> ## Documentation Index
> Fetch the complete documentation index at: https://botcadence-a5dbc408.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Knowledge Base
> Learn how to add files, URLs, and domains to your chat agent's knowledge base for accurate, context-aware responses
A knowledge base is a collection of documents, files, and web content that your chat agent uses to answer questions accurately. By adding sources to your knowledge base, you give your chat agent access to information it can reference when responding to users.
## Prerequisites
Before you begin, ensure you have:
* An account and are logged in
* An organization set up
* A project created
* Permission to add sources (you have this if you're the project creator or admin)
## What is a Knowledge Base?
A knowledge base helps your chat agent:
* **Answer questions accurately** by referencing your documents
* **Provide up-to-date information** from your files and websites
* **Understand your business** by learning from your content
* **Reduce errors** by having access to verified information
Think of it as giving your chat agent a library of information it can read to help answer questions.
## Access the Knowledge Base
After logging in, go to the **Projects** page from the main navigation. Find and click on the project you want to add knowledge sources to. You'll be taken to the project dashboard.
Look for **Knowledge Base** or **Sources** in the project sidebar or navigation menu. Click on it to open the Knowledge Base page. You'll see tabs at the top: **Add Knowledge**, **Files**, **Domains**, and **URLs**.
## Add Knowledge Sources
Click on the **Add Knowledge** tab. You'll see three options for adding sources:
Upload documents from your computer (PDFs, Word documents, text files, etc.).
Click on the **Upload Files** card. You'll see a file upload area.
You can add files using either method:
**Method 1: Drag and Drop**
* Open the folder on your computer where your files are stored
* Drag one or more files into the upload area
* Drop them when you see the area highlight
**Method 2: Browse**
* Click the upload area or the **browse** link
* A file picker window will open
* Navigate to your files and select them
* You can select multiple files at once (hold Ctrl on Windows or Cmd on Mac)
After selecting files, you'll see a list of all selected files. Each file shows:
* File name
* File size
You can remove files before uploading if you change your mind.
Click the **Upload Files** button. You'll see a progress indicator while files are uploading.
Don't close the page while files are uploading.
Once complete, you'll see a success message.
### Supported File Types
* PDF files (.pdf)
* Word documents (.docx)
* Text files (.txt)
* CSV files (.csv)
* Markdown files (.md)
### File Size Limits
* Maximum file size: 10 MB per file
* You can upload multiple files at once
Upload files that contain information relevant to what your chat agent needs to know. Common files include: product manuals, FAQs, company policies, documentation, training materials. Make sure files are clear and well-formatted for best results.
Add individual web pages from the internet that your chat agent can read.
Click on the **Add URLs** card. You'll see two modes: **Single URLs** and **Bulk URLs**.
**Add a URL:**
* Click in the URL input field
* Type or paste a web page address
* Make sure it starts with `http://` or `https://`
* Example: `https://example.com/help-page`
**Add More URLs:**
* Click **Add Another URL** to add more URL fields
* You can add multiple URLs one by one
* Each URL will be processed separately
**Remove URLs:**
* Click the trash icon next to any URL to remove it
* You must keep at least one URL field
Click the **Bulk URLs** button to switch modes.
**Paste URLs:**
* Copy all your URLs from a document or list
* Paste them into the text area
* Put each URL on a separate line
Example:
```
https://example.com/page1
https://example.com/page2
https://example.com/page3
```
**Convert to Individual URLs:**
* Click **Convert to Individual URLs** to switch back to single mode
* This will create separate input fields for each URL
* Useful if you want to review or edit URLs individually
The system checks if URLs are valid. Valid URLs show a link icon you can click to open them. Invalid URLs show an error message in red. Make sure all URLs are valid before submitting.
Click the **Add URLs** button. You'll see a progress indicator. Once complete, you'll see a success message.
### URL Examples
* Documentation pages: `https://docs.example.com/getting-started`
* Blog articles: `https://blog.example.com/article/how-to-guide`
* FAQ pages: `https://support.example.com/faq`
* Help pages: `https://help.example.com/troubleshooting`
Use URLs that contain useful, relevant information. Make sure the pages are publicly accessible (not behind a login). Avoid URLs that require JavaScript to display content (may not work well). Test URLs in a browser first to make sure they load correctly.
Add entire websites that the system will automatically crawl and index.
Click on the **Add Domains** card. You'll see a form to add a domain.
Click in the **Domain** field. Enter the website domain (without http\:// or https\://).
Examples:
* `example.com`
* `docs.example.com`
* `support.example.com`
Don't include `www.` or `http://` - just the domain name.
You have two options:
**Sitemap Strategy:**
* Uses the website's sitemap to find pages
* Faster and more efficient
* Best for well-structured websites with sitemaps
* Select **Sitemap** from the strategy dropdown
**Deep Crawl Strategy:**
* Starts from the homepage and follows links
* More thorough but slower
* Best for websites without sitemaps or when you want to crawl specific sections
* Select **Deep Crawl** from the strategy dropdown
**Maximum Pages:**
* Enter the maximum number of pages to crawl
* Default is usually 50 pages
* Increase if you want more pages indexed
Start with a lower number and increase if needed.
**Maximum Depth (for Deep Crawl):**
* How many levels deep to crawl from the homepage
* Default is usually 3 levels
* Level 1 = homepage, Level 2 = pages linked from homepage, etc.
3 levels is usually enough for most websites.
**Exclude Patterns (Optional):**
* Add URL patterns to exclude from crawling
* Examples:
* `/admin/*` - Excludes admin pages
* `/private/*` - Excludes private sections
* `*.pdf` - Excludes PDF files
* Separate multiple patterns with commas or new lines
Choose how often to automatically re-crawl the domain. Options: Daily, Weekly, Monthly, or Never.
Set to Monthly for most websites, or Weekly for frequently updated sites.
Click the **Add Domain** button. The system will start crawling the website. This process may take several minutes depending on the website size. You can check the status in the "Domains" tab.
Use domains for websites where you want to index many pages automatically. Sitemap strategy is faster and recommended if available. Start with lower page limits and increase if needed. Exclude patterns help avoid crawling irrelevant or private pages.
## View and Manage Your Sources
After adding sources, you can view and manage them in the respective tabs:
### Files Tab
* View all uploaded files
* See file status (Processing, Synced, Failed)
* See file size and upload date
* Delete files you no longer need
* Refresh to check for status updates
### URLs Tab
* View all added URLs
* See URL status (Processing, Synced, Failed)
* See when URLs were added and last synced
* Delete URLs you no longer need
* Refresh to check for status updates
### Domains Tab
* View all added domains
* See domain status (Processing, Synced, Failed)
* See crawl settings and statistics
* Delete domains you no longer need
* Refresh to check for status updates
### Status Meanings
* **Processing:** The source is being indexed (this can take a few minutes)
* **Synced:** The source has been successfully indexed and is available to your chat agent
* **Failed:** There was an error processing the source (check the error message)
## Understanding Source Processing
### What Happens After You Add a Source?
1. **Upload/Submission**
* Files are uploaded to secure storage
* URLs and domains are queued for processing
2. **Processing**
* The system extracts text and content from your sources
* Content is broken into smaller chunks for better understanding
* Information is indexed for fast retrieval
3. **Ready to Use**
* Once status shows "Synced", your chat agent can use the information
* The chat agent will automatically reference these sources when answering questions
### Processing Time
* **Small files (\< 1 MB):** Usually 1-2 minutes
* **Large files (1-10 MB):** Usually 3-5 minutes
* **URLs:** Usually 1-3 minutes per URL
* **Domains:** Can take 10-30 minutes or more depending on size
Processing happens in the background. You can continue working while sources are being processed.
## Best Practices
### Choosing What to Add
**Good Sources:**
* Product documentation and manuals
* Frequently Asked Questions (FAQs)
* Company policies and procedures
* Training materials and guides
* Help articles and tutorials
* Blog posts with useful information
* Support documentation
**Avoid:**
* Personal or sensitive information
* Outdated content
* Duplicate information (adds no value)
* Very large files with mostly irrelevant content
* Password-protected or private content
### Organizing Your Knowledge Base
Start small: Add a few key sources first, then expand. Keep it relevant: Only add sources that help your chat agent answer questions. Update regularly: Remove outdated sources and add new ones. Test after adding: Ask your chat agent questions to see if it uses the new sources correctly.
### File Preparation
**Before Uploading Files:**
* Make sure files are clear and readable
* Remove any unnecessary pages or sections
* Ensure text is selectable (not just images)
* Check file size (keep under 10 MB)
* Use descriptive file names
## Troubleshooting
**Possible reasons and solutions:**
* **File too large:** Make sure files are under 10 MB. Split large files into smaller ones if needed
* **Unsupported format:** Check that your file type is supported (PDF, DOCX, TXT, CSV, MD)
* **Network issues:** Check your internet connection and try again
* **Browser issues:** Try refreshing the page or using a different browser
**What to check:**
* Make sure URLs start with `http://` or `https://`
* Verify the URL is accessible (try opening it in a browser)
* Check that the page doesn't require login
* Make sure the URL is valid (no typos)
**What to do:**
**For files:**
* Check if the file is corrupted or in an unsupported format
* Try re-uploading the file
* Make sure the file contains readable text (not just images)
**For URLs:**
* Verify the URL is accessible
* Check if the page requires JavaScript (may not work)
* Try accessing the URL in a browser first
**For domains:**
* Check if the domain is accessible
* Verify the domain doesn't block crawlers
* Try reducing the maximum pages or depth settings
**What to know:**
* Processing can take several minutes, especially for large files or domains
* Check back in a few minutes - processing happens in the background
* If a source is stuck in "Processing" for more than 30 minutes, try deleting and re-adding it
* Large domains can take an hour or more to fully crawl
**What to check:**
* Make sure sources show "Synced" status (not "Processing" or "Failed")
* Wait a few minutes after sources are synced for the chat agent to update
* Check your chat agent's prompt - make sure it's configured to use knowledge sources
* Try asking specific questions that should be answered by your sources
## Managing Your Knowledge Base
### Deleting Sources
Navigate to the Files, URLs, or Domains tab.
Locate the source you want to delete in the list.
Click the delete/trash icon next to the source. Confirm deletion when prompted.
Deleted sources are permanently removed and cannot be recovered.
### Refreshing Sources
Click the **Refresh** button at the top of the page. This updates the status of all sources. Useful to check if processing has completed.
### Re-syncing Sources
* For domains with automatic sync enabled, they will re-crawl automatically
* For files and URLs, you may need to delete and re-add them if content changes
Keep track of when you add sources so you know when to update them.
## Tips for Success
### Start with Key Information
Begin by adding your most important documents (FAQs, product info, policies). Test your chat agent to see how well it uses the sources. Add more sources based on what questions users are asking.
### Keep Sources Updated
Remove outdated information. Add new documents as your business evolves. Update URLs if pages move or change.
### Quality Over Quantity
It's better to have a few high-quality, relevant sources than many irrelevant ones. Focus on sources that directly help answer user questions. Review and clean up your knowledge base regularly.
### Test Your Knowledge Base
After adding sources, test your chat agent with questions that should be answered by those sources. If the chat agent doesn't use the sources correctly, check:
* Source status (should be "Synced")
* Source content (make sure it's relevant)
* Chat agent prompt (should encourage using knowledge sources)
## Summary
Setting up a knowledge base involves:
1. Access the Knowledge Base from your project
2. Add sources using one of three methods:
* Upload files from your computer
* Add individual URLs
* Add entire domains for automatic crawling
3. Wait for processing (sources are indexed in the background)
4. Verify sources are synced (check status in the respective tabs)
5. Test your chat agent to ensure it uses the knowledge base correctly
The knowledge base is a powerful tool that helps your chat agent provide accurate, helpful answers by referencing your own content. Start with a few key sources and expand as needed!
## Quick Reference
**Supported File Types:**
* PDF (.pdf)
* Word Documents (.docx)
* Text Files (.txt)
* CSV Files (.csv)
* Markdown (.md)
**File Size Limit:** 10 MB per file
**Source Types:**
* Files: Upload documents from your computer
* URLs: Add individual web pages
* Domains: Add entire websites for automatic crawling
**Processing Time:**
* Files: 1-5 minutes
* URLs: 1-3 minutes each
* Domains: 10-30+ minutes depending on size
**Status Types:**
* Processing: Being indexed
* Synced: Ready to use
* Failed: Error occurred (check and retry)