diff --git a/README.md b/README.md index d5c2eea..16afb64 100644 --- a/README.md +++ b/README.md @@ -146,6 +146,7 @@ $ make superuser Additional documentation is available in the `docs/` directory: - [LLM Configuration](docs/llm-configuration.md) - Configure Large Language Models and providers +- [Tools for Agents](docs/tools.md) - Available tools and how to add new ones - [Environment Variables](docs/env.md) - All available environment variables - [Installation Guide](docs/installation.md) - Deploy on a Kubernetes cluster - [Theming](docs/theming.md) - Customize the application appearance diff --git a/docs/tools.md b/docs/tools.md new file mode 100644 index 0000000..6af806e --- /dev/null +++ b/docs/tools.md @@ -0,0 +1,238 @@ +# Tools for the Conversation Agent + +The conversation agent can be extended with various tools that provide additional capabilities such as web search, +weather information, and more. We currently only have web search tools, but more tools can be added as needed. +This document explains how to configure and use these tools. + +## Overview + +Tools are functions that the LLM can call during a conversation to access external data or perform specific actions. +The agent decides when to use these tools based on the user's query and the conversation context. + +## Configuring Tools for a Model + +Tools are configured at the model level in the LLM configuration file. +Each model can have its own set of available tools. + +### Configuration File Location + +Read the [LLM Configuration](llm-configuration.md) document to find out where the configuration file is located +and how to use it. + +### Example Configuration + +```json +{ + "models": [ + { + "hrid": "default-model", + "model_name": "gpt-4", + "human_readable_name": "GPT-4 with Tools", + "provider_name": "default-provider", + "is_active": true, + "system_prompt": "You are a helpful assistant.", + "tools": [ + "web_search_brave", + "get_current_weather" + ] + } + ], + "providers": [ + { + "hrid": "default-provider", + "base_url": "https://api.openai.com/v1", + "api_key": "settings.AI_API_KEY", + "kind": "openai" + } + ] +} +``` + +The `tools` field accepts either: +- A list of tool names: `["tool_name_1", "tool_name_2"]` +- A reference to a settings variable: `"settings.AI_AGENT_TOOLS"` + +## Available Tools + +To make a tool available to be in a model's configuration, it must be registered in the tool registry located at +`src/backend/chat/tools/__init__.py`. + +This is not dynamic - any changes to the tool registry require a code deployment... +We want to add dynamic loading in the future. + +| Tool Name | Description | Documentation | +|------------------------------------------|---------------------------------------------------------------|-----------------------------------------------------------------------------| +| `get_current_weather` | Fake weather tool for testing purposes | [Details](tools/get_current_weather.md) | +| `web_search_tavily` | Web search using Tavily API | [Details](tools/web_search_tavily.md) | +| `web_search_brave` | Web search using Brave Search API with optional summarization | [Details](tools/web_search_brave.md) | +| `web_search_brave_with_document_backend` | Web search using Brave with RAG-based document processing | [Details](tools/web_search_brave.md#web_search_brave_with_document_backend) | +| `web_search_albert_rag` | ⚠️ **Deprecated** - Web search using Albert API with RAG | [Details](tools/web_search_brave.md#deprecated-web_search_albert_rag) | + +## Adding a New Tool + +To add a new tool to the system, follow these steps: + +### 1. Create the Tool Function + +Create a new Python file in `src/backend/chat/tools/` with your tool function. The function should: + +- Have clear type annotations +- Include a comprehensive docstring (the LLM uses this to understand when to use the tool) +- Accept `RunContext` as the first parameter if it needs access to conversation context +- Return appropriate data types + +Example: +```python +"""My custom tool for the chat agent.""" + +from pydantic_ai import RunContext + +def my_custom_tool(ctx: RunContext, param1: str, param2: int) -> dict: + """ + Brief description of what the tool does. + + The LLM uses this description to decide when to call this tool. + + Args: + ctx (RunContext): The run context containing the conversation. + param1 (str): Description of parameter 1. + param2 (int): Description of parameter 2. + + Returns: + dict: Description of the return value. + """ + # Your implementation here + return {"result": "example"} +``` + +### 2. Register the Tool + +Add your tool to the registry in `src/backend/chat/tools/__init__.py`: + +```python +from .my_custom_tool import my_custom_tool + +def get_pydantic_tools_by_name(name: str) -> Tool: + """Get a tool by its name.""" + tool_dict = { + "get_current_weather": Tool(get_current_weather, takes_ctx=False), + "web_search_brave": Tool( + web_search_brave, takes_ctx=False, prepare=only_if_web_search_enabled + ), + # Add your tool here + "my_custom_tool": Tool( + my_custom_tool, + takes_ctx=True, # Set to True if your tool needs RunContext + # prepare=only_if_web_search_enabled # Optional: add conditions + ), + } + return tool_dict[name] +``` + +### 3. Update Imports + +Don't forget to import your tool function at the top of `__init__.py`: + +```python +from .my_custom_tool import my_custom_tool +``` + +### 4. Add to Model Configuration + +Add your tool name to the `tools` list in your LLM configuration file or +to the `AI_AGENT_TOOLS` environment variable for local/test purpose. + +## Tool Preparation: Conditional Tool Availability + +Some tools should only be available under certain conditions. The `prepare` parameter in the `Tool` constructor +allows you to specify a function that determines whether a tool should be included. + +### The `only_if_web_search_enabled` Prepare Function + +This is a built-in prepare function that checks if web search feature is enabled in the conversation context: + +```python +async def only_if_web_search_enabled(ctx, tool_def: ToolDefinition) -> ToolDefinition | None: + """Prepare function to include a tool only if web search is enabled in the context.""" + return tool_def if ctx.deps.web_search_enabled else None +``` + +### Usage + +All web search tools use this prepare function: + +```python +"web_search_brave": Tool( + web_search_brave, + takes_ctx=False, + prepare=only_if_web_search_enabled +), +``` + +This ensures that web search tools are only available when the user or conversation settings have enabled web search functionality. + +### Creating Custom Prepare Functions + +You can create your own prepare functions for custom conditions: + +```python +async def only_if_feature_enabled(ctx, tool_def: ToolDefinition) -> ToolDefinition | None: + """Include tool only if a specific feature is enabled.""" + return tool_def if ctx.deps.feature_enabled else None +``` + +## Web Search Enable/Disable + +Web search tools can be toggled on or off based on conversation settings. When web search is disabled: +- Web search tools are not included in the agent's available tools +- The LLM cannot make web search calls even if it tries +- This is enforced by the `only_if_web_search_enabled` prepare function + +The `web_search_enabled` flag is typically set: +- Per conversation in the conversation settings +- Per user preference +- Through admin configuration + +## Best Practices + +1. **Keep tools focused** - Each tool should do one thing well +2. **Clear documentation** - The LLM relies on docstrings to understand when to use tools +3. **Error handling** - Tools should handle errors gracefully and return meaningful messages +4. **Performance** - Be mindful of API rate limits and timeout values +5. **Security** - Never log sensitive data (API keys, user data, etc.) +6. **Caching** - Use Django's cache framework for expensive operations when appropriate + +## Troubleshooting + +### Tool Not Being Called + +If the LLM isn't calling your tool: +- Check that the tool is registered in `get_pydantic_tools_by_name` +- Verify the tool is in the model's `tools` configuration +- Review the tool's docstring - make it clearer when the tool should be used +- Check if any `prepare` function is preventing the tool from being included + +### Tool Errors + +If a tool is throwing errors: +- Check the logs for detailed error messages +- Verify all required environment variables are set +- Ensure the tool's dependencies are installed +- Test the tool function independently + +We recommend wrapping external API calls in try/except blocks to handle potential issues gracefully and use +the Pydantic AI `ModelRetry` exception to let the LLM manage the errors. + +### Tool Response Issues + +If the LLM isn't using the tool response correctly: +- Ensure the return type is clear and well-structured +- Consider returning a `ToolReturn` object with metadata +- Check if the response format matches what the LLM expects + +## See Also + +- [Web Search Configuration](llm-configuration.md) +- [Architecture](architecture.md) +- [Environment Variables](env.md) + diff --git a/docs/tools/get_current_weather.md b/docs/tools/get_current_weather.md new file mode 100644 index 0000000..7d59c81 --- /dev/null +++ b/docs/tools/get_current_weather.md @@ -0,0 +1,113 @@ +# get_current_weather Tool + +## Overview + +The `get_current_weather` tool is a **fake weather tool** designed for testing and demonstration purposes. It does not connect to any real weather API and always returns hardcoded weather data. + +## Purpose + +This tool is useful for: +- **Testing** the tool calling functionality of LLMs +- **Demonstrating** how tools work without requiring API keys +- **Development** and debugging of the agent system +- **Example implementation** for creating new tools + +⚠️ **Warning**: This tool should **not** be used in production environments. It always returns fake data regardless of the location or conditions. + +## Configuration + +### Add to Model + +To enable this tool for a model, add it to the `tools` list in your LLM configuration: + +```json +{ + "models": [ + { + "hrid": "my-model", + "tools": [ + "get_current_weather" + ] + } + ] +} +``` + +Or via environment variable when using local environment settings: +```ini +AI_AGENT_TOOLS=get_current_weather +``` + +### No Additional Settings Required + +This tool does not require any API keys, environment variables, or additional configuration. + +## Function Signature + +```python +def get_current_weather(location: str, unit: str) -> dict: + """ + Get the current weather in a given location. + + Args: + location (str): The city and state, e.g. San Francisco, CA. + unit (str): The unit of temperature, either 'celsius' or 'fahrenheit'. + + Returns: + dict: A dictionary containing the location, temperature, and unit. + """ +``` + +## Parameters + +| Parameter | Type | Required | Description | +|------------|------|----------|-----------------------------------------------------------------| +| `location` | str | Yes | The city and state (e.g., "San Francisco, CA", "Paris, France") | +| `unit` | str | Yes | Temperature unit: either "celsius" or "fahrenheit" | + +## Return Value + +Returns a dictionary with the following structure: + +```python +{ + "location": str, # The location that was queried + "temperature": int, # Always 22°C or 72°F + "unit": str # The unit that was requested +} +``` + +## How the LLM Uses It + +When a user asks about weather, the LLM will: + +1. **Recognize** the weather-related query +2. **Extract** the location from the user's message +3. **Determine** the appropriate unit (often from context or user preference) +4. **Call** the `get_current_weather` tool +5. **Receive** the fake weather data +6. **Format** a response to the user + +### Example Conversation + +**User**: "What's the weather like in London?" + +**LLM** (internal): *Calls `get_current_weather("London, UK", "celsius")`* + +**Tool Response**: +```json +{ + "location": "London, UK", + "temperature": 22, + "unit": "celsius" +} +``` + +**LLM** (to user): "The current weather in London, UK is 22°C." + +## See Also + +- [Tools Overview](../tools.md) +- [Adding a New Tool](../tools.md#adding-a-new-tool) +- [Testing Tools](../tools.md#testing-your-tools) + diff --git a/docs/tools/web_search_brave.md b/docs/tools/web_search_brave.md new file mode 100644 index 0000000..8a00021 --- /dev/null +++ b/docs/tools/web_search_brave.md @@ -0,0 +1,670 @@ +# Brave Web Search Tools + +## Overview + +The Brave web search tools enable the conversation agent to search the web using the [Brave Search API](https://brave.com/search/api/). +Brave Search is a privacy-focused search engine that provides comprehensive web search results. + +This documentation covers three related tools: +1. **`web_search_brave`** - Standard web search with optional summarization +2. **`web_search_brave_with_document_backend`** - Web search with RAG-based document processing +3. **`web_search_albert_rag`** - ⚠️ **Deprecated** - Use `web_search_brave_with_document_backend` instead + +## Table of Contents + +- [Common Configuration](#common-configuration) +- [web_search_brave](#web_search_brave) +- [web_search_brave_with_document_backend](#web_search_brave_with_document_backend) +- [Deprecated: web_search_albert_rag](#deprecated-web_search_albert_rag) +- [Comparison](#comparison) +- [Best Practices](#best-practices) +- [Troubleshooting](#troubleshooting) + +--- + +## Common Configuration + +### Prerequisites + +1. **Brave Search API Key**: Sign up at [Brave Search API](https://brave.com/search/api/) to get an API key +2. **Environment Variables**: Configure the required settings + +### Common Environment Variables + +All Brave tools share these common settings: + +| Variable | Required | Default | Description | +|---------------------|----------|---------|----------------------------------------------------| +| `BRAVE_API_KEY` | **Yes** | None | Your Brave Search API key | +| `BRAVE_API_TIMEOUT` | No | 5 | API request timeout in seconds | +| `BRAVE_MAX_RESULTS` | No | 8 | Maximum number of search results | +| `BRAVE_CACHE_TTL` | No | 1800 | Cache time-to-live in seconds (30 minutes) | + +### Search Parameters + +Check on the Brave API documentation for more details on these parameters: + +| Variable | Required | Default | Description | +|-------------------------------|----------|------------|---------------------------------------------------| +| `BRAVE_SEARCH_COUNTRY` | No | None | Country code for search (e.g., "US", "FR") | +| `BRAVE_SEARCH_LANG` | No | None | Language code (e.g., "en", "fr") | +| `BRAVE_SEARCH_SAFE_SEARCH` | No | "moderate" | Safe search level: "off", "moderate", or "strict" | +| `BRAVE_SEARCH_SPELLCHECK` | No | True | Enable spell checking | +| `BRAVE_SEARCH_EXTRA_SNIPPETS` | No | True | Fetch extra snippets from pages | + + +Note: even if `BRAVE_SEARCH_EXTRA_SNIPPETS` is enabled, the API may not include them if you don't have a plan for this. +This is why, in `web_search_brave`, we also fetch the page content ourselves when needed. + +### Configuration Example + +```bash +# .env file +BRAVE_API_KEY=BSA-your-api-key-here +BRAVE_MAX_RESULTS=8 +BRAVE_MAX_WORKERS=4 +BRAVE_SEARCH_COUNTRY=US +BRAVE_SEARCH_LANG=en +BRAVE_SEARCH_SAFE_SEARCH=moderate +``` + +### Django Settings + +All Brave settings are defined in `src/backend/conversations/brave_settings.py`: + +```python +class BraveSettings: + """Brave settings for web_search_brave tool.""" + + BRAVE_API_KEY = values.Value( + default=None, + environ_name="BRAVE_API_KEY", + environ_prefix=None, + ) + # ... more settings +``` + +--- + +## web_search_brave + +### Overview + +Standard Brave web search tool with optional LLM-based summarization of page content. + +### Purpose + +- Search the web for up-to-date information +- Extract content from web pages +- Optionally summarize content using an LLM +- Provide structured results with snippets + +### Additional Configuration + +| Variable | Required | Default | Description | +|-------------------------------|----------|---------|-------------------------------------------------| +| `BRAVE_SUMMARIZATION_ENABLED` | No | False | Enable LLM-based summarization of fetched pages | + +### Function Signature + +```python +def web_search_brave(query: str) -> ToolReturn: + """ + Search the web for up-to-date information + + Args: + query (str): The query to search for. + + Returns: + ToolReturn: Formatted search results with metadata + """ +``` + +### Return Value + +Returns a `ToolReturn` object with: + +```python +ToolReturn( + return_value={ + "0": { + "url": "https://example.com/page1", + "title": "Example Page Title", + "snippets": ["Extracted or summarized content..."] + }, + "1": { + "url": "https://example.com/page2", + "title": "Another Page", + "snippets": ["More content..."] + } + }, + metadata={ + "sources": { + "https://example.com/page1", + "https://example.com/page2" + } + } +) +``` + +### How It Works + +1. **Query API**: Sends search query to Brave Search API +2. **Receive Results**: Gets list of matching web pages +3. **Fetch Content**: For results without extra_snippets: + - Fetches the HTML content using `trafilatura` + - Extracts the main text content + - Caches the extracted content +4. **Summarize (Optional)**: If `BRAVE_SUMMARIZATION_ENABLED=True`: + - Sends extracted content to summarization agent + - Receives concise summary focused on the query +5. **Format Results**: Returns structured data with URLs, titles, and snippets + +### Workflow Diagram + +``` +User Query + ↓ +Brave Search API + ↓ +Search Results (URLs, titles, descriptions) + ↓ +[For each result without snippets] + ↓ +Fetch HTML (trafilatura) → Extract Text → Cache + ↓ +[If BRAVE_SUMMARIZATION_ENABLED] + ↓ +Summarization Agent (LLM) + ↓ +Summary Text + ↓ +Format & Return +``` + +### Caching + +Extracted content is cached to avoid repeated fetches: + +```python +cache_key = f"web_search_brave:extract:{url}" +cache.set(cache_key, document, settings.BRAVE_CACHE_TTL) +``` + +**Cache Duration**: Controlled by `BRAVE_CACHE_TTL` (default: 30 minutes) + +### Summarization + +When enabled, the tool uses the `SummarizationAgent` to condense page content: + +```python +prompt = f""" +Based on the following request, summarize the following text in a concise manner, +focusing on the key points regarding the user request. +The result should be up to 30 lines long. + + +{query} + + + +{text} + +""" +``` + +**Note**: Summarization is costly (additional LLM calls). +Use only when necessary, we prefer the document vector search from `web_search_brave_with_document_backend`. + +### Add to Model + +```json +{ + "models": [ + { + "hrid": "my-model", + "tools": [ + "web_search_brave" + ] + } + ] +} +``` + +### Example Usage + +**User**: "What are the new features in Django 5.0?" + +**Tool Call**: `web_search_brave("Django 5.0 new features")` + +**Tool Response**: +```python +{ + "0": { + "url": "https://docs.djangoproject.com/en/5.0/releases/5.0/", + "title": "Django 5.0 release notes", + "snippets": ["Django 5.0 introduces several new features including..."] + }, + # ... more results +} +``` + +### Registration + +```python +"web_search_brave": Tool( + web_search_brave, + takes_ctx=False, + prepare=only_if_web_search_enabled +) +``` + +--- + +## web_search_brave_with_document_backend + +### Overview + +Advanced Brave web search tool that uses RAG (Retrieval-Augmented Generation) +with a document backend for intelligent content processing and retrieval. + +### Purpose + +- Search the web and process results through a RAG system +- Store fetched documents in a temporary vector database +- Perform semantic search across fetched content +- Return the most relevant chunks based on the query + +### Additional Configuration + +| Variable | Required | Default | Description | +|-------------------------------------|----------|------------------|----------------------------------------------| +| `BRAVE_RAG_WEB_SEARCH_CHUNK_NUMBER` | No | 10 | Number of chunks to retrieve from RAG search | +| `RAG_DOCUMENT_SEARCH_BACKEND` | No | AlbertRagBackend | Document backend for RAG processing | + +### Function Signature + +```python +def web_search_brave_with_document_backend(ctx: RunContext, query: str) -> ToolReturn: + """ + Search the web for up-to-date information + + Args: + ctx (RunContext): The run context containing the conversation. + query (str): The query to search for. + + Returns: + ToolReturn: Formatted search results with RAG-enhanced snippets + """ +``` + +### How It Works + +1. **Query API**: Sends search query to Brave Search API +2. **Receive Results**: Gets list of matching web pages +3. **Create Temporary Collection**: Creates a temporary vector database collection +4. **Fetch & Store**: For each result: + - Fetches the HTML content + - Extracts the main text + - Stores in the temporary document backend +5. **RAG Search**: Performs semantic search across stored documents +6. **Map Results**: Maps RAG chunks back to original search results +7. **Format & Return**: Returns structured data with enhanced snippets +8. **Cleanup**: Temporary collection is automatically deleted + +### Workflow Diagram + +``` +User Query + ↓ +Brave Search API + ↓ +Search Results (URLs) + ↓ +Create Temporary Vector Collection + ↓ +[For each URL] + ↓ +Fetch HTML → Extract Text → Store in Vector DB + ↓ +RAG Semantic Search + ↓ +Retrieve Most Relevant Chunks + ↓ +Map Chunks to Original URLs + ↓ +Format & Return + ↓ +Delete Temporary Collection +``` + +### Temporary Collection + +The tool creates a temporary collection with a unique ID: + +```python +with document_store_backend.temporary_collection(f"tmp-{uuid.uuid4()}") as document_store: + # Fetch and store documents + # Perform search + # Collection is automatically deleted on exit +``` + +### RAG Search + +The RAG backend performs semantic search to find the most relevant content: + +```python +rag_results = document_store.search( + query, + results_count=settings.BRAVE_RAG_WEB_SEARCH_CHUNK_NUMBER, +) +``` + +Returns chunks ranked by relevance to the query, not just keyword matching. + +### Token Usage Tracking + +The tool tracks LLM tokens used during RAG processing: + +```python +ctx.usage += RunUsage( + input_tokens=rag_results.usage.prompt_tokens, + output_tokens=rag_results.usage.completion_tokens, +) +``` + +### Document Backend + +The default backend is `AlbertRagBackend`, but you can configure a different one: + +```bash +RAG_DOCUMENT_SEARCH_BACKEND=chat.agent_rag.document_rag_backends.custom_backend.CustomBackend +``` + +### Add to Model + +```json +{ + "models": [ + { + "hrid": "my-model", + "tools": [ + "web_search_brave_with_document_backend" + ] + } + ] +} +``` + +### Example Usage + +**User**: "Explain the concept of async views in Django" + +**Tool Call**: `web_search_brave_with_document_backend(ctx, "Django async views explained")` + +**Tool Response**: +```python +{ + "0": { + "url": "https://docs.djangoproject.com/en/stable/topics/async/", + "title": "Asynchronous support", + "snippets": [ + "Django has support for writing asynchronous views...", + "Async views are declared using Python's async def syntax..." + ] + }, + # ... more results with relevant chunks +} +``` + +### Registration + +```python +"web_search_brave_with_document_backend": Tool( + web_search_brave_with_document_backend, + takes_ctx=True, + prepare=only_if_web_search_enabled, +) +``` + +### Advantages Over Standard web_search_brave + +| Feature | web_search_brave | web_search_brave_with_document_backend | +|-------------------|--------------------------------|----------------------------------------| +| Content Retrieval | Full page or summary | Semantic chunks | +| Relevance | Keyword-based | Semantic similarity | +| Token Efficiency | May include irrelevant content | Only relevant chunks | +| Processing | Simpler, faster | More intelligent, slower | +| Cost | Lower | Higher (RAG processing) | +| Best For | General search | Deep research, technical queries | + +--- + +## Deprecated: web_search_albert_rag + +### ⚠️ Deprecation Notice + +The `web_search_albert_rag` tool is **deprecated** and should not be used in new implementations. + +**Replacement**: Use `web_search_brave_with_document_backend` instead, which provides: +- Better performance +- More control over the RAG backend +- Temporary collections (no cleanup issues) +- Token usage tracking +- Parallel processing support + +### Why Deprecated? + +- Limited to Albert API only +- No control over document backend +- Less flexible than the new approach +- Maintenance burden + +### Timeline + +- **Current**: Still functional but not recommended +- **Future**: Will be removed in a future version + +--- + +## Comparison + +### When to Use Which Tool? + +#### Use `web_search_brave` + +✅ **Best for**: +- General web search queries +- Quick information retrieval +- When speed is important +- Lower cost requirements +- Simple fact-finding + +❌ **Not ideal for**: +- Deep research requiring precise context +- Technical documentation queries +- When semantic relevance is crucial + +#### Use `web_search_brave_with_document_backend` + +✅ **Best for**: +- Complex technical queries +- Research requiring precise context +- When semantic relevance is important +- Questions needing deep understanding +- Documentation and how-to queries + +❌ **Not ideal for**: +- Simple factual queries +- When speed is critical +- Budget-constrained scenarios +- High-volume usage + +--- + +## Best Practices + +### Query Formulation + +Help the LLM formulate effective queries: + +```python +# Good queries +"Python asyncio tutorial 2024" +"Django REST framework authentication" +"React hooks best practices" + +# Poor queries +"tell me about programming" # Too vague +"how do I do the thing with the stuff" # Unclear +``` + +### Performance Optimization + +#### 1. Optimize Cache + +```bash +# Longer cache for stable content +BRAVE_CACHE_TTL=3600 # 1 hour + +# Shorter cache for dynamic content +BRAVE_CACHE_TTL=300 # 5 minutes +``` + +#### 2. Control Result Count + +```bash +# Fewer results = faster responses +BRAVE_MAX_RESULTS=5 + +# More results = more comprehensive +BRAVE_MAX_RESULTS=10 +``` + +### Summarization Best Practices + +Only enable summarization when needed: + +```bash +# Enable for long-form content +BRAVE_SUMMARIZATION_ENABLED=True + +# Disable for speed +BRAVE_SUMMARIZATION_ENABLED=False +``` + +**Cost consideration**: Summarization makes additional LLM calls for each result, +significantly increasing costs (and execution time). + +### RAG Configuration + +For `web_search_brave_with_document_backend`: + +```bash +# More chunks = more context, higher cost +BRAVE_RAG_WEB_SEARCH_CHUNK_NUMBER=10 + +# Fewer chunks = faster, less context +BRAVE_RAG_WEB_SEARCH_CHUNK_NUMBER=5 +``` + +### Search Parameters + +```bash +# Localize results +BRAVE_SEARCH_COUNTRY=FR +BRAVE_SEARCH_LANG=fr + +# Safe search for public deployments +BRAVE_SEARCH_SAFE_SEARCH=strict + +# Enable spell check for better results +BRAVE_SEARCH_SPELLCHECK=True +``` + +--- + +## Troubleshooting + +### Common Issues + +#### 1. No Results Returned + +**Symptoms**: Empty results or no snippets + +**Causes**: +- Query too specific +- Content extraction failed +- Trafilatura couldn't parse the pages + +**Solutions**: +```bash +# Enable extra snippets +BRAVE_SEARCH_EXTRA_SNIPPETS=True + +# Increase result count +BRAVE_MAX_RESULTS=10 + +# Check logs for extraction errors +``` + +#### 2. API Errors + +**Symptoms**: HTTP errors, authentication failures + +**Causes**: +- Invalid API key +- Rate limit exceeded +- API service issues + +**Solutions**: +```bash +# Verify API key is set +echo $BRAVE_API_KEY + +# Check Brave API dashboard for limits +# Implement rate limiting in your application +``` + +#### 3. The tool is not being called +**Symptoms**: LLM doesn't use the tool even when appropriate + +**Causes**: +- Web search not enabled for the conversation +- Tool not in model configuration + +**Solutions**: +- Check conversation settings have `web_search_enabled=True` +- Verify tool is in the model's `tools` list + +--- + +## Security Considerations + +This tool is quite "raw", so be cautious about: +- the results returned by the web search +- the context size which might be large when not using summarization or RAG if long results are returned +- the query content which might include sensitive information +- ... + +### Content Validation + +Be aware that fetched content may contain: +- Malicious scripts (mitigated by text extraction) +- Inappropriate content +- Misinformation +- Biased information + +The LLM should evaluate sources critically. + + +--- + +## See Also + +- [Tools Overview](../tools.md) +- [Tavily Web Search Tool](web_search_tavily.md) +- [LLM Configuration](../llm-configuration.md) +- [Environment Variables](../env.md) +- [Brave Search API Documentation](https://brave.com/search/api/) + diff --git a/docs/tools/web_search_tavily.md b/docs/tools/web_search_tavily.md new file mode 100644 index 0000000..eb00343 --- /dev/null +++ b/docs/tools/web_search_tavily.md @@ -0,0 +1,370 @@ +# web_search_tavily Tool + +## Overview + +The `web_search_tavily` tool enables the conversation agent to search the web for up-to-date +information using the [Tavily Search API](https://tavily.com/). + +## Purpose + +This tool allows the LLM to: +- Access current, real-time information beyond its training data +- Answer questions about recent events, news, or developments +- Provide factual information with sources +- Retrieve specific information from the web + +## Configuration + +### Prerequisites + +1. **Tavily API Key**: Sign up at [Tavily](https://tavily.com/) to get an API key +2. **Environment Variables**: Configure the required settings + +### Environment Variables + +| Variable | Required | Default | Description | +|----------------------|----------|---------|--------------------------------------------| +| `TAVILY_API_KEY` | **Yes** | None | Your Tavily API key | +| `TAVILY_MAX_RESULTS` | No | 5 | Maximum number of search results to return | +| `TAVILY_API_TIMEOUT` | No | 10 | API request timeout in seconds | + +### Configuration Example + +```bash +# .env file +TAVILY_API_KEY=tvly-your-api-key-here +TAVILY_MAX_RESULTS=5 +TAVILY_API_TIMEOUT=10 +``` + +### Add to Model + +To enable this tool for a model, add it to the `tools` list in your LLM configuration: + +```json +{ + "models": [ + { + "hrid": "my-model", + "tools": [ + "web_search_tavily" + ] + } + ] +} +``` + +Or via environment variable when using local environment settings: + +```ini +AI_AGENT_TOOLS=web_search_tavily +``` + +## Function Signature + +```python +def web_search_tavily(query: str) -> list[dict]: + """ + Search the web for up-to-date information + + Args: + query (str): The query to search for. + + Returns: + list[dict]: A list of search results, each represented as a dictionary. + """ +``` + +## Parameters + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------------------| +| `query` | str | Yes | The search query string | + +## Return Value + +Returns a list of dictionaries, each containing: + +```python +{ + "link": str, # URL of the result + "title": str, # Title of the page + "snippet": str # Content snippet from the page +} +``` + +### Example Return Value + +```python +[ + { + "link": "https://example.com/article1", + "title": "Introduction to Python", + "snippet": "Python is a high-level programming language known for its simplicity..." + }, + { + "link": "https://example.com/article2", + "title": "Python Best Practices", + "snippet": "Follow these best practices to write clean and efficient Python code..." + } +] +``` + +## How the LLM Uses It + +When a user asks for current information or specific facts: + +1. **LLM recognizes** the need for external information +2. **Formulates** an appropriate search query +3. **Calls** `web_search_tavily(query="search terms")` +4. **Receives** a list of search results +5. **Synthesizes** the information into a response +6. **Provides** the answer with source references + +### Example Conversation + +**User**: "What are the latest developments in quantum computing?" + +**LLM** (internal): *Calls `web_search_tavily("latest developments quantum computing 2024")`* + +**Tool Response**: +```python +[ + { + "link": "https://techcrunch.com/quantum-news", + "title": "Major Breakthrough in Quantum Computing", + "snippet": "Researchers announced a significant breakthrough..." + }, + # ... more results +] +``` + +**LLM** (to user): "Based on recent sources, there have been several developments in quantum computing. +Researchers recently announced a breakthrough in error correction. Additionally, new quantum processors +with improved qubit stability have been unveiled..." + +## Implementation Details + +### Source Code + +Located at: `src/backend/chat/tools/web_search_tavily.py` + +```python +"""Web search tool using Tavily for the chat agent.""" + +from django.conf import settings + +import requests + + +def web_search_tavily(query: str) -> list[dict]: + """ + Search the web for up-to-date information + + Args: + query (str): The query to search for. + + Returns: + list[dict]: A list of search results, each represented as a dictionary. + """ + url = "https://api.tavily.com/search" + data = { + "query": query, + "api_key": settings.TAVILY_API_KEY, + "max_results": settings.TAVILY_MAX_RESULTS, + } + response = requests.post(url, json=data, timeout=settings.TAVILY_API_TIMEOUT) + response.raise_for_status() + + json_response = response.json() + + raw_search_results = json_response.get("results", []) + + return [ + { + "link": result["url"], + "title": result.get("title", ""), + "snippet": result.get("content"), + } + for result in raw_search_results + ] +``` + +### Registration + +The tool is registered in `src/backend/chat/tools/__init__.py`: + +```python +"web_search_tavily": Tool( + web_search_tavily, + takes_ctx=False, + prepare=only_if_web_search_enabled +) +``` + +Note that: +- `takes_ctx=False` - This tool doesn't need the conversation context +- `prepare=only_if_web_search_enabled` - Only available when web search is enabled + +## Django Settings + +The tool uses these Django settings from `settings.py`: + +```python +# Tavily API +TAVILY_API_KEY = values.Value( + None, # Tavily API key is not set by default + environ_name="TAVILY_API_KEY", + environ_prefix=None, +) +TAVILY_MAX_RESULTS = values.PositiveIntegerValue( + default=5, + environ_name="TAVILY_MAX_RESULTS", + environ_prefix=None, +) +TAVILY_API_TIMEOUT = values.PositiveIntegerValue( + default=10, # seconds + environ_name="TAVILY_API_TIMEOUT", + environ_prefix=None, +) +``` + +## Error Handling + +The tool may raise exceptions in the following cases: + +### Missing API Key +```python +# If TAVILY_API_KEY is not set +AttributeError: 'Settings' object has no attribute 'TAVILY_API_KEY' +``` + +**Solution**: Set the `TAVILY_API_KEY` environment variable + +### API Errors +```python +# If the API request fails +requests.exceptions.HTTPError: 401 Unauthorized +``` + +**Possible causes**: +- Invalid API key +- Exceeded rate limits +- API service unavailable + +### Timeout Errors +```python +# If the request takes too long +requests.exceptions.Timeout +``` + +**Solution**: Increase `TAVILY_API_TIMEOUT` or check network connectivity + +## Best Practices + +### Query Formulation + +The LLM should formulate queries that are: +- **Specific and focused** - Better results with targeted queries +- **Up-to-date** - Include year or "latest" when relevant +- **Clear** - Avoid ambiguous terms +- **Concise** - Remove unnecessary words + +Good query examples: +- ✅ "quantum computing breakthroughs 2024" +- ✅ "latest Python 3.12 features" +- ✅ "climate change COP29 outcomes" + +Poor query examples: +- ❌ "tell me about stuff happening" (too vague) +- ❌ "what is the weather like today in Paris on November 5th 2024 at 3pm" (too specific/long) + +### Rate Limiting + +Be aware of Tavily API rate limits: +- Free tier: Limited requests per month +- Paid tiers: Higher limits + +Monitor your usage and implement caching if needed. + +### Result Count + +The `TAVILY_MAX_RESULTS` setting controls how many results are returned: +- **Lower values (3-5)**: Faster responses, less context for LLM +- **Higher values (8-10)**: More comprehensive, but slower and more expensive + +Recommended: **5 results** for most use cases + +## Troubleshooting + +### Tool Not Being Called + +**Symptoms**: LLM doesn't use web search even when appropriate + +**Possible causes**: +1. Web search not enabled for the conversation +2. Tool not in model configuration +3. API key not set + +**Solutions**: +1. Check conversation settings have `web_search_enabled=True` +2. Verify tool is in the model's `tools` list +3. Confirm `TAVILY_API_KEY` is set + +### No Results Returned + +**Symptoms**: Tool returns empty list + +**Possible causes**: +1. Query too specific +2. No matching results +3. API filtering results + +**Solutions**: +1. Try broader query terms +2. Check Tavily dashboard for query logs +3. Review API response in logs + +### Slow Responses + +**Symptoms**: Tool takes a long time to respond + +**Possible causes**: +1. Network latency +2. Tavily API slow +3. Timeout too high + +**Solutions**: +1. Check network connectivity +2. Monitor Tavily status page +3. Adjust `TAVILY_API_TIMEOUT` if needed + +## Security Considerations + +This tool is quite "raw", and was currently only used for test purpose, so be cautious about: +- the results returned by the web search +- the context size which might be large if many results are returned +- the query content which might include sensitive information +- ... + +## Performance Optimization + +### Query Optimization + +You may want to help the LLM formulate better queries by including something like this in the system prompt: + +``` +When using web search: +- Use specific, focused queries +- Include relevant time periods if needed +- Avoid unnecessary words +- Combine related terms +``` + +## See Also + +- [Tools Overview](../tools.md) +- [Brave Web Search Tool](web_search_brave.md) +- [Web Search Configuration](../llm-configuration.md) +- [Environment Variables](../env.md) +