adjustments desc

docs/tools/data_analysis.md

@@ -0,0 +1,227 @@
+## data_analysis Tool
+
+### Overview
+
+The `data_analysis` tool lets the assistant **analyze tabular files** (CSV / Excel) that the user has uploaded in the current conversation and, optionally, **generate plots** (time series, bar charts, etc.).
+
+Behind the scenes it:
+- finds a tabular attachment in the conversation,
+- generates a **presigned S3 URL** for that file,
+- calls an **external MCP server** (`data_analysis_tool`) with this URL and the user query,
+- receives back a textual analysis and, optionally, a plot image, which is then stored and inserted directly into the conversation.
+
+---
+
+### Prerequisites
+
+- Conversations running locally via `docker-compose` (so that MinIO is available on `minio:9000` in the backend).
+- An MCP server implementing a `data_analysis_tool` HTTP endpoint, listening on:
+  - `http://localhost:8000/mcp` on your host machine.
+- A way for this MCP server to access your MinIO bucket from outside Docker:
+  - we use **ngrok** to expose MinIO’s port `9000` over HTTPS.
+
+---
+
+### Environment Configuration
+
+All the dev defaults are already present in `env.d/development/common`, but you may need to **adapt them to your environment**.
+
+#### 1. Enable the tool and feature flag
+
+In `env.d/development/common`:
+
+- **Tools list**:
+
+```ini
+AI_AGENT_TOOLS=web_search_brave_with_document_backend,data_analysis
+```
+
+- **Feature flag**:
+
+```ini
+FEATURE_FLAG_DATA_ANALYSIS=ENABLED
+```
+
+This allows the model to call `data_analysis` when it thinks it is relevant.
+
+#### 2. Expose MinIO to the MCP server (ngrok)
+
+The MCP server runs **outside** Docker, but the files are stored in MinIO **inside** the `docker-compose` network.  
+To let the MCP server download the file via a presigned URL, you must:
+
+1. Expose MinIO port `9000` with ngrok (on your host):
+
+```bash
+ngrok http 9000
+```
+
+2. Take the HTTPS URL given by ngrok, e.g.:
+
+```text
+https://your-random-subdomain.ngrok-free.app
+```
+
+3. Set it as `AWS_S3_MCP_URL` in `env.d/development/common`:
+
+```ini
+AWS_S3_MCP_URL=https://your-random-subdomain.ngrok-free.app
+```
+
+This value is used here:
+- in `data_analysis.py`, a dedicated S3 client is created with `endpoint_url=settings.AWS_S3_MCP_URL`;
+- the presigned URL given to the MCP server points to this external endpoint, so the MCP process can fetch the file.
+
+> **Important**: keep `AWS_S3_ENDPOINT_URL` pointing to `http://minio:9000` for the backend itself; only `AWS_S3_MCP_URL` needs to be the ngrok HTTPS URL.
+
+#### 3. Data analysis MCP server URL
+
+The URL of the external MCP server is configured in `src/backend/chat/mcp_servers.py`:
+
+```python
+DATA_ANALYSIS_MCP_SERVER = {
+    "data-analysis": {
+        "url": "http://host.docker.internal:8000/mcp",
+    },
+}
+```
+
+From inside the backend container, `host.docker.internal` resolves to your host machine.  
+So you must run your MCP server on the host at `http://localhost:8000/mcp`:
+
+```bash
+# On the host machine (outside Docker)
+uv run your_data_analysis_mcp_server --port 8000  # exemple
+```
+
+Adapt the command to how your MCP server is started; the important part is that it listens on `0.0.0.0:8000` (or `localhost:8000`) with the `/mcp` endpoint.
+
+If you change the MCP server URL, update `DATA_ANALYSIS_MCP_SERVER` accordingly.
+
+---
+
+### How It Works (Backend Side)
+
+High-level flow in `src/backend/chat/tools/data_analysis.py`:
+
+1. **Find attachments**  
+   The tool looks for `ChatConversationAttachment` objects in the current conversation:
+   - only **original** files (`conversion_from` is `NULL` / empty),
+   - excludes markdown conversions,
+   - filters for tabular extensions: `.csv`, `.xls`, `.xlsx`.
+
+2. **Select a document & generate presigned URL**  
+   It picks the **first tabular file** and generates a presigned URL pointing to the S3 object,
+   using the special MCP S3 client (endpoint = `AWS_S3_MCP_URL`).
+
+3. **Call the MCP server**  
+   It then calls the external MCP server:
+   - tool name: `data_analysis_tool`
+   - arguments:
+     - `query`: the natural language instruction from the user,
+     - `document`: the presigned S3 URL,
+     - `document_name`: the original file name.
+
+4. **Parse MCP response**  
+   The MCP server is expected to return a JSON payload (as text), typically containing:
+   - `result`: textual analysis / summary,
+   - optionally `plot_image`: base64-encoded PNG of a plot,
+   - optionally `query_code`: code used to produce the result (e.g. Python / pandas).
+
+5. **Store plot image (optional)**  
+   If `plot_image` is present:
+   - the backend decodes it,
+   - saves it into the same object storage as other media,
+   - generates a browser URL for the frontend using `generate_retrieve_policy`,
+   - stores that URL in `metadata["plot_url"]` of the `ToolReturn`.
+
+6. **Return to the agent**  
+   The `ToolReturn` contains:
+
+   - `return_value` (what the model sees):
+     - `{"result": "<texte d'analyse ...>"}`  
+       (no `plot_url` — the model never sees the URL)
+   - `metadata` (internal use, not seen by the model):
+     - `{"plot_url": "<URL du graphique>", "query_code": "..."}` when a plot exists.
+
+7. **Insertion of the plot in the conversation**  
+   In `pydantic_ai.py`, when the agent receives a tool result from `data_analysis`:
+   - it reads `plot_url` from `event.result.metadata`,
+   - inserts a markdown image `![Graphique de l'analyse](plot_url)` **directly in the streamed response** to the frontend,
+   - the model only has to comment on the results; it is not responsible for embedding the image.
+
+---
+
+### Enabling the Tool in a Model
+
+In your LLM configuration (`conversations/configuration/llm/*.json`), ensure the tool is listed:
+
+```json
+{
+  "models": [
+    {
+      "hrid": "my-model",
+      "tools": [
+        "data_analysis"
+      ]
+    }
+  ]
+}
+```
+
+Or, in a local dev environment, via `env.d/development/common`:
+
+```ini
+AI_AGENT_TOOLS=web_search_brave_with_document_backend,data_analysis
+```
+
+---
+
+### Typical Usage From the User Perspective
+
+1. The user uploads one or more **CSV / Excel** files in the conversation.
+2. Then asks a question like:
+   - “Fais une analyse des soldes par client dans ce fichier.”
+   - “Trace l’évolution du chiffre d’affaires au cours du temps.”
+3. The model detects that a tabular file is available and calls the `data_analysis` tool.
+4. The MCP server:
+   - downloads the file via the presigned URL,
+   - runs the analysis (e.g. pandas),
+   - renvoie un résultat structuré + un graphique encodé en base64.
+5. The backend:
+   - stocke l’image du graphique,
+   - l’insère directement dans le message assistant,
+   - donne au modèle uniquement le texte d’analyse à commenter.
+
+From the user’s point of view, they just see:
+- their question,
+- the assistant’s answer with text **and** a generated chart, without manual configuration.
+
+---
+
+### Troubleshooting
+
+- **The tool is never called**
+  - Check that:
+    - `FEATURE_FLAG_DATA_ANALYSIS=ENABLED` is set,
+    - `AI_AGENT_TOOLS` includes `data_analysis`,
+    - the model in your LLM config has `data_analysis` listed in `tools`.
+
+- **File download error in the MCP**
+  - Check that:
+    - `ngrok http 9000` is running,
+    - `AWS_S3_MCP_URL` is set to the ngrok **HTTPS** URL,
+    - the MCP server can reach this URL (a quick test: `curl <presigned-url>` from the MCP server machine).
+
+- **No plot returned even though a chart was requested**
+  - Inspect the MCP server logs (can it read the file?),
+  - Make sure it returns a `plot_image` field (base64 PNG) in its JSON response.
+
+---
+
+### See Also
+
+- `src/backend/chat/tools/data_analysis.py`
+- `src/backend/chat/mcp_servers.py`
+- [Tools Overview](../tools.md)
+- [Environment Variables](../env.md)
+

src/backend/chat/clients/pydantic_ai.py

@@ -93,6 +93,7 @@ class ContextDeps:
     conversation: models.ChatConversation
     user: User
     web_search_enabled: bool = False
+    data_analysis_enabled: bool = False
 
 
 def get_model_configuration(model_hrid: str):
@@ -131,12 +132,14 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
         # Feature flags
         self._is_document_upload_enabled = is_feature_enabled(self.user, "document_upload")
         self._is_web_search_enabled = is_feature_enabled(self.user, "web_search")
+        self._is_data_analysis_enabled = is_feature_enabled(self.user, "data_analysis")
         self._fake_streaming_delay = settings.FAKE_STREAMING_DELAY
 
         self._context_deps = ContextDeps(
             conversation=conversation,
             user=user,
             web_search_enabled=self._is_web_search_enabled,
+            data_analysis_enabled=self._is_data_analysis_enabled,
         )
 
         self.conversation_agent = ConversationAgent(
@@ -290,15 +293,25 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
                 )
 
             if not document.media_type.startswith("text/"):
+                # For non-text documents (PDF, Excel, images, etc.), we create a separate
+                # markdown attachment that contains the parsed text content.
+                # IMPORTANT: we must **not** overwrite the original binary file.
+                # If `key` is set, it points to the existing original object in storage;
+                # we derive a distinct markdown key from it so the original stays intact.
+                if key:
+                    md_key = f"{key}.md"
+                else:
+                    md_key = f"{self.conversation.pk}/attachments/{document.identifier}.md"
+
                 md_attachment = await models.ChatConversationAttachment.objects.acreate(
                     conversation=self.conversation,
                     uploaded_by=self.user,
-                    key=key or f"{self.conversation.pk}/attachments/{document.identifier}.md",
+                    key=md_key,
                     file_name=f"{document.identifier}.md",
                     content_type="text/markdown",
-                    conversion_from=key,  # might be None
+                    conversion_from=key,  # original storage key, might be None
                 )
-                default_storage.save(md_attachment.key, BytesIO(parsed_content.encode("utf8")))
+                default_storage.save(md_key, BytesIO(parsed_content.encode("utf8")))
                 md_attachment.upload_state = models.AttachmentStatus.READY
                 await md_attachment.asave(update_fields=["upload_state", "updated_at"])
 
@@ -552,6 +565,7 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
 
             _final_output_from_tool = None
             _ui_sources = []
+            _tool_names = {}  # Map tool_call_id to tool_name
 
             # Help Mistral to prevent `Unexpected role 'user' after role 'tool'` error.
             if history and history[-1].kind == "request":
@@ -650,6 +664,8 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
                                     dataclasses.asdict(event),
                                 )
                                 if isinstance(event, FunctionToolCallEvent):
+                                    # Store tool name for later use
+                                    _tool_names[event.tool_call_id] = event.part.tool_name
                                     if not _tool_is_streaming:
                                         yield events_v4.ToolCallPart(
                                             tool_call_id=event.tool_call_id,
@@ -678,10 +694,39 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
                                                     **_new_source_ui.source.model_dump()
                                                 )
 
+                                        # Check if data_analysis tool was used and extract plot_url
+                                        tool_name = _tool_names.get(event.tool_call_id)
+                                        result_content = event.result.content
+                                        plot_url = None
+
+                                        if tool_name == "data_analysis":
+                                            logger.info(
+                                                f"Data analysis tool was used: {event.result}"
+                                            )
+
+                                            # Extract plot_url from metadata (not return_value - le modèle ne doit pas le voir)
+                                            if event.result.metadata:
+                                                plot_url = event.result.metadata.get("plot_url")
+                                                logger.info(
+                                                    f"Extracted plot_url from metadata: {plot_url}"
+                                                )
+
+                                            # Le plot_url n'est PAS dans return_value ni dans content
+                                            # donc le modèle ne le verra jamais - c'est parfait !
+
                                         yield events_v4.ToolResultPart(
                                             tool_call_id=event.tool_call_id,
-                                            result=event.result.content,
+                                            result=result_content,
                                         )
+
+                                        # If we have a plot_url, insert it directly into the stream as markdown image
+                                        if tool_name == "data_analysis" and plot_url:
+                                            logger.info(
+                                                f"Inserting plot_url directly into stream: {plot_url}"
+                                            )
+                                            yield events_v4.TextPart(
+                                                text=f"\n\n![Graphique de l'analyse]({plot_url})\n\n"
+                                            )
                                     elif isinstance(event.result, RetryPromptPart):
                                         yield events_v4.ToolResultPart(
                                             tool_call_id=event.tool_call_id,

src/backend/chat/mcp_servers.py

@@ -8,13 +8,46 @@ MCP_SERVERS = {
         #     "url": "https://api.githubcopilot.com/mcp/",
         #     "headers": {"Authorization": "Bearer XXX"},
         # },
+        # "data-analysis": {
+        #    "url": "http://host.docker.internal:8000/mcp",
+        # },
     }
 }
 
 
+DATA_ANALYSIS_MCP_SERVER = {
+    "data-analysis": {
+        "url": "http://host.docker.internal:8000/mcp",
+    },
+}
+
+
 def get_mcp_servers():
     """Retrieve MCP servers configuration."""
     return [
         MCPServerStreamableHTTP(**server_config)
         for _name, server_config in MCP_SERVERS["mcpServers"].items()
     ]
+
+
+from contextlib import asynccontextmanager
+
+from mcp import ClientSession
+from mcp.client.streamable_http import streamablehttp_client
+
+
+@asynccontextmanager
+async def get_data_analysis_mcp_server():
+    """
+    Connect to the data analysis MCP server and return the initialized session.
+    """
+    server_url = DATA_ANALYSIS_MCP_SERVER["data-analysis"]["url"]
+
+    # Create a streamable HTTP connection to the MCP server.
+    async with streamablehttp_client(server_url) as (read_stream, write_stream, _):
+        # Create a client session using the streams.
+        async with ClientSession(read_stream, write_stream) as session:
+            # Initialize the session (handshake).
+            await session.initialize()
+            # Yield the session so it stays open while being used
+            yield session

src/backend/chat/tools/__init__.py

@@ -2,12 +2,18 @@
 
 from pydantic_ai import Tool, ToolDefinition
 
+from .data_analysis import data_analysis
 from .fake_current_weather import get_current_weather
 from .web_seach_albert_rag import web_search_albert_rag
 from .web_search_brave import web_search_brave, web_search_brave_with_document_backend
 from .web_search_tavily import web_search_tavily
 
 
+async def only_if_data_analysis_enabled(ctx, tool_def: ToolDefinition) -> ToolDefinition | None:
+    """Prepare function to include a tool only if data analysis is enabled in the context."""
+    return tool_def if ctx.deps.data_analysis_enabled else None
+
+
 async def only_if_web_search_enabled(ctx, tool_def: ToolDefinition) -> ToolDefinition | None:
     """Prepare function to include a tool only if web search is enabled in the context."""
     return tool_def if ctx.deps.web_search_enabled else None
@@ -41,6 +47,12 @@ def get_pydantic_tools_by_name(name: str) -> Tool:
             prepare=only_if_web_search_enabled,
             max_retries=2,
         ),
+        "data_analysis": Tool(
+            data_analysis,
+            takes_ctx=True,
+            prepare=only_if_data_analysis_enabled,
+            max_retries=2,
+        ),
     }
 
     return tool_dict[name]  # will raise on purpose if name is not found

src/backend/chat/tools/data_analysis.py

@@ -0,0 +1,151 @@
+import base64
+import json
+import logging
+import uuid
+from io import BytesIO
+
+from django.core.files.storage import default_storage
+from django.db.models import Q
+
+import boto3
+import botocore
+from asgiref.sync import sync_to_async
+from pydantic_ai import RunContext, RunUsage
+from pydantic_ai.messages import ToolReturn
+
+from core.file_upload.utils import generate_retrieve_policy
+
+from chat import models
+from chat.mcp_servers import get_data_analysis_mcp_server
+from conversations import settings
+
+logger = logging.getLogger(__name__)
+
+
+async def data_analysis(ctx: RunContext, query: str) -> ToolReturn:
+    """
+    Call this tool to perform a data analysis or draw a plot from a file and data.
+    When asking for a plot and if the user made no specific instructions, add to the query that the plot should be elegant and easy to read, with harmonious colors.
+
+    Args:
+        query: The query to perform the data analysis/To plot stuff or compute stuff from files and data.
+        The query should be very clear and precise, explaining what results you expect, tables, numbers or plots.
+
+    Returns:
+        The result of the data analysis and/or the plot requested.
+    """
+    # Prepare files - Get all attachments in the conversation (exclude markdown conversions)
+    # Filter for CSV/Excel files that can be analyzed
+    attachments = [
+        attachment
+        async for attachment in models.ChatConversationAttachment.objects.filter(
+            Q(conversion_from__isnull=True) | Q(conversion_from=""),
+            conversation=ctx.deps.conversation,
+            upload_state=models.AttachmentStatus.READY,
+        ).exclude(content_type="text/markdown")
+    ]
+
+    # Filter for tabular files (CSV, Excel)
+    tabular_attachments = [
+        att for att in attachments if att.file_name.endswith((".csv", ".xls", ".xlsx"))
+    ]
+
+    # Prepare tool arguments
+    tool_args = {"query": query}
+
+    # S3 client dedicated to MCP URLs (endpoint = AWS_S3_MCP_URL, e.g. ngrok)
+    mcp_s3_client = boto3.client(
+        "s3",
+        aws_access_key_id=settings.AWS_S3_ACCESS_KEY_ID,
+        aws_secret_access_key=settings.AWS_S3_SECRET_ACCESS_KEY,
+        endpoint_url=settings.AWS_S3_MCP_URL,
+        config=botocore.client.Config(
+            region_name=settings.AWS_S3_REGION_NAME,
+            signature_version=settings.AWS_S3_SIGNATURE_VERSION,
+        ),
+    )
+
+    # If we have tabular files, use the first one (or let the tool handle multiple files)
+    # TODO: Handle multiple files
+    if tabular_attachments:
+        logger.debug(f"Tabular file found: {tabular_attachments[-1].file_name}")
+        # Use the last tabular file found
+        attachment = tabular_attachments[-1]
+        presigned_url = mcp_s3_client.generate_presigned_url(
+            ClientMethod="get_object",
+            Params={"Bucket": default_storage.bucket_name, "Key": attachment.key},
+            ExpiresIn=settings.AWS_S3_RETRIEVE_POLICY_EXPIRATION,
+        )
+        tool_args["document"] = presigned_url
+    else:
+        return ToolReturn(
+            return_value={"error": "No tabular file found, ask the user to upload a tabular file."},
+            content="",
+            metadata={},
+        )
+    tool_args["document_name"] = attachment.file_name
+    logger.debug(f"Tool arguments: {tool_args}")
+
+    # Connect to MCP server and call the tool
+    async with get_data_analysis_mcp_server() as session:
+        tool_result = await session.call_tool(
+            "data_analysis_tool",
+            tool_args,
+        )
+        logger.info(f"Tool result: {tool_result}")
+        logger.info(f"Tool result type: {type(tool_result)}")
+
+        # tool_result is a CallToolResult MCP
+        parsed_result = {}
+        if getattr(tool_result, "content", None):
+            first_content = tool_result.content[0]
+            text = getattr(first_content, "text", str(first_content))
+            try:
+                parsed_result = json.loads(text)
+            except json.JSONDecodeError:
+                parsed_result = {"raw": text}
+        else:
+            parsed_result = {"raw": str(tool_result)}
+
+        # Prepare results
+        result = {
+            "result": str(parsed_result.get("result")),
+        }
+        metadata = {
+            "query": parsed_result.get("query"),
+            "query_code": parsed_result.get("query_code"),
+            "metadata": parsed_result.get("metadata"),
+        }
+
+        # Check if result has plot
+        plot_image_base64 = parsed_result.get("plot_image")
+        plot_url = None
+
+        if plot_image_base64:
+            # Decode base64 image
+            plot_image = base64.b64decode(plot_image_base64)
+
+            plot_filename = f"plot_{uuid.uuid4().hex[:8]}.png"
+            plot_key = f"{ctx.deps.conversation.pk}/plots/{plot_filename}"
+
+            # Save to storage
+            await sync_to_async(default_storage.save)(plot_key, BytesIO(plot_image))
+
+            browser_plot_url = await sync_to_async(generate_retrieve_policy)(plot_key)
+            plot_url = browser_plot_url
+
+            # Do NOT include plot_url in result so the model can't see it.
+            # plot_url will be added to the stream by pydantic_ai.py.
+            # Add a clear message in the content for the model.
+            result["result"] += (
+                "Le graphique a été inséré automatiquement dans la conversation pour l'utilisateur. "
+                "Ne donnes JAMAIS d'url de plot."
+                "Dis à l'utilisateur 'Tu trouveras le graphique ci-dessus.' ou quelque chose comme ça et commente le graphique si besoin."
+            )
+            metadata["plot_url"] = plot_url
+
+    return ToolReturn(
+        return_value=result,
+        content="",
+        metadata=metadata,
+    )

src/backend/conversations/settings.py

@@ -151,6 +151,10 @@ class Base(BraveSettings, Configuration):
         environ_name="AWS_S3_DOMAIN_REPLACE",
         environ_prefix=None,
     )
+    AWS_S3_MCP_URL = values.Value(
+        environ_name="AWS_S3_MCP_URL",
+        environ_prefix=None,
+    )
 
     ATTACHMENT_CHECK_UNSAFE_MIME_TYPES_ENABLED = values.BooleanValue(
         True,

src/backend/core/feature_flags/flags.py

@@ -44,6 +44,7 @@ class FeatureFlags(BaseModel):
     # features
     web_search: FeatureToggle = FeatureToggle.DISABLED
     document_upload: FeatureToggle = FeatureToggle.DISABLED
+    data_analysis: FeatureToggle = FeatureToggle.DISABLED
 
     def __getattr__(self, name: str):
         """Dynamically get specific RAG document search tool feature flags from settings."""