✨(backend) use docling

I save my changes  but this is deprecated
as we want must use docling-serve.

.gitignore

@@ -44,6 +44,9 @@ env.d/development/*
 !env.d/development/*.dist
 env.d/terraform
 
+# Configuration
+**/conversations/configuration/llm/dev.json
+
 # npm
 node_modules
 
@@ -79,3 +82,6 @@ db.sqlite3
 
 # Docker compose override
 compose.override.yml
+
+# Docling
+docling-models

CHANGELOG.md

@@ -8,6 +8,10 @@ and this project adheres to
 
 ## [Unreleased]
 
+### Added
+
+- ✨(backend) add FindRagBackend
+
 ### Changed
 
 - 📦️(front) update react

compose.yml

@@ -71,6 +71,9 @@ services:
       - "host.docker.internal:host-gateway"
     ports:
       - "8071:8000"
+    networks:
+      - default
+      - lasuite
     volumes:
       - ./src/backend:/app
       - ./data/static:/data/static
@@ -89,6 +92,9 @@ services:
     image: nginx:1.25
     ports:
       - "8083:8083"
+    networks:
+      - default
+      - lasuite
     volumes:
       - ./docker/files/etc/nginx/conf.d:/etc/nginx/conf.d:ro
     depends_on:
@@ -177,3 +183,8 @@ services:
       kc_postgresql:
         condition: service_healthy
         restart: true
+
+networks:
+  lasuite:
+    name: lasuite-network
+    driver: bridge

docs/env.md

@@ -95,6 +95,9 @@ These are the environment variables you can set for the `conversations-backend`
 | CACHES_KEY_PREFIX                               | The prefix used to every cache keys.                                                                                              | conversations                                           |
 | THEME_CUSTOMIZATION_FILE_PATH                   | full path to the file customizing the theme. An example is provided in src/backend/conversations/configuration/theme/default.json | BASE_DIR/conversations/configuration/theme/default.json |
 | THEME_CUSTOMIZATION_CACHE_TIMEOUT               | Cache duration for the customization settings                                                                                     | 86400                                                   |
+| FIND_API_KEY                                    | API key of Find                                                                                                                   |                                                         |
+| FIND_API_URL                                    | URL of Find                                                                                                                       | `https://app-find/api`                                  |
+| FIND_API_TIMEOUT                                | Find API timeout                                                                                                                  | 30                                                      |
 
 
 ## conversations-frontend image

docs/llm-configuration.md

@@ -244,9 +244,9 @@ For Mistral AI models using the Etalab platform:
 {
   "models": [
     {
-      "hrid": "mistral-large",
-      "model_name": "mistral-large-latest",
-      "human_readable_name": "Mistral Large (Etalab)",
+      "hrid": "mistral-medium",
+      "model_name": "mistral-medium-2508",
+      "human_readable_name": "Mistral Medium (Etalab)",
       "provider_name": "mistral-etalab",
       "profile": null,
       "settings": {

docs/tools/data_analysis.md

@@ -1,227 +0,0 @@
-## data_analysis Tool
-
-### Overview
-
-The `data_analysis` tool lets the assistant **analyze tabular files** (CSV / Excel) that the user has uploaded in the current conversation and, optionally, **generate plots** (time series, bar charts, etc.).
-
-Behind the scenes it:
-- finds a tabular attachment in the conversation,
-- generates a **presigned S3 URL** for that file,
-- calls an **external MCP server** (`data_analysis_tool`) with this URL and the user query,
-- receives back a textual analysis and, optionally, a plot image, which is then stored and inserted directly into the conversation.
-
----
-
-### Prerequisites
-
-- Conversations running locally via `docker-compose` (so that MinIO is available on `minio:9000` in the backend).
-- An MCP server implementing a `data_analysis_tool` HTTP endpoint, listening on:
-  - `http://localhost:8000/mcp` on your host machine.
-- A way for this MCP server to access your MinIO bucket from outside Docker:
-  - we use **ngrok** to expose MinIO’s port `9000` over HTTPS.
-
----
-
-### Environment Configuration
-
-All the dev defaults are already present in `env.d/development/common`, but you may need to **adapt them to your environment**.
-
-#### 1. Enable the tool and feature flag
-
-In `env.d/development/common`:
-
-- **Tools list**:
-
-```ini
-AI_AGENT_TOOLS=web_search_brave_with_document_backend,data_analysis
-```
-
-- **Feature flag**:
-
-```ini
-FEATURE_FLAG_DATA_ANALYSIS=ENABLED
-```
-
-This allows the model to call `data_analysis` when it thinks it is relevant.
-
-#### 2. Expose MinIO to the MCP server (ngrok)
-
-The MCP server runs **outside** Docker, but the files are stored in MinIO **inside** the `docker-compose` network.  
-To let the MCP server download the file via a presigned URL, you must:
-
-1. Expose MinIO port `9000` with ngrok (on your host):
-
-```bash
-ngrok http 9000
-```
-
-2. Take the HTTPS URL given by ngrok, e.g.:
-
-```text
-https://your-random-subdomain.ngrok-free.app
-```
-
-3. Set it as `AWS_S3_MCP_URL` in `env.d/development/common`:
-
-```ini
-AWS_S3_MCP_URL=https://your-random-subdomain.ngrok-free.app
-```
-
-This value is used here:
-- in `data_analysis.py`, a dedicated S3 client is created with `endpoint_url=settings.AWS_S3_MCP_URL`;
-- the presigned URL given to the MCP server points to this external endpoint, so the MCP process can fetch the file.
-
-> **Important**: keep `AWS_S3_ENDPOINT_URL` pointing to `http://minio:9000` for the backend itself; only `AWS_S3_MCP_URL` needs to be the ngrok HTTPS URL.
-
-#### 3. Data analysis MCP server URL
-
-The URL of the external MCP server is configured in `src/backend/chat/mcp_servers.py`:
-
-```python
-DATA_ANALYSIS_MCP_SERVER = {
-    "data-analysis": {
-        "url": "http://host.docker.internal:8000/mcp",
-    },
-}
-```
-
-From inside the backend container, `host.docker.internal` resolves to your host machine.  
-So you must run your MCP server on the host at `http://localhost:8000/mcp`:
-
-```bash
-# On the host machine (outside Docker)
-uv run your_data_analysis_mcp_server --port 8000  # exemple
-```
-
-Adapt the command to how your MCP server is started; the important part is that it listens on `0.0.0.0:8000` (or `localhost:8000`) with the `/mcp` endpoint.
-
-If you change the MCP server URL, update `DATA_ANALYSIS_MCP_SERVER` accordingly.
-
----
-
-### How It Works (Backend Side)
-
-High-level flow in `src/backend/chat/tools/data_analysis.py`:
-
-1. **Find attachments**  
-   The tool looks for `ChatConversationAttachment` objects in the current conversation:
-   - only **original** files (`conversion_from` is `NULL` / empty),
-   - excludes markdown conversions,
-   - filters for tabular extensions: `.csv`, `.xls`, `.xlsx`.
-
-2. **Select a document & generate presigned URL**  
-   It picks the **first tabular file** and generates a presigned URL pointing to the S3 object,
-   using the special MCP S3 client (endpoint = `AWS_S3_MCP_URL`).
-
-3. **Call the MCP server**  
-   It then calls the external MCP server:
-   - tool name: `data_analysis_tool`
-   - arguments:
-     - `query`: the natural language instruction from the user,
-     - `document`: the presigned S3 URL,
-     - `document_name`: the original file name.
-
-4. **Parse MCP response**  
-   The MCP server is expected to return a JSON payload (as text), typically containing:
-   - `result`: textual analysis / summary,
-   - optionally `plot_image`: base64-encoded PNG of a plot,
-   - optionally `query_code`: code used to produce the result (e.g. Python / pandas).
-
-5. **Store plot image (optional)**  
-   If `plot_image` is present:
-   - the backend decodes it,
-   - saves it into the same object storage as other media,
-   - generates a browser URL for the frontend using `generate_retrieve_policy`,
-   - stores that URL in `metadata["plot_url"]` of the `ToolReturn`.
-
-6. **Return to the agent**  
-   The `ToolReturn` contains:
-
-   - `return_value` (what the model sees):
-     - `{"result": "<texte d'analyse ...>"}`  
-       (no `plot_url` — the model never sees the URL)
-   - `metadata` (internal use, not seen by the model):
-     - `{"plot_url": "<URL du graphique>", "query_code": "..."}` when a plot exists.
-
-7. **Insertion of the plot in the conversation**  
-   In `pydantic_ai.py`, when the agent receives a tool result from `data_analysis`:
-   - it reads `plot_url` from `event.result.metadata`,
-   - inserts a markdown image `![Graphique de l'analyse](plot_url)` **directly in the streamed response** to the frontend,
-   - the model only has to comment on the results; it is not responsible for embedding the image.
-
----
-
-### Enabling the Tool in a Model
-
-In your LLM configuration (`conversations/configuration/llm/*.json`), ensure the tool is listed:
-
-```json
-{
-  "models": [
-    {
-      "hrid": "my-model",
-      "tools": [
-        "data_analysis"
-      ]
-    }
-  ]
-}
-```
-
-Or, in a local dev environment, via `env.d/development/common`:
-
-```ini
-AI_AGENT_TOOLS=web_search_brave_with_document_backend,data_analysis
-```
-
----
-
-### Typical Usage From the User Perspective
-
-1. The user uploads one or more **CSV / Excel** files in the conversation.
-2. Then asks a question like:
-   - “Fais une analyse des soldes par client dans ce fichier.”
-   - “Trace l’évolution du chiffre d’affaires au cours du temps.”
-3. The model detects that a tabular file is available and calls the `data_analysis` tool.
-4. The MCP server:
-   - downloads the file via the presigned URL,
-   - runs the analysis (e.g. pandas),
-   - renvoie un résultat structuré + un graphique encodé en base64.
-5. The backend:
-   - stocke l’image du graphique,
-   - l’insère directement dans le message assistant,
-   - donne au modèle uniquement le texte d’analyse à commenter.
-
-From the user’s point of view, they just see:
-- their question,
-- the assistant’s answer with text **and** a generated chart, without manual configuration.
-
----
-
-### Troubleshooting
-
-- **The tool is never called**
-  - Check that:
-    - `FEATURE_FLAG_DATA_ANALYSIS=ENABLED` is set,
-    - `AI_AGENT_TOOLS` includes `data_analysis`,
-    - the model in your LLM config has `data_analysis` listed in `tools`.
-
-- **File download error in the MCP**
-  - Check that:
-    - `ngrok http 9000` is running,
-    - `AWS_S3_MCP_URL` is set to the ngrok **HTTPS** URL,
-    - the MCP server can reach this URL (a quick test: `curl <presigned-url>` from the MCP server machine).
-
-- **No plot returned even though a chart was requested**
-  - Inspect the MCP server logs (can it read the file?),
-  - Make sure it returns a `plot_image` field (base64 PNG) in its JSON response.
-
----
-
-### See Also
-
-- `src/backend/chat/tools/data_analysis.py`
-- `src/backend/chat/mcp_servers.py`
-- [Tools Overview](../tools.md)
-- [Environment Variables](../env.md)
-

docs/tools/web_search_brave.md

@@ -357,6 +357,7 @@ The RAG backend performs semantic search to find the most relevant content:
 rag_results = document_store.search(
     query,
     results_count=settings.BRAVE_RAG_WEB_SEARCH_CHUNK_NUMBER,
+    **kwargs, # Additional search parameters like session with access_token
 )
 ```
 

src/backend/chat/agent_rag/document_converter/parser.py

@@ -0,0 +1,100 @@
+"""Document parsers for RAG backends."""
+
+import logging
+from io import BytesIO
+from urllib.parse import urljoin
+
+from django.conf import settings
+
+import requests
+from docling.backend.pypdfium2_backend import PyPdfiumDocumentBackend
+from docling.datamodel.base_models import InputFormat
+from docling.datamodel.pipeline_options import PdfPipelineOptions, TableStructureOptions
+from docling.document_converter import DocumentConverter as DoclingDocumentConverter
+from docling.document_converter import PdfFormatOption
+from docling_core.types.io import DocumentStream
+
+from chat.agent_rag.document_converter.markitdown import (
+    DocumentConverter as MarkitdownDocumentConverter,
+)
+
+logger = logging.getLogger(__name__)
+
+
+class BaseParser:
+    """Base class for document parsers."""
+
+    def parse_document(self, name: str, content_type: str, content: BytesIO) -> str:
+        """
+        Parse the document and prepare it for the search operation.
+        This method should handle the logic to convert the document
+        into a format suitable for storage.
+
+        Args:
+            name (str): The name of the document.
+            content_type (str): The MIME type of the document (e.g., "application/pdf").
+            content (BytesIO): The content of the document as a BytesIO stream.
+
+        Returns:
+            str: The document content in Markdown format.
+        """
+        raise NotImplementedError("Must be implemented in subclass.")
+
+
+class AlbertParser(BaseParser):
+    """Document parser using Albert API for PDFs and DocumentConverter for other formats."""
+
+    endpoint = urljoin(settings.ALBERT_API_URL, "/v1/parse-beta")
+
+    def parse_pdf_document(self, name: str, content_type: str, content: bytes) -> str:
+        """Parse PDF document using Albert API."""
+        response = requests.post(
+            self.endpoint,
+            headers={
+                "Authorization": f"Bearer {settings.ALBERT_API_KEY}",
+            },
+            files={
+                "file": (name, content, content_type),
+                "output_format": (None, "markdown"),
+            },
+            timeout=settings.ALBERT_API_PARSE_TIMEOUT,
+        )
+        response.raise_for_status()
+
+        return "\n\n".join(
+            document_page["content"] for document_page in response.json().get("data", [])
+        )
+
+    def parse_document(self, name: str, content_type: str, content: bytes) -> str:
+        """Parse document based on content type."""
+        if content_type == "application/pdf":
+            return self.parse_pdf_document(name=name, content_type=content_type, content=content)
+        return MarkitdownDocumentConverter().convert_raw(
+            name=name, content_type=content_type, content=content
+        )
+
+
+class DoclingParser(BaseParser):
+    """Document parser using Docling's DocumentConverter."""
+
+    artifacts_path = "src/backend/docling-models"
+
+    def __init__(self):
+        pipeline_options = PdfPipelineOptions(artifacts_path=self.artifacts_path)
+        pipeline_options.do_ocr = True
+        pipeline_options.do_table_structure = True
+        pipeline_options.table_structure_options = TableStructureOptions(do_cell_matching=False)
+
+        self.converter = DoclingDocumentConverter(
+            format_options={
+                InputFormat.PDF: PdfFormatOption(
+                    pipeline_options=pipeline_options,
+                    backend=PyPdfiumDocumentBackend
+                )}
+        )
+
+    def parse_document(self, name: str, content_type: str, content: bytes) -> str:
+        """Parse document using Docling's DocumentConverter."""
+        return self.converter.convert(
+            DocumentStream(name=name, stream=BytesIO(content))
+        ).document.export_to_markdown()

src/backend/chat/agent_rag/document_rag_backends/albert_rag_backend.py

@@ -13,7 +13,7 @@ import requests
 
 from chat.agent_rag.albert_api_constants import Searches
 from chat.agent_rag.constants import RAGWebResult, RAGWebResults, RAGWebUsage
-from chat.agent_rag.document_converter.markitdown import DocumentConverter
+from chat.agent_rag.document_converter.parser import DoclingParser
 from chat.agent_rag.document_rag_backends.base_rag_backend import BaseRagBackend
 
 logger = logging.getLogger(__name__)
@@ -26,9 +26,6 @@ class AlbertRagBackend(BaseRagBackend):  # pylint: disable=too-many-instance-att
 
     It provides methods to:
     - Create a collection for the search operation.
-    - Parse documents and convert them to Markdown format:
-       + Handle PDF parsing using the Albert API.
-       + Use the DocumentConverter (markitdown) for other formats.
     - Store parsed documents in the Albert collection.
     - Perform a search operation using the Albert API.
     """
@@ -46,10 +43,9 @@ class AlbertRagBackend(BaseRagBackend):  # pylint: disable=too-many-instance-att
         }
         self._collections_endpoint = urljoin(self._base_url, "/v1/collections")
         self._documents_endpoint = urljoin(self._base_url, "/v1/documents")
-        self._pdf_parser_endpoint = urljoin(self._base_url, "/v1/parse-beta")
         self._search_endpoint = urljoin(self._base_url, "/v1/search")
-
         self._default_collection_description = "Temporary collection for RAG document search"
+        self.parser = DoclingParser()
 
     def create_collection(self, name: str, description: Optional[str] = None) -> str:
         """
@@ -114,59 +110,7 @@ class AlbertRagBackend(BaseRagBackend):  # pylint: disable=too-many-instance-att
             )
             response.raise_for_status()
 
-    def parse_pdf_document(self, name: str, content_type: str, content: BytesIO) -> str:
-        """
-        Parse the PDF document content and return the text content.
-        This method should handle the logic to convert the PDF into
-        a format suitable for the Albert API.
-        """
-        response = requests.post(
-            self._pdf_parser_endpoint,
-            headers=self._headers,
-            files={
-                "file": (
-                    name,
-                    content,
-                    content_type,
-                ),  # Use the name as the filename in the request
-                "output_format": (None, "markdown"),  # Specify the output format as Markdown,
-            },
-            timeout=settings.ALBERT_API_PARSE_TIMEOUT,
-        )
-        response.raise_for_status()
-
-        return "\n\n".join(
-            document_page["content"] for document_page in response.json().get("data", [])
-        )
-
-    def parse_document(self, name: str, content_type: str, content: BytesIO):
-        """
-        Parse the document and prepare it for the search operation.
-        This method should handle the logic to convert the document
-        into a format suitable for the Albert API.
-
-        Args:
-            name (str): The name of the document.
-            content_type (str): The MIME type of the document (e.g., "application/pdf").
-            content (BytesIO): The content of the document as a BytesIO stream.
-
-        Returns:
-            str: The document content in Markdown format.
-        """
-        # Implement the parsing logic here
-        if content_type == "application/pdf":
-            # Handle PDF parsing
-            markdown_content = self.parse_pdf_document(
-                name=name, content_type=content_type, content=content
-            )
-        else:
-            markdown_content = DocumentConverter().convert_raw(
-                name=name, content_type=content_type, content=content
-            )
-
-        return markdown_content
-
-    def store_document(self, name: str, content: str) -> None:
+    def store_document(self, name: str, content: str, **kwargs) -> None:
         """
         Store the document content in the Albert collection.
         This method should handle the logic to send the document content to the Albert API.
@@ -174,6 +118,7 @@ class AlbertRagBackend(BaseRagBackend):  # pylint: disable=too-many-instance-att
         Args:
             name (str): The name of the document.
             content (str): The content of the document in Markdown format.
+            **kwargs: Additional arguments.
         """
         response = requests.post(
             urljoin(self._base_url, self._documents_endpoint),
@@ -188,7 +133,7 @@ class AlbertRagBackend(BaseRagBackend):  # pylint: disable=too-many-instance-att
         logger.debug(response.json())
         response.raise_for_status()
 
-    async def astore_document(self, name: str, content: str) -> None:
+    async def astore_document(self, name: str, content: str, **kwargs) -> None:
         """
         Store the document content in the Albert collection.
         This method should handle the logic to send the document content to the Albert API.
@@ -196,6 +141,7 @@ class AlbertRagBackend(BaseRagBackend):  # pylint: disable=too-many-instance-att
         Args:
             name (str): The name of the document.
             content (str): The content of the document in Markdown format.
+            **kwargs: Additional arguments.
         """
         async with httpx.AsyncClient(timeout=settings.ALBERT_API_TIMEOUT) as client:
             response = await client.post(
@@ -213,13 +159,14 @@ class AlbertRagBackend(BaseRagBackend):  # pylint: disable=too-many-instance-att
             logger.debug(response.json())
             response.raise_for_status()
 
-    def search(self, query, results_count: int = 4) -> RAGWebResults:
+    def search(self, query: str, results_count: int = 4, **kwargs) -> RAGWebResults:
         """
         Perform a search using the Albert API based on the provided query.
 
         Args:
             query (str): The search query.
             results_count (int): The number of results to return.
+            **kwargs: Additional arguments.
 
         Returns:
             RAGWebResults: The search results.
@@ -256,13 +203,14 @@ class AlbertRagBackend(BaseRagBackend):  # pylint: disable=too-many-instance-att
             ),
         )
 
-    async def asearch(self, query, results_count: int = 4) -> RAGWebResults:
+    async def asearch(self, query, results_count: int = 4, **kwargs) -> RAGWebResults:
         """
         Perform an asynchronous search using the Albert API based on the provided query.
 
         Args:
             query (str): The search query.
             results_count (int): The number of results to return.
+            **kwargs: Additional arguments.
 
         Returns:
             RAGWebResults: The search results.

src/backend/chat/agent_rag/document_rag_backends/base_rag_backend.py

@@ -8,6 +8,7 @@ from typing import List, Optional
 from asgiref.sync import sync_to_async
 
 from chat.agent_rag.constants import RAGWebResults
+from chat.agent_rag.document_converter.parser import BaseParser
 
 logger = logging.getLogger(__name__)
 
@@ -38,6 +39,7 @@ class BaseRagBackend:
         self.collection_id = collection_id
         self.read_only_collection_id = read_only_collection_id or []
         self._default_collection_description = "Temporary collection for RAG document search"
+        self.parser: BaseParser = BaseParser()
 
     def get_all_collection_ids(self) -> List[str]:
         """
@@ -53,7 +55,7 @@ class BaseRagBackend:
 
         collection_ids = []
         if self.collection_id:
-            collection_ids.append(int(self.collection_id))
+            collection_ids.append(self.collection_id)
         if self.read_only_collection_id:
             collection_ids.extend(
                 [int(collection_id) for collection_id in self.read_only_collection_id]
@@ -88,9 +90,9 @@ class BaseRagBackend:
         Returns:
             str: The document content in Markdown format.
         """
-        raise NotImplementedError("Must be implemented in subclass.")
+        return self.parser.parse_document(name, content_type, content)
 
-    def store_document(self, name: str, content: str) -> None:
+    def store_document(self, name: str, content: str, **kwargs) -> None:
         """
         Store the document content in the collection.
         This method should handle the logic to send the document content to the API.
@@ -98,10 +100,11 @@ class BaseRagBackend:
         Args:
             name (str): The name of the document.
             content (str): The content of the document in Markdown format.
+            **kwargs: Additional arguments. ex: "user_sub" for access control.
         """
         raise NotImplementedError("Must be implemented in subclass.")
 
-    async def astore_document(self, name: str, content: str) -> None:
+    async def astore_document(self, name: str, content: str, **kwargs) -> None:
         """
         Store the document content in the collection.
         This method should handle the logic to send the document content to the API.
@@ -109,10 +112,13 @@ class BaseRagBackend:
         Args:
             name (str): The name of the document.
             content (str): The content of the document in Markdown format.
+            **kwargs: Additional arguments. ex: "user_sub" for access control.
         """
-        return await sync_to_async(self.store_document)(name=name, content=content)
+        return await sync_to_async(self.store_document)(name=name, content=content, **kwargs)
 
-    def parse_and_store_document(self, name: str, content_type: str, content: BytesIO) -> str:
+    def parse_and_store_document(
+        self, name: str, content_type: str, content: BytesIO, **kwargs
+    ) -> str:
         """
         Parse the document and store it in the Albert collection.
 
@@ -120,12 +126,13 @@ class BaseRagBackend:
             name (str): The name of the document.
             content_type (str): The MIME type of the document (e.g., "application/pdf").
             content (BytesIO): The content of the document as a BytesIO stream.
+            **kwargs: Additional arguments. ex: "user_sub" for access control.
         """
         if not self.collection_id:
             raise RuntimeError("The RAG backend requires collection_id")
 
         document_content = self.parse_document(name, content_type, content)
-        self.store_document(name, document_content)
+        self.store_document(name, document_content, **kwargs)
         return document_content
 
     def delete_collection(self) -> None:
@@ -142,17 +149,27 @@ class BaseRagBackend:
         """
         return await sync_to_async(self.delete_collection)()
 
-    def search(self, query, results_count: int = 4) -> RAGWebResults:
+    def search(self, query: str, results_count: int = 4, **kwargs) -> RAGWebResults:
         """
         Search the collection for the given query.
+
+        Args:
+            query: The search query string.
+            results_count: Number of results to return.
+            **kwargs: Additional arguments. ex: 'session' for OIDC authentication.
         """
         raise NotImplementedError("Must be implemented in subclass.")
 
-    async def asearch(self, query, results_count: int = 4) -> RAGWebResults:
+    async def asearch(self, query: str, results_count: int = 4, **kwargs) -> RAGWebResults:
         """
-        Search the collection for the given query.
+        Search the collection for the given query asynchronously.
+
+        Args:
+            query: The search query string.
+            results_count: Number of results to return.
+            **kwargs: Additional arguments. ex: 'session' for OIDC authentication.
         """
-        return await sync_to_async(self.search)(query=query, results_count=results_count)
+        return await sync_to_async(self.search)(query=query, results_count=results_count, **kwargs)
 
     @classmethod
     @contextmanager

src/backend/chat/agent_rag/document_rag_backends/find_rag_backend.py

@@ -0,0 +1,153 @@
+"""Implementation of the Find API for RAG document search."""
+
+import logging
+import uuid
+from typing import List, Optional
+from urllib.parse import urljoin
+from uuid import uuid4
+
+from django.conf import settings
+from django.utils import timezone
+
+import requests
+
+from chat.agent_rag.constants import RAGWebResult, RAGWebResults, RAGWebUsage
+from chat.agent_rag.document_converter.parser import DoclingParser
+from chat.agent_rag.document_rag_backends.base_rag_backend import BaseRagBackend
+from utils.oidc import with_fresh_access_token
+
+logger = logging.getLogger(__name__)
+
+
+SUPPORTED_LANGUAGE_CODES = ["en", "fr", "de", "nl"]
+
+
+class FindRagBackend(BaseRagBackend):
+    """
+    This class is a placeholder for the Find API implementation.
+    It is designed to be used with the RAG (Retrieval-Augmented Generation) document search system.
+
+    It provides methods to:
+    - Store parsed documents in the Find index.
+    - Perform a search operation using the Find API.
+    """
+
+    def __init__(
+        self,
+        collection_id: Optional[str] = None,
+        read_only_collection_id: Optional[List[str]] = None,
+    ):
+        # Initialize any necessary parameters or configurations here
+        super().__init__(collection_id, read_only_collection_id)
+        self.api_key = settings.FIND_API_KEY
+        self.search_endpoint = "api/v1.0/documents/search/"
+        self.indexing_endpoint = "api/v1.0/documents/index/"
+        self.parser = DoclingParser()
+
+    def create_collection(self, name: str, description: Optional[str] = None) -> str:
+        """
+        init collection_id
+        """
+        self.collection_id = self.collection_id or str(uuid.uuid4())
+        return self.collection_id
+
+    def delete_collection(self) -> None:
+        """
+        Deletion not available
+        """
+        logger.warning("deletion of collections is not yet supported in FindRagBackend")
+
+    def store_document(self, name: str, content: str, **kwargs) -> None:
+        """
+        index document in Find
+
+        Args:
+            name (str): The name of the document.
+            content (str): The content of the document in Markdown format.
+            user_sub (str): The user subject identifier for access control.
+        """
+        logger.debug("index document '%s' in Find", name)
+
+        user_sub = kwargs.get("user_sub")
+        if not user_sub:
+            raise ValueError("user_sub is required to store document in FindRagBackend")
+
+        response = requests.post(
+            urljoin(settings.FIND_API_URL, self.indexing_endpoint),
+            headers={"Authorization": f"Bearer {self.api_key}"},
+            json={
+                "id": str(uuid4()),
+                "title": str(name) or "",
+                "depth": 0,
+                "path": str(name) or "",
+                "numchild": 0,
+                "content": content or "",
+                "created_at": timezone.now().isoformat(),
+                "updated_at": timezone.now().isoformat(),
+                "tags": [f"collection-{self.collection_id}"],
+                "size": len(content.encode("utf-8")),
+                "users": [user_sub],
+                "groups": [],
+                "reach": "authenticated",
+                "is_active": True,
+            },
+            timeout=settings.FIND_API_TIMEOUT,
+        )
+        response.raise_for_status()
+
+    @with_fresh_access_token
+    def search(self, query: str, results_count: int = 4, **kwargs) -> RAGWebResults:
+        """
+        Perform a search using the Find API.
+        Uses the user's OIDC token from the request session.
+
+        Args:
+            query: The search query.
+            results_count: Number of results to return.
+            **kwargs: Additional arguments. Expected: 'session' containing OIDC tokens,
+
+        Returns:
+            RAGWebResults: The search results.
+        """
+        logger.debug("search documents in Find with query '%s'", query)
+
+        response = requests.post(
+            urljoin(settings.FIND_API_URL, self.search_endpoint),
+            headers={"Authorization": f"Bearer {kwargs['session'].get('oidc_access_token')}"},
+            json={
+                "q": query,
+                "tags": [
+                    f"collection-{collection_id}" for collection_id in self.get_all_collection_ids()
+                ],
+                "k": results_count,
+            },
+            timeout=settings.FIND_API_TIMEOUT,
+        )
+        response.raise_for_status()
+
+        return RAGWebResults(
+            data=[
+                RAGWebResult(
+                    url=get_language_value(result["_source"], "title"),
+                    content=get_language_value(result["_source"], "content"),
+                    score=result["_score"],
+                )
+                for result in response.json()
+            ],
+            usage=RAGWebUsage(
+                prompt_tokens=0,
+                completion_tokens=0,
+            ),
+        )
+
+
+def get_language_value(source, language_field):
+    """
+    extract the value of the language field with the correct language_code extension.
+    "title" and "content" have extensions like "title.en" or "title.fr".
+    get_language_value will return the value regardless of the extension.
+    """
+    for language_code in SUPPORTED_LANGUAGE_CODES:
+        if f"{language_field}.{language_code}" in source:
+            return source[f"{language_field}.{language_code}"]
+    raise ValueError(f"No '{language_field}' field with any supported language code in object")

src/backend/chat/agent_rag/document_search/albert_api.py

@@ -11,7 +11,7 @@ import requests
 
 from chat.agent_rag.albert_api_constants import Searches
 from chat.agent_rag.constants import RAGWebResult, RAGWebResults, RAGWebUsage
-from chat.agent_rag.document_converter.markitdown import DocumentConverter
+from chat.agent_rag.document_converter.parser import DoclingParser
 from chat.models import ChatConversation
 
 logger = logging.getLogger(__name__)
@@ -80,58 +80,6 @@ class AlbertRagDocumentSearch:
         self.conversation.collection_id = str(response.json()["id"])
         return True
 
-    def _parse_pdf_document(self, name: str, content_type: str, content: BytesIO) -> str:
-        """
-        Parse the PDF document content and return the text content.
-        This method should handle the logic to convert the PDF into
-        a format suitable for the Albert API.
-        """
-        response = requests.post(
-            self._pdf_parser_endpoint,
-            headers=self._headers,
-            files={
-                "file": (
-                    name,
-                    content,
-                    content_type,
-                ),  # Use the name as the filename in the request
-                "output_format": (None, "markdown"),  # Specify the output format as Markdown,
-            },
-            timeout=settings.ALBERT_API_PARSE_TIMEOUT,
-        )
-        response.raise_for_status()
-
-        return "\n\n".join(
-            document_page["content"] for document_page in response.json().get("data", [])
-        )
-
-    def parse_document(self, name: str, content_type: str, content: BytesIO):
-        """
-        Parse the document and prepare it for the search operation.
-        This method should handle the logic to convert the document
-        into a format suitable for the Albert API.
-
-        Args:
-            name (str): The name of the document.
-            content_type (str): The MIME type of the document (e.g., "application/pdf").
-            content (BytesIO): The content of the document as a BytesIO stream.
-
-        Returns:
-            str: The document content in Markdown format.
-        """
-        # Implement the parsing logic here
-        if content_type == "application/pdf":
-            # Handle PDF parsing
-            markdown_content = self._parse_pdf_document(
-                name=name, content_type=content_type, content=content
-            )
-        else:
-            markdown_content = DocumentConverter().convert_raw(
-                name=name, content_type=content_type, content=content
-            )
-
-        return markdown_content
-
     def _store_document(self, name: str, content: str):
         """
         Store the document content in the Albert collection.
@@ -156,7 +104,7 @@ class AlbertRagDocumentSearch:
         logger.debug(response.json())
         response.raise_for_status()
 
-    def parse_and_store_document(self, name: str, content_type: str, content: BytesIO):
+    def parse_and_store_document(self, name: str, content_type: str, content: bytes):
         """
         Parse the document and store it in the Albert collection.
 
@@ -165,7 +113,9 @@ class AlbertRagDocumentSearch:
             content_type (str): The MIME type of the document (e.g., "application/pdf").
             content (BytesIO): The content of the document as a BytesIO stream.
         """
-        document_content = self.parse_document(name, content_type, content)
+        document_content = DoclingParser().parse_document(
+            name=name, content_type=content_type, content=content
+        )
         self._store_document(name, document_content)
         return document_content
 

src/backend/chat/clients/pydantic_ai.py

@@ -92,8 +92,8 @@ class ContextDeps:
 
     conversation: models.ChatConversation
     user: User
+    session: Optional[Dict] = None
     web_search_enabled: bool = False
-    data_analysis_enabled: bool = False
 
 
 def get_model_configuration(model_hrid: str):
@@ -107,7 +107,14 @@ def get_model_configuration(model_hrid: str):
 class AIAgentService:  # pylint: disable=too-many-instance-attributes
     """Service class for AI-related operations (Pydantic-AI edition)."""
 
-    def __init__(self, conversation: models.ChatConversation, user, model_hrid=None, language=None):
+    def __init__(  # pylint: disable=too-many-arguments,too-many-positional-arguments
+        self,
+        conversation: models.ChatConversation,
+        user,
+        session=None,
+        model_hrid=None,
+        language=None,
+    ):
         """
         Initialize the AI agent service.
 
@@ -132,14 +139,13 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
         # Feature flags
         self._is_document_upload_enabled = is_feature_enabled(self.user, "document_upload")
         self._is_web_search_enabled = is_feature_enabled(self.user, "web_search")
-        self._is_data_analysis_enabled = is_feature_enabled(self.user, "data_analysis")
         self._fake_streaming_delay = settings.FAKE_STREAMING_DELAY
 
         self._context_deps = ContextDeps(
             conversation=conversation,
             user=user,
+            session=session,
             web_search_enabled=self._is_web_search_enabled,
-            data_analysis_enabled=self._is_data_analysis_enabled,
         )
 
         self.conversation_agent = ConversationAgent(
@@ -281,6 +287,7 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
                         name=document.identifier,
                         content_type=document.media_type,
                         content=document_data,
+                        user_sub=self.user.sub,
                     )
                 else:
                     # Remote URL
@@ -290,28 +297,19 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
                     name=document.identifier,
                     content_type=document.media_type,
                     content=document.data,
+                    user_sub=self.user.sub,
                 )
 
             if not document.media_type.startswith("text/"):
-                # For non-text documents (PDF, Excel, images, etc.), we create a separate
-                # markdown attachment that contains the parsed text content.
-                # IMPORTANT: we must **not** overwrite the original binary file.
-                # If `key` is set, it points to the existing original object in storage;
-                # we derive a distinct markdown key from it so the original stays intact.
-                if key:
-                    md_key = f"{key}.md"
-                else:
-                    md_key = f"{self.conversation.pk}/attachments/{document.identifier}.md"
-
                 md_attachment = await models.ChatConversationAttachment.objects.acreate(
                     conversation=self.conversation,
                     uploaded_by=self.user,
-                    key=md_key,
+                    key=key or f"{self.conversation.pk}/attachments/{document.identifier}.md",
                     file_name=f"{document.identifier}.md",
                     content_type="text/markdown",
-                    conversion_from=key,  # original storage key, might be None
+                    conversion_from=key,  # might be None
                 )
-                default_storage.save(md_key, BytesIO(parsed_content.encode("utf8")))
+                default_storage.save(md_attachment.key, BytesIO(parsed_content.encode("utf8")))
                 md_attachment.upload_state = models.AttachmentStatus.READY
                 await md_attachment.asave(update_fields=["upload_state", "updated_at"])
 
@@ -565,7 +563,6 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
 
             _final_output_from_tool = None
             _ui_sources = []
-            _tool_names = {}  # Map tool_call_id to tool_name
 
             # Help Mistral to prevent `Unexpected role 'user' after role 'tool'` error.
             if history and history[-1].kind == "request":
@@ -664,8 +661,6 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
                                     dataclasses.asdict(event),
                                 )
                                 if isinstance(event, FunctionToolCallEvent):
-                                    # Store tool name for later use
-                                    _tool_names[event.tool_call_id] = event.part.tool_name
                                     if not _tool_is_streaming:
                                         yield events_v4.ToolCallPart(
                                             tool_call_id=event.tool_call_id,
@@ -694,39 +689,10 @@ class AIAgentService:  # pylint: disable=too-many-instance-attributes
                                                     **_new_source_ui.source.model_dump()
                                                 )
 
-                                        # Check if data_analysis tool was used and extract plot_url
-                                        tool_name = _tool_names.get(event.tool_call_id)
-                                        result_content = event.result.content
-                                        plot_url = None
-
-                                        if tool_name == "data_analysis":
-                                            logger.info(
-                                                f"Data analysis tool was used: {event.result}"
-                                            )
-
-                                            # Extract plot_url from metadata (not return_value - le modèle ne doit pas le voir)
-                                            if event.result.metadata:
-                                                plot_url = event.result.metadata.get("plot_url")
-                                                logger.info(
-                                                    f"Extracted plot_url from metadata: {plot_url}"
-                                                )
-
-                                            # Le plot_url n'est PAS dans return_value ni dans content
-                                            # donc le modèle ne le verra jamais - c'est parfait !
-
                                         yield events_v4.ToolResultPart(
                                             tool_call_id=event.tool_call_id,
-                                            result=result_content,
+                                            result=event.result.content,
                                         )
-
-                                        # If we have a plot_url, insert it directly into the stream as markdown image
-                                        if tool_name == "data_analysis" and plot_url:
-                                            logger.info(
-                                                f"Inserting plot_url directly into stream: {plot_url}"
-                                            )
-                                            yield events_v4.TextPart(
-                                                text=f"\n\n![Graphique de l'analyse]({plot_url})\n\n"
-                                            )
                                     elif isinstance(event.result, RetryPromptPart):
                                         yield events_v4.ToolResultPart(
                                             tool_call_id=event.tool_call_id,

src/backend/chat/mcp_servers.py

@@ -8,46 +8,13 @@ MCP_SERVERS = {
         #     "url": "https://api.githubcopilot.com/mcp/",
         #     "headers": {"Authorization": "Bearer XXX"},
         # },
-        # "data-analysis": {
-        #    "url": "http://host.docker.internal:8000/mcp",
-        # },
     }
 }
 
 
-DATA_ANALYSIS_MCP_SERVER = {
-    "data-analysis": {
-        "url": "http://host.docker.internal:8000/mcp",
-    },
-}
-
-
 def get_mcp_servers():
     """Retrieve MCP servers configuration."""
     return [
         MCPServerStreamableHTTP(**server_config)
         for _name, server_config in MCP_SERVERS["mcpServers"].items()
     ]
-
-
-from contextlib import asynccontextmanager
-
-from mcp import ClientSession
-from mcp.client.streamable_http import streamablehttp_client
-
-
-@asynccontextmanager
-async def get_data_analysis_mcp_server():
-    """
-    Connect to the data analysis MCP server and return the initialized session.
-    """
-    server_url = DATA_ANALYSIS_MCP_SERVER["data-analysis"]["url"]
-
-    # Create a streamable HTTP connection to the MCP server.
-    async with streamablehttp_client(server_url) as (read_stream, write_stream, _):
-        # Create a client session using the streams.
-        async with ClientSession(read_stream, write_stream) as session:
-            # Initialize the session (handshake).
-            await session.initialize()
-            # Yield the session so it stays open while being used
-            yield session

src/backend/chat/tests/agent_rag/document_converter/run_docling.py

@@ -0,0 +1,29 @@
+"""
+Unit tests for the DocumentConverter.
+
+Only for coverage as the DocumentConverter is a simple wrapper around MarkItDown.
+"""
+
+from io import BytesIO
+
+from docling.document_converter import DocumentConverter
+from docling_core.types.io import DocumentStream
+
+
+def main():
+    """Test that the DocumentConverter calls the underlying MarkItDown converter."""
+    file_path = "test.pdf"
+    converter = DocumentConverter()
+
+    # Convert from file content instead of file path
+    with open(file_path, "rb") as file:
+        content = file.read()
+        stream = DocumentStream(name="test.pdf", stream=BytesIO(content))
+        result = converter.convert(stream)
+        markdown = result.document.export_to_markdown()
+
+    assert markdown == "Document PDF test"
+
+
+if __name__ == "__main__":
+    main()

src/backend/chat/tests/agent_rag/document_converter/test.pdf

@@ -0,0 +1,90 @@
+%PDF-1.4
+%Çì�¢
+5 0 obj
+<</Length 6 0 R/Filter /FlateDecode>>
+stream
+xœMŽ±NÄ0†÷<…Çd¨ÏNœ¸^OÀÀ§l'¦Šc*¨âx’RªÚƒ­ÿóo{
BŽ@=ÿ›ivnq#¦«xì§ÎÕ�.
+ÍQoŽÐÌ„WÆ „#h!¨³»ú‡À˜5Sò_a Œ&¦â§�°•Ÿƒ4‡!¢ÊÅ¿ÿÑϽwÊ%çÑC—Y4[ò/a�ö³n‡D¢
+‹æhû¨Z<nØö‡�1F3Ýaj–·úì«{mùµi:uendstream
+endobj
+6 0 obj
+180
+endobj
+4 0 obj
+<</Type/Page/MediaBox [0 0 595 842]
+/Rotate 0/Parent 3 0 R
+/Resources<</ProcSet[/PDF /Text]
+/Font 8 0 R
+>>
+/Contents 5 0 R
+>>
+endobj
+3 0 obj
+<< /Type /Pages /Kids [
+4 0 R
+] /Count 1
+>>
+endobj
+1 0 obj
+<</Type /Catalog /Pages 3 0 R
+/Metadata 9 0 R
+>>
+endobj
+8 0 obj
+<</R7
+7 0 R>>
+endobj
+7 0 obj
+<</BaseFont/Times-Roman/Type/Font
+/Subtype/Type1>>
+endobj
+9 0 obj
+<</Type/Metadata
+/Subtype/XML/Length 1549>>stream
+<?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'?>
+<?adobe-xap-filters esc="CRLF"?>
+<x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='XMP toolkit 2.9.1-13, framework 1.6'>
+<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#' xmlns:iX='http://ns.adobe.com/iX/1.0/'>
+<rdf:Description rdf:about='uuid:81d69fb9-8bc7-11e4-0000-66b1dd18110c' xmlns:pdf='http://ns.adobe.com/pdf/1.3/'><pdf:Producer>GPL Ghostscript 9.06</pdf:Producer>
+<pdf:Keywords>()</pdf:Keywords>
+</rdf:Description>
+<rdf:Description rdf:about='uuid:81d69fb9-8bc7-11e4-0000-66b1dd18110c' xmlns:xmp='http://ns.adobe.com/xap/1.0/'><xmp:ModifyDate>2014-12-22T00:49:20+01:00</xmp:ModifyDate>
+<xmp:CreateDate>2014-12-22T00:49:20+01:00</xmp:CreateDate>
+<xmp:CreatorTool>PDFCreator Version 1.6.0</xmp:CreatorTool></rdf:Description>
+<rdf:Description rdf:about='uuid:81d69fb9-8bc7-11e4-0000-66b1dd18110c' xmlns:xapMM='http://ns.adobe.com/xap/1.0/mm/' xapMM:DocumentID='uuid:81d69fb9-8bc7-11e4-0000-66b1dd18110c'/>
+<rdf:Description rdf:about='uuid:81d69fb9-8bc7-11e4-0000-66b1dd18110c' xmlns:dc='http://purl.org/dc/elements/1.1/' dc:format='application/pdf'><dc:title><rdf:Alt><rdf:li xml:lang='x-default'>test_word</rdf:li></rdf:Alt></dc:title><dc:creator><rdf:Seq><rdf:li>Seb</rdf:li></rdf:Seq></dc:creator><dc:description><rdf:Seq><rdf:li>()</rdf:li></rdf:Seq></dc:description></rdf:Description>
+</rdf:RDF>
+</x:xmpmeta>
+                                                                        
+                                                                        
+<?xpacket end='w'?>
+endstream
+endobj
+2 0 obj
+<</Producer(GPL Ghostscript 9.06)
+/CreationDate(D:20141222004920+01'00')
+/ModDate(D:20141222004920+01'00')
+/Title(\376\377\000t\000e\000s\000t\000_\000w\000o\000r\000d)
+/Creator(\376\377\000P\000D\000F\000C\000r\000e\000a\000t\000o\000r\000 \000V\000e\000r\000s\000i\000o\000n\000 \0001\000.\0006\000.\0000)
+/Author(\376\377\000S\000e\000b)
+/Keywords()
+/Subject()>>endobj
+xref
+0 10
+0000000000 65535 f 
+0000000484 00000 n 
+0000002268 00000 n 
+0000000425 00000 n 
+0000000284 00000 n 
+0000000015 00000 n 
+0000000265 00000 n 
+0000000577 00000 n 
+0000000548 00000 n 
+0000000643 00000 n 
+trailer
+<< /Size 10 /Root 1 0 R /Info 2 0 R
+/ID [<0CB231047435B33BCE0B1C6881DCF011><0CB231047435B33BCE0B1C6881DCF011>]
+>>
+startxref
+2648
+%%EOF

src/backend/chat/tests/agent_rag/document_converter/test_docling.py

@@ -0,0 +1,18 @@
+"""
+Unit tests for the DoclingParser.
+"""
+from chat.agent_rag.document_converter.parser import DoclingParser
+
+
+def test_document_converter():
+    """Test that the DocumentConverter calls the underlying MarkItDown converter."""
+    file_name = "test"
+    content_type = "application/pdf"
+    file_path = "src/backend/chat/tests/data/test.pdf"
+    parser = DoclingParser()
+
+    with open(file_path, "rb") as file:
+        content = file.read()
+        result = parser.parse_document(name= file_name, content_type= content_type, content= content)
+
+    assert "Document PDF test" in result

src/backend/chat/tests/agent_rag/document_converter/test_markitdown.py

@@ -5,28 +5,21 @@ Only for coverage as the DocumentConverter is a simple wrapper around MarkItDown
 """
 
 from io import BytesIO
-from unittest.mock import MagicMock, patch
 
 from chat.agent_rag.document_converter.markitdown import DocumentConverter
 
 
-@patch("chat.agent_rag.document_converter.markitdown.MarkItDown")
-def test_document_converter(mock_markitdown: MagicMock):
+def test_document_converter():
     """Test that the DocumentConverter calls the underlying MarkItDown converter."""
-    mock_conversion = MagicMock()
-    mock_conversion.text_content = "converted text"
-    mock_markitdown.return_value.convert_stream.return_value = mock_conversion
-
+    file_path = "src/backend/chat/tests/data/test.pdf"
     converter = DocumentConverter()
 
-    result = converter.convert_raw(
-        name="test.pdf",
-        content_type="application/pdf",
-        content=b"test content",
-    )
+    with open(file_path, "rb") as file:
+        content = file.read()
+        result = converter.convert_raw(
+            name="test.pdf",
+            content_type="application/pdf",
+            content=content,
+        )
 
-    assert result == "converted text"
-    converter.converter.convert_stream.assert_called_once()  # pylint: disable=no-member
-    args, kwargs = converter.converter.convert_stream.call_args  # pylint: disable=no-member
-    assert isinstance(args[0], BytesIO)
-    assert kwargs["file_extension"] == ".pdf"
+    assert result == "Document PDF test\n\n"

src/backend/chat/tests/data/test.pdf

@@ -0,0 +1,90 @@
+%PDF-1.4
+%Çì�¢
+5 0 obj
+<</Length 6 0 R/Filter /FlateDecode>>
+stream
+xœMŽ±NÄ0†÷<…Çd¨ÏNœ¸^OÀÀ§l'¦Šc*¨âx’RªÚƒ­ÿóo{
BŽ@=ÿ›ivnq#¦«xì§ÎÕ�.
+ÍQoŽÐÌ„WÆ „#h!¨³»ú‡À˜5Sò_a Œ&¦â§�°•Ÿƒ4‡!¢ÊÅ¿ÿÑϽwÊ%çÑC—Y4[ò/a�ö³n‡D¢
+‹æhû¨Z<nØö‡�1F3Ýaj–·úì«{mùµi:uendstream
+endobj
+6 0 obj
+180
+endobj
+4 0 obj
+<</Type/Page/MediaBox [0 0 595 842]
+/Rotate 0/Parent 3 0 R
+/Resources<</ProcSet[/PDF /Text]
+/Font 8 0 R
+>>
+/Contents 5 0 R
+>>
+endobj
+3 0 obj
+<< /Type /Pages /Kids [
+4 0 R
+] /Count 1
+>>
+endobj
+1 0 obj
+<</Type /Catalog /Pages 3 0 R
+/Metadata 9 0 R
+>>
+endobj
+8 0 obj
+<</R7
+7 0 R>>
+endobj
+7 0 obj
+<</BaseFont/Times-Roman/Type/Font
+/Subtype/Type1>>
+endobj
+9 0 obj
+<</Type/Metadata
+/Subtype/XML/Length 1549>>stream
+<?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'?>
+<?adobe-xap-filters esc="CRLF"?>
+<x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='XMP toolkit 2.9.1-13, framework 1.6'>
+<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#' xmlns:iX='http://ns.adobe.com/iX/1.0/'>
+<rdf:Description rdf:about='uuid:81d69fb9-8bc7-11e4-0000-66b1dd18110c' xmlns:pdf='http://ns.adobe.com/pdf/1.3/'><pdf:Producer>GPL Ghostscript 9.06</pdf:Producer>
+<pdf:Keywords>()</pdf:Keywords>
+</rdf:Description>
+<rdf:Description rdf:about='uuid:81d69fb9-8bc7-11e4-0000-66b1dd18110c' xmlns:xmp='http://ns.adobe.com/xap/1.0/'><xmp:ModifyDate>2014-12-22T00:49:20+01:00</xmp:ModifyDate>
+<xmp:CreateDate>2014-12-22T00:49:20+01:00</xmp:CreateDate>
+<xmp:CreatorTool>PDFCreator Version 1.6.0</xmp:CreatorTool></rdf:Description>
+<rdf:Description rdf:about='uuid:81d69fb9-8bc7-11e4-0000-66b1dd18110c' xmlns:xapMM='http://ns.adobe.com/xap/1.0/mm/' xapMM:DocumentID='uuid:81d69fb9-8bc7-11e4-0000-66b1dd18110c'/>
+<rdf:Description rdf:about='uuid:81d69fb9-8bc7-11e4-0000-66b1dd18110c' xmlns:dc='http://purl.org/dc/elements/1.1/' dc:format='application/pdf'><dc:title><rdf:Alt><rdf:li xml:lang='x-default'>test_word</rdf:li></rdf:Alt></dc:title><dc:creator><rdf:Seq><rdf:li>Seb</rdf:li></rdf:Seq></dc:creator><dc:description><rdf:Seq><rdf:li>()</rdf:li></rdf:Seq></dc:description></rdf:Description>
+</rdf:RDF>
+</x:xmpmeta>
+                                                                        
+                                                                        
+<?xpacket end='w'?>
+endstream
+endobj
+2 0 obj
+<</Producer(GPL Ghostscript 9.06)
+/CreationDate(D:20141222004920+01'00')
+/ModDate(D:20141222004920+01'00')
+/Title(\376\377\000t\000e\000s\000t\000_\000w\000o\000r\000d)
+/Creator(\376\377\000P\000D\000F\000C\000r\000e\000a\000t\000o\000r\000 \000V\000e\000r\000s\000i\000o\000n\000 \0001\000.\0006\000.\0000)
+/Author(\376\377\000S\000e\000b)
+/Keywords()
+/Subject()>>endobj
+xref
+0 10
+0000000000 65535 f 
+0000000484 00000 n 
+0000002268 00000 n 
+0000000425 00000 n 
+0000000284 00000 n 
+0000000015 00000 n 
+0000000265 00000 n 
+0000000577 00000 n 
+0000000548 00000 n 
+0000000643 00000 n 
+trailer
+<< /Size 10 /Root 1 0 R /Info 2 0 R
+/ID [<0CB231047435B33BCE0B1C6881DCF011><0CB231047435B33BCE0B1C6881DCF011>]
+>>
+startxref
+2648
+%%EOF

src/backend/chat/tests/tools/test_web_search_brave.py

@@ -38,9 +38,6 @@ def brave_settings(settings):
     settings.BRAVE_SEARCH_EXTRA_SNIPPETS = True
     settings.BRAVE_SUMMARIZATION_ENABLED = False
     settings.BRAVE_CACHE_TTL = 3600
-    settings.RAG_DOCUMENT_SEARCH_BACKEND = (
-        "chat.agent_rag.document_rag_backends.albert_rag_backend.AlbertRagBackend"
-    )
     settings.BRAVE_RAG_WEB_SEARCH_CHUNK_NUMBER = 5
 
 

src/backend/chat/tests/views/chat/conftest.py

@@ -0,0 +1,17 @@
+"""Common test fixtures for chat views tests."""
+
+from unittest import mock
+
+import pytest
+
+
+@pytest.fixture(autouse=True)
+def mock_process_request():
+    """
+    Mock process_request to bypass OIDC authentication in tests.
+    """
+    with mock.patch(
+        "lasuite.oidc_login.decorators.RefreshOIDCAccessToken.process_request"
+    ) as mocked_process_request:
+        mocked_process_request.return_value = None
+        yield mocked_process_request

src/backend/chat/tests/views/chat/conversations/test_conversation_with_document_upload.py

@@ -8,6 +8,7 @@ import logging
 from io import BytesIO
 from unittest import mock
 
+from django.contrib.sessions.backends.cache import SessionStore
 from django.utils import formats, timezone
 
 import httpx
@@ -41,28 +42,49 @@ from chat.tests.utils import replace_uuids_with_placeholder
 pytestmark = pytest.mark.django_db(transaction=True)
 
 
-@pytest.fixture(autouse=True)
-def ai_settings(settings):
+@pytest.fixture(
+    autouse=True,
+    params=[
+        "chat.agent_rag.document_rag_backends.find_rag_backend.FindRagBackend",
+        "chat.agent_rag.document_rag_backends.albert_rag_backend.AlbertRagBackend",
+    ],
+)
+def ai_settings(request, settings):
     """Fixture to set AI service URLs for testing."""
-    settings.AI_BASE_URL = "https://www.external-ai-service.com/"
-    settings.AI_API_KEY = "test-api-key"
-    settings.AI_MODEL = "test-model"
-    settings.AI_AGENT_INSTRUCTIONS = "You are a helpful test assistant :)"
 
-    # Enable Albert API for document search
-    settings.RAG_DOCUMENT_SEARCH_BACKEND = (
-        "chat.agent_rag.document_rag_backends.albert_rag_backend.AlbertRagBackend"
-    )
-    settings.ALBERT_API_URL = "https://albert.api.etalab.gouv.fr"
-    settings.ALBERT_API_KEY = "albert-api-key"
+    # enable on rag document search tool
+    settings.RAG_DOCUMENT_SEARCH_BACKEND = request.param
     settings.RAG_WEB_SEARCH_PROMPT_UPDATE = (
         "Based on the following document contents:\n\n{search_results}\n\n"
         "Please answer the user's question: {user_prompt}"
     )
 
+    settings.AI_BASE_URL = "https://www.external-ai-service.com/"
+    settings.AI_API_KEY = "test-api-key"
+    settings.AI_MODEL = "test-model"
+    settings.AI_AGENT_INSTRUCTIONS = "You are a helpful test assistant :)"
+
+    # Albert API settings
+    settings.ALBERT_API_URL = "https://albert.api.etalab.gouv.fr"
+    settings.ALBERT_API_KEY = "albert-api-key"
+
+    # Find API settings
+    settings.FIND_API_URL = "https://find.api.example.com"
+    settings.FIND_API_KEY = "find-api-key"
+
     return settings
 
 
+@pytest.fixture(autouse=True)
+def mock_refresh_access_token():
+    """Mock refresh_access_token to bypass token refresh in tests."""
+    with mock.patch("utils.oidc.refresh_access_token") as mocked_refresh_access_token:
+        session = SessionStore()
+        session["oidc_access_token"] = "mocked-access-token"
+        mocked_refresh_access_token.return_value = session
+        yield mocked_refresh_access_token
+
+
 @pytest.fixture(name="sample_pdf_content")
 def fixture_sample_pdf_content():
     """Create a dummy PDF content as BytesIO."""
@@ -81,17 +103,25 @@ def fixture_sample_pdf_content():
     return BytesIO(pdf_data)
 
 
-@pytest.fixture(name="mock_albert_api")
-def fixture_mock_albert_api():
+@pytest.fixture(name="mock_document_api")
+def fixture_mock_document_api():
     """Fixture to mock the Albert API endpoints."""
     # Mock collection creation
+
+    document_name = "sample.pdf"
+    document_content = "This is the content of the PDF."
+    prompt_tokens = 10
+    completion_tokens = 20
+    search_method = "semantic"
+    search_score = 0.9
+
     responses.post(
         "https://albert.api.etalab.gouv.fr/v1/collections",
         json={"id": "123", "name": "test-collection"},
         status=status.HTTP_200_OK,
     )
 
-    # Mock PDF parsing
+    # Mock Albert PDF parsing -> deprecated
     responses.post(
         "https://albert.api.etalab.gouv.fr/v1/parse-beta",
         json={
@@ -101,7 +131,7 @@ def fixture_mock_albert_api():
                     "metadata": {"document_name": "sample.pdf"},
                 }
             ],
-            "usage": {"prompt_tokens": 10, "completion_tokens": 20},
+            "usage": {"prompt_tokens": prompt_tokens, "completion_tokens": completion_tokens},
         },
         status=status.HTTP_200_OK,
     )
@@ -119,20 +149,42 @@ def fixture_mock_albert_api():
         json={
             "data": [
                 {
-                    "method": "semantic",
+                    "method": search_method,
                     "chunk": {
                         "id": 123,
-                        "content": "This is the content of the PDF.",
-                        "metadata": {"document_name": "sample.pdf"},
+                        "content": document_content,
+                        "metadata": {"document_name": document_name},
                     },
-                    "score": 0.9,
+                    "score": search_score,
                 }
             ],
-            "usage": {"prompt_tokens": 10, "completion_tokens": 20},
+            "usage": {"prompt_tokens": prompt_tokens, "completion_tokens": completion_tokens},
         },
         status=status.HTTP_200_OK,
     )
 
+    # Mock document indexing (Find API)
+    responses.post(
+        "https://find.api.example.com/api/v1.0/documents/index/",
+        json={"id": "456", "status": "indexed"},
+        status=status.HTTP_200_OK,
+    )
+
+    # Mock document search (Find API)
+    responses.post(
+        "https://find.api.example.com/api/v1.0/documents/search/",
+        json=[
+            {
+                "_source": {
+                    "title.fr": document_name,
+                    "content.fr": document_content,
+                },
+                "_score": search_score,
+            }
+        ],
+        status=status.HTTP_200_OK,
+    )
+
 
 @pytest.fixture(name="mock_summarization_agent")
 def fixture_mock_summarization_agent():
@@ -219,7 +271,7 @@ def fixture_mock_openai_stream():
 def test_post_conversation_with_document_upload(
     # pylint: disable=too-many-arguments,too-many-positional-arguments
     api_client,
-    mock_albert_api,  # pylint: disable=unused-argument
+    mock_document_api,  # pylint: disable=unused-argument
     sample_pdf_content,
     today_promt_date,
     mock_ai_agent_service,
@@ -548,7 +600,7 @@ def test_post_conversation_with_document_upload_feature_disabled(
 @freeze_time()
 def test_post_conversation_with_document_upload_summarize(  # pylint: disable=too-many-arguments,too-many-positional-arguments  # noqa: PLR0913
     api_client,
-    mock_albert_api,  # pylint: disable=unused-argument
+    mock_document_api,  # pylint: disable=unused-argument
     sample_pdf_content,
     today_promt_date,
     mock_ai_agent_service,
@@ -623,7 +675,7 @@ def test_post_conversation_with_document_upload_summarize(  # pylint: disable=to
         'document discusses various topics."}\n'
         '0:"The document discusses various topics."\n'
         'f:{"messageId":"<mocked_uuid>"}\n'
-        'd:{"finishReason":"stop","usage":{"promptTokens":287,"completionTokens":19}}\n'
+        'd:{"finishReason":"stop","usage":{"promptTokens":283,"completionTokens":19}}\n'
     )
 
     # Check that the conversation was updated

src/backend/chat/tests/views/chat/conversations/test_conversation_with_document_url.py

@@ -37,11 +37,19 @@ from chat.tests.utils import replace_uuids_with_placeholder
 pytestmark = pytest.mark.django_db(transaction=True)
 
 
-@pytest.fixture(autouse=True)
-def ai_settings(settings):
+@pytest.fixture(
+    autouse=True,
+    params=[
+        "chat.agent_rag.document_rag_backends.find_rag_backend.FindRagBackend",
+        "chat.agent_rag.document_rag_backends.albert_rag_backend.AlbertRagBackend",
+    ],
+)
+def ai_settings(request, settings):
     """Fixture to set AI service URLs for testing."""
+    settings.RAG_DOCUMENT_SEARCH_BACKEND = request.param
     settings.AI_BASE_URL = "https://www.external-ai-service.com/"
     settings.AI_API_KEY = "test-api-key"
+    settings.FIND_API_KEY = "find-api-key"
     settings.AI_MODEL = "test-model"
     settings.AI_AGENT_INSTRUCTIONS = "You are a helpful test assistant :)"
     return settings
@@ -85,6 +93,10 @@ def test_post_conversation_with_local_pdf_document_url(
         json={"id": "document_id", "object": "document"},
         status=200,
     )
+    responses.post(
+        "https://app-find/api/v1.0/documents/index/",
+        status=200,
+    )
 
     chat_conversation = ChatConversationFactory(owner__language="en-us")
     api_client.force_authenticate(user=chat_conversation.owner)
@@ -795,6 +807,10 @@ def test_post_conversation_with_local_not_pdf_document_url(
         json={"id": "document_id", "object": "document"},
         status=200,
     )
+    responses.post(
+        "https://app-find/api/v1.0/documents/index/",
+        status=200,
+    )
 
     chat_conversation = ChatConversationFactory(owner__language="en-us")
     api_client.force_authenticate(user=chat_conversation.owner)

src/backend/chat/tools/__init__.py

@@ -2,18 +2,12 @@
 
 from pydantic_ai import Tool, ToolDefinition
 
-from .data_analysis import data_analysis
 from .fake_current_weather import get_current_weather
 from .web_seach_albert_rag import web_search_albert_rag
 from .web_search_brave import web_search_brave, web_search_brave_with_document_backend
 from .web_search_tavily import web_search_tavily
 
 
-async def only_if_data_analysis_enabled(ctx, tool_def: ToolDefinition) -> ToolDefinition | None:
-    """Prepare function to include a tool only if data analysis is enabled in the context."""
-    return tool_def if ctx.deps.data_analysis_enabled else None
-
-
 async def only_if_web_search_enabled(ctx, tool_def: ToolDefinition) -> ToolDefinition | None:
     """Prepare function to include a tool only if web search is enabled in the context."""
     return tool_def if ctx.deps.web_search_enabled else None
@@ -47,12 +41,6 @@ def get_pydantic_tools_by_name(name: str) -> Tool:
             prepare=only_if_web_search_enabled,
             max_retries=2,
         ),
-        "data_analysis": Tool(
-            data_analysis,
-            takes_ctx=True,
-            prepare=only_if_data_analysis_enabled,
-            max_retries=2,
-        ),
     }
 
     return tool_dict[name]  # will raise on purpose if name is not found

src/backend/chat/tools/data_analysis.py

@@ -1,151 +0,0 @@
-import base64
-import json
-import logging
-import uuid
-from io import BytesIO
-
-from django.core.files.storage import default_storage
-from django.db.models import Q
-
-import boto3
-import botocore
-from asgiref.sync import sync_to_async
-from pydantic_ai import RunContext, RunUsage
-from pydantic_ai.messages import ToolReturn
-
-from core.file_upload.utils import generate_retrieve_policy
-
-from chat import models
-from chat.mcp_servers import get_data_analysis_mcp_server
-from conversations import settings
-
-logger = logging.getLogger(__name__)
-
-
-async def data_analysis(ctx: RunContext, query: str) -> ToolReturn:
-    """
-    Call this tool to perform a data analysis or draw a plot from a file and data.
-    When asking for a plot and if the user made no specific instructions, add to the query that the plot should be elegant and easy to read, with harmonious colors.
-
-    Args:
-        query: The query to perform the data analysis/To plot stuff or compute stuff from files and data.
-        The query should be very clear and precise, explaining what results you expect, tables, numbers or plots.
-
-    Returns:
-        The result of the data analysis and/or the plot requested.
-    """
-    # Prepare files - Get all attachments in the conversation (exclude markdown conversions)
-    # Filter for CSV/Excel files that can be analyzed
-    attachments = [
-        attachment
-        async for attachment in models.ChatConversationAttachment.objects.filter(
-            Q(conversion_from__isnull=True) | Q(conversion_from=""),
-            conversation=ctx.deps.conversation,
-            upload_state=models.AttachmentStatus.READY,
-        ).exclude(content_type="text/markdown")
-    ]
-
-    # Filter for tabular files (CSV, Excel)
-    tabular_attachments = [
-        att for att in attachments if att.file_name.endswith((".csv", ".xls", ".xlsx"))
-    ]
-
-    # Prepare tool arguments
-    tool_args = {"query": query}
-
-    # S3 client dedicated to MCP URLs (endpoint = AWS_S3_MCP_URL, e.g. ngrok)
-    mcp_s3_client = boto3.client(
-        "s3",
-        aws_access_key_id=settings.AWS_S3_ACCESS_KEY_ID,
-        aws_secret_access_key=settings.AWS_S3_SECRET_ACCESS_KEY,
-        endpoint_url=settings.AWS_S3_MCP_URL,
-        config=botocore.client.Config(
-            region_name=settings.AWS_S3_REGION_NAME,
-            signature_version=settings.AWS_S3_SIGNATURE_VERSION,
-        ),
-    )
-
-    # If we have tabular files, use the first one (or let the tool handle multiple files)
-    # TODO: Handle multiple files
-    if tabular_attachments:
-        logger.debug(f"Tabular file found: {tabular_attachments[-1].file_name}")
-        # Use the last tabular file found
-        attachment = tabular_attachments[-1]
-        presigned_url = mcp_s3_client.generate_presigned_url(
-            ClientMethod="get_object",
-            Params={"Bucket": default_storage.bucket_name, "Key": attachment.key},
-            ExpiresIn=settings.AWS_S3_RETRIEVE_POLICY_EXPIRATION,
-        )
-        tool_args["document"] = presigned_url
-    else:
-        return ToolReturn(
-            return_value={"error": "No tabular file found, ask the user to upload a tabular file."},
-            content="",
-            metadata={},
-        )
-    tool_args["document_name"] = attachment.file_name
-    logger.debug(f"Tool arguments: {tool_args}")
-
-    # Connect to MCP server and call the tool
-    async with get_data_analysis_mcp_server() as session:
-        tool_result = await session.call_tool(
-            "data_analysis_tool",
-            tool_args,
-        )
-        logger.info(f"Tool result: {tool_result}")
-        logger.info(f"Tool result type: {type(tool_result)}")
-
-        # tool_result is a CallToolResult MCP
-        parsed_result = {}
-        if getattr(tool_result, "content", None):
-            first_content = tool_result.content[0]
-            text = getattr(first_content, "text", str(first_content))
-            try:
-                parsed_result = json.loads(text)
-            except json.JSONDecodeError:
-                parsed_result = {"raw": text}
-        else:
-            parsed_result = {"raw": str(tool_result)}
-
-        # Prepare results
-        result = {
-            "result": str(parsed_result.get("result")),
-        }
-        metadata = {
-            "query": parsed_result.get("query"),
-            "query_code": parsed_result.get("query_code"),
-            "metadata": parsed_result.get("metadata"),
-        }
-
-        # Check if result has plot
-        plot_image_base64 = parsed_result.get("plot_image")
-        plot_url = None
-
-        if plot_image_base64:
-            # Decode base64 image
-            plot_image = base64.b64decode(plot_image_base64)
-
-            plot_filename = f"plot_{uuid.uuid4().hex[:8]}.png"
-            plot_key = f"{ctx.deps.conversation.pk}/plots/{plot_filename}"
-
-            # Save to storage
-            await sync_to_async(default_storage.save)(plot_key, BytesIO(plot_image))
-
-            browser_plot_url = await sync_to_async(generate_retrieve_policy)(plot_key)
-            plot_url = browser_plot_url
-
-            # Do NOT include plot_url in result so the model can't see it.
-            # plot_url will be added to the stream by pydantic_ai.py.
-            # Add a clear message in the content for the model.
-            result["result"] += (
-                "Le graphique a été inséré automatiquement dans la conversation pour l'utilisateur. "
-                "Ne donnes JAMAIS d'url de plot."
-                "Dis à l'utilisateur 'Tu trouveras le graphique ci-dessus.' ou quelque chose comme ça et commente le graphique si besoin."
-            )
-            metadata["plot_url"] = plot_url
-
-    return ToolReturn(
-        return_value=result,
-        content="",
-        metadata=metadata,
-    )

src/backend/chat/tools/document_search_rag.py

@@ -26,7 +26,7 @@ def add_document_rag_search_tool(agent: Agent) -> None:
 
         document_store = document_store_backend(ctx.deps.conversation.collection_id)
 
-        rag_results = document_store.search(query)
+        rag_results = document_store.search(query, session=ctx.deps.session)
 
         ctx.usage += RunUsage(
             input_tokens=rag_results.usage.prompt_tokens,

src/backend/chat/tools/document_summarize.py

@@ -101,7 +101,7 @@ async def document_summarize(  # pylint: disable=too-many-locals
         )
         documents_chunks = chunker(
             [doc[1] for doc in documents],
-            overlap=settings.SUMMARIZATION_OVERLAP_SIZE,
+            # overlap=settings.SUMMARIZATION_OVERLAP_SIZE,
         )
 
         logger.info(

src/backend/chat/tools/web_search_brave.py

@@ -127,7 +127,7 @@ async def _extract_and_summarize_snippets_async(query: str, url: str) -> List[st
         return []
 
 
-async def _fetch_and_store_async(url: str, document_store) -> None:
+async def _fetch_and_store_async(url: str, document_store, **kwargs) -> None:
     """Fetch, extract and store text content from the URL in the document store."""
 
     try:
@@ -136,7 +136,7 @@ async def _fetch_and_store_async(url: str, document_store) -> None:
         logger.debug("Fetched document: %s", document)
 
         if document:
-            await document_store.astore_document(url, document)
+            await document_store.astore_document(url, document, **kwargs)
     except DocumentFetchError as e:
         logger.warning("Failed to fetch and store %s: %s", url, e)
         # Continue with other documents

src/backend/chat/views.py

@@ -7,11 +7,13 @@ from uuid import uuid4
 from django.conf import settings
 from django.core.files.storage import default_storage
 from django.http import Http404, StreamingHttpResponse
+from django.utils.decorators import method_decorator
 
 import langfuse
 import magic
 import posthog
 from lasuite.malware_detection import malware_detection
+from lasuite.oidc_login.decorators import refresh_oidc_access_token
 from rest_framework import decorators, filters, mixins, permissions, status, viewsets
 from rest_framework.exceptions import MethodNotAllowed, PermissionDenied, ValidationError
 from rest_framework.response import Response
@@ -122,6 +124,7 @@ class ChatViewSet(  # pylint: disable=too-many-ancestors, abstract-method
             self.permission_classes = []
         return super().get_permissions()
 
+    @method_decorator(refresh_oidc_access_token)
     @decorators.action(
         methods=["post"],
         detail=True,
@@ -173,6 +176,7 @@ class ChatViewSet(  # pylint: disable=too-many-ancestors, abstract-method
         ai_service = AIAgentService(
             conversation=conversation,
             user=self.request.user,
+            session=request.session,
             model_hrid=model_hrid,
             language=(
                 self.request.user.language

src/backend/conftest.py

@@ -22,7 +22,7 @@ def no_http_requests(monkeypatch):
     Credits: https://blog.jerrycodes.com/no-http-requests/
     """
 
-    allowed_hosts = {"localhost", "minio", "minio:9000"}
+    allowed_hosts = {"localhost", "127.0.0.1", "minio", "minio:9000"}
     original_urlopen = HTTPConnectionPool.urlopen
 
     def urlopen_mock(self, method, url, *args, **kwargs):

src/backend/conversations/settings.py

@@ -151,10 +151,6 @@ class Base(BraveSettings, Configuration):
         environ_name="AWS_S3_DOMAIN_REPLACE",
         environ_prefix=None,
     )
-    AWS_S3_MCP_URL = values.Value(
-        environ_name="AWS_S3_MCP_URL",
-        environ_prefix=None,
-    )
 
     ATTACHMENT_CHECK_UNSAFE_MIME_TYPES_ENABLED = values.BooleanValue(
         True,
@@ -845,6 +841,23 @@ USER QUESTION:
         environ_prefix=None,
     )
 
+    # Find
+    FIND_API_KEY = values.Value(
+        None,
+        environ_name="FIND_API_KEY",
+        environ_prefix=None,
+    )
+    FIND_API_URL = values.Value(
+        "https://app-find/api",
+        environ_name="FIND_API_URL",
+        environ_prefix=None,
+    )
+    FIND_API_TIMEOUT = values.PositiveIntegerValue(
+        default=30,  # seconds
+        environ_name="FIND_API_TIMEOUT",
+        environ_prefix=None,
+    )
+
     # Logging
     # We want to make it easy to log to console but by default we log production
     # to Sentry and don't want to log to console.

src/backend/core/feature_flags/flags.py

@@ -44,7 +44,6 @@ class FeatureFlags(BaseModel):
     # features
     web_search: FeatureToggle = FeatureToggle.DISABLED
     document_upload: FeatureToggle = FeatureToggle.DISABLED
-    data_analysis: FeatureToggle = FeatureToggle.DISABLED
 
     def __getattr__(self, name: str):
         """Dynamically get specific RAG document search tool feature flags from settings."""

src/backend/pyproject.toml

@@ -43,7 +43,9 @@ dependencies = [
     "djangorestframework==3.16.1",
     "drf_spectacular==0.29.0",
     "dockerflow==2024.4.2",
+    "docling",
     "easy_thumbnails==2.10.1",
+    "easyocr",
     "factory_boy==3.3.3",
     "gunicorn==23.0.0",
     "jsonschema==4.25.1",

src/backend/utils/oidc.py

@@ -0,0 +1,54 @@
+"""Utility functions for OIDC token management."""
+
+from functools import wraps
+
+from django.conf import settings
+
+import requests
+from lasuite.oidc_login.backends import get_oidc_refresh_token, store_tokens
+from rest_framework.exceptions import AuthenticationFailed
+
+
+def refresh_access_token(session):
+    """Refresh the OIDC access token using the refresh token."""
+    refresh_token = get_oidc_refresh_token(session)
+    if not refresh_token:
+        raise AuthenticationFailed({"error": "Refresh token is missing from session"})
+
+    response = requests.post(
+        settings.OIDC_OP_TOKEN_ENDPOINT,
+        data={
+            "grant_type": "refresh_token",
+            "client_id": settings.OIDC_RP_CLIENT_ID,
+            "client_secret": settings.OIDC_RP_CLIENT_SECRET,
+            "refresh_token": refresh_token,
+        },
+        timeout=5,
+    )
+    response.raise_for_status()
+    token_info = response.json()
+
+    store_tokens(
+        session,
+        access_token=token_info.get("access_token"),
+        id_token=None,
+        refresh_token=token_info.get("refresh_token"),
+    )
+    return session
+
+
+def with_fresh_access_token(func):
+    """
+    Decorator to handle OIDC token refresh and extraction.
+    Expects 'session' in kwargs and update it with the fresh token.
+    """
+
+    @wraps(func)
+    def wrapper(*args, **kwargs):
+        session = kwargs.pop("session", None)
+        if session is None:
+            raise AuthenticationFailed({"error": "Session is required but not provided"})
+        refreshed_session = refresh_access_token(session)
+        return func(*args, session=refreshed_session, **kwargs)
+
+    return wrapper