⬆️(dependencies) update python dependencies

I am improving the data management of evaluation system

.dockerignore

@@ -0,0 +1,36 @@
+# Python
+__pycache__
+*.pyc
+**/__pycache__
+**/*.pyc
+venv
+.venv
+
+# System-specific files
+.DS_Store
+**/.DS_Store
+
+# Docker
+docker compose.*
+env.d
+
+# Docs
+docs
+*.md
+*.log
+
+# Development/test cache & configurations
+data
+.cache
+.circleci
+.git
+.vscode
+.iml
+.idea
+db.sqlite3
+.mypy_cache
+.pylint.d
+.pytest_cache
+
+# Frontend
+node_modules

.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,6 @@
+<!---
+Thanks for filing an issue 😄 ! Before you submit, please read the following:
+
+Check the other issue templates if you are trying to submit a bug report, feature request, or question
+Search open/closed issues before submitting since someone might have asked the same thing before!
+-->

.github/ISSUE_TEMPLATE/Bug_report.md

@@ -0,0 +1,28 @@
+---
+name: 🐛 Bug Report
+about: If something is not working as expected 🤔.
+
+---
+
+## Bug Report
+
+**Problematic behavior**
+A clear and concise description of the behavior.
+
+**Expected behavior/code**
+A clear and concise description of what you expected to happen (or code).
+
+**Steps to Reproduce**
+1. Do this...
+2. Then this...
+3. And then the bug happens!
+
+**Environment**
+- Drive version:
+- Platform:
+
+**Possible Solution**
+<!--- Only if you have suggestions on a fix for the bug -->
+
+**Additional context/Screenshots**
+Add any other context about the problem here. If applicable, add screenshots to help explain.

.github/ISSUE_TEMPLATE/Feature_request.md

@@ -0,0 +1,23 @@
+---
+name: ✨ Feature Request
+about: I have a suggestion (and may want to build it 💪)!
+
+---
+
+## Feature Request
+
+**Is your feature request related to a problem or unsupported use case? Please describe.**
+A clear and concise description of what the problem is. For example: I need to do some task and I have an issue...
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen. Add any considered drawbacks.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Discovery, Documentation, Adoption, Migration Strategy**
+If you can, explain how users will be able to use this and possibly write out a version the docs (if applicable).
+Maybe a screenshot or design?
+
+**Do you want to work on it through a Pull Request?**
+<!-- Make sure to coordinate with us before you spend too much time working on an implementation! -->

.github/ISSUE_TEMPLATE/Support_question.md

@@ -0,0 +1,22 @@
+---
+name: 🤗 Support Question
+about: If you have a question 💬, or something was not clear from the docs!
+
+---
+
+<!-- ^ Click "Preview" for a nicer view! ^
+We primarily use GitHub as an issue tracker. If however you're encountering an issue not covered in the docs, we may be able to help! -->
+
+---
+
+Please make sure you have read our [main Readme](https://github.com/suitenumerique/find).
+
+Also make sure it was not already answered in [an open or close issue](https://github.com/suitenumerique/find/issues).
+
+If your question was not covered, and you feel like it should be, fire away! We'd love to improve our docs! 👌
+
+**Topic**
+What's the general area of your question: for example, docker setup, database schema, search functionality,...
+
+**Question**
+Try to be as specific as possible so we can help you as best we can. Please be patient 🙏

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,11 @@
+## Purpose
+
+Description...
+
+
+## Proposal
+
+Description...
+
+- [ ] item 1...
+- [ ] item 2...

.github/workflows/docker-hub.yml

@@ -0,0 +1,57 @@
+name: Docker Hub Workflow
+
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - 'main'
+    tags:
+      - 'v*'
+  pull_request:
+    branches:
+      - 'main'
+
+env:
+  DOCKER_USER: 1001:127
+
+jobs:
+  build-and-push-backend:
+    runs-on: ubuntu-latest
+    steps:
+      -
+        name: Checkout repository
+        uses: actions/checkout@v6
+      -
+        name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+      -
+        name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+      -
+        name: Docker meta
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: lasuite/find
+      -
+        name: Login to DockerHub
+        if: github.event_name != 'pull_request'
+        run: echo "${{ secrets.DOCKER_HUB_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_HUB_USER }}" --password-stdin
+      -
+        name: Run trivy scan
+        uses: numerique-gouv/action-trivy-cache@main
+        with:
+          docker-build-args: '--target backend-production -f Dockerfile'
+          docker-image-name: 'docker.io/lasuite/find:${{ github.sha }}'
+        continue-on-error: true
+      -
+        name: Build and push
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          target: backend-production
+          platforms: linux/amd64,linux/arm64
+          build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
+          push: ${{ github.event_name != 'pull_request' }}
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}

.github/workflows/find.yml

@@ -0,0 +1,165 @@
+name: Main Workflow
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - "*"
+
+jobs:
+  lint-git:
+    runs-on: ubuntu-latest
+    if: github.event_name == 'pull_request' # Makes sense only for pull requests
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+      - name: show
+        run: git log
+      - name: Enforce absence of print statements in code
+        run: |
+          ! git diff origin/${{ github.event.pull_request.base.ref }}..HEAD -- . ':(exclude)**/find.yml' | grep "print("
+      - name: Check absence of fixup commits
+        run: |
+          ! git log | grep 'fixup!'
+      - name: Install gitlint
+        run: pip install --user requests gitlint
+      - name: Lint commit messages added to main
+        run: ~/.local/bin/gitlint --commits origin/${{ github.event.pull_request.base.ref }}..HEAD
+
+  check-changelog:
+    runs-on: ubuntu-latest
+    if: |
+      contains(github.event.pull_request.labels.*.name, 'noChangeLog') == false &&
+      github.event_name == 'pull_request'
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v6
+        with:
+          fetch-depth: 50
+      - name: Check that the CHANGELOG has been modified in the current branch
+        run: git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.event.after }} | grep 'CHANGELOG.md'
+
+  lint-changelog:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v6
+      - name: Check CHANGELOG max line length
+        run: |
+          max_line_length=$(cat CHANGELOG.md | grep -Ev "^\[.*\]: https://github.com" | wc -L)
+          if [ $max_line_length -ge 80 ]; then
+            echo "ERROR: CHANGELOG has lines longer than 80 characters."
+            exit 1
+          fi
+
+  lint-back:
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: src/backend
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v6
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version-file: "src/backend/pyproject.toml"
+      - name: Install uv
+        uses: astral-sh/setup-uv@v6
+      - name: Install the project
+        run: uv sync --locked --all-extras
+
+      - name: Check code formatting with ruff
+        run: uv run ruff format . --diff
+      - name: Lint code with ruff
+        run: uv run ruff check .
+      - name: Lint code with pylint
+        run: uv run pylint .
+
+  test-back:
+    runs-on: ubuntu-latest
+
+    defaults:
+      run:
+        working-directory: src/backend
+
+    services:
+      postgres:
+        image: postgres:15
+        env:
+          POSTGRES_DB: find
+          POSTGRES_USER: dinum
+          POSTGRES_PASSWORD: pass
+        ports:
+          - 5432:5432
+        # needed because the postgres container does not provide a healthcheck
+        options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
+
+      opensearch:
+        image: opensearchproject/opensearch:latest
+        env:
+          discovery.type: single-node
+          plugins.security.disabled: true
+          plugins.security.ssl.http.enabled: false
+          OPENSEARCH_INITIAL_ADMIN_PASSWORD: find.PASS123
+        ports:
+          - 9200:9200
+        options: >-
+          --health-cmd "curl -s http://localhost:9200 > /dev/null || exit 1"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 10
+
+    env:
+      DJANGO_CONFIGURATION: Test
+      DJANGO_SETTINGS_MODULE: find.settings
+      DJANGO_SECRET_KEY: ThisIsAnExampleKeyForTestPurposeOnly
+      OIDC_OP_JWKS_ENDPOINT: /endpoint-for-test-purpose-only
+      DB_HOST: localhost
+      DB_NAME: find
+      DB_USER: dinum
+      DB_PASSWORD: pass
+      DB_PORT: 5432
+      STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
+      OPENSEARCH_HOST: localhost
+      OPENSEARCH_PASSWORD: find.PASS123
+      OPENSEARCH_USE_SSL: false
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v6
+
+      - name: Create writable /data
+        run: |
+          sudo mkdir -p /data/media && \
+          sudo mkdir -p /data/static
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version-file: "src/backend/pyproject.toml"
+      - name: Install uv
+        uses: astral-sh/setup-uv@v6
+
+      - name: Wait for OpenSearch to be ready
+        run: |
+          for i in {1..60}; do
+            if curl -s http://localhost:9200 > /dev/null; then
+              echo "OpenSearch is ready"
+              exit 0
+            fi
+            echo "Waiting for OpenSearch... attempt $i"
+            sleep 1
+          done
+          echo "OpenSearch failed to start"
+          exit 1
+
+      - name: Install the dependencies
+        run: uv sync --locked --all-extras
+
+      - name: Run tests
+        run: uv run pytest

.github/workflows/release-helm-chart.yaml

@@ -0,0 +1,40 @@
+name: Release Chart
+run-name: Release Chart
+
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - 'main'
+    tags:
+      - 'v*'
+  pull_request:
+    branches:
+      - 'main'
+  
+jobs:
+  release:
+    if: github.event_name != 'pull_request'
+    permissions:
+      contents: write
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+
+      - name: Cleanup
+        run: rm -rf ./src/helm/extra
+
+      - name: Install Helm
+        uses: azure/setup-helm@v4
+        env:
+          GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
+
+      - name: Publish Helm charts
+        uses: numerique-gouv/helm-gh-pages@master
+        with:
+          charts_dir: ./src/helm
+          linting: on
+          token: ${{ secrets.GITHUB_TOKEN }}

.gitignore

@@ -0,0 +1,86 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+.DS_Store
+.next/
+
+# Translations	# Translations
+*.pot
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+env.d/development/*
+!env.d/development/*.dist
+env.d/terraform
+
+# npm
+node_modules
+
+# Mails
+src/backend/core/templates/mail/
+
+# Typescript client
+src/frontend/tsclient
+
+# Swagger
+**/swagger.json
+
+# Logs
+*.log
+
+# Terraform
+.terraform
+*.tfstate
+*.tfstate.backup
+
+# Test & lint
+.coverage
+.pylint.d
+.pytest_cache
+db.sqlite3
+*history.sqlite    
+.mypy_cache
+
+# Site media
+/data/
+
+# IDEs
+.idea/
+.vscode/
+*.iml
+.devcontainer
+.vscode-server
+
+#pdb
+*.pdbhistory

.gitlint

@@ -0,0 +1,78 @@
+# All these sections are optional, edit this file as you like.
+[general]
+# Ignore certain rules, you can reference them by their id or by their full name
+# ignore=title-trailing-punctuation, T3
+
+# verbosity should be a value between 1 and 3, the commandline -v flags take precedence over this
+# verbosity = 2
+
+# By default gitlint will ignore merge commits. Set to 'false' to disable.
+# ignore-merge-commits=true
+
+# By default gitlint will ignore fixup commits. Set to 'false' to disable.
+# ignore-fixup-commits=true
+
+# By default gitlint will ignore squash commits. Set to 'false' to disable.
+# ignore-squash-commits=true
+
+# Enable debug mode (prints more output). Disabled by default.
+# debug=true
+
+# Set the extra-path where gitlint will search for user defined rules
+# See http://jorisroovers.github.io/gitlint/user_defined_rules for details
+extra-path=gitlint/
+
+# [title-max-length]
+# line-length=80
+
+[title-must-not-contain-word]
+# Comma-separated list of words that should not occur in the title. Matching is case
+# insensitive. It's fine if the keyword occurs as part of a larger word (so "WIPING"
+# will not cause a violation, but "WIP: my title" will.
+words=wip
+
+#[title-match-regex]
+# python like regex (https://docs.python.org/2/library/re.html) that the
+# commit-msg title must be matched to.
+# Note that the regex can contradict with other rules if not used correctly
+# (e.g. title-must-not-contain-word).
+#regex=
+
+# [B1]
+# B1 = body-max-line-length
+# line-length=120
+# [body-min-length]
+# min-length=5
+
+# [body-is-missing]
+# Whether to ignore this rule on merge commits (which typically only have a title)
+# default = True
+# ignore-merge-commits=false
+
+# [body-changed-file-mention]
+# List of files that need to be explicitly mentioned in the body when they are changed
+# This is useful for when developers often erroneously edit certain files or git submodules.
+# By specifying this rule, developers can only change the file when they explicitly reference
+# it in the commit message.
+# files=gitlint/rules.py,README.md
+
+# [author-valid-email]
+# python like regex (https://docs.python.org/2/library/re.html) that the
+# commit author email address should be matched to
+# For example, use the following regex if you only want to allow email addresses from foo.com
+# regex=[^@]+@foo.com
+
+[ignore-by-title]
+# Allow empty body & wrong title pattern only when bots (pyup/greenkeeper)
+# upgrade dependencies
+regex=^(⬆️.*|Update (.*) from (.*) to (.*)|(chore|fix)\(package\): update .*)$
+ignore=B6,UC1
+
+# [ignore-by-body]
+# Ignore certain rules for commits of which the body has a line that matches a regex
+# E.g. Match bodies that have a line that that contain "release"
+# regex=(.*)release(.*)
+#
+# Ignore certain rules, you can reference them by their id or by their full name
+# Use 'all' to ignore all rules
+# ignore=T1,body-min-length

CHANGELOG.md

@@ -0,0 +1,44 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0),
+and this project adheres to
+[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+# Unreleased
+
+## Added
+
+- 👷(docker) add arm64 platform support for image builds
+- ✨(backend) add semantic search
+- ✨(backend) add multi-embedding and chunking
+- ✨(backend) add analyzers to full-text search
+- ✨(backend) handle french, english, german and dutch 
+- ✨(backend) add evaluation command
+- backend application
+- helm chart
+- ✨(backend) allow indexation of documents with either empty content or title.
+- ✨(api) new fulltext 'search/' view with OIDC resource server authentication
+- ✨(backend) limit access to documents : public & authenticated with a
+              list of services
+- 🔧(compose) rename docker network 'lasuite-net' as 'lasuite-network'
+- ✨(backend) add demo service for Drive.
+- ✨(backend) add OPENSEARCH_INDEX_PREFIX setting to prevent naming overlaping
+              issues if the opensearch database is shared between apps.
+- ✨(backend) add tags
+- ✨(backend) adapt to conversation RAG
+- ✨(backend) add deletion endpoint
+- ✨(backend) add path filter
+- ✨(backend) add search_type param
+
+## Changed
+
+- 🏗️(backend) switch Python dependency management to uv
+- ✨(backend) allow deletion by tags
+- ♻️(backend) improve the evaluation command
+
+## Fixed
+
+- 🐛(backend) fix missing index creation in 'index/' view
+- 🐛(backend) fix parallel test execution issues

Dockerfile

@@ -0,0 +1,146 @@
+# Django find
+
+# ---- base image to inherit from ----
+FROM python:3.12-slim-bookworm AS base
+
+# Upgrade system packages to install security updates
+RUN apt-get update && \
+  apt-get -y upgrade && \
+  rm -rf /var/lib/apt/lists/*
+
+# ---- Back-end builder image ----
+FROM base AS back-builder
+
+ENV UV_COMPILE_BYTECODE=1
+ENV UV_LINK_MODE=copy
+
+# Disable Python downloads, because we want to use the system interpreter
+# across both images. If using a managed Python version, it needs to be
+# copied from the build image into the final image;
+ENV UV_PYTHON_DOWNLOADS=0
+
+# install uv
+COPY --from=ghcr.io/astral-sh/uv:0.9.10 /uv /uvx /bin/
+
+WORKDIR /app
+
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=bind,source=src/backend/uv.lock,target=uv.lock \
+    --mount=type=bind,source=src/backend/pyproject.toml,target=pyproject.toml \
+    uv sync --locked --no-install-project --no-dev
+
+COPY ./src/backend /app
+
+RUN --mount=type=cache,target=/root/.cache/uv \
+    uv sync --locked --no-dev
+
+# ---- static link collector ----
+FROM base AS link-collector
+ARG FIND_STATIC_ROOT=/data/static
+
+# Install libpangocairo & rdfind
+RUN apt-get update && \
+    apt-get install -y \
+      libpangocairo-1.0-0 \
+      rdfind && \
+    rm -rf /var/lib/apt/lists/*
+
+# Copy the application from the builder
+COPY --from=back-builder /app /app
+
+ENV PATH="/app/.venv/bin:$PATH"
+
+WORKDIR /app
+
+# collectstatic
+RUN DJANGO_CONFIGURATION=Build \
+    DJANGO_JWT_PRIVATE_SIGNING_KEY=Dummy \
+    OPENSEARCH_PASSWORD=Dummy \
+    python manage.py collectstatic --noinput
+
+# Replace duplicated file by a symlink to decrease the overall size of the
+# final image
+RUN rdfind -makesymlinks true -followsymlinks true -makeresultsfile false ${FIND_STATIC_ROOT}
+
+# ---- Core application image ----
+FROM base AS core
+
+ENV PYTHONUNBUFFERED=1
+
+# Install required system libs
+RUN apt-get update && \
+    apt-get install -y \
+      gettext \
+      libcairo2 \
+      libffi-dev \
+      libgdk-pixbuf2.0-0 \
+      libpango-1.0-0 \
+      libpangocairo-1.0-0 \
+      shared-mime-info && \
+  rm -rf /var/lib/apt/lists/*
+
+# Copy entrypoint
+COPY ./docker/files/usr/local/bin/entrypoint /usr/local/bin/entrypoint
+
+# Give the "root" group the same permissions as the "root" user on /etc/passwd
+# to allow a user belonging to the root group to add new users; typically the
+# docker user (see entrypoint).
+RUN chmod g=u /etc/passwd
+
+# Copy the prepared application (see .dockerignore)
+COPY --from=back-builder /app /app
+
+WORKDIR /app
+
+ENV PATH="/app/.venv/bin:$PATH"
+
+# We wrap commands run in this container by the following entrypoint that
+# creates a user on-the-fly with the container user ID (see USER) and root group
+# ID.
+ENTRYPOINT [ "/usr/local/bin/entrypoint" ]
+
+# ---- Development image ----
+FROM core AS backend-development
+
+# Switch back to the root user to install development dependencies
+USER root:root
+
+# Install psql
+RUN apt-get update && \
+    apt-get install -y postgresql-client && \
+    rm -rf /var/lib/apt/lists/*
+
+# Install development dependencies
+RUN --mount=from=ghcr.io/astral-sh/uv:0.9.10,source=/uv,target=/bin/uv \
+    uv sync --locked --all-extras
+
+# Restore the un-privileged user running the application
+ARG DOCKER_USER
+USER ${DOCKER_USER}
+
+# Target database host (e.g. database engine following docker compose services
+# name) & port
+ENV DB_HOST=postgresql \
+    DB_PORT=5432
+
+# Run django development server
+CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
+
+# ---- Production image ----
+FROM core AS backend-production
+
+ARG FIND_STATIC_ROOT=/data/static
+
+# Gunicorn
+RUN mkdir -p /usr/local/etc/gunicorn
+COPY docker/files/usr/local/etc/gunicorn/find.py /usr/local/etc/gunicorn/find.py
+
+# Un-privileged user running the application
+ARG DOCKER_USER
+USER ${DOCKER_USER}
+
+# Copy statics
+COPY --from=link-collector ${FIND_STATIC_ROOT} ${FIND_STATIC_ROOT}
+
+# The default command runs gunicorn WSGI server in find's main module
+CMD ["gunicorn", "-c", "/usr/local/etc/gunicorn/find.py", "find.wsgi:application"]

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Direction Interministérielle du Numérique - Gouvernement Français
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

Makefile

@@ -0,0 +1,264 @@
+# /!\ /!\ /!\ /!\ /!\ /!\ /!\ DISCLAIMER /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\
+#
+# This Makefile is only meant to be used for DEVELOPMENT purpose as we are
+# changing the user id that will run in the container.
+#
+# PLEASE DO NOT USE IT FOR YOUR CI/PRODUCTION/WHATEVER...
+#
+# /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\
+#
+# Note to developers:
+#
+# While editing this file, please respect the following statements:
+#
+# 1. Every variable should be defined in the ad hoc VARIABLES section with a
+#    relevant subsection
+# 2. Every new rule should be defined in the ad hoc RULES section with a
+#    relevant subsection depending on the targeted service
+# 3. Rules should be sorted alphabetically within their section
+# 4. When a rule has multiple dependencies, you should:
+#    - duplicate the rule name to add the help string (if required)
+#    - write one dependency per line to increase readability and diffs
+# 5. .PHONY rule statement should be written after the corresponding rule
+# ==============================================================================
+# VARIABLES
+
+BOLD := \033[1m
+RESET := \033[0m
+GREEN := \033[1;32m
+
+
+# -- Database
+
+DB_HOST            = postgresql
+DB_PORT            = 5432
+
+# -- Docker
+# Get the current user ID to use for docker run and docker exec commands
+DOCKER_UID          = $(shell id -u)
+DOCKER_GID          = $(shell id -g)
+DOCKER_USER         = $(DOCKER_UID):$(DOCKER_GID)
+COMPOSE             = DOCKER_USER=$(DOCKER_USER) docker compose
+COMPOSE_EXEC        = $(COMPOSE) exec
+COMPOSE_EXEC_APP    = $(COMPOSE_EXEC) app
+COMPOSE_RUN         = $(COMPOSE) run --rm
+COMPOSE_RUN_APP     = $(COMPOSE_RUN) app
+COMPOSE_RUN_CROWDIN = $(COMPOSE_RUN) crowdin crowdin
+WAIT_DB             = @$(COMPOSE_RUN) dockerize -wait tcp://$(DB_HOST):$(DB_PORT) -timeout 60s
+
+# -- Backend
+MANAGE              = $(COMPOSE_RUN_APP) python manage.py
+
+# ==============================================================================
+# RULES
+
+default: help
+
+data/opensearch:
+	@mkdir -p data/opensearch
+
+data/static:
+	@mkdir -p data/static
+
+# -- Project
+
+create-env-files: ## Copy the dist env files to env files
+create-env-files: \
+	env.d/development/common \
+	env.d/development/crowdin \
+	env.d/development/postgresql
+.PHONY: create-env-files
+
+bootstrap: ## Prepare Docker images for the project
+bootstrap: \
+	data/opensearch \
+	data/static \
+	create-env-files \
+	build \
+	migrate \
+	demo \
+	back-i18n-generate \
+	back-i18n-compile
+.PHONY: bootstrap
+
+# -- Docker/compose
+build: ## build the app container
+	@$(COMPOSE) build app --no-cache
+.PHONY: build
+
+down: ## stop and remove containers, networks, images, and volumes
+	@$(COMPOSE) down
+.PHONY: down
+
+logs: ## display app logs (follow mode)
+	@$(COMPOSE) logs -f app
+.PHONY: logs
+
+run: ## start the wsgi (production) and development server
+	@$(COMPOSE) up --force-recreate -d celery
+	@echo "Wait for postgresql to be up..."
+	@$(WAIT_DB)
+.PHONY: run
+
+status: ## an alias for "docker compose ps"
+	@$(COMPOSE) ps
+.PHONY: status
+
+stop: ## stop the development server using Docker
+	@$(COMPOSE) stop
+.PHONY: stop
+
+# -- Backend
+
+demo: ## flush db then create a demo for load testing purpose
+	@$(MAKE) resetdb
+	@$(MANAGE) create_demo
+.PHONY: demo
+
+# Nota bene: Black should come after isort just in case they don't agree...
+lint: ## lint back-end python sources
+lint: \
+  lint-ruff-format \
+  lint-ruff-check \
+  lint-pylint
+.PHONY: lint
+
+lint-ruff-format: ## format back-end python sources with ruff
+	@echo 'lint:ruff-format started…'
+	@$(COMPOSE_RUN_APP) ruff format .
+.PHONY: lint-ruff-format
+
+lint-ruff-check: ## lint back-end python sources with ruff
+	@echo 'lint:ruff-check started…'
+	@$(COMPOSE_RUN_APP) ruff check . --fix
+.PHONY: lint-ruff-check
+
+lint-pylint: ## lint back-end python sources with pylint only on changed files from main
+	@echo 'lint:pylint started…'
+	bin/pylint .
+.PHONY: lint-pylint
+
+test: ## run project tests
+	@$(MAKE) test-back-parallel
+.PHONY: test
+
+test-back: ## run back-end tests
+	@args="$(filter-out $@,$(MAKECMDGOALS))" && \
+	bin/pytest $${args:-${1}}
+.PHONY: test-back
+
+test-back-parallel: ## run all back-end tests in parallel
+	@args="$(filter-out $@,$(MAKECMDGOALS))" && \
+	bin/pytest -n auto $${args:-${1}}
+.PHONY: test-back-parallel
+
+makemigrations:  ## run django makemigrations for the find project.
+	@echo "$(BOLD)Running makemigrations$(RESET)"
+	@$(COMPOSE) up -d postgresql
+	@$(WAIT_DB)
+	@$(MANAGE) makemigrations
+.PHONY: makemigrations
+
+migrate:  ## run django migrations for the find project.
+	@echo "$(BOLD)Running migrations$(RESET)"
+	@$(COMPOSE) up -d postgresql
+	@$(WAIT_DB)
+	@$(MANAGE) migrate
+.PHONY: migrate
+
+superuser: ## Create an admin superuser with password "admin"
+	@echo "$(BOLD)Creating a Django superuser$(RESET)"
+	@$(WAIT_DB)
+	@$(MANAGE) shell -c "from core.models import User; not User.objects.filter(username='admin').exists() and User.objects.create_superuser('admin', 'admin@example.com', 'admin')"
+.PHONY: superuser
+
+back-i18n-compile: ## compile the gettext files
+	@$(MANAGE) compilemessages --ignore=".venv/**/*"
+.PHONY: back-i18n-compile
+
+back-i18n-generate: ## create the .pot files used for i18n
+	@$(MANAGE) makemessages -a --keep-pot
+.PHONY: back-i18n-generate
+
+shell: ## connect to database shell
+	@$(MANAGE) shell #_plus
+.PHONY: dbshell
+
+# -- Database
+
+dbshell: ## connect to database shell
+	docker compose exec app python manage.py dbshell
+.PHONY: dbshell
+
+resetdb: FLUSH_ARGS ?=
+resetdb: ## flush database and create a superuser "admin"
+	@echo "$(BOLD)Flush database$(RESET)"
+	@$(MANAGE) flush $(FLUSH_ARGS)
+	@${MAKE} superuser
+.PHONY: resetdb
+
+env.d/development/common:
+	cp --update=none env.d/development/common.dist env.d/development/common
+
+env.d/development/postgresql:
+	cp --update=none env.d/development/postgresql.dist env.d/development/postgresql
+
+# -- Internationalization
+
+env.d/development/crowdin:
+	cp --update=none env.d/development/crowdin.dist env.d/development/crowdin
+
+crowdin-download: ## Download translated message from crowdin
+	@$(COMPOSE_RUN_CROWDIN) download -c crowdin/config.yml
+.PHONY: crowdin-download
+
+crowdin-download-sources: ## Download sources from Crowdin
+	@$(COMPOSE_RUN_CROWDIN) download sources -c crowdin/config.yml
+.PHONY: crowdin-download-sources
+
+crowdin-upload: ## Upload source translations to crowdin
+	@$(COMPOSE_RUN_CROWDIN) upload sources -c crowdin/config.yml
+.PHONY: crowdin-upload
+
+i18n-compile: ## compile all translations
+i18n-compile: \
+	back-i18n-compile
+.PHONY: i18n-compile
+
+i18n-generate: ## create the .pot files used for i18n
+i18n-generate: \
+	back-i18n-generate
+.PHONY: i18n-generate
+
+i18n-download-and-compile: ## download all translated messages and compile them to be used by all applications
+i18n-download-and-compile: \
+  crowdin-download \
+  i18n-compile
+.PHONY: i18n-download-and-compile
+
+i18n-generate-and-upload: ## generate source translations for all applications and upload them to Crowdin
+i18n-generate-and-upload: \
+  i18n-generate \
+  crowdin-upload
+.PHONY: i18n-generate-and-upload
+
+# -- Misc
+clean: ## restore repository state as it was freshly cloned
+	git clean -idx
+.PHONY: clean
+
+help:
+	@echo "$(BOLD)find Makefile"
+	@echo "Please use 'make $(BOLD)target$(RESET)' where $(BOLD)target$(RESET) is one of:"
+	@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(firstword $(MAKEFILE_LIST)) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "$(GREEN)%-30s$(RESET) %s\n", $$1, $$2}'
+.PHONY: help
+
+# -- K8S
+build-k8s-cluster: ## build the kubernetes cluster using kind
+	./bin/start-kind.sh
+.PHONY: build-k8s-cluster
+
+start-tilt: ## start the kubernetes cluster using kind
+	tilt up -f ./bin/Tiltfile
+.PHONY: build-k8s-cluster
+

README.md

@@ -0,0 +1,75 @@
+# Find
+
+Find can index documents from several applications sharing a common OIDC federation
+and allows users to search documents with their access rights accross all applications
+in the federation.
+
+Find is built on top of [Django Rest
+Framework](https://www.django-rest-framework.org/).
+
+## Getting started
+
+### Prerequisite
+
+Make sure you have a recent version of Docker and [Docker
+Compose](https://docs.docker.com/compose/install) installed on your laptop:
+
+```bash
+$ docker -v
+  Docker version 27.4.1, build b9d17ea
+
+$ docker compose version
+  Docker Compose version v2.32.1
+```
+
+> ⚠️ You may need to run the following commands with `sudo` but this can be
+> avoided by assigning your user to the `docker` group. See docker 
+> [Documentation](https://docs.docker.com/engine/install/linux-postinstall/)
+
+### Project bootstrap
+
+The easiest way to start working on the project is to use GNU Make:
+
+```bash
+$ make bootstrap
+```
+
+This command builds the `app` container, installs dependencies, performs
+database migrations and compile translations. It's a good idea to use this
+command each time you are pulling code from the project repository to avoid
+dependency-releated or migration-releated issues.
+
+Your Docker services should now be up and running 🎉
+
+### Adding content
+
+You can create a basic demo site by running:
+
+    $ make demo
+
+Finally, you can check all available Make rules using:
+
+```bash
+$ make help
+```
+
+### Django admin
+
+You can access the Django admin site at
+[http://localhost:8071/admin](http://localhost:8071/admin).
+
+You first need to create a superuser account:
+
+```bash
+$ make superuser
+```
+
+## Contributing
+
+This project is intended to be community-driven, so please, do not hesitate to
+get in touch if you have any question related to our implementation or design
+decisions.
+
+## License
+
+This work is released under the MIT License (see [LICENSE](./LICENSE)).

bin/Tiltfile

@@ -0,0 +1,47 @@
+load('ext://uibutton', 'cmd_button', 'bool_input', 'location')
+load('ext://namespace', 'namespace_create', 'namespace_inject')
+namespace_create('find')
+
+docker_build(
+    'localhost:5001/find-backend:latest',
+    context='..',
+    dockerfile='../Dockerfile',
+    only=['./src/backend', './docker'],
+    target = 'backend-production',
+    live_update=[
+        sync('../src/backend', '/app'),
+        run(
+            'pip install .',
+            trigger=['../src/backend/pyproject.toml']
+        )
+    ]
+)
+
+
+k8s_yaml(local('cd ../src/helm && helmfile -n find -e dev template .'))
+
+migrate_db = '''
+set -eu
+# get k8s pod name from tilt resource name
+POD_NAME="$(tilt get kubernetesdiscovery find-backend -ojsonpath='{.status.pods[0].name}')"
+kubectl -n find exec "$POD_NAME" -- python manage.py migrate --no-input
+'''
+cmd_button('Migrate db',
+           argv=['sh', '-c', migrate_db],
+           resource='find-backend',
+           icon_name='developer_board',
+           text='Run database migration',
+)
+
+create_demo = '''
+set -eu
+# get k8s pod name from tilt resource name
+POD_NAME="$(tilt get kubernetesdiscovery find-backend -ojsonpath='{.status.pods[0].name}')"
+kubectl -n find exec "$POD_NAME" -- python manage.py create_demo --force
+'''
+cmd_button('Create demo fixtures',
+           argv=['sh', '-c', create_demo],
+           resource='find-backend',
+           icon_name='developer_board',
+           text='Run demo fixtures',
+)

bin/_config.sh

@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+
+set -eo pipefail
+
+REPO_DIR="$(cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd)"
+UNSET_USER=0
+
+COMPOSE_FILE="${REPO_DIR}/compose.yml"
+COMPOSE_PROJECT="find"
+
+
+# _set_user: set (or unset) default user id used to run docker commands
+#
+# usage: _set_user
+#
+# You can override default user ID (the current host user ID), by defining the
+# USER_ID environment variable.
+#
+# To avoid running docker commands with a custom user, please set the
+# $UNSET_USER environment variable to 1.
+function _set_user() {
+
+    if [ $UNSET_USER -eq 1 ]; then
+        USER_ID=""
+        return
+    fi
+
+    # USER_ID = USER_ID or `id -u` if USER_ID is not set
+    USER_ID=${USER_ID:-$(id -u)}
+
+    echo "🙋(user) ID: ${USER_ID}"
+}
+
+# docker_compose: wrap docker compose command
+#
+# usage: docker_compose [options] [ARGS...]
+#
+# options: docker compose command options
+# ARGS   : docker compose command arguments
+function _docker_compose() {
+
+    echo "🐳(compose) project: '${COMPOSE_PROJECT}' file: '${COMPOSE_FILE}'"
+    docker compose \
+        -p "${COMPOSE_PROJECT}" \
+        -f "${COMPOSE_FILE}" \
+        --project-directory "${REPO_DIR}" \
+        "$@"
+}
+
+# _dc_run: wrap docker compose run command
+#
+# usage: _dc_run [options] [ARGS...]
+#
+# options: docker compose run command options
+# ARGS   : docker compose run command arguments
+function _dc_run() {
+    _set_user
+
+    user_args="--user=$USER_ID"
+    if [ -z $USER_ID ]; then
+        user_args=""
+    fi
+
+    _docker_compose run --rm $user_args "$@"
+}
+
+# _dc_exec: wrap docker compose exec command
+#
+# usage: _dc_exec [options] [ARGS...]
+#
+# options: docker compose exec command options
+# ARGS   : docker compose exec command arguments
+function _dc_exec() {
+    _set_user
+
+    echo "🐳(compose) exec command: '\$@'"
+
+    user_args="--user=$USER_ID"
+    if [ -z $USER_ID ]; then
+        user_args=""
+    fi
+
+    _docker_compose exec $user_args "$@"
+}
+
+# _django_manage: wrap django's manage.py command with docker compose
+#
+# usage : _django_manage [ARGS...]
+#
+# ARGS : django's manage.py command arguments
+function _django_manage() {
+    _dc_run app python manage.py "$@"
+}

bin/compose

@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+
+# shellcheck source=bin/_config.sh
+source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
+
+_docker_compose "$@"

bin/install-hooks.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+
+mkdir -p "$(dirname -- "${BASH_SOURCE[0]}")/../.git/hooks/"
+PRE_COMMIT_FILE="$(dirname -- "${BASH_SOURCE[0]}")/../.git/hooks/pre-commit"
+
+cat <<'EOF' >$PRE_COMMIT_FILE
+#!/bin/bash
+
+# directories containing potential secrets
+DIRS="."
+
+bold=$(tput bold)
+normal=$(tput sgr0)
+
+# allow to read user input, assigns stdin to keyboard
+exec </dev/tty
+
+for d in $DIRS; do
+	# find files containing secrets that should be encrypted
+	for f in $(find "${d}" -type f -regex ".*\.enc\..*"); do
+		if ! $(grep -q "unencrypted_suffix" $f); then
+			printf '\xF0\x9F\x92\xA5 '
+			echo "File $f has non encrypted secrets!"
+			exit 1
+		fi
+	done
+done
+EOF
+
+chmod +x $PRE_COMMIT_FILE

bin/manage

@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+
+# shellcheck source=bin/_config.sh
+source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
+
+_django_manage "$@"

bin/pause-kind.sh

@@ -0,0 +1,6 @@
+#!/bin/sh
+set -o errexit
+
+echo "Pausing docker containers running k8s cluster managed with kind"
+
+docker pause $(docker ps --filter name=kind-* --filter status=running --format "{{.Names}}")

bin/pylint

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# shellcheck source=bin/_config.sh
+source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
+
+declare diff_from
+declare -a paths
+declare -a args
+
+# Parse options
+for arg in "$@"
+do
+    case $arg in
+        --diff-only=*)
+            diff_from="${arg#*=}"
+            shift
+            ;;
+        -*)
+            args+=("$arg")
+            shift
+            ;;
+        *)
+            paths+=("$arg")
+            shift
+            ;;
+    esac
+done
+
+if [[ -n "${diff_from}" ]]; then
+    # Run pylint only on modified files located in src/backend
+    # (excluding deleted files and migration files)
+    # shellcheck disable=SC2207
+    paths=($(git diff "${diff_from}" --name-only --diff-filter=d -- src/backend ':!**/migrations/*.py' | grep -E '^src/backend/.*\.py$'))
+fi
+
+# Fix docker vs local path when project sources are mounted as a volume
+read -ra paths <<< "$(echo "${paths[@]}" | sed "s|src/backend/||g")"
+_dc_run app pylint "${paths[@]}" "${args[@]}"

bin/pytest

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+
+source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
+
+_dc_run \
+    -e DJANGO_CONFIGURATION=Test \
+    app \
+    pytest "$@"

bin/resume-kind.sh

@@ -0,0 +1,6 @@
+#!/bin/sh
+set -o errexit
+
+echo "Resuming docker containers running k8s cluster managed with kind"
+
+docker unpause $(docker ps --filter name=kind-* --filter status=paused --format "{{.Names}}")

bin/start-kind.sh

@@ -0,0 +1,103 @@
+#!/bin/sh
+set -o errexit
+
+CURRENT_DIR=$(pwd)
+
+echo "0. Create ca"
+# 0. Create ca
+mkcert -install
+cd /tmp
+mkcert "127.0.0.1.nip.io" "*.127.0.0.1.nip.io"
+cd $CURRENT_DIR
+
+echo "1. Create registry container unless it already exists"
+# 1. Create registry container unless it already exists
+reg_name='kind-registry'
+reg_port='5001'
+if [ "$(docker inspect -f '{{.State.Running}}' "${reg_name}" 2>/dev/null || true)" != 'true' ]; then
+	docker run \
+		-d --restart=always -p "127.0.0.1:${reg_port}:5000" --network bridge --name "${reg_name}" \
+		registry:2
+fi
+
+echo "2. Create kind cluster with containerd registry config dir enabled"
+# 2. Create kind cluster with containerd registry config dir enabled
+# TODO: kind will eventually enable this by default and this patch will
+# be unnecessary.
+#
+# See:
+# https://github.com/kubernetes-sigs/kind/issues/2875
+# https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration
+# See: https://github.com/containerd/containerd/blob/main/docs/hosts.md
+cat <<EOF | kind create cluster --config=-
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+containerdConfigPatches:
+- |-
+  [plugins."io.containerd.grpc.v1.cri".registry]
+    config_path = "/etc/containerd/certs.d"
+nodes:
+- role: control-plane
+  image: kindest/node:v1.27.3
+  kubeadmConfigPatches:
+  - |
+    kind: InitConfiguration
+    nodeRegistration:
+      kubeletExtraArgs:
+        node-labels: "ingress-ready=true"
+  extraPortMappings:
+  - containerPort: 80
+    hostPort: 80
+    protocol: TCP
+  - containerPort: 443
+    hostPort: 443
+    protocol: TCP
+- role: worker
+  image: kindest/node:v1.27.3
+- role: worker
+  image: kindest/node:v1.27.3
+EOF
+
+echo "3. Add the registry config to the nodes"
+# 3. Add the registry config to the nodes
+#
+# This is necessary because localhost resolves to loopback addresses that are
+# network-namespace local.
+# In other words: localhost in the container is not localhost on the host.
+#
+# We want a consistent name that works from both ends, so we tell containerd to
+# alias localhost:${reg_port} to the registry container when pulling images
+REGISTRY_DIR="/etc/containerd/certs.d/localhost:${reg_port}"
+for node in $(kind get nodes); do
+	docker exec "${node}" mkdir -p "${REGISTRY_DIR}"
+	cat <<EOF | docker exec -i "${node}" cp /dev/stdin "${REGISTRY_DIR}/hosts.toml"
+[host."http://${reg_name}:5000"]
+EOF
+done
+
+echo "4. Connect the registry to the cluster network if not already connected"
+# 4. Connect the registry to the cluster network if not already connected
+# This allows kind to bootstrap the network but ensures they're on the same network
+if [ "$(docker inspect -f='{{json .NetworkSettings.Networks.kind}}' "${reg_name}")" = 'null' ]; then
+	docker network connect "kind" "${reg_name}"
+fi
+
+echo "5. Document the local registry"
+# 5. Document the local registry
+# https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry
+cat <<EOF | kubectl apply -f -
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: local-registry-hosting
+  namespace: kube-public
+data:
+  localRegistryHosting.v1: |
+    host: "localhost:${reg_port}"
+    help: "https://kind.sigs.k8s.io/docs/user/local-registry/"
+EOF
+
+echo "6. Install ingress-nginx"
+kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
+kubectl -n ingress-nginx create secret tls mkcert --key /tmp/127.0.0.1.nip.io+1-key.pem --cert /tmp/127.0.0.1.nip.io+1.pem
+kubectl -n ingress-nginx patch deployments.apps ingress-nginx-controller --type 'json' -p '[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value":"--default-ssl-certificate=ingress-nginx/mkcert"}]'

bin/state

@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+set -eo pipefail
+
+source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
+
+project=$(_set_openstack_project)
+echo "Using \"${project}\" project..."
+
+source "${TERRAFORM_DIRECTORY}/${project}/openrc.sh"
+
+# Run Terraform commands in the Hashicorp docker container via docker compose
+# shellcheck disable=SC2068
+DOCKER_USER="$(id -u):$(id -g)" \
+    PROJECT="${project}" \
+    docker compose run --rm \
+    -e OS_AUTH_URL \
+    -e OS_IDENTITY_API_VERSION \
+    -e OS_USER_DOMAIN_NAME \
+    -e OS_PROJECT_DOMAIN_NAME \
+    -e OS_TENANT_ID \
+    -e OS_TENANT_NAME \
+    -e OS_USERNAME \
+    -e OS_PASSWORD \
+    -e OS_REGION_NAME \
+    terraform-state "$@"

bin/update-git-submodule.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+
+git submodule update --init --recursive
+git submodule foreach 'git fetch origin; git checkout $(git rev-parse --abbrev-ref HEAD); git reset --hard origin/$(git rev-parse --abbrev-ref HEAD); git submodule update --recursive; git clean -dfx'

bin/update_openapi_schema

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+
+source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
+
+_dc_run \
+    -e DJANGO_CONFIGURATION=Test \
+    app \
+    python manage.py spectacular \
+    --api-version 'v1.0' \
+    --urlconf 'find.api_urls' \
+    --format openapi-json \
+    --file /app/core/tests/swagger/swagger.json

bin/updatekeys.sh

@@ -0,0 +1,3 @@
+#!/bin/bash
+
+find . -name "*.enc.*" -exec sops updatekeys -y {} \;

compose.yml

@@ -0,0 +1,113 @@
+name: find
+
+services:
+  postgresql:
+    image: postgres:15
+    env_file:
+      - env.d/development/postgresql
+    ports:
+      - "25432:5432"
+
+  opensearch:
+    image: opensearchproject/opensearch:latest
+    environment:
+      - discovery.type=single-node
+      - plugins.security.disabled=true
+      - plugins.security.ssl.http.enabled=false
+      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=find.PASS123
+    ulimits:
+      memlock:
+        soft: -1
+        hard: -1
+    volumes:
+      - ./data/opensearch:/usr/share/opensearch/data
+    ports:
+      - "9200:9200"
+      - "9600:9600"
+    healthcheck:
+      test: ["CMD", "curl", "--head", "-fsS", "http://localhost:9200/"]
+      start_period: 5s
+      interval: 2s
+      timeout: 2s
+      retries: 10
+
+  opensearch-dashboards:
+    image: opensearchproject/opensearch-dashboards:latest
+    ports:
+      - "5601:5601"
+    environment:
+      OPENSEARCH_HOSTS: '["http://opensearch:9200"]'
+      DISABLE_SECURITY_DASHBOARDS_PLUGIN: "true"
+    depends_on:
+      opensearch:
+        condition: service_healthy
+        restart: true
+
+  redis:
+    image: redis:5
+
+  app:
+    build:
+      context: .
+      target: backend-development
+      args:
+        DOCKER_USER: ${DOCKER_USER:-1000}
+    user: ${DOCKER_USER:-1000}
+    image: find:backend-development
+    environment:
+      - PYLINTHOME=/app/.pylint.d
+      - DJANGO_CONFIGURATION=Development
+    env_file:
+      - env.d/development/common
+      - env.d/development/postgresql
+    ports:
+      - "8081:8000"
+    networks:
+      default: {}
+      lasuite:
+        aliases:
+          - find
+    volumes:
+      - ./src/backend:/app
+      - ./data/static:/data/static
+      - /app/.venv
+    depends_on:
+      postgresql:
+        condition: service_started
+      redis:
+        condition: service_started
+      opensearch:
+        condition: service_healthy
+
+  celery:
+    user: ${DOCKER_USER:-1000}
+    image: find:backend-development
+    command: ["celery", "-A", "find.celery_app", "worker", "-l", "DEBUG"]
+    environment:
+      - DJANGO_CONFIGURATION=Development
+    env_file:
+      - env.d/development/common
+      - env.d/development/postgresql
+    volumes:
+      - ./src/backend:/app
+      - ./data/static:/data/static
+      - /app/.venv
+    depends_on:
+      - app
+
+  dockerize:
+    image: jwilder/dockerize
+
+  crowdin:
+    image: crowdin/cli:3.16.0
+    volumes:
+      - ".:/app"
+    env_file:
+      - env.d/development/crowdin
+    user: "${DOCKER_USER:-1000}"
+    working_dir: /app
+
+networks:
+  lasuite:
+    name: lasuite-network
+    driver: bridge

crowdin/config.yml

@@ -0,0 +1,23 @@
+#
+# Your crowdin's credentials
+#
+api_token_env: CROWDIN_API_TOKEN
+project_id_env: CROWDIN_PROJECT_ID
+base_path_env: CROWDIN_BASE_PATH
+
+#
+# Choose file structure in crowdin
+# e.g. true or false
+#
+preserve_hierarchy: true
+
+#
+# Files configuration
+#
+files: [
+ {
+  source : "/backend/locale/django.pot",
+  dest: "/backend-find.pot",
+  translation : "/backend/locale/%locale_with_underscore%/LC_MESSAGES/django.po"
+ },
+]

docker/files/usr/local/bin/entrypoint

@@ -0,0 +1,35 @@
+#!/bin/sh
+#
+# The container user (see USER in the Dockerfile) is an un-privileged user that
+# does not exists and is not created during the build phase (see Dockerfile).
+# Hence, we use this entrypoint to wrap commands that will be run in the
+# container to create an entry for this user in the /etc/passwd file.
+#
+# The following environment variables may be passed to the container to
+# customize running user account:
+#
+#   * USER_NAME: container user name (default: default)
+#   * HOME     : container user home directory (default: none)
+#
+# To pass environment variables, you can either use the -e option of the docker run command:
+#
+#     docker run --rm -e USER_NAME=foo -e HOME='/home/foo' find:latest python manage.py migrate
+#
+# or define new variables in an environment file to use with docker or docker compose:
+#
+#     # env.d/production
+#     USER_NAME=foo
+#     HOME=/home/foo
+#
+#     docker run --rm --env-file env.d/production find:latest python manage.py migrate
+#
+
+echo "🐳(entrypoint) creating user running in the container..."
+if ! whoami > /dev/null 2>&1; then
+  if [ -w /etc/passwd ]; then
+    echo "${USER_NAME:-default}:x:$(id -u):$(id -g):${USER_NAME:-default} user:${HOME}:/sbin/nologin" >> /etc/passwd
+  fi
+fi
+
+echo "🐳(entrypoint) running your command: ${*}"
+exec "$@"

docker/files/usr/local/etc/gunicorn/find.py

@@ -0,0 +1,16 @@
+# Gunicorn-django settings
+bind = ["0.0.0.0:8000"]
+name = "find"
+python_path = "/app"
+
+# Run
+graceful_timeout = 90
+timeout = 90
+workers = 3
+
+# Logging
+# Using '-' for the access log file makes gunicorn log accesses to stdout
+accesslog = "-"
+# Using '-' for the error log file makes gunicorn log errors to stderr
+errorlog = "-"
+loglevel = "info"

docs/architecture.md

@@ -0,0 +1,13 @@
+## Architecture
+
+### Global system architecture
+
+```mermaid
+flowchart TD
+    Docs -- REST API --> Back("Backend (Django)")
+    Back --> DB("Database (PostgreSQL)")
+    Back -- REST API --> Opensearch
+    Back <--> Celery --> DB
+    User -- HTTP --> Dashboard --> Opensearch
+    Back -- REST API --> Embedding Endpoint
+```

docs/env.md

@@ -0,0 +1,116 @@
+# Find variables
+
+Here we describe all environment variables that can be set for the find application.
+
+## find-backend container
+
+These are the environment variables you can set for the `find-backend` container.
+
+| Option                                          | Description                                                                                                                 | default                                                                 |
+|-------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------|
+| API_USERS_LIST_LIMIT                            | Limit on API users                                                                                                          | 5                                                                       |
+| API_USERS_LIST_THROTTLE_RATE_BURST              | Throttle rate for api on burst                                                                                              | 30/minute                                                               |
+| API_USERS_LIST_THROTTLE_RATE_SUSTAINED          | Throttle rate for api                                                                                                       | 180/hour                                                                |
+| CACHES_DEFAULT_TIMEOUT                          | Cache default timeout                                                                                                       | 30                                                                      |
+| CACHES_KEY_PREFIX                               | The prefix used to every cache keys.                                                                                        | docs                                                                    |
+| CHUNK_SIZE                                      | approximate number of characters of document content chunks                                                                 | 512                                                                     |
+| CHUNK_OVERLAP                                   | approximate number of characters of document content overlapping                                                            | 50                                                                      |
+| DB_ENGINE                                       | Engine to use for database connections                                                                                      | django.db.backends.postgresql_psycopg2                                  |
+| DB_HOST                                         | Host of the database                                                                                                        | localhost                                                               |
+| DB_NAME                                         | Name of the database                                                                                                        | impress                                                                 |
+| DB_PASSWORD                                     | Password to authenticate with                                                                                               | pass                                                                    |
+| DB_PORT                                         | Port of the database                                                                                                        | 5432                                                                    |
+| DB_USER                                         | User to authenticate with                                                                                                   | dinum                                                                   |
+| DJANGO_ALLOWED_HOSTS                            | Allowed hosts                                                                                                               | []                                                                      |
+| DJANGO_CELERY_BROKER_TRANSPORT_OPTIONS          | Celery broker transport options                                                                                             | {}                                                                      |
+| DJANGO_CELERY_BROKER_URL                        | Celery broker url                                                                                                           | redis://redis:6379/0                                                    |
+| DJANGO_CORS_ALLOW_ALL_ORIGINS                   | Allow all CORS origins                                                                                                      | false                                                                   |
+| DJANGO_CORS_ALLOWED_ORIGIN_REGEXES              | List of origins allowed for CORS using regulair expressions                                                                 | []                                                                      |
+| DJANGO_CORS_ALLOWED_ORIGINS                     | List of origins allowed for CORS                                                                                            | []                                                                      |
+| DJANGO_CSRF_TRUSTED_ORIGINS                     | CSRF trusted origins                                                                                                        | []                                                                      |
+| DJANGO_EMAIL_BACKEND                            | Email backend library                                                                                                       | django.core.mail.backends.smtp.EmailBackend                             |
+| DJANGO_EMAIL_BRAND_NAME                         | Brand name for email                                                                                                        |                                                                         |
+| DJANGO_EMAIL_FROM                               | Email address used as sender                                                                                                | from@example.com                                                        |
+| DJANGO_EMAIL_HOST                               | Hostname of email                                                                                                           |                                                                         |
+| DJANGO_EMAIL_HOST_PASSWORD                      | Password to authenticate with on the email host                                                                             |                                                                         |
+| DJANGO_EMAIL_HOST_USER                          | User to authenticate with on the email host                                                                                 |                                                                         |
+| DJANGO_EMAIL_LOGO_IMG                           | Logo for the email                                                                                                          |                                                                         |
+| DJANGO_EMAIL_PORT                               | Port used to connect to email host                                                                                          |                                                                         |
+| DJANGO_EMAIL_USE_SSL                            | Use ssl for email host connection                                                                                           | false                                                                   |
+| DJANGO_EMAIL_USE_TLS                            | Use tls for email host connection                                                                                           | false                                                                   |
+| DJANGO_SECRET_KEY                               | Secret key                                                                                                                  |                                                                         |
+| DJANGO_SERVER_TO_SERVER_API_TOKENS              |                                                                                                                             | []                                                                      |
+| DOCUMENT_IMAGE_MAX_SIZE                         | Maximum size of document in bytes                                                                                           | 10485760                                                                |
+| EMBEDDING_API_KEY                               | API key of the embedding api                                                                                                |                                                                         |
+| EMBEDDING_API_MODEL_NAME                        | Name of the embedding model used on the api                                                                                 | embeddings-small                                                        |
+| EMBEDDING_API_PATH                              | URL of the embedding api                                                                                                    |                                                                         |
+| EMBEDDING_DIMENSION                             | Size of the embedding vector                                                                                                | 1024                                                                    |
+| EMBEDDING_REQUEST_TIMEOUT                       | time out in seconds of the embedding requests                                                                               | 10                                                                      |
+| FRONTEND_CSS_URL                                | To add a external css file to the app                                                                                       |                                                                         |
+| FRONTEND_HOMEPAGE_FEATURE_ENABLED               | Frontend feature flag to display the homepage                                                                               | false                                                                   |
+| FRONTEND_THEME                                  | Frontend theme to use                                                                                                       |                                                                         |
+| HYBRID_SEARCH_ENABLED                           | Flag to enable hybrid (an then semantic) search                                                                             | True                                                                    |
+| HYBRID_SEARCH_WEIGHTS                           | Weights used in the weighted sum of the hybrid search score                                                                 | [0.3, 0.7]                                                              |
+| LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD         | Language detection confidence threshold                                                                                     | 0.75                                                                    |
+| LOGGING_LEVEL_LOGGERS_APP                       | Application logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL"                                         | INFO                                                                    |
+| LOGGING_LEVEL_LOGGERS_ROOT                      | Default logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL"                                             | INFO                                                                    |
+| LOGIN_REDIRECT_URL                              | Login redirect url                                                                                                          |                                                                         |
+| LOGIN_REDIRECT_URL_FAILURE                      | Login redirect url on failure                                                                                               |                                                                         |
+| LOGOUT_REDIRECT_URL                             | Logout redirect url                                                                                                         |                                                                         |
+| MALWARE_DETECTION_BACKEND                       | The malware detection backend use from the django-lasuite package                                                           | lasuite.malware_detection.backends.dummy.DummyBackend                   |
+| MALWARE_DETECTION_PARAMETERS                    | A dict containing all the parameters to initiate the malware detection backend                                              | {"callback_path": "core.malware_detection.malware_detection_callback",} |
+| MEDIA_BASE_URL                                  |                                                                                                                             |                                                                         |
+| NO_WEBSOCKET_CACHE_TIMEOUT                      | Cache used to store current editor session key when only users without websocket are editing a document                     | 120                                                                     |
+| OIDC_ALLOW_DUPLICATE_EMAILS                     | Allow duplicate emails                                                                                                      | false                                                                   |
+| OIDC_AUTH_REQUEST_EXTRA_PARAMS                  | OIDC extra auth parameters                                                                                                  | {}                                                                      |
+| OIDC_CREATE_USER                                | Create used on OIDC                                                                                                         | false                                                                   |
+| OIDC_DRF_AUTH_BACKEND                           | DRF Authentication backend class for OIDC                                                                                   | lasuite.oidc_login.backends.OIDCAuthenticationBackend                   |
+| OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION       | Fallback to email for identification                                                                                        | true                                                                    |
+| OIDC_OP_AUTHORIZATION_ENDPOINT                  | Authorization endpoint for OIDC                                                                                             |                                                                         |
+| OIDC_OP_INTROSPECTION_ENDPOINT                  | Introspection endpoint for OIDC                                                                                             |                                                                         |
+| OIDC_OP_JWKS_ENDPOINT                           | JWKS endpoint for OIDC                                                                                                      |                                                                         |
+| OIDC_OP_LOGOUT_ENDPOINT                         | Logout endpoint for OIDC                                                                                                    |                                                                         |
+| OIDC_OP_TOKEN_ENDPOINT                          | Token endpoint for OIDC                                                                                                     |                                                                         |
+| OIDC_OP_USER_ENDPOINT                           | User endpoint for OIDC                                                                                                      |                                                                         |
+| OIDC_REDIRECT_ALLOWED_HOSTS                     | Allowed hosts for OIDC redirect url                                                                                         | []                                                                      |
+| OIDC_REDIRECT_REQUIRE_HTTPS                     | Require https for OIDC redirect url                                                                                         | false                                                                   |
+| OIDC_RP_CLIENT_ID                               | Client id used for OIDC                                                                                                     | impress                                                                 |
+| OIDC_RP_CLIENT_SECRET                           | Client secret used for OIDC                                                                                                 |                                                                         |
+| OIDC_RP_SCOPES                                  | Scopes requested for OIDC                                                                                                   | openid email                                                            |
+| OIDC_RP_SIGN_ALGO                               | verification algorithm used OIDC tokens                                                                                     | RS256                                                                   |
+| OIDC_RS_BACKEND_CLASS                           | Resource server backend class for OIDC                                                                                      | core.authentication.FinderResourceServerBackend                         |
+| OIDC_RS_AUDIENCE_CLAIM                          | The claim used to identify the audience                                                                                     | client_id                                                               |
+| OIDC_RS_CLIENT_ID                               | Client id used for OIDC resource server                                                                                     |                                                                         |
+| OIDC_RS_CLIENT_SECRET                           | Client secret used for OIDC resource server                                                                                 |                                                                         |
+| OIDC_RS_SIGNING_ALGO                            | Signing algorithm                                                                                                           | ES256                                                                   |
+| OIDC_RS_SCOPES                                  | Required scopes for authentication                                                                                          | ["openid"]                                                              |
+| OIDC_RS_PRIVATE_KEY_STR                         | Private key for encryption/decryption                                                                                       |                                                                         |
+| OIDC_RS_ENCRYPTION_KEY_TYPE                     | Encryption key type (RSA, EC, etc.)                                                                                         | RSA                                                                     |
+| OIDC_RS_ENCRYPTION_ALGO                         | Encryption algorithm                                                                                                        | RSA-OAEP                                                                |
+| OIDC_RS_ENCRYPTION_ENCODING                     | Encryption encoding algorithm                                                                                               | A256GCM                                                                 |
+| OIDC_STORE_ID_TOKEN                             | Store OIDC token                                                                                                            | true                                                                    |
+| OIDC_USE_NONCE                                  | Use nonce for OIDC                                                                                                          | true                                                                    |
+| OIDC_USERINFO_FULLNAME_FIELDS                   | OIDC token claims to create full name                                                                                       | ["first_name", "last_name"]                                             |
+| OIDC_USERINFO_SHORTNAME_FIELD                   | OIDC token claims to create shortname                                                                                       | first_name                                                              |
+| OIDC_VERIFY_SSL                                 | Enable SSL validation for OIDC                                                                                              | true                                                                    |
+| OIDC_TIMEOUT                                    | Timeout delay for OID requests                                                                                              |                                                                         |
+| OIDC_PROXY                                      | Defines a proxy for all requests to the OpenID Connect provider                                                             |                                                                         |
+| OPENSEARCH_HOST                                 | Opensearch database url                                                                                                     | default                                                                 |
+| OPENSEARCH_PORT                                 | Port of Opensearch database                                                                                                 | 9200                                                                    |
+| OPENSEARCH_USER                                 | Opensearch database user                                                                                                    | admin                                                                   |
+| OPENSEARCH_PASSWORD                             | Opensearch database user password                                                                                           |                                                                         |
+| OPENSEARCH_USE_SSL                              | Enable SSL connection for Opensearch database                                                                               | true                                                                    |
+| POSTHOG_KEY                                     | Posthog key for analytics                                                                                                   |                                                                         |
+| REDIS_URL                                       | Cache url                                                                                                                   | redis://redis:6379/1                                                    |
+| SENTRY_DSN                                      | Sentry host                                                                                                                 |                                                                         |
+| SESSION_COOKIE_AGE                              | duration of the cookie session                                                                                              | 60*60*12                                                                |
+| SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK |                                                                                                                             | false                                                                   |
+| STORAGES_STATICFILES_BACKEND                    |                                                                                                                             | whitenoise.storage.CompressedManifestStaticFilesStorage                 |
+| THEME_CUSTOMIZATION_CACHE_TIMEOUT               | Cache duration for the customization settings                                                                               | 86400                                                                   |
+| THEME_CUSTOMIZATION_FILE_PATH                   | Full path to the file customizing the theme. An example is provided in src/backend/impress/configuration/theme/default.json | BASE_DIR/impress/configuration/theme/default.json                       |
+| TRASHBIN_CUTOFF_DAYS                            | Trashbin cutoff                                                                                                             | 30                                                                      |
+| TRIGRAMS_BOOST                                  | weight boost applied to trigram score in document matching score                                                            | 0.25                                                                    |
+| TRIGRAMS_MINIMUM_SHOULD_MATCH                   | minimal number or proportion of trigrams having to match to score                                                           | 0.75%                                                                   |
+| USER_OIDC_ESSENTIAL_CLAIMS                      | Essential claims in OIDC token                                                                                              | []                                                                      |
+| Y_PROVIDER_API_BASE_URL                         | Y Provider url                                                                                                              |                                                                         |
+| Y_PROVIDER_API_KEY                              | Y provider API key                                                                                                          |                                                                         |

docs/evaluate-search-engine.md

@@ -0,0 +1,32 @@
+# Search Engine Evaluation Command
+
+## Overview 
+
+this Django command atomizes the evaluation of the search engine by computing 4 metrics: Average Discounted Cumulative Gain, Precision, Recall and F1 score. 
+
+## Usage 
+
+```
+python manage.py evaluate_search_engine <dataset_name> [options]
+```
+
+## Required Arguments
+- `dataset_name`: Name of the evaluation dataset to use. Datasets are located in `evaluation/management/commands/data/evaluation/`
+
+## Optional Arguments
+- `--min_score`: Minimum score threshold; hits below this score are ignored
+- `--keep-index`: Preserve the evaluation index after completion
+- `--force-reindex`: Drop and recreate the index even if it exists
+
+## Examples
+
+````
+# Basic evaluation with default settings
+python manage.py evaluate_search_engine my_dataset
+
+# Evaluation with minimum score threshold
+python manage.py evaluate_search_engine my_dataset --min_score 0.5
+
+# Force reindexing and clean up afterward
+python manage.py evaluate_search_engine my_dataset --force-reindex True --keep-index False
+````

docs/evaluation.md

@@ -0,0 +1,170 @@
+This file keeps for reference the logs of the last best evaluation of the model.
+
+````
+(venv) ➜  find git:(evaluate) ✗ docker compose exec app python manage.py evaluate_search_engine v1 --min_score 0.5   
+
+2025-11-20 18:44:30,903 core.services.opensearch INFO Hybrid search is disabled via HYBRID_SEARCH_ENABLED setting
+2025-11-20 18:44:31,183 core.management.commands.utils INFO Deleting search pipeline hybrid-search-pipeline
+2025-11-20 18:44:31,201 opensearch INFO DELETE http://opensearch:9200/_search/pipeline/hybrid-search-pipeline [status:200 request:0.018s]
+2025-11-20 18:44:31,202 opensearch WARNING GET http://opensearch:9200/_search/pipeline/hybrid-search-pipeline [status:404 request:0.001s]
+2025-11-20 18:44:31,202 core.management.commands.create_search_pipeline INFO Creating search pipeline: hybrid-search-pipeline
+2025-11-20 18:44:31,221 opensearch INFO PUT http://opensearch:9200/_search/pipeline/hybrid-search-pipeline [status:200 request:0.019s]
+2025-11-20 18:44:31,222 opensearch INFO HEAD http://opensearch:9200/evaluation-index [status:200 request:0.001s]
+[INFO] Starting evaluation with 76 documents and 12 queries
+2025-11-20 18:44:31,222 core.services.opensearch INFO embed: 'cours d'histoire de l'antiquité'
+2025-11-20 18:44:31,302 core.services.opensearch INFO Performing hybrid search with embedding: cours d'histoire de l'antiquité
+2025-11-20 18:44:31,317 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.014s]
+
+[QUERY EVALUATION]
+  q: cours d'histoire de l'antiquité
+  expect: ["L'Empire Romain", "L'Égypte Ancienne"]
+  result: ["L'Égypte Ancienne", 'La Sculpture sur Pierre']
+  NDCG: 61.31% 
+  PRECISION: 50.00% 
+  RECALL: 50.00% 
+  F1-SCORE: 50.00% 
+2025-11-20 18:44:31,317 core.services.opensearch INFO embed: 'recette salée végétarienne'
+2025-11-20 18:44:31,392 core.services.opensearch INFO Performing hybrid search with embedding: recette salée végétarienne
+2025-11-20 18:44:31,403 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.010s]
+
+[QUERY EVALUATION]
+  q: recette salée végétarienne
+  expect: ['Ratatouille Provençale', 'Salade de légumes', 'Fondue Savoyarde']
+  result: ['Salade de légumes']
+  NDCG: 46.93% 
+  PRECISION: 100.00% 
+  RECALL: 33.33% 
+  F1-SCORE: 50.00% 
+2025-11-20 18:44:31,403 core.services.opensearch INFO embed: 'art dramatique'
+2025-11-20 18:44:31,475 core.services.opensearch INFO Performing hybrid search with embedding: art dramatique
+2025-11-20 18:44:31,486 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.011s]
+
+[QUERY EVALUATION]
+  q: art dramatique
+  expect: ['Le Théâtre']
+  result: ['Le Théâtre', 'Le Vitrail']
+  NDCG: 100.00% 
+  PRECISION: 50.00% 
+  RECALL: 100.00% 
+  F1-SCORE: 66.67% 
+2025-11-20 18:44:31,487 core.services.opensearch INFO embed: 'art de bouger son corps'
+2025-11-20 18:44:31,556 core.services.opensearch INFO Performing hybrid search with embedding: art de bouger son corps
+2025-11-20 18:44:31,573 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.017s]
+
+[QUERY EVALUATION]
+  q: art de bouger son corps
+  expect: ['La Danse', 'La Danse Contemporaine']
+  result: ['La Danse', 'La Danse Contemporaine']
+  NDCG: 100.00% 
+  PRECISION: 100.00% 
+  RECALL: 100.00% 
+  F1-SCORE: 100.00% 
+2025-11-20 18:44:31,573 core.services.opensearch INFO embed: 'mammifères aquatiques'
+2025-11-20 18:44:31,641 core.services.opensearch INFO Performing hybrid search with embedding: mammifères aquatiques
+2025-11-20 18:44:31,654 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.013s]
+
+[QUERY EVALUATION]
+  q: mammifères aquatiques
+  expect: ['Le Dauphin', 'La Baleine à Bosse']
+  result: ['Le Dauphin', 'La Baleine à Bosse', 'Le Manchot Empereur', 'Le Requin Blanc', 'Le Paresseux']
+  NDCG: 100.00% 
+  PRECISION: 40.00% 
+  RECALL: 100.00% 
+  F1-SCORE: 57.14% 
+2025-11-20 18:44:31,654 core.services.opensearch INFO embed: 'insectes pollinisateurs'
+2025-11-20 18:44:31,733 core.services.opensearch INFO Performing hybrid search with embedding: insectes pollinisateurs
+2025-11-20 18:44:31,746 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.012s]
+
+[QUERY EVALUATION]
+  q: insectes pollinisateurs
+  expect: ["L'Abeille"]
+  result: ["L'Abeille", 'Le Caméléon', 'Le Papillon Monarque']
+  NDCG: 100.00% 
+  PRECISION: 33.33% 
+  RECALL: 100.00% 
+  F1-SCORE: 50.00% 
+2025-11-20 18:44:31,746 core.services.opensearch INFO embed: 'prédateur félin'
+2025-11-20 18:44:31,820 core.services.opensearch INFO Performing hybrid search with embedding: prédateur félin
+2025-11-20 18:44:31,834 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.014s]
+
+[QUERY EVALUATION]
+  q: prédateur félin
+  expect: ["Le Lion d'Afrique", 'Le Guépard']
+  result: ["Le Lion d'Afrique", 'Le Guépard', 'Le Requin Blanc', "L'éléphant", 'Le Hibou Grand-Duc', "L'Ours polaire", 'Le Serpent Python']
+  NDCG: 100.00% 
+  PRECISION: 28.57% 
+  RECALL: 100.00% 
+  F1-SCORE: 44.44% 
+2025-11-20 18:44:31,835 core.services.opensearch INFO embed: 'elephant'
+2025-11-20 18:44:31,906 core.services.opensearch INFO Performing hybrid search with embedding: elephant
+2025-11-20 18:44:31,920 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.013s]
+
+[QUERY EVALUATION]
+  q: elephant
+  expect: ["L'Éléphant d'Asie", "L'éléphant"]
+  result: ["L'éléphant", "L'Éléphant d'Asie"]
+  NDCG: 100.00% 
+  PRECISION: 100.00% 
+  RECALL: 100.00% 
+  F1-SCORE: 100.00% 
+2025-11-20 18:44:31,920 core.services.opensearch INFO embed: 'courir'
+2025-11-20 18:44:31,994 core.services.opensearch INFO Performing hybrid search with embedding: courir
+2025-11-20 18:44:32,010 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.015s]
+
+[QUERY EVALUATION]
+  q: courir
+  expect: ['Il va courir']
+  result: ['Il va courir']
+  NDCG: 100.00% 
+  PRECISION: 100.00% 
+  RECALL: 100.00% 
+  F1-SCORE: 100.00% 
+2025-11-20 18:44:32,011 core.services.opensearch INFO embed: 'football'
+2025-11-20 18:44:32,082 core.services.opensearch INFO Performing hybrid search with embedding: football
+2025-11-20 18:44:32,089 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.007s]
+
+[QUERY EVALUATION]
+  q: football
+  expect: ['Foot']
+  result: ['Foot']
+  NDCG: 100.00% 
+  PRECISION: 100.00% 
+  RECALL: 100.00% 
+  F1-SCORE: 100.00% 
+2025-11-20 18:44:32,089 core.services.opensearch INFO embed: 'couri'
+2025-11-20 18:44:32,156 core.services.opensearch INFO Performing hybrid search with embedding: couri
+2025-11-20 18:44:32,163 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.007s]
+
+[QUERY EVALUATION]
+  q: couri
+  expect: ['Il va courir']
+  result: ['Coq au Vin', 'Il va courir', 'Clafoutis aux Cerises']
+  NDCG: 63.09% 
+  PRECISION: 33.33% 
+  RECALL: 100.00% 
+  F1-SCORE: 50.00% 
+2025-11-20 18:44:32,164 core.services.opensearch INFO embed: 'courrir'
+2025-11-20 18:44:32,231 core.services.opensearch INFO Performing hybrid search with embedding: courrir
+2025-11-20 18:44:32,240 opensearch INFO POST http://opensearch:9200/evaluation-index/_search?search_pipeline=hybrid-search-pipeline&ignore_unavailable=true [status:200 request:0.009s]
+
+[QUERY EVALUATION]
+  q: courrir
+  expect: ['Il va courir']
+  result: ['Il va courir']
+  NDCG: 100.00% 
+  PRECISION: 100.00% 
+  RECALL: 100.00% 
+  F1-SCORE: 100.00% 
+
+============================================================
+[SUMMARY] Average Performance
+============================================================
+  Average NDCG: 89.28%
+  Average Precision: 69.60%
+  Average Recall: 90.28%
+  Average F1-Score: 72.35%
+2025-11-20 18:44:32,241 core.management.commands.utils INFO Deleting search pipeline hybrid-search-pipeline
+2025-11-20 18:44:32,258 opensearch INFO DELETE http://opensearch:9200/_search/pipeline/hybrid-search-pipeline [status:200 request:0.017s]
+
+[SUCCESS] Evaluation completed
+````

docs/release.md

@@ -0,0 +1,58 @@
+# Releasing a new version
+
+Whenever we are cooking a new release (e.g. `0.0.1`) we should follow a standard procedure described below:
+
+1.  Create a new branch named: `release/0.0.1`.
+
+2.  Bump the release number for backend project and Helm files:
+
+    - for backend, update the version number by hand in `pyproject.toml`,
+    - for Helm, update Docker image tag in files located at `src/helm/env.d` for both `preprod` and `production` environments:
+
+    ```yaml
+    image:
+    repository: lasuite/find
+    pullPolicy: Always
+    tag: "v0.0.1" # Replace with your new version number, without forgetting the "v" prefix
+    ...
+
+    The new images don't exist _yet_: they will be created automatically later in the process.
+
+3.  Update the project's `Changelog` following the [keepachangelog](https://keepachangelog.com/en/0.3.0/) recommendations
+
+4.  Commit your changes with the following format: the 🔖 release emoji, the type of release (patch/minor/patch) and the release version:
+
+    ```text
+    🔖(minor) bump release to 0.0.1
+    ```
+
+5.  Open a pull request, wait for an approval from your peers and merge it.
+6.  Checkout and pull changes from the `main` branch to ensure you have the latest updates.
+7.  Tag and push your commit:
+
+    ```bash
+    git tag v0.0.1 && git push origin tag v0.0.1
+    ```
+
+    Doing this triggers the CI and tells it to build the new Docker image versions that you targeted earlier in the Helm files.
+
+8.  Ensure the new [backend](https://hub.docker.com/r/lasuite/find/tags) image tags are on Docker Hub.
+9.  The release is now done!
+
+# Deploying
+
+> [!TIP]
+> The `staging` platform is deployed automatically with every update of the `main` branch.
+
+Making a new release doesn't publish it automatically in production.
+
+Deployment is done by ArgoCD. ArgoCD checks for the `production` tag and automatically deploys the production platform with the targeted commit.
+
+To publish, we mark the commit we want with the `production` tag. ArgoCD is then notified that the tag has changed. It then deploys the Docker image tags specified in the Helm files of the targeted commit.
+
+To publish the release you just made:
+
+```bash
+git tag --force production v0.0.1
+git push --force origin production
+```

docs/setup-for-docs.md

@@ -0,0 +1,39 @@
+# Setup the Find search for Impress
+
+This configuration will enable the fulltext search feature for Docs :
+- Each save on **core.Document** or **core.DocumentAccess** will trigger the indexer
+- The `api/v1.0/documents/search/` will work as a proxy with the Find API for fulltext search.
+
+## Create an index service for Docs
+
+Configure a **Service** for Docs application with these settings
+
+- **Name**: `docs`<br>_request.auth.name of the Docs application._
+- **Client id**: `impress`<br>_Name of the token audience or client_id of the Docs application._
+
+See [how-to-use-indexer.md](how-to-use-indexer.md) for details.
+
+## Configure settings of Docs
+
+Add those Django settings the Docs application to enable the feature.
+
+```shell
+SEARCH_INDEXER_CLASS="core.services.search_indexers.FindDocumentIndexer"
+SEARCH_INDEXER_COUNTDOWN=10  # Debounce delay in seconds for the indexer calls.
+
+# The token from service "docs" of Find application (development).
+SEARCH_INDEXER_SECRET="find-api-key-for-docs-with-exactly-50-chars-length"
+SEARCH_INDEXER_URL="http://find:8000/api/v1.0/documents/index/"
+
+# Search endpoint. Uses the OIDC token for authentication
+SEARCH_INDEXER_QUERY_URL="http://find:8000/api/v1.0/documents/search/"
+```
+
+We also need to enable the **OIDC Token** refresh or the authentication will fail quickly.
+
+```shell
+# Store OIDC tokens in the session
+OIDC_STORE_ACCESS_TOKEN = True  # Store the access token in the session
+OIDC_STORE_REFRESH_TOKEN = True  # Store the encrypted refresh token in the session
+OIDC_STORE_REFRESH_TOKEN_KEY = "your-32-byte-encryption-key=="  # Must be a valid Fernet key (32 url-safe base64-encoded bytes)
+```

docs/setup-for-drive.md

@@ -0,0 +1,43 @@
+# Setup the Find search for Drive
+
+This configuration will enable the fulltext search feature for Docs :
+- Each save on **core.Item** or **core.Item** will trigger the indexer
+- Once indexer service configured, the `api/v1.0/item/search/` will work as a proxy with the Find API for fulltext search.
+
+## Create an index service for Drive
+
+Configure a **Service** for Docs application with these settings
+
+- **Name**: `drive`<br>_request.auth.name of the Docs application._
+- **Client id**: `drive`<br>_Name of the token audience or client_id of the Docs application._
+
+See [how-to-use-indexer.md](how-to-use-indexer.md) for details.
+
+## Configure settings of Drive
+
+Add those Django settings the Docs application to enable the feature.
+
+```python
+SEARCH_INDEXER_CLASS="core.services.search_indexers.SearchIndexer"
+SEARCH_INDEXER_COUNTDOWN=10  # Debounce delay in seconds for the indexer calls.
+
+# The token from service "drive" of Find application (development)
+SEARCH_INDEXER_SECRET=find-api-key-for-driv-with-exactly-50-chars-length
+SEARCH_INDEXER_URL="http://find:8000/api/v1.0/documents/index/"
+
+# Search endpoint. Uses the OIDC token for authentication
+SEARCH_INDEXER_QUERY_URL="http://find:8000/api/v1.0/documents/search/"
+
+# Limit the mimetypes and size of indexable files
+SEARCH_INDEXER_ALLOWED_MIMETYPES=["text/"]
+SEARCH_INDEXER_UPLOAD_MAX_SIZE=2 * 2**20  # 2Mb
+```
+
+We also need to enable the **OIDC Token** refresh or the authentication will fail quickly.
+
+```shell
+# Store OIDC tokens in the session
+OIDC_STORE_ACCESS_TOKEN = True  # Store the access token in the session
+OIDC_STORE_REFRESH_TOKEN = True  # Store the encrypted refresh token in the session
+OIDC_STORE_REFRESH_TOKEN_KEY = "your-32-byte-encryption-key=="  # Must be a valid Fernet key (32 url-safe base64-encoded bytes)
+```

docs/setup-indexer.md

@@ -0,0 +1,161 @@
+# Using the Find indexer
+
+This guide explains how to setup the Find service which provide an API for indexation and fulltext search of
+documents from various sources in a secure way : only the documents within the scope of the user's OIDC token
+are visible.
+
+## Setup Opensearch
+
+### General
+
+Add the following variables to your Django settings to configure Find and enable full-text search.
+
+```python
+# Login for opensearch
+OPENSEARCH_USER=opensearch-user
+OPENSEARCH_PASSWORD=your-opensearch-password
+
+# Host configuration
+OPENSEARCH_HOST=opensearch
+OPENSEARCH_PORT=9200
+
+# Enable SSL for opensearch connection (False in dev mode)
+OPENSEARCH_USE_SSL=True
+
+# Prefix for the index name of the registered services.
+OPENSEARCH_INDEX_PREFIX=find
+```
+
+### Language
+
+Language specific operations are applied to document titles and contents to improve search results. 
+The language is automatically detected  by Find.
+If the language can not be detected no language specific operation are applied and the indexing process is not affected.
+
+Find supports french, english, german and dutch. 
+
+The search process is not language specific, a query can get documents of any language.
+
+Language detection estimates a confidence between 0 and 1. If the confidence is below a threshold the language is considered unrecognized. 
+This threshold can be controlled with LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD environment variable.
+
+```python
+LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD=0.75
+```
+
+### Semantic search
+
+Find offers a semantic search feature. You can either use pure full-text search or a hybrid full-text + semantic search. To enable the hybrid search, add the following settings. 
+
+```python
+# Enable flag
+HYBRID_SEARCH_ENABLED = True
+
+# weighted sum: full_text_weight, semantic_search_weight
+HYBRID_SEARCH_WEIGHTS = 0.7,0.3
+
+# Embedding
+CHUNK_SIZE=512
+CHUNK_OVERLAP=50
+EMBEDDING_API_PATH = https://embedding.api.example.com/full/path/
+EMBEDDING_API_KEY = your-embedding-api-key
+EMBEDDING_REQUEST_TIMEOUT = 10
+EMBEDDING_API_MODEL_NAME = embedding-api-model-name
+EMBEDDING_DIMENSION = 1024
+```
+
+The hybrid search computes a score for full-text and semantic search and combines them through a weighted sum. HYBRID_SEARCH_WEIGHTS contains the weights of full-text and semantic respectively. 
+
+You need to use an embedding api similar to https://albert.api.etalab.gouv.fr/documentation#tag/Embeddings/operation/embeddings_v1_embeddings_post. 
+
+### document chunking
+
+The indexing process embeds documents by converting their content into vector representations (embeddings). When a document exceeds the character dimension defined by CHUNK_SIZE, it's divided into smaller segments (chunks), with each chunk embedded independently. Each chunk must be smaller than the embedding model's context window . 
+
+The chunking algorithm works recursively. It attempts to create the largest possible segments first, then subdivides them further if they still exceed the size limit defined by CHUNK_SIZE. Segments can share overlapping content between them (set CHUNK_OVERLAP=0 to disable overlapping). 
+
+During the search, the matching of a document is the matching of its best chunk.
+
+## trigrams
+
+Find uses trigrams to improve the robustness of the full text search engine to spelling variations and errors. It can be configured by two environment variables. 
+
+````
+TRIGRAMS_BOOST=0.25
+TRIGRAMS_MINIMUM_SHOULD_MATCH=0.75%
+````
+
+`TRIGRAMS_BOOST` is weight boost applied to the trigram score in the document matching. 
+`TRIGRAMS_MINIMUM_SHOULD_MATCH` is the minimal number or proportion of trigrams having to match to score. It is
+either an absolute number or proportion.
+
+## Setup indexation API
+
+Other applications can index their files through the **`/index/`** endpoint with a simple token authentication.
+
+For each application a new **Service** must be created through the admin interface
+(see http://localhost:9071/admin/core/service/add/)
+
+| Field                       | Description                                        |
+|-----------------------------|----------------------------------------------------|
+| Name                        | Name of the service and also the name of the index in Opensearch database |
+| Is active                   | Toggle service availability                        |
+| Client id                   | Calling service client_id (e.g `impress` for docs) |
+| Allowed services for search | List of sub-services. Will add the results from all these index<br>to the search results. |
+| Token (_read-only_)         | Random token for calling service authentication    |
+
+And add the key in the calling application Django settings.
+
+**Development Mode (Docs + Find)**
+
+The command `make demo` will create a working service configuration for `docs` and `drive` with predefined secret keys
+
+```python
+# Docs
+SEARCH_INDEXER_SECRET="find-api-key-for-docs-with-exactly-50-chars-length"
+```
+
+```python
+# Drive
+SEARCH_INDEXER_SECRET="find-api-key-for-driv-with-exactly-50-chars-length"
+```
+
+## Setup search API
+
+The **`/search/`** endpoint is an OIDC ResourceServer view and needs extra Django settings (see [lasuite](https://github.com/suitenumerique/django-lasuite/blob/main/documentation/how-to-use-oidc-resource-server-backend.md) for details)
+
+```shell
+OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/certs
+OIDC_OP_AUTHORIZATION_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/auth
+OIDC_OP_TOKEN_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/token
+OIDC_OP_USER_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/userinfo
+
+# To run Find in development mode along other projects like docs/impress
+# we should to use OIDC endpoints on a common keycloak realm. e.g :
+# OIDC_OP_URL = http://nginx:8083/realms/impress
+#
+# This will cause a conflict with the 'iss' claim validation rule because the docs realm
+# gives {'iss': 'http://localhost:8083/realms/impress'} so it must be the same
+OIDC_OP_URL=http://localhost:8083/realms/impress
+
+# Introspection endpoint is needed to get the "audience" and "sub" from the user token
+OIDC_OP_INTROSPECTION_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/token/introspect
+
+# In development, the resource server use insecure settings
+# OIDC_VERIFY_SSL=False
+
+# Resource server
+OIDC_RS_SCOPES="openid"
+OIDC_RS_SIGN_ALGO=RS256
+
+# This backend allows authentication without any model in database. 
+OIDC_RS_BACKEND_CLASS="core.authentication.FinderResourceServerBackend"
+```
+
+**Development mode (Docs + Find)**
+
+Docs and Find projects stacks can be run together but must have the same keycloak server.
+
+So the endpoints of Docs on the 'nginx' domain for ResourceServer authentication and introspection are also used for Find.
+
+**IMPORTANT:** Keep OIDC_OP_URL on 'localhost' or it will break the OIDC token claims validation : the `iss` claim from the token of the 'impress' users are 'localhost' and not 'nginx'.

docs/system-requirements.md

@@ -0,0 +1,12 @@
+# La Suite Find – System & Requirements
+
+## Ports (dev defaults)
+
+| Port      | Service               |
+| --------- | --------------------- |
+| 8081      | Django (main)         |
+| 8083      | Nginx proxy           |
+| 25432     | PostgreSQL (main)     |
+| 9200      | Opensearch            |
+| 9600      | Opensearch admin      |
+| 5601      | Opensearch dashboard  |

env.d/development/common.dist

@@ -0,0 +1,61 @@
+# Django
+DJANGO_ALLOWED_HOSTS=*
+DJANGO_SECRET_KEY=ThisIsAnExampleKeyForDevPurposeOnly
+DJANGO_SETTINGS_MODULE=find.settings
+DJANGO_SUPERUSER_PASSWORD=admin
+
+# Python
+PYTHONPATH=/app
+
+# find settings
+
+# Backend url
+FIND_BASE_URL="http://localhost:8072"
+
+# Opensearch
+OPENSEARCH_PASSWORD=find.PASS123
+OPENSEARCH_USE_SSL=false
+OPENSEARCH_INDEX_PREFIX=find
+
+# OIDC
+OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/certs
+OIDC_OP_AUTHORIZATION_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/auth
+OIDC_OP_TOKEN_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/token
+OIDC_OP_USER_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/userinfo
+OIDC_OP_INTROSPECTION_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/token/introspect
+
+# To run Find in development mode along other projects like docs/impress
+# we should to use OIDC endpoints on a common keycloak realm. e.g :
+# OIDC_OP_URL = http://nginx:8083/realms/impress
+#
+# This will cause a conflict with the 'iss' claim validation rule because the docs realm
+# gives {'iss': 'http://localhost:8083/realms/impress'} so it must be the same
+OIDC_OP_URL=http://localhost:8083/realms/impress
+
+OIDC_RP_CLIENT_ID=find
+OIDC_RP_CLIENT_SECRET=ThisIsAnExampleKeyForDevPurposeOnly
+OIDC_RP_SIGN_ALGO=RS256
+OIDC_RP_SCOPES="openid email"
+
+OIDC_REDIRECT_ALLOWED_HOSTS=["http://localhost:8083", "http://localhost:3000"]
+OIDC_AUTH_REQUEST_EXTRA_PARAMS={"acr_values": "eidas1"}
+
+# OIDC Resource server
+OIDC_DRF_AUTH_BACKEND="lasuite.oidc_login.backends.OIDCAuthenticationBackend"
+
+OIDC_RS_SCOPES="openid"
+OIDC_RS_CLIENT_ID=impress
+OIDC_RS_CLIENT_SECRET=ThisIsAnExampleKeyForDevPurposeOnly
+OIDC_RS_SIGN_ALGO=RS256
+
+OIDC_RS_BACKEND_CLASS="core.authentication.FinderResourceServerBackend"
+OIDC_RS_ENCRYPTION_KEY_TYPE="RSA"
+
+# Hybrid Search settings
+HYBRID_SEARCH_ENABLED=True
+EMBEDDING_API_KEY=ThisIsAnExampleKeyForDevPurposeOnly
+EMBEDDING_API_PATH=https://albert.api.etalab.gouv.fr/v1/embeddings
+
+## Multi-embedding: chunk documents and embed each chunk
+CHUNK_SIZE=512
+CHUNK_OVERLAP=50

env.d/development/crowdin.dist

@@ -0,0 +1,3 @@
+CROWDIN_API_TOKEN=Your-Api-Token
+CROWDIN_PROJECT_ID=Your-Project-Id
+CROWDIN_BASE_PATH=/app/src

env.d/development/postgresql.dist

@@ -0,0 +1,11 @@
+# Postgresql db container configuration
+POSTGRES_DB=find
+POSTGRES_USER=dinum
+POSTGRES_PASSWORD=pass
+
+# App database configuration
+DB_HOST=postgresql
+DB_NAME=find
+DB_USER=dinum
+DB_PASSWORD=pass
+DB_PORT=5432

gitlint/gitlint_emoji.py

@@ -0,0 +1,37 @@
+"""
+Gitlint extra rule to validate that the message title is of the form
+"<gitmoji>(<scope>) <subject>"
+"""
+from __future__ import unicode_literals
+
+import re
+
+import requests
+
+from gitlint.rules import CommitMessageTitle, LineRule, RuleViolation
+
+
+class GitmojiTitle(LineRule):
+    """
+    This rule will enforce that each commit title is of the form "<gitmoji>(<scope>) <subject>"
+    where gitmoji is an emoji from the list defined in https://gitmoji.carloscuesta.me and
+    subject should be all lowercase
+    """
+
+    id = "UC1"
+    name = "title-should-have-gitmoji-and-scope"
+    target = CommitMessageTitle
+
+    def validate(self, title, _commit):
+        """
+        Download the list possible gitmojis from the project's github repository and check that
+        title contains one of them.
+        """
+        gitmojis = requests.get(
+            "https://raw.githubusercontent.com/carloscuesta/gitmoji/master/packages/gitmojis/src/gitmojis.json"
+        ).json()["gitmojis"]
+        emojis = [item["emoji"] for item in gitmojis]
+        pattern = r"^({:s})\(.*\)\s[a-z].*$".format("|".join(emojis))
+        if not re.search(pattern, title):
+            violation_msg = 'Title does not match regex "<gitmoji>(<scope>) <subject>"'
+            return [RuleViolation(self.id, violation_msg, title)]

index.yaml

@@ -1,28 +0,0 @@
-apiVersion: v1
-entries:
-  find:
-  - apiVersion: v2
-    created: "2026-03-16T08:58:33.405623957Z"
-    digest: 7db8aa465f4f5cf0b56d36ba9eebe0ab7d94536f6b5954362420c1089dacb2c4
-    name: find
-    type: application
-    urls:
-    - https://suitenumerique.github.io/find/find-0.0.3.tgz
-    version: 0.0.3
-  - apiVersion: v2
-    created: "2025-12-16T18:38:32.210537113Z"
-    digest: 9a00ea763fb6a7cb79275a0ccc5f7d83cc7bc1e8ac995544f9a6dcfcf1401f19
-    name: find
-    type: application
-    urls:
-    - https://suitenumerique.github.io/find/find-0.0.2.tgz
-    version: 0.0.2
-  - apiVersion: v2
-    created: "2025-01-27T10:10:29.611386242Z"
-    digest: bfbe5e73c4a7f306788cecfad9d65b22a89ec167257c3e03e72b097501ce1186
-    name: find
-    type: application
-    urls:
-    - https://suitenumerique.github.io/find/find-0.0.1.tgz
-    version: 0.0.1
-generated: "2026-03-16T08:58:33.404908133Z"

renovate.json

@@ -0,0 +1,13 @@
+{
+  "extends": ["github>numerique-gouv/renovate-configuration"],
+  "dependencyDashboard": true,
+  "labels": ["dependencies", "noChangeLog"],
+  "packageRules": [
+    {
+      "enabled": false,
+      "groupName": "ignored python dependencies",
+      "matchManagers": ["pep621"],
+      "matchPackageNames": []
+    }
+  ]
+}

src/backend/.pylintrc

@@ -0,0 +1,468 @@
+[MASTER]
+
+# A comma-separated list of package or module names from where C extensions may
+# be loaded. Extensions are loading into the active Python interpreter and may
+# run arbitrary code
+extension-pkg-whitelist=
+
+# Add files or directories to the blacklist. They should be base names, not
+# paths.
+ignore=migrations,.venv
+
+# Add files or directories matching the regex patterns to the blacklist. The
+# regex matches against base names, not paths.
+ignore-patterns=
+
+# Python code to execute, usually for sys.path manipulation such as
+# pygtk.require().
+#init-hook=
+
+# Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the
+# number of processors available to use.
+jobs=0
+
+# List of plugins (as comma separated values of python modules names) to load,
+# usually to register additional checkers.
+load-plugins=pylint_django,pylint.extensions.no_self_use
+
+# Pickle collected data for later comparisons.
+persistent=yes
+
+# Specify a configuration file.
+#rcfile=
+
+# Allow loading of arbitrary C extensions. Extensions are imported into the
+# active Python interpreter and may run arbitrary code.
+unsafe-load-any-extension=no
+
+
+[MESSAGES CONTROL]
+
+# Only show warnings with the listed confidence levels. Leave empty to show
+# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
+confidence=
+
+# Disable the message, report, category or checker with the given id(s). You
+# can either give multiple identifiers separated by comma (,) or put this
+# option multiple times (only on the command line, not in the configuration
+# file where it should appear only once).You can also use "--disable=all" to
+# disable everything first and then reenable specific checks. For example, if
+# you want to run only the similarities checker, you can use "--disable=all
+# --enable=similarities". If you want to run only the classes checker, but have
+# no Warning level messages displayed, use"--disable=all --enable=classes
+# --disable=W"
+disable=bad-inline-option,
+        deprecated-pragma,
+        django-not-configured,
+        file-ignored,
+        locally-disabled,
+        no-self-use,
+        raw-checker-failed,
+        suppressed-message,
+        useless-suppression
+
+# Enable the message, report, category or checker with the given id(s). You can
+# either give multiple identifier separated by comma (,) or put this option
+# multiple time (only on the command line, not in the configuration file where
+# it should appear only once). See also the "--disable" option for examples.
+enable=c-extension-no-member
+
+
+[REPORTS]
+
+# Python expression which should return a note less than 10 (10 is the highest
+# note). You have access to the variables errors warning, statement which
+# respectively contain the number of errors / warnings messages and the total
+# number of statements analyzed. This is used by the global evaluation report
+# (RP0004).
+evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
+
+# Template used to display messages. This is a python new-style format string
+# used to format the message information. See doc for all details
+#msg-template=
+
+# Set the output format. Available formats are text, parseable, colorized, json
+# and msvs (visual studio).You can also give a reporter class, eg
+# mypackage.mymodule.MyReporterClass.
+output-format=text
+
+# Tells whether to display a full report or only the messages
+reports=no
+
+# Activate the evaluation score.
+score=yes
+
+
+[REFACTORING]
+
+# Maximum number of nested blocks for function / method body
+max-nested-blocks=5
+
+# Complete name of functions that never returns. When checking for
+# inconsistent-return-statements if a never returning function is called then
+# it will be considered as an explicit return statement and no message will be
+# printed.
+never-returning-functions=optparse.Values,sys.exit
+
+
+[LOGGING]
+
+# Logging modules to check that the string format arguments are in logging
+# function parameter format
+logging-modules=logging
+
+
+[SPELLING]
+
+# Limits count of emitted suggestions for spelling mistakes
+max-spelling-suggestions=4
+
+# Spelling dictionary name. Available dictionaries: none. To make it working
+# install python-enchant package.
+spelling-dict=
+
+# List of comma separated words that should not be checked.
+spelling-ignore-words=
+
+# A path to a file that contains private dictionary; one word per line.
+spelling-private-dict-file=
+
+# Tells whether to store unknown words to indicated private dictionary in
+# --spelling-private-dict-file option instead of raising a message.
+spelling-store-unknown-words=no
+
+
+[MISCELLANEOUS]
+
+# List of note tags to take in consideration, separated by a comma.
+notes=FIXME,
+      XXX,
+      TODO
+
+
+[TYPECHECK]
+
+# List of decorators that produce context managers, such as
+# contextlib.contextmanager. Add to this list to register other decorators that
+# produce valid context managers.
+contextmanager-decorators=contextlib.contextmanager
+
+# List of members which are set dynamically and missed by pylint inference
+# system, and so shouldn't trigger E1101 when accessed. Python regular
+# expressions are accepted.
+generated-members=
+
+# Tells whether missing members accessed in mixin class should be ignored. A
+# mixin class is detected if its name ends with "mixin" (case insensitive).
+ignore-mixin-members=yes
+
+# This flag controls whether pylint should warn about no-member and similar
+# checks whenever an opaque object is returned when inferring. The inference
+# can return multiple potential results while evaluating a Python object, but
+# some branches might not be evaluated, which results in partial inference. In
+# that case, it might be useful to still emit no-member and other checks for
+# the rest of the inferred objects.
+ignore-on-opaque-inference=yes
+
+# List of class names for which member attributes should not be checked (useful
+# for classes with dynamically set attributes). This supports the use of
+# qualified names.
+ignored-classes=optparse.Values,thread._local,_thread._local,responses,
+                Template,Contact
+
+# List of module names for which member attributes should not be checked
+# (useful for modules/projects where namespaces are manipulated during runtime
+# and thus existing member attributes cannot be deduced by static analysis. It
+# supports qualified module names, as well as Unix pattern matching.
+ignored-modules=
+
+# Show a hint with possible names when a member name was not found. The aspect
+# of finding the hint is based on edit distance.
+missing-member-hint=yes
+
+# The minimum edit distance a name should have in order to be considered a
+# similar match for a missing member name.
+missing-member-hint-distance=1
+
+# The total number of similar names that should be taken in consideration when
+# showing a hint for a missing member.
+missing-member-max-choices=1
+
+
+[VARIABLES]
+
+# List of additional names supposed to be defined in builtins. Remember that
+# you should avoid to define new builtins when possible.
+additional-builtins=
+
+# Tells whether unused global variables should be treated as a violation.
+allow-global-unused-variables=yes
+
+# List of strings which can identify a callback function by name. A callback
+# name must start or end with one of those strings.
+callbacks=cb_,
+          _cb
+
+# A regular expression matching the name of dummy variables (i.e. expectedly
+# not used).
+dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
+
+# Argument names that match this expression will be ignored. Default to name
+# with leading underscore
+ignored-argument-names=_.*|^ignored_|^unused_
+
+# Tells whether we should check for unused import in __init__ files.
+init-import=no
+
+# List of qualified module names which can have objects that can redefine
+# builtins.
+redefining-builtins-modules=six.moves,past.builtins,future.builtins
+
+
+[FORMAT]
+
+# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
+expected-line-ending-format=
+
+# Regexp for a line that is allowed to be longer than the limit.
+ignore-long-lines=^\s*(# )?<?https?://\S+>?$
+
+# Number of spaces of indent required inside a hanging  or continued line.
+indent-after-paren=4
+
+# String used as indentation unit. This is usually "    " (4 spaces) or "\t" (1
+# tab).
+indent-string='    '
+
+# Maximum number of characters on a single line.
+max-line-length=100
+
+# Maximum number of lines in a module
+max-module-lines=1000
+
+# Allow the body of a class to be on the same line as the declaration if body
+# contains single statement.
+single-line-class-stmt=no
+
+# Allow the body of an if to be on the same line as the test if there is no
+# else.
+single-line-if-stmt=no
+
+
+[SIMILARITIES]
+
+# Ignore comments when computing similarities.
+ignore-comments=yes
+
+# Ignore docstrings when computing similarities.
+ignore-docstrings=yes
+
+# Ignore imports when computing similarities.
+ignore-imports=yes
+
+# Minimum lines number of a similarity.
+# First implementations of CMS wizards have common fields we do not want to factorize for now
+min-similarity-lines=35
+
+
+[BASIC]
+
+# Naming style matching correct argument names
+argument-naming-style=snake_case
+
+# Regular expression matching correct argument names. Overrides argument-
+# naming-style
+#argument-rgx=
+
+# Naming style matching correct attribute names
+attr-naming-style=snake_case
+
+# Regular expression matching correct attribute names. Overrides attr-naming-
+# style
+#attr-rgx=
+
+# Bad variable names which should always be refused, separated by a comma
+bad-names=foo,
+          bar,
+          baz,
+          toto,
+          tutu,
+          tata
+
+# Naming style matching correct class attribute names
+class-attribute-naming-style=any
+
+# Regular expression matching correct class attribute names. Overrides class-
+# attribute-naming-style
+#class-attribute-rgx=
+
+# Naming style matching correct class names
+class-naming-style=PascalCase
+
+# Regular expression matching correct class names. Overrides class-naming-style
+#class-rgx=
+
+# Naming style matching correct constant names
+const-naming-style=UPPER_CASE
+
+# Regular expression matching correct constant names. Overrides const-naming-
+# style
+const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__)|urlpatterns|logger)$
+
+# Minimum line length for functions/classes that require docstrings, shorter
+# ones are exempt.
+docstring-min-length=-1
+
+# Naming style matching correct function names
+function-naming-style=snake_case
+
+# Regular expression matching correct function names. Overrides function-
+# naming-style
+#function-rgx=
+
+# Good variable names which should always be accepted, separated by a comma
+good-names=i,
+           j,
+           k,
+           cm,
+           ex,
+           Run,
+           _
+
+# Include a hint for the correct naming format with invalid-name
+include-naming-hint=no
+
+# Naming style matching correct inline iteration names
+inlinevar-naming-style=any
+
+# Regular expression matching correct inline iteration names. Overrides
+# inlinevar-naming-style
+#inlinevar-rgx=
+
+# Naming style matching correct method names
+method-naming-style=snake_case
+
+# Regular expression matching correct method names. Overrides method-naming-
+# style
+method-rgx=([a-z_][a-z0-9_]{2,50}|setUp|set[Uu]pClass|tearDown|tear[Dd]ownClass|assert[A-Z]\w*|maxDiff|test_[a-z0-9_]+)$
+
+# Naming style matching correct module names
+module-naming-style=snake_case
+
+# Regular expression matching correct module names. Overrides module-naming-
+# style
+#module-rgx=
+
+# Colon-delimited sets of names that determine each other's naming style when
+# the name regexes allow several styles.
+name-group=
+
+# Regular expression which should only match function or class names that do
+# not require a docstring.
+no-docstring-rgx=^_
+
+# List of decorators that produce properties, such as abc.abstractproperty. Add
+# to this list to register other decorators that produce valid properties.
+property-classes=abc.abstractproperty
+
+# Naming style matching correct variable names
+variable-naming-style=snake_case
+
+# Regular expression matching correct variable names. Overrides variable-
+# naming-style
+#variable-rgx=
+
+
+[IMPORTS]
+
+# Allow wildcard imports from modules that define __all__.
+allow-wildcard-with-all=no
+
+# Analyse import fallback blocks. This can be used to support both Python 2 and
+# 3 compatible code, which means that the block might have code that exists
+# only in one or another interpreter, leading to false positives when analysed.
+analyse-fallback-blocks=no
+
+# Deprecated modules which should not be used, separated by a comma
+deprecated-modules=optparse,tkinter.tix
+
+# Create a graph of external dependencies in the given file (report RP0402 must
+# not be disabled)
+ext-import-graph=
+
+# Create a graph of every (i.e. internal and external) dependencies in the
+# given file (report RP0402 must not be disabled)
+import-graph=
+
+# Create a graph of internal dependencies in the given file (report RP0402 must
+# not be disabled)
+int-import-graph=
+
+# Force import order to recognize a module as part of the standard
+# compatibility libraries.
+known-standard-library=
+
+# Force import order to recognize a module as part of a third party library.
+known-third-party=enchant
+
+
+[CLASSES]
+
+# List of method names used to declare (i.e. assign) instance attributes.
+defining-attr-methods=__init__,
+                      __new__,
+                      setUp
+
+# List of member names, which should be excluded from the protected access
+# warning.
+exclude-protected=_asdict,
+                  _fields,
+                  _replace,
+                  _source,
+                  _make
+
+# List of valid names for the first argument in a class method.
+valid-classmethod-first-arg=cls
+
+# List of valid names for the first argument in a metaclass class method.
+valid-metaclass-classmethod-first-arg=mcs
+
+
+[DESIGN]
+
+# Maximum number of arguments for function / method
+max-args=5
+
+# Maximum number of attributes for a class (see R0902).
+max-attributes=7
+
+# Maximum number of boolean expressions in a if statement
+max-bool-expr=5
+
+# Maximum number of branch for function / method body
+max-branches=12
+
+# Maximum number of locals for function / method body
+max-locals=15
+
+# Maximum number of parents for a class (see R0901).
+max-parents=7
+
+# Maximum number of public methods for a class (see R0904).
+max-public-methods=20
+
+# Maximum number of return / yield for function / method body
+max-returns=6
+
+# Maximum number of statements in function / method body
+max-statements=50
+
+# Minimum number of public methods for a class (see R0903).
+min-public-methods=0
+
+
+[EXCEPTIONS]
+
+# Exceptions that will emit a warning when being caught. Defaults to
+# "Exception"
+overgeneral-exceptions=builtins.Exception

src/backend/MANIFEST.in

@@ -0,0 +1,3 @@
+include LICENSE
+include README.md
+recursive-include src/backend/find *.html *.png *.gif *.css *.ico *.jpg *.jpeg *.po *.mo *.eot *.svg *.ttf *.woff *.woff2

src/backend/core/admin.py

@@ -0,0 +1,100 @@
+"""Admin config for find's core app"""
+
+from django.contrib import admin, messages
+from django.shortcuts import redirect, render
+from django.urls import path, reverse
+
+from core.management.commands.create_search_pipeline import (
+    ensure_search_pipeline_exists,
+)
+
+from . import selftests_builtin  # pylint: disable=unused-import
+from .models import Service
+from .selftests import registry
+
+
+@admin.register(Service)
+class ServiceAdmin(admin.ModelAdmin):
+    """Register the serivce model for the admin site"""
+
+    list_display = ("name", "created_at", "is_active")
+    search_fields = ("name",)
+    list_filter = ("is_active", "created_at")
+    ordering = ("-created_at",)
+    readonly_fields = ("created_at", "token")
+    change_list_template = "admin/core/services/change_list.html"
+
+    def get_urls(self):
+        urls = super().get_urls()
+        custom_urls = [
+            path(
+                "ensure-search-pipeline/",
+                self.admin_site.admin_view(self.ensure_search_pipeline_view),
+                name="core_service_ensure_search_pipeline",
+            ),
+        ]
+        return custom_urls + urls
+
+    def ensure_search_pipeline_view(self, request):
+        """Run the management command function to assert the pipeline exists."""
+        changelist_url = reverse("admin:core_service_changelist")
+
+        try:
+            ensure_search_pipeline_exists()
+        except Exception as exc:  # noqa: BLE001# pylint: disable=broad-exception-caught
+            self.message_user(
+                request, f"Failed to ensure search pipeline: {exc}", messages.ERROR
+            )
+        else:
+            self.message_user(
+                request, "Search pipeline presence insured", messages.SUCCESS
+            )
+
+        return redirect(changelist_url)
+
+
+def selftest_view(request):
+    """Display the self-test page and run tests if requested."""
+    # selftests_builtin and registry are imported at module level to ensure tests are registered
+    run_tests = request.GET.get("run", "false").lower() == "true"
+
+    if run_tests:
+        results = registry.run_all()
+        all_passed = all(result.success for result in results)
+    else:
+        results = []
+        all_passed = None
+
+    context = {
+        **admin.site.each_context(request),
+        "title": "System Self-Tests",
+        "results": results,
+        "all_passed": all_passed,
+        "run_tests": run_tests,
+        "available_tests": registry.get_all_tests(),
+    }
+
+    return render(request, "admin/selftest.html", context)
+
+
+# Add custom URL to the default admin site
+def get_admin_urls():
+    """Get URLs with selftest added."""
+
+    def get_urls():
+        urls = get_admin_urls.original_get_urls()
+        custom_urls = [
+            path(
+                "selftest/",
+                admin.site.admin_view(selftest_view),
+                name="selftest",
+            ),
+        ]
+        return custom_urls + urls
+
+    return get_urls
+
+
+# Store original get_urls and override it
+get_admin_urls.original_get_urls = admin.site.get_urls
+admin.site.get_urls = get_admin_urls()

src/backend/core/api.py

@@ -0,0 +1,37 @@
+"""Impress core API endpoints"""
+
+from django.core.exceptions import ValidationError
+
+from pydantic import ValidationError as PydanticValidationError
+from rest_framework import exceptions as drf_exceptions
+from rest_framework import status
+from rest_framework import views as drf_views
+from rest_framework.response import Response
+
+
+def exception_handler(exc, context):
+    """Handle Django ValidationError as an accepted exception.
+
+    For the parameters, see ``exception_handler``
+    This code comes from twidi's gist:
+    https://gist.github.com/twidi/9d55486c36b6a51bdcb05ce3a763e79f
+    """
+    if isinstance(exc, ValidationError):
+        detail = exc.message_dict
+
+        if hasattr(exc, "message"):
+            detail = exc.message
+        elif hasattr(exc, "messages"):
+            detail = exc.messages
+        exc = drf_exceptions.ValidationError(detail=detail)
+
+    elif isinstance(exc, PydanticValidationError):
+        return Response(
+            [
+                {key: error[key] for key in ("msg", "type", "loc")}
+                for error in exc.errors()
+            ],
+            status=status.HTTP_400_BAD_REQUEST,
+        )
+
+    return drf_views.exception_handler(exc, context)

src/backend/core/apps.py

@@ -0,0 +1,26 @@
+"""Find Core application"""
+
+from django.apps import AppConfig
+from django.utils.translation import gettext_lazy as _
+
+from core.management.commands.create_search_pipeline import (
+    ensure_search_pipeline_exists,
+)
+from core.services.opensearch import (
+    check_hybrid_search_enabled,
+)
+
+
+class CoreConfig(AppConfig):
+    """Configuration class for the Find core app."""
+
+    name = "core"
+    app_label = "core"
+    verbose_name = _("Find core application")
+
+    def ready(self):
+        """
+        Ensure search pipeline exists if hybrid search is enabled.
+        """
+        if check_hybrid_search_enabled():
+            ensure_search_pipeline_exists()

src/backend/core/authentication.py

@@ -0,0 +1,76 @@
+"""Token authentication."""
+
+import logging
+
+from django.contrib.auth.models import AnonymousUser
+from django.core.exceptions import ObjectDoesNotExist
+
+from lasuite.oidc_resource_server.backend import ResourceServerBackend
+from rest_framework import authentication, exceptions
+
+from .models import Service
+
+logger = logging.getLogger(__name__)
+
+
+class ServiceTokenAuthentication(authentication.BaseAuthentication):
+    """A custom authentication looking for valid tokens among registered services"""
+
+    model = Service
+
+    def authenticate(self, request):
+        """Authenticate token from the "Authorization" header."""
+        token = request.headers.get("Authorization")
+        if not token:
+            raise exceptions.NotAuthenticated()
+
+        token = token.split(" ")[-1]  # Extract token if prefixed with "Token"
+
+        return self.authenticate_credentials(token)
+
+    def authenticate_credentials(self, token):
+        """Check that the token is registered and valid."""
+        try:
+            service = self.model.objects.only("name").get(token=token, is_active=True)
+        except self.model.DoesNotExist as excpt:
+            raise exceptions.AuthenticationFailed("Invalid token.") from excpt
+
+        # We don't associate tokens with a user
+        return AnonymousUser(), service
+
+
+class ResourceUserManager:
+    """Fake manager that simply returns an instance of user object with the right sub"""
+
+    def get(self, sub):
+        """Returns a ResourceUser for this sub"""
+        return ResourceUser(sub=sub)
+
+
+class ResourceUser:
+    """Fake user model for the ResourceServerBackend.get_object() method"""
+
+    DoesNotExist = ObjectDoesNotExist
+
+    objects = ResourceUserManager()
+
+    def __init__(self, sub: str):
+        self.sub = sub
+        self.pk = None
+        self.is_authenticated = True
+
+
+class FinderResourceServerBackend(ResourceServerBackend):
+    """
+    Custom resource server that uses a ResourceUser object instead of
+    a user model from the database
+    """
+
+    def __init__(self):
+        try:
+            super().__init__()
+        except Exception as e:
+            logger.error(e)
+            raise
+
+        self.UserModel = ResourceUser

src/backend/core/enums.py

@@ -0,0 +1,55 @@
+"""Enums for find's core app."""
+
+from enum import Enum
+
+# Reach
+
+
+class ReachEnum(str, Enum):
+    """Publication options for indexed documents"""
+
+    PUBLIC = "public"
+    AUTHENTICATED = "authenticated"
+    RESTRICTED = "restricted"
+
+
+# Search type
+
+
+class SearchTypeEnum(str, Enum):
+    """Search type options"""
+
+    HYBRID = "hybrid"
+    FULL_TEXT = "full_text"
+
+
+# Fields
+
+CREATED_AT = "created_at"
+DEPTH = "depth"
+PATH = "path"
+NUMCHILD = "numchild"
+REACH = "reach"
+SIZE = "size"
+TAGS = "tags"
+TITLE = "title"
+CONTENT = "content"
+UPDATED_AT = "updated_at"
+USERS = "users"
+GROUPS = "groups"
+
+RELEVANCE = "relevance"
+
+ORDER_BY_OPTIONS = (RELEVANCE, TITLE, CREATED_AT, UPDATED_AT, SIZE, REACH)
+SOURCE_FIELDS = (
+    TITLE,
+    CONTENT,
+    SIZE,
+    DEPTH,
+    PATH,
+    NUMCHILD,
+    CREATED_AT,
+    UPDATED_AT,
+    REACH,
+    TAGS,
+)

src/backend/core/factories.py

@@ -0,0 +1,59 @@
+"""Factories for the find's core app"""
+
+from uuid import uuid4
+
+from django.utils import timezone
+from django.utils.text import slugify
+
+import factory
+from faker import Faker
+
+from . import enums, models
+
+fake = Faker()
+
+
+class DocumentSchemaFactory(factory.DictFactory):
+    """
+    A factory for generating dictionaries that represent a document for
+    indexation for testing and development purposes.
+    """
+
+    id = factory.LazyFunction(lambda: str(uuid4()))
+    title = factory.Sequence(lambda n: f"Test title {n!s}")
+    path = factory.Sequence(lambda n: f"000{n}")
+    content = factory.Sequence(lambda n: f"Test content {n!s}")
+    created_at = factory.LazyFunction(
+        lambda: fake.date_time_this_decade(tzinfo=timezone.get_current_timezone())
+    )
+    size = factory.LazyFunction(lambda: fake.random_int(min=0, max=1024**2))
+    users = factory.LazyFunction(lambda: [str(uuid4()) for _ in range(3)])
+    groups = factory.LazyFunction(lambda: [slugify(fake.word()) for _ in range(3)])
+    reach = factory.Iterator(list(enums.ReachEnum))
+    tags = factory.LazyFunction(lambda: [])
+    depth = 1
+    numchild = 0
+    is_active = True
+
+    @factory.lazy_attribute
+    def updated_at(self):
+        """Ensure updated_at is after created_at and before now"""
+        return fake.date_time_between(
+            start_date=self.created_at,
+            end_date=timezone.now(),
+            tzinfo=timezone.get_current_timezone(),
+        )
+
+
+class ServiceFactory(factory.django.DjangoModelFactory):
+    """
+    A factory for generating service instances for testing and development purposes.
+    """
+
+    name = factory.Sequence(lambda n: f"test-index-{n!s}")
+    created_at = factory.Faker("date_time_this_year", tzinfo=None)
+    is_active = True
+    client_id = "some_client_id"
+
+    class Meta:
+        model = models.Service

src/backend/core/management/commands/create_search_pipeline.py

@@ -0,0 +1,54 @@
+"""
+Handle create the search pipeline command of the hybrid search.
+"""
+
+import logging
+
+from django.conf import settings
+from django.core.management.base import BaseCommand
+
+from opensearchpy.exceptions import NotFoundError
+
+from core.services.opensearch import (
+    opensearch_client,
+)
+
+logger = logging.getLogger(__name__)
+
+
+class Command(BaseCommand):
+    """Handle create the search pipeline command of the hybrid search."""
+
+    help = __doc__
+
+    def handle(self, *args, **options):
+        ensure_search_pipeline_exists()
+
+
+def ensure_search_pipeline_exists():
+    """Create search pipeline for hybrid search if it does not exist"""
+    try:
+        opensearch_client().search_pipeline.get(id=settings.HYBRID_SEARCH_PIPELINE_ID)
+        logger.info("Search pipeline exists already")
+    except NotFoundError:
+        logger.info("Creating search pipeline: %s", settings.HYBRID_SEARCH_PIPELINE_ID)
+        opensearch_client().transport.perform_request(
+            method="PUT",
+            url="/_search/pipeline/" + settings.HYBRID_SEARCH_PIPELINE_ID,
+            body={
+                "description": "Post processor for hybrid search",
+                "phase_results_processors": [
+                    {
+                        "normalization-processor": {
+                            "normalization": {"technique": "min_max"},
+                            "combination": {
+                                "technique": "arithmetic_mean",
+                                "parameters": {
+                                    "weights": settings.HYBRID_SEARCH_WEIGHTS
+                                },
+                            },
+                        }
+                    }
+                ],
+            },
+        )

src/backend/core/management/commands/reindex_with_embedding.py

@@ -0,0 +1,141 @@
+"""
+Handle reindexing of documents with embeddings in OpenSearch.
+"""
+
+import logging
+
+from django.conf import settings
+from django.core.management.base import BaseCommand, CommandError
+
+from opensearchpy.exceptions import NotFoundError
+
+from core.models import get_opensearch_index_name
+from core.services.indexing import chunk_document
+from core.services.opensearch import check_hybrid_search_enabled, opensearch_client
+from core.utils import get_language_value
+
+logger = logging.getLogger(__name__)
+
+
+class Command(BaseCommand):
+    """Reindex all documents with embeddings"""
+
+    help = __doc__
+    opensearch_client_ = opensearch_client()
+
+    def add_arguments(self, parser):
+        parser.add_argument("index_name", type=str)
+
+    def handle(self, *args, **options):
+        """Launch the reindexing with embedding."""
+
+        index_name = get_opensearch_index_name(options["index_name"])
+
+        if not check_hybrid_search_enabled():
+            raise CommandError("Hybrid search is not enabled or properly configured.")
+
+        try:
+            self.opensearch_client_.indices.get(index=index_name)
+        except NotFoundError as error:
+            raise CommandError(f"Index {index_name} does not exist.") from error
+
+        self.stdout.write(f"[INFO] Start reindexing {index_name} with embedding.")
+
+        result = reindex_with_embedding(index_name)
+
+        self.stdout.write(
+            f"[INFO] Reindexing of {index_name} is done.\n"
+            f"nb success embedding: {result['nb_success_embedding']}\n"
+            f"nb failed embedding: {result['nb_failed_embedding']} embedding fails\n"
+        )
+
+
+def reindex_with_embedding(index_name, batch_size=500, scroll="10m"):
+    """
+    Reindex documents from source index to destination index with embeddings.
+
+    returns a dict with the number of successful embeddings and failed embeddings.
+    """
+    opensearch_client_ = opensearch_client()
+    page = opensearch_client_.search(  # pylint: disable=unexpected-keyword-arg
+        index=index_name,
+        scroll=scroll,
+        size=batch_size,
+        seq_no_primary_term=True,
+        body={
+            "query": {
+                "bool": {
+                    "should": [
+                        {
+                            "bool": {
+                                "must_not": [
+                                    {
+                                        "nested": {
+                                            "path": "chunks",
+                                            "query": {"match_all": {}},
+                                        }
+                                    }
+                                ]
+                            }
+                        },
+                        {
+                            "bool": {
+                                "must_not": [
+                                    {
+                                        "term": {
+                                            "embedding_model": settings.EMBEDDING_API_MODEL_NAME
+                                        }
+                                    }
+                                ]
+                            }
+                        },
+                    ],
+                    "minimum_should_match": 1,
+                }
+            }
+        },
+    )
+    nb_failed_embedding = 0
+    nb_success_embedding = 0
+    while len(page["hits"]["hits"]) > 0:
+        actions = []
+        for hit in page["hits"]["hits"]:
+            source = hit["_source"]
+            chunks = chunk_document(
+                get_language_value(source, "title"),
+                get_language_value(source, "content"),
+            )
+            if chunks:
+                actions.append(
+                    {
+                        "update": {
+                            "_id": hit["_id"],
+                            # if_seq_no and if_primary_term ensure we only update indexes
+                            # if the document hasn't changed
+                            "if_seq_no": hit["_seq_no"],
+                            "if_primary_term": hit["_primary_term"],
+                        }
+                    }
+                )
+                actions.append(
+                    {
+                        "doc": {
+                            "chunks": chunks,
+                            "embedding_model": settings.EMBEDDING_API_MODEL_NAME,
+                        }
+                    }
+                )
+                nb_success_embedding += 1
+            else:
+                nb_failed_embedding += 1
+
+        opensearch_client_.bulk(index=index_name, body=actions)
+        page = opensearch_client_.scroll(  # pylint: disable=unexpected-keyword-arg
+            scroll_id=page["_scroll_id"], scroll=scroll
+        )
+
+    opensearch_client_.clear_scroll(scroll_id=page["_scroll_id"])
+    return {
+        "nb_failed_embedding": nb_failed_embedding,
+        "nb_success_embedding": nb_success_embedding,
+    }

src/backend/core/migrations/0001_initial.py

@@ -0,0 +1,64 @@
+# Generated by Django 5.0.7 on 2024-09-01 05:31
+
+import django.contrib.auth.models
+import django.contrib.auth.validators
+import django.utils.timezone
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = [
+        ('auth', '0012_alter_user_first_name_max_length'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Service',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('name', models.SlugField(max_length=20, unique=True)),
+                ('token', models.CharField(max_length=50)),
+                ('created_at', models.DateTimeField(auto_now_add=True)),
+                ('is_active', models.BooleanField(default=True)),
+            ],
+            options={
+                'verbose_name': 'service',
+                'verbose_name_plural': 'services',
+                'db_table': 'find_service',
+                'ordering': ['-is_active', '-created_at'],
+            },
+        ),
+        migrations.CreateModel(
+            name='User',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('password', models.CharField(max_length=128, verbose_name='password')),
+                ('last_login', models.DateTimeField(blank=True, null=True, verbose_name='last login')),
+                ('is_superuser', models.BooleanField(default=False, help_text='Designates that this user has all permissions without explicitly assigning them.', verbose_name='superuser status')),
+                ('username', models.CharField(error_messages={'unique': 'A user with that username already exists.'}, help_text='Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only.', max_length=150, unique=True, validators=[django.contrib.auth.validators.UnicodeUsernameValidator()], verbose_name='username')),
+                ('first_name', models.CharField(blank=True, max_length=150, verbose_name='first name')),
+                ('last_name', models.CharField(blank=True, max_length=150, verbose_name='last name')),
+                ('email', models.EmailField(blank=True, max_length=254, verbose_name='email address')),
+                ('is_staff', models.BooleanField(default=False, help_text='Designates whether the user can log into this admin site.', verbose_name='staff status')),
+                ('is_active', models.BooleanField(default=True, help_text='Designates whether this user should be treated as active. Unselect this instead of deleting accounts.', verbose_name='active')),
+                ('date_joined', models.DateTimeField(default=django.utils.timezone.now, verbose_name='date joined')),
+                ('groups', models.ManyToManyField(blank=True, help_text='The groups this user belongs to. A user will get all permissions granted to each of their groups.', related_name='user_set', related_query_name='user', to='auth.group', verbose_name='groups')),
+                ('user_permissions', models.ManyToManyField(blank=True, help_text='Specific permissions for this user.', related_name='user_set', related_query_name='user', to='auth.permission', verbose_name='user permissions')),
+            ],
+            options={
+                'verbose_name': 'user',
+                'verbose_name_plural': 'users',
+                'abstract': False,
+            },
+            managers=[
+                ('objects', django.contrib.auth.models.UserManager()),
+            ],
+        ),
+        migrations.AddConstraint(
+            model_name='service',
+            constraint=models.CheckConstraint(condition=models.Q(('token__length', 50)), name='token_length_exact_50'),
+        ),
+    ]

src/backend/core/migrations/0002_service_client_id_service_services.py

@@ -0,0 +1,23 @@
+# Generated by Django 5.1.4 on 2025-09-05 04:41
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('core', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='service',
+            name='client_id',
+            field=models.CharField(blank=True, null=True),
+        ),
+        migrations.AddField(
+            model_name='service',
+            name='services',
+            field=models.ManyToManyField(blank=True, to='core.service', verbose_name='Allowed services for search'),
+        ),
+    ]

src/backend/core/models.py

@@ -0,0 +1,75 @@
+"""Models for find's core app"""
+
+import secrets
+import string
+
+from django.conf import settings
+from django.contrib.auth.models import AbstractUser
+from django.db import models
+from django.db.models.functions import Length
+from django.utils.functional import cached_property
+from django.utils.text import slugify
+from django.utils.translation import gettext_lazy as _
+
+models.CharField.register_lookup(Length)
+TOKEN_LENGTH = 50
+
+
+def get_opensearch_index_name(name: str):
+    """Returns the opensearch index for a service name"""
+    return f"{settings.OPENSEARCH_INDEX_PREFIX}-{name}"
+
+
+class User(AbstractUser):
+    """User for the find application"""
+
+
+class Service(models.Model):
+    """Service registered to index its documents to our find"""
+
+    name = models.SlugField(max_length=20, unique=True)
+    token = models.CharField(max_length=TOKEN_LENGTH)
+    created_at = models.DateTimeField(auto_now_add=True)
+    is_active = models.BooleanField(default=True)
+    client_id = models.CharField(blank=True, null=True)
+    services = models.ManyToManyField(
+        "self",
+        verbose_name=_("Allowed services for search"),
+        blank=True,
+    )
+
+    class Meta:
+        db_table = "find_service"
+        verbose_name = _("service")
+        verbose_name_plural = _("services")
+        ordering = ["-is_active", "-created_at"]
+        constraints = [
+            models.CheckConstraint(
+                condition=models.Q(token__length=TOKEN_LENGTH),
+                name="token_length_exact_50",
+            ),
+        ]
+
+    def __str__(self):
+        return self.name
+
+    def save(self, *args, **kwargs):
+        """Automatically slugify the service name and generate a token on creation"""
+        self.name = slugify(self.name)
+        if not self.token:
+            self.token = self.generate_secure_token()
+        super().save(*args, **kwargs)
+
+    @staticmethod
+    def generate_secure_token():
+        """Generate a secure token with with Python secret module"""
+        characters = (
+            string.ascii_letters + string.digits + r"""!#%&'()*+,-./:;<=>?@[\]^_`{|}~"""
+        )
+        token = "".join(secrets.choice(characters) for _ in range(TOKEN_LENGTH))
+        return token
+
+    @cached_property
+    def index_name(self):
+        """Returns the opensearch index for the service"""
+        return get_opensearch_index_name(self.name)

src/backend/core/permissions.py

@@ -0,0 +1,12 @@
+"""Permission classes for find's core app"""
+
+from rest_framework import permissions
+
+
+class IsAuthAuthenticated(permissions.BasePermission):
+    """
+    Allows access only to auth authenticated users.
+    """
+
+    def has_permission(self, request, view):
+        return bool(request.auth)

src/backend/core/schemas.py

@@ -0,0 +1,139 @@
+"""Pydantic model to validate documents before indexation."""
+
+from typing import Annotated, List, Literal, Optional
+
+from django.utils import timezone
+from django.utils.text import slugify
+
+from pydantic import (
+    UUID4,
+    AwareDatetime,
+    BaseModel,
+    BeforeValidator,
+    ConfigDict,
+    Field,
+    conint,
+    field_validator,
+    model_validator,
+)
+
+from . import enums
+from .enums import SearchTypeEnum
+
+
+class DocumentSchema(BaseModel):
+    """Schema for validating the documents submitted to our API for indexing"""
+
+    id: UUID4
+    title: Annotated[str, Field(max_length=300, min_length=0)]
+    depth: Annotated[int, Field(ge=0)]
+    path: Annotated[str, Field(max_length=300)]
+    numchild: Annotated[int, Field(ge=0)]
+    content: Annotated[str, Field(min_length=0)]
+    created_at: AwareDatetime
+    updated_at: AwareDatetime
+    size: Annotated[int, Field(ge=0, le=100 * 1024**3)]  # File size limited to 100GB
+    users: List[Annotated[str, Field(max_length=50)]] = Field(default_factory=list)
+    groups: List[Annotated[str, Field(pattern=r"^[a-z0-9]+(?:-[a-z0-9]+)*$")]] = Field(
+        default_factory=list
+    )
+    reach: Optional[enums.ReachEnum] = Field(default=enums.ReachEnum.RESTRICTED)
+    tags: List[Annotated[str, Field(max_length=100)]] = Field(default_factory=list)
+    is_active: bool
+
+    model_config = ConfigDict(
+        str_min_length=1, str_strip_whitespace=True, use_enum_values=True
+    )
+
+    @field_validator("title")
+    @staticmethod
+    def normalize_title(value):
+        """Normalize the title field by stripping whitespace and converting to lowercase"""
+        return value.strip().lower()
+
+    @field_validator("created_at", "updated_at")
+    @staticmethod
+    def must_be_past(value, info):
+        """Validate that `created_at` and `updated_at` fields are in the past"""
+        if value >= timezone.now():
+            raise ValueError(f"{info.field_name} must be earlier than now")
+        return value
+
+    @model_validator(mode="after")
+    def check_empty_content(self):
+        """Validate that either `title` or `content` are not empty."""
+        if not self.title and not self.content:
+            raise ValueError("Either title or content should have at least 1 character")
+        return self
+
+    @model_validator(mode="after")
+    def check_update_at_after_created_at(self):
+        """Date and time of last modification should be later than date and time of creation"""
+        if self.created_at > self.updated_at:
+            raise ValueError("updated_at must be later than created_at")
+        return self
+
+    @field_validator("groups")
+    @staticmethod
+    def validate_groups(groups):
+        """Validate that group slugs are properly formatted as lowercase and hyphen-separated"""
+        validated_groups = []
+        for value in groups:
+            slug = slugify(value)
+            if value != slug:
+                raise ValueError(
+                    f"Groups must be slugs (lowercase, hyphen-separated): {slug:s}"
+                )
+            validated_groups.append(value)
+        return validated_groups
+
+
+def cleanlist(value):
+    """Build a list of strings from a string, None (empty list) or a list of objects."""
+    if isinstance(value, str):
+        # Convert comma-separated strings to list
+        return [s.strip() for s in value.split(",") if s.strip()]
+
+    if isinstance(value, list):
+        # Clean up list of strings
+        return [str(s).strip() for s in value if s is not None and str(s).strip()]
+
+    if value is None:
+        return []
+
+    raise ValueError()
+
+
+StringListParameter = Annotated[List[str], BeforeValidator(cleanlist)]
+
+
+class SearchQueryParametersSchema(BaseModel):
+    """Schema for validating the querystring on the search API endpoint"""
+
+    q: str
+    services: StringListParameter = Field(default_factory=list)
+    visited: StringListParameter = Field(default_factory=list)
+    reach: Optional[enums.ReachEnum] = None
+    tags: StringListParameter = Field(default_factory=list)
+    path: Optional[str] = None
+    order_by: Optional[Literal[enums.ORDER_BY_OPTIONS]] = Field(default=enums.RELEVANCE)
+    order_direction: Optional[Literal["asc", "desc"]] = Field(default="desc")
+    nb_results: Optional[conint(ge=1, le=300)] = Field(default=50)
+    search_type: Optional[SearchTypeEnum] = Field(default=None)
+
+
+class DeleteDocumentsSchema(BaseModel):
+    """Schema for validating the delete documents request"""
+
+    service: str = Field(max_length=300)
+    document_ids: Optional[List[str]] = Field(default=None)
+    tags: Optional[List[str]] = Field(default=None)
+
+    @model_validator(mode="after")
+    def check_at_least_one_filter(self):
+        """Ensure at least one of document_ids or tags is provided"""
+        if not self.document_ids and not self.tags:
+            raise ValueError(
+                "At least one of 'document_ids' or 'tags' must be provided"
+            )
+        return self

src/backend/core/selftests.py

@@ -0,0 +1,125 @@
+"""
+Selftest registry and base classes for system health checks.
+
+This module provides a modular system for registering and running self-tests
+that check the health of various system components.
+"""
+
+import logging
+from typing import Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+
+class SelfTestResult:
+    """Result of a self-test execution."""
+
+    def __init__(  # pylint: disable=too-many-arguments,too-many-positional-arguments
+        self,
+        name: str,
+        success: bool,
+        message: str,
+        details: Optional[Dict] = None,
+        duration_ms: Optional[float] = None,
+    ):
+        self.name = name
+        self.success = success
+        self.message = message
+        self.details = details or {}
+        self.duration_ms = duration_ms
+
+    def to_dict(self) -> Dict:
+        """Convert result to dictionary."""
+        return {
+            "name": self.name,
+            "success": self.success,
+            "message": self.message,
+            "details": self.details,
+            "duration_ms": self.duration_ms,
+        }
+
+
+class SelfTest:
+    """Base class for self-tests."""
+
+    name: str = "Base Self Test"
+    description: str = "Base self-test class"
+
+    def run(self) -> SelfTestResult:
+        """
+        Execute the self-test.
+
+        Returns:
+            SelfTestResult: The result of the test execution.
+        """
+        raise NotImplementedError("Subclasses must implement the run method")
+
+
+class SelfTestRegistry:
+    """Registry for managing self-tests."""
+
+    def __init__(self):
+        self._tests: Dict[str, SelfTest] = {}
+
+    def register(self, test_class: type[SelfTest]) -> None:
+        """
+        Register a self-test class.
+
+        Args:
+            test_class: The SelfTest subclass to register.
+        """
+        test_instance = test_class()
+        test_id = test_class.__name__
+        if test_id in self._tests:
+            logger.warning("Self-test %s is already registered. Overwriting.", test_id)
+        self._tests[test_id] = test_instance
+        logger.debug("Registered self-test: %s - %s", test_id, test_instance.name)
+
+    def unregister(self, test_class: type[SelfTest]) -> None:
+        """
+        Unregister a self-test class.
+
+        Args:
+            test_class: The SelfTest subclass to unregister.
+        """
+        test_id = test_class.__name__
+        if test_id in self._tests:
+            del self._tests[test_id]
+            logger.debug("Unregistered self-test: %s", test_id)
+
+    def get_all_tests(self) -> List[SelfTest]:
+        """
+        Get all registered tests.
+
+        Returns:
+            List of registered SelfTest instances.
+        """
+        return list(self._tests.values())
+
+    def run_all(self) -> List[SelfTestResult]:
+        """
+        Run all registered tests.
+
+        Returns:
+            List of SelfTestResult objects.
+        """
+        results = []
+        for test in self._tests.values():
+            try:
+                result = test.run()
+                results.append(result)
+            except Exception as e:  # pylint: disable=broad-exception-caught
+                logger.exception("Error running self-test %s: %s", test.name, e)
+                results.append(
+                    SelfTestResult(
+                        name=test.name,
+                        success=False,
+                        message=f"Test failed with exception: {str(e)}",
+                        details={"exception": str(e)},
+                    )
+                )
+        return results
+
+
+# Global registry instance
+registry = SelfTestRegistry()

src/backend/core/selftests_builtin.py

@@ -0,0 +1,236 @@
+"""
+Built-in self-tests for core system components.
+
+This module contains self-tests for database, cache, and OpenSearch connectivity.
+"""
+
+import time
+
+from django.conf import settings
+from django.core.cache import cache
+from django.db import connection
+
+import sentry_sdk
+
+from .selftests import SelfTest, SelfTestResult, registry
+from .services.opensearch import opensearch_client
+
+
+class DatabaseSelfTest(SelfTest):
+    """Test database connectivity."""
+
+    name = "Database Connection"
+    description = "Verify that the database is accessible and responsive"
+
+    def run(self) -> SelfTestResult:
+        """Test database connection by executing a simple query."""
+        start_time = time.time()
+        try:
+            with connection.cursor() as cursor:
+                cursor.execute("SELECT 1")
+                result = cursor.fetchone()
+
+            duration_ms = (time.time() - start_time) * 1000
+
+            if result and result[0] == 1:
+                return SelfTestResult(
+                    name=self.name,
+                    success=True,
+                    message="Database connection successful",
+                    details={
+                        "database": settings.DATABASES["default"]["NAME"],
+                        "engine": settings.DATABASES["default"]["ENGINE"],
+                    },
+                    duration_ms=duration_ms,
+                )
+            return SelfTestResult(
+                name=self.name,
+                success=False,
+                message="Database query returned unexpected result",
+                duration_ms=duration_ms,
+            )
+        except (OSError, ValueError) as e:
+            duration_ms = (time.time() - start_time) * 1000
+            return SelfTestResult(
+                name=self.name,
+                success=False,
+                message=f"Database connection failed: {str(e)}",
+                details={"exception": str(e)},
+                duration_ms=duration_ms,
+            )
+
+
+class CacheSelfTest(SelfTest):
+    """Test cache (Redis) connectivity."""
+
+    name = "Cache Connection"
+    description = "Verify that the cache system is accessible and functional"
+
+    def run(self) -> SelfTestResult:
+        """Test cache by setting and getting a test value."""
+        start_time = time.time()
+        test_key = "selftest:cache:ping"
+        test_value = "pong"
+
+        try:
+            # Try to set a value
+            cache.set(test_key, test_value, timeout=10)
+
+            # Try to get the value back
+            retrieved_value = cache.get(test_key)
+
+            # Clean up
+            cache.delete(test_key)
+
+            duration_ms = (time.time() - start_time) * 1000
+
+            if retrieved_value == test_value:
+                cache_backend = settings.CACHES.get("default", {}).get(
+                    "BACKEND", "unknown"
+                )
+                return SelfTestResult(
+                    name=self.name,
+                    success=True,
+                    message="Cache connection successful",
+                    details={"backend": cache_backend},
+                    duration_ms=duration_ms,
+                )
+            return SelfTestResult(
+                name=self.name,
+                success=False,
+                message="Cache value mismatch",
+                details={
+                    "expected": test_value,
+                    "received": retrieved_value,
+                },
+                duration_ms=duration_ms,
+            )
+        except (OSError, ValueError, TimeoutError) as e:
+            duration_ms = (time.time() - start_time) * 1000
+            return SelfTestResult(
+                name=self.name,
+                success=False,
+                message=f"Cache connection failed: {str(e)}",
+                details={"exception": str(e)},
+                duration_ms=duration_ms,
+            )
+
+
+class OpenSearchSelfTest(SelfTest):
+    """Test OpenSearch connectivity."""
+
+    name = "OpenSearch Connection"
+    description = "Verify that OpenSearch is accessible and responsive"
+
+    def run(self) -> SelfTestResult:
+        """Test OpenSearch connection by checking cluster health."""
+        start_time = time.time()
+        try:
+            client = opensearch_client()
+
+            # Ping the cluster
+            if not client.ping():
+                duration_ms = (time.time() - start_time) * 1000
+                return SelfTestResult(
+                    name=self.name,
+                    success=False,
+                    message="OpenSearch ping failed",
+                    duration_ms=duration_ms,
+                )
+
+            # Get cluster health
+            health = client.cluster.health()
+
+            duration_ms = (time.time() - start_time) * 1000
+
+            return SelfTestResult(
+                name=self.name,
+                success=True,
+                message="OpenSearch connection successful",
+                details={
+                    "cluster_name": health.get("cluster_name", "unknown"),
+                    "status": health.get("status", "unknown"),
+                    "number_of_nodes": health.get("number_of_nodes", 0),
+                    "number_of_data_nodes": health.get("number_of_data_nodes", 0),
+                    "active_shards": health.get("active_shards", 0),
+                    "host": settings.OPENSEARCH_HOST,
+                    "port": settings.OPENSEARCH_PORT,
+                },
+                duration_ms=duration_ms,
+            )
+        except (OSError, ValueError, TimeoutError) as e:
+            duration_ms = (time.time() - start_time) * 1000
+            return SelfTestResult(
+                name=self.name,
+                success=False,
+                message=f"OpenSearch connection failed: {str(e)}",
+                details={"exception": str(e)},
+                duration_ms=duration_ms,
+            )
+
+
+class SentrySelfTest(SelfTest):
+    """Test Sentry connectivity."""
+
+    name = "Sentry Connection"
+    description = "Verify that Sentry is accessible and responsive"
+
+    def run(self) -> SelfTestResult:
+        """Test Sentry connection by sending a test error."""
+        if not sentry_sdk:
+            return SelfTestResult(
+                name=self.name,
+                success=False,
+                message="Sentry SDK not available",
+            )
+
+        # Check if Sentry DSN is configured
+        sentry_dsn = getattr(settings, "SENTRY_DSN", None)
+        if not sentry_dsn:
+            return SelfTestResult(
+                name=self.name,
+                success=False,
+                message="Sentry DSN not configured",
+            )
+
+        start_time = time.time()
+        try:
+            # Send a test error to Sentry to verify connectivity
+            scope = sentry_sdk.get_current_scope()
+            scope.set_extra("selftest", "Sentry connectivity test")
+
+            try:
+                # Raise a fake error that we'll catch and send to Sentry
+                raise ValueError(
+                    "Sentry self-test error - fake error for connectivity check"
+                )
+            except ValueError as test_error:
+                sentry_sdk.capture_exception(test_error)
+
+            duration_ms = (time.time() - start_time) * 1000
+
+            return SelfTestResult(
+                name=self.name,
+                success=True,
+                message="Sentry connection successful",
+                details={"dsn_configured": bool(sentry_dsn)},
+                duration_ms=duration_ms,
+            )
+        except Exception as e:  # noqa: BLE001 pylint: disable=broad-except
+            duration_ms = (time.time() - start_time) * 1000
+            return SelfTestResult(
+                name=self.name,
+                success=False,
+                message=f"Sentry connection failed: {str(e)}",
+                details={"exception": str(e)},
+                duration_ms=duration_ms,
+            )
+
+
+# Register all built-in tests
+registry.register(DatabaseSelfTest)
+registry.register(CacheSelfTest)
+registry.register(OpenSearchSelfTest)
+
+if settings.SENTRY_DSN:
+    registry.register(SentrySelfTest)

src/backend/core/services/embedding.py

@@ -0,0 +1,42 @@
+"""OpenSearch embedding utilities."""
+
+import logging
+
+from django.conf import settings
+
+import requests
+
+logger = logging.getLogger(__name__)
+
+
+def embed_text(text):
+    """
+    Get embedding vector for the given text from any OpenAI-compatible embedding API
+    """
+    logger.info("embed: '%s'", text)
+
+    response = requests.post(
+        settings.EMBEDDING_API_PATH,
+        headers={"Authorization": f"Bearer {settings.EMBEDDING_API_KEY}"},
+        json={
+            "input": text,
+            "model": settings.EMBEDDING_API_MODEL_NAME,
+            "dimensions": settings.EMBEDDING_DIMENSION,
+            "encoding_format": "float",
+        },
+        timeout=settings.EMBEDDING_REQUEST_TIMEOUT,
+    )
+
+    try:
+        response.raise_for_status()
+    except requests.HTTPError as e:
+        logger.warning("embedding API request failed: %s", str(e))
+        return None
+
+    try:
+        embedding = response.json()["data"][0]["embedding"]
+    except (KeyError, IndexError, TypeError):
+        logger.warning("unexpected embedding response format: %s", response.text)
+        return None
+
+    return embedding

src/backend/core/services/indexing.py

@@ -0,0 +1,151 @@
+"""OpenSearch indexing utilities."""
+
+import logging
+
+from django.conf import settings
+from django.core.exceptions import SuspiciousOperation
+
+from langchain_text_splitters import RecursiveCharacterTextSplitter
+from opensearchpy.exceptions import NotFoundError
+from py3langid.langid import MODEL_FILE, LanguageIdentifier
+
+from core.services.opensearch_configuration import (
+    ANALYZERS,
+    FILTERS,
+    MAPPINGS,
+)
+
+from ..models import Service, get_opensearch_index_name
+from .embedding import embed_text
+from .opensearch import check_hybrid_search_enabled, opensearch_client
+
+logger = logging.getLogger(__name__)
+
+
+# see https://pypi.org/project/py3langid/
+LANGUAGE_IDENTIFIER = LanguageIdentifier.from_pickled_model(MODEL_FILE, norm_probs=True)
+LANGUAGE_IDENTIFIER.set_languages(["en", "fr", "de", "nl"])
+
+TEXT_SPLITER = RecursiveCharacterTextSplitter(
+    chunk_size=settings.CHUNK_SIZE,
+    chunk_overlap=settings.CHUNK_OVERLAP,
+)
+
+
+def ensure_index_exists(index_name):
+    """Create index if it does not exist"""
+    try:
+        opensearch_client().indices.get(index=index_name)
+    except NotFoundError:
+        logger.info("Creating index: %s", index_name)
+        opensearch_client().indices.create(
+            index=index_name,
+            body={
+                "settings": {
+                    "index.knn": True,
+                    "analysis": {
+                        "analyzer": ANALYZERS,
+                        "filter": FILTERS,
+                    },
+                },
+                "mappings": MAPPINGS,
+            },
+        )
+
+
+def prepare_document_for_indexing(document):
+    """Prepare document for indexing using nested language structure and handle embedding"""
+    language_code = detect_language_code(f"{document['title']} {document['content']}")
+    return {
+        "id": document["id"],
+        f"title.{language_code}": document["title"],
+        f"content.{language_code}": document["content"],
+        "embedding_model": settings.EMBEDDING_API_MODEL_NAME
+        if check_hybrid_search_enabled()
+        else None,
+        "chunks": chunk_document(
+            document["title"],
+            document["content"],
+        )
+        if check_hybrid_search_enabled()
+        else None,
+        "depth": document["depth"],
+        "path": document["path"],
+        "numchild": document["numchild"],
+        "created_at": document["created_at"],
+        "updated_at": document["updated_at"],
+        "size": document["size"],
+        "users": document["users"],
+        "groups": document["groups"],
+        "reach": document["reach"],
+        "tags": document.get("tags", []),
+        "is_active": document["is_active"],
+    }
+
+
+def chunk_document(title, content):
+    """
+    Chunk a document into multiple pieces and embed them.
+    """
+    chunks = []
+    for idx, chunked_content in enumerate(TEXT_SPLITER.split_text(content)):
+        embedding = embed_text(format_document(title, chunked_content))
+
+        if not embedding:
+            logger.warning(
+                "Failed to embed chunk %d of document '%s'. Document embedding is skipped",
+                idx,
+                title,
+            )
+            # if embedding fails for any chunk, we skip chunking the document
+            return None
+
+        chunks.append(
+            {
+                "index": idx,
+                "content": chunked_content,
+                "embedding": embedding,
+            }
+        )
+
+    logger.info("Document %s chunked into %d pieces", title, len(chunks))
+    return chunks
+
+
+def format_document(title, content):
+    """Get the embedding input format for a document"""
+    return f"<{title}>:<{content}>"
+
+
+def detect_language_code(text):
+    """Detect the language code of the document content."""
+
+    detected_code, confidence = LANGUAGE_IDENTIFIER.classify(text)
+
+    if confidence < settings.LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD:
+        return settings.UNDETERMINED_LANGUAGE_CODE
+
+    return detected_code
+
+
+def get_opensearch_indices(audience, services):
+    """
+    Get OpenSearch indices for the given audience and services.
+    """
+    try:
+        user_service = Service.objects.get(client_id=audience, is_active=True)
+    except Service.DoesNotExist as e:
+        logger.warning("Login failed: No service %s found", audience)
+        raise SuspiciousOperation("Service is not available") from e
+
+    # Find allowed sub-services for this service
+    allowed_services = set(user_service.services.values_list("name", flat=True))
+    allowed_services.add(user_service.name)
+
+    if services:
+        available_service = set(services).intersection(allowed_services)
+
+        if len(available_service) < len(services):
+            raise SuspiciousOperation("Some requested services are not available")
+
+    return [get_opensearch_index_name(service) for service in allowed_services]

src/backend/core/services/opensearch.py

@@ -0,0 +1,66 @@
+"""OpenSearch common utilities."""
+
+import logging
+from functools import cache
+
+from django.conf import settings
+
+from opensearchpy import OpenSearch
+from rest_framework.exceptions import ValidationError
+
+logger = logging.getLogger(__name__)
+
+
+REQUIRED_ENV_VARIABLES = [
+    "OPENSEARCH_HOST",
+    "OPENSEARCH_PORT",
+    "OPENSEARCH_USER",
+    "OPENSEARCH_PASSWORD",
+    "OPENSEARCH_USE_SSL",
+]
+
+
+@cache
+def opensearch_client():
+    """Get OpenSearch client, ensuring required env variables are set"""
+    missing_env_variables = [
+        variable
+        for variable in REQUIRED_ENV_VARIABLES
+        if getattr(settings, variable, None) is None
+    ]
+    if missing_env_variables:
+        raise ValidationError(
+            f"Missing required OpenSearch environment variables: {', '.join(missing_env_variables)}"
+        )
+
+    return OpenSearch(
+        hosts=[{"host": settings.OPENSEARCH_HOST, "port": settings.OPENSEARCH_PORT}],
+        http_auth=(settings.OPENSEARCH_USER, settings.OPENSEARCH_PASSWORD),
+        timeout=50,
+        use_ssl=settings.OPENSEARCH_USE_SSL,
+        verify_certs=False,
+    )
+
+
+@cache
+def check_hybrid_search_enabled():
+    """Check that all required environment variables are set for hybrid search."""
+    if settings.HYBRID_SEARCH_ENABLED is not True:
+        logger.info("Hybrid search is disabled via HYBRID_SEARCH_ENABLED setting")
+        return False
+
+    required_vars = [
+        "HYBRID_SEARCH_WEIGHTS",
+        "EMBEDDING_API_PATH",
+        "EMBEDDING_API_KEY",
+        "EMBEDDING_API_MODEL_NAME",
+        "EMBEDDING_DIMENSION",
+    ]
+    missing_vars = [var for var in required_vars if not getattr(settings, var, None)]
+    if missing_vars:
+        logger.warning(
+            "Missing variables for hybrid search: %s", ", ".join(missing_vars)
+        )
+        return False
+
+    return True

src/backend/core/services/opensearch_configuration.py

@@ -0,0 +1,292 @@
+"""OpenSearch configuration."""
+
+from django.conf import settings
+
+ANALYZERS = {
+    "french_analyzer": {
+        "type": "custom",
+        "tokenizer": "standard",
+        "filter": [
+            "lowercase",
+            "asciifolding",
+            "french_elision",
+            "french_stop",
+            "french_stemmer",
+        ],
+    },
+    "english_analyzer": {
+        "type": "custom",
+        "tokenizer": "standard",
+        "filter": [
+            "lowercase",
+            "asciifolding",
+            "english_stop",
+            "english_stemmer",
+        ],
+    },
+    "german_analyzer": {
+        "type": "custom",
+        "tokenizer": "standard",
+        "filter": [
+            "lowercase",
+            "asciifolding",
+            "german_stop",
+            "german_stemmer",
+        ],
+    },
+    "dutch_analyzer": {
+        "type": "custom",
+        "tokenizer": "standard",
+        "filter": [
+            "lowercase",
+            "asciifolding",
+            "dutch_stop",
+            "dutch_stemmer",
+        ],
+    },
+    "undetermined_language_analyzer": {
+        "type": "custom",
+        "tokenizer": "standard",
+        "filter": [
+            "lowercase",
+            "asciifolding",
+        ],
+    },
+    "trigram_analyzer": {
+        "type": "custom",
+        "tokenizer": "standard",
+        "filter": [
+            "lowercase",
+            "asciifolding",
+            "trigram_filter",
+        ],
+    },
+}
+
+FILTERS = {
+    "french_elision": {
+        "type": "elision",
+        "articles_case": True,
+        "articles": [
+            "l",
+            "m",
+            "t",
+            "qu",
+            "n",
+            "s",
+            "j",
+            "d",
+            "c",
+            "jusqu",
+            "quoiqu",
+            "lorsqu",
+            "puisqu",
+        ],
+    },
+    "french_stop": {
+        "type": "stop",
+        "stopwords": "_french_",
+    },
+    "french_stemmer": {
+        "type": "stemmer",
+        "language": "light_french",
+    },
+    "english_stop": {
+        "type": "stop",
+        "stopwords": "_english_",
+    },
+    "english_stemmer": {
+        "type": "stemmer",
+        "language": "english",
+    },
+    "german_stop": {
+        "type": "stop",
+        "stopwords": "_german_",
+    },
+    "german_stemmer": {
+        "type": "stemmer",
+        "language": "light_german",
+    },
+    "dutch_stop": {
+        "type": "stop",
+        "stopwords": "_dutch_",
+    },
+    "dutch_stemmer": {
+        "type": "stemmer",
+        "language": "dutch",
+    },
+    "trigram_filter": {
+        "type": "ngram",
+        "min_gram": 3,
+        "max_gram": 3,
+    },
+}
+
+MAPPINGS = {
+    "dynamic": "strict",
+    "properties": {
+        "id": {"type": "keyword"},
+        # French
+        "title.fr": {
+            "type": "keyword",
+            "fields": {
+                "text": {
+                    "type": "text",
+                    "analyzer": "french_analyzer",
+                    "fields": {
+                        "trigrams": {
+                            "type": "text",
+                            "analyzer": "trigram_analyzer",
+                        }
+                    },
+                }
+            },
+        },
+        "content.fr": {
+            "type": "text",
+            "analyzer": "french_analyzer",
+            "fields": {
+                "trigrams": {
+                    "type": "text",
+                    "analyzer": "trigram_analyzer",
+                }
+            },
+        },
+        # English
+        "title.en": {
+            "type": "keyword",
+            "fields": {
+                "text": {
+                    "type": "text",
+                    "analyzer": "english_analyzer",
+                    "fields": {
+                        "trigrams": {
+                            "type": "text",
+                            "analyzer": "trigram_analyzer",
+                        }
+                    },
+                }
+            },
+        },
+        "content.en": {
+            "type": "text",
+            "analyzer": "english_analyzer",
+            "fields": {
+                "trigrams": {
+                    "type": "text",
+                    "analyzer": "trigram_analyzer",
+                }
+            },
+        },
+        # German
+        "title.de": {
+            "type": "keyword",
+            "fields": {
+                "text": {
+                    "type": "text",
+                    "analyzer": "german_analyzer",
+                    "fields": {
+                        "trigrams": {
+                            "type": "text",
+                            "analyzer": "trigram_analyzer",
+                        }
+                    },
+                }
+            },
+        },
+        "content.de": {
+            "type": "text",
+            "analyzer": "german_analyzer",
+            "fields": {
+                "trigrams": {
+                    "type": "text",
+                    "analyzer": "trigram_analyzer",
+                }
+            },
+        },
+        # Dutch
+        "title.nl": {
+            "type": "keyword",
+            "fields": {
+                "text": {
+                    "type": "text",
+                    "analyzer": "dutch_analyzer",
+                    "fields": {
+                        "trigrams": {
+                            "type": "text",
+                            "analyzer": "trigram_analyzer",
+                        }
+                    },
+                }
+            },
+        },
+        "content.nl": {
+            "type": "text",
+            "analyzer": "dutch_analyzer",
+            "fields": {
+                "trigrams": {
+                    "type": "text",
+                    "analyzer": "trigram_analyzer",
+                }
+            },
+        },
+        # Undetermined language
+        "title.und": {
+            "type": "keyword",
+            "fields": {
+                "text": {
+                    "type": "text",
+                    "analyzer": "undetermined_language_analyzer",
+                    "fields": {
+                        "trigrams": {
+                            "type": "text",
+                            "analyzer": "trigram_analyzer",
+                        }
+                    },
+                }
+            },
+        },
+        "content.und": {
+            "type": "text",
+            "analyzer": "undetermined_language_analyzer",
+            "fields": {
+                "trigrams": {
+                    "type": "text",
+                    "analyzer": "trigram_analyzer",
+                }
+            },
+        },
+        "depth": {"type": "integer"},
+        "path": {
+            "type": "keyword",
+            "fields": {"text": {"type": "text"}},
+        },
+        "numchild": {"type": "integer"},
+        "created_at": {"type": "date"},
+        "updated_at": {"type": "date"},
+        "size": {"type": "long"},
+        "users": {"type": "keyword"},
+        "groups": {"type": "keyword"},
+        "reach": {"type": "keyword"},
+        "tags": {"type": "keyword"},
+        "is_active": {"type": "boolean"},
+        "chunks": {
+            "type": "nested",
+            "properties": {
+                "index": {"type": "integer"},
+                "content": {"type": "text"},
+                "embedding": {
+                    "type": "knn_vector",
+                    "dimension": settings.EMBEDDING_DIMENSION,
+                    "method": {
+                        "engine": "lucene",
+                        "space_type": "l2",
+                        "name": "hnsw",
+                        "parameters": {},
+                    },
+                },
+            },
+        },
+        "embedding_model": {"type": "keyword"},
+    },
+}

src/backend/core/services/search.py

@@ -0,0 +1,247 @@
+"""OpenSearch search utilities."""
+
+import logging
+
+from django.conf import settings
+
+from core import enums
+from core.enums import SearchTypeEnum
+
+from .embedding import embed_text
+from .opensearch import check_hybrid_search_enabled, opensearch_client
+
+logger = logging.getLogger(__name__)
+
+
+# pylint: disable=too-many-arguments, too-many-positional-arguments
+def search(  # noqa : PLR0913
+    q,
+    nb_results,
+    order_by,
+    order_direction,
+    search_indices,
+    reach,
+    visited,
+    user_sub,
+    groups,
+    tags,
+    search_type,
+    path=None,
+):
+    """Perform an OpenSearch search"""
+    query = get_query(
+        q=q,
+        nb_results=nb_results,
+        reach=reach,
+        visited=visited,
+        user_sub=user_sub,
+        groups=groups,
+        tags=tags,
+        path=path,
+        search_type=search_type,
+    )
+    return opensearch_client().search(  # pylint: disable=unexpected-keyword-arg
+        index=",".join(search_indices),
+        body={
+            "_source": enums.SOURCE_FIELDS,  # limit the fields to return
+            "script_fields": {
+                "number_of_users": {"script": {"source": "doc['users'].size()"}},
+                "number_of_groups": {"script": {"source": "doc['groups'].size()"}},
+            },
+            "sort": get_sort(
+                query_keys=query.keys(),
+                order_by=order_by,
+                order_direction=order_direction,
+            ),
+            "size": nb_results,
+            "query": query,
+        },
+        params=get_params(query_keys=query.keys()),
+        # disable=unexpected-keyword-arg because
+        # ignore_unavailable is not in the method declaration
+        ignore_unavailable=True,
+    )
+
+
+# pylint: disable=too-many-arguments, too-many-positional-arguments
+def get_query(  # noqa : PLR0913
+    q,
+    nb_results,
+    reach,
+    visited,
+    user_sub,
+    groups,
+    tags,
+    search_type,
+    path=None,
+):
+    """Build OpenSearch query body based on parameters"""
+    filter_ = get_filter(reach, visited, user_sub, groups, tags, path)
+
+    if q == "*":
+        logger.info("Performing match_all query")
+        return {
+            "bool": {
+                "must": {"match_all": {}},
+                "filter": {"bool": {"filter": filter_}},
+            },
+        }
+
+    q_vector = vectorize_query(q, search_type)
+
+    if not q_vector:
+        logger.info("Performing full-text search without embedding: %s", q)
+        return get_full_text_query(q, filter_)
+
+    logger.info("Performing hybrid search with embedding: %s", q)
+    return {
+        "hybrid": {
+            "queries": [
+                get_full_text_query(q, filter_),
+                get_semantic_search_query(q_vector, filter_, nb_results),
+            ],
+        }
+    }
+
+
+def vectorize_query(q, search_type):
+    """Vectorize the query if hybrid search is enabled and requested"""
+    hybrid_search_enabled = check_hybrid_search_enabled()
+
+    if search_type == SearchTypeEnum.HYBRID:
+        if not hybrid_search_enabled:
+            logger.warning(
+                "Hybrid search was requested (search_type=hybrid) but is disabled on server",
+            )
+            return None
+
+        return embed_text(q)
+
+    return None
+
+
+def get_semantic_search_query(q_vector, filter_, nb_results):
+    """Build OpenSearch semantic search query"""
+    return {
+        "bool": {
+            "must": {
+                "nested": {
+                    "path": "chunks",
+                    "score_mode": "max",
+                    "query": {
+                        "knn": {
+                            "chunks.embedding": {
+                                "vector": q_vector,
+                                "k": nb_results,
+                            }
+                        }
+                    },
+                }
+            },
+            "filter": filter_,
+        }
+    }
+
+
+def get_full_text_query(q, filter_):
+    """Build OpenSearch full-text query"""
+    return {
+        "bool": {
+            "must": {
+                "bool": {
+                    "should": [
+                        {
+                            "multi_match": {
+                                "query": q,
+                                "fields": [
+                                    "title.*.text^3",
+                                    "content.*",
+                                ],
+                            }
+                        },
+                        {
+                            "multi_match": {
+                                "query": q,
+                                "fields": [
+                                    "title.*.text.trigrams^3",
+                                    "content.*.trigrams",
+                                ],
+                                "boost": settings.TRIGRAMS_BOOST,
+                                "minimum_should_match": settings.TRIGRAMS_MINIMUM_SHOULD_MATCH,
+                            }
+                        },
+                    ],
+                    "minimum_should_match": 1,
+                },
+            },
+            "filter": filter_,
+        }
+    }
+
+
+# pylint: disable=too-many-arguments, too-many-positional-arguments
+def get_filter(  # noqa : PLR0913
+    reach, visited, user_sub, groups, tags, path=None
+):
+    """Build OpenSearch filter"""
+    filters = [
+        {"term": {"is_active": True}},  # filter out inactive documents
+        # Access control filters
+        {
+            "bool": {
+                "should": [
+                    # Public or authenticated (not restricted)
+                    {
+                        "bool": {
+                            "must_not": {
+                                "term": {enums.REACH: enums.ReachEnum.RESTRICTED},
+                            },
+                            "must": {
+                                "terms": {"_id": sorted(visited)},
+                            },
+                        }
+                    },
+                    # Restricted: either user or group must match
+                    {"term": {enums.USERS: user_sub}},
+                    {"terms": {enums.GROUPS: groups}},
+                ],
+                "minimum_should_match": 1,
+            }
+        },
+    ]
+
+    # Optional reach filter
+    if reach is not None:
+        filters.append({"term": {enums.REACH: reach}})
+
+    # Optional tags filter
+    if tags:
+        # logical or: if tags are provided the matching documents should have at least one of them
+        filters.append({"terms": {"tags": tags}})
+
+    # Optional path filter
+    if path:
+        # filter documents that start with the provided path
+        filters.append({"prefix": {"path": path}})
+
+    return filters
+
+
+def get_sort(query_keys, order_by, order_direction):
+    """Build OpenSearch sort clause"""
+    # Add sorting logic based on relevance or specified field
+    if "hybrid" in query_keys:
+        # sorting by other field than "_score" is not supported in hybrid search
+        # see: https://github.com/opensearch-project/neural-search/issues/866
+        return {"_score": {"order": order_direction}}
+    if order_by == enums.RELEVANCE:
+        return {"_score": {"order": order_direction}}
+
+    return {order_by: {"order": order_direction}}
+
+
+def get_params(query_keys):
+    """Build OpenSearch search parameters"""
+    if "hybrid" in query_keys:
+        return {"search_pipeline": settings.HYBRID_SEARCH_PIPELINE_ID}
+    return {}

src/backend/core/templates/admin/core/services/change_list.html

@@ -0,0 +1,13 @@
+{% extends "admin/change_list.html" %}
+
+{% load i18n admin_urls %}
+
+{% block object-tools-items %}
+  <li>
+    <a href="{% url 'admin:core_service_ensure_search_pipeline' %}">
+      {% trans "Ensure search pipeline" %}
+    </a>
+  </li>
+  {{ block.super }}
+{% endblock %}
+

src/backend/core/templates/admin/index.html

@@ -0,0 +1,26 @@
+{% extends "admin/index.html" %}
+{% load i18n %}
+
+{% block sidebar %}
+{{ block.super }}
+
+<div id="content-main">
+    <div class="module" style="margin-top: 20px;">
+        <table>
+            <caption>
+                {% trans 'System Tools' %}
+            </caption>
+            <tbody>
+                <tr>
+                    <th scope="row">
+                        <a href="{% url 'admin:selftest' %}">{% trans 'System Self-Tests' %}</a>
+                    </th>
+                    <td>{% trans 'Run selftest checks' %}</td>
+                </tr>
+            </tbody>
+        </table>
+    </div>
+</div>
+
+{% endblock %}
+

src/backend/core/templates/admin/selftest.html

@@ -0,0 +1,258 @@
+{% extends "admin/base_site.html" %}
+{% load i18n static %}
+
+{% block extrastyle %}
+    {{ block.super }}
+    <style>
+        .selftest-container {
+            margin: 20px;
+        }
+
+        .selftest-module {
+            background: var(--body-bg);
+            border: 1px solid var(--border-color, #ddd);
+            border-radius: 4px;
+            margin-bottom: 20px;
+        }
+
+        .selftest-module h2 {
+            background: var(--primary);
+            color: var(--primary-fg);
+            padding: 10px 15px;
+            margin: 0;
+            font-size: 14px;
+            font-weight: normal;
+        }
+
+        .module-content {
+            padding: 15px;
+        }
+
+        .run-tests-btn {
+            display: inline-block;
+            padding: 10px 20px;
+            background: var(--primary);
+            color: var(--primary-fg);
+            text-decoration: none;
+            border-radius: 4px;
+            font-weight: bold;
+            margin-bottom: 20px;
+            border: none;
+            cursor: pointer;
+        }
+
+        .run-tests-btn:hover {
+            background: var(--primary-hover, #205067);
+        }
+
+        .overall-status {
+            padding: 15px;
+            border-radius: 4px;
+            font-weight: bold;
+            text-align: center;
+        }
+
+        .overall-status.success {
+            background-color: var(--success-bg, #d4edda);
+            color: var(--success-fg, #155724);
+            border: 2px solid var(--success-border, #c3e6cb);
+        }
+
+        .overall-status.failure {
+            background-color: var(--error-bg, #f8d7da);
+            color: var(--error-fg, #721c24);
+            border: 2px solid var(--error-border, #f5c6cb);
+        }
+
+        .test-item {
+            border: 1px solid var(--border-color, #ddd);
+            border-radius: 4px;
+            margin-bottom: 15px;
+            overflow: hidden;
+            background: var(--body-bg);
+        }
+
+        .test-header {
+            padding: 12px 15px;
+            background-color: var(--darkened-bg);
+            display: flex;
+            justify-content: space-between;
+            align-items: center;
+            border-bottom: 1px solid var(--border-color, #ddd);
+        }
+
+        .test-name {
+            font-weight: bold;
+            font-size: 14px;
+            color: var(--body-fg);
+        }
+
+        .test-status {
+            padding: 4px 12px;
+            border-radius: 3px;
+            font-size: 11px;
+            font-weight: bold;
+            text-transform: uppercase;
+        }
+
+        .test-status.success {
+            background-color: #28a745;
+            color: white;
+        }
+
+        .test-status.failure {
+            background-color: #dc3545;
+            color: white;
+        }
+
+        .test-body {
+            padding: 15px;
+            background-color: var(--body-bg);
+        }
+
+        .test-message {
+            margin-bottom: 10px;
+            color: var(--body-fg);
+        }
+
+        .test-details {
+            background-color: var(--darkened-bg);
+            padding: 10px;
+            border-radius: 4px;
+            font-family: "Courier New", Monaco, monospace;
+            font-size: 12px;
+            border: 1px solid var(--border-color, #ddd);
+        }
+
+        .test-details dl {
+            margin: 0;
+        }
+
+        .test-details dt {
+            font-weight: bold;
+            color: var(--body-fg);
+            margin-top: 5px;
+        }
+
+        .test-details dd {
+            margin-left: 20px;
+            color: var(--body-quiet-color);
+            margin-bottom: 5px;
+        }
+
+        .duration {
+            color: var(--body-quiet-color);
+            font-size: 11px;
+            font-style: italic;
+        }
+
+        .available-tests-list {
+            list-style: none;
+            padding: 0;
+            margin: 0;
+        }
+
+        .available-tests-list li {
+            padding: 10px;
+            border-bottom: 1px solid var(--hairline-color, #eee);
+            list-style-type: none;
+        }
+
+        .available-tests-list li:last-child {
+            border-bottom: none;
+        }
+
+        .test-description {
+            color: var(--body-quiet-color);
+            font-size: 13px;
+            display: block;
+            margin-top: 4px;
+        }
+    </style>
+{% endblock %}
+
+{% block breadcrumbs %}
+<div class="breadcrumbs">
+    <a href="{% url 'admin:index' %}">{% trans 'Home' %}</a>
+    &rsaquo; {% trans 'System Self-Tests' %}
+</div>
+{% endblock %}
+
+{% block content %}
+<div class="selftest-container">
+    {% if not run_tests %}
+        <a href="?run=true" class="run-tests-btn">{% trans "Run All Tests" %}</a>
+    {% else %}
+        <a href="{% url 'admin:selftest' %}" class="run-tests-btn" style="opacity: 0.8;">{% trans "Reset" %}</a>
+    {% endif %}
+
+    {% if run_tests %}
+        {% if all_passed is not None %}
+        <div class="selftest-module">
+            <div class="overall-status {% if all_passed %}success{% else %}failure{% endif %}">
+                {% if all_passed %}
+                    ✅ {% trans "All tests passed successfully!" %}
+                {% else %}
+                    ❌ {% trans "Some tests failed. Please check the details below." %}
+                {% endif %}
+            </div>
+        </div>
+        {% endif %}
+
+        <div class="selftest-module">
+            <h2>{% trans "Test Results" %}</h2>
+            <div class="module-content">
+                {% for result in results %}
+                    <div class="test-item">
+                        <div class="test-header">
+                            <span class="test-name">{{ result.name }}</span>
+                            <span class="test-status {% if result.success %}success{% else %}failure{% endif %}">
+                                {% if result.success %}
+                                    ✓ {% trans "Pass" %}
+                                {% else %}
+                                    ✗ {% trans "Fail" %}
+                                {% endif %}
+                            </span>
+                        </div>
+                        <div class="test-body">
+                            <div class="test-message">
+                                {{ result.message }}
+                                {% if result.duration_ms %}
+                                    <span class="duration">({{ result.duration_ms|floatformat:2 }}ms)</span>
+                                {% endif %}
+                            </div>
+                            {% if result.details %}
+                                <div class="test-details">
+                                    <strong>{% trans "Details:" %}</strong>
+                                    <dl>
+                                        {% for key, value in result.details.items %}
+                                            <dt>{{ key }}:</dt>
+                                            <dd>{{ value }}</dd>
+                                        {% endfor %}
+                                    </dl>
+                                </div>
+                            {% endif %}
+                        </div>
+                    </div>
+                {% endfor %}
+            </div>
+        </div>
+    {% else %}
+        <div class="selftest-module">
+            <h2>{% trans "Available Tests" %}</h2>
+            <div class="module-content">
+                <p>{% trans "Click 'Run All Tests' to execute the following system health checks:" %}</p>
+                <ul class="available-tests-list">
+                    {% for test in available_tests %}
+                        <li>
+                            <strong>{{ test.name }}</strong>
+                            <span class="test-description">{{ test.description }}</span>
+                        </li>
+                    {% endfor %}
+                </ul>
+            </div>
+        </div>
+    {% endif %}
+</div>
+{% endblock %}
+

src/backend/core/tests/commands/test_create_search_pipeline.py

@@ -0,0 +1,76 @@
+"""
+Unit test for `create_search_pipeline` command.
+"""
+
+import logging
+
+from django.core.management import call_command
+
+import pytest
+
+from core.services.opensearch import opensearch_client
+from core.tests.utils import (
+    enable_hybrid_search,
+)
+from core.utils import delete_search_pipeline
+
+
+@pytest.fixture(autouse=True)
+def before_each():
+    """Delete search pipeline before each test"""
+    delete_search_pipeline()
+    yield
+    delete_search_pipeline()
+
+
+def test_create_search_pipeline(settings, caplog):
+    """Test command create search pipeline"""
+
+    enable_hybrid_search(settings)
+
+    with caplog.at_level(logging.INFO):
+        call_command("create_search_pipeline")
+
+    assert any(
+        f"Creating search pipeline: {settings.HYBRID_SEARCH_PIPELINE_ID}" in message
+        for message in caplog.messages
+    )
+
+    # calling get works without raising NotFoundError
+    opensearch_client().search_pipeline.get(id=settings.HYBRID_SEARCH_PIPELINE_ID)
+
+
+def test_create_search_pipeline_but_it_exists_already(settings, caplog):
+    """Test command create search pipeline but it already exists"""
+
+    opensearch_client().transport.perform_request(
+        method="PUT",
+        url="/_search/pipeline/" + settings.HYBRID_SEARCH_PIPELINE_ID,
+        body={
+            "description": "Post processor for hybrid search",
+            "phase_results_processors": [
+                {
+                    "normalization-processor": {
+                        "combination": {
+                            "technique": "arithmetic_mean",
+                            "parameters": {"weights": settings.HYBRID_SEARCH_WEIGHTS},
+                        }
+                    }
+                }
+            ],
+        },
+    )
+
+    with caplog.at_level(logging.INFO):
+        call_command("create_search_pipeline")
+
+    assert any(
+        "Search pipeline exists already" in message for message in caplog.messages
+    )
+    assert not any(
+        f"Creating search pipeline: {settings.HYBRID_SEARCH_PIPELINE_ID}" in message
+        for message in caplog.messages
+    )
+
+    # the pipeline is still here
+    opensearch_client().search_pipeline.get(id=settings.HYBRID_SEARCH_PIPELINE_ID)

src/backend/core/tests/commands/test_reindex_with_embedding.py

@@ -0,0 +1,291 @@
+"""
+Unit test for `reindex_with_embedding` command.
+"""
+
+from unittest.mock import patch
+
+from django.core.management import CommandError, call_command
+
+import pytest
+import responses
+
+from core.management.commands.reindex_with_embedding import (
+    check_hybrid_search_enabled as check_hybrid_search_enabled_command,
+)
+from core.management.commands.reindex_with_embedding import (
+    reindex_with_embedding,
+)
+from core.models import get_opensearch_index_name
+from core.services.opensearch import check_hybrid_search_enabled, opensearch_client
+from core.tests.mock import albert_embedding_response
+from core.tests.utils import (
+    enable_hybrid_search,
+)
+from core.utils import (
+    bulk_create_documents,
+    delete_search_pipeline,
+    get_language_value,
+    prepare_index,
+)
+
+SERVICE_NAME = "test-index"
+
+
+@pytest.fixture(autouse=True)
+def before_each():
+    """Clear caches and delete search pipeline before each test"""
+    clear_caches()
+    yield
+    clear_caches()
+
+
+def clear_caches():
+    """Clear caches used in opensearch service and factories"""
+    check_hybrid_search_enabled.cache_clear()
+    # the instance of check_hybrid_search_enabled used in utils.py
+    # is different and must be cleared separately
+    check_hybrid_search_enabled_command.cache_clear()
+    delete_search_pipeline()
+
+
+@responses.activate
+def test_reindex_with_embedding_command(settings):
+    """Test command create indexes with embedding and search pipeline"""
+    # create documents and index them with hybrid search disabled
+    opensearch_client_ = opensearch_client()
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    index_name = get_opensearch_index_name(SERVICE_NAME)
+    prepare_index(index_name, documents)
+
+    # the index has not been embedded in the initial state
+    initial_index = opensearch_client_.search(  # pylint: disable=unexpected-keyword-arg
+        index=index_name, size=3, body={"query": {"match_all": {}}}
+    )
+    assert len(initial_index["hits"]["hits"]) == 3
+    for embedded_hit in initial_index["hits"]["hits"]:
+        assert embedded_hit["_source"]["chunks"] is None
+        assert embedded_hit["_source"]["embedding_model"] is None
+
+    # enable hybrid search
+    enable_hybrid_search(settings)
+    check_hybrid_search_enabled_command.cache_clear()
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+
+    call_command("reindex_with_embedding", SERVICE_NAME)
+
+    opensearch_client_.indices.refresh(index=index_name)
+    embedded_index = opensearch_client_.search(  # pylint: disable=unexpected-keyword-arg
+        index=index_name, size=3, body={"query": {"match_all": {}}}
+    )
+
+    # the source index has been replaced with embedding version
+    assert len(embedded_index["hits"]["hits"]) == 3
+    for embedded_hit in embedded_index["hits"]["hits"]:
+        embedded_source = embedded_hit["_source"]
+        # the index contains a embedding and embedding_model
+        assert len(embedded_source["chunks"]) == 1
+        assert (
+            embedded_source["chunks"][0]["embedding"]
+            == albert_embedding_response.response["data"][0]["embedding"]
+        )
+        assert embedded_source["embedding_model"] == settings.EMBEDDING_API_MODEL_NAME
+        # assert initial value have not been effected
+        initial_hits = [
+            hit_
+            for hit_ in initial_index["hits"]["hits"]
+            if hit_["_id"] == embedded_hit["_id"]
+        ]
+        assert len(initial_hits) == 1
+        initial_source = initial_hits[0]["_source"]
+        assert initial_source["title.en"] == embedded_source["title.en"]
+        assert initial_source["content.en"] == embedded_source["content.en"]
+        assert initial_source["created_at"] == embedded_source["created_at"]
+        assert initial_source["users"] == embedded_source["users"]
+
+
+@responses.activate
+def test_reindex_can_fail_and_restart(settings):
+    """Test command handles embedding errors gracefully and continues processing."""
+    opensearch_client_ = opensearch_client()
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    index_name = get_opensearch_index_name(SERVICE_NAME)
+    prepare_index(index_name, documents)
+
+    # enable hybrid search after first indexing
+    enable_hybrid_search(settings)
+    check_hybrid_search_enabled_command.cache_clear()
+
+    # First call succeeds, second fails, third succeeds
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json={"error": "rate limit exceeded"},
+        status=429,
+    )
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+
+    result = reindex_with_embedding(index_name)
+
+    # assert results reflect 2 successes and 1 failure
+    assert result["nb_success_embedding"] == 2
+    assert result["nb_failed_embedding"] == 1
+
+    # assert the index state
+    opensearch_client_.indices.refresh(index=index_name)
+    embedded_index = opensearch_client_.search(  # pylint: disable=unexpected-keyword-arg
+        index=index_name, size=3, body={"query": {"match_all": {}}}
+    )
+    # Should have 2 documents with embeddings, 1 without due to error
+    embedded_count = 0
+    not_embedded_count = 0
+    for hit in embedded_index["hits"]["hits"]:
+        if hit["_source"].get("chunks"):
+            embedded_count += 1
+            assert (
+                hit["_source"]["embedding_model"] == settings.EMBEDDING_API_MODEL_NAME
+            )
+        else:
+            not_embedded_count += 1
+            assert hit["_source"]["embedding_model"] is None
+    assert embedded_count == 2
+    assert not_embedded_count == 1
+
+    # the command can be run again to index failed items
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    result = reindex_with_embedding(index_name)
+
+    # assert results
+    assert result["nb_success_embedding"] == 1
+    assert result["nb_failed_embedding"] == 0
+
+    # assert there is now 1 more success and 0 failures
+    opensearch_client_.indices.refresh(index=index_name)
+    embedded_index = opensearch_client_.search(  # pylint: disable=unexpected-keyword-arg
+        index=index_name, size=3, body={"query": {"match_all": {}}}
+    )
+    for hit in embedded_index["hits"]["hits"]:
+        for chunk in hit["_source"]["chunks"]:
+            assert (
+                chunk["embedding"]
+                == albert_embedding_response.response["data"][0]["embedding"]
+            )
+        assert hit["_source"]["embedding_model"] == settings.EMBEDDING_API_MODEL_NAME
+
+
+@responses.activate
+def test_reindex_preserves_concurrent_updates(settings):
+    """
+    Test that concurrent document updates don't get overwritten by reindexing.
+    This test simulates the following scenario:
+    • the hybrid search is disabled
+    • documents are created and indexed without indexing
+    • the hybrid search is enabled
+    • the reindexing is triggered
+    • one document is updated while the reindexing is still running
+    Because the updated document is modified after the hybrid search is enabled,
+    it has properly been indexed with embedding, the reindexing command must
+    ignore this document to preserve this latest update.
+    """
+    opensearch_client_ = opensearch_client()
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+        ]
+    )
+    index_name = get_opensearch_index_name(SERVICE_NAME)
+    prepare_index(index_name, documents)
+    enable_hybrid_search(settings)
+
+    updated_title = "updated dog"
+    patch(
+        "core.services.opensearch.opensearch_client_.search",
+        side_effect=opensearch_client_.update(
+            index=index_name,
+            id=documents[1]["id"],
+            body={
+                "doc": {
+                    "title.en": updated_title,
+                }
+            },
+        ),
+    )
+
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    result = reindex_with_embedding(index_name)
+    assert result["nb_success_embedding"] == 2
+    assert result["nb_failed_embedding"] == 0
+
+    opensearch_client_.indices.refresh(index=index_name)
+    embedded_index = opensearch_client_.search(  # pylint: disable=unexpected-keyword-arg
+        index=index_name, size=2, body={"query": {"match_all": {}}}
+    )
+    # Check that the latest update is preserved
+    updated_document = [
+        hit
+        for hit in embedded_index["hits"]["hits"]
+        if get_language_value(hit["_source"], "title") == updated_title
+    ]
+    assert len(updated_document) == 1
+    # Check it was not embedded
+    assert updated_document[0]["_source"]["chunks"] is None
+    assert updated_document[0]["_source"]["embedding_model"] is None
+
+
+def test_reindex_command_but_hybrid_search_is_disabled():
+    """Test the `reindex_with_embedding` command fails when hybrid search is disabled."""
+    with pytest.raises(CommandError) as err:
+        call_command("reindex_with_embedding", SERVICE_NAME)
+
+    assert str(err.value) == "Hybrid search is not enabled or properly configured."
+
+
+def test_reindex_command_but_index_does_not_exist(settings):
+    """Test the `reindex_with_embedding` command fails when the index does not exist."""
+    wrong_index = "wrong-index-name"
+    enable_hybrid_search(settings)
+
+    wrong_index_name = get_opensearch_index_name(wrong_index)
+
+    with pytest.raises(CommandError) as err:
+        call_command("reindex_with_embedding", wrong_index)
+
+    assert str(err.value) == f"Index {wrong_index_name} does not exist."

src/backend/core/tests/conftest.py

@@ -0,0 +1,52 @@
+"""Fixtures for tests in the find core application"""
+
+import pytest
+from faker import Faker
+from lasuite.oidc_resource_server.authentication import (
+    get_resource_server_backend,
+)
+from opensearchpy.exceptions import NotFoundError
+
+from core.services import opensearch
+
+fake = Faker()
+
+
+@pytest.fixture(name="jwt_rs_backend")
+def jwt_resource_server_backend_fixture(settings):
+    """Fixture to switch the backend to the JWTResourceServerBackend."""
+    _original_backend = str(settings.OIDC_RS_BACKEND_CLASS)
+
+    settings.OIDC_RS_BACKEND_CLASS = (
+        "lasuite.oidc_resource_server.backend.JWTResourceServerBackend"
+    )
+    get_resource_server_backend.cache_clear()
+
+    yield
+
+    settings.OIDC_RS_BACKEND_CLASS = _original_backend
+    get_resource_server_backend.cache_clear()
+
+
+@pytest.fixture(autouse=True)
+def cleanup_test_index(settings):
+    """
+    Fixture to set a randomized prefix for all service indexes within the tests
+    and remove them on tear down.
+    """
+    _original_prefix = settings.OPENSEARCH_INDEX_PREFIX
+    prefix = "".join(fake.random_letters(5)).lower()
+    settings.OPENSEARCH_INDEX_PREFIX = prefix
+
+    # Create client here to prevent "teardown" issues when the opensearch settings are
+    # removed for error tests.
+    client = opensearch.opensearch_client()
+
+    yield
+
+    settings.OPENSEARCH_INDEX_PREFIX = _original_prefix
+
+    try:
+        client.indices.delete(index=f"{prefix}-*")
+    except NotFoundError:
+        pass

src/backend/core/tests/test_admin_selftest.py

@@ -0,0 +1,181 @@
+"""Tests for the admin selftest view."""
+
+from unittest.mock import patch
+
+from django.contrib.auth import get_user_model
+from django.urls import reverse
+
+import pytest
+
+from core.selftests import SelfTestResult
+
+pytestmark = pytest.mark.django_db
+
+User = get_user_model()
+
+
+@pytest.fixture(autouse=True)
+def _override_storage_settings(settings):
+    """Override storage settings for all tests."""
+    settings.STORAGES = {
+        "default": {
+            "BACKEND": "django.core.files.storage.FileSystemStorage",
+        },
+        "staticfiles": {
+            "BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage",
+        },
+    }
+
+
+def test_selftest_requires_authentication(client):
+    """Test that the selftest page requires authentication."""
+    selftest_url = reverse("admin:selftest")
+
+    response = client.get(selftest_url)
+
+    # Should redirect to login
+    assert response.status_code == 302
+    assert "/admin/login/" in response.url
+
+
+def test_selftest_requires_staff_permission(client):
+    """Test that only staff users can access the selftest page."""
+    selftest_url = reverse("admin:selftest")
+
+    User.objects.create_user(
+        username="user",
+        email="user@example.com",
+        password="user123",
+    )
+    client.login(username="user", password="user123")
+
+    response = client.get(selftest_url)
+
+    # Regular users should be redirected
+    assert response.status_code == 302
+
+
+def test_selftest_accessible_by_admin(client):
+    """Test that admin users can access the selftest page."""
+    selftest_url = reverse("admin:selftest")
+
+    User.objects.create_superuser(
+        username="admin",
+        email="admin@example.com",
+        password="admin123",
+    )
+    client.login(username="admin", password="admin123")
+
+    response = client.get(selftest_url)
+
+    assert response.status_code == 200
+    assert b"System Self-Tests" in response.content
+
+
+def test_selftest_displays_available_tests(client):
+    """Test that available tests are displayed when not running."""
+    selftest_url = reverse("admin:selftest")
+
+    User.objects.create_superuser(
+        username="admin",
+        email="admin@example.com",
+        password="admin123",
+    )
+    client.login(username="admin", password="admin123")
+
+    response = client.get(selftest_url)
+
+    assert response.status_code == 200
+    assert b"Available Tests" in response.content
+    assert b"Run All Tests" in response.content
+
+
+@patch("core.selftests.registry.run_all")
+def test_selftest_runs_tests(mock_run_all, client):
+    """Test that tests are executed when run=true."""
+    selftest_url = reverse("admin:selftest")
+
+    # Mock the test results
+    mock_run_all.return_value = [
+        SelfTestResult(
+            name="Test 1",
+            success=True,
+            message="Success",
+            duration_ms=10.0,
+        ),
+        SelfTestResult(
+            name="Test 2",
+            success=False,
+            message="Failed",
+            duration_ms=20.0,
+        ),
+    ]
+
+    User.objects.create_superuser(
+        username="admin",
+        email="admin@example.com",
+        password="admin123",
+    )
+    client.login(username="admin", password="admin123")
+
+    response = client.get(selftest_url, {"run": "true"})
+
+    assert response.status_code == 200
+    assert b"Test Results" in response.content
+    assert b"Test 1" in response.content
+    assert b"Test 2" in response.content
+    mock_run_all.assert_called_once()
+
+
+@patch("core.selftests.registry.run_all")
+def test_selftest_displays_success_status(mock_run_all, client):
+    """Test that success status is displayed correctly."""
+    selftest_url = reverse("admin:selftest")
+
+    mock_run_all.return_value = [
+        SelfTestResult(
+            name="Test 1",
+            success=True,
+            message="Success",
+            duration_ms=10.0,
+        ),
+    ]
+
+    User.objects.create_superuser(
+        username="admin",
+        email="admin@example.com",
+        password="admin123",
+    )
+    client.login(username="admin", password="admin123")
+
+    response = client.get(selftest_url, {"run": "true"})
+
+    assert response.status_code == 200
+    assert b"All tests passed successfully" in response.content
+
+
+@patch("core.selftests.registry.run_all")
+def test_selftest_displays_failure_status(mock_run_all, client):
+    """Test that failure status is displayed correctly."""
+    selftest_url = reverse("admin:selftest")
+
+    mock_run_all.return_value = [
+        SelfTestResult(
+            name="Test 1",
+            success=False,
+            message="Failed",
+            duration_ms=10.0,
+        ),
+    ]
+
+    User.objects.create_superuser(
+        username="admin",
+        email="admin@example.com",
+        password="admin123",
+    )
+    client.login(username="admin", password="admin123")
+
+    response = client.get(selftest_url, {"run": "true"})
+
+    assert response.status_code == 200
+    assert b"Some tests failed" in response.content

src/backend/core/tests/test_api_documents_delete.py

@@ -0,0 +1,384 @@
+"""Tests for deleting documents from OpenSearch over the API"""
+
+import opensearchpy
+import pytest
+import responses
+from rest_framework.test import APIClient
+
+from core import factories
+from core.services.opensearch import opensearch_client
+from core.utils import prepare_index
+
+from .utils import build_authorization_bearer, setup_oicd_resource_server
+
+pytestmark = pytest.mark.django_db
+
+
+def test_api_documents_delete_anonymous():
+    """Anonymous requests should not be allowed to delete documents."""
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {"service": "service-name", "document_ids": ["doc1"]},
+        format="json",
+    )
+
+    assert response.status_code == 401
+
+
+@responses.activate
+def test_api_documents_delete_wrong_service_name(settings):
+    """Requests with a wrong service name should return 400 Bad Request."""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {"service": "wrong-service", "document_ids": ["0"]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 400
+    assert response.json()["detail"] == "Invalid request."
+
+
+@responses.activate
+def test_api_documents_delete_success(settings):
+    """Authenticated users should be able to delete documents they have access to."""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+
+    service = factories.ServiceFactory()
+    # Create documents user has access to
+    documents = factories.DocumentSchemaFactory.build_batch(3, users=["user_sub"])
+    prepare_index(service.index_name, documents)
+    document_to_delete_ids = [doc["id"] for doc in documents[:2]]
+
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {"service": service.name, "document_ids": document_to_delete_ids},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert response.json()["nb-deleted-documents"] == 2
+    assert response.json()["undeleted-document-ids"] == []
+
+    opensearch_client_ = opensearch_client()
+    for document in documents:
+        if document["id"] in document_to_delete_ids:
+            with pytest.raises(opensearchpy.exceptions.NotFoundError):
+                opensearch_client_.get(index=service.index_name, id=document["id"])
+        else:
+            doc = opensearch_client_.get(index=service.index_name, id=document["id"])
+            assert doc["found"]
+
+
+@responses.activate
+def test_api_documents_delete_no_access(settings):
+    """Users should not be able to delete documents they don't have access to."""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    service = factories.ServiceFactory()
+    # Create documents where user_sub does NOT have access
+    documents = factories.DocumentSchemaFactory.build_batch(2, users=["other_sub"])
+    prepare_index(service.index_name, documents)
+
+    document_ids = [doc["id"] for doc in documents]
+
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {"service": service.name, "document_ids": document_ids},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert response.json()["nb-deleted-documents"] == 0
+    assert set(response.json()["undeleted-document-ids"]) == set(document_ids)
+
+    # Verify documents not deleted
+    opensearch_client_ = opensearch_client()
+    for doc_id in document_ids:
+        doc = opensearch_client_.get(index=service.index_name, id=doc_id)
+        assert doc["found"]
+
+
+@responses.activate
+def test_api_documents_delete_mixed_access(settings):
+    """Deleting a mix of owned and non-owned documents should only delete owned ones."""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+
+    service = factories.ServiceFactory()
+    # Create documents with different access
+    owned_documents = factories.DocumentSchemaFactory.build_batch(2, users=["user_sub"])
+    other_documents = factories.DocumentSchemaFactory.build_batch(
+        2, users=["other_user"]
+    )
+    prepare_index(service.index_name, owned_documents + other_documents)
+
+    owned_document_ids = [doc["id"] for doc in owned_documents]
+    other_document_ids = [doc["id"] for doc in other_documents]
+    non_existing_document_ids = ["non-existent-1", "non-existent-2"]
+
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {
+            "service": service.name,
+            "document_ids": owned_document_ids
+            + other_document_ids
+            + non_existing_document_ids,
+        },
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert response.json()["nb-deleted-documents"] == 2
+    assert set(response.json()["undeleted-document-ids"]) == set(
+        other_document_ids + non_existing_document_ids
+    )
+
+    # Verify only owned documents are deleted
+    opensearch_client_ = opensearch_client()
+    for document_id in owned_document_ids:
+        with pytest.raises(opensearchpy.exceptions.NotFoundError):
+            opensearch_client_.get(index=service.index_name, id=document_id)
+
+    for document_id in other_document_ids:
+        document = opensearch_client_.get(index=service.index_name, id=document_id)
+        assert document["found"]
+
+
+@responses.activate
+def test_api_documents_delete_invalid_params(settings):
+    """Requests with invalid parameters should return 400 Bad Request."""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    service = factories.ServiceFactory()
+
+    # Missing both document_ids and tags
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {
+            "service": service.name,
+        },
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 400
+    assert (
+        response.json()[0]["msg"]
+        == "Value error, At least one of 'document_ids' or 'tags' must be provided"
+    )
+
+    # Empty document_ids and no tags
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {"service": service.name, "document_ids": []},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 400
+    assert (
+        response.json()[0]["msg"]
+        == "Value error, At least one of 'document_ids' or 'tags' must be provided"
+    )
+
+    # Both empty
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {"service": service.name, "document_ids": [], "tags": []},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 400
+    assert (
+        response.json()[0]["msg"]
+        == "Value error, At least one of 'document_ids' or 'tags' must be provided"
+    )
+
+    # Missing service
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {"document_ids": ["doc1"]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 400
+
+
+@responses.activate
+def test_api_documents_delete_nonexistent_documents(settings):
+    """
+    Deleting non-existent documents should not raise an error
+    and return the list of undeleted ids.
+    """
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    service = factories.ServiceFactory()
+    # Create index but with no documents
+    prepare_index(service.index_name, [])
+
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {"service": service.name, "document_ids": ["non-existent-id"]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert response.json()["nb-deleted-documents"] == 0
+    assert response.json()["undeleted-document-ids"] == ["non-existent-id"]
+
+
+@responses.activate
+def test_api_documents_delete_by_single_tag(settings):
+    """Users should be able to delete documents by tags."""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    service = factories.ServiceFactory()
+
+    document_to_deletes = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["delete-tag", "keep-tag-1"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["delete-tag", "keep-tag-2"]
+        ),
+    ]
+    document_to_keep = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["keep-tag-1", "keep-tag-2"]
+        ),
+        factories.DocumentSchemaFactory.build(users=["user_sub"], tags=["keep-tag-1"]),
+        factories.DocumentSchemaFactory.build(
+            users=["other_user_sub"], tags=["delete-tag"]
+        ),
+    ]
+    prepare_index(service.index_name, document_to_deletes + document_to_keep)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {"service": service.name, "tags": ["delete-tag"]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert response.json()["nb-deleted-documents"] == 2
+    assert response.json()["undeleted-document-ids"] == []
+
+    opensearch_client_ = opensearch_client()
+    for document in document_to_deletes:
+        with pytest.raises(opensearchpy.exceptions.NotFoundError):
+            opensearch_client_.get(index=service.index_name, id=document["id"])
+
+    for document in document_to_keep:
+        doc = opensearch_client_.get(index=service.index_name, id=document["id"])
+        assert doc["found"]
+
+
+@responses.activate
+def test_api_documents_delete_by_multiple_tags(settings):
+    """Users should be able to delete documents matching any of multiple tags."""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    service = factories.ServiceFactory()
+
+    document_to_deletes = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["delete-tag-1", "keep-tag-1"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["delete-tag-1", "delete-tag-2"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["delete-tag-2"]
+        ),
+    ]
+    document_to_keep = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["keep-tag-1", "keep-tag-2"]
+        ),
+        factories.DocumentSchemaFactory.build(users=["user_sub"], tags=["keep-tag-1"]),
+        factories.DocumentSchemaFactory.build(
+            users=["other_user_sub"], tags=["delete-tag-1"]
+        ),
+    ]
+    prepare_index(service.index_name, document_to_deletes + document_to_keep)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {"service": service.name, "tags": ["delete-tag-1", "delete-tag-2"]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert response.json()["nb-deleted-documents"] == 3
+    assert response.json()["undeleted-document-ids"] == []
+
+    opensearch_client_ = opensearch_client()
+    for document in document_to_deletes:
+        with pytest.raises(opensearchpy.exceptions.NotFoundError):
+            opensearch_client_.get(index=service.index_name, id=document["id"])
+
+    for document in document_to_keep:
+        doc = opensearch_client_.get(index=service.index_name, id=document["id"])
+        assert doc["found"]
+
+
+@responses.activate
+def test_api_documents_delete_by_ids_and_tags(settings):
+    """Users should be able to delete documents by both IDs and tags (AND logic)."""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    service = factories.ServiceFactory()
+
+    document_delete_by_tag_and_id = factories.DocumentSchemaFactory.build(
+        users=["user_sub"], tags=["delete-tag"]
+    )
+    document_delete_by_tag_keep_by_id = factories.DocumentSchemaFactory.build(
+        users=["user_sub"], tags=["delete-tag"]
+    )
+    document_keep_by_tag_delete_by_id = factories.DocumentSchemaFactory.build(
+        users=["user_sub"]
+    )
+
+    prepare_index(
+        service.index_name,
+        [
+            document_delete_by_tag_and_id,
+            document_delete_by_tag_keep_by_id,
+            document_keep_by_tag_delete_by_id,
+        ],
+    )
+
+    response = APIClient().post(
+        "/api/v1.0/documents/delete/",
+        {
+            "service": service.name,
+            "document_ids": [
+                document_delete_by_tag_and_id["id"],
+                document_keep_by_tag_delete_by_id["id"],
+            ],
+            "tags": ["delete-tag"],
+        },
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert response.json()["nb-deleted-documents"] == 1
+    assert response.json()["undeleted-document-ids"] == [
+        document_keep_by_tag_delete_by_id["id"]
+    ]
+
+    opensearch_client_ = opensearch_client()
+    with pytest.raises(opensearchpy.exceptions.NotFoundError):
+        opensearch_client_.get(
+            index=service.index_name, id=document_delete_by_tag_and_id["id"]
+        )
+
+    doc = opensearch_client_.get(
+        index=service.index_name, id=document_delete_by_tag_keep_by_id["id"]
+    )
+    assert doc["found"]

src/backend/core/tests/test_api_documents_index_bulk.py

@@ -0,0 +1,432 @@
+"""Tests indexing documents in OpenSearch over the API"""
+
+import datetime
+from unittest import mock
+
+from django.utils import timezone
+
+import pytest
+from opensearchpy import NotFoundError
+from rest_framework.test import APIClient
+
+from core import factories
+from core.services import opensearch
+
+pytestmark = pytest.mark.django_db
+
+
+def test_api_documents_index_bulk_anonymous():
+    """Anonymous requests should not be allowed to index documents in bulk."""
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    response = APIClient().post("/api/v1.0/documents/index/", documents, format="json")
+
+    assert response.status_code == 403
+    assert response.json() == {
+        "detail": "Authentication credentials were not provided."
+    }
+
+
+def test_api_documents_index_bulk_invalid_token():
+    """Requests with invalid tokens should not be allowed to index documents in bulk."""
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        documents,
+        HTTP_AUTHORIZATION="Bearer invalid",
+        format="json",
+    )
+
+    assert response.status_code == 403
+    assert response.json() == {"detail": "Invalid token."}
+
+
+def test_api_documents_index_bulk_success():
+    """A registered service should be able to index documents in bulk with a valid token."""
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        documents,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 201
+    responses = response.json()
+    assert [d["status"] for d in responses] == ["success"] * 3
+
+
+def test_api_documents_index_bulk_ensure_index():
+    """A registered service should be created the opensearch index if needed."""
+    opensearch_client_ = opensearch.opensearch_client()
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    with pytest.raises(NotFoundError):
+        opensearch_client_.indices.get(index=service.index_name)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        documents,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 201
+    responses = response.json()
+    assert len(responses) == 3
+    assert [d["status"] for d in responses] == ["success"] * 3
+
+    # The index has been rebuilt
+    opensearch_client_.indices.get(index=service.index_name)
+
+
+@pytest.mark.parametrize(
+    "field, invalid_value, error_type, error_message",
+    [
+        (
+            "id",
+            "0f9b1c9d-030f-427a-8a0e-6b7c202c5daz",  # invalid UUID b/c contains a z
+            "uuid_parsing",
+            (
+                "Input should be a valid UUID, invalid character: expected an optional "
+                "prefix of `urn:uuid:` followed by [0-9a-fA-F-], found `z` at 36"
+            ),
+        ),
+        ("title", 1, "string_type", "Input should be a valid string"),
+        (
+            "title",
+            "a" * 301,
+            "string_too_long",
+            "String should have at most 300 characters",
+        ),
+        (
+            "depth",
+            -1,
+            "greater_than_equal",
+            "Input should be greater than or equal to 0",
+        ),
+        (
+            "depth",
+            "a",
+            "int_parsing",
+            "Input should be a valid integer, unable to parse string as an integer",
+        ),
+        (
+            "path",
+            "a" * 301,
+            "string_too_long",
+            "String should have at most 300 characters",
+        ),
+        (
+            "numchild",
+            -1,
+            "greater_than_equal",
+            "Input should be greater than or equal to 0",
+        ),
+        (
+            "numchild",
+            "a",
+            "int_parsing",
+            "Input should be a valid integer, unable to parse string as an integer",
+        ),
+        ("content", 1, "string_type", "Input should be a valid string"),
+        (
+            "created_at",
+            "invalid_date",
+            "datetime_from_date_parsing",
+            "Input should be a valid datetime or date, invalid character in year",
+        ),
+        (
+            "updated_at",
+            "invalid_date",
+            "datetime_from_date_parsing",
+            "Input should be a valid datetime or date, invalid character in year",
+        ),
+        (
+            "size",
+            64448894017.3,
+            "int_from_float",
+            "Input should be a valid integer, got a number with a fractional part",
+        ),
+        (
+            "size",
+            "not an integer",
+            "int_parsing",
+            "Input should be a valid integer, unable to parse string as an integer",
+        ),
+        (
+            "users",
+            "33052c8b-3181-4420-aede-f8396fc0f9a1",
+            "list_type",
+            "Input should be a valid list",
+        ),
+        (
+            "users",
+            ["a" * 51],
+            "string_too_long",
+            "String should have at most 50 characters",
+        ),
+        (
+            "groups",
+            ["not_a_slug"],
+            "string_pattern_mismatch",
+            "String should match pattern '^[a-z0-9]+(?:-[a-z0-9]+)*$'",
+        ),
+        (
+            "groups",
+            "not-a-list",
+            "list_type",
+            "Input should be a valid list",
+        ),
+        (
+            "reach",
+            "invalid",
+            "enum",
+            "Input should be 'public', 'authenticated' or 'restricted'",
+        ),
+        (
+            "is_active",
+            "invalid",
+            "bool_parsing",
+            "Input should be a valid boolean, unable to interpret input",
+        ),
+    ],
+)
+def test_api_documents_index_bulk_invalid_document(
+    field, invalid_value, error_type, error_message
+):
+    """Test bulk document indexing with various invalid fields."""
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    # Modify the first document with the invalid value for the specified field
+    documents[0][field] = invalid_value
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        documents,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 400
+    responses = response.json()
+    assert [d["status"] for d in responses] == ["error", "valid", "valid"]
+
+    assert responses[0]["status"] == "error"
+    assert len(responses[0]["errors"]) == 1
+    assert responses[0]["errors"][0]["msg"] == error_message
+    assert responses[0]["errors"][0]["type"] == error_type
+
+
+@pytest.mark.parametrize(
+    "field",
+    [
+        "id",
+        "title",
+        "depth",
+        "path",
+        "numchild",
+        "content",
+        "size",
+        "created_at",
+        "updated_at",
+        "is_active",
+    ],
+)
+def test_api_documents_index_bulk_required(field):
+    """Test bulk document indexing with a required field missing."""
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    del documents[0][field]
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        documents,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 400
+    responses = response.json()
+    assert [d["status"] for d in responses] == ["error", "valid", "valid"]
+
+    assert responses[0]["status"] == "error"
+    assert len(responses[0]["errors"]) == 1
+    assert responses[0]["errors"][0]["msg"] == "Field required"
+    assert responses[0]["errors"][0]["type"] == "missing"
+
+
+@pytest.mark.parametrize(
+    "field,default_value",
+    [
+        ("users", []),
+        ("groups", []),
+        ("reach", "restricted"),
+    ],
+)
+def test_api_documents_index_bulk_default(field, default_value):
+    """Test bulk document indexing while removing optional fields that have default values."""
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    del documents[0][field]
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        documents,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 201
+    responses = response.json()
+    assert [d["status"] for d in responses] == ["success"] * 3
+
+    indexed_document = opensearch.opensearch_client().get(
+        index=service.index_name, id=responses[0]["_id"]
+    )["_source"]
+    assert indexed_document[field] == default_value
+
+
+def test_api_documents_index_bulk_updated_at_before_created_at():
+    """Test bulk document indexing with updated_at before created_at."""
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    documents[0]["updated_at"] = documents[0]["created_at"] - datetime.timedelta(
+        seconds=1
+    )
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        documents,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 400
+    responses = response.json()
+
+    assert [d["status"] for d in responses] == ["error", "valid", "valid"]
+
+    assert responses[0]["status"] == "error"
+    assert len(responses[0]["errors"]) == 1
+    assert (
+        responses[0]["errors"][0]["msg"]
+        == "Value error, updated_at must be later than created_at"
+    )
+    assert responses[0]["errors"][0]["type"] == "value_error"
+
+
+@pytest.mark.parametrize(
+    "field",
+    ["created_at", "updated_at"],
+)
+def test_api_documents_index_bulk_datetime_future(field):
+    """Test bulk document indexing with datetimes in the future."""
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    now = timezone.now()
+    documents[0][field] = now + datetime.timedelta(seconds=3)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        documents,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 400
+    responses = response.json()
+    assert [d["status"] for d in responses] == ["error", "valid", "valid"]
+
+    assert responses[0]["status"] == "error"
+    assert len(responses[0]["errors"]) == 1
+    assert (
+        responses[0]["errors"][0]["msg"]
+        == f"Value error, {field:s} must be earlier than now"
+    )
+    assert responses[0]["errors"][0]["type"] == "value_error"
+
+
+def test_api_documents_index_empty_content_check():
+    """Test bulk document indexing with both empty title & content."""
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    documents[0]["content"] = ""
+    documents[0]["title"] = ""
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        documents,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 400
+    responses = response.json()
+    assert [d["status"] for d in responses] == ["error", "valid", "valid"]
+
+    assert responses[0]["status"] == "error"
+    assert len(responses[0]["errors"]) == 1
+    assert (
+        responses[0]["errors"][0]["msg"]
+        == "Value error, Either title or content should have at least 1 character"
+    )
+    assert responses[0]["errors"][0]["type"] == "value_error"
+
+
+def test_api_documents_index_opensearch_errors():
+    """Test bulk document indexing errors"""
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(3)
+
+    with mock.patch.object(opensearch.opensearch_client(), "bulk") as mock_bulk:
+        mock_bulk.return_value = {
+            "items": [
+                {"index": {"status": 201}},
+                {
+                    "index": {
+                        "status": 400,
+                    }
+                },
+                {"index": {"status": 403, "error": {"reason": "This is forbidden"}}},
+            ]
+        }
+
+        response = APIClient().post(
+            "/api/v1.0/documents/index/",
+            documents,
+            HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+            format="json",
+        )
+
+    assert response.status_code == 201
+    responses = response.json()
+    assert responses == [
+        {
+            "_id": documents[0]["id"],
+            "index": 0,
+            "status": "success",
+        },
+        {
+            "_id": documents[1]["id"],
+            "index": 1,
+            "status": "error",
+            "message": "Unknown error",
+        },
+        {
+            "_id": documents[2]["id"],
+            "index": 2,
+            "status": "error",
+            "message": "This is forbidden",
+        },
+    ]

src/backend/core/tests/test_api_documents_index_single.py

@@ -0,0 +1,673 @@
+"""Tests indexing documents in OpenSearch over the API"""
+
+import datetime
+
+from django.utils import timezone
+
+import pytest
+import responses
+from opensearchpy import NotFoundError
+from rest_framework.test import APIClient
+
+from core import factories
+from core.services import opensearch
+from core.tests.mock import albert_embedding_response
+from core.tests.utils import enable_hybrid_search
+
+pytestmark = pytest.mark.django_db
+
+
+@pytest.fixture(autouse=True)
+def clear_caches():
+    """Clear caches and delete search pipeline before each test"""
+    opensearch.check_hybrid_search_enabled.cache_clear()
+
+
+def test_api_documents_index_single_anonymous():
+    """Anonymous requests should not be allowed to index documents."""
+    document = factories.DocumentSchemaFactory.build()
+
+    response = APIClient().post("/api/v1.0/documents/index/", document, format="json")
+
+    assert response.status_code == 403
+    assert response.json() == {
+        "detail": "Authentication credentials were not provided."
+    }
+
+
+def test_api_documents_index_single_invalid_token():
+    """Requests with invalid tokens should not be allowed to index documents."""
+    document = factories.DocumentSchemaFactory.build()
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION="Bearer invalid",
+        format="json",
+    )
+
+    assert response.status_code == 403
+    assert response.json() == {"detail": "Invalid token."}
+
+
+@responses.activate
+def test_api_documents_index_single_hybrid_enabled_success(settings):
+    """
+    A registered service should be able to index document with a valid token.
+    If hybrid search is enabled, the documents are chunked and embedded.
+    """
+    service = factories.ServiceFactory()
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+
+    document = factories.DocumentSchemaFactory.build()
+    document["content"] = (
+        "a long text to embed." * 100
+    )  # Ensure content is long enough for chunking
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 201
+    assert response.json()["_id"] == str(document["id"])
+
+    new_indexed_document = opensearch.opensearch_client().get(
+        index=service.index_name, id=str(document["id"])
+    )
+    assert new_indexed_document["_version"] == 1
+
+    assert (
+        new_indexed_document["_source"]["title.en"] == document["title"].strip().lower()
+    )
+    assert new_indexed_document["_source"]["content.en"] == document["content"]
+    # only the english fields are indexed
+    assert not "content.fr" in new_indexed_document["_source"]
+
+    # check embedding
+    assert (
+        new_indexed_document["_source"]["chunks"][0]["embedding"]
+        == albert_embedding_response.response["data"][0]["embedding"]
+    )
+    assert (
+        new_indexed_document["_source"]["embedding_model"]
+        == settings.EMBEDDING_API_MODEL_NAME
+    )
+    # Check that the document has been chunked correctly
+    assert (
+        len(new_indexed_document["_source"]["chunks"])
+        == int(
+            len(document["content"]) / (settings.CHUNK_SIZE - settings.CHUNK_OVERLAP)
+        )
+        + 1
+    )
+    for chunk in new_indexed_document["_source"]["chunks"]:
+        assert (
+            chunk["embedding"]
+            == albert_embedding_response.response["data"][0]["embedding"]
+        )
+        assert chunk["content"] in document["content"]
+        assert len(chunk["content"]) < len(document["content"])
+
+
+def test_api_documents_index_language_params():
+    """language_code query param should control which language is indexed."""
+    service = factories.ServiceFactory()
+    document = factories.DocumentSchemaFactory.build()
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 201
+    assert response.json()["_id"] == str(document["id"])
+
+    new_indexed_document = opensearch.opensearch_client().get(
+        index=service.index_name, id=str(document["id"])
+    )
+    language_code = "en"
+    assert (
+        new_indexed_document["_source"][f"title.{language_code}"]
+        == document["title"].strip().lower()
+    )
+    assert (
+        new_indexed_document["_source"][f"content.{language_code}"]
+        == document["content"]
+    )
+    other_language_code = "fr"
+    # only the requested language is indexed
+    assert not f"content.{other_language_code}" in new_indexed_document["_source"]
+
+
+def test_api_documents_index_and_reindex_same_document():
+    """
+    Indexing the same document twice should update it.
+    If the detected language changes the new language code should be used and the
+    former language code should not be present anymore.
+    """
+    service = factories.ServiceFactory()
+    document = factories.DocumentSchemaFactory.build()
+
+    # First indexing with unrecognized language title
+    document["title"] = "planning"
+    APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    new_indexed_document = opensearch.opensearch_client().get(
+        index=service.index_name, id=str(document["id"])
+    )
+    assert new_indexed_document["_version"] == 1
+    assert (
+        new_indexed_document["_source"]["title.und"]
+        == document["title"].strip().lower()
+    )
+    assert new_indexed_document["_source"]["content.und"] == document["content"].strip()
+
+    # Index the same document with a french content
+    document["content"] = "du contenu en francais"
+    APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    new_indexed_document = opensearch.opensearch_client().get(
+        index=service.index_name, id=str(document["id"])
+    )
+    assert new_indexed_document["_version"] == 2
+    # the document is detected as french
+    assert (
+        new_indexed_document["_source"]["title.fr"] == document["title"].strip().lower()
+    )
+    assert new_indexed_document["_source"]["content.fr"] == document["content"]
+    # und field are removed
+    assert "title.und" not in new_indexed_document["_source"]
+    assert "content.und" not in new_indexed_document["_source"]
+
+
+def test_api_documents_index_single_hybrid_disabled_success():
+    """If hybrid search is not enabled, the indexing should have an embedding equal to None."""
+    service = factories.ServiceFactory()
+    document = factories.DocumentSchemaFactory.build()
+    opensearch.check_hybrid_search_enabled.cache_clear()
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 201
+    assert response.json()["_id"] == str(document["id"])
+
+    new_indexed_document = opensearch.opensearch_client().get(
+        index=service.index_name, id=str(document["id"])
+    )
+    assert new_indexed_document["_version"] == 1
+    assert (
+        new_indexed_document["_source"]["title.en"] == document["title"].strip().lower()
+    )
+    assert new_indexed_document["_source"]["content.en"] == document["content"]
+    assert new_indexed_document["_source"]["chunks"] is None
+
+
+def test_api_documents_index_single_ensure_index(settings):
+    """A registered service should be created the opensearch index if needed."""
+    service = factories.ServiceFactory()
+    document = factories.DocumentSchemaFactory.build()
+    opensearch_client_ = opensearch.opensearch_client()
+
+    with pytest.raises(NotFoundError):
+        opensearch_client_.indices.get(index=service.index_name)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 201
+    assert response.json()["_id"] == str(document["id"])
+
+    # The index has been rebuilt
+    data = opensearch_client_.indices.get(index=service.index_name)
+
+    assert data[service.index_name]["mappings"] == {
+        "dynamic": "strict",
+        "properties": {
+            "chunks": {
+                "type": "nested",
+                "properties": {
+                    "content": {"type": "text"},
+                    "embedding": {
+                        "type": "knn_vector",
+                        "dimension": settings.EMBEDDING_DIMENSION,
+                        "method": {
+                            "engine": "lucene",
+                            "space_type": "l2",
+                            "name": "hnsw",
+                            "parameters": {},
+                        },
+                    },
+                    "index": {"type": "integer"},
+                },
+            },
+            "content": {
+                "properties": {
+                    "de": {
+                        "type": "text",
+                        "fields": {
+                            "trigrams": {"type": "text", "analyzer": "trigram_analyzer"}
+                        },
+                        "analyzer": "german_analyzer",
+                    },
+                    "en": {
+                        "type": "text",
+                        "fields": {
+                            "trigrams": {"type": "text", "analyzer": "trigram_analyzer"}
+                        },
+                        "analyzer": "english_analyzer",
+                    },
+                    "fr": {
+                        "type": "text",
+                        "fields": {
+                            "trigrams": {"type": "text", "analyzer": "trigram_analyzer"}
+                        },
+                        "analyzer": "french_analyzer",
+                    },
+                    "nl": {
+                        "type": "text",
+                        "fields": {
+                            "trigrams": {"type": "text", "analyzer": "trigram_analyzer"}
+                        },
+                        "analyzer": "dutch_analyzer",
+                    },
+                    "und": {
+                        "type": "text",
+                        "fields": {
+                            "trigrams": {"type": "text", "analyzer": "trigram_analyzer"}
+                        },
+                        "analyzer": "undetermined_language_analyzer",
+                    },
+                }
+            },
+            "created_at": {"type": "date"},
+            "depth": {"type": "integer"},
+            "embedding_model": {"type": "keyword"},
+            "groups": {"type": "keyword"},
+            "id": {"type": "keyword"},
+            "is_active": {"type": "boolean"},
+            "numchild": {"type": "integer"},
+            "path": {"type": "keyword", "fields": {"text": {"type": "text"}}},
+            "reach": {"type": "keyword"},
+            "size": {"type": "long"},
+            "tags": {"type": "keyword"},
+            "title": {
+                "properties": {
+                    "de": {
+                        "type": "keyword",
+                        "fields": {
+                            "text": {
+                                "type": "text",
+                                "fields": {
+                                    "trigrams": {
+                                        "type": "text",
+                                        "analyzer": "trigram_analyzer",
+                                    }
+                                },
+                                "analyzer": "german_analyzer",
+                            }
+                        },
+                    },
+                    "en": {
+                        "type": "keyword",
+                        "fields": {
+                            "text": {
+                                "type": "text",
+                                "fields": {
+                                    "trigrams": {
+                                        "type": "text",
+                                        "analyzer": "trigram_analyzer",
+                                    }
+                                },
+                                "analyzer": "english_analyzer",
+                            }
+                        },
+                    },
+                    "fr": {
+                        "type": "keyword",
+                        "fields": {
+                            "text": {
+                                "type": "text",
+                                "fields": {
+                                    "trigrams": {
+                                        "type": "text",
+                                        "analyzer": "trigram_analyzer",
+                                    }
+                                },
+                                "analyzer": "french_analyzer",
+                            }
+                        },
+                    },
+                    "nl": {
+                        "type": "keyword",
+                        "fields": {
+                            "text": {
+                                "type": "text",
+                                "fields": {
+                                    "trigrams": {
+                                        "type": "text",
+                                        "analyzer": "trigram_analyzer",
+                                    }
+                                },
+                                "analyzer": "dutch_analyzer",
+                            }
+                        },
+                    },
+                    "und": {
+                        "type": "keyword",
+                        "fields": {
+                            "text": {
+                                "type": "text",
+                                "fields": {
+                                    "trigrams": {
+                                        "type": "text",
+                                        "analyzer": "trigram_analyzer",
+                                    }
+                                },
+                                "analyzer": "undetermined_language_analyzer",
+                            }
+                        },
+                    },
+                }
+            },
+            "updated_at": {"type": "date"},
+            "users": {"type": "keyword"},
+        },
+    }
+
+
+@pytest.mark.parametrize(
+    "field, invalid_value, error_type, error_message",
+    [
+        (
+            "id",
+            "0f9b1c9d-030f-427a-8a0e-6b7c202c5daz",  # invalid UUID b/c contains a z*
+            "uuid_parsing",
+            (
+                "Input should be a valid UUID, invalid character: expected an optional "
+                "prefix of `urn:uuid:` followed by [0-9a-fA-F-], found `z` at 36"
+            ),
+        ),
+        ("title", 1, "string_type", "Input should be a valid string"),
+        (
+            "title",
+            "a" * 301,
+            "string_too_long",
+            "String should have at most 300 characters",
+        ),
+        (
+            "depth",
+            -1,
+            "greater_than_equal",
+            "Input should be greater than or equal to 0",
+        ),
+        (
+            "depth",
+            "a",
+            "int_parsing",
+            "Input should be a valid integer, unable to parse string as an integer",
+        ),
+        (
+            "path",
+            "a" * 301,
+            "string_too_long",
+            "String should have at most 300 characters",
+        ),
+        (
+            "numchild",
+            -1,
+            "greater_than_equal",
+            "Input should be greater than or equal to 0",
+        ),
+        (
+            "numchild",
+            "a",
+            "int_parsing",
+            "Input should be a valid integer, unable to parse string as an integer",
+        ),
+        ("content", 1, "string_type", "Input should be a valid string"),
+        (
+            "created_at",
+            "invalid_date",
+            "datetime_from_date_parsing",
+            "Input should be a valid datetime or date, invalid character in year",
+        ),
+        (
+            "updated_at",
+            "invalid_date",
+            "datetime_from_date_parsing",
+            "Input should be a valid datetime or date, invalid character in year",
+        ),
+        (
+            "size",
+            64448894017.3,
+            "int_from_float",
+            "Input should be a valid integer, got a number with a fractional part",
+        ),
+        (
+            "size",
+            "not an integer",
+            "int_parsing",
+            "Input should be a valid integer, unable to parse string as an integer",
+        ),
+        (
+            "users",
+            "33052c8b-3181-4420-aede-f8396fc0f9a1",
+            "list_type",
+            ("Input should be a valid list"),
+        ),
+        (
+            "users",
+            ["a" * 51],
+            "string_too_long",
+            "String should have at most 50 characters",
+        ),
+        (
+            "groups",
+            ["not_a_slug"],
+            "string_pattern_mismatch",
+            "String should match pattern '^[a-z0-9]+(?:-[a-z0-9]+)*$'",
+        ),
+        (
+            "groups",
+            "not-a-list",
+            "list_type",
+            ("Input should be a valid list"),
+        ),
+        (
+            "reach",
+            "invalid",
+            "enum",
+            "Input should be 'public', 'authenticated' or 'restricted'",
+        ),
+        (
+            "is_active",
+            "invalid",
+            "bool_parsing",
+            "Input should be a valid boolean, unable to interpret input",
+        ),
+    ],
+)
+def test_api_documents_index_single_invalid_document(
+    field, invalid_value, error_type, error_message
+):
+    """Test document indexing with various invalid fields."""
+    service = factories.ServiceFactory()
+    document = factories.DocumentSchemaFactory.build()
+
+    # Modify the document with the invalid value for the specified field
+    document[field] = invalid_value
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 400
+    assert response.data[0]["msg"] == error_message
+    assert response.data[0]["type"] == error_type
+
+
+@pytest.mark.parametrize(
+    "field",
+    [
+        "id",
+        "title",
+        "depth",
+        "path",
+        "numchild",
+        "content",
+        "size",
+        "created_at",
+        "updated_at",
+        "is_active",
+    ],
+)
+def test_api_documents_index_single_required(field):
+    """Test document indexing with a required field missing."""
+    service = factories.ServiceFactory()
+    document = factories.DocumentSchemaFactory.build()
+
+    del document[field]
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 400
+    assert response.data[0]["msg"] == "Field required"
+    assert response.data[0]["type"] == "missing"
+
+
+@pytest.mark.parametrize(
+    "field,default_value",
+    [
+        ("users", []),
+        ("groups", []),
+        ("reach", "restricted"),
+    ],
+)
+def test_api_documents_index_single_default(field, default_value):
+    """Test document indexing while removing optional fields that have default values."""
+    service = factories.ServiceFactory()
+    document = factories.DocumentSchemaFactory.build()
+
+    del document[field]
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 201
+    assert response.json()["_id"] == str(document["id"])
+
+    indexed_document = opensearch.opensearch_client().get(
+        index=service.index_name, id=str(document["id"])
+    )["_source"]
+    assert indexed_document[field] == default_value
+
+
+def test_api_documents_index_single_udpated_at_before_created():
+    """Test document indexing with updated_at before created_at."""
+    service = factories.ServiceFactory()
+    document = factories.DocumentSchemaFactory.build()
+
+    document["updated_at"] = document["created_at"] - datetime.timedelta(seconds=1)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 400
+    assert (
+        response.data[0]["msg"]
+        == "Value error, updated_at must be later than created_at"
+    )
+    assert response.data[0]["type"] == "value_error"
+
+
+@pytest.mark.parametrize(
+    "field",
+    ["created_at", "updated_at"],
+)
+def test_api_documents_index_single_datetime_future(field):
+    """Test document indexing with datetimes in the future."""
+    service = factories.ServiceFactory()
+    document = factories.DocumentSchemaFactory.build()
+
+    now = timezone.now()
+    document[field] = now + datetime.timedelta(seconds=3)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 400
+    assert response.data[0]["msg"] == f"Value error, {field:s} must be earlier than now"
+    assert response.data[0]["type"] == "value_error"
+
+
+def test_api_documents_index_empty_content_check():
+    """Test document indexing with both empty title & content."""
+    service = factories.ServiceFactory()
+    document = factories.DocumentSchemaFactory.build()
+
+    document["content"] = ""
+    document["title"] = ""
+
+    response = APIClient().post(
+        "/api/v1.0/documents/index/",
+        document,
+        HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
+        format="json",
+    )
+
+    assert response.status_code == 400
+    assert (
+        response.data[0]["msg"]
+        == "Value error, Either title or content should have at least 1 character"
+    )
+    assert response.data[0]["type"] == "value_error"

src/backend/core/tests/test_api_documents_search.py

@@ -0,0 +1,995 @@
+"""
+Test suite for searching documents in OpenSearch over the API.
+
+Don't use pytest parametrized tests because batch generation and indexing
+of documents is slow and better done only once.
+"""
+
+import logging
+import operator
+import random
+
+import pytest
+import responses
+from rest_framework.test import APIClient
+
+from core import enums, factories
+from core.services.opensearch import (
+    check_hybrid_search_enabled,
+    opensearch_client,
+)
+from core.utils import bulk_create_documents, prepare_index
+
+from ..enums import SearchTypeEnum
+from .mock import albert_embedding_response
+from .utils import (
+    build_authorization_bearer,
+    enable_hybrid_search,
+    setup_oicd_resource_server,
+)
+
+pytestmark = pytest.mark.django_db
+
+
+@pytest.fixture(autouse=True)
+def before_each():
+    """Clear cached functions before each test to avoid side effects"""
+    clear_caches()
+    yield
+    clear_caches()
+
+
+def clear_caches():
+    """Clear cached functions before each test to avoid side effects"""
+    opensearch_client.cache_clear()
+    check_hybrid_search_enabled.cache_clear()
+
+
+@responses.activate
+def test_api_documents_search_auth_invalid_parameters(settings):
+    """Invalid service parameters should result in a 401 error"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+
+    settings.OIDC_RS_CLIENT_ID = None
+    settings.OIDC_RS_CLIENT_SECRET = None
+
+    service = factories.ServiceFactory()
+    prepare_index(service.index_name, [])
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "a quick fox"},
+        format="json",
+        HTTP_AUTHORIZATION="Bearer unknown",
+    )
+
+    assert response.status_code == 401
+    assert response.json() == {"detail": "Resource Server is improperly configured"}
+
+
+@responses.activate
+def test_api_documents_search_opensearch_env_variables_not_set(settings):
+    """
+    Missing environment variables for OpenSearch client should
+    result in a 500 internal server error
+    """
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    factories.ServiceFactory()
+
+    del settings.OPENSEARCH_HOST  # Remove required settings
+    del settings.OPENSEARCH_PASSWORD
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*"},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 400
+    assert response.json() == [
+        "Missing required OpenSearch environment variables: OPENSEARCH_HOST, OPENSEARCH_PASSWORD"
+    ]
+
+
+@responses.activate
+def test_api_documents_search_query_unknown_user(settings):
+    """Searching a document without an existing user should result in a 401 error"""
+    setup_oicd_resource_server(
+        responses,
+        settings,
+        sub="unknown",
+        introspect=lambda request, user_info: (404, {}, ""),
+    )
+
+    service = factories.ServiceFactory()
+    prepare_index(service.index_name, [])
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "a quick fox"},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 400
+
+
+@responses.activate
+def test_api_documents_search_services_invalid_parameters(settings):
+    """Invalid services parameter should result in a 400 error"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    factories.ServiceFactory()
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        # services should be a list
+        {"q": "a quick fox", "services": {}},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 400
+    assert response.json() == [
+        {
+            "loc": ["services"],
+            "msg": "Value error, ",
+            "type": "value_error",
+        }
+    ]
+
+
+@responses.activate
+def test_api_documents_search_reached_docs_invalid_parameters(settings):
+    """Invalid visited parameters should result in a 400 error"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    factories.ServiceFactory()
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        # visited should be a list
+        {"q": "a quick fox", "visited": {}},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 400
+    assert response.json() == [
+        {
+            "loc": ["visited"],
+            "msg": "Value error, ",
+            "type": "value_error",
+        }
+    ]
+
+
+@responses.activate
+def test_api_documents_search_match_all(settings):
+    """Searching a document with q='*' should match all docs"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    nb_documents = 12
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(
+        nb_documents, reach=random.choice(["public", "authenticated"])
+    )
+    prepare_index(service.index_name, documents)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*", "visited": [doc["id"] for doc in documents]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert len(response.json()) == nb_documents
+
+    assert [r["_id"] for r in response.json()] == [str(doc["id"]) for doc in documents]
+
+
+@responses.activate
+def test_api_documents_full_text_search_query_title(settings):
+    """Searching a document by its title should work as expected"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    service = factories.ServiceFactory()
+
+    documents = bulk_create_documents(
+        [
+            {"title": "The quick brown fox", "content": "the wolf"},
+            {"title": "The blue fox", "content": "the wolf"},
+            {"title": "The brown goat", "content": "the wolf"},
+        ]
+    )
+
+    prepare_index(service.index_name, documents)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "a quick fox", "visited": [doc["id"] for doc in documents]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert len(response.json()) == 2
+
+    fox_response = response.json()[0]
+    fox_document = documents[0]
+    assert list(fox_response.keys()) == ["_index", "_id", "_score", "_source", "fields"]
+    assert fox_response["_id"] == str(documents[0]["id"])
+    assert fox_response["_source"] == {
+        "content.en": "the wolf",
+        "depth": 1,
+        "numchild": 0,
+        "path": fox_document["path"],
+        "size": fox_document["size"],
+        "created_at": fox_document["created_at"].isoformat(),
+        "updated_at": fox_document["updated_at"].isoformat(),
+        "reach": fox_document["reach"],
+        "tags": [],
+        "title.en": fox_document["title"],
+    }
+    assert fox_response["fields"] == {"number_of_users": [1], "number_of_groups": [3]}
+
+    other_fox_response = response.json()[1]
+    other_fox_document = documents[1]
+    assert list(other_fox_response.keys()) == [
+        "_index",
+        "_id",
+        "_score",
+        "_source",
+        "fields",
+    ]
+    assert other_fox_response["_id"] == str(other_fox_document["id"])
+    assert other_fox_response["_source"] == {
+        "content.en": fox_document["content"],
+        "depth": 1,
+        "numchild": 0,
+        "path": other_fox_document["path"],
+        "size": other_fox_document["size"],
+        "created_at": other_fox_document["created_at"].isoformat(),
+        "updated_at": other_fox_document["updated_at"].isoformat(),
+        "reach": other_fox_document["reach"],
+        "tags": [],
+        "title.en": other_fox_document["title"],
+    }
+    assert other_fox_response["fields"] == {
+        "number_of_users": [1],
+        "number_of_groups": [3],
+    }
+
+
+@responses.activate
+def test_api_documents_full_text_search(settings):
+    """
+    Searching a document by its content should work as expected.
+    """
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+
+    service = factories.ServiceFactory()
+    documents = bulk_create_documents(
+        [
+            {"title": "The quick brown fox", "content": "the wolf"},
+            {"title": "The blue fox", "content": "the wolf"},
+            {"title": "The brown goat", "content": "the wolf"},
+        ]
+    )
+    prepare_index(service.index_name, documents)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "a quick fox", "visited": [doc["id"] for doc in documents]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert len(response.json()) == 2
+
+    fox_response = response.json()[0]
+    fox_document = documents[0]
+    assert list(fox_response.keys()) == ["_index", "_id", "_score", "_source", "fields"]
+    assert fox_response["_id"] == str(fox_document["id"])
+    assert fox_response["_score"] > 0
+    assert fox_response["_source"] == {
+        "content.en": "the wolf",
+        "depth": 1,
+        "numchild": 0,
+        "path": fox_document["path"],
+        "size": fox_document["size"],
+        "created_at": fox_document["created_at"].isoformat(),
+        "updated_at": fox_document["updated_at"].isoformat(),
+        "reach": fox_document["reach"],
+        "tags": [],
+        "title.en": fox_document["title"],
+    }
+    assert fox_response["fields"] == {"number_of_users": [1], "number_of_groups": [3]}
+
+    other_fox_response = response.json()[1]
+    other_fox_document = documents[1]
+    assert list(other_fox_response.keys()) == [
+        "_index",
+        "_id",
+        "_score",
+        "_source",
+        "fields",
+    ]
+    assert other_fox_response["_id"] == str(other_fox_document["id"])
+    assert other_fox_response["_score"] > 0
+    assert other_fox_response["_source"] == {
+        "content.en": fox_document["content"],
+        "depth": 1,
+        "numchild": 0,
+        "path": other_fox_document["path"],
+        "size": other_fox_document["size"],
+        "created_at": other_fox_document["created_at"].isoformat(),
+        "updated_at": other_fox_document["updated_at"].isoformat(),
+        "reach": other_fox_document["reach"],
+        "tags": [],
+        "title.en": other_fox_document["title"],
+    }
+    assert other_fox_response["fields"] == {
+        "number_of_users": [1],
+        "number_of_groups": [3],
+    }
+
+
+@responses.activate
+def test_api_documents_search_with_search_type_full_text(settings, caplog):
+    """Test API with search_type=full_text forces full-text search even if hybrid is enabled"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+        ]
+    )
+    prepare_index(service.index_name, documents)
+    with caplog.at_level(logging.INFO):
+        response = APIClient().post(
+            "/api/v1.0/documents/search/",
+            {
+                "q": "wolf",
+                "search_type": SearchTypeEnum.FULL_TEXT,
+                "visited": [doc["id"] for doc in documents],
+            },
+            format="json",
+            HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+        )
+    assert response.status_code == 200
+    assert any(
+        "Performing full-text search without embedding: wolf" in message
+        for message in caplog.messages
+    )
+
+
+@responses.activate
+def test_api_documents_hybrid_search(settings):
+    """Searching a document by its content should work as expected"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    # hybrid search is enabled by default
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )  # mock embedding API
+
+    service = factories.ServiceFactory()
+    documents = bulk_create_documents(
+        [
+            {"title": "The quick brown fox", "content": "the wolf"},
+            {"title": "The blue fox", "content": "the wolf"},
+            {"title": "The brown goat", "content": "the wolf"},
+        ]
+    )
+    prepare_index(service.index_name, documents)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "a quick fox", "visited": [doc["id"] for doc in documents]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert (
+        len(response.json()) == 3
+    )  # semantic search always returns a response of size nb_results
+
+    fox_response = response.json()[0]
+    fox_document = documents[0]
+    assert list(fox_response.keys()) == ["_index", "_id", "_score", "_source", "fields"]
+    assert fox_response["_id"] == str(fox_document["id"])
+    assert fox_response["_score"] > 0
+    assert fox_response["_source"] == {
+        "content.en": fox_document["content"],
+        "depth": 1,
+        "numchild": 0,
+        "path": fox_document["path"],
+        "size": fox_document["size"],
+        "created_at": fox_document["created_at"].isoformat(),
+        "updated_at": fox_document["updated_at"].isoformat(),
+        "reach": fox_document["reach"],
+        "tags": [],
+        "title.en": fox_document["title"],
+    }
+    assert fox_response["fields"] == {"number_of_users": [1], "number_of_groups": [3]}
+
+    other_fox_response = response.json()[1]
+    other_fox_document = documents[1]
+    assert list(other_fox_response.keys()) == [
+        "_index",
+        "_id",
+        "_score",
+        "_source",
+        "fields",
+    ]
+    assert other_fox_response["_id"] == str(other_fox_document["id"])
+    assert other_fox_response["_score"] > 0
+    assert other_fox_response["_source"] == {
+        "content.en": fox_document["content"],
+        "depth": 1,
+        "numchild": 0,
+        "path": other_fox_document["path"],
+        "size": other_fox_document["size"],
+        "created_at": other_fox_document["created_at"].isoformat(),
+        "updated_at": other_fox_document["updated_at"].isoformat(),
+        "reach": other_fox_document["reach"],
+        "tags": [],
+        "title.en": other_fox_document["title"],
+    }
+    assert other_fox_response["fields"] == {
+        "number_of_users": [1],
+        "number_of_groups": [3],
+    }
+
+    no_fox_response = response.json()[2]
+    no_fox_document = documents[2]
+    assert list(no_fox_response.keys()) == [
+        "_index",
+        "_id",
+        "_score",
+        "_source",
+        "fields",
+    ]
+    assert no_fox_response["_id"] == str(no_fox_document["id"])
+    assert no_fox_response["_source"] == {
+        "content.en": fox_document["content"],
+        "depth": 1,
+        "numchild": 0,
+        "path": no_fox_document["path"],
+        "size": no_fox_document["size"],
+        "created_at": no_fox_document["created_at"].isoformat(),
+        "updated_at": no_fox_document["updated_at"].isoformat(),
+        "reach": no_fox_document["reach"],
+        "tags": [],
+        "title.en": no_fox_document["title"],
+    }
+    assert no_fox_response["fields"] == {
+        "number_of_users": [1],
+        "number_of_groups": [3],
+    }
+
+
+@responses.activate
+def test_api_documents_search_ordering_by_fields(settings):
+    """It should be possible to order by several fields"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(
+        4, reach=random.choice(["public", "authenticated"])
+    )
+    prepare_index(service.index_name, documents)
+
+    parameters = [
+        (enums.CREATED_AT, "asc"),
+        (enums.CREATED_AT, "desc"),
+        (enums.UPDATED_AT, "asc"),
+        (enums.UPDATED_AT, "desc"),
+        (enums.SIZE, "asc"),
+        (enums.SIZE, "desc"),
+        (enums.REACH, "asc"),
+        (enums.REACH, "desc"),
+    ]
+
+    for field, direction in parameters:
+        response = APIClient().post(
+            "/api/v1.0/documents/search/",
+            {
+                "q": "*",
+                "order_by": field,
+                "order_direction": direction,
+                "visited": [doc["id"] for doc in documents],
+            },
+            format="json",
+            HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+        )
+
+        assert response.status_code == 200
+        data = response.json()
+        assert len(data) == 4
+
+        # Check that results are sorted by the field as expected
+        compare = operator.le if direction == "asc" else operator.ge
+        for i in range(len(data) - 1):
+            assert compare(data[i]["_source"][field], data[i + 1]["_source"][field])
+
+
+@responses.activate
+def test_api_documents_search_ordering_by_relevance(settings):
+    """It should be possible to order by relevance (score)"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(
+        4, reach=random.choice(["public", "authenticated"])
+    )
+    prepare_index(service.index_name, documents)
+
+    for direction in ["asc", "desc"]:
+        response = APIClient().post(
+            "/api/v1.0/documents/search/",
+            {
+                "q": "*",
+                "order_by": "relevance",
+                "order_direction": direction,
+                "visited": [doc["id"] for doc in documents],
+            },
+            format="json",
+            HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+        )
+
+        assert response.status_code == 200
+        data = response.json()
+        assert len(data) == 4
+
+        # Check that results are sorted by score as expected
+        compare = operator.le if direction == "asc" else operator.ge
+        for i in range(len(data) - 1):
+            assert compare(data[i]["_score"], data[i + 1]["_score"])
+
+
+@responses.activate
+def test_api_documents_search_ordering_by_unknown_field(settings):
+    """Trying to sort by an unknown field should return a 400 error"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    # Setup: Initialize the service and documents only once
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(
+        2, reach=random.choice(["public", "authenticated"])
+    )
+    prepare_index(service.index_name, documents)
+
+    # Define the parameters manually
+    directions = ["asc", "desc"]
+
+    # Perform the parameterized tests
+    for direction in directions:
+        response = APIClient().post(
+            "/api/v1.0/documents/search/",
+            {
+                "q": "*",
+                "order_by": "unknown",
+                "order_direction": direction,
+                "visited": [doc["id"] for doc in documents],
+            },
+            format="json",
+            HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+        )
+
+        assert response.status_code == 400
+        assert response.json() == [
+            {
+                "loc": ["order_by"],
+                "msg": (
+                    "Input should be 'relevance', 'title', 'created_at', "
+                    "'updated_at', 'size' or 'reach'"
+                ),
+                "type": "literal_error",
+            }
+        ]
+
+
+@responses.activate
+def test_api_documents_search_ordering_by_unknown_direction(settings):
+    """Trying to sort with an unknown direction should return a 400 error"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(
+        2, reach=random.choice(["public", "authenticated"])
+    )
+    prepare_index(service.index_name, documents)
+
+    for field in enums.ORDER_BY_OPTIONS:
+        response = APIClient().post(
+            "/api/v1.0/documents/search/",
+            {
+                "q": "*",
+                "order_by": field,
+                "order_direction": "unknown",
+                "visited": [doc["id"] for doc in documents],
+            },
+            format="json",
+            HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+        )
+
+        assert response.status_code == 400
+        assert response.json() == [
+            {
+                "loc": ["order_direction"],
+                "msg": "Input should be 'asc' or 'desc'",
+                "type": "literal_error",
+            }
+        ]
+
+
+@responses.activate
+def test_api_documents_search_filtering_by_reach(settings):
+    """It should be possible to filter results by their reach"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(
+        4, reach=random.choice(["public", "authenticated"])
+    )
+    prepare_index(service.index_name, documents)
+
+    for reach in enums.ReachEnum:
+        response = APIClient().post(
+            "/api/v1.0/documents/search/",
+            {
+                "q": "*",
+                "reach": reach.value,
+                "visited": [doc["id"] for doc in documents],
+            },
+            format="json",
+            HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+        )
+
+        assert response.status_code == 200
+        data = response.json()
+
+        for result in data:
+            assert reach == result["_source"]["reach"]
+
+
+@responses.activate
+def test_api_documents_search_with_nb_results(settings):
+    """nb_size should correctly return results of given size"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(
+        9, reach=random.choice(["public", "authenticated"])
+    )
+    ids = [str(doc["id"]) for doc in documents]
+    prepare_index(service.index_name, documents)
+
+    nb_results = 3
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {
+            "q": "*",
+            "nb_results": nb_results,
+            "visited": [doc["id"] for doc in documents],
+        },
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    data = response.json()
+    assert [r["_id"] for r in data] == ids[0:nb_results]
+
+    nb_results = 6
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {
+            "q": "*",
+            "nb_results": nb_results,
+            "visited": [doc["id"] for doc in documents],
+        },
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+    assert response.status_code == 200
+    data = response.json()
+    assert [r["_id"] for r in data] == ids[0:nb_results]
+
+    nb_results = 10
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {
+            "q": "*",
+            "nb_results": nb_results,
+            "visited": [doc["id"] for doc in documents],
+        },
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+    assert response.status_code == 200
+    data = response.json()
+    # nb_results > total number of documents => returns all documents
+    assert [r["_id"] for r in data] == ids[0:9]
+
+
+@responses.activate
+def test_api_documents_search_nb_results_invalid_parameters(settings):
+    """Invalid nb_results parameters should result in a 400 error"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+    documents = factories.DocumentSchemaFactory.build_batch(
+        4, reach=random.choice(["public", "authenticated"])
+    )
+    prepare_index(service.index_name, documents)
+
+    parameters = [
+        (
+            "invalid",
+            "int_parsing",
+            "Input should be a valid integer, unable to parse string as an integer",
+        ),
+        (-1, "greater_than_equal", "Input should be greater than or equal to 1"),
+        (0, "greater_than_equal", "Input should be greater than or equal to 1"),
+        (350, "less_than_equal", "Input should be less than or equal to 300"),
+    ]
+
+    for nb_results, error_type, error_message in parameters:
+        response = APIClient().post(
+            "/api/v1.0/documents/search/",
+            {"q": "*", "nb_results": nb_results},
+            format="json",
+            HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+        )
+
+        assert response.status_code == 400
+        assert response.data[0]["msg"] == error_message
+        assert response.data[0]["type"] == error_type
+
+
+@responses.activate
+def test_api_documents_search_nb_results_with_filtering(settings):
+    """nb_results should work correctly when combined with filtering by reach"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+    public_documents = factories.DocumentSchemaFactory.build_batch(3, reach="public")
+    public_ids = [str(doc["id"]) for doc in public_documents]
+    private_documents = factories.DocumentSchemaFactory.build_batch(
+        2, reach="authenticated"
+    )
+    prepare_index(service.index_name, public_documents + private_documents)
+
+    nb_results = 3
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {
+            "q": "*",
+            "reach": "public",
+            "nb_results": nb_results,
+            "visited": public_ids,
+        },
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+    assert response.status_code == 200
+    assert [r["_id"] for r in response.json()] == public_ids[0:nb_results]
+
+
+@responses.activate
+def test_api_documents_search_filtering_by_tags(settings):
+    """Test filtering documents by a single tag via API"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+
+    documents = bulk_create_documents(
+        [
+            {
+                "title": "Python document",
+                "content": "About Python",
+                "tags": ["python", "programming"],
+            },
+            {
+                "title": "JavaScript document",
+                "content": "About JavaScript",
+                "tags": ["javascript", "programming"],
+            },
+            {
+                "title": "Untagged document",
+                "content": "No tags here",
+                "tags": [],
+            },
+        ]
+    )
+
+    prepare_index(service.index_name, documents)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {
+            "q": "*",
+            "tags": ["python"],
+            "visited": [doc["id"] for doc in documents],
+        },
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert len(response.json()) == 1
+    assert response.json()[0]["_id"] == str(documents[0]["id"])
+    assert response.json()[0]["_source"]["tags"] == ["python", "programming"]
+
+
+@responses.activate
+def test_api_documents_search_without_tags_filter(settings):
+    """Test that search works normally when no tags filter is provided"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+
+    documents = bulk_create_documents(
+        [
+            {
+                "title": "Document with tags",
+                "content": "Tagged content",
+                "tags": ["python"],
+            },
+            {
+                "title": "Document without tags",
+                "content": "Untagged content",
+                "tags": [],
+            },
+        ]
+    )
+
+    prepare_index(service.index_name, documents)
+
+    # Search without tags parameter - should return all documents
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {
+            "q": "*",
+            "visited": [doc["id"] for doc in documents],
+        },
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert len(response.json()) == 2
+
+
+@responses.activate
+def test_api_documents_search_filtering_by_path(settings):
+    """Test filtering documents by path prefix via API"""
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+
+    documents = bulk_create_documents(
+        [
+            {
+                "title": "Document with tags",
+                "content": "Tagged document",
+                "path": "/path/to/doc1",
+            },
+            {
+                "title": "Document without tags",
+                "content": "Untagged document",
+                "path": "/path/to/doc2",
+            },
+            {
+                "title": "Document without tags",
+                "content": "Untagged document",
+                "path": "other/path/to/doc3",
+            },
+        ]
+    )
+
+    prepare_index(service.index_name, documents)
+
+    path_filter = "/path/to/"
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {
+            "q": "*",
+            "path": path_filter,
+            "visited": [doc["id"] for doc in documents],
+        },
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
+    )
+
+    assert response.status_code == 200
+    assert len(response.json()) == 2
+    for hit in response.json():
+        assert hit["_source"]["path"].startswith(path_filter)

src/backend/core/tests/test_api_documents_search_access_control.py

@@ -0,0 +1,502 @@
+"""
+Test suite for access control when searching documents over the API.
+
+Don't use pytest parametrized tests because batch generation and indexing
+of documents is slow and better done only once.
+"""
+
+import pytest
+import responses
+from rest_framework.test import APIClient
+
+from core import enums, factories
+from core.services.opensearch import opensearch_client
+from core.utils import prepare_index
+
+from .mock import albert_embedding_response
+from .utils import (
+    build_authorization_bearer,
+    setup_oicd_resource_server,
+)
+
+pytestmark = pytest.mark.django_db
+
+
+@responses.activate
+def test_api_documents_search_access_control_anonymous(settings):
+    """Anonymous users should not be allowed to search documents even public."""
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+    documents = []
+    for reach in enums.ReachEnum:
+        documents.extend(factories.DocumentSchemaFactory.build_batch(3, reach=reach))
+    prepare_index(service.index_name, documents)
+
+    response = APIClient().post("/api/v1.0/documents/search/?q=*")
+
+    assert response.status_code == 401
+
+
+@responses.activate
+def test_api_documents_search_access_control(settings):
+    """
+    Authenticated users should only see documents:
+    - for which they are listed in the "users" field
+    - that have a reach set to "authenticated" or "public"
+    - only configured services providers are allowed (e.g docs)
+    (groups is not yet implemnted)
+    """
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    setup_oicd_resource_server(responses, settings, sub="user_sub")
+    token = build_authorization_bearer()
+
+    service = factories.ServiceFactory()
+    documents_reach = factories.DocumentSchemaFactory.build_batch(6)
+    documents_open = [
+        doc for doc in documents_reach if doc["reach"] in ["authenticated", "public"]
+    ]
+    documents_user = factories.DocumentSchemaFactory.build_batch(
+        6, users=["user_sub", "user_sub2"]
+    )
+    expected_ids = [doc["id"] for doc in documents_open + documents_user]
+
+    prepare_index(service.index_name, documents_user + documents_reach)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*", "visited": [d["id"] for d in documents_open]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 200
+    assert sorted([d["_id"] for d in response.json()]) == sorted(expected_ids)
+
+
+@responses.activate
+@pytest.mark.parametrize(
+    "doc_ids,visited,expected",
+    [
+        (["a", "b"], [], []),
+        (["a", "b"], "", []),
+        (["a", "b"], None, []),
+        (["a", "b"], ["other"], []),
+        ([], ["a"], []),
+        (["a", "b"], ["a"], ["a"]),
+        (["a", "b"], ["a", "b", "c"], ["a", "b"]),
+        (["a", "b"], "a,b,c", ["a", "b"]),
+    ],
+)
+def test_api_documents_search_access__only_visited_public(
+    doc_ids, visited, expected, settings
+):
+    """
+    Authenticated users should only see documents with reach="public"
+    that are in "visited" list.
+    """
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    setup_oicd_resource_server(responses, settings, sub="user_sub", audience="docs")
+    token = build_authorization_bearer()
+
+    service = factories.ServiceFactory(client_id="docs")
+
+    docs = [
+        factories.DocumentSchemaFactory(
+            reach=[enums.ReachEnum.PUBLIC, enums.ReachEnum.AUTHENTICATED], id=doc_id
+        )
+        for doc_id in doc_ids
+    ]
+
+    prepare_index(service.index_name, docs)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*", "visited": visited},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 200, response.json()
+    assert sorted([d["_id"] for d in response.json()]) == sorted(expected)
+
+
+@responses.activate
+def test_api_documents_search_access__any_owner_public(settings):
+    """
+    Authenticated users should only see documents with reach="public"
+    that are in "visited" list.
+    """
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    setup_oicd_resource_server(responses, settings, sub="user_sub", audience="docs")
+    token = build_authorization_bearer()
+
+    service = factories.ServiceFactory(client_id="docs")
+
+    docs = factories.DocumentSchemaFactory.build_batch(
+        6,
+        reach=enums.ReachEnum.PUBLIC,
+        users=["user_sub"],
+    )
+
+    other_docs = factories.DocumentSchemaFactory.build_batch(
+        6,
+        reach=enums.ReachEnum.PUBLIC,
+        users=["other_sub"],
+    )
+
+    prepare_index(service.index_name, docs + other_docs)
+
+    expected = [d["id"] for d in docs]
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*", "visited": []},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 200, response.json()
+    assert sorted([d["_id"] for d in response.json()]) == sorted(expected)
+
+
+@responses.activate
+def test_api_documents_search_access__services(settings):
+    """
+    Authenticated users should only see documents of audience
+    service providers (e.g docs)
+    """
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    setup_oicd_resource_server(responses, settings, sub="user_sub", audience="a-client")
+    token = build_authorization_bearer()
+
+    service_a = factories.ServiceFactory(client_id="a-client")
+    service_b = factories.ServiceFactory(client_id="b-client")
+
+    service_a_docs = factories.DocumentSchemaFactory.build_batch(
+        3, reach=enums.ReachEnum.AUTHENTICATED, users=["user_sub"]
+    )
+    service_b_docs = factories.DocumentSchemaFactory.build_batch(
+        3, reach=enums.ReachEnum.AUTHENTICATED, users=["user_sub"]
+    )
+
+    expected_ids = [doc["id"] for doc in service_a_docs]
+
+    prepare_index(service_a.index_name, service_a_docs)
+    prepare_index(service_b.index_name, service_b_docs)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*"},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 200
+    assert sorted([d["_id"] for d in response.json()]) == sorted(expected_ids)
+
+
+@responses.activate
+def test_api_documents_search_access__missing_index(settings):
+    """
+    When the service has no opensearch index, returns an empty list.
+    """
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    setup_oicd_resource_server(responses, settings, sub="user_sub", audience="a-client")
+    token = build_authorization_bearer()
+    factories.ServiceFactory(name="test-index-a", client_id="a-client")
+
+    opensearch_client.cache_clear()
+
+    # a-client has no index. ignore it.
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*"},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 200
+    assert response.json() == []
+
+
+@responses.activate
+def test_api_documents_search_access__related_services(settings):
+    """
+    Authenticated users should only see documents of audience
+    service providers and its related services (e.g drive + docs)
+    """
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    setup_oicd_resource_server(responses, settings, sub="user_sub", audience="c-client")
+    token = build_authorization_bearer()
+
+    service_a = factories.ServiceFactory(name="test-index-a", client_id="a-client")
+    service_b = factories.ServiceFactory(name="test-index-b", client_id="b-client")
+    service_c = factories.ServiceFactory(name="test-index-c", client_id="c-client")
+    service_c.services.set([service_a])
+    service_c.save()
+
+    service_a_docs = factories.DocumentSchemaFactory.build_batch(
+        3, reach=enums.ReachEnum.AUTHENTICATED, users=["user_sub"]
+    )
+    service_b_docs = factories.DocumentSchemaFactory.build_batch(
+        3, reach=enums.ReachEnum.AUTHENTICATED, users=["user_sub"]
+    )
+    service_c_docs = factories.DocumentSchemaFactory.build_batch(
+        3, reach=enums.ReachEnum.AUTHENTICATED, users=["user_sub"]
+    )
+
+    expected_ids = [doc["id"] for doc in service_a_docs + service_c_docs]
+
+    prepare_index(service_a.index_name, service_a_docs)
+    prepare_index(service_b.index_name, service_b_docs)
+    prepare_index(service_c.index_name, service_c_docs)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*"},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 200
+    assert sorted([d["_id"] for d in response.json()]) == sorted(expected_ids)
+
+
+@responses.activate
+def test_api_documents_search_access__related_missing_index(settings):
+    """
+    When the service has no opensearch index, returns the related services data.
+    """
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    setup_oicd_resource_server(responses, settings, sub="user_sub", audience="a-client")
+    token = build_authorization_bearer()
+
+    service_a = factories.ServiceFactory(name="test-index-a", client_id="a-client")
+    service_b = factories.ServiceFactory(name="test-index-b", client_id="b-client")
+    service_c = factories.ServiceFactory(name="test-index-c", client_id="c-client")
+    service_c.services.set([service_a])
+    service_c.save()
+
+    service_b_docs = factories.DocumentSchemaFactory.build_batch(
+        3, reach=enums.ReachEnum.AUTHENTICATED, users=["user_sub"]
+    )
+    service_c_docs = factories.DocumentSchemaFactory.build_batch(
+        3, reach=enums.ReachEnum.AUTHENTICATED, users=["user_sub"]
+    )
+
+    expected_ids = [doc["id"] for doc in service_c_docs]
+
+    prepare_index(service_b.index_name, service_b_docs)
+    prepare_index(service_c.index_name, service_c_docs)
+
+    # a-client has no index. ignore it.
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*"},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 200
+    assert sorted([d["_id"] for d in response.json()]) == sorted(expected_ids)
+
+
+@responses.activate
+def test_api_documents_search_access__request_services(settings):
+    """
+    Authenticated users should only see documents of audience
+    from requested services : 'services' parameter.
+    Raise 400 error if not all requested services are authorized.
+    """
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    setup_oicd_resource_server(responses, settings, sub="user_sub", audience="c-client")
+    token = build_authorization_bearer()
+
+    service_a = factories.ServiceFactory(name="test-index-a", client_id="a-client")
+    service_b = factories.ServiceFactory(name="test-index-b", client_id="b-client")
+    service_c = factories.ServiceFactory(name="test-index-c", client_id="c-client")
+
+    service_a_docs = factories.DocumentSchemaFactory.build_batch(
+        3, reach=enums.ReachEnum.AUTHENTICATED, users=["user_sub"]
+    )
+    service_b_docs = factories.DocumentSchemaFactory.build_batch(
+        3, reach=enums.ReachEnum.AUTHENTICATED, users=["user_sub"]
+    )
+    service_c_docs = factories.DocumentSchemaFactory.build_batch(
+        3, reach=enums.ReachEnum.AUTHENTICATED, users=["user_sub"]
+    )
+
+    expected_ids = [doc["id"] for doc in service_c_docs]
+
+    prepare_index(service_a.index_name, service_a_docs)
+    prepare_index(service_b.index_name, service_b_docs)
+    prepare_index(service_c.index_name, service_c_docs)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*", "services": ["test-index-c"]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 200
+    assert sorted([d["_id"] for d in response.json()]) == sorted(expected_ids)
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*", "services": ["test-index-c", "test-index-b"]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 400
+    assert response.json() == {"detail": "Invalid request."}
+
+
+@responses.activate
+def test_api_documents_search_access__request_inactive_services(settings):
+    """
+    Authenticated users should only see documents of audience
+    from requested services : 'services' parameter.
+    Raise 400 error if not all requested services are active.
+    """
+    setup_oicd_resource_server(responses, settings, sub="user_sub", audience="client")
+    token = build_authorization_bearer()
+
+    factories.ServiceFactory(name="test-index", client_id="client", is_active=False)
+    factories.ServiceFactory(name="test-index-b", client_id="b-client")
+
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*", "services": ["test-index"]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 400
+    assert response.json() == {"detail": "Invalid request."}
+
+    # Event without explicit argument, the client service from the request is not active
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*"},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 400
+    assert response.json() == {"detail": "Invalid request."}
+
+
+@responses.activate
+def test_api_documents_search_access__authenticated(settings):
+    """
+    Authenticated users should only see documents
+    - for which they are listed in the "users" field
+    - that have a reach set to "authenticated" or "public" AND in visited list
+    - only configured services providers are allowed (e.g docs)
+    (groups is not yet implemnted)
+    """
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    setup_oicd_resource_server(responses, settings, sub="user_sub", audience="docs")
+    token = build_authorization_bearer()
+
+    service = factories.ServiceFactory(client_id="docs")
+
+    documents_open = factories.DocumentSchemaFactory.build_batch(
+        2, reach=enums.ReachEnum.PUBLIC
+    ) + factories.DocumentSchemaFactory.build_batch(
+        2, reach=enums.ReachEnum.AUTHENTICATED
+    )
+
+    documents_restricted = factories.DocumentSchemaFactory.build_batch(
+        2, reach=enums.ReachEnum.RESTRICTED
+    )
+
+    documents_user = factories.DocumentSchemaFactory.build_batch(
+        6, users=["user_sub", "user_sub2"]
+    )
+    documents = documents_user + documents_open + documents_restricted
+
+    prepare_index(
+        service.index_name, documents_user + documents_open + documents_restricted
+    )
+
+    # Only owned documents (reach is ignored)
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*"},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 200
+
+    assert sorted([d["_id"] for d in response.json()]) == sorted(
+        [doc["id"] for doc in documents_user]
+    )
+
+    # Owned documents and visited public/authenticated ones.
+    # Restricted ones from another owner are filtered (even if given as visited ones)
+    response = APIClient().post(
+        "/api/v1.0/documents/search/",
+        {"q": "*", "visited": [d["id"] for d in documents]},
+        format="json",
+        HTTP_AUTHORIZATION=f"Bearer {token}",
+    )
+
+    assert response.status_code == 200
+
+    assert sorted([d["_id"] for d in response.json()]) == sorted(
+        [doc["id"] for doc in documents_user + documents_open]
+    )

src/backend/core/tests/test_embedding.py

@@ -0,0 +1,125 @@
+"""
+Test suite for opensearch embedding service
+"""
+
+import logging
+from json import dumps as json_dumps
+
+import pytest
+import responses
+
+from core.services.embedding import embed_text
+from core.services.opensearch import check_hybrid_search_enabled, opensearch_client
+
+from .mock import albert_embedding_response
+from .utils import (
+    check_hybrid_search_enabled as check_hybrid_search_enabled_utils,
+)
+from .utils import (
+    enable_hybrid_search,
+)
+
+pytestmark = pytest.mark.django_db
+
+
+@pytest.fixture(autouse=True)
+def before_each():
+    """Clear caches before each test"""
+    clear_caches()
+    yield
+    clear_caches()
+
+
+def clear_caches():
+    """Clear caches used in opensearch service and factories"""
+    check_hybrid_search_enabled.cache_clear()
+    # the instance of check_hybrid_search_enabled used in utils.py
+    # is different and must be cleared separately
+    check_hybrid_search_enabled_utils.cache_clear()
+    opensearch_client().indices.delete(index="*")
+
+
+@responses.activate
+def test_embed_text_success(settings):
+    """Test embed_text retrieval is successful"""
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    text = "canine pet"
+
+    embedding = embed_text(text)
+
+    assert embedding == albert_embedding_response.response["data"][0]["embedding"]
+
+
+@responses.activate
+def test_embed_401_http_error(settings, caplog):
+    """Test embed_text does not crash and returns None on 401 error"""
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        status=401,
+        body=json_dumps({"message": "Authentication failed."}),
+    )
+    text = "canine pet"
+
+    with caplog.at_level(logging.WARNING):
+        embedding = embed_text(text)
+
+    assert any(
+        "embedding API request failed: 401 Client Error: Unauthorized" in message
+        for message in caplog.messages
+    )
+
+    assert embedding is None
+
+
+@responses.activate
+def test_embed_500_http_error(settings, caplog):
+    """Test embed_text does not crash and returns None on 500 error"""
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        status=500,
+        body=json_dumps({"message": "Internal server error."}),
+    )
+    text = "canine pet"
+
+    with caplog.at_level(logging.WARNING):
+        embedding = embed_text(text)
+
+    assert any(
+        "embedding API request failed: 500 Server Error: Internal Server Error"
+        in message
+        for message in caplog.messages
+    )
+
+    assert embedding is None
+
+
+@responses.activate
+def test_embed_wrong_format(settings, caplog):
+    """Test embed_text does not crash and returns None if api returns a wrong format"""
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json={"wrong": "format"},
+        status=200,
+    )
+    text = "canine pet"
+
+    with caplog.at_level(logging.WARNING):
+        embedding = embed_text(text)
+
+    assert any(
+        "unexpected embedding response format" in message for message in caplog.messages
+    )
+
+    assert embedding is None

src/backend/core/tests/test_indexing.py

@@ -0,0 +1,227 @@
+"""
+Test suite for opensearch indexing service
+"""
+
+import pytest
+import responses
+
+from core.services.indexing import detect_language_code, ensure_index_exists
+from core.services.opensearch import check_hybrid_search_enabled, opensearch_client
+
+from .mock import albert_embedding_response
+from .utils import (
+    check_hybrid_search_enabled as check_hybrid_search_enabled_utils,
+)
+from .utils import (
+    enable_hybrid_search,
+)
+
+pytestmark = pytest.mark.django_db
+
+
+SERVICE_NAME = "test-service"
+
+
+@pytest.fixture(autouse=True)
+def before_each():
+    """Clear caches before each test"""
+    clear_caches()
+    yield
+    clear_caches()
+
+
+def clear_caches():
+    """Clear caches used in opensearch service and factories"""
+    check_hybrid_search_enabled.cache_clear()
+    # the instance of check_hybrid_search_enabled used in utils.py
+    # is different and must be cleared separately
+    check_hybrid_search_enabled_utils.cache_clear()
+    opensearch_client().indices.delete(index="*")
+
+
+@pytest.mark.parametrize(
+    "text, analyzer_name, expected_language_analyzer_tokens, expected_trigram_analyzer_tokens",
+    [
+        (
+            "l'éléphant a couru avec les Gens",
+            "french_analyzer",
+            # lowercase is applied ("Gens" -> "gens")
+            # asciifolding is applied ("éléphant" -> "elephant")
+            # stop words are removed ('avec', 'les')
+            # elisions are removed ("l'")
+            # stemming is applied ("gens" -> "gen")
+            ["elephant", "a", "couru", "gen"],
+            # lowercase is applied ("Gens" -> "gens")
+            # asciifolding is applied ("éléphant" -> "elephant")
+            # words smaller than 3 characters are removed ("a")
+            # trigrams are generated
+            [
+                "l'e",
+                "'el",
+                "ele",
+                "lep",
+                "eph",
+                "pha",
+                "han",
+                "ant",
+                "cou",
+                "our",
+                "uru",
+                "ave",
+                "vec",
+                "les",
+                "gen",
+                "ens",
+            ],
+        ),
+        (
+            "The Elephant is running into a café",
+            "english_analyzer",
+            # lowercase is applied ("Elephant" -> "elephant")
+            # asciifolding is applied ("café" -> "cafe")
+            # stop words are removed ("The", "into", "a")
+            # stemming is applied ("running" -> "run", "elephant" -> "eleph")
+            ["eleph", "run", "cafe"],
+            # lowercase is applied ("Gens" -> "gens")
+            # asciifolding is applied ("café" -> "cafe")
+            # trigrams are generated
+            # words smaller than 3 characters are removed ("a")
+            [
+                "the",
+                "ele",
+                "lep",
+                "eph",
+                "pha",
+                "han",
+                "ant",
+                "run",
+                "unn",
+                "nni",
+                "nin",
+                "ing",
+                "int",
+                "nto",
+                "caf",
+                "afe",
+            ],
+        ),
+        (
+            "Der Käfer läuft über die Straße",
+            "german_analyzer",
+            # lowercase is applied ("Der" -> "der", "Käfer" -> "käfer", "Straße" -> "straße")
+            # asciifolding is applied ("käfer" -> "kafer", "straße" -> "strass")
+            # stop words are removed ("Der", "die")
+            # stemming is applied ("kafer" -> "kaf")
+            ["kaf", "lauft", "uber", "strass"],
+            # lowercase is applied
+            # asciifolding is applied ("käfer" -> "kafer", "straße" -> "strasse")
+            # trigrams are generated
+            [
+                "der",
+                "kaf",
+                "afe",
+                "fer",
+                "lau",
+                "auf",
+                "uft",
+                "ube",
+                "ber",
+                "die",
+                "str",
+                "tra",
+                "ras",
+                "ass",
+                "sse",
+            ],
+        ),
+        (
+            "De Kinderen lopen naar de bakkerij",
+            "dutch_analyzer",
+            # lowercase is applied ("De" -> "de", "Kinderen" -> "kinderen")
+            # stop words are removed ("De", "naar", "de")
+            # stemming is applied ("kinderen" -> "kinder", "lopen" -> "lop")
+            ["kinder", "lop", "bakkerij"],
+            # lowercase is applied
+            # words smaller than 3 characters are removed ("de")
+            # trigrams are generated
+            [
+                "kin",
+                "ind",
+                "nde",
+                "der",
+                "ere",
+                "ren",
+                "lop",
+                "ope",
+                "pen",
+                "naa",
+                "aar",
+                "bak",
+                "akk",
+                "kke",
+                "ker",
+                "eri",
+                "rij",
+            ],
+        ),
+    ],
+)
+@responses.activate
+def test_opensearch_analyzers(
+    settings,
+    text,
+    analyzer_name,
+    expected_language_analyzer_tokens,
+    expected_trigram_analyzer_tokens,
+):
+    """Test the analyzers are correctly configured in OpenSearch"""
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    ensure_index_exists(SERVICE_NAME)
+
+    language_analyzer_response = opensearch_client().indices.analyze(
+        index=SERVICE_NAME,
+        body={
+            "analyzer": analyzer_name,
+            "text": text,
+        },
+    )
+    language_analyzer_tokens = [
+        token_info["token"] for token_info in language_analyzer_response["tokens"]
+    ]
+    response_trigram_analyzer = opensearch_client().indices.analyze(
+        index=SERVICE_NAME,
+        body={
+            "analyzer": "trigram_analyzer",
+            "text": text,
+        },
+    )
+    trigram_analyzer_tokens = [
+        token_info["token"] for token_info in response_trigram_analyzer["tokens"]
+    ]
+
+    assert expected_language_analyzer_tokens == language_analyzer_tokens
+    assert expected_trigram_analyzer_tokens == trigram_analyzer_tokens
+
+
+@pytest.mark.parametrize(
+    "text, expected_language_code",
+    [
+        ("This is a test sentence.", "en"),
+        ("Ceci est une phrase de test.", "fr"),
+        ("Dies ist ein Testsatz.", "de"),
+        ("Dit is een testzin.", "nl"),
+        ("Esta es una oración de prueba.", "und"),  # Spanish, unsupported
+        ("", "und"),
+        ("zefk,l", "und"),
+    ],
+)
+def test_detect_language_code(text, expected_language_code):
+    """Test detect_language_code function"""
+
+    assert detect_language_code(text) == expected_language_code

src/backend/core/tests/test_models_services.py

@@ -0,0 +1,42 @@
+"""Tests Service model for find's core app."""
+
+from django.db import DataError, IntegrityError
+
+import pytest
+
+from core import factories
+
+pytestmark = pytest.mark.django_db
+
+
+def test_models_services_name_unique():
+    """The name field should be unique across services."""
+    service = factories.ServiceFactory()
+
+    with pytest.raises(IntegrityError):
+        factories.ServiceFactory(name=service.name)
+
+
+def test_models_services_name_slugified(settings):
+    """The name field should be slugified."""
+    service = factories.ServiceFactory(name="My service name")
+    assert service.name == "my-service-name"
+    assert service.index_name == f"{settings.OPENSEARCH_INDEX_PREFIX}-my-service-name"
+
+
+def test_models_services_token_50_characters_exact():
+    """The token field should be 50 characters long."""
+    service = factories.ServiceFactory()
+    assert len(service.token) == 50
+
+
+def test_models_services_token_50_characters_less():
+    """The token field should not be less than 50 characters long."""
+    with pytest.raises(IntegrityError):
+        factories.ServiceFactory(token="a" * 49)
+
+
+def test_models_services_token_50_characters_more():
+    """The token field should be 50 characters long."""
+    with pytest.raises(DataError):
+        factories.ServiceFactory(token="a" * 51)

src/backend/core/tests/test_schemas.py

@@ -0,0 +1,26 @@
+"""Test pydantic models & helpers"""
+
+import pytest
+
+from core.schemas import cleanlist
+
+
+def test_cleanlist_empty():
+    """Empty data should return an empty list"""
+    assert cleanlist(None) == []
+    assert cleanlist([]) == []
+    assert cleanlist("") == []
+
+
+def test_cleanlist_error():
+    """Invalid data should raise"""
+    with pytest.raises(ValueError):
+        cleanlist(123)
+
+
+def test_cleanlist():
+    """Should return a list of stripped strings and remove the empty items"""
+    assert cleanlist([1, 2, 3]) == ["1", "2", "3"]
+    assert cleanlist("  1,  2,3   ") == ["1", "2", "3"]
+    assert cleanlist(["1 ", "  2", "3 "]) == ["1", "2", "3"]
+    assert cleanlist([None, 2, 3, ""]) == ["2", "3"]

src/backend/core/tests/test_search.py

@@ -0,0 +1,714 @@
+"""
+Test suite for opensearch search service
+"""
+
+import logging
+import operator
+from json import dumps as json_dumps
+
+import pytest
+import responses
+
+from core import enums, factories
+from core.services import opensearch
+from core.services.opensearch import check_hybrid_search_enabled, opensearch_client
+from core.services.search import search
+from core.utils import bulk_create_documents, delete_search_pipeline, prepare_index
+
+from .mock import albert_embedding_response
+from .utils import (
+    check_hybrid_search_enabled as check_hybrid_search_enabled_utils,
+)
+from .utils import (
+    enable_hybrid_search,
+)
+
+pytestmark = pytest.mark.django_db
+
+SERVICE_NAME = "test-service"
+
+
+def search_params(service):
+    """Build opensearch.search() parameters for tests using the service index name"""
+    return {
+        "nb_results": 20,
+        "order_by": "relevance",
+        "order_direction": "desc",
+        "search_indices": {service.index_name},
+        "reach": None,
+        "user_sub": "user_sub",
+        "groups": [],
+        "visited": [],
+        "tags": [],
+        "search_type": enums.SearchTypeEnum.HYBRID,
+    }
+
+
+@pytest.fixture(autouse=True)
+def before_each():
+    """Clear caches and delete search pipeline before each test"""
+    clear_caches()
+    yield
+    clear_caches()
+
+
+def clear_caches():
+    """Clear caches used in opensearch service and factories"""
+    check_hybrid_search_enabled.cache_clear()
+    # the instance of check_hybrid_search_enabled used in utils.py
+    # is different and must be cleared separately
+    check_hybrid_search_enabled_utils.cache_clear()
+    delete_search_pipeline()
+    opensearch_client().indices.delete(index="*")
+
+
+@responses.activate
+def test_hybrid_search_success(settings, caplog):
+    """Test the hybrid search is successful"""
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+    prepare_index(service.index_name, documents)
+
+    q = "canine pet"
+    with caplog.at_level(logging.INFO):
+        result = search(q=q, **search_params(service))
+
+    assert any(
+        f"Performing hybrid search with embedding: {q}" in message
+        for message in caplog.messages
+    )
+
+    assert result["hits"]["max_score"] > 0.0
+    # hybrid search always returns a response of fixed sized sorted and scored by relevance
+    assert {hit["_source"]["title.en"] for hit in result["hits"]["hits"]} == {
+        doc["title"] for doc in documents
+    }
+
+
+@responses.activate
+def test_hybrid_search_without_embedded_index(settings, caplog):
+    """Test the hybrid search is successful"""
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves"},
+            {"title": "dog", "content": "dogs"},
+            {"title": "cat", "content": "cats"},
+        ]
+    )
+    # index is prepared but hybrid search is not yet enable.
+    # they then won't be embedded.
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+    prepare_index(service.index_name, documents)
+
+    # check embedding is None
+    indexed_documents = opensearch.opensearch_client().search(
+        index=service.index_name, body={"query": {"match_all": {}}}
+    )
+    assert indexed_documents["hits"]["hits"][0]["_source"]["chunks"] is None
+
+    # hybrid search is enabled before to do the first requests
+    enable_hybrid_search(settings)
+
+    q = "canine pet"
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    with caplog.at_level(logging.INFO):
+        result = search(q=q, **search_params(service))
+
+    # the hybrid search is done successfully
+    assert any(
+        f"Performing hybrid search with embedding: {q}" in message
+        for message in caplog.messages
+    )
+
+    # but no match can obviously be found
+    assert result["hits"]["max_score"] == 0.0
+    assert len(result["hits"]["hits"]) == 0
+
+    # The full-text search is still functional
+    q = "wolf"
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    with caplog.at_level(logging.INFO):
+        result = search(q=q, **search_params(service))
+
+    assert any(
+        f"Performing hybrid search with embedding: {q}" in message
+        for message in caplog.messages
+    )
+
+    assert result["hits"]["max_score"] > 0.0
+    assert len(result["hits"]["hits"]) == 1
+    assert (
+        result["hits"]["hits"][0]["_source"][
+            f"title.{settings.UNDETERMINED_LANGUAGE_CODE}"
+        ]
+        == q
+    )
+
+
+def test_fall_back_on_full_text_search_if_hybrid_search_disabled(settings, caplog):
+    """Test the full-text search is done when HYBRID_SEARCH_ENABLED=False"""
+    enable_hybrid_search(settings)
+    settings.HYBRID_SEARCH_ENABLED = False
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+    prepare_index(service.index_name, documents)
+
+    q = "wolf"
+    with caplog.at_level(logging.INFO):
+        result = search(q=q, **search_params(service))
+
+    assert any(
+        "Hybrid search is disabled via HYBRID_SEARCH_ENABLED setting" in message
+        for message in caplog.messages
+    )
+    assert any(
+        f"Performing full-text search without embedding: {q}" in message
+        for message in caplog.messages
+    )
+
+    assert result["hits"]["max_score"] > 0.0
+    assert len(result["hits"]["hits"]) == 1
+    assert result["hits"]["hits"][0]["_source"]["title.en"] == "wolf"
+
+
+@responses.activate
+def test_force_full_text_search_with_search_type_parameter(settings, caplog):
+    """Test the full-text search is done when search_type=FULL_TEXT even if hybrid is enabled"""
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+    prepare_index(service.index_name, documents)
+
+    q = "wolf"
+    with caplog.at_level(logging.INFO):
+        result = search(
+            q=q,
+            **{**search_params(service), "search_type": enums.SearchTypeEnum.FULL_TEXT},
+        )
+
+    assert any(
+        f"Performing full-text search without embedding: {q}" in message
+        for message in caplog.messages
+    )
+
+    assert result["hits"]["max_score"] > 0.0
+    assert len(result["hits"]["hits"]) == 1
+    assert result["hits"]["hits"][0]["_source"]["title.en"] == "wolf"
+
+
+def test_request_hybrid_search_when_server_has_it_disabled(settings, caplog):
+    """Test warning when hybrid search is requested but disabled on server"""
+    enable_hybrid_search(settings)
+    settings.HYBRID_SEARCH_ENABLED = False
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+    prepare_index(service.index_name, documents)
+
+    q = "canine pet"
+    with caplog.at_level(logging.INFO):
+        search(
+            q=q,
+            **{**search_params(service), "search_type": enums.SearchTypeEnum.HYBRID},
+        )
+
+    assert any(
+        "Hybrid search was requested (search_type=hybrid) but is disabled on server"
+        in message
+        for message in caplog.messages
+    )
+    assert any(
+        f"Performing full-text search without embedding: {q}" in message
+        for message in caplog.messages
+    )
+
+
+@responses.activate
+def test_api_documents_search_with_search_type_hybrid(settings, caplog):
+    """Test API with search_type=hybrid uses hybrid search when enabled"""
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    service = factories.ServiceFactory()
+
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+        ]
+    )
+    prepare_index(service.index_name, documents)
+
+    q = "canine pet"
+    with caplog.at_level(logging.INFO):
+        result = search(
+            q=q,
+            **{**search_params(service), "search_type": enums.SearchTypeEnum.HYBRID},
+        )
+
+    assert any(
+        f"Performing hybrid search with embedding: {q}" in message
+        for message in caplog.messages
+    )
+
+    assert result["hits"]["max_score"] > 0.0
+    # hybrid search always returns a response of fixed sized sorted and scored by relevance
+    assert {hit["_source"]["title.en"] for hit in result["hits"]["hits"]} == {
+        doc["title"] for doc in documents
+    }
+
+
+@responses.activate
+def test_fall_back_on_full_text_search_if_embedding_api_fails(settings, caplog):
+    """Test the full-text search is done when the embedding api fails"""
+
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        status=401,
+        body=json_dumps({"message": "Authentication failed."}),
+    )
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+    prepare_index(service.index_name, documents)
+
+    q = "wolf"
+    with caplog.at_level(logging.INFO):
+        result = search(q=q, **search_params(service))
+
+    assert any(
+        "embedding API request failed: 401 Client Error: Unauthorized" in message
+        for message in caplog.messages
+    )
+    assert any(
+        f"Performing full-text search without embedding: {q}" in message
+        for message in caplog.messages
+    )
+    assert result["hits"]["max_score"] > 0.0
+    assert len(result["hits"]["hits"]) == 1
+    assert result["hits"]["hits"][0]["_source"]["title.en"] == "wolf"
+
+
+@responses.activate
+def test_fall_back_on_full_text_search_if_variable_are_missing(settings, caplog):
+    """Test the full-text search is done when variables are missing for hybrid search"""
+    enable_hybrid_search(settings)
+    del settings.HYBRID_SEARCH_WEIGHTS
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+    prepare_index(service.index_name, documents)
+
+    q = "wolf"
+    with caplog.at_level(logging.INFO):
+        result = search(q=q, **search_params(service))
+
+    assert any(
+        "Missing variables for hybrid search: HYBRID_SEARCH_WEIGHTS" in message
+        for message in caplog.messages
+    )
+    assert any(
+        f"Performing full-text search without embedding: {q}" in message
+        for message in caplog.messages
+    )
+    assert result["hits"]["max_score"] > 0.0
+    assert len(result["hits"]["hits"]) == 1
+    assert result["hits"]["hits"][0]["_source"]["title.en"] == "wolf"
+
+
+@responses.activate
+def test_match_all(settings, caplog):
+    """Test match all when q='*' and no semantic search is needed"""
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+    prepare_index(service.index_name, documents)
+
+    q = "*"
+    with caplog.at_level(logging.INFO):
+        result = search(q=q, **search_params(service))
+
+    assert any("Performing match_all query" in message for message in caplog.messages)
+    assert result["hits"]["max_score"] > 0.0
+    assert len(result["hits"]["hits"]) == 3
+
+
+@responses.activate
+def test_search_ordering_by_relevance(settings, caplog):
+    """Test the hybrid supports ordering by relevance asc and desc"""
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    q = "canine pet"
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+    prepare_index(service.index_name, documents)
+
+    for direction in ["asc", "desc"]:
+        with caplog.at_level(logging.INFO):
+            result = search(
+                q=q, **{**search_params(service), "order_direction": direction}
+            )
+
+        # Check that results are sorted by score as expected
+        hits = result["hits"]["hits"]
+        compare = operator.le if direction == "asc" else operator.ge
+        for i in range(len(hits) - 1):
+            assert compare(hits[i]["_score"], hits[i + 1]["_score"])
+
+
+@responses.activate
+def test_hybrid_search_number_of_matches(settings):
+    """
+    In this test full-text search always return 0 documents.
+    The test checks the number of hits returned by hybrid search with different k values.
+    """
+    enable_hybrid_search(settings)
+    responses.add(
+        responses.POST,
+        settings.EMBEDDING_API_PATH,
+        json=albert_embedding_response.response,
+        status=200,
+    )
+
+    documents = bulk_create_documents(
+        [
+            {"title": "wolf", "content": "wolves live in packs and hunt together"},
+            {"title": "dog", "content": "dogs are loyal domestic animals"},
+            {"title": "cat", "content": "cats are curious and independent pets"},
+        ]
+    )
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+    prepare_index(service.index_name, documents)
+
+    q = "pony"  # full-text matches 0 document
+    for nb_results in [1, 2, 3]:  # semantic should match k documents
+        result = search(q=q, **{**search_params(service), "nb_results": nb_results})
+        assert len(result["hits"]["hits"]) == nb_results
+
+
+def test_search_filtering_by_single_tag():
+    """Test filtering documents by a single tag"""
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+
+    documents_to_search = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["tag-to-search"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["tag-to-search", "tag-to-filter"]
+        ),
+    ]
+    documents_to_filter = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["tag-to-filter"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+        ),
+    ]
+    expected_ids = {str(doc["id"]) for doc in documents_to_search}
+
+    prepare_index(service.index_name, documents_to_search + documents_to_filter)
+
+    # Search for documents with tag-to-search tag
+    result = search(q="*", **{**search_params(service), "tags": ["tag-to-search"]})
+    returned_ids = {hit["_id"] for hit in result["hits"]["hits"]}
+
+    assert result["hits"]["total"]["value"] == len(documents_to_search)
+    assert returned_ids == expected_ids
+
+
+def test_search_filtering_by_multiple_tags():
+    """Test filtering documents by multiple tags (OR logic)"""
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+
+    documents_to_search = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["tag-to-search-1"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["tag-to-search-1", "tag-to-filter"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["tag-to-search-2", "tag-to-filter"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["tag-to-search-1", "tag-to-search-2"]
+        ),
+    ]
+    documents_to_filter = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], tags=["tag-to-filter"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+        ),
+    ]
+    expected_ids = {str(doc["id"]) for doc in documents_to_search}
+
+    prepare_index(service.index_name, documents_to_search + documents_to_filter)
+
+    # Search for documents with tag-to-search-1 OR tag-to-search-2 tags
+    result = search(
+        q="*",
+        **{**search_params(service), "tags": ["tag-to-search-1", "tag-to-search-2"]},
+    )
+    returned_ids = {hit["_id"] for hit in result["hits"]["hits"]}
+
+    assert result["hits"]["total"]["value"] == len(documents_to_search)
+    assert returned_ids == expected_ids
+
+
+def test_search_no_tags_filter_returns_all():
+    """Test that not providing tags filter returns all documents"""
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+
+    documents = bulk_create_documents(
+        [
+            {
+                "title": "Document without tags",
+                "content": "Untagged document",
+            },
+            {
+                "title": "Document with tags",
+                "content": "Tagged document",
+                "tags": ["tag-to-search"],
+            },
+        ]
+    )
+    prepare_index(service.index_name, documents)
+
+    # Search without tags filter
+    result = search(q="*", **search_params(service))
+
+    assert result["hits"]["total"]["value"] == len(documents)
+
+
+def test_search_filtering_by_tag_and_query():
+    """Test filtering documents by both tag and query text"""
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+
+    documents_to_search = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], title="title to search", tags=["tag-to-search"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+            title="title to search 1",
+            tags=["tag-to-search", "tag-to-filter"],
+        ),
+    ]
+    documents_to_filter = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], title="title to filter", tags=["tag-to-filter"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], title="title to filter 1", tags=["tag-to-filter"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+            title="title to search 2",
+        ),
+    ]
+    expected_ids = {str(doc["id"]) for doc in documents_to_search}
+
+    prepare_index(service.index_name, documents_to_search + documents_to_filter)
+
+    # Search with both query and tag filter
+    result = search(q="search", **{**search_params(service), "tags": ["tag-to-search"]})
+    returned_ids = {hit["_id"] for hit in result["hits"]["hits"]}
+
+    assert result["hits"]["total"]["value"] == len(documents_to_search)
+    assert returned_ids == expected_ids
+
+
+def test_search_filtering_by_path():
+    """Test filtering documents by path prefix"""
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+
+    documents_to_search = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], path="path/to/search/doc-1"
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], path="path/to/search/doc-2"
+        ),
+    ]
+    documents_to_filter = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], path="path/to/filter/doc-3"
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+        ),
+    ]
+    expected_ids = {str(doc["id"]) for doc in documents_to_search}
+
+    prepare_index(service.index_name, documents_to_search + documents_to_filter)
+
+    path_filter = "path/to/search"
+    result = search(q="*", **{**search_params(service), "path": path_filter})
+    returned_ids = {hit["_id"] for hit in result["hits"]["hits"]}
+
+    assert result["hits"]["total"]["value"] == len(documents_to_search)
+    assert returned_ids == expected_ids
+
+
+def test_search_filtering_by_query_path_and_tag():
+    """Test filtering documents by query text, path prefix and tag combined"""
+    service = factories.ServiceFactory(name=SERVICE_NAME)
+
+    documents_to_search = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+            title="title to search 0",
+            path="path/to/search-0",
+            tags=["tag-to-search"],
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+            title="title to search 1",
+            path="path/to/search/doc1",
+            tags=["tag-to-search", "tag-to-filter"],
+        ),
+    ]
+    documents_to_filter = [
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+            title="title to filter",
+            path="path/to/search/doc-3",
+            tags=["tag-to-search"],
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+            title="title to search 4",
+            path="path/to/filter/doc-4",
+            tags=["tag-to-search"],
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+            title="title to search 4",
+            path="path/to/search/doc-4",
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"], title="title to search 5", tags=["tag-to-search"]
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+            title="title to search 6",
+            path="path/to/search/doc-6",
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+            title="title to search 7",
+        ),
+        factories.DocumentSchemaFactory.build(
+            users=["user_sub"],
+            title="",
+            path="path/to/search/doc-8",
+            tags=["tag-to-search"],
+        ),
+    ]
+    expected_ids = {str(doc["id"]) for doc in documents_to_search}
+
+    prepare_index(service.index_name, documents_to_search + documents_to_filter)
+
+    # Search with query, path and tag filters combined
+    result = search(
+        q="search",
+        **{
+            **search_params(service),
+            "path": "path/to/search",
+            "tags": ["tag-to-search"],
+        },
+    )
+    returned_ids = {hit["_id"] for hit in result["hits"]["hits"]}
+
+    assert result["hits"]["total"]["value"] == len(documents_to_search)
+    assert returned_ids == expected_ids

src/backend/core/tests/test_selftests.py

@@ -0,0 +1,145 @@
+"""Tests for the selftest system."""
+
+from core.selftests import SelfTest, SelfTestRegistry, SelfTestResult
+
+
+class DummySuccessTest(SelfTest):
+    """A test that always succeeds."""
+
+    name = "Dummy Success Test"
+    description = "A test that always passes"
+
+    def run(self) -> SelfTestResult:
+        return SelfTestResult(
+            name=self.name,
+            success=True,
+            message="Test passed successfully",
+            duration_ms=10.0,
+        )
+
+
+class DummyFailureTest(SelfTest):
+    """A test that always fails."""
+
+    name = "Dummy Failure Test"
+    description = "A test that always fails"
+
+    def run(self) -> SelfTestResult:
+        return SelfTestResult(
+            name=self.name,
+            success=False,
+            message="Test failed as expected",
+            duration_ms=5.0,
+        )
+
+
+class DummyExceptionTest(SelfTest):
+    """A test that raises an exception."""
+
+    name = "Dummy Exception Test"
+    description = "A test that throws an exception"
+
+    def run(self) -> SelfTestResult:
+        raise RuntimeError("This test is designed to fail")
+
+
+def test_register_test():
+    """Test that a test can be registered."""
+    registry = SelfTestRegistry()
+    registry.register(DummySuccessTest)
+
+    tests = registry.get_all_tests()
+
+    assert len(tests) == 1
+    assert isinstance(tests[0], DummySuccessTest)
+
+
+def test_register_multiple_tests():
+    """Test that multiple tests can be registered."""
+    registry = SelfTestRegistry()
+    registry.register(DummySuccessTest)
+    registry.register(DummyFailureTest)
+
+    tests = registry.get_all_tests()
+
+    assert len(tests) == 2
+
+
+def test_unregister_test():
+    """Test that a test can be unregistered."""
+    registry = SelfTestRegistry()
+    registry.register(DummySuccessTest)
+    registry.unregister(DummySuccessTest)
+
+    tests = registry.get_all_tests()
+
+    assert len(tests) == 0
+
+
+def test_run_all_success():
+    """Test running all tests when all pass."""
+    registry = SelfTestRegistry()
+    registry.register(DummySuccessTest)
+
+    results = registry.run_all()
+
+    assert len(results) == 1
+    assert results[0].success is True
+
+
+def test_run_all_mixed():
+    """Test running all tests with mixed results."""
+    registry = SelfTestRegistry()
+    registry.register(DummySuccessTest)
+    registry.register(DummyFailureTest)
+
+    results = registry.run_all()
+
+    assert len(results) == 2
+    assert results[0].success is True
+    assert results[1].success is False
+
+
+def test_run_all_with_exception():
+    """Test that exceptions are caught and converted to failures."""
+    registry = SelfTestRegistry()
+    registry.register(DummyExceptionTest)
+
+    results = registry.run_all()
+
+    assert len(results) == 1
+    assert results[0].success is False
+    assert "exception" in results[0].message.lower()
+
+
+def test_result_to_dict():
+    """Test converting a result to a dictionary."""
+    result = SelfTestResult(
+        name="Test Name",
+        success=True,
+        message="Test message",
+        details={"key": "value"},
+        duration_ms=100.5,
+    )
+
+    result_dict = result.to_dict()
+
+    assert result_dict["name"] == "Test Name"
+    assert result_dict["success"] is True
+    assert result_dict["message"] == "Test message"
+    assert result_dict["details"]["key"] == "value"
+    assert result_dict["duration_ms"] == 100.5
+
+
+def test_result_without_optional_fields():
+    """Test creating a result without optional fields."""
+    result = SelfTestResult(
+        name="Test Name",
+        success=False,
+        message="Test failed",
+    )
+
+    result_dict = result.to_dict()
+
+    assert result_dict["details"] == {}
+    assert result_dict["duration_ms"] is None

src/backend/core/tests/utils.py

@@ -0,0 +1,94 @@
+"""Utility functions for Test."""
+
+import base64
+import json
+import logging
+from functools import partial
+
+from core.management.commands.create_search_pipeline import (
+    ensure_search_pipeline_exists,
+)
+from core.services.opensearch import (
+    check_hybrid_search_enabled,
+)
+
+logger = logging.getLogger(__name__)
+
+
+def enable_hybrid_search(settings):
+    """Enable hybrid search settings for tests."""
+    settings.HYBRID_SEARCH_ENABLED = True
+    settings.HYBRID_SEARCH_WEIGHTS = [0.3, 0.7]
+    settings.EMBEDDING_API_KEY = "test-api-key"
+    settings.EMBEDDING_API_PATH = "https://test.embedding.api/v1/embeddings"
+    settings.EMBEDDING_REQUEST_TIMEOUT = 10
+    settings.EMBEDDING_API_MODEL_NAME = "embeddings-small"
+    settings.EMBEDDING_DIMENSION = 1024
+
+    # Clear the cache here or the hybrid search will remain disabled
+    check_hybrid_search_enabled.cache_clear()
+    ensure_search_pipeline_exists()
+
+
+def build_authorization_bearer(token="some_token"):
+    """
+    Build an Authorization Bearer header value from a token.
+
+    This can be used like this:
+    client.post(
+        ...
+        HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer('some_token')}",
+    )
+    """
+    return base64.b64encode(token.encode("utf-8")).decode("utf-8")
+
+
+def setup_oicd_resource_server(
+    responses,
+    settings,
+    sub="some_sub",
+    audience="some_client_id",
+    introspect=None,
+):  # pylint: disable=too-many-arguments
+    """
+    Setup settings for a resource server.
+    Simulate a token introspection.
+    NOTE : Use it with @responses.activate or the fake introspection view will not work.
+    """
+    token_data = {
+        "sub": sub,
+        "iss": "https://oidc.example.com",
+        "aud": audience,
+        "client_id": audience,
+        "scope": "docs",
+        "active": True,
+    }
+
+    settings.OIDC_RS_ENCRYPTION_KEY_TYPE = "RSA"
+    settings.OIDC_RS_ENCRYPTION_ENCODING = "A256GCM"
+    settings.OIDC_RS_ENCRYPTION_ALGO = "RSA-OAEP"
+    settings.OIDC_RS_SIGNING_ALGO = "RS256"
+    settings.OIDC_RS_CLIENT_ID = audience
+    settings.OIDC_RS_CLIENT_SECRET = "some_client_secret"
+    settings.OIDC_RS_SCOPES = ["openid", "docs", "email"]
+
+    settings.OIDC_OP_URL = "https://oidc.example.com"
+    settings.OIDC_OP_INTROSPECTION_ENDPOINT = "https://oidc.example.com/introspect"
+
+    settings.OIDC_VERIFY_SSL = False
+    settings.OIDC_TIMEOUT = 5
+    settings.OIDC_PROXY = None
+    settings.OIDC_CREATE_USER = False
+
+    if callable(introspect):
+        responses.add_callback(
+            responses.POST,
+            settings.OIDC_OP_INTROSPECTION_ENDPOINT,
+            callback=partial(introspect, user_info=token_data),
+        )
+    else:
+        responses.add(
+            responses.POST,
+            settings.OIDC_OP_INTROSPECTION_ENDPOINT,
+            body=json.dumps(token_data),
+        )