From 52ebc6b6fb2ac05fcfdd0b0bdd1f0f1b60f4fb1f Mon Sep 17 00:00:00 2001 From: Fabre Florian Date: Fri, 3 Oct 2025 10:35:50 +0200 Subject: [PATCH] =?UTF-8?q?=F0=9F=93=9D(backend)=20add=20fulltext=20search?= =?UTF-8?q?=20documentation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add documentation for env & Find+Docs configuration in dev mode Signed-off-by: Fabre Florian --- docs/architecture.md | 1 + docs/env.md | 6 ++++++ docs/search.md | 39 +++++++++++++++++++++++++++++++++++++ docs/system-requirements.md | 11 +++++++++++ env.d/development/common | 1 + 5 files changed, 58 insertions(+) create mode 100644 docs/search.md diff --git a/docs/architecture.md b/docs/architecture.md index 230d3245..f858eb01 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -12,6 +12,7 @@ flowchart TD Back --> DB("Database (PostgreSQL)") Back <--> Celery --> DB Back ----> S3("Minio (S3)") + Back -- REST API --> Find ``` ### Architecture decision records diff --git a/docs/env.md b/docs/env.md index 0b3f9b3b..6ccab5e5 100644 --- a/docs/env.md +++ b/docs/env.md @@ -93,6 +93,12 @@ These are the environment variables you can set for the `impress-backend` contai | OIDC_USERINFO_SHORTNAME_FIELD | OIDC token claims to create shortname | first_name | | POSTHOG_KEY | Posthog key for analytics | | | REDIS_URL | Cache url | redis://redis:6379/1 | +| SEARCH_INDEXER_CLASS | Class of the backend for document indexation & search | | +| SEARCH_INDEXER_BATCH_SIZE | Size of each batch for indexation of all documents | 100000 | +| SEARCH_INDEXER_COUNTDOWN | Minimum debounce delay of indexation jobs (in seconds) | 1 | +| SEARCH_INDEXER_URL | Find application endpoint for indexation | | +| SEARCH_INDEXER_SECRET | Token for indexation queries | | +| SEARCH_INDEXER_QUERY_URL | Find application endpoint for search | | | SENTRY_DSN | Sentry host | | | SESSION_COOKIE_AGE | duration of the cookie session | 60*60*12 | | SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK | | false | diff --git a/docs/search.md b/docs/search.md new file mode 100644 index 00000000..63581081 --- /dev/null +++ b/docs/search.md @@ -0,0 +1,39 @@ +# Setup the Find search for Impress + +This configuration will enable the fulltext search feature for Docs : +- Each save on **core.Document** or **core.DocumentAccess** will trigger the indexer +- The `api/v1.0/documents/search/` will work as a proxy with the Find API for fulltext search. + +## Create an index service for Docs + +Configure a **Service** for Docs application with these settings + +- **Name**: `docs`
_request.auth.name of the Docs application._ +- **Client id**: `impress`
_Name of the token audience or client_id of the Docs application._ + +See [how-to-use-indexer.md](how-to-use-indexer.md) for details. + +## Configure settings of Docs + +Add those Django settings the Docs application to enable the feature. + +```shell +SEARCH_INDEXER_CLASS="core.services.search_indexers.FindDocumentIndexer" +SEARCH_INDEXER_COUNTDOWN=10 # Debounce delay in seconds for the indexer calls. + +# The token from service "docs" of Find application (development). +SEARCH_INDEXER_SECRET="find-api-key-for-docs-with-exactly-50-chars-length" +SEARCH_INDEXER_URL="http://find:8000/api/v1.0/documents/index/" + +# Search endpoint. Uses the OIDC token for authentication +SEARCH_INDEXER_QUERY_URL="http://find:8000/api/v1.0/documents/search/" +``` + +We also need to enable the **OIDC Token** refresh or the authentication will fail quickly. + +```shell +# Store OIDC tokens in the session +OIDC_STORE_ACCESS_TOKEN = True # Store the access token in the session +OIDC_STORE_REFRESH_TOKEN = True # Store the encrypted refresh token in the session +OIDC_STORE_REFRESH_TOKEN_KEY = "your-32-byte-encryption-key==" # Must be a valid Fernet key (32 url-safe base64-encoded bytes) +``` diff --git a/docs/system-requirements.md b/docs/system-requirements.md index 8f2daafb..db337d9b 100644 --- a/docs/system-requirements.md +++ b/docs/system-requirements.md @@ -97,6 +97,17 @@ Production deployments differ significantly from development environments. The t | 5433 | PostgreSQL (Keycloak) | | 1081 | MailCatcher | +**With fulltext search service** + +| Port | Service | +| --------- | --------------------- | +| 8081 | Find (Django) | +| 9200 | Opensearch | +| 9600 | Opensearch admin | +| 5601 | Opensearch dashboard | +| 25432 | PostgreSQL (Find) | + + ## 6. Sizing Guidelines **RAM** – start at 8 GB dev / 16 GB staging / 32 GB prod. Postgres and Keycloak are the first to OOM; scale them first. diff --git a/env.d/development/common b/env.d/development/common index 5b1564ad..cdd734b5 100644 --- a/env.d/development/common +++ b/env.d/development/common @@ -36,6 +36,7 @@ OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/c OIDC_OP_AUTHORIZATION_ENDPOINT=http://localhost:8083/realms/impress/protocol/openid-connect/auth OIDC_OP_TOKEN_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/token OIDC_OP_USER_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/userinfo +OIDC_OP_INTROSPECTION_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/token/introspect OIDC_RP_CLIENT_ID=impress OIDC_RP_CLIENT_SECRET=ThisIsAnExampleKeyForDevPurposeOnly