Compare commits

..

7 Commits

Author SHA1 Message Date
charles 5d8506ef16 ⚰️(backend) remove sorting
sorting is not supported with hybrid search.
I should have removed it long ago.
It is not supported also with rerank.
2026-03-17 14:49:04 +01:00
charles 98cf201614 📝(backend) docs
i am adding and improving docs
2026-03-17 14:30:01 +01:00
charles 04ed102575 (backend) add rerank param on search route
We want to be able to disable the reranker at run time
even if it is enabled in settings
2026-03-17 14:30:01 +01:00
charles a25ecfd210 (backend) test reranking function
I am adding test about reranking
2026-03-17 14:30:01 +01:00
charles 6f0d799de8 (backend) implement reranker
I implement a reranker to improve accuracy of search results
2026-03-17 13:46:50 +01:00
charles e1bb25c589 🔧(backend) add flashrank dependency
flashrank is a package for reranking
search results
2026-03-17 13:46:50 +01:00
Charles Englebert 5189db1b2b ♻️(backend) reorganize evaluation data (#59)
I am improving the data management of evaluation system
2026-03-16 09:58:15 +01:00
15 changed files with 610 additions and 171 deletions
+2 -1
View File
@@ -9,7 +9,7 @@ and this project adheres to
# Unreleased
## Added
- ✨(backend) add rescore on `updated_at`
- 👷(docker) add arm64 platform support for image builds
- ✨(backend) add semantic search
- ✨(backend) add multi-embedding and chunking
@@ -31,6 +31,7 @@ and this project adheres to
- ✨(backend) add deletion endpoint
- ✨(backend) add path filter
- ✨(backend) add search_type param
- ✨(backend) implement reranker #47
## Changed
+2
View File
@@ -102,6 +102,8 @@ These are the environment variables you can set for the `find-backend` container
| OPENSEARCH_USE_SSL | Enable SSL connection for Opensearch database | true |
| POSTHOG_KEY | Posthog key for analytics | |
| REDIS_URL | Cache url | redis://redis:6379/1 |
| RERANKER_ENABLED | Flag to enable the re-ranking of search results | false |
| RERANKER_MODEL_NAME | Name of the re-ranking model used | |
| SENTRY_DSN | Sentry host | |
| SESSION_COOKIE_AGE | duration of the cookie session | 60*60*12 |
| SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK | | false |
+49 -31
View File
@@ -1,16 +1,16 @@
# Using the Find indexer
# Setup Find
This guide explains how to setup the Find service which provide an API for indexation and fulltext search of
This guide explains how to setup the Find service which provide an API for indexation and search of
documents from various sources in a secure way : only the documents within the scope of the user's OIDC token
are visible.
## Setup Opensearch
## Core setups
### General
### Opensearch
Add the following variables to your Django settings to configure Find and enable full-text search.
```python
```bash
# Login for opensearch
OPENSEARCH_USER=opensearch-user
OPENSEARCH_PASSWORD=your-opensearch-password
@@ -29,7 +29,7 @@ OPENSEARCH_INDEX_PREFIX=find
### Language
Language specific operations are applied to document titles and contents to improve search results.
The language is automatically detected by Find.
The language is automatically detected by Find.
If the language can not be detected no language specific operation are applied and the indexing process is not affected.
Find supports french, english, german and dutch.
@@ -39,7 +39,7 @@ The search process is not language specific, a query can get documents of any la
Language detection estimates a confidence between 0 and 1. If the confidence is below a threshold the language is considered unrecognized.
This threshold can be controlled with LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD environment variable.
```python
```bash
LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD=0.75
```
@@ -47,49 +47,67 @@ LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD=0.75
Find offers a semantic search feature. You can either use pure full-text search or a hybrid full-text + semantic search. To enable the hybrid search, add the following settings.
```python
```bash
# Enable flag
HYBRID_SEARCH_ENABLED = True
HYBRID_SEARCH_ENABLED=True
# weighted sum: full_text_weight, semantic_search_weight
HYBRID_SEARCH_WEIGHTS = 0.7,0.3
HYBRID_SEARCH_WEIGHTS=0.7,0.3
# Embedding
CHUNK_SIZE=512
CHUNK_OVERLAP=50
EMBEDDING_API_PATH = https://embedding.api.example.com/full/path/
EMBEDDING_API_KEY = your-embedding-api-key
EMBEDDING_REQUEST_TIMEOUT = 10
EMBEDDING_API_MODEL_NAME = embedding-api-model-name
EMBEDDING_DIMENSION = 1024
EMBEDDING_API_PATH=https://embedding.api.example.com/full/path/
EMBEDDING_API_KEY=your-embedding-api-key
EMBEDDING_REQUEST_TIMEOUT=10
EMBEDDING_API_MODEL_NAME=embedding-api-model-name
EMBEDDING_DIMENSION=1024
```
The hybrid search computes a score for full-text and semantic search and combines them through a weighted sum. HYBRID_SEARCH_WEIGHTS contains the weights of full-text and semantic respectively.
You need to use an embedding api similar to https://albert.api.etalab.gouv.fr/documentation#tag/Embeddings/operation/embeddings_v1_embeddings_post.
### document chunking
### Document chunking
The indexing process embeds documents by converting their content into vector representations (embeddings). When a document exceeds the character dimension defined by CHUNK_SIZE, it's divided into smaller segments (chunks), with each chunk embedded independently. Each chunk must be smaller than the embedding model's context window .
The indexing process embeds documents by converting their content into vector representations (embeddings). When a document exceeds the character dimension defined by CHUNK_SIZE, it's divided into smaller segments (chunks), with each chunk embedded independently. Each chunk must be smaller than the embedding model's context window .
```bash
CHUNK_SIZE=512
CHUNK_OVERLAP=50
```
The chunking algorithm works recursively. It attempts to create the largest possible segments first, then subdivides them further if they still exceed the size limit defined by CHUNK_SIZE. Segments can share overlapping content between them (set CHUNK_OVERLAP=0 to disable overlapping).
During the search, the matching of a document is the matching of its best chunk.
## trigrams
### Trigrams
Find uses trigrams to improve the robustness of the full text search engine to spelling variations and errors. It can be configured by two environment variables.
````
```bash
TRIGRAMS_BOOST=0.25
TRIGRAMS_MINIMUM_SHOULD_MATCH=0.75%
````
```
`TRIGRAMS_BOOST` is weight boost applied to the trigram score in the document matching.
`TRIGRAMS_MINIMUM_SHOULD_MATCH` is the minimal number or proportion of trigrams having to match to score. It is
either an absolute number or proportion.
## Setup indexation API
### Reranking
Find offers a reranking feature to reorder the search results based on a cross-encoder method.
To enable the reranking, add the following settings.
```bash
RERANKING_ENABLED=True
RERANKER_MODEL_NAME=ms-marco-MiniLM-L-12-v2
```
Reranking is based on Flashrank. See the [Flashrank documentation](https://github.com/PrithivirajDamodaran/FlashRank?tab=readme-ov-file) to know what models are compatible.
## API setups
### Indexing
Other applications can index their files through the **`/index/`** endpoint with a simple token authentication.
@@ -110,21 +128,21 @@ And add the key in the calling application Django settings.
The command `make demo` will create a working service configuration for `docs` and `drive` with predefined secret keys
```python
```bash
# Docs
SEARCH_INDEXER_SECRET="find-api-key-for-docs-with-exactly-50-chars-length"
SEARCH_INDEXER_SECRET=find-api-key-for-docs-with-exactly-50-chars-length
```
```python
```bash
# Drive
SEARCH_INDEXER_SECRET="find-api-key-for-driv-with-exactly-50-chars-length"
SEARCH_INDEXER_SECRET=find-api-key-for-driv-with-exactly-50-chars-length
```
## Setup search API
### Search
The **`/search/`** endpoint is an OIDC ResourceServer view and needs extra Django settings (see [lasuite](https://github.com/suitenumerique/django-lasuite/blob/main/documentation/how-to-use-oidc-resource-server-backend.md) for details)
```shell
```bash
OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/token
@@ -145,11 +163,11 @@ OIDC_OP_INTROSPECTION_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-
# OIDC_VERIFY_SSL=False
# Resource server
OIDC_RS_SCOPES="openid"
OIDC_RS_SCOPES=openid
OIDC_RS_SIGN_ALGO=RS256
# This backend allows authentication without any model in database.
OIDC_RS_BACKEND_CLASS="core.authentication.FinderResourceServerBackend"
OIDC_RS_BACKEND_CLASS=core.authentication.FinderResourceServerBackend
```
**Development mode (Docs + Find)**
+1 -1
View File
@@ -118,7 +118,7 @@ class SearchQueryParametersSchema(BaseModel):
path: Optional[str] = None
nb_results: Optional[conint(ge=1, le=300)] = Field(default=50)
search_type: Optional[SearchTypeEnum] = Field(default=None)
rescore: bool = Field(default=True)
rerank: Optional[bool] = Field(default=None)
class DeleteDocumentsSchema(BaseModel):
+1 -10
View File
@@ -21,6 +21,7 @@ def embed_text(text):
json={
"input": text,
"model": settings.EMBEDDING_API_MODEL_NAME,
"dimensions": settings.EMBEDDING_DIMENSION,
"encoding_format": "float",
},
timeout=settings.EMBEDDING_REQUEST_TIMEOUT,
@@ -38,14 +39,4 @@ def embed_text(text):
logger.warning("unexpected embedding response format: %s", response.text)
return None
if len(embedding) != settings.EMBEDDING_DIMENSION:
logger.warning(
"unexpected embedding dimension: "
"EMBEDDING_DIMENSION is set to %d "
"but the configured embedding model returned a vector of dimension %d",
settings.EMBEDDING_DIMENSION,
len(embedding),
)
return None
return embedding
+122
View File
@@ -0,0 +1,122 @@
"""Reranking utilities using rerankers library."""
import logging
from functools import cache
from django.conf import settings
from flashrank import Ranker, RerankRequest
from core.services.indexing import format_document
from core.utils import get_language_value
logger = logging.getLogger(__name__)
def rerank(query: str, hits: list[dict]) -> list[dict]:
"""
Rerank search results using the configured reranker model.
Args:
query: The search query string
hits: List of OpenSearch hit objects with _source containing title and content
Returns:
List of reranked results in the same format as input with a _reranked_score
"""
reranker = get_reranker()
if reranker is None:
logger.warning("Could not import reranker, returning original results")
return hits
try:
return _rerank(reranker, query, hits)
except Exception as e: # noqa: BLE001# pylint: disable=broad-exception-caught
logger.error("Reranking failed: %s, returning original results", str(e))
return hits
@cache
def get_reranker() -> Ranker | None:
"""
Get the reranker instance.
Returns None if the reranker library is not available or if initialization fails but
does not raise an exception to avoid crashing the application.
"""
try:
logger.info("Initializing reranker model: %s", settings.RERANKER_MODEL_NAME)
return Ranker(model_name=settings.RERANKER_MODEL_NAME)
except Exception as e: # noqa: BLE001# pylint: disable=broad-exception-caught
logger.error("Failed to initialize reranker: %s", str(e))
return None
def _rerank(ranker: Ranker, query: str, original_hits: list[dict]) -> list[dict]:
"""Rerank the original results using the provided reranker."""
rerank_request = RerankRequest(
query=query, passages=prepare_rerank_data(original_hits)
)
logger.info("Reranking %d results for query: %s", len(original_hits), query)
reranked_results = ranker.rerank(rerank_request)
reranked_hits: list[dict] = []
for reranked_result in reranked_results:
matching_hits = [
hit for hit in original_hits if hit["_id"] == reranked_result["id"]
]
if not matching_hits:
logger.warning(
"Reranked document ID %s not found in original hits, skipping",
reranked_result["id"],
)
continue
if len(matching_hits) > 1:
logger.warning(
"Multiple hits found for document ID %s, using first match",
reranked_result["id"],
)
hit = matching_hits[0]
hit["_reranked_score"] = reranked_result["score"]
reranked_hits.append(hit)
logger.info("Reranking completed, returned %d results", len(reranked_hits))
return reranked_hits
def prepare_rerank_data(original_hits: list[dict]) -> list[dict]:
"""
Prepare the documents for reranking by extracting the title and content from the original hits.
"""
passages = []
for hit in original_hits:
title = get_language_value(hit["_source"], "title")
content = get_language_value(hit["_source"], "content")
passages.append({"id": hit["_id"], "text": format_document(title, content)})
return passages
def should_rerank(is_rerank_requested: bool | None) -> bool:
"""
Determine whether to perform reranking based on the input parameter and settings.
logs warning if reranking was explicitly requested but the reranker is disabled in settings.
falls back to settings.RERANKER_ENABLED if is_rerank_requested is None.
"""
if is_rerank_requested and not settings.RERANKER_ENABLED:
logger.warning(
"Reranking was explicitly requested but the reranker "
"is disabled in settings. Reranking skipped."
)
return False
if is_rerank_requested is None:
return settings.RERANKER_ENABLED
return is_rerank_requested and settings.RERANKER_ENABLED
+11 -34
View File
@@ -9,6 +9,7 @@ from core.enums import SearchTypeEnum
from .embedding import embed_text
from .opensearch import check_hybrid_search_enabled, opensearch_client
from .reranking import rerank, should_rerank
logger = logging.getLogger(__name__)
@@ -25,7 +26,7 @@ def search( # noqa : PLR0913
tags,
search_type,
path=None,
rescore=False,
rerank_requested=None,
):
"""Perform an OpenSearch search"""
query = get_query(
@@ -39,7 +40,7 @@ def search( # noqa : PLR0913
path=path,
search_type=search_type,
)
return opensearch_client().search( # pylint: disable=unexpected-keyword-arg
response = opensearch_client().search( # pylint: disable=unexpected-keyword-arg
index=",".join(search_indices),
body={
"_source": enums.SOURCE_FIELDS, # limit the fields to return
@@ -49,7 +50,6 @@ def search( # noqa : PLR0913
},
"size": nb_results,
"query": query,
"rescore": get_rescore(nb_results=nb_results) if rescore else [],
},
params=get_params(query_keys=query.keys()),
# disable=unexpected-keyword-arg because
@@ -57,6 +57,14 @@ def search( # noqa : PLR0913
ignore_unavailable=True,
)
if should_rerank(rerank_requested) and q != "*":
response["hits"]["hits"] = rerank(
query=q,
hits=response["hits"]["hits"],
)
return response
# pylint: disable=too-many-arguments, too-many-positional-arguments
def get_query( # noqa : PLR0913
@@ -222,37 +230,6 @@ def get_filter( # noqa : PLR0913
return filters
def get_rescore(nb_results):
"""
Build rescore query.
Rescore is based on the `updated_at` field to boost more recently updated documents
"""
return [
{
"window_size": nb_results,
"query": {
"rescore_query_weight": settings.RESCORE_UPDATED_AT_WEIGHT,
"rescore_query": {
"function_score": {
"functions": [
{
"gauss": {
"updated_at": {
"origin": "now",
"offset": settings.RESCORE_UPDATED_AT_OFFSET,
"scale": settings.RESCORE_UPDATED_AT_SCALE,
"decay": settings.RESCORE_UPDATED_AT_DECAY,
}
}
}
],
}
},
},
}
]
def get_params(query_keys):
"""Build OpenSearch search parameters"""
if "hybrid" in query_keys:
@@ -1,7 +1,6 @@
"""Tests indexing documents in OpenSearch over the API"""
import datetime
import logging
from django.utils import timezone
@@ -119,43 +118,6 @@ def test_api_documents_index_single_hybrid_enabled_success(settings):
assert len(chunk["content"]) < len(document["content"])
@responses.activate
def test_api_documents_index_with_wrong_embedding_dimension(settings, caplog):
"""Test embedding with wrong dimension should log a warning and not index the embedding."""
service = factories.ServiceFactory()
enable_hybrid_search(settings)
settings.EMBEDDING_DIMENSION = 8
responses.add(
responses.POST,
settings.EMBEDDING_API_PATH,
json=albert_embedding_response.response,
status=200,
)
document = factories.DocumentSchemaFactory.build()
with caplog.at_level(logging.WARNING):
APIClient().post(
"/api/v1.0/documents/index/",
document,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert any(
"unexpected embedding dimension: EMBEDDING_DIMENSION is set to 8 "
"but the configured embedding model returned a vector of dimension 1024"
in message
for message in caplog.messages
)
new_indexed_document = opensearch.opensearch_client().get(
index=service.index_name, id=str(document["id"])
)
# check embedding
assert new_indexed_document["_source"]["chunks"] is None
def test_api_documents_index_language_params():
"""language_code query param should control which language is indexed."""
service = factories.ServiceFactory()
@@ -7,6 +7,7 @@ of documents is slow and better done only once.
import logging
import random
from unittest import mock
import pytest
import responses
@@ -193,11 +194,7 @@ def test_api_documents_search_match_all(settings):
response = APIClient().post(
"/api/v1.0/documents/search/",
{
"q": "*",
"visited": [doc["id"] for doc in documents],
"rescore": False,
},
{"q": "*", "visited": [doc["id"] for doc in documents]},
format="json",
HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
)
@@ -561,7 +558,6 @@ def test_api_documents_search_with_nb_results(settings):
"q": "*",
"nb_results": nb_results,
"visited": [doc["id"] for doc in documents],
"rescore": False,
},
format="json",
HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
@@ -669,7 +665,6 @@ def test_api_documents_search_nb_results_with_filtering(settings):
"reach": "public",
"nb_results": nb_results,
"visited": public_ids,
"rescore_enable": False,
},
format="json",
HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
@@ -823,3 +818,36 @@ def test_api_documents_search_filtering_by_path(settings):
assert len(response.json()) == 2
for hit in response.json():
assert hit["_source"]["path"].startswith(path_filter)
@responses.activate
@mock.patch("core.services.reranking.rerank")
def test_api_documents_rerank_blocked(
mock_rerank,
settings,
caplog,
):
"""Test can disable rerank at run time even when enabled in settings"""
settings.RERANKER_ENABLED = True
setup_oicd_resource_server(responses, settings, sub="user_sub")
service = factories.ServiceFactory()
documents = bulk_create_documents([{"title": "Fox"}, {"title": "Wolf"}])
prepare_index(service.index_name, documents)
mock_rerank.return_value = documents
with caplog.at_level(logging.INFO):
response = APIClient().post(
"/api/v1.0/documents/search/",
{
"q": "programming",
"rerank": False,
"visited": [doc["id"] for doc in documents],
},
format="json",
HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
)
assert response.status_code == 200
assert not mock_rerank.called
+99
View File
@@ -0,0 +1,99 @@
"""
Test suite for reranking service
"""
import logging
from unittest.mock import MagicMock, patch
import pytest
from core.services.reranking import rerank
pytestmark = pytest.mark.django_db
logger = logging.getLogger(__name__)
ORIGINAL_HITS = [
{
"_id": "doc-0",
"_score": 1.5,
"_source": {
"title.en": "Dogs",
"content.en": "Dogs are domesticated wolves",
},
},
{
"_id": "doc-1",
"_score": 1.2,
"_source": {
"title.en": "Cats",
"content.en": "Cats are popular pets",
},
},
{
"_id": "doc-2",
"_score": 1.0,
"_source": {
"title.en": "Birds",
"content.en": "Birds fly in the sky",
},
},
]
@patch("core.services.reranking.get_reranker")
def test_return_original_results_if_reranker_import_fails(mocked_get_reranker, caplog):
"""Test that original results are returned if reranker import fails"""
mocked_get_reranker.return_value = None
with caplog.at_level(logging.WARNING):
result = rerank("test query", ORIGINAL_HITS)
assert result == ORIGINAL_HITS
assert any(
"Could not import reranker, returning original results" in message
for message in caplog.messages
)
@patch("core.services.reranking.get_reranker")
def test_return_original_results_if_reranking_fails(mocked_get_reranker, caplog):
"""Test that original results are returned if reranking fails"""
mock_reranker = MagicMock()
mock_reranker.rerank.side_effect = Exception("Reranking service error")
mocked_get_reranker.return_value = mock_reranker
q = "test query"
with caplog.at_level(logging.ERROR):
result = rerank(q, ORIGINAL_HITS)
assert result == ORIGINAL_HITS
assert any(
"Reranking failed: Reranking service error, returning original results"
in message
for message in caplog.messages
)
def test_reranking_success(caplog, settings):
"""Test successful reranking of search results"""
settings.RERANKER_MODEL_NAME = "ms-marco-MiniLM-L-12-v2"
q = "cats"
with caplog.at_level(logging.INFO):
reranked_hits = rerank(q, ORIGINAL_HITS.copy())
assert any(
f"Reranking 3 results for query: {q}" in message for message in caplog.messages
)
assert any(
"Reranking completed, returned 3 results" in message
for message in caplog.messages
)
# if the reranking is not too bad it should definitely rerank doc-1 at top position
assert reranked_hits[0]["_id"] == ORIGINAL_HITS[1]["_id"]
# reranked_hits should be sorted by _reranked_score
assert reranked_hits[0]["_reranked_score"] > reranked_hits[1]["_reranked_score"]
assert reranked_hits[1]["_reranked_score"] > reranked_hits[2]["_reranked_score"]
+50 -35
View File
@@ -2,9 +2,9 @@
Test suite for opensearch search service
"""
import datetime
import logging
from json import dumps as json_dumps
from unittest.mock import patch
import pytest
import responses
@@ -677,41 +677,56 @@ def test_search_filtering_by_query_path_and_tag():
assert returned_ids == expected_ids
def test_search_with_rescore(settings):
"""Test rescore feature"""
service = factories.ServiceFactory(name=SERVICE_NAME)
# pylint: disable=too-many-arguments, too-many-positional-arguments
@responses.activate
@patch("core.services.search.rerank")
@pytest.mark.parametrize(
"rerank_requested, reranker_enabled, rerank_called, caplog_messages",
[
(True, True, True, []),
(
True,
False,
False,
[
"Reranking was explicitly requested but the reranker is "
"disabled in settings. Reranking skipped."
],
),
(False, True, False, []),
(False, False, False, []),
(None, False, False, []),
(None, True, True, []),
],
)
def test_search_rerank( # noqa : PLR0913
mock_rerank,
rerank_requested,
reranker_enabled,
rerank_called,
caplog_messages,
settings,
caplog,
):
"""Test that rerank activation according to args and settings"""
settings.RERANKER_ENABLED = reranker_enabled
today = datetime.datetime.today()
forty_days_ago = today - datetime.timedelta(days=40)
documents = [
factories.DocumentSchemaFactory.build(
users=["user_sub"],
title="my first document",
created_at=today,
updated_at=today,
),
factories.DocumentSchemaFactory.build(
users=["user_sub"],
title="another document",
created_at=forty_days_ago,
updated_at=forty_days_ago,
),
]
documents = bulk_create_documents(
[
{"title": "wolf", "content": "wolves live in packs and hunt together"},
{"title": "dog", "content": "dogs are loyal domestic animals"},
{"title": "cat", "content": "cats are curious and independent pets"},
]
)
mock_rerank.return_value = documents
service = factories.ServiceFactory(name=SERVICE_NAME)
prepare_index(service.index_name, documents)
# set a cray big RESCORE_UPDATED_AT_WEIGHT to demonstrate the effect of boosting on rescores
settings.RESCORE_UPDATED_AT_WEIGHT = 200
with caplog.at_level(logging.INFO):
search(
q="canine pet", rerank_requested=rerank_requested, **search_params(service)
)
results = search(
q="another document",
**{
**search_params(service),
"rescore": True,
},
)
hits = results["hits"]["hits"]
# the first document is ranked first because it more recent
# even though the second one matches the query better
assert hits[0]["_source"]["title.en"] == "my first document"
assert hits[1]["_source"]["title.en"] == "another document"
assert mock_rerank.called == rerank_called
for message in caplog_messages:
assert message in caplog.text
+5 -3
View File
@@ -341,6 +341,9 @@ class SearchDocumentView(ResourceServerMixin, views.APIView):
List of public/authenticated documents the user has visited to limit
the document returned to the ones the current user has seen.
Built from linkreach list of a document in docs app.
rerank : bool, optional
Enable or disable reranking of results. If not specified, falls back to
the RERANKER_ENABLED setting.
search_type : str, optional
Type of search to perform: 'hybrid' or 'full_text'.
- 'hybrid': Uses hybrid search if enabled on the server,
@@ -358,7 +361,6 @@ class SearchDocumentView(ResourceServerMixin, views.APIView):
# Get list of groups related to the user from SCIM provider (consider caching result)
audience = self._get_service_provider_audience()
user_sub = self.request.user.sub
groups = []
params = schemas.SearchQueryParametersSchema(**request.data)
# Get index list for search query
@@ -379,7 +381,7 @@ class SearchDocumentView(ResourceServerMixin, views.APIView):
reach=params.reach,
visited=params.visited,
user_sub=user_sub,
groups=groups,
groups=[],
tags=params.tags,
path=params.path,
search_type=params.search_type
@@ -387,7 +389,7 @@ class SearchDocumentView(ResourceServerMixin, views.APIView):
else SearchTypeEnum.HYBRID
if check_hybrid_search_enabled()
else SearchTypeEnum.FULL_TEXT,
rescore=params.rescore,
rerank_requested=params.rerank,
)["hits"]["hits"]
logger.info("found %d results", len(result))
logger.debug("results %s", result)
+7 -11
View File
@@ -318,18 +318,14 @@ class Base(Configuration):
EMBEDDING_DIMENSION = values.IntegerValue(
default=1024, environ_name="EMBEDDING_DIMENSION", environ_prefix=None
)
# rescore
RESCORE_UPDATED_AT_WEIGHT = values.FloatValue(
default=0.2, environ_name="RESCORE_UPDATED_AT_WEIGHT", environ_prefix=None
# Reranking
RERANKER_ENABLED = values.BooleanValue(
default=False, environ_name="RERANKER_ENABLED", environ_prefix=None
)
RESCORE_UPDATED_AT_OFFSET = values.Value(
default="2d", environ_name="RESCORE_UPDATED_AT_OFFSET", environ_prefix=None
)
RESCORE_UPDATED_AT_SCALE = values.Value(
default="6d", environ_name="RESCORE_UPDATED_AT_SCALE", environ_prefix=None
)
RESCORE_UPDATED_AT_DECAY = values.IntegerValue(
default=0.5, environ_name="RESCORE_UPDATED_AT_SCALE", environ_prefix=None
RERANKER_MODEL_NAME = values.Value(
environ_name="RERANKER_MODEL_NAME",
environ_prefix=None,
)
# CORS
+1
View File
@@ -36,6 +36,7 @@ dependencies = [
"drf_spectacular==0.29.0",
"dockerflow==2024.4.2",
"factory_boy==3.3.3",
"Flashrank==0.2.10",
"gunicorn==23.0.0",
"py3langid==0.3.0",
"langchain-text-splitters==1.1.0",
+225
View File
@@ -14,6 +14,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/26/99/fc813cd978842c26c82534010ea849eee9ab3a13ea2b74e95cb9c99e747b/amqp-5.3.1-py3-none-any.whl", hash = "sha256:43b3319e1b4e7d1251833a93d672b4af1e40f3d632d479b98661a95f117880a2", size = 50944, upload-time = "2024-11-12T19:55:41.782Z" },
]
[[package]]
name = "annotated-doc"
version = "0.0.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/57/ba/046ceea27344560984e26a590f90bc7f4a75b06701f653222458922b558c/annotated_doc-0.0.4.tar.gz", hash = "sha256:fbcda96e87e9c92ad167c2e53839e57503ecfda18804ea28102353485033faa4", size = 7288, upload-time = "2025-11-10T22:07:42.062Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1e/d3/26bf1008eb3d2daa8ef4cacc7f3bfdc11818d111f7e2d0201bc6e3b49d45/annotated_doc-0.0.4-py3-none-any.whl", hash = "sha256:571ac1dc6991c450b25a9c2d84a3705e2ae7a53467b5d111c24fa8baabbed320", size = 5303, upload-time = "2025-11-10T22:07:40.673Z" },
]
[[package]]
name = "annotated-types"
version = "0.7.0"
@@ -489,6 +498,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/fc/23/e22da510e1ec1488966330bf76d8ff4bd535cbfc93660eeb7657761a1bb2/faker-40.1.0-py3-none-any.whl", hash = "sha256:a616d35818e2a2387c297de80e2288083bc915e24b7e39d2fb5bc66cce3a929f", size = 1985317, upload-time = "2025-12-29T18:05:58.831Z" },
]
[[package]]
name = "filelock"
version = "3.25.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/94/b8/00651a0f559862f3bb7d6f7477b192afe3f583cc5e26403b44e59a55ab34/filelock-3.25.2.tar.gz", hash = "sha256:b64ece2b38f4ca29dd3e810287aa8c48182bbecd1ae6e9ae126c9b35f1382694", size = 40480, upload-time = "2026-03-11T20:45:38.487Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a4/a5/842ae8f0c08b61d6484b52f99a03510a3a72d23141942d216ebe81fefbce/filelock-3.25.2-py3-none-any.whl", hash = "sha256:ca8afb0da15f229774c9ad1b455ed96e85a81373065fb10446672f64444ddf70", size = 26759, upload-time = "2026-03-11T20:45:37.437Z" },
]
[[package]]
name = "find"
version = "0.0.1"
@@ -504,6 +522,7 @@ dependencies = [
{ name = "dockerflow" },
{ name = "drf-spectacular" },
{ name = "factory-boy" },
{ name = "flashrank" },
{ name = "gunicorn" },
{ name = "langchain-text-splitters" },
{ name = "mozilla-django-oidc" },
@@ -554,6 +573,7 @@ requires-dist = [
{ name = "drf-spectacular-sidecar", marker = "extra == 'dev'", specifier = "==2026.1.1" },
{ name = "factory-boy", specifier = "==3.3.3" },
{ name = "faker", marker = "extra == 'dev'", specifier = "==40.1.0" },
{ name = "flashrank", specifier = "==0.2.10" },
{ name = "gunicorn", specifier = "==23.0.0" },
{ name = "ipdb", marker = "extra == 'dev'", specifier = "==0.13.13" },
{ name = "ipython", marker = "extra == 'dev'", specifier = "==9.8.0" },
@@ -583,6 +603,39 @@ requires-dist = [
]
provides-extras = ["dev"]
[[package]]
name = "flashrank"
version = "0.2.10"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "numpy" },
{ name = "onnxruntime" },
{ name = "requests" },
{ name = "tokenizers" },
{ name = "tqdm" },
]
sdist = { url = "https://files.pythonhosted.org/packages/55/1f/176cb4a857a70c3538f637e19389ab6aed21548a1ba1d1424fccc8bba108/FlashRank-0.2.10.tar.gz", hash = "sha256:f8f82a25c32fdfc668a09dc4089421d6aab8e7f71308424b541f40bb3f01d9db", size = 18905, upload-time = "2025-01-06T13:33:01.657Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ec/99/72639cc1c9221c5bc77a2df1c2d352fe11965553bdf7d3e0856e7fcc8fd6/FlashRank-0.2.10-py3-none-any.whl", hash = "sha256:5d3272ae657d793c132d1e7917ed9e2adf49e0e1c60735583a67b051c6f0434a", size = 14511, upload-time = "2025-01-06T13:32:59.42Z" },
]
[[package]]
name = "flatbuffers"
version = "25.12.19"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e8/2d/d2a548598be01649e2d46231d151a6c56d10b964d94043a335ae56ea2d92/flatbuffers-25.12.19-py2.py3-none-any.whl", hash = "sha256:7634f50c427838bb021c2d66a3d1168e9d199b0607e6329399f04846d42e20b4", size = 26661, upload-time = "2025-12-19T23:16:13.622Z" },
]
[[package]]
name = "fsspec"
version = "2026.2.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/51/7c/f60c259dcbf4f0c47cc4ddb8f7720d2dcdc8888c8e5ad84c73ea4531cc5b/fsspec-2026.2.0.tar.gz", hash = "sha256:6544e34b16869f5aacd5b90bdf1a71acb37792ea3ddf6125ee69a22a53fb8bff", size = 313441, upload-time = "2026-02-05T21:50:53.743Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e6/ab/fb21f4c939bb440104cc2b396d3be1d9b7a9fd3c6c2a53d98c45b3d7c954/fsspec-2026.2.0-py3-none-any.whl", hash = "sha256:98de475b5cb3bd66bedd5c4679e87b4fdfe1a3bf4d707b151b3c07e58c9a2437", size = 202505, upload-time = "2026-02-05T21:50:51.819Z" },
]
[[package]]
name = "grpcio"
version = "1.76.0"
@@ -625,6 +678,22 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },
]
[[package]]
name = "hf-xet"
version = "1.4.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/09/08/23c84a26716382c89151b5b447b4beb19e3345f3a93d3b73009a71a57ad3/hf_xet-1.4.2.tar.gz", hash = "sha256:b7457b6b482d9e0743bd116363239b1fa904a5e65deede350fbc0c4ea67c71ea", size = 672357, upload-time = "2026-03-13T06:58:51.077Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b4/86/b40b83a2ff03ef05c4478d2672b1fc2b9683ff870e2b25f4f3af240f2e7b/hf_xet-1.4.2-cp37-abi3-macosx_10_12_x86_64.whl", hash = "sha256:71f02d6e4cdd07f344f6844845d78518cc7186bd2bc52d37c3b73dc26a3b0bc5", size = 3800339, upload-time = "2026-03-13T06:58:36.245Z" },
{ url = "https://files.pythonhosted.org/packages/64/2e/af4475c32b4378b0e92a587adb1aa3ec53e3450fd3e5fe0372a874531c00/hf_xet-1.4.2-cp37-abi3-macosx_11_0_arm64.whl", hash = "sha256:e9b38d876e94d4bdcf650778d6ebbaa791dd28de08db9736c43faff06ede1b5a", size = 3559664, upload-time = "2026-03-13T06:58:34.787Z" },
{ url = "https://files.pythonhosted.org/packages/3c/4c/781267da3188db679e601de18112021a5cb16506fe86b246e22c5401a9c4/hf_xet-1.4.2-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:77e8c180b7ef12d8a96739a4e1e558847002afe9ea63b6f6358b2271a8bdda1c", size = 4217422, upload-time = "2026-03-13T06:58:27.472Z" },
{ url = "https://files.pythonhosted.org/packages/68/47/d6cf4a39ecf6c7705f887a46f6ef5c8455b44ad9eb0d391aa7e8a2ff7fea/hf_xet-1.4.2-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:c3b3c6a882016b94b6c210957502ff7877802d0dbda8ad142c8595db8b944271", size = 3992847, upload-time = "2026-03-13T06:58:25.989Z" },
{ url = "https://files.pythonhosted.org/packages/2d/ef/e80815061abff54697239803948abc665c6b1d237102c174f4f7a9a5ffc5/hf_xet-1.4.2-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:9d9a634cc929cfbaf2e1a50c0e532ae8c78fa98618426769480c58501e8c8ac2", size = 4193843, upload-time = "2026-03-13T06:58:44.59Z" },
{ url = "https://files.pythonhosted.org/packages/54/75/07f6aa680575d9646c4167db6407c41340cbe2357f5654c4e72a1b01ca14/hf_xet-1.4.2-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:6b0932eb8b10317ea78b7da6bab172b17be03bbcd7809383d8d5abd6a2233e04", size = 4432751, upload-time = "2026-03-13T06:58:46.533Z" },
{ url = "https://files.pythonhosted.org/packages/cd/71/193eabd7e7d4b903c4aa983a215509c6114915a5a237525ec562baddb868/hf_xet-1.4.2-cp37-abi3-win_amd64.whl", hash = "sha256:ad185719fb2e8ac26f88c8100562dbf9dbdcc3d9d2add00faa94b5f106aea53f", size = 3671149, upload-time = "2026-03-13T06:58:57.07Z" },
{ url = "https://files.pythonhosted.org/packages/b4/7e/ccf239da366b37ba7f0b36095450efae4a64980bdc7ec2f51354205fdf39/hf_xet-1.4.2-cp37-abi3-win_arm64.whl", hash = "sha256:32c012286b581f783653e718c1862aea5b9eb140631685bb0c5e7012c8719a87", size = 3533426, upload-time = "2026-03-13T06:58:55.46Z" },
]
[[package]]
name = "httpcore"
version = "1.0.9"
@@ -653,6 +722,26 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" },
]
[[package]]
name = "huggingface-hub"
version = "1.7.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "filelock" },
{ name = "fsspec" },
{ name = "hf-xet", marker = "platform_machine == 'AMD64' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64'" },
{ name = "httpx" },
{ name = "packaging" },
{ name = "pyyaml" },
{ name = "tqdm" },
{ name = "typer" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b4/a8/94ccc0aec97b996a3a68f3e1fa06a4bd7185dd02bf22bfba794a0ade8440/huggingface_hub-1.7.1.tar.gz", hash = "sha256:be38fe66e9b03c027ad755cb9e4b87ff0303c98acf515b5d579690beb0bf3048", size = 722097, upload-time = "2026-03-13T09:36:07.758Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/6f/75/ca21955d6117a394a482c7862ce96216239d0e3a53133ae8510727a8bcfa/huggingface_hub-1.7.1-py3-none-any.whl", hash = "sha256:38c6cce7419bbde8caac26a45ed22b0cea24152a8961565d70ec21f88752bfaa", size = 616308, upload-time = "2026-03-13T09:36:06.062Z" },
]
[[package]]
name = "icdiff"
version = "2.0.7"
@@ -886,6 +975,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/ff/11/22a56b615f1ca84f65fda68b1a53b6e2765f2bdb1c6d885793430a664bfe/langsmith-0.6.3-py3-none-any.whl", hash = "sha256:44fdf8084165513e6bede9dda715e7b460b1b3f57ac69f2ca3f03afa911233ec", size = 282991, upload-time = "2026-01-14T19:26:21.882Z" },
]
[[package]]
name = "markdown-it-py"
version = "4.0.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "mdurl" },
]
sdist = { url = "https://files.pythonhosted.org/packages/5b/f5/4ec618ed16cc4f8fb3b701563655a69816155e79e24a17b651541804721d/markdown_it_py-4.0.0.tar.gz", hash = "sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3", size = 73070, upload-time = "2025-08-11T12:57:52.854Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/94/54/e7d793b573f298e1c9013b8c4dade17d481164aa517d1d7148619c2cedbf/markdown_it_py-4.0.0-py3-none-any.whl", hash = "sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147", size = 87321, upload-time = "2025-08-11T12:57:51.923Z" },
]
[[package]]
name = "matplotlib-inline"
version = "0.2.1"
@@ -907,6 +1008,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/27/1a/1f68f9ba0c207934b35b86a8ca3aad8395a3d6dd7921c0686e23853ff5a9/mccabe-0.7.0-py2.py3-none-any.whl", hash = "sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e", size = 7350, upload-time = "2022-01-24T01:14:49.62Z" },
]
[[package]]
name = "mdurl"
version = "0.1.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" },
]
[[package]]
name = "mozilla-django-oidc"
version = "5.0.2"
@@ -922,6 +1032,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/a8/a9/c1664acf30ef0031ed06650039de1693bddf4cf7f58e07939e1aa80bffb7/mozilla_django_oidc-5.0.2-py3-none-any.whl", hash = "sha256:965a3533b0e299288cdf38ec2f8b550217c302ffe78ce5bd0b2d2f4bc436878b", size = 25910, upload-time = "2025-12-19T16:11:39.729Z" },
]
[[package]]
name = "mpmath"
version = "1.3.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/e0/47/dd32fa426cc72114383ac549964eecb20ecfd886d1e5ccf5340b55b02f57/mpmath-1.3.0.tar.gz", hash = "sha256:7a28eb2a9774d00c7bc92411c19a89209d5da7c4c9a9e227be8330a23a25b91f", size = 508106, upload-time = "2023-03-07T16:47:11.061Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl", hash = "sha256:a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c", size = 536198, upload-time = "2023-03-07T16:47:09.197Z" },
]
[[package]]
name = "numpy"
version = "2.4.1"
@@ -941,6 +1060,25 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/69/80/a828b2d0ade5e74a9fe0f4e0a17c30fdc26232ad2bc8c9f8b3197cf7cf18/numpy-2.4.1-cp312-cp312-win_arm64.whl", hash = "sha256:0e6e8f9d9ecf95399982019c01223dc130542960a12edfa8edd1122dfa66a8a8", size = 10312166, upload-time = "2026-01-10T06:43:03.673Z" },
]
[[package]]
name = "onnxruntime"
version = "1.24.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "flatbuffers" },
{ name = "numpy" },
{ name = "packaging" },
{ name = "protobuf" },
{ name = "sympy" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/d0/7f/dfdc4e52600fde4c02d59bfe98c4b057931c1114b701e175aee311a9bc11/onnxruntime-1.24.3-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:0d244227dc5e00a9ae15a7ac1eba4c4460d7876dfecafe73fb00db9f1d914d91", size = 17342578, upload-time = "2026-03-05T17:19:02.403Z" },
{ url = "https://files.pythonhosted.org/packages/1c/dc/1f5489f7b21817d4ad352bf7a92a252bd5b438bcbaa7ad20ea50814edc79/onnxruntime-1.24.3-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0a9847b870b6cb462652b547bc98c49e0efb67553410a082fde1918a38707452", size = 15150105, upload-time = "2026-03-05T16:34:56.897Z" },
{ url = "https://files.pythonhosted.org/packages/28/7c/fd253da53594ab8efbefdc85b3638620ab1a6aab6eb7028a513c853559ce/onnxruntime-1.24.3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b354afce3333f2859c7e8706d84b6c552beac39233bcd3141ce7ab77b4cabb5d", size = 17237101, upload-time = "2026-03-05T17:18:02.561Z" },
{ url = "https://files.pythonhosted.org/packages/71/5f/eaabc5699eeed6a9188c5c055ac1948ae50138697a0428d562ac970d7db5/onnxruntime-1.24.3-cp312-cp312-win_amd64.whl", hash = "sha256:44ea708c34965439170d811267c51281d3897ecfc4aa0087fa25d4a4c3eb2e4a", size = 12597638, upload-time = "2026-03-05T17:18:52.141Z" },
{ url = "https://files.pythonhosted.org/packages/cc/5c/d8066c320b90610dbeb489a483b132c3b3879b2f93f949fb5d30cfa9b119/onnxruntime-1.24.3-cp312-cp312-win_arm64.whl", hash = "sha256:48d1092b44ca2ba6f9543892e7c422c15a568481403c10440945685faf27a8d8", size = 12270943, upload-time = "2026-03-05T17:18:42.006Z" },
]
[[package]]
name = "opensearch-protobufs"
version = "0.19.0"
@@ -1431,6 +1569,19 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/1c/4c/cc276ce57e572c102d9542d383b2cfd551276581dc60004cb94fe8774c11/responses-0.25.8-py3-none-any.whl", hash = "sha256:0c710af92def29c8352ceadff0c3fe340ace27cf5af1bbe46fb71275bcd2831c", size = 34769, upload-time = "2025-08-08T19:01:45.018Z" },
]
[[package]]
name = "rich"
version = "14.3.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "markdown-it-py" },
{ name = "pygments" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b3/c6/f3b320c27991c46f43ee9d856302c70dc2d0fb2dba4842ff739d5f46b393/rich-14.3.3.tar.gz", hash = "sha256:b8daa0b9e4eef54dd8cf7c86c03713f53241884e814f4e2f5fb342fe520f639b", size = 230582, upload-time = "2026-02-19T17:23:12.474Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/14/25/b208c5683343959b670dc001595f2f3737e051da617f66c31f7c4fa93abc/rich-14.3.3-py3-none-any.whl", hash = "sha256:793431c1f8619afa7d3b52b2cdec859562b950ea0d4b6b505397612db8d5362d", size = 310458, upload-time = "2026-02-19T17:23:13.732Z" },
]
[[package]]
name = "rpds-py"
version = "0.30.0"
@@ -1493,6 +1644,15 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/4d/19/8d77f9992e5cbfcaa9133c3bf63b4fbbb051248802e1e803fed5c552fbb2/sentry_sdk-2.48.0-py2.py3-none-any.whl", hash = "sha256:6b12ac256769d41825d9b7518444e57fa35b5642df4c7c5e322af4d2c8721172", size = 414555, upload-time = "2025-12-16T14:55:40.152Z" },
]
[[package]]
name = "shellingham"
version = "1.5.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/58/15/8b3609fd3830ef7b27b655beb4b4e9c62313a4e8da8c676e142cc210d58e/shellingham-1.5.4.tar.gz", hash = "sha256:8dbca0739d487e5bd35ab3ca4b36e11c4078f3a234bfce294b0a0291363404de", size = 10310, upload-time = "2023-10-24T04:13:40.426Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl", hash = "sha256:7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686", size = 9755, upload-time = "2023-10-24T04:13:38.866Z" },
]
[[package]]
name = "six"
version = "1.17.0"
@@ -1525,6 +1685,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/f1/7b/ce1eafaf1a76852e2ec9b22edecf1daa58175c090266e9f6c64afcd81d91/stack_data-0.6.3-py3-none-any.whl", hash = "sha256:d5558e0c25a4cb0853cddad3d77da9891a08cb85dd9f9f91b9f8cd66e511e695", size = 24521, upload-time = "2023-09-30T13:58:03.53Z" },
]
[[package]]
name = "sympy"
version = "1.14.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "mpmath" },
]
sdist = { url = "https://files.pythonhosted.org/packages/83/d3/803453b36afefb7c2bb238361cd4ae6125a569b4db67cd9e79846ba2d68c/sympy-1.14.0.tar.gz", hash = "sha256:d3d3fe8df1e5a0b42f0e7bdf50541697dbe7d23746e894990c030e2b05e72517", size = 7793921, upload-time = "2025-04-27T18:05:01.611Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl", hash = "sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5", size = 6299353, upload-time = "2025-04-27T18:04:59.103Z" },
]
[[package]]
name = "tenacity"
version = "9.1.2"
@@ -1534,6 +1706,32 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/e5/30/643397144bfbfec6f6ef821f36f33e57d35946c44a2352d3c9f0ae847619/tenacity-9.1.2-py3-none-any.whl", hash = "sha256:f77bf36710d8b73a50b2dd155c97b870017ad21afe6ab300326b0371b3b05138", size = 28248, upload-time = "2025-04-02T08:25:07.678Z" },
]
[[package]]
name = "tokenizers"
version = "0.22.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "huggingface-hub" },
]
sdist = { url = "https://files.pythonhosted.org/packages/73/6f/f80cfef4a312e1fb34baf7d85c72d4411afde10978d4657f8cdd811d3ccc/tokenizers-0.22.2.tar.gz", hash = "sha256:473b83b915e547aa366d1eee11806deaf419e17be16310ac0a14077f1e28f917", size = 372115, upload-time = "2026-01-05T10:45:15.988Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/92/97/5dbfabf04c7e348e655e907ed27913e03db0923abb5dfdd120d7b25630e1/tokenizers-0.22.2-cp39-abi3-macosx_10_12_x86_64.whl", hash = "sha256:544dd704ae7238755d790de45ba8da072e9af3eea688f698b137915ae959281c", size = 3100275, upload-time = "2026-01-05T10:41:02.158Z" },
{ url = "https://files.pythonhosted.org/packages/2e/47/174dca0502ef88b28f1c9e06b73ce33500eedfac7a7692108aec220464e7/tokenizers-0.22.2-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:1e418a55456beedca4621dbab65a318981467a2b188e982a23e117f115ce5001", size = 2981472, upload-time = "2026-01-05T10:41:00.276Z" },
{ url = "https://files.pythonhosted.org/packages/d6/84/7990e799f1309a8b87af6b948f31edaa12a3ed22d11b352eaf4f4b2e5753/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2249487018adec45d6e3554c71d46eb39fa8ea67156c640f7513eb26f318cec7", size = 3290736, upload-time = "2026-01-05T10:40:32.165Z" },
{ url = "https://files.pythonhosted.org/packages/78/59/09d0d9ba94dcd5f4f1368d4858d24546b4bdc0231c2354aa31d6199f0399/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:25b85325d0815e86e0bac263506dd114578953b7b53d7de09a6485e4a160a7dd", size = 3168835, upload-time = "2026-01-05T10:40:38.847Z" },
{ url = "https://files.pythonhosted.org/packages/47/50/b3ebb4243e7160bda8d34b731e54dd8ab8b133e50775872e7a434e524c28/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:bfb88f22a209ff7b40a576d5324bf8286b519d7358663db21d6246fb17eea2d5", size = 3521673, upload-time = "2026-01-05T10:40:56.614Z" },
{ url = "https://files.pythonhosted.org/packages/e0/fa/89f4cb9e08df770b57adb96f8cbb7e22695a4cb6c2bd5f0c4f0ebcf33b66/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1c774b1276f71e1ef716e5486f21e76333464f47bece56bbd554485982a9e03e", size = 3724818, upload-time = "2026-01-05T10:40:44.507Z" },
{ url = "https://files.pythonhosted.org/packages/64/04/ca2363f0bfbe3b3d36e95bf67e56a4c88c8e3362b658e616d1ac185d47f2/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:df6c4265b289083bf710dff49bc51ef252f9d5be33a45ee2bed151114a56207b", size = 3379195, upload-time = "2026-01-05T10:40:51.139Z" },
{ url = "https://files.pythonhosted.org/packages/2e/76/932be4b50ef6ccedf9d3c6639b056a967a86258c6d9200643f01269211ca/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:369cc9fc8cc10cb24143873a0d95438bb8ee257bb80c71989e3ee290e8d72c67", size = 3274982, upload-time = "2026-01-05T10:40:58.331Z" },
{ url = "https://files.pythonhosted.org/packages/1d/28/5f9f5a4cc211b69e89420980e483831bcc29dade307955cc9dc858a40f01/tokenizers-0.22.2-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:29c30b83d8dcd061078b05ae0cb94d3c710555fbb44861139f9f83dcca3dc3e4", size = 9478245, upload-time = "2026-01-05T10:41:04.053Z" },
{ url = "https://files.pythonhosted.org/packages/6c/fb/66e2da4704d6aadebf8cb39f1d6d1957df667ab24cff2326b77cda0dcb85/tokenizers-0.22.2-cp39-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:37ae80a28c1d3265bb1f22464c856bd23c02a05bb211e56d0c5301a435be6c1a", size = 9560069, upload-time = "2026-01-05T10:45:10.673Z" },
{ url = "https://files.pythonhosted.org/packages/16/04/fed398b05caa87ce9b1a1bb5166645e38196081b225059a6edaff6440fac/tokenizers-0.22.2-cp39-abi3-musllinux_1_2_i686.whl", hash = "sha256:791135ee325f2336f498590eb2f11dc5c295232f288e75c99a36c5dbce63088a", size = 9899263, upload-time = "2026-01-05T10:45:12.559Z" },
{ url = "https://files.pythonhosted.org/packages/05/a1/d62dfe7376beaaf1394917e0f8e93ee5f67fea8fcf4107501db35996586b/tokenizers-0.22.2-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:38337540fbbddff8e999d59970f3c6f35a82de10053206a7562f1ea02d046fa5", size = 10033429, upload-time = "2026-01-05T10:45:14.333Z" },
{ url = "https://files.pythonhosted.org/packages/fd/18/a545c4ea42af3df6effd7d13d250ba77a0a86fb20393143bbb9a92e434d4/tokenizers-0.22.2-cp39-abi3-win32.whl", hash = "sha256:a6bf3f88c554a2b653af81f3204491c818ae2ac6fbc09e76ef4773351292bc92", size = 2502363, upload-time = "2026-01-05T10:45:20.593Z" },
{ url = "https://files.pythonhosted.org/packages/65/71/0670843133a43d43070abeb1949abfdef12a86d490bea9cd9e18e37c5ff7/tokenizers-0.22.2-cp39-abi3-win_amd64.whl", hash = "sha256:c9ea31edff2968b44a88f97d784c2f16dc0729b8b143ed004699ebca91f05c48", size = 2747786, upload-time = "2026-01-05T10:45:18.411Z" },
{ url = "https://files.pythonhosted.org/packages/72/f4/0de46cfa12cdcbcd464cc59fde36912af405696f687e53a091fb432f694c/tokenizers-0.22.2-cp39-abi3-win_arm64.whl", hash = "sha256:9ce725d22864a1e965217204946f830c37876eee3b2ba6fc6255e8e903d5fcbc", size = 2612133, upload-time = "2026-01-05T10:45:17.232Z" },
]
[[package]]
name = "tomlkit"
version = "0.14.0"
@@ -1543,6 +1741,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/b5/11/87d6d29fb5d237229d67973a6c9e06e048f01cf4994dee194ab0ea841814/tomlkit-0.14.0-py3-none-any.whl", hash = "sha256:592064ed85b40fa213469f81ac584f67a4f2992509a7c3ea2d632208623a3680", size = 39310, upload-time = "2026-01-13T01:14:51.965Z" },
]
[[package]]
name = "tqdm"
version = "4.67.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/09/a9/6ba95a270c6f1fbcd8dac228323f2777d886cb206987444e4bce66338dd4/tqdm-4.67.3.tar.gz", hash = "sha256:7d825f03f89244ef73f1d4ce193cb1774a8179fd96f31d7e1dcde62092b960bb", size = 169598, upload-time = "2026-02-03T17:35:53.048Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/16/e1/3079a9ff9b8e11b846c6ac5c8b5bfb7ff225eee721825310c91b3b50304f/tqdm-4.67.3-py3-none-any.whl", hash = "sha256:ee1e4c0e59148062281c49d80b25b67771a127c85fc9676d3be5f243206826bf", size = 78374, upload-time = "2026-02-03T17:35:50.982Z" },
]
[[package]]
name = "traitlets"
version = "5.14.3"
@@ -1552,6 +1762,21 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/00/c0/8f5d070730d7836adc9c9b6408dec68c6ced86b304a9b26a14df072a6e8c/traitlets-5.14.3-py3-none-any.whl", hash = "sha256:b74e89e397b1ed28cc831db7aea759ba6640cb3de13090ca145426688ff1ac4f", size = 85359, upload-time = "2024-04-19T11:11:46.763Z" },
]
[[package]]
name = "typer"
version = "0.24.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "annotated-doc" },
{ name = "click" },
{ name = "rich" },
{ name = "shellingham" },
]
sdist = { url = "https://files.pythonhosted.org/packages/f5/24/cb09efec5cc954f7f9b930bf8279447d24618bb6758d4f6adf2574c41780/typer-0.24.1.tar.gz", hash = "sha256:e39b4732d65fbdcde189ae76cf7cd48aeae72919dea1fdfc16593be016256b45", size = 118613, upload-time = "2026-02-21T16:54:40.609Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/4a/91/48db081e7a63bb37284f9fbcefda7c44c277b18b0e13fbc36ea2335b71e6/typer-0.24.1-py3-none-any.whl", hash = "sha256:112c1f0ce578bfb4cab9ffdabc68f031416ebcc216536611ba21f04e9aa84c9e", size = 56085, upload-time = "2026-02-21T16:54:41.616Z" },
]
[[package]]
name = "types-requests"
version = "2.32.4.20250913"