Compare commits

..

6 Commits

Author SHA1 Message Date
charles efe12bcefc (backend) test
i am testing the rescore feature
2026-03-16 09:51:58 +01:00
charles bed2bd5203 (backend) add enable_rescore params
I am making rescore optional.
2026-03-16 09:51:58 +01:00
charles bd5b5f740f ⚰️(backend) remove sorting
sorting is not supported with hybrid search.
I should have removed it long ago.
It is not supported also with rerank.
2026-03-16 09:51:58 +01:00
charles 997e6f3545 (backend) rescore documents on updated_at
We hypothesised that users will be interested in most recently
updated documents.
To improve search results given this hypothesis we want to boost
the score of documents based on the `updated_at` field.
This commit applies a gaussian boost on the updated_at field
2026-03-16 09:51:58 +01:00
charles 70015e8a7d 🚨(backend) remove dimensions params in embed_text
the params dimension is either ignored or crashes
the api call depending on the provider
2026-03-16 09:51:58 +01:00
charles dd197e1f1e ♻️(backend) reorganize evaluation data
I am improving the data management of evaluation system
2026-03-10 15:05:14 +01:00
15 changed files with 171 additions and 610 deletions
+1 -2
View File
@@ -9,7 +9,7 @@ and this project adheres to
# Unreleased
## Added
- ✨(backend) add rescore on `updated_at`
- 👷(docker) add arm64 platform support for image builds
- ✨(backend) add semantic search
- ✨(backend) add multi-embedding and chunking
@@ -31,7 +31,6 @@ and this project adheres to
- ✨(backend) add deletion endpoint
- ✨(backend) add path filter
- ✨(backend) add search_type param
- ✨(backend) implement reranker #47
## Changed
-2
View File
@@ -102,8 +102,6 @@ These are the environment variables you can set for the `find-backend` container
| OPENSEARCH_USE_SSL | Enable SSL connection for Opensearch database | true |
| POSTHOG_KEY | Posthog key for analytics | |
| REDIS_URL | Cache url | redis://redis:6379/1 |
| RERANKER_ENABLED | Flag to enable the re-ranking of search results | false |
| RERANKER_MODEL_NAME | Name of the re-ranking model used | |
| SENTRY_DSN | Sentry host | |
| SESSION_COOKIE_AGE | duration of the cookie session | 60*60*12 |
| SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK | | false |
+31 -49
View File
@@ -1,16 +1,16 @@
# Setup Find
# Using the Find indexer
This guide explains how to setup the Find service which provide an API for indexation and search of
This guide explains how to setup the Find service which provide an API for indexation and fulltext search of
documents from various sources in a secure way : only the documents within the scope of the user's OIDC token
are visible.
## Core setups
## Setup Opensearch
### Opensearch
### General
Add the following variables to your Django settings to configure Find and enable full-text search.
```bash
```python
# Login for opensearch
OPENSEARCH_USER=opensearch-user
OPENSEARCH_PASSWORD=your-opensearch-password
@@ -29,7 +29,7 @@ OPENSEARCH_INDEX_PREFIX=find
### Language
Language specific operations are applied to document titles and contents to improve search results.
The language is automatically detected by Find.
The language is automatically detected by Find.
If the language can not be detected no language specific operation are applied and the indexing process is not affected.
Find supports french, english, german and dutch.
@@ -39,7 +39,7 @@ The search process is not language specific, a query can get documents of any la
Language detection estimates a confidence between 0 and 1. If the confidence is below a threshold the language is considered unrecognized.
This threshold can be controlled with LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD environment variable.
```bash
```python
LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD=0.75
```
@@ -47,67 +47,49 @@ LANGUAGE_DETECTION_CONFIDENCE_THRESHOLD=0.75
Find offers a semantic search feature. You can either use pure full-text search or a hybrid full-text + semantic search. To enable the hybrid search, add the following settings.
```bash
```python
# Enable flag
HYBRID_SEARCH_ENABLED=True
HYBRID_SEARCH_ENABLED = True
# weighted sum: full_text_weight, semantic_search_weight
HYBRID_SEARCH_WEIGHTS=0.7,0.3
HYBRID_SEARCH_WEIGHTS = 0.7,0.3
# Embedding
EMBEDDING_API_PATH=https://embedding.api.example.com/full/path/
EMBEDDING_API_KEY=your-embedding-api-key
EMBEDDING_REQUEST_TIMEOUT=10
EMBEDDING_API_MODEL_NAME=embedding-api-model-name
EMBEDDING_DIMENSION=1024
CHUNK_SIZE=512
CHUNK_OVERLAP=50
EMBEDDING_API_PATH = https://embedding.api.example.com/full/path/
EMBEDDING_API_KEY = your-embedding-api-key
EMBEDDING_REQUEST_TIMEOUT = 10
EMBEDDING_API_MODEL_NAME = embedding-api-model-name
EMBEDDING_DIMENSION = 1024
```
The hybrid search computes a score for full-text and semantic search and combines them through a weighted sum. HYBRID_SEARCH_WEIGHTS contains the weights of full-text and semantic respectively.
You need to use an embedding api similar to https://albert.api.etalab.gouv.fr/documentation#tag/Embeddings/operation/embeddings_v1_embeddings_post.
### Document chunking
### document chunking
The indexing process embeds documents by converting their content into vector representations (embeddings). When a document exceeds the character dimension defined by CHUNK_SIZE, it's divided into smaller segments (chunks), with each chunk embedded independently. Each chunk must be smaller than the embedding model's context window .
```bash
CHUNK_SIZE=512
CHUNK_OVERLAP=50
```
The indexing process embeds documents by converting their content into vector representations (embeddings). When a document exceeds the character dimension defined by CHUNK_SIZE, it's divided into smaller segments (chunks), with each chunk embedded independently. Each chunk must be smaller than the embedding model's context window .
The chunking algorithm works recursively. It attempts to create the largest possible segments first, then subdivides them further if they still exceed the size limit defined by CHUNK_SIZE. Segments can share overlapping content between them (set CHUNK_OVERLAP=0 to disable overlapping).
During the search, the matching of a document is the matching of its best chunk.
### Trigrams
## trigrams
Find uses trigrams to improve the robustness of the full text search engine to spelling variations and errors. It can be configured by two environment variables.
```bash
````
TRIGRAMS_BOOST=0.25
TRIGRAMS_MINIMUM_SHOULD_MATCH=0.75%
```
````
`TRIGRAMS_BOOST` is weight boost applied to the trigram score in the document matching.
`TRIGRAMS_MINIMUM_SHOULD_MATCH` is the minimal number or proportion of trigrams having to match to score. It is
either an absolute number or proportion.
### Reranking
Find offers a reranking feature to reorder the search results based on a cross-encoder method.
To enable the reranking, add the following settings.
```bash
RERANKING_ENABLED=True
RERANKER_MODEL_NAME=ms-marco-MiniLM-L-12-v2
```
Reranking is based on Flashrank. See the [Flashrank documentation](https://github.com/PrithivirajDamodaran/FlashRank?tab=readme-ov-file) to know what models are compatible.
## API setups
### Indexing
## Setup indexation API
Other applications can index their files through the **`/index/`** endpoint with a simple token authentication.
@@ -128,21 +110,21 @@ And add the key in the calling application Django settings.
The command `make demo` will create a working service configuration for `docs` and `drive` with predefined secret keys
```bash
```python
# Docs
SEARCH_INDEXER_SECRET=find-api-key-for-docs-with-exactly-50-chars-length
SEARCH_INDEXER_SECRET="find-api-key-for-docs-with-exactly-50-chars-length"
```
```bash
```python
# Drive
SEARCH_INDEXER_SECRET=find-api-key-for-driv-with-exactly-50-chars-length
SEARCH_INDEXER_SECRET="find-api-key-for-driv-with-exactly-50-chars-length"
```
### Search
## Setup search API
The **`/search/`** endpoint is an OIDC ResourceServer view and needs extra Django settings (see [lasuite](https://github.com/suitenumerique/django-lasuite/blob/main/documentation/how-to-use-oidc-resource-server-backend.md) for details)
```bash
```shell
OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/token
@@ -163,11 +145,11 @@ OIDC_OP_INTROSPECTION_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-
# OIDC_VERIFY_SSL=False
# Resource server
OIDC_RS_SCOPES=openid
OIDC_RS_SCOPES="openid"
OIDC_RS_SIGN_ALGO=RS256
# This backend allows authentication without any model in database.
OIDC_RS_BACKEND_CLASS=core.authentication.FinderResourceServerBackend
OIDC_RS_BACKEND_CLASS="core.authentication.FinderResourceServerBackend"
```
**Development mode (Docs + Find)**
+1 -1
View File
@@ -118,7 +118,7 @@ class SearchQueryParametersSchema(BaseModel):
path: Optional[str] = None
nb_results: Optional[conint(ge=1, le=300)] = Field(default=50)
search_type: Optional[SearchTypeEnum] = Field(default=None)
rerank: Optional[bool] = Field(default=None)
rescore: bool = Field(default=True)
class DeleteDocumentsSchema(BaseModel):
+10 -1
View File
@@ -21,7 +21,6 @@ def embed_text(text):
json={
"input": text,
"model": settings.EMBEDDING_API_MODEL_NAME,
"dimensions": settings.EMBEDDING_DIMENSION,
"encoding_format": "float",
},
timeout=settings.EMBEDDING_REQUEST_TIMEOUT,
@@ -39,4 +38,14 @@ def embed_text(text):
logger.warning("unexpected embedding response format: %s", response.text)
return None
if len(embedding) != settings.EMBEDDING_DIMENSION:
logger.warning(
"unexpected embedding dimension: "
"EMBEDDING_DIMENSION is set to %d "
"but the configured embedding model returned a vector of dimension %d",
settings.EMBEDDING_DIMENSION,
len(embedding),
)
return None
return embedding
-122
View File
@@ -1,122 +0,0 @@
"""Reranking utilities using rerankers library."""
import logging
from functools import cache
from django.conf import settings
from flashrank import Ranker, RerankRequest
from core.services.indexing import format_document
from core.utils import get_language_value
logger = logging.getLogger(__name__)
def rerank(query: str, hits: list[dict]) -> list[dict]:
"""
Rerank search results using the configured reranker model.
Args:
query: The search query string
hits: List of OpenSearch hit objects with _source containing title and content
Returns:
List of reranked results in the same format as input with a _reranked_score
"""
reranker = get_reranker()
if reranker is None:
logger.warning("Could not import reranker, returning original results")
return hits
try:
return _rerank(reranker, query, hits)
except Exception as e: # noqa: BLE001# pylint: disable=broad-exception-caught
logger.error("Reranking failed: %s, returning original results", str(e))
return hits
@cache
def get_reranker() -> Ranker | None:
"""
Get the reranker instance.
Returns None if the reranker library is not available or if initialization fails but
does not raise an exception to avoid crashing the application.
"""
try:
logger.info("Initializing reranker model: %s", settings.RERANKER_MODEL_NAME)
return Ranker(model_name=settings.RERANKER_MODEL_NAME)
except Exception as e: # noqa: BLE001# pylint: disable=broad-exception-caught
logger.error("Failed to initialize reranker: %s", str(e))
return None
def _rerank(ranker: Ranker, query: str, original_hits: list[dict]) -> list[dict]:
"""Rerank the original results using the provided reranker."""
rerank_request = RerankRequest(
query=query, passages=prepare_rerank_data(original_hits)
)
logger.info("Reranking %d results for query: %s", len(original_hits), query)
reranked_results = ranker.rerank(rerank_request)
reranked_hits: list[dict] = []
for reranked_result in reranked_results:
matching_hits = [
hit for hit in original_hits if hit["_id"] == reranked_result["id"]
]
if not matching_hits:
logger.warning(
"Reranked document ID %s not found in original hits, skipping",
reranked_result["id"],
)
continue
if len(matching_hits) > 1:
logger.warning(
"Multiple hits found for document ID %s, using first match",
reranked_result["id"],
)
hit = matching_hits[0]
hit["_reranked_score"] = reranked_result["score"]
reranked_hits.append(hit)
logger.info("Reranking completed, returned %d results", len(reranked_hits))
return reranked_hits
def prepare_rerank_data(original_hits: list[dict]) -> list[dict]:
"""
Prepare the documents for reranking by extracting the title and content from the original hits.
"""
passages = []
for hit in original_hits:
title = get_language_value(hit["_source"], "title")
content = get_language_value(hit["_source"], "content")
passages.append({"id": hit["_id"], "text": format_document(title, content)})
return passages
def should_rerank(is_rerank_requested: bool | None) -> bool:
"""
Determine whether to perform reranking based on the input parameter and settings.
logs warning if reranking was explicitly requested but the reranker is disabled in settings.
falls back to settings.RERANKER_ENABLED if is_rerank_requested is None.
"""
if is_rerank_requested and not settings.RERANKER_ENABLED:
logger.warning(
"Reranking was explicitly requested but the reranker "
"is disabled in settings. Reranking skipped."
)
return False
if is_rerank_requested is None:
return settings.RERANKER_ENABLED
return is_rerank_requested and settings.RERANKER_ENABLED
+34 -11
View File
@@ -9,7 +9,6 @@ from core.enums import SearchTypeEnum
from .embedding import embed_text
from .opensearch import check_hybrid_search_enabled, opensearch_client
from .reranking import rerank, should_rerank
logger = logging.getLogger(__name__)
@@ -26,7 +25,7 @@ def search( # noqa : PLR0913
tags,
search_type,
path=None,
rerank_requested=None,
rescore=False,
):
"""Perform an OpenSearch search"""
query = get_query(
@@ -40,7 +39,7 @@ def search( # noqa : PLR0913
path=path,
search_type=search_type,
)
response = opensearch_client().search( # pylint: disable=unexpected-keyword-arg
return opensearch_client().search( # pylint: disable=unexpected-keyword-arg
index=",".join(search_indices),
body={
"_source": enums.SOURCE_FIELDS, # limit the fields to return
@@ -50,6 +49,7 @@ def search( # noqa : PLR0913
},
"size": nb_results,
"query": query,
"rescore": get_rescore(nb_results=nb_results) if rescore else [],
},
params=get_params(query_keys=query.keys()),
# disable=unexpected-keyword-arg because
@@ -57,14 +57,6 @@ def search( # noqa : PLR0913
ignore_unavailable=True,
)
if should_rerank(rerank_requested) and q != "*":
response["hits"]["hits"] = rerank(
query=q,
hits=response["hits"]["hits"],
)
return response
# pylint: disable=too-many-arguments, too-many-positional-arguments
def get_query( # noqa : PLR0913
@@ -230,6 +222,37 @@ def get_filter( # noqa : PLR0913
return filters
def get_rescore(nb_results):
"""
Build rescore query.
Rescore is based on the `updated_at` field to boost more recently updated documents
"""
return [
{
"window_size": nb_results,
"query": {
"rescore_query_weight": settings.RESCORE_UPDATED_AT_WEIGHT,
"rescore_query": {
"function_score": {
"functions": [
{
"gauss": {
"updated_at": {
"origin": "now",
"offset": settings.RESCORE_UPDATED_AT_OFFSET,
"scale": settings.RESCORE_UPDATED_AT_SCALE,
"decay": settings.RESCORE_UPDATED_AT_DECAY,
}
}
}
],
}
},
},
}
]
def get_params(query_keys):
"""Build OpenSearch search parameters"""
if "hybrid" in query_keys:
@@ -1,6 +1,7 @@
"""Tests indexing documents in OpenSearch over the API"""
import datetime
import logging
from django.utils import timezone
@@ -118,6 +119,43 @@ def test_api_documents_index_single_hybrid_enabled_success(settings):
assert len(chunk["content"]) < len(document["content"])
@responses.activate
def test_api_documents_index_with_wrong_embedding_dimension(settings, caplog):
"""Test embedding with wrong dimension should log a warning and not index the embedding."""
service = factories.ServiceFactory()
enable_hybrid_search(settings)
settings.EMBEDDING_DIMENSION = 8
responses.add(
responses.POST,
settings.EMBEDDING_API_PATH,
json=albert_embedding_response.response,
status=200,
)
document = factories.DocumentSchemaFactory.build()
with caplog.at_level(logging.WARNING):
APIClient().post(
"/api/v1.0/documents/index/",
document,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert any(
"unexpected embedding dimension: EMBEDDING_DIMENSION is set to 8 "
"but the configured embedding model returned a vector of dimension 1024"
in message
for message in caplog.messages
)
new_indexed_document = opensearch.opensearch_client().get(
index=service.index_name, id=str(document["id"])
)
# check embedding
assert new_indexed_document["_source"]["chunks"] is None
def test_api_documents_index_language_params():
"""language_code query param should control which language is indexed."""
service = factories.ServiceFactory()
@@ -7,7 +7,6 @@ of documents is slow and better done only once.
import logging
import random
from unittest import mock
import pytest
import responses
@@ -194,7 +193,11 @@ def test_api_documents_search_match_all(settings):
response = APIClient().post(
"/api/v1.0/documents/search/",
{"q": "*", "visited": [doc["id"] for doc in documents]},
{
"q": "*",
"visited": [doc["id"] for doc in documents],
"rescore": False,
},
format="json",
HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
)
@@ -558,6 +561,7 @@ def test_api_documents_search_with_nb_results(settings):
"q": "*",
"nb_results": nb_results,
"visited": [doc["id"] for doc in documents],
"rescore": False,
},
format="json",
HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
@@ -665,6 +669,7 @@ def test_api_documents_search_nb_results_with_filtering(settings):
"reach": "public",
"nb_results": nb_results,
"visited": public_ids,
"rescore_enable": False,
},
format="json",
HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
@@ -818,36 +823,3 @@ def test_api_documents_search_filtering_by_path(settings):
assert len(response.json()) == 2
for hit in response.json():
assert hit["_source"]["path"].startswith(path_filter)
@responses.activate
@mock.patch("core.services.reranking.rerank")
def test_api_documents_rerank_blocked(
mock_rerank,
settings,
caplog,
):
"""Test can disable rerank at run time even when enabled in settings"""
settings.RERANKER_ENABLED = True
setup_oicd_resource_server(responses, settings, sub="user_sub")
service = factories.ServiceFactory()
documents = bulk_create_documents([{"title": "Fox"}, {"title": "Wolf"}])
prepare_index(service.index_name, documents)
mock_rerank.return_value = documents
with caplog.at_level(logging.INFO):
response = APIClient().post(
"/api/v1.0/documents/search/",
{
"q": "programming",
"rerank": False,
"visited": [doc["id"] for doc in documents],
},
format="json",
HTTP_AUTHORIZATION=f"Bearer {build_authorization_bearer()}",
)
assert response.status_code == 200
assert not mock_rerank.called
-99
View File
@@ -1,99 +0,0 @@
"""
Test suite for reranking service
"""
import logging
from unittest.mock import MagicMock, patch
import pytest
from core.services.reranking import rerank
pytestmark = pytest.mark.django_db
logger = logging.getLogger(__name__)
ORIGINAL_HITS = [
{
"_id": "doc-0",
"_score": 1.5,
"_source": {
"title.en": "Dogs",
"content.en": "Dogs are domesticated wolves",
},
},
{
"_id": "doc-1",
"_score": 1.2,
"_source": {
"title.en": "Cats",
"content.en": "Cats are popular pets",
},
},
{
"_id": "doc-2",
"_score": 1.0,
"_source": {
"title.en": "Birds",
"content.en": "Birds fly in the sky",
},
},
]
@patch("core.services.reranking.get_reranker")
def test_return_original_results_if_reranker_import_fails(mocked_get_reranker, caplog):
"""Test that original results are returned if reranker import fails"""
mocked_get_reranker.return_value = None
with caplog.at_level(logging.WARNING):
result = rerank("test query", ORIGINAL_HITS)
assert result == ORIGINAL_HITS
assert any(
"Could not import reranker, returning original results" in message
for message in caplog.messages
)
@patch("core.services.reranking.get_reranker")
def test_return_original_results_if_reranking_fails(mocked_get_reranker, caplog):
"""Test that original results are returned if reranking fails"""
mock_reranker = MagicMock()
mock_reranker.rerank.side_effect = Exception("Reranking service error")
mocked_get_reranker.return_value = mock_reranker
q = "test query"
with caplog.at_level(logging.ERROR):
result = rerank(q, ORIGINAL_HITS)
assert result == ORIGINAL_HITS
assert any(
"Reranking failed: Reranking service error, returning original results"
in message
for message in caplog.messages
)
def test_reranking_success(caplog, settings):
"""Test successful reranking of search results"""
settings.RERANKER_MODEL_NAME = "ms-marco-MiniLM-L-12-v2"
q = "cats"
with caplog.at_level(logging.INFO):
reranked_hits = rerank(q, ORIGINAL_HITS.copy())
assert any(
f"Reranking 3 results for query: {q}" in message for message in caplog.messages
)
assert any(
"Reranking completed, returned 3 results" in message
for message in caplog.messages
)
# if the reranking is not too bad it should definitely rerank doc-1 at top position
assert reranked_hits[0]["_id"] == ORIGINAL_HITS[1]["_id"]
# reranked_hits should be sorted by _reranked_score
assert reranked_hits[0]["_reranked_score"] > reranked_hits[1]["_reranked_score"]
assert reranked_hits[1]["_reranked_score"] > reranked_hits[2]["_reranked_score"]
+35 -50
View File
@@ -2,9 +2,9 @@
Test suite for opensearch search service
"""
import datetime
import logging
from json import dumps as json_dumps
from unittest.mock import patch
import pytest
import responses
@@ -677,56 +677,41 @@ def test_search_filtering_by_query_path_and_tag():
assert returned_ids == expected_ids
# pylint: disable=too-many-arguments, too-many-positional-arguments
@responses.activate
@patch("core.services.search.rerank")
@pytest.mark.parametrize(
"rerank_requested, reranker_enabled, rerank_called, caplog_messages",
[
(True, True, True, []),
(
True,
False,
False,
[
"Reranking was explicitly requested but the reranker is "
"disabled in settings. Reranking skipped."
],
),
(False, True, False, []),
(False, False, False, []),
(None, False, False, []),
(None, True, True, []),
],
)
def test_search_rerank( # noqa : PLR0913
mock_rerank,
rerank_requested,
reranker_enabled,
rerank_called,
caplog_messages,
settings,
caplog,
):
"""Test that rerank activation according to args and settings"""
settings.RERANKER_ENABLED = reranker_enabled
documents = bulk_create_documents(
[
{"title": "wolf", "content": "wolves live in packs and hunt together"},
{"title": "dog", "content": "dogs are loyal domestic animals"},
{"title": "cat", "content": "cats are curious and independent pets"},
]
)
mock_rerank.return_value = documents
def test_search_with_rescore(settings):
"""Test rescore feature"""
service = factories.ServiceFactory(name=SERVICE_NAME)
today = datetime.datetime.today()
forty_days_ago = today - datetime.timedelta(days=40)
documents = [
factories.DocumentSchemaFactory.build(
users=["user_sub"],
title="my first document",
created_at=today,
updated_at=today,
),
factories.DocumentSchemaFactory.build(
users=["user_sub"],
title="another document",
created_at=forty_days_ago,
updated_at=forty_days_ago,
),
]
prepare_index(service.index_name, documents)
with caplog.at_level(logging.INFO):
search(
q="canine pet", rerank_requested=rerank_requested, **search_params(service)
)
# set a cray big RESCORE_UPDATED_AT_WEIGHT to demonstrate the effect of boosting on rescores
settings.RESCORE_UPDATED_AT_WEIGHT = 200
assert mock_rerank.called == rerank_called
for message in caplog_messages:
assert message in caplog.text
results = search(
q="another document",
**{
**search_params(service),
"rescore": True,
},
)
hits = results["hits"]["hits"]
# the first document is ranked first because it more recent
# even though the second one matches the query better
assert hits[0]["_source"]["title.en"] == "my first document"
assert hits[1]["_source"]["title.en"] == "another document"
+3 -5
View File
@@ -341,9 +341,6 @@ class SearchDocumentView(ResourceServerMixin, views.APIView):
List of public/authenticated documents the user has visited to limit
the document returned to the ones the current user has seen.
Built from linkreach list of a document in docs app.
rerank : bool, optional
Enable or disable reranking of results. If not specified, falls back to
the RERANKER_ENABLED setting.
search_type : str, optional
Type of search to perform: 'hybrid' or 'full_text'.
- 'hybrid': Uses hybrid search if enabled on the server,
@@ -361,6 +358,7 @@ class SearchDocumentView(ResourceServerMixin, views.APIView):
# Get list of groups related to the user from SCIM provider (consider caching result)
audience = self._get_service_provider_audience()
user_sub = self.request.user.sub
groups = []
params = schemas.SearchQueryParametersSchema(**request.data)
# Get index list for search query
@@ -381,7 +379,7 @@ class SearchDocumentView(ResourceServerMixin, views.APIView):
reach=params.reach,
visited=params.visited,
user_sub=user_sub,
groups=[],
groups=groups,
tags=params.tags,
path=params.path,
search_type=params.search_type
@@ -389,7 +387,7 @@ class SearchDocumentView(ResourceServerMixin, views.APIView):
else SearchTypeEnum.HYBRID
if check_hybrid_search_enabled()
else SearchTypeEnum.FULL_TEXT,
rerank_requested=params.rerank,
rescore=params.rescore,
)["hits"]["hits"]
logger.info("found %d results", len(result))
logger.debug("results %s", result)
+11 -7
View File
@@ -318,14 +318,18 @@ class Base(Configuration):
EMBEDDING_DIMENSION = values.IntegerValue(
default=1024, environ_name="EMBEDDING_DIMENSION", environ_prefix=None
)
# Reranking
RERANKER_ENABLED = values.BooleanValue(
default=False, environ_name="RERANKER_ENABLED", environ_prefix=None
# rescore
RESCORE_UPDATED_AT_WEIGHT = values.FloatValue(
default=0.2, environ_name="RESCORE_UPDATED_AT_WEIGHT", environ_prefix=None
)
RERANKER_MODEL_NAME = values.Value(
environ_name="RERANKER_MODEL_NAME",
environ_prefix=None,
RESCORE_UPDATED_AT_OFFSET = values.Value(
default="2d", environ_name="RESCORE_UPDATED_AT_OFFSET", environ_prefix=None
)
RESCORE_UPDATED_AT_SCALE = values.Value(
default="6d", environ_name="RESCORE_UPDATED_AT_SCALE", environ_prefix=None
)
RESCORE_UPDATED_AT_DECAY = values.IntegerValue(
default=0.5, environ_name="RESCORE_UPDATED_AT_SCALE", environ_prefix=None
)
# CORS
-1
View File
@@ -36,7 +36,6 @@ dependencies = [
"drf_spectacular==0.29.0",
"dockerflow==2024.4.2",
"factory_boy==3.3.3",
"Flashrank==0.2.10",
"gunicorn==23.0.0",
"py3langid==0.3.0",
"langchain-text-splitters==1.1.0",
-225
View File
@@ -14,15 +14,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/26/99/fc813cd978842c26c82534010ea849eee9ab3a13ea2b74e95cb9c99e747b/amqp-5.3.1-py3-none-any.whl", hash = "sha256:43b3319e1b4e7d1251833a93d672b4af1e40f3d632d479b98661a95f117880a2", size = 50944, upload-time = "2024-11-12T19:55:41.782Z" },
]
[[package]]
name = "annotated-doc"
version = "0.0.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/57/ba/046ceea27344560984e26a590f90bc7f4a75b06701f653222458922b558c/annotated_doc-0.0.4.tar.gz", hash = "sha256:fbcda96e87e9c92ad167c2e53839e57503ecfda18804ea28102353485033faa4", size = 7288, upload-time = "2025-11-10T22:07:42.062Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/1e/d3/26bf1008eb3d2daa8ef4cacc7f3bfdc11818d111f7e2d0201bc6e3b49d45/annotated_doc-0.0.4-py3-none-any.whl", hash = "sha256:571ac1dc6991c450b25a9c2d84a3705e2ae7a53467b5d111c24fa8baabbed320", size = 5303, upload-time = "2025-11-10T22:07:40.673Z" },
]
[[package]]
name = "annotated-types"
version = "0.7.0"
@@ -498,15 +489,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/fc/23/e22da510e1ec1488966330bf76d8ff4bd535cbfc93660eeb7657761a1bb2/faker-40.1.0-py3-none-any.whl", hash = "sha256:a616d35818e2a2387c297de80e2288083bc915e24b7e39d2fb5bc66cce3a929f", size = 1985317, upload-time = "2025-12-29T18:05:58.831Z" },
]
[[package]]
name = "filelock"
version = "3.25.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/94/b8/00651a0f559862f3bb7d6f7477b192afe3f583cc5e26403b44e59a55ab34/filelock-3.25.2.tar.gz", hash = "sha256:b64ece2b38f4ca29dd3e810287aa8c48182bbecd1ae6e9ae126c9b35f1382694", size = 40480, upload-time = "2026-03-11T20:45:38.487Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a4/a5/842ae8f0c08b61d6484b52f99a03510a3a72d23141942d216ebe81fefbce/filelock-3.25.2-py3-none-any.whl", hash = "sha256:ca8afb0da15f229774c9ad1b455ed96e85a81373065fb10446672f64444ddf70", size = 26759, upload-time = "2026-03-11T20:45:37.437Z" },
]
[[package]]
name = "find"
version = "0.0.1"
@@ -522,7 +504,6 @@ dependencies = [
{ name = "dockerflow" },
{ name = "drf-spectacular" },
{ name = "factory-boy" },
{ name = "flashrank" },
{ name = "gunicorn" },
{ name = "langchain-text-splitters" },
{ name = "mozilla-django-oidc" },
@@ -573,7 +554,6 @@ requires-dist = [
{ name = "drf-spectacular-sidecar", marker = "extra == 'dev'", specifier = "==2026.1.1" },
{ name = "factory-boy", specifier = "==3.3.3" },
{ name = "faker", marker = "extra == 'dev'", specifier = "==40.1.0" },
{ name = "flashrank", specifier = "==0.2.10" },
{ name = "gunicorn", specifier = "==23.0.0" },
{ name = "ipdb", marker = "extra == 'dev'", specifier = "==0.13.13" },
{ name = "ipython", marker = "extra == 'dev'", specifier = "==9.8.0" },
@@ -603,39 +583,6 @@ requires-dist = [
]
provides-extras = ["dev"]
[[package]]
name = "flashrank"
version = "0.2.10"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "numpy" },
{ name = "onnxruntime" },
{ name = "requests" },
{ name = "tokenizers" },
{ name = "tqdm" },
]
sdist = { url = "https://files.pythonhosted.org/packages/55/1f/176cb4a857a70c3538f637e19389ab6aed21548a1ba1d1424fccc8bba108/FlashRank-0.2.10.tar.gz", hash = "sha256:f8f82a25c32fdfc668a09dc4089421d6aab8e7f71308424b541f40bb3f01d9db", size = 18905, upload-time = "2025-01-06T13:33:01.657Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ec/99/72639cc1c9221c5bc77a2df1c2d352fe11965553bdf7d3e0856e7fcc8fd6/FlashRank-0.2.10-py3-none-any.whl", hash = "sha256:5d3272ae657d793c132d1e7917ed9e2adf49e0e1c60735583a67b051c6f0434a", size = 14511, upload-time = "2025-01-06T13:32:59.42Z" },
]
[[package]]
name = "flatbuffers"
version = "25.12.19"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e8/2d/d2a548598be01649e2d46231d151a6c56d10b964d94043a335ae56ea2d92/flatbuffers-25.12.19-py2.py3-none-any.whl", hash = "sha256:7634f50c427838bb021c2d66a3d1168e9d199b0607e6329399f04846d42e20b4", size = 26661, upload-time = "2025-12-19T23:16:13.622Z" },
]
[[package]]
name = "fsspec"
version = "2026.2.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/51/7c/f60c259dcbf4f0c47cc4ddb8f7720d2dcdc8888c8e5ad84c73ea4531cc5b/fsspec-2026.2.0.tar.gz", hash = "sha256:6544e34b16869f5aacd5b90bdf1a71acb37792ea3ddf6125ee69a22a53fb8bff", size = 313441, upload-time = "2026-02-05T21:50:53.743Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e6/ab/fb21f4c939bb440104cc2b396d3be1d9b7a9fd3c6c2a53d98c45b3d7c954/fsspec-2026.2.0-py3-none-any.whl", hash = "sha256:98de475b5cb3bd66bedd5c4679e87b4fdfe1a3bf4d707b151b3c07e58c9a2437", size = 202505, upload-time = "2026-02-05T21:50:51.819Z" },
]
[[package]]
name = "grpcio"
version = "1.76.0"
@@ -678,22 +625,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },
]
[[package]]
name = "hf-xet"
version = "1.4.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/09/08/23c84a26716382c89151b5b447b4beb19e3345f3a93d3b73009a71a57ad3/hf_xet-1.4.2.tar.gz", hash = "sha256:b7457b6b482d9e0743bd116363239b1fa904a5e65deede350fbc0c4ea67c71ea", size = 672357, upload-time = "2026-03-13T06:58:51.077Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b4/86/b40b83a2ff03ef05c4478d2672b1fc2b9683ff870e2b25f4f3af240f2e7b/hf_xet-1.4.2-cp37-abi3-macosx_10_12_x86_64.whl", hash = "sha256:71f02d6e4cdd07f344f6844845d78518cc7186bd2bc52d37c3b73dc26a3b0bc5", size = 3800339, upload-time = "2026-03-13T06:58:36.245Z" },
{ url = "https://files.pythonhosted.org/packages/64/2e/af4475c32b4378b0e92a587adb1aa3ec53e3450fd3e5fe0372a874531c00/hf_xet-1.4.2-cp37-abi3-macosx_11_0_arm64.whl", hash = "sha256:e9b38d876e94d4bdcf650778d6ebbaa791dd28de08db9736c43faff06ede1b5a", size = 3559664, upload-time = "2026-03-13T06:58:34.787Z" },
{ url = "https://files.pythonhosted.org/packages/3c/4c/781267da3188db679e601de18112021a5cb16506fe86b246e22c5401a9c4/hf_xet-1.4.2-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:77e8c180b7ef12d8a96739a4e1e558847002afe9ea63b6f6358b2271a8bdda1c", size = 4217422, upload-time = "2026-03-13T06:58:27.472Z" },
{ url = "https://files.pythonhosted.org/packages/68/47/d6cf4a39ecf6c7705f887a46f6ef5c8455b44ad9eb0d391aa7e8a2ff7fea/hf_xet-1.4.2-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:c3b3c6a882016b94b6c210957502ff7877802d0dbda8ad142c8595db8b944271", size = 3992847, upload-time = "2026-03-13T06:58:25.989Z" },
{ url = "https://files.pythonhosted.org/packages/2d/ef/e80815061abff54697239803948abc665c6b1d237102c174f4f7a9a5ffc5/hf_xet-1.4.2-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:9d9a634cc929cfbaf2e1a50c0e532ae8c78fa98618426769480c58501e8c8ac2", size = 4193843, upload-time = "2026-03-13T06:58:44.59Z" },
{ url = "https://files.pythonhosted.org/packages/54/75/07f6aa680575d9646c4167db6407c41340cbe2357f5654c4e72a1b01ca14/hf_xet-1.4.2-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:6b0932eb8b10317ea78b7da6bab172b17be03bbcd7809383d8d5abd6a2233e04", size = 4432751, upload-time = "2026-03-13T06:58:46.533Z" },
{ url = "https://files.pythonhosted.org/packages/cd/71/193eabd7e7d4b903c4aa983a215509c6114915a5a237525ec562baddb868/hf_xet-1.4.2-cp37-abi3-win_amd64.whl", hash = "sha256:ad185719fb2e8ac26f88c8100562dbf9dbdcc3d9d2add00faa94b5f106aea53f", size = 3671149, upload-time = "2026-03-13T06:58:57.07Z" },
{ url = "https://files.pythonhosted.org/packages/b4/7e/ccf239da366b37ba7f0b36095450efae4a64980bdc7ec2f51354205fdf39/hf_xet-1.4.2-cp37-abi3-win_arm64.whl", hash = "sha256:32c012286b581f783653e718c1862aea5b9eb140631685bb0c5e7012c8719a87", size = 3533426, upload-time = "2026-03-13T06:58:55.46Z" },
]
[[package]]
name = "httpcore"
version = "1.0.9"
@@ -722,26 +653,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" },
]
[[package]]
name = "huggingface-hub"
version = "1.7.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "filelock" },
{ name = "fsspec" },
{ name = "hf-xet", marker = "platform_machine == 'AMD64' or platform_machine == 'aarch64' or platform_machine == 'amd64' or platform_machine == 'arm64' or platform_machine == 'x86_64'" },
{ name = "httpx" },
{ name = "packaging" },
{ name = "pyyaml" },
{ name = "tqdm" },
{ name = "typer" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b4/a8/94ccc0aec97b996a3a68f3e1fa06a4bd7185dd02bf22bfba794a0ade8440/huggingface_hub-1.7.1.tar.gz", hash = "sha256:be38fe66e9b03c027ad755cb9e4b87ff0303c98acf515b5d579690beb0bf3048", size = 722097, upload-time = "2026-03-13T09:36:07.758Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/6f/75/ca21955d6117a394a482c7862ce96216239d0e3a53133ae8510727a8bcfa/huggingface_hub-1.7.1-py3-none-any.whl", hash = "sha256:38c6cce7419bbde8caac26a45ed22b0cea24152a8961565d70ec21f88752bfaa", size = 616308, upload-time = "2026-03-13T09:36:06.062Z" },
]
[[package]]
name = "icdiff"
version = "2.0.7"
@@ -975,18 +886,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/ff/11/22a56b615f1ca84f65fda68b1a53b6e2765f2bdb1c6d885793430a664bfe/langsmith-0.6.3-py3-none-any.whl", hash = "sha256:44fdf8084165513e6bede9dda715e7b460b1b3f57ac69f2ca3f03afa911233ec", size = 282991, upload-time = "2026-01-14T19:26:21.882Z" },
]
[[package]]
name = "markdown-it-py"
version = "4.0.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "mdurl" },
]
sdist = { url = "https://files.pythonhosted.org/packages/5b/f5/4ec618ed16cc4f8fb3b701563655a69816155e79e24a17b651541804721d/markdown_it_py-4.0.0.tar.gz", hash = "sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3", size = 73070, upload-time = "2025-08-11T12:57:52.854Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/94/54/e7d793b573f298e1c9013b8c4dade17d481164aa517d1d7148619c2cedbf/markdown_it_py-4.0.0-py3-none-any.whl", hash = "sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147", size = 87321, upload-time = "2025-08-11T12:57:51.923Z" },
]
[[package]]
name = "matplotlib-inline"
version = "0.2.1"
@@ -1008,15 +907,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/27/1a/1f68f9ba0c207934b35b86a8ca3aad8395a3d6dd7921c0686e23853ff5a9/mccabe-0.7.0-py2.py3-none-any.whl", hash = "sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e", size = 7350, upload-time = "2022-01-24T01:14:49.62Z" },
]
[[package]]
name = "mdurl"
version = "0.1.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" },
]
[[package]]
name = "mozilla-django-oidc"
version = "5.0.2"
@@ -1032,15 +922,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/a8/a9/c1664acf30ef0031ed06650039de1693bddf4cf7f58e07939e1aa80bffb7/mozilla_django_oidc-5.0.2-py3-none-any.whl", hash = "sha256:965a3533b0e299288cdf38ec2f8b550217c302ffe78ce5bd0b2d2f4bc436878b", size = 25910, upload-time = "2025-12-19T16:11:39.729Z" },
]
[[package]]
name = "mpmath"
version = "1.3.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/e0/47/dd32fa426cc72114383ac549964eecb20ecfd886d1e5ccf5340b55b02f57/mpmath-1.3.0.tar.gz", hash = "sha256:7a28eb2a9774d00c7bc92411c19a89209d5da7c4c9a9e227be8330a23a25b91f", size = 508106, upload-time = "2023-03-07T16:47:11.061Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl", hash = "sha256:a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c", size = 536198, upload-time = "2023-03-07T16:47:09.197Z" },
]
[[package]]
name = "numpy"
version = "2.4.1"
@@ -1060,25 +941,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/69/80/a828b2d0ade5e74a9fe0f4e0a17c30fdc26232ad2bc8c9f8b3197cf7cf18/numpy-2.4.1-cp312-cp312-win_arm64.whl", hash = "sha256:0e6e8f9d9ecf95399982019c01223dc130542960a12edfa8edd1122dfa66a8a8", size = 10312166, upload-time = "2026-01-10T06:43:03.673Z" },
]
[[package]]
name = "onnxruntime"
version = "1.24.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "flatbuffers" },
{ name = "numpy" },
{ name = "packaging" },
{ name = "protobuf" },
{ name = "sympy" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/d0/7f/dfdc4e52600fde4c02d59bfe98c4b057931c1114b701e175aee311a9bc11/onnxruntime-1.24.3-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:0d244227dc5e00a9ae15a7ac1eba4c4460d7876dfecafe73fb00db9f1d914d91", size = 17342578, upload-time = "2026-03-05T17:19:02.403Z" },
{ url = "https://files.pythonhosted.org/packages/1c/dc/1f5489f7b21817d4ad352bf7a92a252bd5b438bcbaa7ad20ea50814edc79/onnxruntime-1.24.3-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0a9847b870b6cb462652b547bc98c49e0efb67553410a082fde1918a38707452", size = 15150105, upload-time = "2026-03-05T16:34:56.897Z" },
{ url = "https://files.pythonhosted.org/packages/28/7c/fd253da53594ab8efbefdc85b3638620ab1a6aab6eb7028a513c853559ce/onnxruntime-1.24.3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b354afce3333f2859c7e8706d84b6c552beac39233bcd3141ce7ab77b4cabb5d", size = 17237101, upload-time = "2026-03-05T17:18:02.561Z" },
{ url = "https://files.pythonhosted.org/packages/71/5f/eaabc5699eeed6a9188c5c055ac1948ae50138697a0428d562ac970d7db5/onnxruntime-1.24.3-cp312-cp312-win_amd64.whl", hash = "sha256:44ea708c34965439170d811267c51281d3897ecfc4aa0087fa25d4a4c3eb2e4a", size = 12597638, upload-time = "2026-03-05T17:18:52.141Z" },
{ url = "https://files.pythonhosted.org/packages/cc/5c/d8066c320b90610dbeb489a483b132c3b3879b2f93f949fb5d30cfa9b119/onnxruntime-1.24.3-cp312-cp312-win_arm64.whl", hash = "sha256:48d1092b44ca2ba6f9543892e7c422c15a568481403c10440945685faf27a8d8", size = 12270943, upload-time = "2026-03-05T17:18:42.006Z" },
]
[[package]]
name = "opensearch-protobufs"
version = "0.19.0"
@@ -1569,19 +1431,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/1c/4c/cc276ce57e572c102d9542d383b2cfd551276581dc60004cb94fe8774c11/responses-0.25.8-py3-none-any.whl", hash = "sha256:0c710af92def29c8352ceadff0c3fe340ace27cf5af1bbe46fb71275bcd2831c", size = 34769, upload-time = "2025-08-08T19:01:45.018Z" },
]
[[package]]
name = "rich"
version = "14.3.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "markdown-it-py" },
{ name = "pygments" },
]
sdist = { url = "https://files.pythonhosted.org/packages/b3/c6/f3b320c27991c46f43ee9d856302c70dc2d0fb2dba4842ff739d5f46b393/rich-14.3.3.tar.gz", hash = "sha256:b8daa0b9e4eef54dd8cf7c86c03713f53241884e814f4e2f5fb342fe520f639b", size = 230582, upload-time = "2026-02-19T17:23:12.474Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/14/25/b208c5683343959b670dc001595f2f3737e051da617f66c31f7c4fa93abc/rich-14.3.3-py3-none-any.whl", hash = "sha256:793431c1f8619afa7d3b52b2cdec859562b950ea0d4b6b505397612db8d5362d", size = 310458, upload-time = "2026-02-19T17:23:13.732Z" },
]
[[package]]
name = "rpds-py"
version = "0.30.0"
@@ -1644,15 +1493,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/4d/19/8d77f9992e5cbfcaa9133c3bf63b4fbbb051248802e1e803fed5c552fbb2/sentry_sdk-2.48.0-py2.py3-none-any.whl", hash = "sha256:6b12ac256769d41825d9b7518444e57fa35b5642df4c7c5e322af4d2c8721172", size = 414555, upload-time = "2025-12-16T14:55:40.152Z" },
]
[[package]]
name = "shellingham"
version = "1.5.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/58/15/8b3609fd3830ef7b27b655beb4b4e9c62313a4e8da8c676e142cc210d58e/shellingham-1.5.4.tar.gz", hash = "sha256:8dbca0739d487e5bd35ab3ca4b36e11c4078f3a234bfce294b0a0291363404de", size = 10310, upload-time = "2023-10-24T04:13:40.426Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl", hash = "sha256:7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686", size = 9755, upload-time = "2023-10-24T04:13:38.866Z" },
]
[[package]]
name = "six"
version = "1.17.0"
@@ -1685,18 +1525,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/f1/7b/ce1eafaf1a76852e2ec9b22edecf1daa58175c090266e9f6c64afcd81d91/stack_data-0.6.3-py3-none-any.whl", hash = "sha256:d5558e0c25a4cb0853cddad3d77da9891a08cb85dd9f9f91b9f8cd66e511e695", size = 24521, upload-time = "2023-09-30T13:58:03.53Z" },
]
[[package]]
name = "sympy"
version = "1.14.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "mpmath" },
]
sdist = { url = "https://files.pythonhosted.org/packages/83/d3/803453b36afefb7c2bb238361cd4ae6125a569b4db67cd9e79846ba2d68c/sympy-1.14.0.tar.gz", hash = "sha256:d3d3fe8df1e5a0b42f0e7bdf50541697dbe7d23746e894990c030e2b05e72517", size = 7793921, upload-time = "2025-04-27T18:05:01.611Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl", hash = "sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5", size = 6299353, upload-time = "2025-04-27T18:04:59.103Z" },
]
[[package]]
name = "tenacity"
version = "9.1.2"
@@ -1706,32 +1534,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/e5/30/643397144bfbfec6f6ef821f36f33e57d35946c44a2352d3c9f0ae847619/tenacity-9.1.2-py3-none-any.whl", hash = "sha256:f77bf36710d8b73a50b2dd155c97b870017ad21afe6ab300326b0371b3b05138", size = 28248, upload-time = "2025-04-02T08:25:07.678Z" },
]
[[package]]
name = "tokenizers"
version = "0.22.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "huggingface-hub" },
]
sdist = { url = "https://files.pythonhosted.org/packages/73/6f/f80cfef4a312e1fb34baf7d85c72d4411afde10978d4657f8cdd811d3ccc/tokenizers-0.22.2.tar.gz", hash = "sha256:473b83b915e547aa366d1eee11806deaf419e17be16310ac0a14077f1e28f917", size = 372115, upload-time = "2026-01-05T10:45:15.988Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/92/97/5dbfabf04c7e348e655e907ed27913e03db0923abb5dfdd120d7b25630e1/tokenizers-0.22.2-cp39-abi3-macosx_10_12_x86_64.whl", hash = "sha256:544dd704ae7238755d790de45ba8da072e9af3eea688f698b137915ae959281c", size = 3100275, upload-time = "2026-01-05T10:41:02.158Z" },
{ url = "https://files.pythonhosted.org/packages/2e/47/174dca0502ef88b28f1c9e06b73ce33500eedfac7a7692108aec220464e7/tokenizers-0.22.2-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:1e418a55456beedca4621dbab65a318981467a2b188e982a23e117f115ce5001", size = 2981472, upload-time = "2026-01-05T10:41:00.276Z" },
{ url = "https://files.pythonhosted.org/packages/d6/84/7990e799f1309a8b87af6b948f31edaa12a3ed22d11b352eaf4f4b2e5753/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2249487018adec45d6e3554c71d46eb39fa8ea67156c640f7513eb26f318cec7", size = 3290736, upload-time = "2026-01-05T10:40:32.165Z" },
{ url = "https://files.pythonhosted.org/packages/78/59/09d0d9ba94dcd5f4f1368d4858d24546b4bdc0231c2354aa31d6199f0399/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:25b85325d0815e86e0bac263506dd114578953b7b53d7de09a6485e4a160a7dd", size = 3168835, upload-time = "2026-01-05T10:40:38.847Z" },
{ url = "https://files.pythonhosted.org/packages/47/50/b3ebb4243e7160bda8d34b731e54dd8ab8b133e50775872e7a434e524c28/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:bfb88f22a209ff7b40a576d5324bf8286b519d7358663db21d6246fb17eea2d5", size = 3521673, upload-time = "2026-01-05T10:40:56.614Z" },
{ url = "https://files.pythonhosted.org/packages/e0/fa/89f4cb9e08df770b57adb96f8cbb7e22695a4cb6c2bd5f0c4f0ebcf33b66/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1c774b1276f71e1ef716e5486f21e76333464f47bece56bbd554485982a9e03e", size = 3724818, upload-time = "2026-01-05T10:40:44.507Z" },
{ url = "https://files.pythonhosted.org/packages/64/04/ca2363f0bfbe3b3d36e95bf67e56a4c88c8e3362b658e616d1ac185d47f2/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:df6c4265b289083bf710dff49bc51ef252f9d5be33a45ee2bed151114a56207b", size = 3379195, upload-time = "2026-01-05T10:40:51.139Z" },
{ url = "https://files.pythonhosted.org/packages/2e/76/932be4b50ef6ccedf9d3c6639b056a967a86258c6d9200643f01269211ca/tokenizers-0.22.2-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:369cc9fc8cc10cb24143873a0d95438bb8ee257bb80c71989e3ee290e8d72c67", size = 3274982, upload-time = "2026-01-05T10:40:58.331Z" },
{ url = "https://files.pythonhosted.org/packages/1d/28/5f9f5a4cc211b69e89420980e483831bcc29dade307955cc9dc858a40f01/tokenizers-0.22.2-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:29c30b83d8dcd061078b05ae0cb94d3c710555fbb44861139f9f83dcca3dc3e4", size = 9478245, upload-time = "2026-01-05T10:41:04.053Z" },
{ url = "https://files.pythonhosted.org/packages/6c/fb/66e2da4704d6aadebf8cb39f1d6d1957df667ab24cff2326b77cda0dcb85/tokenizers-0.22.2-cp39-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:37ae80a28c1d3265bb1f22464c856bd23c02a05bb211e56d0c5301a435be6c1a", size = 9560069, upload-time = "2026-01-05T10:45:10.673Z" },
{ url = "https://files.pythonhosted.org/packages/16/04/fed398b05caa87ce9b1a1bb5166645e38196081b225059a6edaff6440fac/tokenizers-0.22.2-cp39-abi3-musllinux_1_2_i686.whl", hash = "sha256:791135ee325f2336f498590eb2f11dc5c295232f288e75c99a36c5dbce63088a", size = 9899263, upload-time = "2026-01-05T10:45:12.559Z" },
{ url = "https://files.pythonhosted.org/packages/05/a1/d62dfe7376beaaf1394917e0f8e93ee5f67fea8fcf4107501db35996586b/tokenizers-0.22.2-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:38337540fbbddff8e999d59970f3c6f35a82de10053206a7562f1ea02d046fa5", size = 10033429, upload-time = "2026-01-05T10:45:14.333Z" },
{ url = "https://files.pythonhosted.org/packages/fd/18/a545c4ea42af3df6effd7d13d250ba77a0a86fb20393143bbb9a92e434d4/tokenizers-0.22.2-cp39-abi3-win32.whl", hash = "sha256:a6bf3f88c554a2b653af81f3204491c818ae2ac6fbc09e76ef4773351292bc92", size = 2502363, upload-time = "2026-01-05T10:45:20.593Z" },
{ url = "https://files.pythonhosted.org/packages/65/71/0670843133a43d43070abeb1949abfdef12a86d490bea9cd9e18e37c5ff7/tokenizers-0.22.2-cp39-abi3-win_amd64.whl", hash = "sha256:c9ea31edff2968b44a88f97d784c2f16dc0729b8b143ed004699ebca91f05c48", size = 2747786, upload-time = "2026-01-05T10:45:18.411Z" },
{ url = "https://files.pythonhosted.org/packages/72/f4/0de46cfa12cdcbcd464cc59fde36912af405696f687e53a091fb432f694c/tokenizers-0.22.2-cp39-abi3-win_arm64.whl", hash = "sha256:9ce725d22864a1e965217204946f830c37876eee3b2ba6fc6255e8e903d5fcbc", size = 2612133, upload-time = "2026-01-05T10:45:17.232Z" },
]
[[package]]
name = "tomlkit"
version = "0.14.0"
@@ -1741,18 +1543,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/b5/11/87d6d29fb5d237229d67973a6c9e06e048f01cf4994dee194ab0ea841814/tomlkit-0.14.0-py3-none-any.whl", hash = "sha256:592064ed85b40fa213469f81ac584f67a4f2992509a7c3ea2d632208623a3680", size = 39310, upload-time = "2026-01-13T01:14:51.965Z" },
]
[[package]]
name = "tqdm"
version = "4.67.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/09/a9/6ba95a270c6f1fbcd8dac228323f2777d886cb206987444e4bce66338dd4/tqdm-4.67.3.tar.gz", hash = "sha256:7d825f03f89244ef73f1d4ce193cb1774a8179fd96f31d7e1dcde62092b960bb", size = 169598, upload-time = "2026-02-03T17:35:53.048Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/16/e1/3079a9ff9b8e11b846c6ac5c8b5bfb7ff225eee721825310c91b3b50304f/tqdm-4.67.3-py3-none-any.whl", hash = "sha256:ee1e4c0e59148062281c49d80b25b67771a127c85fc9676d3be5f243206826bf", size = 78374, upload-time = "2026-02-03T17:35:50.982Z" },
]
[[package]]
name = "traitlets"
version = "5.14.3"
@@ -1762,21 +1552,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/00/c0/8f5d070730d7836adc9c9b6408dec68c6ced86b304a9b26a14df072a6e8c/traitlets-5.14.3-py3-none-any.whl", hash = "sha256:b74e89e397b1ed28cc831db7aea759ba6640cb3de13090ca145426688ff1ac4f", size = 85359, upload-time = "2024-04-19T11:11:46.763Z" },
]
[[package]]
name = "typer"
version = "0.24.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "annotated-doc" },
{ name = "click" },
{ name = "rich" },
{ name = "shellingham" },
]
sdist = { url = "https://files.pythonhosted.org/packages/f5/24/cb09efec5cc954f7f9b930bf8279447d24618bb6758d4f6adf2574c41780/typer-0.24.1.tar.gz", hash = "sha256:e39b4732d65fbdcde189ae76cf7cd48aeae72919dea1fdfc16593be016256b45", size = 118613, upload-time = "2026-02-21T16:54:40.609Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/4a/91/48db081e7a63bb37284f9fbcefda7c44c277b18b0e13fbc36ea2335b71e6/typer-0.24.1-py3-none-any.whl", hash = "sha256:112c1f0ce578bfb4cab9ffdabc68f031416ebcc216536611ba21f04e9aa84c9e", size = 56085, upload-time = "2026-02-21T16:54:41.616Z" },
]
[[package]]
name = "types-requests"
version = "2.32.4.20250913"