Compare commits

..

6 Commits

Author SHA1 Message Date
charles a4f668929c (backend): add command admin
we want to run the indexing from the admin.
in `dmin/core/runindexing/`is a form to do so.

Signed-off-by: charles <charles.englebert@protonmail.com>
2026-03-26 17:41:18 +01:00
charles 209622c523 📝(backend): add docs
i am documenting the index command

Signed-off-by: charles <charles.englebert@protonmail.com>
2026-03-26 17:41:18 +01:00
charles 0076df5782 ♻️(backend): renaming
to me tasks/search.py should be tasks/indexing.py

Signed-off-by: charles <charles.englebert@protonmail.com>
2026-03-26 17:41:18 +01:00
charles e2344dcc17 (backend): add time-bound filtering options on index command
we need to be able to specify time bounds on the bulk index
to allow recovering from checkpoint after crash

Signed-off-by: charles <charles.englebert@protonmail.com>
2026-03-26 17:41:17 +01:00
charles e183ee9ed7 (backend): add crash-safe mode to index command
crash-save mode consist in indexing documents in ascending
updated_at order and save the last document.update_at.
This allows resuming indexing from the last successful batch
in case of a crash.

Signed-off-by: charles <charles.englebert@protonmail.com>
2026-03-25 17:05:40 +01:00
charles 5ce3ab71d3 📝(backend): improve docstring in tasks.search
i think some docstrings, about a counter, were outdated.
i add more details to help understand the logic.

Signed-off-by: charles <charles.englebert@protonmail.com>
2026-03-24 15:42:33 +01:00
46 changed files with 2875 additions and 2662 deletions
+1 -4
View File
@@ -10,14 +10,11 @@ and this project adheres to
- 💄(frontend) improve comments highlights #1961
### Fixed
- 🐛(backend) create_for_owner: add accesses before saving doc content #2124
## [v4.8.3] - 2026-03-23
### Changed
- ✨(backend) improve indexing command with checkpoint recovery and asynchronicity
- ♿️(frontend) improve version history list accessibility #2033
- ♿(frontend) focus skip link on headings and skip grid dropzone #1983
- ♿️(frontend) add sr-only format to export download button #2088
+52
View File
@@ -0,0 +1,52 @@
# Index Command
The `index` management command is used to index documents to the remote search indexer.
It sends an asynchronous task to the Celery worker.
## Usage
### Command Line
```bash
python manage.py index \
--lower-time-bound "2024-01-01T00:00:00" \
--upper-time-bound "2024-01-31T23:59:59" \
--batch-size 200 \
--crash-safe-mode false
```
### Django Admin
The command is available in the Django admin interface:
1. Go to `/admin/`
2. Click on **"Run Indexing"** in the CORE section
3. Fill in the form with the desired parameters
4. Click **"Run Indexing Command"**
## Parameters
### `--batch-size`
- **type:** Integer
- **default:** `50`
- **description:** Number of documents to process per batch. Higher values may improve performance but use more memory.
### `--lower-time-bound`
- **optional**: true
- **type:** ISO 8601 datetime string
- **default:** `None`
- **description:** Only documents updated after this date will be indexed.
### `--upper-time-bound` (optional)
- **optional**: true
- **type:** ISO 8601 datetime string
- **default:** `None`
- **description:** Only documents updated before this date will be indexed.
### `--crash-safe-mode`
- **type:** Boolean flag
- **default:** `True`
- **description:** When enabled, the command orders documents by `updated_at` and stores the last indexed document's timestamp in cache. This allows resuming indexing from the last successful batch in case of a crash. However, it is more computationally expensive due to sorting.b
-3
View File
@@ -60,13 +60,10 @@
"groupName": "ignored js dependencies",
"matchManagers": ["npm"],
"matchPackageNames": [
"@react-pdf/renderer",
"fetch-mock",
"node",
"node-fetch",
"react-resizable-panels",
"stylelint",
"stylelint-config-standard",
"workbox-webpack-plugin"
]
}
+36 -1
View File
@@ -1,8 +1,10 @@
"""Admin classes and registrations for core app."""
from django.conf import settings
from django.contrib import admin, messages
from django.contrib.auth import admin as auth_admin
from django.shortcuts import redirect
from django.core.management import call_command
from django.shortcuts import redirect, render
from django.utils.translation import gettext_lazy as _
from treebeard.admin import TreeAdmin
@@ -227,3 +229,36 @@ class InvitationAdmin(admin.ModelAdmin):
def save_model(self, request, obj, form, change):
obj.issuer = request.user
obj.save()
@admin.register(models.RunIndexing)
class RunIndexingAdmin(admin.ModelAdmin):
"""Admin for running indexing commands."""
def changelist_view(self, request, extra_context=None):
"""Override to avoid querying the database and handle form submission."""
if request.method == "POST":
try:
call_command(
"index",
batch_size=int(request.POST.get("batch_size")),
lower_time_bound=request.POST.get("lower_time_bound"),
upper_time_bound=request.POST.get("upper_time_bound"),
crash_safe_mode=request.POST.get("crash_safe_mode"),
)
messages.success(request, _("Indexing triggered!"))
except Exception as e:
messages.error(
request,
_("Error running indexing command: %(error)s") % {"error": str(e)},
)
return render(
request=request,
template_name="runindexing.html",
context={
**self.admin_site.each_context(request),
"title": "Run Indexing Command",
"default_batch_size": settings.SEARCH_INDEXER_BATCH_SIZE,
},
)
+1 -3
View File
@@ -507,6 +507,7 @@ class ServerCreateDocumentSerializer(serializers.Serializer):
document = models.Document.add_root(
title=validated_data["title"],
content=document_content,
creator=user,
)
@@ -525,9 +526,6 @@ class ServerCreateDocumentSerializer(serializers.Serializer):
role=models.RoleChoices.OWNER,
)
document.content = document_content
document.save()
self._send_email_notification(document, validated_data, email, language)
return document
+37 -11
View File
@@ -3,11 +3,12 @@ Handle search setup that needs to be done at bootstrap time.
"""
import logging
import time
from datetime import datetime
from django.core.management.base import BaseCommand, CommandError
from core.services.search_indexers import get_document_indexer
from core.tasks.indexing import batch_document_indexer_task
logger = logging.getLogger("docs.search.bootstrap_search")
@@ -27,6 +28,31 @@ class Command(BaseCommand):
default=50,
help="Indexation query batch size",
)
parser.add_argument(
"--lower-time-bound",
action="store",
dest="lower_time_bound",
type=datetime.fromisoformat,
default=None,
help="DateTime in ISO format. Only documents updated after this date will be indexed",
)
parser.add_argument(
"--upper-time-bound",
action="store",
dest="upper_time_bound",
type=datetime.fromisoformat,
default=None,
help="DateTime in ISO format. Only documents updated before this date will be indexed",
)
parser.add_argument(
"--crash-safe-mode",
action="store_true",
dest="crash_safe_mode",
default=True,
help="If True, order documents by updated_at and store the last indexed document.updated_at in cache. "
"This allows resuming indexing from the last successful batch in case of a crash "
"but is more computationally expensive due to sorting.",
)
def handle(self, *args, **options):
"""Launch and log search index generation."""
@@ -35,18 +61,18 @@ class Command(BaseCommand):
if not indexer:
raise CommandError("The indexer is not enabled or properly configured.")
logger.info("Starting to regenerate Find index...")
start = time.perf_counter()
batch_size = options["batch_size"]
try:
count = indexer.index(batch_size=batch_size)
batch_document_indexer_task.apply_async(
kwargs={
"lower_time_bound": options["lower_time_bound"],
"upper_time_bound": options["upper_time_bound"],
"batch_size": options["batch_size"],
"crash_safe_mode": options["crash_safe_mode"],
},
)
except Exception as err:
raise CommandError("Unable to regenerate index") from err
raise CommandError("Unable send indexing task to worker") from err
duration = time.perf_counter() - start
logger.info(
"Search index regenerated from %d document(s) in %.2f seconds.",
count,
duration,
"Document indexing task sent to worker",
)
@@ -0,0 +1,23 @@
# Generated by Django 5.2.12 on 2026-03-26 16:34
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0031_clean_onboarding_accesses'),
]
operations = [
migrations.CreateModel(
name='RunIndexing',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
],
options={
'verbose_name': 'Run Indexing',
'managed': False,
},
),
]
+10
View File
@@ -2029,3 +2029,13 @@ class Invitation(BaseModel):
"partial_update": is_admin_or_owner,
"retrieve": is_admin_or_owner,
}
class RunIndexing(models.Model):
"""Proxy model for indexing management in admin."""
class Meta:
"""Meta options."""
managed = False
verbose_name = _("Run Indexing")
+23 -16
View File
@@ -1,5 +1,6 @@
"""Document search index management utilities and indexers"""
import itertools
import logging
from abc import ABC, abstractmethod
from collections import defaultdict
@@ -7,6 +8,7 @@ from functools import cache
from django.conf import settings
from django.contrib.auth.models import AnonymousUser
from django.core.cache import cache as django_cache
from django.core.exceptions import ImproperlyConfigured
from django.utils.module_loading import import_string
@@ -18,6 +20,9 @@ from core.enums import SearchType
logger = logging.getLogger(__name__)
BULK_INDEXER_CHECKPOINT = "bulk-indexer-checkpoint"
@cache
def get_document_indexer():
"""Returns an instance of indexer service if enabled and properly configured."""
@@ -125,7 +130,7 @@ class BaseDocumentIndexer(ABC):
if not self.search_url:
raise ImproperlyConfigured("SEARCH_URL must be set in Django settings.")
def index(self, queryset=None, batch_size=None):
def index(self, queryset=None, batch_size=None, crash_safe_mode=False):
"""
Fetch documents in batches, serialize them, and push to the search backend.
@@ -134,35 +139,37 @@ class BaseDocumentIndexer(ABC):
Defaults to all documents without filter.
batch_size (int, optional): Number of documents per batch.
Defaults to settings.SEARCH_INDEXER_BATCH_SIZE.
crash_safe_mode (bool, optional): If True, order documents by updated_at
and store the last indexed document.updated_at in cache.
This allows resuming indexing from the last successful batch in case of a crash
but is more computationally expensive due to sorting.
"""
last_id = 0
count = 0
queryset = queryset or models.Document.objects.all()
batch_size = batch_size or self.batch_size
while True:
documents_batch = list(
queryset.filter(
id__gt=last_id,
).order_by("id")[:batch_size]
)
if not documents_batch:
break
if crash_safe_mode:
queryset = queryset.order_by("updated_at")
for documents_batch in itertools.batched(queryset.iterator(), batch_size):
doc_paths = [doc.path for doc in documents_batch]
last_id = documents_batch[-1].id
accesses_by_document_path = get_batch_accesses_by_users_and_teams(doc_paths)
serialized_batch = [
self.serialize_document(document, accesses_by_document_path)
for document in documents_batch
if document.content or document.title
]
if serialized_batch:
self.push(serialized_batch)
count += len(serialized_batch)
if not serialized_batch:
continue
self.push(serialized_batch)
count += len(serialized_batch)
if crash_safe_mode:
django_cache.set(
BULK_INDEXER_CHECKPOINT, serialized_batch[-1]["updated_at"]
)
return count
+1 -1
View File
@@ -10,7 +10,7 @@ from django.db.models import signals
from django.dispatch import receiver
from core import models
from core.tasks.search import trigger_batch_document_indexer
from core.tasks.indexing import trigger_batch_document_indexer
from core.utils import get_users_sharing_documents_with_cache_key
+130
View File
@@ -0,0 +1,130 @@
"""Trigger document indexation using celery task."""
from logging import getLogger
from django.conf import settings
from django.core.cache import cache
from django_redis.cache import RedisCache
from core import models
from core.services.search_indexers import (
get_document_indexer,
)
from core.utils import build_indexable_documents_queryset
from impress.celery_app import app
logger = getLogger(__file__)
@app.task
def document_indexer_task(document_id):
"""
Celery Task: Indexes a single document by its ID.
Args:
document_id: Primary key of the document to index.
"""
indexer = get_document_indexer()
if indexer:
logger.info("Start document %s indexation", document_id)
indexer.index(models.Document.objects.filter(pk=document_id))
def batch_indexer_throttle_acquire(timeout: int = 0, atomic: bool = True):
"""
Acquire a throttle lock to prevent multiple batch indexation tasks during countdown.
implements a debouncing pattern: only the first call during the timeout period
will succeed, subsequent calls are skipped until the timeout expires.
Args:
timeout (int): Lock duration in seconds (countdown period).
atomic (bool): Use Redis locks for atomic operations if available.
Returns:
bool: True if lock acquired (first call), False if already held (subsequent calls).
"""
key = "document-batch-indexer-throttle"
# Redis is used as cache database (not in tests). Use the lock feature here
# to ensure atomicity of changes to the throttle flag.
if isinstance(cache, RedisCache) and atomic:
with cache.locks(key):
return batch_indexer_throttle_acquire(timeout, atomic=False)
# cache.add() is atomic test-and-set operation:
# - If key doesn't exist: creates it with timeout and returns True
# - If key already exists: does nothing and returns False
# The key expires after timeout seconds, releasing the lock.
# The value 1 is irrelevant, only the key presence/absence matters.
return cache.add(key, 1, timeout=timeout)
@app.task
def batch_document_indexer_task(lower_time_bound=None, upper_time_bound=None, **kwargs):
"""
Celery Task: Batch indexes all documents modified since timestamp.
Args:
lower_time_bound (datetime|isoformat, optional):
indexes documents updated or deleted after this timestamp.
upper_time_bound (datetime|isoformat, optional):
indexes documents updated or deleted before this timestamp.
"""
indexer = get_document_indexer()
if not indexer:
logger.warning("Indexing task triggered but no indexer configured: skipping")
return
count = indexer.index(
queryset=build_indexable_documents_queryset(
lower_time_bound=lower_time_bound, upper_time_bound=upper_time_bound
),
**kwargs,
)
logger.info("Indexed %d documents", count)
def trigger_batch_document_indexer(document):
"""
Trigger document indexation with optional debounce mechanism.
behavior depends on SEARCH_INDEXER_COUNTDOWN setting:
- if countdown > 0 sec (async batch mode):
* schedules a batch indexation task after countdown in seconds
* uses throttle mechanism to ensure only ONE batch task runs per countdown period
* all documents modified since first trigger are indexed together
- if countdown == 0 sec (sync mode):
* executes indexation synchronously in the current thread
* no batching, no throttling, no Celery task queuing
Args:
document (Document): the document instance that triggered the indexation.
"""
countdown = int(settings.SEARCH_INDEXER_COUNTDOWN)
# DO NOT create a task if indexation is disabled
if not settings.SEARCH_INDEXER_CLASS:
return
if countdown > 0:
# use throttle to ensure only one task is scheduled per countdown period.
# if throttle acquired, schedule batch task; otherwise skip.
if batch_indexer_throttle_acquire(timeout=countdown):
logger.info(
"Add task for batch document indexation from updated_at=%s in %d seconds",
document.updated_at.isoformat(),
countdown,
)
batch_document_indexer_task.apply_async(
kwargs={"lower_time_bound": document.updated_at}, countdown=countdown
)
else:
logger.info("Skip task for batch document %s indexation", document.pk)
else:
document_indexer_task.apply(args=[document.pk])
-95
View File
@@ -1,95 +0,0 @@
"""Trigger document indexation using celery task."""
from logging import getLogger
from django.conf import settings
from django.core.cache import cache
from django.db.models import Q
from django_redis.cache import RedisCache
from core import models
from core.services.search_indexers import (
get_document_indexer,
)
from impress.celery_app import app
logger = getLogger(__file__)
@app.task
def document_indexer_task(document_id):
"""Celery Task : Sends indexation query for a document."""
indexer = get_document_indexer()
if indexer:
logger.info("Start document %s indexation", document_id)
indexer.index(models.Document.objects.filter(pk=document_id))
def batch_indexer_throttle_acquire(timeout: int = 0, atomic: bool = True):
"""
Enable the task throttle flag for a delay.
Uses redis locks if available to ensure atomic changes
"""
key = "document-batch-indexer-throttle"
# Redis is used as cache database (not in tests). Use the lock feature here
# to ensure atomicity of changes to the throttle flag.
if isinstance(cache, RedisCache) and atomic:
with cache.locks(key):
return batch_indexer_throttle_acquire(timeout, atomic=False)
# Use add() here :
# - set the flag and returns true if not exist
# - do nothing and return false if exist
return cache.add(key, 1, timeout=timeout)
@app.task
def batch_document_indexer_task(timestamp):
"""Celery Task : Sends indexation query for a batch of documents."""
indexer = get_document_indexer()
if indexer:
queryset = models.Document.objects.filter(
Q(updated_at__gte=timestamp)
| Q(deleted_at__gte=timestamp)
| Q(ancestors_deleted_at__gte=timestamp)
)
count = indexer.index(queryset)
logger.info("Indexed %d documents", count)
def trigger_batch_document_indexer(document):
"""
Trigger indexation task with debounce a delay set by the SEARCH_INDEXER_COUNTDOWN setting.
Args:
document (Document): The document instance.
"""
countdown = int(settings.SEARCH_INDEXER_COUNTDOWN)
# DO NOT create a task if indexation if disabled
if not settings.SEARCH_INDEXER_CLASS:
return
if countdown > 0:
# Each time this method is called during a countdown, we increment the
# counter and each task decrease it, so the index be run only once.
if batch_indexer_throttle_acquire(timeout=countdown):
logger.info(
"Add task for batch document indexation from updated_at=%s in %d seconds",
document.updated_at.isoformat(),
countdown,
)
batch_document_indexer_task.apply_async(
args=[document.updated_at], countdown=countdown
)
else:
logger.info("Skip task for batch document %s indexation", document.pk)
else:
document_indexer_task.apply(args=[document.pk])
@@ -0,0 +1,43 @@
{% extends "admin/base_site.html" %}
{% load i18n %}
{% block content %}
<form method="POST" >
{% csrf_token %}
<hr style="margin-bottom: 10px;">
<p>
This command triggers the indexing of all documents within the specified time bound.
</p>
<p>
See <b>docs/commands/index.md</b> for more details.
</p>
<hr style="margin-bottom: 20px;">
<div>
<label for="batch_size" style="margin-right: 10px">Batch size</label>
<input id="batch_size" type="number" name="batch_size" value={{ default_batch_size }} style="width: 80px;" >
</div>
<div>
<label for="lower_time_bound" style="margin-right: 10px">Lower time bound (optional)</label>
<input id="lower_time_bound" type="datetime-local" name="lower_time_bound" >
</div>
<div>
<label for="upper_time_bound" style="margin-right: 10px">Upper time bound (optional)</label>
<input id="upper_time_bound" type="datetime-local" name="upper_time_bound" >
</div>
<div>
<label for="crash_safe_mode" style="margin-right: 10px">Crash save mode</label>
<input id="crash_safe_mode" type="checkbox" name="crash_safe_mode" value="true" checked>
</div>
<input type="submit" value="{% translate 'Run Indexing' %}" style="margin-top: 20px;">
</form>
{% endblock %}
+147 -12
View File
@@ -2,21 +2,24 @@
Unit test for `index` command.
"""
from datetime import datetime, timedelta, timezone
from operator import itemgetter
from unittest import mock
from django.core.cache import cache
from django.core.management import CommandError, call_command
from django.db import transaction
import pytest
from core import factories
from core.services.search_indexers import FindDocumentIndexer
from core.services.search_indexers import BULK_INDEXER_CHECKPOINT, FindDocumentIndexer
from core.tests.conftest import create_document_with_updated_at
@pytest.mark.django_db
@pytest.mark.usefixtures("indexer_settings")
def test_index():
def test_index_without_bound_success():
"""Test the command `index` that run the Find app indexer for all the available documents."""
user = factories.UserFactory()
indexer = FindDocumentIndexer()
@@ -37,20 +40,152 @@ def test_index():
}
with mock.patch.object(FindDocumentIndexer, "push") as mock_push:
call_command("index")
call_command("index", crash_safe_mode=False)
push_call_args = [call.args[0] for call in mock_push.call_args_list]
push_call_args = [call.args[0] for call in mock_push.call_args_list]
# called once but with a batch of docs
mock_push.assert_called_once()
# called once but with a batch of docs
mock_push.assert_called_once()
assert sorted(push_call_args[0], key=itemgetter("id")) == sorted(
[
indexer.serialize_document(doc, accesses),
indexer.serialize_document(no_title_doc, accesses),
],
key=itemgetter("id"),
assert sorted(push_call_args[0], key=itemgetter("id")) == sorted(
[
indexer.serialize_document(doc, accesses),
indexer.serialize_document(no_title_doc, accesses),
],
key=itemgetter("id"),
)
# crash_safe_mode deactivated -> no checkpoint stored
assert cache.get(BULK_INDEXER_CHECKPOINT) is None
@pytest.mark.django_db
@pytest.mark.usefixtures("indexer_settings")
def test_index_with_both_bounds_success():
"""Test the command `index` for all documents within time bound."""
cache.clear()
lower_time_bound = datetime(2024, 2, 1, tzinfo=timezone.utc)
upper_time_bound = lower_time_bound + timedelta(days=30)
document_too_early = create_document_with_updated_at(
lower_time_bound - timedelta(days=10)
)
document_in_window_1 = create_document_with_updated_at(
lower_time_bound + timedelta(days=5)
)
document_in_window_2 = create_document_with_updated_at(
lower_time_bound + timedelta(days=15)
)
document_too_late = create_document_with_updated_at(
upper_time_bound + timedelta(days=10)
)
with mock.patch.object(FindDocumentIndexer, "push") as mock_push:
call_command(
"index",
lower_time_bound=lower_time_bound.isoformat(),
upper_time_bound=upper_time_bound.isoformat(),
)
all_push_call_args = [
document["id"]
for call_arg_list in mock_push.call_args_list
for document in call_arg_list.args[0]
]
# Only documents in window should be indexed
assert str(document_too_early.id) not in all_push_call_args
assert str(document_in_window_1.id) in all_push_call_args
assert str(document_in_window_2.id) in all_push_call_args
assert str(document_too_late.id) not in all_push_call_args
# Checkpoint should be set to last indexed document's updated_at
assert (
cache.get(BULK_INDEXER_CHECKPOINT)
== document_in_window_2.updated_at.isoformat()
)
@pytest.mark.django_db
@pytest.mark.usefixtures("indexer_settings")
def test_index_with_crash_recovery():
"""Test resuming indexing from checkpoint after a crash."""
cache.clear()
lower_time_bound = datetime(2024, 2, 1, tzinfo=timezone.utc)
upper_time_bound = lower_time_bound + timedelta(days=60)
batch_size = 2
documents = [
# batch 0
create_document_with_updated_at(lower_time_bound + timedelta(days=5)),
create_document_with_updated_at(lower_time_bound + timedelta(days=10)),
# batch 1
create_document_with_updated_at(lower_time_bound + timedelta(days=20)),
create_document_with_updated_at(lower_time_bound + timedelta(days=25)),
# batch 2 - will crash here
create_document_with_updated_at(lower_time_bound + timedelta(days=30)),
create_document_with_updated_at(lower_time_bound + timedelta(days=35)),
# batch 3
create_document_with_updated_at(lower_time_bound + timedelta(days=40)),
create_document_with_updated_at(lower_time_bound + timedelta(days=45)),
]
# First run: simulate crash on batch 3
with mock.patch.object(FindDocumentIndexer, "push") as mock_push:
def push_with_failure(data):
# Crash when we encounter document at index 4 (batch 2 with batch_size=2)
if str(documents[4].id) in [document["id"] for document in data]:
raise ConnectionError("Simulated indexing error")
mock_push.side_effect = push_with_failure
call_command(
"index",
batch_size=batch_size,
lower_time_bound=lower_time_bound.isoformat(),
upper_time_bound=upper_time_bound.isoformat(),
)
all_push_call_args = [
document["id"]
for call_arg_list in mock_push.call_args_list
for document in call_arg_list.args[0]
]
# First 2 batches should have been indexed and checkpoint saved
checkpoint = cache.get(BULK_INDEXER_CHECKPOINT)
assert checkpoint == documents[3].updated_at.isoformat()
# first 2 batches should be indexed successfully
for i in range(0, 4):
assert str(documents[i].id) in all_push_call_args
# next batch should have been attempted but failed
for i in range(4, 6):
assert str(documents[i].id) in all_push_call_args
# last batches indexing should not have been attempted
for i in range(6, 8):
assert str(documents[i].id) not in all_push_call_args
# Second run: resume from checkpoint
with mock.patch.object(FindDocumentIndexer, "push") as mock_push:
call_command(
"index",
batch_size=batch_size,
lower_time_bound=checkpoint,
upper_time_bound=upper_time_bound.isoformat(),
)
all_push_call_args = [
document["id"]
for call_arg_list in mock_push.call_args_list
for document in call_arg_list.args[0]
]
# first 2 batches should NOT be re-indexed
# except the last document of the last batch which is on the checkpoint boundary
for i in range(0, 3):
assert str(documents[i].id) not in all_push_call_args
# next batches should be indexed including the document at the checkpoint boundary
# which has already been indexed and is re-indexed
for i in range(4, 8):
assert str(documents[i].id) in all_push_call_args
@pytest.mark.django_db
+8 -1
View File
@@ -8,7 +8,7 @@ from django.core.cache import cache
import pytest
import responses
from core import factories
from core import factories, models
from core.tests.utils.urls import reload_urls
USER = "user"
@@ -143,3 +143,10 @@ def user_token():
A fixture to create a user token for testing.
"""
return build_authorization_bearer("some_token")
def create_document_with_updated_at(updated_at, **kwargs):
"""Helper to create a document with a specific updated_at, bypassing auto_now=True."""
document = factories.DocumentFactory(**kwargs)
models.Document.objects.filter(pk=document.pk).update(updated_at=updated_at)
return models.Document.objects.get(pk=document.pk)
@@ -594,43 +594,6 @@ def test_api_documents_create_for_owner_with_converter_exception(
assert response.json() == {"content": ["Could not convert content"]}
@override_settings(SERVER_TO_SERVER_API_TOKENS=["DummyToken"])
def test_api_documents_create_for_owner_access_before_content():
"""
Accesses must exist before content is saved to object storage so the owner
has access to the very first version of the document.
"""
user = factories.UserFactory()
accesses_at_save_time = []
original_save_content = Document.save_content
def capturing_save_content(self, content):
accesses_at_save_time.extend(
list(self.accesses.values_list("user__sub", "role"))
)
return original_save_content(self, content)
data = {
"title": "My Document",
"content": "Document content",
"sub": str(user.sub),
"email": user.email,
}
with patch.object(Document, "save_content", capturing_save_content):
response = APIClient().post(
"/api/v1.0/documents/create-for-owner/",
data,
format="json",
HTTP_AUTHORIZATION="Bearer DummyToken",
)
assert response.status_code == 201
# The owner access must already exist when save_content is called
assert (str(user.sub), "owner") in accesses_at_save_time
@override_settings(SERVER_TO_SERVER_API_TOKENS=["DummyToken"])
def test_api_documents_create_for_owner_with_empty_content():
"""The content should not be empty or a 400 error should be raised."""
+126
View File
@@ -2,6 +2,7 @@
import base64
import uuid
from datetime import datetime, timedelta, timezone
from django.core.cache import cache
@@ -9,6 +10,7 @@ import pycrdt
import pytest
from core import factories, utils
from core.tests.conftest import create_document_with_updated_at
pytestmark = pytest.mark.django_db
@@ -240,3 +242,127 @@ def test_utils_get_value_by_pattern_no_match():
result = utils.get_value_by_pattern(data, r"^title\.")
assert result == []
def test_utils_build_indexable_documents_queryset_no_filters():
"""Test build_indexable_documents_queryset with no filters returns all documents."""
factories.DocumentFactory.create_batch(3)
queryset = utils.build_indexable_documents_queryset()
assert queryset.count() == 3
def test_utils_build_indexable_documents_queryset_oldest_updated_at():
"""Test filtering by oldest_updated_at includes documents updated after timestamp."""
lower_time_bound = datetime(2024, 2, 1, tzinfo=timezone.utc)
document_too_old = create_document_with_updated_at(
updated_at=lower_time_bound - timedelta(days=30)
)
document_at_boundary = create_document_with_updated_at(lower_time_bound)
document_recent = create_document_with_updated_at(
updated_at=lower_time_bound + timedelta(days=15)
)
queryset = utils.build_indexable_documents_queryset(
lower_time_bound=lower_time_bound
)
assert queryset.count() == 2
assert document_too_old not in queryset
assert document_at_boundary in queryset
assert document_recent in queryset
def test_utils_build_indexable_documents_queryset_newest_updated_at():
"""Test filtering by newest_updated_at includes documents updated before timestamp."""
upper_time_bound = datetime(2024, 2, 1, tzinfo=timezone.utc)
document_old = create_document_with_updated_at(
updated_at=upper_time_bound - timedelta(days=30)
)
document_at_boundary = create_document_with_updated_at(upper_time_bound)
document_too_recent = create_document_with_updated_at(
updated_at=upper_time_bound + timedelta(days=15)
)
queryset = utils.build_indexable_documents_queryset(
upper_time_bound=upper_time_bound
)
assert queryset.count() == 2
assert document_old in queryset
assert document_at_boundary in queryset
assert document_too_recent not in queryset
def test_utils_build_indexable_documents_queryset_both_bounds():
"""Test filtering with both oldest and newest bounds."""
lower_time_bound = datetime(2024, 2, 1, tzinfo=timezone.utc)
upper_time_bound = lower_time_bound + timedelta(days=30)
document_too_early = create_document_with_updated_at(
updated_at=lower_time_bound - timedelta(days=10)
)
document_in_window = create_document_with_updated_at(
updated_at=lower_time_bound + timedelta(days=5)
)
other_document_in_window = create_document_with_updated_at(
updated_at=lower_time_bound + timedelta(days=15)
)
document_too_late = create_document_with_updated_at(
updated_at=upper_time_bound + timedelta(days=10)
)
queryset = utils.build_indexable_documents_queryset(
lower_time_bound=lower_time_bound, upper_time_bound=upper_time_bound
)
assert queryset.count() == 2
assert document_too_early not in queryset
assert document_in_window in queryset
assert other_document_in_window in queryset
assert document_too_late not in queryset
def test_utils_build_indexable_documents_queryset_deleted_at():
"""Test filtering includes documents with deleted_at in range."""
lower_time_bound = datetime(2024, 2, 1, tzinfo=timezone.utc)
document_deleted_in_range = create_document_with_updated_at(
updated_at=lower_time_bound - timedelta(days=30),
deleted_at=lower_time_bound + timedelta(days=15),
)
document_updated_before_range = create_document_with_updated_at(
updated_at=lower_time_bound - timedelta(days=30)
)
queryset = utils.build_indexable_documents_queryset(
lower_time_bound=lower_time_bound
)
assert queryset.count() == 1
assert document_deleted_in_range in queryset
assert document_updated_before_range not in queryset
def test_utils_build_indexable_documents_queryset_ancestors_deleted_at():
"""Test filtering includes documents with ancestors_deleted_at in range."""
lower_time_bound = datetime(2024, 2, 1, tzinfo=timezone.utc)
document_ancestor_deleted_in_range = create_document_with_updated_at(
updated_at=lower_time_bound - timedelta(days=30),
ancestors_deleted_at=lower_time_bound + timedelta(days=15),
)
document_updated_before_range = create_document_with_updated_at(
updated_at=lower_time_bound - timedelta(days=30)
)
queryset = utils.build_indexable_documents_queryset(
lower_time_bound=lower_time_bound
)
assert queryset.count() == 1
assert document_ancestor_deleted_in_range in queryset
assert document_updated_before_range not in queryset
+45 -1
View File
@@ -8,7 +8,7 @@ from collections import defaultdict
from django.core.cache import cache
from django.db import models as db
from django.db.models import Subquery
from django.db.models import Q, Subquery
import pycrdt
from bs4 import BeautifulSoup
@@ -168,3 +168,47 @@ def users_sharing_documents_with(user):
elapsed,
)
return result
def build_indexable_documents_queryset(lower_time_bound=None, upper_time_bound=None):
"""
Build a queryset of documents to index filtered by timestamp range.
filters out documents that have been updated, or deleted outside the time range.
Args:
lower_time_bound (datetime, optional):
keeps documents updated or deleted after this timestamp.
upper_time_bound (datetime, optional):
keeps documents updated or deleted before this timestamp.
Returns:
QuerySet: Filtered Document queryset ready for indexation.
"""
queryset = models.Document.objects.all()
conditions = Q()
if lower_time_bound and upper_time_bound:
conditions = (
Q(updated_at__gte=lower_time_bound, updated_at__lte=upper_time_bound)
| Q(deleted_at__gte=lower_time_bound, deleted_at__lte=upper_time_bound)
| Q(
ancestors_deleted_at__gte=lower_time_bound,
ancestors_deleted_at__lte=upper_time_bound,
)
)
elif lower_time_bound:
conditions = (
Q(updated_at__gte=lower_time_bound)
| Q(deleted_at__gte=lower_time_bound)
| Q(ancestors_deleted_at__gte=lower_time_bound)
)
elif upper_time_bound:
conditions = (
Q(updated_at__lte=upper_time_bound)
| Q(deleted_at__lte=upper_time_bound)
| Q(ancestors_deleted_at__lte=upper_time_bound)
)
return queryset.filter(conditions)
@@ -47,9 +47,9 @@ test.describe('Doc AI feature', () => {
await page.locator('.bn-block-outer').last().fill('Anything');
await page.getByText('Anything').selectText();
await expect(
page.locator('button[data-test="convertMarkdown"]'),
).toHaveCount(1);
expect(
await page.locator('button[data-test="convertMarkdown"]').count(),
).toBe(1);
await expect(
page.getByRole('button', { name: config.selector, exact: true }),
).toBeHidden();
@@ -155,11 +155,13 @@ test.describe('Doc Editor', () => {
expect(wsClose.isClosed()).toBeTruthy();
// Check the ws is connected again
webSocket = await page.waitForEvent('websocket', (webSocket) => {
webSocketPromise = page.waitForEvent('websocket', (webSocket) => {
return webSocket
.url()
.includes('ws://localhost:4444/collaboration/ws/?room=');
});
webSocket = await webSocketPromise;
framesentPromise = webSocket.waitForEvent('framesent');
framesent = await framesentPromise;
expect(framesent.payload).not.toBeNull();
@@ -576,10 +578,12 @@ test.describe('Doc Editor', () => {
await page.reload();
responseCanEdit = await page.waitForResponse(
responseCanEditPromise = page.waitForResponse(
(response) =>
response.url().includes(`/can-edit/`) && response.status() === 200,
);
responseCanEdit = await responseCanEditPromise;
expect(responseCanEdit.ok()).toBeTruthy();
jsonCanEdit = (await responseCanEdit.json()) as { can_edit: boolean };
@@ -236,7 +236,7 @@ test.describe('Doc Header', () => {
hasText: randomDoc,
});
await expect(row).toHaveCount(0);
expect(await row.count()).toBe(0);
});
test('it checks the options available if administrator', async ({ page }) => {
@@ -273,7 +273,7 @@ test.describe('Doc Header', () => {
await expect(getMenuItem(page, 'Delete document')).toBeDisabled();
// Click somewhere else to close the options
await page.locator('body').click({ position: { x: 0, y: 0 } });
await page.click('body', { position: { x: 0, y: 0 } });
await page.getByRole('button', { name: 'Share' }).click();
@@ -348,7 +348,7 @@ test.describe('Doc Header', () => {
await expect(getMenuItem(page, 'Delete document')).toBeDisabled();
// Click somewhere else to close the options
await page.locator('body').click({ position: { x: 0, y: 0 } });
await page.click('body', { position: { x: 0, y: 0 } });
await page.getByRole('button', { name: 'Share' }).click();
@@ -418,7 +418,7 @@ test.describe('Doc Header', () => {
await expect(getMenuItem(page, 'Delete document')).toBeDisabled();
// Click somewhere else to close the options
await page.locator('body').click({ position: { x: 0, y: 0 } });
await page.click('body', { position: { x: 0, y: 0 } });
await page.getByRole('button', { name: 'Share' }).click();
@@ -177,5 +177,5 @@ const dragAndDropFiles = async (
return dt;
}, filesData);
await page.locator(selector).dispatchEvent('drop', { dataTransfer });
await page.dispatchEvent(selector, 'drop', { dataTransfer });
};
@@ -43,12 +43,15 @@ test.describe('Doc Tree', () => {
await expect(secondSubPageItem).toBeVisible();
// Check the position of the sub pages
const allSubPageItems = docTree.getByTestId(/^doc-sub-page-item/);
await expect(allSubPageItems).toHaveCount(2);
const allSubPageItems = await docTree
.getByTestId(/^doc-sub-page-item/)
.all();
expect(allSubPageItems.length).toBe(2);
// Check that elements are in the correct order
await expect(allSubPageItems.nth(0).getByText('first move')).toBeVisible();
await expect(allSubPageItems.nth(1).getByText('second move')).toBeVisible();
await expect(allSubPageItems[0].getByText('first move')).toBeVisible();
await expect(allSubPageItems[1].getByText('second move')).toBeVisible();
// Will move the first sub page to the second position
const firstSubPageBoundingBox = await firstSubPageItem.boundingBox();
@@ -88,15 +91,17 @@ test.describe('Doc Tree', () => {
await expect(secondSubPageItem).toBeVisible();
// Check that elements are in the correct order
const allSubPageItemsAfterReload =
docTree.getByTestId(/^doc-sub-page-item/);
await expect(allSubPageItemsAfterReload).toHaveCount(2);
const allSubPageItemsAfterReload = await docTree
.getByTestId(/^doc-sub-page-item/)
.all();
expect(allSubPageItemsAfterReload.length).toBe(2);
await expect(
allSubPageItemsAfterReload.nth(0).getByText('second move'),
allSubPageItemsAfterReload[0].getByText('second move'),
).toBeVisible();
await expect(
allSubPageItemsAfterReload.nth(1).getByText('first move'),
allSubPageItemsAfterReload[1].getByText('first move'),
).toBeVisible();
});
@@ -80,9 +80,9 @@ test.describe('Doc Version', () => {
await expect(panel).toBeVisible();
await expect(page.getByText('History', { exact: true })).toBeVisible();
await expect(page.getByRole('status')).toBeHidden();
const items = panel.locator('.version-item');
await expect(items).toHaveCount(2);
await items.nth(1).click();
const items = await panel.locator('.version-item').all();
expect(items.length).toBe(2);
await items[1].click();
await expect(modal.getByText('Hello World')).toBeVisible();
await expect(modal.getByText('It will create a version')).toBeHidden();
@@ -90,7 +90,7 @@ test.describe('Doc Version', () => {
modal.locator('div[data-content-type="callout"]').first(),
).toBeHidden();
await items.nth(0).click();
await items[0].click();
await expect(modal.getByText('Hello World')).toBeVisible();
await expect(modal.getByText('It will create a version')).toBeVisible();
@@ -101,7 +101,7 @@ test.describe('Doc Version', () => {
modal.getByText('It will create a second version'),
).toBeHidden();
await items.nth(1).click();
await items[1].click();
await expect(modal.getByText('Hello World')).toBeVisible();
await expect(modal.getByText('It will create a version')).toBeHidden();
@@ -3,11 +3,6 @@ import path from 'path';
import { Locator, Page, TestInfo, expect } from '@playwright/test';
import theme_customization from '../../../../../backend/impress/configuration/theme/default.json';
export type BrowserName = 'chromium' | 'firefox' | 'webkit';
export const BROWSERS: BrowserName[] = ['chromium', 'webkit', 'firefox'];
/** Returns a locator for a menu item (handles both menuitem and menuitemradio roles) */
export const getMenuItem = (
context: Page | Locator,
@@ -18,6 +13,11 @@ export const getMenuItem = (
.getByRole('menuitem', { name, exact: options?.exact })
.or(context.getByRole('menuitemradio', { name, exact: options?.exact }));
import theme_customization from '../../../../../backend/impress/configuration/theme/default.json';
export type BrowserName = 'chromium' | 'firefox' | 'webkit';
export const BROWSERS: BrowserName[] = ['chromium', 'webkit', 'firefox'];
export const CONFIG = {
AI_BOT: {
name: 'Docs AI',
+21 -20
View File
@@ -23,7 +23,7 @@
},
"dependencies": {
"@ag-media/react-pdf-table": "2.0.3",
"@ai-sdk/openai": "3.0.45",
"@ai-sdk/openai": "3.0.19",
"@blocknote/code-block": "0.47.1",
"@blocknote/core": "0.47.1",
"@blocknote/mantine": "0.47.1",
@@ -38,20 +38,20 @@
"@emoji-mart/data": "1.2.1",
"@emoji-mart/react": "1.1.1",
"@fontsource-variable/inter": "5.2.8",
"@fontsource-variable/material-symbols-outlined": "5.2.38",
"@fontsource-variable/material-symbols-outlined": "5.2.35",
"@fontsource/material-icons": "5.2.7",
"@gouvfr-lasuite/cunningham-react": "4.2.0",
"@gouvfr-lasuite/integration": "1.0.3",
"@gouvfr-lasuite/ui-kit": "0.19.10",
"@gouvfr-lasuite/ui-kit": "0.19.6",
"@hocuspocus/provider": "3.4.4",
"@mantine/core": "8.3.17",
"@mantine/hooks": "8.3.17",
"@mantine/core": "8.3.14",
"@mantine/hooks": "8.3.14",
"@react-aria/live-announcer": "3.4.4",
"@react-pdf/renderer": "4.3.1",
"@sentry/nextjs": "10.43.0",
"@sentry/nextjs": "10.38.0",
"@tanstack/react-query": "5.90.21",
"@tiptap/extensions": "*",
"ai": "6.0.128",
"ai": "6.0.49",
"canvg": "4.0.3",
"clsx": "2.1.1",
"cmdk": "1.1.1",
@@ -59,28 +59,28 @@
"emoji-datasource-apple": "16.0.0",
"emoji-mart": "5.6.0",
"emoji-regex": "10.6.0",
"i18next": "25.8.18",
"i18next": "25.8.12",
"i18next-browser-languagedetector": "8.2.1",
"idb": "8.0.3",
"lodash": "4.17.23",
"luxon": "3.7.2",
"next": "16.1.7",
"posthog-js": "1.360.2",
"posthog-js": "1.347.2",
"react": "*",
"react-aria-components": "1.16.0",
"react-aria-components": "1.15.1",
"react-dom": "*",
"react-dropzone": "15.0.0",
"react-i18next": "16.5.8",
"react-intersection-observer": "10.0.3",
"react-i18next": "16.5.4",
"react-intersection-observer": "10.0.2",
"react-resizable-panels": "3.0.6",
"react-select": "5.10.2",
"styled-components": "6.3.11",
"styled-components": "6.3.9",
"use-debounce": "10.1.0",
"uuid": "13.0.0",
"y-protocols": "1.0.7",
"yjs": "*",
"zod": "4.3.6",
"zustand": "5.0.12"
"zod": "3.25.28",
"zustand": "5.0.11"
},
"devDependencies": {
"@svgr/webpack": "8.1.0",
@@ -89,25 +89,26 @@
"@testing-library/jest-dom": "6.9.1",
"@testing-library/react": "16.3.2",
"@testing-library/user-event": "14.6.1",
"@types/lodash": "4.17.24",
"@types/lodash": "4.17.23",
"@types/luxon": "3.7.1",
"@types/node": "*",
"@types/react": "*",
"@types/react-dom": "*",
"@vitejs/plugin-react": "6.0.1",
"@vitejs/plugin-react": "5.1.4",
"cross-env": "10.1.0",
"dotenv": "17.3.1",
"eslint-plugin-docs": "*",
"fetch-mock": "9.11.0",
"jsdom": "29.0.0",
"jsdom": "28.1.0",
"node-fetch": "2.7.0",
"prettier": "3.8.1",
"stylelint": "16.26.1",
"stylelint-config-standard": "39.0.1",
"stylelint-prettier": "5.0.3",
"typescript": "*",
"vitest": "4.1.0",
"webpack": "5.105.4",
"vite-tsconfig-paths": "6.1.1",
"vitest": "4.0.18",
"webpack": "5.105.2",
"workbox-webpack-plugin": "7.1.0"
},
"packageManager": "yarn@1.22.22"
@@ -3,7 +3,7 @@ import { useCallback, useEffect, useState } from 'react';
import * as Y from 'yjs';
import { useUpdateDoc } from '@/docs/doc-management/';
import { KEY_LIST_DOC_VERSIONS } from '@/docs/doc-versioning/api/useDocVersions';
import { KEY_LIST_DOC_VERSIONS } from '@/docs/doc-versioning';
import { toBase64 } from '@/utils/string';
import { isFirefox } from '@/utils/userAgent';
@@ -1,9 +1,4 @@
import { afterAll, afterEach, describe, expect, it, vi } from 'vitest';
vi.mock('@/docs/doc-export/components/ModalExport', () => ({
ModalExport: vi.fn(),
}));
const originalEnv = process.env.NEXT_PUBLIC_PUBLISH_AS_MIT;
describe('useModuleExport', () => {
@@ -21,12 +16,12 @@ describe('useModuleExport', () => {
const Export = await import('@/features/docs/doc-export/');
expect(Export.default).toBeUndefined();
});
}, 15000);
it('should load modules when NEXT_PUBLIC_PUBLISH_AS_MIT is false', async () => {
process.env.NEXT_PUBLIC_PUBLISH_AS_MIT = 'false';
const Export = await import('@/features/docs/doc-export/');
expect(Export.default).toHaveProperty('ModalExport');
});
}, 15000);
});
@@ -1,4 +1,6 @@
import { render, screen } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import React from 'react';
import { afterAll, beforeEach, describe, expect, vi } from 'vitest';
import { AppWrapper } from '@/tests/utils';
@@ -38,11 +40,17 @@ describe('DocToolBox - Licence', () => {
render(<DocToolBox doc={doc as any} />, {
wrapper: AppWrapper,
});
const optionsButton = await screen.findByLabelText('Export the document');
await userEvent.click(optionsButton);
// Wait for the export modal to be visible, then assert on its content text.
await screen.findByTestId('modal-export-title');
expect(
await screen.findByLabelText('Export the document'),
screen.getByText(
'Export your document to print or download in .docx, .odt, .pdf or .html(zip) format.',
),
).toBeInTheDocument();
}, 15000);
}, 10000);
test('The export button is not rendered when MIT version is activated', async () => {
process.env.NEXT_PUBLIC_PUBLISH_AS_MIT = 'true';
@@ -60,5 +68,5 @@ describe('DocToolBox - Licence', () => {
expect(
screen.queryByLabelText('Export the document'),
).not.toBeInTheDocument();
}, 15000);
});
});
@@ -1,8 +1,9 @@
import { Button, useModal } from '@gouvfr-lasuite/cunningham-react';
import { useTreeContext } from '@gouvfr-lasuite/ui-kit';
import { useQueryClient } from '@tanstack/react-query';
import dynamic from 'next/dynamic';
import { useRouter } from 'next/router';
import { useState } from 'react';
import { useEffect, useState } from 'react';
import { useTranslation } from 'react-i18next';
import { css } from 'styled-components';
@@ -38,6 +39,7 @@ import {
useDocUtils,
useDuplicateDoc,
} from '@/docs/doc-management';
import { KEY_LIST_DOC_VERSIONS } from '@/docs/doc-versioning';
import { useFocusStore, useResponsiveStore } from '@/stores';
import { useCopyCurrentEditorToClipboard } from '../hooks/useCopyCurrentEditorToClipboard';
@@ -86,6 +88,7 @@ interface DocToolBoxProps {
export const DocToolBox = ({ doc }: DocToolBoxProps) => {
const { t } = useTranslation();
const treeContext = useTreeContext<Doc>();
const queryClient = useQueryClient();
const router = useRouter();
const { isChild, isTopRoot } = useDocUtils(doc);
@@ -111,6 +114,16 @@ export const DocToolBox = ({ doc }: DocToolBoxProps) => {
listInvalidQueries: [KEY_LIST_DOC, KEY_DOC, KEY_LIST_FAVORITE_DOC],
});
useEffect(() => {
if (selectHistoryModal.isOpen) {
return;
}
void queryClient.resetQueries({
queryKey: [KEY_LIST_DOC_VERSIONS],
});
}, [selectHistoryModal.isOpen, queryClient]);
// Emoji Management
const { emoji } = getEmojiAndTitle(doc.title ?? '');
const { updateDocEmoji } = useDocTitleUpdate();
@@ -4,7 +4,7 @@ import {
} from '@gouvfr-lasuite/cunningham-react';
import { useTranslation } from 'react-i18next';
import { useEditorStore } from '@/docs/doc-editor/stores/useEditorStore';
import { useEditorStore } from '../../doc-editor';
export const useCopyCurrentEditorToClipboard = () => {
const { editor } = useEditorStore();
@@ -21,8 +21,8 @@ export const useCopyCurrentEditorToClipboard = () => {
try {
const editorContentFormatted =
asFormat === 'html'
? editor.blocksToHTMLLossy()
: editor.blocksToMarkdownLossy();
? await editor.blocksToHTMLLossy()
: await editor.blocksToMarkdownLossy();
await navigator.clipboard.writeText(editorContentFormatted);
const successMessage =
asFormat === 'markdown'
@@ -44,7 +44,7 @@ export function useUpdateDoc(queryConfig?: UseUpdateDoc) {
...queryConfig,
onSuccess: (data, variables, onMutateResult, context) => {
queryConfig?.listInvalidQueries?.forEach((queryKey) => {
void queryClient.resetQueries({
void queryClient.invalidateQueries({
queryKey: [queryKey],
});
});
@@ -14,11 +14,6 @@ vi.mock('@/stores', () => ({
}),
}));
vi.mock('@gouvfr-lasuite/ui-kit', async () => ({
...(await vi.importActual('@gouvfr-lasuite/ui-kit')),
useTreeContext: () => null,
}));
describe('useDocTitleUpdate', () => {
beforeEach(() => {
vi.clearAllMocks();
@@ -4,12 +4,9 @@ import { useEffect, useState } from 'react';
import * as Y from 'yjs';
import { Box, Text, TextErrors } from '@/components';
import { BlockNoteReader } from '@/docs/doc-editor/components/BlockNoteEditor';
import { DocEditorContainer } from '@/docs/doc-editor/components/DocEditor';
import { BlockNoteReader, DocEditorContainer } from '@/docs/doc-editor/';
import { Doc, base64ToBlocknoteXmlFragment } from '@/docs/doc-management';
import { useDocVersion } from '../api/useDocVersion';
import { Versions } from '../types';
import { Versions, useDocVersion } from '@/docs/doc-versioning/';
import { DocVersionHeader } from './DocVersionHeader';
@@ -1,4 +1,4 @@
import { Doc } from '../doc-management/types';
import { Doc } from '../doc-management';
export interface APIListVersions {
count: number;
@@ -1,4 +1,4 @@
import { act, render, screen, waitFor } from '@testing-library/react';
import { render, screen, waitFor } from '@testing-library/react';
import fetchMock from 'fetch-mock';
import i18next from 'i18next';
import { DateTime } from 'luxon';
@@ -73,9 +73,7 @@ describe('DocsGridItemDate', () => {
});
it(`should render rendered the updated_at field in the correct language`, async () => {
await act(async () => {
await i18next.changeLanguage('fr');
});
await i18next.changeLanguage('fr');
render(
<DocsGridItemDate
@@ -92,9 +90,7 @@ describe('DocsGridItemDate', () => {
expect(screen.getByText('il y a 5 jours')).toBeInTheDocument();
await act(async () => {
await i18next.changeLanguage('en');
});
await i18next.changeLanguage('en');
});
[
@@ -1,7 +1,12 @@
import { describe, expect, it, vi } from 'vitest';
import { afterEach, describe, expect, it, vi } from 'vitest';
vi.mock('@/../package.json', () => ({
default: { version: '0.0.0' },
}));
describe('DocsDB', () => {
beforeEach(() => {
afterEach(() => {
vi.clearAllMocks();
vi.resetModules();
});
@@ -15,16 +20,17 @@ describe('DocsDB', () => {
{ version: '3.0.0', expected: 3000000 },
{ version: '10.20.30', expected: 10020030 },
].forEach(({ version, expected }) => {
it(`correctly computes version for ${version}`, async () => {
it(`correctly computes version for ${version}`, () => {
vi.doMock('@/../package.json', () => ({
default: { version },
}));
const module = await import('../DocsDB');
const result = (module as any).getCurrentVersion();
expect(result).toBe(expected);
expect(result).toBeGreaterThan(previousExpected);
previousExpected = result;
return vi.importActual('../DocsDB').then((module: any) => {
const result = module.getCurrentVersion();
expect(result).toBe(expected);
expect(result).toBeGreaterThan(previousExpected);
previousExpected = result;
});
});
});
});
+8 -4
View File
@@ -1,9 +1,16 @@
/// <reference types="vitest" />
import react from '@vitejs/plugin-react';
import tsconfigPaths from 'vite-tsconfig-paths';
import { defineConfig } from 'vitest/config';
export default defineConfig({
plugins: [react()],
plugins: [
react(),
tsconfigPaths({
root: '.',
projects: ['./tsconfig.json'],
}),
],
test: {
globals: true,
environment: 'jsdom',
@@ -15,7 +22,4 @@ export default defineConfig({
define: {
'process.env.NODE_ENV': 'test',
},
resolve: {
tsconfigPaths: true,
},
});
+4 -4
View File
@@ -32,17 +32,17 @@
},
"resolutions": {
"@tiptap/extensions": "3.19.0",
"@types/node": "24.12.0",
"@types/node": "24.10.13",
"@types/react": "19.2.14",
"@types/react-dom": "19.2.3",
"eslint": "10.0.3",
"eslint": "10.0.1",
"glob": "13.0.6",
"prosemirror-view": "1.41.6",
"react": "19.2.4",
"react-dom": "19.2.4",
"typescript": "5.9.3",
"wrap-ansi": "10.0.0",
"yjs": "13.6.30"
"wrap-ansi": "9.0.2",
"yjs": "13.6.29"
},
"packageManager": "yarn@1.22.22"
}
@@ -2,7 +2,7 @@ const js = require('@eslint/js');
const nextPlugin = require('@next/eslint-plugin-next');
const tanstackQuery = require('@tanstack/eslint-plugin-query');
const { defineConfig } = require('eslint/config');
const importPlugin = require('eslint-plugin-import-x');
const importPlugin = require('eslint-plugin-import');
const jsxA11y = require('eslint-plugin-jsx-a11y');
const prettier = require('eslint-plugin-prettier');
const react = require('eslint-plugin-react');
@@ -18,22 +18,22 @@
},
"dependencies": {
"@eslint/js": "10.0.1",
"@next/eslint-plugin-next": "16.1.7",
"@next/eslint-plugin-next": "16.1.6",
"@tanstack/eslint-plugin-query": "5.91.4",
"@typescript-eslint/eslint-plugin": "8.57.1",
"@typescript-eslint/parser": "8.57.1",
"@typescript-eslint/utils": "8.57.1",
"@vitest/eslint-plugin": "1.6.12",
"eslint-config-next": "16.1.7",
"@typescript-eslint/eslint-plugin": "8.56.0",
"@typescript-eslint/parser": "8.56.0",
"@typescript-eslint/utils": "8.56.0",
"@vitest/eslint-plugin": "1.6.9",
"eslint-config-next": "16.1.6",
"eslint-config-prettier": "10.1.8",
"eslint-plugin-import-x": "4.16.2",
"eslint-plugin-import": "2.32.0",
"eslint-plugin-jest": "29.15.0",
"eslint-plugin-jsx-a11y": "6.10.2",
"eslint-plugin-playwright": "2.10.0",
"eslint-plugin-playwright": "2.5.1",
"eslint-plugin-prettier": "5.5.5",
"eslint-plugin-react": "7.37.5",
"eslint-plugin-react-hooks": "7.0.1",
"eslint-plugin-testing-library": "7.16.0",
"eslint-plugin-react-hooks": "5.2.0",
"eslint-plugin-testing-library": "7.15.4",
"prettier": "3.8.1"
},
"packageManager": "yarn@1.22.22"
+2 -2
View File
@@ -19,8 +19,8 @@
"@types/node": "*",
"eslint-plugin-docs": "*",
"eslint-plugin-import": "2.32.0",
"i18next-parser": "9.4.0",
"jest": "30.3.0",
"i18next-parser": "9.3.0",
"jest": "30.2.0",
"ts-jest": "29.4.6",
"typescript": "*",
"yargs": "18.0.0"
+6 -6
View File
@@ -18,10 +18,10 @@
"dependencies": {
"@blocknote/server-util": "0.47.1",
"@hocuspocus/server": "3.4.4",
"@sentry/node": "10.43.0",
"@sentry/profiling-node": "10.43.0",
"@sentry/node": "10.38.0",
"@sentry/profiling-node": "10.38.0",
"@tiptap/extensions": "*",
"axios": "1.13.6",
"axios": "1.13.5",
"cors": "2.8.6",
"express": "5.2.1",
"express-ws": "5.0.2",
@@ -36,16 +36,16 @@
"@types/express": "5.0.6",
"@types/express-ws": "3.0.6",
"@types/node": "*",
"@types/supertest": "7.2.0",
"@types/supertest": "6.0.3",
"@types/ws": "8.18.1",
"cross-env": "10.1.0",
"eslint-plugin-docs": "*",
"nodemon": "3.1.14",
"nodemon": "3.1.11",
"supertest": "7.2.2",
"ts-node": "10.9.2",
"tsc-alias": "1.8.16",
"typescript": "*",
"vitest": "4.1.0",
"vitest": "4.0.18",
"vitest-mock-extended": "3.1.0",
"ws": "8.19.0"
},
+2037 -2350
View File
File diff suppressed because it is too large Load Diff
-3
View File
@@ -7,9 +7,6 @@
"@html-to/text-cli": "0.5.4",
"mjml": "4.18.0"
},
"resolutions": {
"minimatch": "^9.0.7"
},
"private": true,
"scripts": {
"build-mjml-to-html": "bash ./bin/mjml-to-html",
+13 -6
View File
@@ -110,7 +110,7 @@ boolbase@^1.0.0:
resolved "https://registry.yarnpkg.com/boolbase/-/boolbase-1.0.0.tgz#68dff5fbe60c51eb37725ea9e3ed310dcc1e776e"
integrity sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==
brace-expansion@^2.0.2:
brace-expansion@^2.0.1:
version "2.0.2"
resolved "https://registry.yarnpkg.com/brace-expansion/-/brace-expansion-2.0.2.tgz#54fc53237a613d854c7bd37463aad17df87214e7"
integrity sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==
@@ -562,12 +562,19 @@ mime@^2.4.6:
resolved "https://registry.yarnpkg.com/mime/-/mime-2.6.0.tgz#a2a682a95cd4d0cb1d6257e28f83da7e35800367"
integrity sha512-USPkMeET31rOMiarsBNIHZKLGgvKc/LrjofAnBlOttf5ajRvqiRA8QsenbcooctK6d6Ts6aqZXBA+XbkKthiQg==
minimatch@9.0.1, minimatch@^9.0.3, minimatch@^9.0.4, minimatch@^9.0.7:
version "9.0.9"
resolved "https://registry.yarnpkg.com/minimatch/-/minimatch-9.0.9.tgz#9b0cb9fcb78087f6fd7eababe2511c4d3d60574e"
integrity sha512-OBwBN9AL4dqmETlpS2zasx+vTeWclWzkblfZk7KTA5j3jeOONz/tRCnZomUyvNg83wL5Zv9Ss6HMJXAgL8R2Yg==
minimatch@9.0.1:
version "9.0.1"
resolved "https://registry.yarnpkg.com/minimatch/-/minimatch-9.0.1.tgz#8a555f541cf976c622daf078bb28f29fb927c253"
integrity sha512-0jWhJpD/MdhPXwPuiRkCbfYfSKp2qnn2eOc279qI7f+osl/l+prKSrvhg157zSYvx/1nmgn2NqdT6k2Z7zSH9w==
dependencies:
brace-expansion "^2.0.2"
brace-expansion "^2.0.1"
minimatch@^9.0.3, minimatch@^9.0.4:
version "9.0.5"
resolved "https://registry.yarnpkg.com/minimatch/-/minimatch-9.0.5.tgz#d74f9dd6b57d83d8e98cfb82133b03978bc929e5"
integrity sha512-G6T0ZX48xgozx7587koeX9Ys2NYy6Gmv//P89sEte9V9whIapMNF4idKxnW2QtCcLiTWlb/wfCabAtAFWhhBow==
dependencies:
brace-expansion "^2.0.1"
"minipass@^5.0.0 || ^6.0.2 || ^7.0.0", minipass@^7.1.2:
version "7.1.2"