Compare commits

..

14 Commits

Author SHA1 Message Date
Stephan Meijer 5eca3deefc ♻️(backend) stylistic and consistency changes
Refactored converter services based on PR #1609 review comments:
- Renamed parameter to `data` across all convert methods for consistency
- Replaced recursive call with explicit sequential calls for readability
- Hardcoded CONVERSION_API_SECURE=True in Production class for security
- Removed unused YdocConverter import from viewsets.py
- Updated tests to match new error message wording

Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
2026-01-08 12:30:09 +01:00
Stephan Meijer f5d10360ae Merge branch 'main' into feature/doc-import 2026-01-08 12:11:45 +01:00
Stephan Meijer afa93e4cdd Merge remote-tracking branch 'origin/main' into feature/doc-import 2026-01-08 12:02:35 +01:00
Anthony LC f9c9e444c9 🐛(export) fix heading 4 5 6 in PDF
Heading 4 5 6 in PDF were not correctly
mapped to their corresponding styles in the
exported document.
The new export is now different than the regression
pdf, we need to update the regression PDF to match
the new correct export.
2026-01-08 11:32:58 +01:00
Anthony LC e1d2d9e5c8 (export) simplify export testing
The regression test is asserting most of the
blocks of the editor, we can remove some redundant
tests.
We improved as well the odt and docx tests by
overriding as well the document content like for pdf,
it will assert more blocks and have less code
to maintain.
2026-01-08 11:32:58 +01:00
Anthony LC ab92fc43d6 (export) add PDF regression tests
To avoid regression issues in PDF export
functionality, this commit introduces end-to-end
tests that compare exported PDFs against
known good reference files.
We compare the PDF on most of the blocks
that the editor supports.
If during a Blocknote release or pull request
there are intentional changes, the reference
files would need to be updated accordingly.
It can be done by uncommenting the line
in the test that saves the newly generated
PDF to the assets folder.
2026-01-08 11:32:58 +01:00
Anthony LC 3a3ed0453b ️(export) improve svg width when undefined
We improved the svg width calculation when the
width attribute is undefined by trying to use
the style attribute before falling back to a
default value.
2026-01-08 11:32:27 +01:00
Christopher Spelt 43a1a76a2f (backend) add documents/all endpoint with descendants
External dashboards need to find the latest updated documents across
the entire hierarchy. Currently this requires many API calls to
/documents/ and /documents/{id}/children for each level.
   
This endpoint allows retrieving all accessible documents in a single
request, enabling dashboards to efficiently display recently changed
documents regardless of their position in the hierarchy.
    
Signed-off-by: ChristopherSpelt <christopherspelt@icloud.com>
2026-01-08 09:33:55 +00:00
Cyril 62213812ee 🐛(frontend) fix clickable main content regression
removes accidental clickable area introduced by the skip-to-content feature
2026-01-08 09:32:21 +01:00
Stephan Meijer 970fcd53d6 (backend) add tests for document import feature
Added comprehensive tests covering DocSpec converter service,
converter orchestration, and document creation with file uploads.

Tests validate DOCX and Markdown conversion workflows, error
handling, service availability, and edge cases including empty
files and Unicode filenames.

Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
2025-12-05 23:02:22 +01:00
Stephan Meijer eeeaa0c76d ⬆️(docker) upgrade docspec api to version 2.4.4
Updated docspec service image from 2.0.0 to 2.4.4 to
include latest features and bug fixes.

Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
2025-12-03 15:44:54 +01:00
Anthony LC 9af753a28f (frontend) add import document area in docs grid
Add import document area with drag and drop
support in the docs grid component.
We can now import docx and and md files just
by dropping them into the designated area.

We are using the `react-dropzone` library to
handle the drag and drop functionality.
2025-12-03 15:44:54 +01:00
Anthony LC 75fe73c34b 💄(frontend) adapt the docs grid title bar
Adapt the docs grid title bar to align with the
new design. We will add a upload button in a
future iteration.
2025-12-03 15:10:07 +01:00
Stephan Meijer aebe7d06d2 (backend) Import of documents
We can now import documents in formats .docx and .md.
To do so we added a new container "docspec", which
uses the docspec service to convert
these formats to Blocknote format.

More here: #1567 #1569.
2025-12-03 15:10:06 +01:00
41 changed files with 4050 additions and 2283 deletions
+7 -1
View File
@@ -6,6 +6,12 @@ and this project adheres to
## [Unreleased]
### Added
- ✨ Import of documents #7765
- ✨(backend) add documents/all endpoint with descendants #1553
- ✅(export) add PDF regression tests #1762
## [4.3.0] - 2026-01-05
### Added
@@ -24,7 +30,7 @@ and this project adheres to
- 🐛(frontend) fix tables deletion #1739
- 🐛(frontend) fix children not display when first resize #1753
- 🐛(frontend) fix clickable main content regression #1773
## [4.2.0] - 2025-12-17
+1
View File
@@ -213,6 +213,7 @@ logs: ## display app-dev logs (follow mode)
.PHONY: logs
run-backend: ## Start only the backend application and all needed services
@$(COMPOSE) up --force-recreate -d docspec
@$(COMPOSE) up --force-recreate -d celery-dev
@$(COMPOSE) up --force-recreate -d y-provider-development
@$(COMPOSE) up --force-recreate -d nginx
+8
View File
@@ -231,6 +231,14 @@ services:
condition: service_healthy
restart: true
docspec:
image: ghcr.io/docspecio/api:2.6.0
ports:
- "4000:4000"
networks:
- lasuite
- default
networks:
lasuite:
name: lasuite-network
+14 -14
View File
@@ -7,7 +7,7 @@ Here we describe all environment variables that can be set for the docs applicat
These are the environment variables you can set for the `impress-backend` container.
| Option | Description | default |
|-------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------|
| ----------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
| AI_ALLOW_REACH_FROM | Users that can use AI must be this level. options are "public", "authenticated", "restricted" | authenticated |
| AI_API_KEY | AI key to be used for AI Base url | |
| AI_BASE_URL | OpenAI compatible AI base url | |
@@ -58,13 +58,14 @@ These are the environment variables you can set for the `impress-backend` contai
| DJANGO_EMAIL_USE_TLS | Use tls for email host connection | false |
| DJANGO_SECRET_KEY | Secret key | |
| DJANGO_SERVER_TO_SERVER_API_TOKENS | | [] |
| DOCSPEC_API_URL | URL to endpoint of DocSpec conversion API | |
| DOCUMENT_IMAGE_MAX_SIZE | Maximum size of document in bytes | 10485760 |
| FRONTEND_CSS_URL | To add a external css file to the app | |
| FRONTEND_JS_URL | To add a external js file to the app | |
| FRONTEND_JS_URL | To add a external js file to the app | |
| FRONTEND_HOMEPAGE_FEATURE_ENABLED | Frontend feature flag to display the homepage | false |
| FRONTEND_THEME | Frontend theme to use | |
| LANGUAGE_CODE | Default language | en-us |
| LASUITE_MARKETING_BACKEND | Backend used when SIGNUP_NEW_USER_TO_MARKETING_EMAIL is True. See https://github.com/suitenumerique/django-lasuite/blob/main/documentation/how-to-use-marketing-backend.md | lasuite.marketing.backends.dummy.DummyBackend |
| LASUITE_MARKETING_BACKEND | Backend used when SIGNUP_NEW_USER_TO_MARKETING_EMAIL is True. See https://github.com/suitenumerique/django-lasuite/blob/main/documentation/how-to-use-marketing-backend.md | lasuite.marketing.backends.dummy.DummyBackend |
| LASUITE_MARKETING_PARAMETERS | The parameters to configure LASUITE_MARKETING_BACKEND. See https://github.com/suitenumerique/django-lasuite/blob/main/documentation/how-to-use-marketing-backend.md | {} |
| LOGGING_LEVEL_LOGGERS_APP | Application logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL" | INFO |
| LOGGING_LEVEL_LOGGERS_ROOT | Default logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL" | INFO |
@@ -114,7 +115,6 @@ These are the environment variables you can set for the `impress-backend` contai
| Y_PROVIDER_API_BASE_URL | Y Provider url | |
| Y_PROVIDER_API_KEY | Y provider API key | |
## impress-frontend image
These are the environment variables you can set to build the `impress-frontend` image.
@@ -127,7 +127,7 @@ Example:
```
docker build -f src/frontend/Dockerfile --target frontend-production --build-arg PUBLISH_AS_MIT=false docs-frontend:latest
```
```
If you want to build the front-end application using the yarn build command, you can edit the file `src/frontend/apps/impress/.env` with the `NODE_ENV=production` environment variable and modify it. Alternatively, you can use the listed environment variables with the prefix `NEXT_PUBLIC_` (for example, `NEXT_PUBLIC_PUBLISH_AS_MIT=false`).
@@ -138,18 +138,18 @@ cd src/frontend/apps/impress
NODE_ENV=production NEXT_PUBLIC_PUBLISH_AS_MIT=false yarn build
```
| Option | Description | default |
| ----------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
| API_ORIGIN | backend domain - it uses the current domain if not initialized | |
| SW_DEACTIVATED | To not install the service worker | |
| PUBLISH_AS_MIT | Removes packages whose licences are incompatible with the MIT licence (see below) | true |
| Option | Description | default |
| -------------- | --------------------------------------------------------------------------------- | ------- |
| API_ORIGIN | backend domain - it uses the current domain if not initialized | |
| SW_DEACTIVATED | To not install the service worker | |
| PUBLISH_AS_MIT | Removes packages whose licences are incompatible with the MIT licence (see below) | true |
Packages with licences incompatible with the MIT licence:
* `xl-docx-exporter`: [GPL](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-docx-exporter/LICENSE),
* `xl-pdf-exporter`: [GPL](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-pdf-exporter/LICENSE),
* `xl-multi-column`: [GPL](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-multi-column/LICENSE).
- `xl-docx-exporter`: [GPL](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-docx-exporter/LICENSE),
- `xl-pdf-exporter`: [GPL](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-pdf-exporter/LICENSE),
- `xl-multi-column`: [GPL](https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-multi-column/LICENSE).
In `.env.development`, `PUBLISH_AS_MIT` is set to `false`, allowing developers to test Docs with all its features.
⚠️ If you run Docs in production with `PUBLISH_AS_MIT` set to `false` make sure you fulfill your BlockNote licensing or [subscription](https://www.blocknotejs.org/about#partner-with-us) obligations.
+2
View File
@@ -76,6 +76,8 @@ DJANGO_SERVER_TO_SERVER_API_TOKENS=server-api-token
Y_PROVIDER_API_BASE_URL=http://y-provider-development:4444/api/
Y_PROVIDER_API_KEY=yprovider-api-key
DOCSPEC_API_URL=http://docspec:4000/conversion
# Theme customization
THEME_CUSTOMIZATION_CACHE_TIMEOUT=15
+1 -1
View File
@@ -6,4 +6,4 @@ Y_PROVIDER_API_BASE_URL=http://y-provider:4444/api/
# Throttle
API_DOCUMENT_THROTTLE_RATE=1000/min
API_CONFIG_THROTTLE_RATE=1000/min
API_CONFIG_THROTTLE_RATE=1000/min
+7 -2
View File
@@ -15,10 +15,11 @@ import magic
from rest_framework import serializers
from core import choices, enums, models, utils, validators
from core.services import mime_types
from core.services.ai_services import AI_ACTIONS
from core.services.converter_services import (
ConversionError,
YdocConverter,
Converter,
)
@@ -188,6 +189,7 @@ class DocumentSerializer(ListDocumentSerializer):
content = serializers.CharField(required=False)
websocket = serializers.BooleanField(required=False, write_only=True)
file = serializers.FileField(required=False, write_only=True, allow_null=True)
class Meta:
model = models.Document
@@ -204,6 +206,7 @@ class DocumentSerializer(ListDocumentSerializer):
"deleted_at",
"depth",
"excerpt",
"file",
"is_favorite",
"link_role",
"link_reach",
@@ -461,7 +464,9 @@ class ServerCreateDocumentSerializer(serializers.Serializer):
language = user.language or language
try:
document_content = YdocConverter().convert(validated_data["content"])
document_content = Converter().convert(
validated_data["content"], mime_types.MARKDOWN, mime_types.YJS
)
except ConversionError as err:
raise serializers.ValidationError(
{"content": ["Could not convert content"]}
+87 -8
View File
@@ -41,17 +41,19 @@ from rest_framework.permissions import AllowAny
from core import authentication, choices, enums, models
from core.api.filters import remove_accents
from core.services import mime_types
from core.services.ai_services import AIService
from core.services.collaboration_services import CollaborationService
from core.services.converter_services import (
ConversionError,
Converter,
)
from core.services.converter_services import (
ServiceUnavailableError as YProviderServiceUnavailableError,
)
from core.services.converter_services import (
ValidationError as YProviderValidationError,
)
from core.services.converter_services import (
YdocConverter,
)
from core.services.search_indexers import (
get_document_indexer,
get_visited_document_ids_of,
@@ -388,6 +390,7 @@ class DocumentViewSet(
queryset = models.Document.objects.select_related("creator").all()
serializer_class = serializers.DocumentSerializer
ai_translate_serializer_class = serializers.AITranslateSerializer
all_serializer_class = serializers.ListDocumentSerializer
children_serializer_class = serializers.ListDocumentSerializer
descendants_serializer_class = serializers.ListDocumentSerializer
list_serializer_class = serializers.ListDocumentSerializer
@@ -524,6 +527,28 @@ class DocumentViewSet(
"IN SHARE ROW EXCLUSIVE MODE;"
)
# Remove file from validated_data as it's not a model field
# Process it if present
uploaded_file = serializer.validated_data.pop("file", None)
# If a file is uploaded, convert it to Yjs format and set as content
if uploaded_file:
try:
file_content = uploaded_file.read()
converter = Converter()
converted_content = converter.convert(
file_content,
content_type=uploaded_file.content_type,
accept=mime_types.YJS,
)
serializer.validated_data["content"] = converted_content
serializer.validated_data["title"] = uploaded_file.name
except ConversionError as err:
raise drf.exceptions.ValidationError(
{"file": ["Could not convert file content"]}
) from err
obj = models.Document.add_root(
creator=self.request.user,
**serializer.validated_data,
@@ -858,6 +883,60 @@ class DocumentViewSet(
},
)
@drf.decorators.action(
detail=False,
methods=["get"],
)
def all(self, request, *args, **kwargs):
"""
Returns all documents (including descendants) that the user has access to.
Unlike the list endpoint which only returns top-level documents, this endpoint
returns all documents including children, grandchildren, etc.
"""
user = self.request.user
accessible_documents = self.get_queryset()
accessible_paths = list(accessible_documents.values_list("path", flat=True))
if not accessible_paths:
return self.get_response_for_queryset(self.queryset.none())
# Build query to include all descendants using path prefix matching
descendants_clause = db.Q()
for path in accessible_paths:
descendants_clause |= db.Q(path__startswith=path)
queryset = self.queryset.filter(
descendants_clause, ancestors_deleted_at__isnull=True
)
# Apply existing filters
filterset = ListDocumentFilter(
self.request.GET, queryset=queryset, request=self.request
)
if not filterset.is_valid():
raise drf.exceptions.ValidationError(filterset.errors)
filter_data = filterset.form.cleaned_data
# Filter as early as possible on fields that are available on the model
for field in ["is_creator_me", "title"]:
queryset = filterset.filters[field].filter(queryset, filter_data[field])
queryset = queryset.annotate_user_roles(user)
# Annotate favorite status and filter if applicable as late as possible
queryset = queryset.annotate_is_favorite(user)
for field in ["is_favorite", "is_masked"]:
queryset = filterset.filters[field].filter(queryset, filter_data[field])
# Apply ordering only now that everything is filtered and annotated
queryset = filters.OrderingFilter().filter_queryset(
self.request, queryset, self
)
return self.get_response_for_queryset(queryset)
@drf.decorators.action(
detail=True,
methods=["get"],
@@ -1700,14 +1779,14 @@ class DocumentViewSet(
if base64_content is not None:
# Convert using the y-provider service
try:
yprovider = YdocConverter()
yprovider = Converter()
result = yprovider.convert(
base64.b64decode(base64_content),
"application/vnd.yjs.doc",
mime_types.YJS,
{
"markdown": "text/markdown",
"html": "text/html",
"json": "application/json",
"markdown": mime_types.MARKDOWN,
"html": mime_types.HTML,
"json": mime_types.JSON,
}[content_format],
)
content = result
+77 -11
View File
@@ -1,11 +1,14 @@
"""Y-Provider API services."""
import typing
from base64 import b64encode
from django.conf import settings
import requests
from core.services import mime_types
class ConversionError(Exception):
"""Base exception for conversion-related errors."""
@@ -19,8 +22,73 @@ class ServiceUnavailableError(ConversionError):
"""Raised when the conversion service is unavailable."""
class ConverterProtocol(typing.Protocol):
"""Protocol for converter classes."""
def convert(self, data, content_type, accept):
"""Convert content from one format to another."""
class Converter:
"""Orchestrates conversion between different formats using specialized converters."""
docspec: ConverterProtocol
ydoc: ConverterProtocol
def __init__(self):
self.docspec = DocSpecConverter()
self.ydoc = YdocConverter()
def convert(self, data, content_type, accept):
"""Convert input into other formats using external microservices."""
if content_type == mime_types.DOCX and accept == mime_types.YJS:
blocknote_data = self.docspec.convert(
data, mime_types.DOCX, mime_types.BLOCKNOTE
)
return self.ydoc.convert(
blocknote_data, mime_types.BLOCKNOTE, mime_types.YJS
)
return self.ydoc.convert(data, content_type, accept)
class DocSpecConverter:
"""Service class for DocSpec conversion-related operations."""
def _request(self, url, data, content_type):
"""Make a request to the DocSpec API."""
response = requests.post(
url,
headers={"Accept": mime_types.BLOCKNOTE},
files={"file": ("document.docx", data, content_type)},
timeout=settings.CONVERSION_API_TIMEOUT,
verify=settings.CONVERSION_API_SECURE,
)
response.raise_for_status()
return response
def convert(self, data, content_type, accept):
"""Convert a Document to BlockNote."""
if not data:
raise ValidationError("Input data cannot be empty")
if content_type != mime_types.DOCX or accept != mime_types.BLOCKNOTE:
raise ValidationError(
f"Conversion from {content_type} to {accept} is not supported."
)
try:
return self._request(settings.DOCSPEC_API_URL, data, content_type).content
except requests.RequestException as err:
raise ServiceUnavailableError(
"Failed to connect to DocSpec conversion service",
) from err
class YdocConverter:
"""Service class for conversion-related operations."""
"""Service class for YDoc conversion-related operations."""
@property
def auth_header(self):
@@ -44,29 +112,27 @@ class YdocConverter:
response.raise_for_status()
return response
def convert(
self, text, content_type="text/markdown", accept="application/vnd.yjs.doc"
):
def convert(self, data, content_type=mime_types.MARKDOWN, accept=mime_types.YJS):
"""Convert a Markdown text into our internal format using an external microservice."""
if not text:
raise ValidationError("Input text cannot be empty")
if not data:
raise ValidationError("Input data cannot be empty")
try:
response = self._request(
f"{settings.Y_PROVIDER_API_BASE_URL}{settings.CONVERSION_API_ENDPOINT}/",
text,
data,
content_type,
accept,
)
if accept == "application/vnd.yjs.doc":
if accept == mime_types.YJS:
return b64encode(response.content).decode("utf-8")
if accept in {"text/markdown", "text/html"}:
if accept in {mime_types.MARKDOWN, "text/html"}:
return response.text
if accept == "application/json":
if accept == mime_types.JSON:
return response.json()
raise ValidationError("Unsupported format")
except requests.RequestException as err:
raise ServiceUnavailableError(
"Failed to connect to conversion service",
f"Failed to connect to YDoc conversion service {content_type}, {accept}",
) from err
+8
View File
@@ -0,0 +1,8 @@
"""MIME type constants for document conversion."""
BLOCKNOTE = "application/vnd.blocknote+json"
YJS = "application/vnd.yjs.doc"
MARKDOWN = "text/markdown"
JSON = "application/json"
DOCX = "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
HTML = "text/html"
@@ -0,0 +1,427 @@
"""
Tests for Documents API endpoint in impress's core app: all
The 'all' endpoint returns ALL documents (including descendants) that the user has access to.
This is different from the 'list' endpoint which only returns top-level documents.
"""
from datetime import timedelta
from unittest import mock
from django.utils import timezone
import pytest
from rest_framework.test import APIClient
from core import factories, models
pytestmark = pytest.mark.django_db
@pytest.mark.parametrize("role", models.LinkRoleChoices.values)
@pytest.mark.parametrize("reach", models.LinkReachChoices.values)
def test_api_documents_all_anonymous(reach, role):
"""
Anonymous users should not be able to list any documents via the all endpoint
whatever the link reach and link role.
"""
parent = factories.DocumentFactory(link_reach=reach, link_role=role)
factories.DocumentFactory(parent=parent, link_reach=reach, link_role=role)
response = APIClient().get("/api/v1.0/documents/all/")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 0
def test_api_documents_all_authenticated_with_children():
"""
Authenticated users should see all documents including children,
even though children don't have DocumentAccess records.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Create a document tree: parent -> child -> grandchild
parent = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=parent, user=user, role="owner")
child = factories.DocumentFactory(parent=parent)
grandchild = factories.DocumentFactory(parent=child)
# Verify setup
assert models.DocumentAccess.objects.filter(document=parent).count() == 1
assert models.DocumentAccess.objects.filter(document=child).count() == 0
assert models.DocumentAccess.objects.filter(document=grandchild).count() == 0
response = client.get("/api/v1.0/documents/all/")
assert response.status_code == 200
results = response.json()["results"]
# All three documents should be returned (parent + child + grandchild)
assert len(results) == 3
results_ids = {result["id"] for result in results}
assert results_ids == {str(parent.id), str(child.id), str(grandchild.id)}
depths = {result["depth"] for result in results}
assert depths == {1, 2, 3}
def test_api_documents_all_authenticated_multiple_trees():
"""
Users should see all accessible documents from multiple document trees.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Tree 1: User has access
tree1_parent = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=tree1_parent, user=user)
tree1_child = factories.DocumentFactory(parent=tree1_parent)
# Tree 2: User has access
tree2_parent = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=tree2_parent, user=user)
tree2_child1 = factories.DocumentFactory(parent=tree2_parent)
tree2_child2 = factories.DocumentFactory(parent=tree2_parent)
# Tree 3: User does NOT have access
tree3_parent = factories.DocumentFactory()
factories.DocumentFactory(parent=tree3_parent)
response = client.get("/api/v1.0/documents/all/")
assert response.status_code == 200
results = response.json()["results"]
# Should return 5 documents (tree1: 2, tree2: 3, tree3: 0)
assert len(results) == 5
results_ids = {result["id"] for result in results}
expected_ids = {
str(tree1_parent.id),
str(tree1_child.id),
str(tree2_parent.id),
str(tree2_child1.id),
str(tree2_child2.id),
}
assert results_ids == expected_ids
def test_api_documents_all_authenticated_explicit_access_to_parent_and_child():
"""
When a user has explicit DocumentAccess to both parent AND child,
both should appear in the 'all' endpoint results (unlike 'list' which deduplicates).
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Parent with explicit access
parent = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=parent, user=user)
# Child also has explicit access (e.g., shared separately)
child = factories.DocumentFactory(parent=parent)
factories.UserDocumentAccessFactory(document=child, user=user)
# Grandchild has no explicit access
grandchild = factories.DocumentFactory(parent=child)
# Verify setup
assert models.DocumentAccess.objects.filter(document=parent).count() == 1
assert models.DocumentAccess.objects.filter(document=child).count() == 1
assert models.DocumentAccess.objects.filter(document=grandchild).count() == 0
response = client.get("/api/v1.0/documents/all/")
assert response.status_code == 200
results = response.json()["results"]
# All three should appear
assert len(results) == 3
results_ids = {result["id"] for result in results}
assert results_ids == {str(parent.id), str(child.id), str(grandchild.id)}
# Each document should appear exactly once (no duplicates)
results_ids_list = [result["id"] for result in results]
assert len(results_ids_list) == len(set(results_ids_list)) # No duplicates
def test_api_documents_all_authenticated_via_team(mock_user_teams):
"""
Users should see all documents (including descendants) for documents accessed via teams.
"""
mock_user_teams.return_value = ["team1", "team2"]
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Document tree via team1
parent1 = factories.DocumentFactory()
factories.TeamDocumentAccessFactory(document=parent1, team="team1")
child1 = factories.DocumentFactory(parent=parent1)
# Document tree via team2
parent2 = factories.DocumentFactory()
factories.TeamDocumentAccessFactory(document=parent2, team="team2")
child2 = factories.DocumentFactory(parent=parent2)
# Document tree via unknown team
parent3 = factories.DocumentFactory()
factories.TeamDocumentAccessFactory(document=parent3, team="team3")
factories.DocumentFactory(parent=parent3)
response = client.get("/api/v1.0/documents/all/")
assert response.status_code == 200
results = response.json()["results"]
# Should return 4 documents (team1: 2, team2: 2, team3: 0)
assert len(results) == 4
results_ids = {result["id"] for result in results}
expected_ids = {
str(parent1.id),
str(child1.id),
str(parent2.id),
str(child2.id),
}
assert results_ids == expected_ids
def test_api_documents_all_authenticated_soft_deleted():
"""
Soft-deleted documents and their descendants should not be included.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Active tree
active_parent = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=active_parent, user=user)
active_child = factories.DocumentFactory(parent=active_parent)
# Soft-deleted tree
deleted_parent = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=deleted_parent, user=user)
_deleted_child = factories.DocumentFactory(parent=deleted_parent)
deleted_parent.soft_delete()
response = client.get("/api/v1.0/documents/all/")
assert response.status_code == 200
results = response.json()["results"]
# Should only return active documents
assert len(results) == 2
results_ids = {result["id"] for result in results}
assert results_ids == {str(active_parent.id), str(active_child.id)}
def test_api_documents_all_authenticated_permanently_deleted():
"""
Permanently deleted documents should not be included.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Active tree
active_parent = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=active_parent, user=user)
active_child = factories.DocumentFactory(parent=active_parent)
# Permanently deleted tree (deleted > 30 days ago)
deleted_parent = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=deleted_parent, user=user)
_deleted_child = factories.DocumentFactory(parent=deleted_parent)
fourty_days_ago = timezone.now() - timedelta(days=40)
with mock.patch("django.utils.timezone.now", return_value=fourty_days_ago):
deleted_parent.soft_delete()
response = client.get("/api/v1.0/documents/all/")
assert response.status_code == 200
results = response.json()["results"]
# Should only return active documents
assert len(results) == 2
results_ids = {result["id"] for result in results}
assert results_ids == {str(active_parent.id), str(active_child.id)}
def test_api_documents_all_authenticated_link_reach_restricted():
"""
Documents with link_reach=restricted accessed via LinkTrace should not appear
in the all endpoint results.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Document with direct access (should appear)
parent_with_access = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=parent_with_access, user=user)
child_with_access = factories.DocumentFactory(parent=parent_with_access)
# Document with only LinkTrace and restricted reach (should NOT appear)
parent_restricted = factories.DocumentFactory(
link_reach="restricted", link_traces=[user]
)
factories.DocumentFactory(parent=parent_restricted)
response = client.get("/api/v1.0/documents/all/")
assert response.status_code == 200
results = response.json()["results"]
# Only documents with direct access should appear
assert len(results) == 2
results_ids = {result["id"] for result in results}
assert results_ids == {str(parent_with_access.id), str(child_with_access.id)}
@pytest.mark.parametrize("reach", ["public", "authenticated"])
def test_api_documents_all_authenticated_link_reach_public_or_authenticated(reach):
"""
Documents with link_reach=public or authenticated accessed via LinkTrace
should appear with all their descendants.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Document accessed via LinkTrace with non-restricted reach
parent = factories.DocumentFactory(link_reach=reach, link_traces=[user])
child = factories.DocumentFactory(parent=parent)
grandchild = factories.DocumentFactory(parent=child)
response = client.get("/api/v1.0/documents/all/")
assert response.status_code == 200
results = response.json()["results"]
# All descendants should be included
assert len(results) == 3
results_ids = {result["id"] for result in results}
assert results_ids == {str(parent.id), str(child.id), str(grandchild.id)}
def test_api_documents_all_format():
"""Validate the format of documents as returned by the all endpoint."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
document = factories.DocumentFactory()
access = factories.UserDocumentAccessFactory(document=document, user=user)
child = factories.DocumentFactory(parent=document)
response = client.get("/api/v1.0/documents/all/")
assert response.status_code == 200
content = response.json()
results = content.pop("results")
# Check pagination structure
assert content == {
"count": 2,
"next": None,
"previous": None,
}
# Verify parent document format
parent_result = [r for r in results if r["id"] == str(document.id)][0]
assert parent_result == {
"id": str(document.id),
"abilities": document.get_abilities(user),
"ancestors_link_reach": None,
"ancestors_link_role": None,
"computed_link_reach": document.computed_link_reach,
"computed_link_role": document.computed_link_role,
"created_at": document.created_at.isoformat().replace("+00:00", "Z"),
"creator": str(document.creator.id),
"deleted_at": None,
"depth": 1,
"excerpt": document.excerpt,
"is_favorite": False,
"link_reach": document.link_reach,
"link_role": document.link_role,
"nb_accesses_ancestors": 1,
"nb_accesses_direct": 1,
"numchild": 1,
"path": document.path,
"title": document.title,
"updated_at": document.updated_at.isoformat().replace("+00:00", "Z"),
"user_role": access.role,
}
# Verify child document format
child_result = [r for r in results if r["id"] == str(child.id)][0]
assert child_result["depth"] == 2
assert child_result["user_role"] == access.role # Inherited from parent
assert child_result["nb_accesses_direct"] == 0 # No direct access on child
def test_api_documents_all_distinct():
"""
A document should only appear once even if the user has multiple access paths to it.
"""
user = factories.UserFactory()
other_user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Document with multiple accesses for the same user
document = factories.DocumentFactory(users=[user, other_user])
child = factories.DocumentFactory(parent=document)
response = client.get("/api/v1.0/documents/all/")
assert response.status_code == 200
results = response.json()["results"]
# Should return 2 documents (parent + child), each appearing once
assert len(results) == 2
results_ids = [result["id"] for result in results]
assert results_ids.count(str(document.id)) == 1
assert results_ids.count(str(child.id)) == 1
def test_api_documents_all_comparison_with_list():
"""
The 'all' endpoint should return more documents than 'list' when there are children.
'list' returns only top-level documents, 'all' returns all descendants.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Create a document tree
parent = factories.DocumentFactory()
factories.UserDocumentAccessFactory(document=parent, user=user)
child = factories.DocumentFactory(parent=parent)
grandchild = factories.DocumentFactory(parent=child)
# Call list endpoint
list_response = client.get("/api/v1.0/documents/")
list_results = list_response.json()["results"]
# Call all endpoint
all_response = client.get("/api/v1.0/documents/all/")
all_results = all_response.json()["results"]
# list should return only parent
assert len(list_results) == 1
assert list_results[0]["id"] == str(parent.id)
# all should return parent + child + grandchild
assert len(all_results) == 3
all_ids = {result["id"] for result in all_results}
assert all_ids == {str(parent.id), str(child.id), str(grandchild.id)}
@@ -16,6 +16,7 @@ from rest_framework.test import APIClient
from core import factories
from core.api.serializers import ServerCreateDocumentSerializer
from core.models import Document, Invitation, User
from core.services import mime_types
from core.services.converter_services import ConversionError, YdocConverter
pytestmark = pytest.mark.django_db
@@ -191,7 +192,9 @@ def test_api_documents_create_for_owner_existing(mock_convert_md):
assert response.status_code == 201
mock_convert_md.assert_called_once_with("Document content")
mock_convert_md.assert_called_once_with(
"Document content", mime_types.MARKDOWN, mime_types.YJS
)
document = Document.objects.get()
assert response.json() == {"id": str(document.id)}
@@ -236,7 +239,9 @@ def test_api_documents_create_for_owner_new_user(mock_convert_md):
assert response.status_code == 201
mock_convert_md.assert_called_once_with("Document content")
mock_convert_md.assert_called_once_with(
"Document content", mime_types.MARKDOWN, mime_types.YJS
)
document = Document.objects.get()
assert response.json() == {"id": str(document.id)}
@@ -297,7 +302,9 @@ def test_api_documents_create_for_owner_existing_user_email_no_sub_with_fallback
assert response.status_code == 201
mock_convert_md.assert_called_once_with("Document content")
mock_convert_md.assert_called_once_with(
"Document content", mime_types.MARKDOWN, mime_types.YJS
)
document = Document.objects.get()
assert response.json() == {"id": str(document.id)}
@@ -393,7 +400,9 @@ def test_api_documents_create_for_owner_new_user_no_sub_no_fallback_allow_duplic
HTTP_AUTHORIZATION="Bearer DummyToken",
)
assert response.status_code == 201
mock_convert_md.assert_called_once_with("Document content")
mock_convert_md.assert_called_once_with(
"Document content", mime_types.MARKDOWN, mime_types.YJS
)
document = Document.objects.get()
assert response.json() == {"id": str(document.id)}
@@ -474,7 +483,9 @@ def test_api_documents_create_for_owner_with_default_language(
)
assert response.status_code == 201
mock_convert_md.assert_called_once_with("Document content")
mock_convert_md.assert_called_once_with(
"Document content", mime_types.MARKDOWN, mime_types.YJS
)
assert mock_send.call_args[0][3] == "de-de"
@@ -501,7 +512,9 @@ def test_api_documents_create_for_owner_with_custom_language(mock_convert_md):
assert response.status_code == 201
mock_convert_md.assert_called_once_with("Document content")
mock_convert_md.assert_called_once_with(
"Document content", mime_types.MARKDOWN, mime_types.YJS
)
assert len(mail.outbox) == 1
email = mail.outbox[0]
@@ -537,7 +550,9 @@ def test_api_documents_create_for_owner_with_custom_subject_and_message(
assert response.status_code == 201
mock_convert_md.assert_called_once_with("Document content")
mock_convert_md.assert_called_once_with(
"Document content", mime_types.MARKDOWN, mime_types.YJS
)
assert len(mail.outbox) == 1
email = mail.outbox[0]
@@ -571,7 +586,9 @@ def test_api_documents_create_for_owner_with_converter_exception(
format="json",
HTTP_AUTHORIZATION="Bearer DummyToken",
)
mock_convert_md.assert_called_once_with("Document content")
mock_convert_md.assert_called_once_with(
"Document content", mime_types.MARKDOWN, mime_types.YJS
)
assert response.status_code == 400
assert response.json() == {"content": ["Could not convert content"]}
@@ -0,0 +1,358 @@
"""
Tests for Documents API endpoint in impress's core app: create with file upload
"""
from base64 import b64decode, binascii
from io import BytesIO
from unittest.mock import patch
import pytest
from rest_framework.test import APIClient
from core import factories
from core.models import Document
from core.services import mime_types
from core.services.converter_services import (
ConversionError,
ServiceUnavailableError,
)
pytestmark = pytest.mark.django_db
def test_api_documents_create_with_file_anonymous():
"""Anonymous users should not be allowed to create documents with file upload."""
# Create a fake DOCX file
file_content = b"fake docx content"
file = BytesIO(file_content)
file.name = "test_document.docx"
response = APIClient().post(
"/api/v1.0/documents/",
{
"file": file,
},
format="multipart",
)
assert response.status_code == 401
assert not Document.objects.exists()
@patch("core.services.converter_services.Converter.convert")
def test_api_documents_create_with_docx_file_success(mock_convert):
"""
Authenticated users should be able to create documents by uploading a DOCX file.
The file should be converted to YJS format and the title should be set from filename.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Mock the conversion
converted_yjs = "base64encodedyjscontent"
mock_convert.return_value = converted_yjs
# Create a fake DOCX file
file_content = b"fake docx content"
file = BytesIO(file_content)
file.name = "My Important Document.docx"
response = client.post(
"/api/v1.0/documents/",
{
"file": file,
},
format="multipart",
)
assert response.status_code == 201
document = Document.objects.get()
assert document.title == "My Important Document.docx"
assert document.content == converted_yjs
assert document.accesses.filter(role="owner", user=user).exists()
# Verify the converter was called correctly
mock_convert.assert_called_once_with(
file_content,
content_type=mime_types.DOCX,
accept=mime_types.YJS,
)
@patch("core.services.converter_services.Converter.convert")
def test_api_documents_create_with_markdown_file_success(mock_convert):
"""
Authenticated users should be able to create documents by uploading a Markdown file.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Mock the conversion
converted_yjs = "base64encodedyjscontent"
mock_convert.return_value = converted_yjs
# Create a fake Markdown file
file_content = b"# Test Document\n\nThis is a test."
file = BytesIO(file_content)
file.name = "readme.md"
response = client.post(
"/api/v1.0/documents/",
{
"file": file,
},
format="multipart",
)
assert response.status_code == 201
document = Document.objects.get()
assert document.title == "readme.md"
assert document.content == converted_yjs
assert document.accesses.filter(role="owner", user=user).exists()
# Verify the converter was called correctly
mock_convert.assert_called_once_with(
file_content,
content_type=mime_types.MARKDOWN,
accept=mime_types.YJS,
)
@patch("core.services.converter_services.Converter.convert")
def test_api_documents_create_with_file_and_explicit_title(mock_convert):
"""
When both file and title are provided, the filename should override the title.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Mock the conversion
converted_yjs = "base64encodedyjscontent"
mock_convert.return_value = converted_yjs
# Create a fake DOCX file
file_content = b"fake docx content"
file = BytesIO(file_content)
file.name = "Uploaded Document.docx"
response = client.post(
"/api/v1.0/documents/",
{
"file": file,
"title": "This should be overridden",
},
format="multipart",
)
assert response.status_code == 201
document = Document.objects.get()
# The filename should take precedence
assert document.title == "Uploaded Document.docx"
def test_api_documents_create_with_empty_file():
"""
Creating a document with an empty file should fail with a validation error.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Create an empty file
file = BytesIO(b"")
file.name = "empty.docx"
response = client.post(
"/api/v1.0/documents/",
{
"file": file,
},
format="multipart",
)
assert response.status_code == 400
assert response.json() == {"file": ["The submitted file is empty."]}
assert not Document.objects.exists()
@patch("core.services.converter_services.Converter.convert")
def test_api_documents_create_with_file_conversion_error(mock_convert):
"""
When conversion fails, the API should return a 400 error with appropriate message.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Mock the conversion to raise an error
mock_convert.side_effect = ConversionError("Failed to convert document")
# Create a fake DOCX file
file_content = b"fake invalid docx content"
file = BytesIO(file_content)
file.name = "corrupted.docx"
response = client.post(
"/api/v1.0/documents/",
{
"file": file,
},
format="multipart",
)
assert response.status_code == 400
assert response.json() == {"file": ["Could not convert file content"]}
assert not Document.objects.exists()
@patch("core.services.converter_services.Converter.convert")
def test_api_documents_create_with_file_service_unavailable(mock_convert):
"""
When the conversion service is unavailable, appropriate error should be returned.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Mock the conversion to raise ServiceUnavailableError
mock_convert.side_effect = ServiceUnavailableError(
"Failed to connect to conversion service"
)
# Create a fake DOCX file
file_content = b"fake docx content"
file = BytesIO(file_content)
file.name = "document.docx"
response = client.post(
"/api/v1.0/documents/",
{
"file": file,
},
format="multipart",
)
assert response.status_code == 400
assert response.json() == {"file": ["Could not convert file content"]}
assert not Document.objects.exists()
def test_api_documents_create_without_file_still_works():
"""
Creating a document without a file should still work as before (backward compatibility).
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
response = client.post(
"/api/v1.0/documents/",
{
"title": "Regular document without file",
},
format="json",
)
assert response.status_code == 201
document = Document.objects.get()
assert document.title == "Regular document without file"
assert document.content is None
assert document.accesses.filter(role="owner", user=user).exists()
@patch("core.services.converter_services.Converter.convert")
def test_api_documents_create_with_file_null_value(mock_convert):
"""
Passing file=null should be treated as no file upload.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
response = client.post(
"/api/v1.0/documents/",
{
"title": "Document with null file",
"file": None,
},
format="json",
)
assert response.status_code == 201
document = Document.objects.get()
assert document.title == "Document with null file"
# Converter should not have been called
mock_convert.assert_not_called()
@patch("core.services.converter_services.Converter.convert")
def test_api_documents_create_with_file_preserves_content_format(mock_convert):
"""
Verify that the converted content is stored correctly in the document.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Mock the conversion with realistic base64-encoded YJS data
converted_yjs = "AQMEBQYHCAkKCwwNDg8QERITFBUWFxgZGhscHR4fICA="
mock_convert.return_value = converted_yjs
# Create a fake DOCX file
file_content = b"fake docx with complex formatting"
file = BytesIO(file_content)
file.name = "complex_document.docx"
response = client.post(
"/api/v1.0/documents/",
{
"file": file,
},
format="multipart",
)
assert response.status_code == 201
document = Document.objects.get()
# Verify the content is stored as returned by the converter
assert document.content == converted_yjs
# Verify it's valid base64 (can be decoded)
try:
b64decode(converted_yjs)
except binascii.Error:
pytest.fail("Content should be valid base64-encoded data")
@patch("core.services.converter_services.Converter.convert")
def test_api_documents_create_with_file_unicode_filename(mock_convert):
"""
Test that Unicode characters in filenames are handled correctly.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
# Mock the conversion
converted_yjs = "base64encodedyjscontent"
mock_convert.return_value = converted_yjs
# Create a file with Unicode characters in the name
file_content = b"fake docx content"
file = BytesIO(file_content)
file.name = "文档-télécharger-документ.docx"
response = client.post(
"/api/v1.0/documents/",
{
"file": file,
},
format="multipart",
)
assert response.status_code == 201
document = Document.objects.get()
assert document.title == "文档-télécharger-документ.docx"
@@ -0,0 +1,93 @@
"""Test Converter orchestration services."""
from unittest.mock import MagicMock, patch
from core.services import mime_types
from core.services.converter_services import Converter
@patch("core.services.converter_services.DocSpecConverter")
@patch("core.services.converter_services.YdocConverter")
def test_converter_docx_to_yjs_orchestration(mock_ydoc_class, mock_docspec_class):
"""Test that DOCX to YJS conversion uses both DocSpec and Ydoc converters."""
# Setup mocks
mock_docspec = MagicMock()
mock_ydoc = MagicMock()
mock_docspec_class.return_value = mock_docspec
mock_ydoc_class.return_value = mock_ydoc
# Mock the conversion chain: DOCX -> BlockNote -> YJS
blocknote_data = b'[{"type": "paragraph", "content": "test"}]'
yjs_data = "base64encodedyjs"
mock_docspec.convert.return_value = blocknote_data
mock_ydoc.convert.return_value = yjs_data
# Execute conversion
converter = Converter()
docx_data = b"fake docx data"
result = converter.convert(docx_data, mime_types.DOCX, mime_types.YJS)
# Verify the orchestration
mock_docspec.convert.assert_called_once_with(
docx_data, mime_types.DOCX, mime_types.BLOCKNOTE
)
mock_ydoc.convert.assert_called_once_with(
blocknote_data, mime_types.BLOCKNOTE, mime_types.YJS
)
assert result == yjs_data
@patch("core.services.converter_services.YdocConverter")
def test_converter_markdown_to_yjs_delegation(mock_ydoc_class):
"""Test that Markdown to YJS conversion is delegated to YdocConverter."""
mock_ydoc = MagicMock()
mock_ydoc_class.return_value = mock_ydoc
yjs_data = "base64encodedyjs"
mock_ydoc.convert.return_value = yjs_data
converter = Converter()
markdown_data = "# Test Document"
result = converter.convert(markdown_data, mime_types.MARKDOWN, mime_types.YJS)
mock_ydoc.convert.assert_called_once_with(
markdown_data, mime_types.MARKDOWN, mime_types.YJS
)
assert result == yjs_data
@patch("core.services.converter_services.YdocConverter")
def test_converter_yjs_to_html_delegation(mock_ydoc_class):
"""Test that YJS to HTML conversion is delegated to YdocConverter."""
mock_ydoc = MagicMock()
mock_ydoc_class.return_value = mock_ydoc
html_data = "<p>Test Document</p>"
mock_ydoc.convert.return_value = html_data
converter = Converter()
yjs_data = b"yjs binary data"
result = converter.convert(yjs_data, mime_types.YJS, mime_types.HTML)
mock_ydoc.convert.assert_called_once_with(yjs_data, mime_types.YJS, mime_types.HTML)
assert result == html_data
@patch("core.services.converter_services.YdocConverter")
def test_converter_blocknote_to_yjs_delegation(mock_ydoc_class):
"""Test that BlockNote to YJS conversion is delegated to YdocConverter."""
mock_ydoc = MagicMock()
mock_ydoc_class.return_value = mock_ydoc
yjs_data = "base64encodedyjs"
mock_ydoc.convert.return_value = yjs_data
converter = Converter()
blocknote_data = b'[{"type": "paragraph"}]'
result = converter.convert(blocknote_data, mime_types.BLOCKNOTE, mime_types.YJS)
mock_ydoc.convert.assert_called_once_with(
blocknote_data, mime_types.BLOCKNOTE, mime_types.YJS
)
assert result == yjs_data
@@ -6,6 +6,7 @@ from unittest.mock import MagicMock, patch
import pytest
import requests
from core.services import mime_types
from core.services.converter_services import (
ServiceUnavailableError,
ValidationError,
@@ -21,9 +22,9 @@ def test_auth_header(settings):
def test_convert_empty_text():
"""Should raise ValidationError when text is empty."""
"""Should raise ValidationError when data is empty."""
converter = YdocConverter()
with pytest.raises(ValidationError, match="Input text cannot be empty"):
with pytest.raises(ValidationError, match="Input data cannot be empty"):
converter.convert("")
@@ -36,7 +37,7 @@ def test_convert_service_unavailable(mock_post):
with pytest.raises(
ServiceUnavailableError,
match="Failed to connect to conversion service",
match="Failed to connect to YDoc conversion service",
):
converter.convert("test text")
@@ -52,7 +53,7 @@ def test_convert_http_error(mock_post):
with pytest.raises(
ServiceUnavailableError,
match="Failed to connect to conversion service",
match="Failed to connect to YDoc conversion service",
):
converter.convert("test text")
@@ -83,8 +84,8 @@ def test_convert_full_integration(mock_post, settings):
data="test markdown",
headers={
"Authorization": "Bearer test-key",
"Content-Type": "text/markdown",
"Accept": "application/vnd.yjs.doc",
"Content-Type": mime_types.MARKDOWN,
"Accept": mime_types.YJS,
},
timeout=5,
verify=False,
@@ -108,9 +109,7 @@ def test_convert_full_integration_with_specific_headers(mock_post, settings):
mock_response.raise_for_status.return_value = None
mock_post.return_value = mock_response
result = converter.convert(
b"test_content", "application/vnd.yjs.doc", "text/markdown"
)
result = converter.convert(b"test_content", mime_types.YJS, mime_types.MARKDOWN)
assert result == expected_response
mock_post.assert_called_once_with(
@@ -118,8 +117,8 @@ def test_convert_full_integration_with_specific_headers(mock_post, settings):
data=b"test_content",
headers={
"Authorization": "Bearer test-key",
"Content-Type": "application/vnd.yjs.doc",
"Accept": "text/markdown",
"Content-Type": mime_types.YJS,
"Accept": mime_types.MARKDOWN,
},
timeout=5,
verify=False,
@@ -135,7 +134,7 @@ def test_convert_timeout(mock_post):
with pytest.raises(
ServiceUnavailableError,
match="Failed to connect to conversion service",
match="Failed to connect to YDoc conversion service",
):
converter.convert("test text")
@@ -144,5 +143,5 @@ def test_convert_none_input():
"""Should raise ValidationError when input is None."""
converter = YdocConverter()
with pytest.raises(ValidationError, match="Input text cannot be empty"):
with pytest.raises(ValidationError, match="Input data cannot be empty"):
converter.convert(None)
@@ -0,0 +1,117 @@
"""Test DocSpec converter services."""
from unittest.mock import MagicMock, patch
import pytest
import requests
from core.services import mime_types
from core.services.converter_services import (
DocSpecConverter,
ServiceUnavailableError,
ValidationError,
)
def test_docspec_convert_empty_data():
"""Should raise ValidationError when data is empty."""
converter = DocSpecConverter()
with pytest.raises(ValidationError, match="Input data cannot be empty"):
converter.convert("", mime_types.DOCX, mime_types.BLOCKNOTE)
def test_docspec_convert_none_input():
"""Should raise ValidationError when input is None."""
converter = DocSpecConverter()
with pytest.raises(ValidationError, match="Input data cannot be empty"):
converter.convert(None, mime_types.DOCX, mime_types.BLOCKNOTE)
def test_docspec_convert_unsupported_content_type():
"""Should raise ValidationError when content type is not DOCX."""
converter = DocSpecConverter()
with pytest.raises(
ValidationError, match="Conversion from text/plain to .* is not supported"
):
converter.convert(b"test data", "text/plain", mime_types.BLOCKNOTE)
def test_docspec_convert_unsupported_accept():
"""Should raise ValidationError when accept type is not BLOCKNOTE."""
converter = DocSpecConverter()
with pytest.raises(
ValidationError,
match=f"Conversion from {mime_types.DOCX} to {mime_types.YJS} is not supported",
):
converter.convert(b"test data", mime_types.DOCX, mime_types.YJS)
@patch("requests.post")
def test_docspec_convert_service_unavailable(mock_post):
"""Should raise ServiceUnavailableError when service is unavailable."""
converter = DocSpecConverter()
mock_post.side_effect = requests.RequestException("Connection error")
with pytest.raises(
ServiceUnavailableError,
match="Failed to connect to DocSpec conversion service",
):
converter.convert(b"test data", mime_types.DOCX, mime_types.BLOCKNOTE)
@patch("requests.post")
def test_docspec_convert_http_error(mock_post):
"""Should raise ServiceUnavailableError when HTTP error occurs."""
converter = DocSpecConverter()
mock_response = MagicMock()
mock_response.raise_for_status.side_effect = requests.HTTPError("HTTP Error")
mock_post.return_value = mock_response
with pytest.raises(
ServiceUnavailableError,
match="Failed to connect to DocSpec conversion service",
):
converter.convert(b"test data", mime_types.DOCX, mime_types.BLOCKNOTE)
@patch("requests.post")
def test_docspec_convert_timeout(mock_post):
"""Should raise ServiceUnavailableError when request times out."""
converter = DocSpecConverter()
mock_post.side_effect = requests.Timeout("Request timed out")
with pytest.raises(
ServiceUnavailableError,
match="Failed to connect to DocSpec conversion service",
):
converter.convert(b"test data", mime_types.DOCX, mime_types.BLOCKNOTE)
@patch("requests.post")
def test_docspec_convert_success(mock_post, settings):
"""Test successful DOCX to BlockNote conversion."""
settings.DOCSPEC_API_URL = "http://docspec.test/convert"
settings.CONVERSION_API_TIMEOUT = 5
settings.CONVERSION_API_SECURE = False
converter = DocSpecConverter()
expected_content = b'[{"type": "paragraph", "content": "test"}]'
mock_response = MagicMock()
mock_response.content = expected_content
mock_response.raise_for_status.return_value = None
mock_post.return_value = mock_response
docx_data = b"fake docx binary data"
result = converter.convert(docx_data, mime_types.DOCX, mime_types.BLOCKNOTE)
assert result == expected_content
# Verify the request was made correctly
mock_post.assert_called_once_with(
"http://docspec.test/convert",
headers={"Accept": mime_types.BLOCKNOTE},
files={"file": ("document.docx", docx_data, mime_types.DOCX)},
timeout=5,
verify=False,
)
+6
View File
@@ -709,6 +709,9 @@ class Base(Configuration):
environ_prefix=None,
)
# DocSpec API microservice
DOCSPEC_API_URL = values.Value(environ_name="DOCSPEC_API_URL", environ_prefix=None)
# Conversion endpoint
CONVERSION_API_ENDPOINT = values.Value(
default="convert",
@@ -1054,6 +1057,9 @@ class Production(Base):
# Privacy
SECURE_REFERRER_POLICY = "same-origin"
# Conversion API: Always verify SSL in production
CONVERSION_API_SECURE = True
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
File diff suppressed because one or more lines are too long
@@ -0,0 +1,60 @@
![473389927-e4ff1794-69f3-460a-85f8-fec993cd74d6.png](http://localhost:3000/assets/logo-suite-numerique.png)![497094770-53e5f8e2-c93e-4a0b-a82f-cd184fd03f51.svg](http://localhost:3000/assets/icon-docs.svg)
# Lorem Ipsum import Document
## Introduction
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam auctor, nisl eget ultricies tincidunt, nisl nisl aliquam nisl, eget ultricies nisl nisl eget nisl.
### Subsection 1.1
* **Bold text**: Lorem ipsum dolor sit amet.
* *Italic text*: Consectetur adipiscing elit.
* ~~Strikethrough text~~: Nullam auctor, nisl eget ultricies tincidunt.
1. First item in an ordered list.
2. Second item in an ordered list.
* Indented bullet point.
* Another indented bullet point.
3. Third item in an ordered list.
### Subsection 1.2
**Code block:**
```js
const hello_world = () => {
console.log("Hello, world!");
}
```
**Blockquote:**
> Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam auctor, nisl eget ultricies tincidunt.
**Horizontal rule:**
***
**Table:**
| Syntax | Description |
| --------- | ----------- |
| Header | Title |
| Paragraph | Text |
**Inline code:**
Use the `printf()` function.
**Link:** [Example](http://localhost:3000/)
## Conclusion
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam auctor, nisl eget ultricies tincidunt, nisl nisl aliquam nisl, eget ultricies nisl nisl eget nisl.
@@ -604,7 +604,7 @@ test.describe('Doc Editor', () => {
await verifyDocName(page, randomDoc);
const editor = await openSuggestionMenu({ page });
const { editor } = await openSuggestionMenu({ page });
await page.getByText('Embedded file').click();
await page.getByText('Upload file').click();
@@ -1,18 +1,19 @@
import fs from 'fs';
import path from 'path';
import { expect, test } from '@playwright/test';
import { Download, Page, expect, test } from '@playwright/test';
import cs from 'convert-stream';
import JSZip from 'jszip';
import { PDFParse } from 'pdf-parse';
import {
BrowserName,
TestLanguage,
createDoc,
verifyDocName,
waitForLanguageSwitch,
} from './utils-common';
import { openSuggestionMenu, writeInEditor } from './utils-editor';
import { createRootSubPage } from './utils-sub-pages';
test.beforeEach(async ({ page }) => {
await page.goto('/');
@@ -46,81 +47,14 @@ test.describe('Doc Export', () => {
await expect(page.getByTestId('doc-export-download-button')).toBeVisible();
});
test('it exports the doc with pdf line break', async ({
page,
browserName,
}) => {
const [randomDoc] = await createDoc(
page,
'doc-editor-line-break',
browserName,
1,
);
await verifyDocName(page, randomDoc);
const editor = await writeInEditor({ page, text: 'Hello' });
await page.keyboard.press('Enter');
await openSuggestionMenu({ page });
await page.getByText('Page Break').click();
await expect(
editor.locator('div[data-content-type="pageBreak"]'),
).toBeVisible();
await writeInEditor({ page, text: 'World' });
await page
.getByRole('button', {
name: 'Export the document',
})
.click();
const downloadPromise = page.waitForEvent('download', (download) => {
return download.suggestedFilename().includes(`${randomDoc}.pdf`);
});
void page.getByTestId('doc-export-download-button').click();
const download = await downloadPromise;
expect(download.suggestedFilename()).toBe(`${randomDoc}.pdf`);
const pdfBuffer = await cs.toBuffer(await download.createReadStream());
const pdfParse = new PDFParse({ data: pdfBuffer });
const pdfInfo = await pdfParse.getInfo();
const pdfText = await pdfParse.getText();
expect(pdfInfo.total).toBe(2);
expect(pdfText.pages).toStrictEqual([
{ text: 'Hello', num: 1 },
{ text: 'World', num: 2 },
]);
expect(pdfInfo?.info.Title).toBe(randomDoc);
});
/**
* We override the document content to ensure that the exported DOCX
* contains various elements for testing.
* We don't check the content of the DOCX here, just that the export works
* and the file is correctly named.
*/
test('it exports the doc to docx', async ({ page, browserName }) => {
const [randomDoc] = await createDoc(page, 'doc-editor', browserName, 1);
await verifyDocName(page, randomDoc);
await page.locator('.ProseMirror.bn-editor').click();
await page.locator('.ProseMirror.bn-editor').fill('Hello World');
await page.keyboard.press('Enter');
await page.locator('.bn-block-outer').last().fill('/');
await page.getByText('Resizable image with caption').click();
const fileChooserPromise = page.waitForEvent('filechooser');
await page.getByText('Upload image').click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(path.join(__dirname, 'assets/test.svg'));
const image = page
.locator('.--docs--editor-container img.bn-visual-media')
.first();
await expect(image).toBeVisible();
const randomDoc = await overrideDocContent({ page, browserName });
await page
.getByRole('button', {
@@ -143,29 +77,14 @@ test.describe('Doc Export', () => {
expect(download.suggestedFilename()).toBe(`${randomDoc}.docx`);
});
/**
* We override the document content to ensure that the exported ODT
* contains various elements for testing.
* We don't check the content of the ODT here, just that the export works
* and the file is correctly named.
*/
test('it exports the doc to odt', async ({ page, browserName }) => {
const [randomDoc] = await createDoc(page, 'doc-editor-odt', browserName, 1);
await verifyDocName(page, randomDoc);
await page.locator('.ProseMirror.bn-editor').click();
await page.locator('.ProseMirror.bn-editor').fill('Hello World ODT');
await page.keyboard.press('Enter');
await page.locator('.bn-block-outer').last().fill('/');
await page.getByText('Resizable image with caption').click();
const fileChooserPromise = page.waitForEvent('filechooser');
await page.getByText('Upload image').click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(path.join(__dirname, 'assets/test.svg'));
const image = page
.locator('.--docs--editor-container img.bn-visual-media')
.first();
await expect(image).toBeVisible();
const randomDoc = await overrideDocContent({ page, browserName });
await page
.getByRole('button', {
@@ -358,108 +277,6 @@ test.describe('Doc Export', () => {
expect(pdfText.text).toContain('Hello World');
});
test('it exports the doc with quotes', async ({ page, browserName }) => {
const [randomDoc] = await createDoc(page, 'export-quotes', browserName, 1);
const editor = page.locator('.ProseMirror.bn-editor');
// Trigger slash menu to show menu
await editor.click();
await editor.fill('/');
await page.getByText('Quote or excerpt').click();
await expect(
editor.locator('.bn-block-content[data-content-type="quote"]'),
).toBeVisible();
await editor
.locator('.bn-block-content[data-content-type="quote"]')
.fill('Hello World');
await expect(editor.getByText('Hello World')).toHaveCSS(
'font-style',
'italic',
);
await page
.getByRole('button', {
name: 'Export the document',
})
.click();
await expect(page.getByTestId('doc-export-download-button')).toBeVisible();
const downloadPromise = page.waitForEvent('download', (download) => {
return download.suggestedFilename().includes(`${randomDoc}.pdf`);
});
void page.getByTestId('doc-export-download-button').click();
const download = await downloadPromise;
expect(download.suggestedFilename()).toBe(`${randomDoc}.pdf`);
const pdfBuffer = await cs.toBuffer(await download.createReadStream());
const pdfParse = new PDFParse({ data: pdfBuffer });
const pdfText = await pdfParse.getText();
expect(pdfText.text).toContain('Hello World');
});
test('it exports the doc with multi columns', async ({
page,
browserName,
}) => {
const [randomDoc] = await createDoc(
page,
'doc-multi-columns',
browserName,
1,
);
await page.locator('.bn-block-outer').last().fill('/');
await page.getByText('Three Columns', { exact: true }).click();
await page.locator('.bn-block-column').first().fill('Column 1');
await page.locator('.bn-block-column').nth(1).fill('Column 2');
await page.locator('.bn-block-column').last().fill('Column 3');
expect(await page.locator('.bn-block-column').count()).toBe(3);
await expect(
page.locator('.bn-block-column[data-node-type="column"]').first(),
).toHaveText('Column 1');
await expect(
page.locator('.bn-block-column[data-node-type="column"]').nth(1),
).toHaveText('Column 2');
await expect(
page.locator('.bn-block-column[data-node-type="column"]').last(),
).toHaveText('Column 3');
await page
.getByRole('button', {
name: 'Export the document',
})
.click();
await expect(
page.getByTestId('doc-open-modal-download-button'),
).toBeVisible();
const downloadPromise = page.waitForEvent('download', (download) => {
return download.suggestedFilename().includes(`${randomDoc}.pdf`);
});
void page.getByTestId('doc-export-download-button').click();
const download = await downloadPromise;
expect(download.suggestedFilename()).toBe(`${randomDoc}.pdf`);
const pdfBuffer = await cs.toBuffer(await download.createReadStream());
const pdfParse = new PDFParse({ data: pdfBuffer });
const pdfText = await pdfParse.getText();
expect(pdfText.text).toContain('Column 1');
expect(pdfText.text).toContain('Column 2');
expect(pdfText.text).toContain('Column 3');
});
test('it injects the correct language attribute into PDF export', async ({
page,
browserName,
@@ -506,53 +323,11 @@ test.describe('Doc Export', () => {
expect(pdfString).toContain('/Lang (fr)');
});
test('it exports the doc with interlinking', async ({
test('it exports the doc to PDF and checks regressions', async ({
page,
browserName,
}) => {
const [randomDoc] = await createDoc(
page,
'export-interlinking',
browserName,
1,
);
await verifyDocName(page, randomDoc);
const { name: docChild } = await createRootSubPage(
page,
browserName,
'export-interlink-child',
);
await verifyDocName(page, docChild);
const editor = await openSuggestionMenu({ page });
await page.getByText('Link a doc').first().click();
const input = page.locator(
"span[data-inline-content-type='interlinkingSearchInline'] input",
);
const searchContainer = page.locator('.quick-search-container');
await input.fill('export-interlink');
await expect(searchContainer).toBeVisible();
await expect(searchContainer.getByText(randomDoc)).toBeVisible();
// We are in docChild, we want to create a link to randomDoc (parent)
await searchContainer.getByText(randomDoc).click();
// Search the interlinking link in the editor (not in the document tree)
const interlink = editor
.locator('.--docs--interlinking-link-inline-content')
.first();
await expect(interlink).toContainText(randomDoc);
const downloadPromise = page.waitForEvent('download', (download) => {
return download.suggestedFilename().includes(`${docChild}.pdf`);
});
const randomDoc = await overrideDocContent({ page, browserName });
await page
.getByRole('button', {
@@ -560,77 +335,155 @@ test.describe('Doc Export', () => {
})
.click();
void page.getByTestId('doc-export-download-button').click();
const download = await downloadPromise;
expect(download.suggestedFilename()).toBe(`${docChild}.pdf`);
const pdfBuffer = await cs.toBuffer(await download.createReadStream());
const pdfParse = new PDFParse({ data: pdfBuffer });
const pdfText = await pdfParse.getText();
expect(pdfText.text).toContain(randomDoc);
});
test('it exports the doc with interlinking to odt', async ({
page,
browserName,
}) => {
const [randomDoc] = await createDoc(
page,
'export-interlinking-odt',
browserName,
1,
);
await verifyDocName(page, randomDoc);
const { name: docChild } = await createRootSubPage(
page,
browserName,
'export-interlink-child-odt',
);
await verifyDocName(page, docChild);
const editor = await openSuggestionMenu({ page });
await page.getByText('Link a doc').first().click();
const input = page.locator(
"span[data-inline-content-type='interlinkingSearchInline'] input",
);
const searchContainer = page.locator('.quick-search-container');
await input.fill('export-interlink');
await expect(searchContainer).toBeVisible();
await expect(searchContainer.getByText(randomDoc)).toBeVisible();
// We are in docChild, we want to create a link to randomDoc (parent)
await searchContainer.getByText(randomDoc).click();
// Search the interlinking link in the editor (not in the document tree)
const interlink = editor
.locator('.--docs--interlinking-link-inline-content')
.first();
await expect(interlink).toContainText(randomDoc);
await page
.getByRole('button', {
name: 'Export the document',
})
.click();
await page.getByRole('combobox', { name: 'Format' }).click();
await page.getByRole('option', { name: 'Odt' }).click();
await expect(
page.getByTestId('doc-open-modal-download-button'),
).toBeVisible();
const downloadPromise = page.waitForEvent('download', (download) => {
return download.suggestedFilename().includes(`${docChild}.odt`);
return download.suggestedFilename().includes(`${randomDoc}.pdf`);
});
void page.getByTestId('doc-export-download-button').click();
await page.getByTestId('doc-export-download-button').click();
const download = await downloadPromise;
expect(download.suggestedFilename()).toBe(`${docChild}.odt`);
expect(download.suggestedFilename()).toBe(`${randomDoc}.pdf`);
// If we need to update the PDF regression fixture, uncomment the line below
//await savePDFToAssetFolder(download);
// Assert the generated PDF matches "assets/doc-export-regressions.pdf"
await comparePDFWithAssetFolder(download);
});
});
export const savePDFToAssetFolder = async (download: Download) => {
const pdfBuffer = await cs.toBuffer(await download.createReadStream());
const pdfPath = path.join(__dirname, 'assets', `doc-export-regressions.pdf`);
fs.writeFileSync(pdfPath, pdfBuffer);
};
export const comparePDFWithAssetFolder = async (download: Download) => {
const pdfBuffer = await cs.toBuffer(await download.createReadStream());
// Load reference PDF for comparison
const referencePdfPath = path.join(
__dirname,
'assets',
'doc-export-regressions.pdf',
);
const referencePdfBuffer = fs.readFileSync(referencePdfPath);
// Parse both PDFs
const generatedPdf = new PDFParse({ data: pdfBuffer });
const referencePdf = new PDFParse({ data: referencePdfBuffer });
const [generatedInfo, referenceInfo] = await Promise.all([
generatedPdf.getInfo(),
referencePdf.getInfo(),
]);
const [generatedScreenshot, referenceScreenshot] = await Promise.all([
generatedPdf.getScreenshot(),
referencePdf.getScreenshot(),
]);
generatedScreenshot.pages[0].data;
const [generatedText, referenceText] = await Promise.all([
generatedPdf.getText(),
referencePdf.getText(),
]);
// Compare page count
expect(generatedInfo.total).toBe(referenceInfo.total);
// Compare text content
expect(generatedText.text).toBe(referenceText.text);
// Compare screenshots page by page
for (let i = 0; i < generatedScreenshot.pages.length; i++) {
const genPage = generatedScreenshot.pages[i];
const refPage = referenceScreenshot.pages[i];
expect(genPage.width).toBe(refPage.width);
expect(genPage.height).toBe(refPage.height);
expect(genPage.data).toStrictEqual(refPage.data);
}
};
/**
* Override the document content API response to use a test content
* This test content contains many blocks to facilitate testing
* @param page
*/
export const overrideDocContent = async ({
page,
browserName,
}: {
page: Page;
browserName: BrowserName;
}) => {
// Override content prop with assets/base-content-test-pdf.txt
await page.route(/\**\/documents\/\**/, async (route) => {
const request = route.request();
if (
request.method().includes('GET') &&
!request.url().includes('page=') &&
!request.url().includes('versions') &&
!request.url().includes('accesses') &&
!request.url().includes('invitations')
) {
const response = await route.fetch();
const json = await response.json();
json.content = fs.readFileSync(
path.join(__dirname, 'assets/base-content-test-pdf.txt'),
'utf-8',
);
void route.fulfill({
response,
body: JSON.stringify(json),
});
} else {
await route.continue();
}
});
const [randomDoc] = await createDoc(
page,
'doc-export-override-content',
browserName,
1,
);
await verifyDocName(page, randomDoc);
// Add Image SVG
await page.keyboard.press('Enter');
const { suggestionMenu } = await openSuggestionMenu({ page });
await suggestionMenu.getByText('Resizable image with caption').click();
const fileChooserPromise = page.waitForEvent('filechooser');
await page.getByText('Upload image').click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(path.join(__dirname, 'assets/test.svg'));
const image = page
.locator('.--docs--editor-container img.bn-visual-media[src$=".svg"]')
.first();
await expect(image).toBeVisible();
await page.keyboard.press('Enter');
// Add Image PNG
await openSuggestionMenu({ page });
await suggestionMenu.getByText('Resizable image with caption').click();
const fileChooserPNGPromise = page.waitForEvent('filechooser');
await page.getByText('Upload image').click();
const fileChooserPNG = await fileChooserPNGPromise;
await fileChooserPNG.setFiles(
path.join(__dirname, 'assets/logo-suite-numerique.png'),
);
const imagePng = page
.locator('.--docs--editor-container img.bn-visual-media[src$=".png"]')
.first();
await expect(imagePng).toBeVisible();
return randomDoc;
};
@@ -0,0 +1,172 @@
import { readFileSync } from 'fs';
import path from 'path';
import { Page, expect, test } from '@playwright/test';
import { getEditor } from './utils-editor';
test.beforeEach(async ({ page }) => {
await page.goto('/');
});
test.describe('Doc Import', () => {
test('it imports 2 docs with the import icon', async ({ page }) => {
const fileChooserPromise = page.waitForEvent('filechooser');
await page.getByLabel('Open the upload dialog').click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(path.join(__dirname, 'assets/test_import.docx'));
await fileChooser.setFiles(path.join(__dirname, 'assets/test_import.md'));
await expect(
page.getByText(
'The document "test_import.docx" has been successfully imported',
),
).toBeVisible();
await expect(
page.getByText(
'The document "test_import.md" has been successfully imported',
),
).toBeVisible();
const docsGrid = page.getByTestId('docs-grid');
await expect(docsGrid.getByText('test_import.docx').first()).toBeVisible();
await expect(docsGrid.getByText('test_import.md').first()).toBeVisible();
// Check content of imported md
await docsGrid.getByText('test_import.md').first().click();
const editor = await getEditor({ page });
const contentCheck = async (isMDCheck = false) => {
await expect(
editor.getByRole('heading', {
name: 'Lorem Ipsum import Document',
level: 1,
}),
).toBeVisible();
await expect(
editor.getByRole('heading', {
name: 'Introduction',
level: 2,
}),
).toBeVisible();
await expect(
editor.getByRole('heading', {
name: 'Subsection 1.1',
level: 3,
}),
).toBeVisible();
await expect(
editor
.locator('div[data-content-type="bulletListItem"] strong')
.getByText('Bold text'),
).toBeVisible();
await expect(
editor
.locator('div[data-content-type="codeBlock"]')
.getByText('hello_world'),
).toBeVisible();
await expect(
editor
.locator('div[data-content-type="table"] td')
.getByText('Paragraph'),
).toBeVisible();
await expect(
editor.locator('a[href="http://localhost:3000/"]').getByText('Example'),
).toBeVisible();
/* eslint-disable playwright/no-conditional-expect */
if (isMDCheck) {
await expect(
editor.locator(
'img[src="http://localhost:3000/assets/logo-suite-numerique.png"]',
),
).toBeVisible();
await expect(
editor.locator(
'img[src="http://localhost:3000/assets/icon-docs.svg"]',
),
).toBeVisible();
} else {
await expect(editor.locator('img')).toHaveCount(2);
}
/* eslint-enable playwright/no-conditional-expect */
await expect(
editor.locator('div[data-content-type="divider"] hr'),
).toBeVisible();
};
await contentCheck();
// Check content of imported docx
await page.getByLabel('Back to homepage').first().click();
await docsGrid.getByText('test_import.docx').first().click();
await contentCheck();
});
test('it imports 2 docs with the drag and drop area', async ({ page }) => {
const docsGrid = page.getByTestId('docs-grid');
await expect(docsGrid).toBeVisible();
await dragAndDropFiles(page, "[data-testid='docs-grid']", [
{
filePath: path.join(__dirname, 'assets/test_import.docx'),
fileName: 'test_import.docx',
fileType:
'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
},
{
filePath: path.join(__dirname, 'assets/test_import.md'),
fileName: 'test_import.md',
fileType: 'text/markdown',
},
]);
// Wait for success messages
await expect(
page.getByText(
'The document "test_import.docx" has been successfully imported',
),
).toBeVisible();
await expect(
page.getByText(
'The document "test_import.md" has been successfully imported',
),
).toBeVisible();
await expect(docsGrid.getByText('test_import.docx').first()).toBeVisible();
await expect(docsGrid.getByText('test_import.md').first()).toBeVisible();
});
});
const dragAndDropFiles = async (
page: Page,
selector: string,
files: Array<{ filePath: string; fileName: string; fileType?: string }>,
) => {
const filesData = files.map((file) => ({
bufferData: `data:application/octet-stream;base64,${readFileSync(file.filePath).toString('base64')}`,
fileName: file.fileName,
fileType: file.fileType || '',
}));
const dataTransfer = await page.evaluateHandle(async (filesInfo) => {
const dt = new DataTransfer();
for (const fileInfo of filesInfo) {
const blobData = await fetch(fileInfo.bufferData).then((res) =>
res.blob(),
);
const file = new File([blobData], fileInfo.fileName, {
type: fileInfo.fileType,
});
dt.items.add(file);
}
return dt;
}, filesData);
await page.dispatchEvent(selector, 'drop', { dataTransfer });
};
@@ -42,8 +42,8 @@ test.describe('Doc Version', () => {
// Write more
await writeInEditor({ page, text: 'It will create a version' });
await openSuggestionMenu({ page });
await page.getByText('Add a callout block').click();
const { suggestionMenu } = await openSuggestionMenu({ page });
await suggestionMenu.getByText('Add a callout block').click();
const calloutBlock = page
.locator('div[data-content-type="callout"]')
@@ -109,8 +109,10 @@ test.describe('Language', () => {
}) => {
await createDoc(page, 'doc-toolbar', browserName, 1);
const editor = await openSuggestionMenu({ page });
await expect(page.getByText('Headings', { exact: true })).toBeVisible();
const { editor, suggestionMenu } = await openSuggestionMenu({ page });
await expect(
suggestionMenu.getByText('Headings', { exact: true }),
).toBeVisible();
await editor.click(); // close the menu
@@ -121,6 +123,8 @@ test.describe('Language', () => {
// Trigger slash menu to show french menu
await openSuggestionMenu({ page });
await expect(page.getByText('Titres', { exact: true })).toBeVisible();
await expect(
suggestionMenu.getByText('Titres', { exact: true }),
).toBeVisible();
});
});
@@ -11,7 +11,9 @@ export const openSuggestionMenu = async ({ page }: { page: Page }) => {
await editor.click();
await writeInEditor({ page, text: '/' });
return editor;
const suggestionMenu = page.locator('.bn-suggestion-menu');
return { editor, suggestionMenu };
};
export const writeInEditor = async ({
+1
View File
@@ -62,6 +62,7 @@
"react": "*",
"react-aria-components": "1.13.0",
"react-dom": "*",
"react-dropzone": "14.3.8",
"react-i18next": "16.5.0",
"react-intersection-observer": "10.0.0",
"react-resizable-panels": "3.0.6",
@@ -20,7 +20,7 @@ export type DefinedInitialDataInfiniteOptionsAPI<
QueryKey,
TPageParam
>;
export type UseInfiniteQueryResultAPI<Q> = InfiniteData<Q>;
export type InfiniteQueryConfig<Q> = Omit<
DefinedInitialDataInfiniteOptionsAPI<Q>,
'queryKey' | 'initialData' | 'getNextPageParam' | 'initialPageParam'
@@ -0,0 +1,20 @@
<svg viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
<path
d="M6.12757 9.8486C5.98657 9.6993 5.91709 9.5143 5.91709 9.30858C5.91709 9.10284 5.98679 8.91775 6.13233 8.77221C6.28262 8.62192 6.47291 8.54842 6.68579 8.54842H13.1697C13.3775 8.54842 13.5623 8.62245 13.7061 8.77215C13.8559 8.91601 13.9299 9.10081 13.9299 9.30858C13.9299 9.51737 13.8553 9.70306 13.7085 9.8511C13.5643 10.0024 13.3787 10.0773 13.1697 10.0773H6.68579C6.47291 10.0773 6.28262 10.0038 6.13233 9.85349L6.13076 9.85192L6.12757 9.8486Z"
fill="currentColor"
/>
<path
d="M6.12757 12.83C5.98657 12.6807 5.91709 12.4957 5.91709 12.29C5.91709 12.0843 5.98679 11.8992 6.13233 11.7536C6.28262 11.6033 6.47291 11.5298 6.68579 11.5298H13.1697C13.3775 11.5298 13.5623 11.6039 13.7061 11.7536C13.8559 11.8974 13.9299 12.0822 13.9299 12.29C13.9299 12.4988 13.8553 12.6845 13.7085 12.8325C13.5643 12.9838 13.3787 13.0587 13.1697 13.0587H6.68579C6.47291 13.0587 6.28262 12.9852 6.13233 12.8349L6.13076 12.8333L6.12757 12.83Z"
fill="currentColor"
/>
<path
d="M5.91709 15.2885C5.91709 15.4912 5.98839 15.6726 6.12757 15.82L6.134 15.8266L6.13723 15.8296C6.28833 15.9723 6.47704 16.0401 6.68579 16.0401H9.75263C9.96123 16.0401 10.1502 15.9722 10.2975 15.8249C10.444 15.6784 10.5213 15.4956 10.5213 15.2885C10.5213 15.0768 10.4486 14.8874 10.2999 14.7374C10.1539 14.5842 9.96433 14.5113 9.75263 14.5113H6.68579C6.47293 14.5113 6.28257 14.5847 6.13226 14.735L6.12757 14.7399C5.98486 14.891 5.91709 15.0797 5.91709 15.2885Z"
fill="currentColor"
/>
<path
fill-rule="evenodd"
clip-rule="evenodd"
d="M7.37975 1.24597C7.88425 0.735004 8.61944 0.5 9.54031 0.5H18.6127C19.533 0.5 20.2661 0.734736 20.7653 1.24652C21.2686 1.75666 21.5 2.49628 21.5 3.42147V16.3808C21.5 17.3112 21.2688 18.0521 20.7638 18.5572C20.2645 19.0624 19.532 19.2937 18.6127 19.2937H17.347V20.5338C17.347 21.4641 17.1158 22.2051 16.6108 22.7102C16.1115 23.2153 15.3789 23.4467 14.4597 23.4467H5.3873C4.46721 23.4467 3.73242 23.2149 3.22781 22.7103C2.72908 22.2051 2.5 21.4635 2.5 20.5338V7.57442C2.5 6.64962 2.72942 5.90915 3.22673 5.39893C3.73123 4.88796 4.46643 4.65295 5.3873 4.65295H6.65302V3.42147C6.65302 2.49666 6.88244 1.7562 7.37975 1.24597ZM8.42319 4.65295H14.4597C15.38 4.65295 16.1131 4.88769 16.6122 5.39947C17.1156 5.90962 17.347 6.64923 17.347 7.57442V17.5236H18.5444C18.9636 17.5236 19.2496 17.4163 19.4324 17.2289L19.4337 17.2275C19.6238 17.0374 19.7298 16.7549 19.7298 16.3552V3.4471C19.7298 3.04734 19.6238 2.76485 19.4337 2.57481L19.431 2.57206C19.248 2.37972 18.9625 2.27017 18.5444 2.27017H9.60866C9.19081 2.27017 8.90126 2.37956 8.71212 2.57341C8.52701 2.76329 8.42319 3.04633 8.42319 3.4471V4.65295ZM5.45564 21.6765C5.03728 21.6765 4.74743 21.5697 4.55844 21.3811C4.37372 21.1913 4.27017 20.9084 4.27017 20.5081V7.60005C4.27017 7.19928 4.37399 6.91625 4.55911 6.72636C4.74825 6.53252 5.03779 6.42313 5.45564 6.42313H14.3913C14.8095 6.42313 15.095 6.53268 15.278 6.72501L15.2807 6.72776C15.4708 6.9178 15.5768 7.20029 15.5768 7.60005V20.5081C15.5768 20.9079 15.4708 21.1904 15.2807 21.3804L15.2793 21.3818C15.0966 21.5693 14.8105 21.6765 14.3913 21.6765H5.45564Z"
fill="currentColor"
/>
</svg>

After

Width:  |  Height:  |  Size: 3.1 KiB

@@ -1,21 +1,34 @@
import clsx from 'clsx';
import React from 'react';
import { css } from 'styled-components';
import { Text, TextType } from '@/components';
type IconProps = TextType & {
type IconBase = TextType & {
disabled?: boolean;
};
type IconMaterialProps = IconBase & {
iconName: string;
variant?: 'filled' | 'outlined' | 'symbols-outlined';
icon?: never;
};
type IconSVGProps = IconBase & {
icon: React.ReactNode;
iconName?: never;
variant?: never;
};
export const Icon = ({
className,
iconName,
disabled,
iconName,
icon,
variant = 'outlined',
$theme = 'neutral',
...textProps
}: IconProps) => {
}: IconMaterialProps | IconSVGProps) => {
const hasLabel = 'aria-label' in textProps || 'aria-labelledby' in textProps;
const ariaHidden =
'aria-hidden' in textProps ? textProps['aria-hidden'] : !hasLabel;
@@ -24,15 +37,15 @@ export const Icon = ({
<Text
aria-hidden={ariaHidden}
className={clsx('--docs--icon-bg', className, {
'material-icons-filled': variant === 'filled',
'material-icons': variant === 'outlined',
'material-symbols-outlined': variant === 'symbols-outlined',
'material-icons-filled': variant === 'filled' && iconName,
'material-icons': variant === 'outlined' && iconName,
'material-symbols-outlined': variant === 'symbols-outlined' && iconName,
})}
$theme={disabled ? 'disabled' : $theme}
aria-disabled={disabled}
{...textProps}
>
{iconName}
{iconName ?? icon}
</Text>
);
};
@@ -1,3 +1,7 @@
/**
* Derivated from Blockquote PDF mapping
* @see: https://github.com/TypeCellOS/BlockNote/blob/main/packages/xl-pdf-exporter/src/pdf/defaultSchema/blocks.tsx
*/
import { Text } from '@react-pdf/renderer';
import { DocsExporterPDF } from '../types';
@@ -30,6 +34,15 @@ export const blockMappingHeadingPDF: DocsExporterPDF['mappings']['blockMapping']
const fontSizeEM =
block.props.level === 1 ? 2 : block.props.level === 2 ? 1.5 : 1.17;
const levelFontSizeEM = {
1: 2,
2: 1.5,
3: 1.17,
4: 1,
5: 0.83,
6: 0.67,
}[block.props.level as 1 | 2 | 3 | 4 | 5 | 6];
// Extract plain text for bookmark title
const bookmarkTitle =
extractTextFromBlockContent(block.content) || 'Untitled';
@@ -42,7 +55,7 @@ export const blockMappingHeadingPDF: DocsExporterPDF['mappings']['blockMapping']
title: bookmarkTitle,
}}
style={{
fontSize: fontSizeEM * FONT_SIZE * PIXELS_PER_POINT,
fontSize: levelFontSizeEM * FONT_SIZE * PIXELS_PER_POINT,
fontWeight: 700,
marginTop: `${fontSizeEM * MERGE_RATIO}px`,
marginBottom: `${fontSizeEM * MERGE_RATIO}px`,
@@ -21,9 +21,6 @@ export const blockMappingImagePDF: DocsExporterPDF['mappings']['blockMapping']['
if (blob.type.includes('svg')) {
const svgText = await blob.text();
const FALLBACK_SIZE = 536;
previewWidth = previewWidth || FALLBACK_SIZE;
const result = await convertSvgToPng(svgText, previewWidth);
pngConverted = result.png;
dimensions = { width: result.width, height: result.height };
@@ -36,7 +36,7 @@ export function downloadFile(blob: Blob, filename: string) {
*/
export async function convertSvgToPng(
svgText: string,
width: number,
width?: number,
): Promise<{ png: string; width: number; height: number }> {
// Create a canvas and render the SVG onto it
const canvas = document.createElement('canvas');
@@ -54,26 +54,36 @@ export async function convertSvgToPng(
const svgElement = svgDoc.documentElement;
// Get viewBox or fallback to width/height attributes
let height;
let calculatedHeight: number | undefined;
const svgWidth = svgElement.getAttribute?.('width');
const svgHeight = svgElement.getAttribute?.('height');
const viewBox = svgElement.getAttribute('viewBox')?.split(' ').map(Number);
const originalWidth = svgWidth ? parseInt(svgWidth) : viewBox?.[2];
const originalHeight = svgHeight ? parseInt(svgHeight) : viewBox?.[3];
if (originalWidth && originalHeight) {
const aspectRatio = originalHeight / originalWidth;
height = Math.round(width * aspectRatio);
}
const svg = Canvg.fromString(ctx, svgText);
svg.resize(width, height, true);
const FALLBACK_WIDTH = 536;
// Resize if width provided, preserving aspect ratio
if (originalWidth && originalHeight && width) {
const aspectRatio = originalHeight / originalWidth;
calculatedHeight = Math.round(width * aspectRatio);
svg.resize(width, calculatedHeight, true);
} else if (!width && !originalWidth) {
svg.resize(FALLBACK_WIDTH, undefined, true);
}
await svg.render();
const returnWidth = width || originalWidth || FALLBACK_WIDTH;
const returnHeight = calculatedHeight || returnWidth;
return {
png: canvas.toDataURL('image/png'),
width,
height: height || width,
width: returnWidth,
height: returnHeight,
};
}
@@ -0,0 +1,125 @@
import { VariantType, useToastProvider } from '@openfun/cunningham-react';
import {
UseMutationOptions,
useMutation,
useQueryClient,
} from '@tanstack/react-query';
import { useTranslation } from 'react-i18next';
import {
APIError,
UseInfiniteQueryResultAPI,
errorCauses,
fetchAPI,
} from '@/api';
import { Doc, DocsResponse, KEY_LIST_DOC } from '@/docs/doc-management';
enum ContentTypes {
Docx = 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
Markdown = 'text/markdown',
OctetStream = 'application/octet-stream',
}
export enum ContentTypesAllowed {
Docx = ContentTypes.Docx,
Markdown = ContentTypes.Markdown,
}
const getMimeType = (file: File): string => {
if (file.type) {
return file.type;
}
const extension = file.name.split('.').pop()?.toLowerCase();
switch (extension) {
case 'md':
return ContentTypes.Markdown;
case 'markdown':
return ContentTypes.Markdown;
case 'docx':
return ContentTypes.Docx;
default:
return ContentTypes.OctetStream;
}
};
export const importDoc = async (file: File): Promise<Doc> => {
const form = new FormData();
form.append(
'file',
new File([file], file.name, {
type: getMimeType(file),
lastModified: file.lastModified,
}),
);
const response = await fetchAPI(`documents/`, {
method: 'POST',
body: form,
withoutContentType: true,
});
if (!response.ok) {
throw new APIError('Failed to import the doc', await errorCauses(response));
}
return response.json() as Promise<Doc>;
};
type UseImportDocOptions = UseMutationOptions<Doc, APIError, File>;
export function useImportDoc(props?: UseImportDocOptions) {
const { toast } = useToastProvider();
const queryClient = useQueryClient();
const { t } = useTranslation();
return useMutation<Doc, APIError, File>({
mutationFn: importDoc,
...props,
onSuccess: (...successProps) => {
queryClient.setQueriesData<UseInfiniteQueryResultAPI<DocsResponse>>(
{ queryKey: [KEY_LIST_DOC] },
(oldData) => {
if (!oldData || oldData?.pages.length === 0) {
return oldData;
}
return {
...oldData,
pages: oldData.pages.map((page, index) => {
// Add the new doc to the first page only
if (index === 0) {
return {
...page,
results: [successProps[0], ...page.results],
};
}
return page;
}),
};
},
);
toast(
t('The document "{{documentName}}" has been successfully imported', {
documentName: successProps?.[0].title || '',
}),
VariantType.SUCCESS,
);
props?.onSuccess?.(...successProps);
},
onError: (...errorProps) => {
toast(
t(`The document "{{documentName}}" import has failed`, {
documentName: errorProps?.[1].name || '',
}),
VariantType.ERROR,
);
props?.onError?.(...errorProps);
},
});
}
@@ -1,14 +1,21 @@
import { Button } from '@openfun/cunningham-react';
import { useMemo } from 'react';
import {
Button,
VariantType,
useToastProvider,
} from '@openfun/cunningham-react';
import { useMemo, useState } from 'react';
import { useDropzone } from 'react-dropzone';
import { useTranslation } from 'react-i18next';
import { InView } from 'react-intersection-observer';
import { css } from 'styled-components';
import { Box, Card, Text } from '@/components';
import AllDocs from '@/assets/icons/doc-all.svg';
import { Box, Card, Icon, Text } from '@/components';
import { DocDefaultFilter, useInfiniteDocs } from '@/docs/doc-management';
import { useResponsiveStore } from '@/stores';
import { useInfiniteDocsTrashbin } from '../api';
import { ContentTypesAllowed, useImportDoc } from '../api/useImportDoc';
import { useResponsiveDocGrid } from '../hooks/useResponsiveDocGrid';
import {
@@ -24,6 +31,44 @@ export const DocsGrid = ({
target = DocDefaultFilter.ALL_DOCS,
}: DocsGridProps) => {
const { t } = useTranslation();
const [isDragOver, setIsDragOver] = useState(false);
const { toast } = useToastProvider();
const { getRootProps, getInputProps, open } = useDropzone({
accept: {
[ContentTypesAllowed.Docx]: ['.docx'],
[ContentTypesAllowed.Markdown]: ['.md'],
},
onDrop(acceptedFiles) {
setIsDragOver(false);
for (const file of acceptedFiles) {
importDoc(file);
}
},
onDragEnter: () => {
setIsDragOver(true);
},
onDragLeave: () => {
setIsDragOver(false);
},
onDropRejected(fileRejections) {
toast(
t(
`The document "{{documentName}}" import has failed (only .docx and .md files are allowed)`,
{
documentName: fileRejections?.[0].file.name || '',
},
),
VariantType.ERROR,
);
},
noClick: true,
});
const { mutate: importDoc } = useImportDoc();
const withUpload =
!target ||
target === DocDefaultFilter.ALL_DOCS ||
target === DocDefaultFilter.MY_DOCS;
const { isDesktop } = useResponsiveStore();
const { flexLeft, flexRight } = useResponsiveDocGrid();
@@ -60,21 +105,6 @@ export const DocsGrid = ({
void fetchNextPage();
};
let title = t('All docs');
switch (target) {
case DocDefaultFilter.MY_DOCS:
title = t('My docs');
break;
case DocDefaultFilter.SHARED_WITH_ME:
title = t('Shared with me');
break;
case DocDefaultFilter.TRASHBIN:
title = t('Trashbin');
break;
default:
title = t('All docs');
}
return (
<Box
$position="relative"
@@ -91,16 +121,24 @@ export const DocsGrid = ({
$width="100%"
$css={css`
${!isDesktop ? 'border: none;' : ''}
${isDragOver
? `
border: 2px dashed var(--c--contextuals--border--semantic--brand--primary);
background-color: var(--c--contextuals--background--semantic--brand--tertiary);
`
: ''}
`}
$padding={{
top: 'base',
horizontal: isDesktop ? 'md' : 'xs',
bottom: 'md',
}}
{...(withUpload ? getRootProps({ className: 'dropzone' }) : {})}
>
<Text as="h2" $size="h4" $margin={{ top: '0px', bottom: '10px' }}>
{title}
</Text>
{withUpload && <input {...getInputProps()} />}
<DocGridTitleBar
target={target}
onUploadClick={open}
withUpload={withUpload}
/>
{!hasDocs && !loading && (
<Box $padding={{ vertical: 'sm' }} $align="center" $justify="center">
@@ -110,7 +148,11 @@ export const DocsGrid = ({
</Box>
)}
{hasDocs && (
<Box $gap="6px" $overflow="auto">
<Box
$gap="6px"
$overflow="auto"
$padding={{ vertical: 'sm', horizontal: isDesktop ? 'md' : 'xs' }}
>
<Box role="grid" aria-label={t('Documents grid')}>
<Box role="rowgroup">
<Box
@@ -172,6 +214,73 @@ export const DocsGrid = ({
);
};
const DocGridTitleBar = ({
target,
onUploadClick,
withUpload,
}: {
target: DocDefaultFilter;
onUploadClick: () => void;
withUpload: boolean;
}) => {
const { t } = useTranslation();
const { isDesktop } = useResponsiveStore();
let title = t('All docs');
let icon = <Icon icon={<AllDocs width={24} height={24} />} />;
switch (target) {
case DocDefaultFilter.MY_DOCS:
icon = <Icon iconName="lock" />;
title = t('My docs');
break;
case DocDefaultFilter.SHARED_WITH_ME:
icon = <Icon iconName="group" />;
title = t('Shared with me');
break;
case DocDefaultFilter.TRASHBIN:
icon = <Icon iconName="delete" />;
title = t('Trashbin');
break;
default:
title = t('All docs');
}
return (
<Box
$direction="row"
$padding={{
vertical: 'md',
horizontal: isDesktop ? 'md' : 'xs',
}}
$css={css`
border-bottom: 1px solid var(--c--contextuals--border--surface--primary);
`}
$align="center"
$justify="space-between"
>
<Box $direction="row" $gap="xs" $align="center">
{icon}
<Text as="h2" $size="h4" $margin="none">
{title}
</Text>
</Box>
{withUpload && (
<Button
color="brand"
variant="tertiary"
onClick={(e) => {
e.stopPropagation();
onUploadClick();
}}
aria-label={t('Open the upload dialog')}
>
<Icon iconName="upload_file" $withThemeInherited />
</Button>
)}
</Box>
);
};
const useDocsQuery = (target: DocDefaultFilter) => {
const trashbinQuery = useInfiniteDocsTrashbin(
{
@@ -2,6 +2,7 @@ import { usePathname, useSearchParams } from 'next/navigation';
import { useTranslation } from 'react-i18next';
import { css } from 'styled-components';
import AllDocs from '@/assets/icons/doc-all.svg';
import { Box, Icon, StyledLink, Text } from '@/components';
import { useCunninghamTheme } from '@/cunningham';
import { DocDefaultFilter } from '@/docs/doc-management';
@@ -21,22 +22,22 @@ export const LeftPanelTargetFilters = () => {
const defaultQueries = [
{
icon: 'apps',
icon: <Icon icon={<AllDocs width={24} height={24} />} />,
label: t('All docs'),
targetQuery: DocDefaultFilter.ALL_DOCS,
},
{
icon: 'lock',
icon: <Icon iconName="lock" />,
label: t('My docs'),
targetQuery: DocDefaultFilter.MY_DOCS,
},
{
icon: 'group',
icon: <Icon iconName="group" />,
label: t('Shared with me'),
targetQuery: DocDefaultFilter.SHARED_WITH_ME,
},
{
icon: 'delete',
icon: <Icon iconName="delete" />,
label: t('Trashbin'),
targetQuery: DocDefaultFilter.TRASHBIN,
},
@@ -96,7 +97,7 @@ export const LeftPanelTargetFilters = () => {
}
`}
>
<Icon iconName={query.icon} />
{query.icon}
<Text $size="sm">{query.label}</Text>
</StyledLink>
);
@@ -120,7 +120,7 @@ const MainContent = ({
$css={css`
overflow-y: auto;
overflow-x: clip;
&:focus {
&:focus-visible {
outline: 3px solid ${colorsTokens['brand-400']};
outline-offset: -3px;
}
@@ -69,7 +69,7 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', 'wrong-api-key')
.set('authorization', `Bearer wrong-api-key`)
.set('content-type', 'application/json');
expect(response.status).toBe(401);
@@ -99,7 +99,7 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', apiKey)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'application/json');
expect(response.status).toBe(400);
@@ -114,7 +114,7 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', apiKey)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'application/json')
.send('');
@@ -129,9 +129,10 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', apiKey)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'image/png')
.send('randomdata');
expect(response.status).toBe(415);
expect(response.body).toStrictEqual({ error: 'Unsupported Content-Type' });
});
@@ -141,38 +142,73 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', apiKey)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'text/markdown')
.set('accept', 'image/png')
.send('# Header');
expect(response.status).toBe(406);
expect(response.body).toStrictEqual({ error: 'Unsupported format' });
});
test.each([[apiKey], [`Bearer ${apiKey}`]])(
'POST /api/convert with correct content with Authorization: %s',
async (authHeader) => {
const app = initApp();
test('POST /api/convert BlockNote to Markdown', async () => {
const app = initApp();
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'application/vnd.blocknote+json')
.set('accept', 'text/markdown')
.send(expectedBlocks);
const response = await request(app)
.post('/api/convert')
.set('Origin', origin)
.set('Authorization', authHeader)
.set('content-type', 'text/markdown')
.set('accept', 'application/vnd.yjs.doc')
.send(expectedMarkdown);
expect(response.status).toBe(200);
expect(response.header['content-type']).toBe(
'text/markdown; charset=utf-8',
);
expect(typeof response.text).toBe('string');
expect(response.text.trim()).toBe(expectedMarkdown);
});
expect(response.status).toBe(200);
expect(response.body).toBeInstanceOf(Buffer);
test('POST /api/convert BlockNote to Yjs', async () => {
const app = initApp();
const editor = ServerBlockNoteEditor.create();
const blocks = await editor.tryParseMarkdownToBlocks(expectedMarkdown);
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'application/vnd.blocknote+json')
.set('accept', 'application/vnd.yjs.doc')
.send(blocks)
.responseType('blob');
const editor = ServerBlockNoteEditor.create();
const doc = new Y.Doc();
Y.applyUpdate(doc, response.body);
const blocks = editor.yDocToBlocks(doc, 'document-store');
expect(response.status).toBe(200);
expect(response.header['content-type']).toBe('application/vnd.yjs.doc');
expect(blocks).toStrictEqual(expectedBlocks);
},
);
// Decode the Yjs response and verify it contains the correct blocks
const responseBuffer = Buffer.from(response.body as Buffer);
const ydoc = new Y.Doc();
Y.applyUpdate(ydoc, responseBuffer);
const decodedBlocks = editor.yDocToBlocks(ydoc, 'document-store');
expect(decodedBlocks).toStrictEqual(expectedBlocks);
});
test('POST /api/convert BlockNote to HTML', async () => {
const app = initApp();
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'application/vnd.blocknote+json')
.set('accept', 'text/html')
.send(expectedBlocks);
expect(response.status).toBe(200);
expect(response.header['content-type']).toBe('text/html; charset=utf-8');
expect(typeof response.text).toBe('string');
expect(response.text).toBe(expectedHTML);
});
test('POST /api/convert Yjs to HTML', async () => {
const app = initApp();
@@ -183,10 +219,11 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', apiKey)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'application/vnd.yjs.doc')
.set('accept', 'text/html')
.send(Buffer.from(yjsUpdate));
expect(response.status).toBe(200);
expect(response.header['content-type']).toBe('text/html; charset=utf-8');
expect(typeof response.text).toBe('string');
@@ -202,10 +239,11 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', apiKey)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'application/vnd.yjs.doc')
.set('accept', 'text/markdown')
.send(Buffer.from(yjsUpdate));
expect(response.status).toBe(200);
expect(response.header['content-type']).toBe(
'text/markdown; charset=utf-8',
@@ -223,15 +261,16 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', apiKey)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'application/vnd.yjs.doc')
.set('accept', 'application/json')
.send(Buffer.from(yjsUpdate));
expect(response.status).toBe(200);
expect(response.header['content-type']).toBe(
'application/json; charset=utf-8',
);
expect(Array.isArray(response.body)).toBe(true);
expect(response.body).toBeInstanceOf(Array);
expect(response.body).toStrictEqual(expectedBlocks);
});
@@ -240,15 +279,16 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', apiKey)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'text/markdown')
.set('accept', 'application/json')
.send(expectedMarkdown);
expect(response.status).toBe(200);
expect(response.header['content-type']).toBe(
'application/json; charset=utf-8',
);
expect(Array.isArray(response.body)).toBe(true);
expect(response.body).toBeInstanceOf(Array);
expect(response.body).toStrictEqual(expectedBlocks);
});
@@ -257,11 +297,12 @@ describe('Server Tests', () => {
const response = await request(app)
.post('/api/convert')
.set('origin', origin)
.set('authorization', apiKey)
.set('authorization', `Bearer ${apiKey}`)
.set('content-type', 'application/vnd.yjs.doc')
.set('accept', 'application/json')
.send(Buffer.from('notvalidyjs'));
expect(response.status).toBe(400);
expect(response.body).toStrictEqual({ error: 'Invalid Yjs content' });
expect(response.body).toStrictEqual({ error: 'Invalid content' });
});
});
@@ -14,27 +14,115 @@ interface ErrorResponse {
error: string;
}
type ConversionResponseBody = Uint8Array | string | object | ErrorResponse;
interface InputReader {
supportedContentTypes: string[];
read(data: Buffer): Promise<PartialBlock[]>;
}
interface OutputWriter {
supportedContentTypes: string[];
write(blocks: PartialBlock[]): Promise<ConversionResponseBody>;
}
const editor = ServerBlockNoteEditor.create<
DefaultBlockSchema,
DefaultInlineContentSchema,
DefaultStyleSchema
>();
const ContentTypes = {
XMarkdown: 'text/x-markdown',
Markdown: 'text/markdown',
YJS: 'application/vnd.yjs.doc',
FormUrlEncoded: 'application/x-www-form-urlencoded',
OctetStream: 'application/octet-stream',
HTML: 'text/html',
BlockNote: 'application/vnd.blocknote+json',
JSON: 'application/json',
} as const;
const createYDocument = (blocks: PartialBlock[]) =>
editor.blocksToYDoc(blocks, 'document-store');
const readers: InputReader[] = [
{
// application/x-www-form-urlencoded is interpreted as Markdown for backward compatibility
supportedContentTypes: [
ContentTypes.Markdown,
ContentTypes.XMarkdown,
ContentTypes.FormUrlEncoded,
],
read: (data) => editor.tryParseMarkdownToBlocks(data.toString()),
},
{
supportedContentTypes: [ContentTypes.YJS, ContentTypes.OctetStream],
read: async (data) => {
const ydoc = new Y.Doc();
Y.applyUpdate(ydoc, data);
return editor.yDocToBlocks(ydoc, 'document-store') as PartialBlock[];
},
},
{
supportedContentTypes: [ContentTypes.BlockNote],
read: async (data) => JSON.parse(data.toString()),
},
];
const writers: OutputWriter[] = [
{
supportedContentTypes: [ContentTypes.BlockNote, ContentTypes.JSON],
write: async (blocks) => blocks,
},
{
supportedContentTypes: [ContentTypes.YJS, ContentTypes.OctetStream],
write: async (blocks) => Y.encodeStateAsUpdate(createYDocument(blocks)),
},
{
supportedContentTypes: [ContentTypes.Markdown, ContentTypes.XMarkdown],
write: (blocks) => editor.blocksToMarkdownLossy(blocks),
},
{
supportedContentTypes: [ContentTypes.HTML],
write: (blocks) => editor.blocksToHTMLLossy(blocks),
},
];
const normalizeContentType = (value: string) => value.split(';')[0];
export const convertHandler = async (
req: Request<object, Uint8Array | ErrorResponse, Buffer, object>,
res: Response<Uint8Array | string | object | ErrorResponse>,
res: Response<ConversionResponseBody>,
) => {
if (!req.body || req.body.length === 0) {
res.status(400).json({ error: 'Invalid request: missing content' });
return;
}
const contentType = (req.header('content-type') || 'text/markdown').split(
';',
)[0];
const accept = (req.header('accept') || 'application/vnd.yjs.doc').split(
';',
)[0];
const contentType = normalizeContentType(
req.header('content-type') || ContentTypes.Markdown,
);
const reader = readers.find((reader) =>
reader.supportedContentTypes.includes(contentType),
);
if (!reader) {
res.status(415).json({ error: 'Unsupported Content-Type' });
return;
}
const accept = normalizeContentType(req.header('accept') || ContentTypes.YJS);
const writer = writers.find((writer) =>
writer.supportedContentTypes.includes(accept),
);
if (!writer) {
res.status(406).json({ error: 'Unsupported format' });
return;
}
let blocks:
| PartialBlock<
@@ -44,63 +132,23 @@ export const convertHandler = async (
>[]
| null = null;
try {
// First, convert from the input format to blocks
// application/x-www-form-urlencoded is interpreted as Markdown for backward compatibility
if (
contentType === 'text/markdown' ||
contentType === 'application/x-www-form-urlencoded'
) {
blocks = await editor.tryParseMarkdownToBlocks(req.body.toString());
} else if (
contentType === 'application/vnd.yjs.doc' ||
contentType === 'application/octet-stream'
) {
try {
const ydoc = new Y.Doc();
Y.applyUpdate(ydoc, req.body);
blocks = editor.yDocToBlocks(ydoc, 'document-store') as PartialBlock[];
} catch (e) {
logger('Invalid Yjs content:', e);
res.status(400).json({ error: 'Invalid Yjs content' });
return;
}
} else {
res.status(415).json({ error: 'Unsupported Content-Type' });
try {
blocks = await reader.read(req.body);
} catch (e) {
logger('Invalid content:', e);
res.status(400).json({ error: 'Invalid content' });
return;
}
if (!blocks || blocks.length === 0) {
res.status(500).json({ error: 'No valid blocks were generated' });
return;
}
// Then, convert from blocks to the output format
if (accept === 'application/json') {
res.status(200).json(blocks);
} else {
const yDocument = editor.blocksToYDoc(blocks, 'document-store');
if (
accept === 'application/vnd.yjs.doc' ||
accept === 'application/octet-stream'
) {
res
.status(200)
.setHeader('content-type', 'application/octet-stream')
.send(Y.encodeStateAsUpdate(yDocument));
} else if (accept === 'text/markdown') {
res
.status(200)
.setHeader('content-type', 'text/markdown')
.send(await editor.blocksToMarkdownLossy(blocks));
} else if (accept === 'text/html') {
res
.status(200)
.setHeader('content-type', 'text/html')
.send(await editor.blocksToHTMLLossy(blocks));
} else {
res.status(406).json({ error: 'Unsupported format' });
}
}
res
.status(200)
.setHeader('content-type', accept)
.send(await writer.write(blocks));
} catch (e) {
logger('conversion failed:', e);
res.status(500).json({ error: 'An error occurred' });
+1872 -1767
View File
File diff suppressed because it is too large Load Diff