wip ok

wip foiré
WIP webgl2 + basic blur
2025-03-29 14:05:28 +01:00 · 2025-03-29 14:05:28 +01:00 · 2025-03-29 14:05:28 +01:00 · 2025-03-29 14:05:28 +01:00 · 2025-03-29 14:05:28 +01:00 · 2025-03-28 20:14:32 +01:00
43 changed files with 2832 additions and 2330 deletions
@@ -119,12 +119,9 @@ jobs:
    runs-on: ubuntu-latest
    if: github.event_name != 'pull_request'
    steps:
-      -
-        name: Checkout repository
-        uses: actions/checkout@v4
-      -
-        name: Call argocd github webhook
-        run: |
-          data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/numerique-gouv/lasuite-deploiement"}}'
-          sig=$(echo -n ${data} | openssl dgst -sha1 -hmac "${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET }}" | awk '{print "X-Hub-Signature: sha1="$2}')
-          curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" ${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}
+      - uses: numerique-gouv/action-argocd-webhook-notification@main
+        id: notify
+        with:
+          deployment_repo_path: "${{ secrets.DEPLOYMENT_REPO_URL }}"
+          argocd_webhook_secret: "${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET }}"
+          argocd_url: "${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}"
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2021-2023 DINUM/Etalab
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
@@ -1,4 +1,4 @@
-FROM livekit/livekit-server:latest
+FROM livekit/livekit-server:v1.8.0

 # We inject the nip.io certificate manually because the livekit chart doesn't support volume mounting
 COPY rootCA.pem /etc/ssl/certs/
@@ -2,7 +2,6 @@

 This document is a step-by-step guide that describes how to install Visio on a k8s cluster without AI features.

-
 ## Prerequisites

 - k8s cluster with an nginx-ingress controller
@@ -43,11 +42,11 @@ It will expire on 23 March 2027 🗓
 2. Create kind cluster with containerd registry config dir enabled
 Creating cluster "visio" ...
 ✓ Ensuring node image (kindest/node:v1.27.3) 🖼
- ✓ Preparing nodes 📦  
- ✓ Writing configuration 📜 
- ✓ Starting control-plane 🕹️ 
- ✓ Installing CNI 🔌 
- ✓ Installing StorageClass 💾 
+ ✓ Preparing nodes 📦
+ ✓ Writing configuration 📜
+ ✓ Starting control-plane 🕹️
+ ✓ Installing CNI 🔌
+ ✓ Installing StorageClass 💾
 Set kubectl context to "kind-visio"
 You can now use your cluster with:

@@ -95,13 +94,14 @@ ingress-nginx-admission-create-jgnc9        0/1     Completed   0          2m44s
 ingress-nginx-admission-patch-wrt47         0/1     Completed   0          2m44s
 ingress-nginx-controller-57c548c4cd-9xwt6   1/1     Running     0          2m44s
 ```
-When your k8s cluster is ready, you can start the deployment. This cluster is special because it uses the *.127.0.0.1.nip.io domain and mkcert certificates to have full HTTPS support and easy domain name management.

-Please remember that *.127.0.0.1.nip.io will always resolve to 127.0.0.1, except in the k8s cluster where we configure CoreDNS to answer with the ingress-nginx service IP.
+When your k8s cluster is ready, you can start the deployment. This cluster is special because it uses the \*.127.0.0.1.nip.io domain and mkcert certificates to have full HTTPS support and easy domain name management.
+
+Please remember that \*.127.0.0.1.nip.io will always resolve to 127.0.0.1, except in the k8s cluster where we configure CoreDNS to answer with the ingress-nginx service IP.

 ## Preparation

-### What will you use to authenticate your users ? 
+### What will you use to authenticate your users ?

 Visio uses OIDC, so if you already have an OIDC provider, obtain the necessary information to use it. In the next step, we will see how to configure Django (and thus Visio) to use it. If you do not have a provider, we will show you how to deploy a local Keycloak instance (this is not a production deployment, just a demo).

@@ -153,6 +153,7 @@ keycloak-0              1/1     Running   0          26m
 keycloak-postgresql-0   1/1     Running   0          26m
 redis-master-0          1/1     Running   0          35s
 ```
+
 When the redis is ready we can deploy livekit-server.

 ```
@@ -194,6 +195,7 @@ livekit-livekit-server-5c5fb87f7f-ct6x5   1/1     Running   0          15m
 postgresql-0                              1/1     Running   0          50s
 redis-master-0                            1/1     Running   0          19
 ```
+
 From here important information you will need are :

 ```
@@ -231,3 +233,98 @@ meet-admin               <none>   meet.127.0.0.1.nip.io       localhost   80, 44
 ```

 You can use Visio on https://meet.127.0.0.1.nip.io. The provisioning user in keycloak is meet/meet.
+
+## All options
+
+These are the environmental options available on meet backend.
+
+| Option                                          | Description                              | default                                                                                                                                                       |
+| ----------------------------------------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| DJANGO_ALLOWED_HOSTS                            | Host that are allowed                    | []                                                                                                                                                            |
+| DJANGO_SECRET_KEY                               | Secret key used                          |                                                                                                                                                               |
+| DJANGO_SILENCED_SYSTEM_CHECKS                   | Silence system checks                    | []                                                                                                                                                            |
+| DJANGO_ALLOW_UNSECURE_USER_LISTING              |                                          | false                                                                                                                                                         |
+| DB_ENGINE                                       | Database engine used                     | django.db.backends.postgresql_psycopg2                                                                                                                        |
+| DB_NAME                                         | name of the database                     | meet                                                                                                                                                          |
+| DB_USER                                         | user used to connect to database         | dinum                                                                                                                                                         |
+| DB_PASSWORD                                     | password used to connect to the database | pass                                                                                                                                                          |
+| DB_HOST                                         | hostname of the database                 | localhost                                                                                                                                                     |
+| DB_PORT                                         | port to connect to database              | 5432                                                                                                                                                          |
+| STORAGES_STATICFILES_BACKEND                    | Static file serving engine               | whitenoise.storage.CompressedManifestStaticFilesStorage                                                                                                       |
+| AWS_S3_ENDPOINT_URL                             | S3 host endpoint                         |                                                                                                                                                               |
+| AWS_S3_ACCESS_KEY_ID                            | s3 access key                            |                                                                                                                                                               |
+| AWS_S3_SECRET_ACCESS_KEY                        | s3 secret key                            |                                                                                                                                                               |
+| AWS_S3_REGION_NAME                              | s3 region                                |                                                                                                                                                               |
+| AWS_STORAGE_BUCKET_NAME                         | s3 bucket name                           |                                                                                                                                                               |
+| DJANGO_LANGUAGE_CODE                            | Default language                         | en-us                                                                                                                                                         |
+| REDIS_URL                                       | redis endpoint                           | redis://redis:6379/1                                                                                                                                          |
+| REQUEST_ENTRY_THROTTLE_RATES                    | throttle rates                           | 150/minute                                                                                                                                                    |
+| SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK | deploy check                             | False                                                                                                                                                         |
+| FRONTEND_ANALYTICS                              | analytics information                    | {}                                                                                                                                                            |
+| FRONTEND_SUPPORT                                | crisp frontend support                   | {}                                                                                                                                                            |
+| FRONTEND_SILENCE_LIVEKIT_DEBUG                  | silence livekit debug                    | false                                                                                                                                                         |
+| DJANGO_EMAIL_BACKEND                            | email backend library                    | django.core.mail.backends.smtp.EmailBackend                                                                                                                   |
+| DJANGO_EMAIL_HOST                               | host of the email server                 |                                                                                                                                                               |
+| DJANGO_EMAIL_HOST_USER                          | user to connect to the email server      |                                                                                                                                                               |
+| DJANGO_EMAIL_HOST_PASSWORD                      | password to connect tto the email server |                                                                                                                                                               |
+| DJANGO_EMAIL_PORT                               | por tot connect to the email server      |                                                                                                                                                               |
+| DJANGO_EMAIL_USE_TLS                            | enable tls on email connection           | false                                                                                                                                                         |
+| DJANGO_EMAIL_USE_SSL                            | enable tls on email connection           | false                                                                                                                                                         |
+| DJANGO_EMAIL_FROM                               | email from account                       | from@example.com                                                                                                                                              |
+| DJANGO_CORS_ALLOW_ALL_ORIGINS                   | allow all cors origins                   | true                                                                                                                                                          |
+| DJANGO_CORS_ALLOWED_ORIGINS                     | origins to allow in string               | []                                                                                                                                                            |
+| DJANGO_CORS_ALLOWED_ORIGIN_REGEXES              | origins to allow in regex                | []                                                                                                                                                            |
+| SENTRY_DSN                                      | sentry server                            |                                                                                                                                                               |
+| DJANGO_CELERY_BROKER_URL                        | celery broker host                       | redis://redis:6379/0                                                                                                                                          |
+| DJANGO_CELERY_BROKER_TRANSPORT_OPTIONS          | celery broker options                    | {}                                                                                                                                                            |
+| OIDC_CREATE_USER                                | create oidc user if not exists           | false                                                                                                                                                         |
+| OIDC_VERIFY_SSL                                 | verify ssl for oidc                      | true                                                                                                                                                          |
+| OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION       | fallback to email for identification     | false                                                                                                                                                         |
+| OIDC_RP_SIGN_ALGO                               | token verification algoritm used by oidc | RS256                                                                                                                                                         |
+| OIDC_RP_CLIENT_ID                               | oidc client                              | meet                                                                                                                                                          |
+| OIDC_RP_CLIENT_SECRET                           | oidc client secret                       |                                                                                                                                                               |
+| OIDC_OP_JWKS_ENDPOINT                           | oidc endpoint for JWKS                   |                                                                                                                                                               |
+| OIDC_OP_AUTHORIZATION_ENDPOINT                  | oidc endpoint for authorization          |                                                                                                                                                               |
+| OIDC_OP_TOKEN_ENDPOINT                          | oidc endpoint for token                  |                                                                                                                                                               |
+| OIDC_OP_USER_ENDPOINT                           | oidc endpoint for user                   |                                                                                                                                                               |
+| OIDC_OP_LOGOUT_ENDPOINT                         | oidc endpoint for logout                 |                                                                                                                                                               |
+| OIDC_AUTH_REQUEST_EXTRA_PARAMS                  | extra parameters for oidc request        |                                                                                                                                                               |
+| OIDC_RP_SCOPES                                  | oidc scopes                              | openid email                                                                                                                                                  |
+| LOGIN_REDIRECT_URL                              | login redirect url                       |                                                                                                                                                               |
+| LOGIN_REDIRECT_URL_FAILURE                      | login redurect url for failure           |                                                                                                                                                               |
+| LOGOUT_REDIRECT_URL                             | url to redirect to on logout             |                                                                                                                                                               |
+| OIDC_USE_NONCE                                  | use nonce for oidc                       | true                                                                                                                                                          |
+| OIDC_REDIRECT_REQUIRE_HTTPS                     | require https for oidc                   | false                                                                                                                                                         |
+| OIDC_REDIRECT_ALLOWED_HOSTS                     | allowed redirect hosts for oidc          | []                                                                                                                                                            |
+| OIDC_STORE_ID_TOKEN                             | store oidc ID token                      | true                                                                                                                                                          |
+| ALLOW_LOGOUT_GET_METHOD                         | allow logout through get method          | true                                                                                                                                                          |
+| OIDC_REDIRECT_FIELD_NAME                        | direct field for oidc                    | returnTo                                                                                                                                                      |
+| OIDC_USERINFO_FULLNAME_FIELDS                   | full name claim from OIDC token          | ["given_name", "usual_name"]                                                                                                                                  |
+| OIDC_USERINFO_SHORTNAME_FIELD                   | shortname claim from OIDC token          | given_name                                                                                                                                                    |
+| LIVEKIT_API_KEY                                 | livekit api key                          |                                                                                                                                                               |
+| LIVEKIT_API_SECRET                              | livekit api secret                       |                                                                                                                                                               |
+| LIVEKIT_API_URL                                 | livekit api url                          |                                                                                                                                                               |
+| RESOURCE_DEFAULT_ACCESS_LEVEL                   | default resource access level for rooms  | public                                                                                                                                                        |
+| ALLOW_UNREGISTERED_ROOMS                        | Allow usage of unregistered rooms        | true                                                                                                                                                          |
+| RECORDING_ENABLE                                | record meeting option                    | false                                                                                                                                                         |
+| RECORDING_OUTPUT_FOLDER                         | folder to store meetings                 | recordings                                                                                                                                                    |
+| RECORDING_VERIFY_SSL                            | verify ssl for recording storage         | true                                                                                                                                                          |
+| RECORDING_WORKER_CLASSES                        | worker classes for recording             | {"screen_recording": "core.recording.worker.services.VideoCompositeEgressService","transcript": "core.recording.worker.services.AudioCompositeEgressService"} |
+| RECORDING_EVENT_PARSER_CLASS                    | storage event engine for recording       | core.recording.event.parsers.MinioParser                                                                                                                      |
+| RECORDING_ENABLE_STORAGE_EVENT_AUTH             | enable storage event authorization       | true                                                                                                                                                          |
+| RECORDING_STORAGE_EVENT_ENABLE                  | enable recording storage events          | false                                                                                                                                                         |
+| RECORDING_STORAGE_EVENT_TOKEN                   | recording storage event token            |                                                                                                                                                               |
+| SUMMARY_SERVICE_ENDPOINT                        | summary service endpoint                 |                                                                                                                                                               |
+| SUMMARY_SERVICE_API_TOKEN                       | api token voor summary service           |                                                                                                                                                               |
+| SIGNUP_NEW_USER_TO_MARKETING_EMAIL              | signup users to marketing emails         | false                                                                                                                                                         |
+| MARKETING_SERVICE_CLASS                         | markering class                          | core.services.marketing.BrevoMarketingService                                                                                                                 |
+| BREVO_API_KEY                                   | breva api key for marketing emails       |                                                                                                                                                               |
+| BREVO_API_CONTACT_LIST_IDS                      | brevo api contact list ids               | []                                                                                                                                                            |
+| DJANGO_BREVO_API_CONTACT_ATTRIBUTES             | brevo contact attributes                 | {"VISIO_USER": True}                                                                                                                                          |
+| BREVO_API_TIMEOUT                               | brevo timeout                            | 1                                                                                                                                                             |
+| LOBBY_KEY_PREFIX                                | lobby prefix                             | room_lobby                                                                                                                                                    |
+| LOBBY_WAITING_TIMEOUT                           | lobby waiting tumeout                    | 3                                                                                                                                                             |
+| LOBBY_DENIED_TIMEOUT                            | lobby deny timeout                       | 5                                                                                                                                                             |
+| LOBBY_ACCEPTED_TIMEOUT                          | lobby accept timeout                     | 21600                                                                                                                                                         |
+| LOBBY_NOTIFICATION_TYPE                         | lobby notification types                 | participantWaiting                                                                                                                                            |
+| LOBBY_COOKIE_NAME                               | lobby cookie name                        | lobbyParticipantId                                                                                                                                            |
@@ -121,7 +121,9 @@ class OIDCAuthenticationBackend(MozillaOIDCAuthenticationBackend):
            contact_data = ContactData(
                email=email, attributes={"VISIO_SOURCE": ["SIGNIN"]}
            )
-            marketing_service.create_contact(contact_data, timeout=1)
+            marketing_service.create_contact(
+                contact_data, timeout=settings.BREVO_API_TIMEOUT
+            )
        except (ContactCreationError, ImproperlyConfigured, ImportError):
            pass

@@ -12,7 +12,12 @@ from django.conf import settings
 from django.core.cache import cache

 from asgiref.sync import async_to_sync
-from livekit.api import LiveKitAPI, SendDataRequest, TwirpError  # pylint: disable=E0611
+from livekit.api import (  # pylint: disable=E0611
+    ListRoomsRequest,
+    LiveKitAPI,
+    SendDataRequest,
+    TwirpError,
+)

 from core import models, utils

@@ -221,6 +226,12 @@ class LobbyService:
            color=color,
        )

+        try:
+            self.notify_participants(room_id=room_id)
+        except LobbyNotificationError:
+            # If room not created yet, there is no participants to notify
+            pass
+
        cache_key = self._get_cache_key(room_id, participant_id)
        cache.set(
            cache_key,
@@ -228,12 +239,6 @@ class LobbyService:
            timeout=settings.LOBBY_WAITING_TIMEOUT,
        )

-        try:
-            self.notify_participants(room_id=room_id)
-        except LobbyNotificationError:
-            # If room not created yet, there is no participants to notify
-            pass
-
        return participant

    def _get_participant(
@@ -343,7 +348,18 @@ class LobbyService:
        }

        lkapi = LiveKitAPI(**settings.LIVEKIT_CONFIGURATION)
+
        try:
+            room_response = await lkapi.room.list_rooms(
+                ListRoomsRequest(
+                    names=[str(room_id)],
+                )
+            )
+
+            # Check if the room exists
+            if not room_response.rooms:
+                return
+
            await lkapi.room.send_data(
                SendDataRequest(
                    room=str(room_id),
@@ -776,6 +776,43 @@ def test_update_participant_status_success(mock_cache, lobby_service, participan
    lobby_service._get_cache_key.assert_called_once_with(room.id, participant_id)


+@mock.patch("core.services.lobby.LiveKitAPI")
+def test_notify_participants_success_no_room(mock_livekit_api, lobby_service):
+    """Test the notify_participants method when the LiveKit room doesn't exist yet."""
+    room = RoomFactory(access_level=RoomAccessLevel.RESTRICTED)
+
+    # Set up the mock LiveKitAPI and its behavior
+    mock_api_instance = mock.Mock()
+    mock_api_instance.room = mock.Mock()
+    mock_api_instance.room.send_data = mock.AsyncMock()
+
+    # Create a proper response object with an empty rooms list
+    class MockResponse:
+        """LiveKit API response mock with empty rooms list."""
+
+        rooms = []
+
+    mock_api_instance.room.list_rooms = mock.AsyncMock(return_value=MockResponse())
+    mock_api_instance.aclose = mock.AsyncMock()
+    mock_livekit_api.return_value = mock_api_instance
+
+    # Act
+    lobby_service.notify_participants(room.id)
+
+    # Assert
+    # Verify the API was initialized with correct configuration
+    mock_livekit_api.assert_called_once_with(**settings.LIVEKIT_CONFIGURATION)
+
+    # Verify that the service checked for existing rooms
+    mock_api_instance.room.list_rooms.assert_called_once()
+
+    # Verify the send_data method was not called since no room exists
+    mock_api_instance.room.send_data.assert_not_called()
+
+    # Verify the connection was properly closed
+    mock_api_instance.aclose.assert_called_once()
+
+
@mock.patch("core.services.lobby.LiveKitAPI")
 def test_notify_participants_success(mock_livekit_api, lobby_service):
    """Test successful participant notification."""
@@ -784,6 +821,14 @@ def test_notify_participants_success(mock_livekit_api, lobby_service):
    mock_api_instance = mock.Mock()
    mock_api_instance.room = mock.Mock()
    mock_api_instance.room.send_data = mock.AsyncMock()
+
+    class MockResponse:
+        """LiveKit API response mock with non-empty rooms list."""
+
+        rooms = ["room-1"]
+
+    mock_api_instance.room.list_rooms = mock.AsyncMock(return_value=MockResponse())
+
    mock_api_instance.aclose = mock.AsyncMock()
    mock_livekit_api.return_value = mock_api_instance

@@ -793,6 +838,9 @@ def test_notify_participants_success(mock_livekit_api, lobby_service):
    # Verify the API was called correctly
    mock_livekit_api.assert_called_once_with(**settings.LIVEKIT_CONFIGURATION)

+    # Verify that the service checked for existing rooms
+    mock_api_instance.room.list_rooms.assert_called_once()
+
    # Verify the send_data method was called
    mock_api_instance.room.send_data.assert_called_once()
    send_data_request = mock_api_instance.room.send_data.call_args[0][0]
@@ -817,6 +865,14 @@ def test_notify_participants_error(mock_livekit_api, lobby_service):
    mock_api_instance.room.send_data = mock.AsyncMock(
        side_effect=TwirpError(msg="test error", code=123)
    )
+
+    class MockResponse:
+        """LiveKit API response mock with non-empty rooms list."""
+
+        rooms = ["room-1"]
+
+    mock_api_instance.room.list_rooms = mock.AsyncMock(return_value=MockResponse())
+
    mock_api_instance.aclose = mock.AsyncMock()
    mock_livekit_api.return_value = mock_api_instance

@@ -829,6 +885,9 @@ def test_notify_participants_error(mock_livekit_api, lobby_service):
    # Verify the API was called correctly
    mock_livekit_api.assert_called_once_with(**settings.LIVEKIT_CONFIGURATION)

+    # Verify that the service checked for existing rooms
+    mock_api_instance.room.list_rooms.assert_called_once()
+
    # Verify send_data was called
    mock_api_instance.room.send_data.assert_called_once()

@@ -19,6 +19,7 @@ from django.utils.translation import gettext_lazy as _
 import sentry_sdk
 from configurations import Configuration, values
 from sentry_sdk.integrations.django import DjangoIntegration
+from sentry_sdk.integrations.logging import ignore_logger

 # Build paths inside the project like this: BASE_DIR / 'subdir'.
 BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
@@ -497,6 +498,9 @@ class Base(Configuration):
        converter=lambda x: int(x),  # pylint: disable=unnecessary-lambda
    )
    BREVO_API_CONTACT_ATTRIBUTES = values.DictValue({"VISIO_USER": True})
+    BREVO_API_TIMEOUT = values.PositiveIntegerValue(
+        1, environ_name="BREVO_API_TIMEOUT", environ_prefix=None
+    )

    # Lobby configurations
    LOBBY_KEY_PREFIX = values.Value(
@@ -575,6 +579,9 @@ class Base(Configuration):
            scope = sentry_sdk.get_global_scope()
            scope.set_tag("application", "backend")

+            # Ignore the logs added by the DockerflowMiddleware
+            ignore_logger("request.summary")
+

 class Build(Base):
    """Settings used when the application is built.
@@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"

 [project]
 name = "meet"
-version = "0.1.15"
+version = "0.1.16"
 authors = [{ "name" = "DINUM", "email" = "dev@mail.numerique.gouv.fr" }]
 classifiers = [
    "Development Status :: 5 - Production/Stable",
@@ -25,7 +25,7 @@ license = { file = "LICENSE" }
 readme = "README.md"
 requires-python = ">=3.10"
 dependencies = [
-    "boto3==1.37.4",
+    "boto3==1.37.18",
    "Brotli==1.1.0",
    "brevo-python==1.1.2",
    "celery[redis]==5.4.0",
@@ -48,16 +48,16 @@ dependencies = [
    "june-analytics-python==2.3.0",
    "markdown==3.7",
    "nested-multipart-parser==1.5.0",
-    "psycopg[binary]==3.2.5",
+    "psycopg[binary]==3.2.6",
    "PyJWT==2.10.1",
    "python-frontmatter==1.1.0",
    "requests==2.32.3",
-    "sentry-sdk==2.22.0",
+    "sentry-sdk==2.24.1",
    "url-normalize==1.4.3",
    "whitenoise==6.9.0",
    "mozilla-django-oidc==4.0.1",
    "livekit-api==0.8.2",
-    "aiohttp==3.11.13",
+    "aiohttp==3.11.14",
 ]

 [project.urls]
@@ -72,18 +72,18 @@ dev = [
    "drf-spectacular-sidecar==2025.3.1",
    "freezegun==1.5.1",
    "ipdb==0.13.13",
-    "ipython==9.0.1",
-    "pyfakefs==5.7.4",
+    "ipython==9.0.2",
+    "pyfakefs==5.8.0",
    "pylint-django==2.6.1",
-    "pylint==3.3.4",
+    "pylint==3.3.6",
    "pytest-cov==6.0.0",
    "pytest-django==4.10.0",
    "pytest==8.3.5",
    "pytest-icdiff==0.9",
    "pytest-xdist==3.6.1",
-    "responses==0.25.6",
-    "ruff==0.9.9",
-    "types-requests==2.32.0.20250301",
+    "responses==0.25.7",
+    "ruff==0.11.2",
+    "types-requests==2.32.0.20250306",
 ]

 [tool.setuptools]
@@ -1,7 +1,7 @@
 {
  "name": "meet",
  "private": true,
-  "version": "0.1.15",
+  "version": "0.1.16",
  "type": "module",
  "scripts": {
    "dev": "panda codegen && vite",
@@ -16,45 +16,45 @@
    "@livekit/components-react": "2.8.1",
    "@livekit/components-styles": "1.1.4",
    "@livekit/track-processors": "0.3.3",
-    "@pandacss/preset-panda": "0.53.0",
-    "@react-aria/toast": "3.0.0-beta.19",
+    "@pandacss/preset-panda": "0.53.3",
+    "@react-aria/toast": "3.0.1",
    "@remixicon/react": "4.6.0",
-    "@tanstack/react-query": "5.67.1",
+    "@tanstack/react-query": "5.69.0",
    "crisp-sdk-web": "1.0.25",
    "hoofd": "1.7.3",
-    "i18next": "24.2.2",
+    "i18next": "24.2.3",
    "i18next-browser-languagedetector": "8.0.4",
    "i18next-parser": "9.3.0",
    "i18next-resources-to-backend": "1.2.1",
-    "livekit-client": "2.9.5",
-    "posthog-js": "1.225.1",
+    "livekit-client": "2.9.8",
+    "posthog-js": "1.232.6",
    "react": "18.3.1",
-    "react-aria-components": "1.6.0",
+    "react-aria-components": "1.7.1",
    "react-dom": "18.3.1",
    "react-i18next": "15.1.1",
    "use-sound": "5.0.0",
-    "valtio": "2.1.3",
+    "valtio": "2.1.4",
    "wouter": "3.6.0"
  },
  "devDependencies": {
-    "@pandacss/dev": "0.53.0",
-    "@tanstack/eslint-plugin-query": "5.66.1",
-    "@tanstack/react-query-devtools": "5.67.1",
-    "@types/node": "22.13.9",
+    "@pandacss/dev": "0.53.3",
+    "@tanstack/eslint-plugin-query": "5.68.0",
+    "@tanstack/react-query-devtools": "5.69.0",
+    "@types/node": "22.13.13",
    "@types/react": "18.3.12",
    "@types/react-dom": "18.3.1",
-    "@typescript-eslint/eslint-plugin": "8.26.0",
-    "@typescript-eslint/parser": "8.26.0",
+    "@typescript-eslint/eslint-plugin": "8.28.0",
+    "@typescript-eslint/parser": "8.28.0",
    "@vitejs/plugin-react": "4.3.4",
    "eslint": "8.57.0",
-    "eslint-config-prettier": "10.0.2",
+    "eslint-config-prettier": "10.1.1",
    "eslint-plugin-jsx-a11y": "6.10.2",
    "eslint-plugin-react-hooks": "5.2.0",
    "eslint-plugin-react-refresh": "0.4.19",
    "postcss": "8.5.3",
    "prettier": "3.5.3",
    "typescript": "5.8.2",
-    "vite": "6.2.0",
+    "vite": "6.2.3",
    "vite-tsconfig-paths": "5.1.4"
  }
 }
@@ -35,40 +35,7 @@ const config: Config = {
  exclude: [],
  jsxFramework: 'react',
  outdir: 'src/styled-system',
-  globalFontface: {
-    Marianne: [
-      {
-        src: 'url(/fonts/Marianne-Regular-subset.woff2) format("woff2")',
-        fontWeight: 400,
-        fontStyle: 'normal',
-        fontDisplay: 'swap',
-      },
-      {
-        path: 'url(/fonts/Marianne-Regular_Italic-subset.woff2) format("woff2")',
-        fontWeight: 400,
-        fontStyle: 'italic',
-        fontDisplay: 'swap',
-      },
-      {
-        path: 'url(/fonts/Marianne-Medium-subset.woff2) format("woff2")',
-        fontWeight: 500,
-        fontStyle: 'normal',
-        fontDisplay: 'swap',
-      },
-      {
-        path: 'url(/fonts/Marianne-Bold-subset.woff2) format("woff2")',
-        fontWeight: 700,
-        fontStyle: 'normal',
-        fontDisplay: 'swap',
-      },
-      {
-        path: 'url(/fonts/Marianne-ExtraBold-subset.woff2) format("woff2")',
-        fontWeight: 800,
-        fontStyle: 'normal',
-        fontDisplay: 'swap',
-      },
-    ],
-  },
+  globalFontface: {},
  theme: {
    ...pandaPreset.theme,
    // media queries are defined in em so that zooming with text-only mode triggers breakpoints
@@ -48,7 +48,7 @@ export const useUser = (
  const isLoggedOut = isLoggedIn === false

  return {
-    ...query,
+    refetch: query.refetch,
    user: isLoggedOut ? undefined : (query.data as ApiUser | undefined),
    isLoggedIn,
    logout,
@@ -18,9 +18,9 @@ const Heading = styled('h2', {
    width: 'fit-content',
    marginBottom: 0,
    fontSize: '1.3rem',
-    fontWeight: '600',
+    fontWeight: '700',
    marginTop: '0.75rem',
-    lineHeight: '2rem',
+    lineHeight: '1.7rem',
    maxWidth: '23rem',
    textAlign: 'center',
    textWrap: 'balance',
@@ -120,7 +120,7 @@ const Separator = styled('div', {

 const Heading = styled('h1', {
  base: {
-    fontWeight: '600',
+    fontWeight: '700',
    fontStyle: 'normal',
    fontStretch: 'normal',
    fontOpticalSizing: 'auto',
@@ -298,7 +298,7 @@ export const Join = ({
      case ApiLobbyStatus.TIMEOUT:
        return (
          <VStack alignItems="center" textAlign="center">
-            <H lvl={1} margin={false}>
+            <H lvl={1} margin={false} centered>
              {t('timeoutInvite.title')}
            </H>
            <Text as="p" variant="note">
@@ -310,7 +310,7 @@ export const Join = ({
      case ApiLobbyStatus.DENIED:
        return (
          <VStack alignItems="center" textAlign="center">
-            <H lvl={1} margin={false}>
+            <H lvl={1} margin={false} centered>
              {t('denied.title')}
            </H>
            <Text as="p" variant="note">
@@ -322,7 +322,7 @@ export const Join = ({
      case ApiLobbyStatus.WAITING:
        return (
          <VStack alignItems="center" textAlign="center">
-            <H lvl={1} margin={false}>
+            <H lvl={1} margin={false} centered>
              {t('waiting.title')}
            </H>
            <Text
@@ -346,7 +346,7 @@ export const Join = ({
            }}
          >
            <VStack marginBottom={1}>
-              <H lvl={1} margin="sm">
+              <H lvl={1} margin="sm" centered>
                {t('heading')}
              </H>
              <Field
@@ -37,7 +37,14 @@ export const Admin = () => {
      flexDirection="column"
      alignItems="start"
    >
-      <Text variant="note" wrap="pretty" margin="md">
+      <Text
+        variant="note"
+        wrap="pretty"
+        className={css({
+          textStyle: 'sm',
+        })}
+        margin={'md'}
+      >
        {t('description')}
      </Text>
      <RACSeparator
@@ -68,8 +75,8 @@ export const Admin = () => {
      </Text>
      <Field
        type="radioGroup"
-        label="Type d'accès à la réunion"
-        aria-label="Type d'accès à la réunion"
+        label={t('access.type')}
+        aria-label={t('access.type')}
        labelProps={{
          className: css({
            fontSize: '1rem',
@@ -75,7 +75,7 @@ export function FloatingReaction({
    >
      <span
        className={css({
-          lineHeight: '45px',
+          lineHeight: '57px',
        })}
      >
        {emoji}
@@ -89,9 +89,10 @@ export function FloatingReaction({
            textAlign: 'center',
            borderRadius: '20px',
            paddingX: '0.5rem',
-            paddingY: '0.15rem',
+            paddingBottom: '0.3125rem',
+            paddingTop: '0.15rem',
            boxShadow: '0 2px 4px rgba(0, 0, 0, 0.1)',
-            lineHeight: '14px',
+            lineHeight: '16px',
          })}
        >
          {name}
@@ -17,6 +17,9 @@ import {
  ProcessorType,
 } from '.'

+import { DrawingUtils } from './DrawingUtils'
+import { DrawingUtils2 } from '@/features/rooms/livekit/components/blur/DrawingUtils2.ts'
+
 const PROCESSING_WIDTH = 256
 const PROCESSING_HEIGHT = 144

@@ -25,6 +28,8 @@ const BLUR_CANVAS_ID = 'background-blur-local'

 const DEFAULT_BLUR = '10'

+export class Wip {}
+
 /**
 * This implementation of video blurring is made to be run on CPU for browser that are
 * not compatible with track-processor-js.
@@ -50,8 +55,11 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
  imageSegmenterResult?: ImageSegmenterResult

  // Canvas used for resizing video source and projecting mask.
-  segmentationMaskCanvas?: HTMLCanvasElement
-  segmentationMaskCanvasCtx?: CanvasRenderingContext2D
+  workCanvas?: HTMLCanvasElement
+  // segmentationMaskCanvasCtx?: CanvasRenderingContext2D
+  drawingUtils: any
+  segmentationMaskCanvasCtx: any
+  segmentationMaskCanvas: any

  // Mask containg the inference result.
  segmentationMask?: ImageData
@@ -90,10 +98,11 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {

    this._initVirtualBackgroundImage()
    this._createMainCanvas()
-    this._createMaskCanvas()
+    this._createWorkCanvas()

    const stream = this.outputCanvas!.captureStream()
    const tracks = stream.getVideoTracks()
+
    if (tracks.length == 0) {
      throw new Error('No tracks found for processing')
    }
@@ -159,38 +168,16 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
      baseOptions: {
        modelAssetPath:
          'https://storage.googleapis.com/mediapipe-models/image_segmenter/selfie_segmenter_landscape/float16/latest/selfie_segmenter_landscape.tflite',
-        delegate: 'CPU', // Use CPU for Firefox.
+        // 'https://storage.googleapis.com/mediapipe-models/image_segmenter/deeplab_v3/float32/1/deeplab_v3.tflite',
+        delegate: 'GPU', // Use CPU for Firefox.
      },
      runningMode: 'VIDEO',
      outputCategoryMask: true,
      outputConfidenceMasks: false,
+      canvas: this.workCanvas,
    })
  }

-  /**
-   * Resize the source video to the processing resolution.
-   */
-  async sizeSource() {
-    this.segmentationMaskCanvasCtx?.drawImage(
-      this.videoElement!,
-      0,
-      0,
-      this.videoElement!.videoWidth,
-      this.videoElement!.videoHeight,
-      0,
-      0,
-      PROCESSING_WIDTH,
-      PROCESSING_HEIGHT
-    )
-
-    this.sourceImageData = this.segmentationMaskCanvasCtx?.getImageData(
-      0,
-      0,
-      PROCESSING_WIDTH,
-      PROCESSING_WIDTH
-    )
-  }
-
  /**
   * Run the segmentation.
   */
@@ -198,7 +185,7 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
    const startTimeMs = performance.now()
    return new Promise<void>((resolve) => {
      this.imageSegmenter!.segmentForVideo(
-        this.sourceImageData!,
+        this.videoElement,
        startTimeMs,
        (result: ImageSegmenterResult) => {
          this.imageSegmenterResult = result
@@ -212,92 +199,26 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
   * TODO: future improvement with WebGL.
   */
  async blur() {
-    const mask = this.imageSegmenterResult!.categoryMask!.getAsUint8Array()
-    for (let i = 0; i < mask.length; ++i) {
-      this.segmentationMask!.data[i * 4 + 3] = 255 - mask[i]
-    }
+    // console.log('$$ blur')

-    this.segmentationMaskCanvasCtx!.putImageData(this.segmentationMask!, 0, 0)
+    const legendColor = [[255, 197, 0, 255]]

-    this.outputCanvasCtx!.globalCompositeOperation = 'copy'
-    this.outputCanvasCtx!.filter = 'blur(8px)'
-
-    // Put opacity mask.
-    this.outputCanvasCtx!.drawImage(
-      this.segmentationMaskCanvas!,
-      0,
-      0,
-      PROCESSING_WIDTH,
-      PROCESSING_HEIGHT,
-      0,
-      0,
-      this.videoElement!.videoWidth,
-      this.videoElement!.videoHeight
-    )
-
-    // Draw clear body.
-    this.outputCanvasCtx!.globalCompositeOperation = 'source-in'
-    this.outputCanvasCtx!.filter = 'none'
-    this.outputCanvasCtx!.drawImage(this.videoElement!, 0, 0)
-
-    // Draw blurry background.
-    this.outputCanvasCtx!.globalCompositeOperation = 'destination-over'
-    this.outputCanvasCtx!.filter = `blur(${this.options.blurRadius ?? DEFAULT_BLUR}px)`
-    this.outputCanvasCtx!.drawImage(this.videoElement!, 0, 0)
-  }
-
-  /**
-   * TODO: future improvement with WebGL.
-   */
-  async drawVirtualBackground() {
-    const mask = this.imageSegmenterResult!.categoryMask!.getAsUint8Array()
-    for (let i = 0; i < mask.length; ++i) {
-      this.segmentationMask!.data[i * 4 + 3] = 255 - mask[i]
-    }
-
-    this.segmentationMaskCanvasCtx!.putImageData(this.segmentationMask!, 0, 0)
-
-    this.outputCanvasCtx!.globalCompositeOperation = 'copy'
-    this.outputCanvasCtx!.filter = 'blur(8px)'
-
-    // Put opacity mask.
-    this.outputCanvasCtx!.drawImage(
-      this.segmentationMaskCanvas!,
-      0,
-      0,
-      PROCESSING_WIDTH,
-      PROCESSING_HEIGHT,
-      0,
-      0,
-      this.videoElement!.videoWidth,
-      this.videoElement!.videoHeight
-    )
-
-    // Draw clear body.
-    this.outputCanvasCtx!.globalCompositeOperation = 'source-in'
-    this.outputCanvasCtx!.filter = 'none'
-    this.outputCanvasCtx!.drawImage(this.videoElement!, 0, 0)
-
-    // Draw virtual background.
-    this.outputCanvasCtx!.globalCompositeOperation = 'destination-over'
-    this.outputCanvasCtx!.drawImage(
-      this.virtualBackgroundImage!,
-      0,
-      0,
-      this.outputCanvas!.width,
-      this.outputCanvas!.height
+    this.drawingUtils.applyBackgroundBlur(
+      this.imageSegmenterResult.categoryMask,
+      this.videoElement,
+      3.0
    )
  }

  async process() {
-    await this.sizeSource()
    await this.segment()

    if (this.options.blurRadius) {
      await this.blur()
-    } else {
-      await this.drawVirtualBackground()
    }
+    // } else {
+    //   await this.drawVirtualBackground()
+    // }
    this.timerWorker!.postMessage({
      id: SET_TIMEOUT,
      timeMs: 1000 / 30,
@@ -318,19 +239,33 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
    this.outputCanvasCtx = this.outputCanvas.getContext('2d')!
  }

-  _createMaskCanvas() {
-    this.segmentationMaskCanvas = document.querySelector(
-      `#${SEGMENTATION_MASK_CANVAS_ID}`
-    ) as HTMLCanvasElement
-    if (!this.segmentationMaskCanvas) {
-      this.segmentationMaskCanvas = this._createCanvas(
-        SEGMENTATION_MASK_CANVAS_ID,
+  _createWorkCanvas() {
+    this.workCanvas = document.querySelector(`#workcanvas`)
+    if (!this.workCanvas) {
+      this.workCanvas = this._wip(
+        'workcanvas',
        PROCESSING_WIDTH,
        PROCESSING_HEIGHT
      )
    }
-    this.segmentationMaskCanvasCtx =
-      this.segmentationMaskCanvas.getContext('2d')!
+    this.drawingUtils = new DrawingUtils2(this.workCanvas.getContext('webgl2'))
+  }
+
+  _wip(id: string, width: number, height: number) {
+    const wip = document.querySelector(`#wip`)
+    const element = document.createElement('canvas')
+    element.setAttribute('id', id)
+    element.setAttribute('width', '' + width)
+    element.setAttribute('height', '' + height)
+    element.style.position = 'absolute'
+    element.style.top = '0'
+    element.style.left = '0'
+    element.style.maxWidth = '611px'
+    element.style.maxHeight = '305px'
+
+    wip.appendChild(element)
+
+    return element
  }

  _createCanvas(id: string, width: number, height: number) {
@@ -0,0 +1,158 @@
+import { MPMask } from '@mediapipe/tasks-vision'
+import { CategoryMaskShaderContext } from './wip'
+
+// Type definitions
+type RGBAColor = [number, number, number, number]
+type ImageSource =
+  | HTMLCanvasElement
+  | HTMLImageElement
+  | HTMLVideoElement
+  | ImageData
+
+type CategoryToColorMap = Map<number, RGBAColor> | RGBAColor[]
+
+export class DrawingUtils {
+  private categoryMaskShaderContext?: CategoryMaskShaderContext
+  private readonly contextWebGL?: WebGL2RenderingContext
+
+  /**
+   * Creates a new DrawingUtils class.
+   *
+   * @param gpuContext The WebGL canvas rendering context to render into. If
+   *     your Task is using a GPU delegate, the context must be obtained from
+   * its canvas (provided via `setOptions({ canvas: .. })`).
+   */
+  constructor(gpuContext: WebGL2RenderingContext)
+  /**
+   * Creates a new DrawingUtils class.
+   *
+   * @param cpuContext The 2D canvas rendering context to render into. If
+   *     you are rendering GPU data you must also provide `gpuContext` to allow
+   *     for data conversion.
+   * @param gpuContext A WebGL canvas that is used for GPU rendering and for
+   *     converting GPU to CPU data. If your Task is using a GPU delegate, the
+   *     context must be obtained from  its canvas (provided via
+   *     `setOptions({ canvas: .. })`).
+   */
+  constructor(
+    cpuContext: CanvasRenderingContext2D | OffscreenCanvasRenderingContext2D,
+    gpuContext?: WebGL2RenderingContext
+  )
+  constructor(
+    cpuOrGpuGontext:
+      | CanvasRenderingContext2D
+      | OffscreenCanvasRenderingContext2D
+      | WebGL2RenderingContext,
+    gpuContext?: WebGL2RenderingContext
+  ) {
+    if (
+      (typeof CanvasRenderingContext2D !== 'undefined' &&
+        cpuOrGpuGontext instanceof CanvasRenderingContext2D) ||
+      cpuOrGpuGontext instanceof OffscreenCanvasRenderingContext2D
+    ) {
+      // this.context2d = cpuOrGpuGontext
+      this.contextWebGL = gpuContext
+    } else {
+      // If the first `if` statement is false, then the first argument must be a
+      // WebGL2RenderingContext, since CanvasRenderingContext2D can't be passed
+      // as the first argument. However, typescript isn't smart enough to infer
+      // this so we cast.
+      this.contextWebGL = cpuOrGpuGontext as WebGL2RenderingContext
+    }
+  }
+
+  private getWebGLRenderingContext(): WebGL2RenderingContext {
+    if (!this.contextWebGL) {
+      throw new Error(
+        'GPU rendering requested but WebGL2RenderingContext not provided.'
+      )
+    }
+    return this.contextWebGL
+  }
+
+  private getCategoryMaskShaderContext(): CategoryMaskShaderContext {
+    if (!this.categoryMaskShaderContext) {
+      this.categoryMaskShaderContext = new CategoryMaskShaderContext()
+    }
+    return this.categoryMaskShaderContext
+  }
+
+  /** Draws a category mask on a WebGL2RenderingContext2D. */
+  private drawCategoryMaskWebGL(
+    categoryTexture: WebGLTexture,
+    background: RGBAColor | ImageSource,
+    categoryToColorMap: Map<number, RGBAColor> | RGBAColor[]
+  ): void {
+    const shaderContext = this.getCategoryMaskShaderContext()
+    const gl = this.getWebGLRenderingContext()
+
+    const backgroundImage = Array.isArray(background)
+      ? new ImageData(new Uint8ClampedArray(background), 1, 1)
+      : background
+
+    shaderContext.run(gl, /* flipTexturesVertically= */ true, () => {
+      shaderContext.bindAndUploadTextures(
+        categoryTexture,
+        backgroundImage,
+        categoryToColorMap
+      )
+      gl.clearColor(0, 0, 0, 0)
+      gl.clear(gl.COLOR_BUFFER_BIT)
+      gl.drawArrays(gl.TRIANGLE_FAN, 0, 4)
+      shaderContext.unbindTextures()
+    })
+  }
+
+  /**
+   * Draws a category mask using the provided category-to-color mapping.
+   *
+   * @export
+   * @param mask A category mask that was returned from a segmentation task.
+   * @param categoryToColorMap A map that maps category indices to RGBA
+   *     values. You must specify a map entry for each category.
+   * @param background A color or image to use as the background. Defaults to
+   *     black.
+   */
+  drawCategoryMask(
+    mask: MPMask,
+    categoryToColorMap: Map<number, RGBAColor>,
+    background?: RGBAColor | ImageSource
+  ): void
+  /**
+   * Draws a category mask using the provided color array.
+   *
+   * @export
+   * @param mask A category mask that was returned from a segmentation task.
+   * @param categoryToColorMap An array that maps indices to RGBA values. The
+   *     array's indices must correspond to the category indices of the model
+   *     and an entry must be provided for each category.
+   * @param background A color or image to use as the background. Defaults to
+   *     black.
+   */
+  drawCategoryMask(
+    mask: MPMask,
+    categoryToColorMap: RGBAColor[],
+    background?: RGBAColor | ImageSource
+  ): void
+  /** @export */
+  drawCategoryMask(
+    mask: MPMask,
+    categoryToColorMap: CategoryToColorMap,
+    background: RGBAColor | ImageSource = [0, 0, 0, 255]
+  ): void {
+    this.drawCategoryMaskWebGL(
+      mask.getAsWebGLTexture(),
+      background,
+      categoryToColorMap
+    )
+  }
+
+  /**
+   * Frees all WebGL resources held by this class.
+   * @export
+   */
+  close(): void {
+    this.categoryMaskShaderContext?.close()
+    this.categoryMaskShaderContext = undefined
+  }
+}
@@ -0,0 +1,120 @@
+import { MPMask } from '@mediapipe/tasks-vision'
+import { BackgroundBlurShaderContext } from './wip2.ts'
+
+// Type definitions
+type ImageSource =
+  | HTMLCanvasElement
+  | HTMLImageElement
+  | HTMLVideoElement
+  | ImageData
+
+export class DrawingUtils2 {
+  private backgroundBlurShaderContext?: BackgroundBlurShaderContext
+  private readonly contextWebGL?: WebGL2RenderingContext
+
+  /**
+   * Creates a new DrawingUtils class.
+   *
+   * @param gpuContext The WebGL canvas rendering context to render into. If
+   *     your Task is using a GPU delegate, the context must be obtained from
+   * its canvas (provided via `setOptions({ canvas: .. })`).
+   */
+  constructor(gpuContext: WebGL2RenderingContext)
+  /**
+   * Creates a new DrawingUtils class.
+   *
+   * @param cpuContext The 2D canvas rendering context to render into. If
+   *     you are rendering GPU data you must also provide `gpuContext` to allow
+   *     for data conversion.
+   * @param gpuContext A WebGL canvas that is used for GPU rendering and for
+   *     converting GPU to CPU data. If your Task is using a GPU delegate, the
+   *     context must be obtained from  its canvas (provided via
+   *     `setOptions({ canvas: .. })`).
+   */
+  constructor(
+    cpuContext: CanvasRenderingContext2D | OffscreenCanvasRenderingContext2D,
+    gpuContext?: WebGL2RenderingContext
+  )
+  constructor(
+    cpuOrGpuContext:
+      | CanvasRenderingContext2D
+      | OffscreenCanvasRenderingContext2D
+      | WebGL2RenderingContext,
+    gpuContext?: WebGL2RenderingContext
+  ) {
+    if (
+      (typeof CanvasRenderingContext2D !== 'undefined' &&
+        cpuOrGpuContext instanceof CanvasRenderingContext2D) ||
+      cpuOrGpuContext instanceof OffscreenCanvasRenderingContext2D
+    ) {
+      this.contextWebGL = gpuContext
+    } else {
+      // If the first condition is false, then the first argument must be a
+      // WebGL2RenderingContext
+      this.contextWebGL = cpuOrGpuContext as WebGL2RenderingContext
+    }
+  }
+
+  private getWebGLRenderingContext(): WebGL2RenderingContext {
+    if (!this.contextWebGL) {
+      throw new Error(
+        'GPU rendering requested but WebGL2RenderingContext not provided.'
+      )
+    }
+    return this.contextWebGL
+  }
+
+  private getBackgroundBlurShaderContext(): BackgroundBlurShaderContext {
+    if (!this.backgroundBlurShaderContext) {
+      this.backgroundBlurShaderContext = new BackgroundBlurShaderContext()
+    }
+    return this.backgroundBlurShaderContext
+  }
+
+  /** Applies background blur effect using WebGL2. */
+  private applyBackgroundBlurWebGL(
+    maskTexture: WebGLTexture,
+    background: ImageSource,
+    blurAmount: number = 1.0
+  ): void {
+    const shaderContext = this.getBackgroundBlurShaderContext()
+    const gl = this.getWebGLRenderingContext()
+
+    shaderContext.run(gl, /* flipTexturesVertically= */ true, () => {
+      shaderContext.bindAndUploadTextures(maskTexture, background, blurAmount)
+      gl.clearColor(0, 0, 0, 0)
+      gl.clear(gl.COLOR_BUFFER_BIT)
+      gl.drawArrays(gl.TRIANGLE_FAN, 0, 4)
+      shaderContext.unbindTextures()
+    })
+  }
+
+  /**
+   * Applies a background blur effect using the mask to preserve the foreground.
+   *
+   * @export
+   * @param mask A binary mask that was returned from a segmentation task (1 for foreground, 0 for background).
+   * @param background The input image to blur the background of.
+   * @param blurAmount Optional blur amount parameter (defaults to 1.0, higher values create stronger blur).
+   */
+  applyBackgroundBlur(
+    mask: MPMask,
+    background: ImageSource,
+    blurAmount: number = 1.0
+  ): void {
+    this.applyBackgroundBlurWebGL(
+      mask.getAsWebGLTexture(),
+      background,
+      blurAmount
+    )
+  }
+
+  /**
+   * Frees all WebGL resources held by this class.
+   * @export
+   */
+  close(): void {
+    this.backgroundBlurShaderContext?.close()
+    this.backgroundBlurShaderContext = undefined
+  }
+}
@@ -0,0 +1,324 @@
+/**
+ * Copyright 2023 The MediaPipe Authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+const VERTEX_SHADER = `#version 300 es
+  in vec2 aVertex;
+  in vec2 aTex;
+  out vec2 vTex;
+  void main(void) {
+    gl_Position = vec4(aVertex, 0.0, 1.0);
+    vTex = aTex;
+  }`
+
+const FRAGMENT_SHADER = `#version 300 es
+  precision mediump float;
+  in vec2 vTex;
+  uniform sampler2D inputTexture;
+  out vec4 fragColor;
+  void main() {
+    fragColor = texture(inputTexture, vTex);
+  }
+ `
+
+/** Helper to assert that `value` is not null or undefined.  */
+export function assertExists<T>(value: T, msg: string): NonNullable<T> {
+  if (!value) {
+    throw new Error(`Unable to obtain required WebGL resource: ${msg}`)
+  }
+  return value
+}
+
+/**
+ * Utility class that encapsulates the buffers used by `MPImageShaderContext`.
+ * For internal use only.
+ */
+class MPImageShaderBuffers {
+  constructor(
+    private readonly gl: WebGL2RenderingContext,
+    private readonly vertexArrayObject: WebGLVertexArrayObject,
+    private readonly vertexBuffer: WebGLBuffer,
+    private readonly textureBuffer: WebGLBuffer
+  ) {}
+
+  bind() {
+    this.gl.bindVertexArray(this.vertexArrayObject)
+  }
+
+  unbind() {
+    this.gl.bindVertexArray(null)
+  }
+
+  close() {
+    this.gl.deleteVertexArray(this.vertexArrayObject)
+    this.gl.deleteBuffer(this.vertexBuffer)
+    this.gl.deleteBuffer(this.textureBuffer)
+  }
+}
+
+/**
+ * A class that encapsulates the shaders used by an MPImage. Can be re-used
+ * across MPImages that use the same WebGL2Rendering context.
+ *
+ * For internal use only.
+ */
+export class MPImageShaderContext {
+  protected gl?: WebGL2RenderingContext
+  private framebuffer?: WebGLFramebuffer
+  protected program?: WebGLProgram
+  private vertexShader?: WebGLShader
+  private fragmentShader?: WebGLShader
+  private aVertex?: GLint
+  private aTex?: GLint
+
+  /**
+   * The shader buffers used for passthrough renders that don't modify the
+   * input texture.
+   */
+  private shaderBuffersPassthrough?: MPImageShaderBuffers
+
+  /**
+   * The shader buffers used for passthrough renders that flip the input texture
+   * vertically before conversion to a different type. This is used to flip the
+   * texture to the expected orientation for drawing in the browser.
+   */
+  private shaderBuffersFlipVertically?: MPImageShaderBuffers
+
+  protected getFragmentShader(): string {
+    return FRAGMENT_SHADER
+  }
+
+  protected getVertexShader(): string {
+    return VERTEX_SHADER
+  }
+
+  private compileShader(source: string, type: number): WebGLShader {
+    const gl = this.gl!
+    const shader = assertExists(
+      gl.createShader(type),
+      'Failed to create WebGL shader'
+    )
+    gl.shaderSource(shader, source)
+    gl.compileShader(shader)
+    if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {
+      const info = gl.getShaderInfoLog(shader)
+      throw new Error(`Could not compile WebGL shader: ${info}`)
+    }
+    gl.attachShader(this.program!, shader)
+    return shader
+  }
+
+  protected setupShaders(): void {
+    const gl = this.gl!
+    this.program = assertExists(
+      gl.createProgram()!,
+      'Failed to create WebGL program'
+    )
+
+    this.vertexShader = this.compileShader(
+      this.getVertexShader(),
+      gl.VERTEX_SHADER
+    )
+    this.fragmentShader = this.compileShader(
+      this.getFragmentShader(),
+      gl.FRAGMENT_SHADER
+    )
+
+    gl.linkProgram(this.program)
+    const linked = gl.getProgramParameter(this.program, gl.LINK_STATUS)
+    if (!linked) {
+      const info = gl.getProgramInfoLog(this.program)
+      throw new Error(`Error during program linking: ${info}`)
+    }
+
+    this.aVertex = gl.getAttribLocation(this.program, 'aVertex')
+    this.aTex = gl.getAttribLocation(this.program, 'aTex')
+  }
+
+  protected setupTextures(): void {}
+
+  protected configureUniforms(): void {}
+
+  private createBuffers(flipVertically: boolean): MPImageShaderBuffers {
+    const gl = this.gl!
+    const vertexArrayObject = assertExists(
+      gl.createVertexArray(),
+      'Failed to create vertex array'
+    )
+    gl.bindVertexArray(vertexArrayObject)
+
+    const vertexBuffer = assertExists(
+      gl.createBuffer(),
+      'Failed to create buffer'
+    )
+    gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer)
+    gl.enableVertexAttribArray(this.aVertex!)
+    gl.vertexAttribPointer(this.aVertex!, 2, gl.FLOAT, false, 0, 0)
+    gl.bufferData(
+      gl.ARRAY_BUFFER,
+      new Float32Array([-1, -1, -1, 1, 1, 1, 1, -1]),
+      gl.STATIC_DRAW
+    )
+
+    const textureBuffer = assertExists(
+      gl.createBuffer(),
+      'Failed to create buffer'
+    )
+    gl.bindBuffer(gl.ARRAY_BUFFER, textureBuffer)
+    gl.enableVertexAttribArray(this.aTex!)
+    gl.vertexAttribPointer(this.aTex!, 2, gl.FLOAT, false, 0, 0)
+
+    const bufferData = flipVertically
+      ? [0, 1, 0, 0, 1, 0, 1, 1]
+      : [0, 0, 0, 1, 1, 1, 1, 0]
+    gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(bufferData), gl.STATIC_DRAW)
+
+    gl.bindBuffer(gl.ARRAY_BUFFER, null)
+    gl.bindVertexArray(null)
+
+    return new MPImageShaderBuffers(
+      gl,
+      vertexArrayObject,
+      vertexBuffer,
+      textureBuffer
+    )
+  }
+
+  private getShaderBuffers(flipVertically: boolean): MPImageShaderBuffers {
+    if (flipVertically) {
+      if (!this.shaderBuffersFlipVertically) {
+        this.shaderBuffersFlipVertically = this.createBuffers(
+          /* flipVertically= */ true
+        )
+      }
+      return this.shaderBuffersFlipVertically
+    } else {
+      if (!this.shaderBuffersPassthrough) {
+        this.shaderBuffersPassthrough = this.createBuffers(
+          /* flipVertically= */ false
+        )
+      }
+      return this.shaderBuffersPassthrough
+    }
+  }
+
+  private maybeInitGL(gl: WebGL2RenderingContext): void {
+    if (!this.gl) {
+      this.gl = gl
+    } else if (gl !== this.gl) {
+      throw new Error('Cannot change GL context once initialized')
+    }
+  }
+
+  /** Runs the callback using the shader. */
+  run<T>(
+    gl: WebGL2RenderingContext,
+    flipVertically: boolean,
+    callback: () => T
+  ): T {
+    this.maybeInitGL(gl)
+
+    if (!this.program) {
+      this.setupShaders()
+      this.setupTextures()
+    }
+
+    const shaderBuffers = this.getShaderBuffers(flipVertically)
+    gl.useProgram(this.program!)
+    shaderBuffers.bind()
+    this.configureUniforms()
+    const result = callback()
+    shaderBuffers.unbind()
+
+    return result
+  }
+
+  /**
+   * Creates and configures a texture.
+   *
+   * @param gl The rendering context.
+   * @param filter The setting to use for `gl.TEXTURE_MIN_FILTER` and
+   *     `gl.TEXTURE_MAG_FILTER`. Defaults to `gl.LINEAR`.
+   * @param wrapping The setting to use for `gl.TEXTURE_WRAP_S` and
+   *     `gl.TEXTURE_WRAP_T`. Defaults to `gl.CLAMP_TO_EDGE`.
+   */
+  createTexture(
+    gl: WebGL2RenderingContext,
+    filter?: GLenum,
+    wrapping?: GLenum
+  ): WebGLTexture {
+    this.maybeInitGL(gl)
+    const texture = assertExists(gl.createTexture(), 'Failed to create texture')
+    gl.bindTexture(gl.TEXTURE_2D, texture)
+    gl.texParameteri(
+      gl.TEXTURE_2D,
+      gl.TEXTURE_WRAP_S,
+      wrapping ?? gl.CLAMP_TO_EDGE
+    )
+    gl.texParameteri(
+      gl.TEXTURE_2D,
+      gl.TEXTURE_WRAP_T,
+      wrapping ?? gl.CLAMP_TO_EDGE
+    )
+    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, filter ?? gl.LINEAR)
+    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, filter ?? gl.LINEAR)
+    gl.bindTexture(gl.TEXTURE_2D, null)
+    return texture
+  }
+
+  /**
+   * Binds a framebuffer to the canvas. If the framebuffer does not yet exist,
+   * creates it first. Binds the provided texture to the framebuffer.
+   */
+  bindFramebuffer(gl: WebGL2RenderingContext, texture: WebGLTexture): void {
+    this.maybeInitGL(gl)
+    if (!this.framebuffer) {
+      this.framebuffer = assertExists(
+        gl.createFramebuffer(),
+        'Failed to create framebuffe.'
+      )
+    }
+    gl.bindFramebuffer(gl.FRAMEBUFFER, this.framebuffer)
+    gl.framebufferTexture2D(
+      gl.FRAMEBUFFER,
+      gl.COLOR_ATTACHMENT0,
+      gl.TEXTURE_2D,
+      texture,
+      0
+    )
+  }
+
+  unbindFramebuffer(): void {
+    this.gl?.bindFramebuffer(this.gl.FRAMEBUFFER, null)
+  }
+
+  close() {
+    if (this.program) {
+      const gl = this.gl!
+      gl.deleteProgram(this.program)
+      gl.deleteShader(this.vertexShader!)
+      gl.deleteShader(this.fragmentShader!)
+    }
+    if (this.framebuffer) {
+      this.gl!.deleteFramebuffer(this.framebuffer)
+    }
+    if (this.shaderBuffersPassthrough) {
+      this.shaderBuffersPassthrough.close()
+    }
+    if (this.shaderBuffersFlipVertically) {
+      this.shaderBuffersFlipVertically.close()
+    }
+  }
+}
@@ -36,22 +36,25 @@ export class BackgroundProcessorFactory {
    type: ProcessorType,
    opts: BackgroundOptions
  ): BackgroundProcessorInterface | undefined {
-    if (type === ProcessorType.BLUR) {
-      if (ProcessorWrapper.isSupported) {
-        return new BackgroundBlurTrackProcessorJsWrapper(opts)
-      }
-      if (BackgroundCustomProcessor.isSupported) {
-        return new BackgroundCustomProcessor(opts)
-      }
-    } else if (type === ProcessorType.VIRTUAL) {
-      if (ProcessorWrapper.isSupported) {
-        return new BackgroundVirtualTrackProcessorJsWrapper(opts)
-      }
-      if (BackgroundCustomProcessor.isSupported) {
-        return new BackgroundCustomProcessor(opts)
-      }
-    }
-    return undefined
+    return new BackgroundCustomProcessor(opts)
+
+    // if (type === ProcessorType.BLUR) {
+    //   if (ProcessorWrapper.isSupported) {
+    //
+    //   }
+    //   if (BackgroundCustomProcessor.isSupported) {
+    //     return new BackgroundCustomProcessor(opts)
+    //   }
+    // } else if (type === ProcessorType.VIRTUAL) {
+    //   if (ProcessorWrapper.isSupported) {
+    //     return new BackgroundVirtualTrackProcessorJsWrapper(opts)
+    //   }
+    //   if (BackgroundCustomProcessor.isSupported) {
+    //     return new BackgroundCustomProcessor(opts)
+    //   }
+    // }
+
+    // return undefined
  }

  static deserializeProcessor(data?: ProcessorSerialized) {
@@ -0,0 +1,217 @@
+/**
+ * Copyright 2023 The MediaPipe Authors.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { MPImageShaderContext, assertExists } from './image_shader_context'
+
+import { ImageSource } from '@mediapipe/tasks-vision'
+
+/**
+ * A fragment shader that maps categories to colors based on a background
+ * texture, a mask texture and a 256x1 "color mapping texture" that contains one
+ * color for each pixel.
+ */
+const FRAGMENT_SHADER = `
+  precision mediump float;
+  uniform sampler2D backgroundTexture;
+  uniform sampler2D maskTexture;
+  uniform sampler2D colorMappingTexture;
+  varying vec2 vTex;
+  void main() {
+    vec4 backgroundColor = texture2D(backgroundTexture, vTex);
+    float category = texture2D(maskTexture, vTex).r;
+    vec4 categoryColor = texture2D(colorMappingTexture, vec2(category, 0.0));
+    gl_FragColor = mix(backgroundColor, categoryColor, categoryColor.a);
+  }
+ `
+
+/**
+ * A four channel color with values for red, green, blue and alpha
+ * respectively.
+ */
+export type RGBAColor = [number, number, number, number] | number[]
+
+/**
+ * A category to color mapping that uses either a map or an array to assign
+ * category indexes to RGBA colors.
+ */
+export type CategoryToColorMap = Map<number, RGBAColor> | RGBAColor[]
+
+/** Checks CategoryToColorMap maps for deep equality. */
+function isEqualColorMap(
+  a: CategoryToColorMap,
+  b: CategoryToColorMap
+): boolean {
+  if (a !== b) {
+    return false
+  }
+
+  const aEntries = a.entries()
+  const bEntries = b.entries()
+  for (const [aKey, aValue] of aEntries) {
+    const bNext = bEntries.next()
+    if (bNext.done) {
+      return false
+    }
+
+    const [bKey, bValue] = bNext.value
+    if (aKey !== bKey) {
+      return false
+    }
+
+    if (
+      aValue[0] !== bValue[0] ||
+      aValue[1] !== bValue[1] ||
+      aValue[2] !== bValue[2] ||
+      aValue[3] !== bValue[3]
+    ) {
+      return false
+    }
+  }
+  return !!bEntries.next().done
+}
+
+/** A drawing util class for category masks. */
+export class CategoryMaskShaderContext extends MPImageShaderContext {
+  backgroundTexture?: WebGLTexture
+  colorMappingTexture?: WebGLTexture
+  colorMappingTextureUniform?: WebGLUniformLocation
+  backgroundTextureUniform?: WebGLUniformLocation
+  maskTextureUniform?: WebGLUniformLocation
+  currentColorMap?: CategoryToColorMap
+
+  bindAndUploadTextures(
+    categoryMask: WebGLTexture,
+    background: ImageSource,
+    colorMap: Map<number, number[]> | number[][]
+  ) {
+    const gl = this.gl!
+
+    // Bind category mask
+    gl.activeTexture(gl.TEXTURE0)
+    gl.bindTexture(gl.TEXTURE_2D, categoryMask)
+
+    // TODO: We should avoid uploading textures from CPU to GPU
+    // if the textures haven't changed. This can lead to drastic performance
+    // slowdowns (~50ms per frame). Users can reduce the penalty by passing a
+    // canvas object instead of ImageData/HTMLImageElement.
+    gl.activeTexture(gl.TEXTURE1)
+    gl.bindTexture(gl.TEXTURE_2D, this.backgroundTexture!)
+    gl.texImage2D(
+      gl.TEXTURE_2D,
+      0,
+      gl.RGBA,
+      gl.RGBA,
+      gl.UNSIGNED_BYTE,
+      background
+    )
+
+    // Bind color mapping texture if changed.
+    if (
+      !this.currentColorMap ||
+      !isEqualColorMap(this.currentColorMap, colorMap)
+    ) {
+      this.currentColorMap = colorMap
+
+      const pixels = new Array(256 * 4).fill(0)
+      colorMap.forEach((rgba, index) => {
+        if (rgba.length !== 4) {
+          throw new Error(
+            `Color at index ${index} is not a four-channel value.`
+          )
+        }
+        pixels[index * 4] = rgba[0]
+        pixels[index * 4 + 1] = rgba[1]
+        pixels[index * 4 + 2] = rgba[2]
+        pixels[index * 4 + 3] = rgba[3]
+      })
+      gl.activeTexture(gl.TEXTURE2)
+      gl.bindTexture(gl.TEXTURE_2D, this.colorMappingTexture!)
+      gl.texImage2D(
+        gl.TEXTURE_2D,
+        0,
+        gl.RGBA,
+        256,
+        1,
+        0,
+        gl.RGBA,
+        gl.UNSIGNED_BYTE,
+        new Uint8Array(pixels)
+      )
+    } else {
+      gl.activeTexture(gl.TEXTURE2)
+      gl.bindTexture(gl.TEXTURE_2D, this.colorMappingTexture!)
+    }
+  }
+
+  unbindTextures() {
+    const gl = this.gl!
+    gl.activeTexture(gl.TEXTURE0)
+    gl.bindTexture(gl.TEXTURE_2D, null)
+    gl.activeTexture(gl.TEXTURE1)
+    gl.bindTexture(gl.TEXTURE_2D, null)
+    gl.activeTexture(gl.TEXTURE2)
+    gl.bindTexture(gl.TEXTURE_2D, null)
+  }
+
+  protected override getFragmentShader(): string {
+    return FRAGMENT_SHADER
+  }
+
+  protected override setupTextures(): void {
+    const gl = this.gl!
+    gl.activeTexture(gl.TEXTURE1)
+    this.backgroundTexture = this.createTexture(gl, gl.LINEAR)
+    // Use `gl.NEAREST` to prevent interpolating values in our category to
+    // color map.
+    gl.activeTexture(gl.TEXTURE2)
+    this.colorMappingTexture = this.createTexture(gl, gl.NEAREST)
+  }
+
+  protected override setupShaders(): void {
+    super.setupShaders()
+    const gl = this.gl!
+    this.backgroundTextureUniform = assertExists(
+      gl.getUniformLocation(this.program!, 'backgroundTexture'),
+      'Uniform location'
+    )
+    this.colorMappingTextureUniform = assertExists(
+      gl.getUniformLocation(this.program!, 'colorMappingTexture'),
+      'Uniform location'
+    )
+    this.maskTextureUniform = assertExists(
+      gl.getUniformLocation(this.program!, 'maskTexture'),
+      'Uniform location'
+    )
+  }
+
+  protected override configureUniforms(): void {
+    super.configureUniforms()
+    const gl = this.gl!
+    gl.uniform1i(this.maskTextureUniform!, 0)
+    gl.uniform1i(this.backgroundTextureUniform!, 1)
+    gl.uniform1i(this.colorMappingTextureUniform!, 2)
+  }
+
+  override close(): void {
+    if (this.backgroundTexture) {
+      this.gl!.deleteTexture(this.backgroundTexture)
+    }
+    if (this.colorMappingTexture) {
+      this.gl!.deleteTexture(this.colorMappingTexture)
+    }
+    super.close()
+  }
+}
@@ -0,0 +1,202 @@
+/**
+ * Background Blur Shader Implementation
+ *
+ * Implements a high-quality background blur while preserving
+ * foreground based on a mask. Uses a multi-pass blur approach
+ * for a smoother, more professional-looking effect.
+ *
+ * This implementation INVERTS the mask to blur what's outside the mask.
+ */
+
+import { MPImageShaderContext, assertExists } from './image_shader_context'
+import { ImageSource } from '@mediapipe/tasks-vision'
+
+/**
+ * Two-pass blur with smooth transition between foreground and background.
+ * Uses a larger kernel and multiple passes for a higher quality blur.
+ *
+ * This version inverts the mask interpretation - it blurs what's OUTSIDE the mask.
+ */
+
+const FRAGMENT_SHADER = `#version 300 es
+  precision mediump float;
+  
+  uniform sampler2D backgroundTexture;
+  uniform sampler2D maskTexture;
+  uniform vec2 texelSize;
+  uniform float blurAmount;
+  
+  in vec2 vTex;
+  out vec4 fragColor;
+
+  // Smoothing function for mask transitions
+  vec4 maskSmoothing() {
+    // Sample the mask at the current texel
+    float maskValue = texture(maskTexture, vTex).r;
+    
+    // Apply smoothing to reduce aliasing at mask edges
+    float smoothedMask = smoothstep(0.2, 0.8, maskValue);
+    
+    return vec4(smoothedMask);
+  }
+  
+  // Compute Gaussian weight based on distance and blur amount
+  float gaussianWeight(float x, float y, float sigma) {
+    float sigmaSq = sigma * sigma;
+    float distanceSq = x * x + y * y;
+    
+    // Gaussian function: (1/(2πσ²)) * e^(-((x²+y²)/(2σ²)))
+    return exp(-distanceSq / (2.0 * sigmaSq));
+  }
+
+  void main() {
+    vec4 centerColor = texture(backgroundTexture, vTex);
+    
+    // Apply bilateral filtering on the mask before using it
+    vec4 personMask = maskSmoothing();
+    
+    // Dynamic 5x5 Gaussian blur
+    vec4 blurredColor = vec4(0.0);
+    float totalWeight = 0.0;
+    
+    // Compute effective sigma from blur amount (adjust as needed)
+    float sigma = max(0.1, blurAmount) + 50.0;
+    
+    // Apply 5x5 kernel with dynamic weights
+    for (int y = -10; y <= 10; y++) {
+      for (int x = -10; x <= 10; x++) {
+        // Calculate Gaussian weight for this offset
+        float weight = gaussianWeight(float(x), float(y), sigma);
+        totalWeight += weight;
+        
+        vec2 offset = vec2(float(x), float(y)) * texelSize;
+        vec2 sampleCoord = vTex + offset;
+        blurredColor += texture(backgroundTexture, sampleCoord) * weight;
+      }
+    }
+    
+    // Normalize by total weight
+    blurredColor /= totalWeight;
+    
+    // Blend original color with blurred background based on mask
+    fragColor = mix(centerColor, blurredColor, personMask.r);
+  }
+`;
+
+
+
+/** A drawing util class for high-quality background blur. */
+export class BackgroundBlurShaderContext extends MPImageShaderContext {
+  backgroundTexture?: WebGLTexture
+  maskTextureUniform?: WebGLUniformLocation
+  backgroundTextureUniform?: WebGLUniformLocation
+  texelSizeUniform?: WebGLUniformLocation
+  blurAmountUniform?: WebGLUniformLocation
+
+  bindAndUploadTextures(
+    foregroundMask: WebGLTexture,
+    background: ImageSource,
+    blurAmount: number = 1.0
+  ) {
+    const gl = this.gl!
+
+    // Bind foreground mask
+    gl.activeTexture(gl.TEXTURE0)
+    gl.bindTexture(gl.TEXTURE_2D, foregroundMask)
+
+    // Bind background texture
+    gl.activeTexture(gl.TEXTURE1)
+    gl.bindTexture(gl.TEXTURE_2D, this.backgroundTexture!)
+    gl.texImage2D(
+      gl.TEXTURE_2D,
+      0,
+      gl.RGBA,
+      gl.RGBA,
+      gl.UNSIGNED_BYTE,
+      background
+    )
+
+    // Set texel size (for proper blur scaling)
+    let width = 1;
+    let height = 1;
+
+    if (background instanceof ImageData) {
+      width = background.width;
+      height = background.height;
+    } else if (background instanceof HTMLCanvasElement) {
+      width = background.width;
+      height = background.height;
+    } else if (background instanceof HTMLImageElement) {
+      width = background.naturalWidth;
+      height = background.naturalHeight;
+    } else if (background instanceof HTMLVideoElement) {
+      width = background.videoWidth;
+      height = background.videoHeight;
+    }
+
+    // Ensure we have valid dimensions to prevent division by zero
+    width = Math.max(width, 1);
+    height = Math.max(height, 1);
+
+    gl.uniform2f(this.texelSizeUniform!, 1.0 / width, 1.0 / height)
+
+    // Set blur amount
+    if (this.blurAmountUniform) {
+      gl.uniform1f(this.blurAmountUniform, blurAmount)
+    }
+  }
+
+  unbindTextures() {
+    const gl = this.gl!
+    gl.activeTexture(gl.TEXTURE0)
+    gl.bindTexture(gl.TEXTURE_2D, null)
+    gl.activeTexture(gl.TEXTURE1)
+    gl.bindTexture(gl.TEXTURE_2D, null)
+  }
+
+  protected override getFragmentShader(): string {
+    return FRAGMENT_SHADER
+  }
+
+  protected override setupTextures(): void {
+    const gl = this.gl!
+    gl.activeTexture(gl.TEXTURE1)
+    this.backgroundTexture = this.createTexture(gl, gl.LINEAR)
+  }
+
+  protected override setupShaders(): void {
+    super.setupShaders()
+    const gl = this.gl!
+
+    this.backgroundTextureUniform = assertExists(
+      gl.getUniformLocation(this.program!, 'backgroundTexture'),
+      'Uniform location'
+    )
+    this.maskTextureUniform = assertExists(
+      gl.getUniformLocation(this.program!, 'maskTexture'),
+      'Uniform location'
+    )
+    this.texelSizeUniform = assertExists(
+      gl.getUniformLocation(this.program!, 'texelSize'),
+      'Uniform location'
+    )
+    this.blurAmountUniform = assertExists(
+      gl.getUniformLocation(this.program!, 'blurAmount'),
+      'Uniform location'
+    )
+  }
+
+  protected override configureUniforms(): void {
+    super.configureUniforms()
+    const gl = this.gl!
+    gl.uniform1i(this.maskTextureUniform!, 0)
+    gl.uniform1i(this.backgroundTextureUniform!, 1)
+  }
+
+  override close(): void {
+    if (this.backgroundTexture) {
+      this.gl!.deleteTexture(this.backgroundTexture)
+    }
+    super.close()
+  }
+}
@@ -13,6 +13,7 @@ export const Effects = () => {
    <div
      className={css({
        padding: '0 1.5rem',
+        overflowY: 'scroll',
      })}
    >
      <EffectsConfiguration
@@ -170,16 +170,23 @@ export const EffectsConfiguration = ({
        })}
      >
        {videoTrack && !videoTrack.isMuted ? (
-          <video
-            ref={videoRef}
-            width="100%"
-            muted
-            style={{
-              transform: 'rotateY(180deg)',
-              minHeight: '175px',
-              borderRadius: '8px',
-            }}
-          />
+          <div
+            className={css({
+              position: 'relative',
+            })}
+            id="wip"
+          >
+            <video
+              ref={videoRef}
+              width="100%"
+              muted
+              style={{
+                transform: 'rotateY(180deg)',
+                minHeight: '175px',
+                borderRadius: '8px',
+              }}
+            />
+          </div>
        ) : (
          <div
            style={{
@@ -58,14 +58,14 @@ export class SdkReverseClient {
 * To be used in SDK scope.
 */
 export function useEnsureAuth() {
-  const { isLoggedIn, ...other } = useUser({
+  const { isLoggedIn, refetch } = useUser({
    fetchUserOptions: { attemptSilent: false },
  })

  const startSSO = () => {
    return new Promise<void>((resolve) => {
      SdkReverseClient.waitForAuthenticationAck().then(async () => {
-        await other.refetch()
+        await refetch()
        resolve()
      })
      const params = `scrollbars=no,resizable=no,status=no,location=no,toolbar=no,menubar=no,
@@ -106,7 +106,7 @@ const Marianne = () => {
        className={css({
          letterSpacing: '-.01em',
          textTransform: 'uppercase',
-          fontWeight: '600',
+          fontWeight: '700',
          fontFamily: 'Marianne',
          fontSize: '1.25rem',
          lineHeight: '1.75rem',
@@ -44,7 +44,7 @@ const Marianne = () => {
        className={css({
          letterSpacing: '-.01em',
          textTransform: 'uppercase',
-          fontWeight: '600',
+          fontWeight: '700',
          fontFamily: 'Marianne',
          fontSize: '0.7875rem !important',
        })}
@@ -189,6 +189,7 @@
    "access": {
      "title": "",
      "description": "",
+      "type": "",
      "levels": {
        "public": {
          "label": "",
@@ -188,6 +188,7 @@
    "access": {
      "title": "Room access",
      "description": "These settings will also apply to future occurrences of this meeting.",
+      "type": "Meeting access types",
      "levels": {
        "public": {
          "label": "Open",
@@ -188,6 +188,7 @@
    "access": {
      "title": "Accès à la réunion",
      "description": "Ces paramètres s'appliqueront également aux futures occurrences de cette réunion.",
+      "type": "Type d'accès à la réunion",
      "levels": {
        "public": {
          "label": "Ouvrir",
@@ -188,6 +188,7 @@
    "access": {
      "title": "Toegang tot vergadering",
      "description": "Deze instellingen zijn ook van toepassing op toekomstige sessies van deze vergadering.",
+      "type": "Vergaderings toegangstypen",
      "levels": {
        "public": {
          "label": "Open",
@@ -44,7 +44,7 @@ const FieldWrapper = styled('div', {
 const StyledLabel = styled(Label, {
  base: {
    display: 'block',
-    fontSize: '0.75rem',
+    fontSize: '0.875rem',
  },
  variants: {
    center: {
@@ -218,8 +218,10 @@ export const Field = <T extends object>({
                  {item.description && (
                    <Text
                      variant="note"
+                      wrap={'pretty'}
                      className={css({
                        textStyle: 'sm',
+                        marginBottom: '0.5rem',
                      })}
                    >
                      {item.description}
@@ -30,6 +30,46 @@
  font-display: swap;
 }

+@font-face {
+  font-family: 'Marianne';
+  src: url('/fonts/Marianne-Regular-subset.woff2') format('woff2');
+  font-weight: 400;
+  font-style: normal;
+  font-display: swap;
+}
+
+@font-face {
+  font-family: 'Marianne';
+  src: url('/fonts/Marianne-Regular_Italic-subset.woff2') format('woff2');
+  font-weight: 400;
+  font-style: italic;
+  font-display: swap;
+}
+
+@font-face {
+  font-family: 'Marianne';
+  src: url('/fonts/Marianne-Medium-subset.woff2') format('woff2');
+  font-weight: 500;
+  font-style: normal;
+  font-display: swap;
+}
+
+@font-face {
+  font-family: 'Marianne';
+  src: url('/fonts/Marianne-Bold-subset.woff2') format('woff2');
+  font-weight: 700;
+  font-style: normal;
+  font-display: swap;
+}
+
+@font-face {
+  font-family: 'Marianne';
+  src: url('/fonts/Marianne-ExtraBold-subset.woff2') format('woff2');
+  font-weight: 800;
+  font-style: normal;
+  font-display: swap;
+}
+
 /*
 * to reduce CLS
 * values taken from https://github.com/khempenius/font-fallbacks-dataset/blob/main/font-metric-overrides.csv#L2979
@@ -1,12 +1,12 @@
 {
  "name": "mail_mjml",
-  "version": "0.1.15",
+  "version": "0.1.16",
  "lockfileVersion": 3,
  "requires": true,
  "packages": {
    "": {
      "name": "mail_mjml",
-      "version": "0.1.15",
+      "version": "0.1.16",
      "license": "MIT",
      "dependencies": {
        "@html-to/text-cli": "0.5.4",
@@ -1,6 +1,6 @@
 {
  "name": "mail_mjml",
-  "version": "0.1.15",
+  "version": "0.1.16",
  "description": "An util to generate html and text django's templates from mjml templates",
  "type": "module",
  "dependencies": {
@@ -1,12 +1,12 @@
 {
  "name": "sdk",
-  "version": "0.1.15",
+  "version": "0.1.16",
  "lockfileVersion": 3,
  "requires": true,
  "packages": {
    "": {
      "name": "sdk",
-      "version": "0.1.15",
+      "version": "0.1.16",
      "license": "ISC",
      "workspaces": [
        "./library",
@@ -1,6 +1,6 @@
 {
  "name": "sdk",
-  "version": "0.1.15",
+  "version": "0.1.16",
  "author": "",
  "license": "ISC",
  "description": "",
@@ -1,7 +1,7 @@

 [project]
 name = "summary"
-version = "0.1.15"
+version = "0.1.16"
 dependencies = [
    "fastapi[standard]>=0.105.0",
    "uvicorn>=0.24.0",
@@ -10,14 +10,14 @@ dependencies = [
    "celery==5.4.0",
    "redis==5.2.1",
    "minio==7.2.15",
-    "openai==1.65.2",
+    "openai==1.68.2",
    "requests==2.32.3",
-    "sentry-sdk[fastapi, celery]==2.22.0",
+    "sentry-sdk[fastapi, celery]==2.24.1",
 ]

 [project.optional-dependencies]
 dev = [
-    "ruff==0.9.9",
+    "ruff==0.11.2",
 ]

 [build-system]
@@ -1,10 +1,15 @@
 """API routes related to application tasks."""

+from typing import Optional
+
 from celery.result import AsyncResult
 from fastapi import APIRouter
 from pydantic import BaseModel

-from summary.core.celery_worker import process_audio_transcribe_summarize
+from summary.core.celery_worker import (
+    process_audio_transcribe_summarize,
+    process_audio_transcribe_summarize_v2,
+)


 class TaskCreation(BaseModel):
@@ -13,6 +18,7 @@ class TaskCreation(BaseModel):
    filename: str
    email: str
    sub: str
+    version: Optional[int] = 2


 router = APIRouter(prefix="/tasks")
@@ -21,9 +27,15 @@ router = APIRouter(prefix="/tasks")
@router.post("/")
 async def create_task(request: TaskCreation):
    """Create a task."""
-    task = process_audio_transcribe_summarize.delay(
-        request.filename, request.email, request.sub
-    )
+    if request.version == 1:
+        task = process_audio_transcribe_summarize.delay(
+            request.filename, request.email, request.sub
+        )
+    else:
+        task = process_audio_transcribe_summarize_v2.delay(
+            request.filename, request.email, request.sub
+        )
+
    return {"id": task.id, "message": "Task created"}


@@ -56,6 +56,35 @@ def create_retry_session():
    return session


+def format_segments(transcription_data):
+    """Format transcription segments from WhisperX into a readable conversation format.
+
+    Processes transcription data with segments containing speaker information and text,
+    combining consecutive segments from the same speaker and formatting them as a
+    conversation with speaker labels.
+    """
+    formatted_output = ""
+    if not transcription_data or not hasattr(transcription_data, 'segments'):
+        if isinstance(transcription_data, dict) and 'segments' in transcription_data:
+            segments = transcription_data['segments']
+        else:
+            return "Error: Invalid transcription data format"
+    else:
+        segments = transcription_data.segments
+
+    previous_speaker = None
+
+    for segment in segments:
+        speaker = segment.get('speaker', 'UNKNOWN_SPEAKER')
+        text = segment.get('text', '')
+        if text:
+            if speaker != previous_speaker:
+                formatted_output += f"\n\n *{speaker}*: {text}"
+            else:
+                formatted_output += f" {text}"
+            previous_speaker = speaker
+    return formatted_output
+
 def post_with_retries(url, data):
    """Send POST request with automatic retries."""
    session = create_retry_session()
@@ -140,3 +169,72 @@ def process_audio_transcribe_summarize(filename: str, email: str, sub: str):

    logger.info("Webhook submitted successfully. Status: %s", response.status_code)
    logger.debug("Response body: %s", response.text)
+
+
+@celery.task(max_retries=settings.celery_max_retries)
+def process_audio_transcribe_summarize_v2(filename: str, email: str, sub: str):
+    """Process an audio file by transcribing it and generating a summary.
+
+    This Celery task performs the following operations:
+    1. Retrieves the audio file from MinIO storage
+    2. Transcribes the audio using WhisperX model
+    3. Sends the results via webhook
+
+    """
+    logger.info("Notification received")
+    logger.debug("filename: %s", filename)
+
+    minio_client = Minio(
+        settings.aws_s3_endpoint_url,
+        access_key=settings.aws_s3_access_key_id,
+        secret_key=settings.aws_s3_secret_access_key,
+        secure=settings.aws_s3_secure_access,
+    )
+
+    logger.debug("Connection to the Minio bucket successful")
+
+    audio_file_stream = minio_client.get_object(
+        settings.aws_storage_bucket_name, object_name=filename
+    )
+
+    temp_file_path = save_audio_stream(audio_file_stream)
+    logger.debug("Recording successfully downloaded, filepath: %s", temp_file_path)
+
+    logger.debug("Initiating OpenAI client")
+    openai_client = openai.OpenAI(
+        api_key=settings.openai_api_key, base_url=settings.openai_base_url
+    )
+
+    try:
+        logger.debug("Querying transcription …")
+        with open(temp_file_path, "rb") as audio_file:
+            transcription = openai_client.audio.transcriptions.create(
+                model=settings.openai_asr_model, file=audio_file
+            )
+
+            logger.debug("Transcription: \n %s", transcription)
+    finally:
+        if os.path.exists(temp_file_path):
+            os.remove(temp_file_path)
+            logger.debug("Temporary file removed: %s", temp_file_path)
+
+    formatted_transcription = format_segments(transcription)
+
+    data = {
+        "title": "Transcription",
+        "content": formatted_transcription,
+        "email": email,
+        "sub": sub,
+    }
+
+    logger.debug("Submitting webhook to %s", settings.webhook_url)
+    logger.debug("Request payload: %s", json.dumps(data, indent=2))
+
+    response = post_with_retries(settings.webhook_url, data)
+
+    logger.info("Webhook submitted successfully. Status: %s", response.status_code)
+    logger.debug("Response body: %s", response.text)
+
+    # TODO - integrate summarize the transcript and create a new document.
+
+
Author	SHA1	Message	Date
Arnaud Robin	3bdd12e464	wip ok	2025-03-29 14:05:28 +01:00
Arnaud Robin	b1a3770e61	wip foiré	2025-03-29 14:05:28 +01:00
Arnaud Robin	f501496a0b	WIP webgl2 + basic blur	2025-03-29 14:05:28 +01:00
lebaudantoine	1dde506e05	wip blur broken	2025-03-29 14:05:28 +01:00
lebaudantoine	96310bff75	wip copy/pasted mediapipe	2025-03-29 14:05:28 +01:00
Berry den Hartog	3299ae7c39	📝(docs) describe environmental options for meet backend	2025-03-28 20:14:32 +01:00
Jacques ROUSSEL	93ca4f2bf4	🐛(ci) use github action for argocd webhook notification In order to refactor this notification between alls projetcs, we chooseto use a custom github action	2025-03-28 16:24:17 +01:00
lebaudantoine	19a4e442c5	🔖(minor) bump release to 0.1.16 Misc fixes, release diarization.	2025-03-25 23:27:41 +01:00
lebaudantoine	53f1c9c1d4	💥(summary) replace Whisper with WhisperX for transcription Switch from insanely-fast-whisper to WhisperX for transcription services, gaining word-level granularity not available with FlashAttention in previous API. Whisper was packaged in FastAPI image with basic diarization and speaker-segment association capabilities. Implementation prioritized speed to market over testing. Future iterations will enhance diarization quality and add proper test coverage. API consumers should expect behavioral differences and new capabilities.	2025-03-25 17:09:13 +01:00
lebaudantoine	28538b63da	✨(backend) add env variable to configure marketing service timeout This change allows the marketing service timeout to be easily adjusted via an environment variable, eliminating the need for a new software release. Additionally, the update makes the code more explicit and easier to maintain.	2025-03-25 15:41:47 +01:00
renovate[bot]	94ae5d52c2	⬆️(dependencies) update js dependencies Updated the useUser hook to avoid unwrapping the query object directly, which leads to excessive re-rendering.	2025-03-25 14:53:15 +01:00
lebaudantoine	6b38a3c996	💄(frontend) center titles in prejoin components In non-French languages, the title may be long enough to span two lines. Ensure the text is centered.	2025-03-25 13:38:10 +01:00
lebaudantoine	0af30ec366	✨(backend) notify participants only if the room exists Improves sendData reliability by preventing execution when the room doesn’t exist. This change addresses errors in staging and production where waiting participants arrive before the room owner creates the room. In remote environments, the LiveKit Python SDK doesn’t return a clean Twirp error when the room is missing; instead of a proper "server unknown" response, it raises a ContentTypeError, as if the LiveKit server weren’t responding with a JSON payload, even though the code specifies otherwise. While the issue cannot be reproduced locally, this should help mitigate production errors. Part of a broader effort to enhance data transmission reliability. Importantly, a participant requesting entry to a room before the owner arrives should not be considered an exception.	2025-03-25 13:33:47 +01:00
lebaudantoine	33e017464f	📌(livekit) pin livekit dev version Pin the LiveKit server version to use the same as the staging and production environment. It's a good practice.	2025-03-25 13:18:44 +01:00
lebaudantoine	9e91bbe417	🔇(backend) reduce noisy logs in Sentry Ignore logs from DockerflowMiddleware and minimize unnecessary log submissions.	2025-03-25 09:42:35 +01:00
renovate[bot]	e821982353	⬆️(dependencies) update python dependencies	2025-03-24 20:04:15 +01:00
lebaudantoine	18f4a117ab	🩹(backend) invert operation order in participant handling Invert operation sequence to first notify people in room before setting participant in cache. Fixes infinite loop issue caused by 3s cache timeout for waiting participants when requests take too long. Problem only occurred when notifications were delayed, as faster notification delivery masked the race condition.	2025-03-24 19:39:13 +01:00
lebaudantoine	b5b99d4c52	🐛(frontend) fix effect menu scroll Since refactoring, the effect side panel, was not scrollable anymore. Fixed it.	2025-03-24 17:10:50 +01:00
lebaudantoine	8a4323aa82	🐛(license) fix reuse.software missing support in GitHub Duplicate the MIT License, which are not yet supported by GitHub and its badges.	2025-03-24 14:54:42 +01:00
lebaudantoine	5cbac23f13	💄(frontend) adjust minor visual item Adjust few items from the landing and the interface.	2025-03-24 14:38:27 +01:00
lebaudantoine	d55810784d	🐛(frontend) fix Marianne import The font face definition wasn't working properly. Switch to vanilla css.	2025-03-24 14:38:27 +01:00
lebaudantoine	c2fdefe7b2	💄(frontend) adjust font size based on Marianne recent changes Marianne is bigger than OpenSans, adjust admin side panel style.	2025-03-24 13:53:41 +01:00
lebaudantoine	bf2474d0e1	🌐(frontend) add missing translation The admin side panel was translated too quickly. I forgot a title, fix it.	2025-03-24 13:53:41 +01:00