Compare commits

..

197 Commits

Author SHA1 Message Date
lebaudantoine 8d9ac96d2b wip invalidate cache when room is finished 2025-01-06 19:37:46 +01:00
lebaudantoine 5428aec43e wip fix errors related to re-rendering and web workers 2025-01-05 01:13:17 +01:00
lebaudantoine a2408aebff wip encrypt and encode passphrase while storing it in cache 2025-01-04 23:58:49 +01:00
lebaudantoine 3e3ca6a87b use redis instead of cache 2025-01-04 23:11:45 +01:00
lebaudantoine 9830940d61 rename misleading name
slug is another term in our app
this is the room id
2025-01-04 23:11:22 +01:00
lebaudantoine dd7e3a7c44 wip generate a random passphrase cached by slug 2025-01-04 14:12:45 +01:00
lebaudantoine 1cf26eec19 wip get passphrase from the backend 2025-01-04 13:20:55 +01:00
lebaudantoine 83d4028e84 wip hardcode a passphrase and enable e2ee
inspired by LiveKit repo, follow their example
2025-01-04 13:07:26 +01:00
lebaudantoine ac65404ad6 ⚗️(backend) annotate when and what is required for ee2e
commit will be dropped.
2025-01-04 12:07:28 +01:00
lebaudantoine 4c0230d537 (backend) post email to marketing tools while signing up new users
Submitting new users to the marketing service is currently handled
during signup and is performed only once.

This is a pragmatic first implementation, yet imperfect.

In the future, this should be improved by delegating the call to a Celery
worker or an async task.
2024-12-31 15:09:51 +01:00
lebaudantoine 7309df4115 ♻️(backend) add MarketingService protocol and Brevo implementation
Introduced a MarketingService protocol for typed marketing operations,
allowing easy integration of alternative services.

Implemented a Brevo wrapper following the protocol to decouple
the codebase from the sdk. These implementations are simple and pragmatic.
Feel free to refactor them.
2024-12-31 15:09:51 +01:00
lebaudantoine 18b2dfc497 (backend) install brevo client
Brevo (ex-sendinblue) is a common solution used for marketing and
communications.
2024-12-31 15:09:51 +01:00
lebaudantoine 3282da7c56 🚨(backend) fix Django UnorderedObjectListWarning on models
Found this solution googling on Stack Overflow.

Without a default ordering on a model, Django raises a warning, that
pagination may yield inconsistent results.
2024-12-31 15:09:35 +01:00
renovate[bot] 7f8a6e8685 ⬆️(dependencies) update python dependencies 2024-12-30 13:32:37 +01:00
lebaudantoine 4232c0a303 🔖(patch) bump release to 0.1.12
Release a fix while calling Docs backend API.
2024-12-18 22:54:57 +01:00
lebaudantoine a5454e48b7 🔥(frontend) remove redundant yarn.lock file
We decided to stick to npm for dependencies management.
Having both package-lock.json and yarn.lock isn't clear for
developers, simplify the stack.
2024-12-17 18:53:55 +01:00
dependabot[bot] 591706f363 🔒️(frontend) bump nanoid from 3.3.7 to 3.3.8 in /src/frontend
3.3.8 fixed a way to break Nano ID by passing non-integer size.
Raised as a moderate severity issue by Dependabot.
2024-12-17 18:53:55 +01:00
dependabot[bot] 1ea84b6e6c 🔒️(mail) bump cross-spawn to fix a high severity issue
Bump cross-spawn from 7.0.3 to 7.0.6 in /src/mail
Alert triggered by DependaBot.
2024-12-17 16:43:54 +01:00
renovate[bot] c0d101a326 ⬆️(dependencies) update PyJWT to v2.10.1 [SECURITY] 2024-12-17 16:23:05 +01:00
lebaudantoine 64cfcb6c0f 💩(summary) fix temporarily the summary microservice
The new endpoint requires title and content, breaking
the original implementation. This hotfix ensures staging works
immediately while I plan an LLM-based solution
for title generation.
2024-12-17 11:34:46 +01:00
lebaudantoine cdd69b741a 🚑️(mail) bump package version
Oopsie, I forgot to bump mail package version.
Fix it
2024-12-17 11:34:46 +01:00
Jacques ROUSSEL f5a87cc210 🔐(helm) bump secret in staging
We add a typo in secret so we bump secrets to fix it.
2024-12-17 10:19:40 +01:00
lebaudantoine 4d4ddb9ee8 🔖(patch) bump release to 0.1.11
Weekly release, responsiveness is still under construction.
2024-12-17 00:26:29 +01:00
lebaudantoine 70dbf94f7b 🩹(frontend) avoid duplicating Grist form's URL
It led to an error, with two occurrences of the form's URL being
desynchronized. Fix this minor issue, and refactor the constant
in a constant file to be shared across the app.
2024-12-17 00:08:53 +01:00
Jacques ROUSSEL 0ad37ee6de 🔨(tilt) improve local stack
Improve the local tilt file in order to be abble to start all thing
without any dependencies to DINUM environment
2024-12-16 23:41:09 +01:00
Jacques ROUSSEL ed4f7dcf6c 🔐(helm) add production secrets for recording and transcription
Added all missing secrets for the recording feature
and transcription microservice.
2024-12-16 23:08:04 +01:00
lebaudantoine c54773008c 🚨(backend) fix backend linter
I wrongly merged a backend pr, and broke the linter.
2024-12-16 23:08:04 +01:00
lebaudantoine 6848321bcc 🔧(helm) configure transcription microservice in production
Added one replica for the summary and Celery deployments.
Secret will be added in the upcoming commits.
2024-12-16 23:08:04 +01:00
lebaudantoine f161a5cf6a 🔧(helm) configure recording in production
Enabled recording feature in production. MinIO needs to trigger a webhook
when a new recording is saved. Secret will be updated in the upcoming commits.
2024-12-16 23:08:04 +01:00
lebaudantoine 51270571bc 🔧(helm) update transcription microservice in staging
Updated the webhook URL to the definitive version in docs.

Jacques also updated the webhook secret for authentication
against Impress API. Not tested locally.
2024-12-16 23:08:04 +01:00
Nathan Vasse c6fdeaf1e9 📱(front) add responsive menu on mobile
We want to have a specific responsive menu on mobile browsers.
It also implied to refactor a bit the way the settings modals is opened
because it could be opened from this responsive menu, so in order to achive
that a specific context has been created in order to allow its opening
from any sub component of the control bar.
2024-12-13 15:41:54 +01:00
Nathan Vasse f1959cbb3a ♻️(front) add more customization and event handler to toggles
We want to be able to customize the variant which those toggle uses as
well as being able to trigger an event when the toggle is pressed. This
is going to be useful to close the responsive menu after each clic as
react-aria prevent click event propgation.
2024-12-13 15:41:54 +01:00
Nathan Vasse 84cea2f658 (front) add description to Buttons
We want to display full description below buttons for mobile specific
displays.
2024-12-13 15:41:54 +01:00
Nathan Vasse db1fdb9871 (front) add useIsMobile hook
This hook tells whether the current browser is a mobile one or not.
2024-12-13 15:41:54 +01:00
lebaudantoine a8618239d1 🔧(frontend) enable dynacast optimizations
According to the documentation:

Dynacast dynamically pauses video layers that are
not being consumed by any subscribers, significantly
reducing publishing CPU and bandwidth usage.

Dynacast will be enabled if SVC codecs (VP9/AV1) are used.
Multi-codec simulcast requires dynacast

My goal is to reduce CPU and bandwidth usage for clients.

Dynacast is enabled both in OpenTalk and LiveKit demo app!
2024-12-12 11:25:06 +01:00
lebaudantoine 0eb283b75a 🔧(frontend) enable adaptive stream optimizations
Adaptive stream is a key optimization in large room.

Enabled adaptive stream to automatically manage
the quality of subscribed video tracks, optimizing
for bandwidth and CPU usage.

When video elements are visible, it adjusts
the resolution based on the size of the largest visible
element. If no video elements are visible, it
temporarily pauses the track until they are visible again.

Additionally, introduced support for custom pixel density,
which defaults to `2` for high-density screens
(devicePixelRatio ≥ 3) or `1` for standard screens.
Setting it to `screen` allows the pixel density to match
the actual screen's devicePixelRatio, optimizing video
clarity on ultra-high-definition displays.

This ensures a balance between streaming quality
and resource consumption.

This might also significantly increase the bandwidth
consumed by people streaming on high definition screens.
It needs to be battle tested.

OpenTalk uses a adaptiveStream equals true, while LiveKit
demo app uses 'screen' value. I followed OpenTalk choice,
I was scared 'screen' value creates performance issues
for user with high resolution screen in poor network conditions.
2024-12-12 11:25:06 +01:00
lebaudantoine 0104cabc5e 🔧(frontend) default to vp9 for video codec
As a mandatory codec in WebRTC specifications, VP8 serves as
the baseline for compatibility, making it the default choice
in LiveKit client configuration.

There is room for optimization.

Newer codecs like VP9 offer significant efficiency gains
compared to VP8, with a 23-33% improvement in compression
efficiency. This translates to better video quality at
the same bitrate or reduced bandwidth usage for the same quality.

VP9 is supported in Safari starting from version 15.0+,
and Firefox offers partial support. However, Firefox lacks
support for VP9's Scalable Video Coding (SVC).

With SVC, participants can send a single VP9 stream
with multiple spatial or temporal layers. This allows receivers
to dynamically adjust video quality by using lower layers when
resolution or bandwidth needs to be reduced, improving
adaptability in heterogeneous network conditions.

Simulcast, by contrast, sends multiple separate streams
at different resolutions. While widely supported in VP8 and VP9,
it consumes more bandwidth compared to SVC.

The configuration added here is based on the LiveKit demo app,
which defaults to VP9 when supported. OpenTalk’s configuration
also recommends VP9.

If a browser does not support VP9, LiveKit falls back to VP8 or
other codecs as needed. Notably, LiveKit disables VP9 encoding for
Firefox due to longstanding issues, but it can still decode VP9
streams and encode VP8 for outgoing streams. This ensures
compatibility with other participants, even in mixed environments
where some browsers use VP9 and others fallback to VP8.

In theory, participants do not all need to switch to a single codec,
as both LiveKit and browsers intelligently handle codec negotiation
on a per-participant basis. This dynamic adaptation ensures seamless
communication without manual intervention.

A similar challenge with codec compatibility was raised
in Jitsi two years ago, check issue #10657.

Before any release, this needs to be battle tested
with Firefox 115 browsers.
2024-12-12 11:25:06 +01:00
lebaudantoine dbfba564c5 🔧(backend) fix sentry deprecated scope
`sentry_sdk.configure_scope` is deprecated and will
be removed in the next major version.

(commit taken from people by @qbey)
2024-12-10 12:00:41 +01:00
renovate[bot] 4830206bb2 ⬆️(dependencies) update python dependencies 2024-12-09 18:20:12 +01:00
lebaudantoine e4f30f926c 🩹(summary) fix configuration in staging
@rouja forgot to add the right environment variable for the celery
workers. Fixed it.
2024-12-09 18:15:47 +01:00
lebaudantoine fc61f58596 🔧(summary) update LLM and whisper endpoints
@rouja fixed the issue from OutScale load balancer.
He found the right parameter in OutScale documentation,
that was causing a request timeout.
2024-12-09 18:15:47 +01:00
lebaudantoine e0021dbb80 (frontend) introduce a notification settings
I extended Room settings with a new tab, allowing users to toggle
the sound notification they want.

Their choices will be persisted.
2024-12-09 18:14:46 +01:00
lebaudantoine 83914f8307 (frontend) add notification store for managing user preferences
Introduced a global state to handle user preferences related to notifications.
The first use case is sound notifications, allowing users to disable them
based on feedback.

Additionally, the sound volume is now stored globally, making it easy to
configure in the future if needed. I've lowered the volume of
the notifications to make them more discreet.

Preferences are persisted in local storage, ensuring they are retained
between meetings.
2024-12-09 18:14:46 +01:00
lebaudantoine be54709598 (frontend) create STORAGE_KEYS constant
Add STORAGE_KEYS object to centralize localStorage keys,
ensuring no key overlaps by maintaining a single source
of truth for key declarations across the app.

Might be premature, as only the notification store will be persisted.
2024-12-09 18:14:46 +01:00
lebaudantoine 5b76ea492b (frontend) create utils function for valtio
Valtio allows state persistence in local storage, which is
necessary for the notification store. In this case, I'll need
to persist a `proxyMap`—a utility provided by Valtio to create
an observable map.

However, since `proxyMap` isn't natively serializable,
I'll need to implement two custom functions: one for serialization
and another for deserialization (revival).

Regarding the file structure, I've named the file `utils/valtio`,
but this can be discussed further. The purpose of this file is
to centralize common utility functions related to Valtio
for better organization and reuse.
2024-12-09 18:14:46 +01:00
lebaudantoine 46934a84d1 🚚(frontend) rename NotificationType items
I found the item names unclear, so I updated them for better clarity.

I also removed the unnecessary 'lowered' item and added
a TODO comment about handling the message received notification,
which is not yet implemented in the code.
2024-12-09 18:14:46 +01:00
lebaudantoine deb9ab1a1d (frontend) create a new primitive Switch
Inspired by Robin's design, I've styled a React Aria
Switch component using our DSFR theme.

This is an initial draft and isn't yet pixel-perfect compared
to Robin's design. It also hasn't been integrated into
the form inputs yet.
2024-12-09 18:14:46 +01:00
renovate[bot] d0fd16d7d2 ⬆️(dependencies) update django to v5.1.4 [SECURITY] 2024-12-08 22:21:18 +01:00
lebaudantoine 0b8181e5ce ✏️(backend) fix few typos
Fix few typos in the docstring.
2024-12-06 12:46:46 +01:00
lebaudantoine 8b2365d5f9 (backend) add ODC_VERIFY_SSL configuration
Need to disable SSL while authenticating to unsecure
OIDC provider in dev environment.
2024-12-06 12:46:46 +01:00
lebaudantoine 2dd16d1f40 🩹(backend) add missing environ_prefix on OIDC_CREATE_USER
Misleading, to not have all OIDC-related settings, with environ prefix at None.
Caught myself into troubleshooting few minutes.
2024-12-06 12:46:46 +01:00
lebaudantoine fa9484b630 (frontend) introduce a recording toaster
Notify visually users that the room is being recorded.
Draft, it will be enhance the future.
2024-12-04 18:38:26 +01:00
lebaudantoine 1e0e495cd8 ♻️(frontend) refactor pulse_mic into pulse_background
Make the keyframe more generic with an explicit naming.
2024-12-04 18:38:26 +01:00
lebaudantoine c270299179 🩹(frontend) fix submit button in feedbacks page
Since recent changes in the Color palette, the button was totally
invisible… fixed it.
2024-12-04 18:38:26 +01:00
lebaudantoine d9a84e5f0f 🔥(frontend) remove transcription menu item
This menu item was replaced by a side panel.
2024-12-04 18:38:26 +01:00
lebaudantoine 0f64d3cf3a (frontend) introduce a sidepanel for AI assistant
Introduce the content for the AI assistant panel, which describes the
feature, and offers a button to start and stop a recording.
2024-12-04 18:38:26 +01:00
lebaudantoine d1e008a844 ♻️(frontend) introduce useRoomId hook
Manipulating the room's id from the react-query cache should be
encapsulated in a dedicated hook.
2024-12-04 18:38:26 +01:00
lebaudantoine 3be5a5afc6 ♻️(frontend) package checks in useHasTranscriptAccess hook
This hook will be used by the toggle and the sidepanel to make
sure the users have sufficient permissions to access the transcript
features.
2024-12-04 18:38:26 +01:00
lebaudantoine 31468a5e7c ♻️(frontend) introduce a reusable isTranscriptEnabled
Encapsulate the logic, checking if the feature is enabled in the
backend, in a proper and reusable hook.
2024-12-04 18:38:26 +01:00
lebaudantoine b342b9d526 🚩(frontend) enable transcript toggle with a feature flag
Rely on Posthog for a first iteration on the feature flag feature.
This is a pragmatic choice, relying on an external dependency might
not suitable on the longer term, however, compare to the maturity
of our product, this is the best trade off.
2024-12-04 18:38:26 +01:00
lebaudantoine f7e7c3ba22 🚚(frontend) remove wrong 'tsx' extension
The hook only uses typescript code.
2024-12-04 18:38:26 +01:00
lebaudantoine 3902b02691 📈(frontend) check if analytic is enabled
Few frontend features rely on Posthog. Posthog is not activated in
dev environment. Offer a hook that encapsulates this logic, and
return a boolean flag.
2024-12-04 18:38:26 +01:00
lebaudantoine 7ce4390740 (frontend) initialize transcript sidebar panel
Setup base structure and styling for transcript menu sidebar
2024-12-04 18:38:26 +01:00
Jacques ROUSSEL 94d18cffe4 🔐(secrets) bump secretBump secret
In order to use our openai api we need new secret
2024-12-04 10:49:17 +01:00
lebaudantoine ad0c3eea66 🩹(summary) remove temporary audio files saved
I forgot to remove temporary files downloaded from the Minio bucket.
2024-12-04 10:49:17 +01:00
lebaudantoine 459bbf65a8 🔧(summary) use our self-deployed models
Configure dev and staging environment to use our self-deployed
models (Whisper and LLM). Secrets need to be updated btw.

Because of outscale LB bug, which timeout after 60s, we need to
connect directly to the svc.
2024-12-04 10:49:17 +01:00
lebaudantoine 4b4f16e93f 🔖(patch) bump release to 0.1.10
I made a mistake, and forgot to set summary and celery
deployment to 0 replicas.
2024-12-03 02:31:10 +01:00
lebaudantoine 3e3f964ac5 🚑️(helm) set celery and summary replicas to 0
These two services are not yet needed in production. Disable them.
2024-12-03 02:25:52 +01:00
lebaudantoine bcb111defb 🔖(patch) bump release to 0.1.9
Release the new ui, the logo, the footer, etc.
2024-12-03 02:12:36 +01:00
lebaudantoine 91a3aa4033 📱(frontend) fix few layout issues
For screen between xs and xsm, Dinum typography is creating
an overflow on mobile.

minHeight 'fit-content' is not working as expected on firefox.
2024-12-03 01:59:03 +01:00
lebaudantoine 05b74365fa 🐛(frontend) fix broken animation when motion is reduced
Feedback from one of our users.
I forgot animation can be disabled for accessibility purpose.
Fix it by bypassing the animation if reduce motion is activated.
Oopsie, mybad, I learnt something.
2024-12-03 01:05:06 +01:00
lebaudantoine 4eea4d01cd 🍱(frontend) install Marianne font
Declare Marianne font. Not sure of my configuration, lmk if
my configuration feel wrong @nvasse @manuhabitela.

It's a requirement in the public sector.
2024-12-03 01:05:06 +01:00
lebaudantoine 8caa6b7820 (frontend) add a fav icon
As I introduced the logo, let's add the missing favicon.
2024-12-03 01:05:06 +01:00
lebaudantoine e7e7bb0d09 💩(frontend) introduce an official footer
Based on Florian's feedbacks, this is mandatory as the project is
getting attention.

Bad code, needs a refactor.
2024-12-03 01:05:06 +01:00
lebaudantoine 819d3784f7 🍱(frontend) update homepage assets
Robin made a new design. Update the first image with this lit
design!!
2024-12-03 01:05:06 +01:00
lebaudantoine d83ccb3d9d (frontend) add all official logos
Based on Florian and Stéphanie feedbacks, add the 'Marianne' logo,
and a beta tag on our visio's logo.

I've slightly updated the header. Its responsive is broken in certain
situations.
2024-12-03 01:05:06 +01:00
lebaudantoine a083c8cc3a 💄(frontend) align participant placeholder with recent changes
Use a blue-ish background, instead of a gray one.
We'll continue to iterate on this component.
2024-12-03 01:05:06 +01:00
lebaudantoine a299cae8e8 🩹(frontend) prevent an error if crisp is not defined
While hot reloading the stack, I got an error, Crisp not being
defined at the time of the component rendering. Prevent such issue.
I am not 100% sure this commit is useful, WDYT?
2024-12-03 01:05:06 +01:00
lebaudantoine 60b7a35ed1 💄(frontend) replace grayscale button with a new variant
Button was looking disabled. Use an appropriate contrast.
2024-12-03 01:05:06 +01:00
lebaudantoine 39fdffd9a8 🩹(frontend) harmonize toggle colors in controls
nit-picking, match colors from the central controls
and the minor ones on the right.
2024-12-03 01:05:06 +01:00
lebaudantoine e3779d9a6e 💄(frontend) update color palette
Discussed IRL with Robin and Natan. Previous iteration wasn't
appropriated for a visioconference tool.

Try a trade off between nice colors and DSFR. We will probably
continue to iterate on these.
2024-12-03 01:05:06 +01:00
lebaudantoine b74d6e72f9 🔧(helm) configure staging to notify summary service
Add the relevant configurations, to be tested when deployed.
2024-12-02 14:33:54 +01:00
lebaudantoine 9be826fd14 ⬆️(summary) upgrade openai client to 1.55.3
Got an unexpected error while instantiating the openai client
since I installed sentry deps. I've upgraded openai version
without digging much, and it worked again.
2024-12-02 14:33:54 +01:00
lebaudantoine 4fe01ae2bf 💩(backend) notify the summary service when a new recording is available
Draft a piece of code to try the feature in staging. I'll consolidate this
implementation ASAP, as soon we have a first implementation functional.

What's missing?
- when owners are multiple
- retry when the backend cannot reach the summary service
- factorize the key oneliner, duplicated from the egress service
- optimize SQL query
- unit tests
2024-12-02 14:33:54 +01:00
lebaudantoine 300756b323 🔧(backend) configure sentry in staging and production
Share the same project as the microservice one. Will see in the future
if we need a separate sentry account for the backend. Good enough
at this project stage.
2024-12-02 14:33:54 +01:00
lebaudantoine 0af79912c5 📝(helm) generate documentation for summary microservices
Run the generate-readme script to add the documentation related to
the newly introduced microservice.
2024-12-02 14:33:54 +01:00
lebaudantoine 446270c153 🩹(summary) fix Helm Readme
Oopsie, I forgot to update the Helm readme.
Fixed it.
2024-12-02 14:33:54 +01:00
lebaudantoine e6377ce182 (summary) enable sentry monitoring
Necessary to monitor how the micro service acts.
Will tweak sentry configurations later on.
2024-12-02 14:33:54 +01:00
lebaudantoine a6b6cb7787 🔧(summary) make celery max retries an env configurations
Minor tweak ! It would be useful when we'll scale or microservice.
2024-12-02 14:33:54 +01:00
lebaudantoine 50146e95f4 🩹(summary) use the right s3 env variable
I use 'url' instead of the 'endpoint' as desired.
Leading the Minio client to fail connecting to the bucket.
Fixed it!
2024-12-02 14:33:54 +01:00
lebaudantoine 35741a1bc1 ♻️(summary) reorganize API code
I totally reorganized the API code, to gain in clarity.
2024-12-02 14:33:54 +01:00
lebaudantoine b12b14b277 🔒️(summary) run image as a non-root user
Based on @rouja's comment, this is a bad practice, running a docker
image as root. Added a User instruction with a default non-root user.
2024-12-02 14:33:54 +01:00
lebaudantoine bb30f25953 ⚰️(frontend) remove dead code
Introducing Crisp, we removed the menu item for support. I forgot
to remove its i18n content. My bad
2024-11-29 21:52:28 +01:00
lebaudantoine ebbdebf444 📱(front) improve control bar responsive
We want the bottom bar to stay at the center of the screen.

Co-authored-by: Nathan Vasse <nathan.vasse@gmail.com>
2024-11-29 21:52:28 +01:00
lebaudantoine fde10e7556 🩹(helm) add missing secret for summary service
Oopsie. I forced pushed and dropped this commit.
2024-11-29 19:00:48 +01:00
lebaudantoine 0dbb256e9f 👷(summary) build and push summary image
Copy pasted from the job in charge of building and pushing
the backend image.
2024-11-29 18:39:40 +01:00
lebaudantoine 33c8a3a9e4 💩(summary) declare microservice in staging
Add the minimal extra templates and env values to deploy the micro
service.
2024-11-29 18:39:40 +01:00
lebaudantoine dd0cb61ebc 💩(summary) kubernitize the micro service
Add the micro service to the helm chart.
2024-11-29 18:39:40 +01:00
lebaudantoine e92f084afb 💩(summary) vendor a first draft
This is written in a rush, boostrap the real logic with celery
worker to process meeting recording.

Lack of unit tests is critical, I am not proud of a it. I am
totally in a hurry for the demo. Not sure we will actually keep
this microservice.
2024-11-29 18:39:40 +01:00
lebaudantoine 4a53005ae3 💩(summary) compose the summary stack
Enhance the DX offering a docker compose with reloading.
2024-11-29 18:39:40 +01:00
lebaudantoine d7d82130d9 💩(summary) dockerize FastAPI app
Add a minimal docker file to run the FastAPI server.
2024-11-29 18:39:40 +01:00
lebaudantoine 16bee00eb4 💩(summary) init a FastAPI app
Bootsrap a generic FastAPI app to kick-start the microservice.
2024-11-29 18:39:40 +01:00
Jacques ROUSSEL 0f70b544ed 🔐(secrets) bump secret
Bump secret in order to enable recording on staging
2024-11-29 17:05:29 +01:00
lebaudantoine 2903442a67 🎨(frontend) enhance few naming
I committed in a rush the transcript item, fix my dirty code.
Enhance few naming.
2024-11-29 17:05:29 +01:00
lebaudantoine be884a8ca1 🚩(frontend) display transcript item only for admin/owner
Only room admin/owner should be abble to start a transcript.
2024-11-29 17:05:29 +01:00
lebaudantoine fe70165076 🚩(frontend) display transcript items only when back is enabled
Avoid displaying the feature in the client if it's disabled
in the backend.
2024-11-29 17:05:29 +01:00
lebaudantoine 290ba4274a ⚰️(frontend) remove dead code
With chat refactoring, few lines of code was dead.
Fix it!
2024-11-29 17:05:29 +01:00
lebaudantoine d2f239e6a2 🚧(frontend) add a temporary items to interact with transcription
This is totally temporary, only to test transcription recording.
In the upcoming commits, a proper side panel with all activities
will be created.
2024-11-29 17:05:29 +01:00
lebaudantoine c61fc40671 (frontend) add API to start/stop recording
API calls to interact with backend, minimal call. Might be improved.
2024-11-29 17:05:29 +01:00
lebaudantoine 10705ca3ac 🩹(backend) fix duplication due to a bad rebase
While rebasing, I made few mistake with a settings wrongly renamed.
Oopsie fix this huge mistake.

Plus, a line was duplicated in Event authentication.
2024-11-29 17:05:29 +01:00
lebaudantoine 6b03ebb393 🔧(backend) enable recording and storage hook
Update values for dev and staging environment to enable
recording-related endpoints. A new secret need to be created.

Production values will be added in an upcoming commit.
2024-11-29 17:05:29 +01:00
Jacques ROUSSEL 0627510f10 ♻️(tilt) simplify the local stack
This commit solves few issues:
- sharing the relevant certificates with minio so when triggering the webhook
notification, the minio pod can verify our backend domain certificates.
- making sure everything spawn in the right namespace (LiveKit and the Egress)
without relying on a dirty fix in the make start-tilt.

all these fixes were made by @rouja, I don't fully understand them yet.
He simplified the stack, removing two Kind nodes to make it lightweight.

thx @rouja.
2024-11-29 15:54:24 +01:00
NathanVss f9233c8fb3 Merge pull request #245 from numerique-gouv/feat/video-conference-ui
Enhance Conference UI
2024-11-28 11:49:00 +01:00
Nathan Vasse 0b672ef8da 🚨(front) fix ui enhancement formats
Some formatting were forgotten, here is the fix.
2024-11-28 11:41:38 +01:00
Nathan Vasse 0b0efa5bab 💄(front) improve menu and select ui
We want to introduce two variant for this lists, so it was needed
to create a slot recipe.
2024-11-28 11:30:37 +01:00
Nathan Vasse 0958206057 💄(front) enhance ui first shot
Based on recent sketches, we decided to implement the new ui. This is
a first shot, more will be coming next.
2024-11-28 11:30:32 +01:00
renovate[bot] 4e13ace815 ⬆️(dependencies) update js dependencies 2024-11-25 11:44:30 +01:00
renovate[bot] 30ee0fdce3 ⬆️(dependencies) update python dependencies 2024-11-25 11:35:11 +01:00
renovate[bot] 17453a6cee ⬆️(dependencies) update aiohttp to v3.10.11 [SECURITY] 2024-11-25 10:42:02 +01:00
lebaudantoine c1e8ebc129 💬(frontend) rename Agent Connect mentions
Feedback from C Chausse. Outdated copy writting.
2024-11-25 10:36:01 +01:00
lebaudantoine a1dbf02f9c 🩹(frontend) avoid franglish copywritting
My bad, few english terms were mixed in some french ones.
Fixed it.
2024-11-25 10:35:38 +01:00
lebaudantoine c4206b4b24 🔖(patch) bump release to 0.1.8 2024-11-19 00:24:29 +01:00
lebaudantoine 732df0768c 💄(frontend) adjust home based on Sam feedbacks
Based on Samuel's feedbacks, adjust few details.
2024-11-18 13:31:26 +01:00
lebaudantoine 8a96b6939a 🐛(helm) rename bucket
Wrong copy paste. My bad, bucket's name wasn't matching our project.
2024-11-18 10:04:21 +01:00
lebaudantoine e78b8c69fe 🩹(frontend) add missing key
Fix missing warning, key props was missing on the item of the
for loop.
2024-11-18 01:22:54 +01:00
lebaudantoine 6e49ab3230 💄(frontend) narrow home layout
Avoid having too much white space around the home content.
2024-11-18 01:22:54 +01:00
lebaudantoine 200e2d3c2f 💩(frontend) introduce multistep form
Add an open feedback form step, to allow people giving us some
feedbacks on what's wrong with our product.
2024-11-18 01:22:54 +01:00
lebaudantoine 4ffef3f94a 🚸(frontend) add hover animation on rating Button
Nitpicking, enhance small interactivity with form.
2024-11-18 01:22:54 +01:00
lebaudantoine 8e62f2ecd3 🩹(frontend) fix TextArea export
Wrongly exported the TextArea primitive, mybad.
Fixed !
2024-11-18 01:22:54 +01:00
lebaudantoine d024fb1b95 (frontend) capture from with PostHog survey
Initialize logic related to PostHog, use an API survey to avoid
having element branded with PostHog logo.
2024-11-18 01:22:54 +01:00
lebaudantoine 7e49f0f661 ⬆️(frontend) update PostHog sdk
Required by PostHog app, to benefits from latest survey features.
2024-11-18 01:22:54 +01:00
lebaudantoine fb8c0fd1b5 💄(frontend) refactor feedbacks screen
Enhance initial screen. Allow user returning to the meeting.
Also, add basic and non-functional rating component to mock,
what the form would be.
2024-11-18 01:22:54 +01:00
lebaudantoine b07e4c58b4 💄(frontend) remove slider feature on small screens
Fit the slider content into small screen, by disabling all features
related to the slider. Good enough for a first draft.
2024-11-17 16:48:54 +01:00
lebaudantoine e6bbf2d3c8 💄(frontend) enhance header responsiveness
Quick fix, not optimal. Removed a useless breaking point IMO.
I won't spend too much time on the Header component, I'll
introduce new features needing a heavy layout update.
2024-11-17 16:48:54 +01:00
lebaudantoine 28899a2aaf 🚨(frontend) fix bundler warning on build
Typo while styling the homepage. Fixed it.
Minor issue.
2024-11-17 16:48:54 +01:00
lebaudantoine a5bc66a921 💄(frontend) adjust title's responsiveness
Needed for small and phone screen.
Not a huge fan of having breaking point to manage, but
made an exception for the landing page content.
2024-11-17 16:48:54 +01:00
lebaudantoine adb09410da 💄(frontend) introduce a new breaking point
Needed a trade off between xs and sm breaking points.
Introduce xsm, I am not very satisfied with this naming.
We might refactor xs in xss. Not urgent.
2024-11-17 16:48:54 +01:00
lebaudantoine 061f12e7e2 💄(frontend) update feedback button to banner with context
Refactored the feedback alert button into a more discreet and polished banner.
The banner provides additional context about Visio being under construction.
Also updated the link to the feedback form.
2024-11-17 15:29:26 +01:00
lebaudantoine 24e819a533 🩹(frontend) use browser language if no localStorage for detection
Previously, language detection was failing to read browser settings correctly.
Added explicit detection order to ensure localStorage preferences are checked
before falling back to browser language.
2024-11-17 14:56:32 +01:00
lebaudantoine 5c48ac100a (frontend) allow user register to beta feature
I've created a beta form, allowing interested visitors to
register to be a beta user.

This form is yet created with Grist, could be enhance. I'll
investigate using PostHog.
2024-11-17 01:08:13 +01:00
lebaudantoine 4434f0265d 💫(frontend) add fadein animation on home
Elements' apparition felt quite harsh.
Add an animation to fade in content.
Used a pre-defined animation in panda configs.
2024-11-17 01:08:13 +01:00
lebaudantoine 0522696842 💫(frontend) add animation on interaction with slider
Add a minimal animation to enhance micro-interaction with our app.
I've tried being minimal, please feel free to enhance or tune this
animation.
2024-11-17 01:08:13 +01:00
lebaudantoine 8516782d79 (frontend) add slide feature in the IntroSlider
Avoided installing a dependency for such small piece of code.
I've implemented a naive component, which allow users explore
the slide presenting key feature of our app.

User experience should be ok. However, I might need to optimize
image format and loading strategy. First 'raw' iteration, gonna
optimize it in the future.
2024-11-17 01:08:13 +01:00
lebaudantoine b06880be15 💄(frontend) enhance home page
I've totally rework the homepage, heavily inspired by GMeet.
Goal: make it more pro, and polished.
2024-11-17 01:08:13 +01:00
lebaudantoine 5de9cec688 (frontend) introduce learn more link component
Component providing a 'learn more' feature. Inspired by GMeet.
Naive component.
2024-11-17 01:08:13 +01:00
lebaudantoine 7dfd86873c 🚸(frontend) update A component hover animation
I found original to hide text decoration on hover. It felt wrong
to me, I proposed a new animation. Please feel free to rework it
or enhance it in the future.
2024-11-17 01:08:13 +01:00
lebaudantoine 7cee409d22 🚧(frontend) add intro slider component with feature slides
Created a component for an intro slider to showcase app features.
Each slide will include a minimal illustration and concise descriptions.

Work In Progress, Slide logic will be added in upcomming commits.
2024-11-17 01:08:13 +01:00
lebaudantoine 7ddc448c88 💄(frontend) remove bottom border on header
The bottom border on the header was unnecessary
and removed for a cleaner design.
2024-11-17 01:08:13 +01:00
lebaudantoine 6fcb69bd3c 🩹(frontend) fix horizontal scroll issue
Oooopsie while merging chat, I forgot to test horizontal scrolling.
It ended up creating a scroll issue. Fixed!

This fix might create a new issue for mobile users, I'll refactor
their layout ASAP.
2024-11-16 00:29:58 +01:00
lebaudantoine 6aed4cb751 🚸(frontend) auto-populate prejoin screen with user's default full name
Streamline user joining a room by pre-fillig name field from
their ProConnect data. Reduces friction and typos in the join flow.

This feature will be mostly used by new users. Recurrent users have
their previous choices persisted in local storage.
2024-11-16 00:29:58 +01:00
lebaudantoine f433f59d3f ♻️(frontend) prioritize full name display before email
Why? user friendly, and more human-readable identifier. Should
be enhance later on, to display both info together.
2024-11-16 00:29:58 +01:00
lebaudantoine fba9910705 (frontend) declare full and short name in APIUser type
API now serialize these two data. Save them when requestion /me.
2024-11-16 00:29:58 +01:00
lebaudantoine dcba3330f7 🛂(backend) request given and usual name scopes from ProConnect
Request the necessary scopes from ProConnect service.
Update configurations in every environments.

Note: ask given_name and usual_name scopes to get users' info.

(these scopes should be granted by default by ProConnect when
requesting a client id client secret)
2024-11-16 00:29:58 +01:00
lebaudantoine 82bb5f0f8b (backend) persist OIDC first name and last name while authenticating
Inspired by @sampaccoud's eee2003 commit on impress, adapt the code to be more
Pythonic. Add basic test coverage for user name synchronization on login. User
name fields now update automatically at each login when new data is available.

Note: current logic doesn't handle the case where a user with existing names
logs in with missing first/last names - should we clear the names then?

Removing a field that was present in the initial form is not a valid update
operation.
2024-11-15 23:38:31 +01:00
lebaudantoine 0fd06ef6c0 ♻️(backend) isolate authentication tests when dealing only with email
Refactor a test to narrow down its scope to email-related updates.
2024-11-15 23:38:31 +01:00
lebaudantoine bd4dec6f27 (backend) serialize user name-related fields
Needed in the frontend. Updated existing tests accordingly.
Names are not yet saved while logging the user, it will be
added in the upcomming commits.
2024-11-15 23:38:31 +01:00
lebaudantoine a987830fb3 🗑️(backend) remove useless methods on User
While removing analytics code in 15e922f, I forgot to remove
related code in the User model, used nowwhere else. Oopsie.
Fixed it!
2024-11-15 23:38:31 +01:00
lebaudantoine 7f09636791 (backend) add full_name short_name on User model
Following @sampaccoud's work on impress, add new fields to handle
user names in our application.

@sampaccoud preferred having a full and short names instead of
a basic first and last ones, to follow common good practices, and
avoid having frontend formating names (from my understanding).

Please see commit eee20033 on Impress.
2024-11-15 23:38:31 +01:00
lebaudantoine 3460ec8808 ✏️(backend) fix minor typo
login is a noun, the verb needs a whitespace.
2024-11-15 23:38:31 +01:00
lebaudantoine 86cda75b39 🐛(frontend) tune connection parameters for low bandwidth clients
DINUM users face connection issues in bandwidth-constrained environments.
They cannot reach their room, the peer connection timeouts while negotiating
TURN/TLS handshake (from our current understanding).

I am not sure this issue is linked to those parameters, at least try something.
2024-11-15 23:38:31 +01:00
lebaudantoine 7afa165013 (backend) offer an endpoint to save recording
I've protected this endpoint with a feature flag, and an authentication
class, as it will be exposed on the public internet.

I've tried to keep the viewset logic as minimal as possible, I've
to ship smth and will continue iterating on this piece of code.

At some point, abstracting webhook endpoint and authentication class
might be beneficial for the project. YAGNI as of today.
2024-11-13 19:36:17 +01:00
lebaudantoine e11bdc6d28 ♻️(backend) update is_savable method
A recording is savable only if it's active or stopped. In other
status (error, already saved, etc.) it won't be possible. I might
iterate on this piece of code. Let's ship it.
2024-11-13 19:36:17 +01:00
lebaudantoine 8309545ec6 (backend) add minio event parser
When a new file is uploaded to a Minio Bucket, a webhook can be
configured to notify third parties about the event. Basically,
it's a POST call with a payload providing informations on the
event that just happened.

When a recording worker will stop, it will upload its data to a Minio
bucket, which will trigger the webhook.

Try to introduce the minimalest code to parse these events, discard
them whener it's relevant, and extract the recording ID, thus we
know which recording was successfully saved to the Minio bucket.

In the longer runner, it will trigger a callback.
2024-11-13 19:36:17 +01:00
lebaudantoine 840033fcbc 🔧(frontend) get configs related to the recording feature
Get optional config related to the recording feature. RecordingMode will
be refactored as soon I start working on the recording feature.
2024-11-13 19:23:34 +01:00
lebaudantoine 28ca2d6c37 (backend) expose recording configurations to the client
Inform frontend code, if recording a room is enabled, and which recordings modes
are available. Frontend should adapt its behavior based on this data.
2024-11-13 19:23:34 +01:00
lebaudantoine 4e77458116 🚨(backend) fix Django deprecation warning in RecordingFactory
Addressed a `DeprecationWarning` in `RecordingFactory` related to the
`_after_postgeneration` method, which will stop saving the instance after
postgeneration hooks in the next major release. To resolve this,
`skip_postgeneration_save=True` was added to `RecordingFactory.Meta` to
avoid extraneous save calls. Alternatively, if instance saving is needed,
the save call can be moved to postgeneration hooks or by overriding
`after_postgeneration`.
2024-11-13 18:34:16 +01:00
lebaudantoine d4532eeb64 ♻️(backend) remove unnecessary manipulation of the room name
Avoided unnecessary manipulation of the room name to prevent issues when
starting an egress worker. Previously, hyphens were stripped from the room
name, likely inherited from the legacy setup with Jitsi in Magnify, though
the purpose of this change is unclear and might be an undesired legacy
feature.

To ensure accurate room matching during egress worker requests, this update
removes any manipulation of the room name. This approach minimizes the risk
of errors when initiating recordings and maintains the integrity of the
original room name throughout the process.
2024-11-13 18:34:16 +01:00
lebaudantoine b84628ee95 (backend) add two new endpoints to start and stop a recording
The LiveKit egress worker interactions are proxied through the backend for
security reasons. Allowing clients to directly use tokens with sufficient
grants to start recordings could lead to misuse, enabling users to spam the
egress worker API and potentially initiate a DDOS attack on the egress
service. To prevent this, only users with room-specific privileges can
initiate recordings.

We make sure only one recording at the time can be made on a room.

The requested recording mode is stored so it can be referenced later when
the recording is saved, triggering a callback action as needed.

A feature flag was also introduced for this capability; while this is a simple
approach, a more robust system for managing feature flags could be valuable
long-term. For now, KISS (Keep It Simple, Stupid) applies.

The viewset endpoints were designed to be as straightforward as possible—
let me know if anything can be improved.
2024-11-13 18:34:16 +01:00
lebaudantoine f6f1222f47 (backend) introduce general recording worker concepts
Introducing a new worker service architecture. Sorry for the long commit.
This design adheres to several key principles, primarily the  Single
Responsibility Principle. Dependency Injection and composition are
prioritized over inheritance, enhancing modularity and maintainability.

Interactions between the backend and external workers are encapsulated in
classes implementing a common `WorkerService` interface. I chose Protocol
over an abstract class for agility, aligning closely with static typing
without requiring inheritance. Each `WorkerService` implementation can
independently manage recordings according to its specific requirements.
This flexibility ensures that adding a new worker service, such as for
LiveKit, can be done without any change to existing components.

Configuration management is centralized in a single `WorkerServiceConfig`
class, which loads and provides all settings for different worker
implementations, keeping configurations organized and extensible. The worker
service class itself handles accessing relevant configurations as needed,
simplifying the configuration process.

A basic dictionary in Django settings acts as a factory, responsible for
instantiating the correct worker service based on the client's request mode.
This approach aligns with Django development conventions, emphasizing
simplicity. While a full factory class with a builder pattern could provide
future flexibility, YAGNI (You Aren't Gonna Need It) suggests deferring
such complexity until it’s necessary.

At the core of this design is the worker mediator, which decouples worker
service implementations from the Django ORM and manages database state
according to worker state. The mediator is purposefully limited in
responsibility, handling only what’s essential. It doesn’t instantiate
worker services directly; instead, services are injected via composition,
allowing the mediator to manage any object conforming to the `WorkerService`
interface. This setup preserves flexibility and maintains a clear
separation of responsibilities. The factory create worker services,
the mediator runs it.

(sorry for this long commit)
2024-11-13 18:34:16 +01:00
lebaudantoine 7278613b20 🗃️(backend) merge duplicate user accounts on email
Write the proper ORM code to sanitize the rows in db and avoid
existing users lose access to our app.

Existing duplicate user accounts are merged, and resource accesses
are transferred.
2024-11-12 16:56:58 +01:00
lebaudantoine d370a4db10 🐛(backend) harden email matching against ambiguous cases
Handle case-sensitivity and whitespace in email lookups. Detect and block
multiple matching accounts as security precaution.
2024-11-12 16:56:58 +01:00
lebaudantoine c1bc379744 🧪(backend) add test for email matching
Add test cases for email-based user matching fallback logic:
- String comparison edge cases
- Multiple users with matching email addresses
- Invalid email format handling

Fix will follow in subsequent commit.
2024-11-12 16:56:58 +01:00
lebaudantoine 5ef6359b7c 🛂(backend) fallback to email matching when OIDC sub is not found
When OIDC providers return random values in the "sub" field instead of stable
identifiers, implement email-based user matching as fallback.

Note: Current implementation needs improvement. Tests forthcoming.

Original: @sampaccoud (ff7914f) on Impress
2024-11-12 16:56:58 +01:00
lebaudantoine 04d76acce5 (backend) handle inactive user
Handle case where user is inactive.
Previously this edge case would cause unexpected behavior.

Related to previous commit that added the test coverage.
2024-11-12 16:56:58 +01:00
lebaudantoine eeb71f90bc 🧪(backend) add test for inactive user
Add failing test for case when user is inactive.
This case was highlighted by @qbey and was previously untested.
Fix will follow in subsequent commit.
2024-11-12 16:56:58 +01:00
lebaudantoine 5db4b09106 ♻️(backend) remove redundant create_user method
Remove redundant wrapper method that duplicates SUB validation logic.
Already validated in parent class, following the pattern established by
@sampaccoud in People and Impress modules.
2024-11-12 16:56:58 +01:00
lebaudantoine 11cd85d4eb (backend) handle empty subscription string
Handle case where sub value is an empty string instead of None.
Previously this edge case would cause unexpected behavior.

Related to previous commit that added the test coverage.
2024-11-12 16:56:58 +01:00
lebaudantoine ccbeeba68f 🧪(backend) add test for empty sub string
Add failing test for corner case when sub value is an empty string.
This edge case was discovered by @sampaccoud and was previously untested.
Fix will follow in subsequent commit.
2024-11-12 16:56:58 +01:00
lebaudantoine f3ea0fca71 🐛(frontend) fix missing user label in prejoin screen
Add missing hint to guide user entering their name while
joining a room.

I introduced this bug while merging @manuhabitela works on
the newly prejoin screen.
2024-11-11 21:20:08 +01:00
lebaudantoine cb4acc32e5 🗑️(backend) remove useless analytics settings
Remove useless analytics key settings as we removed tracking
data from the backend code.
2024-11-08 10:45:58 +01:00
lebaudantoine 60f5c8486b 🩹(backend) add missing Django settings for cold storage access
After configuring the cold storage infrastructure, I forgot to add Django
settings needed to access the configuration.
2024-11-08 10:45:58 +01:00
lebaudantoine ed3a26d449 (backend) enable Django Admin on Recording
Manage Recording in the Django Admin. As of today, I have not enforced
a strict policy to avoid edit on recording rows or even creating new
data point directly from the admin. Will do in the future.
2024-11-08 10:36:38 +01:00
lebaudantoine cb4c058c5d (backend) add minimal Recording viewset for room recordings
Implements routes to manage recordings within rooms, following the patterns
established in Impress. The viewset exposes targeted endpoints rather than
full CRUD operations, with recordings being created (soon) through
room-specific routes (e.g. room/123/start-recording).

The implementation draws from @sampaccoud's initial work and advices.

Review focus areas:
- Permission implementation choices
- Serializer design and structure

Credit: Initial work by @sampaccoud
2024-11-08 10:36:31 +01:00
lebaudantoine c504b5262b (backend) introduce Recording model with independent access control
The Recording model is introduced to track recording lifecycle within rooms,
while maintaining strict separation of access controls between rooms and
recordings.

Recordings follow the BaseAccess pattern (similar to Documents in Impress),
providing independent access control from room permissions. This ensures that
joining a room doesn't automatically grant access to previous recordings,
allowing for more flexible permission management.

The implementation was driven by TDD, particularly for the get_abilities
function, resulting in reduced nesting levels and improved readability.

The Recording model is deliberately kept minimal to serve as a foundation for
upcoming AI features while maintaining flexibility for future extensions.

I have avoided LiveKit-specific terminology for better abstraction.

Note: Room access control remains unchanged in this commit, pending future
refactor to use BaseAccess pattern (discussed IRL with @sampaccoud).
2024-11-07 18:06:26 +01:00
lebaudantoine 3b3816b333 ♻️(backend) rename room-specific setting in Resource class
Renames a Django setting in the Resource base class to use resource-agnostic
terminology. While rooms are currently the only resource type, keeping base
class naming generic improves clarity.
2024-11-07 18:06:26 +01:00
lebaudantoine 15e922f9df 🔥(backend) vendor analytics code
Analytics code is now useless, we mostly use
frontend tracking.
2024-11-04 17:49:15 +01:00
lebaudantoine b69f777e5a 🔧(helm) expose MinIO console via ingress
Add ingress configuration for MinIO web console access. Include Nginx proxy
buffer annotations to support large file uploads through web interface.
2024-11-04 15:24:58 +01:00
lebaudantoine 67d004fbda ♻️(backend) refactor try/except when getting a room
Be more Pythonist simplifying try except while tracking when
user is getting a room.
2024-11-04 15:21:24 +01:00
lebaudantoine 4449b578bd 🐛(config) rename hardcoded docker-compose filename
Previously, the Docker Compose filename was hardcoded in _config.sh when used
through utility scripts. In recent commits, I've renamed the filename without
updating this configuration.

Oopsie, running make commands was fine, but running bin scripts
requiring compose failed.
2024-11-04 14:17:31 +01:00
lebaudantoine 1f0d2ce335 🚨(backend) fix Pylint warning
Python module models.py raises too-many-ancestors warning, but linter
still pass, code is rated 10/10.

Inspired from @sampaccoud's commit #eee20032 on impress.
2024-11-04 14:17:31 +01:00
lebaudantoine 0d1d0662ee ♻️(compose) rename docker-compose to compose
Docker Compose now by default search for a 'compose' file.
(according to @jonathanperret).
2024-11-03 22:31:30 +01:00
lebaudantoine 0056b4cc29 🔥(compose) remove docker compose version
The version is now automatically guessed by Docker Compose. This
commit will fix the warning raised about the uselessness of this
setting.

from @sampaccoud
2024-11-03 22:31:30 +01:00
lebaudantoine d7892cde9f (backend) move freezegun to dev dependencies
Freezegun is for testing and should not be installed in the
production image. (from @sampaccoud)
2024-11-03 22:31:30 +01:00
lebaudantoine 372db49e94 🎨(docker) convert to uppercase 'as' keyword
Match 'FROM' casing, to remove a warning.
2024-11-03 22:31:30 +01:00
lebaudantoine f7ed70dc9c (livekit) add Livekit Egress
Egress is already deployed in staging. But, while
working locally on feature relying on Egress, it's not
suitable to test your development or iterate.

Especially I'll need to test the connection between the Egress
and the minio bucket in my next PR.

We faced quite a few issue while starting the whole stack.
Egress didn't want to start. Its connection with the livekit server
while the egress participant was joining the room was not successful.

The Turn part of the livekit server helm chart was activated. We needed
to update few values to in the helm configuration to enabled this turn.

Updated CoreDNS to expose Egress pod. Egress tries connecting to MinIO at
127.0.0.1, where no instance exists. Using minio.127.0.0.1.nip.io resolves
to 127.0.0.1, causing Egress to connect to itself for uploads. The CoreDNS
rewrite directs this to the Ingress IP, correctly routing to MinIO.
2024-10-28 10:26:51 +01:00
lebaudantoine 427b23ca80 (backend) add S3 objects
Inspired by Impress and @sampaccoud's work.

We use Indie Hoster’s Kubernetes objects in staging and production.
In the "dev" environment, we install the `bitnami/minio` chart to mimic
Indie Hoster’s MinIO setup.

To access the MinIO admin interface in dev, use port forwarding;
the interface runs on port 9001.
2024-10-28 10:26:51 +01:00
lebaudantoine 99a1efc538 🐛(helm) deploy livekit in 'meet' namespace
LiveKit Helm chart doesn't support namespace parameterization
like MinIO templates. Egress and server deploy to default namespace,
but frontend and backend need all components in the same namespace.

Workaround: force kubectl context to 'meet' before starting Tilt.
Future improvement needed in chart.

Issue #105 have been opened in livekit-helm repo.
2024-10-28 10:26:51 +01:00
237 changed files with 12749 additions and 3823 deletions
+47
View File
@@ -117,10 +117,57 @@ jobs:
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-summary:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "meet,secrets"
-
name: Checkout repository
uses: actions/checkout@v4
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/meet/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/meet-summary
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
-
name: Build and push
uses: docker/build-push-action@v6
with:
context: ./src/summary
file: ./src/summary/Dockerfile
target: production
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
notify-argocd:
needs:
- build-and-push-frontend
- build-and-push-backend
- build-and-push-summary
runs-on: ubuntu-latest
if: |
github.event_name != 'pull_request'
+7 -7
View File
@@ -1,7 +1,7 @@
# Django Meet
# ---- base image to inherit from ----
FROM python:3.12.6-alpine3.20 as base
FROM python:3.12.6-alpine3.20 AS base
# Upgrade pip to its latest release to speed up dependencies installation
RUN python -m pip install --upgrade pip setuptools
@@ -11,7 +11,7 @@ RUN apk update && \
apk upgrade
# ---- Back-end builder image ----
FROM base as back-builder
FROM base AS back-builder
WORKDIR /builder
@@ -23,7 +23,7 @@ RUN mkdir /install && \
# ---- mails ----
FROM node:20 as mail-builder
FROM node:20 AS mail-builder
COPY ./src/mail /mail/app
@@ -34,7 +34,7 @@ RUN yarn install --frozen-lockfile && \
# ---- static link collector ----
FROM base as link-collector
FROM base AS link-collector
ARG MEET_STATIC_ROOT=/data/static
RUN apk add \
@@ -58,7 +58,7 @@ RUN DJANGO_CONFIGURATION=Build DJANGO_JWT_PRIVATE_SIGNING_KEY=Dummy \
RUN rdfind -makesymlinks true -followsymlinks true -makeresultsfile false ${MEET_STATIC_ROOT}
# ---- Core application image ----
FROM base as core
FROM base AS core
ENV PYTHONUNBUFFERED=1
@@ -93,7 +93,7 @@ WORKDIR /app
ENTRYPOINT [ "/usr/local/bin/entrypoint" ]
# ---- Development image ----
FROM core as backend-development
FROM core AS backend-development
# Switch back to the root user to install development dependencies
USER root:root
@@ -119,7 +119,7 @@ ENV DB_HOST=postgresql \
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
# ---- Production image ----
FROM core as backend-production
FROM core AS backend-production
ARG MEET_STATIC_ROOT=/data/static
+3
View File
@@ -305,3 +305,6 @@ start-tilt: ## start the kubernetes cluster using kind
tilt up -f ./bin/Tiltfile
.PHONY: build-k8s-cluster
start-tilt-keycloak: ## start the kubernetes cluster using kind, without Pro Connect for authentication, use keycloak
DEV_ENV=dev-keycloak tilt up -f ./bin/Tiltfile
.PHONY: build-k8s-cluster
+2
View File
@@ -118,6 +118,8 @@ $ make build-k8s-cluster
Once the Kubernetes cluster is ready, start the application stack locally:
```shell
$ make start-tilt
or
$ make start-tilt-keycloak # start stack without Pro Connect, use keycloak
```
These commands set up and run your application environment using Tilt for local Kubernetes development.
+12 -2
View File
@@ -1,7 +1,6 @@
load('ext://uibutton', 'cmd_button', 'bool_input', 'location')
load('ext://namespace', 'namespace_create', 'namespace_inject')
namespace_create('meet')
docker_build(
'localhost:5001/meet-backend:latest',
context='..',
@@ -28,7 +27,18 @@ docker_build(
]
)
k8s_yaml(local('cd ../src/helm && helmfile -n meet -e dev template .'))
docker_build(
'localhost:5001/meet-summary:latest',
context='../src/summary',
dockerfile='../src/summary/Dockerfile',
only=['.', '../../docker', '../../.dockerignore'],
target = 'production',
live_update=[
sync('../src/summary', '/home/summary'),
]
)
k8s_yaml(local('cd ../src/helm && helmfile -n meet -e ${DEV_ENV:-dev} template .'))
migration = '''
set -eu
+1 -1
View File
@@ -5,7 +5,7 @@ set -eo pipefail
REPO_DIR="$(cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd)"
UNSET_USER=0
COMPOSE_FILE="${REPO_DIR}/docker-compose.yml"
COMPOSE_FILE="${REPO_DIR}/compose.yml"
COMPOSE_PROJECT="meet"
+42 -6
View File
@@ -29,7 +29,7 @@ echo "2. Create kind cluster with containerd registry config dir enabled"
# https://github.com/kubernetes-sigs/kind/issues/2875
# https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration
# See: https://github.com/containerd/containerd/blob/main/docs/hosts.md
cat <<EOF | kind create cluster --config=-
cat <<EOF | kind create cluster --name visio --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
@@ -52,10 +52,6 @@ nodes:
- containerPort: 443
hostPort: 443
protocol: TCP
- role: worker
image: kindest/node:v1.27.3
- role: worker
image: kindest/node:v1.27.3
EOF
echo "3. Add the registry config to the nodes"
@@ -68,7 +64,7 @@ echo "3. Add the registry config to the nodes"
# We want a consistent name that works from both ends, so we tell containerd to
# alias localhost:${reg_port} to the registry container when pulling images
REGISTRY_DIR="/etc/containerd/certs.d/localhost:${reg_port}"
for node in $(kind get nodes); do
for node in $(kind get nodes --name visio); do
docker exec "${node}" mkdir -p "${REGISTRY_DIR}"
cat <<EOF | docker exec -i "${node}" cp /dev/stdin "${REGISTRY_DIR}/hosts.toml"
[host."http://${reg_name}:5000"]
@@ -97,7 +93,47 @@ data:
help: "https://kind.sigs.k8s.io/docs/user/local-registry/"
EOF
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
data:
Corefile: |
.:53 {
errors
health {
lameduck 5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus :9153
forward . /etc/resolv.conf {
max_concurrent 1000
}
rewrite stop {
name regex (.*).127.0.0.1.nip.io ingress-nginx-controller.ingress-nginx.svc.cluster.local answer auto
}
cache 30
loop
reload
loadbalance
}
EOF
kubectl -n kube-system rollout restart deployments/coredns
echo "6. Install ingress-nginx"
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
kubectl -n ingress-nginx create secret tls mkcert --key /tmp/127.0.0.1.nip.io+1-key.pem --cert /tmp/127.0.0.1.nip.io+1.pem
kubectl -n ingress-nginx patch deployments.apps ingress-nginx-controller --type 'json' -p '[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value":"--default-ssl-certificate=ingress-nginx/mkcert"}]'
echo "7. Setup namespace"
kubectl create ns meet
kubectl config set-context --current --namespace=meet
kubectl -n meet create secret generic mkcert --from-file=rootCA.pem="$(mkcert -CAROOT)/rootCA.pem"
-1
View File
@@ -1,4 +1,3 @@
version: '3.8'
services:
postgresql:
+405 -405
View File
File diff suppressed because it is too large Load Diff
+1 -1
Submodule secrets updated: 2ef2610071...2ba12db71d
+2 -2
View File
@@ -447,10 +447,10 @@ max-bool-expr=5
max-branches=12
# Maximum number of locals for function / method body
max-locals=15
max-locals=20
# Maximum number of parents for a class (see R0901).
max-parents=7
max-parents=10
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
+48 -4
View File
@@ -22,7 +22,19 @@ class UserAdmin(auth_admin.UserAdmin):
)
},
),
(_("Personal info"), {"fields": ("sub", "email", "language", "timezone")}),
(
_("Personal info"),
{
"fields": (
"sub",
"email",
"full_name",
"short_name",
"language",
"timezone",
)
},
),
(
_("Permissions"),
{
@@ -52,6 +64,8 @@ class UserAdmin(auth_admin.UserAdmin):
"sub",
"admin_email",
"email",
"full_name",
"short_name",
"is_active",
"is_staff",
"is_superuser",
@@ -60,9 +74,24 @@ class UserAdmin(auth_admin.UserAdmin):
"updated_at",
)
list_filter = ("is_staff", "is_superuser", "is_device", "is_active")
ordering = ("is_active", "-is_superuser", "-is_staff", "-is_device", "-updated_at")
readonly_fields = ("id", "sub", "email", "created_at", "updated_at")
search_fields = ("id", "sub", "admin_email", "email")
ordering = (
"is_active",
"-is_superuser",
"-is_staff",
"-is_device",
"-updated_at",
"full_name",
)
readonly_fields = (
"id",
"sub",
"email",
"full_name",
"short_name",
"created_at",
"updated_at",
)
search_fields = ("id", "sub", "admin_email", "email", "full_name")
class ResourceAccessInline(admin.TabularInline):
@@ -77,3 +106,18 @@ class RoomAdmin(admin.ModelAdmin):
"""Room admin interface declaration."""
inlines = (ResourceAccessInline,)
class RecordingAccessInline(admin.TabularInline):
"""Inline admin class for recording accesses."""
model = models.RecordingAccess
extra = 0
@admin.register(models.Recording)
class RecordingAdmin(admin.ModelAdmin):
"""Recording admin interface declaration."""
inlines = (RecordingAccessInline,)
list_display = ("id", "status", "room", "created_at", "worker_id")
-58
View File
@@ -1,58 +0,0 @@
"""
Meet analytics class.
"""
import uuid
from django.conf import settings
from june import analytics as jAnalytics
class Analytics:
"""Analytics integration
This class wraps the June analytics code to avoid coupling our code directly
with this third-party library. By doing so, we create a generic interface
for analytics that can be easily modified or replaced in the future.
"""
def __init__(self):
key = getattr(settings, "ANALYTICS_KEY", None)
if key is not None:
jAnalytics.write_key = key
self._enabled = key is not None
def _is_anonymous_user(self, user):
"""Check if the user is anonymous."""
return user is None or user.is_anonymous
def identify(self, user, **kwargs):
"""Identify a user"""
if self._is_anonymous_user(user) or not self._enabled:
return
traits = kwargs.pop("traits", {})
traits.update({"email": user.email_anonymized})
jAnalytics.identify(user_id=user.sub, traits=traits, **kwargs)
def track(self, user, **kwargs):
"""Track an event"""
if not self._enabled:
return
event_data = {}
if self._is_anonymous_user(user):
event_data["anonymous_id"] = str(uuid.uuid4())
else:
event_data["user_id"] = user.sub
jAnalytics.track(**event_data, **kwargs)
analytics = Analytics()
+4
View File
@@ -37,6 +37,10 @@ def get_frontend_configuration(request):
"""Returns the frontend configuration dict as configured in settings."""
frontend_configuration = {
"LANGUAGE_CODE": settings.LANGUAGE_CODE,
"recording": {
"is_enabled": settings.RECORDING_ENABLE,
"available_modes": settings.RECORDING_WORKER_CLASSES.keys(),
},
}
frontend_configuration.update(settings.FRONTEND_CONFIGURATION)
return Response(frontend_configuration)
-316
View File
@@ -1,316 +0,0 @@
from rest_framework.decorators import api_view
from rest_framework.response import Response
from minio import Minio
from django.conf import settings
import openai
import logging
from ..models import Room, RoleChoices
import tempfile
import os
import smtplib
import requests
logger = logging.getLogger(__name__)
def get_prompt(transcript, date):
return f"""
Audience: Coworkers.
**Do:**
- Detect the language of the transcript and provide your entire response in the same language.
- If any part of the transcript is unclear or lacks detail, politely inform the user, specifying which areas need further clarification.
- Ensure the accuracy of all information and refrain from adding unverified details.
- Format the response using proper markdown and structured sections.
- Be concise and avoid repeating yourself between the sections.
- Be super precise on nickname
- Be a nit-picker
- Auto-evaluate your response
**Don't:**
- Write something your are not sure.
- Write something that is not mention in the transcript.
- Don't make mistake while mentioning someone
**Task:**
Summarize the provided meeting transcript into clear and well-organized meeting minutes. The summary should be structured into the following sections, excluding irrelevant or inapplicable details:
1. **Summary**: Write a TL;DR of the meeting.
2. **Subjects Discussed**: List the key points or issues in bullet points.
4. **Next Steps**: Provide action items as bullet points, assigning each task to a responsible individual and including deadlines (if mentioned). Format action items as tickable checkboxes. Ensure every action is assigned and, if a deadline is provided, that it is clearly stated.
**Transcript**:
{transcript}
**Response:**
### Summary [Translate this title based on the transcripts language]
[Provide a brief overview of the key points discussed]
### Subjects Discussed [Translate this title based on the transcripts language]
- [Summarize each topic concisely]
### Next Steps [Translate this title based on the transcripts language]
- [ ] Action item [Assign to the responsible individual(s) and include a deadline if applicable, follow this strict format: Action - List of owner(s), deadline.]
"""
def get_room_and_owners(slug):
"""Wip."""
try:
room = Room.objects.get(slug=slug)
owner_accesses = room.accesses.filter(role=RoleChoices.OWNER)
owners = [access.user for access in owner_accesses]
logger.info("Room %s has owners: %s", slug, owners)
except Room.DoesNotExist:
logger.error("Room with slug %s does not exist", slug)
owners = None
room = None
return room, owners
def remove_temporary_file(path):
"""Wip."""
if not path or not os.path.exists(path):
return
os.remove(path)
logger.info("Temporary file %s has been deleted.", path)
def get_blocknote_content(summary):
"""Wip."""
if not settings.BLOCKNOTE_CONVERTER_URL:
logger.error("BLOCKNOTE_CONVERTER_URL is not configured")
return None
headers = {
"Content-Type": "application/json"
}
data = {
"markdown": summary
}
logger.info("Converting summary in BlockNote.js…")
response = requests.post(settings.BLOCKNOTE_CONVERTER_URL, headers=headers, json=data)
if response.status_code != 200:
logger.error(f"Failed to convert summary. Status code: {response.status_code}")
return None
response_data = response.json()
if not 'content' in response_data:
logger.error(f"Content is missing: %s", response_data)
return None
content = response_data['content']
logger.info("Base64 content: %s", content)
return content
def get_document_link(content, email):
"""Wip."""
logger.info("Create a document for %s", email)
if not settings.DOCS_BASE_URL:
logger.error("DOCS_BASE_URL is not configured")
return None
headers = {
"Content-Type": "application/json"
}
data = {
"content": content,
"owner": email
}
logger.info("Querying docs…")
response = requests.post(f"{settings.DOCS_BASE_URL}/api/v1.0/summary/", headers=headers, json=data)
if response.status_code != 200:
logger.error(f"Failed to get document's id. Status code: {response.status_code}")
return None
response_data = response.json()
if not 'id' in response_data:
logger.error(f"ID is missing: %s", response_data)
return None
id = response_data['id']
logger.info("Document's id: %s", id)
return f"{settings.DOCS_BASE_URL}/docs/{id}/"
def email_owner_with_summary(room, link, owner):
"""Wip."""
logger.info("Emailing owner: %s", owner)
try:
room.email_summary(owners=[owner], link=link)
except smtplib.SMTPException:
logger.error("Error while emailing owner")
def strip_room_slug(filename):
"""Wip."""
return filename.split("_")[2].split(".")[0]
def strip_room_date(filename):
"""Wip."""
return filename.split("_")[1].split(".")[0]
def get_minio_client():
"""Wip."""
try:
return Minio(
settings.MINIO_URL,
access_key=settings.MINIO_ACCESS_KEY,
secret_key=settings.MINIO_SECRET_KEY,
)
except Exception as e:
logger.error("An error occurred while creating the Minio client %s: %s", settings.MINIO_URL, str(e))
def download_temporary_file(minio_client, filename):
"""Wip."""
temp_file_path = None
logger.info('downloading file %s', filename)
try:
audio_file_stream = minio_client.get_object(settings.MINIO_BUCKET, object_name=filename)
with tempfile.NamedTemporaryFile(delete=False, suffix='.ogg') as temp_audio_file:
for data in audio_file_stream.stream(32 * 1024):
temp_audio_file.write(data)
temp_file_path = temp_audio_file.name
logger.info('Temporary file created at %s', temp_file_path)
audio_file_stream.close()
audio_file_stream.release_conn()
except Exception as e:
logger.error("An error occurred while accessing the object: %s", str(e))
return temp_file_path
# todo - discuss retry policy if the webhook fail
@api_view(["POST"])
def minio_webhook(request):
data = request.data
logger.info('Minio webhook sent %s', data)
record = data["Records"][0]
s3 = record['s3']
bucket = s3['bucket']
bucket_name = bucket['name']
object = s3['object']
filename = object['key']
if bucket_name != settings.MINIO_BUCKET:
logger.info('Not interested in this bucket: %s', bucket_name)
return Response("Not interested in this bucket")
if object['contentType'] != 'audio/ogg':
logger.info('Not interested in this file type: %s', object['contentType'])
return Response("Not interested in this file type")
room_slug = strip_room_slug(filename)
room_date = strip_room_date(filename)
logger.info('file received %s for room %s', filename, room_slug)
minio_client = get_minio_client()
temp_file_path = None
summary = None
try:
temp_file_path = download_temporary_file(minio_client, filename)
if settings.OPENAI_ENABLE and temp_file_path:
logger.info('Initiating OpenAI client …')
openai_client = openai.OpenAI(
api_key=settings.OPENAI_API_KEY,
)
with open(temp_file_path, "rb") as audio_file:
logger.info('Querying transcription …')
transcript = openai_client.audio.transcriptions.create(
model="whisper-1",
file=audio_file
)
logger.info('Transcript: \n %s', transcript)
prompt = get_prompt(transcript.text, room_date)
logger.info('Prompt: \n %s', prompt)
summary_response = openai_client.chat.completions.create(
model="gpt-4o",
messages=[
{"role": "system", "content": "You are a concise and structured assistant, that summarizes meeting transcripts."},
{"role": "user", "content": prompt}
],
)
summary = summary_response.choices[0].message.content
logger.info('Summary: \n %s', summary)
except Exception as e:
logger.error("An error occurred: %s", str(e))
raise
finally:
remove_temporary_file(temp_file_path)
if not summary:
logger.error("Empty summary.")
return Response("")
room, owners = get_room_and_owners(room_slug)
if not owners or not room:
logger.error("No owners in room %s", room_slug)
return Response("")
content = get_blocknote_content(summary)
if not content:
logger.error("Empty content.")
return Response("")
owner = owners[0]
link = get_document_link(content, owner.email)
if not link:
logger.error("Empty link.")
return Response("")
email_owner_with_summary(room, link, owner)
return Response("")
+41 -5
View File
@@ -1,5 +1,7 @@
"""Permission handlers for the Meet core app."""
from django.conf import settings
from rest_framework import permissions
from ..models import RoleChoices
@@ -65,15 +67,11 @@ class RoomPermissions(permissions.BasePermission):
return obj.is_administrator(user)
class ResourceAccessPermission(permissions.BasePermission):
class ResourceAccessPermission(IsAuthenticated):
"""
Permissions for a room that can only be updated by room administrators.
"""
def has_permission(self, request, view):
"""Only allow authenticated users."""
return request.user.is_authenticated
def has_object_permission(self, request, view, obj):
"""
Check that the logged-in user is administrator of the linked room.
@@ -83,3 +81,41 @@ class ResourceAccessPermission(permissions.BasePermission):
return obj.user == user
return obj.resource.is_administrator(user)
class HasAbilityPermission(IsAuthenticated):
"""Permission class for access objects."""
def has_object_permission(self, request, view, obj):
"""Check permission for a given object."""
return obj.get_abilities(request.user).get(view.action, False)
class HasPrivilegesOnRoom(IsAuthenticated):
"""Check if user has privileges on a given room."""
message = "You must have privileges to start a recording."
def has_object_permission(self, request, view, obj):
"""Determine if user has privileges on room."""
return obj.is_owner(request.user) or obj.is_administrator(request.user)
class IsRecordingEnabled(permissions.BasePermission):
"""Check if the recording feature is enabled."""
message = "Access denied, recording is disabled."
def has_permission(self, request, view):
"""Determine if access is allowed based on settings."""
return settings.RECORDING_ENABLE
class IsStorageEventEnabled(permissions.BasePermission):
"""Check if the storage event feature is enabled."""
message = "Access denied, storage event is disabled."
def has_permission(self, request, view):
"""Determine if access is allowed based on settings."""
return settings.RECORDING_STORAGE_EVENT_ENABLE
+49 -7
View File
@@ -14,8 +14,8 @@ class UserSerializer(serializers.ModelSerializer):
class Meta:
model = models.User
fields = ["id", "email"]
read_only_fields = ["id", "email"]
fields = ["id", "email", "full_name", "short_name"]
read_only_fields = ["id", "email", "full_name", "short_name"]
class ResourceAccessSerializerMixin:
@@ -87,6 +87,15 @@ class NestedResourceAccessSerializer(ResourceAccessSerializer):
user = UserSerializer(read_only=True)
class ListRoomSerializer(serializers.ModelSerializer):
"""Serialize Room model for a list API endpoint."""
class Meta:
model = models.Room
fields = ["id", "name", "slug", "is_public"]
read_only_fields = ["id", "slug"]
class RoomSerializer(serializers.ModelSerializer):
"""Serialize Room model for the API."""
@@ -121,18 +130,51 @@ class RoomSerializer(serializers.ModelSerializer):
del output["configuration"]
if role is not None or instance.is_public:
slug = f"{instance.id!s}".replace("-", "")
room_id = f"{instance.id!s}"
username = request.query_params.get("username", None)
output["livekit"] = {
"url": settings.LIVEKIT_CONFIGURATION["url"],
"room": slug,
"room": room_id,
"token": utils.generate_token(
room=slug, user=request.user, username=username
room=room_id, user=request.user, username=username
),
"passphrase": utils.get_cached_passphrase(room_id)
}
output["is_administrable"] = is_admin
return output
class RecordingSerializer(serializers.ModelSerializer):
"""Serialize Recording for the API."""
room = ListRoomSerializer(read_only=True)
class Meta:
model = models.Recording
fields = ["id", "room", "created_at", "updated_at", "status"]
read_only_fields = fields
class StartRecordingSerializer(serializers.Serializer):
"""Validate start recording requests."""
mode = serializers.ChoiceField(
choices=models.RecordingModeChoices.choices,
required=True,
error_messages={
"required": "Recording mode is required.",
"invalid_choice": "Invalid recording mode. Choose between "
"screen_recording or transcript.",
},
)
def create(self, validated_data):
"""Not implemented as this is a validation-only serializer."""
raise NotImplementedError("StartRecordingSerializer is validation-only")
def update(self, instance, validated_data):
"""Not implemented as this is a validation-only serializer."""
raise NotImplementedError("StartRecordingSerializer is validation-only")
+222 -14
View File
@@ -1,6 +1,7 @@
"""API endpoints"""
import uuid
from logging import getLogger
from django.conf import settings
from django.db.models import Q
@@ -14,17 +15,44 @@ from rest_framework import (
pagination,
viewsets,
)
from rest_framework import (
exceptions as drf_exceptions,
)
from rest_framework import (
response as drf_response,
)
from rest_framework import (
status as drf_status,
)
from core import models, utils
from core.recording.event.authentication import StorageEventAuthentication
from core.recording.event.exceptions import (
InvalidBucketError,
InvalidFileTypeError,
ParsingEventDataError,
)
from core.recording.event.notification import notification_service
from core.recording.event.parsers import get_parser
from core.recording.worker.exceptions import (
RecordingStartError,
RecordingStopError,
)
from core.recording.worker.factories import (
get_worker_service,
)
from core.recording.worker.mediator import (
WorkerServiceMediator,
)
from ..analytics import analytics
from . import permissions, serializers
from livekit import api as livekit_api
# pylint: disable=too-many-ancestors
logger = getLogger(__name__)
class NestedGenericViewSet(viewsets.GenericViewSet):
"""
@@ -184,15 +212,12 @@ class RoomViewSet(
Allow unregistered rooms when activated.
For unregistered rooms we only return a null id and the livekit room and token.
"""
# todo - determine whether encryption is needed store a shared secret in memory or in redis
# todo - check if a secret already exists, else create one.
try:
instance = self.get_object()
analytics.track(
user=self.request.user,
event="Get Room",
properties={"slug": instance.slug},
)
except Http404:
if not settings.ALLOW_UNREGISTERED_ROOMS:
raise
@@ -241,14 +266,119 @@ class RoomViewSet(
role=models.RoleChoices.OWNER,
)
analytics.track(
user=self.request.user,
event="Create Room",
properties={
"slug": room.slug,
},
@decorators.action(
detail=True,
methods=["post"],
url_path="start-recording",
permission_classes=[
permissions.HasPrivilegesOnRoom,
permissions.IsRecordingEnabled,
],
)
def start_room_recording(self, request, pk=None): # pylint: disable=unused-argument
"""Start recording a room."""
serializer = serializers.StartRecordingSerializer(data=request.data)
if not serializer.is_valid():
return drf_response.Response(
{"detail": "Invalid request."}, status=drf_status.HTTP_400_BAD_REQUEST
)
mode = serializer.validated_data["mode"]
room = self.get_object()
# May raise exception if an active or initiated recording already exist for the room
recording = models.Recording.objects.create(room=room, mode=mode)
models.RecordingAccess.objects.create(
user=self.request.user, role=models.RoleChoices.OWNER, recording=recording
)
worker_service = get_worker_service(mode=recording.mode)
worker_manager = WorkerServiceMediator(worker_service=worker_service)
try:
worker_manager.start(recording)
except RecordingStartError:
return drf_response.Response(
{"error": f"Recording failed to start for room {room.slug}"},
status=drf_status.HTTP_500_INTERNAL_SERVER_ERROR,
)
return drf_response.Response(
{"message": f"Recording successfully started for room {room.slug}"},
status=drf_status.HTTP_201_CREATED,
)
@decorators.action(
detail=True,
methods=["post"],
url_path="stop-recording",
permission_classes=[
permissions.HasPrivilegesOnRoom,
permissions.IsRecordingEnabled,
],
)
def stop_room_recording(self, request, pk=None): # pylint: disable=unused-argument
"""Stop room recording."""
room = self.get_object()
try:
recording = models.Recording.objects.get(
room=room, status=models.RecordingStatusChoices.ACTIVE
)
except models.Recording.DoesNotExist as e:
raise drf_exceptions.NotFound(
"No active recording found for this room."
) from e
worker_service = get_worker_service(mode=recording.mode)
worker_manager = WorkerServiceMediator(worker_service=worker_service)
try:
worker_manager.stop(recording)
except RecordingStopError:
return drf_response.Response(
{"error": f"Recording failed to stop for room {room.slug}"},
status=drf_status.HTTP_500_INTERNAL_SERVER_ERROR,
)
return drf_response.Response(
{"message": f"Recording stopped for room {room.slug}."}
)
@decorators.action(
detail=False,
methods=["post"],
url_path="livekit-webhook",
permission_classes=[],
authentication_classes=[],
)
def handle_livekit_webhook(self, request, pk=None): # pylint: disable=unused-argument
"""Handle LiveKit webhook events."""
auth_token = request.headers.get("Authorization")
if not auth_token:
return drf_response.Response(
{"error": "Missing LiveKit authentication token"},
status=drf_status.HTTP_401_UNAUTHORIZED
)
token_verifier = livekit_api.TokenVerifier()
webhook_receiver = livekit_api.WebhookReceiver(token_verifier)
webhook_data = webhook_receiver.receive(request.body.decode("utf-8"), auth_token)
# Todo - livekit triggers a webhook for all events, see if we can restrict webhook to a limited number of events.
# Todo - handle Egress stopped / aborted events.
if webhook_data.event == "room_finished":
room_id = webhook_data.room.name
utils.clear_cache_passphrase(room_id)
return drf_response.Response({"message": f"Event processed"})
class ResourceAccessListModelMixin:
"""List mixin for resource access API."""
@@ -293,3 +423,81 @@ class ResourceAccessViewSet(
permission_classes = [permissions.ResourceAccessPermission]
queryset = models.ResourceAccess.objects.all()
serializer_class = serializers.ResourceAccessSerializer
class RecordingViewSet(
mixins.DestroyModelMixin,
mixins.ListModelMixin,
viewsets.GenericViewSet,
):
"""
API endpoints to access and perform actions on recordings.
"""
pagination_class = Pagination
permission_classes = [permissions.HasAbilityPermission]
queryset = models.Recording.objects.all()
serializer_class = serializers.RecordingSerializer
def get_queryset(self):
"""Restrict recordings to the user's ones."""
user = self.request.user
return (
super()
.get_queryset()
.filter(Q(accesses__user=user) | Q(accesses__team__in=user.get_teams()))
)
@decorators.action(
detail=False,
methods=["post"],
url_path="storage-hook",
authentication_classes=[StorageEventAuthentication],
permission_classes=[permissions.IsStorageEventEnabled],
)
def on_storage_event_received(self, request, pk=None): # pylint: disable=unused-argument
"""Handle incoming storage hook events for recordings."""
parser = get_parser()
try:
recording_id = parser.get_recording_id(request.data)
except ParsingEventDataError as e:
raise drf_exceptions.PermissionDenied(f"Invalid request data: {e}") from e
except InvalidBucketError as e:
raise drf_exceptions.PermissionDenied("Invalid bucket specified") from e
except InvalidFileTypeError as e:
return drf_response.Response(
{"message": f"Ignore this file type, {e}"},
)
try:
recording = models.Recording.objects.get(id=recording_id)
except models.Recording.DoesNotExist as e:
raise drf_exceptions.NotFound("No recording found for this event.") from e
if not recording.is_savable():
raise drf_exceptions.PermissionDenied(
f"Recording with ID {recording_id} cannot be saved because it is either,"
" in an error state or has already been saved."
)
# Attempt to notify external services about the recording
# This is a non-blocking operation - failures are logged but don't interrupt the flow
notification_succeeded = notification_service.notify_external_services(
recording
)
recording.status = (
models.RecordingStatusChoices.NOTIFICATION_SUCCEEDED
if notification_succeeded
else models.RecordingStatusChoices.SAVED
)
recording.save()
return drf_response.Response(
{"message": "Event processed."},
)
+95 -27
View File
@@ -1,6 +1,7 @@
"""Authentication Backends for the Meet core app."""
from django.core.exceptions import SuspiciousOperation
from django.conf import settings
from django.core.exceptions import ImproperlyConfigured, SuspiciousOperation
from django.utils.translation import gettext_lazy as _
import requests
@@ -9,8 +10,11 @@ from mozilla_django_oidc.auth import (
)
from core.models import User
from ..analytics import analytics
from core.services.marketing_service import (
ContactCreationError,
ContactData,
get_marketing_service,
)
class OIDCAuthenticationBackend(MozillaOIDCAuthenticationBackend):
@@ -68,36 +72,100 @@ class OIDCAuthenticationBackend(MozillaOIDCAuthenticationBackend):
user_info = self.get_userinfo(access_token, id_token, payload)
sub = user_info.get("sub")
if sub is None:
if not sub:
raise SuspiciousOperation(
_("User info contained no recognizable user identification")
)
try:
user = User.objects.get(sub=sub)
except User.DoesNotExist:
if self.get_settings("OIDC_CREATE_USER", True):
user = self.create_user(user_info)
else:
user = None
email = user_info.get("email")
user = self.get_existing_user(sub, email)
analytics.identify(user=user)
return user
def create_user(self, claims):
"""Return a newly created User instance."""
sub = claims.get("sub")
if sub is None:
raise SuspiciousOperation(
_("Claims contained no recognizable user identification")
claims = {
"email": email,
"full_name": self.compute_full_name(user_info),
"short_name": user_info.get(settings.OIDC_USERINFO_SHORTNAME_FIELD),
}
if not user and self.get_settings("OIDC_CREATE_USER", True):
user = User.objects.create(
sub=sub,
password="!", # noqa: S106
**claims,
)
user = User.objects.create(
sub=sub,
email=claims.get("email"),
password="!", # noqa: S106
)
if settings.SIGNUP_NEW_USER_TO_MARKETING_EMAIL:
self.signup_to_marketing_email(email)
elif not user:
return None
if not user.is_active:
raise SuspiciousOperation(_("User account is disabled"))
self.update_user_if_needed(user, claims)
return user
@staticmethod
def signup_to_marketing_email(email):
"""Pragmatic approach to newsletter signup during authentication flow.
Details:
1. Uses a very short timeout (1s) to prevent blocking the auth process
2. Silently fails if the marketing service is down/slow to prioritize user experience
3. Trade-off: May miss some signups but ensures auth flow remains fast
Note: For a more robust solution, consider using Async task processing (Celery/Django-Q)
"""
try:
marketing_service = get_marketing_service()
contact_data = ContactData(
email=email, attributes={"VISIO_SOURCE": ["SIGNIN"]}
)
marketing_service.create_contact(contact_data, timeout=1)
except (ContactCreationError, ImproperlyConfigured, ImportError):
pass
def get_existing_user(self, sub, email):
"""Fetch existing user by sub or email."""
try:
return User.objects.get(sub=sub)
except User.DoesNotExist:
if email and settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION:
try:
return User.objects.get(email__iexact=email)
except User.DoesNotExist:
pass
except User.MultipleObjectsReturned as e:
raise SuspiciousOperation(
_("Multiple user accounts share a common email.")
) from e
return None
@staticmethod
def compute_full_name(user_info):
"""Compute user's full name based on OIDC fields in settings."""
full_name = " ".join(
filter(
None,
(
user_info.get(field)
for field in settings.OIDC_USERINFO_FULLNAME_FIELDS
),
)
)
return full_name or None
@staticmethod
def update_user_if_needed(user, claims):
"""Update user claims if they have changed."""
user_fields = vars(user.__class__) # Get available model fields
updated_claims = {
key: value
for key, value in claims.items()
if value and key in user_fields and value != getattr(user, key)
}
if not updated_claims:
return
User.objects.filter(sub=user.sub).update(**updated_claims)
+2 -8
View File
@@ -22,8 +22,6 @@ from mozilla_django_oidc.views import (
OIDCLogoutView as MozillaOIDCOIDCLogoutView,
)
from ..analytics import analytics
class OIDCLogoutView(MozillaOIDCOIDCLogoutView):
"""Custom logout view for handling OpenID Connect (OIDC) logout flow.
@@ -100,10 +98,6 @@ class OIDCLogoutView(MozillaOIDCOIDCLogoutView):
logout_url = self.redirect_url
analytics.track(
user=request.user,
event="Signed Out",
)
if request.user.is_authenticated:
logout_url = self.construct_oidc_logout_url(request)
@@ -151,7 +145,7 @@ class OIDCLogoutCallbackView(MozillaOIDCOIDCLogoutView):
class OIDCAuthenticationCallbackView(MozillaOIDCAuthenticationCallbackView):
"""Custom callback view for handling the silent loging flow."""
"""Custom callback view for handling the silent login flow."""
@property
def failure_url(self):
@@ -168,7 +162,7 @@ class OIDCAuthenticationCallbackView(MozillaOIDCAuthenticationCallbackView):
class OIDCAuthenticationRequestView(MozillaOIDCAuthenticationRequestView):
"""Custom authentication view for handling the silent loging flow."""
"""Custom authentication view for handling the silent login flow."""
def get_extra_params(self, request):
"""Handle 'prompt' extra parameter for the silent login flow
+54
View File
@@ -23,6 +23,8 @@ class UserFactory(factory.django.DjangoModelFactory):
sub = factory.Sequence(lambda n: f"user{n!s}")
email = factory.Faker("email")
full_name = factory.Faker("name")
short_name = factory.Faker("first_name")
language = factory.fuzzy.FuzzyChoice([lang[0] for lang in settings.LANGUAGES])
password = make_password("password")
@@ -32,6 +34,7 @@ class ResourceFactory(factory.django.DjangoModelFactory):
class Meta:
model = models.Resource
skip_postgeneration_save = True
is_public = factory.Faker("boolean", chance_of_getting_true=50)
@@ -45,6 +48,8 @@ class ResourceFactory(factory.django.DjangoModelFactory):
else:
UserResourceAccessFactory(resource=self, user=item[0], role=item[1])
self.save()
class UserResourceAccessFactory(factory.django.DjangoModelFactory):
"""Create fake resource user accesses for testing."""
@@ -65,3 +70,52 @@ class RoomFactory(ResourceFactory):
name = factory.Faker("catch_phrase")
slug = factory.LazyAttribute(lambda o: slugify(o.name))
class RecordingFactory(factory.django.DjangoModelFactory):
"""Create fake recording for testing."""
class Meta:
model = models.Recording
skip_postgeneration_save = True
room = factory.SubFactory(RoomFactory)
status = models.RecordingStatusChoices.INITIATED
mode = models.RecordingModeChoices.SCREEN_RECORDING
worker_id = None
@factory.post_generation
def users(self, create, extracted, **kwargs):
"""Add users to recording from a given list of users with or without roles."""
if create and extracted:
for item in extracted:
if isinstance(item, models.User):
UserRecordingAccessFactory(recording=self, user=item)
else:
UserRecordingAccessFactory(
recording=self, user=item[0], role=item[1]
)
self.save()
class UserRecordingAccessFactory(factory.django.DjangoModelFactory):
"""Create fake recording user accesses for testing."""
class Meta:
model = models.RecordingAccess
recording = factory.SubFactory(RecordingFactory)
user = factory.SubFactory(UserFactory)
role = factory.fuzzy.FuzzyChoice(models.RoleChoices.values)
class TeamRecordingAccessFactory(factory.django.DjangoModelFactory):
"""Create fake recording team accesses for testing."""
class Meta:
model = models.RecordingAccess
recording = factory.SubFactory(RecordingFactory)
team = factory.Sequence(lambda n: f"team{n}")
role = factory.fuzzy.FuzzyChoice(models.RoleChoices.values)
@@ -0,0 +1,67 @@
# Generated by Django 5.1.1 on 2024-11-06 14:31
import django.db.models.deletion
import uuid
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0004_alter_user_language'),
]
operations = [
migrations.CreateModel(
name='Recording',
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('status', models.CharField(choices=[('initiated', 'Initiated'), ('active', 'Active'), ('stopped', 'Stopped'), ('saved', 'Saved'), ('aborted', 'Aborted'), ('failed_to_start', 'Failed to Start'), ('failed_to_stop', 'Failed to Stop')], default='initiated', max_length=20)),
('worker_id', models.CharField(blank=True, help_text='Enter an identifier for the worker recording.This ID is retained even when the worker stops, allowing for easy tracking.', max_length=255, null=True, verbose_name='Worker ID')),
('room', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='recordings', to='core.room', verbose_name='Room')),
],
options={
'verbose_name': 'Recording',
'verbose_name_plural': 'Recordings',
'db_table': 'meet_recording',
'ordering': ('-created_at',),
},
),
migrations.CreateModel(
name='RecordingAccess',
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created on')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated on')),
('team', models.CharField(blank=True, max_length=100)),
('role', models.CharField(choices=[('member', 'Member'), ('administrator', 'Administrator'), ('owner', 'Owner')], default='member', max_length=20)),
('recording', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='accesses', to='core.recording')),
('user', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
],
options={
'verbose_name': 'Recording/user relation',
'verbose_name_plural': 'Recording/user relations',
'db_table': 'meet_recording_access',
'ordering': ('-created_at',),
},
),
migrations.AddConstraint(
model_name='recording',
constraint=models.UniqueConstraint(condition=models.Q(('status__in', ['active', 'initiated'])), fields=('room',), name='unique_initiated_or_active_recording_per_room'),
),
migrations.AddConstraint(
model_name='recordingaccess',
constraint=models.UniqueConstraint(condition=models.Q(('user__isnull', False)), fields=('user', 'recording'), name='unique_recording_user', violation_error_message='This user is already in this recording.'),
),
migrations.AddConstraint(
model_name='recordingaccess',
constraint=models.UniqueConstraint(condition=models.Q(('team__gt', '')), fields=('team', 'recording'), name='unique_recording_team', violation_error_message='This team is already in this recording.'),
),
migrations.AddConstraint(
model_name='recordingaccess',
constraint=models.CheckConstraint(condition=models.Q(models.Q(('team', ''), ('user__isnull', False)), models.Q(('team__gt', ''), ('user__isnull', True)), _connector='OR'), name='check_recording_access_either_user_or_team', violation_error_message='Either user or team must be set, not both.'),
),
]
@@ -0,0 +1,89 @@
from django.db import migrations
from django.db.models import Count
from core.models import RoleChoices
def merge_duplicate_user_accounts(apps, schema_editor):
"""Merge user accounts that share the same email address.
Historical Context:
Previously, ProConnect authentication could return users with the same email
but different sub, leading to duplicate user accounts. While the application
now prevents this scenario, this migration is needed to clean up existing
duplicate accounts to ensure users can continue to connect without being blocked
by unique email constraints.
Performance of this migration is poor, this implementation prioritizes readability
and maintainability. Consider refactoring this code to avoid individual db queries
on each iteration.
"""
User = apps.get_model('core', 'User')
ResourceAccess = apps.get_model('core', 'ResourceAccess')
emails_with_duplicates = (
User.objects.values('email')
.annotate(count=Count('id'))
.filter(count__gt=1)
.values_list('email', flat=True)
)
for email in emails_with_duplicates:
# Keep the oldest user
primary_user = User.objects.filter(email=email).order_by('created_at').first()
duplicate_user_accounts = User.objects.filter(email=email).exclude(id=primary_user.id)
# Get IDs of duplicate accounts to be merged
duplicate_account_ids = list(duplicate_user_accounts.values_list('id', flat=True))
resource_accesses_to_transfer = ResourceAccess.objects.filter(user_id__in=duplicate_account_ids)
# Transfer resource access permissions to primary user
# This process handles role hierarchy where:
# OWNER > ADMIN > MEMBER
for resource_access in resource_accesses_to_transfer:
# Determine if primary user already has access to this resource
existing_primary_access = ResourceAccess.objects.filter(
user_id=primary_user.id,
resource_id=resource_access.resource.id
).first()
if existing_primary_access:
# Skip if primary user is already OWNER as it's the highest privilege level
# No need to modify or downgrade owner access
if existing_primary_access.role == RoleChoices.OWNER:
continue
# Skip if primary user already has the exact same role
# No need to update when roles match
elif existing_primary_access.role == resource_access.role:
continue
# Skip if new role is MEMBER since user already has base access
# All existing access includes at least MEMBER privileges
elif resource_access.role == RoleChoices.MEMBER:
continue
# Update the role only if it represents a higher privilege level
# Preserves existing access record while updating the role
existing_primary_access.role = resource_access.role
existing_primary_access.save()
else:
# Transfer access to primary user
resource_access.user_id = primary_user.id
resource_access.save()
# Delete duplicate accounts - CASCADE will automatically remove any untransferred
# ResourceAccess records and other related data for these users
duplicate_user_accounts.delete()
class Migration(migrations.Migration):
dependencies = [
('core', '0005_recording_recordingaccess_and_more'),
]
operations = [
migrations.RunPython(merge_duplicate_user_accounts, reverse_code=migrations.RunPython.noop),
]
@@ -0,0 +1,18 @@
# Generated by Django 5.1.1 on 2024-11-12 10:59
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0006_merge_duplicate_users'),
]
operations = [
migrations.AddField(
model_name='recording',
name='mode',
field=models.CharField(choices=[('screen_recording', 'SCREEN_RECORDING'), ('transcript', 'TRANSCRIPT')], default='screen_recording', help_text='Defines the mode of recording being called.', max_length=20, verbose_name='Recording mode'),
),
]
@@ -0,0 +1,23 @@
# Generated by Django 5.1.1 on 2024-11-13 09:11
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0007_recording_mode'),
]
operations = [
migrations.AddField(
model_name='user',
name='full_name',
field=models.CharField(blank=True, max_length=100, null=True, verbose_name='full name'),
),
migrations.AddField(
model_name='user',
name='short_name',
field=models.CharField(blank=True, max_length=100, null=True, verbose_name='short name'),
)
]
@@ -0,0 +1,18 @@
# Generated by Django 5.1.3 on 2024-12-02 13:23
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0008_user_full_name_user_short_name'),
]
operations = [
migrations.AlterField(
model_name='recording',
name='status',
field=models.CharField(choices=[('initiated', 'Initiated'), ('active', 'Active'), ('stopped', 'Stopped'), ('saved', 'Saved'), ('aborted', 'Aborted'), ('failed_to_start', 'Failed to Start'), ('failed_to_stop', 'Failed to Stop'), ('notification_succeeded', 'Notification succeeded')], default='initiated', max_length=50),
),
]
+283 -27
View File
@@ -4,6 +4,7 @@ Declare and configure the models for the Meet core application
import uuid
from logging import getLogger
from typing import List
from django.conf import settings
from django.contrib.auth import models as auth_models
@@ -15,8 +16,6 @@ from django.utils.functional import lazy
from django.utils.text import capfirst, slugify
from django.utils.translation import gettext_lazy as _
from django.template.loader import render_to_string
from timezone_field import TimeZoneField
logger = getLogger(__name__)
@@ -40,6 +39,47 @@ class RoleChoices(models.TextChoices):
return role == cls.OWNER
class RecordingStatusChoices(models.TextChoices):
"""Enumeration of possible states for a recording operation."""
INITIATED = "initiated", _("Initiated")
ACTIVE = "active", _("Active")
STOPPED = "stopped", _("Stopped")
SAVED = "saved", _("Saved")
ABORTED = "aborted", _("Aborted")
FAILED_TO_START = "failed_to_start", _("Failed to Start")
FAILED_TO_STOP = "failed_to_stop", _("Failed to Stop")
NOTIFICATION_SUCCEEDED = "notification_succeeded", _("Notification succeeded")
@classmethod
def is_final(cls, status):
"""Determine if the recording status represents a final state.
A final status indicates the recording flow has completed, either
successfully or unsuccessfully.
"""
return status in {
cls.STOPPED,
cls.SAVED,
cls.ABORTED,
cls.FAILED_TO_START,
cls.FAILED_TO_STOP,
}
@classmethod
def is_unsuccessful(cls, status):
"""Determine if the recording status represents an unsuccessful state."""
return status in {cls.ABORTED, cls.FAILED_TO_START, cls.FAILED_TO_STOP}
class RecordingModeChoices(models.TextChoices):
"""Recording mode choices."""
SCREEN_RECORDING = "screen_recording", _("SCREEN_RECORDING")
TRANSCRIPT = "transcript", _("TRANSCRIPT")
class BaseModel(models.Model):
"""
Serves as an abstract base model for other models, ensuring that records are validated
@@ -102,11 +142,14 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
email = models.EmailField(_("identity email address"), blank=True, null=True)
# Unlike the "email" field which stores the email coming from the OIDC token, this field
# stores the email used by staff users to login to the admin site
# stores the email used by staff users to log in to the admin site
admin_email = models.EmailField(
_("admin email address"), unique=True, blank=True, null=True
)
full_name = models.CharField(_("full name"), max_length=100, null=True, blank=True)
short_name = models.CharField(
_("short name"), max_length=100, null=True, blank=True
)
language = models.CharField(
max_length=10,
choices=lazy(lambda: settings.LANGUAGES, tuple)(),
@@ -146,6 +189,7 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
class Meta:
db_table = "meet_user"
ordering = ("-created_at",)
verbose_name = _("user")
verbose_name_plural = _("users")
@@ -165,18 +209,39 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
"""
return []
@property
def email_anonymized(self):
"""Anonymize the email address by replacing the local part with asterisks."""
if not self.email:
return ""
return f"***@{self.email.split('@')[1]}"
def get_resource_roles(resource: models.Model, user: User) -> List[str]:
"""
Get all roles assigned to a user for a specific resource, including team-based roles.
Args:
resource: The resource to check permissions for
user: The user to get roles for
Returns:
List of role strings assigned to the user
"""
if not user.is_authenticated:
return []
# Use pre-annotated roles if available from viewset optimization
if hasattr(resource, "user_roles"):
return resource.user_roles or []
try:
return list(
resource.accesses.filter_user(user)
.values_list("role", flat=True)
.distinct()
)
except (IndexError, models.ObjectDoesNotExist):
return []
class Resource(BaseModel):
"""Model to define access control"""
is_public = models.BooleanField(default=settings.DEFAULT_ROOM_IS_PUBLIC)
is_public = models.BooleanField(default=settings.RESOURCE_DEFAULT_IS_PUBLIC)
users = models.ManyToManyField(
User,
through="ResourceAccess",
@@ -240,6 +305,7 @@ class ResourceAccess(BaseModel):
class Meta:
db_table = "meet_resource_access"
ordering = ("-created_at",)
verbose_name = _("Resource access")
verbose_name_plural = _("Resource accesses")
constraints = [
@@ -329,22 +395,212 @@ class Room(Resource):
super().clean_fields(exclude=exclude)
def email_summary(self, owners, link):
"""Wip"""
class BaseAccessManager(models.Manager):
"""Base manager for handling resource access control."""
template_vars = {
"title": "Votre résumé est prêt",
"link": link,
"room": self.slug,
}
msg_html = render_to_string("mail/html/summary.html", template_vars)
msg_plain = render_to_string("mail/text/invitation.txt", template_vars)
def filter_user(self, user):
"""Filter accesses for a given user, including both direct and team-based access."""
return self.filter(models.Q(user=user) | models.Q(team__in=user.get_teams()))
for owner in owners:
owner.email_user(
subject="Votre résumé est prêt",
from_email=settings.EMAIL_FROM,
message=msg_plain,
html_message=msg_html,
fail_silently=False,
class BaseAccess(BaseModel):
"""Base model for accesses to handle resources."""
user = models.ForeignKey(
User,
on_delete=models.CASCADE,
null=True,
blank=True,
)
team = models.CharField(max_length=100, blank=True)
role = models.CharField(
max_length=20, choices=RoleChoices.choices, default=RoleChoices.MEMBER
)
objects = BaseAccessManager()
class Meta:
abstract = True
def _get_abilities(self, resource, user):
"""
Compute and return abilities for a given user taking into account
the current state of the object.
"""
roles = get_resource_roles(resource, user)
is_owner = RoleChoices.OWNER in roles
has_privileges = is_owner or RoleChoices.ADMIN in roles
# Default values for unprivileged users
set_role_to = set()
can_delete = False
# Special handling when modifying an owner's access
if self.role == RoleChoices.OWNER:
# Prevent orphaning the resource
can_delete = (
is_owner
and resource.accesses.filter(role=RoleChoices.OWNER).count() > 1
)
if can_delete:
set_role_to = {RoleChoices.ADMIN, RoleChoices.OWNER, RoleChoices.MEMBER}
elif has_privileges:
can_delete = True
set_role_to = {RoleChoices.ADMIN, RoleChoices.MEMBER}
if is_owner:
set_role_to.add(RoleChoices.OWNER)
# Remove the current role as we don't want to propose it as an option
set_role_to.discard(self.role)
return {
"destroy": can_delete,
"update": bool(set_role_to),
"partial_update": bool(set_role_to),
"retrieve": bool(roles),
"set_role_to": sorted(r.value for r in set_role_to),
}
class Recording(BaseModel):
"""Model for recordings that take place in a room.
Recording Status Flow:
1. INITIATED: Initial state when recording is requested
2. ACTIVE: Recording is currently in progress
3. STOPPED: Recording has been stopped by user/system
4. SAVED: Recording has been successfully processed and stored
4. NOTIFICATION_SUCCEEDED: External service has been notified of this recording
Error States:
- FAILED_TO_START: Worker failed to initialize recording
- FAILED_TO_STOP: Worker failed during stop operation
- ABORTED: Recording was terminated before completion
Warning: Worker failures may lead to database inconsistency between the actual
recording state and its status in the database.
"""
room = models.ForeignKey(
Room,
on_delete=models.CASCADE,
related_name="recordings",
verbose_name=_("Room"),
)
status = models.CharField(
max_length=50,
choices=RecordingStatusChoices.choices,
default=RecordingStatusChoices.INITIATED,
)
worker_id = models.CharField(
max_length=255,
null=True,
blank=True,
verbose_name=_("Worker ID"),
help_text=_(
"Enter an identifier for the worker recording."
"This ID is retained even when the worker stops, allowing for easy tracking."
),
)
mode = models.CharField(
max_length=20,
choices=RecordingModeChoices.choices,
default=RecordingModeChoices.SCREEN_RECORDING,
verbose_name=_("Recording mode"),
help_text=_("Defines the mode of recording being called."),
)
class Meta:
db_table = "meet_recording"
ordering = ("-created_at",)
verbose_name = _("Recording")
verbose_name_plural = _("Recordings")
constraints = [
models.UniqueConstraint(
fields=["room"],
condition=models.Q(
status__in=[
RecordingStatusChoices.ACTIVE,
RecordingStatusChoices.INITIATED,
]
),
name="unique_initiated_or_active_recording_per_room",
)
]
def __str__(self):
return f"Recording {self.id} ({self.status})"
def get_abilities(self, user):
"""Compute and return abilities for a given user on the recording."""
roles = set(get_resource_roles(self, user))
is_owner_or_admin = bool(
roles.intersection({RoleChoices.OWNER, RoleChoices.ADMIN})
)
is_final_status = RecordingStatusChoices.is_final(self.status)
return {
"destroy": is_owner_or_admin and is_final_status,
"partial_update": False,
"retrieve": is_owner_or_admin,
"stop": is_owner_or_admin and not is_final_status,
"update": False,
}
def is_savable(self) -> bool:
"""Determine if the recording can be saved based on its current status."""
return self.status in {
RecordingStatusChoices.ACTIVE,
RecordingStatusChoices.STOPPED,
}
class RecordingAccess(BaseAccess):
"""Relation model to give access to a recording for a user or a team with a role."""
recording = models.ForeignKey(
Recording,
on_delete=models.CASCADE,
related_name="accesses",
)
class Meta:
db_table = "meet_recording_access"
ordering = ("-created_at",)
verbose_name = _("Recording/user relation")
verbose_name_plural = _("Recording/user relations")
constraints = [
models.UniqueConstraint(
fields=["user", "recording"],
condition=models.Q(user__isnull=False), # Exclude null users
name="unique_recording_user",
violation_error_message=_("This user is already in this recording."),
),
models.UniqueConstraint(
fields=["team", "recording"],
condition=models.Q(team__gt=""), # Exclude empty string teams
name="unique_recording_team",
violation_error_message=_("This team is already in this recording."),
),
models.CheckConstraint(
condition=models.Q(user__isnull=False, team="")
| models.Q(user__isnull=True, team__gt=""),
name="check_recording_access_either_user_or_team",
violation_error_message=_("Either user or team must be set, not both."),
),
]
def __str__(self):
return f"{self.user!s} is {self.role:s} in {self.recording!s}"
def get_abilities(self, user):
"""
Compute and return abilities for a given user on the recording access.
"""
return self._get_abilities(self.recording, user)
@@ -0,0 +1 @@
"""Meet event parser classes, authentication and exceptions."""
@@ -0,0 +1,94 @@
"""Authentication class for storage event token validation."""
import logging
import secrets
from django.conf import settings
from django.utils.translation import gettext_lazy as _
from rest_framework.authentication import BaseAuthentication
from rest_framework.exceptions import AuthenticationFailed
logger = logging.getLogger(__name__)
class MachineUser:
"""Represent a non-interactive system user for automated storage operations."""
def __init__(self) -> None:
self.pk = None
self.username = "storage_event_user"
self.is_active = True
@property
def is_authenticated(self):
"""Indicate if this machine user is authenticated."""
return True
@property
def is_anonymous(self) -> bool:
"""Indicate if this is an anonymous user."""
return False
def get_username(self) -> str:
"""Return the machine user identifier."""
return self.username
class StorageEventAuthentication(BaseAuthentication):
"""Authenticate requests using a Bearer token for storage event integration.
This class validates Bearer tokens for storage events that don't map to database users.
It's designed for S3-compatible storage integrations and similar use cases.
Events are submitted when a webhook is configured on some bucket's events.
"""
AUTH_HEADER = "Authorization"
TOKEN_TYPE = "Bearer" # noqa S105
def authenticate(self, request):
"""Validate the Bearer token from the Authorization header."""
if not settings.RECORDING_ENABLE_STORAGE_EVENT_AUTH:
return MachineUser(), None
required_token = settings.RECORDING_STORAGE_EVENT_TOKEN
if not required_token:
if settings.RECORDING_ENABLE_STORAGE_EVENT_AUTH:
raise AuthenticationFailed(
_("Authentication is enabled but token is not configured.")
)
return MachineUser(), None
auth_header = request.headers.get(self.AUTH_HEADER)
if not auth_header:
logger.warning(
"Authentication failed: Missing Authorization header (ip: %s)",
request.META.get("REMOTE_ADDR"),
)
raise AuthenticationFailed(_("Authorization header is required"))
auth_parts = auth_header.split(" ")
if len(auth_parts) != 2 or auth_parts[0] != self.TOKEN_TYPE:
logger.warning(
"Authentication failed: Invalid authorization header (ip: %s)",
request.META.get("REMOTE_ADDR"),
)
raise AuthenticationFailed(_("Invalid authorization header."))
token = auth_parts[1]
# Use constant-time comparison to prevent timing attacks
if not secrets.compare_digest(token.encode(), required_token.encode()):
logger.warning(
"Authentication failed: Invalid token (ip: %s)",
request.META.get("REMOTE_ADDR"),
)
raise AuthenticationFailed(_("Invalid token"))
return MachineUser(), token
def authenticate_header(self, request):
"""Return the WWW-Authenticate header value."""
return f"{self.TOKEN_TYPE} realm='Storage event API'"
@@ -0,0 +1,17 @@
"""Storage parsers specific exceptions."""
class ParsingEventDataError(Exception):
"""Raised when the request data is malformed, incomplete, or missing."""
class InvalidBucketError(Exception):
"""Raised when the bucket name in the request does not match the expected one."""
class InvalidFileTypeError(Exception):
"""Raised when the file type in the request is not supported."""
class InvalidFilepathError(Exception):
"""Raised when the filepath in the request is invalid."""
@@ -0,0 +1,93 @@
"""Service to notify external services when a new recording is ready."""
import logging
from django.conf import settings
import requests
from core import models
logger = logging.getLogger(__name__)
class NotificationService:
"""Service for processing recordings and notifying external services."""
def notify_external_services(self, recording):
"""Process a recording based on its mode."""
if recording.mode == models.RecordingModeChoices.TRANSCRIPT:
return self._notify_summary_service(recording)
if recording.mode == models.RecordingModeChoices.SCREEN_RECORDING:
logger.warning(
"Screen recording mode not implemented for recording %s", recording.id
)
return False
logger.error(
"Unknown recording mode %s for recording %s",
recording.mode,
recording.id,
)
return False
@staticmethod
def _notify_summary_service(recording):
"""Notify summary service about a new recording."""
if (
not settings.SUMMARY_SERVICE_ENDPOINT
or not settings.SUMMARY_SERVICE_API_TOKEN
):
logger.error("Summary service not configured")
return False
owner_access = (
models.RecordingAccess.objects.select_related("user")
.filter(
role=models.RoleChoices.OWNER,
recording_id=recording.id,
)
.first()
)
if not owner_access:
logger.error("No owner found for recording %s", recording.id)
return False
key = f"{settings.RECORDING_OUTPUT_FOLDER}/{recording.id}.ogg"
payload = {
"filename": key,
"email": owner_access.user.email,
"sub": owner_access.user.sub,
}
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {settings.SUMMARY_SERVICE_API_TOKEN}",
}
try:
response = requests.post(
settings.SUMMARY_SERVICE_ENDPOINT,
json=payload,
headers=headers,
timeout=30,
)
response.raise_for_status()
except requests.HTTPError as exc:
logger.exception(
"Summary service HTTP error for recording %s. URL: %s. Exception: %s",
recording.id,
settings.SUMMARY_SERVICE_ENDPOINT,
exc,
)
return False
return True
notification_service = NotificationService()
+147
View File
@@ -0,0 +1,147 @@
"""Meet storage event parser classes."""
import logging
import re
from dataclasses import dataclass
from functools import lru_cache
from typing import Any, Dict, Optional, Protocol
from django.conf import settings
from django.utils.module_loading import import_string
from .exceptions import (
InvalidBucketError,
InvalidFilepathError,
InvalidFileTypeError,
ParsingEventDataError,
)
logger = logging.getLogger(__name__)
@dataclass
class StorageEvent:
"""Represents a storage event with relevant metadata.
Attributes:
filepath: Identifier for the affected recording
filetype: Type of storage event
bucket_name: When the event occurred
metadata: Additional event data
"""
filepath: str
filetype: str
bucket_name: str
metadata: Optional[Dict[str, Any]]
def __post_init__(self):
if self.filepath is None:
raise TypeError("filepath cannot be None")
if self.filetype is None:
raise TypeError("filetype cannot be None")
if self.bucket_name is None:
raise TypeError("bucket_name cannot be None")
class EventParser(Protocol):
"""Interface for parsing storage events."""
def __init__(self, bucket_name, allowed_filetypes=None):
"""Initialize parser with bucket name and optional allowed filetypes."""
def parse(self, data: Dict) -> StorageEvent:
"""Extract storage event data from raw dictionary input."""
def validate(self, data: StorageEvent) -> None:
"""Verify storage event data meets all requirements."""
def get_recording_id(self, data: Dict) -> str:
"""Extract recording ID from event dictionary."""
@lru_cache(maxsize=1)
def get_parser() -> EventParser:
"""Return cached instance of configured event parser.
Uses function memoization instead of a factory class since the only
varying parameter is the parser class from settings. A factory class
would add unnecessary complexity when a cached function provides the
same singleton behavior with simpler code.
"""
event_parser_cls = import_string(settings.RECORDING_EVENT_PARSER_CLASS)
return event_parser_cls(bucket_name=settings.AWS_STORAGE_BUCKET_NAME)
class MinioParser:
"""Handle parsing and validation of Minio storage events."""
def __init__(self, bucket_name: str, allowed_filetypes=None):
"""Initialize parser with target bucket name and accepted filetypes."""
if not bucket_name:
raise ValueError("Bucket name cannot be None or empty")
self._bucket_name = bucket_name
self._allowed_filetypes = allowed_filetypes or {"audio/ogg", "video/mp4"}
# pylint: disable=line-too-long
self._filepath_regex = re.compile(
r"(?P<url_encoded_folder_path>(?:[^%]+%2F)*)?(?P<recording_id>[0-9a-fA-F\-]{36})\.(?P<extension>[a-zA-Z0-9]+)"
)
@staticmethod
def parse(data):
"""Convert raw Minio event dictionary to StorageEvent object."""
if not data:
raise ParsingEventDataError("Received empty data.")
try:
record = data["Records"][0]
s3 = record["s3"]
bucket_name = s3["bucket"]["name"]
file_object = s3["object"]
filepath = file_object["key"]
filetype = file_object["contentType"]
except (KeyError, IndexError) as e:
raise ParsingEventDataError(f"Missing or malformed key: {e}.") from e
try:
return StorageEvent(
filepath=filepath,
filetype=filetype,
bucket_name=bucket_name,
metadata=None,
)
except TypeError as e:
raise ParsingEventDataError(f"Missing essential data fields: {e}") from e
def validate(self, event_data: StorageEvent) -> str:
"""Verify StorageEvent matches bucket, filetype and filepath requirements."""
if event_data.bucket_name != self._bucket_name:
raise InvalidBucketError(
f"Invalid bucket: expected {self._bucket_name}, got {event_data.bucket_name}"
)
if not event_data.filetype in self._allowed_filetypes:
raise InvalidFileTypeError(
f"Invalid file type, expected {self._allowed_filetypes},"
f"got '{event_data.filetype}'"
)
match = self._filepath_regex.match(event_data.filepath)
if not match:
raise InvalidFilepathError(
f"Invalid filepath structure: {event_data.filepath}"
)
recording_id = match.group("recording_id")
return recording_id
def get_recording_id(self, data):
"""Extract recording ID from Minio event through parsing and validation."""
event_data = self.parse(data)
recording_id = self.validate(event_data)
return recording_id
@@ -0,0 +1 @@
"""Meet worker services classes and exceptions."""
@@ -0,0 +1,21 @@
"""Recording and worker services specific exceptions."""
class WorkerRequestError(Exception):
"""Raised when there is an issue with the worker request"""
class WorkerConnectionError(Exception):
"""Raised when there is an issue connecting to the worker."""
class WorkerResponseError(Exception):
"""Raised when the worker's response is not as expected."""
class RecordingStartError(Exception):
"""Raised when there is an error starting the recording."""
class RecordingStopError(Exception):
"""Raised when there is an error stopping the recording."""
@@ -0,0 +1,75 @@
"""Factory, configurations and Protocol to create worker services"""
import logging
from dataclasses import dataclass
from functools import lru_cache
from typing import Any, ClassVar, Dict, Optional, Protocol, Type
from django.conf import settings
from django.utils.module_loading import import_string
logger = logging.getLogger(__name__)
@dataclass(frozen=True)
class WorkerServiceConfig:
"""Declare Worker Service common configurations"""
output_folder: str
server_configurations: Dict[str, Any]
verify_ssl: Optional[bool]
bucket_args: Optional[dict]
@classmethod
@lru_cache
def from_settings(cls) -> "WorkerServiceConfig":
"""Load configuration from Django settings with caching for efficiency."""
logger.debug("Loading WorkerServiceConfig from settings.")
return cls(
output_folder=settings.RECORDING_OUTPUT_FOLDER,
server_configurations=settings.LIVEKIT_CONFIGURATION,
verify_ssl=settings.RECORDING_VERIFY_SSL,
bucket_args={
"endpoint": settings.AWS_S3_ENDPOINT_URL,
"access_key": settings.AWS_S3_ACCESS_KEY_ID,
"secret": settings.AWS_S3_SECRET_ACCESS_KEY,
"region": settings.AWS_S3_REGION_NAME,
"bucket": settings.AWS_STORAGE_BUCKET_NAME,
"force_path_style": True,
},
)
class WorkerService(Protocol):
"""Define the interface for interacting with a worker service."""
hrid: ClassVar[str]
def __init__(self, config: WorkerServiceConfig):
"""Initialize the service with the given configuration."""
def start(self, room_id: str, recording_id: str) -> str:
"""Start a recording for a specified room."""
def stop(self, worker_id: str) -> str:
"""Stop recording for a specified worker."""
def get_worker_service(mode: str) -> WorkerService:
"""Instantiate a worker service by its mode."""
worker_registry: Dict[str, str] = settings.RECORDING_WORKER_CLASSES
try:
worker_class_path = worker_registry[mode]
except KeyError as e:
raise ValueError(
f"Recording mode '{mode}' not found in RECORDING_WORKER_CLASSES. "
f"Available modes: {list(worker_registry.keys())}"
) from e
worker_class: Type[WorkerService] = import_string(worker_class_path)
config = WorkerServiceConfig.from_settings()
return worker_class(config=config)
@@ -0,0 +1,98 @@
"""Mediator between the worker service and recording instances in the Django ORM."""
import logging
from core.models import Recording, RecordingStatusChoices
from .exceptions import (
RecordingStartError,
RecordingStopError,
WorkerConnectionError,
WorkerRequestError,
WorkerResponseError,
)
from .factories import WorkerService
logger = logging.getLogger(__name__)
class WorkerServiceMediator:
"""Mediate interactions between a worker service and a recording instance.
A mediator class that decouples the worker from Django ORM, handles recording updates
based on worker status, and transforms worker errors into user-friendly exceptions.
Implements Mediator pattern.
"""
def __init__(self, worker_service: WorkerService):
"""Initialize the WorkerServiceMediator with the provided worker service."""
self._worker_service = worker_service
def start(self, recording: Recording):
"""Start the recording process using the worker service.
If the operation is successful, the recording's status will
transition from INITIATED to ACTIVE, else to FAILED_TO_START to keep track of errors.
Args:
recording (Recording): The recording instance to start.
Raises:
RecordingStartError: If there is an error starting the recording.
"""
if recording.status != RecordingStatusChoices.INITIATED:
logger.error("Cannot start recording in %s status.", recording.status)
raise RecordingStartError()
room_name = str(recording.room.id)
try:
worker_id = self._worker_service.start(room_name, recording.id)
except (WorkerRequestError, WorkerConnectionError, WorkerResponseError) as e:
logger.exception(
"Failed to start recording for room %s: %s", recording.room.slug, e
)
recording.status = RecordingStatusChoices.FAILED_TO_START
raise RecordingStartError() from e
else:
recording.worker_id = worker_id
recording.status = RecordingStatusChoices.ACTIVE
finally:
recording.save()
logger.info(
"Worker started for room %s (worker ID: %s)",
recording.room,
recording.worker_id,
)
def stop(self, recording: Recording):
"""Stop the recording process using the worker service.
If the operation is successful, the recording's status will transition
from ACTIVE to STOPPED, else to FAILED_TO_STOP to keep track of errors.
Args:
recording (Recording): The recording instance to stop.
Raises:
RecordingStopError: If there is an error stopping the recording.
"""
if recording.status != RecordingStatusChoices.ACTIVE:
logger.error("Cannot stop recording in %s status.", recording.status)
raise RecordingStopError()
try:
response = self._worker_service.stop(worker_id=recording.worker_id)
except (WorkerConnectionError, WorkerResponseError) as e:
logger.exception(
"Failed to stop recording for room %s: %s", recording.room.slug, e
)
recording.status = RecordingStatusChoices.FAILED_TO_STOP
raise RecordingStopError() from e
else:
recording.status = RecordingStatusChoices[response]
finally:
recording.save()
logger.info("Worker stopped for room %s", recording.room)
@@ -0,0 +1,140 @@
"""Worker services in charge of recording a room."""
# pylint: disable=no-member
import aiohttp
from asgiref.sync import async_to_sync
from livekit import api as livekit_api
from livekit.api.egress_service import EgressService
from .exceptions import WorkerConnectionError, WorkerResponseError
from .factories import WorkerServiceConfig
class BaseEgressService:
"""Base egress defining common methods to manage and interact with LiveKit egress processes."""
def __init__(self, config: WorkerServiceConfig):
self._config = config
self._s3 = livekit_api.S3Upload(**config.bucket_args)
def _get_filepath(self, filename: str, extension: str) -> str:
"""Construct the file path for a given filename and extension.
Unsecure method, doesn't handle paths robustly and securely.
"""
return f"{self._config.output_folder}/{filename}.{extension}"
@async_to_sync
async def _handle_request(self, request, method_name: str):
"""Handle making a request to the LiveKit API and returns the response."""
# Use HTTP connector for local development with Tilt,
# where cluster communications are unsecure
connector = aiohttp.TCPConnector(ssl=self._config.verify_ssl)
async with aiohttp.ClientSession(connector=connector) as session:
client = EgressService(session, **self._config.server_configurations)
method = getattr(client, method_name)
try:
response = await method(request)
except livekit_api.TwirpError as e:
raise WorkerConnectionError(
f"LiveKit client connection error, {e.message}."
) from e
return response
def stop(self, worker_id: str) -> str:
"""Stop an ongoing egress worker.
The StopEgressRequest is shared among all types of egress,
so a single implementation in the base class should be sufficient.
"""
request = livekit_api.StopEgressRequest(
egress_id=worker_id,
)
response = self._handle_request(request, "stop_egress")
if not response.status:
raise WorkerResponseError(
"LiveKit response is missing the recording status."
)
# To avoid exposing EgressStatus values and coupling with LiveKit outside of this class,
# the response status is mapped to simpler "ABORTED", "STOPPED" or "FAILED_TO_STOP" strings.
if response.status == livekit_api.EgressStatus.EGRESS_ABORTED:
return "ABORTED"
if response.status == livekit_api.EgressStatus.EGRESS_ENDING:
return "STOPPED"
return "FAILED_TO_STOP"
def start(self, room_name, recording_id):
"""Start the egress process for a recording (not implemented in the base class).
Each derived class must implement this method, providing the necessary parameters for
its specific egress type (e.g. audio_only, streaming output).
"""
raise NotImplementedError("Subclass must implement this method.")
class VideoCompositeEgressService(BaseEgressService):
"""Record multiple participant video and audio tracks into a single output '.mp4' file."""
hrid = "video-recording-composite-livekit-egress"
def start(self, room_name, recording_id):
"""Start the video composite egress process for a recording."""
# Save room's recording as a mp4 video file.
file_type = livekit_api.EncodedFileType.MP4
filepath = self._get_filepath(filename=recording_id, extension="mp4")
file_output = livekit_api.EncodedFileOutput(
file_type=file_type,
filepath=filepath,
s3=self._s3,
)
request = livekit_api.RoomCompositeEgressRequest(
room_name=room_name,
file_outputs=[file_output],
)
response = self._handle_request(request, "start_room_composite_egress")
if not response.egress_id:
raise WorkerResponseError("Egress ID not found in the response.")
return response.egress_id
class AudioCompositeEgressService(BaseEgressService):
"""Record multiple participant audio tracks into a single output '.ogg' file."""
hrid = "audio-recording-composite-livekit-egress"
def start(self, room_name, recording_id):
"""Start the audio composite egress process for a recording."""
# Save room's recording as an ogg audio file.
file_type = livekit_api.EncodedFileType.OGG
filepath = self._get_filepath(filename=recording_id, extension="ogg")
file_output = livekit_api.EncodedFileOutput(
file_type=file_type,
filepath=filepath,
s3=self._s3,
)
request = livekit_api.RoomCompositeEgressRequest(
room_name=room_name, file_outputs=[file_output], audio_only=True
)
response = self._handle_request(request, "start_room_composite_egress")
if not response.egress_id:
raise WorkerResponseError("Egress ID not found in the response.")
return response.egress_id
@@ -0,0 +1,134 @@
"""Marketing service in charge of pushing data for marketing automation."""
import logging
from dataclasses import dataclass
from functools import lru_cache
from typing import Dict, List, Optional, Protocol
from django.conf import settings
from django.core.exceptions import ImproperlyConfigured
from django.utils.module_loading import import_string
import brevo_python
logger = logging.getLogger(__name__)
class ContactCreationError(Exception):
"""Raised when the contact creation fails."""
@dataclass
class ContactData:
"""Contact data for marketing service integration."""
email: str
attributes: Optional[Dict[str, str]] = None
list_ids: Optional[List[int]] = None
update_enabled: bool = True
class MarketingServiceProtocol(Protocol):
"""Interface for marketing automation service integrations."""
def create_contact(
self, contact_data: ContactData, timeout: Optional[int] = None
) -> dict:
"""Create or update a contact.
Args:
contact_data: Contact information and attributes
timeout: API request timeout in seconds
Returns:
dict: Service response
Raises:
ContactCreationError: If contact creation fails
"""
class BrevoMarketingService:
"""Brevo marketing automation integration.
Handles:
- Contact management and segmentation
- Marketing campaigns and automation
- Email communications
Configuration via Django settings:
- BREVO_API_KEY: API authentication
- BREVO_API_CONTACT_LIST_IDS: Default contact lists
- BREVO_API_CONTACT_ATTRIBUTES: Default contact attributes
"""
def __init__(self):
"""Initialize Brevo (ex-sendinblue) marketing service."""
if not settings.BREVO_API_KEY:
raise ImproperlyConfigured("Brevo API key is required")
configuration = brevo_python.Configuration()
configuration.api_key["api-key"] = settings.BREVO_API_KEY
self._api_client = brevo_python.ApiClient(configuration)
def create_contact(self, contact_data: ContactData, timeout=None) -> dict:
"""Create or update a Brevo contact.
Args:
contact_data: Contact information and attributes
timeout: API request timeout in seconds
Returns:
dict: Brevo API response
Raises:
ContactCreationError: If contact creation fails
ImproperlyConfigured: If required settings are missing
Note:
Contact attributes must be pre-configured in Brevo.
Changes to attributes can impact existing workflows.
"""
if not settings.BREVO_API_CONTACT_LIST_IDS:
raise ImproperlyConfigured(
"Default Brevo List IDs must be configured in settings."
)
contact_api = brevo_python.ContactsApi(self._api_client)
attributes = {
**settings.BREVO_API_CONTACT_ATTRIBUTES,
**(contact_data.attributes or {}),
}
list_ids = (contact_data.list_ids or []) + settings.BREVO_API_CONTACT_LIST_IDS
contact = brevo_python.CreateContact(
email=contact_data.email,
attributes=attributes,
list_ids=list_ids,
update_enabled=contact_data.update_enabled,
)
api_configurations = {}
if timeout is not None:
api_configurations["_request_timeout"] = timeout
try:
response = contact_api.create_contact(contact, **api_configurations)
except brevo_python.rest.ApiException as err:
logger.exception("Failed to create contact in Brevo")
raise ContactCreationError("Failed to create contact in Brevo") from err
return response
@lru_cache(maxsize=1)
def get_marketing_service() -> MarketingServiceProtocol:
"""Return cached instance of configured marketing service."""
marketing_service_cls = import_string(settings.MARKETING_SERVICE_CLASS)
return marketing_service_cls()
@@ -1,42 +1,48 @@
"""Unit tests for the Authentication Backends."""
from django.core.exceptions import SuspiciousOperation
from unittest import mock
from django.core.exceptions import ImproperlyConfigured, SuspiciousOperation
import pytest
from core import models
from core.authentication.backends import OIDCAuthenticationBackend
from core.factories import UserFactory
from core.services import marketing_service
pytestmark = pytest.mark.django_db
def test_authentication_getter_existing_user_no_email(
django_assert_num_queries, monkeypatch
):
def test_authentication_getter_existing_user(monkeypatch):
"""
If an existing user matches the user's info sub, the user should be returned.
If an existing user matches, the user should be returned.
"""
klass = OIDCAuthenticationBackend()
db_user = UserFactory()
db_user = UserFactory(email="foo@mail.com")
def get_userinfo_mocked(*args):
return {"sub": db_user.sub}
return {"sub": db_user.sub, "email": "some@mail.com"}
def get_existing_user(*args):
return db_user
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
monkeypatch.setattr(
OIDCAuthenticationBackend, "get_existing_user", get_existing_user
)
with django_assert_num_queries(1):
user = klass.get_or_create_user(
access_token="test-token", id_token=None, payload=None
)
user = klass.get_or_create_user(
access_token="test-token", id_token=None, payload=None
)
assert user == db_user
def test_authentication_getter_new_user_no_email(monkeypatch):
"""
If no user matches the user's info sub, a user should be created.
If no user matches, a user should be created.
User's info doesn't contain an email, created user's email should be empty.
"""
klass = OIDCAuthenticationBackend()
@@ -58,16 +64,15 @@ def test_authentication_getter_new_user_no_email(monkeypatch):
def test_authentication_getter_new_user_with_email(monkeypatch):
"""
If no user matches the user's info sub, a user should be created.
User's email and name should be set on the identity.
The "email" field on the User model should not be set as it is reserved for staff users.
If no user matches, a user should be created.
User's info contains an email, created user's email should be filled.
"""
klass = OIDCAuthenticationBackend()
email = "meet@example.com"
def get_userinfo_mocked(*args):
return {"sub": "123", "email": email, "first_name": "John", "last_name": "Doe"}
return {"sub": "123", "email": email}
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
@@ -77,6 +82,34 @@ def test_authentication_getter_new_user_with_email(monkeypatch):
assert user.sub == "123"
assert user.email == email
assert user.full_name is None
assert user.short_name is None
assert user.password == "!"
assert models.User.objects.count() == 1
@pytest.mark.parametrize("email", [None, "johndoe@foo.com"])
def test_authentication_getter_new_user_with_names(monkeypatch, email):
"""
If no user matches, a user should be created.
User's info contains name-related field, created user's full and short names should be filled,
whether the email is filled
"""
klass = OIDCAuthenticationBackend()
def get_userinfo_mocked(*args):
return {"sub": "123", "given_name": "John", "usual_name": "Doe", "email": email}
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
user = klass.get_or_create_user(
access_token="test-token", id_token=None, payload=None
)
assert user.sub == "123"
assert user.email == email
assert user.full_name == "John Doe"
assert user.short_name == "John"
assert user.password == "!"
assert models.User.objects.count() == 1
@@ -102,3 +135,419 @@ def test_models_oidc_user_getter_invalid_token(django_assert_num_queries, monkey
klass.get_or_create_user(access_token="test-token", id_token=None, payload=None)
assert models.User.objects.exists() is False
def test_models_oidc_user_getter_empty_sub(django_assert_num_queries, monkeypatch):
"""The user's info contains a sub, but it's an empty string."""
klass = OIDCAuthenticationBackend()
def get_userinfo_mocked(*args):
return {"test": "123", "sub": ""}
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
with (
django_assert_num_queries(0),
pytest.raises(
SuspiciousOperation,
match="User info contained no recognizable user identification",
),
):
klass.get_or_create_user(access_token="test-token", id_token=None, payload=None)
assert models.User.objects.exists() is False
def test_authentication_get_inactive_user(monkeypatch):
"""Test an exception is raised when attempting to authenticate inactive user."""
klass = OIDCAuthenticationBackend()
db_user = UserFactory(is_active=False)
def get_userinfo_mocked(*args):
return {"sub": db_user.sub}
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
with (
pytest.raises(
SuspiciousOperation,
match="User account is disabled",
),
):
klass.get_or_create_user(access_token="test-token", id_token=None, payload=None)
def test_finds_user_by_sub(django_assert_num_queries):
"""Should return user when found by sub, and email is matching."""
klass = OIDCAuthenticationBackend()
db_user = UserFactory(email="foo@mail.com")
with django_assert_num_queries(1):
user = klass.get_existing_user(db_user.sub, db_user.email)
assert user == db_user
def test_finds_user_when_email_fallback_disabled(django_assert_num_queries, settings):
"""Should not return a user when not found by sub, and email fallback is disabled."""
settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = False
klass = OIDCAuthenticationBackend()
db_user = UserFactory(email="foo@mail.com")
with django_assert_num_queries(1):
user = klass.get_existing_user("wrong-sub", db_user.email)
assert user is None
def test_finds_user_when_email_is_none(django_assert_num_queries, settings):
"""Should not return a user when not found by sub, and email is empty."""
settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = True
klass = OIDCAuthenticationBackend()
UserFactory(email="foo@mail.com")
empty_email = ""
with django_assert_num_queries(1):
user = klass.get_existing_user("wrong-sub", empty_email)
assert user is None
def test_finds_user_by_email(django_assert_num_queries, settings):
"""Should return user when found by email, and sub is not matching."""
settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = True
klass = OIDCAuthenticationBackend()
db_user = UserFactory(email="foo@mail.com")
with django_assert_num_queries(2):
user = klass.get_existing_user("wrong-sub", db_user.email)
assert user == db_user
def test_finds_user_case_insensitive_email(django_assert_num_queries, settings):
"""Should match email case-insensitively when falling back to email."""
settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = True
klass = OIDCAuthenticationBackend()
db_user = UserFactory(email="foo@mail.com")
with django_assert_num_queries(2):
user = klass.get_existing_user("wrong-sub", "FOO@MAIL.COM")
assert user == db_user
def test_finds_user_multiple_users_same_email(django_assert_num_queries, settings):
"""Should handle multiple users with same email appropriately."""
settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = True
klass = OIDCAuthenticationBackend()
email = "foo@mail.com"
UserFactory(email=email)
UserFactory(email=email) # Second user with same email
with (
django_assert_num_queries(2),
pytest.raises(
SuspiciousOperation,
match="Multiple user accounts share a common email.",
),
):
klass.get_existing_user("wrong-sub", email)
def test_finds_user_whitespace_email(django_assert_num_queries, settings):
"""Should not match emails with whitespace."""
settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = True
settings.OIDC_CREATE_USER = False
klass = OIDCAuthenticationBackend()
UserFactory(email="foo@mail.com")
with django_assert_num_queries(2):
user = klass.get_existing_user("wrong-sub", " foo@mail.com ")
assert user is None
@pytest.mark.parametrize(
"email",
[
"john.doe@xample.com", # Fullwidth character in domain
"john.doe@еxample.com", # Cyrillic 'е' in domain
"JOHN.DOe@exam𝔭le.com", # Mixed Gothic '𝔭' in domain
"john.doe@exаmple.com", # Cyrillic 'а' (a) in domain
"john.doe@e𝓧𝓪𝓶𝓹𝓵𝓮.com", # Mixed fullwidth and cursive in domain
],
)
def test_authentication_getter_existing_user_email_tricky(email, monkeypatch, settings):
"""Test email matching security against visually similar but non-ASCII domains.
Validates that emails with Unicode characters that visually resemble ASCII
(homoglyphs) are treated as distinct from their ASCII counterparts for security,
per RFC compliance requirements for hostnames.
"""
settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = True
klass = OIDCAuthenticationBackend()
db_user = UserFactory(email="john.doe@example.com")
def get_userinfo_mocked(*args):
return {"sub": "123", "email": email}
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
user = klass.get_or_create_user(
access_token="test-token", id_token=None, payload=None
)
assert user != db_user
@pytest.mark.parametrize(
"given_name, usual_name, email",
[
("Jack", "Doe", "john.doe@example.com"),
("John", "Duy", "john.doe@example.com"),
("John", "Doe", "jack.duy@example.com"),
("Jack", "Duy", "jack.duy@example.com"),
],
)
def test_authentication_getter_existing_user_change_fields(
given_name, usual_name, email, django_assert_num_queries, monkeypatch
):
"""It should update the email or name fields on the user when they change."""
klass = OIDCAuthenticationBackend()
user = UserFactory(
full_name="John Doe", short_name="John", email="john.doe@example.com"
)
def get_userinfo_mocked(*args):
return {
"sub": user.sub,
"email": email,
"given_name": given_name,
"usual_name": usual_name,
}
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
# One and only one additional update query when a field has changed
with django_assert_num_queries(2):
authenticated_user = klass.get_or_create_user(
access_token="test-token", id_token=None, payload=None
)
assert user == authenticated_user
user.refresh_from_db()
assert user.email == email
assert user.full_name == f"{given_name:s} {usual_name:s}"
assert user.short_name == given_name
@pytest.mark.parametrize(
"user_info, expected_name",
[
({"given_name": "John", "family_name": "Doe"}, "John Doe"),
(
{"given_name": "John", "middle_name": "M", "family_name": "Doe"},
"John M Doe",
),
({"family_name": "Doe"}, "Doe"),
({"given_name": "", "family_name": ""}, None),
({}, None),
],
)
def test_compute_full_name(user_info, expected_name, settings):
"""Test full name computation from OIDC user info fields."""
settings.OIDC_USERINFO_FULLNAME_FIELDS = [
"given_name",
"middle_name",
"family_name",
]
klass = OIDCAuthenticationBackend()
assert klass.compute_full_name(user_info) == expected_name
def test_compute_full_name_no_fields(settings):
"""Test full name computation with empty field configuration."""
settings.OIDC_USERINFO_FULLNAME_FIELDS = []
klass = OIDCAuthenticationBackend()
assert klass.compute_full_name({"given_name": "John"}) is None
@pytest.mark.parametrize(
"claims",
[
{"email": "john.doe@example.com"}, # Same data - no change needed
{"email": ""}, # Empty strings should not override
{"non_related_field": "foo"}, # Unrelated fields should be ignored
{}, # Empty claims should not affect user
{"email": None}, # None values should be ignored
],
)
def test_update_user_when_no_update_needed(django_assert_num_queries, claims):
"""Test that user attributes remain unchanged when claims don't require updates."""
user = UserFactory(
full_name="John Doe", short_name="John", email="john.doe@example.com"
)
klass = OIDCAuthenticationBackend()
with django_assert_num_queries(0):
klass.update_user_if_needed(user, claims)
user.refresh_from_db()
assert user.email == "john.doe@example.com"
@mock.patch.object(OIDCAuthenticationBackend, "signup_to_marketing_email")
def test_marketing_signup_new_user_enabled(mock_signup, monkeypatch, settings):
"""Test marketing signup for new user with settings enabled."""
settings.SIGNUP_NEW_USER_TO_MARKETING_EMAIL = True
klass = OIDCAuthenticationBackend()
email = "test@example.com"
def get_userinfo_mocked(*args):
return {"sub": "123", "email": email}
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
user = klass.get_or_create_user("test-token", None, None)
assert user.email == email
mock_signup.assert_called_once_with(email)
@mock.patch.object(OIDCAuthenticationBackend, "signup_to_marketing_email")
def test_marketing_signup_new_user_disabled(mock_signup, monkeypatch, settings):
"""Test no marketing signup for new user with settings disabled."""
settings.SIGNUP_NEW_USER_TO_MARKETING_EMAIL = False
klass = OIDCAuthenticationBackend()
email = "test@example.com"
def get_userinfo_mocked(*args):
return {"sub": "123", "email": email}
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
user = klass.get_or_create_user("test-token", None, None)
assert user.email == email
mock_signup.assert_not_called()
@mock.patch.object(OIDCAuthenticationBackend, "signup_to_marketing_email")
def test_marketing_signup_new_user_default_disabled(mock_signup, monkeypatch):
"""Test no marketing signup for new user with settings by default disabled."""
klass = OIDCAuthenticationBackend()
email = "test@example.com"
def get_userinfo_mocked(*args):
return {"sub": "123", "email": email}
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
user = klass.get_or_create_user("test-token", None, None)
assert user.email == email
mock_signup.assert_not_called()
@pytest.mark.parametrize(
"is_signup_enabled",
[True, False],
)
@mock.patch.object(OIDCAuthenticationBackend, "signup_to_marketing_email")
def test_marketing_signup_existing_user(
mock_signup, monkeypatch, settings, is_signup_enabled
):
"""Test no marketing signup for existing user regardless of settings."""
settings.SIGNUP_NEW_USER_TO_MARKETING_EMAIL = is_signup_enabled
klass = OIDCAuthenticationBackend()
db_user = UserFactory(email="test@example.com")
def get_userinfo_mocked(*args):
return {"sub": db_user.sub, "email": db_user.email}
monkeypatch.setattr(OIDCAuthenticationBackend, "get_userinfo", get_userinfo_mocked)
user = klass.get_or_create_user("test-token", None, None)
assert user == db_user
mock_signup.assert_not_called()
@mock.patch("core.authentication.backends.get_marketing_service")
def test_signup_to_marketing_email_success(mock_marketing):
"""Test successful marketing signup."""
email = "test@example.com"
# Call the method
OIDCAuthenticationBackend.signup_to_marketing_email(email)
# Verify service interaction
mock_service = mock_marketing.return_value
mock_service.create_contact.assert_called_once()
@pytest.mark.parametrize(
"error",
[
ImportError,
ImproperlyConfigured,
],
)
@mock.patch("core.authentication.backends.get_marketing_service")
def test_marketing_signup_handles_service_initialization_errors(
mock_marketing, error, settings
):
"""Tests errors that occur when trying to get/initialize the marketing service."""
settings.SIGNUP_NEW_USER_TO_MARKETING_EMAIL = True
mock_marketing.side_effect = error
# Should not raise any exception
OIDCAuthenticationBackend.signup_to_marketing_email("test@example.com")
@pytest.mark.parametrize(
"error",
[
marketing_service.ContactCreationError,
ImproperlyConfigured,
ImportError,
],
)
@mock.patch("core.authentication.backends.get_marketing_service")
def test_marketing_signup_handles_contact_creation_errors(
mock_marketing, error, settings
):
"""Tests errors that occur during the contact creation process."""
settings.SIGNUP_NEW_USER_TO_MARKETING_EMAIL = True
mock_marketing.return_value.create_contact.side_effect = error
# Should not raise any exception
OIDCAuthenticationBackend.signup_to_marketing_email("test@example.com")
@@ -0,0 +1,145 @@
"""
Test event authentication.
"""
# pylint: disable=E1128
from django.test import RequestFactory
import pytest
from rest_framework.exceptions import AuthenticationFailed
from core.recording.event.authentication import (
MachineUser,
StorageEventAuthentication,
)
def test_successful_authentication(settings):
"""Test successful authentication with valid token."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "Bearer valid-test-token"}
user, token = StorageEventAuthentication().authenticate(request)
assert token == "valid-test-token"
assert isinstance(user, MachineUser)
def test_disabled_authentication_with_header(settings):
"""Authentication should pass when no auth is configured, and header is present."""
settings.RECORDING_STORAGE_EVENT_TOKEN = None
settings.RECORDING_ENABLE_STORAGE_EVENT_AUTH = False
request = RequestFactory().get("/")
request.headers = {"Authorization": "Bearer some-token"}
user, token = StorageEventAuthentication().authenticate(request)
assert token is None
assert isinstance(user, MachineUser)
def test_disabled_authentication_without_header(settings):
"""Authentication should pass when no auth is configured, and no header is present."""
settings.RECORDING_STORAGE_EVENT_TOKEN = None
settings.RECORDING_ENABLE_STORAGE_EVENT_AUTH = False
request = RequestFactory().get("/")
user, token = StorageEventAuthentication().authenticate(request)
assert token is None
assert isinstance(user, MachineUser)
def test_authentication_when_disabled(settings):
"""Authentication should pass when disabled, regardless of token configuration."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "some-token"
settings.RECORDING_ENABLE_STORAGE_EVENT_AUTH = False
request = RequestFactory().get("/")
user, token = StorageEventAuthentication().authenticate(request)
assert token is None
assert isinstance(user, MachineUser)
def test_authentication_fails_when_token_not_configured(settings):
"""Authentication should fail when authentication is enabled but no token is configured."""
# By default RECORDING_ENABLE_STORAGE_EVENT_AUTH should be True
settings.RECORDING_STORAGE_EVENT_TOKEN = None
request = RequestFactory().get("/")
with pytest.raises(
AuthenticationFailed,
match="Authentication is enabled but token is not configured",
):
StorageEventAuthentication().authenticate(request)
def test_missing_auth_header(settings):
"""Test failure when Authorization header is missing."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {}
with pytest.raises(AuthenticationFailed, match="Authorization header is required"):
StorageEventAuthentication().authenticate(request)
def test_invalid_auth_header_format(settings):
"""Test failure when Authorization header has invalid format."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "InvalidFormat"}
with pytest.raises(AuthenticationFailed, match="Invalid authorization header"):
StorageEventAuthentication().authenticate(request)
def test_invalid_token_type(settings):
"""Test failure when token type is not Bearer."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "Basic some-token"}
with pytest.raises(AuthenticationFailed, match="Invalid authorization header"):
StorageEventAuthentication().authenticate(request)
def test_invalid_token(settings):
"""Test failure when token is invalid."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "Bearer wrong-token"}
with pytest.raises(AuthenticationFailed, match="Invalid token"):
StorageEventAuthentication().authenticate(request)
def test_malformed_auth_header(settings):
"""Test failure when Authorization header is malformed."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "Bearer"} # Missing token part
with pytest.raises(AuthenticationFailed, match="Invalid authorization header"):
StorageEventAuthentication().authenticate(request)
def test_authenticate_header():
"""Test the WWW-Authenticate header value."""
request = RequestFactory().get("/")
header = StorageEventAuthentication().authenticate_header(request)
assert header == "Bearer realm='Storage event API'"
def test_multiple_spaces_in_auth_header(settings):
"""Test failure when Authorization header contains multiple spaces."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "Bearer extra-spaces-token"}
with pytest.raises(AuthenticationFailed, match="Invalid authorization header"):
StorageEventAuthentication().authenticate(request)
@@ -0,0 +1,310 @@
"""
Test event parsers.
"""
# pylint: disable=W0212,W0621,W0613
from unittest import mock
from django.conf import settings
import pytest
from core.recording.event.exceptions import (
InvalidBucketError,
InvalidFilepathError,
InvalidFileTypeError,
ParsingEventDataError,
)
from core.recording.event.parsers import (
MinioParser,
StorageEvent,
get_parser,
)
@pytest.fixture
def valid_minio_event():
"""Mock a valid Minio event."""
return {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": "recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
"contentType": "audio/ogg",
},
}
}
]
}
@pytest.fixture
def minio_parser():
"""Mock a Minio parser."""
return MinioParser(bucket_name="test-bucket")
def test_parse_valid_event(minio_parser, valid_minio_event):
"""Test parsing a valid Minio event."""
event = minio_parser.parse(valid_minio_event)
assert isinstance(event, StorageEvent)
assert event.filepath == "recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg"
assert event.filetype == "audio/ogg"
assert event.bucket_name == "test-bucket"
assert event.metadata is None
def test_parse_empty_data(minio_parser):
"""Test parsing empty event data raises error."""
with pytest.raises(ParsingEventDataError, match="Received empty data."):
minio_parser.parse({})
def test_parse_missing_keys(minio_parser):
"""Test parsing event with missing key."""
invalid_minio_event = {
"Records": [
{
"s3": {
"bucket": {"name": None},
# Missing 'object' key
}
}
]
}
with pytest.raises(ParsingEventDataError, match="Missing or malformed key"):
minio_parser.parse(invalid_minio_event)
def test_parse_none_key(minio_parser):
"""Test parsing event with None field."""
invalid_minio_event = {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": "recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
"contentType": None, # 'contentType' should not be None
},
}
}
]
}
with pytest.raises(ParsingEventDataError, match="Missing essential data fields"):
minio_parser.parse(invalid_minio_event)
def test_validate_invalid_bucket(minio_parser):
"""Test validation with wrong bucket name."""
event = StorageEvent(
filepath="recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/ogg",
bucket_name="wrong-bucket",
metadata=None,
)
with pytest.raises(InvalidBucketError):
minio_parser.validate(event)
def test_validate_invalid_filetype(minio_parser):
"""Test validation with unsupported file type."""
event = StorageEvent(
filepath="recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.txt",
filetype="text/plain", # Not included in the default allowed filetypes
bucket_name="test-bucket",
metadata=None,
)
with pytest.raises(InvalidFileTypeError):
minio_parser.validate(event)
@pytest.mark.parametrize(
"invalid_filepath",
[
"invalid_filepath",
"recording/46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
"recording%2F46d1a1212426484d8fb309b5d886f7a8.ogg",
],
)
def test_validate_invalid_filepath(invalid_filepath, minio_parser):
"""Test validation with malformed filepath."""
event = StorageEvent(
filepath=invalid_filepath,
filetype="audio/ogg",
bucket_name="test-bucket",
metadata=None,
)
with pytest.raises(InvalidFilepathError):
minio_parser.validate(event)
def test_validate_valid_event(minio_parser):
"""Test validation with valid event data."""
event = StorageEvent(
filepath="recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/ogg",
bucket_name="test-bucket",
metadata=None,
)
recording_id = minio_parser.validate(event)
assert recording_id == "46d1a121-2426-484d-8fb3-09b5d886f7a8"
def test_get_recording_id_success(minio_parser, valid_minio_event):
"""Test successful extraction of recording ID."""
recording_id = minio_parser.get_recording_id(valid_minio_event)
assert recording_id == "46d1a121-2426-484d-8fb3-09b5d886f7a8"
def test_validate_filepath_with_folder(minio_parser):
"""Test validation of filepath with folder structure."""
event = StorageEvent(
filepath="parent_folder%2Ffolder%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/ogg",
bucket_name="test-bucket",
metadata=None,
)
recording_id = minio_parser.validate(event)
assert recording_id == "46d1a121-2426-484d-8fb3-09b5d886f7a8"
def test_parse_with_video_type(minio_parser):
"""Test parsing event with video file type."""
video_event = {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": "46d1a121-2426-484d-8fb3-09b5d886f7a8.mp4",
"contentType": "video/mp4",
},
}
}
]
}
event = minio_parser.parse(video_event)
assert event.filetype == "video/mp4"
assert event.filepath.endswith(".mp4")
def test_empty_allowed_filetypes():
"""Test MinioParser with empty allowed_filetypes."""
empty_types = set()
parser = MinioParser(bucket_name="test-bucket", allowed_filetypes=empty_types)
assert parser._allowed_filetypes == {"audio/ogg", "video/mp4"}
def test_custom_allowed_filetypes():
"""Test MinioParser with empty allowed_filetypes."""
custom_types = {"audio/mp3", "video/mov"}
parser = MinioParser(bucket_name="test-bucket", allowed_filetypes=custom_types)
assert parser._allowed_filetypes == {"audio/mp3", "video/mov"}
def test_validate_custom_filetypes():
"""Test validation of filepath with folder structure."""
parser = MinioParser(bucket_name="test-bucket", allowed_filetypes={"audio/mp3"})
event = StorageEvent(
filepath="parent_folder%2Ffolder%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/mp3",
bucket_name="test-bucket",
metadata=None,
)
parser.validate(event)
def test_constructor_none_bucket():
"""Test MinioParser constructor with None bucket name."""
with pytest.raises(ValueError, match="Bucket name cannot be None or empty"):
MinioParser(bucket_name=None)
def test_constructor_empty_bucket():
"""Test MinioParser constructor with empty bucket name."""
with pytest.raises(ValueError, match="Bucket name cannot be None or empty"):
MinioParser(bucket_name="")
@pytest.fixture
def clear_lru_cache():
"""Fixture to clear the LRU cache between tests."""
get_parser.cache_clear()
yield
get_parser.cache_clear()
def test_returns_correct_instance(clear_lru_cache):
"""Test if get_parser returns the correct parser instance."""
settings.AWS_STORAGE_BUCKET_NAME = "test-bucket"
parser = get_parser()
assert isinstance(parser, MinioParser)
assert parser._bucket_name == "test-bucket"
def test_caching_behavior(clear_lru_cache):
"""Test if the function properly caches the parser instance."""
settings.AWS_STORAGE_BUCKET_NAME = "test-bucket"
parser1 = get_parser()
parser2 = get_parser()
assert parser1 is parser2 # Check object identity
def test_different_settings_new_instance():
"""Test if changing settings creates a new instance."""
settings.AWS_STORAGE_BUCKET_NAME = "different-bucket"
parser = get_parser()
assert parser._bucket_name == "different-bucket"
def test_import_error_handling(clear_lru_cache):
"""Test handling of import errors for invalid parser class."""
settings.RECORDING_EVENT_PARSER_CLASS = "invalid.parser.path"
with pytest.raises(ImportError):
get_parser()
@mock.patch("core.recording.event.parsers.import_string")
def test_parser_instantiation_called_once(mock_import_string, clear_lru_cache):
"""Test that parser class is instantiated only once due to caching."""
mock_parser_cls = type(
"MockParser",
(),
{
"__init__": lambda self, bucket_name: setattr(
self, "_bucket_name", bucket_name
)
},
)
mock_import_string.return_value = mock_parser_cls
# First call
parser1 = get_parser()
# Second call
parser2 = get_parser()
# Verify import_string was called only once
mock_import_string.assert_called_once_with(settings.RECORDING_EVENT_PARSER_CLASS)
assert parser1 is parser2
def test_cache_clear_behavior(clear_lru_cache, settings):
"""Test that cache clearing creates new instance."""
settings.RECORDING_EVENT_PARSER_CLASS = "core.recording.event.parsers.MinioParser"
parser1 = get_parser()
get_parser.cache_clear()
parser2 = get_parser()
assert parser1 is not parser2 # Should be different instances after cache clear
@@ -0,0 +1,114 @@
"""
Test recordings API endpoints in the Meet core app: delete.
"""
import pytest
from rest_framework.test import APIClient
from ...factories import RecordingFactory, UserFactory, UserRecordingAccessFactory
from ...models import Recording
pytestmark = pytest.mark.django_db
def test_api_recordings_delete_anonymous():
"""Anonymous users should not be allowed to destroy a recording."""
recording = RecordingFactory()
client = APIClient()
response = client.delete(
f"/api/v1.0/recordings/{recording.id!s}/",
)
assert response.status_code == 401
assert Recording.objects.count() == 1
def test_api_recordings_delete_authenticated():
"""
Authenticated users should not be allowed to delete a recording
from which they are not related.
"""
recording = RecordingFactory()
user = UserFactory()
client = APIClient()
client.force_login(user)
response = client.delete(
f"/api/v1.0/recordings/{recording.id!s}/",
)
assert response.status_code == 404
assert Recording.objects.count() == 1
def test_api_recordings_delete_members():
"""
Authenticated users should not be allowed to delete a recording
from which they are only a member.
"""
user = UserFactory()
access = UserRecordingAccessFactory(role="member", user=user)
client = APIClient()
client.force_login(user)
response = client.delete(
f"/api/v1.0/recordings/{access.recording.id}/",
)
assert response.status_code == 403
assert Recording.objects.count() == 1
@pytest.mark.parametrize(
"role",
["owner", "administrator"],
)
def test_api_recordings_delete_active(role):
"""
Authenticated users cannot delete active recordings, even with deletion privileges.
"""
user = UserFactory()
recording = RecordingFactory(status="active")
access = UserRecordingAccessFactory(role=role, user=user, recording=recording)
client = APIClient()
client.force_login(user)
response = client.delete(
f"/api/v1.0/recordings/{access.recording.id}/",
)
assert response.status_code == 403
assert Recording.objects.count() == 1
@pytest.mark.parametrize(
"role",
["owner", "administrator"],
)
def test_api_recordings_delete_final(role):
"""
Authenticated users should not be allowed to delete an active recording
from which they are an admin or owner.
"""
user = UserFactory()
recording = RecordingFactory(status="saved")
access = UserRecordingAccessFactory(role=role, user=user, recording=recording)
client = APIClient()
client.force_login(user)
response = client.delete(
f"/api/v1.0/recordings/{access.recording.id}/",
)
assert response.status_code == 204
assert Recording.objects.count() == 0
@@ -0,0 +1,180 @@
"""
Test recordings API endpoints in the Meet core app: list.
"""
import operator
from unittest import mock
import pytest
from rest_framework.pagination import PageNumberPagination
from rest_framework.test import APIClient
from core import factories
pytestmark = pytest.mark.django_db
def test_api_recordings_list_anonymous():
"""Anonymous users should not be able to list recordings."""
factories.RecordingFactory()
response = APIClient().get("/api/v1.0/recordings/")
assert response.status_code == 401
@pytest.mark.parametrize(
"role",
["administrator", "member", "owner"],
)
def test_api_recordings_list_authenticated_direct(role):
"""
Authenticated users listing recordings, should only see the recordings
to which they are related.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
other_user = factories.UserFactory()
access = factories.UserRecordingAccessFactory(role=role, user=user)
factories.UserRecordingAccessFactory(user=other_user)
recording = access.recording
room = recording.room
response = client.get(
"/api/v1.0/recordings/",
)
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 1
expected_ids = {
str(recording.id),
}
result_ids = {result["id"] for result in results}
assert expected_ids == result_ids
assert results[0] == {
"id": str(recording.id),
"created_at": recording.created_at.isoformat().replace("+00:00", "Z"),
"room": {
"id": str(room.id),
"is_public": room.is_public,
"name": room.name,
"slug": room.slug,
},
"status": "initiated",
"updated_at": recording.updated_at.isoformat().replace("+00:00", "Z"),
}
def test_api_recording_list_authenticated_via_team(mock_user_get_teams):
"""
Authenticated users should be able to list recordings they are a
owner/administrator/member of via a team.
"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
mock_user_get_teams.return_value = ["team1", "team2", "unknown"]
recordings_team1 = [
access.recording
for access in factories.TeamRecordingAccessFactory.create_batch(2, team="team1")
]
recordings_team2 = [
access.recording
for access in factories.TeamRecordingAccessFactory.create_batch(3, team="team2")
]
expected_ids = {
str(recording.id) for recording in recordings_team1 + recordings_team2
}
response = client.get("/api/v1.0/recordings/")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 5
results_id = {result["id"] for result in results}
assert expected_ids == results_id
@mock.patch.object(PageNumberPagination, "get_page_size", return_value=2)
def test_api_recordings_list_pagination(_mock_page_size):
"""Pagination should work as expected."""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
recording_ids = [
str(access.recording_id)
for access in factories.UserRecordingAccessFactory.create_batch(3, user=user)
]
response = client.get("/api/v1.0/recordings/")
assert response.status_code == 200
content = response.json()
assert content["count"] == 3
assert content["next"] == "http://testserver/api/v1.0/recordings/?page=2"
assert content["previous"] is None
assert len(content["results"]) == 2
for item in content["results"]:
recording_ids.remove(item["id"])
# Get page 2
response = client.get(
"/api/v1.0/recordings/?page=2",
)
assert response.status_code == 200
content = response.json()
assert content["count"] == 3
assert content["next"] is None
assert content["previous"], "http://testserver/api/v1.0/recordings/"
assert len(content["results"]) == 1
recording_ids.remove(content["results"][0]["id"])
assert recording_ids == []
def test_api_recordings_list_authenticated_distinct():
"""A recording for a room with several related users should only be listed once."""
user = factories.UserFactory()
other_user = factories.UserFactory()
client = APIClient()
client.force_login(user)
recording = factories.RecordingFactory(users=[user, other_user])
response = client.get("/api/v1.0/recordings/")
assert response.status_code == 200
content = response.json()
assert len(content["results"]) == 1
assert content["results"][0]["id"] == str(recording.id)
def test_api_recordings_list_ordering_default():
"""Recordings should be ordered by descending "updated_at" by default"""
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
factories.RecordingFactory.create_batch(5, users=[user])
response = client.get("/api/v1.0/recordings/")
assert response.status_code == 200
results = response.json()["results"]
assert len(results) == 5
# Check that results are sorted by descending "updated_at" as expected
for i in range(4):
assert operator.ge(results[i]["updated_at"], results[i + 1]["updated_at"])
@@ -0,0 +1,203 @@
"""
Test recordings API endpoints in the Meet core app: save recording.
"""
# pylint: disable=W0621,W0613
import uuid
from unittest import mock
import pytest
from rest_framework.test import APIClient
from ...factories import RecordingFactory
from ...models import Recording, RecordingStatusChoices
from ...recording.event.exceptions import (
InvalidBucketError,
InvalidFileTypeError,
ParsingEventDataError,
)
pytestmark = pytest.mark.django_db
@pytest.fixture
def recording_settings(settings):
"""Configure recording-related and storage event Django settings."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "testAuthToken"
settings.RECORDING_STORAGE_EVENT_ENABLE = True
return settings
@pytest.fixture
def mock_get_parser():
"""Mock 'get_parser' factory function."""
with mock.patch("core.api.viewsets.get_parser") as mock_parser:
yield mock_parser
def test_save_recording_anonymous(settings, client):
"""Anonymous users should not be allowed to save room recordings."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "testAuthToken"
RecordingFactory(status="active")
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
)
assert response.status_code == 401
assert Recording.objects.count() == 1
def test_save_recording_wrong_bearer(settings, client):
"""Requests with incorrect bearer token should be rejected when auth is required."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "testAuthToken"
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer wrongAuthToken",
)
assert response.status_code == 401
def test_save_recording_permission_needed(settings, client):
"""Recordings should not be saved when feature is disabled."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "testAuthToken"
settings.RECORDING_STORAGE_EVENT_ENABLE = False
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 403
def test_save_recording_parsing_error(recording_settings, mock_get_parser, client):
"""Test handling of parsing errors in recording event data."""
mock_parser = mock.Mock()
mock_parser.get_recording_id.side_effect = ParsingEventDataError("Error message")
mock_get_parser.return_value = mock_parser
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 403
assert response.json() == {"detail": "Invalid request data: Error message"}
def test_save_recording_bucket_error(recording_settings, mock_get_parser, client):
"""Test handling of invalid storage bucket errors in recording event data."""
mock_parser = mock.Mock()
mock_parser.get_recording_id.side_effect = InvalidBucketError("Error message")
mock_get_parser.return_value = mock_parser
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 403
assert response.json() == {"detail": "Invalid bucket specified"}
def test_save_recording_filetype_error(recording_settings, mock_get_parser):
"""Test handling of unsupported file types in recording event data."""
mock_parser = mock.Mock()
mock_parser.get_recording_id.side_effect = InvalidFileTypeError(
"unsupported '.json'"
)
mock_get_parser.return_value = mock_parser
client = APIClient()
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 200
assert response.json() == {"message": "Ignore this file type, unsupported '.json'"}
def test_save_recording_unknown_recording(recording_settings, mock_get_parser, client):
"""Test handling of events for non-existent recordings."""
RecordingFactory(status="active")
mock_parser = mock.Mock()
mock_parser.get_recording_id.return_value = uuid.uuid4()
mock_get_parser.return_value = mock_parser
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 404
assert response.json() == {"detail": "No recording found for this event."}
@pytest.mark.parametrize(
"status", ["failed_to_start", "aborted", "failed_to_stop", "saved", "initiated"]
)
def test_save_recording_non_savable_recording(
recording_settings, mock_get_parser, client, status
):
"""Test that recordings in non-savable states cannot be saved."""
recording = RecordingFactory(status=status)
mock_parser = mock.Mock()
mock_parser.get_recording_id.return_value = recording.id
mock_get_parser.return_value = mock_parser
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 403
assert response.json() == {
"detail": f"Recording with ID {recording.id} cannot be saved because it is either,"
" in an error state or has already been saved."
}
@pytest.mark.parametrize("status", ["active", "stopped"])
def test_save_recording_success(recording_settings, mock_get_parser, client, status):
"""Test successful saving of recordings in valid states."""
recording = RecordingFactory(status=status)
mock_parser = mock.Mock()
mock_parser.get_recording_id.return_value = recording.id
mock_get_parser.return_value = mock_parser
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 200
assert response.json() == {"message": "Event processed."}
recording.refresh_from_db()
assert recording.status == RecordingStatusChoices.SAVED
@@ -0,0 +1,171 @@
"""
Test worker service factories.
"""
# pylint: disable=W0212,W0621,W0613
from dataclasses import FrozenInstanceError
from unittest.mock import Mock
from django.test import override_settings
import pytest
from core.recording.worker.factories import (
WorkerService,
WorkerServiceConfig,
get_worker_service,
)
@pytest.fixture(autouse=True)
def clear_lru_cache():
"""Clear the lru_cache before and after each test"""
WorkerServiceConfig.from_settings.cache_clear()
yield
WorkerServiceConfig.from_settings.cache_clear()
@pytest.fixture
def test_settings():
"""Fixture to provide test Django settings"""
mocked_settings = {
"RECORDING_OUTPUT_FOLDER": "/test/output",
"LIVEKIT_CONFIGURATION": {"server": "test.example.com"},
"RECORDING_VERIFY_SSL": True,
"AWS_S3_ENDPOINT_URL": "https://s3.test.com",
"AWS_S3_ACCESS_KEY_ID": "test_key",
"AWS_S3_SECRET_ACCESS_KEY": "test_secret",
"AWS_S3_REGION_NAME": "test-region",
"AWS_STORAGE_BUCKET_NAME": "test-bucket",
}
# Use override_settings to properly patch Django settings
with override_settings(**mocked_settings):
yield test_settings
@pytest.fixture
def default_config(test_settings):
"""Fixture to provide a WorkerServiceConfig instance"""
return WorkerServiceConfig.from_settings()
# Tests
def test_config_initialization(default_config):
"""Test that WorkerServiceConfig is properly initialized from settings"""
assert default_config.output_folder == "/test/output"
assert default_config.server_configurations == {"server": "test.example.com"}
assert default_config.verify_ssl is True
assert default_config.bucket_args == {
"endpoint": "https://s3.test.com",
"access_key": "test_key",
"secret": "test_secret",
"region": "test-region",
"bucket": "test-bucket",
"force_path_style": True,
}
def test_config_immutability(default_config):
"""Test that config instances are immutable after creation"""
with pytest.raises(FrozenInstanceError):
default_config.output_folder = "new/path"
@override_settings(
RECORDING_OUTPUT_FOLDER="/test/output",
LIVEKIT_CONFIGURATION={"server": "test.example.com"},
RECORDING_VERIFY_SSL=True,
AWS_S3_ENDPOINT_URL="https://s3.test.com",
AWS_S3_ACCESS_KEY_ID="test_key",
AWS_S3_SECRET_ACCESS_KEY="test_secret",
AWS_S3_REGION_NAME="test-region",
AWS_STORAGE_BUCKET_NAME="test-bucket",
)
def test_config_caching():
"""Test that from_settings method caches its result"""
# Clear cache before testing caching behavior
WorkerServiceConfig.from_settings.cache_clear()
config1 = WorkerServiceConfig.from_settings()
config2 = WorkerServiceConfig.from_settings()
assert config1 is config2
class MockWorkerService(WorkerService):
"""Mock worker service for testing."""
def __init__(self, config):
self.config = config
@pytest.fixture
def mock_import_string(monkeypatch):
"""Fixture to mock import_string function."""
mock = Mock(return_value=MockWorkerService)
monkeypatch.setattr("core.recording.worker.factories.import_string", mock)
return mock
def test_factory_valid_mode(mock_import_string, settings, default_config):
"""Test getting worker service with valid mode."""
settings.RECORDING_WORKER_CLASSES = {
"test_mode": "path.to.MockWorkerService",
"another_mode": "path.to.AnotherWorkerService",
}
worker = get_worker_service("test_mode")
mock_import_string.assert_called_once_with("path.to.MockWorkerService")
assert isinstance(worker, MockWorkerService)
assert worker.config == default_config
def test_factory_invalid_mode(settings, mock_import_string, default_config):
"""Test getting worker service with invalid mode raises ValueError."""
settings.RECORDING_WORKER_CLASSES = {
"test_mode": "path.to.MockWorkerService",
"another_mode": "path.to.AnotherWorkerService",
}
worker = get_worker_service("test_mode")
mock_import_string.assert_called_once_with("path.to.MockWorkerService")
assert isinstance(worker, MockWorkerService)
with pytest.raises(ValueError) as exc_info:
get_worker_service("invalid_mode")
mock_import_string.assert_not_called()
assert "Recording mode 'invalid_mode' not found" in str(exc_info.value)
assert "Available modes: ['test_mode', 'another_mode']" in str(exc_info.value)
def test_factory_import_error(mock_import_string, settings):
"""Test handling of import errors."""
mock_import_string.side_effect = ImportError("Module not found")
settings.RECORDING_WORKER_CLASSES = {
"test_mode": "path.to.MockWorkerService",
"another_mode": "path.to.AnotherWorkerService",
}
with pytest.raises(ImportError) as exc_info:
get_worker_service("test_mode")
assert "Module not found" in str(exc_info.value)
def test_factory_empty_registry(settings):
"""Test behavior when worker registry is empty."""
settings.RECORDING_WORKER_CLASSES = {}
with pytest.raises(ValueError) as exc_info:
get_worker_service("any_mode")
assert "Available modes: []" in str(exc_info.value)
@@ -0,0 +1,161 @@
"""Test WorkerServiceMediator class."""
# pylint: disable=W0621,W0613
from unittest.mock import Mock
import pytest
from core.factories import RecordingFactory
from core.models import RecordingStatusChoices
from core.recording.worker.exceptions import (
RecordingStartError,
RecordingStopError,
WorkerConnectionError,
WorkerRequestError,
WorkerResponseError,
)
from core.recording.worker.factories import WorkerService
from core.recording.worker.mediator import WorkerServiceMediator
pytestmark = pytest.mark.django_db
@pytest.fixture
def mock_worker_service():
"""Fixture for mock worker service"""
return Mock(spec=WorkerService)
@pytest.fixture
def mediator(mock_worker_service):
"""Fixture for WorkerServiceMediator"""
return WorkerServiceMediator(mock_worker_service)
def test_start_recording_success(mediator, mock_worker_service):
"""Test successful recording start"""
# Setup
worker_id = "test-worker-123"
mock_worker_service.start.return_value = worker_id
mock_recording = RecordingFactory(
status=RecordingStatusChoices.INITIATED, worker_id=None
)
mediator.start(mock_recording)
# Verify worker service call
expected_room_name = str(mock_recording.room.id)
mock_worker_service.start.assert_called_once_with(
expected_room_name, mock_recording.id
)
# Verify recording updates
mock_recording.refresh_from_db()
assert mock_recording.worker_id == worker_id
assert mock_recording.status == RecordingStatusChoices.ACTIVE
@pytest.mark.parametrize(
"error_class", [WorkerRequestError, WorkerConnectionError, WorkerResponseError]
)
def test_mediator_start_recording_worker_errors(
mediator, mock_worker_service, error_class
):
"""Test handling of various worker errors during start"""
# Setup
mock_worker_service.start.side_effect = error_class("Test error")
mock_recording = RecordingFactory(
status=RecordingStatusChoices.INITIATED, worker_id=None
)
# Execute and verify
with pytest.raises(RecordingStartError):
mediator.start(mock_recording)
# Verify recording updates
mock_recording.refresh_from_db()
assert mock_recording.status == RecordingStatusChoices.FAILED_TO_START
assert mock_recording.worker_id is None
@pytest.mark.parametrize(
"status",
[
RecordingStatusChoices.ACTIVE,
RecordingStatusChoices.FAILED_TO_START,
RecordingStatusChoices.FAILED_TO_STOP,
RecordingStatusChoices.STOPPED,
RecordingStatusChoices.SAVED,
RecordingStatusChoices.ABORTED,
],
)
def test_mediator_start_recording_from_forbidden_status(
mediator, mock_worker_service, status
):
"""Test handling of various worker errors during start"""
# Setup
mock_recording = RecordingFactory(status=status)
# Execute and verify
with pytest.raises(RecordingStartError):
mediator.start(mock_recording)
# Verify recording was not updated
mock_recording.refresh_from_db()
assert mock_recording.status == status
def test_mediator_stop_recording_success(mediator, mock_worker_service):
"""Test successful recording stop"""
# Setup
mock_recording = RecordingFactory(
status=RecordingStatusChoices.ACTIVE, worker_id="test-worker-123"
)
mock_worker_service.stop.return_value = "STOPPED"
# Execute
mediator.stop(mock_recording)
# Verify worker service call
mock_worker_service.stop.assert_called_once_with(worker_id=mock_recording.worker_id)
# Verify recording updates
mock_recording.refresh_from_db()
assert mock_recording.status == RecordingStatusChoices.STOPPED
def test_mediator_stop_recording_aborted(mediator, mock_worker_service):
"""Test recording stop when worker returns ABORTED"""
# Setup
mock_recording = RecordingFactory(
status=RecordingStatusChoices.ACTIVE, worker_id="test-worker-123"
)
mock_worker_service.stop.return_value = "ABORTED"
# Execute
mediator.stop(mock_recording)
# Verify recording updates
mock_recording.refresh_from_db()
assert mock_recording.status == RecordingStatusChoices.ABORTED
@pytest.mark.parametrize("error_class", [WorkerConnectionError, WorkerResponseError])
def test_mediator_stop_recording_worker_errors(
mediator, mock_worker_service, error_class
):
"""Test handling of worker errors during stop"""
# Setup
mock_recording = RecordingFactory(
status=RecordingStatusChoices.ACTIVE, worker_id="test-worker-123"
)
mock_worker_service.stop.side_effect = error_class("Test error")
# Execute and verify
with pytest.raises(RecordingStopError):
mediator.stop(mock_recording)
# Verify recording updates
mock_recording.refresh_from_db()
assert mock_recording.status == RecordingStatusChoices.FAILED_TO_STOP
@@ -0,0 +1,368 @@
"""
Test worker service classes.
"""
# pylint: disable=W0212,W0621,W0613,E1101
from unittest.mock import AsyncMock, Mock, patch
import aiohttp
import pytest
from core.recording.worker.exceptions import WorkerConnectionError, WorkerResponseError
from core.recording.worker.factories import WorkerServiceConfig
from core.recording.worker.services import (
AudioCompositeEgressService,
BaseEgressService,
VideoCompositeEgressService,
livekit_api,
)
@pytest.fixture
def config():
"""Fixture to provide a WorkerServiceConfig instance"""
return WorkerServiceConfig(
output_folder="/test/output",
server_configurations={
"host": "test.livekit.io",
"api_key": "test_key",
"api_secret": "test_secret",
},
verify_ssl=True,
bucket_args={
"endpoint": "https://s3.test.com",
"access_key": "test_key",
"secret": "test_secret",
"region": "test-region",
"bucket": "test-bucket",
"force_path_style": True,
},
)
@pytest.fixture
def mock_s3_upload():
"""Fixture for mocked S3Upload"""
with patch("core.recording.worker.services.livekit_api.S3Upload") as mock:
yield mock
@pytest.fixture
def mock_egress_service():
"""Fixture for mocked EgressService"""
with patch("core.recording.worker.services.EgressService") as mock:
yield mock
@pytest.fixture
def service(config, mock_s3_upload):
"""Fixture for BaseEgressService instance"""
return BaseEgressService(config)
@pytest.fixture
def mock_client_session():
"""Fixture for mocked aiohttp.ClientSession"""
with patch("aiohttp.ClientSession") as mock:
mock.return_value.__aenter__ = AsyncMock()
mock.return_value.__aexit__ = AsyncMock()
yield mock
@pytest.fixture
def mock_tcp_connector():
"""Fixture for TCPConnector"""
with patch("aiohttp.TCPConnector") as mock_connector:
mock_connector_instance = Mock()
mock_connector.return_value = mock_connector_instance
yield mock_connector
@pytest.fixture
def video_service(config):
"""Fixture for VideoCompositeEgressService"""
service = VideoCompositeEgressService(config)
service._handle_request = Mock() # Mock the request handler
return service
@pytest.fixture
def audio_service(config):
"""Fixture for AudioCompositeEgressService"""
service = AudioCompositeEgressService(config)
service._handle_request = Mock() # Mock the request handler
return service
def test_base_egress_initialization(config, mock_s3_upload):
"""Test service initialization"""
service = BaseEgressService(config)
assert service._config == config
mock_s3_upload.assert_called_once_with(
endpoint="https://s3.test.com",
access_key="test_key",
secret="test_secret",
region="test-region",
bucket="test-bucket",
force_path_style=True,
)
@pytest.mark.parametrize(
"filename,extension,expected",
[
("test", "mp4", "/test/output/test.mp4"),
("recording123", "ogg", "/test/output/recording123.ogg"),
("live_stream", "m3u8", "/test/output/live_stream.m3u8"),
],
)
def test_base_egress_filepath_construction(service, filename, extension, expected):
"""Test filepath construction with various inputs"""
result = service._get_filepath(filename, extension)
assert result == expected
assert result.startswith(service._config.output_folder)
assert result.endswith(f"{filename}.{extension}")
def test_base_egress_handle_request_success(
config, service, mock_client_session, mock_egress_service, mock_tcp_connector
):
"""Test successful request handling"""
# Setup mock response
mock_response = Mock()
mock_method = AsyncMock(return_value=mock_response)
mock_egress_instance = Mock()
mock_egress_instance.test_method = mock_method
mock_egress_service.return_value = mock_egress_instance
# Create test request
test_request = Mock()
response = service._handle_request(test_request, "test_method")
mock_client_session.assert_called_once_with(
connector=mock_tcp_connector.return_value
)
# Verify EgressService initialization
mock_egress_service.assert_called_once_with(
mock_client_session.return_value.__aenter__.return_value,
**service._config.server_configurations,
)
# Verify method call and response
mock_method.assert_called_once_with(test_request)
assert response == mock_response
def test_base_egress_handle_request_connection_error(service, mock_egress_service):
"""Test handling of connection errors"""
# Setup mock error
mock_method = AsyncMock(
side_effect=livekit_api.TwirpError(msg="Connection failed", code=500)
)
mock_egress_instance = Mock()
mock_egress_instance.test_method = mock_method
mock_egress_service.return_value = mock_egress_instance
# Create test request
test_request = Mock()
# Verify error handling
with pytest.raises(WorkerConnectionError) as exc:
service._handle_request(test_request, "test_method")
assert "LiveKit client connection error" in str(exc.value)
assert "Connection failed" in str(exc.value)
@pytest.mark.parametrize(
"response_status,expected_result",
[
(livekit_api.EgressStatus.EGRESS_ABORTED, "ABORTED"),
(livekit_api.EgressStatus.EGRESS_COMPLETE, "FAILED_TO_STOP"),
(livekit_api.EgressStatus.EGRESS_ENDING, "STOPPED"),
(livekit_api.EgressStatus.EGRESS_FAILED, "FAILED_TO_STOP"),
],
)
def test_base_egress_stop_with_status(service, response_status, expected_result):
"""Test stop method with different response statuses"""
# Mock _handle_request
mock_response = Mock(status=response_status)
service._handle_request = Mock(return_value=mock_response)
# Execute stop
result = service.stop("test_worker_id")
# Verify request and response handling
service._handle_request.assert_called_once_with(
livekit_api.StopEgressRequest(egress_id="test_worker_id"), "stop_egress"
)
assert result == expected_result
def test_base_egress_stop_missing_status(service):
"""Test stop method when response is missing status"""
# Mock _handle_request with missing status
mock_response = Mock(status=None)
service._handle_request = Mock(return_value=mock_response)
# Verify error handling
with pytest.raises(WorkerResponseError) as exc:
service.stop("test_worker_id")
assert "missing the recording status" in str(exc.value)
def test_base_egress_start_not_implemented(service):
"""Test that start method raises NotImplementedError"""
with pytest.raises(NotImplementedError) as exc:
service.start("test_room", "test_recording")
assert "Subclass must implement this method" in str(exc.value)
@pytest.mark.parametrize("verify_ssl", [True, False])
def test_base_egress_ssl_verification_config(verify_ssl):
"""Test SSL verification configuration"""
config = WorkerServiceConfig(
output_folder="/test/output",
server_configurations={
"host": "test.livekit.io",
"api_key": "test_key",
"api_secret": "test_secret",
},
verify_ssl=verify_ssl,
bucket_args={
"endpoint": "https://s3.test.com",
"access_key": "test_key",
"secret": "test_secret",
"region": "test-region",
"bucket": "test-bucket",
"force_path_style": True,
},
)
service = BaseEgressService(config)
# Mock ClientSession to capture connector configuration
with patch("aiohttp.ClientSession") as mock_session:
mock_session.return_value.__aenter__ = AsyncMock()
mock_session.return_value.__aexit__ = AsyncMock()
# Trigger request to verify connector configuration
service._handle_request(Mock(), "test_method")
# Verify SSL configuration
connector = mock_session.call_args[1]["connector"]
assert isinstance(connector, aiohttp.TCPConnector)
assert connector._ssl == verify_ssl
def test_video_composite_egress_hrid(video_service):
"""Test HRID is correct"""
assert video_service.hrid == "video-recording-composite-livekit-egress"
def test_video_composite_egress_start_success(video_service):
"""Test successful start of video composite egress"""
# Setup mock response
egress_id = "test-egress-123"
video_service._handle_request.return_value = Mock(egress_id=egress_id)
# Test parameters
room_name = "test-room"
recording_id = "rec-123"
# Call start
result = video_service.start(room_name, recording_id)
# Verify result
assert result == egress_id
# Verify request construction
video_service._handle_request.assert_called_once()
request = video_service._handle_request.call_args[0][0]
method = video_service._handle_request.call_args[0][1]
# Verify request properties
assert isinstance(request, livekit_api.RoomCompositeEgressRequest)
assert request.room_name == room_name
assert len(request.file_outputs) == 1
assert not request.audio_only # Video service shouldn't set audio_only
# Verify file output configuration
file_output = request.file_outputs[0]
assert file_output.file_type == livekit_api.EncodedFileType.MP4
assert file_output.filepath == f"/test/output/{recording_id}.mp4"
assert file_output.s3.bucket == "test-bucket"
# Verify method name
assert method == "start_room_composite_egress"
def test_video_composite_egress_start_missing_egress_id(video_service):
"""Test handling of missing egress ID in response"""
# Setup mock response without egress_id
video_service._handle_request.return_value = Mock(egress_id=None)
with pytest.raises(WorkerResponseError) as exc_info:
video_service.start("test-room", "rec-123")
assert "Egress ID not found" in str(exc_info.value)
def test_audio_composite_egress_hrid(audio_service):
"""Test HRID is correct"""
assert audio_service.hrid == "audio-recording-composite-livekit-egress"
def test_audio_composite_egress_start_success(audio_service):
"""Test successful start of audio composite egress"""
# Setup mock response
egress_id = "test-egress-123"
audio_service._handle_request.return_value = Mock(egress_id=egress_id)
# Test parameters
room_name = "test-room"
recording_id = "rec-123"
# Call start
result = audio_service.start(room_name, recording_id)
# Verify result
assert result == egress_id
# Verify request construction
audio_service._handle_request.assert_called_once()
request = audio_service._handle_request.call_args[0][0]
method = audio_service._handle_request.call_args[0][1]
# Verify request properties
assert isinstance(request, livekit_api.RoomCompositeEgressRequest)
assert request.room_name == room_name
assert len(request.file_outputs) == 1
assert request.audio_only is True # Audio service should set audio_only
# Verify file output configuration
file_output = request.file_outputs[0]
assert file_output.file_type == livekit_api.EncodedFileType.OGG
assert file_output.filepath == f"/test/output/{recording_id}.ogg"
assert file_output.s3.bucket == "test-bucket"
# Verify method name
assert method == "start_room_composite_egress"
def test_audio_composite_egress_start_missing_egress_id(audio_service):
"""Test handling of missing egress ID in response"""
# Setup mock response without egress_id
audio_service._handle_request.return_value = Mock(egress_id=None)
with pytest.raises(WorkerResponseError) as exc_info:
audio_service.start("test-room", "rec-123")
assert "Egress ID not found" in str(exc_info.value)
@@ -38,7 +38,7 @@ def test_api_rooms_retrieve_anonymous_private_pk():
def test_api_rooms_retrieve_anonymous_private_pk_no_dashes():
"""It should be possible to get a room by its id stripped of its dashes."""
room = RoomFactory(is_public=False)
id_no_dashes = str(room.id).replace("-", "")
id_no_dashes = str(room.id)
client = APIClient()
response = client.get(f"/api/v1.0/rooms/{id_no_dashes:s}/")
@@ -178,7 +178,7 @@ def test_api_rooms_retrieve_anonymous_public(mock_token):
response = client.get(f"/api/v1.0/rooms/{room.id!s}/")
assert response.status_code == 200
expected_name = f"{room.id!s}".replace("-", "")
expected_name = f"{room.id!s}"
assert response.json() == {
"id": str(room.id),
"is_administrable": False,
@@ -220,7 +220,7 @@ def test_api_rooms_retrieve_authenticated_public(mock_token):
)
assert response.status_code == 200
expected_name = f"{room.id!s}".replace("-", "")
expected_name = f"{room.id!s}"
assert response.json() == {
"id": str(room.id),
"is_administrable": False,
@@ -301,6 +301,8 @@ def test_api_rooms_retrieve_members(mock_token, django_assert_num_queries):
"user": {
"id": str(user_access.user.id),
"email": user_access.user.email,
"full_name": user_access.user.full_name,
"short_name": user_access.user.short_name,
},
"resource": str(room.id),
"role": user_access.role,
@@ -310,6 +312,8 @@ def test_api_rooms_retrieve_members(mock_token, django_assert_num_queries):
"user": {
"id": str(other_user_access.user.id),
"email": other_user_access.user.email,
"full_name": other_user_access.user.full_name,
"short_name": other_user_access.user.short_name,
},
"resource": str(room.id),
"role": other_user_access.role,
@@ -318,7 +322,7 @@ def test_api_rooms_retrieve_members(mock_token, django_assert_num_queries):
key=lambda x: x["id"],
)
expected_name = str(room.id).replace("-", "")
expected_name = str(room.id)
assert content_dict == {
"id": str(room.id),
"is_administrable": False,
@@ -374,6 +378,8 @@ def test_api_rooms_retrieve_administrators(mock_token, django_assert_num_queries
"user": {
"id": str(other_user_access.user.id),
"email": other_user_access.user.email,
"full_name": other_user_access.user.full_name,
"short_name": other_user_access.user.short_name,
},
"resource": str(room.id),
"role": other_user_access.role,
@@ -383,6 +389,8 @@ def test_api_rooms_retrieve_administrators(mock_token, django_assert_num_queries
"user": {
"id": str(user_access.user.id),
"email": user_access.user.email,
"full_name": user_access.user.full_name,
"short_name": user_access.user.short_name,
},
"resource": str(room.id),
"role": user_access.role,
@@ -390,7 +398,7 @@ def test_api_rooms_retrieve_administrators(mock_token, django_assert_num_queries
],
key=lambda x: x["id"],
)
expected_name = str(room.id).replace("-", "")
expected_name = str(room.id)
assert content_dict == {
"id": str(room.id),
"is_administrable": True,
@@ -0,0 +1,200 @@
"""
Test rooms API endpoints in the Meet core app: start recording.
"""
# pylint: disable=W0621,W0613
from unittest import mock
import pytest
from rest_framework.test import APIClient
from ...factories import RoomFactory, UserFactory
from ...models import Recording
from ...recording.worker.exceptions import RecordingStartError
pytestmark = pytest.mark.django_db
@pytest.fixture
def mock_worker_service():
"""Mock worker service."""
return mock.Mock()
@pytest.fixture
def mock_worker_service_factory(mock_worker_service):
"""Mock worker service factory."""
with mock.patch(
"core.api.viewsets.get_worker_service",
return_value=mock_worker_service,
) as mock_worker_service_factory:
yield mock_worker_service_factory
@pytest.fixture
def mock_worker_manager(mock_worker_service):
"""Mock worker service mediator."""
with mock.patch("core.api.viewsets.WorkerServiceMediator") as mock_mediator_class:
mock_mediator = mock.Mock()
mock_mediator_class.return_value = mock_mediator
yield mock_mediator
def test_start_recording_anonymous():
"""Anonymous users should not be allowed to start room recordings."""
room = RoomFactory()
client = APIClient()
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{"mode": "screen_recording"},
)
assert response.status_code == 401
assert Recording.objects.count() == 0
def test_start_recording_non_owner_and_non_administrator():
"""Non-owner and Non-Administrator users should not be allowed to start room recordings."""
room = RoomFactory()
user = UserFactory()
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{"mode": "screen_recording"},
)
assert response.status_code == 403
assert Recording.objects.count() == 0
def test_start_recording_recording_disabled(settings):
"""Should fail if recording is disabled for the room."""
settings.RECORDING_ENABLE = False
room = RoomFactory()
user = UserFactory()
# Make user the room owner
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{"mode": "screen_recording"},
)
assert response.status_code == 403
assert response.json() == {"detail": "Access denied, recording is disabled."}
assert Recording.objects.count() == 0
def test_start_recording_missing_mode(settings):
"""Should fail if recording mode is not provided."""
settings.RECORDING_ENABLE = True
room = RoomFactory()
user = UserFactory()
# Make user the room owner
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(f"/api/v1.0/rooms/{room.id}/start-recording/", {})
assert response.status_code == 400
assert response.json() == {"detail": "Invalid request."}
assert Recording.objects.count() == 0
def test_start_recording_worker_error(
mock_worker_service_factory, mock_worker_manager, settings
):
"""Should handle worker service errors appropriately."""
settings.RECORDING_ENABLE = True
room = RoomFactory()
user = UserFactory()
# Make user the room owner
room.accesses.create(user=user, role="owner")
# Configure mock mediator to raise error
mock_start = mock.Mock()
mock_start.side_effect = RecordingStartError("Failed to connect to worker")
mock_worker_manager.start = mock_start
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{"mode": "screen_recording"},
)
mock_worker_service_factory.assert_called_once_with(mode="screen_recording")
assert response.status_code == 500
assert response.json() == {
"error": f"Recording failed to start for room {room.slug}"
}
# Recording object should be created even if worker fails
assert Recording.objects.count() == 1
recording = Recording.objects.first()
assert recording.room == room
assert recording.mode == "screen_recording"
# Verify recording access details
assert recording.accesses.count() == 1
access = recording.accesses.first()
assert access.user == user
assert access.role == "owner"
def test_start_recording_success(
mock_worker_service_factory, mock_worker_manager, settings
):
"""Should successfully start recording when everything is configured correctly."""
settings.RECORDING_ENABLE = True
room = RoomFactory()
user = UserFactory()
# Make user the room owner
room.accesses.create(user=user, role="owner")
mock_start = mock.Mock()
mock_worker_manager.start = mock_start
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{"mode": "screen_recording"},
)
mock_worker_service_factory.assert_called_once_with(mode="screen_recording")
assert response.status_code == 201
assert response.json() == {
"message": f"Recording successfully started for room {room.slug}"
}
# Verify the mediator was called with the recording
recording = Recording.objects.first()
mock_start.assert_called_once_with(recording)
assert recording.room == room
assert recording.mode == "screen_recording"
# Verify recording access details
assert recording.accesses.count() == 1
access = recording.accesses.first()
assert access.user == user
assert access.role == "owner"
@@ -0,0 +1,182 @@
"""
Test rooms API endpoints in the Meet core app: stop recording.
"""
# pylint: disable=W0621,W0613
from unittest import mock
import pytest
from rest_framework.test import APIClient
from ...factories import RecordingFactory, RoomFactory, UserFactory
from ...models import Recording, RecordingStatusChoices
from ...recording.worker.exceptions import RecordingStopError
pytestmark = pytest.mark.django_db
@pytest.fixture
def mock_worker_service():
"""Mock worker service."""
return mock.Mock()
@pytest.fixture
def mock_worker_service_factory(mock_worker_service):
"""Mock worker service factory."""
with mock.patch(
"core.api.viewsets.get_worker_service",
return_value=mock_worker_service,
) as mock_worker_service_factory:
yield mock_worker_service_factory
@pytest.fixture
def mock_worker_manager(mock_worker_service):
"""Mock worker service mediator."""
with mock.patch("core.api.viewsets.WorkerServiceMediator") as mock_mediator_class:
mock_mediator = mock.Mock()
mock_mediator_class.return_value = mock_mediator
yield mock_mediator
def test_stop_recording_anonymous():
"""Anonymous users should not be allowed to stop room recordings."""
room = RoomFactory()
RecordingFactory(room=room, status=RecordingStatusChoices.ACTIVE)
client = APIClient()
response = client.post(f"/api/v1.0/rooms/{room.id}/stop-recording/")
assert response.status_code == 401
# Verify recording status hasn't changed
assert Recording.objects.filter(status=RecordingStatusChoices.ACTIVE).count() == 1
def test_stop_recording_non_owner_and_non_administrator():
"""Non-owner and Non-Administrator users should not be allowed to stop room recordings."""
room = RoomFactory()
user = UserFactory()
RecordingFactory(room=room, status=RecordingStatusChoices.ACTIVE)
client = APIClient()
client.force_login(user)
response = client.post(f"/api/v1.0/rooms/{room.id}/stop-recording/")
assert response.status_code == 403
# Verify recording status hasn't changed
assert Recording.objects.filter(status=RecordingStatusChoices.ACTIVE).count() == 1
def test_stop_recording_recording_disabled(settings):
"""Should fail if recording is disabled for the room."""
settings.RECORDING_ENABLE = False
room = RoomFactory()
user = UserFactory()
# Make user the room owner
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(f"/api/v1.0/rooms/{room.id}/stop-recording/")
assert response.status_code == 403
assert response.json() == {"detail": "Access denied, recording is disabled."}
# Verify no recording exists
assert Recording.objects.count() == 0
def test_stop_recording_no_active_recording(settings):
"""Should fail when there is no active recording for the room."""
settings.RECORDING_ENABLE = True
room = RoomFactory()
user = UserFactory()
# Make user the room owner
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(f"/api/v1.0/rooms/{room.id}/stop-recording/")
assert response.status_code == 404
assert response.json() == {"detail": "No active recording found for this room."}
def test_stop_recording_worker_error(
mock_worker_service_factory, mock_worker_manager, settings
):
"""Should handle worker service errors appropriately."""
settings.RECORDING_ENABLE = True
room = RoomFactory()
user = UserFactory()
recording = RecordingFactory(
room=room,
status=RecordingStatusChoices.ACTIVE,
mode="screen_recording",
)
# Make user the room owner
room.accesses.create(user=user, role="owner")
# Configure mock mediator to raise error
mock_stop = mock.Mock()
mock_stop.side_effect = RecordingStopError("Failed to connect to worker")
mock_worker_manager.stop = mock_stop
client = APIClient()
client.force_login(user)
response = client.post(f"/api/v1.0/rooms/{room.id}/stop-recording/")
mock_worker_service_factory.assert_called_once_with(mode="screen_recording")
mock_stop.assert_called_once_with(recording)
assert response.status_code == 500
assert response.json() == {
"error": f"Recording failed to stop for room {room.slug}"
}
# Verify recording status hasn't changed
assert Recording.objects.filter(status=RecordingStatusChoices.ACTIVE).count() == 1
def test_stop_recording_success(
mock_worker_service_factory, mock_worker_manager, settings
):
"""Should successfully stop recording when everything is configured correctly."""
settings.RECORDING_ENABLE = True
room = RoomFactory()
user = UserFactory()
recording = RecordingFactory(
room=room,
status=RecordingStatusChoices.ACTIVE,
mode="screen_recording",
)
# Make user the room owner
room.accesses.create(user=user, role="owner")
mock_stop = mock.Mock()
mock_worker_manager.stop = mock_stop
client = APIClient()
client.force_login(user)
response = client.post(f"/api/v1.0/rooms/{room.id}/stop-recording/")
mock_worker_service_factory.assert_called_once_with(mode="screen_recording")
mock_stop.assert_called_once_with(recording)
assert response.status_code == 200
assert response.json() == {"message": f"Recording stopped for room {room.slug}."}
# Verify the recording still exists
assert Recording.objects.count() == 1
@@ -0,0 +1,187 @@
"""
Test marketing services.
"""
# pylint: disable=W0621,W0613
from unittest import mock
from django.conf import settings
from django.core.exceptions import ImproperlyConfigured
import brevo_python
import pytest
from core.services.marketing_service import (
BrevoMarketingService,
ContactCreationError,
ContactData,
get_marketing_service,
)
def test_init_missing_api_key(settings):
"""Test initialization with missing API key."""
settings.BREVO_API_KEY = None
with pytest.raises(ImproperlyConfigured, match="Brevo API key is required"):
BrevoMarketingService()
def test_create_contact_missing_list_ids(settings):
"""Test contact creation with missing list IDs."""
settings.BREVO_API_KEY = "test-api-key"
settings.BREVO_API_CONTACT_LIST_IDS = None
settings.BREVO_API_CONTACT_ATTRIBUTES = {"source": "test"}
valid_contact_data = ContactData(
email="test@example.com",
attributes={"first_name": "Test"},
list_ids=[1, 2],
update_enabled=True,
)
brevo_service = BrevoMarketingService()
with pytest.raises(
ImproperlyConfigured, match="Default Brevo List IDs must be configured"
):
brevo_service.create_contact(valid_contact_data)
@mock.patch("brevo_python.ContactsApi")
def test_create_contact_success(mock_contact_api):
"""Test successful contact creation."""
mock_api = mock_contact_api.return_value
settings.BREVO_API_KEY = "test-api-key"
settings.BREVO_API_CONTACT_LIST_IDS = [1, 2, 3, 4]
settings.BREVO_API_CONTACT_ATTRIBUTES = {"source": "test"}
valid_contact_data = ContactData(
email="test@example.com",
attributes={"first_name": "Test"},
list_ids=[1, 2],
update_enabled=True,
)
brevo_service = BrevoMarketingService()
mock_api.create_contact.return_value = {"id": "test-id"}
response = brevo_service.create_contact(valid_contact_data)
assert response == {"id": "test-id"}
mock_api.create_contact.assert_called_once()
contact_arg = mock_api.create_contact.call_args[0][0]
assert contact_arg.email == "test@example.com"
assert contact_arg.attributes == {
**settings.BREVO_API_CONTACT_ATTRIBUTES,
**valid_contact_data.attributes,
}
assert set(contact_arg.list_ids) == {1, 2, 3, 4}
assert contact_arg.update_enabled is True
@mock.patch("brevo_python.ContactsApi")
def test_create_contact_with_timeout(mock_contact_api):
"""Test contact creation with timeout."""
mock_api = mock_contact_api.return_value
settings.BREVO_API_KEY = "test-api-key"
settings.BREVO_API_CONTACT_LIST_IDS = [1, 2, 3, 4]
settings.BREVO_API_CONTACT_ATTRIBUTES = {"source": "test"}
valid_contact_data = ContactData(
email="test@example.com",
attributes={"first_name": "Test"},
list_ids=[1, 2],
update_enabled=True,
)
brevo_service = BrevoMarketingService()
brevo_service.create_contact(valid_contact_data, timeout=30)
mock_api.create_contact.assert_called_once()
assert mock_api.create_contact.call_args[1]["_request_timeout"] == 30
@mock.patch("brevo_python.ContactsApi")
def test_create_contact_api_error(mock_contact_api):
"""Test contact creation API error handling."""
mock_api = mock_contact_api.return_value
settings.BREVO_API_KEY = "test-api-key"
settings.BREVO_API_CONTACT_LIST_IDS = [1, 2, 3, 4]
settings.BREVO_API_CONTACT_ATTRIBUTES = {"source": "test"}
valid_contact_data = ContactData(
email="test@example.com",
attributes={"first_name": "Test"},
list_ids=[1, 2],
update_enabled=True,
)
brevo_service = BrevoMarketingService()
mock_api.create_contact.side_effect = brevo_python.rest.ApiException()
with pytest.raises(ContactCreationError, match="Failed to create contact in Brevo"):
brevo_service.create_contact(valid_contact_data)
@pytest.fixture
def clear_marketing_cache():
"""Clear marketing service cache between tests."""
get_marketing_service.cache_clear()
yield
get_marketing_service.cache_clear()
def test_get_marketing_service_caching(clear_marketing_cache):
"""Test marketing service caching behavior."""
settings.BREVO_API_KEY = "test-api-key"
settings.MARKETING_SERVICE_CLASS = (
"core.services.marketing_service.BrevoMarketingService"
)
service1 = get_marketing_service()
service2 = get_marketing_service()
assert service1 is service2
assert isinstance(service1, BrevoMarketingService)
def test_get_marketing_service_invalid_class(clear_marketing_cache):
"""Test handling of invalid service class."""
settings.MARKETING_SERVICE_CLASS = "invalid.service.path"
with pytest.raises(ImportError):
get_marketing_service()
@mock.patch("core.services.marketing_service.import_string")
def test_service_instantiation_called_once(mock_import_string, clear_marketing_cache):
"""Test service class is instantiated only once."""
settings.BREVO_API_KEY = "test-api-key"
settings.MARKETING_SERVICE_CLASS = (
"core.services.marketing_service.BrevoMarketingService"
)
get_marketing_service.cache_clear()
mock_service_cls = mock.Mock()
mock_service_instance = mock.Mock()
mock_service_cls.return_value = mock_service_instance
mock_import_string.return_value = mock_service_cls
service1 = get_marketing_service()
service2 = get_marketing_service()
mock_import_string.assert_called_once_with(settings.MARKETING_SERVICE_CLASS)
mock_service_cls.assert_called_once()
assert service1 is service2
assert service1 is mock_service_instance
-132
View File
@@ -1,132 +0,0 @@
"""
Test for the Analytics class.
"""
# pylint: disable=W0212
from unittest.mock import patch
from django.contrib.auth.models import AnonymousUser
from django.test.utils import override_settings
import pytest
from core.analytics import Analytics
from core.factories import UserFactory
pytestmark = pytest.mark.django_db
@pytest.fixture(name="mock_june_analytics")
def _mock_june_analytics():
with patch("core.analytics.jAnalytics") as mock:
yield mock
@override_settings(ANALYTICS_KEY="test_key")
def test_analytics_init_enabled(mock_june_analytics):
"""Should enable analytics and set the write key correctly when ANALYTICS_KEY is set."""
analytics = Analytics()
assert analytics._enabled is True
assert mock_june_analytics.write_key == "test_key"
@override_settings(ANALYTICS_KEY=None)
def test_analytics_init_disabled():
"""Should disable analytics when ANALYTICS_KEY is not set."""
analytics = Analytics()
assert analytics._enabled is False
@override_settings(ANALYTICS_KEY="test_key")
def test_analytics_identify_user(mock_june_analytics):
"""Should identify a user with the correct traits when analytics is enabled."""
user = UserFactory(sub="12345", email="user@example.com")
analytics = Analytics()
analytics.identify(user)
mock_june_analytics.identify.assert_called_once_with(
user_id="12345", traits={"email": "***@example.com"}
)
@override_settings(ANALYTICS_KEY="test_key")
def test_analytics_identify_user_with_traits(mock_june_analytics):
"""Should identify a user with additional traits when analytics is enabled."""
user = UserFactory(sub="12345", email="user@example.com")
analytics = Analytics()
analytics.identify(user, traits={"email": "user@example.com", "foo": "foo"})
mock_june_analytics.identify.assert_called_once_with(
user_id="12345", traits={"email": "***@example.com", "foo": "foo"}
)
@override_settings(ANALYTICS_KEY=None)
def test_analytics_identify_not_enabled(mock_june_analytics):
"""Should not call identify when analytics is not enabled."""
user = UserFactory(sub="12345", email="user@example.com")
analytics = Analytics()
analytics.identify(user)
mock_june_analytics.identify.assert_not_called()
@override_settings(ANALYTICS_KEY="test_key")
def test_analytics_identify_no_user(mock_june_analytics):
"""Should not call identify when the user is None."""
analytics = Analytics()
analytics.identify(None)
mock_june_analytics.identify.assert_not_called()
@override_settings(ANALYTICS_KEY="test_key")
def test_analytics_identify_anonymous_user(mock_june_analytics):
"""Should not call identify when the user is anonymous."""
user = AnonymousUser()
analytics = Analytics()
analytics.identify(user)
mock_june_analytics.identify.assert_not_called()
@override_settings(ANALYTICS_KEY="test_key")
def test_analytics_track_event(mock_june_analytics):
"""Should track an event with the correct user and event details when analytics is enabled."""
user = UserFactory(sub="12345")
analytics = Analytics()
analytics.track(user, event="test_event", foo="foo")
mock_june_analytics.track.assert_called_once_with(
user_id="12345", event="test_event", foo="foo"
)
@override_settings(ANALYTICS_KEY=None)
def test_analytics_track_event_not_enabled(mock_june_analytics):
"""Should not call track when analytics is not enabled."""
user = UserFactory(sub="12345")
analytics = Analytics()
analytics.track(user, event="test_event", foo="foo")
mock_june_analytics.track.assert_not_called()
@override_settings(ANALYTICS_KEY="test_key")
@patch("uuid.uuid4", return_value="test_uuid4")
def test_analytics_track_event_no_user(mock_uuid4, mock_june_analytics):
"""Should track an event with a random anonymous user ID when the user is None."""
analytics = Analytics()
analytics.track(None, event="test_event", foo="foo")
mock_june_analytics.track.assert_called_once_with(
anonymous_id="test_uuid4", event="test_event", foo="foo"
)
mock_uuid4.assert_called_once()
@override_settings(ANALYTICS_KEY="test_key")
@patch("uuid.uuid4", return_value="test_uuid4")
def test_analytics_track_event_anonymous_user(mock_uuid4, mock_june_analytics):
"""Should track an event with a random anonymous user ID when the user is anonymous."""
user = AnonymousUser()
analytics = Analytics()
analytics.track(user, event="test_event", foo="foo")
mock_june_analytics.track.assert_called_once_with(
anonymous_id="test_uuid4", event="test_event", foo="foo"
)
mock_uuid4.assert_called_once()
+3 -1
View File
@@ -66,7 +66,7 @@ def test_api_users_list_query_email():
assert response.status_code == 200
user_ids = [user["id"] for user in response.json()["results"]]
assert user_ids == [str(nicole.id), str(frank.id)]
assert user_ids == [str(frank.id), str(nicole.id)]
def test_api_users_retrieve_me_anonymous():
@@ -96,6 +96,8 @@ def test_api_users_retrieve_me_authenticated():
assert response.json() == {
"id": str(user.id),
"email": user.email,
"full_name": user.full_name,
"short_name": user.short_name,
}
@@ -0,0 +1,218 @@
"""
Unit tests for the Recording model
"""
from django.core.exceptions import ValidationError
import pytest
from core.factories import (
RecordingFactory,
RoomFactory,
UserFactory,
UserRecordingAccessFactory,
)
from core.models import Recording, RecordingStatusChoices
pytestmark = pytest.mark.django_db
def test_models_recording_str():
"""The str representation should be the recording ID."""
recording = RecordingFactory()
assert str(recording) == f"Recording {recording.id} (initiated)"
def test_models_recording_ordering():
"""Recordings should be returned ordered by created_at in descending order."""
RecordingFactory.create_batch(3)
recordings = Recording.objects.all()
assert recordings[0].created_at >= recordings[1].created_at
assert recordings[1].created_at >= recordings[2].created_at
def test_models_recording_room_relationship():
"""It should maintain proper relationship with room."""
room = RoomFactory()
recording = RecordingFactory(room=room)
assert recording.room == room
assert recording in room.recordings.all()
def test_models_recording_default_status():
"""A new recording should have INITIATED status by default."""
recording = RecordingFactory()
assert recording.status == RecordingStatusChoices.INITIATED
def test_models_recording_unique_initiated_or_active_per_room():
"""Only one initiated or active recording should be allowed per room."""
room = RoomFactory()
RecordingFactory(room=room, status=RecordingStatusChoices.ACTIVE)
with pytest.raises(ValidationError):
RecordingFactory(room=room, status=RecordingStatusChoices.ACTIVE)
with pytest.raises(ValidationError):
RecordingFactory(room=room, status=RecordingStatusChoices.INITIATED)
def test_models_recording_multiple_finished_allowed():
"""Multiple finished recordings should be allowed in the same room."""
room = RoomFactory()
RecordingFactory(room=room, status=RecordingStatusChoices.SAVED)
RecordingFactory(room=room, status=RecordingStatusChoices.SAVED)
assert room.recordings.count() == 2
# Test get_abilities method
@pytest.mark.parametrize(
"role",
["owner", "administrator"],
)
def test_models_recording_get_abilities_privileges_active(role):
"""Test abilities for active recording and privileged user."""
user = UserFactory()
access = UserRecordingAccessFactory(role=role, user=user)
access.recording.status = RecordingStatusChoices.ACTIVE
abilities = access.recording.get_abilities(user)
assert abilities == {
"destroy": False, # Not final status
"partial_update": False,
"retrieve": True, # Privileged users can always retrieve
"stop": True, # Not final status, so can stop
"update": False,
}
def test_models_recording_get_abilities_member_active():
"""Test abilities for a user who is unprivileged."""
user = UserFactory()
access = UserRecordingAccessFactory(role="member", user=user)
access.recording.status = RecordingStatusChoices.ACTIVE
abilities = access.recording.get_abilities(user)
assert abilities == {
"destroy": False,
"partial_update": False,
"retrieve": False,
"stop": False,
"update": False,
}
@pytest.mark.parametrize(
"role",
["owner", "administrator"],
)
def test_models_recording_get_abilities_privileges_final(role):
"""Test abilities for active recording and privileged user."""
user = UserFactory()
access = UserRecordingAccessFactory(role=role, user=user)
access.recording.status = RecordingStatusChoices.SAVED
abilities = access.recording.get_abilities(user)
assert abilities == {
"destroy": True,
"partial_update": False,
"retrieve": True, # Privileged users can always retrieve
"stop": False, # In final status, so can not stop
"update": False,
}
def test_models_recording_get_abilities_member_final():
"""Test abilities for a user who is unprivileged."""
user = UserFactory()
access = UserRecordingAccessFactory(role="member", user=user)
access.recording.status = RecordingStatusChoices.SAVED
abilities = access.recording.get_abilities(user)
assert abilities == {
"destroy": False,
"partial_update": False,
"retrieve": False,
"stop": False,
"update": False,
}
# Test is_savable method
def test_models_recording_is_savable_normal():
"""Test is_savable for normal recording status."""
recording = RecordingFactory(status=RecordingStatusChoices.ACTIVE)
assert recording.is_savable() is True
@pytest.mark.parametrize(
"status",
[
RecordingStatusChoices.FAILED_TO_STOP,
RecordingStatusChoices.FAILED_TO_START,
RecordingStatusChoices.ABORTED,
],
)
def test_models_recording_is_savable_error(status):
"""Test is_savable for error status."""
recording = RecordingFactory(status=status)
assert recording.is_savable() is False
def test_models_recording_is_savable_already_saved():
"""Test is_savable for already saved recording."""
recording = RecordingFactory(status=RecordingStatusChoices.SAVED)
assert recording.is_savable() is False
def test_models_recording_is_savable_only_initiated():
"""Test is_savable for only initiated recording."""
recording = RecordingFactory(status=RecordingStatusChoices.INITIATED)
assert recording.is_savable() is False
# Test few corner cases
def test_models_recording_worker_id_optional():
"""Worker ID should be optional."""
recording = RecordingFactory(worker_id=None)
assert recording.worker_id is None
recording = RecordingFactory(worker_id="worker-123")
assert recording.worker_id == "worker-123"
def test_models_recording_invalid_status():
"""Test that setting an invalid status raises an error."""
recording = RecordingFactory()
recording.status = "INVALID_STATUS"
with pytest.raises(ValidationError):
recording.save()
def test_models_recording_room_deletion():
"""Test that deleting a room cascades to its recordings."""
room = RoomFactory()
recording = RecordingFactory(room=room)
room.delete()
assert not Recording.objects.filter(id=recording.id).exists()
def test_models_recording_worker_id_very_long():
"""Test worker_id with maximum length."""
long_id = "w" * 255
recording = RecordingFactory(worker_id=long_id)
assert recording.worker_id == long_id
too_long_id = "w" * 256
with pytest.raises(ValidationError):
RecordingFactory(worker_id=too_long_id)
@@ -0,0 +1,312 @@
"""
Unit tests for the RecordingAccess model
"""
from django.contrib.auth.models import AnonymousUser
from django.core.exceptions import ValidationError
import pytest
from core import factories
pytestmark = pytest.mark.django_db
def test_models_recording_accesses_str():
"""
The str representation should include user email, recording ID and role.
"""
user = factories.UserFactory(email="david.bowman@example.com")
access = factories.UserRecordingAccessFactory(
role="member",
user=user,
)
assert (
str(access)
== f"david.bowman@example.com is member in Recording {access.recording.id} (initiated)"
)
def test_models_recording_accesses_unique_user():
"""Recording accesses should be unique for a given couple of user and recording."""
access = factories.UserRecordingAccessFactory()
with pytest.raises(
ValidationError,
match="This user is already in this recording.",
):
factories.UserRecordingAccessFactory(
user=access.user, recording=access.recording
)
def test_models_recording_accesses_several_empty_teams():
"""A recording can have several recording accesses with an empty team."""
access = factories.UserRecordingAccessFactory()
factories.UserRecordingAccessFactory(recording=access.recording)
def test_models_recording_accesses_unique_team():
"""Recording accesses should be unique for a given couple of team and recording."""
access = factories.TeamRecordingAccessFactory()
with pytest.raises(
ValidationError,
match="This team is already in this recording.",
):
factories.TeamRecordingAccessFactory(
team=access.team, recording=access.recording
)
def test_models_recording_accesses_several_null_users():
"""A recording can have several recording accesses with a null user."""
access = factories.TeamRecordingAccessFactory()
factories.TeamRecordingAccessFactory(recording=access.recording)
def test_models_recording_accesses_user_and_team_set():
"""User and team can't both be set on a recording access."""
with pytest.raises(
ValidationError,
match="Either user or team must be set, not both.",
):
factories.UserRecordingAccessFactory(team="my-team")
def test_models_recording_accesses_user_and_team_empty():
"""User and team can't both be empty on a recording access."""
with pytest.raises(
ValidationError,
match="Either user or team must be set, not both.",
):
factories.UserRecordingAccessFactory(user=None)
# get_abilities
def test_models_recording_access_get_abilities_anonymous():
"""Check abilities returned for an anonymous user."""
access = factories.UserRecordingAccessFactory()
abilities = access.get_abilities(AnonymousUser())
assert abilities == {
"destroy": False,
"retrieve": False,
"update": False,
"partial_update": False,
"set_role_to": [],
}
def test_models_recording_access_get_abilities_authenticated():
"""Check abilities returned for an authenticated user."""
access = factories.UserRecordingAccessFactory()
user = factories.UserFactory()
abilities = access.get_abilities(user)
assert abilities == {
"destroy": False,
"retrieve": False,
"update": False,
"partial_update": False,
"set_role_to": [],
}
# - for owner
def test_models_recording_access_get_abilities_for_owner_of_self_allowed():
"""
Check abilities of self access for the owner of a recording when
there is more than one owner left.
"""
access = factories.UserRecordingAccessFactory(role="owner")
factories.UserRecordingAccessFactory(recording=access.recording, role="owner")
abilities = access.get_abilities(access.user)
assert abilities == {
"destroy": True,
"retrieve": True,
"update": True,
"partial_update": True,
"set_role_to": ["administrator", "member"],
}
def test_models_recording_access_get_abilities_for_owner_of_self_last():
"""
Check abilities of self access for the owner of a recording when there is only one owner left.
"""
access = factories.UserRecordingAccessFactory(role="owner")
abilities = access.get_abilities(access.user)
assert abilities == {
"destroy": False,
"retrieve": True,
"update": False,
"partial_update": False,
"set_role_to": [],
}
def test_models_recording_access_get_abilities_for_owner_of_owner():
"""Check abilities of owner access for the owner of a recording."""
access = factories.UserRecordingAccessFactory(role="owner")
factories.UserRecordingAccessFactory(recording=access.recording) # another one
user = factories.UserRecordingAccessFactory(
recording=access.recording, role="owner"
).user
abilities = access.get_abilities(user)
assert abilities == {
"destroy": True,
"retrieve": True,
"update": True,
"partial_update": True,
"set_role_to": ["administrator", "member"],
}
def test_models_recording_access_get_abilities_for_owner_of_administrator():
"""Check abilities of administrator access for the owner of a recording."""
access = factories.UserRecordingAccessFactory(role="administrator")
factories.UserRecordingAccessFactory(recording=access.recording) # another one
user = factories.UserRecordingAccessFactory(
recording=access.recording, role="owner"
).user
abilities = access.get_abilities(user)
assert abilities == {
"destroy": True,
"retrieve": True,
"update": True,
"partial_update": True,
"set_role_to": ["member", "owner"],
}
def test_models_recording_access_get_abilities_for_owner_of_member():
"""Check abilities of member access for the owner of a recording."""
access = factories.UserRecordingAccessFactory(role="member")
factories.UserRecordingAccessFactory(recording=access.recording) # another one
user = factories.UserRecordingAccessFactory(
recording=access.recording, role="owner"
).user
abilities = access.get_abilities(user)
assert abilities == {
"destroy": True,
"retrieve": True,
"update": True,
"partial_update": True,
"set_role_to": ["administrator", "owner"],
}
# - for administrator
def test_models_recording_access_get_abilities_for_administrator_of_owner():
"""Check abilities of owner access for the administrator of a recording."""
access = factories.UserRecordingAccessFactory(role="owner")
factories.UserRecordingAccessFactory(recording=access.recording) # another one
user = factories.UserRecordingAccessFactory(
recording=access.recording, role="administrator"
).user
abilities = access.get_abilities(user)
assert abilities == {
"destroy": False,
"retrieve": True,
"update": False,
"partial_update": False,
"set_role_to": [],
}
def test_models_recording_access_get_abilities_for_administrator_of_administrator():
"""Check abilities of administrator access for the administrator of a recording."""
access = factories.UserRecordingAccessFactory(role="administrator")
factories.UserRecordingAccessFactory(recording=access.recording) # another one
user = factories.UserRecordingAccessFactory(
recording=access.recording, role="administrator"
).user
abilities = access.get_abilities(user)
assert abilities == {
"destroy": True,
"retrieve": True,
"update": True,
"partial_update": True,
"set_role_to": ["member"],
}
def test_models_recording_access_get_abilities_for_administrator_of_member():
"""Check abilities of member access for the administrator of a recording."""
access = factories.UserRecordingAccessFactory(role="member")
factories.UserRecordingAccessFactory(recording=access.recording) # another one
user = factories.UserRecordingAccessFactory(
recording=access.recording, role="administrator"
).user
abilities = access.get_abilities(user)
assert abilities == {
"destroy": True,
"retrieve": True,
"update": True,
"partial_update": True,
"set_role_to": ["administrator"],
}
# - for member
def test_models_recording_access_get_abilities_for_member_of_owner():
"""Check abilities of owner access for the member of a recording."""
access = factories.UserRecordingAccessFactory(role="owner")
factories.UserRecordingAccessFactory(recording=access.recording) # another one
user = factories.UserRecordingAccessFactory(
recording=access.recording, role="member"
).user
abilities = access.get_abilities(user)
assert abilities == {
"destroy": False,
"retrieve": True,
"update": False,
"partial_update": False,
"set_role_to": [],
}
def test_models_recording_access_get_abilities_for_member_of_administrator():
"""Check abilities of administrator access for the member of a recording."""
access = factories.UserRecordingAccessFactory(role="administrator")
factories.UserRecordingAccessFactory(recording=access.recording) # another one
user = factories.UserRecordingAccessFactory(
recording=access.recording, role="member"
).user
abilities = access.get_abilities(user)
assert abilities == {
"destroy": False,
"retrieve": True,
"update": False,
"partial_update": False,
"set_role_to": [],
}
def test_models_recording_access_get_abilities_for_member_of_member_user(
django_assert_num_queries,
):
"""Check abilities of member access for the member of a recording."""
access = factories.UserRecordingAccessFactory(role="member")
factories.UserRecordingAccessFactory(recording=access.recording) # another one
user = factories.UserRecordingAccessFactory(
recording=access.recording, role="member"
).user
with django_assert_num_queries(1):
abilities = access.get_abilities(user)
assert abilities == {
"destroy": False,
"retrieve": True,
"update": False,
"partial_update": False,
"set_role_to": [],
}
@@ -44,12 +44,3 @@ def test_models_users_send_mail_main_missing():
user.email_user("my subject", "my message")
assert str(excinfo.value) == "User has no email address."
def test_models_users_email_anonymized():
"""The user's email should be anonymized if it exists."""
user = factories.UserFactory(email="john.doe@world.com")
assert user.email_anonymized == "***@world.com"
user = factories.UserFactory(email=None)
assert user.email_anonymized == ""
+2 -2
View File
@@ -5,13 +5,14 @@ from django.urls import include, path
from rest_framework.routers import DefaultRouter
from core.api import get_frontend_configuration, viewsets, demo
from core.api import get_frontend_configuration, viewsets
from core.authentication.urls import urlpatterns as oidc_urls
# - Main endpoints
router = DefaultRouter()
router.register("users", viewsets.UserViewSet, basename="users")
router.register("rooms", viewsets.RoomViewSet, basename="rooms")
router.register("recordings", viewsets.RecordingViewSet, basename="recordings")
router.register(
"resource-accesses", viewsets.ResourceAccessViewSet, basename="resource_accesses"
)
@@ -24,7 +25,6 @@ urlpatterns = [
*router.urls,
*oidc_urls,
path("config/", get_frontend_configuration, name="config"),
path("minio-webhook/", demo.minio_webhook, name="demo"),
]
),
),
+41 -1
View File
@@ -14,6 +14,47 @@ from django.conf import settings
from livekit.api import AccessToken, VideoGrants
import secrets
import string
from django.core.cache import cache
from cryptography.fernet import Fernet
import base64
def generate_random_passphrase(length=26):
"""Generate a random passphrase using letters and digits"""
alphabet = string.ascii_letters + string.digits
return ''.join(secrets.choice(alphabet) for _ in range(length))
def build_room_passphrase_key(room_id: str) -> str:
"""Build cache key for room passphrase."""
return f"room_passphrase:{room_id}"
def get_cached_passphrase(room_id: str) -> str:
"""Get or generate encrypted passphrase for a room.
Retrieves existing passphrase from cache or generates,
encrypts and caches a new one if not found.
"""
cypher = Fernet(settings.PASSPHRASE_ENCRYPTION_KEY.encode())
cache_key = build_room_passphrase_key(room_id)
encrypted_passphrase = cache.get(cache_key)
if encrypted_passphrase is None:
passphrase = generate_random_passphrase()
encrypted_passphrase = cypher.encrypt(passphrase.encode()).decode()
cache.set(cache_key, encrypted_passphrase, timeout=86400) # 24 hours
return passphrase
return cypher.decrypt(encrypted_passphrase.encode()).decode()
def clear_room_passphrase(room_id: str) -> None:
"""Remove room passphrase from cache."""
cache.delete(build_room_passphrase_key(room_id))
def generate_color(identity: str) -> str:
"""Generates a consistent HSL color based on a given identity string.
@@ -53,7 +94,6 @@ def generate_token(room: str, user, username: Optional[str] = None) -> str:
room=room,
room_join=True,
room_admin=True,
room_record=True,
can_update_own_metadata=True,
can_publish_sources=[
"camera",
Binary file not shown.
Binary file not shown.
+96 -58
View File
@@ -119,6 +119,25 @@ class Base(Configuration):
},
}
# Media
AWS_S3_ENDPOINT_URL = values.Value(
environ_name="AWS_S3_ENDPOINT_URL", environ_prefix=None
)
AWS_S3_ACCESS_KEY_ID = values.Value(
environ_name="AWS_S3_ACCESS_KEY_ID", environ_prefix=None
)
AWS_S3_SECRET_ACCESS_KEY = values.Value(
environ_name="AWS_S3_SECRET_ACCESS_KEY", environ_prefix=None
)
AWS_S3_REGION_NAME = values.Value(
environ_name="AWS_S3_REGION_NAME", environ_prefix=None
)
AWS_STORAGE_BUCKET_NAME = values.Value(
"meet-media-storage",
environ_name="AWS_STORAGE_BUCKET_NAME",
environ_prefix=None,
)
# Internationalization
# https://docs.djangoproject.com/en/3.1/topics/i18n/
@@ -305,8 +324,14 @@ class Base(Configuration):
OIDC_AUTHENTICATE_CLASS = "core.authentication.views.OIDCAuthenticationRequestView"
OIDC_CALLBACK_CLASS = "core.authentication.views.OIDCAuthenticationCallbackView"
OIDC_CREATE_USER = values.BooleanValue(
default=True,
environ_name="OIDC_CREATE_USER",
default=True, environ_name="OIDC_CREATE_USER", environ_prefix=None
)
OIDC_VERIFY_SSL = values.BooleanValue(
default=True, environ_name="OIDC_VERIFY_SSL", environ_prefix=None
)
OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = values.BooleanValue(
default=False,
environ_name="OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION",
)
OIDC_RP_SIGN_ALGO = values.Value(
"RS256", environ_name="OIDC_RP_SIGN_ALGO", environ_prefix=None
@@ -367,6 +392,16 @@ class Base(Configuration):
OIDC_REDIRECT_FIELD_NAME = values.Value(
"returnTo", environ_name="OIDC_REDIRECT_FIELD_NAME", environ_prefix=None
)
OIDC_USERINFO_FULLNAME_FIELDS = values.ListValue(
default=["given_name", "usual_name"],
environ_name="OIDC_USERINFO_FULLNAME_FIELDS",
environ_prefix=None,
)
OIDC_USERINFO_SHORTNAME_FIELD = values.Value(
default="given_name",
environ_name="OIDC_USERINFO_SHORTNAME_FIELD",
environ_prefix=None,
)
# Video conference configuration
LIVEKIT_CONFIGURATION = {
@@ -376,62 +411,79 @@ class Base(Configuration):
),
"url": values.Value(environ_name="LIVEKIT_API_URL", environ_prefix=None),
}
DEFAULT_ROOM_IS_PUBLIC = values.BooleanValue(
True, environ_name="DEFAULT_ROOM_IS_PUBLIC", environ_prefix=None
RESOURCE_DEFAULT_IS_PUBLIC = values.BooleanValue(
True, environ_name="RESOURCE_DEFAULT_IS_PUBLIC", environ_prefix=None
)
ALLOW_UNREGISTERED_ROOMS = values.BooleanValue(
True, environ_name="ALLOW_UNREGISTERED_ROOMS", environ_prefix=None
)
ANALYTICS_KEY = values.Value(
None, environ_name="ANALYTICS_KEY", environ_prefix=None
)
# todo - totally wip
AWS_S3_ENDPOINT_URL = values.Value(
environ_name="AWS_S3_ENDPOINT_URL", environ_prefix=None
# Recording settings
RECORDING_ENABLE = values.BooleanValue(
False, environ_name="RECORDING_ENABLE", environ_prefix=None
)
AWS_S3_ACCESS_KEY_ID = values.Value(
environ_name="AWS_S3_ACCESS_KEY_ID", environ_prefix=None
RECORDING_OUTPUT_FOLDER = values.Value(
"recordings", environ_name="RECORDING_OUTPUT_FOLDER", environ_prefix=None
)
AWS_S3_SECRET_ACCESS_KEY = values.Value(
environ_name="AWS_S3_SECRET_ACCESS_KEY", environ_prefix=None
RECORDING_VERIFY_SSL = values.BooleanValue(
True, environ_name="RECORDING_VERIFY_SSL", environ_prefix=None
)
AWS_S3_REGION_NAME = values.Value(
environ_name="AWS_S3_REGION_NAME", environ_prefix=None
)
AWS_STORAGE_BUCKET_NAME = values.Value(
"meet-media-storage",
environ_name="AWS_STORAGE_BUCKET_NAME",
RECORDING_WORKER_CLASSES = values.DictValue(
{
"screen_recording": "core.recording.worker.services.VideoCompositeEgressService",
"transcript": "core.recording.worker.services.AudioCompositeEgressService",
},
environ_name="RECORDING_WORKER_CLASSES",
environ_prefix=None,
)
OPENAI_API_KEY = values.Value(
None, environ_name="OPENAI_API_KEY", environ_prefix=None
RECORDING_EVENT_PARSER_CLASS = values.Value(
"core.recording.event.parsers.MinioParser",
environ_name="RECORDING_EVENT_PARSER_CLASS",
environ_prefix=None,
)
OPENAI_ENABLE = values.BooleanValue(
True, environ_name="OPENAI_ENABLE", environ_prefix=None
RECORDING_ENABLE_STORAGE_EVENT_AUTH = values.BooleanValue(
True, environ_name="RECORDING_ENABLE_STORAGE_EVENT_AUTH", environ_prefix=None
)
RECORDING_STORAGE_EVENT_ENABLE = values.BooleanValue(
False, environ_name="RECORDING_STORAGE_EVENT_ENABLE", environ_prefix=None
)
RECORDING_STORAGE_EVENT_TOKEN = values.Value(
None, environ_name="RECORDING_STORAGE_EVENT_TOKEN", environ_prefix=None
)
SUMMARY_SERVICE_ENDPOINT = values.Value(
None, environ_name="SUMMARY_SERVICE_ENDPOINT", environ_prefix=None
)
SUMMARY_SERVICE_API_TOKEN = values.Value(
None, environ_name="SUMMARY_SERVICE_API_TOKEN", environ_prefix=None
)
# todo - totally wip
MINIO_ACCESS_KEY = values.Value(
None, environ_name="MINIO_ACCESS_KEY", environ_prefix=None
# Marketing and communication settings
SIGNUP_NEW_USER_TO_MARKETING_EMAIL = values.BooleanValue(
False,
environ_name="SIGNUP_NEW_USERS_TO_NEWSLETTER",
environ_prefix=None,
help_text=(
"When enabled, new users are automatically added to mailing list "
"for product updates, marketing communications, and customized emails. "
),
)
MINIO_SECRET_KEY = values.Value(
None, environ_name="MINIO_SECRET_KEY", environ_prefix=None
MARKETING_SERVICE_CLASS = values.Value(
"core.services.marketing_service.BrevoMarketingService",
environ_name="MARKETING_SERVICE_CLASS",
environ_prefix=None,
)
MINIO_URL = values.Value(
None, environ_name="MINIO_URL", environ_prefix=None
BREVO_API_KEY = values.Value(
None, environ_name="BREVO_API_KEY", environ_prefix=None
)
MINIO_BUCKET = values.Value(
'livekit-staging-livekit-egress', environ_name="MINIO_BUCKET", environ_prefix=None
BREVO_API_CONTACT_LIST_IDS = values.ListValue(
[],
environ_name="BREVO_API_CONTACT_LIST_IDS",
environ_prefix=None,
converter=lambda x: int(x), # pylint: disable=unnecessary-lambda
)
BREVO_API_CONTACT_ATTRIBUTES = values.DictValue({"VISIO_USER": True})
BLOCKNOTE_CONVERTER_URL = values.Value(
'https://converter-blocknote.osc-fr1.scalingo.io/', environ_name="", environ_prefix=None
)
DOCS_BASE_URL = values.Value(
'https://docs-ia.beta.numerique.gouv.fr', environ_name="", environ_prefix=None
)
PASSPHRASE_ENCRYPTION_KEY = values.Value(environ_name="PASSPHRASE_ENCRYPTION_KEY", environ_prefix=None)
# pylint: disable=invalid-name
@property
@@ -479,8 +531,10 @@ class Base(Configuration):
release=get_release(),
integrations=[DjangoIntegration()],
)
with sentry_sdk.configure_scope() as scope:
scope.set_extra("application", "backend")
# Add the application name to the Sentry scope
scope = sentry_sdk.get_global_scope()
scope.set_tag("application", "backend")
class Build(Base):
@@ -552,8 +606,6 @@ class Test(Base):
CELERY_TASK_ALWAYS_EAGER = values.BooleanValue(True)
ANALYTICS_KEY = None
def __init__(self):
# pylint: disable=invalid-name
self.INSTALLED_APPS += ["drf_spectacular_sidecar"]
@@ -576,20 +628,6 @@ class Production(Base):
ALLOWED_HOSTS=["foo.com", "foo.fr"]
"""
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"handlers": {
"console": {
"class": "logging.StreamHandler",
},
},
"root": {
"handlers": ["console"],
"level": "INFO",
},
}
# Security
ALLOWED_HOSTS = [
*values.ListValue([], environ_name="ALLOWED_HOSTS"),
+23 -23
View File
@@ -7,7 +7,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "meet"
version = "0.1.7"
version = "0.1.12"
authors = [{ "name" = "DINUM", "email" = "dev@mail.numerique.gouv.fr" }]
classifiers = [
"Development Status :: 5 - Production/Stable",
@@ -25,41 +25,40 @@ license = { file = "LICENSE" }
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
"boto3==1.35.19",
"boto3==1.35.90",
"Brotli==1.1.0",
"brevo-python==1.1.2",
"celery[redis]==5.4.0",
"django-configurations==2.5.1",
"django-cors-headers==4.4.0",
"django-cors-headers==4.6.0",
"django-countries==7.6.1",
"django-parler==2.3",
"redis==5.0.8",
"redis==5.2.1",
"django-redis==5.4.0",
"django-storages[s3]==1.14.4",
"django-timezone-field>=5.1",
"django==5.1.1",
"django==5.1.4",
"djangorestframework==3.15.2",
"drf_spectacular==0.27.2",
"drf_spectacular==0.28.0",
"dockerflow==2024.4.2",
"easy_thumbnails==2.10",
"factory_boy==3.3.1",
"freezegun==1.5.1",
"gunicorn==23.0.0",
"jsonschema==4.23.0",
"june-analytics-python==2.3.0",
"markdown==3.7",
"nested-multipart-parser==1.5.0",
"psycopg[binary]==3.2.2",
"PyJWT==2.9.0",
"psycopg[binary]==3.2.3",
"PyJWT==2.10.1",
"python-frontmatter==1.1.0",
"requests==2.32.3",
"sentry-sdk==2.14.0",
"sentry-sdk==2.19.2",
"url-normalize==1.4.3",
"WeasyPrint>=60.2",
"whitenoise==6.7.0",
"whitenoise==6.8.2",
"mozilla-django-oidc==4.0.1",
"livekit-api==0.7.0",
"minio==7.2.9",
"openai==1.51.2"
"livekit-api==0.8.1",
"aiohttp==3.11.11",
]
[project.urls]
@@ -71,20 +70,21 @@ dependencies = [
[project.optional-dependencies]
dev = [
"django-extensions==3.2.3",
"drf-spectacular-sidecar==2024.7.1",
"drf-spectacular-sidecar==2024.12.1",
"freezegun==1.5.1",
"ipdb==0.13.13",
"ipython==8.27.0",
"pyfakefs==5.6.0",
"pylint-django==2.5.5",
"pylint==3.2.7",
"pytest-cov==5.0.0",
"ipython==8.31.0",
"pyfakefs==5.7.3",
"pylint-django==2.6.1",
"pylint==3.3.3",
"pytest-cov==6.0.0",
"pytest-django==4.9.0",
"pytest==8.3.3",
"pytest==8.3.4",
"pytest-icdiff==0.9",
"pytest-xdist==3.6.1",
"responses==0.25.3",
"ruff==0.6.5",
"types-requests==2.32.0.20240914",
"ruff==0.8.4",
"types-requests==2.32.0.20241016",
]
[tool.setuptools]
+5 -5
View File
@@ -1,4 +1,4 @@
FROM node:20-alpine as frontend-deps
FROM node:20-alpine AS frontend-deps
WORKDIR /home/frontend/
@@ -11,11 +11,11 @@ COPY .dockerignore ./.dockerignore
COPY ./src/frontend/ .
### ---- Front-end builder image ----
FROM frontend-deps as meet
FROM frontend-deps AS meet
WORKDIR /home/frontend
FROM frontend-deps as meet-dev
FROM frontend-deps AS meet-dev
WORKDIR /home/frontend
@@ -25,14 +25,14 @@ CMD [ "npm", "run", "dev"]
# Tilt will rebuild Meet target so, we dissociate meet and meet-builder
# to avoid rebuilding the app at every changes.
FROM meet as meet-builder
FROM meet AS meet-builder
WORKDIR /home/frontend
RUN npm run build
# ---- Front-end image ----
FROM nginxinc/nginx-unprivileged:1.26-alpine as frontend-production
FROM nginxinc/nginx-unprivileged:1.26-alpine AS frontend-production
# Un-privileged user running the application
ARG DOCKER_USER
+1 -1
View File
@@ -2,7 +2,7 @@
<html lang="en" data-lk-theme="visio-light">
<head>
<meta charset="UTF-8" />
<link rel="icon" type="image/svg+xml" href="/play-icon.svg" />
<link rel="icon" type="image/svg+xml" href="/assets/icon.svg" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Visio</title>
</head>
+1587 -1675
View File
File diff suppressed because it is too large Load Diff
+29 -29
View File
@@ -1,7 +1,7 @@
{
"name": "meet",
"private": true,
"version": "0.1.7",
"version": "0.1.12",
"type": "module",
"scripts": {
"dev": "panda codegen && vite",
@@ -13,48 +13,48 @@
"check": "prettier --check ./src"
},
"dependencies": {
"@livekit/components-react": "2.6.5",
"@livekit/components-styles": "1.1.3",
"@livekit/components-react": "2.6.9",
"@livekit/components-styles": "1.1.4",
"@livekit/track-processors": "0.3.2",
"@pandacss/preset-panda": "0.46.1",
"@react-aria/toast": "3.0.0-beta.16",
"@remixicon/react": "4.2.0",
"@tanstack/react-query": "5.59.4",
"@pandacss/preset-panda": "0.48.0",
"@react-aria/toast": "3.0.0-beta.18",
"@remixicon/react": "4.5.0",
"@tanstack/react-query": "5.61.3",
"crisp-sdk-web": "1.0.25",
"hoofd": "1.7.1",
"i18next": "23.15.2",
"i18next": "24.0.2",
"i18next-browser-languagedetector": "8.0.0",
"i18next-parser": "9.0.2",
"i18next-resources-to-backend": "1.2.1",
"livekit-client": "2.5.7",
"posthog-js": "1.167.0",
"livekit-client": "2.6.3",
"posthog-js": "1.188.0",
"react": "18.3.1",
"react-aria-components": "1.4.0",
"react-aria-components": "1.5.0",
"react-dom": "18.3.1",
"react-i18next": "15.0.2",
"react-i18next": "15.1.1",
"use-sound": "4.0.3",
"valtio": "2.0.0",
"valtio": "2.1.2",
"wouter": "3.3.5"
},
"devDependencies": {
"@pandacss/dev": "0.46.1",
"@tanstack/eslint-plugin-query": "5.59.2",
"@tanstack/react-query-devtools": "5.59.4",
"@types/node": "20.16.11",
"@types/react": "18.3.11",
"@types/react-dom": "18.3.0",
"@typescript-eslint/eslint-plugin": "8.8.1",
"@typescript-eslint/parser": "8.8.1",
"@vitejs/plugin-react": "4.3.2",
"@pandacss/dev": "0.48.0",
"@tanstack/eslint-plugin-query": "5.61.3",
"@tanstack/react-query-devtools": "5.61.3",
"@types/node": "22.9.3",
"@types/react": "18.3.12",
"@types/react-dom": "18.3.1",
"@typescript-eslint/eslint-plugin": "8.15.0",
"@typescript-eslint/parser": "8.15.0",
"@vitejs/plugin-react": "4.3.3",
"eslint": "8.57.0",
"eslint-config-prettier": "9.1.0",
"eslint-plugin-jsx-a11y": "6.10.0",
"eslint-plugin-react-hooks": "4.6.2",
"eslint-plugin-react-refresh": "0.4.12",
"postcss": "8.4.47",
"eslint-plugin-jsx-a11y": "6.10.2",
"eslint-plugin-react-hooks": "5.0.0",
"eslint-plugin-react-refresh": "0.4.14",
"postcss": "8.4.49",
"prettier": "3.3.3",
"typescript": "5.6.3",
"vite": "5.4.8",
"vite-tsconfig-paths": "5.0.1"
"typescript": "5.7.2",
"vite": "5.4.11",
"vite-tsconfig-paths": "5.1.3"
}
}
+100 -5
View File
@@ -35,11 +35,46 @@ const config: Config = {
exclude: [],
jsxFramework: 'react',
outdir: 'src/styled-system',
globalFontface: {
Marianne: [
{
src: 'url(/fonts/Marianne-Regular-subset.woff2) format("woff2")',
fontWeight: 400,
fontStyle: 'normal',
fontDisplay: 'swap',
},
{
path: 'url(/fonts/Marianne-Regular_Italic-subset.woff2) format("woff2")',
fontWeight: 400,
fontStyle: 'italic',
fontDisplay: 'swap',
},
{
path: 'url(/fonts/Marianne-Medium-subset.woff2) format("woff2")',
fontWeight: 500,
fontStyle: 'normal',
fontDisplay: 'swap',
},
{
path: 'url(/fonts/Marianne-Bold-subset.woff2) format("woff2")',
fontWeight: 700,
fontStyle: 'normal',
fontDisplay: 'swap',
},
{
path: 'url(/fonts/Marianne-ExtraBold-subset.woff2) format("woff2")',
fontWeight: 800,
fontStyle: 'normal',
fontDisplay: 'swap',
},
],
},
theme: {
...pandaPreset.theme,
// media queries are defined in em so that zooming with text-only mode triggers breakpoints
breakpoints: {
xs: '22.6em', // 360px (we assume less than that are old/entry level mobile phones)
xsm: '31.25em', // 500px,
sm: '40em', // 640px
md: '48em', // 768px
lg: '64em', // 1024px
@@ -81,10 +116,10 @@ const config: Config = {
'80%': { transform: 'rotate(20deg)' },
'100%': { transform: 'rotate(0)' },
},
pulse_mic: {
'0%': { color: 'primary', opacity: '1' },
'50%': { color: 'primary', opacity: '0.8' },
'100%': { color: 'primary', opacity: '1' },
pulse_background: {
'0%': { opacity: '1' },
'50%': { opacity: '0.65' },
'100%': { opacity: '1' },
},
},
tokens: defineTokens({
@@ -92,6 +127,66 @@ const config: Config = {
* This way we'll only add the things we need step by step and prevent using lots of differents things.
*/
...pandaPreset.theme.tokens,
colors: defineTokens.colors({
...pandaPreset.theme.tokens.colors,
primaryDark: {
50: { value: '#161622' },
100: { value: '#2D2D46' },
200: { value: '#43436A' },
300: { value: '#5A5A8F' },
400: { value: '#7070B3' },
500: { value: '#8787D7' },
600: { value: '#9D9DDF' },
700: { value: '#B3B3E7' },
800: { value: '#C9C9EE' },
900: { value: '#DFDFF6' },
950: { value: '#F5F5FE' },
action: { value: '#C1C1FB' },
},
primary: {
50: { value: '#F5F5FE' },
100: { value: '#ECECFE' },
200: { value: '#E3E3FB' },
300: { value: '#CACAFB' },
400: { value: '#8585F6' },
500: { value: '#6A6AF4' },
600: { value: '#313178' },
700: { value: '#272747' },
800: { value: '#000091' },
900: { value: '#21213F' },
950: { value: '#1B1B35' },
action: { value: '#1212FF' },
},
greyscale: {
'000': { value: '#FFFFFF' },
50: { value: '#F6F6F6' },
100: { value: '#EEEEEE' },
200: { value: '#E5E5E5' },
250: { value: '#DDDDDD' },
300: { value: '#CECECE' },
400: { value: '#929292' },
500: { value: '#7C7C7C' },
600: { value: '#666666' },
700: { value: '#3A3A3A' },
750: { value: '#353535' },
800: { value: '#2A2A2A' },
900: { value: '#242424' },
950: { value: '#1E1E1E' },
1000: { value: '#161616' },
},
error: {
100: { value: '#261212' },
200: { value: '#6C302E' },
300: { value: '#983533' },
400: { value: '#CA3632' },
500: { value: '#EF413D' },
600: { value: '#EE6A66' },
700: { value: '#F28D8A' },
800: { value: '#F6AFAD' },
900: { value: '#FAD2D1' },
950: { value: '#FFF4F4' },
},
}),
animations: {},
blurs: {},
/* just directly use values as tokens. This allows us to follow a specific design scale,
@@ -193,7 +288,7 @@ const config: Config = {
semanticTokens: defineSemanticTokens({
colors: {
default: {
text: { value: '{colors.gray.900}' },
text: { value: '{colors.greyscale.1000}' },
bg: { value: 'white' },
subtle: { value: '{colors.gray.100}' },
'subtle-text': { value: '{colors.gray.600}' },
File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 13 KiB

+5
View File
@@ -0,0 +1,5 @@
<svg width="160" height="160" viewBox="0 0 160 160" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M5.61135 59.6951C5 62.832 5 66.6949 5 73.1656V88.7307C5 96.7588 5 100.773 6.16756 104.366C7.20062 107.546 8.89041 110.472 11.1274 112.957C13.6555 115.765 17.1318 117.772 24.0842 121.786L37.564 129.568C43.7169 133.121 47.1472 135.101 50.4165 136.054V88.0288C50.4165 85.6281 49.1052 83.4192 46.9976 82.2696L5.61135 59.6951Z" fill="#C9191E"/>
<path d="M118.313 66.7489V91.7898C118.313 95.2521 119.818 98.5435 122.436 100.809L140.459 116.406C141.513 117.298 142.608 118.028 143.744 118.596C144.92 119.123 146.056 119.387 147.151 119.387C149.503 119.387 151.389 118.616 152.809 117.075C154.269 115.493 154.999 113.445 154.999 110.93V47.6577C154.999 45.1431 154.269 43.1151 152.809 41.5738C151.389 39.992 149.503 39.2011 147.151 39.2011C146.056 39.2011 144.92 39.4647 143.744 39.992C142.608 40.5193 141.513 41.2494 140.459 42.1822L122.45 57.7172C119.823 59.983 118.313 63.28 118.313 66.7489Z" fill="#000091"/>
<path d="M11.6345 50.7522C11.1333 50.4788 10.6078 50.2937 10.0757 50.1915C10.4114 49.7628 10.7622 49.3452 11.1276 48.9394C13.6558 46.1315 17.132 44.1245 24.0845 40.1105L37.5643 32.3279C44.5167 28.3139 47.993 26.3069 51.6887 25.5213C54.9587 24.8262 58.3383 24.8262 61.6083 25.5213C65.304 26.3069 68.7803 28.3139 75.7327 32.3279L89.0535 40.0187C89.1066 40.0492 89.1597 40.0798 89.2129 40.1105C96.1653 44.1245 99.6416 46.1316 102.17 48.9394C104.407 51.4238 106.096 54.3506 107.13 57.5301C108.297 61.1235 108.297 65.1375 108.297 73.1656V88.7307C108.297 96.7588 108.297 100.773 107.13 104.366C106.096 107.546 104.407 110.473 102.17 112.957C99.6416 115.765 96.1655 117.772 89.2133 121.785C89.1537 121.82 89.0936 121.855 89.0341 121.889L75.7327 129.568C68.7803 133.582 65.304 135.589 61.6083 136.375C61.4564 136.407 61.3043 136.438 61.1519 136.467L61.1519 88.0288C61.1519 81.6997 57.6948 75.8761 52.1386 72.8455L11.6345 50.7522Z" fill="#000091"/>
</svg>

After

Width:  |  Height:  |  Size: 1.9 KiB

+3
View File
@@ -0,0 +1,3 @@
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M6.66667 4.00098V5.33431H3.33333V12.6676H10.6667V9.33431H12V13.3343C12 13.7025 11.7015 14.001 11.3333 14.001H2.66667C2.29848 14.001 2 13.7025 2 13.3343V4.66764C2 4.29945 2.29848 4.00098 2.66667 4.00098H6.66667ZM14 2.00098V7.33431H12.6667V4.27631L7.47133 9.47231L6.52867 8.52964L11.7233 3.33431H8.66667V2.00098H14Z" fill="#666666"/>
</svg>

After

Width:  |  Height:  |  Size: 484 B

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.
Binary file not shown.
+1 -1
View File
@@ -30,7 +30,7 @@ function App() {
<Route component={NotFoundScreen} />
</Switch>
</Layout>
<ReactQueryDevtools initialIsOpen={false} />
<ReactQueryDevtools initialIsOpen={false} buttonPosition="top-left" />
</I18nProvider>
</Suspense>
</QueryClientProvider>
+5
View File
@@ -1,6 +1,7 @@
import { fetchApi } from './fetchApi'
import { keys } from './queryKeys'
import { useQuery } from '@tanstack/react-query'
import { RecordingMode } from '@/features/rooms/api/startRecording'
export interface ApiConfig {
analytics?: {
@@ -11,6 +12,10 @@ export interface ApiConfig {
id: string
}
silence_livekit_debug_logs?: boolean
recording?: {
is_enabled?: boolean
available_modes?: RecordingMode[]
}
}
const fetchConfig = (): Promise<ApiConfig> => {
Binary file not shown.

After

Width:  |  Height:  |  Size: 651 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 702 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 599 KiB

+6
View File
@@ -0,0 +1,6 @@
<svg width="448" height="172" viewBox="0 0 448 172" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M5.61135 65.6951C5 68.832 5 72.6949 5 79.1656V94.7307C5 102.759 5 106.773 6.16756 110.366C7.20062 113.546 8.89041 116.472 11.1274 118.957C13.6555 121.765 17.1318 123.772 24.0842 127.786L37.564 135.568C43.7169 139.121 47.1472 141.101 50.4165 142.054V94.0288C50.4165 91.6281 49.1052 89.4192 46.9976 88.2696L5.61135 65.6951Z" fill="#C9191E"/>
<path d="M118.313 72.7489V97.7898C118.313 101.252 119.818 104.544 122.436 106.809L140.459 122.406C141.513 123.298 142.608 124.028 143.744 124.596C144.92 125.123 146.056 125.387 147.151 125.387C149.503 125.387 151.389 124.616 152.809 123.075C154.269 121.493 154.999 119.445 154.999 116.93V53.6577C154.999 51.1431 154.269 49.1151 152.809 47.5738C151.389 45.992 149.503 45.2011 147.151 45.2011C146.056 45.2011 144.92 45.4647 143.744 45.992C142.608 46.5193 141.513 47.2494 140.459 48.1822L122.45 63.7172C119.823 65.983 118.313 69.28 118.313 72.7489Z" fill="#000091"/>
<path d="M11.6345 56.7522C11.1333 56.4788 10.6078 56.2937 10.0757 56.1915C10.4114 55.7628 10.7622 55.3452 11.1276 54.9394C13.6558 52.1315 17.132 50.1245 24.0845 46.1105L37.5643 38.3279C44.5167 34.3139 47.993 32.3069 51.6887 31.5213C54.9587 30.8262 58.3383 30.8262 61.6083 31.5213C65.304 32.3069 68.7803 34.3139 75.7327 38.3279L89.0535 46.0187C89.1066 46.0492 89.1597 46.0798 89.2129 46.1105C96.1653 50.1245 99.6416 52.1316 102.17 54.9394C104.407 57.4238 106.096 60.3506 107.13 63.5301C108.297 67.1235 108.297 71.1375 108.297 79.1656V94.7307C108.297 102.759 108.297 106.773 107.13 110.366C106.096 113.546 104.407 116.473 102.17 118.957C99.6416 121.765 96.1655 123.772 89.2133 127.785C89.1537 127.82 89.0936 127.855 89.0341 127.889L75.7327 135.568C68.7803 139.582 65.304 141.589 61.6083 142.375C61.4564 142.407 61.3043 142.438 61.1519 142.467L61.1519 94.0288C61.1519 87.6997 57.6948 81.8761 52.1386 78.8455L11.6345 56.7522Z" fill="#000091"/>
<path d="M193.72 45.7333H211.035L234.771 108.456L258.507 45.7333H275.821L245.435 126H224.107L193.72 45.7333ZM289.019 58.1173C283.859 58.1173 279.501 53.76 279.501 48.6C279.501 43.44 283.859 39.0827 289.019 39.0827C294.179 39.0827 298.421 43.44 298.421 48.6C298.421 53.76 294.179 58.1173 289.019 58.1173ZM281.68 126V68.208H296.243V126H281.68ZM303.29 117.629L312.922 108.915C316.477 113.387 320.72 116.597 326.224 116.597C330.925 116.597 333.333 113.845 333.333 110.405C333.333 100.315 305.698 104.099 305.698 83.8027C305.698 73.5973 314.298 65.9147 326.338 65.9147C335.168 65.9147 343.194 70.1573 347.322 75.776L337.69 84.2613C334.709 80.592 330.925 77.6107 326.453 77.6107C321.866 77.6107 319.688 80.1333 319.688 83.1147C319.688 92.976 347.322 89.536 347.322 109.488C347.093 121.643 337.346 128.293 326.453 128.293C316.133 128.293 308.794 124.165 303.29 117.629ZM363.494 58.1173C358.334 58.1173 353.977 53.76 353.977 48.6C353.977 43.44 358.334 39.0827 363.494 39.0827C368.654 39.0827 372.897 43.44 372.897 48.6C372.897 53.76 368.654 58.1173 363.494 58.1173ZM356.155 126V68.208H370.718V126H356.155ZM411.115 65.9147C429.92 65.9147 442.763 79.7893 442.763 97.104C442.763 114.419 429.92 128.293 411.115 128.293C392.309 128.293 379.467 114.419 379.467 97.104C379.467 79.7893 392.309 65.9147 411.115 65.9147ZM411.344 114.533C420.632 114.533 427.627 107.08 427.627 97.104C427.627 87.0133 420.632 79.6747 411.344 79.6747C401.712 79.6747 394.603 87.0133 394.603 97.104C394.603 107.195 401.712 114.533 411.344 114.533Z" fill="#000091"/>
</svg>

After

Width:  |  Height:  |  Size: 3.4 KiB

-25
View File
@@ -1,25 +0,0 @@
import { css } from '@/styled-system/css'
import { RiExternalLinkLine } from '@remixicon/react'
import { useTranslation } from 'react-i18next'
import { LinkButton } from '@/primitives'
export const Feedback = () => {
const { t } = useTranslation()
return (
<LinkButton
href="https://grist.incubateur.net/o/docs/forms/1YrfNP1QSSy8p2gCxMFnSf/4"
variant="success"
target="_blank"
>
<span className={css({ marginRight: 0.5 })} aria-hidden="true">
💡
</span>
{t('feedbackAlert')}
<RiExternalLinkLine
size={16}
className={css({ marginLeft: 0.5 })}
aria-hidden="true"
/>
</LinkButton>
)
}
@@ -0,0 +1,44 @@
import { css } from '@/styled-system/css'
import { RiErrorWarningLine, RiExternalLinkLine } from '@remixicon/react'
import { useTranslation } from 'react-i18next'
import { Text, A } from '@/primitives'
import { GRIST_FORM } from '@/utils/constants'
export const FeedbackBanner = () => {
const { t } = useTranslation()
return (
<div
className={css({
width: '100%',
backgroundColor: '#E8EDFF',
color: '#0063CB',
display: { base: 'none', sm: 'flex' },
justifyContent: 'center',
padding: '0.5rem 0',
})}
>
<div
className={css({
display: 'inline-flex',
gap: '0.5rem',
alignItems: 'center',
})}
>
<RiErrorWarningLine size={20} />
<Text as="p">{t('feedback.context')}</Text>
<div
className={css({
display: 'flex',
alignItems: 'center',
gap: 0.25,
})}
>
<A href={GRIST_FORM} target="_blank">
{t('feedback.cta')}
</A>
<RiExternalLinkLine size={16} aria-hidden="true" />
</div>
</div>
</div>
)
}
@@ -7,6 +7,7 @@ import { Center } from '@/styled-system/jsx'
export const LoadingScreen = ({
delay = 500,
header = undefined,
footer = undefined,
layout = 'centered',
}: {
delay?: number
@@ -15,7 +16,7 @@ export const LoadingScreen = ({
return (
<DelayedRender delay={delay}>
<Screen layout={layout} header={header}>
<Screen layout={layout} header={header} footer={footer}>
<CenteredContent>
<Center>
<p>{t('loading')}</p>
+1 -1
View File
@@ -21,7 +21,7 @@ export const QueryAware = ({
}
if (status === 'pending') {
return <LoadingScreen header={undefined} />
return <LoadingScreen header={undefined} footer={undefined} />
}
return children
+1 -1
View File
@@ -27,7 +27,7 @@ export const SoundTester = () => {
return (
<>
<Button
invisible
variant="secondaryText"
onPress={() => {
audioRef?.current?.play()
setIsPlaying(true)

Some files were not shown because too many files have changed in this diff Show More