Compare commits

...

613 Commits

Author SHA1 Message Date
renovate[bot] f866529307 ⬆️(dependencies) update python dependencies 2026-03-23 01:05:24 +00:00
renovate[bot] 99eaec9653 ⬆️(dependencies) update next to v15.5.14 [SECURITY] 2026-03-20 10:51:17 +00:00
renovate[bot] 2ab3a67a44 ⬆️(dependencies) update next to v15.5.13 [SECURITY] 2026-03-20 10:30:25 +00:00
Marie PUPO JEAMMET 0fd29e82d5 🧑‍💻(makefile) build front when bootstraping
add frontend-dev to build command to rebuild front image
2026-03-19 11:39:14 +00:00
Eléonore 0776b7d95a 💄(ui-kit) regie v2 ui-kit (#1083)
update regie to ui-kit v2
2026-03-19 10:56:07 +01:00
Eléonore Voisin 19f7bb77c0 🚚(route) prioritize mail domains as default landing page
add line in changelog
2026-03-19 10:30:19 +01:00
Louis 83cae4036c (frontend) update test to expect /mail-domains/ as fallback redirect
The fallback now always redirects to /mail-domains/, so the test
with TEAMS_DISPLAY=true should expect /mail-domains/ instead of /teams/.
2026-03-19 10:30:19 +01:00
Louis ed07eecad4 🐛(frontend) fix fallback redirect to always go to /mail-domains/
The ternary was incorrectly inverted, causing users to be sent to
/teams/ when TEAMS_DISPLAY was off. Simplify to always redirect
to /mail-domains/ as the fallback.
2026-03-19 10:30:19 +01:00
Louis 46ba75b119 🚸(frontend) prioritize mail domains as default landing page
(made with ai, review carefully)
 Swap the redirect priority so that users landing on the root page
are directed to /mail-domains/ instead of /teams/ by default.
This applies both to the permission-based checks and the fallback
using the TEAMS_DISPLAY feature flag.
2026-03-19 10:30:19 +01:00
Louis 1b4d1043af 🍱(static) update logo in invitation email template (#1085)
Replace logo-footer-mail.png with an updated version in the
invitation email template. Remove unused logo-suite-numerique.png.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-19 08:43:29 +01:00
Eléonore bc458b3f68 (datagrid) add sort to mailboxes list + mail domain list (#1057)
add sort to mailboxes list + mail domain list

(invitations) allow delete invitations by an admin (#1052)

allow delete invitations mails domains access by an admin
2026-03-16 16:23:29 +00:00
renovate[bot] 5bc845172b ⬆️(dependencies) update PyJWT to v2.12.0 [SECURITY] 2026-03-16 11:47:31 +00:00
Marie PUPO JEAMMET 8a35e862ff ♻️(tests) finish cleaning dimail token fixture
finish cleaning the fixtures returning valid
token as dimail
2026-03-13 15:20:04 +00:00
Marie PUPO JEAMMET bc0dbcb678 🩹(mailboxes) enforce mailboxes are lowercase
enforce lowercase for new mailboxes, upon creation
and when sending to dimail
2026-03-13 15:20:04 +00:00
Stephan Meijer 68ec85e94a ⬆️(ci) upgrade GitHub Actions workflow steps to latest versions
Update all GitHub Actions to their latest major versions for improved
performance, security patches, and Node.js runtime compatibility.

Signed-off-by: Stephan Meijer <me@stephanmeijer.com>
2026-03-13 14:47:35 +00:00
Marie PUPO JEAMMET ca754d5bcd ⬆️(security) upgrade tornado to fix CVE
Upgrade tornado to fix this CVE
https://github.com/tornadoweb/tornado/security/advisories/GHSA-qjxf-f2mg-c6mc
2026-03-13 14:29:15 +00:00
renovate[bot] 06df255072 ⬆️(dependencies) update python dependencies 2026-03-13 14:29:15 +00:00
Jacob Weisz da1a9e1d7a ✏️(typo) fix readme typo
Small fix of a typo in the README
2026-03-12 18:01:45 +00:00
Louis 74ada8fd6b 🐛(i18n) fix missing translations for status tag labels
(made with ai, review carefully)
Status labels in the Tag component ("enabled", "action required", etc.) were
not translated because `t(status)` used a dynamic variable that i18next-parser
cannot extract.

Replace with a static `translations` map using literal `t()` calls so the keys
are properly detected during extraction and sent to Crowdin for translation.

Co-authored-by: BEY Quentin <quentin.bey@mail.numerique.gouv.fr>
2026-03-12 17:33:22 +00:00
renovate[bot] 0c123bb9de ⬆️(dependencies) update django to v5.2.12 [SECURITY] 2026-03-12 17:17:47 +00:00
renovate[bot] 2a6367c3ac ⬆️(dependencies) update joserfc to v1.6.3 [SECURITY] 2026-03-12 17:17:47 +00:00
Quentin BEY 79bdc6606e ⬆️(dependencies) bump django-lasuite to 0.0.25
Required to use the django-lasuite custom admin.
2026-03-10 14:47:33 +01:00
Quentin BEY c53434411d 🔥(admin) switch to django-lasuite customization
The admin application override has been moved into
the django-lasuite library.
2026-02-26 09:21:56 +01:00
Marie PUPO JEAMMET b9c69e7c08 🚨(lint) fix linter warnings
fix linter warnings after ruff upgrade
2026-02-18 18:13:42 +00:00
renovate[bot] b1e0a24626 ⬆️(dependencies) update python dependencies 2026-02-18 18:13:42 +00:00
mjeammet a1b58fb864 🌐(i18n) update translated strings
Update translated files with new translations
2026-02-16 14:39:42 +00:00
Marie PUPO JEAMMET e1a8cc31f9 🔖(patch) release version 1.23.1
Update all version files and changelog for patch release.
2026-02-16 14:39:42 +00:00
Marie PUPO JEAMMET d0e5aa5952 🚚(exceptions) move invitation-related exception to core
move invitation-related from mailbox_manager to core, as
it will be useful for teams too
2026-02-13 11:09:17 +00:00
Marie PUPO JEAMMET 3b58fb7e1e (invitations) refresh invitations
invitations can now be refreshed, de-expiring
them or moving their expiration date further
in the future.
2026-02-13 11:09:17 +00:00
Marie PUPO JEAMMET 1e02ded7b8 ⬆️(dependencies) bump pillow and cryptography to fix CVEs
Bump with `uv lock --upgrade-package pillow` or `cryptography`
This fixes CVEs https://avd.aquasec.com/nvd/cve-2026-26007 and
https://avd.aquasec.com/nvd/cve-2026-25990
2026-02-12 15:18:31 +00:00
mjeammet 92fcb359c9 🌐(i18n) update translated strings
Update translated files with new translations
2026-02-12 15:18:31 +00:00
Marie PUPO JEAMMET 8c9b35ec47 🔖(minor) release version 1.23.0
Update all version files and changelog for minor release.
2026-02-12 15:18:31 +00:00
Marie PUPO JEAMMET 4e75d675f8 (stats) count aliases in stats endpoint
stats endpoint now return the count of aliases
2026-02-12 13:55:53 +00:00
Marie PUPO JEAMMET 1d4d40aad0 🧑‍💻(demo) add aliases to demo
add aliases to all domains in demo
2026-02-12 13:55:53 +00:00
Quentin BEY aaa9b27c61 🚸(email) we should ignore case when looking for existing emails
Our products mostly rely on email regardless of their case.

Next step would be to normalize email to lower case when storing
them in database (or sending them to dimail if not done yet).
2026-02-12 09:47:24 +01:00
elvoisin a29ef05d8b (front) add icon to button to configure a domain (#1054)
Added an icon to help understanding of the configuration button/tag
2026-02-11 15:39:42 +00:00
elvoisin 40c39bd9ba (invitations) allow delete invitations by an admin (#1052)
allow delete invitations mails domains access by an admin
2026-02-11 10:29:03 +01:00
Quentin BEY 59f9f54b34 💚(crowdin) fix upload job
Since the switch to `uv` we should not use pip.
2026-02-10 15:01:50 +01:00
Marie PUPO JEAMMET 7804583632 (tests) simplify mail domain invitations tests
just a quick simplification for our tests regarding domain invitations
2026-02-06 11:09:43 +00:00
Marie PUPO JEAMMET 63b8984eb2 🐛(domains) fix attemps to send invitations to existing users
Users can only search other users inside their own organization.
This leads to a bug where we try to invite an email already linked to an user
2026-02-06 11:09:43 +00:00
renovate[bot] 0df8f53037 ⬆️(dependencies) update next to v15.5.10 [SECURITY] 2026-02-06 10:41:06 +00:00
Marie PUPO JEAMMET 41084baa0f 🔧(docker) set python version to 3.14.2
set python version to 3.14.2 to match dependencies' expectations
2026-02-05 17:12:34 +00:00
renovate[bot] 32de0b9221 ⬆️(dependencies) update python dependencies 2026-02-05 17:12:34 +00:00
renovate[bot] 1384640a3a ⬆️(dependencies) update django to v5.2.11 [SECURITY] 2026-02-03 21:00:40 +00:00
elvoisin 569aff05a1 (front) add show invitations mails domains access (#1040)
* (front) add show invitations mails domains access

add show invitations to mails domains access

* (front) delete invitations mails domains access

add delete button for delete invitations to mails domains access
2026-02-03 16:06:18 +01:00
Quentin BEY b13f4db536 🔧(actions) migrate from pip to uv
Migrate usage of pip to uv in github actions. How python is setup is
also changed. Doing like this, we will just have to upgrade the python
version requirement in the pyproject file
2026-01-29 23:25:51 +01:00
Quentin BEY 8ace3099b9 🔧(backend) ignore .venv in compile messages command
We have to change the ignored files used in the compoilemessages
command. uv is using .venv directory and not venv
2026-01-29 23:17:03 +01:00
Quentin BEY 38d2125ecf 🏗️(core) migrate from pip to uv
We want to migrate our projects from pip to uv to take the benefits of
the lock file and have reproducible installations.
A first uv.lock file is comitted and the Dockerfile and compose are
modified to work with uv
2026-01-29 23:16:16 +01:00
Marie PUPO JEAMMET 6ae195b90c 🔥(cleanup) remove comment from permissions file
remote remaining commented line from previous PR's tests
2026-01-28 14:49:31 +00:00
Marie PUPO JEAMMET be64abb22f (invitations) can delete domain invitations
add delete method to domain viewset
2026-01-28 14:49:31 +00:00
mjeammet ca56eb0cac 🌐(i18n) update translated strings
Update translated files with new translations
2026-01-26 11:30:57 +00:00
Marie PUPO JEAMMET 99ca34719f 🔖(patch) release version 1.22.2
Update all version files and changelog for patch release.
2026-01-26 11:30:57 +00:00
renovate[bot] 014fb62f95 ⬆️(dependencies) update lodash to v4.17.23 [SECURITY] 2026-01-26 10:48:52 +00:00
Marie PUPO JEAMMET 99433a6722 🐛(aliases) alias destination can be devnull@devnull
devnull@devnull is not considered a valid email address by django's
EmailFieldValidator but it's a special address in dimail's config.

Make "destination" a CharField instead of an EmailField to replace
validator and add devnull to allowlist.
2026-01-23 17:56:53 +00:00
Marie PUPO JEAMMET 5ebc88bcff 🔖(patch) release version 1.22.1
Update all version files and changelog for patch release.
2026-01-22 11:10:44 +00:00
Quentin BEY a65e61bd96 🔒️(organization) the first user is not admin
The first user of a organization is probably not an admin.
This was implemented for first tests but for now it's more
a security issue than something helpful.

FIXES #775
2026-01-22 11:26:58 +01:00
Marie PUPO JEAMMET b8beb56135 🐛(admin) fix broken alias import
fix admin action to sync aliases of a given domain
2026-01-21 17:30:30 +00:00
mjeammet 37e5b5b346 🌐(i18n) update translated strings
Update translated files with new translations
2026-01-20 12:29:49 +00:00
Marie PUPO JEAMMET 31201fbd59 🔖(minor) release version 1.22.0
Update all version files and changelog for minor release.
2026-01-20 12:29:49 +00:00
elvoisin eb0683ffe0 (front) create, manage & delete aliases (#1013)
* (front) add aliases

add list view aliases + creation aliases

* (front) add delete alias

add modale to delete aliases

* 🐛(react-query) remove onMutateResult from mutation callbacks

remove onMutateResult from mutation callbacks
2026-01-19 17:04:57 +01:00
Marie PUPO JEAMMET 54219d25b8 (tests) update auth-related tests after bumping django-lasuite
update the expected number of queries in authentication-related tests
due to bumping django-lasuite to 0.0.22
2026-01-19 10:09:29 +00:00
renovate[bot] 89e7703d53 ⬆️(dependencies) update python dependencies
keep django < 6.0 for compatibility with django-celery-beat
See https://github.com/celery/django-celery-beat/issues/977
2026-01-19 10:09:29 +00:00
Marie PUPO JEAMMET 7d44e54913 (admin) manage aliases from admin
manage aliases from admin, including alias creation,
deletion and sync
2026-01-16 15:19:37 +00:00
Marie PUPO JEAMMET 01583ba94f 🐛(aliases) sort aliases by local part
sort aliases by local part to ease front-end interface
2026-01-16 15:19:37 +00:00
Marie PUPO JEAMMET 5feee53bdd 🔒️(security) upgrade python version to fix vulnerability
Vulnerability in jaraco.context caused security issue
in setuptools and python3. change python version to fix
see https://github.com/advisories/GHSA-58pv-8j8x-9vj2
2026-01-16 11:23:55 +00:00
Marie PUPO JEAMMET 9c62efc9f8 🐛(dimail) authorize alias and mailbox with same local part
a mailbox and alias can coexist despite having the same local part
2026-01-15 10:29:49 +00:00
Marie PUPO JEAMMET d2ef9e0beb 🐛(dimail) ignore oxadmin mailboxes when importing mailboxes
oxadmin mailbox are technical mailboxes used by dimail. It should
not be imported when importing mailboxes from dimail.
2026-01-15 10:29:49 +00:00
Marie PUPO JEAMMET bc1cbef168 (aliases) delete all aliases of a given local part
added a bulk delete method for aliases, when filtering on local part
this is convenient when in need to delete the local part and all its
destinations in a single call
2026-01-12 14:31:53 +00:00
Marie PUPO JEAMMET 8ab1b2e2ef 🔧(settings) separate dimail container from app-dev
separate dimail container from app-dev to avoid setting
dimail container everytime we run the test suite.
2026-01-12 14:31:53 +00:00
Marie PUPO JEAMMET f498e3b6d2 ♻️(lint) mark a few dimail methods as private
mark a few dimail methods as private, to calm linter
2026-01-12 14:31:53 +00:00
Marie PUPO JEAMMET e47573e22f (aliases) delete aliases
delete a single alias using its pk
2026-01-12 14:31:53 +00:00
Sabrina Demagny c7aab6f5b5 🔥(plugins) remove CommuneCreation plugin
Remove the French communes auto-provisioning plugin that handled
SIRET lookup, DNS setup, and MailDomain creation.
2026-01-09 09:38:01 +01:00
renovate[bot] 2144ad5da1 ⬆️(dependencies) update next to v15.4.10 [SECURITY] 2025-12-15 14:43:33 +00:00
renovate[bot] f00deeebe9 ⬆️(dependencies) update next to v15.4.9 [SECURITY] 2025-12-12 12:41:33 +00:00
Marie PUPO JEAMMET 30aea9b350 🔖(minor) release version 1.21.0
Update all version files and changelog for minor release.
2025-12-05 16:34:06 +00:00
Marie PUPO JEAMMET e01b422c13 (mailboxes) check imported mailboxes don't clash with existing alias
check that dimail imported mailboxes don't use the same username
as existing aliases.
2025-12-05 15:03:17 +00:00
Marie PUPO JEAMMET 040f949e5e (aliases) import existing aliases from dimail
import existing aliases from dimail, making sure usernames don't clash with
existing mailboxes. Convenient when people fall out of sync
with dimail or for domains partially operated outside people.
2025-12-05 15:03:17 +00:00
renovate[bot] befcecf33c ⬆️(dependencies) update django to v5.2.9 [SECURITY] 2025-12-04 13:21:36 +00:00
renovate[bot] c5edabf3b0 ⬆️(dependencies) update next to v15.4.8 [SECURITY] 2025-12-04 09:36:36 +00:00
Marie PUPO JEAMMET 3a2b5a6428 🛂(permissions) return 404 to users with no domain access
users having no domain access should be forbidden to know the domain is
managed in people. To this effect, 403_FORBIDDEN responses have been
replaced by 404_NOT_FOUND.
2025-11-21 17:04:20 +00:00
Marie PUPO JEAMMET 15337a2226 (aliases) warn that domain is out of sync when unexpected 404
deleting an alias should trigger a request to dimail. if dimail's
response if a 404, it means people and dimail are out of sync with
each other. log error and warn user
2025-11-20 11:24:59 +00:00
Marie PUPO JEAMMET 0f7e312eb6 (mailboxes) cannot create mailbox with same local part as alias
should not be able to create a mailbox having the same local part as an alias
2025-11-20 11:24:59 +00:00
Marie PUPO JEAMMET 23561cd0e0 🔥(sops) remove obsolete sops file
remove obsolete sops file
2025-11-20 11:24:59 +00:00
Marie PUPO JEAMMET 53d0336755 (aliases) delete aliases
add feature to delete aliases
2025-11-20 11:24:59 +00:00
Marie PUPO JEAMMET b79e12b4be (aliases) list aliases
Can GET a list of all aliases of a domain
2025-11-20 11:24:59 +00:00
Marie PUPO JEAMMET c237bb4b10 (aliases) create aliases
allow domain managers to create aliases on their domain
2025-11-20 11:24:59 +00:00
renovate[bot] 64068efff4 ⬆️(dependencies) update django to v5.2.8 [SECURITY] 2025-11-06 00:11:45 +01:00
renovate[bot] 6866babb32 ⬆️(dependencies) update Brotli to v1.2.0 [SECURITY] 2025-11-06 00:11:04 +01:00
Quentin BEY 80caa062e9 🐛(mail-domains) fix zod resolver use in forms
References:
https://github.com/react-hook-form/react-hook-form/issues/12816
https://github.com/react-hook-form/resolvers/issues/768
2025-11-04 16:10:27 +01:00
Quentin BEY 513dbe6f83 💚(frontend) fix using side effects within waitFor callback
New lint error detected.
2025-11-04 16:10:27 +01:00
Marie PUPO JEAMMET 490fdd2ed4 🐛(tests) fix lint-front and test-front errors
fix dependency import error and failing e2e tests due to
@tanstack upgrade
2025-11-04 16:10:27 +01:00
Marie PUPO JEAMMET fa7d24545f 🔥(config) remove empty duplicate yarn.lock
remove duplicate yarn.lock from previous config attempts
2025-11-04 16:10:27 +01:00
Marie PUPO JEAMMET e5938e144e 📌(frontend) hold next to v15.4.7
Avoid renovate to bump next to its latest major version.
2025-11-04 16:10:27 +01:00
Marie PUPO JEAMMET 39af4313cf 🚨(frontend) adapt signatures to @tanstack/react-query to >5.90
Recent upgrade of @tanstack/react-query to
version >5.90 introduced a breaking change in the
onSuccess and onError callback signatures for
the useMutation hook.
The context parameter has been replaced with an
onMutateResult parameter, which provides
information about the result of the
onMutate callback.

Shamelessly copied from https://github.com/suitenumerique/docs/pull/1375/commits
2025-11-04 16:10:27 +01:00
Marie PUPO JEAMMET abfe32569e ⬇️(dependencies) hold cunningham to v3.2.3
Cunningham v4.0.0 is not yet compatible with lasuite ui-kit
2025-11-04 16:10:27 +01:00
renovate[bot] 964981bbfb ⬆️(dependencies) update js dependencies 2025-11-04 16:10:27 +01:00
mjeammet 33faa5f224 🌐(i18n) update translated strings
Update translated files with new translations
2025-10-22 10:18:21 +00:00
Marie PUPO JEAMMET 99967b450e 🔖(minor) release version 1.20.0
Update all version files and changelog for minor release.
2025-10-22 10:18:21 +00:00
Marie PUPO JEAMMET 71a7bf688f 🐛(mailbox) fix case-sensitive duplicate display names
uniqueness on first name + last name was case-sensitive, which allowed
duplicates
2025-10-22 08:22:48 +00:00
Marie PUPO JEAMMET 302671bc69 ⬆️(dependencies) upgrade node to 22
node 18 reached end-of-life and is now unsupported. we jump straight
to 22 as recommended here
https://nodejs.org/en/blog/announcements/node-18-eol-support
2025-10-21 11:52:51 +00:00
Marie PUPO JEAMMET 4262f469d6 ♻️(tests) refacto dimail tests with fixtures
test_api_mailboxes_create exceeded 1000 lines. By using fixtures, we can
at least factorize dimail response when token is ok or mailbox_data sent
to our API when creating a mailbox.
2025-10-14 10:00:28 +00:00
Marie PUPO JEAMMET b24cb23a83 (models) impose uniqueness on display name, to match ox's constraint
OpenXchange's primary key is display name (= first name + last name).
It must be unique in the domain's context. We don't have context info
but we can impose uniqueness by domain.
2025-10-14 10:00:28 +00:00
Marie PUPO JEAMMET 608f8c6988 🐛(dimail) grab duplicate displayname error
OpenXchange's primary key is display name (= first name + last name).
In absence of clear error message from dimail (yet), we catch errors 500 and
check if they're not due to the display name already existing in the context
2025-10-14 10:00:28 +00:00
renovate[bot] 2ddfef59d8 ⬆️(dependencies) update next to v15.4.7 [SECURITY] 2025-10-13 10:01:30 +00:00
renovate[bot] 6b8cb16bd5 ⬆️(dependencies) update django to v5.2.7 [SECURITY] 2025-10-13 08:50:32 +00:00
Marie PUPO JEAMMET 8b89a1112d 📝(release) update release.md
Update documentation on release
2025-09-23 18:56:56 +02:00
Laurent Bossavit 230ff21220 (mailbox) synchronize password of newly created mailbox with Dimail's
When using La Régie as Identity Provider this allows signing in.
2025-09-23 17:40:08 +02:00
Quentin BEY a37c2a8e83 🔒️(frontend) update alpine packages in production image
Force an update of installed package in the image used for
the frontend in production.
2025-09-23 14:31:13 +02:00
mjeammet 80da8bea74 🌐(i18n) update translated strings
Update translated files with new translations
2025-09-19 17:04:02 +02:00
Marie PUPO JEAMMET 76f4bf36c7 🔖(patch) release version 1.19.1
Update all version files and changelog for patch release.
2025-09-19 17:04:02 +02:00
Eléonore Voisin 78a5d907ca 🐛(fix) add enabled update your mailbox
can update your mailbox as viewer
2025-09-19 16:45:40 +02:00
Marie PUPO JEAMMET 0ad60ab292 ⬆️(backend) bump Django to 5.2.6
Bump Django version to 5.2.6, fixing CVE-2025-57833. See
https://www.djangoproject.com/weblog/2025/sep/03/security-releases/
2025-09-15 14:36:26 +02:00
Quentin BEY ac443d3b6f 🔒️(all) refactor Docker Hub login to use official GitHub actions
Replace custom Docker Hub authentication with standard, secure,
official GitHub actions for improved security and maintainability.

Uses officially supported actions that follow security best practices
and receive regular updates from GitHub.

Avoid unsecure handling of GitHub secrets.

Thanks to @lebaudantoine
2025-09-05 14:55:17 +02:00
Quentin BEY 48ff0e9f3a 🔧(ci) always run all git-lint steps
git-lint steps are independant and we would like to have all checks at
once. Using the `if: always()` instruction should ensure all steps
should be run event if the previous fails.

thanks @lunika
2025-09-05 14:44:20 +02:00
mjeammet 675e719a22 🌐(i18n) update translated strings
Update translated files with new translations
2025-09-03 11:52:42 +02:00
mjeammet 0875fae87b 🌐(i18n) update translated strings
Update translated files with new translations
2025-09-03 11:52:42 +02:00
Marie PUPO JEAMMET 668a296142 🔖(minor) release version 1.19.0
Update all version files and changelog for minor release.
2025-09-03 11:52:42 +02:00
elvoisin f1892b7049 (front) add modal update mailboxes (#954)
* (front) add modal update mailboxes

add modal update mailboxes

* ️(labels) improve aria-labels on domain panel's buttons

improve descriptions on aria-label on domain panel's buttons

---------

Co-authored-by: Marie PUPO JEAMMET <marie.pupojeammet@numerique.gouv.fr>
2025-09-03 11:02:23 +02:00
Marie PUPO JEAMMET 1bfad507ef (api) retrieve mailboxes
add feature to retrieve mailboxes when having the right access
2025-09-02 13:45:26 +02:00
Marie PUPO JEAMMET 72e73bff45 (api) give update rights to domain viewer on own mailbox
Introduces the notion of self in permissions
allowing a domain viewer to update their own mailbox.
2025-09-02 13:45:26 +02:00
Marie PUPO JEAMMET e45cf8dd8b (api) update mailboxes
Allow update of mailboxes. Secondary email, first and last names can be updated
but not domain or local_part.
2025-09-02 13:45:26 +02:00
Marie PUPO JEAMMET 79f8e5276a 💚(ci) fix changelog CI job
The command git watchanged is deprecated and is flag to be removed soon.
It can be easily replace with git log
2025-08-29 15:32:26 +02:00
Marie PUPO JEAMMET 4f97685204 🐛(admin) fix mailbox import from dimail
importing mailboxes from dimail was broken due to a change of format in dimail's
response.
2025-08-29 11:46:33 +02:00
Marie PUPO JEAMMET 935058582e ⬆️(dependencies) upgrade dimail to 0.5.2
manually upgrade dimail to match prod version
2025-08-29 11:46:33 +02:00
Marie PUPO JEAMMET 636b27321b 🧑‍💻(demo) set invitations for local user
prepare invitations for local user. Upon first connexion
to keycloack, invitations are consumed and dev are ready to go
:)
2025-08-26 17:39:58 +02:00
Marie PUPO JEAMMET 2fdddb3e12 🧑‍💻(demo) add mailboxes to demo
add mailboxes to demo domains.
2025-08-26 17:39:58 +02:00
Quentin BEY f7a97e11e8 🎨(ruff) fix linter error after ruff update
Fix the `make lint` errors after dependencies update.
2025-08-26 11:36:05 +02:00
renovate[bot] a56f86965f ⬆️(dependencies) update python dependencies 2025-08-26 09:23:11 +00:00
Hsiaoming Yang 89674ad718 🎨(joserfc) fix import path of RSAKey
`rfc7518` module will be removed in v1.4.0 of joserfc.
2025-08-26 10:59:35 +02:00
Quentin BEY 4e592382dc 💥(sentry) remove DJANGO_ before Sentry DSN env variable
Other "La Suite" projects don't have the "DJANGO_" prefix, so
we align this project with others.
2025-07-24 23:53:47 +02:00
Marie PUPO JEAMMET 1fdd2e052b 🐛(config) remove default sentry_dsn
set default sentry_dsn to None to avoid all outside
deployments sending infos to our sentry instance.
2025-07-23 12:08:28 +02:00
Quentin BEY 25918830ff ⬆️(django-lasuite) bump to version 0.0.11
This allows to accept JWE from new Proconnect introspection endpoint.
2025-07-09 14:47:55 +02:00
Marie PUPO JEAMMET b47282430e 🔖(patch) release version 1.18.2
Update all version files and changelog for patch release.
2025-07-03 11:28:01 +02:00
Marie PUPO JEAMMET ebd3f467c4 🐛(admin) fix read_only preventing mailbox creation in admin
removed read_only on "local_part" and "domain" which prevented
admin user from creating mailboxes from admin
2025-07-02 18:06:02 +02:00
Marie PUPO JEAMMET 0f4ec63a9f 🐛(webhooks) fix special case for tchap integration
we want to use Tchap integration server for our own preprod
but it doesn't follow the same url naming convention as the
other Tchap instances, so it requires a special treatment.
This quick fix should do it.
2025-07-02 17:52:44 +02:00
Eléonore Voisin fdec3b4196 🐛(front) fix missing pagination mail domains
fix position div ref
2025-07-02 17:38:21 +02:00
Marie PUPO JEAMMET b78723ecce 🔖(patch) release version 1.18.1
Update all version files and changelog for patch release.
2025-07-02 11:29:10 +02:00
Marie PUPO JEAMMET df0cea225c 🐛(changelog) add missing changelog
add missing changelog for consistency
2025-07-02 11:29:10 +02:00
mjeammet 3b73cca2f5 🌐(i18n) update translated strings
Update translated files with new translations
2025-07-02 11:29:10 +02:00
elvoisin 06d4d5c9e8 🐛(front) fix missing pagination mail domains (#946)
fix missing pagination mail domains + mailboxes list
2025-07-02 09:03:11 +02:00
mjeammet 405e5fda90 🌐(i18n) update translated strings
Update translated files with new translations
2025-06-30 15:23:08 +02:00
Marie PUPO JEAMMET 8a434448dd 🔖(minor) release version 1.18.0
Update all version files and changelog for minor release.
2025-06-30 15:23:08 +02:00
Marie PUPO JEAMMET 5b5bd312b4 🔧(backend) fix Redis dependency to <6.0.0
Pin the redis dependency to <6.0.0 to fix ResolutionImpossible errors
in future renovate PRs.
See https://github.com/suitenumerique/meet/pull/553
2025-06-30 11:57:28 +02:00
Marie PUPO JEAMMET 99bf301a9c ⬆️(dependencies) update python dependencies
manually upgrade dependencies
2025-06-30 11:57:28 +02:00
Marie PUPO JEAMMET 3dc11c9b52 🐛(webhook) search existing Matrix user before inviting
fix wrong formatting of user id + now searches for existing Matrix
account of the user before inviting them to webhook room.
2025-06-27 16:34:18 +02:00
Marie PUPO JEAMMET 5327baacbd 🔧(settings) rename matrix bot token variable
rename "tchap access token" to "matrix bot access token" to
fit general uses
2025-06-27 16:34:18 +02:00
Manuel Raynaud 709628a26a 🔧(back) remove usage of deprecated db engine (#937)
The db engine postgresql_psycopg2 does not exists anymore in django but
for BC compat it is possible to use it in the configuration and it is
replace by postgresql at runtime. We changed this settings to use the
good one.
2025-06-27 15:18:27 +02:00
Marie PUPO JEAMMET d2bf267160 🐛(webhooks) fix InsecureRequestWarning from webhook calls
Matrix webhook calls were unverified and caused a security warning.
Setting option "verify" to "True" on related calls should help.
2025-06-25 19:54:45 +02:00
Marie PUPO JEAMMET 10dcdfc8c2 🐛(webhook) fix error raised when no secret
Our Tchap webhook contains no secret (as a dedicated access token is provided)
This lead to an error upon trying to join rooms.
2025-06-23 17:28:41 +02:00
elvoisin f30398fbc7 🐛(front) fix button add mail domain (#932)
fix button rules + fix bad wording
2025-06-23 16:27:52 +02:00
Marie PUPO JEAMMET cc39ed5298 (teams) add matrix webhook for teams
A webhook to invite/kick team members to a matrix room.
2025-06-21 00:15:16 +02:00
Marie PUPO JEAMMET 7bebf13d88 🐛(domains) reduce logs around domain invitations
reduce logs and add tests around domain invitations
2025-06-21 00:15:16 +02:00
Quentin BEY e64a34f167 🧑‍💻(keycloak) command to add new client
This introduce a command to create a new client into the "people" realm.
This could be use to create a specific client to test the resource
server mode on a local deployment:
 - run the people stack
 - add the new client, let say, for docs
 - configure the people backend for token introspection
 - make calls from docs backend to people's backend

The new client is not mandatory because the same client could be used
everywhere but this would not demonstrate the fact the introspection
works in a real world configuration.
2025-06-21 00:15:16 +02:00
Quentin BEY 3379d6d499 🔧(git) set LF line endings for all text files
Windows users are by default using CRLF line endings,
which can cause issues with some tools and
environments. This commit sets the `.gitattributes`
file to enforce LF line endings for all text
files in the repository.

Based on the same commit on docs
2025-06-21 00:15:16 +02:00
Quentin BEY 213656fc2e 🧑‍💻(docker) split frontend to another file
This commit aims at improving the user experience:
- Use a dedicated `Dockerfile` for the frontend
- Run the backend and frontend in "watch"/dev mode in Docker
- Do not start all Docker instances for small tasks
2025-06-21 00:15:16 +02:00
Quentin BEY 4dfd682cb6 (resource-server) add SCIM /Me endpoint
This provide a "self-care" SCIM endpoint, authenticated with OIDC token
introspection. This endpoint will be use by services to fetch the user's
team list.

We chose to use the SCIM format (even if this is not a SCIM context) to
make it easier to understand/maintain/plug.
2025-06-21 00:15:16 +02:00
mjeammet 160c45bf35 🌐(i18n) update translated strings
Update translated files with new translations
2025-06-11 11:40:56 +02:00
Marie PUPO JEAMMET 3fdd8a230c 🔖(minor) release version 1.17.0
Update all version files and changelog for minor release.
2025-06-11 11:40:56 +02:00
renovate[bot] b47246826e ⬆️(dependencies) manually update python dependencies
Many of our dependencies still aren't compatible with Redis 6.
This results in a partial upgrade of our dependencies.
2025-06-10 21:59:41 +02:00
Marie PUPO JEAMMET 5429354261 🐛(config) whitelist kube pod
Whitelist our pod's IP address. Based on Visio's PR
https://github.com/suitenumerique/meet/pull/95
2025-06-10 18:12:02 +02:00
Marie PUPO JEAMMET 86c98cc426 🧑‍💻(dimail) modify makefile to setup dimail container upon running demo
Setup dimail container upon running demo so that it's always in sync.
2025-06-10 16:52:04 +02:00
Marie PUPO JEAMMET 0bbce9ffc8 🔥(dimail) remove obsolete user and allow creation in dimail setup
Remove obsolete duplication to dimail database.
See PR https://github.com/suitenumerique/people/pull/886 for more context.
2025-06-10 16:52:04 +02:00
Marie PUPO JEAMMET ce66645294 🐛(demo) fix domains names
Mailbox creation was broken in dev because of wrong format of domains names
2025-06-10 16:52:04 +02:00
Eléonore Voisin 485eb88dd1 (frontend) add crisp script
add crisp chatbox to global layout
2025-06-10 16:13:24 +02:00
renovate[bot] a8b08c4b6d ⬆️(dependencies) update requests to v2.32.4 [SECURITY] 2025-06-10 15:47:00 +02:00
renovate[bot] 23f5a13ccc ⬆️(dependencies) update django to v5.2.2 [SECURITY] 2025-06-10 12:20:27 +00:00
elvoisin 1245c54c61 ️(fix) add error when mailbox create failed (#915)
fix toast error when mailbox create failed
2025-06-10 12:19:31 +00:00
Quentin BEY 95f63fa56d 🔒️(frontend) hide Nginx server version in error responses
Remove version disclosure in /assets/ error pages identified by security
auditor to prevent information leakage vulnerability.
2025-06-05 19:27:24 +02:00
Marie PUPO JEAMMET 4c3891047b (mailboxes) add a button to reset password on enabled mailboxes
Domain admins can now send requests to reset password for any mailbox
on their domains.
2025-05-20 16:45:14 +02:00
Marie PUPO JEAMMET fce9b1e490 🐛(dimail) fix broken auth while resetting passwords
Dimail client's "reset password" method was using basic auth while
dimail expects a token for this endpoint. Fixed it.
2025-05-20 16:45:14 +02:00
Marie PUPO JEAMMET 83bec33bdb ✏️(typo) fix typos
Did you know ? "Successful" actually takes two esses.
2025-05-20 16:45:14 +02:00
Quentin BEY 8fed2606d6 🧑‍💻(frontend) add makefile lint --fix
Add a Makefile command to easily run the automatic fixup for frontend
files.
2025-05-20 15:00:14 +02:00
Marie PUPO JEAMMET c7eb86eaa9 ⬆️(dependencies) manually upgrade python dependencies
Renovate seems to be in a weird place dealing with conflicting versions.
This PR handles the non-problematic upgrades
2025-05-16 19:32:15 +02:00
qbey be1513e106 🌐(i18n) update translated strings
Update translated files with new translations
2025-05-16 11:22:18 +00:00
Quentin BEY 8e85f303ec 🌐(frontend) update some translated messages
Slight cleanup of translations.
2025-05-16 11:55:32 +02:00
Quentin BEY ee564ff6ba 🎨(front) fix mail domain list display
The first implementation was using `div` instead of the proper
components.
2025-05-16 11:55:32 +02:00
renovate[bot] b0355059b7 ⬆️(dependencies) update js dependencies 2025-05-16 11:41:03 +02:00
Marie PUPO JEAMMET 6e792986be (admin) send pending mailboxes from admin
Provides an admin action to send all pending mailboxes for an active domain.
This allows quick fixes when mailboxes fell out of sync.
2025-05-16 11:30:46 +02:00
Quentin BEY fe9fb67fed 🔒️(docker) patch libxml to address CVE
Trivy scan detects some issue:
┌─────────┬────────────────┬──────────┬────────┬───────────────────┬───────────────┬───────────────────────────────────────────────────────────┐
│ Library │ Vulnerability  │ Severity │ Status │ Installed Version │ Fixed Version │                           Title                           │
├─────────┼────────────────┼──────────┼────────┼───────────────────┼───────────────┼───────────────────────────────────────────────────────────┤
│ libxml2 │ CVE-2025-32414 │ HIGH     │ fixed  │ 2.13.4-r5         │ 2.13.4-r6     │ libxml2: Out-of-Bounds Read in libxml2                    │
│         │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2025-32414                │
│         ├────────────────┤          │        │                   │               ├───────────────────────────────────────────────────────────┤
│         │ CVE-2025-32415 │          │        │                   │               │ libxml2: Out-of-bounds Read in xmlSchemaIDCFillNodeTables │
│         │                │          │        │                   │               │ https://avd.aquasec.com/nvd/cve-2025-32415                │
└─────────┴────────────────┴──────────┴────────┴───────────────────┴───────────────┴───────────────────────────────────────────────────────────┘
2025-05-16 10:03:52 +02:00
Quentin BEY 91fbef9066 🎨(front) rewrite the team main page
The first rewrite I made was using `div` instead of the proper
components.
2025-05-14 18:46:49 +02:00
Quentin BEY 4f3c9abe62 🐛(front) improve domain "return" button
The button to return to domain list was reloading the whole page which
was quite long (and somehow failing on staging).
2025-05-14 17:57:29 +02:00
Quentin BEY bd43e4620d 💄(teams) update team list page to match new UI
This is an attempt to quick fix the team page to match the new UI.
2025-05-14 17:57:29 +02:00
Quentin BEY 8c67d4a004 (e2e) add mailboxe (dis/en)able check
This provides a new test to check the action on the mailbox item.
For now we can only enable or disable a mailbox.

We need to create the mailbox in the test, so it exists on Dimail side.
2025-05-14 17:46:26 +02:00
Quentin BEY 78cb3e693c 💚(e2e) remove useless test on "tabs"
This is supposed to be a validation test on accessibility but in fact
it's just a flaky test which does not provide any information. We need
to replace this with something smarter.
2025-05-14 17:46:26 +02:00
Quentin BEY cec8e87edd 🐛(front-mail) dynamically display org name
The organization is not always DINUM.
2025-05-14 17:46:26 +02:00
Quentin BEY 35a700d522 💄(cunningham) fix style for display
Some CSS were not found (like color gold-500).
2025-05-14 17:46:26 +02:00
Quentin BEY c786ddbb82 🐛(frontend) re-enable mailbox actions
This code was commented but seems to work properly.
2025-05-14 17:46:26 +02:00
Quentin BEY cb198a9d04 💩(frontend) restore user name / org header
This was added previously and while the organization is not displayed
elsewhere it's better to keep the information displayed somewhere.
2025-05-14 17:43:37 +02:00
Quentin BEY 3d9645b561 ⚰️(i18n) utils module is not used anymore
The use of this module has been removed during the UI refacto.
2025-05-14 17:43:37 +02:00
Quentin BEY 21cbeded18 (frontend) fix end-to-end tests after refacto
Some test were broken other were flaky: fix them.
All tests are not worthy but since it was easy to fix, we keep them
until we write better tests...
2025-05-14 17:43:37 +02:00
Quentin BEY f0c609ef0b 🐛(frontend) fix create team button
The button to create a new team was not displayed properly anymore.
2025-05-14 17:43:37 +02:00
Quentin BEY aa3d90b686 🐛(frontend) fix browser language detection
The `Default language` E2E test detected the browser language was not
automatically detected when user does not have any cookie, it was always
falling back on the defaut language (en).
2025-05-14 17:43:37 +02:00
Quentin BEY 4a08a9ec92 ⚰️(front) remove unused code after UI v2
The linter was unhappy, so I removed the unused variables or code.
2025-05-14 17:43:37 +02:00
Quentin BEY 5544a40f5f 🎨(front) fix linter issues with --fix
Simply run the lint command with "autofix" to format files.
2025-05-14 17:43:37 +02:00
Eléonore Voisin e274c309cd 🎨(frontend) global UI rewrite to match new design
This commit is the result of several squashed commits which were
complicated to disjoin.
This rewrites the base UI, and the mail management interfaces.
2025-05-14 17:41:30 +02:00
renovate[bot] 560998083d ⬆️(dependencies) update django to v5.2.1 [SECURITY] 2025-05-08 18:32:32 +02:00
Marie PUPO JEAMMET 2d56c57102 (dimail) add reset password method
allow domain owner and admins to reset password for a mailbox
they manage. The request is sent to dimail, which responds with
a new randomly generated password. This new password is sent to
secondary email.
2025-05-05 17:28:00 +02:00
Benoit Serrano b5d86967ff 🐛(docker) fix dockerize no matching manifest for linux/arm64/v8 error
fixing docker config for linux/arm64/v8 error
2025-05-05 17:28:00 +02:00
Marie PUPO JEAMMET 08559d856d ⬆️(dimail) update dimail version to 2.11
update dimail version to last production-ready version.
2025-05-05 17:28:00 +02:00
Marie PUPO JEAMMET 8b17a5470d 🔖(changelog) fix changelog
Fixing changelog after having forgotten to update it in last release (1.16.0)
2025-05-05 12:39:34 +02:00
mjeammet 141c4e7f61 🌐(i18n) update translated strings
Update translated files with new translations
2025-05-05 11:32:27 +02:00
Marie PUPO JEAMMET a5473f62b7 🔖(minor) release version 1.16.0
Update all version files and changelog for minor release.
2025-05-05 11:32:27 +02:00
Quentin BEY 889291c7f3 🔒️(drf) disable browsable HTML API renderer
The `BrowsableAPIRenderer` generates a form to test POST/PUT/... actions
and fill the FK fields with unfiltered data. This issue has been spoted
on visio and fixed https://github.com/suitenumerique/meet/pull/508
2025-04-30 15:58:21 +02:00
Quentin BEY a8d20bacb0 ️(back) use redis as session backend in dev
We want to persist the session during development. Otherwise the session
is reset everytime the server is restart. This behavior make developing
bot a front and back feature a nigthmare, we spend our time login again
and again.

Shamelessly copy/pasted from @lunika 's work
suitenumerique/docs@007854a
2025-04-30 15:11:40 +02:00
renovate[bot] c4a81cf76a ⬆️(dependencies) update python dependencies 2025-04-30 10:49:16 +02:00
Quentin BEY 0a241f0e03 🔧(sentry) add Celery beat task integration
This should provide "cron" monitoring in Sentry.
2025-04-28 15:51:34 +02:00
Laurent Bossavit b389927653 📝(security) add a basic security disclosure policy
This is copied from Docs with only minor changes.
2025-04-24 14:05:22 +02:00
Marie PUPO JEAMMET 056a4bd7ac 🛂(dimail) simplify interop with dimail
In this commit, we stop creating /users and /allows in dimail
for our dbs to be in sync. People with stop impersonating users
in dimail and will create mailboxes using its own credentials.
2025-04-23 16:24:53 +02:00
Quentin BEY 6721328b2d ⬆️(django-lasuite) bump version to v0.0.7
This fixes the userinfo OIDC endpoint format autodetection.
2025-04-23 10:23:09 +02:00
Quentin BEY ab5d8c74d8 (e2e) fix keycloak user email address
Django >= 5.2 add a verification on email address
2025-04-22 17:59:55 +02:00
Quentin BEY 4c14f967b6 (backend) fix test after dependencies update
The queries needs update to manage save/release in database, this should
be improved, but considered ok for now.
2025-04-22 17:59:55 +02:00
renovate[bot] b42cd483c6 ⬆️(dependencies) update python dependencies 2025-04-22 17:59:55 +02:00
Marie PUPO JEAMMET 3735b699cc 🔧(renovate) add dimail-api to renovate scope
Dimail-api is currently outside of renovate' scope, which resulted in us having
to check and update dimail's image manually or, if failing to, discovering new
behaviors by chance or by errors in production. This should fix it.
2025-04-22 14:31:50 +02:00
Quentin BEY 0220875c70 ⬆️(django-lasuite) bump to version 0.0.5
Bump the lib to the latest version:
- update the post_get_or_create_user method signature
- allow silent login for OIDC (will require frontend implementation)
2025-04-10 15:00:03 +02:00
Sabrina Demagny 7a1fc6b626 (mailbox) remove secondary email as required field
The secondary email address is no longer required for all creation
processes and we should not force the user to provide and store an
insecure email address.
2025-04-10 12:59:16 +02:00
Sabrina Demagny 99d7b23dc9 🐛(core) fix AccountService api_key field declaration
Override save is a better way to auto generate api_key if
it is not set.
Default with random secret generate a new migration each time we
run `make makemigrations`.
2025-04-10 10:24:08 +02:00
renovate[bot] 1a52cd63bf ⬆️(dependencies) update js dependencies 2025-04-09 12:12:01 +02:00
Sabrina Demagny 8691f1846d 📝(scripts) enhance release scripts instructions
Add information about settings and helm chart to configure
Add more details about translations PR autogenerated
2025-04-08 22:00:41 +02:00
Sabrina Demagny edbf77c525 💄(domain) enhance admin action label to import mailboxes
So far, "Synchronise from dimail" only import missing mailboxes
from dimail, so this label needs to be more explicit.
2025-04-08 21:39:16 +02:00
Quentin BEY 140d099fce ⬆️(backend) bump django-lasuite to v0.0.2
This will allow the introspected token to not contain the `iss` claim.
2025-04-07 13:55:19 +02:00
sdemagny 133688324b 🌐(i18n) update translated strings
Update translated files with new translations
2025-04-04 17:46:21 +02:00
Sabrina Demagny a7b3cd42bc 🔖(minor) release version 1.15.0
Update all version files and changelog for minor release.
2025-04-04 17:46:21 +02:00
Jacques ROUSSEL ceebf8f7aa 🐛(ci) remove path to trigger relaese helm chart
We had an issue with the automatic helm chart releaser so we decide to
trigger the job on every merge.
2025-04-04 17:18:20 +02:00
Sabrina Demagny 8ef2cc9a37 🧱(helm) add la-suite ingress path
The route was added but not declared in the ingress.
2025-04-04 15:02:20 +02:00
Quentin BEY e2d362bc77 (backend) add django-lasuite dependency
Use the OIDC backends from the new library.
2025-04-04 09:57:12 +02:00
Sabrina Demagny 594d3af0d0 (plugins) add endpoint to list SIRET of active organizations
Allow access to AccountService with right scope to list
SIRET of active communes
2025-04-04 08:47:24 +02:00
Sabrina Demagny 855e20d407 (core) create AccountServiceAuthentication backend
Backend authentication with API Key to AccountService
2025-04-04 08:47:24 +02:00
Sabrina Demagny f60bfc2676 (core) create AccountService model
Create new model to allow access of some API
endpoints with API Key authentification.
Scopes will allow to define permission access on those
endpoints.
2025-04-04 08:47:24 +02:00
Marie PUPO JEAMMET b4de7fda92 🔒️(users) restrict listable users to same organization
This is a quick fix to a security issue. Previously, any user could
list all users. Now /users/ endpoint only lists users from same
organization.
2025-04-03 16:18:25 +02:00
Quentin BEY a009f3ccb7 🐛(plugin) allow simple application name
This allows to use the application name, instead of the full path to the
application configuration in the INSTALLED_PLUGINS setting.
2025-04-03 15:17:53 +02:00
Marie PUPO JEAMMET 2f1843e0e8 🐛(stats) rename stat for clarity
Public statistics on domains was modified to count only enabled
domains. Modify stat name to reflect change.
2025-04-03 14:58:07 +02:00
Quentin BEY 3a044e6b02 📝(helm) update missing documentation
Seems like the readme was not updated after adding the celery beat
worker configuration.
2025-04-03 10:33:47 +02:00
Quentin BEY 7c569a3ca3 🧱(helm) disable createsuperuser job by setting
This provides the way to disable the admin user creation at each
deployment. In production we don't want to persist a generic admin user:
it should be created once, at first deployment then replaced by
nominative accounts.
2025-04-03 10:33:23 +02:00
Quentin BEY e23d236614 (pytest) fail on tests external calls
The backend tests must not try to call the real world.
2025-04-03 09:39:15 +02:00
renovate[bot] 61c3b6ac6b ⬆️(dependencies) update django to v5.1.8 [SECURITY] 2025-04-03 07:47:33 +02:00
Quentin BEY 1eb9dffa48 🐛(contacts) add missing select_related
The new DRF version (3.16.0) adds a check on unique together and needs
more fields to be loaded. To prevent an extra query, we select the owner
value in the DB query.
2025-04-01 10:58:49 +02:00
renovate[bot] d0854851a2 ⬆️(dependencies) update python dependencies 2025-04-01 10:58:49 +02:00
renovate[bot] 6eae92d9e5 ⬆️(dependencies) update js dependencies 2025-03-31 09:39:23 +02:00
Jacques ROUSSEL b02146e4eb 🐛(ci) use github action for argocd webhook notification
In order to refactor this notification between alls projetcs, we
chooseto use a custom github action
2025-03-28 16:32:10 +01:00
Quentin BEY dd43483ce6 🔒️(passwords) add validators for production
This enabled various password validators to enforce password complexity.
2025-03-28 15:43:45 +01:00
Sabrina Demagny 838d1267b2 (domains) allow to re-run check on failed domain
In use we realize that it is also necessary to be able
to re-run dimail check on domain failed
2025-03-28 15:03:15 +01:00
Quentin BEY fbe3aa54d0 🐛(ci) use sha256 to sign argocd webhook call
The argocd webhook call needs now to use sha256 digest now to sign

Copy from docs project commit by @lunika
2025-03-28 11:09:04 +01:00
Sabrina Demagny e4e9a121a4 (organization) add is_active field
Add flag to indicate whether the organization is active.
Prepare organizations provisioning. The organization
will be created with this flag set to False and will
become active when the first user is associated with it.
2025-03-27 18:34:09 +01:00
Sabrina Demagny 3173e096d9 🐛(dimail) enhance sentry log for dimail error
Remove duplicate sentry log and fix failure if response content
has an unexpected format
2025-03-27 18:25:24 +01:00
Marie PUPO JEAMMET 4420bab073 🐛(demo) fix missing support_email field
"support_email" field was missing for all domains created in demo.
this lead to "make demo" and "setup_dimail_db" commands to fail.
2025-03-27 18:06:36 +01:00
Marie PUPO JEAMMET 8cbedeb76e ♻️(dimail) refacto setup_dimail_db to call dimail client
Management command "setup_dimail_db" called dimail directly, thus
creating duplicated code. It now calls "create_domain" and "create_allow"
methods from DimailAPIClient (create_user is left unchanged to create
special users such as dimail admin or people)
2025-03-27 18:06:36 +01:00
Quentin BEY 28fdee868d ♻️(plugins) rewrite plugin system as django app
This allow more flexibility around the installed plugins, this will
allow to add models in plugins if needed.
2025-03-26 19:56:23 +01:00
Quentin BEY 4ced342062 ♻️(core) move app ready code to functions
For readability, we move the code block from the `ready` method to a
dedicated function.

This will allow to add more things to do in the `ready` with more focus.
2025-03-26 19:56:23 +01:00
Laurent Bossavit 2502ff0c99 🔧(dns) make target zone for communes domains configurable
Add a configuration setting tied to an env var, so we can have
a separate zone for staging/preprod.
2025-03-25 19:48:43 +01:00
Laurent Bossavit dc33493739 Revert "(dimail) remove ci-time dependency on dimail to improve CI times"
This reverts commit 81030004e9.
2025-03-25 18:36:17 +01:00
Laurent Bossavit 81030004e9 (dimail) remove ci-time dependency on dimail to improve CI times
The Gitlab instance hosting dimail images has become unstable
resulting in excessive CI failures, and we don't actually depend
on dimail for CI.
2025-03-25 18:10:34 +01:00
Sabrina Demagny 339831f090 🌐(i18n) update translations
Run i18n-download-and-compile to download translations from
Crowdin and compile them
2025-03-25 13:45:24 +01:00
Sabrina Demagny 5178e460c4 (domains) notify support when domain status changes
During the scheduled task to check domains,
send an email notification to domain support if a
status has changed.
2025-03-25 08:44:35 +01:00
Sabrina Demagny feb5d7154b (domains) define domain check interval as a settings
For now, to avoid overloading dimail, we have defined a
time interval between each check request to dimail.
This interval should be configurable for testing and
different environments.
2025-03-25 08:44:35 +01:00
renovate[bot] 7dac39034a ⬆️(dependencies) update js dependencies 2025-03-24 10:55:35 +01:00
renovate[bot] 660fc7c291 ⬆️(dependencies) update python dependencies 2025-03-24 09:34:53 +01:00
renovate[bot] 27fd43b164 ⬆️(dependencies) update next to v15.2.3 [SECURITY] 2025-03-22 12:44:17 +01:00
Laurent Bossavit e63c31f960 🐛(front) disable retries in useQuery and useInfiniteQuery
The default options in TanStack Query don't make sense for these purposes
and were inducing a need for long timeouts in Playwright tests. (Personal
aside: I consider timeouts in Playwright as a testing smell.)
2025-03-20 14:56:15 +01:00
Quentin BEY 6b2ca88ff2 (oidc) add simple introspection backend
This provides a configurable OIDC introspection backend to be able to
call introspection endpoints which returns JSON data instead of an
encrypted JWT.

Two backends are currently defined:

 - ResourceServerBackend` which expect a JSON response
 - JWTResourceServerBackend which implements RFC 9701 and expects
   JWE reponse.

There might be other cases (eg: ResourceServerBackend with JWT, JWS or
JWE, etc. but for now we don't use it, so we follow YAGNI).

This also allow to configure the claim to determine the "audience":

 - client_id: for our Keycloak implementation
 - aud: used by ProConnect
2025-03-20 09:30:18 +01:00
Quentin BEY b771f614e2 🧑‍💻(tilt) setup resource server with kc
This configures the settings to be able to call people as a resource
server when using Keycloak deployment.
2025-03-20 09:30:18 +01:00
Marie PUPO JEAMMET 889a495ea3 🧐(stats) restrict domains count to active domains
Stats are currently counting all domains, including users tests.
Counting enabled domains is more relevant to reflect actual use.
2025-03-19 16:52:11 +01:00
Sabrina Demagny f21716dc68 📝(release) add information about crowdin download
Now a github action downloads last translations from crowdin
when a release branch is created.
We have to merge crowdin translations into release
branch before submit the release PR ;)
2025-03-18 18:29:02 +01:00
Sabrina Demagny 7e1f0b31f9 (scripts) enhance release script input
Remove leading and trailing spaces inserted by mistake in the input.
For the version number, a space can corrupt the version control files.
2025-03-18 18:29:02 +01:00
Sabrina Demagny 666cafe220 📝(dimail) add some info about data required to create mailbox
Prepare generic mailbox implementation
2025-03-18 18:29:02 +01:00
Quentin BEY 1ec98f0948 🧑‍💻(tasks) run management commands
This allows to run management commands from a celery task.
2025-03-18 18:02:53 +01:00
renovate[bot] f0258bbde7 ⬆️(dependencies) update python dependencies 2025-03-17 12:02:44 +01:00
renovate[bot] 773f8cfb4b ⬆️(dependencies) update js dependencies 2025-03-17 11:41:57 +01:00
sdemagny 4003f66243 🌐(i18n) update translated strings
Update translated files with new translations
2025-03-17 11:26:13 +01:00
Sabrina Demagny 5cd8f79f1e 🔖(patch) release version 1.14.1
Update all version files and changelog for patch release.
2025-03-17 11:26:13 +01:00
Sabrina Demagny 9c451e74a6 🔖(minor) release version 1.14.0
Update all version files and changelog for minor release.
2025-03-17 11:06:14 +01:00
Sabrina Demagny f35d17628d (domains) enhance again required action modal content
Enhance DNS required actions and explanations
2025-03-14 18:05:04 +01:00
Sabrina Demagny fa6cfefcd9 📝(release) fix documentation
Add missing part, fix some details and links
2025-03-14 17:22:23 +01:00
Quentin BEY 46ef6eca78 👷(release) download translations from crowdin
When making a release, automatically download translations from Crowdin.
2025-03-14 16:45:41 +01:00
Quentin BEY 9439f454de 📝(release) initiate documentation
This documentation is highly inspired from the `docs` project and
provide details for our project, like using the make command.
2025-03-14 16:45:41 +01:00
Sabrina Demagny db3185e16b (domains) enhance required action modal content
Enhance DNS required actions and explanations
2025-03-14 14:09:02 +01:00
Sabrina Demagny 4c033d7262 🌐(i18n) update translations
Run i18n-download-and-compile to download translations from
Crowdin and compile them
2025-03-14 13:48:33 +01:00
Sabrina Demagny 74655ba378 🐛(domains) fix admin information messages not translated
Using format or f-string breaks translations
2025-03-14 12:26:52 +01:00
Sabrina Demagny 859efa26dc 🐛(mail) fix team invitation subject not translated
Using format or f-string breaks translations
2025-03-14 12:26:52 +01:00
Sabrina Demagny d31b79aaad 🌐(i18n) fix bad translated sentence on crowdin
A special character was inserted by mistake.
2025-03-14 12:26:52 +01:00
Sabrina Demagny 495245a752 🐛(domain) fix flaky test with translated email content
Do not test the content of emails sent with
a random user language
2025-03-14 12:26:52 +01:00
Quentin BEY 03600f243e 🧑‍💻(makefile) fix mails-clean-templates
Fix the `no rule for mails-clean-templates` error when running
`make bootsrap`.
2025-03-14 10:46:17 +01:00
Sabrina Demagny d5eb736343 🌐(i18n) update translations from crowdin
Download and compile translation from Crowdin before release
2025-03-13 17:40:21 +01:00
Quentin BEY 9f1c1ea7dd 🧑‍💻(makefile) enforce generated mail clean
When used locally, you may have removed templates staying and
generarting old translations.
2025-03-13 16:26:32 +01:00
Quentin BEY 803b2c1930 🐛(oauth2) remove ProConnect unknown claims
When we don't have information about the requested claims, they must be
ignored.
2025-03-13 14:52:00 +01:00
Sabrina Demagny e1193bc26d 📝(i18n) add details about translation process
Propose a way to check if every is ok before
upload on crowdin
Add information about crowdin api token
2025-03-13 14:51:07 +01:00
Quentin BEY 285647a8a9 (ci) fix false print detection in commit
The method call `.thumbprint(...)` was detected as a print statement.
Restrict detection to words `print` and `pprint`.
2025-03-13 14:27:26 +01:00
Quentin BEY c4dd4ae3fd 🐛(oauth2) force JWT signed for /userinfo
ProConnect requires the userinfo endpoint to return a signed JWT.
2025-03-13 14:27:26 +01:00
Sabrina Demagny 06f1695071 (mail) add missing text and remove useless translations
Some explanations were forgotten.
Delete useless trans on 'La Régie'
2025-03-13 14:07:31 +01:00
Quentin BEY 34783d0557 🐛(oauth2) add ProConnect scopes
Add missing scopes required by ProConnect evenif we don't fill them.
2025-03-13 11:33:07 +01:00
Quentin BEY 5cc8108e7b 🐛(oauth2) disable PKCE for Proconnect
The PKCE is not available for Proconnect, security is made otherwise.
2025-03-13 10:23:43 +01:00
Quentin BEY 59633d6543 🐛(i18n) force mail build before translation
This prevent the backend translation file to miss the mail translated
content.
I guess this should be managed otherwise, like asking django to look
into the mail template instead of the generated result.
2025-03-12 17:46:21 +01:00
Quentin BEY a6f7c07052 🐛(i18n) fix i18n-generate make command
The `i18n-generate` make command was not downloading the pot from
crowdin before the Django `makemessages` resulting in a pot file never
updated.
2025-03-12 17:46:21 +01:00
Sabrina Demagny 67f8bc32fa 🚀(helm) deploy celery worker and celery beat
This allow to start a celery worker and a celery beat
2025-03-12 17:08:45 +01:00
Sabrina Demagny 0b290d9a5e 🧑‍💻(docker) add flower for dev env
Allow to manage and monitor celery tasks
2025-03-12 17:08:45 +01:00
Sabrina Demagny 68ed5e4d55 (domains) add periodic task to fetch domains status
Add celery crontab to check and update domains status.
This task calls dimail API.
2025-03-12 17:08:45 +01:00
Sabrina Demagny 48264a0b40 🧑‍💻(docker) add celery beat for dev env
Add new container to run celery beat to manage schedule job
2025-03-12 17:08:44 +01:00
Quentin BEY e3bf1d76fa (json) add a test for declared schema
This checks all the defined schema are properly defined.
2025-03-12 15:45:47 +01:00
Quentin BEY f64a592648 (organization) add metadata update command
This allows to update the Organization metadata with default values.
2025-03-12 15:45:47 +01:00
Quentin BEY 7ce5b28af4 (organization) add metadata field
This allows to store custom values which can be reused along the
organization lifetime.
2025-03-12 15:45:47 +01:00
Quentin BEY 3aaddc0493 👷(crowdin) upload main translations to crowdin
This will send the translations to crowdin everytime the main branch is
updated.
2025-03-12 15:14:44 +01:00
Quentin BEY 07ff093b18 👷(github) move dependencies to a separated file
This is inspired from the https://github.com/suitenumerique/docs
project to allow reuse accross several workflows.
2025-03-12 15:14:44 +01:00
Quentin BEY 319a9b18d8 ⬆️(nginx) bump nginx-unprivileged to 1.27
Bump nginx-unprivileged to the latest version
2025-03-12 13:25:59 +01:00
Quentin BEY 403fea94bb (teams) allow broadly available teams
This adds `is_visible_all_services` field to `Teams` to make them
visible to all service providers
2025-03-11 19:15:03 +01:00
Sabrina Demagny 5730b9ea5e (teams) update and enhance team invitation email
- replace logo
- modify wording
- enhance template
2025-03-11 14:09:18 +01:00
Quentin BEY 7f75efacf8 ⚰️(secrets) remove submodule
This submodule is no longer used, as all passwords are now on
vaultwarden.
2025-03-11 13:29:48 +01:00
renovate[bot] 305e2438c5 ⬆️(dependencies) update python dependencies 2025-03-11 12:01:50 +01:00
Sabrina Demagny ebc2b02d22 🐛(domains) use a dedicated mail to invite user to manage domain
- modify models to allow to specify path to mail template
- rename team invitation template
- fix logo and text used for domain invitation email
2025-03-11 11:48:38 +01:00
renovate[bot] 185b87da40 ⬆️(dependencies) update js dependencies 2025-03-11 09:54:26 +01:00
Sabrina Demagny 701aeca763 🐛(mailbox) fix mailbox creation email language
Don't forget to translate mail content before sending.
2025-03-10 15:14:05 +01:00
Sabrina Demagny 7a128393f6 (api) define dimail timeout as a setting
Allow to param dimail timeout for each env
2025-03-10 10:18:14 +01:00
Sabrina Demagny 21993b3272 📝(CONTRIBUTING) describe how to contribute on project
Duplicate docs CONTRIBUTING.md and add part about
process to have a nice commit history.
2025-03-08 13:45:17 +01:00
Sabrina Demagny 63ec61c465 ⬆️(docker) bump crowdin version to 4.6.1
Update crowdin for the best experience :D
2025-03-07 13:34:18 +01:00
Eléonore Voisin 67d9b6462f (frontend) add new access role to domain
add new access role to domain first commit
2025-03-07 11:36:31 +01:00
renovate[bot] ea1f06f6cc ⬆️(dependencies) update django to v5.1.7 [SECURITY] 2025-03-07 10:49:16 +01:00
Quentin BEY b063f690f6 (resource-server) add team invitation endpoint
This allows a service provider to add new members to a team.
2025-03-06 15:17:33 +01:00
Laurent Bossavit ae92ab5dd8 🐛(tests) change domain name factory to be more boring but reliable
MailDomain fixtures now use a boring non-repeating sequence. No
longer will the occasional random CI failure inject excitement
into our workdays; but fear not, there will remain other occasions
to enjoy the art of debugging.
2025-03-06 14:34:41 +01:00
Sabrina Demagny 22419d4779 ✏️(mail) add missing "La Régie"
On invitation email fix "La Régie" naming
2025-03-06 13:38:24 +01:00
Sabrina Demagny 91389181f0 🧑‍💻(admin) add admin for mail domain invitation
Allow to access to mail domain invitation table
on Django admin interface.
2025-03-06 13:28:52 +01:00
Sabrina Demagny 45bafe04de ♻️(backend) rename DomainInvitation
All models relacted to mail domain are prefixed
with "MailDomain". Do the same for mail domain invitations.
2025-03-06 13:01:04 +01:00
Sabrina Demagny c90a74b362 🧑‍💻(tests) improve tests on logged info
Tests on log order make test maintenance difficult.
They are useless and make developers crazy each time
a log information is added :D
2025-03-06 09:57:09 +01:00
Sabrina Demagny 54df9af179 (domains) convert domain invitations to access roles
Use django signals to keep mailbox_manager logic
separated from people core
2025-03-06 09:57:09 +01:00
Marie PUPO JEAMMET 2224acf12d (api) allow invitations for domain management
add an endpoint to allow domain managers to invite someone on people,
using their email address
2025-03-06 09:57:09 +01:00
Marie PUPO JEAMMET 9ee1ef5ba0 🗃️(models) create abstract BaseInvitation and DomainInvitation models
create abstract BaseInvitation models to factorize common elements between
existing Invitation model team-side and new DomainInvitation model
2025-03-05 16:30:24 +01:00
Quentin BEY 7ea381c88a 📝(i18n) describe process for translations
This explains the command to run to be able to translate our project.
2025-03-04 16:20:10 +01:00
Sabrina Demagny 680e05b4a7 🧑‍💻(scripts) improve release script
Fix commits messages
2025-03-04 15:13:00 +01:00
Sabrina Demagny 6e7ebc76d0 🔖(patch) release version 1.13.1
Update all version files and changelog for patch release.
2025-03-04 14:10:29 +01:00
Sabrina Demagny d5b154fbe0 🐛(mailbox) fix migration to fill dn_email field
Fix AttributeError
'Mailbox' object has no attribute 'get_email'
2025-03-04 14:07:16 +01:00
Sabrina Demagny ef4c1da78c 🔖(minor) release version 1.13.0
Update all version files and changelog for minor release.
2025-03-04 10:02:37 +01:00
Laurent Bossavit 4060006a22 💄(domains) improve user experience and avoid repeat fix operations
Adds a loader after clicking on "Re-run check" for a domain
2025-03-03 18:06:15 +01:00
Sabrina Demagny 8b56d97037 👽️(dimail) increase timeout value for check domain API call
In use a timeout of 20 seconds seems more appropriate
2025-03-03 15:52:37 +01:00
renovate[bot] 5fd6579d3c ⬆️(dependencies) update python dependencies 2025-03-03 14:04:57 +01:00
Quentin BEY b4ab36fc0e 📝(oidc) describe the IdP aspect of people
This provides a light documentation about the way to
configure people as an IdentityProvider.
2025-03-03 12:24:43 +01:00
Quentin BEY 160ce92e54 (oidc) add IdP e2e test for login
This is a simple test to assert a user can login via people when setup
as an identity provider.
2025-03-03 12:24:43 +01:00
Quentin BEY a7ab2142f9 🔇(helm) disable sentry on local stack
This is making too much noise when developing using the tilt stack...
2025-03-03 12:24:43 +01:00
Quentin BEY cf4b435c63 🧑‍💻(tilt) allow use of people as an IdP
Few fixes to allow the keycloak dev stack to use people
as an Identity Provider.
This requires the update of the bitnami keycloak chart we
use.
2025-03-03 12:24:43 +01:00
Quentin BEY fd8e0e08c3 💄(oidc) add login page in the frontend
To have a better user experience, we want the login page
to in the frontend.
2025-03-03 12:24:43 +01:00
Quentin BEY 68550f6f7e 🧑‍💻(demo) configure people as an IdP
This configures local environment to test login through people:
- Keycloak configuration of the IdP (people)
- Add Keycloak Application in people

The only user who can login for now is "admin".
2025-03-03 12:24:43 +01:00
Quentin BEY db6cdadd72 (oidc) add django-oauth-toolkit w/ configuration
This allows to use `people` as an identity provider using
OIDC and local users.
This commit is partial, because it does not manage a way to
create "local" users and the login page is the admin one, which
can't be used for non staff users or login with email.
2025-03-03 12:24:43 +01:00
Quentin BEY 8faa049046 🗃️(mailbox_manager) add organization to domain
To be able to provide a SIRET in the ProConnect IdP process
we need to be able to link a mail domain to its organization.
For now this is not mandatory, as we can't detect the organization
and need a frontend process to clarify it.
2025-03-03 12:24:43 +01:00
renovate[bot] a386c61729 ⬆️(dependencies) update js dependencies 2025-03-03 10:09:30 +01:00
Marie PUPO JEAMMET 99cc4d00d5 🔧(api) add sidecar for swagger collection
add drf_spectacular_sidecar to correctly collect swagger
upon collectstatic
2025-02-26 14:59:58 +01:00
Marie PUPO JEAMMET 55b7d1adbd 🔧(swagger) activate swagger in staging
activate swagger in staging for devs from other teams to build interop
more easily
2025-02-25 17:57:40 +01:00
renovate[bot] 7d3f10a4b6 ⬆️(dependencies) update js dependencies 2025-02-24 23:16:46 +01:00
Laurent Bossavit 5f294b8436 📌(renovate) ignore @hookform/resolvers until issue fixed
Version 4.1.0 breaks Zod resolvers, ignore for now.
2025-02-24 23:03:19 +01:00
Laurent Bossavit db338231b7 🧑‍💻(bootstrap) install playwright browsers for local dev
Removal of Playwright browsers installation is a win for CI but
not a reason to degrade developer experience, so we install them
locally as part of the bootstrap process.
2025-02-24 22:52:47 +01:00
renovate[bot] 21033dedbd ⬆️(dependencies) update python dependencies 2025-02-24 16:15:28 +01:00
Sabrina Demagny 2dfa97749f 🧑‍💻(scripts) improve release script
Add explicit links to current and deployment repositories
and change commit message to bump preprod version on deployment repo.
2025-02-21 15:25:05 +01:00
Quentin BEY 7ccd8e3035 🧱(helm) remove extras from helmchart release
The local helm chart provides templates for local use only.
2025-02-21 11:29:20 +01:00
Quentin BEY 8d0fbdfecd 🧱(helm) add resource-server ingress path
The route was added but not declared in the ingress.
2025-02-21 11:17:44 +01:00
Sabrina Demagny a83fb25f6d 🌐(backend) synchronize translations with crowdin again
All translations of backend were synchronized and translated
on crowdin again.
Please use make 'i18n-generate-and-upload' then go to crowdin
to update translations then do 'make i18n-download-and-compile'
2025-02-20 17:31:46 +01:00
Marie PUPO JEAMMET 11c7af205b 📝(tilt) upgrade tilt doc
upgrade tilt doc by remove sops-related topics +
adding linux instructions
2025-02-20 11:11:56 +01:00
Sabrina Demagny f23f1eabd6 🔖(patch) release version 1.12.1
Update all version files and changelog for patch release.
2025-02-20 09:41:18 +01:00
Laurent Bossavit 7025a0787f 👽️(dimail) increase timeout value for API calls
The domain creation endpoint will sometimes take longer than 5s to
complete: increase timeouts.
2025-02-20 09:39:17 +01:00
Sabrina Demagny 70d22eecfa 🔖(minor) release version 1.12.0
Update all version files and changelog for minor release.
2025-02-18 08:51:31 +01:00
Laurent Bossavit 7379d70321 🐛(communes) add missing user creation in domain provisioning
Add user in Dimail for automatically provisioned domains.
2025-02-18 08:30:25 +01:00
Sabrina Demagny 29d0bbb692 (frontend) display button to re-run fetch domain from dimail
Add the button in the modal which describes actions required
to make the domain work
2025-02-17 20:39:03 +01:00
Sabrina Demagny cdb766b0e0 (domains) allow to run all fetch domain data from dimail
Fetch domain status and expected config from dimail.
2025-02-17 19:40:13 +01:00
Sabrina Demagny 38de864d68 (domains) add admin action to fetch domain DNS config
Create a Django admin action to allow retrieval of
the expected domain configuration from dimail.
These values shouldn't change unless external
intervention occurs. An admin command seems sufficient to handle
hypothetical changes.
2025-02-17 19:28:43 +01:00
Sabrina Demagny cad065da84 (frontend) display domain expected config for DNS
If any action is required on the domain, display the
expected domain configuration in the modal window to
inform the user of actions required to operate the domain.
2025-02-17 19:28:43 +01:00
Sabrina Demagny 3893fdf4d7 (domains) get domain expected config for DNS
Call dimail to get DNS configuration values
to make an external domain work and save it in our db.
Add values to serializer for displaying.
2025-02-17 19:28:43 +01:00
Laurent Bossavit d29b5141b1 ️(ci) save time on CI by not downloading already present browser
See https://github.com/microsoft/playwright/issues/23388
2025-02-17 10:30:06 +01:00
renovate[bot] 110fc82250 ⬆️(dependencies) update python dependencies 2025-02-17 09:03:33 +01:00
Sabrina Demagny 95f19f7c6c 🩹(domains) add missing migrations after define ordering
Add missing migration after define ordering by created_at
for Mailbox, MailDomain and MailDomainAccess by created_at.
2025-02-14 18:37:32 +01:00
Sabrina Demagny ab03cd9db9 (domains) check status after creation
Fetch domain status from dimail just after domain creation.
2025-02-14 16:47:53 +01:00
Quentin BEY a811431070 🧑‍💻(tilt) use maildev for local kube
Switch from mailcatcher to maildev for local work purpose.
2025-02-13 15:12:45 +01:00
Quentin BEY d23ac76f36 🧑‍💻(dev) use maildev for local developments
Switch from mailcatcher to maildev for local work purpose.
2025-02-13 15:12:45 +01:00
Sabrina Demagny 9377a96e87 💄(domains) remove useless bold text broken after fbb4797
Fix domain names displaying after fetch status
from dimail failure after commit fbb4797
2025-02-13 09:31:06 +01:00
Sabrina Demagny 25313d3e84 🔒️(docker) patch libssl3 and libcrypto3 to address CVE-2024-12797
Added temporary root privileges to update OpenSSL libraries. Upgrades libssl3
and libcrypto3 to 3.3.3-r0 to fix HIGH severity vulnerability. Properly
switches back to nginx user after updates. Maintains unprivileged execution
while addressing security concern affecting RFC7250 Raw Public Keys
authentication.

Security: CVE-2024-12797
2025-02-13 08:58:28 +01:00
Sabrina Demagny 9cd1b42c3d 💄(frontend) change support email example
The domain of support email suggested was
the same as the example domain name
suggested below.
This was a bit confusing because if the
domain is broken we need to contact someone
with a working email address.
2025-02-12 22:53:06 +01:00
Laurent Bossavit d08198e44d 🔊(prod) move logging config up to Base configuration class
This move makes it possible to set logging configuration on a per-deployment
basis in production.
2025-02-12 15:04:23 +01:00
Sabrina Demagny e55468862d 🧑‍💻(domains) change default ordering
Order by last created models MailDomain, Mailbox and MailDomainAccess.
2025-02-12 10:36:55 +01:00
Laurent Bossavit fbb4797f29 🚨(pytest) fix or suppress warnings during backend tests
Avoid unnecessary noise from testing processes
2025-02-11 16:52:26 +01:00
Sabrina Demagny 0f290df24a (frontend) display required actions details on Domain
On domain page, add link to display modal with needed informations
about actions to do to fix a domain.
2025-02-11 14:07:39 +01:00
Sabrina Demagny a2b2c71448 (api) add required actions to fix domain
Send all informations about required actions
to do to fix a domain and full check domain
health from dimail too.
2025-02-11 13:07:14 +01:00
Sabrina Demagny e1594493a7 🐛(plugin) fix support email
Use a real email to contact support in case
of actions required on the domain in collectivite.fr
2025-02-11 11:15:45 +01:00
renovate[bot] e903c5d4ca ⬆️(dependencies) update python dependencies 2025-02-11 10:57:20 +01:00
renovate[bot] 0f80d5f2db ⬆️(dependencies) update js dependencies 2025-02-11 10:42:56 +01:00
Laurent Bossavit 4cb695c2bf (plugin) add CommuneCreation plugin
Add unit tests and refactor name normalization and zone naming.
2025-02-11 09:53:31 +01:00
Laurent Bossavit a68f8171cb (plugin) add CommuneCreation plugin
Add E2E test to cover the API integration, access grant &c.
2025-02-11 09:53:31 +01:00
Laurent Bossavit 471f69d4ec (plugin) add CommuneCreation plugin
Extend plugin mechanism to be able to grant domain admin in Dimail
2025-02-11 09:53:31 +01:00
Laurent Bossavit dc938d3159 (plugin) add CommuneCreation plugin
Add test for zone creation call
2025-02-11 09:53:31 +01:00
Laurent Bossavit 87907d57de (plugin) add CommuneCreation plugin
Fix earlier test that was in error
2025-02-11 09:53:31 +01:00
Laurent Bossavit 57fd15100e (plugin) add CommuneCreation plugin
Add test for domain spec from Dimail
2025-02-11 09:53:31 +01:00
Laurent Bossavit 34c9dc6cd7 (plugin) add CommuneCreation plugin
Introduces some machinery for testing and executing API orchestrations.
Rolls back some changes in NameFromSiret plugin.
2025-02-11 09:53:31 +01:00
Sabrina Demagny 1be30496be 🌐(frontend) add missing translations
Upload and complete missing translations on crowdin
and download it
2025-02-10 14:46:03 +01:00
Sabrina Demagny 0ca5fa5318 (frontend) new status action required on domain
Display new status flag on mail domain.
2025-02-10 14:25:58 +01:00
Marie PUPO JEAMMET 4d3901b35d (auth) fix empty name from ProConnect
add proconnect scopes for names to be computed automatically
upon user creation
This commit fixes the way names are computed from ProConnect claims
2025-02-10 12:59:16 +01:00
Sabrina Demagny 961bceb64e (domains) store last check domain results
Store results of last dimail check on a domain.
2025-02-10 12:24:37 +01:00
Sabrina Demagny d7957547f8 (frontend) add support email field on domain creation form
Add new field to give email of support to manage actions
required on domain.
2025-02-10 11:37:07 +01:00
Sabrina Demagny 418db6194a (domains) add support email field on MailDomain
Add new field on MailDomain to allow contact support
if some actions are required to fix domain.
2025-02-10 11:37:07 +01:00
Jacques ROUSSEL c2899f66af 🔧(helm) add pdbs to deployments
In order to avoid a service interruption during a Kubernetes
(k8s)upgrade, we add a Pod Disruption Budget (PDB) to deployments.
2025-02-10 11:35:53 +01:00
Sabrina Demagny ba3f6a504f 🌐(backend) add missing translations
Re-run back-i18n-generate and back-i18n-compile
and add some missing translations
2025-02-07 22:11:37 +01:00
Marie PUPO JEAMMET 3de495a489 🚚(github) update all mentions to github repo
github repo moved from numerique-gouv to suitenumerique org
2025-02-07 14:26:27 +01:00
Sabrina Demagny e297a025c3 🔖(minor) release version 1.11.0
Update all version files and changelog for minor release.
2025-02-07 11:46:19 +01:00
Quentin BEY 351c696ef8 🐛(tilt) fix the dev-keycloak configuration
The mailcatcher configuration was missing from the
configuration file when using keycloak.
2025-02-06 10:38:46 +01:00
Sabrina Demagny 579dbdee10 (api) add count mailboxes to MailDomain serializer
Return number of mailboxes of a domain in our API.
2025-02-04 15:22:00 +01:00
Quentin BEY b4a877381a 🐛(teams) disable creation endpoint from abilities
When we don't allow the user to see the team creation button,
we also want to disable the corresponding API.
2025-02-04 15:20:48 +01:00
Quentin BEY 92753082c7 🚑️(teams) hide display add button when disallowed
The frontend should not display "add team" actions when the
user has not the ability.
2025-02-04 15:20:48 +01:00
Sabrina Demagny 4df4172151 (dimail) manage 'action required' status for MailDomain
Adapt fetch domain status call to manage internal and external
fixes required. Use the new status 'action required' to
manage actions expected from support.
Call a new dimail endpoint to run a fix for internal checks
when all external checks are OK.
2025-02-04 14:13:36 +01:00
Laurent Bossavit e7af1fd591 🐛(frontend) improve e2e tests avoiding race condition from mocks
Reorder mock setup with page.route so that it takes place before nav.
2025-02-04 14:03:35 +01:00
Sabrina Demagny 8a2b0d0a76 🐛(backend) fix flaky test after add fr translations
Define user language to test invitation email content.
2025-02-03 15:24:39 +01:00
Sabrina Demagny d48a3ff677 (domains) add action required status on MailDomain
Create a new domain status to handle cases where
action is required from an external domain owner
for a domain to be fully functional.
2025-02-03 15:24:32 +01:00
renovate[bot] 4fb5d87df9 ⬆️(dependencies) update python dependencies 2025-02-03 14:30:56 +01:00
renovate[bot] 6274764f90 ⬆️(dependencies) update js dependencies 2025-02-03 14:18:33 +01:00
Marie PUPO JEAMMET 7f0e231474 (api) restrict mailbox sync to enabled domains
Pending, failed and deactivated domains should not be sync'ed.
2025-02-03 13:47:02 +01:00
Marie PUPO JEAMMET 2c15609c1e 🎨(dimail) factorize dimail response upon successful mailbox creation
dimail's ok response upon mailbox creation is used in several tests.
All those tests now reference proper response available in fixtures.
2025-02-03 13:47:02 +01:00
Marie PUPO JEAMMET cd94dc5091 (dimail) send pending mailboxes upon domain activation
send creation requests to dimail for all pending mailboxes
when domain goes from "pending" to "enabled".
2025-02-03 13:47:02 +01:00
Sabrina Demagny 9d9216cf39 🌐(backend) add missing translations
Regenerate translations, translate some and
replace "Desk" and "Régie" to "La Régie".
2025-02-03 13:24:40 +01:00
Quentin BEY 7dd9eae5d9 💚(argocd) humble try to fix the webhook call
This is an attempt to fix:
`Webhook processing failed: HMAC verification failed`
2025-02-03 13:12:42 +01:00
Quentin BEY 914319c366 💚(argocd) fix deployment command
The command should use organization variables.
2025-02-03 12:48:50 +01:00
Quentin BEY c34ad00fae 💚(docker) fix docker login command
Use the secret from github organization.
2025-02-03 12:35:12 +01:00
Quentin BEY 289879962b 🧑‍💻(tilt) add mailcatcher to the stack
This allows to start a mailcatcher for local developments.
2025-02-03 12:21:34 +01:00
Quentin BEY cd88799943 💚(github) remove secret fetch
The secrets are not managed in the folder anymore.
2025-01-30 15:55:58 +01:00
Quentin BEY 4011c8e8ed 🚑️(plugins) fix name from SIRET specific case
For some SIRET, there are no "liste_enseignes" (null), in such
cases we fallback on the global "company" name.
2025-01-30 15:55:58 +01:00
Sabrina Demagny b98281fa72 🔖(patch) release version 1.10.1
Update all version files and changelog for patch release.
2025-01-27 15:11:01 +01:00
Sabrina Demagny 04bd154bad (scripts) improve release script
One of instructions was in a wrong place.
Release tag on github belongs to the project part.
2025-01-27 14:45:58 +01:00
Marie PUPO JEAMMET 227ecd0700 🐛(dimail) fix imported mailboxes should be enabled instead of pending
Importing mailboxes creates pending mailboxes ... but these mailboxes are
already active and, for most, functional. We thus mark them as "enabled".
2025-01-27 11:23:26 +01:00
renovate[bot] 6d618b4aff ⬆️(dependencies) update python dependencies 2025-01-27 11:09:38 +01:00
renovate[bot] 3a2b2b8a53 ⬆️(dependencies) update js dependencies 2025-01-27 09:33:51 +01:00
Sabrina Demagny f969b118cb (dimail) management command to fetch domain status
Add a management command for a future cron to
to check and update regularly domains status
from dimail.
2025-01-24 19:33:45 +01:00
Sabrina Demagny c48957612b ️(api) add missing cache for stats endpoint
Add cache to display stats for anonymous
and improve some code.
2025-01-24 16:52:02 +01:00
Sabrina Demagny e09e1d1b80 (scripts) adapts release script
Adapts the release script after moving
the deployment part to a new repository.
2025-01-23 18:14:00 +01:00
Marie PUPO JEAMMET 615d5894c9 🔖(minor) release version 1.10.0
Update all version files and changelog for minor release.
heml/env.d/preprod and prod directories are now on another repo.
2025-01-21 17:18:55 +01:00
Laurent Bossavit 914554e45c 🐛(linting) fix linting errors arising from Ruff update
This update affects assertion messages in particular.
2025-01-21 00:42:47 +01:00
renovate[bot] 3f990e93b7 ⬆️(dependencies) update python dependencies 2025-01-21 00:42:47 +01:00
Jacques ROUSSEL 9de20a496e 🐛(ci) fix argocd webhook to auto deploy on staging
Changing the deployment repository broke the automatic deployment of the
main branch. This commit fixes it.
2025-01-20 17:42:24 +01:00
Marie PUPO JEAMMET 870ef424f5 (api) create stats endpoint
create stats endpoint to expose public metrics
2025-01-20 17:32:25 +01:00
renovate[bot] 0e92c1cafa ⬆️(dependencies) update js dependencies 2025-01-20 09:56:30 +01:00
Sabrina Demagny defc18ea40 🐛(backend) fix flaky test with search contact
Sometimes emails generated by faker in data field match search.
So it's necessary to create contacts with empty data field
to search contacts by full_name in tests.
2025-01-17 19:56:35 +01:00
Quentin BEY 4ccea4655b (teams) add treebeard data to serializers
This will allow the frontend to represent teams as a
tree if needed.
2025-01-17 19:00:14 +01:00
Quentin BEY 45fd10fd2d (teams) return parent teams in resource server
Also return the parent teams in the user's team endpoints.
2025-01-17 19:00:14 +01:00
Quentin BEY 201864db3a (teams) return parent teams in API
Also return the parent teams in the user's team endpoints.

This is a first implementation which returns a flat list
of teams (not a tree). This is not really helpful, but
it allows to create hierarchical teams manually (via
admin) if an organization needs it.
2025-01-17 19:00:14 +01:00
Quentin BEY 182f9c1d17 🗃️(teams) add Team dependencies as a tree
This provides the technical way to create Team trees.
The implementation is quite naive.
2025-01-17 19:00:14 +01:00
Quentin BEY cff3d5c123 🐛(tilt) add missing file after previous commit
My previous PR was merged to quickly, I forgot to add
the file to create the secret for Tilt.
2025-01-17 18:11:51 +01:00
Quentin BEY 32a576bbe9 🧑‍💻(tilt) add sync dimail from people btn
Add a button to force sync of dimail accounts from
the people database.
2025-01-17 17:53:14 +01:00
Quentin BEY 010d3674de 🧑‍💻(tilt) add dimail
This adds dimail to the tilt kube deployment
2025-01-17 17:53:14 +01:00
Jacques ROUSSEL 80976e3761 👷(helm) add CI for publishing Helm charts
We need to publish a Helm chart to facilitate separating the code from
the deployment configuration.
2025-01-17 15:26:38 +01:00
Jacques ROUSSEL b848f9eca6 ♻️(dev) refacto tilt stack
To be able to move the repository on the new organization and to
facilitate external developer integration we need to create a standalone
dev stack and use external secret.
2025-01-17 15:26:38 +01:00
Sabrina Demagny cd7135da00 🐛(backend) fix flaky test with team access #646
Faker sometimes creates users whose name starts with "Ms." or "Mr."
The implemented test code computed the order without handling
these cases and failed.
2025-01-17 10:02:27 +01:00
Sabrina Demagny 0a795f6e6f 🧑‍💻(dimail) remove 'NoneType: None' log
Fixes "NoneType: None" log appearing in debug mode.
2025-01-17 08:56:18 +01:00
Sabrina Demagny 6c8329405d 🧑‍💻(setup_dimail_db) create missing access
Add missing John Doe access role on domain test.domain.com
2025-01-17 08:42:10 +01:00
Sabrina Demagny ea3a45ea87 (dimail) improve fetch domain status tests
Add missing test case and add a new fake data
for fetch domain status from dimail
2025-01-16 23:07:14 +01:00
Sabrina Demagny 86451df8b4 ♻️(tests) reorganization of test files
Create two new folders in mailbox_manager tests
2025-01-16 15:57:25 +01:00
Quentin BEY 76fc789eb6 (organization) add admin action for plugin
This allows admin user to run the post creation plugins
from the organization list.
2025-01-16 09:28:38 +01:00
renovate[bot] 632a78c339 ⬆️(dependencies) update django to v5.1.5 [SECURITY] 2025-01-15 10:21:07 +01:00
Laurent Bossavit 20cc173e93 (anct) fetch and display organization names of communes
ANCT-specific extraction of organization names for communes, front
end changes to match.
2025-01-13 15:01:54 +01:00
Marie PUPO JEAMMET d495ef3e19 🧑‍💻(admin) add read-only fields to mailbox admin
mark local_part and domain as read-only fields in admin,
in order to prevent mistakes/temptation. For now, if an local part
needs modification, you can simply delete/recreate the email you want.
Changing the domain is a bigger operation that cannot
be settled simply by changing it Django db.
2025-01-13 08:53:57 +01:00
Sabrina Demagny 4def80214c (frontend) display email if no username
Fallback to email if no username is defined for logged user.
2025-01-12 00:45:49 +01:00
Sabrina Demagny 2428229dbb 🐛(frontend) fix flaky e2e test
Small hack to fix clicking on member action button.
Action button of the third member in the list could not
be clickable every time.
The member list table needs and will be improved soon.
2025-01-12 00:32:56 +01:00
Sabrina Demagny 9b9aa2aa37 🐛(frontend) fix disable mailbox button display
Do not display disable/enable mailbox button
for viewers.
Only owner and admin can manage mailbox and use
this button.
2025-01-10 16:38:07 +01:00
renovate[bot] a28bb5e2a2 ⬆️(dependencies) update js dependencies (#591)
* ⬆️(dependencies) update js dependencies

* ⬆️(dependencies) fixes for React 19

* ⬆️(dependencies) unit test fixes for React 19

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Laurent Bossavit <laurent@MacBook-Pro-de-Laurent.local>
2025-01-10 15:25:38 +01:00
Sabrina Demagny 6962953625 🧑‍💻(admin) add filter for domain
Allow to filter domain by status
2025-01-09 14:12:18 +01:00
Sabrina Demagny b0b718e657 (backend) add admin action to check domain health
Allow to select some domains to check and update status
thanks to a dimail call.
2025-01-09 13:59:00 +01:00
Sabrina Demagny 0abfd49fee (dimail) check domain health
Call dimail to check if a domain still works.
Turn domain into failure status if dimail returns broken state.
And enable domain if dimail returns ok state.
2025-01-09 12:55:23 +01:00
Marie PUPO JEAMMET 5d2e63fc18 🐛(tests) fix flaky back-end test
fixed a test that would fail because of random auth user name being
too close to tested names (assert would find 2 results instead of 1).
Setting the name of auth user should fix the issue permanently.
2025-01-09 10:10:49 +01:00
Marie PUPO JEAMMET 94a443525e 🧑‍💻(admin) add filter and search function to admin mailboxes view
add fields to easily filter and search mailboxes on their statuses,
local part and domains
2025-01-07 17:38:36 +01:00
renovate[bot] 42d7d00772 ⬆️(dependencies) update python dependencies 2025-01-06 10:13:57 +01:00
renovate[bot] 5ec65b1202 ⬆️(dependencies) update next to v15.1.2 [SECURITY] 2025-01-04 13:04:06 +01:00
renovate[bot] 9ba30843c5 ⬆️(dependencies) update python dependencies 2025-01-03 01:16:57 +01:00
Marie PUPO JEAMMET 4de589a7e8 🧑‍💻(mailboxes) add and improve some tests
Add and improve some tests linked to mailbox creation.
2024-12-26 22:22:27 +01:00
Sabrina Demagny 151f030582 🐛(backend) fix dimail call despite mailbox creation failure on our side
When we try to create a duplicate email, a request
to dimail is sent despite a reject on our side.
To solve this issue, we call mailbox creation first
to benefit from validation of our Django model.

Mailbox creation try was called too late after dimail call.
So in attempt to create a duplicated email a request
to dimail was sent despite a failure in our side.
2024-12-26 19:43:04 +01:00
Sabrina Demagny a58152b9a9 📝(scripts) add missing release instructions
Do not forget to check docker image of frontend.
2024-12-26 13:40:35 +01:00
Laurent Bossavit 8fd55a61c5 (e2e) change accounts to facilitate SIRET and add e2e test
We also add registration ID info to the /me endpoint, via serializers
2024-12-23 20:18:44 +01:00
Laurent Bossavit 2435a59078 🧑‍💻(keycloak) add siret attribute and mapper to Keycloak
We can now find organization data as provided by ProConnect in user_info
2024-12-23 20:18:44 +01:00
renovate[bot] fd2c90f50d ⬆️(dependencies) update python dependencies 2024-12-23 17:22:34 +01:00
Quentin BEY 469014ac41 🧑‍💻(user) fix the User.language
The use of a lazy function here make the Django migration
detector to generate a migration every time we run `makemigrations`.
This is not mandatory to have a lazy here as the settings are loaded
once at runtime beginning.
As the choices makes noop migrations, we directly use the setting in
the initial migration.
2024-12-19 22:16:08 +01:00
Marie PUPO JEAMMET e60bae4321 🔖(patch) release version 1.9.1
Update all version files and changelog for patch release.
2024-12-18 15:48:45 +01:00
Marie 2e7f224e30 ⬆️(dimail) upgrade dimail to v0.1.1
Upgrade dimail containers to v0.1.1
2024-12-18 11:38:10 +01:00
Sabrina Demagny 8865e250a0 🔖(minor) release version 1.9.0
Update all version files and changelog for minor release.
2024-12-17 14:48:38 +01:00
Nathan Panchout fa114b1064 (frontend) disable mailbox and allow to create pending mailbox
Allow to disable or enable mailboxes.
And enable mailbox creation button when domain is pending to
allow creation of mailboxes in pending status.
2024-12-17 14:16:06 +01:00
Marie PUPO JEAMMET 38369a8312 🚚(backend) split users test file to improve readability
"test_api_users" was a single test file of 900+ lines.
We used gitfilesplit to split it into several shorter files
for readability.
2024-12-17 14:00:35 +01:00
Marie PUPO JEAMMET d3940f6d09 Split test_api_users.py into users/test_api_users_retrieve.py 2024-12-17 13:48:18 +01:00
Marie PUPO JEAMMET 6061a65e44 Split test_api_users.py into users/test_api_users_create.py 2024-12-17 13:48:18 +01:00
Marie PUPO JEAMMET 4a763396e2 Split test_api_users.py into users/test_api_users_delete.py 2024-12-17 13:48:18 +01:00
Marie PUPO JEAMMET fcc63f9b82 Split test_api_users.py into users/test_api_users_update.py 2024-12-17 13:48:18 +01:00
Marie PUPO JEAMMET 0f4db82b02 Split test_api_users.py into users/test_api_users_list.py 2024-12-17 13:48:18 +01:00
renovate[bot] bcdc57905b ⬆️(dependencies) update python dependencies 2024-12-17 12:12:57 +01:00
Marie PUPO JEAMMET 30832a20e1 ⬆️(dimail) upgrade dimail to 0.0.20
upgrade to 0.0.20 to fix weird anonymous admin behaviour when database
not initialized.
2024-12-17 11:48:52 +01:00
Sabrina Demagny a021e9eec6 🐛(maildomain) fix domain access creation
During a new domain creation, a call to dimail is made
to create user/allow on dimail side before owner role creation
on our side.
So when user/allow creation on dimain side fails,
the owner role is not created on our side.
Therefore the domain is created but invisible on the user interface.
The user will probably try to create the same domain again
and see the error message 'this domain already exists'.
To avoid this we make sure to create owner role on our side
despite dimail failure and set domain to failed status to retry
later dimail access creation.
2024-12-17 10:56:44 +01:00
Quentin BEY fa80edfaa8 🔧(helm) add organization name from siret plugin
This declares the use of the `NameFromSiretOrganizationPlugin`
to allow automatic SIRET -> Name guess when creating an
organization
2024-12-16 16:08:08 +01:00
Quentin BEY 38a5f158b5 (organizations) add siret to name conversion
This adds the plugin system to easily manage
Organization related customizations. This first
plugin tries (best effort) to get a proper name
for the Organization, using its SIRET. This
is French specificities but another plugin can
be defined for other cases.
2024-12-16 16:08:08 +01:00
Quentin BEY 6e14c2e61f (e2e) fix flaky test
The URL may or may not include the "page" query parameter.
2024-12-16 15:51:25 +01:00
Quentin BEY dd9a905dc0 📝(tilt) add startup guide to Tilt use
This provides an "how-to" to deploy a local dev
Kubernetes environment.
2024-12-16 12:35:48 +01:00
Quentin BEY 2f380b49d0 🧑‍💻(dimail) allow account populate from DB
Allow the dimail account creation command to create
accounts from the data in people's database.
2024-12-16 12:18:18 +01:00
Quentin BEY 9953dd2111 💄(frontend) redirect home according to abilities
We try to detect the landing page according to user
permissions (abilities) instead of just the configuration
setting.
This will be improved when the homepage is developed
2024-12-16 11:43:46 +01:00
Sabrina Demagny 5d84e226b7 (maildomain_access) add API endpoint to search users
Add new API endpoint to search for new users
to whom we can assign new roles.
2024-12-13 16:27:48 +01:00
Quentin BEY 6fde76fb46 (contacts) add "abilities" to API endpoint data
Returns the possible actions to the frontend using the
model's `get_abilities`.
2024-12-13 11:52:10 +01:00
Quentin BEY 7154a491f4 ♻️(contacts) switch API to get_abilities
Use the common way to define permissions on the API.

Note: we keep here the notion of "public" contacts,
even if the API does not really allows that. The use
case is not clear for that, but we allow contact w/o
owner to be displayed.
2024-12-13 11:52:10 +01:00
Quentin BEY 019ce99a86 (contacts) filter list API with email
This allows to lookup onto emails for the "magic
filter" on the API list endpoint.
2024-12-12 18:22:53 +01:00
Quentin BEY 579aac264e (contacts) list profile contacts from same org
Allow the contact API to list "profile" contacts for
user of the same organization.
2024-12-12 18:22:53 +01:00
Sabrina Demagny c54f010989 🔖(minor) release version 1.8.0
Update all version files and changelog for minor release.
2024-12-12 17:18:50 +01:00
Sabrina Demagny 841a6c27ba 🐛(backend) fix manage roles on domain admin view
Fix creation of new role to manage mail domain on admin interface
2024-12-12 16:19:51 +01:00
Sabrina Demagny b060bbea51 🐛(mailbox) fix activate mailbox feature
Fix 422 Unprocessable Entity error returned
by dimail because of some missing data sent.
2024-12-12 14:37:22 +01:00
Laurent Bossavit ac24dd91a7 🐛(dimail) ensure Dimail DB is initialized prior to E2E testing
Add a command to the testing workflow to ensure Dimail DB is set up.
2024-12-11 10:29:36 +01:00
Marie PUPO JEAMMET 867c89a2e6 ⬆️(dimail) change dev and tests dimail container version
upgrade dimail container to last version
2024-12-11 10:29:36 +01:00
Marie PUPO JEAMMET 54120eb179 (dimail) automate allows requests to dimail
Automatically send user creation and allow creation requests to dimail
upon creating domain access.
2024-12-11 10:29:36 +01:00
renovate[bot] cf155dc033 ⬆️(dependencies) update python dependencies 2024-12-09 10:47:27 +01:00
Marie PUPO JEAMMET e623c32c9a 🐛(dimail) fix dimail setup command
fix "dimail-setup-db" command
2024-12-09 10:03:12 +01:00
renovate[bot] a1892d2693 ⬆️(dependencies) update django to v5.1.4 [SECURITY] 2024-12-06 23:44:34 +01:00
Quentin BEY 632a96ccf4 🔧(helm) fix the configuration environment
The configuration also defines the Sentry environment
value, which is currently "production" for every Sentry
event.

- dev: Local
- staging: Staging
- preprod: PreProduction
- prod: Production
2024-12-06 09:52:38 +01:00
Quentin BEY d7c654e1aa 📝(contacts) update documentation
This update the documentation to reflect latest
changes:
- drop `Identity`
- `Contact` model updates
2024-12-05 18:40:32 +01:00
Quentin BEY 99b6181944 (e2e) sync keycloak users w/ backend users
The user full name was not the same, it would induce
flaky test, while the user name is updated at user
login from the KC data.
2024-12-05 11:39:58 +01:00
Quentin BEY 514414de16 ♻️(contacts) clean tests after split 2/2
We need to make several cleanup commits to
keep the git history clean.
2024-12-03 16:26:44 +01:00
Quentin BEY f1956f4e86 ♻️(contacts) clean tests after split 1/2
We need to make several cleanup commits to
keep the git history clean.
2024-12-03 16:26:44 +01:00
Quentin BEY f13e2c18b9 Merge branches 'split/core/tests/test_api_contacts.py-core/tests/contacts/test_core_api_contacts_list.py', 'split/core/tests/test_api_contacts.py-core/tests/contacts/test_core_api_contacts_retrieve.py', 'split/core/tests/test_api_contacts.py-core/tests/contacts/test_core_api_contacts_create.py' and 'split/core/tests/test_api_contacts.py-core/tests/contacts/test_core_api_contacts_update.py' into qbey/contacts_api_split_tests 2024-12-03 16:26:44 +01:00
Quentin BEY 4ba1901355 Split core/tests/test_api_contacts.py into core/tests/contacts/test_core_api_contacts_delete.py 2024-12-03 16:26:44 +01:00
Quentin BEY e821163269 Split core/tests/test_api_contacts.py into core/tests/contacts/test_core_api_contacts_list.py 2024-12-03 16:26:44 +01:00
Quentin BEY fc9499e211 Split core/tests/test_api_contacts.py into core/tests/contacts/test_core_api_contacts_retrieve.py 2024-12-03 16:26:44 +01:00
Quentin BEY 9dde2fe7e2 Split core/tests/test_api_contacts.py into core/tests/contacts/test_core_api_contacts_create.py 2024-12-03 16:26:44 +01:00
Quentin BEY 5dad700c1c Split core/tests/test_api_contacts.py into core/tests/contacts/test_core_api_contacts_update.py 2024-12-03 16:26:44 +01:00
Quentin BEY f759d318b9 ♻️(contacts) move user profile to contact
Move the user <-> contact relation for "profile" to
the contact model.

Now the Contact model is the only one to point to
User (and not backward).

Contact:

- FK to User for the owner
- FK to User for the profile
2024-12-03 16:25:25 +01:00
Quentin BEY 60ab61d125 🗃️(contacts) rename base to override
To improve code readability, I propose to rename
the contact field `override`. This comes along
with the fact a contact should not not always
override another (it's the case were I only want
to create some personal contacts).
2024-12-03 16:02:11 +01:00
Quentin BEY 625f122ad5 (contacts) add notes & force full_name
We make the full name mandatory and add a field to
allow user to store personnal notes on the contact.

This also make the "base" contact not mandatory because
user may want to create new contacts out of the blue.
2024-12-03 16:02:11 +01:00
Sabrina Demagny 6d5d0f7ebb 🔧(helm) fix siret validator for staging
This fixes validator for the Organization
registration ID for staging (to enforce SIRET format).
2024-12-03 10:47:43 +01:00
renovate[bot] b7908801d4 ⬆️(dependencies) update python dependencies 2024-12-02 14:18:31 +01:00
renovate[bot] 0785c3a5bc ⬆️(dependencies) update js dependencies 2024-12-02 13:58:40 +01:00
Sabrina Demagny 51eb3474e4 🔖(patch) release version 1.7.1
Update all version files and changelog for patch release.
2024-11-28 18:17:50 +01:00
Laurent Bossavit 10be01f6af 👽️(dimail) adapt dev deployment to modified endpoint of Dimail API
Changed: api.dev.ox.numerique.gouv.fr -> api.ovhdev.dimail1.numerique.gouv.fr
2024-11-28 18:17:15 +01:00
Sabrina Demagny 8e20f86d2c 🔖(minor) release version 1.7.0
Update all version files and changelog for minor release.
2024-11-28 17:29:54 +01:00
Laurent Bossavit 03c79b0412 👽️(dimail) adapt prod deployment to modified endpoint of Dimail API
Changed: api.ox.numerique.gouv.fr -> api.ovhprod.dimail1.numerique.gouv.fr
2024-11-28 17:29:08 +01:00
renovate[bot] a33822b38d ⬆️(dependencies) update js dependencies 2024-11-27 10:13:07 +01:00
Quentin BEY edbd1f0061 (user) add organization data on users API
This will allow the frontend to display data about
organizations when displaying a user or a list of
users.
2024-11-27 10:03:32 +01:00
Quentin BEY 5692c50f21 (organization) add API endpoints
This provides a way to get information about
the organization and update their name for
administrators.
2024-11-27 10:03:32 +01:00
Quentin BEY 6fe4818743 📝(docs) light doc about org & SP
This introduces a very light documentation about
the newly introduced models.
2024-11-27 09:53:32 +01:00
Sabrina Demagny 02c6048d2c (mailbox) allow to activate mailbox
We send a request to dimail API and change mailbox status to enabled.
2024-11-26 15:37:04 +01:00
Sabrina Demagny a6f409f6ed 🐛(dimail) pin dimail-api version
New feature to activate/desactivate mailboxes
is available in v0.0.13
2024-11-26 14:38:38 +01:00
Sabrina Demagny ccb06b3abf (mailbox) allow to disable mailbox
We send a request to dimail API and change mailbox status to disabled.
A disabled mailbox can no longer be used thus access to webmail
is disabled for user.
2024-11-26 14:38:38 +01:00
Sabrina Demagny 3469764697 ♻️(dimail) rename some methods of DimailAPIClient
Use more intuitive and shorter names for actions available in
DimailAPIClient.
2024-11-26 12:17:24 +01:00
renovate[bot] 892a5f10d6 ⬆️(dependencies) update python dependencies 2024-11-26 11:42:39 +01:00
Quentin BEY 0357caa75a 💄(admin) allow header color customization
This allows to use custom colors according to
environment.

FIXES #430
2024-11-26 10:10:59 +01:00
Quentin BEY 9785ce295d 📝(changelog) fix after PR merge
The git rebase moved the changelog line without
conflict so I did not detect it.
2024-11-25 17:01:26 +01:00
Quentin BEY a26e10909d 🐛(admin) add organization on user
User fields has to be listed, this one was forgotten
in the "add organization" commit...
2024-11-25 17:01:26 +01:00
Quentin BEY d6f1cae9e9 🚚(api) split API module in client/resource_server
To improve readability and code sharing we group all
APIs into the same "api" module for each application.

Next submodules might be "scim",
"resource_server_scim", ...

The only shared module is the "permissions" one for now.
2024-11-25 16:05:18 +01:00
Quentin BEY 478a3ffbd1 🎨(dockerfile) fix FromAsCasing warnings
Warning where raised because of the casing change between
`FROM` and `as`.
2024-11-25 16:05:18 +01:00
Quentin BEY 8e6b6318c9 (service_providers) add API endpoints
This allow to display service providers in the frontend.
Not used yet, but will allow to manage organization and
teams related service providers.
2024-11-25 16:05:18 +01:00
Quentin BEY b524369add 🐛(admin) fix organization validators
When updating an Organization in the Django admin, the validator
falsly raises a "duplicated" error because it does not exclude the
current object from the database lookup.
2024-11-25 16:05:18 +01:00
Quentin BEY a991737a59 🔒️(backend) restrict resource server views
We don't want every Service Provider to be able to request
every endpoint if those are not implementing a filtering on
the data returned. To prevent any data leak we enforce the
developers to manually "whitelist" each endpoint and add
the proper filtering when needed.
2024-11-25 16:05:18 +01:00
Quentin BEY a041296f8a (backend) add ServiceProvider
This adds the ServiceProvider notion to allow to better
manage which teams is available for each service provider.
2024-11-25 16:05:18 +01:00
Sabrina Demagny 512d9fe82c 📝(scripts) improve command to do to release
Remove useless option to create tag
2024-11-22 12:04:32 +01:00
Quentin BEY b205ad1d16 🔖(patch) release version 1.6.1
Update all version files and changelog for patch release.
2024-11-22 10:14:47 +01:00
Quentin BEY 0227231370 🚑️(backend) fix claim contains non user field
When we use the feature to get Organization registration
number, the claim contains this value and it does not
match with any user field.
I switched to a whitelist instead of a blacklist (and two
loops, with an if condition on each)
2024-11-22 09:55:28 +01:00
Sabrina Demagny a57070bfb8 🩹(mailbox) fix status of current mailboxes
All mailboxes created so far should be in active status
2024-11-21 23:20:51 +01:00
Quentin BEY 17a1c39dbf 🗃️(teams) remove the slug field in DB
This commit removes the slug field from
the database, now the nullable migration is in
production and the field has been remove from
the code deployed.

FIXES #505
2024-11-20 17:10:05 +01:00
Quentin BEY dfecb83c0a 🔊(backend) update logger config to get info
This sets the default logging level to INFO.
This will help to see what actually happens
in our several deployments.
2024-11-20 16:47:43 +01:00
Quentin BEY 8414a7af4d 🔧(helm) add missing OIDC setting
This setting is mandatory to be able to provision
Organization using their SIRET
2024-11-20 16:36:17 +01:00
Sabrina Demagny 33b364d386 🔖(minor) release version 1.6.0
Update all version files and changelog for minor release.
2024-11-20 14:37:20 +01:00
Laurent Bossavit b7f61e73c2 🐛(dimail) pin dimail-api version to fix main branch
Pin dimail-api version to fix main branch temporarily…
2024-11-20 14:01:07 +01:00
Sabrina Demagny a8e3d8d20e 🔥(teams) remove all search by trigram
Remove trigram search for team access and contact
2024-11-19 23:39:57 +01:00
Laurent Bossavit 43c18cb4e6 (version) convey version information to the /config endpoint and footer
We add the machinery to get version information and display it discreetly.
2024-11-19 18:24:57 +01:00
Laurent Bossavit bbe8f32b96 👷(build) create version.json files on both backend and frontend on push
This supplements the release process. We inject Github metadata into two
version.json files; the 'version' value will depend on the type of event,
for release tag events it should be the same as the release tag (i.e. the
app version). This should make version information available to the /config
endpoint on any push, and the frontend should display the backend version.
(For extra safety we will also want to get the frontend version and display
that, but this commit only supplies the barest necessities.)
2024-11-19 18:24:57 +01:00
Marie PUPO JEAMMET 1e025f034f 🔥(terraform) remove legacy terraform and OpenStack references
Some outdated references to Terraform and OpenStack were missed during
the project quickstart. These are legacy elements inherited from OpenFun.

This commit cleans up the codebase.
2024-11-19 14:04:17 +01:00
renovate[bot] 2f7449f727 ⬆️(dependencies) update js dependencies 2024-11-19 13:51:37 +01:00
Marie PUPO JEAMMET 863c85e3f0 👔(dimail) allow creation of "pending" mailboxes
Previously, mailbox creation was restricted to "enabled" domains.
We now allow users to create mailboxes on pending and failed domains.
Mailboxes thus created have the "pending" mailboxes status.
2024-11-19 10:29:21 +01:00
renovate[bot] 28a972e19e ⬆️(dependencies) update python dependencies 2024-11-18 11:57:28 +01:00
Quentin BEY 90a3e26c7f ♻️(features) rename "TEAMS" flag
To match recent changes we rename the "TEAMS" feature flag
to "TEAMS_DISPLAY".
2024-11-15 10:11:50 +01:00
Quentin BEY 59f3499799 (e2e) add specific accounts for testing
This creates a bunch of accounts with various profiles
to allow testing in a specific "mode"
2024-11-15 10:11:50 +01:00
Nathan Panchout 6123e11dd4 (frontend) use user abilities to show or not the features
use the abilities to show or not the Teams / Mailbox features as well as the
object creation button
2024-11-15 10:11:50 +01:00
Quentin BEY 6be1b63277 🔧(backend) disable contact/teams/mail in prod
We don't want to make these features available for everyone.
2024-11-15 10:11:50 +01:00
Quentin BEY ac853299d3 (backend) add user abilities for front
This allows, on a per user basis, the display of
features.

The main goal here is to allow Team admin or owner
to see the management views.
We also added the same for the two other features
(mailboxes and contacts)

This will be improved later if needed :)
2024-11-15 10:11:50 +01:00
renovate[bot] 4d3097e322 ⬆️(dependencies) update python dependencies 2024-11-14 19:05:05 +01:00
Laurent Bossavit a10f65a51f 💚(ci) call the Dimail container by its actual name (and port)
So that E2E tests in Github Actions can connect to Dimail container.
Previously we were attempting to connect as if from the outside. But
the E2E process is in fact inside the Docker Compose network.
("The tests came from inside the house !")
https://tvtropes.org/pmwiki/pmwiki.php/Main/TheCallsAreComingFromInsideTheHouse
2024-11-14 18:19:55 +01:00
Laurent Bossavit 33e05f7a2d 💚(ci) also save Dimail logs from E2E test runs
To help debug with Dimail interop, save logs from the Dimail container.
Also fix the tests' expectations…
2024-11-14 18:19:55 +01:00
Laurent Bossavit 1e45f1ffd1 (dimail) fix domain creation request to fit latest dimail
Adapt domain creationg request to latest protocol version, also
make error reporting more robust: don't assume utf-8 but use the
response's encoding, don't assume the error is JSON (it won't be
when getting a 500) but reproduce the whole thing instead.
2024-11-14 18:19:55 +01:00
Laurent Bossavit 9c894bdbe9 (dimail) add basic credentials to Develop environment
Make testing easier in a local environment by adding Test credentials.
2024-11-14 18:19:55 +01:00
Laurent Bossavit 20a8edd3aa (dimail) add dimail as a container dependency
Start the Dimail container (in CI and local testing) when starting
the app. The pull_policy should have no effect on CI (because it starts
from a blank slate) but ensure we test against the most recent version
of the chosen tag.
2024-11-14 18:19:55 +01:00
Marie PUPO JEAMMET 0b0b77cead 🐛(dimail) fix unexpected status_code for proper debug
Remove duplicate and catch errors more gracefully. Fixes tests accordingly.
2024-11-14 18:19:55 +01:00
Marie PUPO JEAMMET 21bf431940 (dimail) send domain creation request to dimail
Send domain creation request to dimail when someone creates a domain in people.
2024-11-14 18:19:55 +01:00
Sabrina Demagny 8f30264445 🔖(minor) release version 1.5.0
Update all version files and changelog for minor release.
2024-11-14 15:42:54 +01:00
Laurent Bossavit bde91d55da (ci) separate security scan for frontend too
Separate security scan from build-and-push, so we can make it optional
in CI; this was the case for the backend but frontend was overlooked…
2024-11-13 15:02:50 +01:00
renovate[bot] a328e16e53 ⬆️(dependencies) update js dependencies 2024-11-12 14:48:33 +01:00
Marie PUPO JEAMMET edde9c8d15 (dimail) synchronize mailboxes from dimail to our db
Synchronize mailboxes existing on dimail's api and not on our side,
on domains we are administrating.
2024-11-08 16:40:06 +01:00
Sabrina Demagny a18f06ed27 🐛(mail) fix button display on outlook
In confirmation email of mailbox creation,
button "Go to La Messagerie" disappears on Outlook.
We try to fix here this display bug.
2024-11-08 16:32:18 +01:00
Quentin BEY 72abe04c72 🗃️(teams) remove slug field
After some reflexion, the use of a slug field raises to many
problems without being really needed.

One problem is the slug is made from the group name, but we
don't have unicity on this, so a user might be blocked without
any clue.

We also want to allow group names to be reused (which is already
allowed except for the automatic slug).

The unique ID that will be shared with Service Providers will be
the PK/UUID.
2024-11-06 18:10:02 +01:00
Jacques ROUSSEL 79e92214ab 🔐(secret) add qbey age key
Welcome qbey, allow to run tilt locally
2024-11-06 14:45:08 +01:00
Quentin BEY e5f1151f58 🔧(helm) update settings after previous commit
This adds `siret`to the requested OIDC scopes.
This defines a validator for the Organization
registration ID, to enforce SIRET format.
2024-11-06 14:45:08 +01:00
Quentin BEY ca886c19b0 👔(backend) add Organization model
We introduce the Organization model has a "hat" for all
users and team.

Each User must have a "default" organization.
Each Team must have an organization.

When a User creates a new Team, the team is linked to their
default Organization.

For now the Organization should not be visible to end users
this is a purely technical aspect as it.

The models are also adding a permission to allow User to edit
an Organization, but for now there are no endpoints for that.

Next steps:
- Add an Organization to each User and Team on all environments
  to mark Organization as mandatory in database.
- Add scope to Organization to list the Service Provider list
  allowed for a User in an Organization.
- Add endpoints + frontend to manage Organization's scopes
2024-11-06 14:45:08 +01:00
Laurent Bossavit b602478406 ⬆️(dependencies) remove unneeded dependencies
Remove url-normalize as it is not referenced anywhere
2024-11-05 16:41:13 +01:00
Laurent Bossavit 8b0f942f9b ⬆️(dependencies) update js dependencies
Get react-aria-components only from Cunningham to avoid version clashes
2024-11-05 16:12:42 +01:00
renovate[bot] bd6fe584c6 ⬆️(dependencies) update js dependencies 2024-11-05 16:12:42 +01:00
Laurent Bossavit 821db276bc (ci) add security scan
Separate security scan from build-and-push, so we can make it optional in CI
2024-11-05 15:21:02 +01:00
Jacques ROUSSEL 639490d41e 🚑️(backend) fixe CVEs in backend image
Use alpine version for production image instead of debian in order
to have less CVEs.
2024-11-05 15:21:02 +01:00
Jacques ROUSSEL 748e24cab6 🚑️(frontend) fixe CVEs in frontend image
Use alpine version for production image instead of debian in order
tohave less CVEs.
2024-11-05 15:21:02 +01:00
Jacques ROUSSEL 55c0815c31 (ci) add security scan
Add a security scan for CVE with trivy
2024-11-05 15:21:02 +01:00
Marie PUPO JEAMMET 989239082e (tests) test dimail setup command
Test (and slightly improve) dimail setup command,
which populate database on local dimail container
2024-11-05 11:22:02 +01:00
renovate[bot] 988a091e53 ⬆️(dependencies) update python dependencies 2024-11-04 10:13:48 +01:00
Nathan Panchout 20a19d36b5 ♻️(frontend) delete infinite scroll
In order to be in agreement with the back, we must modify the front
so as to no longer have infinite scroll in the team part.
It is also necessary to modify the tests accordingly
2024-10-31 17:59:14 +01:00
Sabrina Demagny 7d695ab81c 🔥(teams) remove pagination of teams listing
For frontend pagination is useless for teams,
so we remove it.
2024-10-31 17:59:14 +01:00
Nathan Panchout a42e7a10db (frontend) adapt test with new list
After the modifications to the back, some adjustments are expected
on the front side to remove the random aspect of the user search
2024-10-30 19:32:46 +01:00
Sabrina Demagny ad4065e682 🚧(demo) modify data set to fix e2e tests
This part needs more refactoring.
Demo data set needs to be no totally random.
2024-10-30 19:32:46 +01:00
Sabrina Demagny ababcde0d6 🔥(teams) remove search users by trigram
This feature is not necessary for our users now
and we got some strange results so we decided
to remove this feature.
2024-10-30 19:32:46 +01:00
Sabrina Demagny faf8dcc7e5 (teams) register contacts on admin views
Allow to manage contact on admin interface
2024-10-30 15:33:29 +01:00
renovate[bot] 2bbed8de0c ⬆️(dependencies) update python dependencies 2024-10-30 11:31:12 +01:00
Laurent Bossavit a736a9143f 💚(ci) use list reporter in addition to html
This will output lines to the testing jobs' logs so we know how many
tests are run even when jobs are eventually cancelled.
2024-10-30 09:35:49 +01:00
Laurent Bossavit c4ea62dc1f 💚(ci) improve E2E tests
Disable retries and save trace for failed tests.

💚(ci) preserve server logs

Save server logs to the same place as Playwright reports to aid debugging.

💚(ci) move back to 1 worker on CI

At least three reasons
- seems redundant with sharding
- strong suspicion it's the reason for the ValidationError issue
- that way the comment no longer tells a lie ;)

💚(ci) improve E2E tests

Log into CHANGELOG to ensure the new test results impact PR status 🤷

💚(ci) make dummy data creation more robust

This is a QR (Quick Response) fix for the failures in the "Add dummy
data" step in E2E testing. Proper QC (Quality Control) needs a bit
more thought.
2024-10-29 14:58:25 +01:00
Laurent Bossavit 1d1f5cfbb6 🚨(linter) add missing docstrings
Title says all there is to say…
2024-10-29 09:08:35 +01:00
Laurent Bossavit 3934a0bc28 🚨(linter) configure exclusions
See PR for rationale.
2024-10-29 09:08:35 +01:00
Laurent Bossavit fd3ac00ea7 🚨(linter) add D1 pydocstyle rule
See PR for rationale.
2024-10-29 09:08:35 +01:00
Quentin BEY 2d46b7d504 🔧(backend) fix sentry deprecated scope
`sentry_sdk.configure_scope` is deprecated and will
be removed in the next major version.
2024-10-25 15:15:39 +02:00
Quentin BEY 54e5be81e2 🔇(backend) remove Sentry duplicated warning/errors
The `DockerflowMiddleware` job is to add logs, sadly the
Sentry LoggingIntegration also catch these logs (at
level WARNING and above) which results in an extra alert
for the same purpose (the exception + the logger.error).

see https://github.com/mozilla-services/Dockerflow/blob/main/docs/mozlog.md

The fix is to ask Sentry to ignore this specific loggger
(`request.summary`).
2024-10-25 15:15:39 +02:00
Marie PUPO JEAMMET 31944e8083 🔧(backend) restore Sentry default integrations
This reverts 30229e11f9
2024-10-25 15:15:39 +02:00
Sabrina Demagny d1421dbe8c ✏️(scripts) add some details for manual steps
Some instructions are missing to explain manual steps
2024-10-24 17:27:07 +02:00
Marie PUPO JEAMMET c8800259ae 🐛(changelog) move e2e sharding to unreleased
e2e sharding was merged without updating it to unreleased. Fixes that
2024-10-24 17:11:29 +02:00
NathanPanchout bfc2462103 👷(ci) add sharding e2e tests
e2e tests take too long to run. We can easily reduce this time by 2 by adding
shards
2024-10-24 16:35:07 +02:00
Sabrina Demagny 230ed75d8d 🔖(patch) release version 1.4.1
Update all version files and changelog for patch release.
2024-10-23 22:55:56 +02:00
NathanPanchout 5ac3454c5a 🚑️(frontend) fix MailDomainsLayout
The content is not scrollable. We add an auto overflow to solve this problem
2024-10-23 21:44:28 +02:00
Sabrina Demagny bbac34b62a 🔖(minor) release version 1.4.0 (#483)
Update all version files and changelog for minor release.
2024-10-23 19:15:14 +02:00
Sabrina Demagny 38508c272e 📝(CHANGELOG) fix modification declaration
One of unreleased modification declaration was
declared in a wrong place
2024-10-23 18:46:04 +02:00
Sabrina Demagny 10a81f14e6 ✏️(mail) fix typo in mailbox creation email
Whitespace missing in french translation after ':'
2024-10-23 17:56:40 +02:00
Nathan Panchout a08689a64d (frontend) add tabs for mail domain page (#466)
Currently, it is complicated to understand the navigation between mailbox
management and role management for an email domain.
This is why we add tabs with explicit naming
2024-10-23 17:45:42 +02:00
Marie PUPO JEAMMET 30229e11f9 🐛(sentry) fix duplicated sentry errors
errors were sent to sentry twice
2024-10-22 18:03:01 +02:00
Sabrina Demagny d11d1f3bd0 🐛(script) improve and fix release script
Use regex to replace version and add missing
frontend update version.
2024-10-22 00:28:08 +02:00
renovate[bot] 6a22169dae ⬆️(dependencies) update python dependencies 2024-10-21 12:22:49 +02:00
Marie PUPO JEAMMET 6e7a6e9d51 🔖(patch) release version 1.3.1
Update all version files and changelog for patch release.
2024-10-18 14:49:56 +02:00
633 changed files with 62129 additions and 23128 deletions
+24
View File
@@ -0,0 +1,24 @@
# Set the default behavior for all files
* text=auto eol=lf
# Binary files (should not be modified)
*.png binary
*.jpg binary
*.jpeg binary
*.gif binary
*.ico binary
*.mov binary
*.mp4 binary
*.mp3 binary
*.flv binary
*.fla binary
*.swf binary
*.gz binary
*.zip binary
*.7z binary
*.ttf binary
*.woff binary
*.woff2 binary
*.eot binary
*.pdf binary
+2 -2
View File
@@ -9,9 +9,9 @@ We primarily use GitHub as an issue tracker. If however you're encountering an i
---
Please make sure you have read our [main Readme](https://github.com/numerique-gouv/people).
Please make sure you have read our [main Readme](https://github.com/suitenumerique/people).
Also make sure it was not already answered in [an open or close issue](https://github.com/numerique-gouv/people/issues).
Also make sure it was not already answered in [an open or close issue](https://github.com/suitenumerique/people/issues).
If your question was not covered, and you feel like it should be, fire away! We'd love to improve our docs! 👌
+77
View File
@@ -0,0 +1,77 @@
name: Download translations from Crowdin
on:
workflow_dispatch:
push:
branches:
- 'release/**'
jobs:
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '22.x'
with-front-dependencies-installation: true
synchronize-with-crowdin:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Create empty source files
run: |
touch src/backend/locale/django.pot
mkdir -p src/frontend/packages/i18n/locales/desk/
touch src/frontend/packages/i18n/locales/desk/translations-crowdin.json
# crowdin workflow
- name: crowdin action
uses: crowdin/github-action@v2
with:
config: crowdin/config.yml
upload_sources: false
upload_translations: false
download_translations: true
create_pull_request: false
push_translations: false
push_sources: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# A numeric ID, found at https://crowdin.com/project/<projectName>/tools/api
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
# Visit https://crowdin.com/settings#api-key to create this token
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
CROWDIN_BASE_PATH: "../src/"
# frontend i18n
- name: Restore the frontend cache
uses: actions/cache@v5
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: generate translations files
working-directory: src/frontend
run: yarn i18n:deploy
# Create a new PR
- name: Create a new Pull Request with new translated strings
uses: peter-evans/create-pull-request@v7
with:
commit-message: |
🌐(i18n) update translated strings
Update translated files with new translations
title: 🌐(i18n) update translated strings
body: |
## Purpose
update translated strings
## Proposal
- [x] update translated strings
branch: i18n/update-translations
labels: i18n
+76
View File
@@ -0,0 +1,76 @@
name: Update crowdin sources
on:
workflow_dispatch:
push:
branches:
- main
jobs:
install-dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '22.x'
with-front-dependencies-installation: true
with-build_mails: true
synchronize-with-crowdin:
needs: install-dependencies
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
# Backend i18n
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version-file: "src/backend/pyproject.toml"
- name: Install uv
uses: astral-sh/setup-uv@v6
- name: Install the project
run: uv sync --locked --all-extras
working-directory: src/backend
- name: Restore the mail templates
uses: actions/cache@v5
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
fail-on-cache-miss: true
- name: Install gettext
run: |
sudo apt-get update
sudo apt-get install -y gettext pandoc
- name: generate pot files
working-directory: src/backend
run: |
DJANGO_CONFIGURATION=Build uv run python manage.py makemessages -a --keep-pot
# frontend i18n
- name: Restore the frontend cache
uses: actions/cache@v5
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
fail-on-cache-miss: true
- name: generate source translation file
working-directory: src/frontend
run: yarn i18n:extract
# crowdin workflow
- name: crowdin action
uses: crowdin/github-action@v2
with:
config: crowdin/config.yml
upload_sources: true
upload_translations: false
download_translations: false
create_pull_request: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# A numeric ID, found at https://crowdin.com/project/<projectName>/tools/api
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
# Visit https://crowdin.com/settings#api-key to create this token
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
CROWDIN_BASE_PATH: "../src/"
+85
View File
@@ -0,0 +1,85 @@
name: Dependency reusable workflow
on:
workflow_call:
inputs:
node_version:
required: false
default: '22.x'
type: string
with-front-dependencies-installation:
type: boolean
default: false
with-build_mails:
type: boolean
default: false
jobs:
front-dependencies-installation:
if: ${{ inputs.with-front-dependencies-installation == true }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
- name: Restore the frontend cache
uses: actions/cache@v5
id: front-node_modules
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Setup Node.js
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/setup-node@v6
with:
node-version: ${{ inputs.node_version }}
- name: Install dependencies
if: steps.front-node_modules.outputs.cache-hit != 'true'
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Cache install frontend
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/cache@v5
with:
path: "src/frontend/**/node_modules"
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
build-mails:
if: ${{ inputs.with-build_mails == true }}
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/mail
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Restore the mail templates
uses: actions/cache@v5
id: mail-templates
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
- name: Setup Node.js
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/setup-node@v6
with:
node-version: ${{ inputs.node_version }}
- name: Install yarn
if: steps.mail-templates.outputs.cache-hit != 'true'
run: npm install -g yarn
- name: Install node dependencies
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn install --frozen-lockfile
- name: Build mails
if: steps.mail-templates.outputs.cache-hit != 'true'
run: yarn build
- name: Cache mail templates
if: steps.mail-templates.outputs.cache-hit != 'true'
uses: actions/cache@v5
with:
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
+1 -18
View File
@@ -11,26 +11,9 @@ jobs:
notify-argocd:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "people,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/people/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
uses: actions/checkout@v6
-
name: Call argocd github webhook
run: |
+59 -64
View File
@@ -1,4 +1,5 @@
name: Docker Hub Workflow
run-name: Docker Hub Workflow
on:
workflow_dispatch:
@@ -15,42 +16,62 @@ env:
DOCKER_USER: 1001:127
jobs:
trivy-scan:
runs-on: ubuntu-latest
steps:
-
name: Checkout repository
uses: actions/checkout@v6
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
lasuite/people-backend
lasuite/people-frontend
-
name: Run trivy scan (backend)
uses: numerique-gouv/action-trivy-cache@main
with:
docker-build-args: '--target backend-production -f Dockerfile'
docker-image-name: 'docker.io/lasuite/people-backend:${{ github.sha }}'
-
name: Run trivy scan (frontend)
uses: numerique-gouv/action-trivy-cache@main
with:
docker-build-args: '--target frontend-production -f src/frontend/Dockerfile'
docker-image-name: 'docker.io/lasuite/people-frontend:${{ github.sha }}'
build-and-push-backend:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "people,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
uses: actions/checkout@v6
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/people-backend
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/people/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USER }}
password: ${{ secrets.DOCKER_HUB_PASSWORD }}
- name: create-version-json
id: create-version-json
uses: jsdaniell/create-json@v1.2.3
with:
name: "version.json"
json: '{"source":"${{github.repository}}", "version":"${{github.ref_name}}", "commit":"${{github.sha}}", "build": "NA"}'
dir: 'src/backend'
-
name: Build and push
uses: docker/build-push-action@v5
uses: docker/build-push-action@v6
with:
context: .
target: backend-production
@@ -62,41 +83,35 @@ jobs:
build-and-push-frontend:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "people,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
uses: actions/checkout@v6
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/people-frontend
-
name: Load sops secrets
uses: rouja/actions-sops@main
- name: create-version-json
id: create-version-json
uses: jsdaniell/create-json@v1.2.3
with:
secret-file: secrets/numerique-gouv/people/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
name: "version.json"
json: '{"source":"${{github.repository}}", "version":"${{github.ref_name}}", "commit":"${{github.sha}}", "build": "NA"}'
dir: 'src/frontend/apps/desk'
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USER }}
password: ${{ secrets.DOCKER_HUB_PASSWORD }}
-
name: Build and push
uses: docker/build-push-action@v5
uses: docker/build-push-action@v6
with:
context: .
file: ./src/frontend/Dockerfile
target: frontend-production
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
push: ${{ github.event_name != 'pull_request' }}
@@ -110,29 +125,9 @@ jobs:
runs-on: ubuntu-latest
if: github.event_name != 'pull_request'
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
- uses: numerique-gouv/action-argocd-webhook-notification@main
id: notify
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "people,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/people/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/'$GITHUB_REPOSITORY'"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_WEBHOOK_URL
deployment_repo_path: "${{ secrets.DEPLOYMENT_REPO_URL }}"
argocd_webhook_secret: "${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET }}"
argocd_url: "${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}"
-22
View File
@@ -1,22 +0,0 @@
name: Helmfile lint
run-name: Helmfile lint
on:
pull_request:
branches:
- 'main'
jobs:
helmfile-lint:
runs-on: ubuntu-latest
container:
image: ghcr.io/helmfile/helmfile:latest
steps:
-
uses: numerique-gouv/action-helmfile-lint@main
with:
app-id: ${{ secrets.APP_ID }}
age-key: ${{ secrets.SOPS_PRIVATE }}
private-key: ${{ secrets.PRIVATE_KEY }}
helmfile-src: "src/helm"
repositories: "people,secrets"
+102 -107
View File
@@ -9,25 +9,37 @@ on:
- '*'
jobs:
# Call the reusable workflow to install dependencies
dependencies:
uses: ./.github/workflows/dependencies.yml
with:
node_version: '22.x'
with-front-dependencies-installation: true
with-build_mails: true
lint-git:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' # Makes sense only for pull requests
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 0
fetch-depth: 0
- name: show
run: git log
- name: Enforce absence of print statements in code
if: always()
run: |
! git diff origin/${{ github.event.pull_request.base.ref }}..HEAD -- . ':(exclude)**/people.yml' | grep "print("
! git diff origin/${{ github.event.pull_request.base.ref }}..HEAD -- . ':(exclude)**/people.yml' | grep -E "(\bprint\(|\bpprint\()"
- name: Check absence of fixup commits
if: always()
run: |
! git log | grep 'fixup!'
- name: Install gitlint
if: always()
run: pip install --user requests gitlint
- name: Lint commit messages added to main
if: always()
run: ~/.local/bin/gitlint --commits origin/${{ github.event.pull_request.base.ref }}..HEAD
check-changelog:
@@ -37,17 +49,17 @@ jobs:
github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
fetch-depth: 0
fetch-depth: 0
- name: Check that the CHANGELOG has been modified in the current branch
run: git whatchanged --name-only --pretty="" origin/${{ github.event.pull_request.base.ref }}..HEAD | grep CHANGELOG
run: git log --name-only --pretty="" origin/${{ github.event.pull_request.base.ref }}..HEAD | grep CHANGELOG
lint-changelog:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Check CHANGELOG max line length
run: |
max_line_length=$(cat CHANGELOG.md | grep -Ev "^\[.*\]: https://github.com" | wc -L)
@@ -56,45 +68,15 @@ jobs:
exit 1
fi
install-front:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18.x'
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Install dependencies
if: steps.front-node_modules.outputs.cache-hit != 'true'
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Cache install frontend
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
build-front:
runs-on: ubuntu-latest
needs: install-front
needs: dependencies
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Restore the frontend cache
uses: actions/cache@v4
uses: actions/cache@v5
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
@@ -104,20 +86,20 @@ jobs:
run: cd src/frontend/ && yarn ci:build
- name: Cache build frontend
uses: actions/cache@v4
uses: actions/cache@v5
with:
path: src/frontend/apps/desk/out/
key: build-front-${{ github.run_id }}
test-front:
runs-on: ubuntu-latest
needs: install-front
needs: dependencies
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Restore the frontend cache
uses: actions/cache@v4
uses: actions/cache@v5
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
@@ -128,62 +110,71 @@ jobs:
lint-front:
runs-on: ubuntu-latest
needs: install-front
needs: dependencies
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Restore the frontend cache
uses: actions/cache@v4
uses: actions/cache@v5
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Check linting
run: cd src/frontend/ && yarn lint
test-e2e:
runs-on: ubuntu-latest
needs: [build-mails, build-front]
needs: [dependencies, build-front]
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
shardIndex: [1, 2, 3, 4]
shardTotal: [4]
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Set services env variables
run: |
make create-env-files
cat env.d/development/common.e2e.dist >> env.d/development/common
- name: Download mails' templates
uses: actions/download-artifact@v4
- name: Restore the mail templates
uses: actions/cache@v5
id: mail-templates
with:
name: mails-templates
path: src/backend/core/templates/mail
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
fail-on-cache-miss: true
- name: Restore the frontend cache
uses: actions/cache@v4
uses: actions/cache@v5
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Restore the build cache
uses: actions/cache@v4
uses: actions/cache@v5
id: cache-build
with:
path: src/frontend/apps/desk/out/
key: build-front-${{ github.run_id }}
- name: Build and Start Docker Servers
env:
DOCKER_BUILDKIT: 1
COMPOSE_DOCKER_CLI_BUILD: 1
run: |
docker compose build --pull --build-arg BUILDKIT_INLINE_CACHE=1
make update-keycloak-realm-app
make add-dev-rsa-private-key-to-env
make run
- name: Apply DRF migrations
run: |
make migrate
@@ -192,42 +183,35 @@ jobs:
run: |
make demo FLUSH_ARGS='--no-input'
- name: Install Playwright Browsers
run: cd src/frontend/apps/e2e && yarn install
- name: Run e2e tests
run: cd src/frontend/ && yarn e2e:test
run: cd src/frontend/ && yarn e2e:test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
- name: Save logs
if: always()
run: docker compose logs > src/frontend/apps/e2e/report/logs.txt
- name: Save Dimail logs
if: always()
run: docker compose logs dimail > src/frontend/apps/e2e/report/dimail-logs.txt
- uses: actions/upload-artifact@v4
if: always()
with:
name: playwright-report
name: playwright-report-${{ matrix.shardIndex }}
path: src/frontend/apps/e2e/report/
retention-days: 7
build-mails:
tests-e2e-feedback:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/mail
needs: [test-e2e]
if: always()
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: '18'
- name: Install yarn
run: npm install -g yarn
- name: Install node dependencies
run: yarn install --frozen-lockfile
- name: Build mails
run: yarn build
- name: Persist mails' templates
uses: actions/upload-artifact@v4
with:
name: mails-templates
path: src/backend/core/templates/mail
- name: All tests ok
if: ${{ !(contains(needs.*.result, 'failure')) }}
run: exit 0
- name: Some tests failed
if: ${{ contains(needs.*.result, 'failure') }}
run: exit 1
lint-back:
runs-on: ubuntu-latest
@@ -236,23 +220,26 @@ jobs:
working-directory: src/backend
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install Python
uses: actions/setup-python@v5
uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.10'
- name: Install development dependencies
run: pip install --user .[dev]
python-version-file: "src/backend/pyproject.toml"
- name: Install uv
uses: astral-sh/setup-uv@v6
- name: Install the project
run: uv sync --locked --all-extras
- name: Check code formatting with ruff
run: ~/.local/bin/ruff format . --diff
run: uv run ruff format . --diff
- name: Lint code with ruff
run: ~/.local/bin/ruff check .
run: uv run ruff check .
- name: Lint code with pylint
run: ~/.local/bin/pylint .
run: uv run pylint .
test-back:
runs-on: ubuntu-latest
needs: build-mails
needs: dependencies
defaults:
run:
working-directory: src/backend
@@ -282,27 +269,35 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Create writable /data
run: |
sudo mkdir -p /data/media && \
sudo mkdir -p /data/static
- name: Download mails' templates
uses: actions/download-artifact@v4
- name: Restore the mail templates
uses: actions/cache@v5
id: mail-templates
with:
name: mails-templates
path: src/backend/core/templates/mail
- name: Install Python
uses: actions/setup-python@v5
path: "src/backend/core/templates/mail"
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
fail-on-cache-miss: true
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.10'
- name: Install development dependencies
run: pip install --user .[dev]
python-version-file: "src/backend/pyproject.toml"
- name: Install uv
uses: astral-sh/setup-uv@v6
- name: Install the dependencies
run: uv sync --locked --all-extras
- name: Install gettext (required to compile messages)
run: |
sudo apt-get update
sudo apt-get install -y gettext
- name: Generate a MO file from strings extracted from the project
run: python manage.py compilemessages
run: uv run python manage.py compilemessages
- name: Run tests
run: ~/.local/bin/pytest -n 2
run: uv run pytest -n 2
+36
View File
@@ -0,0 +1,36 @@
name: Release Chart
run-name: Release Chart
on:
push:
branches:
- 'main'
jobs:
release:
permissions:
contents: write
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Cleanup
run: |
rm -rf ./src/helm/extra
rm -rf ./src/helm/dimail
rm -rf ./src/helm/maildev
- name: Install Helm
uses: azure/setup-helm@v4
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: Publish Helm charts
uses: numerique-gouv/helm-gh-pages@add-overwrite-option
with:
charts_dir: ./src/helm
linting: on
token: ${{ secrets.GITHUB_TOKEN }}
+2 -5
View File
@@ -42,7 +42,6 @@ env.bak/
venv.bak/
env.d/development/*
!env.d/development/*.dist
env.d/terraform
# npm
node_modules
@@ -59,10 +58,8 @@ src/frontend/tsclient
# Logs
*.log
# Terraform
.terraform
*.tfstate
*.tfstate.backup
# Celery beat
src/backend/celerybeat-schedule
# Test & lint
.coverage
-3
View File
@@ -1,3 +0,0 @@
[submodule "secrets"]
path = secrets
url = ../secrets
-10
View File
@@ -1,10 +0,0 @@
creation_rules:
- path_regex: ./*
key_groups:
- age:
- age15fyxdwmg5mvldtqqus87xspuws2u0cpvwheehrtvkexj4tnsqqysw6re2x # jacques
- age16hnlml8yv4ynwy0seer57g8qww075crd0g7nsundz3pj4wk7m3vqftszg7 # github-repo
- age1plkp8td6zzfcavjusmsfrlk54t9vn8jjxm8zaz7cmnr7kzl2nfnsd54hwg # Anthony Le-Courric
- age12g6f5fse25tgrwweleh4jls3qs52hey2edh759smulwmk5lnzadslu2cp3 # Antoine Lebaud
- age1hnhuzj96ktkhpyygvmz0x9h8mfvssz7ss6emmukags644mdhf4msajk93r # Samuel Paccoud
- age1tl80n23wq6zxegupwn70ew0yp225ua5v4dk800x7g2w6pvlxz46qk592pa # Marie Pupo Jeammet
+462 -8
View File
@@ -8,6 +8,424 @@ and this project adheres to
## [Unreleased]
### Fixed
- ✨(mailboxes) enforce lowercase on mailboxes
- 🐛(i18n) fix missing translations for status tag labels
- 🚚(route) prioritize mail domains as default landing page
### Changed
- 🍱(static) update logo in invitation email template #1085
- ✨(uiV2) use Lasuite UI kit, new layout
## [1.23.1] - 2026-02-16
- ✨(invitations) refresh expired invitations
## [1.23.0] - 2026-02-12
### Added
- ✨(demo) add aliases to demo #1050
- ✨(front) add icon to button to configure a domain
- ✨(datagrid) add sort to mailboxes list + mail domain list
- ✨(invitations) allow delete invitations mails domains access by an admin
- ✨(front) delete invitations mails domains access
- ✨(front) add show invitations mails domains access #1040
- ✨(invitations) can delete domain invitations
### Fixed
- 🐛(domains) fix attemps to send invitations to existing users #953
### Changed
- 🚸(email) we should ignore case when looking for existing emails #1056
- 🏗️(core) migrate from pip to uv
- ✨(front) add show invitations mails domains access #1040
## [1.22.2] - 2026-01-26
### Fixed
- 🐛(aliases) authorize special domain devnull in alias destinations #1029
## [1.22.1] - 2026-01-21
- 🔒️(organization) the first user is not admin #776
- 🐛(admin) fix broken alias import #1021
## [1.22.0] - 2026-01-19
### Added
- ✨(front) create, manage & delete aliases
- ✨(domains) alias sorting and admin
- ✨(aliases) delete all aliases in one call #1002
### Fixed
- 🔒️(security) upgrade python version to fix vulnerability #1010
- 🐛(dimail) ignore oxadmin when importing mailboxes from dimail #986
- ✨(aliases) fix deleting single aliases #1002
### Changed
- 🐛(dimail) allow mailboxes and aliases to have the same local part #986
### Removed
- 🔥(plugins) remove CommuneCreation plugin
## [1.21.0] - 2025-12-05
- ✨(aliases) import existing aliases from dimail
- 🛂(permissions) return 404 to users with no access to domain #985
- ✨(aliases) can create, list and delete aliases #974
## [1.20.0] - 2025-10-22
- ✨(models) impose uniqueness on display name, to match ox's constraint
- 🐛(dimail) catch duplicate displayname error
- ✨(mailbox) synchronize password of newly created mailbox with Dimail's
## [1.19.1] - 2025-09-19
- 🐛(fix) add enabled update your mailbox
## [1.19.0] - 2025-09-03
- ✨(front) add modal update mailboxes #954
### Added
- ✨(api) update mailboxes #934
- ✨(api) give update rights to domain viewer on own mailbox #934
### Fixed
- 🐛(dimail) grab duplicate displayname error #961
### Changed
- 💥(sentry) remove `DJANGO_` before Sentry DSN env variable #957
## [1.18.2] - 2025-07-03
### Fixed
- 🐛(front) fix missing pagination mail domains #950
## [1.18.1] - 2025-07-02
### Fixed
- 🐛(front) fix missing pagination on mail domains #946
## [1.18.0] - 2025-06-30
### Added
- 🐛(front) fix missing pagination mail domains
- 🐛(front) fix button add mail domain
- ✨(teams) add matrix webhook for teams #904
- ✨(resource-server) add SCIM /Me endpoint #895
- 🔧(git) set LF line endings for all text files #928
### Changed
- 🧑‍💻(docker) split frontend to another file #924
### Fixed
- 🐛(webhook) handle user on different home server than room server
## [1.17.0] - 2025-06-11
### Added
- ✨(frontend) add crisp script #914
- ⚡️(fix) add error when mailbox create failed
- ✨(mailbox) allow to reset password on mailboxes #834
## [1.16.0] - 2025-05-05
### Added
- 🔧(sentry) add Celery beat task integration #892
### Changed
- ✨(uiv2) change mail domains
- 🛂(dimail) simplify interop with dimail
- ✨(mailbox) remove secondary email as required field
### Fixed
- 🔒️(drf) disable browsable HTML API renderer #897
## [1.15.0] - 2025-04-04
### Added
- 🧱(helm) add la-suite ingress path
- (backend) add django-lasuite dependency #858
- ✨(plugins) add endpoint to list siret of active organizations #771
- ✨(core) create AccountServiceAuthentication backend #771
- ✨(core) create AccountService model #771
- 🧱(helm) disable createsuperuser job by setting #863
- 🔒️(passwords) add validators for production #850
- ✨(domains) allow to re-run check on domain if status is failed
- ✨(organization) add `is_active` field
- ✨(domains) notify support when domain status changes #668
- ✨(domains) define domain check interval as a settings
- ✨(oidc) add simple introspection backend #832
- 🧑‍💻(tasks) run management commands #814
### Changed
- ♻️(plugins) rewrite plugin system as django app #844
- 🔒️(users) restrict listable users to same organization #846
### Fixed
- 🐛(dimail) enhance sentry log
- 🐛(oauth2) force JWT signed for /userinfo #804
- 🐛(front) disable retries in useQuery and useInfiniteQuery #818
## [1.14.1] - 2025-03-17
## [1.14.0] - 2025-03-17
### Added
- ✨(domains) enhance required action modal content
- ✨(domains) add periodic tasks to fetch domain status
- 🧑‍💻(docker) add celery beat to manage periodic tasks
- ✨(organization) add metadata field #790
- ⬆️(nginx) bump nginx-unprivileged to 1.27 #797
- ✨(teams) allow broadly available teams #796
- ✨(teams) update and enhance team invitation email
- ✨(api) define dimail timeout as a setting
- ✨(frontend) feature modal add new access role to domain
- ✨(api) allow invitations for domain management #708
### Fixed
- 🐛(oauth2) force JWT signed for /userinfo #804
- 🐛(oauth2) add ProConnect scopes #802
- 🐛(domains) use a dedicated mail to invite user to manage domain
- 🐛(mailbox) fix mailbox creation email language
## [1.13.1] - 2025-03-04
### Fixed
- 🐛(mailbox) fix migration to fill dn_email field
## [1.13.0] - 2025-03-04
### Added
- ✨(oidc) people as an identity provider #638
### Fixed
- 💄(domains) improve user experience and avoid repeat fix_domain operations
- 👽️(dimail) increase timeout value for check domain API call
- 🧱(helm) add resource-server ingress path #743
- 🌐(backend) synchronize translations with crowdin again
## [1.12.1] - 2025-02-20
### Fixed
- 👽️(dimail) increase timeout value for API calls
## [1.12.0] - 2025-02-18
### Added
- ✨(domains) allow user to re-run all fetch domain data from dimail
- ✨(domains) display DNS config expected for domain with required actions
- ✨(domains) check status after creation
- ✨(domains) display required actions to do on domain
- ✨(plugin) add CommuneCreation plugin with domain provisioning #658
- ✨(frontend) display action required status on domain
- ✨(domains) store last health check details on MailDomain
- ✨(domains) add support email field on domain
## [1.11.0] - 2025-02-07
### Added
- ✨(api) add count mailboxes to MailDomain serializer
- ✨(dimail) manage 'action required' status for MailDomain
- ✨(domains) add action required status on MailDomain
- ✨(dimail) send pending mailboxes upon domain activation
### Fixed
- ✨(auth) fix empty names from ProConnect #687
- 🚑️(teams) do not display add button when disallowed #676
- 🚑️(plugins) fix name from SIRET specific case #674
- 🐛(api) restrict mailbox sync to enabled domains
## [1.10.1] - 2025-01-27
### Added
- ✨(dimail) management command to fetch domain status
### Changed
- ✨(scripts) adapts release script after moving the deployment part
### Fixed
- 🐛(dimail) fix imported mailboxes should be enabled instead of pending #659
- ⚡️(api) add missing cache for stats endpoint
## [1.10.0] - 2025-01-21
### Added
- ✨(api) create stats endpoint
- ✨(teams) add Team dependencies #560
- ✨(organization) add admin action for plugin #640
- ✨(anct) fetch and display organization names of communes #583
- ✨(frontend) display email if no username #562
- 🧑‍💻(oidc) add ability to pull registration ID (e.g. SIRET) from OIDC #577
### Fixed
- 🐛(frontend) improve e2e tests avoiding race condition from mocks #641
- 🐛(backend) fix flaky test with search contact #605
- 🐛(backend) fix flaky test with team access #646
- 🧑‍💻(dimail) remove 'NoneType: None' log in debug mode
- 🐛(frontend) fix flaky e2e test #636
- 🐛(frontend) fix disable mailbox button display #631
- 🐛(backend) fix dimail call despite mailbox creation failure on our side
- 🧑‍💻(user) fix the User.language infinite migration #611
## [1.9.1] - 2024-12-18
## [1.9.0] - 2024-12-17
### Fixed
- 🐛(backend) fix manage roles on domain admin view
### Added
- ✨(backend) add admin action to check domain health
- ✨(dimail) check domain health
- ✨(frontend) disable mailbox and allow to create pending mailbox
- ✨(organizations) add siret to name conversion #584
- 💄(frontend) redirect home according to abilities #588
- ✨(maildomain_access) add API endpoint to search users #508
## [1.8.0] - 2024-12-12
### Added
- ✨(contacts) add "abilities" to API endpoint data #585
- ✨(contacts) allow filter list API with email
- ✨(contacts) return profile contact from same organization
- ✨(dimail) automate allows requests to dimail
- ✨(contacts) add notes & force full_name #565
### Changed
- ♻️(contacts) move user profile to contact #572
- ♻️(contacts) split api test module in actions #573
### Fixed
- 🐛(backend) fix manage roles on domain admin view
- 🐛(mailbox) fix activate mailbox feature
- 🔧(helm) fix the configuration environment #579
## [1.7.1] - 2024-11-28
## [1.7.0] - 2024-11-28
### Added
- ✨(mailbox) allow to activate mailbox
- ✨(mailbox) allow to disable mailbox
- ✨(backend) add ServiceProvider #522
- 💄(admin) allow header color customization #552
- ✨(organization) add API endpoints #551
### Fixed
- 🐛(admin) add organization on user #555
## [1.6.1] - 2024-11-22
### Fixed
- 🩹(mailbox) fix status of current mailboxes
- 🚑️(backend) fix claim contains non-user field #548
## [1.6.0] - 2024-11-20
### Removed
- 🔥(teams) remove search by trigram for team access and contact
### Added
- ✨(domains) allow creation of "pending" mailboxes
- ✨(teams) allow team management for team admins/owners #509
### Fixed
- 🔊(backend) update logger config to info #542
## [1.5.0] - 2024-11-14
### Removed
- ⬆️(dependencies) remove unneeded dependencies
- 🔥(teams) remove pagination of teams listing
- 🔥(teams) remove search users by trigram
- 🗃️(teams) remove `slug` field
### Added
- ✨(dimail) send domain creation requests to dimail #454
- ✨(dimail) synchronize mailboxes from dimail to our db #453
- ✨(ci) add security scan #429
- ✨(teams) register contacts on admin views
### Fixed
- 🐛(mail) fix display button on outlook
- 💚(ci) improve E2E tests #492
- 🔧(sentry) restore default integrations
- 🔇(backend) remove Sentry duplicated warning/errors
- 👷(ci) add sharding e2e tests #467
- 🐛(dimail) fix unexpected status_code for proper debug #454
## [1.4.1] - 2024-10-23
### Fixed
- 🚑️(frontend) fix MailDomainsLayout
## [1.4.0] - 2024-10-23
### Added
- ✨(frontend) add tabs inside #466
### Fixed
- ✏️(mail) fix typo into mailbox creation email
- 🐛(sentry) fix duplicated sentry errors #479
- 🐛(script) improve and fix release script
## [1.3.1] - 2024-10-18
## [1.3.0] - 2024-10-18
### Added
@@ -15,6 +433,7 @@ and this project adheres to
- ✨(api) add RELEASE version on config endpoint #459
- ✨(backend) manage roles on domain admin view
- ✨(frontend) show version number in footer #369
- 👔(backend) add Organization model
### Changed
@@ -104,11 +523,46 @@ and this project adheres to
- ✨(domains) create and manage domains on admin + API
- ✨(domains) mailbox creation + link to email provisioning API
[unreleased]: https://github.com/numerique-gouv/people/compare/v1.3.0...main
[1.3.0]: https://github.com/numerique-gouv/people/releases/v1.3.0
[1.2.1]: https://github.com/numerique-gouv/people/releases/v1.2.1
[1.2.0]: https://github.com/numerique-gouv/people/releases/v1.2.0
[1.1.0]: https://github.com/numerique-gouv/people/releases/v1.1.0
[1.0.2]: https://github.com/numerique-gouv/people/releases/v1.0.2
[1.0.1]: https://github.com/numerique-gouv/people/releases/v1.0.1
[1.0.0]: https://github.com/numerique-gouv/people/releases/v1.0.0
[unreleased]: https://github.com/suitenumerique/people/compare/v1.23.1...main
[1.23.1]: https://github.com/suitenumerique/people/releases/v1.23.1
[1.23.0]: https://github.com/suitenumerique/people/releases/v1.23.0
[1.22.2]: https://github.com/suitenumerique/people/releases/v1.22.2
[1.22.1]: https://github.com/suitenumerique/people/releases/v1.22.1
[1.22.0]: https://github.com/suitenumerique/people/releases/v1.22.0
[1.21.0]: https://github.com/suitenumerique/people/releases/v1.21.0
[1.20.0]: https://github.com/suitenumerique/people/releases/v1.20.0
[1.19.1]: https://github.com/suitenumerique/people/releases/v1.19.1
[1.19.0]: https://github.com/suitenumerique/people/releases/v1.19.0
[1.18.2]: https://github.com/suitenumerique/people/releases/v1.18.2
[1.18.1]: https://github.com/suitenumerique/people/releases/v1.18.1
[1.18.0]: https://github.com/suitenumerique/people/releases/v1.18.0
[1.17.0]: https://github.com/suitenumerique/people/releases/v1.17.0
[1.16.0]: https://github.com/suitenumerique/people/releases/v1.16.0
[1.15.0]: https://github.com/suitenumerique/people/releases/v1.15.0
[1.14.1]: https://github.com/suitenumerique/people/releases/v1.14.1
[1.14.0]: https://github.com/suitenumerique/people/releases/v1.14.0
[1.13.1]: https://github.com/suitenumerique/people/releases/v1.13.1
[1.13.0]: https://github.com/suitenumerique/people/releases/v1.13.0
[1.12.1]: https://github.com/suitenumerique/people/releases/v1.12.1
[1.12.0]: https://github.com/suitenumerique/people/releases/v1.12.0
[1.11.0]: https://github.com/suitenumerique/people/releases/v1.11.0
[1.10.1]: https://github.com/suitenumerique/people/releases/v1.10.1
[1.10.0]: https://github.com/suitenumerique/people/releases/v1.10.0
[1.9.1]: https://github.com/suitenumerique/people/releases/v1.9.1
[1.9.0]: https://github.com/suitenumerique/people/releases/v1.9.0
[1.8.0]: https://github.com/suitenumerique/people/releases/v1.8.0
[1.7.1]: https://github.com/suitenumerique/people/releases/v1.7.1
[1.7.0]: https://github.com/suitenumerique/people/releases/v1.7.0
[1.6.1]: https://github.com/suitenumerique/people/releases/v1.6.1
[1.6.0]: https://github.com/suitenumerique/people/releases/v1.6.0
[1.5.0]: https://github.com/suitenumerique/people/releases/v1.5.0
[1.4.1]: https://github.com/suitenumerique/people/releases/v1.4.1
[1.4.0]: https://github.com/suitenumerique/people/releases/v1.4.0
[1.3.1]: https://github.com/suitenumerique/people/releases/v1.3.1
[1.3.0]: https://github.com/suitenumerique/people/releases/v1.3.0
[1.2.1]: https://github.com/suitenumerique/people/releases/v1.2.1
[1.2.0]: https://github.com/suitenumerique/people/releases/v1.2.0
[1.1.0]: https://github.com/suitenumerique/people/releases/v1.1.0
[1.0.2]: https://github.com/suitenumerique/people/releases/v1.0.2
[1.0.1]: https://github.com/suitenumerique/people/releases/v1.0.1
[1.0.0]: https://github.com/suitenumerique/people/releases/v1.0.0
+109
View File
@@ -0,0 +1,109 @@
# Contributing to the Project
Thank you for taking the time to contribute! Please follow these guidelines to ensure a smooth and productive workflow. 🚀🚀🚀
To get started with the project, please refer to the [README.md](https://github.com/suitenumerique/docs/blob/main/README.md) for detailed instructions.
Please also check out our [dev handbook](https://suitenumerique.gitbook.io/handbook) to learn our best practices.
## Help us with translations
You can help us with translations on [Crowdin](https://crowdin.com/project/lasuite-people).
Your language is not there? Request it on our Crowdin page 😊.
## Creating an Issue
When creating an issue, please provide the following details:
1. **Title**: A concise and descriptive title for the issue.
2. **Description**: A detailed explanation of the issue, including relevant context or screenshots if applicable.
3. **Steps to Reproduce**: If the issue is a bug, include the steps needed to reproduce the problem.
4. **Expected vs. Actual Behavior**: Describe what you expected to happen and what actually happened.
5. **Labels**: Add appropriate labels to categorize the issue (e.g., bug, feature request, documentation).
## Selecting an issue
We use a [GitHub Project](https://github.com/orgs/suitenumerique/projects/1) in order to prioritize our workload.
Please check in priority the issues that are in the **TODO cette semaine** column.
## Commit Message Format
All commit messages must adhere to the following format:
`<gitmoji>(type) title description`
* <**gitmoji**>: Use a gitmoji to represent the purpose of the commit. For example, ✨ for adding a new feature or 🔥 for removing something, see the list here: <https://gitmoji.dev/>.
* **(type)**: Describe the type of change. Common types include `backend`, `frontend`, `CI`, `docker` etc...
* **title**: A short, descriptive title for the change, starting with a lowercase character.
* **description**: Include additional details about what was changed and why.
### Example Commit Message
```
✨(frontend) add user authentication logic
Implemented login and signup features, and integrated OAuth2 for social login.
```
## Changelog Update
Please add a line to the changelog describing your development. The changelog entry should include a brief summary of the changes, this helps in tracking changes effectively and keeping everyone informed. We usually include the title of the pull request, followed by the pull request ID to finish the log entry. The changelog line should be less than 80 characters in total.
### Example Changelog Message
```
## [Unreleased]
## Added
- ✨(frontend) add AI to the project #321
```
## Pull Requests
It is nice to add information about the purpose of the pull request to help reviewers understand the context and intent of the changes. If you can, add some pictures or a small video to show the changes.
### Don't forget to:
- check your commits
- check the linting: `make lint && make frontend-lint`
- check the tests: `make test`
- add a changelog entry
- squash your commits
- rebase your branch on the latest `main` branch before pushing your changes `git pull --rebase origin main`
### Process to have a nice commit history
In the life time of your PR, you may need to add commits to fix things or add new features.
Commit after commit, your PR will be full of commits but you have to clean it up with the following commands before merging on `main`:
Gradually you can use `--fixup` to add commits to some of previous commit ( for example 1234567890).
```
git commit --fixup=1234567890
```
Then, you can squash your commits with the following command:
```
git rebase --autosquash -i -r HEAD~<number-of-commits>
```
Or you can use:
```
git rebase -i HEAD~<number-of-commits>
```
and move, squash and/or rename your commits manually. You can squash them with previous commit replacing the `pick` by `f`. You can rename them with replacing the `pick` by `r`.
Tada! You have a clean commit history.
Once all the required tests have passed, you can request a review from the project maintainers.
## Code Style
Please maintain consistency in code style. Run any linting tools available to make sure the code is clean and follows the project's conventions.
## Tests
Make sure that all new features or fixes have corresponding tests. Run the test suite before pushing your changes to ensure that nothing is broken.
## Asking for Help
If you need any help while contributing, feel free to open a discussion or ask for guidance in the issue tracker. We are more than happy to assist!
Thank you for your contributions! 👍
+55 -100
View File
@@ -1,79 +1,39 @@
# Django People
# ---- base image to inherit from ----
FROM python:3.10-slim-bullseye as base
# Upgrade pip to its latest release to speed up dependencies installation
RUN python -m pip install --upgrade pip
FROM python:3.14.2-alpine AS base
# Upgrade system packages to install security updates
RUN apt-get update && \
apt-get -y upgrade && \
rm -rf /var/lib/apt/lists/*
### ---- Front-end dependencies image ----
FROM node:20 as frontend-deps
WORKDIR /deps
COPY ./src/frontend/package.json ./package.json
COPY ./src/frontend/yarn.lock ./yarn.lock
COPY ./src/frontend/apps/desk/package.json ./apps/desk/package.json
COPY ./src/frontend/packages/i18n/package.json ./packages/i18n/package.json
COPY ./src/frontend/packages/eslint-config-people/package.json ./packages/eslint-config-people/package.json
RUN yarn --frozen-lockfile
### ---- Front-end builder dev image ----
FROM node:20 as frontend-builder-dev
WORKDIR /builder
COPY --from=frontend-deps /deps/node_modules ./node_modules
COPY ./src/frontend .
WORKDIR ./apps/desk
### ---- Front-end builder image ----
FROM frontend-builder-dev as frontend-builder
RUN yarn build
# ---- Front-end image ----
FROM nginxinc/nginx-unprivileged:1.25 as frontend-production
# Un-privileged user running the application
ARG DOCKER_USER
USER ${DOCKER_USER}
COPY --from=frontend-builder \
/builder/apps/desk/out \
/usr/share/nginx/html
COPY ./src/frontend/apps/desk/conf/default.conf /etc/nginx/conf.d
# Copy entrypoint
COPY ./docker/files/usr/local/bin/entrypoint /usr/local/bin/entrypoint
ENTRYPOINT [ "/usr/local/bin/entrypoint" ]
CMD ["nginx", "-g", "daemon off;"]
RUN apk update && \
apk upgrade
# ---- Back-end builder image ----
FROM base as back-builder
FROM base AS back-builder
WORKDIR /builder
ENV UV_COMPILE_BYTECODE=1
ENV UV_LINK_MODE=copy
# Copy required python dependencies
COPY ./src/backend /builder
# Disable Python downloads, because we want to use the system interpreter
# across both images. If using a managed Python version, it needs to be
# copied from the build image into the final image;
ENV UV_PYTHON_DOWNLOADS=0
RUN mkdir /install && \
pip install --prefix=/install .
# install uv
COPY --from=ghcr.io/astral-sh/uv:0.9.10 /uv /uvx /bin/
WORKDIR /app
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,source=src/backend/uv.lock,target=uv.lock \
--mount=type=bind,source=src/backend/pyproject.toml,target=pyproject.toml \
uv sync --locked --no-install-project --no-dev
COPY src/backend /app
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev
# ---- mails ----
FROM node:20 as mail-builder
FROM node:22 AS mail-builder
COPY ./src/mail /mail/app
@@ -84,24 +44,21 @@ RUN yarn install --frozen-lockfile && \
# ---- static link collector ----
FROM base as link-collector
FROM base AS link-collector
ARG PEOPLE_STATIC_ROOT=/data/static
# Install libpangocairo & rdfind
RUN apt-get update && \
apt-get install -y \
libpangocairo-1.0-0 \
rdfind && \
rm -rf /var/lib/apt/lists/*
# Copy installed python dependencies
COPY --from=back-builder /install /usr/local
# Copy people application (see .dockerignore)
COPY ./src/backend /app/
RUN apk add \
pango \
rdfind
WORKDIR /app
# Copy the application from the builder
COPY --from=back-builder /app /app
ENV PATH="/app/.venv/bin:$PATH"
# collectstatic
RUN DJANGO_CONFIGURATION=Build DJANGO_JWT_PRIVATE_SIGNING_KEY=Dummy \
python manage.py collectstatic --noinput
@@ -111,21 +68,18 @@ RUN DJANGO_CONFIGURATION=Build DJANGO_JWT_PRIVATE_SIGNING_KEY=Dummy \
RUN rdfind -makesymlinks true -followsymlinks true -makeresultsfile false ${PEOPLE_STATIC_ROOT}
# ---- Core application image ----
FROM base as core
FROM base AS core
ENV PYTHONUNBUFFERED=1
# Install required system libs
RUN apt-get update && \
apt-get install -y \
gettext \
libcairo2 \
libffi-dev \
libgdk-pixbuf2.0-0 \
libpango-1.0-0 \
libpangocairo-1.0-0 \
shared-mime-info && \
rm -rf /var/lib/apt/lists/*
RUN apk add \
gettext \
cairo \
libffi-dev \
gdk-pixbuf \
pango \
shared-mime-info
# Copy entrypoint
COPY ./docker/files/usr/local/bin/entrypoint /usr/local/bin/entrypoint
@@ -135,34 +89,35 @@ COPY ./docker/files/usr/local/bin/entrypoint /usr/local/bin/entrypoint
# docker user (see entrypoint).
RUN chmod g=u /etc/passwd
# Copy installed python dependencies
COPY --from=back-builder /install /usr/local
# Copy people application (see .dockerignore)
COPY ./src/backend /app/
# Copy the application from the builder
COPY --from=back-builder /app /app
WORKDIR /app
# Ensure the uv venv is used at runtime
ENV PATH="/app/.venv/bin:$PATH"
# Generate compiled translation messages
RUN DJANGO_CONFIGURATION=Build \
python manage.py compilemessages --ignore=".venv/**/*"
# We wrap commands run in this container by the following entrypoint that
# creates a user on-the-fly with the container user ID (see USER) and root group
# ID.
ENTRYPOINT [ "/usr/local/bin/entrypoint" ]
# ---- Development image ----
FROM core as backend-development
FROM core AS backend-development
# Switch back to the root user to install development dependencies
USER root:root
# Install psql
RUN apt-get update && \
apt-get install -y postgresql-client && \
rm -rf /var/lib/apt/lists/*
RUN apk add postgresql-client
# Uninstall people and re-install it in editable mode along with development
# dependencies
RUN pip uninstall -y people
RUN pip install -e .[dev]
# Install development dependencies
RUN --mount=from=ghcr.io/astral-sh/uv:0.9.10,source=/uv,target=/bin/uv \
uv sync --all-extras --locked
# Restore the un-privileged user running the application
ARG DOCKER_USER
@@ -177,7 +132,7 @@ ENV DB_HOST=postgresql \
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
# ---- Production image ----
FROM core as backend-production
FROM core AS backend-production
ARG PEOPLE_STATIC_ROOT=/data/static
+112 -36
View File
@@ -41,11 +41,9 @@ DOCKER_USER = $(DOCKER_UID):$(DOCKER_GID)
COMPOSE = DOCKER_USER=$(DOCKER_USER) docker compose
COMPOSE_EXEC = $(COMPOSE) exec
COMPOSE_EXEC_APP = $(COMPOSE_EXEC) app-dev
COMPOSE_RUN = $(COMPOSE) run --rm
COMPOSE_RUN = $(COMPOSE) run --rm --no-deps
COMPOSE_RUN_APP = $(COMPOSE_RUN) app-dev
COMPOSE_RUN_CROWDIN = $(COMPOSE_RUN) crowdin crowdin
WAIT_DB = @$(COMPOSE_RUN) dockerize -wait tcp://$(DB_HOST):$(DB_PORT) -timeout 60s
WAIT_KC_DB = $(COMPOSE_RUN) dockerize -wait tcp://kc_postgresql:5432 -timeout 60s
# -- Backend
MANAGE = $(COMPOSE_RUN_APP) python manage.py
@@ -72,29 +70,45 @@ data/static:
create-env-files: ## Copy the dist env files to env files
create-env-files: \
env.d/development/common \
env.d/development/france \
env.d/development/crowdin \
env.d/development/postgresql \
env.d/development/kc_postgresql
.PHONY: create-env-files
add-dev-rsa-private-key-to-env: ## Add a generated RSA private key to the env file
@echo "Generating RSA private key PEM for development..."
@mkdir -p env.d/development/rsa
@openssl genrsa -out env.d/development/rsa/private.pem 2048
@echo -n "\nOAUTH2_PROVIDER_OIDC_RSA_PRIVATE_KEY=\"" >> env.d/development/common
@openssl rsa -in env.d/development/rsa/private.pem -outform PEM >> env.d/development/common
@echo "\"" >> env.d/development/common
@rm -rf env.d/development/rsa
.PHONY: add-dev-rsa-private-key-to-env
update-keycloak-realm-app: ## Create the Keycloak realm for the project
@echo "$(BOLD)Creating Keycloak realm for 'app'$(RESET)"
@sed -i 's|http://app-dev:8000|http://app:8000|g' ./docker/auth/realm.json
.PHONY: update-keycloak-realm-app
bootstrap: ## Prepare Docker images for the project and install frontend dependencies
bootstrap: \
data/media \
data/static \
create-env-files \
build \
run \
run-dev \
migrate \
back-i18n-compile \
mails-install \
mails-build \
dimail-setup-db \
install-front-desk
dimail-setup-db
.PHONY: bootstrap
# -- Docker/compose
build: ## build the app-dev container
@$(COMPOSE) build app-dev
@$(COMPOSE) build app-dev frontend-dev
@$(COMPOSE) build dimail
.PHONY: build
down: ## stop and remove containers, networks, images, and volumes
@@ -105,17 +119,14 @@ logs: ## display app-dev logs (follow mode)
@$(COMPOSE) logs -f app-dev
.PHONY: logs
run: ## start the wsgi (production) and development server
@$(COMPOSE) up --force-recreate -d nginx
@$(COMPOSE) up --force-recreate -d app-dev
@$(COMPOSE) up --force-recreate -d celery-dev
@$(COMPOSE) up --force-recreate -d keycloak
@$(COMPOSE) up -d dimail
@echo "Wait for postgresql to be up..."
@$(WAIT_KC_DB)
@$(WAIT_DB)
run: ## start the wsgi (production) and servers with production Docker images
@$(COMPOSE) up --force-recreate --detach app frontend celery celery-beat nginx maildev
.PHONY: run
run-dev: ## start the servers in development mode (watch) Docker images
@$(COMPOSE) up --force-recreate --detach app-dev dimail frontend-dev celery-dev celery-beat-dev nginx maildev
.PHONY: run-dev
status: ## an alias for "docker compose ps"
@$(COMPOSE) ps
.PHONY: status
@@ -129,13 +140,9 @@ stop: ## stop the development server using Docker
demo: ## flush db then create a demo for load testing purpose
@$(MAKE) resetdb
@$(MANAGE) create_demo
@$(MAKE) dimail-setup-db
.PHONY: demo
reset-dimail-container:
@$(COMPOSE) up --force-recreate -d dimail
@$(MAKE) setup-dimail-db
.PHONY: reset-dimail-container
# Nota bene: Black should come after isort just in case they don't agree...
lint: ## lint back-end python sources
@@ -160,6 +167,14 @@ lint-pylint: ## lint back-end python sources with pylint only on changed files f
bin/pylint --diff-only=origin/main
.PHONY: lint-pylint
lint-front: ## lint front-end sources with eslint
cd $(PATH_FRONT) && yarn lint
.PHONY: lint-front
lint-front-fix: ## fix front-end sources with eslint
cd $(PATH_FRONT) && yarn lint-fix
.PHONY: lint-front-fix
test: ## run project tests
@$(MAKE) test-back-parallel
.PHONY: test
@@ -180,22 +195,16 @@ test-coverage: ## compute, display and save test coverage
makemigrations: ## run django makemigrations for the people project.
@echo "$(BOLD)Running makemigrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) makemigrations $(ARGS)
.PHONY: makemigrations
migrate: ## run django migrations for the people project.
@echo "$(BOLD)Running migrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) migrate $(ARGS)
.PHONY: migrate
showmigrations: ## run django showmigrations for the people project.
@echo "$(BOLD)Running showmigrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) showmigrations $(ARGS)
.PHONY: showmigrations
@@ -206,10 +215,11 @@ superuser: ## Create an admin superuser with password "admin"
.PHONY: superuser
back-i18n-compile: ## compile the gettext files
@$(MANAGE) compilemessages --ignore="venv/**/*"
@$(MANAGE) compilemessages --ignore=".venv/**/*"
.PHONY: back-i18n-compile
back-i18n-generate: ## create the .pot files used for i18n
back-i18n-generate: mails-build crowdin-download-sources
@$(MANAGE) makemessages -a --keep-pot
.PHONY: back-i18n-generate
@@ -231,18 +241,21 @@ resetdb: ## flush database and create a superuser "admin"
.PHONY: resetdb
env.d/development/common:
cp -n env.d/development/common.dist env.d/development/common
cp --update=none env.d/development/common.dist env.d/development/common
env.d/development/france:
cp --update=none env.d/development/france.dist env.d/development/france
env.d/development/postgresql:
cp -n env.d/development/postgresql.dist env.d/development/postgresql
cp --update=none env.d/development/postgresql.dist env.d/development/postgresql
env.d/development/kc_postgresql:
cp -n env.d/development/kc_postgresql.dist env.d/development/kc_postgresql
cp --update=none env.d/development/kc_postgresql.dist env.d/development/kc_postgresql
# -- Internationalization
env.d/development/crowdin:
cp -n env.d/development/crowdin.dist env.d/development/crowdin
cp --update=none env.d/development/crowdin.dist env.d/development/crowdin
crowdin-download: ## Download translated message from Crowdin
@$(COMPOSE_RUN_CROWDIN) download -c crowdin/config.yml
@@ -264,6 +277,7 @@ i18n-compile: \
i18n-generate: ## create the .pot files and extract frontend messages
i18n-generate: \
crowdin-download-sources \
back-i18n-generate \
frontend-i18n-generate
.PHONY: i18n-generate
@@ -282,15 +296,21 @@ i18n-generate-and-upload: \
# -- INTEROPERABILTY
# -- Dimail configuration
dimail-setup-db:
@$(COMPOSE) up --force-recreate -d dimail
@echo "$(BOLD)Populating database of local dimail API container$(RESET)"
@$(MANAGE) setup_dimail_db
@$(MANAGE) setup_dimail_db --populate-from-people
.PHONY: dimail-setup-db
# -- Mail generator
mails-build: ## Convert mjml files to html and text
mails-clean-templates: ## Clean the generated mail templates directory
@echo "$(BOLD)Cleaning mail templates directory$(RESET)"
@rm -rf ./src/backend/core/templates/mail
@mkdir -p ./src/backend/core/templates/mail
.PHONY: mails-clean-templates
mails-build: mails-clean-templates ## Convert mjml files to html and text
@$(MAIL_YARN) build
.PHONY: mails-build
@@ -329,7 +349,7 @@ help:
# Front
install-front-desk: ## Install the frontend dependencies of app Desk
cd $(PATH_FRONT_DESK) && yarn
cd $(PATH_FRONT_DESK) && yarn && yarn playwright install chromium
.PHONY: install-front-desk
run-front-desk: ## Start app Desk
@@ -355,10 +375,66 @@ start-kind: ## Create the kubernetes cluster
./bin/start-kind.sh
.PHONY: start-kind
install-external-secrets: ## install the kubernetes secrets from Vaultwarden
./bin/install-external-secrets.sh
.PHONY: build-k8s-cluster
tilt-up: ## start tilt - k8s local development
tilt up -f ./bin/Tiltfile
.PHONY: tilt-up
start-tilt-keycloak: ## start the kubernetes cluster using kind, without Pro Connect for authentication, use keycloak
DEV_ENV=dev-keycloak tilt up -f ./bin/Tiltfile
.PHONY: build-k8s-cluster
release: ## helper for release and deployment
python scripts/release.py
.PHONY: release
install-secret: ## install the kubernetes secrets from Vaultwarden
if kubectl -n desk get secrets bitwarden-cli-desk; then \
echo "Secret already present"; \
else \
echo "Please provide the following information:"; \
read -p "Enter your vaultwarden email login: " LOGIN; \
read -p "Enter your vaultwarden password: " PASSWORD; \
read -p "Enter your vaultwarden server url: " URL; \
echo "\nCreate vaultwarden secret"; \
echo "apiVersion: v1" > /tmp/secret.yaml; \
echo "kind: Secret" >> /tmp/secret.yaml; \
echo "metadata:" >> /tmp/secret.yaml; \
echo " name: bitwarden-cli-desk" >> /tmp/secret.yaml; \
echo " namespace: desk" >> /tmp/secret.yaml; \
echo "type: Opaque" >> /tmp/secret.yaml; \
echo "stringData:" >> /tmp/secret.yaml; \
echo " BW_HOST: $$URL" >> /tmp/secret.yaml; \
echo " BW_PASSWORD: $$PASSWORD" >> /tmp/secret.yaml; \
echo " BW_USERNAME: $$LOGIN" >> /tmp/secret.yaml; \
kubectl -n desk apply -f /tmp/secret.yaml;\
rm -f /tmp/secret.yaml; \
fi; \
if kubectl get ns external-secrets; then \
echo "External secret already deployed"; \
else \
helm repo add external-secrets https://charts.external-secrets.io; \
helm upgrade --install external-secrets \
external-secrets/external-secrets \
-n external-secrets \
--create-namespace \
--set installCRDs=true; \
fi
.PHONY: build-k8s-cluster
fetch-domain-status:
@$(MANAGE) fetch_domain_status
.PHONY: fetch-domain-status
# -- Keycloak related
create-new-client: ## run the add-keycloak-client.sh script for keycloak.
@echo "$(BOLD)Running add-keycloak-client.sh$(RESET)"
@$(COMPOSE_RUN) \
-v ./scripts/keycloak/add-keycloak-client.sh:/tmp/add-keycloak-client.sh \
--entrypoint="/tmp/add-keycloak-client.sh" \
keycloak \
$(filter-out $@,$(MAKECMDGOALS))
.PHONY: create-new-client
+8 -2
View File
@@ -1,6 +1,6 @@
# People
People is an application to handle users and teams, and distribute permissions accross [La Suite](https://lasuite.numerique.gouv.fr/).
People is an application to handle users and teams, and distribute permissions across [La Suite](https://lasuite.numerique.gouv.fr/).
It is built on top of [Django Rest
Framework](https://www.django-rest-framework.org/).
@@ -33,7 +33,7 @@ The easiest way to start working on the project is to use GNU Make:
$ make bootstrap
```
This command builds the `app` container, installs dependencies, performs
This command builds the `app-dev` container, installs dependencies, performs
database migrations and compile translations. It's a good idea to use this
command each time you are pulling code from the project repository to avoid
dependency-related or migration-related issues.
@@ -46,6 +46,12 @@ Note that if you need to run them afterward, you can use the eponym Make rule:
$ make run
```
or if you want to run them in development mode (with live reloading):
```bash
$ make run-dev
```
You can check all available Make rules using:
```bash
+23
View File
@@ -0,0 +1,23 @@
# Security Policy
## Reporting a Vulnerability
Security is very important to us.
If you have any issue regarding security, please disclose the information responsibly submiting [this form](https://vdp.numerique.gouv.fr/p/Send-a-report?lang=en) and not by creating an issue on the repository. You can also email us at support-regie@numerique.gouv.fr
We appreciate your effort to make People more secure.
## Vulnerability disclosure policy
Working with security issues in an open source project can be challenging, as we are required to disclose potential problems that could be exploited by attackers. With this in mind, our security fix policy is as follows:
1. The Maintainers team will handle the fix as usual (Pull Request,
release).
2. In the release notes, we will include the identification numbers from the
GitHub Advisory Database (GHSA) and, if applicable, the Common Vulnerabilities
and Exposures (CVE) identifier for the vulnerability.
3. Once this grace period has passed, we will publish the vulnerability.
By adhering to this security policy, we aim to address security concerns
effectively and responsibly in our open source software project.
+46 -1
View File
@@ -29,7 +29,9 @@ docker_build(
]
)
k8s_yaml(local('cd ../src/helm && helmfile -n desk -e dev template .'))
# helmfile in docker mount the current working directory and the helmfile.yaml
# requires the keycloak config in another directory
k8s_yaml(local('cd .. && helmfile -n desk -e ${DEV_ENV:-dev} template --file ./src/helm/helmfile.yaml'))
migration = '''
set -eu
@@ -56,3 +58,46 @@ cmd_button('Migrate db',
icon_name='developer_board',
text='Run database migration',
)
# Command to reset DB
reset_db = '''
set -eu
# get k8s pod name from tilt resource name
POD_NAME="$(tilt get kubernetesdiscovery desk-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n desk exec "$POD_NAME" -- python manage.py flush --no-input
kubectl -n desk exec "$POD_NAME" -- python manage.py createsuperuser --username admin@example.com --password admin
'''
cmd_button('Reset DB',
argv=['sh', '-c', reset_db],
resource='desk-backend',
icon_name='developer_board',
text='Reset DB',
)
# Command to create demo data
populate_people_with_demo_data = '''
set -eu
# get k8s pod name from tilt resource name
POD_NAME="$(tilt get kubernetesdiscovery desk-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n desk exec "$POD_NAME" -- python manage.py create_demo --force
'''
cmd_button('Populate with demo data',
argv=['sh', '-c', populate_people_with_demo_data],
resource='desk-backend',
icon_name='developer_board',
text='Populate with demo data',
)
# Command to created domain/users/access from people to dimail
populate_dimail_from_people = '''
set -eu
# get k8s pod name from tilt resource name
POD_NAME="$(tilt get kubernetesdiscovery desk-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n desk exec "$POD_NAME" -- python manage.py setup_dimail_db --populate-from-people
'''
cmd_button('Populate dimail from people',
argv=['sh', '-c', populate_dimail_from_people],
resource='desk-backend',
icon_name='developer_board',
text='Populate dimail from people',
)
+17 -64
View File
@@ -5,7 +5,6 @@ set -eo pipefail
REPO_DIR="$(cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd)"
UNSET_USER=0
TERRAFORM_DIRECTORY="./env.d/terraform"
COMPOSE_FILE="${REPO_DIR}/docker-compose.yml"
COMPOSE_PROJECT="people"
@@ -65,6 +64,23 @@ function _dc_run() {
_docker_compose run --rm $user_args "$@"
}
# _dc_run_no_deps: wrap docker compose run command without dependencies
#
# usage: _dc_run_no_deps [options] [ARGS...]
#
# options: docker compose run command options
# ARGS : docker compose run command arguments
function _dc_run_no_deps() {
_set_user
user_args="--user=$USER_ID"
if [ -z $USER_ID ]; then
user_args=""
fi
_docker_compose run --no-deps --rm $user_args "$@"
}
# _dc_exec: wrap docker compose exec command
#
# usage: _dc_exec [options] [ARGS...]
@@ -92,66 +108,3 @@ function _dc_exec() {
function _django_manage() {
_dc_run "app-dev" python manage.py "$@"
}
# _set_openstack_project: select an OpenStack project from the openrc files defined in the
# terraform directory.
#
# usage: _set_openstack_project
#
# If necessary the script will prompt the user to choose a project from those available
function _set_openstack_project() {
declare prompt
declare -a projects
declare -i default=1
declare -i choice=0
declare -i n_projects
# List projects by looking in the "./env.d/terraform" directory
# and store them in an array
read -r -a projects <<< "$(
find "${TERRAFORM_DIRECTORY}" -maxdepth 1 -mindepth 1 -type d |
sed 's|'"${TERRAFORM_DIRECTORY}\/"'||' |
xargs
)"
nb_projects=${#projects[@]}
if [[ ${nb_projects} -le 0 ]]; then
echo "There are no OpenStack projects defined..." >&2
echo "To add one, create a subdirectory in \"${TERRAFORM_DIRECTORY}\" with the name" \
"of your project and copy your \"openrc.sh\" file into it." >&2
exit 10
fi
if [[ ${nb_projects} -gt 1 ]]; then
prompt="Select an OpenStack project to target:\\n"
for (( i=0; i<nb_projects; i++ )); do
prompt+="[$((i+1))] ${projects[$i]}"
if [[ $((i+1)) -eq ${default} ]]; then
prompt+=" (default)"
fi
prompt+="\\n"
done
prompt+="If your OpenStack project is not listed, add it to the \"env.d/terraform\" directory.\\n"
prompt+="Your choice: "
read -r -p "$(echo -e "${prompt}")" choice
if [[ ${choice} -gt nb_projects ]]; then
(>&2 echo "Invalid choice ${choice} (should be <= ${nb_projects})")
exit 11
fi
if [[ ${choice} -le 0 ]]; then
choice=${default}
fi
fi
project=${projects[$((choice-1))]}
# Check that the openrc.sh file exists for this project
if [ ! -f "${TERRAFORM_DIRECTORY}/${project}/openrc.sh" ]; then
(>&2 echo "Missing \"openrc.sh\" file in \"${TERRAFORM_DIRECTORY}/${project}\". Check documentation.")
exit 12
fi
echo "${project}"
}
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env bash
set -o errexit
CURRENT_DIR=$(pwd)
NAMESPACE=${1:-desk}
SECRET_NAME=${2:-bitwarden-cli-desk}
TEMP_SECRET_FILE=$(mktemp)
cleanup() {
rm -f "${TEMP_SECRET_FILE}"
}
trap cleanup EXIT
# Check if kubectl is available
check_prerequisites() {
if ! command -v kubectl &> /dev/null; then
echo "Error: kubectl is not installed or not in PATH"
exit 1
fi
}
# Check if secret already exists
check_secret_exists() {
kubectl -n "${NAMESPACE}" get secrets "${SECRET_NAME}" &> /dev/null
}
# Collect user input securely
get_user_input() {
echo "Please provide the following information:"
read -p "Enter your Vaultwarden email login: " LOGIN
read -s -p "Enter your Vaultwarden password: " PASSWORD
echo
read -p "Enter your Vaultwarden server url: " URL
}
# Create and apply the secret
create_secret() {
cat > "${TEMP_SECRET_FILE}" << EOF
apiVersion: v1
kind: Secret
metadata:
name: ${SECRET_NAME}
namespace: ${NAMESPACE}
type: Opaque
stringData:
BW_HOST: ${URL}
BW_PASSWORD: ${PASSWORD}
BW_USERNAME: ${LOGIN}
EOF
kubectl -n "${NAMESPACE}" apply -f "${TEMP_SECRET_FILE}"
}
# Install external-secrets using Helm
install_external_secrets() {
if ! kubectl get ns external-secrets &>/dev/null; then
echo "Installing external-secrets…"
helm repo add external-secrets https://charts.external-secrets.io
helm upgrade --install external-secrets \
external-secrets/external-secrets \
-n external-secrets \
--create-namespace \
--set installCRDs=true
else
echo "External secrets already deployed"
fi
}
main() {
check_prerequisites
if check_secret_exists; then
echo "Secret '${SECRET_NAME}' already present in namespace '${NAMESPACE}'"
exit 0
fi
echo -e ${TEMP_SECRET_FILE}
get_user_input
echo -e "\nCreating Vaultwarden secret…"
create_secret
install_external_secrets
echo "Secret installation completed successfully"
}
main "$@"
+1 -1
View File
@@ -35,4 +35,4 @@ fi
# Fix docker vs local path when project sources are mounted as a volume
read -ra paths <<< "$(echo "${paths[@]}" | sed "s|src/backend/||g")"
_dc_run app-dev pylint "${paths[@]}" "${args[@]}"
_dc_run_no_deps app-dev pylint "${paths[@]}" "${args[@]}"
Regular → Executable
+2 -101
View File
@@ -1,102 +1,3 @@
#!/bin/sh
set -o errexit
#!/usr/bin/env bash
CURRENT_DIR=$(pwd)
# 0. Create ca
echo "0. Create ca"
mkcert -install
cd /tmp
mkcert "127.0.0.1.nip.io" "*.127.0.0.1.nip.io"
cd $CURRENT_DIR
# 1. Create registry container unless it already exists
echo "1. Create registry container unless it already exists"
reg_name='kind-registry'
reg_port='5001'
if [ "$(docker inspect -f '{{.State.Running}}' "${reg_name}" 2>/dev/null || true)" != 'true' ]; then
docker run \
-d --restart=always -p "127.0.0.1:${reg_port}:5000" --network bridge --name "${reg_name}" \
registry:2
fi
# 2. Create kind cluster with containerd registry config dir enabled
echo "2. Create kind cluster with containerd registry config dir enabled"
# TODO: kind will eventually enable this by default and this patch will
# be unnecessary.
#
# See:
# https://github.com/kubernetes-sigs/kind/issues/2875
# https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration
# See: https://github.com/containerd/containerd/blob/main/docs/hosts.md
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"
nodes:
- role: control-plane
image: kindest/node:v1.27.3
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
extraPortMappings:
- containerPort: 80
hostPort: 80
protocol: TCP
- containerPort: 443
hostPort: 443
protocol: TCP
- role: worker
image: kindest/node:v1.27.3
- role: worker
image: kindest/node:v1.27.3
EOF
# 3. Add the registry config to the nodes
echo "3. Add the registry config to the nodes"
#
# This is necessary because localhost resolves to loopback addresses that are
# network-namespace local.
# In other words: localhost in the container is not localhost on the host.
#
# We want a consistent name that works from both ends, so we tell containerd to
# alias localhost:${reg_port} to the registry container when pulling images
REGISTRY_DIR="/etc/containerd/certs.d/localhost:${reg_port}"
for node in $(kind get nodes); do
docker exec "${node}" mkdir -p "${REGISTRY_DIR}"
cat <<EOF | docker exec -i "${node}" cp /dev/stdin "${REGISTRY_DIR}/hosts.toml"
[host."http://${reg_name}:5000"]
EOF
done
# 4. Connect the registry to the cluster network if not already connected
echo "4. Connect the registry to the cluster network if not already connected"
# This allows kind to bootstrap the network but ensures they're on the same network
if [ "$(docker inspect -f='{{json .NetworkSettings.Networks.kind}}' "${reg_name}")" = 'null' ]; then
docker network connect "kind" "${reg_name}"
fi
# 5. Document the local registry
echo "5. Document the local registry"
# https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: local-registry-hosting
namespace: kube-public
data:
localRegistryHosting.v1: |
host: "localhost:${reg_port}"
help: "https://kind.sigs.k8s.io/docs/user/local-registry/"
EOF
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
kubectl -n ingress-nginx create secret tls mkcert --key /tmp/127.0.0.1.nip.io+1-key.pem --cert /tmp/127.0.0.1.nip.io+1.pem
kubectl -n ingress-nginx patch deployments.apps ingress-nginx-controller --type 'json' -p '[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value":"--default-ssl-certificate=ingress-nginx/mkcert"}]'
curl https://raw.githubusercontent.com/numerique-gouv/tools/refs/heads/main/kind/create_cluster.sh | bash -s -- desk
-25
View File
@@ -1,25 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
project=$(_set_openstack_project)
echo "Using \"${project}\" project..."
source "${TERRAFORM_DIRECTORY}/${project}/openrc.sh"
# Run Terraform commands in the Hashicorp docker container via docker compose
# shellcheck disable=SC2068
DOCKER_USER="$(id -u):$(id -g)" \
PROJECT="${project}" \
docker compose run --rm \
-e OS_AUTH_URL \
-e OS_IDENTITY_API_VERSION \
-e OS_USER_DOMAIN_NAME \
-e OS_PROJECT_DOMAIN_NAME \
-e OS_TENANT_ID \
-e OS_TENANT_NAME \
-e OS_USERNAME \
-e OS_PASSWORD \
-e OS_REGION_NAME \
terraform-state "$@"
-26
View File
@@ -1,26 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
project=$(_set_openstack_project)
echo "Using \"${project}\" project..."
source "${TERRAFORM_DIRECTORY}/${project}/openrc.sh"
# Run Terraform commands in the Hashicorp docker container via docker compose
# shellcheck disable=SC2068
DOCKER_USER="$(id -u):$(id -g)" \
PROJECT="${project}" \
docker compose run --rm \
-e OS_AUTH_URL \
-e OS_IDENTITY_API_VERSION \
-e OS_USER_DOMAIN_NAME \
-e OS_PROJECT_DOMAIN_NAME \
-e OS_TENANT_ID \
-e OS_TENANT_NAME \
-e OS_USERNAME \
-e OS_PASSWORD \
-e OS_REGION_NAME \
-e TF_VAR_user_name \
terraform "$@"
+157 -48
View File
@@ -1,16 +1,22 @@
services:
postgresql:
image: postgres:16
platform: linux/amd64
env_file:
- env.d/development/postgresql
ports:
- "15432:5432"
healthcheck:
test: [ "CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}" ]
interval: 1s
timeout: 2s
retries: 300
redis:
image: redis:5
mailcatcher:
image: sj26/mailcatcher:latest
maildev:
image: maildev/maildev:latest
ports:
- "1081:1080"
@@ -22,11 +28,13 @@ services:
DOCKER_USER: ${DOCKER_USER:-1000}
user: ${DOCKER_USER:-1000}
image: people:backend-development
pull_policy: never
environment:
- PYLINTHOME=/app/.pylint.d
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/france
- env.d/development/postgresql
ports:
- "8071:8000"
@@ -34,11 +42,69 @@ services:
- ./src/backend:/app
- ./data/media:/data/media
- ./data/static:/data/static
- /app/.venv
depends_on:
- postgresql
- mailcatcher
- redis
postgresql:
condition: service_healthy
restart: true
maildev:
condition: service_started
redis:
condition: service_started
frontend-dev:
user: "${DOCKER_USER:-1000}"
build:
context: .
dockerfile: ./src/frontend/Dockerfile
target: frontend-dev
image: people:frontend-development
pull_policy: never
volumes:
- ./src/frontend:/home/frontend
- /home/frontend/node_modules
- /home/frontend/apps/desk/node_modules
ports:
- "3000:3000"
app:
build:
context: .
target: backend-production
args:
DOCKER_USER: ${DOCKER_USER:-1000}
user: ${DOCKER_USER:-1000}
image: people:backend-production
environment:
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/france
- env.d/development/postgresql
ports:
- "8071:8000"
volumes:
- ./data/media:/data/media
depends_on:
postgresql:
condition: service_healthy
restart: true
redis:
condition: service_started
frontend:
user: "${DOCKER_USER:-1000}"
build:
context: .
dockerfile: ./src/frontend/Dockerfile
target: frontend-production
args:
API_ORIGIN: "http://localhost:8071"
image: people:frontend-production
pull_policy: build
ports:
- "3000:3000"
celery-dev:
user: ${DOCKER_USER:-1000}
image: people:backend-development
@@ -52,27 +118,31 @@ services:
- ./src/backend:/app
- ./data/media:/data/media
- ./data/static:/data/static
- /app/.venv
depends_on:
- app-dev
postgresql:
condition: service_healthy
restart: true
app-dev:
condition: service_started
app:
build:
context: .
target: backend-production
args:
DOCKER_USER: ${DOCKER_USER:-1000}
celery-beat-dev:
user: ${DOCKER_USER:-1000}
image: people:backend-production
image: people:backend-development
command: ["celery", "-A", "people.celery_app", "beat", "-l", "DEBUG"]
environment:
- DJANGO_CONFIGURATION=Demo
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/postgresql
volumes:
- ./src/backend:/app
- ./data/media:/data/media
- ./data/static:/data/static
depends_on:
- postgresql
- redis
postgresql:
condition: service_healthy
restart: true
celery:
user: ${DOCKER_USER:-1000}
@@ -84,7 +154,29 @@ services:
- env.d/development/common
- env.d/development/postgresql
depends_on:
- app
postgresql:
condition: service_healthy
restart: true
app:
condition: service_started
celery-beat:
user: ${DOCKER_USER:-1000}
image: people:backend-production
command: ["celery", "-A", "people.celery_app", "beat", "-l", "INFO"]
environment:
- DJANGO_CONFIGURATION=Demo
env_file:
- env.d/development/common
- env.d/development/postgresql
volumes:
- ./src/backend:/app
- ./data/media:/data/media
- ./data/static:/data/static
depends_on:
postgresql:
condition: service_healthy
restart: true
nginx:
image: nginx:1.25
@@ -93,14 +185,12 @@ services:
volumes:
- ./docker/files/etc/nginx/conf.d:/etc/nginx/conf.d:ro
depends_on:
- app
- keycloak
dockerize:
image: jwilder/dockerize
keycloak:
condition: service_healthy
restart: true
crowdin:
image: crowdin/cli:3.16.0
image: crowdin/cli:4.6.1
volumes:
- ".:/app"
env_file:
@@ -109,40 +199,31 @@ services:
working_dir: /app
node:
image: node:18
image: node:22
user: "${DOCKER_USER:-1000}"
environment:
HOME: /tmp
volumes:
- ".:/app"
terraform-state:
image: hashicorp/terraform:1.6
environment:
- TF_WORKSPACE=${PROJECT:-} # avoid env conflict in local state
user: ${DOCKER_USER:-1000}
working_dir: /app
volumes:
- ./src/terraform/create_state_bucket:/app
terraform:
image: hashicorp/terraform:1.6
user: ${DOCKER_USER:-1000}
working_dir: /app
volumes:
- ./src/terraform:/app
kc_postgresql:
image: postgres:14.3
ports:
- "5433:5432"
env_file:
- env.d/development/kc_postgresql
image: postgres:14.3
ports:
- "5433:5432"
env_file:
- env.d/development/kc_postgresql
healthcheck:
test: [ "CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}" ]
interval: 1s
timeout: 2s
retries: 300
keycloak:
image: quay.io/keycloak/keycloak:20.0.1
volumes:
- ./docker/auth/realm.json:/opt/keycloak/data/import/realm.json
extra_hosts:
- "host.docker.internal:host-gateway"
command:
- start-dev
- --features=preview
@@ -152,6 +233,8 @@ services:
- --hostname-admin-url=http://localhost:8083/
- --hostname-strict=false
- --hostname-strict-https=false
- --health-enabled=true
- --metrics-enabled=true
environment:
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
@@ -162,15 +245,41 @@ services:
KC_DB_USERNAME: people
KC_DB_SCHEMA: public
PROXY_ADDRESS_FORWARDING: 'true'
healthcheck:
test: [ "CMD", "curl", "--head", "-fsS", "http://localhost:8080/health/ready" ]
interval: 1s
timeout: 2s
retries: 300
ports:
- "8080:8080"
depends_on:
- kc_postgresql
kc_postgresql:
condition: service_healthy
restart: true
dimail:
image: registry.mim-libre.fr/dimail/dimail-api:latest
entrypoint: /opt/dimail-api/start-dev.sh
image: registry.mim-libre.fr/dimail/dimail-api:v0.5.2
pull_policy: always
environment:
DIMAIL_MODE: FAKE
DIMAIL_JWT_SECRET: fake_jwt_secret
ports:
- "8001:8000"
flower-dev:
user: ${DOCKER_USER:-1000}
image: people:backend-development
command: ["celery", "-A", "people.celery_app", "flower", "--port=5555"]
environment:
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/postgresql
ports:
- "5555:5555"
volumes:
- ./src/backend:/app
depends_on:
- celery-dev
- redis
+454 -9
View File
@@ -26,7 +26,7 @@
"oauth2DeviceCodeLifespan": 600,
"oauth2DevicePollingInterval": 5,
"enabled": true,
"sslRequired": "external",
"sslRequired": "none",
"registrationAllowed": true,
"registrationEmailAsUsername": false,
"rememberMe": true,
@@ -50,6 +50,9 @@
"firstName": "John",
"lastName": "Doe",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
@@ -58,12 +61,32 @@
],
"realmRoles": ["user"]
},
{
"username": "e2e.marie",
"email": "marie.varzy@gmail.com",
"firstName": "Marie",
"lastName": "Delamairie",
"enabled": true,
"attributes": {
"siret": "21510339100011"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.marie"
}
],
"realmRoles": ["user"]
},
{
"username": "user-e2e-chromium",
"email": "user@chromium.e2e",
"email": "user-e2e-chromium@example.org",
"firstName": "E2E",
"lastName": "Chromium",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
@@ -74,10 +97,13 @@
},
{
"username": "user-e2e-webkit",
"email": "user@webkit.e2e",
"email": "user-e2e-webkit@example.org",
"firstName": "E2E",
"lastName": "Webkit",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
@@ -88,10 +114,13 @@
},
{
"username": "user-e2e-firefox",
"email": "user@firefox.e2e",
"email": "user-e2e-firefox@example.org",
"firstName": "E2E",
"lastName": "Firefox",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
@@ -99,6 +128,261 @@
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-member",
"email": "e2e.team-member@example.com",
"firstName": "E2E",
"lastName": "Group Member",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-member"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-administrator",
"email": "e2e.team-administrator@example.com",
"firstName": "E2E",
"lastName": "Group Administrator",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-administrator"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-owner",
"email": "e2e.team-owner@example.com",
"firstName": "E2E",
"lastName": "Group Owner",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-owner"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.mail-member",
"email": "e2e.mail-member@example.com",
"firstName": "E2E",
"lastName": "Mailbox Member",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.mail-member"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.mail-administrator",
"email": "e2e.mail-administrator@example.com",
"firstName": "E2E",
"lastName": "Mailbox Administrator",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.mail-administrator"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.mail-owner",
"email": "e2e.mail-owner@example.com",
"firstName": "E2E",
"lastName": "Mailbox Owner",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.mail-owner"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-member-mail-member",
"email": "e2e.team-member-mail-member@example.com",
"firstName": "E2E",
"lastName": "Group Member Mailbox Member",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-member-mail-member"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-member-mail-administrator",
"email": "e2e.team-member-mail-administrator@example.com",
"firstName": "E2E",
"lastName": "Group Member Mailbox Administrator",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-member-mail-administrator"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-member-mail-owner",
"email": "e2e.team-member-mail-owner@example.com",
"firstName": "E2E",
"lastName": "Group Member Mailbox Owner",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-member-mail-owner"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-administrator-mail-member",
"email": "e2e.team-administrator-mail-member@example.com",
"firstName": "E2E",
"lastName": "Group Administrator Mailbox Member",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-administrator-mail-member"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-administrator-mail-administrator",
"email": "e2e.team-administrator-mail-administrator@example.com",
"firstName": "E2E",
"lastName": "Group Administrator Mailbox Administrator",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-administrator-mail-administrator"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-administrator-mail-owner",
"email": "e2e.team-administrator-mail-owner@example.com",
"firstName": "E2E",
"lastName": "Group Administrator Mailbox Owner",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-administrator-mail-owner"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-owner-mail-member",
"email": "e2e.team-owner-mail-member@example.com",
"firstName": "E2E",
"lastName": "Group Owner Mailbox Member",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-owner-mail-member"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-owner-mail-administrator",
"email": "e2e.team-owner-mail-administrator@example.com",
"firstName": "E2E",
"lastName": "Group Owner Mailbox Administrator",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-owner-mail-administrator"
}
],
"realmRoles": ["user"]
},
{
"username": "e2e.team-owner-mail-owner",
"email": "e2e.team-owner-mail-owner@example.com",
"firstName": "E2E",
"lastName": "Mailbox Owner",
"enabled": true,
"attributes": {
"siret": "13002526500013"
},
"credentials": [
{
"type": "password",
"value": "password-e2e.team-owner-mail-owner"
}
],
"realmRoles": ["user"]
}
],
"roles": {
@@ -485,9 +769,17 @@
"webAuthnPolicyPasswordlessAvoidSameAuthenticatorRegister": false,
"webAuthnPolicyPasswordlessAcceptableAaguids": [],
"scopeMappings": [
{
"clientScope": "siret",
"roles": [
"user"
]
},
{
"clientScope": "offline_access",
"roles": ["offline_access"]
"roles": [
"offline_access"
]
}
],
"clientScopeMappings": {
@@ -737,6 +1029,7 @@
"acr",
"roles",
"profile",
"siret",
"email"
],
"optionalClientScopes": [
@@ -897,6 +1190,35 @@
}
]
},
{
"id": "eb220fbb-02ac-4105-95a3-727954f6565d",
"name": "siret",
"description": "siret",
"protocol": "openid-connect",
"attributes": {
"include.in.token.scope": "true",
"display.on.consent.screen": "false",
"gui.order": ""
},
"protocolMappers": [
{
"id": "333a4e89-9363-4c36-b56f-79c6b019c6c6",
"name": "siret",
"protocol": "openid-connect",
"protocolMapper": "oidc-usermodel-attribute-mapper",
"consentRequired": false,
"config": {
"aggregate.attrs": "false",
"userinfo.token.claim": "true",
"multivalued": "false",
"user.attribute": "siret",
"id.token.claim": "true",
"access.token.claim": "true",
"claim.name": "siret"
}
}
]
},
{
"id": "af52ccc3-4ecb-49b4-9a67-5d4172f16070",
"name": "role_list",
@@ -1363,7 +1685,8 @@
"email",
"roles",
"web-origins",
"acr"
"acr",
"siret"
],
"defaultOptionalClientScopes": [
"offline_access",
@@ -1386,8 +1709,130 @@
"enabledEventTypes": [],
"adminEventsEnabled": false,
"adminEventsDetailsEnabled": false,
"identityProviders": [],
"identityProviderMappers": [],
"identityProviders": [
{
"alias": "oidc-people-local",
"displayName": "People OIDC (local)",
"internalId": "47aa6d7c-8ac5-4178-934e-66f78e510ee4",
"providerId": "oidc",
"enabled": true,
"updateProfileFirstLoginMode": "on",
"trustEmail": false,
"storeToken": false,
"addReadTokenRoleOnCreate": false,
"authenticateByDefault": false,
"linkOnly": false,
"firstBrokerLoginFlowAlias": "first broker login",
"config": {
"hideOnLoginPage": "false",
"userInfoUrl": "http://app-dev:8000/o/userinfo/",
"validateSignature": "true",
"acceptsPromptNoneForwardFromClient": "false",
"clientId": "people-idp",
"tokenUrl": "http://app-dev:8000/o/token/",
"uiLocales": "false",
"jwksUrl": "http://app-dev:8000/o/.well-known/jwks.json",
"backchannelSupported": "false",
"issuer": "http://app-dev:8000/o",
"useJwksUrl": "true",
"loginHint": "true",
"pkceEnabled": "true",
"pkceMethod": "S256",
"authorizationUrl": "http://localhost:8071/o/authorize/",
"clientAuthMethod": "client_secret_post",
"disableUserInfo": "false",
"syncMode": "IMPORT",
"clientSecret": "local-tests-only",
"passMaxAge": "false",
"defaultScope": "openid given_name usual_name email siret",
"allowedClockSkew": "0"
}
}
],
"identityProviderMappers": [
{
"id": "e55dc88c-7bb5-46fb-95ad-1df701a96282",
"name": "Sub",
"identityProviderAlias": "oidc-people-local",
"identityProviderMapper": "oidc-username-idp-mapper",
"config": {
"template": "${CLAIM.sub}",
"are.claim.values.regex": "false",
"claims": "[{\"key\":\"\",\"value\":\"\"}]",
"syncMode": "FORCE",
"attributes": "[]",
"target": "BROKER_ID"
}
},
{
"id": "7e489676-8cba-49e4-aa1e-dcd1462d33f7",
"name": "given_name",
"identityProviderAlias": "oidc-people-local",
"identityProviderMapper": "hardcoded-attribute-idp-mapper",
"config": {
"claims": "[{\"key\":\"\",\"value\":\"\"}]",
"syncMode": "FORCE",
"are.claim.values.regex": "false",
"attributes": "[]",
"attribute": "firstName"
}
},
{
"id": "30b6b3bc-5738-4936-bf88-c540b8805998",
"name": "usual_name",
"identityProviderAlias": "oidc-people-local",
"identityProviderMapper": "oidc-user-attribute-idp-mapper",
"config": {
"template": "${ALIAS}.${CLAIM.preferred_username}",
"are.claim.values.regex": "false",
"claims": "[{\"key\":\"profile\",\"value\":\"lastName\"}]",
"syncMode": "FORCE",
"claim": "profile",
"user.attribute": "lastName",
"attributes": "[]",
"target": "LOCAL"
}
},
{
"id": "b67caa26-4571-4cfe-9c15-68e022645fc5",
"name": "Username",
"identityProviderAlias": "oidc-people-local",
"identityProviderMapper": "oidc-username-idp-mapper",
"config": {
"template": "${CLAIM.email | lowercase}",
"are.claim.values.regex": "false",
"claims": "[{\"key\":\"\",\"value\":\"\"}]",
"syncMode": "FORCE",
"attributes": "[]",
"target": "BROKER_USERNAME"
}
},
{
"id": "4eef21ce-b5f7-4753-bd58-4e50eb2b5f31",
"name": "Email",
"identityProviderAlias": "oidc-people-local",
"identityProviderMapper": "oidc-user-attribute-idp-mapper",
"config": {
"are.claim.values.regex": "false",
"claims": "[{\"key\":\"\",\"value\":\"\"}]",
"syncMode": "FORCE",
"claim": "email",
"user.attribute": "email",
"attributes": "[]"
}
},
{
"id": "084cdd0e-0794-4388-8474-84c9a7c1b9c8",
"name": "siret",
"identityProviderAlias": "oidc-people-local",
"identityProviderMapper": "oidc-user-attribute-idp-mapper",
"config": {
"syncMode": "FORCE",
"claim": "siret",
"user.attribute": "siret"
}
}
],
"components": {
"org.keycloak.services.clientregistration.policy.ClientRegistrationPolicy": [
{
@@ -1929,7 +2374,7 @@
"authenticatorConfig": "review profile config",
"authenticator": "idp-review-profile",
"authenticatorFlow": false,
"requirement": "REQUIRED",
"requirement": "DISABLED",
"priority": 10,
"autheticatorFlow": false,
"userSetupAllowed": false
+63
View File
@@ -0,0 +1,63 @@
# Internationalization (i18n)
The backend and the frontend of the application are internationalized.
This means that the application can be translated into different languages.
The application is currently available in English and French.
## Local setup
To be able to upload/retrieve translation files to/from Crowdin you need to
setup your local environment with the following environment variables:
```bash
cp ./env.d/development/crowdin.dist ./env.d/development/crowdin
```
Then fill "CROWDIN_API_TOKEN" with your token and "CROWDIN_PROJECT_ID" with the project id
in the `./env.d/development/crowdin` file.
NB: If you don't have a personal Crowdin API token, go to Crowdin interface to generate it:
Settings -> API -> Personal Access Tokens -> New token
This configuration file will be loaded by the crowdin docker instance.
## How to update the translations
### First way (safe way)
1/ Generate the translation files for both frontend and backend
```bash
make i18n-generate
```
Check locally everything is ok before uploading the files to the translation platform.
2/ Upload the files to the translation platform (crowdin):
```bash
make crowdin-upload
```
3/ Fill the missing translations on the translation platform:
=> [https://crowdin.com/project/lasuite-people](https://crowdin.com/project/lasuite-people)
4/ Download the translated files from the translation platform and compile them:
```bash
make i18n-download-and-compile
```
### Second way (faster way)
1/ Generate the translation files for both frontend and backend and upload them to the translation platform (crowdin):
```bash
make i18n-generate-and-upload
```
2/ Fill the missing translations on the translation platform:
=> [https://crowdin.com/project/lasuite-people](https://crowdin.com/project/lasuite-people)
3/ Download the translated files from the translation platform and compile them:
```bash
make i18n-download-and-compile
```
+100
View File
@@ -0,0 +1,100 @@
# People as an Identity Provider
The people project can be configured to act as an Identity Provider for an OIDC federation.
The model containing the "identity" in `people` is the `Mailbox` model.
The connection workflow looks like:
- The user tries to reach a service provider.
- The service provider redirects the user to OIDC federation (in dev we use Keycloak).
- The user asks (or is redirected) to the Identity Provider (people).
- The user enter their email and password.
- If successful, the user is redirected to the OIDC federation with a token.
- The federation make the same Authorization Flow as usual to authenticate on the Service Provider.
## Technical aspects
### The `Mailbox` model
The `Mailbox` model can behave like a usual Django user:
- It has a `password` field (and `last_login`)
- It is considered as `authenticated`.
- To be `active` it must have an `enabled` status.
- To be able to log in, it must also have a `MailboxDomain` with an `enabled` status and an associated `Organization`.
### The `MailboxModelBackend` authentication backend
This authentication backend only allow authentication with `email` and `password` with a `Mailbox` user.
This backend is combined with the `one_time_email_authenticated_session` middleware to restrict
the session to the OIDC login process.
A user connecting with this backend will only be able to access the `/o/authorize/` URL.
### The OIDC validators
There are two validators available:
- `BaseValidator`: This validator retrieves simple claims
- `ProConnectValidator`: This validator is a custom validator to allow ProConnect specific requirements.
## Configuration
The keycloak configuration is not described here, but you may find information in the code base.
### Django settings
The following settings are required to enable the OIDC Identity Provider feature:
- OAUTH2_PROVIDER_OIDC_ENABLED: True
- OAUTH2_PROVIDER_OIDC_RSA_PRIVATE_KEY: ...
Optional:
- OAUTH2_PROVIDER_VALIDATOR_CLASS: "mailbox_oauth2.validators.ProConnectValidator"
If the `OAUTH2_PROVIDER_VALIDATOR_CLASS` is set to the `ProConnectValidator`, the claims will be
automatically defined to match the ProConnect requirements.
### Django OIDC application
To enable an OIDC client to use people, you must declare it.
```python
from oauth2_provider.models import Application
application = Application(
client_id="people-idp",
client_secret="local-tests-only",
client_type=Application.CLIENT_CONFIDENTIAL,
authorization_grant_type=Application.GRANT_AUTHORIZATION_CODE,
name="People Identity Provider",
algorithm=Application.RS256_ALGORITHM,
redirect_uris="http://localhost:8083/realms/people/broker/oidc-people-local/endpoint",
skip_authorization=True,
)
application.clean()
application.save()
```
## Security concerns
### Failed login cooldown
To prevent brute force attacks, 5 failed login will trigger a cooldown of 5 minutes before being able to log in again.
### Password strength
There is currently no constraint on the password strength as it can only be defined by Django administrators,
and later we will generate it.
If at some point the user is allowed to set their password, we will need to enforce a password strength policy.
## What is missing (next steps)
- `Mailbox` password can only be set through Django Admin panel (or shell), the next step will be to generate
a secure password and send it or display it to the end user.
- `MailboxDomain` organization can only be set through Django Admin panel (or shell), we need to provide a
way to auto-select it or allow an administrator to do so.
+34 -19
View File
@@ -2,34 +2,49 @@
## What is dimail ?
The mailing solution provided in La Suite is [Open-XChange](https://www.open-xchange.com/) (OX).
OX not having a provisioning API, 'dimail-api' or 'dimail' was created to allow mail-provisioning through People.
The mailing solution provided in La Suite is [La Messagerie](https://webmail.numerique.gouv.fr/), using [Open-XChange](https://www.open-xchange.com/) (OX). OX not having a provisioning API, 'dimail-api' or 'dimail' was created to allow mail-provisioning through People.
API and its documentation can be found [here](https://api.dev.ox.numerique.gouv.fr/docs#/).
## Architectural links of dimail
## Use of dimail container
As dimail's primary goal is to act as an interface between People and OX, its architecture is similar to that of People. A series of requests are sent from People to dimail upon creating domains, users and accesses.
To ease local development, dimail provides a container that we embark in our docker-compose. In "FAKE" mode, it simulates all responses from Open Exchange.
Bootstraping with command `make bootstrap` creates a container and initializes its database.
Additional commands :
- Reset and populate the database with all the content of your People database with `dimail-setup-db`
## Architecture
### Domains
Upon creating a domain on People, the same domain is created on dimail and will undergo a series of checks. When all checks have passed, the domain is considered valid and mailboxes can be created on it.
Upon creating a domain on People, the same domain is created on dimail and will undergo a series of checks. When all checks have passed, the domain is considered enabled.
### Users
Domains in OX have a field called "context". "Contexts" are shared spaces between domains, allowing users to discover users not only on their domain but on their entire context.
> [!NOTE]
> Contexts are only implemented in La Messagerie and are not currently in use in People. Domains created via People are in their own context.
The ones issuing requests. Dimail users will reflect domains owners and administrators on People, for logging purposes and to allow direct use of dimail, if desired. User reconciliation is made on user uuid provided by ProConnect.
People users can have 3 levels of permissions on a domain:
- Viewers can
- see the domain's information
- list its mailboxes and managers
- Administrators can
- create mailboxes
- invite collaborators to manage domain
- change role of a viewer to administrators
- all of viewers permissions
- Owners can
- promote administrators owners and demote owners
- all of viewers and administrators' permissions
> [!NOTE]
> Contexts are only implemented in La Messagerie and are not currently in use in People. Domains created via People are in their own context.
### Accesses
As for People, an access - a permissions (or "allows" in dimail) - grants an user permission to create objects on a domain.
### Mailboxes
Permissions requests are sent automatically upon :
- dimail database initialisation:
+ permission for dimail user People to create users and domains
- domain creation :
+ permission for dimail user People to manage domain
+ permission for new owner on new domain
- user creation:
+ permission for People to manage user
- access creation, if owner or admin:
+ permission for this user to manage domain
Mailboxes can be created by a domain owners or administrators in People's domain tab.
On enabled domains, mailboxes are created at the same time on dimail (and a confirmation email is sent to the secondary email).
On pending/failed domains, mailboxes are only created locally with "pending" status and are sent to dimail upon domain's (re)activation.
On disabled domains, mailboxes creation is not allowed.
+75
View File
@@ -0,0 +1,75 @@
# Local development with Kubernetes
We use tilt to provide a local development environment for Kubernetes.
Tilt is a tool that helps you develop applications for Kubernetes.
It watches your files for changes, rebuilds your containers, and restarts your pods.
It's like having a conversation with your cluster.
This is particularly useful when working on integrations or to test your helm charts.
Otherwise, you can use your good old docker configuration as described in README.md.
## Prerequisites
This guide assumes you have the following tools installed:
- [Docker](https://docs.docker.com/get-docker/)
- [Kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
- [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/)
* [mkcert](https://github.com/FiloSottile/mkcert)
- [ctlptl](https://github.com/tilt-dev/ctlptl)
- [Tilt](https://docs.tilt.dev/install.html)
* [helm](https://helm.sh/docs/intro/install/)
* [helmfile](https://github.com/helmfile/helmfile)
* [secrets](https://github.com/jkroepke/helm-secrets/wiki/Installation)
[Install_prereq script](https://github.com/numerique-gouv/dk8s/blob/main/scripts/install-prereq.sh) (not tested).
### Helmfile in Docker
```bash
#!/bin/sh
docker run --rm --net=host \
-v "${HOME}/.kube:/root/.kube" \
-v "${HOME}/.config/helm:/root/.config/helm" \
-v "${HOME}/.minikube:/${HOME}/.minikube" \
-v "${PWD}:/wd" \
-e KUBECONFIG=/root/.kube/config \
--workdir /wd ghcr.io/helmfile/helmfile:v0.150.0 helmfile "$@"
```
## Create the kubernetes cluster
Run the following command to create a kubernetes cluster using kind:
```bash
make start-kind
# import your secrets from credentials manager
# ! don't forget "https" before your url
make install-external-secrets
```
That's it! You should now have a local development environment for Kubernetes.
## Start the application
```bash
# You can either start :
# ProConnect stack (but secrets must be set on your local cluster)
make tilt-up
# or standalone environment with keycloak
make start-tilt-keycloak
```
Access your application at https://desk.127.0.0.1.nip.io
## Management
To manage the cluster, you can use k9s.
## Next steps
- Add dimail to the local development environment
- Add a reset demo `cmd_button` to Tilt
+103 -73
View File
@@ -6,16 +6,56 @@ interact efficiently.
Let's see how we could interact with Django's shell to recreate David's environment in the app.
## Base contacts from the organization records
## Profile contact
David Bowman is an exemplary employee at Space Odyssey Corporation. His email is
`david.bowman@spaceodyssey.com` and he is registered in the organization's records via a base
contact as follows:
The system identifies Dave through the `sub` associated with his OIDC session.
The OIDC should also provide his email address.
Note: as some Identity Providers do not provide a constant `sub`, the system should be able to
fallback to the email address to find it the user is already in the database.
When Dave logs in for the first time, the system checks if he is already in the database. If not
a new user is created for him.
Users needs to be linked to an Organization (see [organizations.md](./organizations.md)).
```python
david_base_contact = Contact.objects.create(
space_odyssey, _created = Organization.objects.get_or_create(...)
david_user = User.objects.create(
organization=space_odyssey,
language="en-us",
timezone="America/Los_Angeles",
name="David Bowman",
sub="1234567890",
email="david.bowman@spaceodyssey.com",
)
```
The system then creates a profile contact for Dave, based on the information provided by the OIDC:
```python
david_profile_contact = Contact.objects.create(
owner=david_user,
user=david_user, # this means it's the user's profile contact
full_name="David Bowman",
short_name="David",
data={
"emails": [
{"type": "Work", "value": "david.bowman@spaceodyssey.com"},
],
},
)
```
**Future feature:**
Dave will be prompted to confirm the details of the base contact stored in the database
and fill the details for a better profile.
His profile contact will be updated with the new information.
```python
Contact.objects.filter(user=david_user, owner=david_user).update(
short_name="Dave",
data={
"emails": [
{"type": "Work", "value": "david.bowman@spaceodyssey.com"},
@@ -48,91 +88,81 @@ david_base_contact = Contact.objects.create(
)
```
When David logs-in to the People application for the first time using the corporation's OIDC
Single Sign-On service. A user is created for him on the fly by the system, together with an
identity record representing the OIDC session:
David's profile contact is available to all users of his company/organization.
```python
david_user = User.objects.create(
language="en-us",
timezone="America/Los_Angeles",
)
david_identity = Identity.objects.create(
"user": david_user,
"sub": "2a1b3c4d-5e6f-7a8b-9c0d-1e2f3a4b5c6d",
"email" : "david.bowman@spaceodyssey.com",
"is_main": True,
)
```
## Profile contact
The system identifies Dave through the email associated with his OIDC session and prompts him to
confirm the details of the base contact stored in the database.
When David confirms, giving an alternative short name that he prefers, a contact override is
created on top of the organization's base contact. This new contact is marked as David's profile
on the user:
```python
david_contact = Contact.objects.create(
base=david_base_contact,
owner=david_user,
full_name="David Bowman",
short_name="Dave",
data={}
)
david_user.profile_contact = david_contact
david_user.save()
```
If Dave had not had any existing contact in the organization's records, the profile contact would
have been created independently, without any connection to a base contact:
```python
david_contact = Contact.objects.create(
base=None,
owner=david_user,
full_name="David Bowman",
short_name="Dave",
data={}
)
```
Now, Dave feels like sharing his mobile phone number with his colleagues. He can do this
by editing his contact in the application:
```python
contact.data["phones"] = [
{"type": "Mobile", "value": "(123) 456-7890"},
]
contact.save()
```
## Contact override
During a Space conference he attended, Dave met Dr Ryan Stone, a medical engineer who gave him
her professional email address. Ryan is already present in the system but her email is missing.
her personal email address (because she will quit the company she works for). Ryan is already
present in the system.
Dave can add it to his private version of the contact:
```python
ryan_base_contact = Contact.objects.create(
full_name="Ryan Stone",
data={}
ryan_user = User.objects.create(
organization=space_odyssey,
language="en-us",
timezone="America/Los_Angeles",
name="Ryan Stone",
sub="0987654321",
email="ryan.stone@spaceodyssey.com",
)
ryan_contact = Contact.objects.create(
base=ryan_base_contact,
owner=david_user,
ryan_profile_contact = Contact.objects.create(
user=ryan_user,
owner=ryan_user,
full_name="Ryan Stone",
short_name="Dr Ryan",
data={
"emails": [
{"type": "Work", "value": "ryan.stone@hubblestation.com"},
],
}
},
)
david_ryan_contact = Contact.objects.create(
override=ryan_profile_contact,
owner=david_user,
full_name="Ryan Stone",
short_name="Dr Ryan",
data={
"emails": [
{"type": "Home", "value": "ryan@stone.com"},
],
},
)
```
## Personal contacts
Dave met a lot of people during the last Opensource Experience event. He can add them to his personal contacts:
```python
julie_personal_contact = Contact.objects.create(
owner=david_user,
full_name="Julie Powell",
short_name="Julie",
data={
"emails": [
{"type": "Work", "value": "julie.powell@example.com"},
],
"phones": [
{"type": "Work", "value": "(123) 456-7890"},
],
},
)
```
These contacts are only available to Dave.
## Contacts from shared directories
This is a future feature that will allow Dave to access contacts from shared directories.
Shared directories will provide contacts for all users from several organizations.
## Team Collaboration
Dave wants to form a team with Ryan and other colleagues to work together better on using the organization's digital tools for their projects.
+41
View File
@@ -0,0 +1,41 @@
# Organization model
## Purpose
The initial `Organization` model allows to regroup `User` and `Team` under a same object.
While the purpose was purely technical in the first place, few features emerged afterward.
## Link with the `User` model
Each user must have a "default" organization.
When a user logs in:
- The backend tries to get an existing organization from a "registration ID" or from the user's mail domain
- If the organization does not exist, it is created using the "registration ID" and fallbacks on the user's mail domain
- The user organization is set to this organization.
**Note:** While deploying the new code, we allow the already existing user to not have one.
## Link with the `Team` model
Each team must have an organization. This organization is not defined by the user, but automatically set
using the user's organization.
The team's organization does not restrict the users who can be added to the team.
There is currently no feature relying on the team organization.
**Note:** While deploying the new code, we allow the teams to not have one.
## Organization-related features
### Organization UI
A new interface can be created to allow users to see all members of an organization.
This could be used along with the contacts.
+68
View File
@@ -0,0 +1,68 @@
# Releasing a new version
Whenever we are cooking a new release (e.g. `4.18.1`) we should follow a standard procedure described below:
1. Checkout and pull changes from the `main` branch to ensure you have the latest updates.
```bash
git checkout main
git pull
```
2. The next steps are automated in the `scripts/release.py`, you can run it using:
```bash
make release
```
This script will ask you for the version you want to release and the kind of release (patch, minor, major). It will then:
- Create a new branch named: `release/4.18.1`.
- Bump the release number for backend and frontend project.
- Update the project's `Changelog` following the [keepachangelog](https://keepachangelog.com/en/0.3.0/) recommendations
- Commit your changes with the following format: 🔖 release emoji, type of release (patch/minor/patch) and release version:
```text
🔖(minor) release version 4.18.1
```
- Triggers Crowdin translation action and open related PR
- Open release PR. Wait for an approval from your peers and merge it.
> [!NOTE]
> It also open the PR for pre-prod deployment, see following section.
3. Following release script instructions,
- merge translation and release pulls requests
- tag and push your commit:
```bash
git tag v4.18.1 && git push origin tag v4.18.1
```
This triggers the CI building new Docker images. You can ensure images were successfully built on Docker Hub [here for back-end](https://hub.docker.com/r/lasuite/people-backend/tags) and [here for front-end](https://hub.docker.com/r/lasuite/people-frontend/tags).
# Deploying
## Staging
The `staging` platform is deployed automatically with every update of the `main` branch.
## Pre-prod and production
If you used the release script and had permission to push on [lasuite-deploiement](https://github.com/numerique-gouv/lasuite-deploiement), a deployment branch has been created. You can skip step 1.
Otherwise, for manual preprod and for production deployments :
1. Bump tag version for both front-end and back-end images in the `.gotmpl` file located in `manifests/<your-product>/env.d/<your-environment>/`,
2. Add optional new secrets and variables, if applicable
3. Create a pull request
4. Submit to approval and merge PR
The release is now done! 🎉
+28
View File
@@ -0,0 +1,28 @@
# ServiceProvider model
## Purpose
The `ServiceProvider` model represents a ... service provider, also known as "tools using some data from this project".
## Link with the `Organization` model
An organization can be linked to several service providers.
The goal here, is to allow users of an organization to have access, to a service provider or not.
The first implementation does not have any feature related, but the first feature will probably be
to list applications visible in the "all application menu" (aka "la gaufre").
## Link with the `Team` model
A team can be linked to several service providers.
This is used as a filter when a service provider calls the resource server, only the teams linked to
this service provider are returned. This is mandatory for data segregation: we don't want all service
providers to be able to list all data regarding other service providers.
## Limitations
There is currently no way to provision all the service providers automatically. So when a service provider
creates a team via the resource server, we create the `ServiceProvider` on the fly, without any understandable name.
This will need to be improved later.
+51
View File
@@ -0,0 +1,51 @@
# E2E tests
## Run E2E tests
``` bash
# you need the dockers to be up and running
make bootstrap
# you will need to have few accounts in the database
make demo FLUSH_ARGS='--no-input'
# run the tests
cd src/frontend/apps/e2e
yarn test:ui --workers=1
```
A new browser window will open and you will be able to run the tests.
## Available accounts
The `make demo` command creates the following accounts:
- `e2e.team-<role>@example.com` where `<role>` is one of `administrator`, `member`, `owner`:
this account only belong to a team with the specified role.
- `e2e.mail-<role>@example.com` where `<role>` is one of `administrator`, `member`, `owner`:
this account only have a mailbox with the specified role access.
- `e2e.team-<team_role>-mail-<domain_role>@example.com` with a combination of roles as for the
previous accounts.
For each account, the password is `password-e2e.<role>`, for instance `password-e2e.team-member`.
In the E2E tests you can use these accounts to benefit from there accesses,
using the `keyCloakSignIn(page, browserName, <account_name>)`. The account name is the
username without the prefix `e2e.`.
``` typescript jsx
await keyCloakSignIn(page, browserName, 'mail-owner');
```
The `keyCloakSignIn` function will sign in the user on Keycloak using the proper username and password.
.. note::
This only works because the OIDC setting is set to fallback on user email.
## Add a new account
In case you need to add a new account for specific tests you need:
- to create a new user with the same format in the backend database:
update `[create_demo.py](../src/backend/demo/management/commands/create_demo.py)`
- to create a new account in Keycloak: update [realm.json](../docker/auth/realm.json)
- if the keycloak was running locally, you need to destroy its database and
restart the database and the keycloak containers.
+5 -1
View File
@@ -3,6 +3,7 @@ DJANGO_ALLOWED_HOSTS=*
DJANGO_SECRET_KEY=ThisIsAnExampleKeyForDevPurposeOnly
DJANGO_SETTINGS_MODULE=people.settings
DJANGO_SUPERUSER_PASSWORD=admin
DJANGO_CORS_ALLOWED_ORIGINS=http://localhost:3000
# Python
PYTHONPATH=/app
@@ -10,7 +11,7 @@ PYTHONPATH=/app
# People settings
# Mail
DJANGO_EMAIL_HOST="mailcatcher"
DJANGO_EMAIL_HOST="maildev"
DJANGO_EMAIL_PORT=1025
# Backend url
@@ -65,3 +66,6 @@ IH5em4M/pHIcsqCi1qggBMbdvzHBUtC3R4sK0CpEFHlN+Y59aGazidcN2FPupNJv
MbyqKyC6DAzv4jEEhHaN7oY=
-----END PRIVATE KEY-----
"
# INTEROP
MAIL_PROVISIONING_API_CREDENTIALS="bGFfcmVnaWU6cGFzc3dvcmQ=" # Dev and test key for dimail interop
+7
View File
@@ -1,3 +1,10 @@
# For the CI job test-e2e
SUSTAINED_THROTTLE_RATES="200/hour"
BURST_THROTTLE_RATES="200/minute"
OIDC_ORGANIZATION_REGISTRATION_ID_FIELD="siret"
OAUTH2_PROVIDER_OIDC_ENABLED=True
OAUTH2_PROVIDER_VALIDATOR_CLASS="mailbox_oauth2.validators.ProConnectValidator"
INSTALLED_PLUGINS=plugins.la_suite
+1
View File
@@ -0,0 +1 @@
INSTALLED_PLUGINS=plugins.la_suite
+6 -1
View File
@@ -13,7 +13,12 @@
"enabled": false,
"groupName": "ignored js dependencies",
"matchManagers": ["npm"],
"matchPackageNames": ["fetch-mock", "node", "node-fetch", "eslint"]
"matchPackageNames": ["fetch-mock", "node", "node-fetch", "eslint", "@hookform/resolvers"]
},
{
"groupName": "docker-compose dependencies",
"matchManagers": ["docker-compose"],
"matchPackageNames": ["dimail-api"]
}
]
}
+2 -2
View File
@@ -1,10 +1,10 @@
#!/bin/bash
#!/usr/bin/env bash
mkdir -p "$(dirname -- "${BASH_SOURCE[0]}")/../.git/hooks/"
PRE_COMMIT_FILE="$(dirname -- "${BASH_SOURCE[0]}")/../.git/hooks/pre-commit"
cat <<'EOF' >$PRE_COMMIT_FILE
#!/bin/bash
#!/usr/bin/env bash
# directories containing potential secrets
DIRS="."
+149
View File
@@ -0,0 +1,149 @@
#!/bin/bash
# Script to add a new client to Keycloak using the kcadm.sh CLI
# Usage: ./add-keycloak-client.sh [client_id] [client_secret]
# Default values
CLIENT_ID=${1:-"some-client-id"}
CLIENT_SECRET=${2:-"ThisIsAnExampleKeyForDevPurposeOnly"}
KEYCLOAK_URL=${KEYCLOAK_URL:-"http://keycloak:8080"}
KEYCLOAK_ADMIN=${KEYCLOAK_ADMIN:-"admin"}
KEYCLOAK_ADMIN_PASSWORD=${KEYCLOAK_ADMIN_PASSWORD:-"admin"}
REALM=${REALM:-"people"}
# Check for kcadm.sh in common locations
KCADM_LOCATIONS=(
"/opt/keycloak/bin/kcadm.sh"
"/opt/jboss/keycloak/bin/kcadm.sh"
"/usr/local/bin/kcadm.sh"
"./bin/kcadm.sh"
"$(which kcadm.sh 2>/dev/null)"
)
KCADM=""
for loc in "${KCADM_LOCATIONS[@]}"; do
if [ -x "$loc" ]; then
KCADM="$loc"
break
fi
done
if [ -z "$KCADM" ]; then
echo "Error: kcadm.sh not found. Please specify its location manually."
echo "You can set the KCADM environment variable to the path of kcadm.sh"
exit 1
fi
echo "Using Keycloak Admin CLI at: $KCADM"
echo "Logging in to Keycloak at $KEYCLOAK_URL..."
# Login to Keycloak
$KCADM config credentials \
--server $KEYCLOAK_URL \
--realm master \
--user $KEYCLOAK_ADMIN \
--password $KEYCLOAK_ADMIN_PASSWORD
if [ $? -ne 0 ]; then
echo "Failed to login to Keycloak. Please check your credentials and try again."
exit 1
fi
echo "Successfully logged in to Keycloak."
echo "Creating new client '$CLIENT_ID' in realm '$REALM'..."
# Create a temporary JSON file with client configuration
CLIENT_JSON=$(mktemp)
cat > "$CLIENT_JSON" << EOF
{
"clientId": "$CLIENT_ID",
"name": "",
"description": "",
"rootUrl": "",
"adminUrl": "",
"baseUrl": "",
"surrogateAuthRequired": false,
"enabled": true,
"alwaysDisplayInConsole": false,
"clientAuthenticatorType": "client-secret",
"secret": "$CLIENT_SECRET",
"redirectUris": [
"http://localhost:8070/*",
"http://localhost:8071/*",
"http://localhost:3200/*",
"http://localhost:8088/*",
"http://localhost:3000/*"
],
"webOrigins": [
"http://localhost:3200",
"http://localhost:8088",
"http://localhost:8070",
"http://localhost:3000"
],
"notBefore": 0,
"bearerOnly": false,
"consentRequired": false,
"standardFlowEnabled": true,
"implicitFlowEnabled": false,
"directAccessGrantsEnabled": false,
"serviceAccountsEnabled": false,
"publicClient": false,
"frontchannelLogout": true,
"protocol": "openid-connect",
"attributes": {
"access.token.lifespan": "-1",
"client.secret.creation.time": "$(date +%s)",
"user.info.response.signature.alg": "RS256",
"post.logout.redirect.uris": "http://localhost:8070/*##http://localhost:3200/*##http://localhost:3000/*",
"oauth2.device.authorization.grant.enabled": "false",
"use.jwks.url": "false",
"backchannel.logout.revoke.offline.tokens": "false",
"use.refresh.tokens": "true",
"tls-client-certificate-bound-access-tokens": "false",
"oidc.ciba.grant.enabled": "false",
"backchannel.logout.session.required": "true",
"client_credentials.use_refresh_token": "false",
"acr.loa.map": "{}",
"require.pushed.authorization.requests": "false",
"display.on.consent.screen": "false",
"client.session.idle.timeout": "-1",
"token.response.type.bearer.lower-case": "false"
},
"authenticationFlowBindingOverrides": {},
"fullScopeAllowed": true,
"nodeReRegistrationTimeout": -1,
"defaultClientScopes": [
"web-origins",
"acr",
"roles",
"profile",
"email"
],
"optionalClientScopes": [
"address",
"phone",
"offline_access",
"microprofile-jwt"
]
}
EOF
# Create the client using kcadm.sh
$KCADM create clients -r "$REALM" -f "$CLIENT_JSON"
if [ $? -ne 0 ]; then
echo "Failed to create client. Check the error message above."
rm "$CLIENT_JSON"
exit 1
fi
echo "✅ Client '$CLIENT_ID' created successfully!"
echo " Client ID: $CLIENT_ID"
echo " Client Secret: $CLIENT_SECRET"
# Clean up temporary file
rm "$CLIENT_JSON"
# Display the created client
echo "Client details:"
$KCADM get clients -r "$REALM" --query "clientId=$CLIENT_ID"
+82 -32
View File
@@ -1,5 +1,8 @@
# pylint: disable=line-too-long
import datetime
import os
import re
import sys
from utils import run_command
@@ -17,24 +20,10 @@ def update_files(version):
lines = file.readlines()
for index, line in enumerate(lines):
if line.startswith("version = "):
last_version = line.split("=")[-1]
new_line = line.replace(last_version, f' "{version}"\n')
lines[index] = new_line
lines[index] = re.sub(r'\"(.*?)\"', f'"{version}"', line)
with open(path, 'w+') as file:
file.writelines(lines)
# helm files
sys.stdout.write("Update helm files...\n")
for env in ["preprod", "production"]:
path = f"src/helm/env.d/{env}/values.desk.yaml.gotmpl"
with open(path, 'r') as file:
lines = file.readlines()
for index, line in enumerate(lines):
if line.startswith(" tag: "):
lines[index] = f' tag: "v{version}"\n'
with open(path, 'w+') as file:
file.writelines(lines)
# frontend package.json
sys.stdout.write("Update package.json...\n")
files_to_modify = []
@@ -47,13 +36,24 @@ def update_files(version):
with open(path, 'r') as file:
lines = file.readlines()
for index, line in enumerate(lines):
if line.startswith(' "version": '):
lines[index] = f' "version": "{version}",\n'
if "version" in line:
lines[index] = re.sub(r'"version": \"(.*?)\"', f'"version": "{version}"', line)
with open(path, 'w+') as file:
file.writelines(lines)
return
def update_helm_files(path):
sys.stdout.write("Update helm files...\n")
with open(path, 'r') as file:
lines = file.readlines()
for index, line in enumerate(lines):
if "tag:" in line:
lines[index] = re.sub(r'\"(.*?)\"', f'"v{version}"', line)
with open(path, 'w+') as file:
file.writelines(lines)
def update_changelog(path, version):
"""Update changelog file with release info
"""
@@ -76,7 +76,46 @@ def update_changelog(path, version):
file.writelines(lines)
def prepare_release(version, kind):
def deployment_part(version):
""" Update helm file of preprod on deployment repository numerique-gouv/lasuite-deploiement
"""
# Move to lasuite-deploiement repository
try:
os.chdir('../lasuite-deploiement')
except FileNotFoundError:
sys.stdout.write("\033[1;31m[Error] You must have a clone of https://github.com/numerique-gouv/lasuite-deploiement\x1b[0m")
sys.stdout.write("\033[0;32mPlease clone it and re-run script with option --only-deployment-part\x1b[0m")
sys.stdout.write(" >>> python scripts/release.py --only-deployment-part \x1b[0m")
else:
run_command("git checkout main", shell=True)
run_command("git pull", shell=True)
deployment_branch = f"regie/{version}/preprod"
run_command(f"git checkout -b {deployment_branch}", shell=True)
run_command("git pull --rebase origin main", shell=True)
path = f"manifests/regie/env.d/preprod/values.desk.yaml.gotmpl"
update_helm_files(path)
run_command(f"git add {path}", shell=True)
message = f"""🔖(regie) bump preprod version to {version}"""
run_command(f"git commit -m '{message}'", shell=True)
confirm = input(f"""\033[0;32m
### DEPLOYMENT ###
NEXT COMMAND on lasuite-deploiement repository:
>> git push origin {deployment_branch}
Continue ? (y,n)
\x1b[0m""").strip()
if confirm == 'y':
run_command(f"git push origin {deployment_branch}", shell=True)
sys.stdout.write(f"""\033[1;34m
PLEASE DO THE FOLLOWING INSTRUCTIONS:
--> Check if you need to bump chart version in the manifests/regie/helmfile.yaml file
and settings in the manifests/regie/env.d/preprod/values.desk.yaml.gotmpl file
add commits into {deployment_branch} branch to do it if needed, then...
--> Please submit PR {deployment_branch} and merge code to main [https://github.com/numerique-gouv/lasuite-deploiement]
\x1b[0m""")
def project_part(version, kind):
"""Manage only la regie part with update of CHANGELOG, version files and tag"""
sys.stdout.write('Let\'s go to create branch to release\n')
branch_to_release = f"release/{version}"
run_command(f"git checkout -b {branch_to_release}", shell=True)
@@ -89,28 +128,39 @@ def prepare_release(version, kind):
run_command("git add src/", shell=True)
message = f"""🔖({RELEASE_KINDS[kind]}) release version {version}
Update all version files and changelog for {RELEASE_KINDS[kind]} release."""
Update all version files and changelog for {RELEASE_KINDS[kind]} release."""
run_command(f"git commit -m '{message}'", shell=True)
confirm = input(f"\nNEXT COMMAND: \n>> git push origin {branch_to_release}\nContinue ? (y,n) ")
confirm = input(f"""\033[0;32m
### RELEASE ###
NEXT COMMAND on current repository:
>> git push origin {branch_to_release}
Continue ? (y,n)
\x1b[0m""").strip()
if confirm == 'y':
run_command(f"git push origin {branch_to_release}", shell=True)
sys.stdout.write(f"""
PLEASE DO THE FOLLOWING INSTRUCTIONS:
--> Please submit PR and merge code to main than tag version
>> git tag -a v{version}
\033[1;34mPLEASE DO THE FOLLOWING INSTRUCTIONS:
--> A github action will download last translations from crowdin and create a PR (i18n/update-translations) to merge into the release branch.
Please wait for this, approve and merge this translations PR into the release branch {branch_to_release}.
--> Then please submit PR {branch_to_release} and merge code to main [https://github.com/suitenumerique/people/]
--> Then do:
>> git checkout main
>> git pull
>> git tag v{version}
>> git push origin v{version}
--> Please check docker image: https://hub.docker.com/r/lasuite/people-backend/tags
>> git tag -d preprod
>> git tag preprod
>> git push -f origin preprod
--> Now please generate release on github interface for the current tag v{version}
""")
--> Please check and wait for the docker image v{version} to be published here:
- https://hub.docker.com/r/lasuite/people-backend/tags
- https://hub.docker.com/r/lasuite/people-frontend/tags
--> Now please generate release on github interface for the current tag v{version}
\x1b[0m""")
if __name__ == "__main__":
version, kind = None, None
while not version:
version = input("Enter your release version:")
version = input("Enter your release version:").strip()
while kind not in RELEASE_KINDS:
kind = input("Enter kind of release (p:patch, m:minor, mj:major):")
prepare_release(version, kind)
kind = input("Enter kind of release (p:patch, m:minor, mj:major):").strip()
if "--only-deployment-part" not in sys.argv:
project_part(version, kind)
deployment_part(version)
Submodule secrets deleted from b7ab5f1411
+2 -1
View File
@@ -363,7 +363,8 @@ name-group=
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
# Ignore: private stuff and `Params` class from FactoryBoy
no-docstring-rgx=(^_|^Params$)
# List of decorators that produce properties, such as abc.abstractproperty. Add
# to this list to register other decorators that produce valid properties.
+1
View File
@@ -0,0 +1 @@
"""Root module."""
+30
View File
@@ -0,0 +1,30 @@
"""Global fixtures for the backend tests."""
import pytest
from urllib3.connectionpool import HTTPConnectionPool
@pytest.fixture(autouse=True)
def no_http_requests(monkeypatch):
"""
Prevents HTTP requests from being made during tests.
This is useful for tests that do not require actual HTTP requests
and helps to avoid network-related issues.
Credits: https://blog.jerrycodes.com/no-http-requests/
"""
allowed_hosts = {"localhost"}
original_urlopen = HTTPConnectionPool.urlopen
def urlopen_mock(self, method, url, *args, **kwargs):
if self.host in allowed_hosts:
return original_urlopen(self, method, url, *args, **kwargs)
raise RuntimeError(
f"The test was about to {method} {self.scheme}://{self.host}{url}"
)
monkeypatch.setattr(
"urllib3.connectionpool.HTTPConnectionPool.urlopen", urlopen_mock
)
+1
View File
@@ -0,0 +1 @@
"""Core root module."""
+145 -7
View File
@@ -1,12 +1,17 @@
"""Admin classes and registrations for People's core app."""
from django.contrib import admin
from django.conf import settings
from django.contrib import admin, messages
from django.contrib.auth import admin as auth_admin
from django.utils.translation import gettext_lazy as _
from treebeard.admin import TreeAdmin
from treebeard.forms import movenodeform_factory
from mailbox_manager.admin import MailDomainAccessInline
from . import models
from .plugins.registry import registry as plugin_hooks_registry
class TeamAccessInline(admin.TabularInline):
@@ -18,6 +23,15 @@ class TeamAccessInline(admin.TabularInline):
readonly_fields = ("created_at", "updated_at")
class OrganizationAccessInline(admin.TabularInline):
"""Inline admin class for organization accesses."""
autocomplete_fields = ["user", "organization"]
extra = 0
model = models.OrganizationAccess
readonly_fields = ("created_at", "updated_at")
class TeamWebhookInline(admin.TabularInline):
"""Inline admin class for team webhooks."""
@@ -31,6 +45,7 @@ class TeamWebhookInline(admin.TabularInline):
class UserAdmin(auth_admin.UserAdmin):
"""Admin class for the User model"""
autocomplete_fields = ["organization"]
fieldsets = (
(
None,
@@ -39,10 +54,21 @@ class UserAdmin(auth_admin.UserAdmin):
"id",
"sub",
"password",
"organization",
)
},
),
(
_("Personal info"),
{
"fields": (
"name",
"email",
"language",
"timezone",
)
},
),
(_("Personal info"), {"fields": ("name", "email", "language", "timezone")}),
(
_("Permissions"),
{
@@ -67,9 +93,10 @@ class UserAdmin(auth_admin.UserAdmin):
},
),
)
inlines = (TeamAccessInline, MailDomainAccessInline)
inlines = (TeamAccessInline, MailDomainAccessInline, OrganizationAccessInline)
list_display = (
"get_user",
"organization",
"created_at",
"updated_at",
"is_active",
@@ -97,18 +124,28 @@ class UserAdmin(auth_admin.UserAdmin):
get_user.short_description = _("User")
class TeamServiceProviderInline(admin.TabularInline):
"""Inline admin class for service providers."""
can_delete = False
model = models.Team.service_providers.through
extra = 0
@admin.register(models.Team)
class TeamAdmin(admin.ModelAdmin):
class TeamAdmin(TreeAdmin):
"""Team admin interface declaration."""
inlines = (TeamAccessInline, TeamWebhookInline)
form = movenodeform_factory(models.Team)
inlines = (TeamAccessInline, TeamWebhookInline, TeamServiceProviderInline)
exclude = ("service_providers",) # Handled by the inline
list_display = (
"name",
"slug",
"created_at",
"updated_at",
)
search_fields = ("name",)
readonly_fields = ("path", "depth", "numchild")
@admin.register(models.TeamAccess)
@@ -148,8 +185,8 @@ class InvitationAdmin(admin.ModelAdmin):
)
def get_readonly_fields(self, request, obj=None):
"""Mark all fields read only, i.e. disable update."""
if obj:
# all fields read only = disable update
return self.fields
return self.readonly_fields
@@ -162,5 +199,106 @@ class InvitationAdmin(admin.ModelAdmin):
return super().change_view(request, object_id, extra_context=extra_context)
def save_model(self, request, obj, form, change):
"""Fill in current logged-in user as issuer."""
obj.issuer = request.user
obj.save()
@admin.register(models.Contact)
class ContactAdmin(admin.ModelAdmin):
"""Contact admin interface declaration."""
list_display = (
"full_name",
"owner",
"override",
)
class OrganizationServiceProviderInline(admin.TabularInline):
"""Inline admin class for service providers."""
can_delete = False
model = models.Organization.service_providers.through
extra = 0
@admin.action(description=_("Run post creation plugins"), permissions=["change"])
def run_post_creation_plugins(modeladmin, request, queryset): # pylint: disable=unused-argument
"""Run the post creation plugins for the selected organizations."""
for organization in queryset:
plugin_hooks_registry.execute_hook("organization_created", organization)
messages.success(
request,
_("Post creation plugins have been run for the selected organizations."),
)
@admin.register(models.Organization)
class OrganizationAdmin(admin.ModelAdmin):
"""Admin interface for organizations."""
actions = [run_post_creation_plugins]
list_display = (
"name",
"is_active",
"created_at",
"updated_at",
)
list_filter = ("is_active",)
search_fields = ("name",)
inlines = (OrganizationAccessInline, OrganizationServiceProviderInline)
exclude = ("service_providers",) # Handled by the inline
def get_actions(self, request):
"""Adapt actions list to the context."""
actions = super().get_actions(request)
if not plugin_hooks_registry.get_callbacks("organization_created"):
actions.pop("run_post_creation_plugins", None)
return actions
@admin.register(models.OrganizationAccess)
class OrganizationAccessAdmin(admin.ModelAdmin):
"""Organization access admin interface declaration."""
autocomplete_fields = ("user", "organization")
list_display = (
"user",
"organization",
"role",
"created_at",
"updated_at",
)
@admin.register(models.ServiceProvider)
class ServiceProviderAdmin(admin.ModelAdmin):
"""Admin interface for service providers."""
list_display = (
"name",
"audience_id",
"created_at",
"updated_at",
)
search_fields = ("name", "audience_id")
readonly_fields = ("created_at", "updated_at")
@admin.register(models.AccountService)
class AccountServiceAdmin(admin.ModelAdmin):
"""Admin interface for account services."""
list_display = ("name", "created_at", "updated_at")
readonly_fields = ("api_key", "created_at", "updated_at")
def get_form(self, request, obj=None, change=False, **kwargs):
"""Add help text to the scopes field to provide list of available scopes."""
form = super().get_form(request, obj, change, **kwargs)
form.base_fields[
"scopes"
].help_text = f"Scopes define what the service can access. \
Available scopes: {', '.join(settings.ACCOUNT_SERVICE_SCOPES)}"
return form
+1
View File
@@ -0,0 +1 @@
"""People core client API endpoints"""
@@ -4,28 +4,43 @@ from rest_framework import exceptions, serializers
from timezone_field.rest_framework import TimeZoneSerializerField
from core import models
from core.models import ServiceProvider
class ContactSerializer(serializers.ModelSerializer):
"""Serialize contacts."""
abilities = serializers.SerializerMethodField()
class Meta:
model = models.Contact
fields = [
"id",
"base",
"abilities",
"override",
"data",
"full_name",
"notes",
"owner",
"short_name",
]
read_only_fields = ["id", "owner"]
read_only_fields = ["id", "owner", "abilities"]
extra_kwargs = {
"override": {"required": False},
}
def update(self, instance, validated_data):
"""Make "base" field readonly but only for update/patch."""
validated_data.pop("base", None)
"""Make "override" field readonly but only for update/patch."""
validated_data.pop("override", None)
return super().update(instance, validated_data)
def get_abilities(self, contact) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return contact.get_abilities(request.user)
return {}
class DynamicFieldsModelSerializer(serializers.ModelSerializer):
"""
@@ -34,6 +49,8 @@ class DynamicFieldsModelSerializer(serializers.ModelSerializer):
"""
def __init__(self, *args, **kwargs):
"""Pass arguments to superclass except 'fields', then drop fields not listed therein."""
# Don't pass the 'fields' arg up to the superclass
fields = kwargs.pop("fields", None)
@@ -48,12 +65,40 @@ class DynamicFieldsModelSerializer(serializers.ModelSerializer):
self.fields.pop(field_name)
class OrganizationSerializer(serializers.ModelSerializer):
"""Serialize organizations."""
abilities = serializers.SerializerMethodField()
class Meta:
model = models.Organization
fields = ["id", "name", "registration_id_list", "domain_list", "abilities"]
read_only_fields = ["id", "registration_id_list", "domain_list"]
def get_abilities(self, organization) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
if request:
return organization.get_abilities(request.user)
return {}
class UserOrganizationSerializer(serializers.ModelSerializer):
"""Serialize organizations for users."""
class Meta:
model = models.Organization
fields = ["id", "name", "registration_id_list"]
read_only_fields = ["id", "name", "registration_id_list"]
class UserSerializer(DynamicFieldsModelSerializer):
"""Serialize users."""
timezone = TimeZoneSerializerField(use_pytz=False, required=True)
email = serializers.ReadOnlyField()
name = serializers.ReadOnlyField()
organization = UserOrganizationSerializer(read_only=True)
class Meta:
model = models.User
@@ -62,6 +107,7 @@ class UserSerializer(DynamicFieldsModelSerializer):
"email",
"language",
"name",
"organization",
"timezone",
"is_device",
"is_staff",
@@ -69,6 +115,41 @@ class UserSerializer(DynamicFieldsModelSerializer):
read_only_fields = ["id", "name", "email", "is_device", "is_staff"]
class UserMeSerializer(UserSerializer):
"""
Serialize the current user.
Same as the `UserSerializer` but with abilities.
"""
abilities = serializers.SerializerMethodField()
organization = UserOrganizationSerializer(read_only=True)
class Meta:
model = models.User
fields = [
"email",
"id",
"is_device",
"is_staff",
"language",
"name",
"organization",
"timezone",
# added fields
"abilities",
]
read_only_fields = ["id", "name", "email", "is_device", "is_staff"]
def get_abilities(self, user: models.User) -> dict:
"""Return abilities of the logged-in user on the instance."""
if user != self.context["request"].user: # Should not happen
raise RuntimeError(
"UserMeSerializer.get_abilities: user is not the same as the request user",
)
return user.get_abilities()
class TeamAccessSerializer(serializers.ModelSerializer):
"""Serialize team accesses."""
@@ -170,28 +251,52 @@ class TeamSerializer(serializers.ModelSerializer):
"""Serialize teams."""
abilities = serializers.SerializerMethodField(read_only=True)
slug = serializers.SerializerMethodField()
is_visible_all_services = serializers.BooleanField(required=False, default=True)
service_providers = serializers.PrimaryKeyRelatedField(
queryset=ServiceProvider.objects.all(), many=True, required=False
)
class Meta:
model = models.Team
fields = [
"id",
"name",
"accesses",
"abilities",
"slug",
"accesses",
"created_at",
"depth",
"is_visible_all_services",
"name",
"numchild",
"path",
"service_providers",
"updated_at",
]
read_only_fields = [
"id",
"accesses",
"abilities",
"slug",
"accesses",
"created_at",
"depth",
"numchild",
"path",
"updated_at",
]
def create(self, validated_data):
"""Create a new team with organization enforcement."""
# When called as a resource server, we enforce the team service provider
if sp_audience := self.context.get("from_service_provider_audience", None):
service_provider, _created = models.ServiceProvider.objects.get_or_create(
audience_id=sp_audience
)
validated_data["service_providers"] = [service_provider]
# Note: this is not the purpose of this API to check the user has an organization
return super().create(
validated_data=validated_data
| {"organization_id": self.context["request"].user.organization_id}
)
def get_abilities(self, team) -> dict:
"""Return abilities of the logged-in user on the instance."""
request = self.context.get("request")
@@ -199,10 +304,6 @@ class TeamSerializer(serializers.ModelSerializer):
return team.get_abilities(request.user)
return {}
def get_slug(self, instance):
"""Return slug from the team's name."""
return instance.get_slug()
class InvitationSerializer(serializers.ModelSerializer):
"""Serialize invitations."""
@@ -237,3 +338,12 @@ class InvitationSerializer(serializers.ModelSerializer):
attrs["team_id"] = team_id
attrs["issuer"] = user
return attrs
class ServiceProviderSerializer(serializers.ModelSerializer):
"""Serialize service providers."""
class Meta:
model = models.ServiceProvider
fields = ["id", "audience_id", "name"]
read_only_fields = ["id", "audience_id"]
@@ -1,9 +1,15 @@
"""API endpoints"""
import datetime
import operator
from functools import reduce
from django.conf import settings
from django.contrib.postgres.search import TrigramSimilarity
from django.db.models import Func, Max, OuterRef, Q, Subquery, Value
from django.db.models import OuterRef, Q, Subquery, Value
from django.db.models.functions import Coalesce
from django.utils import timezone
from django.utils.decorators import method_decorator
from django.views.decorators.cache import cache_page
from rest_framework import (
decorators,
@@ -19,10 +25,12 @@ from rest_framework import (
from rest_framework.permissions import AllowAny
from core import models
from core.api import permissions
from core.api.client import serializers
from core.utils.raw_sql import gen_sql_filter_json_array
from . import permissions, serializers
SIMILARITY_THRESHOLD = 0.04
from mailbox_manager import enums
from mailbox_manager import models as domains_models
class NestedGenericViewSet(viewsets.GenericViewSet):
@@ -85,16 +93,23 @@ class SerializerPerActionMixin:
This mixin is useful to avoid to define a serializer class for each action in the
`get_serializer_class` method.
"""
serializer_classes: dict[str, type] = {}
default_serializer_class: type = None
Example:
```
class MyViewSet(SerializerPerActionMixin, viewsets.GenericViewSet):
serializer_class = MySerializer
list_serializer_class = MyListSerializer
retrieve_serializer_class = MyRetrieveSerializer
```
"""
def get_serializer_class(self):
"""
Return the serializer class to use depending on the action.
"""
return self.serializer_classes.get(self.action, self.default_serializer_class)
if serializer_class := getattr(self, f"{self.action}_serializer_class", None):
return serializer_class
return super().get_serializer_class()
class Pagination(pagination.PageNumberPagination):
@@ -130,39 +145,46 @@ class ContactViewSet(
):
"""Contact ViewSet"""
permission_classes = [permissions.IsOwnedOrPublic]
queryset = models.Contact.objects.all()
permission_classes = [permissions.AccessPermission]
queryset = models.Contact.objects.select_related("user", "owner").all()
serializer_class = serializers.ContactSerializer
throttle_classes = [BurstRateThrottle, SustainedRateThrottle]
ordering_fields = ["full_name", "short_name", "created_at"]
ordering = ["full_name"]
def list(self, request, *args, **kwargs):
"""Limit listed users by a query with throttle protection."""
user = self.request.user
queryset = self.filter_queryset(self.get_queryset())
# Exclude contacts that:
# List only contacts that:
queryset = queryset.filter(
# - belong to another user (keep public and owned contacts)
Q(owner__isnull=True) | Q(owner=user),
# - are profile contacts for a user
user__isnull=True,
# - are overriden base contacts
overriding_contacts__isnull=True,
# - is public (owner is None)
Q(owner__isnull=True)
# - are owned by the user
| Q(owner=user)
# - are profile contacts for a user from the same organization
| Q(user__organization_id=user.organization_id),
# - are not overriden by another contact
overridden_by__isnull=True,
)
# Search by case-insensitive and accent-insensitive trigram similarity
# Search by case-insensitive and accent-insensitive on:
# - full name
# - short name
# - email (from data `emails` field)
if query := self.request.GET.get("q", ""):
query = Func(Value(query), function="unaccent")
similarity = TrigramSimilarity(
Func("full_name", function="unaccent"),
query,
) + TrigramSimilarity(Func("short_name", function="unaccent"), query)
queryset = (
queryset.annotate(similarity=similarity)
.filter(
similarity__gte=0.05
) # Value determined by testing (test_api_contacts.py)
.order_by("-similarity")
queryset = queryset.filter(
Q(full_name__unaccent__icontains=query)
| Q(short_name__unaccent__icontains=query)
| Q(
id__in=gen_sql_filter_json_array(
queryset.model,
"data->'emails'",
"value",
query,
)
)
)
serializer = self.get_serializer(queryset, many=True)
@@ -175,21 +197,64 @@ class ContactViewSet(
return super().perform_create(serializer)
class OrganizationViewSet(
mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
viewsets.GenericViewSet,
):
"""
Organization ViewSet
GET /api/organizations/<organization_id>/
Return the organization details for the given id.
PUT /api/organizations/<organization_id>/
Update the organization details for the given id.
PATCH /api/organizations/<organization_id>/
Partially update the organization details for the given id.
"""
permission_classes = [permissions.AccessPermission]
queryset = models.Organization.objects.all()
serializer_class = serializers.OrganizationSerializer
throttle_classes = [BurstRateThrottle, SustainedRateThrottle]
def get_queryset(self):
"""Limit listed organizations to the one the user belongs to."""
return (
super()
.get_queryset()
.filter(pk=self.request.user.organization_id)
.annotate(
user_role=Subquery(
models.OrganizationAccess.objects.filter(
user=self.request.user, organization=OuterRef("pk")
).values("role")[:1]
)
)
)
class UserViewSet(
mixins.UpdateModelMixin, viewsets.GenericViewSet, mixins.ListModelMixin
SerializerPerActionMixin,
mixins.UpdateModelMixin,
viewsets.GenericViewSet,
mixins.ListModelMixin,
):
"""
User viewset for all interactions with user infos and teams.
GET /api/users/&q=query
Return a list of users whose email matches the query. Similarity is
calculated using trigram similarity, allowing for partial,
case-insensitive matches and accented queries.
Return a list of users whose email or name matches the query.
"""
permission_classes = [permissions.IsSelf]
queryset = models.User.objects.all().order_by("-created_at")
queryset = (
models.User.objects.select_related("organization").all().order_by("-created_at")
)
serializer_class = serializers.UserSerializer
get_me_serializer_class = serializers.UserMeSerializer
throttle_classes = [BurstRateThrottle, SustainedRateThrottle]
pagination_class = Pagination
@@ -198,31 +263,21 @@ class UserViewSet(
queryset = self.queryset
if self.action == "list":
# Exclude inactive contacts
# Filter active users
# and users from same organization
queryset = queryset.filter(
is_active=True,
is_active=True, organization_id=self.request.user.organization_id
)
# Exclude all users already in the given team
if team_id := self.request.GET.get("team_id", ""):
queryset = queryset.exclude(teams__id=team_id)
# Search by case-insensitive and accent-insensitive trigram similarity
# Search by case-insensitive and accent-insensitive
if query := self.request.GET.get("q", ""):
similarity = Max(
TrigramSimilarity(
Coalesce(Func("email", function="unaccent"), Value("")),
Func(Value(query), function="unaccent"),
)
+ TrigramSimilarity(
Coalesce(Func("name", function="unaccent"), Value("")),
Func(Value(query), function="unaccent"),
)
)
queryset = (
queryset.annotate(similarity=similarity)
.filter(similarity__gte=SIMILARITY_THRESHOLD)
.order_by("-similarity")
queryset = queryset.filter(
Q(name__unaccent__icontains=query)
| Q(email__unaccent__icontains=query)
)
return queryset
@@ -238,9 +293,7 @@ class UserViewSet(
Return information on currently logged user
"""
user = request.user
return response.Response(
self.serializer_class(user, context={"request": request}).data
)
return response.Response(self.get_serializer(user).data)
class TeamViewSet(
@@ -253,20 +306,57 @@ class TeamViewSet(
):
"""Team ViewSet"""
permission_classes = [permissions.AccessPermission]
permission_classes = [permissions.TeamPermission, permissions.AccessPermission]
serializer_class = serializers.TeamSerializer
filter_backends = [filters.OrderingFilter]
ordering_fields = ["created_at"]
ordering_fields = ["created_at", "name", "path"]
ordering = ["-created_at"]
queryset = models.Team.objects.all()
pagination_class = None
def get_queryset(self):
"""Custom queryset to get user related teams."""
teams_queryset = models.Team.objects.filter(
accesses__user=self.request.user,
)
depth_path = teams_queryset.values("depth", "path")
if not depth_path:
return models.Team.objects.none()
user_role_query = models.TeamAccess.objects.filter(
user=self.request.user, team=OuterRef("pk")
).values("role")[:1]
return models.Team.objects.filter(accesses__user=self.request.user).annotate(
user_role=Subquery(user_role_query)
return (
models.Team.objects.prefetch_related("accesses", "service_providers")
.filter(
reduce(
operator.or_,
(
Q(
# The team the user has access to
depth=d["depth"],
path=d["path"],
)
| Q(
# The parent team the user has access to
depth__lt=d["depth"],
path__startswith=d["path"][: models.Team.steplen],
organization_id=self.request.user.organization_id,
)
for d in depth_path
),
),
)
# Abilities are computed based on logged-in user's role for the team
# and if the user does not have access, it's ok to consider them as a member
# because it's a parent team.
.annotate(
user_role=Coalesce(
Subquery(user_role_query), Value(models.RoleChoices.MEMBER.value)
)
)
)
def perform_create(self, serializer):
@@ -340,6 +430,7 @@ class TeamAccessViewSet(
return context
def get_serializer_class(self):
"""Chooses list or detail serializer according to the action."""
if self.action in {"list", "retrieve"}:
return self.list_serializer_class
return self.detail_serializer_class
@@ -351,20 +442,9 @@ class TeamAccessViewSet(
if self.action in {"list", "retrieve"}:
if query := self.request.GET.get("q", ""):
similarity = Max(
TrigramSimilarity(
Coalesce(Func("user__email", function="unaccent"), Value("")),
Func(Value(query), function="unaccent"),
)
+ TrigramSimilarity(
Coalesce(Func("user__name", function="unaccent"), Value("")),
Func(Value(query), function="unaccent"),
)
)
queryset = (
queryset.annotate(similarity=similarity)
.filter(similarity__gte=SIMILARITY_THRESHOLD)
.order_by("-similarity")
queryset = queryset.filter(
Q(user__email__unaccent__icontains=query)
| Q(user__name__unaccent__icontains=query)
)
# Determine which role the logged-in user has in the team
@@ -506,9 +586,88 @@ class ConfigView(views.APIView):
GET /api/v1.0/config/
Return a dictionary of public settings.
"""
array_settings = ["LANGUAGES", "FEATURES", "RELEASE"]
array_settings = [
"LANGUAGES",
"FEATURES",
"RELEASE",
"COMMIT",
"CRISP_WEBSITE_ID",
]
dict_settings = {}
for setting in array_settings:
dict_settings[setting] = getattr(settings, setting)
return response.Response(dict_settings)
class StatView(views.APIView):
"""API ViewSet for sharing some public metrics."""
permission_classes = [AllowAny]
@method_decorator(cache_page(3600))
def get(self, request):
"""
GET /api/v1.0/stats/
Return a dictionary of public metrics.
"""
context = {
"total_users": models.User.objects.count(),
"mau": models.User.objects.filter(
last_login__gte=timezone.now() - datetime.timedelta(30)
).count(),
"teams": models.Team.objects.count(),
"active_domains": domains_models.MailDomain.objects.filter(
status=enums.MailDomainStatusChoices.ENABLED
).count(),
"mailboxes": domains_models.Mailbox.objects.count(),
"aliases": domains_models.Alias.objects.count(),
}
return response.Response(context)
class ServiceProviderFilter(filters.BaseFilterBackend):
"""
Filter service providers.
"""
def filter_queryset(self, request, queryset, view):
"""
Filter service providers by audience or name.
"""
if name := request.GET.get("name"):
queryset = queryset.filter(name__icontains=name)
if audience_id := request.GET.get("audience_id"):
queryset = queryset.filter(audience_id=audience_id)
return queryset
class ServiceProviderViewSet(
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
viewsets.GenericViewSet,
):
"""
API ViewSet for all interactions with service providers.
GET /api/v1.0/service-providers/
Return a list of service providers.
GET /api/v1.0/service-providers/<service_provider_id>/
Return a service provider.
"""
permission_classes = [permissions.IsAuthenticated]
queryset = models.ServiceProvider.objects.all()
serializer_class = serializers.ServiceProviderSerializer
throttle_classes = [BurstRateThrottle, SustainedRateThrottle]
pagination_class = Pagination
filter_backends = [filters.OrderingFilter, ServiceProviderFilter]
ordering = ["name"]
ordering_fields = ["name", "created_at"]
def get_queryset(self):
"""Filter the queryset to limit results to user's organization."""
queryset = super().get_queryset()
queryset = queryset.filter(organizations__id=self.request.user.organization_id)
return queryset
+44 -2
View File
@@ -4,6 +4,8 @@ from django.core import exceptions
from rest_framework import permissions
from core import models
class IsAuthenticated(permissions.BasePermission):
"""
@@ -12,13 +14,13 @@ class IsAuthenticated(permissions.BasePermission):
"""
def has_permission(self, request, view):
"""Check auth token first."""
return bool(request.auth) if request.auth else request.user.is_authenticated
class IsSelf(IsAuthenticated):
"""
Allows access only to authenticated users. Alternative method checking the presence
of the auth token to avoid hitting the database.
Allows access only to user's own data.
"""
def has_object_permission(self, request, view, obj):
@@ -53,3 +55,43 @@ class AccessPermission(IsAuthenticated):
"""Check permission for a given object."""
abilities = obj.get_abilities(request.user)
return abilities.get(request.method.lower(), False)
class TeamPermission(IsAuthenticated):
"""Permission class for team objects viewset."""
def has_permission(self, request, view):
"""Check permission only when the user tries to create a new team."""
if not super().has_permission(request, view):
return False
if request.method != "POST":
return True
abilities = request.user.get_abilities()
return abilities["teams"]["can_create"]
class TeamInvitationCreationPermission(IsAuthenticated):
"""Permission class that allows only team owners and admins to perform actions."""
def has_permission(self, request, view):
"""Check if user is authenticated and has required role for the team."""
if not super().has_permission(request, view):
return False
# Only check roles for edition operations
if request.method in permissions.SAFE_METHODS:
return True
team_id = view.kwargs.get("team_id")
if not team_id:
return False
team_access = models.TeamAccess.objects.filter(
team_id=team_id,
user=request.user,
role__in=[models.RoleChoices.OWNER, models.RoleChoices.ADMIN],
).exists()
return team_access
@@ -0,0 +1 @@
"""People core resource server API endpoints"""
@@ -0,0 +1 @@
"""SCIM compliant API for resource server"""
@@ -0,0 +1,16 @@
"""Exceptions for SCIM API."""
from django.conf import settings
from core.api.resource_server.scim.response import ScimJsonResponse
def scim_exception_handler(exc, _context):
"""Handle SCIM exceptions and return them in the correct format."""
data = {
"schemas": ["urn:ietf:params:scim:api:messages:2.0:Error"],
"status": str(exc.status_code),
"detail": str(exc.detail) if settings.DEBUG else "",
}
return ScimJsonResponse(data, status=exc.status_code)
@@ -0,0 +1,16 @@
"""SCIM API response classes."""
from rest_framework.response import Response
class ScimJsonResponse(Response):
"""
Custom JSON response class for SCIM API.
This class sets the content type to "application/json+scim" for SCIM
responses.
"""
def __init__(self, *args, **kwargs):
"""JSON response with enforced SCIM content type."""
super().__init__(*args, content_type="application/json+scim", **kwargs)
@@ -0,0 +1,90 @@
"""SCIM serializers for resource server API."""
from django.urls import reverse
from rest_framework import serializers
from core import models
class SCIMUserSerializer(serializers.ModelSerializer):
"""Serialize users in SCIM format."""
schemas = serializers.SerializerMethodField()
userName = serializers.CharField(source="sub")
displayName = serializers.CharField(source="name")
emails = serializers.SerializerMethodField()
meta = serializers.SerializerMethodField()
groups = serializers.SerializerMethodField()
active = serializers.BooleanField(source="is_active")
class Meta:
model = models.User
fields = [
"id",
"schemas",
"active",
"userName",
"displayName",
"emails",
"groups",
"meta",
]
read_only_fields = [
"id",
"schemas",
"active",
"userName",
"displayName",
"emails",
"groups",
"meta",
]
def get_schemas(self, _obj):
"""Return the SCIM schemas for the user."""
return ["urn:ietf:params:scim:schemas:core:2.0:User"]
def get_emails(self, obj):
"""Return the user's email as a list of email objects."""
if obj.email:
return [
{
"value": obj.email,
"primary": True,
"type": "work",
}
]
return []
def get_groups(self, obj):
"""
Return the groups the user belongs to.
WARNING: you need to prefetch the team accesses in the
viewset to avoid N+1 queries.
"""
return [
{
"value": str(team_access.team.external_id),
"display": team_access.team.name,
"type": "direct",
}
for team_access in obj.accesses.all()
]
def get_meta(self, obj):
"""Return metadata about the user."""
request = self.context.get("request")
location = (
f"{request.build_absolute_uri('/').rstrip('/')}{reverse('scim-me-list')}"
if request
else None
)
return {
"resourceType": "User",
"created": obj.created_at.isoformat(),
"lastModified": obj.updated_at.isoformat(),
"location": location,
}
@@ -0,0 +1,62 @@
"""Resource server SCIM API endpoints"""
from django.contrib.auth import get_user_model
from django.db.models import Prefetch, Q
from lasuite.oidc_resource_server.mixins import ResourceServerMixin
from rest_framework import (
viewsets,
)
from core.api import permissions
from core.models import TeamAccess
from . import serializers
from .exceptions import scim_exception_handler
from .response import ScimJsonResponse
User = get_user_model()
class MeViewSet(ResourceServerMixin, viewsets.ViewSet):
"""
SCIM-compliant ViewSet for the /Me endpoint.
This endpoint provides information about the currently authenticated user
in SCIM (System for Cross-domain Identity Management) format.
Features:
- Returns user details in SCIM format.
- Includes the user's teams, restricted to the audience.
Limitations:
- Does not currently support managing Team hierarchies.
Endpoint:
GET /resource-server/v1.0/scim/Me/
Retrieves the authenticated user's details and associated teams.
"""
permission_classes = [permissions.IsAuthenticated]
serializer_class = serializers.SCIMUserSerializer
def get_exception_handler(self):
"""Override the default exception handler to use SCIM-specific handling."""
return scim_exception_handler
def list(self, request, *args, **kwargs):
"""Return the current user's details in SCIM format."""
service_provider_audience = self._get_service_provider_audience()
user = User.objects.prefetch_related(
Prefetch(
"accesses",
queryset=TeamAccess.objects.select_related("team").filter(
Q(team__service_providers__audience_id=service_provider_audience)
| Q(team__is_visible_all_services=True)
),
)
).get(pk=request.user.pk)
serializer = self.serializer_class(user, context={"request": request})
return ScimJsonResponse(serializer.data)
@@ -0,0 +1,78 @@
"""Client serializers for the People core app resource server API."""
from rest_framework import serializers
from core import models
class TeamSerializer(serializers.ModelSerializer):
"""Serialize teams."""
class Meta:
model = models.Team
fields = [
"id",
"created_at",
"depth",
"is_visible_all_services",
"name",
"numchild",
"path",
"updated_at",
]
read_only_fields = [
"id",
"created_at",
"depth",
"numchild",
"path",
"updated_at",
]
def create(self, validated_data):
"""
Create a new team with organization enforcement.
In this context, called as a resource server,
the team service provider is enforced.
When the service provider audience is unknown it is created on the fly.
"""
sp_audience = self.context["from_service_provider_audience"]
service_provider, _created = models.ServiceProvider.objects.get_or_create(
audience_id=sp_audience
)
# Note: this is not the purpose of this API to check the user has an organization
return super().create(
validated_data=validated_data
| {
"organization_id": self.context["request"].user.organization_id,
"service_providers": [service_provider],
},
)
class InvitationSerializer(serializers.ModelSerializer):
"""Serialize invitations."""
class Meta:
model = models.Invitation
fields = ["id", "created_at", "email", "team", "role", "issuer", "is_expired"]
read_only_fields = ["id", "created_at", "team", "issuer", "is_expired"]
def validate(self, attrs):
"""Fill team and issuer from request."""
is_team_available_for_service_provider = models.Team.objects.filter(
id=self.context["team_id"],
service_providers__audience_id=self.context[
"from_service_provider_audience"
],
).exists()
if not is_team_available_for_service_provider:
raise serializers.ValidationError({"team": "Team not found."})
attrs["team_id"] = self.context["team_id"]
attrs["issuer"] = self.context["request"].user # User is authenticated
return attrs
@@ -0,0 +1,202 @@
"""Resource server API endpoints"""
import operator
from functools import reduce
from django.db.models import OuterRef, Prefetch, Q, Subquery, Value
from django.db.models.functions import Coalesce
from lasuite.oidc_resource_server.mixins import ResourceServerMixin
from rest_framework import (
filters,
mixins,
viewsets,
)
from core import models
from core.api import permissions
from core.api.client.viewsets import Pagination
from . import serializers
class TeamViewSet( # pylint: disable=too-many-ancestors
ResourceServerMixin,
mixins.CreateModelMixin,
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
viewsets.GenericViewSet,
):
"""
Team ViewSet dedicated to the resource server.
The DELETE method is not allowed for now, because the use case is
not clear yet and it comes with complexity to know if we can delete
a team or not (eg. if a team has other SP, it might not be deleted
but what do we do then, only remove the current SP?).
GET /resource-server/v1.0/teams/
Return list of Teams of the user and available for the audience.
POST /resource-server/v1.0/teams/
Create a new Team for the user for the audience.
GET /resource-server/v1.0/teams/{team_id}/
Return the Team details if available for the audience.
PUT /resource-server/v1.0/teams/{team_id}/
Update the Team details (only name for now).
"""
permission_classes = [permissions.AccessPermission]
serializer_class = serializers.TeamSerializer
filter_backends = [filters.OrderingFilter]
ordering_fields = ["created_at"]
ordering = ["-created_at"]
queryset = models.Team.objects.all()
pagination_class = Pagination
def get_queryset(self):
"""Custom queryset to get user related teams."""
teams_queryset = models.Team.objects.filter(
accesses__user=self.request.user,
)
depth_path = teams_queryset.values("depth", "path")
if not depth_path:
return models.Team.objects.none()
user_role_query = models.TeamAccess.objects.filter(
user=self.request.user, team=OuterRef("pk")
).values("role")[:1]
service_provider_audience = self._get_service_provider_audience()
service_provider_prefetch = Prefetch(
"service_providers",
queryset=models.ServiceProvider.objects.filter(
audience_id=service_provider_audience
),
)
return (
models.Team.objects.prefetch_related(
"accesses",
service_provider_prefetch,
)
.filter(
reduce(
operator.or_,
(
Q(
# The team the user has access to
depth=d["depth"],
path=d["path"],
)
| Q(
# The parent team the user has access to
depth__lt=d["depth"],
path__startswith=d["path"][: models.Team.steplen],
organization_id=self.request.user.organization_id,
)
for d in depth_path
),
),
Q(service_providers__audience_id=service_provider_audience)
| Q(is_visible_all_services=True),
)
# Abilities are computed based on logged-in user's role for the team
# and if the user does not have access, it's ok to consider them as a member
# because it's a parent team.
.annotate(
user_role=Coalesce(
Subquery(user_role_query), Value(models.RoleChoices.MEMBER.value)
)
)
)
def perform_create(self, serializer):
"""Set the current user as owner of the newly created team."""
team = serializer.save()
models.TeamAccess.objects.create(
team=team,
user=self.request.user,
role=models.RoleChoices.OWNER,
)
class InvitationViewset( # pylint: disable=too-many-ancestors
ResourceServerMixin,
mixins.CreateModelMixin,
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
mixins.DestroyModelMixin,
viewsets.GenericViewSet,
):
"""API ViewSet for user invitations to team via resource server.
GET /resource-server/v1.0/teams/<team_id>/invitations/:<invitation_id>/
Return list of invitations related to that team or one
team access if an id is provided.
POST /resource-server/v1.0/teams/<team_id>/invitations/ with expected data:
- email: str
- role: str [owner|admin|member]
- issuer : User, automatically added from user making query, if allowed
- team : Team, automatically added from requested URI
Return newly created invitation
PUT / PATCH : Not permitted. Instead of updating your invitation,
delete and create a new one.
DELETE /resource-server/v1.0/teams/<team_id>/invitations/<invitation_id>/
Delete targeted invitation
"""
lookup_field = "id"
pagination_class = Pagination
permission_classes = [permissions.AccessPermission]
serializer_class = serializers.InvitationSerializer
def get_permissions(self):
"""Set specific permissions based on the action."""
if self.action == "list":
permission_classes = [permissions.IsAuthenticated]
elif self.action == "create":
permission_classes = [permissions.TeamInvitationCreationPermission]
else:
permission_classes = [permissions.AccessPermission]
return [permission() for permission in permission_classes]
def get_serializer_context(self):
"""Extra context provided to the serializer class."""
context = super().get_serializer_context()
context["team_id"] = self.kwargs["team_id"]
return context
def get_queryset(self):
"""Return the queryset according to the action."""
service_provider_audience = self._get_service_provider_audience()
# Determine which role the logged-in user has in the team
user_role_query = models.TeamAccess.objects.filter(
user=self.request.user, team=self.kwargs["team_id"]
).values("role")[:1]
queryset = (
models.Invitation.objects.select_related("team")
.filter(
team=self.kwargs["team_id"],
# The logged-in user should be part of a team to see its accesses
team__accesses__user=self.request.user,
# The team should be accessible by the service provider audience
team__service_providers__audience_id=service_provider_audience,
)
.annotate(user_role=Subquery(user_role_query))
.order_by("-created_at")
.distinct()
)
return queryset
+65 -7
View File
@@ -1,11 +1,69 @@
"""People Core application"""
# from django.apps import AppConfig
# from django.utils.translation import gettext_lazy as _
import logging
from django.apps import AppConfig
from django.conf import settings
from django.core.management import call_command, get_commands
from django.utils.translation import gettext_lazy as _
from core.utils.io import TeeStringIO
from people.celery_app import app as celery_app
logger = logging.getLogger(__name__)
# class CoreConfig(AppConfig):
# """Configuration class for the People core app."""
def _register_management_commands_as_task():
"""
Register management command which are enabled via MANAGEMENT_COMMAND_AS_TASK setting.
"""
for command_name in settings.MANAGEMENT_COMMAND_AS_TASK:
# Check if the command is a valid management command
try:
app_name = get_commands()[command_name]
except KeyError:
logging.error("Command %s is not a valid management command.", command_name)
continue
# name = "core"
# app_label = "core"
# verbose_name = _("People core application")
command_full_name = ".".join([app_name, command_name])
# Create a closure to capture the current value of command_full_name and command_name
def create_task(cmd_name, cmd_full_name):
@celery_app.task(name=cmd_full_name)
def task_wrapper(*command_args, **command_options):
stdout = TeeStringIO(logging.getLogger(cmd_full_name).info)
stderr = TeeStringIO(logging.getLogger(cmd_full_name).error)
call_command(
cmd_name,
*command_args,
no_color=True,
stdout=stdout,
stderr=stderr,
**command_options,
)
stdout.seek(0)
stderr.seek(0)
return {
"stdout": str(stdout.read()),
"stderr": str(stderr.read()),
}
return task_wrapper
# Create the task with the current values
create_task(command_name, command_full_name)
class CoreConfig(AppConfig):
"""Configuration class for the People core app."""
name = "core"
app_label = "core"
verbose_name = _("People core application")
def ready(self):
"""Run when the application is ready."""
_register_management_commands_as_task()
@@ -0,0 +1 @@
"""Authentication module."""
+149 -102
View File
@@ -1,98 +1,98 @@
"""Authentication Backends for the People core app."""
import logging
from django.conf import settings
from django.contrib.auth import get_user_model
from django.core.exceptions import SuspiciousOperation
from django.utils.translation import gettext_lazy as _
import requests
from mozilla_django_oidc.auth import (
OIDCAuthenticationBackend as MozillaOIDCAuthenticationBackend,
from lasuite.oidc_login.backends import (
OIDCAuthenticationBackend as LaSuiteOIDCAuthenticationBackend,
)
from lasuite.tools.email import get_domain_from_email
from rest_framework.authentication import BaseAuthentication
from rest_framework.exceptions import AuthenticationFailed
from core.models import (
AccountService,
Contact,
Organization,
)
User = get_user_model()
logger = logging.getLogger(__name__)
class OIDCAuthenticationBackend(MozillaOIDCAuthenticationBackend):
class OIDCAuthenticationBackend(LaSuiteOIDCAuthenticationBackend):
"""Custom OpenID Connect (OIDC) Authentication Backend.
This class overrides the default OIDC Authentication Backend to accommodate differences
in the User model, and handles signed and/or encrypted UserInfo response.
"""
def get_userinfo(self, access_token, id_token, payload):
"""Return user details dictionary.
def get_extra_claims(self, user_info):
"""
Return extra claims from user_info.
Parameters:
- access_token (str): The access token.
- id_token (str): The id token (unused).
- payload (dict): The token payload (unused).
Note: The id_token and payload parameters are unused in this implementation,
but were kept to preserve base method signature.
Note: It handles signed and/or encrypted UserInfo Response. It is required by
Agent Connect, which follows the OIDC standard. It forces us to override the
base method, which deal with 'application/json' response.
Args:
user_info (dict): The user information dictionary.
Returns:
- dict: User details dictionary obtained from the OpenID Connect user endpoint.
dict: A dictionary of extra claims.
"""
user_response = requests.get(
self.OIDC_OP_USER_ENDPOINT,
headers={"Authorization": f"Bearer {access_token}"},
verify=self.get_settings("OIDC_VERIFY_SSL", True),
timeout=self.get_settings("OIDC_TIMEOUT", None),
proxies=self.get_settings("OIDC_PROXY", None),
)
user_response.raise_for_status()
userinfo = self.verify_token(user_response.text)
return userinfo
def get_or_create_user(self, access_token, id_token, payload):
"""Return a User based on userinfo. Create a new user if no match is found.
Parameters:
- access_token (str): The access token.
- id_token (str): The ID token.
- payload (dict): The user payload.
Returns:
- User: An existing or newly created User instance.
Raises:
- Exception: Raised when user creation is not allowed and no existing user is found.
"""
user_info = self.get_userinfo(access_token, id_token, payload)
# Get user's full name from OIDC fields defined in settings
full_name = self.compute_full_name(user_info)
email = user_info.get("email")
claims = {
"email": email,
"name": full_name,
}
sub = user_info.get("sub")
if not sub:
raise SuspiciousOperation(
_("User info contained no recognizable user identification")
extra_claims = super().get_extra_claims(user_info)
if settings.OIDC_ORGANIZATION_REGISTRATION_ID_FIELD:
extra_claims[settings.OIDC_ORGANIZATION_REGISTRATION_ID_FIELD] = (
user_info.get(settings.OIDC_ORGANIZATION_REGISTRATION_ID_FIELD)
)
return extra_claims
# if sub is absent, try matching on email
user = self.get_existing_user(sub, email)
def post_get_or_create_user(self, user, claims, is_new_user):
"""
Post-processing after user creation or retrieval.
if user:
if not user.is_active:
raise SuspiciousOperation(_("User account is disabled"))
self.update_user_if_needed(user, claims)
elif self.get_settings("OIDC_CREATE_USER", True):
user = User.objects.create(sub=sub, password="!", **claims) # noqa: S106
Args:
user (User): The user instance.
claims (dict): The claims dictionary.
is_new_user (bool): Indicates if the user was newly created.
return user
Returns:
- None
"""
# Data cleaning, to be removed when user organization is null=False
# or all users have an organization.
# See https://github.com/suitenumerique/people/issues/504
if not user.organization_id:
organization_registration_id = claims.get(
settings.OIDC_ORGANIZATION_REGISTRATION_ID_FIELD
)
domain = get_domain_from_email(claims["email"])
try:
organization, organization_created = (
Organization.objects.get_or_create_from_user_claims(
registration_id=organization_registration_id,
domain=domain,
)
)
if organization_created:
logger.info("Organization %s created", organization)
# For this case, we don't create an OrganizationAccess we will
# manage this manually later, because we don't want the first
# user who log in after the release to be the admin of their
# organization. We will keep organization without admin, and
# we will have to manually clean things up (while there is
# not that much organization in the database).
except ValueError as exc:
# Raised when there is no recognizable organization
# identifier (domain or registration_id)
logger.warning("Unable to update user organization: %s", exc)
else:
user.organization = organization
user.save()
logger.info(
"User %s updated with organization %s", user.pk, organization
)
def create_user(self, claims):
"""Return a newly created User instance."""
@@ -101,39 +101,86 @@ class OIDCAuthenticationBackend(MozillaOIDCAuthenticationBackend):
raise SuspiciousOperation(
_("Claims contained no recognizable user identification")
)
email = claims.get("email")
name = claims.get("name")
return self.UserModel.objects.create(
password="!", # noqa: S106
sub=sub,
email=claims.get("email"),
name=claims.get("name"),
# Extract or create the organization from the data
organization_registration_id = claims.pop(
settings.OIDC_ORGANIZATION_REGISTRATION_ID_FIELD,
None,
)
def compute_full_name(self, user_info):
"""Compute user's full name based on OIDC fields in settings."""
name_fields = settings.USER_OIDC_FIELDS_TO_NAME
full_name = " ".join(
user_info[field] for field in name_fields if user_info.get(field)
)
return full_name or None
def get_existing_user(self, sub, email):
"""Fetch existing user by sub or email."""
domain = get_domain_from_email(email)
try:
return User.objects.get(sub=sub)
except User.DoesNotExist:
if email and settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION:
try:
return User.objects.get(email=email)
except User.DoesNotExist:
pass
return None
organization, organization_created = (
Organization.objects.get_or_create_from_user_claims(
registration_id=organization_registration_id,
domain=domain,
)
)
except ValueError as exc:
raise SuspiciousOperation(
_("Claims contained no recognizable organization identification")
) from exc
def update_user_if_needed(self, user, claims):
"""Update user claims if they have changed."""
has_changed = any(
value and value != getattr(user, key) for key, value in claims.items()
if organization_created:
logger.info("Organization %s created", organization)
logger.info("Creating user %s / %s", sub, email)
user = super().create_user(claims | {"organization": organization})
# Initiate the user's profile
Contact.objects.create(
owner=user,
user=user,
full_name=name or email,
data={
"emails": [
{"type": "Work", "value": email},
],
},
)
if has_changed:
updated_claims = {key: value for key, value in claims.items() if value}
self.UserModel.objects.filter(sub=user.sub).update(**updated_claims)
return user
class AccountServiceAuthentication(BaseAuthentication):
"""Authentication backend for account services using Authorization header.
The Authorization header is used to authenticate the request.
The api key is stored in the AccountService model.
Header format:
Authorization: ApiKey <api_key>
"""
def authenticate(self, request):
"""Authenticate the request. Find the account service and check the api key.
Should return either:
- a tuple of (account_service, api_key) if allowed,
- None to pass on other authentication backends
- raise an AuthenticationFailed exception to stop propagation.
"""
auth_header = request.headers.get("Authorization", "")
if not auth_header:
return None
try:
auth_mode, api_key = auth_header.split(" ")
except (IndexError, ValueError) as err:
raise AuthenticationFailed(_("Invalid authorization header.")) from err
if auth_mode.lower() != "apikey" or not api_key:
raise AuthenticationFailed(_("Invalid authorization header."))
try:
account_service = AccountService.objects.get(api_key=api_key)
except AccountService.DoesNotExist as err:
logger.warning("Invalid api_key: %s", api_key)
raise AuthenticationFailed(_("Invalid api key.")) from err
return (account_service, account_service.api_key)
def authenticate_header(self, request):
"""
Return a string to be used as the value of the `WWW-Authenticate`
header in a `401 Unauthenticated` response, or `None` if the
authentication scheme should return `403 Permission Denied` responses.
"""
return "apikey"
-18
View File
@@ -1,18 +0,0 @@
"""Authentication URLs for the People core app."""
from django.urls import path
from mozilla_django_oidc.urls import urlpatterns as mozzila_oidc_urls
from .views import OIDCLogoutCallbackView, OIDCLogoutView
urlpatterns = [
# Override the default 'logout/' path from Mozilla Django OIDC with our custom view.
path("logout/", OIDCLogoutView.as_view(), name="oidc_logout_custom"),
path(
"logout-callback/",
OIDCLogoutCallbackView.as_view(),
name="oidc_logout_callback",
),
*mozzila_oidc_urls,
]
-137
View File
@@ -1,137 +0,0 @@
"""Authentication Views for the People core app."""
from urllib.parse import urlencode
from django.contrib import auth
from django.core.exceptions import SuspiciousOperation
from django.http import HttpResponseRedirect
from django.urls import reverse
from django.utils import crypto
from mozilla_django_oidc.utils import (
absolutify,
)
from mozilla_django_oidc.views import (
OIDCLogoutView as MozillaOIDCOIDCLogoutView,
)
class OIDCLogoutView(MozillaOIDCOIDCLogoutView):
"""Custom logout view for handling OpenID Connect (OIDC) logout flow.
Adds support for handling logout callbacks from the identity provider (OP)
by initiating the logout flow if the user has an active session.
The Django session is retained during the logout process to persist the 'state' OIDC parameter.
This parameter is crucial for maintaining the integrity of the logout flow between this call
and the subsequent callback.
"""
@staticmethod
def persist_state(request, state):
"""Persist the given 'state' parameter in the session's 'oidc_states' dictionary
This method is used to store the OIDC state parameter in the session, according to the
structure expected by Mozilla Django OIDC's 'add_state_and_verifier_and_nonce_to_session'
utility function.
"""
if "oidc_states" not in request.session or not isinstance(
request.session["oidc_states"], dict
):
request.session["oidc_states"] = {}
request.session["oidc_states"][state] = {}
request.session.save()
def construct_oidc_logout_url(self, request):
"""Create the redirect URL for interfacing with the OIDC provider.
Retrieves the necessary parameters from the session and constructs the URL
required to initiate logout with the OpenID Connect provider.
If no ID token is found in the session, the logout flow will not be initiated,
and the method will return the default redirect URL.
The 'state' parameter is generated randomly and persisted in the session to ensure
its integrity during the subsequent callback.
"""
oidc_logout_endpoint = self.get_settings("OIDC_OP_LOGOUT_ENDPOINT")
if not oidc_logout_endpoint:
return self.redirect_url
reverse_url = reverse("oidc_logout_callback")
id_token = request.session.get("oidc_id_token", None)
if not id_token:
return self.redirect_url
query = {
"id_token_hint": id_token,
"state": crypto.get_random_string(self.get_settings("OIDC_STATE_SIZE", 32)),
"post_logout_redirect_uri": absolutify(request, reverse_url),
}
self.persist_state(request, query["state"])
return f"{oidc_logout_endpoint}?{urlencode(query)}"
def post(self, request):
"""Handle user logout.
If the user is not authenticated, redirects to the default logout URL.
Otherwise, constructs the OIDC logout URL and redirects the user to start
the logout process.
If the user is redirected to the default logout URL, ensure her Django session
is terminated.
"""
logout_url = self.redirect_url
if request.user.is_authenticated:
logout_url = self.construct_oidc_logout_url(request)
# If the user is not redirected to the OIDC provider, ensure logout
if logout_url == self.redirect_url:
auth.logout(request)
return HttpResponseRedirect(logout_url)
class OIDCLogoutCallbackView(MozillaOIDCOIDCLogoutView):
"""Custom view for handling the logout callback from the OpenID Connect (OIDC) provider.
Handles the callback after logout from the identity provider (OP).
Verifies the state parameter and performs necessary logout actions.
The Django session is maintained during the logout process to ensure the integrity
of the logout flow initiated in the previous step.
"""
http_method_names = ["get"]
def get(self, request):
"""Handle the logout callback.
If the user is not authenticated, redirects to the default logout URL.
Otherwise, verifies the state parameter and performs necessary logout actions.
"""
if not request.user.is_authenticated:
return HttpResponseRedirect(self.redirect_url)
state = request.GET.get("state")
if state not in request.session.get("oidc_states", {}):
msg = "OIDC callback state not found in session `oidc_states`!"
raise SuspiciousOperation(msg)
del request.session["oidc_states"][state]
request.session.save()
auth.logout(request)
return HttpResponseRedirect(self.redirect_url)
+8
View File
@@ -1,3 +1,4 @@
# pylint: disable=too-many-ancestors
"""
Core application enums declaration
"""
@@ -23,3 +24,10 @@ class WebhookStatusChoices(models.TextChoices):
FAILURE = "failure", _("Failure")
PENDING = "pending", _("Pending")
SUCCESS = "success", _("Success")
class WebhookProtocolChoices(models.TextChoices):
"""Defines the possible protocols of webhook."""
SCIM = "scim"
MATRIX = "matrix"
+18
View File
@@ -0,0 +1,18 @@
"""
Custom exceptions for mailbox manager app
"""
from django.utils.translation import gettext_lazy as _
from rest_framework import status
from rest_framework.exceptions import APIException
class EmailAlreadyKnownException(APIException):
"""Exception raised when trying to create a user with an already existing email address."""
status_code = status.HTTP_201_CREATED
default_detail = _(
"Email already known. Invitation not sent but access created instead."
)
default_code = "email already known"
+105 -5
View File
@@ -1,4 +1,3 @@
# ruff: noqa: S311
"""
Core application factories
"""
@@ -115,8 +114,62 @@ class ContactFactory(BaseContactFactory):
class Meta:
model = models.Contact
base = factory.SubFactory("core.factories.ContactFactory", base=None, owner=None)
owner = factory.SubFactory("core.factories.UserFactory", profile_contact=None)
owner = factory.SubFactory("core.factories.UserFactory")
class ProfileContactFactory(BaseContactFactory):
"""A factory to create profile contacts for a user"""
class Meta:
model = models.Contact
owner = factory.SelfAttribute(".user")
user = factory.SubFactory("core.factories.UserFactory")
class OverrideContactFactory(BaseContactFactory):
"""A factory to create contacts for a user"""
class Meta:
model = models.Contact
override = factory.SubFactory("core.factories.ContactFactory", owner=None)
owner = factory.SubFactory("core.factories.UserFactory")
class OrganizationFactory(factory.django.DjangoModelFactory):
"""Factory to create organizations for testing purposes."""
name = factory.Faker("company")
class Meta:
model = models.Organization
class Params: # pylint: disable=missing-class-docstring
with_registration_id = factory.Trait(
registration_id_list=factory.List(
[factory.Sequence(lambda n: f"{n:014d}")]
),
)
with_domain = factory.Trait(
domain_list=factory.List([factory.Faker("domain_name")]),
)
class OrganizationAccessFactory(factory.django.DjangoModelFactory):
"""Factory to create organization accesses for testing purposes."""
class Meta:
model = models.OrganizationAccess
user = factory.SubFactory(
"core.factories.UserFactory",
organization=factory.SelfAttribute("..organization"),
)
organization = factory.SubFactory(
"core.factories.OrganizationFactory", with_registration_id=True
)
role = factory.fuzzy.FuzzyChoice(models.OrganizationRoleChoices.values)
class UserFactory(factory.django.DjangoModelFactory):
@@ -126,6 +179,13 @@ class UserFactory(factory.django.DjangoModelFactory):
model = models.User
django_get_or_create = ("sub",)
class Params:
with_organization = factory.Trait(
organization=factory.SubFactory(
OrganizationFactory, with_registration_id=True
),
)
sub = factory.Sequence(lambda n: f"user{n!s}")
email = factory.Faker("email")
name = factory.Faker("name")
@@ -138,7 +198,6 @@ class TeamFactory(factory.django.DjangoModelFactory):
class Meta:
model = models.Team
django_get_or_create = ("name",)
skip_postgeneration_save = True
name = factory.Sequence(lambda n: f"team{n}")
@@ -154,6 +213,13 @@ class TeamFactory(factory.django.DjangoModelFactory):
else:
TeamAccessFactory(team=self, user=user_entry[0], role=user_entry[1])
@factory.post_generation
def service_providers(self, create, extracted, **kwargs):
"""Add service providers to team from a given list of service providers."""
if not create or not extracted:
return
self.service_providers.set(extracted)
class TeamAccessFactory(factory.django.DjangoModelFactory):
"""Create fake team user accesses for testing."""
@@ -173,7 +239,7 @@ class TeamWebhookFactory(factory.django.DjangoModelFactory):
model = models.TeamWebhook
team = factory.SubFactory(TeamFactory)
url = factory.Sequence(lambda n: f"https://example.com/Groups/{n!s}")
url = factory.Sequence(lambda n: f"https://nosuchdomain.xyz/Groups/{n!s}")
class InvitationFactory(factory.django.DjangoModelFactory):
@@ -186,3 +252,37 @@ class InvitationFactory(factory.django.DjangoModelFactory):
email = factory.Faker("email")
role = factory.fuzzy.FuzzyChoice([role[0] for role in models.RoleChoices.choices])
issuer = factory.SubFactory(UserFactory)
class ServiceProviderFactory(factory.django.DjangoModelFactory):
"""A factory to create service providers for testing purposes."""
class Meta:
model = models.ServiceProvider
skip_postgeneration_save = True
audience_id = factory.Faker("uuid4")
@factory.post_generation
def teams(self, create, extracted, **kwargs):
"""Add teams to service provider from a given list."""
if not create or not extracted:
return
self.teams.set(extracted)
@factory.post_generation
def organizations(self, create, extracted, **kwargs):
"""Add organization to service provider from a given list."""
if not create or not extracted:
return
self.organizations.set(extracted)
class AccountServiceFactory(factory.django.DjangoModelFactory):
"""A factory to create account services for testing purposes."""
class Meta:
model = models.AccountService
name = factory.Sequence(lambda n: f"Account Service {n!s}")
api_key = factory.Faker("uuid4")
@@ -0,0 +1,15 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"title": "Organization Metadata",
"properties": {
"is_public_service": {
"type": "boolean",
"title": "Is public service"
},
"is_commune": {
"type": "boolean",
"title": "Is commune"
}
}
}
+1
View File
@@ -0,0 +1 @@
"""Management commands for core app."""
@@ -0,0 +1 @@
"""Management commands for core app."""
@@ -0,0 +1,34 @@
"""
Management command for filling organization metadata with default values.
"""
from django.core.management.base import BaseCommand
from core import models
from core.utils.json_schema import generate_default_from_schema
class Command(BaseCommand):
"""Management command to fill organization metadata with default values."""
help = "Fill organization metadata with default values"
def handle(self, *args, **options):
"""Fill organizations metadata missing values with default values."""
organization_metadata_schema = models.get_organization_metadata_schema()
if not organization_metadata_schema:
message = "No organization metadata schema defined."
self.stdout.write(self.style.ERROR(message))
return
default_metadata = generate_default_from_schema(organization_metadata_schema)
for organization in models.Organization.objects.all():
organization.metadata = {**default_metadata, **organization.metadata}
# Save the organization with the updated metadata
# We don't use bulk update because we want to trigger the clean method
organization.save(update_fields=["metadata", "updated_at"])
message = "Organization metadata filled with default values."
self.stdout.write(self.style.SUCCESS(message))
@@ -0,0 +1,22 @@
"""Management command to list all plugins and their hooks."""
from django.conf import settings
from django.core.management.base import BaseCommand
from core.plugins.registry import registry
class Command(BaseCommand):
"""Management command to list all plugins and their hooks."""
help = "List all plugins and their hooks"
def handle(self, *args, **options):
"""Print plugin information."""
self.stdout.write(self.style.NOTICE("# Listing plugins\n"))
for plugin_app in settings.INSTALLED_PLUGINS:
self.stdout.write(self.style.NOTICE(f" - {plugin_app}\n"))
self.stdout.write(self.style.NOTICE("# Listing loaded hooks\n"))
for hook_name, callbacks in registry.get_registered_hooks():
self.stdout.write(self.style.NOTICE(f" - {hook_name}: {callbacks}\n"))
+1 -1
View File
@@ -48,7 +48,7 @@ class Migration(migrations.Migration):
('sub', models.CharField(help_text='Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_ characters only.', max_length=255, unique=True, validators=[django.core.validators.RegexValidator(message='Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_ characters.', regex='^[\\w.@+-]+\\Z')], verbose_name='sub')),
('email', models.EmailField(blank=True, max_length=254, null=True, verbose_name='email address')),
('name', models.CharField(blank=True, max_length=100, null=True, verbose_name='name')),
('language', models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'))", default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language')),
('language', models.CharField(choices=settings.LANGUAGES, default='en-us', help_text='The language in which the user wants to see the interface.', max_length=10, verbose_name='language')),
('timezone', timezone_field.fields.TimeZoneField(choices_display='WITH_GMT_OFFSET', default='UTC', help_text='The timezone in which the user wants to see times.', use_pytz=False)),
('is_device', models.BooleanField(default=False, help_text='Whether the user is a device or a real user.', verbose_name='device')),
('is_staff', models.BooleanField(default=False, help_text='Whether the user can log into this admin site.', verbose_name='staff status')),
@@ -0,0 +1,62 @@
# Generated by Django 5.1.1 on 2024-10-22 10:07
import core.models
import django.contrib.postgres.fields
import django.db.models.deletion
import uuid
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0001_initial'),
]
operations = [
migrations.CreateModel(
name='Organization',
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created at')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated at')),
('name', models.CharField(max_length=100, verbose_name='name')),
('registration_id_list', django.contrib.postgres.fields.ArrayField(base_field=models.CharField(max_length=128), blank=True, default=list, size=None, verbose_name='registration ID list')),
('domain_list', django.contrib.postgres.fields.ArrayField(base_field=models.CharField(max_length=256), blank=True, default=list, size=None, verbose_name='domain list')),
],
options={
'verbose_name': 'organization',
'verbose_name_plural': 'organizations',
'db_table': 'people_organization',
'constraints': [models.CheckConstraint(condition=models.Q(('registration_id_list__len__gt', 0), ('domain_list__len__gt', 0), _connector='OR'), name='registration_id_or_domain', violation_error_message='An organization must have at least a registration ID or a domain.')],
},
),
migrations.AddField(
model_name='team',
name='organization',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.PROTECT, related_name='teams', to='core.organization'),
),
migrations.AddField(
model_name='user',
name='organization',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.PROTECT, related_name='users', to='core.organization'),
),
migrations.CreateModel(
name='OrganizationAccess',
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created at')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated at')),
('role', models.CharField(choices=[('administrator', 'Administrator')], default='administrator', max_length=20)),
('organization', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='organization_accesses', to='core.organization')),
('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='organization_accesses', to=settings.AUTH_USER_MODEL)),
],
options={
'verbose_name': 'Organization/user relation',
'verbose_name_plural': 'Organization/user relations',
'db_table': 'people_organization_access',
'constraints': [models.UniqueConstraint(fields=('user', 'organization'), name='unique_organization_user', violation_error_message='This user is already in this organization.')],
},
),
]
@@ -0,0 +1,18 @@
# Generated by Django 5.1.2 on 2024-11-04 14:25
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0002_add_organization_and_more'),
]
operations = [
migrations.AlterField(
model_name='team',
name='slug',
field=models.SlugField(max_length=100, null=True),
),
]
@@ -0,0 +1,17 @@
# Generated by Django 5.1.3 on 2024-11-20 10:08
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0003_team_slug_nullable'),
]
operations = [
migrations.RemoveField(
model_name='team',
name='slug',
),
]
@@ -0,0 +1,39 @@
# Generated by Django 5.1.2 on 2024-11-07 16:24
import uuid
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0004_remove_team_slug'),
]
operations = [
migrations.CreateModel(
name='ServiceProvider',
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created at')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated at')),
('name', models.CharField(max_length=256, unique=True, verbose_name='name')),
('audience_id', models.CharField(db_index=True, max_length=256, unique=True, verbose_name='audience id')),
],
options={
'verbose_name': 'service provider',
'verbose_name_plural': 'service providers',
'db_table': 'people_service_provider',
},
),
migrations.AddField(
model_name='organization',
name='service_providers',
field=models.ManyToManyField(blank=True, related_name='organizations', to='core.serviceprovider'),
),
migrations.AddField(
model_name='team',
name='service_providers',
field=models.ManyToManyField(blank=True, related_name='teams', to='core.serviceprovider'),
),
]
@@ -0,0 +1,24 @@
# Generated by Django 5.1.3 on 2024-11-28 09:37
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0005_add_serviceprovider'),
]
operations = [
migrations.AddField(
model_name='contact',
name='notes',
field=models.TextField(blank=True, default='', verbose_name='notes'),
),
migrations.AlterField(
model_name='contact',
name='full_name',
field=models.CharField(default='-', max_length=150, verbose_name='full name'),
preserve_default=False,
),
]
@@ -0,0 +1,47 @@
# Generated by Django 5.1.3 on 2024-12-02 10:04
import django.db.models.deletion
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0006_contact_notes_alter_contact_full_name'),
]
operations = [
migrations.RemoveConstraint(
model_name='contact',
name='base_owner_constraint',
),
migrations.RemoveConstraint(
model_name='contact',
name='base_not_self',
),
migrations.AlterUniqueTogether(
name='contact',
unique_together=set(),
),
migrations.AddField(
model_name='contact',
name='override',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='overridden_by', to='core.contact'),
),
migrations.AddConstraint(
model_name='contact',
constraint=models.CheckConstraint(condition=models.Q(('override__isnull', False), ('owner__isnull', True), _negated=True), name='override_owner_constraint', violation_error_message='A contact overriding a base contact must be owned.'),
),
migrations.AddConstraint(
model_name='contact',
constraint=models.CheckConstraint(condition=models.Q(('override', models.F('id')), _negated=True), name='override_not_self', violation_error_message='A contact cannot override itself.'),
),
migrations.RemoveField(
model_name='contact',
name='base',
),
migrations.AlterUniqueTogether(
name='contact',
unique_together={('owner', 'override')},
),
]
@@ -0,0 +1,47 @@
# Generated by Django 5.1.3 on 2024-12-02 10:58
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
def init_profile_contact(apps, schema_editor):
"""Create a Contact for each User to be used as their profile."""
User = apps.get_model("core", "User")
Contact = apps.get_model("core", "Contact")
for user in User.objects.all(): # This is a small table for now
Contact.objects.create(
owner=user,
user=user,
full_name=user.name or user.email,
data={
"emails": [
{"type": "Work", "value": user.email},
],
},
)
class Migration(migrations.Migration):
dependencies = [
('core', '0007_rename_contact_base_to_override'),
]
operations = [
migrations.RemoveField(
model_name='user',
name='profile_contact',
),
migrations.AddField(
model_name='contact',
name='user',
field=models.OneToOneField(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='profile_contact', to=settings.AUTH_USER_MODEL),
),
migrations.AddConstraint(
model_name='contact',
constraint=models.CheckConstraint(condition=models.Q(('user__isnull', True), models.Q(('owner', models.F('user')), ('owner__isnull', False)), _connector='OR'), name='profile_contact_owner_constraint', violation_error_message='Users can only declare as profile a contact they own.'),
),
migrations.RunPython(init_profile_contact, reverse_code=migrations.RunPython.noop),
]
@@ -0,0 +1,18 @@
# Generated by Django 5.1.3 on 2024-12-05 13:19
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0008_change_user_profile_to_contact'),
]
operations = [
# From 100ms to 78ms for 5 000 contacts on my computer
# This is not a big improvement and may need further investigation
migrations.RunSQL(
"CREATE INDEX json_field_index_emails_value ON people_contact USING GIN ((data->'emails'->'value') jsonb_path_ops);"
)
]
@@ -0,0 +1,58 @@
# Generated by Django 5.1.3 on 2024-11-26 13:55
from django.db import migrations, models
from treebeard.numconv import NumConv
def update_team_paths(apps, schema_editor):
Team = apps.get_model("core", "Team")
alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
steplen = 5
# Initialize NumConv with the specified custom alphabet
converter = NumConv(alphabet)
nodes = Team.objects.all().order_by("created_at")
for i, node in enumerate(nodes, 1):
node.depth = 1 # root nodes are at depth 1
# Use NumConv to encode the index `i` to a base representation and
# pad it to the specified step length using the custom alphabet
node.path = converter.int2str(i).rjust(steplen, alphabet[0])
if nodes:
Team.objects.bulk_update(nodes, ["depth", "path"])
class Migration(migrations.Migration):
dependencies = [
('core', '0009_contacts_add_index_on_data_emails'),
]
operations = [
migrations.AddField(
model_name='team',
name='depth',
field=models.PositiveIntegerField(default=0),
preserve_default=False,
),
migrations.AddField(
model_name='team',
name='numchild',
field=models.PositiveIntegerField(default=0),
),
migrations.AddField(
model_name='team',
name='path',
field=models.CharField(default='', max_length=400, db_collation="C"),
preserve_default=False,
),
migrations.RunPython(update_team_paths, migrations.RunPython.noop),
migrations.AlterField(
model_name="team",
name="path",
field=models.CharField(unique=True, max_length=400, db_collation="C"),
),
]
@@ -0,0 +1,18 @@
# Generated by Django 5.1.7 on 2025-03-11 13:34
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0010_team_depth_team_numchild_team_path_and_more'),
]
operations = [
migrations.AddField(
model_name='team',
name='is_visible_all_services',
field=models.BooleanField(default=False, help_text='Whether this team is visible to all service providers.', verbose_name='is visible for all SP'),
),
]
@@ -0,0 +1,18 @@
# Generated by Django 5.1.7 on 2025-03-10 14:08
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0011_team_is_visible_all_services'),
]
operations = [
migrations.AddField(
model_name='organization',
name='metadata',
field=models.JSONField(blank=True, default=dict, help_text='A JSON object containing the organization metadata', verbose_name='metadata'),
),
]
@@ -0,0 +1,18 @@
# Generated by Django 5.1.7 on 2025-03-27 10:03
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0012_organization_metadata'),
]
operations = [
migrations.AddField(
model_name='organization',
name='is_active',
field=models.BooleanField(default=True, verbose_name='active'),
),
]
@@ -0,0 +1,32 @@
# Generated by Django 5.1.8 on 2025-04-03 20:37
import core.models
import django.contrib.postgres.fields
import uuid
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0013_organization_is_active'),
]
operations = [
migrations.CreateModel(
name='AccountService',
fields=[
('id', models.UUIDField(default=uuid.uuid4, editable=False, help_text='primary key for the record as UUID', primary_key=True, serialize=False, verbose_name='id')),
('created_at', models.DateTimeField(auto_now_add=True, help_text='date and time at which a record was created', verbose_name='created at')),
('updated_at', models.DateTimeField(auto_now=True, help_text='date and time at which a record was last updated', verbose_name='updated at')),
('name', models.CharField(max_length=255, verbose_name='name')),
('api_key', models.CharField(default='Y9-IThtCrKASOvRAKO2MUBd_XvZrnSTGNMMlv-Z1_o8', max_length=255, verbose_name='api key')),
('scopes', django.contrib.postgres.fields.ArrayField(base_field=models.CharField(max_length=255, validators=[core.models.validate_account_service_scope]), help_text='Allowed scopes for this service', size=None, verbose_name='allowed scopes')),
],
options={
'verbose_name': 'Account service',
'verbose_name_plural': 'Account services',
'db_table': 'people_account_service',
},
),
]
@@ -0,0 +1,18 @@
# Generated by Django 5.1.8 on 2025-04-10 07:19
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0014_accountservice'),
]
operations = [
migrations.AlterField(
model_name='accountservice',
name='api_key',
field=models.CharField(max_length=255, verbose_name='api key'),
),
]
@@ -0,0 +1,40 @@
# Generated by Django 5.2 on 2025-06-12 09:58
import uuid
from django.conf import settings
from django.db import migrations, models
def gen_team_uuid(apps, schema_editor):
Team = apps.get_model("core", "Team")
for row in Team.objects.all():
row.external_id = uuid.uuid4()
# Don't need to batch update here, as the table is likely small or empty
row.save(update_fields=["external_id"])
class Migration(migrations.Migration):
dependencies = [
('core', '0015_alter_accountservice_api_key'),
]
operations = [
migrations.AddField(
model_name='team',
name='external_id',
field=models.UUIDField(default=uuid.uuid4, editable=False, help_text='Team external UUID for synchronization with external systems', unique=False, verbose_name='external_id'),
),
migrations.RunPython(gen_team_uuid, reverse_code=migrations.RunPython.noop),
migrations.AlterField(
model_name='team',
name='external_id',
field=models.UUIDField(default=uuid.uuid4, editable=False, help_text='Team external UUID for synchronization with external systems', unique=True, verbose_name='external_id'),
),
#
migrations.AlterField(
model_name='team',
name='users',
field=models.ManyToManyField(related_name='teams', through='core.TeamAccess', through_fields=('team', 'user'), to=settings.AUTH_USER_MODEL),
),
]
@@ -0,0 +1,18 @@
# Generated by Django 5.2.3 on 2025-06-17 14:23
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0016_team_external_id_alter_team_users'),
]
operations = [
migrations.AddField(
model_name='teamwebhook',
name='protocol',
field=models.CharField(choices=[('scim', 'Scim'), ('matrix', 'Matrix')], default='scim'),
),
]
+677 -111
View File
@@ -1,42 +1,77 @@
"""
Declare and configure the models for the People core application
"""
# pylint: disable=too-many-lines import-outside-toplevel
import json
import os
import secrets
import smtplib
import uuid
from contextlib import suppress
from datetime import timedelta
from functools import lru_cache
from logging import getLogger
from typing import Optional, Tuple
from django.conf import settings
from django.contrib.auth import models as auth_models
from django.contrib.auth.base_user import AbstractBaseUser
from django.contrib.postgres.fields import ArrayField
from django.contrib.sites.models import Site
from django.core import exceptions, mail, validators
from django.db import models, transaction
from django.core.exceptions import ValidationError
from django.db import models
from django.template.loader import render_to_string
from django.utils import timezone
from django.utils.functional import lazy
from django.utils.text import slugify
from django.utils.translation import gettext, override
from django.utils.translation import gettext_lazy as _
from django.utils.translation import override
import jsonschema
from timezone_field import TimeZoneField
from treebeard.mp_tree import MP_Node, MP_NodeManager
from core.enums import WebhookStatusChoices
from core.utils.webhooks import scim_synchronizer
from core.enums import WebhookProtocolChoices, WebhookStatusChoices
from core.exceptions import EmailAlreadyKnownException
from core.plugins.registry import registry as plugin_hooks_registry
from core.utils.webhooks import webhooks_synchronizer
from core.validators import get_field_validators_from_setting
logger = getLogger(__name__)
current_dir = os.path.dirname(os.path.abspath(__file__))
contact_schema_path = os.path.join(current_dir, "jsonschema", "contact_data.json")
with open(contact_schema_path, "r", encoding="utf-8") as contact_schema_file:
contact_schema = json.load(contact_schema_file)
class RoleChoices(models.TextChoices):
@lru_cache(maxsize=None)
def get_organization_metadata_schema() -> Optional[dict]:
"""Load the organization metadata schema from the settings."""
if not settings.ORGANIZATION_METADATA_SCHEMA:
logger.info("No organization metadata schema specified")
return None
organization_metadata_schema_path = os.path.join(
current_dir,
"jsonschema",
settings.ORGANIZATION_METADATA_SCHEMA,
)
with open(
organization_metadata_schema_path,
"r",
encoding="utf-8",
) as organization_metadata_schema_file:
organization_metadata_schema = json.load(organization_metadata_schema_file)
logger.info(
"Loaded organization metadata schema from %s", organization_metadata_schema_path
)
return organization_metadata_schema
class RoleChoices(models.TextChoices): # pylint: disable=too-many-ancestors
"""Defines the possible roles a user can have in a team."""
MEMBER = "member", _("Member")
@@ -44,6 +79,17 @@ class RoleChoices(models.TextChoices):
OWNER = "owner", _("Owner")
class OrganizationRoleChoices(models.TextChoices): # pylint: disable=too-many-ancestors
"""
Defines the possible roles a user can have in an organization.
For now, we only have one role, but we might add more in the future.
administrator: The user can manage the organization: change name, add/remove users.
"""
ADMIN = "administrator", _("Administrator")
class BaseModel(models.Model):
"""
Serves as an abstract base model for other models, ensuring that records are validated
@@ -84,10 +130,10 @@ class BaseModel(models.Model):
class Contact(BaseModel):
"""User contacts"""
base = models.ForeignKey(
override = models.ForeignKey(
"self",
on_delete=models.SET_NULL,
related_name="overriding_contacts",
related_name="overridden_by",
null=True,
blank=True,
)
@@ -98,11 +144,19 @@ class Contact(BaseModel):
null=True,
blank=True,
)
full_name = models.CharField(_("full name"), max_length=150, null=True, blank=True)
user = models.OneToOneField(
settings.AUTH_USER_MODEL,
on_delete=models.SET_NULL,
related_name="profile_contact",
blank=True,
null=True,
)
full_name = models.CharField(_("full name"), max_length=150)
short_name = models.CharField(_("short name"), max_length=30, null=True, blank=True)
# avatar =
# notes =
notes = models.TextField(_("notes"), blank=True, default="")
data = models.JSONField(
_("contact information"),
help_text=_("A JSON object containing the contact information"),
@@ -121,31 +175,38 @@ class Contact(BaseModel):
ordering = ("full_name", "short_name")
verbose_name = _("contact")
verbose_name_plural = _("contacts")
unique_together = ("owner", "base")
unique_together = ("owner", "override")
constraints = [
models.CheckConstraint(
condition=~models.Q(base__isnull=False, owner__isnull=True),
name="base_owner_constraint",
condition=~models.Q(override__isnull=False, owner__isnull=True),
name="override_owner_constraint",
violation_error_message="A contact overriding a base contact must be owned.",
),
models.CheckConstraint(
condition=~models.Q(base=models.F("id")),
name="base_not_self",
violation_error_message="A contact cannot be based on itself.",
condition=~models.Q(override=models.F("id")),
name="override_not_self",
violation_error_message="A contact cannot override itself.",
),
# When a user is set, the owner must be the user
models.CheckConstraint(
condition=models.Q(user__isnull=True)
| models.Q(owner=models.F("user"), owner__isnull=False),
name="profile_contact_owner_constraint",
violation_error_message="Users can only declare as profile a contact they own.",
),
]
def __str__(self):
return self.full_name or self.short_name
return self.full_name
def clean(self):
"""Validate fields."""
super().clean()
# Check if the contact points to a base contact that itself points to another base contact
if self.base_id and self.base.base_id:
# Check if the contact overrides a contact that itself overrides another base contact
if self.override_id and self.override.override_id:
raise exceptions.ValidationError(
"A contact cannot point to a base contact that itself points to a base contact."
"A contact cannot override a contact that itself overrides a contact."
)
# Validate the content of the "data" field against our jsonschema definition
@@ -157,6 +218,274 @@ class Contact(BaseModel):
error_message = f"Validation error in '{field_path:s}': {e.message}"
raise exceptions.ValidationError({"data": [error_message]}) from e
def get_abilities(self, user):
"""
Compute and return abilities for a given user on the contact.
Beware that the model allows owner to be None, we are still not
sure about the use case for this and the API does not allow this.
For now, we still consider here, a contact without owner is "public"
so we allow access to it.
"""
is_owner = user.pk == self.owner_id
is_profile_member_or_same_organization = bool(self.user_id) and (
self.user.organization_id == user.organization_id
)
return {
"get": is_owner or is_profile_member_or_same_organization or not self.owner,
"patch": is_owner,
"put": is_owner,
"delete": is_owner and not self.user, # Can't delete a profile contact
}
class ServiceProvider(BaseModel):
"""
Represents a service provider that will consume our information.
Organization uses this model to define the list of SP available to their users.
Team uses this model to define their visibility to the various SP.
"""
name = models.CharField(_("name"), max_length=256, unique=True)
audience_id = models.CharField(
_("audience id"), max_length=256, unique=True, db_index=True
)
class Meta:
db_table = "people_service_provider"
verbose_name = _("service provider")
verbose_name_plural = _("service providers")
def __str__(self):
return self.name
def save(self, *args, **kwargs):
"""Enforce name (even if ugly) from the `audience_id` field."""
if not self.name:
self.name = self.audience_id # ok, same length
return super().save(*args, **kwargs)
class OrganizationManager(models.Manager):
"""
Custom manager for the Organization model, to manage complexity/automation.
"""
def get_or_create_from_user_claims(
self, registration_id: str = None, domain: str = None, **kwargs
) -> Tuple["Organization", bool]:
"""
Get or create an organization using the most fitting information from the user's claims.
We expect to have only one organization per registration_id, but
the registration_id might not be provided.
When the registration_id is not provided, we use the domain to identify the organization.
If both are provided, we use the registration_id first to create missing organization.
Dev note: When a registration_id is provided by the Identity Provider, we don't want
to use the domain to create the organization, because it is less reliable: for example,
a professional user, may have a personal email address, and the domain would be gmail.com
which is not a good identifier for an organization. The domain email is just a fallback
when the registration_id is not provided by the Identity Provider. We can use the domain
to create the organization manually when we are sure about the "safety" of it.
"""
if not any([registration_id, domain]):
raise ValueError("You must provide either a registration_id or a domain.")
filters = models.Q()
if registration_id:
filters |= models.Q(registration_id_list__icontains=registration_id)
if domain:
filters |= models.Q(domain_list__icontains=domain)
with suppress(self.model.DoesNotExist):
# If there are several organizations, we must raise an error and fix the data
# If there is an organization, we return it
return self.get(filters, **kwargs), False
# Manage the case where the organization does not exist: we create one
if registration_id:
return self.create(
name=registration_id, registration_id_list=[registration_id], **kwargs
), True
if domain:
return self.create(name=domain, domain_list=[domain], **kwargs), True
raise ValueError("Should never reach this point.")
def create(self, **kwargs):
"""
Create an organization with the given kwargs.
This method is overridden to call the Organization plugins.
"""
instance = super().create(**kwargs)
plugin_hooks_registry.execute_hook("organization_created", instance)
return instance
class OrganizationAccessManager(models.Manager):
"""
Custom manager for the OrganizationAccess model, to manage complexity/automation.
"""
def create(self, **kwargs):
"""
Create an organization access with the given kwargs.
This method is overridden to call the Organization plugins.
"""
instance = super().create(**kwargs)
plugin_hooks_registry.execute_hook("organization_access_granted", instance)
return instance
class Organization(BaseModel):
"""
Organization model used to regroup Teams.
Each User have an Organization, which corresponds actually to a default organization
because a user can belong to a Team from another organization.
Each Team have an Organization, which is the Organization from the User who created
the Team.
Organization is managed automatically, the User should never choose their Organization.
When creating a User, you must use the `get_or_create` method from the
OrganizationManager to find the proper Organization.
An Organization can have several registration IDs and domains but during automatic
creation process, only one will be used. We may want to allow (manual) organization merge
later, to regroup several registration IDs or domain in the same Organization.
"""
name = models.CharField(_("name"), max_length=100)
registration_id_list = ArrayField(
models.CharField(
max_length=128,
validators=get_field_validators_from_setting(
"ORGANIZATION_REGISTRATION_ID_VALIDATORS"
),
),
verbose_name=_("registration ID list"),
default=list,
blank=True,
# list overlap validation is done in the validate_unique method
)
domain_list = ArrayField(
models.CharField(max_length=256),
verbose_name=_("domain list"),
default=list,
blank=True,
# list overlap validation is done in the validate_unique method
)
metadata = models.JSONField(
_("metadata"),
help_text=_("A JSON object containing the organization metadata"),
blank=True,
default=dict,
)
service_providers = models.ManyToManyField(
ServiceProvider,
related_name="organizations",
blank=True,
)
is_active = models.BooleanField(verbose_name=_("active"), default=True)
objects = OrganizationManager()
class Meta:
db_table = "people_organization"
verbose_name = _("organization")
verbose_name_plural = _("organizations")
constraints = [
models.CheckConstraint(
name="registration_id_or_domain",
condition=models.Q(registration_id_list__len__gt=0)
| models.Q(domain_list__len__gt=0),
violation_error_message=_(
"An organization must have at least a registration ID or a domain."
),
),
# Check a registration ID str can only be present in one
# organization registration ID list
# Check a domain str can only be present in one organization domain list
# Those checks cannot be done with Django constraints
]
def __str__(self):
return f"{self.name} (# {self.pk})"
def clean(self):
"""Validate fields."""
super().clean()
organization_metadata_schema = get_organization_metadata_schema()
if not organization_metadata_schema:
return
try:
jsonschema.validate(self.metadata, organization_metadata_schema)
except jsonschema.ValidationError as e:
# Specify the property in the data in which the error occurred
field_path = ".".join(map(str, e.path))
error_message = f"Validation error in '{field_path:s}': {e.message}"
raise exceptions.ValidationError({"metadata": [error_message]}) from e
def validate_unique(self, exclude=None):
"""
Validate Registration/Domain values in an array field are unique
across all instances.
This can't be done in the field validator because we need to
exclude the current object if already in database.
"""
super().validate_unique(exclude=exclude)
if self.pk:
organization_qs = Organization.objects.exclude(pk=self.pk)
else:
organization_qs = Organization.objects.all()
# Check a registration ID can only be present in one organization
if (
self.registration_id_list
and organization_qs.filter(
registration_id_list__overlap=self.registration_id_list
).exists()
):
raise ValidationError(
"registration_id_list value must be unique across all instances."
)
# Check a domain can only be present in one organization
if (
self.domain_list
and organization_qs.filter(domain_list__overlap=self.domain_list).exists()
):
raise ValidationError(
"domain_list value must be unique across all instances."
)
def get_abilities(self, user):
"""
Compute and return abilities for a given user on the organization.
"""
# Use the role from queryset annotation will raise on purpose if not used properly
is_admin = self.user_role == OrganizationRoleChoices.ADMIN # pylint: disable=no-member
return {
"get": user.organization_id == self.pk,
"patch": is_admin,
"put": is_admin,
"delete": False,
}
class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
"""User model to work with OIDC only authentication."""
@@ -180,16 +509,9 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
)
email = models.EmailField(_("email address"), null=True, blank=True)
name = models.CharField(_("name"), max_length=100, null=True, blank=True)
profile_contact = models.OneToOneField(
Contact,
on_delete=models.SET_NULL,
related_name="user",
blank=True,
null=True,
)
language = models.CharField(
max_length=10,
choices=lazy(lambda: settings.LANGUAGES, tuple)(),
choices=settings.LANGUAGES,
default=settings.LANGUAGE_CODE,
verbose_name=_("language"),
help_text=_("The language in which the user wants to see the interface."),
@@ -218,6 +540,13 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
"Unselect this instead of deleting accounts."
),
)
organization = models.ForeignKey(
Organization,
on_delete=models.PROTECT,
related_name="users",
null=True, # Need to be set to False when everything is migrated
blank=True, # Need to be set to False when everything is migrated
)
objects = auth_models.UserManager()
@@ -234,11 +563,13 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
def save(self, *args, **kwargs):
"""
If it's a new user, give her access to the relevant teams.
If it's a new user, give them access to the relevant teams.
"""
if self._state.adding:
self._convert_valid_invitations()
self._convert_valid_team_invitations()
# a post_save signal to convert domain invitations to domain accesses
# is triggered by the mailbox_manager app
super().save(*args, **kwargs)
@@ -248,12 +579,7 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
if self.email:
self.email = User.objects.normalize_email(self.email)
if self.profile_contact_id and not self.profile_contact.owner == self:
raise exceptions.ValidationError(
"Users can only declare as profile a contact they own."
)
def _convert_valid_invitations(self):
def _convert_valid_team_invitations(self):
"""
Convert valid invitations to team accesses.
Expired invitations are ignored.
@@ -284,14 +610,161 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
raise ValueError("You must first set an email for the user.")
mail.send_mail(subject, message, from_email, [self.email], **kwargs)
def get_abilities(self):
"""
Return the user permissions at the application level (ie feature availability).
class Team(BaseModel):
Note: this is for display purposes only for now.
- Contacts view and creation is globally available (or not) for now.
- Teams view is available if the user is owner or admin of at least
one Team for now. It allows to restrict users knowing the existence of
this feature.
- Teams creation is globally available (or not) for now.
- Mailboxes view is available if the user has access to at least one
MailDomain for now. It allows to restrict users knowing the existence of
this feature.
- Mailboxes creation is globally available (or not) for now.
"""
user_info = (
self._meta.model.objects.filter(pk=self.pk)
.annotate(
teams_can_view=models.Exists(
self.accesses.model.objects.filter( # pylint: disable=no-member
user=models.OuterRef("pk"),
role__in=[RoleChoices.OWNER, RoleChoices.ADMIN],
)
),
mailboxes_can_view=models.Exists(
self.mail_domain_accesses.model.objects.filter( # pylint: disable=no-member
user=models.OuterRef("pk")
)
),
)
.only("id")
.get()
)
teams_can_view = user_info.teams_can_view or settings.FEATURES["TEAMS_DISPLAY"]
mailboxes_can_view = user_info.mailboxes_can_view
return {
"contacts": {
"can_view": settings.FEATURES["CONTACTS_DISPLAY"],
"can_create": (
settings.FEATURES["CONTACTS_DISPLAY"]
and settings.FEATURES["CONTACTS_CREATE"]
),
},
"teams": {
"can_view": teams_can_view,
"can_create": teams_can_view and settings.FEATURES["TEAMS_CREATE"],
},
"mailboxes": {
"can_view": mailboxes_can_view,
"can_create": mailboxes_can_view
and settings.FEATURES["MAILBOXES_CREATE"],
},
}
class OrganizationAccess(BaseModel):
"""
Link table between organization and users,
only for user with specific rights on Organization.
"""
organization = models.ForeignKey(
Organization,
on_delete=models.CASCADE,
related_name="organization_accesses",
)
user = models.ForeignKey(
User,
on_delete=models.CASCADE,
related_name="organization_accesses",
)
role = models.CharField(
max_length=20,
choices=OrganizationRoleChoices.choices,
default=OrganizationRoleChoices.ADMIN,
)
objects = OrganizationAccessManager()
class Meta:
db_table = "people_organization_access"
verbose_name = _("Organization/user relation")
verbose_name_plural = _("Organization/user relations")
constraints = [
models.UniqueConstraint(
fields=["user", "organization"],
name="unique_organization_user",
violation_error_message=_("This user is already in this organization."),
),
]
def __str__(self):
return f"{self.user!s} is {self.role:s} in organization {self.organization!s}"
class TeamManager(MP_NodeManager):
"""
Custom manager for the Team model, to manage complexity/automation.
"""
def create(self, parent_id=None, **kwargs):
"""
Replace the default create method to ease the Team creation process.
Notes:
- the `add_*` methods from django-treebeard does not support the "using db".
Which means it will always use the default db.
- the `add_*` methods from django-treebeard does not support the "force_insert".
"""
if parent_id is None:
return self.model.add_root(**kwargs)
# Retrieve parent object, because django-treebeard uses raw queries for most
# write operations, and raw queries don't update the django objects of the db
# entries they modify. See caveats in the django-treebeard documentation.
# This might be changed later if we never do any operation on the parent object
# before creating the child.
# Beware the N+1 here.
return self.get(pk=parent_id).add_child(**kwargs)
class Team(MP_Node, BaseModel):
"""
Represents the link between teams and users, specifying the role a user has in a team.
When a team is created from here, the user have to choose which Service Providers
can see it.
When a team is created from a Service Provider this one is automatically set in the
Team `service_providers`.
The team `external_id` is used to synchronize the team with external systems, this
is the equivalent of the User `sub` field but for teams (note: `sub` is highly related
to the OIDC standard, while `external_id` is not). The `external_id` is NOT the same as
the `externalId` field in the SCIM standard when importing teams from SCIM.
"""
# Allow up to 80 nested teams with 62^5 (916_132_832) root nodes
alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
# Django treebeard does not allow max_length = None...
steplen = 5
path = models.CharField(max_length=5 * 80, unique=True, db_collation="C")
name = models.CharField(max_length=100)
slug = models.SlugField(max_length=100, unique=True, null=False, editable=False)
external_id = models.UUIDField(
verbose_name=_("external_id"),
help_text=_("Team external UUID for synchronization with external systems"),
unique=True,
default=uuid.uuid4,
editable=False,
)
users = models.ManyToManyField(
User,
@@ -299,6 +772,25 @@ class Team(BaseModel):
through_fields=("team", "user"),
related_name="teams",
)
organization = models.ForeignKey(
Organization,
on_delete=models.PROTECT,
related_name="teams",
null=True, # Need to be set to False when everything is migrated
blank=True, # Need to be set to False when everything is migrated
)
service_providers = models.ManyToManyField(
ServiceProvider,
related_name="teams",
blank=True,
)
is_visible_all_services = models.BooleanField(
_("is visible for all SP"),
default=False,
help_text=_("Whether this team is visible to all service providers."),
)
objects = TeamManager()
class Meta:
db_table = "people_team"
@@ -309,15 +801,6 @@ class Team(BaseModel):
def __str__(self):
return self.name
def save(self, *args, **kwargs):
"""Override save function to compute the slug."""
self.slug = self.get_slug()
return super().save(*args, **kwargs)
def get_slug(self):
"""Compute slug value from name."""
return slugify(self.name)
def get_abilities(self, user):
"""
Compute and return abilities for a given user on the team.
@@ -331,7 +814,7 @@ class Team(BaseModel):
except AttributeError:
try:
role = self.accesses.filter(user=user).values("role")[0]["role"]
except (TeamAccess.DoesNotExist, IndexError):
except TeamAccess.DoesNotExist, IndexError:
role = None
is_owner_or_admin = role in [RoleChoices.OWNER, RoleChoices.ADMIN]
@@ -382,16 +865,12 @@ class TeamAccess(BaseModel):
Override save function to fire webhooks on any addition or update
to a team access.
"""
if self._state.adding:
if self._state.adding and self.team.webhooks.exists():
self.team.webhooks.update(status=WebhookStatusChoices.PENDING)
with transaction.atomic():
instance = super().save(*args, **kwargs)
scim_synchronizer.add_user_to_group(self.team, self.user)
else:
instance = super().save(*args, **kwargs)
# try to synchronize all webhooks
webhooks_synchronizer.add_user_to_group(self.team, self.user)
return instance
return super().save(*args, **kwargs)
def delete(self, *args, **kwargs):
"""
@@ -399,11 +878,12 @@ class TeamAccess(BaseModel):
Don't allow deleting a team access until it is successfully synchronized with all
its webhooks.
"""
self.team.webhooks.update(status=WebhookStatusChoices.PENDING)
with transaction.atomic():
arguments = self.team, self.user
super().delete(*args, **kwargs)
scim_synchronizer.remove_user_from_group(*arguments)
if webhooks := self.team.webhooks.all():
webhooks.update(status=WebhookStatusChoices.PENDING)
# try to synchronize all webhooks
webhooks_synchronizer.remove_user_from_group(self.team, self.user)
super().delete(*args, **kwargs)
def get_abilities(self, user):
"""
@@ -421,7 +901,7 @@ class TeamAccess(BaseModel):
role = self._meta.model.objects.filter(
team=self.team_id, user=user
).values("role")[0]["role"]
except (self._meta.model.DoesNotExist, IndexError):
except self._meta.model.DoesNotExist, IndexError:
role = None
is_team_owner_or_admin = role in [RoleChoices.OWNER, RoleChoices.ADMIN]
@@ -461,6 +941,11 @@ class TeamWebhook(BaseModel):
team = models.ForeignKey(Team, related_name="webhooks", on_delete=models.CASCADE)
url = models.URLField(_("url"))
secret = models.CharField(_("secret"), max_length=255, null=True, blank=True)
protocol = models.CharField(
max_length=None,
default=WebhookProtocolChoices.SCIM,
choices=WebhookProtocolChoices.choices,
)
status = models.CharField(
max_length=10,
default=WebhookStatusChoices.PENDING,
@@ -483,36 +968,21 @@ class TeamWebhook(BaseModel):
return headers
class Invitation(BaseModel):
"""User invitation to teams."""
class BaseInvitation(BaseModel):
"""Abstract base invitation model, surcharged for teams or domains."""
email = models.EmailField(_("email address"), null=False, blank=False)
team = models.ForeignKey(
Team,
on_delete=models.CASCADE,
related_name="invitations",
)
role = models.CharField(
max_length=20, choices=RoleChoices.choices, default=RoleChoices.MEMBER
)
issuer = models.ForeignKey(
User,
on_delete=models.CASCADE,
related_name="invitations",
)
class Meta:
db_table = "people_invitation"
verbose_name = _("Team invitation")
verbose_name_plural = _("Team invitations")
constraints = [
models.UniqueConstraint(
fields=["email", "team"], name="email_and_team_unique_together"
)
]
MAIL_TEMPLATE_HTML = None
MAIL_TEMPLATE_TXT = None
def __str__(self):
return f"{self.email} invited to {self.team}"
class Meta:
abstract = True
def save(self, *args, **kwargs):
"""Make invitations read-only."""
@@ -527,19 +997,95 @@ class Invitation(BaseModel):
super().clean()
# Check if a user already exists for the provided email
if User.objects.filter(email=self.email).exists():
raise exceptions.ValidationError(
{"email": _("This email is already associated to a registered user.")}
)
if User.objects.filter(email__iexact=self.email).exists():
raise EmailAlreadyKnownException
def refresh(self):
"""A simple way to refresh invitation and move expiration date."""
self.clean()
self.updated_at = timezone.now()
@property
def is_expired(self):
"""Calculate if invitation is still valid or has expired."""
if not self.created_at:
if not self.updated_at:
return None
validity_duration = timedelta(seconds=settings.INVITATION_VALIDITY_DURATION)
return timezone.now() > (self.created_at + validity_duration)
return timezone.now() > (self.updated_at + validity_duration)
def _get_mail_subject(self):
"""Get the subject of the invitation."""
return gettext("Invitation to join La Régie!")
def _get_mail_context(self):
"""Get the template variables for the invitation."""
return {
"site": Site.objects.get_current(),
}
def email_invitation(self):
"""Email invitation to the user."""
try:
with override(self.issuer.language):
subject = self._get_mail_subject()
context = self._get_mail_context()
msg_html = render_to_string(self.MAIL_TEMPLATE_HTML, context)
msg_plain = render_to_string(self.MAIL_TEMPLATE_TXT, context)
mail.send_mail(
subject,
msg_plain,
settings.EMAIL_FROM,
[self.email],
html_message=msg_html,
fail_silently=False,
)
except smtplib.SMTPException as exception:
logger.error("invitation to %s was not sent: %s", self.email, exception)
class Invitation(BaseInvitation):
"""User invitation to teams."""
team = models.ForeignKey(
Team,
on_delete=models.CASCADE,
related_name="invitations",
)
role = models.CharField(
max_length=20, choices=RoleChoices.choices, default=RoleChoices.MEMBER
)
MAIL_TEMPLATE_HTML = "mail/html/team_invitation.html"
MAIL_TEMPLATE_TXT = "mail/text/team_invitation.txt"
class Meta:
db_table = "people_invitation"
verbose_name = _("Team invitation")
verbose_name_plural = _("Team invitations")
constraints = [
models.UniqueConstraint(
fields=["email", "team"], name="email_and_team_unique_together"
)
]
def __str__(self):
return f"{self.email} invited to {self.team}"
def _get_mail_subject(self):
"""Get the subject of the team invitation."""
return gettext(
"[La Suite] You have been invited to become a %(role)s of a group"
) % {"role": self.get_role_display().lower()}
def _get_mail_context(self):
"""Get the template variables for the invitation."""
return {
**super()._get_mail_context(),
"team": self.team.name,
"role": self.get_role_display().lower(),
}
def get_abilities(self, user):
"""Compute and return abilities for a given user."""
@@ -554,7 +1100,7 @@ class Invitation(BaseModel):
role = self.team.accesses.filter(user=user).values("role")[0][
"role"
]
except (self._meta.model.DoesNotExist, IndexError):
except self._meta.model.DoesNotExist, IndexError:
role = None
can_delete = role in [RoleChoices.OWNER, RoleChoices.ADMIN]
@@ -566,24 +1112,44 @@ class Invitation(BaseModel):
"put": False,
}
def email_invitation(self):
"""Email invitation to the user."""
try:
with override(self.issuer.language):
template_vars = {
"title": _("Invitation to join Desk!"),
"site": Site.objects.get_current(),
}
msg_html = render_to_string("mail/html/invitation.html", template_vars)
msg_plain = render_to_string("mail/text/invitation.txt", template_vars)
mail.send_mail(
_("Invitation to join Desk!"),
msg_plain,
settings.EMAIL_FROM,
[self.email],
html_message=msg_html,
fail_silently=False,
)
except smtplib.SMTPException as exception:
logger.error("invitation to %s was not sent: %s", self.email, exception)
def validate_account_service_scope(scope):
"""Validate the scope of the account service."""
if scope not in settings.ACCOUNT_SERVICE_SCOPES:
raise ValidationError(f"Invalid scope: {scope}")
class AccountService(BaseModel):
"""Account service model."""
name = models.CharField(_("name"), max_length=255)
api_key = models.CharField(
_("api key"),
max_length=255,
)
scopes = ArrayField(
models.CharField(max_length=255, validators=[validate_account_service_scope]),
verbose_name=_("allowed scopes"),
help_text=_("Allowed scopes for this service"),
)
class Meta:
db_table = "people_account_service"
verbose_name = _("Account service")
verbose_name_plural = _("Account services")
def __str__(self):
return self.name
def save(self, *args, **kwargs):
"""
Override save method to generate a new api_key if it is not set.
"""
if not self.api_key:
self.api_key = secrets.token_urlsafe(32)
return super().save(*args, **kwargs)
@property
def is_authenticated(self):
"""Indicate if the account service is authenticated."""
return True
+1
View File
@@ -0,0 +1 @@
"""Core plugins package."""
+24
View File
@@ -0,0 +1,24 @@
"""Base Django Application Configuration for plugins."""
class BasePluginAppConfigMixIn:
"""
Configuration for the La Suite plugin application.
We cannot use the `AppConfig` class directly because it is not compatible with
the Django way to discover default AppConfig (see `AppConfig.create`).
We use a mixin then, to be able to list plugins using `plugins.la_suite` instead
of `plugins.la_suite.apps.LaSuitePluginConfig`.
Another way would be to force `default` attribute on plugin AppConfig.
"""
def ready(self):
"""
Initialize the hooks registry when the application is ready.
This is called by Django when the application is loaded.
"""
# pylint: disable=import-outside-toplevel
from .registry import registry # noqa: PLC0415
registry.register_app(self.name) # pylint: disable=no-member
+129
View File
@@ -0,0 +1,129 @@
"""Registry for hooks."""
import logging
from typing import Callable, Dict, List, Set
logger = logging.getLogger(__name__)
class HooksRegistry:
"""Registry for hooks."""
_available_hooks = {
"organization_created",
"organization_access_granted",
}
def __init__(self):
"""Initialize the registry."""
self._hooks: Dict[str, List[Callable]] = {
hook_name: [] for hook_name in self._available_hooks
}
self._registered_apps: Set[str] = set()
def register_hook(self, hook_name: str, callback: Callable) -> None:
"""
Register a hook callback.
Args:
hook_name: The name of the hook.
callback: The callback function to register.
"""
try:
self._hooks[hook_name].append(callback)
except KeyError as exc:
logger.exception(
"Failed to register hook '%s' is not a valid hook: %s", hook_name, exc
)
logger.info("Registered hook %s: %s", hook_name, callback)
def get_registered_hooks(self):
"""Get all registered hooks."""
return self._hooks.items()
def register_app(self, app_name: str) -> None:
"""
Register an app as having hooks.
Args:
app_name: The name of the app.
"""
if app_name in self._registered_apps:
return
self._registered_apps.add(app_name)
try:
# Try to import the hooks module from the app
__import__(f"{app_name}.hooks")
logger.info("Registered hooks from app: %s", app_name)
except ImportError:
# It's okay if the app doesn't have a hooks module
logger.info("App %s has no hooks module", app_name)
def get_callbacks(self, hook_name: str) -> List[Callable]:
"""
Get all callbacks for a hook.
Args:
hook_name: The name of the hook.
Returns:
A list of callback functions.
"""
try:
return self._hooks[hook_name]
except KeyError as exc:
logger.exception(
"Failed to get callbacks for hook '%s' is not a valid hook: %s",
hook_name,
exc,
)
return []
def execute_hook(self, hook_name: str, *args, **kwargs):
"""
Execute all callbacks for a hook.
Args:
hook_name: The name of the hook.
*args: Positional arguments to pass to the callbacks.
**kwargs: Keyword arguments to pass to the callbacks.
Returns:
A list of results from the callbacks.
"""
results = []
for callback in self.get_callbacks(hook_name):
try:
result = callback(*args, **kwargs)
results.append(result)
except Exception as e: # pylint: disable=broad-except
logger.exception("Error executing hook %s: %s", hook_name, e)
return results
def reset(self):
"""Function to reset the registry, to be used in test only."""
self._hooks = {hook_name: [] for hook_name in self._available_hooks}
self._registered_apps = set()
# Create a singleton instance of the registry
registry = HooksRegistry()
def register_hook(hook_name: str):
"""
Decorator to register a function as a hook callback.
Args:
hook_name: The name of the hook.
Returns:
A decorator function.
"""
def decorator(func):
registry.register_hook(hook_name, func)
return func
return decorator
+16
View File
@@ -0,0 +1,16 @@
"""URLs for plugins"""
from django.conf import settings
from django.urls import include, path
plugins_urlpatterns = []
# Try to import and include URLs from each installed plugin
for app in settings.INSTALLED_PLUGINS:
try:
plugins_urlpatterns.append(path("", include(f"{app}.urls")))
except ImportError, AttributeError:
# Skip if app doesn't have urls.py
continue
urlpatterns = plugins_urlpatterns
@@ -1,54 +0,0 @@
"""Resource Server Authentication"""
import base64
import binascii
import logging
from django.conf import settings
from django.core.exceptions import ImproperlyConfigured
from mozilla_django_oidc.contrib.drf import OIDCAuthentication
from .backend import ResourceServerBackend, ResourceServerImproperlyConfiguredBackend
from .clients import AuthorizationServerClient
logger = logging.getLogger(__name__)
class ResourceServerAuthentication(OIDCAuthentication):
"""Authenticate clients using the token received from the authorization server."""
def __init__(self):
super().__init__()
try:
authorization_server_client = AuthorizationServerClient(
url=settings.OIDC_OP_URL,
verify_ssl=settings.OIDC_VERIFY_SSL,
timeout=settings.OIDC_TIMEOUT,
proxy=settings.OIDC_PROXY,
url_jwks=settings.OIDC_OP_JWKS_ENDPOINT,
url_introspection=settings.OIDC_OP_INTROSPECTION_ENDPOINT,
)
self.backend = ResourceServerBackend(authorization_server_client)
except ImproperlyConfigured as err:
message = "Resource Server authentication is disabled"
logger.debug("%s. Exception: %s", message, err)
self.backend = ResourceServerImproperlyConfiguredBackend()
def get_access_token(self, request):
"""Retrieve and decode the access token from the request.
This method overcharges the 'get_access_token' method from the parent class,
to support service providers that would base64 encode the bearer token.
"""
access_token = super().get_access_token(request)
try:
access_token = base64.b64decode(access_token).decode("utf-8")
except (binascii.Error, TypeError):
pass
return access_token
-223
View File
@@ -1,223 +0,0 @@
"""Resource Server Backend"""
import logging
from django.conf import settings
from django.contrib import auth
from django.core.exceptions import ImproperlyConfigured, SuspiciousOperation
from joserfc import jwe as jose_jwe
from joserfc import jwt as jose_jwt
from joserfc.errors import InvalidClaimError, InvalidTokenError
from requests.exceptions import HTTPError
from rest_framework.exceptions import AuthenticationFailed
from . import utils
logger = logging.getLogger(__name__)
class ResourceServerBackend:
"""Backend of an OAuth 2.0 resource server.
This backend is designed to authenticate resource owners to our API using the access token
they received from the authorization server.
In the context of OAuth 2.0, a resource server is a server that hosts protected resources and
is capable of accepting and responding to protected resource requests using access tokens.
The resource server verifies the validity of the access tokens issued by the authorization
server to ensure secure access to the resources.
For more information, visit: https://www.oauth.com/oauth2-servers/the-resource-server/
"""
# pylint: disable=too-many-instance-attributes
def __init__(self, authorization_server_client):
# pylint: disable=invalid-name
self.UserModel = auth.get_user_model()
self._client_id = settings.OIDC_RS_CLIENT_ID
self._client_secret = settings.OIDC_RS_CLIENT_SECRET
self._encryption_encoding = settings.OIDC_RS_ENCRYPTION_ENCODING
self._encryption_algorithm = settings.OIDC_RS_ENCRYPTION_ALGO
self._signing_algorithm = settings.OIDC_RS_SIGNING_ALGO
self._scopes = settings.OIDC_RS_SCOPES
if (
not self._client_id
or not self._client_secret
or not authorization_server_client
):
raise ImproperlyConfigured(
"Could not instantiate ResourceServerBackend, some parameters are missing."
)
self._authorization_server_client = authorization_server_client
self._claims_registry = jose_jwt.JWTClaimsRegistry(
iss={"essential": True, "value": self._authorization_server_client.url},
aud={"essential": True, "value": self._client_id},
token_introspection={"essential": True},
)
# pylint: disable=unused-argument
def get_or_create_user(self, access_token, id_token, payload):
"""Maintain API compatibility with OIDCAuthentication class from mozilla-django-oidc
Params 'id_token', 'payload' won't be used, and our implementation will only
support 'get_user', not 'get_or_create_user'.
"""
return self.get_user(access_token)
def get_user(self, access_token):
"""Get user from an access token emitted by the authorization server.
This method will submit the access token to the authorization server for
introspection, to ensure its validity and obtain the associated metadata.
It follows the specifications outlined in RFC7662 https://www.rfc-editor.org/info/rfc7662,
https://datatracker.ietf.org/doc/html/draft-ietf-oauth-jwt-introspection-response-12.
In our eGovernment applications, the standard RFC 7662 doesn't provide sufficient security.
Its introspection response is a plain JSON object. Therefore, we use the draft RFC
that extends RFC 7662 by returning a signed and encrypted JWT for stronger assurance that
the authorization server issued the token introspection response.
"""
jwt = self._introspect(access_token)
claims = self._verify_claims(jwt)
user_info = self._verify_user_info(claims["token_introspection"])
sub = user_info.get("sub")
if sub is None:
message = "User info contained no recognizable user identification"
logger.debug(message)
raise SuspiciousOperation(message)
try:
user = self.UserModel.objects.get(sub=sub)
except self.UserModel.DoesNotExist:
logger.debug("Login failed: No user with %s found", sub)
return None
return user
def _verify_user_info(self, introspection_response):
"""Verify the 'introspection_response' to get valid and relevant user info.
The 'introspection_response' should be still active, and while authenticating
the resource owner should have requested relevant scope to access her data in
our resource server.
Scope should be configured to match between the AS and the RS. The AS will filter
all the scopes the resource owner requested to expose only the relevant ones to
our resource server.
"""
active = introspection_response.get("active", None)
if not active:
message = "Introspection response is not active."
logger.debug(message)
raise SuspiciousOperation(message)
requested_scopes = introspection_response.get("scope", None).split(" ")
if set(self._scopes).isdisjoint(set(requested_scopes)):
message = "Introspection response contains any required scopes."
logger.debug(message)
raise SuspiciousOperation(message)
return introspection_response
def _introspect(self, token):
"""Introspect an access token to the authorization server."""
try:
jwe = self._authorization_server_client.get_introspection(
self._client_id,
self._client_secret,
token,
)
except HTTPError as err:
message = "Could not fetch introspection"
logger.debug("%s. Exception:", message, exc_info=True)
raise SuspiciousOperation(message) from err
private_key = utils.import_private_key_from_settings()
jws = self._decrypt(jwe, private_key=private_key)
try:
public_key_set = self._authorization_server_client.import_public_keys()
except (TypeError, ValueError, AttributeError, HTTPError) as err:
message = "Could get authorization server JWKS"
logger.debug("%s. Exception:", message, exc_info=True)
raise SuspiciousOperation(message) from err
jwt = self._decode(jws, public_key_set)
return jwt
def _decrypt(self, encrypted_token, private_key):
"""Decrypt the token encrypted by the Authorization Server (AS).
Resource Server (RS)'s public key is used for encryption, and its private
key is used for decryption. The RS's public key is exposed to the AS via a JWKS endpoint.
Encryption Algorithm and Encoding should be configured to match between the AS
and the RS.
"""
try:
decrypted_token = jose_jwe.decrypt_compact(
encrypted_token,
private_key,
algorithms=[self._encryption_algorithm, self._encryption_encoding],
)
except Exception as err:
message = "Token decryption failed"
logger.debug("%s. Exception:", message, exc_info=True)
raise SuspiciousOperation(message) from err
return decrypted_token
def _decode(self, encoded_token, public_key_set):
"""Decode the token signed by the Authorization Server (AS).
AS's private key is used for signing, and its public key is used for decoding.
The AS public key is exposed via a JWK endpoint.
Signing Algorithm should be configured to match between the AS and the RS.
"""
try:
token = jose_jwt.decode(
encoded_token.plaintext,
public_key_set,
algorithms=[self._signing_algorithm],
)
except ValueError as err:
message = "Token decoding failed"
logger.debug("%s. Exception:", message, exc_info=True)
raise SuspiciousOperation(message) from err
return token
def _verify_claims(self, token):
"""Verify the claims of the token to ensure authentication security.
By verifying these claims, we ensure that the token was issued by a
trusted authorization server and is intended for this specific
resource server. This prevents various types of attacks, such as
token substitution or misuse of tokens issued for different clients.
"""
try:
self._claims_registry.validate(token.claims)
except (InvalidClaimError, InvalidTokenError) as err:
message = "Failed to validate token's claims"
logger.debug("%s. Exception:", message, exc_info=True)
raise SuspiciousOperation(message) from err
return token.claims
class ResourceServerImproperlyConfiguredBackend:
"""Fallback backend for improperly configured Resource Servers."""
def get_or_create_user(self, access_token, id_token, payload):
"""Indicate that the Resource Server is improperly configured."""
raise AuthenticationFailed("Resource Server is improperly configured")
@@ -1,95 +0,0 @@
"""Resource Server Clients classes"""
from django.core.exceptions import ImproperlyConfigured
import requests
from joserfc.jwk import KeySet
class AuthorizationServerClient:
"""Client for interacting with an OAuth 2.0 authorization server.
An authorization server issues access tokens to client applications after authenticating
and obtaining authorization from the resource owner. It also provides endpoints for token
introspection and JSON Web Key Sets (JWKS) to validate and decode tokens.
This client facilitates communication with the authorization server, including:
- Fetching token introspection responses.
- Fetching JSON Web Key Sets (JWKS) for token validation.
- Setting appropriate headers for secure communication as recommended by RFC drafts.
"""
# ruff: noqa: PLR0913 PLR017
# pylint: disable=too-many-positional-arguments
# pylint: disable=too-many-arguments
def __init__(
self,
url,
url_jwks,
url_introspection,
verify_ssl,
timeout,
proxy,
):
if not url or not url_jwks or not url_introspection:
raise ImproperlyConfigured(
"Could not instantiate AuthorizationServerClient, some parameters are missing."
)
self.url = url
self._url_introspection = url_introspection
self._url_jwks = url_jwks
self._verify_ssl = verify_ssl
self._timeout = timeout
self._proxy = proxy
@property
def _introspection_headers(self):
"""Get HTTP header for the introspection request.
Notify the authorization server that we expect a signed and encrypted response
by setting the appropriate 'Accept' header.
This follows the recommendation from the draft RFC:
https://datatracker.ietf.org/doc/html/draft-ietf-oauth-jwt-introspection-response-12.
"""
return {
"Content-Type": "application/x-www-form-urlencoded",
"Accept": "application/token-introspection+jwt",
}
def get_introspection(self, client_id, client_secret, token):
"""Retrieve introspection response about a token."""
response = requests.post(
self._url_introspection,
data={
"client_id": client_id,
"client_secret": client_secret,
"token": token,
},
headers=self._introspection_headers,
verify=self._verify_ssl,
timeout=self._timeout,
proxies=self._proxy,
)
response.raise_for_status()
return response.text
def get_jwks(self):
"""Retrieve Authorization Server JWKS."""
response = requests.get(
self._url_jwks,
verify=self._verify_ssl,
timeout=self._timeout,
proxies=self._proxy,
)
response.raise_for_status()
return response.json()
def import_public_keys(self):
"""Retrieve and import Authorization Server JWKS."""
jwks = self.get_jwks()
public_keys = KeySet.import_key_set(jwks)
return public_keys

Some files were not shown because too many files have changed in this diff Show More