Our products mostly rely on email regardless of their case.
Next step would be to normalize email to lower case when storing
them in database (or sending them to dimail if not done yet).
Migrate usage of pip to uv in github actions. How python is setup is
also changed. Doing like this, we will just have to upgrade the python
version requirement in the pyproject file
We want to migrate our projects from pip to uv to take the benefits of
the lock file and have reproducible installations.
A first uv.lock file is comitted and the Dockerfile and compose are
modified to work with uv
devnull@devnull is not considered a valid email address by django's
EmailFieldValidator but it's a special address in dimail's config.
Make "destination" a CharField instead of an EmailField to replace
validator and add devnull to allowlist.
The first user of a organization is probably not an admin.
This was implemented for first tests but for now it's more
a security issue than something helpful.
FIXES#775
added a bulk delete method for aliases, when filtering on local part
this is convenient when in need to delete the local part and all its
destinations in a single call
import existing aliases from dimail, making sure usernames don't clash with
existing mailboxes. Convenient when people fall out of sync
with dimail or for domains partially operated outside people.
users having no domain access should be forbidden to know the domain is
managed in people. To this effect, 403_FORBIDDEN responses have been
replaced by 404_NOT_FOUND.
deleting an alias should trigger a request to dimail. if dimail's
response if a 404, it means people and dimail are out of sync with
each other. log error and warn user
Recent upgrade of @tanstack/react-query to
version >5.90 introduced a breaking change in the
onSuccess and onError callback signatures for
the useMutation hook.
The context parameter has been replaced with an
onMutateResult parameter, which provides
information about the result of the
onMutate callback.
Shamelessly copied from https://github.com/suitenumerique/docs/pull/1375/commits
test_api_mailboxes_create exceeded 1000 lines. By using fixtures, we can
at least factorize dimail response when token is ok or mailbox_data sent
to our API when creating a mailbox.
OpenXchange's primary key is display name (= first name + last name).
It must be unique in the domain's context. We don't have context info
but we can impose uniqueness by domain.
OpenXchange's primary key is display name (= first name + last name).
In absence of clear error message from dimail (yet), we catch errors 500 and
check if they're not due to the display name already existing in the context
Replace custom Docker Hub authentication with standard, secure,
official GitHub actions for improved security and maintainability.
Uses officially supported actions that follow security best practices
and receive regular updates from GitHub.
Avoid unsecure handling of GitHub secrets.
Thanks to @lebaudantoine
git-lint steps are independant and we would like to have all checks at
once. Using the `if: always()` instruction should ensure all steps
should be run event if the previous fails.
thanks @lunika
we want to use Tchap integration server for our own preprod
but it doesn't follow the same url naming convention as the
other Tchap instances, so it requires a special treatment.
This quick fix should do it.
The db engine postgresql_psycopg2 does not exists anymore in django but
for BC compat it is possible to use it in the configuration and it is
replace by postgresql at runtime. We changed this settings to use the
good one.
This introduce a command to create a new client into the "people" realm.
This could be use to create a specific client to test the resource
server mode on a local deployment:
- run the people stack
- add the new client, let say, for docs
- configure the people backend for token introspection
- make calls from docs backend to people's backend
The new client is not mandatory because the same client could be used
everywhere but this would not demonstrate the fact the introspection
works in a real world configuration.
Windows users are by default using CRLF line endings,
which can cause issues with some tools and
environments. This commit sets the `.gitattributes`
file to enforce LF line endings for all text
files in the repository.
Based on the same commit on docs
This commit aims at improving the user experience:
- Use a dedicated `Dockerfile` for the frontend
- Run the backend and frontend in "watch"/dev mode in Docker
- Do not start all Docker instances for small tasks
This provide a "self-care" SCIM endpoint, authenticated with OIDC token
introspection. This endpoint will be use by services to fetch the user's
team list.
We chose to use the SCIM format (even if this is not a SCIM context) to
make it easier to understand/maintain/plug.
This provides a new test to check the action on the mailbox item.
For now we can only enable or disable a mailbox.
We need to create the mailbox in the test, so it exists on Dimail side.
This is supposed to be a validation test on accessibility but in fact
it's just a flaky test which does not provide any information. We need
to replace this with something smarter.
The `Default language` E2E test detected the browser language was not
automatically detected when user does not have any cookie, it was always
falling back on the defaut language (en).
This commit is the result of several squashed commits which were
complicated to disjoin.
This rewrites the base UI, and the mail management interfaces.
allow domain owner and admins to reset password for a mailbox
they manage. The request is sent to dimail, which responds with
a new randomly generated password. This new password is sent to
secondary email.
The `BrowsableAPIRenderer` generates a form to test POST/PUT/... actions
and fill the FK fields with unfiltered data. This issue has been spoted
on visio and fixed https://github.com/suitenumerique/meet/pull/508
We want to persist the session during development. Otherwise the session
is reset everytime the server is restart. This behavior make developing
bot a front and back feature a nigthmare, we spend our time login again
and again.
Shamelessly copy/pasted from @lunika 's work
suitenumerique/docs@007854a
In this commit, we stop creating /users and /allows in dimail
for our dbs to be in sync. People with stop impersonating users
in dimail and will create mailboxes using its own credentials.
Dimail-api is currently outside of renovate' scope, which resulted in us having
to check and update dimail's image manually or, if failing to, discovering new
behaviors by chance or by errors in production. This should fix it.
Bump the lib to the latest version:
- update the post_get_or_create_user method signature
- allow silent login for OIDC (will require frontend implementation)
The secondary email address is no longer required for all creation
processes and we should not force the user to provide and store an
insecure email address.
Override save is a better way to auto generate api_key if
it is not set.
Default with random secret generate a new migration each time we
run `make makemigrations`.
Create new model to allow access of some API
endpoints with API Key authentification.
Scopes will allow to define permission access on those
endpoints.
This provides the way to disable the admin user creation at each
deployment. In production we don't want to persist a generic admin user:
it should be created once, at first deployment then replaced by
nominative accounts.
The new DRF version (3.16.0) adds a check on unique together and needs
more fields to be loaded. To prevent an extra query, we select the owner
value in the DB query.
Add flag to indicate whether the organization is active.
Prepare organizations provisioning. The organization
will be created with this flag set to False and will
become active when the first user is associated with it.
Management command "setup_dimail_db" called dimail directly, thus
creating duplicated code. It now calls "create_domain" and "create_allow"
methods from DimailAPIClient (create_user is left unchanged to create
special users such as dimail admin or people)
For readability, we move the code block from the `ready` method to a
dedicated function.
This will allow to add more things to do in the `ready` with more focus.
For now, to avoid overloading dimail, we have defined a
time interval between each check request to dimail.
This interval should be configurable for testing and
different environments.
The default options in TanStack Query don't make sense for these purposes
and were inducing a need for long timeouts in Playwright tests. (Personal
aside: I consider timeouts in Playwright as a testing smell.)
This provides a configurable OIDC introspection backend to be able to
call introspection endpoints which returns JSON data instead of an
encrypted JWT.
Two backends are currently defined:
- ResourceServerBackend` which expect a JSON response
- JWTResourceServerBackend which implements RFC 9701 and expects
JWE reponse.
There might be other cases (eg: ResourceServerBackend with JWT, JWS or
JWE, etc. but for now we don't use it, so we follow YAGNI).
This also allow to configure the claim to determine the "audience":
- client_id: for our Keycloak implementation
- aud: used by ProConnect
Now a github action downloads last translations from crowdin
when a release branch is created.
We have to merge crowdin translations into release
branch before submit the release PR ;)
This prevent the backend translation file to miss the mail translated
content.
I guess this should be managed otherwise, like asking django to look
into the mail template instead of the generated result.
MailDomain fixtures now use a boring non-repeating sequence. No
longer will the occasional random CI failure inject excitement
into our workdays; but fear not, there will remain other occasions
to enjoy the art of debugging.
This configures local environment to test login through people:
- Keycloak configuration of the IdP (people)
- Add Keycloak Application in people
The only user who can login for now is "admin".
This allows to use `people` as an identity provider using
OIDC and local users.
This commit is partial, because it does not manage a way to
create "local" users and the login page is the admin one, which
can't be used for non staff users or login with email.
To be able to provide a SIRET in the ProConnect IdP process
we need to be able to link a mail domain to its organization.
For now this is not mandatory, as we can't detect the organization
and need a frontend process to clarify it.
Removal of Playwright browsers installation is a win for CI but
not a reason to degrade developer experience, so we install them
locally as part of the bootstrap process.
All translations of backend were synchronized and translated
on crowdin again.
Please use make 'i18n-generate-and-upload' then go to crowdin
to update translations then do 'make i18n-download-and-compile'
Create a Django admin action to allow retrieval of
the expected domain configuration from dimail.
These values shouldn't change unless external
intervention occurs. An admin command seems sufficient to handle
hypothetical changes.
If any action is required on the domain, display the
expected domain configuration in the modal window to
inform the user of actions required to operate the domain.
Added temporary root privileges to update OpenSSL libraries. Upgrades libssl3
and libcrypto3 to 3.3.3-r0 to fix HIGH severity vulnerability. Properly
switches back to nginx user after updates. Maintains unprivileged execution
while addressing security concern affecting RFC7250 Raw Public Keys
authentication.
Security: CVE-2024-12797
The domain of support email suggested was
the same as the example domain name
suggested below.
This was a bit confusing because if the
domain is broken we need to contact someone
with a working email address.
Adapt fetch domain status call to manage internal and external
fixes required. Use the new status 'action required' to
manage actions expected from support.
Call a new dimail endpoint to run a fix for internal checks
when all external checks are OK.
Sometimes emails generated by faker in data field match search.
So it's necessary to create contacts with empty data field
to search contacts by full_name in tests.
Also return the parent teams in the user's team endpoints.
This is a first implementation which returns a flat list
of teams (not a tree). This is not really helpful, but
it allows to create hierarchical teams manually (via
admin) if an organization needs it.
To be able to move the repository on the new organization and to
facilitate external developer integration we need to create a standalone
dev stack and use external secret.
Faker sometimes creates users whose name starts with "Ms." or "Mr."
The implemented test code computed the order without handling
these cases and failed.
mark local_part and domain as read-only fields in admin,
in order to prevent mistakes/temptation. For now, if an local part
needs modification, you can simply delete/recreate the email you want.
Changing the domain is a bigger operation that cannot
be settled simply by changing it Django db.
Small hack to fix clicking on member action button.
Action button of the third member in the list could not
be clickable every time.
The member list table needs and will be improved soon.
Call dimail to check if a domain still works.
Turn domain into failure status if dimail returns broken state.
And enable domain if dimail returns ok state.
fixed a test that would fail because of random auth user name being
too close to tested names (assert would find 2 results instead of 1).
Setting the name of auth user should fix the issue permanently.
When we try to create a duplicate email, a request
to dimail is sent despite a reject on our side.
To solve this issue, we call mailbox creation first
to benefit from validation of our Django model.
Mailbox creation try was called too late after dimail call.
So in attempt to create a duplicated email a request
to dimail was sent despite a failure in our side.
The use of a lazy function here make the Django migration
detector to generate a migration every time we run `makemigrations`.
This is not mandatory to have a lazy here as the settings are loaded
once at runtime beginning.
As the choices makes noop migrations, we directly use the setting in
the initial migration.
During a new domain creation, a call to dimail is made
to create user/allow on dimail side before owner role creation
on our side.
So when user/allow creation on dimain side fails,
the owner role is not created on our side.
Therefore the domain is created but invisible on the user interface.
The user will probably try to create the same domain again
and see the error message 'this domain already exists'.
To avoid this we make sure to create owner role on our side
despite dimail failure and set domain to failed status to retry
later dimail access creation.
This adds the plugin system to easily manage
Organization related customizations. This first
plugin tries (best effort) to get a proper name
for the Organization, using its SIRET. This
is French specificities but another plugin can
be defined for other cases.
We try to detect the landing page according to user
permissions (abilities) instead of just the configuration
setting.
This will be improved when the homepage is developed
Use the common way to define permissions on the API.
Note: we keep here the notion of "public" contacts,
even if the API does not really allows that. The use
case is not clear for that, but we allow contact w/o
owner to be displayed.
The configuration also defines the Sentry environment
value, which is currently "production" for every Sentry
event.
- dev: Local
- staging: Staging
- preprod: PreProduction
- prod: Production
Move the user <-> contact relation for "profile" to
the contact model.
Now the Contact model is the only one to point to
User (and not backward).
Contact:
- FK to User for the owner
- FK to User for the profile
To improve code readability, I propose to rename
the contact field `override`. This comes along
with the fact a contact should not not always
override another (it's the case were I only want
to create some personal contacts).
We make the full name mandatory and add a field to
allow user to store personnal notes on the contact.
This also make the "base" contact not mandatory because
user may want to create new contacts out of the blue.
We send a request to dimail API and change mailbox status to disabled.
A disabled mailbox can no longer be used thus access to webmail
is disabled for user.
To improve readability and code sharing we group all
APIs into the same "api" module for each application.
Next submodules might be "scim",
"resource_server_scim", ...
The only shared module is the "permissions" one for now.
When updating an Organization in the Django admin, the validator
falsly raises a "duplicated" error because it does not exclude the
current object from the database lookup.
We don't want every Service Provider to be able to request
every endpoint if those are not implementing a filtering on
the data returned. To prevent any data leak we enforce the
developers to manually "whitelist" each endpoint and add
the proper filtering when needed.
When we use the feature to get Organization registration
number, the claim contains this value and it does not
match with any user field.
I switched to a whitelist instead of a blacklist (and two
loops, with an if condition on each)
This commit removes the slug field from
the database, now the nullable migration is in
production and the field has been remove from
the code deployed.
FIXES#505
This supplements the release process. We inject Github metadata into two
version.json files; the 'version' value will depend on the type of event,
for release tag events it should be the same as the release tag (i.e. the
app version). This should make version information available to the /config
endpoint on any push, and the frontend should display the backend version.
(For extra safety we will also want to get the frontend version and display
that, but this commit only supplies the barest necessities.)
Some outdated references to Terraform and OpenStack were missed during
the project quickstart. These are legacy elements inherited from OpenFun.
This commit cleans up the codebase.
Previously, mailbox creation was restricted to "enabled" domains.
We now allow users to create mailboxes on pending and failed domains.
Mailboxes thus created have the "pending" mailboxes status.
This allows, on a per user basis, the display of
features.
The main goal here is to allow Team admin or owner
to see the management views.
We also added the same for the two other features
(mailboxes and contacts)
This will be improved later if needed :)
So that E2E tests in Github Actions can connect to Dimail container.
Previously we were attempting to connect as if from the outside. But
the E2E process is in fact inside the Docker Compose network.
("The tests came from inside the house !")
https://tvtropes.org/pmwiki/pmwiki.php/Main/TheCallsAreComingFromInsideTheHouse
Adapt domain creationg request to latest protocol version, also
make error reporting more robust: don't assume utf-8 but use the
response's encoding, don't assume the error is JSON (it won't be
when getting a 500) but reproduce the whole thing instead.
Start the Dimail container (in CI and local testing) when starting
the app. The pull_policy should have no effect on CI (because it starts
from a blank slate) but ensure we test against the most recent version
of the chosen tag.
After some reflexion, the use of a slug field raises to many
problems without being really needed.
One problem is the slug is made from the group name, but we
don't have unicity on this, so a user might be blocked without
any clue.
We also want to allow group names to be reused (which is already
allowed except for the automatic slug).
The unique ID that will be shared with Service Providers will be
the PK/UUID.
We introduce the Organization model has a "hat" for all
users and team.
Each User must have a "default" organization.
Each Team must have an organization.
When a User creates a new Team, the team is linked to their
default Organization.
For now the Organization should not be visible to end users
this is a purely technical aspect as it.
The models are also adding a permission to allow User to edit
an Organization, but for now there are no endpoints for that.
Next steps:
- Add an Organization to each User and Team on all environments
to mark Organization as mandatory in database.
- Add scope to Organization to list the Service Provider list
allowed for a User in an Organization.
- Add endpoints + frontend to manage Organization's scopes
In order to be in agreement with the back, we must modify the front
so as to no longer have infinite scroll in the team part.
It is also necessary to modify the tests accordingly
Disable retries and save trace for failed tests.
💚(ci) preserve server logs
Save server logs to the same place as Playwright reports to aid debugging.
💚(ci) move back to 1 worker on CI
At least three reasons
- seems redundant with sharding
- strong suspicion it's the reason for the ValidationError issue
- that way the comment no longer tells a lie ;)
💚(ci) improve E2E tests
Log into CHANGELOG to ensure the new test results impact PR status 🤷💚(ci) make dummy data creation more robust
This is a QR (Quick Response) fix for the failures in the "Add dummy
data" step in E2E testing. Proper QC (Quality Control) needs a bit
more thought.
The `DockerflowMiddleware` job is to add logs, sadly the
Sentry LoggingIntegration also catch these logs (at
level WARNING and above) which results in an extra alert
for the same purpose (the exception + the logger.error).
see https://github.com/mozilla-services/Dockerflow/blob/main/docs/mozlog.md
The fix is to ask Sentry to ignore this specific loggger
(`request.summary`).
Currently, it is complicated to understand the navigation between mailbox
management and role management for an email domain.
This is why we add tabs with explicit naming
Some OIDC identity providers may provide a random value in the "sub"
field instead of an identifying ID. In this case, it may be a good
idea to fallback to matching the user on its email field.
* 🐛(frontend) update access role
Currently, when you click on modify a role, the roles displayed are not the
correct ones. We always took the role of the connected user.
Additionally, there was a warning and a ban on changing roles if you were the
last owner, but it was decided that it was the backend that was throwing the
error and the action was allowed.
* ✅(frontend) adapt tests
We adapt the tests. Remove the check if you are the last owner and use the new
props inside the relevant tests
- access management view is ready to use
get, patch and delete requests once backend
is ready. How to create accesses with post
will come later in a future commit.
- update translations and component tests.
- reduce gap between mail domains feature
logo and mail domain name in top banner
- separate feature into domain, mailboxes
as a member management feature is arriving in
the mail domains feature
- pages/mail-domains/[slug].tsx becomes
pages/mail-domains/[slug]/index.tsx
We modify multiples things :
* settings.py in order to manage the new way to send email with the
scaleway gateway
* helm template to manage new mandatory secret
* helm configuration for staging/preprod/production
- catch errors related to MAIL_PROVISIONING_API_CREDENTIALS
introduced in commit #ba30b1d3eec73718add6585f30c6b7959cb21850.
Intentionally does not parse the error
"Permission denied. Please check your MAIL_PROVISIONING_API_CREDENTIALS."
as it means the user is neither admin or owner of the domain and
should not access the mailbox creation form
- update translations and component tests
Fix following warning messages :
- You have not set a value for the SECURE_HSTS_SECONDS setting.
- Your SECURE_SSL_REDIRECT setting is not set to True.
- add title above the list of members
- allow members filtering with an input
- updates translations and related tests
- add global useDebounce hook to prevent
spamming API with useless requests on
inputs sending requests on value change
- add component and e2e tests
- translate known errors, including already existing
group error, and directly display the other ones
- add component tests
- update translations
- add parseAPIError, a reusable function to
catch errors on the whole frontend app
Closes issue #293♻️(frontend) improve general error catching
- change parseAPIError to make it reusable on all
requests
- update components depending on it
When new secret is added to backend secret, it's not sync at the
beginning of argocd synchronisation and jobs are blocked. Theses new
annotations fix this issue.
we move all business logic from model to serializer.
all API calls (direct and from front) will keep on triggering
expected 3rd party calls while admin actions will uniquely trigger
modifications in our database.
this commit adds a script and 'make' command to populate dimail database
with basic objects: an admin account, a regie account, a domain
and an owner for this domain.
Marie's key was accidentally removed in last commit.
Add her key and restore .sops.yaml file, to easily add keys
for local dev secrets decryption/encryption.
We remove 'secret' field, as it won't be of use in interactions
between la Régie and dimail. Régie credentials will be stored and used
as project variable.
- rename CreateMailboxForm into ModalCreateMailbox,
and useCreateMailDomain into useAddMailDomain
- use useAPIError hook in ModalCreateMailbox.tsx and ModalAddMailDomain
- update translations and tests (include removal of e2e test able
to be asserted by component tests)
- allow to submit form by pressing "Enter" key
- force focus on form when form is submited
but is invalid
- add error 500 handling
- update related e2e tests
- create a temporary Modal component to apply it
a function tracking document update and set
modal elements we want to ignore a tabindex=-1.
- add component tests
- temporary fix. Better to apply them on
cunningham directly
- add temporary fix to language picker to
ignore select on keyboard navigation. Needs
to be fixed directly in Cunningham Select
- update related e2e test
dimail is called twice when creating a mailbox (once for the token,
and once for the post on mailbox endpoint). we want to clarify
the status_codes and messages of each error to inform user and ease debug
Allow desk-staging app to interact with AgentConnect integration env
to introspect a received access token.
Other environment (pre-prod and production) will be configured when
at least one interconnection with a service provider has been
validated.
Pushing to crowdin from the workflow has some side
effects, if 2 branches are pushing to crowdin it
can cause conflicts and delete translations on
Crowdin side.
Better to push to crowdin manually to keep good
control over the translations.
i18next-parser had a compatibility issue with
a dependency (cheerio). The last version
fixed this issue, plus fixed another issue
about a configuration problem.
We can now remove it from the renovate ignore list.
In order to work correctly
@typescript-eslint/eslint-plugin requires
@typescript-eslint/parser to be installed as well.
We fixed the linting issues related to the
upgrade.
I've committed few of my notes for my future myself, helping me to
run locally the stack. These notes are raw, I'll be more than happy
to refine them when I got sufficient time. Please feel free to ping
me if you have any question.
Tests are missing, let's ship it, I'll open an issue.
Without such protection, the whole app would crash if the resource server is
not configured. The fallback backend would return an appropriate error to
the client if the resource server is improperly configured.
Overload mozilla-django-oidc class to support an authentication method
with the resource server backend.
This enables any route of the API to be called with an access token
issued by Agent Connect.
Why:
Many services in La Suite rely on Agent Connect to authenticate their users.
Delegating authentication to Agent Connect is highly beneficial. With a central
party (Agent Connect) handling user authentication, our services can seamlessly
communicate with each other. Our backend must be able to receive and verify
access tokens issued by Agent Connect.
Additionally, it should ensure that the resource owner has granted permission
for our data to the service provider transmitting the access token.
How:
Our backend needs to verify access tokens by introspecting them. This involves
requesting the Authorization Server to validate the access token received in
the authentication header. The Authorization Server validates the token's
integrity, provides authentication and authorization information about
the user currently logged into the service provider requesting data from
the resource server.
The data returned by the Authorization Server to the resource server
is encrypted and signed. To encrypt the introspection token, the Authorization
Server retrieves the resource server's public key from
the new ‘/jwks’ endpoint.
Encryption parameters, such as algorithm and encoding, are configured on
the resource server. Ensure that these parameters match between
the Authorization Server and the resource server.
The resource server verifies the token signature using the Authorization
Server's public key, exposed through its `/jwks` endpoint. Make sure
the signature algorithms match between both servers. Finally, introspection
token claims are verified to adhere to good practices for handling JWTs,
including checks on issuer, audience, and expiration time.
The introspection token contains a subject (`sub`). The resource server uses
this subject to retrieve the requested database user, compatible
with both pairwise and public subjects.
Important:
Agent Connect does not follow RFC 7662 but uses a draft RFC that adds security
(signing/encryption) to the initial specification. Refer to the "References"
section for more information.
References:
The initial RFC describing token introspection is RFC 7662 "OAuth 2.0 Token
Introspection". However, this RFC specifies that the introspection
response is a plain JSON object.
In eGovernment applications, our resource server requires stronger assurance
that the Authorization Server issued the token introspection response.
France Connect's team implemented a stronger version of the spec, returning
a signed and encrypted token introspection response. This version is still
a draft, available under:
"draft-ietf-oauth-jwt-introspection-response".
In OAuth 2.0, the Authorization Server is equivalent to the OIDC provider.
The Authorization Server exposes endpoints for token introspection and JWKS.
I’ve created a client to easily interact with the Authorization Server,
while doing the token introspection in our resource server.
Token introspection will be explained in upcoming commits.
The current OIDC library used in the project doesn’t offer token introspection,
leading to redundancy in the code handling some OIDC/OAuth2 flows.
This overlap makes the code bulky. My goal is to quickly deliver a working
PoC for the resource server, with plans to refactor in the longer run.
Please feel free to provide feedback on the class design.
Introduce a new endpoint, /jwks, which returns a JSON Web Key Set (JWKS).
This set of public crypto keys will be used by external parties to encrypt
data intended for our backend. In the context of the resource server, this key
will be used by the authorization server to encrypt the introspection response.
The current implementation exposes a single public key, with the private key
configurable in the app settings. The private key is represented as a string.
For enhanced security, we might prefer to store this data in a .pem file
excluded from version control.
A few parameters for this key, such as its type and encoding, are configurable
in the settings.
A critique of the current design is its lack of extensibility.
If we decide to offer more than one encryption method, this view will require
refactoring.
Additionally, the current implementation is tightly coupled with joserfc.
This lays the foundation for further improvements.
Please note, this endpoint only public components of the key, there is no
chance for any secret leaking.
We will add a JWKs endpoint to the application
and manipulate signed/encrypted Json Web Token (JWT).
Project lacks tooling for JSON Object Signing and Encryption (JOSE)
manipulations. After a quick benchmark, 'joserfc' has been chosen
as the dependency to add.
joserfc is a Python library that provides a comprehensive
implementation of several essential JOSE standards.
Please find the benchmark:
- Cryptography: Although using only cryptography is feasible, its
interface/API is not as user-friendly.
- pyjwt: While pyjwt is popular, it lacks support for JWK and JWE objects,
which are essential for the requirements.
- python-jose: The latest release of python-jose was in 2021, and the
project seems less active compared to other alternatives.
- Authlib: Authlib is the second most popular library after pyjwt and seems
modern with an active community. However, the parts relevant to the use case
were extracted into a relatively new package named joserfc.
- joserfc: Although joserfc has fewer stars compared to Authlib, it was
extracted from Authlib, which has more than 4k stars, indicating a solid
foundation.
While the low star count of joserfc might raise concerns about its stability, it
is believed to be worth considering its addition. Adding Authlib and refactoring
later, once they finish migrating to joserfc, is also a possibility
Upgrading Django to 5.1 created a severe issue, breaking
mail template-associated features.
The issue originated from the third party 'easy_thumbnail'.
Please refer to the issue #641 on Django's repo. This is
the suggested workaround by @Miketsukami.
We previously accepted '+' as a special caracter during mailbox creation.
We now remove it, as this caracter has a very special meaning and it wouldn't
make sense to create a mail using it.
Latest dimail modification lead to a bug in our app, preventing mailbox creation
from working properly. I swapped old dimail url to new one, mirrored dimail
modification and fixed tests and tada!
By making this email address invariant, we remove failures due to FactoryBoy's
random address being considered as a match to our test query
(and hence returning unexpected number of matches).
Teams feature is not ready for production yet,
so we need to deactivate it on productions environment.
preprod should be a copy of production,
so we need to deactivate it on preprod too.
NEXT_PUBLIC_FEATURE_TEAM is a buid-time env
variable, it is not easy to overload it per
environment.
We will use the config endpoint to get the
feature flag at runtime.
To do so, we are using the ConfigStore.
Add a ConfigProvider to the frontend to provide
configuration to the app.
The configuration is loaded from the config
endpoint and stored in a zustand store.
- user can add an externally created mail domain
from UI and see the mail domain status on mail
domain page and left panel links.
- user can not create mailboxes to domain if mail
domain status is not equal to `enabled`
- update related tests and translations
- manage 403 returned by dimail API when mail domain secret is not valid
- improve some tests
- improve MailboxFactory to mock success for dimail API POST call
- override 403.html to return a nice failing error in django admin
- an error message is displayed on mailbox creation form of frontend
We want people to create new mailboxes in La Régie.
This commit adds integration with intermediary dimail-api,
which will in turn send our email creation request to Open-Xchange.
We want to make a first realease, but the
team feature is not ready yet.
So we will hide it for now by hiding the menu.
We will still let the feature in dev environment.
- update mailbox creation feature by introducing the use of
new mail domain ability field to hide or show
mailbox creation button
- update related e2e tests
- add aria-hidden and empty alt attributes for screen readers
to ignore decorative svg and images.
- remove icon from input field used to name a group
- update translations
- update related e2e and components tests
- change message showed in mailbox list when none exist
- change CTA button text for group management modals
- fix 404 page title
- update translations
- update related e2e tests
Rename member role to viewer role for MailDomainAccess.
A viewer is only allowed to see list of mailboxes
created for a domain. It makes more sense to name it viewer.
- add css rules to highlight focused-visible navigable elements
- update drop down components to make it keyboard navigable
- add e2e keyboard navigation tests asserting it navigates through
all focusable elements from top to bottom on groups index view
when one group exists
@typescript-eslint/eslint-plugin released the
version 8, but it is causing some issues
(@typescript-eslint/no-duplicate-enum-values).
We downgrade it to 7.13.1 in waiting for a fix.
README was not correctly updated after last major modif to the admin auth,
resulting in confusion for users trying to log-in with email instead of sub.
- remove icon on top of mailbox creation form
- change some text positions
- fix fields error style on hover
- update form field names translations
- update related e2e tests
- replace known error causes returned by the API on unsuccessful mailbox
creation requests by meaningful interpolated message shown above the
form
- strengthen form validation rules to be identical as those of
the api endpoint to prevent emitting invalid requests
- designate which form fields are mandatory for accessiblity
- update texts for better ux writting, and their translations
- fix css style input errors
- update related e2e tests.
Remove phone number field from mailbox creation form and requests
to comply with GPDR as it is not an information we need to
collect for creating a mailbox. Deletes translations about phone
number field in mailbox creation form. Updates related e2e tests.
The datagrid was flickering when sorting the member list.
This was due to the fact that the list was getting
empty and then filled again with the sorted list
causing the flickering.
When we use the static mode, Next.js want that we
build our mail-domains pages at build time.
But we can't do that because users can create a
mail at runtime.
So we redirect thanks to ngnix when we see that
a mail page is not found.
When we use the static mode, Next.js want that we
build our team pages at build time. But we can't do that
because users can create a team at runtime.
So we redirect thanks to ngnix when we see that
a team page is not found.
Keys that contains colon where not being
translated correctly. This was due to the
colon being used as a separator for the
key and the value. This was fixed by
replacing the colon with a different
character that is not used in the key
or the value.
fetch-mock relesed the version 10, but it is
causing some issues in the frontend tests for
the moment. The adoption is still low, the
documentation is not updated, so let's
downgrade it to 9.11.0 for now.
We were starting the workflow on push tags,
it is needed for the docker-hub workflow,
but the other workflows does not need to
be triggered on push tags.
mozilla-django-oidc didn't add the `https`
prefix to the redirect_uri.
We set the option SECURE_PROXY_SSL_HEADER to
('HTTP_X_FORWARDED_PROTO', 'https') in the
settings.py file to force the https prefix.
The frontend translations.json could be validated by
the eslint scripts without running manually
prettier. This commit fixes this by running prettier
script automatically once the frontend translations
are extracted.
This commit is part of the replacement of "teams" by "groups" wording task.
It changes words in the "teams" feature and add more meaningful wording in
English and French, and removes the icon used in the title of the group
member deletion modal as it rather lets think we are deleting a group.
Updates of related e2e and rendering tests come along with these changes.
Fix duplicate call on getMe.
The logout process was refactored recently,
the hook dependency is not necessary anymore and
was creating a duplicate call on getMe.
Until now, the front had to know at build time
the url of the backend and the webrtc server
to be able to communicate with them.
It is not optimal because it means that we need
multiple docker image (1 per environment) to have
the app working, it is not very flexible.
This commit will make the frontend "environment free"
by determining these urls at runtime.
The goal of adding a namespace in the templates
is to ensure that resources are deployed
in a specific, possibly isolated part of the Kubernetes cluster.
This helps in organizing resources, managing
permissions, and applying configurations or
limits appropriately within the cluster.
Tilt is a tool for local Kubernetes development.
It makes it easy to see your changes as you
make them, and it rebuilds and redeploys
your app as you change it.
Forcing `platform: linux/amd64` for the PostgreSQL image causes compatibility
issues and performance degradation on Mac ARM chips (M1/M2). Removing the
platform specification allows Docker to select the appropriate architecture
automatically, ensuring better performance and compatibility.
Pylint 3.2.0 introduced a new check `possibly-used-before-assignment`, which
ensures variables are defined regardless of conditional statements.
Some if/else branches were missing defaults. These have been fixed.
Recent updates in the backend views now requires the user agent to be
redirected to the logout endpoint.
The logout endpoint should initiate the logout flow with the OIDC provider,
by redirecting the user to the OIDC provider domain.
Thus, OIDC provider session cookie should be cleared.
E2E tests should be improved later on, when the CI and the development env
use Agent Connect integration environment. The current logout is not working
with the Keycloack configuration.
The default Logout view provided by Mozilla Django OIDC is not suitable
for the Agent Connect Logout flow.
Previously, when a user was logging-out, only its Django session was ended.
However, its session in the OIDC provider was still active.
Agent Connect implements a 'session/end' endpoint, that allows services to
end user session when they logout.
Agent Connect logout triggers cannot work with the default views implemented
by the dependency Mozilla Django OIDC. In their implementation, they decided
to end Django Session before redirecting to the OIDC provider.
The Django session needs to be retained during the logout process.
An OIDC state is saved to the request session, pass to Agent Connect Logout
endpoint, and verified when the backend receives the Logout callback from Agent
Connect. It seems to follow OIDC specifications.
If for any reason, the Logout flow cannot be initiated with Agent Connect,
(missing ID token in cache, unauthenticated user, etc), the user is redirected
to the final URL, without interacting with Agent Connect.
Prepare adding advanced authentication features. Create a dedicated
authentication Python package within the core app.
This code organization will be more extensible.
Add form to create a mailbox for a mail domain. It sends a http
POST request mail-domains/<mail-domain-id>/mailboxes/ on form
submit. The form appears inside a modal.
Installs react-hook-form, zod, and @hookform/resolvers for form
manipulation and field validation.
The upgrade to react@18.3.1 has a compatibility
issue with next@14.2.3. It creates a error warning
about the fetchPriority prop. This commit fixes the
issue by downgrading react to 18.2.0 as it was
before the last upgrade.
Adds a panel based on teams' one. It fetches all mail-domains
the connected user has relationships with and displays them
as a list of links redirecting to
/mail-domains/<my-domain-name>. Updates e2e tests accordingly.
The upgrade to react@18.3.1 has a compatibility
issue with next@14.2.3. It creates a error warning
about the fetchPriority prop. This commit fixes the
issue by downgrading react to 18.2.0 as it was
before the last upgrade.
Simply display all Mailboxes create for a MailDomain.
LDAP connection is not yet available, it will be implemented soon.
Read and create permissions will be refined soon too.
Pylint was randomly failing due to a warning while unpacking emails.
The W0632 (Possible unbalanced tuple unpacking) was triggered.
Replace tuple unpacking by an explicitly accessing the first element of
the array using index.
Previously, there was a difference between Django's `order_by`
behavior and Python's `sorted` function, leading to test failures
under specific conditions. For example, entries such as 'Jose Smith'
and 'Joseph Walker' were not consistently sorted in the same order
between the two methods.
To resolve this issue, we've ensured that sorting the expected
results in the TeamAccess tests are both case-insensitive and
space-insensitive. This adjustment fix tests flakiness
Create the archi to handle the mails feature.
It has a different layout than the other features,
we don't display the sidebar to keep the
user focused on the mail content.
The email field on the user is renamed to "admin_email" for clarity. The
"email" and "name" fields of user's main identity are made available on
the user model so it is easier to access it.
Rework the header based on latest Johann's design, which introduced a
dropdown menu to mange user account.
In this menu, you can find a logout button, which ends up the backend
session by calling the logout endpoint. Please that automatic redirection
when receiving the backend response were disabled. We handle it in our
custom hook, which reload the page.
Has the session cookie have been cleared, on reloading the page, a new
loggin flow is initiated, and the user is redirected to the OIDC provider.
Please note, the homepage design/organization is still under discussion, I
prefered to ship a first increment. The logout feature will be quite useful
in staging to play and test our UI.
Update all assets related to the previous naming.
I gave some asset a more generic name, so if the app's name chang again
we won't have to rename the logo.
Linked but not assets, meta title and description were updated.
Next.js favicon was replaced by the Equipes' one.
next-env.d.ts lot of types for the next.js.
next.js boilerplate don't version the
next-env.d.ts file, so it was being ignored by git.
This was causing the CI to fail with the ts linter.
Fix a flaky jobs
- searched username could be hidden in the options
depends the dummy data generated.
- remove first place assertion when create a new team,
multiple workers make this assertion flaky
Github Actions are transitioning from Node 16 to Node 20. Make sure we use
latest Github Actions versions to clean any deprecation warnings.
The migration is upcoming.
When we restored the frontend cache, we were restoring
old code as well, we don't want that, we want to only
restore the node_modules.
This commit fixes that.
We improve the build-front caching as well, to cache
only the desk build app.
If any request is taking longer than expected, the user could interact with
the frontend, thinking the previous submission was not taken into account,
and would re-submit a request.
It happened to me while sending an invitation. Replaying request can either
lead to inconsistencies in db for the user, or to errors in requests' response.
I propose to disable all interactive button while submitting a request.
It's good enough for this actual sprint, these types of interactivity issues
could be improved later on.
- Add the teamid to the useUsers query, to not get
the users that are already in the team
- Add a check to not select a user or email
that is already selected
Prob:
Next.js transpiles all the files present in the
`pages` directory. But we don't want to transpile
the providers neither the Layout components.
Solution:
We export these components to a core folder.
- Remove Firefox testing, Firefox browser seems unstable with
Playwright, most of the time the failing tests are the one
with Firefox, Firefox is only 3% of the browser.
- Improve some naming in the test creation to avoid
conflit name.
To improve the speed of the CI, we cache the frontend
install. It will even be reused between pull request
until the yarn.lock has a change.
We cache as well the desk build app, in another cache,
this cache persist only per workflow. It will increase the
speed if we have e2e flaky tests and that we have to relaunch
the e2e job.
On small screens, padding and alignment were causing texts to render wrapped.
It might be a personal choice, but I decided to give more space to the text,
thus avoiding text to wrap and ending up in increasing team info's height.
This issue arises when closing / opening the panel menu.
Team panel was quite wide, and took too much space on small screens.
Johann decided to make it closable. Thus, user that needs to navigate quickly
between their team can open it, use it and then close it.
This animation is a first draft, and should be improved later on. Also, some
accessibility issues might ariase, I have ignored them in this first draft.
Johann recently changed the icon use by the 'add team' button. He preferred
having a square one to contrast more with the 'collapse menu' icon that will
be added later on.
Importing the css from Head was causing a flickering
effect on the button, because of the time the css
load. Next.js was emitting as well a warning about
css being loaded from the Head component.
We moved the css import to the _document.tsx file
as recommended by the Next.js documentation.
Introduce a first draft of 404 page based on Johann's design. Please
refer to the Figma file for more info. This page is tested through
tests e2e. It closes the issue #112
This tests was becoming very flaky because we create
teams in parallel with the other tests.
We use another approch, we checks the aria are
changing according to the sort, we check
as well the api request and that the response
is ok.
We can now delete a member from a team.
We take care of usecases like:
- it is the last owner of the team (cannot delete)
- other owner of the team (cannot delete)
- role hierarchy
Django logs some security warnings we can ignored when deploying over K8s.
Inspired by fun project, I added the Django setting SILENCED_SYSTEM_CHECKS,
and silenced the two that were logging a lot of warning.
Email settings were wrongly configured. It led to unsent email and timeout
response from the backend server.
I forgot to enable the SSL while using the Email service from scalingo.
Generate a proper Django secret key ready for production,
using the provided get_random_secret_key() function.
Store its value in a k8s secret. I generated two values one for
dev and one for staging.
Previous values were triggering security logs.
The potential unused keys was the ones from other
PR, but we don't use crowdin with every PR, so we don't
need to keep the unused keys in the translation files.
Based on works from @manuhabitela, introduce a PoC of the future component.
ApplicationsMenu component is still under construction.
This code was committed for the Wednesday 26th demo, to showcase our future
works. This Next.js integration could be improved, and will for sure!
Don't blame me.
We are displaying all team's members in a datagrid. This propose to enable
sorting on columns, to easily find members.
Please note that few issues were faced when activating the sorting on the
Cunningham components. First, custom columns can not be sorted (yet), a PR
has been merged on Cunningham's side. We're waiting for the next release.
Second, when sorting data rows, if any of the column has some null values,
the datagrid sorting state becomes inconsistent. Thx @AntoLC for spotting the
issue. It's work in progress on Cunningham's side to fix the issue.
Finally, Cunningham export only the SortModel type, which is an array, and
doesn't export its items' type. I might have miss something but it feels weird
to redefine its items type.
Columns wiggle on sorting, because they data is set to undefined while fetching
the next batch. it's visually weird, but not a major pain.
Next release of Cunningham will allow us to set the column to a fixed size.
Add a new job in the CI, which notifies ArgoCD through a webhook that a new
docker image has been pushed to the Docker registry. Thus, ArgoCD can sync
and pull the latest image.
Thus, main will be automatically deployed to staging.
TeamActions is a component that control the actions
that a member can performed on the team.
It contains a dropdown menu that contains the actions:
- Edit Team
- Remove Team
We integrate the endpoint to add a new member
to the team with the multi select seach user.
- If it is a unknown email, it will send an invitation,
- If it is a known user, it will add it to the team.
Discussed with @sampaccoud, Renovate PR do not necessitate ChangeLog updates
Our CI system requires a 'noChangeLog' label or an updated ChangeLog. Currently,
we manually add the label for each Renovate PR.
To improve DX, we configured Renovate to apply the label automatically.
Invite the selected members to the team.
To have a successful invitation:
- none user has this email
- an invitation is not pending for this email and
this team
Part of the team logic was in the create team page,
we moved it to the CardCreateTeam component in
the teams feature.
It will be easier to maintain and reuse the logic.
With the recent addition of mails' templates, Django traduction files
needed to be updated.
It seems that recents backend changes were not reflected into the
Django traduction file. Fixed them, and add traductions related to
the invitation email.
Last revision was made on 2024-01-01
build-mails job builds mails Django templates but was not persisting its
output. This steps was present in Joanie CI. It might have been removed,
when converting Circle CI worflows to Github Actions.
Artifacts are passed between build-mails and test-back jobs. test-back
job has now a dependency to build-mails.
When generating an Invitation object within the database, our intention
is to promptly notify the user via email. We send them an invitation
to join Desk.
This code is inspired by Joanie successful order flow.
Johann's design was missing a link to Desk, I simply added a button which
redirect to the staging url. This url is hardcoded, we should refactor it
when we will deploy Desk in pre-prod or prod environments.
Johann's design relied on Marianne font. I implemented a simpler version,
which uses a google font. That's not important for MVP.
Look and feel of this first invitation template is enough to make our PoC
functionnal, which is the more important.
THis feature is inspired by Joanie. Add two new urls to render Emails
HTML and Text templates.
Developpers can render the email template they are working on. When necessary,
run make mails-build, and reload `_debug__/mail/hello_html`, it will re-render
the updated email template.
Also, I have copy/pasted one template extra tags from Joanie, which loads
bas64 string from static images. This code is necessary to render the dummy
template `hello.html`.
Improved code readability, by extracting this well-scoped unit of
logic in a dedicated method. Also, rename active_invitations to match
'valid' vocabulary used elsewhere in the doc. If no valid invitation
exists, early return to avoid nesting.
Integrate multiselect search users in the
modal add members.
We are using react-select to implement the
multiselect search users. We are using this
library in waiting for Cunningham to implement
the multiselect asynch component.
The CI was controlling if the traduction was made
in every PR. It makes the workflow quite grueling
when we have to change the literal, plus the synch
is complicating when we have multiple PR opened.
We remove the CI control on the traduction, we
will do dedicated PR to update the traduction.
We will add the CI control on the traduction in
the future, before a release by example.
When generating a batch of users using Faker, there's a possibility of
generating multiple users with the same email address. This breaches
the uniqueness constraint set on the email field, leading to flaky
tests that may fail when random behavior coincides unfavorably.
Implemented a method inspired by Identity's model to ensure unique
email addresses when creating user batches with Faker.
Updated relevant tests for improved stability.
Important ordering fields for TeamAccess depend on user's
identities data. User and identities has a one-to-many relationship,
which forced us to prefetch the user-related data when listing
team's accesses.
Prefetch get data from the database using two SQL queries, and
join data in Python. User's data were not available in the first
SQL query.
Without annotating the query set with user main identities data,
we could not use default OrderingFilter backend code, which order_by()
the queryset.
Enhance list capabilities, by adding the OrderingFilter as filter backend,
to the TeamAccess viewset.
API response can be ordered by TeamAccess role. More supported ordering
fields will be supported later on.
We created a custom Pagination class, were max_page_size is overriden.
I decided to add bare minimum tests to make sure we can set the maximum
page size using the 'page_size' query parameter.
Our Pagination class inherits from the PageNumberPagination Django class.
However, this base class as not ordering attribute. Thus, setting a
default value wont have any effect on the code.
Why did we end up passing a value to this non-existing attribute? Becasue
we copy/pasted some code sources from Joanie, and Joanie also has this
attribute set to a default value.
If you take a look at DRF pagination style documentation, the only three
attributes they set on the child class are 'page_size', 'max_page_size'
'page_size_query_param'. 'ordering' is not mentionned in the attributes
you may override. However, the CursorPagination class offers the latter
attribute, which may explain why we did end up setting this non-existing
attribute in Joanie.
djangorestframework released version 3.15.0, which includes modifications of
details upon returning 404 errors (see related issue
https://github.com/encode/django-rest-framework/pull/8051).
This commit changes the expected details of 404 responses in our tests,
to match DRF 3.15.0.
Based on @rouja reco, I added a dedicated ingress to serve Django Admin
pages and Django statics. The admin route will be secured by the oauth proxy.
I simply copy/pasted the first ingress template, and adapted it.
This commit is working in progress. I have added an extra chart to take
benefits of the Redis operator developed by Indie hoster.
When using the dev environment, I used bitnami redis chart to deploy
a Redis service with authentication disable.
CSRF trusted origins are set using an environment variable. The env
value was wrongly name to CORS_ALLOWED_ORIGINS, which doesn't exist
in our Django configurations. I fixed this minor issue.
Set OIDC secrets for the dev environment. Please note that we use different
secrets between dev and staging. Why? Benoit created two client id, thus we
could easily tests Agent Connect feature from the local host and the staging
one.
The local host is desk.127.0.0.1.nip.io. If this value change at any time,
please consider asking Benoit to update the host value linked to the dev
client id.
Thx @rouja for your help on deploying Desk. This commit slightly modifies
helm charts and helmfile to prepare the initial project deployment in a
staging environment.
@rouja updates:
- added secrets files for dev and staging environments (dev's one is empty)
- disable ingress by default, to avoid any security issue
- added an extra chart to benefit from Indie hoster Postgres operator
Thx to this commit we deployed a first draft version figured out
that the Django session were broken. We are using a cache session engine,
and wrongly configure cache backend to local memory. Thus, Django server
is not able to resolve the session, and enters in an infinite loop to
log-in the user.
Please note that this Helmfile is uncomplete, it lacks services as
redis, celery, mail ... which are declared in the Docker Compose file
but not yet used in development and production images.
Thus, to run the Desk Helm chart, we only add a postgres database to run the
Django backend server, and apply migrations.
For now, this Helmfile is quite hard to test in dev environment, because the
frontend redirects automatically to the SSO login page. We cannot really
assess if backend and frontend are working properly. We might adjust some
configurations after the first deployment in stagging.
(We are a bit in rush, to respect the current sprint deadline.)
Development values points https://desk.127.0.0.1.nip.io URL. Please note that
the frontend image for now has been built with this URL for the backend address.
Meaning that we either need to rebuild and publish a frontend image with the
staging URL when deploying the project, or enhance our frontend image, to pass
the backend URL at runtime.
First, thanks a LOT @rouja for your help along the way.
This commit propose a first draft of Helm chart to prepare deployment.
It follows Plane's Helm Chart, hosted on the shared team repo,
please https://github.com/numerique-gouv/helm-charts, PR #11
It offers advanced templating function under _helpers.tpl, an auto-generated
README file when running ./generate-readme.sh, and a clear files structure.
The chart itself is quite simple. We have two deployments, one for the
frontend and one for the backend. Both need a dedicated service, which are
exposed using a common ingress. Frontend is accessible from the / path and
backend's from /api path.
Please note, we added a backend job to migrate the database when deploying
backend's pods. This job should be auto-cleaning itself 100s after it completes
to avoid any error when syncing helm.
values.yaml file is quite pristine, all common env variables will be set
in helmfile configuration.
Deploying frontend static files through kubernetes is temporary, we plan to
either remplace it by an external CDN or use minio to host static output in
a S3 bucket within the cluster.
When we pull the translations from crowdin we
get lot of git diff noise with the json file.
We order the keys in the json file to make the
diffs more readable.
`backend-development` and `backend-production` CMD syntaxes were
using a Shell Form. Shell form prevented Unix signals from reaching
our container correctly, such as SIGTERM. Also, the shell process
ends up being the PID 1, instead of our Python scripts.
Docker recommends to use the exec form whenever possible.
In development, sessions are saved in local memory. It's working well,
however it doesn't adapt to a kubernetized setup. Several pods need
to access the current sessions, which need to be stored in a single
source of truth.
With a local memory cache, pods cannot read session saved in other pods.
We end up returning 401 errors, because we cannot authenticate the user.
I preferred setting up a proper cache than storing sessions in database,
because in the long run it would be a performance bottleneck. Cache will
decrease data access latency when reading current sessions.
I added a Redis cache backend to the production settings. Sessions would
be persisted to Redis. In K8s, a Redis operator will make sure the cached
data are not lost.
Two new dependencies were added, redis and django-redis.
I followed the installation guide of django-redis dependency. These
setting were tested deploying the app to a local K8s cluster.
It was quite confusing having development, production and
frontend images' names in the same Docker file. New comers
to the project would have some difficuluties when
differentiating frontend from backend images.
Try to make these naming more explicit and consistent.
Thanks @rouja for your recommendation.
This is a major issue. Docker Images were built and published with a
root user in the CI.
if a user manages to break out of the application running as root in the
container, he may gain root user access on host. In addition, configuring
container to user unprivileged is the best way yo prevent privilege
escalation attacks.
We mitigated this issue by creating a new environment variable DOCKER_USER.
DOCKER_USER is set with id -u and id -g outputs. Then, it is passed as a
build-args when running docker/build-push-action steps.
Ignore frontend dependencies when coping frontend sources to build
the frontend Docker image. It would improve a bit performances locally,
when building the frontend image.
Build and push the frontend image to DockerHub. Backend an Frontend
images will be stored in separate repos: people-backend and people-frontend.
It will be cleaner than managing all images in a single repo and creating
tags to discriminate frontend and backend images.
CI code is not factorized between jobs. Frontend and backend jobs could be
a bit factorized. Hovewer it might be a bit premature, and I prefer having
them decoupled for now. @rouja suggested to introduce a custom github actions
to avoid maintaining the same logic accross different repo.
Please not as the images are built from the same Dockerfile, it's important
to precise the right target.
Updated to Node Image version 20 to align with the frontend image. It will
save us having two different Node versions in the same docker file, and
should not impact mail-builder.
To facilitate deployment on Kubernetes, we've introduced a Docker image for the
frontend. The Next.js project is built, and its static output is served using an
Nginx reverse proxy.
Since DevOps lacks a certified cold storage solution (e.g., S3) for serving
static files, we've decided to containerize the frontend as a quick workaround
for deploying staging environments.
Please note this Docker Image is WIP. One of the main issue still not resolved
concerns environment variables, which are only available when building the
Docker Image. Thus, having different environment variables values between
environment (dev, pre-prod, prod) will require us to build several frontend
images, and tag them with the appropriate target environment.
The `.env.production` values are not the final ones. For now, they were set to
dev values. It allows us to test the frontend image with the development setup.
Important: The frontend image is built-on top of an unprivileged Nginx image,
which exposes by default port 8080 instead of 80 for classic Nginx image.
You can find more info https://github.com/nginxinc/docker-nginx-unprivileged.
The Docker Compose Nginx service is used to proxy OIDC requests to keycloak,
in order to share the same host when initiating an OIDC flow, from outside and
inside docker virtual network.
All Nginx configurations related to serve frontend static build were moved to a
newly created conf file under src/frontend/apps/desk. When starting the frontend
image, we desire to start the minimum Nignx config required to serve frontend
statics.
The current implementation of our product demo via the make command lacks
user identity for a significant portion of generated users, limiting the
realism of the showcased scenarios. As it stands, users created by the make
command lack complete information, such as full names and email addresses,
because they don't have any identity.
I tried to come up with the simplest solution:
We now generate a very small portion of our users with 0 identities. The
probability for users to have only 1 identity is the highest but they
can have up to 4 with decreasing probabilities. I removed the possibility
to set a maximum number of identities as it doesn't bring any value.
3% percent of the identities created will have no email and 3% no name.
Fixes https://github.com/numerique-gouv/people/issues/90
Some tokens were not available from the hook.
We only had the tokens of the currentTheme available
but actually the theme is an augmentation of the
default theme, so we should use the default theme
tokens as a base and then override them with the
currentTheme tokens.
It is what this commit does.
We often need unstyled button to wrap around some content,
we were using Cunningham's button for this purpose,
but it is not the best choice as lot of style is applied
to their buttons.
This component is a simple wrapper around the button
element with all the Box functionalities. Usefull
for wrapping icons by example.
Compute Trigram similarity on user's name, and sum it up
with existing one based on user's email.
This approach is inspired by Contact search feature, which
computes a Trigram similarity score on first name and last
name, to sum up their scores.
With a similarity score influenced by both email and name,
API results would reflect both email and name user's attributes.
As we sum up similarities, I increased the similarity threshold.
Its value is empirical, and was finetuned to avoid breaking
existing tests. Please note, the updated value is closer to the
threshold used to search contacts.
Email or Name can be None. Summing two similarity scores with
one of them None, results in a None total score. To mitigate
this issue, I added a default empty string value, to replace
None values. Thus, the similarity score on this default empty
string value is equal to 0 and not to None anymore.
When testing user search, we generated few identities
with mocked emails.
Name attribute was introduced on Identity model. Currently
names are freely and randomly generated by the factory.
To make this mocked data more realist, mock also identities'
names to match their email.
It should not break existing tests, and will make them more
predictable when introducing advanced search features.
Few minor layout issues were fixed.
First display label and dates inline, so they wrap nicely
when screen's size decreases. It also fixes the text overflow
when the screen's size is tiny.
Then, align screen with the Figma design, where items are
justified on the left of the Team info component.
Nest invitation router below team router and add create endpoints for
authenticated administrators/owners to invite new members to their team,
list valid and expired invitations or delete invite altogether.
Update will not be handled for now. Delete and recreate if needed.
Integrate the design and functionality
for updating a member's role.
Managed use cases:
- when the user is an admin
- when the user is the last owner
- when the user want to update other orner
Integrate the action button dropdown in
the member grid. For the moment it will be
used to update the role of a member.
Manage use cases:
- Does not display when member's role
- Does not display when member is an admin
that wants to update owner role.
Break copy/pasted comment from Joanie in several inline
comments, that are more specific and easy to read.
Hopefully, it will help future myself understanding this
queryset and explaining it.
To compute accesses's abilities, we need to determine
which is the user's role in the team.
We opted for a subquery, which retrieves the user's role
within the team and annotate queryset's results.
The current subquery was broken, and retrieved other
users than the request's user. It led to compute accesses'
abilities based on a randomly picked user.
Abilities on team accesses are computed based on request user role.
Thus, members' roles in relation with user's role matters a lot, to
ensure the abilities were correctly computed.
Complexified the test that lists team accesses while being authenticated.
More members are added to the team with privileged roles. The user
is added last to the less with the less privileged role, "member".
Order matters, because when computing the sub query to determine
user's role within the team, code use the first result value to set the
role to compute abilities.
When running make ruff-check, a warning informs the user that
some config are deprecated, and gives her the step to migrate.
This warning appears after Ruff released its v0.2.0.
Fix it, by keeping our pyproject.toml up to date.
We recently updated Ruff from 0.2.2 to v0.3, which introduced
Ruff 2024.2 style. This new style updated Ruff formatter's behavior,
making our make lint command fails.
Ruff 2024.2 style add a blank line after the module docstring.
Please take a look at Ruff ChangeLog to get more info.
Add serializers to return basic user info when listing /team/<id>/accesses/
endpoint. This will allow front-end to retrieve members info without having
to query API for each user.id.
Downgrade @openfun/cunningham-react to 2.4.0, because of a
compatibility problem with Jest.
We add this package with this version to the ignore list
in renovate.json, when we will have a new compatible version, we will
remove it from the ignore list.
Downgrade @types/react-dom to 18.2.18.
The lastest version seems to have lot of compatibility
issues with other packages:
- @openfun/cunningham-react
- @tanstack/react-query-devtools
- next
We add this package with this version to the ignore list
in renovate.json, when we will have a new compatible version, we will
remove it from the ignore list.
When we run e2e tests with the CI, we are doing lot of
calls to the backend in a short amount of time. This can
lead to a rate limit particulary on the "user/me" endpoint.
To avoid this, we will use different backend settings
for the e2e tests.
Secret settings should not contain any default value as we risk shipping
them to production. The default value can be set via an environment variable
in the `env.d/development/common` file: OIDC_RP_CLIENT_SECRET
We need a name for the user when we display the members in the
frontend. This commit adds the name column to the identity model.
We sync the Keycloak user with the identity model when the user
logs in to fill and udpate the name automatically.
Add the team info component to the team page.
This component shows some informations about the team:
- name
- amount of members
- date created
- date updated
Instead of interacting with Keycloak, the frontend navigate to the
/authenticate endpoint, which starts the Authorization code flow.
When the flow is done, the backend redirect back to the SPA,
passing a session cookie and a csrf cookie.
Done:
- Query GET user/me to determine if user is authenticated yet
- Remove Keycloak js dependency, as all the OIDC logic is handled by the backend
- Store user's data instead of the JWT token
Integrate 'mozilla-django-oidc' dependency, to support
Authorization Code flow, which is required by Agent Connect.
Thus, we provide a secure back channel OIDC flow, and return
to the client only a session cookie.
Done:
- Replace JWT authentication by Session based authentication in DRF
- Update Django settings to make OIDC configurations easily editable
- Add 'mozilla-django-oidc' routes to our router
- Implement a custom Django Authentication class to adapt
'mozilla-django-oidc' to our needs
'mozilla-django-oidc' routes added are:
- /authenticate
- /callback (the redirect_uri called back by the Idp)
- /logout
Backend and Frontend send requests to Keycloak through Nginx.
Thus, all requests from frontend and backend shared a same host
when received by Keycloak.
Otherwise, the flow is initiated from http://localhost:8080. When the Backend
calls token endpoint from Keycloak container at http://keycloak:8080,
the JWT token issuer and sender are mismatching.
Downgrade @types/react-dom to 18.2.18.
The lastest version seems to have lot of compatibility
issues with other packages:
- @openfun/cunningham-react
- @tanstack/react-query-devtools
- next
2 routers exists in Next.js, "app" router and "page" router.
The "app" router has a bug introduced in Next.js 13.4.14, which is
not fixed yet. For the moment we cannot use dynamic routes with
"app" router with an SPA. As advised by the Next.js team, we
migrated to the "pages" router.
Enabled Dockerflow Django app by activating liveness probes. The previously
unavailable routes such as `__heartbeat__` and `__lbheartbeat__` are now
accessible. New endpoints include:
* GET /__version__
* GET /__heartbeat__
* GET /__lbheartbeat__
The local deployment of the Production image through docker-compose was
failing due to issues in the Django configurations, influenced by Joanie.
The bug stemmed from a dependency on a development-specific package
(drf-spectacular-sidecar) while attempting to run the application in
production mode.
Changes Made:
- Introduced new Django settings for local demo environments.
- Uncommented the nginx configuration to address the production image
deployment issues.
Create invitation model, factory and related tests to prepare back-end
for invitation endpoints. We chose to use a separate dedicated model
for separation of concerns, see
https://github.com/numerique-gouv/people/issues/25
*Broken Identity string representation
Resolving a format error in the Identity string representation caused by
potential None values in the email field. This issue was discovered when
attempting to access the User details page in the Django Admin
*Broken User creation form
The replacement of the User's username with an email led to errors in
the UserAdmin class. The base class used the 'username' field in the
'add_fieldsets' attribute. This problem was discovered while attempting to
create a new user in the Django Admin.
Some circular dependency problems started to appear with Jest.
This commit fixes the problem by removing the feature index
file and moving the exports to the respective feature.
We have added workers to playwright to run tests in parallel,
this will help us to run tests faster.
The tests run on a commun database, so to keep the tests
stable between browsers, we created 3 different
users to run the tests, it will avoid to have commun data
stepping on each other.
- Improve the message when a translation is missing in the app,
now it will show the key of the missing translation.
- It also show the translation that are in crowdin but not in the
app.
- Add missing translations
sops_age__list_0__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBIbkIyRi9jTGEzL2RZcGFS\nVkhuN2xMN1FBcm04eGJQUnpvQVRpdmFmaFF3CjdoNVhSM0lrb0lTSXAyUytSKzhG\nNERYaC80OThmUUxCSmt2U2Q0Mno3b0kKLS0tIFRjQzYyOEcwTnRScGhzaGxwaGFL\namxEUXNaeVhWUjYwbzR2Z1Uzb3JsTlEK6tyhbRmcUP4Aql49DPkrYb5tbwvK2EdA\ntvyPQo4pPit+pzgqsVgW+O8Wo4/rYLlITfuVRrOfHEaH3wmf6hziSw==\n-----END AGE ENCRYPTED FILE-----\n
sops_age__list_1__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBIL2NwVjVtVmd1ZE15WmFC\nVGl5cnU1OHVUcXAxWmZCQXBJbXBlcXFUN0RzClpOVXBTek9wbVB6M2s5TWlFbUo3\nelZrT3dsK3p6cEJGTGJoSTRvaEh4NW8KLS0tIGlsL29oSEthU28xV0RERFIxTnZR\nSSs3YjdUT0NyTEtlbjlmZXVOaTd3SzgK5jFJGREfJ/HVytWsCKWFsqM5JoaFSnhv\n538KzzldzcbtWfnY+bQ6A2EBjETwOzCTuQB8axAMj0URXPI+qelKIQ==\n-----END AGE ENCRYPTED FILE-----\n
sops_age__list_2__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBzOWtwSytMNGRmSHNQN0Zz\nV1RTWktXT1c5VjYvNTg2V0x1U3JSTFA1dmtvCnc4RHRodjVxSzcrZXJycCtJMGNy\nZ3ZwbTFDTUc4TTMxbm8rNXQ1ME90Q1UKLS0tIERPUmg1dSt5OWQwaXV0Y2FiYWdt\nVzlhcFNvUS9Nd3A5ZTYwT2lLUU5WVU0KGHSm1BQ3voKs98WiXNLO6hlqoiQmi1/F\n2RCBkE4/gOVyAmJAjOFizaF0Dhd7Ba4KS5l5QylFHloL8XtyixEhog==\n-----END AGE ENCRYPTED FILE-----\n
sops_age__list_3__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBmYlB6cEZKK3NLbm5Ob0hi\nS3M0OFoyTHhVYVA1WmcvSHc2MUErQ2JyaUh3Cnp1NlYvU1BPUnVON29UZWUvWmEz\nOW85N013VUJ3Q1ZZUmMvTWdYclBZTUkKLS0tIG1KODdkazhTUHRPMXZXQnJFNXJY\nOFlSNjZCbkpxcm9tN1dFTkZ4K1lETFUKhiPwKEG71OlTcK/Ul1GKGayLy025vAmo\nNgQUhbqXf7KmqDAmrQNzXTsLsOpRi33l66jFSGkTsFtiXNlmjFljKg==\n-----END AGE ENCRYPTED FILE-----\n
sops_age__list_4__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBJUzI2eDB0MU41YWoxcjAv\nOU8yMlowb3IybCtzcFBBTEVlUzdBczZkQlFNCk5tbnhLR2kweGNFZmx1OFgwaWZU\nTzRKZTBZamJjRFNYOUJlYmxTUVg4S2MKLS0tIDNGdlVVc0hLK1BPTUJVc0tPb1lK\ndVVWVi9GRkxwYXR1cXMrdnlsejJBeGsKKmVWvIMrBYH+UrDMkZPxN8KWnCgA6WK2\nbexMYr2AIF2QMbhck7XW2NuvAwvwjbJMfcd+cp9boe+EcC4YjdJZlw==\n-----END AGE ENCRYPTED FILE-----\n
Thank you for taking the time to contribute! Please follow these guidelines to ensure a smooth and productive workflow. 🚀🚀🚀
To get started with the project, please refer to the [README.md](https://github.com/suitenumerique/docs/blob/main/README.md) for detailed instructions.
Please also check out our [dev handbook](https://suitenumerique.gitbook.io/handbook) to learn our best practices.
## Help us with translations
You can help us with translations on [Crowdin](https://crowdin.com/project/lasuite-people).
Your language is not there? Request it on our Crowdin page 😊.
## Creating an Issue
When creating an issue, please provide the following details:
1.**Title**: A concise and descriptive title for the issue.
2.**Description**: A detailed explanation of the issue, including relevant context or screenshots if applicable.
3.**Steps to Reproduce**: If the issue is a bug, include the steps needed to reproduce the problem.
4.**Expected vs. Actual Behavior**: Describe what you expected to happen and what actually happened.
5.**Labels**: Add appropriate labels to categorize the issue (e.g., bug, feature request, documentation).
## Selecting an issue
We use a [GitHub Project](https://github.com/orgs/suitenumerique/projects/1) in order to prioritize our workload.
Please check in priority the issues that are in the **TODO cette semaine** column.
## Commit Message Format
All commit messages must adhere to the following format:
`<gitmoji>(type) title description`
* <**gitmoji**>: Use a gitmoji to represent the purpose of the commit. For example, ✨ for adding a new feature or 🔥 for removing something, see the list here: <https://gitmoji.dev/>.
***(type)**: Describe the type of change. Common types include `backend`, `frontend`, `CI`, `docker`etc...
***title**: A short, descriptive title for the change, starting with a lowercase character.
***description**: Include additional details about what was changed and why.
### Example Commit Message
```
✨(frontend) add user authentication logic
Implemented login and signup features, and integrated OAuth2 for social login.
```
## Changelog Update
Please add a line to the changelog describing your development. The changelog entry should include a brief summary of the changes, this helps in tracking changes effectively and keeping everyone informed. We usually include the title of the pull request, followed by the pull request ID to finish the log entry. The changelog line should be less than 80 characters in total.
### Example Changelog Message
```
## [Unreleased]
## Added
- ✨(frontend) add AI to the project #321
```
## Pull Requests
It is nice to add information about the purpose of the pull request to help reviewers understand the context and intent of the changes. If you can, add some pictures or a small video to show the changes.
### Don't forget to:
- check your commits
- check the linting: `make lint && make frontend-lint`
- check the tests: `make test`
- add a changelog entry
- squash your commits
- rebase your branch on the latest `main` branch before pushing your changes `git pull --rebase origin main`
### Process to have a nice commit history
In the life time of your PR, you may need to add commits to fix things or add new features.
Commit after commit, your PR will be full of commits but you have to clean it up with the following commands before merging on `main`:
Gradually you can use `--fixup` to add commits to some of previous commit ( for example 1234567890).
```
git commit --fixup=1234567890
```
Then, you can squash your commits with the following command:
```
git rebase --autosquash -i -r HEAD~<number-of-commits>
```
Or you can use:
```
git rebase -i HEAD~<number-of-commits>
```
and move, squash and/or rename your commits manually. You can squash them with previous commit replacing the `pick` by `f`. You can rename them with replacing the `pick` by `r`.
Tada! You have a clean commit history.
Once all the required tests have passed, you can request a review from the project maintainers.
## Code Style
Please maintain consistency in code style. Run any linting tools available to make sure the code is clean and follows the project's conventions.
## Tests
Make sure that all new features or fixes have corresponding tests. Run the test suite before pushing your changes to ensure that nothing is broken.
## Asking for Help
If you need any help while contributing, feel free to open a discussion or ask for guidance in the issue tracker. We are more than happy to assist!
If you have any issue regarding security, please disclose the information responsibly submiting [this form](https://vdp.numerique.gouv.fr/p/Send-a-report?lang=en) and not by creating an issue on the repository. You can also email us at support-regie@numerique.gouv.fr
We appreciate your effort to make People more secure.
## Vulnerability disclosure policy
Working with security issues in an open source project can be challenging, as we are required to disclose potential problems that could be exploited by attackers. With this in mind, our security fix policy is as follows:
1. The Maintainers team will handle the fix as usual (Pull Request,
release).
2. In the release notes, we will include the identification numbers from the
GitHub Advisory Database (GHSA) and, if applicable, the Common Vulnerabilities
and Exposures (CVE) identifier for the vulnerability.
3. Once this grace period has passed, we will publish the vulnerability.
By adhering to this security policy, we aim to address security concerns
effectively and responsibly in our open source software project.
The mailing solution provided in La Suite is [La Messagerie](https://webmail.numerique.gouv.fr/), using [Open-XChange](https://www.open-xchange.com/) (OX). OX not having a provisioning API, 'dimail-api' or 'dimail' was created to allow mail-provisioning through People.
API and its documentation can be found [here](https://api.dev.ox.numerique.gouv.fr/docs#/).
## Use of dimail container
To ease local development, dimail provides a container that we embark in our docker-compose. In "FAKE" mode, it simulates all responses from Open Exchange.
Bootstraping with command `make bootstrap` creates a container and initializes its database.
Additional commands :
- Reset and populate the database with all the content of your People database with `dimail-setup-db`
## Architecture
### Domains
Upon creating a domain on People, the same domain is created on dimail and will undergo a series of checks. When all checks have passed, the domain is considered enabled.
Domains in OX have a field called "context". "Contexts" are shared spaces between domains, allowing users to discover users not only on their domain but on their entire context.
> [!NOTE]
> Contexts are only implemented in La Messagerie and are not currently in use in People. Domains created via People are in their own context.
People users can have 3 levels of permissions on a domain:
- Viewers can
- see the domain's information
- list its mailboxes and managers
- Administrators can
- create mailboxes
- invite collaborators to manage domain
- change role of a viewer to administrators
- all of viewers permissions
- Owners can
- promote administrators owners and demote owners
- all of viewers and administrators' permissions
> [!NOTE]
> Contexts are only implemented in La Messagerie and are not currently in use in People. Domains created via People are in their own context.
### Mailboxes
Mailboxes can be created by a domain owners or administrators in People's domain tab.
On enabled domains, mailboxes are created at the same time on dimail (and a confirmation email is sent to the secondary email).
On pending/failed domains, mailboxes are only created locally with "pending" status and are sent to dimail upon domain's (re)activation.
On disabled domains, mailboxes creation is not allowed.
Whenever we are cooking a new release (e.g. `4.18.1`) we should follow a standard procedure described below:
1. Checkout and pull changes from the `main` branch to ensure you have the latest updates.
```bash
git checkout main
git pull
```
2. The next steps are automated in the `scripts/release.py`, you can run it using:
```bash
make release
```
This script will ask you for the version you want to release and the kind of release (patch, minor, major). It will then:
- Create a new branch named: `release/4.18.1`.
- Bump the release number for backend and frontend project.
- Update the project's `Changelog` following the [keepachangelog](https://keepachangelog.com/en/0.3.0/) recommendations
- Commit your changes with the following format: 🔖 release emoji, type of release (patch/minor/patch) and release version:
```text
🔖(minor) release version 4.18.1
```
- Triggers Crowdin translation action and open related PR
- Open release PR. Wait for an approval from your peers and merge it.
> [!NOTE]
> It also open the PR for pre-prod deployment, see following section.
3. Following release script instructions,
- merge translation and release pulls requests
- tag and push your commit:
```bash
git tag v4.18.1 && git push origin tag v4.18.1
```
This triggers the CI building new Docker images. You can ensure images were successfully built on Docker Hub [here for back-end](https://hub.docker.com/r/lasuite/people-backend/tags) and [here for front-end](https://hub.docker.com/r/lasuite/people-frontend/tags).
# Deploying
## Staging
The `staging` platform is deployed automatically with every update of the `main` branch.
## Pre-prod and production
If you used the release script and had permission to push on [lasuite-deploiement](https://github.com/numerique-gouv/lasuite-deploiement), a deployment branch has been created. You can skip step 1.
Otherwise, for manual preprod and for production deployments :
1. Bump tag version for both front-end and back-end images in the `.gotmpl` file located in `manifests/<your-product>/env.d/<your-environment>/`,
2. Add optional new secrets and variables, if applicable
For detailed information, please refer to the [OAuth 2.0 Resource Server documentation](https://www.oauth.com/oauth2-servers/the-resource-server/) and review the relevant commits that implement the resource server backend.
---
#### Overview
A resource server is a crucial component in an OAuth 2.0 ecosystem. It is responsible for accepting access tokens issued by the authorization server (in this case, Agent Connect), introspecting and validating those tokens, and then returning the requested protected resources to the client.
By implementing a resource server, we can securely share data between different services within La Suite. This ensures that only clients with the appropriate scopes and permissions can access specific resources, thereby enhancing security and maintaining proper access control across services.
---
#### Disclaimer
- Currently compatible only with Agent Connect.
- The development setup requires simplification, with dependencies on Agent Connect ideally mocked.
- Terminology aligns with the specification: what is referred to as a "resource server" is known as a "data provider" in Agent Connect.
- This documentation is WIP.
---
## Running Locally
#### Prerequisites
- **Agent Connect Stack**: Ensure the local Agent Connect stack is running. A solid understanding of its configuration and operation is recommended (advanced level).
- **People Stack**: Make sure the People stack is up and running.
- **Ngrok**: Install and set up Ngrok for secure tunneling.
---
### Update People's configurations
#### Environment variables
Agent Connect includes two pre-configured mocked data providers in its default stack (`bdd-fca-low`).
Use the client ID and client secret from one of these data providers. **Note:** Make sure to retrieve the decrypted secret, as it is stored encrypted in the database. You can find these values in the `dp.js` fixture file, where they are exposed in a comment.
Configure your environment with the following values from Agent Connect:
To enable communication between the Docker networks for People and Agent Connect, update your docker-compose configuration. This setup is required because the Authorization Server and Resource Server will exchange requests over a back-channel, necessitating their accessibility to each other.
1.**Create a Network**: Define a new network to bridge the two Docker networks.
```yaml
networks:
authorization_server:
external: true
driver: bridge
name: "${DESK_NETWORK:-fc_public}"
```
2. **Update Network for `app-dev`**: Ensure your `app-dev` service is connected to both the default network and the new `authorization_server` network.
```yaml
app-dev:
...
networks:
- default
- authorization_server
```
#### Ngrok
To expose your local resource server through an HTTP tunnel, use Ngrok. This is necessary because, in the Agent Connect development stack, the resource server needs to be accessible to a user agent via a publicly accessible URL.
```
$ ngrok http 8071
```
---
### Update AgentConnect's configurations
Modify the AgentConnect configuration to include the local resource server settings.
**Update `fsa1-low` Environment File**:
Add your Ngrok URL and configure the `DATA_APIS` list with the appropriate values.
Adjust the configuration for the data provider to match your local setup. Ensure that the `client_id`, `client_secret`, and other parameters are correctly set and aligned with your environment. This can be configured through environment variables.
```javascript
const dps = [
// Data Provider Configuration
{
uid: "6f21b751-ed06-48b6-a59c-36e1300a368a",
title: "Mock Data Provider - 1",
active: true,
slug: "DESK",
client_id: "***",
client_secret: "***",
// client_secret decrypted : ****
jwks_uri: "https://your-ngrok-url/api/v1.0/jwks", // Update this line
checktoken_signed_response_alg: "ES256",
checktoken_encrypted_response_alg: "RSA-OAEP",
checktoken_encrypted_response_enc: "A256GCM",
},
];
```
**Note**: Ensure that the `jwks_uri` and other cryptographic parameters (e.g., `checktoken_signed_response_alg`, `checktoken_encrypted_response_alg`, and `checktoken_encrypted_response_enc`) match your actual setup and are configured via environment variables where necessary.
---
### Usage
This section is a work in progress. Please note the following important points:
#### User `sub` Matching
Ensure that the `sub` (subject) field for users in AgentConnect matches the corresponding value in the People database. To synchronize this, you can run `make demo`, then edit the user's `sub` field to match the value returned by AgentConnect. For this, you'll need to update the editable field in Django Admin, specifically in `admin.py`. Adjust the `get_readonly_fields` method as follows:
```python
def get_readonly_fields(self, request, obj=None):
"""The 'sub' field should only be editable during creation, not for updates."""
if obj:
return self.readonly_fields
return self.readonly_fields + ["sub"] # update this line adding 'sub'
```
#### Scope for `groups`
Ensure that the `groups` scope is requested from the service provider during authentication with AgentConnect.
#### Resource Server Requests
By default, the `fsa1-low` environment calls the resource server using a POST request.
#### Testing
Most of the testing has been done using the `/users/me` endpoint. Update the `api/viewset.py` configuration to allow both GET and POST methods for this endpoint:
```python
@decorators.action(
detail=False,
methods=["get", "post"], # update this line adding 'post'
@@ -45,8 +45,10 @@ class Migration(migrations.Migration):
('id',models.UUIDField(default=uuid.uuid4,editable=False,help_text='primary key for the record as UUID',primary_key=True,serialize=False,verbose_name='id')),
('created_at',models.DateTimeField(auto_now_add=True,help_text='date and time at which a record was created',verbose_name='created at')),
('updated_at',models.DateTimeField(auto_now=True,help_text='date and time at which a record was last updated',verbose_name='updated at')),
('language',models.CharField(choices="(('en-us', 'English'), ('fr-fr', 'French'))",default='en-us',help_text='The language in which the user wants to see the interface.',max_length=10,verbose_name='language')),
('sub',models.CharField(help_text='Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_ characters only.',max_length=255,unique=True,validators=[django.core.validators.RegexValidator(message='Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_ characters.',regex='^[\\w.@+-]+\\Z')],verbose_name='sub')),
('language',models.CharField(choices=settings.LANGUAGES,default='en-us',help_text='The language in which the user wants to see the interface.',max_length=10,verbose_name='language')),
('timezone',timezone_field.fields.TimeZoneField(choices_display='WITH_GMT_OFFSET',default='UTC',help_text='The timezone in which the user wants to see times.',use_pytz=False)),
('is_device',models.BooleanField(default=False,help_text='Whether the user is a device or a real user.',verbose_name='device')),
('is_staff',models.BooleanField(default=False,help_text='Whether the user can log into this admin site.',verbose_name='staff status')),
@@ -60,7 +62,7 @@ class Migration(migrations.Migration):
('id',models.UUIDField(default=uuid.uuid4,editable=False,help_text='primary key for the record as UUID',primary_key=True,serialize=False,verbose_name='id')),
('created_at',models.DateTimeField(auto_now_add=True,help_text='date and time at which a record was created',verbose_name='created at')),
('updated_at',models.DateTimeField(auto_now=True,help_text='date and time at which a record was last updated',verbose_name='updated at')),
('sub',models.CharField(help_text='Required. 255 characters or fewer. Letters, numbers, and @/./+/-/_ characters only.',max_length=255,unique=True,validators=[django.core.validators.RegexValidator(message='Enter a valid sub. This value may contain only letters, numbers, and @/./+/-/_ characters.',regex='^[\\w.@+-]+\\Z')],verbose_name='sub')),
constraint=models.CheckConstraint(check=models.Q(('base__isnull',False),('owner__isnull',True),_negated=True),name='base_owner_constraint',violation_error_message='A contact overriding a base contact must be owned.'),
migrations.CreateModel(
name='TeamWebhook',
fields=[
('id',models.UUIDField(default=uuid.uuid4,editable=False,help_text='primary key for the record as UUID',primary_key=True,serialize=False,verbose_name='id')),
('created_at',models.DateTimeField(auto_now_add=True,help_text='date and time at which a record was created',verbose_name='created at')),
('updated_at',models.DateTimeField(auto_now=True,help_text='date and time at which a record was last updated',verbose_name='updated at')),
constraint=models.CheckConstraint(check=models.Q(('base',models.F('id')),_negated=True),name='base_not_self',violation_error_message='A contact cannot be based on itself.'),
constraint=models.CheckConstraint(condition=models.Q(('base__isnull',False),('owner__isnull',True),_negated=True),name='base_owner_constraint',violation_error_message='A contact overriding a base contact must be owned.'),
),
migrations.AddConstraint(
model_name='contact',
constraint=models.CheckConstraint(condition=models.Q(('base',models.F('id')),_negated=True),name='base_not_self',violation_error_message='A contact cannot be based on itself.'),
),
migrations.AlterUniqueTogether(
name='contact',
unique_together={('owner','base')},
),
migrations.AddConstraint(
model_name='identity',
constraint=models.UniqueConstraint(fields=('user','email'),name='unique_user_email',violation_error_message='This email address is already declared for this user.'),
('id',models.UUIDField(default=uuid.uuid4,editable=False,help_text='primary key for the record as UUID',primary_key=True,serialize=False,verbose_name='id')),
('created_at',models.DateTimeField(auto_now_add=True,help_text='date and time at which a record was created',verbose_name='created at')),
('updated_at',models.DateTimeField(auto_now=True,help_text='date and time at which a record was last updated',verbose_name='updated at')),
('registration_id_list',django.contrib.postgres.fields.ArrayField(base_field=models.CharField(max_length=128),blank=True,default=list,size=None,verbose_name='registration ID list')),
'constraints':[models.CheckConstraint(condition=models.Q(('registration_id_list__len__gt',0),('domain_list__len__gt',0),_connector='OR'),name='registration_id_or_domain',violation_error_message='An organization must have at least a registration ID or a domain.')],
('id',models.UUIDField(default=uuid.uuid4,editable=False,help_text='primary key for the record as UUID',primary_key=True,serialize=False,verbose_name='id')),
('created_at',models.DateTimeField(auto_now_add=True,help_text='date and time at which a record was created',verbose_name='created at')),
('updated_at',models.DateTimeField(auto_now=True,help_text='date and time at which a record was last updated',verbose_name='updated at')),
'constraints':[models.UniqueConstraint(fields=('user','organization'),name='unique_organization_user',violation_error_message='This user is already in this organization.')],
('id',models.UUIDField(default=uuid.uuid4,editable=False,help_text='primary key for the record as UUID',primary_key=True,serialize=False,verbose_name='id')),
('created_at',models.DateTimeField(auto_now_add=True,help_text='date and time at which a record was created',verbose_name='created at')),
('updated_at',models.DateTimeField(auto_now=True,help_text='date and time at which a record was last updated',verbose_name='updated at')),
constraint=models.CheckConstraint(condition=models.Q(('override__isnull',False),('owner__isnull',True),_negated=True),name='override_owner_constraint',violation_error_message='A contact overriding a base contact must be owned.'),
constraint=models.CheckConstraint(condition=models.Q(('user__isnull',True),models.Q(('owner',models.F('user')),('owner__isnull',False)),_connector='OR'),name='profile_contact_owner_constraint',violation_error_message='Users can only declare as profile a contact they own.'),
field=models.JSONField(blank=True,default=dict,help_text='A JSON object containing the organization metadata',verbose_name='metadata'),
),
]
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
mjeammet