Compare commits

..

173 Commits

Author SHA1 Message Date
Anthony LC 10c53d2a4e 🔖(minor) initial release to 0.1.0
Added
- Coming Soon page
- Impress, project to manage your documents easily
  and collaboratively
2024-05-24 10:49:09 +02:00
Anthony LC 5c9e4ab3e6 🏷️(backend) accept string as saved document
Saved documents has to be a string now.
Before it has to be a json object.
2024-05-24 10:27:34 +02:00
Anthony LC 7aeea18202 🗃️(frontend) add initial pad to webrtcserver
It is the webrtc server that will be responsible
for managing the data of the BlockNote Editor.

So to restore the data of a saved pads, we have to
add the initial pad to the room of the webrtc
server, it will then update the data of BlockNote
Editor.
The webrtc server accept Y.Doc, so we
convert our base64 data from Minio to make it
compatible with the server.

By doing so, we avoid the problem of
data lost when multiple users are connecting
and one user already updated the data of the
BlockNote Editor before saving it.
2024-05-24 10:27:34 +02:00
Anthony LC 64d508c260 (frontend) pad persistance
This commit manage the persistance of the pad.
We save the pad in different ways:
- when the user close the tab or the browser
- when the user leave the page (go to another pad
  by example)
- every 1 minute
----
- We save the pad only if the pad has been modified.
- Pads are collaborative, to not save multiple
  times the same pad, we save the pad only if
  the user is the last to have modified the pad.
----
Because of the collaborative aspect of the pads,
the best way to store our pad is to save the
Y.Doc, to do so the recommended way is to convert
the Y.Doc to a Uint8Array and then to a string
(base64). Our pad are saved as a string in a
object in a Minio bucket.
2024-05-24 10:27:34 +02:00
Anthony LC d75649d18a 🏷️(frontend) adapt some types
Adapt some pad types.
2024-05-24 10:27:34 +02:00
Anthony LC a851720441 (frontend) create useUpdatePad react query hook
Create a react query hook to update a pad.
PATCH /documents/:id
2024-05-24 10:27:34 +02:00
Anthony LC 29f13f5f4b 🚨(frontend) stop throwing errors on id pages
We stopped throwing errors on id pages, often
Next.js just need a rerender to get the id
from the router correctly.
2024-05-24 10:27:34 +02:00
Samuel Paccoud - DINUM 515b686795 (models/api) allow inviting external users to a document by their email
We want to be able to share a document with a person even if this person
does not have an account in impress yet.

This code is ported from https://github.com/numerique-gouv/people.
2024-05-24 08:20:28 +02:00
Samuel Paccoud - DINUM 125284456f 🐛(docker-compose) run node container as logged-in user
Running the container as root will install npm modules with root
access rights and generate issues when running the application with
the logged in user as we do it with Docker Compose.
2024-05-24 08:20:28 +02:00
Anthony LC b7b3c6c058 🧑‍💻(backend) add document in django admin
Add document in django admin to be able
to manage the document in the admin panel.
2024-05-23 10:44:52 +02:00
Anthony LC 644920e732 (frontend) coming soon
Add a coming soon page bind to the
docs.numerique.gouv.fr domain.
2024-05-23 10:44:52 +02:00
Anthony LC dc319578b6 👷(helm) production configuration
This PR adds the production configuration
for the helm chart.
2024-05-23 10:44:52 +02:00
Lebaud Antoine abf08fe3b3 🔧(helm) setup logout flow from Agent Connect
Add the relevant environment configurations to make sure the backend
in dev and staging environments log out the user from Agent Connect.
2024-05-22 16:08:41 +02:00
Lebaud Antoine 76e50ac02f (backend) support Agent Connect Logout flow
The default Logout view provided by Mozilla Django OIDC is not suitable
for the Agent Connect Logout flow.

Previously, when a user was logging-out, only its Django session was ended.
However, its session in the OIDC provider was still active.

Agent Connect implements a 'session/end' endpoint, that allows services to
end user session when they logout.

Agent Connect logout triggers cannot work with the default views implemented
by the dependency Mozilla Django OIDC. In their implementation, they decided
to end Django Session before redirecting to the OIDC provider.

The Django session needs to be retained during the logout process.

An OIDC state is saved to the request session, pass to Agent Connect Logout
endpoint, and verified when the backend receives the Logout callback from Agent
Connect. It seems to follow OIDC specifications.

If for any reason, the Logout flow cannot be initiated with Agent Connect,
(missing ID token in cache, unauthenticated user, etc), the user is redirected
to the final URL, without interacting with Agent Connect.
2024-05-22 16:08:41 +02:00
Lebaud Antoine 5f7e3e620a 🚚(backend) create a dedicated authentication package
Prepare adding advanced authentication features. Create a dedicated
authentication Python package within the core app.

This code organization will be more extensible.
2024-05-22 16:08:41 +02:00
Anthony LC 004a4edfe7 (frontend) add logout button
Rework the header based on latest Johann's design, which introduced a
dropdown menu to manage user account.

In this menu, you can find a logout button, which ends up the backend
session by calling the logout endpoint. Please that automatic redirection
when receiving the backend response were disabled. We handle it in our
custom hook, which reload the page.

Has the session cookie have been cleared, on reloading the page, a new
loggin flow is initiated, and the user is redirected to the OIDC provider.
2024-05-22 16:08:41 +02:00
Anthony LC d221ebe4f9 💚(CI) increase timeout e2e job
It can happen that the e2e job takes longer
than 10 minutes to finish, we increase the
timeout to 15 minutes to avoid the job to fail.
2024-05-14 16:30:49 +02:00
Anthony LC 9903b43b72 ♻️(app-desk) change header html structure
Change the html structure of the header
component to be more aligned.
Remove dummy profil picture.
2024-05-14 16:12:36 +02:00
Anthony LC d23426679a 🐛(app-desk) fix fetchPriority warning
The upgrade to react@18.3.1 has a compatibility
issue with next@14.2.3. It creates a error warning
about the fetchPriority prop. This commit fixes the
issue by downgrading react to 18.2.0 as it was
before the last upgrade.
2024-05-14 16:12:36 +02:00
Anthony LC e2e34a011d (app-impress) add La Gaufre
Add La Gaufre.
It is a button to access easily to the
La Suite products.
2024-05-14 16:12:36 +02:00
Anthony LC c1dd34512b 🚸(app-impress) footer legales pages
- Add legales pages.
- Add links to the footer for the new pages.

Legales pages are based from
https://lasuite.numerique.gouv.fr/
2024-05-14 11:57:19 +02:00
Anthony LC 88d6d8977c (app-impress) add footer
Add footer to the impress app.
2024-05-14 11:57:19 +02:00
Anthony LC 4ffcf164d8 💄(app-impress) lighter border pad editor
Make border pad editor a bit lighter.
2024-05-14 11:04:45 +02:00
Anthony LC c25814b060 ♻️(app-impress) add transition props Box component
Add transition props to Box component.
2024-05-14 11:04:45 +02:00
Anthony LC f88d625622 💄(app-impress) move title document to the top
Move teh title document to the top of the pad editor.
We let the white card only for the pad content.
2024-05-14 11:04:45 +02:00
Anthony LC a26ee3b868 💬(app-impress) replace recent with documents
Replace Recent literal by Documents.
2024-05-14 11:04:45 +02:00
Anthony LC c0d919ae52 💄(app-impress) re-design icon close panel
We deleted the menu, so we have to retouch
a bit the icon close panel to fit the new design.
We change the icon to have something more
intuitive by using material icons.
2024-05-14 11:04:45 +02:00
Anthony LC 944385c83f 🔥(app-impress) remove menu
For the moment we only have 1 menu item,
so we can remove the menu.
2024-05-14 11:04:45 +02:00
Anthony LC ccb6d4cb91 (app-impress) add @blocknote/mantine
The new version of blocknote dissociates
the ux from the @blocknote/react package.
It is now in a separate package @blocknote/mantine.
We add this package to the impress app.
We adapt the toolbar to fit the new package.
2024-05-13 17:21:58 +02:00
renovate[bot] 5853adb8eb ⬆️(dependencies) update js dependencies 2024-05-13 17:21:58 +02:00
Anthony LC 70994de914 💚(ci) improve secrets for createsuperuser
Avoid secrets to be visible from running deployments.
2024-05-13 12:12:52 +02:00
Anthony LC df0ff61faf 👷(CI) configure Minio with the CI
Add the minio container to the CI on the
job test-back. Adapt the conf to have
Minio working with the backend.
2024-05-13 12:12:52 +02:00
Anthony LC b94f08d275 🧑‍💻(helm) add minio in staging environment
Add minio in staging environment.
Set the env var for the backend.
2024-05-13 12:12:52 +02:00
Anthony LC a764e5989c 🧑‍💻(helm) add minio in dev environment
Add minio in dev environment.
Set the env var for the backend.
2024-05-13 12:12:52 +02:00
Samuel Paccoud - DINUM 130e7a8c99 (documents) allow retrieving versions (list and detail)
Versions are retrieved directly from object storage and served on API
endpoints. We make sure a user who is given access to a document will
only see versions that were created after s.he gained access.
2024-05-13 12:12:52 +02:00
Samuel Paccoud - DINUM 8e262da8f5 (documents) add content field as an S3 object
The content field is a writable property on the model which is persisted
in object storage. We take advantage of the versioning, robustness and
scalability of S3.
2024-05-13 12:12:52 +02:00
Samuel Paccoud - DINUM 4956beb040 (docker-compose) add minio for object storage
We will need object storage to store our document content which comes
as a big json object. Storing it in object storage will offer us
versioning, robustness and scalability.
2024-05-13 12:12:52 +02:00
Samuel Paccoud - DINUM 397b9efbce (models/api) add document model and API
We do this by making copies of existing Template and TemplateAccess
models and API. A little refactoring is done to try to limit duplicate
code.
2024-05-13 12:12:52 +02:00
Jacques ROUSSEL 7c6e48975d 💚(ci) improve local deployment with tilt
Change the way we start django app in order to be abble to use django
admin on local deployment
2024-05-13 11:15:43 +02:00
Anthony LC f0a4de361e 👷(helm) command createsuperuser
We sometimes need a superuser in the Django
application, to fix eventual db issues.
This commit adds a superuser on the pods.
2024-05-13 11:15:43 +02:00
Anthony LC f0b3090a73 ♻️(frontend) frontend environment free
Until now, the front had to know at build time
the url of the backend and the webrtc server
to be able to communicate with them.
It is not optimal because it means that we need
multiple docker image (1 per environment) to have
the app working, it is not very flexible.

This commit will make the frontend "environment free"
by determining these urls at runtime.
2024-05-13 11:04:06 +02:00
Anthony LC 88109623ba ♻️(app-desk) replace classname spacings
Replace the classname spacings with the new
spacing system based on props.
2024-05-03 15:47:36 +02:00
Anthony LC d8e7f29072 🎨(app-desk) add margin and padding to Box
Add margin and padding system to Box component.
It proposes the autocompletion.
It is bind with the Cunninghams spacing system.
2024-05-03 15:47:36 +02:00
Jacques ROUSSEL 0c2d097d8d 💚(ci) improve secrets for k8s deployment
Avoid secrets to be visible from running deployments
2024-05-02 16:06:58 +02:00
Anthony LC 5fbb5106a9 🔥(backend) remove code_editor field
code_editor field was removed from the backend.
This filed was used to store the code editor to
generate templates.
2024-05-02 15:24:51 +02:00
Anthony LC c7a5b5f3fb 🤡(backend-demo) remove code_editor from demo
We removed the template code_editor from app, so
we redapt the demo to fit the old template.
The field code_editor is not needed anymore.
2024-05-02 15:24:51 +02:00
Anthony LC d967896874 🔥(app-impress) remove templates
We will generate the templates internally.
We remove the template editor.
2024-05-02 15:24:51 +02:00
Anthony LC b98cc01592 🚚(app-impress) move templates api to pad-tools
Before removing the templates feature, we move the
necessary api to the pad feature.
2024-05-02 15:24:51 +02:00
Anthony LC d9fa75b764 ⬇️(frontend) react 18.3.1 -> 18.2.0
Downgrade react to 18.2.0.
It seems to have a compatibility issue
with @openfun/cunningham-react.
2024-04-29 15:21:25 +02:00
Anthony LC 33b39ce315 ⬇️(eslint-config-people) eslint 9.0.0 -> 8.57.0
Downgrade eslint to 8.57.0.
9.0.0 has breaking changes, the adoption
is still very low, better to wait.
Add it in the renovate.json file.
2024-04-29 15:21:25 +02:00
renovate[bot] f5ddce6b47 ⬆️(dependencies) update js dependencies 2024-04-29 15:21:25 +02:00
renovate[bot] 8d48ded46b ⬆️(dependencies) update gunicorn to v22 [SECURITY] 2024-04-29 10:53:14 +02:00
Jacques ROUSSEL f9bec19c7d 🐛(ci) improve docker-hub
Avoid to notify argocd for nothing
2024-04-29 10:29:56 +02:00
Anthony LC 89aa048fc1 🐛(app-desk) force redirect on resources/[id]
When we use the static mode, Next.js want that we
build our resource pages at build time. But we can't
do that because users can create a resource at runtime.

So we redirect thanks to ngnix when we see that
a resource page is not found.
2024-04-19 13:38:28 +02:00
Anthony LC b2cc789665 ️(docker) separate deps y-webrtc-signaling
We separate the install part of the signaling server
from the frontend-dep stage. It will improve
the build time of both of them.
2024-04-19 13:38:28 +02:00
Anthony LC 9489264555 🔥(eslint) remove uneeded yarn.lock file
The yarn.lock doesn't seem necessary for this
package, so we're removing it.
Update the frontend yarn.lock
2024-04-19 13:38:28 +02:00
Anthony LC e1b94e3443 🌐(app-impress) translate impress in french
Add french translation to impress.
2024-04-19 13:38:28 +02:00
Anthony LC 90c8338a59 🛂(app-impress) allow writing permission on editor
allow writing permission on editor when we are not
the owner to test the POC.
2024-04-19 13:38:28 +02:00
Anthony LC 895aaff2d3 📝(readme) steps to start frontend
Define the steps to start the frontend in
development mode.
2024-04-19 13:38:28 +02:00
Anthony LC bbafce64e7 🐛(app-impress) refresh when multiple Blocknote
When multiple blocknote were created, the editor
didn't display the content of the selected blocknote
but the content of the last blocknote with a new
provider.
We were actually storing the editor, so if the
editor is stored already, we serve it instead of
creating a new one.
2024-04-19 13:38:28 +02:00
Anthony LC e901103c8b 🔨(Tilt) add y-webrtc-signaling
Add docker_build config to build y-webrtc-signaling.
2024-04-19 12:41:06 +02:00
Anthony LC f3f5f190a5 🚀(helm) webrtc helm chart
Make the webrtc helm chart.
The webrtc server is a simple websocket server
that forwards messages between clients.
The server uses the port 443 and the protocol wss.
local_url: wss://impress.127.0.0.1.nip.io/ws
2024-04-19 12:41:06 +02:00
Anthony LC 2dc2341052 🚚(docker) move frontend docker config
Move frontend docker config to
src/frontend/Dockerfile.
2024-04-19 12:41:06 +02:00
Anthony LC c2fb55506d 🧱(docker) dockerhub lasuite/impress-y-webrtc-signaling
Add the signaling server to dockerhub.
We create a special image for the signaling server,
it will be easier for scaling and maintenance.
2024-04-19 12:41:06 +02:00
Anthony LC 428cd74366 🚚(k8s) move tilt config to bin
Move tilt config to bin directory
to keep the root directory clean.
Add makefile cmd about k8s.
2024-04-19 12:41:06 +02:00
Anthony LC 0575c0f086 (app-impress) can duplicate template
Template with mode "is_public" are not editable, except
for the owner. We give the possibility to duplicate a
template, so that the user can edit the new one.
2024-04-18 18:13:59 +02:00
Anthony LC 35d500c08b 👔(backend) display css and code in template endpoint
We would like to duplicate a template from the frontend.
For that we need to access the css and code of the template.
So we add the css and code to the template endpoint.
2024-04-18 18:13:59 +02:00
Anthony LC bf83bf99b3 ♻️(app-impress) create TemplateTools component
A component that will be used to display
the tools for the template editor.
2024-04-18 18:13:59 +02:00
Anthony LC 964efe4c2f 💄(app-impress) simplify the template builder
Remove some features from the template builder
that are not necessary for the MVP.
2024-04-18 18:13:59 +02:00
Anthony LC 75801bf95f 🤡(backend) add template editor json in demo
For the demo to be compatible with the new template
editor, we need to add the json file that setup
correctly the editor.
We are able to read and use it, but not edit it,
because the demo is in is_public mode.
2024-04-18 18:13:59 +02:00
Anthony LC 4de79c2388 💫(ap-impress) improve transition
Improve transition on Box component.
2024-04-18 15:49:42 +02:00
Anthony LC f61047dd6a 💄(app-impress) keep highlighting menu when sub menu
When sub menu was open, the parent menu was not
highlighted.
This commit fixes this issue.
2024-04-18 15:49:42 +02:00
Anthony LC d3b4a576c8 🗑️(app-impress) clean the menu
- replace pad and template icon
- remove unused icons
- remove unused pages
- remove menu items
2024-04-18 15:49:42 +02:00
Anthony LC 7107caf50b 🚸(app-impress) warning when resource not editable
When a resource is public, the users can see it
but not edit it. This commit adds a warning message
to the user to tell them that the resource
is not editable.
2024-04-18 14:44:12 +02:00
Anthony LC 82463be8ec 🐛(app-impress) fix all the resources were public
All the resources were public.
We now update correctly depend the user's choice.
We add a test.
2024-04-18 14:44:12 +02:00
Anthony LC 64cc2a9f2b 🐛(app-impress) fix bug rerendering when empty data
If the data is empty, the editor kept the previous
data and didn't rerender correctly the current editor.
This commit fixes this issue.
2024-04-18 12:48:15 +02:00
Anthony LC b687ad6974 🚚(app-impress) rename PrintToPDFButton to PDFButton
- rename PrintToPDFButton to PDFButton
- rename button "Print the pad" to "Generate PDF"
2024-04-18 12:48:15 +02:00
Anthony LC 5d6ae870fa (app-impress) select template on pad
We add a template select component to the pad toolbox.
This component is a dropdown that allows the user
to select a template for the pad to add in a PDF.
2024-04-18 12:48:15 +02:00
Anthony LC ef2ebf596a 👔(app-impress) add is_public switch button
Add a switch button to the create pad form to
allow the user to set the pad as public or private.
2024-04-17 17:45:03 +02:00
Anthony LC 7463735a38 👔(backend) is_public in document and template serializer
Add is_public field to document and template serializer.
2024-04-17 17:45:03 +02:00
Anthony LC d9cdeb5c68 🚸(app-impress) add warning when body tag not in template
In order to work with the pad, the body tag
must be in the template.
This commit adds a warning to the template editor
when the body tag is not in the template.
2024-04-17 17:15:53 +02:00
Anthony LC 5d25aed765 💫(app-desk) add effect prop on Box component
Add the effect prop on Box component.
2 effects are available: 'show' and 'hide'.
2024-04-17 17:15:53 +02:00
Anthony LC a8248f10d2 👔(app-impress) button to save html and css of the template
We created a button to save the template's html and css.
With the template saved we will be able to use it
in the near future with our pads.
2024-04-17 17:15:53 +02:00
Anthony LC faeb8d137b 👔(backend) add css and code in template serializer
We want to be able to update the css and code of
a template from the update endpoint.
This commit adds the css and code fields to the
TemplateSerializer.
2024-04-17 17:15:53 +02:00
Anthony LC 5074b271ad 🐛(backend) add partial_update action to the abilities
partial_update action was missing for the permission.
So when we were patching the template,
it was not working.
This commit adds the partial_update action
to the abilities.
2024-04-17 17:15:53 +02:00
Anthony LC e13ecdd38d 👔(app-impress) save the templates editor automatically
On almost each change in the code editor,
the template is saved automatically.
It will restore as well the editor when we arrive
on the template editor page.
2024-04-17 17:15:53 +02:00
Anthony LC 3aaa3e179d 👔(backend) add code_editor column on Template
To save the template code editor content,
we need to add a new column on the Template model.
It is a JSONField that will store the code editor content.

We could in the future make an implementation to
save the code editor content in Minio.
2024-04-17 17:15:53 +02:00
Anthony LC 7922c702cd 🚀(app-impress) add template builder editor
We add the template builder editor to
the template feature.
It will be used to create and edit templates.
2024-04-17 17:15:53 +02:00
Anthony LC 8d2a78cf8d (app-impress) create Templates feature
We created the features templates.
It will be used to create, edit and
delete templates.
It is a naive copy of pads.
2024-04-17 17:15:53 +02:00
Anthony LC 7fb8a62e63 🔥(app-impress) remove La Gauffre button
The web host of the La Gauffre app is down,
La Gauffre was not rendered correctly.
La Gauffre button is removed from the header
of the Impress app, in waiting for the
web host to be up again.
2024-04-16 15:01:05 +02:00
Anthony LC 45bf99178b (app-impress) button editor convert from markdown
Some people prefer to write in markdown,
or had already written stuff in markdown,
so this commit adds a button to the editor t
hat will convert the markdown to the editor's format.
2024-04-16 15:01:05 +02:00
Anthony LC 8305cfcf99 🏷️(app-impress) add body type on generate-document endpoint
Adapt the generate-document endpoint
to serve body as html as well.
We add a type to the body parameter to
specify the type of the body.
2024-04-16 13:03:25 +02:00
Anthony LC 1df7c43dd3 🏷️(backend) add body type on generate-document endpoint
We were converting from markdown to html, but the
frontend can provide the body in html format, so
wa can avoid the conversion.

Solution:
Add body type on generate-document endpoint
to allow to choose between markdown and html.
2024-04-16 13:03:25 +02:00
Anthony LC f9705c6ce9 ♻️(backend) api resources list ordering
Give the possibility to order the resources
list by creation date (documents / templates).
By default the list is ordered by
creation date descending.
2024-04-16 11:37:11 +02:00
Anthony LC db9c9ecfe3 🤡(app-impress) remove mock pad endpoints
The backend was updated to include the
pad endpoints.
The mock pad endpoints were removed
in favor of the real ones.
We adapted the types as well to match
the real ones.
2024-04-16 11:37:11 +02:00
Samuel Paccoud - DINUM 3e0739cd0a (models/api) add document model and API
We do this by making copies of existing Template and TemplateAccess
models and API. A little refactoring is done to try to limit duplicate
code.
2024-04-16 11:37:11 +02:00
Anthony LC 0024cc5814 🚨(backend) fix linting issue
Fix linting issue in the backend codebase.
2024-04-12 16:24:23 +02:00
Anthony LC 5587cf69dd 🔧(helm) add command create_demo
We need to get dummy data in the system
for the moment, it will insert a template
for the PDF endpoint.
We need to run this command after the
migration is made.
2024-04-12 16:24:23 +02:00
Anthony LC 68c130bd84 🔧(backend) activate https on oidc redirection
mozilla-django-oidc didn't add the https://
prefix to the redirect_uri.
We set the option SECURE_PROXY_SSL_HEADER to
('HTTP_X_FORWARDED_PROTO', 'https') in the
settings.py file to force the https prefix.
2024-04-12 16:24:23 +02:00
Anthony LC 22e79e7583 🔧(backend) activate container liveness probes
Enabled Dockerflow Django app by activating liveness probes. The previously
unavailable routes such as `__heartbeat__` and `__lbheartbeat__` are now
accessible. New endpoints include:
* GET /__version__
* GET /__heartbeat__
* GET /__lbheartbeat__
2024-04-12 16:24:23 +02:00
Anthony LC 056bad08a5 🔧(backend) configure RedisCache in production settings
In development, sessions are saved in local memory. It's working well,
however it doesn't adapt to a kubernetized setup. Several pods need
to access the current sessions, which need to be stored in a single
source of truth.

With a local memory cache, pods cannot read session saved in other pods.
We end up returning 401 errors, because we cannot authenticate the user.

I preferred setting up a proper cache than storing sessions in database,
because in the long run it would be a performance bottleneck. Cache will
decrease data access latency when reading current sessions.

I added a Redis cache backend to the production settings. Sessions would
be persisted to Redis. In K8s, a Redis operator will make sure the cached
data are not lost.

Two new dependencies were added, redis and django-redis.

I followed the installation guide of django-redis dependency. These
setting were tested deploying the app to a local K8s cluster.
2024-04-12 16:24:23 +02:00
Anthony LC 2f4dd4ee7a 🔐(impress) update impress oidc secrets
Update oidc secrets in dev and staging environments
to match agent connect with impress settings.
2024-04-12 16:24:23 +02:00
Anthony LC 51babc07af 🚚(project) rename desk occurences to impress
Rename desk occurences to impress.
2024-04-12 16:24:23 +02:00
Anthony LC dbb592bc9a 🔨(CI) add Tilt
Tilt is a tool for local Kubernetes development.
It makes it easy to see your changes as you
make them, and it rebuilds and redeploys
your app as you change it.
It is a great tool for local development
and testing of Kubernetes applications.
2024-04-12 16:24:23 +02:00
Anthony LC 0dc1be4cc7 (e2e) test the pdf export of the impress app
Add a test to check the pdf export of the
impress app.
It intercept the download of the PDF and
read its content.
We can assert that the template is correctly
rendered, and that the pad text is present as
well.
2024-04-11 09:21:35 +02:00
Anthony LC a772d8fc29 🤡(backend) demo template
Create the create_demo command.
It creates a demo template for the backend.
We add it in the makefile bootstrap.

We force the template id, this id is the same as
used in the frontend for the moment.
When the template feature will be created in
the frontend side we will not have to force
it anymore.
2024-04-11 09:21:35 +02:00
Anthony LC c6c0c1355d 🙈(project) add .devcontainer in gitignore
.devcontainer is used to create virtual environment
for development. It is not necessary to be
included in the git repository.
2024-04-11 09:21:35 +02:00
Anthony LC cc64b8c8b5 🐛(backend) format html when generate document
When we converted from markdown to html,
we didn't format the html, so it was displayed
like a string in our pdf.
We now format_html the html before we generate
the pdf.
2024-04-11 09:21:35 +02:00
Anthony LC 2665b707db (app-impress) print to pdf
Feature to print the pad to pdf.

It display a button in the pad toolbar,
when clicked it will convert the pad to markdown,
send the markdown with a template to the backend
to convert it to pdf and then download the pdf.
2024-04-11 09:21:35 +02:00
Anthony LC e28a4294fa (app-desk) add useCreatePdfFromMarkdown
Add the hook useCreatePdfFromMarkdown,
it will be used to generate a pdf from a
a template and a markdown.
2024-04-11 09:21:35 +02:00
Anthony LC 9ca63baeeb 🔨(backend) add local CSRF_TRUSTED_ORIGINS
To allow the frontend to make requests to the backend,
we need to add the frontend's origin
to the CSRF_TRUSTED_ORIGINS setting.
2024-04-11 09:21:35 +02:00
Anthony LC a5a8a64350 ♻️(app-impress) refacto pad store to add the editor
To spread the data editor to different feature
of our app, we add the editor in the pad store.
It will give an easy access to the editor data.
2024-04-11 09:21:35 +02:00
Anthony LC 272b18ae02 (y-webrtc-signaling) test signaling server
Create a e2e testcase to test the signaling server.
2024-04-09 16:56:21 +02:00
Anthony LC 37367f2d22 🎨(app-impress) different color per user
We generate a color for each user.
This color is used to highlight the
user's cursor and selection.
2024-04-09 16:56:21 +02:00
Anthony LC 2070a159e1 ️(app-impress) create pad store
Better to store the WebrtcProvider, we will
reuse the provider if one is already created.
2024-04-09 16:56:21 +02:00
Anthony LC a4e8bbd26e 🚚(app-impress) rename usePadStore to usePadPanelStore
This commit renames the `usePadStore`
to `usePadPanelStore` to better
reflect the purpose of the store.
2024-04-09 16:56:21 +02:00
Anthony LC 75bb4b36d4 🔨(y-webrtc-signaling) dockerize y-webrtc-signaling
Dockerize y-webrtc-signaling for futur deployment.
Add a docker-compose file to run easily
the signaling server.
2024-04-09 16:56:21 +02:00
Anthony LC f4e4cbaa56 (y-webrtc-signaling) create signaling server
Create signaling server for WebRTC.
It will be used for the collaborative editor, client
will connect to this server to see their
changes in real-time.
2024-04-09 16:56:21 +02:00
Anthony LC 6b567a0f69 🔧(sops) add secrets
Add secrets to the repository using sops.
2024-04-05 12:42:33 +02:00
Anthony LC 007a9ae4f4 💚(docker) adapt dockerfile with people updates
Adapt the dockerfile to include the new
people updates, plus adapt to impress project.
2024-04-05 12:42:33 +02:00
Anthony LC ca0571cb21 (CI) copy docker-hub github-actions from people
Copy of docker-hub github-actions
from people repository to impress repository.
Replace occurence of people by impress.
2024-04-05 12:42:33 +02:00
Anthony LC 560dbf1c2e 🔧(helm) replace occurences of desk and people
Replace naively all occurences of
desk and people with impress.
2024-04-05 12:42:33 +02:00
Anthony LC 17729e231b (CI) copy helm config from people
Naive copy of helm config from people repository
to impress repository.
2024-04-05 12:42:33 +02:00
Anthony LC 409691d466 🚨(app-desk) improve linter
The linter was passing near the ts errors in
the tests, we improve the linter by adding
a ts checkup.
2024-04-04 17:40:27 +02:00
Anthony LC 603ece466d 🤡(app-impress) mock BlockNoteEditor
BlockNoteEditor uses many libraries not compatible
with Jest.
This commit mocks the BlockNoteEditor
component to allow Jest to run without errors.
2024-04-04 17:40:27 +02:00
Anthony LC 79831154be (app-impress) add basic blocknotes editor
Create PadEditor, the parent component for the editors.
We integrate the BlockNoteEditor into the PadEditor.
2024-04-04 17:40:27 +02:00
Anthony LC da273d6b2b (app-markdown) add necessary library for pad editor
Add the necessary library for the pad editor:
- blocknote editor (Notion like editor)
- yjs and y-webrtc for real-time collaboration
2024-04-04 17:40:27 +02:00
Anthony LC 5ce77c2138 🤡(app-desk) mock pad and pads panel
In order to access to the pad page we mocked
the api calls of the pad and pads panel.
2024-04-04 17:40:27 +02:00
renovate[bot] b7663cca2d ⬆️(dependencies) update js dependencies 2024-04-04 12:38:11 +02:00
Anthony LC 80335ffd15 🔥(markdownPDF) remove versionning app mardownPDF
Remove the app mardownPDF from the versionning,
this app is not used anymore, we don't want
to maintain it.
2024-04-04 12:05:54 +02:00
Anthony LC 6de828c049 🚨(frontend) fix CI probs
The CI highlighted some issues with the tests.
This commit fixes the issues.
2024-04-03 17:36:54 +02:00
Anthony LC 352d4f6d27 🔥(app-impress) remove teams pages
Remove teams pages from the next router.
2024-04-03 17:36:54 +02:00
Anthony LC 3e6f20aedd ️(frontend) clean yarn.lock
The yarn.lock file get full of garbage and old
dependencies after a while. This commit cleans it.
2024-04-03 17:36:54 +02:00
Anthony LC bf19c50146 💚(CI) add frontend jobs
Add the frontend jobs to the CI pipeline.
2024-04-03 17:36:54 +02:00
Anthony LC 1d3f1f793e (e2e) adapt tests to impress app
Adapt e2e test to works with Impress.
2024-04-03 17:36:54 +02:00
renovate[bot] cad206cccf ⬆️(dependencies) update django to v5.0.3 [SECURITY] 2024-04-03 13:10:09 +02:00
Anthony LC 3cd1852b93 🔧(sops) add secrets
Add secrets to the repository using sops.
2024-04-03 13:07:05 +02:00
Anthony LC d19c7f7fd8 👷(project) rename i18n-back to i18n-crowdin
Rename i18n-back to i18n-crowdin.
Change the job to use the 'age' env.
2024-04-03 13:07:05 +02:00
Anthony LC 8afc003004 🐛(docker) switch CMD form from Shell to Exec
`backend-development` and `backend-production` CMD syntaxes were
using a Shell Form. Shell form prevented Unix signals from reaching
our container correctly, such as SIGTERM. Also, the shell process
ends up being the PID 1, instead of our Python scripts.

Docker recommends to use the exec form whenever possible.
2024-04-03 13:07:05 +02:00
Anthony LC 23797f2953 🔧(project) add frontend cmd to Makefile
add frontend cmd to Makefile.
2024-04-03 13:07:05 +02:00
Anthony LC 78bccafb93 🐛(backend) generate po file
Generate the PO file.
It will create the locale folder, and so
init correctly LOCALE_PATHS env.
This should fix the CI.
2024-04-03 13:07:05 +02:00
Anthony LC 42bea9940e 🔥(app-impress) remove addMembers feature
addMembers feature was a part of the People
project, we don't need it in the Impress project.
2024-04-03 11:57:36 +02:00
Anthony LC 55e2a2073d 🔥(app-impress) remove Members feature
Members feature was a part of the People
project, we don't need it in the Impress project.
2024-04-03 11:57:36 +02:00
Anthony LC 1f804ad1f9 🔥(app-impress) remove Teams feature
Teams feature was a part of the People
project, we don't need it in the Impress project.
2024-04-03 11:57:36 +02:00
Anthony LC 3ba388bc49 ⬆️(project) update renovate.json config
Update the renovate config to fit with
what we setup in People project.
2024-04-03 11:57:36 +02:00
Anthony LC b517b0bc39 (app-impress) create feature pad
Create a new feature pad for the app impress.
It will be used to display a pad in the app.
2024-04-03 11:57:36 +02:00
Anthony LC 1cb138ecb3 (app-impress) create feature pads-create
Create a new feature pads-create for the app impress.
It will be used to create a pad in the app.
2024-04-03 11:57:36 +02:00
Anthony LC 1dd33706c0 (app-impress) create feature pads-panel
Create a new feature pads-panel for the app impress.
It will be used to display the pads in the app.
2024-04-03 11:57:36 +02:00
Anthony LC 6fe5f3ef72 🚚(app-impress) rename icon-desk.svg to icon-impress.svg
Rename icon-desk.svg to icon-impress.svg.
2024-04-03 11:57:36 +02:00
Anthony LC 1bf391f26f (app-impress) add layouts
Add layouts folder to the app-impress,
it will contain all the layouts for the app.
For the moment:
- MainLayout.tsx
- PadLayout.tsx
2024-04-03 11:57:36 +02:00
Anthony LC dfafd335f4 🌐(i18n) create package i18n
The package i18n will handle the
internationalization of the applications.
It will parse the frontend code and extract
the translations to be send to the crowdin
platform.
2024-04-02 16:39:17 +02:00
Anthony LC 3098b9f4fc 🚀(app-impress) create the e2e app impress
Create the e2e app impress, it will be used
to test the app impress.
2024-04-02 16:39:17 +02:00
Anthony LC 8fda0bc9b9 🚀(app-impress) create the base app impress
Create the base app impress, based on the
people app.
2024-04-02 16:39:17 +02:00
Anthony LC c64b66c797 🚨(frontend) add lib eslint-config-impress
Create the eslint-config-impress package to share
eslint configuration with all impress projects.
2024-04-02 16:39:17 +02:00
Anthony LC c3e354d727 🚚(frontend) move previous app to mardownPDF app
We will start an monorepo archi, we move the previous
app to a new folder called mardownPDF.
2024-04-02 16:39:17 +02:00
Samuel Paccoud - DINUM ab7d9933e0 ♻️(project) rename project from "publish" to "impress"
The repository was renamed to "impress" but the code was still
mentionning "publish".
2024-03-08 08:53:56 +01:00
Samuel Paccoud - DINUM f581eb8abd (models/api) add RBAC on templates linking accesses to a team name
We want to be able to control who can access a template via roles.
I added this feature on the TeamAccess model assuming that the teams
to which a user belongs can be retrieved via a `get_teams` method on
the user model. The idea is that this method will get the teams either
via a call to an external API or directly from the OIDC token upon
user login. This list of teams will probably have to be cached for
each user.
2024-03-07 21:21:18 +01:00
Lebaud Antoine a23118bee4 🚨(backend) fix minor linting issues
Lint backend tests and sources.
2024-03-03 08:29:00 +01:00
Lebaud Antoine 47aeab76a5 (backend) test the authentication class
Add tests on get_or_create method.
2024-03-03 08:29:00 +01:00
Lebaud Antoine 551468470f 🐛(project) run production image locally with docker-compose
The local deployment of the Production image through docker-compose was
failing due to issues in the Django configurations, influenced by Joanie.

The bug stemmed from a dependency on a development-specific package
(drf-spectacular-sidecar) while attempting to run the application in
production mode.

Changes Made:
- Introduced new Django settings for local demo environments.
2024-03-03 08:29:00 +01:00
Lebaud Antoine 36d6735798 ✏️(project) fix minor typos
Found typos and fixed them.
2024-03-03 08:29:00 +01:00
Lebaud Antoine b1892ded17 (backend) drop JWT authentication in API tests
Force login to bypass authorization checks when necessary.

Note: Generating a session cookie through OIDC flow
is not supported while testing our API.
2024-03-03 08:29:00 +01:00
Lebaud Antoine b9eee3e643 🔧(backend) configure Authorization Code authentication
Integrate 'mozilla-django-oidc' dependency, to support
Authorization Code flow, which is required by Agent Connect.

Thus, we provide a secure back channel OIDC flow, and return
to the client only a session cookie.

Done:
- Replace JWT authentication by Session based authentication in DRF
- Update Django settings to make OIDC configurations easily editable
- Add 'mozilla-django-oidc' routes to our router
- Implement a custom Django Authentication class to adapt
'mozilla-django-oidc' to our needs

'mozilla-django-oidc' routes added are:
- /authenticate
- /callback (the redirect_uri called back by the Idp)
- /logout
2024-03-03 08:29:00 +01:00
Lebaud Antoine 23e92d12fb (project) configure Keycloak server to support OIDC
Create a realm 'impress' and configure it to support
the authorization code flow.
2024-03-03 08:29:00 +01:00
Samuel Paccoud - DINUM 0f9327a1de ♻️(backend) refactor post hackathon to a first working version
This project was copied and hacked to make a POC in a 2-day hackathon.
We need to clean and refactor things in order to get a first version
of the product we want.
2024-02-23 18:41:36 +01:00
Samuel Paccoud 0a3e26486e Merge pull request #1 from numerique-gouv/dependabot/npm_and_yarn/src/mail/semver-5.7.2
Bump semver from 5.7.1 to 5.7.2 in /src/mail
2024-02-16 19:15:23 +01:00
Samuel Paccoud ae2ba59c82 Merge pull request #3 from numerique-gouv/dependabot/pip/src/backend/django-5.0.2
Bump django from 5.0 to 5.0.2 in /src/backend
2024-02-16 19:14:44 +01:00
dependabot[bot] 0f0b0f0de1 Bump django from 5.0 to 5.0.2 in /src/backend
Bumps [django](https://github.com/django/django) from 5.0 to 5.0.2.
- [Commits](https://github.com/django/django/compare/5.0...5.0.2)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-07 17:37:18 +00:00
Samuel Paccoud - DINUM 44ff62c08f (backend) parse YAML metadata
YAML header is processed, to fill Django
templates with the right parametrized values.

Used an external dependency, frontmatter.
2024-02-05 22:59:14 +01:00
Lebaud Antoine 312a680b66 (frontend) set up Vite-based frontend project
Chose Vite for static output efficiency, aligning with project needs.

All API interactions are currently unauthenticated. SSO support
planned soon, using ProConnect.

UX is minimalistic, and showcases the core idea.

Components introduced:
* AppProvider
* Select and TextArea Rhf inputs

API hooks introduced:
* useGeneratePDF, generates a PDF, and downloads it in the client.
* useTemplates, fetches available templates to populate Select options.
2024-01-16 16:35:11 +01:00
dependabot[bot] 7eab2b61a5 Bump semver from 5.7.1 to 5.7.2 in /src/mail
Bumps [semver](https://github.com/npm/node-semver) from 5.7.1 to 5.7.2.
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/v5.7.2/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v5.7.1...v5.7.2)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-09 15:30:25 +00:00
Samuel Paccoud - DINUM 62df0524ac (project) first proof of concept printing pdf from markdown
This is a boilerplate inspired from https://github.com/openfun/joanie
2024-01-09 15:30:36 +01:00
Samuel Paccoud - DINUM 2d81979b0a Initialize project 2023-12-10 16:10:06 +01:00
414 changed files with 41236 additions and 544 deletions
+36
View File
@@ -0,0 +1,36 @@
# Python
__pycache__
*.pyc
**/__pycache__
**/*.pyc
venv
.venv
# System-specific files
.DS_Store
**/.DS_Store
# Docker
docker compose.*
env.d
# Docs
docs
*.md
*.log
# Development/test cache & configurations
data
.cache
.circleci
.git
.vscode
.iml
.idea
db.sqlite3
.mypy_cache
.pylint.d
.pytest_cache
# Frontend
node_modules
+6
View File
@@ -0,0 +1,6 @@
<!---
Thanks for filing an issue 😄 ! Before you submit, please read the following:
Check the other issue templates if you are trying to submit a bug report, feature request, or question
Search open/closed issues before submitting since someone might have asked the same thing before!
-->
+28
View File
@@ -0,0 +1,28 @@
---
name: 🐛 Bug Report
about: If something is not working as expected 🤔.
---
## Bug Report
**Problematic behavior**
A clear and concise description of the behavior.
**Expected behavior/code**
A clear and concise description of what you expected to happen (or code).
**Steps to Reproduce**
1. Do this...
2. Then this...
3. And then the bug happens!
**Environment**
- Impress version:
- Platform:
**Possible Solution**
<!--- Only if you have suggestions on a fix for the bug -->
**Additional context/Screenshots**
Add any other context about the problem here. If applicable, add screenshots to help explain.
+23
View File
@@ -0,0 +1,23 @@
---
name: ✨ Feature Request
about: I have a suggestion (and may want to build it 💪)!
---
## Feature Request
**Is your feature request related to a problem or unsupported use case? Please describe.**
A clear and concise description of what the problem is. For example: I need to do some task and I have an issue...
**Describe the solution you'd like**
A clear and concise description of what you want to happen. Add any considered drawbacks.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Discovery, Documentation, Adoption, Migration Strategy**
If you can, explain how users will be able to use this and possibly write out a version the docs (if applicable).
Maybe a screenshot or design?
**Do you want to work on it through a Pull Request?**
<!-- Make sure to coordinate with us before you spend too much time working on an implementation! -->
@@ -0,0 +1,22 @@
---
name: 🤗 Support Question
about: If you have a question 💬, or something was not clear from the docs!
---
<!-- ^ Click "Preview" for a nicer view! ^
We primarily use GitHub as an issue tracker. If however you're encountering an issue not covered in the docs, we may be able to help! -->
---
Please make sure you have read our [main Readme](https://github.com/numerique-gouv/impress).
Also make sure it was not already answered in [an open or close issue](https://github.com/numerique-gouv/impress/issues).
If your question was not covered, and you feel like it should be, fire away! We'd love to improve our docs! 👌
**Topic**
What's the general area of your question: for example, docker setup, database schema, search functionality,...
**Question**
Try to be as specific as possible so we can help you as best we can. Please be patient 🙏
+11
View File
@@ -0,0 +1,11 @@
## Purpose
Description...
## Proposal
Description...
- [] item 1...
- [] item 2...
+141
View File
@@ -0,0 +1,141 @@
name: Docker Hub Workflow
on:
workflow_dispatch:
push:
branches:
- 'main'
tags:
- 'v*'
pull_request:
branches:
- 'main'
env:
DOCKER_USER: 1001:127
jobs:
build-and-push-backend:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/impress-backend
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: .github/workflows/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
-
name: Build and push
uses: docker/build-push-action@v5
with:
context: .
target: backend-production
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-frontend:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/impress-frontend
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: .github/workflows/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
-
name: Build and push
uses: docker/build-push-action@v5
with:
context: .
file: ./src/frontend/Dockerfile
target: frontend-production
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-y-webrtc-signaling:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/impress-y-webrtc-signaling
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: .github/workflows/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "$DOCKER_HUB_PASSWORD" | docker login -u "$DOCKER_HUB_USER" --password-stdin
-
name: Build and push
uses: docker/build-push-action@v5
with:
context: .
file: ./src/frontend/Dockerfile
target: y-webrtc-signaling
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
notify-argocd:
needs:
- build-and-push-frontend
- build-and-push-backend
runs-on: ubuntu-latest
if: |
github.event_name != 'pull_request'
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: .github/workflows/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/'$GITHUB_REPOSITORY'"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_WEBHOOK_URL
+157
View File
@@ -0,0 +1,157 @@
name: impress Workflow
on:
push:
branches:
- main
tags:
- 'v*'
pull_request:
branches:
- '*'
jobs:
install-front:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18.x'
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Install dependencies
if: steps.front-node_modules.outputs.cache-hit != 'true'
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Cache install frontend
if: steps.front-node_modules.outputs.cache-hit != 'true'
uses: actions/cache@v4
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
build-front:
runs-on: ubuntu-latest
needs: install-front
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Build CI App
run: cd src/frontend/ && yarn ci:build
- name: Cache build frontend
uses: actions/cache@v4
with:
path: src/frontend/apps/impress/out/
key: build-front-${{ github.run_id }}
test-front:
runs-on: ubuntu-latest
needs: install-front
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Test App
run: cd src/frontend/ && yarn app:test
lint-front:
runs-on: ubuntu-latest
needs: install-front
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Check linting
run: cd src/frontend/ && yarn lint
test-e2e:
runs-on: ubuntu-latest
needs: build-front
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set services env variables
run: |
make data/media
make create-env-files
cat env.d/development/common.e2e.dist >> env.d/development/common
- name: Restore the frontend cache
uses: actions/cache@v4
id: front-node_modules
with:
path: 'src/frontend/**/node_modules'
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
- name: Restore the build cache
uses: actions/cache@v4
id: cache-build
with:
path: src/frontend/apps/impress/out/
key: build-front-${{ github.run_id }}
- name: Build and Start Docker Servers
env:
DOCKER_BUILDKIT: 1
COMPOSE_DOCKER_CLI_BUILD: 1
run: |
docker-compose build --pull --build-arg BUILDKIT_INLINE_CACHE=1
make run
- name: Apply DRF migrations
run: |
make migrate
- name: Add dummy data
run: |
make demo FLUSH_ARGS='--no-input'
- name: Install Playwright Browsers
run: cd src/frontend/apps/e2e && yarn install
- name: Run e2e tests
run: cd src/frontend/ && yarn e2e:test
- uses: actions/upload-artifact@v3
if: always()
with:
name: playwright-report
path: src/frontend/apps/e2e/report/
retention-days: 7
+242
View File
@@ -0,0 +1,242 @@
name: impress Workflow
on:
push:
branches:
- main
tags:
- 'v*'
pull_request:
branches:
- '*'
jobs:
lint-git:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' # Makes sense only for pull requests
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: show
run: git log
- name: Enforce absence of print statements in code
run: |
! git diff origin/${{ github.event.pull_request.base.ref }}..HEAD -- . ':(exclude)**/impress.yml' | grep "print("
- name: Check absence of fixup commits
run: |
! git log | grep 'fixup!'
- name: Install gitlint
run: pip install --user requests gitlint
- name: Lint commit messages added to main
run: ~/.local/bin/gitlint --commits origin/${{ github.event.pull_request.base.ref }}..HEAD
check-changelog:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Check that the CHANGELOG has been modified in the current branch
run: git whatchanged --name-only --pretty="" origin..HEAD | grep CHANGELOG
lint-changelog:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Check CHANGELOG max line length
run: |
max_line_length=$(cat CHANGELOG.md | grep -Ev "^\[.*\]: https://github.com" | wc -L)
if [ $max_line_length -ge 80 ]; then
echo "ERROR: CHANGELOG has lines longer than 80 characters."
exit 1
fi
build-mails:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/mail
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: '18'
- name: Install yarn
run: npm install -g yarn
- name: Install node dependencies
run: yarn install --frozen-lockfile
- name: Build mails
run: yarn build
- name: Persist mails' templates
uses: actions/upload-artifact@v4
with:
name: mails-templates
path: src/backend/core/templates/mail
lint-back:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/backend
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: '3.10'
- name: Install development dependencies
run: pip install --user .[dev]
- name: Check code formatting with ruff
run: ~/.local/bin/ruff format impress --diff
- name: Lint code with ruff
run: ~/.local/bin/ruff check impress
- name: Lint code with pylint
run: ~/.local/bin/pylint impress
test-back:
runs-on: ubuntu-latest
needs: build-mails
defaults:
run:
working-directory: src/backend
services:
postgres:
image: postgres:16
env:
POSTGRES_DB: impress
POSTGRES_USER: dinum
POSTGRES_PASSWORD: pass
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
env:
DJANGO_CONFIGURATION: Test
DJANGO_SETTINGS_MODULE: impress.settings
DJANGO_SECRET_KEY: ThisIsAnExampleKeyForTestPurposeOnly
OIDC_OP_JWKS_ENDPOINT: /endpoint-for-test-purpose-only
DB_HOST: localhost
DB_NAME: impress
DB_USER: dinum
DB_PASSWORD: pass
DB_PORT: 5432
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
AWS_S3_ENDPOINT_URL: http://localhost:9000
AWS_S3_ACCESS_KEY_ID: impress
AWS_S3_SECRET_ACCESS_KEY: password
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Create writable /data
run: |
sudo mkdir -p /data/media && \
sudo mkdir -p /data/static
- name: Download mails' templates
uses: actions/download-artifact@v4
with:
name: mails-templates
path: src/backend/core/templates/mail
- name: Start Minio
run: |
docker pull minio/minio
docker run -d --name minio \
-p 9000:9000 \
-e "MINIO_ACCESS_KEY=impress" \
-e "MINIO_SECRET_KEY=password" \
-v /data/media:/data \
minio/minio server --console-address :9001 /data
- name: Configure MinIO
run: |
MINIO=$(docker ps | grep minio/minio | sed -E 's/.*\s+([a-zA-Z0-9_-]+)$/\1/')
docker exec ${MINIO} sh -c \
"mc alias set impress http://localhost:9000 impress password && \
mc alias ls && \
mc mb impress/impress-media-storage && \
mc version enable impress/impress-media-storage"
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: '3.10'
- name: Install development dependencies
run: pip install --user .[dev]
- name: Install gettext (required to compile messages)
run: |
sudo apt-get update
sudo apt-get install -y gettext
- name: Generate a MO file from strings extracted from the project
run: python manage.py compilemessages
- name: Run tests
run: ~/.local/bin/pytest -n 2
i18n-crowdin:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Install gettext (required to make messages)
run: |
sudo apt-get update
sudo apt-get install -y gettext
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: '3.10'
- name: Install development dependencies
working-directory: src/backend
run: pip install --user .[dev]
- name: Generate the translation base file
run: ~/.local/bin/django-admin makemessages --keep-pot --all
- name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: .github/workflows/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '18.x'
cache: 'yarn'
cache-dependency-path: src/frontend/yarn.lock
- name: Install dependencies
run: cd src/frontend/ && yarn install --frozen-lockfile
- name: Extract the frontend translation
run: make frontend-i18n-extract
- name: Upload files to Crowdin
run: |
docker run \
--rm \
-e CROWDIN_API_TOKEN=$CROWDIN_API_TOKEN \
-e CROWDIN_PROJECT_ID=$CROWDIN_PROJECT_ID \
-e CROWDIN_BASE_PATH=$CROWDIN_BASE_PATH \
-v "${{ github.workspace }}:/app" \
crowdin/cli:3.16.0 \
crowdin upload sources -c /app/crowdin/config.yml
+22
View File
@@ -0,0 +1,22 @@
SOPS_PRIVATE=ENC[AES256_GCM,data:53ysyQ9gq2PnAQKNjOL+e+Bu5SQIuOguz8Bo5CpqbpYsF0AmV1WsOutckdClbu6ApqV3m9/Cj1FJ30+L/+j05pvcpqMeehPQwGQ=,iv:VMuML9IXiEqKY9jp+ny76jnQHmewq2rqdBy1wYpZkSI=,tag:aAZgwiWDg1AG4wk3f2Fq4w==,type:str]
CROWDIN_API_TOKEN=ENC[AES256_GCM,data:bwh38oLDH4BpI2H+7oUjtVizyrYvVJ6Av4ECTnyPPthMz6DCaYQn55RXp8rQDgJj4bPRls+JcRVC94zYIjgpkDsbbcqHr620KQKHQHMgoOQ=,iv:hydpwWtCiOkhBpAYyNwDzSjhjfdUJcKX7YX3/PXteN0=,tag:eQLniL5XxkNs5yThUuQHyw==,type:str]
CROWDIN_BASE_PATH=ENC[AES256_GCM,data:LJZE454A6qg=,iv:yIjGACBJSX3S9g7PAHRFn074xL94fHvMLcTKzFYwkwo=,tag:1Z8+UbeDOvTxR80b95KumQ==,type:str]
CROWDIN_PROJECT_ID=ENC[AES256_GCM,data:THoNz661,iv:Ixd0D9tnpEWd2yqZui1HJQEO/h7YsAC1R9Vjj8OHBjA=,tag:wfDHhzaXLD3NwY5zDj24RA==,type:str]
DOCKER_HUB_PASSWORD=ENC[AES256_GCM,data:jj92OOVMtsagOXQ=,iv:r/u8M70PspZMFCbi8a3FvuCDtWt+9YGArPNHZRpHA+k=,tag:WM3vzVkuQZVdHa3wh4satg==,type:str]
DOCKER_HUB_USER=ENC[AES256_GCM,data:btdtLdLApQ==,iv:y1o2zwyzusBS6JiQSEtZwS2zctISo+UgAFhyZ53vbKQ=,tag:ZLkMJydgjMBmbbKq979z7g==,type:str]
ARGOCD_WEBHOOK_URL=ENC[AES256_GCM,data:0TnoZv7vQI+8MZ/7EITx0Mvez66G6BcCzw+Mic+NH2qh0BdZBH8ynkYBleKw9V6TbucgHasa7duL,iv:GeE5tSpjAndThrXrzz8Dk6ah9Bxv6JQCJmKAfsToDi0=,tag:O2pIhA0ge1xygIv0izSMxg==,type:str]
ARGOCD_WEBHOOK_SECRET=ENC[AES256_GCM,data:SrdWdV24lGztyUnFXeOYGAhqTErRFakIm7hBw8n4NKW6ll6AgeZKY6w7pbvgFknQ+NlRd/EK7bYk7CZtPDGU6zM=,iv:IkWxnTWrvzWwNh4RSt3N7iPHA7K7jkzSHa4CHptxxvU=,tag:XFVYBRsuDF/La1/8ADQ2jw==,type:str]
sops_age__list_0__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBESDdJSzBaaVlEbHRjSlIy\naVoyY2l6RVVqVXhOekV4NHdHQjV6Q0IzSEJNCk9JY3BFQ2tFWXBZVFMyWTJUYjdz\nMVdheTd4cjhFREl5MmNncmlobVNyUUUKLS0tIEg1MHBsV2FoRkFlN2JoNlFuTFFS\nNG5yUXZpQVY4Z1FGZmVLUjBqQWhSQTgKfT7hD5LVWg2NOrdyeIiVt6BX/4dt6fpN\nyydn2U0yxMg9fUZ7KkixAaWpChL3rvi3OWM07h6EdsznTwehLiMFTw==\n-----END AGE ENCRYPTED FILE-----\n
sops_age__list_0__map_recipient=age15fyxdwmg5mvldtqqus87xspuws2u0cpvwheehrtvkexj4tnsqqysw6re2x
sops_age__list_1__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBJa3EzbDJBeHcrUE44SXpM\ndlVheHdxc2I4ellwcHlUQkhWL2NiMFpBYUd3CmJxZUZhL0tZVkViQTZFRVRFbndC\nd2ljZUJxczZqSmdqcXlzYkZlZ2t4MTgKLS0tIFFmbHE1NXpOYlRnb2wzSTRVbTQ4\nMDhTNzN6WHovMXFhek5pbXZlMW1PdEkKJlydhV9Es+y2ngMwZMGnuF+JnEV1TGZH\nkWoBHxTSA7WEgwnhGaCe7kuzXrvv2ikrV1Ww7sN4wmqfCGC2sdkPBQ==\n-----END AGE ENCRYPTED FILE-----\n
sops_age__list_1__map_recipient=age16hnlml8yv4ynwy0seer57g8qww075crd0g7nsundz3pj4wk7m3vqftszg7
sops_age__list_2__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAzZEpFZU5maklnN1N4S0kw\nRGFNYzBGR2tFT2d5VzlRYU9NUWVvZld0REQ4CldvTlFtK0RFU0tuNjVhNEM4VzlC\nWjJhUEZVY0l0T05yNVBabXNEdndlbVkKLS0tIGxxdEROcWxpSHczMkN0dkdicnVZ\nT1BXR1hSa2l1SXdYS3RoWWh6NGdWSHcKZJd6HYESjLomY7/S9+eCCN4cFXERipNl\nWtOVZXlufN5BMxX8n8TlKS34oD1t6/CMaZZdmp2SHHslipA+CGRZ5g==\n-----END AGE ENCRYPTED FILE-----\n
sops_age__list_2__map_recipient=age1plkp8td6zzfcavjusmsfrlk54t9vn8jjxm8zaz7cmnr7kzl2nfnsd54hwg
sops_age__list_3__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAzakNpcGkzWlp6NWt1NFU0\ncmhFek1DTU5YS1MyYzRoOGJ2RXdjRU5WcEZBCjN5eUp6WVh0YmdNMzdHTUNJTVZM\ncHZTY3pxbHd0TmhSWmQyVndZS1JjZ00KLS0tIFNxYjZXRHBKbjNxVitQaGlKQVh0\ncHAwbzFyL3hUVmN2dVNQaklIcXZKQjgKr4IO6BoTFO7Km9V/h8tF3UNRCGUXymIw\nnQGL0ZDyIQw7MMBQQ2mksYPSBTFmaejbSd29UkhVnYFuCjJ+LVmX1w==\n-----END AGE ENCRYPTED FILE-----\n
sops_age__list_3__map_recipient=age12g6f5fse25tgrwweleh4jls3qs52hey2edh759smulwmk5lnzadslu2cp3
sops_age__list_4__map_enc=-----BEGIN AGE ENCRYPTED FILE-----\nYWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBpS0hNdDk2Lys4Rk9nVlV1\nWHVwOHcxT3RmZkVSMWh6L0M1bGRrNEt3c1M0CmdseVlqaFZYZjd6KzI0ejdDSG55\nNkFlMGpiOFhMZWtKYkVodGpmUWRsMjgKLS0tIG5ZbVFadk5XVlREZFFEcWNiSDhw\nVnh5b3BURGU4bCtQQzR3b3hxcXdGSlEKBw7E/umovQnucE4oYeuoHFlEtYBMVXPL\n6YjZzBpBxJ+4kZpMvqsXzowQ7ZDEods9pEcuJmHqxrRpLeOrYrykTA==\n-----END AGE ENCRYPTED FILE-----\n
sops_age__list_4__map_recipient=age1qy04neuzwpasmvljqrcvhwnf0kz5cpyteze38c8avp0czewskasszv9pyw
sops_lastmodified=2024-04-03T15:36:15Z
sops_mac=ENC[AES256_GCM,data:1v44C4K4YjV1m7tZKRgj8SiDamdD+L4p3TVwwOl6+05KCOh2uH2ohH+5MH7MTFL489oqaadpjBQfELSJ8h/4fN5MT6+Trbtk5QFLv4moLZx1tSCE1Tuam2cicFem2mlOrxb0pK/tU1qzCLvZke3yvFmiJEa+92u7y96hXM4VR6Y=,iv:23T3Tl5DvRH8zvef7ftbr5GWk+YFfLCzZ/eEzqjMKXY=,tag:TIch+2911w5qleXo55zM0w==,type:str]
sops_unencrypted_suffix=_unencrypted
sops_version=3.8.1
+81
View File
@@ -0,0 +1,81 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
.DS_Store
.next/
# Translations # Translations
*.pot
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
env.d/development/*
!env.d/development/*.dist
env.d/terraform
# npm
node_modules
# Mails
src/backend/core/templates/mail/
# Typescript client
src/frontend/tsclient
# Swagger
**/swagger.json
# Logs
*.log
# Terraform
.terraform
*.tfstate
*.tfstate.backup
# Test & lint
.coverage
.pylint.d
.pytest_cache
db.sqlite3
.mypy_cache
# Site media
/data/
# IDEs
.idea/
.vscode/
*.iml
.devcontainer
+78
View File
@@ -0,0 +1,78 @@
# All these sections are optional, edit this file as you like.
[general]
# Ignore certain rules, you can reference them by their id or by their full name
# ignore=title-trailing-punctuation, T3
# verbosity should be a value between 1 and 3, the commandline -v flags take precedence over this
# verbosity = 2
# By default gitlint will ignore merge commits. Set to 'false' to disable.
# ignore-merge-commits=true
# By default gitlint will ignore fixup commits. Set to 'false' to disable.
# ignore-fixup-commits=true
# By default gitlint will ignore squash commits. Set to 'false' to disable.
# ignore-squash-commits=true
# Enable debug mode (prints more output). Disabled by default.
# debug=true
# Set the extra-path where gitlint will search for user defined rules
# See http://jorisroovers.github.io/gitlint/user_defined_rules for details
extra-path=gitlint/
# [title-max-length]
# line-length=80
[title-must-not-contain-word]
# Comma-separated list of words that should not occur in the title. Matching is case
# insensitive. It's fine if the keyword occurs as part of a larger word (so "WIPING"
# will not cause a violation, but "WIP: my title" will.
words=wip
#[title-match-regex]
# python like regex (https://docs.python.org/2/library/re.html) that the
# commit-msg title must be matched to.
# Note that the regex can contradict with other rules if not used correctly
# (e.g. title-must-not-contain-word).
#regex=
# [B1]
# B1 = body-max-line-length
# line-length=120
# [body-min-length]
# min-length=5
# [body-is-missing]
# Whether to ignore this rule on merge commits (which typically only have a title)
# default = True
# ignore-merge-commits=false
# [body-changed-file-mention]
# List of files that need to be explicitly mentioned in the body when they are changed
# This is useful for when developers often erroneously edit certain files or git submodules.
# By specifying this rule, developers can only change the file when they explicitly reference
# it in the commit message.
# files=gitlint/rules.py,README.md
# [author-valid-email]
# python like regex (https://docs.python.org/2/library/re.html) that the
# commit author email address should be matched to
# For example, use the following regex if you only want to allow email addresses from foo.com
# regex=[^@]+@foo.com
[ignore-by-title]
# Allow empty body & wrong title pattern only when bots (pyup/greenkeeper)
# upgrade dependencies
regex=^(⬆️.*|Update (.*) from (.*) to (.*)|(chore|fix)\(package\): update .*)$
ignore=B6,UC1
# [ignore-by-body]
# Ignore certain rules for commits of which the body has a line that matches a regex
# E.g. Match bodies that have a line that that contain "release"
# regex=(.*)release(.*)
#
# Ignore certain rules, you can reference them by their id or by their full name
# Use 'all' to ignore all rules
# ignore=T1,body-min-length
+13
View File
@@ -0,0 +1,13 @@
creation_rules:
# Here we have
# - Jacques key-id: age15fyxdwmg5mvldtqqus87xspuws2u0cpvwheehrtvkexj4tnsqqysw6re2x
# - github-repo key-id: age16hnlml8yv4ynwy0seer57g8qww075crd0g7nsundz3pj4wk7m3vqftszg7
# - Anthony Le-Courric key-id: age1plkp8td6zzfcavjusmsfrlk54t9vn8jjxm8zaz7cmnr7kzl2nfnsd54hwg
# - Antoine Lebaud key-id: age12g6f5fse25tgrwweleh4jls3qs52hey2edh759smulwmk5lnzadslu2cp3
# - argocd key-id: age1qy04neuzwpasmvljqrcvhwnf0kz5cpyteze38c8avp0czewskasszv9pyw
- age:
age15fyxdwmg5mvldtqqus87xspuws2u0cpvwheehrtvkexj4tnsqqysw6re2x,
age16hnlml8yv4ynwy0seer57g8qww075crd0g7nsundz3pj4wk7m3vqftszg7,
age1plkp8td6zzfcavjusmsfrlk54t9vn8jjxm8zaz7cmnr7kzl2nfnsd54hwg,
age12g6f5fse25tgrwweleh4jls3qs52hey2edh759smulwmk5lnzadslu2cp3,
age1qy04neuzwpasmvljqrcvhwnf0kz5cpyteze38c8avp0czewskasszv9pyw
+20
View File
@@ -0,0 +1,20 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0),
and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.1.0] - 2024-05-24
## Added
- Coming Soon page (#67)
- Impress, project to manage your documents easily and collaboratively.
[unreleased]: https://github.com/numerique-gouv/impress/compare/v0.1.0...main
[0.1.0]: https://github.com/numerique-gouv/impress/releases/v0.1.0
+150
View File
@@ -0,0 +1,150 @@
# Django impress
# ---- base image to inherit from ----
FROM python:3.10-slim-bullseye as base
# Upgrade pip to its latest release to speed up dependencies installation
RUN python -m pip install --upgrade pip
# Upgrade system packages to install security updates
RUN apt-get update && \
apt-get -y upgrade && \
rm -rf /var/lib/apt/lists/*
# ---- Back-end builder image ----
FROM base as back-builder
WORKDIR /builder
# Copy required python dependencies
COPY ./src/backend /builder
RUN mkdir /install && \
pip install --prefix=/install .
# ---- mails ----
FROM node:20 as mail-builder
COPY ./src/mail /mail/app
WORKDIR /mail/app
RUN yarn install --frozen-lockfile && \
yarn build
# ---- static link collector ----
FROM base as link-collector
ARG IMPRESS_STATIC_ROOT=/data/static
# Install libpangocairo & rdfind
RUN apt-get update && \
apt-get install -y \
libpangocairo-1.0-0 \
rdfind && \
rm -rf /var/lib/apt/lists/*
# Copy installed python dependencies
COPY --from=back-builder /install /usr/local
# Copy impress application (see .dockerignore)
COPY ./src/backend /app/
WORKDIR /app
# collectstatic
RUN DJANGO_CONFIGURATION=Build DJANGO_JWT_PRIVATE_SIGNING_KEY=Dummy \
python manage.py collectstatic --noinput
# Replace duplicated file by a symlink to decrease the overall size of the
# final image
RUN rdfind -makesymlinks true -followsymlinks true -makeresultsfile false ${IMPRESS_STATIC_ROOT}
# ---- Core application image ----
FROM base as core
ENV PYTHONUNBUFFERED=1
# Install required system libs
RUN apt-get update && \
apt-get install -y \
gettext \
libcairo2 \
libffi-dev \
libgdk-pixbuf2.0-0 \
libpango-1.0-0 \
libpangocairo-1.0-0 \
shared-mime-info && \
rm -rf /var/lib/apt/lists/*
# Copy entrypoint
COPY ./docker/files/usr/local/bin/entrypoint /usr/local/bin/entrypoint
# Give the "root" group the same permissions as the "root" user on /etc/passwd
# to allow a user belonging to the root group to add new users; typically the
# docker user (see entrypoint).
RUN chmod g=u /etc/passwd
# Copy installed python dependencies
COPY --from=back-builder /install /usr/local
# Copy impress application (see .dockerignore)
COPY ./src/backend /app/
WORKDIR /app
# We wrap commands run in this container by the following entrypoint that
# creates a user on-the-fly with the container user ID (see USER) and root group
# ID.
ENTRYPOINT [ "/usr/local/bin/entrypoint" ]
# ---- Development image ----
FROM core as backend-development
# Switch back to the root user to install development dependencies
USER root:root
# Install psql
RUN apt-get update && \
apt-get install -y postgresql-client && \
rm -rf /var/lib/apt/lists/*
# Uninstall impress and re-install it in editable mode along with development
# dependencies
RUN pip uninstall -y impress
RUN pip install -e .[dev]
# Restore the un-privileged user running the application
ARG DOCKER_USER
USER ${DOCKER_USER}
# Target database host (e.g. database engine following docker compose services
# name) & port
ENV DB_HOST=postgresql \
DB_PORT=5432
# Run django development server
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
# ---- Production image ----
FROM core as backend-production
ARG IMPRESS_STATIC_ROOT=/data/static
# Gunicorn
RUN mkdir -p /usr/local/etc/gunicorn
COPY docker/files/usr/local/etc/gunicorn/impress.py /usr/local/etc/gunicorn/impress.py
# Un-privileged user running the application
ARG DOCKER_USER
USER ${DOCKER_USER}
# Copy statics
COPY --from=link-collector ${IMPRESS_STATIC_ROOT} ${IMPRESS_STATIC_ROOT}
# Copy impress mails
COPY --from=mail-builder /mail/backend/core/templates/mail /app/core/templates/mail
# The default command runs gunicorn WSGI server in impress's main module
CMD ["gunicorn", "-c", "/usr/local/etc/gunicorn/impress.py", "impress.wsgi:application"]
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2023 Direction Interministérielle du Numérique - Gouvernement Français
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+334
View File
@@ -0,0 +1,334 @@
# /!\ /!\ /!\ /!\ /!\ /!\ /!\ DISCLAIMER /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\
#
# This Makefile is only meant to be used for DEVELOPMENT purpose as we are
# changing the user id that will run in the container.
#
# PLEASE DO NOT USE IT FOR YOUR CI/PRODUCTION/WHATEVER...
#
# /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\
#
# Note to developers:
#
# While editing this file, please respect the following statements:
#
# 1. Every variable should be defined in the ad hoc VARIABLES section with a
# relevant subsection
# 2. Every new rule should be defined in the ad hoc RULES section with a
# relevant subsection depending on the targeted service
# 3. Rules should be sorted alphabetically within their section
# 4. When a rule has multiple dependencies, you should:
# - duplicate the rule name to add the help string (if required)
# - write one dependency per line to increase readability and diffs
# 5. .PHONY rule statement should be written after the corresponding rule
# ==============================================================================
# VARIABLES
BOLD := \033[1m
RESET := \033[0m
GREEN := \033[1;32m
# -- Database
DB_HOST = postgresql
DB_PORT = 5432
# -- Docker
# Get the current user ID to use for docker run and docker exec commands
DOCKER_UID = $(shell id -u)
DOCKER_GID = $(shell id -g)
DOCKER_USER = $(DOCKER_UID):$(DOCKER_GID)
COMPOSE = DOCKER_USER=$(DOCKER_USER) docker compose
COMPOSE_EXEC = $(COMPOSE) exec
COMPOSE_EXEC_APP = $(COMPOSE_EXEC) app-dev
COMPOSE_RUN = $(COMPOSE) run --rm
COMPOSE_RUN_APP = $(COMPOSE_RUN) app-dev
COMPOSE_RUN_CROWDIN = $(COMPOSE_RUN) crowdin crowdin
WAIT_DB = @$(COMPOSE_RUN) dockerize -wait tcp://$(DB_HOST):$(DB_PORT) -timeout 60s
# -- Backend
MANAGE = $(COMPOSE_RUN_APP) python manage.py
MAIL_YARN = $(COMPOSE_RUN) -w /app/src/mail node yarn
TSCLIENT_YARN = $(COMPOSE_RUN) -w /app/src/tsclient node yarn
# -- Frontend
PATH_FRONT = ./src/frontend
PATH_FRONT_IMPRESS = $(PATH_FRONT)/apps/impress
# ==============================================================================
# RULES
default: help
data/media:
@mkdir -p data/media
data/static:
@mkdir -p data/static
# -- Project
create-env-files: ## Copy the dist env files to env files
create-env-files: \
env.d/development/common \
env.d/development/crowdin \
env.d/development/postgresql \
env.d/development/kc_postgresql
.PHONY: create-env-files
bootstrap: ## Prepare Docker images for the project
bootstrap: \
data/media \
data/static \
create-env-files \
build \
run \
migrate \
demo \
back-i18n-compile \
mails-install \
mails-build \
install-front-impress
.PHONY: bootstrap
# -- Docker/compose
build: ## build the app-dev container
@$(COMPOSE) build app-dev --no-cache
.PHONY: build
down: ## stop and remove containers, networks, images, and volumes
@$(COMPOSE) down
.PHONY: down
logs: ## display app-dev logs (follow mode)
@$(COMPOSE) logs -f app-dev
.PHONY: logs
run: ## start the wsgi (production) and development server
@$(COMPOSE) up --force-recreate -d nginx
@$(COMPOSE) up --force-recreate -d app-dev
@$(COMPOSE) up --force-recreate -d celery-dev
@$(COMPOSE) up --force-recreate -d keycloak
@$(COMPOSE) up --force-recreate -d y-webrtc-signaling
@echo "Wait for postgresql to be up..."
@$(WAIT_DB)
.PHONY: run
status: ## an alias for "docker compose ps"
@$(COMPOSE) ps
.PHONY: status
stop: ## stop the development server using Docker
@$(COMPOSE) stop
.PHONY: stop
# -- Backend
demo: ## flush db then create a demo for load testing purpose
@$(MAKE) resetdb
@$(MANAGE) create_demo
.PHONY: demo
# Nota bene: Black should come after isort just in case they don't agree...
lint: ## lint back-end python sources
lint: \
lint-ruff-format \
lint-ruff-check \
lint-pylint
.PHONY: lint
lint-ruff-format: ## format back-end python sources with ruff
@echo 'lint:ruff-format started…'
@$(COMPOSE_RUN_APP) ruff format .
.PHONY: lint-ruff-format
lint-ruff-check: ## lint back-end python sources with ruff
@echo 'lint:ruff-check started…'
@$(COMPOSE_RUN_APP) ruff check . --fix
.PHONY: lint-ruff-check
lint-pylint: ## lint back-end python sources with pylint only on changed files from main
@echo 'lint:pylint started…'
bin/pylint --diff-only=origin/main
.PHONY: lint-pylint
test: ## run project tests
@$(MAKE) test-back-parallel
.PHONY: test
test-back: ## run back-end tests
@args="$(filter-out $@,$(MAKECMDGOALS))" && \
bin/pytest $${args:-${1}}
.PHONY: test-back
test-back-parallel: ## run all back-end tests in parallel
@args="$(filter-out $@,$(MAKECMDGOALS))" && \
bin/pytest -n auto $${args:-${1}}
.PHONY: test-back-parallel
makemigrations: ## run django makemigrations for the impress project.
@echo "$(BOLD)Running makemigrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) makemigrations
.PHONY: makemigrations
migrate: ## run django migrations for the impress project.
@echo "$(BOLD)Running migrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) migrate
.PHONY: migrate
superuser: ## Create an admin superuser with password "admin"
@echo "$(BOLD)Creating a Django superuser$(RESET)"
@$(MANAGE) createsuperuser --email admin@example.com --password admin
.PHONY: superuser
back-i18n-compile: ## compile the gettext files
@$(MANAGE) compilemessages --ignore="venv/**/*"
.PHONY: back-i18n-compile
back-i18n-generate: ## create the .pot files used for i18n
@$(MANAGE) makemessages -a --keep-pot
.PHONY: back-i18n-generate
shell: ## connect to database shell
@$(MANAGE) shell #_plus
.PHONY: dbshell
# -- Database
dbshell: ## connect to database shell
docker compose exec app-dev python manage.py dbshell
.PHONY: dbshell
resetdb: FLUSH_ARGS ?=
resetdb: ## flush database and create a superuser "admin"
@echo "$(BOLD)Flush database$(RESET)"
@$(MANAGE) flush $(FLUSH_ARGS)
@${MAKE} superuser
.PHONY: resetdb
env.d/development/common:
cp -n env.d/development/common.dist env.d/development/common
env.d/development/postgresql:
cp -n env.d/development/postgresql.dist env.d/development/postgresql
env.d/development/kc_postgresql:
cp -n env.d/development/kc_postgresql.dist env.d/development/kc_postgresql
# -- Internationalization
env.d/development/crowdin:
cp -n env.d/development/crowdin.dist env.d/development/crowdin
crowdin-download: ## Download translated message from crowdin
@$(COMPOSE_RUN_CROWDIN) download -c crowdin/config.yml
.PHONY: crowdin-download
crowdin-download-sources: ## Download sources from Crowdin
@$(COMPOSE_RUN_CROWDIN) download sources -c crowdin/config.yml
.PHONY: crowdin-download-sources
crowdin-upload: ## Upload source translations to crowdin
@$(COMPOSE_RUN_CROWDIN) upload sources -c crowdin/config.yml
.PHONY: crowdin-upload
i18n-compile: ## compile all translations
i18n-compile: \
back-i18n-compile \
frontend-i18n-compile
.PHONY: i18n-compile
i18n-generate: ## create the .pot files and extract frontend messages
i18n-generate: \
back-i18n-generate \
frontend-i18n-generate
.PHONY: i18n-generate
i18n-download-and-compile: ## download all translated messages and compile them to be used by all applications
i18n-download-and-compile: \
crowdin-download \
i18n-compile
.PHONY: i18n-download-and-compile
i18n-generate-and-upload: ## generate source translations for all applications and upload them to Crowdin
i18n-generate-and-upload: \
i18n-generate \
crowdin-upload
.PHONY: i18n-generate-and-upload
# -- Mail generator
mails-build: ## Convert mjml files to html and text
@$(MAIL_YARN) build
.PHONY: mails-build
mails-build-html-to-plain-text: ## Convert html files to text
@$(MAIL_YARN) build-html-to-plain-text
.PHONY: mails-build-html-to-plain-text
mails-build-mjml-to-html: ## Convert mjml files to html and text
@$(MAIL_YARN) build-mjml-to-html
.PHONY: mails-build-mjml-to-html
mails-install: ## install the mail generator
@$(MAIL_YARN) install
.PHONY: mails-install
# -- TS client generator
tsclient-install: ## Install the Typescript API client generator
@$(TSCLIENT_YARN) install
.PHONY: tsclient-install
tsclient: tsclient-install ## Generate a Typescript API client
@$(TSCLIENT_YARN) generate:api:client:local ../frontend/tsclient
.PHONY: tsclient-install
# -- Misc
clean: ## restore repository state as it was freshly cloned
git clean -idx
.PHONY: clean
help:
@echo "$(BOLD)impress Makefile"
@echo "Please use 'make $(BOLD)target$(RESET)' where $(BOLD)target$(RESET) is one of:"
@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(firstword $(MAKEFILE_LIST)) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "$(GREEN)%-30s$(RESET) %s\n", $$1, $$2}'
.PHONY: help
# Front
install-front-impress: ## Install the frontend dependencies of app Impress
cd $(PATH_FRONT_IMPRESS) && yarn
.PHONY: install-front-impress
run-front-impress: ## Start app Impress
cd $(PATH_FRONT_IMPRESS) && yarn dev
.PHONY: run-front-impress
frontend-i18n-extract: ## Extract the frontend translation inside a json to be used for crowdin
cd $(PATH_FRONT) && yarn i18n:extract
.PHONY: frontend-i18n-extract
frontend-i18n-generate: ## Generate the frontend json files used for crowdin
frontend-i18n-generate: \
crowdin-download-sources \
frontend-i18n-extract
.PHONY: frontend-i18n-generate
frontend-i18n-compile: ## Format the crowin json files used deploy to the apps
cd $(PATH_FRONT) && yarn i18n:deploy
.PHONY: frontend-i18n-compile
# -- K8S
build-k8s-cluster: ## build the kubernetes cluster using kind
./bin/start-kind.sh
.PHONY: build-k8s-cluster
start-tilt: ## start the kubernetes cluster using kind
tilt up -f ./bin/Tiltfile
.PHONY: build-k8s-cluster
+88
View File
@@ -1 +1,89 @@
# Impress
Impress prints your markdown to pdf from predefined templates with user and role based access rights.
Impress is built on top of [Django Rest
Framework](https://www.django-rest-framework.org/) and [Next.js](https://nextjs.org/).
## Getting started
### Prerequisite
Make sure you have a recent version of Docker and [Docker
Compose](https://docs.docker.com/compose/install) installed on your laptop:
```bash
$ docker -v
Docker version 20.10.2, build 2291f61
$ docker compose -v
docker compose version 1.27.4, build 40524192
```
> ⚠️ You may need to run the following commands with `sudo` but this can be
> avoided by assigning your user to the `docker` group.
### Project bootstrap
The easiest way to start working on the project is to use GNU Make:
```bash
$ make bootstrap FLUSH_ARGS='--no-input'
```
Then you can run the following command to start the project in development mode:
```bash
$ make run-front-impress
```
You will be prompted to log in, the default credentials are:
```bash
username: impress
password: impress
```
---
This command builds the `app` container, installs dependencies, performs
database migrations and compile translations. It's a good idea to use this
command each time you are pulling code from the project repository to avoid
dependency-releated or migration-releated issues.
Your Docker services should now be up and running 🎉
Note that if you need to run them afterwards, you can use the eponym Make rule:
```bash
$ make run
```
### Adding content
You can create a basic demo site by running:
$ make demo
Finally, you can check all available Make rules using:
```bash
$ make help
```
### Django admin
You can access the Django admin site at
[http://localhost:8071/admin](http://localhost:8071/admin).
You first need to create a superuser account:
```bash
$ make superuser
```
## Contributing
This project is intended to be community-driven, so please, do not hesitate to
get in touch if you have any question related to our implementation or design
decisions.
## License
This work is released under the MIT License (see [LICENSE](./LICENSE)).
+17
View File
@@ -0,0 +1,17 @@
# Upgrade
All instructions to upgrade this project from one release to the next will be
documented in this file. Upgrades must be run sequentially, meaning you should
not skip minor/major releases while upgrading (fix releases can be skipped).
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
For most upgrades, you just need to run the django migrations with
the following command inside your docker container:
`python manage.py migrate`
(Note : in your development environment, you can `make migrate`.)
## [Unreleased]
+68
View File
@@ -0,0 +1,68 @@
load('ext://uibutton', 'cmd_button', 'bool_input', 'location')
load('ext://namespace', 'namespace_create', 'namespace_inject')
namespace_create('impress')
docker_build(
'localhost:5001/impress-backend:latest',
context='..',
dockerfile='../Dockerfile',
only=['./src/backend', './src/mail', './docker'],
target = 'backend-production',
live_update=[
sync('../src/backend', '/app'),
run(
'pip install -r /app/requirements.txt',
trigger=['./api/requirements.txt']
)
]
)
docker_build(
'localhost:5001/impress-y-webrtc-signaling:latest',
context='..',
dockerfile='../src/frontend/Dockerfile',
only=['./src/frontend/', './docker/', './dockerignore'],
target = 'y-webrtc-signaling',
live_update=[
sync('../src/frontend', '/home/frontend'),
]
)
docker_build(
'localhost:5001/impress-frontend:latest',
context='..',
dockerfile='../src/frontend/Dockerfile',
only=['./src/frontend', './docker', './dockerignore'],
target = 'impress',
live_update=[
sync('../src/frontend', '/home/frontend'),
]
)
k8s_yaml(local('cd ../src/helm && helmfile -n impress -e dev template .'))
migration = '''
set -eu
# get k8s pod name from tilt resource name
POD_NAME="$(tilt get kubernetesdiscovery impress-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n impress exec "$POD_NAME" -- python manage.py makemigrations
'''
cmd_button('Make migration',
argv=['sh', '-c', migration],
resource='impress-backend',
icon_name='developer_board',
text='Run makemigration',
)
pod_migrate = '''
set -eu
# get k8s pod name from tilt resource name
POD_NAME="$(tilt get kubernetesdiscovery impress-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n impress exec "$POD_NAME" -- python manage.py migrate --no-input
'''
cmd_button('Migrate db',
argv=['sh', '-c', pod_migrate],
resource='impress-backend',
icon_name='developer_board',
text='Run database migration',
)
+157
View File
@@ -0,0 +1,157 @@
#!/usr/bin/env bash
set -eo pipefail
REPO_DIR="$(cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd)"
UNSET_USER=0
TERRAFORM_DIRECTORY="./env.d/terraform"
COMPOSE_FILE="${REPO_DIR}/docker-compose.yml"
COMPOSE_PROJECT="impress"
# _set_user: set (or unset) default user id used to run docker commands
#
# usage: _set_user
#
# You can override default user ID (the current host user ID), by defining the
# USER_ID environment variable.
#
# To avoid running docker commands with a custom user, please set the
# $UNSET_USER environment variable to 1.
function _set_user() {
if [ $UNSET_USER -eq 1 ]; then
USER_ID=""
return
fi
# USER_ID = USER_ID or `id -u` if USER_ID is not set
USER_ID=${USER_ID:-$(id -u)}
echo "🙋(user) ID: ${USER_ID}"
}
# docker_compose: wrap docker compose command
#
# usage: docker_compose [options] [ARGS...]
#
# options: docker compose command options
# ARGS : docker compose command arguments
function _docker_compose() {
echo "🐳(compose) project: '${COMPOSE_PROJECT}' file: '${COMPOSE_FILE}'"
docker compose \
-p "${COMPOSE_PROJECT}" \
-f "${COMPOSE_FILE}" \
--project-directory "${REPO_DIR}" \
"$@"
}
# _dc_run: wrap docker compose run command
#
# usage: _dc_run [options] [ARGS...]
#
# options: docker compose run command options
# ARGS : docker compose run command arguments
function _dc_run() {
_set_user
user_args="--user=$USER_ID"
if [ -z $USER_ID ]; then
user_args=""
fi
_docker_compose run --rm $user_args "$@"
}
# _dc_exec: wrap docker compose exec command
#
# usage: _dc_exec [options] [ARGS...]
#
# options: docker compose exec command options
# ARGS : docker compose exec command arguments
function _dc_exec() {
_set_user
echo "🐳(compose) exec command: '\$@'"
user_args="--user=$USER_ID"
if [ -z $USER_ID ]; then
user_args=""
fi
_docker_compose exec $user_args "$@"
}
# _django_manage: wrap django's manage.py command with docker compose
#
# usage : _django_manage [ARGS...]
#
# ARGS : django's manage.py command arguments
function _django_manage() {
_dc_run "app-dev" python manage.py "$@"
}
# _set_openstack_project: select an OpenStack project from the openrc files defined in the
# terraform directory.
#
# usage: _set_openstack_project
#
# If necessary the script will prompt the user to choose a project from those available
function _set_openstack_project() {
declare prompt
declare -a projects
declare -i default=1
declare -i choice=0
declare -i n_projects
# List projects by looking in the "./env.d/terraform" directory
# and store them in an array
read -r -a projects <<< "$(
find "${TERRAFORM_DIRECTORY}" -maxdepth 1 -mindepth 1 -type d |
sed 's|'"${TERRAFORM_DIRECTORY}\/"'||' |
xargs
)"
nb_projects=${#projects[@]}
if [[ ${nb_projects} -le 0 ]]; then
echo "There are no OpenStack projects defined..." >&2
echo "To add one, create a subdirectory in \"${TERRAFORM_DIRECTORY}\" with the name" \
"of your project and copy your \"openrc.sh\" file into it." >&2
exit 10
fi
if [[ ${nb_projects} -gt 1 ]]; then
prompt="Select an OpenStack project to target:\\n"
for (( i=0; i<nb_projects; i++ )); do
prompt+="[$((i+1))] ${projects[$i]}"
if [[ $((i+1)) -eq ${default} ]]; then
prompt+=" (default)"
fi
prompt+="\\n"
done
prompt+="If your OpenStack project is not listed, add it to the \"env.d/terraform\" directory.\\n"
prompt+="Your choice: "
read -r -p "$(echo -e "${prompt}")" choice
if [[ ${choice} -gt nb_projects ]]; then
(>&2 echo "Invalid choice ${choice} (should be <= ${nb_projects})")
exit 11
fi
if [[ ${choice} -le 0 ]]; then
choice=${default}
fi
fi
project=${projects[$((choice-1))]}
# Check that the openrc.sh file exists for this project
if [ ! -f "${TERRAFORM_DIRECTORY}/${project}/openrc.sh" ]; then
(>&2 echo "Missing \"openrc.sh\" file in \"${TERRAFORM_DIRECTORY}/${project}\". Check documentation.")
exit 12
fi
echo "${project}"
}
Executable
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
# shellcheck source=bin/_config.sh
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
_docker_compose "$@"
Executable
+6
View File
@@ -0,0 +1,6 @@
#!/usr/bin/env bash
# shellcheck source=bin/_config.sh
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
_django_manage "$@"
Executable
+38
View File
@@ -0,0 +1,38 @@
#!/usr/bin/env bash
# shellcheck source=bin/_config.sh
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
declare diff_from
declare -a paths
declare -a args
# Parse options
for arg in "$@"
do
case $arg in
--diff-only=*)
diff_from="${arg#*=}"
shift
;;
-*)
args+=("$arg")
shift
;;
*)
paths+=("$arg")
shift
;;
esac
done
if [[ -n "${diff_from}" ]]; then
# Run pylint only on modified files located in src/backend
# (excluding deleted files and migration files)
# shellcheck disable=SC2207
paths=($(git diff "${diff_from}" --name-only --diff-filter=d -- src/backend ':!**/migrations/*.py' | grep -E '^src/backend/.*\.py$'))
fi
# Fix docker vs local path when project sources are mounted as a volume
read -ra paths <<< "$(echo "${paths[@]}" | sed "s|src/backend/||g")"
_dc_run app-dev pylint "${paths[@]}" "${args[@]}"
Executable
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
_dc_run \
-e DJANGO_CONFIGURATION=Test \
app-dev \
pytest "$@"
+103
View File
@@ -0,0 +1,103 @@
#!/bin/sh
set -o errexit
CURRENT_DIR=$(pwd)
echo "0. Create ca"
# 0. Create ca
mkcert -install
cd /tmp
mkcert "127.0.0.1.nip.io" "*.127.0.0.1.nip.io"
cd $CURRENT_DIR
echo "1. Create registry container unless it already exists"
# 1. Create registry container unless it already exists
reg_name='kind-registry'
reg_port='5001'
if [ "$(docker inspect -f '{{.State.Running}}' "${reg_name}" 2>/dev/null || true)" != 'true' ]; then
docker run \
-d --restart=always -p "127.0.0.1:${reg_port}:5000" --network bridge --name "${reg_name}" \
registry:2
fi
echo "2. Create kind cluster with containerd registry config dir enabled"
# 2. Create kind cluster with containerd registry config dir enabled
# TODO: kind will eventually enable this by default and this patch will
# be unnecessary.
#
# See:
# https://github.com/kubernetes-sigs/kind/issues/2875
# https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration
# See: https://github.com/containerd/containerd/blob/main/docs/hosts.md
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"
nodes:
- role: control-plane
image: kindest/node:v1.27.3
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
extraPortMappings:
- containerPort: 80
hostPort: 80
protocol: TCP
- containerPort: 443
hostPort: 443
protocol: TCP
- role: worker
image: kindest/node:v1.27.3
- role: worker
image: kindest/node:v1.27.3
EOF
echo "3. Add the registry config to the nodes"
# 3. Add the registry config to the nodes
#
# This is necessary because localhost resolves to loopback addresses that are
# network-namespace local.
# In other words: localhost in the container is not localhost on the host.
#
# We want a consistent name that works from both ends, so we tell containerd to
# alias localhost:${reg_port} to the registry container when pulling images
REGISTRY_DIR="/etc/containerd/certs.d/localhost:${reg_port}"
for node in $(kind get nodes); do
docker exec "${node}" mkdir -p "${REGISTRY_DIR}"
cat <<EOF | docker exec -i "${node}" cp /dev/stdin "${REGISTRY_DIR}/hosts.toml"
[host."http://${reg_name}:5000"]
EOF
done
echo "4. Connect the registry to the cluster network if not already connected"
# 4. Connect the registry to the cluster network if not already connected
# This allows kind to bootstrap the network but ensures they're on the same network
if [ "$(docker inspect -f='{{json .NetworkSettings.Networks.kind}}' "${reg_name}")" = 'null' ]; then
docker network connect "kind" "${reg_name}"
fi
echo "5. Document the local registry"
# 5. Document the local registry
# https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: local-registry-hosting
namespace: kube-public
data:
localRegistryHosting.v1: |
host: "localhost:${reg_port}"
help: "https://kind.sigs.k8s.io/docs/user/local-registry/"
EOF
echo "6. Install ingress-nginx"
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
kubectl -n ingress-nginx create secret tls mkcert --key /tmp/127.0.0.1.nip.io+1-key.pem --cert /tmp/127.0.0.1.nip.io+1.pem
kubectl -n ingress-nginx patch deployments.apps ingress-nginx-controller --type 'json' -p '[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value":"--default-ssl-certificate=ingress-nginx/mkcert"}]'
Executable
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
set -eo pipefail
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
project=$(_set_openstack_project)
echo "Using \"${project}\" project..."
source "${TERRAFORM_DIRECTORY}/${project}/openrc.sh"
# Run Terraform commands in the Hashicorp docker container via docker compose
# shellcheck disable=SC2068
DOCKER_USER="$(id -u):$(id -g)" \
PROJECT="${project}" \
docker compose run --rm \
-e OS_AUTH_URL \
-e OS_IDENTITY_API_VERSION \
-e OS_USER_DOMAIN_NAME \
-e OS_PROJECT_DOMAIN_NAME \
-e OS_TENANT_ID \
-e OS_TENANT_NAME \
-e OS_USERNAME \
-e OS_PASSWORD \
-e OS_REGION_NAME \
terraform-state "$@"
Executable
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
set -eo pipefail
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
project=$(_set_openstack_project)
echo "Using \"${project}\" project..."
source "${TERRAFORM_DIRECTORY}/${project}/openrc.sh"
# Run Terraform commands in the Hashicorp docker container via docker compose
# shellcheck disable=SC2068
DOCKER_USER="$(id -u):$(id -g)" \
PROJECT="${project}" \
docker compose run --rm \
-e OS_AUTH_URL \
-e OS_IDENTITY_API_VERSION \
-e OS_USER_DOMAIN_NAME \
-e OS_PROJECT_DOMAIN_NAME \
-e OS_TENANT_ID \
-e OS_TENANT_NAME \
-e OS_USERNAME \
-e OS_PASSWORD \
-e OS_REGION_NAME \
-e TF_VAR_user_name \
terraform "$@"
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
_dc_run \
-e DJANGO_CONFIGURATION=Test \
app-dev \
python manage.py spectacular \
--api-version 'v1.0' \
--urlconf 'impress.api_urls' \
--format openapi-json \
--file /app/core/tests/swagger/swagger.json
+3
View File
@@ -0,0 +1,3 @@
#!/bin/bash
find . -name "*.enc.*" -exec sops updatekeys -y {} \;
+29
View File
@@ -0,0 +1,29 @@
#
# Your crowdin's credentials
#
api_token_env: CROWDIN_API_TOKEN
project_id_env: CROWDIN_PROJECT_ID
base_path_env: CROWDIN_BASE_PATH
#
# Choose file structure in crowdin
# e.g. true or false
#
preserve_hierarchy: true
#
# Files configuration
#
files: [
{
source : "/backend/locale/django.pot",
dest: "/backend-impress.pot",
translation : "/backend/locale/%locale_with_underscore%/LC_MESSAGES/django.po"
},
{
source: "/frontend/packages/i18n/locales/impress/translations-crowdin.json",
dest: "/frontend-impress.json",
translation: "/frontend/packages/i18n/locales/impress/%two_letters_code%/translations.json",
skip_untranslated_strings: true,
},
]
+193
View File
@@ -0,0 +1,193 @@
version: '3.8'
services:
postgresql:
image: postgres:16
env_file:
- env.d/development/postgresql
ports:
- "15432:5432"
redis:
image: redis:5
mailcatcher:
image: sj26/mailcatcher:latest
ports:
- "1081:1080"
minio:
user: ${DOCKER_USER:-1000}
image: minio/minio
environment:
- MINIO_ROOT_USER=impress
- MINIO_ROOT_PASSWORD=password
ports:
- '9000:9000'
- '9001:9001'
entrypoint: ""
command: minio server --console-address :9001 /data
volumes:
- ./data/media:/data
createbuckets:
image: minio/mc
depends_on:
- minio
entrypoint: >
sh -c "
/usr/bin/mc alias set impress http://minio:9000 impress password && \
/usr/bin/mc mb impress/impress-media-storage && \
/usr/bin/mc version enable impress/impress-media-storage && \
exit 0;"
app-dev:
build:
context: .
target: backend-development
args:
DOCKER_USER: ${DOCKER_USER:-1000}
user: ${DOCKER_USER:-1000}
image: impress:backend-development
environment:
- PYLINTHOME=/app/.pylint.d
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/postgresql
ports:
- "8071:8000"
volumes:
- ./src/backend:/app
- ./data/static:/data/static
depends_on:
- postgresql
- mailcatcher
- redis
- createbuckets
celery-dev:
user: ${DOCKER_USER:-1000}
image: impress:backend-development
command: ["celery", "-A", "impress.celery_app", "worker", "-l", "DEBUG"]
environment:
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/postgresql
volumes:
- ./src/backend:/app
- ./data/static:/data/static
depends_on:
- app-dev
app:
build:
context: .
target: backend-production
args:
DOCKER_USER: ${DOCKER_USER:-1000}
user: ${DOCKER_USER:-1000}
image: impress:backend-production
environment:
- DJANGO_CONFIGURATION=Demo
env_file:
- env.d/development/common
- env.d/development/postgresql
depends_on:
- postgresql
- redis
- minio
celery:
user: ${DOCKER_USER:-1000}
image: impress:backend-production
command: ["celery", "-A", "impress.celery_app", "worker", "-l", "INFO"]
environment:
- DJANGO_CONFIGURATION=Demo
env_file:
- env.d/development/common
- env.d/development/postgresql
depends_on:
- app
nginx:
image: nginx:1.25
ports:
- "8083:8083"
volumes:
- ./docker/files/etc/nginx/conf.d:/etc/nginx/conf.d:ro
depends_on:
- app
- keycloak
dockerize:
image: jwilder/dockerize
crowdin:
image: crowdin/cli:3.16.0
volumes:
- ".:/app"
env_file:
- env.d/development/crowdin
user: "${DOCKER_USER:-1000}"
working_dir: /app
node:
image: node:18
user: "${DOCKER_USER:-1000}"
environment:
HOME: /tmp
volumes:
- ".:/app"
y-webrtc-signaling:
user: ${DOCKER_USER:-1000}
build:
context: .
dockerfile: ./src/frontend/Dockerfile
target: y-webrtc-signaling
restart: unless-stopped
ports:
- "4444:4444"
volumes:
- ./src/frontend/apps/y-webrtc-signaling:/home/frontend/apps/y-webrtc-signaling
- /home/frontend/apps/y-webrtc-signaling/node_modules/
- /home/frontend/apps/y-webrtc-signaling/dist/
kc_postgresql:
image: postgres:14.3
platform: linux/amd64
ports:
- "5433:5432"
env_file:
- env.d/development/kc_postgresql
keycloak:
image: quay.io/keycloak/keycloak:20.0.1
volumes:
- ./docker/auth/realm.json:/opt/keycloak/data/import/realm.json
command:
- start-dev
- --features=preview
- --import-realm
- --proxy=edge
- --hostname-url=http://localhost:8083
- --hostname-admin-url=http://localhost:8083/
- --hostname-strict=false
- --hostname-strict-https=false
environment:
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: admin
KC_DB: postgres
KC_DB_URL_HOST: kc_postgresql
KC_DB_URL_DATABASE: keycloak
KC_DB_PASSWORD: pass
KC_DB_USERNAME: impress
KC_DB_SCHEMA: public
PROXY_ADDRESS_FORWARDING: 'true'
ports:
- "8080:8080"
depends_on:
- kc_postgresql
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,13 @@
server {
listen 8083;
server_name localhost;
charset utf-8;
location / {
proxy_pass http://keycloak:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
+35
View File
@@ -0,0 +1,35 @@
#!/bin/sh
#
# The container user (see USER in the Dockerfile) is an un-privileged user that
# does not exists and is not created during the build phase (see Dockerfile).
# Hence, we use this entrypoint to wrap commands that will be run in the
# container to create an entry for this user in the /etc/passwd file.
#
# The following environment variables may be passed to the container to
# customize running user account:
#
# * USER_NAME: container user name (default: default)
# * HOME : container user home directory (default: none)
#
# To pass environment variables, you can either use the -e option of the docker run command:
#
# docker run --rm -e USER_NAME=foo -e HOME='/home/foo' impress:latest python manage.py migrate
#
# or define new variables in an environment file to use with docker or docker compose:
#
# # env.d/production
# USER_NAME=foo
# HOME=/home/foo
#
# docker run --rm --env-file env.d/production impress:latest python manage.py migrate
#
echo "🐳(entrypoint) creating user running in the container..."
if ! whoami > /dev/null 2>&1; then
if [ -w /etc/passwd ]; then
echo "${USER_NAME:-default}:x:$(id -u):$(id -g):${USER_NAME:-default} user:${HOME}:/sbin/nologin" >> /etc/passwd
fi
fi
echo "🐳(entrypoint) running your command: ${*}"
exec "$@"
@@ -0,0 +1,16 @@
# Gunicorn-django settings
bind = ["0.0.0.0:8000"]
name = "impress"
python_path = "/app"
# Run
graceful_timeout = 90
timeout = 90
workers = 3
# Logging
# Using '-' for the access log file makes gunicorn log accesses to stdout
accesslog = "-"
# Using '-' for the error log file makes gunicorn log errors to stderr
errorlog = "-"
loglevel = "info"
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
+25
View File
@@ -0,0 +1,25 @@
# Api client TypeScript
The backend application can automatically create a TypeScript client to be used in frontend
applications. It is used in the impress front application itself.
This client is made with [openapi-typescript-codegen](https://github.com/ferdikoomen/openapi-typescript-codegen)
and impress's backend OpenAPI schema (available [here](http://localhost:8071/v1.0/swagger/) if you have the backend running).
## Requirements
We'll need the online OpenAPI schema generated by swagger. Therefore you will first need to
install the backend application.
## Install openApiClientJs
```sh
$ cd src/tsclient
$ yarn install
```
## Generate the client
```sh
yarn generate:api:client:local <output_path_for_generated_client>
```
+41
View File
@@ -0,0 +1,41 @@
# Django
DJANGO_ALLOWED_HOSTS=*
DJANGO_SECRET_KEY=ThisIsAnExampleKeyForDevPurposeOnly
DJANGO_SETTINGS_MODULE=impress.settings
DJANGO_SUPERUSER_PASSWORD=admin
# Python
PYTHONPATH=/app
# impress settings
# Mail
DJANGO_EMAIL_HOST="mailcatcher"
DJANGO_EMAIL_PORT=1025
# Backend url
IMPRESS_BASE_URL="http://localhost:8072"
# Media
STORAGES_STATICFILES_BACKEND=django.contrib.staticfiles.storage.StaticFilesStorage
AWS_S3_ENDPOINT_URL=http://minio:9000
AWS_S3_ACCESS_KEY_ID=impress
AWS_S3_SECRET_ACCESS_KEY=password
# OIDC
OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT=http://localhost:8083/realms/impress/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/token
OIDC_OP_USER_ENDPOINT=http://nginx:8083/realms/impress/protocol/openid-connect/userinfo
OIDC_RP_CLIENT_ID=impress
OIDC_RP_CLIENT_SECRET=ThisIsAnExampleKeyForDevPurposeOnly
OIDC_RP_SIGN_ALGO=RS256
OIDC_RP_SCOPES="openid email"
LOGIN_REDIRECT_URL=http://localhost:3000
LOGIN_REDIRECT_URL_FAILURE=http://localhost:3000
LOGOUT_REDIRECT_URL=http://localhost:3000
OIDC_REDIRECT_ALLOWED_HOSTS=["http://localhost:8083", "http://localhost:3000"]
OIDC_AUTH_REQUEST_EXTRA_PARAMS={"acr_values": "eidas1"}
+3
View File
@@ -0,0 +1,3 @@
# For the CI job test-e2e
SUSTAINED_THROTTLE_RATES="200/hour"
BURST_THROTTLE_RATES="200/minute"
+3
View File
@@ -0,0 +1,3 @@
CROWDIN_API_TOKEN=Your-Api-Token
CROWDIN_PROJECT_ID=Your-Project-Id
CROWDIN_BASE_PATH=/app/src

Some files were not shown because too many files have changed in this diff Show More