Compare commits

...

39 Commits

Author SHA1 Message Date
mascarpon3 72a56d756f Publish find-0.0.3.tgz 2026-03-16 08:58:33 +00:00
mascarpon3 fc7e3fc5c2 Publish find-0.0.3.tgz 2026-03-10 12:49:07 +00:00
qbey d4cab977e0 Publish find-0.0.3.tgz 2026-03-10 12:40:21 +00:00
qbey bb4a4c9c92 Publish find-0.0.3.tgz 2026-02-27 12:32:18 +00:00
qbey 132f45a8b9 Publish find-0.0.3.tgz 2026-02-27 12:22:04 +00:00
qbey d48636eebb Publish find-0.0.3.tgz 2026-02-27 12:15:58 +00:00
lunika c80ce27af5 Publish find-0.0.3.tgz 2026-02-19 21:43:04 +00:00
mascarpon3 fc395fadb9 Publish find-0.0.3.tgz 2026-02-09 15:45:39 +00:00
mascarpon3 4a112b66ff Publish find-0.0.3.tgz 2026-02-09 15:38:25 +00:00
mascarpon3 35317dadac Publish find-0.0.3.tgz 2026-02-09 15:18:52 +00:00
mascarpon3 b7765f4ada Publish find-0.0.3.tgz 2026-01-20 09:36:19 +00:00
qbey f172e18087 Publish find-0.0.3.tgz 2026-01-18 22:25:42 +00:00
mascarpon3 e339a0af23 Publish find-0.0.3.tgz 2026-01-15 16:22:05 +00:00
mascarpon3 de2015490c Publish find-0.0.3.tgz 2026-01-15 14:18:43 +00:00
qbey 21b686b78a Publish find-0.0.3.tgz 2026-01-14 13:57:42 +00:00
qbey 40f57ce8d3 Publish find-0.0.3.tgz 2026-01-13 23:00:25 +00:00
qbey f773a67a8c Publish find-0.0.3.tgz 2026-01-12 21:04:03 +00:00
qbey 0217e2548f Publish find-0.0.3.tgz 2026-01-12 20:50:33 +00:00
qbey dae460367e Publish find-0.0.3.tgz 2026-01-12 11:54:51 +00:00
qbey ad954ebb92 Publish find-0.0.3.tgz 2026-01-09 21:38:23 +00:00
qbey dc5c2037c3 Publish find-0.0.3.tgz 2026-01-09 21:30:45 +00:00
qbey a601365925 Publish find-0.0.3.tgz 2026-01-09 19:51:04 +00:00
qbey 97a04dcfef Publish find-0.0.3.tgz 2026-01-08 17:20:13 +00:00
qbey 4f71ffb56a Publish find-0.0.3.tgz 2026-01-08 16:56:20 +00:00
mascarpon3 d30d69fc7e Publish find-0.0.2.tgz 2025-12-16 18:38:32 +00:00
mascarpon3 f2c447aac6 Publish find-0.0.2.tgz 2025-12-08 14:35:30 +00:00
mascarpon3 c488bfb849 Publish find-0.0.2.tgz 2025-12-08 09:08:35 +00:00
mascarpon3 e0f65be106 Publish find-0.0.2.tgz 2025-12-02 08:35:45 +00:00
joehybird a1e181c13b Publish find-0.0.2.tgz 2025-11-13 09:41:27 +00:00
joehybird ea4d65ee28 Publish find-0.0.2.tgz 2025-11-13 09:26:50 +00:00
joehybird 67a488ef99 Publish find-0.0.2.tgz 2025-11-12 10:09:36 +00:00
joehybird 0eecdd0386 Publish find-0.0.2.tgz 2025-11-12 09:58:33 +00:00
mascarpon3 cc07ea6277 Publish find-0.0.2.tgz 2025-11-10 10:57:37 +00:00
joehybird 06098e0238 Publish find-0.0.2.tgz 2025-10-30 14:00:15 +00:00
lunika 4ef8fa267d Publish find-0.0.2.tgz 2025-01-27 10:28:05 +00:00
lunika 87a92f9383 Publish find-0.0.1.tgz 2025-01-27 10:10:29 +00:00
lunika 0e542197c5 Publish find-0.0.1.tgz 2025-01-15 10:54:30 +00:00
lunika 159a8bb1eb Publish find-0.0.1.tgz 2025-01-14 09:24:35 +00:00
lunika d60c98d830 Publish find-0.0.1.tgz 2025-01-13 09:49:44 +01:00
108 changed files with 28 additions and 6630 deletions
-36
View File
@@ -1,36 +0,0 @@
# Python
__pycache__
*.pyc
**/__pycache__
**/*.pyc
venv
.venv
# System-specific files
.DS_Store
**/.DS_Store
# Docker
docker compose.*
env.d
# Docs
docs
*.md
*.log
# Development/test cache & configurations
data
.cache
.circleci
.git
.vscode
.iml
.idea
db.sqlite3
.mypy_cache
.pylint.d
.pytest_cache
# Frontend
node_modules
-6
View File
@@ -1,6 +0,0 @@
<!---
Thanks for filing an issue 😄 ! Before you submit, please read the following:
Check the other issue templates if you are trying to submit a bug report, feature request, or question
Search open/closed issues before submitting since someone might have asked the same thing before!
-->
-28
View File
@@ -1,28 +0,0 @@
---
name: 🐛 Bug Report
about: If something is not working as expected 🤔.
---
## Bug Report
**Problematic behavior**
A clear and concise description of the behavior.
**Expected behavior/code**
A clear and concise description of what you expected to happen (or code).
**Steps to Reproduce**
1. Do this...
2. Then this...
3. And then the bug happens!
**Environment**
- Drive version:
- Platform:
**Possible Solution**
<!--- Only if you have suggestions on a fix for the bug -->
**Additional context/Screenshots**
Add any other context about the problem here. If applicable, add screenshots to help explain.
-23
View File
@@ -1,23 +0,0 @@
---
name: ✨ Feature Request
about: I have a suggestion (and may want to build it 💪)!
---
## Feature Request
**Is your feature request related to a problem or unsupported use case? Please describe.**
A clear and concise description of what the problem is. For example: I need to do some task and I have an issue...
**Describe the solution you'd like**
A clear and concise description of what you want to happen. Add any considered drawbacks.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Discovery, Documentation, Adoption, Migration Strategy**
If you can, explain how users will be able to use this and possibly write out a version the docs (if applicable).
Maybe a screenshot or design?
**Do you want to work on it through a Pull Request?**
<!-- Make sure to coordinate with us before you spend too much time working on an implementation! -->
@@ -1,22 +0,0 @@
---
name: 🤗 Support Question
about: If you have a question 💬, or something was not clear from the docs!
---
<!-- ^ Click "Preview" for a nicer view! ^
We primarily use GitHub as an issue tracker. If however you're encountering an issue not covered in the docs, we may be able to help! -->
---
Please make sure you have read our [main Readme](https://github.com/suitenumerique/find).
Also make sure it was not already answered in [an open or close issue](https://github.com/suitenumerique/find/issues).
If your question was not covered, and you feel like it should be, fire away! We'd love to improve our docs! 👌
**Topic**
What's the general area of your question: for example, docker setup, database schema, search functionality,...
**Question**
Try to be as specific as possible so we can help you as best we can. Please be patient 🙏
-11
View File
@@ -1,11 +0,0 @@
## Purpose
Description...
## Proposal
Description...
- [] item 1...
- [] item 2...
-52
View File
@@ -1,52 +0,0 @@
name: Deploy
on:
push:
tags:
- 'preprod'
- 'production'
jobs:
notify-argocd:
runs-on: ubuntu-latest
steps:
-
uses: actions/create-github-app-token@v1
id: app-token
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: "drive,secrets"
-
name: Checkout repository
uses: actions/checkout@v2
with:
submodules: recursive
token: ${{ steps.app-token.outputs.token }}
-
name: Load sops secrets
uses: rouja/actions-sops@main
with:
secret-file: secrets/numerique-gouv/drive/secrets.enc.env
age-key: ${{ secrets.SOPS_PRIVATE }}
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/'$GITHUB_REPOSITORY'"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_WEBHOOK_URL
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_PRODUCTION_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_PRODUCTION_WEBHOOK_URL
start-test-on-preprod:
needs:
- notify-argocd
runs-on: ubuntu-latest
if: startsWith(github.event.ref, 'refs/tags/preprod')
steps:
-
name: Debug
run: |
echo "Start test when preprod is ready"
-67
View File
@@ -1,67 +0,0 @@
name: Docker Hub Workflow
on:
workflow_dispatch:
push:
branches:
- 'main'
tags:
- 'v*'
pull_request:
branches:
- 'main'
env:
DOCKER_USER: 1001:127
jobs:
build-and-push-backend:
runs-on: ubuntu-latest
steps:
-
name: Checkout repository
uses: actions/checkout@v4
-
name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: lasuite/find
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
run: echo "${{ secrets.DOCKER_HUB_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_HUB_USER }}" --password-stdin
-
name: Run trivy scan
uses: numerique-gouv/action-trivy-cache@main
with:
docker-build-args: '--target backend-production -f Dockerfile'
docker-image-name: 'docker.io/lasuite/find:${{ github.sha }}'
continue-on-error: true
-
name: Build and push
uses: docker/build-push-action@v6
with:
context: .
target: backend-production
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
notify-argocd:
needs:
- build-and-push-backend
runs-on: ubuntu-latest
if: |
github.event_name != 'pull_request'
steps:
-
name: Checkout repository
uses: actions/checkout@v4
-
name: Call argocd github webhook
run: |
data='{"ref": "'$GITHUB_REF'","repository": {"html_url":"'$GITHUB_SERVER_URL'/'$GITHUB_REPOSITORY'"}}'
sig=$(echo -n ${data} | openssl dgst -sha1 -hmac ''${ARGOCD_WEBHOOK_SECRET}'' | awk '{print "X-Hub-Signature: sha1="$2}')
curl -X POST -H 'X-GitHub-Event:push' -H "Content-Type: application/json" -H "${sig}" --data "${data}" $ARGOCD_WEBHOOK_URL
-142
View File
@@ -1,142 +0,0 @@
name: Main Workflow
on:
push:
branches:
- main
pull_request:
branches:
- "*"
jobs:
lint-git:
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' # Makes sense only for pull requests
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
fetch-depth: 0
- name: show
run: git log
- name: Enforce absence of print statements in code
run: |
! git diff origin/${{ github.event.pull_request.base.ref }}..HEAD -- . ':(exclude)**/find.yml' | grep "print("
- name: Check absence of fixup commits
run: |
! git log | grep 'fixup!'
- name: Install gitlint
run: pip install --user requests gitlint
- name: Lint commit messages added to main
run: ~/.local/bin/gitlint --commits origin/${{ github.event.pull_request.base.ref }}..HEAD
check-changelog:
runs-on: ubuntu-latest
if: |
contains(github.event.pull_request.labels.*.name, 'noChangeLog') == false &&
github.event_name == 'pull_request'
steps:
- name: Checkout repository
uses: actions/checkout@v3
with:
fetch-depth: 50
- name: Check that the CHANGELOG has been modified in the current branch
run: git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.event.after }} | grep 'CHANGELOG.md'
lint-changelog:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Check CHANGELOG max line length
run: |
max_line_length=$(cat CHANGELOG.md | grep -Ev "^\[.*\]: https://github.com" | wc -L)
if [ $max_line_length -ge 80 ]; then
echo "ERROR: CHANGELOG has lines longer than 80 characters."
exit 1
fi
lint-back:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/backend
steps:
- name: Checkout repository
uses: actions/checkout@v2
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.12"
- name: Install development dependencies
run: pip install --user .[dev]
- name: Check code formatting with ruff
run: ~/.local/bin/ruff format . --diff
- name: Lint code with ruff
run: ~/.local/bin/ruff check .
- name: Lint code with pylint
run: ~/.local/bin/pylint .
test-back:
runs-on: ubuntu-latest
defaults:
run:
working-directory: src/backend
services:
postgres:
image: postgres:15
env:
POSTGRES_DB: find
POSTGRES_USER: dinum
POSTGRES_PASSWORD: pass
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
opensearch:
image: opensearchproject/opensearch:latest
env:
discovery.type: single-node
plugins.security.disabled: true
plugins.security.ssl.http.enabled: false
OPENSEARCH_INITIAL_ADMIN_PASSWORD: find.PASS123
ports:
- 9200:9200
env:
DJANGO_CONFIGURATION: Test
DJANGO_SETTINGS_MODULE: find.settings
DJANGO_SECRET_KEY: ThisIsAnExampleKeyForTestPurposeOnly
OIDC_OP_JWKS_ENDPOINT: /endpoint-for-test-purpose-only
DB_HOST: localhost
DB_NAME: find
DB_USER: dinum
DB_PASSWORD: pass
DB_PORT: 5432
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
OPENSEARCH_HOST: localhost
OPENSEARCH_PASSWORD: find.PASS123
OPENSEARCH_USE_SSL: false
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Create writable /data
run: |
sudo mkdir -p /data/media && \
sudo mkdir -p /data/static
- name: Install Python
uses: actions/setup-python@v3
with:
python-version: "3.12"
- name: Install development dependencies
run: pip install --user .[dev]
- name: Run tests
run: ~/.local/bin/pytest
-81
View File
@@ -1,81 +0,0 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
.DS_Store
.next/
# Translations # Translations
*.pot
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
env.d/development/*
!env.d/development/*.dist
env.d/terraform
# npm
node_modules
# Mails
src/backend/core/templates/mail/
# Typescript client
src/frontend/tsclient
# Swagger
**/swagger.json
# Logs
*.log
# Terraform
.terraform
*.tfstate
*.tfstate.backup
# Test & lint
.coverage
.pylint.d
.pytest_cache
db.sqlite3
.mypy_cache
# Site media
/data/
# IDEs
.idea/
.vscode/
*.iml
.devcontainer
-78
View File
@@ -1,78 +0,0 @@
# All these sections are optional, edit this file as you like.
[general]
# Ignore certain rules, you can reference them by their id or by their full name
# ignore=title-trailing-punctuation, T3
# verbosity should be a value between 1 and 3, the commandline -v flags take precedence over this
# verbosity = 2
# By default gitlint will ignore merge commits. Set to 'false' to disable.
# ignore-merge-commits=true
# By default gitlint will ignore fixup commits. Set to 'false' to disable.
# ignore-fixup-commits=true
# By default gitlint will ignore squash commits. Set to 'false' to disable.
# ignore-squash-commits=true
# Enable debug mode (prints more output). Disabled by default.
# debug=true
# Set the extra-path where gitlint will search for user defined rules
# See http://jorisroovers.github.io/gitlint/user_defined_rules for details
extra-path=gitlint/
# [title-max-length]
# line-length=80
[title-must-not-contain-word]
# Comma-separated list of words that should not occur in the title. Matching is case
# insensitive. It's fine if the keyword occurs as part of a larger word (so "WIPING"
# will not cause a violation, but "WIP: my title" will.
words=wip
#[title-match-regex]
# python like regex (https://docs.python.org/2/library/re.html) that the
# commit-msg title must be matched to.
# Note that the regex can contradict with other rules if not used correctly
# (e.g. title-must-not-contain-word).
#regex=
# [B1]
# B1 = body-max-line-length
# line-length=120
# [body-min-length]
# min-length=5
# [body-is-missing]
# Whether to ignore this rule on merge commits (which typically only have a title)
# default = True
# ignore-merge-commits=false
# [body-changed-file-mention]
# List of files that need to be explicitly mentioned in the body when they are changed
# This is useful for when developers often erroneously edit certain files or git submodules.
# By specifying this rule, developers can only change the file when they explicitly reference
# it in the commit message.
# files=gitlint/rules.py,README.md
# [author-valid-email]
# python like regex (https://docs.python.org/2/library/re.html) that the
# commit author email address should be matched to
# For example, use the following regex if you only want to allow email addresses from foo.com
# regex=[^@]+@foo.com
[ignore-by-title]
# Allow empty body & wrong title pattern only when bots (pyup/greenkeeper)
# upgrade dependencies
regex=^(⬆️.*|Update (.*) from (.*) to (.*)|(chore|fix)\(package\): update .*)$
ignore=B6,UC1
# [ignore-by-body]
# Ignore certain rules for commits of which the body has a line that matches a regex
# E.g. Match bodies that have a line that that contain "release"
# regex=(.*)release(.*)
#
# Ignore certain rules, you can reference them by their id or by their full name
# Use 'all' to ignore all rules
# ignore=T1,body-min-length
-3
View File
@@ -1,3 +0,0 @@
[submodule "secrets"]
path = secrets
url = ../secrets
-7
View File
@@ -1,7 +0,0 @@
# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0),
and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
-137
View File
@@ -1,137 +0,0 @@
# Django find
# ---- base image to inherit from ----
FROM python:3.12-slim-bookworm AS base
# Upgrade pip to its latest release to speed up dependencies installation
RUN python -m pip install --upgrade pip
# Upgrade system packages to install security updates
RUN apt-get update && \
apt-get -y upgrade && \
rm -rf /var/lib/apt/lists/*
# ---- Back-end builder image ----
FROM base AS back-builder
WORKDIR /builder
# Copy required python dependencies
COPY ./src/backend /builder
RUN mkdir /install && \
pip install --prefix=/install .
# ---- static link collector ----
FROM base AS link-collector
ARG FIND_STATIC_ROOT=/data/static
# Install libpangocairo & rdfind
RUN apt-get update && \
apt-get install -y \
libpangocairo-1.0-0 \
rdfind && \
rm -rf /var/lib/apt/lists/*
# Copy installed python dependencies
COPY --from=back-builder /install /usr/local
# Copy find application (see .dockerignore)
COPY ./src/backend /app/
WORKDIR /app
# collectstatic
RUN DJANGO_CONFIGURATION=Build \
DJANGO_JWT_PRIVATE_SIGNING_KEY=Dummy \
OPENSEARCH_PASSWORD=Dummy \
python manage.py collectstatic --noinput
# Replace duplicated file by a symlink to decrease the overall size of the
# final image
RUN rdfind -makesymlinks true -followsymlinks true -makeresultsfile false ${FIND_STATIC_ROOT}
# ---- Core application image ----
FROM base AS core
ENV PYTHONUNBUFFERED=1
# Install required system libs
RUN apt-get update && \
apt-get install -y \
gettext \
libcairo2 \
libffi-dev \
libgdk-pixbuf2.0-0 \
libpango-1.0-0 \
libpangocairo-1.0-0 \
shared-mime-info && \
rm -rf /var/lib/apt/lists/*
# Copy entrypoint
COPY ./docker/files/usr/local/bin/entrypoint /usr/local/bin/entrypoint
# Give the "root" group the same permissions as the "root" user on /etc/passwd
# to allow a user belonging to the root group to add new users; typically the
# docker user (see entrypoint).
RUN chmod g=u /etc/passwd
# Copy installed python dependencies
COPY --from=back-builder /install /usr/local
# Copy find application (see .dockerignore)
COPY ./src/backend /app/
WORKDIR /app
# We wrap commands run in this container by the following entrypoint that
# creates a user on-the-fly with the container user ID (see USER) and root group
# ID.
ENTRYPOINT [ "/usr/local/bin/entrypoint" ]
# ---- Development image ----
FROM core AS backend-development
# Switch back to the root user to install development dependencies
USER root:root
# Install psql
RUN apt-get update && \
apt-get install -y postgresql-client && \
rm -rf /var/lib/apt/lists/*
# Uninstall find and re-install it in editable mode along with development
# dependencies
RUN pip uninstall -y find
RUN pip install -e .[dev]
# Restore the un-privileged user running the application
ARG DOCKER_USER
USER ${DOCKER_USER}
# Target database host (e.g. database engine following docker compose services
# name) & port
ENV DB_HOST=postgresql \
DB_PORT=5432
# Run django development server
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
# ---- Production image ----
FROM core AS backend-production
ARG FIND_STATIC_ROOT=/data/static
# Gunicorn
RUN mkdir -p /usr/local/etc/gunicorn
COPY docker/files/usr/local/etc/gunicorn/find.py /usr/local/etc/gunicorn/find.py
# Un-privileged user running the application
ARG DOCKER_USER
USER ${DOCKER_USER}
# Copy statics
COPY --from=link-collector ${FIND_STATIC_ROOT} ${FIND_STATIC_ROOT}
# The default command runs gunicorn WSGI server in find's main module
CMD ["gunicorn", "-c", "/usr/local/etc/gunicorn/find.py", "find.wsgi:application"]
-21
View File
@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2024 Direction Interministérielle du Numérique - Gouvernement Français
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
-264
View File
@@ -1,264 +0,0 @@
# /!\ /!\ /!\ /!\ /!\ /!\ /!\ DISCLAIMER /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\
#
# This Makefile is only meant to be used for DEVELOPMENT purpose as we are
# changing the user id that will run in the container.
#
# PLEASE DO NOT USE IT FOR YOUR CI/PRODUCTION/WHATEVER...
#
# /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\
#
# Note to developers:
#
# While editing this file, please respect the following statements:
#
# 1. Every variable should be defined in the ad hoc VARIABLES section with a
# relevant subsection
# 2. Every new rule should be defined in the ad hoc RULES section with a
# relevant subsection depending on the targeted service
# 3. Rules should be sorted alphabetically within their section
# 4. When a rule has multiple dependencies, you should:
# - duplicate the rule name to add the help string (if required)
# - write one dependency per line to increase readability and diffs
# 5. .PHONY rule statement should be written after the corresponding rule
# ==============================================================================
# VARIABLES
BOLD := \033[1m
RESET := \033[0m
GREEN := \033[1;32m
# -- Database
DB_HOST = postgresql
DB_PORT = 5432
# -- Docker
# Get the current user ID to use for docker run and docker exec commands
DOCKER_UID = $(shell id -u)
DOCKER_GID = $(shell id -g)
DOCKER_USER = $(DOCKER_UID):$(DOCKER_GID)
COMPOSE = DOCKER_USER=$(DOCKER_USER) docker compose
COMPOSE_EXEC = $(COMPOSE) exec
COMPOSE_EXEC_APP = $(COMPOSE_EXEC) app
COMPOSE_RUN = $(COMPOSE) run --rm
COMPOSE_RUN_APP = $(COMPOSE_RUN) app
COMPOSE_RUN_CROWDIN = $(COMPOSE_RUN) crowdin crowdin
WAIT_DB = @$(COMPOSE_RUN) dockerize -wait tcp://$(DB_HOST):$(DB_PORT) -timeout 60s
# -- Backend
MANAGE = $(COMPOSE_RUN_APP) python manage.py
# ==============================================================================
# RULES
default: help
data/opensearch:
@mkdir -p data/opensearch
data/static:
@mkdir -p data/static
# -- Project
create-env-files: ## Copy the dist env files to env files
create-env-files: \
env.d/development/common \
env.d/development/crowdin \
env.d/development/postgresql
.PHONY: create-env-files
bootstrap: ## Prepare Docker images for the project
bootstrap: \
data/opensearch \
data/static \
create-env-files \
build \
migrate \
demo \
back-i18n-generate \
back-i18n-compile
.PHONY: bootstrap
# -- Docker/compose
build: ## build the app container
@$(COMPOSE) build app --no-cache
.PHONY: build
down: ## stop and remove containers, networks, images, and volumes
@$(COMPOSE) down
.PHONY: down
logs: ## display app logs (follow mode)
@$(COMPOSE) logs -f app
.PHONY: logs
run: ## start the wsgi (production) and development server
@$(COMPOSE) up --force-recreate -d celery
@echo "Wait for postgresql to be up..."
@$(WAIT_DB)
.PHONY: run
status: ## an alias for "docker compose ps"
@$(COMPOSE) ps
.PHONY: status
stop: ## stop the development server using Docker
@$(COMPOSE) stop
.PHONY: stop
# -- Backend
demo: ## flush db then create a demo for load testing purpose
@$(MAKE) resetdb
@$(MANAGE) create_demo
.PHONY: demo
# Nota bene: Black should come after isort just in case they don't agree...
lint: ## lint back-end python sources
lint: \
lint-ruff-format \
lint-ruff-check \
lint-pylint
.PHONY: lint
lint-ruff-format: ## format back-end python sources with ruff
@echo 'lint:ruff-format started…'
@$(COMPOSE_RUN_APP) ruff format .
.PHONY: lint-ruff-format
lint-ruff-check: ## lint back-end python sources with ruff
@echo 'lint:ruff-check started…'
@$(COMPOSE_RUN_APP) ruff check . --fix
.PHONY: lint-ruff-check
lint-pylint: ## lint back-end python sources with pylint only on changed files from main
@echo 'lint:pylint started…'
bin/pylint .
.PHONY: lint-pylint
test: ## run project tests
@$(MAKE) test-back-parallel
.PHONY: test
test-back: ## run back-end tests
@args="$(filter-out $@,$(MAKECMDGOALS))" && \
bin/pytest $${args:-${1}}
.PHONY: test-back
test-back-parallel: ## run all back-end tests in parallel
@args="$(filter-out $@,$(MAKECMDGOALS))" && \
bin/pytest -n auto $${args:-${1}}
.PHONY: test-back-parallel
makemigrations: ## run django makemigrations for the find project.
@echo "$(BOLD)Running makemigrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) makemigrations
.PHONY: makemigrations
migrate: ## run django migrations for the find project.
@echo "$(BOLD)Running migrations$(RESET)"
@$(COMPOSE) up -d postgresql
@$(WAIT_DB)
@$(MANAGE) migrate
.PHONY: migrate
superuser: ## Create an admin superuser with password "admin"
@echo "$(BOLD)Creating a Django superuser$(RESET)"
@$(WAIT_DB)
@$(MANAGE) shell -c "from core.models import User; not User.objects.filter(username='admin').exists() and User.objects.create_superuser('admin', 'admin@example.com', 'admin')"
.PHONY: superuser
back-i18n-compile: ## compile the gettext files
@$(MANAGE) compilemessages --ignore="venv/**/*"
.PHONY: back-i18n-compile
back-i18n-generate: ## create the .pot files used for i18n
@$(MANAGE) makemessages -a --keep-pot
.PHONY: back-i18n-generate
shell: ## connect to database shell
@$(MANAGE) shell #_plus
.PHONY: dbshell
# -- Database
dbshell: ## connect to database shell
docker compose exec app python manage.py dbshell
.PHONY: dbshell
resetdb: FLUSH_ARGS ?=
resetdb: ## flush database and create a superuser "admin"
@echo "$(BOLD)Flush database$(RESET)"
@$(MANAGE) flush $(FLUSH_ARGS)
@${MAKE} superuser
.PHONY: resetdb
env.d/development/common:
cp --update=none env.d/development/common.dist env.d/development/common
env.d/development/postgresql:
cp --update=none env.d/development/postgresql.dist env.d/development/postgresql
# -- Internationalization
env.d/development/crowdin:
cp --update=none env.d/development/crowdin.dist env.d/development/crowdin
crowdin-download: ## Download translated message from crowdin
@$(COMPOSE_RUN_CROWDIN) download -c crowdin/config.yml
.PHONY: crowdin-download
crowdin-download-sources: ## Download sources from Crowdin
@$(COMPOSE_RUN_CROWDIN) download sources -c crowdin/config.yml
.PHONY: crowdin-download-sources
crowdin-upload: ## Upload source translations to crowdin
@$(COMPOSE_RUN_CROWDIN) upload sources -c crowdin/config.yml
.PHONY: crowdin-upload
i18n-compile: ## compile all translations
i18n-compile: \
back-i18n-compile
.PHONY: i18n-compile
i18n-generate: ## create the .pot files used for i18n
i18n-generate: \
back-i18n-generate
.PHONY: i18n-generate
i18n-download-and-compile: ## download all translated messages and compile them to be used by all applications
i18n-download-and-compile: \
crowdin-download \
i18n-compile
.PHONY: i18n-download-and-compile
i18n-generate-and-upload: ## generate source translations for all applications and upload them to Crowdin
i18n-generate-and-upload: \
i18n-generate \
crowdin-upload
.PHONY: i18n-generate-and-upload
# -- Misc
clean: ## restore repository state as it was freshly cloned
git clean -idx
.PHONY: clean
help:
@echo "$(BOLD)find Makefile"
@echo "Please use 'make $(BOLD)target$(RESET)' where $(BOLD)target$(RESET) is one of:"
@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(firstword $(MAKEFILE_LIST)) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "$(GREEN)%-30s$(RESET) %s\n", $$1, $$2}'
.PHONY: help
# -- K8S
build-k8s-cluster: ## build the kubernetes cluster using kind
./bin/start-kind.sh
.PHONY: build-k8s-cluster
start-tilt: ## start the kubernetes cluster using kind
tilt up -f ./bin/Tiltfile
.PHONY: build-k8s-cluster
-75
View File
@@ -1,75 +0,0 @@
# Find
Find can index documents from several applications sharing a common OIDC federation
and allows users to search documents with their access rights accross all applications
in the federation.
Find is built on top of [Django Rest
Framework](https://www.django-rest-framework.org/).
## Getting started
### Prerequisite
Make sure you have a recent version of Docker and [Docker
Compose](https://docs.docker.com/compose/install) installed on your laptop:
```bash
$ docker -v
Docker version 27.4.1, build b9d17ea
$ docker compose version
Docker Compose version v2.32.1
```
> ⚠️ You may need to run the following commands with `sudo` but this can be
> avoided by assigning your user to the `docker` group. See docker
> [Documentation](https://docs.docker.com/engine/install/linux-postinstall/)
### Project bootstrap
The easiest way to start working on the project is to use GNU Make:
```bash
$ make bootstrap
```
This command builds the `app` container, installs dependencies, performs
database migrations and compile translations. It's a good idea to use this
command each time you are pulling code from the project repository to avoid
dependency-releated or migration-releated issues.
Your Docker services should now be up and running 🎉
### Adding content
You can create a basic demo site by running:
$ make demo
Finally, you can check all available Make rules using:
```bash
$ make help
```
### Django admin
You can access the Django admin site at
[http://localhost:8071/admin](http://localhost:8071/admin).
You first need to create a superuser account:
```bash
$ make superuser
```
## Contributing
This project is intended to be community-driven, so please, do not hesitate to
get in touch if you have any question related to our implementation or design
decisions.
## License
This work is released under the MIT License (see [LICENSE](./LICENSE)).
-17
View File
@@ -1,17 +0,0 @@
# Upgrade
All instructions to upgrade this project from one release to the next will be
documented in this file. Upgrades must be run sequentially, meaning you should
not skip minor/major releases while upgrading (fix releases can be skipped).
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
For most upgrades, you just need to run the django migrations with
the following command inside your docker container:
`python manage.py migrate`
(Note : in your development environment, you can `make migrate`.)
## [Unreleased]
-47
View File
@@ -1,47 +0,0 @@
load('ext://uibutton', 'cmd_button', 'bool_input', 'location')
load('ext://namespace', 'namespace_create', 'namespace_inject')
namespace_create('find')
docker_build(
'localhost:5001/find-backend:latest',
context='..',
dockerfile='../Dockerfile',
only=['./src/backend', './docker'],
target = 'backend-production',
live_update=[
sync('../src/backend', '/app'),
run(
'pip install .',
trigger=['../src/backend/pyproject.toml']
)
]
)
k8s_yaml(local('cd ../src/helm && helmfile -n find -e dev template .'))
migrate_db = '''
set -eu
# get k8s pod name from tilt resource name
POD_NAME="$(tilt get kubernetesdiscovery find-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n find exec "$POD_NAME" -- python manage.py migrate --no-input
'''
cmd_button('Migrate db',
argv=['sh', '-c', migrate_db],
resource='find-backend',
icon_name='developer_board',
text='Run database migration',
)
create_demo = '''
set -eu
# get k8s pod name from tilt resource name
POD_NAME="$(tilt get kubernetesdiscovery find-backend -ojsonpath='{.status.pods[0].name}')"
kubectl -n find exec "$POD_NAME" -- python manage.py create_demo --force
'''
cmd_button('Create demo fixtures',
argv=['sh', '-c', create_demo],
resource='find-backend',
icon_name='developer_board',
text='Run demo fixtures',
)
-93
View File
@@ -1,93 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
REPO_DIR="$(cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd)"
UNSET_USER=0
COMPOSE_FILE="${REPO_DIR}/compose.yml"
COMPOSE_PROJECT="find"
# _set_user: set (or unset) default user id used to run docker commands
#
# usage: _set_user
#
# You can override default user ID (the current host user ID), by defining the
# USER_ID environment variable.
#
# To avoid running docker commands with a custom user, please set the
# $UNSET_USER environment variable to 1.
function _set_user() {
if [ $UNSET_USER -eq 1 ]; then
USER_ID=""
return
fi
# USER_ID = USER_ID or `id -u` if USER_ID is not set
USER_ID=${USER_ID:-$(id -u)}
echo "🙋(user) ID: ${USER_ID}"
}
# docker_compose: wrap docker compose command
#
# usage: docker_compose [options] [ARGS...]
#
# options: docker compose command options
# ARGS : docker compose command arguments
function _docker_compose() {
echo "🐳(compose) project: '${COMPOSE_PROJECT}' file: '${COMPOSE_FILE}'"
docker compose \
-p "${COMPOSE_PROJECT}" \
-f "${COMPOSE_FILE}" \
--project-directory "${REPO_DIR}" \
"$@"
}
# _dc_run: wrap docker compose run command
#
# usage: _dc_run [options] [ARGS...]
#
# options: docker compose run command options
# ARGS : docker compose run command arguments
function _dc_run() {
_set_user
user_args="--user=$USER_ID"
if [ -z $USER_ID ]; then
user_args=""
fi
_docker_compose run --rm $user_args "$@"
}
# _dc_exec: wrap docker compose exec command
#
# usage: _dc_exec [options] [ARGS...]
#
# options: docker compose exec command options
# ARGS : docker compose exec command arguments
function _dc_exec() {
_set_user
echo "🐳(compose) exec command: '\$@'"
user_args="--user=$USER_ID"
if [ -z $USER_ID ]; then
user_args=""
fi
_docker_compose exec $user_args "$@"
}
# _django_manage: wrap django's manage.py command with docker compose
#
# usage : _django_manage [ARGS...]
#
# ARGS : django's manage.py command arguments
function _django_manage() {
_dc_run app python manage.py "$@"
}
-6
View File
@@ -1,6 +0,0 @@
#!/usr/bin/env bash
# shellcheck source=bin/_config.sh
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
_docker_compose "$@"
-30
View File
@@ -1,30 +0,0 @@
#!/bin/bash
mkdir -p "$(dirname -- "${BASH_SOURCE[0]}")/../.git/hooks/"
PRE_COMMIT_FILE="$(dirname -- "${BASH_SOURCE[0]}")/../.git/hooks/pre-commit"
cat <<'EOF' >$PRE_COMMIT_FILE
#!/bin/bash
# directories containing potential secrets
DIRS="."
bold=$(tput bold)
normal=$(tput sgr0)
# allow to read user input, assigns stdin to keyboard
exec </dev/tty
for d in $DIRS; do
# find files containing secrets that should be encrypted
for f in $(find "${d}" -type f -regex ".*\.enc\..*"); do
if ! $(grep -q "unencrypted_suffix" $f); then
printf '\xF0\x9F\x92\xA5 '
echo "File $f has non encrypted secrets!"
exit 1
fi
done
done
EOF
chmod +x $PRE_COMMIT_FILE
-6
View File
@@ -1,6 +0,0 @@
#!/usr/bin/env bash
# shellcheck source=bin/_config.sh
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
_django_manage "$@"
-6
View File
@@ -1,6 +0,0 @@
#!/bin/sh
set -o errexit
echo "Pausing docker containers running k8s cluster managed with kind"
docker pause $(docker ps --filter name=kind-* --filter status=running --format "{{.Names}}")
-38
View File
@@ -1,38 +0,0 @@
#!/usr/bin/env bash
# shellcheck source=bin/_config.sh
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
declare diff_from
declare -a paths
declare -a args
# Parse options
for arg in "$@"
do
case $arg in
--diff-only=*)
diff_from="${arg#*=}"
shift
;;
-*)
args+=("$arg")
shift
;;
*)
paths+=("$arg")
shift
;;
esac
done
if [[ -n "${diff_from}" ]]; then
# Run pylint only on modified files located in src/backend
# (excluding deleted files and migration files)
# shellcheck disable=SC2207
paths=($(git diff "${diff_from}" --name-only --diff-filter=d -- src/backend ':!**/migrations/*.py' | grep -E '^src/backend/.*\.py$'))
fi
# Fix docker vs local path when project sources are mounted as a volume
read -ra paths <<< "$(echo "${paths[@]}" | sed "s|src/backend/||g")"
_dc_run app pylint "${paths[@]}" "${args[@]}"
-8
View File
@@ -1,8 +0,0 @@
#!/usr/bin/env bash
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
_dc_run \
-e DJANGO_CONFIGURATION=Test \
app \
pytest "$@"
-6
View File
@@ -1,6 +0,0 @@
#!/bin/sh
set -o errexit
echo "Resuming docker containers running k8s cluster managed with kind"
docker unpause $(docker ps --filter name=kind-* --filter status=paused --format "{{.Names}}")
-103
View File
@@ -1,103 +0,0 @@
#!/bin/sh
set -o errexit
CURRENT_DIR=$(pwd)
echo "0. Create ca"
# 0. Create ca
mkcert -install
cd /tmp
mkcert "127.0.0.1.nip.io" "*.127.0.0.1.nip.io"
cd $CURRENT_DIR
echo "1. Create registry container unless it already exists"
# 1. Create registry container unless it already exists
reg_name='kind-registry'
reg_port='5001'
if [ "$(docker inspect -f '{{.State.Running}}' "${reg_name}" 2>/dev/null || true)" != 'true' ]; then
docker run \
-d --restart=always -p "127.0.0.1:${reg_port}:5000" --network bridge --name "${reg_name}" \
registry:2
fi
echo "2. Create kind cluster with containerd registry config dir enabled"
# 2. Create kind cluster with containerd registry config dir enabled
# TODO: kind will eventually enable this by default and this patch will
# be unnecessary.
#
# See:
# https://github.com/kubernetes-sigs/kind/issues/2875
# https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration
# See: https://github.com/containerd/containerd/blob/main/docs/hosts.md
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"
nodes:
- role: control-plane
image: kindest/node:v1.27.3
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
extraPortMappings:
- containerPort: 80
hostPort: 80
protocol: TCP
- containerPort: 443
hostPort: 443
protocol: TCP
- role: worker
image: kindest/node:v1.27.3
- role: worker
image: kindest/node:v1.27.3
EOF
echo "3. Add the registry config to the nodes"
# 3. Add the registry config to the nodes
#
# This is necessary because localhost resolves to loopback addresses that are
# network-namespace local.
# In other words: localhost in the container is not localhost on the host.
#
# We want a consistent name that works from both ends, so we tell containerd to
# alias localhost:${reg_port} to the registry container when pulling images
REGISTRY_DIR="/etc/containerd/certs.d/localhost:${reg_port}"
for node in $(kind get nodes); do
docker exec "${node}" mkdir -p "${REGISTRY_DIR}"
cat <<EOF | docker exec -i "${node}" cp /dev/stdin "${REGISTRY_DIR}/hosts.toml"
[host."http://${reg_name}:5000"]
EOF
done
echo "4. Connect the registry to the cluster network if not already connected"
# 4. Connect the registry to the cluster network if not already connected
# This allows kind to bootstrap the network but ensures they're on the same network
if [ "$(docker inspect -f='{{json .NetworkSettings.Networks.kind}}' "${reg_name}")" = 'null' ]; then
docker network connect "kind" "${reg_name}"
fi
echo "5. Document the local registry"
# 5. Document the local registry
# https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: local-registry-hosting
namespace: kube-public
data:
localRegistryHosting.v1: |
host: "localhost:${reg_port}"
help: "https://kind.sigs.k8s.io/docs/user/local-registry/"
EOF
echo "6. Install ingress-nginx"
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
kubectl -n ingress-nginx create secret tls mkcert --key /tmp/127.0.0.1.nip.io+1-key.pem --cert /tmp/127.0.0.1.nip.io+1.pem
kubectl -n ingress-nginx patch deployments.apps ingress-nginx-controller --type 'json' -p '[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value":"--default-ssl-certificate=ingress-nginx/mkcert"}]'
-25
View File
@@ -1,25 +0,0 @@
#!/usr/bin/env bash
set -eo pipefail
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
project=$(_set_openstack_project)
echo "Using \"${project}\" project..."
source "${TERRAFORM_DIRECTORY}/${project}/openrc.sh"
# Run Terraform commands in the Hashicorp docker container via docker compose
# shellcheck disable=SC2068
DOCKER_USER="$(id -u):$(id -g)" \
PROJECT="${project}" \
docker compose run --rm \
-e OS_AUTH_URL \
-e OS_IDENTITY_API_VERSION \
-e OS_USER_DOMAIN_NAME \
-e OS_PROJECT_DOMAIN_NAME \
-e OS_TENANT_ID \
-e OS_TENANT_NAME \
-e OS_USERNAME \
-e OS_PASSWORD \
-e OS_REGION_NAME \
terraform-state "$@"
-4
View File
@@ -1,4 +0,0 @@
#!/bin/bash
git submodule update --init --recursive
git submodule foreach 'git fetch origin; git checkout $(git rev-parse --abbrev-ref HEAD); git reset --hard origin/$(git rev-parse --abbrev-ref HEAD); git submodule update --recursive; git clean -dfx'
-12
View File
@@ -1,12 +0,0 @@
#!/usr/bin/env bash
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
_dc_run \
-e DJANGO_CONFIGURATION=Test \
app \
python manage.py spectacular \
--api-version 'v1.0' \
--urlconf 'find.api_urls' \
--format openapi-json \
--file /app/core/tests/swagger/swagger.json
-3
View File
@@ -1,3 +0,0 @@
#!/bin/bash
find . -name "*.enc.*" -exec sops updatekeys -y {} \;
-96
View File
@@ -1,96 +0,0 @@
services:
postgresql:
image: postgres:15
env_file:
- env.d/development/postgresql
ports:
- "15432:5432"
opensearch:
image: opensearchproject/opensearch:latest
environment:
- discovery.type=single-node
- plugins.security.disabled=true
- plugins.security.ssl.http.enabled=false
- OPENSEARCH_INITIAL_ADMIN_PASSWORD=find.PASS123
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- ./data/opensearch:/usr/share/opensearch/data
ports:
- "9200:9200"
- "9600:9600"
opensearch-dashboards:
image: opensearchproject/opensearch-dashboards:latest
ports:
- "5601:5601"
environment:
OPENSEARCH_HOSTS: '["http://opensearch:9200"]'
DISABLE_SECURITY_DASHBOARDS_PLUGIN: "true"
depends_on:
- opensearch
redis:
image: redis:5
app:
build:
context: .
target: backend-development
args:
DOCKER_USER: ${DOCKER_USER:-1000}
user: ${DOCKER_USER:-1000}
image: find:backend-development
environment:
- PYLINTHOME=/app/.pylint.d
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/postgresql
ports:
- "8071:8000"
volumes:
- ./src/backend:/app
- ./data/static:/data/static
depends_on:
- postgresql
- opensearch
- redis
celery:
user: ${DOCKER_USER:-1000}
image: find:backend-development
command: ["celery", "-A", "find.celery_app", "worker", "-l", "DEBUG"]
environment:
- DJANGO_CONFIGURATION=Development
env_file:
- env.d/development/common
- env.d/development/postgresql
volumes:
- ./src/backend:/app
- ./data/static:/data/static
depends_on:
- app
dockerize:
image: jwilder/dockerize
crowdin:
image: crowdin/cli:3.16.0
volumes:
- ".:/app"
env_file:
- env.d/development/crowdin
user: "${DOCKER_USER:-1000}"
working_dir: /app
node:
image: node:18
user: "${DOCKER_USER:-1000}"
environment:
HOME: /tmp
volumes:
- ".:/app"
-23
View File
@@ -1,23 +0,0 @@
#
# Your crowdin's credentials
#
api_token_env: CROWDIN_API_TOKEN
project_id_env: CROWDIN_PROJECT_ID
base_path_env: CROWDIN_BASE_PATH
#
# Choose file structure in crowdin
# e.g. true or false
#
preserve_hierarchy: true
#
# Files configuration
#
files: [
{
source : "/backend/locale/django.pot",
dest: "/backend-find.pot",
translation : "/backend/locale/%locale_with_underscore%/LC_MESSAGES/django.po"
},
]
-35
View File
@@ -1,35 +0,0 @@
#!/bin/sh
#
# The container user (see USER in the Dockerfile) is an un-privileged user that
# does not exists and is not created during the build phase (see Dockerfile).
# Hence, we use this entrypoint to wrap commands that will be run in the
# container to create an entry for this user in the /etc/passwd file.
#
# The following environment variables may be passed to the container to
# customize running user account:
#
# * USER_NAME: container user name (default: default)
# * HOME : container user home directory (default: none)
#
# To pass environment variables, you can either use the -e option of the docker run command:
#
# docker run --rm -e USER_NAME=foo -e HOME='/home/foo' find:latest python manage.py migrate
#
# or define new variables in an environment file to use with docker or docker compose:
#
# # env.d/production
# USER_NAME=foo
# HOME=/home/foo
#
# docker run --rm --env-file env.d/production find:latest python manage.py migrate
#
echo "🐳(entrypoint) creating user running in the container..."
if ! whoami > /dev/null 2>&1; then
if [ -w /etc/passwd ]; then
echo "${USER_NAME:-default}:x:$(id -u):$(id -g):${USER_NAME:-default} user:${HOME}:/sbin/nologin" >> /etc/passwd
fi
fi
echo "🐳(entrypoint) running your command: ${*}"
exec "$@"
@@ -1,16 +0,0 @@
# Gunicorn-django settings
bind = ["0.0.0.0:8000"]
name = "find"
python_path = "/app"
# Run
graceful_timeout = 90
timeout = 90
workers = 3
# Logging
# Using '-' for the access log file makes gunicorn log accesses to stdout
accesslog = "-"
# Using '-' for the error log file makes gunicorn log errors to stderr
errorlog = "-"
loglevel = "info"
-31
View File
@@ -1,31 +0,0 @@
# Django
DJANGO_ALLOWED_HOSTS=*
DJANGO_SECRET_KEY=ThisIsAnExampleKeyForDevPurposeOnly
DJANGO_SETTINGS_MODULE=find.settings
DJANGO_SUPERUSER_PASSWORD=admin
# Python
PYTHONPATH=/app
# find settings
# Backend url
FIND_BASE_URL="http://localhost:8072"
# Opensearch
OPENSEARCH_PASSWORD=find.PASS123
OPENSEARCH_USE_SSL=false
# OIDC
OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/find/protocol/openid-connect/certs
OIDC_OP_AUTHORIZATION_ENDPOINT=http://localhost:8083/realms/find/protocol/openid-connect/auth
OIDC_OP_TOKEN_ENDPOINT=http://nginx:8083/realms/find/protocol/openid-connect/token
OIDC_OP_USER_ENDPOINT=http://nginx:8083/realms/find/protocol/openid-connect/userinfo
OIDC_RP_CLIENT_ID=find
OIDC_RP_CLIENT_SECRET=ThisIsAnExampleKeyForDevPurposeOnly
OIDC_RP_SIGN_ALGO=RS256
OIDC_RP_SCOPES="openid email"
OIDC_REDIRECT_ALLOWED_HOSTS=["http://localhost:8083", "http://localhost:3000"]
OIDC_AUTH_REQUEST_EXTRA_PARAMS={"acr_values": "eidas1"}
-3
View File
@@ -1,3 +0,0 @@
CROWDIN_API_TOKEN=Your-Api-Token
CROWDIN_PROJECT_ID=Your-Project-Id
CROWDIN_BASE_PATH=/app/src
-11
View File
@@ -1,11 +0,0 @@
# Postgresql db container configuration
POSTGRES_DB=find
POSTGRES_USER=dinum
POSTGRES_PASSWORD=pass
# App database configuration
DB_HOST=postgresql
DB_NAME=find
DB_USER=dinum
DB_PASSWORD=pass
DB_PORT=5432
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
-37
View File
@@ -1,37 +0,0 @@
"""
Gitlint extra rule to validate that the message title is of the form
"<gitmoji>(<scope>) <subject>"
"""
from __future__ import unicode_literals
import re
import requests
from gitlint.rules import CommitMessageTitle, LineRule, RuleViolation
class GitmojiTitle(LineRule):
"""
This rule will enforce that each commit title is of the form "<gitmoji>(<scope>) <subject>"
where gitmoji is an emoji from the list defined in https://gitmoji.carloscuesta.me and
subject should be all lowercase
"""
id = "UC1"
name = "title-should-have-gitmoji-and-scope"
target = CommitMessageTitle
def validate(self, title, _commit):
"""
Download the list possible gitmojis from the project's github repository and check that
title contains one of them.
"""
gitmojis = requests.get(
"https://raw.githubusercontent.com/carloscuesta/gitmoji/master/packages/gitmojis/src/gitmojis.json"
).json()["gitmojis"]
emojis = [item["emoji"] for item in gitmojis]
pattern = r"^({:s})\(.*\)\s[a-z].*$".format("|".join(emojis))
if not re.search(pattern, title):
violation_msg = 'Title does not match regex "<gitmoji>(<scope>) <subject>"'
return [RuleViolation(self.id, violation_msg, title)]
+28
View File
@@ -0,0 +1,28 @@
apiVersion: v1
entries:
find:
- apiVersion: v2
created: "2026-03-16T08:58:33.405623957Z"
digest: 7db8aa465f4f5cf0b56d36ba9eebe0ab7d94536f6b5954362420c1089dacb2c4
name: find
type: application
urls:
- https://suitenumerique.github.io/find/find-0.0.3.tgz
version: 0.0.3
- apiVersion: v2
created: "2025-12-16T18:38:32.210537113Z"
digest: 9a00ea763fb6a7cb79275a0ccc5f7d83cc7bc1e8ac995544f9a6dcfcf1401f19
name: find
type: application
urls:
- https://suitenumerique.github.io/find/find-0.0.2.tgz
version: 0.0.2
- apiVersion: v2
created: "2025-01-27T10:10:29.611386242Z"
digest: bfbe5e73c4a7f306788cecfad9d65b22a89ec167257c3e03e72b097501ce1186
name: find
type: application
urls:
- https://suitenumerique.github.io/find/find-0.0.1.tgz
version: 0.0.1
generated: "2026-03-16T08:58:33.404908133Z"
-13
View File
@@ -1,13 +0,0 @@
{
"extends": ["github>numerique-gouv/renovate-configuration"],
"dependencyDashboard": true,
"labels": ["dependencies", "noChangeLog"],
"packageRules": [
{
"enabled": false,
"groupName": "ignored python dependencies",
"matchManagers": ["pep621"],
"matchPackageNames": []
}
]
}
-472
View File
@@ -1,472 +0,0 @@
[MASTER]
# A comma-separated list of package or module names from where C extensions may
# be loaded. Extensions are loading into the active Python interpreter and may
# run arbitrary code
extension-pkg-whitelist=
# Add files or directories to the blacklist. They should be base names, not
# paths.
ignore=migrations
# Add files or directories matching the regex patterns to the blacklist. The
# regex matches against base names, not paths.
ignore-patterns=
# Python code to execute, usually for sys.path manipulation such as
# pygtk.require().
#init-hook=
# Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the
# number of processors available to use.
jobs=0
# List of plugins (as comma separated values of python modules names) to load,
# usually to register additional checkers.
load-plugins=pylint_django,pylint.extensions.no_self_use
# Pickle collected data for later comparisons.
persistent=yes
# Specify a configuration file.
#rcfile=
# When enabled, pylint would attempt to guess common misconfiguration and emit
# user-friendly hints instead of false-positive error messages
suggestion-mode=yes
# Allow loading of arbitrary C extensions. Extensions are imported into the
# active Python interpreter and may run arbitrary code.
unsafe-load-any-extension=no
[MESSAGES CONTROL]
# Only show warnings with the listed confidence levels. Leave empty to show
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
confidence=
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
# option multiple times (only on the command line, not in the configuration
# file where it should appear only once).You can also use "--disable=all" to
# disable everything first and then reenable specific checks. For example, if
# you want to run only the similarities checker, you can use "--disable=all
# --enable=similarities". If you want to run only the classes checker, but have
# no Warning level messages displayed, use"--disable=all --enable=classes
# --disable=W"
disable=bad-inline-option,
deprecated-pragma,
django-not-configured,
file-ignored,
locally-disabled,
no-self-use,
raw-checker-failed,
suppressed-message,
useless-suppression
# Enable the message, report, category or checker with the given id(s). You can
# either give multiple identifier separated by comma (,) or put this option
# multiple time (only on the command line, not in the configuration file where
# it should appear only once). See also the "--disable" option for examples.
enable=c-extension-no-member
[REPORTS]
# Python expression which should return a note less than 10 (10 is the highest
# note). You have access to the variables errors warning, statement which
# respectively contain the number of errors / warnings messages and the total
# number of statements analyzed. This is used by the global evaluation report
# (RP0004).
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
# Template used to display messages. This is a python new-style format string
# used to format the message information. See doc for all details
#msg-template=
# Set the output format. Available formats are text, parseable, colorized, json
# and msvs (visual studio).You can also give a reporter class, eg
# mypackage.mymodule.MyReporterClass.
output-format=text
# Tells whether to display a full report or only the messages
reports=no
# Activate the evaluation score.
score=yes
[REFACTORING]
# Maximum number of nested blocks for function / method body
max-nested-blocks=5
# Complete name of functions that never returns. When checking for
# inconsistent-return-statements if a never returning function is called then
# it will be considered as an explicit return statement and no message will be
# printed.
never-returning-functions=optparse.Values,sys.exit
[LOGGING]
# Logging modules to check that the string format arguments are in logging
# function parameter format
logging-modules=logging
[SPELLING]
# Limits count of emitted suggestions for spelling mistakes
max-spelling-suggestions=4
# Spelling dictionary name. Available dictionaries: none. To make it working
# install python-enchant package.
spelling-dict=
# List of comma separated words that should not be checked.
spelling-ignore-words=
# A path to a file that contains private dictionary; one word per line.
spelling-private-dict-file=
# Tells whether to store unknown words to indicated private dictionary in
# --spelling-private-dict-file option instead of raising a message.
spelling-store-unknown-words=no
[MISCELLANEOUS]
# List of note tags to take in consideration, separated by a comma.
notes=FIXME,
XXX,
TODO
[TYPECHECK]
# List of decorators that produce context managers, such as
# contextlib.contextmanager. Add to this list to register other decorators that
# produce valid context managers.
contextmanager-decorators=contextlib.contextmanager
# List of members which are set dynamically and missed by pylint inference
# system, and so shouldn't trigger E1101 when accessed. Python regular
# expressions are accepted.
generated-members=
# Tells whether missing members accessed in mixin class should be ignored. A
# mixin class is detected if its name ends with "mixin" (case insensitive).
ignore-mixin-members=yes
# This flag controls whether pylint should warn about no-member and similar
# checks whenever an opaque object is returned when inferring. The inference
# can return multiple potential results while evaluating a Python object, but
# some branches might not be evaluated, which results in partial inference. In
# that case, it might be useful to still emit no-member and other checks for
# the rest of the inferred objects.
ignore-on-opaque-inference=yes
# List of class names for which member attributes should not be checked (useful
# for classes with dynamically set attributes). This supports the use of
# qualified names.
ignored-classes=optparse.Values,thread._local,_thread._local,responses,
Template,Contact
# List of module names for which member attributes should not be checked
# (useful for modules/projects where namespaces are manipulated during runtime
# and thus existing member attributes cannot be deduced by static analysis. It
# supports qualified module names, as well as Unix pattern matching.
ignored-modules=
# Show a hint with possible names when a member name was not found. The aspect
# of finding the hint is based on edit distance.
missing-member-hint=yes
# The minimum edit distance a name should have in order to be considered a
# similar match for a missing member name.
missing-member-hint-distance=1
# The total number of similar names that should be taken in consideration when
# showing a hint for a missing member.
missing-member-max-choices=1
[VARIABLES]
# List of additional names supposed to be defined in builtins. Remember that
# you should avoid to define new builtins when possible.
additional-builtins=
# Tells whether unused global variables should be treated as a violation.
allow-global-unused-variables=yes
# List of strings which can identify a callback function by name. A callback
# name must start or end with one of those strings.
callbacks=cb_,
_cb
# A regular expression matching the name of dummy variables (i.e. expectedly
# not used).
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
# Argument names that match this expression will be ignored. Default to name
# with leading underscore
ignored-argument-names=_.*|^ignored_|^unused_
# Tells whether we should check for unused import in __init__ files.
init-import=no
# List of qualified module names which can have objects that can redefine
# builtins.
redefining-builtins-modules=six.moves,past.builtins,future.builtins
[FORMAT]
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
expected-line-ending-format=
# Regexp for a line that is allowed to be longer than the limit.
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
# Number of spaces of indent required inside a hanging or continued line.
indent-after-paren=4
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
# tab).
indent-string=' '
# Maximum number of characters on a single line.
max-line-length=100
# Maximum number of lines in a module
max-module-lines=1000
# Allow the body of a class to be on the same line as the declaration if body
# contains single statement.
single-line-class-stmt=no
# Allow the body of an if to be on the same line as the test if there is no
# else.
single-line-if-stmt=no
[SIMILARITIES]
# Ignore comments when computing similarities.
ignore-comments=yes
# Ignore docstrings when computing similarities.
ignore-docstrings=yes
# Ignore imports when computing similarities.
ignore-imports=yes
# Minimum lines number of a similarity.
# First implementations of CMS wizards have common fields we do not want to factorize for now
min-similarity-lines=35
[BASIC]
# Naming style matching correct argument names
argument-naming-style=snake_case
# Regular expression matching correct argument names. Overrides argument-
# naming-style
#argument-rgx=
# Naming style matching correct attribute names
attr-naming-style=snake_case
# Regular expression matching correct attribute names. Overrides attr-naming-
# style
#attr-rgx=
# Bad variable names which should always be refused, separated by a comma
bad-names=foo,
bar,
baz,
toto,
tutu,
tata
# Naming style matching correct class attribute names
class-attribute-naming-style=any
# Regular expression matching correct class attribute names. Overrides class-
# attribute-naming-style
#class-attribute-rgx=
# Naming style matching correct class names
class-naming-style=PascalCase
# Regular expression matching correct class names. Overrides class-naming-style
#class-rgx=
# Naming style matching correct constant names
const-naming-style=UPPER_CASE
# Regular expression matching correct constant names. Overrides const-naming-
# style
const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__)|urlpatterns|logger)$
# Minimum line length for functions/classes that require docstrings, shorter
# ones are exempt.
docstring-min-length=-1
# Naming style matching correct function names
function-naming-style=snake_case
# Regular expression matching correct function names. Overrides function-
# naming-style
#function-rgx=
# Good variable names which should always be accepted, separated by a comma
good-names=i,
j,
k,
cm,
ex,
Run,
_
# Include a hint for the correct naming format with invalid-name
include-naming-hint=no
# Naming style matching correct inline iteration names
inlinevar-naming-style=any
# Regular expression matching correct inline iteration names. Overrides
# inlinevar-naming-style
#inlinevar-rgx=
# Naming style matching correct method names
method-naming-style=snake_case
# Regular expression matching correct method names. Overrides method-naming-
# style
method-rgx=([a-z_][a-z0-9_]{2,50}|setUp|set[Uu]pClass|tearDown|tear[Dd]ownClass|assert[A-Z]\w*|maxDiff|test_[a-z0-9_]+)$
# Naming style matching correct module names
module-naming-style=snake_case
# Regular expression matching correct module names. Overrides module-naming-
# style
#module-rgx=
# Colon-delimited sets of names that determine each other's naming style when
# the name regexes allow several styles.
name-group=
# Regular expression which should only match function or class names that do
# not require a docstring.
no-docstring-rgx=^_
# List of decorators that produce properties, such as abc.abstractproperty. Add
# to this list to register other decorators that produce valid properties.
property-classes=abc.abstractproperty
# Naming style matching correct variable names
variable-naming-style=snake_case
# Regular expression matching correct variable names. Overrides variable-
# naming-style
#variable-rgx=
[IMPORTS]
# Allow wildcard imports from modules that define __all__.
allow-wildcard-with-all=no
# Analyse import fallback blocks. This can be used to support both Python 2 and
# 3 compatible code, which means that the block might have code that exists
# only in one or another interpreter, leading to false positives when analysed.
analyse-fallback-blocks=no
# Deprecated modules which should not be used, separated by a comma
deprecated-modules=optparse,tkinter.tix
# Create a graph of external dependencies in the given file (report RP0402 must
# not be disabled)
ext-import-graph=
# Create a graph of every (i.e. internal and external) dependencies in the
# given file (report RP0402 must not be disabled)
import-graph=
# Create a graph of internal dependencies in the given file (report RP0402 must
# not be disabled)
int-import-graph=
# Force import order to recognize a module as part of the standard
# compatibility libraries.
known-standard-library=
# Force import order to recognize a module as part of a third party library.
known-third-party=enchant
[CLASSES]
# List of method names used to declare (i.e. assign) instance attributes.
defining-attr-methods=__init__,
__new__,
setUp
# List of member names, which should be excluded from the protected access
# warning.
exclude-protected=_asdict,
_fields,
_replace,
_source,
_make
# List of valid names for the first argument in a class method.
valid-classmethod-first-arg=cls
# List of valid names for the first argument in a metaclass class method.
valid-metaclass-classmethod-first-arg=mcs
[DESIGN]
# Maximum number of arguments for function / method
max-args=5
# Maximum number of attributes for a class (see R0902).
max-attributes=7
# Maximum number of boolean expressions in a if statement
max-bool-expr=5
# Maximum number of branch for function / method body
max-branches=12
# Maximum number of locals for function / method body
max-locals=15
# Maximum number of parents for a class (see R0901).
max-parents=7
# Maximum number of public methods for a class (see R0904).
max-public-methods=20
# Maximum number of return / yield for function / method body
max-returns=6
# Maximum number of statements in function / method body
max-statements=50
# Minimum number of public methods for a class (see R0903).
min-public-methods=0
[EXCEPTIONS]
# Exceptions that will emit a warning when being caught. Defaults to
# "Exception"
overgeneral-exceptions=builtins.Exception
-3
View File
@@ -1,3 +0,0 @@
include LICENSE
include README.md
recursive-include src/backend/find *.html *.png *.gif *.css *.ico *.jpg *.jpeg *.po *.mo *.eot *.svg *.ttf *.woff *.woff2
View File
View File
-16
View File
@@ -1,16 +0,0 @@
"""Admin config for find's core app"""
from django.contrib import admin
from .models import Service
@admin.register(Service)
class ServiceAdmin(admin.ModelAdmin):
"""Register the serivce model for the admin site"""
list_display = ("name", "created_at", "is_active")
search_fields = ("name",)
list_filter = ("is_active", "created_at")
ordering = ("-created_at",)
readonly_fields = ("created_at", "token")
-37
View File
@@ -1,37 +0,0 @@
"""Impress core API endpoints"""
from django.core.exceptions import ValidationError
from pydantic import ValidationError as PydanticValidationError
from rest_framework import exceptions as drf_exceptions
from rest_framework import status
from rest_framework import views as drf_views
from rest_framework.response import Response
def exception_handler(exc, context):
"""Handle Django ValidationError as an accepted exception.
For the parameters, see ``exception_handler``
This code comes from twidi's gist:
https://gist.github.com/twidi/9d55486c36b6a51bdcb05ce3a763e79f
"""
if isinstance(exc, ValidationError):
detail = exc.message_dict
if hasattr(exc, "message"):
detail = exc.message
elif hasattr(exc, "messages"):
detail = exc.messages
exc = drf_exceptions.ValidationError(detail=detail)
elif isinstance(exc, PydanticValidationError):
return Response(
[
{key: error[key] for key in ("msg", "type", "loc")}
for error in exc.errors()
],
status=status.HTTP_400_BAD_REQUEST,
)
return drf_views.exception_handler(exc, context)
-35
View File
@@ -1,35 +0,0 @@
"""Token authentication."""
from django.contrib.auth.models import AnonymousUser
from rest_framework import authentication, exceptions
from .models import Service
class ServiceTokenAuthentication(authentication.BaseAuthentication):
"""A custom authentication looking for valid tokens among registered services"""
model = Service
def authenticate(self, request):
"""Authenticate token from the "Authorization" header."""
token = request.headers.get("Authorization")
if not token:
raise exceptions.NotAuthenticated()
token = token.split(" ")[-1] # Extract token if prefixed with "Token"
return self.authenticate_credentials(token)
def authenticate_credentials(self, token):
"""Check that the token is registered and valid."""
try:
service_name = (
self.model.objects.only("name").get(token=token, is_active=True).name
)
except self.model.DoesNotExist as excpt:
raise exceptions.AuthenticationFailed("Invalid token.") from excpt
# We don't associate tokens with a user
return AnonymousUser(), service_name
-12
View File
@@ -1,12 +0,0 @@
"""Enums for find's core app."""
RELEVANCE = "relevance"
CREATED_AT = "created_at"
IS_PUBLIC = "is_public"
SIZE = "size"
TITLE = "title"
UPDATED_AT = "updated_at"
ORDER_BY_OPTIONS = (RELEVANCE, TITLE, CREATED_AT, UPDATED_AT, SIZE, IS_PUBLIC)
SOURCE_FIELDS = (TITLE, SIZE, CREATED_AT, UPDATED_AT, IS_PUBLIC)
-53
View File
@@ -1,53 +0,0 @@
"""Factories for the find's core app"""
from uuid import uuid4
from django.utils import timezone
from django.utils.text import slugify
import factory
from faker import Faker
from .models import Service
fake = Faker()
class DocumentSchemaFactory(factory.DictFactory):
"""
A factory for generating dictionaries that represent a document for
indexation for testing and development purposes.
"""
id = factory.LazyFunction(uuid4)
title = factory.Sequence(lambda n: f"Test title {n!s}")
content = factory.Sequence(lambda n: f"Test content {n!s}")
created_at = factory.LazyFunction(
lambda: fake.date_time_this_decade(tzinfo=timezone.get_current_timezone())
)
size = factory.LazyFunction(lambda: fake.random_int(min=0, max=1024**2))
users = factory.LazyFunction(lambda: [uuid4() for _ in range(3)])
groups = factory.LazyFunction(lambda: [slugify(fake.word()) for _ in range(3)])
is_public = factory.Faker("boolean")
@factory.lazy_attribute
def updated_at(self):
"""Ensure updated_at is after created_at and before now"""
return fake.date_time_between(
start_date=self.created_at,
end_date=timezone.now(),
tzinfo=timezone.get_current_timezone(),
)
class ServiceFactory(factory.django.DjangoModelFactory):
"""
A factory for generating service instances for testing and development purposes.
"""
name = factory.Sequence(lambda n: f"test-index-{n!s}")
created_at = factory.Faker("date_time_this_year", tzinfo=None)
is_active = True
class Meta:
model = Service
@@ -1,64 +0,0 @@
# Generated by Django 5.0.7 on 2024-09-01 05:31
import django.contrib.auth.models
import django.contrib.auth.validators
import django.utils.timezone
from django.db import migrations, models
class Migration(migrations.Migration):
initial = True
dependencies = [
('auth', '0012_alter_user_first_name_max_length'),
]
operations = [
migrations.CreateModel(
name='Service',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('name', models.SlugField(max_length=20, unique=True)),
('token', models.CharField(max_length=50)),
('created_at', models.DateTimeField(auto_now_add=True)),
('is_active', models.BooleanField(default=True)),
],
options={
'verbose_name': 'service',
'verbose_name_plural': 'services',
'db_table': 'find_service',
'ordering': ['-is_active', '-created_at'],
},
),
migrations.CreateModel(
name='User',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('password', models.CharField(max_length=128, verbose_name='password')),
('last_login', models.DateTimeField(blank=True, null=True, verbose_name='last login')),
('is_superuser', models.BooleanField(default=False, help_text='Designates that this user has all permissions without explicitly assigning them.', verbose_name='superuser status')),
('username', models.CharField(error_messages={'unique': 'A user with that username already exists.'}, help_text='Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only.', max_length=150, unique=True, validators=[django.contrib.auth.validators.UnicodeUsernameValidator()], verbose_name='username')),
('first_name', models.CharField(blank=True, max_length=150, verbose_name='first name')),
('last_name', models.CharField(blank=True, max_length=150, verbose_name='last name')),
('email', models.EmailField(blank=True, max_length=254, verbose_name='email address')),
('is_staff', models.BooleanField(default=False, help_text='Designates whether the user can log into this admin site.', verbose_name='staff status')),
('is_active', models.BooleanField(default=True, help_text='Designates whether this user should be treated as active. Unselect this instead of deleting accounts.', verbose_name='active')),
('date_joined', models.DateTimeField(default=django.utils.timezone.now, verbose_name='date joined')),
('groups', models.ManyToManyField(blank=True, help_text='The groups this user belongs to. A user will get all permissions granted to each of their groups.', related_name='user_set', related_query_name='user', to='auth.group', verbose_name='groups')),
('user_permissions', models.ManyToManyField(blank=True, help_text='Specific permissions for this user.', related_name='user_set', related_query_name='user', to='auth.permission', verbose_name='user permissions')),
],
options={
'verbose_name': 'user',
'verbose_name_plural': 'users',
'abstract': False,
},
managers=[
('objects', django.contrib.auth.models.UserManager()),
],
),
migrations.AddConstraint(
model_name='service',
constraint=models.CheckConstraint(check=models.Q(('token__length', 50)), name='token_length_exact_50'),
),
]
-59
View File
@@ -1,59 +0,0 @@
"""Models for find's core app"""
import secrets
import string
from django.contrib.auth.models import AbstractUser
from django.db import models
from django.db.models.functions import Length
from django.utils.text import slugify
from django.utils.translation import gettext_lazy as _
models.CharField.register_lookup(Length)
TOKEN_LENGTH = 50
class User(AbstractUser):
"""User for the find application"""
class Service(models.Model):
"""Service registered to index its documents to our find"""
name = models.SlugField(max_length=20, unique=True)
token = models.CharField(max_length=TOKEN_LENGTH)
created_at = models.DateTimeField(auto_now_add=True)
is_active = models.BooleanField(default=True)
class Meta:
db_table = "find_service"
verbose_name = _("service")
verbose_name_plural = _("services")
ordering = ["-is_active", "-created_at"]
constraints = [
models.CheckConstraint(
check=models.Q(token__length=TOKEN_LENGTH), name="token_length_exact_50"
),
]
def __str__(self):
return self.name
def save(self, *args, **kwargs):
"""Automatically slugify the service name and generate a token on creation"""
self.name = slugify(self.name)
if not self.token:
self.token = self.generate_secure_token()
super().save(*args, **kwargs)
@staticmethod
def generate_secure_token():
"""Generate a secure token with with Python secret module"""
characters = string.ascii_letters + string.digits + string.punctuation
token = "".join(secrets.choice(characters) for _ in range(TOKEN_LENGTH))
return token
@property
def index_name(self):
"""Compute index name from service name"""
return f"find-{self.name:s}"
-46
View File
@@ -1,46 +0,0 @@
"""Opensearch related utils."""
from django.conf import settings
from opensearchpy import OpenSearch
from opensearchpy.exceptions import NotFoundError
client = OpenSearch(
hosts=[{"host": settings.OPENSEARCH_HOST, "port": settings.OPENSEARCH_PORT}],
http_auth=(settings.OPENSEARCH_USER, settings.OPENSEARCH_PASSWORD),
timeout=50,
use_ssl=settings.OPENSEARCH_USE_SSL,
verify_certs=False,
)
def ensure_index_exists(index_name):
"""Create index if it does not exist"""
try:
client.indices.get(index=index_name)
except NotFoundError:
client.indices.create(
index=index_name,
body={
"mappings": {
"dynamic": "strict",
"properties": {
"title": {
"type": "keyword", # Primary field for exact matches and sorting
"fields": {
"text": {
"type": "text" # Sub-field for full-text search
}
},
},
"content": {"type": "text"},
"created_at": {"type": "date"},
"updated_at": {"type": "date"},
"size": {"type": "long"},
"users": {"type": "keyword"},
"groups": {"type": "keyword"},
"is_public": {"type": "boolean"},
},
}
},
)
-12
View File
@@ -1,12 +0,0 @@
"""Permission classes for find's core app"""
from rest_framework import permissions
class IsAuthAuthenticated(permissions.BasePermission):
"""
Allows access only to auth authenticated users.
"""
def has_permission(self, request, view):
return bool(request.auth)
-95
View File
@@ -1,95 +0,0 @@
"""Pydantic model to validate documents before indexation."""
from typing import Annotated, List, Literal, Optional
from django.utils import timezone
from django.utils.text import slugify
from pydantic import (
UUID4,
AwareDatetime,
BaseModel,
ConfigDict,
Field,
conint,
field_validator,
model_validator,
)
from . import enums
class DocumentSchema(BaseModel):
"""Schema for validating the documents submitted to our API for indexing"""
id: UUID4
title: Annotated[str, Field(max_length=300)]
content: str
created_at: AwareDatetime
updated_at: AwareDatetime
size: Annotated[int, Field(ge=0, le=100 * 1024**3)] # File size limited to 100GB
users: List[UUID4] = Field(default_factory=list)
groups: List[Annotated[str, Field(pattern=r"^[a-z0-9]+(?:-[a-z0-9]+)*$")]] = Field(
default_factory=list
)
is_public: bool = Field(default=False)
model_config = ConfigDict(
str_min_length=1, str_strip_whitespace=True, use_enum_values=True
)
@field_validator("title")
@staticmethod
def normalize_title(value):
"""Normalize the title field by stripping whitespace and converting to lowercase"""
return value.strip().lower()
@field_validator("created_at", "updated_at")
@staticmethod
def must_be_past(value, info):
"""Validate that `created_at` and `updated_at` fields are in the past"""
if value >= timezone.now():
raise ValueError(f"{info.field_name} must be earlier than now")
return value
@model_validator(mode="after")
def check_update_at_after_created_at(self):
"""Date and time of last modification should be later than date and time of creation"""
if self.created_at > self.updated_at:
raise ValueError("updated_at must be later than created_at")
return self
@field_validator("groups")
@staticmethod
def validate_groups(groups):
"""Validate that group slugs are properly formatted as lowercase and hyphen-separated"""
validated_groups = []
for value in groups:
slug = slugify(value)
if value != slug:
raise ValueError(
f"Groups must be slugs (lowercase, hyphen-separated): {slug:s}"
)
validated_groups.append(value)
return validated_groups
class SearchQueryParametersSchema(BaseModel):
"""Schema for validating the querystring on the search API endpoint"""
q: str
is_public: Optional[bool] = None
order_by: Optional[Literal[enums.ORDER_BY_OPTIONS]] = Field(default="relevance")
order_direction: Optional[Literal["asc", "desc"]] = Field(default="desc")
page_number: Optional[conint(ge=1)] = Field(default=1)
page_size: Optional[conint(ge=1, le=100)] = Field(default=50)
@model_validator(mode="before")
@staticmethod
def handle_lists(values):
"""Make sure we get strings and ignore multiple values."""
for key, value in values.items():
if isinstance(value, list):
# Take the first item if it's a list
values[key] = value[0] if value else None
return values
View File
@@ -1,291 +0,0 @@
"""Tests indexing documents in OpenSearch over the API"""
import datetime
from django.utils import timezone
import pytest
from rest_framework.test import APIClient
from core import factories, opensearch
pytestmark = pytest.mark.django_db
def test_api_documents_index_bulk_anonymous():
"""Anonymous requests should not be allowed to index documents in bulk."""
documents = factories.DocumentSchemaFactory.build_batch(3)
response = APIClient().post("/api/v1.0/documents/", documents, format="json")
assert response.status_code == 403
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
def test_api_documents_index_bulk_invalid_token():
"""Requests with invalid tokens should not be allowed to index documents in bulk."""
documents = factories.DocumentSchemaFactory.build_batch(3)
response = APIClient().post(
"/api/v1.0/documents/",
documents,
HTTP_AUTHORIZATION="Bearer invalid",
format="json",
)
assert response.status_code == 403
assert response.json() == {"detail": "Invalid token."}
def test_api_documents_index_bulk_success():
"""A registered service should be able to index documents in bulk with a valid token."""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(3)
response = APIClient().post(
"/api/v1.0/documents/",
documents,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 207
responses = response.json()
assert len(responses) == 3
for result in response.json():
assert result["status"] == "success"
@pytest.mark.parametrize(
"field, invalid_value, error_type, error_message",
[
(
"id",
"0f9b1c9d-030f-427a-8a0e-6b7c202c5daz", # invalid UUID b/c contains a z
"uuid_parsing",
(
"Input should be a valid UUID, invalid character: expected an optional "
"prefix of `urn:uuid:` followed by [0-9a-fA-F-], found `z` at 36"
),
),
("title", 1, "string_type", "Input should be a valid string"),
(
"title",
"a" * 301,
"string_too_long",
"String should have at most 300 characters",
),
("content", 1, "string_type", "Input should be a valid string"),
(
"created_at",
"invalid_date",
"datetime_from_date_parsing",
"Input should be a valid datetime or date, invalid character in year",
),
(
"updated_at",
"invalid_date",
"datetime_from_date_parsing",
"Input should be a valid datetime or date, invalid character in year",
),
(
"size",
64448894017.3,
"int_from_float",
"Input should be a valid integer, got a number with a fractional part",
),
(
"size",
"not an integer",
"int_parsing",
"Input should be a valid integer, unable to parse string as an integer",
),
(
"users",
"33052c8b-3181-4420-aede-f8396fc0f9a1",
"list_type",
"Input should be a valid list",
),
(
"users",
["33052c8b-3181-4420-aede-f8396fc0f9az"], # invalid UUID b/c contains a z
"uuid_parsing",
(
"Input should be a valid UUID, invalid character: expected an optional "
"prefix of `urn:uuid:` followed by [0-9a-fA-F-], found `z` at 36"
),
),
(
"groups",
["not_a_slug"],
"string_pattern_mismatch",
"String should match pattern '^[a-z0-9]+(?:-[a-z0-9]+)*$'",
),
(
"groups",
"not-a-list",
"list_type",
"Input should be a valid list",
),
(
"is_public",
"invalid_boolean",
"bool_parsing",
"Input should be a valid boolean, unable to interpret input",
),
],
)
def test_api_documents_index_bulk_invalid_document(
field, invalid_value, error_type, error_message
):
"""Test bulk document indexing with various invalid fields."""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(3)
# Modify the first document with the invalid value for the specified field
documents[0][field] = invalid_value
response = APIClient().post(
"/api/v1.0/documents/",
documents,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 400
responses = response.json()
assert responses[0]["status"] == "error"
assert len(responses[0]["errors"]) == 1
assert responses[0]["errors"][0]["msg"] == error_message
assert responses[0]["errors"][0]["type"] == error_type
for i in [1, 2]:
assert responses[i]["status"] == "valid"
@pytest.mark.parametrize(
"field", ["id", "title", "content", "size", "created_at", "updated_at"]
)
def test_api_documents_index_bulk_required(field):
"""Test bulk document indexing with a required field missing."""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(3)
del documents[0][field]
response = APIClient().post(
"/api/v1.0/documents/",
documents,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 400
responses = response.json()
assert responses[0]["status"] == "error"
assert len(responses[0]["errors"]) == 1
assert responses[0]["errors"][0]["msg"] == "Field required"
assert responses[0]["errors"][0]["type"] == "missing"
for i in [1, 2]:
assert responses[i]["status"] == "valid"
@pytest.mark.parametrize(
"field,default_value",
[
("users", []),
("groups", []),
("is_public", False),
],
)
def test_api_documents_index_bulk_default(field, default_value):
"""Test bulk document indexing while removing optional fields that have default values."""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(3)
del documents[0][field]
response = APIClient().post(
"/api/v1.0/documents/",
documents,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 207
responses = response.json()
assert len(responses) == 3
for result in response.json():
assert result["status"] == "success"
indexed_document = opensearch.client.get(
index=service.index_name, id=responses[0]["_id"]
)["_source"]
assert indexed_document[field] == default_value
def test_api_documents_index_bulk_updated_at_before_created_at():
"""Test bulk document indexing with updated_at before created_at."""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(3)
documents[0]["updated_at"] = documents[0]["created_at"] - datetime.timedelta(
seconds=1
)
response = APIClient().post(
"/api/v1.0/documents/",
documents,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 400
responses = response.json()
assert responses[0]["status"] == "error"
assert len(responses[0]["errors"]) == 1
assert (
responses[0]["errors"][0]["msg"]
== "Value error, updated_at must be later than created_at"
)
assert responses[0]["errors"][0]["type"] == "value_error"
for i in [1, 2]:
assert responses[i]["status"] == "valid"
@pytest.mark.parametrize(
"field",
["created_at", "updated_at"],
)
def test_api_documents_index_bulk_datetime_future(field):
"""Test bulk document indexing with datetimes in the future."""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(3)
now = timezone.now()
documents[0][field] = now + datetime.timedelta(seconds=3)
response = APIClient().post(
"/api/v1.0/documents/",
documents,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 400
responses = response.json()
assert responses[0]["status"] == "error"
assert len(responses[0]["errors"]) == 1
assert (
responses[0]["errors"][0]["msg"]
== f"Value error, {field:s} must be earlier than now"
)
assert responses[0]["errors"][0]["type"] == "value_error"
for i in [1, 2]:
assert responses[i]["status"] == "valid"
@@ -1,255 +0,0 @@
"""Tests indexing documents in OpenSearch over the API"""
import datetime
from django.utils import timezone
import pytest
from rest_framework.test import APIClient
from core import factories, opensearch
pytestmark = pytest.mark.django_db
def test_api_documents_index_single_anonymous():
"""Anonymous requests should not be allowed to index documents."""
document = factories.DocumentSchemaFactory.build()
response = APIClient().post("/api/v1.0/documents/", document, format="json")
assert response.status_code == 403
assert response.json() == {
"detail": "Authentication credentials were not provided."
}
def test_api_documents_index_single_invalid_token():
"""Requests with invalid tokens should not be allowed to index documents."""
document = factories.DocumentSchemaFactory.build()
response = APIClient().post(
"/api/v1.0/documents/",
document,
HTTP_AUTHORIZATION="Bearer invalid",
format="json",
)
assert response.status_code == 403
assert response.json() == {"detail": "Invalid token."}
def test_api_documents_index_single_success():
"""A registered service should be able to index document with a valid token."""
service = factories.ServiceFactory(name="test-service")
document = factories.DocumentSchemaFactory.build()
response = APIClient().post(
"/api/v1.0/documents/",
document,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 201
assert response.json()["_id"] == str(document["id"])
@pytest.mark.parametrize(
"field, invalid_value, error_type, error_message",
[
(
"id",
"0f9b1c9d-030f-427a-8a0e-6b7c202c5daz", # invalid UUID b/c contains a z*
"uuid_parsing",
(
"Input should be a valid UUID, invalid character: expected an optional "
"prefix of `urn:uuid:` followed by [0-9a-fA-F-], found `z` at 36"
),
),
("title", 1, "string_type", "Input should be a valid string"),
(
"title",
"a" * 301,
"string_too_long",
"String should have at most 300 characters",
),
("content", 1, "string_type", "Input should be a valid string"),
(
"created_at",
"invalid_date",
"datetime_from_date_parsing",
"Input should be a valid datetime or date, invalid character in year",
),
(
"updated_at",
"invalid_date",
"datetime_from_date_parsing",
"Input should be a valid datetime or date, invalid character in year",
),
(
"size",
64448894017.3,
"int_from_float",
"Input should be a valid integer, got a number with a fractional part",
),
(
"size",
"not an integer",
"int_parsing",
"Input should be a valid integer, unable to parse string as an integer",
),
(
"users",
"33052c8b-3181-4420-aede-f8396fc0f9a1",
"list_type",
("Input should be a valid list"),
),
(
"users",
["33052c8b-3181-4420-aede-f8396fc0f9az"], # invalid UUID b/c contains a z*
"uuid_parsing",
(
"Input should be a valid UUID, invalid character: expected an optional "
"prefix of `urn:uuid:` followed by [0-9a-fA-F-], found `z` at 36"
),
),
(
"groups",
["not_a_slug"],
"string_pattern_mismatch",
"String should match pattern '^[a-z0-9]+(?:-[a-z0-9]+)*$'",
),
(
"groups",
"not-a-list",
"list_type",
("Input should be a valid list"),
),
(
"is_public",
"invalid_boolean",
"bool_parsing",
"Input should be a valid boolean, unable to interpret input",
),
],
)
def test_api_documents_index_single_invalid_document(
field, invalid_value, error_type, error_message
):
"""Test document indexing with various invalid fields."""
service = factories.ServiceFactory(name="test-service")
document = factories.DocumentSchemaFactory.build()
# Modify the document with the invalid value for the specified field
document[field] = invalid_value
response = APIClient().post(
"/api/v1.0/documents/",
document,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 400
assert response.data[0]["msg"] == error_message
assert response.data[0]["type"] == error_type
@pytest.mark.parametrize(
"field", ["id", "title", "content", "size", "created_at", "updated_at"]
)
def test_api_documents_index_single_required(field):
"""Test document indexing with a required field missing."""
service = factories.ServiceFactory(name="test-service")
document = factories.DocumentSchemaFactory.build()
del document[field]
response = APIClient().post(
"/api/v1.0/documents/",
document,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 400
assert response.data[0]["msg"] == "Field required"
assert response.data[0]["type"] == "missing"
@pytest.mark.parametrize(
"field,default_value",
[
("users", []),
("groups", []),
("is_public", False),
],
)
def test_api_documents_index_single_default(field, default_value):
"""Test document indexing while removing optional fields that have default values."""
service = factories.ServiceFactory(name="test-service")
document = factories.DocumentSchemaFactory.build()
del document[field]
response = APIClient().post(
"/api/v1.0/documents/",
document,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 201
assert response.json()["_id"] == str(document["id"])
indexed_document = opensearch.client.get(
index=service.index_name, id=str(document["id"])
)["_source"]
assert indexed_document[field] == default_value
def test_api_documents_index_single_udpated_at_before_created():
"""Test document indexing with updated_at before created_at."""
service = factories.ServiceFactory(name="test-service")
document = factories.DocumentSchemaFactory.build()
document["updated_at"] = document["created_at"] - datetime.timedelta(seconds=1)
response = APIClient().post(
"/api/v1.0/documents/",
document,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 400
assert (
response.data[0]["msg"]
== "Value error, updated_at must be later than created_at"
)
assert response.data[0]["type"] == "value_error"
@pytest.mark.parametrize(
"field",
["created_at", "updated_at"],
)
def test_api_documents_index_single_datetime_future(field):
"""Test document indexing with datetimes in the future."""
service = factories.ServiceFactory(name="test-service")
document = factories.DocumentSchemaFactory.build()
now = timezone.now()
document[field] = now + datetime.timedelta(seconds=3)
response = APIClient().post(
"/api/v1.0/documents/",
document,
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
format="json",
)
assert response.status_code == 400
assert response.data[0]["msg"] == f"Value error, {field:s} must be earlier than now"
assert response.data[0]["type"] == "value_error"
@@ -1,449 +0,0 @@
"""
Test suite for searching documents in OpenSearch over the API.
Don't use pytest parametrized tests because batch generation and indexing
of documents is slow and better done only once.
"""
import operator
import pytest
from opensearchpy.helpers import bulk
from rest_framework.test import APIClient
from core import enums, factories, opensearch
pytestmark = pytest.mark.django_db
def prepare_index(index_name, documents):
"""Prepare the search index before testing a query on it."""
documents = documents if isinstance(documents, list) else [documents]
# Ensure existence of an empty index
opensearch.client.indices.delete(index=index_name)
opensearch.ensure_index_exists(index_name)
# Bulk index documents
actions = []
for document in documents:
_source = document.copy()
_id = _source.pop("id")
actions.append(
{
"_op_type": "index",
"_index": index_name,
"_id": _id,
"_source": _source,
}
)
bulk(opensearch.client, actions)
opensearch.client.indices.refresh(index=index_name)
assert opensearch.client.count(index=index_name)["count"] == len(documents)
def test_api_documents_search_query_title():
"""Searching a document by its title should work as expected"""
service = factories.ServiceFactory(name="test-service")
document = factories.DocumentSchemaFactory.build(
title="The quick brown fox", content="the wolf"
)
# Add other documents
other_fox_document = factories.DocumentSchemaFactory.build(
title="The blue fox", content="the wolf"
)
no_fox_document = factories.DocumentSchemaFactory.build(
title="The brown goat", content="the wolf"
)
documents = [document, other_fox_document, no_fox_document]
prepare_index(service.index_name, documents)
response = APIClient().get(
"/api/v1.0/documents/",
{"q": "a quick fox"},
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
)
assert response.status_code == 200
assert len(response.json()) == 2
fox_data = response.json()[0]
assert list(fox_data.keys()) == ["_index", "_id", "_score", "_source", "fields"]
assert fox_data["_id"] == str(document["id"])
assert fox_data["_source"] == {
"size": document["size"],
"created_at": document["created_at"].isoformat(),
"updated_at": document["updated_at"].isoformat(),
"is_public": document["is_public"],
"title": "The quick brown fox",
}
assert fox_data["fields"] == {"number_of_users": [3], "number_of_groups": [3]}
other_fox_data = response.json()[1]
assert list(other_fox_data.keys()) == [
"_index",
"_id",
"_score",
"_source",
"fields",
]
assert other_fox_data["_id"] == str(other_fox_document["id"])
assert other_fox_data["_source"] == {
"size": other_fox_document["size"],
"created_at": other_fox_document["created_at"].isoformat(),
"updated_at": other_fox_document["updated_at"].isoformat(),
"is_public": other_fox_document["is_public"],
"title": "The blue fox",
}
assert other_fox_data["fields"] == {"number_of_users": [3], "number_of_groups": [3]}
def test_api_documents_search_query_content():
"""Searching a document by its content should work as expected"""
service = factories.ServiceFactory(name="test-service")
document = factories.DocumentSchemaFactory.build(
title="the wolf", content="The quick brown fox"
)
# Add other documents
other_fox_document = factories.DocumentSchemaFactory.build(
title="the wolf", content="The blue fox"
)
no_fox_document = factories.DocumentSchemaFactory.build(
title="the wolf", content="The brown goat"
)
prepare_index(service.index_name, [document, other_fox_document, no_fox_document])
response = APIClient().get(
"/api/v1.0/documents/",
{"q": "a quick fox"},
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
)
assert response.status_code == 200
assert len(response.json()) == 2
fox_data = response.json()[0]
assert list(fox_data.keys()) == ["_index", "_id", "_score", "_source", "fields"]
assert fox_data["_id"] == str(document["id"])
assert fox_data["_source"] == {
"size": document["size"],
"created_at": document["created_at"].isoformat(),
"updated_at": document["updated_at"].isoformat(),
"is_public": document["is_public"],
"title": document["title"],
}
assert fox_data["fields"] == {"number_of_users": [3], "number_of_groups": [3]}
other_fox_data = response.json()[1]
assert list(other_fox_data.keys()) == [
"_index",
"_id",
"_score",
"_source",
"fields",
]
assert other_fox_data["_id"] == str(other_fox_document["id"])
assert other_fox_data["_source"] == {
"size": other_fox_document["size"],
"created_at": other_fox_document["created_at"].isoformat(),
"updated_at": other_fox_document["updated_at"].isoformat(),
"is_public": other_fox_document["is_public"],
"title": other_fox_document["title"],
}
assert other_fox_data["fields"] == {"number_of_users": [3], "number_of_groups": [3]}
def test_api_documents_search_ordering_by_fields():
"""It should be possible to order by several fields"""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(4)
prepare_index(service.index_name, documents)
parameters = [
(enums.TITLE, "asc"),
(enums.TITLE, "desc"),
(enums.CREATED_AT, "asc"),
(enums.CREATED_AT, "desc"),
(enums.UPDATED_AT, "asc"),
(enums.UPDATED_AT, "desc"),
(enums.SIZE, "asc"),
(enums.SIZE, "desc"),
(enums.IS_PUBLIC, "asc"),
(enums.IS_PUBLIC, "desc"),
]
for field, direction in parameters:
response = APIClient().get(
f"/api/v1.0/documents/?q=*&order_by={field}&order_direction={direction}",
HTTP_AUTHORIZATION=f"Bearer {service.token}",
)
assert response.status_code == 200
responses = response.json()
assert len(responses) == 4
# Check that results are sorted by the field as expected
compare = operator.le if direction == "asc" else operator.ge
for i in range(len(responses) - 1):
assert compare(
responses[i]["_source"][field], responses[i + 1]["_source"][field]
)
def test_api_documents_search_ordering_by_relevance():
"""It should be possible to order by relevance (score)"""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(4)
prepare_index(service.index_name, documents)
for direction in ["asc", "desc"]:
response = APIClient().get(
f"/api/v1.0/documents/?q=*&order_by=relevance&order_direction={direction}",
HTTP_AUTHORIZATION=f"Bearer {service.token}",
)
assert response.status_code == 200
responses = response.json()
assert len(responses) == 4
# Check that results are sorted by score as expected
compare = operator.le if direction == "asc" else operator.ge
for i in range(len(responses) - 1):
assert compare(responses[i]["_score"], responses[i + 1]["_score"])
def test_api_documents_search_ordering_by_unknown_field():
"""Trying to sort by an unknown field should return a 400 error"""
# Setup: Initialize the service and documents only once
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(2)
prepare_index(service.index_name, documents)
# Define the parameters manually
directions = ["asc", "desc"]
# Perform the parameterized tests
for direction in directions:
response = APIClient().get(
f"/api/v1.0/documents/?q=*&order_by=unknown&order_direction={direction}",
HTTP_AUTHORIZATION=f"Bearer {service.token}",
)
assert response.status_code == 400
assert response.json() == [
{
"loc": ["order_by"],
"msg": (
"Input should be 'relevance', 'title', 'created_at', "
"'updated_at', 'size' or 'is_public'"
),
"type": "literal_error",
}
]
def test_api_documents_search_ordering_by_unknown_direction():
"""Trying to sort with an unknown direction should return a 400 error"""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(2)
prepare_index(service.index_name, documents)
for field in enums.ORDER_BY_OPTIONS:
response = APIClient().get(
f"/api/v1.0/documents/?q=*&order_by={field}&order_direction=unknown",
HTTP_AUTHORIZATION=f"Bearer {service.token}",
)
assert response.status_code == 400
assert response.json() == [
{
"loc": ["order_direction"],
"msg": "Input should be 'asc' or 'desc'",
"type": "literal_error",
}
]
def test_api_documents_search_filtering_by_is_public():
"""It should be possible to filter results by their publication status"""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(4)
prepare_index(service.index_name, documents)
parameters = [
[True, "True"],
[False, "False"],
[True, "true"],
[False, "false"],
[True, "1"],
[False, "0"],
]
for public_boolean, public_string in parameters:
response = APIClient().get(
f"/api/v1.0/documents/?q=*&is_public={public_string}",
HTTP_AUTHORIZATION=f"Bearer {service.token}",
)
assert response.status_code == 200
responses = response.json()
for result in responses:
assert public_boolean == result["_source"]["is_public"]
# Pagination
def test_api_documents_search_pagination_basic():
"""Pagination should correctly return documents for the specified page and page size"""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(9)
ids = [str(doc["id"]) for doc in documents]
prepare_index(service.index_name, documents)
# Request the first page with a page size of 3
response = APIClient().get(
"/api/v1.0/documents/?q=*&page_number=1&page_size=3",
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
)
assert response.status_code == 200
responses = response.json()
assert len(responses) == 3 # Page size is 3
assert [r["_id"] for r in responses] == ids[0:3]
# Request the second page with a page size of 3
response = APIClient().get(
"/api/v1.0/documents/?q=*&page_number=2&page_size=3",
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
)
assert response.status_code == 200
responses = response.json()
assert len(responses) == 3
assert [r["_id"] for r in responses] == ids[3:6]
# Request the third page with a page size of 5 (should contain the remaining 3 documents)
response = APIClient().get(
"/api/v1.0/documents/?q=*&page_number=3&page_size=3",
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
)
assert response.status_code == 200
responses = response.json()
assert len(responses) == 3
assert [r["_id"] for r in responses] == ids[6:9]
def test_api_documents_search_pagination_last_page_edge_case():
"""Requesting the last page should return the correct number of remaining documents"""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(8)
ids = [str(doc["id"]) for doc in documents]
prepare_index(service.index_name, documents)
# Request the first page with a page size of 3
response = APIClient().get(
"/api/v1.0/documents/?q=*&page_number=1&page_size=3",
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
)
assert response.status_code == 200
assert len(response.json()) == 3
assert [r["_id"] for r in response.json()] == ids[0:3]
# Request the third page with a page size of 3 (should contain the last 1 document)
response = APIClient().get(
"/api/v1.0/documents/?q=*&page_number=3&page_size=3",
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
)
assert response.status_code == 200
assert len(response.json()) == 2 # Only 2 documents should be on the last page
assert [r["_id"] for r in response.json()] == ids[6:]
def test_api_documents_search_pagination_out_of_bounds():
"""
Requesting a page number that exceeds the total number of pages should return an empty list
"""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(4)
prepare_index(service.index_name, documents)
# Request the fourth page with a page size of 2 (there are only 2 pages)
response = APIClient().get(
"/api/v1.0/documents/?q=*&page_number=4&page_size=2",
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
)
assert response.status_code == 200
assert len(response.json()) == 0 # No documents should be returned
def test_api_documents_search_pagination_invalid_parameters():
"""Invalid pagination parameters should result in a 400 error"""
service = factories.ServiceFactory(name="test-service")
documents = factories.DocumentSchemaFactory.build_batch(4)
prepare_index(service.index_name, documents)
parameters = [
(
"invalid",
10,
"int_parsing",
"Input should be a valid integer, unable to parse string as an integer",
),
(
1,
"invalid",
"int_parsing",
"Input should be a valid integer, unable to parse string as an integer",
),
(-1, 10, "greater_than_equal", "Input should be greater than or equal to 1"),
(1, -10, "greater_than_equal", "Input should be greater than or equal to 1"),
(0, 10, "greater_than_equal", "Input should be greater than or equal to 1"),
(1, 0, "greater_than_equal", "Input should be greater than or equal to 1"),
]
for page_number, page_size, error_type, error_message in parameters:
response = APIClient().get(
f"/api/v1.0/documents/?q=*&page_number={page_number}&page_size={page_size}",
HTTP_AUTHORIZATION=f"Bearer {service.token}",
)
assert response.status_code == 400
assert response.data[0]["msg"] == error_message
assert response.data[0]["type"] == error_type
def test_api_documents_search_pagination_with_filtering():
"""Pagination should work correctly when combined with filtering by is_public"""
service = factories.ServiceFactory(name="test-service")
public_documents = factories.DocumentSchemaFactory.build_batch(3, is_public=True)
public_ids = [str(doc["id"]) for doc in public_documents]
private_documents = factories.DocumentSchemaFactory.build_batch(2, is_public=False)
prepare_index(service.index_name, public_documents + private_documents)
# Filter by public documents, request first page
response = APIClient().get(
"/api/v1.0/documents/?q=*&is_public=true&page_number=1&page_size=2",
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
)
assert response.status_code == 200
assert len(response.json()) == 2
assert [r["_id"] for r in response.json()] == public_ids[0:2]
# Request second page for public documents (remaining 1 document)
response = APIClient().get(
"/api/v1.0/documents/?q=*&is_public=true&page_number=2&page_size=2",
HTTP_AUTHORIZATION=f"Bearer {service.token:s}",
)
assert response.status_code == 200
assert len(response.json()) == 1
assert [r["_id"] for r in response.json()] == public_ids[2:]
@@ -1,47 +0,0 @@
"""Tests Service model for find's core app."""
from django.db import DataError, IntegrityError
import pytest
from core import factories
pytestmark = pytest.mark.django_db
def test_models_services_name_unique():
"""The name field should be unique across services."""
service = factories.ServiceFactory()
with pytest.raises(IntegrityError):
factories.ServiceFactory(name=service.name)
def test_models_services_name_slugified():
"""The name field should be slugified."""
service = factories.ServiceFactory(name="My service name")
assert service.name == "my-service-name"
def test_models_services_index_name():
"""The index name should be computed as a property from the service name."""
service = factories.ServiceFactory(name="My service name")
assert service.index_name == "find-my-service-name"
def test_models_services_token_50_characters_exact():
"""The token field should be 50 characters long."""
service = factories.ServiceFactory()
assert len(service.token) == 50
def test_models_services_token_50_characters_less():
"""The token field should not be less than 50 characters long."""
with pytest.raises(IntegrityError):
factories.ServiceFactory(token="a" * 49)
def test_models_services_token_50_characters_more():
"""The token field should be 50 characters long."""
with pytest.raises(DataError):
factories.ServiceFactory(token="a" * 51)
-9
View File
@@ -1,9 +0,0 @@
"""URL configuration for find's core app."""
from django.urls import path
from .views import DocumentView
urlpatterns = [
path("documents/", DocumentView.as_view(), name="document"),
]
-240
View File
@@ -1,240 +0,0 @@
"""Views for find's core app."""
from pydantic import ValidationError as PydanticValidationError
from rest_framework import status, views
from rest_framework.response import Response
from urllib3.exceptions import ReadTimeoutError
from . import enums, schemas
from .authentication import ServiceTokenAuthentication
from .opensearch import client, ensure_index_exists
from .permissions import IsAuthAuthenticated
class DocumentView(views.APIView):
"""
API view for managing documents in an OpenSearch index.
This view provides functionality for both indexing and searching documents
within an OpenSearch index dedicated to the authenticated service. The class
supports the following operations:
1. **Document Indexing (POST)**:
- Handles both single document and bulk document indexing.
- The index is dynamically determined based on the service authentication token,
ensuring that each service has its own isolated index.
2. **Document Search (GET)**:
- Enables searching through indexed documents with support for various filters
and sorting options.
- The search results can be sorted or filtered via querystring parameters.
"""
authentication_classes = [ServiceTokenAuthentication]
permission_classes = [IsAuthAuthenticated]
@property
def index_name(self):
"""Compute index name from the service name extracted during authentication"""
return f"find-{self.request.auth}"
# pylint: disable=too-many-locals
def post(self, request, *args, **kwargs):
"""
API view for indexing documents into OpenSearch index of the authenticated service.
This view supports both single document indexing and bulk indexing. It handles
the following scenarios based on the type of request data:
1. **Single Document Indexing**: If the request contains a single document (as a
dictionary), it will be indexed into an OpenSearch index named `find-{auth_token}`.
On success, the indexed document is returned with a `201 Created` status. If an
error occurs, a `400 Bad Request` response with an error message is returned.
2. **Bulk Indexing**: If the request contains a list of documents, each document is
validated and indexed in bulk. The response includes a detailed status of each
document, indicating whether it was successfully indexed or if an error occurred.
The HTTP status code for the bulk indexing operation is `207 Multi-Status`, and the
response body contains information about the success or failure of each individual
document.
Methods:
-------
post(request, *args, **kwargs):
Handles POST requests to index either a single document or a list of documents.
- **Single Document**: Expects a dictionary representing a document.
- **Bulk Indexing**: Expects a list of dictionaries, each representing a document.
Request Data:
-------------
For single document indexing:
- Content-Type: application/json
- Body: JSON object representing a document.
For bulk indexing:
- Content-Type: application/json
- Body: JSON array of JSON objects, each representing a document.
Responses:
-----------
- **Single Document Indexing**:
- 201 Created: Returns the indexed document.
- 400 Bad Request: Returns an error message if the document is invalid or if
indexing fails.
- **Bulk Indexing**:
- 207 Multi-Status if all documents formatting is correct, 400 Bad Request otherwise:
- Returns a list of results for all documents, with details of success and indexing
errors.
"""
if isinstance(request.data, list):
# Bulk indexing several documents
results = []
actions = []
has_errors = False
for i, document_data in enumerate(request.data):
try:
document = schemas.DocumentSchema(**document_data)
except PydanticValidationError as excpt:
errors = [
{key: error[key] for key in ("msg", "type", "loc")}
for error in excpt.errors()
]
results.append({"index": i, "status": "error", "errors": errors})
has_errors = True
else:
document_dict = document.model_dump()
_id = document_dict.pop("id")
actions.append({"index": {"_id": _id}})
actions.append(document_dict)
results.append({"index": i, "_id": _id, "status": "valid"})
if has_errors:
return Response(results, status=status.HTTP_400_BAD_REQUEST)
ensure_index_exists(self.index_name)
response = client.bulk(index=self.index_name, body=actions)
for i, item in enumerate(response["items"]):
if item["index"]["status"] != 201:
results[i]["status"] = "error"
results[i]["message"] = (
item["index"].get("error", {}).get("reason", "Unknown error")
)
else:
results[i]["status"] = "success"
return Response(results, status=status.HTTP_207_MULTI_STATUS)
# Indexing a single document
document = schemas.DocumentSchema(**request.data)
document_dict = document.model_dump()
_id = document_dict.pop("id")
try:
client.index(index=self.index_name, body=document_dict, id=_id)
except ReadTimeoutError:
ensure_index_exists(self.index_name)
client.index(index=self.index_name, body=document_dict, id=_id)
return Response(
{"status": "created", "_id": _id}, status=status.HTTP_201_CREATED
)
def get(self, request, *args, **kwargs):
"""
Handle GET requests to perform a search on indexed documents with optional filtering
and ordering.
The search query should be provided as a query parameter 'q'. The method constructs a
search request to OpenSearch using the specified query, with the option to filter by
'is_public' and order by 'relevance', 'created_at', 'updated_at', or 'size'.
The results are further filtered by 'users' and 'groups' based on the authentication
header.
Query Parameters:
---------------
q : str
The search query string. This is a required parameter.
is_public : bool, optional
Filter results based on the 'is_public' field.
order_by : str, optional
Order results by 'relevance', 'created_at', 'updated_at', or 'size'.
Defaults to 'relevance' if not specified.
order_direction : str, optional
Order direction, 'asc' for ascending or 'desc' for descending.
Defaults to 'desc'.
page_number : int, optional
The page number to retrieve.
Defaults to 1 if not specified.
page_size : int, optional
The number of results to return per page.
Defaults to 50 if not specified.
Returns:
--------
Response : rest_framework.response.Response
- 200 OK: Returns a list of search results matching the query.
- 400 Bad Request: If the query parameter 'q' is not provided or invalid.
"""
# Extract and validate query parameters using Pydantic schema
params = schemas.SearchQueryParametersSchema(**request.GET)
# Compute pagination parameters
from_value = (params.page_number - 1) * params.page_size
size_value = params.page_size
# Prepare the search query
search_body = {
"_source": enums.SOURCE_FIELDS, # limit the fields to return
"script_fields": {
"number_of_users": {"script": {"source": "doc['users'].size()"}},
"number_of_groups": {"script": {"source": "doc['groups'].size()"}},
},
"query": {"bool": {"must": [], "filter": []}},
"sort": [],
"from": from_value,
"size": size_value,
}
# Adding the text query
if params.q == "*":
search_body["query"]["bool"]["must"].append({"match_all": {}})
else:
search_body["query"]["bool"]["must"].append(
{
"multi_match": {
"query": params.q,
# Give title more importance over content by a power of 3
"fields": ["title.text^3", "content"],
}
}
)
# Add sorting logic based on relevance or specified field
if params.order_by == enums.RELEVANCE:
search_body["sort"].append({"_score": {"order": params.order_direction}})
else:
search_body["sort"].append(
{params.order_by: {"order": params.order_direction}}
)
# Filter by is_public if provided
if params.is_public is not None:
search_body["query"]["bool"]["filter"].append(
{"term": {enums.IS_PUBLIC: params.is_public}}
)
# Filter by users and groups based on authentication$
# user = request.user
# groups = user.get_teams()
# search_body['query']['bool']['filter'].append({
# "terms": {"users": [user.sub]}
# })
# search_body['query']['bool']['filter'].append({
# "terms": {"groups": list(groups)}
# })
response = client.search(index=self.index_name, body=search_body)
return Response(response["hits"]["hits"], status=status.HTTP_200_OK)
View File
-6
View File
@@ -1,6 +0,0 @@
"""Parameters that define how the demo site will be built."""
NB_OBJECTS = {
"documents": 1000,
"services": 5,
}
@@ -1,198 +0,0 @@
# ruff: noqa: S311, S106
"""create_demo management command"""
import logging
import random
import time
from uuid import uuid4
from django.conf import settings
from django.core.management.base import BaseCommand, CommandError
from django.utils import timezone
from django.utils.text import slugify
from faker import Faker
from opensearchpy.helpers import bulk
from core import factories, opensearch
from demo import defaults
fake = Faker()
logger = logging.getLogger("find.commands.demo.create_demo")
class BulkIndexing:
"""A utility class to index to OpenSearch in bulk by just pushing to a queue."""
BATCH_SIZE = 20000
def __init__(self, stdout, *args, **kwargs):
"""Define actions as a list."""
self.actions = []
self.stdout = stdout
self.logger = logging.getLogger(__name__)
def bulk_index(self):
"""Actually index documents in bulk to OpenSearch."""
_success, failed = bulk(opensearch.client, self.actions, stats_only=False)
if failed:
self.handle_failures(failed)
# Clear the actions list after bulk indexing and display progress
self.actions.clear()
self.stdout.write(".", ending="")
def handle_failures(self, failed):
"""Handle the failed bulk operations."""
for failure in failed:
self.logger.error("Failed to index document: %s", failure)
def push(self, index_name, _id, document):
"""Add a document to queue so that it gets created in bulk."""
self.actions.append(
{
"_op_type": "index",
"_index": index_name,
"_id": _id,
"_source": document,
}
)
if len(self.actions) >= self.BATCH_SIZE:
self.bulk_index()
def flush(self):
"""Flush any remaining documents in the queue."""
if self.actions:
self.bulk_index()
def __del__(self):
"""Ensure that any remaining documents are indexed when the instance is deleted."""
self.flush()
class Timeit:
"""A utility context manager/method decorator to time execution."""
total_time = 0
def __init__(self, stdout, sentence=None):
"""Set the sentence to be displayed for timing information."""
self.sentence = sentence
self.start = None
self.stdout = stdout
def __call__(self, func):
"""Behavior on call for use as a method decorator."""
def timeit_wrapper(*args, **kwargs):
"""wrapper to trigger/stop the timer before/after function call."""
self.__enter__()
result = func(*args, **kwargs)
self.__exit__(None, None, None)
return result
return timeit_wrapper
def __enter__(self):
"""Start timer upon entering context manager."""
self.start = time.perf_counter()
if self.sentence:
self.stdout.write(self.sentence, ending=".")
def __exit__(self, exc_type, exc_value, exc_tb):
"""Stop timer and display result upon leaving context manager."""
if exc_type is not None:
raise exc_type(exc_value)
end = time.perf_counter()
elapsed_time = end - self.start
if self.sentence:
self.stdout.write(f" Took {elapsed_time:g} seconds")
self.__class__.total_time += elapsed_time
return elapsed_time
def generate_document():
"""Generate a realistic document dictionary faster than factory_boy does it."""
created_at = fake.past_datetime(tzinfo=timezone.get_current_timezone())
updated_at = fake.date_time_between(
start_date=created_at,
end_date=timezone.now(),
tzinfo=timezone.get_current_timezone(),
)
return {
"title": fake.sentence(nb_words=10, variable_nb_words=True),
"content": "\n".join(fake.paragraphs(nb=5)),
"created_at": created_at,
"updated_at": updated_at,
"size": random.randint(0, 100 * 1024**2),
"users": [str(uuid4()) for _ in range(3)],
"groups": [slugify(fake.word()) for _ in range(3)],
"is_public": fake.boolean(),
}
def create_demo(stdout):
"""
Create a database with demo data for developers to work in a realistic environment.
"""
opensearch.client.indices.delete("*")
with Timeit(stdout, "Creating services"):
services = factories.ServiceFactory.create_batch(
defaults.NB_OBJECTS["services"]
)
for service in services:
opensearch.ensure_index_exists(service.index_name)
opensearch.client.indices.refresh(index=service.index_name)
with Timeit(stdout, "Creating documents"):
actions = BulkIndexing(stdout)
for _ in range(defaults.NB_OBJECTS["documents"]):
service = random.choice(services)
document = generate_document()
actions.push(service.index_name, uuid4(), document)
actions.flush()
# Check and report on indexed documents
total_indexed = 0
for service in services:
opensearch.client.indices.refresh(index=service.index_name)
indexed = opensearch.client.count(index=service.index_name)["count"]
stdout.write(f" - {service.index_name:s}: {indexed:d} documents")
total_indexed += indexed
stdout.write(f" TOTAL: {total_indexed:d} documents")
class Command(BaseCommand):
"""A management command to create a demo database."""
help = __doc__
def add_arguments(self, parser):
"""Add argument to require forcing execution when not in debug mode."""
parser.add_argument(
"-f",
"--force",
action="store_true",
default=False,
help="Force command execution despite DEBUG is set to False",
)
def handle(self, *args, **options):
"""Handling of the management command."""
if not settings.DEBUG and not options["force"]:
raise CommandError(
(
"This command is not meant to be used in production environment "
"except you know what you are doing, if so use --force parameter"
)
)
create_demo(self.stdout)
View File
@@ -1,29 +0,0 @@
"""Test the `create_demo` management command"""
from unittest import mock
from django.core.management import call_command
from django.test import override_settings
import pytest
from core import models, opensearch
from demo import defaults
pytestmark = pytest.mark.django_db
TEST_NB_OBJECTS = {
"documents": 4,
"services": 2,
}
@override_settings(DEBUG=True)
@mock.patch.dict(defaults.NB_OBJECTS, TEST_NB_OBJECTS)
def test_commands_create_demo():
"""The create_demo management command should create objects as expected."""
call_command("create_demo")
assert models.Service.objects.count() == 2
assert opensearch.client.count()["count"] == 4
View File
-23
View File
@@ -1,23 +0,0 @@
"""find celery configuration file."""
import os
from celery import Celery
from configurations.importer import install
# Set the default Django settings module for the 'celery' program.
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "find.settings")
os.environ.setdefault("DJANGO_CONFIGURATION", "Development")
install(check_options=True)
app = Celery("find")
# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
# should have a `CELERY_` prefix.
app.config_from_object("django.conf:settings", namespace="CELERY")
# Load task modules from all registered Django apps.
app.autodiscover_tasks()
-548
View File
@@ -1,548 +0,0 @@
"""
Django settings for find project.
Generated by 'django-admin startproject' using Django 3.1.5.
For more information on this file, see
https://docs.djangoproject.com/en/3.1/topics/settings/
For the full list of settings and their values, see
https://docs.djangoproject.com/en/3.1/ref/settings/
"""
import json
import os
from django.utils.translation import gettext_lazy as _
import sentry_sdk
from configurations import Configuration, values
from sentry_sdk.integrations.django import DjangoIntegration
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA_DIR = os.path.join("/", "data")
def get_release():
"""
Get the current release of the application
By release, we mean the release from the version.json file à la Mozilla [1]
(if any). If this file has not been found, it defaults to "NA".
[1]
https://github.com/mozilla-services/Dockerflow/blob/master/docs/version_object.md
"""
# Try to get the current release from the version.json file generated by the
# CI during the Docker image build
try:
with open(os.path.join(BASE_DIR, "version.json"), encoding="utf8") as version:
return json.load(version)["version"]
except FileNotFoundError:
return "NA" # Default: not available
class Base(Configuration):
"""
This is the base configuration every configuration (aka environment) should inherit from. It
is recommended to configure third-party applications by creating a configuration mixins in
./configurations and compose the Base configuration with those mixins.
It depends on an environment variable that SHOULD be defined:
* DJANGO_SECRET_KEY
You may also want to override default configuration by setting the following environment
variables:
* DJANGO_SENTRY_DSN
* DB_NAME
* DB_HOST
* DB_PASSWORD
* DB_USER
"""
DEBUG = False
USE_SWAGGER = False
API_VERSION = "v1.0"
# Security
ALLOWED_HOSTS = values.ListValue([])
SECRET_KEY = values.Value(None)
# Application definition
ROOT_URLCONF = "find.urls"
WSGI_APPLICATION = "find.wsgi.application"
# Database
DATABASES = {
"default": {
"ENGINE": values.Value(
"django.db.backends.postgresql_psycopg2",
environ_name="DB_ENGINE",
environ_prefix=None,
),
"NAME": values.Value("find", environ_name="DB_NAME", environ_prefix=None),
"USER": values.Value("dinum", environ_name="DB_USER", environ_prefix=None),
"PASSWORD": values.Value(
"pass", environ_name="DB_PASSWORD", environ_prefix=None
),
"HOST": values.Value(
"localhost", environ_name="DB_HOST", environ_prefix=None
),
"PORT": values.Value(5432, environ_name="DB_PORT", environ_prefix=None),
}
}
DEFAULT_AUTO_FIELD = "django.db.models.AutoField"
# Static files (CSS, JavaScript, Images)
STATIC_URL = "/static/"
STATIC_ROOT = os.path.join(DATA_DIR, "static")
MEDIA_URL = "/media/"
MEDIA_ROOT = os.path.join(DATA_DIR, "media")
SITE_ID = 1
STORAGES = {
"default": {
"BACKEND": "storages.backends.s3.S3Storage",
},
"staticfiles": {
"BACKEND": values.Value(
"whitenoise.storage.CompressedManifestStaticFilesStorage",
environ_name="STORAGES_STATICFILES_BACKEND",
),
},
}
# Internationalization
# https://docs.djangoproject.com/en/3.1/topics/i18n/
# Languages
LANGUAGE_CODE = values.Value("en-us")
# Careful! Languages should be ordered by priority, as this tuple is used to get
# fallback/default languages throughout the app.
LANGUAGES = values.SingleNestedTupleValue(
(
("en-us", _("English")),
("fr-fr", _("French")),
)
)
LOCALE_PATHS = (os.path.join(BASE_DIR, "locale"),)
TIME_ZONE = "UTC"
USE_I18N = True
USE_TZ = True
# Templates
TEMPLATES = [
{
"BACKEND": "django.template.backends.django.DjangoTemplates",
"DIRS": [os.path.join(BASE_DIR, "templates")],
"OPTIONS": {
"context_processors": [
"django.contrib.auth.context_processors.auth",
"django.contrib.messages.context_processors.messages",
"django.template.context_processors.csrf",
"django.template.context_processors.debug",
"django.template.context_processors.i18n",
"django.template.context_processors.media",
"django.template.context_processors.request",
"django.template.context_processors.tz",
],
"loaders": [
"django.template.loaders.filesystem.Loader",
"django.template.loaders.app_directories.Loader",
],
},
},
]
MIDDLEWARE = [
"django.middleware.security.SecurityMiddleware",
"whitenoise.middleware.WhiteNoiseMiddleware",
"django.contrib.sessions.middleware.SessionMiddleware",
"django.middleware.locale.LocaleMiddleware",
"django.middleware.clickjacking.XFrameOptionsMiddleware",
"corsheaders.middleware.CorsMiddleware",
"django.middleware.common.CommonMiddleware",
"django.middleware.csrf.CsrfViewMiddleware",
"django.contrib.auth.middleware.AuthenticationMiddleware",
"django.contrib.messages.middleware.MessageMiddleware",
"dockerflow.django.middleware.DockerflowMiddleware",
]
AUTHENTICATION_BACKENDS = [
"django.contrib.auth.backends.ModelBackend",
]
# Django applications from the highest priority to the lowest
INSTALLED_APPS = [
# find
"core",
"demo",
# Third party apps
"corsheaders",
"dockerflow.django",
"drf_spectacular",
"rest_framework",
# Django
"django.contrib.admin",
"django.contrib.auth",
"django.contrib.contenttypes",
"django.contrib.postgres",
"django.contrib.sessions",
"django.contrib.sites",
"django.contrib.messages",
"django.contrib.staticfiles",
# OIDC third party
"mozilla_django_oidc",
]
# Cache
CACHES = {
"default": {"BACKEND": "django.core.cache.backends.locmem.LocMemCache"},
}
REST_FRAMEWORK = {
"DEFAULT_AUTHENTICATION_CLASSES": (
"rest_framework.authentication.TokenAuthentication",
),
"DEFAULT_PARSER_CLASSES": [
"rest_framework.parsers.JSONParser",
],
"DEFAULT_PERMISSION_CLASSES": [
"rest_framework.permissions.IsAuthenticated",
],
"EXCEPTION_HANDLER": "core.api.exception_handler",
"DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
"PAGE_SIZE": 20,
"DEFAULT_VERSIONING_CLASS": "rest_framework.versioning.URLPathVersioning",
"DEFAULT_SCHEMA_CLASS": "drf_spectacular.openapi.AutoSchema",
}
# OpenSearch settings
OPENSEARCH_HOST = values.Value(
default="opensearch", environ_name="OPENSEARCH_HOST", environ_prefix=None
)
OPENSEARCH_PORT = values.Value(
default=9200, environ_name="OPENSEARCH_PORT", environ_prefix=None
)
OPENSEARCH_USER = values.Value(
default="admin", environ_name="OPENSEARCH_USER", environ_prefix=None
)
OPENSEARCH_PASSWORD = values.SecretValue(
environ_name="OPENSEARCH_PASSWORD", environ_prefix=None
)
OPENSEARCH_USE_SSL = values.BooleanValue(
default=True, environ_name="OPENSEARCH_USE_SSL", environ_prefix=None
)
SPECTACULAR_SETTINGS = {
"TITLE": "Find API",
"DESCRIPTION": "This is the find API schema.",
"VERSION": "0.1.0",
"SERVE_INCLUDE_SCHEMA": False,
"ENABLE_DJANGO_DEPLOY_CHECK": values.BooleanValue(
default=False,
environ_name="SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK",
),
"COMPONENT_SPLIT_REQUEST": True,
# OTHER SETTINGS
"SWAGGER_UI_DIST": "SIDECAR", # shorthand to use the sidecar instead
"SWAGGER_UI_FAVICON_HREF": "SIDECAR",
"REDOC_DIST": "SIDECAR",
}
AUTH_USER_MODEL = "core.User"
# CORS
CORS_ALLOW_CREDENTIALS = True
CORS_ALLOW_ALL_ORIGINS = values.BooleanValue(True)
CORS_ALLOWED_ORIGINS = values.ListValue([])
CORS_ALLOWED_ORIGIN_REGEXES = values.ListValue([])
# Sentry
SENTRY_DSN = values.Value(None, environ_name="SENTRY_DSN")
# Celery
CELERY_BROKER_URL = values.Value("redis://redis:6379/0")
CELERY_BROKER_TRANSPORT_OPTIONS = values.DictValue({})
# Session
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "default"
SESSION_COOKIE_AGE = 60 * 60 * 12
# OIDC - Authorization Code Flow
OIDC_CREATE_USER = values.BooleanValue(
default=True,
environ_name="OIDC_CREATE_USER",
)
OIDC_RP_SIGN_ALGO = values.Value(
"RS256", environ_name="OIDC_RP_SIGN_ALGO", environ_prefix=None
)
OIDC_RP_CLIENT_ID = values.Value(
"find", environ_name="OIDC_RP_CLIENT_ID", environ_prefix=None
)
OIDC_RP_CLIENT_SECRET = values.Value(
None,
environ_name="OIDC_RP_CLIENT_SECRET",
environ_prefix=None,
)
OIDC_OP_JWKS_ENDPOINT = values.Value(
environ_name="OIDC_OP_JWKS_ENDPOINT", environ_prefix=None
)
OIDC_OP_AUTHORIZATION_ENDPOINT = values.Value(
environ_name="OIDC_OP_AUTHORIZATION_ENDPOINT", environ_prefix=None
)
OIDC_OP_TOKEN_ENDPOINT = values.Value(
None, environ_name="OIDC_OP_TOKEN_ENDPOINT", environ_prefix=None
)
OIDC_OP_USER_ENDPOINT = values.Value(
None, environ_name="OIDC_OP_USER_ENDPOINT", environ_prefix=None
)
OIDC_OP_LOGOUT_ENDPOINT = values.Value(
None, environ_name="OIDC_OP_LOGOUT_ENDPOINT", environ_prefix=None
)
OIDC_AUTH_REQUEST_EXTRA_PARAMS = values.DictValue(
{}, environ_name="OIDC_AUTH_REQUEST_EXTRA_PARAMS", environ_prefix=None
)
OIDC_RP_SCOPES = values.Value(
"openid email", environ_name="OIDC_RP_SCOPES", environ_prefix=None
)
LOGIN_REDIRECT_URL = values.Value(
None, environ_name="LOGIN_REDIRECT_URL", environ_prefix=None
)
LOGIN_REDIRECT_URL_FAILURE = values.Value(
None, environ_name="LOGIN_REDIRECT_URL_FAILURE", environ_prefix=None
)
LOGOUT_REDIRECT_URL = values.Value(
None, environ_name="LOGOUT_REDIRECT_URL", environ_prefix=None
)
OIDC_USE_NONCE = values.BooleanValue(
default=True, environ_name="OIDC_USE_NONCE", environ_prefix=None
)
OIDC_REDIRECT_REQUIRE_HTTPS = values.BooleanValue(
default=False, environ_name="OIDC_REDIRECT_REQUIRE_HTTPS", environ_prefix=None
)
OIDC_REDIRECT_ALLOWED_HOSTS = values.ListValue(
default=[], environ_name="OIDC_REDIRECT_ALLOWED_HOSTS", environ_prefix=None
)
OIDC_STORE_ID_TOKEN = values.BooleanValue(
default=True, environ_name="OIDC_STORE_ID_TOKEN", environ_prefix=None
)
ALLOW_LOGOUT_GET_METHOD = values.BooleanValue(
default=True, environ_name="ALLOW_LOGOUT_GET_METHOD", environ_prefix=None
)
# pylint: disable=invalid-name
@property
def ENVIRONMENT(self):
"""Environment in which the application is launched."""
return self.__class__.__name__.lower()
# pylint: disable=invalid-name
@property
def RELEASE(self):
"""
Return the release information.
Delegate to the module function to enable easier testing.
"""
return get_release()
@classmethod
def post_setup(cls):
"""Post setup configuration.
This is the place where you can configure settings that require other
settings to be loaded.
"""
super().post_setup()
# The SENTRY_DSN setting should be available to activate sentry for an environment
if cls.SENTRY_DSN is not None:
sentry_sdk.init(
dsn=cls.SENTRY_DSN,
environment=cls.__name__.lower(),
release=get_release(),
integrations=[DjangoIntegration()],
)
with sentry_sdk.configure_scope() as scope:
scope.set_extra("application", "backend")
class Build(Base):
"""Settings used when the application is built.
This environment should not be used to run the application. Just to build it with non-blocking
settings.
"""
SECRET_KEY = values.Value("DummyKey")
STORAGES = {
"default": {
"BACKEND": "django.core.files.storage.FileSystemStorage",
},
"staticfiles": {
"BACKEND": values.Value(
"whitenoise.storage.CompressedManifestStaticFilesStorage",
environ_name="STORAGES_STATICFILES_BACKEND",
),
},
}
class Development(Base):
"""
Development environment settings
We set DEBUG to True and configure the server to respond from all hosts.
"""
ALLOWED_HOSTS = ["*"]
CORS_ALLOW_ALL_ORIGINS = True
CSRF_TRUSTED_ORIGINS = ["http://localhost:8072", "http://localhost:3000"]
DEBUG = True
SESSION_COOKIE_NAME = "find_sessionid"
USE_SWAGGER = True
def __init__(self):
# pylint: disable=invalid-name
self.INSTALLED_APPS += ["django_extensions", "drf_spectacular_sidecar"]
class Test(Base):
"""Test environment settings"""
LOGGING = values.DictValue(
{
"version": 1,
"disable_existing_loggers": False,
"handlers": {
"console": {
"class": "logging.StreamHandler",
},
},
"loggers": {
"find": {
"handlers": ["console"],
"level": "DEBUG",
},
},
}
)
PASSWORD_HASHERS = [
"django.contrib.auth.hashers.MD5PasswordHasher",
]
USE_SWAGGER = True
CELERY_TASK_ALWAYS_EAGER = values.BooleanValue(True)
def __init__(self):
# pylint: disable=invalid-name
self.INSTALLED_APPS += ["drf_spectacular_sidecar"]
class ContinuousIntegration(Test):
"""
Continuous Integration environment settings
nota bene: it should inherit from the Test environment.
"""
class Production(Base):
"""
Production environment settings
You must define the ALLOWED_HOSTS environment variable in Production
configuration (and derived configurations):
ALLOWED_HOSTS=["foo.com", "foo.fr"]
"""
# Security
ALLOWED_HOSTS = values.ListValue(None)
CSRF_TRUSTED_ORIGINS = values.ListValue([])
SECURE_BROWSER_XSS_FILTER = True
SECURE_CONTENT_TYPE_NOSNIFF = True
# SECURE_PROXY_SSL_HEADER allows to fix the scheme in Django's HttpRequest
# object when your application is behind a reverse proxy.
#
# Keep this SECURE_PROXY_SSL_HEADER configuration only if :
# - your Django app is behind a proxy.
# - your proxy strips the X-Forwarded-Proto header from all incoming requests
# - Your proxy sets the X-Forwarded-Proto header and sends it to Django
#
# In other cases, you should comment the following line to avoid security issues.
# SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")
SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")
# Modern browsers require to have the `secure` attribute on cookies with `Samesite=none`
CSRF_COOKIE_SECURE = True
SESSION_COOKIE_SECURE = True
# Privacy
SECURE_REFERRER_POLICY = "same-origin"
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": values.Value(
"redis://redis:6379/1",
environ_name="REDIS_URL",
environ_prefix=None,
),
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.DefaultClient",
},
},
}
class Feature(Production):
"""
Feature environment settings
nota bene: it should inherit from the Production environment.
"""
class Staging(Production):
"""
Staging environment settings
nota bene: it should inherit from the Production environment.
"""
class PreProduction(Production):
"""
Pre-production environment settings
nota bene: it should inherit from the Production environment.
"""
class Demo(Production):
"""
Demonstration environment settings
nota bene: it should inherit from the Production environment.
"""
STORAGES = {
"default": {
"BACKEND": "django.core.files.storage.FileSystemStorage",
},
"staticfiles": {
"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage",
},
}
-38
View File
@@ -1,38 +0,0 @@
"""URL configuration for the find project"""
from django.conf import settings
from django.contrib import admin
from django.urls import include, path, re_path
from drf_spectacular.views import (
SpectacularJSONAPIView,
SpectacularRedocView,
SpectacularSwaggerView,
)
urlpatterns = [
path("admin/", admin.site.urls),
path(f"api/{settings.API_VERSION}/", include("core.urls")),
]
if settings.USE_SWAGGER or settings.DEBUG:
urlpatterns += [
path(
f"{settings.API_VERSION}/swagger.json",
SpectacularJSONAPIView.as_view(
api_version=settings.API_VERSION,
urlconf="core.urls",
),
name="client-api-schema",
),
path(
f"{settings.API_VERSION}//swagger/",
SpectacularSwaggerView.as_view(url_name="client-api-schema"),
name="swagger-ui-schema",
),
re_path(
f"{settings.API_VERSION}//redoc/",
SpectacularRedocView.as_view(url_name="client-api-schema"),
name="redoc-schema",
),
]
-17
View File
@@ -1,17 +0,0 @@
"""
WSGI config for the find project.
It exposes the WSGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/3.1/howto/deployment/wsgi/
"""
import os
from configurations.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "find.settings")
os.environ.setdefault("DJANGO_CONFIGURATION", "Development")
application = get_wsgi_application()
-15
View File
@@ -1,15 +0,0 @@
#!/usr/bin/env python
"""
find's sandbox management script.
"""
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "find.settings")
os.environ.setdefault("DJANGO_CONFIGURATION", "Development")
from configurations.management import execute_from_command_line
execute_from_command_line(sys.argv)
-132
View File
@@ -1,132 +0,0 @@
#
# find package
#
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "find"
version = "1.1.0"
authors = [{ "name" = "DINUM", "email" = "dev@mail.numerique.gouv.fr" }]
classifiers = [
"Development Status :: 5 - Production/Stable",
"Framework :: Django",
"Framework :: Django :: 5",
"Intended Audience :: Developers",
"License :: OSI Approved :: MIT License",
"Natural Language :: English",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
]
description = "An application to print markdown to pdf from a set of managed templates."
keywords = ["Django", "Contacts", "Templates", "RBAC"]
license = { file = "LICENSE" }
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
"celery[redis]==5.4.0",
"django-configurations==2.5.1",
"django-cors-headers==4.6.0",
"redis==5.2.1",
"django-redis==5.4.0",
"django==5.1.4",
"djangorestframework==3.15.2",
"drf_spectacular==0.28.0",
"dockerflow==2024.4.2",
"factory_boy==3.3.1",
"gunicorn==23.0.0",
"mozilla-django-oidc==4.0.1",
"psycopg[binary]==3.2.3",
"pydantic==2.10.5",
"pyjwt==2.10.1",
"requests==2.32.3",
"sentry-sdk==2.19.2",
"url-normalize==1.4.3",
"opensearch-py==2.8.0",
"whitenoise==6.8.2",
]
[project.urls]
"Bug Tracker" = "https://github.com/suitenumerique/find/issues/new"
"Changelog" = "https://github.com/suitenumerique/find/blob/main/CHANGELOG.md"
"Homepage" = "https://github.com/suitenumerique/find"
"Repository" = "https://github.com/suitenumerique/find"
[project.optional-dependencies]
dev = [
"django-extensions==3.2.3",
"drf-spectacular-sidecar==2024.12.1",
"faker==33.3.0",
"ipdb==0.13.13",
"ipython==8.31.0",
"pyfakefs==5.7.3",
"pylint-django==2.6.1",
"pylint==3.3.3",
"pytest-cov==6.0.0",
"pytest-django==4.9.0",
"pytest==8.3.4",
"pytest-icdiff==0.9",
"pytest-xdist==3.6.1",
"responses==0.25.3",
"ruff==0.9.0",
"types-requests==2.32.0.20241016",
]
[tool.setuptools]
packages = { find = { where = ["."], exclude = ["tests"] } }
zip-safe = true
[tool.distutils.bdist_wheel]
universal = true
[tool.ruff]
exclude = [
".git",
".venv",
"build",
"venv",
"__pycache__",
"*/migrations/*",
]
line-length = 88
[tool.ruff.lint]
select = [
"B", # flake8-bugbear
"BLE", # flake8-blind-except
"C4", # flake8-comprehensions
"DJ", # flake8-django
"I", # isort
"PLC", # pylint-convention
"PLE", # pylint-error
"PLR", # pylint-refactoring
"PLW", # pylint-warning
"RUF100", # Ruff unused-noqa
"RUF200", # Ruff check pyproject.toml
"S", # flake8-bandit
"SLF", # flake8-self
"T20", # flake8-print
]
ignore= ["DJ001", "PLR2004"]
[tool.ruff.lint.isort]
section-order = ["future","standard-library","django","third-party","find","first-party","local-folder"]
sections = { find=["core"], django=["django"] }
[tool.ruff.lint.per-file-ignores]
"**/tests/*" = ["S", "SLF"]
[tool.pytest.ini_options]
addopts = [
"-v",
"--cov-report",
"term-missing",
# Allow test files to have the same name in different directories.
"--import-mode=importlib",
]
python_files = [
"test_*.py",
"tests.py",
]
-7
View File
@@ -1,7 +0,0 @@
#!/usr/bin/env python
"""Setup file for the find module. All configuration stands in the setup.cfg file."""
# coding: utf-8
from setuptools import setup # pylint: disable=import-error
setup()
-73
View File
@@ -1,73 +0,0 @@
djangoSuperUserEmail: ENC[AES256_GCM,data:7b1xfYmr1g0RlBmsHBRA39ZPV/6+1DrtHQ==,iv:/GW7oLxPTZYmRWVPvyAQMoZl1owHM4Fo0XAOtyEh2rA=,tag:DaqoW+dglyAOXMm5+mrDfA==,type:str]
djangoSuperUserPass: ENC[AES256_GCM,data:RQgX,iv:q3CdfmwGfHSTjLXTimDk/1MyoFLviRuwmZa2E7GUzhY=,tag:HCtdtqgSxdJIHFhI8xpegQ==,type:str]
djangoSecretKey: ENC[AES256_GCM,data:mtJCf6mKfj/fJkg4wmfIvvU1vkUEF77BI8TUFikp/M3nPveDXhKmy3Cw3cXFpOYiFZ0=,iv:qwPRKsPS1Jhylj5asbmknXm1xOX3nfp9iccuorUrcj0=,tag:ENVfAt4i3PttoqD8+Kc4wQ==,type:str]
opensearch:
password: ENC[AES256_GCM,data:y4iWSEwW5u1W3vPx,iv:1A9qIUELbqSIdKL+CXn1CshR3g1jMJVgw+VsJHZU9Q8=,tag:6S6RKSC5mnUZP3epcpd5jQ==,type:str]
oidc:
clientId: ENC[AES256_GCM,data:wndPCbysbWDybdHglcG+wkMWk1rrD40hKqFxct9T3TLEGOk/,iv:RH1OdBX1GYIT90sSq0AGz49fFi6dL0m49Pegs6Ko9tQ=,tag:/tKytQwoZkBX1Tf96gAjIA==,type:str]
clientSecret: ENC[AES256_GCM,data:MUJ0wsg+LC2QZ1jZ0Twd3FS3dQevmJq9/97qVI3ARHuJIVlQz0Qah4vE7/iR+sn7ME2o1s1AzV4c1Yx/F3nHBg==,iv:LvinICSzF/8EvrHZD4Jp6lt7g3yxSOEgVHPrc3SShjo=,tag:yvkyyBXmhEkmGL7jZevUCA==,type:str]
sops:
kms: []
gcp_kms: []
azure_kv: []
hc_vault: []
age:
- recipient: age15fyxdwmg5mvldtqqus87xspuws2u0cpvwheehrtvkexj4tnsqqysw6re2x
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSAwdFdnd1RnMkE1blN2YWhK
Q0xYYnpTbk0rVkxEQ2w4a0hrcjZ4TURxQ1ZnCmNyakFSMmU2S0c0cWhLRFpSckk4
bU1kOXlZeFU1S05DN1RCYWpaMTJhM1EKLS0tIHZWREtDVFU3U1djYWsrNzRxeTBW
TmRSbzlRUGpMZHF0NFdDTVpwQURmM1kKUJbmN8UE/7zhVbK5QRzfdCSTiHM3OWai
4ENxdmo1A58De7GE+kb4yMIym6aFa+LnQ0qCVkuZ+Y6i4Ipu/9s3DQ==
-----END AGE ENCRYPTED FILE-----
- recipient: age16hnlml8yv4ynwy0seer57g8qww075crd0g7nsundz3pj4wk7m3vqftszg7
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBsd0ZMZlJscC9LQktDVHVr
UFEzSEI5QUk1VFA5YU93Y1RBNCtpQzg3RDNvCjdKOStUcmsxamNpLzE4VWs2WWNF
MTA3eGNmMU9zSndMZVVvYy9ZMVRuNzgKLS0tICtuT1ZlbVJQMTZNVWRJZllnUXZ1
eDIxczdXOTEza0lYYzZRWERaVUVKdVEKLICxvqmzEOTqDBgcaTMU8XlbpSQPiH6u
gHwTXezsaOwjzdviyAeyhNPq3N1vauM2L8NW/0sWSq9qFZAi92uw6A==
-----END AGE ENCRYPTED FILE-----
- recipient: age1plkp8td6zzfcavjusmsfrlk54t9vn8jjxm8zaz7cmnr7kzl2nfnsd54hwg
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBYcVF3UURjNzFPbzJqRmZu
RGd0QWptWXRQZXVDWVJ4bHA5MllWOEU4N25NCldJa1ZNb0xURFcxMERjbFVMZk01
WHpKY3BWai83SmE4bDl2R2pKR1NGb1UKLS0tIFpJZHpRRUN1aGhTUytTSVA1K1U3
bUJNZUtsSHFZMnRFaUZtZEZmSzN5RncK2hBeMQHxCWCabS0wWV6LC3Shhj5g4LKc
J44oWmIx/Aaz3Jnf73E4aE8/wW5L593oqP3wzT4dermd9OKcEvU2gQ==
-----END AGE ENCRYPTED FILE-----
- recipient: age12g6f5fse25tgrwweleh4jls3qs52hey2edh759smulwmk5lnzadslu2cp3
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBvTlJuQ0pndFFpQ1JxZ3hR
dVZjVDVVd09JcmhQVWpDYnpxSjkrM25VTFZnCkk2SU9QVUJQQWdyUjVPWFRNbU44
RnlKcEJ6RHJyU2lMOTNwbkE1QnBzK2sKLS0tIC80eXNIVTZQa0JjbnpZQkJkRlJq
a1NGa2pNOE9reG84QlMzaDRhQmhjZDgKFihYgODIysk5jPWKSvTckayDx0jKRtKc
FUmsLQC87bJdY6/HGnRSB3xI7ZJYlX12G2lfshOY0ZpQtkRgLtzOBg==
-----END AGE ENCRYPTED FILE-----
- recipient: age1hnhuzj96ktkhpyygvmz0x9h8mfvssz7ss6emmukags644mdhf4msajk93r
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBRcmlzOG44S0VZUVJvTXZ0
aDEybUhBY3kzWG5TMDRpZUtiai9DTE1sU0RFCklKV2Y0eUZOUHhRSEZPamQxMXNp
QmxsZEppMWkrTzRRWnJLTHByNVRuZ1EKLS0tIHQ0bThtaUl2eXE4TlZWck5CbUh4
ZXZnKyt0a0hyTEZRbGhvNGkvTEFsem8Kc0SgSfI8U+nfB9ZqID/dh0IYWl4dBhl3
kpHAn9OiaKXZs9dt6Um2H/ACOOKjRlRYoaybEAeIP5d5RLV8dDVcOw==
-----END AGE ENCRYPTED FILE-----
- recipient: age19drtvac5c0e6t4da24lw8jq9k3hrcwyd3pdj78kxy9kllalxx3dsz9ttx4
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBHelFUdlVrTkIvdnlETDli
dGJHc3pTRjV1VWRNQWhTWkVVNkxIb3QzNTJjCm4zdWVFbmo0Y1IzdzhYYjBuM3B5
QlhHaHVqaXhMUGRPU1lQZUN6RUJ5SDgKLS0tIGd6ZWVEc2pWcDNGSE5OMWhBZm12
b0tsOEExQ1gyTEx3VVR3c1MrRkw3QkUK7pLI3R0p+07URlC3nTsXEnxkbX3KuIxv
JjsDoCnG+p45moQYzSh6tokfIxJKQDt4EJCuQCgHU5twb17aWt9Q7A==
-----END AGE ENCRYPTED FILE-----
lastmodified: "2025-01-08T14:01:47Z"
mac: ENC[AES256_GCM,data:hVzGwnVOD1snyhLF6ov6bzbce0F9kXIaLAYRlKFzIzkFXlMpud/yOlITqIX1LWp5Opbl5VUc7GGp4PkLUm5bi+uRClLr93/qtyZa0Lp71SDyKnvhOKfNaJsmWWxKHHiFsKPT92xxxjf/m4/KK1hpiOy4SjSVtJoIDVTZrSN+RPQ=,iv:U2HqHsf0KO6KibZylg7SoQ0ART72OKwkLWiW11fukrs=,tag:gm4jXekjLJpbvjpypGk6KA==,type:str]
pgp: []
unencrypted_suffix: _unencrypted
version: 3.9.3
@@ -1,95 +0,0 @@
image:
repository: localhost:5001/find-backend
pullPolicy: Always
tag: "latest"
backend:
replicas: 1
envVars:
DJANGO_CSRF_TRUSTED_ORIGINS: https://find.127.0.0.1.nip.io,http://find.127.0.0.1.nip.io
DJANGO_CONFIGURATION: Production
DJANGO_ALLOWED_HOSTS: "*"
DJANGO_SECRET_KEY:
secretKeyRef:
name: find-backend
key: DJANGO_SECRET_KEY
DJANGO_SETTINGS_MODULE: find.settings
DJANGO_SUPERUSER_EMAIL:
secretKeyRef:
name: find-backend
key: DJANGO_SUPERUSER_EMAIL
DJANGO_SUPERUSER_PASSWORD:
secretKeyRef:
name: find-backend
key: DJANGO_SUPERUSER_PASSWORD
DJANGO_EMAIL_HOST: "mailcatcher"
DJANGO_EMAIL_PORT: 1025
DJANGO_EMAIL_USE_SSL: False
OIDC_OP_JWKS_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/jwks
OIDC_OP_AUTHORIZATION_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/authorize
OIDC_OP_TOKEN_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/token
OIDC_OP_USER_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/userinfo
OIDC_OP_LOGOUT_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/session/end
OIDC_RP_CLIENT_ID:
secretKeyRef:
name: find-backend
key: OIDC_RP_CLIENT_ID
OIDC_RP_CLIENT_SECRET:
secretKeyRef:
name: find-backend
key: OIDC_RP_CLIENT_SECRET
OIDC_RP_SIGN_ALGO: RS256
OIDC_RP_SCOPES: "openid email"
OIDC_REDIRECT_ALLOWED_HOSTS: https://find.127.0.0.1.nip.io
OIDC_AUTH_REQUEST_EXTRA_PARAMS: "{'acr_values': 'eidas1'}"
LOGIN_REDIRECT_URL: https://find.127.0.0.1.nip.io
LOGIN_REDIRECT_URL_FAILURE: https://find.127.0.0.1.nip.io
LOGOUT_REDIRECT_URL: https://find.127.0.0.1.nip.io
DB_HOST: postgres-postgresql
DB_NAME: find
DB_USER: dinum
DB_PASSWORD: pass
DB_PORT: 5432
POSTGRES_DB: find
POSTGRES_USER: dinum
POSTGRES_PASSWORD: pass
REDIS_URL: redis://default:pass@redis-master:6379/1
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
OPENSEARCH_HOST: opensearch
OPENSEARCH_USE_SSL: False
OPENSEARCH_PORT: 9200
OPENSEARCH_PASSWORD:
secretKeyRef:
name: find-backend
key: OPENSEARCH_PASSWORD
migrate:
command:
- "/bin/bash"
- "-c"
- |
python manage.py migrate --no-input
restartPolicy: Never
command:
- "gunicorn"
- "-c"
- "/usr/local/etc/gunicorn/find.py"
- "find.wsgi:application"
- "--reload"
createsuperuser:
command:
- "/bin/bash"
- "-c"
- |
python manage.py shell -c "from core.models import User; not User.objects.filter(username='admin').exists() and User.objects.create_superuser('admin', 'admin@example.com', 'admin')"
restartPolicy: Never
ingress:
enabled: true
host: find.127.0.0.1.nip.io
ingressAdmin:
enabled: true
host: find.127.0.0.1.nip.io
-1
View File
@@ -1 +0,0 @@
../../../../secrets/numerique-gouv/impress/env/preprod/secrets.enc.yaml
@@ -1,123 +0,0 @@
image:
repository: lasuite/find-backend
pullPolicy: Always
tag: "v1.1.0"
backend:
migrateJobAnnotations:
argocd.argoproj.io/hook: PreSync
argocd.argoproj.io/hook-delete-policy: HookSucceeded
envVars:
DJANGO_CSRF_TRUSTED_ORIGINS: http://find-preprod.beta.numerique.gouv.fr,https://find-preprod.beta.numerique.gouv.fr
DJANGO_CONFIGURATION: Production
DJANGO_ALLOWED_HOSTS: "*"
DJANGO_SUPERUSER_EMAIL:
secretKeyRef:
name: find-backend
key: DJANGO_SUPERUSER_EMAIL
DJANGO_SECRET_KEY:
secretKeyRef:
name: find-backend
key: DJANGO_SECRET_KEY
DJANGO_SETTINGS_MODULE: find.settings
DJANGO_SUPERUSER_PASSWORD:
secretKeyRef:
name: find-backend
key: DJANGO_SUPERUSER_PASSWORD
DJANGO_EMAIL_HOST: "snap-mail.numerique.gouv.fr"
DJANGO_EMAIL_PORT: 465
DJANGO_EMAIL_USE_SSL: True
DJANGO_SILENCED_SYSTEM_CHECKS: security.W008,security.W004
OIDC_OP_JWKS_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/jwks
OIDC_OP_AUTHORIZATION_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/authorize
OIDC_OP_TOKEN_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/token
OIDC_OP_USER_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/userinfo
OIDC_OP_LOGOUT_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/session/end
OIDC_RP_CLIENT_ID:
secretKeyRef:
name: find-backend
key: OIDC_RP_CLIENT_ID
OIDC_RP_CLIENT_SECRET:
secretKeyRef:
name: find-backend
key: OIDC_RP_CLIENT_SECRET
OIDC_RP_SIGN_ALGO: RS256
OIDC_RP_SCOPES: "openid email"
OIDC_REDIRECT_ALLOWED_HOSTS: https://find-preprod.beta.numerique.gouv.fr
OIDC_AUTH_REQUEST_EXTRA_PARAMS: "{'acr_values': 'eidas1'}"
LOGIN_REDIRECT_URL: https://find-preprod.beta.numerique.gouv.fr
LOGIN_REDIRECT_URL_FAILURE: https://find-preprod.beta.numerique.gouv.fr
LOGOUT_REDIRECT_URL: https://find-preprod.beta.numerique.gouv.fr
DB_HOST:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: host
DB_NAME:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: database
DB_USER:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: username
DB_PASSWORD:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: password
DB_PORT:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: port
REDIS_URL:
secretKeyRef:
name: redis.redis.libre.sh
key: url
AWS_S3_ENDPOINT_URL:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: url
AWS_S3_ACCESS_KEY_ID:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: accessKey
AWS_S3_SECRET_ACCESS_KEY:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: secretKey
AWS_STORAGE_BUCKET_NAME:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: bucket
AWS_S3_REGION_NAME: local
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
OPENSEARCH_PASSWORD:
secretKeyRef:
name: find-backend
key: OPENSEARCH_PASSWORD
createsuperuser:
command:
- "/bin/sh"
- "-c"
- |
python manage.py createsuperuser --email $DJANGO_SUPERUSER_EMAIL --password $DJANGO_SUPERUSER_PASSWORD
restartPolicy: Never
ingress:
enabled: true
host: find-preprod.beta.numerique.gouv.fr
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: htaccess
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'
ingressAdmin:
enabled: true
host: find-preprod.beta.numerique.gouv.fr
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/auth-signin: https://oauth2-proxy-preprod.beta.numerique.gouv.fr/oauth2/start
nginx.ingress.kubernetes.io/auth-url: https://oauth2-proxy-preprod.beta.numerique.gouv.fr/oauth2/auth
@@ -1 +0,0 @@
../../../../secrets/numerique-gouv/impress/env/production/secrets.enc.yaml
@@ -1,123 +0,0 @@
image:
repository: lasuite/find-backend
pullPolicy: Always
tag: "v1.1.0"
backend:
migrateJobAnnotations:
argocd.argoproj.io/hook: PostSync
argocd.argoproj.io/hook-delete-policy: HookSucceeded
envVars:
DJANGO_CSRF_TRUSTED_ORIGINS: https://docs.numerique.gouv.fr
DJANGO_CONFIGURATION: Production
DJANGO_ALLOWED_HOSTS: "*"
DJANGO_SECRET_KEY:
secretKeyRef:
name: find-backend
key: DJANGO_SECRET_KEY
DJANGO_SETTINGS_MODULE: find.settings
DJANGO_SUPERUSER_EMAIL:
secretKeyRef:
name: find-backend
key: DJANGO_SUPERUSER_EMAIL
DJANGO_SUPERUSER_PASSWORD:
secretKeyRef:
name: find-backend
key: DJANGO_SUPERUSER_PASSWORD
DJANGO_EMAIL_HOST: "snap-mail.numerique.gouv.fr"
DJANGO_EMAIL_PORT: 465
DJANGO_EMAIL_USE_SSL: True
DJANGO_SILENCED_SYSTEM_CHECKS: security.W008,security.W004
OIDC_OP_JWKS_ENDPOINT: https://auth.agentconnect.gouv.fr/api/v2/jwks
OIDC_OP_AUTHORIZATION_ENDPOINT: https://auth.agentconnect.gouv.fr/api/v2/authorize
OIDC_OP_TOKEN_ENDPOINT: https://auth.agentconnect.gouv.fr/api/v2/token
OIDC_OP_USER_ENDPOINT: https://auth.agentconnect.gouv.fr/api/v2/userinfo
OIDC_OP_LOGOUT_ENDPOINT: https://auth.agentconnect.gouv.fr/api/v2/session/end
OIDC_RP_CLIENT_ID:
secretKeyRef:
name: find-backend
key: OIDC_RP_CLIENT_ID
OIDC_RP_CLIENT_SECRET:
secretKeyRef:
name: find-backend
key: OIDC_RP_CLIENT_SECRET
OIDC_RP_SIGN_ALGO: RS256
OIDC_RP_SCOPES: "openid email"
OIDC_REDIRECT_ALLOWED_HOSTS: https://docs.numerique.gouv.fr
OIDC_AUTH_REQUEST_EXTRA_PARAMS: "{'acr_values': 'eidas1'}"
LOGIN_REDIRECT_URL: https://docs.numerique.gouv.fr
LOGIN_REDIRECT_URL_FAILURE: https://docs.numerique.gouv.fr
LOGOUT_REDIRECT_URL: https://docs.numerique.gouv.fr
DB_HOST:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: host
DB_NAME:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: database
DB_USER:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: username
DB_PASSWORD:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: password
DB_PORT:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: port
REDIS_URL:
secretKeyRef:
name: redis.redis.libre.sh
key: url
AWS_S3_ENDPOINT_URL:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: url
AWS_S3_ACCESS_KEY_ID:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: accessKey
AWS_S3_SECRET_ACCESS_KEY:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: secretKey
AWS_STORAGE_BUCKET_NAME:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: bucket
AWS_S3_REGION_NAME: local
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
OPENSEARCH_PASSWORD:
secretKeyRef:
name: find-backend
key: OPENSEARCH_PASSWORD
createsuperuser:
command:
- "/bin/sh"
- "-c"
- |
python manage.py createsuperuser --email $DJANGO_SUPERUSER_EMAIL --password $DJANGO_SUPERUSER_PASSWORD
restartPolicy: Never
ingress:
enabled: true
host: docs.numerique.gouv.fr
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: htaccess
nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'
ingressAdmin:
enabled: true
host: docs.numerique.gouv.fr
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt
nginx.ingress.kubernetes.io/auth-signin: https://oauth2-proxy.beta.numerique.gouv.fr/oauth2/start
nginx.ingress.kubernetes.io/auth-url: https://oauth2-proxy.beta.numerique.gouv.fr/oauth2/auth
-1
View File
@@ -1 +0,0 @@
../../../../secrets/numerique-gouv/impress/env/staging/secrets.enc.yaml
@@ -1,120 +0,0 @@
image:
repository: lasuite/find-backend
pullPolicy: Always
tag: "main"
backend:
migrateJobAnnotations:
argocd.argoproj.io/hook: PreSync
argocd.argoproj.io/hook-delete-policy: HookSucceeded
envVars:
DJANGO_CSRF_TRUSTED_ORIGINS: http://find-staging.beta.numerique.gouv.fr,https://find-staging.beta.numerique.gouv.fr
DJANGO_CONFIGURATION: Production
DJANGO_ALLOWED_HOSTS: "*"
DJANGO_SECRET_KEY:
secretKeyRef:
name: find-backend
key: DJANGO_SECRET_KEY
DJANGO_SETTINGS_MODULE: find.settings
DJANGO_SUPERUSER_EMAIL:
secretKeyRef:
name: find-backend
key: DJANGO_SUPERUSER_EMAIL
DJANGO_SUPERUSER_PASSWORD:
secretKeyRef:
name: find-backend
key: DJANGO_SUPERUSER_PASSWORD
DJANGO_EMAIL_HOST: "snap-mail.numerique.gouv.fr"
DJANGO_EMAIL_PORT: 465
DJANGO_EMAIL_USE_SSL: True
DJANGO_SILENCED_SYSTEM_CHECKS: security.W008,security.W004
OIDC_OP_JWKS_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/jwks
OIDC_OP_AUTHORIZATION_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/authorize
OIDC_OP_TOKEN_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/token
OIDC_OP_USER_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/userinfo
OIDC_OP_LOGOUT_ENDPOINT: https://fca.integ01.dev-agentconnect.fr/api/v2/session/end
OIDC_RP_CLIENT_ID:
secretKeyRef:
name: find-backend
key: OIDC_RP_CLIENT_ID
OIDC_RP_CLIENT_SECRET:
secretKeyRef:
name: find-backend
key: OIDC_RP_CLIENT_SECRET
OIDC_RP_SIGN_ALGO: RS256
OIDC_RP_SCOPES: "openid email"
OIDC_REDIRECT_ALLOWED_HOSTS: https://find-staging.beta.numerique.gouv.fr
OIDC_AUTH_REQUEST_EXTRA_PARAMS: "{'acr_values': 'eidas1'}"
LOGIN_REDIRECT_URL: https://find-staging.beta.numerique.gouv.fr
LOGIN_REDIRECT_URL_FAILURE: https://find-staging.beta.numerique.gouv.fr
LOGOUT_REDIRECT_URL: https://find-staging.beta.numerique.gouv.fr
DB_HOST:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: host
DB_NAME:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: database
DB_USER:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: username
DB_PASSWORD:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: password
DB_PORT:
secretKeyRef:
name: postgresql.postgres.libre.sh
key: port
REDIS_URL:
secretKeyRef:
name: redis.redis.libre.sh
key: url
AWS_S3_ENDPOINT_URL:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: url
AWS_S3_ACCESS_KEY_ID:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: accessKey
AWS_S3_SECRET_ACCESS_KEY:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: secretKey
AWS_STORAGE_BUCKET_NAME:
secretKeyRef:
name: find-media-storage.bucket.libre.sh
key: bucket
AWS_S3_REGION_NAME: local
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
OPENSEARCH_PASSWORD:
secretKeyRef:
name: find-backend
key: OPENSEARCH_PASSWORD
createsuperuser:
command:
- "/bin/sh"
- "-c"
- |
python manage.py createsuperuser --email $DJANGO_SUPERUSER_EMAIL --password $DJANGO_SUPERUSER_PASSWORD
restartPolicy: Never
ingress:
enabled: true
host: find-staging.beta.numerique.gouv.fr
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
ingressAdmin:
enabled: true
host: find-staging.beta.numerique.gouv.fr
className: nginx
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/auth-signin: https://oauth2-proxy-preprod.beta.numerique.gouv.fr/oauth2/start
nginx.ingress.kubernetes.io/auth-url: https://oauth2-proxy-preprod.beta.numerique.gouv.fr/oauth2/auth
-5
View File
@@ -1,5 +0,0 @@
apiVersion: v2
name: extra
description: A Helm chart to add some manifests to find
type: application
version: 0.1.0
-7
View File
@@ -1,7 +0,0 @@
apiVersion: core.libre.sh/v1alpha1
kind: Redis
metadata:
name: redis
namespace: {{ .Release.Namespace | quote }}
spec:
disableAuth: false
-7
View File
@@ -1,7 +0,0 @@
apiVersion: core.libre.sh/v1alpha1
kind: Postgres
metadata:
name: postgresql
namespace: {{ .Release.Namespace | quote }}
spec:
database: find
-8
View File
@@ -1,8 +0,0 @@
apiVersion: core.libre.sh/v1alpha1
kind: Bucket
metadata:
name: find-media-storage
namespace: {{ .Release.Namespace | quote }}
spec:
provider: data
versioned: true
-4
View File
@@ -1,4 +0,0 @@
apiVersion: v2
type: application
name: find
version: 0.0.1
-80
View File
@@ -1,80 +0,0 @@
# Find helm chart
## Parameters
### General configuration
| Name | Description | Value |
| ------------------------------------------ | ---------------------------------------------------- | ------------------------ |
| `image.repository` | Repository to use to pull find's container image | `lasuite/find-backend` |
| `image.tag` | find's container tag | `latest` |
| `image.pullPolicy` | Container image pull policy | `IfNotPresent` |
| `image.credentials.username` | Username for container registry authentication | |
| `image.credentials.password` | Password for container registry authentication | |
| `image.credentials.registry` | Registry url for which the credentials are specified | |
| `image.credentials.name` | Name of the generated secret for imagePullSecrets | |
| `nameOverride` | Override the chart name | `""` |
| `fullnameOverride` | Override the full application name | `""` |
| `ingress.enabled` | whether to enable the Ingress or not | `false` |
| `ingress.className` | IngressClass to use for the Ingress | `nil` |
| `ingress.host` | Host for the Ingress | `find.example.com` |
| `ingress.path` | Path to use for the Ingress | `/` |
| `ingress.hosts` | Additional host to configure for the Ingress | `[]` |
| `ingress.tls.enabled` | Weather to enable TLS for the Ingress | `true` |
| `ingress.tls.additional[].secretName` | Secret name for additional TLS config | |
| `ingress.tls.additional[].hosts[]` | Hosts for additional TLS config | |
| `ingress.customBackends` | Add custom backends to ingress | `[]` |
| `ingressAdmin.enabled` | whether to enable the Ingress or not | `false` |
| `ingressAdmin.className` | IngressClass to use for the Ingress | `nil` |
| `ingressAdmin.host` | Host for the Ingress | `find.example.com` |
| `ingressAdmin.path` | Path to use for the Ingress | `/admin` |
| `ingressAdmin.hosts` | Additional host to configure for the Ingress | `[]` |
| `ingressAdmin.tls.enabled` | Weather to enable TLS for the Ingress | `true` |
| `ingressAdmin.tls.additional[].secretName` | Secret name for additional TLS config | |
| `ingressAdmin.tls.additional[].hosts[]` | Hosts for additional TLS config | |
### backend
| Name | Description | Value |
| ----------------------------------------------------- | ---------------------------------------------------------------------------------- | ----------------------------------------------- |
| `backend.command` | Override the backend container command | `[]` |
| `backend.args` | Override the backend container args | `[]` |
| `backend.replicas` | Amount of backend replicas | `3` |
| `backend.shareProcessNamespace` | Enable share process namespace between containers | `false` |
| `backend.sidecars` | Add sidecars containers to backend deployment | `[]` |
| `backend.securityContext` | Configure backend Pod security context | `nil` |
| `backend.envVars` | Configure backend container environment variables | `undefined` |
| `backend.envVars.BY_VALUE` | Example environment variable by setting value directly | |
| `backend.envVars.FROM_CONFIGMAP.configMapKeyRef.name` | Name of a ConfigMap when configuring env vars from a ConfigMap | |
| `backend.envVars.FROM_CONFIGMAP.configMapKeyRef.key` | Key within a ConfigMap when configuring env vars from a ConfigMap | |
| `backend.envVars.FROM_SECRET.secretKeyRef.name` | Name of a Secret when configuring env vars from a Secret | |
| `backend.envVars.FROM_SECRET.secretKeyRef.key` | Key within a Secret when configuring env vars from a Secret | |
| `backend.podAnnotations` | Annotations to add to the backend Pod | `{}` |
| `backend.service.type` | backend Service type | `ClusterIP` |
| `backend.service.port` | backend Service listening port | `80` |
| `backend.service.targetPort` | backend container listening port | `8000` |
| `backend.service.annotations` | Annotations to add to the backend Service | `{}` |
| `backend.migrate.command` | backend migrate command | `["python","manage.py","migrate","--no-input"]` |
| `backend.migrate.restartPolicy` | backend migrate job restart policy | `Never` |
| `backend.probes.liveness.path` | Configure path for backend HTTP liveness probe | `/__heartbeat__` |
| `backend.probes.liveness.targetPort` | Configure port for backend HTTP liveness probe | `undefined` |
| `backend.probes.liveness.initialDelaySeconds` | Configure initial delay for backend liveness probe | `10` |
| `backend.probes.liveness.initialDelaySeconds` | Configure timeout for backend liveness probe | `10` |
| `backend.probes.startup.path` | Configure path for backend HTTP startup probe | `undefined` |
| `backend.probes.startup.targetPort` | Configure port for backend HTTP startup probe | `undefined` |
| `backend.probes.startup.initialDelaySeconds` | Configure initial delay for backend startup probe | `undefined` |
| `backend.probes.startup.initialDelaySeconds` | Configure timeout for backend startup probe | `undefined` |
| `backend.probes.readiness.path` | Configure path for backend HTTP readiness probe | `/__lbheartbeat__` |
| `backend.probes.readiness.targetPort` | Configure port for backend HTTP readiness probe | `undefined` |
| `backend.probes.readiness.initialDelaySeconds` | Configure initial delay for backend readiness probe | `10` |
| `backend.probes.readiness.initialDelaySeconds` | Configure timeout for backend readiness probe | `10` |
| `backend.resources` | Resource requirements for the backend container | `{}` |
| `backend.nodeSelector` | Node selector for the backend Pod | `{}` |
| `backend.tolerations` | Tolerations for the backend Pod | `[]` |
| `backend.affinity` | Affinity for the backend Pod | `{}` |
| `backend.persistence` | Additional volumes to create and mount on the backend. Used for debugging purposes | `{}` |
| `backend.persistence.volume-name.size` | Size of the additional volume | |
| `backend.persistence.volume-name.type` | Type of the additional volume, persistentVolumeClaim or emptyDir | |
| `backend.persistence.volume-name.mountPath` | Path where the volume should be mounted to | |
| `backend.extraVolumeMounts` | Additional volumes to mount on the backend. | `[]` |
| `backend.extraVolumes` | Additional volumes to mount on the backend. | `[]` |
-10
View File
@@ -1,10 +0,0 @@
#!/bin/bash
docker image ls | grep readme-generator-for-helm
if [ "$?" -ne "0" ]; then
git clone https://github.com/bitnami/readme-generator-for-helm.git /tmp/readme-generator-for-helm
cd /tmp/readme-generator-for-helm
docker build -t readme-generator-for-helm:latest .
cd $(dirname -- "${BASH_SOURCE[0]}")
fi
docker run --rm -it -v ./values.yaml:/app/values.yaml -v ./README.md:/app/README.md readme-generator-for-helm:latest readme-generator -v values.yaml -r README.md
-175
View File
@@ -1,175 +0,0 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "find.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "find.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "find.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
find.labels
*/}}
{{- define "find.labels" -}}
helm.sh/chart: {{ include "find.chart" . }}
{{ include "find.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "find.selectorLabels" -}}
app.kubernetes.io/name: {{ include "find.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
transform dictionnary of environment variables
Usage : {{ include "find.env.transformDict" .Values.envVars }}
Example:
envVars:
# Using simple strings as env vars
ENV_VAR_NAME: "envVar value"
# Using a value from a configMap
ENV_VAR_FROM_CM:
configMapKeyRef:
name: cm-name
key: "key_in_cm"
# Using a value from a secret
ENV_VAR_FROM_SECRET:
secretKeyRef:
name: secret-name
key: "key_in_secret"
*/}}
{{- define "find.env.transformDict" -}}
{{- range $key, $value := . }}
- name: {{ $key | quote }}
{{- if $value | kindIs "map" }}
valueFrom: {{ $value | toYaml | nindent 4 }}
{{- else }}
value: {{ $value | quote }}
{{- end }}
{{- end }}
{{- end }}
{{/*
find env vars
*/}}
{{- define "find.common.env" -}}
{{- $topLevelScope := index . 0 -}}
{{- $workerScope := index . 1 -}}
{{- include "find.env.transformDict" $workerScope.envVars -}}
{{- end }}
{{/*
Common labels
Requires array with top level scope and component name
*/}}
{{- define "find.common.labels" -}}
{{- $topLevelScope := index . 0 -}}
{{- $component := index . 1 -}}
{{- include "find.labels" $topLevelScope }}
app.kubernetes.io/component: {{ $component }}
{{- end }}
{{/*
Common selector labels
Requires array with top level scope and component name
*/}}
{{- define "find.common.selectorLabels" -}}
{{- $topLevelScope := index . 0 -}}
{{- $component := index . 1 -}}
{{- include "find.selectorLabels" $topLevelScope }}
app.kubernetes.io/component: {{ $component }}
{{- end }}
{{- define "find.probes.abstract" -}}
{{- if .exec -}}
exec:
{{- toYaml .exec | nindent 2 }}
{{- else if .tcpSocket -}}
tcpSocket:
{{- toYaml .tcpSocket | nindent 2 }}
{{- else -}}
httpGet:
path: {{ .path }}
port: {{ .targetPort }}
{{- end }}
initialDelaySeconds: {{ .initialDelaySeconds | eq nil | ternary 0 .initialDelaySeconds }}
timeoutSeconds: {{ .timeoutSeconds | eq nil | ternary 1 .timeoutSeconds }}
{{- end }}
{{/*
Full name for the backend
Requires top level scope
*/}}
{{- define "find.backend.fullname" -}}
{{ include "find.fullname" . }}-backend
{{- end }}
{{/*
Full name for the webrtc
Requires top level scope
*/}}
{{- define "find.webrtc.fullname" -}}
{{ include "find.fullname" . }}-webrtc
{{- end }}
{{/*
Usage : {{ include "find.secret.dockerconfigjson.name" (dict "fullname" (include "find.fullname" .) "imageCredentials" .Values.path.to.the.image1) }}
*/}}
{{- define "find.secret.dockerconfigjson.name" }}
{{- if (default (dict) .imageCredentials).name }}{{ .imageCredentials.name }}{{ else }}{{ .fullname | trunc 63 | trimSuffix "-" }}-dockerconfig{{ end -}}
{{- end }}
{{/*
Usage : {{ include "find.secret.dockerconfigjson" (dict "fullname" (include "find.fullname" .) "imageCredentials" .Values.path.to.the.image1) }}
*/}}
{{- define "find.secret.dockerconfigjson" }}
{{- if .imageCredentials -}}
apiVersion: v1
kind: Secret
metadata:
name: {{ template "find.secret.dockerconfigjson.name" (dict "fullname" .fullname "imageCredentials" .imageCredentials) }}
annotations:
"helm.sh/hook": pre-install,pre-upgrade
"helm.sh/hook-weight": "-5"
"helm.sh/hook-delete-policy": before-hook-creation
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: {{ template "find.secret.dockerconfigjson.data" .imageCredentials }}
{{- end -}}
{{- end }}
@@ -1,136 +0,0 @@
{{- $envVars := include "find.common.env" (list . .Values.backend) -}}
{{- $fullName := include "find.backend.fullname" . -}}
{{- $component := "backend" -}}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ $fullName }}
namespace: {{ .Release.Namespace | quote }}
labels:
{{- include "find.common.labels" (list . $component) | nindent 4 }}
spec:
replicas: {{ .Values.backend.replicas }}
selector:
matchLabels:
{{- include "find.common.selectorLabels" (list . $component) | nindent 6 }}
template:
metadata:
annotations:
{{- with .Values.backend.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "find.common.selectorLabels" (list . $component) | nindent 8 }}
spec:
{{- if $.Values.image.credentials }}
imagePullSecrets:
- name: {{ include "find.secret.dockerconfigjson.name" (dict "fullname" (include "find.fullname" .) "imageCredentials" $.Values.image.credentials) }}
{{- end}}
shareProcessNamespace: {{ .Values.backend.shareProcessNamespace }}
containers:
{{- with .Values.backend.sidecars }}
{{- toYaml . | nindent 8 }}
{{- end }}
- name: {{ .Chart.Name }}
image: "{{ (.Values.backend.image | default dict).repository | default .Values.image.repository }}:{{ (.Values.backend.image | default dict).tag | default .Values.image.tag }}"
imagePullPolicy: {{ (.Values.backend.image | default dict).pullPolicy | default .Values.image.pullPolicy }}
{{- with .Values.backend.command }}
command:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.backend.args }}
args:
{{- toYaml . | nindent 12 }}
{{- end }}
env:
{{- if $envVars}}
{{- $envVars | indent 12 }}
{{- end }}
{{- with .Values.backend.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
ports:
- name: http
containerPort: {{ .Values.backend.service.targetPort }}
protocol: TCP
{{- if .Values.backend.probes.liveness }}
livenessProbe:
{{- include "find.probes.abstract" (merge .Values.backend.probes.liveness (dict "targetPort" .Values.backend.service.targetPort )) | nindent 12 }}
{{- end }}
{{- if .Values.backend.probes.readiness }}
readinessProbe:
{{- include "find.probes.abstract" (merge .Values.backend.probes.readiness (dict "targetPort" .Values.backend.service.targetPort )) | nindent 12 }}
{{- end }}
{{- if .Values.backend.probes.startup }}
startupProbe:
{{- include "find.probes.abstract" (merge .Values.backend.probes.startup (dict "targetPort" .Values.backend.service.targetPort )) | nindent 12 }}
{{- end }}
{{- with .Values.backend.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
volumeMounts:
{{- range $index, $value := .Values.mountFiles }}
- name: "files-{{ $index }}"
mountPath: {{ $value.path }}
subPath: content
{{- end }}
{{- range $name, $volume := .Values.backend.persistence }}
- name: "{{ $name }}"
mountPath: "{{ $volume.mountPath }}"
{{- end }}
{{- range .Values.backend.extraVolumeMounts }}
- name: {{ .name }}
mountPath: {{ .mountPath }}
subPath: {{ .subPath | default "" }}
readOnly: {{ .readOnly }}
{{- end }}
{{- with .Values.backend.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.backend.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.backend.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
volumes:
{{- range $index, $value := .Values.mountFiles }}
- name: "files-{{ $index }}"
configMap:
name: "{{ include "find.fullname" $ }}-files-{{ $index }}"
{{- end }}
{{- range $name, $volume := .Values.backend.persistence }}
- name: "{{ $name }}"
{{- if eq $volume.type "emptyDir" }}
emptyDir: {}
{{- else }}
persistentVolumeClaim:
claimName: "{{ $fullName }}-{{ $name }}"
{{- end }}
{{- end }}
{{- range .Values.backend.extraVolumes }}
- name: {{ .name }}
{{- if .existingClaim }}
persistentVolumeClaim:
claimName: {{ .existingClaim }}
{{- else if .hostPath }}
hostPath:
{{ toYaml .hostPath | nindent 12 }}
{{- else if .csi }}
csi:
{{- toYaml .csi | nindent 12 }}
{{- else if .configMap }}
configMap:
{{- toYaml .configMap | nindent 12 }}
{{- else if .emptyDir }}
emptyDir:
{{- toYaml .emptyDir | nindent 12 }}
{{- else }}
emptyDir: {}
{{- end }}
{{- end }}
-121
View File
@@ -1,121 +0,0 @@
{{- $envVars := include "find.common.env" (list . .Values.backend) -}}
{{- $fullName := include "find.backend.fullname" . -}}
{{- $component := "backend" -}}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ $fullName }}-migrate
namespace: {{ .Release.Namespace | quote }}
{{- with .Values.backend.migrateJobAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
labels:
{{- include "find.common.labels" (list . $component) | nindent 4 }}
spec:
template:
metadata:
annotations:
{{- with .Values.backend.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "find.common.selectorLabels" (list . $component) | nindent 8 }}
spec:
{{- if $.Values.image.credentials }}
imagePullSecrets:
- name: {{ include "find.secret.dockerconfigjson.name" (dict "fullname" (include "find.fullname" .) "imageCredentials" $.Values.image.credentials) }}
{{- end}}
shareProcessNamespace: {{ .Values.backend.shareProcessNamespace }}
containers:
{{- with .Values.backend.sidecars }}
{{- toYaml . | nindent 8 }}
{{- end }}
- name: {{ .Chart.Name }}
image: "{{ (.Values.backend.image | default dict).repository | default .Values.image.repository }}:{{ (.Values.backend.image | default dict).tag | default .Values.image.tag }}"
imagePullPolicy: {{ (.Values.backend.image | default dict).pullPolicy | default .Values.image.pullPolicy }}
{{- with .Values.backend.migrate.command }}
command:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.backend.args }}
args:
{{- toYaml . | nindent 12 }}
{{- end }}
env:
{{- if $envVars}}
{{- $envVars | indent 12 }}
{{- end }}
{{- with .Values.backend.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.backend.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
volumeMounts:
{{- range $index, $value := .Values.mountFiles }}
- name: "files-{{ $index }}"
mountPath: {{ $value.path }}
subPath: content
{{- end }}
{{- range $name, $volume := .Values.backend.persistence }}
- name: "{{ $name }}"
mountPath: "{{ $volume.mountPath }}"
{{- end }}
{{- range .Values.backend.extraVolumeMounts }}
- name: {{ .name }}
mountPath: {{ .mountPath }}
subPath: {{ .subPath | default "" }}
readOnly: {{ .readOnly }}
{{- end }}
{{- with .Values.backend.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.backend.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.backend.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
restartPolicy: {{ .Values.backend.migrate.restartPolicy }}
volumes:
{{- range $index, $value := .Values.mountFiles }}
- name: "files-{{ $index }}"
configMap:
name: "{{ include "find.fullname" $ }}-files-{{ $index }}"
{{- end }}
{{- range $name, $volume := .Values.backend.persistence }}
- name: "{{ $name }}"
{{- if eq $volume.type "emptyDir" }}
emptyDir: {}
{{- else }}
persistentVolumeClaim:
claimName: "{{ $fullName }}-{{ $name }}"
{{- end }}
{{- end }}
{{- range .Values.backend.extraVolumes }}
- name: {{ .name }}
{{- if .existingClaim }}
persistentVolumeClaim:
claimName: {{ .existingClaim }}
{{- else if .hostPath }}
hostPath:
{{ toYaml .hostPath | nindent 12 }}
{{- else if .csi }}
csi:
{{- toYaml .csi | nindent 12 }}
{{- else if .configMap }}
configMap:
{{- toYaml .configMap | nindent 12 }}
{{- else if .emptyDir }}
emptyDir:
{{- toYaml .emptyDir | nindent 12 }}
{{- else }}
emptyDir: {}
{{- end }}
{{- end }}

Some files were not shown because too many files have changed in this diff Show More