🎉(conversations) bootstrap backend & frontend
This is the first commit which provides all the first stack for a working chat. This is a first implementation with: - Vercel SDK for the frontend part - OpenAI Agent SDK for the backend The stack can use a local LLM with docker ot a remote one. This implementation is more a draft, but it provides the project structure. All tests are working even if we lack a lot of them.
This commit is contained in:
Quentin BEY
@@ -0,0 +1,37 @@
|
||||
# Python
|
||||
__pycache__
|
||||
*.pyc
|
||||
**/__pycache__
|
||||
**/*.pyc
|
||||
venv
|
||||
.venv
|
||||
|
||||
# System-specific files
|
||||
.DS_Store
|
||||
**/.DS_Store
|
||||
|
||||
# Docker
|
||||
docker compose.*
|
||||
env.d
|
||||
|
||||
# Docs
|
||||
docs
|
||||
*.md
|
||||
*.log
|
||||
|
||||
# Development/test cache & configurations
|
||||
data
|
||||
.cache
|
||||
.circleci
|
||||
.git
|
||||
.vscode
|
||||
.iml
|
||||
.idea
|
||||
db.sqlite3
|
||||
.mypy_cache
|
||||
.pylint.d
|
||||
.pytest_cache
|
||||
|
||||
# Frontend
|
||||
node_modules
|
||||
.next
|
||||
@@ -0,0 +1,14 @@
|
||||
DJANGO_CONFIGURATION=Test
|
||||
DJANGO_SETTINGS_MODULE=conversations.settings
|
||||
DJANGO_SECRET_KEY=ThisIsAnExampleKeyForTestPurposeOnly
|
||||
OIDC_OP_JWKS_ENDPOINT=/endpoint-for-test-purpose-only
|
||||
DB_HOST=localhost
|
||||
DB_NAME=conversations
|
||||
DB_USER=dinum
|
||||
DB_PASSWORD=pass
|
||||
DB_PORT=15432
|
||||
|
||||
STORAGES_STATICFILES_BACKEND=django.contrib.staticfiles.storage.StaticFilesStorage
|
||||
AWS_S3_ENDPOINT_URL=http://localhost:9000
|
||||
AWS_S3_ACCESS_KEY_ID=conversations
|
||||
AWS_S3_SECRET_ACCESS_KEY=password
|
||||
@@ -0,0 +1,23 @@
|
||||
# Set the default behavior for all files
|
||||
* text=auto eol=lf
|
||||
|
||||
# Binary files (should not be modified)
|
||||
*.png binary
|
||||
*.jpg binary
|
||||
*.jpeg binary
|
||||
*.gif binary
|
||||
*.ico binary
|
||||
*.mov binary
|
||||
*.mp4 binary
|
||||
*.mp3 binary
|
||||
*.flv binary
|
||||
*.fla binary
|
||||
*.swf binary
|
||||
*.gz binary
|
||||
*.zip binary
|
||||
*.7z binary
|
||||
*.ttf binary
|
||||
*.woff binary
|
||||
*.woff2 binary
|
||||
*.eot binary
|
||||
*.pdf binary
|
||||
@@ -0,0 +1,6 @@
|
||||
<!---
|
||||
Thanks for filing an issue 😄 ! Before you submit, please read the following:
|
||||
|
||||
Check the other issue templates if you are trying to submit a bug report, feature request, or question
|
||||
Search open/closed issues before submitting since someone might have asked the same thing before!
|
||||
-->
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
name: 🐛 Bug Report
|
||||
about: If something is not working as expected 🤔.
|
||||
labels: ["bug", "triage"]
|
||||
---
|
||||
|
||||
## Bug Report
|
||||
|
||||
**Problematic behavior**
|
||||
A clear and concise description of the behavior.
|
||||
|
||||
**Expected behavior/code**
|
||||
A clear and concise description of what you expected to happen (or code).
|
||||
|
||||
**Steps to Reproduce**
|
||||
1. Do this...
|
||||
2. Then this...
|
||||
3. And then the bug happens!
|
||||
|
||||
**Environment**
|
||||
- Docs version:
|
||||
- Instance url:
|
||||
|
||||
**Possible Solution**
|
||||
<!--- Only if you have suggestions on a fix for the bug -->
|
||||
|
||||
**Additional context/Screenshots**
|
||||
Add any other context about the problem here. If applicable, add screenshots to help explain.
|
||||
@@ -0,0 +1,23 @@
|
||||
---
|
||||
name: ✨ Feature Request
|
||||
about: I have a suggestion (and may want to build it 💪)!
|
||||
labels: ["feature", "triage"]
|
||||
---
|
||||
|
||||
## Feature Request
|
||||
|
||||
**Is your feature request related to a problem or unsupported use case? Please describe.**
|
||||
A clear and concise description of what the problem is. For example: I need to do some task and I have an issue...
|
||||
|
||||
**Describe the solution you'd like**
|
||||
A clear and concise description of what you want to happen. Add any considered drawbacks.
|
||||
|
||||
**Describe alternatives you've considered**
|
||||
A clear and concise description of any alternative solutions or features you've considered.
|
||||
|
||||
**Discovery, Documentation, Adoption, Migration Strategy**
|
||||
If you can, explain how users will be able to use this and possibly write out some documentation (if applicable).
|
||||
Maybe add a screenshot or design?
|
||||
|
||||
**Do you want to work on it through a Pull Request?**
|
||||
<!-- Make sure to coordinate with us before you spend too much time working on an implementation! -->
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
name: 🤗 Support Question
|
||||
about: If you have a question 💬, or something was not clear from the docs!
|
||||
labels: ["support", "triage"]
|
||||
---
|
||||
## Support request
|
||||
**Checks before filing**
|
||||
Please make sure you have read our [main Readme](https://github.com/suitenumerique/conversations).
|
||||
|
||||
Also make sure it was not already answered in [an open or close issue](https://github.com/suitenumerique/conversations/issues?q=is%3Aissue%20state%3Aopen%20label%3Asupport).
|
||||
|
||||
If your question was not covered, and you feel like it should be, fire away! We'd love to improve our docs! 👌
|
||||
|
||||
**Topic**
|
||||
What's the general area of your question: for example, docker setup, database schema, search functionality,...
|
||||
|
||||
**Question**
|
||||
Try to be as specific as possible so we can help you as best we can. Please be patient 🙏
|
||||
@@ -0,0 +1,22 @@
|
||||
## Purpose
|
||||
|
||||
Describe the purpose of this pull request.
|
||||
|
||||
|
||||
## Proposal
|
||||
|
||||
- [ ] item 1...
|
||||
- [ ] item 2...
|
||||
|
||||
## External contributions
|
||||
|
||||
Thank you for your contribution! 🎉
|
||||
|
||||
Please ensure the following items are checked before submitting your pull request:
|
||||
- [ ] I have read and followed the [contributing guidelines](https://github.com/suitenumerique/conversations/blob/main/CONTRIBUTING.md)
|
||||
- [ ] I have read and agreed to the [Code of Conduct](https://github.com/suitenumerique/conversations/blob/main/CODE_OF_CONDUCT.md)
|
||||
- [ ] I have signed off my commits with `git commit --signoff` (DCO compliance)
|
||||
- [ ] I have signed my commits with my SSH or GPG key (`git commit -S`)
|
||||
- [ ] My commit messages follow the required format: `<gitmoji>(type) title description`
|
||||
- [ ] I have added a changelog entry under `## [Unreleased]` section (if noticeable change)
|
||||
- [ ] I have added corresponding tests for new features or bug fixes (if applicable)
|
||||
@@ -0,0 +1,151 @@
|
||||
name: Frontend Workflow
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
branches:
|
||||
- "*"
|
||||
|
||||
jobs:
|
||||
|
||||
install-dependencies:
|
||||
uses: ./.github/workflows/dependencies.yml
|
||||
with:
|
||||
node_version: '22.x'
|
||||
with-front-dependencies-installation: true
|
||||
|
||||
test-front:
|
||||
needs: install-dependencies
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "22.x"
|
||||
|
||||
- name: Install yarn
|
||||
run: npm install -g yarn
|
||||
|
||||
- name: Restore the frontend cache
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: "src/frontend/**/node_modules"
|
||||
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
|
||||
fail-on-cache-miss: true
|
||||
|
||||
- name: Test App
|
||||
run: cd src/frontend/ && yarn test
|
||||
|
||||
lint-front:
|
||||
runs-on: ubuntu-latest
|
||||
needs: install-dependencies
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "22.x"
|
||||
|
||||
- name: Install yarn
|
||||
run: npm install -g yarn
|
||||
|
||||
- name: Restore the frontend cache
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: "src/frontend/**/node_modules"
|
||||
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
|
||||
fail-on-cache-miss: true
|
||||
|
||||
- name: Check linting
|
||||
run: cd src/frontend/ && yarn lint
|
||||
|
||||
test-e2e-chromium:
|
||||
runs-on: ubuntu-latest
|
||||
needs: install-dependencies
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "22.x"
|
||||
|
||||
- name: Install yarn
|
||||
run: npm install -g yarn
|
||||
|
||||
- name: Restore the frontend cache
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: "src/frontend/**/node_modules"
|
||||
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
|
||||
fail-on-cache-miss: true
|
||||
|
||||
- name: Set e2e env variables
|
||||
run: cat env.d/development/common.e2e.dist >> env.d/development/common.dist
|
||||
|
||||
- name: Install Playwright Browsers
|
||||
run: cd src/frontend/apps/e2e && yarn install --frozen-lockfile && yarn install-playwright chromium
|
||||
|
||||
- name: Start Docker services
|
||||
run: make bootstrap-e2e FLUSH_ARGS='--no-input'
|
||||
|
||||
- name: Run e2e tests
|
||||
run: cd src/frontend/ && yarn e2e:test --project='chromium'
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: playwright-chromium-report
|
||||
path: src/frontend/apps/e2e/report/
|
||||
retention-days: 7
|
||||
|
||||
test-e2e-other-browser:
|
||||
runs-on: ubuntu-latest
|
||||
needs: test-e2e-chromium
|
||||
timeout-minutes: 40
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "22.x"
|
||||
|
||||
- name: Install yarn
|
||||
run: npm install -g yarn
|
||||
|
||||
- name: Restore the frontend cache
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: "src/frontend/**/node_modules"
|
||||
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
|
||||
fail-on-cache-miss: true
|
||||
|
||||
- name: Set e2e env variables
|
||||
run: cat env.d/development/common.e2e.dist >> env.d/development/common.dist
|
||||
|
||||
- name: Install Playwright Browsers
|
||||
run: cd src/frontend/apps/e2e && yarn install --frozen-lockfile && yarn install-playwright firefox webkit chromium
|
||||
|
||||
- name: Start Docker services
|
||||
run: make bootstrap-e2e FLUSH_ARGS='--no-input'
|
||||
|
||||
- name: Run e2e tests
|
||||
run: cd src/frontend/ && yarn e2e:test --project=firefox --project=webkit
|
||||
|
||||
- uses: actions/upload-artifact@v4
|
||||
if: always()
|
||||
with:
|
||||
name: playwright-other-report
|
||||
path: src/frontend/apps/e2e/report/
|
||||
retention-days: 7
|
||||
@@ -0,0 +1,204 @@
|
||||
name: Main Workflow
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
branches:
|
||||
- "*"
|
||||
|
||||
jobs:
|
||||
install-dependencies:
|
||||
uses: ./.github/workflows/dependencies.yml
|
||||
with:
|
||||
with-build_mails: true
|
||||
|
||||
lint-git:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'pull_request' # Makes sense only for pull requests
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- name: show
|
||||
run: git log
|
||||
- name: Enforce absence of print statements in code
|
||||
run: |
|
||||
! git diff origin/${{ github.event.pull_request.base.ref }}..HEAD -- . ':(exclude)**/conversations.yml' | grep "print("
|
||||
- name: Check absence of fixup commits
|
||||
run: |
|
||||
! git log | grep 'fixup!'
|
||||
- name: Install gitlint
|
||||
run: pip install --user requests gitlint
|
||||
- name: Lint commit messages added to main
|
||||
run: ~/.local/bin/gitlint --commits origin/${{ github.event.pull_request.base.ref }}..HEAD
|
||||
|
||||
check-changelog:
|
||||
runs-on: ubuntu-latest
|
||||
if: |
|
||||
contains(github.event.pull_request.labels.*.name, 'noChangeLog') == false &&
|
||||
github.event_name == 'pull_request'
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
fetch-depth: 50
|
||||
- name: Check that the CHANGELOG has been modified in the current branch
|
||||
run: git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.event.after }} | grep 'CHANGELOG.md'
|
||||
|
||||
lint-changelog:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v2
|
||||
- name: Check CHANGELOG max line length
|
||||
run: |
|
||||
max_line_length=$(cat CHANGELOG.md | grep -Ev "^\[.*\]: https://github.com" | wc -L)
|
||||
if [ $max_line_length -ge 80 ]; then
|
||||
echo "ERROR: CHANGELOG has lines longer than 80 characters."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
lint-spell-mistakes:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name == 'pull_request'
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v2
|
||||
- name: Install codespell
|
||||
run: pip install --user codespell
|
||||
- name: Check for typos
|
||||
run: |
|
||||
codespell \
|
||||
--check-filenames \
|
||||
--ignore-words-list "Dokument,afterAll,excpt,statics" \
|
||||
--skip "./git/" \
|
||||
--skip "**/*.po" \
|
||||
--skip "**/*.pot" \
|
||||
--skip "**/*.json" \
|
||||
--skip "**/yarn.lock"
|
||||
|
||||
lint-back:
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: src/backend
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v2
|
||||
- name: Install Python
|
||||
uses: actions/setup-python@v3
|
||||
with:
|
||||
python-version: "3.13.3"
|
||||
- name: Upgrade pip and setuptools
|
||||
run: pip install --upgrade pip setuptools
|
||||
- name: Install development dependencies
|
||||
run: pip install --user .[dev]
|
||||
- name: Check code formatting with ruff
|
||||
run: ~/.local/bin/ruff format . --diff
|
||||
- name: Lint code with ruff
|
||||
run: ~/.local/bin/ruff check .
|
||||
- name: Lint code with pylint
|
||||
run: ~/.local/bin/pylint .
|
||||
|
||||
test-back:
|
||||
runs-on: ubuntu-latest
|
||||
needs: install-dependencies
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: src/backend
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16
|
||||
env:
|
||||
POSTGRES_DB: conversations
|
||||
POSTGRES_USER: dinum
|
||||
POSTGRES_PASSWORD: pass
|
||||
ports:
|
||||
- 5432:5432
|
||||
# needed because the postgres container does not provide a healthcheck
|
||||
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
|
||||
|
||||
env:
|
||||
DJANGO_CONFIGURATION: Test
|
||||
DJANGO_SETTINGS_MODULE: conversations.settings
|
||||
DJANGO_SECRET_KEY: ThisIsAnExampleKeyForTestPurposeOnly
|
||||
OIDC_OP_JWKS_ENDPOINT: /endpoint-for-test-purpose-only
|
||||
DB_HOST: localhost
|
||||
DB_NAME: conversations
|
||||
DB_USER: dinum
|
||||
DB_PASSWORD: pass
|
||||
DB_PORT: 5432
|
||||
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
|
||||
AWS_S3_ENDPOINT_URL: http://localhost:9000
|
||||
AWS_S3_ACCESS_KEY_ID: conversations
|
||||
AWS_S3_SECRET_ACCESS_KEY: password
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Create writable /data
|
||||
run: |
|
||||
sudo mkdir -p /data/media && \
|
||||
sudo mkdir -p /data/static
|
||||
|
||||
- name: Restore the mail templates
|
||||
uses: actions/cache@v4
|
||||
id: mail-templates
|
||||
with:
|
||||
path: "src/backend/core/templates/mail"
|
||||
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
|
||||
fail-on-cache-miss: true
|
||||
|
||||
- name: Start MinIO
|
||||
run: |
|
||||
docker pull minio/minio
|
||||
docker run -d --name minio \
|
||||
-p 9000:9000 \
|
||||
-e "MINIO_ACCESS_KEY=conversations" \
|
||||
-e "MINIO_SECRET_KEY=password" \
|
||||
-v /data/media:/data \
|
||||
minio/minio server --console-address :9001 /data
|
||||
|
||||
# Tool to wait for a service to be ready
|
||||
- name: Install Dockerize
|
||||
run: |
|
||||
curl -sSL https://github.com/jwilder/dockerize/releases/download/v0.8.0/dockerize-linux-amd64-v0.8.0.tar.gz | sudo tar -C /usr/local/bin -xzv
|
||||
|
||||
- name: Wait for MinIO to be ready
|
||||
run: |
|
||||
dockerize -wait tcp://localhost:9000 -timeout 10s
|
||||
|
||||
- name: Configure MinIO
|
||||
run: |
|
||||
MINIO=$(docker ps | grep minio/minio | sed -E 's/.*\s+([a-zA-Z0-9_-]+)$/\1/')
|
||||
docker exec ${MINIO} sh -c \
|
||||
"mc alias set conversations http://localhost:9000 conversations password && \
|
||||
mc alias ls && \
|
||||
mc mb conversations/conversations-media-storage && \
|
||||
mc version enable conversations/conversations-media-storage"
|
||||
|
||||
- name: Install Python
|
||||
uses: actions/setup-python@v3
|
||||
with:
|
||||
python-version: "3.13.3"
|
||||
|
||||
- name: Install development dependencies
|
||||
run: pip install --user .[dev]
|
||||
|
||||
- name: Install gettext (required to compile messages) and MIME support
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y gettext pandoc shared-mime-info
|
||||
sudo wget https://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/conf/mime.types -O /etc/mime.types
|
||||
|
||||
- name: Generate a MO file from strings extracted from the project
|
||||
run: python manage.py compilemessages
|
||||
|
||||
- name: Run tests
|
||||
run: ~/.local/bin/pytest -n 2
|
||||
@@ -0,0 +1,77 @@
|
||||
name: Download translations from Crowdin
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches:
|
||||
- 'release/**'
|
||||
|
||||
jobs:
|
||||
install-dependencies:
|
||||
uses: ./.github/workflows/dependencies.yml
|
||||
with:
|
||||
node_version: '22.x'
|
||||
with-front-dependencies-installation: true
|
||||
|
||||
synchronize-with-crowdin:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
- name: Create empty source files
|
||||
run: |
|
||||
touch src/backend/locale/django.pot
|
||||
mkdir -p src/frontend/packages/i18n/locales/conversations/
|
||||
touch src/frontend/packages/i18n/locales/conversations/translations-crowdin.json
|
||||
# crowdin workflow
|
||||
- name: crowdin action
|
||||
uses: crowdin/github-action@v2
|
||||
with:
|
||||
config: crowdin/config.yml
|
||||
upload_sources: false
|
||||
upload_translations: false
|
||||
download_translations: true
|
||||
create_pull_request: false
|
||||
push_translations: false
|
||||
push_sources: false
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
# A numeric ID, found at https://crowdin.com/project/<projectName>/tools/api
|
||||
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
|
||||
|
||||
# Visit https://crowdin.com/settings#api-key to create this token
|
||||
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
|
||||
|
||||
CROWDIN_BASE_PATH: "../src/"
|
||||
# frontend i18n
|
||||
- name: Restore the frontend cache
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: "src/frontend/**/node_modules"
|
||||
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
|
||||
fail-on-cache-miss: true
|
||||
- name: generate translations files
|
||||
working-directory: src/frontend
|
||||
run: yarn i18n:deploy
|
||||
# Create a new PR
|
||||
- name: Create a new Pull Request with new translated strings
|
||||
uses: peter-evans/create-pull-request@v7
|
||||
with:
|
||||
commit-message: |
|
||||
🌐(i18n) update translated strings
|
||||
|
||||
Update translated files with new translations
|
||||
title: 🌐(i18n) update translated strings
|
||||
body: |
|
||||
## Purpose
|
||||
|
||||
update translated strings
|
||||
|
||||
## Proposal
|
||||
|
||||
- [x] update translated strings
|
||||
branch: i18n/update-translations
|
||||
labels: i18n
|
||||
@@ -0,0 +1,76 @@
|
||||
name: Update crowdin sources
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
|
||||
jobs:
|
||||
install-dependencies:
|
||||
uses: ./.github/workflows/dependencies.yml
|
||||
with:
|
||||
node_version: '22.x'
|
||||
with-front-dependencies-installation: true
|
||||
with-build_mails: true
|
||||
|
||||
synchronize-with-crowdin:
|
||||
needs: install-dependencies
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
# Backend i18n
|
||||
- name: Install Python
|
||||
uses: actions/setup-python@v3
|
||||
with:
|
||||
python-version: "3.13.3"
|
||||
- name: Upgrade pip and setuptools
|
||||
run: pip install --upgrade pip setuptools
|
||||
- name: Install development dependencies
|
||||
run: pip install --user .
|
||||
working-directory: src/backend
|
||||
- name: Restore the mail templates
|
||||
uses: actions/cache@v4
|
||||
id: mail-templates
|
||||
with:
|
||||
path: "src/backend/core/templates/mail"
|
||||
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
|
||||
fail-on-cache-miss: true
|
||||
- name: Install gettext
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y gettext pandoc
|
||||
- name: generate pot files
|
||||
working-directory: src/backend
|
||||
run: |
|
||||
DJANGO_CONFIGURATION=Build python manage.py makemessages -a --keep-pot
|
||||
# frontend i18n
|
||||
- name: Restore the frontend cache
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: "src/frontend/**/node_modules"
|
||||
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
|
||||
fail-on-cache-miss: true
|
||||
- name: generate source translation file
|
||||
working-directory: src/frontend
|
||||
run: yarn i18n:extract
|
||||
# crowdin workflow
|
||||
- name: crowdin action
|
||||
uses: crowdin/github-action@v2
|
||||
with:
|
||||
config: crowdin/config.yml
|
||||
upload_sources: true
|
||||
upload_translations: false
|
||||
download_translations: false
|
||||
create_pull_request: false
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
# A numeric ID, found at https://crowdin.com/project/<projectName>/tools/api
|
||||
CROWDIN_PROJECT_ID: ${{ secrets.CROWDIN_PROJECT_ID }}
|
||||
|
||||
# Visit https://crowdin.com/settings#api-key to create this token
|
||||
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
|
||||
|
||||
CROWDIN_BASE_PATH: "../src/"
|
||||
@@ -0,0 +1,93 @@
|
||||
name: Dependency reusable workflow
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
node_version:
|
||||
required: false
|
||||
default: '22.x'
|
||||
type: string
|
||||
with-front-dependencies-installation:
|
||||
type: boolean
|
||||
default: false
|
||||
with-build_mails:
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
jobs:
|
||||
front-dependencies-installation:
|
||||
if: ${{ inputs.with-front-dependencies-installation == true }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Restore the frontend cache
|
||||
uses: actions/cache@v4
|
||||
id: front-node_modules
|
||||
with:
|
||||
path: "src/frontend/**/node_modules"
|
||||
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
|
||||
|
||||
- name: Setup Node.js
|
||||
if: steps.front-node_modules.outputs.cache-hit != 'true'
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ inputs.node_version }}
|
||||
|
||||
- name: Install yarn
|
||||
if: steps.front-node_modules.outputs.cache-hit != 'true'
|
||||
run: npm install -g yarn
|
||||
|
||||
- name: Install dependencies
|
||||
if: steps.front-node_modules.outputs.cache-hit != 'true'
|
||||
run: cd src/frontend/ && yarn install --frozen-lockfile
|
||||
|
||||
- name: Cache install frontend
|
||||
if: steps.front-node_modules.outputs.cache-hit != 'true'
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: "src/frontend/**/node_modules"
|
||||
key: front-node_modules-${{ hashFiles('src/frontend/**/yarn.lock') }}
|
||||
|
||||
build-mails:
|
||||
if: ${{ inputs.with-build_mails == true }}
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: src/mail
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Restore the mail templates
|
||||
uses: actions/cache@v4
|
||||
id: mail-templates
|
||||
with:
|
||||
path: "src/backend/core/templates/mail"
|
||||
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
|
||||
|
||||
- name: Setup Node.js
|
||||
if: steps.mail-templates.outputs.cache-hit != 'true'
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: ${{ inputs.node_version }}
|
||||
|
||||
- name: Install yarn
|
||||
if: steps.mail-templates.outputs.cache-hit != 'true'
|
||||
run: npm install -g yarn
|
||||
|
||||
- name: Install node dependencies
|
||||
if: steps.mail-templates.outputs.cache-hit != 'true'
|
||||
run: yarn install --frozen-lockfile
|
||||
|
||||
- name: Build mails
|
||||
if: steps.mail-templates.outputs.cache-hit != 'true'
|
||||
run: yarn build
|
||||
|
||||
- name: Cache mail templates
|
||||
if: steps.mail-templates.outputs.cache-hit != 'true'
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: "src/backend/core/templates/mail"
|
||||
key: mail-templates-${{ hashFiles('src/mail/mjml') }}
|
||||
@@ -0,0 +1,106 @@
|
||||
name: Docker Hub Workflow
|
||||
run-name: Docker Hub Workflow
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches:
|
||||
- 'main'
|
||||
tags:
|
||||
- 'v*'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'main'
|
||||
- 'ci/trivy-fails'
|
||||
|
||||
env:
|
||||
DOCKER_USER: 1001:127
|
||||
|
||||
jobs:
|
||||
build-and-push-backend:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
-
|
||||
name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
-
|
||||
name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
-
|
||||
name: Docker meta
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: lasuite/conversations-backend
|
||||
-
|
||||
name: Login to DockerHub
|
||||
if: github.event_name != 'pull_request'
|
||||
run: echo "${{ secrets.DOCKER_HUB_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_HUB_USER }}" --password-stdin
|
||||
-
|
||||
name: Run trivy scan
|
||||
uses: numerique-gouv/action-trivy-cache@main
|
||||
with:
|
||||
docker-build-args: '--target backend-production -f Dockerfile'
|
||||
docker-image-name: 'docker.io/lasuite/conversations-backend:${{ github.sha }}'
|
||||
-
|
||||
name: Build and push
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
target: backend-production
|
||||
build-args: DOCKER_USER=${{ env.DOCKER_USER }}:-1000
|
||||
push: ${{ github.event_name != 'pull_request' }}
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
|
||||
build-and-push-frontend:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
-
|
||||
name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
-
|
||||
name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
-
|
||||
name: Docker meta
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: lasuite/conversations-frontend
|
||||
-
|
||||
name: Login to DockerHub
|
||||
if: github.event_name != 'pull_request'
|
||||
run: echo "${{ secrets.DOCKER_HUB_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_HUB_USER }}" --password-stdin
|
||||
-
|
||||
name: Run trivy scan
|
||||
uses: numerique-gouv/action-trivy-cache@main
|
||||
with:
|
||||
docker-build-args: '-f src/frontend/Dockerfile --target frontend-production'
|
||||
docker-image-name: 'docker.io/lasuite/conversations-frontend:${{ github.sha }}'
|
||||
-
|
||||
name: Build and push
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
file: ./src/frontend/Dockerfile
|
||||
target: frontend-production
|
||||
build-args: |
|
||||
DOCKER_USER=${{ env.DOCKER_USER }}:-1000
|
||||
push: ${{ github.event_name != 'pull_request' }}
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
|
||||
notify-argocd:
|
||||
needs:
|
||||
- build-and-push-frontend
|
||||
- build-and-push-backend
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event_name != 'pull_request'
|
||||
steps:
|
||||
- uses: numerique-gouv/action-argocd-webhook-notification@main
|
||||
id: notify
|
||||
with:
|
||||
deployment_repo_path: "${{ secrets.DEPLOYMENT_REPO_URL }}"
|
||||
argocd_webhook_secret: "${{ secrets.ARGOCD_PREPROD_WEBHOOK_SECRET }}"
|
||||
argocd_url: "${{ vars.ARGOCD_PREPROD_WEBHOOK_URL }}"
|
||||
@@ -0,0 +1,30 @@
|
||||
name: Helmfile lint
|
||||
run-name: Helmfile lint
|
||||
|
||||
on:
|
||||
push:
|
||||
pull_request:
|
||||
branches:
|
||||
- 'main'
|
||||
|
||||
jobs:
|
||||
helmfile-lint:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: ghcr.io/helmfile/helmfile:v0.171.0
|
||||
steps:
|
||||
-
|
||||
name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
-
|
||||
name: Helmfile lint
|
||||
shell: bash
|
||||
run: |
|
||||
set -e
|
||||
HELMFILE=src/helm/helmfile.yaml
|
||||
environments=$(awk 'BEGIN {in_env=0} /^environments:/ {in_env=1; next} /^---/ {in_env=0} in_env && /^ [^ ]/ {gsub(/^ /,""); gsub(/:.*$/,""); print}' "$HELMFILE")
|
||||
for env in $environments; do
|
||||
echo "################### $env lint ###################"
|
||||
helmfile -e $env -f $HELMFILE lint || exit 1
|
||||
echo -e "\n"
|
||||
done
|
||||
@@ -0,0 +1,34 @@
|
||||
name: Release Chart
|
||||
run-name: Release Chart
|
||||
|
||||
on:
|
||||
push:
|
||||
paths:
|
||||
- src/helm/conversations/**
|
||||
|
||||
jobs:
|
||||
release:
|
||||
# depending on default permission settings for your org (contents being read-only or read-write for workloads), you will have to add permissions
|
||||
# see: https://docs.github.com/en/actions/security-guides/automatic-token-authentication#modifying-the-permissions-for-the-github_token
|
||||
permissions:
|
||||
contents: write
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Cleanup
|
||||
run: rm -rf ./src/helm/extra
|
||||
|
||||
- name: Install Helm
|
||||
uses: azure/setup-helm@v4
|
||||
env:
|
||||
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
|
||||
|
||||
- name: Publish Helm charts
|
||||
uses: numerique-gouv/helm-gh-pages@add-overwrite-option
|
||||
with:
|
||||
charts_dir: ./src/helm
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
+81
@@ -0,0 +1,81 @@
|
||||
# Byte-compiled / optimized / DLL files
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
# Distribution / packaging
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
pip-wheel-metadata/
|
||||
share/python-wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
MANIFEST
|
||||
.DS_Store
|
||||
.next/
|
||||
|
||||
# Translations # Translations
|
||||
*.mo
|
||||
*.pot
|
||||
|
||||
# Environments
|
||||
.venv
|
||||
env/
|
||||
venv/
|
||||
ENV/
|
||||
env.bak/
|
||||
venv.bak/
|
||||
env.d/development/*
|
||||
!env.d/development/*.dist
|
||||
env.d/terraform
|
||||
|
||||
# npm
|
||||
node_modules
|
||||
|
||||
# Mails
|
||||
src/backend/core/templates/mail/
|
||||
|
||||
# Swagger
|
||||
**/swagger.json
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
|
||||
# Terraform
|
||||
.terraform
|
||||
*.tfstate
|
||||
*.tfstate.backup
|
||||
|
||||
# Test & lint
|
||||
.coverage
|
||||
.pylint.d
|
||||
.pytest_cache
|
||||
db.sqlite3
|
||||
.mypy_cache
|
||||
|
||||
# Site media
|
||||
/data/
|
||||
|
||||
# IDEs
|
||||
.idea/
|
||||
.vscode/
|
||||
*.iml
|
||||
.devcontainer
|
||||
|
||||
# Docker compose override
|
||||
compose.override.yml
|
||||
@@ -0,0 +1,78 @@
|
||||
# All these sections are optional, edit this file as you like.
|
||||
[general]
|
||||
# Ignore certain rules, you can reference them by their id or by their full name
|
||||
# ignore=title-trailing-punctuation, T3
|
||||
|
||||
# verbosity should be a value between 1 and 3, the commandline -v flags take precedence over this
|
||||
# verbosity = 2
|
||||
|
||||
# By default gitlint will ignore merge commits. Set to 'false' to disable.
|
||||
# ignore-merge-commits=true
|
||||
|
||||
# By default gitlint will ignore fixup commits. Set to 'false' to disable.
|
||||
# ignore-fixup-commits=true
|
||||
|
||||
# By default gitlint will ignore squash commits. Set to 'false' to disable.
|
||||
# ignore-squash-commits=true
|
||||
|
||||
# Enable debug mode (prints more output). Disabled by default.
|
||||
# debug=true
|
||||
|
||||
# Set the extra-path where gitlint will search for user defined rules
|
||||
# See http://jorisroovers.github.io/gitlint/user_defined_rules for details
|
||||
extra-path=gitlint/
|
||||
|
||||
# [title-max-length]
|
||||
# line-length=80
|
||||
|
||||
[title-must-not-contain-word]
|
||||
# Comma-separated list of words that should not occur in the title. Matching is case
|
||||
# insensitive. It's fine if the keyword occurs as part of a larger word (so "WIPING"
|
||||
# will not cause a violation, but "WIP: my title" will.
|
||||
words=wip
|
||||
|
||||
#[title-match-regex]
|
||||
# python like regex (https://docs.python.org/2/library/re.html) that the
|
||||
# commit-msg title must be matched to.
|
||||
# Note that the regex can contradict with other rules if not used correctly
|
||||
# (e.g. title-must-not-contain-word).
|
||||
#regex=
|
||||
|
||||
# [B1]
|
||||
# B1 = body-max-line-length
|
||||
# line-length=120
|
||||
# [body-min-length]
|
||||
# min-length=5
|
||||
|
||||
# [body-is-missing]
|
||||
# Whether to ignore this rule on merge commits (which typically only have a title)
|
||||
# default = True
|
||||
# ignore-merge-commits=false
|
||||
|
||||
# [body-changed-file-mention]
|
||||
# List of files that need to be explicitly mentioned in the body when they are changed
|
||||
# This is useful for when developers often erroneously edit certain files or git submodules.
|
||||
# By specifying this rule, developers can only change the file when they explicitly reference
|
||||
# it in the commit message.
|
||||
# files=gitlint/rules.py,README.md
|
||||
|
||||
# [author-valid-email]
|
||||
# python like regex (https://docs.python.org/2/library/re.html) that the
|
||||
# commit author email address should be matched to
|
||||
# For example, use the following regex if you only want to allow email addresses from foo.com
|
||||
# regex=[^@]+@foo.com
|
||||
|
||||
[ignore-by-title]
|
||||
# Allow empty body & wrong title pattern only when bots (pyup/greenkeeper)
|
||||
# upgrade dependencies
|
||||
regex=^(⬆️.*|Update (.*) from (.*) to (.*)|(chore|fix)\(package\): update .*)$
|
||||
ignore=B6,UC1
|
||||
|
||||
# [ignore-by-body]
|
||||
# Ignore certain rules for commits of which the body has a line that matches a regex
|
||||
# E.g. Match bodies that have a line that that contain "release"
|
||||
# regex=(.*)release(.*)
|
||||
#
|
||||
# Ignore certain rules, you can reference them by their id or by their full name
|
||||
# Use 'all' to ignore all rules
|
||||
# ignore=T1,body-min-length
|
||||
@@ -8,5 +8,9 @@ and this project adheres to
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- 🎉(conversations) bootstrap backend & frontend #1
|
||||
|
||||
|
||||
[unreleased]: https://github.com/numerique-gouv/conversations/compare/HEAD...main
|
||||
|
||||
+162
@@ -0,0 +1,162 @@
|
||||
# Django conversations
|
||||
|
||||
# ---- base image to inherit from ----
|
||||
FROM python:3.13.3-alpine AS base
|
||||
|
||||
# Upgrade pip to its latest release to speed up dependencies installation
|
||||
RUN python -m pip install --upgrade pip setuptools
|
||||
|
||||
# Upgrade system packages to install security updates
|
||||
RUN apk update && \
|
||||
apk upgrade
|
||||
|
||||
# ---- Back-end builder image ----
|
||||
FROM base AS back-builder
|
||||
|
||||
WORKDIR /builder
|
||||
|
||||
# Install Rust and Cargo using Alpine's package manager
|
||||
RUN apk add --no-cache \
|
||||
build-base \
|
||||
libffi-dev \
|
||||
rust \
|
||||
cargo
|
||||
|
||||
# Copy required python dependencies
|
||||
COPY ./src/backend /builder
|
||||
|
||||
RUN mkdir /install && \
|
||||
pip install --prefix=/install .
|
||||
|
||||
|
||||
# ---- mails ----
|
||||
FROM node:24 AS mail-builder
|
||||
|
||||
COPY ./src/mail /mail/app
|
||||
|
||||
WORKDIR /mail/app
|
||||
|
||||
RUN yarn install --frozen-lockfile && \
|
||||
yarn build
|
||||
|
||||
|
||||
# ---- static link collector ----
|
||||
FROM base AS link-collector
|
||||
ARG CONVERSATIONS_STATIC_ROOT=/data/static
|
||||
|
||||
# Install pango & rdfind
|
||||
RUN apk add \
|
||||
pango \
|
||||
rdfind
|
||||
|
||||
# Copy installed python dependencies
|
||||
COPY --from=back-builder /install /usr/local
|
||||
|
||||
# Copy conversations application (see .dockerignore)
|
||||
COPY ./src/backend /app/
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# collectstatic
|
||||
RUN DJANGO_CONFIGURATION=Build \
|
||||
python manage.py collectstatic --noinput
|
||||
|
||||
# Replace duplicated file by a symlink to decrease the overall size of the
|
||||
# final image
|
||||
RUN rdfind -makesymlinks true -followsymlinks true -makeresultsfile false ${CONVERSATIONS_STATIC_ROOT}
|
||||
|
||||
# ---- Core application image ----
|
||||
FROM base AS core
|
||||
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
|
||||
# Install required system libs
|
||||
RUN apk add \
|
||||
cairo \
|
||||
file \
|
||||
font-noto \
|
||||
font-noto-emoji \
|
||||
gettext \
|
||||
gdk-pixbuf \
|
||||
libffi-dev \
|
||||
pango \
|
||||
shared-mime-info
|
||||
|
||||
RUN wget https://svn.apache.org/repos/asf/httpd/httpd/trunk/docs/conf/mime.types -O /etc/mime.types
|
||||
|
||||
# Copy entrypoint
|
||||
COPY ./docker/files/usr/local/bin/entrypoint /usr/local/bin/entrypoint
|
||||
|
||||
# Give the "root" group the same permissions as the "root" user on /etc/passwd
|
||||
# to allow a user belonging to the root group to add new users; typically the
|
||||
# docker user (see entrypoint).
|
||||
RUN chmod g=u /etc/passwd
|
||||
|
||||
# Copy installed python dependencies
|
||||
COPY --from=back-builder /install /usr/local
|
||||
|
||||
# Copy conversations application (see .dockerignore)
|
||||
COPY ./src/backend /app/
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Generate compiled translation messages
|
||||
RUN DJANGO_CONFIGURATION=Build \
|
||||
python manage.py compilemessages
|
||||
|
||||
|
||||
# We wrap commands run in this container by the following entrypoint that
|
||||
# creates a user on-the-fly with the container user ID (see USER) and root group
|
||||
# ID.
|
||||
ENTRYPOINT [ "/usr/local/bin/entrypoint" ]
|
||||
|
||||
# ---- Development image ----
|
||||
FROM core AS backend-development
|
||||
|
||||
# Switch back to the root user to install development dependencies
|
||||
USER root:root
|
||||
|
||||
# Install psql
|
||||
RUN apk add postgresql-client
|
||||
|
||||
# Uninstall conversations and re-install it in editable mode along with development
|
||||
# dependencies
|
||||
RUN pip uninstall -y conversations
|
||||
RUN pip install -e .[dev]
|
||||
|
||||
# Restore the un-privileged user running the application
|
||||
ARG DOCKER_USER
|
||||
USER ${DOCKER_USER}
|
||||
|
||||
# Target database host (e.g. database engine following docker compose services
|
||||
# name) & port
|
||||
ENV DB_HOST=postgresql \
|
||||
DB_PORT=5432
|
||||
|
||||
# Run django development server
|
||||
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
|
||||
|
||||
# ---- Production image ----
|
||||
FROM core AS backend-production
|
||||
|
||||
# Remove apk cache, we don't need it anymore
|
||||
RUN rm -rf /var/cache/apk/*
|
||||
|
||||
ARG CONVERSATIONS_STATIC_ROOT=/data/static
|
||||
|
||||
# Gunicorn
|
||||
RUN mkdir -p /usr/local/etc/gunicorn
|
||||
COPY docker/files/usr/local/etc/gunicorn/conversations.py /usr/local/etc/gunicorn/conversations.py
|
||||
|
||||
# Un-privileged user running the application
|
||||
ARG DOCKER_USER
|
||||
USER ${DOCKER_USER}
|
||||
|
||||
# Copy statics
|
||||
COPY --from=link-collector ${CONVERSATIONS_STATIC_ROOT} ${CONVERSATIONS_STATIC_ROOT}
|
||||
|
||||
# Copy conversations mails
|
||||
COPY --from=mail-builder /mail/backend/core/templates/mail /app/core/templates/mail
|
||||
|
||||
# The default command runs gunicorn WSGI server in conversations's main module
|
||||
CMD ["gunicorn", "-c", "/usr/local/etc/gunicorn/conversations.py", "conversations.wsgi:application"]
|
||||
@@ -0,0 +1,383 @@
|
||||
# /!\ /!\ /!\ /!\ /!\ /!\ /!\ DISCLAIMER /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\
|
||||
#
|
||||
# This Makefile is only meant to be used for DEVELOPMENT purpose as we are
|
||||
# changing the user id that will run in the container.
|
||||
#
|
||||
# PLEASE DO NOT USE IT FOR YOUR CI/PRODUCTION/WHATEVER...
|
||||
#
|
||||
# /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\ /!\
|
||||
#
|
||||
# Note to developers:
|
||||
#
|
||||
# While editing this file, please respect the following statements:
|
||||
#
|
||||
# 1. Every variable should be defined in the ad hoc VARIABLES section with a
|
||||
# relevant subsection
|
||||
# 2. Every new rule should be defined in the ad hoc RULES section with a
|
||||
# relevant subsection depending on the targeted service
|
||||
# 3. Rules should be sorted alphabetically within their section
|
||||
# 4. When a rule has multiple dependencies, you should:
|
||||
# - duplicate the rule name to add the help string (if required)
|
||||
# - write one dependency per line to increase readability and diffs
|
||||
# 5. .PHONY rule statement should be written after the corresponding rule
|
||||
# ==============================================================================
|
||||
# VARIABLES
|
||||
|
||||
BOLD := \033[1m
|
||||
RESET := \033[0m
|
||||
GREEN := \033[1;32m
|
||||
|
||||
|
||||
# -- Database
|
||||
|
||||
DB_HOST = postgresql
|
||||
DB_PORT = 5432
|
||||
|
||||
# -- Docker
|
||||
# Get the current user ID to use for docker run and docker exec commands
|
||||
DOCKER_UID = $(shell id -u)
|
||||
DOCKER_GID = $(shell id -g)
|
||||
DOCKER_USER = $(DOCKER_UID):$(DOCKER_GID)
|
||||
COMPOSE = DOCKER_USER=$(DOCKER_USER) docker compose
|
||||
COMPOSE_E2E = DOCKER_USER=$(DOCKER_USER) docker compose -f compose.yml -f compose-e2e.yml
|
||||
COMPOSE_EXEC = $(COMPOSE) exec
|
||||
COMPOSE_EXEC_APP = $(COMPOSE_EXEC) app-dev
|
||||
COMPOSE_RUN = $(COMPOSE) run --rm
|
||||
COMPOSE_RUN_APP = $(COMPOSE_RUN) app-dev
|
||||
COMPOSE_RUN_CROWDIN = $(COMPOSE_RUN) crowdin crowdin
|
||||
|
||||
# -- Backend
|
||||
MANAGE = $(COMPOSE_RUN_APP) python manage.py
|
||||
MAIL_YARN = $(COMPOSE_RUN) -w /app/src/mail node yarn
|
||||
|
||||
# -- Frontend
|
||||
PATH_FRONT = ./src/frontend
|
||||
PATH_FRONT_CONVERSATIONS = $(PATH_FRONT)/apps/conversations
|
||||
|
||||
# ==============================================================================
|
||||
# RULES
|
||||
|
||||
default: help
|
||||
|
||||
data/media:
|
||||
@mkdir -p data/media
|
||||
|
||||
data/static:
|
||||
@mkdir -p data/static
|
||||
|
||||
# -- Project
|
||||
|
||||
create-env-files: ## Copy the dist env files to env files
|
||||
create-env-files: \
|
||||
env.d/development/common \
|
||||
env.d/development/crowdin \
|
||||
env.d/development/postgresql \
|
||||
env.d/development/kc_postgresql
|
||||
.PHONY: create-env-files
|
||||
|
||||
pre-bootstrap: \
|
||||
data/media \
|
||||
data/static \
|
||||
create-env-files
|
||||
.PHONY: pre-bootstrap
|
||||
|
||||
post-bootstrap: \
|
||||
migrate \
|
||||
demo \
|
||||
back-i18n-compile \
|
||||
mails-install \
|
||||
mails-build
|
||||
.PHONY: post-bootstrap
|
||||
|
||||
|
||||
bootstrap: ## Prepare Docker developmentimages for the project
|
||||
bootstrap: \
|
||||
pre-bootstrap \
|
||||
build \
|
||||
post-bootstrap \
|
||||
run
|
||||
.PHONY: bootstrap
|
||||
|
||||
bootstrap-e2e: ## Prepare Docker production images to be used for e2e tests
|
||||
bootstrap-e2e: \
|
||||
pre-bootstrap \
|
||||
build-e2e \
|
||||
post-bootstrap \
|
||||
run-e2e
|
||||
.PHONY: bootstrap-e2e
|
||||
|
||||
# -- Docker/compose
|
||||
build: cache ?=
|
||||
build: ## build the project containers
|
||||
@$(MAKE) build-backend cache=$(cache)
|
||||
@$(MAKE) build-frontend cache=$(cache)
|
||||
.PHONY: build
|
||||
|
||||
build-backend: cache ?=
|
||||
build-backend: ## build the app-dev container
|
||||
@$(COMPOSE) build app-dev $(cache)
|
||||
.PHONY: build-backend
|
||||
|
||||
build-frontend: cache ?=
|
||||
build-frontend: ## build the frontend container
|
||||
@$(COMPOSE) build frontend-development $(cache)
|
||||
.PHONY: build-frontend
|
||||
|
||||
build-e2e: cache ?=
|
||||
build-e2e: ## build the e2e container
|
||||
@$(MAKE) build-backend cache=$(cache)
|
||||
@$(COMPOSE_E2E) build frontend $(cache)
|
||||
.PHONY: build-e2e
|
||||
|
||||
down: ## stop and remove containers, networks, images, and volumes
|
||||
@$(COMPOSE_E2E) down
|
||||
.PHONY: down
|
||||
|
||||
logs: ## display app-dev logs (follow mode)
|
||||
@$(COMPOSE) logs -f app-dev
|
||||
.PHONY: logs
|
||||
|
||||
run-backend: ## Start only the backend application and all needed services
|
||||
@$(COMPOSE) up --force-recreate -d nginx app-dev
|
||||
.PHONY: run-backend
|
||||
|
||||
run-frontend: ## Start only the frontend application
|
||||
@$(COMPOSE) up --force-recreate -d frontend-development
|
||||
.PHONY: run-frontend
|
||||
|
||||
run: ## start the wsgi (production) and development server
|
||||
run:
|
||||
@$(MAKE) run-backend
|
||||
@$(MAKE) run-frontend
|
||||
.PHONY: run
|
||||
|
||||
create-compose-with-models: ## override the docker-compose file with models
|
||||
cp -n compose.with_model.override.yml compose.override.yml
|
||||
.PHONY: create-compose-with-models
|
||||
|
||||
run-e2e: ## start the e2e server
|
||||
run-e2e:
|
||||
@$(MAKE) run-backend
|
||||
@$(COMPOSE_E2E) up --force-recreate -d frontend
|
||||
.PHONY: run-e2e
|
||||
|
||||
status: ## an alias for "docker compose ps"
|
||||
@$(COMPOSE_E2E) ps
|
||||
.PHONY: status
|
||||
|
||||
stop: ## stop the development server using Docker
|
||||
@$(COMPOSE_E2E) stop
|
||||
.PHONY: stop
|
||||
|
||||
# -- Backend
|
||||
|
||||
demo: ## flush db then create a demo for load testing purpose
|
||||
@$(MAKE) resetdb
|
||||
@$(MANAGE) create_demo
|
||||
.PHONY: demo
|
||||
|
||||
# Nota bene: Black should come after isort just in case they don't agree...
|
||||
lint: ## lint back-end python sources
|
||||
lint: \
|
||||
lint-ruff-format \
|
||||
lint-ruff-check \
|
||||
lint-pylint
|
||||
.PHONY: lint
|
||||
|
||||
lint-ruff-format: ## format back-end python sources with ruff
|
||||
@echo 'lint:ruff-format started…'
|
||||
@$(COMPOSE_RUN_APP) ruff format .
|
||||
.PHONY: lint-ruff-format
|
||||
|
||||
lint-ruff-check: ## lint back-end python sources with ruff
|
||||
@echo 'lint:ruff-check started…'
|
||||
@$(COMPOSE_RUN_APP) ruff check . --fix
|
||||
.PHONY: lint-ruff-check
|
||||
|
||||
lint-pylint: ## lint back-end python sources with pylint only on changed files from main
|
||||
@echo 'lint:pylint started…'
|
||||
bin/pylint --diff-only=origin/main
|
||||
.PHONY: lint-pylint
|
||||
|
||||
test: ## run project tests
|
||||
@$(MAKE) test-back-parallel
|
||||
.PHONY: test
|
||||
|
||||
test-back: ## run back-end tests
|
||||
@args="$(filter-out $@,$(MAKECMDGOALS))" && \
|
||||
bin/pytest $${args:-${1}}
|
||||
.PHONY: test-back
|
||||
|
||||
test-back-parallel: ## run all back-end tests in parallel
|
||||
@args="$(filter-out $@,$(MAKECMDGOALS))" && \
|
||||
bin/pytest -n auto $${args:-${1}}
|
||||
.PHONY: test-back-parallel
|
||||
|
||||
makemigrations: ## run django makemigrations for the conversations project.
|
||||
@echo "$(BOLD)Running makemigrations$(RESET)"
|
||||
@$(COMPOSE) up -d postgresql
|
||||
@$(MANAGE) makemigrations
|
||||
.PHONY: makemigrations
|
||||
|
||||
migrate: ## run django migrations for the conversations project.
|
||||
@echo "$(BOLD)Running migrations$(RESET)"
|
||||
@$(COMPOSE) up -d postgresql
|
||||
@$(MANAGE) migrate
|
||||
.PHONY: migrate
|
||||
|
||||
superuser: ## Create an admin superuser with password "admin"
|
||||
@echo "$(BOLD)Creating a Django superuser$(RESET)"
|
||||
@$(MANAGE) createsuperuser --email admin@example.com --password admin
|
||||
.PHONY: superuser
|
||||
|
||||
back-i18n-compile: ## compile the gettext files
|
||||
@$(MANAGE) compilemessages --ignore="venv/**/*"
|
||||
.PHONY: back-i18n-compile
|
||||
|
||||
back-i18n-generate: ## create the .pot files used for i18n
|
||||
@$(MANAGE) makemessages -a --keep-pot --all
|
||||
.PHONY: back-i18n-generate
|
||||
|
||||
shell: ## connect to database shell
|
||||
@$(MANAGE) shell #_plus
|
||||
.PHONY: dbshell
|
||||
|
||||
# -- Database
|
||||
|
||||
dbshell: ## connect to database shell
|
||||
docker compose exec app-dev python manage.py dbshell
|
||||
.PHONY: dbshell
|
||||
|
||||
resetdb: FLUSH_ARGS ?=
|
||||
resetdb: ## flush database and create a superuser "admin"
|
||||
@echo "$(BOLD)Flush database$(RESET)"
|
||||
@$(MANAGE) flush $(FLUSH_ARGS)
|
||||
@${MAKE} superuser
|
||||
.PHONY: resetdb
|
||||
|
||||
env.d/development/common:
|
||||
cp -n env.d/development/common.dist env.d/development/common
|
||||
|
||||
env.d/development/postgresql:
|
||||
cp -n env.d/development/postgresql.dist env.d/development/postgresql
|
||||
|
||||
env.d/development/kc_postgresql:
|
||||
cp -n env.d/development/kc_postgresql.dist env.d/development/kc_postgresql
|
||||
|
||||
# -- Internationalization
|
||||
|
||||
env.d/development/crowdin:
|
||||
cp -n env.d/development/crowdin.dist env.d/development/crowdin
|
||||
|
||||
crowdin-download: ## Download translated message from crowdin
|
||||
@$(COMPOSE_RUN_CROWDIN) download -c crowdin/config.yml
|
||||
.PHONY: crowdin-download
|
||||
|
||||
crowdin-download-sources: ## Download sources from Crowdin
|
||||
@$(COMPOSE_RUN_CROWDIN) download sources -c crowdin/config.yml
|
||||
.PHONY: crowdin-download-sources
|
||||
|
||||
crowdin-upload: ## Upload source translations to crowdin
|
||||
@$(COMPOSE_RUN_CROWDIN) upload sources -c crowdin/config.yml
|
||||
.PHONY: crowdin-upload
|
||||
|
||||
i18n-compile: ## compile all translations
|
||||
i18n-compile: \
|
||||
back-i18n-compile \
|
||||
frontend-i18n-compile
|
||||
.PHONY: i18n-compile
|
||||
|
||||
i18n-generate: ## create the .pot files and extract frontend messages
|
||||
i18n-generate: \
|
||||
back-i18n-generate \
|
||||
frontend-i18n-generate
|
||||
.PHONY: i18n-generate
|
||||
|
||||
i18n-download-and-compile: ## download all translated messages and compile them to be used by all applications
|
||||
i18n-download-and-compile: \
|
||||
crowdin-download \
|
||||
i18n-compile
|
||||
.PHONY: i18n-download-and-compile
|
||||
|
||||
i18n-generate-and-upload: ## generate source translations for all applications and upload them to Crowdin
|
||||
i18n-generate-and-upload: \
|
||||
i18n-generate \
|
||||
crowdin-upload
|
||||
.PHONY: i18n-generate-and-upload
|
||||
|
||||
|
||||
# -- Mail generator
|
||||
|
||||
mails-build: ## Convert mjml files to html and text
|
||||
@$(MAIL_YARN) build
|
||||
.PHONY: mails-build
|
||||
|
||||
mails-build-html-to-plain-text: ## Convert html files to text
|
||||
@$(MAIL_YARN) build-html-to-plain-text
|
||||
.PHONY: mails-build-html-to-plain-text
|
||||
|
||||
mails-build-mjml-to-html: ## Convert mjml files to html and text
|
||||
@$(MAIL_YARN) build-mjml-to-html
|
||||
.PHONY: mails-build-mjml-to-html
|
||||
|
||||
mails-install: ## install the mail generator
|
||||
@$(MAIL_YARN) install
|
||||
.PHONY: mails-install
|
||||
|
||||
# -- Misc
|
||||
clean: ## restore repository state as it was freshly cloned
|
||||
git clean -idx
|
||||
.PHONY: clean
|
||||
|
||||
help:
|
||||
@echo "$(BOLD)conversations Makefile"
|
||||
@echo "Please use 'make $(BOLD)target$(RESET)' where $(BOLD)target$(RESET) is one of:"
|
||||
@grep -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(firstword $(MAKEFILE_LIST)) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf "$(GREEN)%-30s$(RESET) %s\n", $$1, $$2}'
|
||||
.PHONY: help
|
||||
|
||||
# Front
|
||||
frontend-development-install: ## install the frontend locally
|
||||
cd $(PATH_FRONT_CONVERSATIONS) && yarn
|
||||
.PHONY: frontend-development-install
|
||||
|
||||
frontend-lint: ## run the frontend linter
|
||||
cd $(PATH_FRONT) && yarn lint
|
||||
.PHONY: frontend-lint
|
||||
|
||||
run-frontend-development: ## Run the frontend in development mode
|
||||
#@$(COMPOSE) stop frontend frontend-development
|
||||
cd $(PATH_FRONT_CONVERSATIONS) && yarn dev
|
||||
.PHONY: run-frontend-development
|
||||
|
||||
frontend-i18n-extract: ## Extract the frontend translation inside a json to be used for crowdin
|
||||
cd $(PATH_FRONT) && yarn i18n:extract
|
||||
.PHONY: frontend-i18n-extract
|
||||
|
||||
frontend-i18n-generate: ## Generate the frontend json files used for crowdin
|
||||
frontend-i18n-generate: \
|
||||
crowdin-download-sources \
|
||||
frontend-i18n-extract
|
||||
.PHONY: frontend-i18n-generate
|
||||
|
||||
frontend-i18n-compile: ## Format the crowin json files used deploy to the apps
|
||||
cd $(PATH_FRONT) && yarn i18n:deploy
|
||||
.PHONY: frontend-i18n-compile
|
||||
|
||||
# -- K8S
|
||||
build-k8s-cluster: ## build the kubernetes cluster using kind
|
||||
./bin/start-kind.sh
|
||||
.PHONY: build-k8s-cluster
|
||||
|
||||
start-tilt: ## start the kubernetes cluster using kind
|
||||
tilt up -f ./bin/Tiltfile
|
||||
.PHONY: build-k8s-cluster
|
||||
|
||||
bump-packages-version: VERSION_TYPE ?= minor
|
||||
bump-packages-version: ## bump the version of the project - VERSION_TYPE can be "major", "minor", "patch"
|
||||
cd ./src/mail && yarn version --no-git-tag-version --$(VERSION_TYPE)
|
||||
cd ./src/frontend/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
|
||||
cd ./src/frontend/apps/e2e/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
|
||||
cd ./src/frontend/apps/conversations/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
|
||||
cd ./src/frontend/packages/eslint-config-conversations/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
|
||||
cd ./src/frontend/packages/i18n/ && yarn version --no-git-tag-version --$(VERSION_TYPE)
|
||||
.PHONY: bump-packages-version
|
||||
@@ -0,0 +1,60 @@
|
||||
load('ext://uibutton', 'cmd_button', 'bool_input', 'location')
|
||||
load('ext://namespace', 'namespace_create', 'namespace_inject')
|
||||
namespace_create('conversations')
|
||||
|
||||
docker_build(
|
||||
'localhost:5001/conversations-backend:latest',
|
||||
context='..',
|
||||
dockerfile='../Dockerfile',
|
||||
only=['./src/backend', './src/mail', './docker'],
|
||||
target = 'backend-production',
|
||||
live_update=[
|
||||
sync('../src/backend', '/app'),
|
||||
run(
|
||||
'pip install -r /app/requirements.txt',
|
||||
trigger=['./api/requirements.txt']
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
docker_build(
|
||||
'localhost:5001/conversations-frontend:latest',
|
||||
context='..',
|
||||
dockerfile='../src/frontend/Dockerfile',
|
||||
only=['./src/frontend', './docker', './.dockerignore'],
|
||||
target = 'conversations',
|
||||
live_update=[
|
||||
sync('../src/frontend', '/home/frontend'),
|
||||
]
|
||||
)
|
||||
|
||||
k8s_resource('conversations-conversations-backend-migrate', resource_deps=['postgres-postgresql'])
|
||||
k8s_resource('conversations-conversations-backend-createsuperuser', resource_deps=['conversations-conversations-backend-migrate'])
|
||||
k8s_resource('conversations-conversations-backend', resource_deps=['conversations-conversations-backend-migrate'])
|
||||
k8s_yaml(local('cd ../src/helm && helmfile -n conversations -e dev template .'))
|
||||
|
||||
migration = '''
|
||||
set -eu
|
||||
# get k8s pod name from tilt resource name
|
||||
POD_NAME="$(tilt get kubernetesdiscovery conversations-backend -ojsonpath='{.status.pods[0].name}')"
|
||||
kubectl -n conversations exec "$POD_NAME" -- python manage.py makemigrations
|
||||
'''
|
||||
cmd_button('Make migration',
|
||||
argv=['sh', '-c', migration],
|
||||
resource='conversations-backend',
|
||||
icon_name='developer_board',
|
||||
text='Run makemigration',
|
||||
)
|
||||
|
||||
pod_migrate = '''
|
||||
set -eu
|
||||
# get k8s pod name from tilt resource name
|
||||
POD_NAME="$(tilt get kubernetesdiscovery conversations-backend -ojsonpath='{.status.pods[0].name}')"
|
||||
kubectl -n conversations exec "$POD_NAME" -- python manage.py migrate --no-input
|
||||
'''
|
||||
cmd_button('Migrate db',
|
||||
argv=['sh', '-c', pod_migrate],
|
||||
resource='conversations-backend',
|
||||
icon_name='developer_board',
|
||||
text='Run database migration',
|
||||
)
|
||||
+155
@@ -0,0 +1,155 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
set -eo pipefail
|
||||
|
||||
REPO_DIR="$(cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd)"
|
||||
UNSET_USER=0
|
||||
|
||||
TERRAFORM_DIRECTORY="./env.d/terraform"
|
||||
COMPOSE_FILE="${REPO_DIR}/compose.yml"
|
||||
|
||||
|
||||
# _set_user: set (or unset) default user id used to run docker commands
|
||||
#
|
||||
# usage: _set_user
|
||||
#
|
||||
# You can override default user ID (the current host user ID), by defining the
|
||||
# USER_ID environment variable.
|
||||
#
|
||||
# To avoid running docker commands with a custom user, please set the
|
||||
# $UNSET_USER environment variable to 1.
|
||||
function _set_user() {
|
||||
|
||||
if [ $UNSET_USER -eq 1 ]; then
|
||||
USER_ID=""
|
||||
return
|
||||
fi
|
||||
|
||||
# USER_ID = USER_ID or `id -u` if USER_ID is not set
|
||||
USER_ID=${USER_ID:-$(id -u)}
|
||||
|
||||
echo "🙋(user) ID: ${USER_ID}"
|
||||
}
|
||||
|
||||
# docker_compose: wrap docker compose command
|
||||
#
|
||||
# usage: docker_compose [options] [ARGS...]
|
||||
#
|
||||
# options: docker compose command options
|
||||
# ARGS : docker compose command arguments
|
||||
function _docker_compose() {
|
||||
|
||||
echo "🐳(compose) file: '${COMPOSE_FILE}'"
|
||||
docker compose \
|
||||
-f "${COMPOSE_FILE}" \
|
||||
--project-directory "${REPO_DIR}" \
|
||||
"$@"
|
||||
}
|
||||
|
||||
# _dc_run: wrap docker compose run command
|
||||
#
|
||||
# usage: _dc_run [options] [ARGS...]
|
||||
#
|
||||
# options: docker compose run command options
|
||||
# ARGS : docker compose run command arguments
|
||||
function _dc_run() {
|
||||
_set_user
|
||||
|
||||
user_args="--user=$USER_ID"
|
||||
if [ -z $USER_ID ]; then
|
||||
user_args=""
|
||||
fi
|
||||
|
||||
_docker_compose run --rm $user_args "$@"
|
||||
}
|
||||
|
||||
# _dc_exec: wrap docker compose exec command
|
||||
#
|
||||
# usage: _dc_exec [options] [ARGS...]
|
||||
#
|
||||
# options: docker compose exec command options
|
||||
# ARGS : docker compose exec command arguments
|
||||
function _dc_exec() {
|
||||
_set_user
|
||||
|
||||
echo "🐳(compose) exec command: '\$@'"
|
||||
|
||||
user_args="--user=$USER_ID"
|
||||
if [ -z $USER_ID ]; then
|
||||
user_args=""
|
||||
fi
|
||||
|
||||
_docker_compose exec $user_args "$@"
|
||||
}
|
||||
|
||||
# _django_manage: wrap django's manage.py command with docker compose
|
||||
#
|
||||
# usage : _django_manage [ARGS...]
|
||||
#
|
||||
# ARGS : django's manage.py command arguments
|
||||
function _django_manage() {
|
||||
_dc_run "app-dev" python manage.py "$@"
|
||||
}
|
||||
|
||||
# _set_openstack_project: select an OpenStack project from the openrc files defined in the
|
||||
# terraform directory.
|
||||
#
|
||||
# usage: _set_openstack_project
|
||||
#
|
||||
# If necessary the script will prompt the user to choose a project from those available
|
||||
function _set_openstack_project() {
|
||||
|
||||
declare prompt
|
||||
declare -a projects
|
||||
declare -i default=1
|
||||
declare -i choice=0
|
||||
declare -i n_projects
|
||||
|
||||
# List projects by looking in the "./env.d/terraform" directory
|
||||
# and store them in an array
|
||||
read -r -a projects <<< "$(
|
||||
find "${TERRAFORM_DIRECTORY}" -maxdepth 1 -mindepth 1 -type d |
|
||||
sed 's|'"${TERRAFORM_DIRECTORY}\/"'||' |
|
||||
xargs
|
||||
)"
|
||||
nb_projects=${#projects[@]}
|
||||
|
||||
if [[ ${nb_projects} -le 0 ]]; then
|
||||
echo "There are no OpenStack projects defined..." >&2
|
||||
echo "To add one, create a subdirectory in \"${TERRAFORM_DIRECTORY}\" with the name" \
|
||||
"of your project and copy your \"openrc.sh\" file into it." >&2
|
||||
exit 10
|
||||
fi
|
||||
|
||||
if [[ ${nb_projects} -gt 1 ]]; then
|
||||
prompt="Select an OpenStack project to target:\\n"
|
||||
for (( i=0; i<nb_projects; i++ )); do
|
||||
prompt+="[$((i+1))] ${projects[$i]}"
|
||||
if [[ $((i+1)) -eq ${default} ]]; then
|
||||
prompt+=" (default)"
|
||||
fi
|
||||
prompt+="\\n"
|
||||
done
|
||||
prompt+="If your OpenStack project is not listed, add it to the \"env.d/terraform\" directory.\\n"
|
||||
prompt+="Your choice: "
|
||||
read -r -p "$(echo -e "${prompt}")" choice
|
||||
|
||||
if [[ ${choice} -gt nb_projects ]]; then
|
||||
(>&2 echo "Invalid choice ${choice} (should be <= ${nb_projects})")
|
||||
exit 11
|
||||
fi
|
||||
|
||||
if [[ ${choice} -le 0 ]]; then
|
||||
choice=${default}
|
||||
fi
|
||||
fi
|
||||
|
||||
project=${projects[$((choice-1))]}
|
||||
# Check that the openrc.sh file exists for this project
|
||||
if [ ! -f "${TERRAFORM_DIRECTORY}/${project}/openrc.sh" ]; then
|
||||
(>&2 echo "Missing \"openrc.sh\" file in \"${TERRAFORM_DIRECTORY}/${project}\". Check documentation.")
|
||||
exit 12
|
||||
fi
|
||||
|
||||
echo "${project}"
|
||||
}
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# shellcheck source=bin/_config.sh
|
||||
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
|
||||
|
||||
_docker_compose "$@"
|
||||
@@ -0,0 +1,30 @@
|
||||
#!/bin/bash
|
||||
|
||||
mkdir -p "$(dirname -- "${BASH_SOURCE[0]}")/../.git/hooks/"
|
||||
PRE_COMMIT_FILE="$(dirname -- "${BASH_SOURCE[0]}")/../.git/hooks/pre-commit"
|
||||
|
||||
cat <<'EOF' >$PRE_COMMIT_FILE
|
||||
#!/bin/bash
|
||||
|
||||
# directories containing potential secrets
|
||||
DIRS="."
|
||||
|
||||
bold=$(tput bold)
|
||||
normal=$(tput sgr0)
|
||||
|
||||
# allow to read user input, assigns stdin to keyboard
|
||||
exec </dev/tty
|
||||
|
||||
for d in $DIRS; do
|
||||
# find files containing secrets that should be encrypted
|
||||
for f in $(find "${d}" -type f -regex ".*\.enc\..*"); do
|
||||
if ! $(grep -q "unencrypted_suffix" $f); then
|
||||
printf '\xF0\x9F\x92\xA5 '
|
||||
echo "File $f has non encrypted secrets!"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
done
|
||||
EOF
|
||||
|
||||
chmod +x $PRE_COMMIT_FILE
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# shellcheck source=bin/_config.sh
|
||||
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
|
||||
|
||||
_django_manage "$@"
|
||||
Executable
+38
@@ -0,0 +1,38 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
# shellcheck source=bin/_config.sh
|
||||
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
|
||||
|
||||
declare diff_from
|
||||
declare -a paths
|
||||
declare -a args
|
||||
|
||||
# Parse options
|
||||
for arg in "$@"
|
||||
do
|
||||
case $arg in
|
||||
--diff-only=*)
|
||||
diff_from="${arg#*=}"
|
||||
shift
|
||||
;;
|
||||
-*)
|
||||
args+=("$arg")
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
paths+=("$arg")
|
||||
shift
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -n "${diff_from}" ]]; then
|
||||
# Run pylint only on modified files located in src/backend
|
||||
# (excluding deleted files and migration files)
|
||||
# shellcheck disable=SC2207
|
||||
paths=($(git diff "${diff_from}" --name-only --diff-filter=d -- src/backend ':!**/migrations/*.py' | grep -E '^src/backend/.*\.py$'))
|
||||
fi
|
||||
|
||||
# Fix docker vs local path when project sources are mounted as a volume
|
||||
read -ra paths <<< "$(echo "${paths[@]}" | sed "s|src/backend/||g")"
|
||||
_dc_run app-dev pylint "${paths[@]}" "${args[@]}"
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
|
||||
|
||||
_dc_run \
|
||||
-e DJANGO_CONFIGURATION=Test \
|
||||
app-dev \
|
||||
pytest "$@"
|
||||
Executable
+2
@@ -0,0 +1,2 @@
|
||||
#!/bin/sh
|
||||
curl https://raw.githubusercontent.com/numerique-gouv/tools/refs/heads/main/kind/create_cluster.sh | bash -s -- conversations
|
||||
@@ -0,0 +1,25 @@
|
||||
#!/usr/bin/env bash
|
||||
set -eo pipefail
|
||||
|
||||
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
|
||||
|
||||
project=$(_set_openstack_project)
|
||||
echo "Using \"${project}\" project..."
|
||||
|
||||
source "${TERRAFORM_DIRECTORY}/${project}/openrc.sh"
|
||||
|
||||
# Run Terraform commands in the Hashicorp docker container via docker compose
|
||||
# shellcheck disable=SC2068
|
||||
DOCKER_USER="$(id -u):$(id -g)" \
|
||||
PROJECT="${project}" \
|
||||
docker compose run --rm \
|
||||
-e OS_AUTH_URL \
|
||||
-e OS_IDENTITY_API_VERSION \
|
||||
-e OS_USER_DOMAIN_NAME \
|
||||
-e OS_PROJECT_DOMAIN_NAME \
|
||||
-e OS_TENANT_ID \
|
||||
-e OS_TENANT_NAME \
|
||||
-e OS_USERNAME \
|
||||
-e OS_PASSWORD \
|
||||
-e OS_REGION_NAME \
|
||||
terraform-state "$@"
|
||||
Executable
+26
@@ -0,0 +1,26 @@
|
||||
#!/usr/bin/env bash
|
||||
set -eo pipefail
|
||||
|
||||
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
|
||||
|
||||
project=$(_set_openstack_project)
|
||||
echo "Using \"${project}\" project..."
|
||||
|
||||
source "${TERRAFORM_DIRECTORY}/${project}/openrc.sh"
|
||||
|
||||
# Run Terraform commands in the Hashicorp docker container via docker compose
|
||||
# shellcheck disable=SC2068
|
||||
DOCKER_USER="$(id -u):$(id -g)" \
|
||||
PROJECT="${project}" \
|
||||
docker compose run --rm \
|
||||
-e OS_AUTH_URL \
|
||||
-e OS_IDENTITY_API_VERSION \
|
||||
-e OS_USER_DOMAIN_NAME \
|
||||
-e OS_PROJECT_DOMAIN_NAME \
|
||||
-e OS_TENANT_ID \
|
||||
-e OS_TENANT_NAME \
|
||||
-e OS_USERNAME \
|
||||
-e OS_PASSWORD \
|
||||
-e OS_REGION_NAME \
|
||||
-e TF_VAR_user_name \
|
||||
terraform "$@"
|
||||
Executable
+12
@@ -0,0 +1,12 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
|
||||
|
||||
_dc_run \
|
||||
-e DJANGO_CONFIGURATION=Test \
|
||||
app-dev \
|
||||
python manage.py spectacular \
|
||||
--api-version 'v1.0' \
|
||||
--urlconf 'conversations.urls' \
|
||||
--format openapi-json \
|
||||
--file /app/core/tests/swagger/swagger.json
|
||||
@@ -0,0 +1,13 @@
|
||||
services:
|
||||
|
||||
frontend:
|
||||
user: "${DOCKER_USER:-1000}"
|
||||
build:
|
||||
context: .
|
||||
dockerfile: ./src/frontend/Dockerfile
|
||||
target: frontend-production
|
||||
args:
|
||||
API_ORIGIN: "http://localhost:8071"
|
||||
image: conversations:frontend-production
|
||||
ports:
|
||||
- "3000:3000"
|
||||
@@ -0,0 +1,12 @@
|
||||
name: conversations
|
||||
|
||||
services:
|
||||
app-dev:
|
||||
models:
|
||||
llm:
|
||||
endpoint_var: AI_BASE_URL
|
||||
model_var: AI_MODEL
|
||||
|
||||
models:
|
||||
llm:
|
||||
model: ai/smollm2
|
||||
+190
@@ -0,0 +1,190 @@
|
||||
name: conversations
|
||||
|
||||
services:
|
||||
postgresql:
|
||||
image: postgres:16
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
|
||||
interval: 1s
|
||||
timeout: 2s
|
||||
retries: 300
|
||||
env_file:
|
||||
- env.d/development/postgresql
|
||||
ports:
|
||||
- "15432:5432"
|
||||
|
||||
redis:
|
||||
image: redis:5
|
||||
|
||||
maildev:
|
||||
image: maildev/maildev:latest
|
||||
ports:
|
||||
- "1081:1080"
|
||||
|
||||
minio:
|
||||
user: ${DOCKER_USER:-1000}
|
||||
image: minio/minio
|
||||
environment:
|
||||
- MINIO_ROOT_USER=conversations
|
||||
- MINIO_ROOT_PASSWORD=password
|
||||
ports:
|
||||
- '9000:9000'
|
||||
- '9001:9001'
|
||||
healthcheck:
|
||||
test: ["CMD", "mc", "ready", "local"]
|
||||
interval: 1s
|
||||
timeout: 20s
|
||||
retries: 300
|
||||
entrypoint: ""
|
||||
command: minio server --console-address :9001 /data
|
||||
volumes:
|
||||
- ./data/media:/data
|
||||
|
||||
createbuckets:
|
||||
image: minio/mc
|
||||
depends_on:
|
||||
minio:
|
||||
condition: service_healthy
|
||||
restart: true
|
||||
entrypoint: >
|
||||
sh -c "
|
||||
/usr/bin/mc alias set conversations http://minio:9000 conversations password && \
|
||||
/usr/bin/mc mb conversations/conversations-media-storage && \
|
||||
/usr/bin/mc version enable conversations/conversations-media-storage && \
|
||||
exit 0;"
|
||||
|
||||
app-dev:
|
||||
build:
|
||||
context: .
|
||||
target: backend-development
|
||||
args:
|
||||
DOCKER_USER: ${DOCKER_USER:-1000}
|
||||
user: ${DOCKER_USER:-1000}
|
||||
image: conversations:backend-development
|
||||
environment:
|
||||
- PYLINTHOME=/app/.pylint.d
|
||||
- DJANGO_CONFIGURATION=Development
|
||||
env_file:
|
||||
- env.d/development/common
|
||||
- env.d/development/postgresql
|
||||
extra_hosts:
|
||||
- "host.docker.internal:host-gateway"
|
||||
ports:
|
||||
- "8071:8000"
|
||||
volumes:
|
||||
- ./src/backend:/app
|
||||
- ./data/static:/data/static
|
||||
depends_on:
|
||||
postgresql:
|
||||
condition: service_healthy
|
||||
restart: true
|
||||
maildev:
|
||||
condition: service_started
|
||||
redis:
|
||||
condition: service_started
|
||||
createbuckets:
|
||||
condition: service_started
|
||||
|
||||
nginx:
|
||||
image: nginx:1.25
|
||||
ports:
|
||||
- "8083:8083"
|
||||
volumes:
|
||||
- ./docker/files/etc/nginx/conf.d:/etc/nginx/conf.d:ro
|
||||
depends_on:
|
||||
app-dev:
|
||||
condition: service_started
|
||||
keycloak:
|
||||
condition: service_healthy
|
||||
restart: true
|
||||
|
||||
frontend-development:
|
||||
user: "${DOCKER_USER:-1000}"
|
||||
build:
|
||||
context: .
|
||||
dockerfile: ./src/frontend/Dockerfile
|
||||
target: conversations-dev
|
||||
args:
|
||||
API_ORIGIN: "http://localhost:8071"
|
||||
image: conversations:frontend-development
|
||||
volumes:
|
||||
- ./src/frontend:/home/frontend
|
||||
- /home/frontend/node_modules
|
||||
- /home/frontend/apps/conversations/node_modules
|
||||
ports:
|
||||
- "3000:3000"
|
||||
|
||||
crowdin:
|
||||
image: crowdin/cli:3.16.0
|
||||
volumes:
|
||||
- ".:/app"
|
||||
env_file:
|
||||
- env.d/development/crowdin
|
||||
user: "${DOCKER_USER:-1000}"
|
||||
working_dir: /app
|
||||
|
||||
node:
|
||||
image: node:22
|
||||
user: "${DOCKER_USER:-1000}"
|
||||
environment:
|
||||
HOME: /tmp
|
||||
volumes:
|
||||
- ".:/app"
|
||||
|
||||
kc_postgresql:
|
||||
image: postgres:17.5
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
|
||||
interval: 1s
|
||||
timeout: 2s
|
||||
retries: 300
|
||||
ports:
|
||||
- "5433:5432"
|
||||
env_file:
|
||||
- env.d/development/kc_postgresql
|
||||
|
||||
keycloak:
|
||||
image: quay.io/keycloak/keycloak:26.3
|
||||
volumes:
|
||||
- ./docker/auth/realm.json:/opt/keycloak/data/import/realm.json
|
||||
command:
|
||||
- start-dev
|
||||
- --features=preview
|
||||
- --import-realm
|
||||
- --proxy=edge
|
||||
- --hostname=http://localhost:8083
|
||||
- --hostname-strict=false
|
||||
- --health-enabled=true
|
||||
- --metrics-enabled=true
|
||||
- --http-enabled=true
|
||||
healthcheck:
|
||||
test: ['CMD-SHELL', '[ -f /tmp/HealthCheck.java ] || echo "public class HealthCheck { public static void main(String[] args) throws java.lang.Throwable { java.net.URI uri = java.net.URI.create(args[0]); System.exit(java.net.HttpURLConnection.HTTP_OK == ((java.net.HttpURLConnection)uri.toURL().openConnection()).getResponseCode() ? 0 : 1); } }" > /tmp/HealthCheck.java && java /tmp/HealthCheck.java http://localhost:9000/health/live']
|
||||
start_period: 10s
|
||||
interval: 1s
|
||||
timeout: 2s
|
||||
retries: 300
|
||||
environment:
|
||||
KC_BOOTSTRAP_ADMIN_USERNAME: admin
|
||||
KC_BOOTSTRAP_ADMIN_PASSWORD: admin
|
||||
KC_DB: postgres
|
||||
KC_DB_URL_HOST: kc_postgresql
|
||||
KC_DB_URL_DATABASE: keycloak
|
||||
KC_DB_PASSWORD: pass
|
||||
KC_DB_USERNAME: conversations
|
||||
KC_DB_SCHEMA: public
|
||||
PROXY_ADDRESS_FORWARDING: 'true'
|
||||
depends_on:
|
||||
kc_postgresql:
|
||||
condition: service_healthy
|
||||
restart: true
|
||||
|
||||
ml-flow:
|
||||
image: ghcr.io/mlflow/mlflow:v3.0.0
|
||||
environment:
|
||||
MLFLOW_TRACKING_URI: http://localhost:5050
|
||||
MLFLOW_ARTIFACT_ROOT: /mlflow/artifacts
|
||||
ports:
|
||||
- "5050:5050"
|
||||
volumes:
|
||||
- ./data/mlflow:/mlflow/artifacts
|
||||
command: mlflow server --host 0.0.0.0:5050
|
||||
@@ -0,0 +1,29 @@
|
||||
#
|
||||
# Your crowdin's credentials
|
||||
#
|
||||
api_token_env: CROWDIN_PERSONAL_TOKEN
|
||||
project_id_env: CROWDIN_PROJECT_ID
|
||||
base_path_env: CROWDIN_BASE_PATH
|
||||
|
||||
#
|
||||
# Choose file structure in crowdin
|
||||
# e.g. true or false
|
||||
#
|
||||
preserve_hierarchy: true
|
||||
|
||||
#
|
||||
# Files configuration
|
||||
#
|
||||
files: [
|
||||
{
|
||||
source : "/backend/locale/django.pot",
|
||||
dest: "/backend-conversations.pot",
|
||||
translation : "/backend/locale/%locale_with_underscore%/LC_MESSAGES/django.po"
|
||||
},
|
||||
{
|
||||
source: "/frontend/packages/i18n/locales/conversations/translations-crowdin.json",
|
||||
dest: "/frontend-conversations.json",
|
||||
translation: "/frontend/packages/i18n/locales/conversations/%two_letters_code%/translations.json",
|
||||
skip_untranslated_strings: true,
|
||||
},
|
||||
]
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,52 @@
|
||||
|
||||
server {
|
||||
listen 8083;
|
||||
server_name localhost;
|
||||
charset utf-8;
|
||||
|
||||
# Proxy auth for media
|
||||
location /media/ {
|
||||
# Auth request configuration
|
||||
auth_request /media-auth;
|
||||
auth_request_set $authHeader $upstream_http_authorization;
|
||||
auth_request_set $authDate $upstream_http_x_amz_date;
|
||||
auth_request_set $authContentSha256 $upstream_http_x_amz_content_sha256;
|
||||
|
||||
# Pass specific headers from the auth response
|
||||
proxy_set_header Authorization $authHeader;
|
||||
proxy_set_header X-Amz-Date $authDate;
|
||||
proxy_set_header X-Amz-Content-SHA256 $authContentSha256;
|
||||
|
||||
# Get resource from Minio
|
||||
proxy_pass http://minio:9000/conversations-media-storage/;
|
||||
proxy_set_header Host minio:9000;
|
||||
|
||||
add_header Content-Security-Policy "default-src 'none'" always;
|
||||
}
|
||||
|
||||
location /media-auth {
|
||||
proxy_pass http://app-dev:8000/api/v1.0/documents/media-auth/;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Original-URL $request_uri;
|
||||
|
||||
# Prevent the body from being passed
|
||||
proxy_pass_request_body off;
|
||||
proxy_set_header Content-Length "";
|
||||
proxy_set_header X-Original-Method $request_method;
|
||||
}
|
||||
|
||||
location / {
|
||||
proxy_pass http://keycloak:8080;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
|
||||
# Increase proxy buffer size to allow keycloak to send large
|
||||
# header responses when a user is created.
|
||||
proxy_buffer_size 128k;
|
||||
proxy_buffers 4 256k;
|
||||
proxy_busy_buffers_size 256k;
|
||||
}
|
||||
}
|
||||
Executable
+35
@@ -0,0 +1,35 @@
|
||||
#!/bin/sh
|
||||
#
|
||||
# The container user (see USER in the Dockerfile) is an un-privileged user that
|
||||
# does not exists and is not created during the build phase (see Dockerfile).
|
||||
# Hence, we use this entrypoint to wrap commands that will be run in the
|
||||
# container to create an entry for this user in the /etc/passwd file.
|
||||
#
|
||||
# The following environment variables may be passed to the container to
|
||||
# customize running user account:
|
||||
#
|
||||
# * USER_NAME: container user name (default: default)
|
||||
# * HOME : container user home directory (default: none)
|
||||
#
|
||||
# To pass environment variables, you can either use the -e option of the docker run command:
|
||||
#
|
||||
# docker run --rm -e USER_NAME=foo -e HOME='/home/foo' conversations:latest python manage.py migrate
|
||||
#
|
||||
# or define new variables in an environment file to use with docker or docker compose:
|
||||
#
|
||||
# # env.d/production
|
||||
# USER_NAME=foo
|
||||
# HOME=/home/foo
|
||||
#
|
||||
# docker run --rm --env-file env.d/production conversations:latest python manage.py migrate
|
||||
#
|
||||
|
||||
echo "🐳(entrypoint) creating user running in the container..."
|
||||
if ! whoami > /dev/null 2>&1; then
|
||||
if [ -w /etc/passwd ]; then
|
||||
echo "${USER_NAME:-default}:x:$(id -u):$(id -g):${USER_NAME:-default} user:${HOME}:/sbin/nologin" >> /etc/passwd
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "🐳(entrypoint) running your command: ${*}"
|
||||
exec "$@"
|
||||
@@ -0,0 +1,16 @@
|
||||
# Gunicorn-django settings
|
||||
bind = ["0.0.0.0:8000"]
|
||||
name = "conversations"
|
||||
python_path = "/app"
|
||||
|
||||
# Run
|
||||
graceful_timeout = 90
|
||||
timeout = 90
|
||||
workers = 3
|
||||
|
||||
# Logging
|
||||
# Using '-' for the access log file makes gunicorn log accesses to stdout
|
||||
accesslog = "-"
|
||||
# Using '-' for the error log file makes gunicorn log errors to stderr
|
||||
errorlog = "-"
|
||||
loglevel = "info"
|
||||
@@ -0,0 +1,19 @@
|
||||
## Architecture
|
||||
|
||||
### Global system architecture
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
User -- HTTP --> Front("Frontend (NextJS SPA)")
|
||||
Front -- REST API --> Back("Backend (Django)")
|
||||
Front -- WebSocket --> Yserver("Microservice Yjs (Express)") -- WebSocket --> CollaborationServer("Collaboration server (Hocuspocus)") -- REST API <--> Back
|
||||
Front -- OIDC --> Back -- OIDC ---> OIDC("Keycloak / ProConnect")
|
||||
Back -- REST API --> Yserver
|
||||
Back --> DB("Database (PostgreSQL)")
|
||||
Back <--> Celery --> DB
|
||||
Back ----> S3("Minio (S3)")
|
||||
```
|
||||
|
||||
### Architecture decision records
|
||||
|
||||
- [ADR-0001-20250106-use-yjs-for-docs-editing](./adr/ADR-0001-20250106-use-yjs-for-docs-editing.md)
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 4.3 KiB |
+127
@@ -0,0 +1,127 @@
|
||||
# Conversations variables
|
||||
|
||||
Here we describe all environment variables that can be set for the conversations application.
|
||||
|
||||
## conversations-backend container
|
||||
|
||||
These are the environment variables you can set for the `conversations-backend` container.
|
||||
|
||||
| Option | Description | default |
|
||||
|-------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------|
|
||||
| DJANGO_ALLOWED_HOSTS | allowed hosts | [] |
|
||||
| DJANGO_SECRET_KEY | secret key | |
|
||||
| DJANGO_SERVER_TO_SERVER_API_TOKENS | | [] |
|
||||
| DB_ENGINE | engine to use for database connections | django.db.backends.postgresql_psycopg2 |
|
||||
| DB_NAME | name of the database | conversations |
|
||||
| DB_USER | user to authenticate with | dinum |
|
||||
| DB_PASSWORD | password to authenticate with | pass |
|
||||
| DB_HOST | host of the database | localhost |
|
||||
| DB_PORT | port of the database | 5432 |
|
||||
| MEDIA_BASE_URL | | |
|
||||
| STORAGES_STATICFILES_BACKEND | | whitenoise.storage.CompressedManifestStaticFilesStorage |
|
||||
| AWS_S3_ENDPOINT_URL | S3 endpoint | |
|
||||
| AWS_S3_ACCESS_KEY_ID | access id for s3 endpoint | |
|
||||
| AWS_S3_SECRET_ACCESS_KEY | access key for s3 endpoint | |
|
||||
| AWS_S3_REGION_NAME | region name for s3 endpoint | |
|
||||
| AWS_STORAGE_BUCKET_NAME | bucket name for s3 endpoint | conversations-media-storage |
|
||||
| DOCUMENT_IMAGE_MAX_SIZE | maximum size of document in bytes | 10485760 |
|
||||
| LANGUAGE_CODE | default language | en-us |
|
||||
| API_USERS_LIST_THROTTLE_RATE_SUSTAINED | throttle rate for api | 180/hour |
|
||||
| API_USERS_LIST_THROTTLE_RATE_BURST | throttle rate for api on burst | 30/minute |
|
||||
| SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK | | false |
|
||||
| TRASHBIN_CUTOFF_DAYS | trashbin cutoff | 30 |
|
||||
| DJANGO_EMAIL_BACKEND | email backend library | django.core.mail.backends.smtp.EmailBackend |
|
||||
| DJANGO_EMAIL_BRAND_NAME | brand name for email | |
|
||||
| DJANGO_EMAIL_HOST | host name of email | |
|
||||
| DJANGO_EMAIL_HOST_USER | user to authenticate with on the email host | |
|
||||
| DJANGO_EMAIL_HOST_PASSWORD | password to authenticate with on the email host | |
|
||||
| DJANGO_EMAIL_LOGO_IMG | logo for the email | |
|
||||
| DJANGO_EMAIL_PORT | port used to connect to email host | |
|
||||
| DJANGO_EMAIL_USE_TLS | use tls for email host connection | false |
|
||||
| DJANGO_EMAIL_USE_SSL | use sstl for email host connection | false |
|
||||
| DJANGO_EMAIL_FROM | email address used as sender | from@example.com |
|
||||
| DJANGO_CORS_ALLOW_ALL_ORIGINS | allow all CORS origins | false |
|
||||
| DJANGO_CORS_ALLOWED_ORIGINS | list of origins allowed for CORS | [] |
|
||||
| DJANGO_CORS_ALLOWED_ORIGIN_REGEXES | list of origins allowed for CORS using regulair expressions | [] |
|
||||
| SENTRY_DSN | sentry host | |
|
||||
| FRONTEND_CSS_URL | To add a external css file to the app | |
|
||||
| FRONTEND_HOMEPAGE_FEATURE_ENABLED | frontend feature flag to display the homepage | false |
|
||||
| FRONTEND_THEME | frontend theme to use | |
|
||||
| POSTHOG_KEY | posthog key for analytics | |
|
||||
| CRISP_WEBSITE_ID | crisp website id for support | |
|
||||
| DJANGO_CELERY_BROKER_URL | celery broker url | redis://redis:6379/0 |
|
||||
| DJANGO_CELERY_BROKER_TRANSPORT_OPTIONS | celery broker transport options | {} |
|
||||
| SESSION_COOKIE_AGE | duration of the cookie session | 60*60*12 |
|
||||
| OIDC_CREATE_USER | create used on OIDC | false |
|
||||
| OIDC_RP_SIGN_ALGO | verification algorithm used OIDC tokens | RS256 |
|
||||
| OIDC_RP_CLIENT_ID | client id used for OIDC | conversations |
|
||||
| OIDC_RP_CLIENT_SECRET | client secret used for OIDC | |
|
||||
| OIDC_OP_JWKS_ENDPOINT | JWKS endpoint for OIDC | |
|
||||
| OIDC_OP_AUTHORIZATION_ENDPOINT | Authorization endpoint for OIDC | |
|
||||
| OIDC_OP_TOKEN_ENDPOINT | Token endpoint for OIDC | |
|
||||
| OIDC_OP_USER_ENDPOINT | User endpoint for OIDC | |
|
||||
| OIDC_OP_LOGOUT_ENDPOINT | Logout endpoint for OIDC | |
|
||||
| OIDC_AUTH_REQUEST_EXTRA_PARAMS | OIDC extra auth parameters | {} |
|
||||
| OIDC_RP_SCOPES | scopes requested for OIDC | openid email |
|
||||
| LOGIN_REDIRECT_URL | login redirect url | |
|
||||
| LOGIN_REDIRECT_URL_FAILURE | login redirect url on failure | |
|
||||
| LOGOUT_REDIRECT_URL | logout redirect url | |
|
||||
| OIDC_USE_NONCE | use nonce for OIDC | true |
|
||||
| OIDC_REDIRECT_REQUIRE_HTTPS | Require https for OIDC redirect url | false |
|
||||
| OIDC_REDIRECT_ALLOWED_HOSTS | Allowed hosts for OIDC redirect url | [] |
|
||||
| OIDC_STORE_ID_TOKEN | Store OIDC token | true |
|
||||
| OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION | faillback to email for identification | true |
|
||||
| OIDC_ALLOW_DUPLICATE_EMAILS | Allow duplicate emails | false |
|
||||
| USER_OIDC_ESSENTIAL_CLAIMS | essential claims in OIDC token | [] |
|
||||
| OIDC_USERINFO_FULLNAME_FIELDS | OIDC token claims to create full name | ["first_name", "last_name"] |
|
||||
| OIDC_USERINFO_SHORTNAME_FIELD | OIDC token claims to create shortname | first_name |
|
||||
| ALLOW_LOGOUT_GET_METHOD | Allow get logout method | true |
|
||||
| AI_API_KEY | AI key to be used for AI Base url | |
|
||||
| AI_BASE_URL | OpenAI compatible AI base url | |
|
||||
| AI_MODEL | AI Model to use | |
|
||||
| AI_AGENT_NAME | Name of the AI agent (useless) | Conversations Assistant |
|
||||
| AI_AGENT_INSTRUCTION | Base instruction for the AI agent | You are a helpful assistant |
|
||||
| Y_PROVIDER_API_KEY | Y provider API key | |
|
||||
| Y_PROVIDER_API_BASE_URL | Y Provider url | |
|
||||
| CONVERSION_API_ENDPOINT | Conversion API endpoint | convert-markdown |
|
||||
| CONVERSION_API_CONTENT_FIELD | Conversion api content field | content |
|
||||
| CONVERSION_API_TIMEOUT | Conversion api timeout | 30 |
|
||||
| CONVERSION_API_SECURE | Require secure conversion api | false |
|
||||
| LOGGING_LEVEL_LOGGERS_ROOT | default logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL" | INFO |
|
||||
| LOGGING_LEVEL_LOGGERS_APP | application logging level. options are "DEBUG", "INFO", "WARN", "ERROR", "CRITICAL" | INFO |
|
||||
| API_USERS_LIST_LIMIT | Limit on API users | 5 |
|
||||
| DJANGO_CSRF_TRUSTED_ORIGINS | CSRF trusted origins | [] |
|
||||
| REDIS_URL | cache url | redis://redis:6379/1 |
|
||||
| CACHES_DEFAULT_TIMEOUT | cache default timeout | 30 |
|
||||
| CACHES_KEY_PREFIX | The prefix used to every cache keys. | conversations |
|
||||
| THEME_CUSTOMIZATION_FILE_PATH | full path to the file customizing the theme. An example is provided in src/backend/conversations/configuration/theme/default.json | BASE_DIR/conversations/configuration/theme/default.json |
|
||||
| THEME_CUSTOMIZATION_CACHE_TIMEOUT | Cache duration for the customization settings | 86400 |
|
||||
|
||||
|
||||
## conversations-frontend image
|
||||
|
||||
These are the environment variables you can set to build the `conversations-frontend` image.
|
||||
|
||||
Depending on how you are building the front-end application, this variable is used in different ways.
|
||||
|
||||
If you want to build the Docker image, this variable is used as an argument in the build command.
|
||||
|
||||
Example:
|
||||
|
||||
```
|
||||
docker build -f src/frontend/Dockerfile --target frontend-production --build-arg API_ORIGIN=https://mybackend.example.com conversations-frontend:latest
|
||||
```
|
||||
|
||||
If you want to build the front-end application using the yarn build command, you can edit the file `src/frontend/apps/conversations/.env` with the `NODE_ENV=production` environment variable and modify it. Alternatively, you can use the listed environment variables with the prefix `NEXT_PUBLIC_`.
|
||||
|
||||
Example:
|
||||
|
||||
```
|
||||
cd src/frontend/apps/conversations
|
||||
NODE_ENV=production NEXT_PUBLIC_API_ORIGIN=https://mybackend.example.com yarn build
|
||||
```
|
||||
|
||||
| Option | Description | default |
|
||||
|-------------------------------------------------|------------------------------------------------------------------------------------| ------------------------------------------------------- |
|
||||
| API_ORIGIN | backend domain - it uses the current domain if not initialized | |
|
||||
| PRODUCT_NAME | to change the default product name displayed in frontend | Conversations |
|
||||
@@ -0,0 +1,134 @@
|
||||
image:
|
||||
repository: lasuite/conversations-backend
|
||||
pullPolicy: Always
|
||||
tag: "latest"
|
||||
|
||||
backend:
|
||||
replicas: 1
|
||||
envVars:
|
||||
DJANGO_CSRF_TRUSTED_ORIGINS: https://conversations.127.0.0.1.nip.io
|
||||
DJANGO_CONFIGURATION: Feature
|
||||
DJANGO_ALLOWED_HOSTS: conversations.127.0.0.1.nip.io
|
||||
DJANGO_SERVER_TO_SERVER_API_TOKENS: secret-api-key
|
||||
DJANGO_SECRET_KEY: AgoodOrAbadKey
|
||||
DJANGO_SETTINGS_MODULE: conversations.settings
|
||||
DJANGO_SUPERUSER_PASSWORD: admin
|
||||
DJANGO_EMAIL_BRAND_NAME: "La Suite Numérique"
|
||||
DJANGO_EMAIL_HOST: "maildev"
|
||||
DJANGO_EMAIL_LOGO_IMG: https://conversations.127.0.0.1.nip.io/assets/logo-suite-numerique.png
|
||||
DJANGO_EMAIL_PORT: 1025
|
||||
DJANGO_EMAIL_USE_SSL: False
|
||||
LOGGING_LEVEL_HANDLERS_CONSOLE: ERROR
|
||||
LOGGING_LEVEL_LOGGERS_ROOT: INFO
|
||||
LOGGING_LEVEL_LOGGERS_APP: INFO
|
||||
OIDC_OP_JWKS_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/conversations/protocol/openid-connect/certs
|
||||
OIDC_OP_AUTHORIZATION_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/conversations/protocol/openid-connect/auth
|
||||
OIDC_OP_TOKEN_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/conversations/protocol/openid-connect/token
|
||||
OIDC_OP_USER_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/conversations/protocol/openid-connect/userinfo
|
||||
OIDC_OP_LOGOUT_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/conversations/protocol/openid-connect/session/end
|
||||
OIDC_RP_CLIENT_ID: conversations
|
||||
OIDC_RP_CLIENT_SECRET: ThisIsAnExampleKeyForDevPurposeOnly
|
||||
OIDC_RP_SIGN_ALGO: RS256
|
||||
OIDC_RP_SCOPES: "openid email"
|
||||
OIDC_VERIFY_SSL: False
|
||||
OIDC_USERINFO_SHORTNAME_FIELD: "given_name"
|
||||
OIDC_USERINFO_FULLNAME_FIELDS: "given_name,usual_name"
|
||||
OIDC_REDIRECT_ALLOWED_HOSTS: https://conversations.127.0.0.1.nip.io
|
||||
OIDC_AUTH_REQUEST_EXTRA_PARAMS: "{'acr_values': 'eidas1'}"
|
||||
LOGIN_REDIRECT_URL: https://conversations.127.0.0.1.nip.io
|
||||
LOGIN_REDIRECT_URL_FAILURE: https://conversations.127.0.0.1.nip.io
|
||||
LOGOUT_REDIRECT_URL: https://conversations.127.0.0.1.nip.io
|
||||
POSTHOG_KEY: "{'id': 'posthog_key', 'host': 'https://product.conversations.127.0.0.1.nip.io'}"
|
||||
DB_HOST: postgresql
|
||||
DB_NAME: conversations
|
||||
DB_USER: dinum
|
||||
DB_PASSWORD: pass
|
||||
DB_PORT: 5432
|
||||
POSTGRES_DB: conversations
|
||||
POSTGRES_USER: dinum
|
||||
POSTGRES_PASSWORD: pass
|
||||
REDIS_URL: redis://default:pass@redis-master:6379/1
|
||||
AWS_S3_ENDPOINT_URL: http://minio.conversations.svc.cluster.local:9000
|
||||
AWS_S3_ACCESS_KEY_ID: root
|
||||
AWS_S3_SECRET_ACCESS_KEY: password
|
||||
AWS_STORAGE_BUCKET_NAME: conversations-media-storage
|
||||
STORAGES_STATICFILES_BACKEND: django.contrib.staticfiles.storage.StaticFilesStorage
|
||||
|
||||
migrate:
|
||||
command:
|
||||
- "/bin/sh"
|
||||
- "-c"
|
||||
- |
|
||||
python manage.py migrate --no-input &&
|
||||
python manage.py create_demo --force
|
||||
restartPolicy: Never
|
||||
|
||||
command:
|
||||
- "gunicorn"
|
||||
- "-c"
|
||||
- "/usr/local/etc/gunicorn/conversations.py"
|
||||
- "conversations.wsgi:application"
|
||||
- "--reload"
|
||||
|
||||
createsuperuser:
|
||||
command:
|
||||
- "/bin/sh"
|
||||
- "-c"
|
||||
- |
|
||||
python manage.py createsuperuser --email admin@example.com --password admin
|
||||
restartPolicy: Never
|
||||
|
||||
# Extra volume to manage our local custom CA and avoid to set ssl_verify: false
|
||||
extraVolumeMounts:
|
||||
- name: certs
|
||||
mountPath: /usr/local/lib/python3.12/site-packages/certifi/cacert.pem
|
||||
subPath: cacert.pem
|
||||
|
||||
# Extra volume to manage our local custom CA and avoid to set ssl_verify: false
|
||||
extraVolumes:
|
||||
- name: certs
|
||||
configMap:
|
||||
name: certifi
|
||||
items:
|
||||
- key: cacert.pem
|
||||
path: cacert.pem
|
||||
frontend:
|
||||
envVars:
|
||||
PORT: 8080
|
||||
NEXT_PUBLIC_API_ORIGIN: https://conversations.127.0.0.1.nip.io
|
||||
|
||||
replicas: 1
|
||||
|
||||
image:
|
||||
repository: lasuite/conversations-frontend
|
||||
pullPolicy: Always
|
||||
tag: "latest"
|
||||
|
||||
posthog:
|
||||
ingress:
|
||||
enabled: false
|
||||
ingressAssets:
|
||||
enabled: false
|
||||
|
||||
ingress:
|
||||
enabled: true
|
||||
host: conversations.127.0.0.1.nip.io
|
||||
|
||||
ingressAdmin:
|
||||
enabled: true
|
||||
host: conversations.127.0.0.1.nip.io
|
||||
|
||||
ingressMedia:
|
||||
enabled: true
|
||||
host: conversations.127.0.0.1.nip.io
|
||||
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/auth-url: https://conversations.127.0.0.1.nip.io/api/v1.0/documents/media-auth/
|
||||
nginx.ingress.kubernetes.io/auth-response-headers: "Authorization, X-Amz-Date, X-Amz-Content-SHA256"
|
||||
nginx.ingress.kubernetes.io/upstream-vhost: minio.conversations.svc.cluster.local:9000
|
||||
nginx.ingress.kubernetes.io/rewrite-target: /conversations-media-storage/$1
|
||||
|
||||
serviceMedia:
|
||||
host: minio.conversations.svc.cluster.local
|
||||
port: 9000
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,8 @@
|
||||
auth:
|
||||
rootUser: root
|
||||
rootPassword: password
|
||||
provisioning:
|
||||
enabled: true
|
||||
buckets:
|
||||
- name: conversations-media-storage
|
||||
versioning: true
|
||||
@@ -0,0 +1,7 @@
|
||||
auth:
|
||||
username: dinum
|
||||
password: pass
|
||||
database: conversations
|
||||
tls:
|
||||
enabled: true
|
||||
autoGenerated: true
|
||||
@@ -0,0 +1,4 @@
|
||||
auth:
|
||||
password: pass
|
||||
architecture: standalone
|
||||
|
||||
@@ -0,0 +1,228 @@
|
||||
# Installation on a k8s cluster
|
||||
|
||||
This document is a step-by-step guide that describes how to install Conversations on a k8s cluster without AI features. It's a teaching document to learn how it works. It needs to be adapted for a production environment.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- k8s cluster with an nginx-ingress controller
|
||||
- an OIDC provider (if you don't have one, we provide an example)
|
||||
- a PostgreSQL server (if you don't have one, we provide an example)
|
||||
- a Memcached server (if you don't have one, we provide an example)
|
||||
- a S3 bucket (if you don't have one, we provide an example)
|
||||
|
||||
### Test cluster
|
||||
|
||||
If you do not have a test cluster, you can install everything on a local Kind cluster. In this case, the simplest way is to use our script **bin/start-kind.sh**.
|
||||
|
||||
To be able to use the script, you need to install:
|
||||
|
||||
- Docker (https://docs.docker.com/desktop/)
|
||||
- Kind (https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
|
||||
- Mkcert (https://github.com/FiloSottile/mkcert#installation)
|
||||
- Helm (https://helm.sh/docs/intro/quickstart/#install-helm)
|
||||
|
||||
```
|
||||
./bin/start-kind.sh
|
||||
% Total % Received % Xferd Average Speed Time Time Time Current
|
||||
Dload Upload Total Spent Left Speed
|
||||
100 4700 100 4700 0 0 92867 0 --:--:-- --:--:-- --:--:-- 94000
|
||||
0. Create ca
|
||||
The local CA is already installed in the system trust store! 👍
|
||||
The local CA is already installed in the Firefox and/or Chrome/Chromium trust store! 👍
|
||||
|
||||
|
||||
Created a new certificate valid for the following names 📜
|
||||
- "127.0.0.1.nip.io"
|
||||
- "*.127.0.0.1.nip.io"
|
||||
|
||||
Reminder: X.509 wildcards only go one level deep, so this won't match a.b.127.0.0.1.nip.io ℹ️
|
||||
|
||||
The certificate is at "./127.0.0.1.nip.io+1.pem" and the key at "./127.0.0.1.nip.io+1-key.pem" ✅
|
||||
|
||||
It will expire on 24 March 2027 🗓
|
||||
|
||||
1. Create registry container unless it already exists
|
||||
2. Create kind cluster with containerd registry config dir enabled
|
||||
Creating cluster "suite" ...
|
||||
✓ Ensuring node image (kindest/node:v1.27.3) 🖼
|
||||
✓ Preparing nodes 📦
|
||||
✓ Writing configuration 📜
|
||||
✓ Starting control-plane 🕹️
|
||||
✓ Installing CNI 🔌
|
||||
✓ Installing StorageClass 💾
|
||||
Set kubectl context to "kind-suite"
|
||||
You can now use your cluster with:
|
||||
|
||||
kubectl cluster-info --context kind-suite
|
||||
|
||||
Thanks for using kind! 😊
|
||||
3. Add the registry config to the nodes
|
||||
4. Connect the registry to the cluster network if not already connected
|
||||
5. Document the local registry
|
||||
configmap/local-registry-hosting created
|
||||
Warning: resource configmaps/coredns is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
|
||||
configmap/coredns configured
|
||||
deployment.apps/coredns restarted
|
||||
6. Install ingress-nginx
|
||||
namespace/ingress-nginx created
|
||||
serviceaccount/ingress-nginx created
|
||||
serviceaccount/ingress-nginx-admission created
|
||||
role.rbac.authorization.k8s.io/ingress-nginx created
|
||||
role.rbac.authorization.k8s.io/ingress-nginx-admission created
|
||||
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
|
||||
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
|
||||
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
|
||||
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
|
||||
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
|
||||
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
|
||||
configmap/ingress-nginx-controller created
|
||||
service/ingress-nginx-controller created
|
||||
service/ingress-nginx-controller-admission created
|
||||
deployment.apps/ingress-nginx-controller created
|
||||
job.batch/ingress-nginx-admission-create created
|
||||
job.batch/ingress-nginx-admission-patch created
|
||||
ingressclass.networking.k8s.io/nginx created
|
||||
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
|
||||
secret/mkcert created
|
||||
deployment.apps/ingress-nginx-controller patched
|
||||
7. Setup namespace
|
||||
namespace/conversations created
|
||||
Context "kind-suite" modified.
|
||||
secret/mkcert created
|
||||
$ kubectl -n ingress-nginx get po
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
ingress-nginx-admission-create-t55ph 0/1 Completed 0 2m56s
|
||||
ingress-nginx-admission-patch-94dvt 0/1 Completed 1 2m56s
|
||||
ingress-nginx-controller-57c548c4cd-2rx47 1/1 Running 0 2m56s
|
||||
```
|
||||
|
||||
When your k8s cluster is ready (the ingress nginx controller is up), you can start the deployment. This cluster is special because it uses the `*.127.0.0.1.nip.io` domain and mkcert certificates to have full HTTPS support and easy domain name management.
|
||||
|
||||
Please remember that `*.127.0.0.1.nip.io` will always resolve to `127.0.0.1`, except in the k8s cluster where we configure CoreDNS to answer with the ingress-nginx service IP.
|
||||
|
||||
## Preparation
|
||||
|
||||
### What do you use to authenticate your users?
|
||||
|
||||
Conversations uses OIDC, so if you already have an OIDC provider, obtain the necessary information to use it. In the next step, we will see how to configure Django (and thus Conversations) to use it. If you do not have a provider, we will show you how to deploy a local Keycloak instance (this is not a production deployment, just a demo).
|
||||
|
||||
```
|
||||
$ kubectl create namespace conversations
|
||||
$ kubectl config set-context --current --namespace=conversations
|
||||
$ helm install keycloak oci://registry-1.docker.io/bitnamicharts/keycloak -f examples/keycloak.values.yaml
|
||||
$ #wait until
|
||||
$ kubectl get po
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
keycloak-0 1/1 Running 0 6m48s
|
||||
keycloak-postgresql-0 1/1 Running 0 6m48s
|
||||
```
|
||||
|
||||
From here the important information you will need are:
|
||||
|
||||
```yaml
|
||||
OIDC_OP_JWKS_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/conversations/protocol/openid-connect/certs
|
||||
OIDC_OP_AUTHORIZATION_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/conversations/protocol/openid-connect/auth
|
||||
OIDC_OP_TOKEN_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/conversations/protocol/openid-connect/token
|
||||
OIDC_OP_USER_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/conversations/protocol/openid-connect/userinfo
|
||||
OIDC_OP_LOGOUT_ENDPOINT: https://keycloak.127.0.0.1.nip.io/realms/conversations/protocol/openid-connect/session/end
|
||||
OIDC_RP_CLIENT_ID: conversations
|
||||
OIDC_RP_CLIENT_SECRET: ThisIsAnExampleKeyForDevPurposeOnly
|
||||
OIDC_RP_SIGN_ALGO: RS256
|
||||
OIDC_RP_SCOPES: "openid email"
|
||||
```
|
||||
|
||||
You can find these values in **examples/keycloak.values.yaml**
|
||||
|
||||
### Find redis server connection values
|
||||
|
||||
Conversations needs a redis so we start by deploying one:
|
||||
|
||||
```
|
||||
$ helm install redis oci://registry-1.docker.io/bitnamicharts/redis -f examples/redis.values.yaml
|
||||
$ kubectl get po
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
keycloak-0 1/1 Running 0 26m
|
||||
keycloak-postgresql-0 1/1 Running 0 26m
|
||||
redis-master-0 1/1 Running 0 35s
|
||||
```
|
||||
|
||||
### Find postgresql connection values
|
||||
|
||||
Conversations uses a postgresql database as backend, so if you have a provider, obtain the necessary information to use it. If you don't, you can install a postgresql testing environment as follow:
|
||||
|
||||
```
|
||||
$ helm install postgresql oci://registry-1.docker.io/bitnamicharts/postgresql -f examples/postgresql.values.yaml
|
||||
$ kubectl get po
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
keycloak-0 1/1 Running 0 28m
|
||||
keycloak-postgresql-0 1/1 Running 0 28m
|
||||
postgresql-0 1/1 Running 0 14m
|
||||
redis-master-0 1/1 Running 0 42s
|
||||
```
|
||||
|
||||
From here the important information you will need are:
|
||||
|
||||
```yaml
|
||||
DB_HOST: postgres-postgresql
|
||||
DB_NAME: conversations
|
||||
DB_USER: dinum
|
||||
DB_PASSWORD: pass
|
||||
DB_PORT: 5432
|
||||
POSTGRES_DB: conversations
|
||||
POSTGRES_USER: dinum
|
||||
POSTGRES_PASSWORD: pass
|
||||
```
|
||||
|
||||
### Find s3 bucket connection values
|
||||
|
||||
Conversations uses an s3 bucket to store documents, so if you have a provider obtain the necessary information to use it. If you don't, you can install a local minio testing environment as follow:
|
||||
|
||||
```
|
||||
$ helm install minio oci://registry-1.docker.io/bitnamicharts/minio -f examples/minio.values.yaml
|
||||
$ kubectl get po
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
keycloak-0 1/1 Running 0 38m
|
||||
keycloak-postgresql-0 1/1 Running 0 38m
|
||||
minio-84f5c66895-bbhsk 1/1 Running 0 42s
|
||||
minio-provisioning-2b5sq 0/1 Completed 0 42s
|
||||
postgresql-0 1/1 Running 0 24m
|
||||
redis-master-0 1/1 Running 0 10m
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
Now you are ready to deploy Conversations without AI. AI requires more dependencies (OpenAI API). To deploy Conversations you need to provide all previous information to the helm chart.
|
||||
|
||||
```
|
||||
$ helm repo add conversations https://suitenumerique.github.io/conversations/
|
||||
$ helm repo update
|
||||
$ helm install conversations conversations/conversations -f examples/conversations.values.yaml
|
||||
$ kubectl get po
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
conversations-conversations-backend-96558758d-xtkbp 0/1 Running 0 79s
|
||||
conversations-conversations-backend-createsuperuser-r7ltc 0/1 Completed 0 79s
|
||||
conversations-conversations-backend-migrate-c949s 0/1 Completed 0 79s
|
||||
conversations-conversations-frontend-6749f644f7-p5s42 1/1 Running 0 79s
|
||||
keycloak-0 1/1 Running 0 48m
|
||||
keycloak-postgresql-0 1/1 Running 0 48m
|
||||
minio-84f5c66895-bbhsk 1/1 Running 0 10m
|
||||
minio-provisioning-2b5sq 0/1 Completed 0 10m
|
||||
postgresql-0 1/1 Running 0 34m
|
||||
redis-master-0 1/1 Running 0 20m
|
||||
```
|
||||
|
||||
## Test your deployment
|
||||
|
||||
In order to test your deployment you have to log into your instance. If you exclusively use our examples you can do:
|
||||
|
||||
```
|
||||
$ kubectl get ingress
|
||||
NAME CLASS HOSTS ADDRESS PORTS AGE
|
||||
conversations-conversations <none> conversations.127.0.0.1.nip.io localhost 80, 443 114s
|
||||
conversations-conversations-admin <none> conversations.127.0.0.1.nip.io localhost 80, 443 114s
|
||||
conversations-conversations-media <none> conversations.127.0.0.1.nip.io localhost 80, 443 114s
|
||||
conversations-conversations-ws <none> conversations.127.0.0.1.nip.io localhost 80, 443 114s
|
||||
keycloak <none> keycloak.127.0.0.1.nip.io localhost 80 49m
|
||||
```
|
||||
|
||||
You can use Conversations at https://conversations.127.0.0.1.nip.io. The provisionning user in keycloak is conversations/conversations.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Releasing a new version
|
||||
|
||||
Whenever we are cooking a new release (e.g. `4.18.1`) we should follow a standard procedure described below:
|
||||
|
||||
1. Create a new branch named: `release/4.18.1`.
|
||||
2. Bump the release number for backend project, frontend projects, and Helm files:
|
||||
|
||||
- for backend, update the version number by hand in `pyproject.toml`,
|
||||
- for each projects (`src/frontend`, `src/frontend/apps/*`, `src/frontend/packages/*`, `src/mail`), run `yarn version --new-version --no-git-tag-version 4.18.1` in their directory. This will update their `package.json` for you,
|
||||
- for Helm, update Docker image tag in files located at `src/helm/env.d` for both `preprod` and `production` environments:
|
||||
|
||||
```yaml
|
||||
image:
|
||||
repository: lasuite/conversations-backend
|
||||
pullPolicy: Always
|
||||
tag: "v4.18.1" # Replace with your new version number, without forgetting the "v" prefix
|
||||
|
||||
...
|
||||
|
||||
frontend:
|
||||
image:
|
||||
repository: lasuite/conversations-frontend
|
||||
pullPolicy: Always
|
||||
tag: "v4.18.1"
|
||||
```
|
||||
|
||||
The new images don't exist _yet_: they will be created automatically later in the process.
|
||||
|
||||
3. Update the project's `Changelog` following the [keepachangelog](https://keepachangelog.com/en/0.3.0/) recommendations
|
||||
|
||||
4. Commit your changes with the following format: the 🔖 release emoji, the type of release (patch/minor/patch) and the release version:
|
||||
|
||||
```text
|
||||
🔖(minor) bump release to 4.18.0
|
||||
```
|
||||
|
||||
5. Open a pull request, wait for an approval from your peers and merge it.
|
||||
6. Checkout and pull changes from the `main` branch to ensure you have the latest updates.
|
||||
7. Tag and push your commit:
|
||||
|
||||
```bash
|
||||
git tag v4.18.1 && git push origin tag v4.18.1
|
||||
```
|
||||
|
||||
Doing this triggers the CI and tells it to build the new Docker image versions that you targeted earlier in the Helm files.
|
||||
|
||||
8. Ensure the new [backend](https://hub.docker.com/r/lasuite/conversations-frontend/tags) and [frontend](https://hub.docker.com/r/lasuite/conversations-frontend/tags) image tags are on Docker Hub.
|
||||
9. The release is now done!
|
||||
|
||||
# Deploying
|
||||
|
||||
> [!TIP]
|
||||
> The `staging` platform is deployed automatically with every update of the `main` branch.
|
||||
|
||||
Making a new release doesn't publish it automatically in production.
|
||||
|
||||
Deployment is done by ArgoCD. ArgoCD checks for the `production` tag and automatically deploys the production platform with the targeted commit.
|
||||
|
||||
To publish, we mark the commit we want with the `production` tag. ArgoCD is then notified that the tag has changed. It then deploys the Docker image tags specified in the Helm files of the targeted commit.
|
||||
|
||||
To publish the release you just made:
|
||||
|
||||
```bash
|
||||
git tag --force production v4.18.1
|
||||
git push --force origin production
|
||||
```
|
||||
@@ -0,0 +1,109 @@
|
||||
# La Suite Docs – System & Requirements (2025-06)
|
||||
|
||||
## 1. Quick-Reference Matrix (single VM / laptop)
|
||||
|
||||
| Scenario | RAM | vCPU | SSD | Notes |
|
||||
| ------------------------- | ----- | ---- | ------- | ------------------------- |
|
||||
| **Solo dev** | 8 GB | 4 | 15 GB | Hot-reload + one IDE |
|
||||
| **Team QA** | 16 GB | 6 | 30 GB | Runs integration tests |
|
||||
| **Prod ≤ 100 live users** | 32 GB | 8 + | 50 GB + | Scale linearly above this |
|
||||
|
||||
Memory is the first bottleneck; CPU matters only when Celery or the Next.js build is saturated.
|
||||
|
||||
> **Note:** Memory consumption varies by operating system. Windows tends to be more memory-hungry than Linux, so consider adding 10-20% extra RAM when running on Windows compared to Linux-based systems.
|
||||
|
||||
## 2. Development Environment Memory Requirements
|
||||
|
||||
| Service | Typical use | Rationale / source |
|
||||
| ------------------------ | ----------------------------- | --------------------------------------------------------------------------------------- |
|
||||
| PostgreSQL | **1 – 2 GB** | `shared_buffers` starting point ≈ 25% RAM ([postgresql.org][1]) |
|
||||
| Keycloak | **≈ 1.3 GB** | 70% of limit for heap + ~300 MB non-heap ([keycloak.org][2]) |
|
||||
| Redis | **≤ 256 MB** | Empty instance ≈ 3 MB; budget 256 MB to allow small datasets ([stackoverflow.com][3]) |
|
||||
| MinIO | **2 GB (dev) / 32 GB (prod)**| Pre-allocates 1–2 GiB; docs recommend 32 GB per host for ≤ 100 Ti storage ([min.io][4]) |
|
||||
| Django API (+ Celery) | **0.8 – 1.5 GB** | Empirical in-house metrics |
|
||||
| Next.js frontend | **0.5 – 1 GB** | Dev build chain |
|
||||
| Y-Provider (y-websocket) | **< 200 MB** | Large 40 MB YDoc called “big” in community thread ([discuss.yjs.dev][5]) |
|
||||
| Nginx | **< 100 MB** | Static reverse-proxy footprint |
|
||||
|
||||
[1]: https://www.postgresql.org/docs/9.1/runtime-config-resource.html "PostgreSQL: Documentation: 9.1: Resource Consumption"
|
||||
[2]: https://www.keycloak.org/high-availability/concepts-memory-and-cpu-sizing "Concepts for sizing CPU and memory resources - Keycloak"
|
||||
[3]: https://stackoverflow.com/questions/45233052/memory-footprint-for-redis-empty-instance "Memory footprint for Redis empty instance - Stack Overflow"
|
||||
[4]: https://min.io/docs/minio/kubernetes/upstream/operations/checklists/hardware.html "Hardware Checklist — MinIO Object Storage for Kubernetes"
|
||||
[5]: https://discuss.yjs.dev/t/understanding-memory-requirements-for-production-usage/198 "Understanding memory requirements for production usage - Yjs Community"
|
||||
|
||||
> **Rule of thumb:** add 2 GB for OS/overhead, then sum only the rows you actually run.
|
||||
|
||||
## 3. Production Environment Memory Requirements
|
||||
|
||||
Production deployments differ significantly from development environments. The table below shows typical memory usage for production services:
|
||||
|
||||
| Service | Typical use | Rationale / notes |
|
||||
| ------------------------ | ----------------------------- | --------------------------------------------------------------------------------------- |
|
||||
| PostgreSQL | **2 – 8 GB** | Higher `shared_buffers` and connection pooling for concurrent users |
|
||||
| OIDC Provider (optional) | **Variable** | Any OIDC-compatible provider (Keycloak, Auth0, Azure AD, etc.) - external or self-hosted |
|
||||
| Redis | **256 MB – 2 GB** | Session storage and caching; scales with active user sessions |
|
||||
| Object Storage (optional)| **External or self-hosted** | Can use AWS S3, Azure Blob, Google Cloud Storage, or self-hosted MinIO |
|
||||
| Django API (+ Celery) | **1 – 3 GB** | Production workloads with background tasks and higher concurrency |
|
||||
| Static Files (Nginx) | **< 200 MB** | Serves Next.js build output and static assets; no development overhead |
|
||||
| Y-Provider (y-websocket) | **200 MB – 1 GB** | Scales with concurrent document editing sessions |
|
||||
| Nginx (Load Balancer) | **< 200 MB** | Reverse proxy, SSL termination, static file serving |
|
||||
|
||||
### Production Architecture Notes
|
||||
|
||||
- **Frontend**: Uses pre-built Next.js static assets served by Nginx (no Node.js runtime needed)
|
||||
- **Authentication**: Any OIDC-compatible provider can be used instead of self-hosted Keycloak
|
||||
- **Object Storage**: External services (S3, Azure Blob) or self-hosted solutions (MinIO) are both viable
|
||||
- **Database**: Consider PostgreSQL clustering or managed database services for high availability
|
||||
- **Scaling**: Horizontal scaling is recommended for Django API and Y-Provider services
|
||||
|
||||
### Minimal Production Setup (Core Services Only)
|
||||
|
||||
| Service | Memory | Notes |
|
||||
|----------------------------------|------------|----------------------------------------|
|
||||
| PostgreSQL | **2 GB** | Core database |
|
||||
| Django API (+ Celery) | **1.5 GB** | Backend services |
|
||||
| Nginx | **100 MB** | Static files + reverse proxy |
|
||||
| Redis | **256 MB** | Session storage |
|
||||
| **Total (without auth/storage)** | **≈ 4 GB** | External OIDC + object storage assumed |
|
||||
|
||||
## 4. Recommended Software Versions
|
||||
|
||||
| Tool | Minimum |
|
||||
| ----------------------- | ------- |
|
||||
| Docker Engine / Desktop | 24.0 |
|
||||
| Docker Compose | v2 |
|
||||
| Git | 2.40 |
|
||||
| **Node.js** | 22+ |
|
||||
| **Python** | 3.13+ |
|
||||
| GNU Make | 4.4 |
|
||||
| Kind | 0.22 |
|
||||
| Helm | 3.14 |
|
||||
| kubectl | 1.29 |
|
||||
| mkcert | 1.4 |
|
||||
|
||||
|
||||
## 5. Ports (dev defaults)
|
||||
|
||||
| Port | Service |
|
||||
| --------- |-----------------------|
|
||||
| 3000 | Next.js |
|
||||
| 8071 | Django |
|
||||
| 4444 | Y-Provider |
|
||||
| 8080 | Keycloak |
|
||||
| 8083 | Nginx proxy |
|
||||
| 9000/9001 | MinIO |
|
||||
| 15432 | PostgreSQL (main) |
|
||||
| 5433 | PostgreSQL (Keycloak) |
|
||||
| 1081 | Maildev |
|
||||
|
||||
## 6. Sizing Guidelines
|
||||
|
||||
**RAM** – start at 8 GB dev / 16 GB staging / 32 GB prod. Postgres and Keycloak are the first to OOM; scale them first.
|
||||
|
||||
> **OS considerations:** Windows systems typically require 10-20% more RAM than Linux due to higher OS overhead. Docker Desktop on Windows also uses additional memory compared to native Linux Docker.
|
||||
|
||||
**CPU** – budget one vCPU per busy container until Celery or Next.js builds saturate.
|
||||
|
||||
**Disk** – SSD; add 10 GB extra for the Docker layer cache.
|
||||
|
||||
**MinIO** – for demos, mount a local folder instead of running MinIO to save 2 GB+ of RAM.
|
||||
@@ -0,0 +1,64 @@
|
||||
# Runtime Theming 🎨
|
||||
|
||||
### How to Use
|
||||
|
||||
To use this feature, simply set the `FRONTEND_CSS_URL` environment variable to the URL of your custom CSS file. For example:
|
||||
|
||||
```javascript
|
||||
FRONTEND_CSS_URL=http://anything/custom-style.css
|
||||
```
|
||||
|
||||
Once you've set this variable, our application will load your custom CSS file and apply the styles to our frontend application.
|
||||
|
||||
### Benefits
|
||||
|
||||
This feature provides several benefits, including:
|
||||
|
||||
* **Easy customization** 🔄: With this feature, you can easily customize the look and feel of our application without requiring any code changes.
|
||||
* **Flexibility** 🌈: You can use any CSS styles you like to create a custom theme that meets your needs.
|
||||
* **Runtime theming** ⏱️: This feature allows you to change the theme of our application at runtime, without requiring a restart or recompilation.
|
||||
|
||||
### Example Use Case
|
||||
|
||||
Let's say you want to change the background color of our application to a custom color. You can create a custom CSS file with the following contents:
|
||||
|
||||
```css
|
||||
body {
|
||||
background-color: #3498db;
|
||||
}
|
||||
```
|
||||
|
||||
Then, set the `FRONTEND_CSS_URL` environment variable to the URL of your custom CSS file. Once you've done this, our application will load your custom CSS file and apply the styles, changing the background color to the custom color you specified.
|
||||
|
||||
----
|
||||
|
||||
# **Footer Configuration** 📝
|
||||
|
||||
The footer is configurable from the theme customization file.
|
||||
|
||||
### Settings 🔧
|
||||
|
||||
```shellscript
|
||||
THEME_CUSTOMIZATION_FILE_PATH=<path>
|
||||
```
|
||||
|
||||
### Example of JSON
|
||||
|
||||
The json must follow some rules: https://github.com/suitenumerique/conversations/blob/main/src/helm/env.d/dev/configuration/theme/demo.json
|
||||
|
||||
`footer.default` is the fallback if the language is not supported.
|
||||
|
||||
|
||||
# **Custom Translations** 📝
|
||||
|
||||
The translations can be partially overridden from the theme customization file.
|
||||
|
||||
### Settings 🔧
|
||||
|
||||
```shellscript
|
||||
THEME_CUSTOMIZATION_FILE_PATH=<path>
|
||||
```
|
||||
|
||||
### Example of JSON
|
||||
|
||||
The json must follow some rules: https://github.com/suitenumerique/conversations/blob/main/src/helm/env.d/dev/configuration/theme/demo.json
|
||||
@@ -0,0 +1,192 @@
|
||||
# Troubleshooting Guide
|
||||
|
||||
## Line Ending Issues on Windows (LF/CRLF)
|
||||
|
||||
### Problem Description
|
||||
|
||||
This project uses **LF (Line Feed: `\n`) line endings** exclusively. Windows users may encounter issues because:
|
||||
|
||||
- **Windows** defaults to CRLF (Carriage Return + Line Feed: `\r\n`) for line endings
|
||||
- **This project** uses LF line endings for consistency across all platforms
|
||||
- **Git** may automatically convert line endings, causing conflicts or build failures
|
||||
|
||||
### Common Symptoms
|
||||
|
||||
- Git shows files as modified even when no changes were made
|
||||
- Error messages like "warning: LF will be replaced by CRLF"
|
||||
- Build failures or linting errors due to line ending mismatches
|
||||
|
||||
### Solutions for Windows Users
|
||||
|
||||
#### Configure Git to Preserve LF (Recommended)
|
||||
|
||||
Configure Git to NOT convert line endings and preserve LF:
|
||||
|
||||
```bash
|
||||
git config core.autocrlf false
|
||||
git config core.eol lf
|
||||
```
|
||||
|
||||
This tells Git to:
|
||||
- Never convert line endings automatically
|
||||
- Always use LF for line endings in working directory
|
||||
|
||||
|
||||
#### Fix Existing Repository with Wrong Line Endings
|
||||
|
||||
If you already have CRLF line endings in your local repository, the **best approach** is to configure Git properly and clone the project again:
|
||||
|
||||
1. **Configure Git first**:
|
||||
```bash
|
||||
git config --global core.autocrlf false
|
||||
git config --global core.eol lf
|
||||
```
|
||||
|
||||
2. **Clone the project fresh** (recommended):
|
||||
```bash
|
||||
# Navigate to parent directory
|
||||
cd ..
|
||||
|
||||
# Remove current repository (backup your changes first!)
|
||||
rm -rf docs
|
||||
|
||||
# Clone again with correct line endings
|
||||
git clone git@github.com:suitenumerique/conversations.git
|
||||
```
|
||||
|
||||
**Alternative**: If you have uncommitted changes and cannot re-clone:
|
||||
|
||||
1. **Backup your changes**:
|
||||
```bash
|
||||
git add .
|
||||
git commit -m "Save changes before fixing line endings"
|
||||
```
|
||||
|
||||
2. **Remove all files from Git's index**:
|
||||
```bash
|
||||
git rm --cached -r .
|
||||
```
|
||||
|
||||
3. **Reset Git configuration** (if not done globally):
|
||||
```bash
|
||||
git config core.autocrlf false
|
||||
git config core.eol lf
|
||||
```
|
||||
|
||||
4. **Re-add all files** (Git will use LF line endings):
|
||||
```bash
|
||||
git add .
|
||||
```
|
||||
|
||||
5. **Commit the changes**:
|
||||
```bash
|
||||
git commit -m "✏️(project) Fix line endings to LF"
|
||||
```
|
||||
|
||||
## Minio Permission Issues on Windows
|
||||
|
||||
### Problem Description
|
||||
|
||||
On Windows, you may encounter permission-related errors when running Minio in development mode with Docker Compose. This typically happens because:
|
||||
|
||||
- **Windows file permissions** don't map well to Unix-style user IDs used in Docker containers
|
||||
- **Docker Desktop** may have issues with user mapping when using the `DOCKER_USER` environment variable
|
||||
- **Minio container** fails to start or access volumes due to permission conflicts
|
||||
|
||||
### Common Symptoms
|
||||
|
||||
- Minio container fails to start with permission denied errors
|
||||
- Error messages related to file system permissions in Minio logs
|
||||
- Unable to create or access buckets in the development environment
|
||||
- Docker Compose showing Minio service as unhealthy or exited
|
||||
|
||||
### Solution for Windows Users
|
||||
|
||||
If you encounter Minio permission issues on Windows, you can temporarily disable user mapping for the Minio service:
|
||||
|
||||
1. **Open the `compose.yml` file**
|
||||
|
||||
2. **Comment out the user directive** in the `minio` service section:
|
||||
```yaml
|
||||
minio:
|
||||
# user: ${DOCKER_USER:-1000} # Comment this line on Windows if permission issues occur
|
||||
image: minio/minio
|
||||
environment:
|
||||
- MINIO_ROOT_USER=conversations
|
||||
- MINIO_ROOT_PASSWORD=password
|
||||
# ... rest of the configuration
|
||||
```
|
||||
|
||||
3. **Restart the services**:
|
||||
```bash
|
||||
make run
|
||||
```
|
||||
|
||||
### Why This Works
|
||||
|
||||
- Commenting out the `user` directive allows the Minio container to run with its default user
|
||||
- This bypasses Windows-specific permission mapping issues
|
||||
- The container will have the necessary permissions to access and manage the mounted volumes
|
||||
|
||||
### Note
|
||||
|
||||
This is a **development-only workaround**. In production environments, proper user mapping and security considerations should be maintained according to your deployment requirements.
|
||||
|
||||
## Frontend File Watching Issues on Windows
|
||||
|
||||
### Problem Description
|
||||
|
||||
Windows users may experience issues with file watching in the frontend-development container. This typically happens because:
|
||||
|
||||
- **Docker on Windows** has known limitations with file change detection
|
||||
- **Node.js file watchers** may not detect changes properly on Windows filesystem
|
||||
- **Hot reloading** fails to trigger when files are modified
|
||||
|
||||
### Common Symptoms
|
||||
|
||||
- Changes to frontend code aren't detected automatically
|
||||
- Hot module replacement doesn't work as expected
|
||||
- Need to manually restart the frontend container after code changes
|
||||
- Console shows no reaction when saving files
|
||||
|
||||
### Solution: Enable WATCHPACK_POLLING
|
||||
|
||||
Add the `WATCHPACK_POLLING=true` environment variable to the frontend-development service in your local environment:
|
||||
|
||||
1. **Modify the `compose.yml` file** by adding the environment variable to the frontend-development service:
|
||||
|
||||
```yaml
|
||||
frontend-development:
|
||||
user: "${DOCKER_USER:-1000}"
|
||||
build:
|
||||
context: .
|
||||
dockerfile: ./src/frontend/Dockerfile
|
||||
target: conversations-dev
|
||||
args:
|
||||
API_ORIGIN: "http://localhost:8071"
|
||||
image: conversations:frontend-development
|
||||
environment:
|
||||
- WATCHPACK_POLLING=true # Add this line for Windows users
|
||||
volumes:
|
||||
- ./src/frontend:/home/frontend
|
||||
- /home/frontend/node_modules
|
||||
- /home/frontend/apps/conversations/node_modules
|
||||
ports:
|
||||
- "3000:3000"
|
||||
```
|
||||
|
||||
2. **Restart your containers**:
|
||||
```bash
|
||||
make run
|
||||
```
|
||||
|
||||
### Why This Works
|
||||
|
||||
- `WATCHPACK_POLLING=true` forces the file watcher to use polling instead of filesystem events
|
||||
- Polling periodically checks for file changes rather than relying on OS-level file events
|
||||
- This is more reliable on Windows but slightly increases CPU usage
|
||||
- Changes to your frontend code should now be detected properly, enabling hot reloading
|
||||
|
||||
### Note
|
||||
|
||||
This setting is primarily needed for Windows users. Linux and macOS users typically don't need this setting as file watching works correctly by default on those platforms.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Django
|
||||
DJANGO_ALLOWED_HOSTS=*
|
||||
DJANGO_SECRET_KEY=ThisIsAnExampleKeyForDevPurposeOnly
|
||||
DJANGO_SETTINGS_MODULE=conversations.settings
|
||||
DJANGO_SUPERUSER_PASSWORD=admin
|
||||
|
||||
# Logging
|
||||
# Set to DEBUG level for dev only
|
||||
LOGGING_LEVEL_HANDLERS_CONSOLE=INFO
|
||||
LOGGING_LEVEL_LOGGERS_ROOT=INFO
|
||||
LOGGING_LEVEL_LOGGERS_APP=INFO
|
||||
|
||||
# Python
|
||||
PYTHONPATH=/app
|
||||
|
||||
# conversations settings
|
||||
|
||||
# Mail
|
||||
DJANGO_EMAIL_BRAND_NAME="La Suite Numérique"
|
||||
DJANGO_EMAIL_HOST="maildev"
|
||||
DJANGO_EMAIL_LOGO_IMG="http://localhost:3000/assets/logo-suite-numerique.png"
|
||||
DJANGO_EMAIL_PORT=1025
|
||||
|
||||
# Backend url
|
||||
CONVERSATIONS_BASE_URL="http://localhost:8072"
|
||||
|
||||
# Media
|
||||
STORAGES_STATICFILES_BACKEND=django.contrib.staticfiles.storage.StaticFilesStorage
|
||||
AWS_S3_ENDPOINT_URL=http://minio:9000
|
||||
AWS_S3_ACCESS_KEY_ID=conversations
|
||||
AWS_S3_SECRET_ACCESS_KEY=password
|
||||
MEDIA_BASE_URL=http://localhost:8083
|
||||
|
||||
# OIDC
|
||||
OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/conversations/protocol/openid-connect/certs
|
||||
OIDC_OP_AUTHORIZATION_ENDPOINT=http://localhost:8083/realms/conversations/protocol/openid-connect/auth
|
||||
OIDC_OP_TOKEN_ENDPOINT=http://nginx:8083/realms/conversations/protocol/openid-connect/token
|
||||
OIDC_OP_USER_ENDPOINT=http://nginx:8083/realms/conversations/protocol/openid-connect/userinfo
|
||||
|
||||
OIDC_RP_CLIENT_ID=conversations
|
||||
OIDC_RP_CLIENT_SECRET=ThisIsAnExampleKeyForDevPurposeOnly
|
||||
OIDC_RP_SIGN_ALGO=RS256
|
||||
OIDC_RP_SCOPES="openid email"
|
||||
|
||||
LOGIN_REDIRECT_URL=http://localhost:3000
|
||||
LOGIN_REDIRECT_URL_FAILURE=http://localhost:3000
|
||||
LOGOUT_REDIRECT_URL=http://localhost:3000
|
||||
|
||||
OIDC_REDIRECT_ALLOWED_HOSTS=["http://localhost:8083", "http://localhost:3000"]
|
||||
OIDC_AUTH_REQUEST_EXTRA_PARAMS={"acr_values": "eidas1"}
|
||||
|
||||
# AI
|
||||
# AI_BASE_URL=https://openaiendpoint.com
|
||||
AI_API_KEY=password
|
||||
# AI_MODEL=llama
|
||||
@@ -0,0 +1,4 @@
|
||||
# For the CI job test-e2e
|
||||
BURST_THROTTLE_RATES="200/minute"
|
||||
DJANGO_SERVER_TO_SERVER_API_TOKENS=test-e2e
|
||||
SUSTAINED_THROTTLE_RATES="200/hour"
|
||||
@@ -0,0 +1,3 @@
|
||||
CROWDIN_PERSONAL_TOKEN=Your-Personal-Token
|
||||
CROWDIN_PROJECT_ID=Your-Project-Id
|
||||
CROWDIN_BASE_PATH=/app/src
|
||||
@@ -0,0 +1,11 @@
|
||||
# Postgresql db container configuration
|
||||
POSTGRES_DB=keycloak
|
||||
POSTGRES_USER=conversations
|
||||
POSTGRES_PASSWORD=pass
|
||||
|
||||
# App database configuration
|
||||
DB_HOST=kc_postgresql
|
||||
DB_NAME=keycloak
|
||||
DB_USER=conversations
|
||||
DB_PASSWORD=pass
|
||||
DB_PORT=5433
|
||||
@@ -0,0 +1,11 @@
|
||||
# Postgresql db container configuration
|
||||
POSTGRES_DB=conversations
|
||||
POSTGRES_USER=dinum
|
||||
POSTGRES_PASSWORD=pass
|
||||
|
||||
# App database configuration
|
||||
DB_HOST=postgresql
|
||||
DB_NAME=conversations
|
||||
DB_USER=dinum
|
||||
DB_PASSWORD=pass
|
||||
DB_PORT=5432
|
||||
@@ -0,0 +1,37 @@
|
||||
"""
|
||||
Gitlint extra rule to validate that the message title is of the form
|
||||
"<gitmoji>(<scope>) <subject>"
|
||||
"""
|
||||
from __future__ import unicode_literals
|
||||
|
||||
import re
|
||||
|
||||
import requests
|
||||
|
||||
from gitlint.rules import CommitMessageTitle, LineRule, RuleViolation
|
||||
|
||||
|
||||
class GitmojiTitle(LineRule):
|
||||
"""
|
||||
This rule will enforce that each commit title is of the form "<gitmoji>(<scope>) <subject>"
|
||||
where gitmoji is an emoji from the list defined in https://gitmoji.carloscuesta.me and
|
||||
subject should be all lowercase
|
||||
"""
|
||||
|
||||
id = "UC1"
|
||||
name = "title-should-have-gitmoji-and-scope"
|
||||
target = CommitMessageTitle
|
||||
|
||||
def validate(self, title, _commit):
|
||||
"""
|
||||
Download the list possible gitmojis from the project's github repository and check that
|
||||
title contains one of them.
|
||||
"""
|
||||
gitmojis = requests.get(
|
||||
"https://raw.githubusercontent.com/carloscuesta/gitmoji/master/packages/gitmojis/src/gitmojis.json"
|
||||
).json()["gitmojis"]
|
||||
emojis = [item["emoji"] for item in gitmojis]
|
||||
pattern = r"^({:s})\(.*\)\s[a-zA-Z].*$".format("|".join(emojis))
|
||||
if not re.search(pattern, title):
|
||||
violation_msg = 'Title does not match regex "<gitmoji>(<scope>) <subject>"'
|
||||
return [RuleViolation(self.id, violation_msg, title)]
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"dependencies": {
|
||||
"@ai-sdk/react": "^1.2.12",
|
||||
"@ai-sdk/ui-utils": "^1.2.11"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"extends": ["github>numerique-gouv/renovate-configuration"],
|
||||
"dependencyDashboard": true,
|
||||
"labels": ["dependencies", "noChangeLog", "automated"],
|
||||
"packageRules": [
|
||||
{
|
||||
"enabled": false,
|
||||
"groupName": "ignored python dependencies",
|
||||
"matchManagers": ["pep621"],
|
||||
"matchPackageNames": []
|
||||
},
|
||||
{
|
||||
"groupName": "allowed redis versions",
|
||||
"matchManagers": ["pep621"],
|
||||
"matchPackageNames": ["redis"],
|
||||
"allowedVersions": "<6.0.0"
|
||||
},
|
||||
{
|
||||
"enabled": false,
|
||||
"groupName": "ignored js dependencies",
|
||||
"matchManagers": ["npm"],
|
||||
"matchPackageNames": [
|
||||
"eslint",
|
||||
"fetch-mock",
|
||||
"node",
|
||||
"node-fetch",
|
||||
"workbox-webpack-plugin"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,472 @@
|
||||
[MASTER]
|
||||
|
||||
# A comma-separated list of package or module names from where C extensions may
|
||||
# be loaded. Extensions are loading into the active Python interpreter and may
|
||||
# run arbitrary code
|
||||
extension-pkg-whitelist=
|
||||
|
||||
# Add files or directories to the blacklist. They should be base names, not
|
||||
# paths.
|
||||
ignore=migrations
|
||||
|
||||
# Add files or directories matching the regex patterns to the blacklist. The
|
||||
# regex matches against base names, not paths.
|
||||
ignore-patterns=
|
||||
|
||||
# Python code to execute, usually for sys.path manipulation such as
|
||||
# pygtk.require().
|
||||
#init-hook=
|
||||
|
||||
# Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the
|
||||
# number of processors available to use.
|
||||
jobs=0
|
||||
|
||||
# List of plugins (as comma separated values of python modules names) to load,
|
||||
# usually to register additional checkers.
|
||||
load-plugins=pylint_django,pylint.extensions.no_self_use
|
||||
|
||||
# Pickle collected data for later comparisons.
|
||||
persistent=yes
|
||||
|
||||
# Specify a configuration file.
|
||||
#rcfile=
|
||||
|
||||
# When enabled, pylint would attempt to guess common misconfiguration and emit
|
||||
# user-friendly hints instead of false-positive error messages
|
||||
suggestion-mode=yes
|
||||
|
||||
# Allow loading of arbitrary C extensions. Extensions are imported into the
|
||||
# active Python interpreter and may run arbitrary code.
|
||||
unsafe-load-any-extension=no
|
||||
|
||||
|
||||
[MESSAGES CONTROL]
|
||||
|
||||
# Only show warnings with the listed confidence levels. Leave empty to show
|
||||
# all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED
|
||||
confidence=
|
||||
|
||||
# Disable the message, report, category or checker with the given id(s). You
|
||||
# can either give multiple identifiers separated by comma (,) or put this
|
||||
# option multiple times (only on the command line, not in the configuration
|
||||
# file where it should appear only once).You can also use "--disable=all" to
|
||||
# disable everything first and then reenable specific checks. For example, if
|
||||
# you want to run only the similarities checker, you can use "--disable=all
|
||||
# --enable=similarities". If you want to run only the classes checker, but have
|
||||
# no Warning level messages displayed, use"--disable=all --enable=classes
|
||||
# --disable=W"
|
||||
disable=bad-inline-option,
|
||||
deprecated-pragma,
|
||||
django-not-configured,
|
||||
file-ignored,
|
||||
locally-disabled,
|
||||
no-self-use,
|
||||
raw-checker-failed,
|
||||
suppressed-message,
|
||||
useless-suppression
|
||||
|
||||
# Enable the message, report, category or checker with the given id(s). You can
|
||||
# either give multiple identifier separated by comma (,) or put this option
|
||||
# multiple time (only on the command line, not in the configuration file where
|
||||
# it should appear only once). See also the "--disable" option for examples.
|
||||
enable=c-extension-no-member
|
||||
|
||||
|
||||
[REPORTS]
|
||||
|
||||
# Python expression which should return a note less than 10 (10 is the highest
|
||||
# note). You have access to the variables errors warning, statement which
|
||||
# respectively contain the number of errors / warnings messages and the total
|
||||
# number of statements analyzed. This is used by the global evaluation report
|
||||
# (RP0004).
|
||||
evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10)
|
||||
|
||||
# Template used to display messages. This is a python new-style format string
|
||||
# used to format the message information. See doc for all details
|
||||
#msg-template=
|
||||
|
||||
# Set the output format. Available formats are text, parseable, colorized, json
|
||||
# and msvs (visual studio).You can also give a reporter class, eg
|
||||
# mypackage.mymodule.MyReporterClass.
|
||||
output-format=text
|
||||
|
||||
# Tells whether to display a full report or only the messages
|
||||
reports=no
|
||||
|
||||
# Activate the evaluation score.
|
||||
score=yes
|
||||
|
||||
|
||||
[REFACTORING]
|
||||
|
||||
# Maximum number of nested blocks for function / method body
|
||||
max-nested-blocks=5
|
||||
|
||||
# Complete name of functions that never returns. When checking for
|
||||
# inconsistent-return-statements if a never returning function is called then
|
||||
# it will be considered as an explicit return statement and no message will be
|
||||
# printed.
|
||||
never-returning-functions=optparse.Values,sys.exit
|
||||
|
||||
|
||||
[LOGGING]
|
||||
|
||||
# Logging modules to check that the string format arguments are in logging
|
||||
# function parameter format
|
||||
logging-modules=logging
|
||||
|
||||
|
||||
[SPELLING]
|
||||
|
||||
# Limits count of emitted suggestions for spelling mistakes
|
||||
max-spelling-suggestions=4
|
||||
|
||||
# Spelling dictionary name. Available dictionaries: none. To make it working
|
||||
# install python-enchant package.
|
||||
spelling-dict=
|
||||
|
||||
# List of comma separated words that should not be checked.
|
||||
spelling-ignore-words=
|
||||
|
||||
# A path to a file that contains private dictionary; one word per line.
|
||||
spelling-private-dict-file=
|
||||
|
||||
# Tells whether to store unknown words to indicated private dictionary in
|
||||
# --spelling-private-dict-file option instead of raising a message.
|
||||
spelling-store-unknown-words=no
|
||||
|
||||
|
||||
[MISCELLANEOUS]
|
||||
|
||||
# List of note tags to take in consideration, separated by a comma.
|
||||
notes=FIXME,
|
||||
XXX,
|
||||
TODO
|
||||
|
||||
|
||||
[TYPECHECK]
|
||||
|
||||
# List of decorators that produce context managers, such as
|
||||
# contextlib.contextmanager. Add to this list to register other decorators that
|
||||
# produce valid context managers.
|
||||
contextmanager-decorators=contextlib.contextmanager
|
||||
|
||||
# List of members which are set dynamically and missed by pylint inference
|
||||
# system, and so shouldn't trigger E1101 when accessed. Python regular
|
||||
# expressions are accepted.
|
||||
generated-members=
|
||||
|
||||
# Tells whether missing members accessed in mixin class should be ignored. A
|
||||
# mixin class is detected if its name ends with "mixin" (case insensitive).
|
||||
ignore-mixin-members=yes
|
||||
|
||||
# This flag controls whether pylint should warn about no-member and similar
|
||||
# checks whenever an opaque object is returned when inferring. The inference
|
||||
# can return multiple potential results while evaluating a Python object, but
|
||||
# some branches might not be evaluated, which results in partial inference. In
|
||||
# that case, it might be useful to still emit no-member and other checks for
|
||||
# the rest of the inferred objects.
|
||||
ignore-on-opaque-inference=yes
|
||||
|
||||
# List of class names for which member attributes should not be checked (useful
|
||||
# for classes with dynamically set attributes). This supports the use of
|
||||
# qualified names.
|
||||
ignored-classes=optparse.Values,thread._local,_thread._local,responses,
|
||||
Template,Contact
|
||||
|
||||
# List of module names for which member attributes should not be checked
|
||||
# (useful for modules/projects where namespaces are manipulated during runtime
|
||||
# and thus existing member attributes cannot be deduced by static analysis. It
|
||||
# supports qualified module names, as well as Unix pattern matching.
|
||||
ignored-modules=
|
||||
|
||||
# Show a hint with possible names when a member name was not found. The aspect
|
||||
# of finding the hint is based on edit distance.
|
||||
missing-member-hint=yes
|
||||
|
||||
# The minimum edit distance a name should have in order to be considered a
|
||||
# similar match for a missing member name.
|
||||
missing-member-hint-distance=1
|
||||
|
||||
# The total number of similar names that should be taken in consideration when
|
||||
# showing a hint for a missing member.
|
||||
missing-member-max-choices=1
|
||||
|
||||
|
||||
[VARIABLES]
|
||||
|
||||
# List of additional names supposed to be defined in builtins. Remember that
|
||||
# you should avoid to define new builtins when possible.
|
||||
additional-builtins=
|
||||
|
||||
# Tells whether unused global variables should be treated as a violation.
|
||||
allow-global-unused-variables=yes
|
||||
|
||||
# List of strings which can identify a callback function by name. A callback
|
||||
# name must start or end with one of those strings.
|
||||
callbacks=cb_,
|
||||
_cb
|
||||
|
||||
# A regular expression matching the name of dummy variables (i.e. expectedly
|
||||
# not used).
|
||||
dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_
|
||||
|
||||
# Argument names that match this expression will be ignored. Default to name
|
||||
# with leading underscore
|
||||
ignored-argument-names=_.*|^ignored_|^unused_
|
||||
|
||||
# Tells whether we should check for unused import in __init__ files.
|
||||
init-import=no
|
||||
|
||||
# List of qualified module names which can have objects that can redefine
|
||||
# builtins.
|
||||
redefining-builtins-modules=six.moves,past.builtins,future.builtins
|
||||
|
||||
|
||||
[FORMAT]
|
||||
|
||||
# Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
|
||||
expected-line-ending-format=
|
||||
|
||||
# Regexp for a line that is allowed to be longer than the limit.
|
||||
ignore-long-lines=^\s*(# )?<?https?://\S+>?$
|
||||
|
||||
# Number of spaces of indent required inside a hanging or continued line.
|
||||
indent-after-paren=4
|
||||
|
||||
# String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
|
||||
# tab).
|
||||
indent-string=' '
|
||||
|
||||
# Maximum number of characters on a single line.
|
||||
max-line-length=100
|
||||
|
||||
# Maximum number of lines in a module
|
||||
max-module-lines=1000
|
||||
|
||||
# Allow the body of a class to be on the same line as the declaration if body
|
||||
# contains single statement.
|
||||
single-line-class-stmt=no
|
||||
|
||||
# Allow the body of an if to be on the same line as the test if there is no
|
||||
# else.
|
||||
single-line-if-stmt=no
|
||||
|
||||
|
||||
[SIMILARITIES]
|
||||
|
||||
# Ignore comments when computing similarities.
|
||||
ignore-comments=yes
|
||||
|
||||
# Ignore docstrings when computing similarities.
|
||||
ignore-docstrings=yes
|
||||
|
||||
# Ignore imports when computing similarities.
|
||||
ignore-imports=yes
|
||||
|
||||
# Minimum lines number of a similarity.
|
||||
# First implementations of CMS wizards have common fields we do not want to factorize for now
|
||||
min-similarity-lines=35
|
||||
|
||||
|
||||
[BASIC]
|
||||
|
||||
# Naming style matching correct argument names
|
||||
argument-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct argument names. Overrides argument-
|
||||
# naming-style
|
||||
#argument-rgx=
|
||||
|
||||
# Naming style matching correct attribute names
|
||||
attr-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct attribute names. Overrides attr-naming-
|
||||
# style
|
||||
#attr-rgx=
|
||||
|
||||
# Bad variable names which should always be refused, separated by a comma
|
||||
bad-names=foo,
|
||||
bar,
|
||||
baz,
|
||||
toto,
|
||||
tutu,
|
||||
tata
|
||||
|
||||
# Naming style matching correct class attribute names
|
||||
class-attribute-naming-style=any
|
||||
|
||||
# Regular expression matching correct class attribute names. Overrides class-
|
||||
# attribute-naming-style
|
||||
#class-attribute-rgx=
|
||||
|
||||
# Naming style matching correct class names
|
||||
class-naming-style=PascalCase
|
||||
|
||||
# Regular expression matching correct class names. Overrides class-naming-style
|
||||
#class-rgx=
|
||||
|
||||
# Naming style matching correct constant names
|
||||
const-naming-style=UPPER_CASE
|
||||
|
||||
# Regular expression matching correct constant names. Overrides const-naming-
|
||||
# style
|
||||
const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__)|urlpatterns|logger)$
|
||||
|
||||
# Minimum line length for functions/classes that require docstrings, shorter
|
||||
# ones are exempt.
|
||||
docstring-min-length=-1
|
||||
|
||||
# Naming style matching correct function names
|
||||
function-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct function names. Overrides function-
|
||||
# naming-style
|
||||
#function-rgx=
|
||||
|
||||
# Good variable names which should always be accepted, separated by a comma
|
||||
good-names=i,
|
||||
j,
|
||||
k,
|
||||
cm,
|
||||
ex,
|
||||
Run,
|
||||
_
|
||||
|
||||
# Include a hint for the correct naming format with invalid-name
|
||||
include-naming-hint=no
|
||||
|
||||
# Naming style matching correct inline iteration names
|
||||
inlinevar-naming-style=any
|
||||
|
||||
# Regular expression matching correct inline iteration names. Overrides
|
||||
# inlinevar-naming-style
|
||||
#inlinevar-rgx=
|
||||
|
||||
# Naming style matching correct method names
|
||||
method-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct method names. Overrides method-naming-
|
||||
# style
|
||||
method-rgx=([a-z_][a-z0-9_]{2,50}|setUp|set[Uu]pClass|tearDown|tear[Dd]ownClass|assert[A-Z]\w*|maxDiff|test_[a-z0-9_]+)$
|
||||
|
||||
# Naming style matching correct module names
|
||||
module-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct module names. Overrides module-naming-
|
||||
# style
|
||||
#module-rgx=
|
||||
|
||||
# Colon-delimited sets of names that determine each other's naming style when
|
||||
# the name regexes allow several styles.
|
||||
name-group=
|
||||
|
||||
# Regular expression which should only match function or class names that do
|
||||
# not require a docstring.
|
||||
no-docstring-rgx=^_
|
||||
|
||||
# List of decorators that produce properties, such as abc.abstractproperty. Add
|
||||
# to this list to register other decorators that produce valid properties.
|
||||
property-classes=abc.abstractproperty
|
||||
|
||||
# Naming style matching correct variable names
|
||||
variable-naming-style=snake_case
|
||||
|
||||
# Regular expression matching correct variable names. Overrides variable-
|
||||
# naming-style
|
||||
#variable-rgx=
|
||||
|
||||
|
||||
[IMPORTS]
|
||||
|
||||
# Allow wildcard imports from modules that define __all__.
|
||||
allow-wildcard-with-all=no
|
||||
|
||||
# Analyse import fallback blocks. This can be used to support both Python 2 and
|
||||
# 3 compatible code, which means that the block might have code that exists
|
||||
# only in one or another interpreter, leading to false positives when analysed.
|
||||
analyse-fallback-blocks=no
|
||||
|
||||
# Deprecated modules which should not be used, separated by a comma
|
||||
deprecated-modules=optparse,tkinter.tix
|
||||
|
||||
# Create a graph of external dependencies in the given file (report RP0402 must
|
||||
# not be disabled)
|
||||
ext-import-graph=
|
||||
|
||||
# Create a graph of every (i.e. internal and external) dependencies in the
|
||||
# given file (report RP0402 must not be disabled)
|
||||
import-graph=
|
||||
|
||||
# Create a graph of internal dependencies in the given file (report RP0402 must
|
||||
# not be disabled)
|
||||
int-import-graph=
|
||||
|
||||
# Force import order to recognize a module as part of the standard
|
||||
# compatibility libraries.
|
||||
known-standard-library=
|
||||
|
||||
# Force import order to recognize a module as part of a third party library.
|
||||
known-third-party=enchant
|
||||
|
||||
|
||||
[CLASSES]
|
||||
|
||||
# List of method names used to declare (i.e. assign) instance attributes.
|
||||
defining-attr-methods=__init__,
|
||||
__new__,
|
||||
setUp
|
||||
|
||||
# List of member names, which should be excluded from the protected access
|
||||
# warning.
|
||||
exclude-protected=_asdict,
|
||||
_fields,
|
||||
_replace,
|
||||
_source,
|
||||
_make
|
||||
|
||||
# List of valid names for the first argument in a class method.
|
||||
valid-classmethod-first-arg=cls
|
||||
|
||||
# List of valid names for the first argument in a metaclass class method.
|
||||
valid-metaclass-classmethod-first-arg=mcs
|
||||
|
||||
|
||||
[DESIGN]
|
||||
|
||||
# Maximum number of arguments for function / method
|
||||
max-args=5
|
||||
|
||||
# Maximum number of attributes for a class (see R0902).
|
||||
max-attributes=7
|
||||
|
||||
# Maximum number of boolean expressions in a if statement
|
||||
max-bool-expr=5
|
||||
|
||||
# Maximum number of branch for function / method body
|
||||
max-branches=12
|
||||
|
||||
# Maximum number of locals for function / method body
|
||||
max-locals=20
|
||||
|
||||
# Maximum number of parents for a class (see R0901).
|
||||
max-parents=10
|
||||
|
||||
# Maximum number of public methods for a class (see R0904).
|
||||
max-public-methods=20
|
||||
|
||||
# Maximum number of return / yield for function / method body
|
||||
max-returns=6
|
||||
|
||||
# Maximum number of statements in function / method body
|
||||
max-statements=50
|
||||
|
||||
# Minimum number of public methods for a class (see R0903).
|
||||
min-public-methods=0
|
||||
|
||||
|
||||
[EXCEPTIONS]
|
||||
|
||||
# Exceptions that will emit a warning when being caught. Defaults to
|
||||
# "Exception"
|
||||
overgeneral-exceptions=builtins.Exception
|
||||
@@ -0,0 +1,3 @@
|
||||
include LICENSE
|
||||
include README.md
|
||||
recursive-include src/backend/conversations *.html *.png *.gif *.css *.ico *.jpg *.jpeg *.po *.mo *.eot *.svg *.ttf *.woff *.woff2
|
||||
@@ -0,0 +1,16 @@
|
||||
"""Admin classes and registrations for chat application."""
|
||||
|
||||
from django.contrib import admin
|
||||
|
||||
from . import models
|
||||
|
||||
|
||||
@admin.register(models.ChatConversation)
|
||||
class ChatConversationAdmin(admin.ModelAdmin):
|
||||
"""Admin class for the ChatConversation model"""
|
||||
|
||||
list_display = (
|
||||
"id",
|
||||
"created_at",
|
||||
"updated_at",
|
||||
)
|
||||
@@ -0,0 +1,451 @@
|
||||
"""This module defines the data structures used in the Vercel AI SDK for chat interactions."""
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, List, Literal, Optional, Union
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
# JSONValue type
|
||||
JSONValue = Union[None, str, int, float, bool, Dict[str, Any], List[Any]]
|
||||
|
||||
|
||||
# ToolCall and ToolResult
|
||||
class ToolCall(BaseModel):
|
||||
"""
|
||||
Represents a call to a tool with arguments.
|
||||
|
||||
Attributes:
|
||||
toolCallId: A unique identifier for the tool call.
|
||||
toolName: The name of the tool being called.
|
||||
args: The arguments passed to the tool.
|
||||
"""
|
||||
|
||||
toolCallId: str
|
||||
toolName: str
|
||||
args: Dict[str, Any]
|
||||
|
||||
|
||||
class ToolResult(BaseModel):
|
||||
"""
|
||||
Represents the result of a tool call including the original call details.
|
||||
|
||||
Attributes:
|
||||
toolCallId: A unique identifier for the tool call.
|
||||
toolName: The name of the tool that was called.
|
||||
args: The arguments that were passed to the tool.
|
||||
result: The result returned by the tool.
|
||||
"""
|
||||
|
||||
toolCallId: str
|
||||
toolName: str
|
||||
args: Dict[str, Any]
|
||||
result: Any
|
||||
|
||||
|
||||
# ToolInvocation union
|
||||
class ToolInvocationPartialCall(ToolCall):
|
||||
"""
|
||||
Represents a tool call that is in progress with partial arguments.
|
||||
|
||||
Attributes:
|
||||
state: The state of the tool invocation, fixed to 'partial-call'.
|
||||
step: Optional step number to track the sequence of tool invocations.
|
||||
"""
|
||||
|
||||
state: Literal["partial-call"]
|
||||
step: Optional[int] = None
|
||||
|
||||
|
||||
class ToolInvocationCall(ToolCall):
|
||||
"""
|
||||
Represents a complete tool call ready for execution.
|
||||
|
||||
Attributes:
|
||||
state: The state of the tool invocation, fixed to 'call'.
|
||||
step: Optional step number to track the sequence of tool invocations.
|
||||
"""
|
||||
|
||||
state: Literal["call"]
|
||||
step: Optional[int] = None
|
||||
|
||||
|
||||
class ToolInvocationResult(ToolResult):
|
||||
"""
|
||||
Represents a completed tool call with its result.
|
||||
|
||||
Attributes:
|
||||
state: The state of the tool invocation, fixed to 'result'.
|
||||
step: Optional step number to track the sequence of tool invocations.
|
||||
"""
|
||||
|
||||
state: Literal["result"]
|
||||
step: Optional[int] = None
|
||||
|
||||
|
||||
ToolInvocation = Union[ToolInvocationPartialCall, ToolInvocationCall, ToolInvocationResult]
|
||||
|
||||
|
||||
# Attachment
|
||||
class Attachment(BaseModel):
|
||||
"""
|
||||
Represents a file attachment that can be sent with a message.
|
||||
|
||||
Attributes:
|
||||
name: Optional name of the attachment, usually the filename.
|
||||
contentType: Optional MIME type of the attachment.
|
||||
url: The URL of the attachment, can be a hosted URL or Data URL.
|
||||
"""
|
||||
|
||||
name: Optional[str] = None
|
||||
contentType: Optional[str] = None
|
||||
url: str
|
||||
|
||||
|
||||
# Reasoning details
|
||||
class ReasoningDetailText(BaseModel):
|
||||
"""
|
||||
Represents a text-based reasoning detail in a message.
|
||||
|
||||
Attributes:
|
||||
type: The type of reasoning detail, fixed to 'text'.
|
||||
text: The text content of the reasoning.
|
||||
signature: Optional signature associated with the reasoning.
|
||||
"""
|
||||
|
||||
type: Literal["text"]
|
||||
text: str
|
||||
signature: Optional[str] = None
|
||||
|
||||
|
||||
class ReasoningDetailRedacted(BaseModel):
|
||||
"""
|
||||
Represents a redacted reasoning detail in a message.
|
||||
|
||||
Attributes:
|
||||
type: The type of reasoning detail, fixed to 'redacted'.
|
||||
data: The redacted content.
|
||||
"""
|
||||
|
||||
type: Literal["redacted"]
|
||||
data: str
|
||||
|
||||
|
||||
ReasoningDetail = Union[ReasoningDetailText, ReasoningDetailRedacted]
|
||||
|
||||
|
||||
# UIParts
|
||||
class TextUIPart(BaseModel):
|
||||
"""
|
||||
Represents a text part of a message.
|
||||
|
||||
Attributes:
|
||||
type: The type of UI part, fixed to 'text'.
|
||||
text: The text content.
|
||||
"""
|
||||
|
||||
type: Literal["text"]
|
||||
text: str
|
||||
|
||||
|
||||
class ReasoningUIPart(BaseModel):
|
||||
"""
|
||||
Represents a reasoning part of a message.
|
||||
|
||||
Attributes:
|
||||
type: The type of UI part, fixed to 'reasoning'.
|
||||
reasoning: The reasoning text.
|
||||
details: List of reasoning details.
|
||||
"""
|
||||
|
||||
type: Literal["reasoning"]
|
||||
reasoning: str
|
||||
details: List[ReasoningDetail]
|
||||
|
||||
|
||||
class ToolInvocationUIPart(BaseModel):
|
||||
"""
|
||||
Represents a tool invocation part of a message.
|
||||
|
||||
Attributes:
|
||||
type: The type of UI part, fixed to 'tool-invocation'.
|
||||
toolInvocation: The tool invocation details.
|
||||
"""
|
||||
|
||||
type: Literal["tool-invocation"]
|
||||
toolInvocation: ToolInvocation
|
||||
|
||||
|
||||
class LanguageModelV1Source(BaseModel):
|
||||
"""
|
||||
Represents source information from a language model.
|
||||
|
||||
Attributes:
|
||||
source_type: The type of source.
|
||||
details: Additional details about the source.
|
||||
"""
|
||||
|
||||
source_type: str
|
||||
details: Dict[str, Any]
|
||||
|
||||
|
||||
class SourceUIPart(BaseModel):
|
||||
"""
|
||||
Represents a source part of a message.
|
||||
|
||||
Attributes:
|
||||
type: The type of UI part, fixed to 'source'.
|
||||
source: The source information.
|
||||
"""
|
||||
|
||||
type: Literal["source"]
|
||||
source: LanguageModelV1Source
|
||||
|
||||
|
||||
class FileUIPart(BaseModel):
|
||||
"""
|
||||
Represents a file part of a message.
|
||||
|
||||
Attributes:
|
||||
type: The type of UI part, fixed to 'file'.
|
||||
mimeType: The MIME type of the file.
|
||||
data: The file data.
|
||||
"""
|
||||
|
||||
type: Literal["file"]
|
||||
mimeType: str
|
||||
data: str
|
||||
|
||||
|
||||
class StepStartUIPart(BaseModel):
|
||||
"""
|
||||
Represents a step boundary part of a message.
|
||||
|
||||
Attributes:
|
||||
type: The type of UI part, fixed to 'step-start'.
|
||||
"""
|
||||
|
||||
type: Literal["step-start"]
|
||||
|
||||
|
||||
UIPart = Union[
|
||||
TextUIPart,
|
||||
ReasoningUIPart,
|
||||
ToolInvocationUIPart,
|
||||
SourceUIPart,
|
||||
FileUIPart,
|
||||
StepStartUIPart,
|
||||
]
|
||||
|
||||
|
||||
# Message and related types
|
||||
class Message(BaseModel):
|
||||
"""
|
||||
Represents a message in a chat conversation.
|
||||
|
||||
Attributes:
|
||||
id: A unique identifier for the message.
|
||||
createdAt: Optional timestamp when the message was created.
|
||||
experimental_attachments: Optional list of attachments.
|
||||
role: The role of the sender (system, user, assistant, or data).
|
||||
annotations: Optional list of annotations.
|
||||
parts: Optional list of UI parts that make up the message content.
|
||||
"""
|
||||
|
||||
id: str
|
||||
createdAt: Optional[datetime] = None
|
||||
content: str # deprecated, use parts instead
|
||||
reasoning: Optional[str] = None # deprecated, use parts instead
|
||||
experimental_attachments: Optional[List[Attachment]] = None
|
||||
role: Literal["system", "user", "assistant", "data"]
|
||||
# data: Optional[JSONValue] = None
|
||||
annotations: Optional[List[JSONValue]] = None
|
||||
toolInvocations: Optional[List[ToolInvocation]] = None # deprecated, use parts instead
|
||||
parts: Optional[List[UIPart]] = None
|
||||
|
||||
|
||||
class UIMessage(Message):
|
||||
"""
|
||||
Represents a message with UI parts for rendering in the user interface.
|
||||
|
||||
Attributes:
|
||||
parts: List of UI parts that make up the message content.
|
||||
"""
|
||||
|
||||
parts: List[UIPart]
|
||||
|
||||
|
||||
class CreateMessage(BaseModel):
|
||||
"""
|
||||
Model for creating a new message.
|
||||
|
||||
Attributes:
|
||||
createdAt: Optional timestamp when the message was created.
|
||||
content: The text content of the message.
|
||||
reasoning: Optional reasoning for the message.
|
||||
experimental_attachments: Optional list of attachments.
|
||||
role: The role of the sender (system, user, assistant, or data).
|
||||
data: Optional JSON value for data messages.
|
||||
annotations: Optional list of annotations.
|
||||
toolInvocations: Optional list of tool invocations.
|
||||
parts: Optional list of UI parts that make up the message content.
|
||||
id: Optional unique identifier for the message.
|
||||
"""
|
||||
|
||||
createdAt: Optional[datetime] = None
|
||||
content: str
|
||||
reasoning: Optional[str] = None
|
||||
experimental_attachments: Optional[List[Attachment]] = None
|
||||
role: Literal["system", "user", "assistant", "data"]
|
||||
data: Optional[JSONValue] = None
|
||||
annotations: Optional[List[JSONValue]] = None
|
||||
toolInvocations: Optional[List[ToolInvocation]] = None
|
||||
parts: Optional[List[UIPart]] = None
|
||||
id: Optional[str] = None
|
||||
|
||||
|
||||
class ChatRequest(BaseModel):
|
||||
"""
|
||||
Represents a request to the chat API.
|
||||
|
||||
Attributes:
|
||||
headers: Optional request headers.
|
||||
body: Optional request body.
|
||||
messages: List of messages in the conversation.
|
||||
data: Optional additional data for the request.
|
||||
"""
|
||||
|
||||
headers: Optional[Dict[str, str]] = None
|
||||
body: Optional[Dict[str, Any]] = None
|
||||
messages: List[Message]
|
||||
data: Optional[JSONValue] = None
|
||||
|
||||
|
||||
class ChatRequestOptions(BaseModel):
|
||||
"""
|
||||
Options for a chat request.
|
||||
|
||||
Attributes:
|
||||
headers: Optional request headers.
|
||||
body: Optional request body.
|
||||
data: Optional additional data for the request.
|
||||
experimental_attachments: Optional list of attachments.
|
||||
allowEmptySubmit: Optional flag to allow empty message submission.
|
||||
"""
|
||||
|
||||
headers: Optional[Dict[str, str]] = None
|
||||
body: Optional[Dict[str, Any]] = None
|
||||
data: Optional[JSONValue] = None
|
||||
experimental_attachments: Optional[List[Attachment]] = None
|
||||
allowEmptySubmit: Optional[bool] = None
|
||||
|
||||
|
||||
class UseChatOptions(BaseModel):
|
||||
"""
|
||||
Options for the useChat hook.
|
||||
|
||||
Attributes:
|
||||
keepLastMessageOnError: Optional flag to keep the last message on error.
|
||||
api: Optional API endpoint.
|
||||
id: Optional unique identifier for the chat.
|
||||
initialMessages: Optional initial messages for the chat.
|
||||
initialInput: Optional initial input for the chat.
|
||||
credentials: Optional credentials for the request.
|
||||
headers: Optional request headers.
|
||||
body: Optional request body.
|
||||
sendExtraMessageFields: Optional flag to send extra message fields.
|
||||
streamProtocol: Optional stream protocol to use.
|
||||
"""
|
||||
|
||||
keepLastMessageOnError: Optional[bool] = None
|
||||
api: Optional[str] = None
|
||||
id: Optional[str] = None
|
||||
initialMessages: Optional[List[Message]] = None
|
||||
initialInput: Optional[str] = None
|
||||
credentials: Optional[str] = None
|
||||
headers: Optional[Dict[str, str]] = None
|
||||
body: Optional[Dict[str, Any]] = None
|
||||
sendExtraMessageFields: Optional[bool] = None
|
||||
streamProtocol: Optional[Literal["data", "text"]] = None
|
||||
|
||||
|
||||
class UseCompletionOptions(BaseModel):
|
||||
"""
|
||||
Options for the useCompletion hook.
|
||||
|
||||
Attributes:
|
||||
api: Optional API endpoint.
|
||||
id: Optional unique identifier for the completion.
|
||||
initialInput: Optional initial input for the completion.
|
||||
initialCompletion: Optional initial completion result.
|
||||
credentials: Optional credentials for the request.
|
||||
headers: Optional request headers.
|
||||
body: Optional request body.
|
||||
streamProtocol: Optional stream protocol to use.
|
||||
"""
|
||||
|
||||
api: Optional[str] = None
|
||||
id: Optional[str] = None
|
||||
initialInput: Optional[str] = None
|
||||
initialCompletion: Optional[str] = None
|
||||
credentials: Optional[str] = None
|
||||
headers: Optional[Dict[str, str]] = None
|
||||
body: Optional[Dict[str, Any]] = None
|
||||
streamProtocol: Optional[Literal["data", "text"]] = None
|
||||
|
||||
|
||||
class LanguageModelUsage(BaseModel):
|
||||
"""
|
||||
Represents the token usage in a language model interaction.
|
||||
|
||||
Attributes:
|
||||
promptTokens: Number of tokens used in the prompt.
|
||||
completionTokens: Number of tokens used in the completion.
|
||||
totalTokens: Total number of tokens used.
|
||||
"""
|
||||
|
||||
promptTokens: int
|
||||
completionTokens: int
|
||||
totalTokens: int
|
||||
|
||||
|
||||
class AssistantMessageContentText(BaseModel):
|
||||
"""
|
||||
Represents text content in an assistant message.
|
||||
|
||||
Attributes:
|
||||
type: The type of content, fixed to 'text'.
|
||||
text: Dictionary containing the text value.
|
||||
"""
|
||||
|
||||
type: Literal["text"]
|
||||
text: Dict[str, str] # {'value': str}
|
||||
|
||||
|
||||
class AssistantMessage(BaseModel):
|
||||
"""
|
||||
Represents a message from the assistant.
|
||||
|
||||
Attributes:
|
||||
id: A unique identifier for the message.
|
||||
role: The role of the sender, fixed to 'assistant'.
|
||||
content: List of content blocks in the message.
|
||||
"""
|
||||
|
||||
id: str
|
||||
role: Literal["assistant"]
|
||||
content: List[AssistantMessageContentText]
|
||||
|
||||
|
||||
class DataMessage(BaseModel):
|
||||
"""
|
||||
Represents a data message.
|
||||
|
||||
Attributes:
|
||||
id: Optional unique identifier for the message.
|
||||
role: The role of the sender, fixed to 'data'.
|
||||
data: The JSON data contained in the message.
|
||||
"""
|
||||
|
||||
id: Optional[str] = None
|
||||
role: Literal["data"]
|
||||
data: JSONValue
|
||||
@@ -0,0 +1,12 @@
|
||||
"""Chat application"""
|
||||
|
||||
from django.apps import AppConfig
|
||||
from django.utils.translation import gettext_lazy as _
|
||||
|
||||
|
||||
class ChatDefaultConfig(AppConfig):
|
||||
"""Configuration class for the chat application."""
|
||||
|
||||
name = "chat"
|
||||
app_label = "chat"
|
||||
verbose_name = _("chat application")
|
||||
@@ -0,0 +1,421 @@
|
||||
"""AIAgentService class for handling AI agent interactions."""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import queue
|
||||
import threading
|
||||
import uuid
|
||||
from contextlib import AsyncExitStack
|
||||
from typing import List
|
||||
|
||||
from django.conf import settings
|
||||
from django.core.exceptions import ImproperlyConfigured
|
||||
|
||||
from agents import Agent, ModelResponse, OpenAIChatCompletionsModel, Runner, Usage
|
||||
from asgiref.sync import sync_to_async
|
||||
from openai import AsyncOpenAI
|
||||
from openai.types.responses import ResponseInputItemParam, ResponseOutputItem
|
||||
from openai.types.responses.response_usage import (
|
||||
InputTokensDetails,
|
||||
OutputTokensDetails,
|
||||
ResponseUsage,
|
||||
)
|
||||
|
||||
from chat.ai_sdk_types import (
|
||||
TextUIPart,
|
||||
ToolInvocationPartialCall,
|
||||
ToolInvocationResult,
|
||||
ToolInvocationUIPart,
|
||||
UIMessage,
|
||||
)
|
||||
from chat.mcp_servers import get_mcp_servers
|
||||
from chat.tools import get_tool_by_name
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def convert_async_generator_to_sync(async_gen):
|
||||
"""Convert an async generator to a sync generator."""
|
||||
q = queue.Queue()
|
||||
sentinel = object()
|
||||
exc_sentinel = object()
|
||||
|
||||
async def run_async_gen():
|
||||
try:
|
||||
async for item in async_gen:
|
||||
q.put(item)
|
||||
except Exception as exc: # pylint: disable=broad-except #noqa: BLE001
|
||||
q.put((exc_sentinel, exc))
|
||||
finally:
|
||||
q.put(sentinel)
|
||||
|
||||
def start_async_loop():
|
||||
asyncio.run(run_async_gen())
|
||||
|
||||
thread = threading.Thread(target=start_async_loop, daemon=True)
|
||||
thread.start()
|
||||
|
||||
try:
|
||||
while True:
|
||||
item = q.get()
|
||||
if item is sentinel:
|
||||
break
|
||||
if isinstance(item, tuple) and item[0] is exc_sentinel:
|
||||
# re-raise the exception in the sync context
|
||||
raise item[1]
|
||||
yield item
|
||||
finally:
|
||||
thread.join()
|
||||
|
||||
|
||||
class AIAgentService:
|
||||
"""Service class for AI-related operations."""
|
||||
|
||||
def __init__(self, conversation):
|
||||
"""Ensure that the AI configuration is set properly."""
|
||||
if settings.AI_BASE_URL is None or settings.AI_API_KEY is None or settings.AI_MODEL is None:
|
||||
raise ImproperlyConfigured("AIChatService configuration not set")
|
||||
|
||||
self.model = OpenAIChatCompletionsModel(
|
||||
model=settings.AI_MODEL,
|
||||
openai_client=AsyncOpenAI(
|
||||
base_url=settings.AI_BASE_URL,
|
||||
api_key=settings.AI_API_KEY,
|
||||
),
|
||||
)
|
||||
self.conversation = conversation
|
||||
|
||||
@staticmethod
|
||||
def _convert_to_openai_messages(
|
||||
messages: List[UIMessage],
|
||||
) -> List[ResponseInputItemParam]:
|
||||
"""Convert UI messages to OpenAI format."""
|
||||
openai_messages = []
|
||||
|
||||
for message in messages:
|
||||
content_parts = []
|
||||
tool_calls = []
|
||||
|
||||
for part in message.parts:
|
||||
match part.type:
|
||||
case "text":
|
||||
content_parts.append(
|
||||
{
|
||||
"type": "input_text" if message.role == "user" else "output_text",
|
||||
"text": part.text,
|
||||
}
|
||||
)
|
||||
|
||||
case "image":
|
||||
content_parts.append(
|
||||
{
|
||||
"type": "input_image",
|
||||
"image_url": part.url,
|
||||
}
|
||||
)
|
||||
|
||||
case "file":
|
||||
content_parts.append(
|
||||
{
|
||||
"type": "input_file",
|
||||
"file_data": part.url,
|
||||
"filename": part.name,
|
||||
}
|
||||
)
|
||||
|
||||
case "tool-invocation":
|
||||
# Extract the tool invocation data
|
||||
tool_invocation = part.toolInvocation
|
||||
tool_calls.append(
|
||||
{
|
||||
"call_id": tool_invocation.toolCallId,
|
||||
"type": "function_call",
|
||||
"name": tool_invocation.toolName,
|
||||
"arguments": json.dumps(tool_invocation.args),
|
||||
"status": tool_invocation.state,
|
||||
}
|
||||
)
|
||||
|
||||
case _:
|
||||
logger.warning("Unrecognized part type: %s in part: %s", part.type, part)
|
||||
|
||||
# Add experimental attachments if they exist
|
||||
if hasattr(message, "experimental_attachments") and message.experimental_attachments:
|
||||
for attachment in message.experimental_attachments:
|
||||
if attachment.contentType.startswith("image"):
|
||||
content_parts.append(
|
||||
{
|
||||
"type": "input_image",
|
||||
"image_url": attachment.url,
|
||||
}
|
||||
)
|
||||
elif attachment.contentType.startswith("text"):
|
||||
content_parts.append(
|
||||
{
|
||||
"type": "input_file",
|
||||
"file_data": attachment.url,
|
||||
"filename": attachment.name,
|
||||
}
|
||||
)
|
||||
|
||||
# Add message with content parts if there are any
|
||||
if content_parts:
|
||||
openai_messages.append(
|
||||
{"role": message.role, "content": content_parts, "type": "message"}
|
||||
)
|
||||
|
||||
# Add tool calls separately, will be merged by `items_to_messages`
|
||||
openai_messages += tool_calls
|
||||
|
||||
return openai_messages
|
||||
|
||||
def stream_text(self, messages: List[UIMessage]):
|
||||
"""Simple generator to convert async generator to sync generator."""
|
||||
async_generator = self.stream_text_async(messages)
|
||||
return convert_async_generator_to_sync(async_generator)
|
||||
|
||||
async def stream_text_async(self, messages: List[UIMessage]):
|
||||
"""Async generator for streaming agent events."""
|
||||
openai_messages = self._convert_to_openai_messages(messages)
|
||||
logger.debug("[stream_data_async] Received messages: %s", openai_messages)
|
||||
|
||||
async with AsyncExitStack() as stack:
|
||||
initialized_mcp_servers = [
|
||||
await stack.enter_async_context(mcp_server) for mcp_server in get_mcp_servers()
|
||||
]
|
||||
agent = Agent(
|
||||
name=settings.AI_AGENT_NAME,
|
||||
instructions=settings.AI_AGENT_INSTRUCTIONS,
|
||||
model=self.model,
|
||||
tools=[get_tool_by_name(tool_name) for tool_name in settings.AI_AGENT_TOOLS],
|
||||
mcp_servers=initialized_mcp_servers,
|
||||
)
|
||||
|
||||
result = Runner.run_streamed(
|
||||
agent,
|
||||
input=openai_messages,
|
||||
)
|
||||
|
||||
async for event in result.stream_events():
|
||||
logger.debug("[stream_text_async] Received event: %s", event)
|
||||
if event.type == "raw_response_event":
|
||||
data = event.data
|
||||
logger.debug("[stream_text_async] - data: %s", data)
|
||||
if data.type == "response.output_text.delta":
|
||||
yield data.delta
|
||||
|
||||
# At the end, save the response and yield the finish message part
|
||||
_response_usage = Usage()
|
||||
for raw_response in result.raw_responses:
|
||||
_response_usage.add(raw_response.usage)
|
||||
|
||||
await sync_to_async(self._update_conversation)(
|
||||
openai_messages, result.raw_responses, _response_usage
|
||||
)
|
||||
|
||||
def stream_data(self, messages: List[UIMessage]):
|
||||
"""Simple generator to convert async generator to sync generator."""
|
||||
async_generator = self.stream_data_async(messages)
|
||||
return convert_async_generator_to_sync(async_generator)
|
||||
|
||||
async def stream_data_async(self, messages: List[UIMessage]):
|
||||
"""Async generator for streaming agent events."""
|
||||
finish_reason = "stop"
|
||||
|
||||
openai_messages = self._convert_to_openai_messages(messages)
|
||||
logger.debug("[stream_data_async] Received messages: %s", openai_messages)
|
||||
|
||||
async with AsyncExitStack() as stack:
|
||||
initialized_mcp_servers = [
|
||||
await stack.enter_async_context(mcp_server) for mcp_server in get_mcp_servers()
|
||||
]
|
||||
|
||||
# websearch_agent = Agent(
|
||||
# name="web search",
|
||||
# instructions=(
|
||||
# "You are a web search agent. "
|
||||
# "Your task is to search the web for up-to-date information."
|
||||
# " You will be called by the main agent to perform web searches."
|
||||
# " You will receive a query and return the search results."
|
||||
# " The results should be in the format of a list of dictionaries, "
|
||||
# "each containing 'link', 'title', and 'snippet' keys."
|
||||
# " If you cannot find any results, return an empty list."
|
||||
# " Do not include any other information in your response."
|
||||
# " Do not include any additional text or explanations."
|
||||
# " You must always annotate the response mentioning the"
|
||||
# " url, title, and snippet."
|
||||
# ),
|
||||
# handoff_description=(
|
||||
# "You are a web search agent. "
|
||||
# "Your task is to search the web for up-to-date information."
|
||||
# ),
|
||||
# model=self.model,
|
||||
# tools=[agent_web_search_tavily],
|
||||
# )
|
||||
|
||||
agent = Agent(
|
||||
name=settings.AI_AGENT_NAME,
|
||||
instructions=settings.AI_AGENT_INSTRUCTIONS,
|
||||
model=self.model,
|
||||
mcp_servers=initialized_mcp_servers,
|
||||
tools=[get_tool_by_name(tool_name) for tool_name in settings.AI_AGENT_TOOLS],
|
||||
# handoffs=[websearch_agent],
|
||||
)
|
||||
result = Runner.run_streamed(
|
||||
agent,
|
||||
input=openai_messages,
|
||||
)
|
||||
|
||||
try:
|
||||
async for event in result.stream_events():
|
||||
logger.debug("[stream_data_async] Received event: %s", event)
|
||||
|
||||
if event.type == "raw_response_event":
|
||||
data = event.data
|
||||
if data.type == "response.output_text.delta":
|
||||
yield f"0:{json.dumps(data.delta)}\n"
|
||||
|
||||
if hasattr(data, "finish_reason") and data.finish_reason:
|
||||
finish_reason = data.finish_reason
|
||||
|
||||
elif event.type == "run_item_stream_event":
|
||||
item = event.item
|
||||
if item.type == "tool_call_item":
|
||||
_tool_call = {
|
||||
"toolCallId": item.raw_item.call_id,
|
||||
"toolName": item.raw_item.name,
|
||||
"args": (
|
||||
json.loads(item.raw_item.arguments)
|
||||
if hasattr(item.raw_item, "arguments")
|
||||
else {}
|
||||
),
|
||||
}
|
||||
yield f"9:{json.dumps(_tool_call)}\n"
|
||||
elif item.type == "tool_call_output_item":
|
||||
_tool_call_result = {
|
||||
"toolCallId": str(item.raw_item["call_id"]),
|
||||
"result": item.raw_item["output"],
|
||||
}
|
||||
yield f"a:{json.dumps(_tool_call_result)}\n"
|
||||
elif event.type == "agent_updated_stream_event":
|
||||
logger.debug(
|
||||
"[stream_data_async] Agent switched to: %s", event.new_agent.name
|
||||
)
|
||||
|
||||
except Exception as e: # pylint: disable=broad-except
|
||||
logger.exception("Error in stream_data_async")
|
||||
yield f"3:{json.dumps(str(e))}\n"
|
||||
finish_reason = "error"
|
||||
|
||||
# At the end, save the response and yield the finish message part
|
||||
_response_usage = Usage()
|
||||
for raw_response in result.raw_responses:
|
||||
_response_usage.add(raw_response.usage)
|
||||
|
||||
await sync_to_async(self._update_conversation)(
|
||||
openai_messages, result.raw_responses, _response_usage
|
||||
)
|
||||
|
||||
_finish_message = {
|
||||
"finishReason": finish_reason,
|
||||
"usage": {
|
||||
"promptTokens": _response_usage.input_tokens,
|
||||
"completionTokens": _response_usage.output_tokens,
|
||||
},
|
||||
}
|
||||
yield f"d:{json.dumps(_finish_message)}\n"
|
||||
|
||||
def _update_conversation(
|
||||
self,
|
||||
input_messages: List[ResponseInputItemParam],
|
||||
result_raw_responses: List[ModelResponse],
|
||||
response_usage: Usage,
|
||||
):
|
||||
ui_messages = []
|
||||
|
||||
self.conversation.openai_messages = input_messages + [
|
||||
output.model_dump()
|
||||
for raw_response in result_raw_responses
|
||||
for output in raw_response.output
|
||||
]
|
||||
|
||||
for raw_response in result_raw_responses:
|
||||
ui_messages += self._convert_openai_output_to_ui_messages(raw_response.output)
|
||||
|
||||
self.conversation.messages = self.conversation.ui_messages + [
|
||||
ui_message.model_dump() for ui_message in ui_messages
|
||||
]
|
||||
|
||||
if self.conversation.agent_usage:
|
||||
total_usage = ResponseUsage(**self.conversation.agent_usage)
|
||||
else:
|
||||
total_usage = ResponseUsage(
|
||||
input_tokens=0,
|
||||
output_tokens=0,
|
||||
input_tokens_details=InputTokensDetails(cached_tokens=0),
|
||||
output_tokens_details=OutputTokensDetails(reasoning_tokens=0),
|
||||
total_tokens=0,
|
||||
)
|
||||
|
||||
total_usage.input_tokens += response_usage.input_tokens # pylint: disable=no-member
|
||||
total_usage.output_tokens += response_usage.output_tokens # pylint: disable=no-member
|
||||
total_usage.input_tokens_details.cached_tokens += (
|
||||
response_usage.input_tokens_details.cached_tokens
|
||||
)
|
||||
total_usage.output_tokens_details.reasoning_tokens += (
|
||||
response_usage.output_tokens_details.reasoning_tokens
|
||||
)
|
||||
total_usage.total_tokens = response_usage.total_tokens
|
||||
|
||||
self.conversation.agent_usage = total_usage.model_dump()
|
||||
|
||||
self.conversation.save()
|
||||
|
||||
def _convert_openai_output_to_ui_messages(
|
||||
self, output: List[ResponseOutputItem]
|
||||
) -> List[UIMessage]:
|
||||
"""Convert OpenAI output to UI messages."""
|
||||
ui_messages = []
|
||||
|
||||
for item in output:
|
||||
if item.type == "message":
|
||||
text_parts = [TextUIPart(type="text", text=item.text) for item in item.content]
|
||||
ui_messages.append(
|
||||
UIMessage(
|
||||
id=str(uuid.uuid4()),
|
||||
role="assistant",
|
||||
parts=text_parts,
|
||||
content="".join(part.text for part in text_parts),
|
||||
)
|
||||
)
|
||||
elif item.type == "function_call":
|
||||
if item.status == "in_progress":
|
||||
tool_invocation = ToolInvocationPartialCall(
|
||||
state="partial-call",
|
||||
step=None,
|
||||
toolCallId=item.call_id,
|
||||
toolName=item.name,
|
||||
args=json.loads(item.arguments),
|
||||
)
|
||||
elif item.status == "completed":
|
||||
tool_invocation = ToolInvocationResult(
|
||||
state="result",
|
||||
step=None,
|
||||
toolCallId=item.call_id,
|
||||
toolName=item.name,
|
||||
args=json.loads(item.arguments),
|
||||
result=json.loads(item.result) if item.result else None,
|
||||
)
|
||||
# elif item.status == "incomplete":
|
||||
else:
|
||||
logger.warning("[stream_data_async] Unhandled message: %s", item)
|
||||
continue
|
||||
|
||||
ui_tool_invocation = ToolInvocationUIPart(
|
||||
type="tool-invocation",
|
||||
toolInvocation=tool_invocation,
|
||||
)
|
||||
ui_messages.append(UIMessage(role="assistant", parts=[ui_tool_invocation]))
|
||||
# Handle other types as needed
|
||||
return ui_messages
|
||||
@@ -0,0 +1,16 @@
|
||||
"""Factories for chat application."""
|
||||
|
||||
import factory.django
|
||||
|
||||
from core.factories import UserFactory
|
||||
|
||||
from . import models
|
||||
|
||||
|
||||
class ChatConversationFactory(factory.django.DjangoModelFactory):
|
||||
"""Factory for creating ChatConversation instances."""
|
||||
|
||||
owner = factory.SubFactory(UserFactory)
|
||||
|
||||
class Meta:
|
||||
model = models.ChatConversation
|
||||
@@ -0,0 +1,23 @@
|
||||
"""MCP servers configuration: will be replaced by models."""
|
||||
|
||||
from agents.mcp import MCPServerStreamableHttp, MCPServerStreamableHttpParams
|
||||
|
||||
MCP_SERVERS = {
|
||||
"mcpServers": {
|
||||
# "github": {
|
||||
# "url": "https://api.githubcopilot.com/mcp/",
|
||||
# "headers": {"Authorization": "Bearer XXX"},
|
||||
# },
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def get_mcp_servers():
|
||||
"""Retrieve MCP servers configuration."""
|
||||
return [
|
||||
MCPServerStreamableHttp(
|
||||
name=name,
|
||||
params=MCPServerStreamableHttpParams(**server),
|
||||
)
|
||||
for name, server in MCP_SERVERS["mcpServers"].items()
|
||||
]
|
||||
@@ -0,0 +1,90 @@
|
||||
# Generated by Django 5.2.3 on 2025-06-26 12:15
|
||||
|
||||
import uuid
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
initial = True
|
||||
|
||||
dependencies = []
|
||||
|
||||
operations = [
|
||||
migrations.CreateModel(
|
||||
name="ChatConversation",
|
||||
fields=[
|
||||
(
|
||||
"id",
|
||||
models.UUIDField(
|
||||
default=uuid.uuid4,
|
||||
editable=False,
|
||||
help_text="primary key for the record as UUID",
|
||||
primary_key=True,
|
||||
serialize=False,
|
||||
verbose_name="id",
|
||||
),
|
||||
),
|
||||
(
|
||||
"created_at",
|
||||
models.DateTimeField(
|
||||
auto_now_add=True,
|
||||
help_text="date and time at which a record was created",
|
||||
verbose_name="created on",
|
||||
),
|
||||
),
|
||||
(
|
||||
"updated_at",
|
||||
models.DateTimeField(
|
||||
auto_now=True,
|
||||
help_text="date and time at which a record was last updated",
|
||||
verbose_name="updated on",
|
||||
),
|
||||
),
|
||||
(
|
||||
"title",
|
||||
models.CharField(
|
||||
blank=True,
|
||||
help_text="Title of the chat conversation",
|
||||
max_length=100,
|
||||
null=True,
|
||||
),
|
||||
),
|
||||
(
|
||||
"ui_messages",
|
||||
models.JSONField(
|
||||
blank=True,
|
||||
default=list,
|
||||
help_text="UI messages for the chat conversation, sent by frontend, not used",
|
||||
),
|
||||
),
|
||||
(
|
||||
"openai_messages",
|
||||
models.JSONField(
|
||||
blank=True,
|
||||
default=list,
|
||||
help_text="OpenAI messages for the chat conversation, not used",
|
||||
),
|
||||
),
|
||||
(
|
||||
"messages",
|
||||
models.JSONField(
|
||||
blank=True,
|
||||
default=list,
|
||||
help_text="Stored messages for the chat conversation, sent to frontend",
|
||||
),
|
||||
),
|
||||
(
|
||||
"agent_usage",
|
||||
models.JSONField(
|
||||
blank=True,
|
||||
default=dict,
|
||||
help_text="Agent usage for the chat conversation, provided by OpenAI API",
|
||||
),
|
||||
),
|
||||
],
|
||||
options={
|
||||
"abstract": False,
|
||||
},
|
||||
),
|
||||
]
|
||||
@@ -0,0 +1,26 @@
|
||||
# Generated by Django 5.2.3 on 2025-06-26 12:15
|
||||
|
||||
import django.db.models.deletion
|
||||
from django.conf import settings
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
initial = True
|
||||
|
||||
dependencies = [
|
||||
("chat", "0001_initial"),
|
||||
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AddField(
|
||||
model_name="chatconversation",
|
||||
name="owner",
|
||||
field=models.ForeignKey(
|
||||
on_delete=django.db.models.deletion.CASCADE,
|
||||
related_name="conversations",
|
||||
to=settings.AUTH_USER_MODEL,
|
||||
),
|
||||
),
|
||||
]
|
||||
@@ -0,0 +1,61 @@
|
||||
"""Models for chat conversations."""
|
||||
|
||||
from django.contrib.auth import get_user_model
|
||||
from django.db import models
|
||||
|
||||
from core.models import BaseModel
|
||||
|
||||
User = get_user_model()
|
||||
|
||||
|
||||
class ChatConversation(BaseModel):
|
||||
"""
|
||||
Model representing a chat conversation.
|
||||
|
||||
This model stores the details of a chat conversation:
|
||||
- `owner`: The user who owns the conversation.
|
||||
- `title`: An optional title for the conversation, provided by frontend,
|
||||
the 100 first characters of the first user input message.
|
||||
- `ui_messages`: A JSON field of UI messages sent by the frontend, all content is
|
||||
overridden at each new request from the frontend.
|
||||
- `openai_messages`: A JSON field of OpenAI messages, only for debug purpose, not used.
|
||||
- `messages`: A JSON field of stored messages for the conversation, sent to frontend
|
||||
when loading the conversation.
|
||||
- `agent_usage`: A JSON field of agent usage statistics for the conversation,
|
||||
"""
|
||||
|
||||
owner = models.ForeignKey(
|
||||
User,
|
||||
related_name="conversations",
|
||||
on_delete=models.CASCADE,
|
||||
null=False,
|
||||
blank=False,
|
||||
)
|
||||
title = models.CharField(
|
||||
max_length=100,
|
||||
blank=True,
|
||||
null=True,
|
||||
help_text="Title of the chat conversation",
|
||||
)
|
||||
|
||||
ui_messages = models.JSONField(
|
||||
default=list,
|
||||
blank=True,
|
||||
help_text="UI messages for the chat conversation, sent by frontend, not used",
|
||||
)
|
||||
openai_messages = models.JSONField(
|
||||
default=list,
|
||||
blank=True,
|
||||
help_text="OpenAI messages for the chat conversation, not used",
|
||||
)
|
||||
messages = models.JSONField(
|
||||
default=list,
|
||||
blank=True,
|
||||
help_text="Stored messages for the chat conversation, sent to frontend",
|
||||
)
|
||||
|
||||
agent_usage = models.JSONField(
|
||||
default=dict,
|
||||
blank=True,
|
||||
help_text="Agent usage for the chat conversation, provided by OpenAI API",
|
||||
)
|
||||
@@ -0,0 +1,16 @@
|
||||
"""Serializers for chat application."""
|
||||
|
||||
from rest_framework import serializers
|
||||
|
||||
from chat import models
|
||||
|
||||
|
||||
class ChatConversationSerializer(serializers.ModelSerializer):
|
||||
"""Serializer for chat conversations."""
|
||||
|
||||
owner = serializers.HiddenField(default=serializers.CurrentUserDefault())
|
||||
|
||||
class Meta: # pylint: disable=missing-class-docstring
|
||||
model = models.ChatConversation
|
||||
fields = ["id", "title", "created_at", "updated_at", "messages", "owner"]
|
||||
read_only_fields = ["id", "created_at", "updated_at", "messages"]
|
||||
@@ -0,0 +1 @@
|
||||
"""Tests for chat application."""
|
||||
@@ -0,0 +1,318 @@
|
||||
"""Tests for OpenAI client message conversion."""
|
||||
|
||||
# pylint: disable=protected-access
|
||||
|
||||
import pytest
|
||||
|
||||
from chat.ai_sdk_types import (
|
||||
Attachment,
|
||||
TextUIPart,
|
||||
ToolInvocationResult,
|
||||
ToolInvocationUIPart,
|
||||
UIMessage,
|
||||
)
|
||||
from chat.clients.openai import AIAgentService
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def openai_settings(settings):
|
||||
"""Create a mock AIAgentService for testing."""
|
||||
settings.AI_BASE_URL = "http://test.url"
|
||||
settings.AI_API_KEY = "test_key"
|
||||
settings.AI_MODEL = "test_model"
|
||||
|
||||
|
||||
def test_convert_simple_text_messages():
|
||||
"""Test conversion of simple text messages"""
|
||||
messages = [
|
||||
UIMessage(
|
||||
id="user1",
|
||||
content="Hello, how are you?",
|
||||
role="user",
|
||||
parts=[TextUIPart(type="text", text="Hello, how are you?")],
|
||||
),
|
||||
UIMessage(
|
||||
id="assistant1",
|
||||
content="I'm doing well, thank you!",
|
||||
role="assistant",
|
||||
parts=[TextUIPart(type="text", text="I'm doing well, thank you!")],
|
||||
),
|
||||
]
|
||||
|
||||
result = AIAgentService._convert_to_openai_messages(messages)
|
||||
|
||||
assert result == [
|
||||
{
|
||||
"role": "user",
|
||||
"type": "message",
|
||||
"content": [{"type": "input_text", "text": "Hello, how are you?"}],
|
||||
},
|
||||
{
|
||||
"role": "assistant",
|
||||
"type": "message",
|
||||
"content": [{"type": "output_text", "text": "I'm doing well, thank you!"}],
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def test_convert_messages_with_tool_invocations():
|
||||
"""Test conversion of messages with tool invocations"""
|
||||
messages = [
|
||||
UIMessage(
|
||||
id="user1",
|
||||
content="What's the weather in Paris?",
|
||||
role="user",
|
||||
parts=[TextUIPart(type="text", text="What's the weather in Paris?")],
|
||||
),
|
||||
UIMessage(
|
||||
id="assistant1",
|
||||
content="The current weather in Paris is 22°C.",
|
||||
role="assistant",
|
||||
toolInvocations=[
|
||||
ToolInvocationResult(
|
||||
toolCallId="tool1",
|
||||
toolName="get_current_weather",
|
||||
args={"location": "Paris", "unit": "celsius"},
|
||||
result="{'location': 'Paris', 'temperature': 22, 'unit': 'celsius'}",
|
||||
state="result",
|
||||
step=0,
|
||||
)
|
||||
],
|
||||
parts=[
|
||||
ToolInvocationUIPart(
|
||||
type="tool-invocation",
|
||||
toolInvocation=ToolInvocationResult(
|
||||
toolCallId="tool1",
|
||||
toolName="get_current_weather",
|
||||
args={"location": "Paris", "unit": "celsius"},
|
||||
result="{'location': 'Paris', 'temperature': 22, 'unit': 'celsius'}",
|
||||
state="result",
|
||||
step=0,
|
||||
),
|
||||
),
|
||||
TextUIPart(type="text", text="The current weather in Paris is 22°C."),
|
||||
],
|
||||
),
|
||||
]
|
||||
|
||||
result = AIAgentService._convert_to_openai_messages(messages)
|
||||
|
||||
assert result == [
|
||||
{
|
||||
"role": "user",
|
||||
"type": "message",
|
||||
"content": [{"type": "input_text", "text": "What's the weather in Paris?"}],
|
||||
},
|
||||
{
|
||||
"role": "assistant",
|
||||
"type": "message",
|
||||
"content": [{"type": "output_text", "text": "The current weather in Paris is 22°C."}],
|
||||
},
|
||||
{
|
||||
"type": "function_call",
|
||||
"call_id": "tool1",
|
||||
"name": "get_current_weather",
|
||||
"arguments": '{"location": "Paris", "unit": "celsius"}',
|
||||
"status": "result",
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def test_convert_messages_with_images():
|
||||
"""Test conversion of messages with images"""
|
||||
messages = [
|
||||
UIMessage(
|
||||
id="user1",
|
||||
content="What is in this image?",
|
||||
role="user",
|
||||
parts=[
|
||||
TextUIPart(type="text", text="What is in this image?"),
|
||||
],
|
||||
experimental_attachments=[
|
||||
Attachment(
|
||||
contentType="image/jpeg",
|
||||
url="data:image/jpeg;base64,/9j/4AAQSkZJRg==",
|
||||
name="image.jpg",
|
||||
)
|
||||
],
|
||||
)
|
||||
]
|
||||
|
||||
result = AIAgentService._convert_to_openai_messages(messages)
|
||||
|
||||
assert result == [
|
||||
{
|
||||
"role": "user",
|
||||
"type": "message",
|
||||
"content": [
|
||||
{"type": "input_text", "text": "What is in this image?"},
|
||||
{"type": "input_image", "image_url": "data:image/jpeg;base64,/9j/4AAQSkZJRg=="},
|
||||
],
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_convert_messages_with_files():
|
||||
"""Test conversion of messages with files"""
|
||||
messages = [
|
||||
UIMessage(
|
||||
id="user1",
|
||||
content="Here is the file",
|
||||
role="user",
|
||||
parts=[
|
||||
TextUIPart(type="text", text="Here is the file"),
|
||||
],
|
||||
experimental_attachments=[
|
||||
Attachment(
|
||||
contentType="text/plain",
|
||||
url="data:text/plain;base64,SGVsbG8gV29ybGQ=",
|
||||
name="example.txt",
|
||||
)
|
||||
],
|
||||
)
|
||||
]
|
||||
|
||||
result = AIAgentService._convert_to_openai_messages(messages)
|
||||
|
||||
assert result == [
|
||||
{
|
||||
"role": "user",
|
||||
"type": "message",
|
||||
"content": [
|
||||
{"type": "input_text", "text": "Here is the file"},
|
||||
{
|
||||
"type": "input_file",
|
||||
"file_data": "data:text/plain;base64,SGVsbG8gV29ybGQ=",
|
||||
"filename": "example.txt",
|
||||
},
|
||||
],
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_convert_messages_with_experimental_attachments():
|
||||
"""Test conversion of messages with experimental attachments"""
|
||||
messages = [
|
||||
UIMessage(
|
||||
id="user1",
|
||||
content="Check these attachments",
|
||||
role="user",
|
||||
parts=[TextUIPart(type="text", text="Check these attachments")],
|
||||
experimental_attachments=[
|
||||
Attachment(
|
||||
contentType="image/jpeg",
|
||||
url="data:image/jpeg;base64,/9j/4AAQSkZJRg==",
|
||||
name="image.jpg",
|
||||
),
|
||||
Attachment(
|
||||
contentType="text/plain",
|
||||
url="data:text/plain;base64,SGVsbG8gV29ybGQ=",
|
||||
name="example.txt",
|
||||
),
|
||||
],
|
||||
)
|
||||
]
|
||||
|
||||
result = AIAgentService._convert_to_openai_messages(messages)
|
||||
|
||||
assert result == [
|
||||
{
|
||||
"role": "user",
|
||||
"type": "message",
|
||||
"content": [
|
||||
{"type": "input_text", "text": "Check these attachments"},
|
||||
{"type": "input_image", "image_url": "data:image/jpeg;base64,/9j/4AAQSkZJRg=="},
|
||||
{
|
||||
"type": "input_file",
|
||||
"file_data": "data:text/plain;base64,SGVsbG8gV29ybGQ=",
|
||||
"filename": "example.txt",
|
||||
},
|
||||
],
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_convert_complex_message_combination():
|
||||
"""Test conversion of complex combination of message types"""
|
||||
messages = [
|
||||
UIMessage(
|
||||
id="user1",
|
||||
content="What is this image and what's the weather in Paris?",
|
||||
role="user",
|
||||
parts=[
|
||||
TextUIPart(type="text", text="What is this image and what's the weather in Paris?"),
|
||||
],
|
||||
experimental_attachments=[
|
||||
Attachment(
|
||||
contentType="image/jpeg",
|
||||
url="data:image/jpeg;base64,/9j/4AAQSkZJRg==",
|
||||
name="eiffel_tower.jpg",
|
||||
)
|
||||
],
|
||||
),
|
||||
UIMessage(
|
||||
id="assistant1",
|
||||
content="This is the Eiffel Tower. The current weather in Paris is 22°C.",
|
||||
role="assistant",
|
||||
toolInvocations=[
|
||||
ToolInvocationResult(
|
||||
toolCallId="tool1",
|
||||
toolName="get_current_weather",
|
||||
args={"location": "Paris", "unit": "celsius"},
|
||||
result="{'location': 'Paris', 'temperature': 22, 'unit': 'celsius'}",
|
||||
state="result",
|
||||
step=0,
|
||||
)
|
||||
],
|
||||
parts=[
|
||||
ToolInvocationUIPart(
|
||||
type="tool-invocation",
|
||||
toolInvocation=ToolInvocationResult(
|
||||
toolCallId="tool1",
|
||||
toolName="get_current_weather",
|
||||
args={"location": "Paris", "unit": "celsius"},
|
||||
result="{'location': 'Paris', 'temperature': 22, 'unit': 'celsius'}",
|
||||
state="result",
|
||||
step=0,
|
||||
),
|
||||
),
|
||||
TextUIPart(
|
||||
type="text",
|
||||
text="This is the Eiffel Tower. The current weather in Paris is 22°C.",
|
||||
),
|
||||
],
|
||||
),
|
||||
]
|
||||
|
||||
result = AIAgentService._convert_to_openai_messages(messages)
|
||||
|
||||
assert result == [
|
||||
{
|
||||
"role": "user",
|
||||
"type": "message",
|
||||
"content": [
|
||||
{
|
||||
"type": "input_text",
|
||||
"text": "What is this image and what's the weather in Paris?",
|
||||
},
|
||||
{"type": "input_image", "image_url": "data:image/jpeg;base64,/9j/4AAQSkZJRg=="},
|
||||
],
|
||||
},
|
||||
{
|
||||
"role": "assistant",
|
||||
"type": "message",
|
||||
"content": [
|
||||
{
|
||||
"type": "output_text",
|
||||
"text": "This is the Eiffel Tower. The current weather in Paris is 22°C.",
|
||||
}
|
||||
],
|
||||
},
|
||||
{
|
||||
"type": "function_call",
|
||||
"call_id": "tool1",
|
||||
"name": "get_current_weather",
|
||||
"arguments": '{"location": "Paris", "unit": "celsius"}',
|
||||
"status": "result",
|
||||
},
|
||||
]
|
||||
@@ -0,0 +1,10 @@
|
||||
"""Common test fixtures for chat application tests."""
|
||||
|
||||
import pytest
|
||||
from rest_framework.test import APIClient
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def api_client():
|
||||
"""Fixture to provide an API client for testing."""
|
||||
return APIClient()
|
||||
@@ -0,0 +1,73 @@
|
||||
"""Test cases for Tavily web search tool."""
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
import responses
|
||||
|
||||
from chat.tools.web_search_tavily import tavily_web_search
|
||||
|
||||
TAVILY_URL = "https://api.tavily.com/search"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def tavily_settings(settings):
|
||||
"""Set up Tavily settings for tests."""
|
||||
settings.TAVILY_API_KEY = "test_api_key"
|
||||
settings.TAVILY_MAX_RESULTS = 3
|
||||
settings.TAVILY_API_TIMEOUT = 5
|
||||
|
||||
|
||||
@responses.activate
|
||||
def test_agent_web_search_tavily_success():
|
||||
"""Test successful Tavily web search."""
|
||||
responses.add(
|
||||
responses.POST,
|
||||
TAVILY_URL,
|
||||
json={
|
||||
"results": [
|
||||
{"url": "https://example.com/1", "title": "Result 1", "content": "Snippet 1"},
|
||||
{"url": "https://example.com/2", "title": "Result 2", "content": "Snippet 2"},
|
||||
]
|
||||
},
|
||||
status=200,
|
||||
)
|
||||
results = tavily_web_search("test query")
|
||||
assert results == [
|
||||
{"link": "https://example.com/1", "title": "Result 1", "snippet": "Snippet 1"},
|
||||
{"link": "https://example.com/2", "title": "Result 2", "snippet": "Snippet 2"},
|
||||
]
|
||||
|
||||
# Check request payload
|
||||
tavily_request = responses.calls[0].request
|
||||
payload = json.loads(tavily_request.body.decode("utf-8"))
|
||||
assert payload["query"] == "test query"
|
||||
assert payload["api_key"] == "test_api_key"
|
||||
assert payload["max_results"] == 3
|
||||
|
||||
|
||||
@responses.activate
|
||||
def test_agent_web_search_tavily_empty_results():
|
||||
"""Test Tavily web search with no results."""
|
||||
responses.add(
|
||||
responses.POST,
|
||||
TAVILY_URL,
|
||||
json={"results": []},
|
||||
status=200,
|
||||
)
|
||||
results = tavily_web_search("no results query")
|
||||
assert results == []
|
||||
|
||||
|
||||
@responses.activate
|
||||
def test_agent_web_search_tavily_http_error():
|
||||
"""Test Tavily web search with HTTP error."""
|
||||
responses.add(
|
||||
responses.POST,
|
||||
TAVILY_URL,
|
||||
status=500,
|
||||
json={"error": "Internal Server Error"},
|
||||
)
|
||||
with pytest.raises(Exception) as exc:
|
||||
tavily_web_search("error query")
|
||||
assert "500" in str(exc.value) or "Internal Server Error" in str(exc.value)
|
||||
@@ -0,0 +1,664 @@
|
||||
"""Unit tests for chat conversation actions in the chat API view."""
|
||||
|
||||
import json
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
import respx
|
||||
from rest_framework import status
|
||||
|
||||
from core.factories import UserFactory
|
||||
|
||||
from chat.factories import ChatConversationFactory
|
||||
|
||||
# enable database transactions for tests:
|
||||
# transaction=True ensures that the data are available in the database
|
||||
# in other threads
|
||||
pytestmark = pytest.mark.django_db(transaction=True)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def ai_settings(settings):
|
||||
"""Fixture to set AI service URLs for testing."""
|
||||
settings.AI_BASE_URL = "https://www.external-ai-service.com/"
|
||||
settings.AI_API_KEY = "test-api-key"
|
||||
settings.AI_MODEL = "test-model"
|
||||
|
||||
return settings
|
||||
|
||||
|
||||
@pytest.fixture(name="mock_openai_stream")
|
||||
def fixture_mock_openai_stream():
|
||||
"""
|
||||
Fixture to mock the OpenAI stream response.
|
||||
|
||||
See https://platform.openai.com/docs/api-reference/chat-streaming/streaming
|
||||
"""
|
||||
openai_stream = (
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"choices": [
|
||||
{
|
||||
"delta": {"content": "Hello"},
|
||||
"index": 0,
|
||||
"finish_reason": None,
|
||||
}
|
||||
],
|
||||
"object": "chat.completion.chunk",
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"choices": [
|
||||
{
|
||||
"delta": {"content": " there"},
|
||||
"index": 0,
|
||||
"finish_reason": "stop",
|
||||
}
|
||||
],
|
||||
"object": "chat.completion.chunk",
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
"data: [DONE]\n\n"
|
||||
)
|
||||
|
||||
async def mock_stream():
|
||||
for line in openai_stream.splitlines(keepends=True):
|
||||
yield line.encode()
|
||||
|
||||
route = respx.post("https://www.external-ai-service.com/chat/completions").mock(
|
||||
return_value=httpx.Response(200, stream=mock_stream())
|
||||
)
|
||||
|
||||
return route
|
||||
|
||||
|
||||
@pytest.fixture(name="mock_openai_stream_image")
|
||||
def fixture_mock_openai_stream_image():
|
||||
"""
|
||||
Mock a very simple OpenAI stream that *mentions* the image
|
||||
in its textual reply (the real test is that the image URL is
|
||||
forwarded in the request body to the AI service).
|
||||
"""
|
||||
openai_stream = (
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"choices": [
|
||||
{
|
||||
"delta": {"content": "I see a cat"},
|
||||
"index": 0,
|
||||
"finish_reason": None,
|
||||
}
|
||||
],
|
||||
"object": "chat.completion.chunk",
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"choices": [
|
||||
{
|
||||
"delta": {"content": " in the picture."},
|
||||
"index": 0,
|
||||
"finish_reason": "stop",
|
||||
}
|
||||
],
|
||||
"object": "chat.completion.chunk",
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
"data: [DONE]\n\n"
|
||||
)
|
||||
|
||||
async def mock_stream():
|
||||
for line in openai_stream.splitlines(keepends=True):
|
||||
yield line.encode()
|
||||
|
||||
route = respx.post("https://www.external-ai-service.com/chat/completions").mock(
|
||||
return_value=httpx.Response(200, stream=mock_stream())
|
||||
)
|
||||
return route
|
||||
|
||||
|
||||
@pytest.fixture(name="mock_openai_stream_tool")
|
||||
def fixture_mock_openai_stream_tool():
|
||||
"""
|
||||
Mock both API calls in the tool call flow:
|
||||
1. First call returns function call
|
||||
2. Second call returns final answer after tool execution
|
||||
"""
|
||||
|
||||
# First response - tool call
|
||||
first_response = (
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"id": "chatcmpl-tool-call",
|
||||
"object": "chat.completion.chunk",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
"delta": {
|
||||
"tool_calls": [
|
||||
{
|
||||
"index": 0,
|
||||
"id": "xLDcIljdsDrz0idal7tATWSMm2jhMj47",
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "get_current_weather",
|
||||
"arguments": '{"location":"Paris", "unit":"celsius"}',
|
||||
},
|
||||
}
|
||||
]
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"id": "chatcmpl-tool-call",
|
||||
"choices": [{"delta": {}, "finish_reason": "tool_calls"}],
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
"data: [DONE]\n\n"
|
||||
)
|
||||
|
||||
# Second response - final answer
|
||||
second_response = (
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"id": "chatcmpl-final",
|
||||
"object": "chat.completion.chunk",
|
||||
"choices": [{"delta": {"role": "assistant"}, "index": 0}],
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"id": "chatcmpl-final",
|
||||
"choices": [
|
||||
{"delta": {"content": "The current weather in Paris is nice"}, "index": 0}
|
||||
],
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
"data: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"id": "chatcmpl-final",
|
||||
"choices": [{"delta": {}, "finish_reason": "stop"}],
|
||||
}
|
||||
)
|
||||
+ "\n\n"
|
||||
"data: [DONE]\n\n"
|
||||
)
|
||||
|
||||
async def mock_first_response_stream():
|
||||
for line in first_response.splitlines(keepends=True):
|
||||
yield line.encode()
|
||||
|
||||
async def mock_second_response_stream():
|
||||
for line in second_response.splitlines(keepends=True):
|
||||
yield line.encode()
|
||||
|
||||
route = respx.post("https://www.external-ai-service.com/chat/completions").mock(
|
||||
side_effect=[
|
||||
httpx.Response(200, stream=mock_first_response_stream()),
|
||||
httpx.Response(200, stream=mock_second_response_stream()),
|
||||
]
|
||||
)
|
||||
|
||||
return route
|
||||
|
||||
|
||||
def test_post_conversation_anonymous(api_client):
|
||||
"""Test posting messages as an anonymous user returns a 401 error."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/conversation/"
|
||||
data = {"messages": [{"role": "user", "content": "Hello there"}]}
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_401_UNAUTHORIZED
|
||||
|
||||
|
||||
def test_post_conversation_other_user(api_client):
|
||||
"""Test posting messages to another user's conversation returns a 404 error."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
other_user = UserFactory()
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/conversation/"
|
||||
data = {"messages": [{"role": "user", "content": "Hello there"}]}
|
||||
api_client.force_login(other_user)
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_404_NOT_FOUND
|
||||
|
||||
|
||||
def test_post_conversation_no_messages(api_client):
|
||||
"""Test posting with no messages returns a 400 error."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/conversation/"
|
||||
data = {"messages": []}
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
assert "No messages provided" in response.data["error"]
|
||||
|
||||
|
||||
def test_post_conversation_invalid_protocol(api_client):
|
||||
"""Test posting with an invalid protocol returns a 400 error."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/conversation/?protocol=invalid"
|
||||
data = {"messages": [{"role": "user", "content": "Hello there"}]}
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
assert "Invalid protocol" in response.data["error"]
|
||||
|
||||
|
||||
@respx.mock
|
||||
def test_post_conversation_data_protocol(api_client, mock_openai_stream):
|
||||
"""Test posting messages to a conversation using the 'data' protocol."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/conversation/?protocol=data"
|
||||
data = {
|
||||
"messages": [
|
||||
{
|
||||
"id": "yuPoOuBkKA4FnKvk",
|
||||
"role": "user",
|
||||
"parts": [{"text": "Hello", "type": "text"}],
|
||||
"content": "Hello",
|
||||
"createdAt": "2025-07-03T15:22:17.105Z",
|
||||
}
|
||||
]
|
||||
}
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.get("Content-Type") == "text/event-stream"
|
||||
assert response.get("x-vercel-ai-data-stream") == "v1"
|
||||
assert response.streaming
|
||||
|
||||
# Wait for the streaming content to be fully received
|
||||
response_content = b"".join(response.streaming_content).decode("utf-8")
|
||||
assert response_content == (
|
||||
'0:"Hello"\n'
|
||||
'0:" there"\n'
|
||||
'd:{"finishReason": "stop", "usage": {"promptTokens": 0, "completionTokens": 0}}\n'
|
||||
)
|
||||
|
||||
assert mock_openai_stream.called
|
||||
|
||||
chat_conversation.refresh_from_db()
|
||||
assert chat_conversation.ui_messages == [
|
||||
{
|
||||
"content": "Hello",
|
||||
"createdAt": "2025-07-03T15:22:17.105Z",
|
||||
"id": "yuPoOuBkKA4FnKvk",
|
||||
"parts": [{"text": "Hello", "type": "text"}],
|
||||
"role": "user",
|
||||
}
|
||||
]
|
||||
|
||||
assert len(chat_conversation.messages) == 2
|
||||
assert chat_conversation.messages[0] == {
|
||||
"content": "Hello",
|
||||
"createdAt": "2025-07-03T15:22:17.105Z",
|
||||
"id": "yuPoOuBkKA4FnKvk",
|
||||
"parts": [{"text": "Hello", "type": "text"}],
|
||||
"role": "user",
|
||||
}
|
||||
|
||||
assert chat_conversation.messages[1].pop("id") # Remove ID for comparison
|
||||
assert chat_conversation.messages[1] == {
|
||||
"annotations": None,
|
||||
"content": "Hello there",
|
||||
"createdAt": None,
|
||||
"experimental_attachments": None,
|
||||
"parts": [{"text": "Hello there", "type": "text"}],
|
||||
"reasoning": None,
|
||||
"role": "assistant",
|
||||
"toolInvocations": None,
|
||||
}
|
||||
|
||||
|
||||
@respx.mock
|
||||
def test_post_conversation_text_protocol(api_client, mock_openai_stream):
|
||||
"""Test posting messages to a conversation using the 'text' protocol."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/conversation/?protocol=text"
|
||||
data = {
|
||||
"messages": [
|
||||
{
|
||||
"id": "yuPoOuBkKA4FnKvk",
|
||||
"role": "user",
|
||||
"parts": [{"text": "Hello", "type": "text"}],
|
||||
"content": "Hello",
|
||||
"createdAt": "2025-07-03T15:22:17.105Z",
|
||||
}
|
||||
]
|
||||
}
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.get("Content-Type") == "text/event-stream"
|
||||
assert response.streaming
|
||||
|
||||
response_content = b"".join(response.streaming_content).decode("utf-8")
|
||||
assert response_content == "Hello there"
|
||||
|
||||
assert mock_openai_stream.called
|
||||
|
||||
chat_conversation.refresh_from_db()
|
||||
assert chat_conversation.ui_messages == [
|
||||
{
|
||||
"content": "Hello",
|
||||
"createdAt": "2025-07-03T15:22:17.105Z",
|
||||
"id": "yuPoOuBkKA4FnKvk",
|
||||
"parts": [{"text": "Hello", "type": "text"}],
|
||||
"role": "user",
|
||||
}
|
||||
]
|
||||
|
||||
assert len(chat_conversation.messages) == 2
|
||||
assert chat_conversation.messages[0] == {
|
||||
"content": "Hello",
|
||||
"createdAt": "2025-07-03T15:22:17.105Z",
|
||||
"id": "yuPoOuBkKA4FnKvk",
|
||||
"parts": [{"text": "Hello", "type": "text"}],
|
||||
"role": "user",
|
||||
}
|
||||
|
||||
assert chat_conversation.messages[1].pop("id")
|
||||
assert chat_conversation.messages[1] == {
|
||||
"annotations": None,
|
||||
"content": "Hello there",
|
||||
"createdAt": None,
|
||||
"experimental_attachments": None,
|
||||
"parts": [{"text": "Hello there", "type": "text"}],
|
||||
"reasoning": None,
|
||||
"role": "assistant",
|
||||
"toolInvocations": None,
|
||||
}
|
||||
|
||||
|
||||
@respx.mock
|
||||
def test_post_conversation_with_image(api_client, mock_openai_stream_image):
|
||||
"""Ensure an image URL is correctly forwarded to the AI service."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/conversation/?protocol=data"
|
||||
|
||||
data = {
|
||||
"messages": [
|
||||
{
|
||||
"id": "7x3hLsq6rB3xp91T",
|
||||
"role": "user",
|
||||
"parts": [{"text": "Hello, what do you see on this picture?", "type": "text"}],
|
||||
"content": "Hello, what do you see on this picture?",
|
||||
"createdAt": "2025-07-07T15:52:27.822Z",
|
||||
"experimental_attachments": [
|
||||
{
|
||||
"url": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD",
|
||||
"name": "FELV-cat.jpg",
|
||||
"contentType": "image/jpeg",
|
||||
}
|
||||
],
|
||||
}
|
||||
]
|
||||
}
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.get("Content-Type") == "text/event-stream"
|
||||
assert response.get("x-vercel-ai-data-stream") == "v1"
|
||||
assert response.streaming
|
||||
|
||||
# Wait for the streaming content to be fully received
|
||||
response_content = b"".join(response.streaming_content).decode("utf-8")
|
||||
assert response_content == (
|
||||
'0:"I see a cat"\n'
|
||||
'0:" in the picture."\n'
|
||||
'd:{"finishReason": "stop", "usage": {"promptTokens": 0, "completionTokens": '
|
||||
"0}}\n"
|
||||
)
|
||||
|
||||
# --- Verify the outgoing HTTP request body contains the image ---
|
||||
request_sent = mock_openai_stream_image.calls[0].request
|
||||
body = json.loads(request_sent.content)
|
||||
|
||||
# Check the exact structure expected by the AI service
|
||||
assert body["messages"] == [
|
||||
{
|
||||
"content": "You are a helpful assistant. Escape formulas or any math "
|
||||
"notation between `$$`, like `$$x^2 + y^2 = z^2$$` or `$$C_l$$`. "
|
||||
"You can use Markdown to format your answers. ",
|
||||
"role": "system",
|
||||
},
|
||||
{
|
||||
"content": [
|
||||
{"text": "Hello, what do you see on this picture?", "type": "text"},
|
||||
{
|
||||
"image_url": {
|
||||
"detail": "auto",
|
||||
"url": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD",
|
||||
},
|
||||
"type": "image_url",
|
||||
},
|
||||
],
|
||||
"role": "user",
|
||||
},
|
||||
]
|
||||
assert body["model"] == "test-model"
|
||||
assert body["stream"] is True
|
||||
|
||||
|
||||
@respx.mock
|
||||
def test_post_conversation_tool_call(api_client, mock_openai_stream_tool, settings):
|
||||
"""Ensure tool calls are correctly forwarded and streamed back."""
|
||||
settings.AI_AGENT_TOOLS = ["get_current_weather"]
|
||||
|
||||
chat_conversation = ChatConversationFactory()
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/conversation/?protocol=data"
|
||||
|
||||
data = {
|
||||
"messages": [
|
||||
{
|
||||
"id": "tool-msg-1",
|
||||
"role": "user",
|
||||
"parts": [{"type": "text", "text": "Weather in Paris?"}],
|
||||
"content": "Weather in Paris?",
|
||||
"createdAt": "2025-07-18T12:00:00Z",
|
||||
}
|
||||
]
|
||||
}
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.get("Content-Type") == "text/event-stream"
|
||||
assert response.get("x-vercel-ai-data-stream") == "v1"
|
||||
assert response.streaming
|
||||
|
||||
# Wait for the streaming content to be fully received
|
||||
response_content = b"".join(response.streaming_content).decode("utf-8")
|
||||
assert response_content == (
|
||||
'9:{"toolCallId": "xLDcIljdsDrz0idal7tATWSMm2jhMj47", "toolName": '
|
||||
'"get_current_weather", "args": {"location": "Paris", "unit": "celsius"}}\n'
|
||||
'a:{"toolCallId": "xLDcIljdsDrz0idal7tATWSMm2jhMj47", "result": '
|
||||
"\"{'location': 'Paris', 'temperature': 22, 'unit': 'celsius'}\"}\n"
|
||||
'0:"The current weather in Paris is nice"\n'
|
||||
'd:{"finishReason": "stop", "usage": {"promptTokens": 0, "completionTokens": '
|
||||
"0}}\n"
|
||||
)
|
||||
|
||||
# --- Verify the outgoing HTTP request body ---
|
||||
request_sent = mock_openai_stream_tool.calls[0].request
|
||||
body = json.loads(request_sent.content)
|
||||
|
||||
assert body["messages"] == [
|
||||
{
|
||||
"content": "You are a helpful assistant. Escape formulas or any math "
|
||||
"notation between `$$`, like `$$x^2 + y^2 = z^2$$` or `$$C_l$$`. "
|
||||
"You can use Markdown to format your answers. ",
|
||||
"role": "system",
|
||||
},
|
||||
{"content": [{"text": "Weather in Paris?", "type": "text"}], "role": "user"},
|
||||
]
|
||||
|
||||
chat_conversation.refresh_from_db()
|
||||
assert chat_conversation.ui_messages == [
|
||||
{
|
||||
"content": "Weather in Paris?",
|
||||
"createdAt": "2025-07-18T12:00:00Z",
|
||||
"id": "tool-msg-1",
|
||||
"parts": [{"text": "Weather in Paris?", "type": "text"}],
|
||||
"role": "user",
|
||||
}
|
||||
]
|
||||
|
||||
assert len(chat_conversation.messages) == 2
|
||||
assert chat_conversation.messages[0] == {
|
||||
"content": "Weather in Paris?",
|
||||
"createdAt": "2025-07-18T12:00:00Z",
|
||||
"id": "tool-msg-1",
|
||||
"parts": [{"text": "Weather in Paris?", "type": "text"}],
|
||||
"role": "user",
|
||||
}
|
||||
|
||||
assert chat_conversation.messages[1].pop("id")
|
||||
assert chat_conversation.messages[1] == {
|
||||
"annotations": None,
|
||||
"content": "The current weather in Paris is nice",
|
||||
"createdAt": None,
|
||||
"experimental_attachments": None,
|
||||
"parts": [{"text": "The current weather in Paris is nice", "type": "text"}],
|
||||
"reasoning": None,
|
||||
"role": "assistant",
|
||||
"toolInvocations": None,
|
||||
}
|
||||
# To be fixed, because in real life, the tool invocation is added to the message...
|
||||
# assert chat_conversation.messages[1] == {
|
||||
# "annotations": None,
|
||||
# "content": "The weather is sunny",
|
||||
# "createdAt": None,
|
||||
# "experimental_attachments": None,
|
||||
# "parts": [
|
||||
# {
|
||||
# "type": "tool-invocation",
|
||||
# "toolInvocation": {
|
||||
# "args": {"unit": "celsius", "location": "Paris"},
|
||||
# "step": 0,
|
||||
# "state": "result",
|
||||
# "result": "{'location': 'Paris', 'temperature': 22, 'unit': 'celsius'}",
|
||||
# "toolName": "get_current_weather",
|
||||
# "toolCallId": "FCBUEY5SpcsaB72P9taJR7Bcx0bAuqOu",
|
||||
# },
|
||||
# },
|
||||
# {"text": "The weather is sunny", "type": "text"},
|
||||
# ],
|
||||
# "reasoning": None,
|
||||
# "role": "assistant",
|
||||
# "toolInvocations": [
|
||||
# {
|
||||
# "args": {"unit": "celsius", "location": "Paris"},
|
||||
# "step": 0,
|
||||
# "state": "result",
|
||||
# "result": "{'location': 'Paris', 'temperature': 22, 'unit': 'celsius'}",
|
||||
# "toolName": "get_current_weather",
|
||||
# "toolCallId": "FCBUEY5SpcsaB72P9taJR7Bcx0bAuqOu",
|
||||
# }
|
||||
# ],
|
||||
# }
|
||||
|
||||
|
||||
@respx.mock
|
||||
def test_post_conversation_tool_call_fails(api_client, mock_openai_stream_tool, settings):
|
||||
"""Ensure tool calls are correctly forwarded and streamed back when failing."""
|
||||
settings.AI_AGENT_TOOLS = []
|
||||
|
||||
chat_conversation = ChatConversationFactory()
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/conversation/?protocol=data"
|
||||
|
||||
data = {
|
||||
"messages": [
|
||||
{
|
||||
"id": "tool-msg-1",
|
||||
"role": "user",
|
||||
"parts": [{"type": "text", "text": "Weather in Paris?"}],
|
||||
"content": "Weather in Paris?",
|
||||
"createdAt": "2025-07-18T12:00:00Z",
|
||||
}
|
||||
]
|
||||
}
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.get("Content-Type") == "text/event-stream"
|
||||
assert response.get("x-vercel-ai-data-stream") == "v1"
|
||||
assert response.streaming
|
||||
|
||||
# Wait for the streaming content to be fully received
|
||||
response_content = b"".join(response.streaming_content).decode("utf-8")
|
||||
assert response_content == (
|
||||
'3:"Tool get_current_weather not found in agent Conversations Assistant"\n'
|
||||
'd:{"finishReason": "error", "usage": {"promptTokens": 0, "completionTokens": '
|
||||
"0}}\n"
|
||||
)
|
||||
|
||||
# --- Verify the outgoing HTTP request body ---
|
||||
request_sent = mock_openai_stream_tool.calls[0].request
|
||||
body = json.loads(request_sent.content)
|
||||
|
||||
assert body["messages"] == [
|
||||
{
|
||||
"content": "You are a helpful assistant. Escape formulas or any math "
|
||||
"notation between `$$`, like `$$x^2 + y^2 = z^2$$` or `$$C_l$$`. "
|
||||
"You can use Markdown to format your answers. ",
|
||||
"role": "system",
|
||||
},
|
||||
{"content": [{"text": "Weather in Paris?", "type": "text"}], "role": "user"},
|
||||
]
|
||||
|
||||
chat_conversation.refresh_from_db()
|
||||
assert chat_conversation.ui_messages == [
|
||||
{
|
||||
"content": "Weather in Paris?",
|
||||
"createdAt": "2025-07-18T12:00:00Z",
|
||||
"id": "tool-msg-1",
|
||||
"parts": [{"text": "Weather in Paris?", "type": "text"}],
|
||||
"role": "user",
|
||||
}
|
||||
]
|
||||
|
||||
assert len(chat_conversation.messages) == 1
|
||||
assert chat_conversation.messages[0] == {
|
||||
"content": "Weather in Paris?",
|
||||
"createdAt": "2025-07-18T12:00:00Z",
|
||||
"id": "tool-msg-1",
|
||||
"parts": [{"text": "Weather in Paris?", "type": "text"}],
|
||||
"role": "user",
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Unit tests for chat conversation creation in the chat API view."""
|
||||
|
||||
import pytest
|
||||
from rest_framework import status
|
||||
|
||||
from core.factories import UserFactory
|
||||
|
||||
from chat.models import ChatConversation
|
||||
|
||||
pytestmark = pytest.mark.django_db
|
||||
|
||||
|
||||
def test_create_conversation(api_client):
|
||||
"""Test creating a new chat conversation as an authenticated user."""
|
||||
user = UserFactory(sub="testuser", email="test@example.com")
|
||||
url = "/api/v1.0/chats/"
|
||||
data = {
|
||||
"title": "New Conversation",
|
||||
}
|
||||
api_client.force_login(user)
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_201_CREATED
|
||||
assert response.data["title"] == "New Conversation"
|
||||
assert response.data["messages"] == []
|
||||
|
||||
# Verify in database
|
||||
conversation = ChatConversation.objects.get(id=response.data["id"])
|
||||
assert conversation.owner == user
|
||||
assert conversation.title == "New Conversation"
|
||||
|
||||
|
||||
def test_create_conversation_other_owner(api_client):
|
||||
"""Test that a user cannot assign another user as the owner of a conversation."""
|
||||
other_user = UserFactory()
|
||||
|
||||
user = UserFactory()
|
||||
url = "/api/v1.0/chats/"
|
||||
data = {
|
||||
"title": "New Conversation",
|
||||
"owner": str(other_user.pk), # Attempt to set another user as owner
|
||||
}
|
||||
api_client.force_login(user)
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_201_CREATED
|
||||
|
||||
# Verify in database
|
||||
conversation = ChatConversation.objects.get(id=response.data["id"])
|
||||
assert conversation.owner == user
|
||||
assert conversation.title == "New Conversation"
|
||||
|
||||
|
||||
def test_create_conversation_anonymous(api_client):
|
||||
"""Test creating a conversation as an anonymous user returns a 401 error."""
|
||||
url = "/api/v1.0/chats/"
|
||||
data = {
|
||||
"title": "New Conversation",
|
||||
}
|
||||
response = api_client.post(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_401_UNAUTHORIZED
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Unit tests for chat conversation deletion in the chat API view."""
|
||||
|
||||
import pytest
|
||||
from rest_framework import status
|
||||
|
||||
from core.factories import UserFactory
|
||||
|
||||
from chat.factories import ChatConversationFactory
|
||||
from chat.models import ChatConversation
|
||||
|
||||
pytestmark = pytest.mark.django_db
|
||||
|
||||
|
||||
def test_delete_conversation(api_client):
|
||||
"""Test deleting a chat conversation as the owner."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
response = api_client.delete(f"/api/v1.0/chats/{chat_conversation.pk}/")
|
||||
|
||||
assert response.status_code == status.HTTP_204_NO_CONTENT
|
||||
|
||||
# Verify deletion in database
|
||||
assert not ChatConversation.objects.filter(id=chat_conversation.pk).exists()
|
||||
|
||||
|
||||
def test_delete_other_user_conversation_fails(api_client):
|
||||
"""Test that deleting another user's conversation returns a 404 error."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
other_user = UserFactory()
|
||||
api_client.force_login(other_user)
|
||||
response = api_client.delete(f"/api/v1.0/chats/{chat_conversation.pk}/")
|
||||
|
||||
assert response.status_code == status.HTTP_404_NOT_FOUND
|
||||
|
||||
# Verify conversation still exists
|
||||
assert ChatConversation.objects.filter(id=chat_conversation.pk).exists()
|
||||
|
||||
|
||||
def test_delete_conversation_anonymous(api_client):
|
||||
"""Test deleting a conversation as an anonymous user returns a 401 error."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
response = api_client.delete(f"/api/v1.0/chats/{chat_conversation.pk}/")
|
||||
|
||||
assert response.status_code == status.HTTP_401_UNAUTHORIZED
|
||||
|
||||
# Verify conversation still exists
|
||||
assert ChatConversation.objects.filter(id=chat_conversation.pk).exists()
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Unit tests for listing chat conversations in the chat API view."""
|
||||
|
||||
import pytest
|
||||
from rest_framework import status
|
||||
|
||||
from core.factories import UserFactory
|
||||
|
||||
from chat.factories import ChatConversationFactory
|
||||
from chat.models import ChatConversation
|
||||
|
||||
pytestmark = pytest.mark.django_db
|
||||
|
||||
|
||||
def test_list_conversations(api_client):
|
||||
"""Test retrieving the list of chat conversations for an authenticated user."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
url = "/api/v1.0/chats/"
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
response = api_client.get(url)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.data["results"][0]["id"] == str(chat_conversation.pk)
|
||||
assert response.data["results"][0]["title"] == chat_conversation.title
|
||||
|
||||
|
||||
def test_list_conversations_empty(api_client):
|
||||
"""Test retrieving an empty list for a user with no conversations."""
|
||||
other_user = UserFactory()
|
||||
url = "/api/v1.0/chats/"
|
||||
api_client.force_login(other_user)
|
||||
response = api_client.get(url)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert len(response.data["results"]) == 0
|
||||
|
||||
|
||||
def test_list_conversations_anonymous(api_client):
|
||||
"""Test listing conversations as an anonymous user returns a 401 error."""
|
||||
url = "/api/v1.0/chats/"
|
||||
response = api_client.get(url)
|
||||
|
||||
assert response.status_code == status.HTTP_401_UNAUTHORIZED
|
||||
|
||||
|
||||
def test_filter_conversations_by_title(api_client):
|
||||
"""Test filtering conversations by title substring."""
|
||||
user = UserFactory(sub="testuser", email="test@example.com")
|
||||
ChatConversation.objects.create(
|
||||
owner=user,
|
||||
title="Test Conversation",
|
||||
ui_messages=[{"role": "user", "content": "Test message"}],
|
||||
)
|
||||
ChatConversation.objects.create(owner=user, title="Another Conversation", ui_messages=[])
|
||||
url = "/api/v1.0/chats/?title=Test"
|
||||
api_client.force_login(user)
|
||||
response = api_client.get(url)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert len(response.data["results"]) == 1
|
||||
assert response.data["results"][0]["title"] == "Test Conversation"
|
||||
|
||||
|
||||
def test_ordering_conversations(api_client):
|
||||
"""Test ordering conversations by creation date."""
|
||||
user = UserFactory(sub="testuser", email="test@example.com")
|
||||
conv1 = ChatConversation.objects.create(owner=user, title="First Conversation", ui_messages=[])
|
||||
conv2 = ChatConversation.objects.create(owner=user, title="Second Conversation", ui_messages=[])
|
||||
url = "/api/v1.0/chats/"
|
||||
api_client.force_login(user)
|
||||
response = api_client.get(url)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.data["results"][0]["id"] == str(conv2.id)
|
||||
assert response.data["results"][1]["id"] == str(conv1.id)
|
||||
|
||||
url = "/api/v1.0/chats/?ordering=created_at"
|
||||
response = api_client.get(url)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.data["results"][0]["id"] == str(conv1.id)
|
||||
assert response.data["results"][1]["id"] == str(conv2.id)
|
||||
@@ -0,0 +1,45 @@
|
||||
"""Unit tests for retrieving chat conversations in the chat API view."""
|
||||
|
||||
import pytest
|
||||
from rest_framework import status
|
||||
|
||||
from core.factories import UserFactory
|
||||
|
||||
from chat.factories import ChatConversationFactory
|
||||
|
||||
pytestmark = pytest.mark.django_db
|
||||
|
||||
|
||||
def test_retrieve_conversation(api_client):
|
||||
"""Test retrieving a single chat conversation as the owner."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/"
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
response = api_client.get(url)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.data["id"] == str(chat_conversation.pk)
|
||||
assert response.data["title"] == chat_conversation.title
|
||||
|
||||
|
||||
def test_retrieve_other_user_conversation_fails(api_client):
|
||||
"""Test that retrieving another user's conversation returns a 404 error."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
|
||||
other_user = UserFactory()
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/"
|
||||
api_client.force_login(other_user)
|
||||
response = api_client.get(url)
|
||||
|
||||
assert response.status_code == status.HTTP_404_NOT_FOUND
|
||||
|
||||
|
||||
def test_retrieve_conversation_anonymous(api_client):
|
||||
"""Test retrieving a conversation as an anonymous user returns a 401 error."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/"
|
||||
response = api_client.get(url)
|
||||
|
||||
assert response.status_code == status.HTTP_401_UNAUTHORIZED
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Unit tests for updating chat conversations in the chat API view."""
|
||||
|
||||
import pytest
|
||||
from rest_framework import status
|
||||
|
||||
from core.factories import UserFactory
|
||||
|
||||
from chat.factories import ChatConversationFactory
|
||||
from chat.models import ChatConversation
|
||||
|
||||
pytestmark = pytest.mark.django_db
|
||||
|
||||
|
||||
def test_update_conversation(api_client):
|
||||
"""Test updating a chat conversation as the owner."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/"
|
||||
data = {"title": "Updated Title"}
|
||||
api_client.force_login(chat_conversation.owner)
|
||||
response = api_client.put(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.data["title"] == "Updated Title"
|
||||
|
||||
# Verify in database
|
||||
conversation = ChatConversation.objects.get(id=chat_conversation.pk)
|
||||
assert conversation.title == "Updated Title"
|
||||
|
||||
|
||||
def test_update_conversation_anonymous(api_client):
|
||||
"""Test updating a conversation as an anonymous user returns a 401 error."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/"
|
||||
data = {"title": "Updated Title"}
|
||||
response = api_client.put(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_401_UNAUTHORIZED
|
||||
|
||||
|
||||
def test_update_other_user_conversation_fails(api_client):
|
||||
"""Test that updating another user's conversation returns a 404 error."""
|
||||
chat_conversation = ChatConversationFactory()
|
||||
|
||||
other_user = UserFactory()
|
||||
url = f"/api/v1.0/chats/{chat_conversation.pk}/"
|
||||
data = {"title": "Updated By Other User"}
|
||||
api_client.force_login(other_user)
|
||||
response = api_client.put(url, data, format="json")
|
||||
|
||||
assert response.status_code == status.HTTP_404_NOT_FOUND
|
||||
@@ -0,0 +1,16 @@
|
||||
"""Tools for the chat agent."""
|
||||
|
||||
from agents import FunctionTool
|
||||
|
||||
from .fake_current_weather import agent_get_current_weather
|
||||
from .web_search_tavily import agent_web_search_tavily
|
||||
|
||||
|
||||
def get_tool_by_name(name: str) -> FunctionTool:
|
||||
"""Get a tool by its name."""
|
||||
tool_dict = {
|
||||
"get_current_weather": agent_get_current_weather,
|
||||
"tavily_web_search": agent_web_search_tavily,
|
||||
}
|
||||
|
||||
return tool_dict[name] # will raise on purpose if name is not found
|
||||
@@ -0,0 +1,39 @@
|
||||
"""Fake weather tool for the chat agent."""
|
||||
|
||||
from agents import function_tool
|
||||
from openai.types.chat import ChatCompletionToolParam
|
||||
from openai.types.shared_params import FunctionDefinition
|
||||
|
||||
current_weather = ChatCompletionToolParam(
|
||||
type="function",
|
||||
function=FunctionDefinition(
|
||||
name="get_current_weather",
|
||||
description="Get the current weather in a given location",
|
||||
parameters={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"location": {
|
||||
"type": "string",
|
||||
"description": "The city and state, e.g. San Francisco, CA",
|
||||
},
|
||||
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
|
||||
},
|
||||
"required": ["location", "unit"],
|
||||
},
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def get_current_weather(location: str, unit: str):
|
||||
"""Get the current weather in a given location."""
|
||||
return {
|
||||
"location": location,
|
||||
"temperature": 22 if unit == "celsius" else 72,
|
||||
"unit": unit,
|
||||
}
|
||||
|
||||
|
||||
@function_tool(name_override="get_current_weather")
|
||||
def agent_get_current_weather(location: str, unit: str) -> dict:
|
||||
"""Get the current weather in a given location."""
|
||||
return get_current_weather(location, unit)
|
||||
@@ -0,0 +1,53 @@
|
||||
"""Web search tool using Tavily for the chat agent."""
|
||||
|
||||
from django.conf import settings
|
||||
|
||||
import requests
|
||||
from agents import function_tool
|
||||
|
||||
|
||||
def tavily_web_search(query: str) -> list[dict]:
|
||||
"""
|
||||
Search the web for up-to-date information
|
||||
|
||||
Args:
|
||||
query (str): The query to search for.
|
||||
|
||||
Returns:
|
||||
list[dict]: A list of search results, each represented as a dictionary.
|
||||
"""
|
||||
url = "https://api.tavily.com/search"
|
||||
data = {
|
||||
"query": query,
|
||||
"api_key": settings.TAVILY_API_KEY,
|
||||
"max_results": settings.TAVILY_MAX_RESULTS,
|
||||
}
|
||||
response = requests.post(url, json=data, timeout=settings.TAVILY_API_TIMEOUT)
|
||||
response.raise_for_status()
|
||||
|
||||
json_response = response.json()
|
||||
|
||||
raw_search_results = json_response.get("results", [])
|
||||
|
||||
return [
|
||||
{
|
||||
"link": result["url"],
|
||||
"title": result.get("title", ""),
|
||||
"snippet": result.get("content"),
|
||||
}
|
||||
for result in raw_search_results
|
||||
]
|
||||
|
||||
|
||||
@function_tool(name_override="tavily_web_search")
|
||||
def agent_web_search_tavily(query: str) -> list[dict]:
|
||||
"""
|
||||
Search the web for up-to-date information
|
||||
|
||||
Args:
|
||||
query (str): The query to search for.
|
||||
|
||||
Returns:
|
||||
list[dict]: A list of search results, each represented as a dictionary.
|
||||
"""
|
||||
return tavily_web_search(query)
|
||||
@@ -0,0 +1,126 @@
|
||||
"""Chat API implementation."""
|
||||
|
||||
import logging
|
||||
|
||||
from django.conf import settings
|
||||
from django.http import StreamingHttpResponse
|
||||
|
||||
from rest_framework import decorators, filters, mixins, permissions, status, viewsets
|
||||
from rest_framework.response import Response
|
||||
|
||||
from core.api.viewsets import Pagination
|
||||
from core.filters import remove_accents
|
||||
|
||||
from chat import models, serializers
|
||||
from chat.ai_sdk_types import UIMessage
|
||||
from chat.clients.openai import AIAgentService
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ChatConversationFilter(filters.BaseFilterBackend):
|
||||
"""Filter conversation."""
|
||||
|
||||
def filter_queryset(self, request, queryset, view):
|
||||
"""Filter conversation by title."""
|
||||
if title := request.GET.get("title"):
|
||||
queryset = queryset.filter(title__unaccent__icontains=remove_accents(title))
|
||||
return queryset
|
||||
|
||||
|
||||
class ChatViewSet(
|
||||
mixins.CreateModelMixin,
|
||||
mixins.RetrieveModelMixin,
|
||||
mixins.ListModelMixin,
|
||||
mixins.DestroyModelMixin,
|
||||
mixins.UpdateModelMixin,
|
||||
viewsets.GenericViewSet,
|
||||
):
|
||||
"""ViewSet for managing chat conversations.
|
||||
|
||||
Provides endpoints to create, retrieve, list, update, and delete chat conversations.
|
||||
|
||||
The chat conversations are filtered by the authenticated user.
|
||||
The `post_conversation` action allows sending messages to the chat and receiving a
|
||||
streaming response with "data" formatted for Vercel AI SDK
|
||||
see https://ai-sdk.dev/docs/ai-sdk-ui/stream-protocol#data-stream-protocol.
|
||||
"""
|
||||
|
||||
pagination_class = Pagination
|
||||
permission_classes = [
|
||||
permissions.IsAuthenticated,
|
||||
]
|
||||
serializer_class = serializers.ChatConversationSerializer
|
||||
filter_backends = [filters.OrderingFilter, ChatConversationFilter]
|
||||
ordering = ["-created_at"]
|
||||
ordering_fields = ["created_at", "updated_at"]
|
||||
|
||||
def get_queryset(self):
|
||||
"""Return the queryset for the chat conversations."""
|
||||
return models.ChatConversation.objects.filter(owner=self.request.user)
|
||||
|
||||
@decorators.action(
|
||||
methods=["post"],
|
||||
detail=True,
|
||||
url_path="conversation",
|
||||
url_name="conversation",
|
||||
)
|
||||
def post_conversation(self, request, pk): # pylint: disable=unused-argument
|
||||
"""Handle POST requests to the chat endpoint.
|
||||
|
||||
Args:
|
||||
request: The HTTP request object containing:
|
||||
- messages: List of message objects with role and content
|
||||
- protocol: Optional protocol parameter ('text' or 'data', defaults to 'data')
|
||||
pk: The primary key of the chat conversation.
|
||||
|
||||
Returns:
|
||||
StreamingHttpResponse: A streaming response containing the chat completion
|
||||
"""
|
||||
protocol = request.query_params.get("protocol", "data")
|
||||
if protocol not in ["text", "data"]:
|
||||
return Response(
|
||||
{"error": "Invalid protocol. Must be 'text' or 'data'"},
|
||||
status=status.HTTP_400_BAD_REQUEST,
|
||||
)
|
||||
|
||||
logger.info("Received messages: %s", request.data.get("messages", []))
|
||||
|
||||
if settings.ML_FLOW_TRACKING_URI:
|
||||
# Set up MLflow experiment, don't import it globally to avoid issues
|
||||
# when running management commands when MLflow is not started
|
||||
import mlflow # pylint: disable=import-outside-toplevel
|
||||
|
||||
mlflow.set_tracking_uri(settings.ML_FLOW_TRACKING_URI)
|
||||
mlflow.set_experiment(settings.ML_FLOW_EXPERIMENT_NAME)
|
||||
|
||||
# Enable automatic tracing for all OpenAI API calls
|
||||
mlflow.openai.autolog()
|
||||
|
||||
# Warning: the messages should be stored more securely in production
|
||||
conversation = self.get_object()
|
||||
conversation.ui_messages = request.data.get("messages", [])
|
||||
conversation.save()
|
||||
|
||||
messages = [UIMessage(**msg) for msg in request.data.get("messages", [])]
|
||||
|
||||
logger.info("Received messages: %s", messages)
|
||||
logger.info("Using protocol: %s", protocol)
|
||||
|
||||
if not messages:
|
||||
return Response({"error": "No messages provided"}, status=status.HTTP_400_BAD_REQUEST)
|
||||
|
||||
ai_service = AIAgentService(conversation=conversation)
|
||||
if protocol == "data":
|
||||
streaming_content = ai_service.stream_data(messages)
|
||||
else:
|
||||
streaming_content = ai_service.stream_text(messages)
|
||||
|
||||
response = StreamingHttpResponse(
|
||||
streaming_content,
|
||||
content_type="text/event-stream",
|
||||
headers={
|
||||
"x-vercel-ai-data-stream": "v1", # This header is used for Vercel AI streaming,
|
||||
},
|
||||
)
|
||||
return response
|
||||
@@ -0,0 +1 @@
|
||||
"""Conversations package"""
|
||||
@@ -0,0 +1,121 @@
|
||||
{
|
||||
"footer": {
|
||||
"default": {
|
||||
"logo": {
|
||||
"src": "/assets/icon-docs.svg",
|
||||
"width": "54px",
|
||||
"alt": "Docs Logo",
|
||||
"withTitle": true
|
||||
},
|
||||
"externalLinks": [
|
||||
{
|
||||
"label": "Github",
|
||||
"href": "https://github.com/suitenumerique/conversations/"
|
||||
},
|
||||
{
|
||||
"label": "DINUM",
|
||||
"href": "https://www.numerique.gouv.fr/dinum/"
|
||||
}
|
||||
],
|
||||
"bottomInformation": {
|
||||
"label": "Unless otherwise stated, all content on this site is under",
|
||||
"link": {
|
||||
"label": "licence etalab-2.0",
|
||||
"href": "https://github.com/etalab/licence-ouverte/blob/master/LO.md"
|
||||
}
|
||||
}
|
||||
},
|
||||
"en": {
|
||||
"legalLinks": [
|
||||
{
|
||||
"label": "Legal Notice",
|
||||
"href": "#"
|
||||
},
|
||||
{
|
||||
"label": "Personal data and cookies",
|
||||
"href": "#"
|
||||
},
|
||||
{
|
||||
"label": "Accessibility",
|
||||
"href": "#"
|
||||
}
|
||||
],
|
||||
"bottomInformation": {
|
||||
"label": "Unless otherwise stated, all content on this site is under",
|
||||
"link": {
|
||||
"label": "licence MIT",
|
||||
"href": "https://github.com/suitenumerique/conversations/blob/main/LICENSE"
|
||||
}
|
||||
}
|
||||
},
|
||||
"fr": {
|
||||
"legalLinks": [
|
||||
{
|
||||
"label": "Mentions légales",
|
||||
"href": "#"
|
||||
},
|
||||
{
|
||||
"label": "Données personnelles et cookies",
|
||||
"href": "#"
|
||||
},
|
||||
{
|
||||
"label": "Accessibilité",
|
||||
"href": "#"
|
||||
}
|
||||
],
|
||||
"bottomInformation": {
|
||||
"label": "Sauf mention contraire, tout le contenu de ce site est sous",
|
||||
"link": {
|
||||
"label": "licence MIT",
|
||||
"href": "https://github.com/suitenumerique/conversations/blob/main/LICENSE"
|
||||
}
|
||||
}
|
||||
},
|
||||
"de": {
|
||||
"legalLinks": [
|
||||
{
|
||||
"label": "Impressum",
|
||||
"href": "#"
|
||||
},
|
||||
{
|
||||
"label": "Personenbezogene Daten und Cookies",
|
||||
"href": "#"
|
||||
},
|
||||
{
|
||||
"label": "Barrierefreiheit",
|
||||
"href": "#"
|
||||
}
|
||||
],
|
||||
"bottomInformation": {
|
||||
"label": "Sofern nicht anders angegeben, steht der gesamte Inhalt dieser Website unter",
|
||||
"link": {
|
||||
"label": "licence MIT",
|
||||
"href": "https://github.com/suitenumerique/conversations/blob/main/LICENSE"
|
||||
}
|
||||
}
|
||||
},
|
||||
"nl": {
|
||||
"legalLinks": [
|
||||
{
|
||||
"label": "Wettelijke bepalingen",
|
||||
"href": "#"
|
||||
},
|
||||
{
|
||||
"label": "Persoonlijke gegevens en cookies",
|
||||
"href": "#"
|
||||
},
|
||||
{
|
||||
"label": "Toegankelijkheid",
|
||||
"href": "#"
|
||||
}
|
||||
],
|
||||
"bottomInformation": {
|
||||
"label": "Tenzij anders vermeld, is alle inhoud van deze site ondergebracht onder",
|
||||
"link": {
|
||||
"label": "licence MIT",
|
||||
"href": "https://github.com/suitenumerique/conversations/blob/main/LICENSE"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Executable
+820
@@ -0,0 +1,820 @@
|
||||
"""
|
||||
Django settings for conversations project.
|
||||
|
||||
Generated by 'django-admin startproject' using Django 3.1.5.
|
||||
|
||||
For more information on this file, see
|
||||
https://docs.djangoproject.com/en/3.1/topics/settings/
|
||||
|
||||
For the full list of settings and their values, see
|
||||
https://docs.djangoproject.com/en/3.1/ref/settings/
|
||||
"""
|
||||
|
||||
import os
|
||||
import tomllib
|
||||
from socket import gethostbyname, gethostname
|
||||
|
||||
import sentry_sdk
|
||||
from configurations import Configuration, values
|
||||
from sentry_sdk.integrations.django import DjangoIntegration
|
||||
from sentry_sdk.integrations.logging import ignore_logger
|
||||
|
||||
# Build paths inside the project like this: BASE_DIR / 'subdir'.
|
||||
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
DATA_DIR = os.getenv("DATA_DIR", os.path.join("/", "data"))
|
||||
|
||||
|
||||
def get_release():
|
||||
"""
|
||||
Get the current release of the application
|
||||
"""
|
||||
try:
|
||||
with open(os.path.join(BASE_DIR, "pyproject.toml"), "rb") as f:
|
||||
pyproject_data = tomllib.load(f)
|
||||
return pyproject_data["project"]["version"]
|
||||
except (FileNotFoundError, KeyError):
|
||||
return "NA" # Default: not available
|
||||
|
||||
|
||||
class Base(Configuration):
|
||||
"""
|
||||
This is the base configuration every configuration (aka environment) should inherit from. It
|
||||
is recommended to configure third-party applications by creating a configuration mixins in
|
||||
./configurations and compose the Base configuration with those mixins.
|
||||
|
||||
It depends on an environment variable that SHOULD be defined:
|
||||
|
||||
* DJANGO_SECRET_KEY
|
||||
|
||||
You may also want to override default configuration by setting the following environment
|
||||
variables:
|
||||
|
||||
* SENTRY_DSN
|
||||
* DB_NAME
|
||||
* DB_HOST
|
||||
* DB_PASSWORD
|
||||
* DB_USER
|
||||
"""
|
||||
|
||||
DEBUG = False
|
||||
USE_SWAGGER = False
|
||||
|
||||
API_VERSION = "v1.0"
|
||||
|
||||
# Security
|
||||
ALLOWED_HOSTS = values.ListValue([])
|
||||
SECRET_KEY = values.Value(None)
|
||||
SERVER_TO_SERVER_API_TOKENS = values.ListValue([])
|
||||
|
||||
# Application definition
|
||||
ROOT_URLCONF = "conversations.urls"
|
||||
WSGI_APPLICATION = "conversations.wsgi.application"
|
||||
|
||||
# Database
|
||||
DATABASES = {
|
||||
"default": {
|
||||
"ENGINE": values.Value(
|
||||
"django.db.backends.postgresql_psycopg2",
|
||||
environ_name="DB_ENGINE",
|
||||
environ_prefix=None,
|
||||
),
|
||||
"NAME": values.Value("conversations", environ_name="DB_NAME", environ_prefix=None),
|
||||
"USER": values.Value("dinum", environ_name="DB_USER", environ_prefix=None),
|
||||
"PASSWORD": values.Value("pass", environ_name="DB_PASSWORD", environ_prefix=None),
|
||||
"HOST": values.Value("localhost", environ_name="DB_HOST", environ_prefix=None),
|
||||
"PORT": values.Value(5432, environ_name="DB_PORT", environ_prefix=None),
|
||||
}
|
||||
}
|
||||
DEFAULT_AUTO_FIELD = "django.db.models.AutoField"
|
||||
|
||||
# Static files (CSS, JavaScript, Images)
|
||||
STATIC_URL = "/static/"
|
||||
STATIC_ROOT = os.path.join(DATA_DIR, "static")
|
||||
MEDIA_URL = "/media/"
|
||||
MEDIA_ROOT = os.path.join(DATA_DIR, "media")
|
||||
MEDIA_BASE_URL = values.Value(None, environ_name="MEDIA_BASE_URL", environ_prefix=None)
|
||||
|
||||
SITE_ID = 1
|
||||
|
||||
STORAGES = {
|
||||
"default": {
|
||||
"BACKEND": "storages.backends.s3.S3Storage",
|
||||
},
|
||||
"staticfiles": {
|
||||
"BACKEND": values.Value(
|
||||
"whitenoise.storage.CompressedManifestStaticFilesStorage",
|
||||
environ_name="STORAGES_STATICFILES_BACKEND",
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
# Media
|
||||
AWS_S3_ENDPOINT_URL = values.Value(environ_name="AWS_S3_ENDPOINT_URL", environ_prefix=None)
|
||||
AWS_S3_ACCESS_KEY_ID = values.Value(environ_name="AWS_S3_ACCESS_KEY_ID", environ_prefix=None)
|
||||
AWS_S3_SECRET_ACCESS_KEY = values.Value(
|
||||
environ_name="AWS_S3_SECRET_ACCESS_KEY", environ_prefix=None
|
||||
)
|
||||
AWS_S3_REGION_NAME = values.Value(environ_name="AWS_S3_REGION_NAME", environ_prefix=None)
|
||||
AWS_STORAGE_BUCKET_NAME = values.Value(
|
||||
"conversations-media-storage",
|
||||
environ_name="AWS_STORAGE_BUCKET_NAME",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
# Internationalization
|
||||
# https://docs.djangoproject.com/en/3.1/topics/i18n/
|
||||
|
||||
# Languages
|
||||
LANGUAGE_CODE = values.Value("en-us")
|
||||
# cookie & language is set from frontend
|
||||
LANGUAGE_COOKIE_NAME = "conversation_language"
|
||||
LANGUAGE_COOKIE_PATH = "/"
|
||||
|
||||
DRF_NESTED_MULTIPART_PARSER = {
|
||||
# output of parser is converted to querydict
|
||||
# if is set to False, dict python is returned
|
||||
"querydict": False,
|
||||
}
|
||||
|
||||
# Careful! Languages should be ordered by priority, as this tuple is used to get
|
||||
# fallback/default languages throughout the app.
|
||||
LANGUAGES = values.SingleNestedTupleValue(
|
||||
(
|
||||
("en-us", "English"),
|
||||
("fr-fr", "Français"),
|
||||
("de-de", "Deutsch"),
|
||||
("nl-nl", "Nederlands"),
|
||||
("es-es", "Español"),
|
||||
)
|
||||
)
|
||||
|
||||
LOCALE_PATHS = (os.path.join(BASE_DIR, "locale"),)
|
||||
|
||||
TIME_ZONE = "UTC"
|
||||
USE_I18N = True
|
||||
USE_TZ = True
|
||||
|
||||
# Templates
|
||||
TEMPLATES = [
|
||||
{
|
||||
"BACKEND": "django.template.backends.django.DjangoTemplates",
|
||||
"DIRS": [os.path.join(BASE_DIR, "templates")],
|
||||
"OPTIONS": {
|
||||
"context_processors": [
|
||||
"django.contrib.auth.context_processors.auth",
|
||||
"django.contrib.messages.context_processors.messages",
|
||||
"django.template.context_processors.csrf",
|
||||
"django.template.context_processors.debug",
|
||||
"django.template.context_processors.i18n",
|
||||
"django.template.context_processors.media",
|
||||
"django.template.context_processors.request",
|
||||
"django.template.context_processors.tz",
|
||||
],
|
||||
"loaders": [
|
||||
"django.template.loaders.filesystem.Loader",
|
||||
"django.template.loaders.app_directories.Loader",
|
||||
],
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
MIDDLEWARE = [
|
||||
"django.middleware.security.SecurityMiddleware",
|
||||
"whitenoise.middleware.WhiteNoiseMiddleware",
|
||||
"django.contrib.sessions.middleware.SessionMiddleware",
|
||||
"django.middleware.locale.LocaleMiddleware",
|
||||
"django.middleware.clickjacking.XFrameOptionsMiddleware",
|
||||
"corsheaders.middleware.CorsMiddleware",
|
||||
"django.middleware.common.CommonMiddleware",
|
||||
"django.middleware.csrf.CsrfViewMiddleware",
|
||||
"django.contrib.auth.middleware.AuthenticationMiddleware",
|
||||
"django.contrib.messages.middleware.MessageMiddleware",
|
||||
"dockerflow.django.middleware.DockerflowMiddleware",
|
||||
]
|
||||
|
||||
AUTHENTICATION_BACKENDS = [
|
||||
"django.contrib.auth.backends.ModelBackend",
|
||||
"core.authentication.backends.OIDCAuthenticationBackend",
|
||||
]
|
||||
|
||||
# Django applications from the highest priority to the lowest
|
||||
INSTALLED_APPS = [
|
||||
"chat",
|
||||
"core",
|
||||
"demo",
|
||||
"drf_spectacular",
|
||||
# Third party apps
|
||||
"corsheaders",
|
||||
"django_filters",
|
||||
"dockerflow.django",
|
||||
"rest_framework",
|
||||
"parler",
|
||||
"easy_thumbnails",
|
||||
# Django
|
||||
"django.contrib.admin",
|
||||
"django.contrib.auth",
|
||||
"django.contrib.contenttypes",
|
||||
"django.contrib.postgres",
|
||||
"django.contrib.sessions",
|
||||
"django.contrib.sites",
|
||||
"django.contrib.messages",
|
||||
"django.contrib.staticfiles",
|
||||
# OIDC third party
|
||||
"mozilla_django_oidc",
|
||||
]
|
||||
|
||||
# Cache
|
||||
CACHES = {
|
||||
"default": {"BACKEND": "django.core.cache.backends.locmem.LocMemCache"},
|
||||
}
|
||||
|
||||
REST_FRAMEWORK = {
|
||||
"DEFAULT_AUTHENTICATION_CLASSES": (
|
||||
"mozilla_django_oidc.contrib.drf.OIDCAuthentication",
|
||||
"rest_framework.authentication.SessionAuthentication",
|
||||
),
|
||||
"DEFAULT_PARSER_CLASSES": [
|
||||
"rest_framework.parsers.JSONParser",
|
||||
"nested_multipart_parser.drf.DrfNestedParser",
|
||||
],
|
||||
"DEFAULT_RENDERER_CLASSES": [
|
||||
# 🔒️ Disable BrowsableAPIRenderer which provides forms allowing a user to
|
||||
# see all the data in the database (ie a serializer with a ForeignKey field
|
||||
# will generate a form with a field with all possible values of the FK).
|
||||
"rest_framework.renderers.JSONRenderer",
|
||||
],
|
||||
"EXCEPTION_HANDLER": "core.api.exception_handler",
|
||||
"DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.PageNumberPagination",
|
||||
"PAGE_SIZE": 20,
|
||||
"DEFAULT_VERSIONING_CLASS": "rest_framework.versioning.URLPathVersioning",
|
||||
"DEFAULT_SCHEMA_CLASS": "drf_spectacular.openapi.AutoSchema",
|
||||
"DEFAULT_THROTTLE_RATES": {
|
||||
"user_list_sustained": values.Value(
|
||||
default="180/hour",
|
||||
environ_name="API_USERS_LIST_THROTTLE_RATE_SUSTAINED",
|
||||
environ_prefix=None,
|
||||
),
|
||||
"user_list_burst": values.Value(
|
||||
default="30/minute",
|
||||
environ_name="API_USERS_LIST_THROTTLE_RATE_BURST",
|
||||
environ_prefix=None,
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
SPECTACULAR_SETTINGS = {
|
||||
"TITLE": "Conversations API",
|
||||
"DESCRIPTION": "This is the conversations API schema.",
|
||||
"VERSION": "1.0.0",
|
||||
"SERVE_INCLUDE_SCHEMA": False,
|
||||
"ENABLE_DJANGO_DEPLOY_CHECK": values.BooleanValue(
|
||||
default=False,
|
||||
environ_name="SPECTACULAR_SETTINGS_ENABLE_DJANGO_DEPLOY_CHECK",
|
||||
),
|
||||
"COMPONENT_SPLIT_REQUEST": True,
|
||||
# OTHER SETTINGS
|
||||
"SWAGGER_UI_DIST": "SIDECAR", # shorthand to use the sidecar instead
|
||||
"SWAGGER_UI_FAVICON_HREF": "SIDECAR",
|
||||
"REDOC_DIST": "SIDECAR",
|
||||
}
|
||||
|
||||
# Mail
|
||||
EMAIL_BACKEND = values.Value("django.core.mail.backends.smtp.EmailBackend")
|
||||
EMAIL_BRAND_NAME = values.Value(None)
|
||||
EMAIL_HOST = values.Value(None)
|
||||
EMAIL_HOST_USER = values.Value(None)
|
||||
EMAIL_HOST_PASSWORD = values.Value(None)
|
||||
EMAIL_LOGO_IMG = values.Value(None)
|
||||
EMAIL_PORT = values.PositiveIntegerValue(None)
|
||||
EMAIL_USE_TLS = values.BooleanValue(False)
|
||||
EMAIL_USE_SSL = values.BooleanValue(False)
|
||||
EMAIL_FROM = values.Value("from@example.com")
|
||||
|
||||
AUTH_USER_MODEL = "core.User"
|
||||
|
||||
# CORS
|
||||
CORS_ALLOW_CREDENTIALS = True
|
||||
CORS_ALLOW_ALL_ORIGINS = values.BooleanValue(False)
|
||||
CORS_ALLOWED_ORIGINS = values.ListValue([])
|
||||
CORS_ALLOWED_ORIGIN_REGEXES = values.ListValue([])
|
||||
|
||||
# Sentry
|
||||
SENTRY_DSN = values.Value(None, environ_name="SENTRY_DSN", environ_prefix=None)
|
||||
|
||||
# Frontend
|
||||
FRONTEND_THEME = values.Value(None, environ_name="FRONTEND_THEME", environ_prefix=None)
|
||||
FRONTEND_HOMEPAGE_FEATURE_ENABLED = values.BooleanValue(
|
||||
default=True,
|
||||
environ_name="FRONTEND_HOMEPAGE_FEATURE_ENABLED",
|
||||
environ_prefix=None,
|
||||
)
|
||||
FRONTEND_CSS_URL = values.Value(None, environ_name="FRONTEND_CSS_URL", environ_prefix=None)
|
||||
|
||||
THEME_CUSTOMIZATION_FILE_PATH = values.Value(
|
||||
os.path.join(BASE_DIR, "conversations/configuration/theme/default.json"),
|
||||
environ_name="THEME_CUSTOMIZATION_FILE_PATH",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
THEME_CUSTOMIZATION_CACHE_TIMEOUT = values.Value(
|
||||
60 * 60 * 24,
|
||||
environ_name="THEME_CUSTOMIZATION_CACHE_TIMEOUT",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
# Posthog
|
||||
POSTHOG_KEY = values.DictValue(None, environ_name="POSTHOG_KEY", environ_prefix=None)
|
||||
|
||||
# Crisp
|
||||
CRISP_WEBSITE_ID = values.Value(None, environ_name="CRISP_WEBSITE_ID", environ_prefix=None)
|
||||
|
||||
# Easy thumbnails
|
||||
THUMBNAIL_EXTENSION = "webp"
|
||||
THUMBNAIL_TRANSPARENCY_EXTENSION = "webp"
|
||||
THUMBNAIL_DEFAULT_STORAGE_ALIAS = "default"
|
||||
THUMBNAIL_ALIASES = {}
|
||||
|
||||
# Session
|
||||
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
|
||||
SESSION_CACHE_ALIAS = "default"
|
||||
SESSION_COOKIE_AGE = values.PositiveIntegerValue(
|
||||
default=60 * 60 * 12, environ_name="SESSION_COOKIE_AGE", environ_prefix=None
|
||||
)
|
||||
|
||||
# OIDC - Authorization Code Flow
|
||||
OIDC_CREATE_USER = values.BooleanValue(
|
||||
default=True,
|
||||
environ_name="OIDC_CREATE_USER",
|
||||
)
|
||||
OIDC_RP_SIGN_ALGO = values.Value("RS256", environ_name="OIDC_RP_SIGN_ALGO", environ_prefix=None)
|
||||
OIDC_RP_CLIENT_ID = values.Value(
|
||||
"conversations", environ_name="OIDC_RP_CLIENT_ID", environ_prefix=None
|
||||
)
|
||||
OIDC_RP_CLIENT_SECRET = values.Value(
|
||||
None,
|
||||
environ_name="OIDC_RP_CLIENT_SECRET",
|
||||
environ_prefix=None,
|
||||
)
|
||||
OIDC_OP_JWKS_ENDPOINT = values.Value(environ_name="OIDC_OP_JWKS_ENDPOINT", environ_prefix=None)
|
||||
OIDC_OP_AUTHORIZATION_ENDPOINT = values.Value(
|
||||
environ_name="OIDC_OP_AUTHORIZATION_ENDPOINT", environ_prefix=None
|
||||
)
|
||||
OIDC_OP_TOKEN_ENDPOINT = values.Value(
|
||||
None, environ_name="OIDC_OP_TOKEN_ENDPOINT", environ_prefix=None
|
||||
)
|
||||
OIDC_OP_USER_ENDPOINT = values.Value(
|
||||
None, environ_name="OIDC_OP_USER_ENDPOINT", environ_prefix=None
|
||||
)
|
||||
OIDC_OP_LOGOUT_ENDPOINT = values.Value(
|
||||
None, environ_name="OIDC_OP_LOGOUT_ENDPOINT", environ_prefix=None
|
||||
)
|
||||
OIDC_AUTH_REQUEST_EXTRA_PARAMS = values.DictValue(
|
||||
{}, environ_name="OIDC_AUTH_REQUEST_EXTRA_PARAMS", environ_prefix=None
|
||||
)
|
||||
OIDC_RP_SCOPES = values.Value(
|
||||
"openid email", environ_name="OIDC_RP_SCOPES", environ_prefix=None
|
||||
)
|
||||
LOGIN_REDIRECT_URL = values.Value(None, environ_name="LOGIN_REDIRECT_URL", environ_prefix=None)
|
||||
LOGIN_REDIRECT_URL_FAILURE = values.Value(
|
||||
None, environ_name="LOGIN_REDIRECT_URL_FAILURE", environ_prefix=None
|
||||
)
|
||||
LOGOUT_REDIRECT_URL = values.Value(
|
||||
None, environ_name="LOGOUT_REDIRECT_URL", environ_prefix=None
|
||||
)
|
||||
OIDC_USE_NONCE = values.BooleanValue(
|
||||
default=True, environ_name="OIDC_USE_NONCE", environ_prefix=None
|
||||
)
|
||||
OIDC_REDIRECT_REQUIRE_HTTPS = values.BooleanValue(
|
||||
default=False, environ_name="OIDC_REDIRECT_REQUIRE_HTTPS", environ_prefix=None
|
||||
)
|
||||
OIDC_REDIRECT_ALLOWED_HOSTS = values.ListValue(
|
||||
default=[], environ_name="OIDC_REDIRECT_ALLOWED_HOSTS", environ_prefix=None
|
||||
)
|
||||
OIDC_STORE_ID_TOKEN = values.BooleanValue(
|
||||
default=True, environ_name="OIDC_STORE_ID_TOKEN", environ_prefix=None
|
||||
)
|
||||
OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = values.BooleanValue(
|
||||
default=True,
|
||||
environ_name="OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION",
|
||||
environ_prefix=None,
|
||||
)
|
||||
OIDC_USE_PKCE = values.BooleanValue(
|
||||
default=False, environ_name="OIDC_USE_PKCE", environ_prefix=None
|
||||
)
|
||||
OIDC_PKCE_CODE_CHALLENGE_METHOD = values.Value(
|
||||
default="S256",
|
||||
environ_name="OIDC_PKCE_CODE_CHALLENGE_METHOD",
|
||||
environ_prefix=None,
|
||||
)
|
||||
OIDC_PKCE_CODE_VERIFIER_SIZE = values.IntegerValue(
|
||||
default=64, environ_name="OIDC_PKCE_CODE_VERIFIER_SIZE", environ_prefix=None
|
||||
)
|
||||
OIDC_STORE_ACCESS_TOKEN = values.BooleanValue(
|
||||
default=False, environ_name="OIDC_STORE_ACCESS_TOKEN", environ_prefix=None
|
||||
)
|
||||
OIDC_STORE_REFRESH_TOKEN = values.BooleanValue(
|
||||
default=False, environ_name="OIDC_STORE_REFRESH_TOKEN", environ_prefix=None
|
||||
)
|
||||
OIDC_STORE_REFRESH_TOKEN_KEY = values.Value(
|
||||
default=None,
|
||||
environ_name="OIDC_STORE_REFRESH_TOKEN_KEY",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
# WARNING: Enabling this setting allows multiple user accounts to share the same email
|
||||
# address. This may cause security issues and is not recommended for production use when
|
||||
# email is activated as fallback for identification (see previous setting).
|
||||
OIDC_ALLOW_DUPLICATE_EMAILS = values.BooleanValue(
|
||||
default=False,
|
||||
environ_name="OIDC_ALLOW_DUPLICATE_EMAILS",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
USER_OIDC_ESSENTIAL_CLAIMS = values.ListValue(
|
||||
default=[], environ_name="USER_OIDC_ESSENTIAL_CLAIMS", environ_prefix=None
|
||||
)
|
||||
|
||||
OIDC_USERINFO_FULLNAME_FIELDS = values.ListValue(
|
||||
default=["first_name", "last_name"],
|
||||
environ_name="OIDC_USERINFO_FULLNAME_FIELDS",
|
||||
environ_prefix=None,
|
||||
)
|
||||
OIDC_USERINFO_SHORTNAME_FIELD = values.Value(
|
||||
default="first_name",
|
||||
environ_name="OIDC_USERINFO_SHORTNAME_FIELD",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
ALLOW_LOGOUT_GET_METHOD = values.BooleanValue(
|
||||
default=True, environ_name="ALLOW_LOGOUT_GET_METHOD", environ_prefix=None
|
||||
)
|
||||
|
||||
# AI service
|
||||
AI_API_KEY = values.Value(None, environ_name="AI_API_KEY", environ_prefix=None)
|
||||
AI_BASE_URL = values.Value(None, environ_name="AI_BASE_URL", environ_prefix=None)
|
||||
AI_MODEL = values.Value(None, environ_name="AI_MODEL", environ_prefix=None)
|
||||
AI_AGENT_NAME = values.Value(
|
||||
"Conversations Assistant",
|
||||
environ_name="AI_AGENT_NAME",
|
||||
environ_prefix=None,
|
||||
)
|
||||
AI_AGENT_INSTRUCTIONS = values.Value(
|
||||
(
|
||||
"You are a helpful assistant. "
|
||||
"Escape formulas or any math notation between `$$`, "
|
||||
"like `$$x^2 + y^2 = z^2$$` or `$$C_l$$`. "
|
||||
"You can use Markdown to format your answers. "
|
||||
),
|
||||
environ_name="AI_AGENT_INSTRUCTIONS",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
# Tools
|
||||
AI_AGENT_TOOLS = values.ListValue(
|
||||
default=[],
|
||||
environ_name="AI_AGENT_TOOLS",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
# Web search
|
||||
TAVILY_API_KEY = values.Value(
|
||||
None, # Tavily API key is not set by default
|
||||
environ_name="TAVILY_API_KEY",
|
||||
environ_prefix=None,
|
||||
)
|
||||
TAVILY_MAX_RESULTS = values.PositiveIntegerValue(
|
||||
default=5,
|
||||
environ_name="TAVILY_MAX_RESULTS",
|
||||
environ_prefix=None,
|
||||
)
|
||||
TAVILY_API_TIMEOUT = values.PositiveIntegerValue(
|
||||
default=10, # seconds
|
||||
environ_name="TAVILY_API_TIMEOUT",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
# Logging
|
||||
# We want to make it easy to log to console but by default we log production
|
||||
# to Sentry and don't want to log to console.
|
||||
LOGGING = {
|
||||
"version": 1,
|
||||
"disable_existing_loggers": False,
|
||||
"formatters": {
|
||||
"simple": {
|
||||
"format": "{asctime} {name} {levelname} {message}",
|
||||
"style": "{",
|
||||
},
|
||||
},
|
||||
"handlers": {
|
||||
"console": {
|
||||
"class": "logging.StreamHandler",
|
||||
"formatter": "simple",
|
||||
},
|
||||
},
|
||||
# Override root logger to send it to console
|
||||
"root": {
|
||||
"handlers": ["console"],
|
||||
"level": values.Value(
|
||||
"INFO", environ_name="LOGGING_LEVEL_LOGGERS_ROOT", environ_prefix=None
|
||||
),
|
||||
},
|
||||
"loggers": {
|
||||
"core": {
|
||||
"handlers": ["console"],
|
||||
"level": values.Value(
|
||||
"INFO",
|
||||
environ_name="LOGGING_LEVEL_LOGGERS_APP",
|
||||
environ_prefix=None,
|
||||
),
|
||||
"propagate": False,
|
||||
},
|
||||
"docs.security": {
|
||||
"handlers": ["console"],
|
||||
"level": values.Value(
|
||||
"INFO",
|
||||
environ_name="LOGGING_LEVEL_LOGGERS_SECURITY",
|
||||
environ_prefix=None,
|
||||
),
|
||||
"propagate": False,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
API_USERS_LIST_LIMIT = values.PositiveIntegerValue(
|
||||
default=5,
|
||||
environ_name="API_USERS_LIST_LIMIT",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
# ML Flow
|
||||
ML_FLOW_TRACKING_URI = values.Value(
|
||||
None,
|
||||
environ_name="ML_FLOW_TRACKING_URI",
|
||||
environ_prefix=None,
|
||||
)
|
||||
ML_FLOW_EXPERIMENT_NAME = values.Value(
|
||||
"conversations-openai-tracing",
|
||||
environ_name="ML_FLOW_EXPERIMENT_NAME",
|
||||
environ_prefix=None,
|
||||
)
|
||||
|
||||
# pylint: disable=invalid-name
|
||||
@property
|
||||
def ENVIRONMENT(self):
|
||||
"""Environment in which the application is launched."""
|
||||
return self.__class__.__name__.lower()
|
||||
|
||||
# pylint: disable=invalid-name
|
||||
@property
|
||||
def RELEASE(self):
|
||||
"""
|
||||
Return the release information.
|
||||
|
||||
Delegate to the module function to enable easier testing.
|
||||
"""
|
||||
return get_release()
|
||||
|
||||
# pylint: disable=invalid-name
|
||||
@property
|
||||
def PARLER_LANGUAGES(self):
|
||||
"""
|
||||
Return languages for Parler computed from the LANGUAGES and LANGUAGE_CODE settings.
|
||||
"""
|
||||
return {
|
||||
self.SITE_ID: tuple({"code": code} for code, _name in self.LANGUAGES),
|
||||
"default": {
|
||||
"fallbacks": [self.LANGUAGE_CODE],
|
||||
"hide_untranslated": False,
|
||||
},
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def post_setup(cls):
|
||||
"""Post setup configuration.
|
||||
This is the place where you can configure settings that require other
|
||||
settings to be loaded.
|
||||
"""
|
||||
super().post_setup()
|
||||
|
||||
# The SENTRY_DSN setting should be available to activate sentry for an environment
|
||||
if cls.SENTRY_DSN is not None:
|
||||
sentry_sdk.init(
|
||||
dsn=cls.SENTRY_DSN,
|
||||
environment=cls.__name__.lower(),
|
||||
release=get_release(),
|
||||
integrations=[DjangoIntegration()],
|
||||
)
|
||||
sentry_sdk.set_tag("application", "backend")
|
||||
|
||||
# Ignore the logs added by the DockerflowMiddleware
|
||||
ignore_logger("request.summary")
|
||||
|
||||
if cls.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION and cls.OIDC_ALLOW_DUPLICATE_EMAILS:
|
||||
raise ValueError(
|
||||
"Both OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION and "
|
||||
"OIDC_ALLOW_DUPLICATE_EMAILS cannot be set to True simultaneously. "
|
||||
)
|
||||
|
||||
|
||||
class Build(Base):
|
||||
"""Settings used when the application is built.
|
||||
|
||||
This environment should not be used to run the application. Just to build it with non-blocking
|
||||
settings.
|
||||
"""
|
||||
|
||||
SECRET_KEY = values.Value("DummyKey")
|
||||
STORAGES = {
|
||||
"default": {
|
||||
"BACKEND": "django.core.files.storage.FileSystemStorage",
|
||||
},
|
||||
"staticfiles": {
|
||||
"BACKEND": values.Value(
|
||||
"whitenoise.storage.CompressedManifestStaticFilesStorage",
|
||||
environ_name="STORAGES_STATICFILES_BACKEND",
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class Development(Base):
|
||||
"""
|
||||
Development environment settings
|
||||
|
||||
We set DEBUG to True and configure the server to respond from all hosts.
|
||||
"""
|
||||
|
||||
ALLOWED_HOSTS = ["*"]
|
||||
CORS_ALLOW_ALL_ORIGINS = True
|
||||
CSRF_TRUSTED_ORIGINS = ["http://localhost:8072", "http://localhost:3000"]
|
||||
DEBUG = True
|
||||
|
||||
SESSION_COOKIE_NAME = "conversations_sessionid"
|
||||
|
||||
USE_SWAGGER = True
|
||||
SESSION_CACHE_ALIAS = "session"
|
||||
CACHES = {
|
||||
"default": {
|
||||
"BACKEND": "django.core.cache.backends.dummy.DummyCache",
|
||||
},
|
||||
"session": {
|
||||
"BACKEND": "django_redis.cache.RedisCache",
|
||||
"LOCATION": values.Value(
|
||||
"redis://redis:6379/2",
|
||||
environ_name="REDIS_URL",
|
||||
environ_prefix=None,
|
||||
),
|
||||
"TIMEOUT": values.IntegerValue(
|
||||
30, # timeout in seconds
|
||||
environ_name="CACHES_DEFAULT_TIMEOUT",
|
||||
environ_prefix=None,
|
||||
),
|
||||
"OPTIONS": {
|
||||
"CLIENT_CLASS": "django_redis.client.DefaultClient",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
def __init__(self):
|
||||
# pylint: disable=invalid-name
|
||||
self.INSTALLED_APPS += ["django_extensions", "drf_spectacular_sidecar"]
|
||||
|
||||
|
||||
class Test(Base):
|
||||
"""Test environment settings"""
|
||||
|
||||
PASSWORD_HASHERS = [
|
||||
"django.contrib.auth.hashers.MD5PasswordHasher",
|
||||
]
|
||||
USE_SWAGGER = True
|
||||
# Static files are not used in the test environment
|
||||
# Tests are raising warnings because the /data/static directory does not exist
|
||||
STATIC_ROOT = None
|
||||
|
||||
os.environ["OPENAI_AGENTS_DISABLE_TRACING"] = "true"
|
||||
|
||||
def __init__(self):
|
||||
# pylint: disable=invalid-name
|
||||
self.INSTALLED_APPS += ["drf_spectacular_sidecar"]
|
||||
|
||||
|
||||
class ContinuousIntegration(Test):
|
||||
"""
|
||||
Continuous Integration environment settings
|
||||
|
||||
nota bene: it should inherit from the Test environment.
|
||||
"""
|
||||
|
||||
|
||||
class Production(Base):
|
||||
"""
|
||||
Production environment settings
|
||||
|
||||
You must define the ALLOWED_HOSTS environment variable in Production
|
||||
configuration (and derived configurations):
|
||||
ALLOWED_HOSTS=["foo.com", "foo.fr"]
|
||||
"""
|
||||
|
||||
# Security
|
||||
# Add allowed host from environment variables.
|
||||
# The machine hostname is added by default,
|
||||
# it makes the application pingable by a load balancer on the same machine by example
|
||||
ALLOWED_HOSTS = [
|
||||
*values.ListValue([], environ_name="ALLOWED_HOSTS"),
|
||||
gethostbyname(gethostname()),
|
||||
]
|
||||
CSRF_TRUSTED_ORIGINS = values.ListValue([])
|
||||
SECURE_BROWSER_XSS_FILTER = True
|
||||
SECURE_CONTENT_TYPE_NOSNIFF = True
|
||||
|
||||
# SECURE_PROXY_SSL_HEADER allows to fix the scheme in Django's HttpRequest
|
||||
# object when your application is behind a reverse proxy.
|
||||
#
|
||||
# Keep this SECURE_PROXY_SSL_HEADER configuration only if :
|
||||
# - your Django app is behind a proxy.
|
||||
# - your proxy strips the X-Forwarded-Proto header from all incoming requests
|
||||
# - Your proxy sets the X-Forwarded-Proto header and sends it to Django
|
||||
#
|
||||
# In other cases, you should comment the following line to avoid security issues.
|
||||
# SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")
|
||||
SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")
|
||||
SECURE_HSTS_SECONDS = 60
|
||||
SECURE_HSTS_PRELOAD = True
|
||||
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
|
||||
SECURE_SSL_REDIRECT = True
|
||||
SECURE_REDIRECT_EXEMPT = [
|
||||
"^__lbheartbeat__",
|
||||
"^__heartbeat__",
|
||||
]
|
||||
|
||||
# Modern browsers require to have the `secure` attribute on cookies with `Samesite=none`
|
||||
CSRF_COOKIE_SECURE = True
|
||||
SESSION_COOKIE_SECURE = True
|
||||
|
||||
# Privacy
|
||||
SECURE_REFERRER_POLICY = "same-origin"
|
||||
|
||||
CACHES = {
|
||||
"default": {
|
||||
"BACKEND": "django_redis.cache.RedisCache",
|
||||
"LOCATION": values.Value(
|
||||
"redis://redis:6379/1",
|
||||
environ_name="REDIS_URL",
|
||||
environ_prefix=None,
|
||||
),
|
||||
"TIMEOUT": values.IntegerValue(
|
||||
30, # timeout in seconds
|
||||
environ_name="CACHES_DEFAULT_TIMEOUT",
|
||||
environ_prefix=None,
|
||||
),
|
||||
"OPTIONS": {
|
||||
"CLIENT_CLASS": "django_redis.client.DefaultClient",
|
||||
},
|
||||
"KEY_PREFIX": values.Value(
|
||||
"conversations",
|
||||
environ_name="CACHES_KEY_PREFIX",
|
||||
environ_prefix=None,
|
||||
),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class Feature(Production):
|
||||
"""
|
||||
Feature environment settings
|
||||
|
||||
nota bene: it should inherit from the Production environment.
|
||||
"""
|
||||
|
||||
|
||||
class Staging(Production):
|
||||
"""
|
||||
Staging environment settings
|
||||
|
||||
nota bene: it should inherit from the Production environment.
|
||||
"""
|
||||
|
||||
|
||||
class PreProduction(Production):
|
||||
"""
|
||||
Pre-production environment settings
|
||||
|
||||
nota bene: it should inherit from the Production environment.
|
||||
"""
|
||||
|
||||
|
||||
class Demo(Production):
|
||||
"""
|
||||
Demonstration environment settings
|
||||
|
||||
nota bene: it should inherit from the Production environment.
|
||||
"""
|
||||
|
||||
STORAGES = {
|
||||
"default": {
|
||||
"BACKEND": "django.core.files.storage.FileSystemStorage",
|
||||
},
|
||||
"staticfiles": {
|
||||
"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage",
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
"""URL configuration for the conversations project"""
|
||||
|
||||
from django.conf import settings
|
||||
from django.conf.urls.static import static
|
||||
from django.contrib import admin
|
||||
from django.contrib.staticfiles.urls import staticfiles_urlpatterns
|
||||
from django.urls import include, path, re_path
|
||||
|
||||
from drf_spectacular.views import (
|
||||
SpectacularJSONAPIView,
|
||||
SpectacularRedocView,
|
||||
SpectacularSwaggerView,
|
||||
)
|
||||
|
||||
urlpatterns = [
|
||||
path("admin/", admin.site.urls),
|
||||
path("", include("core.urls")),
|
||||
]
|
||||
|
||||
if settings.DEBUG:
|
||||
urlpatterns = (
|
||||
urlpatterns
|
||||
+ staticfiles_urlpatterns()
|
||||
+ static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
|
||||
)
|
||||
|
||||
|
||||
if settings.USE_SWAGGER or settings.DEBUG:
|
||||
urlpatterns += [
|
||||
path(
|
||||
f"api/{settings.API_VERSION}/swagger.json",
|
||||
SpectacularJSONAPIView.as_view(
|
||||
api_version=settings.API_VERSION,
|
||||
urlconf="core.urls",
|
||||
),
|
||||
name="client-api-schema",
|
||||
),
|
||||
path(
|
||||
f"api/{settings.API_VERSION}/swagger/",
|
||||
SpectacularSwaggerView.as_view(url_name="client-api-schema"),
|
||||
name="swagger-ui-schema",
|
||||
),
|
||||
re_path(
|
||||
f"api/{settings.API_VERSION}/redoc/",
|
||||
SpectacularRedocView.as_view(url_name="client-api-schema"),
|
||||
name="redoc-schema",
|
||||
),
|
||||
]
|
||||
@@ -0,0 +1,17 @@
|
||||
"""
|
||||
WSGI config for the conversations project.
|
||||
|
||||
It exposes the WSGI callable as a module-level variable named ``application``.
|
||||
|
||||
For more information on this file, see
|
||||
https://docs.djangoproject.com/en/3.1/howto/deployment/wsgi/
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from configurations.wsgi import get_wsgi_application
|
||||
|
||||
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "conversations.settings")
|
||||
os.environ.setdefault("DJANGO_CONFIGURATION", "Development")
|
||||
|
||||
application = get_wsgi_application()
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user