Implement local generation workflow and automation
This commit is contained in:
@@ -0,0 +1,4 @@
|
|||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
automation/reports/
|
||||||
|
automation/state/
|
||||||
@@ -16,3 +16,100 @@ sans perte de cohérence, de mémoire ou de contrôle.
|
|||||||
|
|
||||||
## Statut
|
## Statut
|
||||||
v2 — développement en cours (open-source)
|
v2 — développement en cours (open-source)
|
||||||
|
|
||||||
|
## Suivi
|
||||||
|
- backlog actif: [`TODO_ACTIVE.md`](./TODO_ACTIVE.md)
|
||||||
|
- etat livre: [`TODO_IMPLEMENTE.md`](./TODO_IMPLEMENTE.md)
|
||||||
|
- ordre d'execution recommande: [`docs/EXECUTION_PLAN_2026-03-08.md`](./docs/EXECUTION_PLAN_2026-03-08.md)
|
||||||
|
- runbook local: [`docs/runbooks/LOCAL_GENERATION.md`](./docs/runbooks/LOCAL_GENERATION.md)
|
||||||
|
- comparatif modeles local: [`docs/MODEL_COMPARISON_2026-03-08.md`](./docs/MODEL_COMPARISON_2026-03-08.md)
|
||||||
|
|
||||||
|
## Automation des lots utiles
|
||||||
|
|
||||||
|
Le driver principal des prochains lots utiles est maintenant:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/run_next_lots.py --lot full
|
||||||
|
```
|
||||||
|
|
||||||
|
Points clés:
|
||||||
|
- le manifeste versionné est `automation/next_lots.toml`
|
||||||
|
- le driver réutilise les smokes existants au lieu de dupliquer le pipeline
|
||||||
|
- les opérations sensibles restent semi-autos: en cas de switch Apple ou de restart runtime, le cycle prépare les commandes exactes puis s'arrête avec un état de reprise
|
||||||
|
- reprise:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/run_next_lots.py --resume automation/state/next_lots_state.json
|
||||||
|
```
|
||||||
|
|
||||||
|
- synchronisation seule des plans/TODOs/readmes à partir du dernier état:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/run_next_lots.py --lot tracking_sync --report-only
|
||||||
|
```
|
||||||
|
|
||||||
|
## Generation locale via Mascarade
|
||||||
|
|
||||||
|
`ai-novel-engine` parle un provider OpenAI-compatible. Pour utiliser la
|
||||||
|
generation locale via `mascarade`, pointer simplement le moteur narratif vers le
|
||||||
|
core Python sur `:8100`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export ANE_PROVIDER=openai_compatible
|
||||||
|
export ANE_BASE_URL=http://127.0.0.1:8100
|
||||||
|
export ANE_MODEL=<provider:model>
|
||||||
|
export ANE_MAX_TOKENS=512
|
||||||
|
export ANE_MAX_TOKENS_STRUCTURE=256
|
||||||
|
export ANE_MAX_TOKENS_DRAFT=512
|
||||||
|
export ANE_MAX_TOKENS_CRITIQUE=384
|
||||||
|
export ANE_MAX_TOKENS_REWRITE=512
|
||||||
|
export ANE_MAX_TOKENS_GATE=320
|
||||||
|
export ANE_MAX_TOKENS_REPAIR=384
|
||||||
|
export ANE_MAX_TOKENS_MEMORY=256
|
||||||
|
export ANE_REPAIR_MAX_PASSES=2
|
||||||
|
# optionnel si tu veux forcer un fallback explicite pour la reparation
|
||||||
|
# export ANE_REPAIR_FALLBACK_MODEL=ollama:qwen2.5:7b
|
||||||
|
|
||||||
|
# seulement si MASCARADE_API_KEY est active
|
||||||
|
export ANE_API_KEY=ton-token-mascarade
|
||||||
|
|
||||||
|
python3 -m cli.main generate chapter --chapter 01
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes :
|
||||||
|
- `ANE_MODEL` est requis; le repo n'impose pas de modele par défaut
|
||||||
|
- `ANE_MODEL` selectionne le backend local par prefixe `apple-coreml:` ou `ollama:`
|
||||||
|
- `ANE_MAX_TOKENS` reste le plafond global par défaut
|
||||||
|
- les overrides `ANE_MAX_TOKENS_STRUCTURE`, `..._DRAFT`, `..._CRITIQUE`, `..._REWRITE`, `..._GATE`, `..._REPAIR`, `..._MEMORY` permettent d'ajuster chaque étape
|
||||||
|
- `ANE_REPAIR_MAX_PASSES` borne la boucle `repair`
|
||||||
|
- `ANE_REPAIR_FALLBACK_MODEL` permet de forcer le modele du second passage `repair`
|
||||||
|
- le pipeline narratif reste entierement dans `ai-novel-engine`
|
||||||
|
- `mascarade` sert uniquement de runtime local et de couche OpenAI-compatible
|
||||||
|
- dernier cycle complet termine au 9 mars 2026 :
|
||||||
|
- `apple-coreml:qwen3.5-4b-onnx-q4f16` est `accepted` de bout en bout sous garde-fou
|
||||||
|
- `ollama:qwen2.5:7b` atteint `gate`, exerce `repair` en live, puis finit `quality_blocked` sur `outline_like`
|
||||||
|
- `apple-coreml:stateful-mistral7b-instruct-int4-coreml` reste `preflight_only`
|
||||||
|
- les baselines `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b` sont en rerun automatise separe; ils ne sont plus la reference locale courante
|
||||||
|
- le smoke et `status` exposent maintenant `gate_v1.json`, `quality_blockers`, `failed_stage`, `repair_attempts` et `repair_models`
|
||||||
|
- le runtime Apple local ne sert qu'un `model_id` a la fois; un fallback `repair` vers un autre modele Apple exige donc un switch de service entre runs
|
||||||
|
|
||||||
|
Smoke test local rapide :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/smoke_local_generation.sh \
|
||||||
|
--base-url http://127.0.0.1:8100 \
|
||||||
|
--model "ollama:qwen2.5:1.5b" \
|
||||||
|
--approve
|
||||||
|
```
|
||||||
|
|
||||||
|
Le script cree un workspace temporaire, ecrit une intention de test, lance la vraie CLI publique, fait un warm-up automatique pour `apple-coreml`, puis affiche un resume humain des artefacts et du `meta.json`. En mode `apple-coreml`, il applique par defaut un timeout plus large (`ANE_TIMEOUT=900`) et des budgets de smoke plus courts pour eviter de faire exploser la latence locale. Pour les reruns qualitatifs de reference, fixer explicitement `--timeout 300` et des budgets `ANE_MAX_TOKENS_*` communs. Utiliser `--workspace`, `--chapter`, `--intention`, `--timeout`, `--approve` ou `--reject` si besoin.
|
||||||
|
|
||||||
|
## Etat auto-synchronise
|
||||||
|
## Etat auto-synchronise
|
||||||
|
<!-- AUTO-SYNC:ANE-README:START -->
|
||||||
|
- dernier cycle automatise: 2026-03-09T06:53:02+00:00
|
||||||
|
- reference locale actuelle: aucun accepted, meilleur diagnostic: apple-coreml:qwen2.5-0.5b-instruct-onnx
|
||||||
|
- prochain lot utile: Analyser les runs ayant atteint gate/repair puis resserrer la reference locale autour des meilleurs candidats.
|
||||||
|
- lancer un cycle: `python3 scripts/run_next_lots.py --lot full`
|
||||||
|
- checkpoint manuel en attente: Le runtime Apple sert `qwen2.5-0.5b-instruct-onnx` au lieu de `stateful-mistral7b-instruct-int4-coreml`.
|
||||||
|
<!-- AUTO-SYNC:ANE-README:END -->
|
||||||
|
|||||||
@@ -0,0 +1,74 @@
|
|||||||
|
# TODO actif - AI Novel Engine
|
||||||
|
|
||||||
|
Source de verite des taches restantes pour `ai-novel-engine`.
|
||||||
|
|
||||||
|
Regle:
|
||||||
|
- cocher ici ce qui est fait puis deplacer le lot livre vers `TODO_IMPLEMENTE.md`
|
||||||
|
- ne suivre ici que le travail restant ou les blocages encore ouverts
|
||||||
|
- garder les dependances `mascarade` explicites
|
||||||
|
|
||||||
|
## Deja implemente
|
||||||
|
- [x] P0 Pipeline chapitre `intention -> structure -> draft -> critique -> rewrite -> validation -> memoire`
|
||||||
|
- [x] P0 Normalisation de chapitre avec identifiant canonique `chapitre_XX` et detection des collisions legacy
|
||||||
|
- [x] P0 Provider OpenAI-compatible et branchement local via `mascarade`
|
||||||
|
- [x] P0 Budgets par etape (`ANE_MAX_TOKENS_*`)
|
||||||
|
- [x] P0 Parsing JSON tolerant pour les sorties locales imparfaites
|
||||||
|
- [x] P0 Second passage de reessai pour `critique` et `memory` si le JSON reste invalide apres parsing tolerant
|
||||||
|
- [x] P0 Garde-fou manuscrit dur avant promotion (`gate_v1.json`, heuristiques locales, verdict `quality_blocked`)
|
||||||
|
- [x] P0 Boucle `repair` automatique entre `gate` et `quality_blocked`, avec artefacts `repair_vN.md`, fallback modele et preservation de `draft_v2.md`
|
||||||
|
- [x] P0 Smoke script local `scripts/smoke_local_generation.sh`
|
||||||
|
- [x] P0 Timeout provider remonte maintenant en `ProviderError` et marque correctement `failed_stage` dans `meta.json`
|
||||||
|
- [x] P0 Warm-up Apple du smoke remonte maintenant une erreur lisible au lieu d'une stacktrace brute
|
||||||
|
- [x] P0 Prompts `draft_v1` et `rewrite_v1` durcis pour imposer une prose continue sans titres ni puces
|
||||||
|
- [x] P1 Flags CLI non interactifs `--approve` et `--reject`
|
||||||
|
- [x] P1 `status` enrichi avec les chapitres en echec, en attente et bloques par garde-fou
|
||||||
|
- [x] P1 Resume de smoke humain a partir du `meta.json`
|
||||||
|
- [x] P1 Contrat cross-repo et recovery documentes via les runbooks
|
||||||
|
- [x] P2 `docs/vision.md`, `docs/roadmap.md` et le runbook local ne sont plus des placeholders
|
||||||
|
- [x] P0 Revalidation sous garde-fou de `ollama:qwen2.5:1.5b` -> `quality_blocked`
|
||||||
|
- [x] P0 Revalidation sous garde-fou de `apple-coreml:qwen2.5-0.5b-instruct-onnx` -> `quality_blocked`
|
||||||
|
- [x] P0 Revalidation sous garde-fou de `apple-coreml:qwen3.5-4b-onnx-q4f16` -> `provider_failed` en `rewrite`
|
||||||
|
- [x] P0 Revalidation sous garde-fou de `ollama:qwen2.5:7b` -> `provider_failed` par timeout en `draft`
|
||||||
|
- [x] P0 Comparatif local re-ecrit pour le protocole avec garde-fou dans `docs/MODEL_COMPARISON_2026-03-08.md`
|
||||||
|
- [x] P0 Revalidation reelle sous protocole `gate + repair` borne a `300s` par requete:
|
||||||
|
- `ollama:qwen2.5:1.5b` -> `failed` en `structure`
|
||||||
|
- `apple-coreml:qwen2.5-0.5b-instruct-onnx` -> `failed` en `rewrite`
|
||||||
|
- `apple-coreml:qwen3.5-4b-onnx-q4f16` -> `failed` en `rewrite`
|
||||||
|
- `ollama:qwen2.5:7b` -> `failed` en `rewrite`
|
||||||
|
|
||||||
|
## Actif
|
||||||
|
- [ ] P0 Terminer le lot `baselines`, puis relancer `tracking_sync` sur un etat complet incluant `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b`
|
||||||
|
- [ ] P0 Faire passer `ollama:qwen2.5:7b` de `quality_blocked` a `accepted` en supprimant le residu `outline_like` apres `repair`
|
||||||
|
- [ ] P0 Faire terminer au moins un cycle `python3 scripts/run_next_lots.py --lot full` jusqu'a `tracking_sync` sans checkpoint manuel autre qu'un switch Apple explicite
|
||||||
|
- [ ] P1 Requalifier `apple-coreml:qwen3.5-4b-onnx-q4f16` comme reference Apple stable sur plusieurs cycles, pas sur un seul run accepte
|
||||||
|
- [ ] P1 Rendre la strategie de fallback `repair` consciente des modeles reellement servis: le runtime Apple n'expose qu'un `model_id` a la fois
|
||||||
|
- [ ] P1 Garder l'installation/staging Apple de `qwen2.5-0.5b-instruct-onnx`, `qwen3.5-4b-onnx-q4f16` et `stateful-mistral7b-instruct-int4-coreml` comme prerequis explicite
|
||||||
|
|
||||||
|
## Bloque
|
||||||
|
- [ ] P1 `ollama:qwen2.5:7b` atteint maintenant `gate` et exerce `repair`, mais reste bloque sur `outline_like` apres deux passes
|
||||||
|
- [ ] P1 Le lot `baselines` exige encore un switch Apple explicite avant de pouvoir finir `apple-coreml:qwen2.5-0.5b-instruct-onnx`
|
||||||
|
- [ ] P1 `ollama:qwen2.5:1.5b` reste lent et reste a requalifier une fois `baselines` repris jusqu'au bout
|
||||||
|
- [ ] P1 `apple-coreml:stateful-mistral7b-instruct-int4-coreml` reste preflight-only sur cette machine: preflight froid `:8100` OK en 128 s, preflight chaud OK en 63 s, mais le smoke ANE est reste bloque a `structure` pendant plus de 8 minutes avec les budgets de smoke
|
||||||
|
- [ ] P1 Le host `ollama` natif 0.17.7 sur cette machine echoue sur `qwen2.5:1.5b` avec un crash Metal; la validation ANE reelle passe par un service Docker CPU expose sur `127.0.0.1:11435` et route via `mascarade`
|
||||||
|
- [ ] P1 Le runtime Apple local n'expose qu'un seul `model_id` a la fois; un fallback `repair` vers un autre modele Apple exige donc un switch de service entre deux runs, pas au milieu d'un smoke
|
||||||
|
|
||||||
|
## Prochain ordre
|
||||||
|
- [ ] P0 Finir `python3 scripts/run_next_lots.py --lot baselines`, puis laisser `tracking_sync` recalculer les verdicts complets
|
||||||
|
- [ ] P0 Tuner `rewrite_v1` et la passe `repair` pour eliminer `outline_like` sur `ollama:qwen2.5:7b`
|
||||||
|
- [ ] P1 Rejouer ensuite `python3 scripts/run_next_lots.py --lot priority_models` pour verifier la stabilite de `apple-coreml:qwen3.5-4b-onnx-q4f16` et le sort de `ollama:qwen2.5:7b`
|
||||||
|
- [ ] P1 Garder `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b` comme baselines vitesse ou regressions tant qu'ils n'ont pas de verdict comparable au protocole courant
|
||||||
|
- [ ] P1 Verifier avant tout rerun Apple que `qwen2.5-0.5b-instruct-onnx`, `qwen3.5-4b-onnx-q4f16` et `stateful-mistral7b-instruct-int4-coreml` sont bien installes/stages et que le bon `model_id` est charge sur `:8201`
|
||||||
|
|
||||||
|
## Auto-sync
|
||||||
|
## Auto-sync
|
||||||
|
<!-- AUTO-SYNC:ANE-TODO-ACTIVE:START -->
|
||||||
|
- dernier cycle automatique: 2026-03-09T06:53:02+00:00
|
||||||
|
- modeles accepted: aucun
|
||||||
|
- modeles ayant atteint gate: apple-coreml:qwen2.5-0.5b-instruct-onnx, ollama:qwen2.5:1.5b
|
||||||
|
- quality_blocked: apple-coreml:qwen2.5-0.5b-instruct-onnx, ollama:qwen2.5:1.5b
|
||||||
|
- provider_failed: aucun
|
||||||
|
- prochain lot recommande: Analyser les runs ayant atteint gate/repair puis resserrer la reference locale autour des meilleurs candidats.
|
||||||
|
- checkpoint manuel en attente: Le runtime Apple sert `qwen2.5-0.5b-instruct-onnx` au lieu de `stateful-mistral7b-instruct-int4-coreml`.
|
||||||
|
- commande preparee: `bash scripts/prepare_runtime_step.sh --apple-model stateful-mistral7b-instruct-int4-coreml --resume-state /Users/electron/Documents/Projets_Creatifs/ai-novel-engine/automation/state/next_lots_state.json --ane-script /Users/electron/Documents/Projets_Creatifs/ai-novel-engine/scripts/run_next_lots.py`
|
||||||
|
- reprise: `python3 scripts/run_next_lots.py --resume /Users/electron/Documents/Projets_Creatifs/ai-novel-engine/automation/state/next_lots_state.json`
|
||||||
|
<!-- AUTO-SYNC:ANE-TODO-ACTIVE:END -->
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
# TODO implemente - AI Novel Engine
|
||||||
|
|
||||||
|
Snapshot append-only de ce qui est reellement livre.
|
||||||
|
|
||||||
|
Regle:
|
||||||
|
- n'ajouter ici qu'un lot termine
|
||||||
|
- ne pas y laisser de travail restant
|
||||||
|
- renvoyer vers `TODO_ACTIVE.md` pour les suites et blocages
|
||||||
|
|
||||||
|
## Deja implemente
|
||||||
|
|
||||||
|
### Lot livre - 7 mars 2026
|
||||||
|
- [x] Pipeline chapitre complet `intention -> structure -> draft -> critique -> rewrite -> validation -> memoire`
|
||||||
|
- [x] Artefacts standardises dans `structure/`, `brouillons/`, `manuscrit/`, `memoire/`
|
||||||
|
- [x] Normalisation des chapitres et detection explicite des collisions `chapitre_1` / `chapitre_01`
|
||||||
|
- [x] CLI `status`, `intention create`, `generate chapter --chapter XX` et alias `write --chapter XX`
|
||||||
|
- [x] Provider generique avec implementation OpenAI-compatible et configuration par variables d'environnement
|
||||||
|
- [x] Branchement local via `mascarade` en pointant `ANE_BASE_URL` vers `http://127.0.0.1:8100`
|
||||||
|
- [x] Budgets par etape avec `ANE_MAX_TOKENS_STRUCTURE`, `..._DRAFT`, `..._CRITIQUE`, `..._REWRITE`, `..._MEMORY`
|
||||||
|
- [x] Parsing JSON tolerant pour les etapes `critique` et `memory`
|
||||||
|
- [x] Second passage de reessai cible sur `critique` et `memory` quand la premiere reponse reste invalide
|
||||||
|
- [x] Prompts versionnes par etape dans `prompts/`
|
||||||
|
- [x] Metadonnees pipeline enrichies avec `stage_attempts`, `retry_stages`, `provider.*` et `last_status_message`
|
||||||
|
- [x] Ecriture immediate de l'etape en cours dans `meta.json` avant les appels provider pour rendre les hangs lisibles
|
||||||
|
- [x] CLI non interactive avec `--approve` et `--reject`
|
||||||
|
- [x] `status` enrichi pour les chapitres en echec et en attente
|
||||||
|
- [x] Smoke script local `scripts/smoke_local_generation.sh` branche sur la vraie CLI et avec warm-up Apple via `:8100`
|
||||||
|
- [x] Presets de smoke Apple plus conservateurs et timeout local plus large pour limiter les faux negatifs de warm-up
|
||||||
|
- [x] Runbook local ANE `docs/runbooks/LOCAL_GENERATION.md`
|
||||||
|
- [x] `docs/vision.md` et `docs/roadmap.md` remplaces par une doc exploitable
|
||||||
|
- [x] Suite unitaire `python3 -m unittest discover -s tests -v` verte sur l'etat livre
|
||||||
|
|
||||||
|
### Lot livre - 8 mars 2026
|
||||||
|
- [x] Validation locale `ollama` de bout en bout avec `ollama:qwen2.5:1.5b` via `mascarade`
|
||||||
|
- [x] Validation Apple locale de bout en bout avec `apple-coreml:qwen2.5-0.5b-instruct-onnx`
|
||||||
|
- [x] Validation Apple locale de bout en bout avec `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||||
|
- [x] Comparatif local documente dans `docs/MODEL_COMPARISON_2026-03-08.md`
|
||||||
|
- [x] Runbook local ANE et `README` realignes sur les modeles reellement valides
|
||||||
|
|
||||||
|
### Lot livre - 8 mars 2026 (garde-fou qualite)
|
||||||
|
- [x] Nouvelle etape `gate` entre `rewrite` et la validation auteur
|
||||||
|
- [x] Type `ManuscriptGateReport` et artefact `brouillons/chapitres/chapitre_XX/gate_v1.json`
|
||||||
|
- [x] Heuristiques bloquantes locales `too_short`, `truncated_ending`, `outline_like`
|
||||||
|
- [x] Budget provider `ANE_MAX_TOKENS_GATE`
|
||||||
|
- [x] `--approve` et la promotion manuscrit ne bypassent jamais le garde-fou
|
||||||
|
- [x] `meta.json`, `status` et le smoke exposent `quality_blockers`, `gate_report`, `gate_v1` et les chapitres `quality_blocked`
|
||||||
|
- [x] Revalidation du protocole qualite:
|
||||||
|
- `ollama:qwen2.5:1.5b` -> `quality_blocked` au garde-fou
|
||||||
|
- `apple-coreml:qwen2.5-0.5b-instruct-onnx` -> `quality_blocked` au garde-fou
|
||||||
|
- `apple-coreml:qwen3.5-4b-onnx-q4f16` -> `provider_failed` en `rewrite`
|
||||||
|
- `ollama:qwen2.5:7b` -> `provider_failed` par timeout client en `draft`
|
||||||
|
- [x] Suite unitaire `python3 -m unittest discover -s tests -v` verte avec 27 tests
|
||||||
|
|
||||||
|
### Lot livre - 8 mars 2026 (durcissement prose)
|
||||||
|
- [x] Prompts `draft_v1` et `rewrite_v1` renforces pour interdire titres, puces et labels de plan visibles
|
||||||
|
- [x] Consignes explicites de prose continue, de scene jouee et de fin de phrase complete
|
||||||
|
- [x] Fix runtime cote `mascarade` avec `OLLAMA_TIMEOUT_SECONDS` configurable et timeout explicite
|
||||||
|
- [x] Rerun reel `ollama:qwen2.5:1.5b` complete a nouveau jusqu'au garde-fou (`499` mots), mais reste `quality_blocked`
|
||||||
|
- [x] Rerun reel `apple-coreml:qwen2.5-0.5b-instruct-onnx` complete jusqu'au garde-fou (`538` mots), mais reste `quality_blocked`
|
||||||
|
|
||||||
|
### Lot livre - 8 mars 2026 (repair + reruns bornes)
|
||||||
|
- [x] Boucle `repair` automatique entre `gate` et `quality_blocked`
|
||||||
|
- [x] Preservation de `draft_v2.md` et ajout des artefacts `repair_vN.md`
|
||||||
|
- [x] Budget `ANE_MAX_TOKENS_REPAIR`, limite `ANE_REPAIR_MAX_PASSES` et override `ANE_REPAIR_FALLBACK_MODEL`
|
||||||
|
- [x] `meta.json`, `status` et le smoke exposent `repair_attempts`, `repair_models`, `repair_latest` et le brouillon final candidat
|
||||||
|
- [x] Timeout provider `urllib` remonte maintenant en `ProviderError`, ce qui marque correctement `failed_stage`
|
||||||
|
- [x] Le warm-up Apple du smoke remonte maintenant un message d'erreur lisible
|
||||||
|
- [x] Le fallback `repair` n'essaie plus automatiquement un autre modele `apple-coreml` au milieu d'un meme smoke; `qwen2.5-0.5b` bascule desormais vers un fallback non-Apple
|
||||||
|
- [x] Suite unitaire etendue a 34 tests verts
|
||||||
|
- [x] Reruns reels sous preset qualite borne a `300s` par requete:
|
||||||
|
- `ollama:qwen2.5:1.5b` -> `failed_stage=structure`
|
||||||
|
- `apple-coreml:qwen2.5-0.5b-instruct-onnx` -> `failed_stage=rewrite`
|
||||||
|
- `apple-coreml:qwen3.5-4b-onnx-q4f16` -> `failed_stage=rewrite`
|
||||||
|
- `ollama:qwen2.5:7b` -> `failed_stage=rewrite`
|
||||||
|
- [x] Conclusion du cycle: la boucle `repair` est livree et preparee; le goulot courant reste `rewrite` tant que les meilleurs candidats n'ont pas ete rejoues
|
||||||
|
|
||||||
|
### Lot livre - 9 mars 2026 (automation des lots utiles)
|
||||||
|
- [x] Orchestrateur central `python3 scripts/run_next_lots.py`
|
||||||
|
- [x] Manifeste versionne `automation/next_lots.toml`
|
||||||
|
- [x] Etat de reprise local et rapports machines dans `automation/state/` et `automation/reports/`
|
||||||
|
- [x] Reutilisation des smokes existants `scripts/smoke_local_generation.sh` et `mascarade/scripts/smoke_openai_compat_ane.sh`
|
||||||
|
- [x] Synchronisation directe des plans/TODOs/readmes/runbooks dans des sections `AUTO-SYNC`
|
||||||
|
- [x] Helper `mascarade/scripts/ensure_apple_models.sh` pour verifier ou installer les trois modeles Apple requis
|
||||||
|
- [x] Helper `mascarade/scripts/prepare_runtime_step.sh` pour preparer les checkpoints semi-autos de restart/switch runtime
|
||||||
|
- [x] Attente courte sur `/models` apres un switch Apple pour eviter les faux checkpoints `aucun modele`
|
||||||
|
- [x] Couverture unitaire du manifeste, des checkpoints Apple, du rendu `AUTO-SYNC` et des helpers shell
|
||||||
|
|
||||||
|
### Lot livre - 9 mars 2026 (priority_models automatise)
|
||||||
|
- [x] Cycle reel `python3 scripts/run_next_lots.py --lot priority_models` termine jusqu'a `tracking_sync`
|
||||||
|
- [x] `apple-coreml:qwen3.5-4b-onnx-q4f16` accepte de bout en bout sous protocole `gate + repair`
|
||||||
|
- [x] `ollama:qwen2.5:7b` atteint `gate`, exerce `repair` en live sur deux passes, puis finit `quality_blocked` avec `outline_like`
|
||||||
|
- [x] Le comparatif local, les TODOs, les README et les runbooks disposent maintenant d'un premier resultat `accepted` sous protocole courant
|
||||||
|
|
||||||
|
## Actif
|
||||||
|
- [x] Aucun suivi actif ici. Voir `TODO_ACTIVE.md`.
|
||||||
|
|
||||||
|
## Bloque
|
||||||
|
- [x] Aucun blocage suivi ici. Voir `TODO_ACTIVE.md`.
|
||||||
|
|
||||||
|
## Prochain ordre
|
||||||
|
- [x] Mettre a jour ce fichier uniquement quand un nouveau lot est reellement termine.
|
||||||
|
|
||||||
|
## Auto-sync
|
||||||
|
## Auto-sync
|
||||||
|
<!-- AUTO-SYNC:ANE-TODO-DONE:START -->
|
||||||
|
- orchestrateur `scripts/run_next_lots.py` disponible
|
||||||
|
- manifeste `automation/next_lots.toml` charge
|
||||||
|
- derniers fichiers de suivi synchronisables via marqueurs `AUTO-SYNC`
|
||||||
|
- dernier cycle automatise observe: 2026-03-09T06:53:02+00:00
|
||||||
|
<!-- AUTO-SYNC:ANE-TODO-DONE:END -->
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
[paths]
|
||||||
|
mascarade_repo = "/Users/electron/mascarade"
|
||||||
|
core_base_url = "http://127.0.0.1:8100"
|
||||||
|
apple_runtime_url = "http://127.0.0.1:8201"
|
||||||
|
ollama_tags_url = "http://127.0.0.1:11435/api/tags"
|
||||||
|
apple_model_ready_timeout_seconds = 30
|
||||||
|
apple_model_poll_interval_seconds = 2
|
||||||
|
|
||||||
|
[smoke]
|
||||||
|
chapter = "02"
|
||||||
|
intention = "Chapitre court. Une femme arrive dans une ville de nuit, trouve un indice simple, et finit sur une decision risquee. Style direct, phrases courtes, ton sobre."
|
||||||
|
timeout_seconds = 300
|
||||||
|
|
||||||
|
[preset]
|
||||||
|
ANE_PROVIDER = "openai_compatible"
|
||||||
|
ANE_BASE_URL = "http://127.0.0.1:8100"
|
||||||
|
ANE_TIMEOUT = "300"
|
||||||
|
ANE_MAX_TOKENS_STRUCTURE = "256"
|
||||||
|
ANE_MAX_TOKENS_DRAFT = "768"
|
||||||
|
ANE_MAX_TOKENS_CRITIQUE = "512"
|
||||||
|
ANE_MAX_TOKENS_REWRITE = "768"
|
||||||
|
ANE_MAX_TOKENS_GATE = "384"
|
||||||
|
ANE_MAX_TOKENS_REPAIR = "512"
|
||||||
|
ANE_MAX_TOKENS_MEMORY = "320"
|
||||||
|
ANE_REPAIR_MAX_PASSES = "2"
|
||||||
|
|
||||||
|
[ensure_models]
|
||||||
|
apple_models = [
|
||||||
|
"qwen2.5-0.5b-instruct-onnx",
|
||||||
|
"qwen3.5-4b-onnx-q4f16",
|
||||||
|
"stateful-mistral7b-instruct-int4-coreml",
|
||||||
|
]
|
||||||
|
ollama_models = [
|
||||||
|
"qwen2.5:7b",
|
||||||
|
"qwen2.5:1.5b",
|
||||||
|
]
|
||||||
|
|
||||||
|
[lots.priority_models]
|
||||||
|
models = [
|
||||||
|
"apple-coreml:qwen3.5-4b-onnx-q4f16",
|
||||||
|
"ollama:qwen2.5:7b",
|
||||||
|
]
|
||||||
|
|
||||||
|
[lots.baselines]
|
||||||
|
models = [
|
||||||
|
"apple-coreml:qwen2.5-0.5b-instruct-onnx",
|
||||||
|
"ollama:qwen2.5:1.5b",
|
||||||
|
]
|
||||||
|
|
||||||
|
[lots.preflight_only]
|
||||||
|
models = [
|
||||||
|
"apple-coreml:stateful-mistral7b-instruct-int4-coreml",
|
||||||
|
]
|
||||||
|
|
||||||
|
[tracking.ane]
|
||||||
|
todo_active = "TODO_ACTIVE.md"
|
||||||
|
todo_done = "TODO_IMPLEMENTE.md"
|
||||||
|
plan = "docs/EXECUTION_PLAN_2026-03-08.md"
|
||||||
|
comparison = "docs/MODEL_COMPARISON_2026-03-08.md"
|
||||||
|
readme = "README.md"
|
||||||
|
runbook = "docs/runbooks/LOCAL_GENERATION.md"
|
||||||
|
|
||||||
|
[tracking.mascarade]
|
||||||
|
todo = "TODO_AI_NOVEL_ENGINE.md"
|
||||||
|
plan = "docs/EXECUTION_PLAN_2026-03-08.md"
|
||||||
|
readme = "README.md"
|
||||||
|
runbook = "docs/RUNBOOK_APPLE_LLM_LOCAL.md"
|
||||||
|
|
||||||
|
[next_actions]
|
||||||
|
rewrite_compaction = "Compacter ou reduire `rewrite_v1` et ses budgets pour faire passer au moins `apple-coreml:qwen3.5-4b-onnx-q4f16` ou `ollama:qwen2.5:7b` jusqu'a `gate`."
|
||||||
+211
-61
@@ -1,8 +1,13 @@
|
|||||||
import sys
|
from __future__ import annotations
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from core.chapters import ChapterConflictError, ChapterError, ChapterId, resolve_chapter_file
|
||||||
|
from core.generation.pipeline import GenerationPipeline
|
||||||
|
from core.generation.provider import ProviderConfigurationError, ProviderError
|
||||||
from core.project.loader import ProjectState
|
from core.project.loader import ProjectState
|
||||||
from core.intention.gate import IntentionGate
|
|
||||||
|
|
||||||
|
|
||||||
def cmd_status(root: Path):
|
def cmd_status(root: Path):
|
||||||
@@ -11,92 +16,237 @@ def cmd_status(root: Path):
|
|||||||
|
|
||||||
print("\nAI Novel Engine — Project Status\n")
|
print("\nAI Novel Engine — Project Status\n")
|
||||||
|
|
||||||
if state["current_chapter"] is None:
|
current = state["current_chapter"]
|
||||||
print("📄 Aucun chapitre détecté.")
|
if current is None:
|
||||||
|
print("Chapitre courant : aucun")
|
||||||
else:
|
else:
|
||||||
print(f"📄 Chapitre courant : {state['current_chapter']}")
|
print(f"Chapitre courant : {current}")
|
||||||
|
|
||||||
print(f"📐 Structure présente : {state['has_structure']}")
|
print("\nDossiers:")
|
||||||
print(f"🧠 Mémoire présente : {state['has_memory']}")
|
for label, key in (
|
||||||
print("\n(Prochaine étape : définir une intention)\n")
|
("Structure", "structure"),
|
||||||
|
("Brouillons", "drafts"),
|
||||||
|
("Manuscrit", "manuscript"),
|
||||||
|
("Memoire", "memory"),
|
||||||
|
):
|
||||||
|
print(f"- {label:<10}: {state['directories'][key]}")
|
||||||
|
|
||||||
|
latest_drafts = state["latest_drafts"]
|
||||||
|
if latest_drafts:
|
||||||
|
print("\nDerniers brouillons:")
|
||||||
|
for chapter_slug, draft_name in sorted(latest_drafts.items()):
|
||||||
|
print(f"- {chapter_slug}: {draft_name}")
|
||||||
|
else:
|
||||||
|
print("\nDerniers brouillons: aucun")
|
||||||
|
|
||||||
|
latest_repairs = state["latest_repairs"]
|
||||||
|
if latest_repairs:
|
||||||
|
print("\nDernières réparations:")
|
||||||
|
for chapter_slug, repair_name in sorted(latest_repairs.items()):
|
||||||
|
print(f"- {chapter_slug}: {repair_name}")
|
||||||
|
else:
|
||||||
|
print("\nDernières réparations: aucune")
|
||||||
|
|
||||||
|
failures = state["failed_chapters"]
|
||||||
|
if failures:
|
||||||
|
print("\nChapitres en échec:")
|
||||||
|
for item in failures:
|
||||||
|
retry_suffix = ""
|
||||||
|
if item["retry_stages"]:
|
||||||
|
retry_suffix = f" | réessais JSON: {', '.join(item['retry_stages'])}"
|
||||||
|
status_message = f" | message: {item['last_status_message']}" if item["last_status_message"] else ""
|
||||||
|
print(
|
||||||
|
f"- {item['chapter']}: status={item['status']} | failed_stage={item['failed_stage']} | meta={item['meta_path']}{retry_suffix}{status_message}"
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
print("\nChapitres en échec: aucun")
|
||||||
|
|
||||||
|
quality_blocked = state["quality_blocked_chapters"]
|
||||||
|
if quality_blocked:
|
||||||
|
print("\nBloqués par garde-fou:")
|
||||||
|
for item in quality_blocked:
|
||||||
|
retry_suffix = ""
|
||||||
|
if item["retry_stages"]:
|
||||||
|
retry_suffix = f" | réessais JSON: {', '.join(item['retry_stages'])}"
|
||||||
|
blockers_suffix = ""
|
||||||
|
if item["quality_blockers"]:
|
||||||
|
blockers_suffix = f" | blockers: {', '.join(item['quality_blockers'])}"
|
||||||
|
repair_suffix = ""
|
||||||
|
if item["repair_attempts"]:
|
||||||
|
repair_models = ", ".join(item["repair_models"]) if item["repair_models"] else "provider_courant"
|
||||||
|
repair_suffix = f" | réparations: {item['repair_attempts']} ({repair_models})"
|
||||||
|
status_message = f" | message: {item['last_status_message']}" if item["last_status_message"] else ""
|
||||||
|
print(
|
||||||
|
f"- {item['chapter']}: status={item['status']} | failed_stage={item['failed_stage']} | brouillon={item['draft_path']} | gate={item['gate_path']} | meta={item['meta_path']}{blockers_suffix}{repair_suffix}{retry_suffix}{status_message}"
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
print("\nBloqués par garde-fou: aucun")
|
||||||
|
|
||||||
|
awaiting_acceptance = state["awaiting_acceptance"]
|
||||||
|
if awaiting_acceptance:
|
||||||
|
print("\nEn attente de validation:")
|
||||||
|
for item in awaiting_acceptance:
|
||||||
|
retry_suffix = ""
|
||||||
|
if item["retry_stages"]:
|
||||||
|
retry_suffix = f" | réessais JSON: {', '.join(item['retry_stages'])}"
|
||||||
|
repair_suffix = ""
|
||||||
|
if item["repair_attempts"]:
|
||||||
|
repair_models = ", ".join(item["repair_models"]) if item["repair_models"] else "provider_courant"
|
||||||
|
repair_suffix = f" | réparations: {item['repair_attempts']} ({repair_models})"
|
||||||
|
status_message = f" | message: {item['last_status_message']}" if item["last_status_message"] else ""
|
||||||
|
print(
|
||||||
|
f"- {item['chapter']}: status={item['status']} | brouillon={item['draft_path']} | critique={item['critique_path']} | gate={item['gate_path']}{repair_suffix}{retry_suffix}{status_message}"
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
print("\nEn attente de validation: aucun")
|
||||||
|
|
||||||
|
print("")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def cmd_intention_create(root: Path):
|
def cmd_intention_create(root: Path, chapter_value: str | None = None, input_func=input):
|
||||||
intentions_dir = root / "notes" / "intentions"
|
intentions_dir = root / "notes" / "intentions"
|
||||||
intentions_dir.mkdir(parents=True, exist_ok=True)
|
intentions_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
chap = input("Numéro du chapitre (ex: 08) : ").strip()
|
raw_chapter = chapter_value or input_func("Numéro du chapitre (ex: 08) : ").strip()
|
||||||
if not chap:
|
chapter = ChapterId.parse(raw_chapter)
|
||||||
print("❌ Numéro de chapitre requis.")
|
|
||||||
return
|
|
||||||
|
|
||||||
path = intentions_dir / f"chapitre_{chap}.md"
|
path = resolve_chapter_file(intentions_dir, chapter)
|
||||||
if path.exists():
|
if path.exists():
|
||||||
print(f"⚠️ Une intention existe déjà : {path}")
|
print(f"Une intention existe déjà : {path}")
|
||||||
return
|
return 1
|
||||||
|
|
||||||
print("\nDécris l’intention (finir par Ctrl+D / Ctrl+Z):\n")
|
print("\nDécris l’intention (finir par Ctrl+D / Ctrl+Z):\n")
|
||||||
lines = []
|
lines = []
|
||||||
try:
|
try:
|
||||||
while True:
|
while True:
|
||||||
lines.append(input())
|
lines.append(input_func(""))
|
||||||
except EOFError:
|
except EOFError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
content = "\n".join(lines).strip()
|
content = "\n".join(line for line in lines if line is not None).strip()
|
||||||
if not content:
|
if not content:
|
||||||
print("❌ Intention vide. Annulé.")
|
print("Intention vide. Annulé.")
|
||||||
return
|
return 1
|
||||||
|
|
||||||
path.write_text(
|
canonical_path = intentions_dir / chapter.filename
|
||||||
f"# Intention — Chapitre {chap}\n\n{content}\n",
|
canonical_path.write_text(
|
||||||
encoding="utf-8"
|
f"# Intention — Chapitre {chapter.label}\n\n{content}\n",
|
||||||
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
|
|
||||||
print(f"✅ Intention créée : {path}\n")
|
print(f"Intention créée : {canonical_path}\n")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def cmd_write(root: Path):
|
def _approval_callback_from_flags(force_accept: bool | None):
|
||||||
gate = IntentionGate(root)
|
if force_accept is None:
|
||||||
|
return None
|
||||||
|
return lambda _report, _path: force_accept
|
||||||
|
|
||||||
|
|
||||||
|
def cmd_generate_chapter(
|
||||||
|
root: Path,
|
||||||
|
chapter_value: str,
|
||||||
|
provider=None,
|
||||||
|
input_func=input,
|
||||||
|
*,
|
||||||
|
force_accept: bool | None = None,
|
||||||
|
):
|
||||||
|
pipeline = GenerationPipeline(root, provider=provider, input_func=input_func)
|
||||||
|
outcome = pipeline.generate_chapter(
|
||||||
|
chapter_value,
|
||||||
|
approval_callback=_approval_callback_from_flags(force_accept),
|
||||||
|
)
|
||||||
|
|
||||||
|
print("")
|
||||||
|
print(f"Chapitre traité : {outcome.chapter_id.slug}")
|
||||||
|
print(f"Statut : {outcome.status}")
|
||||||
|
print(f"Brouillon final : {outcome.draft_path}")
|
||||||
|
print(f"Critique : {outcome.critique_path}")
|
||||||
|
print(f"Garde-fou : {outcome.gate_path}")
|
||||||
|
print(f"Métadonnées : {outcome.meta_path}")
|
||||||
|
if outcome.accepted and outcome.manuscript_path is not None:
|
||||||
|
print(f"Manuscrit : {outcome.manuscript_path}")
|
||||||
|
else:
|
||||||
|
print("Manuscrit : non promu")
|
||||||
|
if outcome.quality_blockers:
|
||||||
|
print(f"Blockers : {', '.join(outcome.quality_blockers)}")
|
||||||
|
print("")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _add_approval_flags(parser: argparse.ArgumentParser) -> None:
|
||||||
|
group = parser.add_mutually_exclusive_group()
|
||||||
|
group.add_argument("--approve", action="store_true", help="Promouvoir sans confirmation interactive.")
|
||||||
|
group.add_argument("--reject", action="store_true", help="Refuser sans confirmation interactive.")
|
||||||
|
|
||||||
|
|
||||||
|
def _force_accept_from_namespace(namespace: argparse.Namespace) -> bool | None:
|
||||||
|
if getattr(namespace, "approve", False):
|
||||||
|
return True
|
||||||
|
if getattr(namespace, "reject", False):
|
||||||
|
return False
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def build_parser() -> argparse.ArgumentParser:
|
||||||
|
parser = argparse.ArgumentParser(prog="python3 -m cli.main")
|
||||||
|
subparsers = parser.add_subparsers(dest="command")
|
||||||
|
|
||||||
|
subparsers.add_parser("status")
|
||||||
|
|
||||||
|
intention_parser = subparsers.add_parser("intention")
|
||||||
|
intention_subparsers = intention_parser.add_subparsers(dest="intention_command")
|
||||||
|
intention_create = intention_subparsers.add_parser("create")
|
||||||
|
intention_create.add_argument("--chapter")
|
||||||
|
|
||||||
|
generate_parser = subparsers.add_parser("generate")
|
||||||
|
generate_subparsers = generate_parser.add_subparsers(dest="generate_target")
|
||||||
|
generate_chapter = generate_subparsers.add_parser("chapter")
|
||||||
|
generate_chapter.add_argument("--chapter", required=True)
|
||||||
|
_add_approval_flags(generate_chapter)
|
||||||
|
|
||||||
|
write_parser = subparsers.add_parser("write")
|
||||||
|
write_parser.add_argument("--chapter", required=True)
|
||||||
|
_add_approval_flags(write_parser)
|
||||||
|
|
||||||
|
return parser
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None, root: Path | None = None):
|
||||||
|
args = argv if argv is not None else sys.argv[1:]
|
||||||
|
project_root = root or Path.cwd()
|
||||||
|
parser = build_parser()
|
||||||
|
namespace = parser.parse_args(args)
|
||||||
|
|
||||||
|
if not args or namespace.command == "status":
|
||||||
|
return cmd_status(project_root)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
gate.assert_intention()
|
if namespace.command == "intention" and namespace.intention_command == "create":
|
||||||
except RuntimeError as e:
|
return cmd_intention_create(project_root, chapter_value=namespace.chapter)
|
||||||
print("\n⛔ ÉCRITURE BLOQUÉE\n")
|
|
||||||
print(str(e))
|
|
||||||
print("\n➡️ Utilise : python3 -m cli.main intention create\n")
|
|
||||||
return
|
|
||||||
|
|
||||||
print("\n✅ Intention détectée.")
|
if namespace.command == "generate" and namespace.generate_target == "chapter":
|
||||||
print("✍️ Écriture autorisée (génération non implémentée).\n")
|
return cmd_generate_chapter(
|
||||||
|
project_root,
|
||||||
|
namespace.chapter,
|
||||||
|
force_accept=_force_accept_from_namespace(namespace),
|
||||||
|
)
|
||||||
|
|
||||||
|
if namespace.command == "write":
|
||||||
|
return cmd_generate_chapter(
|
||||||
|
project_root,
|
||||||
|
namespace.chapter,
|
||||||
|
force_accept=_force_accept_from_namespace(namespace),
|
||||||
|
)
|
||||||
|
except (ChapterError, ChapterConflictError, RuntimeError, ProviderConfigurationError, ProviderError) as exc:
|
||||||
|
print(f"\nErreur: {exc}\n")
|
||||||
|
return 1
|
||||||
|
|
||||||
def main():
|
parser.print_help()
|
||||||
root = Path.cwd()
|
return 1
|
||||||
|
|
||||||
# Aucun argument → status
|
|
||||||
if len(sys.argv) == 1:
|
|
||||||
cmd_status(root)
|
|
||||||
return
|
|
||||||
|
|
||||||
# intention create
|
|
||||||
if sys.argv[1] == "intention" and len(sys.argv) >= 3:
|
|
||||||
if sys.argv[2] == "create":
|
|
||||||
cmd_intention_create(root)
|
|
||||||
return
|
|
||||||
|
|
||||||
# write
|
|
||||||
if sys.argv[1] == "write":
|
|
||||||
cmd_write(root)
|
|
||||||
return
|
|
||||||
|
|
||||||
# aide
|
|
||||||
print("\nCommande inconnue.\n")
|
|
||||||
print("Commandes disponibles :")
|
|
||||||
print(" python3 -m cli.main → status")
|
|
||||||
print(" python3 -m cli.main intention create")
|
|
||||||
print(" python3 -m cli.main write\n")
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
main()
|
raise SystemExit(main())
|
||||||
|
|||||||
@@ -0,0 +1,127 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
|
||||||
|
|
||||||
|
_CHAPTER_PATTERN = re.compile(r"^chapitre_(\d+)$", re.IGNORECASE)
|
||||||
|
_DIGITS_PATTERN = re.compile(r"^\d+$")
|
||||||
|
|
||||||
|
|
||||||
|
class ChapterError(ValueError):
|
||||||
|
"""Raised when a chapter identifier is invalid."""
|
||||||
|
|
||||||
|
|
||||||
|
class ChapterConflictError(ChapterError):
|
||||||
|
"""Raised when both canonical and legacy files exist for the same chapter."""
|
||||||
|
|
||||||
|
def __init__(self, chapter: "ChapterId", paths: list[Path]):
|
||||||
|
joined = ", ".join(str(path) for path in paths)
|
||||||
|
super().__init__(
|
||||||
|
f"Conflit de chapitre pour {chapter.slug}: plusieurs fichiers existent ({joined})."
|
||||||
|
)
|
||||||
|
self.chapter = chapter
|
||||||
|
self.paths = paths
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, order=True)
|
||||||
|
class ChapterId:
|
||||||
|
number: int
|
||||||
|
|
||||||
|
def __post_init__(self):
|
||||||
|
if self.number <= 0:
|
||||||
|
raise ChapterError("Le numéro de chapitre doit être strictement positif.")
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def parse(cls, value: object) -> "ChapterId":
|
||||||
|
return cls(parse_chapter_number(value))
|
||||||
|
|
||||||
|
@property
|
||||||
|
def label(self) -> str:
|
||||||
|
return f"{self.number:02d}"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def slug(self) -> str:
|
||||||
|
return f"chapitre_{self.label}"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def filename(self) -> str:
|
||||||
|
return f"{self.slug}.md"
|
||||||
|
|
||||||
|
def __str__(self) -> str:
|
||||||
|
return self.slug
|
||||||
|
|
||||||
|
|
||||||
|
def parse_chapter_number(value: object) -> int:
|
||||||
|
if isinstance(value, ChapterId):
|
||||||
|
return value.number
|
||||||
|
|
||||||
|
if isinstance(value, int) and not isinstance(value, bool):
|
||||||
|
return value
|
||||||
|
|
||||||
|
if isinstance(value, Path):
|
||||||
|
text = value.stem
|
||||||
|
else:
|
||||||
|
text = str(value).strip()
|
||||||
|
|
||||||
|
if not text:
|
||||||
|
raise ChapterError("Numéro de chapitre requis.")
|
||||||
|
|
||||||
|
candidate = text
|
||||||
|
if "/" in candidate or "\\" in candidate or candidate.endswith(".md"):
|
||||||
|
candidate = Path(candidate).stem
|
||||||
|
|
||||||
|
match = _CHAPTER_PATTERN.fullmatch(candidate)
|
||||||
|
if match:
|
||||||
|
return int(match.group(1))
|
||||||
|
|
||||||
|
if _DIGITS_PATTERN.fullmatch(candidate):
|
||||||
|
return int(candidate)
|
||||||
|
|
||||||
|
raise ChapterError(f"Identifiant de chapitre invalide: {value!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def discover_chapter_files(directory: Path) -> list[tuple[ChapterId, Path]]:
|
||||||
|
if not directory.exists():
|
||||||
|
return []
|
||||||
|
|
||||||
|
discovered: list[tuple[ChapterId, Path]] = []
|
||||||
|
for path in sorted(directory.glob("chapitre_*.md")):
|
||||||
|
try:
|
||||||
|
chapter = ChapterId.parse(path.stem)
|
||||||
|
except ChapterError:
|
||||||
|
continue
|
||||||
|
discovered.append((chapter, path))
|
||||||
|
return discovered
|
||||||
|
|
||||||
|
|
||||||
|
def discover_chapter_dirs(directory: Path) -> list[tuple[ChapterId, Path]]:
|
||||||
|
if not directory.exists():
|
||||||
|
return []
|
||||||
|
|
||||||
|
discovered: list[tuple[ChapterId, Path]] = []
|
||||||
|
for path in sorted(directory.glob("chapitre_*")):
|
||||||
|
if not path.is_dir():
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
chapter = ChapterId.parse(path.name)
|
||||||
|
except ChapterError:
|
||||||
|
continue
|
||||||
|
discovered.append((chapter, path))
|
||||||
|
return discovered
|
||||||
|
|
||||||
|
|
||||||
|
def collect_matching_chapter_files(directory: Path, chapter: ChapterId) -> list[Path]:
|
||||||
|
paths = [path for candidate, path in discover_chapter_files(directory) if candidate == chapter]
|
||||||
|
unique_paths = sorted(set(paths))
|
||||||
|
return unique_paths
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_chapter_file(directory: Path, chapter: ChapterId) -> Path:
|
||||||
|
matches = collect_matching_chapter_files(directory, chapter)
|
||||||
|
if len(matches) > 1:
|
||||||
|
raise ChapterConflictError(chapter, matches)
|
||||||
|
if matches:
|
||||||
|
return matches[0]
|
||||||
|
return directory / chapter.filename
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""Generation pipeline primitives."""
|
||||||
@@ -0,0 +1,313 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
|
||||||
|
from core.chapters import ChapterId
|
||||||
|
|
||||||
|
|
||||||
|
def _strip_code_fence(text: str) -> str:
|
||||||
|
payload = text.strip()
|
||||||
|
if not payload.startswith("```"):
|
||||||
|
return payload
|
||||||
|
lines = payload.splitlines()
|
||||||
|
if len(lines) >= 3 and lines[-1].strip() == "```":
|
||||||
|
return "\n".join(lines[1:-1]).strip()
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def _remove_trailing_commas(payload: str) -> str:
|
||||||
|
return re.sub(r",(\s*[}\]])", r"\1", payload)
|
||||||
|
|
||||||
|
|
||||||
|
def _extract_json_object(payload: str) -> str | None:
|
||||||
|
start = payload.find("{")
|
||||||
|
if start == -1:
|
||||||
|
return None
|
||||||
|
|
||||||
|
depth = 0
|
||||||
|
in_string = False
|
||||||
|
escaped = False
|
||||||
|
for index in range(start, len(payload)):
|
||||||
|
char = payload[index]
|
||||||
|
if in_string:
|
||||||
|
if escaped:
|
||||||
|
escaped = False
|
||||||
|
elif char == "\\":
|
||||||
|
escaped = True
|
||||||
|
elif char == '"':
|
||||||
|
in_string = False
|
||||||
|
continue
|
||||||
|
if char == '"':
|
||||||
|
in_string = True
|
||||||
|
elif char == "{":
|
||||||
|
depth += 1
|
||||||
|
elif char == "}":
|
||||||
|
depth -= 1
|
||||||
|
if depth == 0:
|
||||||
|
return payload[start : index + 1]
|
||||||
|
return payload[start:]
|
||||||
|
|
||||||
|
|
||||||
|
def _close_json_delimiters(payload: str) -> str:
|
||||||
|
stack: list[str] = []
|
||||||
|
in_string = False
|
||||||
|
escaped = False
|
||||||
|
for char in payload:
|
||||||
|
if in_string:
|
||||||
|
if escaped:
|
||||||
|
escaped = False
|
||||||
|
elif char == "\\":
|
||||||
|
escaped = True
|
||||||
|
elif char == '"':
|
||||||
|
in_string = False
|
||||||
|
continue
|
||||||
|
if char == '"':
|
||||||
|
in_string = True
|
||||||
|
elif char == "{":
|
||||||
|
stack.append("}")
|
||||||
|
elif char == "[":
|
||||||
|
stack.append("]")
|
||||||
|
elif char in {"}", "]"} and stack and char == stack[-1]:
|
||||||
|
stack.pop()
|
||||||
|
|
||||||
|
repaired = payload.rstrip()
|
||||||
|
if repaired.endswith(","):
|
||||||
|
repaired = repaired[:-1].rstrip()
|
||||||
|
if in_string:
|
||||||
|
repaired += '"'
|
||||||
|
return repaired + "".join(reversed(stack))
|
||||||
|
|
||||||
|
|
||||||
|
def _json_candidates(text: str) -> list[str]:
|
||||||
|
payload = _strip_code_fence(text)
|
||||||
|
candidates = [payload]
|
||||||
|
|
||||||
|
extracted = _extract_json_object(payload)
|
||||||
|
if extracted and extracted not in candidates:
|
||||||
|
candidates.append(extracted)
|
||||||
|
|
||||||
|
repaired: list[str] = []
|
||||||
|
for candidate in list(candidates):
|
||||||
|
trimmed = _remove_trailing_commas(candidate)
|
||||||
|
if trimmed not in candidates and trimmed not in repaired:
|
||||||
|
repaired.append(trimmed)
|
||||||
|
closed = _close_json_delimiters(trimmed)
|
||||||
|
if closed not in candidates and closed not in repaired:
|
||||||
|
repaired.append(closed)
|
||||||
|
candidates.extend(repaired)
|
||||||
|
return candidates
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_json_object(text: str) -> dict[str, object]:
|
||||||
|
last_error: Exception | None = None
|
||||||
|
for candidate in _json_candidates(text):
|
||||||
|
try:
|
||||||
|
data = json.loads(candidate)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
last_error = exc
|
||||||
|
continue
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
raise ValueError("La réponse JSON attendue doit être un objet.")
|
||||||
|
return data
|
||||||
|
raise ValueError(str(last_error) if last_error else "Réponse JSON illisible.")
|
||||||
|
|
||||||
|
|
||||||
|
def _string_list(value: object) -> list[str]:
|
||||||
|
if not isinstance(value, list):
|
||||||
|
return []
|
||||||
|
return [str(item).strip() for item in value if str(item).strip()]
|
||||||
|
|
||||||
|
|
||||||
|
def _record_list(value: object, required_key: str) -> list[dict[str, str]]:
|
||||||
|
if not isinstance(value, list):
|
||||||
|
return []
|
||||||
|
|
||||||
|
normalized: list[dict[str, str]] = []
|
||||||
|
for item in value:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
record = {str(key): str(val).strip() for key, val in item.items() if str(val).strip()}
|
||||||
|
if record.get(required_key):
|
||||||
|
normalized.append(record)
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class StructurePlan:
|
||||||
|
chapter_id: ChapterId
|
||||||
|
markdown: str
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ControlReport:
|
||||||
|
summary: str
|
||||||
|
deviations: list[str]
|
||||||
|
recommendations: list[str]
|
||||||
|
rewrite_required: bool
|
||||||
|
raw: dict[str, object] = field(default_factory=dict)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_response_text(cls, text: str) -> "ControlReport":
|
||||||
|
raw = _parse_json_object(text)
|
||||||
|
summary = str(raw.get("summary", "")).strip() or "Aucun résumé fourni."
|
||||||
|
deviations = _string_list(raw.get("deviations"))
|
||||||
|
recommendations = _string_list(raw.get("recommendations"))
|
||||||
|
rewrite_required = bool(raw.get("rewrite_required", deviations or recommendations))
|
||||||
|
return cls(
|
||||||
|
summary=summary,
|
||||||
|
deviations=deviations,
|
||||||
|
recommendations=recommendations,
|
||||||
|
rewrite_required=rewrite_required,
|
||||||
|
raw=raw,
|
||||||
|
)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"summary": self.summary,
|
||||||
|
"deviations": list(self.deviations),
|
||||||
|
"recommendations": list(self.recommendations),
|
||||||
|
"rewrite_required": self.rewrite_required,
|
||||||
|
}
|
||||||
|
|
||||||
|
def to_markdown(self, chapter_id: ChapterId) -> str:
|
||||||
|
verdict = "oui" if self.rewrite_required else "non"
|
||||||
|
deviations = "\n".join(f"- {item}" for item in self.deviations) or "- Aucun écart majeur."
|
||||||
|
recommendations = "\n".join(f"- {item}" for item in self.recommendations) or "- Aucune recommandation."
|
||||||
|
return (
|
||||||
|
f"# Critique — {chapter_id.slug}\n\n"
|
||||||
|
f"## Résumé\n{self.summary}\n\n"
|
||||||
|
f"## Réécriture requise\n{verdict}\n\n"
|
||||||
|
f"## Écarts\n{deviations}\n\n"
|
||||||
|
f"## Recommandations\n{recommendations}\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class MemoryUpdate:
|
||||||
|
chapter_summary: str
|
||||||
|
characters: list[dict[str, str]]
|
||||||
|
locations: list[dict[str, str]]
|
||||||
|
timeline_events: list[dict[str, str]]
|
||||||
|
raw: dict[str, object] = field(default_factory=dict)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_response_text(cls, text: str) -> "MemoryUpdate":
|
||||||
|
raw = _parse_json_object(text)
|
||||||
|
chapter_summary = str(raw.get("summary", "")).strip() or "Résumé indisponible."
|
||||||
|
characters = _record_list(raw.get("characters"), "name")
|
||||||
|
locations = _record_list(raw.get("locations"), "name")
|
||||||
|
timeline_events = _record_list(raw.get("timeline_events"), "event")
|
||||||
|
return cls(
|
||||||
|
chapter_summary=chapter_summary,
|
||||||
|
characters=characters,
|
||||||
|
locations=locations,
|
||||||
|
timeline_events=timeline_events,
|
||||||
|
raw=raw,
|
||||||
|
)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"summary": self.chapter_summary,
|
||||||
|
"characters": list(self.characters),
|
||||||
|
"locations": list(self.locations),
|
||||||
|
"timeline_events": list(self.timeline_events),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ManuscriptGateReport:
|
||||||
|
ready_for_manuscript: bool
|
||||||
|
summary: str
|
||||||
|
blockers: list[str]
|
||||||
|
recommendations: list[str]
|
||||||
|
heuristic_blockers: list[str]
|
||||||
|
raw: dict[str, object] = field(default_factory=dict)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_response_text(cls, text: str) -> "ManuscriptGateReport":
|
||||||
|
raw = _parse_json_object(text)
|
||||||
|
blockers = _string_list(raw.get("blockers"))
|
||||||
|
heuristic_blockers = _string_list(raw.get("heuristic_blockers"))
|
||||||
|
recommendations = _string_list(raw.get("recommendations"))
|
||||||
|
ready_default = not blockers and not heuristic_blockers
|
||||||
|
ready_for_manuscript = bool(raw.get("ready_for_manuscript", ready_default))
|
||||||
|
summary = str(raw.get("summary", "")).strip() or "Diagnostic manuscrit indisponible."
|
||||||
|
return cls(
|
||||||
|
ready_for_manuscript=ready_for_manuscript and not blockers and not heuristic_blockers,
|
||||||
|
summary=summary,
|
||||||
|
blockers=blockers,
|
||||||
|
recommendations=recommendations,
|
||||||
|
heuristic_blockers=heuristic_blockers,
|
||||||
|
raw=raw,
|
||||||
|
)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_heuristics(
|
||||||
|
cls,
|
||||||
|
*,
|
||||||
|
blockers: list[str],
|
||||||
|
recommendations: list[str],
|
||||||
|
summary: str,
|
||||||
|
) -> "ManuscriptGateReport":
|
||||||
|
return cls(
|
||||||
|
ready_for_manuscript=False,
|
||||||
|
summary=summary,
|
||||||
|
blockers=list(blockers),
|
||||||
|
recommendations=list(recommendations),
|
||||||
|
heuristic_blockers=list(blockers),
|
||||||
|
raw={},
|
||||||
|
)
|
||||||
|
|
||||||
|
def all_blockers(self) -> list[str]:
|
||||||
|
ordered: list[str] = []
|
||||||
|
for value in [*self.heuristic_blockers, *self.blockers]:
|
||||||
|
if value not in ordered:
|
||||||
|
ordered.append(value)
|
||||||
|
return ordered
|
||||||
|
|
||||||
|
def to_dict(self) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"ready_for_manuscript": self.ready_for_manuscript,
|
||||||
|
"summary": self.summary,
|
||||||
|
"blockers": list(self.blockers),
|
||||||
|
"recommendations": list(self.recommendations),
|
||||||
|
"heuristic_blockers": list(self.heuristic_blockers),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GenerationContext:
|
||||||
|
root: Path
|
||||||
|
chapter_id: ChapterId
|
||||||
|
intention_path: Path
|
||||||
|
intention_text: str
|
||||||
|
structure_path: Path
|
||||||
|
draft_dir: Path
|
||||||
|
draft_v1_path: Path
|
||||||
|
critique_path: Path
|
||||||
|
draft_v2_path: Path
|
||||||
|
gate_path: Path
|
||||||
|
meta_path: Path
|
||||||
|
manuscript_path: Path
|
||||||
|
memory_summary_path: Path
|
||||||
|
memory_index_dir: Path
|
||||||
|
story_context: str
|
||||||
|
|
||||||
|
def repair_path(self, attempt: int) -> Path:
|
||||||
|
return self.draft_dir / f"repair_v{attempt}.md"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GenerationOutcome:
|
||||||
|
chapter_id: ChapterId
|
||||||
|
accepted: bool
|
||||||
|
status: str
|
||||||
|
draft_path: Path
|
||||||
|
critique_path: Path
|
||||||
|
gate_path: Path
|
||||||
|
meta_path: Path
|
||||||
|
manuscript_path: Path | None
|
||||||
|
quality_blockers: list[str] = field(default_factory=list)
|
||||||
@@ -0,0 +1,953 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
from typing import Callable, TypeVar
|
||||||
|
|
||||||
|
from core.chapters import ChapterId, resolve_chapter_file
|
||||||
|
from core.generation.models import (
|
||||||
|
ControlReport,
|
||||||
|
GenerationContext,
|
||||||
|
GenerationOutcome,
|
||||||
|
ManuscriptGateReport,
|
||||||
|
MemoryUpdate,
|
||||||
|
StructurePlan,
|
||||||
|
)
|
||||||
|
from core.generation.provider import (
|
||||||
|
clone_provider_with_model,
|
||||||
|
GenerationProvider,
|
||||||
|
GenerationRequest,
|
||||||
|
OpenAICompatibleProvider,
|
||||||
|
ProviderError,
|
||||||
|
build_provider_from_env,
|
||||||
|
)
|
||||||
|
from core.intention.gate import IntentionGate
|
||||||
|
from core.prompts import PromptStore
|
||||||
|
|
||||||
|
|
||||||
|
ApprovalCallback = Callable[[ControlReport, Path], bool]
|
||||||
|
OutputCallback = Callable[[str], None]
|
||||||
|
ParsedStagePayload = TypeVar("ParsedStagePayload")
|
||||||
|
|
||||||
|
|
||||||
|
class GenerationPipeline:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
root: Path,
|
||||||
|
provider: GenerationProvider | None = None,
|
||||||
|
prompt_store: PromptStore | None = None,
|
||||||
|
input_func: Callable[[str], str] = input,
|
||||||
|
output_func: OutputCallback = print,
|
||||||
|
):
|
||||||
|
self.root = root
|
||||||
|
self.provider = provider
|
||||||
|
self.prompt_store = prompt_store or PromptStore(root)
|
||||||
|
self.input_func = input_func
|
||||||
|
self.output_func = output_func
|
||||||
|
self.intention_gate = IntentionGate(root)
|
||||||
|
|
||||||
|
def generate_chapter(
|
||||||
|
self,
|
||||||
|
chapter: object,
|
||||||
|
approval_callback: ApprovalCallback | None = None,
|
||||||
|
) -> GenerationOutcome:
|
||||||
|
chapter_id = ChapterId.parse(chapter)
|
||||||
|
self.intention_gate.assert_intention(chapter_id)
|
||||||
|
context = self._build_context(chapter_id)
|
||||||
|
provider = self.provider or build_provider_from_env()
|
||||||
|
metadata = self._initial_metadata(context, provider)
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
|
||||||
|
structure_plan: StructurePlan | None = None
|
||||||
|
draft_v1: str | None = None
|
||||||
|
control_report: ControlReport | None = None
|
||||||
|
draft_v2: str | None = None
|
||||||
|
gate_report: ManuscriptGateReport | None = None
|
||||||
|
current_candidate_text: str | None = None
|
||||||
|
current_candidate_path: Path = context.draft_v2_path
|
||||||
|
current_stage = "setup"
|
||||||
|
|
||||||
|
try:
|
||||||
|
current_stage = "structure"
|
||||||
|
structure_plan = self._generate_structure(provider, context, metadata)
|
||||||
|
self._write_text(context.structure_path, structure_plan.markdown)
|
||||||
|
self._complete_stage(metadata, current_stage)
|
||||||
|
self._set_status(metadata, "structure_ready", "Structure générée.")
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
|
||||||
|
current_stage = "draft"
|
||||||
|
draft_v1 = self._generate_draft(provider, context, structure_plan, metadata)
|
||||||
|
self._write_text(context.draft_v1_path, draft_v1)
|
||||||
|
self._complete_stage(metadata, current_stage)
|
||||||
|
self._set_status(metadata, "draft_ready", "Brouillon initial généré.")
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
|
||||||
|
current_stage = "critique"
|
||||||
|
control_report = self._generate_control_report(provider, context, structure_plan, draft_v1, metadata)
|
||||||
|
self._write_text(context.critique_path, control_report.to_markdown(context.chapter_id))
|
||||||
|
self._complete_stage(metadata, current_stage)
|
||||||
|
self._set_status(metadata, "critique_ready", "Critique structurée générée.")
|
||||||
|
metadata["control_report"] = control_report.to_dict()
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
|
||||||
|
current_stage = "rewrite"
|
||||||
|
draft_v2 = self._rewrite_draft(provider, context, structure_plan, draft_v1, control_report, metadata)
|
||||||
|
self._write_text(context.draft_v2_path, draft_v2)
|
||||||
|
self._complete_stage(metadata, current_stage)
|
||||||
|
self._set_status(metadata, "rewrite_ready", "Brouillon final généré, contrôle manuscrit en cours.")
|
||||||
|
metadata["draft_final"] = str(context.draft_v2_path)
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
current_candidate_text = draft_v2
|
||||||
|
current_candidate_path = context.draft_v2_path
|
||||||
|
|
||||||
|
current_stage = "gate"
|
||||||
|
gate_report = self._generate_manuscript_gate_report(
|
||||||
|
provider,
|
||||||
|
context,
|
||||||
|
structure_plan,
|
||||||
|
current_candidate_text,
|
||||||
|
metadata,
|
||||||
|
)
|
||||||
|
self._persist_gate_report(metadata, context, gate_report, current_candidate_path)
|
||||||
|
self._complete_stage(metadata, current_stage)
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
|
||||||
|
if not gate_report.ready_for_manuscript:
|
||||||
|
current_candidate_text, current_candidate_path, gate_report = self._repair_until_ready(
|
||||||
|
provider=provider,
|
||||||
|
context=context,
|
||||||
|
structure_plan=structure_plan,
|
||||||
|
current_candidate_text=current_candidate_text,
|
||||||
|
current_candidate_path=current_candidate_path,
|
||||||
|
gate_report=gate_report,
|
||||||
|
metadata=metadata,
|
||||||
|
)
|
||||||
|
if not gate_report.ready_for_manuscript:
|
||||||
|
self._set_status(metadata, "quality_blocked", "Promotion bloquée par le garde-fou manuscrit.")
|
||||||
|
metadata["failed_stage"] = current_stage
|
||||||
|
metadata["finished_at"] = self._timestamp()
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
return GenerationOutcome(
|
||||||
|
chapter_id=chapter_id,
|
||||||
|
accepted=False,
|
||||||
|
status="quality_blocked",
|
||||||
|
draft_path=current_candidate_path,
|
||||||
|
critique_path=context.critique_path,
|
||||||
|
gate_path=context.gate_path,
|
||||||
|
meta_path=context.meta_path,
|
||||||
|
manuscript_path=None,
|
||||||
|
quality_blockers=gate_report.all_blockers(),
|
||||||
|
)
|
||||||
|
|
||||||
|
self._set_status(metadata, "awaiting_acceptance", "Brouillon final prêt pour validation.")
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
|
||||||
|
accepted = (
|
||||||
|
approval_callback(control_report, current_candidate_path)
|
||||||
|
if approval_callback is not None
|
||||||
|
else self._prompt_for_acceptance(control_report, current_candidate_path)
|
||||||
|
)
|
||||||
|
metadata["accepted"] = accepted
|
||||||
|
|
||||||
|
if not accepted:
|
||||||
|
self._set_status(metadata, "rejected", "Promotion refusée par l'auteur.")
|
||||||
|
metadata["finished_at"] = self._timestamp()
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
return GenerationOutcome(
|
||||||
|
chapter_id=chapter_id,
|
||||||
|
accepted=False,
|
||||||
|
status="rejected",
|
||||||
|
draft_path=current_candidate_path,
|
||||||
|
critique_path=context.critique_path,
|
||||||
|
gate_path=context.gate_path,
|
||||||
|
meta_path=context.meta_path,
|
||||||
|
manuscript_path=None,
|
||||||
|
quality_blockers=gate_report.all_blockers() if gate_report is not None else [],
|
||||||
|
)
|
||||||
|
|
||||||
|
current_stage = "memory"
|
||||||
|
self._write_text(context.manuscript_path, current_candidate_text)
|
||||||
|
self._set_status(
|
||||||
|
metadata,
|
||||||
|
"manuscript_promoted",
|
||||||
|
"Brouillon promu dans le manuscrit, mise à jour mémoire en cours.",
|
||||||
|
)
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
memory_update = self._generate_memory_update(provider, context, current_candidate_text, metadata)
|
||||||
|
self._persist_memory(context, memory_update)
|
||||||
|
self._complete_stage(metadata, current_stage)
|
||||||
|
self._set_status(metadata, "accepted", "Chapitre accepté et mémoire mise à jour.")
|
||||||
|
metadata["finished_at"] = self._timestamp()
|
||||||
|
metadata["memory_update"] = memory_update.to_dict()
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
|
||||||
|
return GenerationOutcome(
|
||||||
|
chapter_id=chapter_id,
|
||||||
|
accepted=True,
|
||||||
|
status="accepted",
|
||||||
|
draft_path=current_candidate_path,
|
||||||
|
critique_path=context.critique_path,
|
||||||
|
gate_path=context.gate_path,
|
||||||
|
meta_path=context.meta_path,
|
||||||
|
manuscript_path=context.manuscript_path,
|
||||||
|
quality_blockers=[],
|
||||||
|
)
|
||||||
|
except ProviderError as exc:
|
||||||
|
failed_stage = self._current_running_stage(metadata, current_stage)
|
||||||
|
self._set_status(metadata, "failed", f"Échec à l'étape {failed_stage}: {exc}")
|
||||||
|
metadata["failed_stage"] = failed_stage
|
||||||
|
metadata["error"] = str(exc)
|
||||||
|
metadata["finished_at"] = self._timestamp()
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
raise
|
||||||
|
except ValueError as exc:
|
||||||
|
failed_stage = self._current_running_stage(metadata, current_stage)
|
||||||
|
self._set_status(metadata, "failed", f"Échec à l'étape {failed_stage}: {exc}")
|
||||||
|
metadata["failed_stage"] = failed_stage
|
||||||
|
metadata["error"] = str(exc)
|
||||||
|
metadata["finished_at"] = self._timestamp()
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
raise ProviderError(str(exc)) from exc
|
||||||
|
|
||||||
|
def _build_context(self, chapter_id: ChapterId) -> GenerationContext:
|
||||||
|
intention_path = self.intention_gate.resolve_intention_path(chapter_id)
|
||||||
|
if intention_path is None:
|
||||||
|
raise RuntimeError(f"Aucune intention trouvée pour {chapter_id.slug}.")
|
||||||
|
|
||||||
|
structure_path = resolve_chapter_file(self.root / "structure" / "chapitres", chapter_id)
|
||||||
|
manuscript_path = resolve_chapter_file(self.root / "manuscrit", chapter_id)
|
||||||
|
memory_summary_path = resolve_chapter_file(self.root / "memoire" / "chapitres", chapter_id)
|
||||||
|
draft_dir = self.root / "brouillons" / "chapitres" / chapter_id.slug
|
||||||
|
|
||||||
|
return GenerationContext(
|
||||||
|
root=self.root,
|
||||||
|
chapter_id=chapter_id,
|
||||||
|
intention_path=intention_path,
|
||||||
|
intention_text=intention_path.read_text(encoding="utf-8").strip(),
|
||||||
|
structure_path=structure_path,
|
||||||
|
draft_dir=draft_dir,
|
||||||
|
draft_v1_path=draft_dir / "draft_v1.md",
|
||||||
|
critique_path=draft_dir / "critique_v1.md",
|
||||||
|
draft_v2_path=draft_dir / "draft_v2.md",
|
||||||
|
gate_path=draft_dir / "gate_v1.json",
|
||||||
|
meta_path=draft_dir / "meta.json",
|
||||||
|
manuscript_path=manuscript_path,
|
||||||
|
memory_summary_path=memory_summary_path,
|
||||||
|
memory_index_dir=self.root / "memoire" / "index",
|
||||||
|
story_context=self._build_story_context(chapter_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
def _build_story_context(self, chapter_id: ChapterId) -> str:
|
||||||
|
sections: list[str] = []
|
||||||
|
|
||||||
|
previous_manuscript = self._latest_existing_file(self.root / "manuscrit", chapter_id.number - 1)
|
||||||
|
if previous_manuscript is not None:
|
||||||
|
sections.append(
|
||||||
|
"## Dernier chapitre accepté\n"
|
||||||
|
f"Fichier: {previous_manuscript.name}\n"
|
||||||
|
f"{self._read_excerpt(previous_manuscript)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
previous_memory = self._latest_existing_file(self.root / "memoire" / "chapitres", chapter_id.number - 1)
|
||||||
|
if previous_memory is not None:
|
||||||
|
sections.append(
|
||||||
|
"## Dernier résumé mémoire\n"
|
||||||
|
f"Fichier: {previous_memory.name}\n"
|
||||||
|
f"{self._read_excerpt(previous_memory)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
for name in ("personnages.json", "lieux.json", "chronologie.json"):
|
||||||
|
path = self.root / "memoire" / "index" / name
|
||||||
|
if path.exists():
|
||||||
|
sections.append(f"## {name}\n{self._read_excerpt(path, limit=2000)}")
|
||||||
|
|
||||||
|
if not sections:
|
||||||
|
return "Aucun contexte projet disponible."
|
||||||
|
|
||||||
|
return "\n\n".join(sections)
|
||||||
|
|
||||||
|
def _latest_existing_file(self, directory: Path, max_number: int) -> Path | None:
|
||||||
|
if max_number <= 0 or not directory.exists():
|
||||||
|
return None
|
||||||
|
|
||||||
|
candidates: list[Path] = []
|
||||||
|
for number in range(max_number, 0, -1):
|
||||||
|
path = resolve_chapter_file(directory, ChapterId(number))
|
||||||
|
if path.exists():
|
||||||
|
candidates.append(path)
|
||||||
|
break
|
||||||
|
return candidates[0] if candidates else None
|
||||||
|
|
||||||
|
def _read_excerpt(self, path: Path, limit: int = 4000) -> str:
|
||||||
|
text = path.read_text(encoding="utf-8").strip()
|
||||||
|
if len(text) <= limit:
|
||||||
|
return text
|
||||||
|
return f"{text[:limit].rstrip()}\n[...]"
|
||||||
|
|
||||||
|
def _generate_structure(
|
||||||
|
self,
|
||||||
|
provider: GenerationProvider,
|
||||||
|
context: GenerationContext,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
) -> StructurePlan:
|
||||||
|
prompt = self.prompt_store.render(
|
||||||
|
"structure",
|
||||||
|
chapter_slug=context.chapter_id.slug,
|
||||||
|
intention=context.intention_text,
|
||||||
|
story_context=context.story_context,
|
||||||
|
)
|
||||||
|
self._begin_stage(
|
||||||
|
metadata,
|
||||||
|
context.meta_path,
|
||||||
|
"structure",
|
||||||
|
"Génération de la structure en cours.",
|
||||||
|
)
|
||||||
|
response = provider.generate(GenerationRequest(stage="structure", prompt=prompt))
|
||||||
|
markdown = response.content.strip()
|
||||||
|
if not markdown:
|
||||||
|
raise ProviderError("Le provider a renvoyé une structure vide.")
|
||||||
|
return StructurePlan(chapter_id=context.chapter_id, markdown=f"{markdown}\n")
|
||||||
|
|
||||||
|
def _generate_draft(
|
||||||
|
self,
|
||||||
|
provider: GenerationProvider,
|
||||||
|
context: GenerationContext,
|
||||||
|
structure_plan: StructurePlan,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
) -> str:
|
||||||
|
prompt = self.prompt_store.render(
|
||||||
|
"draft",
|
||||||
|
chapter_slug=context.chapter_id.slug,
|
||||||
|
intention=context.intention_text,
|
||||||
|
structure_markdown=structure_plan.markdown,
|
||||||
|
story_context=context.story_context,
|
||||||
|
)
|
||||||
|
self._begin_stage(
|
||||||
|
metadata,
|
||||||
|
context.meta_path,
|
||||||
|
"draft",
|
||||||
|
"Génération du brouillon initial en cours.",
|
||||||
|
)
|
||||||
|
response = provider.generate(GenerationRequest(stage="draft", prompt=prompt, temperature=0.4))
|
||||||
|
draft = response.content.strip()
|
||||||
|
if not draft:
|
||||||
|
raise ProviderError("Le provider a renvoyé un brouillon vide.")
|
||||||
|
return f"{draft}\n"
|
||||||
|
|
||||||
|
def _generate_control_report(
|
||||||
|
self,
|
||||||
|
provider: GenerationProvider,
|
||||||
|
context: GenerationContext,
|
||||||
|
structure_plan: StructurePlan,
|
||||||
|
draft_v1: str,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
) -> ControlReport:
|
||||||
|
prompt = self.prompt_store.render(
|
||||||
|
"critique",
|
||||||
|
chapter_slug=context.chapter_id.slug,
|
||||||
|
intention=context.intention_text,
|
||||||
|
structure_markdown=structure_plan.markdown,
|
||||||
|
draft_markdown=draft_v1,
|
||||||
|
)
|
||||||
|
return self._generate_json_payload(
|
||||||
|
provider=provider,
|
||||||
|
stage="critique",
|
||||||
|
prompt=prompt,
|
||||||
|
retry_prompt_name="critique_retry",
|
||||||
|
parse_response=ControlReport.from_response_text,
|
||||||
|
metadata=metadata,
|
||||||
|
meta_path=context.meta_path,
|
||||||
|
retry_context={
|
||||||
|
"chapter_slug": context.chapter_id.slug,
|
||||||
|
"intention": context.intention_text,
|
||||||
|
"structure_markdown": structure_plan.markdown,
|
||||||
|
"draft_markdown": draft_v1,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
def _rewrite_draft(
|
||||||
|
self,
|
||||||
|
provider: GenerationProvider,
|
||||||
|
context: GenerationContext,
|
||||||
|
structure_plan: StructurePlan,
|
||||||
|
draft_v1: str,
|
||||||
|
control_report: ControlReport,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
) -> str:
|
||||||
|
prompt = self.prompt_store.render(
|
||||||
|
"rewrite",
|
||||||
|
chapter_slug=context.chapter_id.slug,
|
||||||
|
intention=context.intention_text,
|
||||||
|
structure_markdown=structure_plan.markdown,
|
||||||
|
draft_markdown=draft_v1,
|
||||||
|
critique_json=control_report.to_dict(),
|
||||||
|
)
|
||||||
|
self._begin_stage(
|
||||||
|
metadata,
|
||||||
|
context.meta_path,
|
||||||
|
"rewrite",
|
||||||
|
"Réécriture guidée par la critique en cours.",
|
||||||
|
)
|
||||||
|
response = provider.generate(GenerationRequest(stage="rewrite", prompt=prompt, temperature=0.3))
|
||||||
|
draft = response.content.strip()
|
||||||
|
if not draft:
|
||||||
|
raise ProviderError("Le provider a renvoyé une réécriture vide.")
|
||||||
|
return f"{draft}\n"
|
||||||
|
|
||||||
|
def _repair_until_ready(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
provider: GenerationProvider,
|
||||||
|
context: GenerationContext,
|
||||||
|
structure_plan: StructurePlan,
|
||||||
|
current_candidate_text: str,
|
||||||
|
current_candidate_path: Path,
|
||||||
|
gate_report: ManuscriptGateReport,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
) -> tuple[str, Path, ManuscriptGateReport]:
|
||||||
|
for attempt in range(1, self._repair_max_passes() + 1):
|
||||||
|
repair_model = self._repair_model_for_attempt(provider, attempt)
|
||||||
|
repair_provider = self._provider_for_repair(provider, repair_model)
|
||||||
|
repaired_text = self._repair_draft(
|
||||||
|
provider=repair_provider,
|
||||||
|
context=context,
|
||||||
|
structure_plan=structure_plan,
|
||||||
|
current_candidate=current_candidate_text,
|
||||||
|
gate_report=gate_report,
|
||||||
|
metadata=metadata,
|
||||||
|
attempt=attempt,
|
||||||
|
repair_model=repair_model,
|
||||||
|
)
|
||||||
|
repair_path = context.repair_path(attempt)
|
||||||
|
self._write_text(repair_path, repaired_text)
|
||||||
|
self._complete_stage(metadata, "repair")
|
||||||
|
self._record_repair_attempt(metadata, attempt=attempt, model=self._provider_model_name(repair_provider) or repair_model, path=repair_path)
|
||||||
|
self._set_status(
|
||||||
|
metadata,
|
||||||
|
"repair_ready",
|
||||||
|
f"Réparation prose v{attempt} générée, nouveau contrôle manuscrit en cours.",
|
||||||
|
)
|
||||||
|
metadata["draft_final"] = str(repair_path)
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
|
||||||
|
current_candidate_text = repaired_text
|
||||||
|
current_candidate_path = repair_path
|
||||||
|
gate_report = self._generate_manuscript_gate_report(
|
||||||
|
repair_provider,
|
||||||
|
context,
|
||||||
|
structure_plan,
|
||||||
|
current_candidate_text,
|
||||||
|
metadata,
|
||||||
|
)
|
||||||
|
self._persist_gate_report(metadata, context, gate_report, current_candidate_path)
|
||||||
|
self._complete_stage(metadata, "gate")
|
||||||
|
self._write_metadata(context.meta_path, metadata)
|
||||||
|
if gate_report.ready_for_manuscript:
|
||||||
|
break
|
||||||
|
|
||||||
|
return current_candidate_text, current_candidate_path, gate_report
|
||||||
|
|
||||||
|
def _repair_draft(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
provider: GenerationProvider,
|
||||||
|
context: GenerationContext,
|
||||||
|
structure_plan: StructurePlan,
|
||||||
|
current_candidate: str,
|
||||||
|
gate_report: ManuscriptGateReport,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
attempt: int,
|
||||||
|
repair_model: str | None,
|
||||||
|
) -> str:
|
||||||
|
prompt = self.prompt_store.render(
|
||||||
|
"repair",
|
||||||
|
chapter_slug=context.chapter_id.slug,
|
||||||
|
intention=context.intention_text,
|
||||||
|
structure_markdown=structure_plan.markdown,
|
||||||
|
draft_markdown=current_candidate,
|
||||||
|
gate_json=gate_report.to_dict(),
|
||||||
|
repair_attempt=attempt,
|
||||||
|
repair_model=repair_model or "",
|
||||||
|
story_context=context.story_context,
|
||||||
|
)
|
||||||
|
model_label = repair_model or self._provider_model_name(provider) or "provider_courant"
|
||||||
|
self._begin_stage(
|
||||||
|
metadata,
|
||||||
|
context.meta_path,
|
||||||
|
"repair",
|
||||||
|
f"Réparation prose v{attempt} en cours avec {model_label}.",
|
||||||
|
)
|
||||||
|
response = provider.generate(GenerationRequest(stage="repair", prompt=prompt, temperature=0.2))
|
||||||
|
repaired = response.content.strip()
|
||||||
|
if not repaired:
|
||||||
|
raise ProviderError("Le provider a renvoyé une réparation vide.")
|
||||||
|
return f"{repaired}\n"
|
||||||
|
|
||||||
|
def _generate_manuscript_gate_report(
|
||||||
|
self,
|
||||||
|
provider: GenerationProvider,
|
||||||
|
context: GenerationContext,
|
||||||
|
structure_plan: StructurePlan,
|
||||||
|
draft_v2: str,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
) -> ManuscriptGateReport:
|
||||||
|
self._begin_stage(
|
||||||
|
metadata,
|
||||||
|
context.meta_path,
|
||||||
|
"gate",
|
||||||
|
"Contrôle manuscrit en cours.",
|
||||||
|
)
|
||||||
|
heuristic_report = self._heuristic_gate_report(draft_v2)
|
||||||
|
if heuristic_report is not None:
|
||||||
|
metadata["last_status_message"] = heuristic_report.summary
|
||||||
|
return heuristic_report
|
||||||
|
|
||||||
|
prompt = self.prompt_store.render(
|
||||||
|
"gate",
|
||||||
|
chapter_slug=context.chapter_id.slug,
|
||||||
|
intention=context.intention_text,
|
||||||
|
structure_markdown=structure_plan.markdown,
|
||||||
|
draft_markdown=draft_v2,
|
||||||
|
)
|
||||||
|
return self._generate_json_payload(
|
||||||
|
provider=provider,
|
||||||
|
stage="gate",
|
||||||
|
prompt=prompt,
|
||||||
|
retry_prompt_name="gate_retry",
|
||||||
|
parse_response=ManuscriptGateReport.from_response_text,
|
||||||
|
metadata=metadata,
|
||||||
|
meta_path=context.meta_path,
|
||||||
|
retry_context={
|
||||||
|
"chapter_slug": context.chapter_id.slug,
|
||||||
|
"intention": context.intention_text,
|
||||||
|
"structure_markdown": structure_plan.markdown,
|
||||||
|
"draft_markdown": draft_v2,
|
||||||
|
},
|
||||||
|
begin_stage=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _persist_gate_report(
|
||||||
|
self,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
context: GenerationContext,
|
||||||
|
gate_report: ManuscriptGateReport,
|
||||||
|
draft_path: Path,
|
||||||
|
) -> None:
|
||||||
|
self._write_json(context.gate_path, gate_report.to_dict())
|
||||||
|
metadata["gate_report"] = gate_report.to_dict()
|
||||||
|
metadata["quality_blockers"] = gate_report.all_blockers()
|
||||||
|
metadata["draft_final"] = str(draft_path)
|
||||||
|
|
||||||
|
def _provider_for_repair(self, provider: GenerationProvider, model: str | None) -> GenerationProvider:
|
||||||
|
if not model:
|
||||||
|
return provider
|
||||||
|
return clone_provider_with_model(provider, model)
|
||||||
|
|
||||||
|
def _repair_max_passes(self) -> int:
|
||||||
|
raw = os.environ.get("ANE_REPAIR_MAX_PASSES", "2").strip() or "2"
|
||||||
|
try:
|
||||||
|
value = int(raw)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise ProviderError("ANE_REPAIR_MAX_PASSES doit être un entier positif.") from exc
|
||||||
|
if value <= 0:
|
||||||
|
raise ProviderError("ANE_REPAIR_MAX_PASSES doit être supérieur à zéro.")
|
||||||
|
return value
|
||||||
|
|
||||||
|
def _repair_model_for_attempt(self, provider: GenerationProvider, attempt: int) -> str | None:
|
||||||
|
base_model = self._provider_model_name(provider)
|
||||||
|
if attempt <= 1:
|
||||||
|
return base_model
|
||||||
|
|
||||||
|
override = os.environ.get("ANE_REPAIR_FALLBACK_MODEL", "").strip()
|
||||||
|
candidate = override or self._default_repair_fallback_model(base_model) or base_model
|
||||||
|
if self._is_cross_apple_runtime_switch(base_model, candidate):
|
||||||
|
raise ProviderError(
|
||||||
|
"ANE_REPAIR_FALLBACK_MODEL ne peut pas viser un autre modèle apple-coreml pendant un même smoke. "
|
||||||
|
"Relancer le runtime Apple sur le modèle cible ou utiliser un fallback non-Apple."
|
||||||
|
)
|
||||||
|
return candidate
|
||||||
|
|
||||||
|
def _default_repair_fallback_model(self, model: str | None) -> str | None:
|
||||||
|
mapping = {
|
||||||
|
"ollama:qwen2.5:1.5b": "ollama:qwen2.5:7b",
|
||||||
|
"apple-coreml:qwen2.5-0.5b-instruct-onnx": "ollama:qwen2.5:7b",
|
||||||
|
"apple-coreml:qwen3.5-4b-onnx-q4f16": "ollama:qwen2.5:7b",
|
||||||
|
}
|
||||||
|
if not model:
|
||||||
|
return None
|
||||||
|
return mapping.get(model)
|
||||||
|
|
||||||
|
def _is_cross_apple_runtime_switch(self, base_model: str | None, candidate: str | None) -> bool:
|
||||||
|
if not base_model or not candidate:
|
||||||
|
return False
|
||||||
|
if base_model == candidate:
|
||||||
|
return False
|
||||||
|
return base_model.startswith("apple-coreml:") and candidate.startswith("apple-coreml:")
|
||||||
|
|
||||||
|
def _heuristic_gate_report(self, draft_v2: str) -> ManuscriptGateReport | None:
|
||||||
|
blockers: list[str] = []
|
||||||
|
recommendations: list[str] = []
|
||||||
|
|
||||||
|
if self._word_count(draft_v2) < 180:
|
||||||
|
blockers.append("too_short")
|
||||||
|
recommendations.append("Allonger le chapitre pour produire une scene complete et continue.")
|
||||||
|
|
||||||
|
if self._has_truncated_ending(draft_v2):
|
||||||
|
blockers.append("truncated_ending")
|
||||||
|
recommendations.append("Terminer la scene sur une phrase complete avec une vraie fermeture.")
|
||||||
|
|
||||||
|
if self._is_outline_like(draft_v2):
|
||||||
|
blockers.append("outline_like")
|
||||||
|
recommendations.append("Remplacer les titres et puces par une narration continue en prose.")
|
||||||
|
|
||||||
|
if not blockers:
|
||||||
|
return None
|
||||||
|
|
||||||
|
summary = "Le garde-fou manuscrit a bloque la promotion: " + ", ".join(blockers) + "."
|
||||||
|
return ManuscriptGateReport.from_heuristics(
|
||||||
|
blockers=blockers,
|
||||||
|
recommendations=recommendations,
|
||||||
|
summary=summary,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _word_count(self, text: str) -> int:
|
||||||
|
return len(re.findall(r"[0-9A-Za-zÀ-ÖØ-öø-ÿ]+(?:[-'][0-9A-Za-zÀ-ÖØ-öø-ÿ]+)*", text))
|
||||||
|
|
||||||
|
def _has_truncated_ending(self, text: str) -> bool:
|
||||||
|
for line in reversed(text.splitlines()):
|
||||||
|
stripped = line.strip()
|
||||||
|
if not stripped:
|
||||||
|
continue
|
||||||
|
return not stripped.endswith((".", "!", "?", "…", "»", '"', "'"))
|
||||||
|
return True
|
||||||
|
|
||||||
|
def _is_outline_like(self, text: str) -> bool:
|
||||||
|
detected_markers: set[str] = set()
|
||||||
|
for line in text.splitlines():
|
||||||
|
stripped = line.strip()
|
||||||
|
if not stripped:
|
||||||
|
continue
|
||||||
|
if stripped.startswith("## "):
|
||||||
|
detected_markers.add("heading_level_2")
|
||||||
|
if stripped.startswith("### "):
|
||||||
|
detected_markers.add("heading_level_3")
|
||||||
|
if stripped.startswith("- "):
|
||||||
|
detected_markers.add("bullet_list")
|
||||||
|
lowered = stripped.lower()
|
||||||
|
if "**objectif**" in lowered or "**conflit**" in lowered or "**sortie**" in lowered:
|
||||||
|
detected_markers.add("scene_fields")
|
||||||
|
if "scène" in lowered or "scene" in lowered:
|
||||||
|
detected_markers.add("scene_heading")
|
||||||
|
if len(detected_markers) >= 2:
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _generate_memory_update(
|
||||||
|
self,
|
||||||
|
provider: GenerationProvider,
|
||||||
|
context: GenerationContext,
|
||||||
|
accepted_draft: str,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
) -> MemoryUpdate:
|
||||||
|
prompt = self.prompt_store.render(
|
||||||
|
"memory",
|
||||||
|
chapter_slug=context.chapter_id.slug,
|
||||||
|
accepted_draft=accepted_draft,
|
||||||
|
story_context=context.story_context,
|
||||||
|
)
|
||||||
|
return self._generate_json_payload(
|
||||||
|
provider=provider,
|
||||||
|
stage="memory",
|
||||||
|
prompt=prompt,
|
||||||
|
retry_prompt_name="memory_retry",
|
||||||
|
parse_response=MemoryUpdate.from_response_text,
|
||||||
|
metadata=metadata,
|
||||||
|
meta_path=context.meta_path,
|
||||||
|
retry_context={
|
||||||
|
"chapter_slug": context.chapter_id.slug,
|
||||||
|
"accepted_draft": accepted_draft,
|
||||||
|
"story_context": context.story_context,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
def _generate_json_payload(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
provider: GenerationProvider,
|
||||||
|
stage: str,
|
||||||
|
prompt: str,
|
||||||
|
retry_prompt_name: str,
|
||||||
|
parse_response: Callable[[str], ParsedStagePayload],
|
||||||
|
metadata: dict[str, object],
|
||||||
|
meta_path: Path,
|
||||||
|
retry_context: dict[str, object],
|
||||||
|
begin_stage: bool = True,
|
||||||
|
) -> ParsedStagePayload:
|
||||||
|
if begin_stage:
|
||||||
|
self._begin_stage(
|
||||||
|
metadata,
|
||||||
|
meta_path,
|
||||||
|
stage,
|
||||||
|
f"Génération de l'étape {stage} en cours.",
|
||||||
|
)
|
||||||
|
first_response = provider.generate(
|
||||||
|
GenerationRequest(stage=stage, prompt=prompt, response_format="json", temperature=0.1)
|
||||||
|
)
|
||||||
|
first_payload = first_response.content
|
||||||
|
try:
|
||||||
|
return parse_response(first_payload)
|
||||||
|
except ValueError as first_error:
|
||||||
|
self._mark_stage_attempt(
|
||||||
|
metadata,
|
||||||
|
stage,
|
||||||
|
retry=True,
|
||||||
|
message=f"Réessai JSON sur l'étape {stage} après une réponse invalide.",
|
||||||
|
)
|
||||||
|
self._set_status(metadata, f"{stage}_retrying", f"Réessai JSON sur l'étape {stage} en cours.")
|
||||||
|
self._write_metadata(meta_path, metadata)
|
||||||
|
retry_prompt = self.prompt_store.render(
|
||||||
|
retry_prompt_name,
|
||||||
|
parse_error=str(first_error),
|
||||||
|
invalid_response=self._truncate_retry_payload(first_payload),
|
||||||
|
**retry_context,
|
||||||
|
)
|
||||||
|
retry_response = provider.generate(
|
||||||
|
GenerationRequest(stage=stage, prompt=retry_prompt, response_format="json", temperature=0.0)
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
return parse_response(retry_response.content)
|
||||||
|
except ValueError as second_error:
|
||||||
|
raise ProviderError(
|
||||||
|
f"Le provider a renvoyé un JSON invalide pendant l'étape '{stage}' après deux tentatives: "
|
||||||
|
f"première erreur: {first_error}; seconde erreur: {second_error}"
|
||||||
|
) from second_error
|
||||||
|
|
||||||
|
def _truncate_retry_payload(self, payload: str, limit: int = 2000) -> str:
|
||||||
|
text = payload.strip()
|
||||||
|
if len(text) <= limit:
|
||||||
|
return text
|
||||||
|
return f"{text[:limit].rstrip()}\n[...]"
|
||||||
|
|
||||||
|
def _persist_memory(self, context: GenerationContext, memory_update: MemoryUpdate) -> None:
|
||||||
|
self._write_text(
|
||||||
|
context.memory_summary_path,
|
||||||
|
f"# Mémoire — {context.chapter_id.slug}\n\n{memory_update.chapter_summary}\n",
|
||||||
|
)
|
||||||
|
|
||||||
|
self._merge_index_records(
|
||||||
|
context.memory_index_dir / "personnages.json",
|
||||||
|
context.chapter_id,
|
||||||
|
memory_update.characters,
|
||||||
|
label_key="name",
|
||||||
|
)
|
||||||
|
self._merge_index_records(
|
||||||
|
context.memory_index_dir / "lieux.json",
|
||||||
|
context.chapter_id,
|
||||||
|
memory_update.locations,
|
||||||
|
label_key="name",
|
||||||
|
)
|
||||||
|
self._merge_timeline_records(
|
||||||
|
context.memory_index_dir / "chronologie.json",
|
||||||
|
context.chapter_id,
|
||||||
|
memory_update.timeline_events,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _merge_index_records(
|
||||||
|
self,
|
||||||
|
path: Path,
|
||||||
|
chapter_id: ChapterId,
|
||||||
|
records: list[dict[str, str]],
|
||||||
|
label_key: str,
|
||||||
|
) -> None:
|
||||||
|
existing: dict[str, dict[str, object]] = {}
|
||||||
|
if path.exists():
|
||||||
|
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
if isinstance(payload, dict):
|
||||||
|
existing = payload
|
||||||
|
|
||||||
|
for record in records:
|
||||||
|
label = record[label_key]
|
||||||
|
current = existing.get(label, {"chapters": []})
|
||||||
|
chapters = list(current.get("chapters", []))
|
||||||
|
if chapter_id.slug not in chapters:
|
||||||
|
chapters.append(chapter_id.slug)
|
||||||
|
|
||||||
|
merged = dict(current)
|
||||||
|
merged.update(record)
|
||||||
|
merged["chapters"] = sorted(chapters)
|
||||||
|
existing[label] = merged
|
||||||
|
|
||||||
|
self._write_json(path, existing)
|
||||||
|
|
||||||
|
def _merge_timeline_records(
|
||||||
|
self,
|
||||||
|
path: Path,
|
||||||
|
chapter_id: ChapterId,
|
||||||
|
records: list[dict[str, str]],
|
||||||
|
) -> None:
|
||||||
|
existing: list[dict[str, str]] = []
|
||||||
|
if path.exists():
|
||||||
|
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
if isinstance(payload, list):
|
||||||
|
existing = [
|
||||||
|
{str(key): str(value) for key, value in item.items()}
|
||||||
|
for item in payload
|
||||||
|
if isinstance(item, dict)
|
||||||
|
]
|
||||||
|
|
||||||
|
for record in records:
|
||||||
|
merged = dict(record)
|
||||||
|
merged["chapter"] = chapter_id.slug
|
||||||
|
existing.append(merged)
|
||||||
|
|
||||||
|
self._write_json(path, existing)
|
||||||
|
|
||||||
|
def _prompt_for_acceptance(self, control_report: ControlReport, draft_path: Path) -> bool:
|
||||||
|
self.output_func("")
|
||||||
|
self.output_func(f"Critique: {control_report.summary}")
|
||||||
|
self.output_func(f"Brouillon final: {draft_path}")
|
||||||
|
self.output_func(f"Écarts détectés: {len(control_report.deviations)}")
|
||||||
|
response = self.input_func("Promouvoir ce brouillon vers le manuscrit ? [y/N]: ").strip().lower()
|
||||||
|
return response in {"y", "yes", "o", "oui"}
|
||||||
|
|
||||||
|
def _initial_metadata(self, context: GenerationContext, provider: GenerationProvider) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"chapter": context.chapter_id.slug,
|
||||||
|
"started_at": self._timestamp(),
|
||||||
|
"status": "started",
|
||||||
|
"last_status_message": "Pipeline initialisé.",
|
||||||
|
"completed_stages": [],
|
||||||
|
"accepted": False,
|
||||||
|
"repair_attempts": 0,
|
||||||
|
"repair_models": [],
|
||||||
|
"stage_attempts": {},
|
||||||
|
"retry_stages": [],
|
||||||
|
"quality_blockers": [],
|
||||||
|
"provider": self._provider_metadata(provider),
|
||||||
|
"artifacts": {
|
||||||
|
"intention": str(context.intention_path),
|
||||||
|
"structure": str(context.structure_path),
|
||||||
|
"draft_v1": str(context.draft_v1_path),
|
||||||
|
"critique_v1": str(context.critique_path),
|
||||||
|
"draft_v2": str(context.draft_v2_path),
|
||||||
|
"repair_latest": None,
|
||||||
|
"repairs": [],
|
||||||
|
"gate_v1": str(context.gate_path),
|
||||||
|
"manuscript": str(context.manuscript_path),
|
||||||
|
"memory_summary": str(context.memory_summary_path),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
def _provider_metadata(self, provider: GenerationProvider) -> dict[str, object]:
|
||||||
|
snapshot = {
|
||||||
|
"kind": provider.__class__.__name__,
|
||||||
|
"base_url": None,
|
||||||
|
"model": None,
|
||||||
|
}
|
||||||
|
if isinstance(provider, OpenAICompatibleProvider):
|
||||||
|
snapshot["base_url"] = provider.config.base_url
|
||||||
|
snapshot["model"] = provider.config.model
|
||||||
|
return snapshot
|
||||||
|
|
||||||
|
def _current_running_stage(self, metadata: dict[str, object], fallback: str) -> str:
|
||||||
|
status = str(metadata.get("status", "")).strip()
|
||||||
|
for suffix in ("_running", "_retrying"):
|
||||||
|
if status.endswith(suffix):
|
||||||
|
return status[: -len(suffix)]
|
||||||
|
return fallback
|
||||||
|
|
||||||
|
def _provider_model_name(self, provider: GenerationProvider) -> str | None:
|
||||||
|
metadata = self._provider_metadata(provider)
|
||||||
|
model = metadata.get("model")
|
||||||
|
if not isinstance(model, str):
|
||||||
|
return None
|
||||||
|
return model.strip() or None
|
||||||
|
|
||||||
|
def _complete_stage(self, metadata: dict[str, object], stage: str) -> None:
|
||||||
|
completed = metadata.setdefault("completed_stages", [])
|
||||||
|
if not isinstance(completed, list):
|
||||||
|
completed = []
|
||||||
|
metadata["completed_stages"] = completed
|
||||||
|
if stage not in completed:
|
||||||
|
completed.append(stage)
|
||||||
|
|
||||||
|
def _record_repair_attempt(
|
||||||
|
self,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
*,
|
||||||
|
attempt: int,
|
||||||
|
model: str | None,
|
||||||
|
path: Path,
|
||||||
|
) -> None:
|
||||||
|
metadata["repair_attempts"] = attempt
|
||||||
|
|
||||||
|
repair_models = metadata.setdefault("repair_models", [])
|
||||||
|
if not isinstance(repair_models, list):
|
||||||
|
repair_models = []
|
||||||
|
metadata["repair_models"] = repair_models
|
||||||
|
repair_models.append(model or "provider_courant")
|
||||||
|
|
||||||
|
artifacts = metadata.setdefault("artifacts", {})
|
||||||
|
if not isinstance(artifacts, dict):
|
||||||
|
artifacts = {}
|
||||||
|
metadata["artifacts"] = artifacts
|
||||||
|
repairs = artifacts.setdefault("repairs", [])
|
||||||
|
if not isinstance(repairs, list):
|
||||||
|
repairs = []
|
||||||
|
artifacts["repairs"] = repairs
|
||||||
|
repairs.append(str(path))
|
||||||
|
artifacts["repair_latest"] = str(path)
|
||||||
|
|
||||||
|
def _set_status(self, metadata: dict[str, object], status: str, message: str) -> None:
|
||||||
|
metadata["status"] = status
|
||||||
|
metadata["last_status_message"] = message
|
||||||
|
|
||||||
|
def _mark_stage_attempt(
|
||||||
|
self,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
stage: str,
|
||||||
|
*,
|
||||||
|
retry: bool = False,
|
||||||
|
message: str | None = None,
|
||||||
|
) -> None:
|
||||||
|
stage_attempts = metadata.setdefault("stage_attempts", {})
|
||||||
|
if not isinstance(stage_attempts, dict):
|
||||||
|
stage_attempts = {}
|
||||||
|
metadata["stage_attempts"] = stage_attempts
|
||||||
|
stage_attempts[stage] = int(stage_attempts.get(stage, 0)) + 1
|
||||||
|
|
||||||
|
if retry:
|
||||||
|
retry_stages = metadata.setdefault("retry_stages", [])
|
||||||
|
if isinstance(retry_stages, list) and stage not in retry_stages:
|
||||||
|
retry_stages.append(stage)
|
||||||
|
|
||||||
|
if message:
|
||||||
|
metadata["last_status_message"] = message
|
||||||
|
|
||||||
|
def _begin_stage(
|
||||||
|
self,
|
||||||
|
metadata: dict[str, object],
|
||||||
|
meta_path: Path,
|
||||||
|
stage: str,
|
||||||
|
message: str,
|
||||||
|
) -> None:
|
||||||
|
self._mark_stage_attempt(metadata, stage)
|
||||||
|
self._set_status(metadata, f"{stage}_running", message)
|
||||||
|
self._write_metadata(meta_path, metadata)
|
||||||
|
|
||||||
|
def _write_metadata(self, path: Path, metadata: dict[str, object]) -> None:
|
||||||
|
self._write_json(path, metadata)
|
||||||
|
|
||||||
|
def _write_json(self, path: Path, payload: object) -> None:
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(json.dumps(payload, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
|
||||||
|
def _write_text(self, path: Path, content: str) -> None:
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(content, encoding="utf-8")
|
||||||
|
|
||||||
|
def _timestamp(self) -> str:
|
||||||
|
return datetime.now(timezone.utc).replace(microsecond=0).isoformat()
|
||||||
@@ -0,0 +1,254 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from dataclasses import dataclass, replace
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import socket
|
||||||
|
from typing import Mapping
|
||||||
|
from urllib import error, request
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderError(RuntimeError):
|
||||||
|
"""Raised when a text generation provider fails."""
|
||||||
|
|
||||||
|
|
||||||
|
class ProviderConfigurationError(ProviderError):
|
||||||
|
"""Raised when the provider environment is incomplete."""
|
||||||
|
|
||||||
|
|
||||||
|
STAGE_MAX_TOKENS_ENV = {
|
||||||
|
"structure": "ANE_MAX_TOKENS_STRUCTURE",
|
||||||
|
"draft": "ANE_MAX_TOKENS_DRAFT",
|
||||||
|
"critique": "ANE_MAX_TOKENS_CRITIQUE",
|
||||||
|
"rewrite": "ANE_MAX_TOKENS_REWRITE",
|
||||||
|
"gate": "ANE_MAX_TOKENS_GATE",
|
||||||
|
"repair": "ANE_MAX_TOKENS_REPAIR",
|
||||||
|
"memory": "ANE_MAX_TOKENS_MEMORY",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_positive_int(raw_value: str, *, env_name: str) -> int:
|
||||||
|
try:
|
||||||
|
value = int(raw_value)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise ProviderConfigurationError(f"{env_name} doit être un entier.") from exc
|
||||||
|
if value <= 0:
|
||||||
|
raise ProviderConfigurationError(f"{env_name} doit être supérieur à zéro.")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ProviderConfig:
|
||||||
|
provider: str
|
||||||
|
base_url: str
|
||||||
|
api_key: str
|
||||||
|
model: str
|
||||||
|
timeout: float
|
||||||
|
max_tokens: int
|
||||||
|
stage_max_tokens: Mapping[str, int]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_env(cls, env: Mapping[str, str] | None = None) -> "ProviderConfig":
|
||||||
|
source = env or os.environ
|
||||||
|
provider = source.get("ANE_PROVIDER", "openai_compatible").strip() or "openai_compatible"
|
||||||
|
base_url = source.get("ANE_BASE_URL", "").strip()
|
||||||
|
model = source.get("ANE_MODEL", "").strip()
|
||||||
|
api_key = source.get("ANE_API_KEY", "").strip()
|
||||||
|
timeout_value = source.get("ANE_TIMEOUT", "60").strip() or "60"
|
||||||
|
max_tokens_value = source.get("ANE_MAX_TOKENS", "4096").strip() or "4096"
|
||||||
|
|
||||||
|
try:
|
||||||
|
timeout = float(timeout_value)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise ProviderConfigurationError("ANE_TIMEOUT doit être un nombre.") from exc
|
||||||
|
|
||||||
|
max_tokens = _parse_positive_int(max_tokens_value, env_name="ANE_MAX_TOKENS")
|
||||||
|
stage_max_tokens: dict[str, int] = {}
|
||||||
|
for stage_name, env_name in STAGE_MAX_TOKENS_ENV.items():
|
||||||
|
raw_stage_value = source.get(env_name, "").strip()
|
||||||
|
if not raw_stage_value:
|
||||||
|
continue
|
||||||
|
stage_max_tokens[stage_name] = _parse_positive_int(raw_stage_value, env_name=env_name)
|
||||||
|
|
||||||
|
return cls(
|
||||||
|
provider=provider,
|
||||||
|
base_url=base_url,
|
||||||
|
api_key=api_key,
|
||||||
|
model=model,
|
||||||
|
timeout=timeout,
|
||||||
|
max_tokens=max_tokens,
|
||||||
|
stage_max_tokens=stage_max_tokens,
|
||||||
|
)
|
||||||
|
|
||||||
|
def max_tokens_for_stage(self, stage: str, explicit: int | None = None) -> int:
|
||||||
|
if explicit is not None:
|
||||||
|
return explicit
|
||||||
|
return self.stage_max_tokens.get(stage, self.max_tokens)
|
||||||
|
|
||||||
|
def with_model(self, model: str) -> "ProviderConfig":
|
||||||
|
return replace(self, model=model)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GenerationRequest:
|
||||||
|
stage: str
|
||||||
|
prompt: str
|
||||||
|
response_format: str = "text"
|
||||||
|
temperature: float = 0.2
|
||||||
|
system_prompt: str | None = None
|
||||||
|
max_tokens: int | None = None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GenerationResponse:
|
||||||
|
content: str
|
||||||
|
model: str | None = None
|
||||||
|
raw: dict[str, object] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class GenerationProvider(ABC):
|
||||||
|
@abstractmethod
|
||||||
|
def generate(self, request: GenerationRequest) -> GenerationResponse:
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
|
||||||
|
class OpenAICompatibleProvider(GenerationProvider):
|
||||||
|
def __init__(self, config: ProviderConfig):
|
||||||
|
if not config.base_url:
|
||||||
|
raise ProviderConfigurationError("ANE_BASE_URL est requis pour le provider openai_compatible.")
|
||||||
|
if not config.model:
|
||||||
|
raise ProviderConfigurationError("ANE_MODEL est requis pour le provider openai_compatible.")
|
||||||
|
self.config = config
|
||||||
|
|
||||||
|
def generate(self, prompt_request: GenerationRequest) -> GenerationResponse:
|
||||||
|
payload: dict[str, object] = {
|
||||||
|
"model": self.config.model,
|
||||||
|
"messages": self._build_messages(prompt_request),
|
||||||
|
"temperature": prompt_request.temperature,
|
||||||
|
"max_tokens": self.config.max_tokens_for_stage(
|
||||||
|
prompt_request.stage,
|
||||||
|
prompt_request.max_tokens,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
if prompt_request.response_format == "json":
|
||||||
|
payload["response_format"] = {"type": "json_object"}
|
||||||
|
|
||||||
|
body = json.dumps(payload).encode("utf-8")
|
||||||
|
headers = {"Content-Type": "application/json"}
|
||||||
|
if self.config.api_key:
|
||||||
|
headers["Authorization"] = f"Bearer {self.config.api_key}"
|
||||||
|
|
||||||
|
http_request = request.Request(
|
||||||
|
self._chat_completions_url(),
|
||||||
|
data=body,
|
||||||
|
headers=headers,
|
||||||
|
method="POST",
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
with request.urlopen(http_request, timeout=self.config.timeout) as response:
|
||||||
|
raw_payload = json.loads(response.read().decode("utf-8"))
|
||||||
|
except error.HTTPError as exc:
|
||||||
|
details = exc.read().decode("utf-8", errors="replace")
|
||||||
|
raise ProviderError(
|
||||||
|
f"Le provider a répondu avec HTTP {exc.code} pendant l'étape '{prompt_request.stage}': {details}"
|
||||||
|
) from exc
|
||||||
|
except error.URLError as exc:
|
||||||
|
raise ProviderError(
|
||||||
|
f"Impossible de joindre le provider pendant l'étape '{prompt_request.stage}': {exc.reason}"
|
||||||
|
) from exc
|
||||||
|
except (TimeoutError, socket.timeout) as exc:
|
||||||
|
raise ProviderError(
|
||||||
|
f"Timeout du provider pendant l'étape '{prompt_request.stage}' après {self.config.timeout:.0f}s."
|
||||||
|
) from exc
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise ProviderError(
|
||||||
|
f"Réponse non JSON du provider pendant l'étape '{prompt_request.stage}'."
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
try:
|
||||||
|
choice = raw_payload["choices"][0]
|
||||||
|
message = choice["message"]["content"]
|
||||||
|
except (KeyError, IndexError, TypeError) as exc:
|
||||||
|
raise ProviderError(
|
||||||
|
f"Réponse OpenAI-compatible invalide pendant l'étape '{prompt_request.stage}'."
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
content = self._normalize_message_content(message)
|
||||||
|
return GenerationResponse(
|
||||||
|
content=content,
|
||||||
|
model=str(raw_payload.get("model", self.config.model)),
|
||||||
|
raw=raw_payload,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _build_messages(self, prompt_request: GenerationRequest) -> list[dict[str, str]]:
|
||||||
|
messages: list[dict[str, str]] = []
|
||||||
|
if prompt_request.system_prompt:
|
||||||
|
messages.append({"role": "system", "content": prompt_request.system_prompt})
|
||||||
|
messages.append({"role": "user", "content": prompt_request.prompt})
|
||||||
|
return messages
|
||||||
|
|
||||||
|
def _chat_completions_url(self) -> str:
|
||||||
|
base = self.config.base_url.rstrip("/")
|
||||||
|
if base.endswith("/chat/completions"):
|
||||||
|
return base
|
||||||
|
if base.endswith("/v1"):
|
||||||
|
return f"{base}/chat/completions"
|
||||||
|
return f"{base}/v1/chat/completions"
|
||||||
|
|
||||||
|
def _normalize_message_content(self, message: object) -> str:
|
||||||
|
if isinstance(message, str):
|
||||||
|
return message
|
||||||
|
if isinstance(message, list):
|
||||||
|
parts: list[str] = []
|
||||||
|
for item in message:
|
||||||
|
if isinstance(item, dict) and item.get("type") == "text":
|
||||||
|
parts.append(str(item.get("text", "")))
|
||||||
|
if parts:
|
||||||
|
return "\n".join(parts)
|
||||||
|
raise ProviderError("Le provider n'a pas renvoyé de contenu texte exploitable.")
|
||||||
|
|
||||||
|
|
||||||
|
class MockGenerationProvider(GenerationProvider):
|
||||||
|
def __init__(self, responses: Mapping[str, object]):
|
||||||
|
self._responses = {
|
||||||
|
stage: list(value) if isinstance(value, list) else [value]
|
||||||
|
for stage, value in responses.items()
|
||||||
|
}
|
||||||
|
self.requests: list[GenerationRequest] = []
|
||||||
|
|
||||||
|
def generate(self, prompt_request: GenerationRequest) -> GenerationResponse:
|
||||||
|
self.requests.append(prompt_request)
|
||||||
|
queue = self._responses.get(prompt_request.stage)
|
||||||
|
if not queue:
|
||||||
|
raise ProviderError(f"Aucune réponse mock configurée pour l'étape '{prompt_request.stage}'.")
|
||||||
|
|
||||||
|
next_value = queue.pop(0)
|
||||||
|
if isinstance(next_value, Exception):
|
||||||
|
raise next_value
|
||||||
|
if isinstance(next_value, (dict, list)):
|
||||||
|
content = json.dumps(next_value, ensure_ascii=False)
|
||||||
|
else:
|
||||||
|
content = str(next_value)
|
||||||
|
|
||||||
|
return GenerationResponse(content=content, model="mock")
|
||||||
|
|
||||||
|
|
||||||
|
def build_provider_from_env(env: Mapping[str, str] | None = None) -> GenerationProvider:
|
||||||
|
config = ProviderConfig.from_env(env)
|
||||||
|
if config.provider != "openai_compatible":
|
||||||
|
raise ProviderConfigurationError(
|
||||||
|
f"Provider non supporté: {config.provider}. Utilisez ANE_PROVIDER=openai_compatible."
|
||||||
|
)
|
||||||
|
return OpenAICompatibleProvider(config)
|
||||||
|
|
||||||
|
|
||||||
|
def clone_provider_with_model(provider: GenerationProvider, model: str) -> GenerationProvider:
|
||||||
|
if not model:
|
||||||
|
return provider
|
||||||
|
if isinstance(provider, OpenAICompatibleProvider):
|
||||||
|
if provider.config.model == model:
|
||||||
|
return provider
|
||||||
|
return OpenAICompatibleProvider(provider.config.with_model(model))
|
||||||
|
return provider
|
||||||
Binary file not shown.
+37
-6
@@ -1,5 +1,7 @@
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
from core.chapters import ChapterId, collect_matching_chapter_files
|
||||||
|
|
||||||
class IntentionGate:
|
class IntentionGate:
|
||||||
"""
|
"""
|
||||||
Hard lock: blocks any generation if no explicit intention exists.
|
Hard lock: blocks any generation if no explicit intention exists.
|
||||||
@@ -8,17 +10,46 @@ class IntentionGate:
|
|||||||
def __init__(self, project_root: Path):
|
def __init__(self, project_root: Path):
|
||||||
self.intentions_dir = project_root / "notes" / "intentions"
|
self.intentions_dir = project_root / "notes" / "intentions"
|
||||||
|
|
||||||
def has_intention(self) -> bool:
|
def has_intention(self, chapter: object | None = None) -> bool:
|
||||||
if not self.intentions_dir.exists():
|
if not self.intentions_dir.exists():
|
||||||
return False
|
return False
|
||||||
intentions = list(self.intentions_dir.glob("chapitre_*.md"))
|
if chapter is None:
|
||||||
|
intentions = list(self.intentions_dir.glob("chapitre_*.md"))
|
||||||
|
return len(intentions) > 0
|
||||||
|
chapter_id = ChapterId.parse(chapter)
|
||||||
|
intentions = collect_matching_chapter_files(self.intentions_dir, chapter_id)
|
||||||
return len(intentions) > 0
|
return len(intentions) > 0
|
||||||
|
|
||||||
def assert_intention(self):
|
def resolve_intention_path(self, chapter: object) -> Path | None:
|
||||||
if not self.has_intention():
|
chapter_id = ChapterId.parse(chapter)
|
||||||
|
matches = collect_matching_chapter_files(self.intentions_dir, chapter_id)
|
||||||
|
if not matches:
|
||||||
|
return None
|
||||||
|
if len(matches) > 1:
|
||||||
|
from core.chapters import ChapterConflictError
|
||||||
|
|
||||||
|
raise ChapterConflictError(chapter_id, matches)
|
||||||
|
return matches[0]
|
||||||
|
|
||||||
|
def load_intention(self, chapter: object) -> str:
|
||||||
|
path = self.resolve_intention_path(chapter)
|
||||||
|
if path is None:
|
||||||
|
chapter_id = ChapterId.parse(chapter)
|
||||||
|
raise RuntimeError(f"Aucune intention trouvée pour {chapter_id.slug}.")
|
||||||
|
return path.read_text(encoding="utf-8").strip()
|
||||||
|
|
||||||
|
def assert_intention(self, chapter: object | None = None):
|
||||||
|
if not self.has_intention(chapter):
|
||||||
|
if chapter is None:
|
||||||
|
raise RuntimeError(
|
||||||
|
"Aucune intention trouvée.\n"
|
||||||
|
"L'écriture est volontairement bloquée.\n"
|
||||||
|
"Créez d'abord une intention explicite (CLI: intention create)."
|
||||||
|
)
|
||||||
|
|
||||||
|
chapter_id = ChapterId.parse(chapter)
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
"Aucune intention trouvée.\n"
|
f"Aucune intention trouvée pour {chapter_id.slug}.\n"
|
||||||
"L'écriture est volontairement bloquée.\n"
|
"L'écriture est volontairement bloquée.\n"
|
||||||
"Créez d'abord une intention explicite (CLI: intention create)."
|
"Créez d'abord une intention explicite (CLI: intention create)."
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,952 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
from dataclasses import asdict, dataclass, field
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
import tomllib
|
||||||
|
from typing import Any, Callable, Iterable
|
||||||
|
from urllib import error, request
|
||||||
|
|
||||||
|
from core.chapters import ChapterId
|
||||||
|
from core.project.loader import ProjectState
|
||||||
|
|
||||||
|
|
||||||
|
AUTO_SYNC_TODO_ACTIVE = "ANE-TODO-ACTIVE"
|
||||||
|
AUTO_SYNC_TODO_DONE = "ANE-TODO-DONE"
|
||||||
|
AUTO_SYNC_PLAN = "ANE-PLAN"
|
||||||
|
AUTO_SYNC_COMPARISON = "ANE-COMPARISON"
|
||||||
|
AUTO_SYNC_README = "ANE-README"
|
||||||
|
AUTO_SYNC_RUNBOOK = "ANE-RUNBOOK"
|
||||||
|
AUTO_SYNC_MASCARADE_TODO = "MASCARADE-TODO"
|
||||||
|
AUTO_SYNC_MASCARADE_PLAN = "MASCARADE-PLAN"
|
||||||
|
AUTO_SYNC_MASCARADE_README = "MASCARADE-README"
|
||||||
|
AUTO_SYNC_MASCARADE_RUNBOOK = "MASCARADE-RUNBOOK"
|
||||||
|
|
||||||
|
|
||||||
|
class NextLotsError(RuntimeError):
|
||||||
|
"""Raised when the orchestration flow cannot continue automatically."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class TrackingPaths:
|
||||||
|
ane_todo_active: Path
|
||||||
|
ane_todo_done: Path
|
||||||
|
ane_plan: Path
|
||||||
|
ane_comparison: Path
|
||||||
|
ane_readme: Path
|
||||||
|
ane_runbook: Path
|
||||||
|
mascarade_repo: Path
|
||||||
|
mascarade_todo: Path
|
||||||
|
mascarade_plan: Path
|
||||||
|
mascarade_readme: Path
|
||||||
|
mascarade_runbook: Path
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Manifest:
|
||||||
|
repo_root: Path
|
||||||
|
manifest_path: Path
|
||||||
|
tracking: TrackingPaths
|
||||||
|
core_base_url: str
|
||||||
|
apple_runtime_url: str
|
||||||
|
ollama_tags_url: str
|
||||||
|
apple_model_ready_timeout_seconds: float
|
||||||
|
apple_model_poll_interval_seconds: float
|
||||||
|
smoke_chapter: str
|
||||||
|
smoke_intention: str
|
||||||
|
smoke_timeout_seconds: int
|
||||||
|
preset_env: dict[str, str]
|
||||||
|
required_apple_models: list[str]
|
||||||
|
required_ollama_models: list[str]
|
||||||
|
priority_models: list[str]
|
||||||
|
baseline_models: list[str]
|
||||||
|
preflight_only_models: list[str]
|
||||||
|
next_code_lot: str
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def load(cls, repo_root: Path, manifest_path: Path) -> "Manifest":
|
||||||
|
payload = tomllib.loads(manifest_path.read_text(encoding="utf-8"))
|
||||||
|
paths = payload["paths"]
|
||||||
|
smoke = payload["smoke"]
|
||||||
|
preset = payload["preset"]
|
||||||
|
tracking = payload["tracking"]
|
||||||
|
lots = payload["lots"]
|
||||||
|
ensure_models = payload["ensure_models"]
|
||||||
|
|
||||||
|
mascarade_repo = Path(paths["mascarade_repo"]).expanduser()
|
||||||
|
return cls(
|
||||||
|
repo_root=repo_root,
|
||||||
|
manifest_path=manifest_path,
|
||||||
|
tracking=TrackingPaths(
|
||||||
|
ane_todo_active=repo_root / tracking["ane"]["todo_active"],
|
||||||
|
ane_todo_done=repo_root / tracking["ane"]["todo_done"],
|
||||||
|
ane_plan=repo_root / tracking["ane"]["plan"],
|
||||||
|
ane_comparison=repo_root / tracking["ane"]["comparison"],
|
||||||
|
ane_readme=repo_root / tracking["ane"]["readme"],
|
||||||
|
ane_runbook=repo_root / tracking["ane"]["runbook"],
|
||||||
|
mascarade_repo=mascarade_repo,
|
||||||
|
mascarade_todo=mascarade_repo / tracking["mascarade"]["todo"],
|
||||||
|
mascarade_plan=mascarade_repo / tracking["mascarade"]["plan"],
|
||||||
|
mascarade_readme=mascarade_repo / tracking["mascarade"]["readme"],
|
||||||
|
mascarade_runbook=mascarade_repo / tracking["mascarade"]["runbook"],
|
||||||
|
),
|
||||||
|
core_base_url=str(paths["core_base_url"]).rstrip("/"),
|
||||||
|
apple_runtime_url=str(paths["apple_runtime_url"]).rstrip("/"),
|
||||||
|
ollama_tags_url=str(paths["ollama_tags_url"]).rstrip("/"),
|
||||||
|
apple_model_ready_timeout_seconds=float(paths.get("apple_model_ready_timeout_seconds", 30)),
|
||||||
|
apple_model_poll_interval_seconds=float(paths.get("apple_model_poll_interval_seconds", 2)),
|
||||||
|
smoke_chapter=str(smoke["chapter"]),
|
||||||
|
smoke_intention=str(smoke["intention"]),
|
||||||
|
smoke_timeout_seconds=int(smoke["timeout_seconds"]),
|
||||||
|
preset_env={str(key): str(value) for key, value in preset.items()},
|
||||||
|
required_apple_models=[str(item) for item in ensure_models["apple_models"]],
|
||||||
|
required_ollama_models=[str(item) for item in ensure_models["ollama_models"]],
|
||||||
|
priority_models=[str(item) for item in lots["priority_models"]["models"]],
|
||||||
|
baseline_models=[str(item) for item in lots["baselines"]["models"]],
|
||||||
|
preflight_only_models=[str(item) for item in lots["preflight_only"]["models"]],
|
||||||
|
next_code_lot=str(payload["next_actions"]["rewrite_compaction"]),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class CommandResult:
|
||||||
|
args: list[str]
|
||||||
|
returncode: int
|
||||||
|
stdout: str
|
||||||
|
stderr: str
|
||||||
|
duration_seconds: float
|
||||||
|
|
||||||
|
|
||||||
|
CommandRunner = Callable[[list[str], Path, dict[str, str] | None], CommandResult]
|
||||||
|
JsonFetcher = Callable[[str, float], Any]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ModelRunResult:
|
||||||
|
model: str
|
||||||
|
category: str
|
||||||
|
classification: str = "pending"
|
||||||
|
preflight_ok: bool | None = None
|
||||||
|
preflight_duration_seconds: float | None = None
|
||||||
|
smoke_attempted: bool = False
|
||||||
|
smoke_duration_seconds: float | None = None
|
||||||
|
status: str | None = None
|
||||||
|
accepted: bool = False
|
||||||
|
failed_stage: str | None = None
|
||||||
|
quality_blockers: list[str] = field(default_factory=list)
|
||||||
|
retry_stages: list[str] = field(default_factory=list)
|
||||||
|
repair_attempts: int = 0
|
||||||
|
repair_models: list[str] = field(default_factory=list)
|
||||||
|
draft_path: str | None = None
|
||||||
|
gate_path: str | None = None
|
||||||
|
meta_path: str | None = None
|
||||||
|
manuscript_path: str | None = None
|
||||||
|
notes: list[str] = field(default_factory=list)
|
||||||
|
preflight_log: str | None = None
|
||||||
|
smoke_log: str | None = None
|
||||||
|
workspace: str | None = None
|
||||||
|
apple_model_active: str | None = None
|
||||||
|
completed_stages: list[str] = field(default_factory=list)
|
||||||
|
|
||||||
|
def reached_gate(self) -> bool:
|
||||||
|
return "gate" in self.completed_stages or (self.failed_stage == "gate")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class RunState:
|
||||||
|
version: int
|
||||||
|
manifest_path: str
|
||||||
|
report_dir: str
|
||||||
|
state_path: str
|
||||||
|
lot: str
|
||||||
|
started_at: str
|
||||||
|
updated_at: str
|
||||||
|
step_index: int
|
||||||
|
model_index: int
|
||||||
|
steps: list[dict[str, Any]]
|
||||||
|
results: list[dict[str, Any]]
|
||||||
|
notes: list[str]
|
||||||
|
pending_manual_action: dict[str, Any] | None
|
||||||
|
next_recommended_lot: str
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def new(cls, manifest: Manifest, lot: str, report_dir: Path, state_path: Path, steps: list[dict[str, Any]]) -> "RunState":
|
||||||
|
now = _timestamp()
|
||||||
|
return cls(
|
||||||
|
version=1,
|
||||||
|
manifest_path=str(manifest.manifest_path),
|
||||||
|
report_dir=str(report_dir),
|
||||||
|
state_path=str(state_path),
|
||||||
|
lot=lot,
|
||||||
|
started_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
step_index=0,
|
||||||
|
model_index=0,
|
||||||
|
steps=steps,
|
||||||
|
results=[],
|
||||||
|
notes=[],
|
||||||
|
pending_manual_action=None,
|
||||||
|
next_recommended_lot=manifest.next_code_lot,
|
||||||
|
)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def load(cls, path: Path) -> "RunState":
|
||||||
|
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
return cls(**payload)
|
||||||
|
|
||||||
|
def dump(self, path: Path | None = None) -> None:
|
||||||
|
target = path or Path(self.state_path)
|
||||||
|
self.updated_at = _timestamp()
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
target.write_text(json.dumps(asdict(self), ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
|
||||||
|
def append_result(self, result: ModelRunResult) -> None:
|
||||||
|
self.results.append(asdict(result))
|
||||||
|
self.updated_at = _timestamp()
|
||||||
|
|
||||||
|
def typed_results(self) -> list[ModelRunResult]:
|
||||||
|
return [ModelRunResult(**payload) for payload in self.results]
|
||||||
|
|
||||||
|
|
||||||
|
def _timestamp() -> str:
|
||||||
|
return datetime.now(timezone.utc).replace(microsecond=0).isoformat()
|
||||||
|
|
||||||
|
|
||||||
|
def _default_command_runner(args: list[str], cwd: Path, env: dict[str, str] | None = None) -> CommandResult:
|
||||||
|
merged_env = os.environ.copy()
|
||||||
|
if env:
|
||||||
|
merged_env.update(env)
|
||||||
|
started = time.monotonic()
|
||||||
|
completed = subprocess.run(
|
||||||
|
args,
|
||||||
|
cwd=str(cwd),
|
||||||
|
env=merged_env,
|
||||||
|
text=True,
|
||||||
|
capture_output=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
return CommandResult(
|
||||||
|
args=args,
|
||||||
|
returncode=completed.returncode,
|
||||||
|
stdout=completed.stdout,
|
||||||
|
stderr=completed.stderr,
|
||||||
|
duration_seconds=time.monotonic() - started,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _default_json_fetcher(url: str, timeout: float) -> Any:
|
||||||
|
with request.urlopen(url, timeout=timeout) as response:
|
||||||
|
return json.loads(response.read().decode("utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def _auto_markers(name: str) -> tuple[str, str]:
|
||||||
|
return (
|
||||||
|
f"<!-- AUTO-SYNC:{name}:START -->",
|
||||||
|
f"<!-- AUTO-SYNC:{name}:END -->",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def replace_auto_section(path: Path, marker_name: str, heading: str, body: str) -> None:
|
||||||
|
start_marker, end_marker = _auto_markers(marker_name)
|
||||||
|
text = path.read_text(encoding="utf-8") if path.exists() else ""
|
||||||
|
section = f"{heading}\n{start_marker}\n{body.rstrip()}\n{end_marker}\n"
|
||||||
|
if start_marker in text and end_marker in text:
|
||||||
|
start = text.index(start_marker)
|
||||||
|
end = text.index(end_marker) + len(end_marker)
|
||||||
|
replacement_start = text.rfind("\n", 0, start)
|
||||||
|
if replacement_start == -1:
|
||||||
|
replacement_start = 0
|
||||||
|
else:
|
||||||
|
replacement_start += 1
|
||||||
|
new_text = f"{text[:replacement_start]}{section}{text[end:].lstrip()}"
|
||||||
|
else:
|
||||||
|
suffix = "\n" if text.endswith("\n") else "\n\n"
|
||||||
|
new_text = f"{text}{suffix}{section}"
|
||||||
|
path.write_text(new_text, encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
class NextLotsRunner:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
manifest: Manifest,
|
||||||
|
*,
|
||||||
|
command_runner: CommandRunner = _default_command_runner,
|
||||||
|
json_fetcher: JsonFetcher = _default_json_fetcher,
|
||||||
|
) -> None:
|
||||||
|
self.manifest = manifest
|
||||||
|
self.command_runner = command_runner
|
||||||
|
self.json_fetcher = json_fetcher
|
||||||
|
|
||||||
|
def run(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
lot: str,
|
||||||
|
resume_state: Path | None = None,
|
||||||
|
dry_run: bool = False,
|
||||||
|
report_only: bool = False,
|
||||||
|
) -> int:
|
||||||
|
state_path = self.manifest.repo_root / "automation" / "state" / "next_lots_state.json"
|
||||||
|
if resume_state is not None:
|
||||||
|
state = RunState.load(resume_state)
|
||||||
|
else:
|
||||||
|
report_dir = self._new_report_dir()
|
||||||
|
state = RunState.new(
|
||||||
|
self.manifest,
|
||||||
|
lot=lot,
|
||||||
|
report_dir=report_dir,
|
||||||
|
state_path=state_path,
|
||||||
|
steps=self._steps_for_lot(lot),
|
||||||
|
)
|
||||||
|
state.dump(state_path)
|
||||||
|
|
||||||
|
if report_only:
|
||||||
|
self._sync_tracking(state, dry_run=dry_run)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
while state.step_index < len(state.steps):
|
||||||
|
step = state.steps[state.step_index]
|
||||||
|
step_type = str(step["type"])
|
||||||
|
if step_type == "ensure_models":
|
||||||
|
print("==> lot ensure_models")
|
||||||
|
self._run_ensure_models(state, dry_run=dry_run)
|
||||||
|
state.step_index += 1
|
||||||
|
state.model_index = 0
|
||||||
|
state.dump()
|
||||||
|
continue
|
||||||
|
if step_type == "models":
|
||||||
|
print(f"==> lot {step['name']}")
|
||||||
|
exit_code = self._run_model_step(state, step, dry_run=dry_run)
|
||||||
|
state.dump()
|
||||||
|
if exit_code is not None:
|
||||||
|
self._sync_tracking(state, dry_run=dry_run)
|
||||||
|
return exit_code
|
||||||
|
state.step_index += 1
|
||||||
|
state.model_index = 0
|
||||||
|
state.dump()
|
||||||
|
continue
|
||||||
|
if step_type == "tracking_sync":
|
||||||
|
print("==> lot tracking_sync")
|
||||||
|
self._sync_tracking(state, dry_run=dry_run)
|
||||||
|
state.step_index += 1
|
||||||
|
state.model_index = 0
|
||||||
|
state.dump()
|
||||||
|
continue
|
||||||
|
raise NextLotsError(f"Type de lot non supporté: {step_type}")
|
||||||
|
|
||||||
|
self._write_report_summary(state)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
def _steps_for_lot(self, lot: str) -> list[dict[str, Any]]:
|
||||||
|
if lot == "ensure_models":
|
||||||
|
return [{"type": "ensure_models"}]
|
||||||
|
if lot == "runtime_preflight":
|
||||||
|
queue = [*self.manifest.priority_models, *self.manifest.baseline_models]
|
||||||
|
return [{"type": "models", "name": "runtime_preflight", "models": queue, "preflight_only": True}]
|
||||||
|
if lot == "priority_models":
|
||||||
|
return [
|
||||||
|
{"type": "models", "name": "priority_models", "models": self.manifest.priority_models, "preflight_only": False},
|
||||||
|
{"type": "tracking_sync"},
|
||||||
|
]
|
||||||
|
if lot == "baselines":
|
||||||
|
return [
|
||||||
|
{"type": "models", "name": "baselines", "models": self.manifest.baseline_models, "preflight_only": False},
|
||||||
|
{"type": "models", "name": "preflight_only", "models": self.manifest.preflight_only_models, "preflight_only": True},
|
||||||
|
{"type": "tracking_sync"},
|
||||||
|
]
|
||||||
|
if lot == "tracking_sync":
|
||||||
|
return [{"type": "tracking_sync"}]
|
||||||
|
if lot == "full":
|
||||||
|
return [
|
||||||
|
{"type": "ensure_models"},
|
||||||
|
{"type": "models", "name": "priority_models", "models": self.manifest.priority_models, "preflight_only": False},
|
||||||
|
{"type": "models", "name": "baselines", "models": self.manifest.baseline_models, "preflight_only": False},
|
||||||
|
{"type": "models", "name": "preflight_only", "models": self.manifest.preflight_only_models, "preflight_only": True},
|
||||||
|
{"type": "tracking_sync"},
|
||||||
|
]
|
||||||
|
raise NextLotsError(f"Lot inconnu: {lot}")
|
||||||
|
|
||||||
|
def _new_report_dir(self) -> Path:
|
||||||
|
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
|
||||||
|
report_dir = self.manifest.repo_root / "automation" / "reports" / stamp
|
||||||
|
report_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
return report_dir
|
||||||
|
|
||||||
|
def _run_ensure_models(self, state: RunState, *, dry_run: bool) -> None:
|
||||||
|
if dry_run:
|
||||||
|
state.notes.append("Dry-run: ensure_models non exécuté.")
|
||||||
|
return
|
||||||
|
args = ["bash", "scripts/ensure_apple_models.sh"]
|
||||||
|
result = self.command_runner(args, self.manifest.tracking.mascarade_repo)
|
||||||
|
log_path = Path(state.report_dir) / "ensure_models.log"
|
||||||
|
log_path.write_text(_command_log(result), encoding="utf-8")
|
||||||
|
if result.returncode != 0:
|
||||||
|
raise NextLotsError("ensure_models a échoué.")
|
||||||
|
missing = self._missing_ollama_models()
|
||||||
|
if missing:
|
||||||
|
state.notes.append(
|
||||||
|
"Modèles Ollama manquants: " + ", ".join(missing) + ". Lancer manuellement `ollama pull` sur ces modèles."
|
||||||
|
)
|
||||||
|
|
||||||
|
def _missing_ollama_models(self) -> list[str]:
|
||||||
|
try:
|
||||||
|
payload = self.json_fetcher(self.manifest.ollama_tags_url, 10.0)
|
||||||
|
except Exception:
|
||||||
|
return []
|
||||||
|
models = payload.get("models") if isinstance(payload, dict) else None
|
||||||
|
if not isinstance(models, list):
|
||||||
|
return []
|
||||||
|
names = {
|
||||||
|
str(item.get("name", "")).strip()
|
||||||
|
for item in models
|
||||||
|
if isinstance(item, dict) and str(item.get("name", "")).strip()
|
||||||
|
}
|
||||||
|
return [model for model in self.manifest.required_ollama_models if model not in names]
|
||||||
|
|
||||||
|
def _run_model_step(self, state: RunState, step: dict[str, Any], *, dry_run: bool) -> int | None:
|
||||||
|
models = [str(item) for item in step["models"]]
|
||||||
|
preflight_only = bool(step.get("preflight_only", False))
|
||||||
|
for index in range(state.model_index, len(models)):
|
||||||
|
state.model_index = index
|
||||||
|
model = models[index]
|
||||||
|
category = str(step["name"])
|
||||||
|
state.notes = [f"Modele en cours: {model}"]
|
||||||
|
state.dump()
|
||||||
|
print(f"--> {model}")
|
||||||
|
if dry_run:
|
||||||
|
state.append_result(
|
||||||
|
ModelRunResult(
|
||||||
|
model=model,
|
||||||
|
category=category,
|
||||||
|
classification="dry_run",
|
||||||
|
notes=["Dry-run: aucun preflight ni smoke exécuté."],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
checkpoint = self._checkpoint_if_runtime_manual_step_needed(state, model)
|
||||||
|
if checkpoint is not None:
|
||||||
|
print(f"checkpoint manuel: {checkpoint['reason']}")
|
||||||
|
print(f"commande: {checkpoint['command']}")
|
||||||
|
state.pending_manual_action = checkpoint
|
||||||
|
state.notes = [f"Checkpoint manuel requis pour: {model}"]
|
||||||
|
self._write_report_summary(state)
|
||||||
|
return 3
|
||||||
|
state.pending_manual_action = None
|
||||||
|
result = self._run_model(model, category=category, preflight_only=preflight_only, report_dir=Path(state.report_dir))
|
||||||
|
state.notes = [f"Dernier modele traite: {model} -> {result.classification}"]
|
||||||
|
state.append_result(result)
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _checkpoint_if_runtime_manual_step_needed(self, state: RunState, model: str) -> dict[str, Any] | None:
|
||||||
|
if not self._core_health_ok():
|
||||||
|
return self._build_manual_action(
|
||||||
|
state,
|
||||||
|
args=["bash", "scripts/prepare_runtime_step.sh", "--restart", "core", "--resume-state", state.state_path, "--ane-script", str(self.manifest.repo_root / "scripts" / "run_next_lots.py")],
|
||||||
|
reason="Le core mascarade ne répond pas correctement.",
|
||||||
|
)
|
||||||
|
if not model.startswith("apple-coreml:"):
|
||||||
|
return None
|
||||||
|
target_model = model.split(":", 1)[1]
|
||||||
|
apple_model = self._wait_for_expected_apple_model(target_model)
|
||||||
|
if apple_model == target_model:
|
||||||
|
return None
|
||||||
|
args = [
|
||||||
|
"bash",
|
||||||
|
"scripts/prepare_runtime_step.sh",
|
||||||
|
"--apple-model",
|
||||||
|
target_model,
|
||||||
|
"--resume-state",
|
||||||
|
state.state_path,
|
||||||
|
"--ane-script",
|
||||||
|
str(self.manifest.repo_root / "scripts" / "run_next_lots.py"),
|
||||||
|
]
|
||||||
|
return self._build_manual_action(
|
||||||
|
state,
|
||||||
|
args=args,
|
||||||
|
reason=f"Le runtime Apple sert `{apple_model or 'aucun modèle'}` au lieu de `{target_model}`.",
|
||||||
|
)
|
||||||
|
|
||||||
|
def _build_manual_action(self, state: RunState, *, args: list[str], reason: str) -> dict[str, Any]:
|
||||||
|
result = self.command_runner(args, self.manifest.tracking.mascarade_repo)
|
||||||
|
log_path = Path(state.report_dir) / f"manual_action_{len(state.results):02d}.log"
|
||||||
|
log_path.write_text(_command_log(result), encoding="utf-8")
|
||||||
|
return {
|
||||||
|
"reason": reason,
|
||||||
|
"command": " ".join(args),
|
||||||
|
"log_path": str(log_path),
|
||||||
|
"resume_state": state.state_path,
|
||||||
|
}
|
||||||
|
|
||||||
|
def _core_health_ok(self) -> bool:
|
||||||
|
try:
|
||||||
|
payload = self.json_fetcher(f"{self.manifest.core_base_url}/health", 10.0)
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
return isinstance(payload, dict)
|
||||||
|
|
||||||
|
def _current_apple_model(self) -> str | None:
|
||||||
|
try:
|
||||||
|
payload = self.json_fetcher(f"{self.manifest.apple_runtime_url}/models", 10.0)
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
if isinstance(payload, list) and payload:
|
||||||
|
return str(payload[0]).strip() or None
|
||||||
|
if isinstance(payload, dict):
|
||||||
|
models = payload.get("models")
|
||||||
|
if isinstance(models, list) and models:
|
||||||
|
return str(models[0]).strip() or None
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _wait_for_expected_apple_model(self, target_model: str) -> str | None:
|
||||||
|
deadline = time.monotonic() + max(self.manifest.apple_model_ready_timeout_seconds, 0.0)
|
||||||
|
poll_interval = max(self.manifest.apple_model_poll_interval_seconds, 0.1)
|
||||||
|
last_seen = self._current_apple_model()
|
||||||
|
if last_seen == target_model or self.manifest.apple_model_ready_timeout_seconds <= 0:
|
||||||
|
return last_seen
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
time.sleep(poll_interval)
|
||||||
|
last_seen = self._current_apple_model()
|
||||||
|
if last_seen == target_model:
|
||||||
|
return last_seen
|
||||||
|
return last_seen
|
||||||
|
|
||||||
|
def _run_model(self, model: str, *, category: str, preflight_only: bool, report_dir: Path) -> ModelRunResult:
|
||||||
|
result = ModelRunResult(model=model, category=category, apple_model_active=self._current_apple_model())
|
||||||
|
model_slug = _slugify(model)
|
||||||
|
preflight_args = [
|
||||||
|
"bash",
|
||||||
|
"scripts/smoke_openai_compat_ane.sh",
|
||||||
|
"--url",
|
||||||
|
self.manifest.core_base_url,
|
||||||
|
"--model",
|
||||||
|
model,
|
||||||
|
"--timeout",
|
||||||
|
str(self._timeout_for_model(model)),
|
||||||
|
]
|
||||||
|
preflight = self.command_runner(preflight_args, self.manifest.tracking.mascarade_repo)
|
||||||
|
result.preflight_duration_seconds = preflight.duration_seconds
|
||||||
|
preflight_log = report_dir / f"{model_slug}_preflight.log"
|
||||||
|
preflight_log.write_text(_command_log(preflight), encoding="utf-8")
|
||||||
|
result.preflight_log = str(preflight_log)
|
||||||
|
result.preflight_ok = preflight.returncode == 0
|
||||||
|
if not result.preflight_ok:
|
||||||
|
result.classification = "provider_failed"
|
||||||
|
result.status = "preflight_failed"
|
||||||
|
result.notes.append("Le preflight OpenAI-compatible a échoué.")
|
||||||
|
return result
|
||||||
|
if preflight_only:
|
||||||
|
result.classification = "preflight_only"
|
||||||
|
result.status = "preflight_only"
|
||||||
|
result.notes.append("Smoke complet volontairement sauté pour ce modèle.")
|
||||||
|
return result
|
||||||
|
|
||||||
|
workspace = report_dir / "workspaces" / model_slug
|
||||||
|
workspace.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
smoke_args = [
|
||||||
|
"bash",
|
||||||
|
"scripts/smoke_local_generation.sh",
|
||||||
|
"--base-url",
|
||||||
|
self.manifest.core_base_url,
|
||||||
|
"--model",
|
||||||
|
model,
|
||||||
|
"--chapter",
|
||||||
|
self.manifest.smoke_chapter,
|
||||||
|
"--workspace",
|
||||||
|
str(workspace),
|
||||||
|
"--timeout",
|
||||||
|
str(self.manifest.smoke_timeout_seconds),
|
||||||
|
"--intention",
|
||||||
|
self.manifest.smoke_intention,
|
||||||
|
"--approve",
|
||||||
|
]
|
||||||
|
smoke = self.command_runner(smoke_args, self.manifest.repo_root, env=self.manifest.preset_env)
|
||||||
|
result.smoke_attempted = True
|
||||||
|
result.smoke_duration_seconds = smoke.duration_seconds
|
||||||
|
smoke_log = report_dir / f"{model_slug}_smoke.log"
|
||||||
|
smoke_log.write_text(_command_log(smoke), encoding="utf-8")
|
||||||
|
result.smoke_log = str(smoke_log)
|
||||||
|
result.workspace = str(workspace)
|
||||||
|
|
||||||
|
chapter = ChapterId.parse(self.manifest.smoke_chapter)
|
||||||
|
meta_path = workspace / "brouillons" / "chapitres" / chapter.slug / "meta.json"
|
||||||
|
if not meta_path.exists():
|
||||||
|
result.classification = "provider_failed"
|
||||||
|
result.status = "missing_meta"
|
||||||
|
result.notes.append("Le smoke n'a pas produit de meta.json exploitable.")
|
||||||
|
return result
|
||||||
|
|
||||||
|
payload = json.loads(meta_path.read_text(encoding="utf-8"))
|
||||||
|
result.meta_path = str(meta_path)
|
||||||
|
result.status = str(payload.get("status", "")).strip() or None
|
||||||
|
result.accepted = bool(payload.get("accepted", False))
|
||||||
|
result.failed_stage = str(payload.get("failed_stage", "")).strip() or None
|
||||||
|
result.quality_blockers = _string_list(payload.get("quality_blockers"))
|
||||||
|
result.retry_stages = _string_list(payload.get("retry_stages"))
|
||||||
|
result.repair_attempts = int(payload.get("repair_attempts", 0) or 0)
|
||||||
|
result.repair_models = _string_list(payload.get("repair_models"))
|
||||||
|
result.completed_stages = _string_list(payload.get("completed_stages"))
|
||||||
|
artifacts = payload.get("artifacts", {})
|
||||||
|
if isinstance(artifacts, dict):
|
||||||
|
result.draft_path = _optional_string(artifacts.get("repair_latest")) or _optional_string(artifacts.get("draft_v2"))
|
||||||
|
result.gate_path = _optional_string(artifacts.get("gate_v1"))
|
||||||
|
result.manuscript_path = _optional_string(artifacts.get("manuscript"))
|
||||||
|
|
||||||
|
if result.status == "accepted":
|
||||||
|
result.classification = "accepted"
|
||||||
|
elif result.status == "quality_blocked":
|
||||||
|
result.classification = "quality_blocked"
|
||||||
|
elif smoke.returncode == 0 and result.status == "rejected":
|
||||||
|
result.classification = "provider_failed"
|
||||||
|
else:
|
||||||
|
result.classification = "provider_failed"
|
||||||
|
return result
|
||||||
|
|
||||||
|
def _timeout_for_model(self, model: str) -> int:
|
||||||
|
if model.startswith("apple-coreml:"):
|
||||||
|
return max(600, self.manifest.smoke_timeout_seconds)
|
||||||
|
return max(120, self.manifest.smoke_timeout_seconds)
|
||||||
|
|
||||||
|
def _sync_tracking(self, state: RunState, *, dry_run: bool) -> None:
|
||||||
|
if dry_run:
|
||||||
|
self._write_report_summary(state)
|
||||||
|
return
|
||||||
|
typed_results = state.typed_results()
|
||||||
|
project_state = ProjectState(self.manifest.repo_root).summary()
|
||||||
|
summary = _build_summary(state, typed_results)
|
||||||
|
comparison = _render_comparison_markdown(state, typed_results)
|
||||||
|
active_next = _compute_next_lot_recommendation(typed_results, self.manifest.next_code_lot)
|
||||||
|
|
||||||
|
replace_auto_section(
|
||||||
|
self.manifest.tracking.ane_todo_active,
|
||||||
|
AUTO_SYNC_TODO_ACTIVE,
|
||||||
|
"## Auto-sync",
|
||||||
|
_render_todo_active_sync(summary, active_next),
|
||||||
|
)
|
||||||
|
replace_auto_section(
|
||||||
|
self.manifest.tracking.ane_todo_done,
|
||||||
|
AUTO_SYNC_TODO_DONE,
|
||||||
|
"## Auto-sync",
|
||||||
|
_render_todo_done_sync(summary),
|
||||||
|
)
|
||||||
|
replace_auto_section(
|
||||||
|
self.manifest.tracking.ane_plan,
|
||||||
|
AUTO_SYNC_PLAN,
|
||||||
|
"## Auto-sync",
|
||||||
|
_render_plan_sync(summary, active_next),
|
||||||
|
)
|
||||||
|
replace_auto_section(
|
||||||
|
self.manifest.tracking.ane_comparison,
|
||||||
|
AUTO_SYNC_COMPARISON,
|
||||||
|
"## Auto-sync",
|
||||||
|
comparison,
|
||||||
|
)
|
||||||
|
replace_auto_section(
|
||||||
|
self.manifest.tracking.ane_readme,
|
||||||
|
AUTO_SYNC_README,
|
||||||
|
"## Etat auto-synchronise",
|
||||||
|
_render_readme_sync(summary, active_next),
|
||||||
|
)
|
||||||
|
replace_auto_section(
|
||||||
|
self.manifest.tracking.ane_runbook,
|
||||||
|
AUTO_SYNC_RUNBOOK,
|
||||||
|
"## Etat auto-synchronise",
|
||||||
|
_render_runbook_sync(summary, project_state, active_next),
|
||||||
|
)
|
||||||
|
replace_auto_section(
|
||||||
|
self.manifest.tracking.mascarade_todo,
|
||||||
|
AUTO_SYNC_MASCARADE_TODO,
|
||||||
|
"## Auto-sync",
|
||||||
|
_render_mascarade_todo_sync(summary, active_next),
|
||||||
|
)
|
||||||
|
replace_auto_section(
|
||||||
|
self.manifest.tracking.mascarade_plan,
|
||||||
|
AUTO_SYNC_MASCARADE_PLAN,
|
||||||
|
"## Auto-sync",
|
||||||
|
_render_mascarade_plan_sync(summary, active_next),
|
||||||
|
)
|
||||||
|
replace_auto_section(
|
||||||
|
self.manifest.tracking.mascarade_readme,
|
||||||
|
AUTO_SYNC_MASCARADE_README,
|
||||||
|
"## Etat auto-synchronise",
|
||||||
|
_render_mascarade_readme_sync(summary, active_next),
|
||||||
|
)
|
||||||
|
replace_auto_section(
|
||||||
|
self.manifest.tracking.mascarade_runbook,
|
||||||
|
AUTO_SYNC_MASCARADE_RUNBOOK,
|
||||||
|
"## Etat auto-synchronise",
|
||||||
|
_render_mascarade_runbook_sync(summary, active_next),
|
||||||
|
)
|
||||||
|
self._write_report_summary(state)
|
||||||
|
|
||||||
|
def _write_report_summary(self, state: RunState) -> None:
|
||||||
|
report_dir = Path(state.report_dir)
|
||||||
|
report_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
run_path = report_dir / "run.json"
|
||||||
|
summary_path = report_dir / "SUMMARY.md"
|
||||||
|
run_path.write_text(json.dumps(asdict(state), ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
summary_path.write_text(_render_summary_markdown(state, state.typed_results()), encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def _string_list(value: object) -> list[str]:
|
||||||
|
if not isinstance(value, list):
|
||||||
|
return []
|
||||||
|
return [str(item).strip() for item in value if str(item).strip()]
|
||||||
|
|
||||||
|
|
||||||
|
def _optional_string(value: object) -> str | None:
|
||||||
|
text = str(value).strip() if value is not None else ""
|
||||||
|
return text or None
|
||||||
|
|
||||||
|
|
||||||
|
def _slugify(value: str) -> str:
|
||||||
|
return "".join(char if char.isalnum() else "_" for char in value).strip("_").lower()
|
||||||
|
|
||||||
|
|
||||||
|
def _command_log(result: CommandResult) -> str:
|
||||||
|
return (
|
||||||
|
f"$ {' '.join(result.args)}\n"
|
||||||
|
f"returncode={result.returncode}\n"
|
||||||
|
f"duration_seconds={result.duration_seconds:.2f}\n\n"
|
||||||
|
f"STDOUT\n{result.stdout}\n\nSTDERR\n{result.stderr}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _build_summary(state: RunState, results: list[ModelRunResult]) -> dict[str, Any]:
|
||||||
|
accepted = [item for item in results if item.classification == "accepted"]
|
||||||
|
reached_gate = [item for item in results if item.reached_gate()]
|
||||||
|
quality_blocked = [item for item in results if item.classification == "quality_blocked"]
|
||||||
|
provider_failed = [item for item in results if item.classification == "provider_failed"]
|
||||||
|
return {
|
||||||
|
"started_at": state.started_at,
|
||||||
|
"updated_at": state.updated_at,
|
||||||
|
"pending_manual_action": state.pending_manual_action,
|
||||||
|
"accepted_models": [item.model for item in accepted],
|
||||||
|
"reached_gate_models": [item.model for item in reached_gate],
|
||||||
|
"quality_blocked_models": [item.model for item in quality_blocked],
|
||||||
|
"provider_failed_models": [item.model for item in provider_failed],
|
||||||
|
"results": results,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _compute_next_lot_recommendation(results: list[ModelRunResult], fallback: str) -> str:
|
||||||
|
if any(item.classification == "accepted" for item in results):
|
||||||
|
return "Rejouer uniquement les baselines vitesse puis figer la référence locale dans les README/runbooks."
|
||||||
|
if any(item.reached_gate() for item in results):
|
||||||
|
return "Analyser les runs ayant atteint gate/repair puis resserrer la reference locale autour des meilleurs candidats."
|
||||||
|
return fallback
|
||||||
|
|
||||||
|
|
||||||
|
def _render_todo_active_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||||
|
lines = [
|
||||||
|
f"- dernier cycle automatique: {summary['updated_at']}",
|
||||||
|
f"- modeles accepted: {_comma_or_none(summary['accepted_models'])}",
|
||||||
|
f"- modeles ayant atteint gate: {_comma_or_none(summary['reached_gate_models'])}",
|
||||||
|
f"- quality_blocked: {_comma_or_none(summary['quality_blocked_models'])}",
|
||||||
|
f"- provider_failed: {_comma_or_none(summary['provider_failed_models'])}",
|
||||||
|
f"- prochain lot recommande: {next_lot}",
|
||||||
|
]
|
||||||
|
if summary["pending_manual_action"]:
|
||||||
|
pending = summary["pending_manual_action"]
|
||||||
|
lines.extend(
|
||||||
|
[
|
||||||
|
f"- checkpoint manuel en attente: {pending['reason']}",
|
||||||
|
f"- commande preparee: `{pending['command']}`",
|
||||||
|
f"- reprise: `python3 scripts/run_next_lots.py --resume {pending['resume_state']}`",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _render_todo_done_sync(summary: dict[str, Any]) -> str:
|
||||||
|
lines = [
|
||||||
|
"- orchestrateur `scripts/run_next_lots.py` disponible",
|
||||||
|
"- manifeste `automation/next_lots.toml` charge",
|
||||||
|
"- derniers fichiers de suivi synchronisables via marqueurs `AUTO-SYNC`",
|
||||||
|
f"- dernier cycle automatise observe: {summary['updated_at']}",
|
||||||
|
]
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _render_plan_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||||
|
lines = [
|
||||||
|
f"- dernier verdict automatise: {summary['updated_at']}",
|
||||||
|
f"- accepted: {_comma_or_none(summary['accepted_models'])}",
|
||||||
|
f"- gate atteint: {_comma_or_none(summary['reached_gate_models'])}",
|
||||||
|
f"- prochain lot calcule: {next_lot}",
|
||||||
|
]
|
||||||
|
if summary["pending_manual_action"]:
|
||||||
|
lines.append(f"- checkpoint manuel requis: {summary['pending_manual_action']['reason']}")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _render_readme_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||||
|
lines = [
|
||||||
|
f"- dernier cycle automatise: {summary['updated_at']}",
|
||||||
|
f"- reference locale actuelle: {_reference_label(summary)}",
|
||||||
|
f"- prochain lot utile: {next_lot}",
|
||||||
|
"- lancer un cycle: `python3 scripts/run_next_lots.py --lot full`",
|
||||||
|
]
|
||||||
|
if summary["pending_manual_action"]:
|
||||||
|
lines.append(f"- checkpoint manuel en attente: {summary['pending_manual_action']['reason']}")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _render_runbook_sync(summary: dict[str, Any], project_state: dict[str, Any], next_lot: str) -> str:
|
||||||
|
lines = [
|
||||||
|
f"- dernier cycle automatise: {summary['updated_at']}",
|
||||||
|
f"- chapitre courant detecte: {project_state.get('current_chapter') or 'aucun'}",
|
||||||
|
f"- reference locale actuelle: {_reference_label(summary)}",
|
||||||
|
f"- prochain lot utile: {next_lot}",
|
||||||
|
]
|
||||||
|
if summary["pending_manual_action"]:
|
||||||
|
lines.append(f"- reprise attendue apres action manuelle: {summary['pending_manual_action']['resume_state']}")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _render_mascarade_todo_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||||
|
lines = [
|
||||||
|
f"- dernier cycle ANE automatise: {summary['updated_at']}",
|
||||||
|
f"- accepted via runtime local: {_comma_or_none(summary['accepted_models'])}",
|
||||||
|
f"- gate atteint via runtime local: {_comma_or_none(summary['reached_gate_models'])}",
|
||||||
|
f"- blocage runtime principal: {next_lot}",
|
||||||
|
]
|
||||||
|
if summary["pending_manual_action"]:
|
||||||
|
lines.append(f"- checkpoint runtime manuel: {summary['pending_manual_action']['reason']}")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _render_mascarade_plan_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||||
|
lines = [
|
||||||
|
f"- dernier cycle ANE automatise: {summary['updated_at']}",
|
||||||
|
f"- reference locale ANE: {_reference_label(summary)}",
|
||||||
|
f"- prochain lot ANE a servir: {next_lot}",
|
||||||
|
]
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _render_mascarade_readme_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||||
|
return "\n".join(
|
||||||
|
[
|
||||||
|
f"- dernier cycle ANE automatise: {summary['updated_at']}",
|
||||||
|
f"- etat de reference ANE: {_reference_label(summary)}",
|
||||||
|
f"- prochain lot utile cote pipeline: {next_lot}",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _render_mascarade_runbook_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||||
|
lines = [
|
||||||
|
f"- dernier cycle ANE automatise: {summary['updated_at']}",
|
||||||
|
f"- meilleurs candidats actuels: {_top_candidates(summary['results'])}",
|
||||||
|
f"- prochain lot utile cote ANE: {next_lot}",
|
||||||
|
]
|
||||||
|
if summary["pending_manual_action"]:
|
||||||
|
lines.append(f"- checkpoint runtime manuel: {summary['pending_manual_action']['reason']}")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _reference_label(summary: dict[str, Any]) -> str:
|
||||||
|
if summary["accepted_models"]:
|
||||||
|
return summary["accepted_models"][0]
|
||||||
|
if summary["reached_gate_models"]:
|
||||||
|
return f"aucun accepted, meilleur diagnostic: {summary['reached_gate_models'][0]}"
|
||||||
|
return "aucune reference accepted"
|
||||||
|
|
||||||
|
|
||||||
|
def _top_candidates(results: Iterable[ModelRunResult]) -> str:
|
||||||
|
candidates = []
|
||||||
|
for item in results:
|
||||||
|
if item.model in candidates:
|
||||||
|
continue
|
||||||
|
if item.model.startswith("apple-coreml:qwen3.5-4b") or item.model.startswith("ollama:qwen2.5:7b"):
|
||||||
|
candidates.append(item.model)
|
||||||
|
return ", ".join(candidates) if candidates else "aucun"
|
||||||
|
|
||||||
|
|
||||||
|
def _comma_or_none(items: list[str]) -> str:
|
||||||
|
return ", ".join(items) if items else "aucun"
|
||||||
|
|
||||||
|
|
||||||
|
def _render_comparison_markdown(state: RunState, results: list[ModelRunResult]) -> str:
|
||||||
|
lines = [
|
||||||
|
f"- dernier cycle automatise: {state.updated_at}",
|
||||||
|
"",
|
||||||
|
"| Modele | Categorie | Preflight | Smoke | Classification | Failed stage | Gate | Repairs | Notes |",
|
||||||
|
"|---|---|---|---|---|---|---|---:|---|",
|
||||||
|
]
|
||||||
|
for item in results:
|
||||||
|
lines.append(
|
||||||
|
"| {model} | {category} | {preflight} | {smoke} | {classification} | {failed_stage} | {gate} | {repairs} | {notes} |".format(
|
||||||
|
model=item.model,
|
||||||
|
category=item.category,
|
||||||
|
preflight="OK" if item.preflight_ok else ("KO" if item.preflight_ok is False else "n/a"),
|
||||||
|
smoke="oui" if item.smoke_attempted else "non",
|
||||||
|
classification=item.classification,
|
||||||
|
failed_stage=item.failed_stage or "",
|
||||||
|
gate="oui" if item.reached_gate() else "non",
|
||||||
|
repairs=item.repair_attempts,
|
||||||
|
notes="; ".join(item.notes) if item.notes else "",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def _render_summary_markdown(state: RunState, results: list[ModelRunResult]) -> str:
|
||||||
|
summary = _build_summary(state, results)
|
||||||
|
lines = [
|
||||||
|
"# Résumé du cycle automatique",
|
||||||
|
"",
|
||||||
|
f"- lot: `{state.lot}`",
|
||||||
|
f"- démarré: `{state.started_at}`",
|
||||||
|
f"- mis à jour: `{state.updated_at}`",
|
||||||
|
f"- accepted: {_comma_or_none(summary['accepted_models'])}",
|
||||||
|
f"- gate atteint: {_comma_or_none(summary['reached_gate_models'])}",
|
||||||
|
f"- quality_blocked: {_comma_or_none(summary['quality_blocked_models'])}",
|
||||||
|
f"- provider_failed: {_comma_or_none(summary['provider_failed_models'])}",
|
||||||
|
]
|
||||||
|
if state.pending_manual_action:
|
||||||
|
lines.extend(
|
||||||
|
[
|
||||||
|
"",
|
||||||
|
"## Checkpoint manuel",
|
||||||
|
f"- raison: {state.pending_manual_action['reason']}",
|
||||||
|
f"- commande: `{state.pending_manual_action['command']}`",
|
||||||
|
f"- reprise: `python3 scripts/run_next_lots.py --resume {state.pending_manual_action['resume_state']}`",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
if results:
|
||||||
|
lines.extend(["", "## Résultats", ""])
|
||||||
|
lines.append(_render_comparison_markdown(state, results))
|
||||||
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def build_parser() -> argparse.ArgumentParser:
|
||||||
|
parser = argparse.ArgumentParser(prog="python3 scripts/run_next_lots.py")
|
||||||
|
parser.add_argument("--manifest", default="automation/next_lots.toml")
|
||||||
|
parser.add_argument("--lot", default="full", choices=["full", "ensure_models", "runtime_preflight", "priority_models", "baselines", "tracking_sync"])
|
||||||
|
parser.add_argument("--resume", type=Path)
|
||||||
|
parser.add_argument("--dry-run", action="store_true")
|
||||||
|
parser.add_argument("--report-only", action="store_true")
|
||||||
|
return parser
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None, repo_root: Path | None = None) -> int:
|
||||||
|
parser = build_parser()
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
root = repo_root or Path.cwd()
|
||||||
|
manifest = Manifest.load(root, root / args.manifest)
|
||||||
|
runner = NextLotsRunner(manifest)
|
||||||
|
return runner.run(
|
||||||
|
lot=args.lot,
|
||||||
|
resume_state=args.resume,
|
||||||
|
dry_run=args.dry_run,
|
||||||
|
report_only=args.report_only,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,202 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
import json
|
||||||
|
|
||||||
|
from core.chapters import ChapterId, discover_chapter_dirs, discover_chapter_files
|
||||||
|
|
||||||
|
|
||||||
|
class ProjectState:
|
||||||
|
"""
|
||||||
|
Detects and summarizes the current state of a writing project.
|
||||||
|
Read-only, file-based, human-readable.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, root: Path):
|
||||||
|
self.root = root
|
||||||
|
self.manuscript = root / "manuscrit"
|
||||||
|
self.structure = root / "structure" / "chapitres"
|
||||||
|
self.drafts = root / "brouillons" / "chapitres"
|
||||||
|
self.memory = root / "memoire"
|
||||||
|
self.memory_chapters = self.memory / "chapitres"
|
||||||
|
self.intentions = root / "notes" / "intentions"
|
||||||
|
|
||||||
|
def detect_current_chapter(self) -> str | None:
|
||||||
|
chapters = self.known_chapters()
|
||||||
|
if not chapters:
|
||||||
|
return None
|
||||||
|
return chapters[-1].slug
|
||||||
|
|
||||||
|
def known_chapters(self) -> list[ChapterId]:
|
||||||
|
chapters: set[ChapterId] = set()
|
||||||
|
for chapter, _path in discover_chapter_files(self.intentions):
|
||||||
|
chapters.add(chapter)
|
||||||
|
for chapter, _path in discover_chapter_files(self.structure):
|
||||||
|
chapters.add(chapter)
|
||||||
|
for chapter, _path in discover_chapter_files(self.manuscript):
|
||||||
|
chapters.add(chapter)
|
||||||
|
for chapter, _path in discover_chapter_files(self.memory_chapters):
|
||||||
|
chapters.add(chapter)
|
||||||
|
for chapter, _path in discover_chapter_dirs(self.drafts):
|
||||||
|
chapters.add(chapter)
|
||||||
|
return sorted(chapters)
|
||||||
|
|
||||||
|
def latest_drafts(self) -> dict[str, str]:
|
||||||
|
latest: dict[str, str] = {}
|
||||||
|
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||||
|
meta = self._load_meta(draft_dir)
|
||||||
|
if meta:
|
||||||
|
artifacts = meta.get("artifacts", {})
|
||||||
|
if isinstance(artifacts, dict):
|
||||||
|
repair_latest = artifacts.get("repair_latest")
|
||||||
|
if isinstance(repair_latest, str) and repair_latest.strip():
|
||||||
|
latest[chapter.slug] = Path(repair_latest).name
|
||||||
|
continue
|
||||||
|
|
||||||
|
candidates = sorted(path.name for path in draft_dir.glob("draft_v*.md"))
|
||||||
|
if candidates:
|
||||||
|
latest[chapter.slug] = candidates[-1]
|
||||||
|
return latest
|
||||||
|
|
||||||
|
def latest_repairs(self) -> dict[str, str]:
|
||||||
|
latest: dict[str, str] = {}
|
||||||
|
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||||
|
meta = self._load_meta(draft_dir)
|
||||||
|
if not meta:
|
||||||
|
continue
|
||||||
|
artifacts = meta.get("artifacts", {})
|
||||||
|
if not isinstance(artifacts, dict):
|
||||||
|
continue
|
||||||
|
repair_latest = artifacts.get("repair_latest")
|
||||||
|
if isinstance(repair_latest, str) and repair_latest.strip():
|
||||||
|
latest[chapter.slug] = Path(repair_latest).name
|
||||||
|
return latest
|
||||||
|
|
||||||
|
def failed_chapters(self) -> list[dict[str, object]]:
|
||||||
|
failures: list[dict[str, object]] = []
|
||||||
|
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||||
|
meta = self._load_meta(draft_dir)
|
||||||
|
if not meta or meta.get("status") != "failed":
|
||||||
|
continue
|
||||||
|
failures.append(
|
||||||
|
{
|
||||||
|
"chapter": chapter.slug,
|
||||||
|
"status": str(meta.get("status", "")),
|
||||||
|
"failed_stage": str(meta.get("failed_stage", "")),
|
||||||
|
"meta_path": str(draft_dir / "meta.json"),
|
||||||
|
"retry_stages": self._retry_stages(meta),
|
||||||
|
"last_status_message": str(meta.get("last_status_message", "")).strip(),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return failures
|
||||||
|
|
||||||
|
def quality_blocked_chapters(self) -> list[dict[str, object]]:
|
||||||
|
blocked: list[dict[str, object]] = []
|
||||||
|
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||||
|
meta = self._load_meta(draft_dir)
|
||||||
|
if not meta or meta.get("status") != "quality_blocked":
|
||||||
|
continue
|
||||||
|
artifacts = meta.get("artifacts", {})
|
||||||
|
if not isinstance(artifacts, dict):
|
||||||
|
artifacts = {}
|
||||||
|
raw_blockers = meta.get("quality_blockers")
|
||||||
|
quality_blockers = []
|
||||||
|
if isinstance(raw_blockers, list):
|
||||||
|
quality_blockers = [str(item).strip() for item in raw_blockers if str(item).strip()]
|
||||||
|
blocked.append(
|
||||||
|
{
|
||||||
|
"chapter": chapter.slug,
|
||||||
|
"status": str(meta.get("status", "")),
|
||||||
|
"failed_stage": str(meta.get("failed_stage", "")),
|
||||||
|
"meta_path": str(draft_dir / "meta.json"),
|
||||||
|
"draft_path": str(artifacts.get("repair_latest") or artifacts.get("draft_v2", draft_dir / "draft_v2.md")),
|
||||||
|
"gate_path": str(artifacts.get("gate_v1", draft_dir / "gate_v1.json")),
|
||||||
|
"quality_blockers": quality_blockers,
|
||||||
|
"retry_stages": self._retry_stages(meta),
|
||||||
|
"repair_attempts": int(meta.get("repair_attempts", 0) or 0),
|
||||||
|
"repair_models": self._repair_models(meta),
|
||||||
|
"last_status_message": str(meta.get("last_status_message", "")).strip(),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return blocked
|
||||||
|
|
||||||
|
def awaiting_acceptance(self) -> list[dict[str, object]]:
|
||||||
|
pending: list[dict[str, object]] = []
|
||||||
|
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||||
|
meta = self._load_meta(draft_dir)
|
||||||
|
if not meta or meta.get("status") != "awaiting_acceptance":
|
||||||
|
continue
|
||||||
|
artifacts = meta.get("artifacts", {})
|
||||||
|
if not isinstance(artifacts, dict):
|
||||||
|
artifacts = {}
|
||||||
|
pending.append(
|
||||||
|
{
|
||||||
|
"chapter": chapter.slug,
|
||||||
|
"status": str(meta.get("status", "")),
|
||||||
|
"draft_path": str(artifacts.get("repair_latest") or artifacts.get("draft_v2", draft_dir / "draft_v2.md")),
|
||||||
|
"critique_path": str(artifacts.get("critique_v1", draft_dir / "critique_v1.md")),
|
||||||
|
"gate_path": str(artifacts.get("gate_v1", draft_dir / "gate_v1.json")),
|
||||||
|
"meta_path": str(draft_dir / "meta.json"),
|
||||||
|
"retry_stages": self._retry_stages(meta),
|
||||||
|
"repair_attempts": int(meta.get("repair_attempts", 0) or 0),
|
||||||
|
"repair_models": self._repair_models(meta),
|
||||||
|
"last_status_message": str(meta.get("last_status_message", "")).strip(),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return pending
|
||||||
|
|
||||||
|
def retry_stages(self) -> dict[str, list[str]]:
|
||||||
|
retries: dict[str, list[str]] = {}
|
||||||
|
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||||
|
meta = self._load_meta(draft_dir)
|
||||||
|
if not meta:
|
||||||
|
continue
|
||||||
|
stages = self._retry_stages(meta)
|
||||||
|
if stages:
|
||||||
|
retries[chapter.slug] = stages
|
||||||
|
return retries
|
||||||
|
|
||||||
|
def summary(self) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"project_root": str(self.root),
|
||||||
|
"current_chapter": self.detect_current_chapter(),
|
||||||
|
"known_chapters": [chapter.slug for chapter in self.known_chapters()],
|
||||||
|
"directories": {
|
||||||
|
"structure": self.structure.exists(),
|
||||||
|
"drafts": self.drafts.exists(),
|
||||||
|
"manuscript": self.manuscript.exists(),
|
||||||
|
"memory": self.memory.exists(),
|
||||||
|
},
|
||||||
|
"has_structure": self.structure.exists(),
|
||||||
|
"has_memory": self.memory.exists(),
|
||||||
|
"latest_drafts": self.latest_drafts(),
|
||||||
|
"latest_repairs": self.latest_repairs(),
|
||||||
|
"failed_chapters": self.failed_chapters(),
|
||||||
|
"quality_blocked_chapters": self.quality_blocked_chapters(),
|
||||||
|
"awaiting_acceptance": self.awaiting_acceptance(),
|
||||||
|
"retry_stages": self.retry_stages(),
|
||||||
|
}
|
||||||
|
|
||||||
|
def _load_meta(self, draft_dir: Path) -> dict[str, object] | None:
|
||||||
|
meta_path = draft_dir / "meta.json"
|
||||||
|
if not meta_path.exists():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
payload = json.loads(meta_path.read_text(encoding="utf-8"))
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return None
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
return None
|
||||||
|
return payload
|
||||||
|
|
||||||
|
def _retry_stages(self, meta: dict[str, object]) -> list[str]:
|
||||||
|
raw = meta.get("retry_stages")
|
||||||
|
if not isinstance(raw, list):
|
||||||
|
return []
|
||||||
|
return [str(item).strip() for item in raw if str(item).strip()]
|
||||||
|
|
||||||
|
def _repair_models(self, meta: dict[str, object]) -> list[str]:
|
||||||
|
raw = meta.get("repair_models")
|
||||||
|
if not isinstance(raw, list):
|
||||||
|
return []
|
||||||
|
return [str(item).strip() for item in raw if str(item).strip()]
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
from string import Template
|
||||||
|
import json
|
||||||
|
|
||||||
|
|
||||||
|
class PromptNotFoundError(FileNotFoundError):
|
||||||
|
"""Raised when a prompt file is missing."""
|
||||||
|
|
||||||
|
|
||||||
|
class PromptStore:
|
||||||
|
def __init__(self, root: Path):
|
||||||
|
self.root = root
|
||||||
|
self.prompts_dir = root / "prompts"
|
||||||
|
self.builtin_prompts_dir = Path(__file__).resolve().parents[1] / "prompts"
|
||||||
|
|
||||||
|
def render(self, name: str, **context: object) -> str:
|
||||||
|
path = self.prompts_dir / f"{name}_v1.txt"
|
||||||
|
if not path.exists():
|
||||||
|
path = self.builtin_prompts_dir / f"{name}_v1.txt"
|
||||||
|
if not path.exists():
|
||||||
|
raise PromptNotFoundError(f"Prompt introuvable: {path}")
|
||||||
|
|
||||||
|
template = Template(path.read_text(encoding="utf-8"))
|
||||||
|
normalized = {key: self._normalize(value) for key, value in context.items()}
|
||||||
|
return template.substitute(normalized)
|
||||||
|
|
||||||
|
def _normalize(self, value: object) -> str:
|
||||||
|
if value is None:
|
||||||
|
return ""
|
||||||
|
if isinstance(value, (dict, list)):
|
||||||
|
return json.dumps(value, ensure_ascii=False, indent=2)
|
||||||
|
return str(value)
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
# Plan d'execution - 7 mars 2026
|
||||||
|
|
||||||
|
Ordre recommande pour la suite de `ai-novel-engine`, base sur l'etat reel livre au 7 mars 2026.
|
||||||
|
|
||||||
|
## Lot 1 - Stabilisation locale Apple / Ollama
|
||||||
|
|
||||||
|
### Objectif
|
||||||
|
- verrouiller un run chapitre complet en local via `apple-coreml`
|
||||||
|
- rejouer le meme flux via `ollama`
|
||||||
|
- durcir la fin du pipeline sur les sorties JSON encore fragiles
|
||||||
|
|
||||||
|
### Done quand
|
||||||
|
- un chapitre complet passe jusqu'a la validation interactive puis a la promotion dans `manuscrit/` avec `apple-coreml`
|
||||||
|
- le meme chapitre passe avec `ollama` sans changer le pipeline narratif
|
||||||
|
- `critique` et `memory` disposent d'un second passage de reparation ou de reessai si le JSON reste invalide
|
||||||
|
|
||||||
|
### Risque principal
|
||||||
|
- le service Apple local `:8201` peut rester lent, bloquer une connexion ou degrader la validation sequentielle
|
||||||
|
|
||||||
|
### Dependances
|
||||||
|
- `mascarade` doit garder le shim `/v1/chat/completions` stable
|
||||||
|
- un backend `ollama` local doit etre disponible pour le second passage
|
||||||
|
- les budgets par etape doivent rester ajustables sans changer le pipeline
|
||||||
|
|
||||||
|
## Lot 2 - Workflow auteur et CLI non interactive
|
||||||
|
|
||||||
|
### Objectif
|
||||||
|
- rendre le workflow auteur exploitable en interactif et en batch local
|
||||||
|
- exposer plus clairement l'etat d'echec des chapitres
|
||||||
|
|
||||||
|
### Done quand
|
||||||
|
- `generate chapter` accepte `--approve` et `--reject`
|
||||||
|
- `status` expose les chapitres en echec, le dernier `failed_stage` et le dernier artefact utile
|
||||||
|
- le smoke local affiche un resume lisible sans ouvrir `meta.json`
|
||||||
|
|
||||||
|
### Risque principal
|
||||||
|
- la CLI peut devenir ambigue si les modes interactif et non interactif divergent
|
||||||
|
|
||||||
|
### Dependances
|
||||||
|
- les artefacts de pipeline doivent rester stables
|
||||||
|
- les metadonnees `meta.json` doivent contenir assez d'information pour alimenter `status`
|
||||||
|
|
||||||
|
## Lot 3 - Docs produit et runbooks
|
||||||
|
|
||||||
|
### Objectif
|
||||||
|
- remplacer les placeholders de doc produit
|
||||||
|
- figer les contrats cross-repo et les procedures de recuperation locales
|
||||||
|
|
||||||
|
### Done quand
|
||||||
|
- `docs/vision.md` et `docs/roadmap.md` ne sont plus des placeholders
|
||||||
|
- un runbook court de recuperation Apple local existe
|
||||||
|
- le contrat `mascarade` utile a `ai-novel-engine` est documente une fois, de facon stable
|
||||||
|
|
||||||
|
### Risque principal
|
||||||
|
- la doc peut diverger vite du runtime si elle est redigee avant la stabilisation locale
|
||||||
|
|
||||||
|
### Dependances
|
||||||
|
- le Lot 1 doit etre suffisamment stable pour produire des runbooks fiables
|
||||||
|
- `mascarade` doit figer le perimetre ANE suivi dans `TODO_AI_NOVEL_ENGINE.md`
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
# Plan d'execution - 8 mars 2026
|
||||||
|
|
||||||
|
Plan de reference apres livraison de la boucle `repair`.
|
||||||
|
|
||||||
|
Le plan du 7 mars 2026 reste archive pour historique. L'ordre recommande a date
|
||||||
|
est celui-ci.
|
||||||
|
|
||||||
|
Pilotage operationnel:
|
||||||
|
- lancer les lots avec `python3 scripts/run_next_lots.py --lot <lot>`
|
||||||
|
- utiliser `automation/next_lots.toml` comme source de verite pour l'ordre des smokes, les budgets et les fichiers de suivi
|
||||||
|
- en cas de switch Apple ou de restart runtime, reprendre ensuite avec `python3 scripts/run_next_lots.py --resume automation/state/next_lots_state.json`
|
||||||
|
|
||||||
|
## Lot 1 - Consolider la reference acceptee et finir les baselines
|
||||||
|
|
||||||
|
### Etat constate
|
||||||
|
- la boucle `repair` est livree, testee et visible dans `status` / `meta.json`
|
||||||
|
- `apple-coreml:qwen3.5-4b-onnx-q4f16` a termine un cycle complet et est `accepted`
|
||||||
|
- `ollama:qwen2.5:7b` atteint `gate`, exerce `repair` en live, puis reste `quality_blocked` sur `outline_like`
|
||||||
|
- le lot `baselines` est en cours pour `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b`
|
||||||
|
- le runtime Apple local n'expose qu'un seul `model_id` a la fois, ce qui limite le fallback `repair` entre modeles Apple au sein d'un meme smoke
|
||||||
|
|
||||||
|
### Objectif
|
||||||
|
- finir les baselines pour avoir un comparatif complet du protocole courant
|
||||||
|
- confirmer que la reference `apple-coreml:qwen3.5-4b-onnx-q4f16` est reproductible sur plus d'un cycle
|
||||||
|
- sortir `ollama:qwen2.5:7b` de `quality_blocked` sans degrader la prose utile
|
||||||
|
|
||||||
|
### Done quand
|
||||||
|
- le lot `baselines` est termine et synchronise
|
||||||
|
- `apple-coreml:qwen3.5-4b-onnx-q4f16` reste `accepted` sur un rerun de confirmation
|
||||||
|
- `ollama:qwen2.5:7b` finit soit `accepted`, soit `quality_blocked` avec un diagnostic resserre qui ne soit plus `outline_like`
|
||||||
|
|
||||||
|
### Risque principal
|
||||||
|
- la reference Apple 4B peut rester un succes isole si les switches runtime ou les budgets changent
|
||||||
|
|
||||||
|
### Dependances
|
||||||
|
- garder le garde-fou comme blocage dur
|
||||||
|
- conserver le protocole de comparaison commun et le meme preset qualite
|
||||||
|
- installer ou restager explicitement avant les reruns Apple:
|
||||||
|
- `qwen2.5-0.5b-instruct-onnx`
|
||||||
|
- `qwen3.5-4b-onnx-q4f16`
|
||||||
|
- `stateful-mistral7b-instruct-int4-coreml`
|
||||||
|
- verifier avant chaque rerun Apple que le bon `model_id` est effectivement charge sur `:8201`
|
||||||
|
|
||||||
|
## Lot 2 - Tuner `rewrite` et `repair` pour Ollama 7B
|
||||||
|
|
||||||
|
### Objectif
|
||||||
|
- garder `apple-coreml:qwen3.5-4b-onnx-q4f16` comme reference
|
||||||
|
- faire passer `ollama:qwen2.5:7b` de `quality_blocked` a `accepted`
|
||||||
|
- ne garder les petits modeles que comme baselines vitesse ou regressions
|
||||||
|
|
||||||
|
### Ordre recommande
|
||||||
|
1. finir `apple-coreml:qwen2.5-0.5b-instruct-onnx`
|
||||||
|
2. finir `ollama:qwen2.5:1.5b`
|
||||||
|
3. rejouer `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||||
|
4. rejouer `ollama:qwen2.5:7b`
|
||||||
|
5. `ollama:qwen3.5:9b` seulement si `qwen2.5:7b` termine un smoke complet
|
||||||
|
|
||||||
|
### Done quand
|
||||||
|
- le comparatif distingue clairement:
|
||||||
|
- le modele de reference ANE actuel
|
||||||
|
- le meilleur candidat Apple actuel
|
||||||
|
- le meilleur candidat Ollama actuel
|
||||||
|
- les baselines vitesse a conserver ou a sortir
|
||||||
|
- le meilleur compromis Apple
|
||||||
|
- le candidat vitesse encore insuffisant
|
||||||
|
- les modeles a sortir de la reference locale
|
||||||
|
|
||||||
|
### Risque principal
|
||||||
|
- les meilleurs candidats peuvent rester meilleurs sur la qualite, mais encore hors reference tant que `rewrite` ne passe pas
|
||||||
|
|
||||||
|
### Dependances
|
||||||
|
- chemin Ollama de reference: Docker CPU via `mascarade`
|
||||||
|
- service Apple `:8201` stable pendant tout le smoke
|
||||||
|
- les trois modeles Apple cibles doivent etre installes et visibles cote runtime avant comparaison:
|
||||||
|
- `qwen2.5-0.5b-instruct-onnx`
|
||||||
|
- `qwen3.5-4b-onnx-q4f16`
|
||||||
|
- `stateful-mistral7b-instruct-int4-coreml`
|
||||||
|
- temps borne par requete pour garder des verdicts comparables
|
||||||
|
|
||||||
|
## Lot 3 - Docs et runbooks finaux
|
||||||
|
|
||||||
|
### Objectif
|
||||||
|
- maintenir les README, TODOs, runbooks et le comparatif alignes sur l'etat reel
|
||||||
|
|
||||||
|
### Done quand
|
||||||
|
- les docs distinguent clairement le modele `accepted`, les modeles `quality_blocked` et les baselines encore en rerun
|
||||||
|
- le comparatif et les runbooks renvoient tous vers ce plan du 8 mars 2026
|
||||||
|
- les TODOs n'exposent plus d'items deja livres
|
||||||
|
|
||||||
|
### Risque principal
|
||||||
|
- la doc redevient trop optimiste si elle est mise a jour avant la revalidation complete
|
||||||
|
|
||||||
|
### Dependances
|
||||||
|
- les lots 1 et 2 doivent produire des resultats reels, pas des suppositions
|
||||||
|
|
||||||
|
## Auto-sync
|
||||||
|
## Auto-sync
|
||||||
|
<!-- AUTO-SYNC:ANE-PLAN:START -->
|
||||||
|
- dernier verdict automatise: 2026-03-09T06:53:02+00:00
|
||||||
|
- accepted: aucun
|
||||||
|
- gate atteint: apple-coreml:qwen2.5-0.5b-instruct-onnx, ollama:qwen2.5:1.5b
|
||||||
|
- prochain lot calcule: Analyser les runs ayant atteint gate/repair puis resserrer la reference locale autour des meilleurs candidats.
|
||||||
|
- checkpoint manuel requis: Le runtime Apple sert `qwen2.5-0.5b-instruct-onnx` au lieu de `stateful-mistral7b-instruct-int4-coreml`.
|
||||||
|
<!-- AUTO-SYNC:ANE-PLAN:END -->
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
# Comparatif local ANE - 8 mars 2026
|
||||||
|
|
||||||
|
Comparatif realise avec le protocole courant:
|
||||||
|
|
||||||
|
- meme intention de smoke
|
||||||
|
- meme chapitre `02`
|
||||||
|
- meme CLI publique `generate chapter --chapter 02 --approve`
|
||||||
|
- meme preset qualite:
|
||||||
|
- `ANE_MAX_TOKENS_STRUCTURE=256`
|
||||||
|
- `ANE_MAX_TOKENS_DRAFT=768`
|
||||||
|
- `ANE_MAX_TOKENS_CRITIQUE=512`
|
||||||
|
- `ANE_MAX_TOKENS_REWRITE=768`
|
||||||
|
- `ANE_MAX_TOKENS_GATE=384`
|
||||||
|
- `ANE_MAX_TOKENS_REPAIR=512`
|
||||||
|
- `ANE_MAX_TOKENS_MEMORY=320`
|
||||||
|
- `ANE_REPAIR_MAX_PASSES=2`
|
||||||
|
- meme timeout borne par requete:
|
||||||
|
- `300s`
|
||||||
|
- meme garde-fou manuscrit dur et meme boucle `repair`
|
||||||
|
|
||||||
|
Contexte machine:
|
||||||
|
|
||||||
|
- `ai-novel-engine` pointe vers `mascarade` sur `http://127.0.0.1:8100`
|
||||||
|
- `ollama` est route vers un service Docker CPU expose sur `127.0.0.1:11435`
|
||||||
|
- le host `ollama` natif 0.17.7 reste bloque par un crash Metal sur cette machine
|
||||||
|
- le runtime Apple local n'expose qu'un seul `model_id` a la fois sur `:8201`
|
||||||
|
- dernier cycle complet termine au 9 mars 2026:
|
||||||
|
- `apple-coreml:qwen3.5-4b-onnx-q4f16` est `accepted`
|
||||||
|
- `ollama:qwen2.5:7b` atteint `gate`, exerce `repair` puis finit `quality_blocked`
|
||||||
|
- le lot `baselines` est relance separement pour les petits modeles
|
||||||
|
|
||||||
|
## Resultats
|
||||||
|
|
||||||
|
| Modele | Backend | Preflight | Smoke complet | Statut final | Derniere etape atteinte | Total observe | Prose / narration | JSON / controle | Verdict |
|
||||||
|
|---|---|---|---|---|---|---:|---|---|---|
|
||||||
|
| `apple-coreml:qwen3.5-4b-onnx-q4f16` | `apple-coreml` | OK | oui | `accepted` | `memory` | `711s` | meilleure nuance narrative du lot | critique exploitable, gate vert | reference ANE locale actuelle |
|
||||||
|
| `ollama:qwen2.5:7b` | `ollama` | OK | oui | `quality_blocked` | `gate` | `825s` | correcte, plus sobre que l'Apple 4B | critique exploitable, mais le texte reste trop proche d'un plan | meilleur candidat Ollama, encore bloque |
|
||||||
|
| `apple-coreml:qwen2.5-0.5b-instruct-onnx` | `apple-coreml` | OK | rerun en cours | n/a | n/a | n/a | baseline vitesse a requalifier | n/a | en attente de verdict courant |
|
||||||
|
| `ollama:qwen2.5:1.5b` | `ollama` | OK | rerun en cours | n/a | n/a | n/a | baseline vitesse a requalifier | n/a | en attente de verdict courant |
|
||||||
|
|
||||||
|
Point legacy hors protocole courant:
|
||||||
|
|
||||||
|
| Modele | Backend | Preflight | Smoke complet | Statut final |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| `apple-coreml:stateful-mistral7b-instruct-int4-coreml` | `apple-coreml` | OK | bloque > `8 min` a `structure` | `preflight_only` |
|
||||||
|
|
||||||
|
## Lecture rapide
|
||||||
|
|
||||||
|
### `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||||
|
- passe `structure`, `draft`, `critique`, `rewrite`, `gate` puis `memory`
|
||||||
|
- fournit le premier run `accepted` sous protocole `gate + repair`
|
||||||
|
- devient la reference ANE locale actuelle
|
||||||
|
- doit encore etre confirme sur rerun de stabilite
|
||||||
|
|
||||||
|
### `ollama:qwen2.5:7b`
|
||||||
|
- passe `structure`, `draft`, `critique`, `rewrite` puis `gate`
|
||||||
|
- exerce `repair` en live sur deux passes
|
||||||
|
- reste bloque sur `outline_like`
|
||||||
|
- c'est le meilleur candidat Ollama actuel, mais il lui manque encore une prose plus continue
|
||||||
|
|
||||||
|
### `apple-coreml:qwen2.5-0.5b-instruct-onnx`
|
||||||
|
- rerun baseline en cours via le lot `baselines`
|
||||||
|
- reste utile comme candidat vitesse Apple, pas comme reference qualite tant qu'un verdict courant n'est pas resynchronise
|
||||||
|
|
||||||
|
### `ollama:qwen2.5:1.5b`
|
||||||
|
- rerun baseline en cours via le lot `baselines`
|
||||||
|
- reste un temoin de regression plus qu'un candidat qualite
|
||||||
|
|
||||||
|
## Verdicts
|
||||||
|
|
||||||
|
- **Modele de reference ANE**: `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||||
|
- **Meilleur compromis Apple**: `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||||
|
- **Meilleur compromis Ollama**: `ollama:qwen2.5:7b`
|
||||||
|
- **Modele rapide mais insuffisant**: `apple-coreml:qwen2.5-0.5b-instruct-onnx`
|
||||||
|
- **Modeles a eviter pour la redaction longue sur cette machine**: `ollama:qwen2.5:1.5b` et `apple-coreml:stateful-mistral7b-instruct-int4-coreml`
|
||||||
|
|
||||||
|
## Conclusion du cycle
|
||||||
|
|
||||||
|
Le cycle `priority_models` atteint enfin un objectif produit minimal:
|
||||||
|
|
||||||
|
- la boucle `repair` est implementée, testee et visible dans `status` / `meta.json`
|
||||||
|
- `repair` a maintenant une validation live sur `ollama:qwen2.5:7b`
|
||||||
|
- un premier modele est `accepted` sous protocole courant: `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||||
|
- le prochain enjeu n'est plus de trouver un premier succes, mais de finir les baselines et de sortir `ollama:qwen2.5:7b` de `outline_like`
|
||||||
|
|
||||||
|
Le prochain lot logique n'est plus "ajouter un garde-fou", mais:
|
||||||
|
|
||||||
|
1. finir le lot `baselines`
|
||||||
|
2. confirmer `apple-coreml:qwen3.5-4b-onnx-q4f16` sur rerun
|
||||||
|
3. regler `rewrite` et `repair` pour faire tomber `outline_like` sur `ollama:qwen2.5:7b`
|
||||||
|
4. ne garder `qwen2.5-0.5b` et `qwen2.5:1.5b` que comme baselines vitesse
|
||||||
|
|
||||||
|
## Auto-sync
|
||||||
|
## Auto-sync
|
||||||
|
<!-- AUTO-SYNC:ANE-COMPARISON:START -->
|
||||||
|
- dernier cycle automatise: 2026-03-09T06:53:02+00:00
|
||||||
|
|
||||||
|
| Modele | Categorie | Preflight | Smoke | Classification | Failed stage | Gate | Repairs | Notes |
|
||||||
|
|---|---|---|---|---|---|---|---:|---|
|
||||||
|
| apple-coreml:qwen2.5-0.5b-instruct-onnx | baselines | OK | oui | quality_blocked | gate | oui | 2 | |
|
||||||
|
| ollama:qwen2.5:1.5b | baselines | OK | oui | quality_blocked | gate | oui | 2 | |
|
||||||
|
<!-- AUTO-SYNC:ANE-COMPARISON:END -->
|
||||||
+28
-1
@@ -1 +1,28 @@
|
|||||||
Roadmap v2 du projet.
|
# Roadmap v2
|
||||||
|
|
||||||
|
Roadmap courte et concrete, alignee sur l'etat reel du repo.
|
||||||
|
|
||||||
|
## Priorite 1 - Passer au moins un modele jusqu'a `gate`
|
||||||
|
|
||||||
|
- compacter `rewrite` pour qu'au moins un modele atteigne `gate`
|
||||||
|
- conserver la boucle `repair` et le garde-fou comme blocages durs
|
||||||
|
- viser d'abord `apple-coreml:qwen3.5-4b-onnx-q4f16` et `ollama:qwen2.5:7b`
|
||||||
|
|
||||||
|
## Priorite 2 - Requalifier les modeles plus lourds
|
||||||
|
|
||||||
|
- garder `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b` comme baselines vitesse
|
||||||
|
- rejouer `qwen3.5:9b` seulement si `qwen2.5:7b` termine un smoke complet
|
||||||
|
- maintenir les modeles toujours explicites dans les smokes et la doc
|
||||||
|
- tenir compte du fait que le runtime Apple local ne sert qu'un `model_id` a la fois
|
||||||
|
|
||||||
|
## Priorite 3 - Exploitation locale et docs
|
||||||
|
|
||||||
|
- runbook local ANE centre sur `rewrite`, `gate_v1.json`, `repair_vN.md` et `quality_blocked`
|
||||||
|
- runbook Apple local cote `mascarade` aligne sur les statuts reels
|
||||||
|
- README et suivi croises pointent vers `EXECUTION_PLAN_2026-03-08.md`
|
||||||
|
|
||||||
|
## Source de verite
|
||||||
|
|
||||||
|
- backlog actif: [`../TODO_ACTIVE.md`](../TODO_ACTIVE.md)
|
||||||
|
- etat livre: [`../TODO_IMPLEMENTE.md`](../TODO_IMPLEMENTE.md)
|
||||||
|
- ordre d'execution: [`EXECUTION_PLAN_2026-03-08.md`](./EXECUTION_PLAN_2026-03-08.md)
|
||||||
|
|||||||
@@ -0,0 +1,142 @@
|
|||||||
|
# Runbook local - generation ANE
|
||||||
|
|
||||||
|
Runbook court pour lancer et diagnostiquer la generation locale via `mascarade`.
|
||||||
|
|
||||||
|
Comparatif de reference: [`docs/MODEL_COMPARISON_2026-03-08.md`](../MODEL_COMPARISON_2026-03-08.md)
|
||||||
|
|
||||||
|
## Prerequis
|
||||||
|
|
||||||
|
- `mascarade` repond sur `http://127.0.0.1:8100/health`
|
||||||
|
- le modele reste explicite via `ANE_MODEL` ou `--model`
|
||||||
|
- une intention existe pour le chapitre cible
|
||||||
|
- le garde-fou manuscrit et la boucle `repair` peuvent bloquer la promotion meme avec `--approve`
|
||||||
|
|
||||||
|
## Cycle automatise
|
||||||
|
|
||||||
|
Commande de reference:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/run_next_lots.py --lot full
|
||||||
|
```
|
||||||
|
|
||||||
|
Commandes utiles:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/run_next_lots.py --lot priority_models
|
||||||
|
python3 scripts/run_next_lots.py --resume automation/state/next_lots_state.json
|
||||||
|
python3 scripts/run_next_lots.py --lot tracking_sync --report-only
|
||||||
|
```
|
||||||
|
|
||||||
|
Le driver:
|
||||||
|
- lit `automation/next_lots.toml`
|
||||||
|
- rejoue preflights et smokes dans l'ordre utile du moment
|
||||||
|
- met a jour les sections `AUTO-SYNC` des TODOs, plans, README et runbooks
|
||||||
|
- attend brievement que `/models` reflète le bon `model_id` apres un switch Apple avant de recréer un checkpoint manuel
|
||||||
|
- s'arrete proprement si un restart runtime ou un switch Apple est requis, puis imprime la commande de reprise
|
||||||
|
|
||||||
|
## Contrat local
|
||||||
|
|
||||||
|
`ai-novel-engine` parle uniquement a `mascarade` sur `POST /v1/chat/completions`.
|
||||||
|
|
||||||
|
- format modele: `provider:model`
|
||||||
|
- dernier cycle complet termine au 9 mars 2026:
|
||||||
|
- `apple-coreml:qwen3.5-4b-onnx-q4f16` est `accepted` de bout en bout sous garde-fou
|
||||||
|
- `ollama:qwen2.5:7b` atteint `gate`, exerce `repair` en live, puis finit `quality_blocked` sur `outline_like`
|
||||||
|
- `apple-coreml:stateful-mistral7b-instruct-int4-coreml` reste `preflight_only`
|
||||||
|
- le lot `baselines` pour `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b` est rejoue separement
|
||||||
|
- le runtime Apple local ne sert qu'un seul `model_id` a la fois
|
||||||
|
- le fallback `repair` n'essaie plus de changer de modele `apple-coreml` en plein smoke; tout switch Apple reste une action runtime explicite
|
||||||
|
|
||||||
|
## Smoke Apple
|
||||||
|
|
||||||
|
Preflight minimal cote runtime:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash /Users/electron/mascarade/scripts/smoke_openai_compat_ane.sh \
|
||||||
|
--url http://127.0.0.1:8100 \
|
||||||
|
--model "apple-coreml:qwen3.5-4b-onnx-q4f16"
|
||||||
|
```
|
||||||
|
|
||||||
|
Smoke chapitre complet:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/smoke_local_generation.sh \
|
||||||
|
--base-url http://127.0.0.1:8100 \
|
||||||
|
--model "apple-coreml:qwen3.5-4b-onnx-q4f16" \
|
||||||
|
--approve
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
- le script fait un warm-up automatique via `:8100` quand le modele commence par `apple-coreml:`
|
||||||
|
- le premier chargement Core ML peut etre long
|
||||||
|
- le smoke Apple applique par defaut un timeout plus large et des budgets plus courts; utiliser `--timeout` ou `ANE_MAX_TOKENS_*` pour durcir ou assouplir
|
||||||
|
- `ANE_MAX_TOKENS_GATE` permet de regler le budget du garde-fou LLM
|
||||||
|
- `ANE_MAX_TOKENS_REPAIR` et `ANE_REPAIR_MAX_PASSES` reglent la boucle `repair`
|
||||||
|
- `apple-coreml:qwen2.5-0.5b-instruct-onnx` reste le candidat vitesse Apple a requalifier en baseline
|
||||||
|
- `apple-coreml:qwen3.5-4b-onnx-q4f16` est la reference Apple locale actuelle
|
||||||
|
- `apple-coreml:stateful-mistral7b-instruct-int4-coreml` reste preflight-only sur cette machine: il repond, mais le smoke ANE est reste bloque a `structure` pendant plus de 8 minutes
|
||||||
|
|
||||||
|
## Smoke Ollama
|
||||||
|
|
||||||
|
Preflight:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash /Users/electron/mascarade/scripts/smoke_openai_compat_ane.sh \
|
||||||
|
--url http://127.0.0.1:8100 \
|
||||||
|
--model "ollama:qwen2.5:1.5b"
|
||||||
|
```
|
||||||
|
|
||||||
|
Le provider `ollama` doit apparaitre dans `providers` et le modele cible doit etre deja installe.
|
||||||
|
Sur cette machine, le meilleur candidat Ollama courant est `ollama:qwen2.5:7b`; `qwen2.5:1.5b` reste une baseline a rerun.
|
||||||
|
|
||||||
|
Smoke chapitre complet:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/smoke_local_generation.sh \
|
||||||
|
--base-url http://127.0.0.1:8100 \
|
||||||
|
--model "ollama:qwen2.5:1.5b" \
|
||||||
|
--approve
|
||||||
|
```
|
||||||
|
|
||||||
|
## Lire rapidement le resultat
|
||||||
|
|
||||||
|
Le smoke affiche un resume humain:
|
||||||
|
|
||||||
|
- `backend`
|
||||||
|
- `chapter`
|
||||||
|
- `status`
|
||||||
|
- `accepted`
|
||||||
|
- `failed_stage` si present
|
||||||
|
- `quality_blockers` si presents
|
||||||
|
- `retry_stages` si present
|
||||||
|
- `repair_attempts` et `repair_models` si presents
|
||||||
|
- chemins vers `draft_v2`, `repair_latest`, `critique_v1`, `gate_v1`, `manuscript`, `memory_summary`, `meta.json`
|
||||||
|
|
||||||
|
Le fichier de reference reste:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cat brouillons/chapitres/chapitre_XX/meta.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Champs utiles:
|
||||||
|
- `status`
|
||||||
|
- `last_status_message`
|
||||||
|
- `stage_attempts`
|
||||||
|
- `retry_stages`
|
||||||
|
- `failed_stage`
|
||||||
|
- `quality_blockers`
|
||||||
|
- `repair_attempts`
|
||||||
|
- `repair_models`
|
||||||
|
- `gate_report`
|
||||||
|
- `provider.base_url`
|
||||||
|
- `provider.model`
|
||||||
|
|
||||||
|
## Etat auto-synchronise
|
||||||
|
## Etat auto-synchronise
|
||||||
|
<!-- AUTO-SYNC:ANE-RUNBOOK:START -->
|
||||||
|
- dernier cycle automatise: 2026-03-09T06:53:02+00:00
|
||||||
|
- chapitre courant detecte: chapitre_01
|
||||||
|
- reference locale actuelle: aucun accepted, meilleur diagnostic: apple-coreml:qwen2.5-0.5b-instruct-onnx
|
||||||
|
- prochain lot utile: Analyser les runs ayant atteint gate/repair puis resserrer la reference locale autour des meilleurs candidats.
|
||||||
|
- reprise attendue apres action manuelle: /Users/electron/Documents/Projets_Creatifs/ai-novel-engine/automation/state/next_lots_state.json
|
||||||
|
<!-- AUTO-SYNC:ANE-RUNBOOK:END -->
|
||||||
+45
-1
@@ -1 +1,45 @@
|
|||||||
Vision du projet AI Novel Engine.
|
# Vision AI Novel Engine
|
||||||
|
|
||||||
|
## Positionnement
|
||||||
|
|
||||||
|
AI Novel Engine est un moteur narratif strict, local-first, pour projets longs.
|
||||||
|
|
||||||
|
Le but n'est pas de "discuter avec un chatbot qui écrit un roman". Le but est
|
||||||
|
de garder un pipeline lisible, reproductible et contrôlable par l'auteur:
|
||||||
|
|
||||||
|
`intention -> structure -> draft -> critique -> rewrite -> gate -> validation -> memoire`
|
||||||
|
|
||||||
|
## Ce que porte le produit
|
||||||
|
|
||||||
|
- l'auteur reste decisionnaire a chaque promotion vers le manuscrit
|
||||||
|
- aucune generation sans intention explicite
|
||||||
|
- aucune promotion vers le manuscrit si le garde-fou qualite bloque
|
||||||
|
- la memoire reste externe, inspectable et persistée sur disque
|
||||||
|
- les artefacts intermediaires sont lisibles en Markdown et JSON
|
||||||
|
- le moteur narratif reste decouple du runtime local
|
||||||
|
|
||||||
|
## Architecture cible
|
||||||
|
|
||||||
|
- `ai-novel-engine` porte la logique auteuriale, le pipeline, les prompts et la mémoire
|
||||||
|
- `mascarade` porte le runtime local, le routage provider et le shim OpenAI-compatible
|
||||||
|
- le contrat entre les deux reste minimal:
|
||||||
|
- `POST /v1/chat/completions`
|
||||||
|
- `model=provider:model`
|
||||||
|
- non-streaming
|
||||||
|
- JSON best effort, avec reessai applicatif cote ANE
|
||||||
|
|
||||||
|
## Non-objectifs v1
|
||||||
|
|
||||||
|
- chat libre comme interface principale
|
||||||
|
- studio web riche ou collaboratif
|
||||||
|
- autonomie complete "idee -> manuscrit final"
|
||||||
|
- base de donnees opaque pour la mémoire
|
||||||
|
|
||||||
|
## Critere de valeur
|
||||||
|
|
||||||
|
Le systeme est utile si un auteur peut:
|
||||||
|
|
||||||
|
- relancer un chapitre sans perdre le contexte de travail
|
||||||
|
- comprendre pourquoi une etape a echoue
|
||||||
|
- changer de backend local sans rewriter le pipeline narratif
|
||||||
|
- relire les brouillons, critiques et mises a jour memoire hors de l'IA
|
||||||
|
|||||||
@@ -0,0 +1,32 @@
|
|||||||
|
Tu es le rôle Contrôle du moteur AI Novel Engine.
|
||||||
|
La tentative précédente n'a pas produit un JSON exploitable.
|
||||||
|
Réponds à nouveau avec un seul objet JSON valide.
|
||||||
|
Ne mets aucun texte avant ou après le JSON.
|
||||||
|
Ne mets aucun bloc Markdown.
|
||||||
|
Si la tentative précédente est inutilisable, regénère le diagnostic à partir du contexte.
|
||||||
|
Limite-toi à 1 phrase de résumé, 3 écarts max et 3 recommandations max.
|
||||||
|
|
||||||
|
Chapitre cible: $chapter_slug
|
||||||
|
|
||||||
|
Erreur de parsing observée:
|
||||||
|
$parse_error
|
||||||
|
|
||||||
|
Tentative précédente à corriger si possible:
|
||||||
|
$invalid_response
|
||||||
|
|
||||||
|
Intention:
|
||||||
|
$intention
|
||||||
|
|
||||||
|
Structure attendue:
|
||||||
|
$structure_markdown
|
||||||
|
|
||||||
|
Brouillon à critiquer:
|
||||||
|
$draft_markdown
|
||||||
|
|
||||||
|
Format JSON strict:
|
||||||
|
{
|
||||||
|
"summary": "résumé bref du diagnostic",
|
||||||
|
"rewrite_required": true,
|
||||||
|
"deviations": ["écart 1", "écart 2"],
|
||||||
|
"recommendations": ["recommandation 1", "recommandation 2"]
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
Tu es le rôle Contrôle du moteur AI Novel Engine.
|
||||||
|
Analyse le brouillon et renvoie uniquement un objet JSON valide.
|
||||||
|
Réponse compacte obligatoire.
|
||||||
|
Ne mets aucun texte avant ou après le JSON.
|
||||||
|
Ne mets aucun bloc Markdown.
|
||||||
|
Limite-toi à 1 phrase de résumé, 3 écarts max et 3 recommandations max.
|
||||||
|
|
||||||
|
Chapitre cible: $chapter_slug
|
||||||
|
|
||||||
|
Intention:
|
||||||
|
$intention
|
||||||
|
|
||||||
|
Structure attendue:
|
||||||
|
$structure_markdown
|
||||||
|
|
||||||
|
Brouillon à critiquer:
|
||||||
|
$draft_markdown
|
||||||
|
|
||||||
|
Format JSON strict:
|
||||||
|
{
|
||||||
|
"summary": "résumé bref du diagnostic",
|
||||||
|
"rewrite_required": true,
|
||||||
|
"deviations": ["écart 1", "écart 2"],
|
||||||
|
"recommendations": ["recommandation 1", "recommandation 2"]
|
||||||
|
}
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
Tu es le rôle Production du moteur AI Novel Engine.
|
||||||
|
Tu rédiges un brouillon de chapitre fidèle à l'intention et à la structure.
|
||||||
|
|
||||||
|
Chapitre cible: $chapter_slug
|
||||||
|
|
||||||
|
Intention:
|
||||||
|
$intention
|
||||||
|
|
||||||
|
Structure validée:
|
||||||
|
$structure_markdown
|
||||||
|
|
||||||
|
Contexte projet:
|
||||||
|
$story_context
|
||||||
|
|
||||||
|
Consignes:
|
||||||
|
- répondre uniquement avec le chapitre en Markdown
|
||||||
|
- produire uniquement de la prose narrative continue, sous forme de paragraphes
|
||||||
|
- ne jamais recopier la structure sous forme de plan
|
||||||
|
- interdit: titres Markdown (`#`, `##`, `###`), listes a puces, numerotations, labels `objectif`, `conflit`, `sortie`, section `Tension`, section `Scènes`, code fences
|
||||||
|
- garder une voix cohérente
|
||||||
|
- matérialiser la tension annoncée
|
||||||
|
- ouvrir directement dans l'action ou dans la scene, sans preambule meta
|
||||||
|
- transformer chaque beat de structure en action, perception, decision, consequence et, si utile, dialogue
|
||||||
|
- finir sur une vraie phrase complete avec une ponctuation finale
|
||||||
|
- viser un chapitre bref mais complet, d'au moins 3 paragraphes substantiels
|
||||||
|
- ne pas ajouter d'explication hors texte narratif
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
Tu es le rôle Garde-fou manuscrit du moteur AI Novel Engine.
|
||||||
|
La tentative precedente n'a pas produit un JSON exploitable.
|
||||||
|
Reponds a nouveau avec un seul objet JSON valide.
|
||||||
|
Ne mets aucun texte avant ou apres le JSON.
|
||||||
|
Ne mets aucun bloc Markdown.
|
||||||
|
Si la tentative precedente est inutilisable, regénere le diagnostic a partir du brouillon final.
|
||||||
|
Bloque si le texte ressemble encore a un plan, s'il semble coupe avant sa fin, ou s'il manque une vraie continuite narrative.
|
||||||
|
Limite-toi a 1 phrase de resume, 4 blockers max et 4 recommandations max.
|
||||||
|
|
||||||
|
Chapitre cible: $chapter_slug
|
||||||
|
|
||||||
|
Erreur de parsing observee:
|
||||||
|
$parse_error
|
||||||
|
|
||||||
|
Tentative precedente a corriger si possible:
|
||||||
|
$invalid_response
|
||||||
|
|
||||||
|
Intention:
|
||||||
|
$intention
|
||||||
|
|
||||||
|
Structure attendue:
|
||||||
|
$structure_markdown
|
||||||
|
|
||||||
|
Brouillon final:
|
||||||
|
$draft_markdown
|
||||||
|
|
||||||
|
Format JSON strict:
|
||||||
|
{
|
||||||
|
"ready_for_manuscript": true,
|
||||||
|
"summary": "diagnostic bref",
|
||||||
|
"blockers": ["outline_like"],
|
||||||
|
"recommendations": ["retirer les titres", "terminer la scene"],
|
||||||
|
"heuristic_blockers": []
|
||||||
|
}
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
Tu es le rôle Garde-fou manuscrit du moteur AI Novel Engine.
|
||||||
|
Analyse le brouillon final et renvoie uniquement un objet JSON valide.
|
||||||
|
Ne mets aucun texte avant ou après le JSON.
|
||||||
|
Ne mets aucun bloc Markdown.
|
||||||
|
Le but est de decider si ce texte peut etre promu dans le manuscrit.
|
||||||
|
Bloque si le texte ressemble encore a un plan, s'il semble coupe avant sa fin, ou s'il manque une vraie continuite narrative.
|
||||||
|
Limite-toi a 1 phrase de resume, 4 blockers max et 4 recommandations max.
|
||||||
|
|
||||||
|
Chapitre cible: $chapter_slug
|
||||||
|
|
||||||
|
Intention:
|
||||||
|
$intention
|
||||||
|
|
||||||
|
Structure attendue:
|
||||||
|
$structure_markdown
|
||||||
|
|
||||||
|
Brouillon final:
|
||||||
|
$draft_markdown
|
||||||
|
|
||||||
|
Format JSON strict:
|
||||||
|
{
|
||||||
|
"ready_for_manuscript": true,
|
||||||
|
"summary": "diagnostic bref",
|
||||||
|
"blockers": ["outline_like"],
|
||||||
|
"recommendations": ["retirer les titres", "terminer la scene"],
|
||||||
|
"heuristic_blockers": []
|
||||||
|
}
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
Tu es le rôle Mémoire du moteur AI Novel Engine.
|
||||||
|
La tentative précédente n'a pas produit un JSON exploitable.
|
||||||
|
Réponds à nouveau avec un seul objet JSON valide.
|
||||||
|
Ne mets aucun texte avant ou après le JSON.
|
||||||
|
Ne mets aucun bloc Markdown.
|
||||||
|
Si la tentative précédente est inutilisable, regénère la mémoire à partir du chapitre accepté.
|
||||||
|
Limite-toi à 1 résumé, 3 personnages max, 3 lieux max et 5 événements max.
|
||||||
|
|
||||||
|
Chapitre cible: $chapter_slug
|
||||||
|
|
||||||
|
Erreur de parsing observée:
|
||||||
|
$parse_error
|
||||||
|
|
||||||
|
Tentative précédente à corriger si possible:
|
||||||
|
$invalid_response
|
||||||
|
|
||||||
|
Contexte projet:
|
||||||
|
$story_context
|
||||||
|
|
||||||
|
Chapitre accepté:
|
||||||
|
$accepted_draft
|
||||||
|
|
||||||
|
Format JSON strict:
|
||||||
|
{
|
||||||
|
"summary": "résumé factuel du chapitre",
|
||||||
|
"characters": [
|
||||||
|
{"name": "Nom", "description": "Rôle ou évolution"}
|
||||||
|
],
|
||||||
|
"locations": [
|
||||||
|
{"name": "Lieu", "description": "Ce qui y est établi"}
|
||||||
|
],
|
||||||
|
"timeline_events": [
|
||||||
|
{"event": "Fait important", "order_hint": "optionnel"}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
Tu es le rôle Mémoire du moteur AI Novel Engine.
|
||||||
|
Tu extrais une mémoire exploitable à partir d'un chapitre accepté.
|
||||||
|
Réponds uniquement avec un objet JSON valide.
|
||||||
|
Réponse compacte obligatoire.
|
||||||
|
Ne mets aucun texte avant ou après le JSON.
|
||||||
|
Ne mets aucun bloc Markdown.
|
||||||
|
Limite-toi à 1 résumé, 3 personnages max, 3 lieux max et 5 événements max.
|
||||||
|
|
||||||
|
Chapitre cible: $chapter_slug
|
||||||
|
|
||||||
|
Contexte projet:
|
||||||
|
$story_context
|
||||||
|
|
||||||
|
Chapitre accepté:
|
||||||
|
$accepted_draft
|
||||||
|
|
||||||
|
Format JSON strict:
|
||||||
|
{
|
||||||
|
"summary": "résumé factuel du chapitre",
|
||||||
|
"characters": [
|
||||||
|
{"name": "Nom", "description": "Rôle ou évolution"}
|
||||||
|
],
|
||||||
|
"locations": [
|
||||||
|
{"name": "Lieu", "description": "Ce qui y est établi"}
|
||||||
|
],
|
||||||
|
"timeline_events": [
|
||||||
|
{"event": "Fait important", "order_hint": "optionnel"}
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
Tu es le rôle Réparation prose du moteur AI Novel Engine.
|
||||||
|
Tu dois convertir un brouillon candidat bloqué par le garde-fou en un vrai chapitre lisible.
|
||||||
|
|
||||||
|
Chapitre cible: $chapter_slug
|
||||||
|
Tentative de réparation: $repair_attempt
|
||||||
|
Modèle de réparation: $repair_model
|
||||||
|
|
||||||
|
Intention:
|
||||||
|
$intention
|
||||||
|
|
||||||
|
Structure attendue:
|
||||||
|
$structure_markdown
|
||||||
|
|
||||||
|
Contexte projet:
|
||||||
|
$story_context
|
||||||
|
|
||||||
|
Diagnostic du garde-fou:
|
||||||
|
$gate_json
|
||||||
|
|
||||||
|
Brouillon à réparer:
|
||||||
|
$draft_markdown
|
||||||
|
|
||||||
|
Consignes impératives:
|
||||||
|
- répondre uniquement avec la nouvelle version du chapitre en Markdown
|
||||||
|
- produire uniquement de la prose narrative continue en paragraphes
|
||||||
|
- ne garder aucun titre, aucune puce, aucune numérotation, aucun label de plan visible
|
||||||
|
- supprimer completement les mots `objectif`, `conflit`, `sortie`, `Scène`, `scene`, `Tension` s'ils apparaissent comme labels ou sous-titres
|
||||||
|
- transformer toute structure, note ou checklist en scene(s) jouee(s) avec actions, perceptions et consequences
|
||||||
|
- conserver l'intention et les informations utiles deja presentes
|
||||||
|
- allonger si besoin pour obtenir une scene complete et continue
|
||||||
|
- finir obligatoirement sur une vraie phrase complete avec une ponctuation finale
|
||||||
|
- ne rien ajouter avant ou apres le chapitre
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
Tu es le rôle Réécriture du moteur AI Novel Engine.
|
||||||
|
Tu dois produire une seconde version du chapitre à partir du brouillon et de la critique.
|
||||||
|
|
||||||
|
Chapitre cible: $chapter_slug
|
||||||
|
|
||||||
|
Intention:
|
||||||
|
$intention
|
||||||
|
|
||||||
|
Structure:
|
||||||
|
$structure_markdown
|
||||||
|
|
||||||
|
Brouillon initial:
|
||||||
|
$draft_markdown
|
||||||
|
|
||||||
|
Critique structurée:
|
||||||
|
$critique_json
|
||||||
|
|
||||||
|
Consignes de réécriture:
|
||||||
|
- répondre uniquement avec la version réécrite du chapitre en Markdown
|
||||||
|
- produire uniquement de la prose narrative continue, sous forme de paragraphes
|
||||||
|
- supprimer completement tout titre, toute puce, toute numérotation et tout label de plan
|
||||||
|
- si le brouillon ressemble a une structure ou a des notes, le convertir integralement en scene(s) racontee(s)
|
||||||
|
- ne jamais garder les termes `objectif`, `conflit`, `sortie`, `Tension`, `Scène` comme titres ou labels visibles
|
||||||
|
- conserver l'intention, mais augmenter la continuite dramatique d'une scene a l'autre
|
||||||
|
- materialiser les decisions et leurs consequences
|
||||||
|
- finir sur une vraie phrase complete avec une ponctuation finale
|
||||||
|
- ne rien ajouter avant ou apres le chapitre
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
Tu es le rôle Structure du moteur AI Novel Engine.
|
||||||
|
Tu dois transformer une intention d'auteur en plan de chapitre actionnable.
|
||||||
|
|
||||||
|
Chapitre cible: $chapter_slug
|
||||||
|
|
||||||
|
Intention:
|
||||||
|
$intention
|
||||||
|
|
||||||
|
Contexte projet:
|
||||||
|
$story_context
|
||||||
|
|
||||||
|
Réponds uniquement en Markdown avec cette structure:
|
||||||
|
# Structure — $chapter_slug
|
||||||
|
## Objectif dramatique
|
||||||
|
## Tension
|
||||||
|
## Scènes
|
||||||
|
### Scène 1 — titre
|
||||||
|
- objectif:
|
||||||
|
- conflit:
|
||||||
|
- sortie:
|
||||||
|
|
||||||
|
Ajoute autant de scènes que nécessaire, mais reste concret et opérable pour une génération de chapitre.
|
||||||
Executable
+15
@@ -0,0 +1,15 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
if str(REPO_ROOT) not in sys.path:
|
||||||
|
sys.path.insert(0, str(REPO_ROOT))
|
||||||
|
|
||||||
|
from core.next_lots import main
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
Executable
+296
@@ -0,0 +1,296 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
REPO_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
|
||||||
|
chapter="02"
|
||||||
|
workspace=""
|
||||||
|
base_url="${ANE_BASE_URL:-http://127.0.0.1:8100}"
|
||||||
|
model="${ANE_MODEL:-}"
|
||||||
|
timeout="${ANE_TIMEOUT:-}"
|
||||||
|
decision="approve"
|
||||||
|
intention="Chapitre court. Une femme arrive dans une ville de nuit, trouve un indice simple, et finit sur une decision risquee. Style direct, phrases courtes, ton sobre."
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/smoke_local_generation.sh [options]
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--chapter N Chapter number to generate (default: 02)
|
||||||
|
--workspace DIR Reuse or create the smoke workspace in DIR
|
||||||
|
--intention TEXT Override the default smoke intention
|
||||||
|
--base-url URL OpenAI-compatible base URL (default: ANE_BASE_URL or http://127.0.0.1:8100)
|
||||||
|
--model MODEL Explicit model in provider:model format (required unless ANE_MODEL is already set)
|
||||||
|
--timeout SEC Provider timeout in seconds (default: 900 for apple-coreml, 300 otherwise)
|
||||||
|
--approve Promote without interactive prompt (default)
|
||||||
|
--reject Reject without interactive prompt
|
||||||
|
--no-approve Deprecated alias for --reject
|
||||||
|
-h, --help Show this help
|
||||||
|
|
||||||
|
Environment:
|
||||||
|
ANE_BASE_URL Used if --base-url is not provided
|
||||||
|
ANE_MODEL Used if --model is not provided
|
||||||
|
ANE_TIMEOUT Default: 900 for apple-coreml, 300 otherwise
|
||||||
|
ANE_MAX_TOKENS Default: 192 for apple-coreml, 384 otherwise
|
||||||
|
ANE_MAX_TOKENS_STRUCTURE Default: 96 for apple-coreml, 224 otherwise
|
||||||
|
ANE_MAX_TOKENS_DRAFT Default: 192 for apple-coreml, 384 otherwise
|
||||||
|
ANE_MAX_TOKENS_CRITIQUE Default: 160 for apple-coreml, 512 otherwise
|
||||||
|
ANE_MAX_TOKENS_REWRITE Default: 192 for apple-coreml, 448 otherwise
|
||||||
|
ANE_MAX_TOKENS_GATE Default: 128 for apple-coreml, 384 otherwise
|
||||||
|
ANE_MAX_TOKENS_REPAIR Default: 160 for apple-coreml, 512 otherwise
|
||||||
|
ANE_MAX_TOKENS_MEMORY Default: 128 for apple-coreml, 320 otherwise
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--chapter)
|
||||||
|
chapter="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--workspace)
|
||||||
|
workspace="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--intention)
|
||||||
|
intention="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--base-url)
|
||||||
|
base_url="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--model)
|
||||||
|
model="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--timeout)
|
||||||
|
timeout="${2:-}"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--approve)
|
||||||
|
decision="approve"
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--reject|--no-approve)
|
||||||
|
decision="reject"
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "Unknown option: $1" >&2
|
||||||
|
usage >&2
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ -z "${model}" ]]; then
|
||||||
|
echo "ANE model required. Pass --model provider:model or export ANE_MODEL." >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ -z "${workspace}" ]]; then
|
||||||
|
workspace="$(mktemp -d "${TMPDIR:-/tmp}/ane-local-smoke.XXXXXX")"
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "${workspace}"
|
||||||
|
|
||||||
|
if [[ -z "${timeout}" ]]; then
|
||||||
|
if [[ "${model}" == apple-coreml:* ]]; then
|
||||||
|
timeout="900"
|
||||||
|
else
|
||||||
|
timeout="300"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "${model}" == apple-coreml:* ]]; then
|
||||||
|
export ANE_MAX_TOKENS="${ANE_MAX_TOKENS:-192}"
|
||||||
|
export ANE_MAX_TOKENS_STRUCTURE="${ANE_MAX_TOKENS_STRUCTURE:-96}"
|
||||||
|
export ANE_MAX_TOKENS_DRAFT="${ANE_MAX_TOKENS_DRAFT:-192}"
|
||||||
|
export ANE_MAX_TOKENS_CRITIQUE="${ANE_MAX_TOKENS_CRITIQUE:-160}"
|
||||||
|
export ANE_MAX_TOKENS_REWRITE="${ANE_MAX_TOKENS_REWRITE:-192}"
|
||||||
|
export ANE_MAX_TOKENS_GATE="${ANE_MAX_TOKENS_GATE:-128}"
|
||||||
|
export ANE_MAX_TOKENS_REPAIR="${ANE_MAX_TOKENS_REPAIR:-160}"
|
||||||
|
export ANE_MAX_TOKENS_MEMORY="${ANE_MAX_TOKENS_MEMORY:-128}"
|
||||||
|
else
|
||||||
|
export ANE_MAX_TOKENS="${ANE_MAX_TOKENS:-384}"
|
||||||
|
export ANE_MAX_TOKENS_STRUCTURE="${ANE_MAX_TOKENS_STRUCTURE:-224}"
|
||||||
|
export ANE_MAX_TOKENS_DRAFT="${ANE_MAX_TOKENS_DRAFT:-384}"
|
||||||
|
export ANE_MAX_TOKENS_CRITIQUE="${ANE_MAX_TOKENS_CRITIQUE:-512}"
|
||||||
|
export ANE_MAX_TOKENS_REWRITE="${ANE_MAX_TOKENS_REWRITE:-448}"
|
||||||
|
export ANE_MAX_TOKENS_GATE="${ANE_MAX_TOKENS_GATE:-384}"
|
||||||
|
export ANE_MAX_TOKENS_REPAIR="${ANE_MAX_TOKENS_REPAIR:-512}"
|
||||||
|
export ANE_MAX_TOKENS_MEMORY="${ANE_MAX_TOKENS_MEMORY:-320}"
|
||||||
|
fi
|
||||||
|
|
||||||
|
export ANE_PROVIDER="${ANE_PROVIDER:-openai_compatible}"
|
||||||
|
export ANE_BASE_URL="${base_url}"
|
||||||
|
export ANE_MODEL="${model}"
|
||||||
|
export ANE_TIMEOUT="${timeout}"
|
||||||
|
|
||||||
|
warmup_openai_compatible() {
|
||||||
|
python3 - <<'PY'
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import socket
|
||||||
|
import sys
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
base_url = os.environ["ANE_BASE_URL"].rstrip("/")
|
||||||
|
if base_url.endswith("/v1"):
|
||||||
|
url = f"{base_url}/chat/completions"
|
||||||
|
elif base_url.endswith("/chat/completions"):
|
||||||
|
url = base_url
|
||||||
|
else:
|
||||||
|
url = f"{base_url}/v1/chat/completions"
|
||||||
|
|
||||||
|
payload = {
|
||||||
|
"model": os.environ["ANE_MODEL"],
|
||||||
|
"messages": [{"role": "user", "content": "Respond with exactly: ok"}],
|
||||||
|
"temperature": 0,
|
||||||
|
"max_tokens": 8,
|
||||||
|
}
|
||||||
|
body = json.dumps(payload).encode("utf-8")
|
||||||
|
headers = {"Content-Type": "application/json"}
|
||||||
|
api_key = os.environ.get("ANE_API_KEY", "").strip()
|
||||||
|
if api_key:
|
||||||
|
headers["Authorization"] = f"Bearer {api_key}"
|
||||||
|
|
||||||
|
request = urllib.request.Request(url, data=body, headers=headers, method="POST")
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(request, timeout=float(os.environ.get("ANE_TIMEOUT", "300"))) as response:
|
||||||
|
payload = json.loads(response.read().decode("utf-8"))
|
||||||
|
except (TimeoutError, socket.timeout) as exc:
|
||||||
|
print(f"Warm-up OpenAI-compatible failed: timeout after {os.environ.get('ANE_TIMEOUT', '300')}s", file=sys.stderr)
|
||||||
|
raise SystemExit(1) from exc
|
||||||
|
|
||||||
|
content = payload["choices"][0]["message"]["content"]
|
||||||
|
print(f"Warm-up OpenAI-compatible OK: {content}")
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
prepare_workspace() {
|
||||||
|
python3 - <<'PY'
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
import os
|
||||||
|
|
||||||
|
from core.chapters import ChapterId
|
||||||
|
|
||||||
|
root = Path(os.environ["ANE_SMOKE_ROOT"])
|
||||||
|
chapter_id = ChapterId.parse(os.environ["ANE_SMOKE_CHAPTER"])
|
||||||
|
intention = os.environ["ANE_SMOKE_INTENTION"].strip()
|
||||||
|
|
||||||
|
intentions_dir = root / "notes" / "intentions"
|
||||||
|
intentions_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
intention_path = intentions_dir / chapter_id.filename
|
||||||
|
if not intention_path.exists():
|
||||||
|
intention_path.write_text(
|
||||||
|
f"# Intention — Chapitre {chapter_id.label}\n\n{intention}\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
print_summary() {
|
||||||
|
python3 - <<'PY'
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from core.chapters import ChapterId
|
||||||
|
|
||||||
|
root = Path(os.environ["ANE_SMOKE_ROOT"])
|
||||||
|
chapter_id = ChapterId.parse(os.environ["ANE_SMOKE_CHAPTER"])
|
||||||
|
meta_path = root / "brouillons" / "chapitres" / chapter_id.slug / "meta.json"
|
||||||
|
|
||||||
|
print("")
|
||||||
|
print("Smoke summary")
|
||||||
|
print(f"- workspace: {root}")
|
||||||
|
print(f"- model: {os.environ['ANE_MODEL']}")
|
||||||
|
print(f"- chapter: {chapter_id.slug}")
|
||||||
|
|
||||||
|
if not meta_path.exists():
|
||||||
|
print("- meta: absent")
|
||||||
|
sys.exit(0)
|
||||||
|
|
||||||
|
meta = json.loads(meta_path.read_text(encoding="utf-8"))
|
||||||
|
print(f"- status: {meta.get('status')}")
|
||||||
|
print(f"- accepted: {meta.get('accepted')}")
|
||||||
|
if meta.get("failed_stage"):
|
||||||
|
print(f"- failed_stage: {meta.get('failed_stage')}")
|
||||||
|
quality_blockers = meta.get("quality_blockers") or []
|
||||||
|
if quality_blockers:
|
||||||
|
print(f"- quality_blockers: {', '.join(str(item) for item in quality_blockers)}")
|
||||||
|
retry_stages = meta.get("retry_stages") or []
|
||||||
|
if retry_stages:
|
||||||
|
print(f"- retry_stages: {', '.join(str(item) for item in retry_stages)}")
|
||||||
|
print(f"- repair_attempts: {meta.get('repair_attempts', 0)}")
|
||||||
|
repair_models = meta.get("repair_models") or []
|
||||||
|
if repair_models:
|
||||||
|
print(f"- repair_models: {', '.join(str(item) for item in repair_models)}")
|
||||||
|
print(f"- last_status_message: {meta.get('last_status_message', '')}")
|
||||||
|
artifacts = meta.get("artifacts", {})
|
||||||
|
if isinstance(artifacts, dict):
|
||||||
|
draft_path = artifacts.get("repair_latest") or artifacts.get("draft_v2")
|
||||||
|
if draft_path:
|
||||||
|
print(f"- draft_path: {draft_path}")
|
||||||
|
for label, key in (
|
||||||
|
("draft_v2", "draft_v2"),
|
||||||
|
("critique_v1", "critique_v1"),
|
||||||
|
("repair_latest", "repair_latest"),
|
||||||
|
("gate_v1", "gate_v1"),
|
||||||
|
("manuscript", "manuscript"),
|
||||||
|
("memory_summary", "memory_summary"),
|
||||||
|
):
|
||||||
|
value = artifacts.get(key)
|
||||||
|
if value:
|
||||||
|
print(f"- {label}: {value}")
|
||||||
|
print(f"- meta: {meta_path}")
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
export ANE_SMOKE_ROOT="${workspace}"
|
||||||
|
export ANE_SMOKE_CHAPTER="${chapter}"
|
||||||
|
export ANE_SMOKE_INTENTION="${intention}"
|
||||||
|
|
||||||
|
cd "${REPO_DIR}"
|
||||||
|
prepare_workspace
|
||||||
|
|
||||||
|
if [[ "${ANE_MODEL}" == apple-coreml:* ]]; then
|
||||||
|
echo "Warm-up Apple runtime via ${ANE_BASE_URL} ..."
|
||||||
|
warmup_openai_compatible
|
||||||
|
fi
|
||||||
|
|
||||||
|
decision_flag="--approve"
|
||||||
|
if [[ "${decision}" == "reject" ]]; then
|
||||||
|
decision_flag="--reject"
|
||||||
|
fi
|
||||||
|
|
||||||
|
set +e
|
||||||
|
cli_output="$(
|
||||||
|
cd "${workspace}" && \
|
||||||
|
PYTHONPATH="${REPO_DIR}${PYTHONPATH:+:${PYTHONPATH}}" \
|
||||||
|
python3 -m cli.main generate chapter --chapter "${chapter}" "${decision_flag}" 2>&1
|
||||||
|
)"
|
||||||
|
cli_exit=$?
|
||||||
|
set -e
|
||||||
|
|
||||||
|
printf '%s\n' "${cli_output}"
|
||||||
|
print_summary
|
||||||
|
|
||||||
|
if [[ ${cli_exit} -ne 0 ]]; then
|
||||||
|
exit "${cli_exit}"
|
||||||
|
fi
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,300 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
from core.chapters import ChapterId
|
||||||
|
from core.next_lots import (
|
||||||
|
AUTO_SYNC_TODO_ACTIVE,
|
||||||
|
CommandResult,
|
||||||
|
Manifest,
|
||||||
|
ModelRunResult,
|
||||||
|
NextLotsRunner,
|
||||||
|
RunState,
|
||||||
|
replace_auto_section,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class NextLotsTests(unittest.TestCase):
|
||||||
|
def setUp(self) -> None:
|
||||||
|
self.temp_dir = tempfile.TemporaryDirectory()
|
||||||
|
self.root = Path(self.temp_dir.name) / "ane"
|
||||||
|
self.root.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.mascarade = Path(self.temp_dir.name) / "mascarade"
|
||||||
|
self.mascarade.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
for path in (
|
||||||
|
self.root / "README.md",
|
||||||
|
self.root / "TODO_ACTIVE.md",
|
||||||
|
self.root / "TODO_IMPLEMENTE.md",
|
||||||
|
self.root / "docs" / "EXECUTION_PLAN_2026-03-08.md",
|
||||||
|
self.root / "docs" / "MODEL_COMPARISON_2026-03-08.md",
|
||||||
|
self.root / "docs" / "runbooks" / "LOCAL_GENERATION.md",
|
||||||
|
self.mascarade / "README.md",
|
||||||
|
self.mascarade / "TODO_AI_NOVEL_ENGINE.md",
|
||||||
|
self.mascarade / "docs" / "EXECUTION_PLAN_2026-03-08.md",
|
||||||
|
self.mascarade / "docs" / "RUNBOOK_APPLE_LLM_LOCAL.md",
|
||||||
|
):
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(f"# {path.name}\n", encoding="utf-8")
|
||||||
|
|
||||||
|
manifest_dir = self.root / "automation"
|
||||||
|
manifest_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.manifest_path = manifest_dir / "next_lots.toml"
|
||||||
|
self.manifest_path.write_text(
|
||||||
|
(
|
||||||
|
"[paths]\n"
|
||||||
|
f"mascarade_repo = \"{self.mascarade}\"\n"
|
||||||
|
"core_base_url = \"http://127.0.0.1:8100\"\n"
|
||||||
|
"apple_runtime_url = \"http://127.0.0.1:8201\"\n"
|
||||||
|
"ollama_tags_url = \"http://127.0.0.1:11435/api/tags\"\n\n"
|
||||||
|
"apple_model_ready_timeout_seconds = 0\n"
|
||||||
|
"apple_model_poll_interval_seconds = 0.01\n\n"
|
||||||
|
"[smoke]\n"
|
||||||
|
"chapter = \"02\"\n"
|
||||||
|
"intention = \"Smoke intention\"\n"
|
||||||
|
"timeout_seconds = 300\n\n"
|
||||||
|
"[preset]\n"
|
||||||
|
"ANE_MAX_TOKENS_STRUCTURE = \"256\"\n"
|
||||||
|
"ANE_REPAIR_MAX_PASSES = \"2\"\n\n"
|
||||||
|
"[ensure_models]\n"
|
||||||
|
"apple_models = [\"qwen2.5-0.5b-instruct-onnx\", \"qwen3.5-4b-onnx-q4f16\", \"stateful-mistral7b-instruct-int4-coreml\"]\n"
|
||||||
|
"ollama_models = [\"qwen2.5:7b\", \"qwen2.5:1.5b\"]\n\n"
|
||||||
|
"[lots.priority_models]\n"
|
||||||
|
"models = [\"apple-coreml:qwen3.5-4b-onnx-q4f16\", \"ollama:qwen2.5:7b\"]\n\n"
|
||||||
|
"[lots.baselines]\n"
|
||||||
|
"models = [\"apple-coreml:qwen2.5-0.5b-instruct-onnx\", \"ollama:qwen2.5:1.5b\"]\n\n"
|
||||||
|
"[lots.preflight_only]\n"
|
||||||
|
"models = [\"apple-coreml:stateful-mistral7b-instruct-int4-coreml\"]\n\n"
|
||||||
|
"[tracking.ane]\n"
|
||||||
|
"todo_active = \"TODO_ACTIVE.md\"\n"
|
||||||
|
"todo_done = \"TODO_IMPLEMENTE.md\"\n"
|
||||||
|
"plan = \"docs/EXECUTION_PLAN_2026-03-08.md\"\n"
|
||||||
|
"comparison = \"docs/MODEL_COMPARISON_2026-03-08.md\"\n"
|
||||||
|
"readme = \"README.md\"\n"
|
||||||
|
"runbook = \"docs/runbooks/LOCAL_GENERATION.md\"\n\n"
|
||||||
|
"[tracking.mascarade]\n"
|
||||||
|
"todo = \"TODO_AI_NOVEL_ENGINE.md\"\n"
|
||||||
|
"plan = \"docs/EXECUTION_PLAN_2026-03-08.md\"\n"
|
||||||
|
"readme = \"README.md\"\n"
|
||||||
|
"runbook = \"docs/RUNBOOK_APPLE_LLM_LOCAL.md\"\n\n"
|
||||||
|
"[next_actions]\n"
|
||||||
|
"rewrite_compaction = \"Compacter rewrite\"\n"
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
def tearDown(self) -> None:
|
||||||
|
self.temp_dir.cleanup()
|
||||||
|
|
||||||
|
def test_manifest_loads_tracking_and_models(self) -> None:
|
||||||
|
manifest = Manifest.load(self.root, self.manifest_path)
|
||||||
|
|
||||||
|
self.assertEqual(manifest.priority_models, ["apple-coreml:qwen3.5-4b-onnx-q4f16", "ollama:qwen2.5:7b"])
|
||||||
|
self.assertEqual(manifest.required_apple_models[0], "qwen2.5-0.5b-instruct-onnx")
|
||||||
|
self.assertEqual(manifest.tracking.mascarade_repo, self.mascarade)
|
||||||
|
self.assertEqual(manifest.tracking.ane_todo_active, self.root / "TODO_ACTIVE.md")
|
||||||
|
self.assertEqual(manifest.apple_model_ready_timeout_seconds, 0)
|
||||||
|
|
||||||
|
def test_replace_auto_section_only_replaces_managed_block(self) -> None:
|
||||||
|
path = self.root / "TODO_ACTIVE.md"
|
||||||
|
path.write_text(
|
||||||
|
"# Manual\n\n"
|
||||||
|
"Avant.\n\n"
|
||||||
|
"## Auto-sync\n"
|
||||||
|
"<!-- AUTO-SYNC:ANE-TODO-ACTIVE:START -->\n"
|
||||||
|
"ancien\n"
|
||||||
|
"<!-- AUTO-SYNC:ANE-TODO-ACTIVE:END -->\n\n"
|
||||||
|
"Apres.\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
replace_auto_section(path, AUTO_SYNC_TODO_ACTIVE, "## Auto-sync", "- nouveau")
|
||||||
|
rendered = path.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
self.assertIn("Avant.", rendered)
|
||||||
|
self.assertIn("Apres.", rendered)
|
||||||
|
self.assertIn("- nouveau", rendered)
|
||||||
|
self.assertNotIn("ancien", rendered)
|
||||||
|
|
||||||
|
def test_runner_creates_checkpoint_when_apple_model_differs(self) -> None:
|
||||||
|
manifest = Manifest.load(self.root, self.manifest_path)
|
||||||
|
prepare_calls: list[list[str]] = []
|
||||||
|
|
||||||
|
def command_runner(args: list[str], cwd: Path, env=None) -> CommandResult:
|
||||||
|
if "prepare_runtime_step.sh" in " ".join(args):
|
||||||
|
prepare_calls.append(args)
|
||||||
|
return CommandResult(args=args, returncode=0, stdout="prepared", stderr="", duration_seconds=0.1)
|
||||||
|
|
||||||
|
def json_fetcher(url: str, timeout: float):
|
||||||
|
if url.endswith("/health"):
|
||||||
|
return {"status": "ok"}
|
||||||
|
if url.endswith("/models"):
|
||||||
|
return ["qwen2.5-0.5b-instruct-onnx"]
|
||||||
|
raise AssertionError(url)
|
||||||
|
|
||||||
|
runner = NextLotsRunner(manifest, command_runner=command_runner, json_fetcher=json_fetcher)
|
||||||
|
exit_code = runner.run(lot="priority_models")
|
||||||
|
|
||||||
|
self.assertEqual(exit_code, 3)
|
||||||
|
self.assertEqual(len(prepare_calls), 1)
|
||||||
|
self.assertIn("--apple-model", prepare_calls[0])
|
||||||
|
state = RunState.load(self.root / "automation" / "state" / "next_lots_state.json")
|
||||||
|
self.assertIsNotNone(state.pending_manual_action)
|
||||||
|
self.assertIn("qwen3.5-4b-onnx-q4f16", state.pending_manual_action["reason"])
|
||||||
|
|
||||||
|
def test_runner_waits_for_apple_model_before_checkpointing(self) -> None:
|
||||||
|
manifest = Manifest.load(self.root, self.manifest_path)
|
||||||
|
manifest = Manifest(
|
||||||
|
**{
|
||||||
|
**manifest.__dict__,
|
||||||
|
"apple_model_ready_timeout_seconds": 0.05,
|
||||||
|
"apple_model_poll_interval_seconds": 0.0,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
prepare_calls: list[list[str]] = []
|
||||||
|
model_calls = {"count": 0}
|
||||||
|
|
||||||
|
def command_runner(args: list[str], cwd: Path, env=None) -> CommandResult:
|
||||||
|
if "prepare_runtime_step.sh" in " ".join(args):
|
||||||
|
prepare_calls.append(args)
|
||||||
|
return CommandResult(args=args, returncode=0, stdout="prepared", stderr="", duration_seconds=0.1)
|
||||||
|
|
||||||
|
def json_fetcher(url: str, timeout: float):
|
||||||
|
if url.endswith("/health"):
|
||||||
|
return {"status": "ok"}
|
||||||
|
if url.endswith("/models"):
|
||||||
|
model_calls["count"] += 1
|
||||||
|
if model_calls["count"] == 1:
|
||||||
|
return {"models": []}
|
||||||
|
return ["qwen3.5-4b-onnx-q4f16"]
|
||||||
|
raise AssertionError(url)
|
||||||
|
|
||||||
|
runner = NextLotsRunner(manifest, command_runner=command_runner, json_fetcher=json_fetcher)
|
||||||
|
checkpoint = runner._checkpoint_if_runtime_manual_step_needed(
|
||||||
|
RunState.new(
|
||||||
|
manifest,
|
||||||
|
lot="priority_models",
|
||||||
|
report_dir=self.root / "automation" / "reports" / "sync",
|
||||||
|
state_path=self.root / "automation" / "state" / "next_lots_state.json",
|
||||||
|
steps=[{"type": "models", "name": "priority_models", "models": manifest.priority_models, "preflight_only": False}],
|
||||||
|
),
|
||||||
|
"apple-coreml:qwen3.5-4b-onnx-q4f16",
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertIsNone(checkpoint)
|
||||||
|
self.assertEqual(prepare_calls, [])
|
||||||
|
self.assertGreaterEqual(model_calls["count"], 2)
|
||||||
|
|
||||||
|
def test_run_model_classifies_accepted_from_meta(self) -> None:
|
||||||
|
manifest = Manifest.load(self.root, self.manifest_path)
|
||||||
|
chapter = ChapterId.parse("02")
|
||||||
|
|
||||||
|
def command_runner(args: list[str], cwd: Path, env=None) -> CommandResult:
|
||||||
|
if "smoke_openai_compat_ane.sh" in " ".join(args):
|
||||||
|
return CommandResult(args=args, returncode=0, stdout="ok", stderr="", duration_seconds=0.2)
|
||||||
|
if "smoke_local_generation.sh" in " ".join(args):
|
||||||
|
workspace = Path(args[args.index("--workspace") + 1])
|
||||||
|
meta_path = workspace / "brouillons" / "chapitres" / chapter.slug / "meta.json"
|
||||||
|
meta_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
meta_path.write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"status": "accepted",
|
||||||
|
"accepted": True,
|
||||||
|
"completed_stages": ["structure", "draft", "critique", "rewrite", "gate", "memory"],
|
||||||
|
"retry_stages": ["gate"],
|
||||||
|
"repair_attempts": 1,
|
||||||
|
"repair_models": ["ollama:qwen2.5:7b"],
|
||||||
|
"artifacts": {
|
||||||
|
"repair_latest": str(meta_path.parent / "repair_v1.md"),
|
||||||
|
"gate_v1": str(meta_path.parent / "gate_v1.json"),
|
||||||
|
"manuscript": str(workspace / "manuscrit" / chapter.filename),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
ensure_ascii=False,
|
||||||
|
indent=2,
|
||||||
|
)
|
||||||
|
+ "\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
return CommandResult(args=args, returncode=0, stdout="smoke ok", stderr="", duration_seconds=1.5)
|
||||||
|
raise AssertionError(args)
|
||||||
|
|
||||||
|
runner = NextLotsRunner(
|
||||||
|
manifest,
|
||||||
|
command_runner=command_runner,
|
||||||
|
json_fetcher=lambda url, timeout: {"status": "ok"} if url.endswith("/health") else ["qwen3.5-4b-onnx-q4f16"],
|
||||||
|
)
|
||||||
|
report_dir = self.root / "automation" / "reports" / "test"
|
||||||
|
report_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
result = runner._run_model("ollama:qwen2.5:7b", category="priority_models", preflight_only=False, report_dir=report_dir)
|
||||||
|
|
||||||
|
self.assertEqual(result.classification, "accepted")
|
||||||
|
self.assertEqual(result.repair_attempts, 1)
|
||||||
|
self.assertIn("gate", result.completed_stages)
|
||||||
|
|
||||||
|
def test_tracking_sync_updates_docs_with_auto_sync_sections(self) -> None:
|
||||||
|
manifest = Manifest.load(self.root, self.manifest_path)
|
||||||
|
runner = NextLotsRunner(
|
||||||
|
manifest,
|
||||||
|
command_runner=lambda args, cwd, env=None: CommandResult(args=args, returncode=0, stdout="", stderr="", duration_seconds=0.0),
|
||||||
|
json_fetcher=lambda url, timeout: {"status": "ok"},
|
||||||
|
)
|
||||||
|
state = RunState.new(
|
||||||
|
manifest,
|
||||||
|
lot="tracking_sync",
|
||||||
|
report_dir=self.root / "automation" / "reports" / "sync",
|
||||||
|
state_path=self.root / "automation" / "state" / "next_lots_state.json",
|
||||||
|
steps=[{"type": "tracking_sync"}],
|
||||||
|
)
|
||||||
|
state.results = [
|
||||||
|
asdict(
|
||||||
|
ModelRunResult(
|
||||||
|
model="ollama:qwen2.5:7b",
|
||||||
|
category="priority_models",
|
||||||
|
classification="provider_failed",
|
||||||
|
preflight_ok=True,
|
||||||
|
smoke_attempted=True,
|
||||||
|
status="failed",
|
||||||
|
failed_stage="rewrite",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
runner._sync_tracking(state, dry_run=False)
|
||||||
|
|
||||||
|
self.assertIn("AUTO-SYNC:ANE-TODO-ACTIVE:START", (self.root / "TODO_ACTIVE.md").read_text(encoding="utf-8"))
|
||||||
|
self.assertIn("Compacter rewrite", (self.root / "docs" / "EXECUTION_PLAN_2026-03-08.md").read_text(encoding="utf-8"))
|
||||||
|
self.assertIn("ollama:qwen2.5:7b", (self.root / "docs" / "MODEL_COMPARISON_2026-03-08.md").read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def asdict(result: ModelRunResult) -> dict[str, object]:
|
||||||
|
return {
|
||||||
|
"model": result.model,
|
||||||
|
"category": result.category,
|
||||||
|
"classification": result.classification,
|
||||||
|
"preflight_ok": result.preflight_ok,
|
||||||
|
"preflight_duration_seconds": result.preflight_duration_seconds,
|
||||||
|
"smoke_attempted": result.smoke_attempted,
|
||||||
|
"smoke_duration_seconds": result.smoke_duration_seconds,
|
||||||
|
"status": result.status,
|
||||||
|
"accepted": result.accepted,
|
||||||
|
"failed_stage": result.failed_stage,
|
||||||
|
"quality_blockers": result.quality_blockers,
|
||||||
|
"retry_stages": result.retry_stages,
|
||||||
|
"repair_attempts": result.repair_attempts,
|
||||||
|
"repair_models": result.repair_models,
|
||||||
|
"draft_path": result.draft_path,
|
||||||
|
"gate_path": result.gate_path,
|
||||||
|
"meta_path": result.meta_path,
|
||||||
|
"manuscript_path": result.manuscript_path,
|
||||||
|
"notes": result.notes,
|
||||||
|
"preflight_log": result.preflight_log,
|
||||||
|
"smoke_log": result.smoke_log,
|
||||||
|
"workspace": result.workspace,
|
||||||
|
"apple_model_active": result.apple_model_active,
|
||||||
|
"completed_stages": result.completed_stages,
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user