Implement local generation workflow and automation
This commit is contained in:
L'électron rare
@@ -0,0 +1,4 @@
|
||||
__pycache__/
|
||||
*.pyc
|
||||
automation/reports/
|
||||
automation/state/
|
||||
@@ -16,3 +16,100 @@ sans perte de cohérence, de mémoire ou de contrôle.
|
||||
|
||||
## Statut
|
||||
v2 — développement en cours (open-source)
|
||||
|
||||
## Suivi
|
||||
- backlog actif: [`TODO_ACTIVE.md`](./TODO_ACTIVE.md)
|
||||
- etat livre: [`TODO_IMPLEMENTE.md`](./TODO_IMPLEMENTE.md)
|
||||
- ordre d'execution recommande: [`docs/EXECUTION_PLAN_2026-03-08.md`](./docs/EXECUTION_PLAN_2026-03-08.md)
|
||||
- runbook local: [`docs/runbooks/LOCAL_GENERATION.md`](./docs/runbooks/LOCAL_GENERATION.md)
|
||||
- comparatif modeles local: [`docs/MODEL_COMPARISON_2026-03-08.md`](./docs/MODEL_COMPARISON_2026-03-08.md)
|
||||
|
||||
## Automation des lots utiles
|
||||
|
||||
Le driver principal des prochains lots utiles est maintenant:
|
||||
|
||||
```bash
|
||||
python3 scripts/run_next_lots.py --lot full
|
||||
```
|
||||
|
||||
Points clés:
|
||||
- le manifeste versionné est `automation/next_lots.toml`
|
||||
- le driver réutilise les smokes existants au lieu de dupliquer le pipeline
|
||||
- les opérations sensibles restent semi-autos: en cas de switch Apple ou de restart runtime, le cycle prépare les commandes exactes puis s'arrête avec un état de reprise
|
||||
- reprise:
|
||||
|
||||
```bash
|
||||
python3 scripts/run_next_lots.py --resume automation/state/next_lots_state.json
|
||||
```
|
||||
|
||||
- synchronisation seule des plans/TODOs/readmes à partir du dernier état:
|
||||
|
||||
```bash
|
||||
python3 scripts/run_next_lots.py --lot tracking_sync --report-only
|
||||
```
|
||||
|
||||
## Generation locale via Mascarade
|
||||
|
||||
`ai-novel-engine` parle un provider OpenAI-compatible. Pour utiliser la
|
||||
generation locale via `mascarade`, pointer simplement le moteur narratif vers le
|
||||
core Python sur `:8100`.
|
||||
|
||||
```bash
|
||||
export ANE_PROVIDER=openai_compatible
|
||||
export ANE_BASE_URL=http://127.0.0.1:8100
|
||||
export ANE_MODEL=<provider:model>
|
||||
export ANE_MAX_TOKENS=512
|
||||
export ANE_MAX_TOKENS_STRUCTURE=256
|
||||
export ANE_MAX_TOKENS_DRAFT=512
|
||||
export ANE_MAX_TOKENS_CRITIQUE=384
|
||||
export ANE_MAX_TOKENS_REWRITE=512
|
||||
export ANE_MAX_TOKENS_GATE=320
|
||||
export ANE_MAX_TOKENS_REPAIR=384
|
||||
export ANE_MAX_TOKENS_MEMORY=256
|
||||
export ANE_REPAIR_MAX_PASSES=2
|
||||
# optionnel si tu veux forcer un fallback explicite pour la reparation
|
||||
# export ANE_REPAIR_FALLBACK_MODEL=ollama:qwen2.5:7b
|
||||
|
||||
# seulement si MASCARADE_API_KEY est active
|
||||
export ANE_API_KEY=ton-token-mascarade
|
||||
|
||||
python3 -m cli.main generate chapter --chapter 01
|
||||
```
|
||||
|
||||
Notes :
|
||||
- `ANE_MODEL` est requis; le repo n'impose pas de modele par défaut
|
||||
- `ANE_MODEL` selectionne le backend local par prefixe `apple-coreml:` ou `ollama:`
|
||||
- `ANE_MAX_TOKENS` reste le plafond global par défaut
|
||||
- les overrides `ANE_MAX_TOKENS_STRUCTURE`, `..._DRAFT`, `..._CRITIQUE`, `..._REWRITE`, `..._GATE`, `..._REPAIR`, `..._MEMORY` permettent d'ajuster chaque étape
|
||||
- `ANE_REPAIR_MAX_PASSES` borne la boucle `repair`
|
||||
- `ANE_REPAIR_FALLBACK_MODEL` permet de forcer le modele du second passage `repair`
|
||||
- le pipeline narratif reste entierement dans `ai-novel-engine`
|
||||
- `mascarade` sert uniquement de runtime local et de couche OpenAI-compatible
|
||||
- dernier cycle complet termine au 9 mars 2026 :
|
||||
- `apple-coreml:qwen3.5-4b-onnx-q4f16` est `accepted` de bout en bout sous garde-fou
|
||||
- `ollama:qwen2.5:7b` atteint `gate`, exerce `repair` en live, puis finit `quality_blocked` sur `outline_like`
|
||||
- `apple-coreml:stateful-mistral7b-instruct-int4-coreml` reste `preflight_only`
|
||||
- les baselines `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b` sont en rerun automatise separe; ils ne sont plus la reference locale courante
|
||||
- le smoke et `status` exposent maintenant `gate_v1.json`, `quality_blockers`, `failed_stage`, `repair_attempts` et `repair_models`
|
||||
- le runtime Apple local ne sert qu'un `model_id` a la fois; un fallback `repair` vers un autre modele Apple exige donc un switch de service entre runs
|
||||
|
||||
Smoke test local rapide :
|
||||
|
||||
```bash
|
||||
./scripts/smoke_local_generation.sh \
|
||||
--base-url http://127.0.0.1:8100 \
|
||||
--model "ollama:qwen2.5:1.5b" \
|
||||
--approve
|
||||
```
|
||||
|
||||
Le script cree un workspace temporaire, ecrit une intention de test, lance la vraie CLI publique, fait un warm-up automatique pour `apple-coreml`, puis affiche un resume humain des artefacts et du `meta.json`. En mode `apple-coreml`, il applique par defaut un timeout plus large (`ANE_TIMEOUT=900`) et des budgets de smoke plus courts pour eviter de faire exploser la latence locale. Pour les reruns qualitatifs de reference, fixer explicitement `--timeout 300` et des budgets `ANE_MAX_TOKENS_*` communs. Utiliser `--workspace`, `--chapter`, `--intention`, `--timeout`, `--approve` ou `--reject` si besoin.
|
||||
|
||||
## Etat auto-synchronise
|
||||
## Etat auto-synchronise
|
||||
<!-- AUTO-SYNC:ANE-README:START -->
|
||||
- dernier cycle automatise: 2026-03-09T06:53:02+00:00
|
||||
- reference locale actuelle: aucun accepted, meilleur diagnostic: apple-coreml:qwen2.5-0.5b-instruct-onnx
|
||||
- prochain lot utile: Analyser les runs ayant atteint gate/repair puis resserrer la reference locale autour des meilleurs candidats.
|
||||
- lancer un cycle: `python3 scripts/run_next_lots.py --lot full`
|
||||
- checkpoint manuel en attente: Le runtime Apple sert `qwen2.5-0.5b-instruct-onnx` au lieu de `stateful-mistral7b-instruct-int4-coreml`.
|
||||
<!-- AUTO-SYNC:ANE-README:END -->
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
# TODO actif - AI Novel Engine
|
||||
|
||||
Source de verite des taches restantes pour `ai-novel-engine`.
|
||||
|
||||
Regle:
|
||||
- cocher ici ce qui est fait puis deplacer le lot livre vers `TODO_IMPLEMENTE.md`
|
||||
- ne suivre ici que le travail restant ou les blocages encore ouverts
|
||||
- garder les dependances `mascarade` explicites
|
||||
|
||||
## Deja implemente
|
||||
- [x] P0 Pipeline chapitre `intention -> structure -> draft -> critique -> rewrite -> validation -> memoire`
|
||||
- [x] P0 Normalisation de chapitre avec identifiant canonique `chapitre_XX` et detection des collisions legacy
|
||||
- [x] P0 Provider OpenAI-compatible et branchement local via `mascarade`
|
||||
- [x] P0 Budgets par etape (`ANE_MAX_TOKENS_*`)
|
||||
- [x] P0 Parsing JSON tolerant pour les sorties locales imparfaites
|
||||
- [x] P0 Second passage de reessai pour `critique` et `memory` si le JSON reste invalide apres parsing tolerant
|
||||
- [x] P0 Garde-fou manuscrit dur avant promotion (`gate_v1.json`, heuristiques locales, verdict `quality_blocked`)
|
||||
- [x] P0 Boucle `repair` automatique entre `gate` et `quality_blocked`, avec artefacts `repair_vN.md`, fallback modele et preservation de `draft_v2.md`
|
||||
- [x] P0 Smoke script local `scripts/smoke_local_generation.sh`
|
||||
- [x] P0 Timeout provider remonte maintenant en `ProviderError` et marque correctement `failed_stage` dans `meta.json`
|
||||
- [x] P0 Warm-up Apple du smoke remonte maintenant une erreur lisible au lieu d'une stacktrace brute
|
||||
- [x] P0 Prompts `draft_v1` et `rewrite_v1` durcis pour imposer une prose continue sans titres ni puces
|
||||
- [x] P1 Flags CLI non interactifs `--approve` et `--reject`
|
||||
- [x] P1 `status` enrichi avec les chapitres en echec, en attente et bloques par garde-fou
|
||||
- [x] P1 Resume de smoke humain a partir du `meta.json`
|
||||
- [x] P1 Contrat cross-repo et recovery documentes via les runbooks
|
||||
- [x] P2 `docs/vision.md`, `docs/roadmap.md` et le runbook local ne sont plus des placeholders
|
||||
- [x] P0 Revalidation sous garde-fou de `ollama:qwen2.5:1.5b` -> `quality_blocked`
|
||||
- [x] P0 Revalidation sous garde-fou de `apple-coreml:qwen2.5-0.5b-instruct-onnx` -> `quality_blocked`
|
||||
- [x] P0 Revalidation sous garde-fou de `apple-coreml:qwen3.5-4b-onnx-q4f16` -> `provider_failed` en `rewrite`
|
||||
- [x] P0 Revalidation sous garde-fou de `ollama:qwen2.5:7b` -> `provider_failed` par timeout en `draft`
|
||||
- [x] P0 Comparatif local re-ecrit pour le protocole avec garde-fou dans `docs/MODEL_COMPARISON_2026-03-08.md`
|
||||
- [x] P0 Revalidation reelle sous protocole `gate + repair` borne a `300s` par requete:
|
||||
- `ollama:qwen2.5:1.5b` -> `failed` en `structure`
|
||||
- `apple-coreml:qwen2.5-0.5b-instruct-onnx` -> `failed` en `rewrite`
|
||||
- `apple-coreml:qwen3.5-4b-onnx-q4f16` -> `failed` en `rewrite`
|
||||
- `ollama:qwen2.5:7b` -> `failed` en `rewrite`
|
||||
|
||||
## Actif
|
||||
- [ ] P0 Terminer le lot `baselines`, puis relancer `tracking_sync` sur un etat complet incluant `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b`
|
||||
- [ ] P0 Faire passer `ollama:qwen2.5:7b` de `quality_blocked` a `accepted` en supprimant le residu `outline_like` apres `repair`
|
||||
- [ ] P0 Faire terminer au moins un cycle `python3 scripts/run_next_lots.py --lot full` jusqu'a `tracking_sync` sans checkpoint manuel autre qu'un switch Apple explicite
|
||||
- [ ] P1 Requalifier `apple-coreml:qwen3.5-4b-onnx-q4f16` comme reference Apple stable sur plusieurs cycles, pas sur un seul run accepte
|
||||
- [ ] P1 Rendre la strategie de fallback `repair` consciente des modeles reellement servis: le runtime Apple n'expose qu'un `model_id` a la fois
|
||||
- [ ] P1 Garder l'installation/staging Apple de `qwen2.5-0.5b-instruct-onnx`, `qwen3.5-4b-onnx-q4f16` et `stateful-mistral7b-instruct-int4-coreml` comme prerequis explicite
|
||||
|
||||
## Bloque
|
||||
- [ ] P1 `ollama:qwen2.5:7b` atteint maintenant `gate` et exerce `repair`, mais reste bloque sur `outline_like` apres deux passes
|
||||
- [ ] P1 Le lot `baselines` exige encore un switch Apple explicite avant de pouvoir finir `apple-coreml:qwen2.5-0.5b-instruct-onnx`
|
||||
- [ ] P1 `ollama:qwen2.5:1.5b` reste lent et reste a requalifier une fois `baselines` repris jusqu'au bout
|
||||
- [ ] P1 `apple-coreml:stateful-mistral7b-instruct-int4-coreml` reste preflight-only sur cette machine: preflight froid `:8100` OK en 128 s, preflight chaud OK en 63 s, mais le smoke ANE est reste bloque a `structure` pendant plus de 8 minutes avec les budgets de smoke
|
||||
- [ ] P1 Le host `ollama` natif 0.17.7 sur cette machine echoue sur `qwen2.5:1.5b` avec un crash Metal; la validation ANE reelle passe par un service Docker CPU expose sur `127.0.0.1:11435` et route via `mascarade`
|
||||
- [ ] P1 Le runtime Apple local n'expose qu'un seul `model_id` a la fois; un fallback `repair` vers un autre modele Apple exige donc un switch de service entre deux runs, pas au milieu d'un smoke
|
||||
|
||||
## Prochain ordre
|
||||
- [ ] P0 Finir `python3 scripts/run_next_lots.py --lot baselines`, puis laisser `tracking_sync` recalculer les verdicts complets
|
||||
- [ ] P0 Tuner `rewrite_v1` et la passe `repair` pour eliminer `outline_like` sur `ollama:qwen2.5:7b`
|
||||
- [ ] P1 Rejouer ensuite `python3 scripts/run_next_lots.py --lot priority_models` pour verifier la stabilite de `apple-coreml:qwen3.5-4b-onnx-q4f16` et le sort de `ollama:qwen2.5:7b`
|
||||
- [ ] P1 Garder `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b` comme baselines vitesse ou regressions tant qu'ils n'ont pas de verdict comparable au protocole courant
|
||||
- [ ] P1 Verifier avant tout rerun Apple que `qwen2.5-0.5b-instruct-onnx`, `qwen3.5-4b-onnx-q4f16` et `stateful-mistral7b-instruct-int4-coreml` sont bien installes/stages et que le bon `model_id` est charge sur `:8201`
|
||||
|
||||
## Auto-sync
|
||||
## Auto-sync
|
||||
<!-- AUTO-SYNC:ANE-TODO-ACTIVE:START -->
|
||||
- dernier cycle automatique: 2026-03-09T06:53:02+00:00
|
||||
- modeles accepted: aucun
|
||||
- modeles ayant atteint gate: apple-coreml:qwen2.5-0.5b-instruct-onnx, ollama:qwen2.5:1.5b
|
||||
- quality_blocked: apple-coreml:qwen2.5-0.5b-instruct-onnx, ollama:qwen2.5:1.5b
|
||||
- provider_failed: aucun
|
||||
- prochain lot recommande: Analyser les runs ayant atteint gate/repair puis resserrer la reference locale autour des meilleurs candidats.
|
||||
- checkpoint manuel en attente: Le runtime Apple sert `qwen2.5-0.5b-instruct-onnx` au lieu de `stateful-mistral7b-instruct-int4-coreml`.
|
||||
- commande preparee: `bash scripts/prepare_runtime_step.sh --apple-model stateful-mistral7b-instruct-int4-coreml --resume-state /Users/electron/Documents/Projets_Creatifs/ai-novel-engine/automation/state/next_lots_state.json --ane-script /Users/electron/Documents/Projets_Creatifs/ai-novel-engine/scripts/run_next_lots.py`
|
||||
- reprise: `python3 scripts/run_next_lots.py --resume /Users/electron/Documents/Projets_Creatifs/ai-novel-engine/automation/state/next_lots_state.json`
|
||||
<!-- AUTO-SYNC:ANE-TODO-ACTIVE:END -->
|
||||
@@ -0,0 +1,110 @@
|
||||
# TODO implemente - AI Novel Engine
|
||||
|
||||
Snapshot append-only de ce qui est reellement livre.
|
||||
|
||||
Regle:
|
||||
- n'ajouter ici qu'un lot termine
|
||||
- ne pas y laisser de travail restant
|
||||
- renvoyer vers `TODO_ACTIVE.md` pour les suites et blocages
|
||||
|
||||
## Deja implemente
|
||||
|
||||
### Lot livre - 7 mars 2026
|
||||
- [x] Pipeline chapitre complet `intention -> structure -> draft -> critique -> rewrite -> validation -> memoire`
|
||||
- [x] Artefacts standardises dans `structure/`, `brouillons/`, `manuscrit/`, `memoire/`
|
||||
- [x] Normalisation des chapitres et detection explicite des collisions `chapitre_1` / `chapitre_01`
|
||||
- [x] CLI `status`, `intention create`, `generate chapter --chapter XX` et alias `write --chapter XX`
|
||||
- [x] Provider generique avec implementation OpenAI-compatible et configuration par variables d'environnement
|
||||
- [x] Branchement local via `mascarade` en pointant `ANE_BASE_URL` vers `http://127.0.0.1:8100`
|
||||
- [x] Budgets par etape avec `ANE_MAX_TOKENS_STRUCTURE`, `..._DRAFT`, `..._CRITIQUE`, `..._REWRITE`, `..._MEMORY`
|
||||
- [x] Parsing JSON tolerant pour les etapes `critique` et `memory`
|
||||
- [x] Second passage de reessai cible sur `critique` et `memory` quand la premiere reponse reste invalide
|
||||
- [x] Prompts versionnes par etape dans `prompts/`
|
||||
- [x] Metadonnees pipeline enrichies avec `stage_attempts`, `retry_stages`, `provider.*` et `last_status_message`
|
||||
- [x] Ecriture immediate de l'etape en cours dans `meta.json` avant les appels provider pour rendre les hangs lisibles
|
||||
- [x] CLI non interactive avec `--approve` et `--reject`
|
||||
- [x] `status` enrichi pour les chapitres en echec et en attente
|
||||
- [x] Smoke script local `scripts/smoke_local_generation.sh` branche sur la vraie CLI et avec warm-up Apple via `:8100`
|
||||
- [x] Presets de smoke Apple plus conservateurs et timeout local plus large pour limiter les faux negatifs de warm-up
|
||||
- [x] Runbook local ANE `docs/runbooks/LOCAL_GENERATION.md`
|
||||
- [x] `docs/vision.md` et `docs/roadmap.md` remplaces par une doc exploitable
|
||||
- [x] Suite unitaire `python3 -m unittest discover -s tests -v` verte sur l'etat livre
|
||||
|
||||
### Lot livre - 8 mars 2026
|
||||
- [x] Validation locale `ollama` de bout en bout avec `ollama:qwen2.5:1.5b` via `mascarade`
|
||||
- [x] Validation Apple locale de bout en bout avec `apple-coreml:qwen2.5-0.5b-instruct-onnx`
|
||||
- [x] Validation Apple locale de bout en bout avec `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||
- [x] Comparatif local documente dans `docs/MODEL_COMPARISON_2026-03-08.md`
|
||||
- [x] Runbook local ANE et `README` realignes sur les modeles reellement valides
|
||||
|
||||
### Lot livre - 8 mars 2026 (garde-fou qualite)
|
||||
- [x] Nouvelle etape `gate` entre `rewrite` et la validation auteur
|
||||
- [x] Type `ManuscriptGateReport` et artefact `brouillons/chapitres/chapitre_XX/gate_v1.json`
|
||||
- [x] Heuristiques bloquantes locales `too_short`, `truncated_ending`, `outline_like`
|
||||
- [x] Budget provider `ANE_MAX_TOKENS_GATE`
|
||||
- [x] `--approve` et la promotion manuscrit ne bypassent jamais le garde-fou
|
||||
- [x] `meta.json`, `status` et le smoke exposent `quality_blockers`, `gate_report`, `gate_v1` et les chapitres `quality_blocked`
|
||||
- [x] Revalidation du protocole qualite:
|
||||
- `ollama:qwen2.5:1.5b` -> `quality_blocked` au garde-fou
|
||||
- `apple-coreml:qwen2.5-0.5b-instruct-onnx` -> `quality_blocked` au garde-fou
|
||||
- `apple-coreml:qwen3.5-4b-onnx-q4f16` -> `provider_failed` en `rewrite`
|
||||
- `ollama:qwen2.5:7b` -> `provider_failed` par timeout client en `draft`
|
||||
- [x] Suite unitaire `python3 -m unittest discover -s tests -v` verte avec 27 tests
|
||||
|
||||
### Lot livre - 8 mars 2026 (durcissement prose)
|
||||
- [x] Prompts `draft_v1` et `rewrite_v1` renforces pour interdire titres, puces et labels de plan visibles
|
||||
- [x] Consignes explicites de prose continue, de scene jouee et de fin de phrase complete
|
||||
- [x] Fix runtime cote `mascarade` avec `OLLAMA_TIMEOUT_SECONDS` configurable et timeout explicite
|
||||
- [x] Rerun reel `ollama:qwen2.5:1.5b` complete a nouveau jusqu'au garde-fou (`499` mots), mais reste `quality_blocked`
|
||||
- [x] Rerun reel `apple-coreml:qwen2.5-0.5b-instruct-onnx` complete jusqu'au garde-fou (`538` mots), mais reste `quality_blocked`
|
||||
|
||||
### Lot livre - 8 mars 2026 (repair + reruns bornes)
|
||||
- [x] Boucle `repair` automatique entre `gate` et `quality_blocked`
|
||||
- [x] Preservation de `draft_v2.md` et ajout des artefacts `repair_vN.md`
|
||||
- [x] Budget `ANE_MAX_TOKENS_REPAIR`, limite `ANE_REPAIR_MAX_PASSES` et override `ANE_REPAIR_FALLBACK_MODEL`
|
||||
- [x] `meta.json`, `status` et le smoke exposent `repair_attempts`, `repair_models`, `repair_latest` et le brouillon final candidat
|
||||
- [x] Timeout provider `urllib` remonte maintenant en `ProviderError`, ce qui marque correctement `failed_stage`
|
||||
- [x] Le warm-up Apple du smoke remonte maintenant un message d'erreur lisible
|
||||
- [x] Le fallback `repair` n'essaie plus automatiquement un autre modele `apple-coreml` au milieu d'un meme smoke; `qwen2.5-0.5b` bascule desormais vers un fallback non-Apple
|
||||
- [x] Suite unitaire etendue a 34 tests verts
|
||||
- [x] Reruns reels sous preset qualite borne a `300s` par requete:
|
||||
- `ollama:qwen2.5:1.5b` -> `failed_stage=structure`
|
||||
- `apple-coreml:qwen2.5-0.5b-instruct-onnx` -> `failed_stage=rewrite`
|
||||
- `apple-coreml:qwen3.5-4b-onnx-q4f16` -> `failed_stage=rewrite`
|
||||
- `ollama:qwen2.5:7b` -> `failed_stage=rewrite`
|
||||
- [x] Conclusion du cycle: la boucle `repair` est livree et preparee; le goulot courant reste `rewrite` tant que les meilleurs candidats n'ont pas ete rejoues
|
||||
|
||||
### Lot livre - 9 mars 2026 (automation des lots utiles)
|
||||
- [x] Orchestrateur central `python3 scripts/run_next_lots.py`
|
||||
- [x] Manifeste versionne `automation/next_lots.toml`
|
||||
- [x] Etat de reprise local et rapports machines dans `automation/state/` et `automation/reports/`
|
||||
- [x] Reutilisation des smokes existants `scripts/smoke_local_generation.sh` et `mascarade/scripts/smoke_openai_compat_ane.sh`
|
||||
- [x] Synchronisation directe des plans/TODOs/readmes/runbooks dans des sections `AUTO-SYNC`
|
||||
- [x] Helper `mascarade/scripts/ensure_apple_models.sh` pour verifier ou installer les trois modeles Apple requis
|
||||
- [x] Helper `mascarade/scripts/prepare_runtime_step.sh` pour preparer les checkpoints semi-autos de restart/switch runtime
|
||||
- [x] Attente courte sur `/models` apres un switch Apple pour eviter les faux checkpoints `aucun modele`
|
||||
- [x] Couverture unitaire du manifeste, des checkpoints Apple, du rendu `AUTO-SYNC` et des helpers shell
|
||||
|
||||
### Lot livre - 9 mars 2026 (priority_models automatise)
|
||||
- [x] Cycle reel `python3 scripts/run_next_lots.py --lot priority_models` termine jusqu'a `tracking_sync`
|
||||
- [x] `apple-coreml:qwen3.5-4b-onnx-q4f16` accepte de bout en bout sous protocole `gate + repair`
|
||||
- [x] `ollama:qwen2.5:7b` atteint `gate`, exerce `repair` en live sur deux passes, puis finit `quality_blocked` avec `outline_like`
|
||||
- [x] Le comparatif local, les TODOs, les README et les runbooks disposent maintenant d'un premier resultat `accepted` sous protocole courant
|
||||
|
||||
## Actif
|
||||
- [x] Aucun suivi actif ici. Voir `TODO_ACTIVE.md`.
|
||||
|
||||
## Bloque
|
||||
- [x] Aucun blocage suivi ici. Voir `TODO_ACTIVE.md`.
|
||||
|
||||
## Prochain ordre
|
||||
- [x] Mettre a jour ce fichier uniquement quand un nouveau lot est reellement termine.
|
||||
|
||||
## Auto-sync
|
||||
## Auto-sync
|
||||
<!-- AUTO-SYNC:ANE-TODO-DONE:START -->
|
||||
- orchestrateur `scripts/run_next_lots.py` disponible
|
||||
- manifeste `automation/next_lots.toml` charge
|
||||
- derniers fichiers de suivi synchronisables via marqueurs `AUTO-SYNC`
|
||||
- dernier cycle automatise observe: 2026-03-09T06:53:02+00:00
|
||||
<!-- AUTO-SYNC:ANE-TODO-DONE:END -->
|
||||
@@ -0,0 +1,70 @@
|
||||
[paths]
|
||||
mascarade_repo = "/Users/electron/mascarade"
|
||||
core_base_url = "http://127.0.0.1:8100"
|
||||
apple_runtime_url = "http://127.0.0.1:8201"
|
||||
ollama_tags_url = "http://127.0.0.1:11435/api/tags"
|
||||
apple_model_ready_timeout_seconds = 30
|
||||
apple_model_poll_interval_seconds = 2
|
||||
|
||||
[smoke]
|
||||
chapter = "02"
|
||||
intention = "Chapitre court. Une femme arrive dans une ville de nuit, trouve un indice simple, et finit sur une decision risquee. Style direct, phrases courtes, ton sobre."
|
||||
timeout_seconds = 300
|
||||
|
||||
[preset]
|
||||
ANE_PROVIDER = "openai_compatible"
|
||||
ANE_BASE_URL = "http://127.0.0.1:8100"
|
||||
ANE_TIMEOUT = "300"
|
||||
ANE_MAX_TOKENS_STRUCTURE = "256"
|
||||
ANE_MAX_TOKENS_DRAFT = "768"
|
||||
ANE_MAX_TOKENS_CRITIQUE = "512"
|
||||
ANE_MAX_TOKENS_REWRITE = "768"
|
||||
ANE_MAX_TOKENS_GATE = "384"
|
||||
ANE_MAX_TOKENS_REPAIR = "512"
|
||||
ANE_MAX_TOKENS_MEMORY = "320"
|
||||
ANE_REPAIR_MAX_PASSES = "2"
|
||||
|
||||
[ensure_models]
|
||||
apple_models = [
|
||||
"qwen2.5-0.5b-instruct-onnx",
|
||||
"qwen3.5-4b-onnx-q4f16",
|
||||
"stateful-mistral7b-instruct-int4-coreml",
|
||||
]
|
||||
ollama_models = [
|
||||
"qwen2.5:7b",
|
||||
"qwen2.5:1.5b",
|
||||
]
|
||||
|
||||
[lots.priority_models]
|
||||
models = [
|
||||
"apple-coreml:qwen3.5-4b-onnx-q4f16",
|
||||
"ollama:qwen2.5:7b",
|
||||
]
|
||||
|
||||
[lots.baselines]
|
||||
models = [
|
||||
"apple-coreml:qwen2.5-0.5b-instruct-onnx",
|
||||
"ollama:qwen2.5:1.5b",
|
||||
]
|
||||
|
||||
[lots.preflight_only]
|
||||
models = [
|
||||
"apple-coreml:stateful-mistral7b-instruct-int4-coreml",
|
||||
]
|
||||
|
||||
[tracking.ane]
|
||||
todo_active = "TODO_ACTIVE.md"
|
||||
todo_done = "TODO_IMPLEMENTE.md"
|
||||
plan = "docs/EXECUTION_PLAN_2026-03-08.md"
|
||||
comparison = "docs/MODEL_COMPARISON_2026-03-08.md"
|
||||
readme = "README.md"
|
||||
runbook = "docs/runbooks/LOCAL_GENERATION.md"
|
||||
|
||||
[tracking.mascarade]
|
||||
todo = "TODO_AI_NOVEL_ENGINE.md"
|
||||
plan = "docs/EXECUTION_PLAN_2026-03-08.md"
|
||||
readme = "README.md"
|
||||
runbook = "docs/RUNBOOK_APPLE_LLM_LOCAL.md"
|
||||
|
||||
[next_actions]
|
||||
rewrite_compaction = "Compacter ou reduire `rewrite_v1` et ses budgets pour faire passer au moins `apple-coreml:qwen3.5-4b-onnx-q4f16` ou `ollama:qwen2.5:7b` jusqu'a `gate`."
|
||||
+211
-61
@@ -1,8 +1,13 @@
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
from core.chapters import ChapterConflictError, ChapterError, ChapterId, resolve_chapter_file
|
||||
from core.generation.pipeline import GenerationPipeline
|
||||
from core.generation.provider import ProviderConfigurationError, ProviderError
|
||||
from core.project.loader import ProjectState
|
||||
from core.intention.gate import IntentionGate
|
||||
|
||||
|
||||
def cmd_status(root: Path):
|
||||
@@ -11,92 +16,237 @@ def cmd_status(root: Path):
|
||||
|
||||
print("\nAI Novel Engine — Project Status\n")
|
||||
|
||||
if state["current_chapter"] is None:
|
||||
print("📄 Aucun chapitre détecté.")
|
||||
current = state["current_chapter"]
|
||||
if current is None:
|
||||
print("Chapitre courant : aucun")
|
||||
else:
|
||||
print(f"📄 Chapitre courant : {state['current_chapter']}")
|
||||
print(f"Chapitre courant : {current}")
|
||||
|
||||
print(f"📐 Structure présente : {state['has_structure']}")
|
||||
print(f"🧠 Mémoire présente : {state['has_memory']}")
|
||||
print("\n(Prochaine étape : définir une intention)\n")
|
||||
print("\nDossiers:")
|
||||
for label, key in (
|
||||
("Structure", "structure"),
|
||||
("Brouillons", "drafts"),
|
||||
("Manuscrit", "manuscript"),
|
||||
("Memoire", "memory"),
|
||||
):
|
||||
print(f"- {label:<10}: {state['directories'][key]}")
|
||||
|
||||
latest_drafts = state["latest_drafts"]
|
||||
if latest_drafts:
|
||||
print("\nDerniers brouillons:")
|
||||
for chapter_slug, draft_name in sorted(latest_drafts.items()):
|
||||
print(f"- {chapter_slug}: {draft_name}")
|
||||
else:
|
||||
print("\nDerniers brouillons: aucun")
|
||||
|
||||
latest_repairs = state["latest_repairs"]
|
||||
if latest_repairs:
|
||||
print("\nDernières réparations:")
|
||||
for chapter_slug, repair_name in sorted(latest_repairs.items()):
|
||||
print(f"- {chapter_slug}: {repair_name}")
|
||||
else:
|
||||
print("\nDernières réparations: aucune")
|
||||
|
||||
failures = state["failed_chapters"]
|
||||
if failures:
|
||||
print("\nChapitres en échec:")
|
||||
for item in failures:
|
||||
retry_suffix = ""
|
||||
if item["retry_stages"]:
|
||||
retry_suffix = f" | réessais JSON: {', '.join(item['retry_stages'])}"
|
||||
status_message = f" | message: {item['last_status_message']}" if item["last_status_message"] else ""
|
||||
print(
|
||||
f"- {item['chapter']}: status={item['status']} | failed_stage={item['failed_stage']} | meta={item['meta_path']}{retry_suffix}{status_message}"
|
||||
)
|
||||
else:
|
||||
print("\nChapitres en échec: aucun")
|
||||
|
||||
quality_blocked = state["quality_blocked_chapters"]
|
||||
if quality_blocked:
|
||||
print("\nBloqués par garde-fou:")
|
||||
for item in quality_blocked:
|
||||
retry_suffix = ""
|
||||
if item["retry_stages"]:
|
||||
retry_suffix = f" | réessais JSON: {', '.join(item['retry_stages'])}"
|
||||
blockers_suffix = ""
|
||||
if item["quality_blockers"]:
|
||||
blockers_suffix = f" | blockers: {', '.join(item['quality_blockers'])}"
|
||||
repair_suffix = ""
|
||||
if item["repair_attempts"]:
|
||||
repair_models = ", ".join(item["repair_models"]) if item["repair_models"] else "provider_courant"
|
||||
repair_suffix = f" | réparations: {item['repair_attempts']} ({repair_models})"
|
||||
status_message = f" | message: {item['last_status_message']}" if item["last_status_message"] else ""
|
||||
print(
|
||||
f"- {item['chapter']}: status={item['status']} | failed_stage={item['failed_stage']} | brouillon={item['draft_path']} | gate={item['gate_path']} | meta={item['meta_path']}{blockers_suffix}{repair_suffix}{retry_suffix}{status_message}"
|
||||
)
|
||||
else:
|
||||
print("\nBloqués par garde-fou: aucun")
|
||||
|
||||
awaiting_acceptance = state["awaiting_acceptance"]
|
||||
if awaiting_acceptance:
|
||||
print("\nEn attente de validation:")
|
||||
for item in awaiting_acceptance:
|
||||
retry_suffix = ""
|
||||
if item["retry_stages"]:
|
||||
retry_suffix = f" | réessais JSON: {', '.join(item['retry_stages'])}"
|
||||
repair_suffix = ""
|
||||
if item["repair_attempts"]:
|
||||
repair_models = ", ".join(item["repair_models"]) if item["repair_models"] else "provider_courant"
|
||||
repair_suffix = f" | réparations: {item['repair_attempts']} ({repair_models})"
|
||||
status_message = f" | message: {item['last_status_message']}" if item["last_status_message"] else ""
|
||||
print(
|
||||
f"- {item['chapter']}: status={item['status']} | brouillon={item['draft_path']} | critique={item['critique_path']} | gate={item['gate_path']}{repair_suffix}{retry_suffix}{status_message}"
|
||||
)
|
||||
else:
|
||||
print("\nEn attente de validation: aucun")
|
||||
|
||||
print("")
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_intention_create(root: Path):
|
||||
def cmd_intention_create(root: Path, chapter_value: str | None = None, input_func=input):
|
||||
intentions_dir = root / "notes" / "intentions"
|
||||
intentions_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
chap = input("Numéro du chapitre (ex: 08) : ").strip()
|
||||
if not chap:
|
||||
print("❌ Numéro de chapitre requis.")
|
||||
return
|
||||
raw_chapter = chapter_value or input_func("Numéro du chapitre (ex: 08) : ").strip()
|
||||
chapter = ChapterId.parse(raw_chapter)
|
||||
|
||||
path = intentions_dir / f"chapitre_{chap}.md"
|
||||
path = resolve_chapter_file(intentions_dir, chapter)
|
||||
if path.exists():
|
||||
print(f"⚠️ Une intention existe déjà : {path}")
|
||||
return
|
||||
print(f"Une intention existe déjà : {path}")
|
||||
return 1
|
||||
|
||||
print("\nDécris l’intention (finir par Ctrl+D / Ctrl+Z):\n")
|
||||
lines = []
|
||||
try:
|
||||
while True:
|
||||
lines.append(input())
|
||||
lines.append(input_func(""))
|
||||
except EOFError:
|
||||
pass
|
||||
|
||||
content = "\n".join(lines).strip()
|
||||
content = "\n".join(line for line in lines if line is not None).strip()
|
||||
if not content:
|
||||
print("❌ Intention vide. Annulé.")
|
||||
return
|
||||
print("Intention vide. Annulé.")
|
||||
return 1
|
||||
|
||||
path.write_text(
|
||||
f"# Intention — Chapitre {chap}\n\n{content}\n",
|
||||
encoding="utf-8"
|
||||
canonical_path = intentions_dir / chapter.filename
|
||||
canonical_path.write_text(
|
||||
f"# Intention — Chapitre {chapter.label}\n\n{content}\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
print(f"✅ Intention créée : {path}\n")
|
||||
print(f"Intention créée : {canonical_path}\n")
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_write(root: Path):
|
||||
gate = IntentionGate(root)
|
||||
def _approval_callback_from_flags(force_accept: bool | None):
|
||||
if force_accept is None:
|
||||
return None
|
||||
return lambda _report, _path: force_accept
|
||||
|
||||
|
||||
def cmd_generate_chapter(
|
||||
root: Path,
|
||||
chapter_value: str,
|
||||
provider=None,
|
||||
input_func=input,
|
||||
*,
|
||||
force_accept: bool | None = None,
|
||||
):
|
||||
pipeline = GenerationPipeline(root, provider=provider, input_func=input_func)
|
||||
outcome = pipeline.generate_chapter(
|
||||
chapter_value,
|
||||
approval_callback=_approval_callback_from_flags(force_accept),
|
||||
)
|
||||
|
||||
print("")
|
||||
print(f"Chapitre traité : {outcome.chapter_id.slug}")
|
||||
print(f"Statut : {outcome.status}")
|
||||
print(f"Brouillon final : {outcome.draft_path}")
|
||||
print(f"Critique : {outcome.critique_path}")
|
||||
print(f"Garde-fou : {outcome.gate_path}")
|
||||
print(f"Métadonnées : {outcome.meta_path}")
|
||||
if outcome.accepted and outcome.manuscript_path is not None:
|
||||
print(f"Manuscrit : {outcome.manuscript_path}")
|
||||
else:
|
||||
print("Manuscrit : non promu")
|
||||
if outcome.quality_blockers:
|
||||
print(f"Blockers : {', '.join(outcome.quality_blockers)}")
|
||||
print("")
|
||||
return 0
|
||||
|
||||
|
||||
def _add_approval_flags(parser: argparse.ArgumentParser) -> None:
|
||||
group = parser.add_mutually_exclusive_group()
|
||||
group.add_argument("--approve", action="store_true", help="Promouvoir sans confirmation interactive.")
|
||||
group.add_argument("--reject", action="store_true", help="Refuser sans confirmation interactive.")
|
||||
|
||||
|
||||
def _force_accept_from_namespace(namespace: argparse.Namespace) -> bool | None:
|
||||
if getattr(namespace, "approve", False):
|
||||
return True
|
||||
if getattr(namespace, "reject", False):
|
||||
return False
|
||||
return None
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(prog="python3 -m cli.main")
|
||||
subparsers = parser.add_subparsers(dest="command")
|
||||
|
||||
subparsers.add_parser("status")
|
||||
|
||||
intention_parser = subparsers.add_parser("intention")
|
||||
intention_subparsers = intention_parser.add_subparsers(dest="intention_command")
|
||||
intention_create = intention_subparsers.add_parser("create")
|
||||
intention_create.add_argument("--chapter")
|
||||
|
||||
generate_parser = subparsers.add_parser("generate")
|
||||
generate_subparsers = generate_parser.add_subparsers(dest="generate_target")
|
||||
generate_chapter = generate_subparsers.add_parser("chapter")
|
||||
generate_chapter.add_argument("--chapter", required=True)
|
||||
_add_approval_flags(generate_chapter)
|
||||
|
||||
write_parser = subparsers.add_parser("write")
|
||||
write_parser.add_argument("--chapter", required=True)
|
||||
_add_approval_flags(write_parser)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None, root: Path | None = None):
|
||||
args = argv if argv is not None else sys.argv[1:]
|
||||
project_root = root or Path.cwd()
|
||||
parser = build_parser()
|
||||
namespace = parser.parse_args(args)
|
||||
|
||||
if not args or namespace.command == "status":
|
||||
return cmd_status(project_root)
|
||||
|
||||
try:
|
||||
gate.assert_intention()
|
||||
except RuntimeError as e:
|
||||
print("\n⛔ ÉCRITURE BLOQUÉE\n")
|
||||
print(str(e))
|
||||
print("\n➡️ Utilise : python3 -m cli.main intention create\n")
|
||||
return
|
||||
if namespace.command == "intention" and namespace.intention_command == "create":
|
||||
return cmd_intention_create(project_root, chapter_value=namespace.chapter)
|
||||
|
||||
print("\n✅ Intention détectée.")
|
||||
print("✍️ Écriture autorisée (génération non implémentée).\n")
|
||||
if namespace.command == "generate" and namespace.generate_target == "chapter":
|
||||
return cmd_generate_chapter(
|
||||
project_root,
|
||||
namespace.chapter,
|
||||
force_accept=_force_accept_from_namespace(namespace),
|
||||
)
|
||||
|
||||
if namespace.command == "write":
|
||||
return cmd_generate_chapter(
|
||||
project_root,
|
||||
namespace.chapter,
|
||||
force_accept=_force_accept_from_namespace(namespace),
|
||||
)
|
||||
except (ChapterError, ChapterConflictError, RuntimeError, ProviderConfigurationError, ProviderError) as exc:
|
||||
print(f"\nErreur: {exc}\n")
|
||||
return 1
|
||||
|
||||
def main():
|
||||
root = Path.cwd()
|
||||
|
||||
# Aucun argument → status
|
||||
if len(sys.argv) == 1:
|
||||
cmd_status(root)
|
||||
return
|
||||
|
||||
# intention create
|
||||
if sys.argv[1] == "intention" and len(sys.argv) >= 3:
|
||||
if sys.argv[2] == "create":
|
||||
cmd_intention_create(root)
|
||||
return
|
||||
|
||||
# write
|
||||
if sys.argv[1] == "write":
|
||||
cmd_write(root)
|
||||
return
|
||||
|
||||
# aide
|
||||
print("\nCommande inconnue.\n")
|
||||
print("Commandes disponibles :")
|
||||
print(" python3 -m cli.main → status")
|
||||
print(" python3 -m cli.main intention create")
|
||||
print(" python3 -m cli.main write\n")
|
||||
parser.print_help()
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
raise SystemExit(main())
|
||||
|
||||
@@ -0,0 +1,127 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
|
||||
_CHAPTER_PATTERN = re.compile(r"^chapitre_(\d+)$", re.IGNORECASE)
|
||||
_DIGITS_PATTERN = re.compile(r"^\d+$")
|
||||
|
||||
|
||||
class ChapterError(ValueError):
|
||||
"""Raised when a chapter identifier is invalid."""
|
||||
|
||||
|
||||
class ChapterConflictError(ChapterError):
|
||||
"""Raised when both canonical and legacy files exist for the same chapter."""
|
||||
|
||||
def __init__(self, chapter: "ChapterId", paths: list[Path]):
|
||||
joined = ", ".join(str(path) for path in paths)
|
||||
super().__init__(
|
||||
f"Conflit de chapitre pour {chapter.slug}: plusieurs fichiers existent ({joined})."
|
||||
)
|
||||
self.chapter = chapter
|
||||
self.paths = paths
|
||||
|
||||
|
||||
@dataclass(frozen=True, order=True)
|
||||
class ChapterId:
|
||||
number: int
|
||||
|
||||
def __post_init__(self):
|
||||
if self.number <= 0:
|
||||
raise ChapterError("Le numéro de chapitre doit être strictement positif.")
|
||||
|
||||
@classmethod
|
||||
def parse(cls, value: object) -> "ChapterId":
|
||||
return cls(parse_chapter_number(value))
|
||||
|
||||
@property
|
||||
def label(self) -> str:
|
||||
return f"{self.number:02d}"
|
||||
|
||||
@property
|
||||
def slug(self) -> str:
|
||||
return f"chapitre_{self.label}"
|
||||
|
||||
@property
|
||||
def filename(self) -> str:
|
||||
return f"{self.slug}.md"
|
||||
|
||||
def __str__(self) -> str:
|
||||
return self.slug
|
||||
|
||||
|
||||
def parse_chapter_number(value: object) -> int:
|
||||
if isinstance(value, ChapterId):
|
||||
return value.number
|
||||
|
||||
if isinstance(value, int) and not isinstance(value, bool):
|
||||
return value
|
||||
|
||||
if isinstance(value, Path):
|
||||
text = value.stem
|
||||
else:
|
||||
text = str(value).strip()
|
||||
|
||||
if not text:
|
||||
raise ChapterError("Numéro de chapitre requis.")
|
||||
|
||||
candidate = text
|
||||
if "/" in candidate or "\\" in candidate or candidate.endswith(".md"):
|
||||
candidate = Path(candidate).stem
|
||||
|
||||
match = _CHAPTER_PATTERN.fullmatch(candidate)
|
||||
if match:
|
||||
return int(match.group(1))
|
||||
|
||||
if _DIGITS_PATTERN.fullmatch(candidate):
|
||||
return int(candidate)
|
||||
|
||||
raise ChapterError(f"Identifiant de chapitre invalide: {value!r}")
|
||||
|
||||
|
||||
def discover_chapter_files(directory: Path) -> list[tuple[ChapterId, Path]]:
|
||||
if not directory.exists():
|
||||
return []
|
||||
|
||||
discovered: list[tuple[ChapterId, Path]] = []
|
||||
for path in sorted(directory.glob("chapitre_*.md")):
|
||||
try:
|
||||
chapter = ChapterId.parse(path.stem)
|
||||
except ChapterError:
|
||||
continue
|
||||
discovered.append((chapter, path))
|
||||
return discovered
|
||||
|
||||
|
||||
def discover_chapter_dirs(directory: Path) -> list[tuple[ChapterId, Path]]:
|
||||
if not directory.exists():
|
||||
return []
|
||||
|
||||
discovered: list[tuple[ChapterId, Path]] = []
|
||||
for path in sorted(directory.glob("chapitre_*")):
|
||||
if not path.is_dir():
|
||||
continue
|
||||
try:
|
||||
chapter = ChapterId.parse(path.name)
|
||||
except ChapterError:
|
||||
continue
|
||||
discovered.append((chapter, path))
|
||||
return discovered
|
||||
|
||||
|
||||
def collect_matching_chapter_files(directory: Path, chapter: ChapterId) -> list[Path]:
|
||||
paths = [path for candidate, path in discover_chapter_files(directory) if candidate == chapter]
|
||||
unique_paths = sorted(set(paths))
|
||||
return unique_paths
|
||||
|
||||
|
||||
def resolve_chapter_file(directory: Path, chapter: ChapterId) -> Path:
|
||||
matches = collect_matching_chapter_files(directory, chapter)
|
||||
if len(matches) > 1:
|
||||
raise ChapterConflictError(chapter, matches)
|
||||
if matches:
|
||||
return matches[0]
|
||||
return directory / chapter.filename
|
||||
@@ -0,0 +1 @@
|
||||
"""Generation pipeline primitives."""
|
||||
@@ -0,0 +1,313 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
from core.chapters import ChapterId
|
||||
|
||||
|
||||
def _strip_code_fence(text: str) -> str:
|
||||
payload = text.strip()
|
||||
if not payload.startswith("```"):
|
||||
return payload
|
||||
lines = payload.splitlines()
|
||||
if len(lines) >= 3 and lines[-1].strip() == "```":
|
||||
return "\n".join(lines[1:-1]).strip()
|
||||
return payload
|
||||
|
||||
|
||||
def _remove_trailing_commas(payload: str) -> str:
|
||||
return re.sub(r",(\s*[}\]])", r"\1", payload)
|
||||
|
||||
|
||||
def _extract_json_object(payload: str) -> str | None:
|
||||
start = payload.find("{")
|
||||
if start == -1:
|
||||
return None
|
||||
|
||||
depth = 0
|
||||
in_string = False
|
||||
escaped = False
|
||||
for index in range(start, len(payload)):
|
||||
char = payload[index]
|
||||
if in_string:
|
||||
if escaped:
|
||||
escaped = False
|
||||
elif char == "\\":
|
||||
escaped = True
|
||||
elif char == '"':
|
||||
in_string = False
|
||||
continue
|
||||
if char == '"':
|
||||
in_string = True
|
||||
elif char == "{":
|
||||
depth += 1
|
||||
elif char == "}":
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return payload[start : index + 1]
|
||||
return payload[start:]
|
||||
|
||||
|
||||
def _close_json_delimiters(payload: str) -> str:
|
||||
stack: list[str] = []
|
||||
in_string = False
|
||||
escaped = False
|
||||
for char in payload:
|
||||
if in_string:
|
||||
if escaped:
|
||||
escaped = False
|
||||
elif char == "\\":
|
||||
escaped = True
|
||||
elif char == '"':
|
||||
in_string = False
|
||||
continue
|
||||
if char == '"':
|
||||
in_string = True
|
||||
elif char == "{":
|
||||
stack.append("}")
|
||||
elif char == "[":
|
||||
stack.append("]")
|
||||
elif char in {"}", "]"} and stack and char == stack[-1]:
|
||||
stack.pop()
|
||||
|
||||
repaired = payload.rstrip()
|
||||
if repaired.endswith(","):
|
||||
repaired = repaired[:-1].rstrip()
|
||||
if in_string:
|
||||
repaired += '"'
|
||||
return repaired + "".join(reversed(stack))
|
||||
|
||||
|
||||
def _json_candidates(text: str) -> list[str]:
|
||||
payload = _strip_code_fence(text)
|
||||
candidates = [payload]
|
||||
|
||||
extracted = _extract_json_object(payload)
|
||||
if extracted and extracted not in candidates:
|
||||
candidates.append(extracted)
|
||||
|
||||
repaired: list[str] = []
|
||||
for candidate in list(candidates):
|
||||
trimmed = _remove_trailing_commas(candidate)
|
||||
if trimmed not in candidates and trimmed not in repaired:
|
||||
repaired.append(trimmed)
|
||||
closed = _close_json_delimiters(trimmed)
|
||||
if closed not in candidates and closed not in repaired:
|
||||
repaired.append(closed)
|
||||
candidates.extend(repaired)
|
||||
return candidates
|
||||
|
||||
|
||||
def _parse_json_object(text: str) -> dict[str, object]:
|
||||
last_error: Exception | None = None
|
||||
for candidate in _json_candidates(text):
|
||||
try:
|
||||
data = json.loads(candidate)
|
||||
except json.JSONDecodeError as exc:
|
||||
last_error = exc
|
||||
continue
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError("La réponse JSON attendue doit être un objet.")
|
||||
return data
|
||||
raise ValueError(str(last_error) if last_error else "Réponse JSON illisible.")
|
||||
|
||||
|
||||
def _string_list(value: object) -> list[str]:
|
||||
if not isinstance(value, list):
|
||||
return []
|
||||
return [str(item).strip() for item in value if str(item).strip()]
|
||||
|
||||
|
||||
def _record_list(value: object, required_key: str) -> list[dict[str, str]]:
|
||||
if not isinstance(value, list):
|
||||
return []
|
||||
|
||||
normalized: list[dict[str, str]] = []
|
||||
for item in value:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
record = {str(key): str(val).strip() for key, val in item.items() if str(val).strip()}
|
||||
if record.get(required_key):
|
||||
normalized.append(record)
|
||||
return normalized
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class StructurePlan:
|
||||
chapter_id: ChapterId
|
||||
markdown: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ControlReport:
|
||||
summary: str
|
||||
deviations: list[str]
|
||||
recommendations: list[str]
|
||||
rewrite_required: bool
|
||||
raw: dict[str, object] = field(default_factory=dict)
|
||||
|
||||
@classmethod
|
||||
def from_response_text(cls, text: str) -> "ControlReport":
|
||||
raw = _parse_json_object(text)
|
||||
summary = str(raw.get("summary", "")).strip() or "Aucun résumé fourni."
|
||||
deviations = _string_list(raw.get("deviations"))
|
||||
recommendations = _string_list(raw.get("recommendations"))
|
||||
rewrite_required = bool(raw.get("rewrite_required", deviations or recommendations))
|
||||
return cls(
|
||||
summary=summary,
|
||||
deviations=deviations,
|
||||
recommendations=recommendations,
|
||||
rewrite_required=rewrite_required,
|
||||
raw=raw,
|
||||
)
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"summary": self.summary,
|
||||
"deviations": list(self.deviations),
|
||||
"recommendations": list(self.recommendations),
|
||||
"rewrite_required": self.rewrite_required,
|
||||
}
|
||||
|
||||
def to_markdown(self, chapter_id: ChapterId) -> str:
|
||||
verdict = "oui" if self.rewrite_required else "non"
|
||||
deviations = "\n".join(f"- {item}" for item in self.deviations) or "- Aucun écart majeur."
|
||||
recommendations = "\n".join(f"- {item}" for item in self.recommendations) or "- Aucune recommandation."
|
||||
return (
|
||||
f"# Critique — {chapter_id.slug}\n\n"
|
||||
f"## Résumé\n{self.summary}\n\n"
|
||||
f"## Réécriture requise\n{verdict}\n\n"
|
||||
f"## Écarts\n{deviations}\n\n"
|
||||
f"## Recommandations\n{recommendations}\n"
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MemoryUpdate:
|
||||
chapter_summary: str
|
||||
characters: list[dict[str, str]]
|
||||
locations: list[dict[str, str]]
|
||||
timeline_events: list[dict[str, str]]
|
||||
raw: dict[str, object] = field(default_factory=dict)
|
||||
|
||||
@classmethod
|
||||
def from_response_text(cls, text: str) -> "MemoryUpdate":
|
||||
raw = _parse_json_object(text)
|
||||
chapter_summary = str(raw.get("summary", "")).strip() or "Résumé indisponible."
|
||||
characters = _record_list(raw.get("characters"), "name")
|
||||
locations = _record_list(raw.get("locations"), "name")
|
||||
timeline_events = _record_list(raw.get("timeline_events"), "event")
|
||||
return cls(
|
||||
chapter_summary=chapter_summary,
|
||||
characters=characters,
|
||||
locations=locations,
|
||||
timeline_events=timeline_events,
|
||||
raw=raw,
|
||||
)
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"summary": self.chapter_summary,
|
||||
"characters": list(self.characters),
|
||||
"locations": list(self.locations),
|
||||
"timeline_events": list(self.timeline_events),
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ManuscriptGateReport:
|
||||
ready_for_manuscript: bool
|
||||
summary: str
|
||||
blockers: list[str]
|
||||
recommendations: list[str]
|
||||
heuristic_blockers: list[str]
|
||||
raw: dict[str, object] = field(default_factory=dict)
|
||||
|
||||
@classmethod
|
||||
def from_response_text(cls, text: str) -> "ManuscriptGateReport":
|
||||
raw = _parse_json_object(text)
|
||||
blockers = _string_list(raw.get("blockers"))
|
||||
heuristic_blockers = _string_list(raw.get("heuristic_blockers"))
|
||||
recommendations = _string_list(raw.get("recommendations"))
|
||||
ready_default = not blockers and not heuristic_blockers
|
||||
ready_for_manuscript = bool(raw.get("ready_for_manuscript", ready_default))
|
||||
summary = str(raw.get("summary", "")).strip() or "Diagnostic manuscrit indisponible."
|
||||
return cls(
|
||||
ready_for_manuscript=ready_for_manuscript and not blockers and not heuristic_blockers,
|
||||
summary=summary,
|
||||
blockers=blockers,
|
||||
recommendations=recommendations,
|
||||
heuristic_blockers=heuristic_blockers,
|
||||
raw=raw,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_heuristics(
|
||||
cls,
|
||||
*,
|
||||
blockers: list[str],
|
||||
recommendations: list[str],
|
||||
summary: str,
|
||||
) -> "ManuscriptGateReport":
|
||||
return cls(
|
||||
ready_for_manuscript=False,
|
||||
summary=summary,
|
||||
blockers=list(blockers),
|
||||
recommendations=list(recommendations),
|
||||
heuristic_blockers=list(blockers),
|
||||
raw={},
|
||||
)
|
||||
|
||||
def all_blockers(self) -> list[str]:
|
||||
ordered: list[str] = []
|
||||
for value in [*self.heuristic_blockers, *self.blockers]:
|
||||
if value not in ordered:
|
||||
ordered.append(value)
|
||||
return ordered
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return {
|
||||
"ready_for_manuscript": self.ready_for_manuscript,
|
||||
"summary": self.summary,
|
||||
"blockers": list(self.blockers),
|
||||
"recommendations": list(self.recommendations),
|
||||
"heuristic_blockers": list(self.heuristic_blockers),
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GenerationContext:
|
||||
root: Path
|
||||
chapter_id: ChapterId
|
||||
intention_path: Path
|
||||
intention_text: str
|
||||
structure_path: Path
|
||||
draft_dir: Path
|
||||
draft_v1_path: Path
|
||||
critique_path: Path
|
||||
draft_v2_path: Path
|
||||
gate_path: Path
|
||||
meta_path: Path
|
||||
manuscript_path: Path
|
||||
memory_summary_path: Path
|
||||
memory_index_dir: Path
|
||||
story_context: str
|
||||
|
||||
def repair_path(self, attempt: int) -> Path:
|
||||
return self.draft_dir / f"repair_v{attempt}.md"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GenerationOutcome:
|
||||
chapter_id: ChapterId
|
||||
accepted: bool
|
||||
status: str
|
||||
draft_path: Path
|
||||
critique_path: Path
|
||||
gate_path: Path
|
||||
meta_path: Path
|
||||
manuscript_path: Path | None
|
||||
quality_blockers: list[str] = field(default_factory=list)
|
||||
@@ -0,0 +1,953 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
from typing import Callable, TypeVar
|
||||
|
||||
from core.chapters import ChapterId, resolve_chapter_file
|
||||
from core.generation.models import (
|
||||
ControlReport,
|
||||
GenerationContext,
|
||||
GenerationOutcome,
|
||||
ManuscriptGateReport,
|
||||
MemoryUpdate,
|
||||
StructurePlan,
|
||||
)
|
||||
from core.generation.provider import (
|
||||
clone_provider_with_model,
|
||||
GenerationProvider,
|
||||
GenerationRequest,
|
||||
OpenAICompatibleProvider,
|
||||
ProviderError,
|
||||
build_provider_from_env,
|
||||
)
|
||||
from core.intention.gate import IntentionGate
|
||||
from core.prompts import PromptStore
|
||||
|
||||
|
||||
ApprovalCallback = Callable[[ControlReport, Path], bool]
|
||||
OutputCallback = Callable[[str], None]
|
||||
ParsedStagePayload = TypeVar("ParsedStagePayload")
|
||||
|
||||
|
||||
class GenerationPipeline:
|
||||
def __init__(
|
||||
self,
|
||||
root: Path,
|
||||
provider: GenerationProvider | None = None,
|
||||
prompt_store: PromptStore | None = None,
|
||||
input_func: Callable[[str], str] = input,
|
||||
output_func: OutputCallback = print,
|
||||
):
|
||||
self.root = root
|
||||
self.provider = provider
|
||||
self.prompt_store = prompt_store or PromptStore(root)
|
||||
self.input_func = input_func
|
||||
self.output_func = output_func
|
||||
self.intention_gate = IntentionGate(root)
|
||||
|
||||
def generate_chapter(
|
||||
self,
|
||||
chapter: object,
|
||||
approval_callback: ApprovalCallback | None = None,
|
||||
) -> GenerationOutcome:
|
||||
chapter_id = ChapterId.parse(chapter)
|
||||
self.intention_gate.assert_intention(chapter_id)
|
||||
context = self._build_context(chapter_id)
|
||||
provider = self.provider or build_provider_from_env()
|
||||
metadata = self._initial_metadata(context, provider)
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
|
||||
structure_plan: StructurePlan | None = None
|
||||
draft_v1: str | None = None
|
||||
control_report: ControlReport | None = None
|
||||
draft_v2: str | None = None
|
||||
gate_report: ManuscriptGateReport | None = None
|
||||
current_candidate_text: str | None = None
|
||||
current_candidate_path: Path = context.draft_v2_path
|
||||
current_stage = "setup"
|
||||
|
||||
try:
|
||||
current_stage = "structure"
|
||||
structure_plan = self._generate_structure(provider, context, metadata)
|
||||
self._write_text(context.structure_path, structure_plan.markdown)
|
||||
self._complete_stage(metadata, current_stage)
|
||||
self._set_status(metadata, "structure_ready", "Structure générée.")
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
|
||||
current_stage = "draft"
|
||||
draft_v1 = self._generate_draft(provider, context, structure_plan, metadata)
|
||||
self._write_text(context.draft_v1_path, draft_v1)
|
||||
self._complete_stage(metadata, current_stage)
|
||||
self._set_status(metadata, "draft_ready", "Brouillon initial généré.")
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
|
||||
current_stage = "critique"
|
||||
control_report = self._generate_control_report(provider, context, structure_plan, draft_v1, metadata)
|
||||
self._write_text(context.critique_path, control_report.to_markdown(context.chapter_id))
|
||||
self._complete_stage(metadata, current_stage)
|
||||
self._set_status(metadata, "critique_ready", "Critique structurée générée.")
|
||||
metadata["control_report"] = control_report.to_dict()
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
|
||||
current_stage = "rewrite"
|
||||
draft_v2 = self._rewrite_draft(provider, context, structure_plan, draft_v1, control_report, metadata)
|
||||
self._write_text(context.draft_v2_path, draft_v2)
|
||||
self._complete_stage(metadata, current_stage)
|
||||
self._set_status(metadata, "rewrite_ready", "Brouillon final généré, contrôle manuscrit en cours.")
|
||||
metadata["draft_final"] = str(context.draft_v2_path)
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
current_candidate_text = draft_v2
|
||||
current_candidate_path = context.draft_v2_path
|
||||
|
||||
current_stage = "gate"
|
||||
gate_report = self._generate_manuscript_gate_report(
|
||||
provider,
|
||||
context,
|
||||
structure_plan,
|
||||
current_candidate_text,
|
||||
metadata,
|
||||
)
|
||||
self._persist_gate_report(metadata, context, gate_report, current_candidate_path)
|
||||
self._complete_stage(metadata, current_stage)
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
|
||||
if not gate_report.ready_for_manuscript:
|
||||
current_candidate_text, current_candidate_path, gate_report = self._repair_until_ready(
|
||||
provider=provider,
|
||||
context=context,
|
||||
structure_plan=structure_plan,
|
||||
current_candidate_text=current_candidate_text,
|
||||
current_candidate_path=current_candidate_path,
|
||||
gate_report=gate_report,
|
||||
metadata=metadata,
|
||||
)
|
||||
if not gate_report.ready_for_manuscript:
|
||||
self._set_status(metadata, "quality_blocked", "Promotion bloquée par le garde-fou manuscrit.")
|
||||
metadata["failed_stage"] = current_stage
|
||||
metadata["finished_at"] = self._timestamp()
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
return GenerationOutcome(
|
||||
chapter_id=chapter_id,
|
||||
accepted=False,
|
||||
status="quality_blocked",
|
||||
draft_path=current_candidate_path,
|
||||
critique_path=context.critique_path,
|
||||
gate_path=context.gate_path,
|
||||
meta_path=context.meta_path,
|
||||
manuscript_path=None,
|
||||
quality_blockers=gate_report.all_blockers(),
|
||||
)
|
||||
|
||||
self._set_status(metadata, "awaiting_acceptance", "Brouillon final prêt pour validation.")
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
|
||||
accepted = (
|
||||
approval_callback(control_report, current_candidate_path)
|
||||
if approval_callback is not None
|
||||
else self._prompt_for_acceptance(control_report, current_candidate_path)
|
||||
)
|
||||
metadata["accepted"] = accepted
|
||||
|
||||
if not accepted:
|
||||
self._set_status(metadata, "rejected", "Promotion refusée par l'auteur.")
|
||||
metadata["finished_at"] = self._timestamp()
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
return GenerationOutcome(
|
||||
chapter_id=chapter_id,
|
||||
accepted=False,
|
||||
status="rejected",
|
||||
draft_path=current_candidate_path,
|
||||
critique_path=context.critique_path,
|
||||
gate_path=context.gate_path,
|
||||
meta_path=context.meta_path,
|
||||
manuscript_path=None,
|
||||
quality_blockers=gate_report.all_blockers() if gate_report is not None else [],
|
||||
)
|
||||
|
||||
current_stage = "memory"
|
||||
self._write_text(context.manuscript_path, current_candidate_text)
|
||||
self._set_status(
|
||||
metadata,
|
||||
"manuscript_promoted",
|
||||
"Brouillon promu dans le manuscrit, mise à jour mémoire en cours.",
|
||||
)
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
memory_update = self._generate_memory_update(provider, context, current_candidate_text, metadata)
|
||||
self._persist_memory(context, memory_update)
|
||||
self._complete_stage(metadata, current_stage)
|
||||
self._set_status(metadata, "accepted", "Chapitre accepté et mémoire mise à jour.")
|
||||
metadata["finished_at"] = self._timestamp()
|
||||
metadata["memory_update"] = memory_update.to_dict()
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
|
||||
return GenerationOutcome(
|
||||
chapter_id=chapter_id,
|
||||
accepted=True,
|
||||
status="accepted",
|
||||
draft_path=current_candidate_path,
|
||||
critique_path=context.critique_path,
|
||||
gate_path=context.gate_path,
|
||||
meta_path=context.meta_path,
|
||||
manuscript_path=context.manuscript_path,
|
||||
quality_blockers=[],
|
||||
)
|
||||
except ProviderError as exc:
|
||||
failed_stage = self._current_running_stage(metadata, current_stage)
|
||||
self._set_status(metadata, "failed", f"Échec à l'étape {failed_stage}: {exc}")
|
||||
metadata["failed_stage"] = failed_stage
|
||||
metadata["error"] = str(exc)
|
||||
metadata["finished_at"] = self._timestamp()
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
raise
|
||||
except ValueError as exc:
|
||||
failed_stage = self._current_running_stage(metadata, current_stage)
|
||||
self._set_status(metadata, "failed", f"Échec à l'étape {failed_stage}: {exc}")
|
||||
metadata["failed_stage"] = failed_stage
|
||||
metadata["error"] = str(exc)
|
||||
metadata["finished_at"] = self._timestamp()
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
raise ProviderError(str(exc)) from exc
|
||||
|
||||
def _build_context(self, chapter_id: ChapterId) -> GenerationContext:
|
||||
intention_path = self.intention_gate.resolve_intention_path(chapter_id)
|
||||
if intention_path is None:
|
||||
raise RuntimeError(f"Aucune intention trouvée pour {chapter_id.slug}.")
|
||||
|
||||
structure_path = resolve_chapter_file(self.root / "structure" / "chapitres", chapter_id)
|
||||
manuscript_path = resolve_chapter_file(self.root / "manuscrit", chapter_id)
|
||||
memory_summary_path = resolve_chapter_file(self.root / "memoire" / "chapitres", chapter_id)
|
||||
draft_dir = self.root / "brouillons" / "chapitres" / chapter_id.slug
|
||||
|
||||
return GenerationContext(
|
||||
root=self.root,
|
||||
chapter_id=chapter_id,
|
||||
intention_path=intention_path,
|
||||
intention_text=intention_path.read_text(encoding="utf-8").strip(),
|
||||
structure_path=structure_path,
|
||||
draft_dir=draft_dir,
|
||||
draft_v1_path=draft_dir / "draft_v1.md",
|
||||
critique_path=draft_dir / "critique_v1.md",
|
||||
draft_v2_path=draft_dir / "draft_v2.md",
|
||||
gate_path=draft_dir / "gate_v1.json",
|
||||
meta_path=draft_dir / "meta.json",
|
||||
manuscript_path=manuscript_path,
|
||||
memory_summary_path=memory_summary_path,
|
||||
memory_index_dir=self.root / "memoire" / "index",
|
||||
story_context=self._build_story_context(chapter_id),
|
||||
)
|
||||
|
||||
def _build_story_context(self, chapter_id: ChapterId) -> str:
|
||||
sections: list[str] = []
|
||||
|
||||
previous_manuscript = self._latest_existing_file(self.root / "manuscrit", chapter_id.number - 1)
|
||||
if previous_manuscript is not None:
|
||||
sections.append(
|
||||
"## Dernier chapitre accepté\n"
|
||||
f"Fichier: {previous_manuscript.name}\n"
|
||||
f"{self._read_excerpt(previous_manuscript)}"
|
||||
)
|
||||
|
||||
previous_memory = self._latest_existing_file(self.root / "memoire" / "chapitres", chapter_id.number - 1)
|
||||
if previous_memory is not None:
|
||||
sections.append(
|
||||
"## Dernier résumé mémoire\n"
|
||||
f"Fichier: {previous_memory.name}\n"
|
||||
f"{self._read_excerpt(previous_memory)}"
|
||||
)
|
||||
|
||||
for name in ("personnages.json", "lieux.json", "chronologie.json"):
|
||||
path = self.root / "memoire" / "index" / name
|
||||
if path.exists():
|
||||
sections.append(f"## {name}\n{self._read_excerpt(path, limit=2000)}")
|
||||
|
||||
if not sections:
|
||||
return "Aucun contexte projet disponible."
|
||||
|
||||
return "\n\n".join(sections)
|
||||
|
||||
def _latest_existing_file(self, directory: Path, max_number: int) -> Path | None:
|
||||
if max_number <= 0 or not directory.exists():
|
||||
return None
|
||||
|
||||
candidates: list[Path] = []
|
||||
for number in range(max_number, 0, -1):
|
||||
path = resolve_chapter_file(directory, ChapterId(number))
|
||||
if path.exists():
|
||||
candidates.append(path)
|
||||
break
|
||||
return candidates[0] if candidates else None
|
||||
|
||||
def _read_excerpt(self, path: Path, limit: int = 4000) -> str:
|
||||
text = path.read_text(encoding="utf-8").strip()
|
||||
if len(text) <= limit:
|
||||
return text
|
||||
return f"{text[:limit].rstrip()}\n[...]"
|
||||
|
||||
def _generate_structure(
|
||||
self,
|
||||
provider: GenerationProvider,
|
||||
context: GenerationContext,
|
||||
metadata: dict[str, object],
|
||||
) -> StructurePlan:
|
||||
prompt = self.prompt_store.render(
|
||||
"structure",
|
||||
chapter_slug=context.chapter_id.slug,
|
||||
intention=context.intention_text,
|
||||
story_context=context.story_context,
|
||||
)
|
||||
self._begin_stage(
|
||||
metadata,
|
||||
context.meta_path,
|
||||
"structure",
|
||||
"Génération de la structure en cours.",
|
||||
)
|
||||
response = provider.generate(GenerationRequest(stage="structure", prompt=prompt))
|
||||
markdown = response.content.strip()
|
||||
if not markdown:
|
||||
raise ProviderError("Le provider a renvoyé une structure vide.")
|
||||
return StructurePlan(chapter_id=context.chapter_id, markdown=f"{markdown}\n")
|
||||
|
||||
def _generate_draft(
|
||||
self,
|
||||
provider: GenerationProvider,
|
||||
context: GenerationContext,
|
||||
structure_plan: StructurePlan,
|
||||
metadata: dict[str, object],
|
||||
) -> str:
|
||||
prompt = self.prompt_store.render(
|
||||
"draft",
|
||||
chapter_slug=context.chapter_id.slug,
|
||||
intention=context.intention_text,
|
||||
structure_markdown=structure_plan.markdown,
|
||||
story_context=context.story_context,
|
||||
)
|
||||
self._begin_stage(
|
||||
metadata,
|
||||
context.meta_path,
|
||||
"draft",
|
||||
"Génération du brouillon initial en cours.",
|
||||
)
|
||||
response = provider.generate(GenerationRequest(stage="draft", prompt=prompt, temperature=0.4))
|
||||
draft = response.content.strip()
|
||||
if not draft:
|
||||
raise ProviderError("Le provider a renvoyé un brouillon vide.")
|
||||
return f"{draft}\n"
|
||||
|
||||
def _generate_control_report(
|
||||
self,
|
||||
provider: GenerationProvider,
|
||||
context: GenerationContext,
|
||||
structure_plan: StructurePlan,
|
||||
draft_v1: str,
|
||||
metadata: dict[str, object],
|
||||
) -> ControlReport:
|
||||
prompt = self.prompt_store.render(
|
||||
"critique",
|
||||
chapter_slug=context.chapter_id.slug,
|
||||
intention=context.intention_text,
|
||||
structure_markdown=structure_plan.markdown,
|
||||
draft_markdown=draft_v1,
|
||||
)
|
||||
return self._generate_json_payload(
|
||||
provider=provider,
|
||||
stage="critique",
|
||||
prompt=prompt,
|
||||
retry_prompt_name="critique_retry",
|
||||
parse_response=ControlReport.from_response_text,
|
||||
metadata=metadata,
|
||||
meta_path=context.meta_path,
|
||||
retry_context={
|
||||
"chapter_slug": context.chapter_id.slug,
|
||||
"intention": context.intention_text,
|
||||
"structure_markdown": structure_plan.markdown,
|
||||
"draft_markdown": draft_v1,
|
||||
},
|
||||
)
|
||||
|
||||
def _rewrite_draft(
|
||||
self,
|
||||
provider: GenerationProvider,
|
||||
context: GenerationContext,
|
||||
structure_plan: StructurePlan,
|
||||
draft_v1: str,
|
||||
control_report: ControlReport,
|
||||
metadata: dict[str, object],
|
||||
) -> str:
|
||||
prompt = self.prompt_store.render(
|
||||
"rewrite",
|
||||
chapter_slug=context.chapter_id.slug,
|
||||
intention=context.intention_text,
|
||||
structure_markdown=structure_plan.markdown,
|
||||
draft_markdown=draft_v1,
|
||||
critique_json=control_report.to_dict(),
|
||||
)
|
||||
self._begin_stage(
|
||||
metadata,
|
||||
context.meta_path,
|
||||
"rewrite",
|
||||
"Réécriture guidée par la critique en cours.",
|
||||
)
|
||||
response = provider.generate(GenerationRequest(stage="rewrite", prompt=prompt, temperature=0.3))
|
||||
draft = response.content.strip()
|
||||
if not draft:
|
||||
raise ProviderError("Le provider a renvoyé une réécriture vide.")
|
||||
return f"{draft}\n"
|
||||
|
||||
def _repair_until_ready(
|
||||
self,
|
||||
*,
|
||||
provider: GenerationProvider,
|
||||
context: GenerationContext,
|
||||
structure_plan: StructurePlan,
|
||||
current_candidate_text: str,
|
||||
current_candidate_path: Path,
|
||||
gate_report: ManuscriptGateReport,
|
||||
metadata: dict[str, object],
|
||||
) -> tuple[str, Path, ManuscriptGateReport]:
|
||||
for attempt in range(1, self._repair_max_passes() + 1):
|
||||
repair_model = self._repair_model_for_attempt(provider, attempt)
|
||||
repair_provider = self._provider_for_repair(provider, repair_model)
|
||||
repaired_text = self._repair_draft(
|
||||
provider=repair_provider,
|
||||
context=context,
|
||||
structure_plan=structure_plan,
|
||||
current_candidate=current_candidate_text,
|
||||
gate_report=gate_report,
|
||||
metadata=metadata,
|
||||
attempt=attempt,
|
||||
repair_model=repair_model,
|
||||
)
|
||||
repair_path = context.repair_path(attempt)
|
||||
self._write_text(repair_path, repaired_text)
|
||||
self._complete_stage(metadata, "repair")
|
||||
self._record_repair_attempt(metadata, attempt=attempt, model=self._provider_model_name(repair_provider) or repair_model, path=repair_path)
|
||||
self._set_status(
|
||||
metadata,
|
||||
"repair_ready",
|
||||
f"Réparation prose v{attempt} générée, nouveau contrôle manuscrit en cours.",
|
||||
)
|
||||
metadata["draft_final"] = str(repair_path)
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
|
||||
current_candidate_text = repaired_text
|
||||
current_candidate_path = repair_path
|
||||
gate_report = self._generate_manuscript_gate_report(
|
||||
repair_provider,
|
||||
context,
|
||||
structure_plan,
|
||||
current_candidate_text,
|
||||
metadata,
|
||||
)
|
||||
self._persist_gate_report(metadata, context, gate_report, current_candidate_path)
|
||||
self._complete_stage(metadata, "gate")
|
||||
self._write_metadata(context.meta_path, metadata)
|
||||
if gate_report.ready_for_manuscript:
|
||||
break
|
||||
|
||||
return current_candidate_text, current_candidate_path, gate_report
|
||||
|
||||
def _repair_draft(
|
||||
self,
|
||||
*,
|
||||
provider: GenerationProvider,
|
||||
context: GenerationContext,
|
||||
structure_plan: StructurePlan,
|
||||
current_candidate: str,
|
||||
gate_report: ManuscriptGateReport,
|
||||
metadata: dict[str, object],
|
||||
attempt: int,
|
||||
repair_model: str | None,
|
||||
) -> str:
|
||||
prompt = self.prompt_store.render(
|
||||
"repair",
|
||||
chapter_slug=context.chapter_id.slug,
|
||||
intention=context.intention_text,
|
||||
structure_markdown=structure_plan.markdown,
|
||||
draft_markdown=current_candidate,
|
||||
gate_json=gate_report.to_dict(),
|
||||
repair_attempt=attempt,
|
||||
repair_model=repair_model or "",
|
||||
story_context=context.story_context,
|
||||
)
|
||||
model_label = repair_model or self._provider_model_name(provider) or "provider_courant"
|
||||
self._begin_stage(
|
||||
metadata,
|
||||
context.meta_path,
|
||||
"repair",
|
||||
f"Réparation prose v{attempt} en cours avec {model_label}.",
|
||||
)
|
||||
response = provider.generate(GenerationRequest(stage="repair", prompt=prompt, temperature=0.2))
|
||||
repaired = response.content.strip()
|
||||
if not repaired:
|
||||
raise ProviderError("Le provider a renvoyé une réparation vide.")
|
||||
return f"{repaired}\n"
|
||||
|
||||
def _generate_manuscript_gate_report(
|
||||
self,
|
||||
provider: GenerationProvider,
|
||||
context: GenerationContext,
|
||||
structure_plan: StructurePlan,
|
||||
draft_v2: str,
|
||||
metadata: dict[str, object],
|
||||
) -> ManuscriptGateReport:
|
||||
self._begin_stage(
|
||||
metadata,
|
||||
context.meta_path,
|
||||
"gate",
|
||||
"Contrôle manuscrit en cours.",
|
||||
)
|
||||
heuristic_report = self._heuristic_gate_report(draft_v2)
|
||||
if heuristic_report is not None:
|
||||
metadata["last_status_message"] = heuristic_report.summary
|
||||
return heuristic_report
|
||||
|
||||
prompt = self.prompt_store.render(
|
||||
"gate",
|
||||
chapter_slug=context.chapter_id.slug,
|
||||
intention=context.intention_text,
|
||||
structure_markdown=structure_plan.markdown,
|
||||
draft_markdown=draft_v2,
|
||||
)
|
||||
return self._generate_json_payload(
|
||||
provider=provider,
|
||||
stage="gate",
|
||||
prompt=prompt,
|
||||
retry_prompt_name="gate_retry",
|
||||
parse_response=ManuscriptGateReport.from_response_text,
|
||||
metadata=metadata,
|
||||
meta_path=context.meta_path,
|
||||
retry_context={
|
||||
"chapter_slug": context.chapter_id.slug,
|
||||
"intention": context.intention_text,
|
||||
"structure_markdown": structure_plan.markdown,
|
||||
"draft_markdown": draft_v2,
|
||||
},
|
||||
begin_stage=False,
|
||||
)
|
||||
|
||||
def _persist_gate_report(
|
||||
self,
|
||||
metadata: dict[str, object],
|
||||
context: GenerationContext,
|
||||
gate_report: ManuscriptGateReport,
|
||||
draft_path: Path,
|
||||
) -> None:
|
||||
self._write_json(context.gate_path, gate_report.to_dict())
|
||||
metadata["gate_report"] = gate_report.to_dict()
|
||||
metadata["quality_blockers"] = gate_report.all_blockers()
|
||||
metadata["draft_final"] = str(draft_path)
|
||||
|
||||
def _provider_for_repair(self, provider: GenerationProvider, model: str | None) -> GenerationProvider:
|
||||
if not model:
|
||||
return provider
|
||||
return clone_provider_with_model(provider, model)
|
||||
|
||||
def _repair_max_passes(self) -> int:
|
||||
raw = os.environ.get("ANE_REPAIR_MAX_PASSES", "2").strip() or "2"
|
||||
try:
|
||||
value = int(raw)
|
||||
except ValueError as exc:
|
||||
raise ProviderError("ANE_REPAIR_MAX_PASSES doit être un entier positif.") from exc
|
||||
if value <= 0:
|
||||
raise ProviderError("ANE_REPAIR_MAX_PASSES doit être supérieur à zéro.")
|
||||
return value
|
||||
|
||||
def _repair_model_for_attempt(self, provider: GenerationProvider, attempt: int) -> str | None:
|
||||
base_model = self._provider_model_name(provider)
|
||||
if attempt <= 1:
|
||||
return base_model
|
||||
|
||||
override = os.environ.get("ANE_REPAIR_FALLBACK_MODEL", "").strip()
|
||||
candidate = override or self._default_repair_fallback_model(base_model) or base_model
|
||||
if self._is_cross_apple_runtime_switch(base_model, candidate):
|
||||
raise ProviderError(
|
||||
"ANE_REPAIR_FALLBACK_MODEL ne peut pas viser un autre modèle apple-coreml pendant un même smoke. "
|
||||
"Relancer le runtime Apple sur le modèle cible ou utiliser un fallback non-Apple."
|
||||
)
|
||||
return candidate
|
||||
|
||||
def _default_repair_fallback_model(self, model: str | None) -> str | None:
|
||||
mapping = {
|
||||
"ollama:qwen2.5:1.5b": "ollama:qwen2.5:7b",
|
||||
"apple-coreml:qwen2.5-0.5b-instruct-onnx": "ollama:qwen2.5:7b",
|
||||
"apple-coreml:qwen3.5-4b-onnx-q4f16": "ollama:qwen2.5:7b",
|
||||
}
|
||||
if not model:
|
||||
return None
|
||||
return mapping.get(model)
|
||||
|
||||
def _is_cross_apple_runtime_switch(self, base_model: str | None, candidate: str | None) -> bool:
|
||||
if not base_model or not candidate:
|
||||
return False
|
||||
if base_model == candidate:
|
||||
return False
|
||||
return base_model.startswith("apple-coreml:") and candidate.startswith("apple-coreml:")
|
||||
|
||||
def _heuristic_gate_report(self, draft_v2: str) -> ManuscriptGateReport | None:
|
||||
blockers: list[str] = []
|
||||
recommendations: list[str] = []
|
||||
|
||||
if self._word_count(draft_v2) < 180:
|
||||
blockers.append("too_short")
|
||||
recommendations.append("Allonger le chapitre pour produire une scene complete et continue.")
|
||||
|
||||
if self._has_truncated_ending(draft_v2):
|
||||
blockers.append("truncated_ending")
|
||||
recommendations.append("Terminer la scene sur une phrase complete avec une vraie fermeture.")
|
||||
|
||||
if self._is_outline_like(draft_v2):
|
||||
blockers.append("outline_like")
|
||||
recommendations.append("Remplacer les titres et puces par une narration continue en prose.")
|
||||
|
||||
if not blockers:
|
||||
return None
|
||||
|
||||
summary = "Le garde-fou manuscrit a bloque la promotion: " + ", ".join(blockers) + "."
|
||||
return ManuscriptGateReport.from_heuristics(
|
||||
blockers=blockers,
|
||||
recommendations=recommendations,
|
||||
summary=summary,
|
||||
)
|
||||
|
||||
def _word_count(self, text: str) -> int:
|
||||
return len(re.findall(r"[0-9A-Za-zÀ-ÖØ-öø-ÿ]+(?:[-'][0-9A-Za-zÀ-ÖØ-öø-ÿ]+)*", text))
|
||||
|
||||
def _has_truncated_ending(self, text: str) -> bool:
|
||||
for line in reversed(text.splitlines()):
|
||||
stripped = line.strip()
|
||||
if not stripped:
|
||||
continue
|
||||
return not stripped.endswith((".", "!", "?", "…", "»", '"', "'"))
|
||||
return True
|
||||
|
||||
def _is_outline_like(self, text: str) -> bool:
|
||||
detected_markers: set[str] = set()
|
||||
for line in text.splitlines():
|
||||
stripped = line.strip()
|
||||
if not stripped:
|
||||
continue
|
||||
if stripped.startswith("## "):
|
||||
detected_markers.add("heading_level_2")
|
||||
if stripped.startswith("### "):
|
||||
detected_markers.add("heading_level_3")
|
||||
if stripped.startswith("- "):
|
||||
detected_markers.add("bullet_list")
|
||||
lowered = stripped.lower()
|
||||
if "**objectif**" in lowered or "**conflit**" in lowered or "**sortie**" in lowered:
|
||||
detected_markers.add("scene_fields")
|
||||
if "scène" in lowered or "scene" in lowered:
|
||||
detected_markers.add("scene_heading")
|
||||
if len(detected_markers) >= 2:
|
||||
return True
|
||||
return False
|
||||
|
||||
def _generate_memory_update(
|
||||
self,
|
||||
provider: GenerationProvider,
|
||||
context: GenerationContext,
|
||||
accepted_draft: str,
|
||||
metadata: dict[str, object],
|
||||
) -> MemoryUpdate:
|
||||
prompt = self.prompt_store.render(
|
||||
"memory",
|
||||
chapter_slug=context.chapter_id.slug,
|
||||
accepted_draft=accepted_draft,
|
||||
story_context=context.story_context,
|
||||
)
|
||||
return self._generate_json_payload(
|
||||
provider=provider,
|
||||
stage="memory",
|
||||
prompt=prompt,
|
||||
retry_prompt_name="memory_retry",
|
||||
parse_response=MemoryUpdate.from_response_text,
|
||||
metadata=metadata,
|
||||
meta_path=context.meta_path,
|
||||
retry_context={
|
||||
"chapter_slug": context.chapter_id.slug,
|
||||
"accepted_draft": accepted_draft,
|
||||
"story_context": context.story_context,
|
||||
},
|
||||
)
|
||||
|
||||
def _generate_json_payload(
|
||||
self,
|
||||
*,
|
||||
provider: GenerationProvider,
|
||||
stage: str,
|
||||
prompt: str,
|
||||
retry_prompt_name: str,
|
||||
parse_response: Callable[[str], ParsedStagePayload],
|
||||
metadata: dict[str, object],
|
||||
meta_path: Path,
|
||||
retry_context: dict[str, object],
|
||||
begin_stage: bool = True,
|
||||
) -> ParsedStagePayload:
|
||||
if begin_stage:
|
||||
self._begin_stage(
|
||||
metadata,
|
||||
meta_path,
|
||||
stage,
|
||||
f"Génération de l'étape {stage} en cours.",
|
||||
)
|
||||
first_response = provider.generate(
|
||||
GenerationRequest(stage=stage, prompt=prompt, response_format="json", temperature=0.1)
|
||||
)
|
||||
first_payload = first_response.content
|
||||
try:
|
||||
return parse_response(first_payload)
|
||||
except ValueError as first_error:
|
||||
self._mark_stage_attempt(
|
||||
metadata,
|
||||
stage,
|
||||
retry=True,
|
||||
message=f"Réessai JSON sur l'étape {stage} après une réponse invalide.",
|
||||
)
|
||||
self._set_status(metadata, f"{stage}_retrying", f"Réessai JSON sur l'étape {stage} en cours.")
|
||||
self._write_metadata(meta_path, metadata)
|
||||
retry_prompt = self.prompt_store.render(
|
||||
retry_prompt_name,
|
||||
parse_error=str(first_error),
|
||||
invalid_response=self._truncate_retry_payload(first_payload),
|
||||
**retry_context,
|
||||
)
|
||||
retry_response = provider.generate(
|
||||
GenerationRequest(stage=stage, prompt=retry_prompt, response_format="json", temperature=0.0)
|
||||
)
|
||||
try:
|
||||
return parse_response(retry_response.content)
|
||||
except ValueError as second_error:
|
||||
raise ProviderError(
|
||||
f"Le provider a renvoyé un JSON invalide pendant l'étape '{stage}' après deux tentatives: "
|
||||
f"première erreur: {first_error}; seconde erreur: {second_error}"
|
||||
) from second_error
|
||||
|
||||
def _truncate_retry_payload(self, payload: str, limit: int = 2000) -> str:
|
||||
text = payload.strip()
|
||||
if len(text) <= limit:
|
||||
return text
|
||||
return f"{text[:limit].rstrip()}\n[...]"
|
||||
|
||||
def _persist_memory(self, context: GenerationContext, memory_update: MemoryUpdate) -> None:
|
||||
self._write_text(
|
||||
context.memory_summary_path,
|
||||
f"# Mémoire — {context.chapter_id.slug}\n\n{memory_update.chapter_summary}\n",
|
||||
)
|
||||
|
||||
self._merge_index_records(
|
||||
context.memory_index_dir / "personnages.json",
|
||||
context.chapter_id,
|
||||
memory_update.characters,
|
||||
label_key="name",
|
||||
)
|
||||
self._merge_index_records(
|
||||
context.memory_index_dir / "lieux.json",
|
||||
context.chapter_id,
|
||||
memory_update.locations,
|
||||
label_key="name",
|
||||
)
|
||||
self._merge_timeline_records(
|
||||
context.memory_index_dir / "chronologie.json",
|
||||
context.chapter_id,
|
||||
memory_update.timeline_events,
|
||||
)
|
||||
|
||||
def _merge_index_records(
|
||||
self,
|
||||
path: Path,
|
||||
chapter_id: ChapterId,
|
||||
records: list[dict[str, str]],
|
||||
label_key: str,
|
||||
) -> None:
|
||||
existing: dict[str, dict[str, object]] = {}
|
||||
if path.exists():
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
if isinstance(payload, dict):
|
||||
existing = payload
|
||||
|
||||
for record in records:
|
||||
label = record[label_key]
|
||||
current = existing.get(label, {"chapters": []})
|
||||
chapters = list(current.get("chapters", []))
|
||||
if chapter_id.slug not in chapters:
|
||||
chapters.append(chapter_id.slug)
|
||||
|
||||
merged = dict(current)
|
||||
merged.update(record)
|
||||
merged["chapters"] = sorted(chapters)
|
||||
existing[label] = merged
|
||||
|
||||
self._write_json(path, existing)
|
||||
|
||||
def _merge_timeline_records(
|
||||
self,
|
||||
path: Path,
|
||||
chapter_id: ChapterId,
|
||||
records: list[dict[str, str]],
|
||||
) -> None:
|
||||
existing: list[dict[str, str]] = []
|
||||
if path.exists():
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
if isinstance(payload, list):
|
||||
existing = [
|
||||
{str(key): str(value) for key, value in item.items()}
|
||||
for item in payload
|
||||
if isinstance(item, dict)
|
||||
]
|
||||
|
||||
for record in records:
|
||||
merged = dict(record)
|
||||
merged["chapter"] = chapter_id.slug
|
||||
existing.append(merged)
|
||||
|
||||
self._write_json(path, existing)
|
||||
|
||||
def _prompt_for_acceptance(self, control_report: ControlReport, draft_path: Path) -> bool:
|
||||
self.output_func("")
|
||||
self.output_func(f"Critique: {control_report.summary}")
|
||||
self.output_func(f"Brouillon final: {draft_path}")
|
||||
self.output_func(f"Écarts détectés: {len(control_report.deviations)}")
|
||||
response = self.input_func("Promouvoir ce brouillon vers le manuscrit ? [y/N]: ").strip().lower()
|
||||
return response in {"y", "yes", "o", "oui"}
|
||||
|
||||
def _initial_metadata(self, context: GenerationContext, provider: GenerationProvider) -> dict[str, object]:
|
||||
return {
|
||||
"chapter": context.chapter_id.slug,
|
||||
"started_at": self._timestamp(),
|
||||
"status": "started",
|
||||
"last_status_message": "Pipeline initialisé.",
|
||||
"completed_stages": [],
|
||||
"accepted": False,
|
||||
"repair_attempts": 0,
|
||||
"repair_models": [],
|
||||
"stage_attempts": {},
|
||||
"retry_stages": [],
|
||||
"quality_blockers": [],
|
||||
"provider": self._provider_metadata(provider),
|
||||
"artifacts": {
|
||||
"intention": str(context.intention_path),
|
||||
"structure": str(context.structure_path),
|
||||
"draft_v1": str(context.draft_v1_path),
|
||||
"critique_v1": str(context.critique_path),
|
||||
"draft_v2": str(context.draft_v2_path),
|
||||
"repair_latest": None,
|
||||
"repairs": [],
|
||||
"gate_v1": str(context.gate_path),
|
||||
"manuscript": str(context.manuscript_path),
|
||||
"memory_summary": str(context.memory_summary_path),
|
||||
},
|
||||
}
|
||||
|
||||
def _provider_metadata(self, provider: GenerationProvider) -> dict[str, object]:
|
||||
snapshot = {
|
||||
"kind": provider.__class__.__name__,
|
||||
"base_url": None,
|
||||
"model": None,
|
||||
}
|
||||
if isinstance(provider, OpenAICompatibleProvider):
|
||||
snapshot["base_url"] = provider.config.base_url
|
||||
snapshot["model"] = provider.config.model
|
||||
return snapshot
|
||||
|
||||
def _current_running_stage(self, metadata: dict[str, object], fallback: str) -> str:
|
||||
status = str(metadata.get("status", "")).strip()
|
||||
for suffix in ("_running", "_retrying"):
|
||||
if status.endswith(suffix):
|
||||
return status[: -len(suffix)]
|
||||
return fallback
|
||||
|
||||
def _provider_model_name(self, provider: GenerationProvider) -> str | None:
|
||||
metadata = self._provider_metadata(provider)
|
||||
model = metadata.get("model")
|
||||
if not isinstance(model, str):
|
||||
return None
|
||||
return model.strip() or None
|
||||
|
||||
def _complete_stage(self, metadata: dict[str, object], stage: str) -> None:
|
||||
completed = metadata.setdefault("completed_stages", [])
|
||||
if not isinstance(completed, list):
|
||||
completed = []
|
||||
metadata["completed_stages"] = completed
|
||||
if stage not in completed:
|
||||
completed.append(stage)
|
||||
|
||||
def _record_repair_attempt(
|
||||
self,
|
||||
metadata: dict[str, object],
|
||||
*,
|
||||
attempt: int,
|
||||
model: str | None,
|
||||
path: Path,
|
||||
) -> None:
|
||||
metadata["repair_attempts"] = attempt
|
||||
|
||||
repair_models = metadata.setdefault("repair_models", [])
|
||||
if not isinstance(repair_models, list):
|
||||
repair_models = []
|
||||
metadata["repair_models"] = repair_models
|
||||
repair_models.append(model or "provider_courant")
|
||||
|
||||
artifacts = metadata.setdefault("artifacts", {})
|
||||
if not isinstance(artifacts, dict):
|
||||
artifacts = {}
|
||||
metadata["artifacts"] = artifacts
|
||||
repairs = artifacts.setdefault("repairs", [])
|
||||
if not isinstance(repairs, list):
|
||||
repairs = []
|
||||
artifacts["repairs"] = repairs
|
||||
repairs.append(str(path))
|
||||
artifacts["repair_latest"] = str(path)
|
||||
|
||||
def _set_status(self, metadata: dict[str, object], status: str, message: str) -> None:
|
||||
metadata["status"] = status
|
||||
metadata["last_status_message"] = message
|
||||
|
||||
def _mark_stage_attempt(
|
||||
self,
|
||||
metadata: dict[str, object],
|
||||
stage: str,
|
||||
*,
|
||||
retry: bool = False,
|
||||
message: str | None = None,
|
||||
) -> None:
|
||||
stage_attempts = metadata.setdefault("stage_attempts", {})
|
||||
if not isinstance(stage_attempts, dict):
|
||||
stage_attempts = {}
|
||||
metadata["stage_attempts"] = stage_attempts
|
||||
stage_attempts[stage] = int(stage_attempts.get(stage, 0)) + 1
|
||||
|
||||
if retry:
|
||||
retry_stages = metadata.setdefault("retry_stages", [])
|
||||
if isinstance(retry_stages, list) and stage not in retry_stages:
|
||||
retry_stages.append(stage)
|
||||
|
||||
if message:
|
||||
metadata["last_status_message"] = message
|
||||
|
||||
def _begin_stage(
|
||||
self,
|
||||
metadata: dict[str, object],
|
||||
meta_path: Path,
|
||||
stage: str,
|
||||
message: str,
|
||||
) -> None:
|
||||
self._mark_stage_attempt(metadata, stage)
|
||||
self._set_status(metadata, f"{stage}_running", message)
|
||||
self._write_metadata(meta_path, metadata)
|
||||
|
||||
def _write_metadata(self, path: Path, metadata: dict[str, object]) -> None:
|
||||
self._write_json(path, metadata)
|
||||
|
||||
def _write_json(self, path: Path, payload: object) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(payload, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
def _write_text(self, path: Path, content: str) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(content, encoding="utf-8")
|
||||
|
||||
def _timestamp(self) -> str:
|
||||
return datetime.now(timezone.utc).replace(microsecond=0).isoformat()
|
||||
@@ -0,0 +1,254 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from dataclasses import dataclass, replace
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
from typing import Mapping
|
||||
from urllib import error, request
|
||||
|
||||
|
||||
class ProviderError(RuntimeError):
|
||||
"""Raised when a text generation provider fails."""
|
||||
|
||||
|
||||
class ProviderConfigurationError(ProviderError):
|
||||
"""Raised when the provider environment is incomplete."""
|
||||
|
||||
|
||||
STAGE_MAX_TOKENS_ENV = {
|
||||
"structure": "ANE_MAX_TOKENS_STRUCTURE",
|
||||
"draft": "ANE_MAX_TOKENS_DRAFT",
|
||||
"critique": "ANE_MAX_TOKENS_CRITIQUE",
|
||||
"rewrite": "ANE_MAX_TOKENS_REWRITE",
|
||||
"gate": "ANE_MAX_TOKENS_GATE",
|
||||
"repair": "ANE_MAX_TOKENS_REPAIR",
|
||||
"memory": "ANE_MAX_TOKENS_MEMORY",
|
||||
}
|
||||
|
||||
|
||||
def _parse_positive_int(raw_value: str, *, env_name: str) -> int:
|
||||
try:
|
||||
value = int(raw_value)
|
||||
except ValueError as exc:
|
||||
raise ProviderConfigurationError(f"{env_name} doit être un entier.") from exc
|
||||
if value <= 0:
|
||||
raise ProviderConfigurationError(f"{env_name} doit être supérieur à zéro.")
|
||||
return value
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ProviderConfig:
|
||||
provider: str
|
||||
base_url: str
|
||||
api_key: str
|
||||
model: str
|
||||
timeout: float
|
||||
max_tokens: int
|
||||
stage_max_tokens: Mapping[str, int]
|
||||
|
||||
@classmethod
|
||||
def from_env(cls, env: Mapping[str, str] | None = None) -> "ProviderConfig":
|
||||
source = env or os.environ
|
||||
provider = source.get("ANE_PROVIDER", "openai_compatible").strip() or "openai_compatible"
|
||||
base_url = source.get("ANE_BASE_URL", "").strip()
|
||||
model = source.get("ANE_MODEL", "").strip()
|
||||
api_key = source.get("ANE_API_KEY", "").strip()
|
||||
timeout_value = source.get("ANE_TIMEOUT", "60").strip() or "60"
|
||||
max_tokens_value = source.get("ANE_MAX_TOKENS", "4096").strip() or "4096"
|
||||
|
||||
try:
|
||||
timeout = float(timeout_value)
|
||||
except ValueError as exc:
|
||||
raise ProviderConfigurationError("ANE_TIMEOUT doit être un nombre.") from exc
|
||||
|
||||
max_tokens = _parse_positive_int(max_tokens_value, env_name="ANE_MAX_TOKENS")
|
||||
stage_max_tokens: dict[str, int] = {}
|
||||
for stage_name, env_name in STAGE_MAX_TOKENS_ENV.items():
|
||||
raw_stage_value = source.get(env_name, "").strip()
|
||||
if not raw_stage_value:
|
||||
continue
|
||||
stage_max_tokens[stage_name] = _parse_positive_int(raw_stage_value, env_name=env_name)
|
||||
|
||||
return cls(
|
||||
provider=provider,
|
||||
base_url=base_url,
|
||||
api_key=api_key,
|
||||
model=model,
|
||||
timeout=timeout,
|
||||
max_tokens=max_tokens,
|
||||
stage_max_tokens=stage_max_tokens,
|
||||
)
|
||||
|
||||
def max_tokens_for_stage(self, stage: str, explicit: int | None = None) -> int:
|
||||
if explicit is not None:
|
||||
return explicit
|
||||
return self.stage_max_tokens.get(stage, self.max_tokens)
|
||||
|
||||
def with_model(self, model: str) -> "ProviderConfig":
|
||||
return replace(self, model=model)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GenerationRequest:
|
||||
stage: str
|
||||
prompt: str
|
||||
response_format: str = "text"
|
||||
temperature: float = 0.2
|
||||
system_prompt: str | None = None
|
||||
max_tokens: int | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GenerationResponse:
|
||||
content: str
|
||||
model: str | None = None
|
||||
raw: dict[str, object] | None = None
|
||||
|
||||
|
||||
class GenerationProvider(ABC):
|
||||
@abstractmethod
|
||||
def generate(self, request: GenerationRequest) -> GenerationResponse:
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
class OpenAICompatibleProvider(GenerationProvider):
|
||||
def __init__(self, config: ProviderConfig):
|
||||
if not config.base_url:
|
||||
raise ProviderConfigurationError("ANE_BASE_URL est requis pour le provider openai_compatible.")
|
||||
if not config.model:
|
||||
raise ProviderConfigurationError("ANE_MODEL est requis pour le provider openai_compatible.")
|
||||
self.config = config
|
||||
|
||||
def generate(self, prompt_request: GenerationRequest) -> GenerationResponse:
|
||||
payload: dict[str, object] = {
|
||||
"model": self.config.model,
|
||||
"messages": self._build_messages(prompt_request),
|
||||
"temperature": prompt_request.temperature,
|
||||
"max_tokens": self.config.max_tokens_for_stage(
|
||||
prompt_request.stage,
|
||||
prompt_request.max_tokens,
|
||||
),
|
||||
}
|
||||
if prompt_request.response_format == "json":
|
||||
payload["response_format"] = {"type": "json_object"}
|
||||
|
||||
body = json.dumps(payload).encode("utf-8")
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if self.config.api_key:
|
||||
headers["Authorization"] = f"Bearer {self.config.api_key}"
|
||||
|
||||
http_request = request.Request(
|
||||
self._chat_completions_url(),
|
||||
data=body,
|
||||
headers=headers,
|
||||
method="POST",
|
||||
)
|
||||
|
||||
try:
|
||||
with request.urlopen(http_request, timeout=self.config.timeout) as response:
|
||||
raw_payload = json.loads(response.read().decode("utf-8"))
|
||||
except error.HTTPError as exc:
|
||||
details = exc.read().decode("utf-8", errors="replace")
|
||||
raise ProviderError(
|
||||
f"Le provider a répondu avec HTTP {exc.code} pendant l'étape '{prompt_request.stage}': {details}"
|
||||
) from exc
|
||||
except error.URLError as exc:
|
||||
raise ProviderError(
|
||||
f"Impossible de joindre le provider pendant l'étape '{prompt_request.stage}': {exc.reason}"
|
||||
) from exc
|
||||
except (TimeoutError, socket.timeout) as exc:
|
||||
raise ProviderError(
|
||||
f"Timeout du provider pendant l'étape '{prompt_request.stage}' après {self.config.timeout:.0f}s."
|
||||
) from exc
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ProviderError(
|
||||
f"Réponse non JSON du provider pendant l'étape '{prompt_request.stage}'."
|
||||
) from exc
|
||||
|
||||
try:
|
||||
choice = raw_payload["choices"][0]
|
||||
message = choice["message"]["content"]
|
||||
except (KeyError, IndexError, TypeError) as exc:
|
||||
raise ProviderError(
|
||||
f"Réponse OpenAI-compatible invalide pendant l'étape '{prompt_request.stage}'."
|
||||
) from exc
|
||||
|
||||
content = self._normalize_message_content(message)
|
||||
return GenerationResponse(
|
||||
content=content,
|
||||
model=str(raw_payload.get("model", self.config.model)),
|
||||
raw=raw_payload,
|
||||
)
|
||||
|
||||
def _build_messages(self, prompt_request: GenerationRequest) -> list[dict[str, str]]:
|
||||
messages: list[dict[str, str]] = []
|
||||
if prompt_request.system_prompt:
|
||||
messages.append({"role": "system", "content": prompt_request.system_prompt})
|
||||
messages.append({"role": "user", "content": prompt_request.prompt})
|
||||
return messages
|
||||
|
||||
def _chat_completions_url(self) -> str:
|
||||
base = self.config.base_url.rstrip("/")
|
||||
if base.endswith("/chat/completions"):
|
||||
return base
|
||||
if base.endswith("/v1"):
|
||||
return f"{base}/chat/completions"
|
||||
return f"{base}/v1/chat/completions"
|
||||
|
||||
def _normalize_message_content(self, message: object) -> str:
|
||||
if isinstance(message, str):
|
||||
return message
|
||||
if isinstance(message, list):
|
||||
parts: list[str] = []
|
||||
for item in message:
|
||||
if isinstance(item, dict) and item.get("type") == "text":
|
||||
parts.append(str(item.get("text", "")))
|
||||
if parts:
|
||||
return "\n".join(parts)
|
||||
raise ProviderError("Le provider n'a pas renvoyé de contenu texte exploitable.")
|
||||
|
||||
|
||||
class MockGenerationProvider(GenerationProvider):
|
||||
def __init__(self, responses: Mapping[str, object]):
|
||||
self._responses = {
|
||||
stage: list(value) if isinstance(value, list) else [value]
|
||||
for stage, value in responses.items()
|
||||
}
|
||||
self.requests: list[GenerationRequest] = []
|
||||
|
||||
def generate(self, prompt_request: GenerationRequest) -> GenerationResponse:
|
||||
self.requests.append(prompt_request)
|
||||
queue = self._responses.get(prompt_request.stage)
|
||||
if not queue:
|
||||
raise ProviderError(f"Aucune réponse mock configurée pour l'étape '{prompt_request.stage}'.")
|
||||
|
||||
next_value = queue.pop(0)
|
||||
if isinstance(next_value, Exception):
|
||||
raise next_value
|
||||
if isinstance(next_value, (dict, list)):
|
||||
content = json.dumps(next_value, ensure_ascii=False)
|
||||
else:
|
||||
content = str(next_value)
|
||||
|
||||
return GenerationResponse(content=content, model="mock")
|
||||
|
||||
|
||||
def build_provider_from_env(env: Mapping[str, str] | None = None) -> GenerationProvider:
|
||||
config = ProviderConfig.from_env(env)
|
||||
if config.provider != "openai_compatible":
|
||||
raise ProviderConfigurationError(
|
||||
f"Provider non supporté: {config.provider}. Utilisez ANE_PROVIDER=openai_compatible."
|
||||
)
|
||||
return OpenAICompatibleProvider(config)
|
||||
|
||||
|
||||
def clone_provider_with_model(provider: GenerationProvider, model: str) -> GenerationProvider:
|
||||
if not model:
|
||||
return provider
|
||||
if isinstance(provider, OpenAICompatibleProvider):
|
||||
if provider.config.model == model:
|
||||
return provider
|
||||
return OpenAICompatibleProvider(provider.config.with_model(model))
|
||||
return provider
|
||||
Binary file not shown.
+37
-6
@@ -1,5 +1,7 @@
|
||||
from pathlib import Path
|
||||
|
||||
from core.chapters import ChapterId, collect_matching_chapter_files
|
||||
|
||||
class IntentionGate:
|
||||
"""
|
||||
Hard lock: blocks any generation if no explicit intention exists.
|
||||
@@ -8,17 +10,46 @@ class IntentionGate:
|
||||
def __init__(self, project_root: Path):
|
||||
self.intentions_dir = project_root / "notes" / "intentions"
|
||||
|
||||
def has_intention(self) -> bool:
|
||||
def has_intention(self, chapter: object | None = None) -> bool:
|
||||
if not self.intentions_dir.exists():
|
||||
return False
|
||||
intentions = list(self.intentions_dir.glob("chapitre_*.md"))
|
||||
if chapter is None:
|
||||
intentions = list(self.intentions_dir.glob("chapitre_*.md"))
|
||||
return len(intentions) > 0
|
||||
chapter_id = ChapterId.parse(chapter)
|
||||
intentions = collect_matching_chapter_files(self.intentions_dir, chapter_id)
|
||||
return len(intentions) > 0
|
||||
|
||||
def assert_intention(self):
|
||||
if not self.has_intention():
|
||||
def resolve_intention_path(self, chapter: object) -> Path | None:
|
||||
chapter_id = ChapterId.parse(chapter)
|
||||
matches = collect_matching_chapter_files(self.intentions_dir, chapter_id)
|
||||
if not matches:
|
||||
return None
|
||||
if len(matches) > 1:
|
||||
from core.chapters import ChapterConflictError
|
||||
|
||||
raise ChapterConflictError(chapter_id, matches)
|
||||
return matches[0]
|
||||
|
||||
def load_intention(self, chapter: object) -> str:
|
||||
path = self.resolve_intention_path(chapter)
|
||||
if path is None:
|
||||
chapter_id = ChapterId.parse(chapter)
|
||||
raise RuntimeError(f"Aucune intention trouvée pour {chapter_id.slug}.")
|
||||
return path.read_text(encoding="utf-8").strip()
|
||||
|
||||
def assert_intention(self, chapter: object | None = None):
|
||||
if not self.has_intention(chapter):
|
||||
if chapter is None:
|
||||
raise RuntimeError(
|
||||
"Aucune intention trouvée.\n"
|
||||
"L'écriture est volontairement bloquée.\n"
|
||||
"Créez d'abord une intention explicite (CLI: intention create)."
|
||||
)
|
||||
|
||||
chapter_id = ChapterId.parse(chapter)
|
||||
raise RuntimeError(
|
||||
"Aucune intention trouvée.\n"
|
||||
f"Aucune intention trouvée pour {chapter_id.slug}.\n"
|
||||
"L'écriture est volontairement bloquée.\n"
|
||||
"Créez d'abord une intention explicite (CLI: intention create)."
|
||||
)
|
||||
|
||||
|
||||
@@ -0,0 +1,952 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import time
|
||||
import tomllib
|
||||
from typing import Any, Callable, Iterable
|
||||
from urllib import error, request
|
||||
|
||||
from core.chapters import ChapterId
|
||||
from core.project.loader import ProjectState
|
||||
|
||||
|
||||
AUTO_SYNC_TODO_ACTIVE = "ANE-TODO-ACTIVE"
|
||||
AUTO_SYNC_TODO_DONE = "ANE-TODO-DONE"
|
||||
AUTO_SYNC_PLAN = "ANE-PLAN"
|
||||
AUTO_SYNC_COMPARISON = "ANE-COMPARISON"
|
||||
AUTO_SYNC_README = "ANE-README"
|
||||
AUTO_SYNC_RUNBOOK = "ANE-RUNBOOK"
|
||||
AUTO_SYNC_MASCARADE_TODO = "MASCARADE-TODO"
|
||||
AUTO_SYNC_MASCARADE_PLAN = "MASCARADE-PLAN"
|
||||
AUTO_SYNC_MASCARADE_README = "MASCARADE-README"
|
||||
AUTO_SYNC_MASCARADE_RUNBOOK = "MASCARADE-RUNBOOK"
|
||||
|
||||
|
||||
class NextLotsError(RuntimeError):
|
||||
"""Raised when the orchestration flow cannot continue automatically."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TrackingPaths:
|
||||
ane_todo_active: Path
|
||||
ane_todo_done: Path
|
||||
ane_plan: Path
|
||||
ane_comparison: Path
|
||||
ane_readme: Path
|
||||
ane_runbook: Path
|
||||
mascarade_repo: Path
|
||||
mascarade_todo: Path
|
||||
mascarade_plan: Path
|
||||
mascarade_readme: Path
|
||||
mascarade_runbook: Path
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Manifest:
|
||||
repo_root: Path
|
||||
manifest_path: Path
|
||||
tracking: TrackingPaths
|
||||
core_base_url: str
|
||||
apple_runtime_url: str
|
||||
ollama_tags_url: str
|
||||
apple_model_ready_timeout_seconds: float
|
||||
apple_model_poll_interval_seconds: float
|
||||
smoke_chapter: str
|
||||
smoke_intention: str
|
||||
smoke_timeout_seconds: int
|
||||
preset_env: dict[str, str]
|
||||
required_apple_models: list[str]
|
||||
required_ollama_models: list[str]
|
||||
priority_models: list[str]
|
||||
baseline_models: list[str]
|
||||
preflight_only_models: list[str]
|
||||
next_code_lot: str
|
||||
|
||||
@classmethod
|
||||
def load(cls, repo_root: Path, manifest_path: Path) -> "Manifest":
|
||||
payload = tomllib.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
paths = payload["paths"]
|
||||
smoke = payload["smoke"]
|
||||
preset = payload["preset"]
|
||||
tracking = payload["tracking"]
|
||||
lots = payload["lots"]
|
||||
ensure_models = payload["ensure_models"]
|
||||
|
||||
mascarade_repo = Path(paths["mascarade_repo"]).expanduser()
|
||||
return cls(
|
||||
repo_root=repo_root,
|
||||
manifest_path=manifest_path,
|
||||
tracking=TrackingPaths(
|
||||
ane_todo_active=repo_root / tracking["ane"]["todo_active"],
|
||||
ane_todo_done=repo_root / tracking["ane"]["todo_done"],
|
||||
ane_plan=repo_root / tracking["ane"]["plan"],
|
||||
ane_comparison=repo_root / tracking["ane"]["comparison"],
|
||||
ane_readme=repo_root / tracking["ane"]["readme"],
|
||||
ane_runbook=repo_root / tracking["ane"]["runbook"],
|
||||
mascarade_repo=mascarade_repo,
|
||||
mascarade_todo=mascarade_repo / tracking["mascarade"]["todo"],
|
||||
mascarade_plan=mascarade_repo / tracking["mascarade"]["plan"],
|
||||
mascarade_readme=mascarade_repo / tracking["mascarade"]["readme"],
|
||||
mascarade_runbook=mascarade_repo / tracking["mascarade"]["runbook"],
|
||||
),
|
||||
core_base_url=str(paths["core_base_url"]).rstrip("/"),
|
||||
apple_runtime_url=str(paths["apple_runtime_url"]).rstrip("/"),
|
||||
ollama_tags_url=str(paths["ollama_tags_url"]).rstrip("/"),
|
||||
apple_model_ready_timeout_seconds=float(paths.get("apple_model_ready_timeout_seconds", 30)),
|
||||
apple_model_poll_interval_seconds=float(paths.get("apple_model_poll_interval_seconds", 2)),
|
||||
smoke_chapter=str(smoke["chapter"]),
|
||||
smoke_intention=str(smoke["intention"]),
|
||||
smoke_timeout_seconds=int(smoke["timeout_seconds"]),
|
||||
preset_env={str(key): str(value) for key, value in preset.items()},
|
||||
required_apple_models=[str(item) for item in ensure_models["apple_models"]],
|
||||
required_ollama_models=[str(item) for item in ensure_models["ollama_models"]],
|
||||
priority_models=[str(item) for item in lots["priority_models"]["models"]],
|
||||
baseline_models=[str(item) for item in lots["baselines"]["models"]],
|
||||
preflight_only_models=[str(item) for item in lots["preflight_only"]["models"]],
|
||||
next_code_lot=str(payload["next_actions"]["rewrite_compaction"]),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class CommandResult:
|
||||
args: list[str]
|
||||
returncode: int
|
||||
stdout: str
|
||||
stderr: str
|
||||
duration_seconds: float
|
||||
|
||||
|
||||
CommandRunner = Callable[[list[str], Path, dict[str, str] | None], CommandResult]
|
||||
JsonFetcher = Callable[[str, float], Any]
|
||||
|
||||
|
||||
@dataclass
|
||||
class ModelRunResult:
|
||||
model: str
|
||||
category: str
|
||||
classification: str = "pending"
|
||||
preflight_ok: bool | None = None
|
||||
preflight_duration_seconds: float | None = None
|
||||
smoke_attempted: bool = False
|
||||
smoke_duration_seconds: float | None = None
|
||||
status: str | None = None
|
||||
accepted: bool = False
|
||||
failed_stage: str | None = None
|
||||
quality_blockers: list[str] = field(default_factory=list)
|
||||
retry_stages: list[str] = field(default_factory=list)
|
||||
repair_attempts: int = 0
|
||||
repair_models: list[str] = field(default_factory=list)
|
||||
draft_path: str | None = None
|
||||
gate_path: str | None = None
|
||||
meta_path: str | None = None
|
||||
manuscript_path: str | None = None
|
||||
notes: list[str] = field(default_factory=list)
|
||||
preflight_log: str | None = None
|
||||
smoke_log: str | None = None
|
||||
workspace: str | None = None
|
||||
apple_model_active: str | None = None
|
||||
completed_stages: list[str] = field(default_factory=list)
|
||||
|
||||
def reached_gate(self) -> bool:
|
||||
return "gate" in self.completed_stages or (self.failed_stage == "gate")
|
||||
|
||||
|
||||
@dataclass
|
||||
class RunState:
|
||||
version: int
|
||||
manifest_path: str
|
||||
report_dir: str
|
||||
state_path: str
|
||||
lot: str
|
||||
started_at: str
|
||||
updated_at: str
|
||||
step_index: int
|
||||
model_index: int
|
||||
steps: list[dict[str, Any]]
|
||||
results: list[dict[str, Any]]
|
||||
notes: list[str]
|
||||
pending_manual_action: dict[str, Any] | None
|
||||
next_recommended_lot: str
|
||||
|
||||
@classmethod
|
||||
def new(cls, manifest: Manifest, lot: str, report_dir: Path, state_path: Path, steps: list[dict[str, Any]]) -> "RunState":
|
||||
now = _timestamp()
|
||||
return cls(
|
||||
version=1,
|
||||
manifest_path=str(manifest.manifest_path),
|
||||
report_dir=str(report_dir),
|
||||
state_path=str(state_path),
|
||||
lot=lot,
|
||||
started_at=now,
|
||||
updated_at=now,
|
||||
step_index=0,
|
||||
model_index=0,
|
||||
steps=steps,
|
||||
results=[],
|
||||
notes=[],
|
||||
pending_manual_action=None,
|
||||
next_recommended_lot=manifest.next_code_lot,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def load(cls, path: Path) -> "RunState":
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
return cls(**payload)
|
||||
|
||||
def dump(self, path: Path | None = None) -> None:
|
||||
target = path or Path(self.state_path)
|
||||
self.updated_at = _timestamp()
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_text(json.dumps(asdict(self), ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
def append_result(self, result: ModelRunResult) -> None:
|
||||
self.results.append(asdict(result))
|
||||
self.updated_at = _timestamp()
|
||||
|
||||
def typed_results(self) -> list[ModelRunResult]:
|
||||
return [ModelRunResult(**payload) for payload in self.results]
|
||||
|
||||
|
||||
def _timestamp() -> str:
|
||||
return datetime.now(timezone.utc).replace(microsecond=0).isoformat()
|
||||
|
||||
|
||||
def _default_command_runner(args: list[str], cwd: Path, env: dict[str, str] | None = None) -> CommandResult:
|
||||
merged_env = os.environ.copy()
|
||||
if env:
|
||||
merged_env.update(env)
|
||||
started = time.monotonic()
|
||||
completed = subprocess.run(
|
||||
args,
|
||||
cwd=str(cwd),
|
||||
env=merged_env,
|
||||
text=True,
|
||||
capture_output=True,
|
||||
check=False,
|
||||
)
|
||||
return CommandResult(
|
||||
args=args,
|
||||
returncode=completed.returncode,
|
||||
stdout=completed.stdout,
|
||||
stderr=completed.stderr,
|
||||
duration_seconds=time.monotonic() - started,
|
||||
)
|
||||
|
||||
|
||||
def _default_json_fetcher(url: str, timeout: float) -> Any:
|
||||
with request.urlopen(url, timeout=timeout) as response:
|
||||
return json.loads(response.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _auto_markers(name: str) -> tuple[str, str]:
|
||||
return (
|
||||
f"<!-- AUTO-SYNC:{name}:START -->",
|
||||
f"<!-- AUTO-SYNC:{name}:END -->",
|
||||
)
|
||||
|
||||
|
||||
def replace_auto_section(path: Path, marker_name: str, heading: str, body: str) -> None:
|
||||
start_marker, end_marker = _auto_markers(marker_name)
|
||||
text = path.read_text(encoding="utf-8") if path.exists() else ""
|
||||
section = f"{heading}\n{start_marker}\n{body.rstrip()}\n{end_marker}\n"
|
||||
if start_marker in text and end_marker in text:
|
||||
start = text.index(start_marker)
|
||||
end = text.index(end_marker) + len(end_marker)
|
||||
replacement_start = text.rfind("\n", 0, start)
|
||||
if replacement_start == -1:
|
||||
replacement_start = 0
|
||||
else:
|
||||
replacement_start += 1
|
||||
new_text = f"{text[:replacement_start]}{section}{text[end:].lstrip()}"
|
||||
else:
|
||||
suffix = "\n" if text.endswith("\n") else "\n\n"
|
||||
new_text = f"{text}{suffix}{section}"
|
||||
path.write_text(new_text, encoding="utf-8")
|
||||
|
||||
|
||||
class NextLotsRunner:
|
||||
def __init__(
|
||||
self,
|
||||
manifest: Manifest,
|
||||
*,
|
||||
command_runner: CommandRunner = _default_command_runner,
|
||||
json_fetcher: JsonFetcher = _default_json_fetcher,
|
||||
) -> None:
|
||||
self.manifest = manifest
|
||||
self.command_runner = command_runner
|
||||
self.json_fetcher = json_fetcher
|
||||
|
||||
def run(
|
||||
self,
|
||||
*,
|
||||
lot: str,
|
||||
resume_state: Path | None = None,
|
||||
dry_run: bool = False,
|
||||
report_only: bool = False,
|
||||
) -> int:
|
||||
state_path = self.manifest.repo_root / "automation" / "state" / "next_lots_state.json"
|
||||
if resume_state is not None:
|
||||
state = RunState.load(resume_state)
|
||||
else:
|
||||
report_dir = self._new_report_dir()
|
||||
state = RunState.new(
|
||||
self.manifest,
|
||||
lot=lot,
|
||||
report_dir=report_dir,
|
||||
state_path=state_path,
|
||||
steps=self._steps_for_lot(lot),
|
||||
)
|
||||
state.dump(state_path)
|
||||
|
||||
if report_only:
|
||||
self._sync_tracking(state, dry_run=dry_run)
|
||||
return 0
|
||||
|
||||
while state.step_index < len(state.steps):
|
||||
step = state.steps[state.step_index]
|
||||
step_type = str(step["type"])
|
||||
if step_type == "ensure_models":
|
||||
print("==> lot ensure_models")
|
||||
self._run_ensure_models(state, dry_run=dry_run)
|
||||
state.step_index += 1
|
||||
state.model_index = 0
|
||||
state.dump()
|
||||
continue
|
||||
if step_type == "models":
|
||||
print(f"==> lot {step['name']}")
|
||||
exit_code = self._run_model_step(state, step, dry_run=dry_run)
|
||||
state.dump()
|
||||
if exit_code is not None:
|
||||
self._sync_tracking(state, dry_run=dry_run)
|
||||
return exit_code
|
||||
state.step_index += 1
|
||||
state.model_index = 0
|
||||
state.dump()
|
||||
continue
|
||||
if step_type == "tracking_sync":
|
||||
print("==> lot tracking_sync")
|
||||
self._sync_tracking(state, dry_run=dry_run)
|
||||
state.step_index += 1
|
||||
state.model_index = 0
|
||||
state.dump()
|
||||
continue
|
||||
raise NextLotsError(f"Type de lot non supporté: {step_type}")
|
||||
|
||||
self._write_report_summary(state)
|
||||
return 0
|
||||
|
||||
def _steps_for_lot(self, lot: str) -> list[dict[str, Any]]:
|
||||
if lot == "ensure_models":
|
||||
return [{"type": "ensure_models"}]
|
||||
if lot == "runtime_preflight":
|
||||
queue = [*self.manifest.priority_models, *self.manifest.baseline_models]
|
||||
return [{"type": "models", "name": "runtime_preflight", "models": queue, "preflight_only": True}]
|
||||
if lot == "priority_models":
|
||||
return [
|
||||
{"type": "models", "name": "priority_models", "models": self.manifest.priority_models, "preflight_only": False},
|
||||
{"type": "tracking_sync"},
|
||||
]
|
||||
if lot == "baselines":
|
||||
return [
|
||||
{"type": "models", "name": "baselines", "models": self.manifest.baseline_models, "preflight_only": False},
|
||||
{"type": "models", "name": "preflight_only", "models": self.manifest.preflight_only_models, "preflight_only": True},
|
||||
{"type": "tracking_sync"},
|
||||
]
|
||||
if lot == "tracking_sync":
|
||||
return [{"type": "tracking_sync"}]
|
||||
if lot == "full":
|
||||
return [
|
||||
{"type": "ensure_models"},
|
||||
{"type": "models", "name": "priority_models", "models": self.manifest.priority_models, "preflight_only": False},
|
||||
{"type": "models", "name": "baselines", "models": self.manifest.baseline_models, "preflight_only": False},
|
||||
{"type": "models", "name": "preflight_only", "models": self.manifest.preflight_only_models, "preflight_only": True},
|
||||
{"type": "tracking_sync"},
|
||||
]
|
||||
raise NextLotsError(f"Lot inconnu: {lot}")
|
||||
|
||||
def _new_report_dir(self) -> Path:
|
||||
stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
|
||||
report_dir = self.manifest.repo_root / "automation" / "reports" / stamp
|
||||
report_dir.mkdir(parents=True, exist_ok=True)
|
||||
return report_dir
|
||||
|
||||
def _run_ensure_models(self, state: RunState, *, dry_run: bool) -> None:
|
||||
if dry_run:
|
||||
state.notes.append("Dry-run: ensure_models non exécuté.")
|
||||
return
|
||||
args = ["bash", "scripts/ensure_apple_models.sh"]
|
||||
result = self.command_runner(args, self.manifest.tracking.mascarade_repo)
|
||||
log_path = Path(state.report_dir) / "ensure_models.log"
|
||||
log_path.write_text(_command_log(result), encoding="utf-8")
|
||||
if result.returncode != 0:
|
||||
raise NextLotsError("ensure_models a échoué.")
|
||||
missing = self._missing_ollama_models()
|
||||
if missing:
|
||||
state.notes.append(
|
||||
"Modèles Ollama manquants: " + ", ".join(missing) + ". Lancer manuellement `ollama pull` sur ces modèles."
|
||||
)
|
||||
|
||||
def _missing_ollama_models(self) -> list[str]:
|
||||
try:
|
||||
payload = self.json_fetcher(self.manifest.ollama_tags_url, 10.0)
|
||||
except Exception:
|
||||
return []
|
||||
models = payload.get("models") if isinstance(payload, dict) else None
|
||||
if not isinstance(models, list):
|
||||
return []
|
||||
names = {
|
||||
str(item.get("name", "")).strip()
|
||||
for item in models
|
||||
if isinstance(item, dict) and str(item.get("name", "")).strip()
|
||||
}
|
||||
return [model for model in self.manifest.required_ollama_models if model not in names]
|
||||
|
||||
def _run_model_step(self, state: RunState, step: dict[str, Any], *, dry_run: bool) -> int | None:
|
||||
models = [str(item) for item in step["models"]]
|
||||
preflight_only = bool(step.get("preflight_only", False))
|
||||
for index in range(state.model_index, len(models)):
|
||||
state.model_index = index
|
||||
model = models[index]
|
||||
category = str(step["name"])
|
||||
state.notes = [f"Modele en cours: {model}"]
|
||||
state.dump()
|
||||
print(f"--> {model}")
|
||||
if dry_run:
|
||||
state.append_result(
|
||||
ModelRunResult(
|
||||
model=model,
|
||||
category=category,
|
||||
classification="dry_run",
|
||||
notes=["Dry-run: aucun preflight ni smoke exécuté."],
|
||||
)
|
||||
)
|
||||
continue
|
||||
checkpoint = self._checkpoint_if_runtime_manual_step_needed(state, model)
|
||||
if checkpoint is not None:
|
||||
print(f"checkpoint manuel: {checkpoint['reason']}")
|
||||
print(f"commande: {checkpoint['command']}")
|
||||
state.pending_manual_action = checkpoint
|
||||
state.notes = [f"Checkpoint manuel requis pour: {model}"]
|
||||
self._write_report_summary(state)
|
||||
return 3
|
||||
state.pending_manual_action = None
|
||||
result = self._run_model(model, category=category, preflight_only=preflight_only, report_dir=Path(state.report_dir))
|
||||
state.notes = [f"Dernier modele traite: {model} -> {result.classification}"]
|
||||
state.append_result(result)
|
||||
return None
|
||||
|
||||
def _checkpoint_if_runtime_manual_step_needed(self, state: RunState, model: str) -> dict[str, Any] | None:
|
||||
if not self._core_health_ok():
|
||||
return self._build_manual_action(
|
||||
state,
|
||||
args=["bash", "scripts/prepare_runtime_step.sh", "--restart", "core", "--resume-state", state.state_path, "--ane-script", str(self.manifest.repo_root / "scripts" / "run_next_lots.py")],
|
||||
reason="Le core mascarade ne répond pas correctement.",
|
||||
)
|
||||
if not model.startswith("apple-coreml:"):
|
||||
return None
|
||||
target_model = model.split(":", 1)[1]
|
||||
apple_model = self._wait_for_expected_apple_model(target_model)
|
||||
if apple_model == target_model:
|
||||
return None
|
||||
args = [
|
||||
"bash",
|
||||
"scripts/prepare_runtime_step.sh",
|
||||
"--apple-model",
|
||||
target_model,
|
||||
"--resume-state",
|
||||
state.state_path,
|
||||
"--ane-script",
|
||||
str(self.manifest.repo_root / "scripts" / "run_next_lots.py"),
|
||||
]
|
||||
return self._build_manual_action(
|
||||
state,
|
||||
args=args,
|
||||
reason=f"Le runtime Apple sert `{apple_model or 'aucun modèle'}` au lieu de `{target_model}`.",
|
||||
)
|
||||
|
||||
def _build_manual_action(self, state: RunState, *, args: list[str], reason: str) -> dict[str, Any]:
|
||||
result = self.command_runner(args, self.manifest.tracking.mascarade_repo)
|
||||
log_path = Path(state.report_dir) / f"manual_action_{len(state.results):02d}.log"
|
||||
log_path.write_text(_command_log(result), encoding="utf-8")
|
||||
return {
|
||||
"reason": reason,
|
||||
"command": " ".join(args),
|
||||
"log_path": str(log_path),
|
||||
"resume_state": state.state_path,
|
||||
}
|
||||
|
||||
def _core_health_ok(self) -> bool:
|
||||
try:
|
||||
payload = self.json_fetcher(f"{self.manifest.core_base_url}/health", 10.0)
|
||||
except Exception:
|
||||
return False
|
||||
return isinstance(payload, dict)
|
||||
|
||||
def _current_apple_model(self) -> str | None:
|
||||
try:
|
||||
payload = self.json_fetcher(f"{self.manifest.apple_runtime_url}/models", 10.0)
|
||||
except Exception:
|
||||
return None
|
||||
if isinstance(payload, list) and payload:
|
||||
return str(payload[0]).strip() or None
|
||||
if isinstance(payload, dict):
|
||||
models = payload.get("models")
|
||||
if isinstance(models, list) and models:
|
||||
return str(models[0]).strip() or None
|
||||
return None
|
||||
|
||||
def _wait_for_expected_apple_model(self, target_model: str) -> str | None:
|
||||
deadline = time.monotonic() + max(self.manifest.apple_model_ready_timeout_seconds, 0.0)
|
||||
poll_interval = max(self.manifest.apple_model_poll_interval_seconds, 0.1)
|
||||
last_seen = self._current_apple_model()
|
||||
if last_seen == target_model or self.manifest.apple_model_ready_timeout_seconds <= 0:
|
||||
return last_seen
|
||||
while time.monotonic() < deadline:
|
||||
time.sleep(poll_interval)
|
||||
last_seen = self._current_apple_model()
|
||||
if last_seen == target_model:
|
||||
return last_seen
|
||||
return last_seen
|
||||
|
||||
def _run_model(self, model: str, *, category: str, preflight_only: bool, report_dir: Path) -> ModelRunResult:
|
||||
result = ModelRunResult(model=model, category=category, apple_model_active=self._current_apple_model())
|
||||
model_slug = _slugify(model)
|
||||
preflight_args = [
|
||||
"bash",
|
||||
"scripts/smoke_openai_compat_ane.sh",
|
||||
"--url",
|
||||
self.manifest.core_base_url,
|
||||
"--model",
|
||||
model,
|
||||
"--timeout",
|
||||
str(self._timeout_for_model(model)),
|
||||
]
|
||||
preflight = self.command_runner(preflight_args, self.manifest.tracking.mascarade_repo)
|
||||
result.preflight_duration_seconds = preflight.duration_seconds
|
||||
preflight_log = report_dir / f"{model_slug}_preflight.log"
|
||||
preflight_log.write_text(_command_log(preflight), encoding="utf-8")
|
||||
result.preflight_log = str(preflight_log)
|
||||
result.preflight_ok = preflight.returncode == 0
|
||||
if not result.preflight_ok:
|
||||
result.classification = "provider_failed"
|
||||
result.status = "preflight_failed"
|
||||
result.notes.append("Le preflight OpenAI-compatible a échoué.")
|
||||
return result
|
||||
if preflight_only:
|
||||
result.classification = "preflight_only"
|
||||
result.status = "preflight_only"
|
||||
result.notes.append("Smoke complet volontairement sauté pour ce modèle.")
|
||||
return result
|
||||
|
||||
workspace = report_dir / "workspaces" / model_slug
|
||||
workspace.parent.mkdir(parents=True, exist_ok=True)
|
||||
smoke_args = [
|
||||
"bash",
|
||||
"scripts/smoke_local_generation.sh",
|
||||
"--base-url",
|
||||
self.manifest.core_base_url,
|
||||
"--model",
|
||||
model,
|
||||
"--chapter",
|
||||
self.manifest.smoke_chapter,
|
||||
"--workspace",
|
||||
str(workspace),
|
||||
"--timeout",
|
||||
str(self.manifest.smoke_timeout_seconds),
|
||||
"--intention",
|
||||
self.manifest.smoke_intention,
|
||||
"--approve",
|
||||
]
|
||||
smoke = self.command_runner(smoke_args, self.manifest.repo_root, env=self.manifest.preset_env)
|
||||
result.smoke_attempted = True
|
||||
result.smoke_duration_seconds = smoke.duration_seconds
|
||||
smoke_log = report_dir / f"{model_slug}_smoke.log"
|
||||
smoke_log.write_text(_command_log(smoke), encoding="utf-8")
|
||||
result.smoke_log = str(smoke_log)
|
||||
result.workspace = str(workspace)
|
||||
|
||||
chapter = ChapterId.parse(self.manifest.smoke_chapter)
|
||||
meta_path = workspace / "brouillons" / "chapitres" / chapter.slug / "meta.json"
|
||||
if not meta_path.exists():
|
||||
result.classification = "provider_failed"
|
||||
result.status = "missing_meta"
|
||||
result.notes.append("Le smoke n'a pas produit de meta.json exploitable.")
|
||||
return result
|
||||
|
||||
payload = json.loads(meta_path.read_text(encoding="utf-8"))
|
||||
result.meta_path = str(meta_path)
|
||||
result.status = str(payload.get("status", "")).strip() or None
|
||||
result.accepted = bool(payload.get("accepted", False))
|
||||
result.failed_stage = str(payload.get("failed_stage", "")).strip() or None
|
||||
result.quality_blockers = _string_list(payload.get("quality_blockers"))
|
||||
result.retry_stages = _string_list(payload.get("retry_stages"))
|
||||
result.repair_attempts = int(payload.get("repair_attempts", 0) or 0)
|
||||
result.repair_models = _string_list(payload.get("repair_models"))
|
||||
result.completed_stages = _string_list(payload.get("completed_stages"))
|
||||
artifacts = payload.get("artifacts", {})
|
||||
if isinstance(artifacts, dict):
|
||||
result.draft_path = _optional_string(artifacts.get("repair_latest")) or _optional_string(artifacts.get("draft_v2"))
|
||||
result.gate_path = _optional_string(artifacts.get("gate_v1"))
|
||||
result.manuscript_path = _optional_string(artifacts.get("manuscript"))
|
||||
|
||||
if result.status == "accepted":
|
||||
result.classification = "accepted"
|
||||
elif result.status == "quality_blocked":
|
||||
result.classification = "quality_blocked"
|
||||
elif smoke.returncode == 0 and result.status == "rejected":
|
||||
result.classification = "provider_failed"
|
||||
else:
|
||||
result.classification = "provider_failed"
|
||||
return result
|
||||
|
||||
def _timeout_for_model(self, model: str) -> int:
|
||||
if model.startswith("apple-coreml:"):
|
||||
return max(600, self.manifest.smoke_timeout_seconds)
|
||||
return max(120, self.manifest.smoke_timeout_seconds)
|
||||
|
||||
def _sync_tracking(self, state: RunState, *, dry_run: bool) -> None:
|
||||
if dry_run:
|
||||
self._write_report_summary(state)
|
||||
return
|
||||
typed_results = state.typed_results()
|
||||
project_state = ProjectState(self.manifest.repo_root).summary()
|
||||
summary = _build_summary(state, typed_results)
|
||||
comparison = _render_comparison_markdown(state, typed_results)
|
||||
active_next = _compute_next_lot_recommendation(typed_results, self.manifest.next_code_lot)
|
||||
|
||||
replace_auto_section(
|
||||
self.manifest.tracking.ane_todo_active,
|
||||
AUTO_SYNC_TODO_ACTIVE,
|
||||
"## Auto-sync",
|
||||
_render_todo_active_sync(summary, active_next),
|
||||
)
|
||||
replace_auto_section(
|
||||
self.manifest.tracking.ane_todo_done,
|
||||
AUTO_SYNC_TODO_DONE,
|
||||
"## Auto-sync",
|
||||
_render_todo_done_sync(summary),
|
||||
)
|
||||
replace_auto_section(
|
||||
self.manifest.tracking.ane_plan,
|
||||
AUTO_SYNC_PLAN,
|
||||
"## Auto-sync",
|
||||
_render_plan_sync(summary, active_next),
|
||||
)
|
||||
replace_auto_section(
|
||||
self.manifest.tracking.ane_comparison,
|
||||
AUTO_SYNC_COMPARISON,
|
||||
"## Auto-sync",
|
||||
comparison,
|
||||
)
|
||||
replace_auto_section(
|
||||
self.manifest.tracking.ane_readme,
|
||||
AUTO_SYNC_README,
|
||||
"## Etat auto-synchronise",
|
||||
_render_readme_sync(summary, active_next),
|
||||
)
|
||||
replace_auto_section(
|
||||
self.manifest.tracking.ane_runbook,
|
||||
AUTO_SYNC_RUNBOOK,
|
||||
"## Etat auto-synchronise",
|
||||
_render_runbook_sync(summary, project_state, active_next),
|
||||
)
|
||||
replace_auto_section(
|
||||
self.manifest.tracking.mascarade_todo,
|
||||
AUTO_SYNC_MASCARADE_TODO,
|
||||
"## Auto-sync",
|
||||
_render_mascarade_todo_sync(summary, active_next),
|
||||
)
|
||||
replace_auto_section(
|
||||
self.manifest.tracking.mascarade_plan,
|
||||
AUTO_SYNC_MASCARADE_PLAN,
|
||||
"## Auto-sync",
|
||||
_render_mascarade_plan_sync(summary, active_next),
|
||||
)
|
||||
replace_auto_section(
|
||||
self.manifest.tracking.mascarade_readme,
|
||||
AUTO_SYNC_MASCARADE_README,
|
||||
"## Etat auto-synchronise",
|
||||
_render_mascarade_readme_sync(summary, active_next),
|
||||
)
|
||||
replace_auto_section(
|
||||
self.manifest.tracking.mascarade_runbook,
|
||||
AUTO_SYNC_MASCARADE_RUNBOOK,
|
||||
"## Etat auto-synchronise",
|
||||
_render_mascarade_runbook_sync(summary, active_next),
|
||||
)
|
||||
self._write_report_summary(state)
|
||||
|
||||
def _write_report_summary(self, state: RunState) -> None:
|
||||
report_dir = Path(state.report_dir)
|
||||
report_dir.mkdir(parents=True, exist_ok=True)
|
||||
run_path = report_dir / "run.json"
|
||||
summary_path = report_dir / "SUMMARY.md"
|
||||
run_path.write_text(json.dumps(asdict(state), ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
summary_path.write_text(_render_summary_markdown(state, state.typed_results()), encoding="utf-8")
|
||||
|
||||
|
||||
def _string_list(value: object) -> list[str]:
|
||||
if not isinstance(value, list):
|
||||
return []
|
||||
return [str(item).strip() for item in value if str(item).strip()]
|
||||
|
||||
|
||||
def _optional_string(value: object) -> str | None:
|
||||
text = str(value).strip() if value is not None else ""
|
||||
return text or None
|
||||
|
||||
|
||||
def _slugify(value: str) -> str:
|
||||
return "".join(char if char.isalnum() else "_" for char in value).strip("_").lower()
|
||||
|
||||
|
||||
def _command_log(result: CommandResult) -> str:
|
||||
return (
|
||||
f"$ {' '.join(result.args)}\n"
|
||||
f"returncode={result.returncode}\n"
|
||||
f"duration_seconds={result.duration_seconds:.2f}\n\n"
|
||||
f"STDOUT\n{result.stdout}\n\nSTDERR\n{result.stderr}"
|
||||
)
|
||||
|
||||
|
||||
def _build_summary(state: RunState, results: list[ModelRunResult]) -> dict[str, Any]:
|
||||
accepted = [item for item in results if item.classification == "accepted"]
|
||||
reached_gate = [item for item in results if item.reached_gate()]
|
||||
quality_blocked = [item for item in results if item.classification == "quality_blocked"]
|
||||
provider_failed = [item for item in results if item.classification == "provider_failed"]
|
||||
return {
|
||||
"started_at": state.started_at,
|
||||
"updated_at": state.updated_at,
|
||||
"pending_manual_action": state.pending_manual_action,
|
||||
"accepted_models": [item.model for item in accepted],
|
||||
"reached_gate_models": [item.model for item in reached_gate],
|
||||
"quality_blocked_models": [item.model for item in quality_blocked],
|
||||
"provider_failed_models": [item.model for item in provider_failed],
|
||||
"results": results,
|
||||
}
|
||||
|
||||
|
||||
def _compute_next_lot_recommendation(results: list[ModelRunResult], fallback: str) -> str:
|
||||
if any(item.classification == "accepted" for item in results):
|
||||
return "Rejouer uniquement les baselines vitesse puis figer la référence locale dans les README/runbooks."
|
||||
if any(item.reached_gate() for item in results):
|
||||
return "Analyser les runs ayant atteint gate/repair puis resserrer la reference locale autour des meilleurs candidats."
|
||||
return fallback
|
||||
|
||||
|
||||
def _render_todo_active_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||
lines = [
|
||||
f"- dernier cycle automatique: {summary['updated_at']}",
|
||||
f"- modeles accepted: {_comma_or_none(summary['accepted_models'])}",
|
||||
f"- modeles ayant atteint gate: {_comma_or_none(summary['reached_gate_models'])}",
|
||||
f"- quality_blocked: {_comma_or_none(summary['quality_blocked_models'])}",
|
||||
f"- provider_failed: {_comma_or_none(summary['provider_failed_models'])}",
|
||||
f"- prochain lot recommande: {next_lot}",
|
||||
]
|
||||
if summary["pending_manual_action"]:
|
||||
pending = summary["pending_manual_action"]
|
||||
lines.extend(
|
||||
[
|
||||
f"- checkpoint manuel en attente: {pending['reason']}",
|
||||
f"- commande preparee: `{pending['command']}`",
|
||||
f"- reprise: `python3 scripts/run_next_lots.py --resume {pending['resume_state']}`",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _render_todo_done_sync(summary: dict[str, Any]) -> str:
|
||||
lines = [
|
||||
"- orchestrateur `scripts/run_next_lots.py` disponible",
|
||||
"- manifeste `automation/next_lots.toml` charge",
|
||||
"- derniers fichiers de suivi synchronisables via marqueurs `AUTO-SYNC`",
|
||||
f"- dernier cycle automatise observe: {summary['updated_at']}",
|
||||
]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _render_plan_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||
lines = [
|
||||
f"- dernier verdict automatise: {summary['updated_at']}",
|
||||
f"- accepted: {_comma_or_none(summary['accepted_models'])}",
|
||||
f"- gate atteint: {_comma_or_none(summary['reached_gate_models'])}",
|
||||
f"- prochain lot calcule: {next_lot}",
|
||||
]
|
||||
if summary["pending_manual_action"]:
|
||||
lines.append(f"- checkpoint manuel requis: {summary['pending_manual_action']['reason']}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _render_readme_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||
lines = [
|
||||
f"- dernier cycle automatise: {summary['updated_at']}",
|
||||
f"- reference locale actuelle: {_reference_label(summary)}",
|
||||
f"- prochain lot utile: {next_lot}",
|
||||
"- lancer un cycle: `python3 scripts/run_next_lots.py --lot full`",
|
||||
]
|
||||
if summary["pending_manual_action"]:
|
||||
lines.append(f"- checkpoint manuel en attente: {summary['pending_manual_action']['reason']}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _render_runbook_sync(summary: dict[str, Any], project_state: dict[str, Any], next_lot: str) -> str:
|
||||
lines = [
|
||||
f"- dernier cycle automatise: {summary['updated_at']}",
|
||||
f"- chapitre courant detecte: {project_state.get('current_chapter') or 'aucun'}",
|
||||
f"- reference locale actuelle: {_reference_label(summary)}",
|
||||
f"- prochain lot utile: {next_lot}",
|
||||
]
|
||||
if summary["pending_manual_action"]:
|
||||
lines.append(f"- reprise attendue apres action manuelle: {summary['pending_manual_action']['resume_state']}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _render_mascarade_todo_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||
lines = [
|
||||
f"- dernier cycle ANE automatise: {summary['updated_at']}",
|
||||
f"- accepted via runtime local: {_comma_or_none(summary['accepted_models'])}",
|
||||
f"- gate atteint via runtime local: {_comma_or_none(summary['reached_gate_models'])}",
|
||||
f"- blocage runtime principal: {next_lot}",
|
||||
]
|
||||
if summary["pending_manual_action"]:
|
||||
lines.append(f"- checkpoint runtime manuel: {summary['pending_manual_action']['reason']}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _render_mascarade_plan_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||
lines = [
|
||||
f"- dernier cycle ANE automatise: {summary['updated_at']}",
|
||||
f"- reference locale ANE: {_reference_label(summary)}",
|
||||
f"- prochain lot ANE a servir: {next_lot}",
|
||||
]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _render_mascarade_readme_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||
return "\n".join(
|
||||
[
|
||||
f"- dernier cycle ANE automatise: {summary['updated_at']}",
|
||||
f"- etat de reference ANE: {_reference_label(summary)}",
|
||||
f"- prochain lot utile cote pipeline: {next_lot}",
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def _render_mascarade_runbook_sync(summary: dict[str, Any], next_lot: str) -> str:
|
||||
lines = [
|
||||
f"- dernier cycle ANE automatise: {summary['updated_at']}",
|
||||
f"- meilleurs candidats actuels: {_top_candidates(summary['results'])}",
|
||||
f"- prochain lot utile cote ANE: {next_lot}",
|
||||
]
|
||||
if summary["pending_manual_action"]:
|
||||
lines.append(f"- checkpoint runtime manuel: {summary['pending_manual_action']['reason']}")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _reference_label(summary: dict[str, Any]) -> str:
|
||||
if summary["accepted_models"]:
|
||||
return summary["accepted_models"][0]
|
||||
if summary["reached_gate_models"]:
|
||||
return f"aucun accepted, meilleur diagnostic: {summary['reached_gate_models'][0]}"
|
||||
return "aucune reference accepted"
|
||||
|
||||
|
||||
def _top_candidates(results: Iterable[ModelRunResult]) -> str:
|
||||
candidates = []
|
||||
for item in results:
|
||||
if item.model in candidates:
|
||||
continue
|
||||
if item.model.startswith("apple-coreml:qwen3.5-4b") or item.model.startswith("ollama:qwen2.5:7b"):
|
||||
candidates.append(item.model)
|
||||
return ", ".join(candidates) if candidates else "aucun"
|
||||
|
||||
|
||||
def _comma_or_none(items: list[str]) -> str:
|
||||
return ", ".join(items) if items else "aucun"
|
||||
|
||||
|
||||
def _render_comparison_markdown(state: RunState, results: list[ModelRunResult]) -> str:
|
||||
lines = [
|
||||
f"- dernier cycle automatise: {state.updated_at}",
|
||||
"",
|
||||
"| Modele | Categorie | Preflight | Smoke | Classification | Failed stage | Gate | Repairs | Notes |",
|
||||
"|---|---|---|---|---|---|---|---:|---|",
|
||||
]
|
||||
for item in results:
|
||||
lines.append(
|
||||
"| {model} | {category} | {preflight} | {smoke} | {classification} | {failed_stage} | {gate} | {repairs} | {notes} |".format(
|
||||
model=item.model,
|
||||
category=item.category,
|
||||
preflight="OK" if item.preflight_ok else ("KO" if item.preflight_ok is False else "n/a"),
|
||||
smoke="oui" if item.smoke_attempted else "non",
|
||||
classification=item.classification,
|
||||
failed_stage=item.failed_stage or "",
|
||||
gate="oui" if item.reached_gate() else "non",
|
||||
repairs=item.repair_attempts,
|
||||
notes="; ".join(item.notes) if item.notes else "",
|
||||
)
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _render_summary_markdown(state: RunState, results: list[ModelRunResult]) -> str:
|
||||
summary = _build_summary(state, results)
|
||||
lines = [
|
||||
"# Résumé du cycle automatique",
|
||||
"",
|
||||
f"- lot: `{state.lot}`",
|
||||
f"- démarré: `{state.started_at}`",
|
||||
f"- mis à jour: `{state.updated_at}`",
|
||||
f"- accepted: {_comma_or_none(summary['accepted_models'])}",
|
||||
f"- gate atteint: {_comma_or_none(summary['reached_gate_models'])}",
|
||||
f"- quality_blocked: {_comma_or_none(summary['quality_blocked_models'])}",
|
||||
f"- provider_failed: {_comma_or_none(summary['provider_failed_models'])}",
|
||||
]
|
||||
if state.pending_manual_action:
|
||||
lines.extend(
|
||||
[
|
||||
"",
|
||||
"## Checkpoint manuel",
|
||||
f"- raison: {state.pending_manual_action['reason']}",
|
||||
f"- commande: `{state.pending_manual_action['command']}`",
|
||||
f"- reprise: `python3 scripts/run_next_lots.py --resume {state.pending_manual_action['resume_state']}`",
|
||||
]
|
||||
)
|
||||
if results:
|
||||
lines.extend(["", "## Résultats", ""])
|
||||
lines.append(_render_comparison_markdown(state, results))
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(prog="python3 scripts/run_next_lots.py")
|
||||
parser.add_argument("--manifest", default="automation/next_lots.toml")
|
||||
parser.add_argument("--lot", default="full", choices=["full", "ensure_models", "runtime_preflight", "priority_models", "baselines", "tracking_sync"])
|
||||
parser.add_argument("--resume", type=Path)
|
||||
parser.add_argument("--dry-run", action="store_true")
|
||||
parser.add_argument("--report-only", action="store_true")
|
||||
return parser
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None, repo_root: Path | None = None) -> int:
|
||||
parser = build_parser()
|
||||
args = parser.parse_args(argv)
|
||||
root = repo_root or Path.cwd()
|
||||
manifest = Manifest.load(root, root / args.manifest)
|
||||
runner = NextLotsRunner(manifest)
|
||||
return runner.run(
|
||||
lot=args.lot,
|
||||
resume_state=args.resume,
|
||||
dry_run=args.dry_run,
|
||||
report_only=args.report_only,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,202 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import json
|
||||
|
||||
from core.chapters import ChapterId, discover_chapter_dirs, discover_chapter_files
|
||||
|
||||
|
||||
class ProjectState:
|
||||
"""
|
||||
Detects and summarizes the current state of a writing project.
|
||||
Read-only, file-based, human-readable.
|
||||
"""
|
||||
|
||||
def __init__(self, root: Path):
|
||||
self.root = root
|
||||
self.manuscript = root / "manuscrit"
|
||||
self.structure = root / "structure" / "chapitres"
|
||||
self.drafts = root / "brouillons" / "chapitres"
|
||||
self.memory = root / "memoire"
|
||||
self.memory_chapters = self.memory / "chapitres"
|
||||
self.intentions = root / "notes" / "intentions"
|
||||
|
||||
def detect_current_chapter(self) -> str | None:
|
||||
chapters = self.known_chapters()
|
||||
if not chapters:
|
||||
return None
|
||||
return chapters[-1].slug
|
||||
|
||||
def known_chapters(self) -> list[ChapterId]:
|
||||
chapters: set[ChapterId] = set()
|
||||
for chapter, _path in discover_chapter_files(self.intentions):
|
||||
chapters.add(chapter)
|
||||
for chapter, _path in discover_chapter_files(self.structure):
|
||||
chapters.add(chapter)
|
||||
for chapter, _path in discover_chapter_files(self.manuscript):
|
||||
chapters.add(chapter)
|
||||
for chapter, _path in discover_chapter_files(self.memory_chapters):
|
||||
chapters.add(chapter)
|
||||
for chapter, _path in discover_chapter_dirs(self.drafts):
|
||||
chapters.add(chapter)
|
||||
return sorted(chapters)
|
||||
|
||||
def latest_drafts(self) -> dict[str, str]:
|
||||
latest: dict[str, str] = {}
|
||||
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||
meta = self._load_meta(draft_dir)
|
||||
if meta:
|
||||
artifacts = meta.get("artifacts", {})
|
||||
if isinstance(artifacts, dict):
|
||||
repair_latest = artifacts.get("repair_latest")
|
||||
if isinstance(repair_latest, str) and repair_latest.strip():
|
||||
latest[chapter.slug] = Path(repair_latest).name
|
||||
continue
|
||||
|
||||
candidates = sorted(path.name for path in draft_dir.glob("draft_v*.md"))
|
||||
if candidates:
|
||||
latest[chapter.slug] = candidates[-1]
|
||||
return latest
|
||||
|
||||
def latest_repairs(self) -> dict[str, str]:
|
||||
latest: dict[str, str] = {}
|
||||
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||
meta = self._load_meta(draft_dir)
|
||||
if not meta:
|
||||
continue
|
||||
artifacts = meta.get("artifacts", {})
|
||||
if not isinstance(artifacts, dict):
|
||||
continue
|
||||
repair_latest = artifacts.get("repair_latest")
|
||||
if isinstance(repair_latest, str) and repair_latest.strip():
|
||||
latest[chapter.slug] = Path(repair_latest).name
|
||||
return latest
|
||||
|
||||
def failed_chapters(self) -> list[dict[str, object]]:
|
||||
failures: list[dict[str, object]] = []
|
||||
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||
meta = self._load_meta(draft_dir)
|
||||
if not meta or meta.get("status") != "failed":
|
||||
continue
|
||||
failures.append(
|
||||
{
|
||||
"chapter": chapter.slug,
|
||||
"status": str(meta.get("status", "")),
|
||||
"failed_stage": str(meta.get("failed_stage", "")),
|
||||
"meta_path": str(draft_dir / "meta.json"),
|
||||
"retry_stages": self._retry_stages(meta),
|
||||
"last_status_message": str(meta.get("last_status_message", "")).strip(),
|
||||
}
|
||||
)
|
||||
return failures
|
||||
|
||||
def quality_blocked_chapters(self) -> list[dict[str, object]]:
|
||||
blocked: list[dict[str, object]] = []
|
||||
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||
meta = self._load_meta(draft_dir)
|
||||
if not meta or meta.get("status") != "quality_blocked":
|
||||
continue
|
||||
artifacts = meta.get("artifacts", {})
|
||||
if not isinstance(artifacts, dict):
|
||||
artifacts = {}
|
||||
raw_blockers = meta.get("quality_blockers")
|
||||
quality_blockers = []
|
||||
if isinstance(raw_blockers, list):
|
||||
quality_blockers = [str(item).strip() for item in raw_blockers if str(item).strip()]
|
||||
blocked.append(
|
||||
{
|
||||
"chapter": chapter.slug,
|
||||
"status": str(meta.get("status", "")),
|
||||
"failed_stage": str(meta.get("failed_stage", "")),
|
||||
"meta_path": str(draft_dir / "meta.json"),
|
||||
"draft_path": str(artifacts.get("repair_latest") or artifacts.get("draft_v2", draft_dir / "draft_v2.md")),
|
||||
"gate_path": str(artifacts.get("gate_v1", draft_dir / "gate_v1.json")),
|
||||
"quality_blockers": quality_blockers,
|
||||
"retry_stages": self._retry_stages(meta),
|
||||
"repair_attempts": int(meta.get("repair_attempts", 0) or 0),
|
||||
"repair_models": self._repair_models(meta),
|
||||
"last_status_message": str(meta.get("last_status_message", "")).strip(),
|
||||
}
|
||||
)
|
||||
return blocked
|
||||
|
||||
def awaiting_acceptance(self) -> list[dict[str, object]]:
|
||||
pending: list[dict[str, object]] = []
|
||||
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||
meta = self._load_meta(draft_dir)
|
||||
if not meta or meta.get("status") != "awaiting_acceptance":
|
||||
continue
|
||||
artifacts = meta.get("artifacts", {})
|
||||
if not isinstance(artifacts, dict):
|
||||
artifacts = {}
|
||||
pending.append(
|
||||
{
|
||||
"chapter": chapter.slug,
|
||||
"status": str(meta.get("status", "")),
|
||||
"draft_path": str(artifacts.get("repair_latest") or artifacts.get("draft_v2", draft_dir / "draft_v2.md")),
|
||||
"critique_path": str(artifacts.get("critique_v1", draft_dir / "critique_v1.md")),
|
||||
"gate_path": str(artifacts.get("gate_v1", draft_dir / "gate_v1.json")),
|
||||
"meta_path": str(draft_dir / "meta.json"),
|
||||
"retry_stages": self._retry_stages(meta),
|
||||
"repair_attempts": int(meta.get("repair_attempts", 0) or 0),
|
||||
"repair_models": self._repair_models(meta),
|
||||
"last_status_message": str(meta.get("last_status_message", "")).strip(),
|
||||
}
|
||||
)
|
||||
return pending
|
||||
|
||||
def retry_stages(self) -> dict[str, list[str]]:
|
||||
retries: dict[str, list[str]] = {}
|
||||
for chapter, draft_dir in discover_chapter_dirs(self.drafts):
|
||||
meta = self._load_meta(draft_dir)
|
||||
if not meta:
|
||||
continue
|
||||
stages = self._retry_stages(meta)
|
||||
if stages:
|
||||
retries[chapter.slug] = stages
|
||||
return retries
|
||||
|
||||
def summary(self) -> dict[str, object]:
|
||||
return {
|
||||
"project_root": str(self.root),
|
||||
"current_chapter": self.detect_current_chapter(),
|
||||
"known_chapters": [chapter.slug for chapter in self.known_chapters()],
|
||||
"directories": {
|
||||
"structure": self.structure.exists(),
|
||||
"drafts": self.drafts.exists(),
|
||||
"manuscript": self.manuscript.exists(),
|
||||
"memory": self.memory.exists(),
|
||||
},
|
||||
"has_structure": self.structure.exists(),
|
||||
"has_memory": self.memory.exists(),
|
||||
"latest_drafts": self.latest_drafts(),
|
||||
"latest_repairs": self.latest_repairs(),
|
||||
"failed_chapters": self.failed_chapters(),
|
||||
"quality_blocked_chapters": self.quality_blocked_chapters(),
|
||||
"awaiting_acceptance": self.awaiting_acceptance(),
|
||||
"retry_stages": self.retry_stages(),
|
||||
}
|
||||
|
||||
def _load_meta(self, draft_dir: Path) -> dict[str, object] | None:
|
||||
meta_path = draft_dir / "meta.json"
|
||||
if not meta_path.exists():
|
||||
return None
|
||||
try:
|
||||
payload = json.loads(meta_path.read_text(encoding="utf-8"))
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
if not isinstance(payload, dict):
|
||||
return None
|
||||
return payload
|
||||
|
||||
def _retry_stages(self, meta: dict[str, object]) -> list[str]:
|
||||
raw = meta.get("retry_stages")
|
||||
if not isinstance(raw, list):
|
||||
return []
|
||||
return [str(item).strip() for item in raw if str(item).strip()]
|
||||
|
||||
def _repair_models(self, meta: dict[str, object]) -> list[str]:
|
||||
raw = meta.get("repair_models")
|
||||
if not isinstance(raw, list):
|
||||
return []
|
||||
return [str(item).strip() for item in raw if str(item).strip()]
|
||||
@@ -0,0 +1,34 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from string import Template
|
||||
import json
|
||||
|
||||
|
||||
class PromptNotFoundError(FileNotFoundError):
|
||||
"""Raised when a prompt file is missing."""
|
||||
|
||||
|
||||
class PromptStore:
|
||||
def __init__(self, root: Path):
|
||||
self.root = root
|
||||
self.prompts_dir = root / "prompts"
|
||||
self.builtin_prompts_dir = Path(__file__).resolve().parents[1] / "prompts"
|
||||
|
||||
def render(self, name: str, **context: object) -> str:
|
||||
path = self.prompts_dir / f"{name}_v1.txt"
|
||||
if not path.exists():
|
||||
path = self.builtin_prompts_dir / f"{name}_v1.txt"
|
||||
if not path.exists():
|
||||
raise PromptNotFoundError(f"Prompt introuvable: {path}")
|
||||
|
||||
template = Template(path.read_text(encoding="utf-8"))
|
||||
normalized = {key: self._normalize(value) for key, value in context.items()}
|
||||
return template.substitute(normalized)
|
||||
|
||||
def _normalize(self, value: object) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
if isinstance(value, (dict, list)):
|
||||
return json.dumps(value, ensure_ascii=False, indent=2)
|
||||
return str(value)
|
||||
@@ -0,0 +1,59 @@
|
||||
# Plan d'execution - 7 mars 2026
|
||||
|
||||
Ordre recommande pour la suite de `ai-novel-engine`, base sur l'etat reel livre au 7 mars 2026.
|
||||
|
||||
## Lot 1 - Stabilisation locale Apple / Ollama
|
||||
|
||||
### Objectif
|
||||
- verrouiller un run chapitre complet en local via `apple-coreml`
|
||||
- rejouer le meme flux via `ollama`
|
||||
- durcir la fin du pipeline sur les sorties JSON encore fragiles
|
||||
|
||||
### Done quand
|
||||
- un chapitre complet passe jusqu'a la validation interactive puis a la promotion dans `manuscrit/` avec `apple-coreml`
|
||||
- le meme chapitre passe avec `ollama` sans changer le pipeline narratif
|
||||
- `critique` et `memory` disposent d'un second passage de reparation ou de reessai si le JSON reste invalide
|
||||
|
||||
### Risque principal
|
||||
- le service Apple local `:8201` peut rester lent, bloquer une connexion ou degrader la validation sequentielle
|
||||
|
||||
### Dependances
|
||||
- `mascarade` doit garder le shim `/v1/chat/completions` stable
|
||||
- un backend `ollama` local doit etre disponible pour le second passage
|
||||
- les budgets par etape doivent rester ajustables sans changer le pipeline
|
||||
|
||||
## Lot 2 - Workflow auteur et CLI non interactive
|
||||
|
||||
### Objectif
|
||||
- rendre le workflow auteur exploitable en interactif et en batch local
|
||||
- exposer plus clairement l'etat d'echec des chapitres
|
||||
|
||||
### Done quand
|
||||
- `generate chapter` accepte `--approve` et `--reject`
|
||||
- `status` expose les chapitres en echec, le dernier `failed_stage` et le dernier artefact utile
|
||||
- le smoke local affiche un resume lisible sans ouvrir `meta.json`
|
||||
|
||||
### Risque principal
|
||||
- la CLI peut devenir ambigue si les modes interactif et non interactif divergent
|
||||
|
||||
### Dependances
|
||||
- les artefacts de pipeline doivent rester stables
|
||||
- les metadonnees `meta.json` doivent contenir assez d'information pour alimenter `status`
|
||||
|
||||
## Lot 3 - Docs produit et runbooks
|
||||
|
||||
### Objectif
|
||||
- remplacer les placeholders de doc produit
|
||||
- figer les contrats cross-repo et les procedures de recuperation locales
|
||||
|
||||
### Done quand
|
||||
- `docs/vision.md` et `docs/roadmap.md` ne sont plus des placeholders
|
||||
- un runbook court de recuperation Apple local existe
|
||||
- le contrat `mascarade` utile a `ai-novel-engine` est documente une fois, de facon stable
|
||||
|
||||
### Risque principal
|
||||
- la doc peut diverger vite du runtime si elle est redigee avant la stabilisation locale
|
||||
|
||||
### Dependances
|
||||
- le Lot 1 doit etre suffisamment stable pour produire des runbooks fiables
|
||||
- `mascarade` doit figer le perimetre ANE suivi dans `TODO_AI_NOVEL_ENGINE.md`
|
||||
@@ -0,0 +1,104 @@
|
||||
# Plan d'execution - 8 mars 2026
|
||||
|
||||
Plan de reference apres livraison de la boucle `repair`.
|
||||
|
||||
Le plan du 7 mars 2026 reste archive pour historique. L'ordre recommande a date
|
||||
est celui-ci.
|
||||
|
||||
Pilotage operationnel:
|
||||
- lancer les lots avec `python3 scripts/run_next_lots.py --lot <lot>`
|
||||
- utiliser `automation/next_lots.toml` comme source de verite pour l'ordre des smokes, les budgets et les fichiers de suivi
|
||||
- en cas de switch Apple ou de restart runtime, reprendre ensuite avec `python3 scripts/run_next_lots.py --resume automation/state/next_lots_state.json`
|
||||
|
||||
## Lot 1 - Consolider la reference acceptee et finir les baselines
|
||||
|
||||
### Etat constate
|
||||
- la boucle `repair` est livree, testee et visible dans `status` / `meta.json`
|
||||
- `apple-coreml:qwen3.5-4b-onnx-q4f16` a termine un cycle complet et est `accepted`
|
||||
- `ollama:qwen2.5:7b` atteint `gate`, exerce `repair` en live, puis reste `quality_blocked` sur `outline_like`
|
||||
- le lot `baselines` est en cours pour `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b`
|
||||
- le runtime Apple local n'expose qu'un seul `model_id` a la fois, ce qui limite le fallback `repair` entre modeles Apple au sein d'un meme smoke
|
||||
|
||||
### Objectif
|
||||
- finir les baselines pour avoir un comparatif complet du protocole courant
|
||||
- confirmer que la reference `apple-coreml:qwen3.5-4b-onnx-q4f16` est reproductible sur plus d'un cycle
|
||||
- sortir `ollama:qwen2.5:7b` de `quality_blocked` sans degrader la prose utile
|
||||
|
||||
### Done quand
|
||||
- le lot `baselines` est termine et synchronise
|
||||
- `apple-coreml:qwen3.5-4b-onnx-q4f16` reste `accepted` sur un rerun de confirmation
|
||||
- `ollama:qwen2.5:7b` finit soit `accepted`, soit `quality_blocked` avec un diagnostic resserre qui ne soit plus `outline_like`
|
||||
|
||||
### Risque principal
|
||||
- la reference Apple 4B peut rester un succes isole si les switches runtime ou les budgets changent
|
||||
|
||||
### Dependances
|
||||
- garder le garde-fou comme blocage dur
|
||||
- conserver le protocole de comparaison commun et le meme preset qualite
|
||||
- installer ou restager explicitement avant les reruns Apple:
|
||||
- `qwen2.5-0.5b-instruct-onnx`
|
||||
- `qwen3.5-4b-onnx-q4f16`
|
||||
- `stateful-mistral7b-instruct-int4-coreml`
|
||||
- verifier avant chaque rerun Apple que le bon `model_id` est effectivement charge sur `:8201`
|
||||
|
||||
## Lot 2 - Tuner `rewrite` et `repair` pour Ollama 7B
|
||||
|
||||
### Objectif
|
||||
- garder `apple-coreml:qwen3.5-4b-onnx-q4f16` comme reference
|
||||
- faire passer `ollama:qwen2.5:7b` de `quality_blocked` a `accepted`
|
||||
- ne garder les petits modeles que comme baselines vitesse ou regressions
|
||||
|
||||
### Ordre recommande
|
||||
1. finir `apple-coreml:qwen2.5-0.5b-instruct-onnx`
|
||||
2. finir `ollama:qwen2.5:1.5b`
|
||||
3. rejouer `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||
4. rejouer `ollama:qwen2.5:7b`
|
||||
5. `ollama:qwen3.5:9b` seulement si `qwen2.5:7b` termine un smoke complet
|
||||
|
||||
### Done quand
|
||||
- le comparatif distingue clairement:
|
||||
- le modele de reference ANE actuel
|
||||
- le meilleur candidat Apple actuel
|
||||
- le meilleur candidat Ollama actuel
|
||||
- les baselines vitesse a conserver ou a sortir
|
||||
- le meilleur compromis Apple
|
||||
- le candidat vitesse encore insuffisant
|
||||
- les modeles a sortir de la reference locale
|
||||
|
||||
### Risque principal
|
||||
- les meilleurs candidats peuvent rester meilleurs sur la qualite, mais encore hors reference tant que `rewrite` ne passe pas
|
||||
|
||||
### Dependances
|
||||
- chemin Ollama de reference: Docker CPU via `mascarade`
|
||||
- service Apple `:8201` stable pendant tout le smoke
|
||||
- les trois modeles Apple cibles doivent etre installes et visibles cote runtime avant comparaison:
|
||||
- `qwen2.5-0.5b-instruct-onnx`
|
||||
- `qwen3.5-4b-onnx-q4f16`
|
||||
- `stateful-mistral7b-instruct-int4-coreml`
|
||||
- temps borne par requete pour garder des verdicts comparables
|
||||
|
||||
## Lot 3 - Docs et runbooks finaux
|
||||
|
||||
### Objectif
|
||||
- maintenir les README, TODOs, runbooks et le comparatif alignes sur l'etat reel
|
||||
|
||||
### Done quand
|
||||
- les docs distinguent clairement le modele `accepted`, les modeles `quality_blocked` et les baselines encore en rerun
|
||||
- le comparatif et les runbooks renvoient tous vers ce plan du 8 mars 2026
|
||||
- les TODOs n'exposent plus d'items deja livres
|
||||
|
||||
### Risque principal
|
||||
- la doc redevient trop optimiste si elle est mise a jour avant la revalidation complete
|
||||
|
||||
### Dependances
|
||||
- les lots 1 et 2 doivent produire des resultats reels, pas des suppositions
|
||||
|
||||
## Auto-sync
|
||||
## Auto-sync
|
||||
<!-- AUTO-SYNC:ANE-PLAN:START -->
|
||||
- dernier verdict automatise: 2026-03-09T06:53:02+00:00
|
||||
- accepted: aucun
|
||||
- gate atteint: apple-coreml:qwen2.5-0.5b-instruct-onnx, ollama:qwen2.5:1.5b
|
||||
- prochain lot calcule: Analyser les runs ayant atteint gate/repair puis resserrer la reference locale autour des meilleurs candidats.
|
||||
- checkpoint manuel requis: Le runtime Apple sert `qwen2.5-0.5b-instruct-onnx` au lieu de `stateful-mistral7b-instruct-int4-coreml`.
|
||||
<!-- AUTO-SYNC:ANE-PLAN:END -->
|
||||
@@ -0,0 +1,102 @@
|
||||
# Comparatif local ANE - 8 mars 2026
|
||||
|
||||
Comparatif realise avec le protocole courant:
|
||||
|
||||
- meme intention de smoke
|
||||
- meme chapitre `02`
|
||||
- meme CLI publique `generate chapter --chapter 02 --approve`
|
||||
- meme preset qualite:
|
||||
- `ANE_MAX_TOKENS_STRUCTURE=256`
|
||||
- `ANE_MAX_TOKENS_DRAFT=768`
|
||||
- `ANE_MAX_TOKENS_CRITIQUE=512`
|
||||
- `ANE_MAX_TOKENS_REWRITE=768`
|
||||
- `ANE_MAX_TOKENS_GATE=384`
|
||||
- `ANE_MAX_TOKENS_REPAIR=512`
|
||||
- `ANE_MAX_TOKENS_MEMORY=320`
|
||||
- `ANE_REPAIR_MAX_PASSES=2`
|
||||
- meme timeout borne par requete:
|
||||
- `300s`
|
||||
- meme garde-fou manuscrit dur et meme boucle `repair`
|
||||
|
||||
Contexte machine:
|
||||
|
||||
- `ai-novel-engine` pointe vers `mascarade` sur `http://127.0.0.1:8100`
|
||||
- `ollama` est route vers un service Docker CPU expose sur `127.0.0.1:11435`
|
||||
- le host `ollama` natif 0.17.7 reste bloque par un crash Metal sur cette machine
|
||||
- le runtime Apple local n'expose qu'un seul `model_id` a la fois sur `:8201`
|
||||
- dernier cycle complet termine au 9 mars 2026:
|
||||
- `apple-coreml:qwen3.5-4b-onnx-q4f16` est `accepted`
|
||||
- `ollama:qwen2.5:7b` atteint `gate`, exerce `repair` puis finit `quality_blocked`
|
||||
- le lot `baselines` est relance separement pour les petits modeles
|
||||
|
||||
## Resultats
|
||||
|
||||
| Modele | Backend | Preflight | Smoke complet | Statut final | Derniere etape atteinte | Total observe | Prose / narration | JSON / controle | Verdict |
|
||||
|---|---|---|---|---|---|---:|---|---|---|
|
||||
| `apple-coreml:qwen3.5-4b-onnx-q4f16` | `apple-coreml` | OK | oui | `accepted` | `memory` | `711s` | meilleure nuance narrative du lot | critique exploitable, gate vert | reference ANE locale actuelle |
|
||||
| `ollama:qwen2.5:7b` | `ollama` | OK | oui | `quality_blocked` | `gate` | `825s` | correcte, plus sobre que l'Apple 4B | critique exploitable, mais le texte reste trop proche d'un plan | meilleur candidat Ollama, encore bloque |
|
||||
| `apple-coreml:qwen2.5-0.5b-instruct-onnx` | `apple-coreml` | OK | rerun en cours | n/a | n/a | n/a | baseline vitesse a requalifier | n/a | en attente de verdict courant |
|
||||
| `ollama:qwen2.5:1.5b` | `ollama` | OK | rerun en cours | n/a | n/a | n/a | baseline vitesse a requalifier | n/a | en attente de verdict courant |
|
||||
|
||||
Point legacy hors protocole courant:
|
||||
|
||||
| Modele | Backend | Preflight | Smoke complet | Statut final |
|
||||
|---|---|---|---|---|
|
||||
| `apple-coreml:stateful-mistral7b-instruct-int4-coreml` | `apple-coreml` | OK | bloque > `8 min` a `structure` | `preflight_only` |
|
||||
|
||||
## Lecture rapide
|
||||
|
||||
### `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||
- passe `structure`, `draft`, `critique`, `rewrite`, `gate` puis `memory`
|
||||
- fournit le premier run `accepted` sous protocole `gate + repair`
|
||||
- devient la reference ANE locale actuelle
|
||||
- doit encore etre confirme sur rerun de stabilite
|
||||
|
||||
### `ollama:qwen2.5:7b`
|
||||
- passe `structure`, `draft`, `critique`, `rewrite` puis `gate`
|
||||
- exerce `repair` en live sur deux passes
|
||||
- reste bloque sur `outline_like`
|
||||
- c'est le meilleur candidat Ollama actuel, mais il lui manque encore une prose plus continue
|
||||
|
||||
### `apple-coreml:qwen2.5-0.5b-instruct-onnx`
|
||||
- rerun baseline en cours via le lot `baselines`
|
||||
- reste utile comme candidat vitesse Apple, pas comme reference qualite tant qu'un verdict courant n'est pas resynchronise
|
||||
|
||||
### `ollama:qwen2.5:1.5b`
|
||||
- rerun baseline en cours via le lot `baselines`
|
||||
- reste un temoin de regression plus qu'un candidat qualite
|
||||
|
||||
## Verdicts
|
||||
|
||||
- **Modele de reference ANE**: `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||
- **Meilleur compromis Apple**: `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||
- **Meilleur compromis Ollama**: `ollama:qwen2.5:7b`
|
||||
- **Modele rapide mais insuffisant**: `apple-coreml:qwen2.5-0.5b-instruct-onnx`
|
||||
- **Modeles a eviter pour la redaction longue sur cette machine**: `ollama:qwen2.5:1.5b` et `apple-coreml:stateful-mistral7b-instruct-int4-coreml`
|
||||
|
||||
## Conclusion du cycle
|
||||
|
||||
Le cycle `priority_models` atteint enfin un objectif produit minimal:
|
||||
|
||||
- la boucle `repair` est implementée, testee et visible dans `status` / `meta.json`
|
||||
- `repair` a maintenant une validation live sur `ollama:qwen2.5:7b`
|
||||
- un premier modele est `accepted` sous protocole courant: `apple-coreml:qwen3.5-4b-onnx-q4f16`
|
||||
- le prochain enjeu n'est plus de trouver un premier succes, mais de finir les baselines et de sortir `ollama:qwen2.5:7b` de `outline_like`
|
||||
|
||||
Le prochain lot logique n'est plus "ajouter un garde-fou", mais:
|
||||
|
||||
1. finir le lot `baselines`
|
||||
2. confirmer `apple-coreml:qwen3.5-4b-onnx-q4f16` sur rerun
|
||||
3. regler `rewrite` et `repair` pour faire tomber `outline_like` sur `ollama:qwen2.5:7b`
|
||||
4. ne garder `qwen2.5-0.5b` et `qwen2.5:1.5b` que comme baselines vitesse
|
||||
|
||||
## Auto-sync
|
||||
## Auto-sync
|
||||
<!-- AUTO-SYNC:ANE-COMPARISON:START -->
|
||||
- dernier cycle automatise: 2026-03-09T06:53:02+00:00
|
||||
|
||||
| Modele | Categorie | Preflight | Smoke | Classification | Failed stage | Gate | Repairs | Notes |
|
||||
|---|---|---|---|---|---|---|---:|---|
|
||||
| apple-coreml:qwen2.5-0.5b-instruct-onnx | baselines | OK | oui | quality_blocked | gate | oui | 2 | |
|
||||
| ollama:qwen2.5:1.5b | baselines | OK | oui | quality_blocked | gate | oui | 2 | |
|
||||
<!-- AUTO-SYNC:ANE-COMPARISON:END -->
|
||||
+28
-1
@@ -1 +1,28 @@
|
||||
Roadmap v2 du projet.
|
||||
# Roadmap v2
|
||||
|
||||
Roadmap courte et concrete, alignee sur l'etat reel du repo.
|
||||
|
||||
## Priorite 1 - Passer au moins un modele jusqu'a `gate`
|
||||
|
||||
- compacter `rewrite` pour qu'au moins un modele atteigne `gate`
|
||||
- conserver la boucle `repair` et le garde-fou comme blocages durs
|
||||
- viser d'abord `apple-coreml:qwen3.5-4b-onnx-q4f16` et `ollama:qwen2.5:7b`
|
||||
|
||||
## Priorite 2 - Requalifier les modeles plus lourds
|
||||
|
||||
- garder `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b` comme baselines vitesse
|
||||
- rejouer `qwen3.5:9b` seulement si `qwen2.5:7b` termine un smoke complet
|
||||
- maintenir les modeles toujours explicites dans les smokes et la doc
|
||||
- tenir compte du fait que le runtime Apple local ne sert qu'un `model_id` a la fois
|
||||
|
||||
## Priorite 3 - Exploitation locale et docs
|
||||
|
||||
- runbook local ANE centre sur `rewrite`, `gate_v1.json`, `repair_vN.md` et `quality_blocked`
|
||||
- runbook Apple local cote `mascarade` aligne sur les statuts reels
|
||||
- README et suivi croises pointent vers `EXECUTION_PLAN_2026-03-08.md`
|
||||
|
||||
## Source de verite
|
||||
|
||||
- backlog actif: [`../TODO_ACTIVE.md`](../TODO_ACTIVE.md)
|
||||
- etat livre: [`../TODO_IMPLEMENTE.md`](../TODO_IMPLEMENTE.md)
|
||||
- ordre d'execution: [`EXECUTION_PLAN_2026-03-08.md`](./EXECUTION_PLAN_2026-03-08.md)
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
# Runbook local - generation ANE
|
||||
|
||||
Runbook court pour lancer et diagnostiquer la generation locale via `mascarade`.
|
||||
|
||||
Comparatif de reference: [`docs/MODEL_COMPARISON_2026-03-08.md`](../MODEL_COMPARISON_2026-03-08.md)
|
||||
|
||||
## Prerequis
|
||||
|
||||
- `mascarade` repond sur `http://127.0.0.1:8100/health`
|
||||
- le modele reste explicite via `ANE_MODEL` ou `--model`
|
||||
- une intention existe pour le chapitre cible
|
||||
- le garde-fou manuscrit et la boucle `repair` peuvent bloquer la promotion meme avec `--approve`
|
||||
|
||||
## Cycle automatise
|
||||
|
||||
Commande de reference:
|
||||
|
||||
```bash
|
||||
python3 scripts/run_next_lots.py --lot full
|
||||
```
|
||||
|
||||
Commandes utiles:
|
||||
|
||||
```bash
|
||||
python3 scripts/run_next_lots.py --lot priority_models
|
||||
python3 scripts/run_next_lots.py --resume automation/state/next_lots_state.json
|
||||
python3 scripts/run_next_lots.py --lot tracking_sync --report-only
|
||||
```
|
||||
|
||||
Le driver:
|
||||
- lit `automation/next_lots.toml`
|
||||
- rejoue preflights et smokes dans l'ordre utile du moment
|
||||
- met a jour les sections `AUTO-SYNC` des TODOs, plans, README et runbooks
|
||||
- attend brievement que `/models` reflète le bon `model_id` apres un switch Apple avant de recréer un checkpoint manuel
|
||||
- s'arrete proprement si un restart runtime ou un switch Apple est requis, puis imprime la commande de reprise
|
||||
|
||||
## Contrat local
|
||||
|
||||
`ai-novel-engine` parle uniquement a `mascarade` sur `POST /v1/chat/completions`.
|
||||
|
||||
- format modele: `provider:model`
|
||||
- dernier cycle complet termine au 9 mars 2026:
|
||||
- `apple-coreml:qwen3.5-4b-onnx-q4f16` est `accepted` de bout en bout sous garde-fou
|
||||
- `ollama:qwen2.5:7b` atteint `gate`, exerce `repair` en live, puis finit `quality_blocked` sur `outline_like`
|
||||
- `apple-coreml:stateful-mistral7b-instruct-int4-coreml` reste `preflight_only`
|
||||
- le lot `baselines` pour `apple-coreml:qwen2.5-0.5b-instruct-onnx` et `ollama:qwen2.5:1.5b` est rejoue separement
|
||||
- le runtime Apple local ne sert qu'un seul `model_id` a la fois
|
||||
- le fallback `repair` n'essaie plus de changer de modele `apple-coreml` en plein smoke; tout switch Apple reste une action runtime explicite
|
||||
|
||||
## Smoke Apple
|
||||
|
||||
Preflight minimal cote runtime:
|
||||
|
||||
```bash
|
||||
bash /Users/electron/mascarade/scripts/smoke_openai_compat_ane.sh \
|
||||
--url http://127.0.0.1:8100 \
|
||||
--model "apple-coreml:qwen3.5-4b-onnx-q4f16"
|
||||
```
|
||||
|
||||
Smoke chapitre complet:
|
||||
|
||||
```bash
|
||||
./scripts/smoke_local_generation.sh \
|
||||
--base-url http://127.0.0.1:8100 \
|
||||
--model "apple-coreml:qwen3.5-4b-onnx-q4f16" \
|
||||
--approve
|
||||
```
|
||||
|
||||
Notes:
|
||||
- le script fait un warm-up automatique via `:8100` quand le modele commence par `apple-coreml:`
|
||||
- le premier chargement Core ML peut etre long
|
||||
- le smoke Apple applique par defaut un timeout plus large et des budgets plus courts; utiliser `--timeout` ou `ANE_MAX_TOKENS_*` pour durcir ou assouplir
|
||||
- `ANE_MAX_TOKENS_GATE` permet de regler le budget du garde-fou LLM
|
||||
- `ANE_MAX_TOKENS_REPAIR` et `ANE_REPAIR_MAX_PASSES` reglent la boucle `repair`
|
||||
- `apple-coreml:qwen2.5-0.5b-instruct-onnx` reste le candidat vitesse Apple a requalifier en baseline
|
||||
- `apple-coreml:qwen3.5-4b-onnx-q4f16` est la reference Apple locale actuelle
|
||||
- `apple-coreml:stateful-mistral7b-instruct-int4-coreml` reste preflight-only sur cette machine: il repond, mais le smoke ANE est reste bloque a `structure` pendant plus de 8 minutes
|
||||
|
||||
## Smoke Ollama
|
||||
|
||||
Preflight:
|
||||
|
||||
```bash
|
||||
bash /Users/electron/mascarade/scripts/smoke_openai_compat_ane.sh \
|
||||
--url http://127.0.0.1:8100 \
|
||||
--model "ollama:qwen2.5:1.5b"
|
||||
```
|
||||
|
||||
Le provider `ollama` doit apparaitre dans `providers` et le modele cible doit etre deja installe.
|
||||
Sur cette machine, le meilleur candidat Ollama courant est `ollama:qwen2.5:7b`; `qwen2.5:1.5b` reste une baseline a rerun.
|
||||
|
||||
Smoke chapitre complet:
|
||||
|
||||
```bash
|
||||
./scripts/smoke_local_generation.sh \
|
||||
--base-url http://127.0.0.1:8100 \
|
||||
--model "ollama:qwen2.5:1.5b" \
|
||||
--approve
|
||||
```
|
||||
|
||||
## Lire rapidement le resultat
|
||||
|
||||
Le smoke affiche un resume humain:
|
||||
|
||||
- `backend`
|
||||
- `chapter`
|
||||
- `status`
|
||||
- `accepted`
|
||||
- `failed_stage` si present
|
||||
- `quality_blockers` si presents
|
||||
- `retry_stages` si present
|
||||
- `repair_attempts` et `repair_models` si presents
|
||||
- chemins vers `draft_v2`, `repair_latest`, `critique_v1`, `gate_v1`, `manuscript`, `memory_summary`, `meta.json`
|
||||
|
||||
Le fichier de reference reste:
|
||||
|
||||
```bash
|
||||
cat brouillons/chapitres/chapitre_XX/meta.json
|
||||
```
|
||||
|
||||
Champs utiles:
|
||||
- `status`
|
||||
- `last_status_message`
|
||||
- `stage_attempts`
|
||||
- `retry_stages`
|
||||
- `failed_stage`
|
||||
- `quality_blockers`
|
||||
- `repair_attempts`
|
||||
- `repair_models`
|
||||
- `gate_report`
|
||||
- `provider.base_url`
|
||||
- `provider.model`
|
||||
|
||||
## Etat auto-synchronise
|
||||
## Etat auto-synchronise
|
||||
<!-- AUTO-SYNC:ANE-RUNBOOK:START -->
|
||||
- dernier cycle automatise: 2026-03-09T06:53:02+00:00
|
||||
- chapitre courant detecte: chapitre_01
|
||||
- reference locale actuelle: aucun accepted, meilleur diagnostic: apple-coreml:qwen2.5-0.5b-instruct-onnx
|
||||
- prochain lot utile: Analyser les runs ayant atteint gate/repair puis resserrer la reference locale autour des meilleurs candidats.
|
||||
- reprise attendue apres action manuelle: /Users/electron/Documents/Projets_Creatifs/ai-novel-engine/automation/state/next_lots_state.json
|
||||
<!-- AUTO-SYNC:ANE-RUNBOOK:END -->
|
||||
+45
-1
@@ -1 +1,45 @@
|
||||
Vision du projet AI Novel Engine.
|
||||
# Vision AI Novel Engine
|
||||
|
||||
## Positionnement
|
||||
|
||||
AI Novel Engine est un moteur narratif strict, local-first, pour projets longs.
|
||||
|
||||
Le but n'est pas de "discuter avec un chatbot qui écrit un roman". Le but est
|
||||
de garder un pipeline lisible, reproductible et contrôlable par l'auteur:
|
||||
|
||||
`intention -> structure -> draft -> critique -> rewrite -> gate -> validation -> memoire`
|
||||
|
||||
## Ce que porte le produit
|
||||
|
||||
- l'auteur reste decisionnaire a chaque promotion vers le manuscrit
|
||||
- aucune generation sans intention explicite
|
||||
- aucune promotion vers le manuscrit si le garde-fou qualite bloque
|
||||
- la memoire reste externe, inspectable et persistée sur disque
|
||||
- les artefacts intermediaires sont lisibles en Markdown et JSON
|
||||
- le moteur narratif reste decouple du runtime local
|
||||
|
||||
## Architecture cible
|
||||
|
||||
- `ai-novel-engine` porte la logique auteuriale, le pipeline, les prompts et la mémoire
|
||||
- `mascarade` porte le runtime local, le routage provider et le shim OpenAI-compatible
|
||||
- le contrat entre les deux reste minimal:
|
||||
- `POST /v1/chat/completions`
|
||||
- `model=provider:model`
|
||||
- non-streaming
|
||||
- JSON best effort, avec reessai applicatif cote ANE
|
||||
|
||||
## Non-objectifs v1
|
||||
|
||||
- chat libre comme interface principale
|
||||
- studio web riche ou collaboratif
|
||||
- autonomie complete "idee -> manuscrit final"
|
||||
- base de donnees opaque pour la mémoire
|
||||
|
||||
## Critere de valeur
|
||||
|
||||
Le systeme est utile si un auteur peut:
|
||||
|
||||
- relancer un chapitre sans perdre le contexte de travail
|
||||
- comprendre pourquoi une etape a echoue
|
||||
- changer de backend local sans rewriter le pipeline narratif
|
||||
- relire les brouillons, critiques et mises a jour memoire hors de l'IA
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
Tu es le rôle Contrôle du moteur AI Novel Engine.
|
||||
La tentative précédente n'a pas produit un JSON exploitable.
|
||||
Réponds à nouveau avec un seul objet JSON valide.
|
||||
Ne mets aucun texte avant ou après le JSON.
|
||||
Ne mets aucun bloc Markdown.
|
||||
Si la tentative précédente est inutilisable, regénère le diagnostic à partir du contexte.
|
||||
Limite-toi à 1 phrase de résumé, 3 écarts max et 3 recommandations max.
|
||||
|
||||
Chapitre cible: $chapter_slug
|
||||
|
||||
Erreur de parsing observée:
|
||||
$parse_error
|
||||
|
||||
Tentative précédente à corriger si possible:
|
||||
$invalid_response
|
||||
|
||||
Intention:
|
||||
$intention
|
||||
|
||||
Structure attendue:
|
||||
$structure_markdown
|
||||
|
||||
Brouillon à critiquer:
|
||||
$draft_markdown
|
||||
|
||||
Format JSON strict:
|
||||
{
|
||||
"summary": "résumé bref du diagnostic",
|
||||
"rewrite_required": true,
|
||||
"deviations": ["écart 1", "écart 2"],
|
||||
"recommendations": ["recommandation 1", "recommandation 2"]
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
Tu es le rôle Contrôle du moteur AI Novel Engine.
|
||||
Analyse le brouillon et renvoie uniquement un objet JSON valide.
|
||||
Réponse compacte obligatoire.
|
||||
Ne mets aucun texte avant ou après le JSON.
|
||||
Ne mets aucun bloc Markdown.
|
||||
Limite-toi à 1 phrase de résumé, 3 écarts max et 3 recommandations max.
|
||||
|
||||
Chapitre cible: $chapter_slug
|
||||
|
||||
Intention:
|
||||
$intention
|
||||
|
||||
Structure attendue:
|
||||
$structure_markdown
|
||||
|
||||
Brouillon à critiquer:
|
||||
$draft_markdown
|
||||
|
||||
Format JSON strict:
|
||||
{
|
||||
"summary": "résumé bref du diagnostic",
|
||||
"rewrite_required": true,
|
||||
"deviations": ["écart 1", "écart 2"],
|
||||
"recommendations": ["recommandation 1", "recommandation 2"]
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
Tu es le rôle Production du moteur AI Novel Engine.
|
||||
Tu rédiges un brouillon de chapitre fidèle à l'intention et à la structure.
|
||||
|
||||
Chapitre cible: $chapter_slug
|
||||
|
||||
Intention:
|
||||
$intention
|
||||
|
||||
Structure validée:
|
||||
$structure_markdown
|
||||
|
||||
Contexte projet:
|
||||
$story_context
|
||||
|
||||
Consignes:
|
||||
- répondre uniquement avec le chapitre en Markdown
|
||||
- produire uniquement de la prose narrative continue, sous forme de paragraphes
|
||||
- ne jamais recopier la structure sous forme de plan
|
||||
- interdit: titres Markdown (`#`, `##`, `###`), listes a puces, numerotations, labels `objectif`, `conflit`, `sortie`, section `Tension`, section `Scènes`, code fences
|
||||
- garder une voix cohérente
|
||||
- matérialiser la tension annoncée
|
||||
- ouvrir directement dans l'action ou dans la scene, sans preambule meta
|
||||
- transformer chaque beat de structure en action, perception, decision, consequence et, si utile, dialogue
|
||||
- finir sur une vraie phrase complete avec une ponctuation finale
|
||||
- viser un chapitre bref mais complet, d'au moins 3 paragraphes substantiels
|
||||
- ne pas ajouter d'explication hors texte narratif
|
||||
@@ -0,0 +1,34 @@
|
||||
Tu es le rôle Garde-fou manuscrit du moteur AI Novel Engine.
|
||||
La tentative precedente n'a pas produit un JSON exploitable.
|
||||
Reponds a nouveau avec un seul objet JSON valide.
|
||||
Ne mets aucun texte avant ou apres le JSON.
|
||||
Ne mets aucun bloc Markdown.
|
||||
Si la tentative precedente est inutilisable, regénere le diagnostic a partir du brouillon final.
|
||||
Bloque si le texte ressemble encore a un plan, s'il semble coupe avant sa fin, ou s'il manque une vraie continuite narrative.
|
||||
Limite-toi a 1 phrase de resume, 4 blockers max et 4 recommandations max.
|
||||
|
||||
Chapitre cible: $chapter_slug
|
||||
|
||||
Erreur de parsing observee:
|
||||
$parse_error
|
||||
|
||||
Tentative precedente a corriger si possible:
|
||||
$invalid_response
|
||||
|
||||
Intention:
|
||||
$intention
|
||||
|
||||
Structure attendue:
|
||||
$structure_markdown
|
||||
|
||||
Brouillon final:
|
||||
$draft_markdown
|
||||
|
||||
Format JSON strict:
|
||||
{
|
||||
"ready_for_manuscript": true,
|
||||
"summary": "diagnostic bref",
|
||||
"blockers": ["outline_like"],
|
||||
"recommendations": ["retirer les titres", "terminer la scene"],
|
||||
"heuristic_blockers": []
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
Tu es le rôle Garde-fou manuscrit du moteur AI Novel Engine.
|
||||
Analyse le brouillon final et renvoie uniquement un objet JSON valide.
|
||||
Ne mets aucun texte avant ou après le JSON.
|
||||
Ne mets aucun bloc Markdown.
|
||||
Le but est de decider si ce texte peut etre promu dans le manuscrit.
|
||||
Bloque si le texte ressemble encore a un plan, s'il semble coupe avant sa fin, ou s'il manque une vraie continuite narrative.
|
||||
Limite-toi a 1 phrase de resume, 4 blockers max et 4 recommandations max.
|
||||
|
||||
Chapitre cible: $chapter_slug
|
||||
|
||||
Intention:
|
||||
$intention
|
||||
|
||||
Structure attendue:
|
||||
$structure_markdown
|
||||
|
||||
Brouillon final:
|
||||
$draft_markdown
|
||||
|
||||
Format JSON strict:
|
||||
{
|
||||
"ready_for_manuscript": true,
|
||||
"summary": "diagnostic bref",
|
||||
"blockers": ["outline_like"],
|
||||
"recommendations": ["retirer les titres", "terminer la scene"],
|
||||
"heuristic_blockers": []
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
Tu es le rôle Mémoire du moteur AI Novel Engine.
|
||||
La tentative précédente n'a pas produit un JSON exploitable.
|
||||
Réponds à nouveau avec un seul objet JSON valide.
|
||||
Ne mets aucun texte avant ou après le JSON.
|
||||
Ne mets aucun bloc Markdown.
|
||||
Si la tentative précédente est inutilisable, regénère la mémoire à partir du chapitre accepté.
|
||||
Limite-toi à 1 résumé, 3 personnages max, 3 lieux max et 5 événements max.
|
||||
|
||||
Chapitre cible: $chapter_slug
|
||||
|
||||
Erreur de parsing observée:
|
||||
$parse_error
|
||||
|
||||
Tentative précédente à corriger si possible:
|
||||
$invalid_response
|
||||
|
||||
Contexte projet:
|
||||
$story_context
|
||||
|
||||
Chapitre accepté:
|
||||
$accepted_draft
|
||||
|
||||
Format JSON strict:
|
||||
{
|
||||
"summary": "résumé factuel du chapitre",
|
||||
"characters": [
|
||||
{"name": "Nom", "description": "Rôle ou évolution"}
|
||||
],
|
||||
"locations": [
|
||||
{"name": "Lieu", "description": "Ce qui y est établi"}
|
||||
],
|
||||
"timeline_events": [
|
||||
{"event": "Fait important", "order_hint": "optionnel"}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
Tu es le rôle Mémoire du moteur AI Novel Engine.
|
||||
Tu extrais une mémoire exploitable à partir d'un chapitre accepté.
|
||||
Réponds uniquement avec un objet JSON valide.
|
||||
Réponse compacte obligatoire.
|
||||
Ne mets aucun texte avant ou après le JSON.
|
||||
Ne mets aucun bloc Markdown.
|
||||
Limite-toi à 1 résumé, 3 personnages max, 3 lieux max et 5 événements max.
|
||||
|
||||
Chapitre cible: $chapter_slug
|
||||
|
||||
Contexte projet:
|
||||
$story_context
|
||||
|
||||
Chapitre accepté:
|
||||
$accepted_draft
|
||||
|
||||
Format JSON strict:
|
||||
{
|
||||
"summary": "résumé factuel du chapitre",
|
||||
"characters": [
|
||||
{"name": "Nom", "description": "Rôle ou évolution"}
|
||||
],
|
||||
"locations": [
|
||||
{"name": "Lieu", "description": "Ce qui y est établi"}
|
||||
],
|
||||
"timeline_events": [
|
||||
{"event": "Fait important", "order_hint": "optionnel"}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
Tu es le rôle Réparation prose du moteur AI Novel Engine.
|
||||
Tu dois convertir un brouillon candidat bloqué par le garde-fou en un vrai chapitre lisible.
|
||||
|
||||
Chapitre cible: $chapter_slug
|
||||
Tentative de réparation: $repair_attempt
|
||||
Modèle de réparation: $repair_model
|
||||
|
||||
Intention:
|
||||
$intention
|
||||
|
||||
Structure attendue:
|
||||
$structure_markdown
|
||||
|
||||
Contexte projet:
|
||||
$story_context
|
||||
|
||||
Diagnostic du garde-fou:
|
||||
$gate_json
|
||||
|
||||
Brouillon à réparer:
|
||||
$draft_markdown
|
||||
|
||||
Consignes impératives:
|
||||
- répondre uniquement avec la nouvelle version du chapitre en Markdown
|
||||
- produire uniquement de la prose narrative continue en paragraphes
|
||||
- ne garder aucun titre, aucune puce, aucune numérotation, aucun label de plan visible
|
||||
- supprimer completement les mots `objectif`, `conflit`, `sortie`, `Scène`, `scene`, `Tension` s'ils apparaissent comme labels ou sous-titres
|
||||
- transformer toute structure, note ou checklist en scene(s) jouee(s) avec actions, perceptions et consequences
|
||||
- conserver l'intention et les informations utiles deja presentes
|
||||
- allonger si besoin pour obtenir une scene complete et continue
|
||||
- finir obligatoirement sur une vraie phrase complete avec une ponctuation finale
|
||||
- ne rien ajouter avant ou apres le chapitre
|
||||
@@ -0,0 +1,27 @@
|
||||
Tu es le rôle Réécriture du moteur AI Novel Engine.
|
||||
Tu dois produire une seconde version du chapitre à partir du brouillon et de la critique.
|
||||
|
||||
Chapitre cible: $chapter_slug
|
||||
|
||||
Intention:
|
||||
$intention
|
||||
|
||||
Structure:
|
||||
$structure_markdown
|
||||
|
||||
Brouillon initial:
|
||||
$draft_markdown
|
||||
|
||||
Critique structurée:
|
||||
$critique_json
|
||||
|
||||
Consignes de réécriture:
|
||||
- répondre uniquement avec la version réécrite du chapitre en Markdown
|
||||
- produire uniquement de la prose narrative continue, sous forme de paragraphes
|
||||
- supprimer completement tout titre, toute puce, toute numérotation et tout label de plan
|
||||
- si le brouillon ressemble a une structure ou a des notes, le convertir integralement en scene(s) racontee(s)
|
||||
- ne jamais garder les termes `objectif`, `conflit`, `sortie`, `Tension`, `Scène` comme titres ou labels visibles
|
||||
- conserver l'intention, mais augmenter la continuite dramatique d'une scene a l'autre
|
||||
- materialiser les decisions et leurs consequences
|
||||
- finir sur une vraie phrase complete avec une ponctuation finale
|
||||
- ne rien ajouter avant ou apres le chapitre
|
||||
@@ -0,0 +1,22 @@
|
||||
Tu es le rôle Structure du moteur AI Novel Engine.
|
||||
Tu dois transformer une intention d'auteur en plan de chapitre actionnable.
|
||||
|
||||
Chapitre cible: $chapter_slug
|
||||
|
||||
Intention:
|
||||
$intention
|
||||
|
||||
Contexte projet:
|
||||
$story_context
|
||||
|
||||
Réponds uniquement en Markdown avec cette structure:
|
||||
# Structure — $chapter_slug
|
||||
## Objectif dramatique
|
||||
## Tension
|
||||
## Scènes
|
||||
### Scène 1 — titre
|
||||
- objectif:
|
||||
- conflit:
|
||||
- sortie:
|
||||
|
||||
Ajoute autant de scènes que nécessaire, mais reste concret et opérable pour une génération de chapitre.
|
||||
Executable
+15
@@ -0,0 +1,15 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(REPO_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(REPO_ROOT))
|
||||
|
||||
from core.next_lots import main
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Executable
+296
@@ -0,0 +1,296 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
chapter="02"
|
||||
workspace=""
|
||||
base_url="${ANE_BASE_URL:-http://127.0.0.1:8100}"
|
||||
model="${ANE_MODEL:-}"
|
||||
timeout="${ANE_TIMEOUT:-}"
|
||||
decision="approve"
|
||||
intention="Chapitre court. Une femme arrive dans une ville de nuit, trouve un indice simple, et finit sur une decision risquee. Style direct, phrases courtes, ton sobre."
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/smoke_local_generation.sh [options]
|
||||
|
||||
Options:
|
||||
--chapter N Chapter number to generate (default: 02)
|
||||
--workspace DIR Reuse or create the smoke workspace in DIR
|
||||
--intention TEXT Override the default smoke intention
|
||||
--base-url URL OpenAI-compatible base URL (default: ANE_BASE_URL or http://127.0.0.1:8100)
|
||||
--model MODEL Explicit model in provider:model format (required unless ANE_MODEL is already set)
|
||||
--timeout SEC Provider timeout in seconds (default: 900 for apple-coreml, 300 otherwise)
|
||||
--approve Promote without interactive prompt (default)
|
||||
--reject Reject without interactive prompt
|
||||
--no-approve Deprecated alias for --reject
|
||||
-h, --help Show this help
|
||||
|
||||
Environment:
|
||||
ANE_BASE_URL Used if --base-url is not provided
|
||||
ANE_MODEL Used if --model is not provided
|
||||
ANE_TIMEOUT Default: 900 for apple-coreml, 300 otherwise
|
||||
ANE_MAX_TOKENS Default: 192 for apple-coreml, 384 otherwise
|
||||
ANE_MAX_TOKENS_STRUCTURE Default: 96 for apple-coreml, 224 otherwise
|
||||
ANE_MAX_TOKENS_DRAFT Default: 192 for apple-coreml, 384 otherwise
|
||||
ANE_MAX_TOKENS_CRITIQUE Default: 160 for apple-coreml, 512 otherwise
|
||||
ANE_MAX_TOKENS_REWRITE Default: 192 for apple-coreml, 448 otherwise
|
||||
ANE_MAX_TOKENS_GATE Default: 128 for apple-coreml, 384 otherwise
|
||||
ANE_MAX_TOKENS_REPAIR Default: 160 for apple-coreml, 512 otherwise
|
||||
ANE_MAX_TOKENS_MEMORY Default: 128 for apple-coreml, 320 otherwise
|
||||
EOF
|
||||
}
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--chapter)
|
||||
chapter="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
--workspace)
|
||||
workspace="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
--intention)
|
||||
intention="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
--base-url)
|
||||
base_url="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
--model)
|
||||
model="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
--timeout)
|
||||
timeout="${2:-}"
|
||||
shift 2
|
||||
;;
|
||||
--approve)
|
||||
decision="approve"
|
||||
shift
|
||||
;;
|
||||
--reject|--no-approve)
|
||||
decision="reject"
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option: $1" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z "${model}" ]]; then
|
||||
echo "ANE model required. Pass --model provider:model or export ANE_MODEL." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if [[ -z "${workspace}" ]]; then
|
||||
workspace="$(mktemp -d "${TMPDIR:-/tmp}/ane-local-smoke.XXXXXX")"
|
||||
fi
|
||||
|
||||
mkdir -p "${workspace}"
|
||||
|
||||
if [[ -z "${timeout}" ]]; then
|
||||
if [[ "${model}" == apple-coreml:* ]]; then
|
||||
timeout="900"
|
||||
else
|
||||
timeout="300"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ "${model}" == apple-coreml:* ]]; then
|
||||
export ANE_MAX_TOKENS="${ANE_MAX_TOKENS:-192}"
|
||||
export ANE_MAX_TOKENS_STRUCTURE="${ANE_MAX_TOKENS_STRUCTURE:-96}"
|
||||
export ANE_MAX_TOKENS_DRAFT="${ANE_MAX_TOKENS_DRAFT:-192}"
|
||||
export ANE_MAX_TOKENS_CRITIQUE="${ANE_MAX_TOKENS_CRITIQUE:-160}"
|
||||
export ANE_MAX_TOKENS_REWRITE="${ANE_MAX_TOKENS_REWRITE:-192}"
|
||||
export ANE_MAX_TOKENS_GATE="${ANE_MAX_TOKENS_GATE:-128}"
|
||||
export ANE_MAX_TOKENS_REPAIR="${ANE_MAX_TOKENS_REPAIR:-160}"
|
||||
export ANE_MAX_TOKENS_MEMORY="${ANE_MAX_TOKENS_MEMORY:-128}"
|
||||
else
|
||||
export ANE_MAX_TOKENS="${ANE_MAX_TOKENS:-384}"
|
||||
export ANE_MAX_TOKENS_STRUCTURE="${ANE_MAX_TOKENS_STRUCTURE:-224}"
|
||||
export ANE_MAX_TOKENS_DRAFT="${ANE_MAX_TOKENS_DRAFT:-384}"
|
||||
export ANE_MAX_TOKENS_CRITIQUE="${ANE_MAX_TOKENS_CRITIQUE:-512}"
|
||||
export ANE_MAX_TOKENS_REWRITE="${ANE_MAX_TOKENS_REWRITE:-448}"
|
||||
export ANE_MAX_TOKENS_GATE="${ANE_MAX_TOKENS_GATE:-384}"
|
||||
export ANE_MAX_TOKENS_REPAIR="${ANE_MAX_TOKENS_REPAIR:-512}"
|
||||
export ANE_MAX_TOKENS_MEMORY="${ANE_MAX_TOKENS_MEMORY:-320}"
|
||||
fi
|
||||
|
||||
export ANE_PROVIDER="${ANE_PROVIDER:-openai_compatible}"
|
||||
export ANE_BASE_URL="${base_url}"
|
||||
export ANE_MODEL="${model}"
|
||||
export ANE_TIMEOUT="${timeout}"
|
||||
|
||||
warmup_openai_compatible() {
|
||||
python3 - <<'PY'
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import sys
|
||||
import urllib.request
|
||||
|
||||
base_url = os.environ["ANE_BASE_URL"].rstrip("/")
|
||||
if base_url.endswith("/v1"):
|
||||
url = f"{base_url}/chat/completions"
|
||||
elif base_url.endswith("/chat/completions"):
|
||||
url = base_url
|
||||
else:
|
||||
url = f"{base_url}/v1/chat/completions"
|
||||
|
||||
payload = {
|
||||
"model": os.environ["ANE_MODEL"],
|
||||
"messages": [{"role": "user", "content": "Respond with exactly: ok"}],
|
||||
"temperature": 0,
|
||||
"max_tokens": 8,
|
||||
}
|
||||
body = json.dumps(payload).encode("utf-8")
|
||||
headers = {"Content-Type": "application/json"}
|
||||
api_key = os.environ.get("ANE_API_KEY", "").strip()
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
|
||||
request = urllib.request.Request(url, data=body, headers=headers, method="POST")
|
||||
try:
|
||||
with urllib.request.urlopen(request, timeout=float(os.environ.get("ANE_TIMEOUT", "300"))) as response:
|
||||
payload = json.loads(response.read().decode("utf-8"))
|
||||
except (TimeoutError, socket.timeout) as exc:
|
||||
print(f"Warm-up OpenAI-compatible failed: timeout after {os.environ.get('ANE_TIMEOUT', '300')}s", file=sys.stderr)
|
||||
raise SystemExit(1) from exc
|
||||
|
||||
content = payload["choices"][0]["message"]["content"]
|
||||
print(f"Warm-up OpenAI-compatible OK: {content}")
|
||||
PY
|
||||
}
|
||||
|
||||
prepare_workspace() {
|
||||
python3 - <<'PY'
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import os
|
||||
|
||||
from core.chapters import ChapterId
|
||||
|
||||
root = Path(os.environ["ANE_SMOKE_ROOT"])
|
||||
chapter_id = ChapterId.parse(os.environ["ANE_SMOKE_CHAPTER"])
|
||||
intention = os.environ["ANE_SMOKE_INTENTION"].strip()
|
||||
|
||||
intentions_dir = root / "notes" / "intentions"
|
||||
intentions_dir.mkdir(parents=True, exist_ok=True)
|
||||
intention_path = intentions_dir / chapter_id.filename
|
||||
if not intention_path.exists():
|
||||
intention_path.write_text(
|
||||
f"# Intention — Chapitre {chapter_id.label}\n\n{intention}\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
PY
|
||||
}
|
||||
|
||||
print_summary() {
|
||||
python3 - <<'PY'
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
from core.chapters import ChapterId
|
||||
|
||||
root = Path(os.environ["ANE_SMOKE_ROOT"])
|
||||
chapter_id = ChapterId.parse(os.environ["ANE_SMOKE_CHAPTER"])
|
||||
meta_path = root / "brouillons" / "chapitres" / chapter_id.slug / "meta.json"
|
||||
|
||||
print("")
|
||||
print("Smoke summary")
|
||||
print(f"- workspace: {root}")
|
||||
print(f"- model: {os.environ['ANE_MODEL']}")
|
||||
print(f"- chapter: {chapter_id.slug}")
|
||||
|
||||
if not meta_path.exists():
|
||||
print("- meta: absent")
|
||||
sys.exit(0)
|
||||
|
||||
meta = json.loads(meta_path.read_text(encoding="utf-8"))
|
||||
print(f"- status: {meta.get('status')}")
|
||||
print(f"- accepted: {meta.get('accepted')}")
|
||||
if meta.get("failed_stage"):
|
||||
print(f"- failed_stage: {meta.get('failed_stage')}")
|
||||
quality_blockers = meta.get("quality_blockers") or []
|
||||
if quality_blockers:
|
||||
print(f"- quality_blockers: {', '.join(str(item) for item in quality_blockers)}")
|
||||
retry_stages = meta.get("retry_stages") or []
|
||||
if retry_stages:
|
||||
print(f"- retry_stages: {', '.join(str(item) for item in retry_stages)}")
|
||||
print(f"- repair_attempts: {meta.get('repair_attempts', 0)}")
|
||||
repair_models = meta.get("repair_models") or []
|
||||
if repair_models:
|
||||
print(f"- repair_models: {', '.join(str(item) for item in repair_models)}")
|
||||
print(f"- last_status_message: {meta.get('last_status_message', '')}")
|
||||
artifacts = meta.get("artifacts", {})
|
||||
if isinstance(artifacts, dict):
|
||||
draft_path = artifacts.get("repair_latest") or artifacts.get("draft_v2")
|
||||
if draft_path:
|
||||
print(f"- draft_path: {draft_path}")
|
||||
for label, key in (
|
||||
("draft_v2", "draft_v2"),
|
||||
("critique_v1", "critique_v1"),
|
||||
("repair_latest", "repair_latest"),
|
||||
("gate_v1", "gate_v1"),
|
||||
("manuscript", "manuscript"),
|
||||
("memory_summary", "memory_summary"),
|
||||
):
|
||||
value = artifacts.get(key)
|
||||
if value:
|
||||
print(f"- {label}: {value}")
|
||||
print(f"- meta: {meta_path}")
|
||||
PY
|
||||
}
|
||||
|
||||
export ANE_SMOKE_ROOT="${workspace}"
|
||||
export ANE_SMOKE_CHAPTER="${chapter}"
|
||||
export ANE_SMOKE_INTENTION="${intention}"
|
||||
|
||||
cd "${REPO_DIR}"
|
||||
prepare_workspace
|
||||
|
||||
if [[ "${ANE_MODEL}" == apple-coreml:* ]]; then
|
||||
echo "Warm-up Apple runtime via ${ANE_BASE_URL} ..."
|
||||
warmup_openai_compatible
|
||||
fi
|
||||
|
||||
decision_flag="--approve"
|
||||
if [[ "${decision}" == "reject" ]]; then
|
||||
decision_flag="--reject"
|
||||
fi
|
||||
|
||||
set +e
|
||||
cli_output="$(
|
||||
cd "${workspace}" && \
|
||||
PYTHONPATH="${REPO_DIR}${PYTHONPATH:+:${PYTHONPATH}}" \
|
||||
python3 -m cli.main generate chapter --chapter "${chapter}" "${decision_flag}" 2>&1
|
||||
)"
|
||||
cli_exit=$?
|
||||
set -e
|
||||
|
||||
printf '%s\n' "${cli_output}"
|
||||
print_summary
|
||||
|
||||
if [[ ${cli_exit} -ne 0 ]]; then
|
||||
exit "${cli_exit}"
|
||||
fi
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,300 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
from core.chapters import ChapterId
|
||||
from core.next_lots import (
|
||||
AUTO_SYNC_TODO_ACTIVE,
|
||||
CommandResult,
|
||||
Manifest,
|
||||
ModelRunResult,
|
||||
NextLotsRunner,
|
||||
RunState,
|
||||
replace_auto_section,
|
||||
)
|
||||
|
||||
|
||||
class NextLotsTests(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self.temp_dir = tempfile.TemporaryDirectory()
|
||||
self.root = Path(self.temp_dir.name) / "ane"
|
||||
self.root.mkdir(parents=True, exist_ok=True)
|
||||
self.mascarade = Path(self.temp_dir.name) / "mascarade"
|
||||
self.mascarade.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
for path in (
|
||||
self.root / "README.md",
|
||||
self.root / "TODO_ACTIVE.md",
|
||||
self.root / "TODO_IMPLEMENTE.md",
|
||||
self.root / "docs" / "EXECUTION_PLAN_2026-03-08.md",
|
||||
self.root / "docs" / "MODEL_COMPARISON_2026-03-08.md",
|
||||
self.root / "docs" / "runbooks" / "LOCAL_GENERATION.md",
|
||||
self.mascarade / "README.md",
|
||||
self.mascarade / "TODO_AI_NOVEL_ENGINE.md",
|
||||
self.mascarade / "docs" / "EXECUTION_PLAN_2026-03-08.md",
|
||||
self.mascarade / "docs" / "RUNBOOK_APPLE_LLM_LOCAL.md",
|
||||
):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(f"# {path.name}\n", encoding="utf-8")
|
||||
|
||||
manifest_dir = self.root / "automation"
|
||||
manifest_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.manifest_path = manifest_dir / "next_lots.toml"
|
||||
self.manifest_path.write_text(
|
||||
(
|
||||
"[paths]\n"
|
||||
f"mascarade_repo = \"{self.mascarade}\"\n"
|
||||
"core_base_url = \"http://127.0.0.1:8100\"\n"
|
||||
"apple_runtime_url = \"http://127.0.0.1:8201\"\n"
|
||||
"ollama_tags_url = \"http://127.0.0.1:11435/api/tags\"\n\n"
|
||||
"apple_model_ready_timeout_seconds = 0\n"
|
||||
"apple_model_poll_interval_seconds = 0.01\n\n"
|
||||
"[smoke]\n"
|
||||
"chapter = \"02\"\n"
|
||||
"intention = \"Smoke intention\"\n"
|
||||
"timeout_seconds = 300\n\n"
|
||||
"[preset]\n"
|
||||
"ANE_MAX_TOKENS_STRUCTURE = \"256\"\n"
|
||||
"ANE_REPAIR_MAX_PASSES = \"2\"\n\n"
|
||||
"[ensure_models]\n"
|
||||
"apple_models = [\"qwen2.5-0.5b-instruct-onnx\", \"qwen3.5-4b-onnx-q4f16\", \"stateful-mistral7b-instruct-int4-coreml\"]\n"
|
||||
"ollama_models = [\"qwen2.5:7b\", \"qwen2.5:1.5b\"]\n\n"
|
||||
"[lots.priority_models]\n"
|
||||
"models = [\"apple-coreml:qwen3.5-4b-onnx-q4f16\", \"ollama:qwen2.5:7b\"]\n\n"
|
||||
"[lots.baselines]\n"
|
||||
"models = [\"apple-coreml:qwen2.5-0.5b-instruct-onnx\", \"ollama:qwen2.5:1.5b\"]\n\n"
|
||||
"[lots.preflight_only]\n"
|
||||
"models = [\"apple-coreml:stateful-mistral7b-instruct-int4-coreml\"]\n\n"
|
||||
"[tracking.ane]\n"
|
||||
"todo_active = \"TODO_ACTIVE.md\"\n"
|
||||
"todo_done = \"TODO_IMPLEMENTE.md\"\n"
|
||||
"plan = \"docs/EXECUTION_PLAN_2026-03-08.md\"\n"
|
||||
"comparison = \"docs/MODEL_COMPARISON_2026-03-08.md\"\n"
|
||||
"readme = \"README.md\"\n"
|
||||
"runbook = \"docs/runbooks/LOCAL_GENERATION.md\"\n\n"
|
||||
"[tracking.mascarade]\n"
|
||||
"todo = \"TODO_AI_NOVEL_ENGINE.md\"\n"
|
||||
"plan = \"docs/EXECUTION_PLAN_2026-03-08.md\"\n"
|
||||
"readme = \"README.md\"\n"
|
||||
"runbook = \"docs/RUNBOOK_APPLE_LLM_LOCAL.md\"\n\n"
|
||||
"[next_actions]\n"
|
||||
"rewrite_compaction = \"Compacter rewrite\"\n"
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
def tearDown(self) -> None:
|
||||
self.temp_dir.cleanup()
|
||||
|
||||
def test_manifest_loads_tracking_and_models(self) -> None:
|
||||
manifest = Manifest.load(self.root, self.manifest_path)
|
||||
|
||||
self.assertEqual(manifest.priority_models, ["apple-coreml:qwen3.5-4b-onnx-q4f16", "ollama:qwen2.5:7b"])
|
||||
self.assertEqual(manifest.required_apple_models[0], "qwen2.5-0.5b-instruct-onnx")
|
||||
self.assertEqual(manifest.tracking.mascarade_repo, self.mascarade)
|
||||
self.assertEqual(manifest.tracking.ane_todo_active, self.root / "TODO_ACTIVE.md")
|
||||
self.assertEqual(manifest.apple_model_ready_timeout_seconds, 0)
|
||||
|
||||
def test_replace_auto_section_only_replaces_managed_block(self) -> None:
|
||||
path = self.root / "TODO_ACTIVE.md"
|
||||
path.write_text(
|
||||
"# Manual\n\n"
|
||||
"Avant.\n\n"
|
||||
"## Auto-sync\n"
|
||||
"<!-- AUTO-SYNC:ANE-TODO-ACTIVE:START -->\n"
|
||||
"ancien\n"
|
||||
"<!-- AUTO-SYNC:ANE-TODO-ACTIVE:END -->\n\n"
|
||||
"Apres.\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
replace_auto_section(path, AUTO_SYNC_TODO_ACTIVE, "## Auto-sync", "- nouveau")
|
||||
rendered = path.read_text(encoding="utf-8")
|
||||
|
||||
self.assertIn("Avant.", rendered)
|
||||
self.assertIn("Apres.", rendered)
|
||||
self.assertIn("- nouveau", rendered)
|
||||
self.assertNotIn("ancien", rendered)
|
||||
|
||||
def test_runner_creates_checkpoint_when_apple_model_differs(self) -> None:
|
||||
manifest = Manifest.load(self.root, self.manifest_path)
|
||||
prepare_calls: list[list[str]] = []
|
||||
|
||||
def command_runner(args: list[str], cwd: Path, env=None) -> CommandResult:
|
||||
if "prepare_runtime_step.sh" in " ".join(args):
|
||||
prepare_calls.append(args)
|
||||
return CommandResult(args=args, returncode=0, stdout="prepared", stderr="", duration_seconds=0.1)
|
||||
|
||||
def json_fetcher(url: str, timeout: float):
|
||||
if url.endswith("/health"):
|
||||
return {"status": "ok"}
|
||||
if url.endswith("/models"):
|
||||
return ["qwen2.5-0.5b-instruct-onnx"]
|
||||
raise AssertionError(url)
|
||||
|
||||
runner = NextLotsRunner(manifest, command_runner=command_runner, json_fetcher=json_fetcher)
|
||||
exit_code = runner.run(lot="priority_models")
|
||||
|
||||
self.assertEqual(exit_code, 3)
|
||||
self.assertEqual(len(prepare_calls), 1)
|
||||
self.assertIn("--apple-model", prepare_calls[0])
|
||||
state = RunState.load(self.root / "automation" / "state" / "next_lots_state.json")
|
||||
self.assertIsNotNone(state.pending_manual_action)
|
||||
self.assertIn("qwen3.5-4b-onnx-q4f16", state.pending_manual_action["reason"])
|
||||
|
||||
def test_runner_waits_for_apple_model_before_checkpointing(self) -> None:
|
||||
manifest = Manifest.load(self.root, self.manifest_path)
|
||||
manifest = Manifest(
|
||||
**{
|
||||
**manifest.__dict__,
|
||||
"apple_model_ready_timeout_seconds": 0.05,
|
||||
"apple_model_poll_interval_seconds": 0.0,
|
||||
}
|
||||
)
|
||||
prepare_calls: list[list[str]] = []
|
||||
model_calls = {"count": 0}
|
||||
|
||||
def command_runner(args: list[str], cwd: Path, env=None) -> CommandResult:
|
||||
if "prepare_runtime_step.sh" in " ".join(args):
|
||||
prepare_calls.append(args)
|
||||
return CommandResult(args=args, returncode=0, stdout="prepared", stderr="", duration_seconds=0.1)
|
||||
|
||||
def json_fetcher(url: str, timeout: float):
|
||||
if url.endswith("/health"):
|
||||
return {"status": "ok"}
|
||||
if url.endswith("/models"):
|
||||
model_calls["count"] += 1
|
||||
if model_calls["count"] == 1:
|
||||
return {"models": []}
|
||||
return ["qwen3.5-4b-onnx-q4f16"]
|
||||
raise AssertionError(url)
|
||||
|
||||
runner = NextLotsRunner(manifest, command_runner=command_runner, json_fetcher=json_fetcher)
|
||||
checkpoint = runner._checkpoint_if_runtime_manual_step_needed(
|
||||
RunState.new(
|
||||
manifest,
|
||||
lot="priority_models",
|
||||
report_dir=self.root / "automation" / "reports" / "sync",
|
||||
state_path=self.root / "automation" / "state" / "next_lots_state.json",
|
||||
steps=[{"type": "models", "name": "priority_models", "models": manifest.priority_models, "preflight_only": False}],
|
||||
),
|
||||
"apple-coreml:qwen3.5-4b-onnx-q4f16",
|
||||
)
|
||||
|
||||
self.assertIsNone(checkpoint)
|
||||
self.assertEqual(prepare_calls, [])
|
||||
self.assertGreaterEqual(model_calls["count"], 2)
|
||||
|
||||
def test_run_model_classifies_accepted_from_meta(self) -> None:
|
||||
manifest = Manifest.load(self.root, self.manifest_path)
|
||||
chapter = ChapterId.parse("02")
|
||||
|
||||
def command_runner(args: list[str], cwd: Path, env=None) -> CommandResult:
|
||||
if "smoke_openai_compat_ane.sh" in " ".join(args):
|
||||
return CommandResult(args=args, returncode=0, stdout="ok", stderr="", duration_seconds=0.2)
|
||||
if "smoke_local_generation.sh" in " ".join(args):
|
||||
workspace = Path(args[args.index("--workspace") + 1])
|
||||
meta_path = workspace / "brouillons" / "chapitres" / chapter.slug / "meta.json"
|
||||
meta_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
meta_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"status": "accepted",
|
||||
"accepted": True,
|
||||
"completed_stages": ["structure", "draft", "critique", "rewrite", "gate", "memory"],
|
||||
"retry_stages": ["gate"],
|
||||
"repair_attempts": 1,
|
||||
"repair_models": ["ollama:qwen2.5:7b"],
|
||||
"artifacts": {
|
||||
"repair_latest": str(meta_path.parent / "repair_v1.md"),
|
||||
"gate_v1": str(meta_path.parent / "gate_v1.json"),
|
||||
"manuscript": str(workspace / "manuscrit" / chapter.filename),
|
||||
},
|
||||
},
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
)
|
||||
+ "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
return CommandResult(args=args, returncode=0, stdout="smoke ok", stderr="", duration_seconds=1.5)
|
||||
raise AssertionError(args)
|
||||
|
||||
runner = NextLotsRunner(
|
||||
manifest,
|
||||
command_runner=command_runner,
|
||||
json_fetcher=lambda url, timeout: {"status": "ok"} if url.endswith("/health") else ["qwen3.5-4b-onnx-q4f16"],
|
||||
)
|
||||
report_dir = self.root / "automation" / "reports" / "test"
|
||||
report_dir.mkdir(parents=True, exist_ok=True)
|
||||
result = runner._run_model("ollama:qwen2.5:7b", category="priority_models", preflight_only=False, report_dir=report_dir)
|
||||
|
||||
self.assertEqual(result.classification, "accepted")
|
||||
self.assertEqual(result.repair_attempts, 1)
|
||||
self.assertIn("gate", result.completed_stages)
|
||||
|
||||
def test_tracking_sync_updates_docs_with_auto_sync_sections(self) -> None:
|
||||
manifest = Manifest.load(self.root, self.manifest_path)
|
||||
runner = NextLotsRunner(
|
||||
manifest,
|
||||
command_runner=lambda args, cwd, env=None: CommandResult(args=args, returncode=0, stdout="", stderr="", duration_seconds=0.0),
|
||||
json_fetcher=lambda url, timeout: {"status": "ok"},
|
||||
)
|
||||
state = RunState.new(
|
||||
manifest,
|
||||
lot="tracking_sync",
|
||||
report_dir=self.root / "automation" / "reports" / "sync",
|
||||
state_path=self.root / "automation" / "state" / "next_lots_state.json",
|
||||
steps=[{"type": "tracking_sync"}],
|
||||
)
|
||||
state.results = [
|
||||
asdict(
|
||||
ModelRunResult(
|
||||
model="ollama:qwen2.5:7b",
|
||||
category="priority_models",
|
||||
classification="provider_failed",
|
||||
preflight_ok=True,
|
||||
smoke_attempted=True,
|
||||
status="failed",
|
||||
failed_stage="rewrite",
|
||||
)
|
||||
)
|
||||
]
|
||||
|
||||
runner._sync_tracking(state, dry_run=False)
|
||||
|
||||
self.assertIn("AUTO-SYNC:ANE-TODO-ACTIVE:START", (self.root / "TODO_ACTIVE.md").read_text(encoding="utf-8"))
|
||||
self.assertIn("Compacter rewrite", (self.root / "docs" / "EXECUTION_PLAN_2026-03-08.md").read_text(encoding="utf-8"))
|
||||
self.assertIn("ollama:qwen2.5:7b", (self.root / "docs" / "MODEL_COMPARISON_2026-03-08.md").read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def asdict(result: ModelRunResult) -> dict[str, object]:
|
||||
return {
|
||||
"model": result.model,
|
||||
"category": result.category,
|
||||
"classification": result.classification,
|
||||
"preflight_ok": result.preflight_ok,
|
||||
"preflight_duration_seconds": result.preflight_duration_seconds,
|
||||
"smoke_attempted": result.smoke_attempted,
|
||||
"smoke_duration_seconds": result.smoke_duration_seconds,
|
||||
"status": result.status,
|
||||
"accepted": result.accepted,
|
||||
"failed_stage": result.failed_stage,
|
||||
"quality_blockers": result.quality_blockers,
|
||||
"retry_stages": result.retry_stages,
|
||||
"repair_attempts": result.repair_attempts,
|
||||
"repair_models": result.repair_models,
|
||||
"draft_path": result.draft_path,
|
||||
"gate_path": result.gate_path,
|
||||
"meta_path": result.meta_path,
|
||||
"manuscript_path": result.manuscript_path,
|
||||
"notes": result.notes,
|
||||
"preflight_log": result.preflight_log,
|
||||
"smoke_log": result.smoke_log,
|
||||
"workspace": result.workspace,
|
||||
"apple_model_active": result.apple_model_active,
|
||||
"completed_stages": result.completed_stages,
|
||||
}
|
||||
Reference in New Issue
Block a user