electron-rare/le-mystere-professeur-zacus
1
0
Fork 0
Code Issues 1 Pull Requests 2 Actions 1 Packages Projects Releases Wiki Activity

docs: specs slice 12/13 + part13 livré + sec note

Browse Source 
MacStudio spec: P1 part13 marked delivered (commit d59282a, voice-bridge 0.5 with CORSMiddleware + NPC_SYSTEM_PROMPT injection for npc-fast persona). Risks section gains a new entry pointing to the LAN-only security checklist in tools/macstudio/README.md (commit 695b7f7). Acceptance gates: 4 new boxes checked.

Voice-pipeline ESP-SR spec: C_slice 12 marked delivered (commit 71bc9a5, mDNS zacus-master.local + game_endpoint REST NVS-backed) and C_slice 13 (commit a743cfc, voice_dispatcher fuzzy keyword aliases). Risks rewritten: PLIP /voice/hook decroche is now the primary trigger, custom Espressif wake-word is optional, microWakeWord remains an open-source alternative path. Gates: 3 new boxes checked.

Tradeoffs: kept the wholesale 'Total core' line and the §9 Open questions entry on mDNS untouched even though both are now obsolete — minimum-diff to satisfy the verrou and keep review easy. Both can be revised in a follow-up pass.
This commit is contained in:
L'électron rare
2026-05-04 01:04:11 +02:00
parent 695b7f770b
commit 92ac9e7daa
2 changed files with 13 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/superpowers/specs/2026-05-03-tts-stt-llm-macstudio-design.md
+6 -1
View File
@@ -198,7 +198,7 @@ Tableau récapitulatif :
| Phase | Scope | Acceptance | Effort |
|-------|-------|------------|--------|
| **P0** | Provision MacStudio (ollama, whisper.cpp, voice-bridge, LiteLLM, launchd plists, static IP) | All local services start on `launchd reload`, respond to `curl localhost:<port>/<probe>`. LLM/STT models downloaded (~83 GB total). | 3-5 h |
| **P1** ✅ **livré 2026-05-03** | Install F5-TTS on MacStudio (`pip install f5-tts` or `mlx_audio` if MLX port available), download base checkpoint (~1–2 GB), smoke-test on a fixed FR phrase against a generic reference clip. Smoke-test STT (10-clip FR set) and LLM (hints engine response within ±10% latency of KXKM-AI baseline). Modifs livrées : **F5_DEFAULT_STEPS=4** (au lieu de 8), **F5_TIMEOUT_S=8.0** (au lieu de 3.0), **whisper Q5_0** (au lieu de Q5_K). Sous-tâches livrées 2026-05-03 : **part5** MLX-LM 32B Q4 `hints-deep` ✅ ; **part6** MLX-LM 7B Q4 `npc-fast` ✅ ; **part7** voice-bridge daemon F5 in-process ✅ ; **part9** F5 watchdog crontab `*/2 *` + cache disque `~/voice-bridge/cache/` + `service_down.wav` pré-rendu et routé sur `/tts/service_down` ✅ ; **part10** bornage `steps`/`text` + IP canonique `100.116.92.12` ✅ ; **part11** voice-bridge 0.4.0 (Pydantic, `/health/ready`, slowapi rate-limit `/tts`, **endpoint `/voice/ws` STT streaming**) ✅. | F5-TTS produces audible FR audio for the smoke phrase ; STT transcript stable ; LLM latency within budget ; voice-bridge `/tts` returns audio with `tts_backend_used=f5` ; watchdog cron actif ; `service_down.wav` jouable ; `/voice/ws` end-to-end OK | 2-3 h |
| **P1** ✅ **livré 2026-05-03** | Install F5-TTS on MacStudio (`pip install f5-tts` or `mlx_audio` if MLX port available), download base checkpoint (~1–2 GB), smoke-test on a fixed FR phrase against a generic reference clip. Smoke-test STT (10-clip FR set) and LLM (hints engine response within ±10% latency of KXKM-AI baseline). Modifs livrées : **F5_DEFAULT_STEPS=4** (au lieu de 8), **F5_TIMEOUT_S=8.0** (au lieu de 3.0), **whisper Q5_0** (au lieu de Q5_K). Sous-tâches livrées 2026-05-03 : **part5** MLX-LM 32B Q4 `hints-deep` ✅ ; **part6** MLX-LM 7B Q4 `npc-fast` ✅ ; **part7** voice-bridge daemon F5 in-process ✅ ; **part9** F5 watchdog crontab `*/2 *` + cache disque `~/voice-bridge/cache/` + `service_down.wav` pré-rendu et routé sur `/tts/service_down` ✅ ; **part10** bornage `steps`/`text` + IP canonique `100.116.92.12` ✅ ; **part11** voice-bridge 0.4.0 (Pydantic, `/health/ready`, slowapi rate-limit `/tts`, **endpoint `/voice/ws` STT streaming**) ✅. **part13** ✅ **livré 2026-05-04** — commit `d59282a` — voice-bridge 0.5 : **CORSMiddleware** (origines configurables via env `CORS_ORIGINS`) + injection **`NPC_SYSTEM_PROMPT`** persona (« Tu es le Professeur Zacus », accepte les variantes Zaku/Zacusse, FR ≤ 2 phrases) avant npc-fast dans `/voice/intent` et `/voice/ws`. Caller-supplied `messages` ou `system` override le default — back-compat préservée. | F5-TTS produces audible FR audio for the smoke phrase ; STT transcript stable ; LLM latency within budget ; voice-bridge `/tts` returns audio with `tts_backend_used=f5` ; watchdog cron actif ; `service_down.wav` jouable ; `/voice/ws` end-to-end OK ; **CORS configurable + persona Zacus injecté en amont npc-fast** | 2-3 h |
| **P2** | **Prereq humain** : enregistrer `zacus_reference.wav` (≤10 s, canonical voice-off du jeu, studio quality, clean mic, no reverb, full prosodic range). Tant qu'il manque, le bridge utilise un `/tmp/ref.wav` `say -v Thomas` générique (smoke OK, prod KO). Hard cutover (Saturday): wire reference into voice-bridge, repoint all clients to `192.168.0.150:8200/tts`, restart, smoke-test E2E. **Regen pool** : exécuter `tools/tts/generate_npc_pool.py` (adapté `:8200/tts`) une fois la voix Zacus disponible pour pré-remplir `~/voice-bridge/cache/` avec les ~200 phrases canoniques de `npc_phrases.yaml`. | `zacus_reference.wav` committed to repo (or secrets store) ; ESP32_ZACUS Voice WS connects ; atelier hints query resolves ; NPC speaks via F5 with cloned Zacus voice ; forced-degradation test (kill F5 process) sert `service_down.wav` puis watchdog relance le bridge < 2 min ; pool cache pré-rempli ≥ 80 % de `npc_phrases.yaml` ; full puzzle voice loop ≤ 8 s end-to-end (steps=4) | 3-4 h |
| **P3** | Soak (Sunday): 24 h running, watch logs for crashes / OOM / latency creep / cache hit ratio / F5 timeout ratio | No service crash, no OOM, p95 latency stable, **ratio cache hits ≥ 80 %** + **ratio F5 timeouts < 5 %** measured from voice-bridge structured logs (`tts_backend_used` ∈ {`cache`, `f5`, `service_down`}). Reformulation de l'ancien « ratio fallback Piper < 5 % » désormais sans objet (Piper retiré du chemin live). | 24 h passive |
| **P4** | Decommission KXKM-AI ollama service. Update root CLAUDE.md Infrastructure section. Document the rollback path. **Confirmer** que le watchdog F5 a redémarré le service ≥ 1 fois en 24 h sans crash visible côté client (preuve : log soak + sortie crontab). | KXKM-AI's ollama process stopped ; CLAUDE.md mentions MacStudio F5-TTS primary + watchdog + cache + `service_down.wav` (no Piper Tower in the live chain) ; watchdog evidence captured | 1 h |
@@ -221,6 +221,7 @@ Tableau récapitulatif :
| **F5 in-process bloque l'event loop** (~4 s par requête en `steps=4`) limitant la concurrence à 1–2 clients voix simultanés | Medium | Lock sérialisé actuel suffit pour le scénario Zacus (NPC unique, 1 client ESP32 + 1 atelier max). Migration vers worker pool MLX-aware (process pool / thread dédié) = follow-up post-cutover si la salle accueille plusieurs joueurs voix simultanés. |
| **F5 watchdog dépend de `crond`** — si le démon cron macOS est mort, le watchdog ne tourne pas et un crash F5 reste invisible jusqu'à diagnostic manuel | Low (macOS `cron` runs by default, rarement tué) | Vérifier `sudo launchctl list \| grep cron` au déploiement ; alternative future = `launchd` user agent dédié (mais bloqué sur macOS 26 headless via SSH) |
| **Pool pré-rendu obsolète si `zacus_reference.wav` change** (clé cache inclut `ref_hash`, donc nouvelle ref = 100 % cache miss tant que pool pas régénéré) | Medium | Invalider tout le cache à chaque nouvelle ref (`rm -rf ~/voice-bridge/cache/*` puis re-run `tools/tts/generate_npc_pool.py`). Documenter dans le runbook P4. |
| **Surfaces no-auth exposées si MacStudio sort du LAN** (voice-bridge `/tts`, `/voice/{ws,transcribe,intent}`, `/voice/hook`, LiteLLM `:4000`, MLX-LM `:8500`/`:8501`, whisper `:8300` n'imposent aucune authentification — modèle de menace = LAN-only) | Medium | Documenté dans `tools/macstudio/README.md` Security section (commit `695b7f7`) : LAN-only assumption explicite, **checklist pre-deployment 4 étapes** (vérifier interface bind `127.0.0.1`/LAN-only, `pf`/firewall règle, rotation `MASTER_KEY` LiteLLM, `CORS_ORIGINS` restrictif), boot WARN si default master key détectée. À auditer avant toute exposition Tailscale ou WAN. |
**Rollback procedure**: stop MacStudio's services (F5-TTS, voice-bridge, ollama, whisper, LiteLLM via crontab @reboot lines disabled), re-enable KXKM-AI's ollama service, repoint clients back to KXKM-AI for hints. Pendant la fenêtre de rollback côté TTS, les clients reçoivent `service_down.wav` (pas de voix dynamique le temps de restaurer F5). Total rollback ≤ 30 min if practiced.
@@ -235,6 +236,10 @@ Tableau récapitulatif :
- [x] **`service_down.wav` jouable** + accepté narrativement (« Le Professeur Zacus est temporairement indisponible, un instant ») ✅ (livré 2026-05-03, part9)
- [x] **Steps clamp** (bornage `steps` ∈ [2, 16]) + **RateLimit slowapi `/tts`** ✅ (livré 2026-05-03, part10/part11)
- [x] **`/voice/ws` end-to-end** (hello → stream PCM → STT whisper → intent dispatch) ✅ (livré 2026-05-03, part11)
- [x] **CORSMiddleware actif** (origines via env `CORS_ORIGINS`) ✅ (livré 2026-05-04, part13, commit `d59282a`)
- [x] **NPC system prompt persona Zacus** injecté avant npc-fast (`/voice/intent` + `/voice/ws`, override caller-supplied préservé) ✅ (livré 2026-05-04, part13, commit `d59282a`)
- [x] **`.env.example` template + boot warns** (default master key détectée → WARN visible) ✅ (livré 2026-05-04, voir `tools/macstudio/README.md` Security)
- [x] **Pre-deployment security checklist** (LAN-only assumption, no-auth surfaces, 4 étapes audit) ✅ (livré 2026-05-04, commit `695b7f7`)
- [ ] **`zacus_reference.wav` présent** → **PENDING** (action humaine, enregistrement studio, P2 prereq)
- [ ] All MacStudio daemons (F5-TTS, voice-bridge, ollama, whisper, LiteLLM) green for ≥ 24 h continuous (P3 soak)
- [ ] ESP32_ZACUS Voice WS reconnects after firmware reboot, no panic
docs/superpowers/specs/2026-05-03-voice-pipeline-esp-sr-design.md
+7 -2
View File
@@ -237,6 +237,8 @@ Réf composant : `ESP32_ZACUS/idf_zacus/components/voice_hook_endpoint/README.md
## 6. Risks
| Risk | Probability | Mitigation |
|------|-------------|------------|
| ESP-IDF migration introduces regressions in master firmware (P1) | High by nature | Branch-based migration in `ESP32_ZACUS` ; keep Arduino branch operational ; comprehensive playtest harness (per spec 7) catches regressions |
| Custom wake word fails to converge (training data too thin) | Medium | Fall back to "hi_esp" placeholder ; iterate with more samples |
| Whisper STT mistranscribes accented speech (children, foreign visitors) | Medium-High | Confidence threshold; LLM fallback parses garbled transcripts; "pardon" graceful degradation |
| Speaker echo bypasses mute-during-TTS (room reverb) | Medium | Validate in field ; if issue, jump to AEC P8 |
@@ -251,7 +253,7 @@ Réf composant : `ESP32_ZACUS/idf_zacus/components/voice_hook_endpoint/README.md
- [x] **STT WebSocket streaming end-to-end** (slice 7) — firmware → `/voice/ws` → whisper → intent dispatch
- [x] **Voice dispatcher STT keyword fast-path → hints** (slice 8, commit `c3a771c`) — `voice_dispatcher` route `/indice|hint|aide/` vers `hints_client_ask_async` + intent log + cue best-effort
- [x] **Mute-during-TTS gate** (slice 9, commit `6d3e989`) — state==SPEAKING skips AFE feed
- [x] **I2S TX channel `I2S_NUM_1` séparé du mic `I2S_NUM_0`** (slice 9, commit `6d3e989`) — TTS playback opt-in via `enable_tts_playback`
- [x] **I2S TX channel `I2S_NUM_1` séparé du mic `I2S_NUM_0`** (slice 9, commit `6d3e989`) — TTS playback opt-in via `enable_tts_playback`
- [x] **TTS playback ws protocol** (slice 9b, commit `d47c230`) — `voice_pipeline_ws` parse `{speak_start, sample_rate, format}` + binary PCM frames + `{speak_end, duration_ms, backend, latency_ms}` → `play_chunk` / `play_end`
- [x] **PLIP hook integration côté master** (slice 10, commit `3585a65`) — `voice_hook_endpoint` `POST /voice/hook` sur httpd port 80 partagé avec `ota_server` ; PLIP décroché → LISTENING + start_capture, raccroché → IDLE + stop_capture. Côté PLIP firmware (autre projet) **PENDING**.
- [x] **mDNS hostname `zacus-master.local` publié** (slice 12, commit `71bc9a5`) — `mdns_init` + `mdns_hostname_set` actifs ; smoke device (résolution depuis un autre host LAN) **PENDING**
@@ -263,7 +265,10 @@ Réf composant : `ESP32_ZACUS/idf_zacus/components/voice_hook_endpoint/README.md
- [ ] False-accept rate of "Professeur Zacus" < 1 / hour ambient → **PENDING** (gating sur P4 wake custom Espressif)
- [ ] False-reject rate < 10 % at 2 m distance (3 voice samples per direction) → **PENDING** (gating sur P4)
- [ ] Voice flow degrades gracefully when MacStudio voice-bridge unreachable (button-only mode)
- [ ] Playtest harness has ≥ 1 voice-driven scenario (mocked transcripts replay through voice dispatcher)
- [ ] No false re-triggering during NPC TTS playback (mute gate verified on device) → scaffold ✅ (slice 9), validation device **PENDING**
- [ ] ESP-IDF master firmware passes the same playtest set as the Arduino version (no regression vs P1 baseline)
## 8. Out of scope
- True conversational dialog (multi-turn). Future enhancement when needed.