electron-rare/le-mystere-professeur-zacus
1
0
Fork 0
Code Issues 1 Pull Requests 2 Actions 1 Packages Projects Releases Wiki Activity

docs: CLAUDE.md + README align with P1 stack

Browse Source 
Project root CLAUDE.md: architecture diagram now shows MacStudio voice-bridge (cache → F5 → safe wav), hints engine LLM rewrite via LiteLLM hints-deep (MLX 32B Q4), and MLX-LM inference tier. Infrastructure section adds MacStudio (live stack) and demotes Tower to legacy. tools/macstudio/README.md gets full stack table, aliases, persistence (crontab @reboot + */2 watchdog), resilience (cache, service_down.wav), pool pre-render command, known issues. Home ~/CLAUDE.md edits applied separately (six-node, MacStudio added, Tower line trimmed).
This commit is contained in:
L'électron rare
2026-05-03 21:19:24 +02:00
parent 7336da9da8
commit 0ec3da620f
2 changed files with 98 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+12 -6
View File
@@ -33,18 +33,23 @@ Frontend, desktop, and tooling commands live in their nested CLAUDE.md.
``` ```
YAML scenario → compile_runtime3.py → Runtime 3 IR → ESP32 / Web player YAML scenario → compile_runtime3.py → Runtime 3 IR → ESP32 / Web player
↓ ↓
Voice bridge → Piper TTS (Tower:8001) Voice bridge :8200 (MacStudio)
↳ /tts → F5-TTS-MLX (cache → F5 → safe wav)
↳ /voice/intent → LiteLLM npc-fast (MLX 7B Q4)
↳ /voice/transcribe → whisper.cpp :8300
↓ ↓
Hints engine → /hints/ask Hints engine → /hints/ask (LLM rewrite via hints-deep MLX 32B Q4)
``` ```
Key surfaces: Key surfaces:
- **Scenario IR**: `game/scenarios/zacus_v2.yaml` → `tools/scenario/compile_runtime3.py` → portable Runtime 3 IR. Contract: `specs/ZACUS_RUNTIME_3_SPEC.md`. - **Scenario IR**: `game/scenarios/zacus_v2.yaml` → `tools/scenario/compile_runtime3.py` → portable Runtime 3 IR. Contract: `specs/ZACUS_RUNTIME_3_SPEC.md`.
- **Authoring**: `frontend-v3/` (pnpm monorepo: `apps/atelier/` Scratch-like studio + `apps/dashboard/` game-master live view). - **Authoring**: `frontend-v3/` (pnpm monorepo: `apps/atelier/` Scratch-like studio + `apps/dashboard/` game-master live view).
- **Firmware**: - **Firmware**:
- `ESP32_ZACUS/` submodule — Freenove ESP32-S3 master (NPC engine, voice pipeline, vision/QR, media manager). - `ESP32_ZACUS/` submodule — Freenove ESP32-S3 master (NPC engine, voice pipeline, vision/QR, media manager). Branche `feat/idf-migration` en cours (port Arduino → ESP-IDF 5.4).
- `PLIP_FIRMWARE/` — retro telephone annex (ES8388 dev kit bringup, Si3210 PCB target). REST endpoint consumed by the master ESP32. - `PLIP_FIRMWARE/` — retro telephone annex (ES8388 dev kit bringup, Si3210 PCB target). REST endpoint consumed by the master ESP32.
- **Voice / NPC**: Piper TTS on Tower:8001 (zacus voice = tom-medium). NPC phrases in `game/scenarios/npc_phrases.yaml`. MP3 pool generator: `tools/tts/generate_npc_pool.py`. - **Voice / NPC**: F5-TTS-MLX in-process sur MacStudio (`studio:8200`), voix Zacus clonée via `~/zacus_reference.wav`. Pré-rendu via `tools/tts/generate_npc_pool.py --backend f5`. Service down WAV hardcoded en graceful degradation. NPC phrases : `game/scenarios/npc_phrases.yaml`.
- **Hints engine**: `tools/hints/server.py` FastAPI (LLM rewrite via LiteLLM `hints-deep`, anti-cheat, adaptive level, safety filter `game/scenarios/hints_safety.yaml`).
- **Inference stack**: voir `tools/macstudio/README.md` — MLX-LM (npc + hints), whisper.cpp, F5-TTS, LiteLLM proxy `:4000`. Configs versionnées dans `tools/macstudio/`.
- **MCP hardware**: `tools/dev/mcp_hardware_server.py` (stdio, 6 tools). - **MCP hardware**: `tools/dev/mcp_hardware_server.py` (stdio, 6 tools).
- **Desktop hub**: `desktop/` (Electron, macOS, bundles V3 frontends, talks USB serial). - **Desktop hub**: `desktop/` (Electron, macOS, bundles V3 frontends, talks USB serial).
@@ -76,8 +81,9 @@ Key surfaces:
## Infrastructure ## Infrastructure
- **Tower** (`clems@192.168.0.120`): Piper TTS FR on `:8001` (voices: tom-medium / siwis / upmc). - **MacStudio** (`ssh studio`, M3 Ultra 512 GB) : **stack inférence Zacus complète** — MLX-LM (`npc-fast` 7B Q4 :8501, `hints-deep` 32B Q4 :8500), whisper.cpp `large-v3-turbo-q5_0` :8300, F5-TTS-MLX in-process via voice-bridge :8200 (cache disque + watchdog `*/2` + `service_down.wav` graceful), LiteLLM proxy :4000. Voir `tools/macstudio/README.md`.
- **KXKM-AI** (`kxkm@kxkm-ai` via Tailscale): RTX 4090, GPU inference. - **Tower** (`clems@192.168.0.120`) : Suite Numérique, n8n, LiteLLM Tower (legacy). **Plus dans la chaîne live Zacus** ; container Docker `zacus-tts-piper` survit pour batch historique uniquement (voix EN-only, FR à déployer si besoin).
- **KXKM-AI** (`kxkm@kxkm-ai` via Tailscale) : RTX 4090, GPU inference (fine-tuning, plus utilisé en runtime).
- SSH is key-based only — never `sshpass`. - SSH is key-based only — never `sshpass`.
## Language ## Language
tools/macstudio/README.md
+86 -16
View File
@@ -1,30 +1,81 @@
# MacStudio inference stack — config sources of truth # MacStudio inference stack — config sources of truth
Deployment configs for the MacStudio M3 Ultra (`ssh studio`) hosting Ollama, Deployment configs for the MacStudio M3 Ultra (`ssh studio`,
`clems@MacStudio-de-MonsieurB.local`, 512 GB) hosting MLX-LM, Ollama,
whisper.cpp, F5-TTS, and the LiteLLM proxy. Spec: whisper.cpp, F5-TTS, and the LiteLLM proxy. Spec:
`docs/superpowers/specs/2026-05-03-tts-stt-llm-macstudio-design.md`. `docs/superpowers/specs/2026-05-03-tts-stt-llm-macstudio-design.md`.
## Stack live (post-P1 part1-9)
Inference and voice stack runs entirely on studio. Tower no longer participates
in the live Zacus runtime path.
| Service | Port | Backend | LiteLLM alias(es) | Monitoring |
|---------|------|---------|-------------------|------------|
| MLX-LM 7B Q4 | `:8501` | `mlx_lm.server` | `npc-fast` (~12 tok/s warm) | `curl -s http://studio:8501/v1/models` |
| MLX-LM 32B Q4 | `:8500` | `mlx_lm.server` | `hints-deep` (~4.4 tok/s warm) | `curl -s http://studio:8500/v1/models` |
| Ollama | `:11434` | `ollama serve` | `npc-ollama-7b`, `hints-ollama-72b` (legacy A/B) | `curl -s http://studio:11434/api/tags` |
| whisper.cpp | `:8300` | `whisper-server` (`large-v3-turbo-q5_0`) | `whisper-fr`, `stt-fr` | `pgrep -fl whisper-server` |
| F5-TTS-MLX | `:8200` (in-process) | voice-bridge `/tts` + `/voice/{transcribe,intent,ws}` | `tts-zacus-primary` | `curl -s http://studio:8200/healthz` |
| LiteLLM proxy | `:4000` | `litellm` | (master_key `sk-zacus-local-dev-do-not-share`) | `curl -s http://studio:4000/health/readiness` |
## Files ## Files
| Local path | Remote path on studio | Purpose | | Local path | Remote path on studio | Purpose |
|------------|-----------------------|---------| |------------|-----------------------|---------|
| `litellm-config.yaml` | `~/litellm-config/config.yaml` | LiteLLM proxy (`:4000`) — 6 model aliases (`npc-fast`, `hints-deep`, `whisper-fr`, `stt-fr`, `tts-zacus-primary`, `tts-zacus-fallback`) | | `litellm-config.yaml` | `~/litellm-config/config.yaml` | LiteLLM proxy (`:4000`) — 6+ model aliases (`npc-fast`, `hints-deep`, `whisper-fr`, `stt-fr`, `tts-zacus-primary`, `npc-ollama-7b`, `hints-ollama-72b`) |
| `voice-bridge/main.py` | `~/voice-bridge/main.py` | FastAPI voice-bridge scaffold (`:8200`) — routes `/voice/transcribe` (whisper :8300), `/voice/intent` (LiteLLM `npc-fast`), `/tts` (F5 primary, Piper Tower:8001 fallback) | | `voice-bridge/main.py` | `~/voice-bridge/main.py` | FastAPI voice-bridge (`:8200`) — routes `/voice/transcribe` (whisper :8300), `/voice/intent` (LiteLLM `npc-fast`), `/tts` (F5-TTS-MLX in-process), `/voice/ws` |
| `voice-bridge/requirements.txt` | `~/voice-bridge/requirements.txt` | FastAPI deps (fastapi, httpx, uvicorn) | | `voice-bridge/requirements.txt` | `~/voice-bridge/requirements.txt` | FastAPI deps (fastapi, httpx, uvicorn, f5-tts-mlx) |
| `voice-bridge/watchdog.sh` | `~/voice-bridge/watchdog.sh` | Watchdog: relance voice-bridge si `pgrep` absent |
## Persistence (headless macOS — no launchd) ## Aliases LiteLLM
Model routing exposed via the proxy on `:4000`. Clients hit aliases (never raw
backends) so we can swap MLX ↔ Ollama without code changes.
| Alias | Backend | Use case |
|-------|---------|----------|
| `npc-fast` | MLX-LM 7B Q4 (`:8501`) | NPC dialogue, low-latency intent |
| `hints-deep` | MLX-LM 32B Q4 (`:8500`) | Hint generation, longer reasoning |
| `npc-ollama-7b` | Ollama 7B (`:11434`) | Legacy A/B comparator for `npc-fast` |
| `hints-ollama-72b` | Ollama 72B (`:11434`) | Legacy A/B comparator for `hints-deep` |
| `whisper-fr` / `stt-fr` | whisper.cpp (`:8300`) | French ASR |
| `tts-zacus-primary` | F5-TTS-MLX via voice-bridge (`:8200`) | Zacus voice synthesis |
## Persistence (headless macOS 26 — no launchd)
Studio runs headless; `launchctl` over SSH is broken. All daemons run via Studio runs headless; `launchctl` over SSH is broken. All daemons run via
`nohup` for current uptime + `@reboot` crontab for reboot persistence. `nohup` for current uptime + `@reboot` crontab for reboot persistence. Plus a
`*/2` watchdog entry that relaunches voice-bridge on crash.
```cron ```cron
@reboot /opt/homebrew/bin/node_exporter >> /Users/clems/node_exporter.log 2>&1
@reboot /Users/clems/.local/bin/ollama serve >> /Users/clems/ollama.log 2>&1 @reboot /Users/clems/.local/bin/ollama serve >> /Users/clems/ollama.log 2>&1
@reboot /Users/clems/whisper-install/whisper.cpp/build/bin/whisper-server -m /Users/clems/whisper-install/whisper.cpp/models/ggml-large-v3-turbo-q5_0.bin --port 8300 -t 8 --host 127.0.0.1 -l fr >> /Users/clems/whisper-server.log 2>&1 @reboot /Users/clems/whisper-install/whisper.cpp/build/bin/whisper-server -m /Users/clems/whisper-install/whisper.cpp/models/ggml-large-v3-turbo-q5_0.bin --port 8300 -t 8 --host 127.0.0.1 -l fr >> /Users/clems/whisper-server.log 2>&1
@reboot /Users/clems/.local/bin/litellm --config /Users/clems/litellm-config/config.yaml --port 4000 --host 0.0.0.0 >> /Users/clems/litellm.log 2>&1 @reboot /Users/clems/.local/bin/litellm --config /Users/clems/litellm-config/config.yaml --port 4000 --host 0.0.0.0 >> /Users/clems/litellm.log 2>&1
@reboot /Users/clems/.local/bin/mlx_lm.server --model <hints-32b-q4> --port 8500 >> /Users/clems/mlx-32b.log 2>&1
@reboot /Users/clems/.local/bin/mlx_lm.server --model <npc-7b-q4> --port 8501 >> /Users/clems/mlx-7b.log 2>&1
@reboot /Users/clems/voice-bridge/.venv/bin/uvicorn main:app --host 0.0.0.0 --port 8200 >> /Users/clems/voice-bridge.log 2>&1
*/2 * * * * /Users/clems/voice-bridge/watchdog.sh >> /Users/clems/voice-bridge.watchdog.log 2>&1
``` ```
Edit on studio with `crontab -e`. Edit on studio with `crontab -e`.
## Resilience
- **Watchdog** — `*/2 * * * * /Users/clems/voice-bridge/watchdog.sh` relance
voice-bridge si `pgrep -f "uvicorn main:app"` est vide (auto-recovery sous 2
minutes après un crash).
- **Cache TTS disque** — `~/voice-bridge/cache/` indexé par
`sha256(text + voice_ref + steps)`. Hit cache = retour WAV immédiat (pas de
ré-synthèse). Purge :
```bash
curl -X DELETE http://studio:8200/tts/cache
```
- **Graceful degradation** — `service_down.wav` (hardcoded fallback) servi par
voice-bridge si F5-TTS lève une exception irrécupérable. Le runtime ESP32
reçoit toujours un WAV valide, jamais un 5xx silencieux.
## Sync local → remote ## Sync local → remote
After editing here, push to studio: After editing here, push to studio:
@@ -41,21 +92,40 @@ ssh studio 'pkill -HUP -f "litellm --config" || pgrep -fl litellm' # reload Li
```bash ```bash
ssh studio 'curl -s http://localhost:11434/api/tags | jq .models[].name' # ollama models ssh studio 'curl -s http://localhost:11434/api/tags | jq .models[].name' # ollama models
ssh studio 'curl -s http://localhost:8500/v1/models' # mlx 32B (hints-deep)
ssh studio 'curl -s http://localhost:8501/v1/models' # mlx 7B (npc-fast)
ssh studio 'curl -s http://localhost:4000/health/readiness' # litellm ssh studio 'curl -s http://localhost:4000/health/readiness' # litellm
ssh studio 'curl -s http://localhost:8200/healthz' # voice-bridge
ssh studio 'curl -s -F "file=@/Users/clems/whisper-install/whisper.cpp/samples/jfk.wav" http://localhost:8300/inference' # whisper ssh studio 'curl -s -F "file=@/Users/clems/whisper-install/whisper.cpp/samples/jfk.wav" http://localhost:8300/inference' # whisper
ssh studio 'pgrep -fl ollama; pgrep -fl whisper-server; pgrep -fl litellm' # daemons ssh studio 'pgrep -fl ollama; pgrep -fl whisper-server; pgrep -fl litellm; pgrep -fl mlx_lm.server; pgrep -fl "uvicorn main:app"' # daemons
``` ```
## Known limitations (2026-05-03) ## Pré-render NPC pool (F5-TTS)
- `qwen2.5:72b-instruct-q8_0` via Ollama gives ~2 tok/s on M3 Ultra. Switch Generate the full NPC phrase pool against the live voice-bridge (idempotent —
to MLX-LM backend or Q4_K_M quant if hints latency unacceptable (cible skips already-cached entries). Output: `hotline_tts/f5/<key>.wav` plus
spec: <8 s for 50 tokens, currently ~25 s). `hotline_tts/f5/manifest.json`.
- F5-TTS uses base multilingual checkpoint (no FR-specific). Quality
acceptable but no Zacus voice clone yet — needs `zacus_reference.wav` to ```bash
be recorded (P2). python3 tools/tts/generate_npc_pool.py \
- Voice-bridge scaffold is **not running** as daemon yet. Activate when --backend f5 \
F5-TTS in-process integration is wired. --voice-bridge-url http://studio:8200
```
Cache key derivation must mirror `voice-bridge/main.py::_voice_ref_token`. See
`tools/CLAUDE.md` for backend semantics.
## Known issues / TODO
- **`steps` non borné** côté payload `/tts` — un client malicieux peut envoyer
`steps=999` et saturer la GPU queue. Ajouter `steps: int = Field(ge=1, le=64)`
dans le schéma Pydantic de voice-bridge.
- **Default `--voice-bridge-url 192.168.0.150`** dans certains scripts — IP LAN
studio à confirmer selon le réseau du client. En itinérance préférer le
Tailscale `100.116.92.12` (ou alias DNS `studio`).
- **`zacus_reference.wav`** à enregistrer (action humaine, prereq P2). Sans ce
fichier, F5-TTS-MLX tombe sur le checkpoint multilingue de base — qualité
acceptable mais pas de Zacus voice clone.
## Anti-patterns ## Anti-patterns