le-mystere-professeur-zacus/tools/macstudio/README.md

MacStudio inference stack — config sources of truth

Deployment configs for the MacStudio M3 Ultra (ssh studio, clems@MacStudio-de-MonsieurB.local, 512 GB) hosting MLX-LM, Ollama, whisper.cpp, F5-TTS, and the LiteLLM proxy. Spec: docs/superpowers/specs/2026-05-03-tts-stt-llm-macstudio-design.md.

Stack live (post-P1 part1-9)

Inference and voice stack runs entirely on studio. Tower no longer participates in the live Zacus runtime path.

Service	Port	Backend	LiteLLM alias(es)	Monitoring
MLX-LM 7B Q4	:8501	mlx_lm.server	npc-fast (~12 tok/s warm)	curl -s http://studio:8501/v1/models
MLX-LM 32B Q4	:8500	mlx_lm.server	hints-deep (~4.4 tok/s warm)	curl -s http://studio:8500/v1/models
Ollama	:11434	ollama serve	npc-ollama-7b, hints-ollama-72b (legacy A/B)	curl -s http://studio:11434/api/tags
whisper.cpp	:8300	whisper-server (large-v3-turbo-q5_0)	whisper-fr, stt-fr	pgrep -fl whisper-server
F5-TTS-MLX	:8200 (in-process)	voice-bridge /tts + /voice/{transcribe,intent,ws}	tts-zacus-primary	curl -s http://studio:8200/healthz
LiteLLM proxy	:4000	litellm	(master_key sk-zacus-local-dev-do-not-share)	curl -s http://studio:4000/health/readiness

Files

Local path	Remote path on studio	Purpose
litellm-config.yaml	~/litellm-config/config.yaml	LiteLLM proxy (:4000) — 6+ model aliases (npc-fast, hints-deep, whisper-fr, stt-fr, tts-zacus-primary, npc-ollama-7b, hints-ollama-72b)
voice-bridge/main.py	~/voice-bridge/main.py	FastAPI voice-bridge (:8200) — routes /voice/transcribe (whisper :8300), /voice/intent (LiteLLM npc-fast), /tts (F5-TTS-MLX in-process), /voice/ws
voice-bridge/requirements.txt	~/voice-bridge/requirements.txt	FastAPI deps (fastapi, httpx, uvicorn, f5-tts-mlx)
voice-bridge/watchdog.sh	~/voice-bridge/watchdog.sh	Watchdog: relance voice-bridge si pgrep absent

Aliases LiteLLM

Model routing exposed via the proxy on :4000. Clients hit aliases (never raw backends) so we can swap MLX ↔ Ollama without code changes.

Alias	Backend	Use case
npc-fast	MLX-LM 7B Q4 (:8501)	NPC dialogue, low-latency intent
hints-deep	MLX-LM 32B Q4 (:8500)	Hint generation, longer reasoning
npc-ollama-7b	Ollama 7B (:11434)	Legacy A/B comparator for npc-fast
hints-ollama-72b	Ollama 72B (:11434)	Legacy A/B comparator for hints-deep
whisper-fr / stt-fr	whisper.cpp (:8300)	French ASR
tts-zacus-primary	F5-TTS-MLX via voice-bridge (:8200)	Zacus voice synthesis

Persistence (headless macOS 26 — no launchd)

Studio runs headless; launchctl over SSH is broken. All daemons run via nohup for current uptime + @reboot crontab for reboot persistence. Plus a */2 watchdog entry that relaunches voice-bridge on crash.

@reboot /opt/homebrew/bin/node_exporter >> /Users/clems/node_exporter.log 2>&1
@reboot /Users/clems/.local/bin/ollama serve >> /Users/clems/ollama.log 2>&1
@reboot /Users/clems/whisper-install/whisper.cpp/build/bin/whisper-server -m /Users/clems/whisper-install/whisper.cpp/models/ggml-large-v3-turbo-q5_0.bin --port 8300 -t 8 --host 127.0.0.1 -l fr >> /Users/clems/whisper-server.log 2>&1
@reboot /Users/clems/.local/bin/litellm --config /Users/clems/litellm-config/config.yaml --port 4000 --host 0.0.0.0 >> /Users/clems/litellm.log 2>&1
@reboot /Users/clems/.local/bin/mlx_lm.server --model <hints-32b-q4> --port 8500 >> /Users/clems/mlx-32b.log 2>&1
@reboot /Users/clems/.local/bin/mlx_lm.server --model <npc-7b-q4>   --port 8501 >> /Users/clems/mlx-7b.log  2>&1
@reboot /Users/clems/voice-bridge/.venv/bin/uvicorn main:app --host 0.0.0.0 --port 8200 >> /Users/clems/voice-bridge.log 2>&1
*/2 * * * * /Users/clems/voice-bridge/watchdog.sh >> /Users/clems/voice-bridge.watchdog.log 2>&1

Edit on studio with crontab -e.

Resilience

• Watchdog — */2 * * * * /Users/clems/voice-bridge/watchdog.sh relance voice-bridge si pgrep -f "uvicorn main:app" est vide (auto-recovery sous 2 minutes après un crash).
• Cache TTS disque — ~/voice-bridge/cache/ indexé par sha256(text + voice_ref + steps). Hit cache = retour WAV immédiat (pas de ré-synthèse). Purge :

  curl -X DELETE http://studio:8200/tts/cache
  

• Graceful degradation — service_down.wav (hardcoded fallback) servi par voice-bridge si F5-TTS lève une exception irrécupérable. Le runtime ESP32 reçoit toujours un WAV valide, jamais un 5xx silencieux.

Sync local → remote

After editing here, push to studio:

scp tools/macstudio/litellm-config.yaml     studio:~/litellm-config/config.yaml
scp tools/macstudio/voice-bridge/main.py    studio:~/voice-bridge/main.py
scp tools/macstudio/voice-bridge/requirements.txt studio:~/voice-bridge/requirements.txt

ssh studio 'pkill -HUP -f "litellm --config" || pgrep -fl litellm'   # reload LiteLLM

Health checks

ssh studio 'curl -s http://localhost:11434/api/tags | jq .models[].name'   # ollama models
ssh studio 'curl -s http://localhost:8500/v1/models'                       # mlx 32B (hints-deep)
ssh studio 'curl -s http://localhost:8501/v1/models'                       # mlx 7B  (npc-fast)
ssh studio 'curl -s http://localhost:4000/health/readiness'                # litellm
ssh studio 'curl -s http://localhost:8200/healthz'                         # voice-bridge
ssh studio 'curl -s -F "file=@/Users/clems/whisper-install/whisper.cpp/samples/jfk.wav" http://localhost:8300/inference'  # whisper
ssh studio 'pgrep -fl ollama; pgrep -fl whisper-server; pgrep -fl litellm; pgrep -fl mlx_lm.server; pgrep -fl "uvicorn main:app"'  # daemons

Pré-render NPC pool (F5-TTS)

Generate the full NPC phrase pool against the live voice-bridge (idempotent — skips already-cached entries). Output: hotline_tts/f5/<key>.wav plus hotline_tts/f5/manifest.json.

python3 tools/tts/generate_npc_pool.py \
  --backend f5 \
  --voice-bridge-url http://studio:8200

Cache key derivation must mirror voice-bridge/main.py::_voice_ref_token. See tools/CLAUDE.md for backend semantics.

Known issues / TODO

• steps non borné côté payload /tts — un client malicieux peut envoyer steps=999 et saturer la GPU queue. Ajouter steps: int = Field(ge=1, le=64) dans le schéma Pydantic de voice-bridge.
• Default --voice-bridge-url 192.168.0.150 dans certains scripts — IP LAN studio à confirmer selon le réseau du client. En itinérance préférer le Tailscale 100.116.92.12 (ou alias DNS studio).
• zacus_reference.wav à enregistrer (action humaine, prereq P2). Sans ce fichier, F5-TTS-MLX tombe sur le checkpoint multilingue de base — qualité acceptable mais pas de Zacus voice clone.

Anti-patterns

• Don't edit configs on studio directly; always edit here and scp. Studio is the runtime; this directory is the source of truth.
• Don't commit master_key rotated values — current value is dev-only.