electron-rare/le-mystere-professeur-zacus
1
0
Fork 0
Code Issues 1 Pull Requests 2 Actions 1 Packages Projects Releases Wiki Activity

feat(macstudio): F5 watchdog + cache + safe wav

Browse Source 
- voice-bridge main.py: cache disque indexed by sha256(text+ref+steps), GET /tts/cache/stats + DELETE /tts/cache, GET /tts/service_down, F5 timeout default 8s, default steps 4. - watchdog.sh: crontab */2 keepalive, restarts voice-bridge if pgrep fails. - service_down.wav: hardcoded graceful degradation asset (FR, 6.86s). - generate_npc_pool.py: --backend f5 (calls voice-bridge), manifest with cache_key for server-side cache priming. - tools/CLAUDE.md: documents both backends. - .gitignore: hotline_tts/ pool artefacts (reproducible).
This commit is contained in:
L'électron rare
2026-05-03 21:12:12 +02:00
parent 8fb57032c5
commit a76a882cf1
5 changed files with 677 additions and 122 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+3
View File
@@ -90,3 +90,6 @@ prompt a suivre/
memory/
site/
todos/
# Generated TTS pool artefacts (reproducible from npc_phrases.yaml + voice ref)
hotline_tts/
tools/CLAUDE.md
+8 -1
View File
@@ -9,7 +9,7 @@ Python tooling for scenario compilation, validation, voice pipeline, MCP hardwar
| `scenario/` | Runtime 3 compiler, validator, simulator, pivot verifier, MD/firmware exporters |
| `audio/` | Audio manifest validator |
| `printables/` | Printables manifest validator |
| `tts/` | NPC phrase pool generator (Piper TTS → `hotline_tts/`) |
| `tts/` | NPC phrase pool generator (Piper TTS or voice-bridge F5 → `hotline_tts/`) |
| `stt/` | Speech-to-text helpers |
| `dev/` | TUI dashboard, MCP hardware server, `zacus.sh` CLI (12 actions) |
| `playtest/` | Playtest harness |
@@ -22,6 +22,13 @@ Python tooling for scenario compilation, validation, voice pipeline, MCP hardwar
- Scripts run from repo root; use repo-relative paths via `pathlib.Path(__file__).resolve().parents[N]`.
- All scenario tools share `scenario/runtime3_common.py` for IR types — extend it, don't redefine types per script.
## TTS Pool Backends (`tts/generate_npc_pool.py`)
Two backends, both idempotent (skip existing files; pass `--force` to rebuild):
- **`--backend piper`** (default, legacy): hits Tower :8001 OpenAI-compatible API, MP3 output → `hotline_tts/<key>.mp3`. Manifest: `hotline_tts/manifest.json`.
- **`--backend f5`** (P1 part9b): hits voice-bridge `/tts` on MacStudio, WAV output → `hotline_tts/f5/<key>.wav`. Manifest: `hotline_tts/f5/manifest.json` carries `cache_key`, `server_backend`, `server_cache_hit`, `server_latency_ms` per entry. Cache key derivation must mirror `tools/macstudio/voice-bridge/main.py::_voice_ref_token` — hash the `voice_ref` string verbatim (or sentinel `"default"` when omitted), never the file content.
## Validate Changes
```bash
tools/macstudio/voice-bridge/main.py
+291 -52
View File
@@ -1,27 +1,27 @@
"""voice-bridge — FastAPI daemon (P1 part7, in-process F5-TTS).
"""voice-bridge — FastAPI daemon (P1 part7 + part9b on-disk cache).
Spec: docs/superpowers/specs/2026-05-03-tts-stt-llm-macstudio-design.md §2.5
Routes
------
GET /health → readiness, F5 warm-up status, ref audio used
POST /tts → in-process F5-TTS-MLX, fallback Piper Tower:8001
GET /health → readiness, F5 warm-up, ref audio, cache stats
POST /tts → on-disk cache → F5-TTS-MLX → Piper Tower fallback
GET /tts/cache/stats → cache count / size / hit rate (since boot)
DELETE /tts/cache → clear the on-disk cache (admin key required)
POST /voice/transcribe → multipart proxy → whisper.cpp :8300 /inference
POST /voice/intent → forward → LiteLLM npc-fast (model="npc-fast")
Design notes
------------
* F5-TTS is loaded **once at boot** (single global F5TTS instance) and warmed
up with a short dummy phrase; `generate()` from `f5_tts_mlx.generate` is
bypassed because it (a) reloads the model on each call and (b) tries to
open a sounddevice when `output_path` is None — both unwanted in a daemon.
* Reference audio default: `~/zacus_reference.wav`. If absent, `/tmp/ref.wav`
is generated at boot via `say -v Thomas` + afconvert (24 kHz mono).
* TTS fallback: F5 wrapped in `asyncio.to_thread()` with 3 s timeout. On
timeout / hard error, POST to Piper Tower :8001 /synthesize. If both fail,
503.
* Master key for LiteLLM: `LITELLM_MASTER_KEY` env var (no hardcoded
production secret in source).
up with a short dummy phrase.
* Reference audio default: ``~/zacus_reference.wav``. If absent, ``/tmp/ref.wav``
is generated at boot via ``say -v Thomas`` + afconvert (24 kHz mono).
* On-disk cache (P1 part9b): ``~/voice-bridge/cache/<sha16>.wav`` keyed by
``sha256(text|ref_path|steps)``. The pool generator (tools/tts/generate_npc_pool.py)
uses the same key format so pre-warming is straightforward.
* TTS fallback chain on cache miss: F5 → Piper Tower :8001 → ``service_down.wav``
if both fail. F5 errors / timeouts are logged then bypassed.
"""
from __future__ import annotations
@@ -41,7 +41,7 @@ from typing import Any, Optional
import httpx
import numpy as np
import soundfile as sf
from fastapi import FastAPI, File, HTTPException, Request, Response, UploadFile
from fastapi import FastAPI, File, Header, HTTPException, Request, Response, UploadFile
from fastapi.responses import JSONResponse
# ── config (env-overridable) ────────────────────────────────────────────────
@@ -49,9 +49,9 @@ WHISPER_URL = os.getenv("WHISPER_URL", "http://localhost:8300")
LITELLM_URL = os.getenv("LITELLM_URL", "http://localhost:4000")
LITELLM_KEY = os.environ.get("LITELLM_MASTER_KEY", "sk-zacus-local-dev-do-not-share")
PIPER_URL = os.getenv("PIPER_URL", "http://192.168.0.120:8001")
F5_TIMEOUT_S = float(os.getenv("F5_TIMEOUT_S", "3.0"))
F5_TIMEOUT_S = float(os.getenv("F5_TIMEOUT_S", "8.0"))
F5_MODEL = os.getenv("F5_MODEL", "lucasnewman/f5-tts-mlx")
F5_DEFAULT_STEPS = int(os.getenv("F5_DEFAULT_STEPS", "8"))
F5_DEFAULT_STEPS = int(os.getenv("F5_DEFAULT_STEPS", "4"))
F5_SAMPLE_RATE = 24_000
REF_AUDIO_HOME = Path(os.path.expanduser("~/zacus_reference.wav"))
@@ -60,6 +60,12 @@ REF_AUDIO_TEXT_DEFAULT = (
"Bienvenue dans le mystère du Professeur Zacus."
)
CACHE_DIR = Path(os.path.expanduser(os.getenv("CACHE_DIR", "~/voice-bridge/cache")))
SERVICE_DOWN_WAV = Path(os.path.expanduser(
os.getenv("SERVICE_DOWN_WAV", "~/voice-bridge/service_down.wav")
))
ADMIN_KEY = os.getenv("VOICE_BRIDGE_ADMIN_KEY") # None = no auth on DELETE
# ── logging (single-line JSON) ──────────────────────────────────────────────
logging.basicConfig(
level=logging.INFO,
@@ -81,14 +87,20 @@ def _hash8(text: str) -> str:
_F5_MODEL_OBJ: Any = None
_REF_AUDIO_PATH: Optional[Path] = None
_REF_AUDIO_TEXT: str = REF_AUDIO_TEXT_DEFAULT
_REF_AUDIO_HASH: str = "default"
_WARMUP_MS: Optional[int] = None
_F5_LOAD_ERR: Optional[str] = None
# Serialization lock: F5/MLX operations must run in the asyncio main thread
# (MLX caches per-thread state at model-load time and breaks if reused from
# another thread, even with set_default_stream). The lock prevents two
# (MLX caches per-thread state at model-load time). The lock prevents two
# concurrent /tts calls from clobbering MLX state.
_F5_LOCK: Optional[asyncio.Lock] = None
# Cache index: cache_key → absolute path (warm in-memory after boot scan).
_CACHE_INDEX: dict[str, Path] = {}
_CACHE_LOCK: Optional[asyncio.Lock] = None
_CACHE_HITS: int = 0
_CACHE_MISSES: int = 0
# ── reference audio bootstrap ───────────────────────────────────────────────
def _ensure_ref_audio() -> Path:
@@ -109,7 +121,6 @@ def _ensure_ref_audio() -> Path:
"Cannot bootstrap fallback reference audio: 'say' or 'afconvert' missing"
)
# Generate AIFF then convert to WAV 24 kHz mono PCM 16-bit.
subprocess.run(
[say_bin, "-v", "Thomas", "-o", str(aiff), REF_AUDIO_TEXT_DEFAULT],
check=True,
@@ -128,14 +139,57 @@ def _ensure_ref_audio() -> Path:
return REF_AUDIO_FALLBACK
def _ref_hash(ref_path: Optional[Path]) -> str:
"""Short hash of the reference WAV (file-content based).
Used in /health to fingerprint the boot-time default reference; NOT used
in cache key derivation (see ``_voice_ref_token`` for that), so the pool
generator and the daemon agree on cache keys without sharing the ref file.
"""
if ref_path is None or not ref_path.exists():
return "default"
h = hashlib.sha256()
with open(ref_path, "rb") as fh:
for chunk in iter(lambda: fh.read(65536), b""):
h.update(chunk)
return h.hexdigest()[:16]
def _voice_ref_token(voice_ref: Optional[str]) -> str:
"""Cache-key token derived from the request's ``voice_ref`` field.
Contract (mirrored by ``tools/tts/generate_npc_pool.py``): when the client
omits ``voice_ref`` it gets the daemon's boot-time default reference, so
both sides hash the literal sentinel ``"default"``. Otherwise both sides
hash the ``voice_ref`` string (path or identifier) verbatim — never the
file contents — so the client can compute the cache_key locally without
needing the actual reference WAV.
"""
if not voice_ref:
return "default"
return hashlib.sha256(voice_ref.encode("utf-8")).hexdigest()[:16]
def _cache_key(text: str, ref_token: str, steps: int) -> str:
payload = f"{text}|{ref_token}|{steps}".encode("utf-8")
return hashlib.sha256(payload).hexdigest()[:16]
def _scan_cache_dir() -> dict[str, Path]:
"""Walk CACHE_DIR and build {cache_key: path} from existing .wav files."""
index: dict[str, Path] = {}
if not CACHE_DIR.exists():
return index
for wav in CACHE_DIR.glob("*.wav"):
# Use the file stem as key (set when we wrote it).
index[wav.stem] = wav
return index
# ── F5-TTS in-process synthesizer ───────────────────────────────────────────
def _f5_synthesize_sync(text: str, steps: int) -> bytes:
"""Run F5 inference on the cached model; returns 24 kHz mono WAV bytes.
Must execute on a thread that has a thread-local GPU stream (see
``_f5_thread_init``). The dedicated ``_F5_EXECUTOR`` guarantees this.
"""
import mlx.core as mx # noqa: WPS433 (deferred import for clarity)
"""Run F5 inference on the cached model; returns 24 kHz mono WAV bytes."""
import mlx.core as mx # noqa: WPS433
from f5_tts_mlx.utils import convert_char_to_pinyin
audio_arr, sr = sf.read(str(_REF_AUDIO_PATH))
@@ -173,36 +227,35 @@ def _f5_synthesize_sync(text: str, steps: int) -> bytes:
async def _run_f5(text: str, steps: int) -> bytes:
"""Run synthesis serialized in the asyncio main thread.
F5/MLX state is bound to the thread that loaded the model. Running from
a worker thread triggers ``RuntimeError: There is no Stream(gpu, 0)`` or
``scaled_dot_product_attention(scale: array)`` even with explicit stream
setup, so we keep everything on the event-loop thread and use an
``asyncio.Lock`` to serialize concurrent requests. This blocks the event
loop for the duration of synthesis; that is acceptable for the escape
room workload (max 2 concurrent voice clients).
"""
"""Run synthesis serialized in the asyncio main thread."""
assert _F5_LOCK is not None
async with _F5_LOCK:
return _f5_synthesize_sync(text, steps)
# ── FastAPI app ──────────────────────────────────────────────────────────────
app = FastAPI(title="voice-bridge", version="0.2.0")
app = FastAPI(title="voice-bridge", version="0.3.0")
@app.on_event("startup")
async def _boot() -> None:
global _F5_MODEL_OBJ, _REF_AUDIO_PATH, _WARMUP_MS, _F5_LOAD_ERR
global _F5_LOCK
global _F5_MODEL_OBJ, _REF_AUDIO_PATH, _REF_AUDIO_HASH, _WARMUP_MS
global _F5_LOAD_ERR, _F5_LOCK, _CACHE_INDEX, _CACHE_LOCK
_F5_LOCK = asyncio.Lock()
_CACHE_LOCK = asyncio.Lock()
# Cache directory + initial index (cheap; just lists .wav stems).
CACHE_DIR.mkdir(parents=True, exist_ok=True)
_CACHE_INDEX = _scan_cache_dir()
_jlog("boot_cache_index", dir=str(CACHE_DIR), entries=len(_CACHE_INDEX))
try:
_REF_AUDIO_PATH = _ensure_ref_audio()
_jlog("boot_ref_audio", path=str(_REF_AUDIO_PATH))
except Exception as exc: # pragma: no cover - defensive
_REF_AUDIO_HASH = _ref_hash(_REF_AUDIO_PATH)
_jlog("boot_ref_audio", path=str(_REF_AUDIO_PATH),
ref_hash=_REF_AUDIO_HASH)
except Exception as exc: # pragma: no cover
_F5_LOAD_ERR = f"ref_audio: {exc}"
_jlog("boot_ref_audio_failed", err=str(exc))
return
@@ -215,7 +268,6 @@ async def _boot() -> None:
_jlog("boot_f5_loaded", model=F5_MODEL,
load_ms=int((time.monotonic() - t0) * 1000))
# Warm-up: short dummy synth in the same thread that holds the model.
t1 = time.monotonic()
_ = _f5_synthesize_sync("Test.", steps=F5_DEFAULT_STEPS)
_WARMUP_MS = int((time.monotonic() - t1) * 1000)
@@ -228,13 +280,23 @@ async def _boot() -> None:
@app.get("/health")
async def health() -> dict[str, Any]:
cache_size_b = sum(p.stat().st_size for p in _CACHE_INDEX.values()
if p.exists())
return {
"status": "ok" if _F5_MODEL_OBJ is not None else "degraded",
"version": app.version,
"f5_loaded": _F5_MODEL_OBJ is not None,
"f5_load_error": _F5_LOAD_ERR,
"ref_audio_used": str(_REF_AUDIO_PATH) if _REF_AUDIO_PATH else None,
"ref_hash": _REF_AUDIO_HASH,
"model_warmup_ms": _WARMUP_MS,
"cache": {
"dir": str(CACHE_DIR),
"entries": len(_CACHE_INDEX),
"size_mb": round(cache_size_b / (1024 * 1024), 3),
"hits": _CACHE_HITS,
"misses": _CACHE_MISSES,
},
"backends": {
"whisper": WHISPER_URL,
"litellm": LITELLM_URL,
@@ -243,9 +305,15 @@ async def health() -> dict[str, Any]:
}
# ── /tts (F5 primary, Piper fallback) ───────────────────────────────────────
# ── /tts (cache → F5 primary, Piper fallback, service_down last) ─────────────
async def _piper_fallback(text: str) -> Optional[bytes]:
"""POST text to Piper Tower :8001/synthesize. Returns WAV bytes or None."""
"""POST text to Piper Tower :8001/synthesize. Returns WAV bytes or None.
NOTE (P1 part9b): the Piper backend is kept as a *fallback only* — F5 is
the primary TTS path. This block is intentionally not removed so the
daemon stays operational if F5 fails to load. Marked DEPRECATED for
primary use; see voice-bridge spec §2.5.
"""
try:
async with httpx.AsyncClient(timeout=10.0) as client:
resp = await client.post(
@@ -260,35 +328,116 @@ async def _piper_fallback(text: str) -> Optional[bytes]:
return None
def _service_down_response(request_id: str, latency_ms: int,
err_kind: Optional[str]) -> Optional[Response]:
"""Return service_down.wav when both F5 and Piper are unavailable."""
if not SERVICE_DOWN_WAV.exists():
return None
try:
wav_bytes = SERVICE_DOWN_WAV.read_bytes()
except OSError as exc:
_jlog("service_down_read_err", err=str(exc))
return None
_jlog("tts", request_id=request_id, tts_backend_used="service_down",
latency_ms=latency_ms, error=err_kind)
return Response(
content=wav_bytes,
media_type="audio/wav",
headers={
"X-TTS-Backend": "service_down",
"X-TTS-Cache-Hit": "false",
"X-TTS-Latency-Ms": str(latency_ms),
"X-Request-Id": request_id,
},
)
async def _write_cache(cache_key: str, wav_bytes: bytes) -> None:
"""Persist a freshly synthesized WAV under CACHE_DIR/<key>.wav."""
assert _CACHE_LOCK is not None
out = CACHE_DIR / f"{cache_key}.wav"
try:
# Atomic-ish write: tmp file + rename so partial writes never appear.
tmp = out.with_suffix(".wav.tmp")
tmp.write_bytes(wav_bytes)
tmp.replace(out)
async with _CACHE_LOCK:
_CACHE_INDEX[cache_key] = out
_jlog("cache_write", cache_key=cache_key, bytes=len(wav_bytes))
except OSError as exc:
_jlog("cache_write_err", cache_key=cache_key, err=str(exc))
@app.post("/tts")
async def tts(payload: dict[str, Any]) -> Response:
global _CACHE_HITS, _CACHE_MISSES
request_id = str(uuid.uuid4())
text = (payload.get("text") or payload.get("input") or "").strip()
if not text:
raise HTTPException(status_code=400, detail="payload.text is required")
steps = int(payload.get("steps", F5_DEFAULT_STEPS))
voice_ref = payload.get("voice_ref") # accepted but ignored (single-voice)
voice_ref = payload.get("voice_ref") # accepted; reserved for multi-voice
started = time.monotonic()
backend = "unknown"
# 1) Cache lookup -------------------------------------------------------
cache_key = _cache_key(text, _voice_ref_token(voice_ref), steps)
cached_path = _CACHE_INDEX.get(cache_key)
if cached_path is None:
# Fallback: file may have been dropped in by the pool generator after
# boot; re-check the dir lazily.
candidate = CACHE_DIR / f"{cache_key}.wav"
if candidate.exists():
cached_path = candidate
_CACHE_INDEX[cache_key] = candidate
if cached_path is not None and cached_path.exists():
try:
wav_bytes = cached_path.read_bytes()
_CACHE_HITS += 1
latency_ms = int((time.monotonic() - started) * 1000)
_jlog("tts", request_id=request_id, tts_backend_used="cache",
latency_ms=latency_ms, phrase_len=len(text),
text_hash=_hash8(text), cache_key=cache_key)
return Response(
content=wav_bytes,
media_type="audio/wav",
headers={
"X-TTS-Backend": "cache",
"X-TTS-Cache-Hit": "true",
"X-TTS-Cache-Key": cache_key,
"X-TTS-Latency-Ms": str(latency_ms),
"X-Request-Id": request_id,
},
)
except OSError as exc:
_jlog("cache_read_err", cache_key=cache_key, err=str(exc))
# Fall through and regenerate.
_CACHE_MISSES += 1
err_kind: Optional[str] = None
# 2) F5 synthesis (cache miss) -----------------------------------------
if _F5_MODEL_OBJ is not None and _F5_LOCK is not None:
try:
wav_bytes = await asyncio.wait_for(
_run_f5(text, steps),
timeout=F5_TIMEOUT_S,
)
backend = "f5"
await _write_cache(cache_key, wav_bytes)
latency_ms = int((time.monotonic() - started) * 1000)
_jlog("tts", request_id=request_id, tts_backend_used=backend,
_jlog("tts", request_id=request_id, tts_backend_used="f5",
latency_ms=latency_ms, phrase_len=len(text),
text_hash=_hash8(text), voice_ref=voice_ref, steps=steps)
text_hash=_hash8(text), voice_ref=voice_ref, steps=steps,
cache_key=cache_key)
return Response(
content=wav_bytes,
media_type="audio/wav",
headers={
"X-TTS-Backend": "f5",
"X-TTS-Cache-Hit": "false",
"X-TTS-Cache-Key": cache_key,
"X-TTS-Latency-Ms": str(latency_ms),
"X-Request-Id": request_id,
},
@@ -304,12 +453,11 @@ async def tts(payload: dict[str, Any]) -> Response:
err_kind = "f5_not_loaded"
_jlog("tts_f5_not_loaded", request_id=request_id, load_err=_F5_LOAD_ERR)
# Fallback: Piper Tower
# 3) Piper Tower fallback (DEPRECATED for primary path; safety-net only)
wav_fallback = await _piper_fallback(text)
if wav_fallback is not None:
backend = "piper_fallback"
latency_ms = int((time.monotonic() - started) * 1000)
_jlog("tts", request_id=request_id, tts_backend_used=backend,
_jlog("tts", request_id=request_id, tts_backend_used="piper_fallback",
latency_ms=latency_ms, phrase_len=len(text),
text_hash=_hash8(text), error=err_kind)
return Response(
@@ -317,19 +465,110 @@ async def tts(payload: dict[str, Any]) -> Response:
media_type="audio/wav",
headers={
"X-TTS-Backend": "piper_fallback",
"X-TTS-Cache-Hit": "false",
"X-TTS-Latency-Ms": str(latency_ms),
"X-Request-Id": request_id,
},
)
# 4) service_down.wav last resort --------------------------------------
latency_ms = int((time.monotonic() - started) * 1000)
sd = _service_down_response(request_id, latency_ms, err_kind)
if sd is not None:
return sd
_jlog("tts_all_backends_down", request_id=request_id,
phrase_len=len(text), error=err_kind)
raise HTTPException(
status_code=503,
detail=f"TTS unavailable: F5 ({err_kind}) and Piper fallback both failed",
detail=(
f"TTS unavailable: F5 ({err_kind}), Piper fallback, and "
"service_down.wav all failed"
),
)
# ── /tts/service_down (direct WAV serve, smoke-test friendly) ───────────────
@app.get("/tts/service_down")
async def tts_service_down() -> Response:
"""Serve the pre-rendered fallback WAV directly, no F5 / Piper call.
Useful for smoke-testing the resilience asset without forcing a fault.
Added in P1 part9a (resilience pass).
"""
if not SERVICE_DOWN_WAV.exists():
raise HTTPException(
status_code=503,
detail=f"service_down WAV missing at {SERVICE_DOWN_WAV}",
)
try:
wav_bytes = SERVICE_DOWN_WAV.read_bytes()
except OSError as exc:
raise HTTPException(
status_code=500,
detail=f"service_down WAV unreadable: {exc}",
)
return Response(
content=wav_bytes,
media_type="audio/wav",
headers={
"X-TTS-Backend": "service_down",
"X-TTS-Bytes": str(len(wav_bytes)),
},
)
# ── /tts/cache/{stats,clear} ────────────────────────────────────────────────
@app.get("/tts/cache/stats")
async def cache_stats() -> dict[str, Any]:
total_b = 0
valid_count = 0
for p in _CACHE_INDEX.values():
try:
total_b += p.stat().st_size
valid_count += 1
except OSError:
continue
total = _CACHE_HITS + _CACHE_MISSES
hit_rate = round(_CACHE_HITS / total, 4) if total else 0.0
return {
"dir": str(CACHE_DIR),
"count": valid_count,
"size_mb": round(total_b / (1024 * 1024), 3),
"hits": _CACHE_HITS,
"misses": _CACHE_MISSES,
"hit_rate_since_boot": hit_rate,
}
@app.delete("/tts/cache")
async def cache_clear(
x_admin_key: Optional[str] = Header(default=None, alias="X-Admin-Key"),
) -> dict[str, Any]:
if ADMIN_KEY and x_admin_key != ADMIN_KEY:
raise HTTPException(status_code=403, detail="invalid admin key")
assert _CACHE_LOCK is not None
removed = 0
async with _CACHE_LOCK:
for p in list(_CACHE_INDEX.values()):
try:
p.unlink()
removed += 1
except OSError:
continue
_CACHE_INDEX.clear()
# Also sweep any stragglers not in the index.
for stray in CACHE_DIR.glob("*.wav"):
try:
stray.unlink()
removed += 1
except OSError:
continue
_jlog("cache_cleared", removed=removed)
return {"removed": removed}
# ── /voice/transcribe (multipart proxy → whisper.cpp) ───────────────────────
@app.post("/voice/transcribe")
async def transcribe(request: Request) -> Response:
tools/macstudio/voice-bridge/watchdog.sh
Executable
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
# voice-bridge watchdog
#
# Idempotent: silent exit when service is up; restarts via the same nohup
# pattern as the @reboot crontab line when down. Designed to be invoked
# every 2 minutes by cron.
#
# Layout (on studio):
# ~/voice-bridge/ venv + main.py + service_down.wav + this script
# ~/voice-bridge.log uvicorn stdout/stderr (also reads watchdog notes)
# ~/voice-bridge-watchdog.log cron capture (this script's own stdout/stderr)
#
# Exit codes
# 0 service was already up, or restart issued
# 2 restart attempted but venv/script missing (operator action required)
set -u
VB_DIR="$HOME/voice-bridge"
VB_VENV="$VB_DIR/.venv/bin/python"
VB_LOG="$HOME/voice-bridge.log"
VB_PORT=8200
VB_PATTERN="uvicorn main:app.*--port ${VB_PORT}"
# Service is up: silent exit (no log spam every 2 min).
if pgrep -f "${VB_PATTERN}" >/dev/null 2>&1; then
exit 0
fi
# Down: capture timestamp + log into uvicorn log (operator-visible).
TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
REASON="no process matching '${VB_PATTERN}'"
if [[ ! -x "${VB_VENV}" ]] || [[ ! -f "${VB_DIR}/main.py" ]]; then
echo "${TS} watchdog: cannot restart, missing venv or main.py (${VB_DIR})" >> "${VB_LOG}"
exit 2
fi
echo "${TS} watchdog: restarted at ${TS} (reason: ${REASON})" >> "${VB_LOG}"
nohup "${VB_VENV}" -m uvicorn main:app \
--host 0.0.0.0 --port ${VB_PORT} \
--app-dir "${VB_DIR}" \
>> "${VB_LOG}" 2>&1 < /dev/null &
# Detach so cron doesn't wait on the child.
disown 2>/dev/null || true
exit 0
tools/tts/generate_npc_pool.py
+327 -69
View File
@@ -1,28 +1,47 @@
#!/usr/bin/env python3
"""generate_npc_pool.py — Professor Zacus NPC MP3 pool generator.
"""generate_npc_pool.py — Professor Zacus NPC audio pool generator.
Reads npc_phrases.yaml and synthesizes each phrase via Piper TTS API,
then writes MP3 files under --output directory and a manifest.json index.
Reads npc_phrases.yaml and synthesizes each phrase via either:
- Piper TTS API (legacy backend, MP3 output) — default for backwards compat.
- voice-bridge F5-TTS (P1 part9b backend, WAV output) — preferred for the
Zacus reference voice.
Usage:
python3 generate_npc_pool.py [options]
Both backends write a manifest.json index that maps phrase keys to audio
files. Generators are idempotent: existing files are skipped unless
``--force`` is passed (cf. tools/CLAUDE.md convention).
--tts-url Piper TTS base URL (default: http://192.168.0.120:8001)
--voice TTS voice name (default: zacus)
--output Output directory for MP3 files (default: hotline_tts)
--manifest Path to output manifest JSON (default: hotline_tts/manifest.json)
--phrases Path to npc_phrases.yaml (default: game/scenarios/npc_phrases.yaml)
--dry-run Print keys without calling TTS API
Usage (Piper, legacy):
python3 generate_npc_pool.py
Usage (F5 via voice-bridge):
python3 generate_npc_pool.py \
--backend f5 \
--voice-bridge-url http://192.168.0.150:8200 \
--steps 8
Common flags:
--output Output directory (default: hotline_tts)
--manifest Path to output manifest JSON
--phrases Path to npc_phrases.yaml
--force Re-synthesize even if cached file exists
--dry-run Print phrase keys without calling TTS
--delay Delay between requests (default: 0.3 s)
Backend-specific flags (F5 only):
--voice-bridge-url Base URL of voice-bridge daemon
--steps F5 inference steps (8 = best quality offline pool)
--ref-wav Optional reference WAV path (passed as voice_ref)
"""
from __future__ import annotations
import argparse
import hashlib
import json
import sys
import time
from pathlib import Path
from typing import Generator
from typing import Generator, Optional
# pyyaml is a soft dependency — error early with a clear message
try:
@@ -72,33 +91,57 @@ def load_phrases(phrases_path: Path) -> list[tuple[str, str]]:
# ---------------------------------------------------------------------------
# Key → safe file name
# Filename / cache key helpers
# ---------------------------------------------------------------------------
def key_to_filename(key: str) -> str:
def key_to_filename(key: str, ext: str = "mp3") -> str:
"""Convert a dotted phrase key to a safe file name.
Example: "hints.SCENE_LA_DETECTOR.level_1.0" → "hints__SCENE_LA_DETECTOR__level_1__0.mp3"
"""
safe = key.replace(".", "__")
# Remove any chars outside [a-zA-Z0-9_-]
safe = "".join(c if c.isalnum() or c in "_-" else "_" for c in safe)
return f"{safe}.mp3"
return f"{safe}.{ext}"
def f5_cache_key(text: str, ref_hash: str, steps: int) -> str:
"""Compute the SHA-256 cache key used by both pool & voice-bridge.
Mirrors the voice-bridge cache key format so that pool entries can be
pre-warmed into the daemon cache (drop the .wav into the cache dir and
the daemon will hit it on next request).
"""
payload = f"{text}|{ref_hash}|{steps}".encode("utf-8")
return hashlib.sha256(payload).hexdigest()[:16]
def voice_ref_token(ref_path: Optional[Path]) -> str:
"""Cache-key token derived from the optional reference path.
Mirrors the daemon contract (see ``main.py::_voice_ref_token``):
* When ``ref_path`` is None, the pool gen lets the daemon pick its
boot-time default reference, so both sides hash the literal
sentinel ``"default"``.
* Otherwise both sides hash the path string verbatim — never the
file content — so the client can compute the cache_key without
needing the daemon's reference WAV.
"""
if ref_path is None:
return "default"
return hashlib.sha256(str(ref_path).encode("utf-8")).hexdigest()[:16]
# ---------------------------------------------------------------------------
# TTS synthesis
# Piper TTS synthesis (legacy backend)
# ---------------------------------------------------------------------------
def synthesize(text: str, voice: str, tts_url: str, timeout_s: float = 30.0) -> bytes:
"""POST text to OpenAI-compatible TTS API and return raw MP3 bytes.
def synthesize_piper(text: str, voice: str, tts_url: str, timeout_s: float = 30.0) -> bytes:
"""POST text to OpenAI-compatible Piper API and return raw MP3 bytes.
openedai-speech on Tower:8001 (OpenAI-compatible):
POST /v1/audio/speech
Body: {"model": "tts-1", "voice": "...", "input": "...", "response_format": "mp3"}
Response: audio/mpeg binary
Raises requests.HTTPError on non-2xx responses.
"""
url = tts_url.rstrip("/") + "/v1/audio/speech"
payload = {
@@ -113,110 +156,166 @@ def synthesize(text: str, voice: str, tts_url: str, timeout_s: float = 30.0) ->
# ---------------------------------------------------------------------------
# Main
# voice-bridge F5 synthesis (P1 part9b backend)
# ---------------------------------------------------------------------------
def synthesize_f5(
text: str,
voice_bridge_url: str,
steps: int,
ref_wav: Optional[str] = None,
timeout_s: float = 60.0,
) -> tuple[bytes, dict[str, str]]:
"""POST text to voice-bridge /tts and return (WAV bytes, response headers).
The returned headers include diagnostic info such as ``X-TTS-Backend``
(``f5``, ``cache``, ``piper_fallback``), ``X-TTS-Cache-Hit`` and
``X-TTS-Latency-Ms`` — these get folded into the manifest for traceability.
"""
url = voice_bridge_url.rstrip("/") + "/tts"
payload: dict[str, object] = {"text": text, "steps": steps}
if ref_wav:
payload["voice_ref"] = ref_wav
resp = requests.post(url, json=payload, timeout=timeout_s)
resp.raise_for_status()
return resp.content, dict(resp.headers)
# ---------------------------------------------------------------------------
# CLI
# ---------------------------------------------------------------------------
def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Generate MP3 pool for Professor Zacus NPC phrases via Piper TTS."
description=(
"Generate audio pool for Professor Zacus NPC phrases via Piper TTS "
"or the voice-bridge F5 backend."
)
)
repo_root = Path(__file__).resolve().parents[2]
parser.add_argument(
"--backend",
choices=("piper", "f5"),
default="piper",
help="TTS backend (default: piper for backwards compat).",
)
# Piper-specific
parser.add_argument(
"--tts-url",
default="http://192.168.0.120:8001",
help="Piper TTS base URL (default: http://192.168.0.120:8001)",
help="Piper TTS base URL (default: http://192.168.0.120:8001).",
)
parser.add_argument(
"--voice",
default="zacus",
help="TTS voice name (default: zacus)",
help="Piper voice name (default: zacus).",
)
# F5-specific
parser.add_argument(
"--voice-bridge-url",
default="http://192.168.0.150:8200",
help="voice-bridge base URL (default: http://192.168.0.150:8200).",
)
parser.add_argument(
"--steps",
type=int,
default=8,
help="F5 inference steps (default: 8 = best for offline pool).",
)
parser.add_argument(
"--ref-wav",
type=Path,
default=None,
help=(
"Optional reference WAV passed to voice-bridge as voice_ref. "
"When absent, voice-bridge uses its boot-time default reference."
),
)
# Common
parser.add_argument(
"--output",
type=Path,
default=repo_root / "hotline_tts",
help="Output directory for MP3 files (default: <repo>/hotline_tts)",
help="Output directory (default: <repo>/hotline_tts).",
)
parser.add_argument(
"--manifest",
type=Path,
default=None,
help="Path to output manifest JSON (default: <output>/manifest.json)",
help="Path to output manifest JSON (default: <output>/<backend>/manifest.json for f5, <output>/manifest.json for piper).",
)
parser.add_argument(
"--phrases",
type=Path,
default=repo_root / "game" / "scenarios" / "npc_phrases.yaml",
help="Path to npc_phrases.yaml",
help="Path to npc_phrases.yaml.",
)
parser.add_argument(
"--force",
action="store_true",
help="Re-synthesize even if the cached audio file already exists.",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="Print phrase keys and texts without calling TTS API",
help="Print phrase keys and texts without calling TTS API.",
)
parser.add_argument(
"--delay",
type=float,
default=0.3,
help="Delay in seconds between TTS requests (default: 0.3)",
help="Delay in seconds between TTS requests (default: 0.3).",
)
parser.add_argument(
"--limit",
type=int,
default=0,
help="Process only the first N phrases (0 = all). Useful for smoke tests.",
)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
# ---------------------------------------------------------------------------
# Per-backend run loops
# ---------------------------------------------------------------------------
# Resolve manifest path default
def _run_piper(args: argparse.Namespace, phrases: list[tuple[str, str]]) -> int:
args.output.mkdir(parents=True, exist_ok=True)
manifest_path: Path = args.manifest or (args.output / "manifest.json")
# Validate inputs
if not args.phrases.exists():
print(f"ERROR: phrases file not found: {args.phrases}", file=sys.stderr)
return 1
phrases = load_phrases(args.phrases)
if not phrases:
print("ERROR: no phrases found in YAML file", file=sys.stderr)
return 1
print(f"Loaded {len(phrases)} phrases from {args.phrases}")
if args.dry_run:
print("\n--- DRY RUN --- (no TTS calls)")
for key, text in phrases:
print(f" [{key}] {text[:80]}{'…' if len(text) > 80 else ''}")
print(f"\nTotal: {len(phrases)} phrases")
return 0
# Create output directory
args.output.mkdir(parents=True, exist_ok=True)
manifest: dict[str, str] = {}
failed: list[tuple[str, str, str]] = [] # (key, text, error)
failed: list[tuple[str, str, str]] = []
succeeded = 0
for idx, (key, text) in enumerate(phrases, start=1):
filename = key_to_filename(key)
filename = key_to_filename(key, ext="mp3")
out_path = args.output / filename
rel_path = str(out_path.relative_to(args.output.parent)
if out_path.is_relative_to(args.output.parent)
else out_path)
rel_path = (
str(out_path.relative_to(args.output.parent))
if out_path.is_relative_to(args.output.parent)
else str(out_path)
)
print(f"[{idx}/{len(phrases)}] {key} → {filename}", end="", flush=True)
# Skip if already generated (idempotent re-run)
if out_path.exists() and out_path.stat().st_size > 0:
if out_path.exists() and out_path.stat().st_size > 0 and not args.force:
print(" (cached)")
manifest[key] = rel_path
succeeded += 1
continue
try:
audio_bytes = synthesize(text, args.voice, args.tts_url)
t0 = time.monotonic()
audio_bytes = synthesize_piper(text, args.voice, args.tts_url)
out_path.write_bytes(audio_bytes)
manifest[key] = rel_path
succeeded += 1
print(f" OK ({len(audio_bytes)} bytes)")
dt_ms = int((time.monotonic() - t0) * 1000)
print(f" OK ({len(audio_bytes)} B in {dt_ms} ms)")
except requests.exceptions.ConnectionError as exc:
print(f" FAIL (connection error: {exc})")
failed.append((key, text, f"ConnectionError: {exc}"))
@@ -233,22 +332,181 @@ def main(argv: list[str] | None = None) -> int:
if args.delay > 0:
time.sleep(args.delay)
# Write manifest
manifest_path.parent.mkdir(parents=True, exist_ok=True)
with open(manifest_path, "w", encoding="utf-8") as fh:
json.dump(manifest, fh, ensure_ascii=False, indent=2)
print(f"\nManifest written: {manifest_path} ({len(manifest)} entries)")
# Summary
print(f"\nSummary: {succeeded} succeeded, {len(failed)} failed out of {len(phrases)} phrases")
print(
f"\nSummary: {succeeded} succeeded, {len(failed)} failed "
f"out of {len(phrases)} phrases"
)
if failed:
print("\nFailed phrases:")
for key, text, error in failed:
print(f" [{key}] {error}")
print(f" text: {text[:80]}{'…' if len(text) > 80 else ''}")
return 0 if not failed else 1
def _run_f5(args: argparse.Namespace, phrases: list[tuple[str, str]]) -> int:
f5_dir = args.output / "f5"
f5_dir.mkdir(parents=True, exist_ok=True)
manifest_path: Path = args.manifest or (f5_dir / "manifest.json")
ref_hash = voice_ref_token(args.ref_wav)
print(
f"F5 backend → {args.voice_bridge_url} (steps={args.steps}, "
f"ref_hash={ref_hash})"
)
manifest_entries: dict[str, dict[str, object]] = {}
failed: list[tuple[str, str, str]] = []
succeeded = 0
for idx, (key, text) in enumerate(phrases, start=1):
# Prefer the explicit YAML key for the file name (stable, human-readable);
# the SHA-256 cache_key is also recorded for daemon-side cache pinning.
filename = key_to_filename(key, ext="wav")
out_path = f5_dir / filename
rel_path = (
str(out_path.relative_to(args.output.parent))
if out_path.is_relative_to(args.output.parent)
else str(out_path)
)
cache_key = f5_cache_key(text, ref_hash, args.steps)
print(
f"[{idx}/{len(phrases)}] {key} → {filename} (ck={cache_key})",
end="",
flush=True,
)
if out_path.exists() and out_path.stat().st_size > 0 and not args.force:
print(" (cached)")
manifest_entries[key] = {
"cache_key": cache_key,
"audio_path": rel_path,
"text": text,
"voice_bridge_url": args.voice_bridge_url,
"steps": args.steps,
"ref_hash": ref_hash,
"cached": True,
}
succeeded += 1
continue
try:
t0 = time.monotonic()
wav_bytes, hdrs = synthesize_f5(
text,
voice_bridge_url=args.voice_bridge_url,
steps=args.steps,
ref_wav=str(args.ref_wav) if args.ref_wav else None,
)
out_path.write_bytes(wav_bytes)
dt_ms = int((time.monotonic() - t0) * 1000)
backend_used = hdrs.get("x-tts-backend") or hdrs.get("X-TTS-Backend") or "?"
cache_hit = hdrs.get("x-tts-cache-hit") or hdrs.get("X-TTS-Cache-Hit") or "?"
srv_lat = hdrs.get("x-tts-latency-ms") or hdrs.get("X-TTS-Latency-Ms") or "?"
manifest_entries[key] = {
"cache_key": cache_key,
"audio_path": rel_path,
"text": text,
"voice_bridge_url": args.voice_bridge_url,
"steps": args.steps,
"ref_hash": ref_hash,
"gen_at_ms": int(time.time() * 1000),
"server_backend": backend_used,
"server_cache_hit": cache_hit,
"server_latency_ms": srv_lat,
}
succeeded += 1
print(
f" OK ({len(wav_bytes) // 1024} kB in {dt_ms} ms, "
f"backend={backend_used}, hit={cache_hit})"
)
except requests.exceptions.ConnectionError as exc:
print(f" FAIL (connection error: {exc})")
failed.append((key, text, f"ConnectionError: {exc}"))
print(
"\nFATAL: voice-bridge is unreachable; aborting (no point retrying).",
file=sys.stderr,
)
break
except requests.exceptions.Timeout:
print(" FAIL (timeout)")
failed.append((key, text, "Timeout"))
except requests.exceptions.HTTPError as exc:
body = exc.response.text[:200] if exc.response is not None else ""
print(f" FAIL (HTTP {exc.response.status_code}: {body})")
failed.append((key, text, f"HTTPError: {exc}"))
except Exception as exc: # noqa: BLE001
print(f" FAIL ({type(exc).__name__}: {exc})")
failed.append((key, text, str(exc)))
if args.delay > 0:
time.sleep(args.delay)
manifest = {
"backend": "f5",
"voice_bridge_url": args.voice_bridge_url,
"steps": args.steps,
"ref_hash": ref_hash,
"ref_wav": str(args.ref_wav) if args.ref_wav else None,
"generated_at_ms": int(time.time() * 1000),
"entries": manifest_entries,
}
manifest_path.parent.mkdir(parents=True, exist_ok=True)
with open(manifest_path, "w", encoding="utf-8") as fh:
json.dump(manifest, fh, ensure_ascii=False, indent=2)
print(f"\nManifest written: {manifest_path} ({len(manifest_entries)} entries)")
print(
f"\nSummary: {succeeded} succeeded, {len(failed)} failed "
f"out of {len(phrases)} phrases"
)
if failed:
print("\nFailed phrases:")
for key, text, error in failed:
print(f" [{key}] {error}")
print(f" text: {text[:80]}{'…' if len(text) > 80 else ''}")
return 0 if not failed else 1
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
if not args.phrases.exists():
print(f"ERROR: phrases file not found: {args.phrases}", file=sys.stderr)
return 1
phrases = load_phrases(args.phrases)
if not phrases:
print("ERROR: no phrases found in YAML file", file=sys.stderr)
return 1
if args.limit > 0:
phrases = phrases[: args.limit]
print(
f"Loaded {len(phrases)} phrases from {args.phrases} "
f"(backend={args.backend})"
)
if args.dry_run:
print("\n--- DRY RUN --- (no TTS calls)")
for key, text in phrases:
print(f" [{key}] {text[:80]}{'…' if len(text) > 80 else ''}")
print(f"\nTotal: {len(phrases)} phrases")
return 0
if args.backend == "piper":
return _run_piper(args, phrases)
return _run_f5(args, phrases)
if __name__ == "__main__":
sys.exit(main())