L'électron rare
20aed903ba
- Voice pipeline: ESP32 WebSocket client → voice bridge → LLM → Piper TTS (Tower :8001) - Hints engine: 3 puzzles (LA_440, LEFOU_PIANO, QR_FINALE), anti-cheat, 3 hint levels - MCP hardware server: 6 tools (puzzle, audio, LED, camera, scenario, status), stdio transport - Analytics: ESP32 module + 6 web endpoints + Dashboard UI with chat interface - Security: auth middleware (Bearer NVS), rate limiting, input validation on 30 endpoints - Frontend: code-split (1.1MB → 210KB initial), ErrorBoundary, API timeout, WS reconnect - Tests: 24 Python + 38 TypeScript + 18 MCP = 80 project tests (+ 19 mascarade) - Specs: AI_INTEGRATION_SPEC, MCP_HARDWARE_SERVER_SPEC, QA_TEST_MATRIX_SPEC - Docs: SECURITY, DEPLOYMENT_RUNBOOK, voice pipeline guide, AI architecture map - 6 AI agent definitions (.github/agents/ai_*.md) - TUI orchestration script (tools/dev/zacus_tui.py) - Docker compose TTS for Tower + KXKM-AI - CHANGELOG, README, mkdocs.yml updated - Cycle detection (DFS) in runtime3 validator - Sprint plan: plans/SPRINT_AI_INTEGRATION.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1.6 KiB
1.6 KiB
Custom Agent – AI TTS / Voice Cloning
Scope
Server-side text-to-speech, Professor Zacus voice cloning, and audio streaming to ESP32 devices.
Technologies
- Coqui XTTS-v2 (voice cloning), Piper TTS (fast fallback)
- Docker deployment on mascarade stack
- PCM/Opus streaming over HTTP chunked transfer
Do
- Prepare and curate voice samples for Professor Zacus persona (≥ 30 s clean audio).
- Create Docker Compose service (
zacus-tts) integrated with mascarade stack. - Expose REST API:
POST /tts/generate(text → audio),POST /tts/stream(chunked). - Implement audio format conversion (WAV → PCM 16-bit 16 kHz) for ESP32 I2S playback.
- Cache frequently used phrases to reduce GPU load.
Must Not
- Store voice samples in git; keep them in object storage or Docker volumes.
- Bypass mascarade auth on the TTS API endpoints.
Dependencies
- mascarade Docker stack — networking, auth, service registry.
- ESP32 audio system — I2S DAC output and buffer management.
Test Gates
- Latency < 2 s for a 10-word sentence (first token to last byte).
- Voice similarity > 80% (speaker verification cosine similarity).
References
- Coqui XTTS-v2: https://github.com/coqui-ai/TTS
- Piper TTS: https://github.com/rhasspy/piper
Plan d'action
- Construire et démarrer le service TTS Docker.
- run: docker compose -f docker-compose.ai.yml up -d zacus-tts
- Vérifier la latence de génération.
- run: curl -w '%{time_total}' -X POST http://localhost:5500/tts/generate -d '{"text":"Bonjour explorateurs"}'
- Valider la similarité vocale sur les échantillons de référence.
- run: python3 tools/ai/tts_similarity_bench.py --threshold 0.80