- Voice pipeline: ESP32 WebSocket client → voice bridge → LLM → Piper TTS (Tower :8001) - Hints engine: 3 puzzles (LA_440, LEFOU_PIANO, QR_FINALE), anti-cheat, 3 hint levels - MCP hardware server: 6 tools (puzzle, audio, LED, camera, scenario, status), stdio transport - Analytics: ESP32 module + 6 web endpoints + Dashboard UI with chat interface - Security: auth middleware (Bearer NVS), rate limiting, input validation on 30 endpoints - Frontend: code-split (1.1MB → 210KB initial), ErrorBoundary, API timeout, WS reconnect - Tests: 24 Python + 38 TypeScript + 18 MCP = 80 project tests (+ 19 mascarade) - Specs: AI_INTEGRATION_SPEC, MCP_HARDWARE_SERVER_SPEC, QA_TEST_MATRIX_SPEC - Docs: SECURITY, DEPLOYMENT_RUNBOOK, voice pipeline guide, AI architecture map - 6 AI agent definitions (.github/agents/ai_*.md) - TUI orchestration script (tools/dev/zacus_tui.py) - Docker compose TTS for Tower + KXKM-AI - CHANGELOG, README, mkdocs.yml updated - Cycle detection (DFS) in runtime3 validator - Sprint plan: plans/SPRINT_AI_INTEGRATION.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
13 KiB
MCP Hardware Server Specification
Status
- State: draft
- Date: 2026-03-21
- Depends on:
AI_INTEGRATION_SPEC.md,FIRMWARE_WEB_DATA_CONTRACT.md - Reference: MCP specification (modelcontextprotocol.io), ESP RainMaker MCP pattern, IoT-MCP (Duke CEI)
1) Objective
Define an MCP (Model Context Protocol) server that exposes ESP32-S3 hardware capabilities as LLM-callable tools. This enables natural-language hardware control from mascarade, game master dashboards, and automated scenario orchestration.
2) Architecture
flowchart LR
subgraph LLM["mascarade (LLM Host)"]
Client[MCP Client<br/>call_tool_http]
Registry[MCP Registry]
end
subgraph MCPServer["MCP Hardware Server (Python)"]
Transport[HTTP Transport<br/>JSON-RPC 2.0]
Auth[Bearer Token Auth]
Router[Tool Router]
end
subgraph ESP32["ESP32-S3 Devices"]
API1[Device 1 HTTP API<br/>:8080]
API2[Device 2 HTTP API<br/>:8080]
ESPNOW[ESP-NOW Mesh]
end
Client -->|"JSON-RPC 2.0"| Transport
Registry -.->|"discovery"| Transport
Transport --> Auth --> Router
Router -->|"HTTP REST"| API1
Router -->|"HTTP REST"| API2
API1 <-->|"ESP-NOW"| API2
API1 <-->|"ESP-NOW"| ESPNOW
3) Transport
3.1 HTTP Transport (Primary)
The MCP server runs as a Python HTTP service registered in mascarade's MCP registry.
| Parameter | Value |
|---|---|
| Protocol | JSON-RPC 2.0 over HTTP POST |
| Endpoint | POST /mcp |
| Port | 8790 (configurable via MCP_HARDWARE_PORT) |
| Content-Type | application/json |
| Auth | Bearer token (same as MASCARADE_API_KEY) |
3.2 stdio Transport (Local Development)
For local testing, the server also supports stdio transport per MCP spec:
python -m zacus_mcp_server --transport stdio
3.3 Registration in mascarade
# mascarade MCP registry entry
{
"name": "zacus-hardware",
"description": "ESP32-S3 escape room hardware control",
"transport": "http",
"url": "http://localhost:8790/mcp",
"auth": {"type": "bearer", "token_env": "MASCARADE_API_KEY"},
"enabled_env": "ZACUS_MCP_ENABLED"
}
4) Authentication Model
4.1 MCP Server Auth (mascarade -> MCP Server)
- Bearer token in
Authorizationheader - Token matches
MASCARADE_API_KEYenvironment variable - Requests without valid token receive
401 Unauthorized
4.2 MCP Server -> ESP32 Auth
- Bearer token in
Authorizationheader on ESP32 HTTP API - Token stored in ESP32 NVS (provisioned at setup)
- Per-device token support for multi-device deployments
4.3 Token Hierarchy
mascarade API key
└── MCP server validates incoming requests
└── Per-device ESP32 tokens
└── ESP32 validates hardware commands
5) Tool Definitions
5.1 puzzle_set_state
Control puzzle lock/unlock state and trigger associated effects.
{
"name": "puzzle_set_state",
"description": "Set the state of a puzzle element (lock, unlock, reset). Triggers associated LED and audio effects.",
"inputSchema": {
"type": "object",
"properties": {
"device_id": {
"type": "string",
"description": "Target ESP32 device identifier"
},
"puzzle_id": {
"type": "string",
"description": "Puzzle identifier from scenario runtime",
"enum": ["PUZZLE_FIOLE", "PUZZLE_COFFRE", "PUZZLE_MIROIR", "PUZZLE_ENGRENAGE", "PUZZLE_CRYSTAL", "PUZZLE_BOUSSOLE"]
},
"state": {
"type": "string",
"enum": ["locked", "unlocked", "reset"],
"description": "Target state"
},
"effects": {
"type": "boolean",
"default": true,
"description": "Play associated LED/audio effects on state change"
}
},
"required": ["device_id", "puzzle_id", "state"]
}
}
ESP32 API mapping: POST /api/puzzle with body {"id": "...", "state": "...", "effects": true}
5.2 audio_play
Play audio files or streams on device speakers.
{
"name": "audio_play",
"description": "Play an audio file or stream on the ESP32 speaker. Supports local files (LittleFS) and HTTP URLs.",
"inputSchema": {
"type": "object",
"properties": {
"device_id": {
"type": "string",
"description": "Target ESP32 device identifier"
},
"source": {
"type": "string",
"description": "Audio source: LittleFS path (/audio/hint_01.mp3) or HTTP URL"
},
"volume": {
"type": "integer",
"minimum": 0,
"maximum": 100,
"default": 70,
"description": "Playback volume (0-100)"
},
"loop": {
"type": "boolean",
"default": false,
"description": "Loop playback continuously"
},
"action": {
"type": "string",
"enum": ["play", "stop", "pause", "resume"],
"default": "play"
}
},
"required": ["device_id", "source"]
}
}
ESP32 API mapping: POST /api/audio with body {"src": "...", "vol": 70, "loop": false, "action": "play"}
5.3 led_set
Control LED strips and individual LEDs.
{
"name": "led_set",
"description": "Control LED strips: set color, pattern, brightness. Supports WS2812B addressable LEDs.",
"inputSchema": {
"type": "object",
"properties": {
"device_id": {
"type": "string",
"description": "Target ESP32 device identifier"
},
"zone": {
"type": "string",
"description": "LED zone identifier",
"enum": ["ambient", "puzzle", "alert", "all"]
},
"color": {
"type": "string",
"description": "Hex color (#RRGGBB) or named color",
"pattern": "^(#[0-9a-fA-F]{6}|red|green|blue|white|off|warm|cold|purple|orange)$"
},
"pattern": {
"type": "string",
"enum": ["solid", "breathe", "chase", "rainbow", "pulse", "off"],
"default": "solid"
},
"brightness": {
"type": "integer",
"minimum": 0,
"maximum": 255,
"default": 128
},
"duration_ms": {
"type": "integer",
"description": "Auto-off after duration (0 = indefinite)",
"default": 0
}
},
"required": ["device_id", "zone", "color"]
}
}
ESP32 API mapping: POST /api/led with body {"zone": "...", "color": "...", "pattern": "solid", "bright": 128}
5.4 camera_capture
Capture a snapshot from the OV2640 camera.
{
"name": "camera_capture",
"description": "Capture a JPEG snapshot from the ESP32 camera. Returns base64-encoded image.",
"inputSchema": {
"type": "object",
"properties": {
"device_id": {
"type": "string",
"description": "Target ESP32 device identifier"
},
"resolution": {
"type": "string",
"enum": ["QQVGA", "QVGA", "VGA"],
"default": "QVGA",
"description": "Capture resolution (160x120, 320x240, 640x480)"
},
"quality": {
"type": "integer",
"minimum": 10,
"maximum": 63,
"default": 20,
"description": "JPEG quality (lower = better, 10-63)"
}
},
"required": ["device_id"]
}
}
ESP32 API mapping: GET /api/camera?res=QVGA&q=20 returns image/jpeg
5.5 scenario_advance
Trigger a Runtime 3 transition on the device.
{
"name": "scenario_advance",
"description": "Trigger a Runtime 3 scenario transition. Used by game masters to manually advance or reset the game.",
"inputSchema": {
"type": "object",
"properties": {
"device_id": {
"type": "string",
"description": "Target ESP32 device identifier"
},
"event_type": {
"type": "string",
"enum": ["button", "serial", "timer", "audio_done", "unlock", "espnow", "action", "manual"],
"description": "Event type per Runtime 3 transition model"
},
"event_name": {
"type": "string",
"description": "Event name token (e.g., UNLOCK_COFFRE, MANUAL_ADVANCE)"
},
"target_step_id": {
"type": "string",
"description": "Optional: force transition to specific step (game master override)"
}
},
"required": ["device_id", "event_type", "event_name"]
}
}
ESP32 API mapping: POST /api/scenario/transition with body {"event_type": "...", "event_name": "...", "target": "..."}
5.6 device_status
Query device health and current state.
{
"name": "device_status",
"description": "Get current device status: free memory, current scenario step, uptime, WiFi RSSI, sensor readings.",
"inputSchema": {
"type": "object",
"properties": {
"device_id": {
"type": "string",
"description": "Target ESP32 device identifier"
}
},
"required": ["device_id"]
}
}
ESP32 API mapping: GET /api/status returns JSON status object
6) Message Format (JSON-RPC 2.0)
6.1 Request
{
"jsonrpc": "2.0",
"id": "req-001",
"method": "tools/call",
"params": {
"name": "led_set",
"arguments": {
"device_id": "zacus-main",
"zone": "puzzle",
"color": "#00FF00",
"pattern": "pulse",
"brightness": 200
}
}
}
6.2 Success Response
{
"jsonrpc": "2.0",
"id": "req-001",
"result": {
"content": [
{
"type": "text",
"text": "LED zone 'puzzle' set to #00FF00 pulse at brightness 200 on device zacus-main"
}
]
}
}
6.3 Error Response
{
"jsonrpc": "2.0",
"id": "req-001",
"error": {
"code": -32000,
"message": "Device unreachable",
"data": {
"device_id": "zacus-main",
"detail": "HTTP timeout after 5000ms"
}
}
}
6.4 Error Codes
| Code | Meaning |
|---|---|
| -32700 | Parse error (malformed JSON) |
| -32600 | Invalid request |
| -32601 | Method not found |
| -32602 | Invalid params |
| -32603 | Internal error |
| -32000 | Device unreachable |
| -32001 | Device busy (command in progress) |
| -32002 | Auth failed (ESP32 token) |
| -32003 | Puzzle state conflict |
7) Sequence Diagrams
7.1 LLM-Driven Puzzle Unlock
sequenceDiagram
participant GM as Game Master
participant M as mascarade LLM
participant MCP as MCP Hardware Server
participant E as ESP32-S3
GM->>M: "Deverrouille le coffre dans la salle 2"
M->>M: Parse intent -> puzzle_set_state
M->>MCP: JSON-RPC tools/call puzzle_set_state
MCP->>MCP: Validate params + auth
MCP->>E: POST /api/puzzle {"id":"PUZZLE_COFFRE","state":"unlocked"}
E->>E: Servo unlock + LED green + SFX
E-->>MCP: 200 OK {"state":"unlocked","effects_played":true}
MCP-->>M: JSON-RPC result
M-->>GM: "Le coffre de la salle 2 est maintenant deverrouille."
7.2 Voice Command -> Hardware Action
sequenceDiagram
participant P as Player
participant E as ESP32-S3
participant MCP as MCP Hardware Server
participant M as mascarade LLM
participant T as TTS
P->>E: "Hey Zacus, allume la lumiere UV"
E->>E: ESP-SR: wake + command parse
E->>MCP: JSON-RPC tools/call led_set (zone:puzzle, color:purple)
MCP->>E: POST /api/led {"zone":"puzzle","color":"#7B00FF"}
E->>E: UV LED on
E-->>MCP: 200 OK
MCP-->>E: JSON-RPC result
Note over E: Optional: confirm via TTS
E->>M: Hint request: "confirme action UV"
M->>T: TTS "La lumiere UV est activee"
T-->>E: Audio stream
E->>P: Speaker: "La lumiere UV est activee"
8) Device Discovery
8.1 Static Configuration (Phase 1)
Devices are configured in .env:
ZACUS_DEVICES='[{"id":"zacus-main","host":"192.168.0.50","port":8080,"token":"..."}]'
8.2 mDNS Discovery (Phase 2)
ESP32 devices advertise _zacus._tcp via mDNS. The MCP server discovers devices automatically:
_zacus._tcp.local.
zacus-main._zacus._tcp.local. 8080 TXT "room=main" "version=3.1"
zacus-salle2._zacus._tcp.local. 8080 TXT "room=salle2" "version=3.1"
8.3 ESP-NOW Mesh (Phase 3)
The primary ESP32 acts as a gateway for ESP-NOW mesh devices. The MCP server sends commands to the gateway, which relays via ESP-NOW to secondary devices.
9) Rate Limiting & Safety
| Constraint | Value |
|---|---|
| Max requests per device | 10/s |
| Max concurrent tool calls | 3 |
| Command timeout | 5 s |
| Retry on timeout | 1 retry with 2 s backoff |
| Servo actuation cooldown | 500 ms between movements |
| LED transition min interval | 100 ms |
Safety guards:
- No two conflicting puzzle state changes within 1 s
- Audio volume hard-capped at device level (not bypassable via MCP)
- Camera capture rate limited to 2/s to prevent overheating
- Game master override always available via
scenario_advancewithevent_type: "manual"
10) Implementation Plan
Phase 1: Core Server (2 weeks)
- Python MCP server with HTTP transport
- Tool definitions:
puzzle_set_state,audio_play,led_set,device_status - Bearer auth, static device config
- Register in mascarade MCP registry
- Unit tests (pytest)
Phase 2: Camera + Scenario (2 weeks)
camera_capturetool with base64 responsescenario_advancetool with Runtime 3 integration- mDNS device discovery
- Integration tests with ESP32 hardware
Phase 3: Mesh + Dashboard (3 weeks)
- ESP-NOW mesh relay through gateway device
- Real-time game master dashboard (WebSocket feed)
- Multi-device orchestration (synchronized effects)
- Load testing and latency benchmarks