The BOX-3 touch gamebook had several blocking bugs that made the
voiced+foley story packs unusable. This fixes them end to end.
- Stack overflow on story start: gb_play_task kept a 2 KB buffer on
its 4 KB stack and rebooted the board. Buffers are now static (the
task is a singleton) and the stack is 6 KB.
- No sound at all: the ES8311 codec was never initialised — main.c
wrote raw I2S to a muted DAC. Output (and the mic) now go through
the BSP esp_codec_dev (bsp_audio_codec_speaker/microphone_init,
esp_codec_dev_write/read). The power amplifier (GPIO46) is also
forced on and volume set to 100, as the es8311 pa_pin handling did
not drive it in practice.
- First passage silent: gb_play re-mounted the SD that the gamebook
had already mounted; the failed second bsp_sdcard_mount tore down
the VFS so the first fopen failed. Now we open the file first and
only mount if that fails. A short silence lead-in also primes the
codec/PA ramp.
- No accents: the gamebook used ASCII-only lv_font_montserrat_14/24.
Added custom Montserrat fonts (font_fr_14/24, Latin-1 + œŒŸ « » ’ …,
uncompressed so they render without LV_USE_FONT_COMPRESSED).
- Mic/voice streaming gated behind CONFIG_BOX3_VOICE_STREAMING
(default off): as a gamebook the BOX-3 needs no mic, and the
constant codec reads + bridge reconnects interfered with playback
and flooded the log.
plip_virtual + cmd_exec handle types switched from i2s_chan_handle_t
to esp_codec_dev_handle_t accordingly.
Escape-room mechanic: the retro phone can call the players on its own
and tell them a story when they pick up.
Approach: a story_ring_task rings at a random 15-30 min interval (only
when the line is idle and a story pack is on the SD). Picking up routes
through enter_story_from_ring(), which silences the bell and launches a
random story via plip_gamebook_begin_random() straight into the gamebook
state — no menu. If nobody answers within 1 min the bell stops and the
next ring is rescheduled; hanging up mid-ring also silences it.
The whole feature is gated behind CONFIG_PLIP_AUTO_RING (default n), so
the phone stays silent unless explicitly enabled in menuconfig. The hook
pickup paths (raw SLIC poll + debounced edge) mirror the existing NPC
incoming-call handling, which is unreliable while the bell rings.
Context:
The BOX-3 touch gamebook showed text + tappable choices but was silent.
We want narration through its ES8311 speaker while the text is on
screen — and tapping a choice must cut it instantly.
Approach:
Add a dedicated async, interruptible WAV player to cmd_exec (which owns
the speaker handle), streaming straight from the SD in small chunks so
any length plays without a big malloc. The gamebook plays each passage
WAV on render; a new tap (or returning to the menu) stops it.
Changes:
- cmd_exec.c/.h: cmd_exec_play_file_async(path) + cmd_exec_stop_play().
One persistent task streams a 16 kHz mono 16-bit WAV from the SD in
2 KB chunks (RAM-safe), and a new request / stop flag breaks the
stream mid-clip so playback is interruptible.
- gamebook.c: on entering a passage, play /sdcard/gamebook/<wav> if the
pack has audio (non-fatal when absent → stays text-only); stop the
narration when returning to the library.
Impact:
The BOX-3 reads the story aloud (16 kHz, matches AUDIO_SAMPLE_RATE)
while showing the text, and a tap cuts to the next passage. Audio is
optional — text-only packs still work.
Context:
The Freenove gamebook is screen+buttons, the PLIP is audio+dial. The
ESP32-S3-BOX-3 has a 320x240 touchscreen, so its natural format is a
tap-to-choose "livre dont tu es le heros": read the passage on screen,
tap a choice button to navigate.
Approach:
A new gamebook module reuses the same SD pack as the master (the JSON
passages already carry screen/text/choices[label,goto]) and builds the
UI on the already-running LVGL stack: a scrollable column with the
title, the wrapped passage text, and one tappable button per choice.
Changes:
- New gamebook.c/.h: mounts the SD (bsp_sdcard_mount), reads
/sdcard/gamebook/{library.json, <id>.json}, shows a story list, then
per passage renders title + text + choice buttons; tapping a choice
goes to its target, an ending offers "back to menu". cJSON runs on
PSRAM hooks (the expanded books are ~80 KB JSON); LVGL calls are
wrapped in bsp_display_lock at init (button callbacks already run in
the LVGL task).
- main.c: call gamebook_init() after the scenario server is up; it
takes over the screen when a pack is on the SD, else no-op (the phone
UI stays).
- CMakeLists: build gamebook.c.
Impact:
With a JSON pack on its SD, the BOX-3 is a touch gamebook: tap a story,
tap your choices. Text-only (no audio needed). Hardware-validated:
boots into a 6-story touch library.
Neural-TTS (Kyutai) gamebook narration WAVs run ~1.5 MB per passage at
16 kHz, over the old 512 KB cap. Streamed in 2 KB chunks so the larger
cap stays RAM-safe.
Context:
Two issues on the Freenove gamebook: (1) long passages were clipped —
the reading band was small and text overflowed; (2) the 5-way pad was
decoded wrong, so navigation felt random, and on this unit the physical
Up button is electrically dead (held = no voltage change).
Approach:
Give the reading text the whole screen height in a vertically
scrollable container, split selection from scrolling, and remap the
keys to the pad's MEASURED voltages.
Changes:
- display_ui: gamebook view reworked — compact title, a full-height
scrollable reading container (the body wraps and scrolls, never
clipped), and a one-line choice selector pinned at the bottom; new
display_ui_gamebook_scroll() + reset-on-new-passage; show() gains a
reset_scroll arg; body buffer 512 -> 1100.
- gamebook.c: controls matched to the measured ladder
(key2=LEFT, key4=DOWN, key5=RIGHT, key1=CLICK; key3 unused; Up dead):
Left/Right scroll the text up/down, Down cycles the answer, Click
validates; library nav cycles tiles without needing Up.
Impact:
Long passages are fully readable (they scroll), and all four working
pad buttons do the right thing; nothing depends on the dead Up button.
Hardware-calibrated against the real pad voltages.
Context:
The Freenove plays a screen+buttons "livre dont vous etes le heros".
The PLIP retro phone has no screen, so we want a phone-native version:
an audio choose-your-own-adventure where the narrator reads the passage
and its numbered choices, and the player dials a digit to choose.
Approach:
A self-contained plip_gamebook module reads a phone audio pack from the
SD (narration WAVs already include the spoken "faites le N" prompts),
plays passages through the earpiece, and maps each dialed digit to a
choice. It hooks into the existing conversation state machine and the
existing dialer (rotary/DTMF) — fully offline, no model, no network.
Changes:
- New plip_gamebook.c/.h: loads /sdcard/gamebook/{library.json,
menu.wav, <id>.json, <id>_<passage>.wav}; pickup plays the story
menu, digit picks a story, then each passage plays its WAV and a
dialed digit jumps to that choice; an ending + any digit returns to
the menu. Interrupts narration so a dial is instant.
- conversation.c: new STATE_GAMEBOOK; off-hook from idle launches the
gamebook when a pack is on the SD (else the normal dialtone/NPC flow,
so the phone still works without a pack); each dialed digit is fed to
the gamebook; go_idle() tears it down on hang-up.
- CMakeLists: build plip_gamebook.c.
Impact:
With a phone pack staged on its SD, the PLIP becomes an audio gamebook:
pick up, dial to choose your story and your path, hang up to stop.
Non-breaking when no pack is present. Builds clean.
Context:
Two issues surfaced with the longer kid stories: (1) long passages were
clipped on screen — the body label wrapped in a fixed 196px band and
the tail was simply cut off; (2) old orphan WAVs (demo + archived
adventures) piled up on the SD with no way to remove them, since the
master had POST /game/file but no delete.
Approach:
Make the gamebook body label scroll vertically so the whole passage
goes by, and add a DELETE verb to /game/file mirroring the POST routing
so the host can clean stale assets off the SD.
Changes:
- display_ui: gamebook body label LV_LABEL_LONG_WRAP ->
LV_LABEL_LONG_SCROLL_CIRCULAR; body buffer 512 -> 1100 so a full
~1000-char passage is held and scrolled, not truncated.
- game_endpoint: new DELETE /game/file?path=sd/<…>|apps/<…> (same
whitelist/traversal guard as POST; unlink + 200/404).
- ota_server: max_uri_handlers 22 -> 24 for the extra verb.
Impact:
Long passages are fully readable on the LCD (they scroll), and the SD
gamebook folder can be cleaned remotely. Hardware-validated: 121 orphan
files deleted, library still opens 6 stories.
Context:
The gamebook played a single hard-coded book. We want the master to
boot into a library of several stories, shown as tiles, and let the
player pick one with the pad.
Approach:
Turn the gamebook component into a two-mode machine (LIBRARY / STORY).
On boot it loads /sdcard/gamebook/library.json and shows the stories as
a tile grid; selecting one loads that book's <id>.json and plays it;
finishing a story returns to the library.
Changes:
- display_ui: new library view — a 2x3 grid of title tiles with the
selected one highlighted (display_ui_library_show/hide); it wins the
view selector while open.
- gamebook.c: LIBRARY/STORY modes; load library.json + per-book
<id>.json from the SD; grid navigation (up/down +/-2, left/right
+/-1, click opens); story ending click returns to the library.
- main.c: gamebook_init only installs the pad hook; gamebook_start()
(which opens the library) is now called after the SD + media_manager
are up, so the boot picker actually finds the SD pack.
Impact:
The Freenove boots straight into a story picker and runs any book from
the SD library, fully offline. Hardware-validated: boots into a
6-story library.
Context:
We wanted a "livre dont vous etes le heros" playable on the Freenove
master with NO model in RAM: narration as say()-rendered WAV staged on
the SD card, and navigation on the physical 5-way pad. Nothing in the
firmware drove a branching, button-navigated story before this.
Approach:
A self-contained gamebook component reads a JSON book from the SD,
drives the display and audio, and owns the 5-way pad through a
registered display_ui key hook (so display_ui keeps no dependency on
gamebook — no cycle). A dedicated full-screen gamebook view renders
each page (title / wrapped body / choice list).
Changes:
- New gamebook component: loads /sdcard/gamebook/gamebook.json; per
passage plays its WAV (media_manager, absolute /sdcard path) and
renders the title, wrapped text, and a choice list with a ">" cursor.
- Navigation: D-pad up/down (and left/right) move the cursor, the
center click confirms; endings (no choices) restart on click; a
choice interrupts the current narration (media_manager_stop + play).
- display_ui: add display_ui_set_key_hook (consume keys when a takeover
mode is active) and a dedicated gamebook view (title + height-bounded
body band + reserved bottom menu band) with display_ui_gamebook_show
/ display_ui_gamebook_hide; the gamebook view wins the view selector
while open.
- game_endpoint: POST /game/gamebook[?action=stop] to start/stop.
- ota_server: bump max_uri_handlers 20 -> 22 for the new endpoint.
- main: call gamebook_init() once the buttons are up.
Impact:
The Freenove plays a branching audio gamebook driven entirely by the
pad, fully offline (no gateway, no model). Hardware-validated end to
end: cursor navigation, click select, WAV narration from SD, endings
restart.
Removes a staged file with the same sd/ vs SPIFFS path routing as the
POST handler (mounts SD on demand). Lets the host clean stale clips
when regenerating the SD voice pack. max_uri_handlers 17->18.
Validated: probe WAVs removed from /sdcard/voice (200, then 404).
/debug/ring now takes an optional scene; conversation_arm_incoming
locks it and, after the greeting, queues that scene's local clip
/sdcard/voice/hint_<scene>_l1_0.wav (played back-to-back). For a
scripted offline call CONNECTED skips the gateway/model listen loop
and just waits for hang-up. Scene is cleared in go_idle so a later
dial-out call never inherits it. Validated: greeting 5.05s + WARNING
hint 6.20s from SD on pickup, no gateway/model.
On incoming pickup the GREET state now plays the local clip
/sdcard/voice/greet_<num>.wav when present — the phone greets the
caller with no gateway and no model in RAM. Falls back to the live
gateway greeting (turn_client) only when the pack lacks a clip for
that number.
/game/file only wrote SPIFFS (960 KB, too small for a voice pack). A
path prefixed sd/ (or /sdcard/) now routes to the microSD: the card is
mounted on demand (audio_ensure_sd, independent of hook state) and the
parent dir is created. MAX_BODY 256->512 KB so full say() hints fit
(streamed in 2 KB chunks, RAM-safe). Enable FATFS LFN_HEAP so >8.3
filenames work (greet_0142738200.wav etc). Validated: 40-sample FR
voice pack staged to /sdcard/voice on the bench PLIP (960 MB card).
notify_gateway_scene() + hints puzzle_start lived only in
npc_engine_set_step(), which had NO callers — the runtime->gateway
sync was dead code, so scripted incoming calls would never fire in a
real game. POST /game/step goes through game_endpoint_apply_step(),
which now resolves the step's scene_id (new field on scene_binding,
parsed straight from the IR) and calls the new
npc_engine_set_scene_by_id(). Hardware-validated: STEP_WARNING ->
scene 3 -> gateway notify -> auto-ring Professeur Zacus on the PLIP.
The scene->gateway POST ran synchronously (2 s timeout) on the scene
transition, so an offline gateway lagged EVERY scene change by 2 s.
Fire it on a detached one-shot task instead, so the game's scene
transitions never wait on the network. Frees the heap scene copy and
self-deletes. Builds clean.
On each scene change (where it already signals /hints/puzzle_start), the
npc_engine now also POSTs the active SCENE_* to the voice gateway
/game/step?scene=… (best-effort, esp_http_client, Bearer token). The
phone NPCs then disguise THIS scene's hint automatically during a game.
Gateway URL/token hardcoded in main.c like the hints URL (-> NVS later).
Recalibrate the capture VAD to the quiet handset voice: onset 1.4%->0.7%
FS (was never triggering -> 'no sustained voice'), silence 0.6%->0.34%
FS and end-of-speech window 600->900 ms so a whole sentence is held
instead of cut mid-phrase. Also force slic_ring_stop() on incoming
pickup so the physical bell always stops when answered (phone.c/slic.c
s_ringing could desync, leaving the bell ringing through the call).
The marginal A1S cradle contact flickers open mid-call; treating a
brief open as a hangup dropped the live conversation. Raise the
prolonged-open hangup threshold to 2.5 s and make the resync
asymmetric: a PICKUP is confirmed fast (600 ms, calls answer promptly)
while a HANGUP needs the line to stay open 2.5 s. Brief flickers no
longer end the call; a real hangup still fires ~2.5 s later.
Listen-loop capture now only commits when the caller actually speaks:
phase A waits for a sustained voice onset (3 frames above threshold,
rejecting the PA-mute click), phase B records until silence. Empty
captures are no longer posted, so the NPC never replies to silence.
Also: DC-blocking high-pass on the captured mono, VAD thresholds tuned
to the quiet SLIC handset mic (onset ~1.4%, silence ~0.6%), keep the
PA on during capture (muting it collapsed the mic), and add a
/debug/miccap diagnostic endpoint (raw fixed-duration mic capture).
Incoming-call greeting was silently dropped: audio CMD_PLAY gated on
phone.c's debounced phone_is_offhook(), which lags/misses the pickup
while the bell rings, but the incoming flow established off-hook only
via the raw SLIC poll in conversation.c. The two flags desynced and
playback was skipped as 'on-hook' though the handset was up.
Gate playback on the real SHK line (slic_is_offhook) instead — the
single source of truth. Remove all hook forcing (s_hook_override,
phone_force_offhook, /debug/offhook): the SHK contact is reliable in
hardware, so the firmware must trust it, never override it. Builds
clean (ESP-IDF v5.4.4).
Add es8388_write_reg() public API and GET /debug/es8388?reg=&val=
HTTP handler to read/write codec registers at runtime without
reflashing. Bumps max_uri_handlers 16→17.
STA config: channel=0 (all-channel scan), WIFI_ALL_CHANNEL_SCAN,
WIFI_CONNECT_AP_BY_SIGNAL, failure_retry_cnt=5.
Disconnect handler: retry on every disconnect event, including during
initial association — previously the first failure at boot caused an
immediate abort and a ~30s timeout before IP was acquired.
Validated on hardware: connects reliably on ch1, RSSI -32, IP in ~2.5s.
IDF5 i2s_channel_init_std_mode() constitutes full-duplex ONLY when TX and
RX std_cfg are byte-for-byte identical (memcmp). When din/dout differ
between the two calls, the driver silently moves RX to I2S_NUM_1 which has
no BCLK/WS routing, producing permanent zeros on i2s_channel_read().
Fix: use the same i2s_std_config_t for both TX and RX init calls, with
dout=GPIO26 and din=GPIO35 both set. IDF handles GPIO direction internally.
Also clean up ES8388 register sequence:
- ADCCONTROL2 = 0x00 (LIN1/RIN1 differential, LINE IN header)
- ADCCONTROL6 = 0x00 (clear ADCSMUTE, reset default was 0x30)
- ADCPOWER = 0x00 (full ADC power-up, was 0x09)
- DACCONTROL21 = 0x80 (DAC+ADC normal mode, not line bypass 0xC0)
Verified: peak=2593, rms=2482 over 48000 samples (3s @ 16kHz).
ADCCONTROL6 (reg 0x0E) reset default = 0x30 which has ADCSMUTE=1 (bit5) — ADC output
muted by default. Writing 0x00 unmutes. Without this, ADCINSEL=0x50 (LIN2) is selected
but the signal is suppressed at the ADC output stage → peak=0.
ADCPOWER (reg 0x03) changed from 0x09 (intermediate Espressif open state) to 0x00 (full
power-up: all power-down bits cleared). Value 0x09 = bits 0+3 set (ADCPD_L + MICBIAS_PD)
— MICBIAS_PD in particular means the internal microphone bias is powered off, which can
starve the SLIC line-in path. 0x00 is the correct end state for recording mode per
Espressif esp8388_start(ES_MODULE_ADC) reference.
sdkconfig.defaults: add CONFIG_PLIP_HOOK_GPIO=23 / CONFIG_PLIP_HOOK_ACTIVE_HIGH=y
as explicit defaults so clean builds use the SLIC SHK pin without menuconfig.
ADCCONTROL2 (reg 0x0A): was 0x00 (LINSEL=00 LINPUT1/RINPUT1 — onboard PCB mic),
now 0x50 (LINSEL=01 LINPUT2 / RINSEL=01 RINPUT2 — telephone handset mic via SLIC).
The SLIC K50835F routes the handset mic signal to the ES8388 LINPUT2/RINPUT2 pins.
Writing 0x00 meant the ADC was listening to the empty onboard mic, producing 0/0 RMS.
Writing 0x50 connects the SLIC audio path, enabling voice capture from the handset.
PGA stays at +24 dB (ADCCONTROL1 = 0xBB). ADC power-up sequence unchanged.
- board_config.h: add PLIP_SLIC_RM=18, FR=5, SHK=23, PD=19 (A1S KEY3-6 repurposed)
- slic.c/slic.h: new ESP-IDF module porting Ks0835SlicController:
* slic_init(): RM/FR output LOW, SHK input+pullup, PD open-drain HIGH (power-up)
* slic_is_offhook(): reads SHK GPIO23, HIGH = off-hook (active-high, matches A252ConfigStore default)
* slic_ring_start/stop(): RM HIGH + FreeRTOS task toggles FR at 25 Hz (20 ms period)
- CMakeLists.txt: add slic.c to SRCS, esp_driver_gpio to PRIV_REQUIRES
- Kconfig: PLIP_HOOK_GPIO default 4→23, add PLIP_HOOK_ACTIVE_HIGH (default y)
- phone.c: hook reads SHK GPIO23 via HOOK_OFFHOOK_LEVEL/HOOK_PULSE_OPEN macros (active-HIGH);
phone_ring_start/stop() now drives slic_ring_start/stop() for physical bell + audio tone
- main.c: slic_init() called early in boot_task before audio_init
Root cause fixed: SLIC was never powered (PD never released from reset state).
Hook was read on wrong GPIO (4) with wrong polarity. Ring drove only audio, not bell.
clement
electron
clemsail