ARKit-only-body mode no longer consumes face: the Mac dropped face
entirely (face=0, /pose/mouth gone), so VNDetectFaceLandmarksRequest
and the .face USB frames were pure wasted per-frame compute. Drop the
face request and its send.
The on-device ARView (RealityKit) rendered a black background behind
the SwiftUI skeleton overlay -- it never showed the camera -- so it
only spun up the RealityKit AR render pipeline (the engine:rematerial
console spam) to draw black. Remove the ARView; the screen stays
black + skeleton overlay, identical UX, minus the GPU/thermal cost.
Video (HEVC) and hands are kept: the Mac uses the iPhone video as the
visualizer backdrop and Vision hands drive pinch/fist gestures.
Add camera: ARCamera param to publishUSB and thread frame.camera
through from the session delegate. After the existing .skeleton
send, project all 91 joints to viewport 2D, normalize to [0,1],
and send as Skeleton2DPayload / FrameTag.skeleton2D (tag 6).
Under --iphone-usb, skip all three MediaPipe detectors
(pose, face, hand) and rebuild the MP-33 body from
state.persons_arkit_{2d,joints} via new arkit_body_2d /
arkit_body_3d builders. Add head anchor (ARKIT_HEAD_IDX=51,
needs live verify) as MP33 slot 0. Webcam path unchanged.
Add TAG_SKELETON2D=6, SKEL2D_BYTES=819, decode_skeleton2D() to
iphone_usb_bridge.py; handler in IphoneUSBSource writing
persons_arkit_2d / persons_arkit_2d_t to state; new state fields
in State dataclass. TDD: test_arkit_body.py (round-trip + wrong
length → None). Import check verified.
Add FrameTag.skeleton2D (raw 6) and Skeleton2DPayload for
2D-projected ARKit joints: 91×SIMD2<Float> + 91 valid bytes,
big-endian wire format mirroring SkeletonPayload style.
Note: raw value is 6 (not 5 as drafted) because face=5 is taken.
Add matFistThresh state (default 0.15) with localStorage persistence.
Wire a "Sensibilite poing" slider in Reglages, synced via fisthresh.
Add two fist indicator rows in #morceau-fists driven by fistprogress.
Context: Task 1 wired fistThresh into the SC matrix engine (default
0.15); the generator was not yet emitting this field, so every
.matrix was missing it and the engine fell back to its hardcoded
default.
Approach: TDD — added a failing check to test_global_actions.py,
then edited to_sc so pinchHoldMs ends with a comma and fistThresh
0.15 follows as the last field before the closing paren.
Regenerated 84 presets twice for determinism.
Changes:
- generate_presets.py: pinchHoldMs now ends with comma; fistThresh
0.15 emitted as the final key in every preset's SC Event literal
- test_global_actions.py: EXPECT_FTH block checks to_sc output and
all 84 on-disk .matrix files
- *.matrix (84 files): regenerated with fistThresh field
Impact: all four gates green (GLOBALACT-PY, test_sections,
test_families, GLOBALACT SC); fistThresh is now a first-class
preset field readable at load time by ~matLoadFile.
Held closed-fist (low openness, hysteresis+presence guard) per hand
fires an action after ~matPinchHoldMs: left hand reloads the current
preset, right hand advances to the next. Progress emitted on
/matrix/fistprogress.
- ~matFistEdge edge detector (engage/release/absent + hysteresis)
- ~matFistTick fires once past hold threshold (guarded by ~matFistFired)
- ~matFistReloadCurrent and ~matNextPreset wired to hand 0/1
- /matrix/fisthresh[/get] OSCdefs + ~matFistThreshPush
- fistThresh saved and restored per .matrix file
- 30 Hz poller now calls ~matFistTick alongside ~matPinchTick
- Harness group Q: A-Q all green
Add top-level 'pinchHoldMs': 300 to every generated .matrix so the SC
engine (Task 2) reads it instead of its hardcoded default.
- generate_presets.py: globalSettings ends with ',' and
'pinchHoldMs': 300 is the last field before the closing ')'
- test_global_actions.py: EXPECT_PHMS check added (TDD red then green)
- 84 .matrix files regenerated (28 morceaux x intro/live/outro)
PinchDetector now emits state=1 on engage and state=0 on release
(and on hand-disappear). send_pinch appends the state as the 4th
OSC arg so SC can distinguish hold from lift. Launcher switches
from FINGER_PIANO to PINCH_ENABLE to gate only the pinch detector.
Expand globalActions from 4 to 8 slots:
[\fill, \drop, \breakdown, \evolve,
\muteDrums, \muteMelo, \washReverb, \next]
Drop openThresh/closeThresh from globalSettings (now
engine-side constants via Task 1). Regenerate all 84
.matrix files deterministically.
All gates: GLOBALACT-PY PASS, TEST PASS x2, GLOBALACT PASS
Replace open/close hand detection with 8 pinch slots (hand*4+finger-1).
Global actions array grows from 4 to 8 entries (L/R x index..ring).
Remove ~matHandFlip, ~matHandState, openThresh/closeThresh entirely.
Add the Reglages group to the morceau panel: two threshold sliders
(open/close, the close one being the user-requested "sensibilite"),
a transient-duration slider (1-8 bars), and a breakdown kept-voices
checkbox grid covering all MATRIX_VOICES.
Wired over /matrix/globalsettings with inbound sync and onOpen
get-request. Imports consolidated into a single coherent block;
no duplicate identifiers. DOM container added to index.html;
styles appended to control.css.
Context: Phase 2 Task 1 made the SC engine load a globalSettings Event
from .matrix files (openThresh/closeThresh/transientBars/breakdownKeep)
and default it when absent. Generator was not yet writing the field.
Approach: append globalSettings after globalActions in to_sc() return
string; extend test_global_actions.py with EXPECT_SET check against
both the to_sc sample and all 84 .matrix files on disk.
Changes:
- generate_presets.py: to_sc now ends with globalActions (comma) then
globalSettings Event with default values matching SC defaults
- test_global_actions.py: add EXPECT_SET literal + two check() calls
- *.matrix (84 files): each preset now carries the globalSettings field
Impact: every preset ships with openThresh/closeThresh/transientBars/
breakdownKeep defaults; SC engine reads them on load; no more implicit
fallback needed at runtime for any preset in the library.
When the evolve toggle is OFF, ~matRelanceEvolve now snapshots
~matBaseGrid, mutates once, then restores it to nil. When ON it
keeps the persistent base and re-mutates from it. Fixes stale-
snapshot corruption across preset loads. Case I added to harness.
Add bottom-of-matrix panel with 4 gesture-slot dropdowns wired
over OSC (/matrix/globalaction). State lives in matrix-state.js;
syncs on connect and on incoming OSC echo.
Context: SC engine (Task 2) loads 'globalActions' from .matrix files
and defaults to [\fill, \drop, \evolve, \breakdown] when absent.
Generated presets did not write this field, causing implicit reliance
on the engine default.
Approach: extend to_sc() return string to include the globalActions
key so every preset explicitly carries the default mapping
(LH-open=fill, LH-closed=drop, RH-open=evolve, RH-closed=breakdown).
Changes:
- generate_presets.py: add 'globalActions' line in to_sc() return
- test_global_actions.py: new Python test verifying to_sc() output
and all 84 on-disk .matrix files carry the field
- 84 .matrix files: regenerated with globalActions field present
Impact: presets are self-documenting; SC engine reads the explicit
value rather than applying a silent default.
clement