Commit Graph

843 Commits

Author SHA1 Message Date
L'électron rare 3ffd81afc7 feat(pose): validate hands before analysis
Apply hand_plausible + HandPersistenceGate in _step_stab before
route_hands so gestures and /pose/hands features only see quality-
gated hands. Cache per-tick validated set (_tick_hands/_tick_chir/
_tick_mirror) so all three emitters share one acquisition. Gate
stepped at 30 Hz publisher cadence (audit R6). Non-protocol hands
(raw numpy arrays) pass through via try/except for backward compat.
2026-07-02 14:35:03 +02:00
L'électron rare 65f13b4621 test(pose): gesture input validation gates
Tests (a-f) for plausibility gate, persistence establishment,
chirality alignment, feature path, MediaPipe pass-through.
Update helpers with .x/.y/.c _Kp objects; add HAND_PERSIST_FRAMES=1
to legacy _pub()/_pub_envs() so existing tests need no warmup ticks.
2026-07-02 14:34:53 +02:00
L'électron rare b85a9c93a9 fix(viz): no body mesh when arkit is the source
CI build oscope-of / build-check (push) Has been cancelled
The body-mesh triangles (solid cyan blob reported live) were the one
surface no filter covered: under iphone-usb persons_body is the ARKit
conversion with c=1.0, so garbage/stale joints painted full triangles.
Skip BODY_TRIANGLES whenever the ARKit topology is present (webcam
MediaPipe path unchanged); the body stays the freshness-gated ARKit
wireframe.
2026-07-02 14:26:03 +02:00
L'électron rare 651fd86d37 docs(viz): fix config docstrings + env tables
config.py: PINCH_EXT_MIN docs said "min frames for debounce"
— it is min extended fingers required to fire; fix wording.
HAND_PERSIST_FRAMES said "persist after disappearance" — it
is frames required before a hand is drawn; fix wording.

CLAUDE.md: MEDIAPIPE_DELEGATE default was "cpu" — real
default is "gpu" (config.py); fix the RC0.1+ table row.
Add missing PINCH_ENABLE / VIZ_HUD / CONCERT_MIRROR rows
to the main env-vars table.
2026-07-02 14:24:01 +02:00
L'électron rare d337ed9b29 fix(pose): clear iphone hands on stream drop
iphone_usb_source.py: after stream ends, clear
persons_hands_iphone, persons_hands_chirality, and
persons_hands (when write_hands=True) under state.lock()
to prevent renderer display freeze.

test_iphone_usb_reconnect.py: add
test_hands_cleared_on_stream_drop to verify lists are
empty after simulated USB disconnect.
2026-07-02 14:23:53 +02:00
L'électron rare 0126fc97a8 fix(viz): wire arkit-full + osc port via config
renderer.py: read ARKIT_FULL_SKELETON via VizConfig at
initWithState_ instead of module-level os.environ; store as
self._arkit_full, use getattr fallback for __new__ test bypass.

main.py: pass cfg.iphone_osc_port to IphoneOSCListener
instead of hardcoded 57128; log actual port.

hand_slots.py: import _finite/_coord/_clamp from hand_features
at module level (acyclic: hand_features imports hand_slots only
lazily inside step()); delete the duplicate copies.
2026-07-02 14:23:47 +02:00
L'électron rare 4ea0bb2c2a feat(pose): fist kill-switch, default off
CI build oscope-of / build-check (push) Has been cancelled
User request live 2026-07-02: disable the fist gesture completely
(false held-fist actions during play). fist is forced to 0.0 at the
extractor unless FIST_ENABLE=1; continuous features unchanged. SC
held-fist actions (reload/next) go silent without touching SC.
2026-07-02 14:22:31 +02:00
L'électron rare ba4a1bb103 perf(pose): 30hz gesture loop under iphone-usb
CI build oscope-of / build-check (push) Has been cancelled
iphone-usb path: camera decode + ARKit conversion is cheap (no
MediaPipe inference on Mac). Frame-based gesture calibrations
(debounce/hold/grace/persist) assume 30 Hz. Keep 18 Hz for the
webcam MediaPipe path (M5 CPU protection).
Tests: period formula + MultiWorker.__init__ signature check.
2026-07-02 14:13:52 +02:00
L'électron rare 05d0fc1df1 fix(pose): mirror-coherent gesture fallback
Read state.mirror_2d in _pick_hands and thread it through
_step_stab (route_hands mirror=) and _emit_hands
(HandFeatureExtractor.step mirror=) so the cx fallback slot
assignment matches the renderer panel layout on chirality dropout.

Add mirror kwarg to HandFeatureExtractor.step plumbed to route_hands.
Tests: extractor-level mirror flip + publisher _emit_hands mirror_2d.
2026-07-02 14:13:36 +02:00
L'électron rare caa41fd7f1 test(viz): pin stabilizer defaults, fix seams
CI build oscope-of / build-check (push) Has been cancelled
Detection v3 review follow-ups: pin hand_persist_grace/near_off/
hold_frames defaults; use the module-level route_hands (restores the
monkeypatch seam, drops the dead local alias); document the grace
count-inheritance window in the gate docstring.
2026-07-02 12:00:28 +02:00
L'électron rare e10361c56d docs(pose): fix stale swap note, pin mesh rig
CI build oscope-of / build-check (push) Has been cancelled
hand_slots module docstring still claimed inverted chirality (default
1) after the live A/B validated chirality correct (default 0); align
it. Pin the MESH_RIG ne0 bool convention (chantier 5 review minor).
2026-07-02 11:52:20 +02:00
L'électron rare a7a1563ef8 docs: detection v3 env vars 2026-07-02 11:49:32 +02:00
L'électron rare 5769af3d26 feat(pose): fist near + established gates 2026-07-02 11:48:58 +02:00
L'électron rare f3ca0ac100 feat(pose): gesture slot stabilizer 2026-07-02 11:48:01 +02:00
L'électron rare 4e33a03e1b feat(viz): persistence grace frames 2026-07-02 11:46:01 +02:00
L'électron rare 8b08282aa5 fix(viz): remote log vars + avbody host param
Chantier 5 review: NameError on MULTIHMR_BACKEND=remote (LOG.info
referenced removed locals) and SMPLXTCPSender host param silently
ignored after the VizConfig migration (env-present check restores the
pre-migration contract).
2026-07-02 11:40:55 +02:00
L'électron rare 91b3923431 docs(viz): chantier-5 report + HAND_SWAP_LR doc
CI build oscope-of / build-check (push) Has been cancelled
2026-07-02 11:33:42 +02:00
L'électron rare e2a41c0d8a test(viz): VizConfig and iphone-usb gating tests
61 tests: all VizConfig defaults, four bool conventions,
numeric cast fallback, monkeypatch round-trip, and Part B
gating (_ensure_model never called under --iphone-usb).
Fix hand_swap_lr from_env default: was True, now False.
2026-07-02 11:32:24 +02:00
L'électron rare 65bf3aad08 refactor(viz): remaining consumers read VizConfig
pose_filter _parse_env_* read POSE_FILTER* via VizConfig.
multi_hmr_worker reads MULTIHMR_BACKEND/AUTOCAST/REMOTE via VizConfig.
multihmr_remote reads JPEG/ASYNC/HOST/PORT via VizConfig.
smplx_osc_sender reads AVBODY_HOST/REID/ALPHA via VizConfig.
pose_bridge reads AVBODY_HOST/VDMX_* via VizConfig.
iphone_usb_source reads CONCERT_MIRROR via VizConfig.
lidar_calib reads ICP_LIDAR_EXTRINSIC via VizConfig.
multihmr_coreml reads COREML_COMPUTE_UNITS via VizConfig.
2026-07-02 11:26:57 +02:00
L'électron rare c0cde337c9 refactor(viz): consumers read VizConfig
multi.py reads 5 discrimination thresholds via VizConfig.
main.py reads ICP/Multi-HMR/AV_LIVE_* flags via VizConfig.
2026-07-02 11:24:19 +02:00
L'électron rare e7ce016688 feat(viz): central VizConfig from env
Add data_only_viz/config.py: frozen dataclass VizConfig with typed
fields for all 56 env reads in data_only_viz. from_env() reads
os.environ at call time — no module-level singleton, test-compat.

Migrate action_head_pub._init_finger_piano and renderer.initWithState_
to consume VizConfig. Keep renderer.py ARKIT_FULL module-level (test
uses MetalRenderer.__new__ without calling initWithState_).

HAND_SWAP_LR default changed False (was True): Vision chirality
validated correct live 2026-07-02; swap knob now defaults off.
2026-07-02 11:21:55 +02:00
L'électron rare 5d425b1fbd fix(pose): plumb swap into hand features slots
CI build oscope-of / build-check (push) Has been cancelled
Review C1 (Critical): /pose/hands per-voice L/R mods routed without
the HAND_SWAP_LR swap while pinch/strike used it — the two paths
contradicted each other on an inverted-chirality source. step() now
takes swap, the publisher passes its env-read value, two tests pin
the contract. Also hoists the route_hands imports (M5) and fixes the
stale slot comment (M1).
2026-07-02 11:09:06 +02:00
L'électron rare 32cdfa0d3e fix(pose): swap chirality only, default on
CI build oscope-of / build-check (push) Has been cancelled
User-confirmed live: ARBodyTracker Vision chirality arrives inverted
vs the mirrored view. HAND_SWAP_LR now inverts the chirality mapping
inside route_hands (cx fallback untouched, it is already screen
relative) and defaults to 1. Also: HandPersistenceGate greedy-match
limitation documented + malformed-entry guard (review follow-ups).
2026-07-02 11:00:25 +02:00
L'électron rare e20a4d130a docs: hand router env vars 2026-07-02 10:55:38 +02:00
L'électron rare 366d916e9d refactor(pose): detectors consume routed slots
Convert all four consumers to accept pre-routed [L|None, R|None]:
- finger_strike: remove order_hands_by_cx + _slot_hands; step()
  iterates enumerate(slotted), None slot resets state / synthesizes
  release edges in-place.
- hand_features: step(hands, chirality=None) delegates to route_hands;
  speed buffer indexed by slot, not cx order.
- action_head_pub: _pick_hands returns chirality; _emit_fingers/
  _emit_pinch call route_hands with HAND_SWAP_LR/HAND_NEAR_MIN;
  _emit_hands plumbs chirality to HandFeatureExtractor.
- renderer: panels block replaced with route_hands call; persistence
  gate + hand_plausible mask applied before routing so a ghost hand
  cannot steal a slot; HAND_SWAP_LR read at init.
Test updates: [hand, None] slot lists in test_finger_strike;
_hand() fixture in test_action_head_finger_emit gains realistic size
so near_min=0.10 gate passes.
2026-07-02 10:55:32 +02:00
L'électron rare 9ee7990dcd feat(pose): hand router chirality + near gate
New module hand_slots.route_hands():
- Chirality path: iPhone ARBodyTracker Vision 0/1 -> L/R slots,
  alignment preserved before invalid-hand filtering.
- Fallback: ascending cx sort (mirror-aware).
- swap=True: swaps the two slots at the very end (safety knob,
  HAND_SWAP_LR=1, if a source inverts Vision chirality).
- near_min gate: drops slots below wrist->middle-MCP distance.
18 new tests covering all branches.
2026-07-02 10:55:21 +02:00
L'électron rare 8bc7acbd14 fix(pose): fresh arkit joint arrays per frame
Extract apply_skeleton_joints() as a pure module-level function that
copies prev_arr (or zeros) and writes a new array, never mutating the
old one.  The TAG_SKELETON handler now stores the new array under the
state lock, so readers that captured a reference to the previous array
(multi.py shallow-copy + ArkitFuse.apply outside the lock) always see
a complete, consistent snapshot — eliminating the B4 tearing race.

6 new tests cover: shape, identity (is not), value preservation for
invalid joints, value application for valid joints, wrong-shape reset,
and no-mutation of prev_arr.
2026-07-02 10:41:41 +02:00
L'électron rare 0a4f51ebfb test(viz): pin arkit max-len behavior change
Review follow-up: make the v2 filter's rejection of legacy 0.707
diagonal fixtures explicit instead of silently retuned, and document
the close-up caveat on max_len (ARKIT_BONE_MAX raises it live).
2026-07-02 10:37:02 +02:00
L'électron rare 039abaa4a8 docs: hand filter env vars
CI build oscope-of / build-check (push) Has been cancelled
Add HAND_CONF_MIN, HAND_PERSIST_FRAMES, ARKIT_BONE_MAX to the
Environment variables table in the root CLAUDE.md.
2026-07-02 10:28:23 +02:00
L'électron rare 5bfc63f86d feat(viz): hand persistence gate + conf knobs
HandPersistenceGate (min_frames=3, radius=0.15) suppresses ghost
hands that flash for fewer than min_frames consecutive frames.
Gate is called once per frame in the renderer on persons_hands;
result filters both the video overlay loop and the side panels.

hand_plausible now rejects wrists outside [-0.1, 1.1] (out-of-frame
ghost anchor check).  panel_segments gains a conf_min param.

Env knobs read at renderer init (single parse, no per-frame cost):
  HAND_CONF_MIN (default 0.45, raised from 0.3)
  HAND_PERSIST_FRAMES (default 3)
  ARKIT_BONE_MAX (default 0.5, passed to arkit_segments)

New tests: 12 gate tests + 4 wrist-bounds tests.
2026-07-02 10:27:58 +02:00
L'électron rare ac6768d8a8 fix(viz): arkit bone bounds + length filter
arkit_segments now accepts max_len and bounds params.
Bones whose endpoints fall outside [-bounds, 1+bounds] or whose
length exceeds max_len are skipped (tracking-loss garbage).
Defaults: max_len=0.5, bounds=0.1.  Renderer will pass
max_len=ARKIT_BONE_MAX explicitly (Part 3).

Tests updated to use realistic bone lengths; new tests added
for the out-of-bounds and overlong-bone cases.
2026-07-02 10:27:49 +02:00
L'électron rare 287c4977a9 fix(viz): install key monitors at app launch
NSEvent keyboard monitors were placed at the END of
_start_pose_worker, unreachable: every backend path returns
early (lines 369/384/399/413/440) so monitors were never
installed. ESC, space (concertNext) and keymaps were silently
dead in all real concert paths.

Move the local + global monitor installation into
applicationDidFinishLaunching_, after window focus setup,
non-headless path only. The --multi-hmr headless early-return
(line 95) is unaffected.
2026-07-02 10:15:19 +02:00
L'électron rare b8ae1a668c fix(pose): read real smplx person dataclass
_read_sources used .get("pid")/.get("v3d") (dict API) but
multi_hmr_worker emits SMPLXPerson dataclasses (field: vertices_3d).
AttributeError was swallowed by LOG.exception at 30 Hz.

- Dual-read via isinstance check: dataclass path uses p.pid /
  p.vertices_3d / p.expression; dict path keeps p.get() unchanged
- 3 new tests: real SMPLXPerson emits OSC, expression field accepted,
  legacy dict format still works
2026-07-02 10:14:17 +02:00
L'électron rare 48b04084fa fix(viz): gate arkit overlay on 2d freshness
persons_arkit_2d_t was written by IphoneUSBSource but never
read. A USB drop left the frozen skeleton on screen indefinitely.

- Add arkit_2d_fresh(ts_by_pid, now, max_age=1.0) pure helper
  in hand_display.py (8 TDD tests, all green)
- renderer._update_skeleton: use_arkit now requires at least one
  pid younger than 1 s (replaces bool check)
- Per-pid freshness gate in draw loop skips stale pids
2026-07-02 10:13:05 +02:00
L'électron rare 06b868e377 docs(viz): refactor audit ref (chantiers 1-6)
CI build oscope-of / build-check (push) Has been cancelled
2026-07-02 10:07:00 +02:00
L'électron rare 0a2865e136 feat(viz): left/right hand front-view panels
CI build oscope-of / build-check (push) Has been cancelled
Add panel_rect/panel_segments/panel_frame helpers to hand_display.py
(PANEL_SIDE=0.30, PANEL_MARGIN=0.02, PANEL_INNER=0.10). Segments use
uniform pixel-space scale so the hand aspect ratio is preserved
regardless of view aspect. mirror=True flips X within the panel.

In renderer._update_skeleton add push_panel (no mirror flip) and a
panels block before the segs==0 epilogue: routes persons_hands to
left/right panel via chirality list when aligned, screen-cx fallback
otherwise. Draws frame (pid 7) + wireframe (pid 5 left / 6 right).
Runs on every path including use_arkit. Guards: _mp_bones None, empty
hands, misaligned chirality, height==0.

26 tests pass (10 new for panel helpers).
2026-07-02 09:56:08 +02:00
L'électron rare 3a3e6b113f feat(viz): hud debug panel off by default
The AV-Live data-only text HUD hid the lower-left corner of the
performance display and would collide with the new left-hand panel.
VIZ_HUD=1 re-enables it; the H key toggle guards the absent view.
2026-07-02 09:47:54 +02:00
L'électron rare ef719e76f3 feat(viz): filter hand overlay artifacts
Add hand_display.py with hand_plausible / segment_ok / hand_size pure
helpers; wire into both skeleton hands loops (plausibility gate +
per-segment confidence+spike filter + real min(A.c,B.c) push); remove
filled mesh triangles for hands from _update_mesh (HAND_TRIANGLES block
deleted).
2026-07-02 09:42:51 +02:00
L'électron rare 60640a881e feat(pose): decode hand confidence + chirality
_decode_hands now returns (hands, chirality) tuple instead of a bare
list. Landmark .z keeps the raw wire value; .c = clamp(z, 0, 1).
Chirality byte (1=right, else 0=left) is decoded per hand and stored
in State.persons_hands_chirality under the same lock as
persons_hands_iphone.
2026-07-02 09:37:24 +02:00
L'électron rare 29c66c7151 docs(viz): hand display cleanup spec + plan 2026-07-02 09:33:34 +02:00
L'électron rare 1f19a5db96 feat(pose): pinch gate env vars + docs
CI build oscope-of / build-check (push) Has been cancelled
2026-07-02 09:18:14 +02:00
L'électron rare ea42ba4bb3 feat(pose): pinch engage debounce 2026-07-02 09:14:18 +02:00
L'électron rare 021c53f078 feat(pose): pinch open-hand extension gate 2026-07-02 09:10:06 +02:00
L'électron rare 50bc6f5d71 docs(pose): pinch gate implementation plan 2026-07-02 09:05:26 +02:00
L'électron rare 1510a37059 docs(pose): pinch open-hand gate + debounce spec 2026-07-02 09:00:56 +02:00
clement 095fd51812 Merge branch 'feat/disable-matrix-pinch'
CI build oscope-of / build-check (push) Has been cancelled
2026-06-30 22:51:39 +02:00
clement 1491aef30e feat(matrix): disable finger pinch by default 2026-06-30 22:49:41 +02:00
clement 24ce960992 fix(iphone): mirror skeleton overlay to video 2026-06-30 22:40:31 +02:00
clement 08b7c72e49 feat(iphone): render full arkit body skeleton 2026-06-30 22:14:35 +02:00
clement 86c071d0c9 feat(iphone): arkit skeleton segment builder 2026-06-30 22:09:10 +02:00