Add gesture_quality() pure helper (hand_display.py) that maps a slot's
plausibility, facing, and proximity to a score in [0,1]. Renderer uses
it to modulate panel frame brightness (0.25+0.75*q) and stroke count
(1..3 passes at q≥0, ≥0.5, ≥0.85 / status==3). Status still drives
hue (pid 7/8/9). New gesture_slot_quality field in State written
alongside gesture_slot_status each tick.
Add gauge_segments() for X/Y position gauges around each panel: a
horizontal rail below and a vertical rail on the outer side, each with
a bold two-tick notch at the hand's normalised cx/cy (mirror-aware for
X). Dim rails (conf 0.25) when slot absent. Renderer reads hand_feats
L/R directly from state.
11 new tests (6 gesture_quality + 5 gauge_segments); suite 430 passed.
Add gesture_slot_status [0..3] to State (absent/detected/armed/engaged).
PinchDetector.engaged_slots() exposes per-slot pinch state. Publisher
writes status after each tick via _update_gesture_slot_status().
Renderer draws panel frame always: dim (pid7 conf=0.3) when absent,
normal (pid7) detected, pid8 armed, pid9 double-stroke engaged.
HandPersistenceGate (min_frames=3, radius=0.15) suppresses ghost
hands that flash for fewer than min_frames consecutive frames.
Gate is called once per frame in the renderer on persons_hands;
result filters both the video overlay loop and the side panels.
hand_plausible now rejects wrists outside [-0.1, 1.1] (out-of-frame
ghost anchor check). panel_segments gains a conf_min param.
Env knobs read at renderer init (single parse, no per-frame cost):
HAND_CONF_MIN (default 0.45, raised from 0.3)
HAND_PERSIST_FRAMES (default 3)
ARKIT_BONE_MAX (default 0.5, passed to arkit_segments)
New tests: 12 gate tests + 4 wrist-bounds tests.
_decode_hands now returns (hands, chirality) tuple instead of a bare
list. Landmark .z keeps the raw wire value; .c = clamp(z, 0, 1).
Chirality byte (1=right, else 0=left) is decoded per hand and stored
in State.persons_hands_chirality under the same lock as
persons_hands_iphone.
Add TAG_SKELETON2D=6, SKEL2D_BYTES=819, decode_skeleton2D() to
iphone_usb_bridge.py; handler in IphoneUSBSource writing
persons_arkit_2d / persons_arkit_2d_t to state; new state fields
in State dataclass. TDD: test_arkit_body.py (round-trip + wrong
length → None). Import check verified.
iPhone Vision hands (on-device, .right upright, y already
top-left, rotation-invariant) are stored in a dedicated state
field and used by the strike detector when fresh (FINGER_SOURCE
auto/iphone/mediapipe). More stable than MediaPipe on the
rotated downscaled video. Adds FINGER_DEBUG strike logging.
Three improvements wired end-to-end:
1. Openpos 3D skeleton visible: Skeleton3DRenderer attached to a
RealityKit AnchorEntity in BodyView, toggled by showSkeleton
or vizMode==9. PoseOSCListener now parses /pose3d/count and
/pose3d/kp (plus restored /face/* and /hand/* paths).
2. DINO re-id (dinov2_vits14, ~9 ms ANE forward):
MeshRigger combines Hungarian IoU with cosine similarity over
a per-pid embedding history (deque maxlen=10), weighted by
MULTIHMR_REID_ALPHA (default 0.5). Falls back to pure IoU if
DINO mlpackage absent or scipy missing. state.last_frame_rgb
buffer added so the rigger can crop bbox regions for embedding.
3. PoseFilterChain on pose_world_landmarks:
median (anti-spike) -> Kalman constant-velocity ->
50 ms lookahead -> IK elbow/knee/ankle clamp. Configurable
via POSE_FILTER env (default median+kalman+lookahead+ik).
<2 ms per frame for typical 1-2 persons.
Tests: 5 new in test_dino_reid.py + 6 new in test_pose_filter.py,
all green. Live validated by user: skeleton spawns, mesh stays
stable.
Stream MediaPipe Holistic face landmarks (68 dlib subset of 478),
hand landmarks (21 left + 21 right), and pose world landmarks (33
3D xyz meters) over OSC :57126 to AVLiveBody. Launcher renders
face/hand as SwiftUI Canvas overlay and the 3D skeleton as a
RealityKit armature (sphere joints + cylinder bones, color per
chain) toggled via p / mode openpos.
Multi-HMR worker now also starts MediaPipe Multi in parallel so
both the dense SMPL-X mesh (TCP 57130, PyTorch backend) and the
skeleton/face/hand streams (OSC 57126) feed the launcher from
one Python process.
Launcher AppDelegate forces .regular activation so SwiftPM
binaries actually show their WindowGroup without a bundle.
Tests: 9 new pytest cases (4 body3d + 5 face/hand), all green.
CoreML conversion still produces NaN on v3d/transl; PyTorch
backend is the working path for now.
Ajoute un 10e viz_mode 'openpos' dans data_only_viz pour
afficher le squelette multi-personne sur un fond minimal qui
met en valeur les corps detectes.
state.py :
- viz_mode_names complete avec 'openpos' a l'index 9
- KEYMAP_VIDEO : touche 'p' -> openpos (slot precedemment libre)
shaders/scene.metal :
- nouveau mode_openpos : fond radial sombre avec une grille
de points discrete + pulsation legere sur ~rms. Le bg sert
juste a faire ressortir le skel_pipeline rendu PAR-DESSUS
par le renderer (qui beneficie deja du depth_fog + palette
par pid + epaisseur confiance-adaptee).
- bg_fragment dispatcher : mode == 9 -> mode_openpos
web/public/control.html :
- 10e bouton 'openpos' dans la grille #vizmodes.
Adds RenderSettings ObservableObject (cam opacity, bg brightness,
mesh metallic/roughness/visible, 3 light intensities, FOV, panel
visibility) and a SettingsPanel SwiftUI overlay with sliders and
toggles for each.
ContentView wraps BodyView in a ZStack with a corner button + an
on-receive keyboard handler ('S') that toggles the panel. BodyView
applies every setting in updateNSView : preview opacity, container
bg, ARView FOV, the three DirectionalLight intensities, and
MeshRenderer.applyMaterialSettings rebuilds SimpleMaterial with
the new metallic/roughness on each call.
BodyView also tightens the layer chain that was visually ambiguous :
arView.layer.isOpaque = false (clear background actually honored by
the compositor) and preview.zPosition = -100 so the cam stays
strictly behind the AR layer.
Adds the SMPL-X person record (vertices, joints, translation, betas,
expression) and the corresponding list + last-timestamp fields on the
shared State. Mirrors the NLFPerson shape so renderers can switch
backends without touching the upstream pipeline.
New standalone visualizer: OSC listener, pose bridge,
Apple Vision / CoreML / YOLO pose backends, Euro filter,
fine analysis, mesh topology, holistic renderer.
Metal shader (scene.metal) for GPU-accelerated drawing.
L'électron rare