docs: add AGENTS.md skeleton #1

Merged
electron-rare merged 85 commits from docs/agents-md-tier-1 into main 2026-05-21 10:41:34 +00:00
electron-rare commented 2026-05-21 10:39:39 +00:00 (Migrated from github.com)

Adds an AGENTS.md skeleton for AI coding agents (Claude Code, Aider, Cursor, etc.), per the 2026-05-21 doc audit (Tier-1 sweep). Repo-specific tech stack, conventions, file layout, and domain gotchas.

Generated with Claude Code.

Adds an AGENTS.md skeleton for AI coding agents (Claude Code, Aider, Cursor, etc.), per the 2026-05-21 doc audit (Tier-1 sweep). Repo-specific tech stack, conventions, file layout, and domain gotchas. Generated with Claude Code.
copilot-pull-request-reviewer[bot] (Migrated from github.com) reviewed 2026-05-21 10:45:14 +00:00
copilot-pull-request-reviewer[bot] (Migrated from github.com) left a comment

Pull request overview

This PR introduces a new set of cross-project components for the AV-Live body pipeline (shared wire format + USB transport, new macOS/iOS clients, and Python-side ARKit/LiDAR fusion), and adds an AGENTS.md guidance document for AI coding agents.

Changes:

  • Add shared/AVLiveWire Swift package (frame header, stream demuxer, wire payloads) with unit tests.
  • Add/extend body-tracking clients: iOS iphone-arbody (USB server + HEVC encoder) and macOS avlivebody-mac (usbmuxd client, RealityKit scene, optional Multi-HMR CoreML).
  • Add Python ARKit joint ingestion, ICP LiDAR fusion plumbing, and a new data_feeds/ OSC publisher package; update docs and repo ignores.

Reviewed changes

Copilot reviewed 130 out of 146 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
shared/AVLiveWire/Tests/AVLiveWireTests/WirePayloadsTests.swift Tests for skeleton/video payload encoding/decoding
shared/AVLiveWire/Tests/AVLiveWireTests/StreamDemuxerTests.swift Tests for stream demuxer chunking/resync behaviors
shared/AVLiveWire/Tests/AVLiveWireTests/SmokeTests.swift Smoke test for protocol version
shared/AVLiveWire/Tests/AVLiveWireTests/LoopbackTests.swift End-to-end demux loopback test
shared/AVLiveWire/Tests/AVLiveWireTests/FrameHeaderTests.swift FrameHeader encode/decode validation tests
shared/AVLiveWire/Sources/AVLiveWire/WirePayloads.swift Define SkeletonPayload/VideoPayload wire formats
shared/AVLiveWire/Sources/AVLiveWire/StreamDemuxer.swift Incremental frame demuxer for TCP byte streams
shared/AVLiveWire/Sources/AVLiveWire/FrameHeader.swift Binary frame header + BE helpers
shared/AVLiveWire/Sources/AVLiveWire/AVLiveWire.swift Protocol version namespace
shared/AVLiveWire/Package.swift SwiftPM package definition for AVLiveWire
launcher/Sources/AVLiveLauncher/ProcessManager.swift Disable legacy AV-Live-Body spawn path (archived)
launcher/CLAUDE.md Note about AV-Live-Body archival and new location
launcher/_archive-AV-Live-Body/Tests/AVLiveBodyTests/USBSkeletonConsumerTests.swift Tests for archived USB skeleton mapping
launcher/_archive-AV-Live-Body/Tests/AVLiveBodyTests/USBMuxProtocolTests.swift Tests for archived usbmux protocol codec
launcher/_archive-AV-Live-Body/Tests/AVLiveBodyTests/USBClientTests.swift Tests for archived usbmux client behaviors
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/USBSkeletonConsumer.swift Archived USB consumer (StreamDemuxer + dispatch)
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/USBMuxProtocol.swift Archived usbmux protocol codec
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/USBClient.swift Archived usbmux client + Unix socket transport
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/SkeletonOverlay.swift Archived 2D skeleton overlay rendering
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/SettingsPanel.swift Archived settings panel UI
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/SceneRenderer.swift Archived Metal background renderer
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/RenderSettings.swift Archived render settings expansion
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/PoseOSCListener.swift Archived listener extended with face/hand/pose3d
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/OSCServer.swift Archived TCP OSC server implementation
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/MeshRenderer.swift Archived mesh renderer tracks pelvis position
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/HUDOverlay.swift Archived HUD overlay view
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/FaceHandOverlay.swift Archived SwiftUI Canvas face/hand overlay
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/DataHUDOverlay.swift Archived data feeds HUD UI
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/DataFeedsOSCListener.swift Archived OSC listener for open-data feeds
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/CameraPreviewLayer.swift Archived macOS webcam preview plumbing
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/BodyView.swift Archived RealityKit view wiring (adds 3D skeleton)
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/AVLiveBodyApp.swift Archived app wires listeners + HUD
launcher/_archive-AV-Live-Body/Sources/AVLiveBody/ArkitOSCListener.swift Archived ARKit OSC listener for diagnostics
launcher/_archive-AV-Live-Body/Package.swift Archived SwiftPM package now depends on AVLiveWire + tests
launcher/_archive-AV-Live-Body/ARCHIVED.md Marker doc for archived project
iphone-arbody/README.md iOS app README + build/signing notes
iphone-arbody/project.yml XcodeGen project for ARBodyTracker
iphone-arbody/Config/Shared.xcconfig Shared signing/build settings (includes optional Local)
iphone-arbody/Config/Local.xcconfig.example Template for developer-local team ID
iphone-arbody/Config/Local.xcconfig Developer-local team ID (committed; should be removed)
iphone-arbody/CLAUDE.md Agent guidance for iPhone project + AVLiveWire commands
iphone-arbody/ARBodyTracker.swiftpm/Sources/ARBodyTracker/VideoEncoder.swift HEVC encoder -> VideoPayload
iphone-arbody/ARBodyTracker.swiftpm/Sources/ARBodyTracker/USBServer.swift TCP server sending framed AVLiveWire messages
iphone-arbody/ARBodyTracker.swiftpm/Sources/ARBodyTracker/Info.plist iOS Info.plist for camera/network usage
iphone-arbody/ARBodyTracker.swiftpm/Sources/ARBodyTracker/ARBodyTrackerApp.swift SwiftUI app entrypoint
iphone-arbody/ARBodyTracker.swiftpm/Package.swift SwiftPM executable package for ARBodyTracker
iphone-arbody/ARBodyTracker.swiftpm/.swiftpm/xcode/package.xcworkspace/contents.xcworkspacedata Generated SwiftPM workspace artifact (should not be committed)
docs/ICP_FUSION.md Documentation for LiDAR↔SMPL-X ICP fusion
data_only_viz/tests/test_state_arkit.py Tests for new ARKit state fields
data_only_viz/tests/test_pose_filter.py Tests for pose filter chain behavior/perf
data_only_viz/tests/test_open3d_smoke.py Open3D dependency smoke tests
data_only_viz/tests/test_multihmr_coreml.py Update perf expectation for Multi-HMR CoreML
data_only_viz/tests/test_multihmr_arkit_z.py Tests for ARKit pelvis Z override logic
data_only_viz/tests/test_multi_hmr_worker.py Add predict_once behavior test (naming mismatch)
data_only_viz/tests/test_lidar_receiver.py LiDAR TCP decoder/reader unit tests
data_only_viz/tests/test_lidar_calib.py Extrinsic persistence + Kabsch tests
data_only_viz/tests/test_iphone_osc_listener.py Tests for iPhone OSC listener -> State
data_only_viz/tests/test_icp_fusion.py ICP fusion unit tests (Open3D-gated)
data_only_viz/tests/test_dino_reid.py DINOv2 re-id tests (mlpackage-gated)
data_only_viz/tests/test_arkit_joint_map.py Mapping integrity tests
data_only_viz/tests/test_arkit_fuse.py Tests for ArkitFuse stage
data_only_viz/state.py Extend State with ARKit, LiDAR, joints, last_frame_rgb fields
data_only_viz/smplx_osc_sender.py Wire MeshRigger with optional DINO re-id config
data_only_viz/scripts/quantize_multihmr_int8.py INT8 weight quantization helper script
data_only_viz/scripts/coreml_full_probe.py Update conversion precision note (FP32)
data_only_viz/scripts/calibrate_lidar.py CLI scaffold for LiDAR extrinsic calibration
data_only_viz/scripts/bench_icp_fusion.py ICP fusion benchmark script
data_only_viz/pyproject.toml Add optional lidar extra with Open3D constraint
data_only_viz/multihmr_coreml.py Update CoreML output names + normalization + compute unit selection
data_only_viz/main.py Start iPhone OSC listener; opt-in ICP fusion thread; stop ICP on exit
data_only_viz/lidar_receiver.py LiDAR TCP frame decode + threaded reader
data_only_viz/lidar_calib.py Extrinsic model + JSON I/O + Kabsch solver
data_only_viz/iphone_osc_listener.py Python OSC listener for ARKit joints
data_only_viz/icp_fusion_worker.py Threaded ICP fusion wrapper (currently without state lock)
data_only_viz/CLAUDE.md Document ARKit fusion plumbing and mapping
data_only_viz/arkit_joint_map.py ARKit 91 -> MediaPipe 33 mapping constants
data_feeds/osc_sender.py OSC sender wrapper
data_feeds/main.py Feed runner CLI
data_feeds/feeds/velib.py Velib GBFS specialization
data_feeds/feeds/teleray.py Stub feed
data_feeds/feeds/sytadin.py Stub feed
data_feeds/feeds/prim.py Stub feed
data_feeds/feeds/hubeau.py Hub’Eau feed implementation
data_feeds/feeds/gbfs.py Generic GBFS reader
data_feeds/feeds/eco2mix.py Eco2Mix feed implementation
data_feeds/feeds/carburants.py Stub feed
data_feeds/feeds/base.py Feed base class + scheduler loop
data_feeds/feeds/ais.py Stub feed
data_feeds/feeds/__init__.py Feed registry
data_feeds/config.avlivedata.toml Default feed config file
data_feeds/__init__.py Package marker
CLAUDE.md Add network topology + env var table
avlivebody-mac/Tests/AVLiveBodyTests/USBMuxProtocolTests.swift Tests for usbmux protocol codec
avlivebody-mac/Tests/AVLiveBodyTests/USBClientTests.swift Tests for usbmux client
avlivebody-mac/Tests/AVLiveBodyTests/BodyFusionTests.swift Tests for mesh/skeleton pelvis depth fusion
avlivebody-mac/Tests/AVLiveBodyTests/.gitkeep Keep test dir tracked
avlivebody-mac/Sources/AVLiveBody/VideoQuad.swift RealityKit video background plane
avlivebody-mac/Sources/AVLiveBody/usb/USBSkeletonConsumer.swift macOS USB consumer with video decode callback
avlivebody-mac/Sources/AVLiveBody/usb/USBMuxProtocol.swift usbmux protocol codec
avlivebody-mac/Sources/AVLiveBody/usb/USBClient.swift usbmux client + Unix transport
avlivebody-mac/Sources/AVLiveBody/usb/MultiHMRCoreML.swift CoreML wrapper for Multi-HMR model
avlivebody-mac/Sources/AVLiveBody/usb/BodyFusion.swift Fuse skeleton pelvis depth into Multi-HMR translation
avlivebody-mac/Sources/AVLiveBody/StatusBar.swift USB connection status overlay
avlivebody-mac/Sources/AVLiveBody/SkeletonEntity.swift RealityKit skeleton marker rendering
avlivebody-mac/Sources/AVLiveBody/SceneView.swift SwiftUI bridge to ARView
avlivebody-mac/Sources/AVLiveBody/SceneController.swift Scene setup + update wiring + orbit controls
avlivebody-mac/Sources/AVLiveBody/MeshEntity.swift RealityKit SMPL-X mesh rendering
avlivebody-mac/Sources/AVLiveBody/Info.plist macOS app Info.plist
avlivebody-mac/Sources/AVLiveBody/AxisConvention.swift Axis conversion helper
avlivebody-mac/Sources/AVLiveBody/AVLiveBodyApp.swift App entrypoint wiring USB -> scene + optional Multi-HMR
avlivebody-mac/README.md Build/run docs for new macOS app
avlivebody-mac/project.yml XcodeGen project for macOS app
avlivebody-mac/Config/Shared.xcconfig Shared macOS build settings
avlivebody-mac/Config/Local.xcconfig.example Local signing template
avlivebody-mac/.gitignore Ignore generated projects/build outputs
AGENTS.md AI agent guidance document
.gitignore Ignore iPhone/macos SwiftPM build artifacts + local configs
Files not reviewed (1)
  • iphone-arbody/ARBodyTracker.swiftpm/.swiftpm/xcode/package.xcworkspace/contents.xcworkspacedata: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

## Pull request overview This PR introduces a new set of cross-project components for the AV-Live body pipeline (shared wire format + USB transport, new macOS/iOS clients, and Python-side ARKit/LiDAR fusion), and adds an `AGENTS.md` guidance document for AI coding agents. **Changes:** - Add `shared/AVLiveWire` Swift package (frame header, stream demuxer, wire payloads) with unit tests. - Add/extend body-tracking clients: iOS `iphone-arbody` (USB server + HEVC encoder) and macOS `avlivebody-mac` (usbmuxd client, RealityKit scene, optional Multi-HMR CoreML). - Add Python ARKit joint ingestion, ICP LiDAR fusion plumbing, and a new `data_feeds/` OSC publisher package; update docs and repo ignores. ### Reviewed changes Copilot reviewed 130 out of 146 changed files in this pull request and generated 10 comments. <details> <summary>Show a summary per file</summary> | File | Description | | ---- | ----------- | | shared/AVLiveWire/Tests/AVLiveWireTests/WirePayloadsTests.swift | Tests for skeleton/video payload encoding/decoding | | shared/AVLiveWire/Tests/AVLiveWireTests/StreamDemuxerTests.swift | Tests for stream demuxer chunking/resync behaviors | | shared/AVLiveWire/Tests/AVLiveWireTests/SmokeTests.swift | Smoke test for protocol version | | shared/AVLiveWire/Tests/AVLiveWireTests/LoopbackTests.swift | End-to-end demux loopback test | | shared/AVLiveWire/Tests/AVLiveWireTests/FrameHeaderTests.swift | FrameHeader encode/decode validation tests | | shared/AVLiveWire/Sources/AVLiveWire/WirePayloads.swift | Define SkeletonPayload/VideoPayload wire formats | | shared/AVLiveWire/Sources/AVLiveWire/StreamDemuxer.swift | Incremental frame demuxer for TCP byte streams | | shared/AVLiveWire/Sources/AVLiveWire/FrameHeader.swift | Binary frame header + BE helpers | | shared/AVLiveWire/Sources/AVLiveWire/AVLiveWire.swift | Protocol version namespace | | shared/AVLiveWire/Package.swift | SwiftPM package definition for AVLiveWire | | launcher/Sources/AVLiveLauncher/ProcessManager.swift | Disable legacy AV-Live-Body spawn path (archived) | | launcher/CLAUDE.md | Note about AV-Live-Body archival and new location | | launcher/_archive-AV-Live-Body/Tests/AVLiveBodyTests/USBSkeletonConsumerTests.swift | Tests for archived USB skeleton mapping | | launcher/_archive-AV-Live-Body/Tests/AVLiveBodyTests/USBMuxProtocolTests.swift | Tests for archived usbmux protocol codec | | launcher/_archive-AV-Live-Body/Tests/AVLiveBodyTests/USBClientTests.swift | Tests for archived usbmux client behaviors | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/USBSkeletonConsumer.swift | Archived USB consumer (StreamDemuxer + dispatch) | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/USBMuxProtocol.swift | Archived usbmux protocol codec | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/USBClient.swift | Archived usbmux client + Unix socket transport | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/SkeletonOverlay.swift | Archived 2D skeleton overlay rendering | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/SettingsPanel.swift | Archived settings panel UI | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/SceneRenderer.swift | Archived Metal background renderer | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/RenderSettings.swift | Archived render settings expansion | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/PoseOSCListener.swift | Archived listener extended with face/hand/pose3d | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/OSCServer.swift | Archived TCP OSC server implementation | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/MeshRenderer.swift | Archived mesh renderer tracks pelvis position | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/HUDOverlay.swift | Archived HUD overlay view | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/FaceHandOverlay.swift | Archived SwiftUI Canvas face/hand overlay | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/DataHUDOverlay.swift | Archived data feeds HUD UI | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/DataFeedsOSCListener.swift | Archived OSC listener for open-data feeds | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/CameraPreviewLayer.swift | Archived macOS webcam preview plumbing | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/BodyView.swift | Archived RealityKit view wiring (adds 3D skeleton) | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/AVLiveBodyApp.swift | Archived app wires listeners + HUD | | launcher/_archive-AV-Live-Body/Sources/AVLiveBody/ArkitOSCListener.swift | Archived ARKit OSC listener for diagnostics | | launcher/_archive-AV-Live-Body/Package.swift | Archived SwiftPM package now depends on AVLiveWire + tests | | launcher/_archive-AV-Live-Body/ARCHIVED.md | Marker doc for archived project | | iphone-arbody/README.md | iOS app README + build/signing notes | | iphone-arbody/project.yml | XcodeGen project for ARBodyTracker | | iphone-arbody/Config/Shared.xcconfig | Shared signing/build settings (includes optional Local) | | iphone-arbody/Config/Local.xcconfig.example | Template for developer-local team ID | | iphone-arbody/Config/Local.xcconfig | Developer-local team ID (committed; should be removed) | | iphone-arbody/CLAUDE.md | Agent guidance for iPhone project + AVLiveWire commands | | iphone-arbody/ARBodyTracker.swiftpm/Sources/ARBodyTracker/VideoEncoder.swift | HEVC encoder -> VideoPayload | | iphone-arbody/ARBodyTracker.swiftpm/Sources/ARBodyTracker/USBServer.swift | TCP server sending framed AVLiveWire messages | | iphone-arbody/ARBodyTracker.swiftpm/Sources/ARBodyTracker/Info.plist | iOS Info.plist for camera/network usage | | iphone-arbody/ARBodyTracker.swiftpm/Sources/ARBodyTracker/ARBodyTrackerApp.swift | SwiftUI app entrypoint | | iphone-arbody/ARBodyTracker.swiftpm/Package.swift | SwiftPM executable package for ARBodyTracker | | iphone-arbody/ARBodyTracker.swiftpm/.swiftpm/xcode/package.xcworkspace/contents.xcworkspacedata | Generated SwiftPM workspace artifact (should not be committed) | | docs/ICP_FUSION.md | Documentation for LiDAR↔SMPL-X ICP fusion | | data_only_viz/tests/test_state_arkit.py | Tests for new ARKit state fields | | data_only_viz/tests/test_pose_filter.py | Tests for pose filter chain behavior/perf | | data_only_viz/tests/test_open3d_smoke.py | Open3D dependency smoke tests | | data_only_viz/tests/test_multihmr_coreml.py | Update perf expectation for Multi-HMR CoreML | | data_only_viz/tests/test_multihmr_arkit_z.py | Tests for ARKit pelvis Z override logic | | data_only_viz/tests/test_multi_hmr_worker.py | Add predict_once behavior test (naming mismatch) | | data_only_viz/tests/test_lidar_receiver.py | LiDAR TCP decoder/reader unit tests | | data_only_viz/tests/test_lidar_calib.py | Extrinsic persistence + Kabsch tests | | data_only_viz/tests/test_iphone_osc_listener.py | Tests for iPhone OSC listener -> State | | data_only_viz/tests/test_icp_fusion.py | ICP fusion unit tests (Open3D-gated) | | data_only_viz/tests/test_dino_reid.py | DINOv2 re-id tests (mlpackage-gated) | | data_only_viz/tests/test_arkit_joint_map.py | Mapping integrity tests | | data_only_viz/tests/test_arkit_fuse.py | Tests for ArkitFuse stage | | data_only_viz/state.py | Extend State with ARKit, LiDAR, joints, last_frame_rgb fields | | data_only_viz/smplx_osc_sender.py | Wire MeshRigger with optional DINO re-id config | | data_only_viz/scripts/quantize_multihmr_int8.py | INT8 weight quantization helper script | | data_only_viz/scripts/coreml_full_probe.py | Update conversion precision note (FP32) | | data_only_viz/scripts/calibrate_lidar.py | CLI scaffold for LiDAR extrinsic calibration | | data_only_viz/scripts/bench_icp_fusion.py | ICP fusion benchmark script | | data_only_viz/pyproject.toml | Add optional `lidar` extra with Open3D constraint | | data_only_viz/multihmr_coreml.py | Update CoreML output names + normalization + compute unit selection | | data_only_viz/main.py | Start iPhone OSC listener; opt-in ICP fusion thread; stop ICP on exit | | data_only_viz/lidar_receiver.py | LiDAR TCP frame decode + threaded reader | | data_only_viz/lidar_calib.py | Extrinsic model + JSON I/O + Kabsch solver | | data_only_viz/iphone_osc_listener.py | Python OSC listener for ARKit joints | | data_only_viz/icp_fusion_worker.py | Threaded ICP fusion wrapper (currently without state lock) | | data_only_viz/CLAUDE.md | Document ARKit fusion plumbing and mapping | | data_only_viz/arkit_joint_map.py | ARKit 91 -> MediaPipe 33 mapping constants | | data_feeds/osc_sender.py | OSC sender wrapper | | data_feeds/main.py | Feed runner CLI | | data_feeds/feeds/velib.py | Velib GBFS specialization | | data_feeds/feeds/teleray.py | Stub feed | | data_feeds/feeds/sytadin.py | Stub feed | | data_feeds/feeds/prim.py | Stub feed | | data_feeds/feeds/hubeau.py | Hub’Eau feed implementation | | data_feeds/feeds/gbfs.py | Generic GBFS reader | | data_feeds/feeds/eco2mix.py | Eco2Mix feed implementation | | data_feeds/feeds/carburants.py | Stub feed | | data_feeds/feeds/base.py | Feed base class + scheduler loop | | data_feeds/feeds/ais.py | Stub feed | | data_feeds/feeds/__init__.py | Feed registry | | data_feeds/config.avlivedata.toml | Default feed config file | | data_feeds/__init__.py | Package marker | | CLAUDE.md | Add network topology + env var table | | avlivebody-mac/Tests/AVLiveBodyTests/USBMuxProtocolTests.swift | Tests for usbmux protocol codec | | avlivebody-mac/Tests/AVLiveBodyTests/USBClientTests.swift | Tests for usbmux client | | avlivebody-mac/Tests/AVLiveBodyTests/BodyFusionTests.swift | Tests for mesh/skeleton pelvis depth fusion | | avlivebody-mac/Tests/AVLiveBodyTests/.gitkeep | Keep test dir tracked | | avlivebody-mac/Sources/AVLiveBody/VideoQuad.swift | RealityKit video background plane | | avlivebody-mac/Sources/AVLiveBody/usb/USBSkeletonConsumer.swift | macOS USB consumer with video decode callback | | avlivebody-mac/Sources/AVLiveBody/usb/USBMuxProtocol.swift | usbmux protocol codec | | avlivebody-mac/Sources/AVLiveBody/usb/USBClient.swift | usbmux client + Unix transport | | avlivebody-mac/Sources/AVLiveBody/usb/MultiHMRCoreML.swift | CoreML wrapper for Multi-HMR model | | avlivebody-mac/Sources/AVLiveBody/usb/BodyFusion.swift | Fuse skeleton pelvis depth into Multi-HMR translation | | avlivebody-mac/Sources/AVLiveBody/StatusBar.swift | USB connection status overlay | | avlivebody-mac/Sources/AVLiveBody/SkeletonEntity.swift | RealityKit skeleton marker rendering | | avlivebody-mac/Sources/AVLiveBody/SceneView.swift | SwiftUI bridge to ARView | | avlivebody-mac/Sources/AVLiveBody/SceneController.swift | Scene setup + update wiring + orbit controls | | avlivebody-mac/Sources/AVLiveBody/MeshEntity.swift | RealityKit SMPL-X mesh rendering | | avlivebody-mac/Sources/AVLiveBody/Info.plist | macOS app Info.plist | | avlivebody-mac/Sources/AVLiveBody/AxisConvention.swift | Axis conversion helper | | avlivebody-mac/Sources/AVLiveBody/AVLiveBodyApp.swift | App entrypoint wiring USB -> scene + optional Multi-HMR | | avlivebody-mac/README.md | Build/run docs for new macOS app | | avlivebody-mac/project.yml | XcodeGen project for macOS app | | avlivebody-mac/Config/Shared.xcconfig | Shared macOS build settings | | avlivebody-mac/Config/Local.xcconfig.example | Local signing template | | avlivebody-mac/.gitignore | Ignore generated projects/build outputs | | AGENTS.md | AI agent guidance document | | .gitignore | Ignore iPhone/macos SwiftPM build artifacts + local configs | </details> <details> <summary>Files not reviewed (1)</summary> * **iphone-arbody/ARBodyTracker.swiftpm/.swiftpm/xcode/package.xcworkspace/contents.xcworkspacedata**: Language not supported </details> --- 💡 <a href="/electron-rare/AV-Live/new/main?filename=.github/instructions/*.instructions.md" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Add Copilot custom instructions</a> for smarter, more guided reviews. <a href="https://docs.github.com/en/copilot/customizing-copilot/adding-repository-custom-instructions-for-github-copilot" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Learn how to get started</a>.
@@ -0,0 +5,4 @@
## Project
`AV-Live` — live-coding audio-visual performance system: SuperCollider sound engine, openFrameworks visualiser driven by a Hantek 6022BL oscilloscope, and a SwiftUI menubar launcher orchestrating everything. Public, GPL-3. Repo `electron-rare/AV-Live`, branch `main`. Multi-host: GrosMac (source), macm1 (sink / Multi-HMR + Apple Vision ANE), iPhone 16 Pro (ARKit/LiDAR pub).
copilot-pull-request-reviewer[bot] (Migrated from github.com) commented 2026-05-21 10:45:12 +00:00

The PR title/description says it's a docs-only change adding an AGENTS.md skeleton, but this PR also introduces substantial new Swift/Python code (AVLiveWire package, iOS/macOS apps, ICP fusion pipeline, etc.). Please update the PR title/description to reflect the actual scope, or split the docs addition into a separate PR so reviewers can assess the code changes independently.

The PR title/description says it's a docs-only change adding an `AGENTS.md` skeleton, but this PR also introduces substantial new Swift/Python code (AVLiveWire package, iOS/macOS apps, ICP fusion pipeline, etc.). Please update the PR title/description to reflect the actual scope, or split the docs addition into a separate PR so reviewers can assess the code changes independently.
@@ -0,0 +34,4 @@
# Swift
open launcher/Package.swift # or xcodebuild from CLI
open avlivebody-mac/avlivebody.xcodeproj
copilot-pull-request-reviewer[bot] (Migrated from github.com) commented 2026-05-21 10:45:12 +00:00

The command open avlivebody-mac/avlivebody.xcodeproj appears incorrect/inconsistent with the avlivebody-mac docs and project.yml (project name is AVLiveBody.xcodeproj, and it is generated/ignored). This should point to the generated AVLiveBody.xcodeproj after running xcodegen generate, otherwise new contributors will hit a dead path.

The command `open avlivebody-mac/avlivebody.xcodeproj` appears incorrect/inconsistent with the `avlivebody-mac` docs and `project.yml` (project name is `AVLiveBody.xcodeproj`, and it is generated/ignored). This should point to the generated `AVLiveBody.xcodeproj` after running `xcodegen generate`, otherwise new contributors will hit a dead path.
@@ -0,0 +7,4 @@
@inline(__always)
func arkitToRealityKit(_ v: SIMD3<Float>) -> SIMD3<Float> {
SIMD3<Float>(v.x, -v.y, -v.z)
}
copilot-pull-request-reviewer[bot] (Migrated from github.com) commented 2026-05-21 10:45:14 +00:00

The doc comment says the conversion keeps y pointing up ("y up" -> "y up"), but the implementation negates y (SIMD3(v.x, -v.y, -v.z)). Please either fix the comment to match the actual transform, or adjust the transform so it matches the documented axis convention (otherwise it’s easy to apply the wrong transform across the codebase).

The doc comment says the conversion keeps `y` pointing up ("y up" -> "y up"), but the implementation negates `y` (`SIMD3(v.x, -v.y, -v.z)`). Please either fix the comment to match the actual transform, or adjust the transform so it matches the documented axis convention (otherwise it’s easy to apply the wrong transform across the codebase).
@@ -0,0 +63,4 @@
self._state.lidar_timestamp_ns = frame.timestamp_ns
try:
self._state.icp_metadata = self._worker.run_once(
self._state)
copilot-pull-request-reviewer[bot] (Migrated from github.com) commented 2026-05-21 10:45:13 +00:00

IcpFusionThread writes to shared state (lidar_points, lidar_timestamp_ns, icp_metadata, and potentially persons_smplx[*].vertices_3d via run_once) without holding state.lock(). This contradicts the codebase’s stated threading invariant (and risks races with other workers/UI readers). Please stage/copy the LiDAR frame outside the lock if needed, but wrap all state mutations (including the in-place mesh update) in a with state.lock(): block or introduce a dedicated lock for these fields.

`IcpFusionThread` writes to shared `state` (`lidar_points`, `lidar_timestamp_ns`, `icp_metadata`, and potentially `persons_smplx[*].vertices_3d` via `run_once`) without holding `state.lock()`. This contradicts the codebase’s stated threading invariant (and risks races with other workers/UI readers). Please stage/copy the LiDAR frame outside the lock if needed, but wrap all `state` mutations (including the in-place mesh update) in a `with state.lock():` block or introduce a dedicated lock for these fields.
@@ -585,6 +619,12 @@ class AppDelegate(NSObject):
self._listener.stop()
copilot-pull-request-reviewer[bot] (Migrated from github.com) commented 2026-05-21 10:45:13 +00:00

The iPhone OSC listener (self._iphone_osc) is started in _start_pose_worker, but it is not stopped in applicationWillTerminate_. This can leave a background OSC server thread running during shutdown. Please add a guarded stop/shutdown here (similar to the ICP fusion thread stop) to ensure clean termination.

The iPhone OSC listener (`self._iphone_osc`) is started in `_start_pose_worker`, but it is not stopped in `applicationWillTerminate_`. This can leave a background OSC server thread running during shutdown. Please add a guarded stop/shutdown here (similar to the ICP fusion thread stop) to ensure clean termination.
@@ -122,0 +132,4 @@
persons_hands_mesh: dict = field(default_factory=dict)
persons_hands_mesh_cam_t: dict = field(default_factory=dict)
persons_hands_mesh_last_t: float = 0.0
copilot-pull-request-reviewer[bot] (Migrated from github.com) commented 2026-05-21 10:45:14 +00:00

The comments describe per-pid/per-side "companion arrays" for hand meshes (including a per-hand last-update timestamp), but persons_hands_mesh_last_t is a single float. If the intent is per-pid/per-side freshness tracking, this should likely be a dict keyed by pid/side (or the comment should be updated to reflect a single global timestamp).

The comments describe per-pid/per-side "companion arrays" for hand meshes (including a per-hand last-update timestamp), but `persons_hands_mesh_last_t` is a single float. If the intent is per-pid/per-side freshness tracking, this should likely be a dict keyed by pid/side (or the comment should be updated to reflect a single global timestamp).
@@ -54,0 +63,4 @@
import pytest, numpy as np
rgb = np.zeros((480, 640, 3), dtype=np.uint8)
with pytest.raises(NotImplementedError):
worker.predict_once(rgb)
copilot-pull-request-reviewer[bot] (Migrated from github.com) commented 2026-05-21 10:45:13 +00:00

This test name says returns_none_when_coreml_unavailable, but the assertion expects predict_once to raise NotImplementedError. Either rename the test to match the current behavior, or (if the intended API is to return None) update the implementation and assert accordingly so the test communicates the contract clearly.

This test name says `returns_none_when_coreml_unavailable`, but the assertion expects `predict_once` to raise `NotImplementedError`. Either rename the test to match the current behavior, or (if the intended API is to return `None`) update the implementation and assert accordingly so the test communicates the contract clearly.
@@ -0,0 +4,4 @@
<FileRef
location = "self:">
</FileRef>
</Workspace>
copilot-pull-request-reviewer[bot] (Migrated from github.com) commented 2026-05-21 10:45:12 +00:00

This .swiftpm/xcode/.../contents.xcworkspacedata file is an Xcode/SwiftPM generated artifact. Since .swiftpm/ is already being gitignored, this file should be removed from version control to avoid churn and machine-specific workspace state being committed.

This `.swiftpm/xcode/.../contents.xcworkspacedata` file is an Xcode/SwiftPM generated artifact. Since `.swiftpm/` is already being gitignored, this file should be removed from version control to avoid churn and machine-specific workspace state being committed.
@@ -0,0 +1 @@
DEVELOPMENT_TEAM = K9KK43329X
copilot-pull-request-reviewer[bot] (Migrated from github.com) commented 2026-05-21 10:45:11 +00:00

Config/Local.xcconfig contains a concrete DEVELOPMENT_TEAM value. This file is meant to be developer-local (the repo docs and .gitignore indicate it should be gitignored) and committing a real Team ID can leak personal/org signing configuration. Please remove this file from the repo and keep only Local.xcconfig.example committed.

`Config/Local.xcconfig` contains a concrete `DEVELOPMENT_TEAM` value. This file is meant to be developer-local (the repo docs and `.gitignore` indicate it should be gitignored) and committing a real Team ID can leak personal/org signing configuration. Please remove this file from the repo and keep only `Local.xcconfig.example` committed.
@@ -0,0 +28,4 @@
if start > 0 { buffer.removeFirst(start) }
guard buffer.count >= FrameHeader.byteCount,
let h = FrameHeader(decoding: buffer) else { break }
if h.length > Self.maxPayloadLength {
copilot-pull-request-reviewer[bot] (Migrated from github.com) commented 2026-05-21 10:45:13 +00:00

If FrameHeader(decoding:) returns nil while findMagic() found magic at offset 0 and buffer.count >= byteCount, feed currently breaks, leaving the corrupt header at the front of the buffer forever and preventing resynchronization (even if valid frames follow). Instead of breaking, treat this as a corrupt header and advance the buffer (e.g., drop 1 byte or magic.count) and continue searching for the next magic sequence.

If `FrameHeader(decoding:)` returns nil while `findMagic()` found magic at offset 0 and `buffer.count >= byteCount`, `feed` currently `break`s, leaving the corrupt header at the front of the buffer forever and preventing resynchronization (even if valid frames follow). Instead of breaking, treat this as a corrupt header and advance the buffer (e.g., drop 1 byte or `magic.count`) and continue searching for the next magic sequence.
Sign in to join this conversation.