From 501f1383609fcd6adb56e2105956fca38530f8b0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 16 Aug 2026 23:20:11 +0000 Subject: [PATCH] docs(adr): ADR-324 off-axis head-coupled perspective demo (RF-assisted) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Deep-research ADR answering whether icurtis1/off-axis-sneaker can be used with RuView. Adopts the head-coupled perspective technique (Kooima generalized off-axis projection) via a clean-room implementation — the upstream repo is unlicensed, so no code or assets are reused. Defines a tiered integration: webcam-fine tracking with RF presence gating and multi-person arbitration (Tier A), an explicitly labeled RF-only coarse body-parallax mode (Tier B), and evidence-gated future metric RF head positioning (Tier C). No server changes; existing /ws/sensing and /api/v1/stream/pose streams only. Indexed in the ADR README. Co-Authored-By: claude-flow Claude-Session: https://claude.ai/code/session_01BU3NcEgTpAVvu5QGtw4czT --- ...-off-axis-head-coupled-perspective-demo.md | 261 ++++++++++++++++++ docs/adr/README.md | 1 + 2 files changed, 262 insertions(+) create mode 100644 docs/adr/ADR-324-off-axis-head-coupled-perspective-demo.md diff --git a/docs/adr/ADR-324-off-axis-head-coupled-perspective-demo.md b/docs/adr/ADR-324-off-axis-head-coupled-perspective-demo.md new file mode 100644 index 00000000..af3ba16e --- /dev/null +++ b/docs/adr/ADR-324-off-axis-head-coupled-perspective-demo.md @@ -0,0 +1,261 @@ +# ADR-324: off-axis-mode — RF-assisted head-coupled perspective for the three.js realtime demo + +| Field | Value | +|-------|-------| +| **Status** | Proposed | +| **Date** | 2026-08-16 | +| **Deciders** | ruv | +| **Codename** | **off-axis-mode** | +| **Scope** | New `examples/three.js/demos/07-off-axis-window.html` (client-side only); no server changes | +| **Relates to** | ADR-019 (sensing-only UI), ADR-035 (live sensing UI accuracy), ADR-169 (adam-mode), ADR-170 (yoga-mode), ADR-282 (L0–L5 evidence ladder), ADR-295 (source provenance), ADR-306 (spatial ontology), ADR-307 (persistent tracking), ADR-323 (pose refinement) | +| **Prior art** | [`icurtis1/off-axis-sneaker`](https://github.com/icurtis1/off-axis-sneaker) (reference only — see §2.1 licensing) | +| **Numbering note** | ADR-324 is the next free number in the authoring checkout (322 is unused, 323 is the latest on disk). Re-run the ADR index/collision check immediately before merge and rename if needed. | +| **Tracking issue** | none yet | + +--- + +## 1. Context + +### 1.1 The question this ADR answers + +"Can we use [`icurtis1/off-axis-sneaker`](https://github.com/icurtis1/off-axis-sneaker) +with RuView?" The answer is: **yes for the technique, no for the code, and +only honestly for the RF part.** This ADR records the research behind each of +those three clauses and defines the integration that is actually defensible. + +### 1.2 What off-axis-sneaker is + +`off-axis-sneaker` is a React + TypeScript + Vite web app that renders a GLB +model (a sneaker) in three.js and creates a *head-coupled perspective* +("fish-tank VR" / "window into the screen") illusion: + +- **Tracking input**: MediaPipe Face Mesh (468 facial landmarks) from a + webcam. Head (x, y) comes from the eye midpoint; depth (z) is proxied by + inter-ocular distance. An exponential moving average (default factor 0.3) + smooths jitter; sensitivity multipliers are `strengthX: 4`, `strengthY: 3`, + `strengthZ: 2`. +- **Projection**: `src/utils/offAxisCamera.ts` builds a **true asymmetric + (off-axis) frustum** — `makePerspective(left, right, top, bottom, near, far)` + with `left/right/top/bottom = (screenBound − eyePosition) · (near / + viewerToScreenDistance)` — i.e. Kooima's generalized perspective projection, + plus a matching camera translation. Constants: `nearPlane 0.05`, + `farPlane 1000`, `worldScale 0.01` (cm → world units), `movementScale 1.5`. +- **Calibration**: a wizard captures physical screen width/height (cm), + typical viewing distance, and pixel density, stored locally, so eye position + is computed relative to the *physical* display. + +The technique descends from Johnny Chung Lee's 2007 Wii-remote desktop VR +demo and the fish-tank VR literature (Ware, Arthur & Booth, CHI '93). The +projection math is Robert Kooima's "Generalized Perspective Projection" +(2008). Both are public, well-documented techniques independent of any one +implementation. + +### 1.3 What the illusion physically requires + +The head-coupled illusion is only convincing when the tracked eye position is +**accurate to roughly centimeters** and **low-latency**. The VR literature +puts comfortable motion-to-photon latency below ~20 ms for head-mounted +displays; desktop fish-tank VR tolerates more, but visible lag between head +motion and parallax response is exactly what breaks the "window" illusion. +`CLAIMED` (literature values; no RuView measurement exists for this demo yet). + +### 1.4 What RuView RF sensing can actually supply today + +This is where honesty is mandatory (repo rule: never present WiFi sensing as +camera-grade). + +- **Field-peak position, not metric localization.** + `wifi-densepose-sensing-server/src/field_localize.rs` derives a position + from the strongest peak of the 20×20 `signal_field` carried on + `/ws/sensing` `sensing_update` frames. Its own module doc states the + caveat: the subcarrier→angle mapping is a *representation*; "a single ESP32 + link cannot resolve a true (x, z) room position." The emitted position is + "strongest field peak in the room model," mapped with `X_SCALE 0.6`, + `Z_SCALE 0.5`, gated by `PEAK_THRESHOLD 0.35` — real, live, motion-tracking, + but **not a calibrated person fix** and nowhere near eye-position precision. +- **RF pose is 2-D, normalized, constant-confidence.** The committed Cog + (ADR-101, restated by ADR-323) emits 17 COCO keypoints as normalized 2-D + coordinates with a constant confidence and no per-joint uncertainty. A + "nose" keypoint exists (COCO index 0), but it is not a metric 3-D head fix. +- **Tracks are coarse and pseudonymous by design.** `ruview-track` (ADR-307) + maintains `person_N` tracks with container-level ("kitchen → hallway") + continuity, coarse non-reversible features, and asserts **no accuracy + number** — outputs default to evidence level `L1`. +- **Update cadence and latency are unmeasured for this purpose.** The demo + pipeline runs at ~30 Hz on the MediaPipe side (ADR-170), but no end-to-end + RF motion-to-photon latency has been measured. Any figure quoted for the RF + path must be tagged `MEASURED` with a reproducer before it appears in docs + or UI. + +Conclusion of the capability match: **RF cannot drive a convincing fish-tank +illusion by itself today**, and this ADR does not claim it can. RF *can* +supply things a webcam cannot: camera-free presence, zone-level position, +person count, approach direction, and pseudonymous continuity — including +when the camera is off. + +### 1.5 What this ADR is *not* + +- Not a vendoring of `off-axis-sneaker` (see §2.1 — the repo has no license). +- Not a claim of camera-grade RF head tracking, at any tier. +- Not a backend change: no new server endpoints, no new auth surface, no + schema changes. Purely additive client-side HTML/JS, like ADR-169/170. +- Not a React/Vite/Tailwind adoption. The `examples/three.js/demos/*` are + dependency-light single-file HTML demos and stay that way. + +## 2. Decision + +### 2.1 Licensing: adopt the technique, not the code + +`off-axis-sneaker` publishes **no license**. Under default copyright, its +source cannot be copied, vendored, or translated into this repository. +Decision: + +1. **No code, assets, or models from `off-axis-sneaker` enter this repo.** + The GLB sneaker model is likewise unlicensed for reuse; demos use assets + already present in `examples/`. +2. The off-axis projection is implemented **clean-room from the public + sources**: Kooima's "Generalized Perspective Projection" (2008) — the + `pa/pb/pc` screen-corner formulation — and three.js's documented + `PerspectiveCamera.projectionMatrix` override path. The repository is cited + as prior art in this ADR only. +3. If upstream later adds a permissive license, revisiting reuse requires a + new ADR note, not silent copying. + +### 2.2 Tiered integration — each tier labeled by what it really is + +**Tier A (ships first): webcam-fine + RF-context hybrid.** +`07-off-axis-window.html` uses MediaPipe Face Landmarker (already the pattern +in demo 05) for fine head tracking and the Kooima frustum for rendering — +functionally what off-axis-sneaker does, reimplemented. RuView RF adds the +camera-free layer around it: + +- **Presence-gated camera**: the webcam pipeline starts only when the RF + presence signal (`/ws/sensing` `sensing_update`) says someone is in the + zone, and stops after a configurable RF-vacancy timeout. The privacy + posture improves: the camera is *off* until physics says there is someone + to track. +- **Multi-person arbitration**: when RF reports more than one person, the HUD + says so and the demo holds the last stable perspective instead of jumping + between faces. +- **Pre-warm**: RF approach direction (field-peak trajectory) warms up + MediaPipe and the scene before the person sits down. + +**Tier B (demo mode, prominently labeled): RF-only coarse parallax.** +A toggle drives the off-axis eye position from RF alone — field peak (x, z) +plus the pose nose keypoint when present — through a one-euro filter, a +deadband, and a hard gain clamp. The HUD labels it **"RF coarse body +parallax — not head tracking"** and shows the live evidence level (`L1` +heuristic unless a certificate says otherwise, per ADR-282/ADR-318). The +expected experience is a slow, body-scale parallax sway — a demonstrative +"the room model moves because *you* moved, with no camera" — not a stable +fish-tank illusion. The demo must never present Tier B as equivalent to +Tier A. + +**Tier C (future, explicitly gated, not promised): metric RF head position.** +Only a calibrated multistatic deployment (ADR-297 multi-node semantics, +ADR-311 fusion, ADR-303 ground-truth sync) with an evidence-engine ledger +entry (ADR-304) and a capability certificate (ADR-318) could justify feeding +RF positions into the fine path. No current data supports this; Tier C exists +in this ADR solely so nobody ships it informally without those gates. + +### 2.3 Implementation surface + +- New file `examples/three.js/demos/07-off-axis-window.html` (07, not 06 — + ADR-170 reserves `06-yoga-mode.html`). Single-file demo following the 01–05 + conventions: same CSS custom properties, same HUD/helper-panel pattern, + served from the existing static demo server + (`http://127.0.0.1:8765/examples/three.js/demos/…`). +- A small clean-room module (inline `