Reviews and fixes RuView's 10 most recent substantive issues, plus closes
the wiring gap flagged in the perception-substrate review (docs/adr,
gist, release notes from the PR #1579 review).
## Fixed
- #1526: UI falsely claimed "LIVE — ESP32 Hardware Connected" whenever the
unauthenticated /api/v1/status probe 401'd. Now sends the bearer token
on the probe (the fallback direction was already fixed by ADR-295).
- #1554: top-level `classification` (GET /api/v1/sensing/latest) was the
last UDP packet's single node, not the fused room aggregate, so it
flapped at packet rate with 2+ disagreeing nodes. Now derived from
`RoomInference` (ADR-297's `fuse_room`) at both call sites.
- #1541: per-node MQTT `presence`/`presence_score` read a `classification`
JSON key that does not exist on `NodeInfo` (the real field is
`node_inference`), silently falling back to the room aggregate for
every node — reproducing the exact "lockstep publish" the field report
measured. Also fixed the *existing* regression test for this bug,
which used the same wrong key in its own fixture and so never caught it.
- #1557: pose-fusion's `wsPortMap` only knew port 3000, so a remapped host
port fell through to `localhost:8765` (nothing there for a remote
viewer). WS port is now derived from `location.port` instead of a
2-entry lookup table (the "never render simulated as live" half was
already fixed by ADR-295's `onVerifiedFrame` gate).
- #1525: the no-model pose path already clamps keypoint confidence to a
0.1 floor, but the renderer's own threshold is also 0.1 compared with
`<=`/`>` — a keypoint at exactly the floor was still invisible. Floor
raised to 0.15 to clear the client's gate.
- #1556: `--mqtt-ca-file`/`--mqtt-client-cert`/`--mqtt-client-key` were
parsed and stored but never applied — TLS always used the system trust
store, so a self-signed broker always failed UnknownIssuer. Now builds
`rumqttc::TlsConfiguration::Simple` from the real PEM files (no new TLS
dependency needed). A file that can't be read logs why and falls back
to system trust instead of failing opaquely later.
- #1555: the MQTT availability heartbeat asserted "online" for every known
node on a fixed 30s timer regardless of whether that node's data was
still arriving. Now tracks each node's last-seen broadcast snapshot and
only reports "online" within a 10s freshness window, otherwise
"offline" — a frozen sensor can no longer look available. (The other
half — restarting a publisher that goes permanently silent — needs a
reproduction the reporter themselves weren't certain of; left for a
follow-up rather than guessing at the trigger.)
- #1540: already fixed on main (node-keyed RateLimiter, ADR-297).
- #1521/#1522: not fixable in this repo (published HF model artifact);
replied with the ADR-298 gate status and the exact byte-level fix for
the safetensors header, and corrected the README row that claimed the
file loads with the reference loader.
- #1542, #1527: replied — #1542 is a real, larger firmware+server feature
left open for follow-up; #1527's suggested fixes were already applied,
the one residual sample is an inherent first-frame paint gap.
## Certificate spine wiring (closes the gap flagged in the PR #1579 review)
`ruview-certify` and `ruview-policy` now depend on `ruview-ood` and provide
real `From<ruview_ood::DomainState>` adapters plus a composed entry point,
`ruview_policy::authorize_from_certificate`, matching the adapter contract
`ruview-policy`'s own doc comment already described but that no code
actually implemented. A new cross-crate integration test
(`acceptance_test_b_real_integration`) mints a real signed
`CapabilityCertificate` and proves a real post-drift `ruview_ood::Unknown`
denies a `SafetyCritical` action through the composed pipeline — not two
disconnected unit tests hand-setting the same enum value.
Also:
- Wires `evaluate_linear_head` (ADR-298 model-release gate) into a new CI
job so the checker itself can't silently regress; documents that gating
an actual model publish is still a manual step (no HF automation here).
- Wires `SourceState::export_watermark()` into `start_recording`: a
recording captured while the source is synthetic is now stamped in its
metadata (not the filename or per-line JSON, to avoid breaking
`delete_recording`'s path reconstruction or the training dataset
loader's schema).
- Updates docs/user-guide.md's "Developer Preview" section to describe
what's now genuinely wired vs. still not (no live continuous
calibration/OOD loop in the running server yet).
## Validation
- cargo test --workspace --no-default-features: 4391 passed, 0 failed
- cargo build --release -p wifi-densepose-sensing-server --features mqtt:
clean
- Server smoke-tested end-to-end against the simulator (real startup,
UDP/WS/HTTP listeners, /api/v1/sensing/latest stable across calls)
- Real ESP32-S3 hardware was NOT reachable this session (no COM port
present, zero UDP frames received after 40s bound to 0.0.0.0:5005) —
the multi-node fixes are validated by the new unit/integration tests
and full-workspace regression, not by live hardware.
Co-Authored-By: claude-flow <ruv@ruv.net>
ADR-292 (sensing-server): SourceState enum + pure transition (provenance.rs);
auth-error/unknown can never resolve to LiveVerified; synthetic exports
watermarked. Pose-fusion simulator starts SYNTHETIC and only shows LIVE on a
real decoded frame (#1557); sensing client no longer labels an unauthorized
status endpoint as live (#1526).
ADR-294 (sensing-server): NodeInference distinct from RoomInference (inference.rs);
deterministic freshness-weighted fuse_room; RateLimiter re-keyed to
(NodeId,EntityKind) so nodes do not starve each other (#1541); stale nodes go
unavailable not frozen-online (#1555).
ADR-293 (sensing-server): --udp-bind (default 127.0.0.1) + --udp-allow allowlist
+ fail-closed refusal of routable bind without allowlist unless --udp-insecure-lan
(udp_bind.rs); crate SECURITY.md documents the threat model and the deferred
per-device-auth step two.
ADR-295 (train): model_gates.rs — constant-output, unreachable-boundary (the
issue-1521 degenerate presence head), class-balance, baseline, and
metric-name-provenance gates.
ADR-296 (ci): scripts/csi-data-policy-check.sh (+ allowlist) and a workflow that
fails on tracked CSI-format/oversized-JSONL files; 6/6 self-tests pass.
Per-crate suites reported green by the swarm; CSI policy self-test 6/6 and JS
syntax verified here. Full workspace re-verification deferred until the
concurrent phase-1 spine build frees the target dir (disk pressure).
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_015TcKegTS7QqhWPC2L2SzaS
websocket.service.js (used by the pose/event streams) opened a bare
`new WebSocket(url)` with no ADR-272 ticket exchange, unlike
sensing.service.js which already mints a ticket per connect. Since a
browser cannot set an Authorization header on a WebSocket upgrade, and
the server rejects a long-lived bearer passed as a query string
(CWE-598), the pose stream 401'd whenever auth was on — visible in the
Live Demo tab as "Failed to create WebSocket connection".
Fix: createWebSocketWithTimeout() now strips any `token` query param a
caller put on the URL (pose.service.js does this) and exchanges the
stored bearer for a single-use `?ticket=` via withWsTicket(), done at
this one choke point so every consumer (pose, events, training) and
every reconnect attempt gets a fresh ticket.
Second, smaller bug: pose-fusion/js/main.js auto-connected to a
hardcoded `ws://localhost:8765/ws/sensing`, but the Docker image serves
the sensing WebSocket on :3001 (the same 3000->3001 mapping
sensing.service.js already encodes) — the auto-connect dialed a port
nothing listens on. Reuses that port mapping and tickets both the
auto-connect and the manual "Connect" button.
Bumped the pose-fusion.html cache-buster (v=13 -> v=14) so browsers
actually fetch the updated main.js.
Adds ui/services/websocket.service.test.mjs (4 executed Node tests,
stubbed WebSocket/fetch/localStorage) covering: no-auth passthrough,
ticket exchange + bearer-never-in-URL, stray ?token= stripping, and the
pre-ADR-272 404 fallback. Wired into the CI "Run UI unit tests" step.
Reported with a verified fix in #1461 by wsc7r4zcj4-collab; this PR
implements the same fix against current main with an added regression
test.
Closes#1461
- Add null-safe optional chaining for embPoints and rssiDbm in diagnostic log
- Handle Blob data in _handleLiveFrame (convert to ArrayBuffer before processing)
- Bump cache busters to v=13
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat: dual-modal WASM browser pose estimation demo (ADR-058)
Live webcam video + WiFi CSI fusion for real-time pose estimation.
Two parallel CNN pipelines (ruvector-cnn-wasm) with attention-weighted
fusion and dynamic confidence gating. Three modes: Dual, Video-only,
CSI-only. Includes pre-built WASM package (~52KB) for browser deployment.
- ADR-058: Dual-modal architecture design
- ui/pose-fusion.html: Main demo page with dark theme UI
- 7 JS modules: video-capture, csi-simulator, cnn-embedder, fusion-engine,
pose-decoder, canvas-renderer, main orchestrator
- Pre-built ruvector-cnn-wasm WASM package for browser
- CSI heatmap, embedding space visualization, latency metrics
- WebSocket support for live ESP32 CSI data
- Navigation link added to main dashboard
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix: motion-responsive skeleton + through-wall CSI tracking
- Pose decoder now uses per-cell motion grid to track actual arm/head
positions — raising arms moves the skeleton's arms, head follows
lateral movement
- Motion grid (10x8 cells) tracks intensity per body zone: head,
left/right arm upper/mid, legs
- Through-wall mode: when person exits frame, CSI maintains presence
with slow decay (~10s) and skeleton drifts in exit direction
- CSI simulator persists sensing after video loss, ghost pose renders
with decreasing confidence
- Reduced temporal smoothing (0.45) for faster response to movement
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix: video fills available space + correct WASM path resolution
- Remove fixed aspect-ratio and max-height from video panel so it
fills the available viewport space without scrolling
- Grid uses 1fr row for content area, overflow:hidden on main grid
- Fix WASM path: resolve relative to JS module file using import.meta.url
instead of hardcoded ./pkg/ which resolved incorrectly on gh-pages
- Responsive: mobile still gets aspect-ratio constraint
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat: live ESP32 CSI pipeline + auto-connect WebSocket
- Add auto-connect to local sensing server WebSocket (ws://localhost:8765)
- Demo shows "Live ESP32" when connected to real CSI data
- Add build_firmware.ps1 for native Windows ESP-IDF builds (no Docker)
- Add read_serial.ps1 for ESP32 serial monitor
Pipeline: ESP32 → UDP:5005 → sensing-server → WS:8765 → browser demo
Co-Authored-By: claude-flow <ruv@ruv.net>
* docs: add ADR-059 live ESP32 CSI pipeline + update README with demo links
- ADR-059: Documents end-to-end ESP32 → sensing server → browser pipeline
- README: Add dual-modal pose fusion demo link, update ADR count to 49
- References issue #245
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat: RSSI visualization, RuVector attention WASM, cache-bust fixes
- Add animated RSSI Signal Strength panel with sparkline history
- Fix RuVector WasmMultiHeadAttention retptr calling convention
- Wire up RuVector Multi-Head + Flash Attention in CNN embedder
- Add ambient temporal drift to CSI simulator for visible heatmap animation
- Fix embedding space projection (sparse projection replaces cancelling sum)
- Add auto-scaling to embedding space renderer
- Add cache busters (?v=4) to all ES module imports to prevent stale caches
- Add diagnostic logging for module version verification
- Add RSSI tracking with quality labels and color-coded dBm display
- Includes ruvector-attention-wasm v2.0.5 browser ESM wrapper
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat: 26-keypoint dexterous pose + full RuVector attention pipeline
Pose Decoder (17 → 26 keypoints):
- Add finger approximations: thumb, index, pinky per hand (6 new)
- Add toe tips: left/right foot index (2 new)
- Add neck keypoint (1 new)
- Hand openness driven by arm motion intensity
- Finger positions computed from wrist-elbow axis angles
CNN Embedder (full RuVector WASM pipeline):
- Stage 1: Multi-Head Attention (global spatial reasoning)
- Stage 2: Hyperbolic Attention (hierarchical body-part tree)
- Stage 3: MoE Attention (3 experts: upper/lower/extremities, top-2)
- Blended 40/30/30 weighting → final embedding projection
Canvas Renderer:
- Magenta finger joints with distinct glow
- Cyan toe tips
- White neck keypoint
- Thinner limb lines for hand/foot connections
- Joint count shown in overlay label
CSI Simulator:
- Skip synthetic person state when live ESP32 connected
- Only simulate CSI data in demo mode (was already correct)
Embedding Space:
- Fixed projection: sparse 8-dim projection replaces cancelling sum
- Auto-scaling normalizes point spread to fill canvas
Cache busters bumped to v=5 on all imports.
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix: centroid-based pose tracking for responsive limb movement
Rewrites pose decoder from intensity-based to position-based tracking:
- Arms now track toward motion centroid in each body zone
- Elbow/wrist positions computed along shoulder→centroid vector
- Legs track toward lower-body zone centroids
- Smoothing reduced from 0.45 to 0.25 for responsiveness
- Zone centroids blend 30% old / 70% new each frame
6 body zones with overlapping coverage:
- Head (top 20%, center cols)
- Left/Right Arm (rows 10-60%, outer cols)
- Torso (rows 15-55%, center cols)
- Left/Right Leg (rows 50-100%, half cols each)
Hand openness now driven by arm spread distance + raise amount.
Cache busters v=6.
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix: remove duplicate lAnkleX/rAnkleX declarations in pose-decoder
Stale code block from old intensity-based tracking was left behind,
re-declaring variables already defined by centroid-based tracking.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(demo): wire all 6 RuVector WASM attention mechanisms into pose fusion
- Add WasmLinearAttention and WasmLocalGlobalAttention to browser ESM wrapper
- Add 6 WASM utility functions (batch_normalize, pairwise_distances, etc.)
- Extend CnnEmbedder to 6-stage pipeline: Flash → MHA → Hyperbolic → Linear → MoE → L+G
- Use log-energy softmax blending across all 6 stages
- Wire WASM cosine_similarity and normalize into FusionEngine
- Add RuVector pipeline stats panel to UI (energy, refinement, pose impact)
- Compute embedding-to-joint mapping stats without modifying joint positions
- Center camera prompt with flexbox layout
- Add cache busters v=12
Co-Authored-By: claude-flow <ruv@ruv.net>
