smolkik_adm/RuView
1
0
Fork 0
mirror of https://github.com/ruvnet/RuView.git synced 2026-08-31 12:36:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(nlos): add consumer transient sensing pipeline

Browse Source
This commit is contained in:
rUv
2026-08-22 20:46:26 -04:00
parent 0df48df7b2
commit a373ff666b
100 changed files with 16218 additions and 6698 deletions
Download Patch File Download Diff File Expand all files Collapse all files

251
docs/adr/ADR-328-consumer-nlos-raw-transient-reproduction-pipeline.md Normal file
View File

@@ -0,0 +1,251 @@
# ADR-328: Consumer NLOS raw-transient pipeline and upstream reproduction boundary
| Field | Decision |
|---|---|
| **Status** | Accepted for staged implementation; software contracts may be validated in CI, live-hardware reproduction remains pending until a witness capture passes the protocol |
| **Date** | 2026-08-22 |
| **Owners** | RuView Labs maintainers and sensing research reviewers |
| **Scope** | Commodity optical transient acquisition, calibration, normalization, provenance, upstream reproduction |
| **Extends** | ADR-295, ADR-303, ADR-305, ADR-319, ADR-320 |
| **Related** | ADR-329, ADR-330, ADR-331 |
| **Primary implementation** | `v2/crates/ruview-nlos`, upstream `sidsoma/consumer-nlos`, `harness/ruview` advisory verification |
## Context
Somasundaram et al. introduce motion-induced aperture sampling (MAS) for
consumer time-of-flight LiDAR. Their measurement is not an ordinary depth map.
Each sensor zone records light intensity over time. The strong direct relay-wall
return is followed by much weaker multipath returns whose path lengths constrain
hidden geometry. Multiple frames supply redundant and spatially diverse samples
that improve signal-to-noise ratio and synthesize a larger virtual aperture.
The [Nature paper](https://doi.org/10.1038/s41586-026-10502-x),
[author manuscript](https://arxiv.org/html/2605.17865v1), and
[MIT project page](https://cornar.media.mit.edu/) report 3D reconstruction,
single and multi-object tracking, camera localization, and real-time tracking at
30 Hz. These are upstream `CLAIMED` results until RuView reproduces them with a
named live capture. They are not evidence that Apple exposes the required
measurement to an ordinary iOS application.
The upstream [consumer-nlos implementation](https://github.com/sidsoma/consumer-nlos)
targets ST's P-NUCLEO-53L8A1 research path and captures per-zone histograms. ST
documents the [VL53L8CH](https://www.st.com/en/imaging-and-photonics-solutions/vl53l8ch.html)
compact-normalized-histogram interface and configuration-dependent zone/bin/rate
limits. The exact silicon, expansion board, firmware/API and transient format
must be read from and bound to each capture rather than inferred from the kit
name. The
[P-NUCLEO-53L8A1](https://www.st.com/en/evaluation-tools/p-nucleo-53l8a1.html)
combines the expansion board with an STM32 Nucleo host.
RuView needs an explicit boundary because flattening the input to point clouds,
`Observation.value`, ARKit depth, or CSI destroys the delayed transient that the
inverse problem needs. A successful build or replay also cannot validate the
photon path, relay geometry, ambient-light behavior, or target reflectivity.
## Decision
### 1. Reproduce upstream before modifying its inference state
Phase 1 uses a pinned commit of the upstream implementation and a documented,
upstream-compatible ST assembly with verified raw or compact-normalized
histogram access. The current reference adapter targets VL53L8CH framing, but
the acceptance record stores the observed board/silicon identity and never
promotes a model name inferred from packaging. RuView records the exact upstream
commit, firmware/API digest, scoped enrollment/certificate reference, configuration, calibration
digest, and capture-manifest digest. Upstream code runs as an isolated research
sidecar or separately reviewed tool; it is not silently vendored into the
production Rust dependency graph.
The first geometry follows the upstream plug-and-play arrangement:
1. a planar relay surface fills the sensor field of view;
2. an opaque occluder prevents direct line of sight to the target;
3. a known rigid retroreflective target supplies the initial high-SNR case;
4. an independent ground-truth system measures the hidden trajectory; and
5. an empty-scene background is captured before each experimental block.
Diffuse rigid objects and people are later strata. They cannot be pooled into
the retroreflective acceptance result. The manuscript notes weaker diffuse
returns and an approximately fourth-power distance falloff in its diffuse model;
the project therefore reports performance by target material and range.
### 2. Preserve a first-class transient frame
`v2/crates/ruview-nlos` owns the initial versioned transient scaffold. The table
below is the capture-manifest/next-contract requirement for live promotion, not
a claim that every field already exists in the current v1 Rust/track schema.
The current scaffold implements bounded histogram/provenance/calibration pieces;
missing clock-domain, certificate, world-frame, configuration, and capture
bindings must land in a reviewed schema before L2. The eventual wire/storage
representation retains, at minimum:
| Field group | Required content | Rejection rule |
|---|---|---|
| Identity | schema version, sensor ID or certificate reference, session ID, sequence | unknown schema, unauthenticated identity, duplicate or regressing sequence |
| Time | sensor monotonic timestamp, host receive timestamp, clock domain and uncertainty | future, stale, non-finite, or unbounded clock error |
| Histogram | zone layout, temporal bin count and width, signed/unsigned count encoding, ambient estimate | zero/oversized dimensions, non-finite values, count overflow |
| Geometry | per-zone ray or relay-wall point, sensor intrinsics, world transform, coordinate-frame ID | non-invertible transform, unit mismatch, incompatible frame |
| Calibration | direct-return peak, mask, background reference, calibration and configuration digests | absent, expired, mismatched, or out-of-distribution calibration |
| Provenance | source `LIVE_HARDWARE`/`REPLAY`/`SYNTHETIC`, firmware and capture digests, evidence level | unknown never becomes live; replay and synthetic are visibly distinct |
The HAL may wrap the frame as a modality-specific payload, but it must not
reduce the histogram to a scalar before NLOS preprocessing. Raw transient input
has its own size/rate limits and parser fuzz surface.
### 3. Deterministic normalization stages
The reference normalization pipeline is ordered and individually testable:
1. verify identity, schema, bounds, sequence, timestamps, and calibration;
2. subtract a compatible empty-scene background;
3. locate and mask the strong one-bounce relay-surface return per zone;
4. align each zone's direct peak to the agreed temporal origin;
5. reject saturated, underexposed, or calibration-incompatible zones;
6. transform time/depth into the light-cone coordinate used by MAS; and
7. emit a normalized transient plus quality flags, never an unconditional track.
Calibration is an explicit state machine: `UNAVAILABLE`, `CAPTURING`, `VALID`,
`DEGRADED`, `EXPIRED`, or `REJECTED`. Only `VALID` calibration can produce a
live optical likelihood. Loss of calibration produces `unknown`, not a cached
or synthetic fallback.
### 4. Separate acquisition evidence from inference evidence
The capture manifest is append-only and content-addressed. It binds the raw
stream, exclusions, firmware, configuration, calibration, clock synchronization,
target/relay-surface stratum, and ground-truth source. Track records reference
the capture and model digests. RuVector or a particle filter may improve
temporal inference, but neither can upgrade acquisition provenance or fabricate
unobserved photons.
Raw captures are local research data by default and are not committed to Git.
Only schemas, small non-person fixtures, checksums, and aggregate metrics may be
reviewed in the repository under the data policy.
## Performance decision
The first live reproduction targets end-to-end track updates at **at least 27
Hz**, a preregistered operational definition of “roughly 30 fps.” The rate is
measured from accepted sensor frame through emitted track, not the configured
sensor clock. Dropped, duplicated, replayed, or late frames are not counted.
The frozen zone/bin/integration configuration must itself be documented and
demonstrated capable of this rate; ST configurations documented at 25 Hz cannot
pass merely because another configuration has a 30 Hz maximum.
Implementation budgets are:
| Stage | Budget and behavior |
|---|---|
| Parser and provenance | bounded allocation; reject before copying oversized dimensions |
| Normalization | one pass over zones × bins; reusable buffers; no unbounded capture queue |
| Tracking | bounded particle count and search volume; overload drops stale work rather than growing latency |
| End-to-end | report p50/p95 latency, update rate, frame loss, CPU and memory with named reproducer |
The paper describes 1,000 particles and a 5 cm proximity prior at 30 Hz. Those
are starting parameters, not RuView guarantees. Optimization must preserve a
golden likelihood/track tolerance and may not hide quality loss behind pooled
throughput.
## Security and privacy
1. USB/serial is the initial acquisition transport. A future network bridge
requires authenticated sensor identity, encryption, replay protection,
explicit bind configuration, and a separate threat review. No new
unauthenticated UDP listener is introduced.
2. Firmware and upstream code are supply-chain inputs. Pin commits and toolchain
versions, review licenses, scan dependencies, and verify build/capture
digests. Flashing remains an explicit, confirmed hardware mutation.
3. Malformed frames, NaN/Inf values, integer products, timestamp wrap,
decompression bombs, calibration substitution, and coordinate transforms are
fail-closed parser cases.
4. NLOS presence and trajectory data are sensitive even without imagery. The
approved operator/controller and every required participant receive purpose/
space/time notice and consent controls, pause/withdrawal and a persistent
indicator; raw retention is bounded and session track IDs never claim identity.
5. Safety-critical actuation is outside this ADR. A hypothesis never directly
drives a lock, vehicle, medical device, or emergency decision.
The full attacker/asset analysis is in
`docs/security/consumer-nlos-threat-model.md`.
## Alternatives considered
### Feed ARKit scene depth directly into the MIT algorithm
Rejected. Apple's documented scene-depth surface is processed distance data,
not the per-zone photon-arrival histogram required by the image formation
model. ARKit remains useful for pose and visible geometry under ADR-330.
### Treat CSI as a synthetic optical transient
Rejected. CSI and optical time-of-flight measure different physical channels.
CSI can contribute an independently calibrated likelihood under ADR-329; it
cannot replace the optical measurement or supervise itself.
### Port upstream code to Rust before reproducing it
Rejected for Phase 1. Simultaneously changing hardware, forward model, state
estimator, and language makes failures uninterpretable. The Rust port follows a
pinned upstream baseline and golden captures.
### Vendor all upstream firmware, Python, and data into RuView
Rejected. It increases supply-chain, license, binary-size, and data-governance
risk. The reproduction boundary uses pins, manifests, adapters, and small legal
fixtures instead.
## Consequences
### Positive
1. RuView gains an honest optical-transient modality without corrupting CSI or
scalar HAL semantics.
2. Upstream reproduction and RuView extensions remain experimentally separable.
3. Provenance, calibration, and failure states survive into every downstream
hypothesis.
4. The same contract can later support another histogram-capable ToF sensor.
### Costs and limitations
1. Initial use requires an external ST board and ground-truth rig.
2. Raw histograms and CSI increase sensitive data volume and governance burden.
3. Calibration, reflectivity, relay geometry, ambient light, and motion can
dominate algorithm changes.
4. This decision does not establish through-wall optical sensing, unrestricted
human reconstruction, identity, or production safety.
## Rollout and rollback
| Phase | Enable condition | Rollback trigger | Rollback action |
|---|---|---|---|
| R0 fixtures | schema/parser/property tests pass | parser panic, unbounded allocation, provenance ambiguity | disable crate feature and retain fixtures only |
| R1 upstream live reproduction | approved protocol, verified histogram interface, external ground truth, and `MEASURED` >=27 Hz accepted-update witness | calibration drift, direct line of sight, saturation, provenance gap | invalidate capture; return to controlled geometry |
| R2 Rust shadow | R1 remains valid and golden-capture agreement is within frozen tolerance | likelihood/track divergence or latency regression | keep upstream sidecar authoritative; disable Rust output |
| R3 candidate live | independent witness plus ADR-331 confidence/security/privacy/guardrail gate | research endpoint fails or privacy controls regress | emit `unknown`; withdraw capability certificate |
Rollback never relabels a failed live capture as a passing replay. Existing CSI,
RuVector, WorldGraph, and other RuView runtime paths remain independently usable.
## Objective acceptance mapping
| ID | Requirement | Evidence |
|---|---|---|
| NLOS-328-01 | Preserve bounded per-zone timing histograms and calibration provenance | Rust round-trip, boundary, fuzz/property, golden-vector tests |
| NLOS-328-02 | Reject stale, duplicate, oversized, unauthenticated, non-finite, and calibration-mismatched input | Negative parser and state-machine tests |
| NLOS-328-03 | Keep `LIVE_HARDWARE`, `REPLAY`, and `SYNTHETIC` mutually exclusive | provenance transition tests and capture manifest review |
| NLOS-328-04 | Reproduce hidden-target tracking at roughly 30 fps | `MEASURED` live capture, external ground truth, >=27 accepted updates/s |
| NLOS-328-05 | Prevent synthetic-only acceptance | `ruview nlos verify --require-research-pass` rejection tests |
| NLOS-328-06 | Make upstream/Rust comparison reproducible | pinned commit, firmware/config/calibration/capture SHA-256 digests |
| NLOS-328-07 | Preserve normal RuView operation when NLOS is absent | workspace tests with crate/sensor disabled; advisory harness reports `ABSENT` |
The authoritative experimental steps and statistical gate are in
`docs/research/consumer-nlos-acceptance-protocol.md`.
## References
1. Somasundaram et al., [“Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling”](https://doi.org/10.1038/s41586-026-10502-x), Nature 653, 693–699 (2026).
2. [Author manuscript and methods](https://arxiv.org/html/2605.17865v1).
3. [MIT Consumer NLOS project](https://cornar.media.mit.edu/).
4. [Upstream implementation](https://github.com/sidsoma/consumer-nlos).
5. STMicroelectronics, [VL53L8CH product specification](https://www.st.com/en/imaging-and-photonics-solutions/vl53l8ch.html).
6. STMicroelectronics, [P-NUCLEO-53L8A1 evaluation kit](https://www.st.com/en/evaluation-tools/p-nucleo-53l8a1.html).

285
docs/adr/ADR-329-motion-induced-aperture-csi-fusion-world-model.md Normal file
View File

@@ -0,0 +1,285 @@
# ADR-329: Motion-induced aperture inference, CSI fusion, and RuView world-state integration
| Field | Decision |
|---|---|
| **Status** | Accepted for staged implementation; software fusion is testable with replay, retained performance claims require the live paired protocol |
| **Date** | 2026-08-22 |
| **Owners** | RuView Labs perception, RuVector, RuField, and WorldGraph maintainers |
| **Scope** | MAS forward model, temporal inference, calibrated RF/optical fusion, persistent identity-free state |
| **Depends on** | ADR-273, ADR-295, ADR-301 through ADR-305, ADR-311, ADR-319, ADR-328 |
| **Related** | ADR-330, ADR-331 |
| **Primary implementation** | `v2/crates/ruview-nlos`, `ruview.nlos.track.v1` |
## Context
The consumer-NLOS model converts weak, time-resolved optical multipath into a
posterior over hidden geometry or motion. Its strength is localized geometry
around a relay surface; its weaknesses include low signal-to-noise ratio,
reflectivity, range, aperture coverage, and temporal assumptions. RuView CSI
offers a different failure profile: RF can persist through walls and clutter,
but current commodity CSI generally provides coarser, environment-dependent
spatial evidence.
These modalities should not be concatenated merely because both are
interesting. Fusion is justified only when an independently evaluated RF
likelihood improves an objective endpoint over optical NLOS alone. It is also
unsafe to let a learned temporal store or graph overwrite current sensing
quality. RuVector, RuField, and WorldGraph must preserve uncertainty, freshness,
calibration, and provenance rather than manufacture confidence.
The [consumer-NLOS paper](https://arxiv.org/html/2605.17865v1) expresses tracking
as a sequential posterior and uses a particle filter to carry uncertainty and a
motion prior. This aligns with RuView's temporal and spatial primitives, but the
physics likelihood remains the authority for the optical observation.
## Decision
### 1. Use a typed latent state and explicit likelihood factors
For each session-scoped hidden target hypothesis, the latent state is
\[
X_t = (p_t, v_t, \Sigma_t, q_t, a_t)
\]
where position \(p_t\), velocity \(v_t\), covariance or particle distribution
\(\Sigma_t\), quality state \(q_t\), and bounded attributes \(a_t\) are expressed
in a versioned world coordinate frame. No identity field is inferred.
The fused filter computes
\[
P(X_t \mid L_{1:t}, R_{1:t}) \propto
P(L_t \mid X_t, C_L)^{w_L}
P(R_t \mid X_t, C_R)^{w_R}
P(X_t \mid X_{t-1}),
\]
where \(L_t\) is the normalized optical transient, \(R_t\) is independently
captured CSI evidence, and \(C_L,C_R\) are modality-specific calibration states.
The exponents are bounded reliability weights derived from current quality,
not user-facing confidence decoration.
Missing, stale, rejected or invalid modalities are omitted before factor
evaluation and contribute the multiplicative identity. They are never evaluated
as a zero likelihood raised to a zero weight (`0^0`).
Conditional independence is an approximation. Correlated errors such as motion,
clock drift, shared ground-truth leakage, and environmental change are measured
and documented. If correlation cannot be bounded, the implementation uses a
conservative mixture, covariance intersection, or gating rule instead of
multiplying overconfident factors.
### 2. Keep the MAS physics path inspectable
The optical likelihood follows the upstream structure:
1. transform the normalized transient into light-cone coordinates;
2. precompute a canonical space-time impulse response for the known target
shape or reconstruction basis;
3. propagate a bounded particle set using the frozen motion model;
4. render each particle by indexing the canonical response at current relay-wall
samples and pose;
5. score rendered and observed transients with a normalized, bounded likelihood;
6. normalize, calculate degeneracy diagnostics, and residual-resample; and
7. emit the posterior, entropy/effective-particle count, covariance, and quality.
Optimization may vectorize or cache rendering, but every optimized kernel is
compared against a scalar reference over golden and property-generated inputs.
NaN/Inf, all-zero likelihood, underflow, particle collapse, and out-of-volume
states produce a typed degradation or `unknown` result.
### 3. Calibrate CSI as a likelihood, not a centimetre claim
The initial RF factor is deliberately modest. It may provide:
1. hidden-region presence or absence likelihood;
2. coarse zone occupancy;
3. motion onset/cessation likelihood;
4. a broad position prior with measured covariance; or
5. an empty-volume vote for guarded optical background updates.
RF weights are zero when the CSI sensor identity, room calibration, coordinate
transform, freshness, or out-of-distribution gate is invalid. CSI never sharpens
an optical posterior beyond the calibration evidence that justifies it. A model
trained with LiDAR pseudo-labels is evaluated on a held-out partition with
external ground truth; it may not be scored against its own optical teacher.
### 4. Synchronize before fusion
Both inputs enter a bounded temporal join keyed by tenant, workspace, site,
world-frame ID, and capture session. The join records sensor timestamp, receive
timestamp, clock uncertainty, calibration digest, and maximum pairing skew.
Frames outside the preregistered skew are not interpolated into apparent
coherence. The system may make an optical-only or RF-only observation with the
missing modality named, but it may label output `fused` only when both factors
pass identity, freshness, calibration, and time/space alignment checks.
### 5. Divide responsibilities across RuView primitives
| Component | Responsibility | Explicit non-responsibility |
|---|---|---|
| MAS/particle filter | current optical likelihood and posterior update | long-term identity, authorization, hardware provenance creation |
| CSI adapter/model | current calibrated RF likelihood | optical histogram synthesis, centimetre precision without evidence |
| RuVector | embeddings for trajectory/canonical-response similarity and bounded temporal retrieval | replacing the live Bayesian update or upgrading evidence level |
| RuField | observation confidence, covariance, quality, provenance, calibration, and expiration | presenting unknown/stale data as ground truth |
| WorldGraph | session-scoped hidden-object hypothesis nodes and spatial/temporal relations | identity inference or permanent person graph by default |
| Evidence/witness layer | content digests, acceptance result, lineage and receipts | sensing, fusion, or actuation |
`ruview.nlos.track.v1` is the current shared output contract. It carries
track/session IDs, one source classification, position/velocity, covariance,
freshness/expiry, quality state, modality contribution weights, calibration
hash, evidence level, algorithm revision, and one provenance record. It does
not carry two authenticated modality lineages, a world-frame ID, coordinate
transform digest, or capture-manifest digest. Consequently v1 rejects measured
CSI fusion and `l3_corroborated`; the current CSI path is scope-bound synthetic
L0 regression only. A future v2 contract must add those bindings before F1/F2
measured fusion can be enabled. Consumers reject unknown schema versions and
stale tracks now; coordinate compatibility remains a future contract gate.
### 6. Cross-modal training follows, and cannot contaminate, evaluation
After a frozen rules/physics fusion baseline, LiDAR may supervise an RF model.
Dataset partitions are grouped by capture session, subject/target, room, sensor
configuration, and time block to prevent adjacent-frame and environment
leakage. Optical targets for RF training are probabilistic distributions with
quality masks, not hard ground truth. The test endpoint always uses independent
external ground truth and includes LiDAR-only, CSI-only, and fused arms.
The system retains a non-learned fallback. A learned score or fusion policy must
show stratified improvement and calibration before promotion. Model absence or
error degrades to the last independently validated factor, never direct action.
## Architecture and data flow
```mermaid
flowchart TD
A["Transient frame + optical calibration"] --> B["MAS optical likelihood"]
C["CSI frame + RF calibration"] --> D["RF likelihood"]
B --> E["Bounded temporal Bayesian join"]
D --> E
E --> F["ruview.nlos.track.v1"]
F --> G["RuField + RuVector + WorldGraph"]
```
Only the join creates a fused track. RuVector and WorldGraph are downstream
state consumers, not a shortcut around missing, stale, or rejected modalities.
## Performance decision
1. The live tracking loop is bounded to the newest accepted frame. Backpressure
drops obsolete queued work and records loss; it does not accumulate latency.
2. The particle count, canonical volume, search volume, history length, graph
nodes per session, and temporal-join window are configuration-bounded.
3. The fusion budget is measured independently from optical tracking. Report
p50/p95 sensor-to-track latency and update rate for LiDAR-only and fused arms.
4. Optimization is retained only when numerical agreement stays within the
frozen tolerance and the paired research endpoint does not regress.
5. At least 27 accepted LiDAR-only track updates per second is required before
the fusion endpoint is interpreted. A fast fused path cannot rescue a failed
reproduction.
## Security and privacy
1. The join accepts only authenticated sensor/session identities and compatible
tenant/workspace/world-frame bindings. Cross-tenant or cross-session joins
are impossible by type and policy.
2. Sequence and timestamp replay defenses are per modality. Reusing an old CSI
frame to make an optical track look persistent is rejected and audited.
3. A malicious modality may inject extreme likelihoods. Inputs and weights are
bounded, posterior influence is observable, and single-modality ablations are
recorded for forensic review.
4. Track IDs are random and session-scoped. Raw optical/CSI data is local by
default; downstream stores receive bounded hypotheses and provenance unless
an approved research protocol explicitly retains raw data.
5. Long-term memory uses TTL, purpose limitation, deletion, and tenant isolation.
Similarity is not identity. The graph must not create a biometric profile.
6. No fusion output grants actuation. Governed actions require the independent
ADR-321/327 policy, approval, freshness, and receipt path.
## Alternatives considered
### Concatenate optical and CSI tensors into one end-to-end network
Deferred. It obscures failure attribution, calibration, and missing-modality
behavior before a trustworthy baseline exists. A learned likelihood may be
added after the factorized paired benchmark.
### Use CSI only as a binary veto
Useful as an initial safety rule, but insufficient as the final design because
it discards calibrated spatial/motion information. The typed factor supports a
binary likelihood without fixing the architecture to it.
### Store only the posterior mean
Rejected. NLOS can be multimodal and ambiguous. Covariance, particles or a
bounded distribution summary, posterior entropy, and quality are essential for
honest downstream behavior.
### Let RuVector replace the particle filter
Rejected. Vector memory can retrieve similar histories or canonical responses,
but similarity alone is not the current physics likelihood and cannot enforce
frame-level calibration/provenance.
### Promote fusion if any pooled metric improves
Rejected. The objective endpoint, pairing, strata, confidence interval, and
non-success metrics are frozen before capture. Synthetic-only and pooled-only
gains do not pass.
## Consequences
### Positive
1. Optical localization evidence and RF persistence can complement one another without
conflating their measurement physics.
2. Every downstream state retains uncertainty, freshness, modality, and lineage.
3. The factorized baseline makes ablation, failure analysis, and rollback clear.
4. Cross-modal training has a leakage-resistant target and external evaluation.
### Costs and limitations
1. Coordinate/clock calibration and paired ground truth add operational burden.
2. Conditional-independence violations can make naive multiplication
overconfident; conservative fusion may sacrifice apparent sharpness.
3. Current CSI may not provide enough independent spatial information to meet
the 25 percent endpoint. A negative result is acceptable and stops rollout.
4. The architecture tracks hypotheses; it does not establish identity, intent,
photographic reconstruction, or safety certification.
## Rollout and rollback
| Phase | Behavior | Promotion | Rollback |
|---|---|---|---|
| F0 | optical-only reference plus recorded CSI | deterministic replay and calibration tests | disable RF factor |
| F1 | shadow fusion, output not consumed | ADR-328 live LiDAR-only reproduction is `MEASURED` at >=27 Hz in its capture-manifest witness report and paired live capture is complete | discard shadow output |
| F2 | research-visible fused track with explicit evidence | ADR-328 prerequisite remains valid; `MEASURED` >=25% endpoint has adjusted confidence support plus frozen-protocol, independent-CSI, security/privacy, guardrail and witness review | return to optical-only; invalidate certificate |
| F3 | cross-modal learned likelihood in shadow | held-out external-ground-truth gain and calibration | remove model artifact; retain factorized baseline |
Rollback is a configuration/capability-certificate change. Existing optical,
CSI, RuVector, RuField, and WorldGraph services continue independently.
## Objective acceptance mapping
| ID | Requirement | Evidence |
|---|---|---|
| NLOS-329-01 | Reference MAS likelihood is deterministic and bounded | scalar/golden/property tests, all-zero, omitted-factor/`0^0`, and numeric-extreme tests |
| NLOS-329-02 | Fused output requires valid identity, calibration, freshness, clocks, and coordinate frame for both factors | negative temporal/spatial join matrix plus missing/stale-factor omission tests |
| NLOS-329-03 | Missing/rejected modality cannot be mislabeled fused | modality/provenance state tests |
| NLOS-329-04 | RuVector/RuField/WorldGraph preserve uncertainty, TTL, lineage, and session scope | contract and integration tests |
| NLOS-329-05 | Fusion improves a preregistered objective endpoint | F2/ADR-331 gate under one frozen protocol over >=100 paired `LIVE_HARDWARE` sequences: `MEASURED` >=25% mean-position-error **or** lost-track-rate reduction, successful multiplicity-adjusted interval excludes zero, shared endpoint-pairing digest and full position coverage, all frozen guardrails pass, independent CSI plus privacy/security/witness reviews pass |
| NLOS-329-06 | Evaluation is leakage resistant | grouped partitions, frozen protocol, external ground truth, LiDAR/CSI/fused ablation |
| NLOS-329-07 | Optimization preserves correctness and bounded latency | reference equivalence, benchmark deltas, overload/backpressure tests |
| NLOS-329-08 | Fusion cannot actuate directly | policy/API tests proving hypothesis-only output |
## References
1. Somasundaram et al., [Nature article](https://doi.org/10.1038/s41586-026-10502-x).
2. [Motion-induced aperture model and particle-filter methods](https://arxiv.org/html/2605.17865v1).
3. ADR-273: Unified RF spatial world model.
4. ADR-295: Source provenance state machine.
5. ADR-301/302/303/304/305: calibration, OOD, ground truth, evidence, and authenticated sensor identity.
6. ADR-311: Real sensor fusion.
7. ADR-319: Witness chain.

288
docs/adr/ADR-330-native-and-web-ios-nlos-adapter-boundary.md Normal file
View File

@@ -0,0 +1,288 @@
# ADR-330: Native and web iOS NLOS adapters and Apple API boundary
| Field | Decision |
|---|---|
| **Status** | Accepted; native and web software surfaces are implementable now, direct built-in iPhone transient-NLOS remains unsupported until a documented Apple API and live device evidence exist |
| **Date** | 2026-08-22 |
| **Owners** | RuView Labs iOS, mobile web, API, security, and sensing maintainers |
| **Scope** | Swift package/app, ARKit context adapter, authenticated track transport, web replay/live UI, capability claims |
| **Depends on** | ADR-295, ADR-305, ADR-319, ADR-328, ADR-329 |
| **Related** | ADR-034, ADR-035, ADR-331 |
| **Implementation** | `ui/ios-nlos`, `ui/mobile`, `ruview.nlos.track.v1` |
## Context
The phrase “smartphone-grade LiDAR” describes a performance/cost class; it does
not guarantee that an App Store process can read every internal sensor signal.
The MIT technique needs each SPAD zone's photon-arrival histogram. Apple
documents ARKit APIs for processed
[`sceneDepth`](https://developer.apple.com/documentation/arkit/arframe/scenedepth),
[`smoothedSceneDepth`](https://developer.apple.com/documentation/arkit/arframe/smoothedscenedepth),
world tracking, and
[`sceneReconstruction`](https://developer.apple.com/documentation/arkit/arworldtrackingconfiguration/scenereconstruction).
Apple's [scene-depth point-cloud sample](https://developer.apple.com/documentation/arkit/displaying-a-point-cloud-using-scene-depth)
shows how applications request and unproject processed depth. These public
surfaces do not document the per-zone transient histogram used by the MAS
measurement model.
This is an API assessment, not a claim about undisclosed Apple hardware or
future operating systems. It must be reviewed against official documentation
for each supported iOS/Xcode release. Until the needed signal is documented and
validated on a physical device, RuView must not present ARKit depth as an MIT
NLOS reproduction.
The user still needs two useful iOS surfaces:
1. a native app that compiles a shared contract, uses public ARKit depth/pose for
visible context, and consumes external RuView NLOS tracks; and
2. a web-capable mobile UI that receives authenticated tracks or deterministic
replay without pretending to capture the phone's LiDAR.
## Decision
### 1. Publish one contract with explicit capability levels
All iOS surfaces consume `ruview.nlos.track.v1`. Capability is an enum, not
inferred from device marketing:
| Capability | Meaning | Permitted label |
|---|---|---|
| `unavailable` | no valid track source | unavailable/unknown |
| `replay` | deterministic fixture or recorded stream | `SYNTHETIC` or `REPLAY`, persistently watermarked |
| `arkit_context` | public ARKit pose/depth/mesh only | line-of-sight context; never NLOS |
| `external_live` | authenticated live track from histogram-capable external pipeline | live external NLOS, subject to evidence/certificate |
| `apple_transient_live` | future documented raw-transient Apple adapter | disabled until separate ADR, API proof, device witness, and ADR-331 gate |
Unknown values fail closed. UI copy names the actual source and evidence level.
It never shortens `external_live` to “iPhone sees around corners.”
### 2. Keep the Swift core platform-neutral
`ui/ios-nlos/Package.swift` defines:
1. `RuViewNLOSCore`, a pure Swift contract/validation library that can be unit
tested without ARKit; and
2. `RuViewNLOSApple`, an Apple-only adapter behind `canImport(ARKit)` and runtime
availability/capability checks.
The direct iOS app and shared `RuViewNLOS` scheme use the same validated model.
The current v1 core decoder rejects unknown/excess JSON shape, non-finite or
bounded-range numeric values, invalid covariance diagonals, expired/future
tracks, excessive arrays, duplicate IDs, and illegal provenance/evidence
transitions. It has no world-frame or capture-manifest field yet; those are
live-promotion contract requirements, not claims about this G0 scaffold.
The current Apple adapter is deliberately a static capability probe: it does
not start an `ARSession`, request camera permission, or capture/export any depth.
It documents the public-API boundary and supports the authenticated external
track client. A later, separately reviewed line-of-sight context adapter may
export camera pose, intrinsics, visible scene depth/confidence, smoothed depth,
and mesh metadata with explicit `arkit_context` labeling. Neither implementation
synthesizes photon histograms or changes context into an NLOS evidence source.
### 3. Treat the native app as a client of the external NLOS pipeline
The production data path is intended to be an authenticated, versioned RuView
track endpoint, not direct access to the ST board from UI code. The current v1
client verifies:
1. TLS and the configured RuView service identity;
2. a scoped, revocable pairing token or short-lived session authorization;
3. authenticated server session, envelope session, and schema bindings;
4. monotonic sequence and bounded clock skew;
5. track expiry, covariance, modality contributions, and calibration hash.
L2/live promotion additionally requires tenant/workspace authorization,
coordinate-frame and capture-manifest bindings, scoped enrolled sensor identity,
and a valid witness/capability certificate. Those fields are not silently
inferred from v1.
Network loss, app suspension, sensor unavailability, decode error, and stale
data immediately degrade the capability
and clear or visually expire the live track. Cached data is never silently live.
The initial Swift client further pins concrete bounds: `wss` only; no embedded
URL credentials or fragments; redirects refused; ephemeral URLSession state; a
256 KiB maximum message; an exact bounded JSON model; sequence no larger than
the JavaScript safe integer and strictly increasing within a bound session; at
most a 5 second track lifetime; and pairing tokens of 32–512 visible ASCII bytes
stored as `WhenUnlockedThisDeviceOnly` Keychain data. `SYNTHETIC` frames require
replay transport and the reserved zero calibration hash. Live provenance must
retain a raw/CNH transient kind; a nonzero calibration hash is enforced at L2
calibrated and above, not for every L1 live envelope. These are wire/client
safety rules, not proof that the stream is physically honest.
### 4. Web iOS consumes tickets and replay; it does not capture transients
The NLOS surface in `ui/mobile` implements the same schema and freshness rules.
Live mode first obtains an authenticated, short-lived, single-purpose ticket
from `/api/v1/nlos/ws-ticket`, then connects to the NLOS stream. Long-lived
OAuth tokens and credentials are not placed in query strings, logs, local
storage, or replay files. The current server binds the ticket to its server
session, a 30-second expiry, and one use; the client pins the returned WSS URL
to the configured same authority. Tenant/workspace/audience/origin claims are
required before L2 deployment but are not present in the v1 ticket schema.
Deterministic replay is a first-class developer/demo mode. It is visibly marked
`SYNTHETIC` throughout the view, uses fixed seeds/fixtures, cannot update live
spatial memory, and cannot satisfy the research gate. Bounded exact-key
validation and same-authority ticket checks are implemented. Deployment CSP,
server-enforced origin policy, and reconnect backoff remain required hardening
before live promotion.
The web view visualizes plan/perspective geometry, covariance/quality,
freshness/expiry, modality/provenance, and disconnected/degraded state. It does
not call an undocumented WebKit or ARKit bridge.
Both clients accept at most 1,000 ms of future clock skew and a 5,000 ms
envelope lifetime. The web profile is intentionally stricter under receive
silence: it clears a frame after 1,500 ms without a replacement even when the
publisher supplied a longer TTL. Native clears at the signed envelope expiry.
This conservative display-liveness difference does not alter wire acceptance,
evidence level, or research metrics.
### 5. Keep pose/context fusion separate from evidence fusion
ARKit pose may help align the phone display or a separately calibrated external
sensor. That transform is valid only when extrinsic calibration, timestamps,
coordinate conventions, and uncertainty pass. Scene depth may display the
relay wall or visible geometry. It is not added to the optical NLOS likelihood
unless a future reviewed model defines and evaluates that factor.
If the phone and external sensor are not rigidly mounted, phone pose cannot be
treated as sensor pose without an independently measured time-varying transform.
The UI may still display both frames separately.
### 6. Re-evaluate Apple support through a documented gate
At each major iOS/Xcode intake, a reviewer searches Apple's official SDK headers,
documentation, entitlements, privacy manifest requirements, and App Store rules.
Direct Apple transient support advances only if all are true:
1. a public supported API exposes timing histogram/count data with documented
units, dimensions, timestamps, and device support;
2. use requires no private symbols, jailbreak, reverse engineering, or hidden
entitlement;
3. a physical-device capture proves that the data includes usable multipath;
4. privacy/security review and user disclosure pass; and
5. the same live reproduction/fusion protocol passes under a new adapter ADR.
No marketing article, simulator API, depth-map correlation, or successful
compile satisfies this gate.
## Performance decision
| Surface | Software target | Measurement note |
|---|---|---|
| Swift core | decode/validate without blocking the main actor; bounded memory | benchmark representative max-size track frames |
| Native rendering | newest-frame policy; 30 fps-capable presentation where device allows | rendering rate is not sensing rate |
| Web validation/store | bounded per-message parsing and no unbounded history | reject oversized messages before state update |
| Web rendering | responsive recent-track visualization and backoff under loss | requestAnimationFrame rate is not NLOS update rate |
| Transport | p50/p95 server-to-view latency, reconnects and dropped/stale counts | report separately from sensor-to-track latency |
UI optimization may decimate display history, but it cannot decimate or reorder
the evidence record. Performance tests use `SYNTHETIC` fixtures and are labeled
software evidence only.
## Security and privacy
1. NLOS tracking can reveal a person outside direct view. Native and web apps
require explicit consent, purpose text, a persistent indicator, pause/stop,
and clear source/provenance. Background capture is disabled by default.
2. App Transport Security/TLS and short-lived scoped authorization are required.
Debug cleartext/local exceptions are not release defaults.
3. WebSocket messages are untrusted. Enforce maximum message size, schema,
numeric bounds, sequence, freshness, origin, rate, tenant, and coordinate
frame before rendering or storage.
4. Replay files contain no credentials and no raw person/CSI/transient data by
default. Fixtures are synthetic or approved/de-identified and immutable.
5. Native and web telemetry excludes positions, raw sensor frames, tokens,
calibration secrets, and stable person identifiers. Diagnostics use bounded
counters and redacted errors.
6. The display is advisory. It cannot directly actuate a device or certify that
a hidden region is safe.
## Alternatives considered
### Make the first milestone a direct iPhone NLOS app
Rejected. It makes the research depend on an undocumented measurement. The
external sensor proves the architecture independently while iOS remains a
transport, context, and presentation adapter.
### Use only a native app
Rejected. A web/mobile view lowers review and demo friction, exercises the
versioned protocol, and can run deterministic fixtures. It still must not claim
direct sensor access.
### Use only a web app and skip native Swift
Rejected. ARKit pose/depth and physical-device capability checks require the
native SDK. The pure Swift core also gives an independent decoder implementation.
### Embed a permanent bearer token in the app or WebSocket URL
Rejected. Tokens leak through logs, browser history, proxies, crash reports,
and screenshots. Use short-lived, scoped, one-use tickets.
### Treat replay as a transparent fallback during disconnect
Rejected. It would misrepresent stale/synthetic state as live. Replay is an
explicit operator mode with persistent labeling and separate state.
## Consequences
### Positive
1. Native and web iOS deliver useful live/replay experiences without blocking
the core research on Apple's API choices.
2. The same schema, freshness, provenance, and coordinate rules apply across
Rust, Swift, and TypeScript.
3. Public ARKit pose and visible geometry remain valuable but honestly scoped.
4. Future Apple transient access has a precise, reviewable activation gate.
### Costs and limitations
1. The first live iOS experience needs an external histogram sensor and RuView
host; the phone alone is not the sensing system.
2. Cross-language contract tests and release CI add maintenance.
3. Web presentation depends on an authenticated RuView backend; offline mode is
replay only.
4. Simulator and Linux Swift tests cannot validate ARKit, LiDAR hardware,
App Store behavior, or a physical iOS build.
## Rollout and rollback
| Phase | Enablement | Rollback trigger | Action |
|---|---|---|---|
| I0 | pure Swift/TypeScript contract and deterministic fixtures | decoder disagreement, unbounded input, provenance drift | disable view; fix contract/golden vectors |
| I1 | future separately reviewed ARKit context on supported device | permission/session/transform failure | capability `unavailable`; clear context |
| I2 | authenticated external live tracks with valid, unexpired evidence/capability certificate and privacy approval | stale/replay/auth/tenant/evidence mismatch or approval withdrawal | disconnect; clear live state; retain explicit replay option |
| I3 | future Apple transient adapter | any of five activation gates absent/regressed | remove capability certificate and adapter flag |
The NLOS tab/app can be removed or disabled without affecting core RuView CSI,
Rust sensing, memory, MCP, or orchestration.
## Objective acceptance mapping
| ID | Requirement | Evidence |
|---|---|---|
| NLOS-330-01 | Swift core builds/tests without ARKit | `cd ui/ios-nlos && swift test` on supported Swift host |
| NLOS-330-02 | Native iOS scheme builds with public APIs | macOS `xcodebuild` simulator gate and physical-device smoke evidence, separately labeled |
| NLOS-330-03 | Current Apple capability probe, absent a new adapter ADR, never emits raw-transient/live-NLOS provenance | capability/provenance unit tests and source review |
| NLOS-330-04 | Web contract accepts valid `ruview.nlos.track.v1` and rejects malformed/stale/oversized input | Jest/TypeScript boundary tests |
| NLOS-330-05 | Live web transport uses short-lived authenticated tickets | client/server integration and replay/origin/expiry negative tests |
| NLOS-330-06 | Replay remains persistently `SYNTHETIC` and cannot pass live gate | UI/store tests plus harness research rejection |
| NLOS-330-07 | Missing Xcode/hardware or NLOS backend degrades honestly | advisory verifier skips and offline/disconnect UI tests |
| NLOS-330-08 | Web and native builds remain separable from core runtime | build matrix with NLOS surfaces disabled/absent |
| NLOS-330-09 | Future Apple activation uses only documented public access | official SDK/header diff, entitlement scan, privacy/App Store review, device witness, and separate adapter ADR |
## References
1. Apple, [`ARFrame.sceneDepth`](https://developer.apple.com/documentation/arkit/arframe/scenedepth).
2. Apple, [`ARFrame.smoothedSceneDepth`](https://developer.apple.com/documentation/arkit/arframe/smoothedscenedepth).
3. Apple, [`ARWorldTrackingConfiguration.sceneReconstruction`](https://developer.apple.com/documentation/arkit/arworldtrackingconfiguration/scenereconstruction).
4. Apple, [Displaying a point cloud using scene depth](https://developer.apple.com/documentation/arkit/displaying-a-point-cloud-using-scene-depth).
5. Somasundaram et al., [consumer-NLOS measurement model](https://arxiv.org/html/2605.17865v1).
6. STMicroelectronics, [VL53L8CH raw compact normalized histogram interface](https://www.st.com/en/imaging-and-photonics-solutions/vl53l8ch.html).

331
docs/adr/ADR-331-consumer-nlos-evidence-privacy-benchmark-metaharness.md Normal file
View File

@@ -0,0 +1,331 @@
# ADR-331: Consumer NLOS evidence levels, privacy/security, benchmarks, and optional MetaHarness governance
| Field | Decision |
|---|---|
| **Status** | Accepted; software gates are implemented independently of live-hardware evidence, research promotion remains blocked until the preregistered capture passes |
| **Date** | 2026-08-22 |
| **Owners** | RuView Labs maintainers, security/privacy reviewers, benchmark owners |
| **Scope** | Claim taxonomy, acceptance records, benchmark governance, contributor harness, release/promotion and rollback |
| **Depends on** | ADR-166, ADR-168, ADR-282, ADR-295, ADR-299, ADR-303 through ADR-305, ADR-318/319, ADR-328 through ADR-330 |
| **Implementation** | `harness/ruview`, `docs/security/consumer-nlos-threat-model.md`, `docs/research/consumer-nlos-acceptance-protocol.md` |
## Context
Consumer NLOS is unusually easy to overstate. A point-cloud animation can look
plausible even when driven by replay, stale state, calibration leakage, direct
line of sight, or a reflectivity regime unlike the deployment target. Native and
web builds can validate contracts without measuring one multipath photon. CSI
and LiDAR may also share temporal or labeling leakage, producing an apparent
fusion gain that contains no independent RF information.
The capability reveals presence and trajectories outside direct view, which
raises privacy and misuse risks despite not producing a conventional photograph.
It also has no present safety case for autonomous actuation. The project needs a
promotion gate that separates source integrity, software validation, controlled
laboratory evidence, field generalization, and production evidence.
The existing RuView contributor harness is dependency-free at runtime and uses
MetaHarness/Flywheel/Darwin only as development aids. NLOS verification should
reuse that posture: helpful, deterministic, and removable, never a requirement
for sensing, fusion, memory, routing, MCP, or normal runtime behavior.
## Decision
The `@ruvnet/ruview` packed-size ceiling increases from 160 KiB to 220 KiB to
carry the bounded NLOS contract verifier, research-gate evaluator, and operator
skill. The existing no-source-map, no-runtime-dependency, tarball smoke, and
claim-honesty gates remain mandatory. This is a reviewed capability budget, not
an unbounded exemption; the current dry-run package is approximately 203 KiB.
### 1. Keep three orthogonal labels
Every NLOS result records:
1. **Source provenance**: exactly one logical value for live hardware, replay,
or synthetic input; unknown is never coerced to live. The acceptance record
encodes these as `LIVE_HARDWARE`/`REPLAY`/`SYNTHETIC`, while the
`ruview.nlos.track.v1` wire contract uses `live`/`replay`/`synthetic`.
2. **Claim tag**: `MEASURED` with a named reproducer/manifest, `CLAIMED` with a
primary external source, or `SYNTHETIC` for simulation/fixtures. A build is
“validated software,” not a measured sensing result.
3. **ADR-282 evidence level**:
| Level | Meaning for NLOS |
|---|---|
| L0 | simulation or generated transient/track fixtures only |
| L1 | captured replay; deterministic pipeline behavior, not a fresh live result |
| L2 | controlled laboratory capture with external ground truth and frozen protocol |
| L3 | held-out room plus target/subject validation with leakage-resistant splits |
| L4 | multi-site field pilot under approved privacy/safety operations |
| L5 | production operational evidence, incident monitoring, drift and rollback history |
The upstream paper is cited as `CLAIMED` in RuView until reproduced. The first
RuView live acceptance can reach L2 only. No amount of L0/L1 replay volume
upgrades a capability to L2.
This ADR-282 maturity level is not the similarly named wire
`evidenceLevel` in `ruview.nlos.track.v1`. V1 accepts
`l0_synthetic`/`l1_measured`/`l2_calibrated`; `l3_corroborated` is reserved for
a future contract that can retain authenticated dual-modality lineage. These
values describe one envelope's source/calibration ceiling. They never self-promote
research maturity: a synthetic envelope is ADR L0, a captured replay remains
ADR L1 regardless of its historical wire label, and a live
`l2_calibrated` envelope reaches ADR L2 only through this
frozen external-ground-truth witness protocol. The acceptance JSON therefore
uses separate `claim_tag: MEASURED` and ADR maturity `evidence_level: L2`.
### 2. Separate software, research, and release gates
| Gate | What can pass it | What it proves | What it does not prove |
|---|---|---|---|
| Software | Rust/Swift/TypeScript unit, property, contract, build, replay and security tests | implementations compile and enforce declared invariants | photon capture, NLOS accuracy, physical iPhone support |
| Research | preregistered `LIVE_HARDWARE` capture with external ground truth | objective reproduction and fusion endpoints for named strata | field generalization, safety, identity, production readiness |
| Release/promotion | software + security + privacy + required evidence/certificate + human review | capability may be exposed at its exact evidence level | authority to actuate or claim a higher level |
An unavailable Xcode/hardware toolchain is an explicit `SKIPPED`, not a pass or
failure. A present partial/incompatible surface fails shallow discovery. If no
toolchain executes, the verifier reports `NO_BUILD_TOOLCHAINS_AVAILABLE`, never
a pass. Available-build success is only a subset of Gate A and is forbidden in
a hardware or release claim.
### 3. Preregister the reproduction and fusion endpoint
Before opening the test partition, freeze:
1. upstream commit and firmware/configuration;
2. sensor/CSI identity, calibration and coordinate transforms;
3. scene/target strata, exclusion rules and direct-line-of-sight checks;
4. randomized sequence order and grouped split manifest;
5. background capture and OOD/freshness thresholds;
6. track initialization, lost-track definition and maximum association gap;
7. primary metrics and bootstrap confidence interval procedure; and
8. all model/weight/threshold versions for LiDAR-only and fused arms.
The hard gate requested for this program is:
1. **Reproduction:** LiDAR-only produces at least 27 accepted end-to-end track
updates per second on live hardware, the objective definition of roughly
30 fps.
2. **Fusion:** over at least 100 paired live sequences, fusion achieves
\[
G_e = \frac{E_L-E_F}{E_L} \ge 0.25
\quad\text{or}\quad
G_\ell = \frac{\ell_L-\ell_F}{\ell_L} \ge 0.25,
\]
where \(E\) is the preregistered mean target-position error and \(\ell\) is the
preregistered lost-track rate. Both metrics are reported even if only one is
the success endpoint. Zero denominators do not count as improvement.
The protocol also reports confidence intervals, p95 position error, time to
first lock, false tracks in an empty hidden volume, update/latency distribution,
frame loss, calibration/OOD rejection, and performance per reflectivity, range,
motion, relay surface, room, and RF geometry. These secondary metrics prevent a
single successful aggregate from hiding unacceptable behavior.
### 4. Use a bounded, reviewable acceptance record
`ruview.nlos.acceptance.v1` is repository-contained JSON that references, but
does not embed, sensitive captures. Required fields include:
1. exact schema, `LIVE_HARDWARE` provenance, `MEASURED` claim tag, L2 evidence
level, and `EXTERNAL` ground truth;
2. protocol-frozen-before-capture flag;
3. enrolled external ST VL53L8-series sensor-model label for v1, verified raw/CNH transient kind, full
upstream SHA-1, and SHA-256 digests for protocol, firmware/API, combined and
CSI capture manifests, scoped enrolled identities, calibration and analysis;
4. zero synthetic frames and zero replay frames in the scored set;
5. independently verified CSI, a CSI-only ablation, at least one CSI source and
at least 100 paired sequences;
6. raw aggregate counts/durations/sums from which each arm's update rate, mean
position error, lost-track rate, offered optical rate and fused optical-frame
loss are recomputed; full paired-sequence position coverage and shared
endpoint denominators are mandatory; and
7. preregistered bootstrap seed digest, at least 10,000 resamples,
multiplicity-adjusted intervals, witness review, LOS exclusion, and passed
privacy/security review.
The advisory verifier validates an exact-key record, bounds, provenance,
digests, arithmetic, and thresholds; unknown fields fail so secrets or raw data
cannot silently hitchhike in the acceptance artifact. It cannot independently
prove that a digest corresponds to an honest physical experiment. Human reviewers inspect the immutable manifest,
external ground-truth synchronization, raw-capture access controls, exclusions,
and analysis reproducer before promotion.
### 5. Extend, but do not require, the RuView contributor harness
The dependency-free `@ruvnet/ruview` CLI/MCP registry adds governed advisory
surfaces. MCP/static inspection remains read-only. Local `--run-builds` is an
explicit execution mode, must target a trusted checkout, uses an allowlisted
child environment and redacted bounded tails, and is not a sandbox:
| Surface | Behavior |
|---|---|
| `ruview nlos plan` / `ruview_nlos_plan` | returns four staged phases, measurement invariant, expected surfaces and exit gates |
| `ruview nlos verify` | shallowly discovers expected Rust/native/web manifests/contracts, can explicitly run available local builds in a trusted checkout, and evaluates an optional live acceptance record |
| `ruview_nlos_verify` | read-only MCP inspection of the auto-detected repository and optional repository-confined evidence; rejects repository selection and build execution |
| `consumer-nlos` skill | contributor playbook for measurement boundary, reproduction, fusion, evidence, security, and iOS limitations |
The verifier discovers `v2/crates/ruview-nlos`, `ui/ios-nlos`, and `ui/mobile`.
If an optional surface is absent, it reports `ABSENT`. Once its feature marker
exists, missing, escaping, oversized or incompatible required artifacts are
`MALFORMED` and fail. String/manifest discovery does not replace contract tests.
Evidence files are regular, bounded, repository-confined JSON; path escape,
symlink escape, oversize, malformed JSON, replay, or synthetic input fails.
MetaHarness, Ruflo, Darwin, and Flywheel remain contributor tooling. Direct
crate/app builds without the harness are the evidence that runtime does not
require it; the verifier's status field is not proof by itself. Harness
proposals cannot modify sensing evidence, promote a
model, publish a package, merge code, or authorize hardware. Promotion retains
human review.
For future harness-policy evolution, a proposal is retained only if a frozen
holdout shows more than 2 percent quality lift, contributor cost regression is
below 1 percent, p95 latency regression is below 5 percent, security/legacy
tests do not regress, provenance is verified, and a human approves it. These
thresholds govern contributor tooling only; they do not replace the 25 percent
sensor-fusion endpoint.
### 6. Apply privacy-by-default controls
1. A named controller/operator records purpose, lawful/organizational basis,
approved spaces, experiment window, access list, retention, and deletion.
2. Visible notices and an active sensing indicator are required. Participants
can pause/withdraw where applicable. No hidden deployment is allowed.
3. Raw transient, CSI, ground-truth video/tag data, and joined trajectories are
P0/P1 research data: encrypted locally, access logged, minimized, separated
by tenant/experiment, and deleted on the frozen schedule. They are never
committed to Git or emitted in harness logs.
4. Default outputs are bounded session-scoped tracks with covariance,
provenance and expiry. No face image is needed, but “camera-free” is not
“privacy-free.” No biometric identity or cross-session re-identification is
claimed or enabled by default.
5. Dataset/model publication needs a separate disclosure/re-identification
review, consent/license check, and removal request path.
6. Safety and high-consequence use are out of scope. Tracks are advisory and
never direct actuation authority.
### 7. Make security a release blocker
The threat model in `docs/security/consumer-nlos-threat-model.md` is mandatory.
Release requires no confirmed high/critical finding across:
1. untrusted transient/track/parser boundaries and numeric/resource exhaustion;
2. sensor, session, calibration and capture identity/provenance;
3. replay/stale/future/duplicate and cross-tenant/world-frame attacks;
4. transport authorization, short-lived web tickets, TLS/origin and secret
handling;
5. firmware/upstream/npm/Cargo/Swift supply chain and license review;
6. raw research-data retention, logs, fixtures, telemetry and repository policy;
7. denial/degradation behavior and lack of direct actuation.
Findings are confirmed with focused tests before remediation claims. Automated
scanner output alone is not a confirmed vulnerability or a cleared release.
## Performance and optimization governance
Performance reports name hardware, OS/toolchain, configuration, capture digest,
commit, warmup, sample count, and statistics. Report sensor capture rate,
accepted frame rate, track update rate, sensor-to-track latency, service-to-view
latency, CPU, peak memory, queue drops, and output quality separately.
Optimization sequence:
1. profile parser, normalization, rendering/likelihood, resampling, temporal
join, serialization, and UI independently;
2. add scalar golden vectors and property tests before vectorization/caching;
3. bound buffers/particles/history and prefer newest-frame backpressure;
4. compare LiDAR-only and fused quality after every material optimization; and
5. roll back any change that violates numerical tolerance, security, privacy,
freshness, or the preregistered endpoint.
Configured 30 Hz, display 30/60 fps, and replay throughput are not equivalent
to live end-to-end tracking. Reports must name which one was measured.
## Alternatives considered
### Allow synthetic evidence to pass when hardware is unavailable
Rejected. Synthetic and replay are essential for software QA but cannot measure
the required optical path or independent RF gain.
### Treat the published MIT result as RuView's baseline evidence
Rejected. It is a primary `CLAIMED` reference with different hardware/scenes and
does not validate RuView's integration, privacy, security, or fusion.
### Make MetaHarness a required runtime coordinator
Rejected. Core sensing, fusion, memory, routing, MCP and UI must operate without
development orchestration. An advisory tool cannot be a physical trust anchor.
### Store all raw captures indefinitely for reproducibility
Rejected. Reproducibility uses controlled access, immutable digests, frozen
manifests and a retention schedule. Indefinite person/space/RF data creates
disproportionate risk.
### Promote on the 25 percent point estimate alone
Rejected. Pairing, confidence intervals, strata, secondary harms, calibration,
provenance, security, privacy and human review remain required.
## Consequences
### Positive
1. Reviewers can distinguish compile/replay success from physical NLOS evidence.
2. The 30 fps and 25 percent claims become executable, preregistered gates.
3. Privacy and security are part of experiment design, not a post hoc checklist.
4. The optional harness makes missing toolchains/surfaces explicit without
weakening the independent runtime.
### Costs and limitations
1. Controlled capture, external ground truth, review and data governance take
more time than a visual demo.
2. The acceptance JSON verifies integrity/arithmetic, not physical truth by
itself; witness review remains necessary.
3. L2 success does not establish L3–L5 generalization or safety.
4. A negative fusion result blocks promotion even if each modality is
individually interesting.
## Rollout and rollback
| Stage | Published capability | Required evidence | Rollback |
|---|---|---|---|
| G0 | development preview | L0 synthetic software fixtures, explicitly labeled | disable flag/view/tool; retain tests |
| G1 | captured replay demo | L1 capture manifest and privacy approval | revoke fixture/access; return to synthetic |
| G2 | controlled research result | L2 live protocol + acceptance + security/privacy review | invalidate certificate; remove live label; optical-only or unknown |
| G3+ | held-out/pilot/production | level-specific ADR-282 artifacts and monitoring | level downgrade, stop capture, delete per schedule, incident review |
Rollback preserves audit/witness records and never rewrites provenance. It may
disable NLOS/fusion independently while CSI and the rest of RuView continue.
## Objective acceptance mapping
| ID | Requirement | Evidence |
|---|---|---|
| NLOS-331-01 | Claim tag, provenance and L0–L5 level never alias | type/state tests, UI label tests, claim checker |
| NLOS-331-02 | Synthetic/replay cannot pass research | `evaluateResearchEvidence` negative tests and CLI nonzero with `--require-research-pass` |
| NLOS-331-03 | Reproduction is roughly 30 fps | live accepted update rate >=27 Hz, external-ground-truth manifest |
| NLOS-331-04 | Fusion has objective value | `MEASURED` in the capture-manifest witness report under the frozen protocol over >=100 paired sequences; >=25% mean-error **or** lost-track reduction with adjusted interval excluding zero, shared pairing digest, and all frozen guardrails passing |
| NLOS-331-05 | Harness remains optional | core/app tests without harness dependencies; absent-surface test |
| NLOS-331-06 | Present malformed surface/evidence fails closed | path/bounds/schema/digest/provenance and partial-surface tests |
| NLOS-331-07 | Privacy controls cover collection through deletion | approved experiment record, access/retention/deletion audit |
| NLOS-331-08 | No unaccepted high/critical security finding at release | threat-model review, focused regression tests and dependency/secret scans; `NLOS-SEC-EX-001` was closed by lockfile remediation without an exception, while future findings still require correction or an exact signed, unexpired exception record |
| NLOS-331-09 | Optimization does not trade away quality or freshness | golden/reference equivalence and named benchmark report |
| NLOS-331-10 | No hardware/App Store claim from CI alone | PR/release claim check and explicit physical-device witness field |
| NLOS-331-11 | Optional harness-policy evolution cannot self-promote or regress its frozen gates | `MEASURED` frozen held-out/anchor report proving >2% quality lift, <1% cost regression, <5% p95 latency regression, unchanged security/legacy gates, verified provenance and human approval; otherwise discard proposal |
## References
1. ADR-282: mandatory L0–L5 evidence ladder.
2. ADR-295: source provenance state machine.
3. ADR-303/304/305: ground-truth synchronization, evidence engine, authenticated sensor identity.
4. ADR-318/319: capability certificates and witness chain.
5. MIT, [Consumer NLOS project and 30 Hz demonstration](https://cornar.media.mit.edu/).
6. Somasundaram et al., [Nature paper](https://doi.org/10.1038/s41586-026-10502-x).
7. STMicroelectronics, [VL53L8CH raw histogram interface](https://www.st.com/en/imaging-and-photonics-solutions/vl53l8ch.html).