mirror of
https://github.com/ruvnet/RuView.git
synced 2026-08-31 12:36:09 +00:00
2754af804e
Replace the `Tensor::randn` stubs in occworld-candle's VQVAE encoder (`encode_occupancy`) and decoder (`decode_to_logits`) with a real, deterministic, input-dependent convolutional forward pass. Previously `predict()` emitted trajectory waypoints + confidence that were a function of RANDOM NOISE, independent of the input and silently presented as model output — the exact "AI slop" the project must eliminate. occworld-candle: - New `cnn.rs`: `Encoder2D` (3× Conv2d + GELU, interpolate2d to pin the token grid) and `Decoder2D` (upsample_nearest2d + Conv2d + 1×1 head). Both are deterministic functions of the input — same input → identical output; different input → different output. No randn in any forward path. - Deterministic weight init (`det_fill`, seeded xorshift64*) across all `dummy()` constructors (encoder/decoder, VQ codebook, quant-convs, transformer), so untrained engines are bit-for-bit reproducible. - `InferenceOutput.weights_trained: bool` — honest disclosure flag. `false` for `dummy()` (real but untrained net), `true` only after `load()` reads a real checkpoint. Priors are always from the real forward pass, never faked. - VQ codebook + quant/post-quant convs kept and wired encoder→VQ→decoder. - Centerpiece tests in `tests/predict_honesty.rs` (input-dependence, run-to-run + cross-engine determinism, untrained flag). All three FAIL on the old randn stub (verified by temporarily reinstating randn). pointcloud: - Optimize `to_gaussian_splats` hot path: 9 separate `.iter().sum()` passes per voxel → 2 fused accumulation passes. Bit-identical output. - `benches/splats_bench.rs` (criterion) measures old 9-pass vs new 2-pass with a parity guard. ~1.3× faster on representative cloud sizes. - Confirmed: no `randn`/placeholder in any claimed production path. The remaining synthetic generators (`send_test_frames`, `demo_depth_cloud`) and honestly-flagged heuristics (`heuristic_pose_from_amplitude`, luminance pseudo-depth fallback) are explicitly disclosed, not faked output. DATA-GATED: a trained checkpoint. An untrained-but-real net is the honest deliverable; accuracy is flagged via `weights_trained`, never claimed. Tests: occworld 16 unit + 3 integration + 2 doc, pointcloud 18 — all pass (CPU `Device::Cpu`; CUDA feature is GPU-gated and untouched). Co-Authored-By: claude-flow <ruv@ruv.net>
59 lines
2.7 KiB
Rust
59 lines
2.7 KiB
Rust
//! `wifi-densepose-occworld-candle` — OccWorld TransVQVAE inference in Candle.
|
|
//!
|
|
//! Ports the 72.4 M-parameter OccWorld world model (VQVAE tokeniser +
|
|
//! autoregressive transformer) from Python to native Rust using the
|
|
//! Hugging Face Candle framework. The goal is to eliminate the
|
|
//! 208 ms Python/IPC overhead of the existing `wifi-densepose-worldmodel`
|
|
//! bridge and enable tight integration with the streaming engine.
|
|
//!
|
|
//! ## Module structure
|
|
//!
|
|
//! | Module | Contents |
|
|
//! |-----------------|-------------------------------------------------------|
|
|
//! | `config` | `OccWorldConfig` — hyper-parameters |
|
|
//! | `error` | `OccWorldError` — unified error enum |
|
|
//! | `cnn` | Real conv `Encoder2D` / `Decoder2D` (deterministic) |
|
|
//! | `vqvae` | Class embedding, VQ codebook, quant convolutions |
|
|
//! | `transformer` | Autoregressive transformer (`PlanUAutoRegTransformer`) |
|
|
//! | `model` | SafeTensors weight loading + key mapping |
|
|
//! | `inference` | `OccWorldCandle` end-to-end inference engine |
|
|
//!
|
|
//! ## Implementation status
|
|
//!
|
|
//! The VQVAE encoder/decoder are a **real, deterministic, input-dependent**
|
|
//! convolutional forward pass (`crate::cnn`) — no `randn` anywhere in the
|
|
//! prediction path. All other components (class embedding, VQ codebook,
|
|
//! quant/post-quant convolutions, transformer, trajectory extraction) are
|
|
//! fully implemented. What remains **data-gated** is a *trained* checkpoint:
|
|
//! with `OccWorldCandle::dummy` the weights are deterministically initialised
|
|
//! but untrained, so the model is honest-but-unaccurate. This is surfaced via
|
|
//! [`InferenceOutput::weights_trained`] (`false` until `load` reads a real
|
|
//! checkpoint) — consumers must never treat untrained priors as trained.
|
|
//!
|
|
//! ## Usage
|
|
//!
|
|
//! ```no_run
|
|
//! use wifi_densepose_occworld_candle::inference::OccWorldCandle;
|
|
//! use wifi_densepose_occworld_candle::config::OccWorldConfig;
|
|
//! use candle_core::{Device, DType, Tensor};
|
|
//! use std::path::Path;
|
|
//!
|
|
//! let cfg = OccWorldConfig::default();
|
|
//! let engine = OccWorldCandle::dummy(cfg, Device::Cpu).expect("dummy init");
|
|
//! let past = Tensor::zeros((1, 15, 200, 200, 16), DType::U8, &Device::Cpu).unwrap();
|
|
//! let out = engine.predict(&past).expect("predict");
|
|
//! println!("predicted {} frames in {:.1} ms", out.sem_pred.dim(1).unwrap(), out.inference_ms);
|
|
//! ```
|
|
|
|
pub mod cnn;
|
|
pub mod config;
|
|
pub mod error;
|
|
pub mod inference;
|
|
pub mod model;
|
|
pub mod transformer;
|
|
pub mod vqvae;
|
|
|
|
pub use config::OccWorldConfig;
|
|
pub use error::OccWorldError;
|
|
pub use inference::{InferenceOutput, OccWorldCandle, TrajectoryWaypoint};
|