diff --git a/docs/adr/ADR-324-esp32-micro-llm-research-spike.md b/docs/adr/ADR-324-esp32-micro-llm-research-spike.md new file mode 100644 index 00000000..a6dd8685 --- /dev/null +++ b/docs/adr/ADR-324-esp32-micro-llm-research-spike.md @@ -0,0 +1,111 @@ +# ADR-324: ESP32-S3 micro-LLM inference — bounded research spike, production non-goal + +- **Status**: Proposed +- **Date**: 2026-08-16 +- **Deciders**: ruv +- **Owners**: RuView firmware and edge runtime maintainers +- **Tags**: esp32, llm, edge, research, evidence-labeling, firmware +- **Numbering note**: ADR-324 is the next free number in the authoring + checkout (highest existing: ADR-323; ADR-322 is an observed gap). Re-run the + ADR index/collision check immediately before merge and rename if needed. +- **Extends**: ADR-028, ADR-039, ADR-040, ADR-045, ADR-110, ADR-175, ADR-304 +- **Supersedes**: None +- **Companion research**: `docs/research/esp32-micro-llm-inference.md` +- **Companion ADR**: ADR-325 (flash-resident weight streaming) + +## Executive decision + +RuView will treat on-device micro-LLM inference on ESP32-S3 as a **bounded, +optional research spike on dedicated companion hardware**, and explicitly a +**non-goal for production CSI node firmware**. No LLM code, model weights, or +partition changes enter `firmware/esp32-csi-node/` under this ADR. Every +externally sourced performance number is `CLAIMED` until reproduced on RuView +silicon with a captured witness log. + +## Context + +Public demonstrations in 2026 (slvDev's `esp32-ai`, building on +`karpathy/llama2.c` and `DaveBben/esp32-llm`) show a 28.9 M-parameter +TinyStories transformer running at 9.5–9.88 tok/s (`CLAIMED`) on an ESP32-S3 +with 512 KB SRAM, 8 MB PSRAM, and 16 MB flash, using 4-bit quantization and +Gemma-3n-style Per-Layer Embeddings memory-mapped from flash. The companion +research document analyzes the techniques and coverage in depth. + +The question for RuView is whether any of this belongs on our sensing nodes. +The material facts are: + +1. **It does not fit our fleet.** Production nodes use the ADR-045 8 MB + partition map (2× 2 MB OTA + 1.875 MB SPIFFS) or the 4 MB variant. The + 14.9 MB model artifact cannot be stored on either (`SYNTHETIC`, partition + arithmetic). Replication requires a 16 MB-flash S3 SKU we do not deploy. +2. **It contends with the product.** The demos own both cores and run no + radio traffic during inference. RuView nodes dedicate core 0 to + WiFi + 20 Hz CSI capture and core 1 to the Tier 1–2 DSP pipeline and WASM + modules (ADR-039). LLM inference would degrade the primary sensing path. +3. **The model class has no task capability.** TinyStories-scale models + generate short fiction only — no instruction following, no factual recall. + Every on-node use case we examined is served better by deterministic + templates or is blocked on datasets that do not exist. +4. **The memory techniques are genuinely valuable** — for RuView's own edge + models, which is split out as ADR-325 so its fate is independent of the + LLM demo's. + +The rejected premise is: "an LLM now runs on our chip, therefore our product +should run an LLM." Feasibility of a demo is not fitness for a sensing +appliance. + +## Decision + +1. **Production non-goal.** `firmware/esp32-csi-node/` gains no LLM inference + path, no model partitions, and no LLM-derived output fields. Any future + reversal requires a new ADR with `MEASURED` evidence of zero CSI-path + regression. +2. **Bounded research spike (optional, unscheduled).** If funded, the spike + runs on a dedicated ESP32-S3 N16R8-class devkit that performs no CSI + capture, following the validation plan in the research document: + unmodified `esp32-ai` reproduction with witness log; WiFi-coexistence + penalty measurement; a ≤3.5 M-parameter fit-our-flash retrain solely to + characterize the quality cliff. +3. **Evidence labeling.** All numbers from external sources remain `CLAIMED` + in every RuView document, README, and communication until step 2 produces + `MEASURED` rows backed by a committed witness log (WITNESS-LOG discipline + per ADR-028). Builds and simulators are not hardware evidence. +4. **Trust boundaries.** Third-party inference code and weights undergo + license review at intake; model weights are never committed to this + repository; fetched artifacts are SHA-256 pinned. Micro-LLM output is + never admitted into the evidence engine (ADR-304), never labeled as + perception, and never enters a safety-relevant path (fall/presence + alerts). +5. **ESP32-C6 exclusion.** The spike targets S3 only; the single-core RISC-V + C6 without PIE SIMD is out of scope (ADR-110 unaffected). + +## Options considered + +- **Do nothing.** Cheapest, but discards transferable memory techniques and + leaves recurring "can we run an LLM on the nodes?" questions undocumented. + Rejected in favor of a written boundary. +- **Integrate into production firmware as a Tier 3.** Rejected: flash + impossibility on deployed SKUs, core/PSRAM contention with sensing, + no task capability at this model scale, safety-path hallucination risk. +- **Companion-hardware research spike (chosen).** Isolates risk, produces + `MEASURED` data, feeds ADR-325. +- **Server-side LLM only.** Remains the correct place for any real language + capability; this ADR does not change server architecture. + +## Consequences + +- Contributors get a citable "no" for production LLM integration and a + citable "yes, like this" for research. +- The spike, if run, costs one devkit and bounded engineering time, and + produces the repository's first `MEASURED` micro-LLM data. +- Risk accepted: the field moves fast; this ADR may need revisiting if + instruction-capable models reach ~10 M parameters. The evidence-labeling + and non-contention requirements would still hold. + +## References + +- `docs/research/esp32-micro-llm-inference.md` (sources, techniques, fit + analysis) +- ADR-028 (capability audit / witness discipline), ADR-039 (edge tiers), + ADR-045 (partition map), ADR-175 (measured quantization precedent), + ADR-304 (evidence engine), ADR-325 (companion decision) diff --git a/docs/adr/ADR-325-flash-resident-edge-model-weight-streaming.md b/docs/adr/ADR-325-flash-resident-edge-model-weight-streaming.md new file mode 100644 index 00000000..2b96bb10 --- /dev/null +++ b/docs/adr/ADR-325-flash-resident-edge-model-weight-streaming.md @@ -0,0 +1,111 @@ +# ADR-325: Flash-resident weight streaming for RuView edge models + +- **Status**: Proposed +- **Date**: 2026-08-16 +- **Deciders**: ruv +- **Owners**: RuView firmware and edge runtime maintainers +- **Tags**: esp32, memory, quantization, edge, inference, firmware, wasm +- **Numbering note**: ADR-325 follows ADR-324 in the authoring checkout. + Re-run the ADR index/collision check immediately before merge and rename if + needed. +- **Extends**: ADR-028, ADR-039, ADR-040, ADR-045, ADR-102, ADR-163, ADR-175 +- **Supersedes**: None +- **Companion research**: `docs/research/esp32-micro-llm-inference.md` +- **Companion ADR**: ADR-324 (micro-LLM research spike) + +## Executive decision + +RuView adopts a **design investigation** into flash-resident, memory-mapped, +quantized weight storage for its **own** edge inference models on ESP32-S3 — +the durable technique behind the 2026 micro-LLM demos — decoupled from any +language-model ambition. The target outcome is that future neural +presence/pose/vitals edge models are bounded by *flash* capacity rather than +by the ~7.36 MB of free PSRAM or ~280 KB of free SRAM, on the existing 8 MB +fleet SKU. This ADR authorizes design and benchmarking only; any partition +map or wire change requires its own follow-up ADR with `MEASURED` evidence. + +## Context + +Today RuView's on-node intelligence is DSP (ADR-039 Tiers 1–2) plus WASM3 +modules in 640 KB of PSRAM arenas (ADR-040/041), with neural inference done +server-side. If RuView ever ships a neural edge model (e.g. an on-node +presence classifier to cut the Tier 2 heuristic's false-positive rate), the +naive approach loads all weights into PSRAM/SRAM, capping model size and +competing with WASM arenas and CSI buffers. + +The `esp32-ai` demonstration (`CLAIMED`, see companion research) validated a +different memory contract on identical silicon: keep only activations and hot +weights in SRAM, the compute-dense core in PSRAM, and the bulk of parameters +4-bit-quantized in flash, accessed via the ESP32-S3 MMU's memory-mapped +read path with sparse, cache-friendly gathers (~450 bytes/token in the demo). +The same contract applies to non-LLM models: embedding/lookup layers, +frozen feature banks, and per-domain calibration tables are all +sparse-access, read-only structures. + +RuView precedents: ADR-175 established measured INT8 quantization discipline; +ADR-163 established edge latency measurement; ADR-028 established witness +evidence. This ADR composes them. + +## Decision + +1. **Adopt the memory contract as a design target** for future RuView edge + models on ESP32-S3: read-only quantized weight banks live in a dedicated + flash region and are memory-mapped, not copied; PSRAM holds compute-dense + weights and scratch; SRAM holds activations. INT8 remains the default + quantization (ADR-175 precedent); sub-byte formats require their own + measured accuracy gate. +2. **Fit the fleet, not the demo.** All designs must fit the ADR-045 8 MB + partition map's current free space (1.875 MB SPIFFS region or a future + dedicated `model` data partition of comparable size). Designs requiring a + 16 MB SKU are research-only under ADR-324. +3. **Benchmark before build.** The first deliverable is a microbenchmark + suite on real S3 silicon measuring: memory-mapped flash gather latency and + bandwidth (sequential vs. strided), ESP-DSP SIMD matmul throughput + against PSRAM- vs. flash-resident operands, and CSI-pipeline interference + (CSI packets/s at 20 Hz with the benchmark running on core 1). Results + are committed as `MEASURED` with witness logs; until then, all sizing in + this ADR is `SYNTHETIC`/`CLAIMED`. +4. **Integrity and provenance.** Any model bank flashed to a node carries a + SHA-256 recorded in NVS and reported in the node hello; unsigned or + mismatched banks are rejected at mount time. Model artifacts are never + committed to this repository; distribution follows the edge module + registry path (ADR-102) with the same review gates as WASM modules. +5. **Non-goals.** No production partition change, no wire-format change, no + model training, and no LLM integration under this ADR. WASM-visible + weight access (a `model_read` host API) is deferred until the benchmark + shows the flash path sustains it alongside CSI capture. + +## Options considered + +- **Status quo (all weights in RAM).** Simple, but caps any future edge model + at well under the free PSRAM after WASM arenas, and couples model growth to + RAM contention with the sensing pipeline. Rejected as the default. +- **Flash-resident streaming (chosen for investigation).** Raises the + capacity ceiling ~an order of magnitude on existing SKUs; costs MMU window + management and benchmark-verified bandwidth budgeting. +- **16 MB SKU migration.** Hardware cost and fleet churn for an unproven + need; remains available later and is orthogonal to this technique. +- **Keep all neural inference server-side.** Remains the default for heavy + models; this ADR only lowers the barrier for small on-node models where + latency/privacy justify them. + +## Consequences + +- Positive: future edge models are flash-bounded (~1.8 MB usable today, + `SYNTHETIC`) instead of RAM-bounded; techniques arrive pre-validated by a + measured benchmark rather than by external claims; provenance gating + extends the existing module-registry trust model to weights. +- Negative/risks: flash read contention with OTA and SPIFFS logging must be + measured, not assumed; MMU mapping bugs are a new failure class at the + hardware boundary (input validation at the mount path is mandatory); + benchmark effort is spent even if no neural edge model ever ships. +- The ADR-324 spike and this benchmark share kernels; either can proceed + without the other. + +## References + +- `docs/research/esp32-micro-llm-inference.md` (technique analysis, sources) +- ADR-028 (witness evidence), ADR-039 (edge tiers and core budget), ADR-040/ + ADR-041 (WASM sensing), ADR-045 (partition map), ADR-102 (edge module + registry), ADR-163 (edge latency measurement), ADR-175 (measured INT8 + quantization) diff --git a/docs/research/esp32-micro-llm-inference.md b/docs/research/esp32-micro-llm-inference.md new file mode 100644 index 00000000..d092512d --- /dev/null +++ b/docs/research/esp32-micro-llm-inference.md @@ -0,0 +1,217 @@ +# Deep research: micro-LLM inference on ESP32-S3-class microcontrollers + +- **Date**: 2026-08-16 +- **Status**: Research survey (no RuView hardware evidence yet) +- **Related ADRs**: ADR-324 (micro-LLM research spike), ADR-325 (flash-resident + weight streaming), ADR-028 (ESP32 capability audit), ADR-039 (edge + intelligence tiers), ADR-040 (WASM programmable sensing), ADR-045 (8 MB + partition table), ADR-110 (ESP32-C6 extension), ADR-175 (INT8 quantization, + measured) +- **Evidence policy**: Every performance or capability number in this document + is tagged `MEASURED` (reproduced on RuView hardware with a witness log), + `CLAIMED` (asserted by an external source, not reproduced by us), or + `SYNTHETIC` (derived arithmetic, no hardware run). As of this writing there + are **zero** `MEASURED` micro-LLM numbers in this repository. + +## 1. Executive summary + +In mid-2026 a developer known as slvDev (YouTube channel "The Stack") published +`esp32-ai`, a demonstration that a 28.9 M-parameter transformer trained on +TinyStories runs entirely on an ~$8 ESP32-S3 at roughly 9.5–9.88 tokens/s +(`CLAIMED`). The result was widely covered (Tom's Hardware, The Register, +CNX Software, Hackster, XDA, geeky-gadgets). The enabling ideas are: + +1. **Per-Layer Embeddings (PLE)**, borrowed from Google's Gemma 3n: ~25 M of + the 28.9 M parameters live in a memory-mapped lookup table in flash that is + *read*, not computed on — only ~450 bytes of table data are touched per + token (`CLAIMED`). +2. **4-bit quantization**, compressing the model to 14.9 MB so it fits a + 16 MB flash part (`CLAIMED`). +3. **Memory-hierarchy mapping**: activations and norm weights in ~512 KB SRAM, + the ~3.9 M-parameter dense core and output head in 8 MB PSRAM, the PLE + table memory-mapped in flash (`CLAIMED`). +4. **A modified `llama2.c` inference loop** with ESP32-S3 hardware-aware + optimizations, building on earlier work by DaveBben. + +This is a genuine engineering milestone — roughly 100× more parameters than +the prior ESP32 record (~260 K) — but the model is a toy: it generates short +children's stories and, per its own author, "will not answer questions, follow +instructions, write code, or know facts" (`CLAIMED`). + +**Bottom line for RuView**: the demo does not run on RuView production nodes +as-is (our fleet is 8 MB/4 MB flash; the demo needs 16 MB), it contends +directly with the CSI sensing workload for both cores and PSRAM, and a +TinyStories model has no useful task capability for RF perception. The +*techniques*, however — flash-resident memory-mapped quantized weights, PLE- +style table lookup, ESP-DSP SIMD kernels, dual-core scheduling — are directly +transferable to RuView's own edge models and are the durable value of this +research. ADR-324 and ADR-325 turn those two conclusions into decisions. + +## 2. Landscape and timeline + +| Year | Project | Model | Claimed throughput | Hardware | Notes | +|---|---|---|---|---|---| +| 2023 | `karpathy/llama2.c` | any Llama-2-architecture checkpoint | n/a (host) | PC | Single-file C inference engine; upstream of everything below | +| 2024 | `DaveBben/esp32-llm` | 260 K params (tinyllamas, TinyStories) | 19.13 tok/s (`CLAIMED`) | ESP32-S3FH4R2, 2 MB PSRAM | First notable ESP32 LLM port; ESP-DSP SIMD dot products, both cores, 240 MHz CPU, 80 MHz PSRAM, enlarged instruction cache | +| 2026 | `slvDev/esp32-ai` ("The Stack") | 28.9 M params, 4-bit, 14.9 MB, TinyStories | 9.5–9.88 tok/s end-to-end, ~94.9 ms/token compute (`CLAIMED`) | ESP32-S3, 512 KB SRAM, 8 MB PSRAM, **16 MB flash** | First known application of Gemma-3n-style PLE at this scale of hardware; MIT license; SHA-256-verified model fetch | +| 2026 | `wladimiravila/esp32s3-distributed-ai` | 56 M params split across 3 boards | not verified by us | 3× ESP32-S3 over ESP-NOW | "Split-PLE + KV cache, fully offline" (`CLAIMED`); shows the scaling direction | + +Context for scale: a 28.9 M-parameter model is ~4 orders of magnitude smaller +than frontier LLMs and ~2 orders smaller than the smallest generally useful +instruction-following models (~1–3 B). TinyStories (Eldan & Li, 2023, +arXiv:2305.07759) exists precisely because sub-100 M models can produce +coherent English only when the training distribution is radically constrained +(vocabulary of a young child, short narrative form). + +## 3. Technique deep dive + +### 3.1 Per-Layer Embeddings (PLE) / flash-resident lookup tables + +Gemma 3n introduced PLE to cut the *resident* memory footprint of a model: +a large share of parameters is restructured into per-layer embedding tables +that are gathered by token id rather than multiplied against activations. +Gathers are sparse — a handful of rows per token — so the table can live in +slow, cheap storage (here: memory-mapped SPI flash via the ESP32-S3 MMU) +without putting flash bandwidth on the critical path. `esp32-ai` claims only +~450 bytes of table reads per token (`CLAIMED`), which at ~10 tok/s is a +trivial ~4.5 KB/s of flash read traffic. + +Why this matters generally: it converts the dominant constraint from +*RAM capacity* to *flash capacity*, and flash is the cheapest memory on these +parts. The compute-active core shrinks to ~3.9 M parameters (~2 MB at 4-bit), +which fits comfortably in PSRAM. + +### 3.2 Quantization + +- 4-bit weight quantization: 28.9 M params → 14.9 MB total artifact + (`CLAIMED`). Sub-byte weights on Xtensa cost unpack instructions; the + scheme's viability at 9.5 tok/s suggests unpacking is amortized inside the + SIMD kernels. +- RuView precedent: ADR-175 measured INT8 quantization of a RuView pose model + and is our internal quality bar for how quantization claims must be + validated (accuracy delta on a held-out split, not vibes). + +### 3.3 Memory-hierarchy mapping (claimed layout of `esp32-ai`) + +| Tier | Size | Speed class | Contents | +|---|---|---|---| +| Internal SRAM | 512 KB | fastest | activations, normalization weights, hot scratch | +| Octal/Quad PSRAM | 8 MB | ~80 MHz SPI | dense transformer core (~3.9 M params) + output head, KV cache | +| SPI flash (memory-mapped) | 16 MB | slowest | 25 M-param PLE table (14.9 MB artifact total) | + +### 3.4 Compute optimizations (from `esp32-llm`, inherited by successors) + +- ESP-DSP dot-product kernels using the ESP32-S3's PIE 128-bit SIMD + extensions (the S3's differentiator vs. plain ESP32/C-series). +- Both Xtensa LX7 cores active during matmuls (second core normally runs the + WiFi stack — these demos do not run WiFi during inference). +- 240 MHz CPU, 80 MHz PSRAM clock, enlarged instruction cache. +- DaveBben's measured jump to 19.13 tok/s on a 260 K model (`CLAIMED`) came + primarily from SIMD + dual-core, an indication of how memory- and + compute-bound the naive loop is. + +### 3.5 What the model actually does + +TinyStories-class models emit grammatical, mostly-coherent short fiction. +They have no instruction following, no factual recall, no structured-output +reliability. Every secondary source and the author agree on this. Any plan +that assumes "small LLM = small ChatGPT" is wrong at this parameter scale. + +## 4. Fit analysis against RuView hardware and workload + +### 4.1 Flash: the demo does not fit our fleet + +RuView production partition table (ADR-045, `partitions_display.csv`): +8 MB flash = 2× 2 MB OTA app slots + 1.875 MB SPIFFS + NVS/PHY/otadata. +The 4 MB variant (issue #265) is tighter still. A 14.9 MB model artifact +**cannot be stored on either** — not in SPIFFS, not memory-mapped, not at +all (`SYNTHETIC`, from partition arithmetic). Direct replication requires a +16 MB-flash S3 variant (e.g. N16R8 modules), which is a *hardware SKU change*, +not a firmware change. + +Fitting inside the existing 1.875 MB SPIFFS would cap a 4-bit model near +~3.5 M total parameters (`SYNTHETIC`) — a regime where even TinyStories +coherence degrades sharply. + +### 4.2 RAM: feasible in isolation, contended in practice + +Current firmware budget (firmware README): ~35 KB SRAM used with ~280 KB SRAM +free; 640 KB PSRAM for WASM arenas with ~7.36 MB PSRAM free. A ~2 MB dense +core + KV cache would fit free PSRAM (`SYNTHETIC`). But: + +### 4.3 Compute and scheduling: direct conflict with the day job + +The published demos own **both cores** and disable radio work during +inference. RuView nodes run WiFi + 20 Hz CSI capture on core 0 and the +Tier 1–2 DSP pipeline (plus WASM3 modules) on core 1 (ADR-039). CSI capture +is the product; a micro-LLM competing for core 1 and PSRAM bandwidth would +degrade the primary sensing path. On-node LLM inference is therefore only +plausible as (a) a mutually exclusive duty mode, or (b) a dedicated companion +node that does no CSI capture. + +### 4.4 ESP32-C6 is out of scope + +The C6 (ADR-110 research target) is a single-core RISC-V without the S3's +PIE SIMD and typically without PSRAM — no published micro-LLM result exists +for it and the arithmetic is unfavorable. S3-only. + +### 4.5 Capability fit: what would a node even say? + +RuView's edge outputs are compact typed packets (32-byte vitals, ADR-039) and +server-side semantics. Candidate LLM uses on-node and their honest status: + +| Candidate use | Verdict | Why | +|---|---|---| +| Natural-language event narration ("someone fell in the kitchen") | Weak | A template engine does this deterministically in <1 KB; a TinyStories-class model adds hallucination risk to a safety-relevant path with zero benefit | +| On-device Q&A / configuration assistant | No | Requires instruction following; not available at this scale | +| Semantic compression of CSI events | Research-only | Would require a custom domain model trained on RuView event→text pairs; no dataset exists today | +| Demonstrating the platform's headroom / marketing | Real but bounded | Legitimate as a clearly labeled demo, never as a shipped claim | + +The durable value is the **memory architecture**, not the language model: +flash-resident memory-mapped quantized weights + PLE-style tables would let +RuView's *own* future neural presence/pose edge models grow well beyond +current RAM budgets on 8 MB parts (ADR-325). + +## 5. Risks and honest-labeling obligations + +- All throughput/size numbers above are `CLAIMED` until reproduced on RuView + silicon with a captured boot/runtime log (witness-log discipline, cf. + ADR-028/WITNESS-LOG-028). A successful build or QEMU run is not hardware + evidence. +- Third-party code and model weights (`esp32-ai` MIT; `llama2.c` MIT; + `esp32-llm` license to be verified at intake) require license review before + entering any RuView tree; model weights are unreviewed generated artifacts + and must not be committed (repository non-negotiables). +- Never present micro-LLM output as perception evidence. It must not enter + the evidence engine (ADR-304) or any published sensing claim. +- Marketing risk: "LLM on our sensor" invites camera-grade-style + overclaiming. Any public statement must carry the demo framing and + evidence tags. + +## 6. Recommended validation plan (if the ADR-324 spike is funded) + +1. Acquire an ESP32-S3 N16R8 (16 MB flash / 8 MB PSRAM) devkit — companion + hardware, not a fleet SKU. +2. Reproduce `esp32-ai` unmodified; capture serial witness log (boot, model + SHA-256, tok/s over ≥500 tokens). This produces the first `MEASURED` row. +3. Measure tok/s with WiFi stack active vs. disabled to quantify the + coexistence penalty. +4. Attempt a ≤3.5 M-parameter retrain fitting the 8 MB partition map, solely + to characterize the quality cliff (`SYNTHETIC` sizing above). +5. Separately benchmark the transferable kernels (ESP-DSP SIMD matmul, + memory-mapped flash weight streaming) against RuView's own edge-model + workloads — this feeds ADR-325 regardless of the LLM outcome. + +## 7. Sources + +- Geeky-Gadgets coverage (task prompt): https://www.geeky-gadgets.com/run-llm-esp32-microcontroller/ +- Project repo: https://github.com/slvDev/esp32-ai (MIT) +- Prior art: https://github.com/DaveBben/esp32-llm ; https://github.com/karpathy/llama2.c +- Distributed follow-on: https://github.com/wladimiravila/esp32s3-distributed-ai +- Tom's Hardware: https://www.tomshardware.com/tech-industry/artificial-intelligence/ai-developer-runs-28-9-million-parameter-model-on-usd10-esp32-s3-microcontroller-uses-googles-per-layer-embeddings-technique-stores-table-on-16mb-flash-memory +- The Register: https://www.theregister.com/edge-and-iot/2026/08/04/dev-proves-llms-will-run-on-anything-even-a-10-microcontroller/5283088 +- CNX Software: https://www.cnx-software.com/2026/08/03/28-9m-parameter-llm-runs-locally-on-esp32-s3-at-9-tokens-s/ +- Hackster: https://www.hackster.io/news/running-a-28-9m-parameter-llm-on-an-8-microcontroller-173f1f370708 +- XDA: https://www.xda-developers.com/someone-squeezed-a-289m-llm-onto-an-esp32-s3-and-so-can-you/ +- TinyStories: Eldan & Li 2023, arXiv:2305.07759 +- Gemma 3n / Per-Layer Embeddings: Google AI developer announcements, 2025