diff --git a/ALIGNMENT.md b/ALIGNMENT.md index d65d937..2672e8e 100644 --- a/ALIGNMENT.md +++ b/ALIGNMENT.md @@ -79,6 +79,8 @@ The original a-immersive was mobile-first ("designed for touch / foldable phone ## Recently resolved (delete after a few weeks) +- 2026-07-18: Browser curve maths unified onto the canonical `nisps/core/math.hpp` catalog at P4. The retired TS mirror had silently divergent maths for `exp`/`log` (k=4 vs the C++ k=1-normalised pair), `sigmoid` (slope 8 vs 6) and `cubic` (smoothstep vs x³) — browser-shaped params now behave firmware-exact. `linear/square/sqrt/centred-power` were already identical; the four changed curves were re-baselined in `manifold/tests/fixtures/curves-golden.json`. + - 2026-07-14: Defect "WASM MLP architecture is fixed" resolved by one-core-engine P2: `nisps/ml/` is storage-policied (`MLPCore`); the browser MLP is runtime-shaped (`DynamicStorage`), `nisps_ml_create` honours dims, `nisps_ml_reshape` warm-starts. Firmware keeps the zero-heap fixed template (`.text` +0.30%, within contract). - 2026-04-29: Three-implementation ML duplication (firmware `memlp`, `nisps-core`, JS engine) collapsed to single `nisps/` C++ codebase. diff --git a/MAP.md b/MAP.md index 2e1b94e..44d12e4 100644 --- a/MAP.md +++ b/MAP.md @@ -7,6 +7,7 @@ MEMLNaut-NISPS — Neural Interactive Shaping of Parameter Spaces. One C++20 cod ### `nisps/` — platform-agnostic C++20 library (the only ML/DSP/engine code) - `nisps/core/` — `perf.hpp` (memory section attrs), `types.hpp`, `concepts.hpp` (`MLEngine`, `AudioEngine`, `Mode`), `fixed_buffer.hpp`, `ring_buffer.hpp` (SPSC lock-free, replaces pico/util/queue), `rng.hpp` (xoshiro256+ deterministic), `math.hpp` (fast_sigmoid, `Curve` enum + `apply_curve`). - `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (SGD + grad clipping), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackControllerCore` — the "Down Action" state machine: Avoid (geometric push-away default / Diffuse legacy) / RandomiseOutputs / RandomiseMlp / ExploreAndPlace; storage-policied like the MLP, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `replay.hpp` (`ReplayView` — reward-tagged memory: dedup/deepen, k-NN positive centroid with deterministic tie-break, proportional decay+eviction), `geo_push.hpp` (push-away target computation, upstream InterfaceRL @ 0a541cc), `warm_start.hpp` (overlapping-weights copy for reshape), `stats.hpp`. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`. +- `nisps/pipeline/` — the control-rate input/output processing chains (P4): `input_chain.hpp` (`InputChain` — invert→deadzone→circular clamp→momentum-modulated zoom→centred power→EMA→momentum; caller-supplied dt, internal clock, fixed velocity ring, serialisable state) and `output_chain.hpp` (`OutputChain` — curve→EMA→slew→freeze(+mask), capacity-templated). Behaviour contract = the retired manifold TS pipelines, pinned by `manifold/tests/fixtures/` and parity stage 7. - `nisps/dsp/` — `biquad.hpp`, `delay.hpp`, `reverb.hpp`, `filter.hpp`, `env.hpp`, `osc.hpp`, `pitch_shift.hpp`, `dc_blocker.hpp`. Lean primitives extracted from maximilian; daisysp PitchShifter replaced with custom granular impl. - `nisps/engines/` — eight audio engines, each satisfying `AudioEngine`: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp` (sequencer, NoOp audio), `elysiamorf.hpp` (sequencer, NoOp audio), `analysis.hpp` (input-side spectral features). Plus `base.hpp` (`NoOpEngine`, engine_id "thru"). - `nisps/modes/` — platform-agnostic modes binding `{ML config, engine, voice space lambdas, abstract I/O channels}`. Files: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `slp_workshop.hpp` (`SLPWorkshopMode` — the Synth Library Portland workshop build; reuses the MEMLCelium engine + MLP shape, foregrounds the Jolt + OU explore gestures), `breakor.hpp`, `elysiamorf.hpp`, `sound_analysis_midi.hpp`, `external_synth_midi.hpp` (`ExternalSynthMIDIMode` — joystick→MLP→MIDI CC for an external synth; compile-time device from `nisps/midi`; `consteval pick_cc_slots` curates which params fill the NOut slots; NoOpEngine, `kRouteOutputsToEngine=false`). `base.hpp` provides a CRTP scaffold eliminating the duplication that previously plagued firmware modes. `voice_space.hpp` holds engine-side voice space dispatch helpers. `generated/` contains codegen output (do not edit by hand). @@ -35,7 +36,8 @@ alongside the live vanilla a-immersive at `/`). Built 2026-06-27/28; see `docs/s anchor + locked decisions) and the `docs/specs/*-spec.md` set. - `manifold/src/engine/` — the parity-tested TS engine LIFTED from `playground/src` (same `nisps.wasm`), made framework-neutral: `wasm-iml.ts` (rewired off Solid stores onto an injected `EngineSink`), `engine-host.ts` + - `worklet/nisps-processor.ts` (audio), `input-pipeline.ts`/`output-pipeline.ts`/`curves.ts`, `wasm-worker.ts`, + `worklet/nisps-processor.ts` (audio), thin WASM wrappers over the core pipelines + curve catalog (the TS + `input-pipeline`/`output-pipeline`/`curves` implementations died at P4), `wasm-worker.ts`, `spine.ts` (the reactive spine BELOW React — `setInput` derives processed→ml→routed eagerly off-render), `engine-api.ts` (`EngineApi` façade incl. `feedback.*` wrappers over the `nisps_ml_feedback_*` C ABI), `EngineProvider.tsx`/`useEngine.ts` (React binding via `useSyncExternalStore` version counter). nisps.js is @@ -141,7 +143,7 @@ the "BUILD DELTAS" block at the top of `docs/specs/vcv-module.md`). `src/MEMLNau - Firmware mode selection is compile-time only — `#define MEMLNAUT_MODE_TYPE` in the `.ino`. - `nisps/` follows Chris's RP2350 perf rules globally: no heap, `static const float` for non-trivial constants, strict `.f` suffix, memory section attrs (`NISPS_AUDIO_MEM`, `NISPS_AUDIO_FUNC`, `NISPS_APP_SRAM`, `NISPS_HOT`, `NISPS_FORCE_INLINE`). - C++ identifiers: `PascalCase` types, `snake_case` functions/variables, `kPascalCase` constexpr. JSON keys `snake_case`. TS types `PascalCase`, components `PascalCase.tsx`, modules `kebab-case.ts`. -- `Curve` enum lives in `nisps/core/math.hpp` (lowercase: `linear/exp/log/square/sqrt/sigmoid/cubic`); generated mode headers re-export via `using Curve = ::nisps::Curve;`. TS mirror at `manifold/src/engine/curves.ts` with same names (moves into core at P4). +- `Curve` enum lives in `nisps/core/math.hpp` (lowercase: `linear/exp/log/square/sqrt/sigmoid/cubic`, plus the parameterised `centered_power` free function); generated mode headers re-export via `using Curve = ::nisps::Curve;`. Since P4 there is NO TS mirror — the browser samples the WASM catalog (`nisps_curve_apply(+batch)`). - Modes are TSX components composed of primitives; mode parameter contracts are JSON schemas with codegen → C++ types (TS codegen returns at P5). **No declarative JSON UI.** - WASM and firmware share the same C++; the browser MLP is runtime-shaped (`MLPCore`, since P2): `nisps_ml_create` honours `(input, output, hidden[3])` with non-positive/null args defaulting to `32→[10,14,18]→126`; `nisps_ml_reshape` warm-starts a new shape. Firmware keeps compile-time `MLP<...>` (zero heap). Modes currently still use a slice of the default 126 outputs (per-mode dims become schema-real at P5). - Cross-platform parity: `scripts/parity-check.sh` enforces native vs WASM agreement within 1e-5. diff --git a/docs/AGENT-REFERENCE.md b/docs/AGENT-REFERENCE.md index 58b30d0..a2bae59 100644 --- a/docs/AGENT-REFERENCE.md +++ b/docs/AGENT-REFERENCE.md @@ -82,7 +82,7 @@ Build: `scripts/build-firmware.sh [VARIANT]`. Verified compiling for PAFSynth, C manifold/ # Vite + React + TypeScript (the sole browser app) ├── src/ │ ├── engine/ # framework-neutral TS engine spine: wasm-iml, engine-host + worklet, -│ │ # input/output pipelines + curves (move into core at P4), spine.ts, EngineProvider +│ │ # thin WASM wrappers over nisps/pipeline + the curve catalog (P4), spine.ts, EngineProvider │ ├── primitives/ # 12 design primitives as typed React │ ├── console/ # convertible Console (CompositeStage, Dock, Drawers, Manifold canvas, …) │ ├── dock/, backends/, inputs/, feedback/, settings/, serial/, midi-devices/ diff --git a/docs/specs/plans/one-core-engine-refactor.md b/docs/specs/plans/one-core-engine-refactor.md index 48bad35..9f52356 100644 --- a/docs/specs/plans/one-core-engine-refactor.md +++ b/docs/specs/plans/one-core-engine-refactor.md @@ -150,19 +150,21 @@ Each phase ends green on its test gate and is independently landable. File phase `10c3e55c`: the explore/place wiring is linker-GC'd out of the PAFSynth ELF (pre-existing, discovered during P3; the gestures are dead code on that build until fixed). -### P4 — Input/output pipelines + curves into core (≈3 days) +### P4 — Input/output pipelines + curves into core (≈3 days) — ✅ landed 2026-07-18 -- New `nisps/pipeline/`: input chain and output chain as plain structs satisfying the perf contract - (usable per-sample on firmware if ever wanted; per-pointer-event in browser — JS↔WASM call cost is - trivial at that rate). -- Curves: single catalog in `nisps/core/math.hpp` (already canonical); bindings expose - `nisps_curve_apply(id, x)` and batch variant; TS `curves.ts` (both copies) deleted, UI curve *previews* - render by sampling the WASM. -- Manifold `input-pipeline.ts` / `output-pipeline.ts` become thin calls into the main-thread WASM instance - (state lives C++-side, per-instance, serialisable for persistence). -- Golden TS-vs-C++ tests retire; replaced by direct use. -- **Gate:** spine e2e unchanged; recorded-gesture regression: same pointer trace → same routed output - pre/post migration (capture fixture before starting). +- ✅ New `nisps/pipeline/`: `input_chain.hpp` + `output_chain.hpp` (capacity-templated), perf-contract + structs, caller-supplied dt with an internal clock (no wall time — deterministic), serialisable state. +- ✅ Curves: `nisps/core/math.hpp` is the single catalog (+ parameterised `centered_power`); bindings expose + `nisps_curve_apply(id, x, param)` + batch. TS `curves.ts` deleted. NOTE: the TS mirror's `exp/log/ + sigmoid/cubic` maths silently diverged from the canonical catalog — the browser now behaves + firmware-exact (re-baselined in `curves-golden.json` with a provenance note; ALIGNMENT entry). +- ✅ Manifold pipelines are thin WASM calls (one `nisps_pipeline_create` handle per WasmIML; state C++-side; + config stays TS-side and is pushed over the 15-float wire layout). +- ✅ Golden TS-vs-C++ tests retired → the fixture suite drives the WASM directly. +- **Gate met:** spine e2e unchanged (27 e2e green); recorded-gesture regression against the P1 pre-migration + fixtures passes — non-momentum configs at ≤5e-7, momentum configs at 1e-2 with the drift PROVEN inherent + f32-vs-f64 (a byte-faithful f32 reference reproduces the WASM to <6e-8; documented in the test + fixtures + README). Parity stage 7 (v5): native↔WASM pipeline/curve floats bit-identical. ### P5 — Schema/codegen serves manifold (≈2 days)