memlnaut-nisps/MAP.md

184 lines
25 KiB
Markdown
Raw Normal View History

2026-04-15 17:32:32 +02:00
# MAP
MEMLNaut-NISPS — Neural Interactive Shaping of Parameter Spaces. One C++20 codebase (`nisps/`) compiles to two targets: (1) Arduino/RP2350 firmware for the MEMLNaut hardware, (2) WASM in the Manifold React browser app that runs the same engines + ML through an AudioWorklet. (The former SolidJS playground was retired 2026-07-13 — branch `archive/playground-solidjs`, tag `playground-solidjs-final`; the browser-only C15 engine lives only there for now.) See `CLAUDE.md` for the long-form architecture narrative and `ALIGNMENT.md` for current strategic gaps.
2026-04-15 17:32:32 +02:00
## Layout
### `nisps/` — platform-agnostic C++20 library (the only ML/DSP/engine code)
- `nisps/core/``perf.hpp` (hot-path/inlining attrs), `types.hpp`, `concepts.hpp` (`MLEngine`, `AudioEngine`, `Mode`), `ring_buffer.hpp` (SPSC lock-free cross-core channel, replaces pico/util/queue), `event_queue.hpp` (single-threaded in-engine event FIFO — deliberately NOT RingBuffer, which is an atomics-based cross-thread channel), `rng.hpp` (xoshiro256+ deterministic), `math.hpp` (fast_sigmoid, `Curve` enum + `apply_curve`).
refactor(ml): one runtime-configurable training default (S26) The operator's call: "there should be one default learning rate and one default max iterations and they should both be configurable at runtime." There were SIX copies, not the four the audit described, and they did not agree: nisps/ml/mlp.hpp no-arg train() hardcoding 1.f / 1000u / 0.001f — and firmware's ONLY training path calls exactly this, so firmware had no runtime knob at all wasm-iml.ts train() and trainAsync() TS default params (x2) engine-api.ts learningRate ?? 1.0, with no maxIterations knob vcv/src/iml.hpp 200 / 0.1 / 0.00001 — silently divergent external_synth_midi.hpp its own kDefaultLearningRate/kDefaultMaxIterations schemas/modes/*.json x9, identical, read by nobody at runtime Now: schemas/ml_defaults.json is the single declaration (validated against a sibling meta-schema, matching the midi_device.schema.json convention), codegen emits it to C++ and TS in the same run, and MLPCore carries a TrainConfig whose default member initialisers read the generated constant. set_train_config()/nisps_ml_set_train_config() make it runtime-overridable on every target; the explicit-argument train() overload is untouched. min_error joins the tuple — it was duplicated identically and belongs with the other two. The per-mode ml block loses default_learning_rate/default_max_iterations. default_spread stays (genuinely wired on both targets) and input_channels stays (codegen-time validated, real information for sound_analysis_midi). VCV BEHAVIOUR CHANGE, deliberate: MEMLNaut.cpp constructs IML positionally and relies on those defaults, so the module moves to 1000/1.0/0.001 — 5x the max iterations, 10x the learning rate, and a 100x looser early-stop threshold. The old values were never justified anywhere; they arrived with fbc68eb alongside an unrelated module rewrite and no tuning rationale. Firmware and WASM have shipped 1.0/1000 all along. It is now runtime-settable if this turns out worse. The generated header lands in nisps/ml/generated/, not nisps/modes/generated/ where the rest of codegen output lives: training hyperparameters are an ML fact, and nisps/ml sits below nisps/modes, so emitting them there would make mlp.hpp include upward. The agent that built this flagged the directory-crossing rather than hiding it; this is the fix. CI's generated-freshness gate learns the new directory. Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS (max delta 2.38e-7), lint clean, manifold typecheck + 17 unit + 33 e2e (which exercise train() and trainAsync() through a real browser).
2026-07-21 17:20:10 +02:00
- `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore<Storage>`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP<NIn,NH1,NH2,NH3,NOut>` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (SGD + grad clipping), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise<N>` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackControllerCore<FbStorage>` — the "Down Action" state machine: Avoid (geometric push-away default / Diffuse legacy) / RandomiseOutputs / RandomiseMlp / ExploreAndPlace; storage-policied like the MLP, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `replay.hpp` (`ReplayView` — reward-tagged memory: dedup/deepen, k-NN positive centroid with deterministic tie-break, proportional decay+eviction), `geo_push.hpp` (push-away target computation, upstream InterfaceRL @ 0a541cc), `warm_start.hpp` (overlapping-weights copy for reshape), `stats.hpp`. `generated/ml_defaults.hpp` is codegen output (do not edit): `nisps::ml::generated::kMlTrainDefaults`, the ONE learning-rate / max-iterations / min-error default shared by firmware, WASM and VCV (source `schemas/ml_defaults.json`); `MLPCore::set_train_config()` and `nisps_ml_set_train_config()` override it at runtime. It lives under `ml/` rather than `modes/generated/` because `nisps/ml` sits below `nisps/modes` — mlp.hpp must not include upward. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`.
- `nisps/pipeline/` — the control-rate input/output processing chains (P4): `input_chain.hpp` (`InputChain` — invert→deadzone→circular clamp→momentum-modulated zoom→centred power→EMA→momentum; caller-supplied dt, internal clock, fixed velocity ring, serialisable state) and `output_chain.hpp` (`OutputChain<NMax>` — curve→EMA→slew→freeze(+mask), capacity-templated). Behaviour contract = the retired manifold TS pipelines, pinned by `manifold/tests/fixtures/` and parity stage 7.
- `nisps/dsp/``biquad.hpp`, `delay.hpp`, `reverb.hpp`, `filter.hpp`, `env.hpp`, `osc.hpp`, `pitch_shift.hpp`, `dc_blocker.hpp`, plus the sequencer primitives shared by the sequencer engines: `ratio_seq.hpp` and `seq_clock.hpp` (bar phasor + MIDI clock + bpm). Lean primitives extracted from maximilian; daisysp PitchShifter replaced with custom granular impl.
- `nisps/engines/` — eight audio engines, each satisfying `AudioEngine`: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp` (sequencer, NoOp audio), `elysiamorf.hpp` (sequencer, NoOp audio), `analysis.hpp` (input-side spectral features). Plus `base.hpp` (`NoOpEngine`, engine_id "thru").
- `nisps/modes/` — platform-agnostic modes binding `{ML config, engine, voice space lambdas, abstract I/O channels}`. Files: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `slp_workshop.hpp` (`SLPWorkshopMode` — the Synth Library Portland workshop build; reuses the MEMLCelium engine + MLP shape, foregrounds the Jolt + OU explore gestures), `breakor.hpp`, `elysiamorf.hpp`, `sound_analysis_midi.hpp`, `external_synth_midi.hpp` (`ExternalSynthMIDIMode<const MidiDevice&, NOut>` — joystick→MLP→MIDI CC for an external synth; compile-time device from `nisps/midi`; `consteval pick_cc_slots` curates which params fill the NOut slots; NoOpEngine, `kRouteOutputsToEngine=false`). `base.hpp` provides a CRTP scaffold eliminating the duplication that previously plagued firmware modes. `generated/` contains codegen output (do not edit by hand): per-mode `k<Mode>Schema` ParamSchema instances, the `<Mode>MLP` type aliases built from the schema's own dims, and `schema_types.hpp` which now owns the `ParamSchema` struct itself. Mode headers no longer hand-write either their schema aggregate or their net shape.
- `nisps/wasm/bindings.cpp` — flat C API exported to WASM (Emscripten target only).
- `nisps/midi/generated/midi_devices.hpp` — codegen output: no-heap `constexpr` external-MIDI-synth templates (`nisps::midi::generated`; `MidiParam`/`MidiDevice` + `kMidiDevices` registry). Source = `schemas/midi_devices/`; do not edit by hand.
- `nisps/CMakeLists.txt` + `nisps/build/` — host-target builds + ctest.
build(firmware): migrate to PlatformIO and vendor memllib (plan §5) One cut, no dual path. Closes ALIGNMENT defect 3 ("Arduino-CLI build machinery is actively hostile") and vision bullet 4. platformio.ini carries 16 [env:], one per variant, each passing -DMEMLNAUT_MODE_TYPE; selftest passes -DNISPS_SELFTEST=1 instead. The env list IS the registry now — the .ino comment-registry and the NISPS_ST_* token-paste table are deleted rather than migrated. L12 noted that table was already silently missing the currently-shipped SLPWorkshop variant, which is the whole argument against having a second list. Also deleted: the Python/sed machinery that rewrote the COMMITTED .ino on every build, the sketch symlink forest, the global TFT_eSPI User_Setup.h mutation (now -D flags — TFT_eSPI's own documented PlatformIO recipe), the UF2 boot-mount detection stack (upload_protocol=picotool talks to the bootloader directly), and build-firmware-arch.sh entirely. Scripts 683 -> 435 lines. memllib is vendored at lib/memllib/ from upstream e291192; no submodules remain. VENDORED.md records provenance and the re-sync procedure. S9: a firmware-build CI job compiles three representative envs against a cached toolchain and reports per-variant flash/RAM. Firmware is in an automated gate for the FIRST time. The old ci.yml comment justified excluding it as "low verification value" — an assessment that did not survive contact, since the SelfTest variant sat broken for an unknown period calling a DisplayDriver method that did not exist at the pinned memllib commit, and nothing noticed because nothing built it. Verified: all 16 envs build from an empty cache, each within ~520 bytes of the arduino-cli binary it replaces, flash and RAM. Measured as .text+.rodata / .data+.bss+vector+uninitialized — NOT PlatformIO's console line, which double-counts .data on this board. This does not prove the hardware boots; no flash+smoke test was possible and that stays an operator chokepoint. slpworkshop 248232/145028 pafsynth 256880/149716 selftest 216228/17960 (all 16 in the CI log format; none exceeds 2% of a 16 MB flash) Two traps recorded so nobody rediscovers them: vendoring memllib's subdirs without a src/ wrapper makes PlatformIO's library builder silently compile NOTHING while still linking; and project build_flags land BEFORE the framework's own -std=gnu++17 -Os, so build_unflags is required. CORRECTION carried in this commit: the firmware sizes in c19d846's message and the first version of the memllib recon doc were wrong — SLPWorkshop 145348, PAFSynth 145300, SelfTest 141840. They came from building variants in sequence through a SHARED incremental arduino-cli build directory, which reused stale objects and under-reported by ~75 KB. Clean-cache rebuilds of the identical commit give 216736/18492 for SelfTest. The real cost of the memllib upstream bump is +216 bytes flash, not +316. Never measure firmware size through a reused build dir. HISTORY NOTE: this commit and the docs commit before it were rebuilt (force-push, 2026-07-21) so that each contains only what its message describes. The first versions had the firmware deletions stranded in the docs commit by a shared-index race between concurrent agents; content is byte-identical to the originals. Gates: run-all-tests.sh ALL GREEN (nisps/ untouched by this change beyond include paths); 16/16 pio envs build.
2026-07-21 20:17:58 +02:00
### `firmware/` — PlatformIO project + hardware glue
- `firmware/MEMLNaut-NISPS/platformio.ini`**the variant registry**: one `[env:<alias>]` per firmware variant (16 of them), each passing `-DMEMLNAUT_MODE_TYPE=<alias>`; `selftest` passes `-DNISPS_SELFTEST=1` instead. There is no second list to keep in sync. Shared `[env]` base pins the platform wrapper + arduino-pico framework, sets `-std=gnu++20 -O3` (via `build_unflags`, because the framework appends its own `-std=gnu++17 -Os` AFTER project flags), reaches `nisps/` with `-I${PROJECT_DIR}/../..`, and carries the TFT_eSPI panel config as `-D` flags. Build: `pio run -e <alias>`, or `scripts/build-firmware.sh [--all]`.
- `firmware/MEMLNaut-NISPS/src/main.cpp` — entry point (was `MEMLNaut-NISPS.ino`). Forks on `NISPS_SELFTEST`: normal modes run the engine/ML path; the `SelfTest` variant delegates all four entry points to `glue/selftest.hpp`.
Stream 6: extract firmware glue under firmware/ Move the Arduino sketch into firmware/MEMLNaut-NISPS/ and bridge the hardware (memllib) to the platform-agnostic nisps/ library through a slim glue layer. Delete the legacy root-level *AudioApp.hpp, modes/MEMLNautMode*.hpp, voicespaces/, IMLInterface.hpp, XiasriAnalysis, and the src/memlp submodule. Glue layout (firmware/MEMLNaut-NISPS/glue/): audio_driver.hpp - bridge memllib block callback to Mode::process via per-Mode templated trampoline (no virtual dispatch) peripherals.hpp - joystick/pots/buttons -> Mode::set_input + ML primitives midi_io.hpp - MIDI in -> mode.note_on/update_bpm/set_playing, drains mode ControlEvent ring -> MIDI UART mode_select.hpp - using-aliases mapping MEMLNautMode<Name> to nisps::modes::*Mode (build script rewrites the #define MEMLNAUT_MODE_TYPE line) input_router.hpp / output_router.hpp - top-level wire/drain entry points The sketch tree uses src/{memllib,daisysp,nisps} symlinks because Arduino-CLI rejects ".." in include paths from sketch-tree headers. mode_select.hpp #undefs Arduino's sq/min/max/abs/round macros before including nisps headers (some nisps engines use those identifiers as method names). The audio bridge struct is extern in the header and defined in the .ino because inline + __not_in_flash section attribute collide at link time. Verification: arduino-cli compile succeeds for PAFSynth, ChannelStrip, and BreakOr (rp2040:rp2040:solderparty_rp2350_stamp_xl:opt=Optimize3, -std=gnu++20). Host C++ tests under nisps/build still pass (3 binaries, 110+ tests). Build script (scripts/build-firmware.sh) updated to point at the new sketch path; mode-rewrite logic unchanged. Closes meml-gkm.
2026-04-29 16:05:38 +02:00
- `firmware/MEMLNaut-NISPS/glue/` — hardware bindings:
- `audio_driver.hpp` — bridges memllib `AudioDriver` callback → `Mode::process(stereosample_t)`.
feat(firmware): reposition gesture — relocate an existing example to a new input Add a "grab → move → drop" gesture that moves an existing positive example's output to a new input position, preserving the output. This is the new core's home for upstream InterfaceRL's drag-store/reposition-commit, distinct from Explore→Place (which places newly-auditioned scratchpad sounds). - nisps/ml/feedback.hpp: begin_reposition()/commit_reposition()/repositioning(). Reuses the Placing state (static_output holds the carried vector) but a reposition_ flag makes commit AND the mode-switch teardown SKIP the weight restore — the real net is never set aside here, so restoring snapshot_ would clobber the live trained net. Guards cancel_place + abort_explore_place. - firmware glue: state-gate Toggle B. Exploring → reroll/nudge (unchanged); Idle → MomB1 grab, MomB2 drop (commit + add_example + train). The 4D variant has no joystick button, so the gesture lives on the momentary toggle. Also fix a stale top-of-file control-map comment that contradicted the bindings. - tests: 4 reposition cases (hold without snapshot; commit stores carried output with no restore; mode-switch aborts without clobber; begin-only-Idle). - MAP.md: document the full ExploreAndPlace lifecycle + reposition wiring. Audio-hold (carrying the sound audibly during the move) remains the existing unwired static_output() TODO and affects Explore→Place identically. Firmware compile unverified (no arduino-cli); host tests + lint pass.
2026-06-28 23:44:57 +02:00
- `peripherals.hpp` — joystick / pots / buttons → `Mode::set_input` and ML primitives. Wires the shared `FeedbackController` ExploreAndPlace lifecycle (MomA1 = enter/exit explore, MomA2 = freeze/place, TogB2 = commit; MomB1/MomB2 = reroll/nudge while exploring **or** grab/drop *reposition* while idle) plus the adaptive-learning gestures: **TogB1** = Jolt (held weight morph), **RVX1** = exploration amount (OU output walk). Reposition relocates an existing positive example's output to a new input position (`feedback.hpp` `begin_reposition`/`commit_reposition`) — no scratchpad, no weight restore.
- `midi_io.hpp` — MIDI in → mode `note_on`/`update_bpm`/`set_playing`; drains `ControlEvent` ring → MIDI UART.
build(firmware): migrate to PlatformIO and vendor memllib (plan §5) One cut, no dual path. Closes ALIGNMENT defect 3 ("Arduino-CLI build machinery is actively hostile") and vision bullet 4. platformio.ini carries 16 [env:], one per variant, each passing -DMEMLNAUT_MODE_TYPE; selftest passes -DNISPS_SELFTEST=1 instead. The env list IS the registry now — the .ino comment-registry and the NISPS_ST_* token-paste table are deleted rather than migrated. L12 noted that table was already silently missing the currently-shipped SLPWorkshop variant, which is the whole argument against having a second list. Also deleted: the Python/sed machinery that rewrote the COMMITTED .ino on every build, the sketch symlink forest, the global TFT_eSPI User_Setup.h mutation (now -D flags — TFT_eSPI's own documented PlatformIO recipe), the UF2 boot-mount detection stack (upload_protocol=picotool talks to the bootloader directly), and build-firmware-arch.sh entirely. Scripts 683 -> 435 lines. memllib is vendored at lib/memllib/ from upstream e291192; no submodules remain. VENDORED.md records provenance and the re-sync procedure. S9: a firmware-build CI job compiles three representative envs against a cached toolchain and reports per-variant flash/RAM. Firmware is in an automated gate for the FIRST time. The old ci.yml comment justified excluding it as "low verification value" — an assessment that did not survive contact, since the SelfTest variant sat broken for an unknown period calling a DisplayDriver method that did not exist at the pinned memllib commit, and nothing noticed because nothing built it. Verified: all 16 envs build from an empty cache, each within ~520 bytes of the arduino-cli binary it replaces, flash and RAM. Measured as .text+.rodata / .data+.bss+vector+uninitialized — NOT PlatformIO's console line, which double-counts .data on this board. This does not prove the hardware boots; no flash+smoke test was possible and that stays an operator chokepoint. slpworkshop 248232/145028 pafsynth 256880/149716 selftest 216228/17960 (all 16 in the CI log format; none exceeds 2% of a 16 MB flash) Two traps recorded so nobody rediscovers them: vendoring memllib's subdirs without a src/ wrapper makes PlatformIO's library builder silently compile NOTHING while still linking; and project build_flags land BEFORE the framework's own -std=gnu++17 -Os, so build_unflags is required. CORRECTION carried in this commit: the firmware sizes in c19d846's message and the first version of the memllib recon doc were wrong — SLPWorkshop 145348, PAFSynth 145300, SelfTest 141840. They came from building variants in sequence through a SHARED incremental arduino-cli build directory, which reused stale objects and under-reported by ~75 KB. Clean-cache rebuilds of the identical commit give 216736/18492 for SelfTest. The real cost of the memllib upstream bump is +216 bytes flash, not +316. Never measure firmware size through a reused build dir. HISTORY NOTE: this commit and the docs commit before it were rebuilt (force-push, 2026-07-21) so that each contains only what its message describes. The first versions had the firmware deletions stranded in the docs commit by a shared-index race between concurrent agents; content is byte-identical to the originals. Gates: run-all-tests.sh ALL GREEN (nisps/ untouched by this change beyond include paths); 16/16 pio envs build.
2026-07-21 20:17:58 +02:00
- `mode_select.hpp` — type aliases mapping firmware mode identifiers to `nisps::modes::*Mode` C++ types, selected by the `-D` from platformio.ini. Includes the six `MEMLNautModeExtSynth*` external-synth variants (one per device template in `nisps/midi`, e.g. `MEMLNautModeExtSynthSub37`). The `NISPS_ST_*`/`NISPS_ST_CAT` token-paste table and the `SelfTestRig` tag type are GONE — selftest is now just an env with its own `-D`.
- `selftest.hpp` — standalone guided hardware self-test rig (`SelfTest` variant; no engine/ML). Step-driven state machine on a `SelfTestView`: TFT prompts the operator through every control, auto-advances on detection, encoder-press skips. Ends with optional L/R/BOTH sine-sweep headphone check (core 1 block callback) + MIDI loopback-cable test. Lives firmware-side (touches TFT + raw pins) so it stays out of platform-agnostic `nisps/`.
- `output_router.hpp` — top-level `drain_outputs()` entry point. (Inputs are wired directly by `peripherals.hpp`'s `bind_peripherals()`.)
- `settings_view.hpp``wire_settings(mode)`: adds on-device settings views to the MEMLNaut display carousel (TFT + rotary encoder). Joystick Dual/Single toggle for the 4-input ("two 2-D joystick") modes — "Single" pins ML input channels 2,3 to neutral via `ModeBase::set_input_pinned` (no net rebuild). Registered in the `.ino` after `addSystemInfoView()`.
build(firmware): migrate to PlatformIO and vendor memllib (plan §5) One cut, no dual path. Closes ALIGNMENT defect 3 ("Arduino-CLI build machinery is actively hostile") and vision bullet 4. platformio.ini carries 16 [env:], one per variant, each passing -DMEMLNAUT_MODE_TYPE; selftest passes -DNISPS_SELFTEST=1 instead. The env list IS the registry now — the .ino comment-registry and the NISPS_ST_* token-paste table are deleted rather than migrated. L12 noted that table was already silently missing the currently-shipped SLPWorkshop variant, which is the whole argument against having a second list. Also deleted: the Python/sed machinery that rewrote the COMMITTED .ino on every build, the sketch symlink forest, the global TFT_eSPI User_Setup.h mutation (now -D flags — TFT_eSPI's own documented PlatformIO recipe), the UF2 boot-mount detection stack (upload_protocol=picotool talks to the bootloader directly), and build-firmware-arch.sh entirely. Scripts 683 -> 435 lines. memllib is vendored at lib/memllib/ from upstream e291192; no submodules remain. VENDORED.md records provenance and the re-sync procedure. S9: a firmware-build CI job compiles three representative envs against a cached toolchain and reports per-variant flash/RAM. Firmware is in an automated gate for the FIRST time. The old ci.yml comment justified excluding it as "low verification value" — an assessment that did not survive contact, since the SelfTest variant sat broken for an unknown period calling a DisplayDriver method that did not exist at the pinned memllib commit, and nothing noticed because nothing built it. Verified: all 16 envs build from an empty cache, each within ~520 bytes of the arduino-cli binary it replaces, flash and RAM. Measured as .text+.rodata / .data+.bss+vector+uninitialized — NOT PlatformIO's console line, which double-counts .data on this board. This does not prove the hardware boots; no flash+smoke test was possible and that stays an operator chokepoint. slpworkshop 248232/145028 pafsynth 256880/149716 selftest 216228/17960 (all 16 in the CI log format; none exceeds 2% of a 16 MB flash) Two traps recorded so nobody rediscovers them: vendoring memllib's subdirs without a src/ wrapper makes PlatformIO's library builder silently compile NOTHING while still linking; and project build_flags land BEFORE the framework's own -std=gnu++17 -Os, so build_unflags is required. CORRECTION carried in this commit: the firmware sizes in c19d846's message and the first version of the memllib recon doc were wrong — SLPWorkshop 145348, PAFSynth 145300, SelfTest 141840. They came from building variants in sequence through a SHARED incremental arduino-cli build directory, which reused stale objects and under-reported by ~75 KB. Clean-cache rebuilds of the identical commit give 216736/18492 for SelfTest. The real cost of the memllib upstream bump is +216 bytes flash, not +316. Never measure firmware size through a reused build dir. HISTORY NOTE: this commit and the docs commit before it were rebuilt (force-push, 2026-07-21) so that each contains only what its message describes. The first versions had the firmware deletions stranded in the docs commit by a shared-index race between concurrent agents; content is byte-identical to the originals. Gates: run-all-tests.sh ALL GREEN (nisps/ untouched by this change beyond include paths); 16/16 pio envs build.
2026-07-21 20:17:58 +02:00
- `firmware/MEMLNaut-NISPS/lib/memllib/`**vendored** memllib (was the `src/memllib` submodule): hardware abstraction (audio driver, TFT display, MIDI, peripherals), ~1.9 MB / 100 files, `examples/` dropped. `VENDORED.md` records the upstream commit and the re-sync procedure; `LICENSE` is MPL-2.0, copied verbatim. **Sources must sit under `lib/memllib/src/`** — PlatformIO's library builder falls back to a flat root-only scan without it and silently compiles nothing while still linking (see VENDORED.md).
- `firmware/README.md` — structure + build instructions.
- `firmware/useq-celium/` — standalone RP2040 firmware (PlatformIO, Arduino-Pico core) that turns a uSEQ module + CV expander into a USB→CV/gate converter driven by the manifold `cvgate` backend. `shared/protocol.h` is the v2 wire-protocol single source of truth (mirrored by `manifold/src/backends/useq-protocol.ts`); `main/` (USB serial → CV13 + GATE13, I2C → expander) and `expander/` (I2C slave → CV411). Wire spec: `docs/specs/useq-cv-protocol.md`. Restored from the April-2026 "uSEQ-Celium" mode.
### `manifold/` — Vite + React + TS convertible-mode app (the sole browser app)
The Manifold "convertible" Console on the real engine, deployed at `meml.lnfinitemonkeys.org/next` (staging,
alongside the live vanilla a-immersive at `/`). Built 2026-06-27/28; see `docs/specs/plans/BUILD-PLAN.md` (resume
anchor + locked decisions) and the `docs/specs/*-spec.md` set.
- `manifold/src/engine/` — the parity-tested TS engine (same `nisps.wasm`), made
framework-neutral: `wasm-iml.ts` (rewired off Solid stores onto an injected `EngineSink`), `engine-host.ts` +
`worklet/nisps-processor.ts` (audio), thin WASM wrappers over the core pipelines + curve catalog (the TS
`input-pipeline`/`output-pipeline`/`curves` implementations died at P4), `wasm-worker.ts`,
`spine.ts` (the reactive spine BELOW React — `setInput` derives processed→ml→routed eagerly off-render),
`engine-api.ts` (`EngineApi` façade incl. `feedback.*` wrappers over the `nisps_ml_feedback_*` C ABI),
`EngineProvider.tsx`/`useEngine.ts` (React binding via `useSyncExternalStore` version counter). nisps.js is
loaded via fetch+indirect-eval (Emscripten MODULARIZE glue has no ES exports), base-aware via `document.baseURI`
for the `/next` sub-path.
- `manifold/src/primitives/` — the 7 design primitives as typed React (Badge, Button, PillToggle, Slider, Switch, VirtualJoystick, XYPad). Five unused ones were deleted in the 2026-07 sweep (L22).
- `manifold/src/console/` — the convertible Console: `ConsoleApp`, `CompositeStage` (single-divider convertible
with snap/magnetism/minimap-demotion), `OutputStage`/`SandwichStage`/`ParticleStage`/`Manifold` (canvas,
rect↔circular + feedback markers), `Dock` (top Mode selector + 5 vertically-centred drawers), `Drawers`
(Learning/Inputs/Outputs/Settings/Help), `VerdictCluster` (mode-aware), `OutputEditor`/`CurvePad`, `icons.tsx`
(monochrome currentColor SVG), `model.ts` (`MF_MODES` catalogue — schema-backed modes DERIVED from
`manifold/src/modes/generated/`; carries per-mode `ml` net shape + `engineId`), `output-mode.ts`.
- `manifold/src/modes/generated/` — codegen output (`*_schema.ts`, do NOT hand-edit): `ModeSchema`
consts (mode_id, engine_id, ml dims, params, voice_spaces, ui) — the SOURCE OF TRUTH for `MF_MODES`.
Switching mode reshapes the WASM net to the mode's `ml` dims (ConsoleApp P5.3; boot mode paf_synth →
4→[10,10,14]→33).
- `manifold/src/dock/``OutputControlRow` (off/fixed/live + mute + solo/arm + min/max/curve), `output-state.ts`,
`OutputsBackendConfig.tsx` (per-backend specialised Outputs panel — the sole per-backend editor).
- `manifold/src/backends/``OutputBackend` adapter + `BackendManager` (spine consumer); `midi-backend.ts`
(WebMIDI), `osc-backend.ts`+`osc-client.ts` (OSC-over-WS), `vcv-backend.ts` (VCV-over-WS), `cv-backend.ts`
(`UseqCvBackend` — uSEQ CV/gate over USB Web Serial, backend id `cvgate`) + `useq-protocol.ts` (v2 wire
protocol, mirrors `firmware/useq-celium/shared/protocol.h`; `useq-protocol.test.ts` runs via `bun test`),
`particle-backend.ts`, `passthrough-backend.ts`, `presets.ts` (named presets), `manager.ts`.
- `manifold/src/midi-devices/` — external-synth device templates. `generated/` is codegen output from
`schemas/midi_devices/` (`MIDI_DEVICES` catalogue + `MIDI_DEVICES_BY_ID`, params by name+CC). The MIDI Outputs
config (`dock/OutputsBackendConfig.tsx`) reads it for the device picker + param-select that fills the CC table.
feat(manifold): MIDI + game controller inputs; widen ML net to N-D Wire the modular input layer into the Console and reshape the browser engine so input axes are genuine independent dimensions. Inputs (manifold/src/inputs/): - gamepad-source: emit press+release edges with standard-mapping labels (enables hold-and-move); single/double-stick already present. - midi-input-source: single-device selection + batch "MIDI Learn" (every CC swept while armed becomes an axis); notes stay discrete. - input-layer: compose() forwards each axis 1:1 (no mean-blend); add onReducedInput so the manifold tracks gamepad/MIDI position. - types: InputAction.phase, InputMode. Console (manifold/src/console/): - ConsoleApp: bind gamepad buttons to verdicts (RB up / LB down / X randomise / Y nudge / B undo / A-hold reposition); mirror composed position onto the manifold. - Drawers: rebuilt Inputs drawer (source picker, gamepad legend, MIDI device picker + batch-learn flow, learned-control meters). Engine (nisps/wasm, manifold/src/engine): - DefaultMLP widened MLP<2,..> -> MLP<32,..> (32 = MAX_AXES); each active axis gets a dedicated slot, unused slots held at 0 (inert). Rebuilt nisps.wasm (playground + manifold). - spine/engine-api: setInputs writes the full N-D vector (was dropping arr[2+]); primary pair keeps the 2-D pipeline; process() re-ticks the whole vector via spine.reprocess(). Tests: - parity_check/parity_wasm: ParityMLP -> 32 inputs, widen example bufs. - CMakeLists: build parity binary with -ffp-contract=off so native matches FMA-free WASM (training amplified the gap past 1e-5). Inputs dock is still an exclusive picker; mixing toggles, reshape modal, and the >2-D slider view (inputs-spec.md) are groundwork-laid but not yet wired. See docs/redesign/midi-gamepad-inputs-worklog.md.
2026-06-28 21:05:30 +02:00
- `manifold/src/inputs/` — modular INPUT layer feeding the ML head. The Inputs dock picks ONE exclusive mode
(`InputMode` = `internal` | `gamepad` | `midi`; Internal/XY-pad is default). `input-layer.ts` owns a single rAF
docs: the specs disposition pass (plan §8) Roughly 20k lines were deleted from this repo in the last week and much of the corpus still described the pre-deletion world in the present tense. Executes the §8 table: archive the retired, reclassify the executed, prune the stale. aimmersive-clone-spec -> _archive/ with a deprecated-by note feedback-modes-port-spec -> plans/, kind: plan, status: executed manifold-parity-features -> plans/, kind: plan, status: active playground-2.0-rewrite -> status: superseded engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten present-tense against the shipped engine/ MAIN.md six contradicted claims fixed; registry resynced vcv-module.md pruned to the current 8->16 contract and made the single .nisps format spec vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table inputs/backends/dock trio grounding sections marked historical, dead cites fixed Two rows of the §8 table were themselves wrong, corrected here: the deleted full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no §6.3), and codegen/README.md was already a MAP pointer with no port-solidjs trigger left to remove. Beyond the table — found by sweeping every backticked path in the changed docs against `git ls-files`, which is how these should have been caught before: manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current: SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a shot.spec.ts that does not exist. The whole stage table was keyed on a `focus` axis that no longer exists — selection is now sandwich > particles > composite. This matters more than the rest: CLAUDE.md tells every agent to read ONBOARDING.md first for Manifold work, so it was actively teaching a fiction. Rewritten against ConsoleApp.tsx. MAP.md claimed the input layer reduces axes to the engine arity with an "even/odd blend". input-layer.ts says the opposite in its own header: one dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and mean-blending was removed deliberately because it diluted every source. AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed), per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at nisps::FixedBuffer (deleted). Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/ Delta and cachedNovelty behind a live context-menu toggle that nothing reads; vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to 9000, so out of the box they do not meet. Firmware-build docs are deliberately untouched — the PlatformIO migration lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
loop composing the active source's axes → **one dedicated engine input slot per axis, 1:1, no blending** → one
`setInputs`, plus an `onReducedInput` callback the manifold tracks. The WASM net is over-provisioned to a
32-input head (`MAX_AXES`, `nisps/wasm/bindings.cpp`); unused slots are zero-padded and a zero input is inert,
so idle sources cannot perturb the net. Mean-blending was removed deliberately — it diluted every source and
biased the net toward idle sources' resting values. Changing the ACTIVE axis count offers a reshape
(`ConsoleApp` → `ReshapeModal`): new net at the new arity, warm-started from overlapping weights, examples and
feedback state reset; declining keeps the over-provisioned head. Sources: `xy-pad-source` (push-driven),
feat(manifold): MIDI + game controller inputs; widen ML net to N-D Wire the modular input layer into the Console and reshape the browser engine so input axes are genuine independent dimensions. Inputs (manifold/src/inputs/): - gamepad-source: emit press+release edges with standard-mapping labels (enables hold-and-move); single/double-stick already present. - midi-input-source: single-device selection + batch "MIDI Learn" (every CC swept while armed becomes an axis); notes stay discrete. - input-layer: compose() forwards each axis 1:1 (no mean-blend); add onReducedInput so the manifold tracks gamepad/MIDI position. - types: InputAction.phase, InputMode. Console (manifold/src/console/): - ConsoleApp: bind gamepad buttons to verdicts (RB up / LB down / X randomise / Y nudge / B undo / A-hold reposition); mirror composed position onto the manifold. - Drawers: rebuilt Inputs drawer (source picker, gamepad legend, MIDI device picker + batch-learn flow, learned-control meters). Engine (nisps/wasm, manifold/src/engine): - DefaultMLP widened MLP<2,..> -> MLP<32,..> (32 = MAX_AXES); each active axis gets a dedicated slot, unused slots held at 0 (inert). Rebuilt nisps.wasm (playground + manifold). - spine/engine-api: setInputs writes the full N-D vector (was dropping arr[2+]); primary pair keeps the 2-D pipeline; process() re-ticks the whole vector via spine.reprocess(). Tests: - parity_check/parity_wasm: ParityMLP -> 32 inputs, widen example bufs. - CMakeLists: build parity binary with -ffp-contract=off so native matches FMA-free WASM (training amplified the gap past 1e-5). Inputs dock is still an exclusive picker; mixing toggles, reshape modal, and the >2-D slider view (inputs-spec.md) are groundwork-laid but not yet wired. See docs/redesign/midi-gamepad-inputs-worklog.md.
2026-06-28 21:05:30 +02:00
`gamepad-source` (sticks→axes single/double; buttons emit press+release actions, bound in `ConsoleApp` to
verdicts — LB/RB=down/up, X/Y/B=randomise/nudge/undo, A-hold=reposition), `midi-input-source` (device picker +
BATCH "MIDI Learn": every CC swept while armed becomes an axis, shown as read-only meters). `useInputLayer.ts`
is the React binding; `base-source.ts` shared status/action plumbing; `types.ts` the adapter contract.
`backends/base-backend.ts` is its output-side counterpart (status + throttle + lastSent) used by the midi/osc/vcv transports.
- `manifold/src/feedback/``controller.ts` (Explore-and-place scratchpad + geometric-dislike + solo; a thin
driver over the shared C++ core).
- `manifold/src/settings/``settings-store.ts` (monochrome icons, input-map shape, corner radius).
- `manifold/src/serial/``memlnaut-serial.ts` Web Serial scaffold + `EditorPanel.tsx` (MEMLNaut Editor mode).
- `manifold/src/engine/exploration.ts` — Jolt press + OU explore gestures (Learning drawer): a thin
timer-driver over the shared C++ core via the `nisps_ml_jolt_*`/`nisps_ml_ou_*` bindings (the interim
TS math and `jolt.ts`/`ou-explore.ts` were deleted when P3 landed).
- `manifold/src/debug/probe.ts``window.__nisps` (`?debug=1`). `manifold/tests/e2e/``smoke`,
`probe-api` (15-test engine-contract port), `spine` (spine invariant + probe-survives-mode-switch).
E2E on the VPS runs via non-snap node (see BUILD-PLAN). `manifold/tests/fixtures/` — golden parity
fixtures (gesture trace, curves, input/output pipelines) captured 2026-07-13 pre-P4, guarded by
`tests/pipeline-golden.test.ts` (in `bun run test`). `manifold/osc-bridge/` — Deno WS↔UDP-OSC bridge.
### `vcv/` — VCV Rack 2 plugin (MEMLNaut module, WIP)
Native C++ Rack module: ML CV-mapper with RL feedback + a browser bridge. **8 inputs × 16 outputs + per-output
LED rings**, palette from the frontend tokens, WS↔OSC browser bridge (see `docs/specs/vcv-module.md`).
`src/MEMLNaut.cpp` (module, 8→[16,24,16]→16), `src/iml.hpp` (**thin adapter over `nisps::ml::MLPCore<DynamicStorage>`
+ core `nisps::Rng`** — P6 reunification 2026-07-18, closes vcv-module.md delta #5; behaviour is now core-exact,
pinned by `tests/cpp/test_vcv_iml_parity.cpp`), `src/osc_server.hpp` (bridge, transport-only), `src/plugin.{hpp,cpp}`,
`res/*.svg` (panels), `Makefile` (needs `RACK_DIR`). Builds against the current `../nisps/` core via relative
includes; no `nisps-core`.
### `schemas/` — JSON parameter contracts (firmware/browser source of truth)
- `schemas/schema.json` — Draft 2020-12 meta-schema validating mode files.
- `schemas/modes/<mode>.json` (×9) — each mode's params, ranges, defaults, curves, voice spaces, ML config. (`slp_workshop.json` reuses `engine_id: memlcelium`.)
- `schemas/modes/params_notes.md` — provenance notes and judgement calls per mode.
- `schemas/midi_device.schema.json` — Draft 2020-12 meta-schema for external-MIDI-synth templates.
- `schemas/midi_devices/<device>.json` (×6) — CC-controllable external synths (Moog Sub 37 / Sub Phatty, Creamware Pro-12 ASB, Elektron Analog Keys, ASM Hydrasynth, Roland JD-800). Each param: `{id, cc, label, min, max, default, group}`. Canonical source for both firmware + browser device pickers. Verified-CC provenance + sources live in `schemas/midi_devices/sources/synth-midi-cc.json` (a `sources/` subdir, because the generator ajv-validates every `*.json` directly under `midi_devices/` as a device template).
### `codegen/` — schema → C++/TS code
- `codegen/generate.ts` — Bun script: validates schemas via ajv (incl. the P5 firmware-fit check: exactly 3 hidden layers, dims ≤4096), emits per-mode C++ `nisps/modes/generated/<mode>_schema.hpp` (`constexpr`, `nisps::modes::generated`) AND TS `manifold/src/modes/generated/<mode>_schema.ts` (+ `types.ts`, `index.ts`). Idempotent; golden-tested in `run-all-tests.sh` stage 5. The TS output is the SOURCE OF TRUTH consumed by `MF_MODES` (`manifold/src/console/model.ts`).
- `codegen/generate-midi-devices.ts` — separate Bun script (isolated from the mode golden test): validates `schemas/midi_devices/` via ajv, emits `nisps/midi/generated/midi_devices.hpp` (no-heap `constexpr`) and `manifold/src/midi-devices/generated/{types,devices,index}.ts`. Idempotent.
- `codegen/lib.ts` — helpers shared by both generators. `codegen/tests/golden/` — golden snapshot for paf_synth (C++ + TS).
### `tests/cpp/` — host C++ tests
- Per-component tests: `test_dsp_*.cpp`, `test_engine_*.cpp`, `test_mlp_*.cpp`, `test_mode_*.cpp`, `test_ring_buffer.cpp`, `test_rng.cpp`, `test_math.cpp`. Helpers in `test_helpers.hpp`.
- Verification: `ml_golden_vectors.cpp`, `engine_impulse.cpp` (+ `engine_impulse_baseline.bin`), `parity_check.cpp` + `parity_wasm.mjs` + `parity_diff.mjs` — native-vs-WASM bit-equivalence within 1e-5.
### `scripts/` — build + verify entry points
- `build-firmware.sh`, `flash-firmware.sh`, `build-and-flash-firmware.sh`, `firmware-common.sh` — Arduino-CLI wrapper for RP2350 target with C++20 flag.
- `build-wasm.sh` — Emscripten compile producing `manifold/public/nisps.{wasm,js}`.
- `build-cpp-tests.sh` — CMake configure + build + ctest (Ninja).
- `parity-check.sh` — runs native + WASM and diffs binary outputs.
- `lint-cpp.sh``.f` literal warn + heap/`Arduino.h` violation fail.
- `run-all-tests.sh` — master verification script.
### `.github/workflows/`
- `ci.yml` — GitHub Actions: cmake build + ctest + WASM build + parity check + lint + Playwright (cpp-tests + manifold-tests jobs). Firmware compile is documented as manual.
### `src/` — submodule + vendored trees
build(firmware): migrate to PlatformIO and vendor memllib (plan §5) One cut, no dual path. Closes ALIGNMENT defect 3 ("Arduino-CLI build machinery is actively hostile") and vision bullet 4. platformio.ini carries 16 [env:], one per variant, each passing -DMEMLNAUT_MODE_TYPE; selftest passes -DNISPS_SELFTEST=1 instead. The env list IS the registry now — the .ino comment-registry and the NISPS_ST_* token-paste table are deleted rather than migrated. L12 noted that table was already silently missing the currently-shipped SLPWorkshop variant, which is the whole argument against having a second list. Also deleted: the Python/sed machinery that rewrote the COMMITTED .ino on every build, the sketch symlink forest, the global TFT_eSPI User_Setup.h mutation (now -D flags — TFT_eSPI's own documented PlatformIO recipe), the UF2 boot-mount detection stack (upload_protocol=picotool talks to the bootloader directly), and build-firmware-arch.sh entirely. Scripts 683 -> 435 lines. memllib is vendored at lib/memllib/ from upstream e291192; no submodules remain. VENDORED.md records provenance and the re-sync procedure. S9: a firmware-build CI job compiles three representative envs against a cached toolchain and reports per-variant flash/RAM. Firmware is in an automated gate for the FIRST time. The old ci.yml comment justified excluding it as "low verification value" — an assessment that did not survive contact, since the SelfTest variant sat broken for an unknown period calling a DisplayDriver method that did not exist at the pinned memllib commit, and nothing noticed because nothing built it. Verified: all 16 envs build from an empty cache, each within ~520 bytes of the arduino-cli binary it replaces, flash and RAM. Measured as .text+.rodata / .data+.bss+vector+uninitialized — NOT PlatformIO's console line, which double-counts .data on this board. This does not prove the hardware boots; no flash+smoke test was possible and that stays an operator chokepoint. slpworkshop 248232/145028 pafsynth 256880/149716 selftest 216228/17960 (all 16 in the CI log format; none exceeds 2% of a 16 MB flash) Two traps recorded so nobody rediscovers them: vendoring memllib's subdirs without a src/ wrapper makes PlatformIO's library builder silently compile NOTHING while still linking; and project build_flags land BEFORE the framework's own -std=gnu++17 -Os, so build_unflags is required. CORRECTION carried in this commit: the firmware sizes in c19d846's message and the first version of the memllib recon doc were wrong — SLPWorkshop 145348, PAFSynth 145300, SelfTest 141840. They came from building variants in sequence through a SHARED incremental arduino-cli build directory, which reused stale objects and under-reported by ~75 KB. Clean-cache rebuilds of the identical commit give 216736/18492 for SelfTest. The real cost of the memllib upstream bump is +216 bytes flash, not +316. Never measure firmware size through a reused build dir. HISTORY NOTE: this commit and the docs commit before it were rebuilt (force-push, 2026-07-21) so that each contains only what its message describes. The first versions had the firmware deletions stranded in the docs commit by a shared-index race between concurrent agents; content is byte-identical to the originals. Gates: run-all-tests.sh ALL GREEN (nisps/ untouched by this change beyond include paths); 16/16 pio envs build.
2026-07-21 20:17:58 +02:00
- **There are no submodules.** `src/memllib` was one until the Phase 4 PlatformIO migration; it is now vendored at `firmware/MEMLNaut-NISPS/lib/memllib/`. Fresh clones need no `git submodule` step.
### Top-level docs
- `CLAUDE.md` — long-form architecture narrative.
- `MAP.md` — this file.
- `ALIGNMENT.md` — strategic gaps + open mission questions, dated, opinionated.
2026-04-15 17:32:32 +02:00
- `README.md` — short quickstart.
2026-07-11 23:19:01 +02:00
- `AGENTS.md` — canonical agent contract: architecture, build/test, scope, and Ergo workflow.
2026-04-15 17:32:32 +02:00
## Entry points
- **Firmware**: `scripts/build-firmware.sh [VARIANT]` (interactive prompt if omitted), `scripts/flash-firmware.sh`, `scripts/build-and-flash-firmware.sh`. Target: `rp2040:rp2040:solderparty_rp2350_stamp_xl:opt=Optimize3`, `-std=gnu++20`.
- **Manifold dev**: `cd manifold && bun install && bun run dev` (Vite, COOP/COEP headers).
- **Manifold build**: `cd manifold && bun run build`.
- **WASM rebuild**: `bash scripts/build-wasm.sh` (needs `emcc`).
- **Host C++ tests**: `bash scripts/build-cpp-tests.sh`.
- **Parity check**: `bash scripts/parity-check.sh`.
- **All tests**: `bash scripts/run-all-tests.sh`.
- **Playwright**: `cd manifold && node node_modules/.bin/playwright test` (non-snap node runner on the VPS — BUILD-PLAN gotcha; `bunx playwright test` works elsewhere).
refactor(ml): one runtime-configurable training default (S26) The operator's call: "there should be one default learning rate and one default max iterations and they should both be configurable at runtime." There were SIX copies, not the four the audit described, and they did not agree: nisps/ml/mlp.hpp no-arg train() hardcoding 1.f / 1000u / 0.001f — and firmware's ONLY training path calls exactly this, so firmware had no runtime knob at all wasm-iml.ts train() and trainAsync() TS default params (x2) engine-api.ts learningRate ?? 1.0, with no maxIterations knob vcv/src/iml.hpp 200 / 0.1 / 0.00001 — silently divergent external_synth_midi.hpp its own kDefaultLearningRate/kDefaultMaxIterations schemas/modes/*.json x9, identical, read by nobody at runtime Now: schemas/ml_defaults.json is the single declaration (validated against a sibling meta-schema, matching the midi_device.schema.json convention), codegen emits it to C++ and TS in the same run, and MLPCore carries a TrainConfig whose default member initialisers read the generated constant. set_train_config()/nisps_ml_set_train_config() make it runtime-overridable on every target; the explicit-argument train() overload is untouched. min_error joins the tuple — it was duplicated identically and belongs with the other two. The per-mode ml block loses default_learning_rate/default_max_iterations. default_spread stays (genuinely wired on both targets) and input_channels stays (codegen-time validated, real information for sound_analysis_midi). VCV BEHAVIOUR CHANGE, deliberate: MEMLNaut.cpp constructs IML positionally and relies on those defaults, so the module moves to 1000/1.0/0.001 — 5x the max iterations, 10x the learning rate, and a 100x looser early-stop threshold. The old values were never justified anywhere; they arrived with fbc68eb alongside an unrelated module rewrite and no tuning rationale. Firmware and WASM have shipped 1.0/1000 all along. It is now runtime-settable if this turns out worse. The generated header lands in nisps/ml/generated/, not nisps/modes/generated/ where the rest of codegen output lives: training hyperparameters are an ML fact, and nisps/ml sits below nisps/modes, so emitting them there would make mlp.hpp include upward. The agent that built this flagged the directory-crossing rather than hiding it; this is the fix. CI's generated-freshness gate learns the new directory. Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS (max delta 2.38e-7), lint clean, manifold typecheck + 17 unit + 33 e2e (which exercise train() and trainAsync() through a real browser).
2026-07-21 17:20:10 +02:00
- **Codegen**: `cd codegen && bun run generate.ts` (regenerates `nisps/modes/generated/` + `nisps/ml/generated/` C++ and `manifold/src/modes/generated/` TS).
2026-04-15 17:32:32 +02:00
## Conventions
build(firmware): migrate to PlatformIO and vendor memllib (plan §5) One cut, no dual path. Closes ALIGNMENT defect 3 ("Arduino-CLI build machinery is actively hostile") and vision bullet 4. platformio.ini carries 16 [env:], one per variant, each passing -DMEMLNAUT_MODE_TYPE; selftest passes -DNISPS_SELFTEST=1 instead. The env list IS the registry now — the .ino comment-registry and the NISPS_ST_* token-paste table are deleted rather than migrated. L12 noted that table was already silently missing the currently-shipped SLPWorkshop variant, which is the whole argument against having a second list. Also deleted: the Python/sed machinery that rewrote the COMMITTED .ino on every build, the sketch symlink forest, the global TFT_eSPI User_Setup.h mutation (now -D flags — TFT_eSPI's own documented PlatformIO recipe), the UF2 boot-mount detection stack (upload_protocol=picotool talks to the bootloader directly), and build-firmware-arch.sh entirely. Scripts 683 -> 435 lines. memllib is vendored at lib/memllib/ from upstream e291192; no submodules remain. VENDORED.md records provenance and the re-sync procedure. S9: a firmware-build CI job compiles three representative envs against a cached toolchain and reports per-variant flash/RAM. Firmware is in an automated gate for the FIRST time. The old ci.yml comment justified excluding it as "low verification value" — an assessment that did not survive contact, since the SelfTest variant sat broken for an unknown period calling a DisplayDriver method that did not exist at the pinned memllib commit, and nothing noticed because nothing built it. Verified: all 16 envs build from an empty cache, each within ~520 bytes of the arduino-cli binary it replaces, flash and RAM. Measured as .text+.rodata / .data+.bss+vector+uninitialized — NOT PlatformIO's console line, which double-counts .data on this board. This does not prove the hardware boots; no flash+smoke test was possible and that stays an operator chokepoint. slpworkshop 248232/145028 pafsynth 256880/149716 selftest 216228/17960 (all 16 in the CI log format; none exceeds 2% of a 16 MB flash) Two traps recorded so nobody rediscovers them: vendoring memllib's subdirs without a src/ wrapper makes PlatformIO's library builder silently compile NOTHING while still linking; and project build_flags land BEFORE the framework's own -std=gnu++17 -Os, so build_unflags is required. CORRECTION carried in this commit: the firmware sizes in c19d846's message and the first version of the memllib recon doc were wrong — SLPWorkshop 145348, PAFSynth 145300, SelfTest 141840. They came from building variants in sequence through a SHARED incremental arduino-cli build directory, which reused stale objects and under-reported by ~75 KB. Clean-cache rebuilds of the identical commit give 216736/18492 for SelfTest. The real cost of the memllib upstream bump is +216 bytes flash, not +316. Never measure firmware size through a reused build dir. HISTORY NOTE: this commit and the docs commit before it were rebuilt (force-push, 2026-07-21) so that each contains only what its message describes. The first versions had the firmware deletions stranded in the docs commit by a shared-index race between concurrent agents; content is byte-identical to the originals. Gates: run-all-tests.sh ALL GREEN (nisps/ untouched by this change beyond include paths); 16/16 pio envs build.
2026-07-21 20:17:58 +02:00
- Firmware mode selection is compile-time only — one `-DMEMLNAUT_MODE_TYPE` per `[env:]` in `platformio.ini`.
- `nisps/` follows Chris's RP2350 perf rules: no heap, `static const float` for non-trivial constants, strict `.f` suffix. `perf.hpp` now carries only `NISPS_HOT`/`NISPS_FORCE_INLINE`; the three dead/misshapen SRAM-section macros were deleted in the 2026-07 sweep (S21/L13).
- C++ identifiers: `PascalCase` types, `snake_case` functions/variables, `kPascalCase` constexpr. JSON keys `snake_case`. TS types `PascalCase`, components `PascalCase.tsx`, modules `kebab-case.ts`.
- `Curve` enum lives in `nisps/core/math.hpp` (lowercase: `linear/exp/log/square/sqrt/sigmoid/cubic`, plus the parameterised `centered_power` free function); generated mode headers re-export via `using Curve = ::nisps::Curve;`. Since P4 there is NO TS mirror — the browser samples the WASM catalog (`nisps_curve_apply(+batch)`).
- Modes are TSX components composed of primitives; mode parameter contracts are JSON schemas with codegen → C++ **and** TS types (`MF_MODES` derives params/ml-config from the generated schemas since P5; labels/ordering stay a manifold overlay). **No declarative JSON UI.**
- WASM and firmware share the same C++; the browser MLP is runtime-shaped (`MLPCore<DynamicStorage>`, since P2): `nisps_ml_create` honours `(input, output, hidden[3])` with non-positive/null args defaulting to `32→[10,14,18]→126`; `nisps_ml_reshape` warm-starts a new shape. Firmware keeps compile-time `MLP<...>` (zero heap). Per-mode dims are schema-real on both targets since P5.3 (the browser reshapes on mode switch).
- Cross-platform parity: `scripts/parity-check.sh` enforces native vs WASM agreement within 1e-5.
2026-04-15 17:32:32 +02:00
## Gotchas
build(firmware): migrate to PlatformIO and vendor memllib (plan §5) One cut, no dual path. Closes ALIGNMENT defect 3 ("Arduino-CLI build machinery is actively hostile") and vision bullet 4. platformio.ini carries 16 [env:], one per variant, each passing -DMEMLNAUT_MODE_TYPE; selftest passes -DNISPS_SELFTEST=1 instead. The env list IS the registry now — the .ino comment-registry and the NISPS_ST_* token-paste table are deleted rather than migrated. L12 noted that table was already silently missing the currently-shipped SLPWorkshop variant, which is the whole argument against having a second list. Also deleted: the Python/sed machinery that rewrote the COMMITTED .ino on every build, the sketch symlink forest, the global TFT_eSPI User_Setup.h mutation (now -D flags — TFT_eSPI's own documented PlatformIO recipe), the UF2 boot-mount detection stack (upload_protocol=picotool talks to the bootloader directly), and build-firmware-arch.sh entirely. Scripts 683 -> 435 lines. memllib is vendored at lib/memllib/ from upstream e291192; no submodules remain. VENDORED.md records provenance and the re-sync procedure. S9: a firmware-build CI job compiles three representative envs against a cached toolchain and reports per-variant flash/RAM. Firmware is in an automated gate for the FIRST time. The old ci.yml comment justified excluding it as "low verification value" — an assessment that did not survive contact, since the SelfTest variant sat broken for an unknown period calling a DisplayDriver method that did not exist at the pinned memllib commit, and nothing noticed because nothing built it. Verified: all 16 envs build from an empty cache, each within ~520 bytes of the arduino-cli binary it replaces, flash and RAM. Measured as .text+.rodata / .data+.bss+vector+uninitialized — NOT PlatformIO's console line, which double-counts .data on this board. This does not prove the hardware boots; no flash+smoke test was possible and that stays an operator chokepoint. slpworkshop 248232/145028 pafsynth 256880/149716 selftest 216228/17960 (all 16 in the CI log format; none exceeds 2% of a 16 MB flash) Two traps recorded so nobody rediscovers them: vendoring memllib's subdirs without a src/ wrapper makes PlatformIO's library builder silently compile NOTHING while still linking; and project build_flags land BEFORE the framework's own -std=gnu++17 -Os, so build_unflags is required. CORRECTION carried in this commit: the firmware sizes in c19d846's message and the first version of the memllib recon doc were wrong — SLPWorkshop 145348, PAFSynth 145300, SelfTest 141840. They came from building variants in sequence through a SHARED incremental arduino-cli build directory, which reused stale objects and under-reported by ~75 KB. Clean-cache rebuilds of the identical commit give 216736/18492 for SelfTest. The real cost of the memllib upstream bump is +216 bytes flash, not +316. Never measure firmware size through a reused build dir. HISTORY NOTE: this commit and the docs commit before it were rebuilt (force-push, 2026-07-21) so that each contains only what its message describes. The first versions had the firmware deletions stranded in the docs commit by a shared-index race between concurrent agents; content is byte-identical to the originals. Gates: run-all-tests.sh ALL GREEN (nisps/ untouched by this change beyond include paths); 16/16 pio envs build.
2026-07-21 20:17:58 +02:00
- Firmware needs PlatformIO: `nix-shell -p platformio-core`. Use `platformio-core`, NOT `platformio` — the latter is nixpkgs' bubblewrap-wrapped FHS build and fails without a working user namespace. First build pulls ~1-2 GB into `~/.platformio`.
- `firmware/MEMLNaut-NISPS/glue/mode_select.hpp` `#undef`s Arduino macros (`sq`, `min`, `max`, `abs`, `round`) before pulling nisps headers — engines use those identifiers as method names.
build(firmware): migrate to PlatformIO and vendor memllib (plan §5) One cut, no dual path. Closes ALIGNMENT defect 3 ("Arduino-CLI build machinery is actively hostile") and vision bullet 4. platformio.ini carries 16 [env:], one per variant, each passing -DMEMLNAUT_MODE_TYPE; selftest passes -DNISPS_SELFTEST=1 instead. The env list IS the registry now — the .ino comment-registry and the NISPS_ST_* token-paste table are deleted rather than migrated. L12 noted that table was already silently missing the currently-shipped SLPWorkshop variant, which is the whole argument against having a second list. Also deleted: the Python/sed machinery that rewrote the COMMITTED .ino on every build, the sketch symlink forest, the global TFT_eSPI User_Setup.h mutation (now -D flags — TFT_eSPI's own documented PlatformIO recipe), the UF2 boot-mount detection stack (upload_protocol=picotool talks to the bootloader directly), and build-firmware-arch.sh entirely. Scripts 683 -> 435 lines. memllib is vendored at lib/memllib/ from upstream e291192; no submodules remain. VENDORED.md records provenance and the re-sync procedure. S9: a firmware-build CI job compiles three representative envs against a cached toolchain and reports per-variant flash/RAM. Firmware is in an automated gate for the FIRST time. The old ci.yml comment justified excluding it as "low verification value" — an assessment that did not survive contact, since the SelfTest variant sat broken for an unknown period calling a DisplayDriver method that did not exist at the pinned memllib commit, and nothing noticed because nothing built it. Verified: all 16 envs build from an empty cache, each within ~520 bytes of the arduino-cli binary it replaces, flash and RAM. Measured as .text+.rodata / .data+.bss+vector+uninitialized — NOT PlatformIO's console line, which double-counts .data on this board. This does not prove the hardware boots; no flash+smoke test was possible and that stays an operator chokepoint. slpworkshop 248232/145028 pafsynth 256880/149716 selftest 216228/17960 (all 16 in the CI log format; none exceeds 2% of a 16 MB flash) Two traps recorded so nobody rediscovers them: vendoring memllib's subdirs without a src/ wrapper makes PlatformIO's library builder silently compile NOTHING while still linking; and project build_flags land BEFORE the framework's own -std=gnu++17 -Os, so build_unflags is required. CORRECTION carried in this commit: the firmware sizes in c19d846's message and the first version of the memllib recon doc were wrong — SLPWorkshop 145348, PAFSynth 145300, SelfTest 141840. They came from building variants in sequence through a SHARED incremental arduino-cli build directory, which reused stale objects and under-reported by ~75 KB. Clean-cache rebuilds of the identical commit give 216736/18492 for SelfTest. The real cost of the memllib upstream bump is +216 bytes flash, not +316. Never measure firmware size through a reused build dir. HISTORY NOTE: this commit and the docs commit before it were rebuilt (force-push, 2026-07-21) so that each contains only what its message describes. The first versions had the firmware deletions stranded in the docs commit by a shared-index race between concurrent agents; content is byte-identical to the originals. Gates: run-all-tests.sh ALL GREEN (nisps/ untouched by this change beyond include paths); 16/16 pio envs build.
2026-07-21 20:17:58 +02:00
- `nisps_firmware::g_active_mode_bridge` is `extern` in `glue/audio_driver.hpp` and defined in `src/main.cpp`; combining `inline` with `__not_in_flash` produces a comdat conflict at link time.
- `pio run`'s own "Flash: NN%" console line double-counts `.data` on this board (PlatformIO's generic size checker counts every PROGBITS+ALLOC section). Compare `arm-none-eabi-size -A` — flash = `.text+.rodata` — before believing a size regression.
- `nisps_modes_tests` builds against generated schemas under `nisps/modes/generated/`; if you add a new mode, regenerate via `bun run codegen/generate.ts` before building.
## Smells / strategic concerns
See `ALIGNMENT.md`.
## Specs
- **Root**: `docs/specs/`
- **Entry**: `MAIN.md`
- **Layout**: flat (with `plans/`, `recon/`, `_archive/` subdirs)
- **Index**: none (intentionally — generate when a consumer exists)
- **Skill**: invoke `/specs` to review/maintain/add/navigate.
- **Conventions**: Four-genre ontology — `kind: spec` (timeless contract, wins by intent), `kind: plan` (status: active|executed|superseded, never authority for behaviour), `kind: finding` (dated, immutable, exempt from drift lint), ADRs in `docs/adr/`.
The corpus holds platform-level specs (engine architecture, I/O backends, feedback design), implementation specs (port specs, wire protocols), feature specs (e.g. slp-workshop-firmware.md), historical findings (dated research artifacts), and finite build plans. Before changing behaviour a spec covers, find it via `/specs`; the spec wins by intent — if it's wrong, update it in the same commit as the code.