Move the Arduino sketch into firmware/MEMLNaut-NISPS/ and bridge the
hardware (memllib) to the platform-agnostic nisps/ library through a
slim glue layer. Delete the legacy root-level *AudioApp.hpp,
modes/MEMLNautMode*.hpp, voicespaces/, IMLInterface.hpp, XiasriAnalysis,
and the src/memlp submodule.
Glue layout (firmware/MEMLNaut-NISPS/glue/):
audio_driver.hpp - bridge memllib block callback to Mode::process
via per-Mode templated trampoline (no virtual dispatch)
peripherals.hpp - joystick/pots/buttons -> Mode::set_input + ML primitives
midi_io.hpp - MIDI in -> mode.note_on/update_bpm/set_playing,
drains mode ControlEvent ring -> MIDI UART
mode_select.hpp - using-aliases mapping MEMLNautMode<Name> to
nisps::modes::*Mode (build script rewrites the
#define MEMLNAUT_MODE_TYPE line)
input_router.hpp / output_router.hpp - top-level wire/drain entry points
The sketch tree uses src/{memllib,daisysp,nisps} symlinks because
Arduino-CLI rejects ".." in include paths from sketch-tree headers.
mode_select.hpp #undefs Arduino's sq/min/max/abs/round macros before
including nisps headers (some nisps engines use those identifiers as
method names). The audio bridge struct is extern in the header and
defined in the .ino because inline + __not_in_flash section attribute
collide at link time.
Verification: arduino-cli compile succeeds for PAFSynth, ChannelStrip,
and BreakOr (rp2040:rp2040:solderparty_rp2350_stamp_xl:opt=Optimize3,
-std=gnu++20). Host C++ tests under nisps/build still pass (3 binaries,
110+ tests). Build script (scripts/build-firmware.sh) updated to point
at the new sketch path; mode-rewrite logic unchanged.
Closes meml-gkm.
Stream 9 (part 1/2) — shared infrastructure for the per-mode TSX layer.
- ModeShell.tsx: common chrome (header, voice-space PillToggle, audio
start/stop, control axes, training controls, drawer, status line).
Modes only have to author primary input + output JSX.
- mode-runtime.ts: useModeRuntime(schema) hook. Owns input pipeline,
WASM ML calls, output pipeline, and a 50ms-throttled engine-host
bridge. Exposes setInput/processedOutputs/training/audio surfaces
so mode TSX stays declarative.
- mode-helpers.ts: schema → SliderBank config + Float32Array →
slider-range value mapping. Pure functions.
- ModeSwitcher.tsx: top-of-page <select> wired to modeStore.
ModeShell and the runtime read controlStore (Boldness/Memory/Precision)
to derive learning rate + RL noise cap so axes affect training without
needing per-mode plumbing.
Stream 7 wires nisps/ml + nisps/engines into the playground via Emscripten.
Highlights:
- nisps/wasm/bindings.cpp: flat C API per architecture.md §6.2. Fixed-arch
MLP<2, 10, 14, 18, 126>; engine string→type dispatch table with NoOp
fallback.
- scripts/build-wasm.sh: emcc invocation, MODULARIZE=1, exports listed
explicitly; produces playground/public/nisps.{js,wasm}.
- playground/src/ml/wasm-iml.ts: main-thread MLP host (sync inference,
sync training, RL ops, weights I/O, layer stats, localStorage).
- playground/src/ml/wasm-worker.ts: disposable Web Worker for off-thread
async training, owns its own WASM instance.
- playground/src/ml/dataset.ts: Float32Array-backed FIFO with sample-weight
modes (uniform/global/local/combined). Port of legacy dataset.js.
- playground/src/audio/engine-host.ts: AudioContext + AudioWorkletNode
lifecycle, with start/stop/setEngine/setParams.
- playground/src/audio/worklet/nisps-processor.ts: WASM-loading
AudioWorkletProcessor that runs engine.process_block per 128-sample
block. Loads its own WASM instance from main-thread-supplied bytes
(no fetch in worklet).
- playground/src/stores/ml-store.ts: wired stub methods to WasmIML
singleton; lazy initialize().
- playground/src/debug/probe.ts: window.__nisps now calls real WasmIML
via the store; lazy-init on first use.
Verified:
- bash scripts/build-wasm.sh succeeds (94 KB nisps.wasm)
- bun run typecheck OK
- bun run build OK (production bundle)
- vite dev server serves /nisps.{js,wasm} with COOP/COEP
Known limitation: WASM is fixed at one MLP shape. Multi-arch deferred —
documented in nisps/wasm/README.md.
One header per mode, each ~50-130 LOC built atop ModeBase:
- `paf_synth.hpp` (4 inputs → 33 outputs, 7 voice spaces, note_on/off)
- `channel_strip.hpp` (4 → 24, 6 voice spaces)
- `xiasri.hpp` (4 → 24, 1 'Direct' voice space)
- `verb_fx.hpp` (4 → 47, 12 voice spaces)
- `memlcelium.hpp` (4 → 56, dual synth + 2-track sequencer; set_playing/update_bpm)
- `breakor.hpp` (4 → 56, 8-track ratio sequencer; pumps engine NoteOn/Off/Clock
events into ControlEvent ring buffer)
- `elysiamorf.hpp` (4 → 40, 8-track FM-pair MIDI-CC sequencer; pumps CC events)
- `sound_analysis_midi.hpp` (10 → 8; owns AnalysisEngine for input feature
extraction PLUS NoOpEngine 'thru' for audio passthrough; ML outputs
converted to MIDI CC events via on_post_inference; opts out of
output→engine routing via ModeRoutesOutputsToEngine specialisation)
Every mode satisfies `nisps::Mode` (verified via `static_assert` in each
header). Schema `output_size` is verified against engine `param_count()`
at compile time inside ModeBase.
All hardware-specific glue (MEMLNaut::Instance, pico/util/queue, MIDIInOut,
display widgets, button callbacks) is intentionally absent — that lives in
firmware/glue and playground/src/modes per architecture.md §4.3.
Adds two foundational pieces shared by every concrete mode:
- `base.hpp`: defines `nisps::ParamSchema` (the aggregate type the
`Mode` concept's `param_schema()` returns by const-reference),
`nisps::ControlEvent`, and `ModeBase<Derived, EngineT, MLPType,
NInputs>` — a CRTP scaffold absorbing input forwarding, ML
inference + voice space mapping, engine.set_params() routing, and
a per-mode ControlEvent ring buffer for MIDI/I2C events.
`ModeRoutesOutputsToEngine<Derived>` lets a derived mode (e.g.
SoundAnalysisMIDI) opt out of routing ML outputs to engine
params when its outputs become MIDI CCs instead.
- `voice_space.hpp`: `VoiceSpaceEntry` + `make_voice_space_entries()`
+ `find_voice_space()` helpers. Voice-space *mapping code* lives
inside engines (mirroring firmware); mode layer just selects
which voice space the engine uses by index/name.
No heap, no virtuals, no Pico/Arduino dependencies; satisfies stream
4 of architecture.md.
Stream 3's NoOpEngine used engine_id()=='noop' but the schema for
sound_analysis_midi.json declares engine_id: 'thru' (matches firmware's
ThruAudioApp naming). The class name stays NoOpEngine — it accurately
describes what process() does — but the schema-facing identifier is now
'thru'. BreakOr/Elysiamorf compose NoOpEngine by type, not engine_id, so
they're unaffected.
Stream 5 shipped a temporary local Curve enum (PascalCase) while stream 1
was in flight. Stream 1 has now landed nisps/core/math.hpp with the
authoritative lowercase Curve enum per architecture spec. Generated C++
headers now include core/math.hpp and re-export the enum into the
generated namespace via 'using Curve = ::nisps::Curve;'.
- codegen/generate.ts: emit lowercase enum values + include math.hpp
- regenerated all 8 mode schema headers
- updated golden snapshot to match
Verified all 8 headers compile clean with -std=c++20 -Wall -Wextra.
Sixteen primitives at src/primitives/, each with its own .module.css and a
.demo.tsx that wires the primitive to live local state. All primitives
are typed (strict), pointer-event based (touch + mouse), keyboard-accessible
where it makes sense, and free of business logic.
Primitives:
- Slider, SliderBank: value sliders with optional curve mapping;
collapsible sections.
- VirtualJoystick, XYPad: 2D pointer input, returning [0,1]^2.
- Heatmap: NxN canvas rendering with three color modes
(luminance/variance/divergence) and a configurable palette.
- OutputDisplay: bar chart of N output values.
- TrainingControls: thumbs-up/down/undo + train/randomise + status.
- Drawer: portal'd slide-in panel from left or right; ESC + scrim close.
- ControlAxis: compound-axis slider (Boldness/Memory/Precision shape)
with active-preset hint, endpoint labels, double-tap-to-relink.
- ProgressRing: SVG circular progress with optional inline label.
- PillToggle: segmented radio control.
- ParamEditor: min/max/curve/mute/pin/fixedValue editor for a single param.
- JoyMap: zoom minimap with adaptive grid + zoom window, vanishing trail
with Catmull-Rom spline + tap-to-return, dual concentric noise rings,
region pin overlays, frozen overlay.
- WeightHealth: 10-bin histogram + dead/saturating/healthy status glow.
- GradientFlow: per-layer bar chart with vanishing/exploding/converged
color coding.
- LossPlot: log-scale line chart of training loss history.
App.tsx now wires the /dev/primitives route to a lazy-loaded
PrimitivesShowcase that renders all sixteen demos in a responsive grid.
Lazy import keeps the home page bundle ~20 kB while the showcase ships
its own ~42 kB chunk.
Stream 8 of the rewrite (meml-911).
Seven Solid stores wired up around the architecture's reactivity model
(§7.1). Each store uses createStore for object state + createSignal for
Float32Arrays where appropriate; setters mutate the store and schedule a
debounced (200ms) localStorage write through src/stores/persistence.ts.
- bus.ts: typed signal bus with prefix wildcards (ml.*, ui.*, mode.*,
pin.*, snap.*). Singleton coreBus for app-wide events.
- ml-store.ts: shape final, methods stub-throw "not implemented" until
stream 7 wires WASM. Outputs and weights are separate Float32Array
signals so the store proxy doesn't run on every audio-rate tick.
- input-store.ts: full input pipeline config (zoom, anchor, deadzone,
curve, smoothing, momentum, invert) + persisted live state.
- output-store.ts: globalCurve, smoothing, slewRate, freezeOutput,
freezeMask. Mask not persisted (engine-specific).
- mode-store.ts: activeModeId + per-mode { paramName → ParamOverride }.
- control-store.ts: Boldness/Memory/Precision compound axes with
interpolation tables and offset-based overrides (trim-pot model).
Tables and 6 built-in CONTROL_PRESETS mirror legacy
js/ui/control-surface.js exactly. interpolateAxis() exposed for
testing. Stream 10 wires resolveParams() into other stores.
- session-store.ts: ring-buffered snapshot stack (max 20), A/B
capture/toggle/accept/revert, region pins (max 5), param pins
(toggle + mask builder), named session presets.
- index.ts: public re-exports for components and modes.
Stream 8 of the rewrite (meml-911).
Pure-TS ports of the legacy input + output pipelines:
- src/input/pipeline.ts: deadzone → circular clamp → zoom → centered
power curve → EMA smoothing → momentum-as-zoom. Exposed as a
pure function processInput(raw, cfg, state, dt) so the input-store
can hold the state. Math is intentionally bit-equivalent to the
legacy js/ui/input-pipeline.js implementation.
- src/output/pipeline.ts: global power curve → EMA smoothing →
slew-rate limit → freeze gate (global + per-output mask).
- src/output/curves.ts: named curve catalog (linear/exp/log/square/
sqrt/sigmoid/cubic/centered_power) — the TS half of the contract
defined in nisps/core/math.hpp (stream 1). Golden-vector tests
in stream 11 will keep them in lockstep.
Stream 8 of the rewrite (meml-911).
Fresh playground/ now sits on:
- Vite 5 + vite-plugin-solid + Solid.js 1.9
- TypeScript strict (jsx preserve, target ES2022)
- index.html → main.tsx → App.tsx with a tiny pushState/popstate router
- Design tokens (dark immersive palette, JetBrains Mono) at src/styles/tokens.css
- COOP/COEP headers in vite.config.ts (server + preview) so SharedArrayBuffer
is available for the C15 + AudioWorklet wiring stream 7 will land.
- Debug-probe stub at src/debug/probe.ts: window.__nisps with placeholder
methods + __ready=false marker. Stream 10 fills it in. Keeping the install
point stable from day one means Playwright tests can rely on the global
existing.
Stores, pipelines, and primitives ride in the next commits.
Stream 8 of the rewrite (meml-911).
Clears the old vanilla-JS / multi-app playground (a-immersive.html,
b-workbench.html, c-journey.html, designs.html, index.html, plus the
js/, css/, c15/, faust/, firmware/, osc-bridge/, wasm/ subtrees and the
serve/test scripts). The fresh SolidJS + Vite scaffold lands in the next
commits per the rewrite plan in .local/architecture.md.
Reference docs (SPEC-controls.md, SPEC-shapeseq.md,
PLAN-solidjs-migration.md, ARCHITECTURE.md, README.md, TODOS.md, devlog/)
have been moved to .local/playground-archive/ (out of git) so future
streams can still consult them.
Stream 8 of the rewrite (meml-911).
Output of \`bun run codegen/generate.ts\` for the 8 mode schemas.
These files are checked in so consumers don't need bun on every
build, but they remain regenerable and byte-identical.
C++ (nisps/modes/generated/):
schema_types.hpp + 8 <mode_id>_schema.hpp files. All compile
cleanly with g++ -std=c++20 -Wall -Wextra -fsyntax-only -I nisps.
TS (playground/src/modes/generated/):
types.ts + index.ts + 8 <mode_id>_schema.ts files. Type-check
cleanly with tsc --strict.
bun-runnable TypeScript script (codegen/generate.ts) that:
- Validates each schemas/modes/*.json against the meta-schema via
ajv (Draft 2020-12).
- Cross-checks params.length == ml.output_size and
ml.input_channels.length == ml.input_size.
- Emits constexpr C++ data into nisps/modes/generated/ (one
schema_types.hpp + one <mode_id>_schema.hpp per mode). Uses
std::string_view + std::array; no std::vector, no heap, .f
suffixed float literals (perf contract §3.3).
- Emits TS modules into playground/src/modes/generated/ (one
types.ts + one <mode_id>_schema.ts + index.ts barrel).
- Idempotent: re-running yields byte-identical output.
- Exits non-zero on schema validation failure.
Reference templates live in codegen/templates/ (not consumed at
codegen time -- for human reviewers).
Golden test (codegen/tests/golden_test.ts) snapshots
paf_synth_schema.{hpp,ts} and verifies regeneration matches the
golden + that a second run is idempotent.
Until stream 1 lands nisps/core/math.hpp, schema_types.hpp ships
its own minimal Curve enum with a TODO marker pointing at the
eventual include.
CMakeLists.txt:
- Native host build by default; Emscripten-target detection plumbed but
WASM emit deferred to stream 7 (playground build script).
- Header-only INTERFACE library `nisps_core`.
- Host test executable `nisps_core_tests` compiled with -Wall -Wextra
-Werror -Wpedantic (Chris's rules: clean build is non-negotiable).
tests/cpp/test_helpers.hpp:
- Minimal NISPS_TEST / NISPS_EXPECT / NISPS_EXPECT_NEAR macros, no external
deps. Rationale documented in-file: Catch2/doctest would add ~10MB and 30s
for what is currently <100 LOC of test runtime.
22 unit tests covering FixedBuffer (5), RingBuffer (5), Rng (7), math (5).
All green; verified via `cmake --build nisps/build && ./nisps/build/nisps_core_tests`.
Source: archive/playground-redesign-2026-snapshot branch (commit 287e3a1).
Reference-only — playground is being rewritten in SolidJS, so the redesign's
implementation will not be merged, but its design intent is preserved here
for the rewrite team.
DISLIKE now calls joltNetworks() immediately for instant sound change.
DRAG release now outputs savedAction to audio queue immediately rather
than waiting for training to converge.
b290144 made matrix cells opt-in to prevent joystick-silences-voice,
but that broke modular-ui.updateLive(): the matrix DOM stopped
reflecting live MLP outputs because matrixIndexCache was empty when
buildMatrixIndex() walked paramMeta. This was the same regression
6072fe8 had previously fixed.
Fix it structurally at the DSP layer instead: amp_val now computes
as `clamp(base_amp + max(0, mod_amp)) * level * vel_gain`, so matrix
d08_amp cells can only boost the amp floor — never cut it. base_amp
defaults to 1.0 (always audible), and presets that want classic
envelope-gated voices (slow pad, plucky bass, crystal, morphing
drone) drop base_amp to 0 and layer a positive ADSR→amp route on top.
With the DSP guard in place, all 480 matrix cells can safely live
in paramMeta again, and updateLive() gets its live visual feedback
back. Revert the opt-in gate in _rebuildParamMeta and the 32-param
test counts, and add a regression test asserting that every matrix
destination has 48 cells in paramMeta — that's what updateLive needs.
Modular sub-engines computed amp_val as a pure function of mod_amp (the
matrix d08_amp destination sum), so once the MLP drove the matrix cells
every joystick movement had a chance to silence the voice: matrix cells
have signed range [-1, 1], sigmoid outputs near 0.5 denormalise to 0,
and the amp gate collapsed. Additive survived in scattered regions
because it only has one kill-switch (d08_amp); subtractive and fm were
almost always dead because they also have d05_cutoff and d01_op1_level.
Two changes:
1. DSP: each sub-engine gets a base_amp hslider (default 1.0) so
amp_val = clamp(base_amp + mod_amp) * level * vel_gain. At the
default the voice is always fully open and d08_amp modulation is
purely additive decoration; drop base_amp to 0 for classic
ADSR-gated VCA behaviour.
2. ModularEngine._rebuildParamMeta: restore the _exposedMatrixCells
gate (default empty). paramCount drops from 512 to 32 (4 ADSR * 4
+ 8 LFO * 2); matrix cells are opt-in via setExposeMatrixCell.
_applyDefaultPatch no longer writes s00_d08_amp since base_amp
keeps the voice audible without routing.
Tests updated for the new 32-param baseline; matrix-cell persistence
test now calls setExposeMatrixCell(1, 5, true) before asserting the
cell lands in paramMeta. Drive-by: engine-switching test bumped from
3 to 4 engine cards (stale since the modular engine was added).
Previous fix pulled matrix cells out of paramMeta to avoid the
default-patch-clobbering silence issue. Side effect: paramCount dropped
from 512 to 32, the heatmap strip and synth visualizer shrank
dramatically, and moving the joystick no longer animated matrix cells
(the user's two most recent complaints).
Better approach: put matrix cells back in paramMeta (512 outputs), but
when iml.exampleCount === 0 substitute the normalised default-patch
vector for the raw MLP output in routeOutputs. The engine sees the
default patch exactly, audio works, and the user still sees 512 cells
in the heatmap / matrix grid. As soon as they capture their first
training example the MLP resumes driving everything normally.
ModularEngine.getDefaultNormalizedOutputs() returns the normalised
default value per paramMeta entry, reading from _lastRawByLabel first
(so user edits via click or _setRawByLabel propagate) and falling back
to the walk-entry init field.
modular-ui setCell now prefers engine.setParam over _setRawByLabel when
the cell is in paramMeta, so writes flow through the normal tracking
path and are visible to getDefaultNormalizedOutputs on the next tick.
Verified in headless Chromium: switching to modular, cold start, reading
engine._lastRawByLabel after routeOutputs ticks shows ampRaw=1,
attackRaw=0.01, enableRaw=1 — the default patch is preserved. paramCount
is 512 again. 0 console errors.
The matrix-seeding edit from the previous commit added a second
`const destNames = engine.destNames || [];` inside rebuildMatrixGrid
while the same name was already declared earlier in the function body.
Chrome threw SyntaxError at parse time, which in turn prevented
a-app.js from wiring window.__nisps at all, which is what made the
earlier FaustWorkletProcessor error look like the root cause — it
was just the next thing that broke once the parse error let partial
modules load.
Verified end-to-end in headless Chromium via Playwright: engine
switches to modular, paramCount=32, audioCtx running, worklet node
live, zero console errors.
Also updates the Modular engine card description to reflect the new
default (32 params = 4 ADSR × 4 + 8 LFO × 2, matrix cells opt-in).
The globalThis.FaustWorkletProcessor attachment on the base class isn't
enough in practice: Chrome's AudioWorkletGlobalScope isolates top-level
class declarations between separate addModule() calls, so a subclass
script loaded via a second addModule() still throws ReferenceError on
its 'extends FaustWorkletProcessor' clause.
Workaround: fetch both files, concatenate, and addModule() a single
blob URL. The base class and subclass end up in the same script
evaluation context and the extends clause resolves. Processor names
are memoised on a static Set so repeated sub-engine swaps don't try
to re-register (which would throw).
Two fixes driven by user reports:
1. MLP wasn't affecting the sound. Matrix cells are now opt-in to the
MLP output vector rather than always-driven. Default modular paramMeta
is 32 mod-source params (4 ADSR * A/D/S/R + 8 LFO * rate/morph), down
from 512. The default patch's MM_Matrix/s00_d08_amp=1.0 now survives
the first inference tick because it's not in paramMeta — ADSR1 stays
routed to amp and the MLP drives envelope shape per joystick position.
Matrix cells still clickable as direct-DSP knobs in modular-ui: setCell
now routes through engine._setRawByLabel(). A later UI pass can add a
"expose to MLP" menu entry that calls engine.setExposeMatrixCell(s,d).
Also removes the earlier exampleCount-based routing gate; no longer
needed now that the default patch is stable.
2. Per-group curve drawer (hover over section labels on the synth
visualizer) is now available for all synth engines, not just C15.
Refactored showGroupDrawer behind a getSectionView(sectionIndex)
helper that returns a uniform view for either C15 (via SYNTH_SECTIONS
+ groupOverrides) or non-C15 (via nonC15Sections + nonC15GroupCurves
+ engineParamOverrides).
Group-level curve persists across sub-engine swaps by group name, so
e.g. tuning the "ADSR 1" curve survives a switch between subtractive
and fm without being reset.
ModularEngine: adds setExposeMatrixCell(s,d,exposed) +
getExposedMatrixCells() + clears exposed cells on sub-engine swap.
Modular mode's default patch sets MM_Matrix/s00_d08_amp = 1.0 via
_setRawByLabel at engine init, but on the first inference tick
routeOutputs writes the untrained MLP's output (~0.5 normalised) to
every paramMeta index. For matrix cells the raw range is [-1, 1], so
normalised 0.5 denormalises to 0 — clobbering the default amp routing
and killing all audio.
Skip the setParam loop for modular when iml.exampleCount === 0, so the
worklet keeps running on the default patch we already pushed at init.
Once the user captures at least one training example the MLP has a
target to reproduce, and routing resumes normally.
Only affects modular mode; other engines are unchanged.
Two fixes for Modular mode:
1. faust-worklet-processor.js: explicitly attach FaustWorkletProcessor
to globalThis. Class declarations at the top of a classic script are
lexically scoped to that script's evaluation context and do not
propagate across separate addModule() calls, so the base class was
invisible to subclass processors when they loaded in AudioWorklet-
GlobalScope. Symptom: "FaustWorkletProcessor is not defined" at the
extends clause of modular-subtractive-processor.js.
2. modular-ui.js + a-app.js: wire live matrix cell updates. Phase C
wired matrix cells for writes (click cycles, precise editor) but
not for reads — the MLP-driven values never propagated to the DOM.
modular-ui now exposes updateLive(outputs), called from routeOutputs
on every inference tick. Throttled to ~20 fps internally to avoid
DOM thrashing. Only visible sources (within adsrCount/lfoCount) are
updated; muted slots stay dark.
If faust isn't in PATH but nix-shell is, transparently re-exec the
script via `nix-shell -p faust --run` instead of erroring out with a
hint. NISPS_BUILD_IN_NIX_SHELL guards against infinite recursion if the
shell somehow still lacks faust.
