Joystick -> MLP -> MIDI CC for an external hardware synth, using the
compile-time device templates from nisps/midi/generated. ExternalSynthMIDIMode
<const MidiDevice&, NOut> mirrors SoundAnalysisMIDIMode (NoOpEngine,
kRouteOutputsToEngine=false, pushes ControlChange events). A consteval
pick_cc_slots curates which NOut params fill the output slots (prefers musical
params over Bank Select/global). Adds six flashable variants
(MEMLNautModeExtSynth{Sub37,SubPhatty,Pro12,AnalogKeys,Hydrasynth,JD800}) wired
into mode_select.hpp + the .ino variant list + NISPS_ST guards, and a src/nisps/
midi symlink so the sketch tree can reach nisps/midi.
Host-compile-verified under C++20 (incl. via the sketch include path) and
lint-clean; full arduino-cli build + flash is on the hardware (no toolchain here.
Add a durable, committed source of truth for external MIDI synth control:
- synth-midi-cc.json: verified CC maps + provenance/sources (8 devices researched)
- schemas/midi_device.schema.json + schemas/midi_devices/*.json: 6 CC-controllable
device templates (Moog Sub 37/Sub Phatty, Creamware Pro-12 ASB, Elektron Analog
Keys, ASM Hydrasynth, Roland JD-800), params keyed {id, cc, label, min, max,
default, group}.
- codegen/generate-midi-devices.ts (isolated from the mode golden test) emits both
nisps/midi/generated/midi_devices.hpp (no-heap constexpr, firmware+WASM) and
manifold/src/midi-devices/generated/ (typed catalogue for the browser).
- codegen/seed-midi-devices.ts: reproducible seed from the research artifact.
Lets a performer pick a device and address its parameters by name (not CC number)
on both the firmware and the Manifold browser engine.
Stream 7 wires nisps/ml + nisps/engines into the playground via Emscripten.
Highlights:
- nisps/wasm/bindings.cpp: flat C API per architecture.md §6.2. Fixed-arch
MLP<2, 10, 14, 18, 126>; engine string→type dispatch table with NoOp
fallback.
- scripts/build-wasm.sh: emcc invocation, MODULARIZE=1, exports listed
explicitly; produces playground/public/nisps.{js,wasm}.
- playground/src/ml/wasm-iml.ts: main-thread MLP host (sync inference,
sync training, RL ops, weights I/O, layer stats, localStorage).
- playground/src/ml/wasm-worker.ts: disposable Web Worker for off-thread
async training, owns its own WASM instance.
- playground/src/ml/dataset.ts: Float32Array-backed FIFO with sample-weight
modes (uniform/global/local/combined). Port of legacy dataset.js.
- playground/src/audio/engine-host.ts: AudioContext + AudioWorkletNode
lifecycle, with start/stop/setEngine/setParams.
- playground/src/audio/worklet/nisps-processor.ts: WASM-loading
AudioWorkletProcessor that runs engine.process_block per 128-sample
block. Loads its own WASM instance from main-thread-supplied bytes
(no fetch in worklet).
- playground/src/stores/ml-store.ts: wired stub methods to WasmIML
singleton; lazy initialize().
- playground/src/debug/probe.ts: window.__nisps now calls real WasmIML
via the store; lazy-init on first use.
Verified:
- bash scripts/build-wasm.sh succeeds (94 KB nisps.wasm)
- bun run typecheck OK
- bun run build OK (production bundle)
- vite dev server serves /nisps.{js,wasm} with COOP/COEP
Known limitation: WASM is fixed at one MLP shape. Multi-arch deferred —
documented in nisps/wasm/README.md.
One header per mode, each ~50-130 LOC built atop ModeBase:
- `paf_synth.hpp` (4 inputs → 33 outputs, 7 voice spaces, note_on/off)
- `channel_strip.hpp` (4 → 24, 6 voice spaces)
- `xiasri.hpp` (4 → 24, 1 'Direct' voice space)
- `verb_fx.hpp` (4 → 47, 12 voice spaces)
- `memlcelium.hpp` (4 → 56, dual synth + 2-track sequencer; set_playing/update_bpm)
- `breakor.hpp` (4 → 56, 8-track ratio sequencer; pumps engine NoteOn/Off/Clock
events into ControlEvent ring buffer)
- `elysiamorf.hpp` (4 → 40, 8-track FM-pair MIDI-CC sequencer; pumps CC events)
- `sound_analysis_midi.hpp` (10 → 8; owns AnalysisEngine for input feature
extraction PLUS NoOpEngine 'thru' for audio passthrough; ML outputs
converted to MIDI CC events via on_post_inference; opts out of
output→engine routing via ModeRoutesOutputsToEngine specialisation)
Every mode satisfies `nisps::Mode` (verified via `static_assert` in each
header). Schema `output_size` is verified against engine `param_count()`
at compile time inside ModeBase.
All hardware-specific glue (MEMLNaut::Instance, pico/util/queue, MIDIInOut,
display widgets, button callbacks) is intentionally absent — that lives in
firmware/glue and playground/src/modes per architecture.md §4.3.
Adds two foundational pieces shared by every concrete mode:
- `base.hpp`: defines `nisps::ParamSchema` (the aggregate type the
`Mode` concept's `param_schema()` returns by const-reference),
`nisps::ControlEvent`, and `ModeBase<Derived, EngineT, MLPType,
NInputs>` — a CRTP scaffold absorbing input forwarding, ML
inference + voice space mapping, engine.set_params() routing, and
a per-mode ControlEvent ring buffer for MIDI/I2C events.
`ModeRoutesOutputsToEngine<Derived>` lets a derived mode (e.g.
SoundAnalysisMIDI) opt out of routing ML outputs to engine
params when its outputs become MIDI CCs instead.
- `voice_space.hpp`: `VoiceSpaceEntry` + `make_voice_space_entries()`
+ `find_voice_space()` helpers. Voice-space *mapping code* lives
inside engines (mirroring firmware); mode layer just selects
which voice space the engine uses by index/name.
No heap, no virtuals, no Pico/Arduino dependencies; satisfies stream
4 of architecture.md.
Stream 3's NoOpEngine used engine_id()=='noop' but the schema for
sound_analysis_midi.json declares engine_id: 'thru' (matches firmware's
ThruAudioApp naming). The class name stays NoOpEngine — it accurately
describes what process() does — but the schema-facing identifier is now
'thru'. BreakOr/Elysiamorf compose NoOpEngine by type, not engine_id, so
they're unaffected.
Stream 5 shipped a temporary local Curve enum (PascalCase) while stream 1
was in flight. Stream 1 has now landed nisps/core/math.hpp with the
authoritative lowercase Curve enum per architecture spec. Generated C++
headers now include core/math.hpp and re-export the enum into the
generated namespace via 'using Curve = ::nisps::Curve;'.
- codegen/generate.ts: emit lowercase enum values + include math.hpp
- regenerated all 8 mode schema headers
- updated golden snapshot to match
Verified all 8 headers compile clean with -std=c++20 -Wall -Wextra.
Output of \`bun run codegen/generate.ts\` for the 8 mode schemas.
These files are checked in so consumers don't need bun on every
build, but they remain regenerable and byte-identical.
C++ (nisps/modes/generated/):
schema_types.hpp + 8 <mode_id>_schema.hpp files. All compile
cleanly with g++ -std=c++20 -Wall -Wextra -fsyntax-only -I nisps.
TS (playground/src/modes/generated/):
types.ts + index.ts + 8 <mode_id>_schema.ts files. Type-check
cleanly with tsc --strict.
CMakeLists.txt:
- Native host build by default; Emscripten-target detection plumbed but
WASM emit deferred to stream 7 (playground build script).
- Header-only INTERFACE library `nisps_core`.
- Host test executable `nisps_core_tests` compiled with -Wall -Wextra
-Werror -Wpedantic (Chris's rules: clean build is non-negotiable).
tests/cpp/test_helpers.hpp:
- Minimal NISPS_TEST / NISPS_EXPECT / NISPS_EXPECT_NEAR macros, no external
deps. Rationale documented in-file: Catch2/doctest would add ~10MB and 30s
for what is currently <100 LOC of test runtime.
22 unit tests covering FixedBuffer (5), RingBuffer (5), Rng (7), math (5).
All green; verified via `cmake --build nisps/build && ./nisps/build/nisps_core_tests`.
