feat(firmware): ExternalSynthMIDIMode + 6 device variants

Joystick -> MLP -> MIDI CC for an external hardware synth, using the
compile-time device templates from nisps/midi/generated. ExternalSynthMIDIMode
<const MidiDevice&, NOut> mirrors SoundAnalysisMIDIMode (NoOpEngine,
kRouteOutputsToEngine=false, pushes ControlChange events). A consteval
pick_cc_slots curates which NOut params fill the output slots (prefers musical
params over Bank Select/global). Adds six flashable variants
(MEMLNautModeExtSynth{Sub37,SubPhatty,Pro12,AnalogKeys,Hydrasynth,JD800}) wired
into mode_select.hpp + the .ino variant list + NISPS_ST guards, and a src/nisps/
midi symlink so the sketch tree can reach nisps/midi.

Host-compile-verified under C++20 (incl. via the sketch include path) and
lint-clean; full arduino-cli build + flash is on the hardware (no toolchain here.
This commit is contained in:
monkey-w1n5t0n 2026-06-28 20:13:08 +02:00
parent b3c0e71bb7
commit 5d3785a6a8
5 changed files with 184 additions and 2 deletions

4
MAP.md
View file

@ -9,7 +9,7 @@ MEMLNaut-NISPS — Neural Interactive Shaping of Parameter Spaces. One C++20 cod
- `nisps/ml/` — MLP class template `MLP<NIn, NH1, NH2, NH3, NOut>`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (SGD + grad clipping), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `feedback.hpp` (`FeedbackController<MLP_T>` — the 3-mode "Down Action" negative-feedback state machine: Avoid / RandomiseOutputs / RandomiseMlp; header-only, zero-heap, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `stats.hpp`.
- `nisps/dsp/``biquad.hpp`, `delay.hpp`, `reverb.hpp`, `filter.hpp`, `env.hpp`, `osc.hpp`, `pitch_shift.hpp`, `dc_blocker.hpp`. Lean primitives extracted from maximilian; daisysp PitchShifter replaced with custom granular impl.
- `nisps/engines/` — eight audio engines, each satisfying `AudioEngine`: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp` (sequencer, NoOp audio), `elysiamorf.hpp` (sequencer, NoOp audio), `analysis.hpp` (input-side spectral features). Plus `base.hpp` (`NoOpEngine`, engine_id "thru").
- `nisps/modes/` eight platform-agnostic modes binding `{ML config, engine, voice space lambdas, abstract I/O channels}`. Files: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp`, `elysiamorf.hpp`, `sound_analysis_midi.hpp`. `base.hpp` provides a CRTP scaffold eliminating the duplication that previously plagued firmware modes. `voice_space.hpp` holds engine-side voice space dispatch helpers. `generated/` contains codegen output (do not edit by hand).
- `nisps/modes/` — platform-agnostic modes binding `{ML config, engine, voice space lambdas, abstract I/O channels}`. Files: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp`, `elysiamorf.hpp`, `sound_analysis_midi.hpp`, `external_synth_midi.hpp` (`ExternalSynthMIDIMode<const MidiDevice&, NOut>` — joystick→MLP→MIDI CC for an external synth; compile-time device from `nisps/midi`; `consteval pick_cc_slots` curates which params fill the NOut slots; NoOpEngine, `kRouteOutputsToEngine=false`). `base.hpp` provides a CRTP scaffold eliminating the duplication that previously plagued firmware modes. `voice_space.hpp` holds engine-side voice space dispatch helpers. `generated/` contains codegen output (do not edit by hand).
- `nisps/wasm/bindings.cpp` — flat C API exported to WASM (Emscripten target only).
- `nisps/midi/generated/midi_devices.hpp` — codegen output: no-heap `constexpr` external-MIDI-synth templates (`nisps::midi::generated`; `MidiParam`/`MidiDevice` + `kMidiDevices` registry). Source = `schemas/midi_devices/`; do not edit by hand.
- `nisps/CMakeLists.txt` + `nisps/build/` — host-target builds + ctest.
@ -20,7 +20,7 @@ MEMLNaut-NISPS — Neural Interactive Shaping of Parameter Spaces. One C++20 cod
- `audio_driver.hpp` — bridges memllib `AudioDriver` callback → `Mode::process(stereosample_t)`.
- `peripherals.hpp` — joystick / pots / buttons → `Mode::set_input` and ML primitives.
- `midi_io.hpp` — MIDI in → mode `note_on`/`update_bpm`/`set_playing`; drains `ControlEvent` ring → MIDI UART.
- `mode_select.hpp` — type aliases mapping firmware mode identifiers to `nisps::modes::*Mode` C++ types. Build script rewrites the active line. Also defines the `MEMLNautModeSelfTest` pseudo-variant (tag type) + the `NISPS_ST_*`/`NISPS_ST_CAT` token-paste macros the `.ino` uses to compute `NISPS_SELFTEST`.
- `mode_select.hpp` — type aliases mapping firmware mode identifiers to `nisps::modes::*Mode` C++ types. Build script rewrites the active line. Includes the six `MEMLNautModeExtSynth*` external-synth variants (one per device template in `nisps/midi`, e.g. `MEMLNautModeExtSynthSub37`). Also defines the `MEMLNautModeSelfTest` pseudo-variant (tag type) + the `NISPS_ST_*`/`NISPS_ST_CAT` token-paste macros the `.ino` uses to compute `NISPS_SELFTEST`. Note: `src/nisps/` exposes each referenced top-level nisps subdir as a symlink — `midi` was added alongside `core/dsp/engines/ml/modes`.
- `selftest.hpp` — standalone guided hardware self-test rig (`SelfTest` variant; no engine/ML). Step-driven state machine on a `SelfTestView`: TFT prompts the operator through every control, auto-advances on detection, encoder-press skips. Ends with optional L/R/BOTH sine-sweep headphone check (core 1 block callback) + MIDI loopback-cable test. Lives firmware-side (touches TFT + raw pins) so it stays out of platform-agnostic `nisps/`.
- `input_router.hpp`, `output_router.hpp` — top-level `wire_inputs()` / `drain_outputs()` entry points.
- `firmware/MEMLNaut-NISPS/src/{memllib,daisysp,nisps}` — symlinks (Arduino-CLI requires sketch-tree includes; preprocessor refuses `..` in headers).

View file

@ -42,6 +42,13 @@
// #define MEMLNAUT_MODE_TYPE MEMLNautModeChannelStrip
#define MEMLNAUT_MODE_TYPE MEMLNautModePAFSynth
// #define MEMLNAUT_MODE_TYPE MEMLNautModeMEMLCelium
// ---- External-synth MIDI-CC variants (control an external hardware synth) ----
// #define MEMLNAUT_MODE_TYPE MEMLNautModeExtSynthSub37
// #define MEMLNAUT_MODE_TYPE MEMLNautModeExtSynthSubPhatty
// #define MEMLNAUT_MODE_TYPE MEMLNautModeExtSynthPro12
// #define MEMLNAUT_MODE_TYPE MEMLNautModeExtSynthAnalogKeys
// #define MEMLNAUT_MODE_TYPE MEMLNautModeExtSynthHydrasynth
// #define MEMLNAUT_MODE_TYPE MEMLNautModeExtSynthJD800
// #define MEMLNAUT_MODE_TYPE MEMLNautModeSelfTest
// NISPS_SELFTEST == 1 iff the selected variant is the guided hardware self-test

View file

@ -41,6 +41,7 @@
#include "../src/nisps/modes/breakor.hpp"
#include "../src/nisps/modes/channel_strip.hpp"
#include "../src/nisps/modes/elysiamorf.hpp"
#include "../src/nisps/modes/external_synth_midi.hpp"
#include "../src/nisps/modes/memlcelium.hpp"
#include "../src/nisps/modes/paf_synth.hpp"
#include "../src/nisps/modes/sound_analysis_midi.hpp"
@ -58,6 +59,17 @@ using MEMLNautModeVerbFX = ::nisps::modes::VerbFXMode;
using MEMLNautModeElysiamorfs = ::nisps::modes::ElysiamorfMode;
using MEMLNautModeMEMLCelium = ::nisps::modes::MEMLCeliumMode;
// ---- External-synth MIDI-CC variants (one flashable variant per device) ----
// Joystick -> MLP -> MIDI CC for the named external hardware synth. Device
// templates live in nisps/midi/generated/midi_devices.hpp (source of truth:
// schemas/midi_devices/). Each drives a curated 8-param subset (pick_cc_slots).
using MEMLNautModeExtSynthSub37 = ::nisps::modes::ExternalSynthMIDIMode<::nisps::midi::generated::kMoogSub37, 8u>;
using MEMLNautModeExtSynthSubPhatty = ::nisps::modes::ExternalSynthMIDIMode<::nisps::midi::generated::kMoogSubPhatty, 8u>;
using MEMLNautModeExtSynthPro12 = ::nisps::modes::ExternalSynthMIDIMode<::nisps::midi::generated::kCreamwarePro12Asb, 8u>;
using MEMLNautModeExtSynthAnalogKeys = ::nisps::modes::ExternalSynthMIDIMode<::nisps::midi::generated::kElektronAnalogKeys, 8u>;
using MEMLNautModeExtSynthHydrasynth = ::nisps::modes::ExternalSynthMIDIMode<::nisps::midi::generated::kAsmHydrasynth, 8u>;
using MEMLNautModeExtSynthJD800 = ::nisps::modes::ExternalSynthMIDIMode<::nisps::midi::generated::kRolandJd800, 8u>;
// ---- SelfTest pseudo-variant ----
// A standalone guided hardware self-test (see glue/selftest.hpp). It is NOT a
// nisps Mode — it drives the display + raw peripherals directly. We expose a
@ -78,6 +90,12 @@ using MEMLNautModeSelfTest = ::nisps_firmware::selftest::SelfTestRig;
#define NISPS_ST_MEMLNautModeVerbFX 0
#define NISPS_ST_MEMLNautModeElysiamorfs 0
#define NISPS_ST_MEMLNautModeMEMLCelium 0
#define NISPS_ST_MEMLNautModeExtSynthSub37 0
#define NISPS_ST_MEMLNautModeExtSynthSubPhatty 0
#define NISPS_ST_MEMLNautModeExtSynthPro12 0
#define NISPS_ST_MEMLNautModeExtSynthAnalogKeys 0
#define NISPS_ST_MEMLNautModeExtSynthHydrasynth 0
#define NISPS_ST_MEMLNautModeExtSynthJD800 0
#define NISPS_ST_MEMLNautModeSelfTest 1
#define NISPS_ST_CAT_(x) NISPS_ST_##x
#define NISPS_ST_CAT(x) NISPS_ST_CAT_(x)

View file

@ -0,0 +1 @@
../../../../nisps/midi

View file

@ -0,0 +1,156 @@
// nisps/modes/external_synth_midi.hpp — Drive an external MIDI synth via CC.
//
// A platform-agnostic mode that maps the abstract joystick inputs through the
// MLP to MIDI Control Change messages for an EXTERNAL hardware synth, using a
// compile-time-selected device template from nisps/midi/generated/midi_devices.hpp
// (the same templates the Manifold browser picker uses).
//
// - Audio engine = NoOpEngine (engine_id "thru"); no audio is produced.
// - ML inputs (4) = the abstract joystick channels (joy_x/y/z/w).
// - ML outputs (NOut) = MIDI CC values pushed into the ControlEvent ring for
// the firmware glue to dispatch (output_router -> midi_io::queueCC).
// - Device is chosen at COMPILE TIME (one flashable variant per synth, e.g.
// MEMLNautModeExtSynthSub37). Which of the device's params occupy the NOut
// output slots is chosen by `pick_cc_slots` (prefers musical params over
// Bank Select / global housekeeping CCs). A runtime `enabled_` mask can
// mute individual slots (set_param_enabled); rich per-param selection is the
// browser's job — the hardware drives this curated subset.
//
// `kRouteOutputsToEngine = false` (specialised below) so ModeBase skips routing
// ML outputs into engine params, exactly like SoundAnalysisMIDIMode.
#pragma once
#include <array>
#include <cstddef>
#include <cstdint>
#include <span>
#include <string_view>
#include "../core/concepts.hpp"
#include "../core/perf.hpp"
#include "../core/types.hpp"
#include "../engines/base.hpp" // NoOpEngine
#include "../midi/generated/midi_devices.hpp"
#include "../ml/mlp.hpp"
#include "base.hpp"
namespace nisps::modes {
template <const ::nisps::midi::generated::MidiDevice& Device, std::size_t NOut>
class ExternalSynthMIDIMode;
// Compile-time slot picker: choose which NOut device params the MLP drives.
// Prefers "musical" params (any group except "global"); fills any remaining
// slots from the leftover (global) params. Deterministic, zero runtime cost.
template <const ::nisps::midi::generated::MidiDevice& Device, std::size_t NOut>
consteval std::array<std::uint8_t, NOut> pick_cc_slots() {
std::array<std::uint8_t, NOut> slots{};
std::size_t n = 0u;
for (std::size_t i = 0u; i < Device.params.size() && n < NOut; ++i) {
if (Device.params[i].group != std::string_view{"global"}) {
slots[n++] = static_cast<std::uint8_t>(i);
}
}
for (std::size_t i = 0u; i < Device.params.size() && n < NOut; ++i) {
if (Device.params[i].group == std::string_view{"global"}) {
slots[n++] = static_cast<std::uint8_t>(i);
}
}
return slots;
}
} // namespace nisps::modes
namespace nisps {
// Specialise: this mode does not route ML outputs to engine params.
template <const ::nisps::midi::generated::MidiDevice& D, std::size_t N>
struct ModeRoutesOutputsToEngine<modes::ExternalSynthMIDIMode<D, N>> : std::false_type {};
} // namespace nisps
namespace nisps::modes {
template <const ::nisps::midi::generated::MidiDevice& Device, std::size_t NOut = 8u>
class ExternalSynthMIDIMode : public ModeBase<
ExternalSynthMIDIMode<Device, NOut>,
NoOpEngine,
ml::MLP<4u, 10u, 14u, 18u, NOut>,
4u> {
static_assert(Device.params.size() >= NOut,
"device template has fewer params than the mode's output count");
public:
using Base = ModeBase<ExternalSynthMIDIMode<Device, NOut>, NoOpEngine,
ml::MLP<4u, 10u, 14u, 18u, NOut>, 4u>;
using Base::Base;
static constexpr std::string_view mode_id() noexcept { return Device.device_id; }
static constexpr const ParamSchema& param_schema() noexcept { return kSchema; }
// Map the NOut MLP outputs onto the selected device CCs and queue them.
void on_post_inference() noexcept {
const auto outs = this->ml_.outputs();
const std::uint8_t ch =
Device.default_channel > 0u ? static_cast<std::uint8_t>(Device.default_channel - 1u) : 0u;
for (std::size_t i = 0u; i < NOut && i < outs.size(); ++i) {
if (!enabled_[i]) continue;
const auto& p = Device.params[kSlots[i]];
float v = outs[i];
if (v < 0.f) v = 0.f;
else if (v > 1.f) v = 1.f;
const float lo = static_cast<float>(p.min);
const float hi = static_cast<float>(p.max);
ControlEvent ce{};
ce.kind = ControlEvent::Kind::ControlChange;
ce.channel = ch;
ce.data1 = p.cc;
ce.data2 = static_cast<std::uint8_t>(lo + v * (hi - lo) + 0.5f);
(void)this->push_control_event(ce);
}
}
// Runtime mute of an individual output slot (hardware has no rich selector;
// this is the hook a future TFT menu / glue would drive).
void set_param_enabled(std::size_t slot, bool on) noexcept {
if (slot < NOut) enabled_[slot] = on;
}
static constexpr std::size_t slot_count() noexcept { return NOut; }
// The device param backing output slot i (for glue / display by name).
static constexpr const ::nisps::midi::generated::MidiParam& slot_param(std::size_t i) noexcept {
return Device.params[kSlots[i]];
}
private:
static constexpr auto kSlots = pick_cc_slots<Device, NOut>();
std::array<bool, NOut> enabled_ = [] {
std::array<bool, NOut> a{};
for (auto& b : a) b = true;
return a;
}();
static constexpr std::array<std::string_view, 4> kInputs{
std::string_view{"joy_x"}, std::string_view{"joy_y"},
std::string_view{"joy_z"}, std::string_view{"joy_w"}};
static constexpr std::array<std::size_t, 3> kHidden{10u, 14u, 18u};
static constexpr std::array<generated::Param, 0> kNoParams{};
static constexpr std::array<std::string_view, 0> kNoVoiceSpaces{};
static constexpr generated::UIConfig kUI{generated::PrimaryInput::Joystick, false, false};
static inline constexpr ParamSchema kSchema = ParamSchema{
Device.device_id,
std::string_view{"thru"},
std::span<const std::string_view>(kInputs),
4u,
std::span<const std::size_t>(kHidden),
NOut,
0.6f,
1.0f,
1000u,
std::span<const generated::Param>(kNoParams),
std::span<const std::string_view>(kNoVoiceSpaces),
kUI,
};
};
} // namespace nisps::modes