w1n5t0n/memlnaut-nisps
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
memlnaut-nisps/docs/specs/MAIN.md
monkey-w1n5t0n 9ad1f78ddd docs: the specs disposition pass (plan §8) 
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.

  aimmersive-clone-spec      -> _archive/ with a deprecated-by note
  feedback-modes-port-spec   -> plans/, kind: plan, status: executed
  manifold-parity-features   -> plans/, kind: plan, status: active
  playground-2.0-rewrite     -> status: superseded
  engine-architecture        434 -> ~120 lines; seam + spine kept, rewritten
                             present-tense against the shipped engine/
  MAIN.md                    six contradicted claims fixed; registry resynced
  vcv-module.md              pruned to the current 8->16 contract and made the
                             single .nisps format spec
  vcv/NISPS-FORMAT.md        DELETED — documented a v1 format that no longer loads
  vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
  inputs/backends/dock trio  grounding sections marked historical, dead cites fixed

Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.

Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:

  manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
  SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
  shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
  axis that no longer exists — selection is now sandwich > particles >
  composite. This matters more than the rest: CLAUDE.md tells every agent to
  read ONBOARDING.md first for Manifold work, so it was actively teaching a
  fiction. Rewritten against ConsoleApp.tsx.

  MAP.md claimed the input layer reduces axes to the engine arity with an
  "even/odd blend". input-layer.ts says the opposite in its own header: one
  dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
  mean-blending was removed deliberately because it diluted every source.

  AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
  per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
  nisps::FixedBuffer (deleted).

Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.

Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00

13 KiB
Raw Blame History

kind stability layer
spec stable cross-cutting

MEMLNaut-NISPS Spec Corpus

Neural Interactive Shaping of Parameter Spaces — a research platform for interactive ML control of audio. One C++20 codebase compiles to two targets: RP2350 firmware for the MEMLNaut hardware, and WASM in the Manifold React browser app running the same engines + ML through an AudioWorklet. (The SolidJS playground was retired 2026-07-13, P1 of plans/one-core-engine-refactor.md; archived on archive/playground-solidjs.)

0. Spec Corpus Ontology

This corpus distinguishes four document kinds by temporal stance and mutability:

  • kind: spec (flat in docs/specs/) — timeless present-tense contracts. Wins over code by intent. Drift-lintable; updated when the spec itself changes or code breaks it.
  • kind: plan (in docs/specs/plans/) — future-tense, finite prescriptions carrying status: active | executed | superseded. Burn down as work lands; NEVER citable as authority for shipped behaviour (executed items migrate to MAP.md per the repo's doc-sync rule).
  • kind: finding (in docs/specs/recon/) — past-tense, dated, IMMUTABLE. Never edited, only superseded; exempt from drift lint. Research artifacts capturing ground truth at a specific moment.
  • ADRs (in docs/adr/) — decision rationale ("why we chose X"). Outside the spec root; used as backlinks from specs that encode those decisions.

Archived documents live in docs/specs/_archive/ with deprecated-by/superseded-by front matter.

Source Files (Architectural Roots)

The codebase's most architecturally significant areas referenced by this corpus:

Layer Path Purpose
Shared C++20 nisps/core/, nisps/ml/, nisps/dsp/, nisps/engines/, nisps/modes/, nisps/pipeline/ One audio+ML library compiling to both firmware and WASM
Schema/Codegen schemas/, codegen/ Parameter contracts + code generation (C++ headers, TS types)
Firmware firmware/MEMLNaut-NISPS/glue/ Hardware bindings (audio, peripherals, MIDI, settings)
Browser (Manifold) manifold/src/engine/, manifold/src/console/ Headless TS engine + AudioWorklet + the convertible React Console

1. Frame

What it is. MEMLNaut-NISPS is a single C++20 codebase that compiles once to two distinct targets:

  1. Firmware — runs on RP2350 hardware (MEMLNaut instrument). Real-time audio engines + interactive ML (4-layer MLP, SGD + RL feedback) with deterministic RNG + dual-core orchestration (audio on core 1, control on core 0).
  2. Browser (WASM) — runs in the React Manifold front-end via AudioWorklet. The same eight nisps/ engines as firmware — neither target is a superset. (The browser-only C15 synth lives only on the retired-playground archive branch; browser audio-input — mic — is not wired, so the analysis-driven modes are hardware-only today.)

The unifying constraint: one source tree, one ML architecture, cross-platform parity (native ↔ WASM within 1e-5 numerical tolerance). Parameter shapes are JSON schemas with codegen producing both C++ headers and TypeScript types.

Performance contract (applies to nisps/, inert in WASM but kept for consistency):

  • No heap in audio/inference paths; use std::array (the one deliberate exception: nisps/ml/dynamic_storage.hpp, browser/VCV-only, #errors on RP2350 builds, sole lint allowlist entry).
  • Float literals have the .f suffix; non-trivial float constants are static const.
  • No virtual dispatch in the audio path (C++20 concepts, not interfaces).
  • Deterministic per-instance RNG; all random state is local and seeded at construction.
  • Hot-path attributes from nisps/core/perf.hpp: NISPS_HOT / NISPS_FORCE_INLINE (the former SRAM-section macro regime was deleted 2026-07 as dead).

2. Cross-Cutting Contracts

Parity discipline

Native ↔ WASM bit-equivalence within 1e-5. The C++ core and its WASM compilation are asserted to produce identical outputs given the same input + same seed. See tests/cpp/parity_check.cpp (native test) + scripts/parity-check.sh (verify runner).

Schema/codegen contract

Each mode has a schemas/modes/<mode>.json describing parameters (name, label, range, default, curve, group), ML config (input/output sizes, hidden layers), voice spaces (names; bodies are inline lambdas in the engine), and UI config. The meta-schema at schemas/schema.json validates these. Codegen (bun run codegen/generate.ts) is idempotent and produces both language outputs:

  • nisps/modes/generated/<mode>_schema.hpp — constexpr C++ data under nisps::modes::generated (+ nisps/ml/generated/ml_defaults.hpp from schemas/ml_defaults.json).
  • manifold/src/modes/generated/<mode>_schema.ts (+ types.ts, index.ts) — the source of truth for the Manifold MF_MODES catalogue.

A second generator, codegen/generate-midi-devices.ts, emits nisps/midi/generated/midi_devices.hpp + manifold/src/midi-devices/generated/ from schemas/midi_devices/.

Schema changes ship with the regenerated C++ and TypeScript in the same commit; CI re-runs both generators and fails on any diff (plus the golden test).

No-heap + deterministic RNG

These are inert in WASM but kept globally to avoid surprises when porting. Lint catches violations: bash scripts/lint-cpp.sh warns on missing .f suffix and fails on heap use under nisps/ (allowlist: dynamic_storage.hpp), plus Arduino.h use outside firmware glue.

Sub-specs (Behavioural + Binding)

File Stability Layer Summary
backends-spec.md evolving binding Output backends (synth, particles, MIDI, OSC, CV/gate, VCV) unified behind one OutputBackend interface + manager. Largely built (manifold/src/backends/); carries a 2026-07 grounding note — the full-state OSC sync legs it designed were deleted.
dock-spec.md evolving behavioural Console dock drawers + the per-output control row (off/fixed/live tri-state, mute, solo/arm, min/max/curve). Built with operator restructuring (top Mode selector + 5 drawers, 2 depths); grounding note maps spec→shipped.
engine-architecture.md stable cross-cutting The surviving browser-engine contract: the headless EngineApi seam + the reactive-spine invariant, as realised in manifold/src/engine/. Trimmed 2026-07; superseded framing documented in its header.
inputs-spec.md evolving binding Modular input layer (XY pad, MIDI input, gamepad single/double-stick) composing into an N-dim vector. Built as manifold/src/inputs/ (runtime-reshaped net, exclusive-mode UI; mixing groundwork laid — operator decision §7.7 pending).
slp-workshop-firmware.md evolving binding SLP-Workshop (Synth Library Portland workshop build). Part I (Jolt + OU explore adaptive-learning gestures) shipped & stable. Part II (output-mode evolution, gate sequences, Manifold config) planned/evolving.
useq-cv-protocol.md stable binding uSEQ-CV wire protocol v2 (USB Web Serial ↔ uSEQ main module → CV/gate jacks + I2C expander). Single source of truth: firmware/useq-celium/shared/protocol.h, mirrored in manifold/src/backends/useq-protocol.ts.
vcv-module.md evolving binding MEMLNaut VCV Rack 2 module: CV-to-CV mapper (8 in × 16 out, LED rings) on the shared nisps/ core (P6). Also the single .nisps/patch format spec (v3 flat weights). Pruned 2026-07-21 to the current contract.

Archived (in _archive/): aimmersive-clone-spec.md — the SolidJS clone spec for the deployed vanilla a-immersive app; the clone was never built and its target framework was retired. Kept as a behavioural reference for the still-deployed vanilla app.

Plans (Finite, Burn-Down)

File Status Summary
plans/BUILD-PLAN.md executed Manifold build resume anchor (2026-06-27). The app shipped to /next/; survivors (non-snap-node e2e invocation, naming/copy decisions) noted in its header; deploy is now the CI-gated webhook.
plans/feedback-modes-port-spec.md executed The "Down Action" negative-feedback port into nisps/. Landed, then evolved past it — docs/adr/rl-feedback-design.md + nisps/ml/feedback.hpp are the surviving truth (the ADR explicitly supersedes its §2.5/§7 AVOID decision).
plans/manifold-parity-features-spec.md active Five playground features for Manifold. Jolt + OU-explore: executed by other means (one-core P3 core bindings). Session presets, pins, control surface: still the live prescription, feeding the curated/advanced split (simplification-plan §6.5c).
plans/playground-2.0-rewrite-plan.md superseded SolidJS clean-room rewrite plan (2026-06-17). Its target (the playground) was retired; many of its ideas shipped in Manifold instead. Still cited as design source by the parity-features plan.
plans/one-core-engine-refactor.md active Firmware+Manifold core reunification (2026-07-13). All six phases landed on main by 2026-07-18; stays active only for the two hardware-verification chokepoints (its §6) before flipping to executed.
plans/simplification-plan.md active Phased burn-down of the 2026-07-21 simplification audit. Phases 0 (CI/verification), 2 (behaviour bugs) and 3 (truth consolidation, minus S26) burned down 2026-07-21; Phase 1 deletions largely landed; PlatformIO (§5) in flight; §7 lists remaining operator decisions.

Findings (Immutable, Dated Research)

Exempt from drift lint.

File Date Summary
recon/findings-design-and-manifold.md 2026-06-27 Build-oriented brief synthesizing five redesign docs + Manifold token export + ConsoleApp.jsx. VERIFIED from source.
recon/findings-engine-surface.md 2026-06-27 Engine surface audit (deployed JS, WASM, C++ core). Uncovers fixed-2-input WASM gap, loss-history plumbing gap, C15 placeholder.
recon/findings-feedback-behaviour.md 2026-06-27 Current RL feedback behaviour audit. Deployed JS + C++ core use undirected Gaussian noise (not geometric push-away); firmware InterfaceRL implements the true geometric mode.
recon/midi-gamepad-inputs-worklog.md 2026-06-27 Work log of feat/midi-inputs branch. Modular input sources (MIDI, gamepad), 32-input WASM foundation, gamepad→verdict wiring. Exclusive picker (not mixing) shipped; mixing engine groundwork done but UI deferred.
recon/upstream-firmware-survey.md 2026-06-27 Git archaeology of the MusicallyEmbodiedML ecosystem. origin/main (C++20/SolidJS rewrite) and upstream/main (old .ino firmware) forked at 6efbe9c (2026-04-14); 49 commits upstream not in origin are ports, not merges.
recon/playground-2026.md 2026-04-12 Design intent snapshot of an unfinished playground UI redesign. Reference-only for SolidJS rewrite; do NOT merge into vanilla playground (those files moved on independently).
recon/simplification-audit-2026-07.md 2026-07-21 Full-repo smell/bloat/spec audit vs the five-bullet one-core vision (66-agent workflow, adversarially verified). 113 findings: CI red since 2026-07-13, memllib pin unpushed, ungated deploys, mode layer unshared, no curated/advanced split, dead-mass inventory. Mitigations in plans/simplification-plan.md.
recon/memllib-usage-inventory.md 2026-07-21 Per-file inventory of the memllib surface the firmware actually compiles — the prerequisite for the §7.5 ownership decision. Result: all of memllib bar examples/ is load-bearing (~1.8 MB / 84 files); no small subset to vendor.

ADRs (Decision Rationale)

File Decision Audience
adr/rl-feedback-design.md Rationale for the Explore-and-Place + Geometric-Dislike feedback model choices Implementation team

Cross-References

  • MAP.md — the neutral inventory of the codebase as it stands (ground truth for what exists).
  • CLAUDE.md — agent contract (gates, hard constraints); docs/AGENT-REFERENCE.md — the long-form reference.
  • ALIGNMENT.md — strategic gaps + open mission questions (dated, opinionated gap diagnosis).
  • manifold/ONBOARDING.md — agent-oriented breakdown of the Manifold React app (run/build/deploy/test, UI/engine-spine/WASM layering, convertible Stages, Dock+drawers, gotchas).
  • docs/redesign/manifold-export/ — design-token and UI-kit asset export from Manifold design project (not part of this corpus; static asset reference, not migrated).

Open / Deferred

  • Index generation (.index.json): intentionally not generated yet. No consumer defined; add when crawler/search infra lands.
  • Per-spec §-numbering normalisation: deferred to a future maintainer pass. Current state mixes flat (§1.1, §1.2…) and three-level nesting (§6.x.y).
  • ADR corpus: currently minimal (only RL feedback design). Strategic decisions captured in spec Open/Deferred sections; promotion to dedicated ADRs is a future pass.
  • Spec-touched validator: git-aware lint rule that flags when a code diff touches spec-cited files without also touching the spec. Useful as a pre-commit hook or PR check; infrastructure TBD.