memlnaut-nisps/docs/specs/plans/manifold-parity-features-spec.md

487 lines
24 KiB
Markdown
Raw Normal View History

---
docs: the specs disposition pass (plan §8) Roughly 20k lines were deleted from this repo in the last week and much of the corpus still described the pre-deletion world in the present tense. Executes the §8 table: archive the retired, reclassify the executed, prune the stale. aimmersive-clone-spec -> _archive/ with a deprecated-by note feedback-modes-port-spec -> plans/, kind: plan, status: executed manifold-parity-features -> plans/, kind: plan, status: active playground-2.0-rewrite -> status: superseded engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten present-tense against the shipped engine/ MAIN.md six contradicted claims fixed; registry resynced vcv-module.md pruned to the current 8->16 contract and made the single .nisps format spec vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table inputs/backends/dock trio grounding sections marked historical, dead cites fixed Two rows of the §8 table were themselves wrong, corrected here: the deleted full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no §6.3), and codegen/README.md was already a MAP pointer with no port-solidjs trigger left to remove. Beyond the table — found by sweeping every backticked path in the changed docs against `git ls-files`, which is how these should have been caught before: manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current: SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a shot.spec.ts that does not exist. The whole stage table was keyed on a `focus` axis that no longer exists — selection is now sandwich > particles > composite. This matters more than the rest: CLAUDE.md tells every agent to read ONBOARDING.md first for Manifold work, so it was actively teaching a fiction. Rewritten against ConsoleApp.tsx. MAP.md claimed the input layer reduces axes to the engine arity with an "even/odd blend". input-layer.ts says the opposite in its own header: one dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and mean-blending was removed deliberately because it diluted every source. AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed), per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at nisps::FixedBuffer (deleted). Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/ Delta and cachedNovelty behind a live context-menu toggle that nothing reads; vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to 9000, so out of the box they do not meet. Firmware-build docs are deliberately untouched — the PlatformIO migration lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
kind: plan
status: active
---
# Manifold Parity Features — Session Presets · Pins · Jolt · OU-Explore · Control Surface
docs: the specs disposition pass (plan §8) Roughly 20k lines were deleted from this repo in the last week and much of the corpus still described the pre-deletion world in the present tense. Executes the §8 table: archive the retired, reclassify the executed, prune the stale. aimmersive-clone-spec -> _archive/ with a deprecated-by note feedback-modes-port-spec -> plans/, kind: plan, status: executed manifold-parity-features -> plans/, kind: plan, status: active playground-2.0-rewrite -> status: superseded engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten present-tense against the shipped engine/ MAIN.md six contradicted claims fixed; registry resynced vcv-module.md pruned to the current 8->16 contract and made the single .nisps format spec vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table inputs/backends/dock trio grounding sections marked historical, dead cites fixed Two rows of the §8 table were themselves wrong, corrected here: the deleted full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no §6.3), and codegen/README.md was already a MAP pointer with no port-solidjs trigger left to remove. Beyond the table — found by sweeping every backticked path in the changed docs against `git ls-files`, which is how these should have been caught before: manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current: SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a shot.spec.ts that does not exist. The whole stage table was keyed on a `focus` axis that no longer exists — selection is now sandwich > particles > composite. This matters more than the rest: CLAUDE.md tells every agent to read ONBOARDING.md first for Manifold work, so it was actively teaching a fiction. Rewritten against ConsoleApp.tsx. MAP.md claimed the input layer reduces axes to the engine arity with an "even/odd blend". input-layer.ts says the opposite in its own header: one dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and mean-blending was removed deliberately because it diluted every source. AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed), per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at nisps::FixedBuffer (deleted). Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/ Delta and cachedNovelty behind a live context-menu toggle that nothing reads; vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to 9000, so out of the box they do not meet. Firmware-build docs are deliberately untouched — the PlatformIO migration lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
*Date: 2026-07-12; status updated 2026-07-21. Prescriptive plan for porting five playground
features into `manifold/`. Now part-executed:*
- ***§3 Jolt and §4 OU-explore: EXECUTED by other means*** *— one-core-engine P3 landed the core
bindings (this document's own preferred route, §0 principle 1). The shipped C ABI is
`nisps_ml_jolt_{press,step,release,active}` and `nisps_ml_explore_{intensity,get_intensity,apply}`
— not the §3.2/§4.2 names proposed below — driven by `manifold/src/engine/exploration.ts`.
§3/§4 are kept as behavioural reference; the code wins.*
- ***§1 session presets, §2 pins, §5 control surface: NOT implemented.*** *These remain the live
prescription, feeding the curated/advanced-split work (`simplification-plan.md` §6.5c).*
**Traces up to:** `playground-2.0-rewrite-plan.md` (superseded plan — its target died but these
sections are still the cited design source) §2.3 (Feel drawer), §2.6 (pins), §3.2 (fanout,
control-point tri-state), §3.8 (composed-layers presets, persist helper);
`../slp-workshop-firmware.md` §34 (Jolt / OU-explore, Part I shipped in firmware);
`../dock-spec.md` (drawer depths, tri-state semantics); BUILD-PLAN locked decisions
(parity-tested engine; British spelling; the synth is "Powerful Synth Engine", never the forbidden
string).
**Reference implementations:** the playground versions are the behavioural ground truth for
docs: the specs disposition pass (plan §8) Roughly 20k lines were deleted from this repo in the last week and much of the corpus still described the pre-deletion world in the present tense. Executes the §8 table: archive the retired, reclassify the executed, prune the stale. aimmersive-clone-spec -> _archive/ with a deprecated-by note feedback-modes-port-spec -> plans/, kind: plan, status: executed manifold-parity-features -> plans/, kind: plan, status: active playground-2.0-rewrite -> status: superseded engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten present-tense against the shipped engine/ MAIN.md six contradicted claims fixed; registry resynced vcv-module.md pruned to the current 8->16 contract and made the single .nisps format spec vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table inputs/backends/dock trio grounding sections marked historical, dead cites fixed Two rows of the §8 table were themselves wrong, corrected here: the deleted full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no §6.3), and codegen/README.md was already a MAP pointer with no port-solidjs trigger left to remove. Beyond the table — found by sweeping every backticked path in the changed docs against `git ls-files`, which is how these should have been caught before: manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current: SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a shot.spec.ts that does not exist. The whole stage table was keyed on a `focus` axis that no longer exists — selection is now sandwich > particles > composite. This matters more than the rest: CLAUDE.md tells every agent to read ONBOARDING.md first for Manifold work, so it was actively teaching a fiction. Rewritten against ConsoleApp.tsx. MAP.md claimed the input layer reduces axes to the engine arity with an "even/odd blend". input-layer.ts says the opposite in its own header: one dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and mean-blending was removed deliberately because it diluted every source. AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed), per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at nisps::FixedBuffer (deleted). Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/ Delta and cachedNovelty behind a live context-menu toggle that nothing reads; vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to 9000, so out of the box they do not meet. Firmware-build docs are deliberately untouched — the PlatformIO migration lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
constants and algorithms — `src/features/session-preset.ts`, `src/stores/session-store.ts` (pins),
`src/ml/jolt.ts`, `src/output/ou-explore.ts`, `src/stores/control-store.ts` +
`src/features/control-routing.ts`, all on the retired-playground archive (branch
`archive/playground-solidjs`, tag `playground-solidjs-final` — these files no longer exist on
main). Where this spec and the playground disagree, this spec wins (each divergence is called out
and justified inline).
---
## 0. Principles applied throughout
1. **C++ owns gesture math where a C++ class exists.** `nisps::ml::Jolt` and
docs: the specs disposition pass (plan §8) Roughly 20k lines were deleted from this repo in the last week and much of the corpus still described the pre-deletion world in the present tense. Executes the §8 table: archive the retired, reclassify the executed, prune the stale. aimmersive-clone-spec -> _archive/ with a deprecated-by note feedback-modes-port-spec -> plans/, kind: plan, status: executed manifold-parity-features -> plans/, kind: plan, status: active playground-2.0-rewrite -> status: superseded engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten present-tense against the shipped engine/ MAIN.md six contradicted claims fixed; registry resynced vcv-module.md pruned to the current 8->16 contract and made the single .nisps format spec vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table inputs/backends/dock trio grounding sections marked historical, dead cites fixed Two rows of the §8 table were themselves wrong, corrected here: the deleted full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no §6.3), and codegen/README.md was already a MAP pointer with no port-solidjs trigger left to remove. Beyond the table — found by sweeping every backticked path in the changed docs against `git ls-files`, which is how these should have been caught before: manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current: SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a shot.spec.ts that does not exist. The whole stage table was keyed on a `focus` axis that no longer exists — selection is now sandwich > particles > composite. This matters more than the rest: CLAUDE.md tells every agent to read ONBOARDING.md first for Manifold work, so it was actively teaching a fiction. Rewritten against ConsoleApp.tsx. MAP.md claimed the input layer reduces axes to the engine arity with an "even/odd blend". input-layer.ts says the opposite in its own header: one dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and mean-blending was removed deliberately because it diluted every source. AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed), per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at nisps::FixedBuffer (deleted). Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/ Delta and cachedNovelty behind a live context-menu toggle that nothing reads; vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to 9000, so out of the box they do not meet. Firmware-build docs are deliberately untouched — the PlatformIO migration lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
`nisps::ml::OUNoise<N>` (`nisps/ml/jolt.hpp`, `nisps/ml/ou_noise.hpp`) are bound to WASM
rather than re-ported to TS — deterministic, firmware-parity noise for free. *(Done — this
landed via one-core-engine P3; see the status block above.)*
2. **No new mechanisms where an existing one already expresses the idea.** Param pins are the
existing `off|fixed|live` tri-state + arm mask, not a parallel pin system (§2.2).
3. **Transparent defaults.** Every default constant appears in this spec as a number with its
source, and the control surface UI must *show* the derived per-param values live (§5.6) —
the user should never wonder what an axis is secretly doing.
4. **One persistence pattern.** All new persisted state goes through a single versioned
`persist<T>(key, version, migrate)` helper modelled on `settings-store.ts`
docs: the specs disposition pass (plan §8) Roughly 20k lines were deleted from this repo in the last week and much of the corpus still described the pre-deletion world in the present tense. Executes the §8 table: archive the retired, reclassify the executed, prune the stale. aimmersive-clone-spec -> _archive/ with a deprecated-by note feedback-modes-port-spec -> plans/, kind: plan, status: executed manifold-parity-features -> plans/, kind: plan, status: active playground-2.0-rewrite -> status: superseded engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten present-tense against the shipped engine/ MAIN.md six contradicted claims fixed; registry resynced vcv-module.md pruned to the current 8->16 contract and made the single .nisps format spec vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table inputs/backends/dock trio grounding sections marked historical, dead cites fixed Two rows of the §8 table were themselves wrong, corrected here: the deleted full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no §6.3), and codegen/README.md was already a MAP pointer with no port-solidjs trigger left to remove. Beyond the table — found by sweeping every backticked path in the changed docs against `git ls-files`, which is how these should have been caught before: manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current: SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a shot.spec.ts that does not exist. The whole stage table was keyed on a `focus` axis that no longer exists — selection is now sandwich > particles > composite. This matters more than the rest: CLAUDE.md tells every agent to read ONBOARDING.md first for Manifold work, so it was actively teaching a fiction. Rewritten against ConsoleApp.tsx. MAP.md claimed the input layer reduces axes to the engine arity with an "even/odd blend". input-layer.ts says the opposite in its own header: one dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and mean-blending was removed deliberately because it diluted every source. AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed), per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at nisps::FixedBuffer (deleted). Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/ Delta and cachedNovelty behind a live context-menu toggle that nothing reads; vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to 9000, so out of the box they do not meet. Firmware-build docs are deliberately untouched — the PlatformIO migration lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
(localStorage, debounced 200 ms), per `playground-2.0-rewrite-plan.md` §3.8.
5. **Probe parity.** Each feature activates its currently-inert `window.__nisps` methods
(`manifold/src/debug/probe.ts`) so Playwright can drive it headlessly. New e2e specs are part
of each feature's acceptance criteria.
---
## 1. Session presets (composed layers)
### 1.1 Model
docs: the specs disposition pass (plan §8) Roughly 20k lines were deleted from this repo in the last week and much of the corpus still described the pre-deletion world in the present tense. Executes the §8 table: archive the retired, reclassify the executed, prune the stale. aimmersive-clone-spec -> _archive/ with a deprecated-by note feedback-modes-port-spec -> plans/, kind: plan, status: executed manifold-parity-features -> plans/, kind: plan, status: active playground-2.0-rewrite -> status: superseded engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten present-tense against the shipped engine/ MAIN.md six contradicted claims fixed; registry resynced vcv-module.md pruned to the current 8->16 contract and made the single .nisps format spec vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table inputs/backends/dock trio grounding sections marked historical, dead cites fixed Two rows of the §8 table were themselves wrong, corrected here: the deleted full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no §6.3), and codegen/README.md was already a MAP pointer with no port-solidjs trigger left to remove. Beyond the table — found by sweeping every backticked path in the changed docs against `git ls-files`, which is how these should have been caught before: manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current: SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a shot.spec.ts that does not exist. The whole stage table was keyed on a `focus` axis that no longer exists — selection is now sandwich > particles > composite. This matters more than the rest: CLAUDE.md tells every agent to read ONBOARDING.md first for Manifold work, so it was actively teaching a fiction. Rewritten against ConsoleApp.tsx. MAP.md claimed the input layer reduces axes to the engine arity with an "even/odd blend". input-layer.ts says the opposite in its own header: one dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and mean-blending was removed deliberately because it diluted every source. AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed), per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at nisps::FixedBuffer (deleted). Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/ Delta and cachedNovelty behind a live context-menu toggle that nothing reads; vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to 9000, so out of the box they do not meet. Firmware-build docs are deliberately untouched — the PlatformIO migration lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
Adopt the composed-layers model prescribed by `playground-2.0-rewrite-plan.md` §3.8 rather than
the playground's monolithic `SessionPresetPayload`. A preset is a bundle of independently optional
layers:
```ts
interface SessionPresetV1 {
v: 1;
id: string; // `preset-${epoch36}`
name: string;
createdAt: number;
layers: {
control?: ControlLayer; // axes + offsets + presetId (§5)
pipelines?: PipelineLayer; // input/output pipeline params
routing?: RoutingLayer; // per-output MFParam state/min/max/curve/mute + backend specs
mode?: ModeLayer; // modeId + outputMode
weights?: string; // base64url Float32Array — OPT-IN only
};
}
```
- `ControlLayer` = `{ boldness, memory, precision, presetId, offsets }` exactly as §5.4 state.
- `PipelineLayer` = the configurable params of `engine/input-pipeline.ts` and
`output-pipeline.ts` (deadzone, zoom, invert, curve, smoothing, momentum; global curve,
smoothing, slew, freeze).
- `RoutingLayer` = serialised `MFParam[]` control fields + the per-backend specs already handled
by the `OutputsBackendConfig` preset bar. The existing backend preset bar becomes a *view onto
the routing layer* — one storage model, two entry points (see §1.4).
- `weights` uses the playground's base64url `Float32Array` codec verbatim
(`session-preset.ts:6398`): standard base64 with `+/``-_`, padding stripped; byte length
rounded down to a 4-byte boundary on decode; size-mismatch on restore is silently skipped.
Restore is best-effort and layer-wise: absent layers leave current state untouched; unknown keys
inside a layer are ignored (forward compatibility).
### 1.2 Persistence
- Key: `mf-session`, via the shared `persist<T>` helper, version field `v: 1`, 200 ms debounce.
- Stored shape: `{ v: 1, presets: SessionPresetV1[], regionPins: RegionPin[] }` (pins ride in
the same document, §2).
- Weights are **excluded by default** on save; the save UI offers an explicit
"include network weights" toggle (default off). Rationale: weight blobs are ~KB-scale ×
N presets and stale weights surprise users; matches playground `withWeights=false` default.
### 1.3 URL sharing
Carry over both playground URL forms (`session-preset.ts:249283`):
- **Compact axis form** — `?boldness=0.5&memory=0.5&precision=0.3`: applies only the control
layer; each value clamped to [0,1].
- **Full form** — `?session=<base64url(JSON of SessionPresetV1.layers)>`: applied on first load,
after engine ready, before first render of the dock. Never include weights in a generated
share URL (URL length); the share button produces the full form minus `weights`.
URL params are read once at boot in `App.tsx` (same place the `?debug=1` probe is installed) and
then stripped from the address bar via `history.replaceState` so a reload doesn't re-apply over
user changes.
### 1.4 UI
- **Save/restore/rename/delete** lives in a preset bar at the top of the **route drawer's
expanded depth**, reusing the exact interaction pattern of
`dock/OutputsBackendConfig.tsx` (save-as / restore / rename / delete). The existing
backend-scoped preset bar remains; a session preset is the superset bundle.
- **Share** button beside the bar copies the URL to the clipboard and flashes confirmation.
- Restoring a preset that contains a `mode` layer while a different mode is active shows the
same confirm modal used for net-reset (input-layer reshape) — mode switches discard training
state and must not be silent.
### 1.5 Probe + acceptance
- Probe: `saveSessionPreset(name, withWeights?)`, `loadSessionPreset(id)`, `listSessionPresets()`,
`buildShareUrl()`, `applySession(json)`.
- e2e: save → mutate axes/routing → restore → assert state round-trips; share URL → fresh page
load with `?session=` → assert layers applied; weights round-trip when opted in
(`getWeights()` equality within exact bits).
---
## 2. Region & param pins
### 2.1 Region pins
**Type (unchanged from playground `session-store.ts:3444`):**
```ts
interface RegionPin {
id: string; // `pin-${epoch36}-${rand}`
x: number; y: number; // bottom-left corner, [0,1] input space
width: number; height: number; // [0,1]
colourSlot: number; // 0..4 → --pin-1..--pin-5 tokens
createdAt: number;
}
```
- **Cap: 5 pins, FIFO** (oldest evicted). Colour slot defaults to `pins.length % 5`.
- **Colour tokens** join `manifold/src/styles/tokens/colors.css` with the playground values:
fills `--pin-1..5` at 0.25 alpha (orange `255,106,0` / cyan `0,204,255` / purple
`180,100,255` / green `80,200,120` / tan `255,200,80`), borders same hues at 0.7 alpha,
2 px logical border width.
- **Creation gesture:** long-press (600 ms — the constant Manifold.tsx already uses) on the
Manifold stage pins the *currently visible region* (whole surface when unzoomed; the zoom
window once input zoom exists in manifold — until then, `{x:0,y:0,w:1,h:1}` is not useful, so
the long-press instead opens a small radial affordance: "pin region around cursor"
creating a 0.25×0.25 region centred on the press point, clamped to [0,1]²).
*Divergence from playground (which pins the zoom window) because manifold has no input zoom
yet; revisit when the inputs workstream lands zoom.*
- **Removal:** tap a pin border → small popover with colour dot + "remove"; "clear pins" action
in the settings drawer.
- **Rendering:** `Manifold.tsx` already accepts `pins: Pin[]` — extend the `Pin` type in
`console/types.ts` to the full `RegionPin` shape (it currently has only `x/y/colour`), and
render rects behind trail/markers, in front of the grid. Long-press must not fire when the
press initiates a drag > 8 px (existing drag threshold logic) or while `picking` is true
(explore-and-place anchor picking takes priority).
- **Persistence:** in the `mf-session` document (§1.2).
- **Semantics:** purely annotational in v1 (visual memory of "good areas"). They do **not**
constrain training. A future spec may bind snapshot anchors to pins.
### 2.2 Param pins = the existing tri-state (decision, not new code)
The playground's separate `ParamPin` list + `paramPinMask()` is **not ported**. Manifold already
has a strictly more expressive mechanism, per `dock-spec.md` §3.3:
- `off` — excluded from model control *and* pinned out of training;
- `fixed` — held at a static value, pinned out of training;
- `live` — model-driven, trainable.
"Pinning a param" in manifold = setting it to `fixed` (hold current value) via the existing
`OutputControlRow` segmented control or by clicking the value readout in `OutputStage`.
**Gap this spec closes:** the tri-state must actually feed the training mask. Requirement: the
arm/focus mask sent through `_nisps_ml_feedback_set_focus` (and any `moveWeights`-style
perturbation mask) is computed as `armed ∧ (state === 'live')``off`/`fixed` columns are never
perturbed and never accumulate gradient. Owner: `dock/output-state.ts` `buildArmMask()`.
Probe: `pinParam(i)` / `unpinParam(i)` map to `setParam(i, {state:'fixed'|'live'})`; the inert
playground-compat methods in `probe.ts` are rewired to these.
Acceptance: e2e asserts (a) a `fixed` output's value is bit-stable across training and jolt;
(b) region pins survive reload; (c) 6th region pin evicts the 1st.
---
## 3. Jolt
### 3.1 What it is
A held-gesture weight morph: while held, N randomly-chosen weights each glide (EMA) toward their
own random target, re-rolling the target on arrival — a continuous, directional "stirring" of
the network. On release the net freezes where it landed, and the learning rate ramps back up
from 0 so training doesn't immediately fight the new position. Ported from upstream
`InterfaceRL`; C++ implementation shipped in firmware (SLP workshop Part I).
### 3.2 Binding, not porting
Expose the existing `nisps::ml::Jolt` through the C ABI. New bindings in
`nisps/wasm/bindings.cpp` + `scripts/build-wasm.sh` EXPORTED_FUNCS:
```
_nisps_ml_jolt_press(ml) // picks indices+targets from the MLP's weight count
_nisps_ml_jolt_step(ml) // one tick: mutates weights in place, then caller reprocesses
_nisps_ml_jolt_release(ml)
_nisps_ml_jolt_active(ml) -> i32
_nisps_ml_jolt_lr_scale(ml) -> f32 // 0 while held; ramps 0→1 after release
_nisps_ml_jolt_tick_lr_ramp(ml)
_nisps_ml_jolt_seed(ml, u64)
```
The Jolt instance lives beside the MLP handle inside the WASM module (one per net), seeded from
the session seed so runs are reproducible. Parity CI: extend `tests/cpp/parity_check.cpp` with a
golden jolt sequence (seed → press → k steps → weight vector) asserted native-vs-WASM.
docs: the specs disposition pass (plan §8) Roughly 20k lines were deleted from this repo in the last week and much of the corpus still described the pre-deletion world in the present tense. Executes the §8 table: archive the retired, reclassify the executed, prune the stale. aimmersive-clone-spec -> _archive/ with a deprecated-by note feedback-modes-port-spec -> plans/, kind: plan, status: executed manifold-parity-features -> plans/, kind: plan, status: active playground-2.0-rewrite -> status: superseded engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten present-tense against the shipped engine/ MAIN.md six contradicted claims fixed; registry resynced vcv-module.md pruned to the current 8->16 contract and made the single .nisps format spec vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table inputs/backends/dock trio grounding sections marked historical, dead cites fixed Two rows of the §8 table were themselves wrong, corrected here: the deleted full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no §6.3), and codegen/README.md was already a MAP pointer with no port-solidjs trigger left to remove. Beyond the table — found by sweeping every backticked path in the changed docs against `git ls-files`, which is how these should have been caught before: manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current: SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a shot.spec.ts that does not exist. The whole stage table was keyed on a `focus` axis that no longer exists — selection is now sandwich > particles > composite. This matters more than the rest: CLAUDE.md tells every agent to read ONBOARDING.md first for Manifold work, so it was actively teaching a fiction. Rewritten against ConsoleApp.tsx. MAP.md claimed the input layer reduces axes to the engine arity with an "even/odd blend". input-layer.ts says the opposite in its own header: one dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and mean-blending was removed deliberately because it diluted every source. AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed), per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at nisps::FixedBuffer (deleted). Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/ Delta and cachedNovelty behind a live context-menu toggle that nothing reads; vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to 9000, so out of the box they do not meet. Firmware-build docs are deliberately untouched — the PlatformIO migration lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
### 3.3 Constants (defaults — from `nisps/ml/jolt.hpp` `JoltParams`, upstream-verbatim)
| Constant | Value | Meaning |
|---|---|---|
| `num_weights` | 40 | weights morphed simultaneously (hard cap 64) |
| `morph_rate` | 0.017 | EMA per tick: `w += 0.017·(target w)` |
| `target_min / max` | 1.2 / 0.9 | random target range |
| `target_epsilon` | 0.05 | re-roll target when within this distance |
| `lr_ramp_step` | 0.001 | post-release LR ramp per tick (~5 s at 200 Hz) |
| tick period | 5 ms (200 Hz) | `JOLT_TICK_MS`, matches firmware control rate |
### 3.4 Manifold integration
- **Driver:** a `setInterval(5 ms)` in `FeedbackController` while active: `jolt_step`
`engine.reprocess()` (the spine already re-ticks the last input after weight changes) → bump
version. LR suppression: training calls multiply their LR by `jolt_lr_scale()`; the ramp is
ticked from the same driver until it reaches 1.0.
- **Snapshot before press:** entering jolt pushes an auto-snapshot (tag `before jolt`) so undo
recovers the pre-jolt net — same behaviour as playground `mode-runtime.ts:551`.
- **UI:** a press-and-hold **⚡ Jolt** button in the **learn drawer** (condensed depth, beside
the VerdictCluster-adjacent controls) and keyboard hold `J`. Visual: the Manifold noise ring
pulses while active. Pointer-cancel/blur must release (never a stuck jolt).
- **Interlocks:** disabled while `exploring` (explore-and-place owns the net during a scratchpad
session); `off`/`fixed` outputs are unaffected by definition (§2.2 mask applies only to
training — jolt perturbs *weights*, so v1 explicitly documents that jolt ignores per-output
pins; a per-column jolt mask is out of scope).
### 3.5 Probe + acceptance
Probe: `joltPress()`, `joltRelease()`, `joltActive()`. e2e: press → 200 ms → assert outputs
changed and `getWeights()` differs; release → assert weights stable; undo restores pre-jolt
weights; determinism: same seed + same step count ⇒ identical weight vector twice.
---
## 4. OU-explore noise
### 4.1 What it is
OrnsteinUhlenbeck drift added to the *output vector* (not the weights): each output channel
carries a state that takes small random steps while being pulled back toward zero, producing
smooth, temporally-correlated wander around whatever the net currently outputs. Learning stays
live throughout — it is an audition aid, not a training mode. C++ `OUNoise<N>` shipped in
firmware (SLP workshop Part I).
Discrete update per channel (EulerMaruyama, μ = 0):
```
state += theta · (state) · dt + noise_scale · N(0,1)
out = clamp(out + state, 0, 1)
noise_scale = stationary_std · √(2·theta·dt), stationary_std = intensity · 0.65
```
### 4.2 Binding
Instantiate `nisps::ml::OUNoise<126>` (the WASM output width) beside the MLP handle. Bindings:
```
_nisps_ml_ou_set_intensity(ml, f32) // [0,1] → stationary_std = v·0.65; 0 disables
_nisps_ml_ou_intensity(ml) -> f32
_nisps_ml_ou_apply(ml, out_ptr, n) // advance state + add + clamp, in place
_nisps_ml_ou_reset(ml)
_nisps_ml_ou_set_theta(ml, f32) _nisps_ml_ou_set_dt(ml, f32)
_nisps_ml_ou_seed(ml, u64)
```
### 4.3 Constants (defaults — `nisps/ml/ou_noise.hpp`, upstream-verbatim)
| Constant | Value |
|---|---|
| `theta` (mean reversion) | 0.02 |
| `dt` | 0.001 |
| `kOUMaxAmplitude` (full-scale stationary std) | 0.65 |
| default intensity | 0 (**inert** — `apply()` is a no-op, preserving current behaviour/parity) |
| idle tick period | 30 ms (`EXPLORE_TICK_MS`) |
### 4.4 Manifold integration
- **Where in the chain:** in the spine, after the TS output-pipeline (curve→smoothing→slew→
freeze) and before the backend send — i.e. `ou_apply` on the routed buffer. This matches the
playground ordering (post-pipeline, pre-overrides). `fixed`/`off` outputs are re-asserted
*after* `ou_apply` so pinned values do not wander (this is the §2.2 guarantee).
- **Idle ticking:** when intensity > 0 and no input events arrive, a 30 ms interval calls
`reprocess()` so the wander is audible with a static joystick. Interval torn down and
`ou_reset()` called when intensity returns to 0.
- **UI:** an **Explore** slider (01) in the **learn drawer** condensed depth, next to the
existing Noise/Spread knobs; the Manifold noise ring radius reflects
`noiseCap + ou_intensity·0.65` so the visual language stays truthful.
- **Interaction with Jolt:** independent and composable (jolt stirs weights, OU wanders
outputs); both default inert.
### 4.5 Probe + acceptance
Probe: `setExploreIntensity(v)`, `getExploreIntensity()`. e2e: intensity 0.5 + static input ⇒
outputs vary over 500 ms with bounded step size; intensity 0 ⇒ outputs bit-stable; `fixed`
output does not wander; determinism under seeded runs.
---
## 5. Boldness / Memory / Precision control surface
### 5.1 Concept
Three compound axes, each a `[0,1]` fader that drives ~56 underlying parameters through a
piecewise-linear lookup table, with per-param trim-pot offsets on top. This is the "Feel"
surface from `playground-2.0-rewrite-plan.md` §2.3, ported with the playground's tables as the
**transparent defaults** — reproduced in full below so the mapping is reviewable here, not
buried in code.
### 5.2 Axis tables (defaults — `playground/src/stores/control-store.ts:3854`, verbatim)
Numeric params interpolate linearly between breakpoints; discrete params snap to the upper row
at t ≥ 0.75.
**Boldness** — "how hard do moves hit"
| axis | zoom | noiseCap | noiseGrowth | learningRate | weightDecay | noiseDistribution |
|---|---|---|---|---|---|---|
| 0.0 | 0.1 | 0.02 | 1.1 | 0.1 | 0.15 | gaussian |
| 0.5 | 0.5 | 0.12 | 1.5 | 1.0 | 0.06 | gaussian |
| 1.0 | 1.0 | 0.30 | 2.5 | 3.0 | 0.00 | cauchy |
**Memory** — "how much does the net remember"
| axis | maxExamples | exampleDecay | memoryWeightDecay | noiseDecay | convergenceThreshold |
|---|---|---|---|---|---|
| 0.0 | 5 | 0.3 | 0.20 | 0.85 | 1e-3 |
| 0.5 | 50 | 0.7 | 0.06 | 0.97 | 1e-5 |
| 1.0 | 500 | 1.0 | 0.00 | 0.995 | 1e-8 |
**Precision** — "how surgical is the gesture"
| axis | inputCurve | deadzone | smoothing | slewRate | momentumZoom |
|---|---|---|---|---|---|
| 0.0 | 1.0 | 0.00 | 0.00 | 1.0 | off |
| 0.5 | 1.5 | 0.05 | 0.15 | 0.3 | off |
| 1.0 | 3.0 | 0.15 | 0.40 | 0.1 | off |
Routing targets in manifold: `zoom/deadzone/inputCurve/smoothing/momentumZoom`
`engine/input-pipeline` config; `slewRate``engine/output-pipeline`; `noiseCap/noiseGrowth/
noiseDecay/learningRate/weightDecay` → `FeedbackController` + training params;
`maxExamples/exampleDecay/convergenceThreshold``engine/dataset.ts` + training loop.
`noiseDistribution` and `momentumZoom` are carried in the schema but may no-op until their
consumers exist in manifold (each no-op must be logged once at boot — no silent dead params).
`spread` is deliberately **not** axis-driven (stays an independent lab knob, as in playground).
### 5.3 Presets (defaults — verbatim)
| id | label | boldness | memory | precision |
|---|---|---|---|---|
| `default` | Default | 0.5 | 0.5 | 0.3 |
| `first-touch` | First Touch | 0.2 | 0.7 | 0.6 |
| `jazz-hands` | Jazz Hands | 0.8 | 0.2 | 0.0 |
| `sculptor` | Sculptor | 0.3 | 0.9 | 0.8 |
| `improviser` | Improviser | 0.6 | 0.3 | 0.2 |
| `microscope` | Microscope | 0.1 | 1.0 | 1.0 |
Boot default = the `default` preset (0.5 / 0.5 / 0.3). **This replaces ConsoleApp's current
ad-hoc `{0.55, 0.4, 0.5}` initial state.** Moving any axis manually clears `presetId` (chips
show "custom").
### 5.4 State + trim-pot offsets
```ts
interface ControlState {
boldness: number; memory: number; precision: number; // [0,1]
offsets: Record<'boldness'|'memory'|'precision', Record<string, number>>;
presetId: string | null;
}
```
Resolution: interpolate all three tables, merge, then add each axis's offsets to numeric params
(`resolved[k] = table[k] + offset[k]`), clamp to each param's legal range. Offsets are created
when the user adjusts a *derived* param directly (§5.6) — the delta from the current table value
becomes the offset ("trim-pot"). Double-tap on a derived param's readout clears its offset
("re-link"); a per-axis "re-link all" clears the axis's offset map. A dot marks any param with a
non-zero offset.
### 5.5 Fanout (architecture requirement)
docs: the specs disposition pass (plan §8) Roughly 20k lines were deleted from this repo in the last week and much of the corpus still described the pre-deletion world in the present tense. Executes the §8 table: archive the retired, reclassify the executed, prune the stale. aimmersive-clone-spec -> _archive/ with a deprecated-by note feedback-modes-port-spec -> plans/, kind: plan, status: executed manifold-parity-features -> plans/, kind: plan, status: active playground-2.0-rewrite -> status: superseded engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten present-tense against the shipped engine/ MAIN.md six contradicted claims fixed; registry resynced vcv-module.md pruned to the current 8->16 contract and made the single .nisps format spec vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table inputs/backends/dock trio grounding sections marked historical, dead cites fixed Two rows of the §8 table were themselves wrong, corrected here: the deleted full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no §6.3), and codegen/README.md was already a MAP pointer with no port-solidjs trigger left to remove. Beyond the table — found by sweeping every backticked path in the changed docs against `git ls-files`, which is how these should have been caught before: manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current: SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a shot.spec.ts that does not exist. The whole stage table was keyed on a `focus` axis that no longer exists — selection is now sandwich > particles > composite. This matters more than the rest: CLAUDE.md tells every agent to read ONBOARDING.md first for Manifold work, so it was actively teaching a fiction. Rewritten against ConsoleApp.tsx. MAP.md claimed the input layer reduces axes to the engine arity with an "even/odd blend". input-layer.ts says the opposite in its own header: one dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and mean-blending was removed deliberately because it diluted every source. AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed), per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at nisps::FixedBuffer (deleted). Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/ Delta and cachedNovelty behind a live context-menu toggle that nothing reads; vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to 9000, so out of the box they do not meet. Firmware-build docs are deliberately untouched — the PlatformIO migration lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
Per `playground-2.0-rewrite-plan.md` §3.2: resolution is a memoised derivation per target param — in
React terms one `useMemo` producing the resolved record + an effect per target store that
writes only on change. The playground's `JSON.stringify`-signature-inside-effect pattern
(`control-routing.ts:34`) is explicitly **not** ported. Resolution runs off the render cycle for
engine targets (direct setter calls on the engine config objects), with React state only for
what the UI displays.
### 5.6 UI — transparency is the feature
- **Placement:** the **learn drawer** gains a top "Feel" section (this keeps manifold at five
drawers; if the section crowds the condensed depth, promoting Feel to its own drawer is a
pre-approved fallback — note it in the implementation PR).
- *Condensed:* three vertical `ControlAxis` faders (the existing primitive, finally wired) +
six preset chips.
- *Expanded:* under each fader, the live list of its derived params — name, resolved value
(table + offset), and a mini trim-pot. This satisfies the "reasonable and transparent
defaults" requirement: the mapping is always visible in the UI, not just in this document.
- Axis moves are continuous (no apply button); param writes are rate-limited to one per frame.
- The existing `preset = 'Sculpt'` placeholder state in ConsoleApp is removed in favour of
`presetId`.
### 5.7 Persistence, probe, acceptance
- Persisted as the `control` slice of `mf-session` (also the `control` layer of session
presets, §1 — one shape, two containers).
- Probe: `setAxis(name, v)`, `getAxes()`, `resolveControlParams()`, `applyControlPreset(id)`,
`setAxisOffset(axis, param, v)`.
- e2e: boldness 0→1 sweeps noiseCap 0.02→0.30 and zoom 0.1→1.0 (assert via
`resolveControlParams()` + engine config readback); preset chip sets all three axes; manual
axis move clears presetId; offset survives axis movement (offset is additive, not absolute);
double-tap re-links; state round-trips through reload and through a session preset.
---
## 6. Cross-cutting
### 6.1 Build & sequencing (proposed ergo breakdown, post-sign-off)
1. **W1 — WASM bindings**: jolt + OU C ABI, EXPORTED_FUNCS, parity golden tests. (Blocks W4, W5.)
2. **W2 — `persist<T>` helper + `mf-session` document** (control slice, pins, presets scaffold).
3. **W3 — Control surface**: state, tables, fanout, Feel section, presets, offsets. (Needs W2.)
4. **W4 — Jolt + OU in FeedbackController + learn drawer UI.** (Needs W1.)
5. **W5 — Region pins** (tokens, gesture, rendering, popover) + tri-state training-mask
guarantee (§2.2). (Needs W2.)
6. **W6 — Session presets**: layers, preset bar, URL sharing, probe. (Needs W2, W3.)
Each lands with its e2e spec; smoke suite additions run in the same CI gate as the existing
`tests/e2e/smoke.spec.ts`.
### 6.2 Out of scope (explicit)
- Snapshot-with-weights / snapshot DAG, A/B compare (separate spec — the biggest remaining
parity gap, but architecturally entangled with the C++ feedback snapshot lifecycle).
- 2D heatmap sampler, weight health, gradient flow (blocked on loss-history C API plumbing).
- Input zoom (inputs workstream) — noted where it interacts with region pins (§2.1).
- Per-column jolt masking (§3.4).
### 6.3 Open questions for review
1. §2.1 creation gesture: is the 0.25×0.25 press-centred region an acceptable stand-in until
input zoom lands, or should region pins wait for the zoom feature?
2. §5.6: Feel section inside the learn drawer vs a sixth drawer — reviewer's call if condensed
depth gets crowded.
3. §1.1 RoutingLayer: should backend connection details (OSC URL, MIDI port id) be included in
session presets, or excluded as machine-specific? Proposal: exclude port/URL, include
per-output mappings.
4. Memory-axis params (`maxExamples`, `exampleDecay`, `convergenceThreshold`): confirm the
manifold training loop actually consumes these; if any are dead in the current engine, they
land as logged no-ops (§5.2) with an ergo follow-up.
---
*Doc-sync rule applies: when implementation diverges from this spec, update the spec in the same
commit. When a workstream ships and is verified, migrate its section to ONBOARDING.md ("what
is") and prune it here.*