feat(manifold): edit I/O as identity-aware cards

This commit is contained in:
monkey-w1n5t0n 2026-07-25 15:16:37 +02:00
parent 3879d57b4c
commit 0010097d01
30 changed files with 848 additions and 284 deletions

13
MAP.md
View file

@ -58,7 +58,7 @@ anchor + locked decisions) and the `docs/specs/*-spec.md` set.
consts (mode_id, engine_id, ml dims, params, voice_spaces, ui) — the SOURCE OF TRUTH for `MF_MODES`.
Switching mode reshapes the WASM net to the mode's `ml` dims (ConsoleApp P5.3; boot mode paf_synth →
4→[10,10,14]→33).
- `manifold/src/dock/``OutputControlRow` (off/fixed/live + mute + solo/arm + min/max/curve), `output-state.ts`,
- `manifold/src/dock/``OutputControlRow` (add/delete card identity + off/fixed/live + mute + solo/arm + min/max/curve), `output-state.ts`,
`OutputsBackendConfig.tsx` (per-backend specialised Outputs panel — the sole per-backend editor).
- `manifold/src/backends/``OutputBackend` adapter + `BackendManager` (spine consumer); `midi-backend.ts`
(WebMIDI), `osc-backend.ts`+`osc-client.ts` (OSC-over-WS), `vcv-backend.ts` (VCV-over-WS), `cv-backend.ts`
@ -74,9 +74,10 @@ anchor + locked decisions) and the `docs/specs/*-spec.md` set.
`setInputs`, plus an `onReducedInput` callback the manifold tracks. The WASM net is over-provisioned to a
32-input head (`MAX_AXES`, `nisps/wasm/bindings.cpp`); unused slots are zero-padded and a zero input is inert,
so idle sources cannot perturb the net. Mean-blending was removed deliberately — it diluted every source and
biased the net toward idle sources' resting values. Changing the ACTIVE axis count offers a reshape
(`ConsoleApp` → `ReshapeModal`): new net at the new arity, warm-started from overlapping weights, examples and
feedback state reset; declining keeps the over-provisioned head. Sources: `xy-pad-source` (push-driven),
biased the net toward idle sources' resting values. Active-axis edits follow the persistent I/O policy:
keep-capacity permutes stable identities in place until more slots are required; exact-I/O reconstructs
to the active count. Surviving weights and (under adapt policy) examples are identity-remapped; feedback
scratch state resets. Sources: `xy-pad-source` (push-driven),
`gamepad-source` (sticks→axes single/double; buttons emit press+release actions, bound in `ConsoleApp` to
verdicts — LB/RB=down/up, X/Y/B=randomise/nudge/undo, A-hold=reposition), `midi-input-source` (device picker +
BATCH "MIDI Learn": every CC swept while armed becomes an axis, shown as read-only meters). `useInputLayer.ts`
@ -84,11 +85,13 @@ anchor + locked decisions) and the `docs/specs/*-spec.md` set.
`backends/base-backend.ts` is its output-side counterpart (status + throttle + lastSent) used by the midi/osc/vcv transports.
- `manifold/src/feedback/``controller.ts` (Explore-and-place scratchpad + geometric-dislike + solo; a thin
driver over the shared C++ core).
- `manifold/src/settings/``settings-store.ts` (monochrome icons, input-map shape, corner radius, and the opt-in legacy Xavier/spread feature flag; Manifold randomisation is full-range uniform by default).
- `manifold/src/settings/``settings-store.ts` (monochrome icons, input-map shape, I/O resize policy, corner radius, and the opt-in legacy Xavier/spread feature flag; Manifold randomisation is full-range uniform by default).
- `manifold/src/serial/``memlnaut-serial.ts` Web Serial scaffold + `EditorPanel.tsx` (MEMLNaut Editor mode).
- `manifold/src/engine/exploration.ts` — Jolt press + OU explore gestures (Learning drawer): a thin
timer-driver over the shared C++ core via the `nisps_ml_jolt_*`/`nisps_ml_ou_*` bindings (the interim
TS math and `jolt.ts`/`ou-explore.ts` were deleted when P3 landed).
- `manifold/src/engine/io-reshape.ts` — the deep identity-migration module for I/O card edits:
exact-vs-capacity reconstruction decisions, flat weight remapping, and example vector adaptation.
- `manifold/src/debug/probe.ts``window.__nisps` (`?debug=1`). `manifold/tests/e2e/``smoke`,
`probe-api` (engine-contract port), `spine` (spine invariant + probe-survives-mode-switch),
`geo-dislike`, `reshape`, `schema-modes`, `training-health` (the loss/layer-stats panel + its

View file

@ -115,7 +115,7 @@ Two WASM instances at runtime:
C API is in `nisps/wasm/bindings.cpp`. Build: `bash scripts/build-wasm.sh` (~94KB output to `manifold/public/`).
The browser MLP is runtime-shaped since P2 (`MLPCore<DynamicStorage>`): `nisps_ml_create` honours `(input, output, hidden[3])`; non-positive/null args default to `32→[10,14,18]→126`. `nisps_ml_reshape` swaps in a new shape warm-started from the overlapping weights (examples + feedback state reset). Per-mode dims have been schema-real since P5.3 on both targets — modes no longer slice a shared 126-wide default.
The browser MLP is runtime-shaped since P2 (`MLPCore<DynamicStorage>`): `nisps_ml_create` honours `(input, output, hidden[3])`; non-positive/null args default to `32→[10,14,18]→126`. Raw `nisps_ml_reshape` reconstructs and prefix-warm-starts a new shape. Manifold's higher-level `engine/io-reshape.ts` seam adds stable input/output identity: it can permute weights and examples without reconstruction while capacity suffices, or reconstruct with arbitrary surviving-dimension remaps. Persistent settings select capacity-vs-exact arity and adapt-vs-clear examples (neutral new-input/output defaults 0/0.5); feedback/exploration scratch state resets on either identity edit. Per-mode dims have been schema-real since P5.3 on both targets — modes no longer slice a shared 126-wide default.
### Known limitations

View file

@ -121,10 +121,11 @@ for the narrow pane.
- **Output modes** (the TOP dock selector, NOT the same axis as `focus`): `src/console/output-mode.ts`
defines `OUTPUT_MODES` = **particles** (default) / midi / osc / cv / synth / editor, each mapping to a
`BackendId`. `DEFAULT_OUTPUT_MODE='particles'`. `outputDisplayCount()` is the shared presentation
boundary for the stage and routing rows: MIDI uses its configured CC count, while backends without
a separate count present the full mode parameter set. The condensed Outputs panel reports this as an
`N outputs` chip. This does not reshape the MLP or clear examples.
`BackendId`. `DEFAULT_OUTPUT_MODE='particles'`. `outputDisplayCount()` is the shared active-card
boundary for the stage, backend context, and routing rows. MIDI starts with eight cards; every
backend can add a card or delete any individual card in condensed and expanded Outputs drawers.
The `N outputs` chip always reports that same set. `MFParam.id` is semantic identity; array position
is not. Settings decides whether edits retain spare network capacity or keep exact arity.
- `src/console/output-mode.ts`, `types.ts`, `model.ts` are the shared vocabulary — read these first
when touching anything cross-cutting:
- `types.ts`: `Focus`, `OutputMode`, `DrawerKey`, `DrawerDepth`, `FeedbackModeUI`, `SoloMode`,
@ -236,25 +237,25 @@ a setting → `--r-*` tokens.
- `input-layer.ts` — composition hub. One rAF loop polls sources, pulls all axes into a vector,
forwards N→engine. **`MAX_AXES = 32`** (WASM net over-provisioned to 32 inputs). **Dedicated
dimensions, NO mean-blending** — each active axis drives its own engine slot 1:1; unused slots
zero-padded (inert). Changing axis count requires a **net reset** (UI confirm modal).
zero-padded (inert). Changing the layout uses the same persisted identity-aware I/O policy as output
cards.
- `base-source.ts` + sources: `xy-pad-source.ts` (push, 2 axes), `gamepad-source.ts` (single=2 /
double=4 axes, deadzone 0.08), `midi-input-source.ts` (Web MIDI, batch CC-learn, multi-port).
- `useInputLayer.ts` — React binding; manages exclusive input mode + gamepad stick mode + MIDI
device/learn map; exposes `pushPad`, `sources`, `channelLayout`, etc.
- **Reshape (P2.3, live):** the net is now **runtime-shaped**. It boots at the default
- **I/O migration (P2.3, live):** the net is **runtime-shaped**. It boots at the default
over-provisioned 32-input head (zero-padding preserved), and `EngineApi.reshape({ inputSize, … })`
`WasmIML.reshape` swaps in a new net at the requested arity, **warm-started** from the overlapping
weights (`nisps_ml_reshape`; C-side dataset + feedback state RESET). When the active axis layout
CHANGES to a count ≠ the net's arity, `ConsoleApp` offers the swap behind `ReshapeModal.tsx`
(reset-on-reshape confirm; declining keeps the zero-padded head). Never offered on load. The
spine tolerates the arity change (buffers resize, version bumps); the training worker
`WasmIML.reshape`. `engine/io-reshape.ts` owns the identity map: **Keep capacity** (default)
permutes weights/examples in place and reconstructs only when active I/O outgrows the net;
**Exact I/O** reconstructs whenever active arity changes. Existing examples either adapt by
deleting removed dimensions and inserting the saved neutral placeholders (input 0, output 0.5 by
default), or clear, according to the persistent Settings choice. Feedback replay/exploration state
resets because it has no stable-ID contract. The spine tolerates arity changes; the training worker
(`wasm-worker.ts`) carries the current dims in its train message and re-creates its mirror net to
match. Debug: `window.__nisps.reshape(nIn)` / `.describe()`. See the `manifold-mixed-inputs` memory
for the locked design (adaptive slider viz when >2 dims is still pending).
match. The raw debug `window.__nisps.reshape(nIn)` remains a low-level reconstruct-and-clear call.
- **Per-mode net dims (P5.3):** switching INSTRUMENT mode reshapes the net to that mode's schema
`ml` config (`MFMode.ml` — input/hidden/output + legacy spread) via a `ConsoleApp` effect keyed on
`[engine, modeId]`. No confirm modal (switching instrument is deliberate); the axis-count
`ReshapeModal` above is for input-LAYOUT changes only. The effect depends on `engine`, so on boot
`[engine, modeId]`. No confirm modal (switching instrument is deliberate). The effect depends on `engine`, so on boot
it fires once WASM is ready and lands the boot mode's dims (**paf_synth → 4→[10,10,14]→33**, weights
809 — NOT the 32→126 default). The reshape-offer effect reads the engine's CURRENT `inputSize`
live, so a mode switch that changes arity doesn't spuriously prompt (its baseline tracks axis
@ -297,7 +298,8 @@ a setting → `--r-*` tokens.
### Misc
- `src/serial/memlnaut-serial.ts`**STUB** Web Serial scaffold for the MEMLNaut Editor mode (protocol TODO). `EditorPanel.tsx` is its UI.
- `src/settings/settings-store.ts` — localStorage settings (`mf-settings`): icon style, input-map
shape, corner radius, and the opt-in legacy Xavier/spread feature flag.
shape, exact-vs-capacity network resizing, adapt-vs-clear examples + neutral new-dimension values,
corner radius, and the opt-in legacy Xavier/spread feature flag.
- `src/midi-devices/` — codegen'd external-synth device templates.
- `src/debug/probe.ts``window.__nisps` synchronous probe (engine/audio/bus). Some playground
feature-store methods are present-but-inert (not ported yet) to keep the surface stable.

File diff suppressed because one or more lines are too long

Binary file not shown.

View file

@ -33,7 +33,7 @@ export interface OutputMapping {
/** What a backend needs to know about the active mode/output set. */
export interface BackendContext {
modeId: string;
/** Model output dims in use (≤ 126). */
/** Active output cards routed to this backend (may be < model capacity). */
outputCount: number;
/** Per-output baseline mapping, length === outputCount. */
mappings: OutputMapping[];

View file

@ -53,6 +53,8 @@ export class BackendManager {
* click no longer silently drops the second request. Cleared before the
* re-run so it can only ever chain one hop at a time (no unbounded loop). */
private pendingId: BackendId | null = null;
/** Reused active-prefix buffer when model capacity exceeds active cards. */
private routedScratch = new Float32Array(0);
private statusListeners = new Set<(id: BackendId, s: BackendStatus) => void>();
private offBackendStatus: (() => void) | null = null;
@ -77,7 +79,17 @@ export class BackendManager {
this.active.setInputVector(this.engine.inputVector());
}
const routed = this.engine.routedOutput();
if (routed) this.active.send(routed);
if (!routed) return;
const activeCount = this.ctx?.outputCount ?? routed.length;
if (activeCount >= routed.length) {
this.active.send(routed);
} else {
if (this.routedScratch.length !== activeCount) {
this.routedScratch = new Float32Array(activeCount);
}
this.routedScratch.set(routed.subarray(0, activeCount));
this.active.send(this.routedScratch);
}
});
}
@ -127,6 +139,9 @@ export class BackendManager {
/** Provide / refresh the BackendContext (mappings + names) for the active set. */
setContext(ctx: BackendContext): void {
this.ctx = ctx;
if (this.routedScratch.length !== ctx.outputCount) {
this.routedScratch = new Float32Array(ctx.outputCount);
}
this.active?.setContext?.(ctx);
}

View file

@ -14,6 +14,7 @@ import type { BackendId } from '../dock/output-state';
/** The per-output config a preset captures (a slice of MFParam). */
export interface OutputPresetRow {
id?: string;
name: string;
status: MFParam['status'];
muted?: boolean;
@ -71,6 +72,7 @@ export function listPresets(backend: BackendId): OutputPreset[] {
/** Project the live params into preset rows. */
export function rowsFromParams(params: MFParam[]): OutputPresetRow[] {
return params.map((p) => ({
id: p.id,
name: p.name,
status: p.status,
muted: p.muted,
@ -140,7 +142,7 @@ export function renamePreset(backend: BackendId, from: string, to: string): bool
*/
export function applyPreset(params: MFParam[], preset: OutputPreset): MFParam[] {
return params.map((p, i) => {
const r = preset.rows[i];
const r = preset.rows.find((row) => row.id === p.id) ?? preset.rows[i];
if (!r) return p;
return {
...p,

View file

@ -30,7 +30,7 @@
import { useEffect, useMemo, useRef, useState } from 'react';
import type { CSSProperties } from 'react';
import { useEngine, useEngineVersion, ExplorationController } from '../engine';
import { MF_MODES, modeEngineId, shapeValues } from './model';
import { MF_MODES, createOutputParam, modeEngineId, shapeValues } from './model';
import type { MFParam } from './model';
import { CompositeStage } from './CompositeStage';
import { ParticleStage } from './ParticleStage';
@ -39,7 +39,6 @@ import { OutputStage } from './OutputStage';
import { Manifold } from './Manifold';
import { VerdictCluster } from './VerdictCluster';
import { Dock } from './Dock';
import { ReshapeModal } from './ReshapeModal';
import type {
ConsoleCtx,
DrawerDepth,
@ -57,6 +56,11 @@ import { DEFAULT_OUTPUT_MODE, outputDisplayCount, outputModeDescriptor } from '.
import { useSettings, resolveInputMap } from '../settings/settings-store';
import { useBackendManager } from '../backends';
import { useInputLayer } from '../inputs';
import {
completeDimensionMap,
resizeTarget,
type DimensionMap,
} from '../engine/io-reshape';
/**
* Console debug seam, installed on `window.__mf` under `?debug=1` (see the
@ -102,6 +106,7 @@ export function ConsoleApp() {
const [modeId, setModeId] = useState('paf_synth');
const mode = MF_MODES.find((m) => m.id === modeId) ?? MF_MODES[0];
const [params, setParams] = useState<MFParam[]>(() => mode.params.map((p) => ({ ...p })));
const [paramsModeId, setParamsModeId] = useState(mode.id);
const [pos, setPos] = useState<[number, number]>([0.5, 0.5]);
const [noiseCap, setNoiseCap] = useState(0.12);
@ -146,10 +151,13 @@ export function ConsoleApp() {
// Per-backend transport settings (backends-spec §2.3/§2.4). Persisted via the
// named-preset system; these are the live working values.
const [midiOutputId, setMidiOutputId] = useState<string | null>(null);
const [midiCcCount, setMidiCcCount] = useState(8);
const displayOutputCount = outputDisplayCount(outputMode, params.length, {
midi: midiCcCount,
const [outputCounts, setOutputCounts] = useState<Partial<Record<OutputMode, number>>>({
midi: 8,
});
const midiCcCount = outputDisplayCount('midi', params.length, outputCounts);
const setMidiCcCount = (n: number) =>
setOutputCounts((counts) => ({ ...counts, midi: Math.max(1, Math.floor(n)) }));
const displayOutputCount = outputDisplayCount(outputMode, params.length, outputCounts);
const [oscUrl, setOscUrl] = useState('ws://localhost:8765');
const [oscSendRaw, setOscSendRaw] = useState(false);
// VCV bridge: WS URL of the Deno bridge that relays to the VCV module over UDP
@ -249,9 +257,9 @@ export function ConsoleApp() {
// level — the true gradient column-freeze (`train_masked`) is the C++ step.
useEffect(() => {
controllerRef.current?.setSoloMode(soloMode);
controllerRef.current?.setArmMask(buildArmMask(params));
controllerRef.current?.setArmMask(buildArmMask(params.slice(0, displayOutputCount)));
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [engine, params, soloMode]);
}, [engine, params, displayOutputCount, soloMode]);
// Keep the audio backend pointed at the current mode (audio itself is gated
// behind a user gesture — see startAudio below).
@ -276,6 +284,7 @@ export function ConsoleApp() {
// reset transient state on mode switch
useEffect(() => {
setParams(mode.params.map((p) => ({ ...p })));
setParamsModeId(modeId);
setPos([0.5, 0.5]);
setExamples(0);
setFollow(false);
@ -301,43 +310,55 @@ export function ConsoleApp() {
// `pushPad`; MIDI + gamepad are pulled by the layer's own rAF loop.
const inputs = useInputLayer(engine);
// ---- Reshape offer (runtime-shaped net, P2) --------------------------------
// When the ACTIVE input layout CHANGES (source added/removed, MIDI-learn axes
// change, gamepad stick mode) to an axis count that differs from the net's
// current input arity, offer a warm-started reshape behind a confirm modal
// (locked decision: "reshapeable N-D net, reset-on-reshape modal"). We only
// offer on a genuine CHANGE — never on first load — so the default 32-input
// over-provisioned head is preserved untouched, and declining keeps the
// zero-padding path working. Once per layout change (debounced by the ref).
const [reshapeTarget, setReshapeTarget] = useState<number | null>(null);
const prevAxisCountRef = useRef<number | null>(null);
// ---- Identity-aware input-card migration -----------------------------------
// Input sources already expose add/remove cards (MIDI learns, gamepad axes).
// Preserve their semantic coordinates by source+label when those rows change.
const inputSlotsRef = useRef<Map<string, number> | null>(null);
const inputLayoutKey = inputs.channelLayout
.map((channel) => `${channel.source}\u0000${channel.label}`)
.join('\u0001');
useEffect(() => {
if (!engine) return;
const n = inputs.axisCount;
// Ignore the boot transient (sources attach a frame after mount, so the
// count ramps 0 → 2) and any "no active axes" lull — neither is a layout the
// user chose, and treating 0 as a baseline would make the first real layout
// look like a change and prompt on load.
if (n < 1) return;
const keys = inputLayoutKey ? inputLayoutKey.split('\u0001') : [];
if (keys.length < 1) return;
const inSize = inputs.engineInputSize;
// First established layout is the baseline (default load) — record, never
// prompt. This is the default over-provisioned case that must stay untouched.
if (prevAxisCountRef.current === null) {
prevAxisCountRef.current = n;
const previous = inputSlotsRef.current;
if (previous === null) {
inputSlotsRef.current = new Map(keys.map((key, i) => [key, i]));
return;
}
if (n === prevAxisCountRef.current) return; // arity unchanged → no offer
prevAxisCountRef.current = n;
// Offer iff the new active layout no longer matches the net's arity.
if (n !== inSize) setReshapeTarget(n);
else setReshapeTarget(null);
}, [engine, inputs.axisCount, inputs.engineInputSize]);
const confirmReshape = () => {
const n = reshapeTarget;
setReshapeTarget(null);
if (engine && n != null) engine.reshape({ inputSize: n });
};
const activeMap: DimensionMap = keys.map((key) => previous.get(key) ?? null);
const target =
resizeTarget(keys.length, inSize, settings.networkResizePolicy) ?? inSize;
const inputMap = completeDimensionMap(activeMap, target, inSize);
const needsMigration =
target !== inSize || inputMap.some((oldIndex, newIndex) => oldIndex !== newIndex);
if (needsMigration) {
engine.reshape(
{ inputSize: target },
randomisationSpread,
{
inputMap,
examples: settings.exampleResizePolicy,
addedInputValue: settings.addedInputExampleValue,
addedOutputValue: settings.addedOutputExampleValue,
},
);
controllerRef.current?.resetAfterIoChange();
syncController();
setExamples(engine.getState().exampleCount);
}
inputSlotsRef.current = new Map(keys.map((key, i) => [key, i]));
}, [
engine,
inputLayoutKey,
inputs.engineInputSize,
randomisationSpread,
settings.networkResizePolicy,
settings.exampleResizePolicy,
settings.addedInputExampleValue,
settings.addedOutputExampleValue,
]);
// ---- Console debug seam (`?debug=1`) ----------------------------------------
// There is no instrument-mode picker in the UI yet (ctx.modes/setModeId are
@ -389,6 +410,10 @@ export function ConsoleApp() {
// spine and forwards routed outputs to the active backend; switching Mode
// tears down the old backend, starts the new one, and gates synth audio
// (mute on non-synth modes). MIDI/OSC config + names ride the shared params.
const backendParams = useMemo(
() => params.slice(0, displayOutputCount),
[params, displayOutputCount],
);
const {
manager: backendManager,
status: backendStatus,
@ -401,7 +426,7 @@ export function ConsoleApp() {
engine,
outputBackend,
modeId,
params,
backendParams,
{ outputId: midiOutputId, ccCount: midiCcCount },
{ url: oscUrl, sendRaw: oscSendRaw },
{ url: vcvUrl, sendRaw: vcvSendRaw },
@ -636,6 +661,104 @@ export function ConsoleApp() {
});
}, [version]);
/**
* The single output-card mutation seam. Callers provide the desired active
* cards; this module decides whether to keep capacity, permute in place, or
* reconstruct, then makes card order and MLP coordinates agree again.
*/
const applyOutputCards = (activeCards: MFParam[], spareCards: MFParam[]) => {
if (activeCards.length < 1) return;
const oldCapacity = engine?.architecture.outputSize ?? mode.ml.outputSize;
const target =
resizeTarget(activeCards.length, oldCapacity, settings.networkResizePolicy) ??
oldCapacity;
const activeMap: DimensionMap = activeCards.map((param) => {
const index = param.engineIndex;
return index !== undefined && index >= 0 && index < oldCapacity ? index : null;
});
const outputMap = completeDimensionMap(activeMap, target, oldCapacity);
const oldByIndex = new Map<number, MFParam>();
for (const param of params) {
if (param.engineIndex !== undefined) oldByIndex.set(param.engineIndex, param);
}
const placed = new Set(activeCards.map((param) => param.id));
const ordered: MFParam[] = [...activeCards];
for (const oldIndex of outputMap.slice(activeCards.length)) {
if (oldIndex === null) continue;
const param = oldByIndex.get(oldIndex);
if (param && !placed.has(param.id)) {
ordered.push(param);
placed.add(param.id);
}
}
for (const param of [...spareCards, ...params]) {
if (!placed.has(param.id)) {
ordered.push(param);
placed.add(param.id);
}
}
const needsMigration =
target !== oldCapacity ||
outputMap.some((oldIndex, newIndex) => oldIndex !== newIndex);
if (engine && needsMigration) {
engine.reshape(
{ outputSize: target },
randomisationSpread,
{
outputMap,
examples: settings.exampleResizePolicy,
addedInputValue: settings.addedInputExampleValue,
addedOutputValue: settings.addedOutputExampleValue,
},
);
controllerRef.current?.resetAfterIoChange();
syncController();
setExamples(engine.getState().exampleCount);
}
setParams(
ordered.map((param, index) => ({
...param,
engineIndex: index < target ? index : undefined,
})),
);
setOutputCounts((counts) => ({ ...counts, [outputMode]: activeCards.length }));
};
const addOutput = () => {
const active = params.slice(0, displayOutputCount);
const next = params[displayOutputCount] ?? createOutputParam(params.length);
const spares = params
.slice(displayOutputCount + (params[displayOutputCount] ? 1 : 0))
.filter((param) => param.id !== next.id);
applyOutputCards([...active, next], spares);
};
const deleteOutput = (index: number) => {
if (displayOutputCount <= 1 || index < 0 || index >= displayOutputCount) return;
const active = params.slice(0, displayOutputCount);
const [removed] = active.splice(index, 1);
applyOutputCards(active, [removed, ...params.slice(displayOutputCount)]);
};
// Switching output targets (or changing the saved resize policy) reconciles
// the network to that target's persisted active-card count.
useEffect(() => {
if (paramsModeId !== modeId) return;
const count = outputDisplayCount(outputMode, params.length, outputCounts);
applyOutputCards(params.slice(0, count), params.slice(count));
// `params` is deliberately read as the current card catalogue but omitted:
// applyOutputCards itself replaces it, so including it would self-trigger.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [
engine,
modeId,
paramsModeId,
outputMode,
outputCounts[outputMode],
settings.networkResizePolicy,
]);
// Ref-mirror of everything the two global-listener effects below close over
// that is NOT already React-stable (verdict/navigation handlers are plain
// consts re-created every render; `pos`/`inputs.inputMode` are per-render
@ -804,6 +927,8 @@ export function ConsoleApp() {
},
params,
setParam,
addOutput,
deleteOutput,
displayOutputCount,
outputMode,
setOutputMode,
@ -840,8 +965,11 @@ export function ConsoleApp() {
setSoloMode,
exploring,
learningPaused,
armedCount: params.filter((p) => p.armed).length,
clearArmed: () => setParams((ps) => ps.map((p) => (p.armed ? { ...p, armed: false } : p))),
armedCount: params.slice(0, displayOutputCount).filter((p) => p.armed).length,
clearArmed: () =>
setParams((ps) =>
ps.map((p, i) => (i < displayOutputCount && p.armed ? { ...p, armed: false } : p)),
),
// exploration gestures (Jolt + OU explore)
joltActive,
onJoltPress,
@ -1123,14 +1251,6 @@ export function ConsoleApp() {
}}
/>
{reshapeTarget !== null && (
<ReshapeModal
target={reshapeTarget}
current={inputs.engineInputSize}
onConfirm={confirmReshape}
onCancel={() => setReshapeTarget(null)}
/>
)}
</div>
);
}

View file

@ -30,6 +30,7 @@ import { shapeValues } from './model';
import { outputModeDescriptor } from './output-mode';
import { useSettings, unfocusedIconCss } from '../settings/settings-store';
import type { UnfocusedIconColour, InputMapMode } from '../settings/settings-store';
import type { ExampleResizePolicy, NetworkResizePolicy } from '../engine/io-reshape';
import { EditorPanel } from '../serial/EditorPanel';
import { TrainingHealth } from './TrainingHealth';
import {
@ -632,6 +633,9 @@ function RoutingDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
<Chip>off {counts.off || 0}</Chip>
<Chip tone="var(--danger)">muted {mutedN}</Chip>
<BackendStatusChip ctx={ctx} />
<Button size="sm" variant="ghost" onClick={ctx.addOutput}>
+ output
</Button>
</div>
{ModeConfig(ctx, depth)}
{/* Specialised per-backend config + named-preset bar (MIDI/OSC); hidden when condensed. */}
@ -651,10 +655,11 @@ function RoutingDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
const labelled = { ...p, name: nameFor(i, p.name) };
return (
<OutputControlRow
key={i}
key={p.id}
param={labelled}
value={values[i] ?? 0}
onChange={(patch) => ctx.setParam(i, patch)}
onDelete={activeParams.length > 1 ? () => ctx.deleteOutput(i) : undefined}
showCurve={expanded}
/>
);
@ -681,6 +686,14 @@ const INPUT_MAP_OPTS: { value: InputMapMode; label: string }[] = [
{ value: 'rectangular', label: 'Rectangular' },
{ value: 'circular', label: 'Circular' },
];
const NETWORK_RESIZE_OPTS: { value: NetworkResizePolicy; label: string }[] = [
{ value: 'capacity', label: 'Keep capacity' },
{ value: 'exact', label: 'Exact I/O' },
];
const EXAMPLE_RESIZE_OPTS: { value: ExampleResizePolicy; label: string }[] = [
{ value: 'adapt', label: 'Adapt' },
{ value: 'clear', label: 'Clear' },
];
function SettingsDrawer({ depth }: { ctx: ConsoleCtx; depth: DrawerDepth }) {
const { settings, set } = useSettings();
@ -728,6 +741,47 @@ function SettingsDrawer({ depth }: { ctx: ConsoleCtx; depth: DrawerDepth }) {
</p>
)}
<SectionLabel>I/O editing</SectionLabel>
<div style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)' }}>Network size</div>
<PillToggle
value={settings.networkResizePolicy}
onChange={(v) => set('networkResizePolicy', v as NetworkResizePolicy)}
options={NETWORK_RESIZE_OPTS}
ariaLabel="Network resize policy"
/>
<div style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)' }}>Existing examples</div>
<PillToggle
value={settings.exampleResizePolicy}
onChange={(v) => set('exampleResizePolicy', v as ExampleResizePolicy)}
options={EXAMPLE_RESIZE_OPTS}
ariaLabel="Example resize policy"
/>
{depth === 'expanded' && (
<>
<p style={{ fontSize: 9, color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}>
Keep capacity edits mappings in place and reconstructs only when the active cards outgrow
the network. Exact I/O keeps network arity equal to the cards. Surviving dimensions retain
their identity and weights; exploration scratch state resets after an I/O edit.
</p>
<Slider
label="New input example value"
value={settings.addedInputExampleValue}
min={0}
max={1}
step={0.05}
onChange={(v) => set('addedInputExampleValue', v)}
/>
<Slider
label="New output example value"
value={settings.addedOutputExampleValue}
min={0}
max={1}
step={0.05}
onChange={(v) => set('addedOutputExampleValue', v)}
/>
</>
)}
<SectionLabel>Chrome</SectionLabel>
<Slider
label="Corner radius"

View file

@ -136,7 +136,7 @@ export function OutputStage({ params, values, onChange, compact = false }: Outpu
const placeRight = i > params.length - 4;
return (
<div
key={i}
key={p.id}
style={{
position: 'relative',
flex: 1,

View file

@ -1,86 +0,0 @@
/**
* ReshapeModal confirm modal for reshaping the runtime-shaped net (P2).
*
* Offered when the composed ACTIVE input-axis count changes to something other
* than the net's current input arity (see ConsoleApp's reshape-offer effect).
* The locked decision (manifold-mixed-inputs memory) is a "reset-on-reshape"
* flow: warm-start the weights from the current net, reset examples + explore
* state. Declining keeps the current (over-provisioned, zero-padded) net so
* the default 32-input head behaviour never changes unless the user opts in.
*
* Small inline modal in the house design language (no external dialog dep).
*/
import { Button } from '../primitives';
export interface ReshapeModalProps {
/** Target input arity (the current active axis count). */
target: number;
/** The net's current input arity, for the copy. */
current: number;
onConfirm: () => void;
onCancel: () => void;
}
export function ReshapeModal({ target, current, onConfirm, onCancel }: ReshapeModalProps) {
return (
<div
role="dialog"
aria-modal="true"
aria-label="Reshape the net"
onClick={onCancel}
style={{
position: 'fixed',
inset: 0,
zIndex: 100,
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
background: 'rgba(0,0,0,0.55)',
backdropFilter: 'blur(2px)',
}}
>
<div
onClick={(e) => e.stopPropagation()}
style={{
width: 'min(420px, 90vw)',
background: 'var(--bg-1)',
border: '1px solid var(--line)',
borderRadius: 'var(--r-2)',
boxShadow: '0 12px 40px rgba(0,0,0,0.5)',
fontFamily: 'var(--font-mono)',
color: 'var(--fg)',
padding: 'var(--sp-4, 18px)',
display: 'flex',
flexDirection: 'column',
gap: 14,
}}
>
<div
style={{
fontSize: 'var(--fs-sm)',
textTransform: 'uppercase',
letterSpacing: '0.08em',
color: 'var(--accent)',
}}
>
Reshape the net?
</div>
<p style={{ margin: 0, fontSize: 'var(--fs-sm)', lineHeight: 1.7, color: 'var(--fg)' }}>
Reshape the net to {target} input{target === 1 ? '' : 's'}? Weights are warm-started from
the current {current}-input net; examples and exploration state reset.
</p>
<p style={{ margin: 0, fontSize: 9, lineHeight: 1.6, color: 'var(--fg-dim)' }}>
Decline to keep the current net the extra axes stay zero-padded (inert).
</p>
<div style={{ display: 'flex', gap: 8, justifyContent: 'flex-end' }}>
<Button size="sm" variant="ghost" onClick={onCancel}>
Keep current
</Button>
<Button size="sm" variant="primary" onClick={onConfirm}>
Reshape to {target}
</Button>
</div>
</div>
</div>
);
}

View file

@ -101,6 +101,10 @@ export const DEFAULT_MODE_ML: ModeML = {
* are re-declared here loosely to avoid a consoledock import cycle.
*/
export interface MFParam {
/** Stable semantic identity; array position is presentation, not identity. */
id: string;
/** Current MLP output coordinate, absent while the card is inactive/spare. */
engineIndex?: number;
name: string;
group: string;
status: ParamStatus;
@ -166,7 +170,17 @@ function mkParams(spec: Spec): MFParam[] {
const out: MFParam[] = [];
for (const [group, names] of spec) {
names.forEach((name) =>
out.push({ name, group, status: 'live', val: 0.5, min: 0, max: 1, curve: 0.5 }),
out.push({
id: `${group}:${name}:${out.length}`,
engineIndex: out.length,
name,
group,
status: 'live',
val: 0.5,
min: 0,
max: 1,
curve: 0.5,
}),
);
}
return out;
@ -190,7 +204,9 @@ function mlFromSchema(schema: ModeSchema): ModeML {
* are surfaced as ENGINE-unit metadata for display only.
*/
function paramsFromSchema(schema: ModeSchema): MFParam[] {
return schema.params.map((p) => ({
return schema.params.map((p, i) => ({
id: `${schema.mode_id}:${p.name}`,
engineIndex: i,
name: p.name,
group: p.group,
status: 'live' as ParamStatus,
@ -369,12 +385,32 @@ export function shapeValues(params: MFParam[], engineOut: Float32Array | null):
if (p.status === 'fixed') {
return p.min + Math.max(0, Math.min(1, p.val ?? 0.5)) * (p.max - p.min);
}
const raw = engineOut && i < engineOut.length ? engineOut[i] : 0.5;
const engineIndex = p.engineIndex ?? i;
const raw = engineOut && engineIndex < engineOut.length ? engineOut[engineIndex] : 0.5;
const v = p.min + applyCurve(raw, p.curve) * (p.max - p.min);
return Math.max(0, Math.min(1, v));
});
}
/** Create a backend-agnostic output card after every schema output is active. */
export function createOutputParam(index: number): MFParam {
const suffix = index + 1;
const unique =
typeof crypto !== 'undefined' && typeof crypto.randomUUID === 'function'
? crypto.randomUUID()
: `${Date.now()}-${index}`;
return {
id: `custom:${unique}`,
name: `Output ${suffix}`,
group: 'custom',
status: 'live',
val: 0.5,
min: 0,
max: 1,
curve: 0.5,
};
}
/**
* Map a mode id the audio-engine backend id. Routes on `MFMode.engineId`
* (schema truth `slp_workshop`'s schema already declares `engine_id:

View file

@ -82,12 +82,9 @@ export function outputModeDescriptor(id: OutputMode): OutputModeDescriptor {
}
/**
* Number of output controls the active backend presents.
*
* The model may expose more parameters than a backend currently maps (MIDI is
* the live example: its CC count is adjustable). Keep that presentation
* boundary separate from the model arity so changing a backend count does not
* silently reshape the net and clear its examples.
* Number of active output cards the selected backend presents. Model capacity
* may be larger under the persisted keep-capacity policy; exact-I/O makes this
* count the model arity. The identity-aware engine action owns that distinction.
*/
export function outputDisplayCount(
id: OutputMode,

View file

@ -76,6 +76,8 @@ export interface ConsoleCtx {
params: MFParam[];
/** Patch one output row in the shared store (drives stage + dock in sync). */
setParam: (i: number, patch: Partial<MFParam>) => void;
addOutput: () => void;
deleteOutput: (i: number) => void;
/** Active backend outputs currently presented by the stage + routing rows. */
displayOutputCount: number;
@ -85,7 +87,7 @@ export interface ConsoleCtx {
/** Available Web MIDI output ports (for the MIDI config picker). */
midiPorts: { id: string; name: string }[];
refreshMidiPorts: () => void;
/** MIDI backend settings (selected port + number of CCs mapped). */
/** MIDI backend settings (selected port + active output-card count). */
midiOutputId: string | null;
setMidiOutputId: (id: string | null) => void;
midiCcCount: number;

View file

@ -60,11 +60,19 @@ export interface OutputControlRowProps {
/** Live (computed) value for the value bar. */
value: number;
onChange: (patch: Partial<MFParam>) => void;
/** Remove this semantic output from the active card set. */
onDelete?: () => void;
/** Show the curve pad inline (expand depth); hidden in compact rows. */
showCurve?: boolean;
}
export function OutputControlRow({ param, value, onChange, showCurve = false }: OutputControlRowProps) {
export function OutputControlRow({
param,
value,
onChange,
onDelete,
showCurve = false,
}: OutputControlRowProps) {
const gc = `var(${GROUP_COLOR[param.group] || '--accent'})`;
const muted = param.muted ?? false;
const armed = param.armed ?? false;
@ -99,6 +107,28 @@ export function OutputControlRow({ param, value, onChange, showCurve = false }:
{param.name}
</span>
<span style={{ fontSize: 9, color: 'var(--fg-dim)' }}>{param.group}</span>
{onDelete && (
<button
type="button"
aria-label={`Delete ${param.name} output`}
title={`Delete ${param.name} output`}
onClick={onDelete}
style={{
width: 22,
height: 22,
borderRadius: 'var(--r-1)',
border: '1px solid var(--line)',
background: 'transparent',
color: 'var(--danger)',
cursor: 'pointer',
fontFamily: 'var(--font-mono)',
fontSize: 14,
lineHeight: 1,
}}
>
×
</button>
)}
<GlyphToggle
on={muted}
glyph="M"

View file

@ -354,19 +354,9 @@ function MidiConfig({ ctx }: { ctx: ConsoleCtx }) {
</option>
))}
</select>
<label style={{ display: 'flex', gap: 6, alignItems: 'center', fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)' }}>
CCs
<input
type="number"
min={1}
max={ctx.params.length}
value={ctx.midiCcCount}
onChange={(e) =>
ctx.setMidiCcCount(Math.max(1, Math.min(ctx.params.length, num(e.target.value, ctx.midiCcCount))))
}
style={{ ...cellInput, width: 60 }}
/>
</label>
<span style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)' }}>
{ctx.midiCcCount} CC output{ctx.midiCcCount === 1 ? '' : 's'} · add/delete cards below
</span>
<span style={{ fontSize: 9, color: statusColor }}>{s.message}</span>
</div>
@ -478,7 +468,7 @@ function OscConfig({ ctx }: { ctx: ConsoleCtx }) {
</tr>
</thead>
<tbody>
{ctx.params.map((p, i) => {
{ctx.params.slice(0, ctx.displayOutputCount).map((p, i) => {
const o = p.osc ?? defaultOscSpec(p.name);
return (
<tr key={i}>
@ -558,7 +548,7 @@ function VcvConfig({ ctx }: { ctx: ConsoleCtx }) {
</tr>
</thead>
<tbody>
{ctx.params.map((p, i) => {
{ctx.params.slice(0, ctx.displayOutputCount).map((p, i) => {
const v = p.vcv ?? defaultVcvSpec();
return (
<tr key={i}>
@ -622,7 +612,7 @@ function CvConfig({ ctx }: { ctx: ConsoleCtx }) {
</tr>
</thead>
<tbody>
{ctx.params.map((p, i) => {
{ctx.params.slice(0, ctx.displayOutputCount).map((p, i) => {
const c = (p.cv as { channel: CvChannelId; gateThreshold: number } | undefined) ?? defaultCvSpec(i);
const isGate = c.channel.startsWith('gate');
return (

View file

@ -21,6 +21,7 @@ import type { InputConfig, OutputConfig } from './pipeline-types';
import { Spine, type BackendSend } from './spine';
import type { EngineId, FeedbackMode, LayerStats } from './types';
import { WasmIML } from './wasm-iml';
import type { IoMigration } from './io-reshape';
export interface EngineFeedbackApi {
/** Positive feedback (thumbs-up). Returns the FeedbackAction int. */
@ -315,17 +316,17 @@ export class EngineApi {
}
/**
* Reshape the net to new dims (runtime-shaped MLP; one-core-engine P2). The
* new net is warm-started from the current net's overlapping weights; the
* dataset + feedback/exploration state RESET (front-end shows a confirm modal
* first). Returns true on success. On success the spine re-reads its arity and
* re-ticks the last input so outputs/audio reflect the new net.
* Reconfigure runtime I/O. Without a migration plan this is the legacy
* reconstruct-and-clear operation. With one, stable dimension maps preserve
* semantic weights and optionally adapt examples; same-shape permutations do
* not reconstruct the MLP. Feedback/exploration scratch state always resets.
*/
reshape(
dims: { inputSize?: number; outputSize?: number; hidden?: [number, number, number] },
spread = this.spread_,
migration?: IoMigration,
): boolean {
const ok = this.iml.reshape(dims, spread);
const ok = this.iml.reshape(dims, spread, migration);
if (ok) this.process();
return ok;
}

View file

@ -25,6 +25,18 @@ export type { WasmIMLOptions } from './wasm-iml';
export { EngineHost } from './engine-host';
export { Dataset } from './dataset';
export {
completeDimensionMap,
remapFlatWeights,
remapVector,
resizeTarget,
} from './io-reshape';
export type {
DimensionMap,
ExampleResizePolicy,
IoMigration,
NetworkResizePolicy,
} from './io-reshape';
export { noopSink } from './sink';
export type { EngineSink, EngineStatePatch } from './sink';

View file

@ -0,0 +1,154 @@
/**
* Identity-aware I/O migration for the runtime-shaped MLP.
*
* A dimension map is destination-first: `map[newIndex]` is the old dimension
* whose meaning survives there, or `null` for a newly-created dimension.
* Keeping this algebra in one framework-neutral module prevents card order,
* flat-weight layout, and example migration from becoming three independent
* sources of truth.
*/
import type { MLArchitecture } from './types';
export type DimensionMap = ReadonlyArray<number | null>;
export type ExampleResizePolicy = 'adapt' | 'clear';
export type NetworkResizePolicy = 'capacity' | 'exact';
export interface IoMigration {
inputMap?: DimensionMap;
outputMap?: DimensionMap;
examples: ExampleResizePolicy;
addedInputValue: number;
addedOutputValue: number;
}
/** Fill the inactive tail of an active dimension map with unused old slots. */
export function completeDimensionMap(
active: DimensionMap,
newSize: number,
oldSize: number,
): Array<number | null> {
const out = Array.from({ length: newSize }, (_, i) => active[i] ?? null);
const used = new Set<number>();
for (const oldIndex of out) {
if (oldIndex !== null && oldIndex >= 0 && oldIndex < oldSize) used.add(oldIndex);
}
const spare: number[] = [];
for (let i = 0; i < oldSize; ++i) if (!used.has(i)) spare.push(i);
let next = 0;
for (let i = active.length; i < newSize && next < spare.length; ++i) {
out[i] = spare[next++];
}
return out;
}
function prefixMap(size: number, oldSize: number): Array<number | null> {
return Array.from({ length: size }, (_, i) => (i < oldSize ? i : null));
}
interface FlatLayout {
dims: [number, number, number, number, number];
weightOffsets: [number, number, number, number];
biasOffsets: [number, number, number, number];
}
function flatLayout(arch: MLArchitecture): FlatLayout {
const dims: FlatLayout['dims'] = [
arch.inputSize,
arch.hidden[0],
arch.hidden[1],
arch.hidden[2],
arch.outputSize,
];
const weightOffsets: number[] = [];
let offset = 0;
for (let layer = 0; layer < 4; ++layer) {
weightOffsets.push(offset);
offset += dims[layer] * dims[layer + 1];
}
const biasOffsets: number[] = [];
for (let layer = 0; layer < 4; ++layer) {
biasOffsets.push(offset);
offset += dims[layer + 1];
}
return {
dims,
weightOffsets: weightOffsets as FlatLayout['weightOffsets'],
biasOffsets: biasOffsets as FlatLayout['biasOffsets'],
};
}
/**
* Overlay all surviving semantic coordinates from `oldWeights` onto the
* freshly-initialised destination weights. Hidden layers retain prefix
* identity; input columns and output rows follow the supplied maps.
*/
export function remapFlatWeights(
oldWeights: Float32Array,
oldArch: MLArchitecture,
freshWeights: Float32Array,
newArch: MLArchitecture,
inputMap?: DimensionMap,
outputMap?: DimensionMap,
): Float32Array {
const src = flatLayout(oldArch);
const dst = flatLayout(newArch);
const maps: Array<Array<number | null>> = [
Array.from(inputMap ?? prefixMap(newArch.inputSize, oldArch.inputSize)),
prefixMap(newArch.hidden[0], oldArch.hidden[0]),
prefixMap(newArch.hidden[1], oldArch.hidden[1]),
prefixMap(newArch.hidden[2], oldArch.hidden[2]),
Array.from(outputMap ?? prefixMap(newArch.outputSize, oldArch.outputSize)),
];
const out = new Float32Array(freshWeights);
for (let layer = 0; layer < 4; ++layer) {
const oldIn = src.dims[layer];
const oldOut = src.dims[layer + 1];
const newIn = dst.dims[layer];
const newOut = dst.dims[layer + 1];
const inputIndices = maps[layer];
const outputIndices = maps[layer + 1];
for (let node = 0; node < newOut; ++node) {
const oldNode = outputIndices[node];
if (oldNode === null || oldNode < 0 || oldNode >= oldOut) continue;
for (let input = 0; input < newIn; ++input) {
const oldInput = inputIndices[input];
if (oldInput === null || oldInput < 0 || oldInput >= oldIn) continue;
out[dst.weightOffsets[layer] + node * newIn + input] =
oldWeights[src.weightOffsets[layer] + oldNode * oldIn + oldInput];
}
out[dst.biasOffsets[layer] + node] = oldWeights[src.biasOffsets[layer] + oldNode];
}
}
return out;
}
/** Remap one feature/label vector, filling genuinely new dimensions neutrally. */
export function remapVector(
source: ArrayLike<number>,
map: DimensionMap | undefined,
newSize: number,
placeholder: number,
): Float32Array {
const indices = map ?? prefixMap(newSize, source.length);
const out = new Float32Array(newSize);
for (let i = 0; i < newSize; ++i) {
const oldIndex = indices[i];
out[i] =
oldIndex !== null && oldIndex >= 0 && oldIndex < source.length
? source[oldIndex]
: placeholder;
}
return out;
}
/** Decide whether an arity edit needs a real network reconstruction. */
export function resizeTarget(
activeCount: number,
currentCapacity: number,
policy: NetworkResizePolicy,
): number | null {
const wanted = Math.max(1, Math.floor(activeCount));
if (policy === 'exact') return wanted === currentCapacity ? null : wanted;
return wanted > currentCapacity ? wanted : null;
}

View file

@ -75,6 +75,8 @@ export interface NispsModule {
// RandomiseMlp). Mode ints: 0=Avoid 1=RandomiseOutputs 2=RandomiseMlp.
// Action return ints come from FeedbackController::on_*; see feedback.hpp.
_nisps_ml_feedback_set_mode(ml: number, mode: number): void;
/** Reset index-aligned feedback/exploration state without rebuilding the MLP. */
_nisps_ml_feedback_reset(ml: number): void;
_nisps_ml_feedback_get_mode(ml: number): number;
_nisps_ml_feedback_exploring(ml: number): number; // 1 = exploring
_nisps_ml_feedback_set_focus(ml: number, mask_ptr: number, n: number): void;

View file

@ -38,6 +38,11 @@ import {
type NispsModuleFactory,
} from './types';
import { createTrainer, type WasmTrainer } from './wasm-worker';
import {
remapFlatWeights,
remapVector,
type IoMigration,
} from './io-reshape';
/** Default architecture matches `nisps/wasm/bindings.cpp` instantiation. */
const DEFAULT_INPUT_SIZE = 2;
@ -352,98 +357,145 @@ export class WasmIML {
// -------------------------------------------------------------------
/**
* Swap the net for one at new dims, warm-started from the overlapping weights
* of the current net (`nisps_ml_reshape`). Any omitted dim keeps its current
* value. Returns true on success (false = C-side rejected / no change).
*
* The C side RESETS its dataset/examples and feedback/exploration state on
* reshape, so this method also clears the TS `Dataset` mirror, reallocates
* every dim-dependent heap buffer, refreshes `weightCount`, and pushes the new
* shape + zeroed example/output state through the sink so React re-reads.
* Apply one identity-aware I/O edit. A changed shape reconstructs the C++ MLP;
* a same-shape permutation edits weights/examples in place.
*/
reshape(
dims: { inputSize?: number; outputSize?: number; hidden?: readonly [number, number, number] },
spread = 0,
migration?: IoMigration,
): boolean {
const oldArch = this.arch_;
const wantIn = dims.inputSize ?? this.arch_.inputSize;
const wantOut = dims.outputSize ?? this.arch_.outputSize;
const wantHidden = dims.hidden ?? this.arch_.hidden;
const shapeChanged =
wantIn !== oldArch.inputSize ||
wantOut !== oldArch.outputSize ||
wantHidden.some((n, i) => n !== oldArch.hidden[i]);
const oldWeights = migration ? this.getWeights() : null;
const examples =
migration?.examples === 'adapt'
? Array.from({ length: this.dataset.size }, (_, i) => ({
features: new Float32Array(this.dataset.feature(i)),
labels: new Float32Array(this.dataset.label(i)),
}))
: [];
const hiddenPtr = this.module._malloc(wantHidden.length * 4);
new Int32Array(this.module.HEAP32.buffer, hiddenPtr, wantHidden.length).set(wantHidden);
const ok = this.module._nisps_ml_reshape(
this.mlHandle,
wantIn,
wantOut,
hiddenPtr,
wantHidden.length,
spread,
);
this.module._free(hiddenPtr);
if (ok !== 1) return false;
if (shapeChanged || !migration) {
const hiddenPtr = this.module._malloc(wantHidden.length * 4);
new Int32Array(this.module.HEAP32.buffer, hiddenPtr, wantHidden.length).set(wantHidden);
const ok = this.module._nisps_ml_reshape(
this.mlHandle,
wantIn,
wantOut,
hiddenPtr,
wantHidden.length,
spread,
);
this.module._free(hiddenPtr);
if (ok !== 1) return false;
// Re-describe the (new) instance and refresh the weight count.
this.module._nisps_ml_describe(this.mlHandle, this.describePtr);
const d = new Int32Array(this.module.HEAP32.buffer, this.describePtr, 7);
this.arch_ = {
inputSize: d[0],
hidden: [d[1], d[2], d[3]],
outputSize: d[4],
numLayers: d[5],
maxExamples: d[6],
};
this.weightCount_ = this.module._nisps_ml_weight_count(this.mlHandle);
// nisps_ml_reshape never varies max_examples (no such parameter exists on
// that C API) — it always reconstructs at kDefaultMaxExamples, same as
// create(). The Dataset mirror's cap is therefore still correct; only its
// contents are cleared below, matching the C++ side's dataset reset.
this.module._nisps_ml_describe(this.mlHandle, this.describePtr);
const d = new Int32Array(this.module.HEAP32.buffer, this.describePtr, 7);
this.arch_ = {
inputSize: d[0],
hidden: [d[1], d[2], d[3]],
outputSize: d[4],
numLayers: d[5],
maxExamples: d[6],
};
this.weightCount_ = this.module._nisps_ml_weight_count(this.mlHandle);
// Reallocate every dim-dependent heap buffer. Freeing first then reallocating
// means a later malloc may sbrk-grow the heap and detach earlier views, so we
// rebind() all of them afterwards.
this.featuresBuf.free();
this.labelsBuf.free();
this.weightsBuf.free();
this.statsBuf.free();
this.batchInBuf.free();
this.batchOutBuf.free();
this.pinMaskBuf.free();
this.feedbackBuf.free();
this.outProcBuf.free();
this.pipeMaskBuf.free();
this.featuresBuf = new HeapBuffer(this.module, this.arch_.inputSize);
this.labelsBuf = new HeapBuffer(this.module, this.arch_.outputSize);
this.weightsBuf = new HeapBuffer(this.module, this.weightCount_);
this.statsBuf = new HeapBuffer(this.module, this.arch_.numLayers * 4);
this.batchInBuf = new HeapBuffer(this.module, WasmIML.MAX_BATCH * this.arch_.inputSize);
this.batchOutBuf = new HeapBuffer(this.module, WasmIML.MAX_BATCH * this.arch_.outputSize);
this.pinMaskBuf = new HeapU8(this.module, this.arch_.outputSize);
this.feedbackBuf = new HeapBuffer(this.module, this.arch_.outputSize);
this.outProcBuf = new HeapBuffer(this.module, this.arch_.outputSize);
this.pipeMaskBuf = new HeapU8(this.module, this.arch_.outputSize);
this.featuresBuf.rebind();
this.labelsBuf.rebind();
this.weightsBuf.rebind();
this.statsBuf.rebind();
this.batchInBuf.rebind();
this.batchOutBuf.rebind();
this.pinMaskBuf.rebind();
this.feedbackBuf.rebind();
this.outProcBuf.rebind();
this.pipeMaskBuf.rebind();
// Fixed-size pipeline buffers were not reallocated but a grow above may have
// detached their views — rebind so later writes hit the live heap.
this.inCfgBuf.rebind();
this.inXYBuf.rebind();
this.curveBuf.rebind();
// A malloc may grow WASM memory, so all retained fixed-size views are
// rebound after reallocating the dimension-dependent buffers.
this.featuresBuf.free();
this.labelsBuf.free();
this.weightsBuf.free();
this.statsBuf.free();
this.batchInBuf.free();
this.batchOutBuf.free();
this.pinMaskBuf.free();
this.feedbackBuf.free();
this.outProcBuf.free();
this.pipeMaskBuf.free();
this.featuresBuf = new HeapBuffer(this.module, this.arch_.inputSize);
this.labelsBuf = new HeapBuffer(this.module, this.arch_.outputSize);
this.weightsBuf = new HeapBuffer(this.module, this.weightCount_);
this.statsBuf = new HeapBuffer(this.module, this.arch_.numLayers * 4);
this.batchInBuf = new HeapBuffer(this.module, WasmIML.MAX_BATCH * this.arch_.inputSize);
this.batchOutBuf = new HeapBuffer(this.module, WasmIML.MAX_BATCH * this.arch_.outputSize);
this.pinMaskBuf = new HeapU8(this.module, this.arch_.outputSize);
this.feedbackBuf = new HeapBuffer(this.module, this.arch_.outputSize);
this.outProcBuf = new HeapBuffer(this.module, this.arch_.outputSize);
this.pipeMaskBuf = new HeapU8(this.module, this.arch_.outputSize);
this.featuresBuf.rebind();
this.labelsBuf.rebind();
this.weightsBuf.rebind();
this.statsBuf.rebind();
this.batchInBuf.rebind();
this.batchOutBuf.rebind();
this.pinMaskBuf.rebind();
this.feedbackBuf.rebind();
this.outProcBuf.rebind();
this.pipeMaskBuf.rebind();
this.inCfgBuf.rebind();
this.inXYBuf.rebind();
this.curveBuf.rebind();
} else {
// Pure permutations do not recreate the network. Feedback replay,
// scratchpad, Jolt, and OU state are index-aligned, so reset only those.
this.module._nisps_ml_feedback_reset(this.mlHandle);
const hasNewDimension =
migration.inputMap?.some((oldIndex) => oldIndex === null) ||
migration.outputMap?.some((oldIndex) => oldIndex === null);
if (hasNewDimension) {
// Establish fresh deterministic values, then the remap below overlays
// every surviving coordinate. Only genuinely new columns/rows remain.
this.module._nisps_ml_draw_weights(this.mlHandle, spread);
}
}
// C-side dataset/examples reset on reshape → clear the TS mirror to match.
if (migration && oldWeights) {
this.setWeights(
remapFlatWeights(
oldWeights,
oldArch,
this.getWeights(),
this.arch_,
migration.inputMap,
migration.outputMap,
),
);
}
// Rebuild both example mirrors from the same adapted vectors. The C++
// reshape already cleared its side; same-shape migration needs this call.
this.dataset.clear();
this.module._nisps_ml_clear_examples(this.mlHandle);
if (migration?.examples === 'adapt') {
for (const example of examples) {
const features = remapVector(
example.features,
migration.inputMap,
this.arch_.inputSize,
migration.addedInputValue,
);
const labels = remapVector(
example.labels,
migration.outputMap,
this.arch_.outputSize,
migration.addedOutputValue,
);
this.dataset.add(Array.from(features), Array.from(labels));
this.copyExampleToWasm_(Array.from(features), Array.from(labels));
}
}
this.lastLoss_ = null;
this.resetInput();
this.resetOutput();
// The lazy training worker's mirror net is now stale (wrong arity). Dropping
// it makes the next trainAsync re-create it; the train protocol also carries
// the current dims so a fresh worker matches (see wasm-worker.ts).
// The worker mirror is stale after any weight, example, or shape migration.
if (this.trainer) {
this.trainer.dispose();
this.trainer = null;
@ -452,7 +504,7 @@ export class WasmIML {
this.sink.setState({
inputSize: this.arch_.inputSize,
outputSize: this.arch_.outputSize,
exampleCount: 0,
exampleCount: this.dataset.size,
lastLoss: null,
lossHistory: [],
});
@ -460,6 +512,8 @@ export class WasmIML {
this.sink.emit('ml.reshaped', {
inputSize: this.arch_.inputSize,
outputSize: this.arch_.outputSize,
reconstructed: shapeChanged || !migration,
exampleCount: this.dataset.size,
});
this.scheduleSave_();
return true;

View file

@ -176,6 +176,16 @@ export class FeedbackController {
this.spread = spread;
}
/**
* An I/O identity edit resets the core's index-aligned scratch/replay state.
* Mirror that reset locally without issuing another core transition.
*/
resetAfterIoChange(): void {
this.exploringFlag = false;
this.pickingFlag = false;
this.anchors = [];
}
/**
* Set the arm/solo mask. The dock builds this from the per-output `armed`
* flags (dock/output-state.ts buildArmMask). We RESPECT it at the example

View file

@ -23,9 +23,10 @@
* behaviour) that diluted every source and biased the net toward idle
* sources' resting values.
*
* Changing the ACTIVE axis count offers a reshape (ConsoleApp ReshapeModal):
* a new net at the new arity, warm-started from the overlapping weights, with
* examples + feedback state reset. Declining keeps this over-provisioned head.
* Changing the ACTIVE layout runs through ConsoleApp's persistent I/O policy:
* keep-capacity remaps stable dimensions in place until capacity is exceeded;
* exact-I/O reconstructs to the active arity. Example adaptation/clearing is a
* separate persisted choice.
*/
import type { InputAction, InputSource } from './types';

View file

@ -17,6 +17,10 @@
* read + mutate settings without a render tree.
*/
import { useSyncExternalStore } from 'react';
import type {
ExampleResizePolicy,
NetworkResizePolicy,
} from '../engine/io-reshape';
/** Resting (unfocused) icon colour choice. Focused is always --accent. */
export type UnfocusedIconColour = 'off-white' | 'white' | 'orange';
@ -43,6 +47,17 @@ export interface Settings {
* Learning-drawer control. Off means full-range uniform randomisation.
*/
xavierSpreadEnabled: boolean;
/**
* `capacity` keeps the current network while the edited card set fits;
* `exact` reconstructs whenever active I/O arity changes.
*/
networkResizePolicy: NetworkResizePolicy;
/** What a required I/O migration does with existing training examples. */
exampleResizePolicy: ExampleResizePolicy;
/** Neutral feature value inserted into old examples for a new input. */
addedInputExampleValue: number;
/** Neutral label value inserted into old examples for a new output. */
addedOutputExampleValue: number;
}
export const DEFAULT_SETTINGS: Settings = {
@ -51,6 +66,10 @@ export const DEFAULT_SETTINGS: Settings = {
inputMap: 'follow-mode',
cornerRadius: 2,
xavierSpreadEnabled: false,
networkResizePolicy: 'capacity',
exampleResizePolicy: 'adapt',
addedInputExampleValue: 0,
addedOutputExampleValue: 0.5,
};
const STORAGE_KEY = 'mf-settings';

View file

@ -28,6 +28,7 @@ class FakeBackend implements OutputBackend {
readonly id: BackendId;
startCalls = 0;
teardownCalls = 0;
sent: number[][] = [];
private release: (() => void) | null = null;
private hold: boolean;
@ -59,7 +60,9 @@ class FakeBackend implements OutputBackend {
this.teardownCalls++;
}
send(): void {}
send(routed: Float32Array): void {
this.sent.push(Array.from(routed));
}
status(): BackendStatus {
return { state: 'ready', message: 'fake' };
@ -121,6 +124,32 @@ test('rapid repeated switches to the SAME pending id only apply it once', async
expect(osc.startCalls).toBe(1);
});
test('capacity slots are not forwarded beyond the active output-card count', async () => {
let notify: (() => void) | null = null;
const engine: ManagerEngine = {
subscribe: (cb) => {
notify = cb;
return () => {};
},
routedOutput: () => new Float32Array([0.1, 0.2, 0.3, 0.4]),
audio: { setMuted: () => {} },
};
const midi = new FakeBackend('midi');
const manager = new BackendManager(engine, { midi });
manager.setContext({
modeId: 'test',
outputCount: 2,
mappings: [],
names: [],
});
await manager.setActive('midi');
notify!();
expect(midi.sent).toEqual([
Array.from(new Float32Array([0.1, 0.2])),
]);
});
// NOTE: `manifold/package.json`'s `test` script is `bun test src
// tests/pipeline-golden.test.ts` — an explicit file list, not a directory
// glob, so this file (like any other new file under tests/) is NOT picked up

View file

@ -8,7 +8,7 @@ declare global {
}
}
test('output sliders follow the active backend output count', async ({ page }) => {
test('output cards add/delete in both drawer depths and capacity mode avoids reconstruction', async ({ page }) => {
await loadProbe(page);
// Drive the real UI path: output target → Outputs drawer → expanded MIDI
@ -18,14 +18,25 @@ test('output sliders follow the active backend output count', async ({ page }) =
await page.getByRole('button', { name: 'MIDI', exact: true }).click();
await page.getByTitle('Outputs').click();
await page.getByTitle('Expand').click();
await page.getByLabel('CCs').fill('2');
// MIDI starts with eight output cards. Delete six real cards instead of
// editing a detached numeric count.
await expect(page.getByText('8 outputs', { exact: true })).toBeVisible();
for (let i = 0; i < 6; ++i) {
await page.getByRole('button', { name: /^Delete .* output$/ }).last().click();
}
await expect(page.getByText('2 outputs', { exact: true })).toBeVisible();
// Default "Keep capacity" policy edits semantic mappings in place.
expect(await page.evaluate(() => window.__nisps!.getOutputs().length)).toBe(33);
// The count remains visible in condensed chrome after leaving the advanced
// config, and closing the drawer reveals the same number of stage columns.
// config. Adding in condensed depth updates that same card set.
await page.getByTitle('Condense').click();
await expect(page.getByText('2 outputs', { exact: true })).toBeVisible();
await page.getByRole('button', { name: '+ output', exact: true }).click();
await expect(page.getByText('3 outputs', { exact: true })).toBeVisible();
await page.getByTitle('Close').click();
await expect(page.getByTestId('output-stage')).toHaveAttribute('data-output-count', '2');
await expect(page.getByTestId('output-stage')).toHaveAttribute('data-output-count', '3');
// Backends without a configured count present the full mode output set.
const fullCount = await page.evaluate(() => window.__mf!.paramCount());
@ -33,3 +44,34 @@ test('output sliders follow the active backend output count', async ({ page }) =
await page.getByRole('button', { name: 'OSC', exact: true }).click();
await expect(page.getByTestId('output-stage')).toHaveAttribute('data-output-count', String(fullCount));
});
test('exact I/O persists and adapts examples across a deleted output identity', async ({ page }) => {
await loadProbe(page);
await page.getByTitle('Settings').click();
await page.getByRole('radio', { name: 'Exact I/O' }).click();
await page.getByTitle('Close').click();
await page.getByTitle(/^Mode:/).click();
await page.getByRole('button', { name: 'MIDI', exact: true }).click();
await expect.poll(() => page.evaluate(() => window.__nisps!.getOutputs().length)).toBe(8);
const added = await page.evaluate(() => {
const probe = window.__nisps!;
probe.setFeedbackMode('explore_and_place');
return probe.addExample([0.1, 0.2, 0.3, 0.4], Array.from(probe.getOutputs()));
});
expect(added).toBe(true);
expect(await page.evaluate(() => window.__nisps!.getExampleCount())).toBe(1);
await page.getByTitle('Outputs').click();
await page.getByRole('button', { name: /^Delete .* output$/ }).first().click();
await expect.poll(() => page.evaluate(() => window.__nisps!.getOutputs().length)).toBe(7);
expect(await page.evaluate(() => window.__nisps!.getExampleCount())).toBe(1);
expect(await page.evaluate(() => window.__nisps!.getFeedbackMode())).toBe('explore_and_place');
const persisted = await page.evaluate(() => JSON.parse(localStorage.getItem('mf-settings') ?? '{}'));
expect(persisted.networkResizePolicy).toBe('exact');
expect(persisted.exampleResizePolicy).toBe('adapt');
expect(persisted.addedOutputExampleValue).toBe(0.5);
});

View file

@ -0,0 +1,49 @@
import { expect, test } from 'bun:test';
import type { MLArchitecture } from '../src/engine/types';
import {
completeDimensionMap,
remapFlatWeights,
remapVector,
resizeTarget,
} from '../src/engine/io-reshape';
function arch(inputSize: number, outputSize: number): MLArchitecture {
return { inputSize, hidden: [2, 2, 2], outputSize, numLayers: 4, maxExamples: 8 };
}
test('capacity policy only reconstructs when active I/O exceeds capacity', () => {
expect(resizeTarget(2, 8, 'capacity')).toBeNull();
expect(resizeTarget(9, 8, 'capacity')).toBe(9);
expect(resizeTarget(2, 8, 'exact')).toBe(2);
expect(resizeTarget(8, 8, 'exact')).toBeNull();
});
test('a middle deletion keeps surviving identities and moves unused slots to the tail', () => {
expect(completeDimensionMap([0, 2], 4, 4)).toEqual([0, 2, 1, 3]);
});
test('example vectors remove deleted dimensions and fill additions with placeholders', () => {
expect(Array.from(remapVector([10, 20, 30], [0, 2], 2, -1))).toEqual([10, 30]);
expect(Array.from(remapVector([10, 20], [0, null, 1], 3, 0.5))).toEqual([10, 0.5, 20]);
});
test('weight remap preserves an arbitrary output row and bias by identity', () => {
const oldArch = arch(2, 3);
const newArch = arch(2, 2);
// 4 layers: weights 4 + 4 + 4 + 6, then biases 2 + 2 + 2 + 3 = 27.
const oldWeights = Float32Array.from({ length: 27 }, (_, i) => i + 1);
const freshWeights = new Float32Array(24).fill(-1);
const remapped = remapFlatWeights(
oldWeights,
oldArch,
freshWeights,
newArch,
undefined,
[0, 2],
);
// Final-layer weights begin at 12. Old rows: [13,14], [15,16], [17,18].
expect(Array.from(remapped.slice(12, 16))).toEqual([13, 14, 17, 18]);
// Destination output biases are the final two entries; old output biases
// are [25,26,27], so output identity 2 must retain 27 rather than 26.
expect(Array.from(remapped.slice(22, 24))).toEqual([25, 27]);
});

View file

@ -401,6 +401,8 @@ int nisps_ml_reshape(void* ml, int input_size, int output_size,
const MlDims d = sanitise_dims(input_size, output_size, hidden, n_hidden);
if (!d.ok) return 0;
const auto feedback_mode = h->feedback.mode();
const auto avoid_style = h->feedback.avoid_style();
BrowserMLP fresh(h->seed64, d.n_in, std::span<const std::size_t>(d.hidden, 3u), d.n_out);
if (!fresh.valid()) return 0;
fresh.draw_weights(spread);
@ -409,6 +411,8 @@ int nisps_ml_reshape(void* ml, int input_size, int output_size,
BrowserFeedback fb(h->seed64 ^ kFeedbackSalt, d.n_out, fresh.weight_count(),
kFeedbackUndoDepth, d.n_in, kFeedbackReplayCap);
if (!fb.valid()) return 0;
fb.set_avoid_style(avoid_style);
fb.set_mode(feedback_mode, fresh);
h->mlp = static_cast<BrowserMLP&&>(fresh);
h->feedback = static_cast<BrowserFeedback&&>(fb);
@ -614,6 +618,28 @@ void nisps_ml_feedback_set_mode(void* ml, int mode) {
h->feedback.set_mode(m, h->mlp);
}
// Reset only the index-aligned feedback/exploration state. Same-capacity I/O
// edits remap weights and examples without reconstructing the network, but
// replay/scratch vectors have no stable parameter-identity contract.
EMSCRIPTEN_KEEPALIVE
void nisps_ml_feedback_reset(void* ml) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
const auto feedback_mode = h->feedback.mode();
const auto avoid_style = h->feedback.avoid_style();
BrowserFeedback fb(h->seed64 ^ kFeedbackSalt, h->n_out(), h->mlp.weight_count(),
kFeedbackUndoDepth, h->n_in(), kFeedbackReplayCap);
if (!fb.valid()) return;
fb.set_avoid_style(avoid_style);
fb.set_mode(feedback_mode, h->mlp);
h->feedback = static_cast<BrowserFeedback&&>(fb);
h->feedback_static_scratch.assign(h->n_out(), 0.f);
h->mlp.reset_optimizer_state();
h->jolt.release();
h->ou.reset();
h->jolt_scratch.assign(h->mlp.weight_count(), 0.f);
}
EMSCRIPTEN_KEEPALIVE
int nisps_ml_feedback_get_mode(void* ml) {
if (!ml) return 0;

View file

@ -46,7 +46,7 @@ EXPORTED_FUNCS='[
"_nisps_ml_clear_examples",
"_nisps_ml_weight_count","_nisps_ml_get_weights","_nisps_ml_set_weights",
"_nisps_ml_draw_weights",
"_nisps_ml_feedback_set_mode","_nisps_ml_feedback_get_mode",
"_nisps_ml_feedback_set_mode","_nisps_ml_feedback_reset","_nisps_ml_feedback_get_mode",
"_nisps_ml_feedback_exploring",
"_nisps_ml_feedback_set_focus","_nisps_ml_feedback_down",
"_nisps_ml_feedback_up","_nisps_ml_feedback_static_output",