w1n5t0n/memlnaut-nisps
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge remote-tracking branch 'origin/main' into workshop/synth-fw-audit

Browse source 
# Conflicts:
#	nisps/modes/base.hpp
This commit is contained in:
monkey-w1n5t0n 2026-06-28 22:29:41 +02:00
commit c45aff4411
21 changed files with 1595 additions and 200 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
CLAUDE.md
View file

@ -58,7 +58,8 @@ firmware/MEMLNaut-NISPS/
│ ├── midi_io.hpp # MIDI in → mode handlers; drain ControlEvent ring → MIDI UART │ ├── midi_io.hpp # MIDI in → mode handlers; drain ControlEvent ring → MIDI UART
│ ├── mode_select.hpp # type aliases firmware mode name → nisps::modes::*Mode │ ├── mode_select.hpp # type aliases firmware mode name → nisps::modes::*Mode
│ ├── input_router.hpp # wire_inputs() entry point │ ├── input_router.hpp # wire_inputs() entry point
│ └── output_router.hpp # drain_outputs() entry point │ ├── output_router.hpp # drain_outputs() entry point
│ └── settings_view.hpp # wire_settings(): TFT/rotary menu (Joystick Dual/Single for 4-in modes)
└── src/{memllib,daisysp,nisps} # symlinks (Arduino-CLI sketch tree convention) └── src/{memllib,daisysp,nisps} # symlinks (Arduino-CLI sketch tree convention)
``` ```

1
MAP.md
View file

@ -23,6 +23,7 @@ MEMLNaut-NISPS — Neural Interactive Shaping of Parameter Spaces. One C++20 cod
- `mode_select.hpp` — type aliases mapping firmware mode identifiers to `nisps::modes::*Mode` C++ types. Build script rewrites the active line. Includes the six `MEMLNautModeExtSynth*` external-synth variants (one per device template in `nisps/midi`, e.g. `MEMLNautModeExtSynthSub37`). Also defines the `MEMLNautModeSelfTest` pseudo-variant (tag type) + the `NISPS_ST_*`/`NISPS_ST_CAT` token-paste macros the `.ino` uses to compute `NISPS_SELFTEST`. Note: `src/nisps/` exposes each referenced top-level nisps subdir as a symlink — `midi` was added alongside `core/dsp/engines/ml/modes`. - `mode_select.hpp` — type aliases mapping firmware mode identifiers to `nisps::modes::*Mode` C++ types. Build script rewrites the active line. Includes the six `MEMLNautModeExtSynth*` external-synth variants (one per device template in `nisps/midi`, e.g. `MEMLNautModeExtSynthSub37`). Also defines the `MEMLNautModeSelfTest` pseudo-variant (tag type) + the `NISPS_ST_*`/`NISPS_ST_CAT` token-paste macros the `.ino` uses to compute `NISPS_SELFTEST`. Note: `src/nisps/` exposes each referenced top-level nisps subdir as a symlink — `midi` was added alongside `core/dsp/engines/ml/modes`.
- `selftest.hpp` — standalone guided hardware self-test rig (`SelfTest` variant; no engine/ML). Step-driven state machine on a `SelfTestView`: TFT prompts the operator through every control, auto-advances on detection, encoder-press skips. Ends with optional L/R/BOTH sine-sweep headphone check (core 1 block callback) + MIDI loopback-cable test. Lives firmware-side (touches TFT + raw pins) so it stays out of platform-agnostic `nisps/`. - `selftest.hpp` — standalone guided hardware self-test rig (`SelfTest` variant; no engine/ML). Step-driven state machine on a `SelfTestView`: TFT prompts the operator through every control, auto-advances on detection, encoder-press skips. Ends with optional L/R/BOTH sine-sweep headphone check (core 1 block callback) + MIDI loopback-cable test. Lives firmware-side (touches TFT + raw pins) so it stays out of platform-agnostic `nisps/`.
- `input_router.hpp`, `output_router.hpp` — top-level `wire_inputs()` / `drain_outputs()` entry points. - `input_router.hpp`, `output_router.hpp` — top-level `wire_inputs()` / `drain_outputs()` entry points.
- `settings_view.hpp` — `wire_settings(mode)`: adds on-device settings views to the MEMLNaut display carousel (TFT + rotary encoder). Joystick Dual/Single toggle for the 4-input ("two 2-D joystick") modes — "Single" pins ML input channels 2,3 to neutral via `ModeBase::set_input_pinned` (no net rebuild). Registered in the `.ino` after `addSystemInfoView()`.
- `firmware/MEMLNaut-NISPS/src/{memllib,daisysp,nisps}` — symlinks (Arduino-CLI requires sketch-tree includes; preprocessor refuses `..` in headers). - `firmware/MEMLNaut-NISPS/src/{memllib,daisysp,nisps}` — symlinks (Arduino-CLI requires sketch-tree includes; preprocessor refuses `..` in headers).
- `firmware/README.md` — structure + build instructions. - `firmware/README.md` — structure + build instructions.

484
assets/media/learn/index.html Normal file
View file

@ -0,0 +1,484 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Manifold — the parameter landscape</title>
<style>
:root{
--bg:#0d0d0d; --bg-1:#141414; --bg-2:#1c1c1c; --bg-3:#242424;
--fg:#e8e8e8; --fg-mute:#9a9a9a; --fg-dim:#5a5a5a;
--line:#2a2a2a; --line-strong:#3a3a3a;
--accent:#ff6a00; --accent-2:#00ccff; --good:#6bc26b; --bad:#5aa6c8;
--font-mono:'JetBrains Mono','Fira Code','SF Mono',Menlo,Consolas,monospace;
}
*{box-sizing:border-box}
html,body{margin:0;height:100%;background:var(--bg);color:var(--fg);
font-family:var(--font-mono);font-size:14px;-webkit-font-smoothing:antialiased;overflow:hidden}
#app{position:fixed;inset:0}
#scene{position:absolute;inset:0;width:100%;height:100%;display:block;touch-action:none;cursor:grab}
#scene:active{cursor:grabbing}
/* floating HUD blocks */
.hud{position:absolute;z-index:5}
/* three zones, left → right: input · MLP stack · outputs */
.hud-l{left:24px;top:50%;transform:translateY(-50%);display:flex;flex-direction:column;gap:10px;align-items:flex-start}
.hud-r{right:24px;top:50%;transform:translateY(-50%)}
.hud-bc{left:50%;bottom:22px;transform:translateX(-50%)}
.hud-tr{right:18px;top:18px}
.zonecap{font-size:9px;letter-spacing:.16em;text-transform:uppercase;color:var(--fg-dim);margin-bottom:2px}
/* output fader bank (1-D projection of each layer's value at the point) */
.outs{display:flex;gap:13px;align-items:flex-start;background:rgba(20,20,20,.55);
border:1px solid var(--line);border-radius:12px;padding:12px 14px;backdrop-filter:blur(4px)}
.fader{display:flex;flex-direction:column;align-items:center;gap:6px;width:30px}
.fader .track{position:relative;width:8px;height:200px;background:var(--bg-3);
border:1px solid var(--line);border-radius:999px;overflow:hidden}
.fader .fill{position:absolute;left:0;right:0;bottom:0;border-radius:999px;transition:height .05s linear}
.fader .handle{position:absolute;left:50%;width:16px;height:3px;border-radius:2px;
transform:translate(-50%,1.5px);box-shadow:0 0 8px currentColor}
.fader .flab{font-size:9px;letter-spacing:.03em;text-transform:uppercase;max-width:34px;text-align:center;line-height:1.1}
.fader .fval{font-size:10px;font-variant-numeric:tabular-nums;color:var(--fg)}
/* XY pad */
#ndpad{display:block;border:1px solid var(--line);border-radius:8px;background:var(--bg-2);
cursor:crosshair;touch-action:none}
.padcap{font-size:10px;letter-spacing:.1em;text-transform:uppercase;color:var(--fg-dim)}
/* parameter numberbox */
.kbox{display:flex;align-items:center;gap:8px;background:var(--bg-1);border:1px solid var(--line);
border-radius:8px;padding:5px 8px;width:fit-content}
.kbox .lab{font-size:10px;letter-spacing:.08em;text-transform:uppercase;color:var(--fg-mute)}
.kbox .val{min-width:1.4ch;text-align:center;color:var(--fg);font-size:15px;font-variant-numeric:tabular-nums}
.step{width:24px;height:24px;display:grid;place-items:center;padding:0;font-size:15px;line-height:1;
background:var(--bg-2);border:1px solid var(--line-strong);border-radius:6px;color:var(--fg);cursor:pointer}
.step:hover{background:var(--bg-3)}
.step:disabled{opacity:.3;cursor:default}
/* action bar */
.actionbar{display:flex;align-items:center;gap:8px;background:rgba(20,20,20,.82);
border:1px solid var(--line);border-radius:12px;padding:8px;backdrop-filter:blur(6px)}
.act{display:flex;flex-direction:column;align-items:center;gap:3px;min-width:58px;padding:8px 6px;
background:var(--bg-2);border:1px solid var(--line-strong);border-radius:9px;color:var(--fg);
cursor:pointer;font-family:inherit;transition:border-color .12s,background .12s}
.act:hover{background:var(--bg-3)}
.act .ic{font-size:18px;line-height:1}
.act .tx{font-size:9px;letter-spacing:.04em;color:var(--fg-mute);text-transform:uppercase}
.act.up:hover{border-color:var(--accent);box-shadow:0 0 12px rgba(255,106,0,.25)}
.act.down:hover{border-color:var(--accent-2);box-shadow:0 0 12px rgba(0,204,255,.22)}
.act:disabled{opacity:.35;cursor:default;box-shadow:none}
.sep{width:1px;align-self:stretch;background:var(--line);margin:2px 2px}
/* mode toggle */
.modeseg{display:inline-flex;border:1px solid var(--line-strong);border-radius:9px;overflow:hidden}
.modeseg button{border:0;border-radius:0;padding:0 12px;height:100%;min-height:54px;font-family:inherit;
font-size:11px;letter-spacing:.04em;background:var(--bg-2);color:var(--fg-mute);cursor:pointer}
.modeseg button.on{background:var(--accent);color:#0d0d0d}
/* info toggle */
#infoBtn{width:34px;height:34px;border-radius:50%;border:1px solid var(--line-strong);background:rgba(20,20,20,.82);
color:var(--fg-mute);font-size:15px;cursor:pointer;backdrop-filter:blur(6px)}
#infoBtn:hover{color:var(--accent);border-color:var(--accent)}
/* fading rotate hint */
#hint{position:absolute;left:50%;top:18px;transform:translateX(-50%);z-index:4;
font-size:11px;letter-spacing:.08em;color:var(--fg-dim);pointer-events:none;
transition:opacity 1s ease;opacity:.85}
#hint.gone{opacity:0}
/* loading */
#loading{position:absolute;inset:0;display:grid;place-items:center;z-index:9;background:var(--bg);
color:var(--fg-dim);font-size:13px;letter-spacing:.1em}
#loading.gone{display:none}
#loading b{color:var(--accent);letter-spacing:-.01em}
/* prose drawer (revealed by the info switch) */
#prose{position:absolute;top:0;right:0;bottom:0;width:min(440px,86vw);z-index:8;overflow:auto;
background:rgba(13,13,13,.96);border-left:1px solid var(--line);padding:40px 26px 64px;
transform:translateX(102%);transition:transform .28s ease}
body:not(.minimal) #prose{transform:none}
#prose h1{font-size:24px;margin:0 0 8px}
#prose h1 b{color:var(--accent)}
#prose p{color:var(--fg-mute);line-height:1.6;margin:0 0 12px}
#prose p b{color:var(--fg)}
#prose .lbl{font-size:11px;letter-spacing:.14em;text-transform:uppercase;color:var(--accent);margin:26px 0 8px}
#prose .ph{font-size:10px;letter-spacing:.1em;text-transform:uppercase;color:var(--fg-dim);
border:1px dashed var(--line-strong);border-radius:6px;padding:6px 9px;display:inline-block;margin-bottom:14px}
#prose .bars{display:flex;flex-direction:column;gap:7px;margin-top:6px}
#prose .ro{display:flex;align-items:center;gap:10px}
#prose .ro span{width:74px;font-size:10px;text-transform:uppercase;letter-spacing:.05em;color:var(--fg-mute)}
#prose .bar{flex:1;height:7px;background:var(--bg-3);border:1px solid var(--line);border-radius:999px;overflow:hidden;position:relative}
#prose .bar i{position:absolute;left:0;top:0;bottom:0;border-radius:999px}
#prose .num{width:3.2ch;text-align:right;font-variant-numeric:tabular-nums;color:var(--fg);font-size:12px}
#prose a{color:var(--accent-2);text-decoration:none}
</style>
</head>
<body class="minimal">
<div id="app">
<canvas id="scene"></canvas>
<div id="hint">drag to rotate · top-down view = the flat 2-D map</div>
<!-- ZONE 1 · input: the 2-D control + parameter count (left) -->
<div class="hud hud-l">
<div class="zonecap">input · 2-D control</div>
<canvas id="ndpad" width="140" height="140"></canvas>
<div class="kbox">
<span class="lab">params</span>
<button class="step" id="kdown" title="fewer parameters">−</button>
<span class="val" id="kval">1</span>
<button class="step" id="kup" title="more parameters">+</button>
</div>
</div>
<!-- ZONE 3 · outputs: vertical faders, one per layer (right) -->
<div class="hud hud-r">
<div class="zonecap">outputs</div>
<div class="outs" id="outs"></div>
</div>
<!-- action bar (bottom-center) -->
<div class="hud hud-bc actionbar">
<button class="act" id="aRand"><span class="ic">🎲</span><span class="tx">randomise</span></button>
<button class="act" id="aNudge"><span class="ic">〜</span><span class="tx">nudge</span></button>
<button class="act" id="aUndo"><span class="ic">↶</span><span class="tx">undo</span></button>
<div class="sep"></div>
<button class="act up" id="aUp"><span class="ic">👍</span><span class="tx">keep</span></button>
<button class="act down" id="aDown"><span class="ic">👎</span><span class="tx" id="downTx">push away</span></button>
<div class="sep"></div>
<div class="modeseg">
<button id="mPush" class="on">push&#8209;away</button>
<button id="mExp">explore</button>
</div>
</div>
<button id="infoBtn" class="hud hud-tr" title="show / hide explanation">ⓘ</button>
<div id="loading"><span>loading the <b>&nbsp;NISPS&nbsp;</b> engine…</span></div>
</div>
<!-- placeholder explanatory copy (revealed by the ⓘ switch; to be reworked) -->
<div id="prose">
<h1>The <b>parameter landscape</b></h1>
<p class="ph">placeholder copy — to be reworked</p>
<p>Two numbers go in — your <b>X / Y control position</b> — and a whole fistful of parameters comes
out. You can't picture a dozen dimensions at once, so picture them <b>stacked</b>: each <b>slab</b> is one
parameter, drawn as a 3-D landscape whose height at any spot is that parameter's value. <b>One probe</b>
reads every slab at the same control position.</p>
<p>Seen straight from above the slabs overlap into a single flat map — it looks 2-D. <b>Tilt it</b> and the
reveal lands: it was a stack of 3-D surfaces all along. Add parameters to grow the sandwich.</p>
<p>Every surface here is the <b>real NISPS network</b> sampled across the input space. <b>👍 keep</b> teaches
the net that this sound belongs at this spot and the landscape bends to honour it; <b>👎</b> either pushes the
mapping away from the current sound or (in explore mode) re-rolls a fresh candidate to audition. <b>nudge</b>
and <b>randomise</b> perturb the net; <b>undo</b> steps back.</p>
<div class="lbl">Values under the probe</div>
<div class="bars" id="ndbars"></div>
<p style="margin-top:24px"><a href="../">← back to the instrument</a> · <a href="../animations/">animated explainers →</a></p>
</div>
</div>
<script>
(() => {
'use strict';
const clamp=(v,a=0,b=1)=>v<a?a:v>b?b:v;
const cssv=n=>getComputedStyle(document.documentElement).getPropertyValue(n).trim();
const MONO=cssv('--font-mono')||'monospace';
const NAMES=['Cutoff','Resonance','Drive','Tone','Decay','Air','Pitch','Width'];
const ABBR =['Cut','Res','Drv','Ton','Dec','Air','Pit','Wid'];
const MAXK=NAMES.length;
const OUT=126, IN=2, GRID=12, N=GRID, PTS=(N+1)*(N+1);
let K=1; // parameters (slabs) visualised — from the same net
let cx=0.5, cy=0.5; // 2-D control position in [0,1]
let mode='pushaway'; // 'pushaway' | 'explore'
let exploring=false; // explore-and-place scratchpad session active
let ready=false, auto=true, autoT=0.6;
// camera — START TOP-DOWN (looks 2-D); drag tilts to reveal the 3-D stack
let yaw=0, pitch=Math.PI/2, dragRot=false, lastX=0, lastY=0;
// ===== engine (vanilla wrapper over nisps.wasm; same C ABI the manifold uses) =====
const E={
mod:null,h:0,bIn:0,bOut:0,bF:0,bL:0,bW:0,bFb:0,wc:0,
async init(){
// nisps.{js,wasm} sit one level up at /next/ in production; probe a few bases.
let base=null,src=null;
for(const b of ['../','','./']){
try{const r=await fetch(b+'nisps.js');if(r.ok){src=await r.text();base=b;break;}}catch(e){}
}
if(base===null) throw new Error('nisps.js not found');
if(!window.createNispsModule)(0,eval)(src);
this.mod=await window.createNispsModule({locateFile:p=>p.endsWith('.wasm')?base+'nisps.wasm':p});
const m=this.mod;
this.h=m._nisps_ml_create(IN,OUT,0,0,(Date.now()>>>0));
this.wc=m._nisps_ml_weight_count(this.h);
this.bF=m._malloc(IN*4); this.bL=m._malloc(OUT*4); this.bFb=m._malloc(OUT*4);
this.bW=m._malloc(this.wc*4);
this.bIn=m._malloc(PTS*IN*4); this.bOut=m._malloc(PTS*OUT*4);
},
inferXY(x,y){const m=this.mod;m._nisps_ml_set_input(this.h,0,x);m._nisps_ml_set_input(this.h,1,y);
m._nisps_ml_process(this.h);const p=m._nisps_ml_outputs(this.h);
return new Float32Array(m.HEAPF32.buffer,p,OUT);},
inferGrid(){const m=this.mod;const inv=new Float32Array(m.HEAPF32.buffer,this.bIn,PTS*IN);
let q=0;for(let j=0;j<=N;j++)for(let i=0;i<=N;i++){inv[q++]=i/N;inv[q++]=j/N;}
m._nisps_ml_infer_batch(this.h,this.bIn,PTS,this.bOut);
return new Float32Array(new Float32Array(m.HEAPF32.buffer,this.bOut,PTS*OUT));},
addExample(f,l){const m=this.mod;const fv=new Float32Array(m.HEAPF32.buffer,this.bF,IN);
const lv=new Float32Array(m.HEAPF32.buffer,this.bL,OUT);fv[0]=f[0];fv[1]=f[1];
for(let i=0;i<OUT;i++)lv[i]=l[i]||0;m._nisps_ml_add_example(this.h,this.bF,this.bL);},
train(lr,it,me){return this.mod._nisps_ml_train(this.h,lr,it,me,0);},
draw(s){this.mod._nisps_ml_draw_weights(this.h,s);},
move(sp,s){this.mod._nisps_ml_move_weights(this.h,sp,s,0);},
setMode(i){this.mod._nisps_ml_feedback_set_mode(this.h,i);},
down(out,sp,s){const m=this.mod;const ov=new Float32Array(m.HEAPF32.buffer,this.bFb,OUT);
ov.fill(0);for(let i=0;i<OUT;i++)ov[i]=out[i]||0;m._nisps_ml_feedback_down(this.h,this.bFb,sp,s,0);},
enterExplore(s){this.mod._nisps_ml_feedback_enter_explore(this.h,s);},
reroll(s){this.mod._nisps_ml_feedback_reroll(this.h,s);},
coreNudge(a){this.mod._nisps_ml_feedback_nudge(this.h,a);},
coreUndo(){this.mod._nisps_ml_feedback_undo(this.h);},
exitExplore(){this.mod._nisps_ml_feedback_exit_explore(this.h);},
getWeights(){this.mod._nisps_ml_get_weights(this.h,this.bW);
return new Float32Array(new Float32Array(this.mod.HEAPF32.buffer,this.bW,this.wc));},
setWeights(w){const v=new Float32Array(this.mod.HEAPF32.buffer,this.bW,this.wc);v.set(w);
this.mod._nisps_ml_set_weights(this.h,this.bW);},
};
// Low spread = polarised weights = strong spatial relief (measured); high spread
// flattens the surface toward 0.5. Keep it low so the landscapes show real terrain.
const SPREAD=0.1, NUDGE=0.05, BOOT_SPREAD=0.05;
let grid=null, probeOut=new Float32Array(OUT);
function H(k,i,j){return grid?grid[(j*(N+1)+i)*OUT+k]:0.5;}
function resample(){grid=E.inferGrid();}
function updateProbe(){const o=E.inferXY(cx,cy);probeOut=new Float32Array(o);}
// ---- weight-snapshot undo (push-away mode; explore mode uses the core ring) ----
const undoStack=[];
function pushUndo(){undoStack.push(E.getWeights());if(undoStack.length>24)undoStack.shift();}
function popUndo(){const w=undoStack.pop();if(w){E.setWeights(w);return true;}return false;}
// ---- feedback examples marked on the 2-D control plane (like the main app) ----
// {x,y,sign} — sign +1 kept (👍), -1 pushed away (👎). undo pops the last.
const marks=[];
// ===== actions (all backed by the real engine) =====
function act(name){
if(!ready)return;
if(mode==='pushaway'){
if(name==='rand'){pushUndo();E.draw(SPREAD);}
else if(name==='nudge'){pushUndo();E.move(0.04,SPREAD);}
else if(name==='undo'){popUndo();marks.pop();}
else if(name==='up'){pushUndo();const o=new Float32Array(E.inferXY(cx,cy));E.addExample([cx,cy],o);E.train(1.0,200,1e-4);marks.push({x:cx,y:cy,sign:1});}
else if(name==='down'){pushUndo();const o=new Float32Array(E.inferXY(cx,cy));E.down(o,0.15,SPREAD);marks.push({x:cx,y:cy,sign:-1});}
}else{ // explore-and-place
if(name==='rand'||name==='down'){
if(!exploring){E.setMode(3);E.enterExplore(SPREAD);exploring=true;}else E.reroll(SPREAD);
}else if(name==='nudge'){
if(!exploring){E.setMode(3);E.enterExplore(SPREAD);exploring=true;}else E.coreNudge(NUDGE);
}else if(name==='undo'){
if(exploring)E.coreUndo();else{popUndo();marks.pop();}
}else if(name==='up'){
const o=new Float32Array(E.inferXY(cx,cy));
if(exploring){E.exitExplore();exploring=false;}
E.addExample([cx,cy],o);E.train(1.0,200,1e-4);marks.push({x:cx,y:cy,sign:1});
}
}
resample();updateProbe();reflectButtons();redraw();
}
function setMode(m){
if(m===mode)return;
if(mode==='explore'&&exploring){E.exitExplore();exploring=false;}
mode=m;E.setMode(m==='explore'?3:0);
document.getElementById('mPush').classList.toggle('on',m==='pushaway');
document.getElementById('mExp').classList.toggle('on',m==='explore');
document.getElementById('downTx').textContent=m==='explore'?'skip':'push away';
if(ready){resample();updateProbe();reflectButtons();redraw();}
}
function reflectButtons(){
const ud=document.getElementById('aUndo');
ud.disabled=(mode==='explore'?false:undoStack.length===0)&&!exploring;
}
// ===== 3-D projection: yaw + pitch orbit, bounding-sphere fit, X inverted =====
const SX=300,SZ=300,LAYER_GAP=120,ZAMP=92,MARGIN=44;
// Anchor the view to the TOP slab (k=0) at a FIXED world Y and a FIXED screen
// position, independent of K. So adding parameters grows the stack DOWNWARD and
// the top slab always stays the top — it never re-centres onto new layers.
const AWY=ZAMP*0.5, ANCHOR_FY=0.42;
let CW=800,CH=600,scale=1;
function computeView(){
let R=1; // bounding-sphere radius about the top anchor
for(let k=0;k<K;k++)for(const z of[0,1])for(const gx of[-0.5,0.5])for(const gy of[-0.5,0.5]){
const X=gx*SX,Y=(-k*LAYER_GAP+z*ZAMP)-AWY,Z=gy*SZ;const d=Math.sqrt(X*X+Y*Y+Z*Z);if(d>R)R=d;}
scale=(Math.min(CW,CH)/2-MARGIN)/R;
}
function project(gx,gy,zv,k){
const X=gx*SX, Y=(-k*LAYER_GAP+zv*ZAMP)-AWY, Z=-gy*SZ; // Z negated: pad-up = map-up
const cyA=Math.cos(yaw),syA=Math.sin(yaw);
const x1=X*cyA-Z*syA, z1=X*syA+Z*cyA;
const cpA=Math.cos(pitch),spA=Math.sin(pitch);
const y2=Y*cpA-z1*spA, z2=Y*spA+z1*cpA;
return {x:CW/2+x1*scale,y:CH*ANCHOR_FY-y2*scale,depth:z2,wx:X,wy:Y,wz:Z};
}
const DPR=Math.max(1,Math.min(2,window.devicePixelRatio||1));
const scene=document.getElementById('scene'),sctx=scene.getContext('2d');
const pad=document.getElementById('ndpad'),pctx=pad.getContext('2d');
pad.width=140*DPR;pad.height=140*DPR;pctx.scale(DPR,DPR);
function resize(){
const r=scene.getBoundingClientRect();CW=Math.max(320,Math.round(r.width));CH=Math.max(320,Math.round(r.height));
scene.width=CW*DPR;scene.height=CH*DPR;sctx.setTransform(DPR,0,0,DPR,0,0);computeView();
}
const hueOf=k=>K>1?(26+(196-26)*k/(K-1)):26;
const colCss=(k,a)=>`hsla(${hueOf(k).toFixed(0)},85%,60%,${a})`;
const LIGHT=(()=>{const v=[0.35,0.86,0.36],m=Math.hypot(v[0],v[1],v[2]);return[v[0]/m,v[1]/m,v[2]/m];})();
function renderScene(){
const c=sctx;c.clearRect(0,0,CW,CH);
const quads=[],labels=[];
for(let k=0;k<K;k++){
const hue=hueOf(k),P=[];
for(let j=0;j<=N;j++){P[j]=[];for(let i=0;i<=N;i++)P[j][i]=project(i/N-0.5,j/N-0.5,H(k,i,j),k);}
for(let j=0;j<N;j++)for(let i=0;i<N;i++){
const a=P[j][i],b=P[j][i+1],d=P[j+1][i+1],e=P[j+1][i];
const ux=d.wx-a.wx,uy=d.wy-a.wy,uz=d.wz-a.wz,vx=b.wx-e.wx,vy=b.wy-e.wy,vz=b.wz-e.wz;
const nx=uy*vz-uz*vy,ny=uz*vx-ux*vz,nz=ux*vy-uy*vx,nm=Math.hypot(nx,ny,nz)||1;
const L=clamp(0.32+0.62*Math.abs((nx*LIGHT[0]+ny*LIGHT[1]+nz*LIGHT[2])/nm));
quads.push({a,b,d,e,hue,L,depth:(a.depth+b.depth+d.depth+e.depth)/4});
}
const lp=project(-0.5,0.5,H(k,0,N),k);labels.push({x:lp.x,y:lp.y,hue,name:NAMES[k]});
}
quads.sort((p,q)=>q.depth-p.depth);
for(const Q of quads){
c.fillStyle=`hsla(${Q.hue},80%,${(26+Q.L*44).toFixed(0)}%,0.66)`;
c.strokeStyle=`hsla(${Q.hue},80%,82%,0.18)`;c.lineWidth=0.6;
c.beginPath();c.moveTo(Q.a.x,Q.a.y);c.lineTo(Q.b.x,Q.b.y);c.lineTo(Q.d.x,Q.d.y);c.lineTo(Q.e.x,Q.e.y);c.closePath();
c.fill();c.stroke();
}
c.font='11px '+MONO;c.textAlign='right';c.textBaseline='middle';
for(const Lb of labels){c.fillStyle=`hsla(${Lb.hue},85%,74%,0.95)`;c.fillText(Lb.name,Lb.x-6,Lb.y);}
// probe column at (cx,cy) through every slab
const gx=cx-0.5,gy=cy-0.5;
const top=project(gx,gy,1.18,0),bot=project(gx,gy,-0.18,K-1);
c.strokeStyle='rgba(255,255,255,0.32)';c.lineWidth=1;c.setLineDash([3,4]);
c.beginPath();c.moveTo(top.x,top.y);c.lineTo(bot.x,bot.y);c.stroke();c.setLineDash([]);
for(let k=0;k<K;k++){
const p=project(gx,gy,clamp(probeOut[k]),k),fl=project(gx,gy,0,k);
c.strokeStyle=colCss(k,0.5);c.lineWidth=1;c.beginPath();c.moveTo(p.x,p.y);c.lineTo(fl.x,fl.y);c.stroke();
c.fillStyle=colCss(k,1);c.shadowColor=colCss(k,1);c.shadowBlur=12;
c.beginPath();c.arc(p.x,p.y,6,0,Math.PI*2);c.fill();c.shadowBlur=0;
c.fillStyle='#0d0d0d';c.beginPath();c.arc(p.x,p.y,2.2,0,Math.PI*2);c.fill();
}
}
function drawPad(){
pctx.clearRect(0,0,140,140);pctx.fillStyle='#1c1c1c';pctx.fillRect(0,0,140,140);
pctx.strokeStyle='rgba(255,255,255,0.06)';pctx.lineWidth=1;
for(let i=1;i<4;i++){const t=i/4*140;pctx.beginPath();pctx.moveTo(t,0);pctx.lineTo(t,140);pctx.stroke();
pctx.beginPath();pctx.moveTo(0,t);pctx.lineTo(140,t);pctx.stroke();}
// feedback examples (👍 +, 👎 −) — same convention as the main app
const TAU=Math.PI*2;
for(const m of marks){const mx=m.x*140,my=(1-m.y)*140,col=m.sign>0?'#ff6a00':'#00ccff';
pctx.globalAlpha=0.28;pctx.fillStyle=col;pctx.beginPath();pctx.arc(mx,my,8,0,TAU);pctx.fill();
pctx.globalAlpha=1;pctx.strokeStyle=col;pctx.lineWidth=1.5;pctx.beginPath();pctx.arc(mx,my,5,0,TAU);pctx.stroke();
pctx.fillStyle=col;pctx.font='9px '+MONO;pctx.textAlign='center';pctx.textBaseline='middle';
pctx.fillText(m.sign>0?'+':'–',mx,my+0.5);}
const dx=cx*140,dy=(1-cy)*140;
pctx.strokeStyle='rgba(255,255,255,0.1)';pctx.beginPath();pctx.moveTo(0,dy);pctx.lineTo(140,dy);
pctx.moveTo(dx,0);pctx.lineTo(dx,140);pctx.stroke();
pctx.fillStyle='#ff6a00';pctx.shadowColor='#ff6a00';pctx.shadowBlur=12;
pctx.beginPath();pctx.arc(dx,dy,6,0,Math.PI*2);pctx.fill();pctx.shadowBlur=0;
pctx.fillStyle='#0d0d0d';pctx.beginPath();pctx.arc(dx,dy,2.4,0,Math.PI*2);pctx.fill();
}
// readout bars (prose drawer only)
const barsEl=document.getElementById('ndbars');let barI=[],barN=[];
function buildBars(){barsEl.innerHTML='';barI=[];barN=[];
for(let k=0;k<K;k++){const ro=document.createElement('div');ro.className='ro';
ro.innerHTML=`<span style="color:${colCss(k,0.95)}">${NAMES[k]}</span>`+
`<div class="bar"><i style="background:${colCss(k,1)}"></i></div><div class="num">0.50</div>`;
barsEl.appendChild(ro);barI[k]=ro.querySelector('i');barN[k]=ro.querySelector('.num');}}
function updateBars(){if(!barI.length)return;for(let k=0;k<K;k++){const v=clamp(probeOut[k]);
barI[k].style.width=(v*100)+'%';barN[k].textContent=v.toFixed(2);}}
// output faders — the 1-D projection of each layer's value at the current point
const TRACK_H=200;
const outsEl=document.getElementById('outs');let fFill=[],fHan=[],fVal=[];
function buildOuts(){outsEl.innerHTML='';fFill=[];fHan=[];fVal=[];
for(let k=0;k<K;k++){const col=colCss(k,1);const f=document.createElement('div');f.className='fader';
f.innerHTML=`<div class="track"><div class="fill" style="background:${col}"></div>`+
`<div class="handle" style="background:${col};color:${col}"></div></div>`+
`<div class="flab" style="color:${colCss(k,0.95)}" title="${NAMES[k]}">${ABBR[k]}</div><div class="fval">0.50</div>`;
outsEl.appendChild(f);fFill[k]=f.querySelector('.fill');fHan[k]=f.querySelector('.handle');fVal[k]=f.querySelector('.fval');}}
function updateOuts(){if(!fFill.length)return;for(let k=0;k<K;k++){const v=clamp(probeOut[k]);
fFill[k].style.height=(v*100)+'%';fHan[k].style.bottom=(v*TRACK_H)+'px';fVal[k].textContent=v.toFixed(2);}}
function redraw(){renderScene();drawPad();updateBars();updateOuts();}
// ===== interaction =====
function setK(n){K=clamp(n,1,MAXK);document.getElementById('kval').textContent=K;
document.getElementById('kup').disabled=K>=MAXK;document.getElementById('kdown').disabled=K<=1;
computeView();buildBars();buildOuts();updateProbe&&(probeOut=new Float32Array(E.inferXY(cx,cy)));redraw();}
function stopAuto(){auto=false;document.getElementById('hint').classList.add('gone');}
// pad → control position
function setFromPad(e){const r=pad.getBoundingClientRect();
cx=clamp((e.clientX-r.left)/r.width);cy=clamp(1-(e.clientY-r.top)/r.height);}
let padDrag=false;
pad.addEventListener('pointerdown',e=>{padDrag=true;pad.setPointerCapture(e.pointerId);stopAuto();setFromPad(e);updateProbe();redraw();});
pad.addEventListener('pointermove',e=>{if(padDrag){setFromPad(e);updateProbe();redraw();}});
pad.addEventListener('pointerup',()=>padDrag=false);
// scene → orbit (pitch inverted; pitch up to π/2 = exact top-down)
scene.addEventListener('pointerdown',e=>{dragRot=true;scene.setPointerCapture(e.pointerId);stopAuto();lastX=e.clientX;lastY=e.clientY;});
scene.addEventListener('pointermove',e=>{if(!dragRot)return;
yaw+=(e.clientX-lastX)*0.01;
pitch=clamp(pitch+(e.clientY-lastY)*0.008,0.06,Math.PI/2); // vertical (Y) drag — inverted
lastX=e.clientX;lastY=e.clientY;redraw();});
scene.addEventListener('pointerup',()=>dragRot=false);
scene.addEventListener('pointercancel',()=>dragRot=false);
// buttons
const B=(id,fn)=>document.getElementById(id).addEventListener('click',()=>{stopAuto();fn();});
B('aRand',()=>act('rand'));B('aNudge',()=>act('nudge'));B('aUndo',()=>act('undo'));
B('aUp',()=>act('up'));B('aDown',()=>act('down'));
B('kup',()=>setK(K+1));B('kdown',()=>setK(K-1));
document.getElementById('mPush').addEventListener('click',()=>{stopAuto();setMode('pushaway');});
document.getElementById('mExp').addEventListener('click',()=>{stopAuto();setMode('explore');});
document.getElementById('infoBtn').addEventListener('click',()=>{
document.body.classList.toggle('minimal');
document.getElementById('infoBtn').textContent=document.body.classList.contains('minimal')?'ⓘ':'✕';});
window.addEventListener('resize',()=>{resize();redraw();});
function loop(){
if(auto&&ready&&!padDrag&&!dragRot){
autoT+=0.006;cx=0.5+0.42*Math.sin(autoT*0.9);cy=0.5+0.42*Math.sin(autoT*1.27+1.3);
updateProbe();redraw();
}
requestAnimationFrame(loop);
}
// ===== boot =====
(async()=>{
try{
await E.init();
E.draw(BOOT_SPREAD); // a fresh net is near-flat; draw for visible relief
window.__learn={E,setK,resample,redraw,
range(k){let lo=1,hi=0;for(let j=0;j<=N;j++)for(let i=0;i<=N;i++){const v=H(k,i,j);if(v<lo)lo=v;if(v>hi)hi=v;}return[lo,hi];}};
resize();setK(1);resample();updateProbe();reflectButtons();
ready=true;document.getElementById('loading').classList.add('gone');
redraw();requestAnimationFrame(loop);
// fade the hint after a while
setTimeout(()=>{if(auto)document.getElementById('hint').classList.add('gone');},6000);
}catch(err){
console.error('[learn] engine init failed:',err);
document.getElementById('loading').innerHTML='<span style="color:#c8553a">engine failed to load — see console</span>';
}
})();
})();
</script>
</body>
</html>

3
firmware/MEMLNaut-NISPS/MEMLNaut-NISPS.ino
View file

@ -31,6 +31,7 @@
#include "glue/midi_io.hpp" #include "glue/midi_io.hpp"
#include "glue/mode_select.hpp" #include "glue/mode_select.hpp"
#include "glue/output_router.hpp" #include "glue/output_router.hpp"
#include "glue/settings_view.hpp"
// ---- Mode selection ---- // ---- Mode selection ----
// Build script rewrites which line is uncommented. // Build script rewrites which line is uncommented.
@ -142,6 +143,8 @@ void setup() {
} }
MEMLNaut::Instance()->addSystemInfoView(); MEMLNaut::Instance()->addSystemInfoView();
// Settings menu (e.g. Joystick: Dual/Single for the 4-input modes).
nisps_firmware::wire_settings(g_mode);
Serial.println("Finished initialising core 0."); Serial.println("Finished initialising core 0.");
} }

64
firmware/MEMLNaut-NISPS/glue/settings_view.hpp Normal file
View file

@ -0,0 +1,64 @@
// glue/settings_view.hpp — on-device settings menu views (TFT + rotary encoder).
//
// Adds entries to the MEMLNaut display carousel (the same DisplayDriver the
// SystemView and SelfTest use). Navigation: rotate to move between views;
// press to focus a view; rotate-while-focused to change its value; press
// again to unfocus. See src/memllib/.../display/DisplayDriver.hpp.
//
// Currently provides:
// * Joystick: Dual / Single — for the 4-input ("two 2-D joystick") modes,
// toggles whether the SECOND joystick (input channels 2,3) feeds the ML
// engine. "Single" pins those channels to the neutral value (0.5) via the
// platform-agnostic ModeBase pin mechanism, so the network is never
// rebuilt and trained state survives toggling. Default is Dual (the 4-D
// variant naturally uses both joysticks).
//
// Only registered for modes whose input arity is exactly 4 (the two-joystick
// modes); other modes (e.g. SoundAnalysisMIDI, whose joystick channels are not
// 2,3) get no joystick toggle.
#pragma once
#include <array>
#include <cstddef>
#include <memory>
#include <span>
#include "../src/memllib/hardware/memlnaut/MEMLNaut.hpp"
#include "../src/memllib/hardware/memlnaut/display/SingleSelectView.hpp"
namespace nisps_firmware {
// The second 2-D joystick maps to ML input channels 2 and 3 (joy_z / joy_w).
inline constexpr std::size_t kSecondJoystickCh0 = 2u;
inline constexpr std::size_t kSecondJoystickCh1 = 3u;
// Add the settings views for `mode` to the display carousel. Must run AFTER
// MEMLNaut::Initialize() (so the display + TFT exist) and after
// addSystemInfoView(), on core 0.
template <typename ModeT>
inline void wire_settings(ModeT& mode) {
if constexpr (ModeT::input_channel_count() == 4u) {
MEMLNaut* meml = MEMLNaut::Instance();
if (!meml || !meml->disp) return;
// Kept alive for the program lifetime (the carousel holds a shared_ptr
// too, but this guarantees ownership regardless of view churn).
static std::shared_ptr<SingleSelectView> joystick_view;
joystick_view = std::make_shared<SingleSelectView>("Joystick");
meml->disp->AddView(joystick_view); // calls Setup() → creates the selector
// Index 0 = Dual (default, no pinning); index 1 = Single.
std::array<String, 2u> options = { String("Dual"), String("Single") };
joystick_view->setOptions(std::span<String>(options.data(), options.size()));
joystick_view->setNewVoiceCallback([&mode](std::size_t idx) {
const bool single = (idx == 1u);
mode.set_input_pinned(kSecondJoystickCh0, single);
mode.set_input_pinned(kSecondJoystickCh1, single);
});
} else {
(void)mode;
}
}
} // namespace nisps_firmware

265
manifold/ONBOARDING.md Normal file
View file

@ -0,0 +1,265 @@
# Manifold UI — Agent Onboarding
> **Purpose of this doc:** everything an agent needs to make a tweak or fix to the Manifold
> front-end without re-grepping the tree. Read this + the relevant section's source files; you
> should not need to read the 40KB design specs for routine work (they're pointers at the end).
>
> **Voice:** ground-truth map. If something here contradicts the code, the code wins — fix this doc
> in the same commit.
---
## 1. What Manifold is
A **convertible-mode React front-end** on the real, parity-tested NISPS ML+audio engine (same
`nisps/` C++ core that the firmware and the `playground/` use, compiled to WASM). It lets you drive
a neural net's mapping from a small input space (XY pad / gamepad / MIDI) to many outputs, shape
that mapping with interactive ML feedback ("explore & place" / dislike), and route the outputs to
several backends (built-in synth, MIDI, OSC, particles, VCV, MEMLNaut hardware).
- **Stack:** Vite + React 18 + TypeScript (strict). No CSS-in-JS — design-token CSS variables.
- **Lives at:** `manifold/` (beside `playground/`). Deploys to **https://meml.lnfinitemonkeys.org/next/**.
- **Synth is labelled "Powerful Synth Engine" / "Built-in Synth" — NEVER "C15".** British spelling.
---
## 2. Run / build / deploy / test
```bash
cd manifold
bun install
bun run dev # Vite dev server, port 5273 (COOP/COEP headers set)
bun run typecheck # tsc --noEmit ← run before every commit
bun run build # tsc --noEmit && vite build → dist/
bun run preview # serve dist/ on :4273 (honours COOP/COEP — needed by WASM/worklet)
bun run test:e2e # Playwright smoke (needs `bun run build` first; runs against preview)
```
- **Typecheck is the cheap gate.** It catches most regressions; run it after any edit.
- **E2E on the VPS** needs a non-snap node runner (bun is snap-confined and hides libs from
Chromium): `PLAYWRIGHT_BROWSERS_PATH=/home/w1n5t0n/snap/bun-js/87/.cache/ms-playwright node node_modules/.bin/playwright test`. Preview via bun is fine. The smoke spec (`tests/e2e/smoke.spec.ts`)
asserts: engine WASM loads, spine invariant (setInputs → outputs change), feedback runs, console
renders, **no "C15" in the bundle**, no console errors. `shot.spec.ts` takes screenshots.
- **Deploy is automatic on push to GitHub `main`** → webhook → builds `manifold/` → rsyncs to the
live `/next/` subdir. See the `manifold-deploy-pipeline` memory for the full chain and gotchas
(the `cp index.html a-immersive.html` 403 workaround; git-ignored `bun.lock`).
- **`?debug=1`** installs `window.__nisps` (synchronous probe for Playwright/console — see
`src/debug/probe.ts`). **`base: './'`** in `vite.config.ts` keeps asset URLs relative so one
`dist/` mounts at both `/` and `/next/`. **WASM URLs must resolve via `document.baseURI`**, never
hardcoded (see the assetUrl gotcha in §6).
---
## 3. The big picture — three layers
```
┌─────────────────────────────────────────────────────────────────┐
│ UI layer (React) src/console/ src/dock/ src/primitives/
│ ConsoleApp = spine of the UI: holds state, picks a Stage, │
│ renders the Dock. "Convertible" = swappable Stages. │
└───────────────┬─────────────────────────────────────────────────-┘
│ reads engine.version (useSyncExternalStore), reads buffers imperatively
│ writes via engine.setInput / setParam / feedback.*
┌───────────────▼─────────────────────────────────────────────────┐
│ Engine layer (NO React) src/engine/ src/inputs/ src/feedback/ src/backends/
│ Spine = reactive store below React. Per-frame ML inference is │
│ eager + synchronous, OFF the render cycle. React only watches │
│ a monotonic version counter. │
└───────────────┬─────────────────────────────────────────────────┘
│ C ABI
┌───────────────▼─────────────────────────────────────────────────┐
│ WASM (nisps.wasm) — built from the C++ nisps/ core │
│ Loaded TWICE: main thread (ML inference + training + feedback) │
│ and AudioWorklet (audio DSP, separate instance). │
└──────────────────────────────────────────────────────────────────┘
```
**Golden rule of the reactivity model:** components subscribe to `engine.version()` via
`useEngineVersion()` and then read live `Float32Array`s **imperatively** (`engine.getOutputs()`,
`engine.routedOutput()`). They do **not** get outputs as React state — that was a deliberate perf
decision. Buffers are reused frame-to-frame; never assume a fresh array.
---
## 4. UI layer (where most tweaks happen)
### Entry & composition
- `src/App.tsx` → mounts `<EngineProvider>` (async WASM load, shows `Loading`) → `<ConsoleApp focus="composite" />`. Installs the `?debug=1` probe once the engine is live.
- `src/console/ConsoleApp.tsx` (**~1040 lines — the UI spine**). Holds nearly all UI state:
`focus`, `modeId`, `params` (`MFParam[]`), `pos` (2D input), feedback state
(`feedbackMode`/`soloMode`/`exploring`/`picking`/`anchorCount`), output config
(`outputMode`/`midiOutputId`/`oscUrl`/`vcvUrl`), and UI flags (`sandwich`, `split`, `stripPinned`,
`snapshots`, `markers`, `health`, `rev`). Builds the flat `ConsoleCtx` passed to the Dock + drawers.
### The "convertible" Stages (one renders at a time, chosen by `focus` + `outputMode`)
| Stage | File | Renders when | What it is |
|---|---|---|---|
| Manifold | `Manifold.tsx` | `focus==='in'` (default input view) | Full-bleed 2D input surface; canvas trail + pins + feedback markers; pointer → `onMove`. |
| OutputStage | `OutputStage.tsx` | `focus==='out'` | Full-bleed output columns; drag bars set value; `InputMini` docked in a corner. |
| SplitStage | `SplitStage.tsx` | `focus==='split'` | Manifold left, OutputStage right, equal width. |
| CompositeStage | `CompositeStage.tsx` | `focus==='composite'` (**app default / hero**) | Draggable split-ratio; magnet-snaps to 0.14/0.33/0.5/0.66/0.86; collapses a side to a corner minimap at extremes. |
| SandwichStage | `SandwichStage.tsx` | `sandwich===true` (overrides) | 3D parameter-landscape view (input → MLP heatmap grid → outputs); drag to orbit. |
| ParticleStage | `ParticleStage.tsx` | `outputMode==='particles'` | Flow-field visualiser (`flow-field.ts`, 400-particle Canvas2D port) + macro-axis bar + corner joystick. |
- **Output modes** (the TOP dock selector, NOT the same axis as `focus`): `src/console/output-mode.ts`
defines `OUTPUT_MODES` = **particles** (default) / midi / osc / synth / editor, each mapping to a
`BackendId`. `DEFAULT_OUTPUT_MODE='particles'`.
- `src/console/output-mode.ts`, `types.ts`, `model.ts` are the shared vocabulary — read these first
when touching anything cross-cutting:
- `types.ts`: `Focus`, `OutputMode`, `DrawerKey`, `DrawerDepth`, `FeedbackModeUI`, `SoloMode`,
`Pin`, `FeedbackMarker`, `Snapshot`, and **`ConsoleCtx`** (the flat context handed to the dock).
- `model.ts`: the static instrument catalogue `MF_MODES`, `MFParam`, `ParamStatus`
(`off|fixed|live`), `ParamGroup`, plus `shapeValues()` (applies min/max/curve to raw engine
outputs) and `seededGradient()`.
### The Dock (right-edge rail) — `src/console/Dock.tsx` + `Drawers.tsx`
- 48px right rail: **TOP** = mode selector (the 5 output modes, popover); **MIDDLE** = 5 drawer
icons, vertically centred macOS-dock style; **BOTTOM** = sandwich toggle.
- Five drawers (`DRAWERS` in `Drawers.tsx`, each has `.render(ctx, depth)` — condensed 360px panel
vs expanded 80vw×80vh modal):
- **learn** — feedback mode (explore-and-place / geometric-dislike) + solo mode + per-output arm.
- **inputs** — enable/configure input sources (XY pad / MIDI / gamepad).
- **route** (label "Outputs") — per-output control matrix + per-backend config.
- **settings** — icon style (monochrome/colour), input-map shape (xy/joystick/rect/circular), corner radius.
- **help** — keymap pills + loop explanation.
- `src/dock/` holds the output-routing internals used by the `route` drawer:
- `output-state.ts` — the per-output control model: `OutputControl` (state/muted/armed/min/max/
curve/fixedValue + backend specs `MidiCcSpec`/`OscSpec`/`VcvSpec`), `toOutputControl()`,
`buildArmMask()` (solo focus).
- `OutputControlRow.tsx` — one output row: name · M(mute) · S(solo/arm) · off|fixed|live · dual-range
· curve pad · live value. **Writes eagerly to the shared `MFParam` store via `onChange`.**
- `OutputsBackendConfig.tsx` — preset bar (save/restore/rename/delete) + per-backend config (MIDI
CC#/channel, OSC path/range, VCV polarity).
- `BackendAdvanced.tsx` — full-depth modal version of the same editors.
### Primitives — `src/primitives/` (barrel: `index.ts`)
`Button`, `Slider`, `PillToggle`, `Panel`, `Badge`, `Switch`, `StatusLine`, `XYPad`,
`VirtualJoystick`, `ControlAxis`, `CurvePlot`, `Sparkline`. Dumb, reusable, no engine knowledge.
Side-effect import of `styles/primitives.css` styles the range inputs.
### Other shared UI files
- `shared-ui.tsx` — `AltitudeNav` (focus IN/DUAL/OUT/FLEX switcher) + `MiniMeters` (read-only output bars).
- `icons.tsx` — monochrome inline-SVG icons (mode icons + drawer icons + `GLYPH_FALLBACK` for when monochrome is off).
- `ReadoutStrip.tsx` — thin horizontal heatmap strip (pinned, `focus==='in'`); same per-output control as OutputStage.
- `VerdictCluster.tsx` — floating bottom-centre feedback UI (perturb ▽ / undo ↺ / commit △ + A/B); labels adapt to feedback mode.
- `CurvePad.tsx` — square canvas curve editor (vertical drag reshapes [0,1]; ~0.43 ≈ linear). Used in OutputEditor + OutputControlRow.
- `InputMini.tsx` — compact XYPad/joystick docked in a corner when input is demoted.
- `OutputEditor.tsx` — inline min/max/curve popup for a single output (hover/click on a bar).
### Styling — `src/styles/`
CSS-variable design tokens, no CSS-in-JS. `tokens.css` `@import`s `tokens/{base,colors,fonts,
typography,spacing,effects}.css`. All vars on `:root`, consumed via `var(--name)`. Dark theme: deep
black bg, warm-orange `--accent` (#ff6a00), cyan `--accent-2` (#00ccff). Corner radius is driven by
a setting → `--r-*` tokens.
---
## 5. Engine / inputs / feedback / backends (touch when behaviour, not chrome, changes)
### Engine — `src/engine/` (no React except the two binding files)
- `spine.ts` — **the reactive store.** `setInput(x,y)`/`setInputs(arr)` drive raw input synchronously
through input-pipeline → `WasmIML.processInto()` → output-pipeline → backend, all off the render
cycle, reusing buffers. Bumps a monotonic `version`. `reprocess()` re-ticks the last input after a
weight change (stores the full N-D vector so extra axes survive).
- `engine-api.ts` — **`EngineApi`, the framework-neutral facade** everything in the UI talks to:
`setInput/setInputs`, `getOutputs/routedOutput`, training (`addExample/train/trainAsync/evalLoss`),
weights (`getWeights/setWeights/process/randomise`), `subscribe/version/on`, plus nested
`.feedback` and `.audio` facades.
- `EngineProvider.tsx` / `useEngine.ts` — the **only** React coupling. `useEngine()` returns the API
(null until WASM ready); `useEngineVersion()` = `useSyncExternalStore(subscribe, version)`.
- `engine-host.ts` — main-thread audio wiring: AudioContext (user-gesture gated), fetch `nisps.wasm`,
register + feed the worklet.
- `wasm-iml.ts` (**~750 lines**) — the ML interface to `nisps.wasm`: one MLP handle, dataset, heap
buffers, feedback C-ABI bindings (`nisps_ml_feedback_*`), lazy training worker.
- `wasm-worker.ts` — off-thread training worker. `worklet/nisps-processor.ts` — the AudioWorklet's
separate WASM instance (raw `WebAssembly.instantiate`, no Emscripten glue; 128-sample blocks).
- `input-pipeline.ts` — per-axis: invert → deadzone → circular clamp → zoom → centred power curve →
EMA smoothing → momentum. **First 2 axes get the full pad pipeline; axes 2+ feed raw to the spine.**
- `output-pipeline.ts` — global power curve → per-output EMA smoothing → slew limit → freeze gate.
- `dataset.ts` — JS-side example store + sample-weight modes (uniform/recency/spatial/combined).
- `curves.ts` — math primitives. **Must stay lockstep with C++ `nisps/core/math.hpp`** (golden tests
compare WASM vs TS). `sink.ts` — `EngineSink` framework boundary. `types.ts` — C-ABI surface types.
### Inputs — `src/inputs/`
- `input-layer.ts` — composition hub. One rAF loop polls sources, pulls all axes into a vector,
forwards N→engine. **`MAX_AXES = 32`** (WASM net over-provisioned to 32 inputs). **Dedicated
dimensions, NO mean-blending** — each active axis drives its own engine slot 1:1; unused slots
zero-padded (inert). Changing axis count requires a **net reset** (UI confirm modal).
- `base-source.ts` + sources: `xy-pad-source.ts` (push, 2 axes), `gamepad-source.ts` (single=2 /
double=4 axes, deadzone 0.08), `midi-input-source.ts` (Web MIDI, batch CC-learn, multi-port).
- `useInputLayer.ts` — React binding; manages exclusive input mode + gamepad stick mode + MIDI
device/learn map; exposes `pushPad`, `sources`, `channelLayout`, etc.
- **Reshape status:** the WASM head is over-provisioned to 32 but the spine's `setInputs` historically
treated the head as effectively 2-D — confirm current behaviour in `spine.ts`/`input-layer.ts`
before relying on >2 active dims. See the `manifold-mixed-inputs` memory for the locked design
(reshapeable net, reset-on-reshape modal, adaptive slider viz when >2 dims).
### Feedback — `src/feedback/`
- `controller.ts` (**~490 lines**) — `FeedbackController`, framework-neutral, owned by ConsoleApp.
Two modes: **explore-and-place** (default, Mode 2, positive-only — drives the C++ core's
snapshot/scratchpad/undo lifecycle; caller accumulates anchors and trains on finalise with
warm-start) and **geometric-dislike** (Mode 1, selectable). Solo/arm via per-output mask.
- `rng.ts` — `SeededRng` (deterministic xorshift32 + gaussian). **Stand-in** until the C++ nudge owns
the stream — not bit-identical to `nisps::Rng`.
- **`--- C++ GAP ---` markers** flag behaviour approximated in TS pending C++ port: true geometric
push (firmware k-NN), feedback nudge → `nisps_ml_feedback_nudge`, loss-history plumbing. Grep for
`C++ GAP` before changing feedback maths.
### Backends — `src/backends/`
- `manager.ts` — `BackendManager`, the **single consumer of the engine spine** for output: subscribes,
reads `routedOutput()` each version bump, calls `active.send(routed)`. Gates audio (mutes synth when
a non-synth backend is active).
- Registered: `midi` (`midi-backend.ts`, CC out, ~20Hz throttle), `osc` (`osc-client.ts` → WebSocket
to the Deno bridge in `osc-bridge/bridge.ts`), `synth`/`cvgate` (`passthrough-backend.ts`),
`particles` (`particle-backend.ts`, no-op — the visualiser is a separate `flow-field.ts` consumer),
`vcv` (`vcv-backend.ts`, bidirectional). `mapping.ts`, `presets.ts`, `useBackendManager.ts` support
config. `backend.ts` defines the `OutputBackend` interface + `OutputMapping`.
### Misc
- `src/serial/memlnaut-serial.ts` — **STUB** Web Serial scaffold for the MEMLNaut Editor mode (protocol TODO). `EditorPanel.tsx` is its UI.
- `src/settings/settings-store.ts` — localStorage settings (`mf-settings`): icon style, input-map shape, corner radius.
- `src/midi-devices/` — codegen'd external-synth device templates.
- `src/debug/probe.ts` — `window.__nisps` synchronous probe (engine/audio/bus). Some playground
feature-store methods are present-but-inert (not ported yet) to keep the surface stable.
---
## 6. Gotchas & non-obvious rules
1. **Outputs are read imperatively, not via React state.** Subscribe to `version`, then call
`getOutputs()`/`routedOutput()`. Buffers are reused — copy if you need to retain.
2. **Asset URLs must resolve against `document.baseURI`, not `location.origin`.** `base: './'` +
the `/next/` subpath mean a hardcoded `/nisps.wasm` 404s. The `assetUrl()` helper (in
`engine-host.ts`, `wasm-iml.ts`, `wasm-worker.ts`) handles this — use it.
3. **`nisps.js` is non-module Emscripten glue (no ES exports).** Workers/worklet fetch + indirect-eval
to install the global `createNispsModule`; the worklet uses raw `WebAssembly.instantiate`.
4. **WASM architecture is hardwired at build time** (`nisps/wasm/bindings.cpp`): 2→126 outputs,
fixed hidden layers, 32 input slots. Requesting other sizes in options is a logged warning, not
dynamic. Changing it means rebuilding WASM (`bash scripts/build-wasm.sh`) and updating the parity
test (`tests/cpp/parity_check.cpp`) or parity CI goes red.
5. **`curves.ts` ↔ `nisps/core/math.hpp` must stay lockstep** (golden-vector parity tests).
6. **COOP/COEP headers are mandatory** for the WASM/worklet path — set in `vite.config.ts` for
dev+preview, and at nginx server scope in prod (inherited by `/next/`).
7. **Never emit "C15"** in code or bundle — the smoke test fails on it. Synth is "Powerful Synth
Engine"/"Built-in Synth".
8. **Two `MFParam` writers** (stage controls AND dock rows) share one store — edits in either re-render both.
---
## 7. Deeper references (only when this doc isn't enough)
All in `docs/redesign/` (at the repo root, not under `manifold/`):
- `BUILD-PLAN.md` — locked decisions + the 12-step build sequence + spec pointers (the resume anchor).
- `engine-architecture.md` — full engine/spine/WASM design.
- `dock-spec.md` — dock + drawers spec. `inputs-spec.md` — mixed-input design. `backends-spec.md` — backends.
- `rl-feedback-design.md` + `feedback-modes-port-spec.md` + `findings-feedback-behaviour.md` — feedback modes (Mode 1/Mode 2).
- `aimmersive-clone-spec.md` / `playground-2026.md` — the a-immersive feature parity target.
- `src/backends/README.md` — backend wiring notes.
**Memories** (auto-loaded): `manifold-build` (status + locked decisions), `manifold-mixed-inputs`
(N-D input design), `manifold-deploy-pipeline` (the auto-deploy chain), `single-double-joystick-toggle`.
---
*Keep this in sync with the code — update it in the same commit as any change to UI structure, the
engine spine, the dock/drawers, or the build/deploy commands.*

98
manifold/src/console/ConsoleApp.tsx
View file

@ -28,6 +28,7 @@ import { MF_MODES, modeEngineId, seededGradient, shapeValues } from './model';
import type { MFParam } from './model'; import type { MFParam } from './model';
import { CompositeStage } from './CompositeStage'; import { CompositeStage } from './CompositeStage';
import { ParticleStage } from './ParticleStage'; import { ParticleStage } from './ParticleStage';
import { SandwichStage } from './SandwichStage';
import { SplitStage } from './SplitStage'; import { SplitStage } from './SplitStage';
import { OutputStage } from './OutputStage'; import { OutputStage } from './OutputStage';
import { InputMini } from './InputMini'; import { InputMini } from './InputMini';
@ -105,8 +106,10 @@ export function ConsoleApp({ focus: initialFocus = 'composite' }: ConsoleAppProp
const [tame, setTame] = useState(0.85); const [tame, setTame] = useState(0.85);
const [health, setHealth] = useState(0.8); const [health, setHealth] = useState(0.8);
const [rev, setRev] = useState(1); const [rev, setRev] = useState(1);
const [active, setActive] = useState<DrawerKey | null>('learn'); const [active, setActive] = useState<DrawerKey | null>(null);
const [depth, setDepth] = useState<DrawerDepth>('peek'); const [depth, setDepth] = useState<DrawerDepth>('condensed');
// Sandwich (parameter-landscape) centre-stage toggle — dock-bottom layers icon.
const [sandwich, setSandwich] = useState(false);
// Learning-behaviour store (dock-spec §1; rl-feedback-design). Default // Learning-behaviour store (dock-spec §1; rl-feedback-design). Default
// feedback mode = "Explore and place"; default solo = "Mask gradients". // feedback mode = "Explore and place"; default solo = "Mask gradients".
@ -219,8 +222,8 @@ export function ConsoleApp({ focus: initialFocus = 'composite' }: ConsoleAppProp
setFollow(false); setFollow(false);
setPins([]); setPins([]);
setMarkers([]); setMarkers([]);
setActive('learn'); setActive(null);
setDepth('peek'); setDepth('condensed');
if (engine) engine.setInput(0.5, 0.5); if (engine) engine.setInput(0.5, 0.5);
// eslint-disable-next-line react-hooks/exhaustive-deps // eslint-disable-next-line react-hooks/exhaustive-deps
}, [modeId]); }, [modeId]);
@ -408,6 +411,23 @@ export function ConsoleApp({ focus: initialFocus = 'composite' }: ConsoleAppProp
setRev((r) => r + 1); setRev((r) => r + 1);
}; };
/**
* Nudge — a small bounded weight perturbation of the current net (the left
* half of the secondary pill). Routes to the scratchpad while exploring,
* otherwise nudges the real net directly.
*/
const nudgeNet = () => {
const c = controllerRef.current;
if (feedbackMode === 'explore-and-place' && c?.getState().exploring) {
c.nudge();
} else {
engine?.feedback.nudge(0.05);
engine?.process();
}
syncController();
setRev((r) => r + 1);
};
/** /**
* Undo. While exploring (Mode 2) this pops the scratchpad undo ring (reroll / * Undo. While exploring (Mode 2) this pops the scratchpad undo ring (reroll /
* nudge). Otherwise it falls back to the UI snapshot stack (visual A/B seed). * nudge). Otherwise it falls back to the UI snapshot stack (visual A/B seed).
@ -523,8 +543,8 @@ export function ConsoleApp({ focus: initialFocus = 'composite' }: ConsoleAppProp
}; };
if (map[e.key]) { if (map[e.key]) {
setActive((a) => (a === map[e.key] ? null : map[e.key])); setActive((a) => (a === map[e.key] ? null : map[e.key]));
setDepth('peek'); setDepth('condensed');
} else if (e.key === '\\') setDepth((d) => (d === 'full' ? 'peek' : 'full')); } else if (e.key === '\\') setDepth((d) => (d === 'expanded' ? 'condensed' : 'expanded'));
else if (focus === 'composite' && e.key === '[') { else if (focus === 'composite' && e.key === '[') {
e.preventDefault(); e.preventDefault();
setSplit((s) => Math.max(0, s - 0.04)); setSplit((s) => Math.max(0, s - 0.04));
@ -771,7 +791,55 @@ export function ConsoleApp({ focus: initialFocus = 'composite' }: ConsoleAppProp
bottom: 0, bottom: 0,
}} }}
> >
{outputMode === 'particles' ? ( {sandwich ? (
// Sandwich centre-stage: shrunken input (left) · landscape stack
// (centre, fills) · compact outputs (right). Replaces the Mode stage.
<div style={{ position: 'absolute', inset: 0, display: 'flex' }}>
<div
style={{
width: 'clamp(200px, 24%, 320px)',
flex: '0 0 auto',
position: 'relative',
borderRight: '1px solid var(--line)',
}}
>
<Manifold
pos={pos}
onMove={onMove}
noiseCap={noiseCap}
pins={pins}
markers={markers}
variant={inputMapVariant}
frozen={false}
follow={follow}
onLongPress={addPin}
picking={picking}
onPickLocation={onPickLocation}
/>
</div>
<div style={{ flex: 1, minWidth: 0, position: 'relative' }}>
{engine && (
<SandwichStage
engine={engine}
version={version}
pos={pos}
layerCount={Math.min(params.length, 8)}
names={params.map((p) => p.name)}
/>
)}
</div>
<div
style={{
width: 'clamp(200px, 24%, 320px)',
flex: '0 0 auto',
position: 'relative',
borderLeft: '1px solid var(--line)',
}}
>
<OutputStage params={params} values={values} onChange={setParam} compact />
</div>
</div>
) : outputMode === 'particles' ? (
<ParticleStage pos={pos} onMove={onMove} /> <ParticleStage pos={pos} onMove={onMove} />
) : focus === 'composite' ? ( ) : focus === 'composite' ? (
<CompositeStage <CompositeStage
@ -852,6 +920,8 @@ export function ConsoleApp({ focus: initialFocus = 'composite' }: ConsoleAppProp
onUndo={undo} onUndo={undo}
onCommit={commit} onCommit={commit}
onReroll={reroll} onReroll={reroll}
onNudge={nudgeNet}
onRandomise={reroll}
canUndo={feedbackMode === 'explore-and-place' && exploring ? undoDepth > 0 : snapshots.length > 0} canUndo={feedbackMode === 'explore-and-place' && exploring ? undoDepth > 0 : snapshots.length > 0}
ab={ab} ab={ab}
onToggleAB={toggleAB} onToggleAB={toggleAB}
@ -951,7 +1021,19 @@ export function ConsoleApp({ focus: initialFocus = 'composite' }: ConsoleAppProp
</div> </div>
</div> </div>
<Dock ctx={ctx} active={active} setActive={setActive} depth={depth} setDepth={setDepth} /> <Dock
ctx={ctx}
active={active}
setActive={setActive}
depth={depth}
setDepth={setDepth}
sandwich={sandwich}
setSandwich={(v) => {
setSandwich(v);
// reveal the full 3-zone layout: a drawer would overlay the output zone
if (v) setActive(null);
}}
/>
</div> </div>
); );
} }

154
manifold/src/console/Dock.tsx
View file

@ -1,6 +1,8 @@
/** /**
* The Console dock — right-edge 48px rail + one mutually-exclusive drawer with * The Console dock — right-edge 48px rail + one mutually-exclusive drawer with
* three depth states (Peek 320 / Expand 520 / Full modal). * two depth states: condensed (360px side panel) and expanded (centred modal,
* ~80% of the viewport). A vertically-centred tab on the panel's left edge
* toggles between them; drawers are closed by default.
* *
* RESTRUCTURED (operator dock restructure): * RESTRUCTURED (operator dock restructure):
* - TOP, pinned: the **Mode** selector — the active OUTPUT BACKEND/target * - TOP, pinned: the **Mode** selector — the active OUTPUT BACKEND/target
@ -22,12 +24,12 @@ import type { ConsoleCtx, DrawerDepth, DrawerKey, OutputMode } from './types';
import { OUTPUT_MODES } from './output-mode'; import { OUTPUT_MODES } from './output-mode';
import { useSettings, unfocusedIconCss } from '../settings/settings-store'; import { useSettings, unfocusedIconCss } from '../settings/settings-store';
import { import {
ModeIcon,
ParticleIcon, ParticleIcon,
MidiIcon, MidiIcon,
OscIcon, OscIcon,
SynthIcon, SynthIcon,
EditorIcon, EditorIcon,
SandwichIcon,
CloseIcon, CloseIcon,
ExpandIcon, ExpandIcon,
GLYPH_FALLBACK, GLYPH_FALLBACK,
@ -79,17 +81,20 @@ function ModeSelector({
width: 40, width: 40,
height: 40, height: 40,
borderRadius: 'var(--r-2)', borderRadius: 'var(--r-2)',
border: '1px solid var(--accent)', border: '1px solid #ffffff',
background: 'rgba(255,106,0,0.12)', background: 'rgba(255,106,0,0.12)',
color: 'var(--accent)', color: 'var(--accent)',
cursor: 'pointer', cursor: 'pointer',
fontSize: 18, fontFamily: 'var(--font-mono)',
fontSize: 22,
fontWeight: 700,
lineHeight: 1,
display: 'flex', display: 'flex',
alignItems: 'center', alignItems: 'center',
justifyContent: 'center', justifyContent: 'center',
}} }}
> >
{mono ? <ModeIcon /> : '⊞'} M
</button> </button>
{open && ( {open && (
<div <div
@ -173,11 +178,14 @@ export interface DockProps {
setActive: (k: DrawerKey | null) => void; setActive: (k: DrawerKey | null) => void;
depth: DrawerDepth; depth: DrawerDepth;
setDepth: (d: DrawerDepth) => void; setDepth: (d: DrawerDepth) => void;
/** Sandwich (parameter-landscape) centre-stage toggle. */
sandwich: boolean;
setSandwich: (v: boolean) => void;
} }
const ORDER: DrawerKey[] = ['learn', 'inputs', 'route', 'settings', 'help']; const ORDER: DrawerKey[] = ['learn', 'inputs', 'route', 'settings', 'help'];
export function Dock({ ctx, active, setActive, depth, setDepth }: DockProps) { export function Dock({ ctx, active, setActive, depth, setDepth, sandwich, setSandwich }: DockProps) {
const { settings } = useSettings(); const { settings } = useSettings();
const mono = settings.monochromeIcons; const mono = settings.monochromeIcons;
const restColour = unfocusedIconCss(settings.unfocusedIconColour); const restColour = unfocusedIconCss(settings.unfocusedIconColour);
@ -192,7 +200,7 @@ export function Dock({ ctx, active, setActive, depth, setDepth }: DockProps) {
title={s.label} title={s.label}
onClick={() => { onClick={() => {
setActive(on ? null : key); setActive(on ? null : key);
setDepth('peek'); setDepth('condensed');
}} }}
style={{ style={{
position: 'relative', position: 'relative',
@ -230,48 +238,105 @@ export function Dock({ ctx, active, setActive, depth, setDepth }: DockProps) {
); );
}; };
const width = depth === 'expand' ? 520 : 320; const expanded = depth === 'expanded';
const full = depth === 'full';
const section = active ? DRAWERS[active] : null; const section = active ? DRAWERS[active] : null;
return ( return (
<> <>
{section && ( {/* Dim backdrop behind the centred modal; click to collapse back. */}
<aside {section && expanded && (
style={ <div
full onClick={() => setDepth('condensed')}
? { style={{
position: 'fixed', position: 'fixed',
inset: 0, inset: 0,
zIndex: 85,
background: 'rgba(0,0,0,0.45)',
backdropFilter: 'blur(2px)',
WebkitBackdropFilter: 'blur(2px)',
}}
/>
)}
{section && (
// Shell: positions/sizes the drawer and is the anchor for the left-edge
// tab. overflow:visible so the tab (at left:-22) is NOT clipped; the
// inner wrapper handles scrolling.
<aside
style={
expanded
? {
position: 'fixed',
top: '50%',
left: '50%',
transform: 'translate(-50%, -50%)',
width: '80vw',
height: '80vh',
zIndex: 90, zIndex: 90,
background: 'var(--glass)', background: 'var(--glass)',
backdropFilter: 'blur(16px)', backdropFilter: 'blur(16px)',
WebkitBackdropFilter: 'blur(16px)', WebkitBackdropFilter: 'blur(16px)',
padding: 'var(--sp-6)', border: '1px solid var(--glass-line)',
display: 'flex', borderRadius: 'var(--r-3)',
flexDirection: 'column', boxShadow: 'var(--shadow-2)',
gap: 'var(--sp-3)', overflow: 'visible',
} }
: { : {
position: 'absolute', position: 'absolute',
top: 0, top: 0,
right: 48, right: 48,
bottom: 0, bottom: 0,
width, width: 360,
zIndex: 35, zIndex: 35,
background: 'var(--glass)', background: 'var(--glass)',
backdropFilter: 'blur(16px)', backdropFilter: 'blur(16px)',
WebkitBackdropFilter: 'blur(16px)', WebkitBackdropFilter: 'blur(16px)',
borderLeft: '1px solid var(--glass-line)', borderLeft: '1px solid var(--glass-line)',
boxShadow: '-8px 0 24px rgba(0,0,0,0.4)', boxShadow: '-8px 0 24px rgba(0,0,0,0.4)',
padding: 'var(--sp-4)', overflow: 'visible',
display: 'flex',
flexDirection: 'column',
gap: 'var(--sp-2)',
overflow: 'auto',
animation: 'mfDrawerIn var(--dur-med) var(--ease-console)', animation: 'mfDrawerIn var(--dur-med) var(--ease-console)',
} }
} }
>
{/* Expand/collapse tab — vertically centred on the panel's left edge. */}
<button
type="button"
title={expanded ? 'Condense' : 'Expand'}
onClick={() => setDepth(expanded ? 'condensed' : 'expanded')}
style={{
position: 'absolute',
left: -22,
top: '50%',
transform: 'translateY(-50%)',
width: 22,
height: 56,
borderRadius: 'var(--r-2) 0 0 var(--r-2)',
border: '1px solid var(--glass-line)',
borderRight: 'none',
background: 'var(--glass)',
backdropFilter: 'blur(14px)',
WebkitBackdropFilter: 'blur(14px)',
color: 'var(--fg-mute)',
cursor: 'pointer',
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
zIndex: 1,
}}
>
<ExpandIcon size={12} />
</button>
{/* Inner scroll wrapper — holds header + content, scrolls independently. */}
<div
style={{
height: '100%',
boxSizing: 'border-box',
overflow: 'auto',
padding: expanded ? 'var(--sp-6)' : 'var(--sp-4)',
display: 'flex',
flexDirection: 'column',
gap: expanded ? 'var(--sp-3)' : 'var(--sp-2)',
borderRadius: 'inherit',
}}
> >
<header <header
style={{ style={{
@ -298,24 +363,6 @@ export function Dock({ ctx, active, setActive, depth, setDepth }: DockProps) {
{depth} {depth}
</span> </span>
<div style={{ marginLeft: 'auto', display: 'flex', gap: 4 }}> <div style={{ marginLeft: 'auto', display: 'flex', gap: 4 }}>
{!full && (
<button
type="button"
title={depth === 'peek' ? 'More' : 'Peek'}
onClick={() => setDepth(depth === 'peek' ? 'expand' : 'peek')}
style={dockMini}
>
<ExpandIcon size={12} />
</button>
)}
<button
type="button"
title="Open full"
onClick={() => setDepth(full ? 'peek' : 'full')}
style={dockMini}
>
<ExpandIcon size={12} />
</button>
<button type="button" title="Close" onClick={() => setActive(null)} style={dockMini}> <button type="button" title="Close" onClick={() => setActive(null)} style={dockMini}>
<CloseIcon size={12} /> <CloseIcon size={12} />
</button> </button>
@ -327,11 +374,11 @@ export function Dock({ ctx, active, setActive, depth, setDepth }: DockProps) {
flexDirection: 'column', flexDirection: 'column',
gap: 'var(--sp-2)', gap: 'var(--sp-2)',
flex: 1, flex: 1,
...(full ? { maxWidth: 720 } : {}),
}} }}
> >
{section.render(ctx, depth)} {section.render(ctx, depth)}
</div> </div>
</div>
</aside> </aside>
)} )}
@ -368,6 +415,27 @@ export function Dock({ ctx, active, setActive, depth, setDepth }: DockProps) {
> >
{ORDER.map(iconBtn)} {ORDER.map(iconBtn)}
</div> </div>
{/* Sandwich toggle — pinned at the BOTTOM of the rail. */}
<button
type="button"
title="Parameter sandwich (layer view)"
onClick={() => setSandwich(!sandwich)}
style={{
width: 40,
height: 40,
borderRadius: 'var(--r-2)',
cursor: 'pointer',
border: `1px solid ${sandwich ? 'var(--accent)' : 'transparent'}`,
background: sandwich ? 'rgba(255,106,0,0.14)' : 'transparent',
color: sandwich ? 'var(--accent)' : mono ? restColour : 'var(--fg-mute)',
display: 'flex',
alignItems: 'center',
justifyContent: 'center',
transition: 'background var(--dur-fast), color var(--dur-fast)',
}}
>
{mono ? <SandwichIcon /> : '≣'}
</button>
</nav> </nav>
</> </>
); );

69
manifold/src/console/Drawers.tsx
View file

@ -1,6 +1,6 @@
/** /**
* Console — the FIVE real dock drawers (operator dock restructure). Each renderer * Console — the FIVE real dock drawers (operator dock restructure). Each renderer
* takes (ctx, depth); what shows is gated by depth (peek | expand | full): * takes (ctx, depth); what shows is gated by depth (condensed | expanded):
* *
* learn — Learning : feedback-mode selector, solo/arm chooser, live params * learn — Learning : feedback-mode selector, solo/arm chooser, live params
* inputs — Inputs : input source * inputs — Inputs : input source
@ -162,13 +162,13 @@ function LearningDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
<SectionLabel>Down action · feedback mode</SectionLabel> <SectionLabel>Down action · feedback mode</SectionLabel>
<Segmented value={ctx.feedbackMode} onChange={ctx.setFeedbackMode} options={FEEDBACK_OPTS} /> <Segmented value={ctx.feedbackMode} onChange={ctx.setFeedbackMode} options={FEEDBACK_OPTS} />
{depth !== 'peek' && ( {depth === 'expanded' && (
<p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}> <p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}>
{FEEDBACK_DESC[ctx.feedbackMode]} {FEEDBACK_DESC[ctx.feedbackMode]}
</p> </p>
)} )}
{depth !== 'peek' && ( {depth === 'expanded' && (
<> <>
<SectionLabel>Solo / arm scope</SectionLabel> <SectionLabel>Solo / arm scope</SectionLabel>
<div style={{ display: 'flex', gap: 6, alignItems: 'center', flexWrap: 'wrap' }}> <div style={{ display: 'flex', gap: 6, alignItems: 'center', flexWrap: 'wrap' }}>
@ -204,7 +204,7 @@ function LearningDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
</> </>
)} )}
{depth === 'full' && ( {depth === 'expanded' && (
<> <>
<SectionLabel>Feedback lab</SectionLabel> <SectionLabel>Feedback lab</SectionLabel>
<p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}> <p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}>
@ -339,21 +339,21 @@ function InputsDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
<SectionLabel>Input source</SectionLabel> <SectionLabel>Input source</SectionLabel>
<Segmented value={inp.inputMode} onChange={inp.setInputMode} options={INPUT_MODE_OPTS} /> <Segmented value={inp.inputMode} onChange={inp.setInputMode} options={INPUT_MODE_OPTS} />
{active && depth !== 'peek' && ( {active && depth === 'expanded' && (
<span style={{ fontSize: 9, color: STATUS_TONE[active.status.state] ?? 'var(--fg-dim)' }}> <span style={{ fontSize: 9, color: STATUS_TONE[active.status.state] ?? 'var(--fg-dim)' }}>
{active.status.message} {active.status.message}
</span> </span>
)} )}
{/* ---- Internal (XY pad / manifold) ---- */} {/* ---- Internal (XY pad / manifold) ---- */}
{inp.inputMode === 'internal' && depth !== 'peek' && ( {inp.inputMode === 'internal' && depth === 'expanded' && (
<p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}> <p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}>
Drag the on-screen manifold / XY pad. Two axes feed the net directly — this is the default. Drag the on-screen manifold / XY pad. Two axes feed the net directly — this is the default.
</p> </p>
)} )}
{/* ---- Game Controller ---- */} {/* ---- Game Controller ---- */}
{inp.inputMode === 'gamepad' && depth !== 'peek' && ( {inp.inputMode === 'gamepad' && depth === 'expanded' && (
<> <>
<SectionLabel>Sticks</SectionLabel> <SectionLabel>Sticks</SectionLabel>
<Segmented <Segmented
@ -384,7 +384,7 @@ function InputsDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
)} )}
{/* ---- MIDI ---- */} {/* ---- MIDI ---- */}
{inp.inputMode === 'midi' && depth !== 'peek' && ( {inp.inputMode === 'midi' && depth === 'expanded' && (
<> <>
<SectionLabel>Device</SectionLabel> <SectionLabel>Device</SectionLabel>
{inp.midiInputs.length === 0 ? ( {inp.midiInputs.length === 0 ? (
@ -466,7 +466,7 @@ function InputsDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
)} )}
{/* ---- Reshape note (only when >2 axes feed the fixed WASM head) ---- */} {/* ---- Reshape note (only when >2 axes feed the fixed WASM head) ---- */}
{reshaping && depth !== 'peek' && ( {reshaping && depth === 'expanded' && (
<p style={{ fontSize: 9, color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}> <p style={{ fontSize: 9, color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}>
{/* TODO(workstream F, docs/redesign/inputs-spec.md — "multiple WASM modules + {/* TODO(workstream F, docs/redesign/inputs-spec.md — "multiple WASM modules +
warm-start"): give every axis its own genuine input dimension by (re)loading a warm-start"): give every axis its own genuine input dimension by (re)loading a
@ -509,7 +509,7 @@ function ModeConfig(ctx: ConsoleCtx, depth: DrawerDepth) {
<span style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)' }}>audio starts on the play gesture</span> <span style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)' }}>audio starts on the play gesture</span>
</div> </div>
<Slider label="master volume" value={ctx.volume} min={0} max={1} step={0.01} onChange={ctx.setVolume} /> <Slider label="master volume" value={ctx.volume} min={0} max={1} step={0.01} onChange={ctx.setVolume} />
{depth !== 'peek' && ( {depth === 'expanded' && (
<> <>
<SectionLabel>Tempo</SectionLabel> <SectionLabel>Tempo</SectionLabel>
<Slider label="bpm" value={ctx.bpm} min={40} max={220} step={1} onChange={ctx.setBpm} /> <Slider label="bpm" value={ctx.bpm} min={40} max={220} step={1} onChange={ctx.setBpm} />
@ -530,7 +530,7 @@ function ModeConfig(ctx: ConsoleCtx, depth: DrawerDepth) {
</> </>
); );
case 'particles': case 'particles':
return depth !== 'peek' ? ( return depth === 'expanded' ? (
<p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}> <p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}>
Flow-field visualiser driven by the first {Math.min(20, ctx.params.length)} outputs (no audio). Flow-field visualiser driven by the first {Math.min(20, ctx.params.length)} outputs (no audio).
{/* TODO(backends-spec §4): port FlowFieldVisualizer + visual preset chips. */} {/* TODO(backends-spec §4): port FlowFieldVisualizer + visual preset chips. */}
@ -539,13 +539,13 @@ function ModeConfig(ctx: ConsoleCtx, depth: DrawerDepth) {
case 'midi': case 'midi':
// The full MIDI config (port picker, CC count, per-output CC/channel/name) // The full MIDI config (port picker, CC count, per-output CC/channel/name)
// + preset bar render via OutputsBackendConfig in RoutingDrawer below. // + preset bar render via OutputsBackendConfig in RoutingDrawer below.
return depth !== 'peek' ? ( return depth === 'expanded' ? (
<p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}> <p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}>
Each output sends a real Web MIDI CC. Pick a port and set CC# / channel per output. Each output sends a real Web MIDI CC. Pick a port and set CC# / channel per output.
</p> </p>
) : null; ) : null;
case 'osc': case 'osc':
return depth !== 'peek' ? ( return depth === 'expanded' ? (
<p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}> <p style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}>
Each output sends to an OSC path with a physical range, over the WebSocket bridge. Each output sends to an OSC path with a physical range, over the WebSocket bridge.
</p> </p>
@ -568,23 +568,8 @@ function RoutingDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
const nameFor = (idx: number, fallback: string) => const nameFor = (idx: number, fallback: string) =>
ctx.outputMode === 'particles' ? VISUAL_NAMES[idx] ?? fallback : fallback; ctx.outputMode === 'particles' ? VISUAL_NAMES[idx] ?? fallback : fallback;
if (depth === 'full') { const expanded = depth === 'expanded';
return ( const rows = expanded ? ctx.params : ctx.params.slice(0, 6);
<>
<div style={{ display: 'flex', gap: 6, flexWrap: 'wrap', alignItems: 'center' }}>
<Chip tone="var(--accent)">{modeDesc.label}</Chip>
<BackendStatusChip ctx={ctx} />
</div>
{ModeConfig(ctx, depth)}
{/* Specialised, editable per-backend config (MIDI/OSC) + named-preset bar. */}
<OutputsBackendConfig ctx={ctx} backend={modeDesc.backend} />
<SectionLabel>Advanced · {modeDesc.label}</SectionLabel>
<BackendAdvanced backend={modeDesc.backend} params={ctx.params} setParam={ctx.setParam} />
</>
);
}
const rows = depth === 'peek' ? ctx.params.slice(0, 6) : ctx.params;
return ( return (
<> <>
<div style={{ display: 'flex', gap: 6, flexWrap: 'wrap', alignItems: 'center' }}> <div style={{ display: 'flex', gap: 6, flexWrap: 'wrap', alignItems: 'center' }}>
@ -596,15 +581,15 @@ function RoutingDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
<BackendStatusChip ctx={ctx} /> <BackendStatusChip ctx={ctx} />
</div> </div>
{ModeConfig(ctx, depth)} {ModeConfig(ctx, depth)}
{/* Specialised per-backend config + named-preset bar (MIDI/OSC); hidden at peek. */} {/* Specialised per-backend config + named-preset bar (MIDI/OSC); hidden when condensed. */}
{depth !== 'peek' && <OutputsBackendConfig ctx={ctx} backend={modeDesc.backend} />} {expanded && <OutputsBackendConfig ctx={ctx} backend={modeDesc.backend} />}
<SectionLabel>Outputs · M mute · S arm · off/fixed/live</SectionLabel> <SectionLabel>Outputs · M mute · S arm · off/fixed/live</SectionLabel>
<div <div
style={{ style={{
display: 'flex', display: 'flex',
flexDirection: 'column', flexDirection: 'column',
gap: 4, gap: 4,
maxHeight: depth === 'peek' ? 220 : 460, maxHeight: expanded ? 460 : 220,
overflow: 'auto', overflow: 'auto',
}} }}
> >
@ -617,14 +602,20 @@ function RoutingDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
param={labelled} param={labelled}
value={values[i] ?? 0} value={values[i] ?? 0}
onChange={(patch) => ctx.setParam(i, patch)} onChange={(patch) => ctx.setParam(i, patch)}
showCurve={depth !== 'peek'} showCurve={expanded}
/> />
); );
})} })}
</div> </div>
{depth === 'peek' && ctx.params.length > 6 && ( {!expanded && ctx.params.length > 6 && (
<span style={{ fontSize: 9, color: 'var(--fg-dim)' }}>+{ctx.params.length - 6} more — expand to edit</span> <span style={{ fontSize: 9, color: 'var(--fg-dim)' }}>+{ctx.params.length - 6} more — expand to edit</span>
)} )}
{expanded && (
<>
<SectionLabel>Advanced · {modeDesc.label}</SectionLabel>
<BackendAdvanced backend={modeDesc.backend} params={ctx.params} setParam={ctx.setParam} />
</>
)}
</> </>
); );
} }
@ -654,7 +645,7 @@ function SettingsDrawer({ depth }: { ctx: ConsoleCtx; depth: DrawerDepth }) {
onChange={(v) => set('monochromeIcons', v)} onChange={(v) => set('monochromeIcons', v)}
label="Monochrome icons" label="Monochrome icons"
/> />
{depth !== 'peek' && ( {depth === 'expanded' && (
<> <>
<div style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)' }}>Unfocused icon colour</div> <div style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)' }}>Unfocused icon colour</div>
<PillToggle <PillToggle
@ -683,7 +674,7 @@ function SettingsDrawer({ depth }: { ctx: ConsoleCtx; depth: DrawerDepth }) {
onChange={(v) => set('inputMap', v as InputMapMode)} onChange={(v) => set('inputMap', v as InputMapMode)}
options={INPUT_MAP_OPTS} options={INPUT_MAP_OPTS}
/> />
{depth !== 'peek' && ( {depth === 'expanded' && (
<p style={{ fontSize: 9, color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}> <p style={{ fontSize: 9, color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}>
The 2D input surface: a rectangular XY map or a circular joystick-style disc. "Follow mode" The 2D input surface: a rectangular XY map or a circular joystick-style disc. "Follow mode"
uses the active mode's declared input (joystick → circular, else rectangular). uses the active mode's declared input (joystick → circular, else rectangular).
@ -700,7 +691,7 @@ function SettingsDrawer({ depth }: { ctx: ConsoleCtx; depth: DrawerDepth }) {
unit="px" unit="px"
onChange={(v) => set('cornerRadius', Math.round(v))} onChange={(v) => set('cornerRadius', Math.round(v))}
/> />
{depth !== 'peek' && ( {depth === 'expanded' && (
<p style={{ fontSize: 9, color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}> <p style={{ fontSize: 9, color: 'var(--fg-dim)', margin: 0, lineHeight: 1.6 }}>
Roundness of buttons, control rows, dock icons and panels. Pills and the circular verdict Roundness of buttons, control rows, dock icons and panels. Pills and the circular verdict
buttons are intentionally exempt. Default 2px. buttons are intentionally exempt. Default 2px.
@ -716,7 +707,7 @@ function SettingsDrawer({ depth }: { ctx: ConsoleCtx; depth: DrawerDepth }) {
const KEYS: [string, string][] = [ const KEYS: [string, string][] = [
['1–5', 'open drawers'], ['1–5', 'open drawers'],
['\\', 'full depth'], ['\\', 'expand drawer'],
['space / ↑', 'commit +'], ['space / ↑', 'commit +'],
['↓', 'perturb / down −'], ['↓', 'perturb / down −'],
['z', 'undo'], ['z', 'undo'],

18
manifold/src/console/Manifold.tsx
View file

@ -285,22 +285,8 @@ export function Manifold({
ctx.stroke(); ctx.stroke();
} }
if (nc > 0.001) { // Noise-cap exploration rings hidden for now (kept for possible later use).
const breathe = 1 + Math.sin(now / 600) * 0.06; void nc;
const rCap = nc * Math.min(W, H) * 0.5 * breathe;
const rCur = rCap * 0.55;
ctx.setLineDash([4, 5]);
ctx.strokeStyle = 'rgba(255,106,0,0.4)';
ctx.lineWidth = 1.5;
ctx.beginPath();
ctx.arc(px, py, rCap, 0, Math.PI * 2);
ctx.stroke();
ctx.strokeStyle = 'rgba(255,106,0,0.7)';
ctx.beginPath();
ctx.arc(px, py, rCur, 0, Math.PI * 2);
ctx.stroke();
ctx.setLineDash([]);
}
ctx.shadowColor = fz ? cyan : accent; ctx.shadowColor = fz ? cyan : accent;
ctx.shadowBlur = 18; ctx.shadowBlur = 18;

31
manifold/src/console/OutputEditor.tsx
View file

@ -2,6 +2,7 @@
* OutputEditor — the per-output menu (state · min · max · static value · curve). * OutputEditor — the per-output menu (state · min · max · static value · curve).
* Opens on hover over an output column/cell. Ported from `OutputEditor.jsx`. * Opens on hover over an output column/cell. Ported from `OutputEditor.jsx`.
*/ */
import { useLayoutEffect, useRef, useState } from 'react';
import type { CSSProperties, ReactNode } from 'react'; import type { CSSProperties, ReactNode } from 'react';
import { CurvePad } from './CurvePad'; import { CurvePad } from './CurvePad';
import type { MFParam, ParamStatus } from './model'; import type { MFParam, ParamStatus } from './model';
@ -51,7 +52,7 @@ function MiniSlider({
disabled={disabled} disabled={disabled}
onChange={(e) => onChange(parseFloat(e.target.value))} onChange={(e) => onChange(parseFloat(e.target.value))}
className="mf-slider-input" className="mf-slider-input"
style={{ width: '100%' }} style={{ width: '100%', ['--mf-pct' as string]: `${Math.max(0, Math.min(1, value))}` } as CSSProperties}
/> />
</label> </label>
); );
@ -67,8 +68,35 @@ export interface OutputEditorProps {
export function OutputEditor({ param, onChange, onHold, onLeave, place }: OutputEditorProps) { export function OutputEditor({ param, onChange, onHold, onLeave, place }: OutputEditorProps) {
const isLive = param.status === 'live'; const isLive = param.status === 'live';
const ref = useRef<HTMLDivElement>(null);
// After the popover lands at its anchored spot, nudge it back on-screen so it
// never clips the viewport edges (matters when the window isn't full-screen).
const [shift, setShift] = useState<{ dx: number; dy: number }>({ dx: 0, dy: 0 });
useLayoutEffect(() => {
const el = ref.current;
if (!el) return;
const fit = () => {
const r = el.getBoundingClientRect();
// r already includes the current shift — back it out to get the natural
// anchored box, then derive the shift that pulls it inside the viewport.
const left = r.left - shift.dx;
const top = r.top - shift.dy;
const M = 8; // keep an 8px gutter from each edge
let dx = 0;
let dy = 0;
if (left + r.width > window.innerWidth - M) dx = window.innerWidth - M - (left + r.width);
if (left + dx < M) dx = M - left;
if (top + r.height > window.innerHeight - M) dy = window.innerHeight - M - (top + r.height);
if (top + dy < M) dy = M - top; // if too tall, prefer pinning the top
if (dx !== shift.dx || dy !== shift.dy) setShift({ dx, dy });
};
fit();
window.addEventListener('resize', fit);
return () => window.removeEventListener('resize', fit);
});
return ( return (
<div <div
ref={ref}
onPointerEnter={onHold} onPointerEnter={onHold}
onPointerLeave={onLeave} onPointerLeave={onLeave}
style={{ style={{
@ -76,6 +104,7 @@ export function OutputEditor({ param, onChange, onHold, onLeave, place }: Output
width: 196, width: 196,
zIndex: 80, zIndex: 80,
...place, ...place,
transform: `translate(${shift.dx}px, ${shift.dy}px)`,
background: 'var(--glass)', background: 'var(--glass)',
backdropFilter: 'blur(14px)', backdropFilter: 'blur(14px)',
WebkitBackdropFilter: 'blur(14px)', WebkitBackdropFilter: 'blur(14px)',

198
manifold/src/console/SandwichStage.tsx Normal file
View file

@ -0,0 +1,198 @@
/**
* SandwichStage — the N-dimensional "parameter sandwich" as a centre stage,
* driven by the REAL engine (the same `EngineApi` instance the rest of the
* console uses). Ported from the standalone /learn renderer.
*
* Each layer is one output channel of the MLP, sampled across the 2-D input
* grid via `engine.inferBatch` → that surface IS the parameter's landscape.
* The grid is re-sampled only when the net changes (`version`); a per-frame
* rAF reads `engine.getOutputs()` for the live probe at the current input.
*
* Three-zone metaphor (input → MLP stack → outputs): this component owns the
* MIDDLE zone. ConsoleApp docks a shrunken Manifold (left) and OutputStage
* (right) around it. Drag the canvas to orbit; it lands top-down (a flat 2-D
* map) and tilts into the 3-D stack. The stack is centred and grows to fill
* the available height as more parameters are shown.
*/
import { useEffect, useRef } from 'react';
import type { EngineApi } from '../engine';
const GRID = 12;
const N = GRID;
const SX = 300, SZ = 300, LAYER_GAP = 120, ZAMP = 92, MARGIN = 34;
// Reference scale: each layer adds a fixed ~px gap (so the stack GROWS with the
// layer count) until it would overflow, at which point the fit scale caps it.
const REF_SCALE = 0.8;
export interface SandwichStageProps {
engine: EngineApi;
/** Engine version counter — re-sample the landscape grid when it bumps. */
version: number;
/** Current 2-D input (drives the probe column). */
pos: [number, number];
/** How many output channels to render as layers (top→down). */
layerCount: number;
/** Per-layer labels (the mode's param names). */
names: string[];
}
export function SandwichStage({ engine, version, pos, layerCount, names }: SandwichStageProps) {
const wrapRef = useRef<HTMLDivElement>(null);
const canvasRef = useRef<HTMLCanvasElement>(null);
const gridRef = useRef<Float32Array | null>(null);
const posRef = useRef(pos);
posRef.current = pos;
const Kref = useRef(layerCount);
Kref.current = Math.max(1, layerCount);
const namesRef = useRef(names);
namesRef.current = names;
const yaw = useRef(0);
const pitch = useRef(Math.PI / 2); // top-down (looks 2-D) until you tilt it
const view = useRef({ cw: 800, ch: 600, scale: 1, ceny: 0 });
const OUT = engine.architecture.outputSize;
// ---- re-sample the landscape grid whenever the net changes ----
useEffect(() => {
const pts: [number, number][] = [];
for (let j = 0; j <= N; j++) for (let i = 0; i <= N; i++) pts.push([i / N, j / N]);
try {
gridRef.current = engine.inferBatch(pts);
} catch {
gridRef.current = null;
}
}, [engine, version]);
// ---- render loop ----
useEffect(() => {
const canvas = canvasRef.current;
const wrap = wrapRef.current;
if (!canvas || !wrap) return;
const ctx = canvas.getContext('2d');
if (!ctx) return;
const DPR = Math.max(1, Math.min(2, window.devicePixelRatio || 1));
const computeView = () => {
const v = view.current;
const K = Kref.current;
v.ceny = -(K - 1) * LAYER_GAP / 2 + ZAMP / 2; // centroid (centre vertically)
let R = 1;
for (let k = 0; k < K; k++) for (const z of [0, 1]) for (const gx of [-0.5, 0.5]) for (const gy of [-0.5, 0.5]) {
const X = gx * SX, Y = (-k * LAYER_GAP + z * ZAMP) - v.ceny, Z = gy * SZ;
const d = Math.sqrt(X * X + Y * Y + Z * Z);
if (d > R) R = d;
}
const fit = (Math.min(v.cw, v.ch) / 2 - MARGIN) / R;
v.scale = Math.min(REF_SCALE, fit); // grow with K, then cap to fit
};
const resize = () => {
const r = wrap.getBoundingClientRect();
view.current.cw = Math.max(160, Math.round(r.width));
view.current.ch = Math.max(160, Math.round(r.height));
canvas.width = view.current.cw * DPR;
canvas.height = view.current.ch * DPR;
ctx.setTransform(DPR, 0, 0, DPR, 0, 0);
computeView();
};
const ro = new ResizeObserver(resize);
ro.observe(wrap);
resize();
const project = (gx: number, gy: number, zv: number, k: number) => {
const v = view.current;
const X = gx * SX, Y = (-k * LAYER_GAP + zv * ZAMP) - v.ceny, Z = -gy * SZ;
const cyA = Math.cos(yaw.current), syA = Math.sin(yaw.current);
const x1 = X * cyA - Z * syA, z1 = X * syA + Z * cyA;
const cpA = Math.cos(pitch.current), spA = Math.sin(pitch.current);
const y2 = Y * cpA - z1 * spA, z2 = Y * spA + z1 * cpA;
return { x: v.cw / 2 + x1 * v.scale, y: v.ch / 2 - y2 * v.scale, depth: z2, wx: X, wy: Y, wz: Z };
};
const hueOf = (k: number, K: number) => (K > 1 ? 26 + (196 - 26) * k / (K - 1) : 26);
const colCss = (k: number, K: number, a: number) => `hsla(${hueOf(k, K).toFixed(0)},85%,60%,${a})`;
const LIGHT = (() => { const v = [0.35, 0.86, 0.36]; const m = Math.hypot(v[0], v[1], v[2]); return [v[0] / m, v[1] / m, v[2] / m]; })();
const clamp = (x: number) => (x < 0 ? 0 : x > 1 ? 1 : x);
let raf = 0;
const draw = () => {
const v = view.current;
const K = Kref.current;
const grid = gridRef.current;
computeView();
ctx.clearRect(0, 0, v.cw, v.ch);
const H = (k: number, i: number, j: number) => (grid ? grid[(j * (N + 1) + i) * OUT + k] : 0.5);
const quads: { a: any; b: any; d: any; e: any; hue: number; L: number; depth: number }[] = [];
const labels: { x: number; y: number; hue: number; name: string }[] = [];
for (let k = 0; k < K; k++) {
const hue = hueOf(k, K);
const P: { x: number; y: number; depth: number; wx: number; wy: number; wz: number }[][] = [];
for (let j = 0; j <= N; j++) { P[j] = []; for (let i = 0; i <= N; i++) P[j][i] = project(i / N - 0.5, j / N - 0.5, H(k, i, j), k); }
for (let j = 0; j < N; j++) for (let i = 0; i < N; i++) {
const a = P[j][i], b = P[j][i + 1], d = P[j + 1][i + 1], e = P[j + 1][i];
const ux = d.wx - a.wx, uy = d.wy - a.wy, uz = d.wz - a.wz, vx = b.wx - e.wx, vy = b.wy - e.wy, vz = b.wz - e.wz;
const nx = uy * vz - uz * vy, ny = uz * vx - ux * vz, nz = ux * vy - uy * vx, nm = Math.hypot(nx, ny, nz) || 1;
const L = clamp(0.32 + 0.62 * Math.abs((nx * LIGHT[0] + ny * LIGHT[1] + nz * LIGHT[2]) / nm));
quads.push({ a, b, d, e, hue, L, depth: (a.depth + b.depth + d.depth + e.depth) / 4 });
}
const lp = project(-0.5, 0.5, H(k, 0, N), k);
labels.push({ x: lp.x, y: lp.y, hue, name: namesRef.current[k] || `p${k}` });
}
quads.sort((p, q) => q.depth - p.depth);
for (const Q of quads) {
ctx.fillStyle = `hsla(${Q.hue},80%,${(26 + Q.L * 44).toFixed(0)}%,0.66)`;
ctx.strokeStyle = `hsla(${Q.hue},80%,82%,0.18)`; ctx.lineWidth = 0.6;
ctx.beginPath(); ctx.moveTo(Q.a.x, Q.a.y); ctx.lineTo(Q.b.x, Q.b.y); ctx.lineTo(Q.d.x, Q.d.y); ctx.lineTo(Q.e.x, Q.e.y); ctx.closePath();
ctx.fill(); ctx.stroke();
}
ctx.font = '11px var(--font-mono, monospace)'; ctx.textAlign = 'right'; ctx.textBaseline = 'middle';
for (const Lb of labels) { ctx.fillStyle = `hsla(${Lb.hue},85%,74%,0.95)`; ctx.fillText(Lb.name, Lb.x - 6, Lb.y); }
// probe column at the current input, marker on each slab from the LIVE output
const out = engine.getOutputs();
const gx = posRef.current[0] - 0.5, gy = posRef.current[1] - 0.5;
const top = project(gx, gy, 1.18, 0), bot = project(gx, gy, -0.18, K - 1);
ctx.strokeStyle = 'rgba(255,255,255,0.3)'; ctx.lineWidth = 1; ctx.setLineDash([3, 4]);
ctx.beginPath(); ctx.moveTo(top.x, top.y); ctx.lineTo(bot.x, bot.y); ctx.stroke(); ctx.setLineDash([]);
for (let k = 0; k < K; k++) {
const val = clamp(out[k] ?? 0.5);
const p = project(gx, gy, val, k), fl = project(gx, gy, 0, k);
ctx.strokeStyle = colCss(k, K, 0.5); ctx.lineWidth = 1;
ctx.beginPath(); ctx.moveTo(p.x, p.y); ctx.lineTo(fl.x, fl.y); ctx.stroke();
ctx.fillStyle = colCss(k, K, 1); ctx.shadowColor = colCss(k, K, 1); ctx.shadowBlur = 12;
ctx.beginPath(); ctx.arc(p.x, p.y, 6, 0, Math.PI * 2); ctx.fill(); ctx.shadowBlur = 0;
ctx.fillStyle = 'var(--bg, #0d0d0d)'; ctx.beginPath(); ctx.arc(p.x, p.y, 2.2, 0, Math.PI * 2); ctx.fill();
}
raf = requestAnimationFrame(draw);
};
raf = requestAnimationFrame(draw);
return () => { cancelAnimationFrame(raf); ro.disconnect(); };
}, [engine, OUT]);
// ---- drag to orbit (yaw + pitch; pitch up to π/2 = exact top-down) ----
const rotating = useRef(false);
const last = useRef({ x: 0, y: 0 });
return (
<div ref={wrapRef} style={{ position: 'absolute', inset: 0, overflow: 'hidden' }}>
<canvas
ref={canvasRef}
style={{ position: 'absolute', inset: 0, width: '100%', height: '100%', cursor: 'grab', touchAction: 'none' }}
onPointerDown={(e) => {
rotating.current = true;
(e.currentTarget as HTMLCanvasElement).setPointerCapture?.(e.pointerId);
last.current = { x: e.clientX, y: e.clientY };
}}
onPointerMove={(e) => {
if (!rotating.current) return;
yaw.current += (e.clientX - last.current.x) * 0.01;
pitch.current = Math.max(0.06, Math.min(Math.PI / 2, pitch.current + (e.clientY - last.current.y) * 0.008));
last.current = { x: e.clientX, y: e.clientY };
}}
onPointerUp={() => { rotating.current = false; }}
onPointerCancel={() => { rotating.current = false; }}
/>
<div style={{ position: 'absolute', top: 10, left: '50%', transform: 'translateX(-50%)', fontSize: 9, letterSpacing: '0.12em', color: 'var(--fg-dim)', pointerEvents: 'none', fontFamily: 'var(--font-mono)' }}>
PARAMETER SANDWICH · drag to rotate · top-down = the flat 2-D map
</div>
</div>
);
}

126
manifold/src/console/VerdictCluster.tsx
View file

@ -18,6 +18,7 @@
import { useRef, useState } from 'react'; import { useRef, useState } from 'react';
import type { CSSProperties } from 'react'; import type { CSSProperties } from 'react';
import type { FeedbackModeUI } from './types'; import type { FeedbackModeUI } from './types';
import { DiceIcon } from './icons';
function ThumbIcon({ size = 24, down = false }: { size?: number; down?: boolean }) { function ThumbIcon({ size = 24, down = false }: { size?: number; down?: boolean }) {
return ( return (
@ -44,6 +45,10 @@ export interface VerdictClusterProps {
onUndo: () => void; onUndo: () => void;
onCommit: () => void; onCommit: () => void;
onReroll: () => void; onReroll: () => void;
/** Small bounded weight perturbation of the current net (left half of the pill). */
onNudge: () => void;
/** Full re-roll of the current net (right half of the pill). */
onRandomise: () => void;
canUndo: boolean; canUndo: boolean;
ab: 'A' | 'B'; ab: 'A' | 'B';
onToggleAB: () => void; onToggleAB: () => void;
@ -62,6 +67,8 @@ export function VerdictCluster({
onUndo, onUndo,
onCommit, onCommit,
onReroll, onReroll,
onNudge,
onRandomise,
canUndo, canUndo,
ab, ab,
onToggleAB, onToggleAB,
@ -116,6 +123,22 @@ export function VerdictCluster({
...extra, ...extra,
}); });
// Two halves of the secondary pill (nudge | randomise).
const pillHalf = (extra: CSSProperties): CSSProperties => ({
flex: 1,
height: 30,
border: 0,
background: 'transparent',
color: 'var(--fg-mute)',
cursor: 'pointer',
fontFamily: 'var(--font-mono)',
fontSize: 'var(--fs-xs)',
letterSpacing: '0.04em',
textTransform: 'uppercase',
transition: 'background var(--dur-fast), color var(--dur-fast)',
...extra,
});
return ( return (
<div <div
onPointerEnter={() => setHover(true)} onPointerEnter={() => setHover(true)}
@ -125,6 +148,17 @@ export function VerdictCluster({
bottom: 28, bottom: 28,
left: '50%', left: '50%',
transform: 'translateX(-50%)', transform: 'translateX(-50%)',
display: 'flex',
flexDirection: 'column',
alignItems: 'center',
gap: 'var(--sp-2)',
opacity: hover || firstSession ? 1 : 0.55,
transition: 'opacity var(--dur-med) var(--ease-console)',
zIndex: 40,
}}
>
<div
style={{
display: 'flex', display: 'flex',
alignItems: 'center', alignItems: 'center',
gap: 'var(--sp-3)', gap: 'var(--sp-3)',
@ -135,9 +169,6 @@ export function VerdictCluster({
border: '1px solid var(--glass-line)', border: '1px solid var(--glass-line)',
borderRadius: 'var(--r-pill)', borderRadius: 'var(--r-pill)',
boxShadow: 'var(--shadow-2)', boxShadow: 'var(--shadow-2)',
opacity: hover || firstSession ? 1 : 0.55,
transition: 'opacity var(--dur-med) var(--ease-console)',
zIndex: 40,
}} }}
> >
<button <button
@ -148,18 +179,15 @@ export function VerdictCluster({
onPointerLeave={() => { onPointerLeave={() => {
if (lp.current) clearTimeout(lp.current); if (lp.current) clearTimeout(lp.current);
}} }}
style={big( style={big({
explore background: explore && exploring ? 'rgba(255,106,0,0.22)' : 'rgba(255,106,0,0.16)',
? exploring color: 'var(--accent)',
? { background: 'rgba(0,204,255,0.16)', color: 'var(--accent-2)' } })}
: { background: 'rgba(0,204,255,0.10)', color: 'var(--accent-2)' }
: { background: 'rgba(255,68,102,0.16)', color: 'var(--danger)' },
)}
onMouseEnter={(e) => (e.currentTarget.style.transform = 'scale(1.08)')} onMouseEnter={(e) => (e.currentTarget.style.transform = 'scale(1.08)')}
onMouseLeave={(e) => (e.currentTarget.style.transform = 'scale(1)')} onMouseLeave={(e) => (e.currentTarget.style.transform = 'scale(1)')}
> >
{/* Explore-mode down is a re-roll/explore (↻), not a dislike thumb. */} {/* Explore mode = dice (re-roll into a scratchpad); dislike mode = thumb-down. */}
{explore ? <span style={{ fontSize: 24, lineHeight: 1 }}>↻</span> : <ThumbIcon down />} {explore ? <DiceIcon size={24} /> : <ThumbIcon down />}
</button> </button>
<button <button
@ -168,9 +196,9 @@ export function VerdictCluster({
onClick={onUndo} onClick={onUndo}
disabled={!canUndo} disabled={!canUndo}
style={big({ style={big({
width: 48, width: 36,
height: 48, height: 36,
fontSize: 20, fontSize: 16,
background: 'var(--bg-2)', background: 'var(--bg-2)',
color: 'var(--fg-mute)', color: 'var(--fg-mute)',
opacity: canUndo ? 1 : 0.4, opacity: canUndo ? 1 : 0.4,
@ -184,11 +212,19 @@ export function VerdictCluster({
type="button" type="button"
title={upTitle} title={upTitle}
onClick={onCommit} onClick={onCommit}
style={big({ style={big(
background: picking ? 'rgba(0,204,255,0.22)' : 'rgba(255,106,0,0.18)', picking
color: picking ? 'var(--accent-2)' : 'var(--accent)', ? {
boxShadow: picking ? '0 0 16px var(--accent-2)' : '0 0 16px var(--glow-accent)', background: 'rgba(0,204,255,0.22)',
})} color: 'var(--accent-2)',
boxShadow: '0 0 16px var(--accent-2)',
}
: {
background: 'rgba(107,194,107,0.20)',
color: 'var(--good)',
boxShadow: '0 0 16px rgba(107,194,107,0.45)',
},
)}
onMouseEnter={(e) => (e.currentTarget.style.transform = 'scale(1.08)')} onMouseEnter={(e) => (e.currentTarget.style.transform = 'scale(1.08)')}
onMouseLeave={(e) => (e.currentTarget.style.transform = 'scale(1)')} onMouseLeave={(e) => (e.currentTarget.style.transform = 'scale(1)')}
> >
@ -196,5 +232,55 @@ export function VerdictCluster({
{explore && exploring ? <span style={{ fontSize: 22, lineHeight: 1 }}>⌖</span> : <ThumbIcon />} {explore && exploring ? <span style={{ fontSize: 22, lineHeight: 1 }}>⌖</span> : <ThumbIcon />}
</button> </button>
</div> </div>
{/* Secondary pill — two halves: nudge (small perturbation) | randomise (full re-roll). */}
<div
style={{
display: 'flex',
alignItems: 'stretch',
width: 220,
background: 'var(--glass)',
backdropFilter: 'blur(14px)',
WebkitBackdropFilter: 'blur(14px)',
border: '1px solid var(--glass-line)',
borderRadius: 'var(--r-pill)',
boxShadow: 'var(--shadow-2)',
overflow: 'hidden',
}}
>
<button
type="button"
title="Nudge — small bounded perturbation of the current net"
onClick={onNudge}
style={pillHalf({ borderRight: '1px solid var(--glass-line)' })}
onMouseEnter={(e) => {
e.currentTarget.style.background = 'rgba(255,106,0,0.12)';
e.currentTarget.style.color = 'var(--accent)';
}}
onMouseLeave={(e) => {
e.currentTarget.style.background = 'transparent';
e.currentTarget.style.color = 'var(--fg-mute)';
}}
>
nudge
</button>
<button
type="button"
title="Randomise — full re-roll of the current net"
onClick={onRandomise}
style={pillHalf({})}
onMouseEnter={(e) => {
e.currentTarget.style.background = 'rgba(255,106,0,0.12)';
e.currentTarget.style.color = 'var(--accent)';
}}
onMouseLeave={(e) => {
e.currentTarget.style.background = 'transparent';
e.currentTarget.style.color = 'var(--fg-mute)';
}}
>
randomise
</button>
</div>
</div>
); );
} }

29
manifold/src/console/icons.tsx
View file

@ -50,6 +50,19 @@ export function ModeIcon({ size = 18, style }: IconProps) {
); );
} }
/** Sandwich — stacked parameter-landscape slabs (the layer toggle). */
export function SandwichIcon({ size = 18, style }: IconProps) {
return svg(
size,
style,
<>
<path d="M3 6.5 12 3l9 3.5L12 10 3 6.5Z" />
<path d="M3 12 12 15.5 21 12" />
<path d="M3 17 12 20.5 21 17" />
</>,
);
}
/** Learning — a brain-ish node graph. */ /** Learning — a brain-ish node graph. */
export function LearningIcon({ size = 18, style }: IconProps) { export function LearningIcon({ size = 18, style }: IconProps) {
return svg( return svg(
@ -117,6 +130,22 @@ export function HelpIcon({ size = 18, style }: IconProps) {
); );
} }
/** Dice — explore / re-roll into a scratchpad (a die showing five pips). */
export function DiceIcon({ size = 22, style }: IconProps) {
return svg(
size,
style,
<>
<rect x="4" y="4" width="16" height="16" rx="3" />
<circle cx="8.5" cy="8.5" r="1.1" fill="currentColor" stroke="none" />
<circle cx="15.5" cy="8.5" r="1.1" fill="currentColor" stroke="none" />
<circle cx="12" cy="12" r="1.1" fill="currentColor" stroke="none" />
<circle cx="8.5" cy="15.5" r="1.1" fill="currentColor" stroke="none" />
<circle cx="15.5" cy="15.5" r="1.1" fill="currentColor" stroke="none" />
</>,
);
}
/** Close (✕). */ /** Close (✕). */
export function CloseIcon({ size = 14, style }: IconProps) { export function CloseIcon({ size = 14, style }: IconProps) {
return svg(size, style, <path d="M6 6l12 12M18 6 6 18" />); return svg(size, style, <path d="M6 6l12 12M18 6 6 18" />);

8
manifold/src/console/shared-ui.tsx
View file

@ -163,7 +163,13 @@ export function CompactAxis({
value={value} value={value}
onChange={(e) => onChange(parseFloat(e.target.value))} onChange={(e) => onChange(parseFloat(e.target.value))}
className="mf-slider-input" className="mf-slider-input"
style={{ width: 120, ['--mf-axis-accent' as string]: accent } as CSSProperties} style={
{
width: 120,
['--mf-axis-accent' as string]: accent,
['--mf-pct' as string]: `${Math.max(0, Math.min(1, value))}`,
} as CSSProperties
}
/> />
<span <span
style={{ style={{

2
manifold/src/console/types.ts
View file

@ -43,7 +43,7 @@ export interface Snapshot {
} }
export type DrawerKey = 'learn' | 'inputs' | 'route' | 'settings' | 'help'; export type DrawerKey = 'learn' | 'inputs' | 'route' | 'settings' | 'help';
export type DrawerDepth = 'peek' | 'expand' | 'full'; export type DrawerDepth = 'condensed' | 'expanded';
export type Focus = 'in' | 'split' | 'out' | 'composite'; export type Focus = 'in' | 'split' | 'out' | 'composite';
export interface Axes { export interface Axes {

4
manifold/src/dock/OutputControlRow.tsx
View file

@ -8,7 +8,7 @@
* (ConsoleApp owns it) — never a second data path (dock-spec §3.2, §8). * (ConsoleApp owns it) — never a second data path (dock-spec §3.2, §8).
*/ */
import { useRef } from 'react'; import { useRef } from 'react';
import type { PointerEvent as ReactPointerEvent } from 'react'; import type { CSSProperties, PointerEvent as ReactPointerEvent } from 'react';
import type { MFParam, ParamStatus } from '../console/model'; import type { MFParam, ParamStatus } from '../console/model';
import { CurvePad } from '../console/CurvePad'; import { CurvePad } from '../console/CurvePad';
@ -288,7 +288,7 @@ export function OutputControlRow({ param, value, onChange, showCurve = false }:
value={param.val} value={param.val}
onChange={(e) => onChange({ val: parseFloat(e.target.value) })} onChange={(e) => onChange({ val: parseFloat(e.target.value) })}
className="mf-slider-input" className="mf-slider-input"
style={{ flex: 1 }} style={{ flex: 1, ['--mf-pct' as string]: `${Math.max(0, Math.min(1, param.val))}` } as CSSProperties}
/> />
</label> </label>
)} )}

15
manifold/src/styles/primitives.css
View file

@ -13,13 +13,21 @@
*/ */
/* ---- Slider (.mf-slider-input) ---- */ /* ---- Slider (.mf-slider-input) ---- */
/* Base reset so the custom track/thumb styling applies even when a consumer
doesn't set these inline (e.g. OutputControlRow's held slider). The colored
fill tracks `--mf-pct` (0..1); it falls back to 0 if a consumer forgets it. */
.mf-slider-input {
-webkit-appearance: none;
appearance: none;
background: transparent;
}
.mf-slider-input::-webkit-slider-runnable-track { .mf-slider-input::-webkit-slider-runnable-track {
height: 4px; height: 4px;
border-radius: 999px; border-radius: 999px;
background: linear-gradient( background: linear-gradient(
to right, to right,
var(--accent) 0%, var(--accent) 0%,
var(--accent) calc(var(--mf-pct) * 100%), var(--accent) calc(var(--mf-pct, 0) * 100%),
var(--bg-3) 0% var(--bg-3) 0%
); );
} }
@ -28,6 +36,11 @@
border-radius: 999px; border-radius: 999px;
background: var(--bg-3); background: var(--bg-3);
} }
.mf-slider-input::-moz-range-progress {
height: 4px;
border-radius: 999px;
background: var(--accent);
}
.mf-slider-input::-webkit-slider-thumb { .mf-slider-input::-webkit-slider-thumb {
-webkit-appearance: none; -webkit-appearance: none;
appearance: none; appearance: none;

4
manifold/tests/e2e/smoke.spec.ts
View file

@ -50,7 +50,9 @@ test('engine loads, spine propagates, console renders', async ({ page }) => {
// 4. The convertible Console rendered (assert on the dock drawer rail, which // 4. The convertible Console rendered (assert on the dock drawer rail, which
// is always present — the old "MEMLNaut" wordmark only shows outside // is always present — the old "MEMLNaut" wordmark only shows outside
// Particle mode now that the particle top bar is a heatmap strip). // Particle mode now that the particle top bar is a heatmap strip).
await expect(page.getByText('Learning')).toBeVisible(); // Drawers are closed by default, so assert on the rail button's title
// rather than the (hidden) drawer-header text.
await expect(page.getByTitle('Learning')).toBeVisible();
// 5. No "C15" anywhere in the rendered UI. // 5. No "C15" anywhere in the rendered UI.
const body = await page.evaluate(() => document.body.innerText); const body = await page.evaluate(() => document.body.innerText);

37
nisps/modes/base.hpp
View file

@ -146,7 +146,7 @@ class ModeBase {
// Run an inference at default inputs so engine has params on first // Run an inference at default inputs so engine has params on first
// process() call, even if no input has been touched. // process() call, even if no input has been touched.
for (std::size_t i = 0u; i < NInputs; ++i) { for (std::size_t i = 0u; i < NInputs; ++i) {
ml_.set_input(i, input_channels_[i]); ml_.set_input(i, effective_input(i));
} }
ml_.process(); ml_.process();
if constexpr (kRouteOutputsToEngine) { if constexpr (kRouteOutputsToEngine) {
@ -165,6 +165,34 @@ class ModeBase {
input_dirty_ = true; input_dirty_ = true;
} }
// ---- Input neutralization (single/double controller toggle) ----
//
// A pinned channel feeds `pin_value_` (neutral, default 0.5) to the MLP
// instead of its live value, without rebuilding/resizing the network.
// Glue toggles which channels are pinned (e.g. single-joystick mode pins
// the second 2D controller's two channels). The stored live value is left
// untouched, so unpinning resumes from the controller's current position.
NISPS_FORCE_INLINE void set_input_pinned(std::size_t idx, bool pinned) noexcept {
if (idx >= NInputs) return;
input_pinned_[idx] = pinned;
input_dirty_ = true;
}
NISPS_FORCE_INLINE bool is_input_pinned(std::size_t idx) const noexcept {
return idx < NInputs && input_pinned_[idx];
}
NISPS_FORCE_INLINE void set_pin_value(float v) noexcept {
if (v < 0.f) v = 0.f;
else if (v > 1.f) v = 1.f;
pin_value_ = v;
input_dirty_ = true;
}
float pin_value() const noexcept { return pin_value_; }
// Effective value fed to the MLP for channel i (pin override applied).
NISPS_FORCE_INLINE float effective_input(std::size_t i) const noexcept {
return input_pinned_[i] ? pin_value_ : input_channels_[i];
}
NISPS_HOT void tick_control() noexcept { NISPS_HOT void tick_control() noexcept {
if constexpr (requires(Derived& d) { d.on_pre_inference(); }) { if constexpr (requires(Derived& d) { d.on_pre_inference(); }) {
static_cast<Derived&>(*this).on_pre_inference(); static_cast<Derived&>(*this).on_pre_inference();
@ -177,9 +205,10 @@ class ModeBase {
if (jolt_.active()) apply_jolt_(); if (jolt_.active()) apply_jolt_();
jolt_.tick_lr_ramp(); jolt_.tick_lr_ramp();
// Forward (possibly Derived-mutated) channels into the MLP. // Forward (possibly Derived-mutated) channels into the MLP, applying
// the per-channel pin override.
for (std::size_t i = 0u; i < NInputs; ++i) { for (std::size_t i = 0u; i < NInputs; ++i) {
ml_.set_input(i, input_channels_[i]); ml_.set_input(i, effective_input(i));
} }
ml_.process(); ml_.process();
if constexpr (kRouteOutputsToEngine) { if constexpr (kRouteOutputsToEngine) {
@ -294,6 +323,8 @@ class ModeBase {
EngineT engine_{}; EngineT engine_{};
MLPType ml_; MLPType ml_;
std::array<float, NInputs> input_channels_{}; std::array<float, NInputs> input_channels_{};
std::array<bool, NInputs> input_pinned_{}; // false => live
float pin_value_ = 0.5f; // neutral
bool input_dirty_ = false; bool input_dirty_ = false;
std::size_t voice_space_idx_ = 0u; std::size_t voice_space_idx_ = 0u;
RingBuffer<ControlEvent, kModeEventBufferSize> events_{}; RingBuffer<ControlEvent, kModeEventBufferSize> events_{};

56
tests/cpp/test_mode_paf_synth.cpp
View file

@ -88,6 +88,62 @@ NISPS_TEST(paf_synth_mode_input_clamping) {
m.set_input(99u, 0.5f); m.set_input(99u, 0.5f);
} }
NISPS_TEST(paf_synth_mode_pinned_inputs_neutralized) {
// Single-joystick mode pins the second 2D controller's channels (2,3).
// A pinned channel must feed the neutral pin value (default 0.5) to the
// MLP regardless of its live value, and pinning must not resize the model.
PAFSynthMode m;
m.setup(48000.f);
NISPS_EXPECT(m.pin_value() == 0.5f);
// Reference: channels 2,3 driven live to 0.5 (the neutral value).
m.set_input(0, 0.2f);
m.set_input(1, 0.8f);
m.set_input(2, 0.5f);
m.set_input(3, 0.5f);
m.tick_control();
std::array<float, 33> ref{};
{
const auto outs = m.ml().outputs();
for (std::size_t i = 0u; i < outs.size(); ++i) ref[i] = outs[i];
}
// Pin channels 2,3, then drive them to arbitrary values. Output must match
// the reference (they are neutralized to 0.5).
m.set_input_pinned(2, true);
m.set_input_pinned(3, true);
NISPS_EXPECT(m.is_input_pinned(2));
m.set_input(2, 0.0f);
m.set_input(3, 1.0f);
m.tick_control();
for (std::size_t i = 0u; i < 33u; ++i) {
NISPS_EXPECT(std::fabs(m.ml().outputs()[i] - ref[i]) < 1e-6f);
}
// The active controller (channels 0,1) still affects the output.
m.set_input(0, 0.9f);
m.tick_control();
bool changed = false;
for (std::size_t i = 0u; i < 33u; ++i) {
if (std::fabs(m.ml().outputs()[i] - ref[i]) > 1e-6f) { changed = true; break; }
}
NISPS_EXPECT(changed);
// Unpinning restores sensitivity on channels 2,3.
m.set_input_pinned(2, false);
m.set_input_pinned(3, false);
m.set_input(0, 0.2f); // restore active channels to the reference pose
m.set_input(1, 0.8f);
m.set_input(2, 0.0f);
m.set_input(3, 1.0f);
m.tick_control();
bool differs = false;
for (std::size_t i = 0u; i < 33u; ++i) {
if (std::fabs(m.ml().outputs()[i] - ref[i]) > 1e-6f) { differs = true; break; }
}
NISPS_EXPECT(differs);
}
NISPS_TEST(paf_synth_mode_engine_and_ml_accessors) { NISPS_TEST(paf_synth_mode_engine_and_ml_accessors) {
PAFSynthMode m; PAFSynthMode m;
m.setup(48000.f); m.setup(48000.f);