b290144 made matrix cells opt-in to prevent joystick-silences-voice,
but that broke modular-ui.updateLive(): the matrix DOM stopped
reflecting live MLP outputs because matrixIndexCache was empty when
buildMatrixIndex() walked paramMeta. This was the same regression
6072fe8 had previously fixed.
Fix it structurally at the DSP layer instead: amp_val now computes
as `clamp(base_amp + max(0, mod_amp)) * level * vel_gain`, so matrix
d08_amp cells can only boost the amp floor — never cut it. base_amp
defaults to 1.0 (always audible), and presets that want classic
envelope-gated voices (slow pad, plucky bass, crystal, morphing
drone) drop base_amp to 0 and layer a positive ADSR→amp route on top.
With the DSP guard in place, all 480 matrix cells can safely live
in paramMeta again, and updateLive() gets its live visual feedback
back. Revert the opt-in gate in _rebuildParamMeta and the 32-param
test counts, and add a regression test asserting that every matrix
destination has 48 cells in paramMeta — that's what updateLive needs.
Modular sub-engines computed amp_val as a pure function of mod_amp (the
matrix d08_amp destination sum), so once the MLP drove the matrix cells
every joystick movement had a chance to silence the voice: matrix cells
have signed range [-1, 1], sigmoid outputs near 0.5 denormalise to 0,
and the amp gate collapsed. Additive survived in scattered regions
because it only has one kill-switch (d08_amp); subtractive and fm were
almost always dead because they also have d05_cutoff and d01_op1_level.
Two changes:
1. DSP: each sub-engine gets a base_amp hslider (default 1.0) so
amp_val = clamp(base_amp + mod_amp) * level * vel_gain. At the
default the voice is always fully open and d08_amp modulation is
purely additive decoration; drop base_amp to 0 for classic
ADSR-gated VCA behaviour.
2. ModularEngine._rebuildParamMeta: restore the _exposedMatrixCells
gate (default empty). paramCount drops from 512 to 32 (4 ADSR * 4
+ 8 LFO * 2); matrix cells are opt-in via setExposeMatrixCell.
_applyDefaultPatch no longer writes s00_d08_amp since base_amp
keeps the voice audible without routing.
Tests updated for the new 32-param baseline; matrix-cell persistence
test now calls setExposeMatrixCell(1, 5, true) before asserting the
cell lands in paramMeta. Drive-by: engine-switching test bumped from
3 to 4 engine cards (stale since the modular engine was added).
Previous fix pulled matrix cells out of paramMeta to avoid the
default-patch-clobbering silence issue. Side effect: paramCount dropped
from 512 to 32, the heatmap strip and synth visualizer shrank
dramatically, and moving the joystick no longer animated matrix cells
(the user's two most recent complaints).
Better approach: put matrix cells back in paramMeta (512 outputs), but
when iml.exampleCount === 0 substitute the normalised default-patch
vector for the raw MLP output in routeOutputs. The engine sees the
default patch exactly, audio works, and the user still sees 512 cells
in the heatmap / matrix grid. As soon as they capture their first
training example the MLP resumes driving everything normally.
ModularEngine.getDefaultNormalizedOutputs() returns the normalised
default value per paramMeta entry, reading from _lastRawByLabel first
(so user edits via click or _setRawByLabel propagate) and falling back
to the walk-entry init field.
modular-ui setCell now prefers engine.setParam over _setRawByLabel when
the cell is in paramMeta, so writes flow through the normal tracking
path and are visible to getDefaultNormalizedOutputs on the next tick.
Verified in headless Chromium: switching to modular, cold start, reading
engine._lastRawByLabel after routeOutputs ticks shows ampRaw=1,
attackRaw=0.01, enableRaw=1 — the default patch is preserved. paramCount
is 512 again. 0 console errors.
The matrix-seeding edit from the previous commit added a second
`const destNames = engine.destNames || [];` inside rebuildMatrixGrid
while the same name was already declared earlier in the function body.
Chrome threw SyntaxError at parse time, which in turn prevented
a-app.js from wiring window.__nisps at all, which is what made the
earlier FaustWorkletProcessor error look like the root cause — it
was just the next thing that broke once the parse error let partial
modules load.
Verified end-to-end in headless Chromium via Playwright: engine
switches to modular, paramCount=32, audioCtx running, worklet node
live, zero console errors.
Also updates the Modular engine card description to reflect the new
default (32 params = 4 ADSR × 4 + 8 LFO × 2, matrix cells opt-in).
The globalThis.FaustWorkletProcessor attachment on the base class isn't
enough in practice: Chrome's AudioWorkletGlobalScope isolates top-level
class declarations between separate addModule() calls, so a subclass
script loaded via a second addModule() still throws ReferenceError on
its 'extends FaustWorkletProcessor' clause.
Workaround: fetch both files, concatenate, and addModule() a single
blob URL. The base class and subclass end up in the same script
evaluation context and the extends clause resolves. Processor names
are memoised on a static Set so repeated sub-engine swaps don't try
to re-register (which would throw).
Two fixes driven by user reports:
1. MLP wasn't affecting the sound. Matrix cells are now opt-in to the
MLP output vector rather than always-driven. Default modular paramMeta
is 32 mod-source params (4 ADSR * A/D/S/R + 8 LFO * rate/morph), down
from 512. The default patch's MM_Matrix/s00_d08_amp=1.0 now survives
the first inference tick because it's not in paramMeta — ADSR1 stays
routed to amp and the MLP drives envelope shape per joystick position.
Matrix cells still clickable as direct-DSP knobs in modular-ui: setCell
now routes through engine._setRawByLabel(). A later UI pass can add a
"expose to MLP" menu entry that calls engine.setExposeMatrixCell(s,d).
Also removes the earlier exampleCount-based routing gate; no longer
needed now that the default patch is stable.
2. Per-group curve drawer (hover over section labels on the synth
visualizer) is now available for all synth engines, not just C15.
Refactored showGroupDrawer behind a getSectionView(sectionIndex)
helper that returns a uniform view for either C15 (via SYNTH_SECTIONS
+ groupOverrides) or non-C15 (via nonC15Sections + nonC15GroupCurves
+ engineParamOverrides).
Group-level curve persists across sub-engine swaps by group name, so
e.g. tuning the "ADSR 1" curve survives a switch between subtractive
and fm without being reset.
ModularEngine: adds setExposeMatrixCell(s,d,exposed) +
getExposedMatrixCells() + clears exposed cells on sub-engine swap.
Modular mode's default patch sets MM_Matrix/s00_d08_amp = 1.0 via
_setRawByLabel at engine init, but on the first inference tick
routeOutputs writes the untrained MLP's output (~0.5 normalised) to
every paramMeta index. For matrix cells the raw range is [-1, 1], so
normalised 0.5 denormalises to 0 — clobbering the default amp routing
and killing all audio.
Skip the setParam loop for modular when iml.exampleCount === 0, so the
worklet keeps running on the default patch we already pushed at init.
Once the user captures at least one training example the MLP has a
target to reproduce, and routing resumes normally.
Only affects modular mode; other engines are unchanged.
Two fixes for Modular mode:
1. faust-worklet-processor.js: explicitly attach FaustWorkletProcessor
to globalThis. Class declarations at the top of a classic script are
lexically scoped to that script's evaluation context and do not
propagate across separate addModule() calls, so the base class was
invisible to subclass processors when they loaded in AudioWorklet-
GlobalScope. Symptom: "FaustWorkletProcessor is not defined" at the
extends clause of modular-subtractive-processor.js.
2. modular-ui.js + a-app.js: wire live matrix cell updates. Phase C
wired matrix cells for writes (click cycles, precise editor) but
not for reads — the MLP-driven values never propagated to the DOM.
modular-ui now exposes updateLive(outputs), called from routeOutputs
on every inference tick. Throttled to ~20 fps internally to avoid
DOM thrashing. Only visible sources (within adsrCount/lfoCount) are
updated; muted slots stay dark.
If faust isn't in PATH but nix-shell is, transparently re-exec the
script via `nix-shell -p faust --run` instead of erroring out with a
hint. NISPS_BUILD_IN_NIX_SHELL guards against infinite recursion if the
shell somehow still lacks faust.
showGroupDrawer indexes into the hardcoded SYNTH_SECTIONS table and
groupOverrides, which only exist for the shaper-feedback engine. For
additive/fm/modular engines, paramToSection is rebuilt dynamically from
paramMeta groups, so region.index points into a larger dynamic section
list and SYNTH_SECTIONS[region.index] is undefined. Hovering a section
label in Modular mode threw "Cannot read properties of undefined
(reading 'name')".
These engines have their own param UIs; the C15 group drawer doesn't
apply. Guard and early-return.
New "Modular" engine in a-immersive with three hot-swappable Faust
sub-engines (subtractive/additive/fm) sharing a common modulation pool:
16 ADSR slots + 32 LFO slots (single-knob sine->tri->square->saw
wavemorph) routed through a 48-source x 10-destination matrix per
engine. Per-connection scalar amounts in [-1, 1], summed at each
destination. Default MLP output count is 512 (32 mod-source params +
480 matrix cells); model reinits on sub-engine swap, count change, or
engine-param exposure toggle.
Faust layer:
- mod-pool.lib: shared ADSR/LFO/source-bus library
- gen-modular-dsp.py: byte-reproducible generator (source of truth)
- modular-subtractive: faithful Minimoog (3 osc, ladder filter, no envs)
- modular-additive: 64-partial, spectral shape + formants, no envs/LFOs
- modular-fm: 4-op matrix + self-feedback, no envs
- All three share d08=amp, d09=pan conventions
- MODULAR_DESTINATIONS.md: authoritative destination table
JS layer:
- ModularEngine: self-contained SynthEngine with getState/setState,
setSubEngine, setModSourceCount, setExposeEngineParam
- modular-ui: drawer with sub-engine toggle, ADSR/LFO count steppers,
per-slot enable switches, matrix grid editor (tap-cycle, long-press
precise, right-click menu, negative amounts), preset overlay
- modular-presets: 6 named presets (Slow pad, Plucky bass, Crystal,
DX bell, Morphing drone + default)
- a-app.js: Modular mode registered, paramMeta:change -> resizeMLP,
modular DSP state persisted under modularDspState, window.__nisps
debug hooks for programmatic control
Tests: tests/e2e/modular-mode.spec.js (11 Playwright tests, all passing
including DSP state survives reload, sub-engine swap keeps paramCount,
preset apply verification).
Also fixes a pre-existing build.sh bug where the -e flag caused faust
to overwrite .wasm outputs with expanded DSP source text, leaving
additive/fm-matrix/eoc-* committed as invalid WebAssembly. Rebuilt all
affected engines with the corrected script. Added an early-message
buffer to faust-worklet-processor.js so setParam calls arriving before
wasm instantiation are queued rather than dropped (needed when the user
configures modular state before clicking Start Audio).
Add #shapeseq-container div with a canvas and chain mount point,
positioned above the bottom controls. Visible when ?shapeseq=1 is
set and synth mode is active.
Move the lazy AudioCanvas initialization above the outputCountForMode()
call in setOutputMode() so that the real output count is available
instead of the 12-output fallback.
Add serializeShapeSeqState/restoreShapeSeqState for complete ShapeSeq
state capture (chain config, primitive states, projection knobs, voice
mode, MLP weights, freeze state, clock config). SessionPresetManager
lazily imports the session module when a ShapeSeq engine is registered,
including ShapeSeq state in save/load without a hard dependency.
When frozen, param rows dim to indicate frozen state. Tapping a row
toggles it to "live" (highlighted, receives ML deltas). Unfreezing
clears all visual indicators.
Replace the inline chain builder with a full-screen slide-up drawer
(overlay + glass backdrop + drag handle). Param sliders now subscribe
to a vanilla signal (createArraySignal) for O(changed params) updates
instead of querySelectorAll sweeps. Card list uses keyed reconciliation.
Mono mode (default) kills the previous note before firing a new one —
classic monophonic synth behavior. Poly mode retriggers same-pitch
notes without killing others. All active notes are released on mode
switch.
Unified mode: a single MLP drives both timbre and sequence — the timbre
MLP's outputs are partitioned and a configurable slice feeds the
sequencer pipeline via setSequenceOutputsFromTimbre().
Dual mode (default): separate MLPs for timbre and sequence with
independent training. The sequencer owns its own IML instance.
Two freeze modes for capturing and preserving musical state:
- Algorithm freeze: snapshot all primitive params + seeds. Individual
params can be selectively unfrozen for ML-driven exploration via
DeltaController (applies MLP deltas to frozen base values with
per-param boundary enforcement).
- Pattern freeze: snapshot the realized note pattern. The sequencer
bypasses the primitive chain entirely and loops the frozen pattern.
Extracts _runPipeline() from setSequenceInputs() for reuse.
The dirty-check that skips re-evaluation when inputs haven't changed now
also checks a generation counter. Config changes (tempo, step count,
projection, chain edits) bump the counter, ensuring the pipeline re-runs
even when joystick position is static.
The sequence MLP output count is now configurable (default 16). Hidden
layer widths scale with the output count to keep inference cheap while
providing sufficient capacity. The sequencer exposes setOutputCount()
to resize the MLP at runtime.
Replace the composable transform chain (VelocityCurve, GateThreshold,
RangeMap, OctaveFolder, StutterMap, presets) with 3 straightforward
post-chain knobs: velocityCurve, gateThreshold, and pitchRange. The
old system was over-engineered for a research prototype — the new API
is easier to wire to UI controls and reason about.
PitchWalker was classified as a generator but always consumed the
incoming pattern's triggers. Reclassify as a processor so it modifies
the pattern produced by an upstream generator (clonePattern instead of
createPattern). Add reEvalOnLoop flag on the Primitive base class; when
set, the sequencer re-evaluates the chain at each loop start so
stateful primitives produce evolving patterns across loops.
Generators can now have independent step counts. When merging patterns
of different lengths, both are tiled to their LCM. Chain exposes
setGeneratorStepCount/getGeneratorStepCount for polyrhythmic control.
Replace the pitch*127 encoding hack with a dedicated midiNote field on
pattern steps. IntervalLock now writes integer MIDI notes to step.midiNote
while preserving the original [0,1] pitch for other consumers. The clock
and sequencer bridge pass midiNote through the event pipeline.
Also reworks mergePatterns to be trigger-aware: when only one generator
fires at a step, use its values directly instead of averaging with the
other generator's defaults.
The EventBus was duplicated inside shapeseq/ but is used by other
playground code too. Move the canonical implementation to js/event-bus.js
and turn the shapeseq copy into a re-export shim.
- Per-file loop count (0-4) with stepper UI on each waveform cell;
dynamic MLP output sizing (8 range + N_loops × 4 position outputs)
- Beat-synced loop length: global bars/beats selector (2/4/8/16/32/64)
with per-loop overrides via long-press dropdown, persistent badges
- Global param range groups: dual-thumb range sliders (volume, pitch,
position, filter) with ML-driven min/max and per-loop position mapping
- Pitch lock toggle (1×) to freeze playback rate across all submodes
- Fix: _resizeOutputs now syncs _loopOutputs to prevent stale reads
- Fix: granular visuals consolidated per-clip to prevent multi-loop flicker
Tests cover:
- Default state (C15, 126 params, heatmap cells)
- Engine switcher UI (3 cards, active state)
- Switch to Additive (48 params, outputs bounded, training works)
- Switch to FM Matrix (55 params, outputs bounded, training works)
- Round-trip switching (C15→Add→C15, C15→FM→Add→C15)
- Warm-start weight preservation across resize
- EOC chain accessibility across engines
- SynthVisualizer visibility across engines
- No console errors during any switch
Also fixes: faustJsonToParamMeta now filters [hidden:1] params,
giving correct counts (48 additive, 55 FM) instead of including
freq/gate/_vel control inputs.
When switching from C15 to Additive or FM engines, the MLP was resized
to 0 outputs because paramMeta wasn't loaded until init() (which needs
AudioContext). The SynthVisualizer also disappeared because its section
map was hardcoded to C15's 18 param groups.
- Add FaustEngineBase.loadParamMeta() for eager JSON-only loading
- Call loadParamMeta() before setActiveEngine() in engine switch handler
- Add SynthVisualizer.rebuild(paramMeta) to derive sections from groups
- Add rebuildParamToSection() / restoreC15ParamToSection() for engine-agnostic
param-to-section mapping
- Fix tooltip to use engineParamOverrides for non-C15 engines
- Engine switcher now lives in the Synth drawer (was incorrectly in NISPS params drawer)
- Params drawer renamed to "NISPS" (it's for ML engine tuning, not synth engine selection)
- Output mode button says "Synth" (generic) not "C15 Shaper-Feedback" (engine-specific)
- Dock icon for params drawer now labelled "NISPS" with correct tooltip
Support opus, webm, mp4, wma, aif, aiff in addition to existing formats.
Add "+" button to control bar for importing files without drag-and-drop (mobile).
Additive engine: 5 presets across 3 tiers (Spectral Basics, Formant Play,
Harmonic Sculptor, Phase Explorer, Full Spectrum). FM engine: 5 presets
across 3 tiers (Simple FM, Bell Tones, Matrix Explorer, Feedback Machine,
Full Matrix).
Engine-aware preset system: applyPreset() dispatches to C15 groupOverrides
or new flat engineParamOverrides depending on active engine. Preset selector
dropdown rebuilds on engine switch. URL ?preset param searches all engines.
Resolved conflicts: merged all 6 module imports in eoc-chain-ui.js and index.js.
All moduleFactory cases now return real Faust WASM implementations — no stubs remain.
