Claude (stream-10)
54821fffd0
Builds the SolidJS-side feature parity with the legacy a-immersive
playground. Every Phase 1–4 feature called out in
.local/recon/04-playground.md now flows through the runtime → stores →
primitives pipeline.
Phase 1 (compound axes / zoom / drawer / trail):
- control-routing.ts: resolves Boldness/Memory/Precision into
input/output/exploration store writes via createEffect.
- SettingsDrawer (new): tabbed sections for Input / Training /
Exploration / Output / Per-param overrides / Advanced. Per-param
ParamEditor list wires through modeStore.setOverride.
- JoyMap navigator added to ModeShell; tap-to-return + long-press
region pin handlers wired into the runtime.
- Anchor mode toggle (auto/sticky/center) exposed via drawer.
Phase 2 (pinning / history / A/B):
- features/snapshots.ts: bridge sessionStore push/pop to actual
Float32Array weights. Auto-snapshot before train/randomize/thumbs-down.
- Undo button + long-press snapshot list popup wired in ModeShell.
- A/B capture/toggle/accept/revert in SettingsDrawer.
- features/overrides.ts buildPinMask() composes override `pinned`
flags + sessionStore param pins; passed to mlStore.moveWeights.
- Region pinning long-press → addRegionPin + auto-snapshot tag
"pinned baseline".
Phase 3 (refinement / exploration):
- features/heatmap-sampler.ts: 16x16 grid via mlStore.inferBatch,
three colour modes, throttled 5/sec, refreshed on ml.trained /
ml.delta_update bus events.
- exploration-store.ts (new): RL noise level, floor/cap/growth/decay,
spread, learning rate, weight decay, auto-explore, pressure.
- Auto-explore timer in mode-runtime drives moveWeights at user
interval, zoom-scaled intensity.
- Pressure feedback: window pointerdown/up timestamps mapped to
explorationStore.setPressure; feeds growNoise/decayNoise.
Phase 4 (output / persistence / polish):
- features/weight-health.ts: histogram, status, per-layer L2 deltas,
vanishing/exploding/converged classification.
- features/session-preset.ts: full state capture/restore +
base64url URL sharing; main.tsx applies on load.
- SettingsDrawer Advanced tab renders WeightHealth, GradientFlow,
LayerStats, Heatmap, session preset save/load/share.
Override application:
- features/overrides.ts applies per-param mute/freeze/curve/range
between MLP outputs and the engine. Mode runtime exposes
`paramOutputs` (length = schema.params) which all modes now bind to
OutputDisplay so ranges and mutes are visible.
- Freeze flags push into outputStore.freezeMask reactively.
Mic input:
- features/mic-input.ts: getUserMedia + AnalyserNode-derived
{energy, brightness, pitch, aperiodicity}. Mode runtime feeds
them into channels 2..(input_size-1) when active. ModeShell shows
a Mic toggle for audio_in modes.
Debug probe (probe.ts):
- Synchronous bypass for snapshot, A/B, pins, overrides, axes,
spread, output freeze, heatmap, weight health, session presets,
URL params, bus emit/on. Untracked reads/writes throughout to
avoid SolidJS reactivity surprises in tests.
Modes:
- All 8 firmware mode TSX files now use the default SettingsDrawer
(no per-mode SliderBank scaffolding). They bind OutputDisplay to
runtime.paramOutputs (post-override) instead of processedOutputs.
Build status:
- bun run typecheck: clean
- bun run build: clean
- dev server smoke: index, /modes, mode-runtime, SettingsDrawer,
features/* all serve.
Note: bd close meml-5wg failed because Dolt server unreachable from
this worktree. Issue should be closed manually by orchestrator.
96 lines
3.5 KiB
TypeScript
96 lines
3.5 KiB
TypeScript
/**
|
|
* SoundAnalysisMIDIMode — sound analysis → MIDI CC output (audio_in input).
|
|
*
|
|
* Schema declares `primary_input: 'audio_in'` and `engine_id: 'thru'` (no
|
|
* synthesis). Stream 10 wires the mic into channels 2..5 via ModeShell's
|
|
* Mic button + the runtime's MicInput. The joystick is used as a fallback
|
|
* for channels 0..1 and during testing without mic permissions.
|
|
*
|
|
* MIDI output routing is read-only here — the SettingsDrawer's per-param
|
|
* override editor lets users tighten the active range / mute params; the
|
|
* actual MIDI device wiring is left for the engine host to handle (or for
|
|
* a future stream that adds WebMIDI sender). For now CC values are
|
|
* visible in the live OutputDisplay.
|
|
*/
|
|
|
|
import { Component, Show } from 'solid-js';
|
|
import { ModeShell } from './ModeShell';
|
|
import { useModeRuntime } from './mode-runtime';
|
|
import { VirtualJoystick } from '../primitives/VirtualJoystick';
|
|
import { OutputDisplay } from '../primitives/OutputDisplay';
|
|
import { LossPlot } from '../primitives/LossPlot';
|
|
import { SoundAnalysisMidiSchema } from './generated/sound_analysis_midi_schema';
|
|
|
|
export const SoundAnalysisMIDIMode: Component = () => {
|
|
const schema = SoundAnalysisMidiSchema;
|
|
const runtime = useModeRuntime(schema);
|
|
|
|
return (
|
|
<ModeShell
|
|
schema={schema}
|
|
runtime={runtime}
|
|
drawerTitle="Sound analysis → MIDI settings"
|
|
primaryInput={() => (
|
|
<>
|
|
<Show
|
|
when={schema.ui.primary_input === 'audio_in'}
|
|
fallback={
|
|
<VirtualJoystick
|
|
size={260}
|
|
onMove={(x, y) => runtime.setInput(x, y)}
|
|
position={runtime.pipedInput}
|
|
ariaLabel="Sound analysis joystick"
|
|
/>
|
|
}
|
|
>
|
|
<div
|
|
style={{
|
|
display: 'flex',
|
|
'flex-direction': 'column',
|
|
'align-items': 'center',
|
|
gap: 'var(--sp-3)',
|
|
padding: 'var(--sp-4)',
|
|
background: 'var(--bg-2)',
|
|
border: `1px ${runtime.mic.started() ? 'solid var(--accent-2)' : 'dashed var(--line-strong)'}`,
|
|
'border-radius': 'var(--r-2)',
|
|
'min-width': '260px',
|
|
'min-height': '200px',
|
|
'justify-content': 'center',
|
|
color: 'var(--fg-mute)',
|
|
'text-align': 'center',
|
|
}}
|
|
>
|
|
<strong style={{ color: runtime.mic.started() ? 'var(--accent-2)' : 'var(--fg-mute)' }}>
|
|
{runtime.mic.started() ? '🎤 Mic active' : 'Mic input'}
|
|
</strong>
|
|
<span style={{ 'font-size': 'var(--fs-xs)' }}>
|
|
{runtime.mic.started()
|
|
? 'Audio features → channels 2..5. Joystick below drives channels 0..1.'
|
|
: 'Tap "Mic" in the header to feed mic features into the model.'}
|
|
</span>
|
|
<VirtualJoystick
|
|
size={200}
|
|
onMove={(x, y) => runtime.setInput(x, y)}
|
|
position={runtime.pipedInput}
|
|
ariaLabel="Manual joystick fallback"
|
|
/>
|
|
</div>
|
|
</Show>
|
|
</>
|
|
)}
|
|
outputArea={() => (
|
|
<>
|
|
<OutputDisplay
|
|
values={runtime.paramOutputs}
|
|
width={360}
|
|
height={120}
|
|
color="var(--info)"
|
|
/>
|
|
<LossPlot history={runtime.training.lossHistory} width={360} height={70} />
|
|
</>
|
|
)}
|
|
/>
|
|
);
|
|
};
|
|
|
|
export default SoundAnalysisMIDIMode;
|