406 lines
20 KiB
HTML
406 lines
20 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
|
|
<meta name="apple-mobile-web-app-capable" content="yes">
|
|
<meta name="mobile-web-app-capable" content="yes">
|
|
<title>NISPS Immersive</title>
|
|
<link rel="preconnect" href="https://fonts.googleapis.com">
|
|
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
|
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:wght@400;500;700&display=swap" rel="stylesheet">
|
|
<link rel="stylesheet" href="css/a-immersive.css">
|
|
</head>
|
|
<body>
|
|
<!-- Back button -->
|
|
<a href="designs.html" class="back-btn" title="Back to designs">
|
|
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
|
<path d="M10 2L4 8l6 6"/>
|
|
</svg>
|
|
</a>
|
|
|
|
<!-- Synth quick controls (play button + drawer, next to back button) -->
|
|
<div class="synth-quick-controls hidden" id="synth-quick-controls">
|
|
<button class="play-btn" id="quick-play" title="Start audio + arpeggiator">
|
|
<svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor" id="quick-play-icon"><path d="M4 2l10 6-10 6z"/></svg>
|
|
</button>
|
|
<div class="play-drawer" id="play-drawer">
|
|
<label>Vol <input type="range" id="quick-vol" min="0" max="1" step="0.01" value="0.5"></label>
|
|
<label>BPM <input type="range" id="quick-bpm" min="40" max="240" step="1" value="120"><span id="quick-bpm-val">120</span></label>
|
|
</div>
|
|
<select class="preset-select" id="synth-preset-select" title="Synth preset">
|
|
<option value="">Manual</option>
|
|
<optgroup label="Beginner">
|
|
<option value="beginner-1">1.1</option>
|
|
<option value="beginner-2">1.2</option>
|
|
<option value="beginner-3">1.3</option>
|
|
<option value="beginner-4">1.4</option>
|
|
</optgroup>
|
|
<optgroup label="Intermediate">
|
|
<option value="intermediate-1">2.1</option>
|
|
<option value="intermediate-2">2.2</option>
|
|
<option value="intermediate-3">2.3</option>
|
|
<option value="intermediate-4">2.4</option>
|
|
</optgroup>
|
|
<optgroup label="Advanced">
|
|
<option value="advanced-1">3.1</option>
|
|
<option value="advanced-2">3.2</option>
|
|
<option value="advanced-3">3.3</option>
|
|
</optgroup>
|
|
<optgroup label="Expert">
|
|
<option value="expert-1">4.1</option>
|
|
<option value="expert-2">4.2</option>
|
|
</optgroup>
|
|
</select>
|
|
</div>
|
|
|
|
<!-- MIDI CC quick controls (shown in midi-cc mode, mirrors synth quick controls layout) -->
|
|
<div class="midi-cc-quick-controls hidden" id="midi-cc-quick-controls">
|
|
<button class="play-btn" id="midi-cc-enable-btn" title="Enable MIDI output">
|
|
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5">
|
|
<rect x="2" y="3" width="12" height="10" rx="1.5"/><path d="M4 9v2M6 7v4M8 8v3M10 7v4M12 9v2"/>
|
|
</svg>
|
|
</button>
|
|
<select class="preset-select" id="midi-cc-output-select" title="MIDI output device">
|
|
<option value="">No device</option>
|
|
</select>
|
|
<select class="preset-select" id="midi-cc-quick-preset" title="MIDI CC preset" style="max-width:130px">
|
|
<option value="">Manual</option>
|
|
</select>
|
|
<span class="synth-status" id="midi-cc-status" style="font-size:0.65rem;opacity:0.7"></span>
|
|
</div>
|
|
|
|
<!-- Audio Canvas mode container -->
|
|
<div id="audio-canvas-wrap" style="display:none;position:absolute;inset:0;z-index:1"></div>
|
|
|
|
<!-- Fullscreen flow field canvas -->
|
|
<canvas id="vis-canvas"></canvas>
|
|
|
|
<!-- Synth visualization canvas -->
|
|
<canvas id="synth-vis-canvas"></canvas>
|
|
|
|
<!-- Heatmap strip (top) — bars are click/draggable -->
|
|
<div class="heatmap-strip" id="heatmap-strip">
|
|
<div class="heatmap-cells" id="heatmap-cells"></div>
|
|
<div class="heatmap-tooltip" id="heatmap-tooltip"></div>
|
|
</div>
|
|
|
|
<!-- Floating joystick with integrated map -->
|
|
<div class="joystick-container" id="joystick-container">
|
|
<div class="joystick-glow" id="joystick-glow"></div>
|
|
<canvas id="joy-map" width="160" height="160"></canvas>
|
|
<div class="noise-ring" id="noise-ring"></div>
|
|
<div class="follow-badge hidden" id="follow-badge">FOLLOW</div>
|
|
<div id="gamepad-status" style="color: #ff6a00; font-size: 0.6rem; text-align: center; position: absolute; bottom: -16px; left: 0; right: 0; pointer-events: none;"></div>
|
|
</div>
|
|
|
|
<!-- Hand tracking PIP (replaces joystick position when active) -->
|
|
<div class="hand-pip hidden" id="hand-pip">
|
|
<video id="hand-video" playsinline autoplay muted></video>
|
|
<canvas id="hand-overlay"></canvas>
|
|
<div class="hand-status" id="hand-status">Loading...</div>
|
|
<div class="gesture-indicator" id="gesture-indicator">
|
|
<svg class="gesture-ring" viewBox="0 0 36 36">
|
|
<circle class="gesture-ring-bg" cx="18" cy="18" r="16" />
|
|
<circle class="gesture-ring-progress" cx="18" cy="18" r="16" />
|
|
</svg>
|
|
<span class="gesture-label" id="gesture-label"></span>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- RL floating buttons (with undo between/below) -->
|
|
<div class="rl-buttons" id="rl-buttons">
|
|
<button class="rl-btn rl-down" id="btn-thumbsdown" title="Explore more"><span class="rl-icon">−</span><span class="key-num">1</span></button>
|
|
<button class="rl-btn rl-up" id="btn-thumbsup" title="Keep this"><span class="rl-icon">+</span><span class="key-num">2</span></button>
|
|
<button class="rl-undo-btn" id="btn-undo" title="Undo last action">
|
|
<svg width="12" height="12" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
|
<path d="M3 7h6a4 4 0 0 1 0 8H7"/><path d="M3 7l3-3M3 7l3 3"/>
|
|
</svg>
|
|
</button>
|
|
</div>
|
|
|
|
<!-- Status line (floating, minimal) -->
|
|
<div class="status-line" id="status-line">
|
|
<span id="status-text">0 examples · untrained</span>
|
|
</div>
|
|
|
|
<!-- Right-side dock (macOS-style) -->
|
|
<div class="dock" id="dock">
|
|
<button class="dock-icon" data-drawer="training" title="Training">
|
|
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round">
|
|
<path d="M3 8h10"/><rect x="1" y="5" width="3" height="6" rx="0.5"/><rect x="12" y="5" width="3" height="6" rx="0.5"/><rect x="3" y="6.5" width="2" height="3" rx="0.3"/><rect x="11" y="6.5" width="2" height="3" rx="0.3"/>
|
|
</svg>
|
|
<span class="dock-label">Train</span>
|
|
</button>
|
|
<button class="dock-icon" data-drawer="mode" title="Input / Output mode">
|
|
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round">
|
|
<circle cx="5" cy="5" r="2"/><circle cx="11" cy="11" r="2"/><path d="M3 13L13 3"/>
|
|
</svg>
|
|
<span class="dock-label">Mode</span>
|
|
</button>
|
|
<button class="dock-icon" data-drawer="synth" title="Synth controls">
|
|
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round">
|
|
<path d="M2 10c2-4 4-4 6 0s4-4 6 0"/>
|
|
</svg>
|
|
<span class="dock-label">Synth</span>
|
|
</button>
|
|
<button class="dock-icon" data-drawer="params" title="Engine parameters">
|
|
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round">
|
|
<circle cx="8" cy="8" r="3"/><path d="M8 1v2M8 13v2M1 8h2M13 8h2M3.05 3.05l1.41 1.41M11.54 11.54l1.41 1.41M3.05 12.95l1.41-1.41M11.54 4.46l1.41-1.41"/>
|
|
</svg>
|
|
<span class="dock-label">Engine</span>
|
|
</button>
|
|
<button class="dock-icon" data-drawer="help" title="Help">
|
|
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round">
|
|
<circle cx="8" cy="8" r="6"/><path d="M6 6a2 2 0 0 1 4 0c0 1.5-2 1.5-2 3"/><circle cx="8" cy="12" r="0.5" fill="currentColor"/>
|
|
</svg>
|
|
<span class="dock-label">Help</span>
|
|
</button>
|
|
</div>
|
|
|
|
<!-- Drawer panels (stack on right side, left of dock) -->
|
|
<div class="drawer-stack" id="drawer-stack">
|
|
<!-- Training drawer -->
|
|
<div class="drawer hidden" id="drawer-training" data-drawer="training">
|
|
<div class="drawer-header">
|
|
<span>Training</span>
|
|
<button class="drawer-close" data-drawer="training">×</button>
|
|
</div>
|
|
<div class="drawer-body">
|
|
<div class="action-row" id="examples-actions">
|
|
<button class="action-btn" id="btn-add-example">Add Example</button>
|
|
<button class="action-btn accent" id="btn-train">Train</button>
|
|
<button class="action-btn dim" id="btn-clear-examples">Clear Ex</button>
|
|
<button class="action-btn dim" id="btn-clear">Clear All</button>
|
|
<button class="action-btn dim" id="btn-randomize">Randomize</button>
|
|
</div>
|
|
<div class="preset-row" id="preset-row">
|
|
<span class="preset-label">Presets</span>
|
|
<button class="preset-chip" data-preset="calm-to-chaotic">Calm/Chaos</button>
|
|
<button class="preset-chip" data-preset="rainbow-sweep">Rainbow</button>
|
|
<button class="preset-chip" data-preset="vortex">Vortex</button>
|
|
<button class="preset-chip" data-preset="spiral">Spiral</button>
|
|
<button class="preset-chip" data-preset="embers">Embers</button>
|
|
</div>
|
|
<div class="loss-section">
|
|
<label>Loss History</label>
|
|
<canvas id="loss-canvas" width="280" height="80"></canvas>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Mode drawer -->
|
|
<div class="drawer hidden" id="drawer-mode" data-drawer="mode">
|
|
<div class="drawer-header">
|
|
<span>Mode</span>
|
|
<button class="drawer-close" data-drawer="mode">×</button>
|
|
</div>
|
|
<div class="drawer-body">
|
|
<div class="mode-group">
|
|
<span class="mode-label">Input</span>
|
|
<div class="pill-toggle pill-toggle-sm" id="input-toggle">
|
|
<button class="pill-opt active" data-input="joystick">Joystick</button>
|
|
<button class="pill-opt" data-input="hands">Hands</button>
|
|
</div>
|
|
</div>
|
|
<div class="mode-group">
|
|
<span class="mode-label">Output</span>
|
|
<div class="pill-toggle pill-toggle-sm" id="output-toggle-float">
|
|
<button class="pill-opt active" data-mode="visual">Visual</button>
|
|
<button class="pill-opt" data-mode="synth" id="synth-mode-btn">C15 Shaper-Feedback</button>
|
|
<button class="pill-opt" data-mode="midi-cc">MIDI CC</button>
|
|
<button class="pill-opt" data-mode="audio-canvas">Audio Canvas</button>
|
|
</div>
|
|
</div>
|
|
<div class="mode-group">
|
|
<button class="follow-pill" id="follow-pill" title="Toggle follow mode">Follow</button>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Synth drawer -->
|
|
<div class="drawer hidden" id="drawer-synth" data-drawer="synth">
|
|
<div class="drawer-header">
|
|
<span>Synth</span>
|
|
<button class="drawer-close" data-drawer="synth">×</button>
|
|
</div>
|
|
<div class="drawer-body" id="synth-panel">
|
|
<div class="synth-row">
|
|
<button class="action-btn accent" id="synth-start">Start Audio</button>
|
|
<span class="synth-status" id="synth-status">Stopped</span>
|
|
</div>
|
|
<div class="synth-row">
|
|
<label>Volume</label>
|
|
<input type="range" id="synth-volume" min="0" max="1" step="0.01" value="0.5">
|
|
</div>
|
|
<div class="synth-row">
|
|
<label>Arpeggiator</label>
|
|
<button class="action-btn" id="arp-toggle">Play</button>
|
|
</div>
|
|
<div class="synth-row">
|
|
<label>Progression</label>
|
|
<select id="arp-progression">
|
|
<option value="I-vi-IV-V">I-vi-IV-V</option>
|
|
<option value="I-IV-vi-V">I-IV-vi-V</option>
|
|
<option value="i-VI-III-VII">i-VI-III-VII</option>
|
|
<option value="I-V-vi-IV">I-V-vi-IV</option>
|
|
</select>
|
|
</div>
|
|
<div class="synth-row">
|
|
<label>Tempo <span id="tempo-val">120</span></label>
|
|
<input type="range" id="arp-tempo" min="40" max="240" step="1" value="120">
|
|
</div>
|
|
<div class="synth-row">
|
|
<label>Octaves <span id="octaves-val">2</span></label>
|
|
<input type="range" id="arp-octaves" min="1" max="5" step="1" value="2">
|
|
</div>
|
|
<div class="synth-row">
|
|
<label>Oct Offset <span id="offset-val">0</span></label>
|
|
<input type="range" id="arp-offset" min="-2" max="3" step="1" value="0">
|
|
</div>
|
|
<div class="synth-row" id="midi-row" style="display:none">
|
|
<label>MIDI Input</label>
|
|
<button class="action-btn" id="midi-toggle">Enable</button>
|
|
<select id="midi-select" style="flex:1;min-width:0"></select>
|
|
</div>
|
|
<div class="synth-row" id="midi-status-row" style="display:none">
|
|
<span class="synth-status" id="midi-status"></span>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- MIDI CC drawer (param management) -->
|
|
<div class="drawer hidden" id="drawer-midi-cc" data-drawer="midi-cc">
|
|
<div class="drawer-header">
|
|
<span>MIDI CC</span>
|
|
<button class="drawer-close" data-drawer="midi-cc">×</button>
|
|
</div>
|
|
<div class="drawer-body" id="midi-cc-panel">
|
|
<div class="synth-row">
|
|
<label>Preset</label>
|
|
<select id="midi-cc-preset-select" style="flex:1;min-width:0">
|
|
<option value="">Manual</option>
|
|
</select>
|
|
<button class="action-btn" id="midi-cc-file-import" title="Load preset from JSON file">File</button>
|
|
</div>
|
|
<div class="synth-row">
|
|
<label>Parameters</label>
|
|
<span id="midi-cc-count">8</span>
|
|
<button class="action-btn" id="midi-cc-add" title="Add CC parameter">+</button>
|
|
<button class="action-btn" id="midi-cc-remove" title="Remove last CC parameter">−</button>
|
|
</div>
|
|
<div class="synth-row">
|
|
<button class="action-btn" id="midi-cc-import">Paste JSON</button>
|
|
<button class="action-btn" id="midi-cc-export">Copy JSON</button>
|
|
</div>
|
|
<div id="midi-cc-param-list"></div>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Params drawer (NISPS engine tuning) -->
|
|
<div class="drawer hidden" id="drawer-params" data-drawer="params">
|
|
<div class="drawer-header">
|
|
<span>Engine</span>
|
|
<button class="drawer-close" data-drawer="params">×</button>
|
|
</div>
|
|
<div class="drawer-body">
|
|
<div class="engine-params" id="engine-params"></div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Help modal (unchanged) -->
|
|
<div class="help-overlay hidden" id="help-overlay">
|
|
<div class="help-modal" id="help-modal">
|
|
<button class="help-close" id="help-close">×</button>
|
|
<h2>Welcome to NISPS</h2>
|
|
<p class="help-subtitle">Neural Interactive Shaping of Parameter Spaces</p>
|
|
|
|
<div class="help-section">
|
|
<h3>What is this?</h3>
|
|
<p>NISPS lets you <strong>teach a neural network</strong> how to map a 2D joystick position to a rich set of visual or audio parameters. Move the joystick, shape the outputs you want, and the network learns your preferences in real time.</p>
|
|
</div>
|
|
|
|
<div class="help-section">
|
|
<h3>How it works</h3>
|
|
<p>A small neural network takes your joystick X/Y position as input and produces dozens of output parameters. You teach it by either:</p>
|
|
<ul>
|
|
<li><strong>Examples</strong> — set the parameter sliders to what you want at a given joystick position, add the example, then hit Train. Do this a few times from different positions and the network interpolates between them.</li>
|
|
<li><strong>Feedback (RL)</strong> — move the joystick around. If you like what you see/hear, press <strong>+</strong> (keep this). If you don't, press <strong>−</strong> (explore more). The network gradually learns what you prefer.</li>
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="help-section">
|
|
<h3>What to expect</h3>
|
|
<ul>
|
|
<li><strong>Moving the joystick</strong> changes all outputs in real time through the network</li>
|
|
<li><strong>Randomize</strong> shuffles the network weights — instant new mapping</li>
|
|
<li><strong>Train</strong> fits the network to your saved examples</li>
|
|
<li><strong>+ / − feedback</strong> nudges the network: + reinforces the current mapping, − adds exploration noise</li>
|
|
<li>Switch between <strong>Visual</strong> (particle flow field) and <strong>Synth</strong> (C15 synthesizer) output modes</li>
|
|
<li>The <strong>heatmap bar</strong> at the top shows all output parameters at a glance — drag any bar to set its value</li>
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="help-section">
|
|
<h3>Controls</h3>
|
|
<table class="help-keys">
|
|
<tr><th colspan="2">Touch / Mouse</th></tr>
|
|
<tr><td>Drag joystick</td><td>Move through parameter space</td></tr>
|
|
<tr><td>+ / − buttons</td><td>Positive / negative feedback</td></tr>
|
|
<tr><td>Undo (between +/−)</td><td>Revert last feedback action</td></tr>
|
|
<tr><td>Drag heatmap bar</td><td>Set parameter value directly</td></tr>
|
|
<tr><td>Right-side dock</td><td>Training, Mode, Synth, Params drawers</td></tr>
|
|
<tr><th colspan="2">Keyboard</th></tr>
|
|
<tr><td><kbd>1</kbd></td><td>Negative feedback (−)</td></tr>
|
|
<tr><td><kbd>2</kbd></td><td>Positive feedback (+)</td></tr>
|
|
<tr><td><kbd>Z</kbd></td><td>Undo</td></tr>
|
|
<tr><th colspan="2">Gamepad (Steam Deck, Xbox, etc.)</th></tr>
|
|
<tr><td>Left stick</td><td>Joystick control</td></tr>
|
|
<tr><td>LB (left bumper)</td><td>Negative feedback</td></tr>
|
|
<tr><td>RB (right bumper)</td><td>Positive feedback</td></tr>
|
|
<tr><td>A</td><td>Train</td></tr>
|
|
<tr><td>X</td><td>Randomize</td></tr>
|
|
<tr><td>B</td><td>Clear examples</td></tr>
|
|
</table>
|
|
</div>
|
|
|
|
<div class="help-section">
|
|
<h3>Hand Tracking</h3>
|
|
<p>Toggle to <strong>Hands</strong> mode in the Mode drawer to use your webcam for input.</p>
|
|
<ul>
|
|
<li><strong>Right hand</strong> controls parameters — palm position, finger curls, spread, rotation, and pinch map to 14 input dimensions</li>
|
|
<li><strong>Left hand</strong> gives feedback via gestures:</li>
|
|
<ul>
|
|
<li><strong>1 finger</strong> (index) held 0.4s → positive feedback (+)</li>
|
|
<li><strong>2 fingers</strong> (index + middle) held 0.4s → negative feedback (−)</li>
|
|
</ul>
|
|
<li>If only one hand is visible, it's treated as the tracking hand — use keyboard/buttons for feedback</li>
|
|
<li>Camera permission is requested only when you switch to Hands mode</li>
|
|
</ul>
|
|
</div>
|
|
|
|
<div class="help-section">
|
|
<h3>Synth Controls</h3>
|
|
<ul>
|
|
<li><strong>Presets</strong> — use the dropdown (top left, next to play) to choose a parameter preset. Lower tiers (1.x) expose fewer parameters for simpler exploration; higher tiers (3.x, 4.x) unlock cross-modulation, feedback, and the full engine. You can tweak any preset afterwards using the controls below.</li>
|
|
<li><strong>Hover any bar</strong> in the synth visualizer to see parameter name, current value, range, and curve</li>
|
|
<li><strong>Hover a group name</strong> (e.g. "Env A", "Osc B") at the top to open its control drawer:</li>
|
|
<ul>
|
|
<li><strong>Group curve</strong> — drag the top graph vertically to shape the response curve for the whole group</li>
|
|
<li><strong>Per-param curve</strong> — drag each small graph to shape that parameter individually</li>
|
|
<li><strong>Min/Max range</strong> — dual-thumb slider constrains the parameter's output range</li>
|
|
<li><strong>Mute (M)</strong> — removes the parameter from NISPS control</li>
|
|
</ul>
|
|
<li><strong>MIDI input</strong> — connect a MIDI controller and it will be detected automatically</li>
|
|
<li><strong>Arpeggiator</strong> — open the Synth drawer in the dock to access tempo, octave range, offset, and chord progression controls</li>
|
|
</ul>
|
|
</div>
|
|
|
|
<button class="help-got-it" id="help-got-it">Got it</button>
|
|
</div>
|
|
</div>
|
|
|
|
<script type="module" src="js/a-app.js"></script>
|
|
</body>
|
|
</html>
|