What is this?

NISPS lets you teach a neural network how to map a 2D joystick position to a rich set of visual or audio parameters. Move the joystick, shape the outputs you want, and the network learns your preferences in real time.

How it works

A small neural network takes your joystick X/Y position as input and produces dozens of output parameters. You teach it by either:

• Examples — set the parameter sliders to what you want at a given joystick position, add the example, then hit Train. Do this a few times from different positions and the network interpolates between them.
• Feedback (RL) — move the joystick around. If you like what you see/hear, press + (keep this). If you don't, press − (explore more). The network gradually learns what you prefer.

What to expect

• Moving the joystick changes all outputs in real time through the network
• Randomize shuffles the network weights — instant new mapping
• Train fits the network to your saved examples
• + / − feedback nudges the network: + reinforces the current mapping, − adds exploration noise
• Switch between Visual (particle flow field) and Synth (C15 synthesizer) output modes
• The heatmap bar at the top shows all output parameters at a glance

Controls

Hand Tracking

Toggle to Hands mode in the bottom bar to use your webcam for input.

• Right hand controls parameters — palm position, finger curls, spread, rotation, and pinch map to 14 input dimensions
• Left hand gives feedback via gestures:
• 1 finger (index) held 0.4s → positive feedback (+)
• 2 fingers (index + middle) held 0.4s → negative feedback (−)
• If only one hand is visible, it's treated as the tracking hand — use keyboard/buttons for feedback
• Camera permission is requested only when you switch to Hands mode

Tips

• Start with Randomize a few times to hear/see different mappings, then use feedback to refine
• Use Follow mode to let the joystick wander automatically while you give feedback
• In Synth mode, press the play button (top left, pulses orange when audio is off) to start audio, then explore
• Tap a heatmap cell to see which parameter it controls

Synth Controls

• Presets — use the dropdown (top left, next to play) to choose a parameter preset. Lower tiers (1.x) expose fewer parameters for simpler exploration; higher tiers (3.x, 4.x) unlock cross-modulation, feedback, and the full engine. You can tweak any preset afterwards using the controls below.
• Hover any bar in the synth visualizer to see parameter name, current value, range, and curve
• Hover a group name (e.g. "Env A", "Osc B") at the top to open its control drawer:
• Group curve — drag the top graph vertically to shape the response curve for the whole group (adjusts all param curves relatively)
• Per-param curve — drag each small graph to shape that parameter individually (down = exponential, up = logarithmic)
• Min/Max range — dual-thumb slider constrains the parameter's output range
• Mute (M) — removes the parameter from NISPS control; its bar disappears and you get a manual value slider instead
• MIDI input — connect a MIDI controller and it will be detected automatically; MIDI notes trigger the synth directly
• ShapeSeq — in synth mode, the sequencer UI appears: a circular step visualizer shows the current pattern; the chain builder below lets you add/remove/reorder algorithmic primitives; press Play to start the generative sequencer