p5.js in one line

Any p5.js sketch (OpenProcessing, the p5 editor, your own page) can be played from a MIDI controller, from a controller drawn on screen, or from a phone, by adding one line at the top:

document.head.prepend(Object.assign(document.createElement('script'), { src: 'https://openaudiovisual.com/p5/oav-p5.js' }));

On OpenProcessing you can instead paste the URL into Sketch → Libraries → + (custom library). Both work in p5 1.x and 2.x global mode, and nothing has to be called in setup(). A sketch that carries this line, playable right now: example 15.

Read it like the mouse#

function draw() {
  background(knob1 * 255);                 // knob 1, 0..1, like mouseX
  circle(width / 2, height / 2, 50 + knob2 * 300);
}

function notePressed(note, velocity) {     // like keyPressed(): any key, or a pad
  circle(random(width), random(height), velocity * 100);
}

Knob 1 is knob 1 on whichever controller played it: the controller profiles say which MIDI message each knob sends (MiniLab 3's knob 1 is CC 74, LPD8's is CC 70). On a controller OAV has no profile for, the first knob you turn becomes knob1, the next knob2. A sketch that already has its own note, velocity or knob1 keeps it.

Note names#

Wherever a note number goes, a note name goes too (0.8):

function draw() {
  if (noteIsDown('C4')) circle(width / 2, height / 2, 100);   // the same as noteIsDown(60)
}

function mousePressed() {
  midi.play('C#4', 0.8);                    // sends note 61, as a key would
}

A name is a letter A to G (either case), an optional # or b (♯ and ♭ too), and the octave number: C4, c#4, Db3, F♯2, C-1 (0) up to G9 (127). C4 is 60, the scientific pitch notation the course modules use; some software (Ableton, Yamaha) calls 60 "C3", so a name copied from there is one octave off. These are also the names Strudel writes (c4 e4 g4, # for sharp, b for flat). Anything else, such as H4 or C-2, stops with a one-line card saying how to write it. notePressed(note, velocity) still hands you the number: noteName(note) writes it back as a name.

With OP Configurator#

OP Configurator 3000 (OPC, built into OpenProcessing) names a variable and its range. With this line on the page the controller is wired to those variables in order, with no mapping step:

OPC.slider('radius', 60, 10, 300);   // knob 1
OPC.slider('speed', 1, 0, 5, 0.1);   // knob 2
OPC.toggle('trails', true);          // pad 1 flips it

Knobs drive the sliders in the order they are declared, the faders continue after the knobs, and pads drive toggles and buttons. OPC's panel moves with the knob, and parameterChanged() fires as usual. OpenProcessing's OPC panel can also learn hardware by itself (its MIDI switch); once that mapping is seen working, hardware is left to it and this line keeps wiring the on-screen controller and the phone, which OpenProcessing cannot see. Avoid OPC names that are also p5 functions (hue, fill): p5 1.x clashes with them.

One knob, one job. If the sketch also reads knob1 itself, knob 1 is the sketch's: the first OPC slider no longer follows it (drag it on OPC's panel), the other knobs stay wired, and a hint says so once. The first line of the 🎹 panel is the wiring, for example knob1 → knob1 (sketch; radius by its slider) · knob2 → speed (OPC).

The microphone#

The same line reads the microphone, and the sketch reads it the same way: as variables.

function draw() {
  background(230, 40, 10);
  fill(200 + bass * 160, 70, 100);                      // bass: the low end, 0..1
  circle(width / 2, height / 2, 50 + micLevel * 300);   // micLevel: how loud, 0..1
}

function beatDetected(level) {                          // like keyPressed(): a beat just landed
  // flash, spawn, jump to the next scene
}

It is off until the sketch asks. Nothing is requested when the page loads. The first time the sketch reads micLevel, bass, mid, treble or beat, or calls startMic(), the browser asks for the microphone; until the answer arrives every value is 0. A sketch that defines beatDetected() and reads nothing starts it on the first click or key. In the 🎹 panel the 🎤 button turns it on and off by hand and shows the level, and it says so when the browser blocked the microphone (click it again after allowing it from the address bar).

• Click first, if the browser wants one. Browsers keep audio quiet until the page has been clicked. This file asks as soon as draw() reads a value and starts the microphone again on the first click or key if the browser held it back, so no userStartAudio() is needed.
• Next to p5.sound. The analysis is @openav/audio (an AnalyserNode), loaded from next to this file the first time it is needed, with its own AudioContext. p5.sound (p5.AudioIn, p5.FFT) can run on the same page; the two microphones do not share anything, and none of these names is a p5 or p5.sound name. It works in p5 1.x and 2.x global mode.
• How loud is loud. micLevel is a loudness in decibels, measured from the room: its bottom follows the room's own noise (it settles in a second or two, and rises at most 1.5 dB a second, so a voice never becomes the new silence), and −10 dBFS, shouting into a laptop, is 1. So 50 + micLevel * 300 visibly grows when someone talks. A quiet or far microphone: micSensitivity(3) (+9.5 dB on micLevel, ×3 on the bands, still capped at 1). The raw numbers are OAVp5.mic.rms (0..1), OAVp5.mic.db and OAVp5.mic.floor (the room, in dBFS). OAVp5.mic.on('beat', fn) is the same as defining beatDetected, and returns a function that stops it.
• Your own names win. A sketch that already has a bass or mid (a let, a function, a library) keeps it, and this file never starts the microphone for it.
• Off switch. ?mic=click (or window.OAV_P5 = { mic: 'click' }) means only 🎤 and startMic() turn it on; mic=off means nothing can start it (the names stay 0). https or localhost is required, as for any microphone page.
• A sketch with a microphone, ready to open: p5/examples/mic/.

Sound: what is played is heard#

function setup() {
  createCanvas(windowWidth, windowHeight);
  startSound('piano');          // every key is heard: a keyboard, the 🎹 on screen, a phone
}

The instruments are those of @openav/sound: piano (the Salamander grand; its 2 MB of samples load the first time, and the electric piano plays meanwhile, or instead if they cannot load), epiano, organ, strings, choir, harp, marimba, vibraphone, xylophone, glockenspiel, music-box and the synth/… presets. In the 🎹 panel, 🔊 turns sound on and off and shows a picker with every instrument and the loading progress.

• Off until asked. Nothing loads before startSound() or 🔊; then @openav/sound from next to this file and Tone.js 15 from jsDelivr (the option tone=<url> loads it from elsewhere).
• The first click. Browsers play audio only after the page has been clicked, and a key on a MIDI keyboard does not count. A sketch that calls startSound() in setup() is heard from the first click or key; until then the 🔊 on the 🎹 button is yellow.
• Pads are drums. Notes on MIDI channel 10 (MiniLab 3 and LPD8 pads, the General MIDI drum channel) play the drum kit, not the piano. The sustain pedal holds notes.
• No code at all. ?oav_sound=piano on the page (any instrument id; on OpenProcessing ?sketch_oav_sound=piano) makes any sketch heard from the first click; sound=off keeps it silent.
• A sketch that plays and draws, ready to open: p5/examples/keys/.

Drum machine#

function setup() {
  startDrums('breakbeat', 110);   // a 16-step grid on screen; click a cell to change the beat
}

function kickDetected(level) {    // the microphone's own callback
  // swell, flash, spawn
}

function drumHit(lane, velocity) {   // every hit: 'kick', 'snare', 'hat' or 'clap'
}

A microphone sketch moves over unchanged. Each hit calls the same functions the microphone does, so a sketch written for the microphone reacts to the machine as it is. Once a sketch has started the machine, defining kickDetected() no longer opens the microphone on the first click (a sketch that reads micLevel still asks, and then both call it).

• Patterns. 'four on floor', breakbeat, 'half time', latin, sparse, your own startDrums({ kick: 'x...x...x...x...', snare: '....x.......x...', hat: 'x.x.x.x.x.x.x.x.' }) (a lane left out is silent), or one line of rhythm (below).
• The card. The lanes are named in the page's language (大鼓 小鼓 鈸 拍手 in a Chinese browser), the bar it is in shows beside the tempo, and the step that sounds is a yellow column (13:1 against the dark card, readable on a projector).
• Where the grid sits. At the top centre: on OpenProcessing between the 🎹 and OPC's icon and clear of the console, and below the 🎹 panel while that is open. – folds it, × stops the machine, and 🥁 in the 🎹 panel opens and closes it with no code at all. 🥁 brings the grid up stopped: tick its play box to hear it (a sketch's own startDrums() plays at once).
• The kit. @openav/drums' synthesized kit (no samples), heard from the first click like sound; the callbacks run from the start either way.

On time, however heavy the sketch. The hits are handed to the kit ahead of time, on its audio clock: a heartbeat in a Web Worker wakes every 25 ms and schedules what falls in the next 0.1 s at its exact time. Measured on the kit's own output with a draw() that keeps the page busy 30 ms a frame, the kick-to-kick interval varies by 0.000 ms (0.5, which played each hit in the frame it was due, varied by 15 ms). The callbacks still come in a frame: the one in which the hit is heard (its time plus the output latency the browser reports), right before draw(), in the same order as before.

• A tab in the background keeps playing and counting: drumBar follows the clock on the wall and the cues fire, with one barStarted a bar. The hits and stepStarted are not told while nobody can see them, and nothing is told twice when the tab comes back, so a sketch that adds a particle per hit does not burst.
• A draw() longer than 0.1 s makes the machine look 0.3 s ahead from then on (🩺 says so); a click on a cell is then heard up to 0.3 s later.
• A Bluetooth speaker is late by 0.15–0.25 s; where the browser reports it the callbacks wait for it, and ?oav_drumlatency=200 (milliseconds) sets it by hand.
• The safety net: ?oav_drums=frame (on OpenProcessing ?sketch_oav_drums=frame) brings back 0.5's machine, moved by the frames.

One line of rhythm#

function setup() {
  startDrums("bd*4, [~ sd]*2, hh*8", 110);   // a kick on every beat, a snare on 2 and 4, eight hats
}

Any text that is not a pattern name is a line of rhythm; names, grids and numbers work exactly as before. The line shows over the grid, and the word that sounds lights up and fades.

The whole list is in @openav/pattern, loaded from next to oav-p5.js with the drums. Drums the grid has no lane for (crash, ride, toms, open hat) get a row of their own; a hit off the sixteen steps (a triplet) is drawn where it falls inside its step. A mistake in startDrums("…") shows a card in the page's language with the character it is at, why, and how to fix it, and the machine plays 'four on floor' meanwhile; 'x...x...' on its own (one lane of the grid) says to put it inside { kick: … }.

A line copied from strudel.cc works too: startDrums('s("bd*4, [~ sd]*2").fast(2)') plays [bd*4, [~ sd]*2]*2. s("…"), sound("…"), with ", ' or ` quotes, several of them ($: s(…)) as layers, .fast(n) and .slow(n); setcpm(110/4) or setcps(…) sets the tempo when startDrums() is given none. .bank("…") is left out (OAV plays its own synthesized kit) and anything else in the chain (.jux(rev), .every(…), .gain(…)) gets a card that says OAV does not do it yet and points at Open in Strudel; the rest plays.

The way of writing one line of rhythm comes from TidalCycles / Strudel (strudel.cc); OAV implements it independently from their public documentation.

The cells and the line edit each other#

Since 0.7 the line over the grid is always there (a preset shows the line its grid is: 'four on floor' is bd*4, [~ sd]*2, hh*8) and both ways work:

• Click a cell: the line is rewritten the way a person would write it, and the machine plays the new line at once. Steady hits become bd*4 / hh*8, one hit per group [~ sd]*2, a Euclidean rhythm bd(3,8) (with its rotation: bd(3,8,2) starts two steps later), anything else beat by beat: bd [~ ~ ~ bd] bd*2 ~. Lines are joined with , . When the line holds more than sixteen steps can (triplets, < >, /2, ?, bd:2), the first click shows a card saying what would go, and the next click rewrites it.
• Type in the line: click it, change it; Enter, or a pause of 400 ms, plays it and the cells follow. A mistake marks the word in red, a card says what is wrong, and the last line that worked keeps playing (not 'four on floor'). Escape drops what is not played yet. Typed keys never reach the sketch's keyPressed().
• ↶ on the card (or ⌘Z / Ctrl+Z after touching the card) goes back one line; OAVp5.drums.history lists them.
• Copy puts startDrums("…", 110) on the clipboard, ready to paste into setup().
• Open in Strudel opens the same line at strudel.cc in a new tab, as s("…") with setcpm(bpm/4) (one OAV bar is one Strudel cycle of four beats; strudel.cc's docs: setcpm(bpm/bpc)). OAVp5.drums.strudelUrl is the link.

function rhythmChanged(src, how) {   // how: 'code' · 'cell' · 'type' · 'undo' · 'menu'
  line = src;                        // e.g. draw the line on the canvas, so a recording shows it
}

OAVp5.drums.src is the line that plays, OAVp5.drums.edit("…") plays a line as if typed, OAVp5.drums.undo() goes back. Keys that swap the rhythm, step back and open Strudel: p5/examples/rhythm-lab/.

The rhythm on the canvas#

The card is part of the page, not the canvas, so a recording never shows it. drawRhythm() draws the rhythm on the canvas:

function draw() {
  background(0);
  drawRhythm(20, 20, 360, 90, 'grid');          // the sixteen steps, a playhead
  drawRhythm(20, 130, 300, 300, 'circle');      // a ring of 16 per drum, a hand turned by drumPhase
  drawRhythm(340, 130, 300, 300, 'spiral', { bars: 4, colors: { kick: '#ff7b54' } });   // the last 4 bars, wound outwards
}

drawRhythm(x, y, w, h, style, options): style is 'grid', 'circle' or 'spiral'; a hit lights up the moment it is heard (the clock drumPhase reads) and a triplet sits where it falls. Options: colors ({ kick: '#f66', … }, any p5 color), color (one for all), off (the empty steps), playhead (a color, or false), bg, bars (spiral). Drawn inside push() / pop() with p5's own shapes, in 2D and WEBGL, p5 1.x and 2.x; in WEBGL (0, 0) is the middle of the canvas: drawRhythm(-width / 2, -height / 2, width, height, 'circle'). With no box it draws a strip along the bottom ('grid') or a square in the middle. A sketch with all three: p5/examples/rhythm-visual/.

Timeline: bars and cues#

The drum machine is also the clock: sixteen steps a bar, counted from startDrums(). Declare what happens at which bar:

function setup() {
  startDrums('breakbeat', 110);
  cue(1, 'calm', 12);             // bar 1, and again every 12 bars
  cue(5, 'bloom', 12);
  cue(9, 'storm', 12);
}

function cueFired(name, bar) {    // or read cueName in draw()
  look = SCENES[name];
}

The same cue() twice adds nothing, so it may also sit in draw(). On one step the cues fire in the order they were declared, then barStarted, stepStarted and that step's hits. A silent clock: startDrums({}). The whole of it, ready to open: p5/examples/drums/; a line of rhythm with drumPhase and beatPhase: p5/examples/rhythm/; the rhythm drawn on the canvas: p5/examples/rhythm-visual/.

Values over time: lag, env, speed, spring, trail#

A knob or micLevel jumps; what looks good is how it moves between the jumps. TouchDesigner does this with CHOPs; the same operators are here, one call each in draw(), nothing to declare, and seconds are seconds (the same curve at 30 and at 144 fps):

circle(width / 2, height / 2, 50 + lag(micLevel, 0.03, 0.4) * 300);   // a VU meter: opens at once, closes slowly
scale(1 + env('kick', 0.01, 0.3));                                     // the drum machine: every kick swells it, gone in 0.3 s
rotate(spring(knob1, 100, 6) * TWO_PI);                                // flick a knob: it overshoots and settles

Seconds, the same as in TouchDesigner. A lag's seconds mean what a Lag CHOP's do: about the time the value takes to follow 90% of a jump, so lag(x, 0.1) here moves like a Lag CHOP set to 0.1 (Lag CHOP). Underneath it is a one-pole lag with a time constant of seconds ÷ 2.3 (ln 10); @openav/mapping's smooth keeps its own 63% time constant. Attack and decay are lengths in both.

Nothing to declare. The first lag() in draw() keeps the first lag's memory, the second the second, every frame; spring(), speed(), trail() … are counted each on their own. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. The sure way is a name, a string at the end: lag(p.size, 0.1, 0.5, 'p' + p.id). A named one is the same wherever it is called, draw(), setup() or mousePressed(); an unnamed one outside draw() has nothing to remember by and returns its input (speed 0, trail []), and a card says that too. env(), trigger() and count('kick') are about events and work anywhere. A sketch with its own speed or count keeps it; OAVp5.chop.speed(…) is the same function under another name.

Seeing the values. scope(value, 'label') returns value, so it can wrap an expression: circle(x, y, scope(lag(micLevel, 0.03, 0.4), 'lag') * 300). Each label is one line in a small oscilloscope across the page from the 🎹: the last 4 seconds and the value now; scope(micLevel, 'mic') beside it shows what the lag did. It lives on the page, not on the canvas: never in a recording or a 📷, never in the way of a click; hints=off or scope=off hides it. To draw it in the work: stroke(255); drawTrail(trail(micLevel, 3), 0, height - 100, width, 100) (or drawTrail(micLevel, …), which keeps its own 2 seconds): 0 at the bottom of the box, 1 at the top (lo, hi after h), the sketch's stroke() (white under noStroke()).

The math is @openav/chop, loaded from next to this file the first time one of these is called; until it arrives (a few frames) each returns its input as it is. What each parameter means follows Derivative's TouchDesigner documentation; the code is OAV's own. Raw against lagged, side by side, with the scope: p5/examples/lag/. env, spring, speed, count and slope driving a disc that a bass drum opens, with startDrums() as the only input: p5/examples/chop/.

Recording: a video with its sound#

function keyPressed() {
  if (key === 'r') recordingIsOn ? stopRecording() : startRecording();   // startRecording(10): ten seconds, then it stops
}

function recordingSaved(info) {   // info.name, info.seconds, info.bytes, info.format ('mp4' or 'webm')
}

In the 🎹 panel, ⏺ starts and stops a take with no code; while it runs, the 🎹 button shows a red dot and the time (a click on the dot stops it). 📷 saves the canvas as a PNG.

• What is in the video. The p5 canvas at its own pixel size (a retina screen doubles it), 30 frames a second, and every sound this file makes: startSound(), the drum machine and the pads, the microphone while it is on, and p5.sound 1.x, mixed into one track while the speakers play as before. The 🎹, the drum grid, the cards and OPC's panel are not on the canvas, so they are not in the video.
• The file. MP4 (H.264 + AAC), which Instagram, YouTube and every editor take, wherever the browser can write it: Chrome 126 and later, Edge, Safari. Elsewhere (Firefox, an older Chrome) WebM, and a card says Instagram will not take it. It is named oav-20261015-183005.mp4; on a computer it lands in Downloads, on a phone a Share button on the card saves it to the photo library or sends it to Instagram.
• Checked after every take. A card says what was recorded (format, length, size, with or without sound) and, when something is wrong, what and why: nothing reached the recording although a sound was on (no click on the page yet, nothing played), WebM, an almost empty file, a black video of a canvas that was not black (Safari recording a WebGL canvas, WebKit 229611), an image from another site on the canvas (no CORS), no recorder in this browser.
• The microphone is in the take while it is on: wear headphones, or the speakers are recorded twice. record=nomic leaves it out.
• p5.sound. p5.sound 1.x (the one for p5 1.x) is recorded. The p5.sound made for p5 2.x connects each sound straight to the speakers, with no output a recording can tap: it is not in the take, and a card says so. Switch the sketch to p5 1.11, or use startSound() and startDrums(). Sound the sketch makes with its own Tone.js or Web Audio code is not recorded either.
• Stills. 📷 (or OAVp5.record.shot()) saves the canvas alone as oav-<date>-<time>.png, without the 🎹 or the cards; from code, p5's own saveCanvas('name', 'png') does the same.
• Options. record=nomic (no microphone) or record=off (no ⏺ or 📷; startRecording() does nothing), recfps=60, reclimit=600 (seconds), title=my-piece (in the file name); for one take, startRecording({ seconds, fps, mic: false, name, canvas, bitrate }).
• The whole of it, ready to open: p5/examples/record/.

Three inputs#

All three arrive as the same MIDI bytes, so the sketch cannot tell them apart (midi.last.source can). Safari and iPad have no Web MIDI; the on-screen controller and the phone still work there. A sketch that uses Web MIDI itself (navigator.requestMIDIAccess, WebMidi.js) also hears the screen and the phone, as one more input called OAV Virtual MIDI.

Other programs are a fourth input (0.7). What a program plays into OAV Virtual MIDI's output — Strudel's .midi('OAV Virtual MIDI') — reaches the sketch at the moment it was stamped with (midi.last.source is virtual-out); an external MIDI clock moves drumPhase, drumBar and a drum machine; and link=<room> brings another tab's or another computer's programs into this sketch. All of it: With Strudel.

When something goes wrong#

Nobody watches OpenProcessing's console while playing, so problems are said on the page: a small card beside the 🎹, never over the middle of the canvas, in the browser's language, once per page, with a × to close it. Each card says what happened, why, and what to press, and the console gets the same sentence.

• The microphone: refused (the lock left of the address → allow → click the canvas), refused by the computer's settings, by an embed without allow="microphone", missing, held by another app (Zoom, OBS), an http: page, or the browser still asking.
• MIDI: no Web MIDI (Safari, iPad: use the 🎹 or a phone), or the permission refused. Only for a sketch that plays from MIDI; a sketch that uses none of this never sees a card.
• Your own code: an error inside notePressed(), kickDetected(), drumHit() and the other callbacks shows the message with the file and line, and the sketch keeps running. A sketch that declares its own note or knob1 (let, var, function) keeps it, and the card says what to read instead (midi.last.note, knob(1)).
• Sound and drums: waiting for the first click, the piano still loading (the electric piano meanwhile), an instrument that failed (the fallback plays), Tone.js unreachable, the drum machine playing silently until a click.
• Recording: started (what it hears, how to stop), saved (format, length, size, sound), a take with no sound in it, WebM (not for Instagram), a black take, an almost empty file, a canvas with another site's image, no recorder in this browser.

?oav_hints=off (or window.OAV_P5 = { hints: 'off' }) keeps them in the console only; OAVp5.hints lists what was said.

🩺 Check-up. 🩺 in the 🎹 panel (or OAVp5.check()) shows the version, the page, the browser, the microphone, MIDI, sound, drums, recording (MP4 or WebM here, the last take), the wiring, the sketch's own names and the hints so far, as one card to screenshot, with a copy button and a link to the full device check.

In Hydra#

Hydra has a loader of its own; the same file works there, and Hydra's arrow functions read the variables each frame:

await loadScript('https://openaudiovisual.com/p5/oav-p5.js')
osc(10, 0.1, () => knob1 * 2).out()                     // knob 1 sets the colour offset

osc(10, 0.1, () => 1 + micLevel * 3).out()              // the microphone, asked for when this line first runs

note, velocity, knob1 … knob8, fader1 …, pad1 …, drumStep and the 🎹 on-screen controller are there the same way. Hydra is not p5, so there are no notePressed() callbacks to define; subscribe instead: midi.on('notePressed', (note, velocity) => { … }) or OAVp5.drums.on('hit', (lane, velocity) => { … }) (each returns a function that unsubscribes). Checked against hydra-synth 1.4.0, the library the editor runs (p5/test/hydra-smoke.cjs).

Notes#

• Why prepend. p5's friendly-error checker reads the last <script> on the page as your sketch. Put first, this file is never mistaken for your code.
• Why one asynchronous line is enough. p5 starts global mode on the window's load event, and a script inserted while the page is loading holds load back until it has run, so knob1 already exists in setup().
• When the line does not load (offline, a typo in the URL), code that reads knob1 stops with knob1 is not defined.
• Embedding a sketch elsewhere. OpenProcessing's own pages allow MIDI and the microphone in the sketch frame. The embed code it gives you does not: add allow="midi" to that <iframe> if the embedded sketch should hear a hardware controller, and allow="microphone" if it should hear the microphone.
• Options, MIDI-level access and tests: p5/README.md. Examples: p5/examples/particles/, p5/examples/opc/, p5/examples/mic/, p5/examples/keys/, p5/examples/drums/, p5/examples/rhythm/, p5/examples/rhythm-visual/, p5/examples/rhythm-lab/, p5/examples/record/, p5/examples/lag/, p5/examples/chop/, p5/examples/with-strudel/ and p5/examples/note-names/.