Every variable, function, event and option of oav-p5, and every export of every package. Names, signatures and source links are read from the code on each build; what is written about them is checked against the code in CI, so a name cannot be added, renamed or removed without this page noticing.
You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.
Declare it the way you declare keyPressed():
js
function controlChange(cc, value, channel) {
// a control moved
}
It hears every control, including the ones that are not a knob or a fader, such as the mod wheel and the pedals, which knobMoved() never reports. The latest value of every controller number is also in OAVp5.cc, so a sketch that only wants to look can read OAVp5.cc[1] in draw().
For one incoming message the order is: midiMessage(), then the specific one (noteOn(), controlChange(), pitchBend()), then the friendly one (padPressed(), knobMoved(), notePressed() and their partners).
Before it has a value
Defining the function is enough. oav-p5 asks the browser for Web MIDI when it loads (the default, ?oav_hardware=auto), and a sketch that defines it also gets a card if Web MIDI is missing or blocked. Keys from 🎹, a phone, OAVp5.input() and other programs reach it too. OAVp5.on('controlChange', fn) does the same without defining a global function, and returns a function that stops it.
Parameters
Name
Type
Description
cc
Number
The controller number, 0 to 127 (1 is the mod wheel, 64 the sustain pedal, 74 the first knob on a MiniLab 3).
value
Number
Where the control is now, 0 to 1 (MIDI's 0 to 127 divided by 127).
channel
Number
The MIDI channel, 1 to 16.
Notes
An error inside it does not stop the music: the next message is still handled, and the message shows in the console and on a card beside the 🎹, with the file and line.
let shown = 'move a knob, the mod wheel or a pedal';
function setup() {
createCanvas(400, 300);
textAlign(CENTER, CENTER);
textSize(24);
}
function controlChange(cc, value, channel) {
shown = 'CC ' + cc + ' = ' + nf(value, 1, 2) + ' (channel ' + channel + ')';
}
function draw() {
background(20);
fill(255);
text(shown, width / 2, height / 2);
}
Shows the controller number, its value and the channel of whatever you moved last. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
let sustain = false;
function setup() {
createCanvas(400, 300);
}
function controlChange(cc, value) {
if (cc === 64) sustain = value >= 0.5; // the sustain pedal: 64 and up counts as down
}
function mousePressed() {
OAVp5.input([0xB0, 64, 127]); // no pedal? press the mouse
}
function mouseReleased() {
OAVp5.input([0xB0, 64, 0]);
}
function draw() {
background(sustain ? 90 : 20);
}
The background lights up while the sustain pedal is down. No pedal? Hold the mouse button: it sends the same message.
Related
knobMoved()You define this function; oav-p5 calls it whenever a knob moves.
faderMoved()You define this function; oav-p5 calls it whenever a fader moves.
OAVp5.ccAn array of 128 numbers, one per MIDI controller number: where each control last was, 0 to 1. OAVp5.cc[1] is the mod wheel, OAVp5.cc[64] the sustain pedal.
midiMessage()You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
OAVp5.on()Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
OAVp5.input()Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
The position of a controller fader (slider), from 0 to 1: fader1 is the first fader, up to fader8.
Works exactly like knob1, for the controller's faders, numbered in the order the controller's profile lists them (as printed on the device). A controller with only four faders, like the MiniLab 3, has fader1 to fader4.
Type
Number
Range
0..1
Changes when the fader moves; 0 until it has moved.
Before it has a value
Nothing to call. Web MIDI is asked for at load, exactly as for a knob (knob1); reading a fader also makes a missing or refused Web MIDI show as a card.
let level = 0;
function setup() {
createCanvas(400, 300);
}
function faderMoved(n, value) {
if (n === 1) level = value; // fader 1 only
}
function draw() {
background(20);
fill(255);
rect(40, height - 40, level * (width - 80), 24);
}
A bar that follows fader 1, driven by the event instead of a read.
Related
fader1The position of a controller fader (slider), from 0 to 1: fader1 is the first fader, up to fader8.
knobMoved()You define this function; oav-p5 calls it whenever a knob moves.
controlChange()You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.
Returns the position of knob number n, from 0 to 1: the same as knob1 to knob8, for any n.
Use it when the number of the knob is a variable: for (let i = 1; i <= 16; i++) bar(i, knob(i));. It also works in a sketch that has its own knob1, which oav-p5 then leaves alone.
Parameters
Name
Type
Description
n
Number
Which knob, counting from 1 (as printed on the controller).
Returns
Number The knob's position, 0..1; 0 until it has been turned.
The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
Read it in draw() the way you read mouseX: background(knob1 * 255). Knobs are numbered the way they are printed on the controller, so knob 1 is knob 1 on any controller OAV has a profile for: the MiniLab 3's first knob is CC 74, the LPD8's is CC 70. The sketch never needs to know a CC number.
On a controller OAV has no profile for, the first knob you turn becomes knob1, the next one knob2, and so on, in the order you turn them. Past eight, use knob(n).
Type
Number
Range
0..1
Changes when the knob moves, not once per frame; the sketch just reads the latest value. 0 until the knob has been turned.
Before it has a value
Nothing to call. With the default hardware=auto, oav-p5 asks the browser for Web MIDI as soon as it loads, whatever the sketch reads (Chrome, Edge and Firefox have it; Safari and iPad do not: use 🎹 or a phone there). Reading a knob also tells oav-p5 the sketch plays from MIDI, so when Web MIDI is missing or refused a card beside the 🎹 says so; a sketch that uses none of this never sees that card. ?oav_hardware=click waits for the 🎹 panel instead.
Notes
A sketch that already has its own knob1 (a let, a function) keeps it, and a card beside the 🎹 says what to read instead (knob(1)).
With OP Configurator, knobs also drive the OPC sliders in the order they are declared. A knob the sketch reads itself stays the sketch's: that slider no longer follows it.
let last = 'turn a knob';
function setup() {
createCanvas(400, 300);
textAlign(CENTER, CENTER);
textSize(32);
}
function knobMoved(n, value) {
last = 'knob ' + n + ': ' + nf(value, 1, 2);
}
function draw() {
background(20);
fill(255);
text(last, width / 2, height / 2);
}
Shows which knob moved and where it is. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
Related
knob1The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
faderMoved()You define this function; oav-p5 calls it whenever a fader moves.
controlChange()You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.
function setup() {
createCanvas(400, 300);
noStroke();
}
function draw() {
background(20);
const f = frameCount - lastNoteFrameCount; // frames since a key went down
fill(120, 190, 255);
circle(200, 150, max(0, 200 - f * 5)); // shrinks by 5 px a frame
}
Each key you press (on 🎹 too) brings back a big circle that shrinks 5 px a frame.
Related
lastNoteTsWhen the last key went down, on the millis() clock: millis() - lastNoteTs is how many ms ago.
noteThe MIDI number (0 to 127) of the last key pressed.
notePressed()You define this function; oav-p5 calls it each time a key (or a pad) goes down.
lastKickFrameCountThe frameCount of the frame that saw the last kick (snare, hat, beat): frameCount - lastKickFrameCount is how many frames ago.
You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
Declare it the way you declare keyPressed():
js
function midiMessage(bytes, source) {
// anything that came in
}
It is the widest net: it is called first for every note, control change and pitch bend, and it is the only function that hears the messages oav-p5 has no friendlier name for, such as channel pressure (0xD0), polyphonic aftertouch (0xA0) and program change (0xC0). The OAVp5.last object keeps the latest note, control change or pitch bend, but not those.
It does not hear system messages: the MIDI clock, start, stop and song position go to OAVp5.clock, and system-exclusive messages are dropped. A note-on with velocity 0 arrives here as it was sent (a note-on with 0), while noteOff() is the one that treats it as a release.
For one incoming message the order is: midiMessage(), then the specific one (noteOn(), controlChange(), pitchBend()), then the friendly one (padPressed(), knobMoved(), notePressed() and their partners).
Before it has a value
Defining the function is enough. oav-p5 asks the browser for Web MIDI when it loads (the default, ?oav_hardware=auto), and a sketch that defines it also gets a card if Web MIDI is missing or blocked. Keys from 🎹, a phone, OAVp5.input() and other programs reach it too. OAVp5.on('midiMessage', fn) does the same without defining a global function, and returns a function that stops it.
Parameters
Name
Type
Description
bytes
Array<Number>
The message as three numbers [status, data1, data2]. The status byte holds the kind in its top half and the channel in its bottom half (0x90 is a note-on on channel 1, 0x93 on channel 4). The data bytes are 0 to 127, and a byte the message does not have is 0.
source
String
Where it came from: 'hardware' (a real controller), 'screen' (the 🎹), 'phone', 'virtual-out' (a program playing into OAV Virtual MIDI), 'link' (another page in the same room), or whatever label you gave OAVp5.input().
Notes
An error inside it does not stop the music: the next message is still handled, and the message shows in the console and on a card beside the 🎹, with the file and line.
let pressure = 0;
function setup() {
createCanvas(400, 300);
}
function midiMessage(bytes) {
if ((bytes[0] & 0xF0) === 0xD0) pressure = bytes[1] / 127; // channel pressure (aftertouch)
}
function mouseMoved() {
OAVp5.input([0xD0, constrain(round(mouseX / width * 127), 0, 127)], 'mouse'); // no aftertouch? move the mouse
}
function draw() {
background(20);
circle(width / 2, height / 2, 40 + pressure * 260);
}
The circle follows aftertouch, which no friendlier function reports. No keyboard with aftertouch? Move the mouse across the canvas: it sends the same message.
Related
noteOn()You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
controlChange()You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.
pitchBend()You define this function; oav-p5 calls it whenever the pitch wheel moves, with its position from −1 to 1.
OAVp5.lastThe latest note, control change or pitch bend that arrived: its type, number, value, channel, where it came from and which control it is. null before any.
OAVp5.on()Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
OAVp5.input()Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
The MIDI number (0 to 127) of the last key pressed.
Middle C is 60, A4 is 69. noteName(note) writes the number as a name (60 → 'C4') and noteFreq(note) as a frequency in Hz. velocity says how hard it was pressed. For every key, not just the last, define notePressed().
Type
Number
Range
0..127
Set when a key goes down and kept after it is released; 0 before any key has been pressed.
Before it has a value
Nothing to call. Web MIDI is asked for at load (see knob1). Unlike a knob, merely reading note does not tell oav-p5 the sketch plays from MIDI: defining notePressed() or reading a knob or pad does, which is what makes a missing or refused Web MIDI show as a card. 🎹 and a phone always work.
Notes
A sketch with its own note (a let note, a function) keeps it; read midi.last.note instead. oav-p5 says so in a card the first time.
function setup() {
createCanvas(400, 300);
textAlign(CENTER, CENTER);
textSize(64);
}
function draw() {
background(20);
fill(255);
text(noteName(note), width / 2, height / 2); // the last key, as a name
}
Shows the name of the last key you played. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
Related
velocityHow hard the last key was pressed, from 0 to 1.
noteIsPressedTrue while at least one key is held down.
notePressed()You define this function; oav-p5 calls it each time a key (or a pad) goes down.
noteName()Turns a MIDI number into a note name: noteName(61) is 'C#4'.
True while the key n (a MIDI number or a name like 'C4') is held down.
The MIDI version of keyIsDown(). noteIsDown('C4') is the same as noteIsDown(60): C4 is middle C. A name that is not a note ('H4', 'C-2') stops the sketch with a one-line card saying how to write it.
Parameters
Name
Type
Description
n
Number | String
The key: a MIDI number (60) or a note name ('C4', 'c#4', 'Db3').
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
fill(noteIsDown('C4') ? color(255, 211, 107) : 80);
circle(width / 2, height / 2, 140); // lights up while middle C is held
}
A circle that lights up only while C4 (middle C) is held. On the on-screen keyboard, middle C is the C labelled C4.
Related
noteIsPressedTrue while at least one key is held down.
noteNumber()Turns a note name like 'C4' into its MIDI number, 0 to 127.
notePressed()You define this function; oav-p5 calls it each time a key (or a pad) goes down.
function noteOff(note, channel) {
// a key or pad came up
}
It is called for a note-off message and also for a note-on with velocity 0, which many keyboards send instead. There is no velocity: oav-p5 does not pass the release velocity.
For one incoming message the order is: midiMessage(), then the specific one (noteOn(), controlChange(), pitchBend()), then the friendly one (padPressed(), knobMoved(), notePressed() and their partners).
Before it has a value
Defining the function is enough. oav-p5 asks the browser for Web MIDI when it loads (the default, ?oav_hardware=auto), and a sketch that defines it also gets a card if Web MIDI is missing or blocked. Keys from 🎹, a phone, OAVp5.input() and other programs reach it too. OAVp5.on('noteOff', fn) does the same without defining a global function, and returns a function that stops it.
Parameters
Name
Type
Description
note
Number
The MIDI number of the key that came up, 0 to 127.
channel
Number
The MIDI channel, 1 to 16.
Notes
An error inside it does not stop the music: the next message is still handled, and the message shows in the console and on a card beside the 🎹, with the file and line.
const down = {};
let held = 'play a key and let it go';
function setup() {
createCanvas(400, 300);
textAlign(CENTER, CENTER);
textSize(24);
}
function noteOn(note) { down[note] = millis(); }
function noteOff(note) {
if (down[note] !== undefined) {
held = noteName(note) + ' was held ' + round(millis() - down[note]) + ' ms';
delete down[note];
}
}
function draw() {
background(20);
fill(255);
text(held, width / 2, height / 2);
}
Pairs each noteOn() with its noteOff() and says how long the key was held. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
Related
noteOn()You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
noteReleased()You define this function; oav-p5 calls it each time a key (or a pad) is let go.
padReleased()You define this function; oav-p5 calls it each time a drum pad is let go.
midiMessage()You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
OAVp5.on()Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
Declare it the way you declare keyPressed():
js
function noteOn(note, velocity, channel) {
// a key or pad went down
}
It is called once per note-on message, as it came in. It is the level below notePressed(): the same moment, but with the channel, and for pads too, even when the sketch defines padPressed(). A note-on with velocity 0, which some keyboards send instead of a note-off, is not a note-on here: it calls noteOff().
For one incoming message the order is: midiMessage(), then the specific one (noteOn(), controlChange(), pitchBend()), then the friendly one (padPressed(), knobMoved(), notePressed() and their partners).
Before it has a value
Defining the function is enough. oav-p5 asks the browser for Web MIDI when it loads (the default, ?oav_hardware=auto), and a sketch that defines it also gets a card if Web MIDI is missing or blocked. Keys from 🎹, a phone, OAVp5.input() and other programs reach it too. OAVp5.on('noteOn', fn) does the same without defining a global function, and returns a function that stops it.
Parameters
Name
Type
Description
note
Number
The MIDI number of the key, 0 to 127 (60 is middle C). noteName(note) turns it into a name.
velocity
Number
How hard, above 0 and up to 1 (MIDI's 1 to 127 divided by 127).
channel
Number
The MIDI channel, 1 to 16. Pads on the MiniLab 3 and the LPD8 send on channel 10.
Notes
An error inside it does not stop the music: the next message is still handled, and the message shows in the console and on a card beside the 🎹, with the file and line.
Keys leave blue circles and pads leave orange squares, placed by pitch, bigger for harder hits, fading after a moment. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
Related
noteOff()You define this function; oav-p5 calls it for every MIDI note-off, with the key and the channel.
notePressed()You define this function; oav-p5 calls it each time a key (or a pad) goes down.
padPressed()You define this function; oav-p5 calls it each time a drum pad is hit.
midiMessage()You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
OAVp5.on()Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
OAVp5.input()Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
You define this function; oav-p5 calls it each time a key (or a pad) goes down.
Declare it the way you declare keyPressed():
js
function notePressed(note, velocity) {
// a key went down
}
It is called once per key, with the key and how hard. A pad also calls it, unless the sketch defines padPressed().
Before it has a value
Defining it is enough: oav-p5 calls it from every source, and if Web MIDI is missing or refused it says so in a card beside the 🎹 (Web MIDI itself is asked for at load, see knob1). Keys from 🎹 and from a phone call it too.
Parameters
Name
Type
Description
note
Number
The MIDI number of the key (60 is middle C). noteName(note) turns it into a name.
velocity
Number
How hard, 0..1.
Notes
An error inside it does not stop the music: the next key still plays, and the message shows in the console and on a card beside the 🎹, with the file and line.
let held = {};
function setup() {
createCanvas(400, 300);
}
function notePressed(note) { held[note] = millis(); }
function noteReleased(note) { delete held[note]; }
function draw() {
background(20);
fill(255);
text(Object.keys(held).map((n) => noteName(+n)).join(' '), 20, 40); // the keys down now
}
Lists the names of the keys that are down right now.
Related
notePressed()You define this function; oav-p5 calls it each time a key (or a pad) goes down.
padReleased()You define this function; oav-p5 calls it each time a drum pad is let go.
noteOff()You define this function; oav-p5 calls it for every MIDI note-off, with the key and the channel.
noteIsDown()True while the key n (a MIDI number or a name like 'C4') is held down.
How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.
Read it like a held mouse button that also knows how hard: if (pad1 > 0) fill(255, 90, 31). Pads are numbered in the order the controller's profile lists them. To react once per hit rather than ask every frame, define padPressed().
Type
Number
Range
0..1
The velocity of the hit while the pad is held (1 is as hard as the controller sends), and 0 as soon as it is released.
Before it has a value
Nothing to call. Web MIDI is asked for at load, exactly as for a knob (knob1); reading a pad also makes a missing or refused Web MIDI show as a card.
let down = {};
function setup() {
createCanvas(400, 300);
}
function padPressed(n) { down[n] = true; }
function padReleased(n) { down[n] = false; }
function draw() {
background(20);
fill(255, 90, 31);
for (let i = 1; i <= 8; i++) if (down[i]) square(10 + (i - 1) * 48, 130, 40); // lit while held
}
A square per pad, lit from press to release.
Related
padPressed()You define this function; oav-p5 calls it each time a drum pad is hit.
pad1How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.
noteReleased()You define this function; oav-p5 calls it each time a key (or a pad) is let go.
You define this function; oav-p5 calls it whenever the pitch wheel moves, with its position from −1 to 1.
Declare it the way you declare keyPressed():
js
function pitchBend(value, channel) {
// the wheel moved
}
A pitch wheel usually springs back to the middle when you let go, so you will hear a stream of values ending at 0. MIDI sends the position as 14 bits (0 to 16383, with 8192 in the middle). oav-p5 hands it to you as −1 to 1. The latest position is also in OAVp5.bend.
For one incoming message the order is: midiMessage(), then the specific one (noteOn(), controlChange(), pitchBend()), then the friendly one (padPressed(), knobMoved(), notePressed() and their partners).
Before it has a value
Defining the function is enough. oav-p5 asks the browser for Web MIDI when it loads (the default, ?oav_hardware=auto), and a sketch that defines it also gets a card if Web MIDI is missing or blocked. Keys from 🎹, a phone, OAVp5.input() and other programs reach it too. OAVp5.on('pitchBend', fn) does the same without defining a global function, and returns a function that stops it.
Parameters
Name
Type
Description
value
Number
The wheel's position: −1 all the way down, 0 in the middle, 1 all the way up.
channel
Number
The MIDI channel, 1 to 16.
Notes
Pitch bend is not mapped to anything else: there is no bend variable, only this function and OAVp5.bend.
An error inside it does not stop the music: the next message is still handled, and the message shows in the console and on a card beside the 🎹, with the file and line.
let bend = 0;
function setup() {
createCanvas(400, 300);
}
function pitchBend(value) {
bend = value;
}
function draw() {
background(20);
circle(width / 2 + bend * 150, height / 2, 50); // wheel up: right, wheel down: left
}
The circle slides left and right with the pitch wheel and returns to the middle when it springs back. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
let bend = 0;
function setup() {
createCanvas(400, 300);
}
function pitchBend(value) { bend = value; }
function mouseDragged() {
OAVp5.input([0xE0, 0, constrain(round(mouseX / width * 127), 0, 127)]); // no wheel? drag the mouse
}
function draw() {
background(20);
const x = spring(width / 2 + bend * 150, 120, 8); // a spring smooths the steps
circle(x, height / 2, 50);
}
The same wheel through spring(). No wheel? Drag the mouse across the canvas: it sends the same message.
Related
OAVp5.bendWhere the pitch wheel is, from −1 (all the way down) through 0 (middle) to 1 (all the way up).
controlChange()You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.
midiMessage()You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
OAVp5.on()Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
OAVp5.input()Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
Where the pitch wheel is, from −1 (all the way down) through 0 (middle) to 1 (all the way up).
Read it in draw() like a knob that goes both ways: x = width / 2 + OAVp5.bend * 150. MIDI sends the position as 14 bits (0 to 16383, with 8192 in the middle). oav-p5 maps the two halves to −1..0 and 0..1, so full travel reaches exactly −1 and 1. To be told on every movement, define pitchBend().
Type
Number
Range
-1..1
Changes when the wheel moves. 0 before it has moved, and again when a spring-loaded wheel returns to the middle.
Notes
midi.bend is the same number. There is no plain bend variable.
function setup() {
createCanvas(400, 300);
}
function mouseDragged() {
OAVp5.input([0xE0, 0, constrain(round(mouseX / width * 127), 0, 127)]); // no wheel? drag the mouse
}
function draw() {
background(20);
circle(width / 2 + OAVp5.bend * 150, height / 2, 50); // wheel up: right, wheel down: left
text('bend ' + nf(OAVp5.bend, 1, 2), 20, 30);
}
The circle follows the pitch wheel. No wheel? Drag the mouse across the canvas: it sends the same message.
Related
pitchBend()You define this function; oav-p5 calls it whenever the pitch wheel moves, with its position from −1 to 1.
OAVp5.ccAn array of 128 numbers, one per MIDI controller number: where each control last was, 0 to 1. OAVp5.cc[1] is the mod wheel, OAVp5.cc[64] the sustain pedal.
OAVp5.noteAn array of 128 numbers, one per MIDI key: how hard that key is held, 0 to 1, and 0 when it is up. OAVp5.note[60] is middle C.
An array of 128 numbers, one per MIDI controller number: where each control last was, 0 to 1. OAVp5.cc[1] is the mod wheel, OAVp5.cc[64] the sustain pedal.
The array version of controlChange(): read any controller in draw() without defining a function. It holds every CC, including the ones that are not a knob or a fader, such as the mod wheel (1), the volume (7) and the sustain pedal (64, which counts as down at 0.5 and above). The MIDI channel is ignored: the latest value of that controller number wins, whichever channel it came on.
Type
Array<Number>
Range
128 numbers, 0..1 each
The index is the controller (CC) number, 0 to 127; the value is the last one received divided by 127. Changes when a message arrives. 0 before a control has moved, which is not the same as "at 0".
Notes
midi.cc is the same array. The friendly names knob1 and fader1 are looked up through the controller's profile and read the same data.
function setup() {
createCanvas(400, 300);
}
function mouseDragged() {
OAVp5.input([0xB0, 1, constrain(round(mouseX / width * 127), 0, 127)]); // no wheel? drag the mouse
}
function draw() {
background(OAVp5.cc[64] >= 0.5 ? 90 : 20); // sustain pedal down: lighter
circle(width / 2, height / 2, 40 + OAVp5.cc[1] * 300); // mod wheel: size
}
The mod wheel (CC 1) sets the size of the circle and the sustain pedal (CC 64) lightens the background. No controller? Drag the mouse across the canvas: it sends CC 1.
Related
controlChange()You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.
knob1The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
fader1The position of a controller fader (slider), from 0 to 1: fader1 is the first fader, up to fader8.
OAVp5.noteAn array of 128 numbers, one per MIDI key: how hard that key is held, 0 to 1, and 0 when it is up. OAVp5.note[60] is middle C.
OAVp5.bendWhere the pitch wheel is, from −1 (all the way down) through 0 (middle) to 1 (all the way up).
Asks the browser for Web MIDI now, and returns a promise that says whether a real controller can be heard.
It does what ?oav_hardware=auto does at load, at a moment you pick: from a button, after an intro, or when hardware is click or off. It works whatever the option says. After a refusal you can call it again to retry, but browsers usually remember a refusal and answer at once, so the way out is the lock icon left of the address.
Before you call it
A browser with Web MIDI (Chrome, Edge, Firefox), on https or localhost. Some browsers want a click first: call it from mousePressed() or a button handler if setup() is too early. Safari and iPad have none: the promise resolves false and OAVp5.hardware is 'unavailable'.
Returns
Promise<Boolean> Resolves to true once the browser has allowed Web MIDI (with or without a device plugged in), and to false if it refused or there is no Web MIDI in this browser. Calling it again while it is asking, or after it said yes, gives the same answer.
window.OAV_P5 = { hardware: 'click' }; // do not ask at loadlet answer = 'click the canvas to ask for Web MIDI';
function setup() {
createCanvas(400, 300);
}
function mousePressed() {
OAVp5.connectHardware().then((ok) => { answer = ok ? 'Web MIDI allowed' : 'no Web MIDI for this page'; });
}
function draw() {
background(20);
fill(255);
text(answer, 20, 30);
text('state: ' + OAVp5.hardware, 20, 60);
}
Nothing is asked at load. Click the canvas: the browser asks, the state goes from "off" through "asking" to "connected", "waiting" or "denied", and the text follows.
Related
OAVp5.hardwareWhere Web MIDI stands: 'off', 'asking', 'connected', 'waiting', 'denied' or 'unavailable'.
OAVp5.devicesThe names of the MIDI controllers plugged in, as the browser reports them: an array of text, empty when there is none.
hardwareWhen oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.
The names of the MIDI controllers plugged in, as the browser reports them: an array of text, empty when there is none.
Use it for a line like "playing: Arturia MiniLab3", or to tell whether a real controller is there. The names are whatever the operating system calls the port, so match them loosely (name.toLowerCase().includes('minilab')). The 🎹 panel uses them to follow the controller that is plugged in.
Type
Array<String>
One entry per connected input port. Replaced whenever a device is plugged in or out. Empty until Web MIDI has been allowed (OAVp5.hardware), and it never lists the on-screen controller or the phone.
Notes
midi.devices is the same array, replaced as a whole, not changed in place: read it again after a change.
Where Web MIDI stands: 'off', 'asking', 'connected', 'waiting', 'denied' or 'unavailable'.
'off': Web MIDI has not been asked for yet (see hardware). 'asking': waiting for the browser's permission answer. 'connected': allowed, and at least one device is plugged in (OAVp5.devices lists them). 'waiting': allowed, but no device yet: plug one in and it becomes 'connected' by itself. 'denied': the permission was refused. 'unavailable': the browser has no Web MIDI (Safari, iPad) or could not open it.
It is only about real hardware. The on-screen controller and a phone work in every state.
Shows the state of Web MIDI and the devices found. The background turns green once a controller is connected. Plug one in to see it change.
Related
OAVp5.devicesThe names of the MIDI controllers plugged in, as the browser reports them: an array of text, empty when there is none.
OAVp5.connectHardware()Asks the browser for Web MIDI now, and returns a promise that says whether a real controller can be heard.
hardwareWhen oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.
OAVp5.check()Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
A list of the MIDI numbers of the keys that are down right now, lowest first.
The short version of OAVp5.note: only the keys that are down, so OAVp5.held.length is how many, OAVp5.held[0] is the lowest and OAVp5.held.map((n) => noteName(n)) spells the chord. noteIsPressed is OAVp5.held.length > 0.
Type
Array<Number>
Empty when nothing is held. A key is added when its note-on arrives and removed at its note-off, and the list is kept sorted from low to high.
Notes
midi.held is the same array. Read it, do not change it. It is the same array object every time, so copy it ([...OAVp5.held]) if you want to keep a moment of it.
function setup() {
createCanvas(400, 300);
textAlign(CENTER, CENTER);
textSize(40);
}
function mousePressed() { OAVp5.play(60 + floor(random(12)), 0.8); } // no keys? clickfunction draw() {
background(20);
fill(255);
text(OAVp5.held.map((n) => noteName(n)).join(' ') || 'play a chord', width / 2, height / 2);
}
Spells the keys held down, lowest first. Click the canvas to hold one more key. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
Related
OAVp5.noteAn array of 128 numbers, one per MIDI key: how hard that key is held, 0 to 1, and 0 when it is up. OAVp5.note[60] is middle C.
noteIsPressedTrue while at least one key is held down.
noteIsDown()True while the key n (a MIDI number or a name like 'C4') is held down.
noteName()Turns a MIDI number into a note name: noteName(61) is 'C#4'.
Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
It is the door every input goes through, and you can knock on it yourself: a sketch can test itself without a controller, drive itself from a timer, or take messages from a WebSocket or another library. The sketch cannot tell the difference: knob1, notePressed(), OAVp5.note, the sound if startSound() is on and the drum cues all react. OAVp5.play() is the friendlier way to send a note.
Parameters
Name
Type
Description
bytes
Array<Number>
One message, [status, data1, data2]: [0x90, 60, 100] is a note-on for key 60 with velocity 100 on channel 1, [0xB0, 74, 127] a control change, [0xE0, 0, 64] a pitch bend in the middle. The status byte sets the kind (top half) and the channel (bottom half). Only the first three numbers are read, and the data bytes are held to 0 to 127. System messages are not for this: the clock bytes (0xF8, 0xFA, 0xFB, 0xFC, 0xF2) go to OAVp5.clock, and anything else from 0xF0 up is ignored.
source
String
A label for where it came from, which shows up as OAVp5.last.source and as the second argument of midiMessage(). Any text. 'hardware' and 'virtual-out' are taken as those real sources and are not repeated to a sketch's own Web MIDI code. optionaldefault 'screen'
device
String
A profile id such as 'akai-lpd8': the numbering of knob1, pad1 … follows that controller for this message. Left out, it is the controller on the on-screen panel. optional
Notes
midi.input(…) is the same function. It returns nothing. A message with a source other than 'screen' also lights the keys of the on-screen controller if it is open.
function setup() {
createCanvas(400, 300);
OAVp5.input([0xB0, 70, 127], 'test', 'akai-lpd8'); // CC 70 is knob 1 on an LPD8
}
function draw() {
background(20);
fill(255);
text('knob1 = ' + knob1 + ' (from the LPD8 numbering)', 20, 30);
text('last: ' + OAVp5.last.control + ' from ' + OAVp5.last.source, 20, 60);
}
The third argument says which controller the message comes from, so CC 70 is knob 1 here, whatever profile the panel shows.
Related
OAVp5.play()Plays a key as if a controller did: midi.play(60, 0.8) presses middle C, midi.play(60, 0) lets it go.
OAVp5.lastThe latest note, control change or pitch bend that arrived: its type, number, value, channel, where it came from and which control it is. null before any.
OAVp5.on()Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
midiMessage()You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
noteOn()You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
controlChange()You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.
True while the key n (a MIDI number or a name like 'C4') is held down: the same as noteIsDown(n).
It is noteIsDown() under the OAVp5 name, for a sketch that cannot use the plain one: OAVp5.isDown('C4'). A name that is not a note stops the sketch with a card that says how to write one.
Parameters
Name
Type
Description
n
Number | String
The key: a MIDI number (60) or a note name ('C4', 'c#4', 'Db3').
function setup() {
createCanvas(400, 300);
textAlign(CENTER, CENTER);
textSize(32);
}
function mousePressed() { for (const n of ['C4', 'E4', 'G4']) OAVp5.play(n, 0.8); } // no keys? click for a chordfunction mouseReleased() { for (const n of ['C4', 'E4', 'G4']) OAVp5.play(n, 0); }
function draw() {
const chord = OAVp5.isDown('C4') && OAVp5.isDown('E4') && OAVp5.isDown('G4');
background(chord ? color(60, 100, 40) : 20);
fill(255);
text(chord ? 'C major' : 'hold C4 + E4 + G4', width / 2, height / 2);
}
The background turns green while C, E and G are held together. Hold the mouse down to play the chord. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
Related
noteIsDown()True while the key n (a MIDI number or a name like 'C4') is held down.
OAVp5.noteAn array of 128 numbers, one per MIDI key: how hard that key is held, 0 to 1, and 0 when it is up. OAVp5.note[60] is middle C.
OAVp5.heldA list of the MIDI numbers of the keys that are down right now, lowest first.
noteNumber()Turns a note name like 'C4' into its MIDI number, 0 to 127.
The latest note, control change or pitch bend that arrived: its type, number, value, channel, where it came from and which control it is. null before any.
A single object that is replaced on every message, so read it, do not keep it. It is how a sketch finds out where a message came from (OAVp5.last.source) or which knob moved (OAVp5.last.control), and what to read when the sketch has its own note or velocity that keeps the plain names: OAVp5.last.note and OAVp5.last.value.
It keeps only notes, control changes and pitch bends. Aftertouch and program change reach midiMessage() but are not stored here.
Type
Object | null
{ type, note | cc, value, channel, source, control }. type is 'noteOn', 'noteOff', 'controlChange' or 'pitchBend' (a note-on with velocity 0 is a 'noteOff'). note is there for notes, cc for control changes. value is the velocity 0..1 (0 for a note-off), the control's position 0..1, or the wheel −1..1. channel is 1 to 16. source is 'hardware', 'screen', 'phone', 'virtual-out', 'link' or the label you gave OAVp5.input(). control is 'knob1', 'fader2', 'pad3' … when the controller's profile knows it, otherwise null.
Notes
midi.last is the same object. Clock messages (tick, start, stop) are not here: see OAVp5.clock.
Prints the last message as an object. Click the canvas to send one from the mouse, and watch source. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
Related
midiMessage()You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
noteOn()You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
noteThe MIDI number (0 to 127) of the last key pressed.
velocityHow hard the last key was pressed, from 0 to 1.
OAVp5.input()Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
An array of 128 numbers, one per MIDI key: how hard that key is held, 0 to 1, and 0 when it is up. OAVp5.note[60] is middle C.
Read every key at once, in draw(): OAVp5.note[60] for one key, or a loop over a range to draw a piano roll. It is the array version of noteIsDown(), with the velocity in it. It is not the variable note, which is just the number of the last key pressed. OAVp5.held is the same information as a short list of the keys that are down.
Type
Array<Number>
Range
128 numbers, 0..1 each
The index is the MIDI note number, 0 to 127. A key that is down holds the velocity of its note-on divided by 127, a key that is up (or never pressed) holds 0. Changes when a message arrives, not once per frame.
Notes
midi.note is the same array. A note-on with velocity 0 sets the key to 0, like a note-off.
function setup() {
createCanvas(400, 300);
noStroke();
}
function draw() {
background(20);
for (let n = 36; n < 96; n++) { // five octaves, C2 to B6const v = OAVp5.note[n];
fill(v > 0 ? color(255, 211, 107) : 60);
rect(20 + (n - 36) * 6, height - 40 - v * 200, 5, 40 + v * 200);
}
}
A bar for each key: it rises while the key is down, higher for a harder press. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
const v = lag(OAVp5.note[60], 0.05, 0.4); // middle C, softened
circle(width / 2, height / 2, 40 + v * 260);
}
Middle C (60) swells the circle by how hard it is held, and the circle shrinks back slowly.
Related
OAVp5.heldA list of the MIDI numbers of the keys that are down right now, lowest first.
OAVp5.isDown()True while the key n (a MIDI number or a name like 'C4') is held down: the same as noteIsDown(n).
noteIsDown()True while the key n (a MIDI number or a name like 'C4') is held down.
noteThe MIDI number (0 to 127) of the last key pressed.
velocityHow hard the last key was pressed, from 0 to 1.
OAVp5.ccAn array of 128 numbers, one per MIDI controller number: where each control last was, 0 to 1. OAVp5.cc[1] is the mod wheel, OAVp5.cc[64] the sustain pedal.
Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
The event functions (notePressed(), knobMoved(), noteOn() …) are defined once, by name. OAVp5.on('notePressed', fn) is the other way: any number of listeners, added and removed while the sketch runs, with no global names taken. It is how a library, a class or a Hydra sketch listens, and the way to listen for a while and stop.
A listener counts as a defined function for the pad rule: once anything listens to 'padPressed', a pad stops calling notePressed(). An error inside fn shows in the console and on a card, and the other listeners still run. For the microphone's events use OAVp5.mic.on().
Parameters
Name
Type
Description
name
String
The event: 'notePressed', 'noteReleased', 'knobMoved', 'faderMoved', 'padPressed', 'padReleased', 'noteOn', 'noteOff', 'controlChange', 'pitchBend' or 'midiMessage'. 'noteon' and 'noteoff' work in any case. Anything else throws an error that lists the names.
fn
Function
Called with the same arguments the function of that name would get (notePressed(note, velocity), knobMoved(n, value) …).
Returns
Function A function that takes fn off again: const off = OAVp5.on(…); off();.
Counts the notes with a listener instead of a global function. Click the canvas to play one. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
let heard = 0;
let off;
function setup() {
createCanvas(400, 300);
off = OAVp5.on('noteOn', () => {
heard++;
if (heard === 5) off(); // stop listening after five notes
});
}
function mousePressed() { OAVp5.play(60 + floor(random(12)), 0.8); }
function draw() {
background(20);
fill(255);
text('notes heard: ' + heard + (heard >= 5 ? ' (stopped listening)' : ''), 20, 30);
}
The number counts up to five and then stops, even though you keep playing: the listener took itself off.
Related
OAVp5.mic.on()Listens to a microphone event ('beat', 'kick', 'snare' or 'hat') without defining a global function, and returns a function that stops listening.
notePressed()You define this function; oav-p5 calls it each time a key (or a pad) goes down.
noteOn()You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
knobMoved()You define this function; oav-p5 calls it whenever a knob moves.
OAVp5.input()Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
The 🎹 panel as an object: open(), close() and toggle() show and hide it from code.
The panel behind the 🎹 button holds the on-screen controller and the buttons for 🎤, 🔊, 🥁, ⏺, 📷, 📱 and 🩺. OAVp5.panel.open(), close() and toggle() do what pressing 🎹 does, from your code: open it on a key, or open it at the start for a workshop (?oav_panel=open does the same at load).
Notes
With ?oav_panel=off there is no 🎹 and these do nothing.
function setup() {
createCanvas(400, 300);
}
function keyPressed() {
if (key === 'p') OAVp5.panel.toggle();
}
function draw() {
background(20);
fill(255);
text('press p to show or hide the 🎹 panel', 20, 30);
}
Press p: the 🎹 panel opens and, pressed again, closes.
Related
OAVp5.panel.open()Opens the 🎹 panel (open()), closes it (close()) or flips it (toggle()).
panelWhether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
compactShows the 🎹 panel as a small box in the corner, a third of the window high, instead of a full-width bar: on or off.
OAVp5.phone()Shows the 📱 card: a QR code that sends a phone to the remote page, whose MIDI tab then plays this sketch. Returns the room name.
Opens the 🎹 panel (open()), closes it (close()) or flips it (toggle()).
Same as pressing the 🎹 button. The first time the panel opens, oav-p5 loads the on-screen controller (a few seconds on a slow connection, which the panel says while it loads), and if hardware is click it asks for Web MIDI then. OAVp5.panel.open() when it is already open does nothing new.
Returns
Promise<Element | null> A promise. open() (and toggle() when it opens) resolves to the panel's element once the on-screen controller inside it has loaded, and the panel is on screen at once, before that. close() (and toggle() when it closes) resolves to null, and so does everything when the panel option is off.
Notes
The three are listed together here: OAVp5.panel.close and OAVp5.panel.toggle take no arguments either. midi.panel.open() is the same function.
function setup() {
createCanvas(400, 300);
}
function keyPressed() {
if (key === 'o') OAVp5.panel.open();
if (key === 'c') OAVp5.panel.close();
}
function draw() {
background(20);
fill(255);
text('o: open the 🎹 panel, c: close it', 20, 30);
}
function setup() {
createCanvas(400, 300);
}
function mousePressed() {
OAVp5.panel.open().then((panel) => console.log('the panel is', panel ? 'open and loaded' : 'not available'));
}
function draw() {
background(20);
fill(255);
text('click the canvas to open the panel', 20, 30);
}
Click the canvas: the panel opens at once, and the console says when the controller inside it has loaded.
Related
OAVp5.panelThe 🎹 panel as an object: open(), close() and toggle() show and hide it from code.
panelWhether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
OAVp5.phone()Shows the 📱 card: a QR code that sends a phone to the remote page, whose MIDI tab then plays this sketch. Returns the room name.
OAVp5.check()Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
Shows the 📱 card: a QR code that sends a phone to the remote page, whose MIDI tab then plays this sketch. Returns the room name.
The same as the 📱 button in the panel. The card shows a QR code, the room and the link underneath; opening the link on a phone gives that phone an on-screen controller whose MIDI tab plays your sketch over the relay, as one more input (OAVp5.last.source is 'phone'). Calling it again hides the card. The first call connects to the relay and loads a small QR-code library from jsDelivr.
Before you call it
A network connection, to the relay (wss://remote.cheyuwu.com/relay unless you set another) and, for the picture of the QR code, to jsDelivr. Without the library the card shows the link as text.
Returns
String The name of the room the phone joins: the room option, or the link room, or a name made up for this browser (p5-…) and kept in it.
Examples
js
function setup() {
createCanvas(400, 300);
}
function mousePressed() {
const room = OAVp5.phone(); // the QR code card opens
console.log('phones join room', room);
}
function notePressed(note) {
background(random(255), random(255), random(255)); // a phone key changes the colour
}
function draw() {
fill(255);
text('click the canvas, scan the QR code, open the MIDI tab', 20, 30);
}
static Click the canvas and scan the QR code with a phone: the keys on its MIDI tab change the background of this sketch.
To be confirmed
A phone playing a sketch through the relay was not tried for this page: the headless check cannot reach the relay, so the example is not run. The behaviour is as the code and the project's README describe it.
Related
roomThe name of the room phones join to play this page: by default one made up per browser.
relayThe address of the relay that phones and link rooms talk through: a WebSocket URL.
remoteThe address of the remote page that the 📱 QR code opens on a phone.
OAVp5.panel.open()Opens the 🎹 panel (open()), closes it (close()) or flips it (toggle()).
Plays a key as if a controller did: midi.play(60, 0.8) presses middle C, midi.play(60, 0) lets it go.
It sends the same MIDI bytes a keyboard would, with midi.last.source set to 'screen', so everything reacts: notePressed(), noteIsDown(), note, the sound if startSound() is on. midi and OAVp5 are the same object, so OAVp5.play and midi.play are one function.
Parameters
Name
Type
Description
note
Number | String
The key: a MIDI number or a name ('C#4').
velocity
Number
How hard, 0..1; 0 releases the key. Leave it out for full force. optionaldefault 1
Notes
The key stays down until you play it again with velocity 0.
Lists which controller knob, fader or pad drives which OpenProcessing OPC variable right now: [{ control: 'knob1', name: 'radius' }, …].
With OP Configurator, OPC.slider('radius', 10) declares a variable and oav-p5 wires the controller to the OPC variables in order: knob 1 to the first slider, knob 2 to the second, the faders after the knobs, pad 1 to the first toggle or button. This is the list of who drives what, the same one the panel prints under the controller and the console says once. Use it when a knob moves the wrong thing.
Parameters
Name
Type
Description
dev
String
Whose numbering to use: 'screen' (the controller shown in the panel) or a profile id such as 'akai-lpd8'. It matters for the faders, which are numbered after the knobs. optionaldefault 'screen'
Returns
Array<Object> One { control, name } per wired variable: control is 'knob1', 'fader2', 'pad1' …, name is the OPC variable. A control the sketch reads itself also has sketch: true, and its slider no longer follows it. An empty array when there is no OPC on the page or opc is off.
Prints the controls and the OPC variables they drive, one per line. Outside OpenProcessing there is no OPC, so it says nothing is wired.
To be confirmed
Not tried with OpenProcessing's real OPC library: it was checked against a stand-in object with the same options shape ({ type: 'slider', name, min, max }). Outside OpenProcessing the result is an empty array.
Related
opcWhether oav-p5 wires the controller to the variables of OP Configurator (OpenProcessing's sliders and buttons): on or off.
knob1The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
knob()Returns the position of knob number n, from 0 to 1: the same as knob1 to knob8, for any n.
pad1How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.
OAVp5.check()Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
Returns the frequency in Hz of a note: noteFreq('A4') is 440.
For a sketch that makes its own sound (p5.sound, Web Audio): osc.freq(noteFreq(note)). If you just want the sound, startSound() plays every key for you.
Parameters
Name
Type
Description
v
Number | String
A note name or a MIDI number. A number may be fractional (69.5 is a quarter tone above A4) and is not held to 0..127.
Returns
Number Hz, equal temperament with A4 = 69 = 440 Hz.
Turns a MIDI number into a note name: noteName(61) is 'C#4'.
notePressed(note, velocity) hands you the number; noteName(note) writes it back as a name for the screen or a lookup table. The same C4-is-60 convention as noteNumber().
Parameters
Name
Type
Description
v
Number | String
A MIDI number (or a name, which is normalised).
o
Object
Options: { flats: true } writes 'Db4' instead of 'C#4'; { octave: false } drops the octave number ('C#'). optional
Returns
String The name, sharps by default: noteName(60) is 'C4'.
Turns a note name like 'C4' into its MIDI number, 0 to 127.
A name is a letter A to G (either case), an optional # or b (♯ and ♭ too), and the octave: C4, c#4, Db3, C-1 (0) up to G9 (127). C4 is 60. Anything else, such as 'H4' or 'C-2', throws an error, and a card beside the 🎹 says how to write a note.
You rarely call it yourself: every place that takes a note number also takes a name, such as noteIsDown() and midi.play().
Parameters
Name
Type
Description
v
String | Number
A note name ('C4', 'c#4', 'Db3', 'F♯2') or a number. A number comes back rounded and held to 0..127.
Returns
Number The MIDI number: noteNumber('C4') is 60, noteNumber('A4') is 69.
How much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
Use them like micLevel, one number per band: fill(200 + bass * 160, 70, 100) pushes the colour with the low end. The three share one entry because they behave the same; only the frequency range differs.
Unlike micLevel they are not measured in decibels: each is the average strength of the frequencies in its band, scaled by micSensitivity() and held to 1. A band that is faint in your room can stay close to 0; raise the sensitivity rather than multiplying by a large number.
Type
Number
Range
0..1
The energy in 20–250 Hz (bass), 250–2000 Hz (mid) and 2000–8000 Hz (treble). Updated once per display frame while the microphone is on; 0 otherwise.
Before it has a value
Reading any of them asks for the microphone the first time, exactly like micLevel; until you allow it they are 0.
Notes
A sketch that already has its own variable, function or library called bass, mid or treble keeps it: oav-p5 leaves that name alone and never starts the microphone for it. Assigning one later turns the name back into a plain variable.
True for one frame each time the microphone (or the drum machine) hears a beat.
It behaves like mouseIsPressed while a button is down, except the "press" lasts one frame: if (beat) flash = 1; is all it takes to react to the music. A beat is a jump in loudness over what the room has been doing, with at least 100 ms between two beats; two beats between two frames count as one.
When the drum machine is playing, beat is also true for a frame on every kick and snare, so a sketch written for the microphone moves over to the machine unchanged.
Type
Boolean
True in the first draw() after a beat, and false until the next one. Reading it twice in the same frame gives the same answer.
Before it has a value
Reading beat asks for the microphone the first time, like micLevel. If you would rather be called, define beatDetected().
let flash = 0;
function setup() {
createCanvas(400, 300);
}
function draw() {
if (beat) flash = 1; // true for exactly one frame per beat
background(20 + flash * 200);
flash *= 0.9; // fade back down
}
The screen flashes on every beat and fades out. Clap or tap the desk. Asks for the microphone
let count = 0;
function setup() {
createCanvas(400, 300);
textAlign(CENTER, CENTER);
textSize(96);
}
function draw() {
if (beat) count++;
background(20);
fill(255);
text(count, width / 2, height / 2);
}
A counter that goes up by one per beat, never twice for the same one. Asks for the microphone
Related
beatDetected()You define this function; oav-p5 calls it each time the microphone (or the drum machine) hears a beat.
micLevelHow loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
kickDetected()You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
You define this function; oav-p5 calls it each time the microphone (or the drum machine) hears a beat.
Declare it the way you declare keyPressed():
js
function beatDetected(level) {
// flash, spawn, jump to the next scene
}
It is called once per beat, right when the beat lands, not once per frame. Use beat instead if you would rather ask each frame.
Before it has a value
A sketch that defines beatDetected() and reads no microphone value starts the microphone on the first click or key. When the drum machine is on, the same function is called for its kicks and snares and the microphone is not opened for it. OAVp5.mic.on('beat', fn) hears the same beats without defining a global and returns a function that stops listening, but a listener alone never starts the microphone: read micLevel or call startMic() as well.
Parameters
Name
Type
Description
level
Number
How loud it was at that moment, 0..1 (the value of micLevel).
Notes
If your function throws, the sketch keeps running: the error appears in the console and as a card beside the 🎹 with the file and line.
let flash = 0;
function setup() {
createCanvas(400, 300);
}
function beatDetected(level) {
flash = 0.4 + level * 0.6; // louder beat, brighter flash
}
function draw() {
background(20 + flash * 200);
flash *= 0.9;
}
A flash per beat, brighter for a louder one. Asks for the microphone
Related
beatTrue for one frame each time the microphone (or the drum machine) hears a beat.
kickDetected()You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
micLevelHow loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
The microphone splits what it hears into three drum-like bands and calls the matching function when each band jumps: 20–120 Hz for the kick, 150–800 Hz for the snare, 6–14 kHz for the hat. There is no machine learning in it, just a transient in a frequency band, so a bass voice can trigger a kick.
When the drum machine plays, each of its hits calls the same functions: kick for the kick, snare for a snare, clap or rim, hat for a closed, open or pedal hat. A sketch written for the microphone therefore works with startDrums() unchanged.
While the drum machine plays, only its hits call these functions (from oav-p5 0.8.1). A microphone that is on at the same time mostly hears the drums coming out of the speakers, so its hits would call the same function twice and hear a kick as a snare. micLevel, bass, mid and treble still follow the microphone; the microphone's hits come back when the drums stop.
Before it has a value
Defining one of them is not enough to open the microphone once the drum machine has started; reading micLevel (or calling startMic()) is. Without the drum machine, a sketch that defines any of them and reads nothing starts the microphone on the first click or key.
Parameters
Name
Type
Description
level
Number
How strong the hit was, 0..1 (for the drum machine, its velocity).
Two circles, orange for kicks and blue for snares, each swelling and shrinking back on its own hits. Asks for the microphone
Related
beatDetected()You define this function; oav-p5 calls it each time the microphone (or the drum machine) hears a beat.
beatTrue for one frame each time the microphone (or the drum machine) hears a beat.
bassHow much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
True in the one frame after a kick (snare, hat) is heard, from the drum machine or the microphone, like mouseIsPressed for a hit.
The same hits that call kickDetected(), snareDetected() and hatDetected(), as a value you read in draw() instead of a function you write. Hand it to lag() and it becomes a number that jumps to 1 on the hit and fades: lag(kickIsDetected, 0, 0.3) is 1 on a kick and back near 0 about 0.3 s later, ready for an alpha or a size. lag() takes true as 1.
While the drum machine plays, only its hits count. Reading these does not ask for the microphone by itself: with no drums, it counts as wanting the microphone's hits, the way defining kickDetected() does, so the first click on the page opens it.
Type
Boolean
True for exactly one p5 frame per hit, however often that frame reads it; false otherwise. Since oav-p5 0.8.2.
The frameCount of the frame that saw the last kick (snare, hat, beat): frameCount - lastKickFrameCount is how many frames ago.
The frame-counting twin of lastKickTs. frameCount - lastKickFrameCount is 0 in the frame where kickIsDetected is true, then 1, 2, 3 …, so a shape can last a fixed number of frames after each hit. Use the Ts version when it should last a fixed time whatever the frame rate.
Type
Number
A frame number, on the same count as frameCount; -Infinity before the first hit. Since oav-p5 0.8.3.
function setup() {
createCanvas(400, 300);
startDrums('bd*4, ~ sd ~ sd', 110);
noStroke();
}
function draw() {
background(20);
const k = frameCount - lastKickFrameCount; // frames since the kickif (k < 10) { // the ring stays for 10 frames after each kick
fill(255, 90, 31);
circle(200, 200, 60 + k * 16);
}
}
After every kick a circle shows for ten frames, growing each frame.
Related
lastKickTsWhen the last kick (snare, hat, beat) came, on the millis() clock: millis() - lastKickTs is how many ms ago.
kickIsDetectedTrue in the one frame after a kick (snare, hat) is heard, from the drum machine or the microphone, like mouseIsPressed for a hit.
lastNoteFrameCountThe frameCount of the frame that saw the last key go down: frameCount - lastNoteFrameCount is how many frames ago.
When the last kick (snare, hat, beat) came, on the millis() clock: millis() - lastKickTs is how many ms ago.
Set by every hit that calls kickDetected(), snareDetected(), hatDetected() or beatDetected(): the drum machine's, or the microphone's when no drums play. Use it for anything that depends on how long ago: exp(-(millis() - lastKickTs) / 200) is 1 on the kick and fades with a 0.2 s time constant.
Type
Number
Milliseconds on the same clock as millis(); -Infinity before the first hit, so millis() - lastKickTs is then Infinity. Since oav-p5 0.8.2.
Use it to show a hint until sound arrives: if (!micIsOn) text("Allow the microphone", 20, 30);. Reading it never starts the microphone (unlike micLevel), so it is safe to check anywhere. The reason behind a false is in OAVp5.mic.state (off, asking, on, denied or unavailable) and, when something failed, OAVp5.mic.error.
Type
Boolean
True once the browser has said yes and sound is coming in; false while the microphone is off, still asking, blocked or missing.
How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
Read it in draw() the way you read mouseX: circle(width / 2, height / 2, 50 + micLevel * 300) grows when someone talks. It is a loudness in decibels, which is how an ear hears it, measured from the room itself: the bottom of the scale follows the room's own noise (it settles within a second or two and rises at most 1.5 dB a second, so a voice never becomes the new silence), and −10 dBFS, shouting into a laptop, is 1.
In practice a quiet room reads 0, a quiet voice about 0.4 to 0.6, shouting close to 1. If the microphone is quiet or far away, raise it with micSensitivity().
It follows the browser's animation frames rather than draw(), so a sketch running at frameRate(10) still reads the latest value, and the number stops changing while the tab is hidden (browsers pause animation frames there).
Type
Number
Range
0..1
Updated once per display frame while the microphone is on. It is 0 while the microphone is off, still asking, blocked or missing.
Before it has a value
Nothing to call. The first time a sketch reads micLevel (or bass, mid, treble, beat), oav-p5 asks the browser for the microphone, and every value stays 0 until you allow it. The page needs https or localhost. If the browser kept audio quiet until a click, the first click or key starts it. ?oav_mic=click makes only the 🎤 button and startMic() turn it on; ?oav_mic=off means nothing can.
Notes
In the 🎹 panel the 🎤 button turns the microphone on and off by hand and shows the level. When the browser blocked it, a card beside the 🎹 says why and what to press: click the lock left of the address, set Microphone to Allow, then click the canvas.
On openprocessing.org the sketch frame allows the microphone and MIDI (checked 2026-10-19: its allow list contains microphone and midi), and the first read of micLevel there asks for the microphone. An embed code copied from OpenProcessing does not carry that: add allow="microphone" to that <iframe>.
The raw numbers behind it are OAVp5.mic.rms (0..1), OAVp5.mic.db and OAVp5.mic.floor (the room's own noise, in dBFS).
function setup() {
createCanvas(400, 300);
micSensitivity(3); // a quiet or far-away microphone
}
function draw() {
background(20);
fill(255);
if (!micIsOn) text('Allow the microphone, then make a sound', 20, 30);
rect(20, height - 40, micLevel * (width - 40), 20);
}
A bar that fills with the level. micIsOn tells you whether the browser has said yes yet. Asks for the microphone
To be confirmed
On openprocessing.org the pasted sketch runs, loads oav-p5 and asks for the microphone, but the sound itself was not tried there: the browser used for the check blocks the microphone, so the allow-and-listen step is still unconfirmed on that site.
Related
bassHow much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
beatTrue for one frame each time the microphone (or the drum machine) hears a beat.
micIsOnWhether the microphone is running right now.
micSensitivity()Makes the microphone more or less sensitive: micSensitivity(2) is twice as sensitive, micSensitivity() says what it is now.
startMic()Turns the microphone on and returns a promise that says whether it worked.
lag()Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
Makes the microphone more or less sensitive: micSensitivity(2) is twice as sensitive, micSensitivity() says what it is now.
The number multiplies what the microphone hears. On micLevel, 2 adds 6 dB; on bass, mid and treble it multiplies by 2 (they stay held to 1). Use it for a quiet microphone or a far-away voice: micSensitivity(3) in setup() adds about 9.5 dB.
Parameters
Name
Type
Description
v
Number
The new sensitivity, greater than 0 (kept between 0.05 and 20). Leave it out, or pass something that is not greater than 0, to change nothing. optional
Returns
Number The sensitivity now in effect (1 until you change it).
Notes
OAVp5.mic.gain is the same knob under the name oav-p5 0.3.0 used.
function setup() {
createCanvas(400, 300);
micSensitivity(3); // three times as sensitive
}
function draw() {
background(20);
circle(width / 2, height / 2, 50 + micLevel * 300);
text('sensitivity ' + micSensitivity(), 20, 30); // no argument: just reads it
}
The same loudness circle on a microphone that needed a boost; the number shows the sensitivity in effect. Asks for the microphone
Related
micLevelHow loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
bassHow much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
startMic()Turns the microphone on and returns a promise that says whether it worked.
Turns the microphone on and returns a promise that says whether it worked.
You rarely need it: reading micLevel and its friends already starts the microphone. Call it when you want to choose the moment (a button, the end of an intro) or know the answer: startMic().then((ok) => { … }). Calling it again while it is on, or while the browser is still asking, returns the same result.
Before you call it
Browsers want a click before audio plays: call it from mousePressed() or a button handler if setup() is too early. It works whatever the mic option is, except ?oav_mic=off, where nothing can start the microphone.
Returns
Promise<Boolean> Resolves to true once the microphone is on, false if it was blocked, is missing, or the mic option is off.
Notes
After stopMic(), only startMic() (or 🎤) turns it back on; reading micLevel does not.
Turns the microphone off and sets every microphone value back to 0.
It stops the microphone's tracks and closes its audio context, so the browser's "recording" indicator goes away. micLevel, bass, mid, treble and the rest read 0 again, and micIsOn is false.
Returns
Promise<Boolean> Resolves to false, the new state of the microphone.
Notes
After stopMic(), reading micLevel does not turn the microphone back on: that needs startMic() or the 🎤 button.
The microphone under the plain names: its state, the numbers of micLevel and bass that never turn it on when read, the raw loudness, start, stop and events.
Everything the microphone names (micLevel, bass, micIsOn, startMic() …) read from here. The difference is that reading OAVp5.mic.level never asks for the microphone: the plain micLevel does, the first time. So OAVp5.mic is for code that must look without waking anything: a status line, a library, a sketch with ?oav_mic=click.
The AudioAnalyzer from @openav/audio that listens to the microphone, while the microphone is on; null otherwise.
A way into the audio itself when the 0 to 1 numbers are not enough: read the spectrum of the microphone with the browser's own OAVp5.mic.analyzer.analyser.getByteFrequencyData(array), or connect your own Web Audio nodes to it. oav-p5 updates the analyzer once per frame, so do not call its update() yourself.
Type
AudioAnalyzer | null
Made when the microphone starts and dropped when it stops. It has its own AudioContext (OAVp5.mic.analyzer.ctx) and the Web Audio AnalyserNode it reads from (OAVp5.mic.analyzer.analyser).
Notes
This is advanced. Check it is not null first, and read it again after stopMic() and startMic(): it is a new object each time.
let bins;
function setup() {
createCanvas(400, 300);
noStroke();
startMic();
}
function draw() {
background(20);
const a = OAVp5.mic.analyzer;
if (!a) return; // the microphone is not on yet
bins = bins || new Uint8Array(a.analyser.frequencyBinCount);
a.analyser.getByteFrequencyData(bins); // the spectrum, 0..255 per frequency bin
fill(255, 211, 107);
for (let i = 0; i < 64; i++) {
const h = bins[i] / 255 * (height - 20);
rect(20 + i * 5.8, height - 10 - h, 4, h);
}
}
The lowest 64 bins of the microphone's spectrum as bars, read straight from the analyser. Whistle and watch the bars move. Asks for the microphone
Related
OAVp5.mic.start()Turns the microphone on and returns a promise that says whether it worked: the same as startMic().
OAVp5.mic.stateWhere the microphone is: 'off', 'asking', 'on', 'denied' or 'unavailable'.
bassHow much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
micLevelHow loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
The same 0 to 1 low, middle and high energy as bass, mid and treble, read without ever turning the microphone on.
The plain bass, mid and treble ask for the microphone when first read, and a sketch that already has a variable of that name keeps it. OAVp5.mic.bass, .mid and .treble are the numbers themselves: always there, never starting anything. They are also the answer when the plain names are taken: on strudel.cc, where Strudel owns bass, this is what the card tells you to read.
Type
Number
Range
0..1
bass is 20–250 Hz, mid 250–2000 Hz and treble 2000–8000 Hz, scaled by the sensitivity and held to 1. Updated once per display frame while the microphone is on; 0 otherwise.
Notes
The three share this entry because they behave the same: only the frequency range differs.
The loudness of the microphone in decibels relative to full scale (dBFS): 0 is as loud as it gets, −100 is silence or the microphone being off.
Decibels are what an ear hears: each 6 dB is twice the amplitude, so a number in dB moves the way loudness feels. micLevel maps it onto 0 to 1 between the room's noise floor (+6 dB) and −10 dBFS. Use db directly when you want your own scale, for example map(OAVp5.mic.db, -60, -10, 0, 1).
Type
Number
Range
-100..0
20 * log10(rms), measured from OAVp5.mic.rms. −100 when the level is below about −100 dB and while the microphone is off. Updated once per display frame.
Why the microphone did not start, as the browser's error name such as 'NotAllowedError'; null while there is no error.
For a sketch that wants to say more than a card does: if (OAVp5.mic.error === 'NotAllowedError') …. The card beside the 🎹 already tells the user what to press for each of these, in their language; this is the same fact for your code.
Type
String | null
null at the start and again once the microphone is on. After a failure: the browser's error name ('NotAllowedError' or 'SecurityError' for a refusal, 'NotFoundError' when there is no microphone, 'NotReadableError' when another program holds it) or 'no getUserMedia' on a page that cannot ask.
The room's own noise as oav-p5 has measured it, in dBFS: the level that micLevel treats as silence (plus 6 dB).
A humming classroom should not read as loud, so micLevel is measured from the room itself: this floor follows the quietest recent readings. It drops within a second when the room gets quieter and rises at most 1.5 dB a second when it gets noisier, so a voice never becomes the new silence. 6 dB above the floor reads 0 on micLevel.
Type
Number
Range
-70..-45
Starts at −60 and is always held between −70 and −45. Updated once per display frame while the microphone is on.
Notes
Stopping the microphone does not reset it: the next start continues from the last floor.
Shows the level, the room's noise floor and the resulting micLevel. Keep quiet for a few seconds and watch the floor settle. Asks for the microphone
Related
micLevelHow loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
OAVp5.mic.dbThe loudness of the microphone in decibels relative to full scale (dBFS): 0 is as loud as it gets, −100 is silence or the microphone being off.
OAVp5.mic.rmsThe raw loudness of the microphone: a smoothed RMS, 0 to 1, before oav-p5 turns it into the decibel scale of micLevel.
OAVp5.mic.levelThe same 0 to 1 loudness as micLevel, read without ever turning the microphone on.
The same 0 to 1 loudness as micLevel, read without ever turning the microphone on.
micLevel is a getter that returns this number and, the first time it is read, asks for the microphone. OAVp5.mic.level is the number itself, so reading it is safe anywhere: before the user has chosen, with ?oav_mic=click, or in a library. Start the microphone with startMic() or 🎤, then it moves. How it is measured (decibels, the room as zero, −10 dBFS as 1) is described under micLevel.
Type
Number
Range
0..1
Updated once per display frame while the microphone is on; 0 while it is off, asking, blocked or missing.
The loudness circle, written with the raw number instead of micLevel. Allow the microphone and talk. Asks for the microphone
Related
micLevelHow loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
OAVp5.mic.rmsThe raw loudness of the microphone: a smoothed RMS, 0 to 1, before oav-p5 turns it into the decibel scale of micLevel.
OAVp5.mic.dbThe loudness of the microphone in decibels relative to full scale (dBFS): 0 is as loud as it gets, −100 is silence or the microphone being off.
OAVp5.mic.floorThe room's own noise as oav-p5 has measured it, in dBFS: the level that micLevel treats as silence (plus 6 dB).
startMic()Turns the microphone on and returns a promise that says whether it worked.
Listens to a microphone event ('beat', 'kick', 'snare' or 'hat') without defining a global function, and returns a function that stops listening.
The listener version of beatDetected(), kickDetected(), snareDetected() and hatDetected(): any number of them, added and removed while the sketch runs, with no global names taken. An error inside fn shows in the console and on a card.
Before you call it
Listening does not turn the microphone on: call startMic(), press 🎤, or read micLevel too, or nothing will ever fire. (Defining the global beatDetected() is different: oav-p5 sees that function and starts the microphone at the first click.) These events come from the microphone only, not from the drum machine.
Parameters
Name
Type
Description
name
String
'beat', 'kick', 'snare' or 'hat'. Anything else throws an error that lists them.
Function A function that takes fn off again: const off = OAVp5.mic.on(…); off();.
Notes
When the drum machine plays, the global beatDetected(), kickDetected() and the others are also called for its hits. A listener added with OAVp5.mic.on() hears the microphone only.
The raw loudness of the microphone: a smoothed RMS, 0 to 1, before oav-p5 turns it into the decibel scale of micLevel.
The root mean square of the samples in the latest frame, smoothed a little. It is linear, the way a waveform is drawn, so it suits a quick check or your own scale. micLevel is built from it: OAVp5.mic.db is the same number in dBFS, and OAVp5.mic.floor is the room's own noise, in dBFS too.
Type
Number
Range
0..1
Updated once per display frame while the microphone is on; 0 when it is off. A speaking voice into a laptop is a small number, in the range 0.01 to 0.05 according to the comment in the code, which is why micLevel works in decibels.
function setup() {
createCanvas(400, 300);
startMic();
}
function draw() {
background(20);
fill(255);
text('rms ' + nf(OAVp5.mic.rms, 1, 4), 20, 30);
rect(20, 60, min(OAVp5.mic.rms * 10, 1) * (width - 40), 20); // magnified 10 times: a voice is small
}
The raw RMS as a number and a bar, magnified ten times because speech is a small number. Allow the microphone and talk. Asks for the microphone
To be confirmed
Typical values for speech and music depend on the microphone and the room. The 0.01 to 0.05 for a voice is the code comment's figure and was not measured here.
Related
OAVp5.mic.dbThe loudness of the microphone in decibels relative to full scale (dBFS): 0 is as loud as it gets, −100 is silence or the microphone being off.
OAVp5.mic.floorThe room's own noise as oav-p5 has measured it, in dBFS: the level that micLevel treats as silence (plus 6 dB).
OAVp5.mic.levelThe same 0 to 1 loudness as micLevel, read without ever turning the microphone on.
micLevelHow loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
The microphone's sensitivity: 1 normally, 2 for twice as sensitive. micSensitivity() sets it; OAVp5.mic.gain is the old name.
The number multiplies what the microphone hears: on micLevel 2 adds 6 dB, on bass, mid and treble it multiplies by 2. Read it to show the current setting, and set it with micSensitivity(), which checks the number.
Type
Number
Range
0.05..20
Starts at 1. micSensitivity(v) keeps what you give it between 0.05 and 20.
Notes
OAVp5.mic.gain is the same value under the name oav-p5 0.3.0 used. Assigning to either works (OAVp5.mic.gain = 2) but is not held between 0.05 and 20, and gain falls back to 1 for a number that is not above 0.
Turns the microphone on and returns a promise that says whether it worked: the same as startMic().
startMic() is this function: see it for what must happen first (a click, https or localhost). OAVp5.mic.start() also lifts the "stopped by you" mark that stopMic() leaves, which is what lets a later read of micLevel switch the microphone on again.
Returns
Promise<Boolean> Resolves to true once the microphone is on, false if it was blocked, is missing, or the mic option is off.
Where the microphone is: 'off', 'asking', 'on', 'denied' or 'unavailable'.
'off': not started, or stopped. 'asking': the browser is waiting for the user's answer. 'on': sound is coming in, and micIsOn is true. 'denied': the permission was refused (by the user, by the computer's settings, or by an iframe without allow="microphone"). 'unavailable': there is no microphone, none the browser can open, or the page is not on https or localhost. When it is not 'on', OAVp5.mic.error has the browser's reason and a card beside the 🎹 says what to press.
function setup() {
createCanvas(400, 300);
startMic();
}
function draw() {
background(OAVp5.mic.state === 'on' ? 20 : color(90, 30, 30));
fill(255);
text('microphone: ' + OAVp5.mic.state, 20, 30);
if (OAVp5.mic.state === 'denied') text('click the lock left of the address and allow it', 20, 60);
}
The background is red until the microphone is on. The state goes "asking", then "on" when you allow it, or "denied" when you refuse. Asks for the microphone
Related
micIsOnWhether the microphone is running right now.
OAVp5.mic.errorWhy the microphone did not start, as the browser's error name such as 'NotAllowedError'; null while there is no error.
startMic()Turns the microphone on and returns a promise that says whether it worked.
micWho may turn the microphone on: auto (the first read of micLevel and friends), click (only 🎤 or startMic()) or off (nobody).
OAVp5.check()Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
Turns the microphone off and sets every microphone value back to 0: the same as stopMic().
stopMic() is this function. It stops the microphone's tracks and closes its audio context, so the browser's "recording" indicator goes away, and OAVp5.mic.level, bass, mid, treble, rms read 0 and db reads −100. After it, only OAVp5.mic.start() or 🎤 turns the microphone back on: reading micLevel does not.
Returns
Promise<Boolean> Resolves to false, the new state of the microphone.
Listens to the microphone, an audio element or any Web Audio node, and turns it into loudness, three bands, onsets and kick, snare and hat hits.
An AudioAnalyzer wraps one Web Audio AnalyserNode. You give it a sound (enableMic(), enableElement(el), or connect your own node into input()), call update() once per frame, and it returns a plain object of numbers, which it also publishes as audio/* signals if you gave it a signals bus. This is the analysis under the microphone of a p5 sketch: micLevel is its rms turned into decibels, bass, mid and treble are its low, mid and high, and beat and kickDetected() come from its onset and kick.
update() returns null until a source is enabled. After that it returns { rms, peak, low, mid, high, centroid, onset, kick, snare, hat }. rms is the smoothed loudness and peak the loudest sample of the moment, both 0..1 of full scale (from float samples where the browser has them, so a quiet room is measured, not rounded to one 8-bit step). A voice is a small rms: scale it, or map it through a window. low, mid and high are the average strength of the spectrum in 20–250 Hz, 250–2000 Hz and 2000–8000 Hz, 0..1, taken from the analyser's byte spectrum, so they are not decibels and a faint band stays near 0. centroid is the spectrum's centre of mass, 0..1 of the way from 0 Hz to half the sample rate (the brightness). onset is true for the one update() in which the loudness jumped over its adaptive floor, with at least 100 ms between two onsets.
kick, snare and hat are { hit, level, env }, found by looking for a sudden rise in one band of the spectrum, with no machine learning: kick 20–120 Hz, snare 150–800 Hz, hat 6000–14000 Hz. hit is true on the update that found one (at least 90, 90 and 60 ms apart), level is the band's strength now, and env is 0..1: it jumps to min(1, level × 2) on a hit and then shrinks by 12% every update(), about 120 ms at 60 frames a second. Because it decays per update, it falls faster if you call update() less often.
With a signals bus, these are published: audio/rms, audio/peak, audio/band/low, audio/band/mid, audio/band/high, audio/centroid, audio/kick/env, audio/snare/env and audio/hat/env (continuous, 0..1), and the pulses audio/onset{ rms }, audio/kick{ level }, audio/snare{ level } and audio/hat{ level }. Pulses fire only when something was found.
Before you call it
Browsers start audio only after a user gesture. enableMic() asks for the microphone, and the analyser's AudioContext stays suspended until the page has had a click or a key: update() calls resume() on every frame, so the numbers start by themselves after the first click. Until then they are all 0. The page needs https or localhost for the microphone.
Parameters
Name
Type
Description
signals
Signals
A Signals bus to publish into. With one, the audio/* signals are defined when you construct the analyzer and set on every update(). Without one, update() only returns the numbers. optionaldefault null
fftSize
Number
The size of the analyser's FFT, a power of two from 32 to 32768. The spectrum has half as many bins. optionaldefault 2048
smooth
Number
How much of the previous rms is kept each update, 0..1. Higher is smoother and slower: rms = rms × smooth + now × (1 − smooth). optionaldefault 0.7
Asks the browser for the microphone, with echo cancellation, noise suppression and auto gain off, and connects it. Returns a promise that resolves to true. It rejects when the permission is refused or there is no microphone, with the browser's error (NotAllowedError, NotFoundError…).
Analyses an <audio> or <video> element: enableElement(el) returns true. The element keeps playing to the speakers. Web Audio takes an element in only once, so a second enableElement() for the same element throws a browser error.
Returns the AnalyserNode (creating the context first), so you can bring your own sound: node.connect(audio.input()). Any node of audio.ctx works; for Tone.js, connect its raw node. Nothing is routed to the speakers by this.
Call it once per frame. Reads the analyser, works out the numbers, publishes the signals if there is a bus, and returns the object { rms, peak, low, mid, high, centroid, onset, kick, snare, hat }, or null when no source is enabled yet. It also resumes a suspended context.
Notes
It makes its own AudioContext, so p5.sound or Tone.js can run beside it. To analyse a sound that lives in another context, connect a node of audio.ctx into input(), or use enableElement().
enableMic() turns off echo cancellation, noise suppression and automatic gain, because they change the loudness you want to measure. On a laptop that plays the sound it listens to, expect the speakers to leak into the microphone.
Examples
js
import { AudioAnalyzer } from'@openav/audio';
const audio = new AudioAnalyzer();
await audio.enableMic(); // the browser asks for the microphoneconst g = document.getElementById('c').getContext('2d');
(function frame() {
const f = audio.update(); // { rms, peak, low, mid, high, ... }
g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
g.fillStyle = '#ffd166';
g.fillRect(20, 130, Math.min(1, f.rms * 6) * 360, 40); // a voice is a small rms: scale it
requestAnimationFrame(frame);
})();
module Allow the microphone, click the page once, and talk: a bar grows with the loudness. Asks for the microphone
js
import { AudioAnalyzer } from'@openav/audio';
const audio = new AudioAnalyzer(); // no signals bus: update() just returns the numbersconst input = audio.input(); // makes the AudioContext (audio.ctx) and the AnalyserNodeconst osc = new OscillatorNode(audio.ctx, { frequency: 100 });
osc.connect(input); // any node of audio.ctx can feed the analyser
osc.start();
let frames = 0;
(function frame() {
const f = audio.update(); // also resumes the context once the page has had a clickif (++frames < 40) return requestAnimationFrame(frame);
console.log(audio.ctx.state, f.low > f.high); // running true: a 100 Hz tone sits in the low band
console.log(f.rms > 0.1, f.centroid < 0.1); // true true
})();
module No microphone needed: a 100 Hz tone is fed into the analyser through input(). After 40 frames the log says the low band is louder than the high one.
js
import { AudioAnalyzer } from'@openav/audio';
import { Signals } from'@openav/core';
const signals = new Signals();
const audio = new AudioAnalyzer({ signals }); // defines the audio/* signals at once
console.log(signals.list().length); // 13
signals.on('audio/onset', ({ rms }) => console.log('onset', rms.toFixed(2)));
signals.on('audio/kick', ({ level }) => console.log('kick', level.toFixed(2)));
console.log(signals.get('audio/rms')); // 0, nothing heard yet
console.log(audio.update()); // null until a source is enabled
module With a signals bus the analyzer defines thirteen audio/* signals up front. Listen to a pulse with signals.on; the handlers run once a source is enabled and a hit is found.
To be confirmed
Typical rms values for speech differ between the notes of this project: the handbook (docs/inputs.md) says speech often sits around 0.05 to 0.2, a comment in oav-p5 says 0.01 to 0.05. They depend on the microphone and the room, and have not been measured here, so no typical number is given.
Related
micLevelHow loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
bassHow much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
beatTrue for one frame each time the microphone (or the drum machine) hears a beat.
kickDetected()You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
micSensitivity()Makes the microphone more or less sensitive: micSensitivity(2) is twice as sensitive, micSensitivity() says what it is now.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
Whether notes are being heard right now: true once startSound() has finished, false before and after.
Read it in draw() like mouseIsPressed: if (!soundIsOn) text('click to hear', 20, 30);. It changes only when the state of the sound changes, not every frame. Reading it never starts anything (unlike micLevel), so it is safe to check anywhere.
false does not mean startSound() was never called. To tell "loading" from "waiting for the click" from "failed", read OAVp5.sound.state, and OAVp5.sound.error for the reason of a failure.
Type
Boolean
True only while OAVp5.sound.state is 'on'. It is false at the start, while sound waits for a click, while it loads, after stopSound() and after an error.
Before it has a value
It becomes true only after startSound() (or 🔊 in the 🎹 panel, or ?oav_sound=piano), and after the click the browser wants.
Notes
A sketch that already declares its own variable called soundIsOn keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.sound.state === 'on' instead.
Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
Call it once in setup(): startSound('organ'). From then on every note you play is heard: a MIDI keyboard, the 🎹 on screen, a phone, midi.play(60, 0.8). Your own notePressed() still gets the note, and the sound is made before the sketch is told, so a slow callback never delays it. Pads on MIDI channel 10 (the General MIDI drum channel) play the drum kit instead of the instrument, and the sustain pedal holds notes.
The instrument ids are piano, epiano, organ, strings, choir, harp, marimba, vibraphone, xylophone, glockenspiel, music-box and the synth presets synth/pad, synth/saw-lead, synth/square-lead, synth/pluck, synth/supersaw, synth/bass, synth/acid, synth/brass, synth/bell, synth/sub and synth/chip. Calling startSound(id) while sound plays switches the instrument. An id that does not exist is listed in the console together with the real ones: an instrument that is already playing stays, and at the first call the warm pad synth/pad plays.
Nothing is loaded before the first call (or before 🔊 in the 🎹 panel): @openav/sound comes from next to oav-p5.js and Tone.js 15 from jsDelivr (the tone option points it somewhere else). Only the piano plays recorded samples, about 2 MB the first time; the electric piano sounds meanwhile (or, when you switch from another instrument, that one keeps sounding), and OAVp5.sound.instrument changes to piano when the samples are in. Every other instrument is made by Tone.js and has no files to load.
Before you call it
A browser lets a page make sound only after a click or a key press on the page, and a key on a MIDI keyboard does not count. Calling startSound() in setup() is fine: OAVp5.sound.state is 'waiting' and a card beside the 🎹 says "Sound is ready but silent"; the first click or key press turns sound on. soundIsOn tells when it has.
Parameters
Name
Type
Description
id
String
An instrument id, for example 'organ' or 'synth/pluck' (the list is in the description). Leave it out to keep the instrument chosen before; at the very first call that is the piano. A value that is not a non-empty string counts as left out. optionaldefault 'piano'
Returns
Promise<Boolean> Resolves to true once sound is on. When the browser wants a click first, that is after the first click or key press on the page. Resolves to false when the loading option sound is off, when stopSound() came first, or when Tone.js could not be loaded.
Notes
No code at all: ?oav_sound=piano on the page address (any instrument id; on OpenProcessing ?sketch_oav_sound=piano) makes any sketch heard from the first click, and ?oav_sound=off keeps it silent: startSound() then does nothing and resolves to false. In the 🎹 panel, 🔊 turns sound on and off and shows a picker with every instrument.
Sound is separate from the drum machine: startDrums() has its own kit and is heard from the first click without startSound().
let playing = 60;
function setup() {
createCanvas(400, 300);
startSound('synth/pluck'); // every key you play is heard
}
function mousePressed() {
playing = 48 + floor(mouseX / 25); // low on the left, high on the right
midi.play(playing, 0.8);
}
function mouseReleased() {
midi.play(playing, 0);
}
function draw() {
background(soundIsOn ? 30 : 90);
fill(255);
textAlign(CENTER, CENTER);
text(soundIsOn ? 'click to play a note' : 'click once to turn sound on', width / 2, height / 2);
}
The first click turns sound on and the background darkens; the clicks after that play a note, higher to the right.
const IDS = ['synth/pluck', 'organ', 'marimba', 'piano'];
function setup() {
createCanvas(400, 300);
startSound(IDS[0]);
}
function keyPressed() {
const i = Number(key) - 1;
if (IDS[i]) startSound(IDS[i]); // switches while it plays
}
function draw() {
background(20);
fill(255);
const s = OAVp5.sound.status; // the instrument being loaded
text('keys 1-4 pick an instrument', 20, 30);
text('playing: ' + OAVp5.sound.instrument, 20, 60);
text('loading: ' + s.id + ' (' + s.state + ', ' + nf(s.progress * 100, 1, 0) + '%)', 20, 90);
}
Press 4 for the piano: "loading" shows its samples arriving while "playing" still names the instrument you hear, and flips to piano when it is ready.
Related
stopSound()Turns sound off and lets every ringing note go; the instrument stays loaded, so startSound() brings it back at once.
soundIsOnWhether notes are being heard right now: true once startSound() has finished, false before and after.
OAVp5.sound.stateWhere sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.
OAVp5.sound.instrumentThe id of the instrument you hear right now, or null before sound has loaded.
notePressed()You define this function; oav-p5 calls it each time a key (or a pad) goes down.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
Turns sound off and lets every ringing note go; the instrument stays loaded, so startSound() brings it back at once.
It releases every note that is ringing, sets soundIsOn to false and OAVp5.sound.state to 'off'. Notes you play afterwards are not heard (the sketch's notePressed() is still called). Tone.js and the instrument stay loaded: startSound() afterwards is on again at once, without a new download and without waiting for another click.
It also works while sound is still loading or waiting for the first click: the pending startSound() then resolves to false.
Returns
Promise<Boolean> A promise that is already resolved to false, the new state of the sound.
Notes
It does not touch the drum machine (use stopDrums()) or the microphone (stopMic()).
The object behind the sound names: the state, the instrument, the error, the loading progress and the loaded parts of @openav/sound.
OAVp5.sound (also reachable as midi.sound) is what startSound(), stopSound() and soundIsOn are built on. Read state to know where sound is, instrument for what you hear, status for the loading progress and error for a failure. start(id) and stop() are the same as the two global functions. shell, engine and module hand you the loaded @openav/sound parts for the rare case that you need them.
Notes
For a sketch the global names are enough. This object is for tools, the console and debugging: OAVp5.sound.state is the first thing to look at when nothing is heard.
The message of the last thing that went wrong while loading sound, or null when nothing has.
It is the error the browser reported when @openav/sound or Tone.js could not be loaded. A card beside the 🎹 already tells the person at the keyboard what to do (try another network, or point the tone option at another copy of Tone.js); read this value to log it or to show your own message. To know whether sound is failing right now, use OAVp5.sound.state, not this.
Type
String | null
Set when the state becomes 'error'. It stays until the page is reloaded: a later try that works does not clear it.
The id of the instrument you hear right now, or null before sound has loaded.
After a first startSound('piano') the piano's samples (about 2 MB) download first, and the electric piano plays meanwhile, so the first key is never silent: instrument says 'epiano' until the piano is ready and 'piano' after. When you switch to the piano from another instrument, that one keeps sounding and instrument names it until the samples are in. How far the loading is, and which instrument it is on, is in OAVp5.sound.status.
Type
String | null
Changes when the instrument changes, which for a sampled instrument is when its samples are in.
Where sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.
'off' is where it starts and where stopSound() leaves it. startSound() moves it to 'loading' while @openav/sound and Tone.js arrive, to 'waiting' when everything is ready but the browser has not yet had its click, and to 'on' when notes are heard. 'error' means the sound parts could not be loaded (a network, a school firewall or an ad blocker); the message is in OAVp5.sound.error, and calling startSound() again tries once more.
soundIsOn is the same thing reduced to state === 'on'.
Type
String
Range
off | waiting | loading | on | error
Starts as 'off' and changes whenever the sound moves to another stage.
How the instrument is loading: { id, state, progress, error }, with progress from 0 to 1.
It is about the instrument, not about the whole sound: OAVp5.sound.state is the whole sound. Only the piano has files to load, so for every other instrument progress is 1 almost at once. While a sampler loads, OAVp5.sound.instrument is still the one you hear and status.id is the one on its way (at the very first call it may name the electric piano that stands in, and then the piano).
A new object on every read. id is the instrument the loader is working on, state is 'idle' before anything loads, 'loading', 'ready' or 'error', and progress is the fraction of the files loaded.
The 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.
text
id category name
piano keys Grand piano (Salamander), 30 samples, CC-BY 3.0
epiano keys Electric piano (two FM pairs and a tremolo)
xylophone mallets Xylophone (sounds an octave up)
marimba mallets Marimba
vibraphone mallets Vibraphone (motor tremolo, key-up damps)
glockenspiel mallets Glockenspiel (two octaves up)
music-box mallets Music box (two octaves up)
strings strings String ensemble
harp plucked Harp (Karplus-Strong)
organ organ Drawbar organ (rotary tremolo)
choir voice Choir pad (aah)
synth/pad synth Warm pad (the default)
synth/saw-lead synth Saw lead
synth/square-lead synth Square lead
synth/pluck synth Pluck
synth/supersaw synth Supersaw
synth/bass synth Analog bass
synth/acid synth Acid
synth/brass synth Synth brass
synth/bell synth FM bell
synth/sub synth Sub bass
synth/chip synth Chip (8-bit)
Loading the package registers all of them, except those whose id you registered first. Each one is a small file in the package's instruments/ folder. The piano loads its samples only when it is picked, and falls back to the electric piano if they cannot be fetched. The organ, the strings, the choir, the electric piano and the vibraphone bring params of their own.
Type
Object[]
An array of 22 instrument definitions, in the order of the table.
Examples
js
import { BUILTIN_INSTRUMENTS, getInstrument } from'@openav/sound';
console.log(BUILTIN_INSTRUMENTS.length); // 22
console.log(BUILTIN_INSTRUMENTS.slice(0, 3).map((d) => d.id).join(' ')); // piano epiano xylophone
console.log(getInstrument('piano') === BUILTIN_INSTRUMENTS[0]); // true: they are the registered ones
console.log(getInstrument('organ').params.map((p) => p.key).join(' ')); // drawbars rotor
module How many built-in instruments there are, the first three, and that the registry holds these very objects. The organ lists the two params of its own.
Related
listInstruments()Lists every registered instrument, in the order they were registered, or only the ones of one category.
SYNTH_PRESETSThe 11 synth presets as plain data: pad, saw lead, square lead, pluck, supersaw, bass, acid, brass, bell, sub and chip.
DEFAULT_INSTRUMENTThe instrument used when none is named: 'synth/pad', the warm three-saw pad that was the framework's first voice.
registerInstrument()Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
getInstrument()Finds a registered instrument definition by its id, or null.
The instrument categories in picker order: keys, mallets, strings, plucked, organ, voice, synth, drums and other.
The category of an instrument definition picks its group in the picker. A category that is not in this list is allowed: it sorts after these, and its label is its id. Every category also has a Chinese name, which the picker uses on a Chinese page.
Type
Object[]
Nine items { id, name: { en, zh } }, such as { id: 'mallets', name: { en: 'Mallets', zh: … } }.
module Logs the default instrument and its name, and that an engine with no options starts on it.
Related
toneEngine()The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
BUILTIN_INSTRUMENTSThe 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.
SYNTH_PRESETSThe 11 synth presets as plain data: pad, saw lead, square lead, pluck, supersaw, bass, acid, brass, bell, sub and chip.
instrumentGroups(lang = 'en', list = listInstruments())
Groups the instruments by category in picker order: [{ id, label, items }], with empty groups left out.
It is the data behind the picker, so you can build your own menu, a list or a grid of buttons. A category that is not in CATEGORIES is allowed; it sorts after the known ones and its label is its id.
Parameters
Name
Type
Description
lang
String
The language of the group labels. optionaldefault 'en'
list
Object[]
The instruments to group. Left out, all the registered ones. optionaldefault listInstruments()
Returns
Object[][{ id, label, items: [definition…] }]: the groups in the order of CATEGORIES, then any categories of your own, sorted by name.
Examples
js
import { instrumentGroups } from'@openav/sound';
for (const g of instrumentGroups('en')) console.log(g.label + ' (' + g.items.length + ')');
// Keys (2), Mallets (5), Strings (1), Plucked (1), Organ (1), Voice (1), Synth (11)
console.log(instrumentGroups('zh')[1].label); // the Chinese label of the mallets group
module The seven groups the built-in instruments make, with how many each holds (no drums yet, so that group is left out), and a group label in Chinese.
Related
listInstruments()Lists every registered instrument, in the order they were registered, or only the ones of one category.
CATEGORIESThe instrument categories in picker order: keys, mallets, strings, plucked, organ, voice, synth, drums and other.
instrumentName()Gives an instrument's display name in a language, falling back to English.
mountSoundPicker()Puts an instrument menu on the page: one group per category, a loading bar for samplers and the credit line an instrument asks for.
Gives an instrument's display name in a language, falling back to English.
Instruments carry name as a string or as { en, zh }. This hides which, so your own menu can just call instrumentName(def, lang).
Parameters
Name
Type
Description
def
Object
An instrument definition. null or undefined gives ''.
lang
String
'en', 'zh'… A longer tag such as 'zh-TW' is tried by its first two letters. optionaldefault 'en'
Returns
String The name in that language, else the English one. A definition whose name is a plain string gives that string.
Examples
js
import { getInstrument, instrumentName } from'@openav/sound';
const bar = getInstrument('glockenspiel');
console.log(instrumentName(bar), instrumentName(bar, 'zh')); // Glockenspiel, then its Chinese name
console.log(instrumentName(bar, 'fr')); // Glockenspiel, English as the fallback
console.log(JSON.stringify(instrumentName(null))); // ""
module The name in English, in Chinese, in a language that has no entry (English again), and for nothing at all.
Related
instrumentGroups()Groups the instruments by category in picker order: [{ id, label, items }], with empty groups left out.
getInstrument()Finds a registered instrument definition by its id, or null.
mountSoundPicker()Puts an instrument menu on the page: one group per category, a loading bar for samplers and the credit line an instrument asks for.
Lists every registered instrument, in the order they were registered, or only the ones of one category.
The 22 built-in instruments are registered when @openav/sound loads, so a fresh page lists those. For a list shaped for a menu, with the groups in order, use instrumentGroups().
Parameters
Name
Type
Description
category
String
In the options { category }: only the instruments of this category ('mallets'). optionaldefault null
module How many instruments there are, and which ones are mallets and keys.
Related
instrumentGroups()Groups the instruments by category in picker order: [{ id, label, items }], with empty groups left out.
getInstrument()Finds a registered instrument definition by its id, or null.
BUILTIN_INSTRUMENTSThe 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.
CATEGORIESThe instrument categories in picker order: keys, mallets, strings, plucked, organ, voice, synth, drums and other.
Makes a struck-bar or tine instrument from data: a list of partials that ring and die away, and a short noise for the mallet.
A xylophone bar, a glockenspiel bar and a music-box tine are the same physics with different numbers: a strike excites a few modes, each rings at its own multiple of the pitch and dies at its own rate. So a mallet instrument here is data, not code. The built-in xylophone is this short:
Harder strikes brighten: the upper partials grow faster with velocity, as a real mallet does. The voices are raw Web Audio nodes built for each strike and left to the garbage collector.
A credit line, a string or { text, url }. optionaldefault null
params
Object[]
Params of its own: [{ key, label, min, max, def }]. Handled by the set function. optional
defaults
Object
Starting values for the shared params. The default opens the filter, because bars are bright. optionaldefault { cutoff: 8000 }
partials
Object[]
The modes of the bar: [{ ratio, gain, decay }]. ratio is the frequency as a multiple of the note (1 is the note itself, 3 a twelfth above it), gain the level relative to the others and decay the seconds to near silence at middle C. High notes die faster than low ones. A partial above 45% of the sample rate is skipped. Required: without it this throws.
strike
Object
The mallet, a few milliseconds of band-passed noise: { ratio, q, gain, decay }, where ratio sets its pitch as a multiple of the note, q its width, gain its level and decay its length in seconds. optionaldefault null
damp
Number
Seconds in which letting go of a key mutes the bar. 0 lets the bars ring free (xylophone, glockenspiel); a vibraphone has a damper. optionaldefault 0
pitchDecay
Number
How much faster high notes die than low ones. The decay is scaled by (261.63 / frequency) ^ pitchDecay, held between 0.2 and 3. 0 makes every note last the same. optionaldefault 0.6
transpose
Number
Semitones to shift what is played (the xylophone sounds an octave up: 12). optionaldefault 0
gain
Number
The instrument's level, in dB. optionaldefault 0
attack
Number
Seconds the partials take to reach full level. optionaldefault 0.0015
tremolo
Object
A level wobble over all voices, the vibraphone's motor: { frequency, depth, spread }. optionaldefault null
maxVoices
Number
How many notes can ring at once. The oldest is faded out in 30 ms when it is exceeded. optionaldefault 40
set
Function
set({ key, value, tremolo, Tone }), called when one of the instrument's own params changes, so it can change the tremolo or anything else. optionaldefault null
Returns
Object An instrument definition, not yet registered: pass it to registerInstrument(). tail is set to min(8, longest decay × 1.5 + 0.2) seconds.
module Two partials and a mallet make a steel-bar instrument that validates and registers. It is only built, and heard, once the sound engine plays it.
Related
registerInstrument()Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
registerSynthPreset()Adds a synth preset to every picker as synth/ plus its id: describe the voice, the envelope and the effects as data.
validateInstrument()Checks an instrument definition and lists everything wrong with it as readable sentences. An empty list means it is valid.
BUILTIN_INSTRUMENTSThe 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.
Puts an instrument menu on the page: one group per category, a loading bar for samplers and the credit line an instrument asks for.
A native <select> with one <optgroup> per category (CATEGORIES), so it works with a keyboard and on a phone and needs no styling to be usable. It lists whatever is registered, and it refreshes itself when an instrument is registered or removed (onInstrumentsChange()). Choosing one calls sound.setInstrument(id, { remember: true }). While a sampler loads it shows a bar and a percentage; if the load fails it says so, and says which instrument plays instead. A credit line appears under the menu for an instrument that has one: the Salamander piano is CC-BY, and its credit is always shown.
Parameters
Name
Type
Description
container
HTMLElement
The element to put the picker in.
sound
Sound
The Sound to control. The menu follows its instrument and its loading.
lang
'en' | 'zh'
The language of the labels and instrument names. The default is 'zh' when the page's <html lang> starts with zh, else 'en'. optionaldefault from <html lang>
className
String
The class of the picker. The parts get -row, -label, -status, -meter and -credit added to it, for your own styling. The picker brings no CSS. optionaldefault 'oav-sound-pick'
label
String
The label in front of the menu. Left out, it is the word "instrument" (in Chinese when lang is 'zh'). optionaldefault null
enableOnPick
Boolean
Choosing an instrument also calls sound.enable() if the sound is off. A choice is a click, which is the gesture a browser wants before it plays audio. optionaldefault true
Returns
Object{ el, select, refresh(), dispose() }: the element it made, its <select>, a function that rebuilds the list, and a function that takes it away.
module A menu of the 22 built-in instruments, in groups, appears on the page. Registering one more adds it to the menu at once.
Related
SoundMakes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
instrumentGroups()Groups the instruments by category in picker order: [{ id, label, items }], with empty groups left out.
onInstrumentsChange()Calls your function whenever an instrument is registered or removed, so a menu can refresh. Returns a function that stops listening.
registerInstrument()Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
CATEGORIESThe instrument categories in picker order: keys, mallets, strings, plucked, organ, voice, synth, drums and other.
Calls your function whenever an instrument is registered or removed, so a menu can refresh. Returns a function that stops listening.
mountSoundPicker() listens this way, which is why an instrument registered after the picker was built still shows up. A replacement with { replace: true } also reports register. A call to unregisterInstrument() for an id that is not there reports nothing.
Parameters
Name
Type
Description
cb
Function
cb({ type, id }) with type'register' or 'unregister'. An error inside it is logged and does not stop the others.
Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
A sound you can pick is plain data plus one factory. Registering it is all it takes: import the file before you start, and every mountSoundPicker() on the page lists it under its category, with no change to the package. The registry lives on globalThis, so two copies of this module on one page share one list. Built-in instruments only fill empty slots: register your own 'piano' first and you keep it. The definition:
text
id lowercase letters, digits and "-", at most one "/": 'kalimba', 'me/kalimba'
name a string, or { en, zh }
category keys · mallets · strings · plucked · organ · voice · synth · drums · other, or your own
create create(Tone, ctx) → { noteOn(note, vel01, time), noteOff(note, time), set?(key, v),
releaseAll?(time), ready?: Promise, dispose() }
params your own knobs [{ key, label, min, max, def }], shown as sound/<key> while it plays
defaults starting values for the shared params: { cutoff, space, volume }
credit a string, or { text, url }, shown in the picker
gain level in dB transpose semitones tail seconds the old one may ring after a switch
fallback the id of an instrument to play if `ready` rejects
create(Tone, ctx) runs on enable and on every switch to the instrument, with Tone.js already loaded. ctx is { output, rawOutput, rawContext, baseUrl, options, onProgress }: connect Tone nodes to output (raw Web Audio nodes to rawOutput), baseUrl is where sample folders live, options is what was passed in toneEngine({ instruments: { <id>: … } }), and onProgress(0..1) shows a loading bar. If ready is a promise, the picker shows progress and the old sound keeps playing until it resolves. An instrument that makes mallet sounds from data can use modalInstrument(), and one that is a Tone synth with settings, registerSynthPreset().
Parameters
Name
Type
Description
def
Object
The instrument definition, see the table below.
replace
Boolean
In the options { replace }: allow replacing an instrument that has the same id. Without it a duplicate id throws. optionaldefault false
Returns
Object The definition you gave. It throws a TypeError listing everything wrong if the definition is not valid (see validateInstrument()), and an Error for a duplicate id.
static A whole instrument in one file: an FM synth with a knob of its own. Import it before the sound starts, and "Kalimba" appears under Mallets in every picker.
Related
validateInstrument()Checks an instrument definition and lists everything wrong with it as readable sentences. An empty list means it is valid.
unregisterInstrument()Removes a registered sound, for tests and hot reload. Returns whether it was there.
getInstrument()Finds a registered instrument definition by its id, or null.
modalInstrument()Makes a struck-bar or tine instrument from data: a list of partials that ring and die away, and a short noise for the mallet.
registerSynthPreset()Adds a synth preset to every picker as synth/ plus its id: describe the voice, the envelope and the effects as data.
toneEngine()The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
CATEGORIESThe instrument categories in picker order: keys, mallets, strings, plucked, organ, voice, synth, drums and other.
Adds a synth preset to every picker as synth/ plus its id: describe the voice, the envelope and the effects as data.
A preset names a voice, its settings in Tone.js's vocabulary, and an optional effects chain, and nothing else. id is lowercase letters, digits and - (it becomes synth/<id>), name is a string or { en, zh }. voice is 'synth' (oscillator and envelope), 'mono' (adds a resonant filter with its own envelope), 'fm' or 'am'. The Tone settings are oscillator, envelope, filter, filterEnvelope, harmonicity, modulationIndex, modulation, modulationEnvelope, detune and portamento. The rest: octave (a whole number of octaves, -1 for a bass), gain (dB, to level it against the others), polyphony, defaults (starting cutoff, space, volume), params, category and credit, as in registerInstrument().
effects is a list of { type, …settings } that the sound passes through in order. The types are chorus, delay, pingpong, distortion, vibrato, tremolo, phaser, autofilter, filter, eq, compressor, chebyshev and widener; the other keys are the effect's own Tone settings, including wet. A preset that is wrong throws a TypeError with the reasons.
Parameters
Name
Type
Description
preset
Object
The preset: { id, name, voice, … }, see below.
opts
Object
The options of registerInstrument(): { replace: true } swaps an existing one. optional
Returns
Object The instrument definition that was registered (synthInstrument() of the preset).
The 30 notes the Salamander piano has samples for: every minor third from A0 (MIDI 21) to C8 (108).
The sampler shifts the pitch of the notes in between by at most a semitone and a half, which is inaudible on a piano. There is only one velocity layer, so loud against soft is a gain curve, not a different hammer.
Type
Number[]
[21, 24, 27, …, 108]: 30 MIDI note numbers, three semitones apart (the A, C, D♯ and F♯ of every octave).
module The full map has 30 files; a map of two notes; and the full address of one sample.
Related
SALAMANDER_NOTESThe 30 notes the Salamander piano has samples for: every minor third from A0 (MIDI 21) to C8 (108).
SAMPLES_URLThe address of the folder with the built-in sample sets, when nobody says otherwise: openaudiovisual.com/packages/sound/samples/.
BUILTIN_INSTRUMENTSThe 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.
The address of the folder with the built-in sample sets, when nobody says otherwise: openaudiovisual.com/packages/sound/samples/.
It is the default baseUrl of toneEngine(). The piano reads its 30 mp3 files from the salamander/ folder under it. The server sends CORS headers, so any page can load them. createShow() points the engine at the copy next to the page instead. To use your own copy, give baseUrl, or instruments: { piano: { baseUrl } } for the piano alone.
Type
String
'https://openaudiovisual.com/packages/sound/samples/', ending in a slash.
Examples
js
import { SAMPLES_URL, toneEngine } from'@openav/sound';
console.log(SAMPLES_URL); // https://openaudiovisual.com/packages/sound/samples/const local = toneEngine({ instrument: 'piano', baseUrl: '/my/samples/' }); // the piano then reads /my/samples/salamander/…
module Logs the default folder and shows how to point the engine at your own.
Related
toneEngine()The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
TONE_CDNThe address Tone.js is loaded from when the engine is enabled: Tone 15.0.4 on jsDelivr.
salamanderMap()Builds the sample map of the Salamander piano for Tone.Sampler: { 'A0': 'A0.mp3', 'D#1': 'Ds1.mp3', … }.
Makes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
Sound is the shell that wires an engine into an app. Once await sound.enable() has run, every midi/note/on and midi/note/off on the bus is played, whatever sent it: a MIDI keyboard, the on-screen piano (mountKeys()), a SimPlayer, a TakePlayer or a phone. Notes on a channel in the engine's skipChannels are left out (the Tone engine skips channel 10, which belongs to the drum machine).
The engine's params become sound/* params, so a knob, the timeline or a hand can play the filter the same way it plays the visuals: sound is performable state, not a side effect. Every frame, call sound.update(state) with the resolved param values, and the sound/* ones are pushed into the engine. The instrument is a discrete choice (setInstrument() or the picker); what you perform continuously stays a param.
Before you call it
sound.enable() from a click: browsers start audio only after a user gesture. With toneEngine() it also loads Tone.js from the network (jsDelivr) the first time. Before enable() nothing is heard, and the bus is not listened to.
Parameters
Name
Type
Description
signals
Signals
The Signals bus. Sound listens to midi/note/on{ note, vel, ch }, midi/note/off{ note, ch } and midi/cc/64 (the sustain pedal, down at 0.5 or more), and pulses sound/instrument{ id, name } when the instrument changes.
params
Params
A Params registry to add the engine's params to, named sound/<key> (sound/cutoff, sound/space, sound/volume and the instrument's own). Left out, no params are registered. optionaldefault null
engine
Object
The sound engine, such as toneEngine() or drumEngine(). The contract is { params, enable(), noteOn(note, vel01), noteOff(note), set(key, value), dispose() }, and optionally setInstrument, instrument, status, subscribe, sustain(down) and skipChannels.
instrument
String
The instrument to start with, for an engine that has instruments ('piano', 'synth/pluck'). An unknown id is warned about in the console and ignored. optionaldefault null
remember
Boolean | String
Keep the instrument the picker last chose, per page, in localStorage, and let it win over instrument the next time. A string is your own storage key; the default key is openav:sound:<page path>. false turns it off. Choices made from code are not remembered unless you ask (setInstrument(id, { remember: true })). optionaldefault true
picker
Boolean
Whether a console should show the instrument picker. It is stored as sound.picker for whoever builds the interface; Sound itself draws nothing. optionaldefault true
Starts the engine and begins listening to the bus. Returns a promise of true. Call it from a click. Calling it again returns the same promise; if it fails you can try again. Emits { type: 'enabled', id } to onChange listeners.
Switches instrument, live: setInstrument(id, { remember = false }). Held notes are released on the old one and the new one takes over once it is ready (a sampler keeps the old sound while its files load). Returns a promise: true once the instrument plays, false if the id is unknown, the engine has no instruments or the choice was replaced by a newer one. remember: true stores the choice for this page.
Listens for what happens: cb({ type, id, progress, error }) with type'instrument', 'loading', 'ready', 'error' or 'enabled'. Returns a function that stops listening.
Stops listening to the bus, drops the listeners and disposes the engine.
Notes
From a p5 sketch you do not build this: startSound() does it, loading this package only when it is called.
Examples
js
import { Sound } from'@openav/sound';
import { Signals } from'@openav/core';
const signals = new Signals();
const heard = [];
const engine = { // the smallest engine: it only writes down what it is told
params: [{ key: 'cutoff', label: 'Cutoff', min: 100, max: 8000, def: 2500 }],
async enable() {},
noteOn: (note, vel) => heard.push('on ' + note + ' ' + vel),
noteOff: (note) => heard.push('off ' + note),
set: (key, value) => heard.push(key + '=' + value),
dispose() {},
};
const sound = new Sound({ signals, engine });
await sound.enable();
signals.pulse('midi/note/on', { note: 60, vel: 0.8, ch: 1 });
signals.pulse('midi/note/off', { note: 60, ch: 1 });
sound.update({ 'sound/cutoff': 900, hue: 0.3 }); // only the sound/* keys go to the engine
console.log(heard.join(', ')); // on 60 0.8, off 60, cutoff=900
console.log(sound.enabled, sound.choosable, sound.instrument); // true false null
module A bus with a stand-in engine: notes on the bus reach the engine, and of the state only sound/cutoff does. The engine has no instruments, so choosable is false.
js
import { Signals } from'@openav/core';
import { mountKeys } from'@openav/keys';
import { Sound, toneEngine, mountSoundPicker } from'@openav/sound';
const signals = new Signals();
const sound = new Sound({ signals, engine: toneEngine({ instrument: 'piano' }) });
mountKeys(document.getElementById('app'), { signals }); // an on-screen piano that publishes midi/note/on
mountSoundPicker(document.getElementById('app'), sound); // choosing an instrument also turns the sound on
document.body.addEventListener('click', () => sound.enable(), { once: true });
sound.onChange((e) => console.log(e.type, e.id));
static Click anywhere to start the sound, then click the piano keys: you hear the Salamander grand piano. Pick another instrument from the menu while you play.
To be confirmed
The Tone.js engine was not run for this page: there was no network and no speakers. What is written here comes from the code and from the package tests, which use a stand-in for Tone.js, and the examples that run use a stand-in engine.
Related
toneEngine()The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
mountSoundPicker()Puts an instrument menu on the page: one group per category, a loading bar for samplers and the credit line an instrument asks for.
registerInstrument()Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
SOUND_PARAMSThe three params every instrument shares: cutoff (the filter), space (the reverb) and volume.
startSound()Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
The three params every instrument shares: cutoff (the filter), space (the reverb) and volume.
cutoff is the low-pass filter's frequency in Hz, space is how much reverb is mixed in (the wet amount, 0..1), volume is the level in dB. They sit after the instrument on the output chain, so the timeline, a knob or a hand keep performing them across instrument switches. In a show they are the params sound/cutoff, sound/space and sound/volume. An instrument may suggest other starting values through its defaults, and automation and overrides always win over a suggestion. An instrument's own params cannot use these keys.
Type
Object[]
[{ key: 'cutoff', min: 100, max: 8000, def: 2500 }, { key: 'space', min: 0, max: 1, def: 0.3 }, { key: 'volume', min: -36, max: 0, def: -8 }], each also with a label.
Reads the modules.sound option of createShow() (true, an instrument id, or an object) into one normalized object.
It is how createShow({ modules: { sound: … } }) understands its option, so sound: true, sound: 'piano' and sound: { instrument: 'organ', remember: false, picker: false } all work. You only need it if you write something that takes the same option.
Parameters
Name
Type
Description
value
Boolean | String | Object
true for the defaults, an instrument id ('piano'), or { instrument, remember, picker, engine, …toneEngine options }. Anything falsy gives null.
Returns
Object | null{ engine, instrument, remember, picker, engineOptions } (engine and instrument are null when not given, remember and picker are true), or null for a falsy value. engineOptions holds the rest of an object: baseUrl, cdn, instruments…. An instrument that is not a string, or a value of another kind, throws a TypeError.
module The three accepted shapes, a falsy value that turns the sound off, and the error for something that is none of them.
Related
SoundMakes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
toneEngine()The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
The 11 synth presets as plain data: pad, saw lead, square lead, pluck, supersaw, bass, acid, brass, bell, sub and chip.
Each preset is what synthInstrument() turns into an instrument synth/<id>. They are data in Tone.js's own vocabulary (oscillator, envelope, filter, filterEnvelope, harmonicity…), a good place to look when you write a preset of your own for registerSynthPreset(). The pad is the framework's original voice: three detuned sawtooth waves with a quick attack and a long release.
module Lists the preset ids and reads a few settings of the bass: a mono synth with a filter, an octave down.
Related
synthInstrument()Turns a synth preset into an instrument definition whose id is synth/ followed by the preset id, without registering it.
registerSynthPreset()Adds a synth preset to every picker as synth/ plus its id: describe the voice, the envelope and the effects as data.
validateSynthPreset()Checks a synth preset and lists what is wrong with it. An empty list means it is valid.
BUILTIN_INSTRUMENTSThe 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.
Turns a synth preset into an instrument definition whose id is synth/ followed by the preset id, without registering it.
The definition's create() builds one Tone.PolySynth of the preset's voice (synth, mono, fm or am), with the Tone settings the preset carries, a chain of its effects, and its polyphony (16 voices if left out). Use it when you want the definition for yourself (to adjust it before registering); otherwise registerSynthPreset() does both steps.
Object An instrument definition: id is 'synth/' + preset.id, name the preset's (or its id), category the preset's or 'synth', transpose is octave × 12, gain, defaults, params and credit come from the preset, tail is min(8, release × 1.5 + 0.5) seconds and preset is the preset itself.
The address Tone.js is loaded from when the engine is enabled: Tone 15.0.4 on jsDelivr.
A module URL that toneEngine() imports on enable(). To host Tone.js yourself, pass your own as the cdn option of toneEngine().
Type
String
'https://cdn.jsdelivr.net/npm/tone@15.0.4/+esm'.
Examples
js
import { TONE_CDN, toneEngine } from'@openav/sound';
console.log(TONE_CDN); // https://cdn.jsdelivr.net/npm/tone@15.0.4/+esmconst own = toneEngine({ cdn: '/vendor/tone.mjs' }); // your own copy, used when enable() runs
module Logs the default address and shows where an own copy would be given.
Related
toneEngine()The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
SAMPLES_URLThe address of the folder with the built-in sample sets, when nobody says otherwise: openaudiovisual.com/packages/sound/samples/.
The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
Tone.js stays a neighbour, not a dependency: it is loaded from the network the first time enable() runs, so an app that never turns the sound on pays nothing. After enable() the instrument's output goes through a low-pass filter (cutoff, starting at 2500 Hz), a reverb (space, the wet amount, starting at 0.3), a volume (volume, in dB, starting at -8) and a limiter at -1 dB, so a loud organ never clips the room. These three are the SOUND_PARAMS shared by every instrument. Their starting values can follow the instrument (a piano wants the filter open), and the instrument may add params of its own.
setInstrument(id) switches live and returns a promise of whether it worked. Before enable() it only remembers the choice. Held notes are released on the old instrument and its tail is left to ring for a few seconds. A sampler (the piano) keeps the old sound until its files are in; if they cannot load, the engine plays the instrument's fallback (the piano falls back to the electric piano), so a performance never goes silent because of one failed download. subscribe(cb) tells you about instrument, loading, ready and error events. sustain(down) is the pedal: note-offs wait until it comes up. releaseAll() is a panic button.
You usually pass the engine to a Sound, which is what listens to the bus. Used on its own, noteOn(60, 0.8) plays a note.
Before you call it
await engine.enable() from a click: it loads Tone.js from the network, then starts its audio. Before that, noteOn() does nothing and setInstrument() only remembers the choice.
Parameters
Name
Type
Description
cdn
String
Where Tone.js is imported from when the engine is enabled. The default is TONE_CDN. A module URL, so you can host your own copy. optionaldefault TONE_CDN
instrument
String
The instrument to start with. An unknown id is warned about and synth/pad is used. optionaldefault 'synth/pad'
baseUrl
String
The folder that holds the built-in sample sets (salamander/…). The default is SAMPLES_URL. optionaldefault SAMPLES_URL
instruments
Object
Options for single instruments, handed to their create() as ctx.options: { piano: { baseUrl: '/my/salamander/' } }. optionaldefault {}
drumChannel
Number | null
Notes on this MIDI channel belong to a drum engine (the drum machine sends General MIDI channel 10) and are not played here. null plays everything. optionaldefault 10
module Looks at an engine before it is enabled: its instrument, the params it will offer (the strings suggest a different starting cutoff and space, and add two of their own), and that channel 10 is left to the drums.
js
import { toneEngine } from'@openav/sound';
const engine = toneEngine({ instrument: 'epiano' });
document.body.addEventListener('click', async () => {
await engine.enable(); // loads Tone.js from the network, starts the audio
engine.set('space', 0.6); // more reverb
[60, 64, 67].forEach((note) => engine.noteOn(note, 0.8)); // a C major chord
setTimeout(() => engine.releaseAll(), 1500);
}, { once: true });
static Click the page: an electric piano plays a C major chord, with more reverb than usual, and lets go after a second and a half.
To be confirmed
The Tone.js engine was not run for this page: there was no network and no speakers. What is written here comes from the code and from the package tests, which use a stand-in for Tone.js, and the examples that run use a stand-in engine.
Related
SoundMakes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
registerInstrument()Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
SOUND_PARAMSThe three params every instrument shares: cutoff (the filter), space (the reverb) and volume.
TONE_CDNThe address Tone.js is loaded from when the engine is enabled: Tone 15.0.4 on jsDelivr.
SAMPLES_URLThe address of the folder with the built-in sample sets, when nobody says otherwise: openaudiovisual.com/packages/sound/samples/.
drumEngine()A drum kit made of Web Audio synthesis, with no sample files: General MIDI drum notes go in, drum sounds come out.
module Removes an instrument; the second time there is nothing to remove.
Related
registerInstrument()Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
getInstrument()Finds a registered instrument definition by its id, or null.
onInstrumentsChange()Calls your function whenever an instrument is registered or removed, so a menu can refresh. Returns a function that stops listening.
Checks an instrument definition and lists everything wrong with it as readable sentences. An empty list means it is valid.
It checks: id (lowercase letters, digits and -, at most one /), name (a string, or { en, zh } with at least en), category (a string), create (a function), params (each with a word as key, numbers min < max, a def within them, and a key that is not one of the shared three), defaults (only cutoff, space or volume, each within its range), credit (a string, or { text, url }), gain, transpose and tail (numbers) and fallback (another instrument's id). registerInstrument() runs it and throws with these sentences.
Parameters
Name
Type
Description
def
any
The definition to check.
Returns
String[] Sentences, one per problem. [] when valid. It never throws.
Examples
js
import { validateInstrument } from'@openav/sound';
const ok = { id: 'me/ok', name: 'OK', category: 'other', create() {} };
console.log(validateInstrument(ok)); // []
console.log(validateInstrument({ ...ok, id: 'Piano', params: [{ key: 'cutoff', min: 0, max: 1, def: 0 }] }));
// [ 'id "Piano": lowercase letters, digits and "-", at most one "/" (e.g. "synth/pad")',// 'params[0] (cutoff): "cutoff" is a shared sound param; set its starting value with defaults: { cutoff: … }' ]
console.log(validateInstrument(null)); // [ 'an instrument definition is an object' ]
module A good definition gives an empty list; a definition with a capital letter in the id and a param that is already shared gets two sentences.
Related
registerInstrument()Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
validateSynthPreset()Checks a synth preset and lists what is wrong with it. An empty list means it is valid.
SOUND_PARAMSThe three params every instrument shares: cutoff (the filter), space (the reverb) and volume.
Checks a synth preset and lists what is wrong with it. An empty list means it is valid.
It checks the id (lowercase letters, digits and -), the voice (synth, mono, fm or am), that every effect has a type the package knows, and that octave is a whole number. It does not check the Tone settings, which are Tone.js's to judge.
Parameters
Name
Type
Description
p
Object
The preset to check.
Returns
String[] Sentences, one per problem. [] when valid.
How far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
A beat is a quarter note (four steps). 1 - beatPhase is biggest right on the beat and shrinks until the next one, so circle(width / 2, height / 2, 60 + (1 - beatPhase) * 80) pulses with the tempo, with no callback needed. drumPhase is the same thing across a whole bar.
Type
Number
Range
0..1
Read fresh each time you look at it. 0 on the beat, close to 1 just before the next one, then 0 again. It is 0 before the machine has played and holds still while it is stopped.
Before it has a value
Start the machine with startDrums(); until it plays, beatPhase is 0.
function setup() {
createCanvas(400, 300);
startDrums('four on floor', 120);
noStroke();
}
function draw() {
background(20);
fill(255, 120, 90);
circle(width / 2, height / 2, 60 + (1 - beatPhase) * 100); // biggest on the beat
}
A circle that pulses four times per bar: biggest on each beat, then shrinking until the next one.
Related
drumPhaseHow far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
drumStepWhich of the 16 steps of the bar the drum machine just played, from 0 to 15.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
beatTrue for one frame each time the microphone (or the drum machine) hears a beat.
Draws the rhythm on the canvas as a grid, a circle or a spiral, with a playhead and the hits lighting up, so a recording has it too.
Call it in draw() after background(): drawRhythm(20, 100, 360, 100). Each row is one drum that plays in the pattern, a hit lights up the moment it is heard (the clock drumPhase reads), and a hit between two steps, such as a triplet, sits where it falls. The grid card is part of the page and a recording never shows it, but the canvas is what a recording takes, so this is how the rhythm gets into the video.
With no box it draws a strip along the bottom for 'grid' and a square in the middle for the other two. It draws with p5's own shapes between push() and pop(), so it leaves your styles alone and works in 2D and in WEBGL, in p5 1.x and 2.x. In WEBGL the origin is the middle of the canvas: drawRhythm(-width / 2, -height / 2, width, height, 'circle') fills it.
Before you call it
Start the machine with startDrums() first. Before it plays you get an empty four-lane grid.
Parameters
Name
Type
Description
x
Number
Left edge of the box to draw in. In WEBGL the middle of the canvas is (0, 0). If any of x, y, w, h is left out, the default box is used. optional
y
Number
Top edge of the box. optional
w
Number
Width of the box. optional
h
Number
Height of the box. optional
style
String | Object
'grid' (the 16 steps with a playhead), 'circle' (a ring of 16 per drum with a hand turned by drumPhase) or 'spiral' (the last few bars wound outwards, now at the tip). An object here is taken as o, with style inside it. optionaldefault 'grid'
o
Object
Options: colors ({ kick: '#f66', … }, any p5 colour per drum), color (one colour for all), off (the empty steps), playhead (a colour, or false for none), bg (a background for the box) and bars (spiral only, 1 to 16, 4 if left out). optional
Returns
Booleantrue if it drew something, false when the box is empty (w or h is not above 0) or there is no p5 global mode to draw with.
Notes
OAVp5.drums.draw(x, y, w, h, style, o) is the same function as a method.
function setup() {
createCanvas(400, 300, WEBGL);
startDrums('four on floor', 120);
}
function draw() {
background(14, 16, 26);
drawRhythm(-width / 2, -height / 2, width, height, 'circle'); // (0, 0) is the middle in WEBGL
}
In a WEBGL canvas the box starts at (-width / 2, -height / 2), because (0, 0) is the middle.
Related
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
drumPhaseHow far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
drumStepWhich of the 16 steps of the bar the drum machine just played, from 0 to 15.
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
rhythmChanged()An event you define: it is called with the new line of rhythm every time the rhythm changes.
The tempo of the drum machine in beats per minute.
A beat is a quarter note, four steps, so one step lasts 60 / drumBpm / 4 seconds and one bar 240 / drumBpm: at 120 a bar takes 2 seconds. To change the tempo call startDrums(bpm): startDrums(drumBpm + 5) speeds up by five. Writing to drumBpm yourself changes nothing.
startDrums() keeps the tempo between 30 and 300; the slider on the grid goes from 60 to 180.
Type
Number
Range
30..300
112 until you set another one. It follows startDrums() and the tempo slider on the grid.
Notes
A sketch that declares its own variable called drumBpm keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.drums.bpm instead.
function setup() {
createCanvas(400, 300);
startDrums('four on floor', 100);
textAlign(CENTER, CENTER);
textSize(32);
}
function keyPressed() {
if (keyCode === UP_ARROW) startDrums(drumBpm + 5);
if (keyCode === DOWN_ARROW) startDrums(drumBpm - 5);
}
function draw() {
background(20);
fill(255);
text(round(drumBpm) + ' bpm', width / 2, height / 2);
}
Press the up and down arrow keys to change the tempo by 5; the number shows it.
Related
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
drumPhaseHow far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
beatPhaseHow far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
drumBarWhich bar the drum machine is in, counted from 1 when it started.
An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
Define function drumHit(lane, velocity) { … } and oav-p5 calls it for every hit of every drum, the way it calls notePressed() for a key. It is called in the frame in which the hit is heard, right before draw().
The same hit also calls the microphone's functions, so you can pick the grain you want: kickDetected() for the kick, snareDetected() for snare, clap and rim, hatDetected() for the three hats, and beatDetected() (with beat true for a frame) for kick, snare, clap and rim, once per step. Toms, crash, ride and cowbell reach only drumHit(). On one step the order is: cueFired(), barStarted(), stepStarted(), then the hits.
Parameters
Name
Type
Description
lane
String
The drum that sounded: 'kick', 'snare', 'hat' or 'clap' for the grid, and also 'openhat', 'pedalhat', 'rim', 'lowtom', 'tom', 'hightom', 'crash', 'ride' or 'cowbell' for a line of rhythm.
velocity
Number
How hard, from 0.15 to 1. It varies a little from hit to hit, because the machine plays slightly softer or louder on purpose.
Notes
While the tab is hidden nobody sees a frame: the clock keeps counting, but no hit is told, and nothing is told twice when the tab comes back. An error thrown inside drumHit() is printed, with its file and line, and the next hit still comes.
let last = '';
function setup() {
createCanvas(400, 300);
startDrums('bd*4, cr ~ ~ ~, lt(3,8), hh*8', 100);
textAlign(CENTER, CENTER);
textSize(32);
}
function drumHit(lane, velocity) {
last = lane;
}
function draw() {
background(20);
fill(255);
text(last, width / 2, height / 2);
}
The name of the latest drum that sounded: a line of rhythm brings crash and low tom, which only drumHit() reports.
Related
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
kickDetected()You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
beatDetected()You define this function; oav-p5 calls it each time the microphone (or the drum machine) hears a beat.
stepStarted()An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.
barStarted()An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
OAVp5.drums.on()Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
notePressed()You define this function; oav-p5 calls it each time a key (or a pad) goes down.
How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
Use it for anything that should move continuously with the music: rotate(drumPhase * TWO_PI) turns once per bar. drumStep says the same thing in 16 jumps; drumPhase is the smooth version. The value is what the listener hears, which is the machine's position corrected for the latency of the speakers, so a drawing driven by it lands on the beat you hear.
A bar has four beats; beatPhase is the same idea for one beat.
Type
Number
Range
0..1
Read fresh each time you look at it, so it is never a frame old. 0 at the start of a bar, close to 1 at its end, then 0 again. It is 0 before the machine has played and holds still while it is stopped.
Before it has a value
Start the machine with startDrums(); until it plays, drumPhase is 0.
Notes
A sketch that declares its own drumPhase keeps it, and oav-p5 does not write that name. OAVp5.drums.phase is the same number under a name nothing else can take.
function setup() {
createCanvas(400, 300);
startDrums('four on floor', 120);
strokeWeight(4);
}
function draw() {
background(20);
stroke(255, 211, 107);
translate(width / 2, height / 2);
rotate(drumPhase * TWO_PI - HALF_PI);
line(0, 0, 110, 0); // a hand that turns once per bar
}
A clock hand that turns exactly once every bar, smoothly.
function setup() {
createCanvas(400, 300);
startDrums('breakbeat', 110);
noStroke();
}
function draw() {
background(20);
const a = drumStep / 16 * TWO_PI - HALF_PI; // hops from step to stepconst b = drumPhase * TWO_PI - HALF_PI; // moves all the time
fill(255, 120, 90);
circle(200 + cos(a) * 100, 150 + sin(a) * 100, 20);
fill(120, 200, 255);
circle(200 + cos(b) * 70, 150 + sin(b) * 70, 20);
}
Two dots on two circles: the orange one hops from step to step, the blue one runs smoothly.
Related
beatPhaseHow far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
drumStepWhich of the 16 steps of the bar the drum machine just played, from 0 to 15.
drumBarWhich bar the drum machine is in, counted from 1 when it started.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
drawRhythm()Draws the rhythm on the canvas as a grid, a circle or a spiral, with a playhead and the hits lighting up, so a recording has it too.
Use it to tell "the drums are not running" from "the drums are at step 0": drumStep is 0 in both cases. if (drumsAreOn) { … } is also the right guard before you read drumPhase. It changes only when the machine starts or stops, and reading it never starts anything.
For the reason behind a false read OAVp5.drums.state: 'loading' while the module arrives, 'stopped' when the grid is up but not playing, 'error' when it could not load.
Type
Boolean
True while the machine plays, false at the start, after stopDrums(), after an error and while the grid's own checkbox is unticked.
Before it has a value
It becomes true after startDrums() (or 🥁 in the 🎹 panel) has loaded the machine, usually within a fraction of a second.
Notes
A sketch that declares its own variable called drumsAreOn keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.drums.state === 'on' instead.
function setup() {
createCanvas(400, 300);
startDrums('half time', 100);
textAlign(CENTER, CENTER);
}
function draw() {
background(20);
fill(drumsAreOn ? color(120, 220, 140) : color(220, 160, 60));
circle(width / 2, height / 2, 80);
fill(255);
text(drumsAreOn ? 'drums are on' : 'drums are not running', width / 2, height - 40);
}
An amber circle for a moment while the machine loads, then green.
To be confirmed
After stopDrums() and a second startDrums() it stayed false while the machine played again (Chromium, oav-p5 0.8.0). Not confirmed whether that is intended.
Related
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
stopDrums()Stops the drum machine and hides its card.
drumStepWhich of the 16 steps of the bar the drum machine just played, from 0 to 15.
soundIsOnWhether notes are being heard right now: true once startSound() has finished, false before and after.
micIsOnWhether the microphone is running right now.
recordingIsOnWhether a recording is running right now.
An event you define: it is called with the new line of rhythm every time the rhythm changes.
The line over the grid and the cells edit each other, so the rhythm can change without your code: a click on a cell, a line typed into the card, ↶. rhythmChanged(src, how) tells the sketch, so it can show the line on the canvas (where a recording has it) or store it. It is also called once after startDrums(), a moment after the rhythm module has loaded, with how equal to 'code'.
Parameters
Name
Type
Description
src
String
The line of rhythm that plays now, for example 'bd*4, [~ sd]*2, hh*8'. A preset or a grid you drew comes as the line it is written as.
how
String
What changed it: 'code' (startDrums()), 'cell' (a click on a cell of the grid), 'type' (typed into the line on the card, or OAVp5.drums.edit()), 'undo' (↶ or OAVp5.drums.undo()) or 'menu' (a preset picked on the card).
let current = '';
let how = '';
function setup() {
createCanvas(400, 300);
startDrums('bd*4, [~ sd]*2, hh*8', 110);
textAlign(CENTER, CENTER);
}
function rhythmChanged(src, mode) {
current = src;
how = mode;
}
function mousePressed() {
OAVp5.drums.edit('bd(3,8), [~ sd]*2, hh*16'); // as if typed on the card
}
function draw() {
background(20);
fill(255);
text(current, width / 2, height / 2 - 10);
text('changed by: ' + how, width / 2, height / 2 + 20);
text('click the canvas, or click a cell of the grid', width / 2, height - 30);
}
The line that plays and what changed it: "code" at the start, "type" after a click on the canvas, "cell" when you click a cell of the grid.
Related
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
OAVp5.drums.srcThe line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
OAVp5.drums.edit()Plays a line of rhythm as if it was typed on the card, and returns whether it worked and, if not, what is wrong with it.
OAVp5.drums.undo()Goes back one line of rhythm, like ↶ on the card; returns true if there was an earlier line.
OAVp5.drums.historyThe earlier lines of rhythm, oldest first: every click on a cell, typed line or menu pick leaves the line before it here.
OAVp5.drums.on()Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
drawRhythm()Draws the rhythm on the canvas as a grid, a circle or a spiral, with a playhead and the hits lighting up, so a recording has it too.
Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
Call it once in setup(): startDrums('breakbeat', 120). A grid of 16 steps by 4 lanes (kick, snare, hat, clap) appears at the top of the page, with the tempo and the bar it is in, and plays a synthesized drum kit (no sample files). The grid is part of the page, not of the canvas: a recording never shows it, so use drawRhythm() to draw the rhythm on the canvas. A click on a cell changes the beat, and 🥁 in the 🎹 panel starts and stops the machine with no code at all.
Calling it again while it plays changes the pattern or the tempo without restarting: startDrums(90) only changes the tempo, startDrums('latin') only the pattern. A line of rhythm is any text that is not a preset name. A word is a drum (bd kick, sd snare, hh hat, oh open hat, cp clap, rim, ltmtht toms, cr crash, rd ride, cb cowbell), ~ is a rest, words separated by spaces share the bar evenly, [ ] groups, *4 repeats, , puts another line on top, < > takes one per bar and bd(3,8) spreads 3 hits over 8 steps. A mistake shows a card that points at the character and says how to fix it, and 'four on floor' plays meanwhile.
A line copied from strudel.cc works too: startDrums('s("bd*4, [~ sd]*2").fast(2)'). s("…") and sound("…") with .fast(n) and .slow(n) are read, setcpm(110/4) sets the tempo if you give none, .bank(…) is left out (oav-p5 plays its own kit), and any other call in the chain gets a card that says it is not supported yet. The rest plays.
Before you call it
Nothing before the call. The counting, drumHit() and the other callbacks start at once, but a browser lets the page make sound only after a click or key press on it, so the drums are silent until the first one. A card at the top says so after a moment.
Parameters
Name
Type
Description
pattern
String | Object | Number
What to play. A preset name ('four on floor', 'breakbeat', 'half time', 'latin', 'sparse'). A line of rhythm such as 'bd*4, [~ sd]*2, hh*8', or one copied from strudel.cc. Your own grid { kick: 'x...x...x...x...', snare, hat, clap } (x or 1 is a hit, a lane you leave out is silent, {} is a silent clock). A number alone is the tempo. Leave it out to keep what plays now ('four on floor' at the start). optional
bpm
Number
The tempo in beats per minute (a beat is four steps). Kept between 30 and 300. Leave it out to keep the current tempo: 112 at the start, or the one a copied Strudel line sets with setcpm(…). optional
Returns
Promise<Boolean> Resolves to true once the machine plays (or when a later startDrums() took over). Resolves to false when stopDrums() came first, when there is no page to put the grid on, or when @openav/drums could not be loaded.
Notes
Drum sound does not need startSound(): the machine has its own kit. Pads on MIDI channel 10 play the same kit once startSound() is on.
The card has – to fold the grid away and × to stop the machine. A tab in the background keeps counting and playing. The loading options drums (lookahead or frame) and drumlatency tune the clock.
let swell = 0;
function setup() {
createCanvas(400, 300);
startDrums('breakbeat', 120); // the grid appears at the top of the page
}
function kickDetected(level) {
swell = 1;
}
function draw() {
background(20);
swell *= 0.9;
circle(width / 2, height / 2, 60 + swell * 120);
}
A grid appears at the top of the page. Click the canvas once to hear it: the circle jumps on every kick.
function setup() {
createCanvas(400, 300);
startDrums({}, 90); // no lane has a hit: nothing to hear, the clock still counts
noStroke();
}
function draw() {
background(20);
for (let i = 0; i < 16; i++) {
fill(i === drumStep ? 255 : 90);
circle(30 + i * 22.5, height / 2, i === drumStep ? 18 : 8);
}
}
A silent clock: sixteen dots with the current step enlarged, running at 90 bpm with no sound at all.
Related
stopDrums()Stops the drum machine and hides its card.
drumsAreOnWhether the drum machine is playing right now.
drumHit()An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
drumStepWhich of the 16 steps of the bar the drum machine just played, from 0 to 15.
drumPhaseHow far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
drawRhythm()Draws the rhythm on the canvas as a grid, a circle or a spiral, with a playhead and the hits lighting up, so a recording has it too.
cue()Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
rhythmChanged()An event you define: it is called with the new line of rhythm every time the rhythm changes.
beatTrue for one frame each time the microphone (or the drum machine) hears a beat.
The machine stops playing, the grid disappears and drumsAreOn becomes false. The numbers keep the last value they had: drumStep, drumBar and cueName stay where they were, and drumPhase and beatPhase hold still. startDrums() afterwards begins again from bar 1.
It also works while the machine is still loading: the pending startDrums() then resolves to false. The × on the card does the same thing.
Returns
Promise<Boolean> A promise that is already resolved to false, the new state of the machine.
function setup() {
createCanvas(400, 300);
startDrums('four on floor', 120);
textAlign(CENTER, CENTER);
}
function mousePressed() {
stopDrums();
}
function draw() {
background(drumsAreOn ? 40 : 100);
fill(255);
text(drumsAreOn ? 'click to stop the drums' : 'stopped at step ' + drumStep, width / 2, height / 2);
}
Click the canvas: the grid disappears, the background turns light and the text shows the step it stopped on.
To be confirmed
After stopDrums() and a second startDrums() on the same page the machine plays and counts again, but drumsAreOn and OAVp5.drums.state stayed false and 'off' when this was tried (Chromium, oav-p5 0.8.0). Whether that is intended is not confirmed.
Related
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
drumsAreOnWhether the drum machine is playing right now.
stopSound()Turns sound off and lets every ringing note go; the instrument stays loaded, so startSound() brings it back at once.
start, stop and draw are the same as startDrums(), stopDrums() and drawRhythm(). For a sketch the global names are enough. This object is for tools, libraries (Hydra, for one), the console and debugging.
Four lines read from the drums object while the machine plays.
Related
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
OAVp5.drums.stateWhere the drum machine is: 'off', 'loading', 'on', 'stopped' or 'error'.
OAVp5.drums.on()Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
OAVp5.soundThe object behind the sound names: the state, the instrument, the error, the loading progress and the loaded parts of @openav/sound.
OAVp5.recordThe object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.
The line to paste back into setup(); click cells on the grid and it follows.
Related
OAVp5.drums.srcThe line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
OAVp5.drums.strudelUrlThe rhythm that plays now as a link that opens it at strudel.cc, or an empty string when there is none.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
Plays a line of rhythm as if it was typed on the card, and returns whether it worked and, if not, what is wrong with it.
Unlike startDrums(text), which plays 'four on floor' when the line has a mistake, edit() keeps the line that was playing and tells you why the new one failed, so a live-coding sketch can show the message. The line goes into OAVp5.drums.history and rhythmChanged() is called with how equal to 'type'. A mistake also shows the card that points at the character.
Before you call it
The machine and the rhythm module must have loaded: call startDrums() first and give it a moment.
Parameters
Name
Type
Description
text
String
The line of rhythm, for example 'bd*4, [~ sd]*2, hh*8', or a line copied from strudel.cc.
Returns
Object{ ok: true, error: null, text } when the line plays. { ok: false, error, text } when it has a mistake: error has code, index and length (where the bad word is in text), a message and a hint, and the last line that worked keeps playing. Before the machine has loaded: { ok: false, error: null }.
const TRIES = ['bd*4, hh*8', 'bd(3,8), [~ sd]*2', 'bd*4, xyz*8'];
let n = 0;
let result = '';
function setup() {
createCanvas(400, 300);
startDrums('bd*4, [~ sd]*2, hh*8', 110);
textAlign(CENTER, CENTER);
}
function mousePressed() {
const r = OAVp5.drums.edit(TRIES[n % TRIES.length]);
result = r.ok ? 'ok: ' + r.text : 'wrong: ' + r.error.message;
n++;
}
function draw() {
background(20);
fill(255);
text('click to try the next line', width / 2, 40);
text(result, width / 2, height / 2 - 10);
text('playing: ' + OAVp5.drums.src, width / 2, height / 2 + 30);
}
Click three times: the first two lines play; the third has a mistake, so the message is shown and the line before it keeps playing.
Related
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
OAVp5.drums.undo()Goes back one line of rhythm, like ↶ on the card; returns true if there was an earlier line.
OAVp5.drums.historyThe earlier lines of rhythm, oldest first: every click on a cell, typed line or menu pick leaves the line before it here.
rhythmChanged()An event you define: it is called with the new line of rhythm every time the rhythm changes.
OAVp5.drums.srcThe line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
The earlier lines of rhythm, oldest first: every click on a cell, typed line or menu pick leaves the line before it here.
OAVp5.drums.undo() takes the last line off this list and plays it. Changes made by startDrums() are not recorded here, only the ones made on the card or with OAVp5.drums.edit(). The array is the machine's own, so treat it as read-only.
Type
Array<String>
Empty at the start, and at most 50 lines long. The line that plays now is not in it: that is OAVp5.drums.src.
A snapshot of what the machine hears right now: the bar, the step, both phases and which words of the line are lit.
It is what the card uses to light the word that sounds. Read it when you want the line and its lit words in one go, for example to draw the line on the canvas with the sounding word marked. Before the machine has loaded it answers with zeros and an empty text.
Type
Object
Range
{ bar, step, phase, beatPhase, tokens, mode, text }
A new object on every read. phase and beatPhase are drumPhase and beatPhase. tokens is a list of { start, end }: the places in text of the words that sound now. mode is 'string' for a line of rhythm and 'grid' for a preset or a grid you drew. text is the line.
Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
The functions you define (drumHit, barStarted …) are one listener each. on() is for the cases where that does not fit: several listeners for one event, a library or a Hydra sketch that has no drumHit() of its own, a listener you want to remove again. It is the same idea as midi.on(). An error thrown inside fn is printed and the other listeners still run.
What the drum machine plays, as a name: the preset's name, 'custom' for a grid, or the line of rhythm itself.
After startDrums('breakbeat') it is 'breakbeat', after startDrums({ kick: 'x...x...x...x...' }) it is 'custom', and after startDrums('bd*4, hh*8') it is that line. A click on a cell of a preset makes it 'custom'. To get the line of rhythm in every case, read OAVp5.drums.src: a preset also has one.
Type
String
'four on floor' until you choose another. It follows what plays, including clicks on the grid.
Shows "breakbeat"; click a cell of the grid and it turns into "custom".
Related
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
OAVp5.drums.srcThe line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
rhythmChanged()An event you define: it is called with the new line of rhythm every time the rhythm changes.
The line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
Show it on the canvas so a recording has it (text(OAVp5.drums.src, 20, 30)), or keep it to come back to the rhythm later with startDrums(src). rhythmChanged() tells you when it changes, and OAVp5.drums.copyText is the same line wrapped in a startDrums("…") call.
Type
String
Read fresh each time. A preset or a grid you drew shows as the line it is written as, and it changes with every click on a cell and every typed edit.
Before it has a value
It is '' until startDrums() has loaded the machine and the rhythm module, a fraction of a second.
The preset "half time" written as the line of rhythm it is; click cells on the grid and the line is rewritten.
Related
OAVp5.drums.patternWhat the drum machine plays, as a name: the preset's name, 'custom' for a grid, or the line of rhythm itself.
OAVp5.drums.copyTextThe text the Copy button on the card puts on the clipboard: the rhythm that plays now as a ready startDrums("…", bpm) call.
OAVp5.drums.edit()Plays a line of rhythm as if it was typed on the card, and returns whether it worked and, if not, what is wrong with it.
rhythmChanged()An event you define: it is called with the new line of rhythm every time the rhythm changes.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
Where the drum machine is: 'off', 'loading', 'on', 'stopped' or 'error'.
'loading' is the moment after startDrums() while the module arrives, 'on' is playing, 'stopped' is a grid that is up but not playing (its checkbox is unticked), 'off' is where it starts and where stopDrums() leaves it, and 'error' means @openav/drums could not be loaded. drumsAreOn is state === 'on'.
Type
String
Range
off | loading | on | stopped | error
Starts as 'off' and changes when the machine loads, plays, is stopped or fails.
Shows "loading" for a moment, then "on". Untick the checkbox on the grid and it reads "stopped".
To be confirmed
After stopDrums() and a second startDrums() the state stayed 'off' while the machine played again (Chromium, oav-p5 0.8.0). Not confirmed whether that is intended.
Related
drumsAreOnWhether the drum machine is playing right now.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
stopDrums()Stops the drum machine and hides its card.
OAVp5.drumsThe object behind the drum names: the state of the machine, its counters, the line of rhythm, its history and the loaded machine.
The rhythm that plays now as a link that opens it at strudel.cc, or an empty string when there is none.
It holds the line as s("…") with setcpm(bpm/4) (one bar here is one Strudel cycle of four beats), the same thing the "Open in Strudel" link on the card opens. Open it in a new tab: window.open(OAVp5.drums.strudelUrl) from a click handler.
Type
String
A https://strudel.cc/#… address, read fresh each time. '' before the machine has loaded.
function setup() {
createCanvas(400, 300);
startDrums('bd*4, [~ sd]*2, hh*8', 110);
textAlign(CENTER, CENTER);
}
function mousePressed() {
if (OAVp5.drums.strudelUrl) window.open(OAVp5.drums.strudelUrl, '_blank');
}
function draw() {
background(20);
fill(255);
text('click to open this rhythm in strudel.cc', width / 2, height / 2 - 15);
text(OAVp5.drums.strudelUrl.slice(0, 40) + '…', width / 2, height / 2 + 15);
}
Shows the start of the link; a click opens it at strudel.cc in a new tab.
To be confirmed
Not tried: whether the sketch frame of OpenProcessing lets window.open open the link.
Related
OAVp5.drums.srcThe line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
OAVp5.drums.copyTextThe text the Copy button on the card puts on the clipboard: the rhythm that plays now as a ready startDrums("…", bpm) call.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
Goes back one line of rhythm, like ↶ on the card; returns true if there was an earlier line.
The line before the current one is played again and removed from the history. rhythmChanged() is called with how equal to 'undo'. On the card the same thing is ⌘Z or Ctrl+Z after touching it.
Returns
Booleantrue if it went back one line, false if OAVp5.drums.history was empty or the machine is not loaded.
A drum kit made of Web Audio synthesis, with no sample files: General MIDI drum notes go in, drum sounds come out.
It follows the small engine contract of @openav/sound, so the same shell and the same signals drive it: enable() makes the AudioContext, noteOn(note, vel01, time) plays a drum, set(key, value) sets a param. The kit has thirteen voices (see GM): kick, rim, snare, clap, low tom, hat, pedal hat, tom, open hat, high tom, crash, ride and cowbell. A note the kit does not know (any melodic note) plays nothing and returns null.
noteOn(note, vel01, time) returns { time, cancel() }, or null before enable() or for a note it does not play. time is on the kit's own clock (engine.context.currentTime); left out, or already past, the hit sounds now. cancel() silences a hit that has not started, which is how DrumTransport takes back what it scheduled when the tempo changes. hit('crash', 0.9) plays a voice by name (or a note number) for code that is not a MIDI pad. There is one param, kitVolume, from −30 to 0 dB (default −6).
Once enabled, engine.context is its AudioContext and engine.output is the node that goes to the speakers, which a recorder can tap. Before that both are null. noteOff() does nothing (drums ring out) and dispose() closes the context.
Before you call it
await kit.enable() from a click: browsers start audio only after a user gesture. mountDrums() and DrumTransport with autoEnable do that for you on the first hit.
Parameters
Name
Type
Description
samples
Object
Play audio files instead of the synthesis: { 36: 'kick.wav', 38: 'snare.wav' }, MIDI note number to URL. They are fetched and decoded by enable(); a file that fails to load is skipped without an error, and a note of the kit then plays its synthesized sound instead. optionaldefault null
module Before enable() the kit has a param but no sound and no AudioContext: noteOn() returns null.
js
import { drumEngine } from'@openav/drums';
const kit = drumEngine();
document.body.addEventListener('click', async () => {
await kit.enable(); // from a click: audio may start
kit.hit('kick');
kit.hit('snare', 0.9, kit.context.currentTime + 0.5); // half a second from now
kit.hit('crash', 0.8, kit.context.currentTime + 1);
});
static Click the page: a kick, then a snare half a second later and a crash after a second. The times are on the kit's own audio clock.
Related
DrumTransportThe runtime around a DrumScheduler: it hands every hit to the kit ahead of time on its audio clock, and tells the page each step and hit when it is heard.
mountDrums()Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
GMThe General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.
voiceForNote()Tells which drum of the kit a General MIDI drum note plays, or null for a note it does not play.
SoundMakes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
toneEngine()The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
The name of a drum for the screen, in English or in Chinese: drumName('lowtom') is 'low tom'.
The drum names inside code stay English everywhere ('kick', 'snare'), and this is for the screen: the row labels of the drum machine card use it, and with 'zh' they are written in Chinese.
Parameters
Name
Type
Description
voice
String
The drum: kick, snare, hat, clap, openhat, pedalhat, rim, lowtom, tom, hightom, crash, ride or cowbell.
lang
'en' | 'zh'
'zh' for Chinese; anything else is English. optionaldefault 'en'
Returns
String The name, or voice itself if it is not a drum of the kit.
Examples
js
import { drumName } from'@openav/drums';
console.log(drumName('kick'), drumName('kick', 'zh')); // kick, then the Chinese name of the kick
console.log(drumName('openhat', 'zh'), drumName('lowtom', 'zh')); // Chinese names of the open hat and the low tom
console.log(drumName('banjo', 'zh')); // banjo, not a drum: returned as is
module Logs a few drums in English and in Chinese, and what a name that is not a drum gives back.
Related
GMThe General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.
mountDrums()Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
WORDSThe drum words a line of rhythm may use and the drum each one plays: bd is kick, sd is snare, hh is hat.
new DrumScheduler({ bpm = 112, swing = 0, humanize = 0.35, lookahead = 0.1, pattern = 'four on floor', seed = 0, random = Math.random } = {})
Pure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
A DrumScheduler knows what plays in which bar, and when a bar happens, but it owns no clock: every time you give it is in seconds on one clock of your choosing, so it runs the same under a fake clock in a test as under the audio clock in a page. DrumTransport is the runtime that feeds it the real one. Use the scheduler on its own when you want drum events as data, for your own sound engine or a test.
The loop is: start(at) puts the downbeat of bar 1 at time at. Then, again and again, advance(now) schedules everything up to now + lookahead and returns the new events, in order. due(t) hands out the events whose time has come (they stay in queue until then). Two kinds of event come out: a step { kind: 'step', i, n, bar, pos, time } for every 16th, and a hit { kind: 'hit', voice, vel, pos, n, bar, token, word, layer, dur, time }. pos counts bars from the start (0 is the downbeat of bar 1, 1.5 is half way through bar 2), time is in your seconds, i is the step 0..15 and n counts steps since the start. For a grid, the step comes before its hits. token, word, layer and dur come from a line of rhythm and are null or undefined for a grid.
Positions turn into seconds by multiplication from the last tempo anchor, never by adding up frame times, so a thousand bars later nothing has drifted (the test measures under a microsecond). A tempo change adds an anchor, so the past keeps the tempo it had.
Parameters
Name
Type
Description
bpm
Number
The tempo, in beats per minute. A value that is not above 0 becomes 112. optionaldefault 112
swing
Number
0 to 0.5: every odd 16th is that share of a 16th late. 0 is straight. optionaldefault 0
humanize
Number
0..1: a few milliseconds late and a little softer or louder, so it grooves instead of ticking. Use 0 for exact, repeatable times. optionaldefault 0.35
A preset name, your own grid { kick: 'x...x...', … } (a lane left out is silent), or a compiled line of rhythm: anything with eventsInBar(bar, { seed }), such as compile(text). With a line, { text } is the text it came from. An unknown name plays 'four on floor'. Returns the new pattern.
Bar bar of what plays, on the 16 steps of the four lanes: { grid, offGrid, dropped }. offGrid counts hits that sat between steps and were moved to the nearest one, dropped lists the drums the grid has no lane for.
The hits of bar bar (1 is the first) as [{ voice, pos, … }], pos 0..1 in the bar, from whichever source plays. A line that fails or names a drum the kit does not have leaves warning set and skips that hit.
A new tempo from time at on. Left out, from the end of what is already scheduled, so nothing is taken back. Returns the events taken back ([] if none). Does nothing for a value that is not above 0.
Takes back everything scheduled at or after time at; the next advance() makes it again from the pattern and tempo as they are then. Returns the events taken back. Events already handed out by due() are never taken back.
Schedules everything up to now + lookahead and returns the new events in order (they stay in queue until due() hands them out). If the page was frozen so long that more than 64 bars are due, it skips ahead instead of returning minutes of hits at once, and counts them in skipped. Does nothing, and returns [], when stopped.
The events whose time is t or earlier, in order, removed from the queue.
Examples
js
import { DrumScheduler } from'@openav/drums';
const s = new DrumScheduler({ bpm: 120, humanize: 0 });
s.start(0); // the downbeat of bar 1 is at time 0
s.advance(1.9); // schedule up to 1.9 + 0.1 seconds: one whole barconst heard = s.due(0.5); // what is heard by 0.5 s
console.log(heard.filter((e) => e.kind === 'hit').map((e) => e.voice + '@' + e.time).join(' '));
// kick@0 hat@0 hat@0.25 kick@0.5 snare@0.5 hat@0.5
console.log(s.phaseAt(1).bar, s.posAt(2), s.timeAt(1)); // 0.5 1 2
module No clock, no audio: the scheduler is asked for one bar of four on the floor at 120 bpm and logs the hits heard in the first half second, with their times in seconds.
module A line of rhythm goes in: its first bar is read as hits, then as a 16-step grid, and a tempo change halves the speed at time 12 without moving what came before.
Related
DrumTransportThe runtime around a DrumScheduler: it hands every hit to the kit ahead of time on its audio clock, and tells the page each step and hit when it is heard.
DrumSequencerA 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
compile()Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
toGrid()Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.
PATTERNSThe five preset grids of the drum machine: 'four on floor', breakbeat, 'half time', latin and sparse.
GMThe General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.
A 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
It is the simplest clock: no audio, no timers, no DOM. Make it with an onHit that does something, call toggle(true), then call update(dt) once per frame with the seconds since the last frame. It walks through the 16 steps of a bar (a 16th lasts 60 / bpm / 4 seconds) and calls onHit(lane, velocity) for each hit in the four lanes kick, snare, hat and clap. A hit is told in the frame in which it is due, so it can be late by up to a frame, and nothing happens in a hidden tab. For a steady beat under a heavy frame, use DrumTransport.
The grid is seq.grid: { kick: [0|1 × 16], snare, hat, clap }. seq.pattern is the preset name, or 'custom' once you changed a cell or gave your own grid. seq.playing, seq.bpm, seq.swing and seq.humanize are plain properties you can read and set.
Parameters
Name
Type
Description
onHit
Function
onHit(lane, velocity) for every hit: lane is 'kick', 'snare', 'hat' or 'clap', velocity is 0.15 to 1. You decide what a hit does: play a sound, draw, publish a signal.
onStep
Function
onStep(i, n) for every 16th step: i is 0..15, n counts steps since the start. It comes before that step's hits. A timeline counts bars on it. optionaldefault null
bpm
Number
The tempo, in beats per minute. It can be changed later: seq.bpm = 90. optionaldefault 112
swing
Number
0 to 0.5: every odd 16th is that share of a 16th late. 0 is straight. optionaldefault 0.12
humanize
Number
0..1: some hits a few milliseconds late and a little softer or louder. Use 0 for exact, repeatable velocities (0.9 on the beat, 0.6 between). optionaldefault 0.35
pattern
String | Object
A preset name from PATTERNS or your own grid. optionaldefault 'four on floor'
A preset name ('breakbeat'), or your own grid { kick: 'x...x...x...x...', snare: …, hat: …, clap: … }; a lane you leave out is silent, and pattern becomes 'custom'. An unknown name plays 'four on floor'.
toggle(on) starts or stops; with no argument it flips. Starting begins from step 0, and a hit left over from before a stop never lands after the next start. Returns the new state.
Moves the clock forward by dt seconds and calls onStep and onHit for what falls in that time. Does nothing while stopped. Call it once per frame.
Examples
js
import { DrumSequencer } from'@openav/drums';
const hits = [];
const seq = new DrumSequencer({
onHit: (lane, vel) => hits.push(lane + ' ' + vel.toFixed(1)),
bpm: 120, swing: 0, humanize: 0,
});
seq.toggle(true);
for (let i = 0; i < 32; i++) seq.update(1 / 60); // about half a second, in 60 fps frames
console.log(hits.join(', '));
// kick 0.9, hat 0.9, hat 0.6, kick 0.9, snare 0.9, hat 0.9
console.log(seq.pattern, seq.grid.kick.join('')); // four on floor 1000100010001000
module Runs half a second of four on the floor with no sound and no screen, and logs the hits: a kick and a hat on the downbeat, the off-beat hat, then kick, snare and hat on beat 2.
js
import { DrumSequencer } from'@openav/drums';
const g = document.getElementById('c').getContext('2d');
let lit = 0;
const seq = new DrumSequencer({
bpm: 110, pattern: { kick: 'x...x...x...x...', snare: '....x.......x...', hat: 'x.x.x.x.x.x.x.x.' },
onStep: (i) => { lit = i; }, // onHit would be the place to play a sound
});
seq.toggle(true);
let last = performance.now();
(function frame(now) {
seq.update((now - last) / 1000); last = now;
g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
for (let i = 0; i < 16; i++) { g.fillStyle = i === lit ? '#ffd166' : '#2a3348'; g.fillRect(12 + i * 23, 120, 18, 60); }
requestAnimationFrame(frame);
})(last);
module Sixteen blocks on the canvas, and a bright one runs along them in time with the beat. There is no sound: onHit is where you would play one.
Related
DrumTransportThe runtime around a DrumScheduler: it hands every hit to the kit ahead of time on its audio clock, and tells the page each step and hit when it is heard.
DrumSchedulerPure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
PATTERNSThe five preset grids of the drum machine: 'four on floor', breakbeat, 'half time', latin and sparse.
laneSteps()Turns a lane written as text, like 'x...x...x...x...', or as a list into 16 steps of 0 and 1.
mountDrums()Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
GMThe General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.
The runtime around a DrumScheduler: it hands every hit to the kit ahead of time on its audio clock, and tells the page each step and hit when it is heard.
A page that plays a hit in the frame it is due is late by up to a frame, and silent when its tab is hidden. DrumTransport fixes both. A timer (the heartbeat, from a Web Worker, 25 ms by default) looks 0.1 seconds ahead on the kit's own audio clock and hands the kit every hit with its exact start time. The page is told separately, in the frame in which each step and hit is heard: its time plus the output latency the browser reports. So the beat stays steady under a heavy draw() and in a hidden tab, and the callbacks still arrive in order: a step before its hits, hits in time order.
Call start(), then frame() once per animation frame. The heartbeat calls pump() for you. Tempo and pattern changes (setBpm(), setPattern(), toggleCell()) take back what was already handed over after now and make it again, so no hit is doubled or lost. Before the kit's AudioContext runs (no click yet) the clock runs on now() and the callbacks run but nothing sounds; once it runs, the hits from then on sound, and none from before.
A hidden page, or a frame loop that catches up more than stale seconds late, tells its steps quietly: ev.quiet is true, the clock moves and cues fire, but onHit is not called, and of several bar starts at once only the last has ev.announce. Nothing floods the page when the tab comes back. If a draw() ever holds the thread longer than the lookahead, the lookahead grows to 0.3 seconds (info().extended).
It has the getters playing, pattern, grid, mode, text, bpm, swing, humanize and lookahead of its scheduler; bpm, swing and humanize can also be set. You usually get one through mountDrums() with clock: 'lookahead'; build your own when you want the clock without the card.
Parameters
Name
Type
Description
scheduler
DrumScheduler
The DrumScheduler that decides what is played when. optionaldefault new DrumScheduler()
engine
Object
The sound engine, usually a drumEngine(). If it has a context property (an AudioContext, or null before it is enabled), hits are scheduled ahead on that clock. If not, each hit is played by calling engine.noteOn(note, vel) at the moment it is heard. optionaldefault null
onStep
Function
onStep(i, n, ev) for every 16th step when it is heard. ev.quiet and ev.announce say whether it was told quietly (below). optionaldefault null
onHit
Function
onHit(voice, vel, ev) for every drum hit when it is heard. optionaldefault null
onLate
Function
onLate(margin, transport): a hit was handed to the kit after its time, because the page held the main thread longer than the lookahead. margin is how many seconds early it was (negative or tiny). optionaldefault null
autoEnable
Boolean
Call engine.enable() by start() when the engine has no AudioContext yet. optionaldefault false
now
Function
A function that returns the time in seconds. Pass your own to drive the transport from a fake clock in a test. optionaldefault performance.now() / 1000
hidden
Function
A function that returns true while nobody can see the page. optionaldefault checks document.visibilityState
heartbeat
'auto' | 'interval' | false
Who calls pump(): 'auto' a Web Worker timer (a hidden tab does not slow a worker), falling back to setInterval; 'interval' always setInterval; false (or 'none') nobody, and you call pump() or frame() yourself. optionaldefault 'auto'
tick
Number
Seconds between two heartbeats. optionaldefault 0.025
latency
'auto' | Number
Seconds between a hit being scheduled and being heard. 'auto' asks the browser (getOutputTimestamp(), outputLatency); a number sets it by hand. optionaldefault 'auto'
stale
Number
A step that is told more than this many seconds late (a frame loop catching up) is told quietly. optionaldefault 0.25
startDelay
Number
When the audio clock is running, the first downbeat is put this many seconds after start(), so the first hit is not already late. optionaldefault 0.05
The master time line, in seconds: { sched, heard, live, ct }. sched is the earliest time a hit can still be handed to the kit, heard is what the listener hears right now, live is whether the audio clock runs. Until it does, it is now().
Starts from bar 1 (again, if it was playing) and returns true. When the audio clock runs, the first downbeat is startDelay seconds away. Starts the heartbeat.
A preset name, your own grid, or a compiled line of rhythm (with { text }, the line it came from). The change is heard at once. Returns the new pattern.
toggleCell(lane, i) flips one cell of the grid (lane is kick, snare, hat or clap, i is 0..15). A line of rhythm is first turned into the grid it plays.
A new tempo from now on. What was handed to the kit after now is taken back and made again at the new tempo. The same tempo again does nothing. Returns the tempo.
Takes back everything handed to the kit after now (with 10 ms of margin, so a hit already sounding is not cut) and makes it again from the pattern and tempo as they are. The setters call it for you.
The heartbeat: hand the next hits to the kit. When the page is hidden, or no frame has run for 0.1 seconds, it also tells the page what is heard. Call it yourself only with heartbeat: false.
Call it once per animation frame. Hands over what is due, then tells the page every step and hit heard by now through onStep and onHit. Returns those events.
How the clock is doing: { clock, heartbeat, why, lookahead, late, extended, audio, latency, skipped, warning }. heartbeat is 'worker', 'interval' or 'none', late counts hits handed over late.
Plays on an external clock (Strudel, a DAW, a linked page): follow({ pos, at, bpm }) says that position pos (bars from its start, 0 is the downbeat of bar 1) is heard at at (the now clock, in seconds) at tempo bpm. Stopped, it starts so bar 1 lands on the external 0. Playing, an error over jump bars (0.125) restarts the same way; a smaller one is closed over catchUp seconds by a tempo a little faster or slower (at most pull of it, 0.08), then the external tempo again. An error under still (0.0005 seconds) is left alone. The second argument is { catchUp, jump, pull, still }. Returns the error in seconds (positive: the machine was behind). Call it with every new reading of the external clock.
Stops following. The machine keeps the tempo it has, on its own clock.
Examples
js
import { DrumScheduler, DrumTransport } from'@openav/drums';
let T = 100; // a clock we move by hand, in secondsconst log = [];
const transport = new DrumTransport({
scheduler: new DrumScheduler({ bpm: 120, humanize: 0 }),
heartbeat: false, now: () => T, // no timer, our own clock
onStep: (i) => { if (i % 4 === 0) log.push('step ' + i); },
onHit: (voice, vel) => { if (voice !== 'hat') log.push(voice); },
});
transport.start();
for (let k = 0; k < 40; k++) { T += 1 / 20; transport.frame(); } // two seconds in 20 fps frames
console.log(log.join(', '));
// step 0, kick, step 4, kick, snare, step 8, kick, step 12, kick, snare
console.log(transport.playing, transport.phase().bar.toFixed(2)); // true 1.00
module Runs a whole bar of four on the floor on a clock we move by hand, and logs what the page is told: the downbeats of each beat, the kicks and the snares on 2 and 4.
js
import { DrumScheduler, DrumTransport, drumEngine } from'@openav/drums';
const kit = drumEngine();
const transport = new DrumTransport({
scheduler: new DrumScheduler({ bpm: 100, swing: 0.1, pattern: 'breakbeat' }),
engine: kit, autoEnable: true, // the click that starts it also starts the audio
onHit: (voice, vel) => console.log(voice, vel.toFixed(2)),
});
document.body.addEventListener('click', () => transport.start(), { once: true });
(function frame() { transport.frame(); requestAnimationFrame(frame); })();
static Click the page and a breakbeat starts on the kit's audio clock; the console logs each hit when it is heard. Switch to another tab: the beat keeps going.
Related
DrumSchedulerPure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
mountDrums()Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
drumEngine()A drum kit made of Web Audio synthesis, with no sample files: General MIDI drum notes go in, drum sounds come out.
DrumSequencerA 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
drumPhaseHow far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
The General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.
Drum pads and drum machines send these notes on MIDI channel 10. The drum machine sends them as midi/note/on{ note, vel, ch: 10 } for every hit, so anything that listens to MIDI hears the drums. The reverse, from a note to a drum, is voiceForNote(), which also knows the neighbouring General MIDI notes (35 is also a kick, 40 is also a snare).
Turns a lane written as text, like 'x...x...x...x...', or as a list into 16 steps of 0 and 1.
It is how every grid you write is read: { kick: 'x...x...x...x...' } in DrumSequencer, DrumScheduler and startDrums(). 'x... x... x... x...' and '1000100010001000' are the same lane. Note that the dot is a rest here, and only here: in a line of rhythm (see compile()) a rest is ~.
Parameters
Name
Type
Description
v
String | Array
The lane. In text, x, X and 1 are hits and everything else is a rest; spaces and | are ignored, so they can help the eye. In a list, 1 and true are hits.
Returns
Number[] An array of 16 numbers, 1 for a hit and 0 for a rest. A short lane is padded with rests, a long one is cut at 16.
module Logs a lane written with spaces, a lane written as a list, and that any length comes out as 16 steps.
Related
PATTERNSThe five preset grids of the drum machine: 'four on floor', breakbeat, 'half time', latin and sparse.
DrumSequencerA 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
DrumSchedulerPure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
fromGrid()Turns a grid of steps, like the drum machine card, into the one line of rhythm a person would write for it.
Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
mountDrums(el, { signals }) builds a drum machine card inside el: a play checkbox, a preset menu, a tempo slider, the bar it is in, and a grid with a lane per drum (kick, snare, hat, clap, and a row of their own for the crash, ride, toms and open hat that a line of rhythm uses). A click on a cell toggles it. The returned object lets you do from code what the card does with the mouse: play(true), setPattern('breakbeat'), setBpm(96).
You must call drums.update(dt) every frame, with the seconds since the last one. It moves the machine (with the frame clock), tells your onStep and onHit what is heard, paints the grid and lets the drum/*/env signals decay. A machine that is never updated does not move.
Hits travel the same road as a drummer heard through the microphone, so a sketch written for the microphone moves over unchanged: drum/kick, drum/snare, drum/hat, drum/clap as pulses with { level }, drum/<voice>/env as continuous 0..1 values, and midi/note/on{ note, vel, ch: 10 } with the General MIDI drum note (see GM). A voice a line of rhythm adds, like drum/crash, is defined the first time it is hit.
Give it the pattern module (rhythm) and the line of rhythm is always shown over the grid: a preset shows the line its grid is ('four on floor' is bd*4, [~ sd]*2, hh*8), and the word that sounds lights up. Then the two edit each other. A click on a cell rewrites the line (with fromGrid()). Typing in the line changes the cells: Enter, or 400 ms without a key, plays what was typed. A mistake is marked red, onEditError is told, and the last line that worked keeps playing. Escape drops what was not played yet. Every change is kept in history; ↶ on the card, or Cmd/Ctrl+Z after touching the card, goes back one. Copy puts the line on the clipboard and Open in Strudel opens it at strudel.cc in a new tab (strudelUrl()).
What the returned object offers:
text
play(on) start (true) or stop (false) like the checkbox; no argument flips; returns whether it plays
setPattern(p, {text}) a preset name, your own grid, or a compiled line (text: the line it came from); the card follows
setBpm(bpm) the tempo, like the slider; returns the tempo
update(dt) call every frame
deliver() tell onStep/onHit what is heard by now without painting (call it right before you draw)
phase() { bar, beat, pos }: how far through the bar and the beat the listener is, 0..1
now() { bar, step, phase, beatPhase, tokens, mode, text } for the display
info() { clock, heartbeat, lookahead, late, extended, audio, latency, ... } the health of the clock
useRhythm(mod) hand over @openav/pattern later (it can load on its own)
src() the line that plays now ('' with a grid and no rhythm module)
strudelUrl() the line as a strudel.cc link, or ''
edit(text) play a line as if typed: { ok, error }; a mistake keeps the last line playing
history, undo() the lines before, oldest first; undo goes back one and returns whether it did
copy() put copyText() on the clipboard: Promise<{ ok, text }>
barEvents(bar) the hits of a bar: [{ voice, pos, ... }]
voices() the rows on the grid now
seq, engine, root the DrumSequencer or DrumTransport, the sound engine, the card's element
dispose() stop, release the kit and take the card out
Before you call it
A page with a DOM, and update(dt) called every frame. Sound waits for the browser: the kit's AudioContext is made by the first hit (or by play() with the lookahead clock), and until the page has had a click it stays suspended. Callbacks and signals run from the start; the sound comes in on the first click, and nothing from before it is played late.
Parameters
Name
Type
Description
container
HTMLElement
The element to put the card in. dispose() takes it out again.
signals
Signals
A Signals bus. Every hit is published as drum/<voice> (a pulse { level }), drum/<voice>/env (0..1, decays) and midi/note/on{ note, vel, ch: 10 }. Without one the machine plays and calls your functions, and publishes nothing. optional
engine
Object
The sound engine. Left out, the machine makes its own drumEngine(). Anything with enable() and noteOn(note, vel01, time) works. optionaldefault null
autoEnableEngine
Boolean
Switch the engine on by the first hit (or by play() with the lookahead clock), so no extra click is needed beyond the one the browser wants. optionaldefault true
onStep
Function
onStep(i, n, ev), once for every 16th step when it is heard. i is 0..15 in the bar, n counts steps since the start, before that step's hits. It is the clock a timeline counts bars on. optionaldefault null
onHit
Function
onHit(voice, vel, ev), once for every drum hit when it is heard. voice is a name like 'kick', vel is 0..1. optionaldefault null
onTakeover
Function
onTakeover({ text, bar, offGrid, dropped, varied }): without a rhythm module, a click on a cell turned a line of rhythm into the grid it was playing. offGrid hits moved to the nearest step and dropped drums had no row. optionaldefault null
bpm
Number
The starting tempo, in beats per minute. The slider on the card goes from 60 to 180. optionaldefault 112
pattern
String | Object
A preset name from PATTERNS or your own grid { kick: 'x...x...', … }. Not a line of rhythm: to start with one, call drums.edit(line) or drums.setPattern(compile(line), { text: line }). optionaldefault 'four on floor'
clock
'frame' | 'lookahead'
'frame': the machine moves in update(dt) and each hit sounds in the frame it is due. 'lookahead': hits are handed to the kit ahead of time on its audio clock, so the beat is steady under a heavy frame and in a hidden tab ('audio' means the same). See DrumTransport. optionaldefault 'frame'
lang
'en' | 'zh'
'zh' names the lanes and the buttons in Chinese. Anything else is English. optionaldefault 'en'
heartbeat
'auto' | 'interval' | false
Only for clock: 'lookahead'. Who wakes the scheduler: 'auto' a Web Worker timer (falls back to setInterval where a worker cannot start), 'interval' always setInterval, false nobody (you call drums.update() often enough). optionaldefault 'auto'
latency
'auto' | Number
Only for clock: 'lookahead'. How long after a hit is scheduled it is heard, in seconds. 'auto' asks the browser for its output latency; a number sets it by hand (a Bluetooth speaker is 0.15 to 0.25). optionaldefault 'auto'
now
Function
Only for clock: 'lookahead'. A function that returns the time in seconds. The default is performance.now() / 1000. Replace it to run the machine on a fake clock in a test. optional
hidden
Function
Only for clock: 'lookahead'. A function that returns true while nobody can see the page. The default looks at document.visibilityState. optional
rhythm
Module
The @openav/pattern module (import * as rhythm from '@openav/pattern', anything with compile and fromGrid). With it the line of rhythm over the grid is always there and editable, and the cells and the line rewrite each other. It can also be handed over later with drums.useRhythm(mod). optionaldefault null
onRhythm
Function
onRhythm(line, { how, prev }), once for every new line. how is 'code', 'cell', 'type', 'undo' or 'menu'. optionaldefault null
onLossy
Function
onLossy({ text, bar, lost }): a click on a cell would lose something the 16 steps cannot hold (triplets, < >, ?, bd:2). lost is the same as in toGrid(). Left out, a note appears on the card. The next click rewrites the line anyway. optionaldefault null
onEditError
Function
onEditError(error, text) when what was typed is not a line of rhythm (the mistake is marked in red and the last line that worked keeps playing), and onEditError(null) once it is fixed. optionaldefault null
copyText
Function
copyText(line, bpm) returns what the Copy button puts on the clipboard. The default is the line itself. optionaldefault null
strudelComment
String
A comment to put at the head of the code that Open in Strudel sends. optionaldefault ''
With clock: 'frame' the first bar of a compiled line is played on the 16-step grid of four lanes: hits between steps move to the nearest step, and drums with no lane (crash, ride, toms, open hat) are left out. The console says so. clock: 'lookahead' plays a line exactly as written.
In a p5 sketch you do not call it: startDrums() builds this card for you, with the lookahead clock, and calls update() every frame.
Examples
js
import { mountDrums } from'@openav/drums';
const drums = mountDrums(document.getElementById('app'), { bpm: 120, pattern: 'breakbeat' });
drums.play(true); // start, like ticking the checkboxlet last = performance.now();
(function frame(now) {
drums.update((now - last) / 1000); // the machine only moves when you call this
last = now;
requestAnimationFrame(frame);
})(last);
module A drum machine card appears and plays a breakbeat; the yellow column runs along the grid. Click the page once to hear the kit; click the cells to change the beat.
js
import * as rhythm from'@openav/pattern';
import { mountDrums } from'@openav/drums';
const told = [];
const drums = mountDrums(document.getElementById('app'), {
rhythm, bpm: 110, clock: 'lookahead',
onRhythm: (line, { how }) => told.push(how + ': ' + line),
onEditError: (error) => told.push(error ? 'error ' + error.code : 'fixed'),
});
console.log(drums.src()); // bd*4, [~ sd]*2, hh*8
console.log(drums.edit('bd(3,8), cr ~ ~ ~').ok); // true: the crash gets a row of its own
console.log(drums.edit('bd [sd').ok); // false: the last line that worked keeps playing
console.log(drums.src()); // bd(3,8), cr ~ ~ ~
console.log(drums.undo(), drums.src()); // true bd*4, [~ sd]*2, hh*8
console.log(told.join(' | '));
// code: bd*4, [~ sd]*2, hh*8 | type: bd(3,8), cr ~ ~ ~ | fixed | error unclosed | undo: bd*4, [~ sd]*2, hh*8 | fixedlet last = performance.now();
(function frame(now) { drums.update((now - last) / 1000); last = now; requestAnimationFrame(frame); })(last);
module With the pattern module the line is shown over the grid. The code types two lines for you, one of them wrong, then goes back; type into the line yourself and the cells follow.
js
import { mountDrums } from'@openav/drums';
import { Signals } from'@openav/core';
const signals = new Signals();
const g = document.getElementById('c').getContext('2d');
const drums = mountDrums(document.getElementById('app'), { signals, clock: 'lookahead', bpm: 100 });
drums.play(true);
let last = performance.now();
(function frame(now) {
drums.update((now - last) / 1000); last = now;
g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
g.fillStyle = '#ff7b54';
g.beginPath();
g.arc(200, 150, 30 + signals.get('drum/kick/env') * 90, 0, Math.PI * 2); // 0..1, decays after each hit
g.fill();
requestAnimationFrame(frame);
})(last);
module A circle on the canvas swells with every kick: the drum machine publishes drum/kick/env, and the sketch only reads the signal.
To be confirmed
The figure of 0.15 to 0.25 seconds for a Bluetooth speaker (the latency option) is from the handbook (docs/p5.md); it was not measured here.
Related
DrumTransportThe runtime around a DrumScheduler: it hands every hit to the kit ahead of time on its audio clock, and tells the page each step and hit when it is heard.
DrumSequencerA 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
drumEngine()A drum kit made of Web Audio synthesis, with no sample files: General MIDI drum notes go in, drum sounds come out.
compile()Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
drumHit()An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
The five preset grids of the drum machine: 'four on floor', breakbeat, 'half time', latin and sparse.
The preset names are what startDrums(), DrumSequencer and mountDrums() accept as a pattern. A preset is just data, so you can start from one: { ...PATTERNS.latin, clap: 'x...x...x...x...' } is latin with a clap on every beat.
The five presets as text, one lane per column:
text
kick snare hat clap
four on floor 1000100010001000 0000100000001000 1010101010101010 0000000000000000
breakbeat 1000000110100000 0000100000001001 1010101010101010 0000100000000000
half time 1000000000001000 0000000010000000 1010101010101010 0000000010000000
latin 1000001000100000 0010010000100100 1011101110111011 0000000000000000
sparse 1000000000100000 0000000000000000 0010001000100010 0000000010000000
Type
Object
Five presets, each { kick, snare, hat, clap } as 16-character strings of 1 and 0.
Examples
js
import { PATTERNS, DrumSequencer } from'@openav/drums';
console.log(Object.keys(PATTERNS).join(' | ')); // four on floor | breakbeat | half time | latin | sparse
console.log(PATTERNS['half time'].snare); // 0000000010000000const seq = new DrumSequencer({ onHit() {} });
seq.setPattern({ ...PATTERNS.latin, clap: 'x...x...x...x...' }); // latin plus a clap on every beat
console.log(seq.pattern, seq.grid.clap.join('')); // custom 1000100010001000
module Lists the presets, reads one lane, and builds a variation of latin by spreading it into a new grid.
Related
laneSteps()Turns a lane written as text, like 'x...x...x...x...', or as a list into 16 steps of 0 and 1.
DrumSequencerA 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
DrumSchedulerPure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
mountDrums()Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
Tells which drum of the kit a General MIDI drum note plays, or null for a note it does not play.
Pads and drum machines send notes on channel 10, and several notes map to the same drum: 35 and 36 are kicks, 38 and 40 snares, 41 and 43 low toms, 45 and 47 toms, 48 and 50 high toms, 49, 52, 55 and 57 crashes, 51, 53 and 59 rides. A note that is not in the table, such as a melodic 60, gives null: the kit stays silent instead of guessing, because a wrong cymbal on a kick pad is worse than nothing.
Parameters
Name
Type
Description
note
Number
A MIDI note number.
Returns
String | null The drum name ('kick', 'crash'…), or null.
Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
compile() is the one function you need to turn a line of rhythm into hits. compile('bd sd hh').eventsInBar(1) answers "what happens in bar 1?" with a list of hits, sorted by pos. Bars count from 1, and one bar is one cycle of the line: the words share the bar equally, so bd sd hh puts a kick at 0, a snare at 1/3 and a hat at 2/3.
Each hit from eventsInBar(bar, { seed }) is { word, voice, n, pos, dur, vel, layer, token }. word is what was written, voice the drum it plays ('kick'), pos where in the bar, 0 up to (not including) 1. dur is how long it lasts, in bars, and may run past the bar (bd/2 lasts two). vel is always 1. layer is the index of the comma part it came from. token is { start, end }, the character range of the word in src, so you can light the word while it sounds. n is the sample number from bd:2, or null. n is kept for you but does not change the sound.
It never throws. A line it cannot read comes back as { ok: false, error }, and eventsInBar() returns [] for every bar, so check ok before you trust the hits. error.index and error.length mark the characters that are wrong, error.message and error.hint are English sentences, and error.code is one of empty, unclosed, unexpected, bad-number, bad-euclid, unknown-word or unsupported. An unknown-word error also carries suggestion, the closest word or null. The words and the symbols you can write are listed below.
text
bd sd hh steps share the bar equally
~ or - a rest
[bd sd] a group takes one step and splits it again
bd*4 faster: 4 times in its step (decimals work: bd*1.5)
bd/2 slower: once every 2 bars
<bd sd> one per bar, in turn
bd, hh*8 a comma stacks layers
bd(3,8) Euclidean: 3 hits spread over 8 steps
bd(3,8,2) the same, starting 2 steps later
bd!2 bd ! repeat the step (each copy takes a step)
bd@3 bd _ _ make the step 3 times as long
hh*8? hh*8?0.3 drop each hit by chance (1/2, or 0.3)
bd:2 sample number, kept in n
Modifiers chain in the order written (bd(3,8)*2, hh*8?). Word case does not matter (BD is bd). ? is decided by seed and the time of the hit, so the same seed and bar always give the same hits, while other bars and other seeds differ. Not supported, and refused with an unsupported error and a hint: . (Strudel's grouping), |, { }, %, .. and ^. A dot matters most: lane strings like 'x...x...' are not a line of rhythm, and a rest is ~.
Parameters
Name
Type
Description
src
String
The line of rhythm, for example 'bd sd hh'. Anything that is not a string is turned into one (null and undefined become an empty line, which is an error).
words
Object
Which words mean which drum: { word: voice }. The default is WORDS. To add a word, spread the default: { ...WORDS, boom: 'kick' }. The voice should be one the drum kit plays, see GM. optionaldefault WORDS
Returns
Object{ ok, error, src, voices, layers, period, eventsInBar(bar, { seed }) }. ok is true when the line was read. error is null, or { code, index, length, message, hint }. voices lists the drums the line uses, in the order they first appear. layers counts the comma parts. period is the number of bars until the written rhythm repeats.
Notes
Time inside is exact fractions, and each bar is computed on its own. Bar 1 000 000 of [bd sd sn]*3 lands on the same ninths as bar 1, and asking for bar 500 is as cheap as asking for bar 2.
From a p5 sketch you do not call it: startDrums() takes the line and compiles it for you. Use compile() yourself when you want the hits as data, for a score, a drawing or a test.
The way of writing comes from TidalCycles and Strudel. This is an independent implementation written from their public documentation. Where it differs from strudel.cc on purpose (a rotation of n steps or more wraps around, *0 and (9,8) are errors, unknown names are errors), the package README says so.
Examples
js
import { compile } from'@openav/pattern';
const p = compile('bd sd hh');
for (const e of p.eventsInBar(1)) console.log(e.word, e.voice, e.pos.toFixed(2));
// bd kick 0.00// sd snare 0.33// hh hat 0.67
module Logs the three hits of bar 1: the word, the drum it plays, and where in the bar. Open the console to see them.
js
import { compile } from'@openav/pattern';
const p = compile('<bd sd>, hh*4');
console.log(p.period); // 2for (const bar of [1, 2, 3]) {
const words = p.eventsInBar(bar).filter((e) => e.layer === 0).map((e) => e.word);
console.log('bar', bar, words.join(' '));
}
// bar 1 bd// bar 2 sd// bar 3 bdconst bad = compile('bd [sd');
console.log(bad.ok, bad.error.code, bad.error.index); // false unclosed 3
console.log(bad.eventsInBar(1)); // []
module A line that changes every bar: <bd sd> alternates, so period is 2. A line with a mistake comes back with ok: false, the error code and the character it starts at.
js
import { compile } from'@openav/pattern';
const p = compile('bd(3,8), [~ sd]*2, hh*8');
const g = document.getElementById('c').getContext('2d');
g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
const colors = ['#ff7b54', '#ffd166', '#6f9bff'];
for (const e of p.eventsInBar(1)) {
g.fillStyle = colors[e.layer];
g.fillRect(20 + e.pos * 360, 40 + e.layer * 70, e.dur * 360 - 4, 40); // dur is in bars
}
module Draws bar 1 on the canvas, one row per comma part: each hit is a block at its position, as long as it lasts.
Related
parse()Reads a line of rhythm into a tree of nodes, each with its character range. It does not look words up in the drum kit.
WORDSThe drum words a line of rhythm may use and the drum each one plays: bd is kick, sd is snare, hh is hat.
euclid()Spreads a number of hits as evenly as possible over a number of steps, the Euclidean rhythm. euclid(3, 8) is the tresillo.
toGrid()Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.
DrumSchedulerPure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
Spreads a number of hits as evenly as possible over a number of steps, the Euclidean rhythm. euclid(3, 8) is the tresillo.
It is the function behind bd(3,8) in a line of rhythm: compile() calls it for every (k,n). Use it on its own when you want the pattern as data, for a drawing or your own sequencer. euclid(3, 8) is x..x..x., and euclid(5, 8) is x.xx.xx..
It follows Bjorklund's procedure as Toussaint (2005) describes it. rotate moves the pattern later, the direction strudel.cc plays: euclid(3, 8, 2) is x.x..x.., the hits of euclid(3, 8) moved two steps to the right.
Parameters
Name
Type
Description
pulses
Number
How many hits. A negative number plays the gaps instead: euclid(-3, 8) is the five steps euclid(3, 8) leaves out. More hits than steps fills every step. The number is cut to a whole number.
steps
Number
How many steps. Fewer than 1 gives an empty array.
rotate
Number
Start the pattern this many steps later. Negative starts earlier, and a number of steps or more wraps around. optionaldefault 0
Returns
Boolean[] An array of steps booleans, true where there is a hit.
fromGrid(grid, { steps = 16, words = null, order = null } = {})
Turns a grid of steps, like the drum machine card, into the one line of rhythm a person would write for it.
It is what happens when you click a cell of the drum machine card: the grid has to come back as text. Each lane becomes one layer, joined with , , and it picks the shortest honest spelling. Steady hits become bd*4 or hh*8, one hit per group [~ sd]*2, a Euclidean rhythm bd(3,8), and anything else is written beat by beat, like bd [~ ~ ~ bd] bd*2 ~. It never throws.
The line is exact: compile() of the result lands on the same steps, in every bar. The rules are in the package README, and the tests check a thousand random grids.
Parameters
Name
Type
Description
grid
Object
The steps: { lane: 'x...x...' } or { lane: [1, 0, 0, 0, …] }. A lane is named by its drum (kick) or its word (bd). x, X, 1 and true are hits, | and spaces are ignored. A lane with no name the kit knows is left out.
steps
Number
How many steps each lane has. optionaldefault 16
words
Object
Keep your own spelling: { kick: 'kick' } writes kick*4 instead of bd*4. optionaldefault null
order
String[]
Drums to put first, in this order, for example ['kick', 'snare']. The rest follow in the order of the grid's keys. optionaldefault null
Returns
String One line of rhythm that plays exactly the grid. A grid with no hit anywhere gives '~'.
module A lane can be an array of 0 and 1. The hits at steps 0, 6 and 12 are recognised as a Euclidean rhythm. words keeps your own spelling, and an empty grid is one rest.
Related
toGrid()Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.
compile()Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
euclid()Spreads a number of hits as evenly as possible over a number of steps, the Euclidean rhythm. euclid(3, 8) is the tresillo.
toStrudel()Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.
Reads code copied from strudel.cc, like s("bd*4").fast(2), into the line of rhythm OAV plays, plus the tempo it found.
It lets someone paste what they made in Strudel. It reads s("…") and sound("…") with ", ' or backtick quotes. Several of them (also with a label like $: or name: in front) become layers. A .fast(n) or .slow(n) is folded into the line as […]*n or […]/n. setcpm(x) gives bpm as x times 4, setcps(x) as x times 240. .bank(…) goes to ignored, because OAV plays its own kit. Any other call or statement goes to unsupported, and the rest still plays. It never throws.
It only reshapes text: it does not check the words. fromStrudel('s("bd kik")') gives ok: true with src: 'bd kik', so pass src to compile() and look at its ok. startDrums() accepts Strudel code directly and does all of this for you.
Parameters
Name
Type
Description
code
String
The Strudel code, as copied from the strudel.cc editor.
Returns
Object{ strudel, ok, src, bpm, ignored, unsupported }. strudel is false when the text has no s(…) or sound(…) in it. ok is true when there is a line of rhythm in src. bpm is the tempo from setcpm() or setcps(), else null. ignored lists what was left out on purpose (.bank(…)), unsupported what OAV does not do (.jux(rev), .gain(…)).
module Reads a typical Strudel snippet: .fast(2) is folded into the line, the tempo is found, .bank() is ignored and .jux() is reported as unsupported.
js
import { fromStrudel } from'@openav/pattern';
console.log(fromStrudel('note("c e g")').strudel); // false, no s("…") in it
console.log(fromStrudel('s("bd kik")').ok); // true: the words are not checked
console.log(fromStrudel('setcps(0.5)\ns("bd")').bpm); // 120
module The edges: text with no s(…) is not Strudel for OAV, a misspelled word still passes, and setcps(0.5) is 120 bpm.
Related
toStrudel()Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.
strudelUrl()Makes a strudel.cc link that opens the line of rhythm in the Strudel editor.
compile()Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
Reads a line of rhythm into a tree of nodes, each with its character range. It does not look words up in the drum kit.
parse() is the first half of compile(): it checks the grammar and builds the tree, and stops there. Use it when you want to look at the structure of a line, for example to colour its words or find a mistake. For the hits, call compile().
The root is always a layer whose layers are the top-level comma parts, and each of those is a seq. The node types are step{ word, n }, rest, seq{ steps: [{ node, weight }] }, layer{ layers }, alt{ steps } (one per bar), fast and slow{ factor, node }, euclid{ pulses, slots, rotate, node } and maybe{ chance, id, node }. Every node has start and end, its character range in src. Weights and factors are fractions [numerator, denominator], so bd*1.5 has the factor [3, 2].
Parameters
Name
Type
Description
src
String
The line of rhythm, for example 'bd [sd sd]'.
Returns
Object{ ok: true, tree, error: null }, or { ok: false, tree: null, error } with the same { code, index, length, message, hint } error as compile().
Notes
Because parse() never looks at the drum kit, parse('kik') succeeds while compile('kik') fails with unknown-word. Everything else (an unclosed bracket, *0, (9,8)) fails in both.
module Logs the type of each step of the first layer (a plain step, then a group made faster), and the error code of a line with an unclosed bracket.
js
import { parse } from'@openav/pattern';
// light every word of the line: each step node knows its charactersconst src = 'bd*4, [~ sd]*2, hh*8';
const words = [];
const walk = (node) => {
if (node.type === 'step') words.push(src.slice(node.start, node.end) + '@' + node.start);
for (const s of node.steps || []) walk(s.node);
for (const l of node.layers || []) walk(l);
if (node.node) walk(node.node);
};
walk(parse(src).tree);
console.log(words.join(' ')); // bd@0 sd@9 hh@16
module Walks the tree and logs each word with the character it starts at, the data you need to colour words in an editor.
Related
compile()Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
toGrid()Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.
toStrudel()Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.
Makes a strudel.cc link that opens the line of rhythm in the Strudel editor.
The format is the one strudel.cc shows in its own address bar. It is what the drum machine card's Open in Strudel link uses. Open it in a new tab: a strudel.cc tab that is already open does not read a changed #.
Parameters
Name
Type
Description
src
String
The line of rhythm.
opts
Object
The options of toStrudel(): { bpm, comment, words }. optionaldefault {}
Returns
String A URL: 'https://strudel.cc/#' and the code from toStrudel(), as UTF-8 base64, URL-encoded.
Notes
OAV only builds the link. No Strudel code runs in, or ships with, OAV.
module Builds the link, decodes it again to show the code inside, and puts it on the page as a link that opens in a new tab.
To be confirmed
Not opened in strudel.cc from this build: the link format was read off strudel.cc's address bar on 2026-10-15 (README), and a strudel.cc change could break it.
Related
toStrudel()Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.
fromStrudel()Reads code copied from strudel.cc, like s("bd*4").fast(2), into the line of rhythm OAV plays, plus the tempo it found.
mountDrums()Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
toGrid(p, { bar = 1, steps = 16, seed = 0, bars = 16, words = WORDS } = {})
Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.
It is the way back from fromGrid(), and what the drum machine card uses to draw the cells of a line. Each hit goes to the step it is nearest to. The lost fields say what that cost: offGrid is the number of hits that sit between steps (triplets) and were moved to the nearest one. stacked counts two hits of one drum on one step. varies is true when other bars differ (< >, /2). random is true when the line has a ?. samples is true when it has a sample number like bd:2. exact is true only when none of these happened.
A line that fails to compile does not throw: you get ok: false.
Which bar to show, counting from 1. optionaldefault 1
steps
Number
How many steps the grid has. optionaldefault 16
seed
Number
Decides each ?, the same way as in eventsInBar(). optionaldefault 0
bars
Number
How many bars to look through to find out whether the other bars differ (lost.varies). At most the line's period, and never more than 64. optionaldefault 16
words
Object
Which words mean which drum, the same as in compile(). optionaldefault WORDS
Returns
Object{ ok, grid, words, voices, exact, lost }. grid is { voice: [0 or 1 × steps] }. words is the word the line used for each drum. voices lists the drums. exact is true when the grid holds the bar completely. lost counts what it cannot: { offGrid, stacked, varies, random, samples }. If the line cannot be read, ok is false and the rest is empty.
module A line that fits the sixteen steps: exact is true, and each drum has its row of 0 and 1.
js
import { toGrid } from'@openav/pattern';
console.log(toGrid('bd [sd sd sd]').lost.offGrid); // 2, the triplet sits between steps
console.log(toGrid('<bd sd>', { bar: 2 }).lost.varies); // true
console.log(toGrid('hh*8?').lost.random); // true
console.log(toGrid('bd:2').exact); // false, the sample number is lost
console.log(toGrid('bd . sd').ok); // false, not a line of rhythm
module Five lines and what each costs on a grid: triplets move, alternating bars and chance cannot be shown, and a line that does not compile has no grid at all.
Related
fromGrid()Turns a grid of steps, like the drum machine card, into the one line of rhythm a person would write for it.
compile()Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
DrumSchedulerPure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
mountDrums()Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
toStrudel(src, { bpm = null, comment = '', words = WORDS } = {})
Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.
The words become the names Strudel's drum sounds use: kick becomes bd, pedalhat becomes hh. A rotation of n steps or more in a Euclidean part, like bd(3,8,10), is written as the rotation OAV plays (bd(3,8,2)), because strudel.cc would play it unrotated. A line that does not read is passed through unchanged, so the result may not play in Strudel: check it with compile() first. It never throws.
To open the result in the Strudel editor, use strudelUrl().
Parameters
Name
Type
Description
src
String
The line of rhythm.
bpm
Number
The tempo. When it is above 0 a setcpm(bpm/4) line is written, because one OAV bar is one Strudel cycle of four beats. optionaldefault null
comment
String
A line to put first as a // comment. optionaldefault ''
words
Object
Which words mean which drum, the same as in compile(). optionaldefault WORDS
Returns
String The Strudel code, one to three lines, ending in s("…").
module Logs three lines of Strudel code: the comment, the tempo, and the pattern with Strudel's names for the drums.
js
import { toStrudel } from'@openav/pattern';
console.log(toStrudel('bd(3,8,10)')); // s("bd(3,8,2)")
console.log(toStrudel('bd . sd')); // s("bd . sd"), a line that does not read passes through
module A rotation past the number of steps is written the way OAV plays it, and a line with a mistake is not touched.
Related
strudelUrl()Makes a strudel.cc link that opens the line of rhythm in the Strudel editor.
fromStrudel()Reads code copied from strudel.cc, like s("bd*4").fast(2), into the line of rhythm OAV plays, plus the tempo it found.
compile()Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
The drum words a line of rhythm may use and the drum each one plays: bd is kick, sd is snare, hh is hat.
The short words are the ones TidalCycles and Strudel use: bd kick, sd and sn snare, hh hat, oh open hat, cp clap, rim and rs rim, lt low tom, mt tom, ht high tom, cr crash, rd ride, cb cowbell. Each drum's own name works too: kick, snare, hat, openhat, pedalhat, clap, lowtom, tom, hightom, crash, ride, cowbell. The values are the drum names of GM.
It is the default of the words option of compile(), toGrid() and toStrudel(). To teach a line a new word, pass a copy with your word added.
Type
Object
A frozen object of 26 { word: voice } pairs. Keys are lowercase, and compile() ignores the case of what you write.
An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
Define function barStarted(bar) { … } to do something once per bar: function barStarted(bar) { drift += 9; }. It is called in the frame where the first step of the bar is heard, right before draw(), and the first call is barStarted(1) when the machine starts. Within a step, cueFired() comes before it and stepStarted() after.
Parameters
Name
Type
Description
bar
Number
The bar that starts, counted from 1.
Notes
While the tab is hidden it is still called once per bar, but nothing is told twice when the tab comes back. stepStarted() and the hits are not told in that time.
let shade = 200;
function setup() {
createCanvas(400, 300);
colorMode(HSB, 360, 100, 100);
startDrums('four on floor', 150);
}
function barStarted(bar) {
shade = (shade + 40) % 360; // a new colour every bar
}
function draw() {
background(shade, 60, 80);
fill(0, 0, 100);
textAlign(CENTER, CENTER);
textSize(48);
text(drumBar, width / 2, height / 2);
}
The background changes colour at the start of every bar, while the number counts the bars.
Related
stepStarted()An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.
cueFired()An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
drumBarWhich bar the drum machine is in, counted from 1 when it started.
drumHit()An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
OAVp5.drums.on()Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
The drum machine counts the bars, so it can run a plan: cue(1, 'calm', 12); cue(5, 'bloom', 12); cue(9, 'storm', 12); in setup() makes three scenes that come round every 12 bars. When a cue is due, cueFired(name, bar) is called and cueName holds the name; read it in draw() if you prefer that to a callback. Several cues on one step fire in the order you declared them, before barStarted(), stepStarted() and the hits of that step.
Calling cue() twice with the same values adds nothing, so it is safe in draw() too. Counting starts with startDrums(), and a cue only fires when the machine passes its place: declare the cues right after startDrums() in setup() so that a cue at bar 1 is not missed. After the machine is stopped and started again, the cues fire again from bar 1.
Parameters
Name
Type
Description
bar
Number
The bar, counted from 1 (the first bar). A fraction is a place inside a bar: 4.5 is halfway through bar 4 (the nearest of the 16 steps is used). A value below 1, or one that is not a number, prints a warning in the console and adds nothing.
name
Any
What to call it, usually a string. It is passed to cueFired() and becomes cueName.
every
Number
Repeat every this many bars after the first time (fractions work, at least one step). 0 or left out: only once. optionaldefault 0
Returns
undefined Nothing. OAVp5.drums.cue() is the same function and returns the cue it made.
Notes
A cue declared after its bar has passed fires at its next repeat if it has an every, otherwise never until the machine is started again. Without a running machine no cue fires at all.
function setup() {
createCanvas(400, 300);
startDrums('latin', 120);
textAlign(CENTER, CENTER);
textSize(28);
}
function draw() {
cue(2, 'drop', 4); // the same call every frame adds nothing
background(20);
fill(255);
text(cueName === '' ? 'waiting for the first cue' : 'last cue: ' + cueName, width / 2, height / 2);
}
cue() inside draw() is safe: the text changes to "last cue: drop" at bar 2 and stays that way.
Related
cueFired()An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
cueNameThe name of the last cue that fired, or an empty string before the first one.
barStarted()An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
drumBarWhich bar the drum machine is in, counted from 1 when it started.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
OAVp5.drums.cue()The same as cue(bar, name, every), but it returns the cue it made (or the one that was already there), and null when bar is not valid.
An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
Define function cueFired(name, bar) { … } and it is called at the step where a cue is due, before barStarted(), stepStarted() and the hits of that step. A good place to change a scene: function cueFired(name) { look = SCENES[name]; }. If you only need the latest name, read cueName in draw() instead.
The bar the cue fired in, counted from 1. For a cue in the middle of a bar (4.5) it is the whole bar it falls in (4).
Notes
Cues keep firing while the tab is hidden, so the scene is right when you come back. Several cues on one step call it once each, in the order they were declared.
let scene = '-';
let when = 0;
function setup() {
createCanvas(400, 300);
startDrums('four on floor', 150);
cue(1, 'intro');
cue(3, 'verse', 4);
cue(5, 'chorus', 4);
textAlign(CENTER, CENTER);
textSize(32);
}
function cueFired(name, bar) {
scene = name + ' (bar ' + bar + ')';
when = millis();
}
function draw() {
background(20 + max(0, 1 - (millis() - when) / 400) * 120); // a flash on every cue
fill(255);
text(scene, width / 2, height / 2);
}
The screen flashes and names the scene each time a cue fires: intro once, then verse and chorus take turns.
Related
cue()Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
cueNameThe name of the last cue that fired, or an empty string before the first one.
barStarted()An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
stepStarted()An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.
OAVp5.drums.on()Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
The name of the last cue that fired, or an empty string before the first one.
Read it in draw() instead of waiting for cueFired(): look = SCENES[cueName];. It is the name of the last cue only, so it is the right thing for "which scene am I in", and cueFired() is the right thing for "something just began". Cues are declared with cue().
Type
String
Range
'' or a cue name
Changes when a cue fires, and keeps the name until the next one. It is '' at the start.
Notes
A sketch that declares its own variable called cueName keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.drums.lastCue instead.
The background and the word follow the last cue: blue "calm", pink "bloom", grey "storm", every six bars.
Related
cue()Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
cueFired()An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
drumBarWhich bar the drum machine is in, counted from 1 when it started.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
Which bar the drum machine is in, counted from 1 when it started.
A bar is 16 steps, four beats. Use it to give a piece a shape: floor((drumBar - 1) / 4) % 3 is the section number when you change every four bars. To make something happen at an exact bar, cue() is neater than testing drumBar in draw(). barStarted() is the function that is called at the start of each bar.
Type
Number
Range
0, 1, 2, 3 …
0 before the machine has played its first step, then 1, 2, 3 … with no upper limit. It keeps its last value when the machine stops.
Before it has a value
Start the machine with startDrums(). Stopping and starting again counts from bar 1 again.
Notes
A sketch that declares its own variable called drumBar keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.drums.bar instead.
The background changes colour every two bars and the text names the bar and the section.
Related
barStarted()An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
drumStepWhich of the 16 steps of the bar the drum machine just played, from 0 to 15.
cue()Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
OAVp5.drums.countHow many steps the drum machine has played since it started, from 0; -1 before the first one.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
Which of the 16 steps of the bar the drum machine just played, from 0 to 15.
A bar has 16 steps (sixteenth notes) and a beat has 4, so drumStep % 4 === 0 is a beat and drumStep === 0 the start of a bar. At 120 bpm a step lasts 125 ms. Use it to light a cell or pick a frame: fill(i === drumStep ? 255 : 80). If you would rather be called on each step, define stepStarted(). For something that moves smoothly instead of in 16 jumps, use drumPhase.
Type
Number
Range
0..15
Changes in the frame in which a step is heard, right before draw(). 0 at the start, and it keeps its last value when the machine stops.
Before it has a value
Start the machine with startDrums(). Before that, and when the machine is stopped, the value is 0 or the last one, so check drumsAreOn if the difference matters.
Notes
A sketch that declares its own variable called drumStep keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.drums.step instead. While the tab is hidden the clock keeps counting but nothing else is told.
function setup() {
createCanvas(400, 300);
startDrums('four on floor', 120);
noStroke();
}
function draw() {
background(20);
for (let i = 0; i < 16; i++) {
fill(drumsAreOn && i === drumStep ? 255 : 70);
rect(20 + i * 22, 130, 18, 40); // sixteen cells, the current one lights up
}
}
Sixteen cells in a row with the current step lit, one cell at a time, from left to right.
function setup() {
createCanvas(400, 300);
startDrums('breakbeat', 110);
textAlign(CENTER, CENTER);
textSize(18);
}
function draw() {
background(drumStep % 4 === 0 ? 70 : 20); // a flash on every beat
fill(255);
text('step ' + drumStep + ' of 16' + (drumStep === 0 ? ' (bar starts)' : ''), width / 2, height / 2);
}
The background lights up on every beat (steps 0, 4, 8, 12) and the text counts the steps.
Related
stepStarted()An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.
drumBarWhich bar the drum machine is in, counted from 1 when it started.
drumPhaseHow far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
beatPhaseHow far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
drumsAreOnWhether the drum machine is playing right now.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.
Define function stepStarted(step, bar) { … } for something on every step: if (step % 4 === 0) ping = 1; is a pulse on the beat. It is called in the frame where the step is heard, right before draw(), after barStarted() and before the hits of that step. drumStep holds the same number if you would rather read than be called.
Parameters
Name
Type
Description
step
Number
The step that starts, 0 to 15. Every 4 steps is a beat.
bar
Number
The bar it is in, counted from 1.
Notes
It is not called while the tab is hidden, so a sketch that adds a particle per step does not burst when you come back.
let ping = 0;
function setup() {
createCanvas(400, 300);
startDrums('breakbeat', 120);
noStroke();
}
function stepStarted(step, bar) {
if (step % 4 === 0) ping = 1; // on every beat
}
function draw() {
background(20);
ping *= 0.9;
fill(255, 160, 80);
circle(width / 2, height / 2, 40 + ping * 120);
}
A circle that pulses on every beat (every fourth step) and shrinks back in between.
Related
drumStepWhich of the 16 steps of the bar the drum machine just played, from 0 to 15.
barStarted()An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
drumHit()An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
cueFired()An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
OAVp5.drums.on()Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
How many steps the drum machine has played since it started, from 0; -1 before the first one.
It is the single clock the others are cut from: drumBar is floor(count / 16) + 1, drumStep is count % 16, and cue() fires when count reaches the step of its bar. It is handy when you want one number that only grows: a position on a very long timeline, or an index into a list.
Type
Number
Range
-1, 0, 1, 2 …
Goes up by one at every step and never goes back to 0 at a new bar. It starts again from 0 when the machine is started again.
Before it has a value
Start the machine with startDrums(); it is -1 until the first step has played.
The running step count and the bar and step it is made of.
Related
drumBarWhich bar the drum machine is in, counted from 1 when it started.
drumStepWhich of the 16 steps of the bar the drum machine just played, from 0 to 15.
cue()Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
The same as cue(bar, name, every), but it returns the cue it made (or the one that was already there), and null when bar is not valid.
Use it when you want to hold on to the cue, for example to find out which step it fires at. For a plain plan, cue() is shorter. The list of every cue is OAVp5.drums.cues.
Parameters
Name
Type
Description
bar
Number
The bar, counted from 1; 4.5 is halfway through bar 4. See cue().
name
Any
What to call the cue.
every
Number
Repeat every this many bars; 0 or left out: once. optional
Returns
Object | null The cue: { bar, name, every, at, period, key } with at and period counted in steps. null if bar is below 1 or not a number. A repeat of an earlier call gives back the cue that exists.
let c = null;
function setup() {
createCanvas(400, 300);
startDrums('four on floor', 120);
c = OAVp5.drums.cue(3.5, 'half', 4);
textAlign(CENTER, CENTER);
}
function draw() {
background(20);
fill(255);
text(c ? 'cue "' + c.name + '" at step ' + c.at + ', every ' + c.period + ' steps' : 'no cue', width / 2, height / 2);
}
Bar 3.5 is step 40 of the count, and "every 4 bars" is 64 steps.
Related
cue()Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
OAVp5.drums.cuesThe list of every cue declared so far, in the order they were declared.
cueFired()An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
cue()Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
OAVp5.drums.cue()The same as cue(bar, name, every), but it returns the cue it made (or the one that was already there), and null when bar is not valid.
cueNameThe name of the last cue that fired, or an empty string before the first one.
Runs a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
A segment module is the code that performs one segment (the dawn, the storm, the finale). The director calls its hooks as the show moves through the score:
js
{
id: 'dusk', // the id of the segment it performs
enter(api, ctx) {}, // the segment began, or a seek landed inside it
update(api, ctx) {}, // every frame while the segment (and its linger) is current
exit(api, ctx) {}, // it ended, or a seek left it: give back what you took
param(key, value, ctx) { return value; }, // optional: rewrite a timeline param meanwhile
cue(name, ctx) {}, // optional: one of its cues was just passed
reset(api) {}, // optional: R was pressed, forget everything
}
Every hook gets a ctx: p (progress through the segment, 0 to 1; above 1 during linger; exactly 1 while holding), t (seconds into the segment), dt, dur, T (show time), segment, cue(name) and cueT(name) (where a cue sits, as a progress and in show seconds), phase ('active', 'linger', or 'failed' inside exit() after a crash), holding, held (seconds waited at a hold: a clock for ambient motion while T is frozen), playing and api.
A module is entered once and exited once, however the show got there: a seek into the middle of a segment calls enter with the right p, so a module must be able to start from the middle. A module that throws is switched off alone: the director calls its exit() once with phase: 'failed', logs one console.error, and never calls it again; the other modules and the show carry on. Reset brings it back.
In a show, createShow() builds the director from its score option and calls update() every frame after the timeline has advanced; the director's param() plugs into timeline.layer. Without createShow() you do that wiring yourself: director.update(timeline.t, dt, { holding: timeline.holding, playing: timeline.playing }).
Parameters
Name
Type
Description
score
Score
The Score to perform. Required: without it the constructor throws "Director needs a score".
modules
Array | Object
The segment modules: an array (matched to segments by each module's id) or an object keyed by segment id. A segment with no module is simply not performed. optionaldefault []
api
Object | Function
What every module hook receives as its first argument (also ctx.api). A function is called once, the first time it is needed, and its result is used. optionaldefault {}
onStatus
Function | null
Called with status() when the segment, the holding state or the list of switched-off modules changes, not every frame. optionaldefault null
onCue
Function | null
Called for each cue the playhead passes: { name, cue, T, n, segment }. optionaldefault null
maxStep
Number
A jump bigger than this many seconds between two update() calls counts as a seek, so the cues in between are not fired. Raise it if you rehearse at a very high timeline.rate. optionaldefault 2
Call once per frame, after the timeline advanced: update(T, dt, { holding, playing }) with T the show time. Enters, updates and exits the modules and fires the cues passed since the previous call.
The timeline's param layer: param(key, value, T) lets the modules of the current segments rewrite the value (a module returns a number to replace it). A performer's override still wins afterwards.
For the desk and the backstage: { cut, T, total, segment, next, holding, held, active, failed, errors, missing }, with the current segment's progress and time remaining, the next segment and its countdown, and who is switched off.
Notes
The last segment is never exited by time: while the show sits at its end (or past it), the last module stays entered. It exits on a seek that leaves it, or on reset().
A new segment's module is entered before the old one exits, and during linger both run at the same time.
A module whose module or id matches no segment, or a segment that names a module nobody registered (listed in status().missing), is simply not performed. The show keeps going.
Examples
js
import { Score, Director } from'@openav/score';
const score = new Score({ cuts: [{ id: 'full', segments: [
{ id: 'dawn', dur: 4, cues: { birds: 0.5 } },
{ id: 'noon', dur: 4 },
] }] });
const say = (id) => ({
id,
enter(api, ctx) { console.log(id, 'enter, p =', ctx.p); },
exit() { console.log(id, 'exit'); },
cue(name) { console.log(id, 'cue', name); },
});
const director = new Director({ score, modules: [say('dawn'), say('noon')] });
for (let T = 0; T <= 8; T += 1) director.update(T, 1, { playing: true }); // one call per frame
module The console prints dawn enter, p = 0, dawn cue birds (second 2), then noon enter, p = 0 and dawn exit at second 4. The last segment stays entered to the end.
module While dawn is current its module bends only the sun param, following its own progress: the console prints 0.25 0.9, then 0.5.
js
import { Score, Director } from'@openav/score';
console.error = (message) => console.log('logged:', String(message).slice(0, 62)); // show the director's error line hereconst score = new Score({ cuts: [{ id: 'full', segments: [{ id: 'dawn', dur: 4 }, { id: 'noon', dur: 4 }] }] });
const dawn = { id: 'dawn', update() { thrownew Error('boom'); } };
const noon = { id: 'noon', enter() { console.log('noon is still performed'); } };
const director = new Director({ score, modules: [dawn, noon] });
director.update(1, 1, { playing: true }); // dawn throws: it is switched off, once
director.update(5, 1, { playing: true }); // the show goes on
console.log(director.status().failed, director.status().errors);
director.reset(); // brings dawn back
console.log(director.status().failed);
module A crash is contained: the console prints logged: [score] segment module "dawn" crashed in update: boom and was (normally a red error line), then noon is still performed, [ 'dawn' ] { dawn: 'update: boom' }, and after the reset [].
Related
ScoreA show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.
ScoreErrorThe error a Score throws when it cannot be built: its problems array lists every reason at once, and its message lists them too.
TimelineAnswers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
mountConsole()Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.
A show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.
A timeline written as a table of absolute seconds cannot grow: make one scene longer and every later cue, curve and if (t > 466) is wrong. In a score you write only how long each segment is, and Score computes every start, end, cue time, countdown and scene from those lengths. Change one dur and everything after it moves; a second version of the show is a few lines of data.
A cut is one version of the show; a segment is one stretch of it:
js
{ id: 'full', title: 'A day', segments: [
{ id: 'standby', dur: 6, hold: true, acts: ['Press Space when the room is ready'] },
{ id: 'dawn', dur: 24, cues: { birds: 0.3 } }, // a ratio of the segment
{ id: 'noon', dur: 30, cues: { flare: { s: 8 } } }, // or seconds into it
{ id: 'dusk', dur: 24, linger: 8 },
] }
{ id: 'short', from: 'full', scale: 0.5 } // the same show at half the length
dur is the only time a segment has. cues are named moments (a ratio 0 <= r < 1, or { s: seconds }), acts is what the human operator does (strings shown on the desk), linger keeps a segment's module running for some seconds after it ends, hold stops playback at the end of the segment until the operator releases it, and module names the module that performs it when that is not the segment's own id. A derived cut scales dur, linger and { s } cues, so a cue keeps its place in the music.
Everything is checked when the score is built. Duplicate ids, a dur of 0 or below, a from that is not a cut, a cut derived from itself, a cue outside its segment: each throws a ScoreError that lists every problem at once. In a show, createShow() builds the score from its score option and hands it to a Timeline and a Director; ?cut=<id> in the page address picks the cut.
Parameters
Name
Type
Description
cuts
Array | Object
The cuts this show can be performed as: an array of cuts, or an object keyed by cut id. A cut is { id, title?, start?, segments: [...] }, or a derived one, { id, title?, from, scale }. Required: without a cut it throws.
cut
String
The id of the cut to perform. Defaults to the first one; an id that is not in cuts is an error. optionaldefault the first cut
A moment in show seconds: 'dusk' is the segment's start, 'dusk.end' its end, 'dusk.glow' a cue declared in the cut. NaN when the name means nothing here.
Time T expressed on the cut this one was derived from (T / scale). Equal to T when the cut is not derived. This is how automation written once against the full cut stretches with a short one.
The segments as a scenes array for a Timeline, with acts, hold and cues (cue times in show seconds).
Notes
Plain fields: cut (the id being performed), title, segments (each with start, end, dur, index, cues …), total (the length of the show), startT (where reset goes), scale and base.
Segment and cue ids must not contain a dot: the dot is how cueTime('dusk.glow') separates a segment from its cue. A cue cannot be called end.
If two cuts differ in structure and not only in length, do not share automation between them. Drive the params from the segment modules instead, where ctx.p already scales for free.
module The console shows a table of the three segments (dawn 0 to 24, noon 24 to 54, dusk 54 to 80), then 80 noon 32: the show is 80 seconds long, second 40 is in noon, and the flare cue sits at 32.
module The timeline takes its scenes and total from the score, and reads the haze curve (written once for the full cut) stretched: full 80 a@0 b@20 c@60 0.25, then short 40 a@0 b@10 c@30 0.5.
Related
ScoreErrorThe error a Score throws when it cannot be built: its problems array lists every reason at once, and its message lists them too.
DirectorRuns a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
TimelineAnswers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
The error a Score throws when it cannot be built: its problems array lists every reason at once, and its message lists them too.
You rarely build one yourself: new Score() throws it. error.problems is an array of strings and error.message is the same list as one text, headed "Score is invalid (3 problems)". error.name is 'ScoreError', and it is a normal Error, so instanceof ScoreError and instanceof Error both work.
It lists every problem it found, not only the first, so one run of the show shows you everything to fix. Nothing half-valid can reach the stage.
Parameters
Name
Type
Description
problems
Array<String>
The reasons, one string each. Each one names the cut, the segment and what is wrong.
module Three mistakes, all reported at once: the console prints true 3 and then one line for each (a dur of 0, a repeated id, a from that is not a cut).
Related
ScoreA show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.
DirectorRuns a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
The curve names a keyframe can use to say how its segment gets there: linear, ease, in, out, step, each a function from 0..1 to 0..1.
In a Timeline, the third item of a keyframe names the curve of the segment that ends at that keyframe: [30, 320, 'ease'] means "arrive at 320 at second 30 the ease way". linear is the default, and an unknown name or a typo also falls back to linear, so a mistake never stops a show. You can pass your own function instead of a name.
linear goes straight. ease is a smoothstep: slow, fast, slow. in starts slowly (f²). out starts fast and lands softly (1 − (1 − f)²). step is special: the segment holds the old value until the keyframe, then jumps. The step function stored here returns 0 and is never called to interpolate; it is listed so that the object is the whole vocabulary.
A param declared with step in its Params schema holds each keyframe whatever curve you name.
Examples
js
import { CURVES } from'@openav/timeline';
for (const name of ['linear', 'ease', 'in', 'out']) {
console.log(name, CURVES[name](0.5)); // halfway through the segment
}
module The console prints how far each curve has got at the halfway point: linear 0.5, ease 0.5, in 0.25, out 0.75.
module One lane, three curves: the console prints a slow start (8 0.63), the arrival at 16 1.60, a long hold (19 1.60) until the step jumps at 20 (20 3.40), then an out curve down to 30 0.80.
js
import { Params } from'@openav/core';
import { Timeline } from'@openav/timeline';
const params = new Params([{ key: 'speed', min: 0, max: 4, def: 0 }]);
const timeline = new Timeline({ params, automation: {
speed: [[0, 0], [10, 4, (f) => f * f * f]], // your own curve: a function of 0..1
} });
console.log(timeline.valueAt('speed', 5));
module A function works wherever a name does. Halfway through, a cubic curve is only an eighth of the way: the console prints 0.5 (4 × 0.125).
Related
TimelineAnswers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
approach()One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.
How far before the boundary the playhead parks when a hold stops the show: one microsecond, so it still belongs to the scene that is waiting.
A scene with hold: true stops playback at its end. The Timeline parks HOLD_EPS (0.000001 seconds) before the boundary rather than on it, so sceneIndexAt(t) and the director still say "the held scene", timeline.t reads 5.999999 for a hold at 6, and the next scene has not started. release() then moves the playhead onto the boundary itself.
You only need it when you compare the playhead with a boundary yourself: while holding, timeline.t is boundary - HOLD_EPS, not exactly boundary.
module The show parks just before second 6, still inside the held scene: the console prints true wait true (6 - timeline.t is HOLD_EPS, apart from rounding).
Related
TimelineAnswers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
DirectorRuns a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
ScoreA show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.
Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
timeline.state(t) returns { key: value } for every param at time t, interpolated from automation. That is the whole job: the timeline owns time, and a world never reads the clock. In a show, createShow() builds the timeline from its timeline option, Loop calls advance(dt) every frame, and what state() returns goes into params.resolve(), where a hand or a knob that has overridden a param still wins.
Between two keyframes the value moves in a straight line. The optional third item of a keyframe says how the segment that ends at that keyframe gets there: 'linear', 'ease', 'in', 'out', 'step', or your own function of 0..1 (see CURVES). Before the first keyframe the value is the first one, after the last it is the last one, and a param with no keyframes stays at its def. A param declared with step holds each keyframe instead of interpolating.
Scenes give the show its shape: { id: 'storm', t: 90, title: 'Storm', note: 'clusters welcome' }. onSceneChange is the hook to fire sound, lights or prompts, and the console shows the title and note to the performer. A scene with hold: true stops playback at its end until someone calls release(). Scene cues are { name, t } with t in show seconds (not seconds into the scene), and they fire only when playback passes them: a seek, a jump or a reset never does.
Before you call it
Time only moves when you call play() and then advance(dt) every frame (a show does this for you). state(t) and valueAt(key, t) work at any moment, playing or not. The callbacks do not fire for the starting scene until the first advance() or seek().
Parameters
Name
Type
Description
params
Params | Array
The Params of the show, or a plain schema array. The timeline reads the schema to know which params to resolve. Required.
automation
Object
The keyframes of each param: { hue: [[0, 200], [30, 320, 'ease'], [60, 40]] }. A keyframe is [seconds, value, curve?], sorted by time. optionaldefault {}
scenes
Array
The scenes: { id, t, title?, note?, acts?, hold?, cues? }, sorted by t. A negative t is a pre-show standby. Ignored when a score is given. optionaldefault []
total
Number
The length of the show in seconds. Playback stops by itself there. (createShow() passes 120 unless you say otherwise.) Ignored when a score is given. optionaldefault 600
score
Score | null
A Score: the scenes, the total and the start then come from it, and automation is read at score.baseTime(t), so a derived cut stretches the curves with it. optionaldefault null
start
Number | 'first' | null
Where reset() goes and where the playhead starts: a number of seconds, or 'first' for the start of the first scene (a standby at −30, say). null means 0, or the score's start. optionaldefault null
loop
Boolean | String | Object
true wraps the show at its end and plays on; 'scene' repeats the scene the playhead is in; a scene id repeats that scene; { from, to } repeats a range (seconds or scene ids). It is a plain property: set timeline.loop any time. optionaldefault false
onCue
Function | null
The same as calling timeline.onCue(cb) once: cb(cue, { cause }). optionaldefault null
Every param at time t as { key: value }. This is the base state that params.resolve() then lets overrides win over. If layer is set, each value passes through it.
Register cb(index, scene, { cause }), called when the playhead lands in a different scene. cause is 'play', 'release', 'seek', 'jump', 'reset' or 'loop'. There is no way to remove it again.
Move the playhead to t, held between the first scene's start (when negative) or 0, and total. Leaving a hold this way releases it. cause (default 'seek') is what onSceneChange and onSeek report. A seek never fires cues.
Call once per frame with the seconds that passed. Moves t by dt × rate while playing, parks at a hold, wraps a loop, stops at total, and fires scene changes and cues on the way. Returns whether it is still playing.
Arm a lane: while the timeline plays, whatever source() returns is written into automation[key] as keyframes. source returns a number while someone is touching the control and null when not; that closes the take. Throws for an unknown key or a source that is not a function. Returns the recorder ({ key, take, active, commit(), stop() }).
Write a take (an array of [t, value]) into automation[key]: thin it, replace the keyframes inside its time span, keep the ones outside. Returns { from, to, count }, or null for fewer than two points.
Notes
These are plain fields you can read and set: t, playing, holding, rate (1 is real time; 4 rehearses at four times the speed), total, start, scenes, automation, score, loop, loops (laps since the last reset), and layer.
layer is (key, value, t) => value. A Director plugs in here so segment modules can bend a param; the performer's override is still applied afterwards and wins.
With a score, anything you pass as scenes or total is ignored, and reset() goes to the score's start.
module The console prints the hue at four moments: 0 200, 15 260, 30 320, 45 180. Nothing plays; state(t) just looks the answer up.
js
import { Params } from'@openav/core';
import { Timeline } from'@openav/timeline';
const params = new Params([{ key: 'energy', min: 0, max: 1, def: 0.2 }]);
const timeline = new Timeline({
params,
total: 20,
scenes: [
{ id: 'standby', t: 0, hold: true },
{ id: 'go', t: 6, cues: [{ name: 'flash', t: 10 }] },
],
});
timeline.onSceneChange((i, scene, { cause }) => console.log('scene', scene.id, cause));
timeline.onCue((cue) => console.log('cue', cue.name, 'at', cue.t));
timeline.play();
for (let i = 0; i < 8; i++) timeline.advance(1); // one second per call
console.log('holding:', timeline.holding, 'at', timeline.t.toFixed(3));
timeline.release(); // the operator says gofor (let i = 0; i < 6; i++) timeline.advance(1);
module The console prints scene standby play, then holding: true at 6.000 (the show waits at the end of the standby), then scene go release, then cue flash at 10.
js
import { Params } from'@openav/core';
import { Timeline } from'@openav/timeline';
const params = new Params([{ key: 'hue', min: 0, max: 360, def: 100 }]);
const timeline = new Timeline({ params, total: 30, automation: { hue: [[0, 100], [30, 100]] } });
let knob = null; // a number while a hand holds the knob
timeline.record('hue', () => knob);
timeline.onRecord((take) => console.log('take', take.from, 'to', take.to, take.cause));
timeline.play();
for (let i = 0; i < 20; i++) {
knob = i >= 5 && i < 12 ? 100 + (i - 5) * 20 : null; // turned from second 5 to 11
timeline.advance(1);
}
console.log(JSON.stringify(timeline.automation.hue));
module The knob turns for seven seconds; the console prints take 6 to 12 release, then the new keyframes: the take replaced the middle of the curve and the old ends stayed.
Related
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
LoopThe frame clock: calls your function every animation frame with the seconds since the last one, and keeps ticking when the tab is hidden.
ScoreA show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.
DirectorRuns a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
CURVESThe curve names a keyframe can use to say how its segment gets there: linear, ease, in, out, step, each a function from 0..1 to 0..1.
HOLD_EPSHow far before the boundary the playhead parks when a hold stops the show: one microsecond, so it still belongs to the scene that is waiting.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
mountConsole()Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.
Counts how often on has turned on, or how many 'kick' (or any other event) have happened; with a limit it loops 0, 1, … limit − 1.
A counter you do not have to keep yourself: count(mouseIsPressed) is the number of clicks so far, count(beat, 4) walks 0 1 2 3 0 1 … on the beat, and count('kick') counts the kicks of the drum machine. A switch has to go off and on again to be counted twice.
Event counts start when count('kick') is first called, so earlier kicks are not counted. An event of your own fired with trigger() is counted from its first firing, whenever count() is first called.
Before you call it
The event names work like env(): 'kick', 'snare', 'hat' and 'beat' come from the microphone, or from the drum machine once startDrums() has been called, and the first count('kick') asks for the microphone if the drum machine is not on. The first call loads @openav/chop, which takes a few frames.
Parameters
Name
Type
Description
on
Boolean | Number | String
A switch: a boolean, or a number above 0.5, and each time it goes from off to on counts one. Or an event name ('kick', 'beat', 'note', or one of your own fired with trigger()): how many of those events have happened.
limit
Number
Loop at this number: 4 gives 0 1 2 3 0 1 … Left out, or 0 or less, it keeps counting up. optionaldefault 0
name
String | Number
Only for the switch form (count(beat, 4, 'a')): a name for its memory, as the last argument. The event-name form needs none. optionalread from arguments
Returns
Number The count, a whole number from 0. It is 0 until @openav/chop has loaded (a few frames), and for an unnamed switch outside draw().
Notes
Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
A sketch with its own variable or function called count keeps it. OAVp5.chop.count(…) is this function under another name.
beatTrue for one frame each time the microphone (or the drum machine) hears a beat.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
OAVp5.chopThe CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
Draws a line through values on the canvas, so it is in the work and in a recording: the oldest at x, lo at the bottom of the box, hi at the top.
The picture scope() shows in the corner, but drawn on the canvas: stroke(255); drawTrail(trail(micLevel, 3), 0, height - 100, width, 100). The line is one shape in the sketch's own stroke() colour and weight (white while noStroke() is in effect), with no fill, and the style you had before is put back afterwards.
The oldest value is at the left edge x, the newest at x + w. A number instead of an array is the shortcut: drawTrail(micLevel, 0, 0, width, 100) draws the last 2 seconds without a separate trail() call.
Before you call it
Call it in draw() after createCanvas(). With a number, the first call loads @openav/chop, which takes a few frames, and nothing is drawn until then.
Parameters
Name
Type
Description
values
Array<Number> | Number
An array such as the one trail() returns. Or a single number: then drawTrail keeps its own trail of the last 2 seconds.
x
Number
Left edge of the box. optionaldefault 0
y
Number
Top edge of the box. optionaldefault 0
w
Number
Width of the box. Left out, it is the canvas width. optionaldefault width
h
Number
Height of the box. Left out, it is the canvas height. optionaldefault height
lo
Number
The value drawn at the bottom of the box. optionaldefault 0
hi
Number
The value drawn at the top of the box. Values outside lo..hi sit on the edge of the box. If hi equals lo, the line runs through the middle. optionaldefault 1
name
String | Number
Only when values is a single number: a name for its own trail, as the 8th argument (after hi). optionalread from arguments
Returns
Array<Number> The array that was drawn: the one you passed, or the number's own trail. Nothing is drawn when it has fewer than 2 values.
Notes
Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
OAVp5.chop.drawTrail(…) is this function under another name.
function setup() {
createCanvas(400, 300);
noFill();
stroke(255);
}
function draw() {
background(20);
const wave = sin(frameCount * 0.05); // -1..1
drawTrail(trail(wave, 3), 20, 100, width - 40, 100, -1, 1); // lo and hi say what the box shows
}
A wave that goes below 0: lo = -1 and hi = 1 put the bottom of the box at -1 and the top at 1, so the whole wave fits.
Related
trail()The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).
scope()Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
lag()Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
OAVp5.chopThe CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.
Call it every frame and use the number like a size or a brightness: circle(width / 2, height / 2, 60 + env('kick', 0.01, 0.3) * 200) swells on every kick. The number rises to 1 in attack seconds after an event and falls back to 0 in decay seconds, quick at first and landing softly, the way a bass drum's swell looks right.
An event that arrives while the last pulse is still sounding restarts from where the pulse is, never from zero, so fast hits pile up into a plateau instead of flickering. The attack is measured from 0: a retrigger from halfway up takes half the attack.
The events: 'kick', 'snare' and 'hat' are the hits kickDetected() hears (from the microphone, or from the drum machine when it plays), 'beat' is beat, 'note' is any key going down, 'bar' and 'step' come from the drum machine, a cue's name fires when cue() does, and any other name is yours to fire with trigger(). The switch form, env(mouseIsPressed, 0.02, 0.5), fires on every press.
Before you call it
'kick', 'snare', 'hat' and 'beat' need something that makes them. If the sketch has not started the drum machine (startDrums()), the first env('kick') asks for the microphone, like reading micLevel, and ?oav_mic=off means nothing ever fires. Once the drum machine is started, the microphone is not opened for it. 'note' needs a key: play one, press 🎹, or call OAVp5.play(). A name of your own needs trigger(): if nothing fires it within 4 seconds, a card says so.
Parameters
Name
Type
Description
event
String | Boolean | Number
What to react to. A name: 'kick', 'snare', 'hat', 'beat' (from the microphone or the drum machine), 'note' (any key), 'bar', 'step' (the drum machine), a cue's name, or an event of your own fired with trigger(). Or a switch: a boolean, or a number above 0.5, and each time it turns on counts as an event. Left out: 'beat'. optionaldefault 'beat'
attack
Number
Seconds the rise from 0 to 1 takes. optionaldefault 0.01
decay
Number
Seconds the fall from 1 back to 0 takes. optionaldefault 0.3
name
String | Number
Only for the switch form (env(mouseIsPressed, …, 'a')): a name for its memory, as the last argument. A named event such as 'kick' needs none. optionalread from arguments
Returns
Number 0..1: 0 when nothing has happened, 1 at the top of the pulse. Until @openav/chop has loaded (a few frames) it returns 0.
Notes
A named event needs no place in draw(): env('kick') works in setup() or mousePressed() too. Only the switch form remembers anything between frames, and an unnamed switch outside draw() returns 0. Calls in draw() are matched by order, like lag().
OAVp5.chop.env(…) is this function under another name.
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
const k = env(mouseIsPressed, 0.02, 0.5); // 1 right after each press, then it fades
circle(width / 2, height / 2, 60 + k * 200);
}
Click the canvas: the circle jumps open and fades back each time.
function setup() {
createCanvas(400, 300);
}
function mousePressed() {
OAVp5.play(48 + floor(random(24)), 0.8); // play a key, as a controller would
}
function draw() {
background(20);
const k = env('note', 0.005, 0.8); // every key makes it flash
fill(255, 211, 107);
circle(width / 2, height / 2, 40 + k * 240);
}
Click the canvas, or play a key on a controller or the 🎹: every key pulses the circle.
count()Counts how often on has turned on, or how many 'kick' (or any other event) have happened; with a limit it loops 0, 1, … limit − 1.
lag()Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
beatTrue for one frame each time the microphone (or the drum machine) hears a beat.
kickDetected()You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
OAVp5.chopThe CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
Wrap any number that jumps: circle(width / 2, height / 2, 50 + lag(micLevel, 0.03, 0.4) * 300) is a VU meter. It opens within 0.03 seconds when someone speaks and takes 0.4 seconds to close. up is the time to follow a rise, down the time to follow a fall. Leave down out and both are up. Leave both out and they are 0.1.
The seconds are TouchDesigner's: lag(x, 0.1) moves like a Lag CHOP set to 0.1, so the numbers you learn here carry over. That is about the time the output takes to cover 90% of a jump (underneath, a one-pole lag whose time constant is the seconds divided by 2.3). The curve is the same at 30 and at 144 frames per second.
The first value it sees is taken as it is, so nothing glides in from 0. A boolean counts as 0 or 1: lag(mouseIsPressed, 0.2) is a soft switch.
Before you call it
Nothing to call or declare: write it in draw(). The first call loads @openav/chop from next to oav-p5.js, which takes a few frames.
Parameters
Name
Type
Description
value
Number | Boolean
The number to follow. A boolean counts as 0 or 1. Anything that is not a finite number (a string, NaN) goes straight through and the lag keeps what it had.
up
Number
Seconds to follow a rise: about how long the output takes to cover 90% of a jump upward. Negative numbers count as 0. optionaldefault 0.1
down
Number
Seconds to follow a fall. Left out, it is the same as up. optionaldefault up
name
String | Number
A name for this call's memory, written as the last argument. Optional. Without one, oav-p5 tells calls apart by their order in draw(); with one, the call is the same wherever it is made. optionalread from arguments
Returns
Number The smoothed value. It returns value as it is until @openav/chop has loaded (a few frames), and for an unnamed call outside draw().
Notes
Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
A sketch with its own variable or function called lag keeps it. OAVp5.chop.lag(…) is this function under another name.
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
const x = lag(mouseX, 0.5); // follows the mouse, half a second behind
circle(x, height / 2, 40);
}
A circle that chases the mouse: it never jumps, it glides, and takes about half a second to catch up.
function setup() {
createCanvas(400, 300);
noStroke();
}
function draw() {
background(20);
for (let i = 0; i < 6; i++) {
const seconds = 0.1 + i * 0.12;
const x = lag(mouseX, seconds, seconds, 'dot' + i); // a name per dot
fill(255, 211, 107);
circle(x, 40 + i * 44, 24);
}
}
Six dots follow the mouse, each a little slower than the one above. The names keep each dot's memory its own even if you later add or remove dots.
Related
spring()Follows value like a mass on a spring: it overshoots, bounces and settles (TouchDesigner's Spring CHOP).
env()A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.
micLevelHow loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
knob1The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
trail()The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).
scope()Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
OAVp5.chopThe CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
It is a CHOP's viewer: each label is one line in a small plot across the page from the 🎹, with the last 4 seconds of the value and its number now. Call it twice in a frame with two labels and you get two lines to compare, raw against smoothed. The scale is at least 0 to 1 and grows to hold whatever the values reach.
It lives on the page, not on the canvas: never in a recording or in 📷, and it never takes a click from the sketch. To draw the same line inside the work, use drawTrail().
Before you call it
Only inside draw(): anywhere else it just returns value. ?oav_scope=off hides it, and so does ?oav_hints=off, which is the one thing to know if it does not appear. See scope. The first call loads @openav/chop, which takes a few frames.
Parameters
Name
Type
Description
value
Number | Boolean
The number to watch. A boolean is drawn as 0 or 1. Anything that is not a finite number is returned as it is and not drawn.
label
String
The name of this line in the legend. Each label is one line, and a label longer than 9 characters is cut short there. Without one the line is numbered. optional
Returns
Number | Boolean Exactly value, so circle(x, y, scope(lag(micLevel, 0.03, 0.4), 'lag') * 300) works.
Notes
Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
OAVp5.chop.scope(…) is this function under another name. The lines keep coming from the last second: stop calling it for a second and its line leaves the plot.
The corner plot shows two lines: the square wave and what lag() makes of it, rising in 0.3 seconds and falling in 0.6.
Related
drawTrail()Draws a line through values on the canvas, so it is in the work and in a recording: the oldest at x, lo at the bottom of the box, hi at the top.
trail()The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).
lag()Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
scopeWhether scope() draws its oscilloscope in the corner of the page: on or off. hints=off turns it off too.
hintsWhether oav-p5 shows its explaining cards beside the 🎹 (blocked microphone, no Web MIDI, an error in your callback): on or off.
How fast value is changing, per second: positive while it rises, negative while it falls, 0 while it holds still (TouchDesigner's Slope CHOP).
Ask how hard something is moving rather than where it is: abs(slope(mouseX)) is the mouse speed in pixels per second, and slope(micLevel) jumps up at the start of a word. It is as noisy as its input, so smooth it when it flickers: lag(abs(slope(x)), 0.05, 0.3).
Before you call it
Nothing to call or declare: write it in draw(). The first call loads @openav/chop, which takes a few frames.
Parameters
Name
Type
Description
value
Number | Boolean
The number to watch. A boolean counts as 0 or 1. Anything that is not a finite number keeps the last slope.
name
String | Number
A name for this call's memory, written as the last argument. Optional. Without one, oav-p5 tells calls apart by their order in draw(); with one, the call is the same wherever it is made. optionalread from arguments
Returns
Number The change per second, in the units of value (pixels per second for a pixel position). 0 on the first call, and 0 until @openav/chop has loaded or for an unnamed call outside draw().
Notes
Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
OAVp5.chop.slope(…) is this function under another name.
Adds rate to a running total every second, so rotate(speed(TWO_PI)) turns once a second and the rate can change without the total ever jumping.
Use the total to turn or move something: rotate(speed(TWO_PI)) is one turn a second, x = speed(60) % width slides 60 pixels a second and wraps around. You pass the rate, not the position, so the rate can follow a knob or the microphone (speed(micLevel * TWO_PI) spins faster the louder it gets) and the angle never jumps, which frameCount * rate would do.
Time is in seconds, the same at any frame rate. One frame counts for at most a quarter of a second, so coming back to a hidden tab does not make the shape leap.
Before you call it
Nothing to call or declare: write it in draw(). The first call loads @openav/chop, which takes a few frames.
Parameters
Name
Type
Description
rate
Number | Boolean
How much to add per second, in the unit you use the result in (radians for rotate(), pixels for a position). It may change every frame. A boolean counts as 0 or 1.
name
String | Number
A name for this call's memory, written as the last argument. Optional. Without one, oav-p5 tells calls apart by their order in draw(); with one, the call is the same wherever it is made. optionalread from arguments
Returns
Number The running total, 0 at the start. It is 0 until @openav/chop has loaded (a few frames), and for an unnamed call outside draw().
Notes
Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
Plenty of sketches have a variable called speed. If yours does, it keeps the name, and OAVp5.chop.speed(…) is this function under another name (third example).
let speed = 2; // the sketch's own variable called speedfunction setup() {
createCanvas(400, 300);
rectMode(CENTER);
}
function draw() {
background(20);
translate(width / 2, height / 2);
rotate(OAVp5.chop.speed(speed)); // the same function, under another name
rect(0, 0, 120, 120);
}
The sketch keeps its own speed (2 radians a second) and reaches the function as OAVp5.chop.speed.
Related
lag()Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
spring()Follows value like a mass on a spring: it overshoots, bounces and settles (TouchDesigner's Spring CHOP).
slope()How fast value is changing, per second: positive while it rises, negative while it falls, 0 while it holds still (TouchDesigner's Slope CHOP).
OAVp5.chopThe CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
Follows value like a mass on a spring: it overshoots, bounces and settles (TouchDesigner's Spring CHOP).
Wrap a target that jumps: circle(width / 2, height / 2, spring(mouseIsPressed ? 300 : 100, 100, 6)). The size does not just glide to the new target (that is lag()): it swings past it, comes back and settles.
Stiffness and damping are the two knobs. A stiff, lightly damped spring rings like a ruler on a table. A soft, heavily damped one just eases. A big jump or a low frame rate does not break it: the motion is the same at any frame rate.
Before you call it
Nothing to call or declare: write it in draw(). The first call loads @openav/chop, which takes a few frames.
Parameters
Name
Type
Description
value
Number | Boolean
The target the spring is pulled toward. A boolean counts as 0 or 1.
stiffness
Number
How fast it swings: 100 swings about 1.6 times a second. Higher is snappier. Negative numbers count as 0. optionaldefault 100
damping
Number
How soon it settles: 6 gives two or three bounces, 20 and up none. The edge where it stops bouncing is 2 * sqrt(stiffness). optionaldefault 6
name
String | Number
A name for this call's memory, written as the last argument. Optional. Without one, oav-p5 tells calls apart by their order in draw(); with one, the call is the same wherever it is made. optionalread from arguments
Returns
Number The spring's position now. The first value it sees is taken as it is. It returns value as it is until @openav/chop has loaded (a few frames), and for an unnamed call outside draw().
Notes
Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
OAVp5.chop.spring(…) is this function under another name.
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
const x = spring(mouseX, 60, 4); // soft and bouncyconst y = spring(mouseY, 60, 4);
circle(x, y, 40);
}
A circle hung on the mouse by a soft spring: it swings around the pointer instead of sticking to it.
function setup() {
createCanvas(400, 300);
noStroke();
}
function draw() {
background(20);
const target = mouseIsPressed ? 340 : 60;
const dampings = [2, 6, 20];
for (let i = 0; i < 3; i++) {
const x = spring(target, 100, dampings[i]); // same pull, three dampings
circle(x, 60 + i * 90, 36);
}
}
Hold the mouse: three circles are pulled right by the same spring with different damping. The top one bounces many times, the middle one a few, the bottom one not at all.
Related
lag()Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
speed()Adds rate to a running total every second, so rotate(speed(TWO_PI)) turns once a second and the rate can change without the total ever jumping.
slope()How fast value is changing, per second: positive while it rises, negative while it falls, 0 while it holds still (TouchDesigner's Slope CHOP).
OAVp5.chopThe CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).
A history to draw or measure: drawTrail(trail(micLevel, 3), 20, 20, width - 40, 100) plots the last three seconds of the microphone level, and Math.max(...trail(x, 2)) is a two-second peak. The values are sampled 60 times a second whatever the frame rate, and the gaps between two frames are interpolated, so 30 fps and 144 fps give the same shape.
Change seconds between frames and the history is resized, keeping the newest values. The array you got last frame is never changed afterwards.
Before you call it
Nothing to call or declare: write it in draw(). The first call loads @openav/chop, which takes a few frames.
Parameters
Name
Type
Description
value
Number | Boolean
The number to record. A boolean counts as 0 or 1. Anything that is not a finite number is skipped.
seconds
Number
How many seconds of history to keep, from 0.05 to 600. optionaldefault 2
name
String | Number
A name for this call's memory, written as the last argument. Optional. Without one, oav-p5 tells calls apart by their order in draw(); with one, the call is the same wherever it is made. optionalread from arguments
Returns
Array<Number> About seconds * 60 + 1 numbers (121 for 2 seconds), a new array every call. The length is fixed from the first call and filled with the first value, so you can draw it from frame one. It is an empty array until @openav/chop has loaded, and for an unnamed call outside draw().
Notes
Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
OAVp5.chop.trail(…) is this function under another name. To see the line on the canvas use drawTrail(), to see it in the corner of the page use scope().
function setup() {
createCanvas(400, 300);
noStroke();
}
function draw() {
background(20);
const t = trail(mouseX, 1); // where the mouse was during the last secondfor (let i = 0; i < t.length; i += 6) {
fill(255, 211, 107, 10 + (i / t.length) * 200);
circle(t[i], mouseY, 24);
}
}
A ghost trail behind the pointer: the older the copy, the fainter it is.
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
const t = trail(mouseX, 3); // the last 3 seconds of the mouse positionconst left = Math.min(...t), right = Math.max(...t);
noStroke();
fill(255, 211, 107, 90);
rect(left, 0, right - left, height); // the band the mouse covered
stroke(255);
drawTrail(t, 0, 20, width, 60, 0, width);
}
A band shows how far left and right the mouse went in the last three seconds, with the trail itself drawn across the top.
Related
drawTrail()Draws a line through values on the canvas, so it is in the work and in a recording: the oldest at x, lo at the bottom of the box, hi at the top.
scope()Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
lag()Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
slope()How fast value is changing, per second: positive while it rises, negative while it falls, 0 while it holds still (TouchDesigner's Slope CHOP).
OAVp5.chopThe CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
Fires an event of your own, by name, for env() and count() to react to.
Call it where the thing happens: function mousePressed() { trigger('boom'); }. Then read it anywhere in draw() with env('boom', 0.01, 0.6) (a pulse that fades) or count('boom') (how many times so far). It returns nothing.
It works anywhere, not only in draw(), and each call is one event, remembered with the time it happened. You can fire the built-in names by hand as well: trigger('kick') pulses env('kick') as a real kick would.
Parameters
Name
Type
Description
name
String
The name of the event. Anything is fine except the built-in meanings: 'kick', 'snare', 'hat', 'beat', 'note', 'bar' and 'step' are also fired by the microphone, the keys and the drum machine. Left out: 'trigger'. optionaldefault 'trigger'
Notes
OAVp5.chop.trigger(…) is this function under another name.
function setup() {
createCanvas(400, 300);
}
function mousePressed() {
trigger('boom'); // an event of our own
}
function draw() {
background(20);
const k = env('boom', 0.01, 0.6);
circle(width / 2, height / 2, 60 + k * 220);
}
Click the canvas: each click fires "boom" and the circle pulses once.
The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
Every function of this group is also here: OAVp5.chop.lag, .env, .trigger, .speed, .spring, .trail, .slope, .count, .scope and .drawTrail are the same functions as the plain names, for a sketch whose own speed or count keeps the plain name.
The rest is for looking inside: state says whether the module is loaded, module is the module itself, slots() lists every memory in use and reset() forgets them.
Notes
This is advanced: a sketch that only calls lag() and friends never needs it.
The @openav/chop module itself once it has loaded, otherwise null: the classes and functions the CHOP functions are built on.
For reaching below the p5-style calls: OAVp5.chop.module.DEFAULTS lists the defaults the functions use, and new OAVp5.chop.module.Lag({ up: 0.1 }) makes a lag you keep yourself. Check it is not null first.
Type
Object | null
null until the first CHOP function has been called and the module has arrived, then the module (Lag, Spring, Trail, envelope(), DEFAULTS and the rest).
Notes
This is advanced. The module is also published as @openav/chop for sketches and shows that do not go through oav-p5.
Forgets the memory named name (a lag, a speed total, a count …), or every memory when you give no name: the next call starts fresh.
The call that follows starts from nothing, as if it were the first one: a speed() total is 0 again, a lag() takes its input as it is, a count() is back at 0. Use it to restart a named effect, for example on a key press.
Calling it with no name also drops the unnamed memories. They are made again by order on the next frame, so nothing breaks, but every unnamed value starts over too. Event counts from trigger() are not memories of a call and are not touched.
Parameters
Name
Type
Description
name
String
The name you gave a call as its last argument ('spin' in speed(1, 'spin')). The memory of that name is forgotten in every function that has one. Left out, every memory is forgotten, named or not. optional
Lists every memory the CHOP functions are using right now: its kind, how it is found, and its latest value.
For finding out why two calls seem to share a memory, or whether a loop made hundreds: OAVp5.chop.slots().length is how many there are. It reads the state and changes nothing.
Returns
Array<Object> One { kind, key, mode, value } per memory. kind is the function ('lag', 'spring', …). key is '#1', '#2' for calls found by order, a name for named ones, or the line of code for calls matched by line. mode says which: 'order', 'name' or 'site'. value is its latest output (missing for a trail).
Notes
This is advanced. 🩺 and tests use it; a sketch rarely needs to.
Where the @openav/chop module is: 'off', 'loading', 'on' or 'error'.
off until one of these functions is first called, loading while @openav/chop arrives (a few frames), on once it is in, error if it could not be loaded: a card says so and every function then passes its input through. trigger() on its own never loads it, because it only records that something happened.
Type
String
Range
'off' | 'loading' | 'on' | 'error'
Changes the first time a CHOP function is called and once more when the module has arrived. off before any call.
One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.
The smallest piece of smoothing. Call it once per frame, value = approach(value, target, 0.2, dt), and value chases target: 63% of the way after seconds, 90% after 2.3 × seconds, 99% after 4.6 × seconds. It is exact for any dt, so a sketch at 30 fps and one at 144 fps draw the same curve.
seconds here is a time constant, which is shorter than the "seconds" of TouchDesigner's Lag CHOP. Lag takes the TouchDesigner kind and divides it by TD_LAG_RATIO before calling this function. @openav/mapping's smooth steps with approach and means the time constant too.
Before you call it
Nothing. It is pure arithmetic with no memory: you keep current yourself and pass in the seconds since the last call. For a value with its own memory use Lag.
Parameters
Name
Type
Description
current
Number
Where the value is now.
target
Number
Where it is heading.
seconds
Number
The time constant: after this many seconds the gap to target has closed by 63% (1 − 1/e). 0, a negative number or NaN jumps straight to target, whatever dt is.
dt
Number
Seconds since the last step. 0, a negative number or NaN leaves current where it is.
Returns
Number The new value, between current and target.
Examples
js
import { approach } from'@openav/chop';
let value = 0;
const seen = [];
for (let frame = 0; frame < 6; frame++) {
value = approach(value, 1, 0.1, 1 / 60); // chase 1, time constant 0.1 s
seen.push(value.toFixed(3));
}
document.getElementById('app').textContent = seen.join(' ');
module Six frames of a value chasing 1: it covers about 15% of the remaining gap each frame (0.154, 0.283, 0.393 …).
js
import { approach } from'@openav/chop';
// one second of time constant: the gap is 63% closed after 1 s, 90% after 2.3 sconst seconds = 1;
const after = (t) => approach(0, 1, seconds, t).toFixed(3);
document.getElementById('app').textContent =
`after 1 s: ${after(1)} after 2.3 s: ${after(2.3026)} after 4.6 s: ${after(4.6052)}`;
module Prints 0.632, 0.900 and 0.990: the three numbers worth remembering.
Related
LagFollows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.
TD_LAG_RATIOMath.LN10, about 2.303: how many time constants a lag needs to cover 90% of a jump, which is what TouchDesigner's Lag seconds mean.
lag()Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
Counts how many times its input turned on (true, or a number above threshold); with a limit it loops 0 … limit − 1. The Count CHOP.
count.step(input, limit) counts a rising edge, the moment the input goes from off to on, not the time it stays on. Hold a button for ten seconds and that is one. With limit 4 the result walks 0 1 2 3 0 1 …, which is how a beat becomes "beat 1 of 4".
The counter keeps counting underneath even when you pass a limit, so you can change the limit later without losing your place. A value that is not a number counts as off.
In a p5 sketch use count(): count(beat, 4), or count('kick') to count the drum machine's kicks.
Before you call it
Nothing to load. Call step every frame, even while the input is off: it has to see the "off" to notice the next "on". It has no dt, because a count does not depend on time.
Parameters
Name
Type
Description
threshold
Number
A number counts as "on" when it is above this. A boolean true is 1, false is 0. optionaldefault 0.5
The default settings of every CHOP in the package, as one frozen object: lag, envelope, spring, trail and count.
For documentation and for interfaces that show the defaults, such as the placeholder text of a field. The classes and envelope() have these same numbers written in their own signatures; changing this object changes nothing about how they behave.
The level, 0 to peak, of an attack-and-decay envelope at time t, for events that happened at the given times; the Trigger CHOP.
Keep a list of the times something happened (a kick, a key press, a cue) and ask for the level at the time you are drawing: scale(1 + envelope(kicks, now, 0.01, 0.3)). It rises in a straight line to 1 in attack seconds, then falls to 0 in decay seconds, quick at first and landing softly.
It is a pure function of the times and of t: ask twice, or at 30 fps and at 144 fps, and the same moment gives the same level; attack and decay may change every frame (a knob on the decay). A new event while the last one still sounds restarts from where the envelope is, never from 0, so fast hits pile up into a plateau instead of flickering. The attack counts from 0 to the peak, so a retrigger from halfway up takes half the attack.
In a p5 sketch you do not import it: env() and trigger() keep the event list for you and read the drum machine and the microphone.
Before you call it
Nothing to load. You record the event times yourself (for example kicks.push(performance.now() / 1000) when a kick arrives) and pass the same clock as t.
Parameters
Name
Type
Description
times
Number[]
When the events happened, in seconds on any clock you like, oldest first. An empty array (or none) gives 0. Events later than t have not happened yet and are ignored. Only the last few matter, so you can trim the list.
t
Number
The moment to read, on the same clock as times.
attack
Number
Seconds to rise from 0 to the peak. 0 starts at the peak at once. optionaldefault 0.01
decay
Number
Seconds to fall back to 0 after the peak: it is exactly 0 from attack + decay seconds after an event. optionaldefault 0.3
peak
Number
The top of the envelope. optionaldefault 1
Returns
Number 0 to peak: peak just after an event, 0 once it has died away.
Examples
js
import { envelope } from'@openav/chop';
const kicks = [1]; // one kick, at 1 sconst times = [0.9, 1, 1.005, 1.01, 1.1, 1.2, 1.31];
document.getElementById('app').innerHTML = times
.map((t) => `${t} s: ${envelope(kicks, t).toFixed(2)}`)
.join('<br>');
module Before the kick 0.00, half way up the attack 0.50 at 1.005 s, the peak 1.00 at 1.01 s, then 0.49, 0.13 and 0.00 at 1.31 s.
module A disc that swells on every kick (twice a second) and shrinks back within 0.3 s.
js
import { envelope } from'@openav/chop';
// three hits 50 ms apart pile up; one lone hit at the same moment is much lowerconst pile = envelope([0, 0.05, 0.1], 0.11);
const lone = envelope([0], 0.11);
document.getElementById('app').textContent =
`pile-up: ${pile.toFixed(2)} lone hit: ${lone.toFixed(2)}`;
module Prints "pile-up: 0.95 lone hit: 0.44": each new hit restarts from where the envelope is, so quick hits hold it high.
Related
envelopeStage()The level of one envelope elapsed seconds after its trigger: a straight rise to peak, then a fall to 0 in decay seconds.
env()A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.
LagFollows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.
CountCounts how many times its input turned on (true, or a number above threshold); with a limit it loops 0 … limit − 1. The Count CHOP.
envelopeStage(elapsed, attack, decay, from = 0, peak = 1)
The level of one envelope elapsed seconds after its trigger: a straight rise to peak, then a fall to 0 in decay seconds.
The building block of envelope(), which strings several of these together for a list of events. On its own it answers: "how high is a kick that happened elapsed seconds ago?" With the defaults of envelope() (attack 0.01, decay 0.3) it is 0.5 at 5 ms, 1 at 10 ms, 0.49 at 100 ms and 0 from 310 ms on.
The fall is quick at first and lands softly: that is the shape a kick drum swell looks right with. It reaches 0 in exactly decay seconds, not asymptotically.
Parameters
Name
Type
Description
elapsed
Number
Seconds since the trigger. Below 0 (or NaN) gives from.
attack
Number
Seconds to rise from 0 to peak. 0 or less starts at the peak at once.
decay
Number
Seconds to fall from peak to 0, along (1 − u)² where u runs 0 to 1. 0 or less drops to 0 right after the attack.
from
Number
The level at the moment of the trigger. The attack still counts from 0 to the peak, so starting from halfway up takes half the attack. optionaldefault 0
peak
Number
The top of the envelope. optionaldefault 1
Returns
Number The level, from 0 to peak.
Notes
Advanced. Most sketches never call it: use envelope(), or env() in p5.
new Lag({ up = 0.1, down = up, value = null } = {})
Follows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.
Make one Lag per value you want to smooth, then call step(input, dt) once per frame. new Lag({ up: 0.03, down: 0.4 }) is a VU meter: it opens almost at once and closes slowly. up and down are in TouchDesigner's seconds: with up: 1, a jump from 0 to 1 is at 0.9 one second later, at 30, 60 or 144 fps alike.
Each step is the exact answer for an input held at its new value over dt seconds, so the frame rate does not change the curve. A boolean counts as 0 or 1 (lag.step(mouseIsPressed, dt) is a soft switch). A value that is not a number, such as NaN, comes back untouched and leaves the memory as it was.
In a p5 sketch you do not import this: lag() keeps one Lag per call for you.
Before you call it
Nothing to load or start. Pass the seconds since the previous frame as dt (in a browser: the difference of two performance.now() / 1000). The first step only stores the input, so the output equals the input on frame one.
Parameters
Name
Type
Description
up
Number
Seconds to follow a rise, while the input is above the output. TouchDesigner's seconds: about the time to cover 90% of a jump. 0 follows at once. optionaldefault 0.1
down
Number
Seconds to follow a fall, while the input is below the output. Leave it out and it is the same as up. optionaldefault up
value
Number
A starting value. Leave it out and the first input is taken as it is, so nothing glides in from 0. optionaldefault null
step(input, dt, up?, down?): advance by dt seconds toward input and return the new output. up and down override the constructor's for this call only, so a knob can change them every frame. A non-number input is returned as it is.
reset(value?): forget the memory. With no argument the next input is taken as it is again; with a number the output restarts from there. Returns the Lag, so calls chain.
Notes
Public fields: up, down and value, the last output (null until the first step). You can change up and down any time.
The time constant underneath is seconds / TD_LAG_RATIO (see TD_LAG_RATIO), not seconds. If you want the plain 63% kind, call approach().
Examples
js
import { Lag } from'@openav/chop';
const lag = new Lag({ up: 0.03, down: 0.4 }); // opens fast, closes slowlyconst seen = [];
for (let frame = 0; frame < 12; frame++) {
const input = frame >= 2 && frame < 6 ? 1 : 0; // on for four frames
seen.push(lag.step(input, 1 / 60).toFixed(2));
}
document.getElementById('app').textContent = seen.join(' ');
module The output shoots up within two frames when the input turns on, then sags back slowly after it turns off (0.00 0.00 0.72 0.92 0.98 0.99 0.90 0.82 …).
js
import { Lag } from'@openav/chop';
const ctx = document.getElementById('c').getContext('2d');
ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
ctx.lineWidth = 2;
const input = (t) => (t < 0.25 || t > 1.25 ? 0 : 1); // a pulse, 1 s longfor (const [fps, color] of [[30, '#555'], [30, '#fa4'], [144, '#4af']]) {
const lag = new Lag({ up: 0.1, down: 0.4 });
ctx.strokeStyle = color; ctx.beginPath();
for (let k = 0; k <= fps * 2; k++) {
const t = k / fps;
const y = color === '#555' ? input(t) : lag.step(input(t), k ? 1 / fps : 0);
ctx.lineTo(20 + t * 180, 260 - y * 220);
}
ctx.stroke();
}
module A one-second pulse (grey) through a Lag at 30 fps (orange) and at 144 fps (blue): it rises in about 0.1 s, falls over about 0.4 s, and the two curves lie on top of each other.
js
import { Lag } from'@openav/chop';
const lag = new Lag(); // up and down are both 0.1
lag.step(0.5, 0); // the first input is taken as it isconst nan = lag.step(NaN, 1 / 60); // not a number: passed through, memory keptconst kept = lag.value; // still 0.5const toward = lag.step(true, 1 / 60); // a boolean counts as 1, so it moves up from 0.5
document.getElementById('app').textContent =
`NaN in: ${nan}, memory still ${kept}, true in: ${toward.toFixed(2)}`;
module Prints "NaN in: NaN, memory still 0.5, true in: 0.66": a non-number leaves the memory alone, and a boolean counts as 0 or 1.
Related
approach()One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.
TD_LAG_RATIOMath.LN10, about 2.303: how many time constants a lag needs to cover 90% of a jump, which is what TouchDesigner's Lag seconds mean.
lag()Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
SpringFollows its input like a mass on a spring: it overshoots, swings back and settles; the Spring CHOP with the input as position.
envelope()The level, 0 to peak, of an attack-and-decay envelope at time t, for events that happened at the given times; the Trigger CHOP.
How fast the input is changing, in units per second; 0 on the first step. The Slope CHOP.
slope.step(value, dt) returns (value − previous value) / dt. A value rising by 3 every second gives 3 at any frame rate; a value that stands still gives 0; a falling one gives a negative number. Use it for "how hard did the knob just move": scale(1 + abs(slope.step(knob, dt))).
It has no smoothing: a single jump of the input is one big spike for one step. Put the result through a Lag if you want it to fade. A dt of 0 or less changes nothing and returns the last slope, and a value that is not a number returns the last slope too.
Adds rate every second to a running total: a rate in units per second becomes a position; the Speed CHOP.
speed.step(rate, dt) adds rate * dt to the total and returns it. With rate 1 the total grows by 1 every second; rotate(speed.step(TWO_PI, dt)) is one turn a second. Feed it a rate that changes (a knob, micLevel) and the motion speeds up and slows down smoothly, without the jump you get from multiplying frameCount by it.
It is the integral of the rate, so it is the same at any frame rate. A boolean rate counts as 1 per second while true. A rate that is not a number, or a dt of 0 or less, adds nothing.
In a p5 sketch use speed(), which keeps one for each call.
Before you call it
Nothing to load. Pass the seconds since the previous frame as dt.
Parameters
Name
Type
Description
value
Number
The starting total. Anything that is not a finite number starts at 0. optionaldefault 0
reset(value?): set the total back to value (0 if left out). Returns the Speed.
Notes
The total is the public field value.
Examples
js
import { Speed } from'@openav/chop';
const speed = new Speed();
const seen = [];
for (let i = 0; i < 4; i++) seen.push(speed.step(2, 0.5).toFixed(1)); // 2 units a second
document.getElementById('app').textContent = seen.join(' ');
module Prints 1.0 2.0 3.0 4.0: two units a second, half a second per step.
js
import { Speed } from'@openav/chop';
const ctx = document.getElementById('c').getContext('2d');
const spin = new Speed();
let last = performance.now();
function frame(now) {
const dt = (now - last) / 1000; last = now;
const angle = spin.step(Math.PI * 2, dt); // one turn a second
ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
ctx.strokeStyle = '#fc4'; ctx.lineWidth = 6;
ctx.beginPath(); ctx.moveTo(200, 150);
ctx.lineTo(200 + Math.cos(angle) * 90, 150 + Math.sin(angle) * 90); ctx.stroke();
requestAnimationFrame(frame);
}
requestAnimationFrame(frame);
module A hand that turns once a second, at any frame rate. Change the rate to a changing number (a level, a knob) and it speeds up and slows down smoothly.
Related
speed()Adds rate to a running total every second, so rotate(speed(TWO_PI)) turns once a second and the rate can change without the total ever jumping.
SlopeHow fast the input is changing, in units per second; 0 on the first step. The Slope CHOP.
LagFollows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.
new Spring({ stiffness = 100, damping = 6, value = null } = {})
Follows its input like a mass on a spring: it overshoots, swings back and settles; the Spring CHOP with the input as position.
Make one Spring per value, call step(target, dt) once per frame and draw what it returns. With the defaults (stiffness 100, damping 6), a jump of the input from 0 to 1 sends the output to 1.37 after a third of a second, and it is within 1% of 1 from about 1.4 s on. Raise the damping to 20 and it glides to the target without overshooting.
Each step is the exact answer from springStep(), so 30 fps and 144 fps give the same motion and a long frame cannot blow it up. The input is the position it is pulled toward; a boolean counts as 0 or 1; a value that is not a number is returned as it is and changes nothing.
Nothing to load. Pass the seconds since the previous frame as dt. The first step only stores the input (the spring starts at rest on it).
Parameters
Name
Type
Description
stiffness
Number
How hard the spring pulls, so how fast it swings. With nothing damping it, it swings about √stiffness / 2π times a second (100 is about 1.6). optionaldefault 100
damping
Number
How soon it settles. 6 with a stiffness of 100 bounces a few times; 2 * √stiffness (20 for 100) and above never bounces. optionaldefault 6
value
Number
A starting position. Leave it out and the first input is taken as it is, so it does not swing in from 0. optionaldefault null
step(target, dt, stiffness?, damping?): advance the spring by dt seconds toward target and return the new position. stiffness and damping override the constructor's for this call.
One exact step of a damped spring of mass 1 pulled toward target; returns [position, velocity].
Solves x'' = −stiffness·(x − target) − damping·x' exactly over dt for a target that stays still during the step. It handles all three cases (under-damped: it bounces; critically damped; over-damped: it creeps) and a huge dt never blows up, which is why a sketch can fall behind for a whole second and the spring just lands on the target.
damping = 2 * Math.sqrt(stiffness) is the edge where it stops bouncing: with stiffness 100 that is a damping of 20. With no spring (stiffness 0) only the damping acts and the velocity just fades.
Parameters
Name
Type
Description
x
Number
The position now.
v
Number
The velocity now, in units per second.
target
Number
Where the other end of the spring is held.
stiffness
Number
How hard it pulls. Negative counts as 0.
damping
Number
How much friction slows it. Negative counts as 0.
dt
Number
Seconds to advance. 0 or less (or NaN) returns [x, v] unchanged.
Returns
[Number, Number] The new position and velocity.
Notes
Advanced. The Spring class holds the position and the velocity for you and calls this.
Examples
js
import { springStep } from'@openav/chop';
let x = 0, v = 0;
const seen = [];
for (let frame = 0; frame < 8; frame++) {
[x, v] = springStep(x, v, 1, 100, 6, 1 / 60); // pulled toward 1
seen.push(x.toFixed(2));
}
document.getElementById('app').textContent = seen.join(' ');
module Eight frames of a spring released at 0 and pulled toward 1: it accelerates (0.01, 0.05, 0.11 …).
js
import { springStep } from'@openav/chop';
const [x, v] = springStep(0, 0, 1, 100, 6, 10); // a ten-second step in one go
document.getElementById('app').textContent = `after 10 s: position ${x.toFixed(3)}, velocity ${v.toFixed(3)}`;
module A ten-second step lands on the target (position 1.000, velocity 0.000) instead of exploding.
Related
SpringFollows its input like a mass on a spring: it overshoots, swings back and settles; the Spring CHOP with the input as position.
spring()Follows value like a mass on a spring: it overshoots, bounces and settles (TouchDesigner's Spring CHOP).
approach()One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.
Math.LN10, about 2.303: how many time constants a lag needs to cover 90% of a jump, which is what TouchDesigner's Lag seconds mean.
TouchDesigner's Lag CHOP quotes its seconds as roughly the time the output takes to follow 90% of a change. A one-pole lag reaches 90% after ln 10 time constants, so seconds / TD_LAG_RATIO is the time constant to hand to approach(). Lag does exactly that, which is why lag(x, 0.1) in a p5 sketch moves like a Lag CHOP set to 0.1.
You only need it when you mix the two kinds of seconds: a time constant tau is tdSeconds / TD_LAG_RATIO, and tdSeconds is tau * TD_LAG_RATIO. Attack and decay of an envelope() are plain lengths and are not converted.
Type
Number
Exactly Math.LN10, 2.302585092994046. A fixed number: it never changes.
Examples
js
import { approach, TD_LAG_RATIO } from'@openav/chop';
const tdSeconds = 0.5; // as you would type it into a TouchDesigner Lag CHOPconst tau = tdSeconds / TD_LAG_RATIO; // the time constant approach() wantsconst reached = approach(0, 1, tau, tdSeconds);
document.getElementById('app').textContent =
`tau = ${tau.toFixed(3)} s, after ${tdSeconds} s the value is ${reached.toFixed(2)}`;
module Prints "tau = 0.217 s, after 0.5 s the value is 0.90": half a second of Lag CHOP seconds is 90% of the way.
Related
approach()One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.
LagFollows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.
lag()Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
Remembers the input over the last few seconds as a list of values, oldest first, at a fixed 60 values a second; the Trail CHOP.
trail.step(value, dt) returns an array of seconds * rate past values followed by this step's input at the end, so the array is seconds * rate + 1 long (121 for the defaults). Draw it across the canvas as a line: that is a scope of the value.
The length is fixed from the first step, filled with the first value, so a sketch can draw the whole line from frame one. The values are interpolated between steps, so 30 fps and 144 fps give the same line. A gap longer than the whole window (a tab that was hidden) restarts it filled with the new value.
Nothing to load. Pass the seconds since the previous frame as dt. A value that is not a number leaves the history as it is and returns the current list.
Parameters
Name
Type
Description
seconds
Number
How many seconds of history to keep. 0 or less falls back to 2. optionaldefault 2
rate
Number
Values per second in the list. 0 or less falls back to 60. optionaldefault 60
module A scope of a sine wave that scrolls across the canvas: the newest value is at the right edge, with three seconds of history behind it (flat on the left until that much has passed).
Related
trail()The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).
drawTrail()Draws a line through values on the canvas, so it is in the work and in a recording: the oldest at x, lo at the bottom of the box, hi at the top.
scope()Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
SlopeHow fast the input is changing, in units per second; 0 on the first step. The Slope CHOP.
Loops a count to 0 … limit − 1: wrapCount(5, 4) is 1. A limit of 0 or less leaves the count as it is.
The modulo that Count uses, made safe for negative numbers: wrapCount(-1, 4) is 3, where -1 % 4 is -1. It is useful on its own for anything that loops, such as a step number: wrapCount(frameCount, 16).
Parameters
Name
Type
Description
total
Number
The count so far.
limit
Number
How many values to loop through. A fraction is rounded down. 0, a negative number, or no value at all returns total unchanged.
Returns
Numbertotal looped into 0 … limit − 1 (never negative, even for a negative total).
Examples
js
import { wrapCount } from'@openav/chop';
const row = [0, 3, 4, 5, 9, -1].map((n) => wrapCount(n, 4));
document.getElementById('app').textContent =
`0 3 4 5 9 -1 in a loop of 4: ${row.join(' ')} no limit: ${wrapCount(5, 0)}`;
module Prints 0 3 0 1 1 3, then 5 when there is no limit.
Related
CountCounts how many times its input turned on (true, or a number above threshold); with a limit it loops 0 … limit − 1. The Count CHOP.
count()Counts how often on has turned on, or how many 'kick' (or any other event) have happened; with a limit it loops 0, 1, … limit − 1.
Use it to keep things out of the video: if (!recordingIsOn) text('press r to record', 20, 30);, because everything on the canvas is recorded. Use it for a key that toggles: recordingIsOn ? stopRecording() : startRecording(). It changes when a take starts and when it ends, and reading it never starts anything. For the other stages (starting, saving) read OAVp5.record.state.
Type
Boolean
True while a take runs. It is false at the start, while the recorder is still loading, from the moment stopRecording() is called (the file is then being made) and after the take.
Notes
A sketch that declares its own variable called recordingIsOn keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.record.state === 'on' instead.
An event you define: it is called when a recording has been saved and checked, with an info object about the file.
Define function recordingSaved(info) { … } to know when the file is done: stopRecording() or the end of seconds leads to it. By then the file has already been handed to the browser to download. The same info is given to the promise of stopRecording() and to OAVp5.record.on('saved', fn), and stays in OAVp5.record.last.
info.url works as a link to the file (createA(info.url, 'play') or a <video>) until a minute after the next take is saved. warnings is what the card after the take says in words; an empty list means nothing looked wrong.
Parameters
Name
Type
Description
info
Object
name (the file name), format ('mp4' or 'webm'), type (the exact MIME type), seconds (length, to a tenth), bytes, width and height (even numbers for MP4), fps, audio (the video has a sound track), heard (something sounded), sources (which sounds were in it: 'sound', 'drums', 'mic', 'p5sound'), warnings (what looked wrong: 'recEmpty', 'recWebm', 'recNoAudio', 'recSilent', 'recBlack', usually an empty list), blob (the file) and url (an address for the blob).
let note = '';
function setup() {
createCanvas(400, 300);
startDrums('breakbeat', 120);
}
function keyPressed() {
if (key === 'r') startRecording(4);
}
function recordingSaved(info) {
note = info.heard ? 'with sound: ' + info.sources.join(', ') : 'silent: click the page before you record';
if (info.warnings.length) note += ' (' + info.warnings.join(', ') + ')';
}
function draw() {
background(20);
fill(255);
if (!recordingIsOn) text(note || 'click once, then press r', 20, 30);
}
Press r (after one click on the page): the text says whether the video has sound, from which sources, and lists anything that looked wrong.
Related
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
stopRecording()Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.
OAVp5.record.lastThe info of the last recording that was saved (the same object recordingSaved got), or null before the first one.
OAVp5.record.on()Listens to the recording from anywhere: OAVp5.record.on('saved', fn) or on('start', fn), and it returns a function that stops listening.
recordingIsOnWhether a recording is running right now.
Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
Start it from a key or a click: if (key === 'r') startRecording();. What goes into the video is the canvas alone, at its own pixel size (a retina screen doubles it), and everything your sketch sounds like: the instrument of startSound(), the drum machine, the pads, the microphone while it is on, and p5.sound 1.x, mixed into one track while the speakers play as before. The 🎹, the drum grid and the cards are part of the page, not of the canvas, so they are never in the video. A sound that comes on during the take joins it.
The file is MP4 (H.264 + AAC, which Instagram, YouTube and every editor take) where the browser can write it, and WebM where not; a card says Instagram will not take a WebM. It is named oav-20261015-183005.mp4, with name in the middle when you give one. When the take is done it is downloaded, and on a phone that can share files a Share button appears on the card. After stopRecording(), or when seconds has run out, recordingSaved(info) is called with the details. Without seconds a take stops by itself at the longest length (300 seconds unless the reclimit option says otherwise).
Without bitrate the quality follows the canvas: 0.16 bits per pixel per frame, kept between 4 and 16 megabits a second (24 above 30 frames a second). Calling startRecording() while a take is running starts nothing new. After every take a card says what was recorded and, when something looks wrong, what and why: nothing reached the sound track although a sound was on, WebM, a file that is almost empty, a black video of a canvas that was not black.
Before you call it
A browser that has MediaRecorder: where it is missing, a card says so and the promise resolves to false. The picture is recorded at once, but the page can only make sound after a click or key press on it, so a take that begins before the first one has nothing to hear at first. A canvas that shows an image from another site (no CORS) cannot be recorded, and a card says so.
Parameters
Name
Type
Description
o
Number | Object
A number is the length in seconds: startRecording(10) stops by itself after ten. An object can hold seconds (the same, never more than the longest take, 300 by default), fps (frames a second, at most 60, 30 by default), mic (false keeps the microphone out of the video), name (a word for the file name), canvas (the canvas to record: a <canvas>, a p5 canvas or a createGraphics() buffer; the biggest p5 canvas on the page by default) and bitrate (bits a second). Leave it out to record until stopRecording(). optional
Returns
Promise<Boolean> Resolves to true once the recording is running (or when one already is). Resolves to false when the loading option record is off, when there is no canvas to record, when this browser has no MediaRecorder, or when it could not start; a card says why.
Notes
In the 🎹 panel, ⏺ starts and stops a take with no code, and 📷 saves the canvas as a PNG (see OAVp5.record.shot()). While a take runs the 🎹 button shows a red dot and the time. The loading options record (on, nomic or off), recfps, reclimit and title set the defaults.
The microphone is in the take while it is on: wear headphones, or the speakers are recorded twice. p5.sound for p5 2.x connects each sound straight to the speakers, so it cannot be recorded (a card says so); use p5 1.11, or startSound() and startDrums(). Sound that your sketch makes with its own Tone.js or Web Audio code is not recorded either.
Whatever you draw on the canvas is in the video, so show hints only while recordingIsOn is false.
function setup() {
createCanvas(400, 300);
}
function keyPressed() {
if (key === 'r') startRecording(5); // five seconds, then it stops and downloads
}
function draw() {
background(20);
circle(200 + sin(frameCount * 0.05) * 120, 150, 50);
if (!recordingIsOn) {
fill(255);
text('press r to record 5 seconds', 20, 30); // not drawn while recording
}
}
Press r: after five seconds an MP4 of the moving circle is downloaded, without the hint text.
function setup() {
createCanvas(400, 300);
startDrums('four on floor', 110);
startSound('synth/pluck');
}
function keyPressed() {
if (key === 'r') {
if (recordingIsOn) stopRecording();
else startRecording({ name: 'my piece', fps: 60 });
}
if (key === 'a') midi.play(60, 0.8);
}
function kickDetected(level) {
background(255 * level);
}
function draw() {
background(20);
if (!recordingIsOn) {
fill(255);
text('click once, then: a plays a note, r starts and stops a recording', 20, 30);
}
}
Click once so the browser allows sound, then press r to start and r again to stop: the file holds the picture, the drums and the notes you played.
let layer;
function setup() {
createCanvas(400, 300);
layer = createGraphics(200, 200);
}
function keyPressed() {
if (key === 'g') startRecording({ canvas: layer, seconds: 3 }); // record the buffer, not the main canvas
}
function draw() {
background(20);
layer.background(40);
layer.circle(100 + sin(frameCount * 0.1) * 60, 100, 40);
image(layer, 100, 50);
fill(255);
text('press g to record the small square only (3 s)', 20, 30);
}
Press g: the video is 200 by 200, the buffer alone, not the 400 by 300 canvas.
Related
stopRecording()Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.
recordingIsOnWhether a recording is running right now.
recordingSaved()An event you define: it is called when a recording has been saved and checked, with an info object about the file.
OAVp5.recordThe object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.
OAVp5.record.shot()Saves the canvas alone as a PNG picture (not the 🎹, not the cards) and downloads it.
startSound()Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
drawRhythm()Draws the rhythm on the canvas as a grid, a circle or a spiral, with a playhead and the hits lighting up, so a recording has it too.
Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.
It ends a take that has no seconds, or ends one early. recordingIsOn turns false at once, while the file is made (OAVp5.record.state is 'saving' until then), so the promise or recordingSaved() is how you know the file is ready. Both come after a fraction of a second, a little more for a long take.
Returns
Promise<Object | null> Resolves, once the file is saved and checked, to the same info that recordingSaved(info) gets. Resolves to null at once when nothing is recording, or when the recording was still starting (nothing was recorded then).
let message = 'press r to record, r again to stop';
function setup() {
createCanvas(400, 300);
}
function keyPressed() {
if (key !== 'r') return;
if (recordingIsOn) {
stopRecording().then((info) => {
if (info) message = 'saved ' + info.name; // the file is ready
});
} else {
startRecording();
}
}
function draw() {
background(20);
circle(200 + sin(frameCount * 0.05) * 120, 150, 50);
if (!recordingIsOn) {
fill(255);
text(message, 20, 30);
}
}
Press r to start and r to stop: once the file is ready the text names it.
Related
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
recordingIsOnWhether a recording is running right now.
recordingSaved()An event you define: it is called when a recording has been saved and checked, with an info object about the file.
OAVp5.record.stateWhere the recording is: 'off', 'starting', 'on' or 'saving'.
The object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.
OAVp5.record (also reachable as midi.record) is what startRecording(), stopRecording() and recordingIsOn are built on. It holds the state of the recording, how many seconds it has run, the last file's info, the sources in the take, and the fps and limit in force. formats() says what the browser can write, shot() saves a PNG of the canvas and on() listens to 'start' and 'saved'. start and stop are the same as the two global functions.
Notes
For a sketch the global names are enough. This object is for tools, the console and debugging.
Says what this browser can record: { recorder, mp4, webm, mime }.
Use it before you offer a recording button, or to tell the person which file they will get: MP4 goes to Instagram and every editor, WebM does not. It asks the browser each time you call it.
Returns
Objectrecorder is true when the browser has MediaRecorder at all, mp4 and webm are true when it can write that format, and mime is the MIME type a recording asks for first (MP4 where possible), or '' when none works.
The frame rate (fps) and the longest take in seconds (limit) that the loading options recfps and reclimit set.
They are there to read: OAVp5.record.limit tells a sketch how long a take can be. Writing to them changes nothing, because a take reads the options, not these two. To record at another frame rate once, pass startRecording({ fps: 60 }); to change the limit, set the reclimit option before oav-p5 loads (window.OAV_P5 = { reclimit: 600 }).
Type
Number
Range
fps 1..60, limit 1 …
fps is 30 and limit is 300 unless the options say otherwise (fps is kept between 1 and 60).
The info of the last recording that was saved (the same object recordingSaved got), or null before the first one.
Handy when you did not keep info in recordingSaved(): show the last file name, or put OAVp5.record.last.url in a link. The address stays valid until a minute after the next take is saved.
Type
Object | null
Replaced each time a take is saved. See recordingSaved() for its fields.
Listens to the recording from anywhere: OAVp5.record.on('saved', fn) or on('start', fn), and it returns a function that stops listening.
recordingSaved() is a single function that you define. on() is for the cases where that does not fit: several listeners, a library, a listener you want to remove again. It is the same idea as OAVp5.drums.on() and midi.on(). An error thrown inside fn is printed and the other listeners still run.
Parameters
Name
Type
Description
name
String
'start' (a take began) or 'saved' (a file is saved, like recordingSaved()). Any other name throws an error that lists the two.
fn
Function
For 'start' it gets { width, height, fps, type, seconds } (seconds is 0 when no length was asked for). For 'saved' it gets the same info as recordingSaved().
let status = 'press r to record 2 seconds';
function setup() {
createCanvas(400, 300);
OAVp5.record.on('start', (i) => { status = 'recording ' + i.width + 'x' + i.height + ' at ' + i.fps + ' fps'; });
OAVp5.record.on('saved', (i) => { status = 'saved ' + i.name; });
}
function keyPressed() {
if (key === 'r') startRecording(2);
}
function draw() {
background(20);
circle(200 + sin(frameCount * 0.05) * 120, 150, 50);
if (!recordingIsOn) {
fill(255);
text(status, 20, 30);
}
}
Press r: the text reports the start (size and frame rate) and then the saved file name, from two listeners.
Related
recordingSaved()An event you define: it is called when a recording has been saved and checked, with an info object about the file.
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
OAVp5.drums.on()Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
OAVp5.recordThe object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.
How long the running take has been going, in seconds, and the length of the last take after it.
Use it as a clock that starts with the take: const t = recordingIsOn ? OAVp5.record.seconds : 0; makes an animation that always begins at 0 in the video. It is the time since the recording began, not the time of the sketch, so it does not count the moment before startRecording().
Type
Number
Range
0 …
0 at the start, then refreshed four times a second while recording. When the take is saved it is that take's length, and the next take sets it back to 0.
function setup() {
createCanvas(400, 300);
}
function keyPressed() {
if (key === 'r') startRecording(4);
}
function draw() {
background(20);
const t = recordingIsOn ? OAVp5.record.seconds : 0; // 0 at the first frame of the video
fill(255, 160, 80);
circle(40 + t * 80, 150, 40); // slides right, 80 px a secondif (!recordingIsOn) {
fill(255);
text('press r: the video starts with the circle at the left', 20, 30);
}
}
Press r: the circle jumps to the left edge and slides right for four seconds, so the video always begins the same way.
Related
recordingIsOnWhether a recording is running right now.
OAVp5.record.stateWhere the recording is: 'off', 'starting', 'on' or 'saving'.
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
OAVp5.record.lastThe info of the last recording that was saved (the same object recordingSaved got), or null before the first one.
Saves the canvas alone as a PNG picture (not the 🎹, not the cards) and downloads it.
It is the button 📷 in the 🎹 panel as a function. The file is named oav-20261015-183005.png (with name in the middle when you give one) and is downloaded at once; on a phone that can share files a Share button appears on a card instead. The picture has the size of the canvas in pixels, so a retina screen gives twice the width and height. p5's own saveCanvas('name', 'png') does the same thing without the card.
Parameters
Name
Type
Description
o
Object
Options: canvas (the canvas to save: a <canvas>, a p5 canvas or a createGraphics() buffer; the biggest p5 canvas on the page by default) and name (a word for the file name). optional
Returns
Promise<Object | null> Resolves to { name, blob, url, width, height, bytes }, or to null when there is no canvas or it cannot be read (an image from another site on it, no CORS).
Notes
With the loading option record=off the 📷 button is gone and startRecording() does nothing, but this function still saves the PNG (tried).
function setup() {
createCanvas(400, 300);
}
function keyPressed() {
if (key === 'p') OAVp5.record.shot({ name: 'still' });
}
function draw() {
background(20);
circle(mouseX, mouseY, 60);
fill(255);
text('press p to save a PNG of the canvas', 20, 30);
}
Move the circle with the mouse and press p: a PNG of the canvas is downloaded.
Related
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
OAVp5.recordThe object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.
recordingSaved()An event you define: it is called when a recording has been saved and checked, with an info object about the file.
Which sounds the running take is hearing right now: a list of 'sound', 'drums', 'mic' and 'p5sound'; empty when no take runs.
It is what the take has connected, not whether something is audible at this moment. A source that is switched on halfway joins the list within a quarter of a second. After the take, recordingSaved() gives the final list in info.sources.
Type
Array<String>
A new list on every read. 'sound' is the instrument of startSound(), 'drums' the drum machine and the pads, 'mic' the microphone, 'p5sound' p5.sound 1.x.
function setup() {
createCanvas(400, 300);
startDrums('breakbeat', 120);
startSound('synth/pluck');
textSize(14);
}
function keyPressed() {
if (key === 'r') recordingIsOn ? stopRecording() : startRecording();
}
function draw() {
background(20);
fill(255);
text(recordingIsOn ? 'in the take: ' + OAVp5.record.sources.join(', ') : 'press r to start a take', 20, 30);
}
Press r: the text lists what the take has connected, here the drum machine and the instrument.
Related
recordingSaved()An event you define: it is called when a recording has been saved and checked, with an info object about the file.
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
startSound()Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
startMic()Turns the microphone on and returns a promise that says whether it worked.
Where the recording is: 'off', 'starting', 'on' or 'saving'.
recordingIsOn is state === 'on'. Read the state when the other stages matter: a button can show "starting…" and "saving…" and refuse a new take while one is being saved.
Type
String
Range
off | starting | on | saving
Starts as 'off'. A take goes 'starting' (the recorder is loading, the first time), 'on' (recording), 'saving' (the file is being made) and back to 'off'.
The audio MIME types the recorder tries for a sound-only file, best first: AAC in .m4a, then Opus.
AAC in an .m4a file comes first because it imports into every DAW and editor. This is the list a Recorder with audioOnly: true uses, through pickMime().
Type
String[]
A frozen list of 5 strings: audio/mp4;codecs=mp4a.40.2, audio/webm;codecs=opus, audio/ogg;codecs=opus, audio/mp4, audio/webm.
Examples
js
import { AUDIO_MIMES, pickMime, extFor } from'@openav/record';
const mime = pickMime((m) => MediaRecorder.isTypeSupported(m), { audioOnly: true });
console.log(AUDIO_MIMES[0]); // audio/mp4;codecs=mp4a.40.2
console.log(mime, extFor(mime)); // the one this browser would use, and its file extension
module Logs the best candidate for a sound file, and the one this browser would pick with its extension.
Related
VIDEO_MIMESThe video MIME types the recorder tries, best first: MP4 with H.264 and AAC, then WebM.
pickMime()Picks the first MIME type from a list that the browser can record, using the browser's own check.
extFor()Gives the file extension for a recorded MIME type: video/mp4 is 'mp4', audio/mp4 is 'm4a', anything unknown is 'bin'.
RecorderRecords a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
Collects the sound of your work (and, if you add it, a microphone) into one recordable track, without sending anything to the speakers.
Recording the speakers with a microphone gives you the room. Tapping the Web Audio graph gives you the instrument exactly as it was synthesized, on the same clock as the canvas. An AudioTap has two buses and two outputs. The work bus is the sound of the piece; the extra bus is for things that belong in the video but not in the sound file, such as a microphone. tap.stream and tap.tracks are work and extras together (give them to the video Recorder); tap.workStream and tap.workTracks are the work alone (for the separate audio file). Both are a single track, on purpose: Chrome records only the first audio track of a stream.
The tap never routes anything to the speakers, so adding a microphone cannot feed back. To record a sketch you did not write (p5.sound, Tone.js, Sound), call captureDestination()before the work starts its sound: from then on every node that connects to a speaker destination is tapped too.
Before you call it
An AudioContext that is running. Browsers start audio only after a click, so call tap.resume() from a click handler. Without the click the tracks exist but carry silence.
Parameters
Name
Type
Description
context
AudioContext
The AudioContext to mix in, usually the work's own. One is made if you leave it out; it needs Web Audio to exist. optionaldefault null
sources
Array
Things to add to the work bus at once: AudioNodes, MediaStreams or MediaStreamTracks (see add()). optionaldefault []
captureDestination
Boolean
Also call captureDestination() right away. optionaldefault false
Taps a source: add(source, { bus = 'work', gain = 1 }). A node of this tap's context is connected directly, a node of another AudioContext is bridged through a MediaStream, and a MediaStream or MediaStreamTrack is brought in as a source node. bus: 'extra' reaches the video only. Returns a function that removes it. Throws a TypeError for anything else.
From now on, every node that connects to an AudioDestinationNode (the speakers), in any context, is tapped too, by a hook on AudioNode.prototype.connect. Install it before the work starts its sound: a node connected earlier is invisible. It can be called again without doubling. Returns a function that uninstalls it.
Adds a microphone (or any audio input) to the video's mix, not to the audio file. { deviceId, gain = 1, echoCancellation = false, noiseSuppression = false, autoGainControl = false }: those three are off by default because they eat piano attacks and room tone. Returns a promise of the MediaStream. A microphone added before is replaced. Needs the user's permission.
Lets go of everything the tap made, including the hook and the microphone. The context itself is left alone.
Examples
js
import { AudioTap } from'@openav/record';
const tap = new AudioTap(); // makes its own AudioContextconst osc = new OscillatorNode(tap.context, { frequency: 440 });
const remove = tap.add(osc); // the work bus
osc.start();
await tap.resume(); // from a click in a real pageawaitnew Promise((done) => setTimeout(done, 300));
console.log(tap.tracks.length, tap.workTracks.length); // 1 1
console.log(tap.level() > 0.1); // true: sound is reaching the recording
remove();
module A 440 Hz tone is tapped and the meter confirms that sound reaches the recording. The tracks are ready to hand to a Recorder.
js
import { AudioTap } from'@openav/record';
const tap = new AudioTap();
const uninstall = tap.captureDestination(); // before the work starts its soundconst ctx = new AudioContext(); // "someone else's" sketch, in its own contextconst osc = new OscillatorNode(ctx, { frequency: 330 });
osc.connect(ctx.destination); // goes to the speakers, and into the tap
osc.start();
awaitnew Promise((done) => setTimeout(done, 300));
console.log(tap.level() > 0.05); // true
uninstall();
tap.dispose();
module A sound made in another AudioContext, connected to the speakers the usual way, is picked up by the tap too: this is how you record a sketch you did not write.
Related
RecorderRecords a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
mixTracks()Mixes several audio tracks into one, because Chrome records only the first audio track of a stream.
CompositorDraws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
SoundMakes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
Builds the getUserMedia options for a camera: it asks for 1920 by 1080 at 30 fps, but only as an ideal, never as a demand.
Size and rate are ideal, never exact: a camera that cannot do 1920 by 1080 should open at what it can, not throw in front of an audience. It is pure, so you can look at what openCamera() will ask for. The stream has no audio: a microphone is added through AudioTap.
Parameters
Name
Type
Description
deviceId
String
A specific camera, from listCameras(). It is asked for as exact: if it is gone, opening fails. optionaldefault ''
width
Number
The width to ask for, as ideal. optionaldefault 1920
height
Number
The height to ask for, as ideal. optionaldefault 1080
fps
Number
The frame rate to ask for, as ideal. optionaldefault 30
facingMode
String
'user' or 'environment' on a phone. Ignored when a deviceId is given. optionaldefault ''
Makes a copy of a signal value that is safe to write as JSON: rounded numbers, plain arrays, no functions, nothing too deep.
The rules: numbers are rounded to 6 decimals, and NaN and infinities become null. Typed arrays become plain arrays. A bigint becomes a number. Objects and arrays are copied down to depth 4 and anything deeper becomes null. A function, symbol or undefined becomes null in an array and the key is dropped from an object. EventLog runs every value through it.
Parameters
Name
Type
Description
v
any
The value to clean.
depth
Number
How deep we already are. Leave it out; it is used by the function calling itself. optionaldefault 0
Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
A performance video is two things in one picture: your work and the performer. The Compositor makes that picture. It owns a canvas (comp.canvas) of exactly the size of the file you want, whatever your window is, so a vertical 4K video from a laptop really is 4K. It is the canvas you hand to a Recorder. To show it on the page as a preview, put comp.canvas in the DOM and shrink it with CSS.
comp.start() redraws every animation frame (and, because a hidden tab freezes animation frames, from a Web Worker timer too, so a recording does not freeze with the tab). comp.draw() draws one frame now: call it yourself right after your work renders if you have your own loop. comp.rects is the last layout, { work, cam } in output pixels, and comp.fps is the rate it has been drawing at.
The camera is optional and never owned: setCamera(stream) shows it and setCamera(null) lets go, and you stop the tracks yourself. The camera's area can show a panel instead, anything with draw(ctx, rect, now), for example a drawing of the controller being played. The area for the camera is reserved before the camera arrives, so the picture does not jump.
Parameters
Name
Type
Description
size
String | Object | Number[]
The size of the output, which is the size of the file. A preset id from PRESETS ('vertical-4k'), '1080x1920', [w, h] or { w, h }. Each side is rounded up to an even number. See resolveSize(). optionaldefault 'vertical-1080p'
layout
String
Where the work and the camera go: stack, pip, side or work (see LAYOUTS). optionaldefault 'stack'
layoutOptions
Object
Passed on to layoutRects(): fit, camFit, camAnchor, minCam, split, pip. optionaldefault {}
getWorkCanvas
Function
Returns the canvas of your work, or null. It is asked every frame, because a p5 sketch may replace its canvas and a WebGL work may resize it. The compositor never owns the work. optionaldefault null
background
String
The colour behind everything, any canvas colour string. optionaldefault '#000'
mirrorCam
Boolean
Show the camera as a mirror, as performers expect. Keep it equal to the mirror of a hand tracker drawn on top. optionaldefault true
maxFps
Number
Draw at most this often when running. A 120 Hz display does not need 120 composites a second. optionaldefault 60
onDraw
Function
onDraw(ctx, rects, comp), called after the work and the camera are drawn, to put an overlay on top. A context state is saved and restored around it. An error in it is warned about once and does not stop the recording. optionaldefault null
canvas
HTMLCanvasElement
Draw into this canvas instead of making a new one. Its width and height are set to the output size. optionaldefault null
Changes the output size (the same values as the size option). Do it between takes: a recorder that is already running keeps the size it started with. Returns the compositor.
Changes the layout and, if you pass options, merges them into layoutOptions (the pip object is merged too). Throws for a name that is not in LAYOUTS. Returns the compositor.
Shows a camera: a MediaStream (played in a hidden muted <video> made here), a <video> or a canvas, drawn as it is. null removes it, and its tracks keep running. Returns a promise of the camera size { w, h } once frames arrive, or null (no camera, or none within 5 seconds).
Draws a panel where the camera goes: { draw(ctx, rect, now), aspect }. draw is called every frame with a context clipped to the area, rect is { x, y, w, h } in output pixels and aspect (width divided by height, default 16/9) shapes a picture-in-picture corner. setPanel(null) brings the camera back, which stayed attached. The compositor never disposes a panel. Throws if the panel has no draw.
Draws over the camera in the camera's own coordinates: fn(ctx, w, h) gets a context moved to where the whole camera frame lands and clipped to the visible camera area, which is what a hand or pose tracker's skeleton(ctx, w, h) expects. Call it from onDraw. Does nothing without a camera, or with a panel in its place.
Stops and lets go of the camera (its tracks are yours to stop).
Notes
A WebGL work: the compositor copies its canvas in the same animation frame it was drawn in, which works in Chrome. If you see black, create the WebGL context with preserveDrawingBuffer: true, or call comp.draw() right after your render and do not call start().
Webcams usually stop at 1080p30, so a 4K stack layout enlarges the camera about 1.6 times. The work above it stays sharp because it is drawn at the output size, as long as the work's own canvas is at least as large as its area.
In a p5 sketch you do not need this: startRecording() records the canvas and its sound for you. The Compositor is for a page that records the work together with a camera.
Examples
js
import { Compositor } from'@openav/record';
const work = document.createElement('canvas'); // the work: any canvas
work.width = 600; work.height = 600;
const g = work.getContext('2d');
g.fillStyle = '#c33'; g.fillRect(0, 0, 600, 600);
const comp = new Compositor({ size: '540x960', layout: 'stack', getWorkCanvas: () => work });
const rects = comp.draw(); // one frame, now
console.log(comp.size, rects.work.h, rects.cam.box.h); // { w: 540, h: 960 } 540 420
comp.canvas.style.width = '135px'; // a preview, scaled down with CSS
document.getElementById('app').append(comp.canvas);
module A square work is drawn into a 540 by 960 frame: the work takes the top 540 pixels and 420 are kept for the camera. The result appears on the page as a small preview.
js
import { Compositor, openCamera } from'@openav/record';
const work = document.getElementById('c'); // the 400 x 300 canvas of the pageconst wg = work.getContext('2d');
const comp = new Compositor({ size: 'landscape-1080p', layout: 'pip', getWorkCanvas: () => work,
layoutOptions: { pip: { corner: 'br', size: 0.25 } } });
comp.start(); // redraws every frameconst size = await comp.setCamera(await openCamera()); // the browser asks for the camera
console.log(size, comp.rects.cam.w); // { w: 1920, h: 1080 } 480 (what the camera gives may differ)let x = 0;
(function animate() { // the work moves, the composite follows
wg.fillStyle = '#111'; wg.fillRect(0, 0, 400, 300);
wg.fillStyle = '#ffd166'; wg.fillRect(x = (x + 4) % 380, 120, 20, 60);
requestAnimationFrame(animate);
})();
comp.canvas.style.width = '480px';
document.getElementById('app').append(comp.canvas);
module The animated canvas fills a 1080p frame with the camera in a corner. Allow the camera when the browser asks; the composite appears below the work.
js
import { Compositor } from'@openav/record';
const work = document.getElementById('c');
const comp = new Compositor({
size: [720, 720], layout: 'work', getWorkCanvas: () => work,
onDraw(ctx, rects) { // an overlay, on top of everything
ctx.fillStyle = 'rgba(255, 255, 255, 0.85)';
ctx.font = '32px sans-serif';
ctx.fillText('take 1', rects.work.x + 20, rects.work.y + 48);
},
});
work.getContext('2d').fillRect(0, 0, 400, 300);
comp.draw();
document.getElementById('app').append(comp.canvas);
comp.canvas.style.width = '240px';
module The work alone in a square frame, with a caption written over it by onDraw. The caption is part of the picture, so it is in the recording.
Related
RecorderRecords a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
layoutRects()Works out where the work and the camera go in an output frame, in whole pixels, for one of the four layouts.
PRESETSThe output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.
openCamera()Opens a camera and returns its MediaStream: ideally 1920 by 1080 at 30 fps, or whatever the camera can do.
AudioTapCollects the sound of your work (and, if you add it, a microphone) into one recordable track, without sending anything to the speakers.
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
new EventLog({ now = defaultNow, filter = null, throttleMs = 0 } = {})
Keeps a timestamped list of what happened during a take, such as notes and parameter changes, on the same clock as the recording.
A video keeps the picture and the sound. The log keeps the score of the take: every note, pad hit and (throttled) hand position, in milliseconds from the moment recording started. With it a take can be re-rendered at another size, re-synced to a better audio take, or turned into MIDI later. Start it with the recorder's start time so event times line up with the video's first frame: log.start(recorder.t0).
log.attach(signals) logs every signal a Signals bus publishes; pulses are always kept and continuous signals obey throttleMs. Or add events yourself with log.add(name, value). Values are cleaned into plain JSON data by cleanValue(). toJSON() gives { format: 'openav-eventlog', version: 1, startedAt, durationMs, count, events }, and EventLog.fromJSON(json) reads it back. toTake() turns the log into a MIDI take; TakeRecorder is a log that does exactly that on midi/*.
Properties: events is the list [{ t, name, value }] (t in ms, to a tenth), t0 the clock time of t = 0, startedAt the wall-clock time (ISO text) and durationMs is null until stop().
Parameters
Name
Type
Description
now
Function
The clock, in milliseconds. The default is the same clock as Recorder.t0. Pass your own in a test. optionaldefault performance.now
filter
Function
filter(name, value, meta), returns whether to keep an event. It applies to add() too. optionaldefault null
throttleMs
Number
For continuous signals: keep at most one event per name in this many milliseconds. Pulses (note on, note off…) are never throttled. optionaldefault 0
Logs one event: add(name, value, { at, pulse, meta }), and returns it as { t, name, value }, or null when the log is not running, the filter refused it, it happened before t = 0, or it was throttled. pulse: false makes it subject to throttleMs; at is the clock time (default now).
The log as a MIDI take { v: 1, name, createdAt, durationMs, events, meta } (see normalizeTake()). { name = 'take', meta = {} }. Every event is kept; filter to midi/* first, or use a TakeRecorder.
Static. Rebuilds a stopped log from toJSON() output or its text. Throws a TypeError if it is not an openav-eventlog.
Examples
js
import { EventLog } from'@openav/record';
let clock = 1000; // a clock we move by hand, in msconst log = new EventLog({ now: () => clock });
log.start();
clock = 1100; log.add('midi/note/on', { note: 60, vel: 0.8 });
clock = 1200; log.mark('chorus');
clock = 1500; log.stop();
console.log(log.events.map((e) => e.t + ' ' + e.name).join(' | ')); // 100 midi/note/on | 200 mark
console.log(log.durationMs, log.running, log.toJSON().count); // 500 false 2
console.log(log.add('late')); // null: the take is over
module A take of half a second on a clock moved by hand: a note at 100 ms, a marker at 200 ms. After stop() nothing more is accepted.
js
import { EventLog } from'@openav/record';
import { Signals } from'@openav/core';
const signals = new Signals();
const log = new EventLog({ throttleMs: 100 }); // knobs: at most one event per 100 ms
log.start();
const detach = log.attach(signals);
signals.pulse('midi/note/on', { note: 64, vel: 0.7 }); // a pulse: always keptfor (let i = 0; i < 10; i++) signals.set('param/hue', i / 10); // continuous: throttled
detach();
console.log(log.events.length, log.events[0].name); // 2 midi/note/on
console.log(EventLog.fromJSON(log.toJSON()).events.length); // 2
module A bus is attached: the note pulse is kept, and ten changes of a continuous signal in a few milliseconds become one event. The log survives a round trip through JSON.
Related
TakeRecorderRecords what you play: it listens to every midi/* signal on a bus from start() to stop() and gives you a take.
RecorderRecords a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
cleanValue()Makes a copy of a signal value that is safe to write as JSON: rounded numbers, plain arrays, no functions, nothing too deep.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
toMidiFile()Writes a take as a Standard MIDI File (type 0) that opens in any DAW.
Turns a point in the camera's own 0..1 coordinates, like a hand landmark, into a pixel in the output frame.
Landmarks from a hand or pose tracker come as fractions of the camera picture. To draw something on the output at the place a landmark is, you need pixels of the output, where the camera may be enlarged and cropped. framePoint() does that. It gives the same point whether or not part of the camera is cropped away, because it uses frame, the whole camera picture placed in the output. Use the same mirror in the tracker and in the compositor.
Across the camera picture, 0 left to 1 right, in the displayed (mirrored if mirrorCam) orientation.
v
Number
Down the camera picture, 0 top to 1 bottom.
Returns
Object | null{ x, y } in output pixels, or null when there is no camera (or its size is not known yet).
Examples
js
import { layoutRects, framePoint } from'@openav/record';
const r = layoutRects('stack', { w: 1080, h: 1920 }, { workW: 1000, workH: 1000, camW: 1920, camH: 1080 });
console.log(framePoint(r.cam, 0.5, 0.5)); // { x: 540, y: 1500 }: the middle of the camera
console.log(framePoint(r.cam, 0, 0)); // { x: -206.67, y: 1080 }: the left edge is cropped away, off the frame
console.log(framePoint(null, 0.5, 0.5)); // null: no camera
module The centre of the camera lands in the middle of its area. The camera's left edge is outside the frame because the 16:9 picture is cropped to fit.
Related
layoutRects()Works out where the work and the camera go in an output frame, in whole pixels, for one of the four layouts.
CompositorDraws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
Reads a Standard MIDI File into a take whose events are the signals a keyboard playing that file would publish.
It reads type 0 and type 1 files (the tracks are merged), running status, tempo changes and SMPTE time division; system exclusive messages are skipped. A file that is not a MIDI file throws a TypeError (fromMidiFile: not a MIDI file (no MThd)), and one that ends early a RangeError. Play the result into a show with TakePlayer: it drives a work like someone playing the file on a keyboard.
Parameters
Name
Type
Description
bytes
Uint8Array | ArrayBuffer | Number[]
The bytes of the .mid file, for example await file.arrayBuffer().
name
String
The name of the take. Left out, it is the track name in the file, then 'take'. optional
families
'both' | 'legacy' | 'channel'
Which signal spellings to produce, as in signalsOfMidi(). The default is both, like hardware. optionaldefault 'both'
Returns
Object A take. meta.bpm is the first tempo in the file (not set for SMPTE time).
module A round trip: the two notes come back as four events (both spellings), or two with families: 'legacy'. Bytes that are not a MIDI file are refused.
Related
toMidiFile()Writes a take as a Standard MIDI File (type 0) that opens in any DAW.
signalsOfMidi()Turns one MIDI event into the signal events a keyboard playing it would publish, so a file can drive a work like live playing.
TakePlayerPlays a take back into a bus at the right times, so the work, its sound and its chord detector cannot tell it from you playing.
normalizeTake()Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.
Works out where the work and the camera go in an output frame, in whole pixels, for one of the four layouts.
It is pure arithmetic: no DOM, no clock. The Compositor calls it whenever a size changes. A 1080 by 1920 frame with a square work and stack splits into exactly 1080 by 1080 and 1080 by 840, with no seam row. The crop of a camera is in the displayed orientation, so with the mirror on, an anchor of x: 0.2 still means "keep the left of what you see".
stack puts the work across the full width at the top and the camera below. If that would leave the camera under minCam of the height (a square or landscape frame), the work gets split of the height and is letterboxed in it. side is the same turned sideways. pip fills the frame with the work and puts the camera in a corner as a rounded inset. work is the work alone.
Parameters
Name
Type
Description
layout
String
One of LAYOUTS. Any other name throws a TypeError.
out
Object
The output size { w, h } in pixels. It must be positive, or this throws.
src
Object
The sizes of the sources: { workW, workH, camW, camH }. A missing or zero value means unknown. An unknown work counts as square. With an unknown camera the camera's area is still reserved, with crop: null and frame: null, so the frame does not jump when the camera arrives. optionaldefault {}
opts
Object
Options: fit ('contain' default, or 'cover', for the work), camFit ('cover' default, or 'contain'), camAnchor ({ x, y }, 0..1, default the middle: which part a cover-crop keeps, in what you see), minCam (0.2: the share of the height in stack, or of the width in side, that the camera must keep), split (0.6: the work's share when it would leave the camera less than minCam) and pip ({ corner: 'br' | 'bl' | 'tr' | 'tl', size: 0.3, margin: 0.03, radius: 0.08, aspect: 0 }: the inset's width as a share of the frame, its distance from the corner as a share of the short side, its corner radius as a share of the inset's short side, and its shape, 0 being the camera's own). optionaldefault {}
Returns
Object{ layout, out, work, cam }. work and cam are { x, y, w, h, crop, frame, box } in output pixels (all whole numbers), and cam also has radius. crop is the part of the source that is drawn (source pixels, may be fractional) or null for the whole source. frame is where the whole source frame would land, cropped parts falling outside the rect, which is what overlays need. box is the area it was placed in. cam is null for the work layout.
Examples
js
import { layoutRects } from'@openav/record';
const r = layoutRects('stack', { w: 1080, h: 1920 }, { workW: 1000, workH: 1000, camW: 1920, camH: 1080 });
console.log(r.work.x, r.work.y, r.work.w, r.work.h); // 0 0 1080 1080
console.log(r.cam.x, r.cam.y, r.cam.w, r.cam.h); // 0 1080 1080 840
console.log(r.cam.crop); // the middle of the 16:9 camera that fills 1080x840
module A square work and a 16:9 camera in a vertical frame: the work is the top 1080 pixels, the camera fills the 840 below it with a middle crop.
The four layout names, in picker order: 'stack', 'pip', 'side', 'work'.
stack: the work across the full width at the top, the camera fills the rest below (for vertical video: Reels, Shorts, TikTok). pip: the work fills the frame and the camera is a rounded inset in a corner (for square frames, or a work that needs all of the frame). side: the work at full height on the left, the camera fills the right (landscape). work: the work alone. layoutRects() says exactly where everything lands, and the layout option of the Compositor takes one of these names; any other name throws.
Type
String[]
['stack', 'pip', 'side', 'work'], a frozen-by-convention list (do not change it).
Examples
js
import { LAYOUTS, layoutRects } from'@openav/record';
console.log(LAYOUTS.join(' ')); // stack pip side workfor (const name of LAYOUTS) {
const r = layoutRects(name, { w: 1080, h: 1920 }, { workW: 1000, workH: 1000, camW: 1920, camH: 1080 });
console.log(name, r.work.w + 'x' + r.work.h, r.cam ? r.cam.w + 'x' + r.cam.h : 'no camera');
}
// stack 1080x1080 1080x840// pip 1080x1080 324x182// side 648x648 432x1920 (a vertical frame is too narrow for side: the work gets 60% of the width)// work 1080x1080 no camera
module Runs the four layouts on one vertical frame with a square work and a 16:9 camera, and logs the size of each area.
Related
layoutRects()Works out where the work and the camera go in an output frame, in whole pixels, for one of the four layouts.
CompositorDraws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
PRESETSThe output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.
Lists the video inputs of this computer as { deviceId, groupId, label }.
Browsers hide the labels (and sometimes the ids) until the page has had camera permission once, so call it again after openCamera() to get the real names. Needs https or localhost: elsewhere there is no navigator.mediaDevices and the promise rejects.
Returns
Promise<Object[]> A promise of a list of { deviceId, groupId, label }. A device without a label is called Camera 1, Camera 2… by its position.
Examples
js
import { listCameras, openCamera } from'@openav/record';
console.log((await listCameras()).length > 0); // true; the labels are generic until permission is givenconst stream = await openCamera(); // the browser asks for the cameraconst cams = await listCameras(); // now with their real names
console.log(cams.map((c) => c.label).join(', '));
stream.getTracks().forEach((t) => t.stop());
module Lists the cameras before and after permission is given; the second list has the real names. Stop the tracks when you are done.
Related
listMicrophones()Lists the audio inputs of this computer as { deviceId, groupId, label }.
openCamera()Opens a camera and returns its MediaStream: ideally 1920 by 1080 at 30 fps, or whatever the camera can do.
cameraConstraints()Builds the getUserMedia options for a camera: it asks for 1920 by 1080 at 30 fps, but only as an ideal, never as a demand.
Lists the audio inputs of this computer as { deviceId, groupId, label }.
The same as listCameras() for audio, with the same rule about labels: they are generic until the page has had microphone permission once. Pass a deviceId to AudioTap.addMicrophone() to use one.
Returns
Promise<Object[]> A promise of a list of { deviceId, groupId, label }. A device without a label is called Microphone 1, Microphone 2… by its position.
Reads a take as plain MIDI events (note on and off, controllers, pitch bend), picking one naming family so nothing is counted twice.
A take made on hardware holds the same note several times, under several names. This picks one family for the notes: midi/note/* if present, else midi/ch/*, else one device's midi/<device>/note/*, else raw midi/virtual bytes. Controllers and pitch bend come from midi/ch/* if present, else the classic names (as channel 1), else one device's. toMidiFile() and takeStats() are built on it.
Parameters
Name
Type
Description
take
Object
The take.
Returns
Object[] A list in the shape of @openav/midi: { t, type: 'noteon', ch, note, vel }, noteoff, { t, type: 'cc', ch, cc, value }, { t, type: 'pitchbend', ch, value } (0..16383), chanat and polyat. vel and value are 0..127.
Mixes several audio tracks into one, because Chrome records only the first audio track of a stream.
A Recorder calls this itself when you give it more than one live audio track, and closes the mix on stop(). Call it yourself when you want the one track for something else. It makes an AudioContext, so the page needs to have had a click. For several sounds that come from Web Audio nodes, AudioTap is the better tool: it mixes them without a detour through streams.
Parameters
Name
Type
Description
tracks
MediaStreamTrack[]
The audio tracks to mix.
Returns
Object{ track, close() }: the one mixed MediaStreamTrack, and a function that closes the AudioContext that mixes it.
Examples
js
import { AudioTap, mixTracks } from'@openav/record';
const tap = new AudioTap();
const mix = mixTracks([tap.tracks[0], tap.workTracks[0]]); // two tracks in, one comes out
console.log(mix.track.kind, mix.track.readyState); // audio live
mix.close(); // closes the mixing context
tap.dispose();
module Two audio tracks go in and one live audio track comes out; close() shuts the mixer down.
Related
RecorderRecords a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
AudioTapCollects the sound of your work (and, if you add it, a microphone) into one recordable track, without sending anything to the speakers.
Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.
A take is what an EventLog filtered to midi/* records, as plain data: t is milliseconds from the start, name is a signal name such as midi/note/on, value is its payload, and meta holds free fields like { bpm, device, notes }. normalizeTake() makes any input safe: events that are not objects, have no name or have a t that is not a number are dropped; t is rounded to a tenth of a millisecond and held at 0 or more; the events are sorted by t (events with equal times keep their order); name defaults to 'take'; durationMs is at least the time of the last event. A new object is returned.
Every function in this package that takes a take runs it through here first, so you can pass raw data.
Parameters
Name
Type
Description
input
Object | String
A take, or its JSON text. A string is parsed with JSON.parse. Anything that is not an object throws a TypeError.
Returns
Object A new take: { v: 1, name, createdAt, durationMs, events: [{ t, name, value }], meta }.
Tells whether a recorded event is a key going down or up, whatever its spelling, and which note on which channel.
A hardware keyboard publishes the same note under several names, and this function knows them all: midi/note/on and midi/note/off (family legacy, with value: { note, vel, ch }), midi/ch/<ch>/note/<note> (family channel, a number, 0 for off), midi/<device>/note/on|off (family slug:<device>) and raw bytes on midi/virtual (family virtual, { data: [status, note, vel] }). A key-down with velocity 0 in raw bytes counts as off. A channel of 0, as the on-screen piano says, counts as channel 1.
Parameters
Name
Type
Description
ev
Object
An event { name, value } of a take.
Returns
Object | null{ key, on, note, ch, family, vel }, or null if it is not a note event. key is '<ch>:<note>', on is true for a key-down, family says which spelling it came in, vel is 0..127.
Gives the events that release a sounding note, in the same spelling it was played in.
This is how a TakePlayer and a TakeRecorder make sure no note is left sounding: for legacy it is midi/note/off, for channel it is midi/ch/<ch>/note/<note> with 0, for virtual it is a raw note-off byte triple, for a device family midi/<device>/note/off.
Opens a camera and returns its MediaStream: ideally 1920 by 1080 at 30 fps, or whatever the camera can do.
Give the stream to Compositor.setCamera(). The caller owns the stream: stop its tracks when you are done, the compositor never does. Webcams usually top out at 1080p30, so a 4K frame enlarges the camera; the work above it stays sharp. It needs https or localhost, and the user's permission.
Parameters
Name
Type
Description
opts
Object
The options of cameraConstraints(): { deviceId, width, height, fps, facingMode }. optionaldefault {}
Returns
Promise<MediaStream> A promise of the stream, with one video track and no audio. It rejects with the browser's error when permission is refused or the camera is missing.
Examples
js
import { openCamera } from'@openav/record';
const stream = await openCamera(); // the browser asks for the cameraconst video = document.createElement('video');
video.muted = true; video.srcObject = stream; await video.play();
console.log(video.videoWidth > 0); // true once frames arrive
document.getElementById('app').append(video);
video.style.width = '240px';
module Allow the camera and a small live picture of it appears on the page.
Related
CompositorDraws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
cameraConstraints()Builds the getUserMedia options for a camera: it asks for 1920 by 1080 at 30 fps, but only as an ideal, never as a demand.
listCameras()Lists the video inputs of this computer as { deviceId, groupId, label }.
Picks the first MIME type from a list that the browser can record, using the browser's own check.
A Recorder calls this for you. It takes the browser's check as an argument so that it is pure and can be tried with a fake one, like the examples below.
Parameters
Name
Type
Description
isTypeSupported
Function
The browser's check, for example (m) => MediaRecorder.isTypeSupported(m). Not a function gives ''. A check that throws counts as "no" for that type.
module Logs the numbers of one preset and shows that an unknown id gives null.
Related
PRESETSThe output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.
resolveSize()Turns any way of naming a size, like 'vertical-4k', '1080x1920' or [w, h], into { w, h }, rounded up to even numbers.
The output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.
text
id size fps layout
vertical-1080p 1080x1920 60 stack
vertical-4x5 1440x1800 60 stack
vertical-2.7k 1520x2704 30 stack
vertical-4k 2160x3840 30 stack
landscape-1080p 1920x1080 60 side
landscape-16x10 1920x1200 60 side
landscape-2.7k 2704x1520 30 side
landscape-4k 3840x2160 30 side
square-1080 1080x1080 60 pip
square-2048 2048x2048 30 pip
square-2160 2160x2160 30 pip
Every side is even, because H.264 and VP9 encode 4:2:0 colour, which needs even sizes. Use an id as the size of a Compositor; presetById() looks one up.
Type
Object[]
A frozen list of 11 presets. Each is { id, group, w, h, label, labelZh, layout, fps }: group is 'vertical', 'landscape' or 'square', label and labelZh are names for a picker, layout is the layout suggested for the shape and fps is recommendedFps().
Snaps the key presses of a take to a rhythmic grid, so a loose performance lands on the beat.
Each key-down moves to the nearest grid line (60000 / bpm / grid milliseconds apart). The key-up of a note moves with it, so the length of every note is kept. Everything that is not a note (controllers, pitch bend) stays where it was. A note written in two spellings at the same time (midi/note/on and midi/ch/1/note/60) moves by the same amount in both.
Parameters
Name
Type
Description
take
Object
The take.
bpm
Number
The tempo. Left out, it is take.meta.bpm, and then 120. optional
grid
Number
Grid steps per beat: 4 is sixteenths, 2 is eighths, 1 is quarters. optionaldefault 4
strength
Number
How far to move, 0..1. 1 is all the way, 0.5 is half way to the grid, 0 changes nothing. optionaldefault 1
offsetMs
Number
Shift the grid later by this many milliseconds, for a take that does not start on a beat. optionaldefault 0
module Two notes a little off the beat snap to 500 and 1000 ms, and each note-off moves with its note. With a strength of one half they only go halfway.
Related
trimSilence()Cuts the silence before the first key press of a take, so the first note comes at the moment you choose.
takeStats()Counts what is in a take: notes, events, length, the channels used, and the lowest and highest note.
toMidiFile()Writes a take as a Standard MIDI File (type 0) that opens in any DAW.
normalizeTake()Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.
A video bitrate that keeps fast generative detail clean: 0.12 bits per pixel per frame, held between 4 and 80 Mb/s.
Particles and thin lines are the first things a low bitrate smears, so this is higher than a streaming bitrate on purpose: the file is a master that will be edited and encoded again. It is the default videoBitsPerSecond of a Recorder that records a canvas.
Parameters
Name
Type
Description
w
Number
Width in pixels.
h
Number
Height in pixels.
fps
Number
The frame rate. Left out, it is recommendedFps() for the size. optionaldefault recommendedFps(w, h)
Returns
Number Bits per second, rounded to 100 000. 1080 by 1920 at 60 fps is 14 900 000, 4K at 30 fps is 29 900 000.
The frame rate to record a frame size at: 60 up to about 2.6 megapixels, 30 above that.
60 fps holds up to about 1080 by 1920 (and 1440 by 1800, 1920 by 1200). A 4K canvas at 60 means 500 million pixels a second through the compositor and the encoder, which drops frames on a laptop in the middle of a performance, and 30 is what cameras shoot 4K at anyway. Pass the result as fps to a Recorder.
Parameters
Name
Type
Description
w
Number
Width in pixels.
h
Number
Height in pixels.
Returns
30 | 60 60 when w × h is at most 2 600 000 pixels, otherwise 30.
module Logs the rate for a 1080p vertical frame, a 4K vertical frame and a 16:10 landscape frame.
Related
recommendedBitrate()A video bitrate that keeps fast generative detail clean: 0.12 bits per pixel per frame, held between 4 and 80 Mb/s.
RecorderRecords a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
PRESETSThe output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.
Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
new Recorder({ canvas, audioTracks }) then start() and, later, await stop(). The canvas track and the audio track go into the same MediaRecorder, which stamps both from one clock: nothing has to be lined up afterwards. stop() resolves, after the last chunk, with { mimeType, ext, bytes, chunks, durationMs, width, height, fps, audio, blob, error }: the blob is the file, ext the extension to give it (mp4, webm, m4a…), audio whether it has sound. Give the blob to URL.createObjectURL() for a download link.
Which file you get depends on the browser: it records MP4 (H.264 and AAC) when it can, WebM otherwise. A WebM from Chrome has no duration in its header; players cope, editors may want ffmpeg -i in.webm -c copy out.webm. The file is a master to edit, so the default bitrate is high.
To keep a long take out of the tab's memory, pass ondata(blob, seq) to start(): each chunk (every timeslice ms) is handed over in order, and a promise you return is awaited before the next chunk and before stop() resolves, so uploads stay ordered. With ondata the chunks are not kept and blob is null, unless you pass keep: true. To line a take up with an EventLog, start the log with recorder.t0, the performance.now() of the moment recording started.
Before you call it
A browser with MediaRecorder (Recorder.supported tells), and a canvas or a stream, or audioOnly: true: otherwise the constructor throws a TypeError. Mixing several audio tracks makes an AudioContext, which needs the page to have had a click. A camera or microphone in the stream needs the user's permission, as always.
Parameters
Name
Type
Description
canvas
HTMLCanvasElement
What to record, usually compositor.canvas. optionaldefault null
stream
MediaStream
A ready video stream to record instead of a canvas. optionaldefault null
fps
Number
The capture rate of the canvas. Use recommendedFps(w, h): 30 for 2.7K and 4K. optionaldefault 60
audioTracks
MediaStreamTrack[]
The sound, for example tap.tracks of an AudioTap. Several tracks are mixed into one, because Chrome records only the first audio track of a stream. optionaldefault []
audioOnly
Boolean
Record only the audio tracks: the sound as a file of its own. Then canvas and stream are not needed. optionaldefault false
videoBitsPerSecond
Number
The video bitrate. The default is recommendedBitrate() of the canvas size and fps (12 Mb/s when there is no canvas). optional
audioBitsPerSecond
Number
The audio bitrate. optionaldefault 192000
mimeType
String
Force one MIME type. The default is the first one the browser supports from VIDEO_MIMES or AUDIO_MIMES, see pickMime(). optional
prefer
'mp4' | 'webm'
Try this container first when picking the MIME type. optionaldefault null
Starts recording and returns the recorder. { timeslice = 1000, ondata, keep }: how often (ms) a chunk is handed over, a function for each chunk, and whether to keep the chunks for the blob. Starting a recorder that already records does nothing. Throws if the browser has no MediaRecorder.
Stops, waits for the last chunk and every ondata promise, stops the canvas track it made and returns a promise of the result object. Calling it again returns the same result. Throws if the recorder was never started.
Static getter. True when this browser can record a canvas: it has MediaRecorder and canvas.captureStream.
Examples
js
import { Recorder } from'@openav/record';
const canvas = document.getElementById('c');
const g = canvas.getContext('2d');
let x = 0;
(function animate() { // something moving to record
g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
g.fillStyle = '#ffd166'; g.fillRect(x = (x + 5) % 380, 120, 20, 60);
requestAnimationFrame(animate);
})();
const rec = new Recorder({ canvas, fps: 30 });
rec.start();
awaitnew Promise((done) => setTimeout(done, 1000)); // record for one secondconst take = await rec.stop();
console.log(take.ext, take.width, take.height, take.fps, take.audio, take.blob.size > 0);
// mp4 400 300 30 false true (webm where the browser cannot write mp4)const a = document.getElementById('app').appendChild(document.createElement('a'));
a.href = URL.createObjectURL(take.blob); a.download = 'take.' + take.ext; a.textContent = 'Download';
module Records one second of the moving square and logs what the file is. A Download link appears on the page.
js
import { Recorder, AudioTap } from'@openav/record';
const tap = new AudioTap(); // the sound that goes into the fileconst osc = new OscillatorNode(tap.context, { frequency: 220 });
tap.add(osc); osc.start();
await tap.resume(); // from a click, in a real pageconst video = new Recorder({ canvas: document.getElementById('c'), fps: 30, audioTracks: tap.tracks });
const sound = new Recorder({ audioOnly: true, audioTracks: tap.workTracks }); // the sound as its own file
video.start(); sound.start();
awaitnew Promise((done) => setTimeout(done, 1000));
const [v, a] = await Promise.all([video.stop(), sound.stop()]);
console.log(v.ext, v.audio, a.ext, a.width); // mp4 true m4a 0
module One second of video with a 220 Hz tone in it, and the same tone as a separate audio file. The picture and the sound of the video enter the same recorder.
js
import { Recorder } from'@openav/record';
const chunks = [];
const rec = new Recorder({ canvas: document.getElementById('c'), fps: 30 });
rec.start({
timeslice: 250,
ondata: async (blob, seq) => { chunks.push(blob.size); }, // send it somewhere here; the next chunk waits
});
awaitnew Promise((done) => setTimeout(done, 1000));
const take = await rec.stop();
console.log(take.blob, take.chunks === chunks.length, take.bytes === chunks.reduce((a, b) => a + b, 0)); // null true true
module With ondata the chunks are handed to you one by one, in order, and the result carries no blob: this is how a long take stays out of the tab's memory.
To be confirmed
Whether data arrives while the recording is still running depends on the container and the browser. In headless Chromium, writing MP4, bytes stayed 0 for a one-second take with a timeslice of 200 and the chunks came at stop(). Other browsers and WebM were not tried.
Related
CompositorDraws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
AudioTapCollects the sound of your work (and, if you add it, a microphone) into one recordable track, without sending anything to the speakers.
EventLogKeeps a timestamped list of what happened during a take, such as notes and parameter changes, on the same clock as the recording.
pickMime()Picks the first MIME type from a list that the browser can record, using the browser's own check.
recommendedFps()The frame rate to record a frame size at: 60 up to about 2.6 megapixels, 30 above that.
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
stopRecording()Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.
Turns any way of naming a size, like 'vertical-4k', '1080x1920' or [w, h], into { w, h }, rounded up to even numbers.
It is what the Compositor uses for its size and setSize(). A size that cannot be read ('wat', a negative number, a missing side) throws a TypeError: record: unknown size "wat". Even sizes matter because H.264 and VP9 encode 4:2:0 colour.
Parameters
Name
Type
Description
size
String | Number[] | Object
A preset id from PRESETS, 'WxH' (also with × or *), [w, h] or { w, h }.
Returns
Object{ w, h }, whole numbers, each rounded up to the next even number.
module Four ways of naming a size: a preset, text, a list, and a mistake that throws. Odd sides become even.
Related
PRESETSThe output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.
presetById()Finds one size preset by its id: presetById('vertical-4k').
CompositorDraws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
layoutRects()Works out where the work and the camera go in an output frame, in whole pixels, for one of the four layouts.
Turns one MIDI event into the signal events a keyboard playing it would publish, so a file can drive a work like live playing.
It is what fromMidiFile() uses to build a take from a .mid file: each event of the file becomes the signals a keyboard would have published, so TakePlayer can play it into a show. Call it yourself when you have MIDI events from somewhere else and want them on a Signals bus.
Parameters
Name
Type
Description
ev
Object
A MIDI event { type, ch, note?, vel?, cc?, value? }. type is noteon, noteoff, cc, pitchbend (value 0..16383), chanat or polyat; velocities and values are 0..127.
families
'both' | 'legacy' | 'channel'
Which spellings to make: the classic names (midi/note/on, midi/cc/64, midi/bend), the per-channel ones (midi/ch/1/note/60, midi/ch/1/cc/64…), or both, as hardware does. optionaldefault 'both'
device
String
A device name to put in the classic note events. optionaldefault null
Returns
Object[] A list of { name, value }. Note velocity becomes 0..1 (vel and velocity for the classic spelling), controllers 0..1, pitch bend -1..1. chanat and polyat exist only in the per-channel spelling.
The version number of the take format, which is 1. Every take has v: 1.
A take is a plain object { v, name, createdAt, durationMs, events, meta }, see normalizeTake(). validateTake() complains when v is not this number, and normalizeTake() writes it.
Plays a take back into a bus at the right times, so the work, its sound and its chord detector cannot tell it from you playing.
Give it a bus and a take, call play(), and it publishes the take's events into the bus at their times: the same signals a keyboard would have, so a work's visuals, its sound (Sound) and its chord detector respond exactly as to your hands. That is what lets a work's preview play itself. Pass clock: 'manual' and call player.update(dt) from your loop for full control.
No stuck notes, ever: the player remembers every key it pressed, in the spelling it pressed it, and the sustain pedal, and lets go of them on pause, stop, seek and every loop wrap. With yieldToLive, playing a key yourself makes the take fall silent; with resumeAfter: 8 it comes back 8 seconds after you stop. Its state is 'playing', 'paused', 'yielded' or 'stopped', and position (milliseconds), loops, playing and yielded are plain properties.
Parameters
Name
Type
Description
signals
Signals
The Signals bus to publish into. It also defines take/playing (0 or 1) and take/position (0..1). optionaldefault null
take
Object | String
A take or its JSON. You can load() another one later. optionaldefault null
loop
Boolean
Start again at the end. Off, it stops there. optionaldefault true
speed
Number
Playback speed: 0.5 is half speed. optionaldefault 1
yieldToLive
Boolean
Fall silent when someone plays live: a key-down that the player did not send (hardware, the on-screen piano, a phone) pauses the take and releases its notes. optionaldefault false
resumeAfter
Number
Seconds without live playing before a take that gave way resumes. 0 means it stays paused. optionaldefault 0
isLive
Function
isLive(name, value), what counts as live playing. The default is any key-down that did not come from the player; knobs do not count. optionaldefault any key-down
chase
Boolean
When you seek(), send again the last value of every controller before that point (the sustain pedal, knobs), so the take is in the state it would have been in. optionaldefault true
clock
'raf' | 'timer' | 'manual'
Who drives the time: 'raf' (requestAnimationFrame, the default in a page), 'timer' (a 10 ms timer: tighter for sound and it keeps going, slower, in a background tab) or 'manual' (you call update(dt) or tick(); the default where there is no requestAnimationFrame). optional
maxStepMs
Number
The longest time one step may move forward. After a gap (a frozen tab) the take slows down instead of firing a burst of notes. optionaldefault 250
onEvent
Function
onEvent({ t, name, value }, player), for every event as it is played. optionaldefault null
onState
Function
onState(state, reason), when it changes: 'playing', 'paused', 'yielded' or 'stopped'. optionaldefault null
now
Function
The clock, in milliseconds. optionaldefault performance.now
Getter. 'playing', 'yielded' (gave way to live playing), 'paused' (anywhere after the start, also at the end of a take that does not loop) or 'stopped' (at the start). The onState callback still says 'stopped' with the reason 'end' when a take that does not loop ends.
Plays from where it is, from the start after the end, or from from milliseconds: play({ from }). Returns false when there is nothing to play (no take, or an empty one), otherwise true.
Jumps to ms milliseconds (held between 0 and the length), releasing the sounding notes, and with chase sends the controllers' values at that point. Returns the player.
Stops, releases every note and stops listening to the bus.
Examples
js
import { TakePlayer } from'@openav/record';
import { Signals } from'@openav/core';
const take = { v: 1, name: 'riff', durationMs: 600, events: [
{ t: 100, name: 'midi/note/on', value: { note: 60, vel: 0.9, ch: 1 } },
{ t: 400, name: 'midi/note/off', value: { note: 60, ch: 1 } },
] };
const signals = new Signals();
const heard = [];
signals.on('midi/note/on', (v) => heard.push('on ' + v.note));
signals.on('midi/note/off', (v) => heard.push('off ' + v.note));
const player = new TakePlayer({ signals, take, loop: false, clock: 'manual' });
console.log(player.state, player.duration); // stopped 600
player.play();
for (let i = 0; i < 20; i++) player.update(0.05); // one second, in 50 ms steps
console.log(heard.join(', '), player.state); // on 60, off 60 paused
module A two-event take is played by hand-cranked time into a bus: the listeners hear the note start and end, and because loop is off the player stays at the end of the take (state reads 'paused', position is 600).
js
import { TakePlayer } from'@openav/record';
import { Signals } from'@openav/core';
const take = { v: 1, name: 'loop', durationMs: 800, events: [
{ t: 0, name: 'midi/note/on', value: { note: 64, vel: 0.8, ch: 1 } },
{ t: 600, name: 'midi/note/off', value: { note: 64, ch: 1 } },
] };
const signals = new Signals();
const log = [];
signals.on('midi/note/on', (v) => log.push('note ' + v.note));
const player = new TakePlayer({ signals, take, yieldToLive: true, resumeAfter: 1, clock: 'manual',
onState: (state, reason) => log.push(state + (reason ? ' (' + reason + ')' : '')) });
player.play();
player.update(0.1);
signals.pulse('midi/note/on', { note: 72, vel: 1, ch: 1 }); // someone plays live: the take gives way
log.push('live');
for (let i = 0; i < 25; i++) player.update(0.05); // 1.25 s of silence: it comes back
console.log(log.join(' | '));
// playing | note 64 | note 72 | yielded (live) | live | playing (resume)
module A looping take gives way when a key is played live and returns one second after the playing stops. The log shows the order of events.
Related
TakeRecorderRecords what you play: it listens to every midi/* signal on a bus from start() to stop() and gives you a take.
EventLogKeeps a timestamped list of what happened during a take, such as notes and parameter changes, on the same clock as the recording.
fromMidiFile()Reads a Standard MIDI File into a take whose events are the signals a keyboard playing that file would publish.
noteOffFor()Gives the events that release a sounding note, in the same spelling it was played in.
SoundMakes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
new TakeRecorder({ signals, filter = /^midi\//, name = 'take', meta = {}, overdub = false, throttleMs = 0, now = defaultNow } = {})
Records what you play: it listens to every midi/* signal on a bus from start() to stop() and gives you a take.
A TakeRecorder is an EventLog that listens to midi/* on a bus. Press record: rec.start(). Play. Press stop: const take = rec.stop(). The take is plain data, so it can be saved as JSON, turned into a .mid (toMidiFile()) or played back (TakePlayer). It has every member of EventLog (mark(), between(), toJSON()…).
No stuck notes: a recorder stopped in the middle of a chord closes the keys that are still held (and a sustain pedal that is still down) in the take itself, at the stop time. The events are the signals as they were published, so a take from hardware holds both naming families (midi/note/on and midi/ch/1/note/60) and the device names, and playback publishes all of them. Pass recorder.t0 of a video Recorder to start() to line the take up with the video.
module Records four events from a bus on a clock moved by hand. The last note was still held, so the take closes it at the stop time; the audio/rms signal is not part of the take.
Related
TakePlayerPlays a take back into a bus at the right times, so the work, its sound and its chord detector cannot tell it from you playing.
EventLogKeeps a timestamped list of what happened during a take, such as notes and parameter changes, on the same clock as the recording.
toMidiFile()Writes a take as a Standard MIDI File (type 0) that opens in any DAW.
trimSilence()Cuts the silence before the first key press of a take, so the first note comes at the moment you choose.
RecorderRecords a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
Counts what is in a take: notes, events, length, the channels used, and the lowest and highest note.
Good for showing what a take is before it is played or saved. Notes are counted through midiEventsOf(), which picks one naming family, so the count is the number of keys pressed.
Parameters
Name
Type
Description
take
Object
The take.
Returns
Object{ notes, events, durationMs, channels, low, high }. notes counts key-downs, once each even when the take holds two spellings of them. channels is a sorted list of MIDI channels. low and high are MIDI note numbers, or null for a take without notes.
Writes a take as a Standard MIDI File (type 0) that opens in any DAW.
One track holds note on and off with velocity, controllers (the sustain pedal is CC 64), pitch bend, and channel and poly aftertouch, each on its own channel; channel 10 stays the drums. The take's name becomes the track name, and a 4/4 time signature is written only so a DAW's grid looks sane. Times are kept to the millisecond whatever the tempo, so the file plays like the take. Only one naming family of the take is used, so nothing doubles (midiEventsOf()). To save it, wrap it in a Blob of type audio/midi.
Parameters
Name
Type
Description
take
Object
The take.
ppq
Number
Ticks per quarter note. optionaldefault 960
bpm
Number
The tempo written in the file. Left out, it is take.meta.bpm, and then 120. optional
Cuts the silence before the first key press of a take, so the first note comes at the moment you choose.
Everything before the first key-down is cut away, but not lost: a controller value set during the silence, such as the sustain pedal or a knob, is kept and moved to the start (to t = 0 if it was before the cut), so the take still starts in the same state. If the take has no key-down at all, it is moved so its first event is at keep.
Parameters
Name
Type
Description
take
Object
The take.
keep
Number
Milliseconds of lead-in to keep before the first key-down. optionaldefault 0
Returns
Object A new take. Everything moved earlier by the same amount, and durationMs is shorter by that amount. A take that starts at once comes back as it is.
Checks a take and lists everything wrong with it as readable sentences. An empty list means it is valid.
It checks: v is TAKE_VERSION, name is a string, durationMs is a number from 0 up, events is an array, each event is an object with a signal name and a t of 0 or more, the events are in time order and none is after durationMs, and meta, if present, is an object. Use it on a take you read from a file; to repair one, use normalizeTake().
Parameters
Name
Type
Description
take
any
The take to check.
Returns
String[] Sentences such as v must be 1 (got 2) or events[3] is out of order (t 100 < 200). [] when valid. It never throws.
Examples
js
import { validateTake } from'@openav/record';
const good = { v: 1, name: 'riff', durationMs: 1500, events: [{ t: 500, name: 'midi/note/on', value: { note: 60 } }] };
console.log(validateTake(good)); // []
console.log(validateTake({ v: 2, events: [{ t: -1, name: '' }] }));
// ["v must be 1 (got 2)", "name must be a string", "durationMs must be a number ≥ 0",// "events[0].name must be a signal name", "events[0].t must be ms ≥ 0"]
console.log(validateTake(null)); // ["not an object"]
module A valid take gives an empty list; a broken one lists every problem; something that is not even an object says so.
Related
normalizeTake()Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.
TAKE_VERSIONThe version number of the take format, which is 1. Every take has v: 1.
The video MIME types the recorder tries, best first: MP4 with H.264 and AAC, then WebM.
MP4 comes first because it opens everywhere a performance video goes next (an editor, Instagram, a phone's gallery) without a remux, and Chrome encodes H.264 in hardware. The first string asks for a profile (avc1.640033, High profile level 5.1, enough for 4K at 30 and 1080 by 1920 at 60) because Chrome says no to the generic avc1,mp4a but yes to the qualified one, so both are in the list for other browsers. Plain video/mp4 is for Safari. pickMime() walks this list.
Type
String[]
A frozen list of 6 strings: video/mp4;codecs=avc1.640033,mp4a.40.2, video/mp4;codecs=avc1,mp4a, video/webm;codecs=vp9,opus, video/webm;codecs=vp8,opus, video/webm, video/mp4.
Whether an external MIDI clock (from a DAW, Strudel or a drum machine) moves drumPhase, drumBpm and the drum machine: follow or off.
With follow, an external MIDI clock (24 ticks a beat, plus start, stop and song position) is followed: drumPhase, beatPhase, drumBar, drumStep and drumBpm read it, and a drum machine that is up takes its tempo and its downbeat from it, starting and stopping with it. OAVp5.clock tells you what it hears. With nothing sending a clock, nothing changes.
off ignores every clock message: OAVp5.clock.info.state stays 'none'. Use it when something on the page or the machine sends a clock you do not want the sketch to follow.
Type
String
Values
follow | off
default
follow
Anything except off means follow.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { clock: … }; above the loader line wins, then ?oav_clock=… on the page address, then ?clock=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_clock=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { clock: 'off' };
function setup() {
createCanvas(400, 300);
OAVp5.input([0xFA]); // MIDI start
setInterval(() => OAVp5.input([0xF8]), 20); // clock ticks: 24 a beat, here 50 a second
}
function draw() {
background(20);
fill(255);
text('clock option: ' + OAVp5.options.clock, 20, 30);
text('external clock: ' + OAVp5.clock.info.state, 20, 60); // none: the clock is ignored
}
The sketch sends itself a made-up MIDI clock, and with clock: 'off' it is ignored: the state stays "none". Delete the first line and it reads "running".
Related
OAVp5.clockWhat oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
drumPhaseHow far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
beatPhaseHow far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
linkJoins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
Shows the 🎹 panel as a small box in the corner, a third of the window high, instead of a full-width bar: on or off.
Normally the panel opened by 🎹 is a bar across the whole window, up to half its height. With compact it is a box at most 560 pixels wide and a third of the window high, in the corner the 🎹 sits in, so the page beside it stays in view. It is the default on strudel.cc, where the code editor needs the room.
Type
String
Values
on | off
default
off (on in Strudel)
Only on turns it on. OAVp5.options.compact reads back as true or false.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { compact: … }; above the loader line wins, then ?oav_compact=… on the page address, then ?compact=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_compact=… (the project's README says OpenProcessing forwards it to the sketch).
Which side of the window the 🎹 button, its cards and the compact panel sit on: right or left.
Moves the 🎹 to the left or right edge, with the cards that appear beside it. scope() draws on the opposite side, so the two never meet. On openprocessing.org the default is left, because OpenProcessing's own icon sits at the top right.
Type
String
Values
right | left
default
right (left on OpenProcessing)
Anything except left means right.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { corner: … }; above the loader line wins, then ?oav_corner=… on the page address, then ?corner=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_corner=… (the project's README says OpenProcessing forwards it to the sketch).
Prints [oav-p5 debug] lines in the console about clicks, the 🎹 button and loading, for finding out why the 🎹 does not respond.
Two seconds after the 🎹 is placed, a line says whether the 🎹 in the page is oav-p5's own copy and where oav-p5 loads the rest of OAV from. Every click then adds a line with the pointer position, what was clicked, and what is on top at that spot, which is how you find a page element sitting over the 🎹. It also says when the panel opens, when the page replaced the 🎹 and which name asked for the microphone.
Type
String
Values
1 | 0
default
'' (off)
Any value except an empty one and 0 turns it on.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { debug: … }; above the loader line wins, then ?oav_debug=… on the page address, then ?debug=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_debug=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { debug: 1 };
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
fill(255);
text('open the console, then click the canvas', 20, 30); // [oav-p5 debug] lines appear
}
Open the console: two seconds after load a debug line talks about the 🎹, and every click on the canvas adds one saying what was under the pointer.
Related
hintsWhether oav-p5 shows its explaining cards beside the 🎹 (blocked microphone, no Web MIDI, an error in your callback): on or off.
panelWhether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
OAVp5.check()Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
Which edge of the window the 🎹 button and its cards sit on, bottom or top; set it and the panel starts on that edge too.
Moves the 🎹, its cards and the panel to the top or bottom edge. On openprocessing.org the default is top, because OpenProcessing's console floats over the bottom of the sketch. The drum machine's card sits below a panel docked at the top.
The panel itself has two buttons for whoever is looking at it: Dock cycles bottom → top → floating (drag a floating panel by its bar), and Size cycles small → medium → large. This browser remembers the choice, and it wins over the option. When the sketch sets no dock, the panel starts at the bottom, also on openprocessing.org (from 2026-10-20: at the top it covered the sketch). The 🎹 moves out of the panel's way only when the two share an edge. Reloading the same page (Cmd+Enter on OpenProcessing) opens the panel again if it was open, and restarts drums started from its 🥁 on the same line and tempo.
Type
String
Values
bottom | top
default
bottom (top on OpenProcessing)
Anything except top means bottom.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { dock: … }; above the loader line wins, then ?oav_dock=… on the page address, then ?dock=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_dock=… (the project's README says OpenProcessing forwards it to the sketch).
How long it takes a drum hit to be heard after oav-p5 hands it to the audio clock, so pictures land with the sound: auto or milliseconds.
The drum machine hands each hit to the audio clock slightly ahead of time, and tells the sketch (drumHit(), kickDetected(), env('kick')) when you will hear it, which is the scheduled moment plus the output latency. auto uses the latency the browser reports. If the browser's figure is smaller than what your output really takes, the picture runs ahead of the sound, and giving the number yourself fixes that. The project's README names a Bluetooth speaker as the case and suggests trying 200.
Type
String | Number
Values
auto | 0 to 500 (milliseconds)
default
auto
A number is held between 0 and 500. Anything that is not a number means auto. OAVp5.options.drumlatency reads back in seconds (0.2 for 200).
Notes
Only the default drum machine (drums=lookahead) uses it: the frame-driven one has nothing scheduled ahead.
Set it before oav-p5 loads. window.OAV_P5 = { drumlatency: … }; above the loader line wins, then ?oav_drumlatency=… on the page address, then ?drumlatency=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_drumlatency=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { drumlatency: 200 };
function setup() {
createCanvas(400, 300);
startDrums('bd*4, [~ sd]*2, hh*8', 108);
}
function draw() {
background(20);
fill(255);
text('drumlatency: ' + OAVp5.options.drumlatency + ' s', 20, 30);
circle(width / 2, height / 2, 50 + env('kick', 0.01, 0.3) * 200); // told 0.2 s after the hit is scheduled
}
The circle swells on each kick, told 200 milliseconds after the hit was handed to the audio clock, to match a speaker that is that late. Click the canvas once if you hear nothing.
To be confirmed
No Bluetooth speaker was tried: the suggested 200 comes from the project's README, not from a measurement made for this page.
Related
drumsHow the drum machine is clocked: lookahead (hits scheduled ahead on the audio clock) or frame (moved by animation frames, 0.5's way).
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
drumHit()An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
env()A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.
How the drum machine is clocked: lookahead (hits scheduled ahead on the audio clock) or frame (moved by animation frames, 0.5's way).
With lookahead, a heartbeat in a Web Worker hands the hits to the audio clock a little ahead of time, so the beat stays steady when frames stutter or the tab is busy, and the sketch is told when each hit is heard. With frame, the machine is moved by the animation frames instead, as it was in oav-p5 0.5. It is the safety net: where the lookahead machine misbehaves, frame brings the old behaviour back, and the timing is then only as steady as the frame rate.
Type
String
Values
lookahead | frame
default
lookahead
Anything except frame means lookahead.
Notes
OAVp5.drums.clock and the 🩺 check-up say which clock is running.
Set it before oav-p5 loads. window.OAV_P5 = { drums: … }; above the loader line wins, then ?oav_drums=… on the page address, then ?drums=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_drums=… (the project's README says OpenProcessing forwards it to the sketch).
The drum machine is moved by the animation frames, and the text says so. Click the canvas once if you hear nothing.
Related
drumlatencyHow long it takes a drum hit to be heard after oav-p5 hands it to the audio clock, so pictures land with the sound: auto or milliseconds.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
OAVp5.check()Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
When oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.
With auto, oav-p5 asks for Web MIDI as soon as it loads, so a plugged-in controller plays the sketch without anyone pressing anything. The browser remembers your answer per site. With click, it asks only when the 🎹 panel is first opened. With off, it never asks by itself, and the on-screen controller and a phone still play the sketch.
Only auto and click ask. Any other value acts like off.
Notes
On strudel.cc (or a page where Strudel's own functions are present) the default is click, because Strudel's midin() asks for MIDI itself. Setting hardware yourself wins.
Safari and iPad have no Web MIDI at all: when oav-p5 asks (or you call OAVp5.connectHardware()), the state there becomes 'unavailable'.
Set it before oav-p5 loads. window.OAV_P5 = { hardware: … }; above the loader line wins, then ?oav_hardware=… on the page address, then ?hardware=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_hardware=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { hardware: 'click' };
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
fill(255);
text('hardware option: ' + OAVp5.options.hardware, 20, 30);
text('Web MIDI state: ' + OAVp5.hardware, 20, 60); // stays "off" until you open the 🎹 panel
}
The page does not ask for Web MIDI when it loads, and the state stays "off". Press 🎹: that opens the panel and asks. Delete the first line and it asks at once.
Related
OAVp5.hardwareWhere Web MIDI stands: 'off', 'asking', 'connected', 'waiting', 'denied' or 'unavailable'.
OAVp5.connectHardware()Asks the browser for Web MIDI now, and returns a promise that says whether a real controller can be heard.
OAVp5.devicesThe names of the MIDI controllers plugged in, as the browser reports them: an array of text, empty when there is none.
panelWhether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
strudelWhether oav-p5 treats the page as Strudel, the live-coding editor, and behaves politely beside it: auto, on or off.
Whether oav-p5 shows its explaining cards beside the 🎹 (blocked microphone, no Web MIDI, an error in your callback): on or off.
When something goes wrong, a small card beside the 🎹 says what happened, why, and what to press, in the browser's language, once per page. hints: 'off' takes the cards off the page: the console still says the same thing and OAVp5.hints still lists what was said. Use it for a finished piece that goes on a wall or into a recording.
It also hides the corner oscilloscope of scope(), which is part of the same on-page help. scope=off hides only that.
Type
String
Values
on | off
default
on
Only off turns the cards off. Any other value, including the text false, leaves them on.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { hints: … }; above the loader line wins, then ?oav_hints=… on the page address, then ?hints=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_hints=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { hints: 'off' };
function setup() {
createCanvas(400, 300);
lag(1); // an unnamed lag() outside draw(): oav-p5 has something to say about that
}
function draw() {
background(20);
fill(255);
text('cards said: ' + OAVp5.hints.length + ' (none on the page)', 20, 30);
text(OAVp5.hints.map((h) => h.what).join('\n'), 20, 60);
}
oav-p5 notices the lag() in setup() and would normally show a card. With hints off nothing appears on the page, but the console and the list in OAVp5.hints still have it.
window.OAV_P5 = { hints: false }; // not 'off': the value is turned into the text "false"function setup() {
createCanvas(400, 300);
lag(1);
}
function draw() {
background(20);
fill(255);
text('the card is still beside the 🎹', 20, 30);
}
The gotcha: false is not 'off', so the card still appears.
Related
OAVp5.hintsThe list of cards oav-p5 has shown or would have shown so far (a blocked microphone, no Web MIDI, an error in your callback), each with what, why and what to press.
scopeWhether scope() draws its oscilloscope in the corner of the page: on or off. hints=off turns it off too.
debugPrints [oav-p5 debug] lines in the console about clicks, the 🎹 button and loading, for finding out why the 🎹 does not respond.
OAVp5.check()Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
Joins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
Every oav-p5 page with the same room (another tab, another computer) gets what this page's programs play into the virtual MIDI output (notes, controllers and the clock), over the phone relay, and plays it at the moment it sounds here. On the other side, OAVp5.last.source is 'link'. Nothing it receives is sent on, so two pages cannot loop. The room is also the phones' room, so one QR code plays every linked page.
OAVp5.link.info tells you the room, whether it is connected and how many other pages were heard. The 🎹 grows a 🔗: green when another page is there, yellow while connecting or alone, red when the relay is gone. linkdelay adds a buffer.
Type
String
Values
a room name
default
'' (no room)
Only letters, digits, _ and - are kept, and only the first 32 characters.
Before it has a value
A network connection to the relay (relay, wss://remote.cheyuwu.com/relay unless you set another). Without it the 🔗 is red and nothing is shared.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { link: … }; above the loader line wins, then ?oav_link=… on the page address, then ?link=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_link=… (the project's README says OpenProcessing forwards it to the sketch).
static Open the same address in two tabs: each shows the room and, once connected, one other page. Anything one tab's program plays into OAV Virtual MIDI is heard by the other.
To be confirmed
The headless check cannot reach a relay, so this example is not run. Behaviour between two pages was read from the code and the project's README, not seen here.
Related
OAVp5.linkWhat the link to other oav-p5 pages in the same room is doing: only OAVp5.link.info, and null unless a room is set.
OAVp5.link.infoThe state of the link: its room, state (connecting, open, closed), how many other pages are there, and the trip time. null when no room is set.
linkdelayExtra milliseconds of buffer on the notes a linked page receives, to smooth out an uneven network: 0 to 2000.
relayThe address of the relay that phones and link rooms talk through: a WebSocket URL.
roomThe name of the room phones join to play this page: by default one made up per browser.
Extra milliseconds of buffer on the notes a linked page receives, to smooth out an uneven network: 0 to 2000.
Only matters together with link: without a room nothing is received. Each note arriving from another page is played at the moment it sounded there, which already allows for the trip. linkdelay adds this much on top, so a note that took longer than usual is still on time. The price is that everything from the other page arrives that much later.
Type
Number
Values
0 to 2000 (milliseconds)
default
0
Held between 0 and 2000. Anything that is not a number is 0.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { linkdelay: … }; above the loader line wins, then ?oav_linkdelay=… on the page address, then ?linkdelay=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_linkdelay=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { linkdelay: 80 };
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
fill(255);
text('linkdelay: ' + OAVp5.options.linkdelay + ' ms', 20, 30);
text('link: ' + (OAVp5.link.info ? 'room ' + OAVp5.link.info.room : 'no room, so nothing to delay'), 20, 60); // add link: 'my-room' to use it
}
Without a link room the number is only stored: the sketch says there is nothing to delay. Add link: 'my-room' to the options and it applies to every note that arrives.
Related
linkJoins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
OAVp5.link.infoThe state of the link: its room, state (connecting, open, closed), how many other pages are there, and the trip time. null when no room is set.
relayThe address of the relay that phones and link rooms talk through: a WebSocket URL.
Who may turn the microphone on: auto (the first read of micLevel and friends), click (only 🎤 or startMic()) or off (nobody).
With auto, the first time a sketch reads micLevel, bass, mid, treble or beat, oav-p5 asks the browser for the microphone. With click, reading them does nothing: the microphone starts only when someone presses 🎤 in the 🎹 panel or the sketch calls startMic(), so the permission prompt arrives when you choose. With off, nothing opens the microphone, not even startMic(), which resolves false.
off is a promise to the page: a gallery piece that must never listen can say so. click is for pieces where a permission prompt out of nowhere would be rude, or where the sound only matters after an intro. The values of the microphone stay 0 while it is off.
Type
String
Values
auto | click | off
default
auto
Anything else means auto.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { mic: … }; above the loader line wins, then ?oav_mic=… on the page address, then ?mic=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_mic=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { mic: 'click' };
function setup() {
createCanvas(400, 300);
}
function mousePressed() {
startMic(); // now the browser asks
}
function draw() {
background(20);
circle(width / 2, height / 2, 50 + micLevel * 300); // reading micLevel no longer asks
fill(255);
text('microphone: ' + OAVp5.mic.state + ' (click the canvas to start it)', 20, 30);
}
Reading micLevel does not ask for the microphone: the state stays "off" and the circle stays small. Click the canvas and the browser asks. Asks for the microphone
Whether oav-p5 wires the controller to the variables of OP Configurator (OpenProcessing's sliders and buttons): on or off.
On OpenProcessing, OPC.slider('radius', 10) declares a variable with a slider. With opc on, oav-p5 then wires the controller to the OPC variables in order: knob 1 to the first slider, knob 2 to the second, the faders after the knobs, pad 1 to the first toggle or button. OPC's own panel moves with the knob. OAVp5.wiring() lists who drives what.
off leaves OPC's variables to their own sliders. A control the sketch reads itself (knob1 in draw()) is the sketch's anyway: the slider at that place stops following it.
Type
String
Values
on | off
default
on
Only off turns it off.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { opc: … }; above the loader line wins, then ?oav_opc=… on the page address, then ?opc=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_opc=… (the project's README says OpenProcessing forwards it to the sketch).
Outside OpenProcessing there is no OPC, so the count is 0 either way. On OpenProcessing with sliders declared, it is 0 with opc: 'off' and the number of sliders and buttons without it.
To be confirmed
Not tried with OpenProcessing's real OPC library. The wiring was checked against a stand-in object with the same options shape, and the example here runs where no OPC exists, so it only shows the option arriving.
Related
OAVp5.wiring()Lists which controller knob, fader or pad drives which OpenProcessing OPC variable right now: [{ control: 'knob1', name: 'radius' }, …].
knob1The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
knob()Returns the position of knob number n, from 0 to 1: the same as knob1 to knob8, for any n.
pad1How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.
Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
closed shows just the small 🎹 button, and pressing it opens the panel: the on-screen controller, the buttons for 🎤, 🔊, 🥁, ⏺, 📷, 📱 and 🩺. open opens the panel as soon as the page loads, for a workshop where everyone needs the on-screen controller at once. off removes the 🎹 button altogether, for a piece in an exhibition: there is no button and OAVp5.panel.open() does nothing. The cards that explain a problem still appear in the corner, unless hints is off too.
Type
String
Values
closed | open | off
default
closed
Any value except open and off means closed.
Notes
With off everything the panel does is still there as code: startMic(), startSound(), startRecording() and the options keep working, and the console still says what goes wrong.
Set it before oav-p5 loads. window.OAV_P5 = { panel: … }; above the loader line wins, then ?oav_panel=… on the page address, then ?panel=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_panel=… (the project's README says OpenProcessing forwards it to the sketch).
Which controller the on-screen 🎹 shows, by profile id such as akai-lpd8; by default it follows the one that is plugged in.
The on-screen controller of the 🎹 panel draws this device, and knob1, fader1 and pad1 mean the first knob, fader and pad printed on it. A sketch written with the LPD8 in mind, for instance, can ask for it with profile: 'akai-lpd8' so that whoever has no LPD8 still gets one on screen. The picker inside the panel changes it by hand.
If you do not set it, the panel starts as the MiniLab 3 and then follows a real controller as soon as one that OAV has a profile for is plugged in. If you do set it, that choice stays. An id that does not exist is reported in the console with the list of known ids, and the panel shows no controller.
Type
String
Values
a profile id
default
arturia-minilab3
Any id that @openav/midi has a profile for: arturia-minilab3, akai-lpd8, korg-nanokontrol2, generic-keys49 and the rest. When you set nothing, the panel follows the controller that is plugged in.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { profile: … }; above the loader line wins, then ?oav_profile=… on the page address, then ?profile=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_profile=… (the project's README says OpenProcessing forwards it to the sketch).
The frame rate of recorded video, 1 to 60 frames a second.
How many frames a second the recorder takes from the canvas, for startRecording() and the ⏺ button. 30 suits most places Instagram and YouTube accept. 60 is smoother for fast motion and makes a bigger file. One take can ask for its own rate: startRecording({ fps: 60 }).
Type
Number
Values
1 to 60
default
30
Rounded to a whole number and held between 1 and 60. Anything that is not a number means 30.
Notes
It is read back as OAVp5.options.recfps and as OAVp5.record.fps.
Set it before oav-p5 loads. window.OAV_P5 = { recfps: … }; above the loader line wins, then ?oav_recfps=… on the page address, then ?recfps=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_recfps=… (the project's README says OpenProcessing forwards it to the sketch).
The longest one recording may last, in seconds: a take stops by itself there.
A long take fills the tab's memory, so every recording stops by itself at this length, saves what it has and shows a card saying so. Raise it for a long piece, record in parts, or give one take its own length: startRecording(10) stops after ten seconds, and a number above the limit is cut down to the limit.
Type
Number
Values
1 or more (seconds)
default
300
At least 1. Anything that is not a number means 300 (five minutes).
Notes
It is read back as OAVp5.options.reclimit and as OAVp5.record.limit.
Set it before oav-p5 loads. window.OAV_P5 = { reclimit: … }; above the loader line wins, then ?oav_reclimit=… on the page address, then ?reclimit=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_reclimit=… (the project's README says OpenProcessing forwards it to the sketch).
Whether recording is available and whether it includes the microphone: on, nomic or off.
With on, ⏺ and 📷 are in the 🎹 panel and startRecording() records the canvas together with the sound the sketch makes, the microphone included when it is on. nomic keeps the microphone out of the recording. With it in and no headphones, the speakers are recorded a second time through the microphone, so leave it out when you play sound out loud. off takes ⏺ and 📷 away and makes startRecording() do nothing.
off is a promise to the page, like mic=off: a piece that must not be captured can say so. One take can still choose its own: startRecording({ mic: false }).
Type
String
Values
on | nomic | off
default
on
Anything else means on.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { record: … }; above the loader line wins, then ?oav_record=… on the page address, then ?record=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_record=… (the project's README says OpenProcessing forwards it to the sketch).
The address of the relay that phones and link rooms talk through: a WebSocket URL.
The 📱 button shows a QR code that sends a phone to the remote page, and the phone's MIDI tab plays your sketch through this relay. link rooms use it too. The default is the project's public relay. To keep your traffic on your own machine, run the relay from a checkout of the repository with node packages/relay/server.js and point relay at ws://localhost:7458/relay.
Type
String (URL)
Values
a ws:// or wss:// address
default
wss://remote.cheyuwu.com/relay
Used as given.
Notes
Nothing connects to it at load. It is used when you open the 📱 card (OAVp5.phone()) or set link.
Set it before oav-p5 loads. window.OAV_P5 = { relay: … }; above the loader line wins, then ?oav_relay=… on the page address, then ?relay=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_relay=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { relay: 'ws://localhost:7458/relay' };
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
fill(255);
text('relay: ' + OAVp5.options.relay, 20, 30); // used when you open 📱 or set link
}
The sketch only prints the address in use: a phone or a linked page would now connect to a relay on your own computer.
Related
roomThe name of the room phones join to play this page: by default one made up per browser.
remoteThe address of the remote page that the 📱 QR code opens on a phone.
linkJoins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
OAVp5.phone()Shows the 📱 card: a QR code that sends a phone to the remote page, whose MIDI tab then plays this sketch. Returns the room name.
The address of the remote page that the 📱 QR code opens on a phone.
The phone's page is where the on-screen controller on the phone lives. The QR code and the link under it are this address plus the relay, the room and the MIDI tab. Change it when you host the remote page yourself, for example on a local network where the phone cannot reach the public site.
Type
String (URL)
Values
an https:// address
default
https://openaudiovisual.com/packages/remote/
Used as given, with ?relay=…&room=…&tab=midi added.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { remote: … }; above the loader line wins, then ?oav_remote=… on the page address, then ?remote=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_remote=… (the project's README says OpenProcessing forwards it to the sketch).
The name of the room phones join to play this page: by default one made up per browser.
A phone and a page meet in a room on the relay. Left empty, oav-p5 picks a name like p5-k3x9a the first time and keeps it in this browser (localStorage), so the same QR code keeps working on reload. If link is set, its room is used. Set room yourself when you want a name people can type or remember, such as one per class.
A phone needs only the room name to join, so pick a name people cannot guess if that matters. OAVp5.phone() returns the room in use.
Type
String
Values
a room name
default
'' (random per browser)
Left empty, a name like p5-k3x9a is made up and kept in this browser. Only letters, digits, _ and - are kept, and only the first 32 characters. If nothing is left, the room is default.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { room: … }; above the loader line wins, then ?oav_room=… on the page address, then ?room=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_room=… (the project's README says OpenProcessing forwards it to the sketch).
Whether scope() draws its oscilloscope in the corner of the page: on or off. hints=off turns it off too.
With off, scope() draws nothing in the corner and only returns its value, so a sketch can leave its scope() calls in and hide the plot for a show. drawTrail(), which draws on the canvas, is not affected.
Type
String
Values
on | off
default
on
Only off turns it off.
Notes
If the plot does not show up, check hints too: hints=off hides it as well.
Set it before oav-p5 loads. window.OAV_P5 = { scope: … }; above the loader line wins, then ?oav_scope=… on the page address, then ?scope=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_scope=… (the project's README says OpenProcessing forwards it to the sketch).
The circle still breathes, but there is no plot in the corner of the page. Delete the first line and it comes back.
Related
scope()Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
drawTrail()Draws a line through values on the canvas, so it is in the work and in a recording: the oldest at x, lo at the bottom of the box, hi at the top.
hintsWhether oav-p5 shows its explaining cards beside the 🎹 (blocked microphone, no Web MIDI, an error in your callback): on or off.
Whether oav-p5 replaces navigator.requestMIDIAccess so sketches with their own Web MIDI code also hear the 🎹 and phones: on or off.
With on, a sketch that calls navigator.requestMIDIAccess() itself (WebMidi.js does) gets the real ports and one more input called "OAV Virtual MIDI", which carries whatever the on-screen controller and the phone play. It also works in Safari and on iPad, where there is no Web MIDI: the sketch gets the virtual port alone, so a WebMidi.js sketch still runs there. Programs can play into its output as well (Strudel's .midi('OAV Virtual MIDI')).
off leaves the browser's own requestMIDIAccess alone. Use it when a page already has its own MIDI handling that must not see an extra port.
Type
String
Values
on | off
default
on
Only off turns it off.
Notes
Order matters: oav-p5 has to load before the code that asks for MIDI, which is why the loader line goes first. A page that opened MIDI earlier keeps the browser's own answer, and 🩺 says so.
Set it before oav-p5 loads. window.OAV_P5 = { shim: … }; above the loader line wins, then ?oav_shim=… on the page address, then ?shim=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_shim=… (the project's README says OpenProcessing forwards it to the sketch).
With the shim off, the sketch gets only the browser's own answer (the real inputs, or a refusal). Delete the first line and "OAV Virtual MIDI" is in the list. The browser may ask you for MIDI permission.
Related
OAVp5.input()Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
hardwareWhen oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.
midiMessage()You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
OAVp5.lastThe latest note, control change or pitch bend that arrived: its type, number, value, channel, where it came from and which control it is. null before any.
Whether notes are heard: click (only after startSound() or 🔊), off (never), or an instrument id that starts at the first click.
With click, nothing makes a sound until the sketch calls startSound() or someone presses 🔊 in the 🎹 panel. With an instrument id, every key, pad, the on-screen controller and a phone are heard from the first click on the page, with no code at all, so any MIDI sketch becomes playable by adding the option to the page address: ?oav_sound=piano. off is a promise to the page: nothing sounds, startSound() resolves false and the 🔊 button is hidden.
Browsers let a page play sound only after a click, and a MIDI key is not a click, so the first sound waits for one: a card says "sound is ready but silent" until then.
Type
String
Values
click | off | an instrument id
default
click
An instrument id such as piano, organ, strings, harp or music-box. on and auto start the default instrument (the piano). An id that does not exist plays the default synth (synth/pad) and the console lists the ids that exist.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { sound: … }; above the loader line wins, then ?oav_sound=… on the page address, then ?sound=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_sound=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { sound: 'organ' };
function setup() {
createCanvas(400, 300);
}
function mousePressed() {
OAVp5.play(60 + floor(random(12)), 0.8); // a key, as a controller would play it
}
function draw() {
background(20);
fill(255);
text('sound: ' + OAVp5.sound.state + ' (' + OAVp5.sound.instrument + ')', 20, 30);
text('click the canvas to play a note', 20, 60);
}
No startSound() call: the organ is heard from your first click, which also plays a note. The state goes from "waiting" to "on".
Whether oav-p5 treats the page as Strudel, the live-coding editor, and behaves politely beside it: auto, on or off.
Strudel is a live-coding editor, not a sketch, so a few things change when oav-p5 sees it: the names Strudel owns (note, velocity, beat) are left alone and one card says what to read instead, MIDI is asked for by Strudel's own midin() rather than at load (hardware becomes click), the 🎹 panel becomes the compact box, and the 🎹 sits above Strudel's side-panel buttons. oav-p5 only looks at the address and at the names on the page: nothing of Strudel is called.
on forces this behaviour on any page, off forces the plain one even on strudel.cc. Settings you give yourself (hardware, compact) still win over what Strudel mode would choose.
Type
String
Values
auto | on | off
default
auto
auto means on strudel.cc, or on a page where Strudel's own functions (note() and s() with stack() or samples()) are already there. Anything else means auto.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { strudel: … }; above the loader line wins, then ?oav_strudel=… on the page address, then ?strudel=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_strudel=… (the project's README says OpenProcessing forwards it to the sketch).
On an ordinary page, strudel on switches to Strudel's manners: Web MIDI is not asked for at load and the panel will be the compact box.
Related
hardwareWhen oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.
compactShows the 🎹 panel as a small box in the corner, a third of the window high, instead of a full-width bar: on or off.
linkJoins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
clockWhether an external MIDI clock (from a DAW, Strudel or a drum machine) moves drumPhase, drumBpm and the drum machine: follow or off.
OAVp5.clockWhat oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
Whether the on-screen controller may send one anonymous page-load count to the project: off or on.
In oav-p5 it is off. With on, when the 🎹 panel builds the on-screen controller, @openav/midi sends one anonymous hit per page load to the project's Google Analytics, so the project can see where its controllers end up. The @openav/midi README (section Telemetry) lists exactly what it sends: the kind of embed, the profile, the page's origin (never the path, query or title) and the version. Nothing anyone plays is sent and no cookie is set.
Type
String
Values
off | on
default
off
Only on turns it on.
Notes
According to that README nothing is sent from localhost, local network addresses, file:// or openaudiovisual.com itself, or when the browser sends Do Not Track or Global Privacy Control.
Set it before oav-p5 loads. window.OAV_P5 = { telemetry: … }; above the loader line wins, then ?oav_telemetry=… on the page address, then ?telemetry=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_telemetry=… (the project's README says OpenProcessing forwards it to the sketch).
A name that goes into the file name of recordings and images, as in oav-my-piece-20261015-183012.mp4.
Without it a recording is oav-20261015-183012.mp4. With title: 'my-piece' it is oav-my-piece-20261015-183012.mp4, so takes from several pieces sort themselves. One take can have its own name: startRecording({ name: 'take-2' }) replaces it.
Type
String
Values
a name
default
'' (no name)
Spaces and the characters a file name cannot hold become -, and only the first 40 characters are used.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { title: … }; above the loader line wins, then ?oav_title=… on the page address, then ?title=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_title=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { title: 'my-piece' };
let saved = 'press r to record 2 seconds';
function setup() {
createCanvas(400, 300);
}
function keyPressed() {
if (key === 'r') startRecording(2);
}
function recordingSaved(info) {
saved = 'saved as ' + info.name;
}
function draw() {
background(20);
fill(255);
text(saved, 20, 30);
}
Press r: two seconds are recorded and downloaded, and the sketch shows the file name, which starts with oav-my-piece-.
Related
startRecording()Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
recordingSaved()An event you define: it is called when a recording has been saved and checked, with an info object about the file.
recordWhether recording is available and whether it includes the microphone: on, nomic or off.
Where Tone.js, the sound engine behind startSound(), is loaded from: an ES module address. Empty means jsDelivr.
Sound (startSound(), the sound option, the 🔊 button) uses Tone.js 15, which oav-p5 loads the first time sound is wanted. By default it comes from jsDelivr. Point tone at your own copy when the page is behind a school firewall or an ad blocker that stops the CDN, or when you serve everything yourself. If it cannot be loaded, a card says which host was unreachable and what to try.
Type
String (URL)
Values
an ES module URL
default
'' (jsDelivr)
The address of Tone.js version 15 as an ES module. Left empty, it is https://cdn.jsdelivr.net/npm/tone@15.0.4/+esm, and OAVp5.options.tone reads ''.
Notes
Set it before oav-p5 loads. window.OAV_P5 = { tone: … }; above the loader line wins, then ?oav_tone=… on the page address, then ?tone=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_tone=… (the project's README says OpenProcessing forwards it to the sketch).
window.OAV_P5 = { tone: 'https://cdn.jsdelivr.net/npm/tone@15.0.4/+esm' };
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
fill(255);
text('Tone.js from: ' + OAVp5.options.tone, 20, 30); // the address here is the built-in one
}
Prints where Tone.js would come from. The address here is the built-in one, so nothing changes. Swap in your own copy to host it yourself.
Related
soundWhether notes are heard: click (only after startSound() or 🔊), off (never), or an instrument id that starts at the first click.
startSound()Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
soundIsOnWhether notes are being heard right now: true once startSound() has finished, false before and after.
Three names, one object. OAVp5 is always there. oavMidi is always there too. midi is there unless the page already had a midi of its own, in which case it is left alone and the console says to read OAVp5 instead. OAVp5.options holds the loading options as oav-p5 read them: OAVp5.options.mic, and so on for each of the loading options (for example mic).
Before it has a value
Only the loader line. If it has not loaded, OAVp5 is not defined, and a sketch that reads it stops with "OAVp5 is not defined".
Notes
midi and oavMidi are the same object under other names.
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
fill(255);
text('same object: ' + (OAVp5 === midi && OAVp5 === oavMidi), 20, 30);
text('version: ' + OAVp5.version, 20, 60);
text('MIDI hardware: ' + OAVp5.hardware, 20, 90);
text('microphone: ' + OAVp5.mic.state, 20, 120); // reading the state does not start it
text('external clock: ' + OAVp5.clock.info.state, 20, 150);
}
A small status board: the three names are one object, then the version and the state of MIDI, the microphone and the clock.
Related
OAVp5.versionThe version of oav-p5 that loaded, as text such as '0.8.0'.
OAVp5.on()Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
OAVp5.input()Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
OAVp5.check()Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
OAVp5.micThe microphone under the plain names: its state, the numbers of micLevel and bass that never turn it on when read, the raw loudness, start, stop and events.
OAVp5.clockWhat oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
OAVp5.chopThe CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
The same thing the 🩺 button in the panel does. When something does not work for a student, OAVp5.check() in the console (or 🩺) gives a card that says what this browser did: whether the page is in an iframe, whether the microphone is on or blocked, whether Web MIDI exists and which devices it found, the last message, the sound and drum state, whether MP4 recording works, what the controller is wired to, which of the sketch's own names stepped in the way, and which cards have appeared. It has a copy button and a link to the site's full device check.
Parameters
Name
Type
Description
opts
Object
{ show: false } skips the card on the page and only returns the report (the text still goes to the console). Left out, the card is shown. optional
Returns
Object The report: { version, time, page, browser, microphone, midi, sound, drums, wiring, own, hints, external, recording, text }. text is the whole card as plain lines, the other fields are the same facts as data (microphone.state, midi.hardware, midi.devices …).
Notes
It is a snapshot taken when you call it, not a live view. Calling it again replaces the card.
let report;
function setup() {
createCanvas(400, 300);
report = OAVp5.check({ show: false }); // the report, without the card on the page
}
function draw() {
background(20);
fill(255);
textSize(10);
text(report.text, 10, 10, width - 20, height - 20);
}
Prints the check-up text on the canvas: version, browser, microphone, MIDI and the rest, one line each.
function setup() {
createCanvas(400, 300);
}
function mousePressed() {
OAVp5.check(); // the card appears over the page
}
function draw() {
background(20);
fill(255);
text('click the canvas for the check-up card', 20, 30);
}
Click the canvas: the check-up card opens over the page, with a copy button, ready to screenshot.
Related
OAVp5.hintsThe list of cards oav-p5 has shown or would have shown so far (a blocked microphone, no Web MIDI, an error in your callback), each with what, why and what to press.
OAVp5.versionThe version of oav-p5 that loaded, as text such as '0.8.0'.
debugPrints [oav-p5 debug] lines in the console about clicks, the 🎹 button and loading, for finding out why the 🎹 does not respond.
OAVp5.mic.stateWhere the microphone is: 'off', 'asking', 'on', 'denied' or 'unavailable'.
OAVp5.hardwareWhere Web MIDI stands: 'off', 'asking', 'connected', 'waiting', 'denied' or 'unavailable'.
What oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
Some programs keep the time and everyone else follows: Strudel (midicmd("clock*96, <start stop>/8")), a DAW over a virtual cable, a drum machine on a MIDI cable. They send 24 clock messages per beat, plus start, continue, stop and song position. oav-p5 counts them: OAVp5.clock.info says whether a clock is running, OAVp5.clock.bpm its tempo and OAVp5.clock.phase how far through the bar and the beat it is now.
The first clock message loads the counting code (@openav/midi's MidiClock) from next to oav-p5.js, which takes a few frames. Until then the members read none, null and null.
This is advanced: most sketches only read drumPhase and let it follow a clock.
function setup() {
createCanvas(400, 300);
OAVp5.input([0xFA]); // MIDI start
setInterval(() => OAVp5.input([0xF8]), 20); // clock ticks: 24 a beat, here 50 a second (125 BPM)
}
function draw() {
background(20);
fill(255);
text('clock: ' + OAVp5.clock.info.state, 20, 30);
text('tempo: ' + (OAVp5.clock.bpm ? nf(OAVp5.clock.bpm, 1, 1) : '-') + ' BPM', 20, 60);
const p = OAVp5.clock.phase;
if (p) circle(40 + p.beat * (width - 80), 150, 30); // sweeps across once a beat
}
The sketch sends itself a made-up MIDI clock at about 125 BPM. A circle sweeps across once per beat, and the state and tempo are printed.
Related
OAVp5.clock.infoA snapshot of the external clock: state (none, running, stopped or lost), bpm, whether the drums follow it, where it came from, and counts.
OAVp5.clock.bpmThe tempo of the external MIDI clock in beats a minute, or null while there is none.
OAVp5.clock.phaseWhere the external clock is right now: { pos, bar, beat, bpm, running }, or null while there is no tempo.
clockWhether an external MIDI clock (from a DAW, Strudel or a drum machine) moves drumPhase, drumBpm and the drum machine: follow or off.
drumPhaseHow far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
beatPhaseHow far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
The tempo of the external MIDI clock in beats a minute, or null while there is none.
The raw number, not rounded. Use it for anything that should run at the clock's tempo: OAVp5.clock.bpm / 60 is beats a second. If the drum machine follows the clock, drumBpm shows it rounded to a tenth.
Type
Number | null
Fitted over the last 24 clock ticks, so it settles within a moment and does not jump with one late tick. null before two ticks have arrived or when clock=off.
function setup() {
createCanvas(400, 300);
OAVp5.input([0xFA]); // MIDI start
setInterval(() => OAVp5.input([0xF8]), 20); // clock ticks: 24 a beat, here 50 a second (125 BPM)
}
function draw() {
background(20);
fill(255);
textSize(48);
textAlign(CENTER, CENTER);
const bpm = OAVp5.clock.bpm;
text(bpm ? nf(bpm, 1, 1) + ' BPM' : 'no clock', width / 2, height / 2);
}
Prints the tempo of the made-up clock the sketch sends itself: about 125 BPM once a few ticks have arrived.
Related
OAVp5.clockWhat oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
OAVp5.clock.infoA snapshot of the external clock: state (none, running, stopped or lost), bpm, whether the drums follow it, where it came from, and counts.
OAVp5.clock.phaseWhere the external clock is right now: { pos, bar, beat, bpm, running }, or null while there is no tempo.
drumBpmThe tempo of the drum machine in beats per minute.
A snapshot of the external clock: state (none, running, stopped or lost), bpm, whether the drums follow it, where it came from, and counts.
state is 'none' until a clock message has come, 'running' while the clock is going, 'stopped' after a MIDI stop (the position holds and clocks may still arrive), and 'lost' when no clock message has arrived for 0.6 seconds. bpm is the tempo fitted over the last 24 ticks, rounded to two decimals, or null before two ticks. running is true while the transport runs, which a stream of clocks with no start message counts as too (implicit is true then).
following is true when the drums read this clock: it is running, has a tempo and clock=off is not set. source says where the messages came from: 'hardware', 'screen', 'phone', 'virtual-out' (a program playing into OAV Virtual MIDI) or 'link'. clocks, dupes, starts, stops and continues count what arrived; a start sent three times with one timestamp counts once and adds to dupes.
Type
Object
A new object each time you read it: { state, bpm, running, following, source, implicit, clocks, dupes, starts, stops, continues }. Before any clock message it is { state: 'none', bpm: null, running: false, following: false, source: null, clocks: 0 }.
Prints the clock's snapshot while the sketch sends itself a made-up clock: the state goes from "none" to "running" within a moment.
Related
OAVp5.clockWhat oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
OAVp5.clock.bpmThe tempo of the external MIDI clock in beats a minute, or null while there is none.
OAVp5.clock.phaseWhere the external clock is right now: { pos, bar, beat, bpm, running }, or null while there is no tempo.
clockWhether an external MIDI clock (from a DAW, Strudel or a drum machine) moves drumPhase, drumBpm and the drum machine: follow or off.
Where the external clock is right now: { pos, bar, beat, bpm, running }, or null while there is no tempo.
A bar is four beats. bar and beat count up from 0 and fall back to 0 at the next downbeat, the same way drumPhase and beatPhase do. After a stop the position holds. A clock with no start message gets its downbeat from the first tick after a silence of half a second, so a phase is there anyway.
Type
Object | null
A new object each time you read it: pos is the number of bars since the start (0 is the downbeat of bar 1), bar and beat are 0..1 through the current bar and beat, bpm the tempo and running whether the transport is moving. null until a tempo is known.
function setup() {
createCanvas(400, 300);
OAVp5.input([0xFA]); // MIDI start
setInterval(() => OAVp5.input([0xF8]), 20); // clock ticks: 24 a beat, here 50 a second (125 BPM)
}
function draw() {
background(20);
const p = OAVp5.clock.phase;
if (!p) return;
noStroke();
fill(255, 211, 107);
circle(40 + p.bar * (width - 80), 100, 30); // once a bar (four beats)
fill(120, 190, 255);
circle(40 + p.beat * (width - 80), 200, 30); // once a beat
}
Two circles sweep from left to right on the made-up clock: the blue one once per beat, the yellow one once per bar of four beats.
Related
OAVp5.clockWhat oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
OAVp5.clock.infoA snapshot of the external clock: state (none, running, stopped or lost), bpm, whether the drums follow it, where it came from, and counts.
drumPhaseHow far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
beatPhaseHow far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
How the drum machine keeps time: which clock moves it, how far ahead it schedules and how long the callbacks wait for the speakers.
This is the diagnostic behind the 🩺 check-up. When drum hits feel late or early, look here first: the handbook puts a Bluetooth speaker at 0.15 to 0.25 seconds of latency, and ?oav_drumlatency=200 sets it by hand. late, skipped, warning and why are further notes the check-up reads.
A new object on every read. clock is 'lookahead' (hits are handed to the audio clock ahead of time, the default) or 'frame' (moved by the frames, the drums option). heartbeat is 'worker' (a Web Worker wakes it every 25 ms), 'interval' (the fallback), 'frame' with the frame clock, or 'none' before the machine exists. lookahead is the seconds scheduled ahead and extended is true when a slow draw() made it look further ahead. latency is the seconds the callbacks wait for the output of the speakers, audio whether the kit's audio clock is running, and preDraw the p5 hook that delivers the callbacks right before draw().
What exactly late, skipped, warning and why count was not checked.
Related
OAVp5.drumsThe object behind the drum names: the state of the machine, its counters, the line of rhythm, its history and the loaded machine.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
drumPhaseHow far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
The drum machine from @openav/drums that the grid on the page belongs to, or null before it has loaded.
It is what mountDrums() of @openav/drums returns, with seq (the sequencer), engine (the kit), play(on), setPattern(p), setBpm(bpm), phase(), now(), info() and the rhythm functions. OAVp5.drums.machine.play(false) stops the sound and the counting but leaves the grid up: OAVp5.drums.state then says 'stopped'. It is an advanced handle; the names on this page cover what a sketch normally needs.
Type
Object | null
Set once, when startDrums() has loaded the machine. null before that and after a failed load.
Before it has a value
It is null until startDrums() (or 🥁) has loaded the machine. Check for null before you use it.
function setup() {
createCanvas(400, 300);
startDrums('four on floor', 120);
textAlign(CENTER, CENTER);
}
function mousePressed() {
const m = OAVp5.drums.machine;
if (m) m.play(!m.seq.playing); // pause and resume, the grid stays
}
function draw() {
background(OAVp5.drums.state === 'stopped' ? 80 : 20);
fill(255);
text('click to pause or resume (' + OAVp5.drums.state + ')', width / 2, height / 2);
}
A click pauses the machine and the grid stays on the page; the state reads "stopped" until the next click.
Related
OAVp5.drumsThe object behind the drum names: the state of the machine, its counters, the line of rhythm, its history and the loaded machine.
OAVp5.drums.stateWhere the drum machine is: 'off', 'loading', 'on', 'stopped' or 'error'.
startDrums()Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
The list of cards oav-p5 has shown or would have shown so far (a blocked microphone, no Web MIDI, an error in your callback), each with what, why and what to press.
Each situation is said once per page. The list holds them all, including the ones the page did not show because hints=off is set, so it is the place to look when a sketch does something odd and nothing is on screen: console.log(OAVp5.hints).
Type
Array<Object>
Starts empty and only grows. Each item is { id, kind, what, why, next }: id names the situation ('micDenied', 'chopOut:lag'), kind is 'error', 'warn' or 'info', and what, why, next are the three lines of the card in the browser's language.
Notes
A card that went away because the situation ended (the microphone came on) stays in the list.
function setup() {
createCanvas(400, 300);
lag(1); // an unnamed lag() outside draw(): oav-p5 has something to say
}
function draw() {
background(20);
fill(255);
text(OAVp5.hints.length + ' card(s) so far', 20, 30);
text(OAVp5.hints.map((h) => h.kind + ': ' + h.what).join('\n'), 20, 60);
}
Lists what oav-p5 has said so far: here, that lag() was called outside draw(). The same sentence is in the console.
Related
hintsWhether oav-p5 shows its explaining cards beside the 🎹 (blocked microphone, no Web MIDI, an error in your callback): on or off.
OAVp5.check()Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
debugPrints [oav-p5 debug] lines in the console about clicks, the 🎹 button and loading, for finding out why the 🎹 does not respond.
What the link to other oav-p5 pages in the same room is doing: only OAVp5.link.info, and null unless a room is set.
A link joins oav-p5 pages that share a room: what one page's programs play into OAV Virtual MIDI (notes, controllers and the clock) is heard by every other page in the room, at the moment it sounds there. You switch it on with ?oav_link=my-room, and OAVp5.link.info is where you look at how it is going. There is nothing else to call: the link is made and closed by the option.
Notes
This is advanced. It needs a network connection to the relay (relay).
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
fill(255);
text(OAVp5.link.info ? 'linked' : 'not linked: add link: \'my-room\' to the options', 20, 30);
}
Says whether this page is linked. Without a link room it is not, and the sketch says so.
Related
OAVp5.link.infoThe state of the link: its room, state (connecting, open, closed), how many other pages are there, and the trip time. null when no room is set.
linkJoins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
linkdelayExtra milliseconds of buffer on the notes a linked page receives, to smooth out an uneven network: 0 to 2000.
relayThe address of the relay that phones and link rooms talk through: a WebSocket URL.
The state of the link: its room, state (connecting, open, closed), how many other pages are there, and the trip time. null when no room is set.
Read it in draw() for a status line, or look at the 🔗 on the 🎹, which turns green when another page is there, yellow while connecting or alone, and red when the relay is gone. The 🩺 check-up prints the same facts. A new object is made on every read.
Type
Object | null
null without a link room. With one: { room, state, peers, rtt, trip, late, sent, received, … }. state is 'connecting', 'open' or 'closed' (the relay is gone and it retries). peers is how many other pages were heard in the last 6 seconds. rtt is the shortest round trip to the relay in the last 30 seconds and trip the one-way trip estimated from it, both in milliseconds. late counts notes that arrived after the moment they should have sounded and were played at once. sent and received count messages. The object also has diff, lead, msgs, dupes and id, which explain the clock alignment (packages/relay/link.js).
function setup() {
createCanvas(400, 300);
}
function draw() {
background(20);
fill(255);
text('link.info: ' + JSON.stringify(OAVp5.link.info), 20, 30); // null: no room
}
Without a link room it prints null.
To be confirmed
The values between two real pages were not observed here: the headless check cannot reach a relay. The fields and their meanings are from the code (packages/relay/link.js) and the project's README.
Related
OAVp5.linkWhat the link to other oav-p5 pages in the same room is doing: only OAVp5.link.info, and null unless a room is set.
linkJoins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
linkdelayExtra milliseconds of buffer on the notes a linked page receives, to smooth out an uneven network: 0 to 2000.
relayThe address of the relay that phones and link rooms talk through: a WebSocket URL.
The loaded parts of @openav/sound: the Sound shell, its Tone.js engine and the module itself. All three are null until sound has loaded.
OAVp5.sound.shell is the Sound instance of @openav/sound, OAVp5.sound.engine is the Tone.js engine it plays through (noteOn, noteOff, sustain, releaseAll, setInstrument) and OAVp5.sound.module is the whole module, for example OAVp5.sound.module.listInstruments(), which lists every instrument id. They are advanced handles: the sketch-level names (startSound(), soundIsOn) cover everything a sketch normally needs.
Type
Object | null
Set once, when startSound() has loaded @openav/sound (the engine and the shell at the same moment, the module a little before). null before that, and after a failed load.
Before it has a value
Call startSound() first and wait until OAVp5.sound.state is no longer 'loading', or check for null before you use them.
The version of oav-p5 that loaded, as text such as '0.8.0'.
Useful in a bug report or to check which build a page got: openaudiovisual.com/p5/oav-p5.js is always the latest, while a jsDelivr address with a tag is frozen. The same number is on the first line of the 🩺 check-up.
Type
String
Set when oav-p5 loads and never changes.
Notes
If the file loads twice (the loader line and the OpenProcessing library list), the first one wins and the second does nothing.
OAVp5The object oav-p5 hangs everything on: MIDI state, the panel, the microphone, the clock, the check-up. midi and oavMidi are the same object.
OAVp5.check()Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
The resting value of a pitch-bend wheel: 8192, the middle of the 14-bit range 0..16383.
Use it when you build a pitch-bend message by hand: { type: 'pitchbend', ch: 1, value: BEND_CENTER } is a wheel at rest. Spring-loaded wheels in a profile (spring) go back to this value when released.
Type
Number
Range
8192
A constant. A pitch-bend value equal to it means "not bent" and becomes 0 in bendToUnit().
Turns a 14-bit pitch-bend value (0..16383) into −1..1, with the centre (8192) exactly 0.
MIDI centres pitch bend at 8192, so there are 8192 steps below the centre and 8191 above. Below the centre it divides by 8192 and above by 8191, so the bottom reaches exactly −1, the top exactly +1 and the rest position exactly 0. This is the number published as midi/bend and midi/ch/<ch>/bend.
Parameters
Name
Type
Description
v
Number
A pitch-bend value, 0..16383 (what parseMessage() puts in value).
module Prints −1, 0, 1, then about 0.5, then 5000: the round trip is exact.
Related
unitToBend()Turns a −1..1 number into a 14-bit pitch-bend value (0..16383), the inverse of bendToUnit().
BEND_CENTERThe resting value of a pitch-bend wheel: 8192, the middle of the 14-bit range 0..16383.
genericSignals()Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
The kinds of control a profile may contain: knob, encoder, fader, pad, button, keys, wheel, strip.
What each sends: knob and fader a CC (0..1); encoder a CC that is either relative (position plus a /delta signal) or absolute; pad a note (velocity while held, with a /hit pulse) or a CC; button a note or CC, momentary or toggle; keys a range of notes; wheel and strip a CC or pitch bend, optionally bipolar and spring-loaded.
A plain array. A control's type must be one of these or validateProfile() reports it.
Examples
js
import { CONTROL_TYPES, PROFILES } from'@openav/midi';
const count = {};
for (const p of PROFILES) for (const c of p.controls) count[c.type] = (count[c.type] || 0) + 1;
console.log(CONTROL_TYPES.map((t) => t + ': ' + (count[t] || 0)).join(', '));
module Prints how many controls of each type the shipped profiles hold, in the order of the list.
Related
validateProfile()Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
RESERVED_IDSControl ids a profile may not use, because they are signal suffixes: last, raw, delta, hit, on, off, pressure.
normalizeProfile()Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
controllerRoutes(profile, bindings, { short = profile.short, ...extra } = {})
Builds mapper routes for "this control drives that param": controllerRoutes(profile, { knob1: 'hue' }) routes the knob's signal to the param hue.
This is rule 1 of the project (continuous control goes through params): a world does not read midi/lpd8/k1 in update(), it declares a param and lets a route drive it. An id the profile does not have throws profile akai-lpd8 has no control "x", so a typo is loud.
Parameters
Name
Type
Description
profile
Object
The device's profile (raw or normalized): used to check the ids and to know which controls are bipolar.
bindings
Object
{ controlId: target }, where target is a param name or a route object ({ target, inMin, inMax, … }). A pad written as { target: 'burst', on: 'hit' } listens to the /hit pulse instead of its velocity.
short
String
The signal namespace. The default is the profile's short; give another when you run the device under a different one. optionaldefault profile.short
extra
Object
Every other key of the third argument ({ inMin: 0, inMax: 0.5 }) is copied onto each route as a default; a key in the route object itself wins. optional
Returns
Object[][{ source, target, … }], ready for mapper.addRoute(route). A bipolar control (a pitch wheel) gets inMin: -1, inMax: 1. Any other key in the third argument is copied onto every route.
Examples
js
import { controllerRoutes, profileById } from'@openav/midi';
const routes = controllerRoutes(profileById('lpd8'), {
k1: 'hue', // knob 1 drives the param "hue"
k2: { target: 'size', inMin: 0, inMax: 1 }, // a route object, extra keys are kept
pad1: { target: 'burst', on: 'hit' }, // fire on the hit, not on the velocity
});
for (const r of routes) console.log(r.source, '->', r.target);
module Prints midi/lpd8/k1 -> hue, midi/lpd8/k2 -> size and midi/lpd8/pad1/hit -> burst: the signal each control drives its param from.
Related
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
profileSignals()Lists every signal name a profile can publish under its own namespace (midi/lpd8/… for the LPD8), for docs, tests and mapping screens.
linkControllers()Links the on-screen controllers to phones over the room relay: tells phones which device is on screen, echoes this machine's moves to them, and mirrors theirs here.
Builds a Web MIDI MIDIAccess that always has one more input and output of its own, so a sketch written for a MIDI keyboard works with none plugged in.
A 2018 sketch that calls navigator.requestMIDIAccess() (or WebMidi.js) and waits for a keyboard shows nothing on a laptop without one, and on an iPad has no Web MIDI at all. Hand it this access instead and the on-screen controller becomes a real-looking MIDI input: the sketch's own unmodified code receives midimessage events with the same bytes a keyboard sends.
The virtual OUTPUT is a real-looking port too: a program that plays out to it (Strudel's .midi('OAV Virtual MIDI'), a WebMidi.js sketch) calls send(bytes, timestamp), and onOutput hears both. Real ports can be merged in with addReal(realAccess): they come FIRST in inputs, the virtual one last, so a sketch that only listens to inputs[0] takes the hardware when it is there and the virtual controller when it is not. Real ports pass through untouched, so nothing is delivered twice.
Parameters
Name
Type
Description
name
String
The name of the virtual port, as the sketch sees it. optionaldefault 'OAV Virtual MIDI'
manufacturer
String
The manufacturer the virtual ports report. optionaldefault 'open-audiovisual'
id
String
The port id prefix: the ports are <id>-in and <id>-out. optionaldefault 'oav-virtual'
sysex
Boolean
What access.sysexEnabled reports. optionaldefault false
onOutput
Function | null
(bytes, timestamp) => … called when a program sends to the virtual OUTPUT. timestamp is when the bytes should sound (performance.now() milliseconds), or undefined for "now". optionaldefault null
onClear
Function | null
Called when the program calls output.clear() (Web MIDI: drop what is still waiting). optionaldefault null
Returns
Object{ access, input, output, emit(bytes), addReal(realAccess), real, requests, dispose() }. Give access to the program. emit() plays bytes into the virtual input.
Notes
access.inputs and access.outputs are Maps, as in Web MIDI. input.listening counts the listeners a program put on the port, and requests counts how often virtualRequestMIDIAccess() handed the access out.
Examples
js
import { createVirtualMIDIAccess } from'@openav/midi';
const v = createVirtualMIDIAccess({
name: 'Demo MIDI',
onOutput: (bytes, timestamp) => console.log('program played', bytes, timestamp),
});
const input = [...v.access.inputs.values()][0];
const output = [...v.access.outputs.values()][0];
console.log(input.name, '|', input.listening, 'listeners');
input.onmidimessage = (e) => console.log('program heard', [...e.data]); // what a 2018 sketch does
v.emit([0x90, 60, 100]); // the on-screen keyboard plays: program heard [144, 60, 100]
output.send([0x90, 62, 80]); // the program plays out: program played [144, 62, 80] undefined
module Prints the port name with 0 listeners, then program heard [144, 60, 100], then program played [144, 62, 80] undefined (no timestamp means "now").
To be confirmed
Not tried for this page: a real WebMidi.js or Strudel sketch talking to the virtual ports, and how the onOutput timestamps line up with a hardware port. The example calls the ports directly.
Related
virtualRequestMIDIAccess()Wraps navigator.requestMIDIAccess so every caller gets the virtual access: real ports join if they answer in time, and the virtual input stays without Web MIDI.
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
parseEmbedOptions()Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
Returns a short readable line for a parsed MIDI event, for logs and learn screens: CC ch1 #74=64, Bend ch1 0.50.
The formats are Note On ch1 n60 v100 (two spaces after "On"), Note Off ch1 n60, CC ch1 #74=64, Bend ch1 0.50 (the bend as −1..1 with two decimals), Pressure ch1 50, Poly AT ch1 n60 33 and Program ch1 5. The Midi engine writes one of these to its log for each message it parses (clock is left out).
String The line. null gives ?, and a type without a format (clock, sysex) gives its type name.
Notes
Easily confused with describeMessage() and describeDetail(), which write the shorter lines the <oav-controller> readout shows (CC 70 = 53). This one takes an event and is meant for logs.
module Prints Note On ch1 n60 v100, CC ch1 #74=64 and Bend ch1 0.50.
Related
parseMessage()Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
describeMessage()Writes one parsed MIDI message the way a person reads it: CC 70 = 53, Note 36 · vel 90, Bend 12288.
describeDetail()Writes the readout line for a control event payload: midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1.
Recognises our own MIDI messages coming back: remember what was sent, and an identical message arriving within a short window is an echo, not a performer.
A virtual cable or a DAW can send your message straight back on an input, and the engine would treat it as a performer. sent(bytes) records a message and isEcho(bytes) asks whether an arriving one is that message returning. Each sent message cancels at most ONE echo, so two real identical messages are not both eaten.
A note-on with velocity 0 counts as a note-off, in both directions, because a DAW may rewrite one as the other. System messages (clock and the like) are never recorded. Midi has one (midi.echo, with the echoWindow option) and drops the echoes it finds, counted in midi.echoDropped.
Parameters
Name
Type
Description
window
Number
Milliseconds an echo may take to come back. 0 turns the guard off. optionaldefault 40
max
Number
How many sent messages to remember. Only the newest max are kept. optionaldefault 256
import { EchoGuard } from'@openav/midi';
const guard = new EchoGuard({ window: 40 });
guard.sent([0x90, 60, 100], 1000); // we sent note-on 60 at t = 1000 ms
console.log(guard.isEcho([0x90, 60, 100], 1010)); // true: it came back 10 ms later
console.log(guard.isEcho([0x90, 60, 100], 1011)); // false: one send explains one echo
guard.sent([0x90, 62, 100], 2000);
console.log(guard.isEcho([0x90, 62, 100], 2100)); // false: 100 ms later is a performer
module Prints true, false, false: an echo is used up once, and a message that arrives too late is a real note.
Related
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
isLive()Says whether a MIDI port can still be sent to: it exists and its state is not 'disconnected'.
Turns an event object back into MIDI bytes, the inverse of parseMessage(): { type: 'cc', ch: 1, cc: 74, value: 64 } becomes [0xb0, 74, 64].
encodeMessage({ type: 'noteon', ch: 10, note: 36, vel: 127 }) is [0x99, 36, 127]. The channel is held to 1..16 and every data byte is rounded and held to 0..127, so a bad value cannot produce an invalid message. A missing vel is 100 for noteon and 0 for noteoff.
A pitchbendvalue (0..16383) is split into two 7-bit bytes, low byte first, as MIDI requires. The package uses this for the on-screen controller and for LED feedback.
Parameters
Name
Type
Description
ev
Object
An event: type is noteon, noteoff, polyat, cc, program, chanat or pitchbend, with the fields parseMessage() uses for that type. ch defaults to 1.
Returns
number[] | null The bytes, ready for output.send(). null for a type it cannot make (clock, sysex, anything unknown).
Examples
js
import { encodeMessage, parseMessage } from'@openav/midi';
const bytes = encodeMessage({ type: 'noteon', ch: 10, note: 36, vel: 127 });
console.log(bytes); // [153, 36, 127]
console.log(parseMessage(bytes)); // back to the same event
module Prints [153, 36, 127] (0x99, 0x24, 0x7f), then the event it came from.
module Three results: the pitch-bend centre, a value held to 127, and null for a message type it cannot make.
Related
parseMessage()Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
describe()Returns a short readable line for a parsed MIDI event, for logs and learn screens: CC ch1 #74=64, Bend ch1 0.50.
unitToBend()Turns a −1..1 number into a 14-bit pitch-bend value (0..16383), the inverse of bendToUnit().
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
The inverse of genericSignals(): turns one generic MIDI signal (midi/ch/10/note/36, midi/cc/74) back into the MIDI event it came from.
It lets anything that only hears a Signals bus (a recorded take playing back, the on-screen piano, a phone) drive a controller model as if the bytes had arrived. Values come back on the 0..127 scale they were published from (value × 127, rounded), so the round trip through genericSignals() is exact.
A note signal with a value above 0 is a noteon (velocity at least 1), and 0 is a noteoff. The on-screen piano publishes channel 0, which is read as channel. A hardware message is published under both families, so a listener that wants each message once should use followMidiSignals() instead of calling this on every name.
Parameters
Name
Type
Description
name
String
A signal name. Only the generic names starting with midi/ are understood.
value
any
The signal's value: a number 0..1 (−1..1 for bend), or the { note, vel, velocity, ch } payload of midi/note/on and midi/note/off.
channel
Number
The channel to use for the legacy names that carry none (midi/cc/<n>, midi/bend). optionaldefault 1
Returns
{ family, ev } | nullfamily is 'channel' or 'legacy', and ev is the event. null for a device name (midi/minilab3/…), midi/virtual and anything else.
module Prints a noteon on channel 10 with velocity 64, a CC 74 on channel 4 with value 127, then null.
Related
genericSignals()Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
followMidiSignals()Listens to a Signals bus and hands you each MIDI message back exactly once, as an event, although hardware publishes it under two names.
ControllerCanvasDraws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
module Prints IAC Driver Bus 1, Minilab3 MIDI, null, null.
Related
pickOutput()Chooses which MIDI output to send to from a list: the port you named, else the first match of a preference list, else the first live port.
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
Finds a profile in a list by id or short name, or returns null; no validation, no loading.
Use it when you hold your own list, for example the profiles a picker offers (controller.controllers.profiles), and want the same id-or-short lookup the element uses.
Parameters
Name
Type
Description
ref
String
A profile id or short name.
profiles
Object[]
The list to search (required here, unlike profileById(), which uses the shipped ones).
profileById()Looks up a shipped controller profile by its id ('akai-lpd8') or its short signal name ('lpd8'), or returns null.
resolveProfile()Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.
Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
Two families, both published for every hardware port and every virtual controller. The short ones are what most worlds read: a noteon becomes the pulse midi/note/on with { note, vel (0..1), velocity (1..127), ch, device }, a noteoff becomes midi/note/off with { note, ch, device }, a cc becomes midi/cc/<n> (0..1), and a pitchbend becomes midi/bend (−1..1).
The per-channel ones are for two devices that share a CC number, or a work that cares about the channel: midi/ch/<ch>/note/<n> (velocity 0..1 while held, 0 on release), midi/ch/<ch>/cc/<n>, midi/ch/<ch>/bend, midi/ch/<ch>/pressure (channel aftertouch) and midi/ch/<ch>/poly/<n> (polyphonic aftertouch). Pressure and poly exist only in this family. program, clock and the other system messages produce nothing.
There is deliberately no midi/cc/<ch>/<n>: under prefix subscriptions it would collide with "CC 1 on any channel".
module The first prints the pulse payload { note: 60, vel: 0.787…, velocity: 100, ch: 1, device: 'my-keys' }, the second only the per-channel name.
Related
eventOfSignal()The inverse of genericSignals(): turns one generic MIDI signal (midi/ch/10/note/36, midi/cc/74) back into the MIDI event it came from.
publish()Publishes a list of { name, value, pulse?, min?, max? } into a Signals registry, declaring each signal the first time it is seen.
parseMessage()Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
followMidiSignals()Listens to a Signals bus and hands you each MIDI message back exactly once, as an event, although hardware publishes it under two names.
Groups a profile's controls by their group (knob, pad, fader …), returning { group: [control ids] }.
The groups follow the TouchDesigner tool's out CHOPs (out_knob, out_slider, out_pad …). A control gets its type as group unless the profile gives another. MidiController has a method of the same name that returns the current values instead of the ids.
Parameters
Name
Type
Description
p
Object
A normalized profile, so every control has its group filled in.
module Prints the group names and then the eight knob ids.
Related
profileSignals()Lists every signal name a profile can publish under its own namespace (midi/lpd8/… for the LPD8), for docs, tests and mapping screens.
normalizeProfile()Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
Builds the lookup from a MIDI message to the control it moves: "cc:1:74" maps to the knob listening for CC 74 on channel 1.
This is how MidiController decides which control a message belongs to. Use it to build your own router. Each value is a list because a message can move more than one control.
Parameters
Name
Type
Description
p
Object
A normalized profile (each control needs its ch).
Returns
Map<string, { control, note? }[]> The keys are cc:<ch>:<cc>, note:<ch>:<note>, polyat:<ch>:<note>, pitchbend:<ch>, chanat:<ch>, program:<ch>. A keys control gets one entry per note of its range.
module Prints ['k1'] for CC 70, ['pad1'] for note 36 on channel 10, and undefined for a message the device does not send.
Related
normalizeProfile()Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
Says whether a MIDI port can still be sent to: it exists and its state is not 'disconnected'.
A port that was unplugged can still be an object in your hands. Sending to it gets nowhere, and the engine's try/catch swallows the error, so nothing on screen says why. The output rules (pickOutput(), reselectOutput()) therefore skip every port that is not live.
Parameters
Name
Type
Description
p
MIDIPort | null
A port shaped like a MIDIOutput: it only needs a state.
Returns
Booleanfalse for null and for a disconnected port.
Tells a URL to a profile file apart from a profile id: true for ./pads.json, https://…, /p/x, false for 'akai-lpd8'.
It is the test resolveProfile() uses to decide between looking an id up and fetching a file. A bare name that happens to end in .json also counts as a URL.
Parameters
Name
Type
Description
ref
any
What the page or attribute gave. Anything that is not a string is false.
Returns
Booleantrue when the string starts with http:, https:, blob: or data:, starts with /, ./ or ../, or ends in .json (optionally followed by ?… or #…).
Examples
js
import { isProfileUrl } from'@openav/midi/embed';
for (const ref of ['akai-lpd8', './pads.json', 'https://x.org/my.json', '/p/x', 'my.json']) {
console.log(ref, isProfileUrl(ref));
}
module Prints false for the id and true for the other four.
Related
resolveProfile()Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.
loadProfile()Loads a controller profile by id, short name or URL (a JSON file anywhere CORS allows), checking any file or object you give it; returns a promise.
Links the on-screen controllers to phones over the room relay: tells phones which device is on screen, echoes this machine's moves to them, and mirrors theirs here.
A phone's MIDI tab publishes the device's signals itself (the controller lives there), so this side only does three things. When you select another device it tells the phones with relay.config('midi', { profile }). When a knob, fader, encoder, wheel, strip or toggle button on THIS machine moves from hardware or the screen, it queues relay.feedback('midi/<short>/<id>', value) so the phone's knobs follow. When a phone plays (midi/virtual) it mirrors that on this screen, state and view only, because the phone already published the signals.
This is what @openav/remote's show-side host and the lab's midi module call. You need it only when you wire phones into a page yourself.
Parameters
Name
Type
Description
relay
Object
A relay client with config(name, value) and feedback(signalName, value) (the show runner's RelayClient).
signals
Signals
The bus the phones' midi/virtual messages arrive on.
controllers
MidiControllers
The manager whose controllers are linked.
Returns
{ flush(), dispose() }flush() sends the queued feedback to the relay (call it at a steady pace, for example once per frame). dispose() stops listening.
Examples
js
import { Signals } from'@openav/core';
import { MidiControllers, PROFILES, linkControllers } from'@openav/midi';
const signals = new Signals();
const mcs = new MidiControllers({ profiles: PROFILES, signals, initial: 'akai-lpd8' });
// a stand-in for the relay client: it prints what would go to the phonesconst relay = {
config: (name, value) => console.log('config', name, value),
feedback: (name, value) => console.log('feedback', name, value.toFixed(2)),
};
const link = linkControllers(relay, signals, mcs);
mcs.current.setValue('k1', 0.5); // a knob moves on this machine
link.flush(); // feedback midi/lpd8/k1 0.50
mcs.select('korg-nanokontrol2'); // config midi { profile: 'korg-nanokontrol2' }
link.dispose();
module Prints feedback midi/lpd8/k1 0.50 after the flush, then config midi { profile: 'korg-nanokontrol2' } when the device on screen changes.
Related
MidiControllersTies hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
controllerRoutes()Builds mapper routes for "this control drives that param": controllerRoutes(profile, { knob1: 'hue' }) routes the knob's signal to the param hue.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
Loads a controller profile by id, short name or URL (a JSON file anywhere CORS allows), checking any file or object you give it; returns a promise.
It is resolveProfile() with the shipped profiles and the page's base URL filled in. <oav-controller profile="./my-device.json"> calls it to read your file. A bad file says why instead of drawing nothing.
Parameters
Name
Type
Description
ref
String | Object
An id ('akai-lpd8'), a short name ('lpd8'), a URL or path to a profile JSON, or a profile object.
o
Object
Options passed on to resolveProfile(): fetch, base, and profiles (default: every shipped profile). base defaults to the page's document.baseURI, so a relative path is read from the page. optionaldefault {}
Returns
Promise<Object> The raw profile. Rejects with every problem listed when a file or object is invalid, or with unknown MIDI profile "x" (known: …) when an id does not exist.
module Prints akai-lpd8 16, then the beginning of the error for an id that does not exist (it goes on to list every known id).
js
import { loadProfile, profileById } from'@openav/midi';
// a profile file of your own; here a fake fetch stands in for the networkconst mine = { ...structuredClone(profileById('lpd8')), id: 'my-lpd8', short: 'mylpd8', name: 'My LPD8' };
const fetch = async (url) => { console.log('fetching', url); return { ok: true, json: async () => mine }; };
const p = await loadProfile('./profiles/my-lpd8.json', { fetch, base: 'https://example.com/show/' });
console.log(p.name);
module Prints the full URL it fetched (https://example.com/show/profiles/my-lpd8.json), then My LPD8.
Related
resolveProfile()Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.
profileById()Looks up a shipped controller profile by its id ('akai-lpd8') or its short signal name ('lpd8'), or returns null.
validateProfile()Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
Finds the device profile that owns a MIDI port name, such as 'Minilab3 MIDI', by testing each profile's match patterns.
Only device profiles are considered (a generic layout never matches by name). A profile matches when one of its match.ports regular expressions (case-insensitive) hits the name. The score starts at 1, gains 1 when a match.prefer pattern also hits, and loses 2 when a match.avoid pattern hits. The highest positive score wins, so a port a device only exposes for something else (Minilab3 DIN THRU, Minilab3 MCU/HUI) does not match.
This is how MidiControllers turns a plugged-in device into the right on-screen controller. When a device exposes several ports, pickPort() chooses which one to listen to.
{ profile, score } | null The best match, or null when no device profile fits.
Examples
js
import { matchProfile, PROFILES } from'@openav/midi';
for (const name of ['LPD8 mk2', 'Minilab3 MIDI', 'Minilab3 DIN THRU', 'Some Unknown Synth']) {
const m = matchProfile(name, PROFILES);
console.log(name, '->', m ? m.profile.id + ' (score ' + m.score + ')' : null);
}
module Prints akai-lpd8 (score 1), arturia-minilab3 (score 2), then null for the DIN THRU port and for an unknown synth.
Related
pickPort()Of several ports one device exposes ("Minilab3 MIDI", "Minilab3 DIN THRU", "Minilab3 MCU/HUI"), picks the one to listen to.
MidiControllersTies hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
PROFILESEvery controller profile that ships with the package, as an array of plain JSON objects: real devices first, then the generic layouts.
The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
Create it, then await midi.enable(). It resolves true once the browser has given access and false when there is no Web MIDI (Safari, iPad) or the permission was refused; the reason is written to the log (onMessage). Nothing is listened to or sent before that.
In: every message from every listened input goes to the callbacks you added with listen(fn) (raw bytes, including clock), then is parsed and published as genericSignals() names. With two or more inputs it also publishes midi/<slug>/note/on, midi/<slug>/note/off, midi/<slug>/cc/<n> and midi/<slug>/bend, so devices do not collide. The slug comes from the port name (Minilab3 MIDI becomes minilab3-midi), so a device that drops and reconnects keeps its identity. Clock messages are not logged and publish nothing.
Out: send(bytes), noteOn(), noteOff() and cc() go to midi.out, the chosen output port, and are remembered for echo filtering. If that port is unplugged the engine moves to the next best live port (your choice by name, then prefer, then the first), moves back when your choice returns, and never lets an unrelated device steal the output. Every switch writes one line to the log and one console.warn, and calls the onOutput subscribers.
Hooks you can assign: onCC(cc, value01, ch), onNote(note, velocity01, on, ch), onMessage({ dir, text }) (the log), onSend(bytes) (sees everything sent, for a meter) and onDeviceChange(devices). Fields: enabled, inputs, outputs, out, access, echoDropped.
Before you call it
Call await midi.enable() first. Browsers with Web MIDI (Chrome, Edge, Firefox) ask the visitor for permission at that moment, so call it from a click when you can. Safari and iPad have no Web MIDI: enable() gives false.
Parameters
Name
Type
Description
signals
Signals | null
Publish every input message here as midi/… signals (see genericSignals()). Without it the engine still calls listen() callbacks and the on… hooks. optionaldefault null
filterOut
String | RegExp | null
Input port NAMES never to listen to, as a regular expression (a string is read as its source, case-insensitive). The default 'IAC' is the macOS bus you send into Ableton or TouchDesigner, so what you send never comes back as input. null listens to everything. optionaldefault 'IAC'
requestAccess
Function | null
Where the MIDIAccess comes from, called as requestAccess({ sysex: false }) and returning a promise. The default is navigator.requestMIDIAccess. Pass your own for a test, a shim, or a host that wraps Web MIDI for old sketches and hands the REAL one here. optionaldefault null
echoWindow
Number
Milliseconds. An input message identical to one just SENT within this window is our own echo and is dropped (counted in echoDropped). 0 turns it off. optionaldefault 40
prefer
Array<String | RegExp>
Output priority after the port you chose by name: the first entry that matches a live port wins (a string matches as a substring), else the first port. Example: ['IAC']. optionaldefault []
Asks the browser for Web MIDI and wires the inputs. preferredOut is the output you want, by exact name and then by substring. Resolves true, or false when Web MIDI is missing or refused.
Subscribe to output changes (hot-plug, switch back, your own selectOutput): fn({ text, reason, next, out }). reason is found, fallback, back, none or select. Returns an unsubscribe function.
Sends to the output with exactly this name from now on, and remembers it as YOUR choice: if it is unplugged and plugged in again, the output returns to it. Gives false when no output has that name.
The output port a device answers on (for LED feedback): the one with the same name as its input, or the first whose name matches a RegExp. null when there is none.
Subscribe to the raw bytes of every listened input: fn(bytes, { slug, name, input }). Called before parsing, clock included. Returns an unsubscribe function.
Sends raw bytes to the output port, and to the onSend hook. Unless echo is false they are also remembered, so the same bytes coming back within echoWindow ms are dropped. Does nothing when there is no output.
Silences everything: all-notes-off, all-sound-off and sustain-off on all 16 channels, plus a note-off for every note. It goes through send(), so meters see the sweep.
Notes
createShow() creates one and enables it for you. createController() and <oav-controller> share one engine per page, so a page with five controllers asks for MIDI once.
The engine sends MIDI data only and plays no audio. To hear the notes you need a synth on the other end (a DAW on the IAC bus, a hardware synth), or @openav/sound.
Examples
js
import { Midi } from'@openav/midi';
// a stand-in for navigator.requestMIDIAccess, so this runs without a deviceconst output = { name: 'Demo out', state: 'connected', send: (bytes) => console.log('out', bytes) };
const access = { inputs: new Map(), outputs: new Map([['o', output]]) };
const midi = new Midi({ requestAccess: async () => access });
console.log(await midi.enable()); // true
midi.noteOn(60, 100); // out [144, 60, 100]
midi.noteOff(60); // out [128, 60, 0]
module Prints true, then the two messages that reach the output port: note-on and note-off for middle C.
module Prints knob 1.00: a message arriving on the input port became the signal midi/cc/74.
js
import { Midi } from'@openav/midi';
const port = (name) => ({ name, state: 'connected', send: () => {} });
const iac = port('IAC Driver Bus 1'), synth = port('Synth');
const access = { inputs: new Map(), outputs: new Map([['a', iac], ['b', synth]]) };
const midi = new Midi({ requestAccess: async () => access });
midi.onOutput(({ reason, text }) => console.log(reason, '-', text));
await midi.enable('IAC');
console.log(midi.out.name); // IAC Driver Bus 1
access.outputs.delete('a'); iac.state = 'disconnected';
access.onstatechange(); // what the browser does when a port is unplugged
console.log(midi.out.name); // Synth
module Prints the chosen port, then a fallback line saying the output moved to "Synth" (the engine also writes it with console.warn), then Synth.
js
import { Midi } from'@openav/midi';
const midi = new Midi();
if (await midi.enable()) {
console.log('inputs:', midi.devices().map((d) => d.name));
} else {
console.log('no Web MIDI here, or the permission was refused');
}
static The real thing: the browser asks for MIDI permission, then the console lists your input devices (or says there is no Web MIDI).
To be confirmed
Not tried for this page: the real permission prompt and real MIDI hardware in each browser. The examples run against a stand-in for MIDIAccess, and the browser behaviour (Chrome, Edge and Firefox ask for permission, Safari and iPad have no Web MIDI) is as the package README describes it.
Related
EchoGuardRecognises our own MIDI messages coming back: remember what was sent, and an identical message arriving within a short window is an echo, not a performer.
pickOutput()Chooses which MIDI output to send to from a list: the port you named, else the first match of a preference list, else the first live port.
reselectOutput()Decides what to send to after the port list changed or the current port died, and returns the one line to print about it.
genericSignals()Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
MidiControllersTies hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
ScoreMidiTurns a show's segment changes and cues into MIDI notes and CCs on one channel, so a DAW or lighting desk can follow the show.
new MidiController(profile, { signals = null, sink = null, short = null, onLed = null } = {})
One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
The one idea: the on-screen controller and the hardware speak the SAME language, MIDI bytes. setValue(), press() and the other virtual hands encode the exact message the real control would send and feed it through ingest(), the same door the hardware uses. So a virtual knob and a real one can never drift apart, and what your work receives is identical whichever hand moved it. Pass the bytes of a real device to ingest(bytes, { source: 'hw' }).
The source of a message decides what is published, so every event is published exactly once. 'hw': the device lives here, and Midi already published the generic midi/… names, so only midi/<short>/… is added. 'ui': a virtual hand, so it publishes midi/<short>/…, the generic names, and midi/virtual (the raw bytes, for Web MIDI shims). 'mirror': someone else published it (a phone, feedback), so only the state and the views move.
Signals per control (values are normalised, raw keeps the MIDI number): midi/<short>/<id> (0..1, −1..1 for a bipolar wheel) and /raw; an encoder's /delta; a pad's /hit pulse { vel, velocity, note, ch } and /pressure; a keys control's /on and /off pulses and midi/<short>/n<note> while a key is held; and midi/<short>/last, a pulse saying what was touched last. profileSignals() lists them for a profile.
Each control keeps a state { value, raw, held, pressure, pos } in controller.state (read it with get(id)). controller.keys maps each held key's note to its velocity. controller.profile is the working copy (learning changes it), controller.base the normalized profile before any learning, controller.lastEvent the last message with its text, and controller.learning and controller.learned track learn mode.
Parameters
Name
Type
Description
profile
Object
A raw or normalized profile (see PROFILES). It is copied, so changing the object later does nothing.
signals
Signals | null
Publish the controller's signals here. optionaldefault null
sink
Function | null
(name, value, { pulse, min, max }) => … called for each signal instead of signals (a relay, a phone). When both are given, sink wins. optionaldefault null
short
String | null
Overrides the signal namespace, so two controllers of the same device publish under different names. optionaldefault null
onLed
Function | null
(bytes) => … called with the MIDI that lights or dims a control's LED on the hardware, whenever its state changes. optionaldefault null
The current values grouped like the TouchDesigner tool's out CHOPs: { knob: [v1, v2 …], pad: […] }. Keys controls are left out. (Unlike groupsOf(), which lists ids.)
Subscribe to changes: fn(id, state, info) with info{ source, ev, note, bytes }. When the mapping or learn state changes instead, it is called as fn(null, null, { meta: true }). Returns an unsubscribe function.
Subscribe to every message that passes through, once, after the state has moved: fn({ bytes, ev, source, ids }), where ids are the controls it moved (none for a message the profile does not know). An embed forwards source: 'ui' messages to a MIDI output here. Returns an unsubscribe function.
Feeds MIDI bytes in. source is 'hw' (default), 'ui' or 'mirror'. Returns the parsed event, or null. Clock, sysex and system messages are returned without touching any control.
Moves an absolute control (knob, fader, wheel, strip, absolute encoder) to v: 0..1, or −1..1 for a bipolar one. Returns the bytes it sent, or null when the value did not change. For a relative encoder it turns by the matching steps.
Presses a pad, button or key with a velocity from 0 to 1 (a pad or key velocity is held to 1..127; a button always sends 127). A key needs its note. A toggle button flips. Returns the bytes.
Arms learn mode for one control: the next compatible hardware message is bound to it. Calling it again for the same id cancels. Returns the armed id or null.
The learned bindings as JSON, { controlId: { msg, ch, cc, note, from, to } }, containing only what differs from the profile. Persist it per device (localStorage, a file).
Goes back to the profile's own bindings and forgets what was learned.
Notes
Inside a show, do not read midi/lpd8/k1 in update(): route it to a param with controllerRoutes(). You rarely create a MidiController yourself: MidiControllers makes one per profile, and createController() wraps all of it.
Examples
js
import { MidiController, profileById } from'@openav/midi';
const lpd8 = new MidiController(profileById('lpd8'));
lpd8.onChange((id, st, info) => id && console.log(info.source, id, st.value.toFixed(2), st.raw));
lpd8.ingest([0xb0, 70, 127], { source: 'hw' }); // the real knob 1 moved to the top
lpd8.setValue('k2', 0.5); // a finger on the on-screen knob 2
lpd8.press('pad1', 0.8); // a finger on pad 1
module Prints three lines: hw k1 1.00 127, ui k2 0.50 64, ui pad1 0.80 102. The same model, three hands.
js
import { Signals } from'@openav/core';
import { MidiController, profileById } from'@openav/midi';
const signals = new Signals();
const mc = new MidiController(profileById('minilab3'), { signals });
signals.on('midi/minilab3/pad3/hit', (hit) => console.log('pad 3 hit', hit));
mc.press('pad3', 0.5);
console.log(signals.get('midi/minilab3/pad3')); // 0.50...: velocity while held
mc.release('pad3');
console.log(signals.get('midi/minilab3/pad3')); // 0
module Prints the /hit pulse { vel: 0.50…, velocity: 64, note: 38, ch: 10 }, then the pad's value while held and after release.
js
import { MidiController, profileById } from'@openav/midi';
// a generic layout learns whatever you moveconst mc = new MidiController(profileById('generic-8k8p'));
mc.learn('k3'); // arm control k3
mc.ingest([0xb0, 50, 99], { source: 'hw' }); // the next CC from the hardware is bound to it
console.log(JSON.stringify(mc.exportMapping())); // {"k3":{"msg":"cc","ch":1,"cc":50}}const again = new MidiController(profileById('generic-8k8p'));
again.applyMapping(mc.exportMapping());
again.ingest([0xb0, 50, 10], { source: 'hw' });
console.log(again.get('k3').raw); // 10
module Prints the learned binding of k3, then 10: a second controller with the saved mapping reacts to the same CC.
Related
MidiControllersTies hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
controllerRoutes()Builds mapper routes for "this control drives that param": controllerRoutes(profile, { knob1: 'hue' }) routes the knob's signal to the param hue.
profileSignals()Lists every signal name a profile can publish under its own namespace (midi/lpd8/… for the LPD8), for docs, tests and mapping screens.
parseMessage()Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
Ties hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
There is one MidiController per profile, created the first time it is asked for (get(id)), so a virtual MiniLab and a real MiniLab are the SAME object: whichever hand moves it, the work receives the same midi/minilab3/… signals. current is the controller the panel shows.
With a midi attached it matches every input port to a profile by name (matchProfile(), one port per device through pickPort()), switches current to the first device it recognises, sends the LED colours to its output port, and routes each port's messages to that profile's controller. A port nobody recognises can be given to a profile by hand with assign(slug, profileId); that choice is remembered per port name.
It remembers in storage under openav.midi.map.<profile id> (learned mappings, restored when the controller is created) and openav.midi.assign (manual port assignments).
Parameters
Name
Type
Description
profiles
Object[]
The profiles to manage, usually PROFILES. Required: the first one is shown when initial is not given. optional
signals
Signals | null
Passed to every MidiController it creates. optionaldefault null
sink
Function | null
Passed on to every controller, as in MidiController. optionaldefault null
midi
Midi | null
An enabled (or about-to-be-enabled) Midi to follow. Leave it out for no hardware, and call attach(midi) later. optionaldefault null
initial
String | null
The profile id shown before any hardware arrives. optionaldefault null
storage
Storage | null
Where learned mappings and manual port assignments are remembered: anything with getItem and setItem. The default is localStorage when the page has it, otherwise nothing is remembered. optionaldefault safeStorage()
Starts following a hardware Midi (now, or later: an embed connects only when asked). Switching to another engine detaches the first. Returns the manager.
The MidiController for a profile id, created on first use with any learned mapping restored. Throws unknown MIDI profile "x" for an id that was never registered.
Subscribe to { type: 'select' } (the device on screen changed) and { type: 'ports' } (ports or their assignments changed): fn(event, manager). Returns an unsubscribe function.
The hardware ports and who they drive: [{ slug, name, listening, profileId, auto }]. auto is true when the match was made by name, false when you assigned it.
Routes a port to a profile by hand (an unknown device to a generic layout), or un-routes it with null. It then selects that profile. persist (default true) remembers the choice for this port name.
module Prints 1 (a finger on the on-screen LPD8 knob became a signal), then korg-nanokontrol2 after switching the device on screen.
js
import { Signals } from'@openav/core';
import { Midi, MidiControllers, PROFILES } from'@openav/midi';
// a fake Minilab3 on a fake MIDIAccessconst keys = { name: 'Minilab3 MIDI', state: 'connected', onmidimessage: null };
const access = { inputs: new Map([['k', keys]]), outputs: new Map() };
const signals = new Signals();
const midi = new Midi({ signals, requestAccess: async () => access });
await midi.enable();
const mcs = new MidiControllers({ profiles: PROFILES, signals, midi });
console.log(mcs.current.id, '<-', mcs.hardwareFor('arturia-minilab3'));
keys.onmidimessage({ data: Uint8Array.of(0xb0, 74, 127) }); // the real knob 1
console.log(signals.get('midi/minilab3/knob1')); // 1
module Prints arturia-minilab3 <- Minilab3 MIDI (the port was recognised by name), then 1: a message from the port moved knob 1 of the on-screen MiniLab 3.
To be confirmed
LED feedback (the colours sent back to a device when it is recognised) was not tried on real hardware. Some controls in the shipped profiles are marked verified: false: their messages come from the manual but nobody has confirmed them on a real unit yet, and Learn rebinds them.
Related
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
matchProfile()Finds the device profile that owns a MIDI port name, such as 'Minilab3 MIDI', by testing each profile's match patterns.
pickPort()Of several ports one device exposes ("Minilab3 MIDI", "Minilab3 DIN THRU", "Minilab3 MCU/HUI"), picks the one to listen to.
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
mountMidiPanel()Mounts a controller panel: a toolbar (device picker, hardware status, port picker, Learn, last message) above a playable faceplate, docked at the bottom or full size.
Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
It throws invalid MIDI profile "<id>": followed by every problem when the profile is not valid. Otherwise it fills in: channel (1), learn (true for a generic layout), stack (true), sources and match (empty lists), and per control label (its id), group (its type), ch (the profile's channel), an encoder's relative (offset64), a button's mode (momentary) and a pitch-bend wheel's bipolar and spring (both true). Sections get label, flow (row) and spans, and portrait is built from the sections, top to bottom, with each section's height as its weight when you did not write one.
MidiController, ControllerCanvas and the view all work on the normalized form, which is what controller.profile gives you.
module Prints the defaults it filled in, then the error message listing what is wrong with { id: 'X' }.
Related
validateProfile()Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
PROFILESEvery controller profile that ships with the package, as an array of plain JSON objects: real devices first, then the generic layouts.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
indexProfile()Builds the lookup from a MIDI message to the control it moves: "cc:1:74" maps to the knob listening for CC 74 on channel 1.
Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
parseMessage([0x90, 60, 100]) is { type: 'noteon', ch: 1, note: 60, vel: 100 }. Channels are numbered 1 to 16, the way they are printed on a device (the status byte stores 0 to 15), and data bytes are cut to 0..127. Every other function in this package that handles MIDI speaks these event objects.
The type is noteon and noteoff (note, vel), polyat (polyphonic aftertouch: note, value), cc (cc, value), program (value), chanat (channel pressure: value) or pitchbend (value 0..16383, centre 8192). System messages come back as clock, start, continue, stop, sysex (with data, the whole message) or system (with the status byte).
A note-on with velocity 0 comes back as noteoff, because that is how many devices say "released". Bytes that start with a data byte give null: the browser has already resolved running status before it hands you the message.
Parameters
Name
Type
Description
data
Array | Uint8Array
The bytes of ONE message, the way Web MIDI hands them over (event.data): a status byte first, then up to two data bytes.
Returns
Object | null The event, or null when the input is empty, starts with a data byte (below 0x80) or is a channel status it has no case for.
Notes
It is pure: no DOM, no Web MIDI, no state. encodeMessage() is its inverse, and describe() turns the result into a line of text.
Examples
js
import { parseMessage } from'@openav/midi';
console.log(parseMessage([0x90, 60, 100])); // note on, middle C
console.log(parseMessage([0x90, 60, 0])); // velocity 0 reads as note off
console.log(parseMessage([0xe0, 0, 64])); // pitch bend, centre
module Open the console: three event objects, a noteon, a noteoff and a pitchbend with value: 8192.
js
import { parseMessage } from'@openav/midi';
// what a Web MIDI port gives you: event.data is a Uint8Arrayconst ev = parseMessage(Uint8Array.of(0xb0, 74, 100));
if (ev && ev.type === 'cc') {
console.log('CC', ev.cc, 'on channel', ev.ch, '=', (ev.value / 127).toFixed(2));
}
module Prints CC 74 on channel 1 = 0.79: a control-change value turned into the 0 to 1 range a sketch wants.
Related
encodeMessage()Turns an event object back into MIDI bytes, the inverse of parseMessage(): { type: 'cc', ch: 1, cc: 74, value: 64 } becomes [0xb0, 74, 64].
describe()Returns a short readable line for a parsed MIDI event, for logs and learn screens: CC ch1 #74=64, Bend ch1 0.50.
genericSignals()Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
Chooses which MIDI output to send to from a list: the port you named, else the first match of a preference list, else the first live port.
The order is: 1) the port named preferred, 2) the first port that matches each entry of prefer, in order, 3) the first port. Only live ports (see isLive()) are considered, so a disconnected port is never picked.
It is a pure function over plain objects, so you can test the rules without any hardware. The Midi engine applies them through reselectOutput().
Parameters
Name
Type
Description
outputs
MIDIOutput[]
The candidate ports (objects with a name and a state).
preferred
String
The port you asked for: tried by exact name, then as a substring. optionaldefault ''
prefer
Array<String | RegExp>
Fallback order. The first entry that matches a live port wins (a string is a substring). optionaldefault []
Returns
MIDIOutput | null The chosen port, or null when no port is live.
Examples
js
import { pickOutput } from'@openav/midi';
const port = (name, state = 'connected') => ({ name, state });
const outs = [port('Synth A'), port('IAC Driver Bus 1'), port('Synth B')];
console.log(pickOutput(outs).name); // Synth A
console.log(pickOutput(outs, { preferred: 'IAC' }).name); // IAC Driver Bus 1
console.log(pickOutput(outs, { prefer: [/synth b/i] }).name); // Synth B
console.log(pickOutput([port('X', 'disconnected')])); // null
module Prints Synth A, IAC Driver Bus 1, Synth B and null: the first port, your choice, the preference list, and nothing live.
Related
reselectOutput()Decides what to send to after the port list changed or the current port died, and returns the one line to print about it.
isLive()Says whether a MIDI port can still be sent to: it exists and its state is not 'disconnected'.
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
Of several ports one device exposes ("Minilab3 MIDI", "Minilab3 DIN THRU", "Minilab3 MCU/HUI"), picks the one to listen to.
A port scores 1 when its name matches one of the profile's match.ports (otherwise it is out), plus 1 for a prefer match and minus 2 for an avoid match. Only positive scores count and the highest wins. It is the second half of device detection after matchProfile().
Parameters
Name
Type
Description
ports
Object[]
Candidate ports: objects with a name.
profile
Object
The device's profile (its match.ports, prefer and avoid).
Returns
Object | null The best port, or null when none of them fits the profile.
module Prints Minilab3 MIDI, then null when no port belongs to the device.
Related
matchProfile()Finds the device profile that owns a MIDI port name, such as 'Minilab3 MIDI', by testing each profile's match patterns.
MidiControllersTies hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
Looks up a shipped controller profile by its id ('akai-lpd8') or its short signal name ('lpd8'), or returns null.
Only profiles from PROFILES are searched. A profile you loaded from a URL is not in there: use resolveProfile(), which also accepts ids. It does not normalize or validate: pass the result to normalizeProfile() when you need the filled-in form.
Parameters
Name
Type
Description
id
String
A profile id or its short name (the part in midi/<short>/…).
Returns
Object | null The raw profile from PROFILES, the same object each time, or null.
Examples
js
import { profileById } from'@openav/midi';
const lpd8 = profileById('lpd8'); // short name
console.log(lpd8.id, '|', lpd8.name, '|', lpd8.controls.length, 'controls');
console.log(profileById('akai-lpd8') === lpd8); // true: the same profile by its id
console.log(profileById('no-such-box')); // null
module Prints the LPD8's id, name and 16 controls, then true, then null.
Related
PROFILESEvery controller profile that ships with the package, as an array of plain JSON objects: real devices first, then the generic layouts.
resolveProfile()Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.
normalizeProfile()Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
findProfile()Finds a profile in a list by id or short name, or returns null; no validation, no loading.
Every controller profile that ships with the package, as an array of plain JSON objects: real devices first, then the generic layouts.
A profile describes one controller as data: what it looks like (the faceplate sections), every control on it (type and the exact MIDI message it sends), LED feedback where the device accepts it, how to recognise its port name, and the document each number came from. Each one is a JSON file in packages/midi/profiles/; the full field list is in the package README and the controllers page shows them all.
A device profile cites its sources and is matched to ports by name, so plugging the device in picks it automatically. A generic layout (generic-8k8p, generic-keys49 …) never auto-matches: you assign a device to it and it learns the controls in the order you move them.
The order is the order a device picker shows. Each item is a raw profile (see normalizeProfile() for the filled-in form): id, short, name, kind ('device' or 'generic'), face, sections, controls and, for a device, match and sources.
Notes
A control marked verified: false (with a note saying why) has a message that comes from the manual or a guess and has not been confirmed on a real unit. The faceplate shows a small orange ? on its label, and Learn rebinds it in one move.
Examples
js
import { PROFILES } from'@openav/midi';
for (const p of PROFILES.slice(0, 4)) {
console.log(p.id, '|', p.short, '|', p.kind, '|', p.controls.length, 'controls');
}
console.log(PROFILES.filter((p) => p.kind === 'generic').map((p) => p.id));
module Prints the first four devices (id, short name, kind, control count), then the ids of the generic layouts.
Related
profileById()Looks up a shipped controller profile by its id ('akai-lpd8') or its short signal name ('lpd8'), or returns null.
normalizeProfile()Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
validateProfile()Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
matchProfile()Finds the device profile that owns a MIDI port name, such as 'Minilab3 MIDI', by testing each profile's match patterns.
Lists every signal name a profile can publish under its own namespace (midi/lpd8/… for the LPD8), for docs, tests and mapping screens.
For each control: midi/<short>/<id> and /raw; an encoder adds /delta; a pad adds /hit (and /pressure when it has aftertouch). A keys control has only /on, /off and one midi/<short>/n<note> per key. These are the names a MidiController publishes, so a route from controllerRoutes() or a signals.on() can use them.
Parameters
Name
Type
Description
p
Object
A profile (its short and controls are read).
Returns
string[] Signal names, starting with midi/<short>/last.
module Prints how many signals the LPD8 can publish and the first five: midi/lpd8/last, then pad 1's value, raw value and hit, then pad 2.
Related
controllerRoutes()Builds mapper routes for "this control drives that param": controllerRoutes(profile, { knob1: 'hue' }) routes the knob's signal to the param hue.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
groupsOf()Groups a profile's controls by their group (knob, pad, fader …), returning { group: [control ids] }.
Publishes a list of { name, value, pulse?, min?, max? } into a Signals registry, declaring each signal the first time it is seen.
For a new continuous signal it defines the range from min and max (0 and 1 when missing) and source: 'midi'. A pulse is defined as kind: 'pulse'. Then it calls signals.pulse(name, value) or signals.set(name, value).
The Midi engine and MidiController already do this for you. Call it yourself when you read MIDI some other way, for example from a WebSocket, and want the same names in your show.
Parameters
Name
Type
Description
signals
Signals
The registry from @openav/core. null makes the call do nothing.
list
Object[]
Usually the result of genericSignals(). pulse: true fires a pulse, anything else sets a value.
Examples
js
import { Signals } from'@openav/core';
import { parseMessage, genericSignals, publish } from'@openav/midi';
const signals = new Signals();
signals.on('midi/cc/74', (v) => console.log('cc 74 is now', v.toFixed(2)));
// bytes from anywhere: a WebSocket, a file, a test
publish(signals, genericSignals(parseMessage([0xb0, 74, 100])));
console.log(signals.get('midi/ch/1/cc/74'));
module Prints cc 74 is now 0.79, then the same value read from the per-channel name.
Related
genericSignals()Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
parseMessage()Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
The four ways an endless encoder can write a turn as a CC value: offset64, offset16, twos and signbit.
Use it to check or list the encodings, for example when you build a profile editor. validateProfile() reports an encoder whose relative is not in this list.
Type
string[]
Range
['offset64', 'offset16', 'twos', 'signbit']
A plain array. relativeDelta() and relativeValue() take one of these as mode, and a profile's relative field must be one of them (or false for an absolute encoder).
Turns the CC value an endless encoder sends into signed steps: with offset64, 65 is +1, 63 is −1 and 64 is no move.
An endless encoder has no position, so it sends how far it moved, and devices disagree on how to write it. offset64: 64 is no move, 65 is +1, 63 is −1. offset16: 16 is no move, 17 is +1, 15 is −1. twos: 1..63 are +1..+63 and 127..65 are −1..−63 (two's complement). signbit: 1..63 are positive, 65..127 are −1..−63 (sign and magnitude).
An unknown mode throws unknown relative mode "x". MidiController uses this for every encoder control whose profile says relative, and publishes the result as midi/<short>/<id>/delta.
Parameters
Name
Type
Description
value
Number
The CC value, 0..127 (only the low 7 bits are read).
mode
String
How the device writes a turn: one of RELATIVE_MODES. A profile names it in the control's relative field. optionaldefault 'offset64'
Returns
Number The signed step count, positive for clockwise.
module Prints 1, −1, −1 and −3: the same kind of turn written four ways.
Related
relativeValue()The inverse of relativeDelta(): the CC value that says "turned by this many steps" in a given encoder encoding.
RELATIVE_MODESThe four ways an endless encoder can write a turn as a CC value: offset64, offset16, twos and signbit.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
The inverse of relativeDelta(): the CC value that says "turned by this many steps" in a given encoder encoding.
A virtual encoder sends the same encoding a hardware one does, so the on-screen controller and the real device look identical to everything downstream. relativeValue(-1) is 63 and relativeValue(3, 'twos') is 3. With offset16 the result is held at 0, so it can only go down to −16.
Decides what to send to after the port list changed or the current port died, and returns the one line to print about it.
It switches only when the current port is gone or disconnected (to the best live one, or to none), or when the port you named is back while a stand-in is in use (back to it). In every other case it stays: plugging in an unrelated device never steals the output. The text is plain ASCII, such as MIDI out: switched to "Synth" ("IAC Driver Bus 1" was unplugged).
This is the rule behind the Midi engine's output hot-plug. It never touches Web MIDI, so a test can call it with plain objects.
Parameters
Name
Type
Description
current
MIDIOutput | null
The port in use now. optional
outputs
MIDIOutput[]
The ports there are now. optional
preferred
String
The port you chose by name (the one to go back to). optionaldefault ''
prefer
Array<String | RegExp>
Fallback order, as in pickOutput(). optionaldefault []
Returns
{ next, changed, reason, text }next is the port to use. changed says whether it differs from current. reason is same, found, fallback, back or none. text is the line to log (an empty string when nothing changed).
Examples
js
import { reselectOutput } from'@openav/midi';
const iac = { name: 'IAC Driver Bus 1', state: 'connected' };
const synth = { name: 'Synth', state: 'connected' };
// the IAC bus is unplugged: only the synth is leftconst r = reselectOutput({ current: iac, outputs: [synth], preferred: 'IAC Driver Bus 1' });
console.log(r.reason, r.changed, r.next.name);
console.log(r.text);
// the IAC bus comes back while the synth stands in: go back to your choiceconst back = reselectOutput({ current: synth, outputs: [synth, iac], preferred: 'IAC Driver Bus 1' });
console.log(back.reason, back.next.name);
module Prints fallback true Synth and its sentence, then back IAC Driver Bus 1.
Related
pickOutput()Chooses which MIDI output to send to from a list: the port you named, else the first match of a preference list, else the first live port.
isLive()Says whether a MIDI port can still be sent to: it exists and its state is not 'disconnected'.
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
Control ids a profile may not use, because they are signal suffixes: last, raw, delta, hit, on, off, pressure.
A profile's short has a separate reserved list (cc, note, bend, ch, virtual, raw, clock) because those are the generic midi/… families. Both are checked by validateProfile().
A plain array. last is the signal midi/<short>/last, and the others are the suffixes a control's signals use (/raw, /delta, /hit, /on, /off, /pressure), so validateProfile() rejects them as control ids.
Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.
Errors are written for a person: no MIDI profile given, unknown MIDI profile "x" (known: …), could not load MIDI profile <url> (404) and invalid MIDI profile (<where>): followed by the list from validateProfile().
Parameters
Name
Type
Description
ref
String | Object
A profile object (checked and returned), an id or short name (looked up in profiles), or a URL (fetched). A URL is a string starting with http:, https:, blob:, data:, /, ./ or ../, or ending in .json.
profiles
Object[]
Where ids are looked up. The default is an empty list, so pass PROFILES (or use loadProfile(), which does). optionaldefault []
fetch
Function
Replaces fetch, for tests or special loading. It must return { ok, status?, json() }. optionaldefault globalThis.fetch
base
String
A base URL that a relative path is resolved against. optionaldefault undefined
module Prints korg-nanokontrol2, akai-lpd8, then the error header and the first two problems of an invalid object.
Related
loadProfile()Loads a controller profile by id, short name or URL (a JSON file anywhere CORS allows), checking any file or object you give it; returns a promise.
findProfile()Finds a profile in a list by id or short name, or returns null; no validation, no loading.
isProfileUrl()Tells a URL to a profile file apart from a profile id: true for ./pads.json, https://…, /p/x, false for 'akai-lpd8'.
validateProfile()Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
Turns a show's segment changes and cues into MIDI notes and CCs on one channel, so a DAW or lighting desk can follow the show.
The score knows when a segment starts and when a cue passes. An outside program cannot read the score, but it can read MIDI: launch a clip on a note, jump a scene on a CC. Create a ScoreMidi, then call out.segment({ index, cause }) and out.cue({ name, n }) (createShow({ score: { …, midi: { channel: 15 } } }) does this for you).
Markers go out only while the show plays (cause is play or release). A reset never sends, and seek and jump send only with scrub: true. Everything goes through the engine's own noteOn, noteOff and cc, so the MIDI OUT meter shows it and panic() can silence it. Off unless you create it.
Parameters
Name
Type
Description
midi
Midi
What to send through: a Midi (anything with noteOn, noteOff and cc). Required: without it the constructor throws. optional
channel
Number
The MIDI channel, 1..16 (anything else throws). optionaldefault 15
segment
false | Object
{ note, cc, velocity }: segment number i sends note note + i and/or CC cc with value i. velocity defaults to 100, note to 36 (give note: null to send only the CC). false sends no segment markers. optionaldefault { note: 36 }
cue
false | Object
{ note, notes, cc, velocity }: every cue sends note (or its own note from the map notes, by cue name) and/or CC cc with the cue's number as value. note defaults to 84 (give note: null to send only the CC). false sends no cue markers. optionaldefault { note: 84 }
length
Number
Seconds before a marker note's note-off. A marker note is a trigger, not a held note. optionaldefault 0.15
scrub
Boolean
Also mark jumps the playhead made by hand (seek, jump). Off by default, so dragging the scrubber does not launch clips in the DAW. optionaldefault false
enabled
Boolean
Set to false (or later, out.enabled = false) to send nothing. optionaldefault true
timer
Function
(fn, ms) => … used to schedule the note-offs. It is setTimeout; pass your own in a test. optionaldefault (fn, ms) => setTimeout(fn, ms)
Returns what a segment change would send, [{ type: 'note' | 'cc', number, value }], without sending it. An empty list when markers are off for this cause.
Sends the markers for a cue that was passed. evt is { name, n }: the cue's name and its order in the cut. Returns the messages it sent.
Examples
js
import { ScoreMidi } from'@openav/midi';
// anything with noteOn / noteOff / cc will do; here it just printsconst midi = {
noteOn: (note, vel, ch) => console.log('note on ', note, vel, 'ch', ch),
noteOff: (note, ch) => console.log('note off', note, 'ch', ch),
cc: (cc, val, ch) => console.log('cc', cc, '=', val, 'ch', ch),
};
const out = new ScoreMidi({
midi, channel: 15,
segment: { note: 36, cc: 20 },
cue: { note: 84, notes: { 'dusk.glow': 90 }, cc: 21 },
});
out.segment({ index: 2, cause: 'play' }); // note 38 and CC 20 = 2
out.cue({ name: 'dusk.glow', n: 4 }); // note 90 and CC 21 = 4
out.segment({ index: 1, cause: 'seek' }); // nothing: scrub is off
module Prints note 38 and CC 20 = 2 for segment 2, note 90 and CC 21 = 4 for the cue, nothing for the seek, and the note-offs 0.15 s later.
Related
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
Turns a −1..1 number into a 14-bit pitch-bend value (0..16383), the inverse of bendToUnit().
It does not hold u to −1..1, so clamp it first if it can overshoot. encodeMessage() clamps the final value into 0..16383 anyway, so a pitch-bend message built from an overshoot stays valid.
Parameters
Name
Type
Description
u
Number
A position from −1 (fully down) to 1 (fully up). 0 is the centre.
Returns
Number The rounded 14-bit value: unitToBend(0) is 8192, unitToBend(1) is 16383.
Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
It reports all problems at once, not just the first. It checks: id and short are lower-case [a-z0-9_-] and short is not a reserved family name (cc, note, bend, ch, virtual, raw, clock), name and kind exist, a device cites at least one source, match regexes compile, face has a size, every control has a unique non-reserved id (see RESERVED_IDS), a known type and msg, numbers in range and no two controls listening to the same message, encoders name their relative mode, and the sections (and portrait, if given) show every control exactly once, inside the faceplate and within their grid.
This is the check behind normalizeProfile() (it throws with this list), resolveProfile() and the test that validates every shipped profile. Run it on your own file before you try it in a page.
Parameters
Name
Type
Description
p
Object
A raw profile object (for example parsed from your JSON file).
Returns
string[] The problems, in plain words. [] is valid.
module Prints [] for a shipped profile, then a list of what is wrong with the made-up one (bad id, reserved short name, no source, CC out of range, control not shown).
Related
normalizeProfile()Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
resolveProfile()Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.
CONTROL_TYPESThe kinds of control a profile may contain: knob, encoder, fader, pad, button, keys, wheel, strip.
RESERVED_IDSControl ids a profile may not use, because they are signal suffixes: last, raw, delta, hit, on, off, pressure.
virtualRequestMIDIAccess({ real = null, make, waitMs = 1200 })
Wraps navigator.requestMIDIAccess so every caller gets the virtual access: real ports join if they answer in time, and the virtual input stays without Web MIDI.
If the real request answers within waitMs (1200 ms by default), its ports are there from the start. If it answers later they hot-plug in as statechange events. If it is refused or missing (Safari, iPad), the virtual input is still there. A permission prompt nobody answers cannot hang the sketch: it gets the access after waitMs anyway.
Parameters
Name
Type
Description
real
Function | null
The original requestMIDIAccess, already bound (navigator.requestMIDIAccess.bind(navigator)), or null when there is none. The real request still happens, permission prompt and all. optionaldefault null
make
Function
(opts) => … returning the object of createVirtualMIDIAccess(). Required. Return the same one each time to share it. optional
waitMs
Number
How long to wait for the real request before answering anyway. optionaldefault 1200
Returns
Function A function (opts?) => Promise<MIDIAccess> to put where navigator.requestMIDIAccess was.
module Prints ['Demo MIDI'], then true 1: the caller got the virtual access, and it was asked for once.
js
import { createVirtualMIDIAccess, virtualRequestMIDIAccess } from'@openav/midi';
const virtual = createVirtualMIDIAccess();
navigator.requestMIDIAccess = virtualRequestMIDIAccess({
real: navigator.requestMIDIAccess ? navigator.requestMIDIAccess.bind(navigator) : null,
make: () => virtual,
});
// from now on every sketch on the page that asks for MIDI gets the virtual input too
static The real wiring: put it before the sketch runs and its old MIDI code needs no change.
Related
createVirtualMIDIAccess()Builds a Web MIDI MIDIAccess that always has one more input and output of its own, so a sketch written for a MIDI keyboard works with none plugged in.
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
Returns a copy of a profile with every control moved to another MIDI channel, for a unit that is set to a channel other than the profile's.
Explicit LED channels (led.ch) stay as they are, since LED feedback may use a different channel from the control. This is what the channel option of createController() and the channel attribute of <oav-controller> call.
Parameters
Name
Type
Description
profile
Object
A profile.
ch
Number
The channel, 1..16. A falsy value (0, null) returns the same profile object untouched.
Returns
Object A deep copy, validated, with channel and every control's ch set. The original is not changed.
module Prints 10 -> 3: the shipped profile keeps its channel, the copy has every control on channel 3.
Related
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
normalizeProfile()Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
The default look of ControllerCanvas: background, body, text, idle colours, accent, glow and fonts. Pass any subset as the theme option.
Colours are CSS colour strings. bg: null lets the video or page underneath show through, and accent: '#7ea6ff' makes the panel monochrome-with-one-colour, which suits a branded video.
Type
Object
A frozen object (you cannot change it; pass theme instead). Keys: bg (the area around the device; null leaves what is underneath), body (the faceplate; null uses the profile's face.body), ink (big text), label (control labels), dim (faceplate and section lettering), idle (a pad at rest), button, cap (knob caps), well (fader slots and strip wells), screen, screenInk, screenDim, accent (one colour for everything that moves; null keeps each control's own), glow (0 flat to 1 like the on-screen controller) and the fonts font and mono.
module Prints the defaults (#060709 null 1), then the panel's theme with your three keys replaced and everything else (such as ink) kept.
Related
ControllerCanvasDraws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
controllerGeometry()Computes where everything goes when a profile's faceplate is drawn into a pixel rectangle: the face, every section, every control cell, and the notes strip.
Builds the control event payload for one control change: id, value, raw, signal, the parsed message, its bytes and where it came from.
Always there: profile (the profile id), id, type, label, value, raw, signal, message, bytes, source and time (performance.now(), rounded). value is 0..1 (−1..1 for a bipolar wheel); a pad is its velocity while held and 0 on release; a button is 0 or 1; for keys it is that note's velocity. raw is the MIDI number (0..127, pitch bend 0..16383). signal is the name the rest of open-audiovisual uses (midi/lpd8/k1; for a key midi/minilab3/n60). message is the parsed MIDI ({ type: 'cc', ch: 1, cc: 70, value: 53 }) and bytes the raw bytes. source is 'ui' (a finger, a mouse, set()), 'hardware' (a real port, or bytes you ingest()) or 'mirror'.
Sometimes there: note and velocity (note messages, velocity 0 for a release), delta (a relative encoder: the signed steps), held (a pad or button), and pressure (a pad, when it is above 0). The control, noteon and noteoff events carry it as their detail.
module Prints the readout line midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1, then the bytes [176, 74, 53], ui and the signal name.
Related
describeDetail()Writes the readout line for a control event payload: midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1.
describeMessage()Writes one parsed MIDI message the way a person reads it: CC 70 = 53, Note 36 · vel 90, Bend 12288.
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
ControllerHostThe object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
toParent()Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
The on-screen controller is DOM: perfect to play, impossible to record, because a page cannot capture its own DOM into a video. This draws the same faceplate, every section where it sits on the device and in the same colours, straight into a 2D context. It is vector and drawn at the output size, so a 4K frame is as sharp as a 1080 one, and it costs a few hundred path calls a frame. @openav/record's Compositor#setPanel takes one.
Call panel.draw(ctx, { x, y, w, h }) every frame, or panel.renderTo(canvas) to fill a canvas of its own. What a video needs that a screen does not: a touch has to stay visible for a few frames (30 fps is 33 ms; a staccato note can be shorter), so released keys and pads fade out instead of switching off, the control a hand just moved glows, and the keyboard follows the device's octave buttons. panel.aspect is the faceplate's width ÷ height, for reserving an area of that shape. With neither signals nor controller the panel just stays at rest.
Parameters
Name
Type
Description
profile
String | Object
Profile id, short name or object. An unknown id throws ControllerCanvas: unknown profile "x". optionaldefault 'arturia-minilab3'
signals
Signals | null
Follow the generic MIDI names on this bus (midi/ch/<ch>/…, midi/note/on|off, midi/cc/<n>, midi/bend): live hardware, the on-screen controller, the QWERTY piano, phones and a recorded take playing back all move the panel. Each message counts once, though hardware publishes it under two names. optionaldefault null
controller
MidiController | null
Follow this existing controller instead of the bus: exactly that device, with its learned mapping. A recorded take does not reach it. When given, signals is ignored. optionaldefault null
mapping
Object | null
Learned bindings to apply to the panel's own controller: the exportMapping() of the live device. optionaldefault null
theme
Object
Any subset of CANVAS_THEME: colours, glow, fonts. optionaldefault {}
notes
Boolean | 'auto'
Write the held notes large under the faceplate. 'auto' does it when the area has room, true makes room, false never. optionaldefault 'auto'
screen
Boolean
The device's own screen (MiniLab 3, KeyStep) shows the last touch, as the hardware's does: Knob 3 · 87, C4 · 100. optionaldefault true
padding
Number
Space around the faceplate, as a share of the area's short side. optionaldefault 0.05
now
Function
The clock in milliseconds that the fades use; default performance.now(). The now you pass to draw() must be on the same clock. optionaldefault defaultNow
Draws into rect ({ x, y, w, h } in pixels) of a 2D context. Call it every frame. It saves and restores the context. now is on the same clock as the now option, in milliseconds.
It is a silent model: it publishes nothing back to the bus. panel.dispose() stops listening to the bus or controller (a controller you passed in is left as it was).
Examples
js
import { Signals } from'@openav/core';
import { ControllerCanvas, genericSignals, publish } from'@openav/midi';
const signals = new Signals();
const panel = new ControllerCanvas({ profile: 'minilab3', signals });
// a chord, a pad and a knob, published as the hardware wouldconst play = (ev) => publish(signals, genericSignals(ev));
play({ type: 'noteon', ch: 1, note: 60, vel: 100 });
play({ type: 'noteon', ch: 1, note: 64, vel: 100 });
play({ type: 'noteon', ch: 10, note: 36, vel: 127 });
play({ type: 'cc', ch: 1, cc: 74, value: 100 });
panel.renderTo(document.getElementById('c')); // fills the 400 x 300 canvas
module The Arturia MiniLab 3 is drawn into the canvas with C4 and E4 lit, pad 1 lit, and knob 1 turned up. The device screen reads Knob 1 · 100.
js
import { Signals } from'@openav/core';
import { ControllerCanvas, genericSignals, publish } from'@openav/midi';
const signals = new Signals();
const panel = new ControllerCanvas({ profile: 'lpd8', signals, theme: { accent: '#7ea6ff' } });
const ctx = document.getElementById('c').getContext('2d');
const send = (ev) => publish(signals, genericSignals(ev));
let frame = 0;
function tick() {
const pad = 36 + Math.floor(frame / 20) % 8;
if (frame % 20 === 0) send({ type: 'noteon', ch: 10, note: pad, vel: 110 }); // hit a padif (frame % 20 === 8) send({ type: 'noteoff', ch: 10, note: pad }); // let go: it fades out
panel.draw(ctx, { x: 0, y: 0, w: 400, h: 300 });
if (++frame < 120) requestAnimationFrame(tick);
}
tick();
module A different pad of the LPD8 lights every third of a second and then fades, all in one accent colour, as it would in a recorded video.
Related
controllerGeometry()Computes where everything goes when a profile's faceplate is drawn into a pixel rectangle: the face, every section, every control cell, and the notes strip.
CANVAS_THEMEThe default look of ControllerCanvas: background, body, text, idle colours, accent, glow and fonts. Pass any subset as the theme option.
followMidiSignals()Listens to a Signals bus and hands you each MIDI message back exactly once, as an event, although hardware publishes it under two names.
ControllerViewThe playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
eventOfSignal()The inverse of genericSignals(): turns one generic MIDI signal (midi/ch/10/note/36, midi/cc/74) back into the MIDI event it came from.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
Computes where everything goes when a profile's faceplate is drawn into a pixel rectangle: the face, every section, every control cell, and the notes strip.
The faceplate keeps its own proportions, centred in the rect. The sections are laid out the way the on-screen controller's CSS grid does (row or column flow, spans), so the canvas drawing and the DOM agree. It is pure, so it is the part to test, and the part to use when you want to draw something of your own on top of a control.
Parameters
Name
Type
Description
profile
Object
A normalized profile (MidiController#profile).
rect
{ x, y, w, h }
The area, in pixels.
padding
Number
Space around the faceplate, as a share of the rect's short side (0..1). optionaldefault 0.05
notes
Boolean | 'auto'
Reserve a strip under the faceplate for the held notes: true always, false never, 'auto' when the rect has room. optionaldefault 'auto'
Returns
{ u, face, notes, sections, controls, deco }u is the pixel size of one face unit. face is { x, y, w, h }. notes is the strip or null. sections is a list with id, label, the box and its grid. controls is a Map from control id to { id, type, section, x, y, w, h }. deco lists the decorations (screen, logo, lettering) in pixels.
Examples
js
import { controllerGeometry, normalizeProfile, profileById } from'@openav/midi';
const profile = normalizeProfile(profileById('lpd8'));
const g = controllerGeometry(profile, { x: 0, y: 0, w: 400, h: 300 });
console.log(g.face); // the faceplate, centred, in pixels
console.log(g.controls.get('pad1')); // where pad 1 is
console.log(g.controls.size, 'controls', g.notes ? 'with a notes strip' : 'without a notes strip');
module Prints the faceplate rectangle (370 wide, centred), pad 1's cell in pixels, then 16 controls with a notes strip.
Related
ControllerCanvasDraws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
keyboardGeometry()Computes the rectangles of piano keys from one note to another inside a rect: white keys side by side, black keys over the seams at 60% of the height.
normalizeProfile()Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
The object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
Events, with ctl.on(name, fn) (returns an unsubscribe function): control (every knob, pad, key or button change), noteon and noteoff (the same payload, for pads, keys and note buttons), connect and disconnect ({ port, profile }: a hardware port started or stopped driving it), status, profilechange ({ profile, name }) and '*', which gets (name, detail) for all of them. The control payload is the one described under controlDetail(). ready and error exist only on the <oav-controller> element.
ctl.status is { hardware, input, output, outputState }. hardware is off, asking, waiting (listening, but the device is not plugged in), connected, unavailable (the browser has no Web MIDI) or denied (permission refused). outputState is off, asking, open, missing (no port matches), unavailable or denied. ctl.profileId, ctl.controller and ctl.profile say which device is on screen; ctl.controllers, ctl.view, ctl.midi and ctl.output are the objects underneath.
What a hand plays on the screen leaves through the output port as real MIDI; moves of your real hardware are not forwarded (your software already hears the device), and input with the same name as the chosen output (the loopMIDI echo) is muted.
Before you call it
connectHardware(), setOutput() and requestMidi() are the only calls that ask the browser for MIDI. Chrome, Edge and Firefox show a permission prompt, so call them from a click when you can. Without Web MIDI (Safari, iPad) the controller still plays and fires events, and status says unavailable.
Parameters
Name
Type
Description
profile
String | Object
Profile id, short name or object. optionaldefault 'arturia-minilab3'
profiles
Object[]
The profiles a picker or follow may switch to. Default: every shipped one. optionaldefault PROFILES
signals
Signals | null
Also publish midi/<short>/… into a show. optionaldefault null
sink
Function | null
Publish through this function instead (a relay), as in MidiController. optionaldefault null
channel
Number | null
Move every control to this MIDI channel (your unit is set to another one). See withChannel(). optionaldefault null
follow
Boolean
When a different known device is plugged in, show that one. optionaldefault false
midi
Midi | null
Use this engine (a show's) instead of the page-shared one. optionaldefault null
requestAccess
Function | null
Where the MIDIAccess comes from, as in Midi. Giving it makes the controller use its own engine. optionaldefault null
filterOut
String | RegExp
The input-port filter of Midi. Anything but the default 'IAC' also gives the controller its own engine. optionaldefault 'IAC'
storage
Storage | null
Where learned mappings are kept, as in MidiControllers. Default: localStorage when available. optionaldefault undefined
telemetry
Boolean
false sends no anonymous page-load hit. optionaldefault true
embedKind
String
Which kind of embed this is for the hit: 'headless' (default), 'element' or 'iframe'. optionaldefault 'headless'
Moves a control like a hand: knobs and faders 0..1, bipolar wheels −1..1, pads and buttons above 0 press (with that velocity) and 0 release, keys set('keys', velocity, { note: 60 }). With quiet: true only the faceplate moves: no event and no MIDI out (two-way binding). Returns the bytes sent, or null when nothing changed. See setControl().
Lets go of everything and brings every control home (knobs and faders to 0, bipolar wheels to the centre), sending what a hand would. With quiet: true it only moves the faceplate.
MIDI arriving from anywhere (a WebSocket, another page, a port you manage): the faceplate follows. One message ([0xb0, 74, 64]) or a list of them. source is 'hardware' (default), 'ui' or 'quiet'. Returns the parsed event, or a list of them.
Draws the playable faceplate into el; it fills el and follows its size, so give el a height. Returns the ControllerView. layout is auto, face or stack.
Learn mode: tap a control on the screen, then move the one on your hardware (or ingest() it). Toggles by default; learn(true) and learn(false) set it. Returns the new state.
Shows another device: an id, short name, URL or profile object. Held notes are released first. Resolves with the new controller and fires profilechange. Rejects with the reason when the profile is unknown or invalid.
Follows the real device: its port drives the on-screen one (asks the browser for Web MIDI). port is a name or /regex/ for routing a renamed or unknown device by hand. Resolves true when MIDI is available, false otherwise; check status.hardware for connected or waiting.
Sends what is played here to a MIDI output: a name, a substring ('IAC', 'loopMIDI') or a /regex/, matched as in findPort(). null turns it off. Resolves with the port name, or null when there is none (status.outputState says why).
Removes everything this controller added: listeners, the view, its share of the MIDI engine. Held notes are released first.
Examples
js
import { ControllerHost } from'@openav/midi';
// same as createController('lpd8'); the class is exported for instanceof and subclassingconst ctl = new ControllerHost('lpd8', { channel: 3 });
console.log(ctl instanceof ControllerHost, ctl.profileId, ctl.status.hardware);
console.log(ctl.set('k1', 1)); // [178, 70, 127]: channel 3 is status byte 0xb2await ctl.setProfile('nano2'); // show another device
console.log(ctl.profileId, ctl.profile.controls.every((c) => c.ch === 3));
module Prints true akai-lpd8 off, then the bytes of knob 1 on channel 3, then the new device with the channel override kept.
Related
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
ControllerViewThe playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
controlDetail()Builds the control event payload for one control change: id, value, raw, signal, the parsed message, its bytes and where it came from.
The playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
Two shapes, chosen from the box it lives in. face: the device as it looks, sections where they sit on the real faceplate, scaled to fit (desktop, iPad, a phone held sideways). stack: the same controls re-gridded top to bottom for a tall box (a phone held upright), as the profile's portrait says. Nothing scrolls and nothing shrinks below a thumb. Square grid devices (Launchpad) stay face. mode fixes one; 'auto' follows the box.
Every control repaints from the controller's state, so a finger here and a knob on the desk look the same, and a move from hardware gets a brief glow ring so you can see your hand arrive. Touch: drag a knob (right or up raises it), drag or click a fader or strip, strike a pad (higher on the pad is softer, lower is harder), hold a button, play the keys (multi-touch works). The mouse wheel turns knobs and faders.
Keyboard: Tab reaches every control except the piano keys; the arrow keys, Page Up/Down, Home and End move knobs, faders and strips (Shift for fine steps); Space or Enter strikes pads and buttons, held while the key is held. Learn mode (setLearning(true)) turns every tap into "bind this control", and view.onLearnTap = (id) => … tells you which.
Before you call it
A browser DOM and a container with a height. createController(…).mount(el) is the usual way to get one.
Parameters
Name
Type
Description
container
HTMLElement
The element to fill. It needs a size: the view is 100% wide and tall, so give it a height. It can be inside a shadow root: the view's CSS goes into that root, not the page.
controller
MidiController
The model to show and drive.
mode
String
'auto', 'face' or 'stack' (below). optionaldefault 'auto'
minTouch
Number
Pixels a control needs before face gives way to stack. optionaldefault 38
Stops listening, disconnects the resize observer and removes the view from the page.
Examples
js
import { ControllerView, MidiController, profileById } from'@openav/midi';
const app = document.getElementById('app');
app.style.height = '240px';
const lpd8 = new MidiController(profileById('lpd8'));
const view = new ControllerView(app, lpd8); // fills #app
lpd8.press('pad1'); // pad 1 lights up
setTimeout(() => lpd8.release('pad1'), 600);
module The AKAI LPD8 appears in the page and pad 1 lights for a moment. Try dragging a knob or tapping the pads.
js
import { ControllerView, MidiController, profileById } from'@openav/midi';
const app = document.getElementById('app');
app.style.height = '360px';
app.style.width = '220px'; // a tall, narrow boxconst view = new ControllerView(app, new MidiController(profileById('lpd8')));
console.log(view.mode); // 'stack': re-gridded for a tall box
view.setMode('face');
console.log(view.mode); // 'face': the device as it looks
module Prints stack, then face after forcing the layout: the same controls in the two shapes.
Related
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
ControllerCanvasDraws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
mountMidiPanel()Mounts a controller panel: a toolbar (device picker, hardware status, port picker, Learn, last message) above a playable faceplate, docked at the bottom or full size.
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
VIEW_CSSThe stylesheet text of the on-screen controller (ControllerView), as a string.
createController(profile = 'arturia-minilab3', o = {})
Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
Four lines are enough: const ctl = createController('akai-lpd8'), ctl.mount(el) for the playable faceplate (optional, it works headless too), ctl.on('control', fn) to hear every knob, pad and key whether it came from the screen or the hardware, and ctl.connectHardware() to follow the real device. ctl.set(id, value) moves a control the way a hand would.
Controllers on one page share one Midi engine, so the browser asks for MIDI permission once. Nothing asks until you call connectHardware(), setOutput() or requestMidi(). Pass signals to publish midi/<short>/… into a show.
Parameters
Name
Type
Description
profile
String | Object
A profile id ('akai-lpd8'), a short name ('lpd8') or a profile object. For a URL, load it first with loadProfile(). An unknown id throws unknown MIDI profile "x" (known: …). optionaldefault 'arturia-minilab3'
o
Object
Options, the same as the second argument of ControllerHost: profiles, signals, sink, channel, follow, midi, requestAccess, filterOut, storage, telemetry, embedKind. optionaldefault {}
Returns
ControllerHost The controller. Its methods are listed under ControllerHost.
Notes
Like <oav-controller>, it sends one anonymous hit per page load (which kind of embed and which device; nothing anyone plays) to the project's analytics, and nothing from localhost or the LAN. Turn it off with createController(profile, { telemetry: false }). The README's Telemetry section has the details.
Examples
js
import { createController } from'@openav/midi';
const ctl = createController('akai-lpd8');
ctl.on('control', (e) => console.log(e.signal, e.value.toFixed(2), '<-', e.source));
ctl.set('k1', 0.5); // a hand on knob 1
ctl.set('pad1', 1); // press pad 1
ctl.set('pad1', 0); // let go
console.log(ctl.values().k1, ctl.get('nope'));
module Prints three control events (knob 0.50, pad 1.00, pad 0.00, each from 'ui'), then the knob's value and null for an id that does not exist.
js
import { createController } from'@openav/midi';
// MIDI out to a DAW: a stand-in for the browser's Web MIDI shows what would be sentconst iac = { name: 'IAC Driver Bus 1', state: 'connected', send: (bytes) => console.log('to the DAW', bytes) };
const requestAccess = async () => ({ inputs: new Map(), outputs: new Map([['o', iac]]) });
const ctl = createController('akai-lpd8', { requestAccess });
console.log(await ctl.setOutput('IAC')); // IAC Driver Bus 1
ctl.set('pad1', 1); // to the DAW [153, 36, 127]
ctl.set('pad1', 0); // to the DAW [137, 36, 0]
ctl.ingest([0x99, 37, 90]); // hardware-style input: the faceplate follows, nothing is sent out
module Prints the port name, then the two messages a hand on the screen sends. The last line prints nothing: what arrives from hardware is not forwarded.
module The MiniLab 3 appears in the page with knob 1 turned up, and the console prints its signal value (0.80…). Play it with the mouse.
Related
ControllerHostThe object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
loadProfile()Loads a controller profile by id, short name or URL (a JSON file anywhere CORS allows), checking any file or object you give it; returns a promise.
MidiThe Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
ControllerViewThe playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
Registers the controller element under another tag name (importing @openav/midi/element already registers it as oav-controller).
A tag name can be registered only once per page, so use this when oav-controller clashes with another library, or when you want two differently-named copies. The default tag registers the exported OavController class itself; any other name registers a subclass of it.
Parameters
Name
Type
Description
tag
String
The new tag name. It must contain a hyphen, as every custom element does. optionaldefault 'oav-controller'
Returns
Function | null The element class registered for that tag (an already-registered tag returns what is there). null when the page has no customElements.
Examples
js
import { defineController } from'@openav/midi/element';
defineController('my-pads'); // now <my-pads> works tooconst app = document.getElementById('app');
app.innerHTML = '<my-pads profile="generic-16p"></my-pads>';
const el = app.firstElementChild;
await el.ready;
console.log(el.tagName, el.profileData.id); // MY-PADS generic-16p
module A 16-pad generic controller appears from a tag you named yourself; the console prints MY-PADS generic-16p.
Related
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
Writes the readout line for a control event payload: midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1.
This is the text <oav-controller readout> shows under the faceplate. Use it for your own log or caption. A payload without a message shows only the signal and the value.
Parameters
Name
Type
Description
d
Object | null
A payload from controlDetail() (or a control event's detail). null gives an empty string.
Returns
String The signal name, the value (two decimals, or a whole number from 100 up), then ←, the message as describeMessage() writes it and the channel.
Writes one parsed MIDI message the way a person reads it: CC 70 = 53, Note 36 · vel 90, Bend 12288.
These are the shorter lines of the <oav-controller> readout. describe() writes the longer log lines (CC ch1 #70=53).
Parameters
Name
Type
Description
m
Object | null
An event from parseMessage(). null gives an empty string.
Returns
StringCC 70 = 53, Note 36 · vel 90, Note 36 off, Bend 12288 (the raw 14-bit value), Pressure 50, Poly AT 36 = 12, Program 5; any other type gives its name.
module Prints CC 70 = 53, Note 36 · vel 90, Note 36 off and Bend 12288.
Related
describeDetail()Writes the readout line for a control event payload: midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1.
describe()Returns a short readable line for a parsed MIDI event, for logs and learn screens: CC ch1 #74=64, Bend ch1 0.50.
parseMessage()Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
The attributes the oav-controller element reads, which are also the query parameters of the iframe page: profile, layout, hardware, midi-out and more.
The iframe page copies every one of these from its URL onto the element. The extra parameters bg, pad, frame, origin and lang belong to the iframe page and are read by parseEmbedOptions(). What each attribute means is under OavController.
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
parseEmbedOptions()Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
EMBED_EVENTSThe nine events the oav-controller element dispatches and the iframe posts to its page, from ready and control to profilechange and error.
The nine events the oav-controller element dispatches and the iframe posts to its page, from ready and control to profilechange and error.
The iframe also posts a tenth type, values, as the answer to the get command; it is not an element event, so it is not in this list. The payloads are described under OavController and controlDetail().
A plain array, in this order. The element dispatches each as a CustomEvent that bubbles and is composed; the iframe page forwards each to window.parent with toParent().
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
toParent()Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
controlDetail()Builds the control event payload for one control change: id, value, raw, signal, the parsed message, its bytes and where it came from.
EMBED_ATTRSThe attributes the oav-controller element reads, which are also the query parameters of the iframe page: profile, layout, hardware, midi-out and more.
The three layouts a controller can have: auto, face (the device as it looks) and stack (re-gridded for a tall box).
A profile can opt out of stack (stack: false); square grid devices such as the Launchpad always stay face. See preferredAspect().
Type
string[]
Range
['auto', 'face', 'stack']
A plain array. parseEmbedOptions() turns any other layout value into 'auto', and the layout command of the iframe accepts only these.
Examples
js
import { EMBED_LAYOUTS, parseEmbedOptions } from'@openav/midi/embed';
console.log(EMBED_LAYOUTS);
console.log(parseEmbedOptions({ layout: 'stack' }).layout); // stack
console.log(parseEmbedOptions({ layout: 'sideways' }).layout); // auto: not in the list
module Prints the three names, then stack, then auto.
Related
parseEmbedOptions()Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
preferredAspect()Works out the width ÷ height a controller wants in a box of a given width: its faceplate when every control stays finger-sized, a taller stack otherwise.
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
withChannel()Returns a copy of a profile with every control moved to another MIDI channel, for a unit that is set to a channel other than the profile's.
parseEmbedOptions()Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
Checks the bg colour of the iframe page and returns a safe value: transparent, a hex colour, or a plain colour name; anything else is null.
The iframe page puts the result into a CSS variable, so a value that could break out of CSS (red;}body{) is refused. It does not check that a name is a real colour: any 3 to 20 letters pass, and CSS ignores the ones it does not know.
Parameters
Name
Type
Description
v
String
The value from the query string.
Returns
String | null'transparent'; '#' plus 3, 4, 6 or 8 hex digits (the # is optional in the input, so abc becomes #abc); or 3 to 20 letters, lower-cased ('RED' becomes 'red'); otherwise null.
module Prints transparent #0b0b0c #000 red, then three nulls.
Related
parseEmbedOptions()Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
Reads a boolean attribute: present means on, unless it says false, 0, off or no.
This is why <oav-controller readout> and readout="true" both turn the readout on and readout="off" turns it off. Any other word counts as on, so readout="ask" is true.
Parameters
Name
Type
Description
v
any
The attribute value: null/undefined/false (absent) or a string. A bare attribute is the empty string.
Returns
Boolean Absent is false; true is true; a string is true unless it is (case-insensitively, trimmed) false, 0, off or no.
parseEmbedOptions()Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
embedMode()Reads hardware and midi-out: off, ask (a button, so the permission prompt waits for a click) or on, plus which port.
Reads hardware and midi-out: off, ask (a button, so the permission prompt waits for a click) or on, plus which port.
Absent or false/0/off/no is off. ask is ask. on, true, yes, auto is on without a port. Any other text is on, and the text is the port name or /regex/ to look for (see findPort()): embedMode('IAC') is { mode: 'on', port: 'IAC' }.
Parameters
Name
Type
Description
v
any
The attribute value.
bare
String
What a bare attribute (no value) means: 'on' for hardware (the default), 'ask' for midi-out, because there is no sensible default port to send to. optionaldefault 'on'
Returns
{ mode, port }mode is 'off', 'ask' or 'on'. port is a string only when the value names one.
module Prints four { mode, port } objects: off, on, ask, and on with a named port.
Related
parseEmbedOptions()Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
embedFlag()Reads a boolean attribute: present means on, unless it says false, 0, off or no.
findPort()Finds a MIDI port by name: a /regex/flags, then the exact name, then a case-insensitive substring; null when nothing matches.
The brightness of a fading touch: 1 at the moment it happened, 0 after ms, eased so it still reads on a 30 fps frame.
ControllerCanvas fades released keys (180 ms), pads (450 ms), the glow of a touched control (650 ms), the last-touch text under the faceplate (1600 ms) and on the device screen (3200 ms) with it. A video frame is 33 ms at 30 fps, so a staccato note that was 20 ms long would be gone before the next frame without a fade.
Parameters
Name
Type
Description
now
Number
The current time in milliseconds.
at
Number | null
When the touch happened, on the same clock. null or undefined gives 0.
ms
Number
How long the fade lasts. Not above 0 gives 0.
Returns
Number From 1 down to 0. The curve is the square of the remaining time, so it falls quickly at first.
Examples
js
import { fadeLevel } from'@openav/midi';
for (const now of [100, 200, 300, 400, 500]) console.log(now, fadeLevel(now, 100, 400));
module Prints 1 at the moment of the touch, then 0.5625, 0.25, 0.0625 and 0 at the end of the fade.
Related
ControllerCanvasDraws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
Listens to a Signals bus and hands you each MIDI message back exactly once, as an event, although hardware publishes it under two names.
Hardware and the on-screen controllers publish every message twice: the legacy name (midi/note/on) and then the per-channel one (midi/ch/1/note/60) in one call. The QWERTY piano and some takes publish only the legacy name. A per-channel name counts at once; a legacy name waits for a microtask and is dropped if its per-channel twin shows up first. So a relative encoder never turns twice, a legacy CC (which has no channel) never lands on the wrong channel, and order is kept.
ControllerCanvas uses it. Use it yourself to turn a bus back into MIDI events, for example to drive your own visual from a recorded take.
Parameters
Name
Type
Description
signals
Signals
The bus. It needs onAny(cb).
onEvent
Function
(ev) => … called with an event of parseMessage() for each message.
channel
Number
The channel given to legacy CC and bend names, which carry none. optionaldefault 1
defer
Function
(fn) => … that schedules the flush of legacy-only names. Default queueMicrotask; replaceable in tests. optionaldefault (fn) => queueMicrotask(fn)
Returns
{ flush(), dispose() }flush() delivers legacy-only messages that are still waiting. dispose() stops listening and drops them.
Examples
js
import { Signals } from'@openav/core';
import { followMidiSignals, genericSignals, publish } from'@openav/midi';
const signals = new Signals();
const follower = followMidiSignals(signals, (ev) => console.log(ev.type, ev.note ?? ev.cc, ev.ch));
// the hardware publishes a message under both names; the follower reports it once
publish(signals, genericSignals({ type: 'noteon', ch: 2, note: 60, vel: 100 }));
publish(signals, genericSignals({ type: 'cc', ch: 2, cc: 74, value: 64 }));
// the QWERTY piano publishes only the short name: it arrives after a microtask, or at flush()
signals.pulse('midi/note/on', { note: 64, vel: 0.8, ch: 0 });
follower.flush();
follower.dispose();
module Prints three lines: noteon 60 2, cc 74 2, then noteon 64 1 for the piano (channel 0 becomes the default channel 1).
Related
eventOfSignal()The inverse of genericSignals(): turns one generic MIDI signal (midi/ch/10/note/36, midi/cc/74) back into the MIDI event it came from.
genericSignals()Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
ControllerCanvasDraws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
The commands a page may send into an embed iframe: set, ingest, reset, profile, layout, get, press, release.
What each does: set{ id, value, note?, quiet? } moves a control, press{ id, velocity?, note? } and release{ id, note? } hold and let go, ingest{ bytes } feeds MIDI (one message or a list), reset brings everything home, profile{ profile } switches the device, layout{ layout } changes the layout, and get asks for every value (the iframe answers with a values message).
A plain array. fromParent() ignores a message whose type is not one of these.
Examples
js
import { FRAME_COMMANDS, toFrame } from'@openav/midi/embed';
console.log(FRAME_COMMANDS.join(' '));
console.log(toFrame('set', { id: 'k1', value: 0.5 })); // the message you post to the iframe
module Prints the eight commands, then the message { target: 'openav', v: 1, type: 'set', id: 'k1', value: 0.5 }.
Related
fromParent()Frame side: validates a command from the page and returns a clean command, or null (the message is ignored).
toFrame()Page side: builds a command to post into an embed iframe: { target: 'openav', v: 1, type, ...args }.
EMBED_EVENTSThe nine events the oav-controller element dispatches and the iframe posts to its page, from ready and control to profilechange and error.
Page side: reads a message that came from an embed iframe, returns { type, frame, detail, v }, or null when it is not one of ours.
Other scripts and extensions post to a page all the time, so always filter. Check event.origin too when the page cares who sent it: this function checks the shape, not the sender.
Parameters
Name
Type
Description
data
any
The event.data of a message event.
Returns
{ type, frame, detail, v } | nullnull unless data is an object with source: 'openav' and a string type.
Examples
js
import { fromFrame, toParent } from'@openav/midi/embed';
// what a page does with every 'message' event; here the message is made by handconst data = toParent('control', { id: 'k1', signal: 'midi/lpd8/k1', value: 1 }, { frame: 'pads' });
const msg = fromFrame(data);
if (msg && msg.type === 'control') console.log(msg.frame, msg.detail.signal, msg.detail.value);
console.log(fromFrame({ type: 'control' }), fromFrame('hello')); // null null
module Prints pads midi/lpd8/k1 1, then null null for messages that are not ours.
js
window.addEventListener('message', (e) => {
const msg = fromFrame(e.data);
if (!msg) return; // not from an openav iframeif (msg.frame === 'pads' && msg.type === 'control') {
console.log(msg.detail.id, msg.detail.value);
}
});
static The listener to put on the page that holds the iframe.
Related
toParent()Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
toFrame()Page side: builds a command to post into an embed iframe: { target: 'openav', v: 1, type, ...args }.
fromParent()Frame side: validates a command from the page and returns a clean command, or null (the message is ignored).
POST_SOURCEThe word that marks a message as an open-audiovisual embed message: 'openav', sent as source by the iframe and as target by the page.
Frame side: validates a command from the page and returns a clean command, or null (the message is ignored).
It is strict, because anything on the web can post to an iframe. The message needs target: 'openav' and a type from FRAME_COMMANDS. Control ids must be lower-case [a-z0-9_-]. value must be a finite number. note must be an integer 0..127 (else null). A press velocity is held to 0..1. ingest takes one message of 1 to 3 bytes whose first byte is a status byte (128 or more), or up to 256 such messages. profile takes an id or an https:// URL (not http://). layout must be in EMBED_LAYOUTS.
Parameters
Name
Type
Description
data
any
The event.data of a message event.
Returns
Object | null A command such as { type: 'set', id: 'k1', value: 0.5, note: null, quiet: false }, or null.
Examples
js
import { fromParent, toFrame } from'@openav/midi/embed';
const show = (data) => console.log(JSON.stringify(fromParent(data)));
show(toFrame('set', { id: 'k1', value: '0.5' })); // value is read as a number
show(toFrame('press', { id: 'pad1', velocity: 2 })); // velocity held to 1
show(toFrame('ingest', { bytes: [0xb0, 70, 64] }));
// all of these are ignored: null
show({ source: 'openav', type: 'set', id: 'k1', value: 1 }); // wrong key: source, not target
show(toFrame('set', { id: '<img>', value: 1 })); // bad id
show(toFrame('profile', { profile: 'http://x.org/p.json' })); // http, not https
show(toFrame('eval', { code: 'alert(1)' })); // not a command
module Prints three clean commands, then four nulls for messages the frame would ignore.
Related
toFrame()Page side: builds a command to post into an embed iframe: { target: 'openav', v: 1, type, ...args }.
fromFrame()Page side: reads a message that came from an embed iframe, returns { type, frame, detail, v }, or null when it is not one of ours.
toParent()Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
FRAME_COMMANDSThe commands a page may send into an embed iframe: set, ingest, reset, profile, layout, get, press, release.
Computes the rectangles of piano keys from one note to another inside a rect: white keys side by side, black keys over the seams at 60% of the height.
A range never starts on a black key: a black key with no white key before it is left out, the way the on-screen keyboard does. White keys share the width equally, so 48 to 72 is 15 white keys and 10 black ones.
Parameters
Name
Type
Description
from
Number
The first MIDI note.
to
Number
The last MIDI note.
rect
{ x, y, w, h }
The area the keys fill.
Returns
{ note, white, x, y, w, h }[] One item per key, the white keys first and then the black ones (the draw order, so the black keys end up on top).
Examples
js
import { keyboardGeometry } from'@openav/midi';
const keys = keyboardGeometry(60, 72, { x: 0, y: 0, w: 400, h: 100 });
console.log(keys.filter((k) => k.white).length, 'white,', keys.filter((k) => !k.white).length, 'black');
console.log(keys[0].note, keys[0].x, keys[0].w); // 60 (C4) is the first white key
module Prints 8 white, 5 black (one octave and the next C), then the first key's note and its position.
Related
ControllerCanvasDraws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
keyWindow()Tells which note a drawn keyboard should start at after a note outside its printed range arrives, so it follows the device's octave buttons.
controllerGeometry()Computes where everything goes when a profile's faceplate is drawn into a pixel rectangle: the face, every section, every control cell, and the notes strip.
Tells which note a drawn keyboard should start at after a note outside its printed range arrives, so it follows the device's octave buttons.
A keyboard control catches notes outside its printed range, because the device's Oct+/Oct− buttons shift what the keys send. When one arrives, the drawing moves and stays there, as the device does. A 48 to 72 MiniLab with Oct+ pressed (a note of 76) shows 60 to 84. A range narrower than an octave just moves to reach the note.
Parameters
Name
Type
Description
from
Number
The first note of the printed range.
to
Number
The last note of the printed range.
base
Number
The first note shown now.
note
Number
A note that just went down.
Returns
Number The new first note: base when the note fits, otherwise base moved by the fewest octaves that reach it, held to the MIDI range.
Examples
js
import { keyWindow } from'@openav/midi';
console.log(keyWindow(48, 72, 48, 60)); // 48: inside the window, unchanged
console.log(keyWindow(48, 72, 48, 76)); // 60: Oct+ on the MiniLab
console.log(keyWindow(48, 72, 48, 40)); // 36: Oct-
console.log(keyWindow(48, 72, 48, 127)); // 103: held to the MIDI range
module Prints 48, 60, 36 and 103.
Related
keyboardGeometry()Computes the rectangles of piano keys from one note to another inside a rect: white keys side by side, black keys over the seams at 60% of the height.
ControllerCanvasDraws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
Turns a MIDI note number into its name with the octave, middle C being C4: midiNoteName(60) is 'C4', midiNoteName(61) is 'C#4'.
This is the on-screen keyboard's convention. Some software (Ableton Live, Yamaha) calls 60 "C3": the number is the same, only the octave label differs. In a p5 sketch, noteName() does the same and also writes flats when you ask.
Parameters
Name
Type
Description
n
Number
A MIDI note number, 0..127.
Returns
String Sharps only ('C#4', never 'Db4'). 0 is 'C-1' and 127 is 'G9'.
Mounts a controller panel: a toolbar (device picker, hardware status, port picker, Learn, last message) above a playable faceplate, docked at the bottom or full size.
The bar shows which device is on screen (the picker switches it), a chip saying whether a hardware port drives it, a port picker when hardware is attached, a Learn button (click a control, then move the one on the hardware; a second button resets the mapping), and a tiny monitor that prints the last message, so you see the hardware talking before you see what it moves. Plugging in a known device switches the picker and the faceplate.
Parameters
Name
Type
Description
parent
HTMLElement
The element to add the panel to. In full mode the panel is position: absolute; inset: 0, so give parent a position (relative) and a size.
mcs
MidiControllers
The manager that holds the devices and the hardware.
mode
String
'full' fills parent. 'dock' is a drawer fixed to the bottom of the window, 40% of the window height by default, with a grip to drag its height (remembered in localStorage). optionaldefault 'full'
open
Boolean
Whether it starts open. A closed dock builds its view only when opened. optionaldefault true
id
String | null
An element id for the panel. optionaldefault null
onToggle
Function | null
(open) => … called when the panel opens or closes. optionaldefault null
closeButton
Boolean
Show a ✕ button. The default is on in dock mode only. optionaldefault mode === 'dock'
profiles
String[] | null
Limit the device picker to these profile ids. optionaldefault null
contained
Boolean
Dock inside parent (absolute) instead of the window (fixed). optionaldefault false
Returns
{ el, view, isOpen, open(), close(), toggle(), dispose() }el is the panel element, view the ControllerView (once it exists), isOpen a boolean. open(), close() and toggle() return the same object; close() also releases held notes.
Notes
The labels on the toolbar (Learn, Reset, the hardware chip) are written in Chinese only, whatever the page language is. <oav-controller> has an English and a Chinese toolbar; this panel is the one the lab and the phone tab use.
Examples
js
import { MidiControllers, PROFILES, mountMidiPanel } from'@openav/midi';
const app = document.getElementById('app');
app.style.cssText = 'position: relative; height: 320px';
const mcs = new MidiControllers({ profiles: PROFILES, initial: 'akai-lpd8' });
const panel = mountMidiPanel(app, mcs, { mode: 'full' });
console.log(panel.isOpen, panel.view.constructor.name); // true ControllerView
mcs.current.press('pad2'); // the pad lights up in the panel
setTimeout(() => mcs.current.release('pad2'), 500);
module A panel with the toolbar and the AKAI LPD8 appears in the page; pad 2 lights for half a second. Switch the device in the picker.
Related
MidiControllersTies hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
ControllerViewThe playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
PANEL_CSSThe stylesheet text of the controller panel (mountMidiPanel), as a string.
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
Load @openav/midi/element (https://openaudiovisual.com/packages/midi/element.js) once and write <oav-controller profile="akai-lpd8" readout></oav-controller>. Importing the file registers the tag by itself. It is a shadow-DOM element, so the page's CSS cannot bend it and it cannot bend the page. It plays with a mouse, a finger, a pen and the keyboard, with or without Web MIDI: on Safari and iPad it still plays and fires events, and only the hardware and MIDI-out parts say they are unavailable.
Attributes, each also a property of the same name (midi-out is el.midiOut): profile (an id, a short name or a URL to a profile JSON; default arturia-minilab3), layout (auto, face as it looks, stack re-gridded for a tall box), hardware (off by default; bare or on follows the real device, ask shows a button so the permission prompt waits for a click, or a port name or /regex/), midi-out (off by default; bare or ask lets the visitor pick, or a port name, substring or /regex/ such as IAC or loopMIDI), channel (1 to 16: move every control to that channel), picker, readout (the last message under the faceplate), learn (a Learn button), follow (show whatever known device is plugged in), theme (dark or light: the toolbar only, the device stays the device), telemetry (off sends no anonymous hit) and lang (the toolbar language: zh… is Chinese, anything else English; it reads the nearest lang on the page).
Events are CustomEvents that bubble and cross the shadow root, so el.addEventListener('control', e => e.detail) works anywhere above it: ready{ profile, name, controls }, control (see controlDetail()), noteon, noteoff, connect, disconnect, status, profilechange and error{ message }. EMBED_EVENTS lists them. Methods: set(), get(), values(), reset(), press(), release(), ingest(), send(), on(), connectHardware(), disconnectHardware(), setOutput() and toggleLearn().
It fills its width, and its height follows the faceplate (a taller stack on a phone) unless you set one. CSS variables: --oav-bg, --oav-fg, --oav-accent, --oav-line, --oav-font, and --oav-ratio (width ÷ height). Parts: ::part(toolbar), ::part(body), ::part(readout).
Before you call it
Wait for the controller to exist: await el.ready (a promise that resolves with the element), or listen for the ready event. Before that, set(), press(), release() and ingest() return null and print one console warning, and get() also returns null. It never asks for MIDI permission unless hardware, midi-out or Learn is used.
Standard custom-element callback. Changing profile, layout, hardware, midi-out, follow, readout or theme updates the live element. Changing channel or lang rebuilds it.
Moves a control like a hand: knobs and faders 0..1, pads and buttons above 0 press and 0 release, keys set('keys', velocity, { note: 60 }). quiet: true moves only the faceplate: no event, no MIDI out. A number as the third argument is taken as note. Returns the bytes sent, or null.
Turns Learn mode on or off (no argument toggles) and shows a hint in the readout. Turning it on also starts listening to hardware. Returns the new state.
The attributes the element reacts to: EMBED_ATTRS plus lang.
Notes
For a place that only takes an iframe, the same element is served at /embed/controller/ (query parameters mirror the attributes). Hardware and MIDI out inside the frame need allow="midi" on the <iframe>. See parseEmbedOptions() and toParent().
If a page imports both @openav/midi and @openav/midi/element, the browser loads index.js twice, because the element imports its own copy under a ?v= stamp. Checks across the two copies fail: el.core instanceof ControllerHost is false. Talk to the element through its methods and events instead.
Like createController(), it sends one anonymous hit per page load, never from localhost or the LAN. telemetry="off" stops it.
Examples
js
// importing the file registers <oav-controller>import'@openav/midi/element';
const app = document.getElementById('app');
app.innerHTML = '<oav-controller profile="akai-lpd8" readout></oav-controller>';
const el = app.firstElementChild;
await el.ready; // the controller exists now
el.on('control', (d) => console.log(d.signal, d.value.toFixed(2), d.source));
el.set('k1', 0.5); // turns knob 1 on the screen
el.ingest([0xb0, 71, 127]); // MIDI from "elsewhere": knob 2 follows
module An LPD8 appears with a message line under it. The console prints midi/lpd8/k1 0.50 ui, then midi/lpd8/k2 1.00 hardware. Drag the knobs or tap the pads.
js
import'@openav/midi/element';
const app = document.getElementById('app');
app.innerHTML = '<oav-controller profile="lpd8" picker style="max-width: 420px"></oav-controller>';
const el = app.firstElementChild;
await el.ready;
el.addEventListener('profilechange', (e) => console.log('now showing', e.detail.name));
el.profile = 'nano2'; // the same as setAttribute('profile', 'nano2')
console.log(el.status.hardware, el.channel); // off null
module The toolbar has a device picker. Setting el.profile switches the faceplate to the Korg nanoKONTROL2 and the console prints its name, then off null.
html
<script type="module" src="https://openaudiovisual.com/packages/midi/element.js"></script>
<!-- plays on its own; the real device and MIDI out stay off until you ask -->
<oav-controller profile="akai-lpd8" readout></oav-controller>
<!-- a button asks for the real device on click; a MIDI output named like IAC receives what you play -->
<oav-controller profile="arturia-minilab3" hardware="ask" midi-out="IAC" picker learn></oav-controller>
static The two lines you paste into any page. The first plays with the mouse; the second shows a Connect button and a Learn button.
To be confirmed
Not tried for this page: hardware and midi-out inside an iframe in each browser, and a real MIDI device behind the element. The allow="midi" requirement is as the package README states it.
Related
defineController()Registers the controller element under another tag name (importing @openav/midi/element already registers it as oav-controller).
createController()Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
ControllerHostThe object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
parseEmbedOptions()Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
EMBED_ATTRSThe attributes the oav-controller element reads, which are also the query parameters of the iframe page: profile, layout, hardware, midi-out and more.
EMBED_EVENTSThe nine events the oav-controller element dispatches and the iframe posts to its page, from ready and control to profilechange and error.
controlDetail()Builds the control event payload for one control change: id, value, raw, signal, the parsed message, its bytes and where it came from.
The stylesheet text of the controller panel (mountMidiPanel), as a string.
Advanced: it is exported for pages that want to read or override the rules. You never need it to mount a panel.
Type
String
CSS for the .mcp… classes. mountMidiPanel() injects it by itself, once per document (a <style id="openav-midi-panel-css"> in the head).
Examples
js
import { PANEL_CSS } from'@openav/midi';
console.log(PANEL_CSS.includes('.mcp.dock')); // the drawer rules
module Prints true: the panel's rules are readable CSS text.
Related
mountMidiPanel()Mounts a controller panel: a toolbar (device picker, hardware status, port picker, Learn, last message) above a playable faceplate, docked at the bottom or full size.
VIEW_CSSThe stylesheet text of the on-screen controller (ControllerView), as a string.
Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
It is the one place that decides what the attributes and the iframe's query string mean, so the element and the iframe always agree. Defaults: profile is arturia-minilab3, layoutauto, themedark, hardware and midiOut off (so embedding never asks for MIDI permission), the booleans false, telemetry true, and channel, bg, frame, origin, lang are null.
Rules worth knowing: an unknown layout becomes auto; theme is light or dark; midi-out may also be written midiout or midiOut, and alone it means ask; frame may also be written id and is cut to 64 characters; origin is kept only as https://host or http://host with no path (anything else, such as javascript:, becomes null), lang is cut to 16 characters, and bg goes through embedColor().
Parameters
Name
Type
Description
src
Element | URLSearchParams | Object | Function
Where the options live: an element (read with getAttribute), a URLSearchParams (the iframe page passes its own), a plain object { profile: 'lpd8' }, or a function (name) => value | null.
Returns
Object{ profile, layout, hardware, midiOut, channel, learn, picker, readout, follow, telemetry, theme, bg, frame, origin, lang }. hardware and midiOut are { mode, port } as in embedMode().
module Prints every default as JSON, then midiOut as {"mode":"ask","port":null} next to hardware as off, then null.
Related
embedMode()Reads hardware and midi-out: off, ask (a button, so the permission prompt waits for a click) or on, plus which port.
embedFlag()Reads a boolean attribute: present means on, unless it says false, 0, off or no.
embedChannel()Reads the channel attribute: a whole number from 1 to 16, or null for anything else.
embedColor()Checks the bg colour of the iframe page and returns a safe value: transparent, a hex colour, or a plain colour name; anything else is null.
EMBED_ATTRSThe attributes the oav-controller element reads, which are also the query parameters of the iframe page: profile, layout, hardware, midi-out and more.
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
Works out the width ÷ height a controller wants in a box of a given width: its faceplate when every control stays finger-sized, a taller stack otherwise.
<oav-controller> uses it to set its height, so a controller sizes itself from its width. In auto, the faceplate's own proportions are used while the smallest control stays at least minTouch pixels at that width; otherwise it becomes a stack, about 0.8 to 1.5 times as tall as it is wide depending on how many bands the profile's portrait has. Square grid devices (the Launchpad, stack: false) always keep their face.
Parameters
Name
Type
Description
profile
Object
A normalized profile (face, sections, portrait, stack).
width
Number
The box width in pixels.
layout
String
'auto', 'face' or 'stack'. optionaldefault 'auto'
minTouch
Number
The smallest control size in pixels that still counts as finger-sized. optionaldefault 38
Returns
{ mode, aspect }mode is 'face' or 'stack', and aspect is the width ÷ height to give the box.
module Prints four results: a wide face, a tall stack for a narrow box, the face when forced, and the square Launchpad.
Related
OavControllerThe oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
EMBED_LAYOUTSThe three layouts a controller can have: auto, face (the device as it looks) and stack (re-gridded for a tall box).
ControllerViewThe playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
Says where a control rests: knobs and faders at their default (0), bipolar wheels at the centre, pads, buttons and keys at 0.
reset() on ControllerHost uses it to bring every control home. An endless encoder has no real position, so only its drawn position goes to the middle and no MIDI is sent for it.
Parameters
Name
Type
Description
c
Object
A control definition from a normalized profile.
Returns
Number 0 for keys, pads, buttons and bipolar controls; otherwise the control's def, else 0.5 for an endless encoder and 0 for the rest.
Examples
js
import { normalizeProfile, profileById } from'@openav/midi';
import { restValue } from'@openav/midi/embed';
const p = normalizeProfile(profileById('minilab3'));
for (const id of ['knob1', 'pitch', 'enc', 'pad1', 'keys']) {
console.log(id, restValue(p.controls.find((c) => c.id === id)));
}
module Prints 0 for the knob, the pitch strip (centre), the pad and the keys, and 0.5 for the endless encoder.
Related
setControl()Moves one control the way a hand would, whatever its type, and returns the MIDI bytes it sent: setControl(mc, 'k1', 0.5).
ControllerHostThe object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
Moves one control the way a hand would, whatever its type, and returns the MIDI bytes it sent: setControl(mc, 'k1', 0.5).
What value means depends on the type. Knob, fader, wheel, strip and encoder: 0..1 (−1..1 for a bipolar one). Pad and momentary button: above 0 presses (with that velocity, held to 1) and 0 releases. Toggle button: 0.5 or more is on. Keys: value is the velocity, above 0 presses that note and 0 releases it. This is what set() on ControllerHost and <oav-controller> call.
Parameters
Name
Type
Description
mc
MidiController
The controller.
id
String
The control id. An unknown id throws <profile> has no control "x" (controls: …).
value
Number
The value (below). Anything that is not a finite number throws set("id"): value must be a number.
note
Number | null
For a keys control: which note. The default is the first key of its range. optionaldefault null
source
String
'ui' (default) publishes and reaches MIDI out. 'mirror' moves only the faceplate (two-way binding). optionaldefault 'ui'
Returns
number[] | null The bytes sent, or null when nothing changed (for example releasing a pad that is not held).
Examples
js
import { MidiController, profileById } from'@openav/midi';
import { setControl } from'@openav/midi/embed';
const mc = new MidiController(profileById('lpd8'));
console.log(setControl(mc, 'k1', 0.5)); // [176, 70, 64] : a knob
console.log(setControl(mc, 'pad1', 1)); // [153, 36, 127] : a pad pressed
console.log(setControl(mc, 'pad1', 0)); // [137, 36, 0] : released
console.log(setControl(mc, 'pad1', 0)); // null : nothing to release
module Prints the three messages the control types send, then null.
Related
valuesOf()Reads every control's current value from a controller as { id: number }; for keys it is 1 while any key is held.
MidiControllerOne MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
ControllerHostThe object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
restValue()Says where a control rests: knobs and faders at their default (0), bipolar wheels at the centre, pads, buttons and keys at 0.
Page side: builds a command to post into an embed iframe: { target: 'openav', v: 1, type, ...args }.
It does not check the command: the iframe does, with fromParent(), and ignores what it does not understand. Post to the frame's real origin, not *, when you can.
Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
The iframe page calls parent.postMessage(toParent(name, detail, { frame }), target) for every event, where target is the ?origin= parameter or *. With ?origin=https://your.site the frame posts only to that origin and accepts commands only from it.
Writes what a device's screen says about a touch: Knob 1 · 87, C4 · 100, Pad 3 · 96, Pitch +0.42.
ControllerCanvas draws this text on the device's screen (MiniLab 3, KeyStep) so a video shows what the hardware's own screen would. It is pure, so it is easy to reuse for a caption or a log.
Parameters
Name
Type
Description
c
Object
The control's definition (from the normalized profile). null gives an empty string.
st
Object
Its state: { raw, value, held, pos }.
note
Number | null
For a keys control, the note that was touched. optionaldefault null
Returns
String The line. A control whose label is only a number is named after its type (Knob 1). A pad shows its velocity only while held, a button shows on or off, a relative encoder shows its position as a percentage, and a bipolar wheel shows a signed value.
module Prints the four lines shown in the comments.
Related
ControllerCanvasDraws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
midiNoteName()Turns a MIDI note number into its name with the octave, middle C being C4: midiNoteName(60) is 'C4', midiNoteName(61) is 'C#4'.
module Prints 16 controls, then k1 = 1 | pad2 = 0.50 | pad1 = 0: one number for every control of the LPD8.
Related
setControl()Moves one control the way a hand would, whatever its type, and returns the MIDI bytes it sent: setControl(mc, 'k1', 0.5).
ControllerHostThe object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
The stylesheet text of the on-screen controller (ControllerView), as a string.
Advanced. You never need it to show a controller. It is exported for pages that build their own shadow DOM, a server-rendered preview or a theme that wants to read the rules. Colours use the CSS variables --oav-focus, --body, --c and --cap that the view sets per control.
Type
String
CSS for the .mcv… classes. The view injects it by itself, once per document, or once per shadow root when it lives inside one (a <style id="openav-midi-view-css">).
Examples
js
import { VIEW_CSS } from'@openav/midi';
console.log(typeof VIEW_CSS, VIEW_CSS.length > 1000);
console.log(VIEW_CSS.includes('.mcv-pad')); // the rules for pads
module Prints string true, then true: it is plain CSS text you can read or inject yourself.
Related
ControllerViewThe playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
PANEL_CSSThe stylesheet text of the controller panel (mountMidiPanel), as a string.
An on-screen piano you can play with the mouse, a finger or the computer keyboard, that calls a function for every note.
new KeysPiano(el, { onNote }) builds the keys inside el. Whatever plays a key (a click or a touch on it, sweeping across the keys with the button held, the computer keyboard, or your own call to press()) ends up in onNote(note, vel01, on): on is true for a key going down and false for one coming up, and vel01 is 0..1 (0 on the way up). The black and white pattern follows the real pitch, so a piano can start on any note. A key that is already down ignores a second press().
The computer keyboard is off until you set piano.captureEnabled = true, and it is up to you to pass the page's keydown and keyup events to handleKeyDown(e) and handleKeyUp(e) (see PIANO_KEYMAP for which letter is which note). mountKeys() does all of that for you.
Parameters
Name
Type
Description
container
HTMLElement
The element the keys are built in. It gets the class oav-keys and is emptied first.
opts
Object
Options: base (lowest key, default 48), octaves (default 2), semitones (the number of keys, default octaves × 12 + 1), velocity (0..127 for the mouse and the computer keyboard, default 100), onNote(note, vel01, on), onOctave(base), fill (size the keys in percent of the container, so it can be any width), keyWidth (px per white key when not filling, default 27), minBase (24) and maxBase (84), the range the octave keys can move base in. optionaldefault {}
Moves the piano to start on note n, held between minBase and maxBase. Sounding keys are released first, so no note sticks. onOctave(base) is called with the new value when it changed.
Presses a key as if it was played: press(60, 0.8). The key lights up and onNote(60, 0.8, true) is called. Pressing a key that is already down does nothing.
Give it a keydown event. Z and X move the piano an octave down and up, a letter in PIANO_KEYMAP presses its note (Shift adds 20 to the velocity), and the other letters are swallowed. Returns true when the event was used. A key held down (e.repeat) does not retrigger.
Give it a keyup event: it releases the note that key pressed. Returns true when the event was used.
Notes
handleKeyDown(e) returns true when the piano used the event, and then you should call e.preventDefault(). While capturing, every letter a–z counts as used, even those that play no note, so a stray letter cannot trigger a hotkey; digits, space and the arrows pass through. Held keys with Ctrl, Cmd or Alt are never used.
Examples
js
import { KeysPiano } from'@openav/keys';
const piano = new KeysPiano(document.getElementById('app'), {
base: 60, octaves: 1,
onNote: (note, vel, on) => console.log(on ? 'on' : 'off', piano.noteName(note), vel.toFixed(2)),
});
piano.press(64, 0.5); // on E4 0.50
piano.release(64); // off E4 0.00
module A one-octave piano from middle C appears on the page. Click its keys and the console logs each note by name.
js
import { KeysPiano } from'@openav/keys';
const piano = new KeysPiano(document.getElementById('app'), {
fill: true, // as wide as the container
onNote: (note, vel, on) => { if (on) console.log('note', note); },
});
piano.captureEnabled = true; // the computer keyboard plays
window.addEventListener('keydown', (e) => { if (piano.handleKeyDown(e)) e.preventDefault(); });
window.addEventListener('keyup', (e) => piano.handleKeyUp(e));
window.addEventListener('blur', () => piano.releaseAll());
module Click the page, then press A, S, D, F… to play from C3, Z and X to change octave. The letter keys no longer reach the rest of the page.
Related
mountKeys()Puts a playable piano on the page in one call, with a keyboard switch and a self-playing performer, and publishes every note as midi/note/on and midi/note/off.
SimPlayerA hands-free performer: it presses and releases notes on its own, in a scale, so a demo can play itself.
PIANO_KEYMAPWhich computer-keyboard letter plays which note: a is the base note, w a semitone higher, s two.
notePressed()You define this function; oav-p5 calls it each time a key (or a pad) goes down.
noteName()Turns a MIDI number into a note name: noteName(61) is 'C#4'.
Puts a playable piano on the page in one call, with a keyboard switch and a self-playing performer, and publishes every note as midi/note/on and midi/note/off.
mountKeys() is the whole keyboard department in one line: a KeysPiano you can play with the mouse, a finger or the computer keyboard, and a SimPlayer that plays by itself. Above the piano are two checkboxes, both off to begin with: keyboard (the QWERTY keys play, Z and X move an octave, Shift plays harder) and simulate performance.
Everything is published as the standard midi/note/on and midi/note/off, the same names a real MIDI keyboard uses, so a world, a sound engine or a chord detector cannot tell a QWERTY performance from hardware. Call keys.update(dt) once per frame, or the simulated performer does not move.
Before you call it
A page with a DOM. The keyboard checkbox must be ticked before the computer keyboard plays, on purpose: while it is on, every letter key is taken by the piano, so hotkeys on letters stop working. A key listener goes on window until dispose().
Parameters
Name
Type
Description
container
HTMLElement
The element to put the piano in. It gets a small bar of checkboxes and the piano under it. dispose() empties it again.
signals
Signals
The Signals bus to publish into. Every key down is midi/note/on{ note, vel, ch: 0 } (vel 0..1) and every key up midi/note/off{ note, ch: 0 }. Without a bus the piano still works and calls chord. optional
chord
ChordDetector
Anything with noteOn(note, vel01) and noteOff(note), usually a chord detector from @openav/chord, to be fed every note directly. optionaldefault null
base
Number
The MIDI note of the lowest key (48 is C3). The simulated performer plays an octave above it. optionaldefault 48
octaves
Number
How many octaves the piano shows. It has octaves × 12 + 1 keys, so two octaves end on a C. optionaldefault 2
sim
Boolean
Show the "simulate performance" checkbox. The performer exists either way: keys.sim.toggle(true) starts it from code. optionaldefault true
capture
Boolean
Show the "keyboard" checkbox that lets the computer keyboard play the piano. With false there is no checkbox and captureBox is null; you can still set keys.piano.captureEnabled = true. optionaldefault true
Returns
Object{ piano, sim, update(dt), dispose(), captureBox }: the KeysPiano, the SimPlayer, a function to call every frame with the seconds since the last one (it drives the performer), a function that takes everything down again, and the "keyboard" checkbox (or null).
Notes
In a p5 sketch you do not need this: the 🎹 panel of oav-p5 is a piano like this one, and notePressed() hears it. mountKeys() is for a page that has no oav-p5.
Examples
js
import { mountKeys } from'@openav/keys';
import { Signals } from'@openav/core';
const signals = new Signals();
signals.on('midi/note/on', ({ note, vel }) => console.log('on', note, vel.toFixed(2)));
signals.on('midi/note/off', ({ note }) => console.log('off', note));
const keys = mountKeys(document.getElementById('app'), { signals, base: 48 });
keys.piano.press(60, 0.8); // as if C4 was clicked
keys.piano.release(60);
// on 60 0.80// off 60
module A two-octave piano appears on the page with its two checkboxes; click a key and the console logs the same events the code below sends.
js
import { mountKeys } from'@openav/keys';
import { Signals } from'@openav/core';
const signals = new Signals();
const heard = [];
signals.on('midi/note/on', ({ note }) => heard.push(note));
const keys = mountKeys(document.getElementById('app'), { signals, octaves: 3, capture: false });
keys.sim.toggle(true); // the performer plays by itselfconst t0 = performance.now();
let last = t0;
(function frame(now) {
keys.update((now - last) / 1000); last = now; // seconds since the last frameif (now - t0 < 3000) return requestAnimationFrame(frame);
console.log(heard.length > 0, keys.captureBox); // true null
keys.dispose(); // the piano is taken down
})(t0);
module A three-octave piano plays itself for three seconds, its keys lighting up, and then it is taken down. capture: false leaves out the keyboard checkbox.
Related
KeysPianoAn on-screen piano you can play with the mouse, a finger or the computer keyboard, that calls a function for every note.
SimPlayerA hands-free performer: it presses and releases notes on its own, in a scale, so a demo can play itself.
PIANO_KEYMAPWhich computer-keyboard letter plays which note: a is the base note, w a semitone higher, s two.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
notePressed()You define this function; oav-p5 calls it each time a key (or a pad) goes down.
Which computer-keyboard letter plays which note: a is the base note, w a semitone higher, s two.
The layout the music software calls the DAW standard: the A S D F G H J K L ; row is the white keys and the W E T Y U O P row above it is the black keys, placed where they sit on a piano. With a piano that starts on C (base 48, 60…) it reads C, C♯, D, D♯, E, F, F♯, G, G♯, A, A♯, B, C, C♯, D, D♯, E. The keys are lowercase letters; KeysPiano.handleKeyDown(e) lowercases the key first. Z and X are not in the map: they move the octave.
module Looks up the note of a few letters on a piano that starts on middle C: k is the C an octave up, and q plays nothing.
Related
KeysPianoAn on-screen piano you can play with the mouse, a finger or the computer keyboard, that calls a function for every note.
mountKeys()Puts a playable piano on the page in one call, with a keyboard switch and a self-playing performer, and publishes every note as midi/note/on and midi/note/off.
new SimPlayer({ press, release, base = 60, scale = [0, 2, 4, 7, 9], density = 1 } = {})
A hands-free performer: it presses and releases notes on its own, in a scale, so a demo can play itself.
It plays like someone noodling. Every gesture it picks at random: mostly a step up or down the scale, sometimes a leap, now and then a triad stacked in the scale, and real rests. Notes last 0.18 to 0.68 seconds (a triad twice as long), the next gesture comes 0.22 to 0.77 seconds later (a rest lasts 0.6 to 2 seconds), all at the default density of 1, and the notes are kept between 36 and 96. Because it presses and releases through your own two functions, whatever listens downstream cannot tell it from a person.
It starts off. Call toggle(true) and then update(dt) once per frame with the seconds since the last frame; it does nothing while it is off. toggle(false) releases every note it is holding, so none sticks. The notes are random (it uses Math.random, there is no seed).
Parameters
Name
Type
Description
press
Function
press(note, vel01): called when a note starts, with its MIDI number and a velocity between 0.45 and 0.85. Usually the press of a KeysPiano, so the keys light up.
release
Function
release(note): called when the note ends.
base
Number
The MIDI note the scale starts on. optionaldefault 60
scale
Number[]
The notes it may play, as semitones above base. The default is a pentatonic scale, which sounds fine whatever happens. optionaldefault [0, 2, 4, 7, 9]
density
Number
How busy it is. The gaps between notes are divided by it, so 2 plays about twice as often. optionaldefault 1
Advance by dt seconds: release the notes that are due and, when it is time, start the next gesture. Call it every frame. Does nothing while the performer is off.
Examples
js
import { SimPlayer } from'@openav/keys';
const played = [];
const sim = new SimPlayer({
press: (note, vel) => played.push(note),
release: () => {},
base: 60, density: 2,
});
sim.toggle(true);
for (let i = 0; i < 600; i++) sim.update(1 / 60); // ten seconds, in frames
console.log(played.length > 5, Math.min(...played) >= 36, Math.max(...played) <= 96); // true true true
sim.toggle(false);
module Runs the performer for ten simulated seconds without any sound or screen and logs that it played a good number of notes, all inside 36 to 96.
js
import { KeysPiano, SimPlayer } from'@openav/keys';
const piano = new KeysPiano(document.getElementById('app'), { base: 48, octaves: 2 });
const sim = new SimPlayer({
press: (n, v) => piano.press(n, v), // through the piano: the keys light up
release: (n) => piano.release(n),
base: 60, scale: [0, 2, 3, 5, 7, 8, 10], // a natural minor scale
});
sim.toggle(true);
const t0 = performance.now();
let last = t0;
(function frame(now) {
sim.update((now - last) / 1000); last = now;
if (now - t0 < 4000) requestAnimationFrame(frame); else sim.toggle(false);
})(t0);
module A piano plays itself in a minor scale for four seconds: the keys light up as the notes go down and up.
Related
KeysPianoAn on-screen piano you can play with the mouse, a finger or the computer keyboard, that calls a function for every note.
mountKeys()Puts a playable piano on the page in one call, with a keyboard switch and a self-playing performer, and publishes every note as midi/note/on and midi/note/off.
An advanced helper: the orthonormal basis [x, y, z] of a bone, rebuilt from two joints and the palm normal the way LeapJS expects.
The sensor gives joints, not rotations, but LeapJS sketches read a rotation basis for every bone, so V6Converter uses this to fill in bases. You only need it if you write your own converter.
Parameters
Name
Type
Description
prev
Array<Number>
The joint at the base of the bone, [x, y, z] in millimetres.
next
Array<Number>
The joint at the tip end of the bone.
palmNormal
Array<Number>
The palm normal; -palmNormal becomes the "back of the hand" axis (y).
left
Boolean
true for a left hand: the x axis is negated, so the determinant is negative, which is how LeapJS tells left from right.
Returns
Array Three rows [x, y, z], each a unit vector rounded to 3 decimals. z runs from the tip back to the base (LeapJS's Bone.direction() is -z).
Examples
js
import { boneBasis } from'@openav/leap';
// a bone pointing straight ahead (-z), palm facing down: the basis is the identity
console.log(boneBasis([0, 200, 0], [0, 200, -40], [0, -1, 0], false));
// the same bone on a left hand: x flips
console.log(boneBasis([0, 200, 0], [0, 200, -40], [0, -1, 0], true)[0]);
module The console prints the identity [ [ 1, 0, 0 ], [ 0, 1, 0 ], [ 0, 0, 1 ] ], then [ -1, 0, 0 ] for the left hand.
Related
palmBasis()An advanced helper: the orthonormal basis [x, y, z] of the palm, from its direction and normal, in the rows LeapJS reads as hand.r.
V6ConverterConverts raw hand lines into LeapJS v6 frames, keeping the little state the protocol needs: time visible, smoothed positions and fingertip velocities.
rawToV6()One-off conversion of a raw frame to a LeapJS v6 frame, with a fresh converter each time (no history, so no velocities).
The Leap bridge's default port: 6437, the port the original Leap service used, so old leap.js sketches find it unchanged.
The default of startBridge({ port }) and of --port. The environment variable LEAP_BRIDGE_PORT changes it for the command line. This is the port of the Leap bridge; the out-bridge has its own (DEFAULT_PORT of @openav/out, 6450).
Related
DEFAULT_URLThe default address LeapInput connects to: ws://127.0.0.1:6437/v6.json, the bridge on this machine speaking the LeapJS protocol.
The default address LeapInput connects to: ws://127.0.0.1:6437/v6.json, the bridge on this machine speaking the LeapJS protocol.
Port 6437 is the port the original Leap service used, so old leap.js sketches find the bridge without changing anything. …/raw on the same port serves the raw lines instead. In a show, ?leap=ws://host:port/v6.json in the page address points a page at another bridge, and ?leap=sim starts the simulator.
Related
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
deviceEvent({ device = null, service = true } = {})
A LeapJS device event telling the client whether a device is attached and streaming, plus our own service flag.
LeapJS (protocol 5 and up) turns it into deviceAttached, deviceStreaming and streamingStarted. The bridge sends one on connect and on every status change. LeapInput maps it to its status: no service is 'no-service', a service without a device is 'no-device', and a streaming device is 'tracking'.
Parameters
Name
Type
Description
device
String | null
The device id, or null for no device. optionaldefault null
service
Boolean
Whether the tracking service is running. LeapJS ignores this; LeapInput uses it to tell "no tracking service" from "no device". optionaldefault true
Returns
Object{ event: { type: 'deviceEvent', state: { attached, streaming, id, type: 'Peripheral', service } } }; id is 'none' without a device.
The five finger names in order, thumb index middle ring pinky: the order of hand.fingers and the names in the fingertip signals.
hand.fingers[0] is the thumb and hand.fingers[4] the pinky, so FINGERS[i] names hand.fingers[i]. The fingertip signals are leap/hand/<side>/<finger>/x|y|z with these names, for example leap/hand/right/index/y.
Related
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
handsOf()Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
leapSignals()The list of every signal LeapInput publishes, with its range, as [[name, meta], …], ready to hand to signals.define().
Turns a hand's direction and palm normal into { pitch, yaw, roll } in radians, with the LeapJS conventions.
A flat hand, palm down, fingers away from you, is { pitch: 0, yaw: 0, roll: 0 }. The three angles are the leap/hand/<side>/pitch|yaw|roll signals, which LeapInput computes with this function from each hand.
Parameters
Name
Type
Description
direction
Array<Number>
The direction the fingers point, [x, y, z] (a unit vector).
normal
Array<Number>
The palm normal [x, y, z], pointing out of the palm (down when the palm faces the sensor).
Returns
Object{ pitch, yaw, roll }, each −π..π: pitch is the fingers tipping up, yaw turning toward +x (the right), roll the palm turning over.
Notes
When the fingers point straight up there is no yaw or roll to measure, and the function answers π (3.14) for them. Expect that at the top of a gesture, and do not map those two signals when the hand is vertical.
The bones to draw for a hand, as pairs of points: each finger's joints in turn, the line across the knuckles, and the forearm.
It only lists the pairs; you choose how to draw them. LeapInput.skeleton() calls it for you. Use it directly when you draw with p5, three.js or your own projection.
Turns pinch and grab strengths into clean start and end events, per hand plus a both-hands pinch, using hysteresis so a wobbling value fires once.
A pinch strength hovers around any single threshold while the fingers are "almost touching", and one threshold then fires start, end, start, end sixty times a second. So every gesture is a trigger with two thresholds: it starts above on and only ends below off. The defaults are a pinch starting above 0.86 and ending below 0.5, a both-hands pinch entering when both hands are above 0.8 and leaving when either drops below 0.5, and a grab (fist) above 0.8 and below 0.5. A hand that disappears ends all of its gestures.
LeapInput already runs one (leap.gestures) and turns its events into the leap/…/pinch-start pulses. Use HandGestures on its own when you have hands from somewhere else.
Parameters
Name
Type
Description
thresholds
Object
Overrides for the defaults in THRESHOLDS: { pinch: { on, off }, bothPinch: { on, off }, grab: { on, off } }. Keys you leave out keep their defaults. optionaldefault THRESHOLDS
update(hands) takes [{ side: 'left' | 'right', pinch, grab, … }] and returns the events that just happened, in a stable order: { type: 'pinch-start' | 'pinch-end' | 'grab-start' | 'grab-end', side, hand }, then { type: 'both-pinch-start' | 'both-pinch-end', side: 'both', hands }. Call it once per frame.
module The console prints [], [ 'pinch-start' ], [], [ 'pinch-end' ]: the pinch starts above 0.86 and holds until it drops under 0.5.
js
import { HandGestures } from'@openav/leap';
const gestures = new HandGestures({ pinch: { on: 0.7, off: 0.3 } }); // easier to start, harder to let goconst hand = (side, pinch) => ({ side, pinch, grab: 0 });
const names = (events) => events.map((e) => e.type + ':' + e.side);
console.log(names(gestures.update([hand('right', 0.75), hand('left', 0.82)]))); // single pinches only: both need 0.8
console.log(names(gestures.update([hand('right', 0.9), hand('left', 0.9)]))); // now both hands pinch
console.log(names(gestures.update([]))); // both hands leave: everything ends
module The console prints [ 'pinch-start:left', 'pinch-start:right' ], then [ 'both-pinch-start:both' ], then the three ends when the hands leave.
Related
HysteresisOne Schmitt trigger: update(v) says 'start' when the value rises above on, and 'end' only when it falls below off.
THRESHOLDSThe default hysteresis levels for the gestures: pinch 0.86 / 0.5, both-hands pinch 0.8 / 0.5, grab 0.8 / 0.5 (start above on, end below off).
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
mockFrame()One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
This is the hand shape worlds draw from (leap.hands). A raw frame carries the fingers inline; a v6 frame carries them as pointables joined to their hand by handId, and handsOf() does that join. Both give the same hand, so code written against leap.hands does not care which URL the bridge was reached on.
Parameters
Name
Type
Description
msg
Object
A parsed message: a raw frame (t: 'f') or a v6 frame (with hands and pointables).
Returns
Array The hands, each { id, side: 'left' | 'right', confidence, pinch, grab, palm, velocity, normal, direction, width, fingers, extended, elbow, wrist, timeVisible }. Positions are in millimetres; fingers is 5 arrays of joints (the last is the tip), extended 5 booleans.
Examples
js
import { mockFrame, rawToV6, handsOf } from'@openav/leap';
const raw = mockFrame(2.0);
const fromRaw = handsOf(raw)[0];
const fromV6 = handsOf(rawToV6(raw))[0]; // the same frame as LeapJS would send it
console.log(fromRaw.side, fromRaw.palm, fromRaw.pinch); // right [43.6, 233.2, 53.6] 1
console.log(fromV6.side, fromV6.palm, fromV6.pinch); // the same hand
console.log(handsOf(null), handsOf({})); // [] []
module Both formats give the same hand: the console prints right [ 43.6, 233.2, 53.6 ] 1 twice, then [] [].
Related
handBones()The bones to draw for a hand, as pairs of points: each finger's joints in turn, the line across the knuckles, and the forearm.
rawToV6()One-off conversion of a raw frame to a LeapJS v6 frame, with a fresh converter each time (no history, so no velocities).
mockFrame()One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
One Schmitt trigger: update(v) says 'start' when the value rises above on, and 'end' only when it falls below off.
The gap between on and off is what stops a value that hovers near one threshold from flickering. Each call to update() returns 'start' (it just became active), 'end' (it just stopped) or null (nothing changed). A missing or NaN value while active counts as an end, so a lost sensor lets go. active tells you whether it is on now.
Parameters
Name
Type
Description
on
Number
The level the value must rise above to start.
off
Number
The level the value must fall below to end. It must be lower than on, or the constructor throws "Hysteresis: off must be below on".
module The console prints [ null, 'start', null, 'end', null ]: 0.6 is below 0.8 but above 0.4, so it is still held.
Related
HandGesturesTurns pinch and grab strengths into clean start and end events, per hand plus a both-hands pinch, using hysteresis so a wobbling value fires once.
THRESHOLDSThe default hysteresis levels for the gestures: pinch 0.86 / 0.5, both-hands pinch 0.8 / 0.5, grab 0.8 / 0.5 (start above on, end below off).
The classic Leap interaction box, { center: [0, 200, 0], size: [235.247, 235.247, 147.751] } in millimetres, that positions are normalised against.
A frozen object: the centre sits 200 mm above the sensor, and the box measures 235.247 mm along x, 235.247 mm along y and 147.751 mm along z (the sensor's own axes: x across, y up, z toward the player). normalizePoint() maps it to 0..1, and the console panel draws it as an outline so you can see when a hand leaves it. Pass your own box to new LeapInput({ box }) to change where 0 and 1 are.
Related
normalizePoint()Maps a point in millimetres to 0..1 inside the interaction box, clamped by default: the same normalisation the leap/* signals use.
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
mountLeapPanel()Mounts a small panel that shows whether the Leap is alive (bridge, service, device, hands) and draws what it sees, with a button to simulate hands.
Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
Browsers cannot talk to the Leap Motion sensor directly, so a small program on the same machine, the bridge (startBridge(), ws://127.0.0.1:6437), reads it and serves the hands over a WebSocket. LeapInput connects to that bridge and does one job: it turns hands into normalised signals and never touches params or worlds. In a show, createShow({ modules: { leap: true } }) builds one and returns it as show.leap; the console's L1 panel shows which link is missing.
It publishes, for each side (left, right): leap/hand/<side>/present (0 or 1), x, y, z (the palm in the interaction box, 0..1, y up, z toward the player), pinch and grab (0..1), roll, pitch, yaw (−π..π radians), speed (0..1, 1 = 1 m/s), and <finger>/x|y|z for the five fingertips. Shared: leap/status (0 no bridge, 1 bridge but no tracking service, 2 service but no device, 3 tracking), leap/hands (0..2) and leap/both/distance (palm to palm, 1 = 40 cm). The full list with its ranges is what leapSignals() returns.
Gestures are pulses with a payload { side, x, y, z, strength }: leap/hand/<side>/pinch-start|pinch-end and grab-start|grab-end, and leap/both/pinch-start|pinch-end (payload { x, y, z, distance } at the midpoint). They use hysteresis, so a pinch hovering near the threshold fires once and not sixty times a second (see HandGestures). A hand that leaves the view lets go: its present, pinch, grab and speed go to 0 and its gestures end.
For continuous control, map a signal to a param ({ source: 'leap/hand/right/y', target: 'energy' }) so the timeline, a knob and a hand are interchangeable. Use leap.hands (joints in millimetres) only to draw the hand itself.
Before you call it
For real hands: the bridge must be running (node packages/leap/bridge/leap-bridge.mjs, or --mock for animated made-up hands) and you must call connect(). Nothing is published until the first frame arrives. No sensor and no bridge: feed it hands with ingest(), or call simulate() to turn the mouse into hands.
Parameters
Name
Type
Description
signals
Signals
The Signals registry to publish into. Without one the input still works (hands, onFrame) but publishes nothing. optionaldefault null
url
String
The bridge address: the LeapJS protocol (the default, DEFAULT_URL) or …/raw for the raw line format. Any service that still speaks the LeapJS v6 protocol works too. optionaldefault ws://127.0.0.1:6437/v6.json
reconnect
Boolean
Whether to try again after the connection drops, waiting 0.5 s, then 1 s, 2 s … up to 5 s between tries. optionaldefault true
fingers
Boolean
Publish the fingertip signals (3 per finger, 30 across two hands). false leaves them out and keeps the signal list short. optionaldefault true
box
Object
The interaction box { center, size } in millimetres that positions are normalised against. The default is INTERACTION_BOX. optionaldefault INTERACTION_BOX
thresholds
Object
Overrides for the gesture hysteresis, such as { pinch: { on: 0.9, off: 0.6 } } (see THRESHOLDS). Keys you leave out keep their defaults. optionaldefault THRESHOLDS
WebSocket
Function
The WebSocket constructor to use. Pass your own to test without a network. optionaldefault globalThis.WebSocket
Feed one parsed message, exactly what the socket would deliver: a v6 header, a v6 frame, a device event, or a raw status or frame line. Public so that tests, recordings and the simulator take the very path real frames take.
No sensor? Mouse and touch become hands: move = right palm (x across, y up the screen), wheel = depth, press = pinch, right button or Alt+press = grab, hold Shift to drive the left hand, a second finger on a touch screen = left hand. { target = window, hz = 60 }. Returns stop(). Throws when there is no target to listen on.
A point in millimetres to canvas pixels: project(p, w, h, { view, box }). view is 'oblique' (default: the performer's eye, up the canvas = higher and farther, so a flat hand shows its fingers), 'front' (x across, y up) or 'top' (z down the canvas). Returns [x, y].
Draw the hands onto a 2D canvas context: skeleton(ctx, w, h, { view, box, colorLeft, colorRight, lineWidth, hands }). The palm and fingertips are dots and a pinch gets a ring. Points outside the box still draw.
Fields to read: status ('idle', 'connecting', 'bridge', 'no-service', 'no-device', 'tracking', 'simulated', 'closed'), hands (an array; each hand has id, side, palm, fingers (5 × 5 joints), pinch, grab, velocity, normal, direction … in millimetres), frame (the latest message as it came), frames, fps, device, serviceVersion, gestures and box.
While the simulator runs it owns the hands: real frames from the bridge are ignored, and what the bridge says is remembered and restored when the simulator stops.
The signals are normalised (y = 1 is up). leap.hands and the LeapJS frames keep the sensor's own millimetre units, with +z toward the player.
How the bridge talks, the formats and the troubleshooting table are in the package README.
Examples
js
import { Signals } from'@openav/core';
import { LeapInput, mockFrame } from'@openav/leap';
const signals = new Signals();
const leap = new LeapInput({ signals });
leap.ingest(mockFrame(2.0)); // a made-up frame: the same path real frames take
console.log(leap.status, leap.hands.length); // tracking 2
console.log(signals.get('leap/hand/right/pinch')); // 1
leap.skeleton(document.getElementById('c').getContext('2d'), 400, 300);
module The console prints tracking 2 and 1 (the mock right hand is pinching at second 2), and both hands are drawn on the canvas.
js
import { Signals } from'@openav/core';
import { LeapInput, mockFrame } from'@openav/leap';
const signals = new Signals();
const leap = new LeapInput({ signals });
signals.onAny((name, value, meta) => {
if (meta.kind === 'pulse') console.log(name, value.strength.toFixed(2));
});
for (let i = 0; i <= 360; i++) leap.ingest(mockFrame(i / 60)); // six seconds of mock hands at 60 frames a second
console.log(signals.get('leap/hands'), signals.get('leap/status'));
module Six seconds of mock hands, with the pulses listed as they fire: the right hand pinches twice (leap/hand/right/pinch-start 0.86, then pinch-end 0.50, again) and the left fist closes once (leap/hand/left/grab-start 0.80). It ends with 2 3 (two hands, status 3 = tracking).
module Move the mouse over the page: a right hand follows it, and the text shows its height (y). Press the button to pinch, right-click to make a fist.
static With the bridge running and a hand over the sensor, the console follows the link (leap connecting, leap bridge, leap tracking) and prints a line for every pinch.
To be confirmed
Not tried with a real Leap Motion sensor here: the examples feed made-up hands (mockFrame) through the same ingest() path.
On a public https page, whether Chrome asks for permission before the page may reach 127.0.0.1 (the package README says it needs the "loopback-network" permission) was not tried here.
Related
leapSignals()The list of every signal LeapInput publishes, with its range, as [[name, meta], …], ready to hand to signals.define().
mockFrame()One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
HandGesturesTurns pinch and grab strengths into clean start and end events, per hand plus a both-hands pinch, using hysteresis so a wobbling value fires once.
mountLeapPanel()Mounts a small panel that shows whether the Leap is alive (bridge, service, device, hands) and draws what it sees, with a button to simulate hands.
normalizePoint()Maps a point in millimetres to 0..1 inside the interaction box, clamped by default: the same normalisation the leap/* signals use.
handsOf()Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
The list of every signal LeapInput publishes, with its range, as [[name, meta], …], ready to hand to signals.define().
new LeapInput({ signals }) already declares all of these, so you rarely call this yourself. It is useful when you want the list: to build a signal picker, to check that a route's source exists, or to declare the signals before the sensor is connected. The names it returns are the vocabulary of the Signals reference for Leap.
Parameters
Name
Type
Description
fingers
Boolean
Include the fingertip signals. false leaves out the 30 of them. optionaldefault true
Returns
Array The pairs [name, meta]: 63 with fingers, 33 without. Each meta has source: 'leap', and min and max (the roll, pitch and yaw ones in radians, −π..π) or kind: 'pulse'.
module The console prints 63 33, then the roll range's lower end (−π) and pulse.
Related
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
mockFrame()One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
This is the bridge's --mock stream and the way to try Leap code without a sensor. The right hand pinches for about a second out of every 3, the left hand pinches once every 12 seconds, and at those moments (for example second 8) both are pinched, so the both-hands pinch fires too. The left fist comes and goes slowly. The output goes through exactly the same conversion and publishing as a real Leap Motion Controller.
Parameters
Name
Type
Description
t
Number
Time in seconds. The result depends only on t, so it is the same every time.
id
Number
The frame number written into the frame. optionaldefault Math.round(t * 60)
hands
Number
How many hands: 2 (right and left), 1 (the right only) or 0. optionaldefault 2
Returns
Object A frame in the raw line format: { t: 'f', id, fps: 60, hands: [hand, …] }, the right hand (side: 'R') first. Feed it to LeapInput.ingest() or handsOf().
Examples
js
import { mockFrame } from'@openav/leap';
for (const t of [0.5, 2.0, 8.0]) {
const frame = mockFrame(t);
console.log(t, frame.hands.map((h) => h.side + ' pinch ' + h.pinch).join(' · '));
}
module The console prints 0.5 R pinch 0 · L pinch 0, 2 R pinch 1 · L pinch 0 and 8 R pinch 1 · L pinch 1 (the both-hands pinch).
Related
mockHand()Builds one made-up hand, with all five fingers and the arm, from a palm position plus pinch, grab, roll, yaw and pitch.
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
handsOf()Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
Builds one made-up hand, with all five fingers and the arm, from a palm position plus pinch, grab, roll, yaw and pitch.
It places a plausible hand in space: palm, knuckles, fingers curled by grab, thumb and index meeting as pinch goes to 1, wrist and elbow. LeapInput.simulate() builds its hands from the mouse with it, and mockFrame() with the animated ones. Use it to put a hand exactly where a test needs it.
Parameters
Name
Type
Description
id
Number
The hand's id. optionaldefault 1
side
'L' | 'R'
'R' for a right hand, 'L' for a left. optionaldefault 'R'
palm
Array<Number>
The palm position [x, y, z] in millimetres (+x right, +y up, +z toward the player). optionaldefault [0, 200, 0]
pinch
Number
Thumb to index, 0..1. At 1 the two fingertips meet. optionaldefault 0
grab
Number
How closed the fist is, 0..1. optionaldefault 0
roll
Number
Radians; positive rolls toward the thumb side. optionaldefault 0
yaw
Number
Radians; positive turns the fingers toward +x. optionaldefault 0
pitch
Number
Radians; positive tips the fingers up. optionaldefault 0
vel
Array<Number>
Palm velocity in millimetres per second. optionaldefault [0, 0, 0]
conf
Number
Tracking confidence, 0..1. optionaldefault 1
width
Number
Palm width in millimetres. optionaldefault 85
Returns
Object A hand in the raw line format: { id, side, conf, pinch, grab, palm, vel, n, dir, w, f, ext, arm }, where f is 5 fingers × 5 joints (thumb to pinky; the last joint of each is the tip). Wrap hands in { t: 'f', id, fps, hands: [...] } to make a frame.
Examples
js
import { mockHand } from'@openav/leap';
const open = mockHand({ palm: [0, 200, 0] });
const pinched = mockHand({ palm: [0, 200, 0], pinch: 1 });
const gap = (hand) => Math.hypot(...hand.f[0][4].map((v, i) => v - hand.f[1][4][i])).toFixed(0);
console.log('thumb to index tip, open:', gap(open), 'mm pinched:', gap(pinched), 'mm');
module The console measures the distance between the thumb tip and the index tip: 100 mm for the open hand and 0 mm when pinched.
Related
mockFrame()One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
handsOf()Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
handBones()The bones to draw for a hand, as pairs of points: each finger's joints in turn, the line across the knuckles, and the forearm.
Mounts a small panel that shows whether the Leap is alive (bridge, service, device, hands) and draws what it sees, with a button to simulate hands.
One status line says which link is the broken one, in order: no bridge, bridge but no tracking service, service but no device, tracking (with the number of hands and the frame rate). Under it, a small front view draws the interaction box (a hand outside it is visibly outside, its signals sitting clamped at the edge) and the hand skeletons. It redraws only on frames and status changes, so nothing runs while the sensor is idle.
In a show you do not call it: the console's L1 · Input panel mounts it when you declare modules: { leap: true }.
The width of the drawing in CSS pixels; the height follows (about 0.62 of the width). optionaldefault 300
simulate
Boolean
Show the "simulate" button (mouse and touch become hands). optionaldefault true
simTarget
HTMLElement | Window
What the simulator listens on. Defaults to leap.simTarget, then the window. optional
Returns
Object{ el, render(), destroy() }: the panel's root element, a function that redraws it, and one that removes it and unsubscribes.
Examples
js
import { LeapInput, mockFrame, mountLeapPanel } from'@openav/leap';
const leap = new LeapInput();
const panel = mountLeapPanel(document.getElementById('app'), leap, { width: 360 });
leap.ingest(mockFrame(2.0)); // a made-up frame stands in for the sensor
panel.render();
module A panel appears in the page: a green dot, the status line tracking · 2 hands · 60 fps · R:pinch (the right hand is pinching), a "simulate" button, and the two mock hands drawn in a small box.
Related
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
mountConsole()Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.
Maps a point in millimetres to 0..1 inside the interaction box, clamped by default: the same normalisation the leap/* signals use.
The classic box is centred 200 mm above the sensor and about 235 × 235 × 148 mm (INTERACTION_BOX). A palm at the centre reads [0.5, 0.5, 0.5]; a palm far outside it reads 0 or 1 on that axis, which is why the signals "sit clamped at the edge" when a hand leaves the box. Pass clamp = false when you want to know how far outside it is.
Parameters
Name
Type
Description
p
Array<Number>
The point [x, y, z] in millimetres.
box
Object
The box { center, size } in millimetres. optionaldefault INTERACTION_BOX
clamp
Boolean
Hold each value to 0..1. false lets a point outside the box go beyond. optionaldefault true
Returns
Array<Number>[x, y, z], each (p − center) / size + 0.5: 0.5 is the centre of the box, x grows to the right, y upward, z toward the player.
An advanced helper: the orthonormal basis [x, y, z] of the palm, from its direction and normal, in the rows LeapJS reads as hand.r.
Like boneBasis(), it exists so that the LeapJS frames V6Converter writes have a r that hand.rotationAngle() can use. Worlds that draw hands read leap.hands and handAngles() instead.
Parameters
Name
Type
Description
direction
Array<Number>
The direction the fingers point.
normal
Array<Number>
The palm normal.
left
Boolean
true for a left hand: x is negated.
Returns
Array Three rows [x, y, z] (x sideways, y the back of the hand, z toward the wrist), rounded to 3 decimals.
module The console prints the identity [ [ 1, 0, 0 ], [ 0, 1, 0 ], [ 0, 0, 1 ] ].
Related
boneBasis()An advanced helper: the orthonormal basis [x, y, z] of a bone, rebuilt from two joints and the palm normal the way LeapJS expects.
handAngles()Turns a hand's direction and palm normal into { pitch, yaw, roll } in radians, with the LeapJS conventions.
V6ConverterConverts raw hand lines into LeapJS v6 frames, keeping the little state the protocol needs: time visible, smoothed positions and fingertip velocities.
Which protocol version a /vN.json path asks for, as a number, or null when the path is not one.
The bridge uses it to decide whether a WebSocket upgrade is a LeapJS client at all: /v6.json yes, /raw is handled separately, anything else is refused with a 404.
Parameters
Name
Type
Description
pathname
String
The URL path, such as '/v6.json'.
Returns
Number | null The number in /v<number>.json, or null. It does not clip to 6: /v7.json gives 7 (the bridge then answers it as version 6).
One-off conversion of a raw frame to a LeapJS v6 frame, with a fresh converter each time (no history, so no velocities).
Handy for tests and for looking at what a legacy sketch would receive. For a stream, keep one V6Converter instead, so that timeVisible and tipVelocity can accumulate.
Parameters
Name
Type
Description
raw
Object
A raw frame { t: 'f', id, fps, hands }.
tUs
Number
The time in microseconds. Defaults to now. optional
opts
Object
Options for the V6Converter: { box, smoothing }. optional
Returns
Object The v6 frame. Every field the LeapJS Frame, Hand, Pointable, Finger and Bone constructors read is there.
module The console prints the frame's keys (id timestamp currentFrameRate hands pointables interactionBox gestures r s t), then 2 10 right 1.
Related
V6ConverterConverts raw hand lines into LeapJS v6 frames, keeping the little state the protocol needs: time visible, smoothed positions and fingertip velocities.
handsOf()Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
v6Header()The first message of a LeapJS v6 connection: { serviceVersion, version }, with version held to 1..6 because LeapJS throws above 6.
mockFrame()One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
The ports the Leap bridge refuses to use: [7457], which belongs to The Last Input and the backstage monitor.
startBridge({ port: 7457 }) throws "port 7457 belongs to The Last Input" immediately, so the bridge can never take the port the monitor listens on (node packages/monitor/server.js). The out-bridge refuses the same one.
Related
MonitorFeedStreams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.
Starts the Leap bridge on this machine: a loopback WebSocket and HTTP server that serves the sensor's hands to every page (Node only).
This is what node packages/leap/bridge/leap-bridge.mjs runs (add --mock for made-up hands, --port, --hz, --bin, --quiet). It serves ws://127.0.0.1:6437/v6.json (the LeapJS protocol, which every 2014 to 2021 browser sketch speaks), ws://…/raw (the raw lines), the same lines as server-sent events on GET /raw and /hands, and GET /health (source, service, device, fps, hands, clients). Any page origin may read it, as the original Leap service allowed.
Call it from your own Node program when you want the bridge inside it, for example in a test. startBridge() throws at once for port 7457 (it belongs to The Last Input and the monitor) and for a non-loopback host; a port already in use rejects the returned promise with a message that suggests curl http://127.0.0.1:6437/health.
Before you call it
Node 22 or later, and for real hands the Ultraleap Hand Tracking service plus Xcode's clang to build the reader on first start. With mock: true neither is needed.
Parameters
Name
Type
Description
port
Number
The port to listen on, DEFAULT_PORT unless you say otherwise. 0 picks any free one. The ports in RESERVED_PORTS (7457) are refused. optionaldefault DEFAULT_PORT
host
String
The address to bind. Only loopback (127.0.0.1, ::1, localhost) is allowed: hand data never leaves the machine. optionaldefault '127.0.0.1'
mock
Boolean
true serves two animated made-up hands (mockFrame()) instead of reading a sensor. optionaldefault false
hz
Number
The most frames per second sent to each client (held to 1..120). A slow client skips frames; it never queues them. optionaldefault 60
bin
String | null
The path to the leap-stream reader program. By default the bridge compiles its C reader on first start (needs the Xcode command line tools) and caches it. optionaldefault null
log
Function
Called with a line of text for each event worth knowing (clients coming and going, status changes). optionaldefault () => {}
Returns
Promise<Object> Resolves once listening, with { port, state, health(), close() }: the real port (useful with port: 0), the live state, a function returning the /health object, and a function that stops everything and returns a Promise.
static In Node: the first message is the handshake, handshake: { serviceVersion: 'mock', version: 6, bridge: 'open-audiovisual' }, then a frame arrives and the program prints hands in the frame: 2 and stops the bridge.
Related
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
mockFrame()One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
V6ConverterConverts raw hand lines into LeapJS v6 frames, keeping the little state the protocol needs: time visible, smoothed positions and fingertip velocities.
DEFAULT_URLThe default address LeapInput connects to: ws://127.0.0.1:6437/v6.json, the bridge on this machine speaking the LeapJS protocol.
The four levels of the leap/status signal by name: off 0, bridge 1, service 2, tracking 3.
A frozen object { off: 0, bridge: 1, service: 2, tracking: 3 }: 0 no bridge, 1 the bridge is there but the tracking service is not, 2 the service is there but no device, 3 tracking. Use it to read the signal without magic numbers: signals.get('leap/status') === STATUS.tracking. Note it is the number published as leap/status; leap.status is a different thing, a string ('tracking', 'no-device' …), and the simulator counts as 3.
Notes
The code in LeapInput writes the numbers itself and does not read this object, so STATUS is a convenience for your own code. The strings map as: tracking and simulated to 3, no-device to 2, bridge and no-service to 1, the rest to 0.
Related
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
deviceEvent()A LeapJS device event telling the client whether a device is attached and streaming, plus our own service flag.
mountLeapPanel()Mounts a small panel that shows whether the Leap is alive (bridge, service, device, hands) and draws what it sees, with a button to simulate hands.
The default hysteresis levels for the gestures: pinch 0.86 / 0.5, both-hands pinch 0.8 / 0.5, grab 0.8 / 0.5 (start above on, end below off).
A frozen object { pinch: { on, off }, bothPinch: { on, off }, grab: { on, off } }. The numbers come from The Last Input, tuned on a Leap Motion Controller 2, so one bridge means one feel. Change them per input with new LeapInput({ thresholds: { pinch: { on: 0.9, off: 0.6 } } }) or new HandGestures(...); you cannot change this object itself.
Related
HandGesturesTurns pinch and grab strengths into clean start and end events, per hand plus a both-hands pinch, using hysteresis so a wobbling value fires once.
HysteresisOne Schmitt trigger: update(v) says 'start' when the value rises above on, and 'end' only when it falls below off.
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
Converts raw hand lines into LeapJS v6 frames, keeping the little state the protocol needs: time visible, smoothed positions and fingertip velocities.
Every browser sketch written for the old Leap service (LeapJS, 2014 to 2021) speaks the v6 protocol, so the bridge serves it on /v6.json. This class does the conversion: one converter per stream. It is stateful where the protocol is: a hand's timeVisible grows while it stays in view and starts over when it leaves, stabilized… positions are a light low-pass, and tipVelocity is a finite difference between two frames.
LeapC gives joints, not bone rotations, so the bone bases are rebuilt from the joints and the palm normal (boneBasis()). Gestures are not synthesised: gestures is always [].
Parameters
Name
Type
Description
box
Object
The interaction box written into every frame's interactionBox. optionaldefault INTERACTION_BOX
smoothing
Number
0..1: how strongly stabilizedPalmPosition and stabilizedTipPosition lag behind (0 follows exactly, close to 1 barely moves). optionaldefault 0.5
convert(raw, tUs = now): a raw frame { t: 'f', id, fps, hands } and its time in microseconds, to a v6 frame { id, timestamp, currentFrameRate, hands, pointables, interactionBox, gestures: [], r, s, t }. Pointable ids are handId × 10 + finger.
Examples
js
import { V6Converter, mockFrame } from'@openav/leap';
const converter = new V6Converter();
converter.convert(mockFrame(1.0), 1_000_000); // microsecondsconst frame = converter.convert(mockFrame(1.1), 1_100_000);
console.log(frame.hands[0].timeVisible); // 0.1: seen for a tenth of a second
console.log(frame.pointables.length, frame.pointables[1].tipVelocity); // 10 [ -21, 17, 28 ] mm per second
module The console prints 0.1, then 10 pointables and the index fingertip's velocity [ -21, 17, 28 ] (millimetres per second between the two frames).
Related
rawToV6()One-off conversion of a raw frame to a LeapJS v6 frame, with a fresh converter each time (no history, so no velocities).
v6Header()The first message of a LeapJS v6 connection: { serviceVersion, version }, with version held to 1..6 because LeapJS throws above 6.
deviceEvent()A LeapJS device event telling the client whether a device is attached and streaming, plus our own service flag.
handsOf()Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
v6Header(serviceVersion = 'unknown', version = PROTOCOL_VERSION)
The first message of a LeapJS v6 connection: { serviceVersion, version }, with version held to 1..6 because LeapJS throws above 6.
The bridge sends this (plus bridge: 'open-audiovisual') as soon as a client connects to /v6.json, and LeapInput turns it into the status 'bridge'. Ask for 7 and you get 6.
Parameters
Name
Type
Description
serviceVersion
String
The tracking service version to report, such as '5.20.0'. optionaldefault 'unknown'
version
Number
The protocol version the client asked for. Held to 1..6. optionaldefault PROTOCOL_VERSION
Returns
Object{ serviceVersion: String, version: Number }.
Tracks up to two hands from the camera with 21 landmarks each and publishes palm position and thumb-to-finger pinch distances as hand/* signals.
The fine-control tracker: each hand gives two precise continuous controllers, thumb to index and thumb to middle finger, so two hands are four controllers you can pinch. The distances are normalised by the size of your palm (wrist to the base of the middle finger), so moving toward or away from the camera does not change a pinch: stage-robust. It runs on the device with MediaPipe Tasks Vision and downloads nothing until enable(). In a show, createShow({ modules: { hands: true } }) builds one (show.hands) and the L1 panel gets a 🖐 hands button.
For each of left and right it publishes hand/<side>/present (0 or 1), hand/<side>/x and /y (the palm, 0..1; y flipped so 1 is raised), hand/<side>/pinch/index and hand/<side>/pinch/middle (0..1, 0 = touching, 1 = wide open) and hand/<side>/spread (index fingertip to little fingertip, 0..1). A side with no hand in view gets present 0 and keeps its other values. Note that these names do not start with pose/, and are different from the body tracker's.
A pinch is a distance, so for "pinch closes" use a route with invert: { source: 'hand/right/pinch/index', target: 'tightness', invert: true, smooth: 0.1 }. If you want pinch start and end as events, LeapInput's gestures do that; HandTracker only gives the continuous distances.
Before you call it
Call await hands.enable(): it loads the MediaPipe library from jsDelivr and the hand model from Google's model storage (so it needs the network), asks for the camera (https or localhost, and the user must allow it) and starts the loop. Until then the signals sit at their starting values.
Parameters
Name
Type
Description
signals
Signals | null
The Signals registry to publish into. Without one the tracker still runs and keeps hands, but publishes nothing. optionaldefault null
mirror
Boolean
Treat the camera as a mirror: x is flipped, and the left and right labels MediaPipe gives are swapped so the signals match the body. optionaldefault true
enable({ videoEl = null } = {}): ask for the camera (640 × 360), load the model (up to two hands) and start detecting. videoEl is a <video> to use; without one an off-screen video is created. Resolves true; rejects if the camera is refused or the library or model cannot load.
Draw the hands onto a 2D canvas context: skeleton(ctx, w, h, { colorLeft, colorRight, lineWidth = 2 }), the 21 landmarks joined into finger bones, with bigger dots at the thumb, index and middle fingertips (the pinch control points). Left is green and right is yellow by default.
Notes
Fields to read: hands ([{ handed: 'left' | 'right', lm }], lm the 21 landmarks { x, y, z } in the camera image, y downward and not mirrored), running, mirror, video.
Landmark indexes used: wrist 0, thumb tip 4, index tip 8, middle tip 12, little finger tip 20, base of the middle finger 9 (the palm position and the size reference).
Examples
js
import { Signals } from'@openav/core';
import { HandTracker } from'@openav/pose';
const signals = new Signals();
const hands = new HandTracker({ signals });
document.addEventListener('click', () => hands.enable(), { once: true }); // the camera prompt, then the model
signals.on('hand/right/pinch/index', (distance) => {
if (distance < 0.1) console.log('right hand pinches thumb and index');
});
static Click the page and allow the camera; hold a hand up and pinch thumb to index finger: the console prints right hand pinches thumb and index while they touch.
module An open right hand is drawn in yellow on the canvas from made-up landmarks, with bigger dots at the thumb, index and middle fingertips: the overlay skeleton() gives you over the camera picture.
To be confirmed
Not tried here with a live camera, so the real range of the pinch values (for example how close to 0 a firm pinch gets) is not measured.
Related
PoseTrackerTracks one body from the camera and publishes wrist positions, wrist speed, hand spread, height and lean as pose/* signals.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
Tracks one body from the camera and publishes wrist positions, wrist speed, hand spread, height and lean as pose/* signals.
This is the body tracker: one person, 33 landmarks (the BlazePose set), run by MediaPipe Tasks Vision on the device. No video leaves the machine, and nothing is downloaded until you call enable(). For fine finger control use HandTracker instead. In a show, createShow({ modules: { pose: true } }) builds one and the console's L1 panel gets a 🕺 body button that calls enable(); it is show.pose.
Signals, all normalised, with y flipped so that 1 means raised and 0 means the bottom of the frame (performance meaning beats pixel meaning): pose/present (0 or 1: someone is visible), pose/hand/left/x and /y, pose/hand/right/x and /y (wrist positions, 0..1), pose/hand/left/v and pose/hand/right/v (wrist speed, 0..1, clamped), pose/hands/spread (distance between the wrists, 0..1), pose/height (nose height, 0..1: crouching to standing) and pose/lean (shoulder-line tilt, −1..1). When nobody is in view only pose/present goes to 0; the others keep their last values.
Map these to params ({ source: 'pose/hand/right/y', target: 'energy', smooth: 0.1 }) so a raised hand does what a knob would. The detection runs on every animation frame while running.
Before you call it
Call await pose.enable(): it loads the MediaPipe library from jsDelivr and the model from Google's model storage (so it needs the network), asks the browser for the camera (the page needs https or localhost, and the user must allow it), and starts the loop. Until then every signal sits at its starting value.
Parameters
Name
Type
Description
signals
Signals | null
The Signals registry to publish into. Without one the tracker still runs and keeps landmarks, but publishes nothing. optionaldefault null
mirror
Boolean
Treat the camera as a mirror: x is flipped, so moving toward the viewer's right makes x grow. Natural for performers. false keeps the camera's own left and right. optionaldefault true
enable({ videoEl = null } = {}) asks for the camera (640 × 360), loads the model and starts detecting. videoEl is a <video> element to use; without one an off-screen video is created. Resolves true; rejects if the camera is refused or the library or model cannot be loaded.
Draw the body onto a 2D canvas context: skeleton(ctx, w, h, { color, lineWidth = 2 }), the bones of the torso, arms and legs plus dots at the nose, shoulders, wrists and hips. It follows the same mirroring as the signals but not the y flip (it draws in screen space). Does nothing without landmarks. Call it every frame after drawing the background.
Notes
Fields to read: landmarks (the latest 33 raw points { x, y, z, visibility }, x and y 0..1 in the camera image, y downward and not mirrored; null when nobody is seen), running, mirror, video.
Landmark indexes (BlazePose): nose 0, shoulders 11 and 12, wrists 15 and 16, hips 23 and 24.
The model is MediaPipe's "lite" pose landmarker on the GPU, one pose at a time. If two people are in view, only one is tracked.
Examples
js
import { Signals } from'@openav/core';
import { PoseTracker } from'@openav/pose';
const signals = new Signals();
const pose = new PoseTracker({ signals });
document.addEventListener('click', async () => {
await pose.enable(); // camera prompt, then the model downloads
console.log('tracking');
}, { once: true });
setInterval(() => {
console.log('raised hand', signals.get('pose/hand/right/y').toFixed(2), 'present', signals.get('pose/present'));
}, 500);
static Click the page and allow the camera: the console prints tracking, then every half second the height of the right wrist (1 = raised) and whether someone is in view.
module A stick figure with one arm raised is drawn on the canvas from made-up landmarks: the overlay skeleton() draws over the camera picture, without needing a camera.
static Click to start the camera, then raise your right hand: the bar at the bottom of the canvas grows with it, because the pose signal is mapped to a param.
To be confirmed
Not tried here with a live camera. In particular, which person's hand pose/hand/left/* follows in practice was not checked: the code takes landmark 16 for left and landmark 15 for right when mirror is true (the other way round when false), and the package docs say left is the viewer's left.
Whether the camera permission prompt needs a click first was not tested; the package README calls enable() from a button.
Related
HandTrackerTracks up to two hands from the camera with 21 landmarks each and publishes palm position and thumb-to-finger pinch distances as hand/* signals.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
Groups notes struck close together into one gesture and analyses it: chord type, consonance, triad and cluster flags, published as chord/* signals.
This is not a music-theory library. It is a performance semantics layer: raw notes in, one analysed event per gesture out, with features that are made to drive a work. A single note is a seed, a chord a colony, a consonant chord a flourish, a dissonant one decay; rename them for your own piece. In a show, createShow({ modules: { chord: true } }) builds one ({ chord: { window: 120 } } tunes it) and connects it to midi/note/on and midi/note/off, so a MIDI keyboard, the on-screen piano and phones all feed it.
Each closed gesture publishes chord/consonance (−1..1), chord/count (0..10), chord/root (the lowest note) and the pulse chord/event, whose payload is the whole analysis: { notes, count, root, vel, pcs, consonance, isConsonant, isDissonant, isTriad, thirdsFraction, dissonanceLevel, chordType }. chordType is one of single, major, minor, sus2, sus4, dim, aug, maj7, min7, dom7, halfdim7, maj9, min9, dom9, six, min6, cluster (a very dissonant pile) or chord (anything else). Inversions resolve to the root-position name.
consonance is the average of an interval weight for every pair of notes (an octave or fifth is high, a semitone or tritone is negative). isConsonant is consonance > 0.25 and isDissonant is consonance < −0.15, both only for two notes or more. dissonanceLevel is 0 for none, 1 for mild (a storm warning: foreshadow it) and 2 for severe (three or more notes that are a true cluster: commit to the decay). isTriad is true for major, minor, sus2, sus4, dim and aug, and thirdsFraction is the share of neighbouring notes exactly 3 or 4 semitones apart, so "genuinely stacked thirds" can be told from "some consonant notes".
Before you call it
Feed it with noteOn(note, vel) (and noteOff(note)), or let createShow do it from midi/note/on|off. Nothing is published until window milliseconds after the last noteOn, so the result is asynchronous. analyze() is the exception: it is instant and needs no setup.
Parameters
Name
Type
Description
window
Number
How many milliseconds after the lastnoteOn the gesture is closed and analysed. Notes struck within this time of each other are one chord. optionaldefault 80
onChord
Function | null
Called with the analysis object each time a gesture is closed. optionaldefault null
signals
Signals | null
A Signals registry to publish chord/* into. The signals are declared when the detector is built. optionaldefault null
A note was struck: noteOn(note, vel = 0.8), note a MIDI number and vel 0..1. Restarts the window timer; striking the same note twice in a gesture counts once, with the harder velocity.
A pure function: analyze(notes, vels = []) takes MIDI note numbers (lowest first) and returns the analysis object at once, without timers or signals. vel is the mean of vels, or 0.8 without them.
Notes
Pass notes to analyze() lowest first: root is the first note, and the detector sorts for you but a direct call does not.
A major second like [60, 62] is typed cluster (no template fits and its consonance is below −0.3) even though its dissonanceLevel is only 1. Level 2 needs three or more notes.
The gesture window does not know about held notes: a note already down when you play two more does not join them.
module Inversions resolve to the root-position name: the console prints major, major (E-G-C is C major again), dom7, dim, sus4, and chord for the bare fifth, which matches no template.
Related
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
Sends OSC messages from the browser to TouchDesigner, Max, Reaper or a spatialiser, batched once per frame, through a small local bridge.
A browser cannot send UDP, so OscOut posts batches of messages to a ~100-line Node program, packages/osc/bridges/osc-bridge.js, which sends them on as OSC packets. Start it with node packages/osc/bridges/osc-bridge.js 7456 192.168.1.50 3456 (HTTP port, target host, UDP port; the defaults are 7456 127.0.0.1 3456). Then send(address, args) as often as you like and flush() once per frame: only the latest message per address survives a frame, because a position stream does not need its in-between values.
Numbers go out as OSC floats (what Spat, Reaper and TouchDesigner expect), and anything else as a string. createShow() does not create an OscOut: make one, call enable(), flush it from onFrame, and if you like hand it to the worlds as show.stage.io.osc and to the Layers panel as show.app.osc (which then shows the OSC rate).
Before you call it
The bridge must be running, and you must await osc.enable() first: it probes GET /health and answers false if nobody is there. Until it succeeds flush() just throws the queue away. On a public https page, Chrome asks permission before the page may reach 127.0.0.1.
Parameters
Name
Type
Description
url
String
The address of the OSC bridge (its HTTP port, not the UDP target). optionaldefault 'http://127.0.0.1:7456'
Probe the bridge (GET /health) and, if it answers, switch on. Resolves true, or false when nothing is listening. Any HTTP answer counts as present. Calls onStatus(ok, info); info is the UDP target the bridge reports.
Queue one message: send(address, args). args is an array, or a single value that is wrapped in one. Nothing leaves until flush(). Calls the onMsg(address, args) hook if you set one.
Send the queue as one POST, the latest message per address. Call once per frame. Async; it does nothing while a previous flush is still in flight, and throws the queue away when enabled is false.
Notes
Fields and hooks: enabled, udp (the target the bridge reported), onStatus = (ok, info) => {} and onMsg = (address, args) => {} (high frequency: use sparingly).
If the bridge disappears, the first failed flush calls onStatus(false, 'bridge lost'), and after more than 30 failed flushes in a row enabled turns false by itself, with no further message. Call enable() again to resume. The messages of a failed flush are lost.
Every number goes out as a 32-bit float, whole numbers included (the bridge's own header comment mentions integers, but its code only writes floats). If the receiver needs integers, convert there.
Examples
js
import { OscOut } from'@openav/osc';
const osc = new OscOut(); // http://127.0.0.1:7456
console.log(await osc.enable()); // true when the bridge answersfunction frame(x, y, z) {
osc.send('/source/3/xyz', [x, y, z]); // queue as often as you like
osc.flush(); // once per frame
}
frame(0.1, 0.5, 0.9);
static With node packages/osc/bridges/osc-bridge.js running, the console prints true and the bridge forwards /source/3/xyz as an OSC packet with three floats.
js
import { OscOut } from'@openav/osc';
// a stand-in for the bridge, so this runs anywhere
globalThis.fetch = async (url, options = {}) => {
if (options.body) console.log('POST', url, options.body);
return { json: async () => ({ ok: true, udp: '127.0.0.1:3456' }) };
};
const osc = new OscOut();
osc.onStatus = (ok, info) => console.log('status', ok, info);
await osc.enable();
osc.send('/source/1/xyz', [0.1, 0.2, 0.3]);
osc.send('/source/1/xyz', [0.4, 0.5, 0.6]); // the same address: replaces the first
osc.send('/gain', 0.8); // a single value is wrapped in an arrayawait osc.flush();
console.log('rate', osc.rate()); // 2 messages went out
module Three send() calls become two messages: the console prints status true 127.0.0.1:3456, then the one POST with /source/1/xyz carrying [0.4,0.5,0.6] and /gain carrying [0.8], then rate 2.
js
import { OscOut } from'@openav/osc';
const osc = new OscOut('http://127.0.0.1:7499'); // an address where no bridge runs
osc.onStatus = (ok, info) => console.log('status', ok, info);
console.log(await osc.enable()); // false: nobody answers
osc.send('/gain', 0.5);
await osc.flush(); // nothing is sent while disabled: the queue is dropped
module With no bridge, enable() does not throw: the console prints status false bridge not running (node bridges/osc-bridge.js), then false. (The browser may also log one failed request.)
To be confirmed
Not tried here against TouchDesigner, Max or Reaper: the examples use a stand-in for the bridge, and what the bridge does with the packets is read from its code (floats for numbers, strings for the rest).
Related
OutClientThe page side of the out-bridge: set(channel, value) sends a number to a named hardware channel, at most once per frame, and keeps the connection alive.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
The default of startBridge({ port }) and of --port; the environment variable OUT_BRIDGE_PORT changes it for the command line. It is the out-bridge's port: the Leap bridge has its own (DEFAULT_PORT of @openav/leap, 6437).
Related
OUT_DEFAULT_URLThe address OutClient connects to by default: ws://127.0.0.1:6450/out, the out-bridge on this machine.
The watchdog time when the config does not say: 500 milliseconds without a value, and a channel is put on its safe value.
It is the default of timeoutMs in the bridge config (the minimum allowed is 20). OutClient re-sends held values every third of whatever the bridge reports, so a still slider is not mistaken for a dead page. Lower it for hardware that must stop quickly, but keep it above the slowest frame your page may have.
Related
OutClientThe page side of the out-bridge: set(channel, value) sends a number to a named hardware channel, at most once per frame, and keeps the connection alive.
Turns a channel glob like light/* into the regular expression the bridge matches channel names with: only * is special, and it crosses slashes.
The match of a channel rule and the keys of safe in the bridge config are globs, and this is how both are read. Remember that * also crosses slashes: light/* fits light/0/r, and * alone fits every channel.
Parameters
Name
Type
Description
glob
String
The pattern. * matches any run of characters, slashes included. Everything else, including ., ? and +, matches itself.
Returns
RegExp An anchored expression: globToRegExp('light/*') is /^light\/.*$/.
Examples
js
import { globToRegExp } from'@openav/out/bridge';
const lights = globToRegExp('light/*');
console.log(lights.test('light/0/r'), lights.test('motor/1')); // true false
console.log(globToRegExp('*').test('any/channel/at/all')); // true
console.log(globToRegExp('a.b').test('axb')); // false: the dot is only a dot
static In Node: the console prints true false, then true, then false.
Related
loadConfig()Reads the out-bridge config from a JSON file and returns it as an object (Node only); throws when the file cannot be read.
Reads the out-bridge config from a JSON file and returns it as an object (Node only); throws when the file cannot be read.
A thin helper around fs.readFileSync and JSON.parse. A missing file or invalid JSON throws one error that names the file (its text is written in Chinese in the code). The command line uses it for --config out.json; in your own program you may just as well pass an object to startBridge({ config }).
Parameters
Name
Type
Description
file
String
The path of the JSON file, such as out.json (copy packages/out/bridge/out.example.json to start).
Returns
Object The parsed config, unchecked: startBridge() is what validates it.
static In Node: the console prints 1 { 'light/*': 0 }, then cannot load: true: the error names the missing file.
Related
globToRegExp()Turns a channel glob like light/* into the regular expression the bridge matches channel names with: only * is special, and it crosses slashes.
The address OutClient connects to by default: ws://127.0.0.1:6450/out, the out-bridge on this machine.
Port 6450 is the bridge's DEFAULT_PORT and /out is the only WebSocket path it serves. In the lab, ?out=ws://host:port/out points a page at another bridge, ?out=sim records values without one and ?out=0 turns it off.
Related
OutClientThe page side of the out-bridge: set(channel, value) sends a number to a named hardware channel, at most once per frame, and keeps the connection alive.
new OutClient({ url = OUT_DEFAULT_URL, id = null, WebSocket: WS, schedule, maxBackoffMs = 5000, minBackoffMs = 250 } = {})
The page side of the out-bridge: set(channel, value) sends a number to a named hardware channel, at most once per frame, and keeps the connection alive.
Hardware needs a number, and a page has a number: out.set('light/0/r', 0.8). The channel name is yours (a-z A-Z 0-9 _ - . /, up to 128 characters) and the bridge (startBridge()) decides from its config where that channel goes: a WLED strip, a serial motor board, an OSC address. Values are plain numbers; the convention is 0..1. Several set() calls on one channel inside one frame leave as one value, the last. In a lab work lab.out is this class.
It keeps the connection alive on its own: it reconnects with a growing delay (250 ms doubling up to 5 s), resends everything it knows after a reconnect (a restarted bridge has forgotten it all), and while the socket is open it re-sends the held values every timeoutMs / 3 of the bridge's watchdog, so a slider that stands still is not mistaken for a frozen page. If the page really freezes, that heartbeat stops and the bridge puts the hardware on its safe values.
Before you call it
Call connect() (the bridge must be running: node packages/out/bridge/out-bridge.mjs --config out.json), or simulate() for no hardware at all. set() before the socket is open is not lost: it waits in last and goes out on open.
Parameters
Name
Type
Description
url
String
The bridge's WebSocket address. The default is OUT_DEFAULT_URL, ws://127.0.0.1:6450/out. optionaldefault OUT_DEFAULT_URL
id
String | null
A name for this page (the work's id), sent to the bridge in its "hello" and shown in its log. optionaldefault null
WebSocket
Function
The WebSocket constructor to use. Defaults to the global one (also in Node 22). Pass your own to test without a network. optional
schedule
Function
A function that runs a callback once, on the next frame. The default is requestAnimationFrame (or a 16 ms timer when the tab is hidden or there is no requestAnimationFrame). optional
maxBackoffMs
Number
The longest wait between reconnection attempts, in milliseconds. optionaldefault 5000
minBackoffMs
Number
The first wait between reconnection attempts. It doubles each time up to maxBackoffMs. optionaldefault 250
Open the socket and keep it open (reconnects with a capped, doubling delay). Returns the client, so new OutClient(…).connect() chains. Does nothing in simulate mode, or when there is no WebSocket constructor.
Close the socket and stop reconnecting. The status becomes 'closed'. The bridge sees the last client leave and puts every channel with a safe value back on it.
set(channel, value): send one number. The value goes through Number(); a channel that is not a string or a value that is not finite is ignored. Recorded in last at once, sent on the next frame.
batch({ channel: value, … }): the same as one set() per entry.
Notes
Fields to read: status ('closed', 'connecting', 'open', 'sim'), last ({ channel: last value given to set() }, also in simulate mode), sent (values actually put on the wire: less than the number of set() calls because of coalescing) and bridge (the bridge's last status message).
On the wire a single value is { "type": "set", "ch", "v" } and several are { "type": "batch", "items": [{ "ch", "v" }, …] }, after a { "type": "hello", "id" }.
On a public https page, Chrome asks permission before the page may reach 127.0.0.1; if it is refused, the status stays 'connecting'.
Examples
js
import { OutClient } from'@openav/out';
const out = new OutClient({ id: 'my-work' }); // ws://127.0.0.1:6450/out
out.onStatus((status) => console.log('out', status)); // connecting, open
out.connect();
out.set('light/0/r', 0.8); // waits for the socket, then goes outfunction frame(t) {
out.set('light/0/g', 0.5 + 0.5 * Math.sin(t / 1000)); // call as often as you like: one value a frame leaves
requestAnimationFrame(frame);
}
requestAnimationFrame(frame);
static With the out-bridge running, the console prints out connecting, then out open, and the green channel breathes between 0 and 1.
js
import { OutClient } from'@openav/out';
class FakeSocket { // stands in for the bridge's socket, so this runs anywhere
constructor() { this.readyState = 1; setTimeout(() => this.onopen(), 0); }
send(text) { console.log('wire:', text); }
close() {}
}
const out = new OutClient({ id: 'demo', WebSocket: FakeSocket, schedule: (fn) => setTimeout(fn, 0) });
out.connect();
setTimeout(() => {
out.set('light/0/r', 0.8);
out.set('light/0/r', 0.9); // same channel, same frame: only 0.9 leaves
out.batch({ 'light/0/g': 0.2, 'motor/1': 0.5 });
setTimeout(() => { console.log('sent', out.sent, out.status); out.close(); }, 50);
}, 20);
module The console prints the hello, then one batch of three values (light/0/r is 0.9, not 0.8), then sent 3 open.
js
import { OutClient } from'@openav/out';
const out = new OutClient().simulate(); // no bridge, no hardware
out.set('light/0/r', 1);
out.set('light/0/r', 0.25);
out.set('typo', 'not a number'); // ignored: not finite
console.log(out.status, out.last); // sim { 'light/0/r': 0.25 }
module Simulate mode records instead of sending: the console prints sim { 'light/0/r': 0.25 }; the bad value was ignored.
To be confirmed
Not tried here with real hardware (a WLED strip, a motor board): the examples use a stand-in socket and the bridge's log sink.
Related
OUT_DEFAULT_URLThe address OutClient connects to by default: ws://127.0.0.1:6450/out, the out-bridge on this machine.
OscOutSends OSC messages from the browser to TouchDesigner, Max, Reaper or a spatialiser, batched once per frame, through a small local bridge.
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
The ports the out-bridge refuses to use: [7457], which belongs to The Last Input and the backstage monitor.
startBridge({ port: 7457 }) throws at once, so the bridge can never take the port the monitor listens on. The Leap bridge refuses the same one (RESERVED_PORTS of @openav/leap).
Related
MonitorFeedStreams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.
Starts the out-bridge: a loopback WebSocket server that routes channel values from the page to hardware sinks, with a watchdog that falls back to safe values (Node only).
This is what node packages/out/bridge/out-bridge.mjs --config out.json runs (also --port, --no-watchdog, --quiet; the environment variable OUT_BRIDGE_PORT). It serves ws://127.0.0.1:6450/out, which understands { type: 'hello', id }, { type: 'set', ch, v }, { type: 'batch', items } and { type: 'status' }, and answers with { type: 'status', … }; GET /health reports the sinks, the channel count, the clients and the time since the last value. Channel names are a-z A-Z 0-9 _ - . / up to 128 characters, values must be finite numbers, and there is room for 4096 channels.
The config routes channels: channels is a list of { match, sink, … } and the first rule whose match glob fits the channel name takes it (* means anything, slashes included); a channel no rule fits is counted and dropped. The sinks are log (stdout), udp-json (UDP, {"ch","v"}), osc (UDP, address "/" + channel, one float32), ddp (WLED over UDP 4048: light/<i>/r|g|b, 0..1, pixel index from 0, up to 10000 pixels) and serial (a device path, ch value lines or JSON, latest value only). safe maps a channel glob to its safe value, timeoutMs (default 500, at least 20) is the watchdog time, and allowOrigins lists the page origins that may connect (without it any page on this machine can).
The watchdog: a channel that has a safe value and gets no new value for timeoutMs is sent its safe value once (a new value re-arms it). When the last client disconnects, every channel the bridge has seen that has a safe value gets it at once, and so does a clean shutdown. It does not cover channels the bridge has never heard of, nor a bridge killed with SIGKILL.
It throws at once for port 7457 (it belongs to The Last Input and the monitor), for a host that is not loopback, and for a bad config (no channels, an unknown sink, a safe value that is not a number, a timeoutMs under 20). A port already in use rejects the returned promise.
Before you call it
Node 22 or later. The sinks need what they talk to: a WLED device for ddp, the device file for serial, a listener for osc.
Parameters
Name
Type
Description
port
Number
The port to listen on, DEFAULT_PORT unless you say otherwise. 0 picks any free one. The ports in RESERVED_PORTS (7457) are refused. optionaldefault DEFAULT_PORT
host
String
The address to bind. Only loopback is allowed. optionaldefault '127.0.0.1'
config
Object | null
The routing: { channels: [{ match, sink, … }], safe: { glob: value }, timeoutMs, allowOrigins } (see the description). null sends every channel to the log sink. loadConfig(file) reads it from a JSON file. optionaldefault null
watchdog
Boolean
false switches the watchdog off: no safe values are ever sent (the tests use it to prove the watchdog is what does the work). optionaldefault true
log
Function
Called with a line of text for diagnostics: clients coming and going, safe values sent. optionaldefault () => {}
print
Function
What the log sink prints each value with ("channel value"). The default is stdout. optionaldefault (s) => console.log(s)
Returns
Promise<Object> Resolves once listening, with { port, state, health(), close() }: the real port, the counters (received, rejected, unrouted, safeSent …), a function returning the /health object, and a function that puts every known channel on its safe value, stops the server and returns a Promise.
Notes
The error messages the bridge throws and sends back to a page are written in Chinese in the code, so a page may show them in Chinese whatever its language.
Examples
js
import { startBridge } from'@openav/out/bridge';
import { OutClient } from'@openav/out';
const bridge = await startBridge({
port: 0, // any free port
config: { channels: [{ match: '*', sink: 'log' }], safe: { 'light/*': 0 }, timeoutMs: 2000 },
print: (line) => console.log('sink:', line),
});
const out = new OutClient({ url: 'ws://127.0.0.1:' + bridge.port + '/out', id: 'demo' }).connect();
setTimeout(() => out.set('light/0/r', 0.8), 100); // reaches the sink
setTimeout(() => out.close(), 300); // the last client leaves: safe values go out
setTimeout(() => bridge.close(), 600);
static In Node: the console prints sink: light/0/r 0.8, then sink: light/0/r 0: when the page goes away the bridge puts the light channel on its safe value.
To be confirmed
The ddp (WLED) and serial sinks were not tried here with real hardware; what the description says about them comes from the package README and the code. The log sink and the watchdog were run.
Related
OutClientThe page side of the out-bridge: set(channel, value) sends a number to a named hardware channel, at most once per frame, and keeps the connection alive.
loadConfig()Reads the out-bridge config from a JSON file and returns it as an object (Node only); throws when the file cannot be read.
globToRegExp()Turns a channel glob like light/* into the regular expression the bridge matches channel names with: only * is special, and it crosses slashes.
DEFAULT_TIMEOUT_MSThe watchdog time when the config does not say: 500 milliseconds without a value, and a channel is put on its safe value.
new LearnWizard({ mapper, keys, onChange = null })
Binds a list of params one after another: it arms learn for the first, you turn a knob, it binds it and arms the next. Skip, back and stop steer it.
Binding twelve params one by one with the learn chip means twelve clicks and twelve guesses about what to touch next. The wizard walks the list for you: start() arms learn for keys[0], wizard.current says which param it is waiting for ('Turn a knob for: hue (1/12)'), you move a control, it is bound and the next one is armed. At the end learn is disarmed. The console's 🎛 map knobs in order button is this class.
It is pure logic on top of the mapper, with no screen of its own. Learn only ever adds routes, so running the wizard twice puts a second control on a param instead of losing the first. While it runs, it owns learn mode: call sync() every frame so that a click on some other learn chip cannot leave it waiting for nothing.
Call every frame: if learn mode was switched off or pointed elsewhere while the wizard runs, it re-arms the current param.
Notes
Fields to read: active, done (the run reached the end), index, total, order, and added ([{ key, source }], what was bound in this run).
While it runs the wizard wraps the mapper's onLearn and puts yours back when it ends, so your own onLearn keeps being called.
Examples
js
import { Signals, Params } from'@openav/core';
import { Mapper, LearnWizard } from'@openav/mapping';
const signals = new Signals();
for (const name of ['knob/1', 'knob/2']) signals.define(name, { min: 0, max: 1 });
const params = new Params([{ key: 'hue', min: 0, max: 360, def: 0 }, { key: 'size', min: 0, max: 1, def: 0.2 }]);
const mapper = new Mapper({ signals, params });
const wizard = new LearnWizard({ mapper, keys: () => params.schema.map((p) => p.key) });
wizard.start();
console.log('waiting for', wizard.current); // hueconst turn = (name) => { signals.set(name, 0); signals.set(name, 0.9); }; // a knob that moves a lot
turn('knob/1'); console.log('waiting for', wizard.current); // size
turn('knob/2'); console.log(wizard.done, wizard.added);
module The console prints waiting for hue, waiting for size, then true and the two bindings [ { key: 'hue', source: 'knob/1' }, { key: 'size', source: 'knob/2' } ].
Related
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
mountConsole()Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.
Fetches a map file from an address (what ?map= calls) and puts it on the mapper; it never throws and never blocks the show.
Fetches the address with credentials: 'omit', and importMap()s the text. Any failure (refused address, no fetch, the server answered 404, offline, a site that blocks cross-origin reads, a timeout, a file that is not a map) lands in the report and in mapper.report, where the console shows it, and the routes stay as they were. It is async, so createShow() does not wait for it and the show starts regardless.
A map on another site must be served with CORS (Access-Control-Allow-Origin: *); raw GitHub files and gists are. In a show you rarely call it: open the page with ?map=<url>, or pass map: 'maps/a.json' to createShow().
The address of the map file, checked by parseMapUrl() first.
base
String
The page address that relative addresses resolve against. optionaldefault location.href
fetchImpl
Function
The function that does the request. Replace it to test without a network. optionaldefault fetch
timeoutMs
Number
Give up after this many milliseconds. optionaldefault 8000
Returns
Promise<Object> The report: what mapper.importMap() returned (source: 'url', address, added, skipped …), or { ok: false, source: 'url', address, error }. The same report is kept in mapper.report.
module With a stand-in for the network, the console prints true https://openaudiovisual.com/x/maps/stage-left.json 1: the route came in.
js
import { Signals, Params } from'@openav/core';
import { Mapper, loadMapUrl } from'@openav/mapping';
const mapper = new Mapper({ signals: new Signals(), params: new Params([]) });
const report = await loadMapUrl(mapper, 'http://example.com/map.json', { base: 'https://openaudiovisual.com/' });
console.log(report.ok, report.error); // false 'a map from another site must be served over https'
console.log(mapper.report === report); // true: the console would show it too
module A refused address is a report, not an exception: the console prints false a map from another site must be served over https, then true.
Related
parseMapUrl()Decides whether a ?map= address may be fetched: same origin, or an absolute https URL, and nothing else.
parseMap()Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
The text "openav.map": the value of format in every map file. A file with another format is refused.
A map file is JSON with "format": "openav.map", a version, a world name and a list of routes. parseMap() refuses anything whose format is not this string, so you cannot import some other tool's JSON by accident. The one exception is a bare array (what mapper.toJSON() returns), which is read as version 0.
Related
MAP_VERSIONThe newest map file version this build reads and writes: 1. A file with a higher version is refused with a message.
MAP_MAX_BYTESThe longest map text parseMap() will read: 262144, which is 256 KB. Longer text is refused without being parsed.
parseMap()Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
serializeMap()Builds the portable map file object from a list of routes: { format, version, world, exported, routes }.
The longest map text parseMap() will read: 262144, which is 256 KB. Longer text is refused without being parsed.
The limit is compared with the length of the text (characters), before any parsing, so a huge file cannot freeze the page. A map with more than 500 routes is refused as well, whatever its size.
Related
parseMap()Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
MAP_FORMATThe text "openav.map": the value of format in every map file. A file with another format is refused.
MAP_VERSIONThe newest map file version this build reads and writes: 1. A file with a higher version is refused with a message.
The newest map file version this build reads and writes: 1. A file with a higher version is refused with a message.
serializeMap() writes this number and parseMap() accepts any version from 1 up to it. A newer file gets the message "this build of OAV reads up to version 1. Update OAV or re-export it." A file with no valid version is refused too. Only a bare array of routes counts as version 0.
Related
MAP_FORMATThe text "openav.map": the value of format in every map file. A file with another format is refused.
MAP_MAX_BYTESThe longest map text parseMap() will read: 262144, which is 256 KB. Longer text is refused without being parsed.
parseMap()Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
The mapper is the only component that connects signals to params, and everything it knows is a list of routes, plain objects. mapper.addRoute({ source: 'midi/cc/74', target: 'bloom' }) makes the param bloom follow the signal midi/cc/74. One signal may drive several params, one param may listen to several signals (the last writer wins), and learn adds routes, it never replaces them. In a show, createShow() builds one, loads the saved routes (or adds the ones you declared in routes) and returns it as show.mapper.
A route needs only source and target; addRoute() fills in the rest and returns the stored route:
text
source, target the signal name and the param key (required)
inMin, inMax 0, 1 the part of the signal that counts; outside it is clamped
outMin, outMax the param's min and max AT THE MOMENT THE ROUTE IS ADDED (0 and 1 if the param does not exist yet)
curve 'linear' | 'exp' (x squared) | 'log' (square root) | 'smooth' (smoothstep); an unknown name is linear
invert false flip the value (1 - x) before the curve
smooth 0 glide time in seconds; 0 = instant
enabled absent = on; false mutes the route
id assigned; used by removeRoute()
When a signal arrives, each enabled route with that source does five things in order: take the window ((value - inMin) / (inMax - inMin), clamped to 0..1), invert if asked, apply the curve, scale into outMin..outMax, and write. With smooth: 0 the param is overridden at once. With smooth > 0 the result becomes a target that update(dt) glides toward: after smooth seconds the param has covered about 63% of the way. If the target param is a pulse param, the route fires it instead: on every event of a pulse signal, or when a continuous source rises past 0.5.
Learn is generalised MIDI-learn: mapper.learn('bloom'), then move something. The first signal that moves by at least 5% of its declared range (a knob, a hand, a phone tilt), or the first pulse that fires (a key, a pad), becomes the source of a new route with the default settings. Calling learn('bloom') again cancels it. Because any pulse binds at once, stop the drum machine before you learn.
Before you call it
Create the signals and the params first, then the mapper. A smoothed route (smooth > 0) writes its param only inside update(dt), so call mapper.update(dt) every frame (createShow() does).
Parameters
Name
Type
Description
signals
Signals
The Signals registry to listen to. The mapper subscribes to every signal at once, so a route works the moment it is added.
params
Params
The Params to write into. A route writes its target as an override.
profile
String
The name this mapper saves its routes under in localStorage (openav.map.<profile>), and the world name in an exported map file. createShow() uses the world's name. optionaldefault 'default'
onChange
Function | null
Called with no arguments whenever the routes change or the import report changes (for a UI to redraw). optionaldefault null
onLearn
Function | null
Called as onLearn(route, signalName) when learn mode binds a signal. optionaldefault null
Put a map file on this mapper: the JSON text or an already parsed value, mode'replace' (default) or 'add'. Routes whose param is missing, or whose signal this show cannot produce, are skipped and listed. An unreadable file changes nothing. Returns the report (also kept in mapper.report) and saves at once.
A route writes its param as an override, and overrides are sticky: as soon as a mapped signal moves, the timeline stops driving that param until params.clearOverride(key).
A smoothed route keeps writing its last target every frame once its signal has arrived, so its param stays overridden even after clearOverride and even when the route is muted with enabled = false. Remove the route to give the param back to the timeline.
With createShow(), routes are added before the sound engine registers its sound/* params, so a route to sound/cutoff needs an explicit outMin: 100, outMax: 8000.
import { Signals, Params } from'@openav/core';
import { Mapper } from'@openav/mapping';
const signals = new Signals();
const params = new Params([{ key: 'bloom', min: 0, max: 100, def: 10 }]);
const mapper = new Mapper({ signals, params });
mapper.addRoute({ source: 'midi/cc/74', target: 'bloom', curve: 'exp' });
signals.set('midi/cc/74', 0.5);
console.log(params.resolve().bloom); // 25: 0.5 through the exp curve is 0.25 of the range 0..100
module The console prints 25: the knob is halfway, the exp curve squares it to 0.25, and that is a quarter of the way along 0..100.
js
import { Signals, Params } from'@openav/core';
import { Mapper } from'@openav/mapping';
const signals = new Signals();
const params = new Params([{ key: 'hue', min: 0, max: 360, def: 0 }]);
const mapper = new Mapper({ signals, params });
mapper.addRoute({ source: 'audio/rms', target: 'hue', inMax: 0.3, smooth: 0.5 });
signals.set('audio/rms', 0.3); // a voice at 0.3 is the whole range
console.log(params.resolve().hue); // 0: nothing moved yet
mapper.update(0.5); // half a second = one time constant
console.log(params.resolve().hue.toFixed(1)); // 227.6: about 63% of 360
module A smoothed route only moves inside update(dt): the console prints 0, then 227.6 after half a second (63% of the way to 360).
js
import { Signals, Params } from'@openav/core';
import { Mapper } from'@openav/mapping';
const signals = new Signals();
const params = new Params([{ key: 'bloom', min: 0, max: 100, def: 10 }]);
const mapper = new Mapper({ signals, params, onLearn: (route, name) => console.log('bound', name) });
signals.define('knob/a');
signals.set('knob/a', 0.5);
mapper.learn('bloom'); // arm learn for the param
signals.set('knob/a', 0.51); // moved 1%: not enough
signals.set('knob/a', 0.8); // moved 30%: bound
console.log(mapper.routesFor('bloom').map((r) => r.source), mapper.learnTarget);
module Learn waits for a real movement: the console prints bound knob/a, then [ 'knob/a' ] null (learn disarms itself once it has bound a signal).
js
import { Signals, Params } from'@openav/core';
import { Mapper } from'@openav/mapping';
function rig(profile) {
const signals = new Signals();
signals.define('audio/rms');
const params = new Params([{ key: 'bloom', min: 0, max: 100, def: 0 }]);
return { signals, params, mapper: new Mapper({ signals, params, profile }) };
}
const a = rig('garden');
a.mapper.addRoute({ source: 'audio/rms', target: 'bloom', curve: 'log', inMax: 0.3 });
const file = a.mapper.exportMap(); // the portable fileconst report = rig('garden').mapper.importMap(file); // on another machine or in another page
console.log(file.format, file.version, report.ok, report.added, report.skipped);
module A wiring travels as a file: the console prints openav.map 1 true 1 [].
Related
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
LearnWizardBinds a list of params one after another: it arms learn for the first, you turn a knob, it binds it and arms the next. Skip, back and stop steer it.
parseMap()Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
serializeMap()Builds the portable map file object from a list of routes: { format, version, world, exported, routes }.
loadMapUrl()Fetches a map file from an address (what ?map= calls) and puts it on the mapper; it never throws and never blocks the show.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
approach()One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.
Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
Use it to validate a file before you import it, or to show what is inside one. It refuses, with a plain-language error, text that is not JSON, text longer than MAP_MAX_BYTES, an object whose format is not MAP_FORMAT, a missing or newer version, no routes list, and more than 500 routes.
What it accepts is cleaned: a route without a string signal and param is dropped (and counted in dropped), unknown keys disappear, numbers must be finite, an unknown curve becomes 'linear', smooth is held to 0..60 seconds, and only invert: true inverts. A route is on unless it says enabled: false.
Parameters
Name
Type
Description
input
String | Object | Array
The JSON text of a map file, an already parsed map object, or a bare array of routes (version 0).
Returns
Object{ ok: true, version, world, routes, dropped } or { ok: false, error }. Each route is { signal, param, curve, inMin, inMax, outMin?, outMax?, smooth, invert, enabled }; dropped counts the entries that were not routes.
Examples
js
import { parseMap } from'@openav/mapping';
const good = parseMap('{"format":"openav.map","version":1,"routes":[{"signal":"audio/rms","param":"bloom","curve":"wobble"},{"signal":7}]}');
console.log(good.ok, good.routes[0].curve, good.dropped); // true linear 1
console.log(parseMap('not json')); // { ok: false, error: 'not valid JSON' }
console.log(parseMap('{"format":"other","version":1}')); // refused: not an OAV map
module The console prints true linear 1 (the unknown curve became linear, the broken route was dropped), then the two refusals with their reasons.
Related
serializeMap()Builds the portable map file object from a list of routes: { format, version, world, exported, routes }.
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
MAP_FORMATThe text "openav.map": the value of format in every map file. A file with another format is refused.
MAP_VERSIONThe newest map file version this build reads and writes: 1. A file with a higher version is refused with a message.
MAP_MAX_BYTESThe longest map text parseMap() will read: 262144, which is 256 KB. Longer text is refused without being parsed.
loadMapUrl()Fetches a map file from an address (what ?map= calls) and puts it on the mapper; it never throws and never blocks the show.
Decides whether a ?map= address may be fetched: same origin, or an absolute https URL, and nothing else.
A map file named in the page address is somebody else's data, so the rules are strict. Accepted: an address on the same origin as base (any scheme, so plain http://localhost works for a page served from there), or an absolute https: address. Refused: an empty address, one longer than 2048 characters, one that is not a URL, one carrying a user name or password, http: to another host, and data:, javascript:, file:, blob: and every other scheme. loadMapUrl() runs this check before it fetches anything.
Parameters
Name
Type
Description
value
String
The address as typed: absolute, or relative to base.
base
String
The page address that relative addresses resolve against, and that "same origin" is judged by. Usually location.href.
Returns
Object{ ok: true, url } with the resolved address, or { ok: false, error }.
Examples
js
import { parseMapUrl } from'@openav/mapping';
const base = 'https://openaudiovisual.com/examples/01-hello-particles/';
console.log(parseMapUrl('maps/stage-left.json', base)); // ok: same origin, resolved
console.log(parseMapUrl('https://example.com/my-map.json', base)); // ok: https
console.log(parseMapUrl('http://example.com/my-map.json', base)); // refused: http to another site
console.log(parseMapUrl('javascript:alert(1)', base)); // refused: not http or https
module The console prints two accepted results (the first with the full resolved address https://openaudiovisual.com/examples/01-hello-particles/maps/stage-left.json) and two refusals with their reasons.
Related
loadMapUrl()Fetches a map file from an address (what ?map= calls) and puts it on the mapper; it never throws and never blocks the show.
parseMap()Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
serializeMap(routes, { world = 'default', now = new Date() } = {})
Builds the portable map file object from a list of routes: { format, version, world, exported, routes }.
Routes in a map file are named by signal and param, never by id, so the file can travel between worlds. mapper.exportMap() calls this for you; use serializeMap() directly when you build routes by hand or want a fixed date. Missing inMin/inMax become 0 and 1, missing curve is 'linear', and the output range is 0..1 unless the route has outMin/outMax.
Parameters
Name
Type
Description
routes
Array
Routes in the mapper's own shape (what mapper.toJSON() returns: source, target, inMin, outMax …).
world
String
The world name written into the file. Importing into another world drops the output ranges. optionaldefault 'default'
now
Date | String
The time written as exported (an ISO string). Pass a fixed date to get the same file twice. optionaldefault new Date()
module The console prints the whole file as one line of JSON, starting {"format":"openav.map","version":1,"world":"garden","exported":"2026-10-19T00:00:00.000Z","routes":[{"signal":"midi/cc/74","param":"bloom","curve":"exp","range":{"in":[0,1],"out":[0,100]} ….
Related
parseMap()Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
MAP_FORMATThe text "openav.map": the value of format in every map file. A file with another format is refused.
MAP_VERSIONThe newest map file version this build reads and writes: 1. A file with a higher version is refused with a message.
A minimal event emitter: on(event, cb), emit(event, payload), off(event, cb). No wildcards, no async.
new Bus() is a tiny publish and subscribe helper for your own code: subscribe to an event name with on(), fire it with emit(), and every callback gets the one payload you pass. Callbacks run in the order they subscribed, synchronously, inside emit().
It is not the thing shows talk through. Inputs and worlds talk through Signals (named values with a range) and Params; nothing else in the framework uses a Bus, and createShow() does not create one.
Fire an event with one payload: emit(event, payload). A callback that throws is caught and logged as [bus] <event> handler:; the rest still run.
Examples
js
import { Bus } from'@openav/core';
const bus = new Bus();
const off = bus.on('hit', (strength) => console.log('hit', strength));
bus.emit('hit', 0.8); // hit 0.8
off(); // unsubscribe
bus.emit('hit', 0.9); // nothing: nobody is listening
module The console prints hit 0.8 once; after unsubscribing, the second emit is silent.
Related
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
The frame clock: calls your function every animation frame with the seconds since the last one, and keeps ticking when the tab is hidden.
new Loop((dt) => { … }).start() runs your function once per requestAnimationFrame, with dt in seconds. It is the one clock a show has: the timeline advances, mapped signals settle, the world draws and the backstage feed is sent from inside it, in the order the architecture page lists. createShow() builds and starts one for you and returns it as show.loop.
Browsers freeze requestAnimationFrame in a hidden tab, but a performance must not stop because someone switched windows. So while the loop runs, a Web Worker beats every 33 ms; whenever no animation frame has arrived for 50 ms, the loop steps itself. A hidden tab therefore ticks at roughly 15 frames a second with a normal-sized dt, and show time stays real time. Where the page forbids workers (a strict content security policy) that fallback is simply absent.
loop.fps is the frame rate, rounded, refreshed about every half second (the very first value comes from the first frame alone); it is 0 before the first frame. loop.running says whether it is going.
Before you call it
Call start(): nothing ticks until you do. It needs a browser (requestAnimationFrame, performance); in Node there is no clock to run it.
Parameters
Name
Type
Description
onFrame
Function
Called as onFrame(dt, now): dt is the seconds since the previous frame (never more than maxDt), now the time in milliseconds. An error thrown inside is caught and logged as [loop]; the clock keeps going.
maxDt
Number
The longest dt you will ever be handed, in seconds. A stall (a long garbage collection, a dragged window) is cut to this, so a world never has to cope with a one-second jump. optionaldefault 0.1
Stop the clock and shut the hidden-tab worker down. start() begins again.
Notes
Do not read the real clock inside a world: take time from dt (or from the timeline), so a show at 30 fps and one at 144 fps do the same thing.
Examples
js
import { Loop } from'@openav/core';
let elapsed = 0;
const loop = new Loop((dt) => {
elapsed += dt;
document.getElementById('app').textContent = elapsed.toFixed(1) + ' s · ' + loop.fps + ' fps';
if (elapsed > 3) loop.stop();
});
loop.start();
module A clock counts up in the page and stops itself after three seconds.
js
import { Loop } from'@openav/core';
const ctx = document.getElementById('c').getContext('2d');
let x = 0;
const loop = new Loop((dt) => {
x = (x + 120 * dt) % 400; // 120 pixels a second, whatever the frame rate
ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
ctx.fillStyle = '#7ea6ff'; ctx.fillRect(x, 130, 40, 40);
}, { maxDt: 0.05 });
loop.start();
module A square crosses the canvas at 120 pixels a second, the same speed at any frame rate because it moves by dt.
Related
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
TimelineAnswers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
A param is the way a world takes continuous control: { key: 'bloom', min: 0, max: 1, def: 0.5 }. The world never asks who set it. Every frame each param resolves to one number: the override if there is one, otherwise the base value (usually the Timeline's automation at the current time), otherwise def. resolve() does exactly that and hands the world a { key: value } object.
An override is sticky. After params.override('bloom', 0.8) (what a slider, a MIDI knob or a mapped route does) the timeline stops moving bloom until clearOverride('bloom'). That matches the stage: when a person grabs a control, the score gives way. Overrides are clamped into min..max and snapped to step when they are written.
A param with pulse: true is a trigger, not a level: nobody reads a value from it. Fire it with firePulse(key) (a pad, a route from a pulse signal, a console button) and listen with onPulse(key, cb).
In a show, createShow() builds one Params and registers every world's schema into it with add(), then returns it as show.params.
Parameters
Name
Type
Description
schema
Array
The param declarations: { key, label?, min, max, def, step?, pulse?, group? }. key is required; min, max and def give the range and the value when nothing else sets it. The array is kept, not copied. optionaldefault []
resolve(base = {}) gives { key: value } for every declared param: the override, else base[key] (for example timeline.state()), else def. A pulse param has no def, so its value is undefined.
Notes
step snaps with ordinary floating-point arithmetic: with step: 0.1, an override of 0.3 reads back as 0.30000000000000004. On a timeline a stepped param also holds each keyframe instead of interpolating.
resolve() does not clamp the base value, only overrides are clamped when written. Keep your automation inside min..max.
The fields a world may declare (label, options, surface, group …) are listed in Writing a world.
Examples
js
import { Params } from'@openav/core';
const params = new Params([{ key: 'bloom', min: 0, max: 1, def: 0.5 }]);
console.log(params.resolve().bloom); // 0.5 (no automation, no override: def)
params.override('bloom', 0.8); // what a slider or a mapped knob does
console.log(params.resolve({ bloom: 0.1 }).bloom); // 0.8 (the override beats the base)
params.clearOverride('bloom');
console.log(params.resolve({ bloom: 0.1 }).bloom); // 0.1 (the base is back)
module The console prints 0.5, 0.8, 0.1: def, then an override that beats the timeline, then the timeline again.
module A stepped param snaps (the console shows 2) and a pulse param fires its listener (fired burst).
js
import { Params } from'@openav/core';
const params = new Params([{ key: 'bloom', min: 0, max: 1, def: 0.5 }]);
console.log(params.get('bloom')); // the declaration, not a value
params.override('bloom', 5);
console.log(params.resolve().bloom); // 1 (clamped to max)
params.override('typo', 0.3); // unknown key: ignored
console.log(params.isOverridden('typo')); // false
module The gotchas: get() returns the declaration { key, min, max, def }, an override is clamped to the range (1), and an unknown key is silently ignored (false).
Related
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
TimelineAnswers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
StageRuns one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
new Signals() makes an empty registry. In a show you rarely build it yourself: createShow() creates one and returns it as show.signals. A signal is a name plus a value. Input packages publish with set() and pulse(); everything else reads with get() and norm(), or subscribes with on() and onAny().
There are two kinds. A continuous signal (the default) always holds a current value, usually between 0 and 1: a knob, the loudness of the room, the height of a wrist. A pulse signal (kind: 'pulse') is an event: its value is the payload of the last firing and it is null until the first one: a key struck, a kick heard, a pinch started. Read a level whenever you like; subscribe to an event with on().
Names are paths with the source first: midi/cc/74, audio/rms, pose/hand/right/y, breath/pressure. Declare a signal with its range before you publish it: signals.define('breath/pressure', { min: 0, max: 1 }). set() on a name nobody declared defines it on the spot as a continuous signal from 0 to 1, so declare first whenever your numbers are in degrees, MIDI notes or any other range.
A knob, a raised hand and a chord are equal citizens here. Whoever reads a signal (the Mapper, a world that listens for events, the monitor) does not know and does not care what produced it.
Declare a signal and its range: define(name, { kind, min, max, unit, description, source }). Safe to call again: later declarations merge into earlier ones. A continuous signal starts at min, a pulse at null. Returns the registry.
Publish a value: set(name, value). Subscribers run every time, even when the value did not change. An undeclared name is defined on the spot as continuous 0..1.
Fire an event: pulse(name, payload = 1). It is set() under another name; declare the signal with kind: 'pulse' first, or it is registered as continuous.
Subscribe to every signal: cb(name, value, meta) runs for each one. The mapper's learn mode and the monitor use it. Returns a function that unsubscribes.
Every signal as [{ name, value, at, kind, min, max, unit, description, source }]; at is the time of the last set() (performance.now(), 0 if never). What the monitor and meters read.
Notes
signals.meta is a Map of name → declaration and signals.values a Map of name → last value. Read them if you must; write through define() and set() so subscribers hear about it.
A subscriber that throws is caught and logged as [signals] <name>; the other subscribers still run and set() does not throw.
module A level and an event side by side: the console shows 25 0.75, then sigh { depth: 0.8 }.
js
import { Signals } from'@openav/core';
const signals = new Signals();
signals.set('wind/speed', 12); // never declared, so it is 0..1
console.log(signals.norm('wind/speed')); // 1 (12 is far past the range)
signals.define('wind/speed', { min: 0, max: 20 });
console.log(signals.norm('wind/speed')); // 0.6
module The gotcha: an undeclared signal is 0..1, so 12 reads as 1. Declare the range and the same value reads 0.6 (the console shows 1, then 0.6).
Related
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
LoopThe frame clock: calls your function every animation frame with the seconds since the last one, and keeps ticking when the tab is hidden.
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
BusA minimal event emitter: on(event, cb), emit(event, payload), off(event, cb). No wildcards, no async.
ChordDetectorGroups notes struck close together into one gesture and analyses it: chord type, consonance, triad and cluster flags, published as chord/* signals.
Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
Every example is a world plus one createShow() call. It builds the Signals registry, the Params, the Stage with your world, the Timeline, the Mapper (with your routes), the inputs and outputs you declare in modules, the director's desk (mountConsole()), the backstage feed (MonitorFeed) and the frame Loop, starts the loop, and hands you every part. Nothing is hidden: show.signals, show.params, show.timeline, show.mapper, show.stage and the rest are the very objects the frame loop runs. When the framework grows a module, every show can have it just by declaring it.
The modules keys: midi (on by default, false turns it off, { controllers: true } or { controllers: { profile, profiles, open } } docks an on-screen MIDI controller), keys (the on-screen piano, QWERTY and the simulated performer: on by default, false turns them off, { base, octaves, sim, capture } tunes them), drums (true or { engine, autoEnableEngine }), audio (any truthy value: the microphone analyser, started from the 🎤 button), chord (true or { window }: a ChordDetector wired to every played note), hands (true: HandTracker), pose (true: PoseTracker), leap (true or { url, simulate, fingers, box }: LeapInput; ?leap=sim or ?leap=ws://… in the address overrides), remote (true or { room, … }: phones as controllers) and sound (true, an instrument id such as 'piano', or { instrument, remember, picker, engine }: instruments in the page, loaded only when declared). Parts that fetch things load lazily, so a show that does not declare them pays nothing.
The page shell is fixed once so every show is the same. Without mount, the page body becomes a two-column grid: the stage, and a 340 px side panel holding the piano and the console (an upright phone gets the stage on top and the panel below: see isStacked()). Every frame, in this order:
text
keys, drums, audio update on-screen performers and the microphone
onFrame(dt, show) your hook
timeline.advance(dt) time moves
director.update(...) with a score: segment modules enter, update, exit
mapper.update(dt) smoothed routes settle
stage.frame(dt, state) params resolve, world updates and renders
sound, remote sound params reach the synth, values echo to phones
console.render(state) the desk reflects reality
monitor.frame(snapshot) the backstage hears about it
Before you call it
await it: it is async because it waits for the world's init() (and for lazily loaded modules). It needs a page (document, location, localStorage), and an import map or bundler that resolves every @openav/* package it imports (copy the one from examples/01-hello-particles/index.html). Browsers start audio only after a click, so sound and the microphone start from the console's buttons.
Parameters
Name
Type
Description
world
Object | null
The world to perform: { name, params, init, update, render, dispose } (see Stage). It is activated first, and createShow waits for its init(). optionaldefault null
worlds
Array
More worlds to register, after world. All of their params are registered up front; switch with show.stage.activate(name). optionaldefault []
timeline
Object
The options of the Timeline: { total, automation, scenes, start, loop, onCue }. With a score, its scenes and total win. optionaldefault { total: 120 }
routes
Array
Routes for the Mapper. If a saved profile exists it is loaded, and any declared route whose source → target pair is missing from it is still added. optionaldefault []
modules
Object
The inputs and outputs to switch on, by key (see the description): midi, keys, drums, audio, chord, hands, pose, leap, remote, sound. optionaldefault {}
artwork
Object | null
{ title, artist, year, note }: a credit line at the bottom right of the stage and a <meta name="artwork"> tag in the head. optionaldefault null
hint
String
A line of HTML at the bottom left of the stage, for example 'play the piano · Space plays the timeline'. It is inserted as HTML, so never put untrusted text in it. optionaldefault ''
profile
String | false | null
The name the mapper saves its routes under in localStorage (default: the first world's name, else 'show'). false ignores the saved routes when the show starts, so the routes come from your code on every load. optionaldefault null
map
String | false | null
A map file to load on start: a same-origin or https address (see loadMapUrl()). Left out, ?map=<url> in the page address is used; false ignores ?map=. optionaldefault null
onFrame
Function | null
Called as onFrame(dt, show) every frame, after the on-screen performers and the microphone update and before the timeline advances. The place for assembly-level logic such as envelopes, or osc.flush(). optionaldefault null
mount
Object | null
{ stage, side }: elements or selectors to render into instead of the generated full-page layout. optionaldefault null
telemetry
Boolean
Allow one anonymous oav_show_start hit per page load. false sends nothing. optionaldefault true
score
Object | null
{ cuts, cut, modules, api, onStatus, onCue, maxStep, midi }: the show's structure as segment lengths, performed by segment modules (see Score and Director). ?cut=<id> in the page address picks the version. optionaldefault null
Returns
Promise<Object> The show: { signals, params, stage, timeline, mapper, midi, controllers, midiPanel, keys, drums, sound, audio, hands, pose, leap, chord, remote, score, director, scoreMidi, console, app, loop }. A part you did not ask for is null. With a score there is also show.show, the rehearsal controls. The same object is window.openav.
Notes
The mapper saves its routes under the profile name every 3 seconds, so a learned or edited route survives a reload and wins over a changed declaration in your code (the same source → target pair keeps its saved curve and smoothing). profile: false makes the routes come from code on every load, but the mapper still saves under the world's name from then on. To start clean, localStorage.removeItem('openav.map.<world name>').
Routes are added before the sound engine registers its sound/* params, so a declared route to sound/cutoff needs an explicit outMin: 100, outMax: 8000.
show.params.resolve() without an argument gives the def of every param. To read what the world sees this frame, pass the timeline's base state: show.params.resolve(show.timeline.state()).
Unless telemetry: false, one anonymous oav_show_start hit goes to Google Analytics per page load. Per the code it is silent on localhost, LAN addresses, file:// and openaudiovisual.com itself, and it honours Do Not Track and Global Privacy Control. Nothing played, no path, no cookie and no stable id is sent.
With a score, window.openav.show has table(), goto('dusk'), goto('dusk.glow'), goto(2), seek(T), cue(name), status(), next(), prev() and cuts, for rehearsal from the devtools console.
module A blue dot on the stage grows slowly while the timeline plays, with the desk (transport, layers, the size slider) on the right. The console prints 23 true: twenty-three parts, and the timeline is playing.
js
import { createShow } from'@openav/show';
const world = {
name: 'glow',
params: [{ key: 'size', min: 0, max: 1, def: 0.3 }, { key: 'burst', pulse: true }],
init() {},
update(dt, state) { this.last = state; },
};
const show = await createShow({
world,
routes: [{ source: 'midi/cc/1', target: 'size' }, { source: 'pad/1', target: 'burst' }],
modules: { midi: false, keys: false },
profile: false, // routes come from this code on every load
telemetry: false,
});
show.params.onPulse('burst', () => console.log('burst!'));
show.signals.set('midi/cc/1', 0.9); // what a knob would do
console.log(show.params.isOverridden('size'), show.params.resolve(show.timeline.state()).size); // true 0.9
show.signals.set('pad/1', 0);
show.signals.set('pad/1', 1); // a continuous source rising past 0.5 fires a pulse param
setTimeout(() => console.log('the world heard', world.last.size), 200);
module A signal drives a param through a route: the console prints true 0.9, then burst! as the pad crosses 0.5, then the world heard 0.9.
module With a score the timeline takes its length from it: the console prints 20 20, then dusk enters after the jump, and dusk 10.
js
import { createShow } from'@openav/show';
const show = await createShow({
world, // your world
timeline: { total: 120, automation, scenes },
routes: [{ source: 'audio/rms', target: 'energy', inMax: 0.3, smooth: 0.1 }],
modules: { keys: { base: 48 }, chord: true, audio: true, sound: 'piano' },
artwork: { title: 'Untitled', artist: 'You', year: 2026 },
hint: 'play the piano · Space plays the timeline',
});
show.signals.on('chord/event', (a) => console.log(a.chordType));
static The whole set: an on-screen piano, chord analysis, the microphone and a grand piano that sounds after a click on the console's sound button. Play a chord and the console prints its type (major).
To be confirmed
The modules that need hardware, a permission or a download (microphone, camera, Leap, phones, the Tone.js sound engine, a MIDI device) were not run here; they are described from the code and the package docs. The examples switch midi and keys off so that they run anywhere.
Related
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
LoopThe frame clock: calls your function every animation frame with the seconds since the last one, and keeps ticking when the tab is hidden.
StageRuns one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
TimelineAnswers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
mountConsole()Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.
MonitorFeedStreams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.
ScoreA show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.
DirectorRuns a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
ChordDetectorGroups notes struck close together into one gesture and analyses it: chord type, consonance, triad and cluster flags, published as chord/* signals.
LeapInputReads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
isStacked()Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
The shell of createShow() has one layout rule: wide windows get the stage and a 340 px side panel next to each other, and an upright phone or small tablet gets the stage on top and the panel below it, scrolling. The rule is the media query STACK_QUERY, and isStacked() is the very same decision as a plain function, so a test (or your own layout code) can use it without a browser. A phone on its side is never taller than wide, so it stays in the wide layout.
Parameters
Name
Type
Description
width
Number
The window width in pixels.
height
Number
The window height in pixels.
Returns
Booleantrue when width <= 820 and height > width.
Examples
js
import { isStacked, stageHeight } from'@openav/show/layout';
console.log(isStacked(390, 844)); // true: a phone held upright
console.log(isStacked(844, 390)); // false: the same phone on its side
console.log(isStacked(820, 1180)); // true: a small tablet held upright
console.log(isStacked(1024, 1366)); // false: wider than 820, even though it is upright
console.log(stageHeight(844)); // 490: the stage keeps 58% of the window height
module The console prints true, false, true, false, then 490.
Related
stageHeight()The stage height in pixels for a window of this height when the layout is stacked: 58% of it, rounded.
STACK_QUERYThe media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).
STACK_MAX_WIDTHThe widest window that can get the stacked layout: 820 pixels (a small tablet held upright).
STAGE_SHAREHow much of the window height the stage keeps when the layout is stacked: 58 (percent).
SHELL_CSSThe CSS text createShow() puts in the page: the two-column grid, the 340 px side panel, the hint and credit styles, and the stacked layout.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
The CSS text createShow() puts in the page: the two-column grid, the 340 px side panel, the hint and credit styles, and the stacked layout.
A string. createShow() adds it once as <style id="openav-shell-css"> (unless one with that id is already there). It sets the page body to a grid of the stage and a 340 px side panel, with the classes .oav-stage, .oav-side, .oav-keys-slot, .oav-desk, .oav-hint and .oav-credit, and the stacked layout from STACK_QUERY. Read it to see what the shell does to your page, or add a <style id="openav-shell-css"> of your own before calling createShow() to replace it entirely.
Examples
js
import { SHELL_CSS } from'@openav/show/layout';
console.log(SHELL_CSS.includes('.oav-stage'), SHELL_CSS.includes('340px'), SHELL_CSS.length > 500); // true true true
console.log(SHELL_CSS.split('\n').slice(1, 4).join('\n')); // the first rules: body is a grid
module The console prints true true true, then the first lines of the stylesheet (the body grid).
Related
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
STACK_QUERYThe media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).
isStacked()Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
The widest window that can get the stacked layout: 820 pixels (a small tablet held upright).
Used inside STACK_QUERY and isStacked(), so the CSS rule and the function cannot drift apart. Above 820 pixels the stage and the 340 px panel always sit side by side.
Related
STACK_QUERYThe media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).
isStacked()Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
STAGE_SHAREHow much of the window height the stage keeps when the layout is stacked: 58 (percent).
STACK_QUERY = "(max-width: 820px) and (orientation: portrait)"
The media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).
It is a string you can hand to matchMedia() to ask the browser the same question the shell's CSS asks: matchMedia(STACK_QUERY).matches. isStacked(width, height) is the same decision from numbers.
Examples
js
import { STACK_QUERY } from'@openav/show/layout';
const query = matchMedia(STACK_QUERY);
console.log(STACK_QUERY, query.matches); // the window this runs in is not an upright phone: false
query.addEventListener('change', (e) => console.log('stacked now?', e.matches));
module The console prints the query and whether this window matches it (usually false on a desktop); the last line logs again whenever the window is resized across the rule.
Related
isStacked()Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
STACK_MAX_WIDTHThe widest window that can get the stacked layout: 820 pixels (a small tablet held upright).
STAGE_SHAREHow much of the window height the stage keeps when the layout is stacked: 58 (percent).
SHELL_CSSThe CSS text createShow() puts in the page: the two-column grid, the 340 px side panel, the hint and credit styles, and the stacked layout.
How much of the window height the stage keeps when the layout is stacked: 58 (percent).
The stage must stay the main thing on a phone, so it keeps 58% of the height and the console below it scrolls. stageHeight() turns this number into pixels.
Related
stageHeight()The stage height in pixels for a window of this height when the layout is stacked: 58% of it, rounded.
isStacked()Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
STACK_QUERYThe media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).
The stage height in pixels for a window of this height when the layout is stacked: 58% of it, rounded.
When the shell stacks, the stage must stay the main thing, so it takes STAGE_SHARE percent of the window height (the CSS uses 58vh, and 58dvh where the browser has it) and the panel gets the rest. This is the same number as a function, for tests and your own layout code.
module The console prints 667 387, 844 490 and 1180 684.
Related
isStacked()Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
STAGE_SHAREHow much of the window height the stage keeps when the layout is stacked: 58 (percent).
STACK_QUERYThe media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
Makes a 2D canvas that fills its container and keeps its size in step with it, at up to 2 times the device pixel ratio.
A convenience for worlds that draw with the 2D canvas API; a world may use it or ignore it. The canvas is display: block and as big as the container. fit() sets its pixel size to the container size times the device pixel ratio (never more than 2) and scales the context so you keep drawing in CSS pixels. Call fit() at the top of render(): it only does work when the size changed, and it returns { w, h } to draw with. A ResizeObserver also calls it when the container is resized.
Call dispose() in the world's dispose(): it stops the observer and removes the canvas.
Parameters
Name
Type
Description
container
HTMLElement
The element to put the canvas in (a world's container, usually).
alpha
Boolean
Whether the canvas may be transparent. The default false is faster and gives an opaque canvas. optionaldefault false
Returns
Object{ canvas, ctx, fit, dispose }: the <canvas>, its 2D context, fit() and dispose(). fit() returns { w, h }, the container size in CSS pixels.
Notes
If the container has no height (an empty <div> in normal flow is 0 pixels tall), the canvas is 0 × 0 and nothing shows. Give the container a size, or use it inside a Stage, whose layer fills the stage.
module A blue square slides left and right across a canvas that fills a 200-pixel-high strip of the page, and stays sharp when the window is resized.
Related
StageRuns one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
new Stage({ container, params, signals, io = {} })
Runs one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
A world is any system with this shape, and every method is optional:
js
{
name: 'garden', // unique: the stage registers worlds by name
params: [{ key: 'growth', min: 0, max: 1, def: 0.5 }], // what it can be performed with
init({ container, signals, params }) {}, // when it becomes active (may be async)
update(dt, state, io) {}, // every frame: state = the resolved params
render() {}, // every frame, right after update
dispose() {}, // when another world is activated
}
The stage owns the lifecycle and the param aggregation and deliberately not a renderer: draw with a 2D canvas, p5, three.js, SVG, the DOM, whatever you like. The shell never knows the work: switching works means swapping the world and its data, and the shell stays.
A world takes continuous control from state (the params) and may subscribe to signals for discrete events (a note struck, a chord resolved). If it reads signals.get('midi/cc/74') inside update(), it works with your controller today, but the timeline, a hand or another controller can no longer drive it. In a show, createShow() builds the stage from its world and worlds options and returns it as show.stage.
Before you call it
Register your worlds, then await stage.activate(name) once; until a world is active, frame() returns null and does nothing. Then call stage.frame(dt, baseState) every frame (a show's loop does).
Parameters
Name
Type
Description
container
HTMLElement
The element the worlds draw into. It becomes position: relative if it was static. The stage adds its own layer inside it, so overlays you put in the container (hints, start buttons) are not wiped when a world is switched. optional
params
Params
The Params the worlds' schemas are absorbed into, and that frame() resolves. optional
signals
Signals
The Signals registry handed to every world, so it can subscribe to events. optional
io
Object
Extra output handles for the worlds' update(dt, state, io), such as { midi, osc }. signals is always there. optionaldefault {}
Switch to the world with this name. Async: it disposes the previous world (an error in its dispose() is logged, not thrown), empties the stage layer, then awaits the new world's init({ container, signals, params }). Does nothing if that world is already active; rejects with "unknown world" for a name that was never registered.
Per frame: frame(dt, baseState) resolves params.resolve(baseState) (override, else the base, else def), calls the active world's update(dt, state, io) and then render(), and returns the resolved state, or null with no active world. An exception in either call is caught and logged as [stage] update: or [stage] render:; the show keeps running.
Notes
Check the name before you activate: activate('nope') disposes the current world first and only then rejects, and the stage keeps pointing at the disposed world (active and activeName do not change), so the next frame() still calls its update() and render().
Fields to read: active (the active world object), activeName, worlds (a Map of name → world), container, layer (the element worlds are placed in) and io ({ signals, ...io }, what update receives; add handles later with stage.io.osc = osc).
An exception in init() is not caught: activate() rejects. Exceptions in update() and render() are caught. A world must release what it takes (GPU objects, DOM, subscriptions) in dispose(), because worlds get switched live.
Examples
js
import { Signals, Params } from'@openav/core';
import { Stage } from'@openav/stage';
const container = document.getElementById('app');
container.style.cssText = 'width: 400px; height: 120px;'; // give the stage some roomconst stage = new Stage({ container, params: new Params(), signals: new Signals() });
stage.register({
name: 'label',
params: [{ key: 'size', min: 10, max: 60, def: 20 }],
init({ container }) { this.el = document.createElement('div'); container.appendChild(this.el); },
update(dt, state) { this.size = state.size; },
render() { this.el.style.cssText = 'color: #7ea6ff; font: ' + this.size + 'px monospace'; this.el.textContent = 'size ' + this.size; },
dispose() { this.el.remove(); },
});
await stage.activate('label'); // init runs now
console.log(stage.frame(0.016, { size: 40 })); // { size: 40 }: the base state, since nothing overrides it
console.log(stage.frame(0.016, {})); // { size: 20 }: no base either, so def
module The console prints { size: 40 }, then { size: 20 }, and the page shows the text size 20 in blue: the world drew itself from the resolved params.
js
import { Signals, Params } from'@openav/core';
import { Stage } from'@openav/stage';
const signals = new Signals();
const stage = new Stage({ container: document.getElementById('app'), params: new Params(), signals });
const world = (name) => ({
name, params: [{ key: 'calm', min: 0, max: 1, def: 0.5 }],
init() { console.log(name, 'init'); },
dispose() { console.log(name, 'dispose'); },
});
stage.register(world('calm')).register(world('storm'));
await stage.activate('calm'); // calm initawait stage.activate('storm'); // calm dispose, then storm initawait stage.activate('storm'); // already active: nothing happens
console.log(stage.activeName, stage.params.schema.length); // storm 1: both worlds share the one 'calm' paramawait stage.activate('nope').catch((error) => console.log(error.message)); // unknown world: storm is disposed first, then this rejects
module The console prints calm init, calm dispose, storm init, then storm 1 (the shared param is registered once), then storm dispose and the error message for an unknown world: the failed switch has already disposed the world that was playing.
Related
createCanvas()Makes a 2D canvas that fills its container and keeps its size in step with it, at up to 2 times the device pixel ratio.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
webtoeWorld()Turns a WebToe node patch into a world: the patch runs in an iframe and every changed param is posted into it.
TimelineAnswers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
webtoeWorld({ name = 'webtoe', app = 'https://webtoe.openaudiovisual.com/', project = null, params = [], extra = null, } = {})
Turns a WebToe node patch into a world: the patch runs in an iframe and every changed param is posted into it.
WebToe is the sister project: a web-native, node-based dataflow engine that speaks TouchDesigner. webtoeWorld() embeds its app in an iframe that fills the stage (with allow="camera; microphone"), and each frame it posts the resolved params into the iframe as { type: 'webtoe:ext', values }. Inside the patch, any parameter written as ext('name', fallback) follows the value called name, and uses fallback until one arrives.
The contract stays pure: the world reads params and never inputs. Which knob, hand or timeline drives speed is the mapper's business, as for any world. To send less, the world posts only when at least one value differs from the last post.
Before you call it
The page must be able to load the WebToe app in an iframe (the network, by default https://webtoe.openaudiovisual.com/), and the patch file must be reachable by the app. The world does its work when it is activated, so use it with createShow({ world }), or stage.register() and stage.activate().
Parameters
Name
Type
Description
name
String
The world's name: what Stage registers it under and what you pass to stage.activate(). Give each patch its own. optionaldefault 'webtoe'
app
String
The WebToe app to embed in the iframe. Change it to run your own copy of the app. optionaldefault 'https://webtoe.openaudiovisual.com/'
project
String | null
The address of a .webtoe.json patch file. It is handed to the app as ?project=<address>, and the app fetches it, so the file's server must allow the app's origin (CORS). Make the address absolute, for example new URL('./garden.webtoe.json', location.href).href. optionaldefault null
params
Array
The param schema of the world: [{ key, min, max, def }, …]. The keys are the names the patch reads with ext('name', fallback). optionaldefault []
extra
Function | null
extra(state, out), called every frame: add derived values to out (a copy of the state) that the patch can read too, such as out.hueRad = state.hue * Math.PI / 180. optionaldefault null
Returns
Object A world { name, params, init, update, render, dispose } to pass to createShow({ world }) or stage.register().
Notes
The message goes to the iframe with the target origin *, so any page the iframe happens to be showing receives the values. If you point app at a site you do not trust, it can read your params.
module An iframe fills the container and, once it has loaded, shows the message the world posted: {"type":"webtoe:ext","values":{"speed":0.8,"hue":180,"hueRad":3.14}}.
js
import { createShow } from'@openav/show';
import { webtoeWorld } from'@openav/world-webtoe';
const world = webtoeWorld({
name: 'garden-patch',
project: new URL('./garden.webtoe.json', location.href).href, // the app fetches it: CORS must allow it
params: [
{ key: 'speed', min: 0, max: 1, def: 0.5 },
{ key: 'hue', min: 0, max: 360, def: 205 },
],
});
const show = await createShow({ world, routes: [{ source: 'midi/cc/1', target: 'speed' }] });
static The patch from garden.webtoe.json fills the stage, and turning MIDI knob 1 changes its speed through the mapper. Every ext('speed', …) in the patch follows.
To be confirmed
Not run against the real WebToe app here (the examples use a stand-in page, with no network). Because a value is only posted when it changed, a patch that finishes loading after the first frame may not receive values that never move again until one of them changes; whether that happens with the live app was not checked.
Related
StageRuns one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
The "latest phone" name of a phone signal: aliasOf('phone/ab12/tilt/x') is 'phone/any/tilt/x'. null for anything else.
A route written in a World file cannot know the id of a stranger's phone. The alias fixes that: every phone signal is also filed under phone/any/…, which always holds the value from whichever phone sent last, so { source: 'phone/any/tilt/x', target: 'hue' } can be written before anyone has joined. bindSignals() and localSensors() both file the alias for you.
Parameters
Name
Type
Description
name
String
A signal name. Only phone/<id>/… has an alias.
alias
String
The word that replaces the id. Leave it out for any. optionaldefault 'any'
Returns
String | null The alias name, or null when name is not a phone signal or already uses the alias.
module Prints phone/any/tilt/x, then null twice (already an alias; not a phone signal), then phone/latest/knock.
Related
bindSignals()Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
localSensors()For a show page that runs on the phone itself: the phone's own sensors go straight into a Signals as phone/local/…, with a phone/any/… copy.
fileSignal()Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
Attaches the room relay to a Node http.Server: WebSocket clients that connect to path are put in rooms and routed by role. Returns { hub, close() }.
This is the server half, in Node, with no dependencies (the WebSocket codec is in the package). node serve.js already does it for you at /relay. To use it in your own server: attachRelay(server) before or after server.listen(). Clients connect to ws://host/relay?role=controller|runner|monitor&room=<name>&id=<device>: a bad role becomes controller, a bad room name becomes default, and the id is cut to 64 characters.
Who may talk to whom: a controller's signal and batch go to the runner and the monitors; a runner's go to the monitors only; a runner's feedback and config go to the controllers and monitors; a monitor says nothing. A config is kept per key and replayed to controllers that join later, and the room forgets it when its runner leaves. Rooms never hear each other. A message over 256 KB is ignored and a frame over 1 MB closes the connection.
There is no authentication: anyone who can reach the server and knows a room name can join as a controller. Use a closed network, or a room name nobody will guess.
Before you call it
A Node program, imported from @openav/relay/server (a different entry point from @openav/relay, which is the browser side). Standalone: node packages/relay/server.js [port] (default 7458; /health answers with the room names; RELAY_HOST=127.0.0.1 binds to localhost behind a reverse proxy).
Parameters
Name
Type
Description
server
http.Server
A Node http.Server (or anything with the same upgrade event, such as an https server). The relay listens for its upgrade events.
path
String
The URL path clients connect to. Any other path is refused. optionaldefault '/relay'
log
Function
(line) => {}, called with a line such as + controller ab12 @main when a client joins and - controller ab12 @main when it leaves. The default says nothing. optionaldefault () => {}
heartbeatMs
Number
How often to ping every client, in milliseconds. A client that has not answered by the next ping is dropped, so a dead phone goes within one to two periods. optionaldefault 20000
Returns
{ hub, close }hub is the RoomHub (hub.rooms is a Map of the rooms; hub.counts(room) gives { controllers, runners, monitors }). close() drops every connected client; it does not close your server.
static A relay on port 7458 in about five lines. Open a show page with ?relay=ws://localhost:7458/relay and the terminal logs a line such as [relay] + runner show-k3x9 @default when it connects.
Related
RelayClientThe browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
bindSignals()Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
mountRemoteHost()The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
bindSignals(client, signals, { alias = 'any', meta = signalMeta } = {})
Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
Call it once on the show page, after making the runner's client: bindSignals(show, signals). It sets client.onSignal so that each signal that arrives goes through fileSignal(), and a phone signal is filed twice, under its own name (phone/ab12/tilt/x) and under the alias (phone/any/tilt/x). Routes written at author time use the alias.
A callback that was already in client.onSignal keeps running after the filing, so you can still watch the raw messages.
Parameters
Name
Type
Description
client
RelayClient
The runner's client (any object with an onSignal field works).
signals
Signals
The registry to fill (from @openav/core).
alias
String | null
The word used in the copy, phone/<alias>/…. null or an empty string turns the copy off. optionaldefault 'any'
meta
Function
Replace signalMeta with your own declarations, as in fileSignal(). optionaldefault signalMeta
Returns
Function Call it to unbind: client.onSignal goes back to what it was before.
Examples
js
import { bindSignals } from'@openav/relay';
import { Signals } from'@openav/core';
const signals = new Signals();
const client = {}; // anything with an onSignal field; a RelayClient in real useconst unbind = bindSignals(client, signals);
// what the relay calls when a phone's message arrives
client.onSignal('phone/ab12/tilt/x', 0.4, {});
client.onSignal('phone/ab12/knock', { strength: 1 }, { pulse: true });
unbind(); // client.onSignal is back to what it was
document.getElementById('app').innerHTML = [
'phone/any/tilt/x = ' + signals.get('phone/any/tilt/x'),
'knock kind: ' + signals.meta.get('phone/any/knock').kind,
'unbound: ' + (client.onSignal === undefined),
].join('<br>');
module Prints the tilt under the alias (0.4), the knock declared as a pulse, and "true" once unbind() has put onSignal back.
js
import { RelayClient, bindSignals } from'@openav/relay';
import { Signals } from'@openav/core';
const signals = new Signals();
const show = new RelayClient({ role: 'runner', room: 'main' });
const seen = [];
show.onSignal = (name) => seen.push(name); // our own listener first
bindSignals(show, signals); // it still runs after the filing
show.onSignal('phone/ab12/tilt/y', 0.2, {});
document.getElementById('app').textContent = seen.join(', ') + ' | ' + signals.get('phone/any/tilt/y');
module A listener that was there before bindSignals() still hears every message: prints "phone/ab12/tilt/y | 0.2".
Related
RelayClientThe browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
fileSignal()Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
aliasOf()The "latest phone" name of a phone signal: aliasOf('phone/ab12/tilt/x') is 'phone/any/tilt/x'. null for anything else.
signalMeta()What the relay knows about a signal name: its range and unit, or that it is a pulse. Returns null for names it does not know.
mountRemoteHost()The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
The short random id of this browser, kept in localStorage, that names this phone in its signals, as in phone/ab12/tilt/x.
The first call makes an id and stores it under key; later calls read it back. RelayClient uses it as its default id, so a phone keeps the same name across reloads and its signals stay phone/<same id>/….
If localStorage is blocked (some private windows), nothing can be stored and every call makes a new id.
Parameters
Name
Type
Description
key
String
The localStorage key. The default is the one the lab uses, so /control and /remote on one phone count as one device. optionaldefault 'lab.deviceId'
Returns
String A short id of about five letters and digits, such as k3x9a. The same one on every call, in the same browser.
Examples
js
import { deviceId } from'@openav/relay';
const first = deviceId();
const again = deviceId();
document.getElementById('app').textContent =
`id ${first}, the same on the next call: ${first === again}`;
module Prints a short id and "true": the second call reads back what the first one stored.
Related
RelayClientThe browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
PhoneSensorsTurns a phone's tilt, acceleration, rotation, knocks and front-camera light into named signals, in screen axes, and hands them to the io you give it.
Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
This is the receiving half of the wire format, and every receiver files a message the same way through it: bindSignals() for a show that has a relay, localSensors() for a show on a phone, and a host that runs its own socket. In order: if the registry has never heard the name, declare it with what signalMeta() says (a pulse is never mistaken for a knob, and meters get their range); fill in the missing velocity spelling with normalizeValue(); then signals.pulse(name, value) for a pulse or signals.set(name, value) for the rest.
Parameters
Name
Type
Description
signals
Signals
The registry to file it in (from @openav/core).
name
String
The signal name. It must be a non-empty string of at most 128 characters, or nothing is filed.
value
any
The payload.
pulse
Boolean
True when the message was sent as an event. A name that signalMeta() knows to be a pulse (or was declared as one) is a pulse either way. optionaldefault false
meta
Function
Replace signalMeta with your own (name, { pulse }) => metadata | null, to declare names the relay does not know. optionaldefault signalMeta
source
String
A label stored in the declared metadata (signals.meta.get(name).source), such as 'relay' or 'local'. optionaldefault ''
Returns
Booleantrue when it was filed, false for a bad name.
module Prints a velocity of 0.79 (100 / 127) that the sender never wrote, the kind "pulse", the range -1..1 for tilt, and "false" for the empty name.
Related
bindSignals()Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
signalMeta()What the relay knows about a signal name: its range and unit, or that it is a pulse. Returns null for names it does not know.
normalizeValue()Fills in the missing velocity spelling of a note message: vel (0..1) or velocity (1..127). Every other message passes through unchanged.
localSensors()For a show page that runs on the phone itself: the phone's own sensors go straight into a Signals as phone/local/…, with a phone/any/… copy.
aliasOf()The "latest phone" name of a phone signal: aliasOf('phone/ab12/tilt/x') is 'phone/any/tilt/x'. null for anything else.
Fills in the missing velocity spelling of a note message: vel (0..1) or velocity (1..127). Every other message passes through unchanged.
@openav/midi publishes vel as 0..1, a hardware keyboard talks in velocity 1..127, and senders (an old phone page, a TouchOSC bridge, a sequencer) give one or the other. A sketch is written against one spelling, so the receiving end fills in the other: velocity becomes vel = velocity / 127, and vel becomes velocity = Math.round(vel * 127).
You rarely call it yourself: fileSignal() does it for every message it files.
Parameters
Name
Type
Description
name
String
The signal name. Only midi/note/on, midi/note/off and the same with a device in the middle (midi/<device>/note/on) are touched.
value
any
The payload. Only an object with vel or velocity is changed (a copy, never the original).
Returns
any The payload with both vel and velocity. If both were there already, the same object comes back untouched.
module The first two gain the spelling they lacked (vel 1 and velocity 64); the third works with a device in the name; the fourth is not a note and stays as it is.
Related
fileSignal()Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
bindSignals()Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
new RelayClient({ role = 'controller', room = 'default', id = deviceId(), url = null, batchHz = 30, resendMs = 1000, onStatus, onSignal, onFeedback, onConfig } = {})
The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
Make one client per page, give it a role and a room, and call connect(). A phone calls phone.set('phone/me/tilt/x', 0.3) for values that change continuously and phone.send('midi/note/on', { note: 60 }, true) for events. The show passes its client to bindSignals(), and every phone signal lands in its Signals. The relay itself is attachRelay() on a Node server.
Continuous values are cheap on purpose: set() only stores the value (and does nothing if it is the same as the last), then every 1 / batchHz seconds the changed ones leave in one batch, and every resendMs all of them leave again. Events are the opposite: send() leaves at once and is never batched or repeated, because a note-on delivered twice is a bug.
It looks after the connection itself. When the socket closes it reconnects after 0.5 s, then 1 s, 2 s, 4 s and every 5 s, until you call close(). A ping goes out every 2 s and the answer becomes latency in milliseconds. status is 'connecting', 'open' or 'closed', and peers is how many controllers, runners and monitors the room has.
Before you call it
A relay server for it to reach, such as node serve.js (it hosts one at /relay) or node packages/relay/server.js. Until the socket is open, set() keeps the latest values and sends them when it opens, but send(), feedback() and config() messages are dropped, not queued.
Parameters
Name
Type
Description
role
'controller' | 'runner' | 'monitor'
Who this page is. A controller (a phone) sends signals to the runner and the monitors. A runner (the show) hears the controllers, and can send feedback and config back. A monitor only listens. The server treats any other word as controller. optionaldefault 'controller'
room
String
The room name. Pages in different rooms never hear each other. 1 to 32 characters of A–Z a–z 0–9 _ -; the server turns anything else into default. optionaldefault 'default'
id
String
This device's id, the <id> in phone/<id>/…. The default is deviceId(), which is kept in localStorage. The server keeps its first 64 characters. optionaldefault deviceId()
url
String
The full WebSocket address, with the query already on it: wss://host/relay?role=controller&room=main&id=ab12. Nothing is added to it. Leave it out and the client uses ?relay=ws://… from the page's address, or else the page's own /relay (wss: on an https page), and adds role, room and id itself. optionaldefault null
batchHz
Number
How many times a second the values you set() are flushed to the relay, as one batch message. optionaldefault 30
resendMs
Number
Every this many milliseconds, send every stored value again, not only the changed ones. This is what lets a phone that joins late, or a show page that reloads, catch up. optionaldefault 1000
onStatus
Function
(client) => {}, called when the connection opens or closes, when a pong gives a new latency, and when the peer counts change. Read client.status, client.latency and client.peers inside it. optional
onSignal
Function
(name, value, msg) => {}, for every signal that arrives (a runner or monitor hears the controllers; a monitor also hears the runner). A batch calls it once per item. msg.pulse is true for a pulse. bindSignals() fills this in for you. optional
onFeedback
Function
(name, value) => {}, on a controller: a value the runner sent with feedback(), to show in a meter or to move a fader. optional
onConfig
Function
(key, data) => {}, on a controller: a standing config the runner sent with config(), such as the surface layout. A phone that joins late gets the stored ones right away. optional
connect(): open the socket and start the timers (flush, re-send, ping). Returns the client, so new RelayClient({…}).connect() works. Calling it again after close() starts over.
send(name, value, pulse?, extra?): send one signal now. pulse true marks it as an event. extra is an object whose keys are added to the message. Dropped if the socket is not open.
feedback(name, value): runner only. A value for the controllers to display or to echo, flushed like set() and re-sent every resendMs. Nothing happens on a controller or a monitor.
config(key, data): runner only. Send a standing config at once, for example config('surface', { layout }). The relay keeps it per key for controllers that join later, and forgets it when the runner leaves.
Notes
Public fields: role, room, id, url, batchHz, resendMs, the callbacks, status, latency (milliseconds, null until the first pong), peers and sent (how many messages have gone out).
After a reconnect the runner has to send its config() again: the client does not remember it. mountRemoteHost() does that for you.
Examples
js
import { RelayClient, bindSignals } from'@openav/relay';
import { Signals } from'@openav/core';
// the show page: every phone signal lands in a Signals registryconst signals = new Signals();
const show = new RelayClient({ role: 'runner', room: 'main' }).connect();
bindSignals(show, signals);
// the phone page: a continuous value and an eventconst phone = new RelayClient({ role: 'controller', room: 'main' }).connect();
phone.set('phone/me/tilt/x', 0.3);
phone.send('midi/note/on', { note: 60 }, true);
static The two ends of one room, as they would sit on two pages (the show and a phone): once the socket is open, the show's signals hold phone/me/tilt/x and, as an alias, phone/any/tilt/x.
js
import { RelayClient, bindSignals } from'@openav/relay';
import { Signals } from'@openav/core';
// A stand-in for the relay so this runs without a server: a controller's messages reach the runner,// and the runner's feedback and config reach the controller (the same routing the real relay does).const sockets = [];
globalThis.WebSocket = class {
constructor(url) {
this.role = new URL(url).searchParams.get('role');
this.readyState = 1;
sockets.push(this);
setTimeout(() => this.onopen?.());
}
send(text) {
const type = JSON.parse(text).type;
const to = this.role === 'controller' ? 'runner' : (type === 'feedback' || type === 'config') ? 'controller' : null;
for (const s of sockets) if (s.role === to) s.onmessage?.({ data: text });
}
close() {}
};
const signals = new Signals();
const show = new RelayClient({ role: 'runner', url: 'ws://relay/x?role=runner&room=demo&id=show' }).connect();
bindSignals(show, signals);
const got = [];
const phone = new RelayClient({
role: 'controller', url: 'ws://relay/x?role=controller&room=demo&id=ab12',
onFeedback: (name, value) => got.push('feedback ' + name + ' = ' + value),
onConfig: (key, data) => got.push('config ' + key + ' = ' + JSON.stringify(data)),
}).connect();
phone.set('phone/ab12/tilt/x', 0.3);
show.feedback('surface/main/level', 0.4);
show.config('surface', { title: 'demo' });
setTimeout(() => {
got.push('tilt in the show: ' + signals.get('phone/any/tilt/x'));
document.getElementById('app').innerHTML = got.join('<br>');
phone.close(); show.close();
}, 200);
module Prints the config and the feedback the phone got back from the show, then the phone's tilt as the show hears it (under its alias phone/any/tilt/x). Same calls as against a real relay.
To be confirmed
The reconnect delays (0.5 s, 1 s, 2 s, 4 s, then 5 s) are read from the code, not measured against a relay that goes down and comes back. The second example runs the client against a stand-in socket, not a real WebSocket server.
Related
bindSignals()Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
attachRelay()Attaches the room relay to a Node http.Server: WebSocket clients that connect to path are put in rooms and routed by role. Returns { hub, close() }.
deviceId()The short random id of this browser, kept in localStorage, that names this phone in its signals, as in phone/ab12/tilt/x.
mountRemoteHost()The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
What the relay knows about a signal name: its range and unit, or that it is a pulse. Returns null for names it does not know.
A Signals registry needs to know a name's range before a meter can scale it or a mapping can tell a knob from an event. fileSignal() calls this the first time it hears a name. What it knows: phone/<id>/tilt/* is −1..1, accel/* is −20..20 m/s², rot/* is −360..360 °/s, orient/* is 0..360 °, touch/count is 0..10, knock is a pulse, and every other phone/… name (touch, light) is 0..1. surface/… names are 0..1 (…/raw and …/delta are unscaled, {}). midi/note/on, midi/note/off, midi/virtual and the hit, on, off, last names of a controller are pulses, and midi/cc/* is 0..1.
Parameters
Name
Type
Description
name
String
A signal name such as phone/ab12/tilt/x, surface/main/hue or midi/note/on.
pulse
Boolean
Pass { pulse: true } when the message was sent as an event: the answer is then always { kind: 'pulse' }. optionaldefault false
Returns
Object | null Metadata for signals.define(): { min, max, unit? }, { kind: 'pulse' }, or {} for raw values. null when the name is not one it knows.
module Prints what the relay knows about seven names, from {"min":-1,"max":1} for tilt to null for a name it has never heard of.
Related
fileSignal()Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
bindSignals()Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
aliasOf()The "latest phone" name of a phone signal: aliasOf('phone/ab12/tilt/x') is 'phone/any/tilt/x'. null for anything else.
Makes an element a multi-touch pad: each finger publishes touch/x, touch/y and touch/down (0..1 inside the element), and touch/count says how many are down.
The first finger down is slot 0 and publishes touch/x, touch/y, touch/down; the second is slot 1 and publishes touch/1/x, touch/1/y, touch/1/down, and so on. A slot is given back when its finger lifts, so the lowest free number is used next. x runs 0 at the left edge to 1 at the right, y runs 0 at the top to 1 at the bottom, both held inside the element and rounded to 4 decimals. touch/down is 1 or 0, and touch/count is the number of fingers down.
It publishes touch/down 0 and touch/count 0 at once, so a receiver knows the pad is there before anyone touches it. It uses pointer events, so a mouse works too (one finger).
Parameters
Name
Type
Description
pad
HTMLElement
The element to listen on. Give it touch-action: none in CSS, or the browser may take a drag for a scroll and cancel the touch.
io
{ set(name, value) }
Where the values go, as for PhoneSensors: only set is used, and the names have no phone/<id>/ in front.
onMove
Function
(slot, x, y, down) => {}, when a finger touches and each time it moves. x and y are 0..1 and down is true. Use it to draw the finger. optional
onEnd
Function
(slot) => {}, when a finger lifts or the touch is cancelled. optional
Notes
The sense tab uses this on its touch pad, with io writing to a RelayClient under phone/<id>/.
Examples
js
import { attachTouchPad } from'@openav/remote';
const pad = document.createElement('div');
pad.style.cssText = 'width:200px;height:200px;background:#222;touch-action:none';
document.getElementById('app').append(pad);
const seen = [];
attachTouchPad(pad, { set: (name, value) => seen.push(name + '=' + value) });
// a finger lands a quarter of the way across, halfway down, then moves rightconst r = pad.getBoundingClientRect();
const touch = (type, id, fx, fy) => pad.dispatchEvent(new PointerEvent(type, {
pointerId: id, clientX: r.left + fx * r.width, clientY: r.top + fy * r.height, bubbles: true,
}));
touch('pointerdown', 1, 0.25, 0.5);
touch('pointermove', 1, 0.5, 0.5);
touch('pointerup', 1, 0.5, 0.5);
pad.after(Object.assign(document.createElement('p'), { innerHTML: seen.join('<br>') }));
module A 200 px square appears, and under it the values a finger would publish: touch/down=0, touch/count=0 at the start, then x, y, down and count as it lands, moves and lifts.
module Two fingers get slots 0 and 1; after the first lifts, the next finger takes slot 0 again. The list shows the callbacks and the count.
Related
PhoneSensorsTurns a phone's tilt, acceleration, rotation, knocks and front-camera light into named signals, in screen axes, and hands them to the io you give it.
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
RelayClientThe browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
For a show page that runs on the phone itself: the phone's own sensors go straight into a Signals as phone/local/…, with a phone/any/… copy.
A remote phone reaches the show through a relay. When the show page is open on the phone itself, there is no one to relay to, so this reads the sensors in place. The names and thresholds are the same as a remote phone's (phone/local/tilt/x is −1..1, phone/local/knock is a pulse), declared the same way by fileSignal(), and mirrored to phone/any/…, so a route written for "the latest phone", phone/any/tilt/x, works whether the phone is in your hand as a controller or is the show.
Before you call it
Call .start() on the result from a tap (iOS asks for permission only inside a gesture, and only on https pages). Until then nothing is published. See PhoneSensors.
Parameters
Name
Type
Description
signals
Signals
The registry to fill (from @openav/core).
prefix
String
What goes in front of each sensor name. optionaldefault 'phone/local/'
alias
String | null
The word for the copy, phone/<alias>/…. null turns the copy off. optionaldefault 'any'
source
String
Stored as the signals' source in signals.meta. optionaldefault 'local'
hooks
Object
The hooks of PhoneSensors: onKnock, onSample, notify. optionaldefault {}
Returns
PhoneSensors Nothing is listening yet: call .start() from a tap.
Examples
js
import { localSensors } from'@openav/remote';
import { Signals } from'@openav/core';
const signals = new Signals();
const own = localSensors(signals);
await own.start(); // on a phone: from a tap
window.dispatchEvent(new DeviceOrientationEvent('deviceorientation', { alpha: 0, beta: 22.5, gamma: -45 }));
own.stop();
document.getElementById('app').innerHTML = [
'phone/local/tilt/x = ' + signals.get('phone/local/tilt/x'),
'phone/any/tilt/y = ' + signals.get('phone/any/tilt/y'),
'source: ' + signals.meta.get('phone/local/tilt/x').source,
].join('<br>');
module Prints the tilt under both names (-1 and 0.5) and the source "local". On a phone the events come from the sensors instead of from this made-up one.
module A different prefix and no alias: prints "phone/me/tilt/x = 0.2, copy exists: false".
Related
PhoneSensorsTurns a phone's tilt, acceleration, rotation, knocks and front-camera light into named signals, in screen axes, and hands them to the io you give it.
fileSignal()Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
aliasOf()The "latest phone" name of a phone signal: aliasOf('phone/ab12/tilt/x') is 'phone/any/tilt/x'. null for anything else.
bindSignals()Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
Puts a small card on the show page with a QR code and the URL a phone should open, and a line that says whether the relay is reachable and how many phones are in.
The URL is the phone page (packages/remote/ next to this file) with ?room=<room> on it, and with ?relay=… as well when the show uses the public relay or a relay of your own, so the phone ends up in the same room on the same relay. On localhost it asks node serve.js for this machine's LAN address at /__info and puts that in the URL, because a phone cannot open localhost. The QR code is drawn from the URL; if the URL is too long for the encoder, the code is left out and the URL stays.
The status line reads relay ok · 2 phones · room demo when the connection is open (and adds · public relay on the public one), relay offline — is it node serve.js? when a local relay cannot be reached, and can't reach the relay (…) — retrying for a shared one. On a phone held upright the card starts folded, because the stage is small; tap it to open or fold it.
Parameters
Name
Type
Description
parent
HTMLElement
The element to add the card to. The card is position: absolute at its top left, so the parent should be positioned (position: relative) and big enough to show it.
host
Object
The host from mountRemoteHost(). The card reads host.room, host.relayMode and host.relayUrl, and takes over host.onStatus to write the status line.
Returns
HTMLElement The card's element, already in parent. Remove it with .remove().
module A card appears at the top left with a QR code, the phone URL ending in ?room=demo, and the line "relay ok · 2 phones · room demo".
Related
mountRemoteHost()The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
RelayClientThe browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
It is what a phone opens: lockViewport() so nothing scrolls or zooms, keepAwake() so the screen stays on, one RelayClient as a controller that every tab shares, and the tabs. Sense is a tilt ball, readouts, a multi-touch pad (attachTouchPad()) and the sensors (PhoneSensors) behind a start button. Keys is a piano that is one row in landscape and two stacked rows in portrait. Control is a Surface. MIDI is a virtual controller.
The control tab's layout comes from, in order: surfaceUrl or ?surface=, metaUrl or ?meta=, whatever layout the show publishes over the relay (the config surface), and finally a small generic panel so the page is never blank. The signals are phone/<id>/… (sense), midi/note/on|off and midi/cc/64 (keys), and surface/<page>/<id> (control). The app opens on ?tab=, else the tab used last in this browser, else sense.
Before you call it
A page of its own (it takes over the whole screen) and a relay to reach: the same origin's /relay unless relayUrl or ?relay= says otherwise. Without a show in the room the header says it is waiting for the work to open.
Parameters
Name
Type
Description
root
HTMLElement
Where to put the app. It fills the whole page (position: fixed), so pass document.body.
opts
Object
{ room, relayUrl, surfaceUrl, metaUrl }, each optional. room is the relay room (else ?room= in the address, else window.__REMOTE__.room, else default). relayUrl is a ws:// or wss:// relay (else ?relay=; only those two schemes are accepted, so a link cannot point the phone anywhere else). surfaceUrl is a layout JSON for the control tab and metaUrl a JSON with a params array, which the control tab turns into autoSurface(). The address can also say ?tab=sense|keys|control|midi. optionaldefault {}
Returns
Object{ relay, show(tab), surface, dispose() }: the RelayClient, a function that switches tab, the control tab's Surface (a getter, null until that tab has been opened), and dispose(), which closes the relay, unlocks the viewport, lets go of the wake lock and removes the app.
Notes
The words on the app's tabs and header are in Chinese.
Examples
js
import { mountRemote } from'@openav/remote';
// the whole phone app: this is what /packages/remote/ doesconst app = mountRemote(document.body, { room: 'main' });
// later: app.show('keys') switches tab, app.dispose() takes everything down
static On a phone: the status bar with the connection and latency, and the four tabs (sense, keys, control and MIDI, labelled in Chinese) along the bottom.
To be confirmed
Not opened on a real phone for this entry: how the tabs behave is read from packages/remote/app.js and docs/remote.md.
Related
mountRemoteHost()The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
PhoneSensorsTurns a phone's tilt, acceleration, rotation, knocks and front-camera light into named signals, in screen axes, and hands them to the io you give it.
attachTouchPad()Makes an element a multi-touch pad: each finger publishes touch/x, touch/y and touch/down (0..1 inside the element), and touch/count says how many are down.
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
RelayClientThe browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
createShow({ modules: { remote: true } }) calls this for you, so the world never has to know about phones. Calling it yourself does five things: it makes a RelayClient as the room's runner; files every phone signal into signals with bindSignals() (so phone/any/… always holds the latest phone); builds the layout from surface, world.surface or autoSurface() of the params; adds the surface/… → param routes to the mapper; and, when you call host.frame(dt, state) every frame, sends the resolved param values back about feedbackHz times a second so the phones' faders follow the timeline.
Making the host does not connect it: call host.connect() when you want the socket. On every (re)connection it publishes the layout as relay config surface, so a phone that joins later gets the right panel. host.layout, host.routes and host.bindings are there to look at straight away, and host.dispose() closes the socket.
Before you call it
A relay for connect() to reach: node serve.js hosts one at /relay, a static site uses the public relay (see pickRelay()), or pass url. Building the host needs none of that.
Parameters
Name
Type
Description
signals
Signals
The registry that phone signals are filed in. optional
params
Params
The world's params. Its schema is what the control surface is built from. optional
mapper
Mapper
The mapper that gets the surface/… → param routes (a route it already has is not added twice). optional
world
Object
The world: its params are used when params.schema is empty, its name becomes the layout's title, and its surface (a hand-written layout) is used instead of a generated one. optionaldefault null
An explicit layout. It wins over world.surface. optionaldefault null
feedbackHz
Number
How many times a second host.frame() sends the param values back to the phones. optionaldefault 10
url
String
A relay to use instead of this page's own /relay, such as wss://host/relay. role, room and id are added to it for you. The runner id is then show- plus four random characters, so two tabs of one show do not collide; without url it is show. optionaldefault null
mode
'local' | 'public' | 'custom'
What pickRelay() decided. Only the words of the join card depend on it; the host reports it as host.relayMode. optionaldefault null
controllers
MidiControllers
The on-screen MIDI controllers of @openav/midi: the phone's MIDI tab then shows the same device, and what is played on either screen shows on the other. optionaldefault null
module Prints what the host built from two params: a layout titled Bloom with a fader and a button, the two routes, and that the mapper now has them.
js
import { mountRemoteHost, mountJoinCard } from'@openav/remote/host';
// signals, params, mapper and world as aboveconst host = mountRemoteHost({ signals, params, mapper, world, room: 'main' }).connect();
mountJoinCard(document.body, host); // the QR code and URL for phonesfunction frame(dt, state) {
host.frame(dt, state); // echo the params back to the phones
}
static The usual wiring on a show page: connect, show the join card, and call host.frame() from the frame loop. A phone that scans the card moves the faders.
Related
mountJoinCard()Puts a small card on the show page with a QR code and the URL a phone should open, and a line that says whether the relay is reachable and how many phones are in.
RelayClientThe browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
bindSignals()Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
autoSurface()Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
routesFromLayout()Writes the mapper routes for a hand-written layout: every widget that names a param in target gets a route from its signal to that param.
feedbackFor()Turns the resolved param state into the normalized values the phone's widgets should show right now: [{ name, value }].
attachRelay()Attaches the room relay to a Node http.Server: WebSocket clients that connect to path are put in rooms and routed by role. Returns { hub, close() }.
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
Turns a phone's tilt, acceleration, rotation, knocks and front-camera light into named signals, in screen axes, and hands them to the io you give it.
This is the phone half of the sense tab, with no page in it: make one with an io, call start() from a tap, and signals flow. From the phone's own events it produces tilt/x and tilt/y (−1..1, which is ±45° from the resting pose, x to the right and y toward you, in screen axes, so a phone turned sideways still reports "tilt right is +x"), orient/alpha|beta|gamma (degrees), accel/x|y|z (m/s², gravity included), rot/alpha|beta|gamma (degrees per second), knock (a pulse { strength, delta, t }) and, if you turn the camera on, light (0..1).
A knock is a jolt: when the acceleration vector changes by more than threshold (10 m/s² by default) between two motion samples, and at least 130 ms after the last knock, io.send('knock', { strength, delta, t }, true) fires. delta is the size of the change, and strength runs from 0.15 just over the threshold up to 1. The phone also vibrates for 10 ms where it can. Hold the phone how you want to play it and call calibrate() to make that pose the zero of the tilt.
For the show page itself running on a phone, localSensors() wraps this class and files straight into a Signals.
Before you call it
Call start() from a tap or click, not on page load. iOS asks for motion permission only inside a user gesture, and it only gives motion sensors to pages on https://; touch and keys work over plain http://. start() always resolves true: look at denied for a refusal. If no sensor data arrives within 2.5 s, hooks.notify says so (a desktop, or an iPhone page on http). setCamera(true) also needs a gesture and HTTPS on iOS.
Parameters
Name
Type
Description
io
{ set(name, value), send(name, value, pulse?) }
Where the signals go. set gets the continuous ones, send the knocks (with pulse true). The names have no phone/<id>/ in front: tilt/x, accel/z. A RelayClient with a prefix, or a Signals registry, can sit behind it.
hooks
{ onKnock?, onSample?, notify? }
Optional callbacks. onKnock(strength) after each knock, onSample(name, value) for every continuous sample (to draw a readout), notify(message) with a one-line problem report such as a refused permission. The messages are written in Chinese. optionaldefault {}
start(): async. Ask for the motion permission where the browser requires it, then listen to deviceorientation and devicemotion. Resolves true even when the answer was no (then denied is true and the listeners are attached anyway). Calling it again does nothing.
setCamera(on): async. true opens the front camera and publishes light (the mean brightness of a 32 × 24 picture, 0..1) ten times a second; false stops it. Resolves true when done, false if the camera is not available (then hooks.notify says why).
PhoneSensors.screenTilt(beta, gamma, angle?): static. Turn the two raw tilt angles into screen axes for a screen angle of 0, 90, 180 or 270 degrees. Returns { x, y } in degrees.
Notes
Public fields: io, hooks, started, gotSensor (true once any sensor data has arrived), denied (iOS said no; it stays no until the Settings app is changed), rest and deg (tilt in degrees), threshold (the knock threshold, 10) and COOLDOWN (130 ms).
Examples
js
import { PhoneSensors } from'@openav/remote';
const seen = [];
const sensors = new PhoneSensors({
set: (name, value) => seen.push(name + ' = ' + value),
send: (name, value) => seen.push(name + ' ' + JSON.stringify(value)),
});
await sensors.start(); // on a phone: call this from a tap// the event a phone fires; here a made-up one, tipped 22.5° toward you and 45° to the left
window.dispatchEvent(new DeviceOrientationEvent('deviceorientation', { alpha: 5, beta: 22.5, gamma: -45 }));
sensors.stop();
document.getElementById('app').innerHTML = seen.join('<br>');
module Prints tilt/x = -1 (45° left is the end of the range), tilt/y = 0.5, then the three raw orient angles.
js
import { PhoneSensors } from'@openav/remote';
const seen = [];
const sensors = new PhoneSensors(
{ set() {}, send: (name, v) => seen.push(name + ': strength ' + v.strength + ', delta ' + v.delta) },
{ onKnock: (s) => seen.push('onKnock(' + s + ')') },
);
await sensors.start();
awaitnew Promise((r) => setTimeout(r, 200)); // the 130 ms knock cooldown counts from page load
navigator.vibrate = () => true; // a knock also buzzes the phone; a stand-in for a desktop pageconst motion = (x) => window.dispatchEvent(new DeviceMotionEvent('devicemotion', {
accelerationIncludingGravity: { x, y: 0, z: 9.8 },
}));
motion(0); // at rest, gravity only
motion(15); // a jolt: the acceleration changed by 15 m/s²
motion(15.5); // too small to count
sensors.stop();
seen.push('threshold now ' + sensors.cycleThreshold());
document.getElementById('app').innerHTML = seen.join('<br>');
module Prints one knock with strength 0.35 and delta 15 (and the onKnock call), then the new threshold 12 after cycleThreshold().
js
import { PhoneSensors } from'@openav/remote';
// the same tilt (beta 10, gamma 20) as the screen turns: x is always "to the right of the screen"const rows = [0, 90, 180, 270].map((angle) => {
const { x, y } = PhoneSensors.screenTilt(10, 20, angle);
return'screen angle ' + angle + ': x ' + x + ', y ' + y;
});
document.getElementById('app').innerHTML = rows.join('<br>');
module Prints x and y for the four screen angles: (20, 10), (10, -20), (-20, -10), (-10, 20).
To be confirmed
Not tried on a real iPhone or Android phone for this entry: the iOS permission flow, the HTTPS requirement and the 2.5 s "no data" message are read from the code and from docs/remote.md. The examples feed the class real DeviceOrientationEvent and DeviceMotionEvent objects in a desktop browser instead.
Related
localSensors()For a show page that runs on the phone itself: the phone's own sensors go straight into a Signals as phone/local/…, with a phone/any/… copy.
attachTouchPad()Makes an element a multi-touch pad: each finger publishes touch/x, touch/y and touch/down (0..1 inside the element), and touch/count says how many are down.
RelayClientThe browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
signalMeta()What the relay knows about a signal name: its range and unit, or that it is a pulse. Returns null for names it does not know.
Packs widgets into a grid first-fit, row by row, and never overlaps; fixed placements are kept and block their cells. Returns the placements and the rows used.
Each item takes the first free spot found by scanning left to right, then top to bottom: what a person does laying out a controller. The grid grows downward as far as it must, so rows is a result, not a limit. resolvePage() uses it for widgets without a position and for the automatic reflow.
Parameters
Name
Type
Description
items
{ id, w, h }[]
What to place, in order. Every item needs a numeric w and h in cells, at least 1. A w wider than the grid is cut to cols.
cols
Number
How many columns the grid has.
fixed
{ id, x, y, w, h }[]
Placements that are already decided. They come first in the result and the other items go around them. optionaldefault []
Returns
{ placements, rows }placements is a list of { id, x, y, w, h } (the fixed ones first), rows is how many rows they fill.
Notes
An item with no w (or a w that is not a number) never finds a spot and the function never returns. Give every item a w and an h; DEFAULT_SIZE has the natural size of each widget type.
module a and b share the first row, c takes a full row, d goes below: rows is 5.
js
import { autoArrange } from'@openav/surface/layout';
// a button that must stay at the top left; the new knob goes around itconst { placements } = autoArrange(
[{ id: 'knob', w: 2, h: 1 }],
4,
[{ id: 'go', x: 0, y: 0, w: 2, h: 1 }],
);
document.getElementById('app').textContent = placements.map((p) => p.id + ' at ' + p.x + ',' + p.y).join(' ');
module Prints "go at 0,0 knob at 2,0": the fixed placement comes first and the knob takes the first free cells.
Related
resolvePage()Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.
stretchRows()Stretches a last row that stops short to the edge of the grid: five faders in four columns leave one lonely fader, and this makes it four columns wide.
DEFAULT_SIZEThe natural size of each widget type, in grid cells, as { w, h }: a layout only has to give x, y, w, h when it cares.
Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
This is how a world that declares what it can be performed with gets a phone panel for free: const { layout, routes, bindings } = autoSurface(world.params), then routes.forEach((r) => mapper.addRoute(r)) and new Surface(el, { layout, sink }). Each param becomes the widget widgetTypeFor() picks. Each group of params gets its own page (a param without a group goes on main), long groups continue on main2, main3 …, and a pair from pairs becomes one XY pad with two routes, …/x and …/y.
Widget ids are the param keys (anything outside A–Z a–z 0–9 _ - becomes _), colours go round cyan, amber, magenta, lime, violet, coral, and a radio's options come from the param or are the numbers. A pulse param gets no binding, since there is no value to show. The routes already scale the normalized signal to the param's min and max. A world that comes with its own layout uses routesFromLayout() instead.
Parameters
Name
Type
Description
params
Object[]
The param declarations: { key, label?, min, max, def, step?, pulse?, group?, options?, surface? }. surface: { type, color, label } on a param overrides the choice. optionaldefault []
perPage
Number
How much fits on a page, counted in weights: a fader or a meter weighs 1, an XY pad 4, anything else 2. The default 8 is eight faders, four knobs or two XY pads. A page that scrolls is a page that fails, so a full page starts the next. optionaldefault 8
style
'auto' | 'fader' | 'knob'
Faders or knobs for the continuous params. 'auto' picks faders up to six, knobs beyond. optionaldefault 'auto'
pairs
string[][]
Pairs of param keys to drive with one XY pad: [['cx', 'cy']]. The first key is x and the second is y. optionaldefault []
meters
string[]
Param keys to also show as read-only meters, on a page called Meters. optionaldefault []
smooth
Number
Seconds of smoothing on the continuous routes, to hide the steps of a 30 Hz network (zipper noise). Pulses, toggles and radios are never smoothed. optionaldefault 0.04
title
String
The layout's title. optionaldefault ''
Returns
{ layout, routes, bindings }layout is a layout JSON for Surface. routes is a list of mapper routes { source, target, inMin, inMax, curve, smooth, outMin, outMax }, from surface/<page>/<id> to each param. bindings is { param, name, page, widget, min, max, readonly? } for feedbackFor().
module Prints the one page main with a fader for hue and size, a radio for mode and a button for burst, then the four routes, smoothed by 0.04 s except the radio and the button.
module cx and cy share one XY pad, density is a fader and also a read-only meter on its own page, and glow, in group look, gets a page of its own.
js
import { autoSurface } from'@openav/surface/auto';
// twenty params do not fit on one page: more than six continuous ones become knobs, four to a pageconst params = Array.from({ length: 20 }, (_, i) => ({ key: 'p' + i, min: 0, max: 1, def: 0 }));
const { layout } = autoSurface(params);
document.getElementById('app').textContent = layout.pages
.map((p) => p.id + ': ' + p.widgets.length + ' ' + p.widgets[0].type + 's').join(' ');
module Prints "main1: 4 knobs main2: 4 knobs …" up to main5: the page limit counts a knob as 2, so four fit in the default 8.
Related
widgetTypeFor()Picks the widget type for one param declaration: a button for a pulse, a toggle, a radio, a fader or a knob.
feedbackFor()Turns the resolved param state into the normalized values the phone's widgets should show right now: [{ name, value }].
routesFromLayout()Writes the mapper routes for a hand-written layout: every widget that names a param in target gets a route from its signal to that param.
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
mountRemoteHost()The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
DEFAULT_SIZEThe natural size of each widget type, in grid cells, as { w, h }: a layout only has to give x, y, w, h when it cares.
True when the page can ask for fullscreen: the browser has requestFullscreen (or the WebKit one). The code treats iPhone Safari as not having it.
Use it to decide whether to show a fullscreen button: if (canFullscreen()) button.hidden = false. It only checks that the function exists; whether a request succeeds is up to the browser, which wants it to come from a tap (toggleFullscreen()).
Returns
Booleantrue or false. false without a document.
Examples
js
import { canFullscreen } from'@openav/surface';
document.getElementById('app').textContent = 'this browser can ask for fullscreen: ' + canFullscreen();
module Prints "true" in desktop Chrome, Firefox and Safari, and in Android Chrome.
To be confirmed
Not checked on an iPhone or iPad: that iPhone Safari has no fullscreen function is what the code's own comment says, not something measured here.
Related
toggleFullscreen()Enters fullscreen if the page is not in it, and leaves it if it is. Call it from a tap, because browsers refuse otherwise.
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
The seven colour names a widget's color may be: cyan, amber, magenta, lime, violet, coral, white.
{ id: 'hue', type: 'knob', color: 'amber' } makes the widget's fill, its value readout and its pressed outline amber. A widget without a colour uses --oav-accent of THEME_VARS. autoSurface() hands out the first six in turn (white is left for you). A name that is not in the list gets no colour of its own and falls back to the accent; validateLayout() does not check it.
module Prints cyan, amber, magenta, lime, violet, coral, white.
Related
THEME_VARSThe look of every control as 15 CSS custom properties on :root (--oav-bg, --oav-accent …): override them to re-skin the whole surface.
autoSurface()Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
WIDGET_TYPESThe names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
The natural size of each widget type, in grid cells, as { w, h }: a layout only has to give x, y, w, h when it cares.
A widget without a w or h takes the size of its type, held to the number of columns of the grid. autoArrange() and resolvePage() use it, and so does autoSurface() when it counts how many widgets fit on a page. A type it does not know counts as 2×2.
Type
Object
Fourteen keys, one per type: fader 1×4, knob 2×2, encoder 2×2, button 2×1, toggle 2×1, xy 4×4, bank 4×4, radio 4×1, pads 4×4, keyboard 8×4, number 2×1, text 4×1, label 2×1, meter 1×4 (width × height).
Examples
js
import { DEFAULT_SIZE } from'@openav/surface';
const { w, h } = DEFAULT_SIZE.knob;
document.getElementById('app').textContent = 'a knob is ' + w + ' cells wide and ' + h + ' tall; a fader is ' +
DEFAULT_SIZE.fader.w + ' by ' + DEFAULT_SIZE.fader.h;
module Prints "a knob is 2 cells wide and 2 tall; a fader is 1 by 4".
Related
WIDGET_TYPESThe names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
autoArrange()Packs widgets into a grid first-fit, row by row, and never overlaps; fixed placements are kept and block their cells. Returns the placements and the rows used.
resolvePage()Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.
Turns the resolved param state into the normalized values the phone's widgets should show right now: [{ name, value }].
It is pure and cheap, so the show calls it about ten times a second and sends each result to the phones with relay.feedback(name, value); host.frame() of mountRemoteHost() does exactly that. On the phone, Surface's feedback() moves the widget, so a fader follows the timeline and other controllers.
Parameters
Name
Type
Description
state
Object
The resolved param values, such as params.resolve(…): { hue: 90, size: 5.5 }.
{ name, value }[] One entry per binding that has a number in state: name is the widget's signal, value is the param scaled to 0..1 and held in range. A param that is missing, not a number, or with min equal to max is skipped.
module Prints surface/main/hue = 0.25 (90 of 0 to 360) and surface/main/size = 0.5 (5.5 of 1 to 10).
Related
autoSurface()Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
mountRemoteHost()The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
RelayClientThe browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
Says whether a grid { cols, rows } is 'landscape' (at least as many columns as rows) or 'portrait'.
resolvePage() compares it with the orientation of the screen: a base grid of 8 by 4 is used as it is on a landscape screen and reflowed on a portrait one.
Parameters
Name
Type
Description
g
{ cols, rows }
The grid.
Returns
'landscape' | 'portrait''landscape' when g.cols >= g.rows, else 'portrait'.
Vibrates the phone for ms milliseconds where the browser can (Android Chrome); silent everywhere else, and enabled false turns it off.
It calls navigator.vibrate(ms) when that exists and swallows any error. Browsers without it, such as Safari on iPhone, simply do nothing, and Chrome only vibrates after the person has touched the page. Surface calls it for buttons, toggles and radios, and its haptics: false option turns that off.
Parameters
Name
Type
Description
ms
Number
How long to vibrate, in milliseconds. A button press in the surface uses 10 to 14.
enabled
Boolean
Pass false to do nothing: a page can offer a "haptics" switch and hand it here. optionaldefault true
Examples
js
import { haptic } from'@openav/surface';
document.querySelector('button').addEventListener('pointerdown', () => haptic(20)); // a short buzz on every press
static On an Android phone the button buzzes for 20 ms each time it is pressed; elsewhere nothing happens.
js
import { haptic } from'@openav/surface';
const log = [];
navigator.vibrate = (ms) => { log.push('vibrate ' + ms + ' ms'); returntrue; }; // a stand-in, so there is something to see
haptic(20);
haptic(12, false); // switched off: nothing is called
haptic(30, true);
document.getElementById('app').innerHTML = log.join('<br>');
module Prints "vibrate 20 ms" and "vibrate 30 ms": the call with enabled false made no vibration.
To be confirmed
Not felt on a real Android phone for this entry. The example uses a stand-in for navigator.vibrate to show when it is called.
Related
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
keepAwake()Asks the browser to keep the screen on (a screen wake lock) and asks again when the tab comes back. Returns { release() }.
lockViewport()Makes a page behave like an instrument panel instead of a web page: no scrolling, no pinch zoom, no rubber-banding. Returns a function that undoes most of it.
Writes the surface's stylesheet into the page, once: the 15 theme variables on :root and the rules for every widget. Safe to call again.
Surface and mountRemote() call it for you, so you need it only when you build a widget by yourself, or when you want the variables (var(--oav-accent)) for your own elements. It adds one <style id="openav-surface-css"> to the head, and does nothing if that already exists, or if there is no document. To change the look, set a variable after that: document.documentElement.style.setProperty('--oav-accent', '#ff5fb3').
module A fader and a knob drawn in pink: they have no colour of their own, so they take the accent that was overridden.
Related
THEME_VARSThe look of every control as 15 CSS custom properties on :root (--oav-bg, --oav-accent …): override them to re-skin the whole surface.
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
WIDGET_CLASSESThe widget classes, one per type, that Surface builds from a layout: WIDGET_CLASSES.fader, .knob, .xy and so on (advanced).
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
Asks the browser to keep the screen on (a screen wake lock) and asks again when the tab comes back. Returns { release() }.
A phone that is a controller dims and locks within a minute unless something holds the screen on. keepAwake() requests navigator.wakeLock.request('screen') right away if the page is visible. The operating system drops the lock whenever the page is hidden, so it asks again each time the page becomes visible. It is best effort and silent: a browser with no wake lock, or one that says no, just means the screen may still dim, and it never throws.
Before you call it
A page that is visible, on https or localhost (the wake lock only exists on secure pages). mountRemote() calls it for you.
Returns
{ release }release() stops asking and lets go of the lock.
Examples
js
import { keepAwake } from'@openav/surface';
const awake = keepAwake(); // the screen stays on while this page is visible// when the show is over:// awake.release();
static On a phone or a laptop the screen stops dimming while the page is open and visible.
js
import { keepAwake } from'@openav/surface';
const log = [];
// a stand-in for the browser's wake lock, so there is something to see on any machine
Object.defineProperty(navigator, 'wakeLock', { configurable: true, value: {
request: async (type) => { log.push('request ' + type); return { release() { log.push('lock released'); } }; },
} });
const awake = keepAwake();
awaitnew Promise((r) => setTimeout(r, 50));
awake.release();
log.push('release() called');
document.getElementById('app').innerHTML = log.join('<br>');
module Prints "request screen", then "lock released" and "release() called": one request when the page is visible, and release() lets the lock go.
To be confirmed
Not tried on a real iPhone or iPad: Safari may refuse a wake-lock request that is not made inside a tap, in which case the screen can still dim. The second example only shows the calls, with a stand-in for navigator.wakeLock.
Related
lockViewport()Makes a page behave like an instrument panel instead of a web page: no scrolling, no pinch zoom, no rubber-banding. Returns a function that undoes most of it.
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
Makes a page behave like an instrument panel instead of a web page: no scrolling, no pinch zoom, no rubber-banding. Returns a function that undoes most of it.
It sets <meta name="viewport"> to width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no,viewport-fit=cover (making the tag if there is none) and adds a style that gives html and body no margin, the full height, overflow: hidden, overscroll-behavior: none and touch-action: none. iOS ignores user-scalable=no, so it also cancels gesturestart and gesturechange (pinch zoom) and the default of every touchmove, except one that starts in an input or textarea, so a text box still works.
This is for a page that is a control panel, such as mountRemote(), which calls it. Do not call it on a page that has to scroll. Without a document (in Node) it returns a function that does nothing.
Returns
Function Call it to remove the stylesheet and the listeners. The viewport <meta> stays as it was set.
module Prints the viewport string, "hidden" while locked, and "visible" after unlock(): the stylesheet goes away, the meta tag stays.
Related
keepAwake()Asks the browser to keep the screen on (a screen wake lock) and asks again when the tab comes back. Returns { release() }.
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
Returns a copy of a layout with the defaults filled in: ids, titles, grids and the version. The input is not changed.
A page without an id becomes page1, page2 …; without a title it gets its id (or Page 1); without a grid it gets { cols: 8, rows: 6 }. A widget without an id becomes its type and its position in the list, fader1; an id already used on the page gets _ added until it is unique. Every other field is kept as it is. Surface normalizes the layout it is given, and so do signalNames() and the show side of the remote.
Parameters
Name
Type
Description
raw
Object
The layout JSON. undefined or an object without pages gives a layout with no pages.
Returns
Object{ version: 1, title, pages } with every page and widget complete.
Notes
The default grid has 6 rows. autoSurface() writes its own grids of 8 by 4, so this default only matters for layouts that leave the grid out.
Says whether a box is 'landscape' or 'portrait' from its width and height; a square counts as landscape.
Surface calls it with the size of its container, not the size of the screen, so a panel in a half-width pane can be portrait while the phone is landscape. Use gridOrientation() for the shape of a grid.
Parameters
Name
Type
Description
w
Number
The width, in any unit.
h
Number
The height, in the same unit.
Returns
'landscape' | 'portrait''landscape' when w >= h, else 'portrait'.
Examples
js
import { orientationOf } from'@openav/surface/layout';
document.getElementById('app').innerHTML = [[844, 390], [390, 844], [500, 500]]
.map(([w, h]) => w + ' x ' + h + ': ' + orientationOf(w, h)).join('<br>');
module Prints landscape for 844 x 390, portrait for 390 x 844, and landscape for the square.
Related
gridOrientation()Says whether a grid { cols, rows } is 'landscape' (at least as many columns as rows) or 'portrait'.
resolvePage()Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
Decides how many piano keys, in how many rows, fit a box: one wide row, or two stacked rows when the box is tall, so keys stay a playable size.
A piano is two and a half metres wide and a phone held upright is six centimetres, so shrinking the keys makes a toy. Instead the keyboard changes shape with its box. A wide box gets one row with as many octaves as fit at minKeyPx or wider. A tall box gets two rows, the upper one continuing the pitch range where the lower stops, within reach of a thumb. The rows end on E or B and the next one starts on the next white key, so no black key is left alone at the seam. Notes stop at 127. The keyboard widget of Surface calls this every time its box changes size.
Parameters
Name
Type
Description
width
Number
The width of the box, in pixels. optional
height
Number
The height of the box, in pixels. optional
base
Number
The MIDI note of the lowest key. 48 is C3 (C4 is 60). optionaldefault 48
minKeyPx
Number
The narrowest a white key may be, in pixels. At least 3 white keys are kept. optionaldefault 38
maxWhites
Number
The most white keys in a row. Two stacked rows are limited to 14 each. optionaldefault 22
stackBelow
Number
Stack into two rows when width ÷ height is below this. optionaldefault 2.2
forceRows
1 | 2
1 or 2 to decide yourself; 0 lets the shape of the box decide. optionaldefault 0
Returns
{ stacked, whites, keyPx, rows }stacked is true for two rows, whites the white keys in a row, keyPx the width of a white key. rows is a list of { base, semitones, whites }, the top row first (the higher notes), the way two keyboards are read: base is the MIDI note of the first key and semitones how many notes the row spans.
module Prints one row of 21 white keys (notes 48 to 83) for the wide box, and two rows of 10 (notes 65 to 81 on top, 48 to 64 below) for the tall one.
Related
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
WIDGET_TYPESThe names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.
A phone gets turned, so a layout has to survive it. The rules, in order: 1. If the page has a portrait or landscape entry for this orientation, use its grid and its place ({ widgetId: { x, y, w, h } }); widgets it does not mention keep their own position, or are packed into free cells (source is 'explicit'). 2. Else, if the base grid has the right shape (more columns than rows is landscape) and every widget has an x and a y, use the base ('base'). 3. Else reflow: the same widgets in reading order, packed again into half the columns (portrait, at least 2) or twice as many (landscape, up to 16), with a lone last row stretched to the edge by stretchRows() ('reflow').
Rows are a count, not pixels, so the result always fills the screen and nothing scrolls. A widget without w and h takes the size of its type from DEFAULT_SIZE. Surface calls this for every page each time the orientation changes.
Parameters
Name
Type
Description
page
Object
One page of a layout: { grid, widgets, portrait?, landscape? }.
orientation
'portrait' | 'landscape'
Which way the screen is held.
Returns
{ grid, placements, source }grid is { cols, rows }, placements a list of { id, x, y, w, h }, source is 'explicit', 'base' or 'reflow'.
Examples
js
import { resolvePage } from'@openav/surface/layout';
// eight faders in a row: right for a landscape screen, too wide for a portrait oneconst page = {
grid: { cols: 8, rows: 4 },
widgets: Array.from({ length: 8 }, (_, i) => ({ id: 'f' + i, type: 'fader', x: i, y: 0, w: 1, h: 4 })),
};
const out = ['landscape', 'portrait'].map((o) => {
const r = resolvePage(page, o);
return o + ': ' + r.source + ', ' + r.grid.cols + ' columns x ' + r.grid.rows + ' rows';
});
document.getElementById('app').innerHTML = out.join('<br>');
module Prints "landscape: base, 8 columns x 4 rows" and "portrait: reflow, 4 columns x 8 rows": turned upright, the eight faders become two rows of four.
module Prints "explicit": a goes where the portrait entry says, the knob and the button (which have no place) fill the free cells.
Related
autoArrange()Packs widgets into a grid first-fit, row by row, and never overlaps; fixed placements are kept and block their cells. Returns the placements and the rows used.
stretchRows()Stretches a last row that stops short to the edge of the grid: five faders in four columns leave one lonely fader, and this makes it four columns wide.
orientationOf()Says whether a box is 'landscape' or 'portrait' from its width and height; a square counts as landscape.
gridOrientation()Says whether a grid { cols, rows } is 'landscape' (at least as many columns as rows) or 'portrait'.
normalizeLayout()Returns a copy of a layout with the defaults filled in: ids, titles, grids and the version. The input is not changed.
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
Writes the mapper routes for a hand-written layout: every widget that names a param in target gets a route from its signal to that param.
Give a widget a target and the layout file is enough to perform a world. target is a param key, or an array for the widgets with several signals: an xy sends x to the first and y to the second, a bank sends channel i to the i-th. Every other widget uses its first (or only) target. The routes already scale the 0..1 signal to the param's min and max. mountRemoteHost() uses this for a layout from world.surface or from its surface option.
Parameters
Name
Type
Description
layout
Object
The layout JSON, whose widgets may have a target.
params
Object[]
The param declarations. A target that is not among them is skipped without a warning. optionaldefault []
smooth
Number
Seconds of smoothing on continuous routes. A widget's own smooth wins. Pulse params and button, toggle and radio widgets are never smoothed. optionaldefault 0.04
Returns
Object[] Routes in the same shape autoSurface() makes: { source, target, inMin, inMax, curve, smooth, outMin, outMax }, with curve taken from the widget (linear if it has none).
module Prints surface/p/size → size, surface/p/pos/x → cx, surface/p/pos/y → cy, surface/p/go → burst: the label has no target, so no route.
Related
autoSurface()Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
signalNames()Lists every signal name a layout can publish, so a mapping screen, a doc or a test can show them without touching the panel.
mountRemoteHost()The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
Lists every signal name a layout can publish, so a mapping screen, a doc or a test can show them without touching the panel.
It follows the table in WIDGET_TYPES: an XY pad gives /x, /y, /down; a bank gives /1 to /count; pads give /hit and /0 to /<rows × cols − 1>; an encoder gives its name and /delta; a keyboard gives midi/note/on, midi/note/off, midi/cc/64 and midi/virtual; a label or a meter gives nothing. The …/raw names that a widget with a range also sends are not in the list.
Parameters
Name
Type
Description
layout
Object
The layout JSON (it is normalized first).
Returns
string[] Unique names such as surface/main/hue, surface/main/pad/x, midi/note/on, in page and widget order.
module Prints surface/main/hue, then pos/x, pos/y, pos/down, then mix/1 and mix/2: the label gives nothing.
Related
WIDGET_TYPESThe names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
normalizeLayout()Returns a copy of a layout with the defaults filled in: ids, titles, grids and the version. The input is not changed.
routesFromLayout()Writes the mapper routes for a hand-written layout: every widget that names a param in target gets a route from its signal to that param.
Makes a sink for a Surface that writes straight into a Signals registry: for a panel on the same page as the show, with no phone and no relay.
A name it has not met is declared first: { kind: 'pulse' } when info.pulse is set, otherwise { min: 0, max: 1 }. Then a pulse fires with signals.pulse() and anything else is signals.set(). So a route in the mapper, { source: 'surface/main/hue', target: 'hue' }, works whether the finger is on this page or on a phone.
It does no scaling: the …/raw signal that a widget with a range sends is filed like any other name, with the range 0..1 it was declared with, so ignore those or give them their own declaration.
Parameters
Name
Type
Description
signals
Signals
The registry to write into (from @openav/core).
Returns
Function A (name, value, info) => {} to pass as Surface's sink.
module Prints "level = 0.4, kind continuous" and "go = 1, kind pulse": the sink declared each name the first time it saw it.
Related
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
bindSignals()Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
fileSignal()Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
Stretches a last row that stops short to the edge of the grid: five faders in four columns leave one lonely fader, and this makes it four columns wide.
A row is stretched only when it is safe: its widgets are all the same height, side by side with no gaps from the left edge, and nothing else sits to their right. The extra columns are dealt out one at a time from left to right, so a short row of three in four columns gives the first widget two columns. A row that does not qualify is left alone, and nothing ever overlaps. resolvePage() calls it at the end of a reflow.
Parameters
Name
Type
Description
placements
{ id, x, y, w, h }[]
Placements, for example from autoArrange(). Not changed: copies come back.
cols
Number
How many columns the grid has.
Returns
Object[] The placements, with the short rows widened.
module Prints f0 to f3 at width 1 both times, and f4 going from width 1 to width 4.
Related
autoArrange()Packs widgets into a grid first-fit, row by row, and never overlaps; fixed placements are kept and block their cells. Returns the placements and the rows used.
resolvePage()Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.
A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
Pass new Surface(container, { layout, sink }) and a panel appears: faders, knobs, an XY pad, pads, a keyboard, whatever the layout names, each with its live number. Widgets are dumb on purpose. Fingers go in, named normalized signals come out of sink, and the surface never knows what the work is. Run it on the show page with signalSink(), or on a phone with a sink that sends to the relay: the same layout works for both. A world's params can grow the whole layout by themselves with autoSurface().
The grid always fills the container and nothing scrolls. When the container is turned, the page is laid out again for the new orientation by resolvePage(): a portrait or landscape variant in the layout if there is one, else the base grid, else the same widgets packed again into half or double the columns. Touching a widget outlines it in its colour, and a double-tap on a fader or knob returns it to its default.
Runner to widget goes the other way: feedback(name, value) moves a fader to follow the timeline, or fills a meter. A widget under a finger ignores feedback while it is touched and for about half a second after, so it does not fight the hand (a button only while it is held).
Before you call it
A browser page (it builds DOM) and a container with a size. The panel injects its look once with injectTheme(). Nothing else to start: it works on the first touch. On a phone page, lockViewport() stops the page from scrolling and zooming under the fingers.
Parameters
Name
Type
Description
container
HTMLElement
Where the panel goes. The panel is position: absolute and fills it, so the container needs a position (position: relative) and a size, or the panel has none.
layout
Object
The layout JSON (see validateLayout()): pages, a grid, widgets. A bad layout throws an error that lists every problem. optional
sink
Function
(name, value, info) => {}, called for every signal a touch makes. name is like surface/main/hue, value is normalized 0..1 (an event is info.pulse, a string for a text box). When a widget's range is not 0..1, info.raw holds the unscaled value, and a second call follows with the name …/raw. signalSink() writes into a Signals; a phone passes one that calls relay.set. optional
haptics
Boolean
Buzz the phone briefly on a button, a toggle, a radio choice or a double-tap reset, where the browser can (Android Chrome). false turns it off. optionaldefault true
tabs
Boolean
Show a tab strip when the layout has more than one page. false hides it, and you switch with setPage(). optionaldefault true
orientation
'portrait' | 'landscape'
Force the orientation. Left out, it follows the container's shape (wider than tall is landscape) and changes when the container does. optionaldefault null
onPage
Function
(pageId) => {}, when the page changes, and once at the start with the first page. optionaldefault null
setLayout(raw): rebuild the panel from a layout. Throws an Error that begins invalid surface layout: and lists every problem if validateLayout() finds any. Keeps the current page when the new layout has one with the same id. Returns the surface.
feedback(name, value): runner to widget. name is the full signal name such as surface/main/level, value is normalized 0..1. A name that no widget listens to is ignored.
dispose(): stop watching the container's size, remove the widgets and take the panel out of the page.
Notes
Public fields: layout (the normalized layout), root (the panel's element), orientation, pageId, widgets (a Map of "page/id" to widget), sink, haptics.
The constructor appends the panel to the container before it checks the layout, so a layout that throws there leaves an empty panel behind. Run validateLayout() first if the layout comes from a file or a person.
module A dark panel with a fader, an XY pad and a button. Touch one and the line under it shows the signal it sent, for example surface/main/size = 0.75 (raw 7.75).
module A bad layout throws and lists every problem (an unknown type "dial", a duplicate id "x"); the panel that was already there stays as it was.
Related
signalSink()Makes a sink for a Surface that writes straight into a Signals registry: for a panel on the same page as the show, with no phone and no relay.
validateLayout()Checks a layout and returns a list of readable problems; an empty list means it is fine.
autoSurface()Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
resolvePage()Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.
injectTheme()Writes the surface's stylesheet into the page, once: the 15 theme variables on :root and the rules for every widget. Safe to call again.
lockViewport()Makes a page behave like an instrument panel instead of a web page: no scrolling, no pinch zoom, no rubber-banding. Returns a function that undoes most of it.
WIDGET_TYPESThe names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
The look of every control as 15 CSS custom properties on :root (--oav-bg, --oav-accent …): override them to re-skin the whole surface.
The design is a set of rules rather than decoration, because a controller is used in the dark, fast and half-looking: dark cards on near-black, the only saturated thing on screen is a value, every touch target at least 44 px (--oav-touch-min), the number you are changing always visible, and a pressed widget outlines itself in its colour. Everything is a CSS variable, so a work re-skins the surface by overriding a few of them, never by editing widget code.
injectTheme() writes these values into the page once. Override after that, on document.documentElement or in your own stylesheet, with a rule that comes later on :root.
Type
Object
Property name to its default value: --oav-bg #0a0c11, --oav-surface #141922, --oav-surface-2 #1b2230, --oav-line #283044, --oav-text #e8ecf5, --oav-dim #8691a8, --oav-accent #7ea6ff, --oav-ok #3ddc84, --oav-warn #f5a524, --oav-bad #e5484d, --oav-radius 14px, --oav-gap 8px, --oav-touch-min 44px, --oav-font (the system font) and --oav-mono (a monospace font).
Enters fullscreen if the page is not in it, and leaves it if it is. Call it from a tap, because browsers refuse otherwise.
It asks for the whole page (document.documentElement) with the browser's navigation bar hidden, and quietly does nothing on a browser without fullscreen (see canFullscreen()). The phone app puts it on a ⛶ button in the status bar.
Before you call it
A user gesture: call it from a click or pointerdown handler. Called on page load it is refused by the browser.
static A button that puts the page into fullscreen and out again, and is hidden where the browser has no fullscreen.
Related
canFullscreen()True when the page can ask for fullscreen: the browser has requestFullscreen (or the WebKit one). The code treats iPhone Safari as not having it.
mountRemote()The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
Checks a layout and returns a list of readable problems; an empty list means it is fine.
It looks for: no object, or no non-empty pages array (it stops there); a page without an id, with a duplicate id or an id outside A–Z a–z 0–9 _ -; a widget whose type is not in WIDGET_TYPES, without an id, with a duplicate id on its page or an id outside those characters (the id becomes part of a signal name); and a min equal to its max. It reports every problem it finds, not only the first.
It does not look at positions, grids or targets: a widget that overlaps another, or a target that is not a param, is not a problem here. Surface's setLayout() runs it and throws with the list when it is not empty.
Parameters
Name
Type
Description
layout
Object
The raw layout JSON.
Returns
string[] One sentence per problem, with where it is (pages[0].widgets[2]: unknown type "dial"). [] when the layout is fine.
module Prints "good: []", then five problems for the bad layout: the page id, the unknown type, the duplicate id, the missing id and min equal to max.
Related
normalizeLayout()Returns a copy of a layout with the defaults filled in: ids, titles, grids and the version. The input is not changed.
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
WIDGET_TYPESThe names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
The widget classes, one per type, that Surface builds from a layout: WIDGET_CLASSES.fader, .knob, .xy and so on (advanced).
Most pages never touch it: Surface looks each spec.type up here. It is exported for code that wants a single widget without a whole panel. spec is the widget's entry of the layout ({ id, type, label, min, max, … }) and ctx is { page, out(name, value, info), haptic(ms) }, where out is where the widget's signals go and page is the page id used in their names. A widget's element is .el; put it in the page yourself.
Adding your own key here does not make a new layout type: validateLayout() only accepts the names in WIDGET_TYPES.
Type
Object
An object with the same 14 keys as WIDGET_TYPES, each a class: new WIDGET_CLASSES[type](spec, ctx).
Notes
Advanced. The keyboard widget wraps KeysPiano from @openav/keys.
Examples
js
import { WIDGET_CLASSES } from'@openav/surface';
const sent = [];
const ctx = { page: 'solo', out: (name, value) => sent.push(name + ' = ' + value), haptic() {} };
const toggle = new WIDGET_CLASSES.toggle({ id: 'mute', type: 'toggle', label: 'Mute' }, ctx);
document.getElementById('app').append(toggle.el);
toggle.set(true); // what a tap does
toggle.set(false);
document.getElementById('app').append(Object.assign(document.createElement('pre'), { textContent: sent.join('\n') }));
module A lone toggle on the page, and under it what it sent when it was switched on and off: surface/solo/mute = 1, then 0.
Related
SurfaceA TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
WIDGET_TYPESThe names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
The names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
Each widget publishes under surface/<page>/<id>, normalized 0..1; when its range is not 0..1 the unscaled value also goes out as …/raw. What each type sends:
text
fader, knob, number <id>
button <id> 1 while held, 0 on release
toggle <id> 1 or 0
encoder <id> (phase 0..1, wraps) and <id>/delta (turns)
xy <id>/x <id>/y <id>/down
bank <id>/1 … <id>/N (count, default 8)
radio <id> = index / (n - 1), raw = index
pads <id>/hit {pad,row,col,vel} and <id>/<n> while held
keyboard midi/note/on midi/note/off midi/cc/64
text <id> a string, as a pulse
label, meter nothing: they only receive feedback
Fields every widget accepts: id, type, label, min, max, def, step, unit, color (one of COLOR_NAMES) and, for hand-written layouts, target, the param key that routesFromLayout() turns into a route. Others by type: orient (v, h or auto) on a fader, spring: true on an XY pad, count and labels on a bank, options on a radio, rows, cols and labels on pads, pulse on a button.
Picks the widget type for one param declaration: a button for a pulse, a toggle, a radio, a fader or a knob.
The rules, first match wins: a surface: { type } hint on the param; pulse is a button; step: 1 over a range of 1 (two values) is a toggle; step: 1 over a range of 2 to 6 (three to seven values) is a radio; style'knob' or 'fader' as asked; otherwise a fader, or a knob when continuousCount is above 6. A missing min is 0 and a missing max is 1. autoSurface() calls it for every param.
Parameters
Name
Type
Description
p
Object
A param: { key, min?, max?, step?, pulse?, surface? }.
continuousCount
Number
How many continuous params the panel has in all. More than 6 turns faders into knobs. optionaldefault 0
style
'auto' | 'fader' | 'knob'
Force faders or knobs for the continuous params. optionaldefault 'auto'
module Prints hue → fader, mode → radio, mirror → toggle, burst → button, and a knob for hue once the panel has nine continuous params.
Related
autoSurface()Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
WIDGET_TYPESThe names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.
The desk reads the same objects the loop runs and owns nothing. From top to bottom it has the transport (play, previous and next scene, reset, the clock) and a scrubber with a block for every scene (with a tick for every cue, and a dashed end for a hold, when the scenes have them); with a score, the Director panel; the Layers overview (L1 input, L2 mapping, L3 world, L4 output); L1 · Input (MIDI devices, 🎤 mic, 🖐 hands, 🕺 body, leap); L2 · Mapping (the routes with a mute box and ✕, 🎛 map knobs in order, ⬇ export, ⬆ import, clear); L3 · Params (a slider per param with ✕ to clear the override and a learn chip; a yellow label means overridden); L4 · sound when there is a sound engine; and the Signals meters with the MIDI log. Every panel header collapses its section.
Keyboard, on the whole window: Space play and pause (at a hold: release it), → and ← next and previous scene (→ at a hold releases it), R back to the start, T performance mode (a fullscreen teleprompter for the performer: scene title, note and acts, the clock, the next scene, a HOLD banner), F fullscreen, Esc stops the mapping wizard or leaves keyboard-piano capture. Typing in an input field, a held key and any combination with ⌘, Ctrl or Alt are ignored. While the on-screen piano captures the keyboard the letters (R, T, F) belong to the piano; Space, the arrows and Esc always work.
Before you call it
Call desk.render(state) every frame after stage.frame() (a show's loop does): the desk only redraws when you tell it to. state is the resolved params, params.resolve(timeline.state()).
Parameters
Name
Type
Description
root
HTMLElement
The element to build the desk in. It gets the class oav-console, and the desk's styles are added once to the head.
app
Object
The objects the desk reads: { timeline, params, mapper, signals, stage } are needed, and midi, controllers, keys, sound, audio, hands, pose, leap, chord, remote, drums, loop, osc, score, director are optional. A missing part hides its section. createShow() passes show.app.
opts
Object
{ layers: false } and { signals: false } leave out the Layers and Signals panels; { learnOrder: ['hue', 'size'] } sets the order of the 🎛 map knobs in order wizard (default: the params as declared, skipping hidden ones and any with wizard: false). optionaldefault {}
Returns
Object{ render(state), perf, director, wizard, dispose() }: render redraws the desk, perf is the performance mode (toggle(), active, render()), director the Director panel (or null), wizard the knob-mapping wizard, and dispose() removes the keyboard handler and empties the element.
Notes
The keyboard handler is on window, so two desks on one page both answer the keys. dispose() removes it. The performance mode's teleprompter is appended to document.body, not to root.
Touch a slider (or a mapped control) and that param leaves the timeline's hands until you press its ✕: the score gives way when a person grabs a control.
module The desk appears in the page: a transport with a clock running, a scrubber with two scene blocks (Dawn, Noon), the layers overview and a slider each for Hue and Size. The Hue slider follows the timeline until you drag it.
js
import { Signals, Params } from'@openav/core';
import { Mapper } from'@openav/mapping';
import { Timeline } from'@openav/timeline';
import { Stage } from'@openav/stage';
import { mountConsole } from'@openav/console';
const signals = new Signals();
const params = new Params([{ key: 'hue', min: 0, max: 360, def: 200 }]);
const timeline = new Timeline({ params, total: 30 });
const app = { timeline, params, signals, mapper: new Mapper({ signals, params }), stage: new Stage({ container: document.createElement('div'), params, signals }) };
const desk = mountConsole(document.getElementById('app'), app, { layers: false, signals: false });
desk.render(params.resolve(timeline.state()));
console.log(desk.perf.active, desk.perf.toggle(), desk.perf.active); // false true true: the T key, in code
desk.perf.toggle(); // back to the desk
desk.dispose(); // removes the key handler and empties the element
console.log(document.getElementById('app').children.length); // 0
module A desk without the Layers and Signals panels: the console prints false true true as performance mode is switched on from code, then 0 after dispose() has emptied the element.
Related
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
TimelineAnswers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
MapperConnects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
LearnWizardBinds a list of params one after another: it arms learn for the first, you turn a knob, it binds it and arms the next. Skip, back and stop steer it.
DirectorRuns a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
MonitorFeedStreams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.
Where the backstage relay is unless told otherwise: the page's own host on port 7457 as a ws:// address, and null on an https page or where there is no page.
It is the default url of MonitorFeed. Not on an https page: the browser blocks ws:// there (mixed content, logged as an error), and a page published over https (openaudiovisual.com, a tunnel) has no relay beside it. A show that runs one behind TLS passes { url: 'wss://…' } explicitly.
Parameters
Name
Type
Description
loc
Object
Anything with protocol and hostname, like location. Pass a fake one to test. optionaldefault globalThis.location
Returns
String | null'ws://<hostname>:7457', or null when loc is missing or its protocol is https:.
module The console prints ws://192.168.1.20:7457, then null twice.
Related
MonitorFeedStreams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.
snapshotOf()Builds the usual snapshot of a show from its parts: clock, scene, world, frame rate, resolved params, overrides and every signal.
Streams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.
A show without a backstage is a rehearsal. Run the relay on the machine that serves the show (node packages/monitor/server.js, port 7457, which also serves the backstage page at / and GET /health), and a stage manager's phone or laptop on the same network opens http://<that machine>:7457/ to see the clock, the scene, the world, the frame rate, the overrides and every signal and param live, with a red "stage feed lost" banner when the stage stops sending for 4 seconds. The show page joins as ?role=stage; every other socket is a monitor. createShow() already creates and connects one, so you only build a MonitorFeed for a show you assembled by hand.
It is made to cost nothing when left on: with no relay it reconnects with a growing delay (about 5 seconds, growing by 1.6 each time up to 30), though the browser logs each refused connection, and that message is expected. frame() throttles itself, so call it every frame with a snapshot (see snapshotOf()).
Before you call it
Call connect() once (a show does), then frame(snapshot) every frame. frame() does nothing while connected is false. The relay must be reachable from the page: on an https page the browser blocks ws:// (mixed content), so give a wss:// address explicitly.
Parameters
Name
Type
Description
url
String | null
The relay address, such as 'ws://192.168.1.20:7457' (?role=stage is added for you). The default comes from defaultMonitorUrl(): the page's own host on port 7457, and null on an https page. With null, connect() does nothing. optionaldefault defaultMonitorUrl()
hz
Number
The most snapshots sent per second. optionaldefault 15
Open the socket. Does nothing without a url. If the connection drops or cannot be made, it tries again after a growing delay. The onStatus(ok) hook is called when it connects and when it drops.
Send a snapshot (anything that survives JSON.stringify; snapshotOf() builds the usual one). The message is { at: Date.now(), ...snapshot }. Throttled to hz: calls closer together than that are dropped, and nothing is sent while disconnected.
Notes
Fields to read and set: url (with ?role=stage appended, or null), hz, connected, ws, and the hook onStatus = (ok) => {}.
On a public https page a hand-assembled show that connects to ws://127.0.0.1:7457 meets Chrome's Local Network Access prompt (from Chrome 147 for WebSockets, per the package README): press Allow, or open the page from http://localhost.
Examples
js
import { MonitorFeed } from'@openav/monitor';
class FakeSocket { // stands in for the relay, so this runs anywhere
constructor(url) { console.log('connecting', url); setTimeout(() => this.onopen(), 0); }
send(text) { console.log('sent', Object.keys(JSON.parse(text)).join(' ')); }
}
globalThis.WebSocket = FakeSocket;
const monitor = new MonitorFeed({ url: 'ws://192.168.1.20:7457', hz: 15 });
monitor.onStatus = (ok) => console.log('connected', ok);
monitor.connect();
setTimeout(() => {
for (let i = 0; i < 5; i++) monitor.frame({ t: i, scene: 'Dawn' }); // five calls in the same instant
}, 20);
module The console prints connecting ws://192.168.1.20:7457?role=stage, connected true, and then only one sent at t scene: the other four calls came inside one throttle interval and were dropped.
js
import { Loop } from'@openav/core';
import { MonitorFeed, snapshotOf } from'@openav/monitor';
// a hand-assembled show: timeline, params, signals, stage and loop are yoursconst monitor = new MonitorFeed({ url: 'ws://192.168.1.20:7457' });
monitor.connect();
const loop = new Loop((dt) => {
timeline.advance(dt);
const state = stage.frame(dt, timeline.state());
monitor.frame(snapshotOf({ timeline, params, signals, stage, loop }, state)); // throttled for you
});
loop.start();
static With the relay running (node packages/monitor/server.js), a phone on the same network that opens http://<this machine>:7457/ sees the clock, the scene, the frame rate and every signal and param move.
To be confirmed
The relay itself (packages/monitor/server.js) and the backstage page were not run here; the examples use a stand-in socket.
Related
snapshotOf()Builds the usual snapshot of a show from its parts: clock, scene, world, frame rate, resolved params, overrides and every signal.
defaultMonitorUrl()Where the backstage relay is unless told otherwise: the page's own host on port 7457 as a ws:// address, and null on an https page or where there is no page.
createShow()Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
mountConsole()Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.
Builds the usual snapshot of a show from its parts: clock, scene, world, frame rate, resolved params, overrides and every signal.
The data that MonitorFeed.frame() sends and the backstage page draws. Each part is optional and read with a safe fallback, so a hand-assembled show can pass only what it has. The snapshot is plain JSON, so you can also log it, store it for a rehearsal recording or show it in your own page.
Parameters
Name
Type
Description
timeline
Timeline
The Timeline: gives t, playing, holding, total, sceneIndex and scene. optional
params
Params
The Params: gives the list of keys that are overridden. optional
signals
Signals
The Signals: every signal as { n, v, k } (name, value, kind). optional
stage
Stage
The Stage: gives the active world's name. optional
The resolved param values of this frame (what stage.frame() returns). {} when you pass nothing.
Returns
Object{ t, playing, holding, total, sceneIndex, scene, world, fps, state, overrides, signals }. scene is the current scene's title (or its id); holding is true while the show waits at a hold; signals is [{ n, v, k }] with numbers rounded to 3 decimals. A part you leave out gives zeros, empty strings and empty lists.
module The console prints the snapshot: t: 35, scene: 'noon' (a scene without a title shows its id), world: 'garden', fps: 60, state: { hue: 90 }, overrides: [ 'hue' ] and signals: [ { n: 'midi/cc/74', v: 0.457, k: 'continuous' } ].
Related
MonitorFeedStreams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.
TimelineAnswers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
ParamsThe values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
SignalsThe registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
StageRuns one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
LoopThe frame clock: calls your function every animation frame with the seconds since the last one, and keeps ticking when the tab is hidden.
The list of URLs loadKernel() tries for a kernel, in order: your urls, then window.__LAB__.vendor, then the unpkg default.
The order lets a page choose where its kernels come from without touching the code that uses them: pass urls for one page, set window.__LAB__.vendor for a whole host (the lab does, to serve the kernels itself), and the unpkg default is the last resort. Calling it yourself is useful to see where a page would look.
Parameters
Name
Type
Description
kind
'smlib' | 'manifold'
Which kernel.
opts
Object
urls: { smlib, manifold }, each a URL or a list of URLs. optionaldefault {}
Returns
string[] The URLs, first to try first, without repeats.
Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
Pure and synchronous, with no DOM. Duplicate vertices are merged first (within 1e-6 of the bounding-box diagonal), because without that every un-indexed mesh (three.js toNonIndexed(), an STL, any triangle soup) reads as open. In the report: triangles (after merging; a face that collapses is dropped), bbox as { min, max, size } in mm, volume in mm³ (always positive; a mesh wound inside out is measured the right way round), watertight (every edge used by exactly two faces, in opposite directions) with the counts openEdges, nonManifoldEdges and inconsistentEdges behind it, and grams: { PETG, PLA }, the weight of the volume as solid plastic (1.27 and 1.24 g/cm³).
overhang is { angle, area, ratio }: the area (mm²) of faces that look down steeper than angle allows, and its share of the whole surface. Faces that rest on the lowest 0.2 mm (the bed) do not count. minWall is { value, min, median, samples } or null: rays are shot into the part from sampled faces, and value is the 5th percentile of the thicknesses found, which is how thin most of the wall is. min is the single thinnest sample, which on a part with holes drilled at an angle is a knife edge and says little about the wall, so it has its own milder note.
warnings is a list of one sentence each, in Chinese: not watertight (with the three counts), overhang above overhangWarn, a wall thinner than minWall, a knife edge thinner than half of it, and a size beyond bed. An empty list means nothing to report. fab.check(x) is checkMesh(meshOf(x), opts).
Parameters
Name
Type
Description
input
TriMesh
A mesh { vertices, triangles, units? }: vertices a Float32Array or Float64Array of x, y, z, x, y, z …, triangles a Uint32Array with three vertex numbers per face, counter-clockwise seen from outside, Z up, in millimetres. Use meshOf() to get one from something else.
opts
Object
angle (45): the overhang angle in degrees. minWall (1.2): the thinnest wall allowed, in mm. bed ([320, 320, 325]): the size of the print bed in mm. wallSamples (2000): how many faces to measure the wall from; fewer is faster. overhangWarn (0.15): the share of overhanging area that earns a warning. weld (true): merge duplicate vertices first; false skips it. optionaldefault {}
Returns
Object{ units, triangles, bbox, volume, watertight, openEdges, nonManifoldEdges, inconsistentEdges, overhang, minWall, grams, warnings }. See the description.
Notes
grams is the weight of a solid part: it does not know about infill or wall count, so a sliced print weighs less.
module Prints closed: true, size 10 x 10 x 10 mm, volume 166.67 mm³, PLA 0.207 g and a wall of 3.33 mm.
js
import { checkMesh } from'@openav/fab';
// the same tetrahedron with its bottom face left out: a hole in the meshconst open = {
vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
triangles: new Uint32Array([0, 1, 3, 0, 3, 2, 1, 2, 3]),
units: 'mm',
};
const r = checkMesh(open, { bed: [5, 5, 5] }); // and a printer bed that is too small
document.getElementById('app').innerHTML = [
'watertight: ' + r.watertight + ', open edges: ' + r.openEdges,
'warnings: ' + r.warnings.length,
].join('<br>');
module Prints watertight: false with 3 open edges, and 2 warnings: the open mesh and the 10 mm part that does not fit a 5 mm bed.
js
import { checkMesh } from'@openav/fab';
// the same tetrahedron, but with every triangle listed again for each corner: 12 vertices, none shared (a "triangle soup")const tetra = [0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10];
const faces = [[0, 2, 1], [0, 1, 3], [0, 3, 2], [1, 2, 3]];
const soup = {
vertices: Float64Array.from(faces.flatMap((f) => f.flatMap((i) => tetra.slice(i * 3, i * 3 + 3)))),
triangles: Uint32Array.from({ length: 12 }, (_, i) => i),
units: 'mm',
};
document.getElementById('app').innerHTML = [
'merged first (default): ' + checkMesh(soup).watertight,
'weld: false: ' + checkMesh(soup, { weld: false }).watertight,
].join('<br>');
module Prints true with the default merge and false with weld: false: a soup of loose triangles looks open until its vertices are merged.
Related
FabChecks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.
weldMesh()Merges vertices that sit closer together than a tolerance and drops the faces that collapse; returns a new mesh with shared vertices.
meshOf()Turns whatever you made into a triangle mesh { vertices, triangles, units }: a mesh, a three.js object, a mesh-like object or a smlibwasm shape.
toStlBytes()Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.
Where the CAD kernels are loaded from when nothing else says so: smlibwasm 0.3.0 and manifold-3d 3.5.4, both from unpkg.
The last entry of candidateUrls(). Versions are pinned, so the kernels a page gets do not change under it. To use another copy, pass urls to Fab or loadKernel() rather than changing this object.
Exports a shape as STL, OBJ, GLB, STEP or DXF and resolves to { bytes, mime, ext, report }: the same as fab.export(), without the Fab class.
stl, obj and glb come from the mesh: toStlBytes(), toObjText(), toGlbBytes(). step and dxf come straight from the smlibwasm kernel (toStep, toDxf) and are refused for anything that is not a kernel shape, because a mesh has no curved surfaces to write; an error says so. A kernel solid's glb uses the kernel's toGlb when it has one. The report is made on the way, so a file and its check always agree.
Parameters
Name
Type
Description
x
Object
The shape: anything meshOf() accepts, and for step a smlibwasm Part, for dxf a smlibwasm Shape or Sketch.
fmt
String
'stl', 'obj', 'step', 'dxf' or 'glb', in any case. 'stp' means step and 'gltf' means glb. Another word throws an Error that lists the formats.
opts
Object
The options of meshOf() (scale, tessellator) and of checkMesh() (angle, minWall, bed …), and name, the object name of an OBJ file. optionaldefault {}
kernel
{ smlib }
{ smlib: () => Promise<module> }, where to get the smlibwasm module for step, dxf, and the glb of a Part. fab.export() passes its own. optionaldefault null
Returns
Promise<Object>bytes is a Uint8Array, mime is from MIME, ext is the extension without the dot ('step' even if you asked for 'stp'), and report is the checkMesh() report of the mesh (for step and dxf it can be null when the shape cannot be meshed, such as a 2D sketch).
module Two refusals, each with a message in Chinese: a mesh cannot become STEP (it has no curved surfaces), and "xyz" is not a format.
js
import { exportAs } from'@openav/fab';
// a stand-in for the smlibwasm kernel: STEP only works for a kernel solid, and the kernel writes itconst solid = { faces: () => [], bbox: () => ({}), tessellate: () => ({ positions: [0, 0, 0, 10, 0, 0, 0, 10, 0], indices: [0, 1, 2] }) };
const kernel = { smlib: async () => ({ toStep: () => 'ISO-10303-21;' }) };
const out = await exportAs(solid, 'stp', {}, kernel);
document.getElementById('app').textContent = out.ext + ' (' + out.mime + '): ' + new TextDecoder().decode(out.bytes);
module Prints "step (model/step): ISO-10303-21;": the alias stp was understood and the bytes came from the kernel's toStep.
Related
FabChecks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.
MIMEThe media type of each export format: MIME.stl is 'model/stl', and obj, step, dxf and glb the same way.
toStlBytes()Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.
toObjText()Writes a triangle mesh as OBJ text: welded vertices (v x y z), then faces (f a b c), one object, in millimetres.
toGlbBytes()Writes a triangle mesh as a minimal binary glTF (.glb): one mesh with welded positions and indices, in millimetres.
checkMesh()Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
loadKernel()Imports a CAD kernel (smlibwasm or manifold-3d) from the first URL that works, and resolves to the ready module.
Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.
Make one Fab and hand it whatever you made: fab.check(x) returns a manufacturability report (is it closed, how much material, how much overhang, how thin is the thinnest wall, how many grams), and await fab.export(x, 'stl') returns the file as bytes. x can be a plain triangle mesh { vertices, triangles }, a three.js Mesh, Object3D or BufferGeometry (recognised by its shape, three.js is never imported), or a smlibwasm Part or Shape. meshOf() says exactly what is accepted.
Formats: stl, obj and glb work for any input; step only for a smlibwasm Part, and dxf only for a smlibwasm Shape or Sketch, because a mesh has no curved-surface information. The CAD kernels are not bundled: the first time one is needed, fab.smlib() or fab.manifold() imports it from a URL (unpkg by default) and remembers it.
fab.save() writes the file through fab.saver if you set one (the lab's dev server does), and otherwise as a browser download. Zero runtime dependencies: nothing is fetched until you ask for a kernel.
Before you call it
Nothing for meshes and three.js objects. A smlibwasm Part has to be tessellated by the kernel: call await fab.smlib() once before the synchronous check(), or use checkAsync(), meshOfAsync() and export(), which load it for you. Loading needs a network, or urls / window.__LAB__.vendor pointing somewhere that has the kernel.
Parameters
Name
Type
Description
opts
Object
{ scale, urls, importer, download, context }, all optional. scale is millimetres per unit for a three.js object (10 means one three.js unit is 10 mm). urls is { smlib, manifold }, a URL or a list of them, to load the CAD kernels from. importer(url) replaces the dynamic import() that loads a kernel. download(bytes, info) replaces the browser download that save() falls back to. context() returns { id, params } for save() to put in info. optionaldefault {}
smlib(): a promise of the smlibwasm module, imported on the first call and remembered (the same promise comes back). A failed load is forgotten, so the next call tries again.
export(x, fmt, opts?): a promise of { bytes, mime, ext, report }; see exportAs(). It uses this Fab's scale and loads the kernel for step, dxf and the glb of a Part.
save(x, fmt, { name, params, note, … }?): export, then hand the bytes to fab.saver(bytes, info) if it is a function; info is { id, fmt, ext, mime, name, params, note, report }. If the saver is missing, throws, or returns nothing, the file is downloaded instead, through opts.download or an <a download> click. Returns what the saver returned, or { downloaded: true, file }, plus report.
Notes
Public fields: opts and saver (set it to async (bytes, info) => ({ file, … }); it starts as null). The file name for save() is name, else the id from context(), else fab.
Examples
js
import { Fab } from'@openav/fab';
// a tetrahedron: three 10 mm edges meeting at the origin (x, y, z), counter-clockwise faces seen from outsideconst mesh = {
vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
units: 'mm',
};
const report = new Fab().check(mesh);
document.getElementById('app').innerHTML = [
'watertight: ' + report.watertight,
'volume: ' + report.volume.toFixed(1) + ' mm³',
'PETG: ' + report.grams.PETG + ' g',
'overhang: ' + Math.round(report.overhang.ratio * 100) + '%',
'warnings: ' + report.warnings.length,
].join('<br>');
module Prints watertight: true, volume 166.7 mm³ (10 × 10 × 10 ÷ 6), 0.212 g of PETG, no overhang and no warnings.
module Prints the download of tetra.stl (the stand-in download replaces the browser's), then the saver's answer for tetra.obj.
To be confirmed
The STEP and DXF export, and the tessellation of a real smlibwasm Part, were not run for this entry: they need the real kernel from unpkg. The examples use a stand-in kernel to show the routing.
Related
checkMesh()Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
exportAs()Exports a shape as STL, OBJ, GLB, STEP or DXF and resolves to { bytes, mime, ext, report }: the same as fab.export(), without the Fab class.
meshOf()Turns whatever you made into a triangle mesh { vertices, triangles, units }: a mesh, a three.js object, a mesh-like object or a smlibwasm shape.
loadKernel()Imports a CAD kernel (smlibwasm or manifold-3d) from the first URL that works, and resolves to the ready module.
toStlBytes()Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.
Turns a mesh-like object (positions and indices under any of the usual names) into a triangle mesh, multiplying the positions by scale.
This is what meshOf() uses for a kernel's tessellation result, whichever way the kernel spells it. It throws an Error (in Chinese) when m is empty or has no positions. It does not check that the faces point outward or that the mesh is closed: that is checkMesh().
Parameters
Name
Type
Description
m
Object
An object with the points under positions, vertices, points or position, and the faces under indices, triangles, faces, index or triVerts. Points may be flat numbers, nested arrays ([[x, y, z], …]) or objects ({ x, y, z }). Without faces, every three points are one triangle. m.units is kept (default 'mm').
scale
Number
Multiplies every position, for example millimetres per unit. optionaldefault 1
Returns
TriMesh{ vertices: Float64Array, triangles: Uint32Array, units }.
Examples
js
import { fromMeshLike } from'@openav/fab';
// no faces given: every three points are one triangle; positions in centimetres, scaled to millimetresconst mesh = fromMeshLike({ positions: [0, 0, 0, 1, 0, 0, 0, 1, 0] }, 10);
document.getElementById('app').textContent =
'vertices: ' + [...mesh.vertices].join(', ') + ' | triangles: ' + [...mesh.triangles].join(', ') + ' | ' + mesh.units;
meshOf()Turns whatever you made into a triangle mesh { vertices, triangles, units }: a mesh, a three.js object, a mesh-like object or a smlibwasm shape.
meshOfAsync()Like meshOf(), but it waits for a tessellate() that returns a promise, and can load the smlibwasm kernel to tessellate a solid.
checkMesh()Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
Reads a binary STL file back into a triangle mesh: every face has its own three vertices, in 32-bit floats.
It reads what toStlBytes() writes, or any binary STL: the normals and the two attribute bytes are skipped, and the result is a triangle soup. Give it to checkMesh(), which merges the vertices itself, or to weldMesh(). The text (ASCII) form of STL, which starts with solid, is not supported and would read as garbage.
Parameters
Name
Type
Description
bytes
Uint8Array
The contents of a binary STL file. The face count is read from the header; the file's length is not checked against it.
import { toStlBytes, fromStlBytes, checkMesh } from'@openav/fab';
const tetra = {
vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
};
const back = fromStlBytes(toStlBytes(tetra)); // a round trip through the file format
document.getElementById('app').innerHTML = [
'faces: ' + back.triangles.length / 3 + ', vertices: ' + back.vertices.length / 3,
'still closed after the check merges them: ' + checkMesh(back).watertight,
].join('<br>');
module Prints 4 faces and 12 vertices (no sharing), and "true": the check welds them and finds the shape closed.
Related
toStlBytes()Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.
weldMesh()Merges vertices that sit closer together than a tolerance and drops the faces that collapse; returns a new mesh with shared vertices.
checkMesh()Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
Imports a CAD kernel (smlibwasm or manifold-3d) from the first URL that works, and resolves to the ready module.
The kernels are not bundled, because open-audiovisual has no dependencies and no build step: the first time a page needs one, it is imported from a URL. The order is candidateUrls(): opts.urls[kind], then window.__LAB__.vendor[kind], then unpkg (DEFAULT_URLS). If every URL fails it rejects with an Error (in Chinese) that names each URL and why it failed. It does not remember anything: fab.smlib() and fab.manifold() keep the promise for you, so call those unless you want to manage it yourself.
Before you call it
A network, or urls pointing at a place that serves the kernel (the lab uses window.__LAB__.vendor), or an importer that returns one.
Parameters
Name
Type
Description
kind
'smlib' | 'manifold'
Which kernel: 'smlib' for smlibwasm (solid modelling, STEP and DXF) or 'manifold' for manifold-3d (fast mesh booleans).
opts
Object
urls: { smlib, manifold }, a URL or a list of URLs for each kernel. importer: async (url) => module, to use instead of the dynamic import(). optionaldefault {}
Returns
Promise<Object> The smlibwasm module as it came, or the manifold-3d module with its WebAssembly started (setup() called).
Examples
js
import { loadKernel } from'@openav/fab';
const tried = [];
// a stand-in importer so nothing is fetched: the first URL fails, the second one worksconst importer = async (url) => {
tried.push(url);
if (url.includes('mirror-down')) thrownew Error('503');
return { name: 'manifold stand-in', default: async () => ({ setup() { tried.push('setup() called'); } }) };
};
const wasm = await loadKernel('manifold', { urls: { manifold: ['https://mirror-down.example/m.js', 'https://mirror.example/m.js'] }, importer });
document.getElementById('app').innerHTML = tried.join('<br>') + '<br>result has setup: ' + (typeof wasm.setup);
module Prints the two URLs in the order they were tried, then "setup() called": manifold's WebAssembly is started for you.
Turns whatever you made into a triangle mesh { vertices, triangles, units }: a mesh, a three.js object, a mesh-like object or a smlibwasm shape.
Everything is recognised by its shape and nothing is imported, so three.js and smlibwasm are never loaded for you. A three.js object has its matrixWorld applied (a mirrored object keeps its faces facing outward), indexed or not, and Y up is turned into Z up as (x, y, z) → (x, −z, y), then everything is multiplied by scale. A missing input, or something it cannot read, throws an Error whose message (in Chinese) says what to do.
It is synchronous, so it cannot wait for a kernel: a smlibwasm solid without opts.tessellator throws a message that tells you to load the kernel first, and a tessellate() that returns a promise throws too. Use meshOfAsync() for those.
Parameters
Name
Type
Description
x
Object
The thing to convert. Accepted, tried in this order: a mesh { vertices, triangles } (returned as it is); a three.js Mesh, Object3D (every visible mesh below it) or BufferGeometry; anything with a tessellate() method; an object with positions / vertices / points and, optionally, indices / triangles / faces / index; a smlibwasm solid or shape (given opts.tessellator).
opts
Object
scale: millimetres per unit for a three.js or mesh-like input (default 1). A mesh { vertices, triangles } is never scaled. tessellator(shape): a function that turns a smlibwasm shape into a mesh (the Fab class passes the kernel's once it is loaded). tessellate: an argument passed on to the shape's own tessellate(). optionaldefault {}
Returns
TriMesh{ vertices, triangles, units }: counter-clockwise faces seen from outside, Z up, millimetres.
Examples
js
import { meshOf } from'@openav/fab';
// what a three.js Mesh looks like to fab: a triangle in the XY plane, no index, an identity matrixWorldconst threeLike = {
isMesh: true,
geometry: { attributes: { position: { array: new Float32Array([0, 0, 0, 1, 0, 0, 0, 1, 0]), itemSize: 3, count: 3 } }, index: null },
matrixWorld: { elements: [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1] },
};
const m = meshOf(threeLike, { scale: 10 }); // one three.js unit is 10 mm
document.getElementById('app').textContent = 'vertices: ' + [...m.vertices].join(', ') + ' (' + m.units + ')';
module Prints "vertices: 0, 0, 0, 10, 0, 0, 0, 0, 10 (mm)": scaled by 10, and three.js's up (y = 1) became fab's up (z = 10).
js
import { meshOf } from'@openav/fab';
// a mesh-like object: nested points and faces, as some libraries hand them outconst m = meshOf({ points: [[0, 0, 0], [20, 0, 0], [0, 20, 0]], faces: [[0, 1, 2]] });
// the vertices come back flat, as numbers: x, y, z, x, y, z …
document.getElementById('app').textContent = [...m.vertices].join(' ') + ' | faces: ' + [...m.triangles].join(' ');
module Prints the nine numbers of the three corners and the face "0 1 2": nested arrays are flattened.
Related
meshOfAsync()Like meshOf(), but it waits for a tessellate() that returns a promise, and can load the smlibwasm kernel to tessellate a solid.
fromMeshLike()Turns a mesh-like object (positions and indices under any of the usual names) into a triangle mesh, multiplying the positions by scale.
checkMesh()Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
FabChecks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.
Like meshOf(), but it waits for a tessellate() that returns a promise, and can load the smlibwasm kernel to tessellate a solid.
Use it where the conversion may have to wait: a shape whose tessellate() is asynchronous, or a smlibwasm Part when the kernel is not loaded yet. With opts.kernel.smlib it awaits the module and uses its tessellate(part); an error says so if the module has none. Everything else is passed on to meshOf(). fab.meshOfAsync() fills in kernel for you.
The options of meshOf(), and kernel: { smlib: () => Promise<module> }, where to get the smlibwasm module from when a solid has to be tessellated and no tessellator was given. optionaldefault {}
Returns
Promise<TriMesh> The same mesh meshOf() would give.
module Prints "3 vertices, 1 face": the promise was awaited and the result turned into a mesh.
js
import { meshOfAsync } from'@openav/fab';
// a stand-in for the smlibwasm module: a solid has no tessellate() of its own, the module's function cuts itconst part = { faces: () => [], bbox: () => ({}) };
const kernel = { smlib: async () => ({ tessellate: (p) => ({ vertices: [0, 0, 0, 5, 0, 0, 0, 5, 0], triangles: [0, 1, 2] }) }) };
const mesh = await meshOfAsync(part, { kernel });
document.getElementById('app').textContent = 'tessellated by the kernel: ' + mesh.triangles.length / 3 + ' face';
module Prints "tessellated by the kernel: 1 face": the module was fetched through kernel.smlib() and its tessellate did the work.
Related
meshOf()Turns whatever you made into a triangle mesh { vertices, triangles, units }: a mesh, a three.js object, a mesh-like object or a smlibwasm shape.
FabChecks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.
loadKernel()Imports a CAD kernel (smlibwasm or manifold-3d) from the first URL that works, and resolves to the ready module.
The media type of each export format: MIME.stl is 'model/stl', and obj, step, dxf and glb the same way.
For the type of a Blob or the Content-Type of an upload when you handle the bytes yourself: new Blob([bytes], { type: MIME.stl }). exportAs() already puts the right one in its result.
Type
Object
{ stl: 'model/stl', obj: 'model/obj', step: 'model/step', dxf: 'image/vnd.dxf', glb: 'model/gltf-binary' }. Its keys are the formats exportAs() accepts.
Writes a triangle mesh as a minimal binary glTF (.glb): one mesh with welded positions and indices, in millimetres.
Vertices are welded first with weldMesh(). The file has POSITION and indices only: no normals, no materials, no colours, so a viewer shades it flat or works the normals out itself. The positions stay in millimetres and Z up, and asset.extras says { units: 'mm', up: 'Z' }. glTF itself assumes Y up, so a viewer that does not read the extras shows the part lying on its side.
await fab.export(part, 'glb') of a smlibwasm Part uses the kernel's own toGlb when it has one, and this function otherwise.
Parameters
Name
Type
Description
mesh
TriMesh
A mesh { vertices, triangles }, in millimetres.
Returns
Uint8Array A glTF 2.0 binary: the glTF header, one JSON chunk, one binary chunk with 32-bit float positions and 32-bit indices. The length is a multiple of 4.
Writes a triangle mesh as OBJ text: welded vertices (v x y z), then faces (f a b c), one object, in millimetres.
Vertices are welded first with weldMesh(), so a triangle soup comes out with shared vertices and a small file. There are no normals, no texture coordinates and no materials. await fab.export(x, 'obj', { name }) is the same thing with the conversions done for you.
Parameters
Name
Type
Description
mesh
TriMesh
A mesh { vertices, triangles }, in millimetres.
name
String
The object name on the o line. Spaces become underscores. optionaldefault 'fab'
Returns
String The text: a comment line, o <name>, one v line per welded vertex (6 decimals at most) and one f line per face, vertex numbers counting from 1.
Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.
Every face becomes its own 50 bytes: STL has no shared vertices. The normal comes from the order of the corners (right-hand rule), so counter-clockwise faces seen from outside give outward normals. Positions are written as they are, as float32: nothing is converted to millimetres, so give it a mesh that is already in mm. await fab.export(x, 'stl') is the same thing with the conversions and the check done for you.
Parameters
Name
Type
Description
mesh
TriMesh
A mesh { vertices, triangles }, in millimetres.
Returns
Uint8Array The file: an 80-byte header (the text open-audiovisual fab STL (mm)), the face count as a little-endian 32-bit integer, then per face the normal and three corners as 32-bit floats and two zero bytes.
Merges vertices that sit closer together than a tolerance and drops the faces that collapse; returns a new mesh with shared vertices.
A triangle soup, where every face has three vertices of its own (what an STL file or three.js toNonIndexed() gives), has no neighbours as far as topology goes: every edge looks open. Welding gives the faces shared vertices, so checkMesh() can see that they join. checkMesh(), toObjText() and toGlbBytes() call it themselves; toStlBytes() does not need it.
Parameters
Name
Type
Description
mesh
TriMesh
A mesh { vertices, triangles }. It is not changed.
tolerance
Number
Vertices closer than this (in each of x, y and z) become one, in the mesh's units. Left out: 1e-6 times the diagonal of the bounding box. optional
Returns
TriMesh{ vertices: Float64Array, triangles: Uint32Array, units } with each position once, and no face that has two corners on the same vertex.
module Prints "soup: 12 vertices, welded: 4 vertices, 4 faces".
Related
checkMesh()Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
fromStlBytes()Reads a binary STL file back into a triangle mesh: every face has its own three vertices, in 32-bit floats.
toObjText()Writes a triangle mesh as OBJ text: welded vertices (v x y z), then faces (f a b c), one object, in millimetres.
toGlbBytes()Writes a triangle mesh as a minimal binary glTF (.glb): one mesh with welded positions and indices, in millimetres.
Lists every MIDI controller profile that ships with the framework: id, name, maker, kind, control mix, how many controls are unverified, and its page URL.
No input: send "arguments": {}. It reads the profiles in packages/midi/profiles/, the same ones the controller pages and <oav-controller> use. Use it to find the id for read_profile, or to check that a controller is already covered before writing a new profile with scaffold_profile.
Returns
Object{ count, controllers } as JSON text. Each controller is { id, short, name, maker, kind, controls, unverified, url }: id such as arturia-minilab3, short the signal namespace (midi/<short>/…), kind'device' or 'generic', controls the mix as { knob: 8, pad: 8, keys: 25 } (a keyboard counts one per key), unverified how many of those controls still carry "verified": false, url like https://openaudiovisual.com/controllers/<id>/.
static The reply lists every shipped profile, starting with arturia-minilab3 (8 knobs, 4 faders, 8 pads, 25 keys …), each with its page URL.
Related
read_profileReads one controller profile in full, plus a readable MIDI map: for each control its type, the exact message, the signal name and whether the number is verified.
validate_profileChecks a controller profile with validateProfile() and against the shipped ones: same id, same short name, or port patterns that claim the same device.
scaffold_profileGenerates a valid skeleton profile for a new controller from maker, model and control counts. Every number is an unverified placeholder; nothing is written.
Lists the example works of this checkout: for each one its folder, its page title and the first comment line of its main.js.
The first call an agent makes in an unfamiliar checkout: what works already exist, so it can pick a donor for scaffold_world. It takes no input: send "arguments": {}.
Before it has a value
The server running: node packages/mcp/server.js, which Claude Code starts from the .mcp.json of the repository (server name openav). Other clients need the same command and args in their MCP settings. An MCP client sends initialize once before the first call. Over stdio every message is one line of JSON: the request bodies on this page are spread over several lines to be read, and a client writes them on one line for you.
Returns
Object[][{ dir, title, summary }], one per folder in examples/, as JSON text. dir is like examples/01-hello-particles, title is the <title> of its index.html, summary is the first line of its main.js that starts with // (the slashes taken off). A file that is missing gives an empty string.
static Send this to the server over stdio: the reply carries one text item, a JSON list of the examples with their folders, titles and one-line summaries.
Related
scaffold_worldCreates a new numbered example by copying a donor example's files; then the agent edits the World in main.js. This tool writes to disk.
read_docReturns the text of one framework document: the agent briefing, a handbook chapter or the README, chosen by name.
Returns the text of one framework document: the agent briefing, a handbook chapter or the README, chosen by name.
Input: doc (required), one of architecture, writing-a-world, signals, show-control, score, roadmap, agents, readme, introduction, getting-started, concepts, inputs, mapping, controllers, remote, packages, agents-guide, troubleshooting, lab-integration. The tool's own description tells an agent to start with agents (the conventions and the two unbreakable rules), then writing-a-world.
Most names are docs/<name>.md. Two are not, and they are easy to mix up: agents is AGENTS.md at the root of the repository (the briefing), agents-guide is docs/agents.md (the handbook chapter), and readme is README.md.
Returns
String The Markdown of the file, as it is on disk, in one text item: not JSON. The whole file comes back, which can be tens of kilobytes.
Notes
A name that is not in the list, or no doc at all, is an error with a plain Node message (The "path" argument must be of type string. Received undefined), not a list of the valid names. A client that follows the input schema cannot send one: the schema has the names as an enum.
Reads one controller profile in full, plus a readable MIDI map: for each control its type, the exact message, the signal name and whether the number is verified.
Input: id (required), a profile id or its short, for example arturia-minilab3 or minilab3. Where a control says verified: false, its number is a placeholder or a guess that no source has confirmed yet. An unknown id is an error that lists every id it knows. list_controllers shows what is there.
Returns
Object{ id, url, unverified, profile, mapping } as JSON text. profile is the JSON as shipped. mapping has one row per control: { control, type, message, signal, verified }, where message reads like CC 74, ch 1, note 36, ch 10 or notes 48–72 (25 keys), ch 1, and signal is midi/<short>/<id> (for a keyboard midi/<short>/<id>/on · /off · n<note>); a row may also carry led and note.
static Asks for the MiniLab 3 by its short name: the reply holds its full profile and a mapping whose first knob reads "CC …, ch 1" with the signal midi/minilab3/knob1.
Related
list_controllersLists every MIDI controller profile that ships with the framework: id, name, maker, kind, control mix, how many controls are unverified, and its page URL.
validate_profileChecks a controller profile with validateProfile() and against the shipped ones: same id, same short name, or port patterns that claim the same device.
scaffold_profileGenerates a valid skeleton profile for a new controller from maker, model and control counts. Every number is an unverified placeholder; nothing is written.
Runs the framework's whole test suite (node --test on every file in tests/), the same gate CI uses, and says whether it passed.
No input: send "arguments": {}. An agent calls it after every change to know whether the repository is still green. It runs node --test over every tests/*.test.js of the checkout the server lives in, waits for it to finish and gives up after 60 seconds.
Returns
Object When the run finishes cleanly: { pass, fail, ok } (numbers from the report, and ok true only when fail is 0), as JSON text. Otherwise { ok: false, output }, where output holds the lines of the report that say not ok or give the totals.
Notes
At the time of writing, in the checkout this was written from, the whole suite takes about 50 seconds and prints about 3.6 MB of report, while the tool reads at most 1 MiB of it. Past that the child process is stopped, and the tool answers { ok: false, output } with only the failing lines found in that first megabyte, even if every test passes. Until that changes, do not read ok: false with an empty output as a failing test: run node --test tests/*.test.js in a terminal.
static Runs the suite: the reply is { pass, fail, ok } when it ends cleanly, or { ok: false, output } with the failing lines. Allow about a minute.
Related
scaffold_worldCreates a new numbered example by copying a donor example's files; then the agent edits the World in main.js. This tool writes to disk.
validate_profileChecks a controller profile with validateProfile() and against the shipped ones: same id, same short name, or port patterns that claim the same device.
Generates a valid skeleton profile for a new controller from maker, model and control counts. Every number is an unverified placeholder; nothing is written.
Input, all required: maker (such as Arturia), model (such as MiniLab 3) and controls, a list of { type, count, from? } that describes the faceplate, for example [{"type":"knob","count":8},{"type":"pad","count":8},{"type":"keys","count":25,"from":48}]. type is one of knob, encoder, fader, pad, button, keys, wheel, strip; count is a whole number from 1 to 128; from is the lowest note of a keys section (default 48).
The skeleton passes validateProfile() and it is deliberately honest: every control has "verified": false, the source list holds one TODO entry, and the numbers are placeholders. Knobs, faders and strips get consecutive CC numbers from 20, pads get notes from 36 (skipping a keys range), buttons get momentary CCs, encoders get relative CCs, the first wheel is a pitch bend and any further wheel falls back to a CC. The id comes from maker and model (acme-pad-pro-16), short is the model without hyphens, and match.ports is a pattern made of the model's words.
Wrong input is an error with the reason: no maker or model, controls empty or not a list, an unknown type, a count outside 1..128, keys outside notes 0..127, or more controls than there are CC numbers (about 108) or pad notes. After that the agent sources every number from the maker's MIDI chart, runs validate_profile, and saves the file in packages/midi/profiles/; next_steps lists these.
Returns
Object{ profile, problems, unverified, next_steps } as JSON text, plus warnings when the id or short name collides with a shipped profile. profile is the skeleton, problems is what validateProfile() says about it (empty), unverified is how many controls are placeholders (all of them), next_steps is what to do to turn it into a real profile.
static Asks for a skeleton for an invented "Acme Pad Pro 16": the reply has id acme-pad-pro-16, 25 control entries (the keyboard is one entry of 25 keys, so unverified is 49), no problems, and the next steps.
Related
validate_profileChecks a controller profile with validateProfile() and against the shipped ones: same id, same short name, or port patterns that claim the same device.
read_profileReads one controller profile in full, plus a readable MIDI map: for each control its type, the exact message, the signal name and whether the number is verified.
list_controllersLists every MIDI controller profile that ships with the framework: id, name, maker, kind, control mix, how many controls are unverified, and its page URL.
Creates a new numbered example by copying a donor example's files; then the agent edits the World in main.js. This tool writes to disk.
Input: slug (required), a kebab-case name made of lower-case letters, digits and hyphens, such as jellyfish-bloom; and donor (optional, default 01-hello-particles), the name of a folder in examples/ to copy. Good donors: 01-hello-particles (a minimal canvas), 02-chord-garden (driven by chords), 03-pose-field (the camera), 05-prebiotic-flake (p5 and sound). The new folder is examples/<number>-<slug>, where the number is the highest one already in examples/ plus 1, with two digits (28).
It copies every file of the donor folder (the donors have no sub-folders, and a sub-folder would stop the copy with an error), so the new example starts as a working copy. A slug with anything else in it, or an unknown donor, or a folder that already exists is an error (slug must be kebab-case, unknown donor: …, already exists: …).
Returns
Object{ created, next_steps } as JSON text: created is the new folder (examples/28-jellyfish-bloom), next_steps is three lines: edit the World in main.js, edit the <title> and hint in index.html, then open it with node serve.js.
Notes
Send slug every time. It is marked required in the input schema, but the tool does not check that it is there: without it the folder is created with the name examples/<number>-undefined.
donor is used as a folder name without further checks, so give it a plain name from examples/.
Checks a controller profile with validateProfile() and against the shipped ones: same id, same short name, or port patterns that claim the same device.
Input: profile (required), the profile as an object or as a JSON string. A string that is not valid JSON does not raise an error: the answer is { ok: false, problems: ['not valid JSON: …'], conflicts: [] }. The checks cover the id and short name (lower-case, and short must not be one of the reserved generic names), a name and a kind (device or generic), a source for every device profile, the port regular expressions, the faceplate size, every control (a known type and message, cc or note in 0..127, two controls never on the same message, an encoder's relative mode) and that every control is shown on the faceplate.
The conflicts are what the validator cannot see by itself: id and short are taken by a shipped profile (the short becomes the signal namespace midi/<short>/…), and port means a regular expression in match.ports is identical to a shipped one or matches its name, so the framework could not tell the two devices apart when one is plugged in. A profile for a new device passes when ok is true; a clone of a shipped one comes back with all three kinds.
Returns
Object{ ok, problems, conflicts } as JSON text. problems is a list of sentences from validateProfile() (controls[0] (a): cc must be 0..127). conflicts is a list of { with, kind, detail }: with is the shipped profile, kind is 'id', 'short' or 'port'. ok is true only when both are empty.
static Sends a profile with several mistakes: the reply has ok: false and a list of problems, among them the id that is not lower-case and "cc must be 0..127".
Related
scaffold_profileGenerates a valid skeleton profile for a new controller from maker, model and control counts. Every number is an unverified placeholder; nothing is written.
read_profileReads one controller profile in full, plus a readable MIDI map: for each control its type, the exact message, the signal name and whether the number is verified.
list_controllersLists every MIDI controller profile that ships with the framework: id, name, maker, kind, control mix, how many controls are unverified, and its page URL.