open-audiovisual
Search GitHub

API reference

Look a name up

Every variable, function, event and option of oav-p5, and every export of every package. Names, signatures and source links are read from the code on each build; what is written about them is checked against the code in CI, so a name cannot be added, renamed or removed without this page noticing.

489 names · 489 described · oav-p5 0.8.4 · framework 0.1.0

One-page cheat sheet → · The same data for tools: reference.json · Adding a name? How to document it

MIDI: knobs, faders, pads, keys#

Read a controller like the mouse (knob1, notePressed), and the raw MIDI messages behind it. 39

oav-p5

controlChange()#

eventoav-p5
function controlChange(cc, value, channel) {}

You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.

Declare it the way you declare keyPressed():

js
function controlChange(cc, value, channel) {
  // a control moved
}

It hears every control, including the ones that are not a knob or a fader, such as the mod wheel and the pedals, which knobMoved() never reports. The latest value of every controller number is also in OAVp5.cc, so a sketch that only wants to look can read OAVp5.cc[1] in draw().

For one incoming message the order is: midiMessage(), then the specific one (noteOn(), controlChange(), pitchBend()), then the friendly one (padPressed(), knobMoved(), notePressed() and their partners).

Before it has a value

Defining the function is enough. oav-p5 asks the browser for Web MIDI when it loads (the default, ?oav_hardware=auto), and a sketch that defines it also gets a card if Web MIDI is missing or blocked. Keys from 🎹, a phone, OAVp5.input() and other programs reach it too. OAVp5.on('controlChange', fn) does the same without defining a global function, and returns a function that stops it.

Parameters

NameTypeDescription
ccNumberThe controller number, 0 to 127 (1 is the mod wheel, 64 the sustain pedal, 74 the first knob on a MiniLab 3).
valueNumberWhere the control is now, 0 to 1 (MIDI's 0 to 127 divided by 127).
channelNumberThe MIDI channel, 1 to 16.

Notes

  • An error inside it does not stop the music: the next message is still handled, and the message shows in the console and on a card beside the 🎹, with the file and line.

Examples

jsOpenProcessing ↗
let shown = 'move a knob, the mod wheel or a pedal';

function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(24);
}

function controlChange(cc, value, channel) {
  shown = 'CC ' + cc + ' = ' + nf(value, 1, 2) + '  (channel ' + channel + ')';
}

function draw() {
  background(20);
  fill(255);
  text(shown, width / 2, height / 2);
}
Shows the controller number, its value and the channel of whatever you moved last. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
jsOpenProcessing ↗
let sustain = false;

function setup() {
  createCanvas(400, 300);
}

function controlChange(cc, value) {
  if (cc === 64) sustain = value >= 0.5;   // the sustain pedal: 64 and up counts as down
}

function mousePressed() {
  OAVp5.input([0xB0, 64, 127]);   // no pedal? press the mouse
}

function mouseReleased() {
  OAVp5.input([0xB0, 64, 0]);
}

function draw() {
  background(sustain ? 90 : 20);
}
The background lights up while the sustain pedal is down. No pedal? Hold the mouse button: it sends the same message.

Related

  • knobMoved() You define this function; oav-p5 calls it whenever a knob moves.
  • faderMoved() You define this function; oav-p5 calls it whenever a fader moves.
  • OAVp5.cc An array of 128 numbers, one per MIDI controller number: where each control last was, 0 to 1. OAVp5.cc[1] is the mod wheel, OAVp5.cc[64] the sustain pedal.
  • midiMessage() You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
  • OAVp5.on() Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
  • OAVp5.input() Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.

p5/oav-p5.js, line 255

fader()#

functionoav-p5
fader(n)

Returns the position of fader number n, from 0 to 1: the same as fader1 to fader8, for any n.

The function form of fader1, for when the fader's number is a variable.

Parameters

NameTypeDescription
nNumberWhich fader, counting from 1.

Returns

Number The fader's position, 0..1; 0 until it has moved.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  for (let i = 1; i <= 4; i++) circle(i * 80, height - 40 - fader(i) * 200, 30);
}
Four dots that rise with four faders.

Related

  • fader1 The position of a controller fader (slider), from 0 to 1: fader1 is the first fader, up to fader8.
  • knob() Returns the position of knob number n, from 0 to 1: the same as knob1 to knob8, for any n.
  • pad() Returns how hard pad number n is pressed, 0 to 1: the same as pad1 to pad16, for any n.

p5/oav-p5.js, line 361

fader1#

variableoav-p5
fader1

Also: fader2 fader3 fader4 fader5 fader6 fader7 fader8

The position of a controller fader (slider), from 0 to 1: fader1 is the first fader, up to fader8.

Works exactly like knob1, for the controller's faders, numbered in the order the controller's profile lists them (as printed on the device). A controller with only four faders, like the MiniLab 3, has fader1 to fader4.

Type
Number
Range
0..1

Changes when the fader moves; 0 until it has moved.

Before it has a value

Nothing to call. Web MIDI is asked for at load, exactly as for a knob (knob1); reading a fader also makes a missing or refused Web MIDI show as a card.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  rect(40, height - 40, fader1 * (width - 80), 24);   // fader 1 is a level bar
}
A bar that follows fader 1. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.

Related

  • fader() Returns the position of fader number n, from 0 to 1: the same as fader1 to fader8, for any n.
  • knob1 The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
  • faderMoved() You define this function; oav-p5 calls it whenever a fader moves.

p5/oav-p5.js, line 1

faderMoved()#

eventoav-p5
function faderMoved(n, value) {}

You define this function; oav-p5 calls it whenever a fader moves.

The fader twin of knobMoved(): function faderMoved(n, value) { … }.

Parameters

NameTypeDescription
nNumberWhich fader (1 is the first).
valueNumberWhere it is now, 0..1.

Examples

jsOpenProcessing ↗
let level = 0;

function setup() {
  createCanvas(400, 300);
}

function faderMoved(n, value) {
  if (n === 1) level = value;   // fader 1 only
}

function draw() {
  background(20);
  fill(255);
  rect(40, height - 40, level * (width - 80), 24);
}
A bar that follows fader 1, driven by the event instead of a read.

Related

  • fader1 The position of a controller fader (slider), from 0 to 1: fader1 is the first fader, up to fader8.
  • knobMoved() You define this function; oav-p5 calls it whenever a knob moves.
  • controlChange() You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.

p5/oav-p5.js, line 254

knob()#

functionoav-p5
knob(n)

Returns the position of knob number n, from 0 to 1: the same as knob1 to knob8, for any n.

Use it when the number of the knob is a variable: for (let i = 1; i <= 16; i++) bar(i, knob(i));. It also works in a sketch that has its own knob1, which oav-p5 then leaves alone.

Parameters

NameTypeDescription
nNumberWhich knob, counting from 1 (as printed on the controller).

Returns

Number The knob's position, 0..1; 0 until it has been turned.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
}

function draw() {
  background(20);
  fill(255);
  for (let i = 1; i <= 8; i++) {
    rect(i * 44 - 20, height - 20, 30, -10 - knob(i) * 250);   // one bar per knob
  }
}
Eight bars, one per knob. Knobs you have not touched stay at the bottom.

Related

  • knob1 The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
  • fader() Returns the position of fader number n, from 0 to 1: the same as fader1 to fader8, for any n.
  • pad() Returns how hard pad number n is pressed, 0 to 1: the same as pad1 to pad16, for any n.

p5/oav-p5.js, line 360

knob1#

variableoav-p5
knob1

Also: knob2 knob3 knob4 knob5 knob6 knob7 knob8

The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.

Read it in draw() the way you read mouseX: background(knob1 * 255). Knobs are numbered the way they are printed on the controller, so knob 1 is knob 1 on any controller OAV has a profile for: the MiniLab 3's first knob is CC 74, the LPD8's is CC 70. The sketch never needs to know a CC number.

On a controller OAV has no profile for, the first knob you turn becomes knob1, the next one knob2, and so on, in the order you turn them. Past eight, use knob(n).

Type
Number
Range
0..1

Changes when the knob moves, not once per frame; the sketch just reads the latest value. 0 until the knob has been turned.

Before it has a value

Nothing to call. With the default hardware=auto, oav-p5 asks the browser for Web MIDI as soon as it loads, whatever the sketch reads (Chrome, Edge and Firefox have it; Safari and iPad do not: use 🎹 or a phone there). Reading a knob also tells oav-p5 the sketch plays from MIDI, so when Web MIDI is missing or refused a card beside the 🎹 says so; a sketch that uses none of this never sees that card. ?oav_hardware=click waits for the 🎹 panel instead.

Notes

  • A sketch that already has its own knob1 (a let, a function) keeps it, and a card beside the 🎹 says what to read instead (knob(1)).
  • With OP Configurator, knobs also drive the OPC sliders in the order they are declared. A knob the sketch reads itself stays the sketch's: that slider no longer follows it.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(knob1 * 255);   // knob 1 sets the brightness
}
Turn knob 1 and the whole canvas goes from black to white. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  colorMode(HSB, 1);
}

function draw() {
  background(0.65, 0.5, 0.15);
  fill(knob2, 0.8, 1);                        // knob 2: colour
  circle(width / 2, height / 2, 40 + knob1 * 300);   // knob 1: size
}
Two knobs, two jobs: size and colour.

Related

  • knob() Returns the position of knob number n, from 0 to 1: the same as knob1 to knob8, for any n.
  • fader1 The position of a controller fader (slider), from 0 to 1: fader1 is the first fader, up to fader8.
  • pad1 How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.
  • knobMoved() You define this function; oav-p5 calls it whenever a knob moves.
  • noteIsDown() True while the key n (a MIDI number or a name like 'C4') is held down.

p5/oav-p5.js, line 288

knobMoved()#

eventoav-p5
function knobMoved(n, value) {}

You define this function; oav-p5 calls it whenever a knob moves.

Like mouseMoved() for knobs:

js
function knobMoved(n, value) {
  // knob n is now at value
}

It is called for every step the knob sends, so many times per second while it turns. To read a knob once a frame, use knob1 instead.

Parameters

NameTypeDescription
nNumberWhich knob (1 is the first, as printed on the controller).
valueNumberWhere it is now, 0..1.

Examples

jsOpenProcessing ↗
let last = 'turn a knob';

function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(32);
}

function knobMoved(n, value) {
  last = 'knob ' + n + ': ' + nf(value, 1, 2);
}

function draw() {
  background(20);
  fill(255);
  text(last, width / 2, height / 2);
}
Shows which knob moved and where it is. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.

Related

  • knob1 The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
  • faderMoved() You define this function; oav-p5 calls it whenever a fader moves.
  • controlChange() You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.

p5/oav-p5.js, line 254

lastNoteFrameCount#

variableoav-p5
lastNoteFrameCount

The frameCount of the frame that saw the last key go down: frameCount - lastNoteFrameCount is how many frames ago.

The frame-counting twin of lastNoteTs: 0 in the first frame after the key, then 1, 2, 3 …

Type
Number

A frame number, on the same count as frameCount, set with note; -Infinity before any key. Since oav-p5 0.8.3.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
}

function draw() {
  background(20);
  const f = frameCount - lastNoteFrameCount;    // frames since a key went down
  fill(120, 190, 255);
  circle(200, 150, max(0, 200 - f * 5));        // shrinks by 5 px a frame
}
Each key you press (on 🎹 too) brings back a big circle that shrinks 5 px a frame.

Related

  • lastNoteTs When the last key went down, on the millis() clock: millis() - lastNoteTs is how many ms ago.
  • note The MIDI number (0 to 127) of the last key pressed.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • lastKickFrameCount The frameCount of the frame that saw the last kick (snare, hat, beat): frameCount - lastKickFrameCount is how many frames ago.

p5/oav-p5.js, line 333

lastNoteTs#

variableoav-p5
lastNoteTs

When the last key went down, on the millis() clock: millis() - lastNoteTs is how many ms ago.

Any key counts: a MIDI keyboard, the 🎹 on screen, a phone. Releasing a key does not change it.

Type
Number

Milliseconds on the same clock as millis(), set with note and velocity; -Infinity before any key. Since oav-p5 0.8.2.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
}

function draw() {
  background(20);
  const ago = millis() - lastNoteTs;    // ms since a key went down
  fill(255, 200 * exp(-ago / 400) + 30);
  circle(200, 150, 60 + 160 * exp(-ago / 400));
}
Each key you press (on 🎹 too) makes the circle bright and big, then it dims.

Related

  • note The MIDI number (0 to 127) of the last key pressed.
  • velocity How hard the last key was pressed, from 0 to 1.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • lastKickTs When the last kick (snare, hat, beat) came, on the millis() clock: millis() - lastKickTs is how many ms ago.

p5/oav-p5.js, line 330

midiMessage()#

eventoav-p5
function midiMessage(bytes, source) {}

You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.

Declare it the way you declare keyPressed():

js
function midiMessage(bytes, source) {
  // anything that came in
}

It is the widest net: it is called first for every note, control change and pitch bend, and it is the only function that hears the messages oav-p5 has no friendlier name for, such as channel pressure (0xD0), polyphonic aftertouch (0xA0) and program change (0xC0). The OAVp5.last object keeps the latest note, control change or pitch bend, but not those.

It does not hear system messages: the MIDI clock, start, stop and song position go to OAVp5.clock, and system-exclusive messages are dropped. A note-on with velocity 0 arrives here as it was sent (a note-on with 0), while noteOff() is the one that treats it as a release.

For one incoming message the order is: midiMessage(), then the specific one (noteOn(), controlChange(), pitchBend()), then the friendly one (padPressed(), knobMoved(), notePressed() and their partners).

Before it has a value

Defining the function is enough. oav-p5 asks the browser for Web MIDI when it loads (the default, ?oav_hardware=auto), and a sketch that defines it also gets a card if Web MIDI is missing or blocked. Keys from 🎹, a phone, OAVp5.input() and other programs reach it too. OAVp5.on('midiMessage', fn) does the same without defining a global function, and returns a function that stops it.

Parameters

NameTypeDescription
bytesArray<Number>The message as three numbers [status, data1, data2]. The status byte holds the kind in its top half and the channel in its bottom half (0x90 is a note-on on channel 1, 0x93 on channel 4). The data bytes are 0 to 127, and a byte the message does not have is 0.
sourceStringWhere it came from: 'hardware' (a real controller), 'screen' (the 🎹), 'phone', 'virtual-out' (a program playing into OAV Virtual MIDI), 'link' (another page in the same room), or whatever label you gave OAVp5.input().

Notes

  • An error inside it does not stop the music: the next message is still handled, and the message shows in the console and on a card beside the 🎹, with the file and line.

Examples

jsOpenProcessing ↗
let shown = 'play something';

function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(20);
}

function midiMessage(bytes, source) {
  shown = bytes.map((b) => '0x' + b.toString(16).toUpperCase()).join(' ') + '  from ' + source;
}

function draw() {
  background(20);
  fill(255);
  text(shown, width / 2, height / 2);
}
Prints the raw bytes of the last message and where it came from. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
jsOpenProcessing ↗
let pressure = 0;

function setup() {
  createCanvas(400, 300);
}

function midiMessage(bytes) {
  if ((bytes[0] & 0xF0) === 0xD0) pressure = bytes[1] / 127;   // channel pressure (aftertouch)
}

function mouseMoved() {
  OAVp5.input([0xD0, constrain(round(mouseX / width * 127), 0, 127)], 'mouse');   // no aftertouch? move the mouse
}

function draw() {
  background(20);
  circle(width / 2, height / 2, 40 + pressure * 260);
}
The circle follows aftertouch, which no friendlier function reports. No keyboard with aftertouch? Move the mouse across the canvas: it sends the same message.

Related

  • noteOn() You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
  • controlChange() You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.
  • pitchBend() You define this function; oav-p5 calls it whenever the pitch wheel moves, with its position from −1 to 1.
  • OAVp5.last The latest note, control change or pitch bend that arrived: its type, number, value, channel, where it came from and which control it is. null before any.
  • OAVp5.on() Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
  • OAVp5.input() Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.

p5/oav-p5.js, line 255

note#

variableoav-p5
note

The MIDI number (0 to 127) of the last key pressed.

Middle C is 60, A4 is 69. noteName(note) writes the number as a name (60 → 'C4') and noteFreq(note) as a frequency in Hz. velocity says how hard it was pressed. For every key, not just the last, define notePressed().

Type
Number
Range
0..127

Set when a key goes down and kept after it is released; 0 before any key has been pressed.

Before it has a value

Nothing to call. Web MIDI is asked for at load (see knob1). Unlike a knob, merely reading note does not tell oav-p5 the sketch plays from MIDI: defining notePressed() or reading a knob or pad does, which is what makes a missing or refused Web MIDI show as a card. 🎹 and a phone always work.

Notes

  • A sketch with its own note (a let note, a function) keeps it; read midi.last.note instead. oav-p5 says so in a card the first time.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(64);
}

function draw() {
  background(20);
  fill(255);
  text(noteName(note), width / 2, height / 2);   // the last key, as a name
}
Shows the name of the last key you played. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.

Related

  • velocity How hard the last key was pressed, from 0 to 1.
  • noteIsPressed True while at least one key is held down.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • noteName() Turns a MIDI number into a note name: noteName(61) is 'C#4'.

p5/oav-p5.js, line 326

noteIsDown()#

functionoav-p5
noteIsDown(n)

True while the key n (a MIDI number or a name like 'C4') is held down.

The MIDI version of keyIsDown(). noteIsDown('C4') is the same as noteIsDown(60): C4 is middle C. A name that is not a note ('H4', 'C-2') stops the sketch with a one-line card saying how to write it.

Parameters

NameTypeDescription
nNumber | StringThe key: a MIDI number (60) or a note name ('C4', 'c#4', 'Db3').

Returns

Boolean True if that key is down right now.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(noteIsDown('C4') ? color(255, 211, 107) : 80);
  circle(width / 2, height / 2, 140);   // lights up while middle C is held
}
A circle that lights up only while C4 (middle C) is held. On the on-screen keyboard, middle C is the C labelled C4.

Related

  • noteIsPressed True while at least one key is held down.
  • noteNumber() Turns a note name like 'C4' into its MIDI number, 0 to 127.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.

p5/oav-p5.js, line 363

noteIsPressed#

variableoav-p5
noteIsPressed

True while at least one key is held down.

To ask about one particular key, use noteIsDown(). midi.held lists every key held right now, lowest first.

Type
Boolean

Like keyIsPressed, for MIDI keys. False when nothing is held.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(noteIsPressed ? 90 : 20);   // lighter while any key is down
}
The background lightens while any key is held.

Related

  • noteIsDown() True while the key n (a MIDI number or a name like 'C4') is held down.
  • note The MIDI number (0 to 127) of the last key pressed.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.

p5/oav-p5.js, line 326

noteOff()#

eventoav-p5
function noteOff(note, channel) {}

You define this function; oav-p5 calls it for every MIDI note-off, with the key and the channel.

The partner of noteOn():

js
function noteOff(note, channel) {
  // a key or pad came up
}

It is called for a note-off message and also for a note-on with velocity 0, which many keyboards send instead. There is no velocity: oav-p5 does not pass the release velocity.

For one incoming message the order is: midiMessage(), then the specific one (noteOn(), controlChange(), pitchBend()), then the friendly one (padPressed(), knobMoved(), notePressed() and their partners).

Before it has a value

Defining the function is enough. oav-p5 asks the browser for Web MIDI when it loads (the default, ?oav_hardware=auto), and a sketch that defines it also gets a card if Web MIDI is missing or blocked. Keys from 🎹, a phone, OAVp5.input() and other programs reach it too. OAVp5.on('noteOff', fn) does the same without defining a global function, and returns a function that stops it.

Parameters

NameTypeDescription
noteNumberThe MIDI number of the key that came up, 0 to 127.
channelNumberThe MIDI channel, 1 to 16.

Notes

  • An error inside it does not stop the music: the next message is still handled, and the message shows in the console and on a card beside the 🎹, with the file and line.

Examples

jsOpenProcessing ↗
const down = {};
let held = 'play a key and let it go';

function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(24);
}

function noteOn(note) { down[note] = millis(); }

function noteOff(note) {
  if (down[note] !== undefined) {
    held = noteName(note) + ' was held ' + round(millis() - down[note]) + ' ms';
    delete down[note];
  }
}

function draw() {
  background(20);
  fill(255);
  text(held, width / 2, height / 2);
}
Pairs each noteOn() with its noteOff() and says how long the key was held. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.

Related

  • noteOn() You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
  • noteReleased() You define this function; oav-p5 calls it each time a key (or a pad) is let go.
  • padReleased() You define this function; oav-p5 calls it each time a drum pad is let go.
  • midiMessage() You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
  • OAVp5.on() Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.

p5/oav-p5.js, line 255

noteOn()#

eventoav-p5
function noteOn(note, velocity, channel) {}

You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.

Declare it the way you declare keyPressed():

js
function noteOn(note, velocity, channel) {
  // a key or pad went down
}

It is called once per note-on message, as it came in. It is the level below notePressed(): the same moment, but with the channel, and for pads too, even when the sketch defines padPressed(). A note-on with velocity 0, which some keyboards send instead of a note-off, is not a note-on here: it calls noteOff().

For one incoming message the order is: midiMessage(), then the specific one (noteOn(), controlChange(), pitchBend()), then the friendly one (padPressed(), knobMoved(), notePressed() and their partners).

Before it has a value

Defining the function is enough. oav-p5 asks the browser for Web MIDI when it loads (the default, ?oav_hardware=auto), and a sketch that defines it also gets a card if Web MIDI is missing or blocked. Keys from 🎹, a phone, OAVp5.input() and other programs reach it too. OAVp5.on('noteOn', fn) does the same without defining a global function, and returns a function that stops it.

Parameters

NameTypeDescription
noteNumberThe MIDI number of the key, 0 to 127 (60 is middle C). noteName(note) turns it into a name.
velocityNumberHow hard, above 0 and up to 1 (MIDI's 1 to 127 divided by 127).
channelNumberThe MIDI channel, 1 to 16. Pads on the MiniLab 3 and the LPD8 send on channel 10.

Notes

  • An error inside it does not stop the music: the next message is still handled, and the message shows in the console and on a card beside the 🎹, with the file and line.

Examples

jsOpenProcessing ↗
let shown = 'play a key';

function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(24);
}

function noteOn(note, velocity, channel) {
  shown = noteName(note) + '  velocity ' + nf(velocity, 1, 2) + '  channel ' + channel;
}

function draw() {
  background(20);
  fill(255);
  text(shown, width / 2, height / 2);
}
Shows the key you played, how hard and on which channel. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
jsOpenProcessing ↗
let hits = [];

function setup() {
  createCanvas(400, 300);
  noStroke();
}

function noteOn(note, velocity, channel) {
  hits.push({ note, velocity, pad: channel === 10, born: millis() });   // channel 10: the pads
}

function draw() {
  background(20);
  hits = hits.filter((h) => millis() - h.born < 1500);
  for (const h of hits) {
    const fade = 1 - (millis() - h.born) / 1500;
    const x = map(h.note, 24, 96, 20, width - 20);
    fill(h.pad ? color(255, 90, 31, fade * 255) : color(120, 190, 255, fade * 255));
    if (h.pad) square(x - 25, 125, 50 + h.velocity * 40);
    else circle(x, 150, 20 + h.velocity * 80);
  }
}
Keys leave blue circles and pads leave orange squares, placed by pitch, bigger for harder hits, fading after a moment. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.

Related

  • noteOff() You define this function; oav-p5 calls it for every MIDI note-off, with the key and the channel.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • padPressed() You define this function; oav-p5 calls it each time a drum pad is hit.
  • midiMessage() You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
  • OAVp5.on() Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
  • OAVp5.input() Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.

p5/oav-p5.js, line 255

notePressed()#

eventoav-p5
function notePressed(note, velocity) {}

You define this function; oav-p5 calls it each time a key (or a pad) goes down.

Declare it the way you declare keyPressed():

js
function notePressed(note, velocity) {
  // a key went down
}

It is called once per key, with the key and how hard. A pad also calls it, unless the sketch defines padPressed().

Before it has a value

Defining it is enough: oav-p5 calls it from every source, and if Web MIDI is missing or refused it says so in a card beside the 🎹 (Web MIDI itself is asked for at load, see knob1). Keys from 🎹 and from a phone call it too.

Parameters

NameTypeDescription
noteNumberThe MIDI number of the key (60 is middle C). noteName(note) turns it into a name.
velocityNumberHow hard, 0..1.

Notes

  • An error inside it does not stop the music: the next key still plays, and the message shows in the console and on a card beside the 🎹, with the file and line.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  background(20);
  noStroke();
}

function notePressed(note, velocity) {
  fill(random(255), random(255), random(255), 200);
  circle(map(note, 36, 96, 20, width - 20), height / 2, 10 + velocity * 120);
}
Every key leaves a circle: low keys on the left, high on the right, harder keys bigger.

Related

  • noteReleased() You define this function; oav-p5 calls it each time a key (or a pad) is let go.
  • padPressed() You define this function; oav-p5 calls it each time a drum pad is hit.
  • noteOn() You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
  • note The MIDI number (0 to 127) of the last key pressed.
  • noteName() Turns a MIDI number into a note name: noteName(61) is 'C#4'.

p5/oav-p5.js, line 254

noteReleased()#

eventoav-p5
function noteReleased(note) {}

You define this function; oav-p5 calls it each time a key (or a pad) is let go.

The partner of notePressed(), like keyReleased() to keyPressed():

js
function noteReleased(note) {
  // a key came up
}

Parameters

NameTypeDescription
noteNumberThe MIDI number of the key that was released.

Examples

jsOpenProcessing ↗
let held = {};

function setup() {
  createCanvas(400, 300);
}

function notePressed(note) { held[note] = millis(); }
function noteReleased(note) { delete held[note]; }

function draw() {
  background(20);
  fill(255);
  text(Object.keys(held).map((n) => noteName(+n)).join('  '), 20, 40);   // the keys down now
}
Lists the names of the keys that are down right now.

Related

  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • padReleased() You define this function; oav-p5 calls it each time a drum pad is let go.
  • noteOff() You define this function; oav-p5 calls it for every MIDI note-off, with the key and the channel.
  • noteIsDown() True while the key n (a MIDI number or a name like 'C4') is held down.

p5/oav-p5.js, line 254

pad()#

functionoav-p5
pad(n)

Returns how hard pad number n is pressed, 0 to 1: the same as pad1 to pad16, for any n.

The function form of pad1.

Parameters

NameTypeDescription
nNumberWhich pad, counting from 1.

Returns

Number The velocity while the pad is held (0..1), 0 when it is released.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
}

function draw() {
  background(20);
  for (let i = 1; i <= 8; i++) {
    fill(255, 90, 31, 60 + pad(i) * 195);
    square(20 + ((i - 1) % 4) * 92, 40 + floor((i - 1) / 4) * 110, 80);
  }
}
A grid of eight squares that light up under the pads you hit.

Related

  • pad1 How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.
  • knob() Returns the position of knob number n, from 0 to 1: the same as knob1 to knob8, for any n.
  • padPressed() You define this function; oav-p5 calls it each time a drum pad is hit.

p5/oav-p5.js, line 362

pad1#

variableoav-p5
pad1

Also: pad2 pad3 pad4 pad5 pad6 pad7 pad8 pad9 pad10 pad11 pad12 pad13 pad14 pad15 pad16

How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.

Read it like a held mouse button that also knows how hard: if (pad1 > 0) fill(255, 90, 31). Pads are numbered in the order the controller's profile lists them. To react once per hit rather than ask every frame, define padPressed().

Type
Number
Range
0..1

The velocity of the hit while the pad is held (1 is as hard as the controller sends), and 0 as soon as it is released.

Before it has a value

Nothing to call. Web MIDI is asked for at load, exactly as for a knob (knob1); reading a pad also makes a missing or refused Web MIDI show as a card.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255, 90, 31);
  circle(width / 2, height / 2, 40 + pad1 * 260);   // swells while pad 1 is held
}
The circle swells while pad 1 is down, more for a harder hit. On the on-screen controller, click the first pad.

Related

  • pad() Returns how hard pad number n is pressed, 0 to 1: the same as pad1 to pad16, for any n.
  • padPressed() You define this function; oav-p5 calls it each time a drum pad is hit.
  • padReleased() You define this function; oav-p5 calls it each time a drum pad is let go.
  • knob1 The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.

p5/oav-p5.js, line 1

padPressed()#

eventoav-p5
function padPressed(n, velocity) {}

You define this function; oav-p5 calls it each time a drum pad is hit.

js
function padPressed(n, velocity) {
  // pad n was hit
}

When the sketch defines padPressed(), a pad calls it and not notePressed(); without it, a pad calls notePressed() like any key.

Parameters

NameTypeDescription
nNumberWhich pad (1 is the first).
velocityNumberHow hard, 0..1.

Examples

jsOpenProcessing ↗
let hits = [];

function setup() {
  createCanvas(400, 300);
  noStroke();
}

function padPressed(n, velocity) {
  hits.push({ x: 40 + ((n - 1) % 4) * 105, y: 70 + floor((n - 1) / 4) * 130, r: 20 + velocity * 60 });
}

function draw() {
  background(20, 40);
  fill(255, 90, 31);
  for (const h of hits) { circle(h.x, h.y, h.r); h.r *= 0.95; }
  hits = hits.filter((h) => h.r > 2);
}
Each hit pops a circle where its pad would sit on a 4×2 grid, and it fades away.

Related

  • padReleased() You define this function; oav-p5 calls it each time a drum pad is let go.
  • pad1 How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.

p5/oav-p5.js, line 254

padReleased()#

eventoav-p5
function padReleased(n) {}

You define this function; oav-p5 calls it each time a drum pad is let go.

The partner of padPressed(): function padReleased(n) { … }. Without it, a pad calls noteReleased().

Parameters

NameTypeDescription
nNumberWhich pad (1 is the first).

Examples

jsOpenProcessing ↗
let down = {};

function setup() {
  createCanvas(400, 300);
}

function padPressed(n) { down[n] = true; }
function padReleased(n) { down[n] = false; }

function draw() {
  background(20);
  fill(255, 90, 31);
  for (let i = 1; i <= 8; i++) if (down[i]) square(10 + (i - 1) * 48, 130, 40);   // lit while held
}
A square per pad, lit from press to release.

Related

  • padPressed() You define this function; oav-p5 calls it each time a drum pad is hit.
  • pad1 How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.
  • noteReleased() You define this function; oav-p5 calls it each time a key (or a pad) is let go.

p5/oav-p5.js, line 254

pitchBend()#

eventoav-p5
function pitchBend(value, channel) {}

You define this function; oav-p5 calls it whenever the pitch wheel moves, with its position from −1 to 1.

Declare it the way you declare keyPressed():

js
function pitchBend(value, channel) {
  // the wheel moved
}

A pitch wheel usually springs back to the middle when you let go, so you will hear a stream of values ending at 0. MIDI sends the position as 14 bits (0 to 16383, with 8192 in the middle). oav-p5 hands it to you as −1 to 1. The latest position is also in OAVp5.bend.

For one incoming message the order is: midiMessage(), then the specific one (noteOn(), controlChange(), pitchBend()), then the friendly one (padPressed(), knobMoved(), notePressed() and their partners).

Before it has a value

Defining the function is enough. oav-p5 asks the browser for Web MIDI when it loads (the default, ?oav_hardware=auto), and a sketch that defines it also gets a card if Web MIDI is missing or blocked. Keys from 🎹, a phone, OAVp5.input() and other programs reach it too. OAVp5.on('pitchBend', fn) does the same without defining a global function, and returns a function that stops it.

Parameters

NameTypeDescription
valueNumberThe wheel's position: −1 all the way down, 0 in the middle, 1 all the way up.
channelNumberThe MIDI channel, 1 to 16.

Notes

  • Pitch bend is not mapped to anything else: there is no bend variable, only this function and OAVp5.bend.
  • An error inside it does not stop the music: the next message is still handled, and the message shows in the console and on a card beside the 🎹, with the file and line.

Examples

jsOpenProcessing ↗
let bend = 0;

function setup() {
  createCanvas(400, 300);
}

function pitchBend(value) {
  bend = value;
}

function draw() {
  background(20);
  circle(width / 2 + bend * 150, height / 2, 50);   // wheel up: right, wheel down: left
}
The circle slides left and right with the pitch wheel and returns to the middle when it springs back. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
jsOpenProcessing ↗
let bend = 0;

function setup() {
  createCanvas(400, 300);
}

function pitchBend(value) { bend = value; }

function mouseDragged() {
  OAVp5.input([0xE0, 0, constrain(round(mouseX / width * 127), 0, 127)]);   // no wheel? drag the mouse
}

function draw() {
  background(20);
  const x = spring(width / 2 + bend * 150, 120, 8);   // a spring smooths the steps
  circle(x, height / 2, 50);
}
The same wheel through spring(). No wheel? Drag the mouse across the canvas: it sends the same message.

Related

  • OAVp5.bend Where the pitch wheel is, from −1 (all the way down) through 0 (middle) to 1 (all the way up).
  • controlChange() You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.
  • midiMessage() You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
  • OAVp5.on() Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
  • OAVp5.input() Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.

p5/oav-p5.js, line 255

velocity#

variableoav-p5
velocity

How hard the last key was pressed, from 0 to 1.

MIDI sends velocity as 0 to 127; oav-p5 hands you it divided by 127, so 1 is the hardest hit the controller can send.

Type
Number
Range
0..1

Set together with note when a key goes down; kept after release; 0 before any key.

Notes

  • A sketch with its own velocity keeps it; read midi.last.value.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255, 90, 31);
  circle(width / 2, height / 2, 20 + velocity * 300);   // a harder key makes a bigger circle
}
The circle stays at the size of the last key's velocity.

Related

  • note The MIDI number (0 to 127) of the last key pressed.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • noteIsPressed True while at least one key is held down.

p5/oav-p5.js, line 326

OAVp5.bend#

variableoav-p5
OAVp5.bend

Where the pitch wheel is, from −1 (all the way down) through 0 (middle) to 1 (all the way up).

Read it in draw() like a knob that goes both ways: x = width / 2 + OAVp5.bend * 150. MIDI sends the position as 14 bits (0 to 16383, with 8192 in the middle). oav-p5 maps the two halves to −1..0 and 0..1, so full travel reaches exactly −1 and 1. To be told on every movement, define pitchBend().

Type
Number
Range
-1..1

Changes when the wheel moves. 0 before it has moved, and again when a spring-loaded wheel returns to the middle.

Notes

  • midi.bend is the same number. There is no plain bend variable.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function mouseDragged() {
  OAVp5.input([0xE0, 0, constrain(round(mouseX / width * 127), 0, 127)]);   // no wheel? drag the mouse
}

function draw() {
  background(20);
  circle(width / 2 + OAVp5.bend * 150, height / 2, 50);   // wheel up: right, wheel down: left
  text('bend ' + nf(OAVp5.bend, 1, 2), 20, 30);
}
The circle follows the pitch wheel. No wheel? Drag the mouse across the canvas: it sends the same message.

Related

  • pitchBend() You define this function; oav-p5 calls it whenever the pitch wheel moves, with its position from −1 to 1.
  • OAVp5.cc An array of 128 numbers, one per MIDI controller number: where each control last was, 0 to 1. OAVp5.cc[1] is the mod wheel, OAVp5.cc[64] the sustain pedal.
  • OAVp5.note An array of 128 numbers, one per MIDI key: how hard that key is held, 0 to 1, and 0 when it is up. OAVp5.note[60] is middle C.

p5/oav-p5.js, line 549

OAVp5.cc#

variableoav-p5
OAVp5.cc

An array of 128 numbers, one per MIDI controller number: where each control last was, 0 to 1. OAVp5.cc[1] is the mod wheel, OAVp5.cc[64] the sustain pedal.

The array version of controlChange(): read any controller in draw() without defining a function. It holds every CC, including the ones that are not a knob or a fader, such as the mod wheel (1), the volume (7) and the sustain pedal (64, which counts as down at 0.5 and above). The MIDI channel is ignored: the latest value of that controller number wins, whichever channel it came on.

Type
Array<Number>
Range
128 numbers, 0..1 each

The index is the controller (CC) number, 0 to 127; the value is the last one received divided by 127. Changes when a message arrives. 0 before a control has moved, which is not the same as "at 0".

Notes

  • midi.cc is the same array. The friendly names knob1 and fader1 are looked up through the controller's profile and read the same data.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function mouseDragged() {
  OAVp5.input([0xB0, 1, constrain(round(mouseX / width * 127), 0, 127)]);   // no wheel? drag the mouse
}

function draw() {
  background(OAVp5.cc[64] >= 0.5 ? 90 : 20);   // sustain pedal down: lighter
  circle(width / 2, height / 2, 40 + OAVp5.cc[1] * 300);   // mod wheel: size
}
The mod wheel (CC 1) sets the size of the circle and the sustain pedal (CC 64) lightens the background. No controller? Drag the mouse across the canvas: it sends CC 1.

Related

  • controlChange() You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.
  • knob1 The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
  • fader1 The position of a controller fader (slider), from 0 to 1: fader1 is the first fader, up to fader8.
  • OAVp5.note An array of 128 numbers, one per MIDI key: how hard that key is held, 0 to 1, and 0 when it is up. OAVp5.note[60] is middle C.
  • OAVp5.bend Where the pitch wheel is, from −1 (all the way down) through 0 (middle) to 1 (all the way up).

p5/oav-p5.js, line 258

OAVp5.connectHardware()#

functionoav-p5
OAVp5.connectHardware()

Asks the browser for Web MIDI now, and returns a promise that says whether a real controller can be heard.

It does what ?oav_hardware=auto does at load, at a moment you pick: from a button, after an intro, or when hardware is click or off. It works whatever the option says. After a refusal you can call it again to retry, but browsers usually remember a refusal and answer at once, so the way out is the lock icon left of the address.

Before you call it

A browser with Web MIDI (Chrome, Edge, Firefox), on https or localhost. Some browsers want a click first: call it from mousePressed() or a button handler if setup() is too early. Safari and iPad have none: the promise resolves false and OAVp5.hardware is 'unavailable'.

Returns

Promise<Boolean> Resolves to true once the browser has allowed Web MIDI (with or without a device plugged in), and to false if it refused or there is no Web MIDI in this browser. Calling it again while it is asking, or after it said yes, gives the same answer.

Examples

jsOpenProcessing ↗
window.OAV_P5 = { hardware: 'click' };   // do not ask at load

let answer = 'click the canvas to ask for Web MIDI';

function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  OAVp5.connectHardware().then((ok) => { answer = ok ? 'Web MIDI allowed' : 'no Web MIDI for this page'; });
}

function draw() {
  background(20);
  fill(255);
  text(answer, 20, 30);
  text('state: ' + OAVp5.hardware, 20, 60);
}
Nothing is asked at load. Click the canvas: the browser asks, the state goes from "off" through "asking" to "connected", "waiting" or "denied", and the text follows.

Related

  • OAVp5.hardware Where Web MIDI stands: 'off', 'asking', 'connected', 'waiting', 'denied' or 'unavailable'.
  • OAVp5.devices The names of the MIDI controllers plugged in, as the browser reports them: an array of text, empty when there is none.
  • hardware When oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.

p5/oav-p5.js, line 274

OAVp5.devices#

variableoav-p5
OAVp5.devices

The names of the MIDI controllers plugged in, as the browser reports them: an array of text, empty when there is none.

Use it for a line like "playing: Arturia MiniLab3", or to tell whether a real controller is there. The names are whatever the operating system calls the port, so match them loosely (name.toLowerCase().includes('minilab')). The 🎹 panel uses them to follow the controller that is plugged in.

Type
Array<String>

One entry per connected input port. Replaced whenever a device is plugged in or out. Empty until Web MIDI has been allowed (OAVp5.hardware), and it never lists the on-screen controller or the phone.

Notes

  • midi.devices is the same array, replaced as a whole, not changed in place: read it again after a change.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text(OAVp5.devices.length ? 'playing: ' + OAVp5.devices.join(', ') : 'no MIDI controller plugged in', 20, 30);
}
Names the controllers found, or says there are none. Plug a controller in while the page is open and the line changes.

Related

  • OAVp5.hardware Where Web MIDI stands: 'off', 'asking', 'connected', 'waiting', 'denied' or 'unavailable'.
  • OAVp5.connectHardware() Asks the browser for Web MIDI now, and returns a promise that says whether a real controller can be heard.
  • profile Which controller the on-screen 🎹 shows, by profile id such as akai-lpd8; by default it follows the one that is plugged in.
  • hardware When oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.

p5/oav-p5.js, line 598

OAVp5.hardware#

variableoav-p5
OAVp5.hardware

Where Web MIDI stands: 'off', 'asking', 'connected', 'waiting', 'denied' or 'unavailable'.

'off': Web MIDI has not been asked for yet (see hardware). 'asking': waiting for the browser's permission answer. 'connected': allowed, and at least one device is plugged in (OAVp5.devices lists them). 'waiting': allowed, but no device yet: plug one in and it becomes 'connected' by itself. 'denied': the permission was refused. 'unavailable': the browser has no Web MIDI (Safari, iPad) or could not open it.

It is only about real hardware. The on-screen controller and a phone work in every state.

Type
String
Range
'off' | 'asking' | 'connected' | 'waiting' | 'denied' | 'unavailable'

Changes when the browser answers and whenever a MIDI device is plugged in or out.

Notes

  • In the 🩺 check-up, waiting is written as "no device found".

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(OAVp5.hardware === 'connected' ? color(20, 60, 30) : 20);
  fill(255);
  text('Web MIDI: ' + OAVp5.hardware, 20, 30);
  text('devices: ' + (OAVp5.devices.join(', ') || 'none'), 20, 60);
}
Shows the state of Web MIDI and the devices found. The background turns green once a controller is connected. Plug one in to see it change.

Related

  • OAVp5.devices The names of the MIDI controllers plugged in, as the browser reports them: an array of text, empty when there is none.
  • OAVp5.connectHardware() Asks the browser for Web MIDI now, and returns a promise that says whether a real controller can be heard.
  • hardware When oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.
  • OAVp5.check() Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.

p5/oav-p5.js, line 260

OAVp5.held#

variableoav-p5
OAVp5.held

A list of the MIDI numbers of the keys that are down right now, lowest first.

The short version of OAVp5.note: only the keys that are down, so OAVp5.held.length is how many, OAVp5.held[0] is the lowest and OAVp5.held.map((n) => noteName(n)) spells the chord. noteIsPressed is OAVp5.held.length > 0.

Type
Array<Number>

Empty when nothing is held. A key is added when its note-on arrives and removed at its note-off, and the list is kept sorted from low to high.

Notes

  • midi.held is the same array. Read it, do not change it. It is the same array object every time, so copy it ([...OAVp5.held]) if you want to keep a moment of it.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(40);
}

function mousePressed() { OAVp5.play(60 + floor(random(12)), 0.8); }   // no keys? click

function draw() {
  background(20);
  fill(255);
  text(OAVp5.held.map((n) => noteName(n)).join('  ') || 'play a chord', width / 2, height / 2);
}
Spells the keys held down, lowest first. Click the canvas to hold one more key. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.

Related

  • OAVp5.note An array of 128 numbers, one per MIDI key: how hard that key is held, 0 to 1, and 0 when it is up. OAVp5.note[60] is middle C.
  • noteIsPressed True while at least one key is held down.
  • noteIsDown() True while the key n (a MIDI number or a name like 'C4') is held down.
  • noteName() Turns a MIDI number into a note name: noteName(61) is 'C#4'.

p5/oav-p5.js, line 258

OAVp5.input()#

functionoav-p5
OAVp5.input(bytes, [source], [device])

Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.

It is the door every input goes through, and you can knock on it yourself: a sketch can test itself without a controller, drive itself from a timer, or take messages from a WebSocket or another library. The sketch cannot tell the difference: knob1, notePressed(), OAVp5.note, the sound if startSound() is on and the drum cues all react. OAVp5.play() is the friendlier way to send a note.

Parameters

NameTypeDescription
bytesArray<Number>One message, [status, data1, data2]: [0x90, 60, 100] is a note-on for key 60 with velocity 100 on channel 1, [0xB0, 74, 127] a control change, [0xE0, 0, 64] a pitch bend in the middle. The status byte sets the kind (top half) and the channel (bottom half). Only the first three numbers are read, and the data bytes are held to 0 to 127. System messages are not for this: the clock bytes (0xF8, 0xFA, 0xFB, 0xFC, 0xF2) go to OAVp5.clock, and anything else from 0xF0 up is ignored.
sourceStringA label for where it came from, which shows up as OAVp5.last.source and as the second argument of midiMessage(). Any text. 'hardware' and 'virtual-out' are taken as those real sources and are not repeated to a sketch's own Web MIDI code. optional default 'screen'
deviceStringA profile id such as 'akai-lpd8': the numbering of knob1, pad1 … follows that controller for this message. Left out, it is the controller on the on-screen panel. optional

Notes

  • midi.input(…) is the same function. It returns nothing. A message with a source other than 'screen' also lights the keys of the on-screen controller if it is open.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(24);
}

function mousePressed() {
  OAVp5.input([0x90, 60 + floor(random(12)), 100]);   // a note-on, as a key would send it
}

function notePressed(note, velocity) {
  background(20);
  fill(255);
  text(noteName(note) + ' (sent by hand)', width / 2, height / 2);
}
Click the canvas: the sketch hears a note-on exactly as if you had played a key, and names it.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function mouseDragged() {
  OAVp5.input([0xB0, 74, constrain(round(mouseX / width * 127), 0, 127)]);   // CC 74 is knob 1 on a MiniLab 3
}

function draw() {
  background(20);
  circle(width / 2, height / 2, 40 + knob1 * 300);
}
Drag the mouse across the canvas: it sends CC 74, which is knob 1, and the circle follows.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  OAVp5.input([0xB0, 70, 127], 'test', 'akai-lpd8');   // CC 70 is knob 1 on an LPD8
}

function draw() {
  background(20);
  fill(255);
  text('knob1 = ' + knob1 + ' (from the LPD8 numbering)', 20, 30);
  text('last: ' + OAVp5.last.control + ' from ' + OAVp5.last.source, 20, 60);
}
The third argument says which controller the message comes from, so CC 70 is knob 1 here, whatever profile the panel shows.

Related

  • OAVp5.play() Plays a key as if a controller did: midi.play(60, 0.8) presses middle C, midi.play(60, 0) lets it go.
  • OAVp5.last The latest note, control change or pitch bend that arrived: its type, number, value, channel, where it came from and which control it is. null before any.
  • OAVp5.on() Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
  • midiMessage() You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
  • noteOn() You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
  • controlChange() You define this function; oav-p5 calls it for every MIDI control change (a knob, a fader, the mod wheel, the sustain pedal), whether or not the controller is known.

p5/oav-p5.js, line 273

OAVp5.isDown()#

functionoav-p5
OAVp5.isDown(n)

True while the key n (a MIDI number or a name like 'C4') is held down: the same as noteIsDown(n).

It is noteIsDown() under the OAVp5 name, for a sketch that cannot use the plain one: OAVp5.isDown('C4'). A name that is not a note stops the sketch with a card that says how to write one.

Parameters

NameTypeDescription
nNumber | StringThe key: a MIDI number (60) or a note name ('C4', 'c#4', 'Db3').

Returns

Boolean True if that key is down right now.

Notes

  • midi.isDown(…) is the same function.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(32);
}

function mousePressed() { for (const n of ['C4', 'E4', 'G4']) OAVp5.play(n, 0.8); }   // no keys? click for a chord
function mouseReleased() { for (const n of ['C4', 'E4', 'G4']) OAVp5.play(n, 0); }

function draw() {
  const chord = OAVp5.isDown('C4') && OAVp5.isDown('E4') && OAVp5.isDown('G4');
  background(chord ? color(60, 100, 40) : 20);
  fill(255);
  text(chord ? 'C major' : 'hold C4 + E4 + G4', width / 2, height / 2);
}
The background turns green while C, E and G are held together. Hold the mouse down to play the chord. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.

Related

  • noteIsDown() True while the key n (a MIDI number or a name like 'C4') is held down.
  • OAVp5.note An array of 128 numbers, one per MIDI key: how hard that key is held, 0 to 1, and 0 when it is up. OAVp5.note[60] is middle C.
  • OAVp5.held A list of the MIDI numbers of the keys that are down right now, lowest first.
  • noteNumber() Turns a note name like 'C4' into its MIDI number, 0 to 127.

p5/oav-p5.js, line 271

OAVp5.last#

variableoav-p5
OAVp5.last

The latest note, control change or pitch bend that arrived: its type, number, value, channel, where it came from and which control it is. null before any.

A single object that is replaced on every message, so read it, do not keep it. It is how a sketch finds out where a message came from (OAVp5.last.source) or which knob moved (OAVp5.last.control), and what to read when the sketch has its own note or velocity that keeps the plain names: OAVp5.last.note and OAVp5.last.value.

It keeps only notes, control changes and pitch bends. Aftertouch and program change reach midiMessage() but are not stored here.

Type
Object | null

{ type, note | cc, value, channel, source, control }. type is 'noteOn', 'noteOff', 'controlChange' or 'pitchBend' (a note-on with velocity 0 is a 'noteOff'). note is there for notes, cc for control changes. value is the velocity 0..1 (0 for a note-off), the control's position 0..1, or the wheel −1..1. channel is 1 to 16. source is 'hardware', 'screen', 'phone', 'virtual-out', 'link' or the label you gave OAVp5.input(). control is 'knob1', 'fader2', 'pad3' … when the controller's profile knows it, otherwise null.

Notes

  • midi.last is the same object. Clock messages (tick, start, stop) are not here: see OAVp5.clock.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textSize(16);
}

function mousePressed() { OAVp5.input([0x90, 60 + floor(random(12)), 100], 'mouse'); }   // no controller? click

function draw() {
  background(20);
  fill(255);
  text(OAVp5.last ? JSON.stringify(OAVp5.last, null, 1) : 'play something', 20, 30);
}
Prints the last message as an object. Click the canvas to send one from the mouse, and watch source. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.

Related

  • midiMessage() You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
  • noteOn() You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
  • note The MIDI number (0 to 127) of the last key pressed.
  • velocity How hard the last key was pressed, from 0 to 1.
  • OAVp5.input() Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.

p5/oav-p5.js, line 524

OAVp5.note#

variableoav-p5
OAVp5.note

An array of 128 numbers, one per MIDI key: how hard that key is held, 0 to 1, and 0 when it is up. OAVp5.note[60] is middle C.

Read every key at once, in draw(): OAVp5.note[60] for one key, or a loop over a range to draw a piano roll. It is the array version of noteIsDown(), with the velocity in it. It is not the variable note, which is just the number of the last key pressed. OAVp5.held is the same information as a short list of the keys that are down.

Type
Array<Number>
Range
128 numbers, 0..1 each

The index is the MIDI note number, 0 to 127. A key that is down holds the velocity of its note-on divided by 127, a key that is up (or never pressed) holds 0. Changes when a message arrives, not once per frame.

Notes

  • midi.note is the same array. A note-on with velocity 0 sets the key to 0, like a note-off.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
}

function draw() {
  background(20);
  for (let n = 36; n < 96; n++) {   // five octaves, C2 to B6
    const v = OAVp5.note[n];
    fill(v > 0 ? color(255, 211, 107) : 60);
    rect(20 + (n - 36) * 6, height - 40 - v * 200, 5, 40 + v * 200);
  }
}
A bar for each key: it rises while the key is down, higher for a harder press. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const v = lag(OAVp5.note[60], 0.05, 0.4);   // middle C, softened
  circle(width / 2, height / 2, 40 + v * 260);
}
Middle C (60) swells the circle by how hard it is held, and the circle shrinks back slowly.

Related

  • OAVp5.held A list of the MIDI numbers of the keys that are down right now, lowest first.
  • OAVp5.isDown() True while the key n (a MIDI number or a name like 'C4') is held down: the same as noteIsDown(n).
  • noteIsDown() True while the key n (a MIDI number or a name like 'C4') is held down.
  • note The MIDI number (0 to 127) of the last key pressed.
  • velocity How hard the last key was pressed, from 0 to 1.
  • OAVp5.cc An array of 128 numbers, one per MIDI controller number: where each control last was, 0 to 1. OAVp5.cc[1] is the mod wheel, OAVp5.cc[64] the sustain pedal.

p5/oav-p5.js, line 258

OAVp5.on()#

functionoav-p5
OAVp5.on(name, fn)

Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.

The event functions (notePressed(), knobMoved(), noteOn() …) are defined once, by name. OAVp5.on('notePressed', fn) is the other way: any number of listeners, added and removed while the sketch runs, with no global names taken. It is how a library, a class or a Hydra sketch listens, and the way to listen for a while and stop.

A listener counts as a defined function for the pad rule: once anything listens to 'padPressed', a pad stops calling notePressed(). An error inside fn shows in the console and on a card, and the other listeners still run. For the microphone's events use OAVp5.mic.on().

Parameters

NameTypeDescription
nameStringThe event: 'notePressed', 'noteReleased', 'knobMoved', 'faderMoved', 'padPressed', 'padReleased', 'noteOn', 'noteOff', 'controlChange', 'pitchBend' or 'midiMessage'. 'noteon' and 'noteoff' work in any case. Anything else throws an error that lists the names.
fnFunctionCalled with the same arguments the function of that name would get (notePressed(note, velocity), knobMoved(n, value) …).

Returns

Function A function that takes fn off again: const off = OAVp5.on(…); off();.

Notes

  • midi.on(…) is the same function.

Examples

jsOpenProcessing ↗
let heard = 0;

function setup() {
  createCanvas(400, 300);
  OAVp5.on('noteOn', (note, velocity) => { heard++; });
}

function mousePressed() { OAVp5.play(60 + floor(random(12)), 0.8); }   // no keys? click

function draw() {
  background(20);
  fill(255);
  text('notes heard: ' + heard, 20, 30);
}
Counts the notes with a listener instead of a global function. Click the canvas to play one. Plug in a controller, or press 🎹 (bottom right) and use the on-screen one.
jsOpenProcessing ↗
let heard = 0;
let off;

function setup() {
  createCanvas(400, 300);
  off = OAVp5.on('noteOn', () => {
    heard++;
    if (heard === 5) off();   // stop listening after five notes
  });
}

function mousePressed() { OAVp5.play(60 + floor(random(12)), 0.8); }

function draw() {
  background(20);
  fill(255);
  text('notes heard: ' + heard + (heard >= 5 ? ' (stopped listening)' : ''), 20, 30);
}
The number counts up to five and then stops, even though you keep playing: the listener took itself off.

Related

  • OAVp5.mic.on() Listens to a microphone event ('beat', 'kick', 'snare' or 'hat') without defining a global function, and returns a function that stops listening.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • noteOn() You define this function; oav-p5 calls it for every MIDI note-on, with the key, how hard and the channel.
  • knobMoved() You define this function; oav-p5 calls it whenever a knob moves.
  • OAVp5.input() Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.

p5/oav-p5.js, line 262

OAVp5.panel#

objectoav-p5
OAVp5.panel

The 🎹 panel as an object: open(), close() and toggle() show and hide it from code.

The panel behind the 🎹 button holds the on-screen controller and the buttons for 🎤, 🔊, 🥁, ⏺, 📷, 📱 and 🩺. OAVp5.panel.open(), close() and toggle() do what pressing 🎹 does, from your code: open it on a key, or open it at the start for a workshop (?oav_panel=open does the same at load).

Notes

  • With ?oav_panel=off there is no 🎹 and these do nothing.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function keyPressed() {
  if (key === 'p') OAVp5.panel.toggle();
}

function draw() {
  background(20);
  fill(255);
  text('press p to show or hide the 🎹 panel', 20, 30);
}
Press p: the 🎹 panel opens and, pressed again, closes.

Related

  • OAVp5.panel.open() Opens the 🎹 panel (open()), closes it (close()) or flips it (toggle()).
  • panel Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
  • compact Shows the 🎹 panel as a small box in the corner, a third of the window high, instead of a full-width bar: on or off.
  • OAVp5.phone() Shows the 📱 card: a QR code that sends a phone to the remote page, whose MIDI tab then plays this sketch. Returns the room name.

p5/oav-p5.js, line 275

OAVp5.panel.open()#

functionoav-p5
OAVp5.panel.open()

Also: OAVp5.panel.close OAVp5.panel.toggle

Opens the 🎹 panel (open()), closes it (close()) or flips it (toggle()).

Same as pressing the 🎹 button. The first time the panel opens, oav-p5 loads the on-screen controller (a few seconds on a slow connection, which the panel says while it loads), and if hardware is click it asks for Web MIDI then. OAVp5.panel.open() when it is already open does nothing new.

Returns

Promise<Element | null> A promise. open() (and toggle() when it opens) resolves to the panel's element once the on-screen controller inside it has loaded, and the panel is on screen at once, before that. close() (and toggle() when it closes) resolves to null, and so does everything when the panel option is off.

Notes

  • The three are listed together here: OAVp5.panel.close and OAVp5.panel.toggle take no arguments either. midi.panel.open() is the same function.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function keyPressed() {
  if (key === 'o') OAVp5.panel.open();
  if (key === 'c') OAVp5.panel.close();
}

function draw() {
  background(20);
  fill(255);
  text('o: open the 🎹 panel,  c: close it', 20, 30);
}
Press o to open the 🎹 panel and c to close it.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  OAVp5.panel.open().then((panel) => console.log('the panel is', panel ? 'open and loaded' : 'not available'));
}

function draw() {
  background(20);
  fill(255);
  text('click the canvas to open the panel', 20, 30);
}
Click the canvas: the panel opens at once, and the console says when the controller inside it has loaded.

Related

  • OAVp5.panel The 🎹 panel as an object: open(), close() and toggle() show and hide it from code.
  • panel Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
  • OAVp5.phone() Shows the 📱 card: a QR code that sends a phone to the remote page, whose MIDI tab then plays this sketch. Returns the room name.
  • OAVp5.check() Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.

p5/oav-p5.js, line 275

OAVp5.phone()#

functionoav-p5
OAVp5.phone()

Shows the 📱 card: a QR code that sends a phone to the remote page, whose MIDI tab then plays this sketch. Returns the room name.

The same as the 📱 button in the panel. The card shows a QR code, the room and the link underneath; opening the link on a phone gives that phone an on-screen controller whose MIDI tab plays your sketch over the relay, as one more input (OAVp5.last.source is 'phone'). Calling it again hides the card. The first call connects to the relay and loads a small QR-code library from jsDelivr.

Before you call it

A network connection, to the relay (wss://remote.cheyuwu.com/relay unless you set another) and, for the picture of the QR code, to jsDelivr. Without the library the card shows the link as text.

Returns

String The name of the room the phone joins: the room option, or the link room, or a name made up for this browser (p5-…) and kept in it.

Examples

js
function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  const room = OAVp5.phone();   // the QR code card opens
  console.log('phones join room', room);
}

function notePressed(note) {
  background(random(255), random(255), random(255));   // a phone key changes the colour
}

function draw() {
  fill(255);
  text('click the canvas, scan the QR code, open the MIDI tab', 20, 30);
}
static Click the canvas and scan the QR code with a phone: the keys on its MIDI tab change the background of this sketch.

To be confirmed

A phone playing a sketch through the relay was not tried for this page: the headless check cannot reach the relay, so the example is not run. The behaviour is as the code and the project's README describe it.

Related

  • room The name of the room phones join to play this page: by default one made up per browser.
  • relay The address of the relay that phones and link rooms talk through: a WebSocket URL.
  • remote The address of the remote page that the 📱 QR code opens on a phone.
  • OAVp5.panel.open() Opens the 🎹 panel (open()), closes it (close()) or flips it (toggle()).

p5/oav-p5.js, line 276

OAVp5.play()#

functionoav-p5
OAVp5.play(note, [velocity])

Plays a key as if a controller did: midi.play(60, 0.8) presses middle C, midi.play(60, 0) lets it go.

It sends the same MIDI bytes a keyboard would, with midi.last.source set to 'screen', so everything reacts: notePressed(), noteIsDown(), note, the sound if startSound() is on. midi and OAVp5 are the same object, so OAVp5.play and midi.play are one function.

Parameters

NameTypeDescription
noteNumber | StringThe key: a MIDI number or a name ('C#4').
velocityNumberHow hard, 0..1; 0 releases the key. Leave it out for full force. optional default 1

Notes

  • The key stays down until you play it again with velocity 0.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(32);
}

function mousePressed() { midi.play('C4', 0.8); }
function mouseReleased() { midi.play('C4', 0); }

function draw() {
  background(noteIsDown('C4') ? 90 : 20);
  fill(255);
  text('hold the mouse: C4 is ' + (noteIsDown('C4') ? 'down' : 'up'), width / 2, height / 2);
}
The mouse plays middle C: the sketch hears it the same as a key.

Related

  • noteIsDown() True while the key n (a MIDI number or a name like 'C4') is held down.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • noteNumber() Turns a note name like 'C4' into its MIDI number, 0 to 127.
  • OAVp5.input() Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.

p5/oav-p5.js, line 269

OAVp5.wiring()#

functionoav-p5
OAVp5.wiring([dev])

Lists which controller knob, fader or pad drives which OpenProcessing OPC variable right now: [{ control: 'knob1', name: 'radius' }, …].

With OP Configurator, OPC.slider('radius', 10) declares a variable and oav-p5 wires the controller to the OPC variables in order: knob 1 to the first slider, knob 2 to the second, the faders after the knobs, pad 1 to the first toggle or button. This is the list of who drives what, the same one the panel prints under the controller and the console says once. Use it when a knob moves the wrong thing.

Parameters

NameTypeDescription
devStringWhose numbering to use: 'screen' (the controller shown in the panel) or a profile id such as 'akai-lpd8'. It matters for the faders, which are numbered after the knobs. optional default 'screen'

Returns

Array<Object> One { control, name } per wired variable: control is 'knob1', 'fader2', 'pad1' …, name is the OPC variable. A control the sketch reads itself also has sketch: true, and its slider no longer follows it. An empty array when there is no OPC on the page or opc is off.

Notes

  • midi.wiring(…) is the same function.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  const wired = OAVp5.wiring();
  text(wired.length ? wired.map((w) => w.control + ' -> ' + w.name).join('\n') : 'nothing wired (no OPC here)', 20, 30);
}
Prints the controls and the OPC variables they drive, one per line. Outside OpenProcessing there is no OPC, so it says nothing is wired.

To be confirmed

Not tried with OpenProcessing's real OPC library: it was checked against a stand-in object with the same options shape ({ type: 'slider', name, min, max }). Outside OpenProcessing the result is an empty array.

Related

  • opc Whether oav-p5 wires the controller to the variables of OP Configurator (OpenProcessing's sliders and buttons): on or off.
  • knob1 The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
  • knob() Returns the position of knob number n, from 0 to 1: the same as knob1 to knob8, for any n.
  • pad1 How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.
  • OAVp5.check() Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.

p5/oav-p5.js, line 278

Note names and pitch#

C4, c#4, Db3: write a note as a name, convert to number and Hz. 3

oav-p5

noteFreq()#

functionoav-p5
noteFreq(v)

Returns the frequency in Hz of a note: noteFreq('A4') is 440.

For a sketch that makes its own sound (p5.sound, Web Audio): osc.freq(noteFreq(note)). If you just want the sound, startSound() plays every key for you.

Parameters

NameTypeDescription
vNumber | StringA note name or a MIDI number. A number may be fractional (69.5 is a quarter tone above A4) and is not held to 0..127.

Returns

Number Hz, equal temperament with A4 = 69 = 440 Hz.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(36);
}

function draw() {
  background(20);
  fill(255);
  text('A4 = ' + noteFreq('A4') + ' Hz\nC4 = ' + noteFreq('C4').toFixed(2) + ' Hz', width / 2, height / 2);
}
Prints 440 Hz and 261.63 Hz.

Related

  • noteNumber() Turns a note name like 'C4' into its MIDI number, 0 to 127.
  • noteName() Turns a MIDI number into a note name: noteName(61) is 'C#4'.
  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.

p5/oav-p5.js, line 244

noteName()#

functionoav-p5
noteName(v, [o])

Turns a MIDI number into a note name: noteName(61) is 'C#4'.

notePressed(note, velocity) hands you the number; noteName(note) writes it back as a name for the screen or a lookup table. The same C4-is-60 convention as noteNumber().

Parameters

NameTypeDescription
vNumber | StringA MIDI number (or a name, which is normalised).
oObjectOptions: { flats: true } writes 'Db4' instead of 'C#4'; { octave: false } drops the octave number ('C#'). optional

Returns

String The name, sharps by default: noteName(60) is 'C4'.

Examples

jsOpenProcessing ↗
let shown = 'play a key';

function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(64);
}

function notePressed(note) {
  shown = noteName(note) + '  ' + noteName(note, { flats: true });
}

function draw() {
  background(20);
  fill(255);
  text(shown, width / 2, height / 2);
}
Shows each key twice, with sharps and with flats (they differ on black keys). Open 🎹 and play.

Related

  • noteNumber() Turns a note name like 'C4' into its MIDI number, 0 to 127.
  • noteFreq() Returns the frequency in Hz of a note: noteFreq('A4') is 440.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • note The MIDI number (0 to 127) of the last key pressed.

p5/oav-p5.js, line 239

noteNumber()#

functionoav-p5
noteNumber(v)

Turns a note name like 'C4' into its MIDI number, 0 to 127.

A name is a letter A to G (either case), an optional # or b (♯ and ♭ too), and the octave: C4, c#4, Db3, C-1 (0) up to G9 (127). C4 is 60. Anything else, such as 'H4' or 'C-2', throws an error, and a card beside the 🎹 says how to write a note.

You rarely call it yourself: every place that takes a note number also takes a name, such as noteIsDown() and midi.play().

Parameters

NameTypeDescription
vString | NumberA note name ('C4', 'c#4', 'Db3', 'F♯2') or a number. A number comes back rounded and held to 0..127.

Returns

Number The MIDI number: noteNumber('C4') is 60, noteNumber('A4') is 69.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(40);
}

function draw() {
  background(20);
  fill(255);
  text('C4 = ' + noteNumber('C4') + '\nA4 = ' + noteNumber('A4') + '\nF#2 = ' + noteNumber('F#2'), width / 2, height / 2);
}
Prints 60, 69 and 42.

Related

  • noteName() Turns a MIDI number into a note name: noteName(61) is 'C#4'.
  • noteFreq() Returns the frequency in Hz of a note: noteFreq('A4') is 440.
  • noteIsDown() True while the key n (a MIDI number or a name like 'C4') is held down.

p5/oav-p5.js, line 225

Microphone and audio analysis#

How loud (micLevel), bass / mid / treble, beats; @openav/audio underneath. 26

oav-p5

bass#

variableoav-p5
bass

Also: mid treble

How much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.

Use them like micLevel, one number per band: fill(200 + bass * 160, 70, 100) pushes the colour with the low end. The three share one entry because they behave the same; only the frequency range differs.

Unlike micLevel they are not measured in decibels: each is the average strength of the frequencies in its band, scaled by micSensitivity() and held to 1. A band that is faint in your room can stay close to 0; raise the sensitivity rather than multiplying by a large number.

Type
Number
Range
0..1

The energy in 20–250 Hz (bass), 250–2000 Hz (mid) and 2000–8000 Hz (treble). Updated once per display frame while the microphone is on; 0 otherwise.

Before it has a value

Reading any of them asks for the microphone the first time, exactly like micLevel; until you allow it they are 0.

Notes

  • A sketch that already has its own variable, function or library called bass, mid or treble keeps it: oav-p5 leaves that name alone and never starts the microphone for it. Assigning one later turns the name back into a plain variable.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  colorMode(HSB, 360, 100, 100);
}

function draw() {
  background(230, 40, 10);
  fill(200 + bass * 160, 70, 100);   // low sounds push the colour toward magenta
  circle(width / 2, height / 2, 80 + bass * 220);
}
A circle that swells and changes colour with the low end: tap the desk or play something with a kick. Asks for the microphone
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
}

function draw() {
  background(20);
  const bands = [bass, mid, treble];
  for (let i = 0; i < 3; i++) {
    fill(255, 80 + i * 70, 120);
    rect(60 + i * 110, height - 20, 80, -20 - bands[i] * 240);
  }
}
Three bars, one per band, from low to high. Whistle for the right one, hum for the left. Asks for the microphone

To be confirmed

Typical values for speech and for music depend on the microphone and the room; no typical numbers are given here until they have been measured.

Related

  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • beat True for one frame each time the microphone (or the drum machine) hears a beat.
  • micSensitivity() Makes the microphone more or less sensitive: micSensitivity(2) is twice as sensitive, micSensitivity() says what it is now.
  • kickDetected() You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.

p5/oav-p5.js, line 1017

beat#

variableoav-p5
beat

True for one frame each time the microphone (or the drum machine) hears a beat.

It behaves like mouseIsPressed while a button is down, except the "press" lasts one frame: if (beat) flash = 1; is all it takes to react to the music. A beat is a jump in loudness over what the room has been doing, with at least 100 ms between two beats; two beats between two frames count as one.

When the drum machine is playing, beat is also true for a frame on every kick and snare, so a sketch written for the microphone moves over to the machine unchanged.

Type
Boolean

True in the first draw() after a beat, and false until the next one. Reading it twice in the same frame gives the same answer.

Before it has a value

Reading beat asks for the microphone the first time, like micLevel. If you would rather be called, define beatDetected().

Examples

jsOpenProcessing ↗
let flash = 0;

function setup() {
  createCanvas(400, 300);
}

function draw() {
  if (beat) flash = 1;        // true for exactly one frame per beat
  background(20 + flash * 200);
  flash *= 0.9;               // fade back down
}
The screen flashes on every beat and fades out. Clap or tap the desk. Asks for the microphone
jsOpenProcessing ↗
let count = 0;

function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(96);
}

function draw() {
  if (beat) count++;
  background(20);
  fill(255);
  text(count, width / 2, height / 2);
}
A counter that goes up by one per beat, never twice for the same one. Asks for the microphone

Related

  • beatDetected() You define this function; oav-p5 calls it each time the microphone (or the drum machine) hears a beat.
  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • kickDetected() You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 1020

beatDetected()#

eventoav-p5
function beatDetected(level) {}

You define this function; oav-p5 calls it each time the microphone (or the drum machine) hears a beat.

Declare it the way you declare keyPressed():

js
function beatDetected(level) {
  // flash, spawn, jump to the next scene
}

It is called once per beat, right when the beat lands, not once per frame. Use beat instead if you would rather ask each frame.

Before it has a value

A sketch that defines beatDetected() and reads no microphone value starts the microphone on the first click or key. When the drum machine is on, the same function is called for its kicks and snares and the microphone is not opened for it. OAVp5.mic.on('beat', fn) hears the same beats without defining a global and returns a function that stops listening, but a listener alone never starts the microphone: read micLevel or call startMic() as well.

Parameters

NameTypeDescription
levelNumberHow loud it was at that moment, 0..1 (the value of micLevel).

Notes

  • If your function throws, the sketch keeps running: the error appears in the console and as a card beside the 🎹 with the file and line.

Examples

jsOpenProcessing ↗
let flash = 0;

function setup() {
  createCanvas(400, 300);
}

function beatDetected(level) {
  flash = 0.4 + level * 0.6;   // louder beat, brighter flash
}

function draw() {
  background(20 + flash * 200);
  flash *= 0.9;
}
A flash per beat, brighter for a louder one. Asks for the microphone

Related

  • beat True for one frame each time the microphone (or the drum machine) hears a beat.
  • kickDetected() You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 865

kickDetected()#

eventoav-p5
function kickDetected(level) {}

Also: snareDetected hatDetected

You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.

The microphone splits what it hears into three drum-like bands and calls the matching function when each band jumps: 20–120 Hz for the kick, 150–800 Hz for the snare, 6–14 kHz for the hat. There is no machine learning in it, just a transient in a frequency band, so a bass voice can trigger a kick.

When the drum machine plays, each of its hits calls the same functions: kick for the kick, snare for a snare, clap or rim, hat for a closed, open or pedal hat. A sketch written for the microphone therefore works with startDrums() unchanged.

While the drum machine plays, only its hits call these functions (from oav-p5 0.8.1). A microphone that is on at the same time mostly hears the drums coming out of the speakers, so its hits would call the same function twice and hear a kick as a snare. micLevel, bass, mid and treble still follow the microphone; the microphone's hits come back when the drums stop.

Before it has a value

Defining one of them is not enough to open the microphone once the drum machine has started; reading micLevel (or calling startMic()) is. Without the drum machine, a sketch that defines any of them and reads nothing starts the microphone on the first click or key.

Parameters

NameTypeDescription
levelNumberHow strong the hit was, 0..1 (for the drum machine, its velocity).

Examples

jsOpenProcessing ↗
let kick = 0, snare = 0;

function setup() {
  createCanvas(400, 300);
  noStroke();
}

function kickDetected(level) { kick = level; }
function snareDetected(level) { snare = level; }

function draw() {
  background(20);
  fill(255, 90, 31);
  circle(120, 150, 40 + kick * 160);
  fill(120, 190, 255);
  circle(280, 150, 40 + snare * 160);
  kick *= 0.9;
  snare *= 0.9;
}
Two circles, orange for kicks and blue for snares, each swelling and shrinking back on its own hits. Asks for the microphone

Related

  • beatDetected() You define this function; oav-p5 calls it each time the microphone (or the drum machine) hears a beat.
  • beat True for one frame each time the microphone (or the drum machine) hears a beat.
  • bass How much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 865

kickIsDetected#

variableoav-p5
kickIsDetected

Also: snareIsDetected hatIsDetected

True in the one frame after a kick (snare, hat) is heard, from the drum machine or the microphone, like mouseIsPressed for a hit.

The same hits that call kickDetected(), snareDetected() and hatDetected(), as a value you read in draw() instead of a function you write. Hand it to lag() and it becomes a number that jumps to 1 on the hit and fades: lag(kickIsDetected, 0, 0.3) is 1 on a kick and back near 0 about 0.3 s later, ready for an alpha or a size. lag() takes true as 1.

While the drum machine plays, only its hits count. Reading these does not ask for the microphone by itself: with no drums, it counts as wanting the microphone's hits, the way defining kickDetected() does, so the first click on the page opens it.

Type
Boolean

True for exactly one p5 frame per hit, however often that frame reads it; false otherwise. Since oav-p5 0.8.2.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, ~ sd ~ sd', 110);
  noStroke();
}

function draw() {
  background(20);
  fill(255, 90, 31, 255 * lag(kickIsDetected, 0, 0.3));    // a kick lights it, then it fades
  circle(130, 200, 140);
  fill(120, 190, 255, 255 * lag(snareIsDetected, 0, 0.3));
  circle(270, 200, 140);
}
The orange circle flashes on every kick and fades out, the blue one on every snare.

Related

  • kickDetected() You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
  • beat True for one frame each time the microphone (or the drum machine) hears a beat.
  • lastKickTs When the last kick (snare, hat, beat) came, on the millis() clock: millis() - lastKickTs is how many ms ago.
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).

p5/oav-p5.js, line 1044

lastKickFrameCount#

variableoav-p5
lastKickFrameCount

Also: lastSnareFrameCount lastHatFrameCount lastBeatFrameCount

The frameCount of the frame that saw the last kick (snare, hat, beat): frameCount - lastKickFrameCount is how many frames ago.

The frame-counting twin of lastKickTs. frameCount - lastKickFrameCount is 0 in the frame where kickIsDetected is true, then 1, 2, 3 …, so a shape can last a fixed number of frames after each hit. Use the Ts version when it should last a fixed time whatever the frame rate.

Type
Number

A frame number, on the same count as frameCount; -Infinity before the first hit. Since oav-p5 0.8.3.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, ~ sd ~ sd', 110);
  noStroke();
}

function draw() {
  background(20);
  const k = frameCount - lastKickFrameCount;    // frames since the kick
  if (k < 10) {                                 // the ring stays for 10 frames after each kick
    fill(255, 90, 31);
    circle(200, 200, 60 + k * 16);
  }
}
After every kick a circle shows for ten frames, growing each frame.

Related

  • lastKickTs When the last kick (snare, hat, beat) came, on the millis() clock: millis() - lastKickTs is how many ms ago.
  • kickIsDetected True in the one frame after a kick (snare, hat) is heard, from the drum machine or the microphone, like mouseIsPressed for a hit.
  • lastNoteFrameCount The frameCount of the frame that saw the last key go down: frameCount - lastNoteFrameCount is how many frames ago.

p5/oav-p5.js, line 333

lastKickTs#

variableoav-p5
lastKickTs

Also: lastSnareTs lastHatTs lastBeatTs

When the last kick (snare, hat, beat) came, on the millis() clock: millis() - lastKickTs is how many ms ago.

Set by every hit that calls kickDetected(), snareDetected(), hatDetected() or beatDetected(): the drum machine's, or the microphone's when no drums play. Use it for anything that depends on how long ago: exp(-(millis() - lastKickTs) / 200) is 1 on the kick and fades with a 0.2 s time constant.

Type
Number

Milliseconds on the same clock as millis(); -Infinity before the first hit, so millis() - lastKickTs is then Infinity. Since oav-p5 0.8.2.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, hh*8', 100);
  noStroke();
}

function draw() {
  background(20);
  const ago = millis() - lastKickTs;    // ms since the last kick
  fill(255, 90, 31);
  circle(200, 200, 40 + 160 * exp(-ago / 200));
  fill(255);
  text(isFinite(ago) ? round(ago) + ' ms since the kick' : 'no kick yet', 20, 290);
}
The circle swells on each kick and shrinks back; the line says how many ms ago the kick was.

Related

  • kickIsDetected True in the one frame after a kick (snare, hat) is heard, from the drum machine or the microphone, like mouseIsPressed for a hit.
  • kickDetected() You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
  • lastNoteTs When the last key went down, on the millis() clock: millis() - lastNoteTs is how many ms ago.

p5/oav-p5.js, line 330

micIsOn#

variableoav-p5
micIsOn

Whether the microphone is running right now.

Use it to show a hint until sound arrives: if (!micIsOn) text("Allow the microphone", 20, 30);. Reading it never starts the microphone (unlike micLevel), so it is safe to check anywhere. The reason behind a false is in OAVp5.mic.state (off, asking, on, denied or unavailable) and, when something failed, OAVp5.mic.error.

Type
Boolean

True once the browser has said yes and sound is coming in; false while the microphone is off, still asking, blocked or missing.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(micIsOn ? 20 : 90);
  fill(255);
  textAlign(CENTER, CENTER);
  text(micIsOn ? 'listening' : 'allow the microphone', width / 2, height / 2);
}
Grey with a prompt until the microphone is on, then dark. (Nothing here reads micLevel, so nothing asks for the microphone: press 🎤 in the 🎹 panel.)

Related

  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • startMic() Turns the microphone on and returns a promise that says whether it worked.
  • stopMic() Turns the microphone off and sets every microphone value back to 0.

p5/oav-p5.js, line 326

micLevel#

variableoav-p5
micLevel

How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).

Read it in draw() the way you read mouseX: circle(width / 2, height / 2, 50 + micLevel * 300) grows when someone talks. It is a loudness in decibels, which is how an ear hears it, measured from the room itself: the bottom of the scale follows the room's own noise (it settles within a second or two and rises at most 1.5 dB a second, so a voice never becomes the new silence), and −10 dBFS, shouting into a laptop, is 1.

In practice a quiet room reads 0, a quiet voice about 0.4 to 0.6, shouting close to 1. If the microphone is quiet or far away, raise it with micSensitivity().

It follows the browser's animation frames rather than draw(), so a sketch running at frameRate(10) still reads the latest value, and the number stops changing while the tab is hidden (browsers pause animation frames there).

Type
Number
Range
0..1

Updated once per display frame while the microphone is on. It is 0 while the microphone is off, still asking, blocked or missing.

Before it has a value

Nothing to call. The first time a sketch reads micLevel (or bass, mid, treble, beat), oav-p5 asks the browser for the microphone, and every value stays 0 until you allow it. The page needs https or localhost. If the browser kept audio quiet until a click, the first click or key starts it. ?oav_mic=click makes only the 🎤 button and startMic() turn it on; ?oav_mic=off means nothing can.

Notes

  • In the 🎹 panel the 🎤 button turns the microphone on and off by hand and shows the level. When the browser blocked it, a card beside the 🎹 says why and what to press: click the lock left of the address, set Microphone to Allow, then click the canvas.
  • On openprocessing.org the sketch frame allows the microphone and MIDI (checked 2026-10-19: its allow list contains microphone and midi), and the first read of micLevel there asks for the microphone. An embed code copied from OpenProcessing does not carry that: add allow="microphone" to that <iframe>.
  • The raw numbers behind it are OAVp5.mic.rms (0..1), OAVp5.mic.db and OAVp5.mic.floor (the room's own noise, in dBFS).

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  circle(width / 2, height / 2, 50 + micLevel * 300);
}
Allow the microphone when the browser asks, then talk: the circle grows with your voice. Asks for the microphone
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const level = lag(micLevel, 0.03, 0.4);   // opens fast, closes slowly
  fill(255, 120 + level * 135, 60);
  circle(width / 2, height / 2, 50 + level * 300);
}
The same circle through lag(): it opens at once when someone speaks and closes slowly, like a VU meter. Asks for the microphone
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  micSensitivity(3);   // a quiet or far-away microphone
}

function draw() {
  background(20);
  fill(255);
  if (!micIsOn) text('Allow the microphone, then make a sound', 20, 30);
  rect(20, height - 40, micLevel * (width - 40), 20);
}
A bar that fills with the level. micIsOn tells you whether the browser has said yes yet. Asks for the microphone

To be confirmed

On openprocessing.org the pasted sketch runs, loads oav-p5 and asks for the microphone, but the sound itself was not tried there: the browser used for the check blocks the microphone, so the allow-and-listen step is still unconfirmed on that site.

Related

  • bass How much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
  • beat True for one frame each time the microphone (or the drum machine) hears a beat.
  • micIsOn Whether the microphone is running right now.
  • micSensitivity() Makes the microphone more or less sensitive: micSensitivity(2) is twice as sensitive, micSensitivity() says what it is now.
  • startMic() Turns the microphone on and returns a promise that says whether it worked.
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).

p5/oav-p5.js, line 1016

micSensitivity()#

functionoav-p5
micSensitivity([v])

Makes the microphone more or less sensitive: micSensitivity(2) is twice as sensitive, micSensitivity() says what it is now.

The number multiplies what the microphone hears. On micLevel, 2 adds 6 dB; on bass, mid and treble it multiplies by 2 (they stay held to 1). Use it for a quiet microphone or a far-away voice: micSensitivity(3) in setup() adds about 9.5 dB.

Parameters

NameTypeDescription
vNumberThe new sensitivity, greater than 0 (kept between 0.05 and 20). Leave it out, or pass something that is not greater than 0, to change nothing. optional

Returns

Number The sensitivity now in effect (1 until you change it).

Notes

  • OAVp5.mic.gain is the same knob under the name oav-p5 0.3.0 used.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  micSensitivity(3);   // three times as sensitive
}

function draw() {
  background(20);
  circle(width / 2, height / 2, 50 + micLevel * 300);
  text('sensitivity ' + micSensitivity(), 20, 30);   // no argument: just reads it
}
The same loudness circle on a microphone that needed a boost; the number shows the sensitivity in effect. Asks for the microphone

Related

  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • bass How much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
  • startMic() Turns the microphone on and returns a promise that says whether it worked.

p5/oav-p5.js, line 1047

startMic()#

functionoav-p5
startMic()

Turns the microphone on and returns a promise that says whether it worked.

You rarely need it: reading micLevel and its friends already starts the microphone. Call it when you want to choose the moment (a button, the end of an intro) or know the answer: startMic().then((ok) => { … }). Calling it again while it is on, or while the browser is still asking, returns the same result.

Before you call it

Browsers want a click before audio plays: call it from mousePressed() or a button handler if setup() is too early. It works whatever the mic option is, except ?oav_mic=off, where nothing can start the microphone.

Returns

Promise<Boolean> Resolves to true once the microphone is on, false if it was blocked, is missing, or the mic option is off.

Notes

  • After stopMic(), only startMic() (or 🎤) turns it back on; reading micLevel does not.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
}

function mousePressed() {
  startMic().then((ok) => console.log(ok ? 'microphone on' : 'no microphone'));
}

function draw() {
  background(20);
  fill(255);
  text(micIsOn ? 'level ' + nf(micLevel, 1, 2) : 'click to start the microphone', width / 2, height / 2);
}
Nothing happens until you click; then the browser asks, and the number starts moving. Asks for the microphone

Related

  • stopMic() Turns the microphone off and sets every microphone value back to 0.
  • micIsOn Whether the microphone is running right now.
  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).

p5/oav-p5.js, line 1045

stopMic()#

functionoav-p5
stopMic()

Turns the microphone off and sets every microphone value back to 0.

It stops the microphone's tracks and closes its audio context, so the browser's "recording" indicator goes away. micLevel, bass, mid, treble and the rest read 0 again, and micIsOn is false.

Returns

Promise<Boolean> Resolves to false, the new state of the microphone.

Notes

  • After stopMic(), reading micLevel does not turn the microphone back on: that needs startMic() or the 🎤 button.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
}

function keyPressed() {
  if (key === 'm') { if (micIsOn) stopMic(); else startMic(); }
}

function draw() {
  background(20);
  fill(255);
  circle(width / 2, height / 2, 50 + micLevel * 300);
  text('press m to turn the microphone ' + (micIsOn ? 'off' : 'on'), width / 2, height - 24);
}
Press m to switch the microphone on and off; the circle drops to its smallest size when it is off. Asks for the microphone

Related

  • startMic() Turns the microphone on and returns a promise that says whether it worked.
  • micIsOn Whether the microphone is running right now.
  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).

p5/oav-p5.js, line 1048

OAVp5.mic#

objectoav-p5
OAVp5.mic

The microphone under the plain names: its state, the numbers of micLevel and bass that never turn it on when read, the raw loudness, start, stop and events.

Everything the microphone names (micLevel, bass, micIsOn, startMic() …) read from here. The difference is that reading OAVp5.mic.level never asks for the microphone: the plain micLevel does, the first time. So OAVp5.mic is for code that must look without waking anything: a status line, a library, a sketch with ?oav_mic=click.

The members: state, error, sensitivity (also gain), level, bass (also mid, treble), rms, db, floor, analyzer, start(), stop() and on().

Notes

  • This is advanced. For most sketches the plain names are all you need.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  if (OAVp5.mic.state === 'on') OAVp5.mic.stop(); else OAVp5.mic.start();
}

function draw() {
  background(20);
  fill(255);
  text('microphone: ' + OAVp5.mic.state + ' (click the canvas to switch it)', 20, 30);   // reading this asks for nothing
  circle(width / 2, height / 2, 50 + OAVp5.mic.level * 300);
}
A circle that follows the microphone, which only starts when you click: reading OAVp5.mic never turns it on by itself. Asks for the microphone

Related

  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • bass How much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
  • micIsOn Whether the microphone is running right now.
  • startMic() Turns the microphone on and returns a promise that says whether it worked.
  • stopMic() Turns the microphone off and sets every microphone value back to 0.
  • mic Who may turn the microphone on: auto (the first read of micLevel and friends), click (only 🎤 or startMic()) or off (nobody).

p5/oav-p5.js, line 882

OAVp5.mic.analyzer#

variableoav-p5
OAVp5.mic.analyzer

The AudioAnalyzer from @openav/audio that listens to the microphone, while the microphone is on; null otherwise.

A way into the audio itself when the 0 to 1 numbers are not enough: read the spectrum of the microphone with the browser's own OAVp5.mic.analyzer.analyser.getByteFrequencyData(array), or connect your own Web Audio nodes to it. oav-p5 updates the analyzer once per frame, so do not call its update() yourself.

Type
AudioAnalyzer | null

Made when the microphone starts and dropped when it stops. It has its own AudioContext (OAVp5.mic.analyzer.ctx) and the Web Audio AnalyserNode it reads from (OAVp5.mic.analyzer.analyser).

Notes

  • This is advanced. Check it is not null first, and read it again after stopMic() and startMic(): it is a new object each time.

Examples

jsOpenProcessing ↗
let bins;

function setup() {
  createCanvas(400, 300);
  noStroke();
  startMic();
}

function draw() {
  background(20);
  const a = OAVp5.mic.analyzer;
  if (!a) return;   // the microphone is not on yet
  bins = bins || new Uint8Array(a.analyser.frequencyBinCount);
  a.analyser.getByteFrequencyData(bins);   // the spectrum, 0..255 per frequency bin
  fill(255, 211, 107);
  for (let i = 0; i < 64; i++) {
    const h = bins[i] / 255 * (height - 20);
    rect(20 + i * 5.8, height - 10 - h, 4, h);
  }
}
The lowest 64 bins of the microphone's spectrum as bars, read straight from the analyser. Whistle and watch the bars move. Asks for the microphone

Related

  • OAVp5.mic.start() Turns the microphone on and returns a promise that says whether it worked: the same as startMic().
  • OAVp5.mic.state Where the microphone is: 'off', 'asking', 'on', 'denied' or 'unavailable'.
  • bass How much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).

p5/oav-p5.js, line 871

OAVp5.mic.bass#

variableoav-p5
OAVp5.mic.bass

Also: OAVp5.mic.mid OAVp5.mic.treble

The same 0 to 1 low, middle and high energy as bass, mid and treble, read without ever turning the microphone on.

The plain bass, mid and treble ask for the microphone when first read, and a sketch that already has a variable of that name keeps it. OAVp5.mic.bass, .mid and .treble are the numbers themselves: always there, never starting anything. They are also the answer when the plain names are taken: on strudel.cc, where Strudel owns bass, this is what the card tells you to read.

Type
Number
Range
0..1

bass is 20–250 Hz, mid 250–2000 Hz and treble 2000–8000 Hz, scaled by the sensitivity and held to 1. Updated once per display frame while the microphone is on; 0 otherwise.

Notes

  • The three share this entry because they behave the same: only the frequency range differs.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
  startMic();
}

function draw() {
  background(20);
  const bands = [OAVp5.mic.bass, OAVp5.mic.mid, OAVp5.mic.treble];
  for (let i = 0; i < 3; i++) {
    fill(255, 80 + i * 70, 120);
    rect(60 + i * 110, height - 20, 80, -20 - bands[i] * 240);
  }
}
Three bars for low, middle and high, written with the raw members. Whistle for the right one, hum for the left. Asks for the microphone

Related

  • bass How much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
  • OAVp5.mic.level The same 0 to 1 loudness as micLevel, read without ever turning the microphone on.
  • micSensitivity() Makes the microphone more or less sensitive: micSensitivity(2) is twice as sensitive, micSensitivity() says what it is now.
  • startMic() Turns the microphone on and returns a promise that says whether it worked.

p5/oav-p5.js, line 871

OAVp5.mic.db#

variableoav-p5
OAVp5.mic.db

The loudness of the microphone in decibels relative to full scale (dBFS): 0 is as loud as it gets, −100 is silence or the microphone being off.

Decibels are what an ear hears: each 6 dB is twice the amplitude, so a number in dB moves the way loudness feels. micLevel maps it onto 0 to 1 between the room's noise floor (+6 dB) and −10 dBFS. Use db directly when you want your own scale, for example map(OAVp5.mic.db, -60, -10, 0, 1).

Type
Number
Range
-100..0

20 * log10(rms), measured from OAVp5.mic.rms. −100 when the level is below about −100 dB and while the microphone is off. Updated once per display frame.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startMic();
}

function draw() {
  background(20);
  fill(255);
  text(nf(OAVp5.mic.db, 1, 1) + ' dBFS', 20, 30);
  const own = constrain(map(OAVp5.mic.db, -60, -10, 0, 1), 0, 1);   // an own scale: -60 dB to -10 dB
  circle(width / 2, height / 2, 50 + own * 300);
}
The level in dBFS, and a circle on an own scale from −60 to −10 dB. Allow the microphone and talk. Asks for the microphone

Related

  • OAVp5.mic.rms The raw loudness of the microphone: a smoothed RMS, 0 to 1, before oav-p5 turns it into the decibel scale of micLevel.
  • OAVp5.mic.floor The room's own noise as oav-p5 has measured it, in dBFS: the level that micLevel treats as silence (plus 6 dB).
  • OAVp5.mic.level The same 0 to 1 loudness as micLevel, read without ever turning the microphone on.
  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).

p5/oav-p5.js, line 872

OAVp5.mic.error#

variableoav-p5
OAVp5.mic.error

Why the microphone did not start, as the browser's error name such as 'NotAllowedError'; null while there is no error.

For a sketch that wants to say more than a card does: if (OAVp5.mic.error === 'NotAllowedError') …. The card beside the 🎹 already tells the user what to press for each of these, in their language; this is the same fact for your code.

Type
String | null

null at the start and again once the microphone is on. After a failure: the browser's error name ('NotAllowedError' or 'SecurityError' for a refusal, 'NotFoundError' when there is no microphone, 'NotReadableError' when another program holds it) or 'no getUserMedia' on a page that cannot ask.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startMic();
}

function draw() {
  background(20);
  fill(255);
  text('state: ' + OAVp5.mic.state, 20, 30);
  text('error: ' + OAVp5.mic.error, 20, 60);   // null while all is well
}
Prints the state and the error. Refuse the microphone to see "NotAllowedError". Asks for the microphone

Related

  • OAVp5.mic.state Where the microphone is: 'off', 'asking', 'on', 'denied' or 'unavailable'.
  • startMic() Turns the microphone on and returns a promise that says whether it worked.
  • micIsOn Whether the microphone is running right now.

p5/oav-p5.js, line 871

OAVp5.mic.floor#

variableoav-p5
OAVp5.mic.floor

The room's own noise as oav-p5 has measured it, in dBFS: the level that micLevel treats as silence (plus 6 dB).

A humming classroom should not read as loud, so micLevel is measured from the room itself: this floor follows the quietest recent readings. It drops within a second when the room gets quieter and rises at most 1.5 dB a second when it gets noisier, so a voice never becomes the new silence. 6 dB above the floor reads 0 on micLevel.

Type
Number
Range
-70..-45

Starts at −60 and is always held between −70 and −45. Updated once per display frame while the microphone is on.

Notes

  • Stopping the microphone does not reset it: the next start continues from the last floor.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startMic();
}

function draw() {
  background(20);
  fill(255);
  text('level ' + nf(OAVp5.mic.db, 1, 1) + ' dBFS', 20, 30);
  text('room noise ' + nf(OAVp5.mic.floor, 1, 1) + ' dBFS', 20, 60);
  text('micLevel ' + nf(OAVp5.mic.level, 1, 2), 20, 90);
}
Shows the level, the room's noise floor and the resulting micLevel. Keep quiet for a few seconds and watch the floor settle. Asks for the microphone

Related

  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • OAVp5.mic.db The loudness of the microphone in decibels relative to full scale (dBFS): 0 is as loud as it gets, −100 is silence or the microphone being off.
  • OAVp5.mic.rms The raw loudness of the microphone: a smoothed RMS, 0 to 1, before oav-p5 turns it into the decibel scale of micLevel.
  • OAVp5.mic.level The same 0 to 1 loudness as micLevel, read without ever turning the microphone on.

p5/oav-p5.js, line 872

OAVp5.mic.level#

variableoav-p5
OAVp5.mic.level

The same 0 to 1 loudness as micLevel, read without ever turning the microphone on.

micLevel is a getter that returns this number and, the first time it is read, asks for the microphone. OAVp5.mic.level is the number itself, so reading it is safe anywhere: before the user has chosen, with ?oav_mic=click, or in a library. Start the microphone with startMic() or 🎤, then it moves. How it is measured (decibels, the room as zero, −10 dBFS as 1) is described under micLevel.

Type
Number
Range
0..1

Updated once per display frame while the microphone is on; 0 while it is off, asking, blocked or missing.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startMic();
}

function draw() {
  background(20);
  circle(width / 2, height / 2, 50 + OAVp5.mic.level * 300);
}
The loudness circle, written with the raw number instead of micLevel. Allow the microphone and talk. Asks for the microphone

Related

  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • OAVp5.mic.rms The raw loudness of the microphone: a smoothed RMS, 0 to 1, before oav-p5 turns it into the decibel scale of micLevel.
  • OAVp5.mic.db The loudness of the microphone in decibels relative to full scale (dBFS): 0 is as loud as it gets, −100 is silence or the microphone being off.
  • OAVp5.mic.floor The room's own noise as oav-p5 has measured it, in dBFS: the level that micLevel treats as silence (plus 6 dB).
  • startMic() Turns the microphone on and returns a promise that says whether it worked.

p5/oav-p5.js, line 871

OAVp5.mic.on()#

functionoav-p5
OAVp5.mic.on(name, fn)

Listens to a microphone event ('beat', 'kick', 'snare' or 'hat') without defining a global function, and returns a function that stops listening.

The listener version of beatDetected(), kickDetected(), snareDetected() and hatDetected(): any number of them, added and removed while the sketch runs, with no global names taken. An error inside fn shows in the console and on a card.

Before you call it

Listening does not turn the microphone on: call startMic(), press 🎤, or read micLevel too, or nothing will ever fire. (Defining the global beatDetected() is different: oav-p5 sees that function and starts the microphone at the first click.) These events come from the microphone only, not from the drum machine.

Parameters

NameTypeDescription
nameString'beat', 'kick', 'snare' or 'hat'. Anything else throws an error that lists them.
fnFunctionCalled as fn(level) with how strong the hit was, 0 to 1: the same number beatDetected(level) and kickDetected(level) get.

Returns

Function A function that takes fn off again: const off = OAVp5.mic.on(…); off();.

Notes

  • When the drum machine plays, the global beatDetected(), kickDetected() and the others are also called for its hits. A listener added with OAVp5.mic.on() hears the microphone only.

Examples

jsOpenProcessing ↗
let flash = 0;

function setup() {
  createCanvas(400, 300);
  startMic();
  OAVp5.mic.on('beat', (level) => { flash = 0.4 + level * 0.6; });   // louder beat, brighter flash
}

function draw() {
  background(20 + flash * 200);
  flash *= 0.9;
}
A flash on every beat the microphone hears, brighter for a louder one. Clap or tap the desk. Asks for the microphone

Related

  • beatDetected() You define this function; oav-p5 calls it each time the microphone (or the drum machine) hears a beat.
  • kickDetected() You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
  • OAVp5.on() Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
  • beat True for one frame each time the microphone (or the drum machine) hears a beat.
  • startMic() Turns the microphone on and returns a promise that says whether it worked.

p5/oav-p5.js, line 875

OAVp5.mic.rms#

variableoav-p5
OAVp5.mic.rms

The raw loudness of the microphone: a smoothed RMS, 0 to 1, before oav-p5 turns it into the decibel scale of micLevel.

The root mean square of the samples in the latest frame, smoothed a little. It is linear, the way a waveform is drawn, so it suits a quick check or your own scale. micLevel is built from it: OAVp5.mic.db is the same number in dBFS, and OAVp5.mic.floor is the room's own noise, in dBFS too.

Type
Number
Range
0..1

Updated once per display frame while the microphone is on; 0 when it is off. A speaking voice into a laptop is a small number, in the range 0.01 to 0.05 according to the comment in the code, which is why micLevel works in decibels.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startMic();
}

function draw() {
  background(20);
  fill(255);
  text('rms ' + nf(OAVp5.mic.rms, 1, 4), 20, 30);
  rect(20, 60, min(OAVp5.mic.rms * 10, 1) * (width - 40), 20);   // magnified 10 times: a voice is small
}
The raw RMS as a number and a bar, magnified ten times because speech is a small number. Allow the microphone and talk. Asks for the microphone

To be confirmed

Typical values for speech and music depend on the microphone and the room. The 0.01 to 0.05 for a voice is the code comment's figure and was not measured here.

Related

  • OAVp5.mic.db The loudness of the microphone in decibels relative to full scale (dBFS): 0 is as loud as it gets, −100 is silence or the microphone being off.
  • OAVp5.mic.floor The room's own noise as oav-p5 has measured it, in dBFS: the level that micLevel treats as silence (plus 6 dB).
  • OAVp5.mic.level The same 0 to 1 loudness as micLevel, read without ever turning the microphone on.
  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).

p5/oav-p5.js, line 872

OAVp5.mic.sensitivity#

variableoav-p5
OAVp5.mic.sensitivity

Also: OAVp5.mic.gain

The microphone's sensitivity: 1 normally, 2 for twice as sensitive. micSensitivity() sets it; OAVp5.mic.gain is the old name.

The number multiplies what the microphone hears: on micLevel 2 adds 6 dB, on bass, mid and treble it multiplies by 2. Read it to show the current setting, and set it with micSensitivity(), which checks the number.

Type
Number
Range
0.05..20

Starts at 1. micSensitivity(v) keeps what you give it between 0.05 and 20.

Notes

  • OAVp5.mic.gain is the same value under the name oav-p5 0.3.0 used. Assigning to either works (OAVp5.mic.gain = 2) but is not held between 0.05 and 20, and gain falls back to 1 for a number that is not above 0.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  micSensitivity(3);   // a quiet or far-away microphone
}

function draw() {
  background(20);
  fill(255);
  text('sensitivity ' + OAVp5.mic.sensitivity, 20, 30);   // 3
  circle(width / 2, height / 2, 50 + micLevel * 300);
}
micSensitivity(3) sets the number, and OAVp5.mic.sensitivity reads it back. Asks for the microphone

Related

  • micSensitivity() Makes the microphone more or less sensitive: micSensitivity(2) is twice as sensitive, micSensitivity() says what it is now.
  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • bass How much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.

p5/oav-p5.js, line 869

OAVp5.mic.start()#

functionoav-p5
OAVp5.mic.start()

Turns the microphone on and returns a promise that says whether it worked: the same as startMic().

startMic() is this function: see it for what must happen first (a click, https or localhost). OAVp5.mic.start() also lifts the "stopped by you" mark that stopMic() leaves, which is what lets a later read of micLevel switch the microphone on again.

Returns

Promise<Boolean> Resolves to true once the microphone is on, false if it was blocked, is missing, or the mic option is off.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  OAVp5.mic.start().then((ok) => console.log(ok ? 'microphone on' : 'no microphone'));
}

function draw() {
  background(20);
  fill(255);
  text(OAVp5.mic.state === 'on' ? 'listening' : 'click the canvas to start the microphone', 20, 30);
}
Nothing happens until you click: then the browser asks, and the text changes when the microphone is on. Asks for the microphone

Related

  • startMic() Turns the microphone on and returns a promise that says whether it worked.
  • OAVp5.mic.stop() Turns the microphone off and sets every microphone value back to 0: the same as stopMic().
  • OAVp5.mic.state Where the microphone is: 'off', 'asking', 'on', 'denied' or 'unavailable'.
  • mic Who may turn the microphone on: auto (the first read of micLevel and friends), click (only 🎤 or startMic()) or off (nobody).

p5/oav-p5.js, line 873

OAVp5.mic.state#

variableoav-p5
OAVp5.mic.state

Where the microphone is: 'off', 'asking', 'on', 'denied' or 'unavailable'.

'off': not started, or stopped. 'asking': the browser is waiting for the user's answer. 'on': sound is coming in, and micIsOn is true. 'denied': the permission was refused (by the user, by the computer's settings, or by an iframe without allow="microphone"). 'unavailable': there is no microphone, none the browser can open, or the page is not on https or localhost. When it is not 'on', OAVp5.mic.error has the browser's reason and a card beside the 🎹 says what to press.

Type
String
Range
'off' | 'asking' | 'on' | 'denied' | 'unavailable'

Changes when you start or stop the microphone and when the browser answers. 'off' until something starts it. Reading it never starts it.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startMic();
}

function draw() {
  background(OAVp5.mic.state === 'on' ? 20 : color(90, 30, 30));
  fill(255);
  text('microphone: ' + OAVp5.mic.state, 20, 30);
  if (OAVp5.mic.state === 'denied') text('click the lock left of the address and allow it', 20, 60);
}
The background is red until the microphone is on. The state goes "asking", then "on" when you allow it, or "denied" when you refuse. Asks for the microphone

Related

  • micIsOn Whether the microphone is running right now.
  • OAVp5.mic.error Why the microphone did not start, as the browser's error name such as 'NotAllowedError'; null while there is no error.
  • startMic() Turns the microphone on and returns a promise that says whether it worked.
  • mic Who may turn the microphone on: auto (the first read of micLevel and friends), click (only 🎤 or startMic()) or off (nobody).
  • OAVp5.check() Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.

p5/oav-p5.js, line 868

OAVp5.mic.stop()#

functionoav-p5
OAVp5.mic.stop()

Turns the microphone off and sets every microphone value back to 0: the same as stopMic().

stopMic() is this function. It stops the microphone's tracks and closes its audio context, so the browser's "recording" indicator goes away, and OAVp5.mic.level, bass, mid, treble, rms read 0 and db reads −100. After it, only OAVp5.mic.start() or 🎤 turns the microphone back on: reading micLevel does not.

Returns

Promise<Boolean> Resolves to false, the new state of the microphone.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startMic();
}

function mousePressed() {
  OAVp5.mic.stop();   // the browser's recording indicator goes away
}

function draw() {
  background(20);
  fill(255);
  text('microphone: ' + OAVp5.mic.state + ' (click the canvas to stop it)', 20, 30);
  circle(width / 2, height / 2, 50 + OAVp5.mic.level * 300);
}
Click the canvas: the microphone stops, the state says "off" and the circle drops to its smallest size. Asks for the microphone

Related

  • stopMic() Turns the microphone off and sets every microphone value back to 0.
  • OAVp5.mic.start() Turns the microphone on and returns a promise that says whether it worked: the same as startMic().
  • OAVp5.mic.state Where the microphone is: 'off', 'asking', 'on', 'denied' or 'unavailable'.
  • micIsOn Whether the microphone is running right now.

p5/oav-p5.js, line 873

@openav/audio

AudioAnalyzer#

class@openav/audio
new AudioAnalyzer({ signals = null, fftSize = 2048, smooth = 0.7 } = {})

Listens to the microphone, an audio element or any Web Audio node, and turns it into loudness, three bands, onsets and kick, snare and hat hits.

An AudioAnalyzer wraps one Web Audio AnalyserNode. You give it a sound (enableMic(), enableElement(el), or connect your own node into input()), call update() once per frame, and it returns a plain object of numbers, which it also publishes as audio/* signals if you gave it a signals bus. This is the analysis under the microphone of a p5 sketch: micLevel is its rms turned into decibels, bass, mid and treble are its low, mid and high, and beat and kickDetected() come from its onset and kick.

update() returns null until a source is enabled. After that it returns { rms, peak, low, mid, high, centroid, onset, kick, snare, hat }. rms is the smoothed loudness and peak the loudest sample of the moment, both 0..1 of full scale (from float samples where the browser has them, so a quiet room is measured, not rounded to one 8-bit step). A voice is a small rms: scale it, or map it through a window. low, mid and high are the average strength of the spectrum in 20–250 Hz, 250–2000 Hz and 2000–8000 Hz, 0..1, taken from the analyser's byte spectrum, so they are not decibels and a faint band stays near 0. centroid is the spectrum's centre of mass, 0..1 of the way from 0 Hz to half the sample rate (the brightness). onset is true for the one update() in which the loudness jumped over its adaptive floor, with at least 100 ms between two onsets.

kick, snare and hat are { hit, level, env }, found by looking for a sudden rise in one band of the spectrum, with no machine learning: kick 20–120 Hz, snare 150–800 Hz, hat 6000–14000 Hz. hit is true on the update that found one (at least 90, 90 and 60 ms apart), level is the band's strength now, and env is 0..1: it jumps to min(1, level × 2) on a hit and then shrinks by 12% every update(), about 120 ms at 60 frames a second. Because it decays per update, it falls faster if you call update() less often.

With a signals bus, these are published: audio/rms, audio/peak, audio/band/low, audio/band/mid, audio/band/high, audio/centroid, audio/kick/env, audio/snare/env and audio/hat/env (continuous, 0..1), and the pulses audio/onset { rms }, audio/kick { level }, audio/snare { level } and audio/hat { level }. Pulses fire only when something was found.

Before you call it

Browsers start audio only after a user gesture. enableMic() asks for the microphone, and the analyser's AudioContext stays suspended until the page has had a click or a key: update() calls resume() on every frame, so the numbers start by themselves after the first click. Until then they are all 0. The page needs https or localhost for the microphone.

Parameters

NameTypeDescription
signalsSignalsA Signals bus to publish into. With one, the audio/* signals are defined when you construct the analyzer and set on every update(). Without one, update() only returns the numbers. optional default null
fftSizeNumberThe size of the analyser's FFT, a power of two from 32 to 32768. The spectrum has half as many bins. optional default 2048
smoothNumberHow much of the previous rms is kept each update, 0..1. Higher is smoother and slower: rms = rms × smooth + now × (1 − smooth). optional default 0.7

Members

NameDescription
enableMic()methodAsks the browser for the microphone, with echo cancellation, noise suppression and auto gain off, and connects it. Returns a promise that resolves to true. It rejects when the permission is refused or there is no microphone, with the browser's error (NotAllowedError, NotFoundError…).
enableElement(el)methodAnalyses an <audio> or <video> element: enableElement(el) returns true. The element keeps playing to the speakers. Web Audio takes an element in only once, so a second enableElement() for the same element throws a browser error.
input()methodReturns the AnalyserNode (creating the context first), so you can bring your own sound: node.connect(audio.input()). Any node of audio.ctx works; for Tone.js, connect its raw node. Nothing is routed to the speakers by this.
update()methodCall it once per frame. Reads the analyser, works out the numbers, publishes the signals if there is a bus, and returns the object { rms, peak, low, mid, high, centroid, onset, kick, snare, hat }, or null when no source is enabled yet. It also resumes a suspended context.

Notes

  • It makes its own AudioContext, so p5.sound or Tone.js can run beside it. To analyse a sound that lives in another context, connect a node of audio.ctx into input(), or use enableElement().
  • enableMic() turns off echo cancellation, noise suppression and automatic gain, because they change the loudness you want to measure. On a laptop that plays the sound it listens to, expect the speakers to leak into the microphone.

Examples

js
import { AudioAnalyzer } from '@openav/audio';

const audio = new AudioAnalyzer();
await audio.enableMic();                      // the browser asks for the microphone
const g = document.getElementById('c').getContext('2d');

(function frame() {
  const f = audio.update();                   // { rms, peak, low, mid, high, ... }
  g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
  g.fillStyle = '#ffd166';
  g.fillRect(20, 130, Math.min(1, f.rms * 6) * 360, 40);   // a voice is a small rms: scale it
  requestAnimationFrame(frame);
})();
module Allow the microphone, click the page once, and talk: a bar grows with the loudness. Asks for the microphone
js
import { AudioAnalyzer } from '@openav/audio';

const audio = new AudioAnalyzer();             // no signals bus: update() just returns the numbers
const input = audio.input();                    // makes the AudioContext (audio.ctx) and the AnalyserNode
const osc = new OscillatorNode(audio.ctx, { frequency: 100 });
osc.connect(input);                             // any node of audio.ctx can feed the analyser
osc.start();

let frames = 0;
(function frame() {
  const f = audio.update();                     // also resumes the context once the page has had a click
  if (++frames < 40) return requestAnimationFrame(frame);
  console.log(audio.ctx.state, f.low > f.high);  // running true: a 100 Hz tone sits in the low band
  console.log(f.rms > 0.1, f.centroid < 0.1);    // true true
})();
module No microphone needed: a 100 Hz tone is fed into the analyser through input(). After 40 frames the log says the low band is louder than the high one.
js
import { AudioAnalyzer } from '@openav/audio';
import { Signals } from '@openav/core';

const signals = new Signals();
const audio = new AudioAnalyzer({ signals });         // defines the audio/* signals at once
console.log(signals.list().length);                    // 13
signals.on('audio/onset', ({ rms }) => console.log('onset', rms.toFixed(2)));
signals.on('audio/kick', ({ level }) => console.log('kick', level.toFixed(2)));
console.log(signals.get('audio/rms'));                 // 0, nothing heard yet
console.log(audio.update());                           // null until a source is enabled
module With a signals bus the analyzer defines thirteen audio/* signals up front. Listen to a pulse with signals.on; the handlers run once a source is enabled and a hit is found.

To be confirmed

Typical rms values for speech differ between the notes of this project: the handbook (docs/inputs.md) says speech often sits around 0.05 to 0.2, a comment in oav-p5 says 0.01 to 0.05. They depend on the microphone and the room, and have not been measured here, so no typical number is given.

Related

  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • bass How much of the microphone's sound is low (bass), middle (mid) or high (treble), each from 0 to 1.
  • beat True for one frame each time the microphone (or the drum machine) hears a beat.
  • kickDetected() You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
  • micSensitivity() Makes the microphone more or less sensitive: micSensitivity(2) is twice as sensitive, micSensitivity() says what it is now.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).

import { AudioAnalyzer } from '@openav/audio'; packages/audio/index.js, line 24

Sound and instruments#

Every key you play is heard: startSound(), @openav/sound. 33

oav-p5

soundIsOn#

variableoav-p5
soundIsOn

Whether notes are being heard right now: true once startSound() has finished, false before and after.

Read it in draw() like mouseIsPressed: if (!soundIsOn) text('click to hear', 20, 30);. It changes only when the state of the sound changes, not every frame. Reading it never starts anything (unlike micLevel), so it is safe to check anywhere.

false does not mean startSound() was never called. To tell "loading" from "waiting for the click" from "failed", read OAVp5.sound.state, and OAVp5.sound.error for the reason of a failure.

Type
Boolean

True only while OAVp5.sound.state is 'on'. It is false at the start, while sound waits for a click, while it loads, after stopSound() and after an error.

Before it has a value

It becomes true only after startSound() (or 🔊 in the 🎹 panel, or ?oav_sound=piano), and after the click the browser wants.

Notes

  • A sketch that already declares its own variable called soundIsOn keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.sound.state === 'on' instead.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startSound('organ');
}

function draw() {
  background(20);
  fill(soundIsOn ? color(120, 220, 140) : color(220, 160, 60));
  circle(width / 2, height / 2, 80);
  fill(255);
  textAlign(CENTER);
  text(soundIsOn ? 'sound is on' : 'click the canvas to turn sound on', width / 2, height - 40);
}
An amber circle until you click, then a green one: that is the moment soundIsOn turns true.

Related

  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • stopSound() Turns sound off and lets every ringing note go; the instrument stays loaded, so startSound() brings it back at once.
  • OAVp5.sound.state Where sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.
  • micIsOn Whether the microphone is running right now.
  • drumsAreOn Whether the drum machine is playing right now.
  • recordingIsOn Whether a recording is running right now.

p5/oav-p5.js, line 316

startSound()#

functionoav-p5
startSound([id])

Also: OAVp5.sound.start

Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.

Call it once in setup(): startSound('organ'). From then on every note you play is heard: a MIDI keyboard, the 🎹 on screen, a phone, midi.play(60, 0.8). Your own notePressed() still gets the note, and the sound is made before the sketch is told, so a slow callback never delays it. Pads on MIDI channel 10 (the General MIDI drum channel) play the drum kit instead of the instrument, and the sustain pedal holds notes.

The instrument ids are piano, epiano, organ, strings, choir, harp, marimba, vibraphone, xylophone, glockenspiel, music-box and the synth presets synth/pad, synth/saw-lead, synth/square-lead, synth/pluck, synth/supersaw, synth/bass, synth/acid, synth/brass, synth/bell, synth/sub and synth/chip. Calling startSound(id) while sound plays switches the instrument. An id that does not exist is listed in the console together with the real ones: an instrument that is already playing stays, and at the first call the warm pad synth/pad plays.

Nothing is loaded before the first call (or before 🔊 in the 🎹 panel): @openav/sound comes from next to oav-p5.js and Tone.js 15 from jsDelivr (the tone option points it somewhere else). Only the piano plays recorded samples, about 2 MB the first time; the electric piano sounds meanwhile (or, when you switch from another instrument, that one keeps sounding), and OAVp5.sound.instrument changes to piano when the samples are in. Every other instrument is made by Tone.js and has no files to load.

Before you call it

A browser lets a page make sound only after a click or a key press on the page, and a key on a MIDI keyboard does not count. Calling startSound() in setup() is fine: OAVp5.sound.state is 'waiting' and a card beside the 🎹 says "Sound is ready but silent"; the first click or key press turns sound on. soundIsOn tells when it has.

Parameters

NameTypeDescription
idStringAn instrument id, for example 'organ' or 'synth/pluck' (the list is in the description). Leave it out to keep the instrument chosen before; at the very first call that is the piano. A value that is not a non-empty string counts as left out. optional default 'piano'

Returns

Promise<Boolean> Resolves to true once sound is on. When the browser wants a click first, that is after the first click or key press on the page. Resolves to false when the loading option sound is off, when stopSound() came first, or when Tone.js could not be loaded.

Notes

  • No code at all: ?oav_sound=piano on the page address (any instrument id; on OpenProcessing ?sketch_oav_sound=piano) makes any sketch heard from the first click, and ?oav_sound=off keeps it silent: startSound() then does nothing and resolves to false. In the 🎹 panel, 🔊 turns sound on and off and shows a picker with every instrument.
  • Sound is separate from the drum machine: startDrums() has its own kit and is heard from the first click without startSound().

Examples

jsOpenProcessing ↗
let playing = 60;

function setup() {
  createCanvas(400, 300);
  startSound('synth/pluck');   // every key you play is heard
}

function mousePressed() {
  playing = 48 + floor(mouseX / 25);   // low on the left, high on the right
  midi.play(playing, 0.8);
}

function mouseReleased() {
  midi.play(playing, 0);
}

function draw() {
  background(soundIsOn ? 30 : 90);
  fill(255);
  textAlign(CENTER, CENTER);
  text(soundIsOn ? 'click to play a note' : 'click once to turn sound on', width / 2, height / 2);
}
The first click turns sound on and the background darkens; the clicks after that play a note, higher to the right.
jsOpenProcessing ↗
const KEYS = { a: 60, s: 62, d: 64, f: 65, g: 67, h: 69, j: 71, k: 72 };
let dots = [];

function setup() {
  createCanvas(400, 300);
  noStroke();
  startSound('epiano');
}

function keyPressed() {
  if (KEYS[key]) midi.play(KEYS[key], 0.8);
}

function keyReleased() {
  if (KEYS[key]) midi.play(KEYS[key], 0);
}

function notePressed(n, v) {
  dots.push({ x: map(n, 60, 72, 30, width - 30), y: height, v });
}

function draw() {
  background(20);
  for (const d of dots) {
    fill(255, 160, 80);
    circle(d.x, d.y, 20 + d.v * 30);
    d.y -= 2;
  }
  dots = dots.filter((d) => d.y > -40);
  fill(255);
  text(soundIsOn ? 'play with a s d f g h j k' : 'press a key once to turn sound on', 20, 24);
}
Press a, s, d, f, g, h, j, k: each key sounds and a dot floats up from the matching place.
jsOpenProcessing ↗
const IDS = ['synth/pluck', 'organ', 'marimba', 'piano'];

function setup() {
  createCanvas(400, 300);
  startSound(IDS[0]);
}

function keyPressed() {
  const i = Number(key) - 1;
  if (IDS[i]) startSound(IDS[i]);   // switches while it plays
}

function draw() {
  background(20);
  fill(255);
  const s = OAVp5.sound.status;   // the instrument being loaded
  text('keys 1-4 pick an instrument', 20, 30);
  text('playing: ' + OAVp5.sound.instrument, 20, 60);
  text('loading: ' + s.id + ' (' + s.state + ', ' + nf(s.progress * 100, 1, 0) + '%)', 20, 90);
}
Press 4 for the piano: "loading" shows its samples arriving while "playing" still names the instrument you hear, and flips to piano when it is ready.

Related

  • stopSound() Turns sound off and lets every ringing note go; the instrument stays loaded, so startSound() brings it back at once.
  • soundIsOn Whether notes are being heard right now: true once startSound() has finished, false before and after.
  • OAVp5.sound.state Where sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.
  • OAVp5.sound.instrument The id of the instrument you hear right now, or null before sound has loaded.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 1413

stopSound()#

functionoav-p5
stopSound()

Also: OAVp5.sound.stop

Turns sound off and lets every ringing note go; the instrument stays loaded, so startSound() brings it back at once.

It releases every note that is ringing, sets soundIsOn to false and OAVp5.sound.state to 'off'. Notes you play afterwards are not heard (the sketch's notePressed() is still called). Tone.js and the instrument stay loaded: startSound() afterwards is on again at once, without a new download and without waiting for another click.

It also works while sound is still loading or waiting for the first click: the pending startSound() then resolves to false.

Returns

Promise<Boolean> A promise that is already resolved to false, the new state of the sound.

Notes

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startSound('synth/pluck');
  textAlign(CENTER, CENTER);
}

function keyPressed() {
  if (key === 's') {
    if (soundIsOn) stopSound(); else startSound();
  }
}

function draw() {
  background(soundIsOn ? 40 : 100);
  fill(255);
  text('press s: sound is ' + (soundIsOn ? 'on' : 'off'), width / 2, height / 2);
}
Press s to switch sound off and on; the background is dark while sound is on.

Related

  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • soundIsOn Whether notes are being heard right now: true once startSound() has finished, false before and after.
  • OAVp5.sound.state Where sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.
  • stopDrums() Stops the drum machine and hides its card.
  • stopMic() Turns the microphone off and sets every microphone value back to 0.

p5/oav-p5.js, line 1414

OAVp5.sound#

objectoav-p5
OAVp5.sound

The object behind the sound names: the state, the instrument, the error, the loading progress and the loaded parts of @openav/sound.

OAVp5.sound (also reachable as midi.sound) is what startSound(), stopSound() and soundIsOn are built on. Read state to know where sound is, instrument for what you hear, status for the loading progress and error for a failure. start(id) and stop() are the same as the two global functions. shell, engine and module hand you the loaded @openav/sound parts for the rare case that you need them.

Notes

  • For a sketch the global names are enough. This object is for tools, the console and debugging: OAVp5.sound.state is the first thing to look at when nothing is heard.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startSound('harp');
}

function draw() {
  background(20);
  fill(255);
  const s = OAVp5.sound;
  text('state: ' + s.state, 20, 30);
  text('instrument: ' + s.instrument, 20, 60);
  text('error: ' + s.error, 20, 90);
}
Three lines of the sound object: the state goes from "loading" or "waiting" to "on" after the first click.

Related

  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • stopSound() Turns sound off and lets every ringing note go; the instrument stays loaded, so startSound() brings it back at once.
  • soundIsOn Whether notes are being heard right now: true once startSound() has finished, false before and after.
  • OAVp5.sound.state Where sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.
  • OAVp5.sound.status How the instrument is loading: { id, state, progress, error }, with progress from 0 to 1.
  • OAVp5.drums The object behind the drum names: the state of the machine, its counters, the line of rhythm, its history and the loaded machine.
  • OAVp5.record The object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.

p5/oav-p5.js, line 1325

OAVp5.sound.error#

variableoav-p5
OAVp5.sound.error

The message of the last thing that went wrong while loading sound, or null when nothing has.

It is the error the browser reported when @openav/sound or Tone.js could not be loaded. A card beside the 🎹 already tells the person at the keyboard what to do (try another network, or point the tone option at another copy of Tone.js); read this value to log it or to show your own message. To know whether sound is failing right now, use OAVp5.sound.state, not this.

Type
String | null

Set when the state becomes 'error'. It stays until the page is reloaded: a later try that works does not clear it.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startSound('strings');
  textAlign(CENTER, CENTER);
}

function draw() {
  background(OAVp5.sound.error ? color(120, 30, 30) : 20);
  fill(255);
  text(OAVp5.sound.error ? OAVp5.sound.error : 'no error', width / 2, height / 2);
}
Stays on "no error" when sound loads; if Tone.js cannot be reached the background turns red and the message is printed.

Related

  • OAVp5.sound.state Where sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.
  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • soundIsOn Whether notes are being heard right now: true once startSound() has finished, false before and after.

p5/oav-p5.js, line 1318

OAVp5.sound.instrument#

variableoav-p5
OAVp5.sound.instrument

The id of the instrument you hear right now, or null before sound has loaded.

After a first startSound('piano') the piano's samples (about 2 MB) download first, and the electric piano plays meanwhile, so the first key is never silent: instrument says 'epiano' until the piano is ready and 'piano' after. When you switch to the piano from another instrument, that one keeps sounding and instrument names it until the samples are in. How far the loading is, and which instrument it is on, is in OAVp5.sound.status.

Type
String | null

Changes when the instrument changes, which for a sampled instrument is when its samples are in.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startSound('piano');
  textAlign(CENTER, CENTER);
}

function draw() {
  background(20);
  fill(255);
  text(OAVp5.sound.instrument || 'nothing loaded yet', width / 2, height / 2);
}
Shows the instrument you hear: epiano while the piano samples arrive, then piano.

Related

  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • OAVp5.sound.status How the instrument is loading: { id, state, progress, error }, with progress from 0 to 1.
  • OAVp5.sound.state Where sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.

p5/oav-p5.js, line 1318

OAVp5.sound.state#

variableoav-p5
OAVp5.sound.state

Where sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.

'off' is where it starts and where stopSound() leaves it. startSound() moves it to 'loading' while @openav/sound and Tone.js arrive, to 'waiting' when everything is ready but the browser has not yet had its click, and to 'on' when notes are heard. 'error' means the sound parts could not be loaded (a network, a school firewall or an ad blocker); the message is in OAVp5.sound.error, and calling startSound() again tries once more.

soundIsOn is the same thing reduced to state === 'on'.

Type
String
Range
off | waiting | loading | on | error

Starts as 'off' and changes whenever the sound moves to another stage.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startSound('marimba');
  textAlign(CENTER, CENTER);
  textSize(28);
}

function draw() {
  background(20);
  const hints = { off: 'off', loading: 'loading…', waiting: 'click to hear', on: 'listening', error: 'failed' };
  fill(255);
  text(hints[OAVp5.sound.state], width / 2, height / 2);
}
One word for each stage; "click to hear" is what the sketch shows between a ready sound and the first click.

Related

  • soundIsOn Whether notes are being heard right now: true once startSound() has finished, false before and after.
  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • stopSound() Turns sound off and lets every ringing note go; the instrument stays loaded, so startSound() brings it back at once.
  • OAVp5.sound.error The message of the last thing that went wrong while loading sound, or null when nothing has.
  • OAVp5.sound.status How the instrument is loading: { id, state, progress, error }, with progress from 0 to 1.

p5/oav-p5.js, line 1317

OAVp5.sound.status#

objectoav-p5
OAVp5.sound.status

How the instrument is loading: { id, state, progress, error }, with progress from 0 to 1.

It is about the instrument, not about the whole sound: OAVp5.sound.state is the whole sound. Only the piano has files to load, so for every other instrument progress is 1 almost at once. While a sampler loads, OAVp5.sound.instrument is still the one you hear and status.id is the one on its way (at the very first call it may name the electric piano that stands in, and then the piano).

Type
Object
Range
{ id, state: idle | loading | ready | error, progress: 0..1, error }

A new object on every read. id is the instrument the loader is working on, state is 'idle' before anything loads, 'loading', 'ready' or 'error', and progress is the fraction of the files loaded.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startSound('piano');
  noStroke();
}

function draw() {
  background(20);
  const s = OAVp5.sound.status;
  fill(60);
  rect(40, 140, 320, 16);
  fill(120, 220, 140);
  rect(40, 140, 320 * s.progress, 16);
  fill(255);
  text(s.id + ': ' + s.state, 40, 120);
}
A progress bar for the piano samples; it fills within a few seconds and the label changes to "ready".

Related

  • OAVp5.sound.instrument The id of the instrument you hear right now, or null before sound has loaded.
  • OAVp5.sound.state Where sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.
  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.

p5/oav-p5.js, line 1322

@openav/sound

BUILTIN_INSTRUMENTS#

constant@openav/sound
BUILTIN_INSTRUMENTS = array, 22 items

The 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.

text
id                 category  name
piano              keys      Grand piano (Salamander), 30 samples, CC-BY 3.0
epiano             keys      Electric piano (two FM pairs and a tremolo)
xylophone          mallets   Xylophone (sounds an octave up)
marimba            mallets   Marimba
vibraphone         mallets   Vibraphone (motor tremolo, key-up damps)
glockenspiel       mallets   Glockenspiel (two octaves up)
music-box          mallets   Music box (two octaves up)
strings            strings   String ensemble
harp               plucked   Harp (Karplus-Strong)
organ              organ     Drawbar organ (rotary tremolo)
choir              voice     Choir pad (aah)
synth/pad          synth     Warm pad (the default)
synth/saw-lead     synth     Saw lead
synth/square-lead  synth     Square lead
synth/pluck        synth     Pluck
synth/supersaw     synth     Supersaw
synth/bass         synth     Analog bass
synth/acid         synth     Acid
synth/brass        synth     Synth brass
synth/bell         synth     FM bell
synth/sub          synth     Sub bass
synth/chip         synth     Chip (8-bit)

Loading the package registers all of them, except those whose id you registered first. Each one is a small file in the package's instruments/ folder. The piano loads its samples only when it is picked, and falls back to the electric piano if they cannot be fetched. The organ, the strings, the choir, the electric piano and the vibraphone bring params of their own.

Type
Object[]

An array of 22 instrument definitions, in the order of the table.

Examples

js
import { BUILTIN_INSTRUMENTS, getInstrument } from '@openav/sound';

console.log(BUILTIN_INSTRUMENTS.length);                                       // 22
console.log(BUILTIN_INSTRUMENTS.slice(0, 3).map((d) => d.id).join(' '));      // piano epiano xylophone
console.log(getInstrument('piano') === BUILTIN_INSTRUMENTS[0]);                // true: they are the registered ones
console.log(getInstrument('organ').params.map((p) => p.key).join(' '));        // drawbars rotor
module How many built-in instruments there are, the first three, and that the registry holds these very objects. The organ lists the two params of its own.

Related

  • listInstruments() Lists every registered instrument, in the order they were registered, or only the ones of one category.
  • SYNTH_PRESETS The 11 synth presets as plain data: pad, saw lead, square lead, pluck, supersaw, bass, acid, brass, bell, sub and chip.
  • DEFAULT_INSTRUMENT The instrument used when none is named: 'synth/pad', the warm three-saw pad that was the framework's first voice.
  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • getInstrument() Finds a registered instrument definition by its id, or null.

import { BUILTIN_INSTRUMENTS } from '@openav/sound'; packages/sound/index.js, line 61

CATEGORIES#

constant@openav/sound
CATEGORIES = array, 9 items

The instrument categories in picker order: keys, mallets, strings, plucked, organ, voice, synth, drums and other.

The category of an instrument definition picks its group in the picker. A category that is not in this list is allowed: it sorts after these, and its label is its id. Every category also has a Chinese name, which the picker uses on a Chinese page.

Type
Object[]

Nine items { id, name: { en, zh } }, such as { id: 'mallets', name: { en: 'Mallets', zh: … } }.

Examples

js
import { CATEGORIES } from '@openav/sound';

console.log(CATEGORIES.map((c) => c.id).join(' '));   // keys mallets strings plucked organ voice synth drums other
console.log(CATEGORIES[1].name.en, CATEGORIES[1].name.zh);
module Logs the nine ids in order and the English and Chinese name of the second.

Related

  • instrumentGroups() Groups the instruments by category in picker order: [{ id, label, items }], with empty groups left out.
  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • listInstruments() Lists every registered instrument, in the order they were registered, or only the ones of one category.

import { CATEGORIES } from '@openav/sound'; Also from @openav/sound/registry packages/sound/registry.js, line 32

DEFAULT_INSTRUMENT#

constant@openav/sound
DEFAULT_INSTRUMENT = "synth/pad"

The instrument used when none is named: 'synth/pad', the warm three-saw pad that was the framework's first voice.

toneEngine() starts on it, and falls back to it when you name an instrument that is not registered. Its sound has not changed since the first version.

Type
String

'synth/pad'.

Examples

js
import { DEFAULT_INSTRUMENT, getInstrument, instrumentName, toneEngine } from '@openav/sound';

console.log(DEFAULT_INSTRUMENT, '-', instrumentName(getInstrument(DEFAULT_INSTRUMENT)));   // synth/pad - Warm pad
console.log(toneEngine().instrument === DEFAULT_INSTRUMENT);                                 // true
module Logs the default instrument and its name, and that an engine with no options starts on it.

Related

  • toneEngine() The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
  • BUILTIN_INSTRUMENTS The 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.
  • SYNTH_PRESETS The 11 synth presets as plain data: pad, saw lead, square lead, pluck, supersaw, bass, acid, brass, bell, sub and chip.

import { DEFAULT_INSTRUMENT } from '@openav/sound'; packages/sound/index.js, line 59

getInstrument()#

function@openav/sound
getInstrument(id)

Finds a registered instrument definition by its id, or null.

Use it to check that an id exists before you pass it to toneEngine() or setInstrument(), or to read an instrument's credit, defaults or params.

Parameters

NameTypeDescription
idStringThe id, for example 'piano' or 'synth/pluck'.

Returns

Object | null The definition you registered ({ id, name, category, create, … }), or null.

Examples

js
import { getInstrument, instrumentName } from '@openav/sound';

const piano = getInstrument('piano');
console.log(instrumentName(piano), '/', piano.category);   // Grand piano (Salamander) / keys
console.log(piano.credit.text);                             // Salamander Grand Piano · Alexander Holm · CC-BY 3.0
console.log(getInstrument('theremin'));                     // null
module Reads the name, category and credit of the built-in piano, and shows that an id that is not registered gives null.

Related

  • listInstruments() Lists every registered instrument, in the order they were registered, or only the ones of one category.
  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • instrumentName() Gives an instrument's display name in a language, falling back to English.

import { getInstrument } from '@openav/sound'; Also from @openav/sound/registry packages/sound/registry.js, line 102

instrumentGroups()#

function@openav/sound
instrumentGroups(lang = 'en', list = listInstruments())

Groups the instruments by category in picker order: [{ id, label, items }], with empty groups left out.

It is the data behind the picker, so you can build your own menu, a list or a grid of buttons. A category that is not in CATEGORIES is allowed; it sorts after the known ones and its label is its id.

Parameters

NameTypeDescription
langStringThe language of the group labels. optional default 'en'
listObject[]The instruments to group. Left out, all the registered ones. optional default listInstruments()

Returns

Object[] [{ id, label, items: [definition…] }]: the groups in the order of CATEGORIES, then any categories of your own, sorted by name.

Examples

js
import { instrumentGroups } from '@openav/sound';

for (const g of instrumentGroups('en')) console.log(g.label + ' (' + g.items.length + ')');
// Keys (2), Mallets (5), Strings (1), Plucked (1), Organ (1), Voice (1), Synth (11)
console.log(instrumentGroups('zh')[1].label);   // the Chinese label of the mallets group
module The seven groups the built-in instruments make, with how many each holds (no drums yet, so that group is left out), and a group label in Chinese.

Related

  • listInstruments() Lists every registered instrument, in the order they were registered, or only the ones of one category.
  • CATEGORIES The instrument categories in picker order: keys, mallets, strings, plucked, organ, voice, synth, drums and other.
  • instrumentName() Gives an instrument's display name in a language, falling back to English.
  • mountSoundPicker() Puts an instrument menu on the page: one group per category, a loading bar for samplers and the credit line an instrument asks for.

import { instrumentGroups } from '@openav/sound'; Also from @openav/sound/registry packages/sound/registry.js, line 118

instrumentName()#

function@openav/sound
instrumentName(def, lang = 'en')

Gives an instrument's display name in a language, falling back to English.

Instruments carry name as a string or as { en, zh }. This hides which, so your own menu can just call instrumentName(def, lang).

Parameters

NameTypeDescription
defObjectAn instrument definition. null or undefined gives ''.
langString'en', 'zh'… A longer tag such as 'zh-TW' is tried by its first two letters. optional default 'en'

Returns

String The name in that language, else the English one. A definition whose name is a plain string gives that string.

Examples

js
import { getInstrument, instrumentName } from '@openav/sound';

const bar = getInstrument('glockenspiel');
console.log(instrumentName(bar), instrumentName(bar, 'zh'));   // Glockenspiel, then its Chinese name
console.log(instrumentName(bar, 'fr'));                         // Glockenspiel, English as the fallback
console.log(JSON.stringify(instrumentName(null)));              // ""
module The name in English, in Chinese, in a language that has no entry (English again), and for nothing at all.

Related

  • instrumentGroups() Groups the instruments by category in picker order: [{ id, label, items }], with empty groups left out.
  • getInstrument() Finds a registered instrument definition by its id, or null.
  • mountSoundPicker() Puts an instrument menu on the page: one group per category, a loading bar for samplers and the credit line an instrument asks for.

import { instrumentName } from '@openav/sound'; Also from @openav/sound/registry packages/sound/registry.js, line 111

listInstruments()#

function@openav/sound
listInstruments({ category = null } = {})

Lists every registered instrument, in the order they were registered, or only the ones of one category.

The 22 built-in instruments are registered when @openav/sound loads, so a fresh page lists those. For a list shaped for a menu, with the groups in order, use instrumentGroups().

Parameters

NameTypeDescription
categoryStringIn the options { category }: only the instruments of this category ('mallets'). optional default null

Returns

Object[] A new array of instrument definitions.

Examples

js
import { listInstruments } from '@openav/sound';

console.log(listInstruments().length);                                       // 22
console.log(listInstruments({ category: 'mallets' }).map((d) => d.id).join(' '));
// xylophone marimba vibraphone glockenspiel music-box
console.log(listInstruments({ category: 'keys' }).map((d) => d.id).join(' '));    // piano epiano
module How many instruments there are, and which ones are mallets and keys.

Related

  • instrumentGroups() Groups the instruments by category in picker order: [{ id, label, items }], with empty groups left out.
  • getInstrument() Finds a registered instrument definition by its id, or null.
  • BUILTIN_INSTRUMENTS The 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.
  • CATEGORIES The instrument categories in picker order: keys, mallets, strings, plucked, organ, voice, synth, drums and other.

import { listInstruments } from '@openav/sound'; Also from @openav/sound/registry packages/sound/registry.js, line 105

modalInstrument()#

function@openav/sound
modalInstrument({ id, name, category = 'mallets', credit = null, params, defaults = { cutoff: 8000 }, partials, strike = null, damp = 0, pitchDecay = 0.6, transpose = 0, gain = 0, attack = 0.0015, tremolo = null, maxVoices = 40, set = null, })

Makes a struck-bar or tine instrument from data: a list of partials that ring and die away, and a short noise for the mallet.

A xylophone bar, a glockenspiel bar and a music-box tine are the same physics with different numbers: a strike excites a few modes, each rings at its own multiple of the pitch and dies at its own rate. So a mallet instrument here is data, not code. The built-in xylophone is this short:

js
modalInstrument({
  id: 'xylophone', name: { en: 'Xylophone', zh: '…' }, transpose: 12,
  partials: [{ ratio: 1, gain: 1, decay: 0.9 }, { ratio: 3, gain: 0.32, decay: 0.32 }, { ratio: 6.1, gain: 0.1, decay: 0.1 }],
  strike: { ratio: 5, q: 1.1, gain: 0.35, decay: 0.012 },
  pitchDecay: 0.75,
});

Harder strikes brighten: the upper partials grow faster with velocity, as a real mallet does. The voices are raw Web Audio nodes built for each strike and left to the garbage collector.

Parameters

NameTypeDescription
idStringThe instrument id (see registerInstrument()).
nameString | ObjectThe name: a string, or { en, zh }.
categoryStringThe picker group. optional default 'mallets'
creditString | ObjectA credit line, a string or { text, url }. optional default null
paramsObject[]Params of its own: [{ key, label, min, max, def }]. Handled by the set function. optional
defaultsObjectStarting values for the shared params. The default opens the filter, because bars are bright. optional default { cutoff: 8000 }
partialsObject[]The modes of the bar: [{ ratio, gain, decay }]. ratio is the frequency as a multiple of the note (1 is the note itself, 3 a twelfth above it), gain the level relative to the others and decay the seconds to near silence at middle C. High notes die faster than low ones. A partial above 45% of the sample rate is skipped. Required: without it this throws.
strikeObjectThe mallet, a few milliseconds of band-passed noise: { ratio, q, gain, decay }, where ratio sets its pitch as a multiple of the note, q its width, gain its level and decay its length in seconds. optional default null
dampNumberSeconds in which letting go of a key mutes the bar. 0 lets the bars ring free (xylophone, glockenspiel); a vibraphone has a damper. optional default 0
pitchDecayNumberHow much faster high notes die than low ones. The decay is scaled by (261.63 / frequency) ^ pitchDecay, held between 0.2 and 3. 0 makes every note last the same. optional default 0.6
transposeNumberSemitones to shift what is played (the xylophone sounds an octave up: 12). optional default 0
gainNumberThe instrument's level, in dB. optional default 0
attackNumberSeconds the partials take to reach full level. optional default 0.0015
tremoloObjectA level wobble over all voices, the vibraphone's motor: { frequency, depth, spread }. optional default null
maxVoicesNumberHow many notes can ring at once. The oldest is faded out in 30 ms when it is exceeded. optional default 40
setFunctionset({ key, value, tremolo, Tone }), called when one of the instrument's own params changes, so it can change the tremolo or anything else. optional default null

Returns

Object An instrument definition, not yet registered: pass it to registerInstrument(). tail is set to min(8, longest decay × 1.5 + 0.2) seconds.

Examples

js
import { modalInstrument, registerInstrument, validateInstrument, getInstrument, unregisterInstrument } from '@openav/sound';

const bar = modalInstrument({
  id: 'me/steel-bar', name: { en: 'Steel bar', zh: '鋼片' },
  partials: [{ ratio: 1, gain: 1, decay: 2.5 }, { ratio: 2.76, gain: 0.4, decay: 1.2 }],
  strike: { ratio: 6, q: 1, gain: 0.3, decay: 0.01 },
});
console.log(bar.id, bar.category, bar.tail, validateInstrument(bar));   // me/steel-bar mallets 3.95 []
registerInstrument(bar);                                                  // now it is in every picker, under Mallets
console.log(getInstrument('me/steel-bar') === bar);                       // true
unregisterInstrument('me/steel-bar');
module Two partials and a mallet make a steel-bar instrument that validates and registers. It is only built, and heard, once the sound engine plays it.

Related

  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • registerSynthPreset() Adds a synth preset to every picker as synth/ plus its id: describe the voice, the envelope and the effects as data.
  • validateInstrument() Checks an instrument definition and lists everything wrong with it as readable sentences. An empty list means it is valid.
  • BUILTIN_INSTRUMENTS The 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.

import { modalInstrument } from '@openav/sound'; packages/sound/instruments/modal.js, line 20

mountSoundPicker()#

function@openav/sound
mountSoundPicker(container, sound, { lang = (globalThis.document?.documentElement?.lang || 'en').toLowerCase().startsWith('zh') ? 'zh' : 'en', className = 'oav-sound-pick', label = null, enableOnPick = true, } = {})

Puts an instrument menu on the page: one group per category, a loading bar for samplers and the credit line an instrument asks for.

A native <select> with one <optgroup> per category (CATEGORIES), so it works with a keyboard and on a phone and needs no styling to be usable. It lists whatever is registered, and it refreshes itself when an instrument is registered or removed (onInstrumentsChange()). Choosing one calls sound.setInstrument(id, { remember: true }). While a sampler loads it shows a bar and a percentage; if the load fails it says so, and says which instrument plays instead. A credit line appears under the menu for an instrument that has one: the Salamander piano is CC-BY, and its credit is always shown.

Parameters

NameTypeDescription
containerHTMLElementThe element to put the picker in.
soundSoundThe Sound to control. The menu follows its instrument and its loading.
lang'en' | 'zh'The language of the labels and instrument names. The default is 'zh' when the page's <html lang> starts with zh, else 'en'. optional default from <html lang>
classNameStringThe class of the picker. The parts get -row, -label, -status, -meter and -credit added to it, for your own styling. The picker brings no CSS. optional default 'oav-sound-pick'
labelStringThe label in front of the menu. Left out, it is the word "instrument" (in Chinese when lang is 'zh'). optional default null
enableOnPickBooleanChoosing an instrument also calls sound.enable() if the sound is off. A choice is a click, which is the gesture a browser wants before it plays audio. optional default true

Returns

Object { el, select, refresh(), dispose() }: the element it made, its <select>, a function that rebuilds the list, and a function that takes it away.

Examples

js
import { mountSoundPicker, registerInstrument, unregisterInstrument, Sound } from '@openav/sound';
import { Signals } from '@openav/core';

const engine = {                                     // a stand-in engine with instruments, so nothing is loaded
  instrument: 'synth/pad', params: [], enable() {}, noteOn() {}, noteOff() {}, set() {},
  setInstrument: async (id) => { engine.instrument = id; return true; }, subscribe: () => () => {},
};
const sound = new Sound({ signals: new Signals(), engine, remember: false });
const picker = mountSoundPicker(document.getElementById('app'), sound, { lang: 'en' });
console.log(picker.select.options.length, picker.select.value);    // 22 synth/pad

registerInstrument({ id: 'me/kalimba', name: 'Kalimba', category: 'mallets', create: () => ({ noteOn() {}, noteOff() {}, dispose() {} }) });
console.log(picker.select.options.length);                          // 23: the menu refreshed itself
unregisterInstrument('me/kalimba');
module A menu of the 22 built-in instruments, in groups, appears on the page. Registering one more adds it to the menu at once.

Related

  • Sound Makes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
  • instrumentGroups() Groups the instruments by category in picker order: [{ id, label, items }], with empty groups left out.
  • onInstrumentsChange() Calls your function whenever an instrument is registered or removed, so a menu can refresh. Returns a function that stops listening.
  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • CATEGORIES The instrument categories in picker order: keys, mallets, strings, plucked, organ, voice, synth, drums and other.

import { mountSoundPicker } from '@openav/sound'; Also from @openav/sound/picker packages/sound/picker.js, line 18

onInstrumentsChange()#

function@openav/sound
onInstrumentsChange(cb)

Calls your function whenever an instrument is registered or removed, so a menu can refresh. Returns a function that stops listening.

mountSoundPicker() listens this way, which is why an instrument registered after the picker was built still shows up. A replacement with { replace: true } also reports register. A call to unregisterInstrument() for an id that is not there reports nothing.

Parameters

NameTypeDescription
cbFunctioncb({ type, id }) with type 'register' or 'unregister'. An error inside it is logged and does not stop the others.

Returns

Function Call it to stop listening.

Examples

js
import { onInstrumentsChange, registerInstrument, unregisterInstrument } from '@openav/sound';

const seen = [];
const stop = onInstrumentsChange((e) => seen.push(e.type + ' ' + e.id));
const def = { id: 'me/x', name: 'X', category: 'other', create: () => ({ noteOn() {}, noteOff() {}, dispose() {} }) };
registerInstrument(def);
unregisterInstrument('me/x');
stop();
registerInstrument(def); unregisterInstrument('me/x');        // nobody is listening now
console.log(seen.join(' | '));                                  // register me/x | unregister me/x
module Records two changes while listening, then stops listening and shows that the next two are not reported.

Related

  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • unregisterInstrument() Removes a registered sound, for tests and hot reload. Returns whether it was there.
  • mountSoundPicker() Puts an instrument menu on the page: one group per category, a loading bar for samplers and the credit line an instrument asks for.

import { onInstrumentsChange } from '@openav/sound'; Also from @openav/sound/registry packages/sound/registry.js, line 129

registerInstrument()#

function@openav/sound
registerInstrument(def, { replace = false } = {})

Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.

A sound you can pick is plain data plus one factory. Registering it is all it takes: import the file before you start, and every mountSoundPicker() on the page lists it under its category, with no change to the package. The registry lives on globalThis, so two copies of this module on one page share one list. Built-in instruments only fill empty slots: register your own 'piano' first and you keep it. The definition:

text
id          lowercase letters, digits and "-", at most one "/": 'kalimba', 'me/kalimba'
name        a string, or { en, zh }
category    keys · mallets · strings · plucked · organ · voice · synth · drums · other, or your own
create      create(Tone, ctx) → { noteOn(note, vel01, time), noteOff(note, time), set?(key, v),
                                  releaseAll?(time), ready?: Promise, dispose() }
params      your own knobs [{ key, label, min, max, def }], shown as sound/<key> while it plays
defaults    starting values for the shared params: { cutoff, space, volume }
credit      a string, or { text, url }, shown in the picker
gain        level in dB      transpose   semitones      tail   seconds the old one may ring after a switch
fallback    the id of an instrument to play if `ready` rejects

create(Tone, ctx) runs on enable and on every switch to the instrument, with Tone.js already loaded. ctx is { output, rawOutput, rawContext, baseUrl, options, onProgress }: connect Tone nodes to output (raw Web Audio nodes to rawOutput), baseUrl is where sample folders live, options is what was passed in toneEngine({ instruments: { <id>: … } }), and onProgress(0..1) shows a loading bar. If ready is a promise, the picker shows progress and the old sound keeps playing until it resolves. An instrument that makes mallet sounds from data can use modalInstrument(), and one that is a Tone synth with settings, registerSynthPreset().

Parameters

NameTypeDescription
defObjectThe instrument definition, see the table below.
replaceBooleanIn the options { replace }: allow replacing an instrument that has the same id. Without it a duplicate id throws. optional default false

Returns

Object The definition you gave. It throws a TypeError listing everything wrong if the definition is not valid (see validateInstrument()), and an Error for a duplicate id.

Examples

js
import { registerInstrument, getInstrument, listInstruments, unregisterInstrument } from '@openav/sound';

const def = {
  id: 'me/kalimba', name: { en: 'Kalimba', zh: '拇指琴' }, category: 'mallets',
  create: () => ({ noteOn() {}, noteOff() {}, dispose() {} }),     // a silent stand-in for the example
};
registerInstrument(def);
console.log(getInstrument('me/kalimba') === def);                   // true
console.log(listInstruments({ category: 'mallets' }).map((d) => d.id).join(' '));
// xylophone marimba vibraphone glockenspiel music-box me/kalimba

try { registerInstrument(def); } catch (e) { console.log(e.message); }
// [sound] instrument "me/kalimba" is already registered; pass { replace: true } to swap it on purpose
registerInstrument({ ...def, name: 'Kalimba 2' }, { replace: true });   // on purpose
unregisterInstrument('me/kalimba');
module Registers an instrument, finds it in its category, is refused when it registers the same id twice, and swaps it on purpose with replace.
js
import { registerInstrument } from '@openav/sound';

registerInstrument({
  id: 'kalimba',
  name: { en: 'Kalimba', zh: '拇指琴' },
  category: 'mallets',
  params: [{ key: 'decay', label: 'Decay (s)', min: 0.2, max: 4, def: 1.2 }],   // becomes sound/decay
  create(Tone, { output }) {
    const synth = new Tone.PolySynth(Tone.FMSynth, {
      harmonicity: 5.1, modulationIndex: 2,
      envelope: { attack: 0.001, decay: 1.2, sustain: 0, release: 1 },
    }).connect(output);
    const hz = (note) => 440 * 2 ** ((note - 69) / 12);
    return {
      noteOn: (note, vel, time) => synth.triggerAttack(hz(note), time, vel),
      noteOff: (note, time) => synth.triggerRelease(hz(note), time),
      set: (key, value) => key === 'decay' && synth.set({ envelope: { decay: value } }),
      dispose: () => synth.dispose(),
    };
  },
});
static A whole instrument in one file: an FM synth with a knob of its own. Import it before the sound starts, and "Kalimba" appears under Mallets in every picker.

Related

  • validateInstrument() Checks an instrument definition and lists everything wrong with it as readable sentences. An empty list means it is valid.
  • unregisterInstrument() Removes a registered sound, for tests and hot reload. Returns whether it was there.
  • getInstrument() Finds a registered instrument definition by its id, or null.
  • modalInstrument() Makes a struck-bar or tine instrument from data: a list of partials that ring and die away, and a short noise for the mallet.
  • registerSynthPreset() Adds a synth preset to every picker as synth/ plus its id: describe the voice, the envelope and the effects as data.
  • toneEngine() The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
  • CATEGORIES The instrument categories in picker order: keys, mallets, strings, plucked, organ, voice, synth, drums and other.

import { registerInstrument } from '@openav/sound'; Also from @openav/sound/registry packages/sound/registry.js, line 86

registerSynthPreset()#

function@openav/sound
registerSynthPreset(preset, opts)

Adds a synth preset to every picker as synth/ plus its id: describe the voice, the envelope and the effects as data.

A preset names a voice, its settings in Tone.js's vocabulary, and an optional effects chain, and nothing else. id is lowercase letters, digits and - (it becomes synth/<id>), name is a string or { en, zh }. voice is 'synth' (oscillator and envelope), 'mono' (adds a resonant filter with its own envelope), 'fm' or 'am'. The Tone settings are oscillator, envelope, filter, filterEnvelope, harmonicity, modulationIndex, modulation, modulationEnvelope, detune and portamento. The rest: octave (a whole number of octaves, -1 for a bass), gain (dB, to level it against the others), polyphony, defaults (starting cutoff, space, volume), params, category and credit, as in registerInstrument().

effects is a list of { type, …settings } that the sound passes through in order. The types are chorus, delay, pingpong, distortion, vibrato, tremolo, phaser, autofilter, filter, eq, compressor, chebyshev and widener; the other keys are the effect's own Tone settings, including wet. A preset that is wrong throws a TypeError with the reasons.

Parameters

NameTypeDescription
presetObjectThe preset: { id, name, voice, … }, see below.
optsObjectThe options of registerInstrument(): { replace: true } swaps an existing one. optional

Returns

Object The instrument definition that was registered (synthInstrument() of the preset).

Examples

js
import { registerSynthPreset, getInstrument, unregisterInstrument } from '@openav/sound';

const def = registerSynthPreset({
  id: 'glass', name: { en: 'Glass', zh: '玻璃' },
  voice: 'fm', harmonicity: 3.01, modulationIndex: 14,
  envelope: { attack: 0.002, decay: 1.4, sustain: 0, release: 1.2 },
  effects: [{ type: 'chorus', frequency: 1.5, depth: 0.4, wet: 0.3 }],
});
console.log(def.id, getInstrument('synth/glass') === def);   // synth/glass true: it is in every picker now
unregisterInstrument('synth/glass');
module Registers an FM bell-like sound as plain data. It appears under Synth in the picker as "Glass".

Related

  • synthInstrument() Turns a synth preset into an instrument definition whose id is synth/ followed by the preset id, without registering it.
  • SYNTH_PRESETS The 11 synth presets as plain data: pad, saw lead, square lead, pluck, supersaw, bass, acid, brass, bell, sub and chip.
  • validateSynthPreset() Checks a synth preset and lists what is wrong with it. An empty list means it is valid.
  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • modalInstrument() Makes a struck-bar or tine instrument from data: a list of partials that ring and die away, and a short noise for the mallet.

import { registerSynthPreset } from '@openav/sound'; packages/sound/instruments/synth.js, line 180

SALAMANDER_NOTES#

constant@openav/sound
SALAMANDER_NOTES = array, 30 items

The 30 notes the Salamander piano has samples for: every minor third from A0 (MIDI 21) to C8 (108).

The sampler shifts the pitch of the notes in between by at most a semitone and a half, which is inaudible on a piano. There is only one velocity layer, so loud against soft is a gain curve, not a different hammer.

Type
Number[]

[21, 24, 27, …, 108]: 30 MIDI note numbers, three semitones apart (the A, C, D♯ and F♯ of every octave).

Examples

js
import { SALAMANDER_NOTES } from '@openav/sound';

console.log(SALAMANDER_NOTES.length, SALAMANDER_NOTES[0], SALAMANDER_NOTES.at(-1));   // 30 21 108
console.log(SALAMANDER_NOTES.slice(0, 4).join(' '));                                    // 21 24 27 30
module The count, the lowest and highest notes, and the first four.

Related

  • salamanderMap() Builds the sample map of the Salamander piano for Tone.Sampler: { 'A0': 'A0.mp3', 'D#1': 'Ds1.mp3', … }.
  • SAMPLES_URL The address of the folder with the built-in sample sets, when nobody says otherwise: openaudiovisual.com/packages/sound/samples/.

import { SALAMANDER_NOTES } from '@openav/sound'; packages/sound/instruments/piano.js, line 20

salamanderMap()#

function@openav/sound
salamanderMap(notes = SALAMANDER_NOTES)

Builds the sample map of the Salamander piano for Tone.Sampler: { 'A0': 'A0.mp3', 'D#1': 'Ds1.mp3', … }.

The piano instrument uses it to fetch its files from baseUrl + 'salamander/'. Use it if you want to load the same files into a sampler of your own.

Parameters

NameTypeDescription
notesNumber[]Which MIDI notes to include. The default is SALAMANDER_NOTES, all 30. optional default SALAMANDER_NOTES

Returns

Object An object from Tone note names to file names. The files spell a sharp as "s": D#1 is Ds1.mp3.

Examples

js
import { salamanderMap, SAMPLES_URL } from '@openav/sound';

const map = salamanderMap();
console.log(Object.keys(map).length, map['A0'], map['D#1']);   // 30 A0.mp3 Ds1.mp3
console.log(salamanderMap([60, 63]));                            // { C4: 'C4.mp3', 'D#4': 'Ds4.mp3' }
console.log(SAMPLES_URL + 'salamander/' + map['C4']);            // the address of one file
module The full map has 30 files; a map of two notes; and the full address of one sample.

Related

  • SALAMANDER_NOTES The 30 notes the Salamander piano has samples for: every minor third from A0 (MIDI 21) to C8 (108).
  • SAMPLES_URL The address of the folder with the built-in sample sets, when nobody says otherwise: openaudiovisual.com/packages/sound/samples/.
  • BUILTIN_INSTRUMENTS The 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.

import { salamanderMap } from '@openav/sound'; packages/sound/instruments/piano.js, line 23

SAMPLES_URL#

constant@openav/sound
SAMPLES_URL = "https://openaudiovisual.com/packages/sound/samples/"

The address of the folder with the built-in sample sets, when nobody says otherwise: openaudiovisual.com/packages/sound/samples/.

It is the default baseUrl of toneEngine(). The piano reads its 30 mp3 files from the salamander/ folder under it. The server sends CORS headers, so any page can load them. createShow() points the engine at the copy next to the page instead. To use your own copy, give baseUrl, or instruments: { piano: { baseUrl } } for the piano alone.

Type
String

'https://openaudiovisual.com/packages/sound/samples/', ending in a slash.

Examples

js
import { SAMPLES_URL, toneEngine } from '@openav/sound';

console.log(SAMPLES_URL);   // https://openaudiovisual.com/packages/sound/samples/
const local = toneEngine({ instrument: 'piano', baseUrl: '/my/samples/' });   // the piano then reads /my/samples/salamander/…
module Logs the default folder and shows how to point the engine at your own.

Related

  • toneEngine() The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
  • TONE_CDN The address Tone.js is loaded from when the engine is enabled: Tone 15.0.4 on jsDelivr.
  • salamanderMap() Builds the sample map of the Salamander piano for Tone.Sampler: { 'A0': 'A0.mp3', 'D#1': 'Ds1.mp3', … }.

import { SAMPLES_URL } from '@openav/sound'; packages/sound/index.js, line 57

Sound#

class@openav/sound
new Sound({ signals, params = null, engine, instrument = null, remember = true, picker = true })

Makes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.

Sound is the shell that wires an engine into an app. Once await sound.enable() has run, every midi/note/on and midi/note/off on the bus is played, whatever sent it: a MIDI keyboard, the on-screen piano (mountKeys()), a SimPlayer, a TakePlayer or a phone. Notes on a channel in the engine's skipChannels are left out (the Tone engine skips channel 10, which belongs to the drum machine).

The engine's params become sound/* params, so a knob, the timeline or a hand can play the filter the same way it plays the visuals: sound is performable state, not a side effect. Every frame, call sound.update(state) with the resolved param values, and the sound/* ones are pushed into the engine. The instrument is a discrete choice (setInstrument() or the picker); what you perform continuously stays a param.

Before you call it

sound.enable() from a click: browsers start audio only after a user gesture. With toneEngine() it also loads Tone.js from the network (jsDelivr) the first time. Before enable() nothing is heard, and the bus is not listened to.

Parameters

NameTypeDescription
signalsSignalsThe Signals bus. Sound listens to midi/note/on { note, vel, ch }, midi/note/off { note, ch } and midi/cc/64 (the sustain pedal, down at 0.5 or more), and pulses sound/instrument { id, name } when the instrument changes.
paramsParamsA Params registry to add the engine's params to, named sound/<key> (sound/cutoff, sound/space, sound/volume and the instrument's own). Left out, no params are registered. optional default null
engineObjectThe sound engine, such as toneEngine() or drumEngine(). The contract is { params, enable(), noteOn(note, vel01), noteOff(note), set(key, value), dispose() }, and optionally setInstrument, instrument, status, subscribe, sustain(down) and skipChannels.
instrumentStringThe instrument to start with, for an engine that has instruments ('piano', 'synth/pluck'). An unknown id is warned about in the console and ignored. optional default null
rememberBoolean | StringKeep the instrument the picker last chose, per page, in localStorage, and let it win over instrument the next time. A string is your own storage key; the default key is openav:sound:<page path>. false turns it off. Choices made from code are not remembered unless you ask (setInstrument(id, { remember: true })). optional default true
pickerBooleanWhether a console should show the instrument picker. It is stored as sound.picker for whoever builds the interface; Sound itself draws nothing. optional default true

Members

NameDescription
enable()methodStarts the engine and begins listening to the bus. Returns a promise of true. Call it from a click. Calling it again returns the same promise; if it fails you can try again. Emits { type: 'enabled', id } to onChange listeners.
update(state)methodPer frame: pushes every sound/<key> value of state into the engine's set(key, value). Other keys of state are ignored. Does nothing before enable().
instrumentgetterGetter. The id of the current instrument ('piano'), or null for an engine without instruments, such as the drum kit.
statusgetterGetter. { id, state, progress, error }: state is 'idle', 'loading', 'ready' or 'error' and progress is 0..1 while a sampler loads.
choosablegetterGetter. Whether the engine has instruments to choose from.
setInstrument(id, { remember = false } = {})methodSwitches instrument, live: setInstrument(id, { remember = false }). Held notes are released on the old one and the new one takes over once it is ready (a sampler keeps the old sound while its files load). Returns a promise: true once the instrument plays, false if the id is unknown, the engine has no instruments or the choice was replaced by a newer one. remember: true stores the choice for this page.
onChange(cb)methodListens for what happens: cb({ type, id, progress, error }) with type 'instrument', 'loading', 'ready', 'error' or 'enabled'. Returns a function that stops listening.
dispose()methodStops listening to the bus, drops the listeners and disposes the engine.

Notes

  • From a p5 sketch you do not build this: startSound() does it, loading this package only when it is called.

Examples

js
import { Sound } from '@openav/sound';
import { Signals } from '@openav/core';

const signals = new Signals();
const heard = [];
const engine = {                                    // the smallest engine: it only writes down what it is told
  params: [{ key: 'cutoff', label: 'Cutoff', min: 100, max: 8000, def: 2500 }],
  async enable() {},
  noteOn: (note, vel) => heard.push('on ' + note + ' ' + vel),
  noteOff: (note) => heard.push('off ' + note),
  set: (key, value) => heard.push(key + '=' + value),
  dispose() {},
};
const sound = new Sound({ signals, engine });
await sound.enable();
signals.pulse('midi/note/on', { note: 60, vel: 0.8, ch: 1 });
signals.pulse('midi/note/off', { note: 60, ch: 1 });
sound.update({ 'sound/cutoff': 900, hue: 0.3 });     // only the sound/* keys go to the engine
console.log(heard.join(', '));                        // on 60 0.8, off 60, cutoff=900
console.log(sound.enabled, sound.choosable, sound.instrument);   // true false null
module A bus with a stand-in engine: notes on the bus reach the engine, and of the state only sound/cutoff does. The engine has no instruments, so choosable is false.
js
import { Signals } from '@openav/core';
import { mountKeys } from '@openav/keys';
import { Sound, toneEngine, mountSoundPicker } from '@openav/sound';

const signals = new Signals();
const sound = new Sound({ signals, engine: toneEngine({ instrument: 'piano' }) });
mountKeys(document.getElementById('app'), { signals });        // an on-screen piano that publishes midi/note/on
mountSoundPicker(document.getElementById('app'), sound);       // choosing an instrument also turns the sound on
document.body.addEventListener('click', () => sound.enable(), { once: true });
sound.onChange((e) => console.log(e.type, e.id));
static Click anywhere to start the sound, then click the piano keys: you hear the Salamander grand piano. Pick another instrument from the menu while you play.

To be confirmed

The Tone.js engine was not run for this page: there was no network and no speakers. What is written here comes from the code and from the package tests, which use a stand-in for Tone.js, and the examples that run use a stand-in engine.

Related

  • toneEngine() The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
  • mountSoundPicker() Puts an instrument menu on the page: one group per category, a loading bar for samplers and the credit line an instrument asks for.
  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • SOUND_PARAMS The three params every instrument shares: cutoff (the filter), space (the reverb) and volume.
  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).

import { Sound } from '@openav/sound'; packages/sound/index.js, line 90

SOUND_PARAMS#

constant@openav/sound
SOUND_PARAMS = array, 3 items

The three params every instrument shares: cutoff (the filter), space (the reverb) and volume.

cutoff is the low-pass filter's frequency in Hz, space is how much reverb is mixed in (the wet amount, 0..1), volume is the level in dB. They sit after the instrument on the output chain, so the timeline, a knob or a hand keep performing them across instrument switches. In a show they are the params sound/cutoff, sound/space and sound/volume. An instrument may suggest other starting values through its defaults, and automation and overrides always win over a suggestion. An instrument's own params cannot use these keys.

Type
Object[]

[{ key: 'cutoff', min: 100, max: 8000, def: 2500 }, { key: 'space', min: 0, max: 1, def: 0.3 }, { key: 'volume', min: -36, max: 0, def: -8 }], each also with a label.

Examples

js
import { SOUND_PARAMS } from '@openav/sound';

console.log(SOUND_PARAMS.map((p) => p.key + ' ' + p.min + '..' + p.max + ' (' + p.def + ')').join(' | '));
// cutoff 100..8000 (2500) | space 0..1 (0.3) | volume -36..0 (-8)
module Logs the range and the default of each shared param.

Related

  • toneEngine() The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
  • Sound Makes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.

import { SOUND_PARAMS } from '@openav/sound'; Also from @openav/sound/registry packages/sound/registry.js, line 24

soundOptions()#

function@openav/sound
soundOptions(value)

Reads the modules.sound option of createShow() (true, an instrument id, or an object) into one normalized object.

It is how createShow({ modules: { sound: … } }) understands its option, so sound: true, sound: 'piano' and sound: { instrument: 'organ', remember: false, picker: false } all work. You only need it if you write something that takes the same option.

Parameters

NameTypeDescription
valueBoolean | String | Objecttrue for the defaults, an instrument id ('piano'), or { instrument, remember, picker, engine, …toneEngine options }. Anything falsy gives null.

Returns

Object | null { engine, instrument, remember, picker, engineOptions } (engine and instrument are null when not given, remember and picker are true), or null for a falsy value. engineOptions holds the rest of an object: baseUrl, cdn, instruments…. An instrument that is not a string, or a value of another kind, throws a TypeError.

Examples

js
import { soundOptions } from '@openav/sound';

console.log(soundOptions('piano').instrument);                                  // piano
console.log(soundOptions({ instrument: 'organ', remember: false, baseUrl: 'x/' }));
// { engine: null, instrument: 'organ', remember: false, picker: true, engineOptions: { baseUrl: 'x/' } }
console.log(soundOptions(false));                                               // null
try { soundOptions([]); } catch (e) { console.log(e.message); }
// modules.sound: true, an instrument id, or { instrument, remember, picker, engine, baseUrl, cdn, instruments }
module The three accepted shapes, a falsy value that turns the sound off, and the error for something that is none of them.

Related

  • Sound Makes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
  • toneEngine() The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { soundOptions } from '@openav/sound'; packages/sound/index.js, line 74

SYNTH_PRESETS#

constant@openav/sound
SYNTH_PRESETS = array, 11 items

The 11 synth presets as plain data: pad, saw lead, square lead, pluck, supersaw, bass, acid, brass, bell, sub and chip.

Each preset is what synthInstrument() turns into an instrument synth/<id>. They are data in Tone.js's own vocabulary (oscillator, envelope, filter, filterEnvelope, harmonicity…), a good place to look when you write a preset of your own for registerSynthPreset(). The pad is the framework's original voice: three detuned sawtooth waves with a quick attack and a long release.

Type
Object[]

An array of 11 presets { id, name: { en, zh }, voice, … }: pad, saw-lead, square-lead, pluck, supersaw, bass, acid, brass, bell, sub, chip.

Examples

js
import { SYNTH_PRESETS } from '@openav/sound';

console.log(SYNTH_PRESETS.map((p) => p.id).join(' '));
// pad saw-lead square-lead pluck supersaw bass acid brass bell sub chip
const bass = SYNTH_PRESETS.find((p) => p.id === 'bass');
console.log(bass.voice, bass.octave, bass.oscillator.type);   // mono -1 fatsawtooth
module Lists the preset ids and reads a few settings of the bass: a mono synth with a filter, an octave down.

Related

  • synthInstrument() Turns a synth preset into an instrument definition whose id is synth/ followed by the preset id, without registering it.
  • registerSynthPreset() Adds a synth preset to every picker as synth/ plus its id: describe the voice, the envelope and the effects as data.
  • validateSynthPreset() Checks a synth preset and lists what is wrong with it. An empty list means it is valid.
  • BUILTIN_INSTRUMENTS The 22 instruments that come with the package, as definitions: a grand piano, mallets, strings, harp, organ, choir and 11 synth presets.

import { SYNTH_PRESETS } from '@openav/sound'; packages/sound/instruments/synth.js, line 30

synthInstrument()#

function@openav/sound
synthInstrument(preset)

Turns a synth preset into an instrument definition whose id is synth/ followed by the preset id, without registering it.

The definition's create() builds one Tone.PolySynth of the preset's voice (synth, mono, fm or am), with the Tone settings the preset carries, a chain of its effects, and its polyphony (16 voices if left out). Use it when you want the definition for yourself (to adjust it before registering); otherwise registerSynthPreset() does both steps.

Parameters

NameTypeDescription
presetObjectThe preset, see registerSynthPreset(). A preset that validateSynthPreset() refuses throws a TypeError.

Returns

Object An instrument definition: id is 'synth/' + preset.id, name the preset's (or its id), category the preset's or 'synth', transpose is octave × 12, gain, defaults, params and credit come from the preset, tail is min(8, release × 1.5 + 0.5) seconds and preset is the preset itself.

Examples

js
import { synthInstrument, validateInstrument } from '@openav/sound';

const def = synthInstrument({ id: 'glass', name: 'Glass', voice: 'fm', octave: -1, gain: -3, envelope: { release: 2 } });
console.log(def.id, def.category, def.transpose, def.gain, def.tail);   // synth/glass synth -12 -3 3.5
console.log(validateInstrument(def));                                    // []
try { synthInstrument({ id: 'Glass!' }); } catch (e) { console.log(e instanceof TypeError); }   // true
module A preset becomes a valid instrument definition: an octave down is 12 semitones, and the tail follows the release. A bad id throws.

Related

  • registerSynthPreset() Adds a synth preset to every picker as synth/ plus its id: describe the voice, the envelope and the effects as data.
  • SYNTH_PRESETS The 11 synth presets as plain data: pad, saw lead, square lead, pluck, supersaw, bass, acid, brass, bell, sub and chip.
  • validateSynthPreset() Checks a synth preset and lists what is wrong with it. An empty list means it is valid.
  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.

import { synthInstrument } from '@openav/sound'; packages/sound/instruments/synth.js, line 141

TONE_CDN#

constant@openav/sound
TONE_CDN = "https://cdn.jsdelivr.net/npm/tone@15.0.4/+esm"

The address Tone.js is loaded from when the engine is enabled: Tone 15.0.4 on jsDelivr.

A module URL that toneEngine() imports on enable(). To host Tone.js yourself, pass your own as the cdn option of toneEngine().

Type
String

'https://cdn.jsdelivr.net/npm/tone@15.0.4/+esm'.

Examples

js
import { TONE_CDN, toneEngine } from '@openav/sound';

console.log(TONE_CDN);   // https://cdn.jsdelivr.net/npm/tone@15.0.4/+esm
const own = toneEngine({ cdn: '/vendor/tone.mjs' });   // your own copy, used when enable() runs
module Logs the default address and shows where an own copy would be given.

Related

  • toneEngine() The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.
  • SAMPLES_URL The address of the folder with the built-in sample sets, when nobody says otherwise: openaudiovisual.com/packages/sound/samples/.

import { TONE_CDN } from '@openav/sound'; packages/sound/index.js, line 54

toneEngine()#

function@openav/sound
toneEngine({ cdn = TONE_CDN, instrument = DEFAULT_INSTRUMENT, baseUrl = SAMPLES_URL, instruments: perInstrument = {}, drumChannel = 10, } = {})

The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.

Tone.js stays a neighbour, not a dependency: it is loaded from the network the first time enable() runs, so an app that never turns the sound on pays nothing. After enable() the instrument's output goes through a low-pass filter (cutoff, starting at 2500 Hz), a reverb (space, the wet amount, starting at 0.3), a volume (volume, in dB, starting at -8) and a limiter at -1 dB, so a loud organ never clips the room. These three are the SOUND_PARAMS shared by every instrument. Their starting values can follow the instrument (a piano wants the filter open), and the instrument may add params of its own.

setInstrument(id) switches live and returns a promise of whether it worked. Before enable() it only remembers the choice. Held notes are released on the old instrument and its tail is left to ring for a few seconds. A sampler (the piano) keeps the old sound until its files are in; if they cannot load, the engine plays the instrument's fallback (the piano falls back to the electric piano), so a performance never goes silent because of one failed download. subscribe(cb) tells you about instrument, loading, ready and error events. sustain(down) is the pedal: note-offs wait until it comes up. releaseAll() is a panic button.

You usually pass the engine to a Sound, which is what listens to the bus. Used on its own, noteOn(60, 0.8) plays a note.

Before you call it

await engine.enable() from a click: it loads Tone.js from the network, then starts its audio. Before that, noteOn() does nothing and setInstrument() only remembers the choice.

Parameters

NameTypeDescription
cdnStringWhere Tone.js is imported from when the engine is enabled. The default is TONE_CDN. A module URL, so you can host your own copy. optional default TONE_CDN
instrumentStringThe instrument to start with. An unknown id is warned about and synth/pad is used. optional default 'synth/pad'
baseUrlStringThe folder that holds the built-in sample sets (salamander/…). The default is SAMPLES_URL. optional default SAMPLES_URL
instrumentsObjectOptions for single instruments, handed to their create() as ctx.options: { piano: { baseUrl: '/my/salamander/' } }. optional default {}
drumChannelNumber | nullNotes on this MIDI channel belong to a drum engine (the drum machine sends General MIDI channel 10) and are not played here. null plays everything. optional default 10

Returns

Object An engine: { params, sharedParams, instrument, status, skipChannels, hasInstrument(id), subscribe(cb), setInstrument(id), enable(), noteOn(note, vel01, time), noteOff(note, time), sustain(down), releaseAll(), set(key, value), dispose() }.

Examples

js
import { toneEngine } from '@openav/sound';

const engine = toneEngine({ instrument: 'strings' });      // nothing is loaded yet
console.log(engine.instrument, engine.status.state);        // strings idle
console.log(engine.params.map((p) => p.key + ' ' + p.def).join(', '));
// cutoff 6000, space 0.45, volume -8, attack 0.35, vibrato 0.35   (the shared three, then the strings' own)
console.log(engine.skipChannels, engine.hasInstrument('piano'));   // [10] true
console.log(toneEngine({ drumChannel: null }).skipChannels);       // []
module Looks at an engine before it is enabled: its instrument, the params it will offer (the strings suggest a different starting cutoff and space, and add two of their own), and that channel 10 is left to the drums.
js
import { toneEngine } from '@openav/sound';

const engine = toneEngine({ instrument: 'epiano' });
document.body.addEventListener('click', async () => {
  await engine.enable();                                // loads Tone.js from the network, starts the audio
  engine.set('space', 0.6);                             // more reverb
  [60, 64, 67].forEach((note) => engine.noteOn(note, 0.8));   // a C major chord
  setTimeout(() => engine.releaseAll(), 1500);
}, { once: true });
static Click the page: an electric piano plays a C major chord, with more reverb than usual, and lets go after a second and a half.

To be confirmed

The Tone.js engine was not run for this page: there was no network and no speakers. What is written here comes from the code and from the package tests, which use a stand-in for Tone.js, and the examples that run use a stand-in engine.

Related

  • Sound Makes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • SOUND_PARAMS The three params every instrument shares: cutoff (the filter), space (the reverb) and volume.
  • TONE_CDN The address Tone.js is loaded from when the engine is enabled: Tone 15.0.4 on jsDelivr.
  • SAMPLES_URL The address of the folder with the built-in sample sets, when nobody says otherwise: openaudiovisual.com/packages/sound/samples/.
  • drumEngine() A drum kit made of Web Audio synthesis, with no sample files: General MIDI drum notes go in, drum sounds come out.

import { toneEngine } from '@openav/sound'; packages/sound/index.js, line 226

unregisterInstrument()#

function@openav/sound
unregisterInstrument(id)

Removes a registered sound, for tests and hot reload. Returns whether it was there.

A picker refreshes itself. It does not stop an engine that is playing the instrument: switch away first.

Parameters

NameTypeDescription
idStringThe id of the instrument.

Returns

Boolean True if it existed. The listeners of onInstrumentsChange() hear { type: 'unregister', id } only then.

Examples

js
import { registerInstrument, unregisterInstrument, getInstrument } from '@openav/sound';

registerInstrument({ id: 'me/temp', name: 'Temp', category: 'other', create: () => ({ noteOn() {}, noteOff() {}, dispose() {} }) });
console.log(unregisterInstrument('me/temp'), getInstrument('me/temp'));   // true null
console.log(unregisterInstrument('me/temp'));                              // false
module Removes an instrument; the second time there is nothing to remove.

Related

  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • getInstrument() Finds a registered instrument definition by its id, or null.
  • onInstrumentsChange() Calls your function whenever an instrument is registered or removed, so a menu can refresh. Returns a function that stops listening.

import { unregisterInstrument } from '@openav/sound'; Also from @openav/sound/registry packages/sound/registry.js, line 96

validateInstrument()#

function@openav/sound
validateInstrument(def)

Checks an instrument definition and lists everything wrong with it as readable sentences. An empty list means it is valid.

It checks: id (lowercase letters, digits and -, at most one /), name (a string, or { en, zh } with at least en), category (a string), create (a function), params (each with a word as key, numbers min < max, a def within them, and a key that is not one of the shared three), defaults (only cutoff, space or volume, each within its range), credit (a string, or { text, url }), gain, transpose and tail (numbers) and fallback (another instrument's id). registerInstrument() runs it and throws with these sentences.

Parameters

NameTypeDescription
defanyThe definition to check.

Returns

String[] Sentences, one per problem. [] when valid. It never throws.

Examples

js
import { validateInstrument } from '@openav/sound';

const ok = { id: 'me/ok', name: 'OK', category: 'other', create() {} };
console.log(validateInstrument(ok));                          // []
console.log(validateInstrument({ ...ok, id: 'Piano', params: [{ key: 'cutoff', min: 0, max: 1, def: 0 }] }));
// [ 'id "Piano": lowercase letters, digits and "-", at most one "/" (e.g. "synth/pad")',
//   'params[0] (cutoff): "cutoff" is a shared sound param; set its starting value with defaults: { cutoff: … }' ]
console.log(validateInstrument(null));                        // [ 'an instrument definition is an object' ]
module A good definition gives an empty list; a definition with a capital letter in the id and a param that is already shared gets two sentences.

Related

  • registerInstrument() Adds a sound to every picker on the page: an id, a name, a category and one create() function that builds the instrument.
  • validateSynthPreset() Checks a synth preset and lists what is wrong with it. An empty list means it is valid.
  • SOUND_PARAMS The three params every instrument shares: cutoff (the filter), space (the reverb) and volume.

import { validateInstrument } from '@openav/sound'; Also from @openav/sound/registry packages/sound/registry.js, line 49

validateSynthPreset()#

function@openav/sound
validateSynthPreset(p)

Checks a synth preset and lists what is wrong with it. An empty list means it is valid.

It checks the id (lowercase letters, digits and -), the voice (synth, mono, fm or am), that every effect has a type the package knows, and that octave is a whole number. It does not check the Tone settings, which are Tone.js's to judge.

Parameters

NameTypeDescription
pObjectThe preset to check.

Returns

String[] Sentences, one per problem. [] when valid.

Examples

js
import { validateSynthPreset } from '@openav/sound';

console.log(validateSynthPreset({ id: 'glass', voice: 'fm', effects: [{ type: 'chorus' }] }));   // []
console.log(validateSynthPreset({ id: 'Glass', voice: 'laser', effects: [{ type: 'zz' }], octave: 1.5 }));
// [ 'id "Glass": lowercase letters, digits and "-" (it becomes "synth/<id>")',
//   'voice "laser": one of synth, mono, fm, am',
//   'effects[0].type "zz": one of chorus, delay, pingpong, distortion, vibrato, tremolo, phaser, autofilter, filter, eq, compressor, chebyshev, widener',
//   'octave: a whole number' ]
module A good preset gives an empty list; a preset with four mistakes gets one sentence for each.

Related

  • registerSynthPreset() Adds a synth preset to every picker as synth/ plus its id: describe the voice, the envelope and the effects as data.
  • synthInstrument() Turns a synth preset into an instrument definition whose id is synth/ followed by the preset id, without registering it.
  • validateInstrument() Checks an instrument definition and lists everything wrong with it as readable sentences. An empty list means it is valid.

import { validateSynthPreset } from '@openav/sound'; packages/sound/instruments/synth.js, line 128

Drum machine and rhythm#

startDrums(), one line of rhythm, drumPhase; @openav/drums and @openav/pattern. 40

oav-p5

beatPhase#

variableoav-p5
beatPhase

Also: OAVp5.drums.beatPhase

How far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.

A beat is a quarter note (four steps). 1 - beatPhase is biggest right on the beat and shrinks until the next one, so circle(width / 2, height / 2, 60 + (1 - beatPhase) * 80) pulses with the tempo, with no callback needed. drumPhase is the same thing across a whole bar.

Type
Number
Range
0..1

Read fresh each time you look at it. 0 on the beat, close to 1 just before the next one, then 0 again. It is 0 before the machine has played and holds still while it is stopped.

Before it has a value

Start the machine with startDrums(); until it plays, beatPhase is 0.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 120);
  noStroke();
}

function draw() {
  background(20);
  fill(255, 120, 90);
  circle(width / 2, height / 2, 60 + (1 - beatPhase) * 100);   // biggest on the beat
}
A circle that pulses four times per bar: biggest on each beat, then shrinking until the next one.

Related

  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
  • drumStep Which of the 16 steps of the bar the drum machine just played, from 0 to 15.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • beat True for one frame each time the microphone (or the drum machine) hears a beat.

p5/oav-p5.js, line 1447

drawRhythm()#

functionoav-p5
drawRhythm([x], [y], [w], [h], [style], [o])

Also: OAVp5.drums.draw

Draws the rhythm on the canvas as a grid, a circle or a spiral, with a playhead and the hits lighting up, so a recording has it too.

Call it in draw() after background(): drawRhythm(20, 100, 360, 100). Each row is one drum that plays in the pattern, a hit lights up the moment it is heard (the clock drumPhase reads), and a hit between two steps, such as a triplet, sits where it falls. The grid card is part of the page and a recording never shows it, but the canvas is what a recording takes, so this is how the rhythm gets into the video.

With no box it draws a strip along the bottom for 'grid' and a square in the middle for the other two. It draws with p5's own shapes between push() and pop(), so it leaves your styles alone and works in 2D and in WEBGL, in p5 1.x and 2.x. In WEBGL the origin is the middle of the canvas: drawRhythm(-width / 2, -height / 2, width, height, 'circle') fills it.

Before you call it

Start the machine with startDrums() first. Before it plays you get an empty four-lane grid.

Parameters

NameTypeDescription
xNumberLeft edge of the box to draw in. In WEBGL the middle of the canvas is (0, 0). If any of x, y, w, h is left out, the default box is used. optional
yNumberTop edge of the box. optional
wNumberWidth of the box. optional
hNumberHeight of the box. optional
styleString | Object'grid' (the 16 steps with a playhead), 'circle' (a ring of 16 per drum with a hand turned by drumPhase) or 'spiral' (the last few bars wound outwards, now at the tip). An object here is taken as o, with style inside it. optional default 'grid'
oObjectOptions: colors ({ kick: '#f66', … }, any p5 colour per drum), color (one colour for all), off (the empty steps), playhead (a colour, or false for none), bg (a background for the box) and bars (spiral only, 1 to 16, 4 if left out). optional

Returns

Boolean true if it drew something, false when the box is empty (w or h is not above 0) or there is no p5 global mode to draw with.

Notes

  • OAVp5.drums.draw(x, y, w, h, style, o) is the same function as a method.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 120);
}

function draw() {
  background(14, 16, 26);
  drawRhythm(20, 100, 360, 100);   // the 16 steps with a playhead
}
The pattern as a grid on the canvas itself: a white playhead crosses it, and every hit lights up as it sounds.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('bd(3,8), [~ sd]*2, hh*8, ~ ~ ~ cp', 104);
}

function draw() {
  background(14, 16, 26);
  drawRhythm(10, 10, 180, 180, 'circle');
  drawRhythm(210, 10, 180, 180, 'spiral', { bars: 4, colors: { kick: '#ff7b54' } });
  drawRhythm(20, 220, 360, 60, 'grid', { bg: 'rgba(255, 255, 255, 0.05)' });
}
The same rhythm three ways: a ring per drum with a hand, the last four bars as a spiral, and the grid along the bottom.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300, WEBGL);
  startDrums('four on floor', 120);
}

function draw() {
  background(14, 16, 26);
  drawRhythm(-width / 2, -height / 2, width, height, 'circle');   // (0, 0) is the middle in WEBGL
}
In a WEBGL canvas the box starts at (-width / 2, -height / 2), because (0, 0) is the middle.

Related

  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
  • drumStep Which of the 16 steps of the bar the drum machine just played, from 0 to 15.
  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • rhythmChanged() An event you define: it is called with the new line of rhythm every time the rhythm changes.

p5/oav-p5.js, line 1871

drumBpm#

variableoav-p5
drumBpm

Also: OAVp5.drums.bpm

The tempo of the drum machine in beats per minute.

A beat is a quarter note, four steps, so one step lasts 60 / drumBpm / 4 seconds and one bar 240 / drumBpm: at 120 a bar takes 2 seconds. To change the tempo call startDrums(bpm): startDrums(drumBpm + 5) speeds up by five. Writing to drumBpm yourself changes nothing.

startDrums() keeps the tempo between 30 and 300; the slider on the grid goes from 60 to 180.

Type
Number
Range
30..300

112 until you set another one. It follows startDrums() and the tempo slider on the grid.

Notes

  • A sketch that declares its own variable called drumBpm keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.drums.bpm instead.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 100);
  textAlign(CENTER, CENTER);
  textSize(32);
}

function keyPressed() {
  if (keyCode === UP_ARROW) startDrums(drumBpm + 5);
  if (keyCode === DOWN_ARROW) startDrums(drumBpm - 5);
}

function draw() {
  background(20);
  fill(255);
  text(round(drumBpm) + ' bpm', width / 2, height / 2);
}
Press the up and down arrow keys to change the tempo by 5; the number shows it.

Related

  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
  • beatPhase How far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
  • drumBar Which bar the drum machine is in, counted from 1 when it started.

p5/oav-p5.js, line 317

drumHit()#

eventoav-p5
function drumHit(lane, velocity) {}

An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.

Define function drumHit(lane, velocity) { … } and oav-p5 calls it for every hit of every drum, the way it calls notePressed() for a key. It is called in the frame in which the hit is heard, right before draw().

The same hit also calls the microphone's functions, so you can pick the grain you want: kickDetected() for the kick, snareDetected() for snare, clap and rim, hatDetected() for the three hats, and beatDetected() (with beat true for a frame) for kick, snare, clap and rim, once per step. Toms, crash, ride and cowbell reach only drumHit(). On one step the order is: cueFired(), barStarted(), stepStarted(), then the hits.

Parameters

NameTypeDescription
laneStringThe drum that sounded: 'kick', 'snare', 'hat' or 'clap' for the grid, and also 'openhat', 'pedalhat', 'rim', 'lowtom', 'tom', 'hightom', 'crash', 'ride' or 'cowbell' for a line of rhythm.
velocityNumberHow hard, from 0.15 to 1. It varies a little from hit to hit, because the machine plays slightly softer or louder on purpose.

Notes

  • While the tab is hidden nobody sees a frame: the clock keeps counting, but no hit is told, and nothing is told twice when the tab comes back. An error thrown inside drumHit() is printed, with its file and line, and the next hit still comes.

Examples

jsOpenProcessing ↗
const COLORS = { kick: '#ff6b6b', snare: '#ffd166', hat: '#6fd3ff', clap: '#c792ff' };
const lit = { kick: 0, snare: 0, hat: 0, clap: 0 };

function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 120);
  noStroke();
}

function drumHit(lane, velocity) {
  lit[lane] = velocity;
}

function draw() {
  background(20);
  Object.keys(lit).forEach((lane, i) => {
    fill(COLORS[lane]);
    circle(60 + i * 93, height / 2, 30 + lit[lane] * 70);
    lit[lane] *= 0.85;
  });
}
Four circles, one per drum: each swells with the hit and shrinks back, and a harder hit swells more.
jsOpenProcessing ↗
let last = '';

function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, cr ~ ~ ~, lt(3,8), hh*8', 100);
  textAlign(CENTER, CENTER);
  textSize(32);
}

function drumHit(lane, velocity) {
  last = lane;
}

function draw() {
  background(20);
  fill(255);
  text(last, width / 2, height / 2);
}
The name of the latest drum that sounded: a line of rhythm brings crash and low tom, which only drumHit() reports.

Related

  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • kickDetected() You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
  • beatDetected() You define this function; oav-p5 calls it each time the microphone (or the drum machine) hears a beat.
  • stepStarted() An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.
  • barStarted() An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
  • OAVp5.drums.on() Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.

p5/oav-p5.js, line 1498

drumPhase#

variableoav-p5
drumPhase

Also: OAVp5.drums.phase

How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.

Use it for anything that should move continuously with the music: rotate(drumPhase * TWO_PI) turns once per bar. drumStep says the same thing in 16 jumps; drumPhase is the smooth version. The value is what the listener hears, which is the machine's position corrected for the latency of the speakers, so a drawing driven by it lands on the beat you hear.

A bar has four beats; beatPhase is the same idea for one beat.

Type
Number
Range
0..1

Read fresh each time you look at it, so it is never a frame old. 0 at the start of a bar, close to 1 at its end, then 0 again. It is 0 before the machine has played and holds still while it is stopped.

Before it has a value

Start the machine with startDrums(); until it plays, drumPhase is 0.

Notes

  • A sketch that declares its own drumPhase keeps it, and oav-p5 does not write that name. OAVp5.drums.phase is the same number under a name nothing else can take.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 120);
  strokeWeight(4);
}

function draw() {
  background(20);
  stroke(255, 211, 107);
  translate(width / 2, height / 2);
  rotate(drumPhase * TWO_PI - HALF_PI);
  line(0, 0, 110, 0);   // a hand that turns once per bar
}
A clock hand that turns exactly once every bar, smoothly.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 110);
  noStroke();
}

function draw() {
  background(20);
  const a = drumStep / 16 * TWO_PI - HALF_PI;   // hops from step to step
  const b = drumPhase * TWO_PI - HALF_PI;       // moves all the time
  fill(255, 120, 90);
  circle(200 + cos(a) * 100, 150 + sin(a) * 100, 20);
  fill(120, 200, 255);
  circle(200 + cos(b) * 70, 150 + sin(b) * 70, 20);
}
Two dots on two circles: the orange one hops from step to step, the blue one runs smoothly.

Related

  • beatPhase How far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
  • drumStep Which of the 16 steps of the bar the drum machine just played, from 0 to 15.
  • drumBar Which bar the drum machine is in, counted from 1 when it started.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • drawRhythm() Draws the rhythm on the canvas as a grid, a circle or a spiral, with a playhead and the hits lighting up, so a recording has it too.

p5/oav-p5.js, line 728

drumsAreOn#

variableoav-p5
drumsAreOn

Whether the drum machine is playing right now.

Use it to tell "the drums are not running" from "the drums are at step 0": drumStep is 0 in both cases. if (drumsAreOn) { … } is also the right guard before you read drumPhase. It changes only when the machine starts or stops, and reading it never starts anything.

For the reason behind a false read OAVp5.drums.state: 'loading' while the module arrives, 'stopped' when the grid is up but not playing, 'error' when it could not load.

Type
Boolean

True while the machine plays, false at the start, after stopDrums(), after an error and while the grid's own checkbox is unticked.

Before it has a value

It becomes true after startDrums() (or 🥁 in the 🎹 panel) has loaded the machine, usually within a fraction of a second.

Notes

  • A sketch that declares its own variable called drumsAreOn keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.drums.state === 'on' instead.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('half time', 100);
  textAlign(CENTER, CENTER);
}

function draw() {
  background(20);
  fill(drumsAreOn ? color(120, 220, 140) : color(220, 160, 60));
  circle(width / 2, height / 2, 80);
  fill(255);
  text(drumsAreOn ? 'drums are on' : 'drums are not running', width / 2, height - 40);
}
An amber circle for a moment while the machine loads, then green.

To be confirmed

After stopDrums() and a second startDrums() it stayed false while the machine played again (Chromium, oav-p5 0.8.0). Not confirmed whether that is intended.

Related

  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • stopDrums() Stops the drum machine and hides its card.
  • drumStep Which of the 16 steps of the bar the drum machine just played, from 0 to 15.
  • soundIsOn Whether notes are being heard right now: true once startSound() has finished, false before and after.
  • micIsOn Whether the microphone is running right now.
  • recordingIsOn Whether a recording is running right now.

p5/oav-p5.js, line 316

rhythmChanged()#

eventoav-p5
function rhythmChanged(src, how) {}

An event you define: it is called with the new line of rhythm every time the rhythm changes.

The line over the grid and the cells edit each other, so the rhythm can change without your code: a click on a cell, a line typed into the card, ↶. rhythmChanged(src, how) tells the sketch, so it can show the line on the canvas (where a recording has it) or store it. It is also called once after startDrums(), a moment after the rhythm module has loaded, with how equal to 'code'.

Parameters

NameTypeDescription
srcStringThe line of rhythm that plays now, for example 'bd*4, [~ sd]*2, hh*8'. A preset or a grid you drew comes as the line it is written as.
howStringWhat changed it: 'code' (startDrums()), 'cell' (a click on a cell of the grid), 'type' (typed into the line on the card, or OAVp5.drums.edit()), 'undo' (↶ or OAVp5.drums.undo()) or 'menu' (a preset picked on the card).

Examples

jsOpenProcessing ↗
let current = '';
let how = '';

function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, [~ sd]*2, hh*8', 110);
  textAlign(CENTER, CENTER);
}

function rhythmChanged(src, mode) {
  current = src;
  how = mode;
}

function mousePressed() {
  OAVp5.drums.edit('bd(3,8), [~ sd]*2, hh*16');   // as if typed on the card
}

function draw() {
  background(20);
  fill(255);
  text(current, width / 2, height / 2 - 10);
  text('changed by: ' + how, width / 2, height / 2 + 20);
  text('click the canvas, or click a cell of the grid', width / 2, height - 30);
}
The line that plays and what changed it: "code" at the start, "type" after a click on the canvas, "cell" when you click a cell of the grid.

Related

  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • OAVp5.drums.src The line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
  • OAVp5.drums.edit() Plays a line of rhythm as if it was typed on the card, and returns whether it worked and, if not, what is wrong with it.
  • OAVp5.drums.undo() Goes back one line of rhythm, like ↶ on the card; returns true if there was an earlier line.
  • OAVp5.drums.history The earlier lines of rhythm, oldest first: every click on a cell, typed line or menu pick leaves the line before it here.
  • OAVp5.drums.on() Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
  • drawRhythm() Draws the rhythm on the canvas as a grid, a circle or a spiral, with a playhead and the hits lighting up, so a recording has it too.

p5/oav-p5.js, line 1637

startDrums()#

functionoav-p5
startDrums([pattern], [bpm])

Also: OAVp5.drums.start

Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

Call it once in setup(): startDrums('breakbeat', 120). A grid of 16 steps by 4 lanes (kick, snare, hat, clap) appears at the top of the page, with the tempo and the bar it is in, and plays a synthesized drum kit (no sample files). The grid is part of the page, not of the canvas: a recording never shows it, so use drawRhythm() to draw the rhythm on the canvas. A click on a cell changes the beat, and 🥁 in the 🎹 panel starts and stops the machine with no code at all.

The machine is also a clock and a source of events. Every hit calls drumHit() and the microphone's own functions (kickDetected(), snareDetected(), hatDetected(), beatDetected()), so a sketch written for the microphone moves over unchanged. drumStep, drumBar, drumPhase and beatPhase say where the machine is, and cue() lets you plan what happens at which bar.

Calling it again while it plays changes the pattern or the tempo without restarting: startDrums(90) only changes the tempo, startDrums('latin') only the pattern. A line of rhythm is any text that is not a preset name. A word is a drum (bd kick, sd snare, hh hat, oh open hat, cp clap, rim, lt mt ht toms, cr crash, rd ride, cb cowbell), ~ is a rest, words separated by spaces share the bar evenly, [ ] groups, *4 repeats, , puts another line on top, < > takes one per bar and bd(3,8) spreads 3 hits over 8 steps. A mistake shows a card that points at the character and says how to fix it, and 'four on floor' plays meanwhile.

A line copied from strudel.cc works too: startDrums('s("bd*4, [~ sd]*2").fast(2)'). s("…") and sound("…") with .fast(n) and .slow(n) are read, setcpm(110/4) sets the tempo if you give none, .bank(…) is left out (oav-p5 plays its own kit), and any other call in the chain gets a card that says it is not supported yet. The rest plays.

Before you call it

Nothing before the call. The counting, drumHit() and the other callbacks start at once, but a browser lets the page make sound only after a click or key press on it, so the drums are silent until the first one. A card at the top says so after a moment.

Parameters

NameTypeDescription
patternString | Object | NumberWhat to play. A preset name ('four on floor', 'breakbeat', 'half time', 'latin', 'sparse'). A line of rhythm such as 'bd*4, [~ sd]*2, hh*8', or one copied from strudel.cc. Your own grid { kick: 'x...x...x...x...', snare, hat, clap } (x or 1 is a hit, a lane you leave out is silent, {} is a silent clock). A number alone is the tempo. Leave it out to keep what plays now ('four on floor' at the start). optional
bpmNumberThe tempo in beats per minute (a beat is four steps). Kept between 30 and 300. Leave it out to keep the current tempo: 112 at the start, or the one a copied Strudel line sets with setcpm(…). optional

Returns

Promise<Boolean> Resolves to true once the machine plays (or when a later startDrums() took over). Resolves to false when stopDrums() came first, when there is no page to put the grid on, or when @openav/drums could not be loaded.

Notes

  • Drum sound does not need startSound(): the machine has its own kit. Pads on MIDI channel 10 play the same kit once startSound() is on.
  • The card has – to fold the grid away and × to stop the machine. A tab in the background keeps counting and playing. The loading options drums (lookahead or frame) and drumlatency tune the clock.

Examples

jsOpenProcessing ↗
let swell = 0;

function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 120);   // the grid appears at the top of the page
}

function kickDetected(level) {
  swell = 1;
}

function draw() {
  background(20);
  swell *= 0.9;
  circle(width / 2, height / 2, 60 + swell * 120);
}
A grid appears at the top of the page. Click the canvas once to hear it: the circle jumps on every kick.
jsOpenProcessing ↗
const LINES = ['bd*4, [~ sd]*2, hh*8', 'bd(3,8), [~ sd]*2, hh*16', 'bd ~ ~ bd, ~ sd ~ ~, [hh hh]*4'];
let n = 0;

function setup() {
  createCanvas(400, 300);
  startDrums(LINES[0], 110);
  textAlign(CENTER, CENTER);
}

function mousePressed() {
  n = (n + 1) % LINES.length;
  startDrums(LINES[n]);   // changes the rhythm while it plays
}

function draw() {
  background(20);
  fill(255);
  text('click to change the rhythm', width / 2, height / 2 - 20);
  text(OAVp5.drums.src, width / 2, height / 2 + 20);
}
Click the canvas to step through three rhythms: the line of text under the caption is the one that plays now, and the grid at the top follows it.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums({}, 90);   // no lane has a hit: nothing to hear, the clock still counts
  noStroke();
}

function draw() {
  background(20);
  for (let i = 0; i < 16; i++) {
    fill(i === drumStep ? 255 : 90);
    circle(30 + i * 22.5, height / 2, i === drumStep ? 18 : 8);
  }
}
A silent clock: sixteen dots with the current step enlarged, running at 90 bpm with no sound at all.

Related

  • stopDrums() Stops the drum machine and hides its card.
  • drumsAreOn Whether the drum machine is playing right now.
  • drumHit() An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
  • drumStep Which of the 16 steps of the bar the drum machine just played, from 0 to 15.
  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
  • drawRhythm() Draws the rhythm on the canvas as a grid, a circle or a spiral, with a playhead and the hits lighting up, so a recording has it too.
  • cue() Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
  • rhythmChanged() An event you define: it is called with the new line of rhythm every time the rhythm changes.
  • beat True for one frame each time the microphone (or the drum machine) hears a beat.

p5/oav-p5.js, line 1731

stopDrums()#

functionoav-p5
stopDrums()

Also: OAVp5.drums.stop

Stops the drum machine and hides its card.

The machine stops playing, the grid disappears and drumsAreOn becomes false. The numbers keep the last value they had: drumStep, drumBar and cueName stay where they were, and drumPhase and beatPhase hold still. startDrums() afterwards begins again from bar 1.

It also works while the machine is still loading: the pending startDrums() then resolves to false. The × on the card does the same thing.

Returns

Promise<Boolean> A promise that is already resolved to false, the new state of the machine.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 120);
  textAlign(CENTER, CENTER);
}

function mousePressed() {
  stopDrums();
}

function draw() {
  background(drumsAreOn ? 40 : 100);
  fill(255);
  text(drumsAreOn ? 'click to stop the drums' : 'stopped at step ' + drumStep, width / 2, height / 2);
}
Click the canvas: the grid disappears, the background turns light and the text shows the step it stopped on.

To be confirmed

After stopDrums() and a second startDrums() on the same page the machine plays and counts again, but drumsAreOn and OAVp5.drums.state stayed false and 'off' when this was tried (Chromium, oav-p5 0.8.0). Whether that is intended is not confirmed.

Related

  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • drumsAreOn Whether the drum machine is playing right now.
  • stopSound() Turns sound off and lets every ringing note go; the instrument stays loaded, so startSound() brings it back at once.

p5/oav-p5.js, line 1732

OAVp5.drums#

objectoav-p5
OAVp5.drums

The object behind the drum names: the state of the machine, its counters, the line of rhythm, its history and the loaded machine.

OAVp5.drums (also reachable as midi.drums) is what startDrums(), drumStep and the rest are built on. It holds the numbers (state, step, bar, count, bpm, phase, beatPhase), the rhythm (pattern, src, copyText, strudelUrl, history, edit(), undo()), the cues (cue(), cues, lastCue), a way to listen (on()), what the machine hears right now (now) and handles for the curious (machine, clock).

start, stop and draw are the same as startDrums(), stopDrums() and drawRhythm(). For a sketch the global names are enough. This object is for tools, libraries (Hydra, for one), the console and debugging.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('latin', 110);
  textSize(14);
}

function draw() {
  background(20);
  fill(255);
  const d = OAVp5.drums;
  text('state: ' + d.state, 20, 30);
  text('pattern: ' + d.pattern, 20, 60);
  text('bar ' + d.bar + ', step ' + d.step + ', count ' + d.count, 20, 90);
  text('bpm: ' + d.bpm, 20, 120);
}
Four lines read from the drums object while the machine plays.

Related

  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • OAVp5.drums.state Where the drum machine is: 'off', 'loading', 'on', 'stopped' or 'error'.
  • OAVp5.drums.on() Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
  • OAVp5.sound The object behind the sound names: the state, the instrument, the error, the loading progress and the loaded parts of @openav/sound.
  • OAVp5.record The object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.

p5/oav-p5.js, line 1471

OAVp5.drums.copyText#

variableoav-p5
OAVp5.drums.copyText

The text the Copy button on the card puts on the clipboard: the rhythm that plays now as a ready startDrums("…", bpm) call.

Click cells, type a line, find a rhythm you like, and paste this into setup() to keep it. The tempo is the current one, rounded to two decimals.

Type
String

For example startDrums("bd*4, [~ sd]*2, hh*8", 110). Read fresh each time; '' before the machine has loaded.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 120);
  textSize(12);
}

function draw() {
  background(20);
  fill(255);
  text(OAVp5.drums.copyText, 20, 20, width - 40, 200);
}
The line to paste back into setup(); click cells on the grid and it follows.

Related

  • OAVp5.drums.src The line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
  • OAVp5.drums.strudelUrl The rhythm that plays now as a link that opens it at strudel.cc, or an empty string when there is none.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 1467

OAVp5.drums.edit()#

functionoav-p5
OAVp5.drums.edit(text)

Plays a line of rhythm as if it was typed on the card, and returns whether it worked and, if not, what is wrong with it.

Unlike startDrums(text), which plays 'four on floor' when the line has a mistake, edit() keeps the line that was playing and tells you why the new one failed, so a live-coding sketch can show the message. The line goes into OAVp5.drums.history and rhythmChanged() is called with how equal to 'type'. A mistake also shows the card that points at the character.

Before you call it

The machine and the rhythm module must have loaded: call startDrums() first and give it a moment.

Parameters

NameTypeDescription
textStringThe line of rhythm, for example 'bd*4, [~ sd]*2, hh*8', or a line copied from strudel.cc.

Returns

Object { ok: true, error: null, text } when the line plays. { ok: false, error, text } when it has a mistake: error has code, index and length (where the bad word is in text), a message and a hint, and the last line that worked keeps playing. Before the machine has loaded: { ok: false, error: null }.

Examples

jsOpenProcessing ↗
const TRIES = ['bd*4, hh*8', 'bd(3,8), [~ sd]*2', 'bd*4, xyz*8'];
let n = 0;
let result = '';

function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, [~ sd]*2, hh*8', 110);
  textAlign(CENTER, CENTER);
}

function mousePressed() {
  const r = OAVp5.drums.edit(TRIES[n % TRIES.length]);
  result = r.ok ? 'ok: ' + r.text : 'wrong: ' + r.error.message;
  n++;
}

function draw() {
  background(20);
  fill(255);
  text('click to try the next line', width / 2, 40);
  text(result, width / 2, height / 2 - 10);
  text('playing: ' + OAVp5.drums.src, width / 2, height / 2 + 30);
}
Click three times: the first two lines play; the third has a mistake, so the message is shown and the line before it keeps playing.

Related

  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • OAVp5.drums.undo() Goes back one line of rhythm, like ↶ on the card; returns true if there was an earlier line.
  • OAVp5.drums.history The earlier lines of rhythm, oldest first: every click on a cell, typed line or menu pick leaves the line before it here.
  • rhythmChanged() An event you define: it is called with the new line of rhythm every time the rhythm changes.
  • OAVp5.drums.src The line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.

p5/oav-p5.js, line 1465

OAVp5.drums.history#

variableoav-p5
OAVp5.drums.history

The earlier lines of rhythm, oldest first: every click on a cell, typed line or menu pick leaves the line before it here.

OAVp5.drums.undo() takes the last line off this list and plays it. Changes made by startDrums() are not recorded here, only the ones made on the card or with OAVp5.drums.edit(). The array is the machine's own, so treat it as read-only.

Type
Array<String>

Empty at the start, and at most 50 lines long. The line that plays now is not in it: that is OAVp5.drums.src.

Examples

jsOpenProcessing ↗
const LINES = ['bd*4, hh*8', 'bd(3,8), hh*8', 'bd ~ ~ bd, [~ sd]*2, hh*16'];
let n = 0;

function setup() {
  createCanvas(400, 300);
  startDrums(LINES[0], 110);
  textSize(14);
}

function mousePressed() {
  n = (n + 1) % LINES.length;
  OAVp5.drums.edit(LINES[n]);
}

function draw() {
  background(20);
  fill(255);
  text('click to edit the line', 20, 30);
  text('history: ' + OAVp5.drums.history.length + ' earlier lines', 20, 60);
  text(OAVp5.drums.history.join('\n'), 20, 90);
}
Each click edits the line and adds the one before it to the history list.

Related

  • OAVp5.drums.undo() Goes back one line of rhythm, like ↶ on the card; returns true if there was an earlier line.
  • OAVp5.drums.edit() Plays a line of rhythm as if it was typed on the card, and returns whether it worked and, if not, what is wrong with it.
  • OAVp5.drums.src The line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
  • rhythmChanged() An event you define: it is called with the new line of rhythm every time the rhythm changes.

p5/oav-p5.js, line 1461

OAVp5.drums.now#

objectoav-p5
OAVp5.drums.now

A snapshot of what the machine hears right now: the bar, the step, both phases and which words of the line are lit.

It is what the card uses to light the word that sounds. Read it when you want the line and its lit words in one go, for example to draw the line on the canvas with the sounding word marked. Before the machine has loaded it answers with zeros and an empty text.

Type
Object
Range
{ bar, step, phase, beatPhase, tokens, mode, text }

A new object on every read. phase and beatPhase are drumPhase and beatPhase. tokens is a list of { start, end }: the places in text of the words that sound now. mode is 'string' for a line of rhythm and 'grid' for a preset or a grid you drew. text is the line.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, [~ sd]*2, hh*8', 80);
  textFont('monospace');
  textSize(18);
  noStroke();
}

function draw() {
  background(20);
  const n = OAVp5.drums.now;
  fill(255, 211, 107);
  for (const t of n.tokens) {
    rect(20 + textWidth(n.text.slice(0, t.start)), 150, textWidth(n.text.slice(t.start, t.end)), 3);   // underline the sounding words
  }
  fill(255);
  text(n.text, 20, 140);
}
The line of rhythm with a yellow underline that follows the words as they sound.

Related

  • OAVp5.drums.src The line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
  • beatPhase How far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
  • drumStep Which of the 16 steps of the bar the drum machine just played, from 0 to 15.
  • drumBar Which bar the drum machine is in, counted from 1 when it started.

p5/oav-p5.js, line 1449

OAVp5.drums.on()#

functionoav-p5
OAVp5.drums.on(name, fn)

Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.

The functions you define (drumHit, barStarted …) are one listener each. on() is for the cases where that does not fit: several listeners for one event, a library or a Hydra sketch that has no drumHit() of its own, a listener you want to remove again. It is the same idea as midi.on(). An error thrown inside fn is printed and the other listeners still run.

Parameters

NameTypeDescription
nameStringWhat to listen to: 'hit' (like drumHit()), 'step' (like stepStarted()), 'bar' (like barStarted()), 'cue' (like cueFired()) or 'rhythm' (like rhythmChanged()). Any other name throws an error that lists them.
fnFunctionCalled with the same arguments as the function it matches: (lane, velocity), (step, bar), (bar), (name, bar) or (src, how).

Returns

Function A function: call it to stop listening.

Examples

jsOpenProcessing ↗
let bars = 0;
let off = null;

function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 160);
  off = OAVp5.drums.on('bar', (bar) => { bars++; });   // returns a function that stops it
  textAlign(CENTER, CENTER);
}

function mousePressed() {
  if (off) { off(); off = null; }   // stop counting
}

function draw() {
  background(20);
  fill(255);
  text('bars counted: ' + bars, width / 2, height / 2);
  text(off ? 'click to stop counting' : 'stopped', width / 2, height / 2 + 30);
}
A counter that goes up once per bar; a click removes the listener and the number stops.

Related

  • drumHit() An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
  • stepStarted() An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.
  • barStarted() An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
  • cueFired() An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
  • rhythmChanged() An event you define: it is called with the new line of rhythm every time the rhythm changes.
  • OAVp5.drums The object behind the drum names: the state of the machine, its counters, the line of rhythm, its history and the loaded machine.

p5/oav-p5.js, line 1441

OAVp5.drums.pattern#

variableoav-p5
OAVp5.drums.pattern

What the drum machine plays, as a name: the preset's name, 'custom' for a grid, or the line of rhythm itself.

After startDrums('breakbeat') it is 'breakbeat', after startDrums({ kick: 'x...x...x...x...' }) it is 'custom', and after startDrums('bd*4, hh*8') it is that line. A click on a cell of a preset makes it 'custom'. To get the line of rhythm in every case, read OAVp5.drums.src: a preset also has one.

Type
String

'four on floor' until you choose another. It follows what plays, including clicks on the grid.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 110);
  textAlign(CENTER, CENTER);
  textSize(24);
}

function draw() {
  background(20);
  fill(255);
  text(OAVp5.drums.pattern, width / 2, height / 2);
}
Shows "breakbeat"; click a cell of the grid and it turns into "custom".

Related

  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • OAVp5.drums.src The line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
  • rhythmChanged() An event you define: it is called with the new line of rhythm every time the rhythm changes.

p5/oav-p5.js, line 1437

OAVp5.drums.src#

variableoav-p5
OAVp5.drums.src

The line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.

Show it on the canvas so a recording has it (text(OAVp5.drums.src, 20, 30)), or keep it to come back to the rhythm later with startDrums(src). rhythmChanged() tells you when it changes, and OAVp5.drums.copyText is the same line wrapped in a startDrums("…") call.

Type
String

Read fresh each time. A preset or a grid you drew shows as the line it is written as, and it changes with every click on a cell and every typed edit.

Before it has a value

It is '' until startDrums() has loaded the machine and the rhythm module, a fraction of a second.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('half time', 100);
  textAlign(CENTER, CENTER);
}

function draw() {
  background(20);
  fill(255);
  text(OAVp5.drums.src || 'loading…', width / 2, height / 2);
}
The preset "half time" written as the line of rhythm it is; click cells on the grid and the line is rewritten.

Related

  • OAVp5.drums.pattern What the drum machine plays, as a name: the preset's name, 'custom' for a grid, or the line of rhythm itself.
  • OAVp5.drums.copyText The text the Copy button on the card puts on the clipboard: the rhythm that plays now as a ready startDrums("…", bpm) call.
  • OAVp5.drums.edit() Plays a line of rhythm as if it was typed on the card, and returns whether it worked and, if not, what is wrong with it.
  • rhythmChanged() An event you define: it is called with the new line of rhythm every time the rhythm changes.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 1457

OAVp5.drums.state#

variableoav-p5
OAVp5.drums.state

Where the drum machine is: 'off', 'loading', 'on', 'stopped' or 'error'.

'loading' is the moment after startDrums() while the module arrives, 'on' is playing, 'stopped' is a grid that is up but not playing (its checkbox is unticked), 'off' is where it starts and where stopDrums() leaves it, and 'error' means @openav/drums could not be loaded. drumsAreOn is state === 'on'.

Type
String
Range
off | loading | on | stopped | error

Starts as 'off' and changes when the machine loads, plays, is stopped or fails.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('sparse', 90);
  textAlign(CENTER, CENTER);
  textSize(28);
}

function draw() {
  background(20);
  fill(255);
  text(OAVp5.drums.state, width / 2, height / 2);
}
Shows "loading" for a moment, then "on". Untick the checkbox on the grid and it reads "stopped".

To be confirmed

After stopDrums() and a second startDrums() the state stayed 'off' while the machine played again (Chromium, oav-p5 0.8.0). Not confirmed whether that is intended.

Related

  • drumsAreOn Whether the drum machine is playing right now.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • stopDrums() Stops the drum machine and hides its card.
  • OAVp5.drums The object behind the drum names: the state of the machine, its counters, the line of rhythm, its history and the loaded machine.

p5/oav-p5.js, line 1436

OAVp5.drums.strudelUrl#

variableoav-p5
OAVp5.drums.strudelUrl

The rhythm that plays now as a link that opens it at strudel.cc, or an empty string when there is none.

It holds the line as s("…") with setcpm(bpm/4) (one bar here is one Strudel cycle of four beats), the same thing the "Open in Strudel" link on the card opens. Open it in a new tab: window.open(OAVp5.drums.strudelUrl) from a click handler.

Type
String

A https://strudel.cc/#… address, read fresh each time. '' before the machine has loaded.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, [~ sd]*2, hh*8', 110);
  textAlign(CENTER, CENTER);
}

function mousePressed() {
  if (OAVp5.drums.strudelUrl) window.open(OAVp5.drums.strudelUrl, '_blank');
}

function draw() {
  background(20);
  fill(255);
  text('click to open this rhythm in strudel.cc', width / 2, height / 2 - 15);
  text(OAVp5.drums.strudelUrl.slice(0, 40) + '…', width / 2, height / 2 + 15);
}
Shows the start of the link; a click opens it at strudel.cc in a new tab.

To be confirmed

Not tried: whether the sketch frame of OpenProcessing lets window.open open the link.

Related

  • OAVp5.drums.src The line of rhythm that plays now, such as 'bd*4, [~ sd]*2, hh*8'; an empty string until the machine has loaded.
  • OAVp5.drums.copyText The text the Copy button on the card puts on the clipboard: the rhythm that plays now as a ready startDrums("…", bpm) call.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 1459

OAVp5.drums.undo()#

functionoav-p5
OAVp5.drums.undo()

Goes back one line of rhythm, like ↶ on the card; returns true if there was an earlier line.

The line before the current one is played again and removed from the history. rhythmChanged() is called with how equal to 'undo'. On the card the same thing is ⌘Z or Ctrl+Z after touching it.

Returns

Boolean true if it went back one line, false if OAVp5.drums.history was empty or the machine is not loaded.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, hh*8', 110);
  textAlign(CENTER, CENTER);
}

function mousePressed() {
  OAVp5.drums.edit('bd(3,8), hh*16');
}

function keyPressed() {
  if (key === 'z') OAVp5.drums.undo();
}

function draw() {
  background(20);
  fill(255);
  text(OAVp5.drums.src, width / 2, height / 2 - 10);
  text('click: change the line   z: undo', width / 2, height / 2 + 20);
}
Click to change the line, press z to go back to the one before.

Related

  • OAVp5.drums.history The earlier lines of rhythm, oldest first: every click on a cell, typed line or menu pick leaves the line before it here.
  • OAVp5.drums.edit() Plays a line of rhythm as if it was typed on the card, and returns whether it worked and, if not, what is wrong with it.
  • rhythmChanged() An event you define: it is called with the new line of rhythm every time the rhythm changes.

p5/oav-p5.js, line 1463

@openav/drums

drumEngine()#

function@openav/drums
drumEngine({ samples = null } = {})

A drum kit made of Web Audio synthesis, with no sample files: General MIDI drum notes go in, drum sounds come out.

It follows the small engine contract of @openav/sound, so the same shell and the same signals drive it: enable() makes the AudioContext, noteOn(note, vel01, time) plays a drum, set(key, value) sets a param. The kit has thirteen voices (see GM): kick, rim, snare, clap, low tom, hat, pedal hat, tom, open hat, high tom, crash, ride and cowbell. A note the kit does not know (any melodic note) plays nothing and returns null.

noteOn(note, vel01, time) returns { time, cancel() }, or null before enable() or for a note it does not play. time is on the kit's own clock (engine.context.currentTime); left out, or already past, the hit sounds now. cancel() silences a hit that has not started, which is how DrumTransport takes back what it scheduled when the tempo changes. hit('crash', 0.9) plays a voice by name (or a note number) for code that is not a MIDI pad. There is one param, kitVolume, from −30 to 0 dB (default −6).

Once enabled, engine.context is its AudioContext and engine.output is the node that goes to the speakers, which a recorder can tap. Before that both are null. noteOff() does nothing (drums ring out) and dispose() closes the context.

Before you call it

await kit.enable() from a click: browsers start audio only after a user gesture. mountDrums() and DrumTransport with autoEnable do that for you on the first hit.

Parameters

NameTypeDescription
samplesObjectPlay audio files instead of the synthesis: { 36: 'kick.wav', 38: 'snare.wav' }, MIDI note number to URL. They are fetched and decoded by enable(); a file that fails to load is skipped without an error, and a note of the kit then plays its synthesized sound instead. optional default null

Returns

Object An engine: { params, context, output, enable(), noteOn(note, vel01, time), hit(what, vel01, time), noteOff(), set(key, value), dispose() }.

Examples

js
import { drumEngine, GM } from '@openav/drums';

const kit = drumEngine();
console.log(kit.params.map((p) => p.key + ' ' + p.min + '..' + p.max + ' (' + p.def + ')').join());
// kitVolume -30..0 (-6)
console.log(kit.noteOn(GM.kick, 1), kit.context);   // null null: not enabled yet
module Before enable() the kit has a param but no sound and no AudioContext: noteOn() returns null.
js
import { drumEngine } from '@openav/drums';

const kit = drumEngine();
document.body.addEventListener('click', async () => {
  await kit.enable();                                          // from a click: audio may start
  kit.hit('kick');
  kit.hit('snare', 0.9, kit.context.currentTime + 0.5);        // half a second from now
  kit.hit('crash', 0.8, kit.context.currentTime + 1);
});
static Click the page: a kick, then a snare half a second later and a crash after a second. The times are on the kit's own audio clock.

Related

  • DrumTransport The runtime around a DrumScheduler: it hands every hit to the kit ahead of time on its audio clock, and tells the page each step and hit when it is heard.
  • mountDrums() Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
  • GM The General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.
  • voiceForNote() Tells which drum of the kit a General MIDI drum note plays, or null for a note it does not play.
  • Sound Makes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
  • toneEngine() The sound engine that plays the registered instruments with Tone.js through a filter, a reverb and a limiter.

import { drumEngine } from '@openav/drums'; packages/drums/index.js, line 38

drumName()#

function@openav/drums
drumName(voice, lang = 'en')

The name of a drum for the screen, in English or in Chinese: drumName('lowtom') is 'low tom'.

The drum names inside code stay English everywhere ('kick', 'snare'), and this is for the screen: the row labels of the drum machine card use it, and with 'zh' they are written in Chinese.

Parameters

NameTypeDescription
voiceStringThe drum: kick, snare, hat, clap, openhat, pedalhat, rim, lowtom, tom, hightom, crash, ride or cowbell.
lang'en' | 'zh''zh' for Chinese; anything else is English. optional default 'en'

Returns

String The name, or voice itself if it is not a drum of the kit.

Examples

js
import { drumName } from '@openav/drums';

console.log(drumName('kick'), drumName('kick', 'zh'));         // kick, then the Chinese name of the kick
console.log(drumName('openhat', 'zh'), drumName('lowtom', 'zh')); // Chinese names of the open hat and the low tom
console.log(drumName('banjo', 'zh'));                           // banjo, not a drum: returned as is
module Logs a few drums in English and in Chinese, and what a name that is not a drum gives back.

Related

  • GM The General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.
  • mountDrums() Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
  • WORDS The drum words a line of rhythm may use and the drum each one plays: bd is kick, sd is snare, hh is hat.

import { drumName } from '@openav/drums'; packages/drums/index.js, line 809

DrumScheduler#

class@openav/drums
new DrumScheduler({ bpm = 112, swing = 0, humanize = 0.35, lookahead = 0.1, pattern = 'four on floor', seed = 0, random = Math.random } = {})

Pure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.

A DrumScheduler knows what plays in which bar, and when a bar happens, but it owns no clock: every time you give it is in seconds on one clock of your choosing, so it runs the same under a fake clock in a test as under the audio clock in a page. DrumTransport is the runtime that feeds it the real one. Use the scheduler on its own when you want drum events as data, for your own sound engine or a test.

The loop is: start(at) puts the downbeat of bar 1 at time at. Then, again and again, advance(now) schedules everything up to now + lookahead and returns the new events, in order. due(t) hands out the events whose time has come (they stay in queue until then). Two kinds of event come out: a step { kind: 'step', i, n, bar, pos, time } for every 16th, and a hit { kind: 'hit', voice, vel, pos, n, bar, token, word, layer, dur, time }. pos counts bars from the start (0 is the downbeat of bar 1, 1.5 is half way through bar 2), time is in your seconds, i is the step 0..15 and n counts steps since the start. For a grid, the step comes before its hits. token, word, layer and dur come from a line of rhythm and are null or undefined for a grid.

Positions turn into seconds by multiplication from the last tempo anchor, never by adding up frame times, so a thousand bars later nothing has drifted (the test measures under a microsecond). A tempo change adds an anchor, so the past keeps the tempo it had.

Parameters

NameTypeDescription
bpmNumberThe tempo, in beats per minute. A value that is not above 0 becomes 112. optional default 112
swingNumber0 to 0.5: every odd 16th is that share of a 16th late. 0 is straight. optional default 0
humanizeNumber0..1: a few milliseconds late and a little softer or louder, so it grooves instead of ticking. Use 0 for exact, repeatable times. optional default 0.35
lookaheadNumberSeconds advance() schedules beyond now. optional default 0.1
patternString | ObjectA preset name from PATTERNS, your own grid, or a compiled line of rhythm (see setPattern()). optional default 'four on floor'
seedNumberPassed to the rhythm line as eventsInBar(bar, { seed }), so each ? decides the same way every time. optional default 0
randomFunctionWhere humanize gets its randomness: a function that returns a number from 0 up to 1. Pass your own for repeatable output. optional default Math.random

Members

NameDescription
bpmgetterGetter and setter. The tempo; setting it does what setBpm(bpm) does.
stepDur()methodSeconds one 16th lasts at the current tempo: 60 / bpm / 4.
setPattern(p, { text } = {})methodA preset name, your own grid { kick: 'x...x...', … } (a lane left out is silent), or a compiled line of rhythm: anything with eventsInBar(bar, { seed }), such as compile(text). With a line, { text } is the text it came from. An unknown name plays 'four on floor'. Returns the new pattern.
toggleCell(lane, i)methodFlips one cell of the grid: toggleCell('snare', 4). The pattern becomes 'custom'. A line of rhythm is first turned into the grid it plays in bar 1.
gridOf(bar = 1)methodBar bar of what plays, on the 16 steps of the four lanes: { grid, offGrid, dropped }. offGrid counts hits that sat between steps and were moved to the nearest one, dropped lists the drums the grid has no lane for.
takeover(bar = 1)methodThe line of rhythm becomes the grid it plays in bar, and from then on the cells play. Returns what gridOf(bar) returns.
barEvents(bar)methodThe hits of bar bar (1 is the first) as [{ voice, pos, … }], pos 0..1 in the bar, from whichever source plays. A line that fails or names a drum the kit does not have leaves warning set and skips that hit.
posAt(t)methodWhere the music is at time t, in bars from the start. The downbeat of bar 1 is 0, and before it the value is negative.
timeAt(p)methodThe time when position p sounds, before swing and humanize. The reverse of posAt().
bpmAt(t)methodThe tempo at time t.
phaseAt(t)method{ pos, bar, beat } at time t: bar and beat are 0..1 through the bar and through the beat. 0 before the first downbeat.
start(at)methodStarts: the downbeat of bar 1 is at time at. Clears the queue and returns the scheduler.
stop()methodStops, and returns the events that were scheduled and not yet heard, so you can take them back from the kit.
setBpm(bpm, at)methodA new tempo from time at on. Left out, from the end of what is already scheduled, so nothing is taken back. Returns the events taken back ([] if none). Does nothing for a value that is not above 0.
rewind(at)methodTakes back everything scheduled at or after time at; the next advance() makes it again from the pattern and tempo as they are then. Returns the events taken back. Events already handed out by due() are never taken back.
advance(now)methodSchedules everything up to now + lookahead and returns the new events in order (they stay in queue until due() hands them out). If the page was frozen so long that more than 64 bars are due, it skips ahead instead of returning minutes of hits at once, and counts them in skipped. Does nothing, and returns [], when stopped.
due(t)methodThe events whose time is t or earlier, in order, removed from the queue.

Examples

js
import { DrumScheduler } from '@openav/drums';

const s = new DrumScheduler({ bpm: 120, humanize: 0 });
s.start(0);                                   // the downbeat of bar 1 is at time 0
s.advance(1.9);                               // schedule up to 1.9 + 0.1 seconds: one whole bar
const heard = s.due(0.5);                     // what is heard by 0.5 s
console.log(heard.filter((e) => e.kind === 'hit').map((e) => e.voice + '@' + e.time).join(' '));
// kick@0 hat@0 hat@0.25 kick@0.5 snare@0.5 hat@0.5
console.log(s.phaseAt(1).bar, s.posAt(2), s.timeAt(1));   // 0.5 1 2
module No clock, no audio: the scheduler is asked for one bar of four on the floor at 120 bpm and logs the hits heard in the first half second, with their times in seconds.
js
import { DrumScheduler } from '@openav/drums';
import { compile } from '@openav/pattern';

const s = new DrumScheduler({ bpm: 120, humanize: 0 });
s.setPattern(compile('bd [~ sd]*2'), { text: 'bd [~ sd]*2' });
console.log(s.mode, s.pattern);                          // string bd [~ sd]*2
console.log(s.barEvents(1).map((e) => e.voice + '@' + e.pos).join(' '));   // kick@0 snare@0.625 snare@0.875
const g = s.gridOf(1);
console.log(g.grid.snare.join(''), g.offGrid);           // 0000000000100010 0

s.start(10);
s.advance(10);
s.setBpm(60, 12);                                        // half the tempo from time 12
console.log(s.posAt(12), s.posAt(14));                   // 1 1.5
module A line of rhythm goes in: its first bar is read as hits, then as a 16-step grid, and a tempo change halves the speed at time 12 without moving what came before.

Related

  • DrumTransport The runtime around a DrumScheduler: it hands every hit to the kit ahead of time on its audio clock, and tells the page each step and hit when it is heard.
  • DrumSequencer A 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
  • compile() Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
  • toGrid() Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.
  • PATTERNS The five preset grids of the drum machine: 'four on floor', breakbeat, 'half time', latin and sparse.
  • GM The General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.

import { DrumScheduler } from '@openav/drums'; packages/drums/index.js, line 296

DrumSequencer#

class@openav/drums
new DrumSequencer({ onHit, onStep = null, bpm = 112, swing = 0.12, humanize = 0.35, pattern = 'four on floor' } = {})

A 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.

It is the simplest clock: no audio, no timers, no DOM. Make it with an onHit that does something, call toggle(true), then call update(dt) once per frame with the seconds since the last frame. It walks through the 16 steps of a bar (a 16th lasts 60 / bpm / 4 seconds) and calls onHit(lane, velocity) for each hit in the four lanes kick, snare, hat and clap. A hit is told in the frame in which it is due, so it can be late by up to a frame, and nothing happens in a hidden tab. For a steady beat under a heavy frame, use DrumTransport.

The grid is seq.grid: { kick: [0|1 × 16], snare, hat, clap }. seq.pattern is the preset name, or 'custom' once you changed a cell or gave your own grid. seq.playing, seq.bpm, seq.swing and seq.humanize are plain properties you can read and set.

Parameters

NameTypeDescription
onHitFunctiononHit(lane, velocity) for every hit: lane is 'kick', 'snare', 'hat' or 'clap', velocity is 0.15 to 1. You decide what a hit does: play a sound, draw, publish a signal.
onStepFunctiononStep(i, n) for every 16th step: i is 0..15, n counts steps since the start. It comes before that step's hits. A timeline counts bars on it. optional default null
bpmNumberThe tempo, in beats per minute. It can be changed later: seq.bpm = 90. optional default 112
swingNumber0 to 0.5: every odd 16th is that share of a 16th late. 0 is straight. optional default 0.12
humanizeNumber0..1: some hits a few milliseconds late and a little softer or louder. Use 0 for exact, repeatable velocities (0.9 on the beat, 0.6 between). optional default 0.35
patternString | ObjectA preset name from PATTERNS or your own grid. optional default 'four on floor'

Members

NameDescription
setPattern(p)methodA preset name ('breakbeat'), or your own grid { kick: 'x...x...x...x...', snare: …, hat: …, clap: … }; a lane you leave out is silent, and pattern becomes 'custom'. An unknown name plays 'four on floor'.
toggleCell(lane, i)methodFlips one cell: toggleCell('kick', 4). lane must be one of the four lanes. pattern becomes 'custom'.
toggle(on = !this.playing)methodtoggle(on) starts or stops; with no argument it flips. Starting begins from step 0, and a hit left over from before a stop never lands after the next start. Returns the new state.
stepDur()methodSeconds one 16th lasts at the current tempo: 60 / bpm / 4.
update(dt)methodMoves the clock forward by dt seconds and calls onStep and onHit for what falls in that time. Does nothing while stopped. Call it once per frame.

Examples

js
import { DrumSequencer } from '@openav/drums';

const hits = [];
const seq = new DrumSequencer({
  onHit: (lane, vel) => hits.push(lane + ' ' + vel.toFixed(1)),
  bpm: 120, swing: 0, humanize: 0,
});
seq.toggle(true);
for (let i = 0; i < 32; i++) seq.update(1 / 60);   // about half a second, in 60 fps frames
console.log(hits.join(', '));
// kick 0.9, hat 0.9, hat 0.6, kick 0.9, snare 0.9, hat 0.9
console.log(seq.pattern, seq.grid.kick.join(''));   // four on floor 1000100010001000
module Runs half a second of four on the floor with no sound and no screen, and logs the hits: a kick and a hat on the downbeat, the off-beat hat, then kick, snare and hat on beat 2.
js
import { DrumSequencer } from '@openav/drums';

const g = document.getElementById('c').getContext('2d');
let lit = 0;
const seq = new DrumSequencer({
  bpm: 110, pattern: { kick: 'x...x...x...x...', snare: '....x.......x...', hat: 'x.x.x.x.x.x.x.x.' },
  onStep: (i) => { lit = i; },          // onHit would be the place to play a sound
});
seq.toggle(true);
let last = performance.now();
(function frame(now) {
  seq.update((now - last) / 1000); last = now;
  g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
  for (let i = 0; i < 16; i++) { g.fillStyle = i === lit ? '#ffd166' : '#2a3348'; g.fillRect(12 + i * 23, 120, 18, 60); }
  requestAnimationFrame(frame);
})(last);
module Sixteen blocks on the canvas, and a bright one runs along them in time with the beat. There is no sound: onHit is where you would play one.

Related

  • DrumTransport The runtime around a DrumScheduler: it hands every hit to the kit ahead of time on its audio clock, and tells the page each step and hit when it is heard.
  • DrumScheduler Pure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
  • PATTERNS The five preset grids of the drum machine: 'four on floor', breakbeat, 'half time', latin and sparse.
  • laneSteps() Turns a lane written as text, like 'x...x...x...x...', or as a list into 16 steps of 0 and 1.
  • mountDrums() Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
  • GM The General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.

import { DrumSequencer } from '@openav/drums'; packages/drums/index.js, line 210

DrumTransport#

class@openav/drums
new DrumTransport({ scheduler = new DrumScheduler(), engine = null, onStep = null, onHit = null, onLate = null, autoEnable = false, now = perfNow, hidden = pageHidden, heartbeat = 'auto', tick = 0.025, latency = 'auto', stale = 0.25, startDelay = 0.05 } = {})

The runtime around a DrumScheduler: it hands every hit to the kit ahead of time on its audio clock, and tells the page each step and hit when it is heard.

A page that plays a hit in the frame it is due is late by up to a frame, and silent when its tab is hidden. DrumTransport fixes both. A timer (the heartbeat, from a Web Worker, 25 ms by default) looks 0.1 seconds ahead on the kit's own audio clock and hands the kit every hit with its exact start time. The page is told separately, in the frame in which each step and hit is heard: its time plus the output latency the browser reports. So the beat stays steady under a heavy draw() and in a hidden tab, and the callbacks still arrive in order: a step before its hits, hits in time order.

Call start(), then frame() once per animation frame. The heartbeat calls pump() for you. Tempo and pattern changes (setBpm(), setPattern(), toggleCell()) take back what was already handed over after now and make it again, so no hit is doubled or lost. Before the kit's AudioContext runs (no click yet) the clock runs on now() and the callbacks run but nothing sounds; once it runs, the hits from then on sound, and none from before.

A hidden page, or a frame loop that catches up more than stale seconds late, tells its steps quietly: ev.quiet is true, the clock moves and cues fire, but onHit is not called, and of several bar starts at once only the last has ev.announce. Nothing floods the page when the tab comes back. If a draw() ever holds the thread longer than the lookahead, the lookahead grows to 0.3 seconds (info().extended).

It has the getters playing, pattern, grid, mode, text, bpm, swing, humanize and lookahead of its scheduler; bpm, swing and humanize can also be set. You usually get one through mountDrums() with clock: 'lookahead'; build your own when you want the clock without the card.

Parameters

NameTypeDescription
schedulerDrumSchedulerThe DrumScheduler that decides what is played when. optional default new DrumScheduler()
engineObjectThe sound engine, usually a drumEngine(). If it has a context property (an AudioContext, or null before it is enabled), hits are scheduled ahead on that clock. If not, each hit is played by calling engine.noteOn(note, vel) at the moment it is heard. optional default null
onStepFunctiononStep(i, n, ev) for every 16th step when it is heard. ev.quiet and ev.announce say whether it was told quietly (below). optional default null
onHitFunctiononHit(voice, vel, ev) for every drum hit when it is heard. optional default null
onLateFunctiononLate(margin, transport): a hit was handed to the kit after its time, because the page held the main thread longer than the lookahead. margin is how many seconds early it was (negative or tiny). optional default null
autoEnableBooleanCall engine.enable() by start() when the engine has no AudioContext yet. optional default false
nowFunctionA function that returns the time in seconds. Pass your own to drive the transport from a fake clock in a test. optional default performance.now() / 1000
hiddenFunctionA function that returns true while nobody can see the page. optional default checks document.visibilityState
heartbeat'auto' | 'interval' | falseWho calls pump(): 'auto' a Web Worker timer (a hidden tab does not slow a worker), falling back to setInterval; 'interval' always setInterval; false (or 'none') nobody, and you call pump() or frame() yourself. optional default 'auto'
tickNumberSeconds between two heartbeats. optional default 0.025
latency'auto' | NumberSeconds between a hit being scheduled and being heard. 'auto' asks the browser (getOutputTimestamp(), outputLatency); a number sets it by hand. optional default 'auto'
staleNumberA step that is told more than this many seconds late (a frame loop catching up) is told quietly. optional default 0.25
startDelayNumberWhen the audio clock is running, the first downbeat is put this many seconds after start(), so the first hit is not already late. optional default 0.05

Members

NameDescription
playinggetterGetter. Whether the machine is playing.
patterngetterGetter. The preset name, 'custom' for your own grid, or the line of rhythm text.
gridgetterGetter. The grid as { kick: [0|1 × 16], snare, hat, clap }. For a line of rhythm it is what bar 1 looks like on the grid.
modegetterGetter. 'grid' or 'string' (a compiled line of rhythm).
textgetterGetter. The line of rhythm that plays, or '' for a grid.
bpmgetterGetter and setter. The tempo; setting it does what setBpm() does.
swinggetterGetter and setter. 0 to 0.5: how much later every odd 16th is, as a share of a 16th. Setting it makes the scheduled hits again.
humanizegetterGetter and setter. 0..1, a few milliseconds of lateness and a little change of velocity. Setting it makes the scheduled hits again.
lookaheadgetterGetter. Seconds handed to the kit ahead of time (0.1; it grows to 0.3 after a hit was late).
stepDur()methodSeconds one 16th lasts at the current tempo: 60 / bpm / 4.
times()methodThe master time line, in seconds: { sched, heard, live, ct }. sched is the earliest time a hit can still be handed to the kit, heard is what the listener hears right now, live is whether the audio clock runs. Until it does, it is now().
start()methodStarts from bar 1 (again, if it was playing) and returns true. When the audio clock runs, the first downbeat is startDelay seconds away. Starts the heartbeat.
stop()methodStops, takes back what was handed to the kit and has not started, and stops the heartbeat. Returns false.
toggle(on = !this.playing)methodtoggle(on) starts or stops; with no argument it flips. Starting begins at bar 1.
setPattern(p, o)methodA preset name, your own grid, or a compiled line of rhythm (with { text }, the line it came from). The change is heard at once. Returns the new pattern.
toggleCell(lane, i)methodtoggleCell(lane, i) flips one cell of the grid (lane is kick, snare, hat or clap, i is 0..15). A line of rhythm is first turned into the grid it plays.
takeover(bar)methodTurns the line of rhythm into the grid it plays in bar bar, so the cells can be edited. Returns { grid, offGrid, dropped }.
setBpm(v)methodA new tempo from now on. What was handed to the kit after now is taken back and made again at the new tempo. The same tempo again does nothing. Returns the tempo.
rewind()methodTakes back everything handed to the kit after now (with 10 ms of margin, so a hit already sounding is not cut) and makes it again from the pattern and tempo as they are. The setters call it for you.
pump()methodThe heartbeat: hand the next hits to the kit. When the page is hidden, or no frame has run for 0.1 seconds, it also tells the page what is heard. Call it yourself only with heartbeat: false.
frame()methodCall it once per animation frame. Hands over what is due, then tells the page every step and hit heard by now through onStep and onHit. Returns those events.
phase()method{ bar, beat, pos } where the listener is: how far through the bar and through the beat, 0..1, and bars since the start. It holds still while stopped.
info()methodHow the clock is doing: { clock, heartbeat, why, lookahead, late, extended, audio, latency, skipped, warning }. heartbeat is 'worker', 'interval' or 'none', late counts hits handed over late.
dispose()methodStops the machine.
heardAt(at)methodThe scheduler's time line (what is heard) at a moment at of the now clock. For follow().
follow({ pos = 0, at, bpm } = {}, { catchUp = 0.5, jump = 0.125, pull = 0.08, still = 0.0005 } = {})methodPlays on an external clock (Strudel, a DAW, a linked page): follow({ pos, at, bpm }) says that position pos (bars from its start, 0 is the downbeat of bar 1) is heard at at (the now clock, in seconds) at tempo bpm. Stopped, it starts so bar 1 lands on the external 0. Playing, an error over jump bars (0.125) restarts the same way; a smaller one is closed over catchUp seconds by a tempo a little faster or slower (at most pull of it, 0.08), then the external tempo again. An error under still (0.0005 seconds) is left alone. The second argument is { catchUp, jump, pull, still }. Returns the error in seconds (positive: the machine was behind). Call it with every new reading of the external clock.
followInfo()method{ bpm, err, jumps, steers, at } while following (err is the last correction in seconds), and null if it never followed.
unfollow()methodStops following. The machine keeps the tempo it has, on its own clock.

Examples

js
import { DrumScheduler, DrumTransport } from '@openav/drums';

let T = 100;                                         // a clock we move by hand, in seconds
const log = [];
const transport = new DrumTransport({
  scheduler: new DrumScheduler({ bpm: 120, humanize: 0 }),
  heartbeat: false, now: () => T,                    // no timer, our own clock
  onStep: (i) => { if (i % 4 === 0) log.push('step ' + i); },
  onHit: (voice, vel) => { if (voice !== 'hat') log.push(voice); },
});
transport.start();
for (let k = 0; k < 40; k++) { T += 1 / 20; transport.frame(); }   // two seconds in 20 fps frames
console.log(log.join(', '));
// step 0, kick, step 4, kick, snare, step 8, kick, step 12, kick, snare
console.log(transport.playing, transport.phase().bar.toFixed(2));  // true 1.00
module Runs a whole bar of four on the floor on a clock we move by hand, and logs what the page is told: the downbeats of each beat, the kicks and the snares on 2 and 4.
js
import { DrumScheduler, DrumTransport, drumEngine } from '@openav/drums';

const kit = drumEngine();
const transport = new DrumTransport({
  scheduler: new DrumScheduler({ bpm: 100, swing: 0.1, pattern: 'breakbeat' }),
  engine: kit, autoEnable: true,                      // the click that starts it also starts the audio
  onHit: (voice, vel) => console.log(voice, vel.toFixed(2)),
});
document.body.addEventListener('click', () => transport.start(), { once: true });
(function frame() { transport.frame(); requestAnimationFrame(frame); })();
static Click the page and a breakbeat starts on the kit's audio clock; the console logs each hit when it is heard. Switch to another tab: the beat keeps going.

Related

  • DrumScheduler Pure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
  • mountDrums() Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
  • drumEngine() A drum kit made of Web Audio synthesis, with no sample files: General MIDI drum notes go in, drum sounds come out.
  • DrumSequencer A 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.

import { DrumTransport } from '@openav/drums'; packages/drums/index.js, line 473

GM#

constant@openav/drums
GM = object, 13 keys

The General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.

Drum pads and drum machines send these notes on MIDI channel 10. The drum machine sends them as midi/note/on { note, vel, ch: 10 } for every hit, so anything that listens to MIDI hears the drums. The reverse, from a note to a drum, is voiceForNote(), which also knows the neighbouring General MIDI notes (35 is also a kick, 40 is also a snare).

Type
Object

{ kick: 36, rim: 37, snare: 38, clap: 39, lowtom: 41, hat: 42, pedalhat: 44, tom: 45, openhat: 46, hightom: 48, crash: 49, ride: 51, cowbell: 56 }: drum name to MIDI note number.

Examples

js
import { GM, voiceForNote } from '@openav/drums';

console.log(GM.kick, GM.snare, GM.hat);        // 36 38 42
console.log(Object.keys(GM).length);            // 13
console.log(voiceForNote(GM.crash));            // crash
module Logs the notes of three drums, how many drums the kit has, and that the note of a drum maps back to the drum.

Related

  • voiceForNote() Tells which drum of the kit a General MIDI drum note plays, or null for a note it does not play.
  • drumEngine() A drum kit made of Web Audio synthesis, with no sample files: General MIDI drum notes go in, drum sounds come out.
  • WORDS The drum words a line of rhythm may use and the drum each one plays: bd is kick, sd is snare, hh is hat.
  • drumHit() An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.

import { GM } from '@openav/drums'; packages/drums/index.js, line 23

laneSteps()#

function@openav/drums
laneSteps(v)

Turns a lane written as text, like 'x...x...x...x...', or as a list into 16 steps of 0 and 1.

It is how every grid you write is read: { kick: 'x...x...x...x...' } in DrumSequencer, DrumScheduler and startDrums(). 'x... x... x... x...' and '1000100010001000' are the same lane. Note that the dot is a rest here, and only here: in a line of rhythm (see compile()) a rest is ~.

Parameters

NameTypeDescription
vString | ArrayThe lane. In text, x, X and 1 are hits and everything else is a rest; spaces and | are ignored, so they can help the eye. In a list, 1 and true are hits.

Returns

Number[] An array of 16 numbers, 1 for a hit and 0 for a rest. A short lane is padded with rests, a long one is cut at 16.

Examples

js
import { laneSteps } from '@openav/drums';

console.log(laneSteps('x... x... x... x...').join(''));   // 1000100010001000
console.log(laneSteps([1, 0, true]).join(''));            // 1010000000000000
console.log(laneSteps('x').length, laneSteps('x'.repeat(40)).length);   // 16 16
module Logs a lane written with spaces, a lane written as a list, and that any length comes out as 16 steps.

Related

  • PATTERNS The five preset grids of the drum machine: 'four on floor', breakbeat, 'half time', latin and sparse.
  • DrumSequencer A 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
  • DrumScheduler Pure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
  • fromGrid() Turns a grid of steps, like the drum machine card, into the one line of rhythm a person would write for it.

import { laneSteps } from '@openav/drums'; packages/drums/index.js, line 199

mountDrums()#

function@openav/drums
mountDrums(container, { signals, engine = null, autoEnableEngine = true, onStep = null, onHit = null, onTakeover = null, bpm = 112, pattern = 'four on floor', clock = 'frame', lang = 'en', heartbeat = 'auto', latency = 'auto', now, hidden, rhythm = null, onRhythm = null, onLossy = null, onEditError = null, copyText = null, strudelComment = '' } = {})

Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.

mountDrums(el, { signals }) builds a drum machine card inside el: a play checkbox, a preset menu, a tempo slider, the bar it is in, and a grid with a lane per drum (kick, snare, hat, clap, and a row of their own for the crash, ride, toms and open hat that a line of rhythm uses). A click on a cell toggles it. The returned object lets you do from code what the card does with the mouse: play(true), setPattern('breakbeat'), setBpm(96).

You must call drums.update(dt) every frame, with the seconds since the last one. It moves the machine (with the frame clock), tells your onStep and onHit what is heard, paints the grid and lets the drum/*/env signals decay. A machine that is never updated does not move.

Hits travel the same road as a drummer heard through the microphone, so a sketch written for the microphone moves over unchanged: drum/kick, drum/snare, drum/hat, drum/clap as pulses with { level }, drum/<voice>/env as continuous 0..1 values, and midi/note/on { note, vel, ch: 10 } with the General MIDI drum note (see GM). A voice a line of rhythm adds, like drum/crash, is defined the first time it is hit.

Give it the pattern module (rhythm) and the line of rhythm is always shown over the grid: a preset shows the line its grid is ('four on floor' is bd*4, [~ sd]*2, hh*8), and the word that sounds lights up. Then the two edit each other. A click on a cell rewrites the line (with fromGrid()). Typing in the line changes the cells: Enter, or 400 ms without a key, plays what was typed. A mistake is marked red, onEditError is told, and the last line that worked keeps playing. Escape drops what was not played yet. Every change is kept in history; ↶ on the card, or Cmd/Ctrl+Z after touching the card, goes back one. Copy puts the line on the clipboard and Open in Strudel opens it at strudel.cc in a new tab (strudelUrl()).

What the returned object offers:

text
play(on)               start (true) or stop (false) like the checkbox; no argument flips; returns whether it plays
setPattern(p, {text})  a preset name, your own grid, or a compiled line (text: the line it came from); the card follows
setBpm(bpm)            the tempo, like the slider; returns the tempo
update(dt)             call every frame
deliver()              tell onStep/onHit what is heard by now without painting (call it right before you draw)
phase()                { bar, beat, pos }: how far through the bar and the beat the listener is, 0..1
now()                  { bar, step, phase, beatPhase, tokens, mode, text } for the display
info()                 { clock, heartbeat, lookahead, late, extended, audio, latency, ... } the health of the clock
useRhythm(mod)         hand over @openav/pattern later (it can load on its own)
src()                  the line that plays now ('' with a grid and no rhythm module)
strudelUrl()           the line as a strudel.cc link, or ''
edit(text)             play a line as if typed: { ok, error }; a mistake keeps the last line playing
history, undo()        the lines before, oldest first; undo goes back one and returns whether it did
copy()                 put copyText() on the clipboard: Promise<{ ok, text }>
barEvents(bar)         the hits of a bar: [{ voice, pos, ... }]
voices()               the rows on the grid now
seq, engine, root      the DrumSequencer or DrumTransport, the sound engine, the card's element
dispose()              stop, release the kit and take the card out

Before you call it

A page with a DOM, and update(dt) called every frame. Sound waits for the browser: the kit's AudioContext is made by the first hit (or by play() with the lookahead clock), and until the page has had a click it stays suspended. Callbacks and signals run from the start; the sound comes in on the first click, and nothing from before it is played late.

Parameters

NameTypeDescription
containerHTMLElementThe element to put the card in. dispose() takes it out again.
signalsSignalsA Signals bus. Every hit is published as drum/<voice> (a pulse { level }), drum/<voice>/env (0..1, decays) and midi/note/on { note, vel, ch: 10 }. Without one the machine plays and calls your functions, and publishes nothing. optional
engineObjectThe sound engine. Left out, the machine makes its own drumEngine(). Anything with enable() and noteOn(note, vel01, time) works. optional default null
autoEnableEngineBooleanSwitch the engine on by the first hit (or by play() with the lookahead clock), so no extra click is needed beyond the one the browser wants. optional default true
onStepFunctiononStep(i, n, ev), once for every 16th step when it is heard. i is 0..15 in the bar, n counts steps since the start, before that step's hits. It is the clock a timeline counts bars on. optional default null
onHitFunctiononHit(voice, vel, ev), once for every drum hit when it is heard. voice is a name like 'kick', vel is 0..1. optional default null
onTakeoverFunctiononTakeover({ text, bar, offGrid, dropped, varied }): without a rhythm module, a click on a cell turned a line of rhythm into the grid it was playing. offGrid hits moved to the nearest step and dropped drums had no row. optional default null
bpmNumberThe starting tempo, in beats per minute. The slider on the card goes from 60 to 180. optional default 112
patternString | ObjectA preset name from PATTERNS or your own grid { kick: 'x...x...', … }. Not a line of rhythm: to start with one, call drums.edit(line) or drums.setPattern(compile(line), { text: line }). optional default 'four on floor'
clock'frame' | 'lookahead''frame': the machine moves in update(dt) and each hit sounds in the frame it is due. 'lookahead': hits are handed to the kit ahead of time on its audio clock, so the beat is steady under a heavy frame and in a hidden tab ('audio' means the same). See DrumTransport. optional default 'frame'
lang'en' | 'zh''zh' names the lanes and the buttons in Chinese. Anything else is English. optional default 'en'
heartbeat'auto' | 'interval' | falseOnly for clock: 'lookahead'. Who wakes the scheduler: 'auto' a Web Worker timer (falls back to setInterval where a worker cannot start), 'interval' always setInterval, false nobody (you call drums.update() often enough). optional default 'auto'
latency'auto' | NumberOnly for clock: 'lookahead'. How long after a hit is scheduled it is heard, in seconds. 'auto' asks the browser for its output latency; a number sets it by hand (a Bluetooth speaker is 0.15 to 0.25). optional default 'auto'
nowFunctionOnly for clock: 'lookahead'. A function that returns the time in seconds. The default is performance.now() / 1000. Replace it to run the machine on a fake clock in a test. optional
hiddenFunctionOnly for clock: 'lookahead'. A function that returns true while nobody can see the page. The default looks at document.visibilityState. optional
rhythmModuleThe @openav/pattern module (import * as rhythm from '@openav/pattern', anything with compile and fromGrid). With it the line of rhythm over the grid is always there and editable, and the cells and the line rewrite each other. It can also be handed over later with drums.useRhythm(mod). optional default null
onRhythmFunctiononRhythm(line, { how, prev }), once for every new line. how is 'code', 'cell', 'type', 'undo' or 'menu'. optional default null
onLossyFunctiononLossy({ text, bar, lost }): a click on a cell would lose something the 16 steps cannot hold (triplets, < >, ?, bd:2). lost is the same as in toGrid(). Left out, a note appears on the card. The next click rewrites the line anyway. optional default null
onEditErrorFunctiononEditError(error, text) when what was typed is not a line of rhythm (the mistake is marked in red and the last line that worked keeps playing), and onEditError(null) once it is fixed. optional default null
copyTextFunctioncopyText(line, bpm) returns what the Copy button puts on the clipboard. The default is the line itself. optional default null
strudelCommentStringA comment to put at the head of the code that Open in Strudel sends. optional default ''

Returns

Object { seq, engine, root, clock, play(on), setPattern(p, { text }), setBpm(bpm), update(dt), deliver(), phase(), now(), info(), dispose(), useRhythm(mod), src(), strudelUrl(), history, undo(), edit(text), copy(), copyText(), barEvents(bar), voices() } (and the getters rhythm and copied).

Notes

  • With clock: 'frame' the first bar of a compiled line is played on the 16-step grid of four lanes: hits between steps move to the nearest step, and drums with no lane (crash, ride, toms, open hat) are left out. The console says so. clock: 'lookahead' plays a line exactly as written.
  • In a p5 sketch you do not call it: startDrums() builds this card for you, with the lookahead clock, and calls update() every frame.

Examples

js
import { mountDrums } from '@openav/drums';

const drums = mountDrums(document.getElementById('app'), { bpm: 120, pattern: 'breakbeat' });
drums.play(true);                          // start, like ticking the checkbox
let last = performance.now();
(function frame(now) {
  drums.update((now - last) / 1000);       // the machine only moves when you call this
  last = now;
  requestAnimationFrame(frame);
})(last);
module A drum machine card appears and plays a breakbeat; the yellow column runs along the grid. Click the page once to hear the kit; click the cells to change the beat.
js
import * as rhythm from '@openav/pattern';
import { mountDrums } from '@openav/drums';

const told = [];
const drums = mountDrums(document.getElementById('app'), {
  rhythm, bpm: 110, clock: 'lookahead',
  onRhythm: (line, { how }) => told.push(how + ': ' + line),
  onEditError: (error) => told.push(error ? 'error ' + error.code : 'fixed'),
});
console.log(drums.src());                         // bd*4, [~ sd]*2, hh*8
console.log(drums.edit('bd(3,8), cr ~ ~ ~').ok);  // true: the crash gets a row of its own
console.log(drums.edit('bd [sd').ok);              // false: the last line that worked keeps playing
console.log(drums.src());                          // bd(3,8), cr ~ ~ ~
console.log(drums.undo(), drums.src());            // true bd*4, [~ sd]*2, hh*8
console.log(told.join(' | '));
// code: bd*4, [~ sd]*2, hh*8 | type: bd(3,8), cr ~ ~ ~ | fixed | error unclosed | undo: bd*4, [~ sd]*2, hh*8 | fixed
let last = performance.now();
(function frame(now) { drums.update((now - last) / 1000); last = now; requestAnimationFrame(frame); })(last);
module With the pattern module the line is shown over the grid. The code types two lines for you, one of them wrong, then goes back; type into the line yourself and the cells follow.
js
import { mountDrums } from '@openav/drums';
import { Signals } from '@openav/core';

const signals = new Signals();
const g = document.getElementById('c').getContext('2d');
const drums = mountDrums(document.getElementById('app'), { signals, clock: 'lookahead', bpm: 100 });
drums.play(true);

let last = performance.now();
(function frame(now) {
  drums.update((now - last) / 1000); last = now;
  g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
  g.fillStyle = '#ff7b54';
  g.beginPath();
  g.arc(200, 150, 30 + signals.get('drum/kick/env') * 90, 0, Math.PI * 2);   // 0..1, decays after each hit
  g.fill();
  requestAnimationFrame(frame);
})(last);
module A circle on the canvas swells with every kick: the drum machine publishes drum/kick/env, and the sketch only reads the signal.

To be confirmed

The figure of 0.15 to 0.25 seconds for a Bluetooth speaker (the latency option) is from the handbook (docs/p5.md); it was not measured here.

Related

  • DrumTransport The runtime around a DrumScheduler: it hands every hit to the kit ahead of time on its audio clock, and tells the page each step and hit when it is heard.
  • DrumSequencer A 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
  • drumEngine() A drum kit made of Web Audio synthesis, with no sample files: General MIDI drum notes go in, drum sounds come out.
  • compile() Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • drumHit() An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).

import { mountDrums } from '@openav/drums'; packages/drums/index.js, line 830

PATTERNS#

constant@openav/drums
PATTERNS = object, 5 keys

The five preset grids of the drum machine: 'four on floor', breakbeat, 'half time', latin and sparse.

The preset names are what startDrums(), DrumSequencer and mountDrums() accept as a pattern. A preset is just data, so you can start from one: { ...PATTERNS.latin, clap: 'x...x...x...x...' } is latin with a clap on every beat.

The five presets as text, one lane per column:

text
                  kick              snare             hat               clap
four on floor     1000100010001000  0000100000001000  1010101010101010  0000000000000000
breakbeat         1000000110100000  0000100000001001  1010101010101010  0000100000000000
half time         1000000000001000  0000000010000000  1010101010101010  0000000010000000
latin             1000001000100000  0010010000100100  1011101110111011  0000000000000000
sparse            1000000000100000  0000000000000000  0010001000100010  0000000010000000
Type
Object

Five presets, each { kick, snare, hat, clap } as 16-character strings of 1 and 0.

Examples

js
import { PATTERNS, DrumSequencer } from '@openav/drums';

console.log(Object.keys(PATTERNS).join(' | '));   // four on floor | breakbeat | half time | latin | sparse
console.log(PATTERNS['half time'].snare);          // 0000000010000000

const seq = new DrumSequencer({ onHit() {} });
seq.setPattern({ ...PATTERNS.latin, clap: 'x...x...x...x...' });   // latin plus a clap on every beat
console.log(seq.pattern, seq.grid.clap.join(''));  // custom 1000100010001000
module Lists the presets, reads one lane, and builds a variation of latin by spreading it into a new grid.

Related

  • laneSteps() Turns a lane written as text, like 'x...x...x...x...', or as a list into 16 steps of 0 and 1.
  • DrumSequencer A 16-step, four-lane drum pattern clock that you move forward with update(dt) every frame and that calls your function for each hit.
  • DrumScheduler Pure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
  • mountDrums() Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

import { PATTERNS } from '@openav/drums'; packages/drums/index.js, line 173

voiceForNote()#

function@openav/drums
voiceForNote(note)

Tells which drum of the kit a General MIDI drum note plays, or null for a note it does not play.

Pads and drum machines send notes on channel 10, and several notes map to the same drum: 35 and 36 are kicks, 38 and 40 snares, 41 and 43 low toms, 45 and 47 toms, 48 and 50 high toms, 49, 52, 55 and 57 crashes, 51, 53 and 59 rides. A note that is not in the table, such as a melodic 60, gives null: the kit stays silent instead of guessing, because a wrong cymbal on a kick pad is worse than nothing.

Parameters

NameTypeDescription
noteNumberA MIDI note number.

Returns

String | null The drum name ('kick', 'crash'…), or null.

Examples

js
import { voiceForNote } from '@openav/drums';

console.log(voiceForNote(36), voiceForNote(35));   // kick kick
console.log(voiceForNote(49), voiceForNote(57));   // crash crash
console.log(voiceForNote(60));                      // null, a melodic note
module Logs which drum a few notes play: two notes for one drum, and a melodic note that plays none.

Related

  • GM The General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.
  • drumEngine() A drum kit made of Web Audio synthesis, with no sample files: General MIDI drum notes go in, drum sounds come out.
  • WORDS The drum words a line of rhythm may use and the drum each one plays: bd is kick, sd is snare, hh is hat.

import { voiceForNote } from '@openav/drums'; packages/drums/index.js, line 34

@openav/pattern

compile()#

function@openav/pattern
compile(src, { words = WORDS } = {})

Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.

compile() is the one function you need to turn a line of rhythm into hits. compile('bd sd hh').eventsInBar(1) answers "what happens in bar 1?" with a list of hits, sorted by pos. Bars count from 1, and one bar is one cycle of the line: the words share the bar equally, so bd sd hh puts a kick at 0, a snare at 1/3 and a hat at 2/3.

Each hit from eventsInBar(bar, { seed }) is { word, voice, n, pos, dur, vel, layer, token }. word is what was written, voice the drum it plays ('kick'), pos where in the bar, 0 up to (not including) 1. dur is how long it lasts, in bars, and may run past the bar (bd/2 lasts two). vel is always 1. layer is the index of the comma part it came from. token is { start, end }, the character range of the word in src, so you can light the word while it sounds. n is the sample number from bd:2, or null. n is kept for you but does not change the sound.

It never throws. A line it cannot read comes back as { ok: false, error }, and eventsInBar() returns [] for every bar, so check ok before you trust the hits. error.index and error.length mark the characters that are wrong, error.message and error.hint are English sentences, and error.code is one of empty, unclosed, unexpected, bad-number, bad-euclid, unknown-word or unsupported. An unknown-word error also carries suggestion, the closest word or null. The words and the symbols you can write are listed below.

text
bd sd hh      steps share the bar equally
~  or  -       a rest
[bd sd]        a group takes one step and splits it again
bd*4           faster: 4 times in its step (decimals work: bd*1.5)
bd/2           slower: once every 2 bars
<bd sd>        one per bar, in turn
bd, hh*8       a comma stacks layers
bd(3,8)        Euclidean: 3 hits spread over 8 steps
bd(3,8,2)      the same, starting 2 steps later
bd!2  bd !     repeat the step (each copy takes a step)
bd@3  bd _ _   make the step 3 times as long
hh*8?  hh*8?0.3   drop each hit by chance (1/2, or 0.3)
bd:2           sample number, kept in n

Modifiers chain in the order written (bd(3,8)*2, hh*8?). Word case does not matter (BD is bd). ? is decided by seed and the time of the hit, so the same seed and bar always give the same hits, while other bars and other seeds differ. Not supported, and refused with an unsupported error and a hint: . (Strudel's grouping), |, { }, %, .. and ^. A dot matters most: lane strings like 'x...x...' are not a line of rhythm, and a rest is ~.

Parameters

NameTypeDescription
srcStringThe line of rhythm, for example 'bd sd hh'. Anything that is not a string is turned into one (null and undefined become an empty line, which is an error).
wordsObjectWhich words mean which drum: { word: voice }. The default is WORDS. To add a word, spread the default: { ...WORDS, boom: 'kick' }. The voice should be one the drum kit plays, see GM. optional default WORDS

Returns

Object { ok, error, src, voices, layers, period, eventsInBar(bar, { seed }) }. ok is true when the line was read. error is null, or { code, index, length, message, hint }. voices lists the drums the line uses, in the order they first appear. layers counts the comma parts. period is the number of bars until the written rhythm repeats.

Notes

  • Time inside is exact fractions, and each bar is computed on its own. Bar 1 000 000 of [bd sd sn]*3 lands on the same ninths as bar 1, and asking for bar 500 is as cheap as asking for bar 2.
  • From a p5 sketch you do not call it: startDrums() takes the line and compiles it for you. Use compile() yourself when you want the hits as data, for a score, a drawing or a test.
  • The way of writing comes from TidalCycles and Strudel. This is an independent implementation written from their public documentation. Where it differs from strudel.cc on purpose (a rotation of n steps or more wraps around, *0 and (9,8) are errors, unknown names are errors), the package README says so.

Examples

js
import { compile } from '@openav/pattern';

const p = compile('bd sd hh');
for (const e of p.eventsInBar(1)) console.log(e.word, e.voice, e.pos.toFixed(2));
// bd kick 0.00
// sd snare 0.33
// hh hat 0.67
module Logs the three hits of bar 1: the word, the drum it plays, and where in the bar. Open the console to see them.
js
import { compile } from '@openav/pattern';

const p = compile('<bd sd>, hh*4');
console.log(p.period);                                   // 2
for (const bar of [1, 2, 3]) {
  const words = p.eventsInBar(bar).filter((e) => e.layer === 0).map((e) => e.word);
  console.log('bar', bar, words.join(' '));
}
// bar 1 bd
// bar 2 sd
// bar 3 bd

const bad = compile('bd [sd');
console.log(bad.ok, bad.error.code, bad.error.index);   // false unclosed 3
console.log(bad.eventsInBar(1));                         // []
module A line that changes every bar: <bd sd> alternates, so period is 2. A line with a mistake comes back with ok: false, the error code and the character it starts at.
js
import { compile } from '@openav/pattern';

const p = compile('bd(3,8), [~ sd]*2, hh*8');
const g = document.getElementById('c').getContext('2d');
g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
const colors = ['#ff7b54', '#ffd166', '#6f9bff'];
for (const e of p.eventsInBar(1)) {
  g.fillStyle = colors[e.layer];
  g.fillRect(20 + e.pos * 360, 40 + e.layer * 70, e.dur * 360 - 4, 40);   // dur is in bars
}
module Draws bar 1 on the canvas, one row per comma part: each hit is a block at its position, as long as it lasts.

Related

  • parse() Reads a line of rhythm into a tree of nodes, each with its character range. It does not look words up in the drum kit.
  • WORDS The drum words a line of rhythm may use and the drum each one plays: bd is kick, sd is snare, hh is hat.
  • euclid() Spreads a number of hits as evenly as possible over a number of steps, the Euclidean rhythm. euclid(3, 8) is the tresillo.
  • toGrid() Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.
  • DrumScheduler Pure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

import { compile } from '@openav/pattern'; packages/pattern/index.js, line 498

euclid()#

function@openav/pattern
euclid(pulses, steps, rotate = 0)

Spreads a number of hits as evenly as possible over a number of steps, the Euclidean rhythm. euclid(3, 8) is the tresillo.

It is the function behind bd(3,8) in a line of rhythm: compile() calls it for every (k,n). Use it on its own when you want the pattern as data, for a drawing or your own sequencer. euclid(3, 8) is x..x..x., and euclid(5, 8) is x.xx.xx..

It follows Bjorklund's procedure as Toussaint (2005) describes it. rotate moves the pattern later, the direction strudel.cc plays: euclid(3, 8, 2) is x.x..x.., the hits of euclid(3, 8) moved two steps to the right.

Parameters

NameTypeDescription
pulsesNumberHow many hits. A negative number plays the gaps instead: euclid(-3, 8) is the five steps euclid(3, 8) leaves out. More hits than steps fills every step. The number is cut to a whole number.
stepsNumberHow many steps. Fewer than 1 gives an empty array.
rotateNumberStart the pattern this many steps later. Negative starts earlier, and a number of steps or more wraps around. optional default 0

Returns

Boolean[] An array of steps booleans, true where there is a hit.

Examples

js
import { euclid } from '@openav/pattern';

const show = (bits) => bits.map((b) => (b ? 'x' : '.')).join('');
console.log(show(euclid(3, 8)));      // x..x..x.
console.log(show(euclid(5, 8)));      // x.xx.xx.
console.log(show(euclid(3, 8, 2)));   // x.x..x..
module Logs three rhythms as text: x is a hit and . is a rest. The third is the first one moved two steps later.
js
import { euclid } from '@openav/pattern';

const g = document.getElementById('c').getContext('2d');
g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
g.fillStyle = '#ffd166';
euclid(5, 16).forEach((hit, i) => {          // 5 hits on 16 steps
  g.globalAlpha = hit ? 1 : 0.2;
  g.fillRect(12 + i * 23, 120, 18, 60);
});
console.log(euclid(5, 16).filter(Boolean).length);   // 5
module Draws sixteen blocks and lights five of them, spread as evenly as sixteen steps allow.

Related

  • compile() Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
  • toGrid() Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.
  • fromGrid() Turns a grid of steps, like the drum machine card, into the one line of rhythm a person would write for it.

import { euclid } from '@openav/pattern'; packages/pattern/index.js, line 69

fromGrid()#

function@openav/pattern
fromGrid(grid, { steps = 16, words = null, order = null } = {})

Turns a grid of steps, like the drum machine card, into the one line of rhythm a person would write for it.

It is what happens when you click a cell of the drum machine card: the grid has to come back as text. Each lane becomes one layer, joined with , , and it picks the shortest honest spelling. Steady hits become bd*4 or hh*8, one hit per group [~ sd]*2, a Euclidean rhythm bd(3,8), and anything else is written beat by beat, like bd [~ ~ ~ bd] bd*2 ~. It never throws.

The line is exact: compile() of the result lands on the same steps, in every bar. The rules are in the package README, and the tests check a thousand random grids.

Parameters

NameTypeDescription
gridObjectThe steps: { lane: 'x...x...' } or { lane: [1, 0, 0, 0, …] }. A lane is named by its drum (kick) or its word (bd). x, X, 1 and true are hits, | and spaces are ignored. A lane with no name the kit knows is left out.
stepsNumberHow many steps each lane has. optional default 16
wordsObjectKeep your own spelling: { kick: 'kick' } writes kick*4 instead of bd*4. optional default null
orderString[]Drums to put first, in this order, for example ['kick', 'snare']. The rest follow in the order of the grid's keys. optional default null

Returns

String One line of rhythm that plays exactly the grid. A grid with no hit anywhere gives '~'.

Examples

js
import { fromGrid } from '@openav/pattern';

console.log(fromGrid({
  kick: 'x...x...x...x...',
  snare: '....x.......x...',
  hat: 'x.x.x.x.x.x.x.x.',
}));
// bd*4, [~ sd]*2, hh*8
module Logs the line for a classic four-on-the-floor grid: a kick on every beat, a snare on 2 and 4, eight hats.
js
import { fromGrid, compile } from '@openav/pattern';

const grid = { kick: [1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0], hat: 'x.x.x.x.x.x.x.x.' };
const line = fromGrid(grid, { order: ['kick'] });
console.log(line);                          // bd(3,8), hh*8
console.log(compile(line).ok);              // true
console.log(fromGrid(grid, { words: { kick: 'kick' } }));   // kick(3,8), hh*8
console.log(fromGrid({}));                  // ~
module A lane can be an array of 0 and 1. The hits at steps 0, 6 and 12 are recognised as a Euclidean rhythm. words keeps your own spelling, and an empty grid is one rest.

Related

  • toGrid() Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.
  • compile() Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
  • euclid() Spreads a number of hits as evenly as possible over a number of steps, the Euclidean rhythm. euclid(3, 8) is the tresillo.
  • toStrudel() Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.

import { fromGrid } from '@openav/pattern'; packages/pattern/index.js, line 628

fromStrudel()#

function@openav/pattern
fromStrudel(code)

Reads code copied from strudel.cc, like s("bd*4").fast(2), into the line of rhythm OAV plays, plus the tempo it found.

It lets someone paste what they made in Strudel. It reads s("…") and sound("…") with ", ' or backtick quotes. Several of them (also with a label like $: or name: in front) become layers. A .fast(n) or .slow(n) is folded into the line as […]*n or […]/n. setcpm(x) gives bpm as x times 4, setcps(x) as x times 240. .bank(…) goes to ignored, because OAV plays its own kit. Any other call or statement goes to unsupported, and the rest still plays. It never throws.

It only reshapes text: it does not check the words. fromStrudel('s("bd kik")') gives ok: true with src: 'bd kik', so pass src to compile() and look at its ok. startDrums() accepts Strudel code directly and does all of this for you.

Parameters

NameTypeDescription
codeStringThe Strudel code, as copied from the strudel.cc editor.

Returns

Object { strudel, ok, src, bpm, ignored, unsupported }. strudel is false when the text has no s(…) or sound(…) in it. ok is true when there is a line of rhythm in src. bpm is the tempo from setcpm() or setcps(), else null. ignored lists what was left out on purpose (.bank(…)), unsupported what OAV does not do (.jux(rev), .gain(…)).

Examples

js
import { fromStrudel, compile } from '@openav/pattern';

const r = fromStrudel('setcpm(110/4)\ns("bd*4, [~ sd]*2").bank("RolandTR909").fast(2).jux(rev)');
console.log(r.src);            // [bd*4, [~ sd]*2]*2
console.log(r.bpm);            // 110
console.log(r.ignored);        // ['.bank("RolandTR909")']
console.log(r.unsupported);    // ['.jux(rev)']
console.log(compile(r.src).ok); // true
module Reads a typical Strudel snippet: .fast(2) is folded into the line, the tempo is found, .bank() is ignored and .jux() is reported as unsupported.
js
import { fromStrudel } from '@openav/pattern';

console.log(fromStrudel('note("c e g")').strudel);   // false, no s("…") in it
console.log(fromStrudel('s("bd kik")').ok);          // true: the words are not checked
console.log(fromStrudel('setcps(0.5)\ns("bd")').bpm); // 120
module The edges: text with no s(…) is not Strudel for OAV, a misspelled word still passes, and setcps(0.5) is 120 bpm.

Related

  • toStrudel() Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.
  • strudelUrl() Makes a strudel.cc link that opens the line of rhythm in the Strudel editor.
  • compile() Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

import { fromStrudel } from '@openav/pattern'; packages/pattern/index.js, line 727

parse()#

function@openav/pattern
parse(src)

Reads a line of rhythm into a tree of nodes, each with its character range. It does not look words up in the drum kit.

parse() is the first half of compile(): it checks the grammar and builds the tree, and stops there. Use it when you want to look at the structure of a line, for example to colour its words or find a mistake. For the hits, call compile().

The root is always a layer whose layers are the top-level comma parts, and each of those is a seq. The node types are step { word, n }, rest, seq { steps: [{ node, weight }] }, layer { layers }, alt { steps } (one per bar), fast and slow { factor, node }, euclid { pulses, slots, rotate, node } and maybe { chance, id, node }. Every node has start and end, its character range in src. Weights and factors are fractions [numerator, denominator], so bd*1.5 has the factor [3, 2].

Parameters

NameTypeDescription
srcStringThe line of rhythm, for example 'bd [sd sd]'.

Returns

Object { ok: true, tree, error: null }, or { ok: false, tree: null, error } with the same { code, index, length, message, hint } error as compile().

Notes

  • Because parse() never looks at the drum kit, parse('kik') succeeds while compile('kik') fails with unknown-word. Everything else (an unclosed bracket, *0, (9,8)) fails in both.

Examples

js
import { parse } from '@openav/pattern';

const r = parse('bd [sd hh]*2');
console.log(r.ok);                                          // true
console.log(r.tree.layers[0].steps.map((s) => s.node.type)); // ['step', 'fast']
console.log(parse('bd [sd').error.code);                    // unclosed
module Logs the type of each step of the first layer (a plain step, then a group made faster), and the error code of a line with an unclosed bracket.
js
import { parse } from '@openav/pattern';

// light every word of the line: each step node knows its characters
const src = 'bd*4, [~ sd]*2, hh*8';
const words = [];
const walk = (node) => {
  if (node.type === 'step') words.push(src.slice(node.start, node.end) + '@' + node.start);
  for (const s of node.steps || []) walk(s.node);
  for (const l of node.layers || []) walk(l);
  if (node.node) walk(node.node);
};
walk(parse(src).tree);
console.log(words.join(' '));   // bd@0 sd@9 hh@16
module Walks the tree and logs each word with the character it starts at, the data you need to colour words in an editor.

Related

  • compile() Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
  • toGrid() Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.
  • toStrudel() Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.

import { parse } from '@openav/pattern'; packages/pattern/index.js, line 123

SHORT#

constant@openav/pattern
SHORT = object, 13 keys

The short word Tidal and Strudel write for each drum: kick is bd, snare is sd. The other direction from WORDS.

fromGrid() writes with these words when you do not give it your own spelling, and toStrudel() uses them (with hh for pedalhat) to speak Strudel.

Type
Object

A frozen object of 13 { voice: word } pairs, one per drum of the kit. pedalhat has no short word, so it maps to 'pedalhat', OAV's own.

Examples

js
import { SHORT } from '@openav/pattern';

console.log(SHORT.kick, SHORT.snare, SHORT.hat, SHORT.hightom);   // bd sd hh ht
console.log(SHORT.pedalhat);                                       // pedalhat
module Logs the short word for a few drums, and the one drum that has none.

Related

  • WORDS The drum words a line of rhythm may use and the drum each one plays: bd is kick, sd is snare, hh is hat.
  • fromGrid() Turns a grid of steps, like the drum machine card, into the one line of rhythm a person would write for it.
  • toStrudel() Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.

import { SHORT } from '@openav/pattern'; packages/pattern/index.js, line 548

strudelUrl()#

function@openav/pattern
strudelUrl(src, opts = {})

Makes a strudel.cc link that opens the line of rhythm in the Strudel editor.

The format is the one strudel.cc shows in its own address bar. It is what the drum machine card's Open in Strudel link uses. Open it in a new tab: a strudel.cc tab that is already open does not read a changed #.

Parameters

NameTypeDescription
srcStringThe line of rhythm.
optsObjectThe options of toStrudel(): { bpm, comment, words }. optional default {}

Returns

String A URL: 'https://strudel.cc/#' and the code from toStrudel(), as UTF-8 base64, URL-encoded.

Notes

  • OAV only builds the link. No Strudel code runs in, or ships with, OAV.

Examples

js
import { strudelUrl } from '@openav/pattern';

const url = strudelUrl('bd*4, [~ sd]*2, hh*8', { bpm: 110 });
console.log(url.startsWith('https://strudel.cc/#'));            // true
const code = atob(decodeURIComponent(url.split('#')[1]));
console.log(code);                                              // setcpm(110/4)
                                                                // s("bd*4, [~ sd]*2, hh*8")

const a = document.getElementById('app').appendChild(document.createElement('a'));
a.href = url; a.target = '_blank'; a.textContent = 'Open in Strudel';
module Builds the link, decodes it again to show the code inside, and puts it on the page as a link that opens in a new tab.

To be confirmed

Not opened in strudel.cc from this build: the link format was read off strudel.cc's address bar on 2026-10-15 (README), and a strudel.cc change could break it.

Related

  • toStrudel() Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.
  • fromStrudel() Reads code copied from strudel.cc, like s("bd*4").fast(2), into the line of rhythm OAV plays, plus the tempo it found.
  • mountDrums() Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.

import { strudelUrl } from '@openav/pattern'; packages/pattern/index.js, line 872

toGrid()#

function@openav/pattern
toGrid(p, { bar = 1, steps = 16, seed = 0, bars = 16, words = WORDS } = {})

Shows one bar of a line of rhythm on a grid of steps, and tells you what the grid cannot hold.

It is the way back from fromGrid(), and what the drum machine card uses to draw the cells of a line. Each hit goes to the step it is nearest to. The lost fields say what that cost: offGrid is the number of hits that sit between steps (triplets) and were moved to the nearest one. stacked counts two hits of one drum on one step. varies is true when other bars differ (< >, /2). random is true when the line has a ?. samples is true when it has a sample number like bd:2. exact is true only when none of these happened.

A line that fails to compile does not throw: you get ok: false.

Parameters

NameTypeDescription
pString | ObjectA line of rhythm, or the result of compile().
barNumberWhich bar to show, counting from 1. optional default 1
stepsNumberHow many steps the grid has. optional default 16
seedNumberDecides each ?, the same way as in eventsInBar(). optional default 0
barsNumberHow many bars to look through to find out whether the other bars differ (lost.varies). At most the line's period, and never more than 64. optional default 16
wordsObjectWhich words mean which drum, the same as in compile(). optional default WORDS

Returns

Object { ok, grid, words, voices, exact, lost }. grid is { voice: [0 or 1 × steps] }. words is the word the line used for each drum. voices lists the drums. exact is true when the grid holds the bar completely. lost counts what it cannot: { offGrid, stacked, varies, random, samples }. If the line cannot be read, ok is false and the rest is empty.

Examples

js
import { toGrid } from '@openav/pattern';

const g = toGrid('bd*4, [~ sd]*2, hh*8');
console.log(g.exact);                        // true
console.log(g.grid.snare.join(''));          // 0000100000001000
console.log(g.words);                        // { kick: 'bd', snare: 'sd', hat: 'hh' }
module A line that fits the sixteen steps: exact is true, and each drum has its row of 0 and 1.
js
import { toGrid } from '@openav/pattern';

console.log(toGrid('bd [sd sd sd]').lost.offGrid);      // 2, the triplet sits between steps
console.log(toGrid('<bd sd>', { bar: 2 }).lost.varies); // true
console.log(toGrid('hh*8?').lost.random);               // true
console.log(toGrid('bd:2').exact);                      // false, the sample number is lost
console.log(toGrid('bd . sd').ok);                      // false, not a line of rhythm
module Five lines and what each costs on a grid: triplets move, alternating bars and chance cannot be shown, and a line that does not compile has no grid at all.

Related

  • fromGrid() Turns a grid of steps, like the drum machine card, into the one line of rhythm a person would write for it.
  • compile() Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
  • DrumScheduler Pure drum logic with no timers and no audio: a grid or a line of rhythm goes in, timed events come out, ahead of time, on any clock you pass.
  • mountDrums() Puts a drum machine on the page: a step grid, a transport and a line of rhythm you can type into. It publishes every hit as drum/* signals.

import { toGrid } from '@openav/pattern'; packages/pattern/index.js, line 656

toStrudel()#

function@openav/pattern
toStrudel(src, { bpm = null, comment = '', words = WORDS } = {})

Turns a line of rhythm into Strudel code: OAV's drum words become Strudel's, with a tempo line and a comment if you give them.

The words become the names Strudel's drum sounds use: kick becomes bd, pedalhat becomes hh. A rotation of n steps or more in a Euclidean part, like bd(3,8,10), is written as the rotation OAV plays (bd(3,8,2)), because strudel.cc would play it unrotated. A line that does not read is passed through unchanged, so the result may not play in Strudel: check it with compile() first. It never throws.

To open the result in the Strudel editor, use strudelUrl().

Parameters

NameTypeDescription
srcStringThe line of rhythm.
bpmNumberThe tempo. When it is above 0 a setcpm(bpm/4) line is written, because one OAV bar is one Strudel cycle of four beats. optional default null
commentStringA line to put first as a // comment. optional default ''
wordsObjectWhich words mean which drum, the same as in compile(). optional default WORDS

Returns

String The Strudel code, one to three lines, ending in s("…").

Examples

js
import { toStrudel } from '@openav/pattern';

console.log(toStrudel('kick*4, pedalhat*8', { bpm: 110, comment: 'from OAV' }));
// // from OAV
// setcpm(110/4)
// s("bd*4, hh*8")
module Logs three lines of Strudel code: the comment, the tempo, and the pattern with Strudel's names for the drums.
js
import { toStrudel } from '@openav/pattern';

console.log(toStrudel('bd(3,8,10)'));   // s("bd(3,8,2)")
console.log(toStrudel('bd . sd'));      // s("bd . sd"), a line that does not read passes through
module A rotation past the number of steps is written the way OAV plays it, and a line with a mistake is not touched.

Related

  • strudelUrl() Makes a strudel.cc link that opens the line of rhythm in the Strudel editor.
  • fromStrudel() Reads code copied from strudel.cc, like s("bd*4").fast(2), into the line of rhythm OAV plays, plus the tempo it found.
  • compile() Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.

import { toStrudel } from '@openav/pattern'; packages/pattern/index.js, line 832

WORDS#

constant@openav/pattern
WORDS = object, 26 keys

The drum words a line of rhythm may use and the drum each one plays: bd is kick, sd is snare, hh is hat.

The short words are the ones TidalCycles and Strudel use: bd kick, sd and sn snare, hh hat, oh open hat, cp clap, rim and rs rim, lt low tom, mt tom, ht high tom, cr crash, rd ride, cb cowbell. Each drum's own name works too: kick, snare, hat, openhat, pedalhat, clap, lowtom, tom, hightom, crash, ride, cowbell. The values are the drum names of GM.

It is the default of the words option of compile(), toGrid() and toStrudel(). To teach a line a new word, pass a copy with your word added.

Type
Object

A frozen object of 26 { word: voice } pairs. Keys are lowercase, and compile() ignores the case of what you write.

Examples

js
import { WORDS, compile } from '@openav/pattern';

console.log(WORDS.bd, WORDS.sn, WORDS.rs, WORDS.mt);   // kick snare rim tom
console.log(Object.keys(WORDS).length);                 // 26

const mine = compile('boom tss boom tss', { words: { ...WORDS, boom: 'kick', tss: 'hat' } });
console.log(mine.ok, mine.voices.join(' '));            // true kick hat
module Looks up a few words, counts them, and teaches a line two words of your own by spreading WORDS into a new object.

Related

  • SHORT The short word Tidal and Strudel write for each drum: kick is bd, snare is sd. The other direction from WORDS.
  • compile() Reads one line of rhythm text, like 'bd*4, [~ sd]*2, hh*8', and gives back the drum hits of any bar you ask for. It never throws.
  • GM The General MIDI drum note each drum of the kit plays: GM.kick is 36, GM.snare is 38.
  • voiceForNote() Tells which drum of the kit a General MIDI drum note plays, or null for a note it does not play.

import { WORDS } from '@openav/pattern'; packages/pattern/index.js, line 31

Timeline, cues and score#

Bars and cues in a sketch; @openav/timeline and @openav/score for a whole show. 16

oav-p5

barStarted()#

eventoav-p5
function barStarted(bar) {}

An event you define: it is called at the first step of every bar of the drum machine, with the bar number.

Define function barStarted(bar) { … } to do something once per bar: function barStarted(bar) { drift += 9; }. It is called in the frame where the first step of the bar is heard, right before draw(), and the first call is barStarted(1) when the machine starts. Within a step, cueFired() comes before it and stepStarted() after.

Parameters

NameTypeDescription
barNumberThe bar that starts, counted from 1.

Notes

  • While the tab is hidden it is still called once per bar, but nothing is told twice when the tab comes back. stepStarted() and the hits are not told in that time.

Examples

jsOpenProcessing ↗
let shade = 200;

function setup() {
  createCanvas(400, 300);
  colorMode(HSB, 360, 100, 100);
  startDrums('four on floor', 150);
}

function barStarted(bar) {
  shade = (shade + 40) % 360;   // a new colour every bar
}

function draw() {
  background(shade, 60, 80);
  fill(0, 0, 100);
  textAlign(CENTER, CENTER);
  textSize(48);
  text(drumBar, width / 2, height / 2);
}
The background changes colour at the start of every bar, while the number counts the bars.

Related

  • stepStarted() An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.
  • cueFired() An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
  • drumBar Which bar the drum machine is in, counted from 1 when it started.
  • drumHit() An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
  • OAVp5.drums.on() Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.

p5/oav-p5.js, line 1490

cue()#

functionoav-p5
cue(bar, name, [every])

Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.

The drum machine counts the bars, so it can run a plan: cue(1, 'calm', 12); cue(5, 'bloom', 12); cue(9, 'storm', 12); in setup() makes three scenes that come round every 12 bars. When a cue is due, cueFired(name, bar) is called and cueName holds the name; read it in draw() if you prefer that to a callback. Several cues on one step fire in the order you declared them, before barStarted(), stepStarted() and the hits of that step.

Calling cue() twice with the same values adds nothing, so it is safe in draw() too. Counting starts with startDrums(), and a cue only fires when the machine passes its place: declare the cues right after startDrums() in setup() so that a cue at bar 1 is not missed. After the machine is stopped and started again, the cues fire again from bar 1.

Parameters

NameTypeDescription
barNumberThe bar, counted from 1 (the first bar). A fraction is a place inside a bar: 4.5 is halfway through bar 4 (the nearest of the 16 steps is used). A value below 1, or one that is not a number, prints a warning in the console and adds nothing.
nameAnyWhat to call it, usually a string. It is passed to cueFired() and becomes cueName.
everyNumberRepeat every this many bars after the first time (fractions work, at least one step). 0 or left out: only once. optional default 0

Returns

undefined Nothing. OAVp5.drums.cue() is the same function and returns the cue it made.

Notes

  • A cue declared after its bar has passed fires at its next repeat if it has an every, otherwise never until the machine is started again. Without a running machine no cue fires at all.

Examples

jsOpenProcessing ↗
let scene = 'calm';

function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 140);
  cue(1, 'calm', 4);    // bars 1, 5, 9 ...
  cue(3, 'storm', 4);   // bars 3, 7, 11 ...
  textAlign(CENTER, CENTER);
  textSize(32);
}

function cueFired(name, bar) {
  scene = name;
}

function draw() {
  background(scene === 'storm' ? color(130, 30, 40) : color(20, 40, 100));
  fill(255);
  text(scene + ' (bar ' + drumBar + ')', width / 2, height / 2);
}
The picture swaps between "calm" (blue) and "storm" (red) every two bars, and the text names the scene and the bar.
jsOpenProcessing ↗
let flash = 0;

function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 130);
  cue(2.5, 'flash', 2);   // halfway through bar 2, then every 2 bars
  textAlign(CENTER, CENTER);
}

function cueFired(name, bar) {
  if (name === 'flash') flash = 1;
}

function draw() {
  flash *= 0.92;
  background(20 + flash * 200);
  fill(255 - flash * 255);
  text('a flash halfway through every second bar (bar ' + drumBar + ')', width / 2, height / 2);
}
The screen flashes in the middle of bar 2 and again in the middle of every second bar after it.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('latin', 120);
  textAlign(CENTER, CENTER);
  textSize(28);
}

function draw() {
  cue(2, 'drop', 4);   // the same call every frame adds nothing
  background(20);
  fill(255);
  text(cueName === '' ? 'waiting for the first cue' : 'last cue: ' + cueName, width / 2, height / 2);
}
cue() inside draw() is safe: the text changes to "last cue: drop" at bar 2 and stays that way.

Related

  • cueFired() An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
  • cueName The name of the last cue that fired, or an empty string before the first one.
  • barStarted() An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
  • drumBar Which bar the drum machine is in, counted from 1 when it started.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • OAVp5.drums.cue() The same as cue(bar, name, every), but it returns the cue it made (or the one that was already there), and null when bar is not valid.

p5/oav-p5.js, line 1733

cueFired()#

eventoav-p5
function cueFired(name, bar) {}

An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.

Define function cueFired(name, bar) { … } and it is called at the step where a cue is due, before barStarted(), stepStarted() and the hits of that step. A good place to change a scene: function cueFired(name) { look = SCENES[name]; }. If you only need the latest name, read cueName in draw() instead.

Parameters

NameTypeDescription
nameAnyThe name you gave to cue().
barNumberThe bar the cue fired in, counted from 1. For a cue in the middle of a bar (4.5) it is the whole bar it falls in (4).

Notes

  • Cues keep firing while the tab is hidden, so the scene is right when you come back. Several cues on one step call it once each, in the order they were declared.

Examples

jsOpenProcessing ↗
let scene = '-';
let when = 0;

function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 150);
  cue(1, 'intro');
  cue(3, 'verse', 4);
  cue(5, 'chorus', 4);
  textAlign(CENTER, CENTER);
  textSize(32);
}

function cueFired(name, bar) {
  scene = name + ' (bar ' + bar + ')';
  when = millis();
}

function draw() {
  background(20 + max(0, 1 - (millis() - when) / 400) * 120);   // a flash on every cue
  fill(255);
  text(scene, width / 2, height / 2);
}
The screen flashes and names the scene each time a cue fires: intro once, then verse and chorus take turns.

Related

  • cue() Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
  • cueName The name of the last cue that fired, or an empty string before the first one.
  • barStarted() An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
  • stepStarted() An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.
  • OAVp5.drums.on() Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.

p5/oav-p5.js, line 1487

cueName#

variableoav-p5
cueName

Also: OAVp5.drums.lastCue

The name of the last cue that fired, or an empty string before the first one.

Read it in draw() instead of waiting for cueFired(): look = SCENES[cueName];. It is the name of the last cue only, so it is the right thing for "which scene am I in", and cueFired() is the right thing for "something just began". Cues are declared with cue().

Type
String
Range
'' or a cue name

Changes when a cue fires, and keeps the name until the next one. It is '' at the start.

Notes

  • A sketch that declares its own variable called cueName keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.drums.lastCue instead.

Examples

jsOpenProcessing ↗
const SCENES = { calm: '#3b5bdb', bloom: '#e64980', storm: '#495057' };

function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 150);
  cue(1, 'calm', 6);
  cue(3, 'bloom', 6);
  cue(5, 'storm', 6);
  textAlign(CENTER, CENTER);
  textSize(32);
}

function draw() {
  background(SCENES[cueName] || '#111');   // nothing yet: dark
  fill(255);
  text(cueName, width / 2, height / 2);
}
The background and the word follow the last cue: blue "calm", pink "bloom", grey "storm", every six bars.

Related

  • cue() Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
  • cueFired() An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
  • drumBar Which bar the drum machine is in, counted from 1 when it started.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 317

drumBar#

variableoav-p5
drumBar

Also: OAVp5.drums.bar

Which bar the drum machine is in, counted from 1 when it started.

A bar is 16 steps, four beats. Use it to give a piece a shape: floor((drumBar - 1) / 4) % 3 is the section number when you change every four bars. To make something happen at an exact bar, cue() is neater than testing drumBar in draw(). barStarted() is the function that is called at the start of each bar.

Type
Number
Range
0, 1, 2, 3 …

0 before the machine has played its first step, then 1, 2, 3 … with no upper limit. It keeps its last value when the machine stops.

Before it has a value

Start the machine with startDrums(). Stopping and starting again counts from bar 1 again.

Notes

  • A sketch that declares its own variable called drumBar keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.drums.bar instead.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 150);
  textAlign(CENTER, CENTER);
  textSize(64);
}

function draw() {
  background(20);
  fill(255);
  text(drumBar, width / 2, height / 2);
}
A big number that goes up by one at the start of every bar.
jsOpenProcessing ↗
const SECTIONS = ['#3b5bdb', '#e8590c', '#2f9e44'];

function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 160);
  noStroke();
}

function draw() {
  const section = floor((max(drumBar, 1) - 1) / 2) % SECTIONS.length;   // a new section every 2 bars
  background(SECTIONS[section]);
  fill(255);
  text('bar ' + drumBar + ', section ' + (section + 1), 20, 30);
}
The background changes colour every two bars and the text names the bar and the section.

Related

  • barStarted() An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
  • drumStep Which of the 16 steps of the bar the drum machine just played, from 0 to 15.
  • cue() Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
  • OAVp5.drums.count How many steps the drum machine has played since it started, from 0; -1 before the first one.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 316

drumStep#

variableoav-p5
drumStep

Also: OAVp5.drums.step

Which of the 16 steps of the bar the drum machine just played, from 0 to 15.

A bar has 16 steps (sixteenth notes) and a beat has 4, so drumStep % 4 === 0 is a beat and drumStep === 0 the start of a bar. At 120 bpm a step lasts 125 ms. Use it to light a cell or pick a frame: fill(i === drumStep ? 255 : 80). If you would rather be called on each step, define stepStarted(). For something that moves smoothly instead of in 16 jumps, use drumPhase.

Type
Number
Range
0..15

Changes in the frame in which a step is heard, right before draw(). 0 at the start, and it keeps its last value when the machine stops.

Before it has a value

Start the machine with startDrums(). Before that, and when the machine is stopped, the value is 0 or the last one, so check drumsAreOn if the difference matters.

Notes

  • A sketch that declares its own variable called drumStep keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.drums.step instead. While the tab is hidden the clock keeps counting but nothing else is told.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 120);
  noStroke();
}

function draw() {
  background(20);
  for (let i = 0; i < 16; i++) {
    fill(drumsAreOn && i === drumStep ? 255 : 70);
    rect(20 + i * 22, 130, 18, 40);   // sixteen cells, the current one lights up
  }
}
Sixteen cells in a row with the current step lit, one cell at a time, from left to right.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 110);
  textAlign(CENTER, CENTER);
  textSize(18);
}

function draw() {
  background(drumStep % 4 === 0 ? 70 : 20);   // a flash on every beat
  fill(255);
  text('step ' + drumStep + ' of 16' + (drumStep === 0 ? '  (bar starts)' : ''), width / 2, height / 2);
}
The background lights up on every beat (steps 0, 4, 8, 12) and the text counts the steps.

Related

  • stepStarted() An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.
  • drumBar Which bar the drum machine is in, counted from 1 when it started.
  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
  • beatPhase How far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
  • drumsAreOn Whether the drum machine is playing right now.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 316

stepStarted()#

eventoav-p5
function stepStarted(step, bar) {}

An event you define: it is called at every one of the 16 steps of the drum machine, with the step and the bar.

Define function stepStarted(step, bar) { … } for something on every step: if (step % 4 === 0) ping = 1; is a pulse on the beat. It is called in the frame where the step is heard, right before draw(), after barStarted() and before the hits of that step. drumStep holds the same number if you would rather read than be called.

Parameters

NameTypeDescription
stepNumberThe step that starts, 0 to 15. Every 4 steps is a beat.
barNumberThe bar it is in, counted from 1.

Notes

  • It is not called while the tab is hidden, so a sketch that adds a particle per step does not burst when you come back.

Examples

jsOpenProcessing ↗
let ping = 0;

function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 120);
  noStroke();
}

function stepStarted(step, bar) {
  if (step % 4 === 0) ping = 1;   // on every beat
}

function draw() {
  background(20);
  ping *= 0.9;
  fill(255, 160, 80);
  circle(width / 2, height / 2, 40 + ping * 120);
}
A circle that pulses on every beat (every fourth step) and shrinks back in between.

Related

  • drumStep Which of the 16 steps of the bar the drum machine just played, from 0 to 15.
  • barStarted() An event you define: it is called at the first step of every bar of the drum machine, with the bar number.
  • drumHit() An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
  • cueFired() An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.
  • OAVp5.drums.on() Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.

p5/oav-p5.js, line 1491

OAVp5.drums.count#

variableoav-p5
OAVp5.drums.count

How many steps the drum machine has played since it started, from 0; -1 before the first one.

It is the single clock the others are cut from: drumBar is floor(count / 16) + 1, drumStep is count % 16, and cue() fires when count reaches the step of its bar. It is handy when you want one number that only grows: a position on a very long timeline, or an index into a list.

Type
Number
Range
-1, 0, 1, 2 …

Goes up by one at every step and never goes back to 0 at a new bar. It starts again from 0 when the machine is started again.

Before it has a value

Start the machine with startDrums(); it is -1 until the first step has played.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('half time', 140);
  textAlign(CENTER, CENTER);
  textSize(28);
}

function draw() {
  background(20);
  fill(255);
  const n = OAVp5.drums.count;
  text('step ' + n + '  =  bar ' + (floor(n / 16) + 1) + ', step ' + (n % 16), width / 2, height / 2);
}
The running step count and the bar and step it is made of.

Related

  • drumBar Which bar the drum machine is in, counted from 1 when it started.
  • drumStep Which of the 16 steps of the bar the drum machine just played, from 0 to 15.
  • cue() Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 1437

OAVp5.drums.cue()#

functionoav-p5
OAVp5.drums.cue(bar, name, [every])

The same as cue(bar, name, every), but it returns the cue it made (or the one that was already there), and null when bar is not valid.

Use it when you want to hold on to the cue, for example to find out which step it fires at. For a plain plan, cue() is shorter. The list of every cue is OAVp5.drums.cues.

Parameters

NameTypeDescription
barNumberThe bar, counted from 1; 4.5 is halfway through bar 4. See cue().
nameAnyWhat to call the cue.
everyNumberRepeat every this many bars; 0 or left out: once. optional

Returns

Object | null The cue: { bar, name, every, at, period, key } with at and period counted in steps. null if bar is below 1 or not a number. A repeat of an earlier call gives back the cue that exists.

Examples

jsOpenProcessing ↗
let c = null;

function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 120);
  c = OAVp5.drums.cue(3.5, 'half', 4);
  textAlign(CENTER, CENTER);
}

function draw() {
  background(20);
  fill(255);
  text(c ? 'cue "' + c.name + '" at step ' + c.at + ', every ' + c.period + ' steps' : 'no cue', width / 2, height / 2);
}
Bar 3.5 is step 40 of the count, and "every 4 bars" is 64 steps.

Related

  • cue() Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
  • OAVp5.drums.cues The list of every cue declared so far, in the order they were declared.
  • cueFired() An event you define: it is called when the bar of a cue declared with cue() has come, with the cue's name and the bar.

p5/oav-p5.js, line 1439

OAVp5.drums.cues#

variableoav-p5
OAVp5.drums.cues

The list of every cue declared so far, in the order they were declared.

Read it to show a plan on screen or to check what is declared. It is the machine's own list, so treat it as read-only; declare cues with cue().

Type
Array<Object>

Each item is { bar, name, every, at, period, key }: the values you passed, and at and period counted in steps. An empty list at the start.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 120);
  cue(1, 'calm', 8);
  cue(5, 'storm', 8);
  textSize(14);
}

function draw() {
  background(20);
  fill(255);
  OAVp5.drums.cues.forEach((c, i) => {
    text('bar ' + c.bar + (c.every ? ' (every ' + c.every + ')' : '') + ': ' + c.name, 20, 30 + i * 24);
  });
}
The plan as a list: one line per cue.

Related

  • cue() Plans something for a bar: at bar bar the function cueFired(name, bar) is called and cueName becomes name; with every, again every that many bars.
  • OAVp5.drums.cue() The same as cue(bar, name, every), but it returns the cue it made (or the one that was already there), and null when bar is not valid.
  • cueName The name of the last cue that fired, or an empty string before the first one.

p5/oav-p5.js, line 1437

@openav/score

Director#

class@openav/score
new Director({ score, modules = [], api = {}, onStatus = null, onCue = null, maxStep = 2 } = {})

Runs a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.

A segment module is the code that performs one segment (the dawn, the storm, the finale). The director calls its hooks as the show moves through the score:

js
{
  id: 'dusk',                                  // the id of the segment it performs
  enter(api, ctx)  {},                         // the segment began, or a seek landed inside it
  update(api, ctx) {},                         // every frame while the segment (and its linger) is current
  exit(api, ctx)   {},                         // it ended, or a seek left it: give back what you took
  param(key, value, ctx) { return value; },    // optional: rewrite a timeline param meanwhile
  cue(name, ctx)   {},                         // optional: one of its cues was just passed
  reset(api)       {},                         // optional: R was pressed, forget everything
}

Every hook gets a ctx: p (progress through the segment, 0 to 1; above 1 during linger; exactly 1 while holding), t (seconds into the segment), dt, dur, T (show time), segment, cue(name) and cueT(name) (where a cue sits, as a progress and in show seconds), phase ('active', 'linger', or 'failed' inside exit() after a crash), holding, held (seconds waited at a hold: a clock for ambient motion while T is frozen), playing and api.

A module is entered once and exited once, however the show got there: a seek into the middle of a segment calls enter with the right p, so a module must be able to start from the middle. A module that throws is switched off alone: the director calls its exit() once with phase: 'failed', logs one console.error, and never calls it again; the other modules and the show carry on. Reset brings it back.

In a show, createShow() builds the director from its score option and calls update() every frame after the timeline has advanced; the director's param() plugs into timeline.layer. Without createShow() you do that wiring yourself: director.update(timeline.t, dt, { holding: timeline.holding, playing: timeline.playing }).

Parameters

NameTypeDescription
scoreScoreThe Score to perform. Required: without it the constructor throws "Director needs a score".
modulesArray | ObjectThe segment modules: an array (matched to segments by each module's id) or an object keyed by segment id. A segment with no module is simply not performed. optional default []
apiObject | FunctionWhat every module hook receives as its first argument (also ctx.api). A function is called once, the first time it is needed, and its result is used. optional default {}
onStatusFunction | nullCalled with status() when the segment, the holding state or the list of switched-off modules changes, not every frame. optional default null
onCueFunction | nullCalled for each cue the playhead passes: { name, cue, T, n, segment }. optional default null
maxStepNumberA jump bigger than this many seconds between two update() calls counts as a seek, so the cues in between are not fired. Raise it if you rehearse at a very high timeline.rate. optional default 2

Members

NameDescription
apigetterThe api object the modules receive (a function given as api is called on first use).
register(mod)methodAdd a segment module (it replaces one with the same id). Throws when the module has no string id. Returns the director.
moduleFor(seg)methodThe module that performs a segment: the one named by its module field, else the one with its own id, else null.
holdinggetterWhether the show is parked at a hold, as told by the last update().
update(T, dt = 0, state = {})methodCall once per frame, after the timeline advanced: update(T, dt, { holding, playing }) with T the show time. Enters, updates and exits the modules and fires the cues passed since the previous call.
param(key, value, T = this._T)methodThe timeline's param layer: param(key, value, T) lets the modules of the current segments rewrite the value (a module returns a number to replace it). A performer's override still wins afterwards.
seek(T)methodThe playhead jumped to T: modules whose segment no longer covers T exit now, and the next update() enters the new ones with the right p.
reset()methodBack to the start: every active module exits, every module hears reset(api), and switched-off modules come back.
status()methodFor the desk and the backstage: { cut, T, total, segment, next, holding, held, active, failed, errors, missing }, with the current segment's progress and time remaining, the next segment and its countdown, and who is switched off.

Notes

  • The last segment is never exited by time: while the show sits at its end (or past it), the last module stays entered. It exits on a seek that leaves it, or on reset().
  • A new segment's module is entered before the old one exits, and during linger both run at the same time.
  • A module whose module or id matches no segment, or a segment that names a module nobody registered (listed in status().missing), is simply not performed. The show keeps going.

Examples

js
import { Score, Director } from '@openav/score';

const score = new Score({ cuts: [{ id: 'full', segments: [
  { id: 'dawn', dur: 4, cues: { birds: 0.5 } },
  { id: 'noon', dur: 4 },
] }] });
const say = (id) => ({
  id,
  enter(api, ctx) { console.log(id, 'enter, p =', ctx.p); },
  exit() { console.log(id, 'exit'); },
  cue(name) { console.log(id, 'cue', name); },
});
const director = new Director({ score, modules: [say('dawn'), say('noon')] });
for (let T = 0; T <= 8; T += 1) director.update(T, 1, { playing: true });   // one call per frame
module The console prints dawn enter, p = 0, dawn cue birds (second 2), then noon enter, p = 0 and dawn exit at second 4. The last segment stays entered to the end.
js
import { Score, Director } from '@openav/score';

const score = new Score({ cuts: [{ id: 'full', segments: [{ id: 'dawn', dur: 4 }, { id: 'noon', dur: 4 }] }] });
const sun = { id: 'dawn', param: (key, value, ctx) => (key === 'sun' ? ctx.p : value) };
const director = new Director({ score, modules: [sun] });

director.update(1, 1, { playing: true });
console.log(director.param('sun', 0.9, 1), director.param('moon', 0.9, 1));   // 0.25 0.9
director.update(2, 1, { playing: true });
console.log(director.param('sun', 0.9, 2));                                     // 0.5
module While dawn is current its module bends only the sun param, following its own progress: the console prints 0.25 0.9, then 0.5.
js
import { Score, Director } from '@openav/score';

console.error = (message) => console.log('logged:', String(message).slice(0, 62));   // show the director's error line here

const score = new Score({ cuts: [{ id: 'full', segments: [{ id: 'dawn', dur: 4 }, { id: 'noon', dur: 4 }] }] });
const dawn = { id: 'dawn', update() { throw new Error('boom'); } };
const noon = { id: 'noon', enter() { console.log('noon is still performed'); } };
const director = new Director({ score, modules: [dawn, noon] });

director.update(1, 1, { playing: true });     // dawn throws: it is switched off, once
director.update(5, 1, { playing: true });     // the show goes on
console.log(director.status().failed, director.status().errors);
director.reset();                              // brings dawn back
console.log(director.status().failed);
module A crash is contained: the console prints logged: [score] segment module "dawn" crashed in update: boom and was (normally a red error line), then noon is still performed, [ 'dawn' ] { dawn: 'update: boom' }, and after the reset [].

Related

  • Score A show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.
  • ScoreError The error a Score throws when it cannot be built: its problems array lists every reason at once, and its message lists them too.
  • Timeline Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • mountConsole() Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.

import { Director } from '@openav/score'; packages/score/director.js, line 38

Score#

class@openav/score
new Score({ cuts, cut } = {})

A show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.

A timeline written as a table of absolute seconds cannot grow: make one scene longer and every later cue, curve and if (t > 466) is wrong. In a score you write only how long each segment is, and Score computes every start, end, cue time, countdown and scene from those lengths. Change one dur and everything after it moves; a second version of the show is a few lines of data.

A cut is one version of the show; a segment is one stretch of it:

js
{ id: 'full', title: 'A day', segments: [
  { id: 'standby', dur: 6, hold: true, acts: ['Press Space when the room is ready'] },
  { id: 'dawn',    dur: 24, cues: { birds: 0.3 } },          // a ratio of the segment
  { id: 'noon',    dur: 30, cues: { flare: { s: 8 } } },     // or seconds into it
  { id: 'dusk',    dur: 24, linger: 8 },
] }
{ id: 'short', from: 'full', scale: 0.5 }                     // the same show at half the length

dur is the only time a segment has. cues are named moments (a ratio 0 <= r < 1, or { s: seconds }), acts is what the human operator does (strings shown on the desk), linger keeps a segment's module running for some seconds after it ends, hold stops playback at the end of the segment until the operator releases it, and module names the module that performs it when that is not the segment's own id. A derived cut scales dur, linger and { s } cues, so a cue keeps its place in the music.

Everything is checked when the score is built. Duplicate ids, a dur of 0 or below, a from that is not a cut, a cut derived from itself, a cue outside its segment: each throws a ScoreError that lists every problem at once. In a show, createShow() builds the score from its score option and hands it to a Timeline and a Director; ?cut=<id> in the page address picks the cut.

Parameters

NameTypeDescription
cutsArray | ObjectThe cuts this show can be performed as: an array of cuts, or an object keyed by cut id. A cut is { id, title?, start?, segments: [...] }, or a derived one, { id, title?, from, scale }. Required: without a cut it throws.
cutStringThe id of the cut to perform. Defaults to the first one; an id that is not in cuts is an error. optional default the first cut

Members

NameDescription
cutsgetterEvery cut, for a version picker: [{ id, title, total, from, scale, base }].
validate()methodCheck everything again; throws a ScoreError that lists every problem, and returns true when sound. The constructor already calls it.
table()methodOne row per segment, { index, id, title, start, end, dur, linger, hold }, for console.table() and docs.
segment(id)methodThe segment with this id, or undefined.
indexAt(T)methodThe index of the segment playing at show time T. A segment owns [start, end). Before 0 (or NaN) answers 0, after the end the last one.
segmentAt(T)methodThe segment playing at show time T (see indexAt).
cueTime(name)methodA moment in show seconds: 'dusk' is the segment's start, 'dusk.end' its end, 'dusk.glow' a cue declared in the cut. NaN when the name means nothing here.
cues()methodEvery cue of the cut in time order: [{ name, cue, T, p, n, segment }], where n is its place in that order.
cuesBetween(T0, T1)methodThe cues a playhead passed going from T0 to T1: those with T0 < time <= T1, in order. Nothing when T1 <= T0.
baseTime(T)methodTime T expressed on the cut this one was derived from (T / scale). Equal to T when the cut is not derived. This is how automation written once against the full cut stretches with a short one.
scenes()methodThe segments as a scenes array for a Timeline, with acts, hold and cues (cue times in show seconds).

Notes

  • Plain fields: cut (the id being performed), title, segments (each with start, end, dur, index, cues …), total (the length of the show), startT (where reset goes), scale and base.
  • Segment and cue ids must not contain a dot: the dot is how cueTime('dusk.glow') separates a segment from its cue. A cue cannot be called end.
  • If two cuts differ in structure and not only in length, do not share automation between them. Drive the params from the segment modules instead, where ctx.p already scales for free.

Examples

js
import { Score } from '@openav/score';

const score = new Score({ cuts: [{ id: 'full', segments: [
  { id: 'dawn', dur: 24, cues: { birds: 0.25 } },
  { id: 'noon', dur: 30, cues: { flare: { s: 8 } } },
  { id: 'dusk', dur: 26 },
] }] });
console.table(score.table());
console.log(score.total, score.segmentAt(40).id, score.cueTime('noon.flare'));
module The console shows a table of the three segments (dawn 0 to 24, noon 24 to 54, dusk 54 to 80), then 80 noon 32: the show is 80 seconds long, second 40 is in noon, and the flare cue sits at 32.
js
import { Score } from '@openav/score';

const cuts = [
  { id: 'full', segments: [{ id: 'dawn', dur: 24, cues: { birds: 0.25 } }, { id: 'noon', dur: 30, cues: { flare: { s: 8 } } }] },
  { id: 'short', from: 'full', scale: 0.5 },
];
for (const cut of ['full', 'short']) {
  const score = new Score({ cuts, cut });
  console.log(cut, score.total, score.cueTime('dawn.birds'), score.cueTime('noon.flare'));
}
module The same show at two lengths: full 54 6 32, then short 27 3 16. Every time is halved and the cues stay in the same place in the music.
js
import { Params } from '@openav/core';
import { Score } from '@openav/score';
import { Timeline } from '@openav/timeline';

const cuts = [
  { id: 'full', segments: [{ id: 'a', dur: 20 }, { id: 'b', dur: 40, hold: true }, { id: 'c', dur: 20 }] },
  { id: 'short', from: 'full', scale: 0.5 },
];
const params = new Params([{ key: 'haze', min: 0, max: 1, def: 0 }]);
for (const cut of ['full', 'short']) {
  const timeline = new Timeline({ params, score: new Score({ cuts, cut }), automation: { haze: [[0, 0], [80, 1]] } });
  console.log(cut, timeline.total, timeline.scenes.map((s) => s.id + '@' + s.t).join(' '), timeline.valueAt('haze', 20));
}
module The timeline takes its scenes and total from the score, and reads the haze curve (written once for the full cut) stretched: full 80 a@0 b@20 c@60 0.25, then short 40 a@0 b@10 c@30 0.5.

Related

  • ScoreError The error a Score throws when it cannot be built: its problems array lists every reason at once, and its message lists them too.
  • Director Runs a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
  • Timeline Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { Score } from '@openav/score'; packages/score/score.js, line 48

ScoreError#

class@openav/score
new ScoreError(problems)

The error a Score throws when it cannot be built: its problems array lists every reason at once, and its message lists them too.

You rarely build one yourself: new Score() throws it. error.problems is an array of strings and error.message is the same list as one text, headed "Score is invalid (3 problems)". error.name is 'ScoreError', and it is a normal Error, so instanceof ScoreError and instanceof Error both work.

It lists every problem it found, not only the first, so one run of the show shows you everything to fix. Nothing half-valid can reach the stage.

Parameters

NameTypeDescription
problemsArray<String>The reasons, one string each. Each one names the cut, the segment and what is wrong.

Examples

js
import { Score, ScoreError } from '@openav/score';

try {
  new Score({ cuts: [
    { id: 'a', segments: [{ id: 'x', dur: 0 }, { id: 'x', dur: 5 }] },
    { id: 'b', from: 'missing', scale: 1 },
  ] });
} catch (error) {
  console.log(error instanceof ScoreError, error.problems.length);
  for (const problem of error.problems) console.log('-', problem);
}
module Three mistakes, all reported at once: the console prints true 3 and then one line for each (a dur of 0, a repeated id, a from that is not a cut).

Related

  • Score A show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.
  • Director Runs a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.

import { ScoreError } from '@openav/score'; packages/score/score.js, line 30

@openav/timeline

CURVES#

constant@openav/timeline
CURVES = object, 5 keys

The curve names a keyframe can use to say how its segment gets there: linear, ease, in, out, step, each a function from 0..1 to 0..1.

In a Timeline, the third item of a keyframe names the curve of the segment that ends at that keyframe: [30, 320, 'ease'] means "arrive at 320 at second 30 the ease way". linear is the default, and an unknown name or a typo also falls back to linear, so a mistake never stops a show. You can pass your own function instead of a name.

linear goes straight. ease is a smoothstep: slow, fast, slow. in starts slowly (f²). out starts fast and lands softly (1 − (1 − f)²). step is special: the segment holds the old value until the keyframe, then jumps. The step function stored here returns 0 and is never called to interpolate; it is listed so that the object is the whole vocabulary.

A param declared with step in its Params schema holds each keyframe whatever curve you name.

Examples

js
import { CURVES } from '@openav/timeline';

for (const name of ['linear', 'ease', 'in', 'out']) {
  console.log(name, CURVES[name](0.5));        // halfway through the segment
}
module The console prints how far each curve has got at the halfway point: linear 0.5, ease 0.5, in 0.25, out 0.75.
js
import { Params } from '@openav/core';
import { Timeline } from '@openav/timeline';

const params = new Params([{ key: 'speed', min: 0, max: 4, def: 0 }]);
const timeline = new Timeline({ params, automation: {
  speed: [[0, 0.3], [16, 1.6, 'in'], [20, 3.4, 'step'], [30, 0.8, 'out']],
} });
for (const t of [8, 16, 19, 20, 25, 30]) console.log(t, timeline.valueAt('speed', t).toFixed(2));
module One lane, three curves: the console prints a slow start (8 0.63), the arrival at 16 1.60, a long hold (19 1.60) until the step jumps at 20 (20 3.40), then an out curve down to 30 0.80.
js
import { Params } from '@openav/core';
import { Timeline } from '@openav/timeline';

const params = new Params([{ key: 'speed', min: 0, max: 4, def: 0 }]);
const timeline = new Timeline({ params, automation: {
  speed: [[0, 0], [10, 4, (f) => f * f * f]],      // your own curve: a function of 0..1
} });
console.log(timeline.valueAt('speed', 5));
module A function works wherever a name does. Halfway through, a cubic curve is only an eighth of the way: the console prints 0.5 (4 × 0.125).

Related

  • Timeline Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
  • approach() One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.

import { CURVES } from '@openav/timeline'; packages/timeline/index.js, line 28

HOLD_EPS#

constant@openav/timeline
HOLD_EPS = 0.000001

How far before the boundary the playhead parks when a hold stops the show: one microsecond, so it still belongs to the scene that is waiting.

A scene with hold: true stops playback at its end. The Timeline parks HOLD_EPS (0.000001 seconds) before the boundary rather than on it, so sceneIndexAt(t) and the director still say "the held scene", timeline.t reads 5.999999 for a hold at 6, and the next scene has not started. release() then moves the playhead onto the boundary itself.

You only need it when you compare the playhead with a boundary yourself: while holding, timeline.t is boundary - HOLD_EPS, not exactly boundary.

Examples

js
import { Params } from '@openav/core';
import { Timeline, HOLD_EPS } from '@openav/timeline';

const params = new Params([]);
const timeline = new Timeline({ params, total: 20, scenes: [{ id: 'wait', t: 0, hold: true }, { id: 'go', t: 6 }] });
timeline.play();
for (let i = 0; i < 10; i++) timeline.advance(1);
console.log(timeline.holding, timeline.currentScene().id, Math.abs(6 - timeline.t - HOLD_EPS) < 1e-9);
module The show parks just before second 6, still inside the held scene: the console prints true wait true (6 - timeline.t is HOLD_EPS, apart from rounding).

Related

  • Timeline Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
  • Director Runs a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
  • Score A show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.

import { HOLD_EPS } from '@openav/timeline'; packages/timeline/index.js, line 38

Timeline#

class@openav/timeline
new Timeline({ params, automation = {}, scenes = [], total = 600, score = null, start = null, loop = false, onCue = null })

Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.

timeline.state(t) returns { key: value } for every param at time t, interpolated from automation. That is the whole job: the timeline owns time, and a world never reads the clock. In a show, createShow() builds the timeline from its timeline option, Loop calls advance(dt) every frame, and what state() returns goes into params.resolve(), where a hand or a knob that has overridden a param still wins.

Between two keyframes the value moves in a straight line. The optional third item of a keyframe says how the segment that ends at that keyframe gets there: 'linear', 'ease', 'in', 'out', 'step', or your own function of 0..1 (see CURVES). Before the first keyframe the value is the first one, after the last it is the last one, and a param with no keyframes stays at its def. A param declared with step holds each keyframe instead of interpolating.

Scenes give the show its shape: { id: 'storm', t: 90, title: 'Storm', note: 'clusters welcome' }. onSceneChange is the hook to fire sound, lights or prompts, and the console shows the title and note to the performer. A scene with hold: true stops playback at its end until someone calls release(). Scene cues are { name, t } with t in show seconds (not seconds into the scene), and they fire only when playback passes them: a seek, a jump or a reset never does.

Before you call it

Time only moves when you call play() and then advance(dt) every frame (a show does this for you). state(t) and valueAt(key, t) work at any moment, playing or not. The callbacks do not fire for the starting scene until the first advance() or seek().

Parameters

NameTypeDescription
paramsParams | ArrayThe Params of the show, or a plain schema array. The timeline reads the schema to know which params to resolve. Required.
automationObjectThe keyframes of each param: { hue: [[0, 200], [30, 320, 'ease'], [60, 40]] }. A keyframe is [seconds, value, curve?], sorted by time. optional default {}
scenesArrayThe scenes: { id, t, title?, note?, acts?, hold?, cues? }, sorted by t. A negative t is a pre-show standby. Ignored when a score is given. optional default []
totalNumberThe length of the show in seconds. Playback stops by itself there. (createShow() passes 120 unless you say otherwise.) Ignored when a score is given. optional default 600
scoreScore | nullA Score: the scenes, the total and the start then come from it, and automation is read at score.baseTime(t), so a derived cut stretches the curves with it. optional default null
startNumber | 'first' | nullWhere reset() goes and where the playhead starts: a number of seconds, or 'first' for the start of the first scene (a standby at −30, say). null means 0, or the score's start. optional default null
loopBoolean | String | Objecttrue wraps the show at its end and plays on; 'scene' repeats the scene the playhead is in; a scene id repeats that scene; { from, to } repeats a range (seconds or scene ids). It is a plain property: set timeline.loop any time. optional default false
onCueFunction | nullThe same as calling timeline.onCue(cb) once: cb(cue, { cause }). optional default null

Members

NameDescription
valueAt(key, t = this.t)methodOne param at time t (default: now): the automation value with the curves applied, before the layer and before any override.
state(t = this.t)methodEvery param at time t as { key: value }. This is the base state that params.resolve() then lets overrides win over. If layer is set, each value passes through it.
sceneIndexAt(t = this.t)methodThe index of the scene that contains time t (the first scene before any scene starts).
sceneEnd(i)methodThe end of scene i in seconds: the start of the next scene, or total for the last one.
currentScene(t = this.t)methodThe scene object that contains time t (default: now).
onSceneChange(cb)methodRegister cb(index, scene, { cause }), called when the playhead lands in a different scene. cause is 'play', 'release', 'seek', 'jump', 'reset' or 'loop'. There is no way to remove it again.
onSeek(cb)methodRegister cb(t, kind), called when the playhead is moved by hand: kind is 'seek', 'jump' or 'reset'. Never called while playing.
play()methodStart playing. Parked at a hold? Then this releases it instead.
pause()methodStop playing and close any take being recorded.
toggle()methodPlay if paused, pause if playing (at a hold: release). Returns whether it is playing now.
seek(t, cause = 'seek')methodMove the playhead to t, held between the first scene's start (when negative) or 0, and total. Leaving a hold this way releases it. cause (default 'seek') is what onSceneChange and onSeek report. A seek never fires cues.
jumpScene(d)methodGo to the start of the scene d steps away (1 next, -1 previous), stopping at the first and last. It is a seek with the cause 'jump'.
next()methodThe → key: release a hold and keep playing, otherwise jump to the next scene.
prev()methodThe ← key: jump to the previous scene.
release()methodLet a hold go: the next scene starts now and playback continues. Returns false when nothing was holding.
reset()methodBack to the start (start, default 0): stopped, not holding, loop count cleared.
advance(dt)methodCall once per frame with the seconds that passed. Moves t by dt × rate while playing, parks at a hold, wraps a loop, stops at total, and fires scene changes and cues on the way. Returns whether it is still playing.
loopRange(t = this.t)methodThe range being looped at time t, as { a, b } in seconds, or null when loop is off or empty.
onLoop(cb)methodRegister cb({ t, from, to, count }), called each time the playhead wraps round the loop.
cues()methodEvery cue of every scene in time order, as { …cue, scene: sceneId, sceneIndex }.
onCue(cb)methodRegister cb(cue, { cause }), called when playback passes a scene cue (cause 'play') or fireCue() sets one off ('manual').
fireCue(c)methodSet a cue off by hand (the scrubber's ◆ click). Takes an item of cues() or any { name, t }.
record(key, source, opts = {})methodArm a lane: while the timeline plays, whatever source() returns is written into automation[key] as keyframes. source returns a number while someone is touching the control and null when not; that closes the take. Throws for an unknown key or a source that is not a function. Returns the recorder ({ key, take, active, commit(), stop() }).
recordinggetterWhether any lane is armed.
stopRecording()methodClose every armed recorder, committing what each holds.
onRecord(cb)methodRegister cb({ key, from, to, count, cause, keyframes }), called when a take closes. count 0 means it was too short to keep.
mergeTake(key, raw, opts = {})methodWrite a take (an array of [t, value]) into automation[key]: thin it, replace the keyframes inside its time span, keep the ones outside. Returns { from, to, count }, or null for fewer than two points.

Notes

  • These are plain fields you can read and set: t, playing, holding, rate (1 is real time; 4 rehearses at four times the speed), total, start, scenes, automation, score, loop, loops (laps since the last reset), and layer.
  • layer is (key, value, t) => value. A Director plugs in here so segment modules can bend a param; the performer's override is still applied afterwards and wins.
  • With a score, anything you pass as scenes or total is ignored, and reset() goes to the score's start.

Examples

js
import { Params } from '@openav/core';
import { Timeline } from '@openav/timeline';

const params = new Params([{ key: 'hue', min: 0, max: 360, def: 200 }]);
const timeline = new Timeline({
  params,
  total: 60,
  automation: { hue: [[0, 200], [30, 320], [60, 40, 'ease']] },
});
for (const t of [0, 15, 30, 45]) console.log(t, timeline.state(t).hue);
module The console prints the hue at four moments: 0 200, 15 260, 30 320, 45 180. Nothing plays; state(t) just looks the answer up.
js
import { Params } from '@openav/core';
import { Timeline } from '@openav/timeline';

const params = new Params([{ key: 'energy', min: 0, max: 1, def: 0.2 }]);
const timeline = new Timeline({
  params,
  total: 20,
  scenes: [
    { id: 'standby', t: 0, hold: true },
    { id: 'go', t: 6, cues: [{ name: 'flash', t: 10 }] },
  ],
});
timeline.onSceneChange((i, scene, { cause }) => console.log('scene', scene.id, cause));
timeline.onCue((cue) => console.log('cue', cue.name, 'at', cue.t));

timeline.play();
for (let i = 0; i < 8; i++) timeline.advance(1);   // one second per call
console.log('holding:', timeline.holding, 'at', timeline.t.toFixed(3));
timeline.release();                                  // the operator says go
for (let i = 0; i < 6; i++) timeline.advance(1);
module The console prints scene standby play, then holding: true at 6.000 (the show waits at the end of the standby), then scene go release, then cue flash at 10.
js
import { Params } from '@openav/core';
import { Timeline } from '@openav/timeline';

const params = new Params([{ key: 'hue', min: 0, max: 360, def: 100 }]);
const timeline = new Timeline({ params, total: 30, automation: { hue: [[0, 100], [30, 100]] } });

let knob = null;                                  // a number while a hand holds the knob
timeline.record('hue', () => knob);
timeline.onRecord((take) => console.log('take', take.from, 'to', take.to, take.cause));

timeline.play();
for (let i = 0; i < 20; i++) {
  knob = i >= 5 && i < 12 ? 100 + (i - 5) * 20 : null;   // turned from second 5 to 11
  timeline.advance(1);
}
console.log(JSON.stringify(timeline.automation.hue));
module The knob turns for seven seconds; the console prints take 6 to 12 release, then the new keyframes: the take replaced the middle of the curve and the old ends stayed.

Related

  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
  • Loop The frame clock: calls your function every animation frame with the seconds since the last one, and keeps ticking when the tab is hidden.
  • Score A show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.
  • Director Runs a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
  • CURVES The curve names a keyframe can use to say how its segment gets there: linear, ease, in, out, step, each a function from 0..1 to 0..1.
  • HOLD_EPS How far before the boundary the playhead parks when a hold stops the show: one microsecond, so it still belongs to the scene that is waiting.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • mountConsole() Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.

import { Timeline } from '@openav/timeline'; packages/timeline/index.js, line 40

Values over time (CHOP)#

lag(), env(), spring(), trail(): the way TouchDesigner shapes a value. 28

oav-p5

count()#

functionoav-p5
count(on, [limit], [name])

Also: OAVp5.chop.count

Counts how often on has turned on, or how many 'kick' (or any other event) have happened; with a limit it loops 0, 1, … limit − 1.

A counter you do not have to keep yourself: count(mouseIsPressed) is the number of clicks so far, count(beat, 4) walks 0 1 2 3 0 1 … on the beat, and count('kick') counts the kicks of the drum machine. A switch has to go off and on again to be counted twice.

Event counts start when count('kick') is first called, so earlier kicks are not counted. An event of your own fired with trigger() is counted from its first firing, whenever count() is first called.

Before you call it

The event names work like env(): 'kick', 'snare', 'hat' and 'beat' come from the microphone, or from the drum machine once startDrums() has been called, and the first count('kick') asks for the microphone if the drum machine is not on. The first call loads @openav/chop, which takes a few frames.

Parameters

NameTypeDescription
onBoolean | Number | StringA switch: a boolean, or a number above 0.5, and each time it goes from off to on counts one. Or an event name ('kick', 'beat', 'note', or one of your own fired with trigger()): how many of those events have happened.
limitNumberLoop at this number: 4 gives 0 1 2 3 0 1 … Left out, or 0 or less, it keeps counting up. optional default 0
nameString | NumberOnly for the switch form (count(beat, 4, 'a')): a name for its memory, as the last argument. The event-name form needs none. optional read from arguments

Returns

Number The count, a whole number from 0. It is 0 until @openav/chop has loaded (a few frames), and for an unnamed switch outside draw().

Notes

  • Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
  • A sketch with its own variable or function called count keeps it. OAVp5.chop.count(…) is this function under another name.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(96);
}

function draw() {
  background(20);
  fill(255);
  text(count(mouseIsPressed), width / 2, height / 2);   // clicks so far
}
A counter that goes up by one on every click. Holding the button down counts once.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
  startDrums('bd*4, [~ sd]*2, hh*8', 108);
}

function draw() {
  background(20);
  const nth = count('kick', 4);   // 0 1 2 3 0 1 ...
  for (let i = 0; i < 4; i++) {
    fill(i === nth ? color(255, 90, 31) : 70);
    circle(100 + i * 67, 150, i === nth ? 50 : 30);
  }
}
Four dots light up one after another, one step per kick of the drum machine. Click the canvas once if you hear nothing.
jsOpenProcessing ↗
const colors = ['#ff5a1f', '#78beff', '#ffd36b'];

function setup() {
  createCanvas(400, 300);
}

function mousePressed() { trigger('boom'); }

function draw() {
  background(colors[count('boom', 3)]);   // 0 1 2 0 1 2 ...
  fill(20);
  text('clicks: ' + count('boom'), 20, 30);
}
Each click cycles the background through three colours, and the text counts the clicks without end.

Related

  • env() A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.
  • trigger() Fires an event of your own, by name, for env() and count() to react to.
  • beat True for one frame each time the microphone (or the drum machine) hears a beat.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 3057

drawTrail()#

functionoav-p5
drawTrail(values, [x], [y], [w], [h], [lo], [hi], [name])

Also: OAVp5.chop.drawTrail

Draws a line through values on the canvas, so it is in the work and in a recording: the oldest at x, lo at the bottom of the box, hi at the top.

The picture scope() shows in the corner, but drawn on the canvas: stroke(255); drawTrail(trail(micLevel, 3), 0, height - 100, width, 100). The line is one shape in the sketch's own stroke() colour and weight (white while noStroke() is in effect), with no fill, and the style you had before is put back afterwards.

The oldest value is at the left edge x, the newest at x + w. A number instead of an array is the shortcut: drawTrail(micLevel, 0, 0, width, 100) draws the last 2 seconds without a separate trail() call.

Before you call it

Call it in draw() after createCanvas(). With a number, the first call loads @openav/chop, which takes a few frames, and nothing is drawn until then.

Parameters

NameTypeDescription
valuesArray<Number> | NumberAn array such as the one trail() returns. Or a single number: then drawTrail keeps its own trail of the last 2 seconds.
xNumberLeft edge of the box. optional default 0
yNumberTop edge of the box. optional default 0
wNumberWidth of the box. Left out, it is the canvas width. optional default width
hNumberHeight of the box. Left out, it is the canvas height. optional default height
loNumberThe value drawn at the bottom of the box. optional default 0
hiNumberThe value drawn at the top of the box. Values outside lo..hi sit on the edge of the box. If hi equals lo, the line runs through the middle. optional default 1
nameString | NumberOnly when values is a single number: a name for its own trail, as the 8th argument (after hi). optional read from arguments

Returns

Array<Number> The array that was drawn: the one you passed, or the number's own trail. Nothing is drawn when it has fewer than 2 values.

Notes

  • Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
  • OAVp5.chop.drawTrail(…) is this function under another name.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const wave = 0.5 + 0.5 * sin(frameCount * 0.05);
  stroke(255, 211, 107);
  strokeWeight(2);
  drawTrail(wave, 20, 100, width - 40, 100);   // a number: keeps its own 2 seconds
}
A moving sine line drawn on the canvas itself, so it would appear in a recording.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noFill();
  strokeWeight(2);
}

function draw() {
  background(20);
  const raw = frameCount % 120 < 60 ? 1 : 0;
  const smooth = lag(raw, 0.3, 0.6);
  stroke(95, 224, 138);
  drawTrail(trail(raw, 3), 20, 60, width - 40, 80);       // 0 at the bottom, 1 at the top
  stroke(255, 211, 107);
  drawTrail(trail(smooth, 3), 20, 60, width - 40, 80);    // same box: drawn on top
}
The raw square wave in green and its lagged version in yellow, in the same box so you can see the delay.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noFill();
  stroke(255);
}

function draw() {
  background(20);
  const wave = sin(frameCount * 0.05);   // -1..1
  drawTrail(trail(wave, 3), 20, 100, width - 40, 100, -1, 1);   // lo and hi say what the box shows
}
A wave that goes below 0: lo = -1 and hi = 1 put the bottom of the box at -1 and the top at 1, so the whole wave fits.

Related

  • trail() The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).
  • scope() Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 3152

env()#

functionoav-p5
env([event], [attack], [decay], [name])

Also: OAVp5.chop.env

A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.

Call it every frame and use the number like a size or a brightness: circle(width / 2, height / 2, 60 + env('kick', 0.01, 0.3) * 200) swells on every kick. The number rises to 1 in attack seconds after an event and falls back to 0 in decay seconds, quick at first and landing softly, the way a bass drum's swell looks right.

An event that arrives while the last pulse is still sounding restarts from where the pulse is, never from zero, so fast hits pile up into a plateau instead of flickering. The attack is measured from 0: a retrigger from halfway up takes half the attack.

The events: 'kick', 'snare' and 'hat' are the hits kickDetected() hears (from the microphone, or from the drum machine when it plays), 'beat' is beat, 'note' is any key going down, 'bar' and 'step' come from the drum machine, a cue's name fires when cue() does, and any other name is yours to fire with trigger(). The switch form, env(mouseIsPressed, 0.02, 0.5), fires on every press.

Before you call it

'kick', 'snare', 'hat' and 'beat' need something that makes them. If the sketch has not started the drum machine (startDrums()), the first env('kick') asks for the microphone, like reading micLevel, and ?oav_mic=off means nothing ever fires. Once the drum machine is started, the microphone is not opened for it. 'note' needs a key: play one, press 🎹, or call OAVp5.play(). A name of your own needs trigger(): if nothing fires it within 4 seconds, a card says so.

Parameters

NameTypeDescription
eventString | Boolean | NumberWhat to react to. A name: 'kick', 'snare', 'hat', 'beat' (from the microphone or the drum machine), 'note' (any key), 'bar', 'step' (the drum machine), a cue's name, or an event of your own fired with trigger(). Or a switch: a boolean, or a number above 0.5, and each time it turns on counts as an event. Left out: 'beat'. optional default 'beat'
attackNumberSeconds the rise from 0 to 1 takes. optional default 0.01
decayNumberSeconds the fall from 1 back to 0 takes. optional default 0.3
nameString | NumberOnly for the switch form (env(mouseIsPressed, …, 'a')): a name for its memory, as the last argument. A named event such as 'kick' needs none. optional read from arguments

Returns

Number 0..1: 0 when nothing has happened, 1 at the top of the pulse. Until @openav/chop has loaded (a few frames) it returns 0.

Notes

  • A named event needs no place in draw(): env('kick') works in setup() or mousePressed() too. Only the switch form remembers anything between frames, and an unnamed switch outside draw() returns 0. Calls in draw() are matched by order, like lag().
  • OAVp5.chop.env(…) is this function under another name.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const k = env(mouseIsPressed, 0.02, 0.5);   // 1 right after each press, then it fades
  circle(width / 2, height / 2, 60 + k * 200);
}
Click the canvas: the circle jumps open and fades back each time.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
  startDrums('bd*4, [~ sd]*2, hh*8', 108);   // a drum machine, so no microphone is needed
}

function draw() {
  background(20);
  const kick = env('kick', 0.01, 0.3);       // 1 on every kick, 0 again 0.3 s later
  const snare = env('snare', 0.005, 0.15);
  fill(255, 90, 31);
  circle(120, 150, 40 + kick * 160);
  fill(120, 190, 255);
  circle(280, 150, 40 + snare * 160);
}
Two circles swell on the drum machine's kick and snare. Click the canvas once if you hear nothing: browsers wait for a click before they play sound.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  OAVp5.play(48 + floor(random(24)), 0.8);   // play a key, as a controller would
}

function draw() {
  background(20);
  const k = env('note', 0.005, 0.8);         // every key makes it flash
  fill(255, 211, 107);
  circle(width / 2, height / 2, 40 + k * 240);
}
Click the canvas, or play a key on a controller or the 🎹: every key pulses the circle.

Related

  • trigger() Fires an event of your own, by name, for env() and count() to react to.
  • count() Counts how often on has turned on, or how many 'kick' (or any other event) have happened; with a limit it loops 0, 1, … limit − 1.
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
  • beat True for one frame each time the microphone (or the drum machine) hears a beat.
  • kickDetected() You define these; oav-p5 calls them when the microphone (or the drum machine) hears a kick, a snare or a hat.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 3016

lag()#

functionoav-p5
lag(value, [up], [down], [name])

Also: OAVp5.chop.lag

Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).

Wrap any number that jumps: circle(width / 2, height / 2, 50 + lag(micLevel, 0.03, 0.4) * 300) is a VU meter. It opens within 0.03 seconds when someone speaks and takes 0.4 seconds to close. up is the time to follow a rise, down the time to follow a fall. Leave down out and both are up. Leave both out and they are 0.1.

The seconds are TouchDesigner's: lag(x, 0.1) moves like a Lag CHOP set to 0.1, so the numbers you learn here carry over. That is about the time the output takes to cover 90% of a jump (underneath, a one-pole lag whose time constant is the seconds divided by 2.3). The curve is the same at 30 and at 144 frames per second.

The first value it sees is taken as it is, so nothing glides in from 0. A boolean counts as 0 or 1: lag(mouseIsPressed, 0.2) is a soft switch.

Before you call it

Nothing to call or declare: write it in draw(). The first call loads @openav/chop from next to oav-p5.js, which takes a few frames.

Parameters

NameTypeDescription
valueNumber | BooleanThe number to follow. A boolean counts as 0 or 1. Anything that is not a finite number (a string, NaN) goes straight through and the lag keeps what it had.
upNumberSeconds to follow a rise: about how long the output takes to cover 90% of a jump upward. Negative numbers count as 0. optional default 0.1
downNumberSeconds to follow a fall. Left out, it is the same as up. optional default up
nameString | NumberA name for this call's memory, written as the last argument. Optional. Without one, oav-p5 tells calls apart by their order in draw(); with one, the call is the same wherever it is made. optional read from arguments

Returns

Number The smoothed value. It returns value as it is until @openav/chop has loaded (a few frames), and for an unnamed call outside draw().

Notes

  • Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
  • A sketch with its own variable or function called lag keeps it. OAVp5.chop.lag(…) is this function under another name.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const x = lag(mouseX, 0.5);   // follows the mouse, half a second behind
  circle(x, height / 2, 40);
}
A circle that chases the mouse: it never jumps, it glides, and takes about half a second to catch up.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const open = lag(mouseIsPressed, 0.15, 0.8);   // a boolean counts as 0 or 1
  fill(255, 211, 107);
  circle(width / 2, height / 2, 40 + open * 240);
}
Hold the mouse down: the circle swells quickly and, when you let go, shrinks back slowly.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
}

function draw() {
  background(20);
  for (let i = 0; i < 6; i++) {
    const seconds = 0.1 + i * 0.12;
    const x = lag(mouseX, seconds, seconds, 'dot' + i);   // a name per dot
    fill(255, 211, 107);
    circle(x, 40 + i * 44, 24);
  }
}
Six dots follow the mouse, each a little slower than the one above. The names keep each dot's memory its own even if you later add or remove dots.

Related

  • spring() Follows value like a mass on a spring: it overshoots, bounces and settles (TouchDesigner's Spring CHOP).
  • env() A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.
  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • knob1 The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
  • trail() The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).
  • scope() Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 3008

scope()#

functionoav-p5
scope(value, [label])

Also: OAVp5.chop.scope

Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.

It is a CHOP's viewer: each label is one line in a small plot across the page from the 🎹, with the last 4 seconds of the value and its number now. Call it twice in a frame with two labels and you get two lines to compare, raw against smoothed. The scale is at least 0 to 1 and grows to hold whatever the values reach.

It lives on the page, not on the canvas: never in a recording or in 📷, and it never takes a click from the sketch. To draw the same line inside the work, use drawTrail().

Before you call it

Only inside draw(): anywhere else it just returns value. ?oav_scope=off hides it, and so does ?oav_hints=off, which is the one thing to know if it does not appear. See scope. The first call loads @openav/chop, which takes a few frames.

Parameters

NameTypeDescription
valueNumber | BooleanThe number to watch. A boolean is drawn as 0 or 1. Anything that is not a finite number is returned as it is and not drawn.
labelStringThe name of this line in the legend. Each label is one line, and a label longer than 9 characters is cut short there. Without one the line is numbered. optional

Returns

Number | Boolean Exactly value, so circle(x, y, scope(lag(micLevel, 0.03, 0.4), 'lag') * 300) works.

Notes

  • Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
  • OAVp5.chop.scope(…) is this function under another name. The lines keep coming from the last second: stop calling it for a second and its line leaves the plot.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const wave = 0.5 + 0.5 * sin(frameCount * 0.05);
  circle(width / 2, height / 2, 40 + scope(wave, 'wave') * 200);   // watch it in the corner
}
The circle breathes, and a small plot in the corner of the page draws the same number as a moving line.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const raw = frameCount % 120 < 60 ? 1 : 0;   // a square wave
  scope(raw, 'raw');
  const smooth = scope(lag(raw, 0.3, 0.6), 'lag');
  fill(255, 211, 107);
  circle(width / 2, height / 2, 40 + smooth * 200);
}
The corner plot shows two lines: the square wave and what lag() makes of it, rising in 0.3 seconds and falling in 0.6.

Related

  • drawTrail() Draws a line through values on the canvas, so it is in the work and in a recording: the oldest at x, lo at the bottom of the box, hi at the top.
  • trail() The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
  • scope Whether scope() draws its oscilloscope in the corner of the page: on or off. hints=off turns it off too.
  • hints Whether oav-p5 shows its explaining cards beside the 🎹 (blocked microphone, no Web MIDI, an error in your callback): on or off.

p5/oav-p5.js, line 3072

slope()#

functionoav-p5
slope(value, [name])

Also: OAVp5.chop.slope

How fast value is changing, per second: positive while it rises, negative while it falls, 0 while it holds still (TouchDesigner's Slope CHOP).

Ask how hard something is moving rather than where it is: abs(slope(mouseX)) is the mouse speed in pixels per second, and slope(micLevel) jumps up at the start of a word. It is as noisy as its input, so smooth it when it flickers: lag(abs(slope(x)), 0.05, 0.3).

Before you call it

Nothing to call or declare: write it in draw(). The first call loads @openav/chop, which takes a few frames.

Parameters

NameTypeDescription
valueNumber | BooleanThe number to watch. A boolean counts as 0 or 1. Anything that is not a finite number keeps the last slope.
nameString | NumberA name for this call's memory, written as the last argument. Optional. Without one, oav-p5 tells calls apart by their order in draw(); with one, the call is the same wherever it is made. optional read from arguments

Returns

Number The change per second, in the units of value (pixels per second for a pixel position). 0 on the first call, and 0 until @openav/chop has loaded or for an unnamed call outside draw().

Notes

  • Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
  • OAVp5.chop.slope(…) is this function under another name.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textSize(16);
}

function draw() {
  background(20);
  const v = abs(slope(mouseX));   // pixels per second
  fill(255);
  text(nf(v, 1, 0) + ' px/s', 20, 120);
  rect(20, 140, min(v / 10, width - 40), 24);
}
Move the mouse sideways: the bar and the number show how fast you are going, and drop to 0 when you stop.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const r = spring(mouseIsPressed ? 140 : 60, 100, 6);   // a radius that bounces
  const moving = lag(abs(slope(r)), 0.05, 0.3);          // how fast it is changing, smoothed
  noFill();
  stroke(255, 211, 107);
  strokeWeight(2 + min(14, moving / 40));
  circle(width / 2, height / 2, r * 2);
}
Hold the mouse: the ring grows and bounces, and its outline is thickest while it is changing fastest.

Related

  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
  • speed() Adds rate to a running total every second, so rotate(speed(TWO_PI)) turns once a second and the rate can change without the total ever jumping.
  • spring() Follows value like a mass on a spring: it overshoots, bounces and settles (TouchDesigner's Spring CHOP).
  • trail() The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 3052

speed()#

functionoav-p5
speed(rate, [name])

Also: OAVp5.chop.speed

Adds rate to a running total every second, so rotate(speed(TWO_PI)) turns once a second and the rate can change without the total ever jumping.

Use the total to turn or move something: rotate(speed(TWO_PI)) is one turn a second, x = speed(60) % width slides 60 pixels a second and wraps around. You pass the rate, not the position, so the rate can follow a knob or the microphone (speed(micLevel * TWO_PI) spins faster the louder it gets) and the angle never jumps, which frameCount * rate would do.

Time is in seconds, the same at any frame rate. One frame counts for at most a quarter of a second, so coming back to a hidden tab does not make the shape leap.

Before you call it

Nothing to call or declare: write it in draw(). The first call loads @openav/chop, which takes a few frames.

Parameters

NameTypeDescription
rateNumber | BooleanHow much to add per second, in the unit you use the result in (radians for rotate(), pixels for a position). It may change every frame. A boolean counts as 0 or 1.
nameString | NumberA name for this call's memory, written as the last argument. Optional. Without one, oav-p5 tells calls apart by their order in draw(); with one, the call is the same wherever it is made. optional read from arguments

Returns

Number The running total, 0 at the start. It is 0 until @openav/chop has loaded (a few frames), and for an unnamed call outside draw().

Notes

  • Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
  • Plenty of sketches have a variable called speed. If yours does, it keeps the name, and OAVp5.chop.speed(…) is this function under another name (third example).

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  rectMode(CENTER);
}

function draw() {
  background(20);
  translate(width / 2, height / 2);
  rotate(speed(TWO_PI));   // one turn a second
  rect(0, 0, 120, 120);
}
A square that turns once a second, steadily.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  rectMode(CENTER);
}

function draw() {
  background(20);
  const rate = map(mouseX, 0, width, -TWO_PI, TWO_PI);   // turns per second, left to right
  translate(width / 2, height / 2);
  rotate(speed(rate));
  rect(0, 0, 120, 120);
}
Move the mouse: the left side spins the square one way, the middle stops it, the right side spins it the other way, and it never jumps.
jsOpenProcessing ↗
let speed = 2;   // the sketch's own variable called speed

function setup() {
  createCanvas(400, 300);
  rectMode(CENTER);
}

function draw() {
  background(20);
  translate(width / 2, height / 2);
  rotate(OAVp5.chop.speed(speed));   // the same function, under another name
  rect(0, 0, 120, 120);
}
The sketch keeps its own speed (2 radians a second) and reaches the function as OAVp5.chop.speed.

Related

  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
  • spring() Follows value like a mass on a spring: it overshoots, bounces and settles (TouchDesigner's Spring CHOP).
  • slope() How fast value is changing, per second: positive while it rises, negative while it falls, 0 while it holds still (TouchDesigner's Slope CHOP).
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 3034

spring()#

functionoav-p5
spring(value, [stiffness], [damping], [name])

Also: OAVp5.chop.spring

Follows value like a mass on a spring: it overshoots, bounces and settles (TouchDesigner's Spring CHOP).

Wrap a target that jumps: circle(width / 2, height / 2, spring(mouseIsPressed ? 300 : 100, 100, 6)). The size does not just glide to the new target (that is lag()): it swings past it, comes back and settles.

Stiffness and damping are the two knobs. A stiff, lightly damped spring rings like a ruler on a table. A soft, heavily damped one just eases. A big jump or a low frame rate does not break it: the motion is the same at any frame rate.

Before you call it

Nothing to call or declare: write it in draw(). The first call loads @openav/chop, which takes a few frames.

Parameters

NameTypeDescription
valueNumber | BooleanThe target the spring is pulled toward. A boolean counts as 0 or 1.
stiffnessNumberHow fast it swings: 100 swings about 1.6 times a second. Higher is snappier. Negative numbers count as 0. optional default 100
dampingNumberHow soon it settles: 6 gives two or three bounces, 20 and up none. The edge where it stops bouncing is 2 * sqrt(stiffness). optional default 6
nameString | NumberA name for this call's memory, written as the last argument. Optional. Without one, oav-p5 tells calls apart by their order in draw(); with one, the call is the same wherever it is made. optional read from arguments

Returns

Number The spring's position now. The first value it sees is taken as it is. It returns value as it is until @openav/chop has loaded (a few frames), and for an unnamed call outside draw().

Notes

  • Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
  • OAVp5.chop.spring(…) is this function under another name.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const size = spring(mouseIsPressed ? 300 : 100, 100, 6);   // overshoots, then settles
  circle(width / 2, height / 2, size);
}
Hold the mouse: the circle springs open, overshoots and settles. Let go and it bounces back down.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const x = spring(mouseX, 60, 4);   // soft and bouncy
  const y = spring(mouseY, 60, 4);
  circle(x, y, 40);
}
A circle hung on the mouse by a soft spring: it swings around the pointer instead of sticking to it.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
}

function draw() {
  background(20);
  const target = mouseIsPressed ? 340 : 60;
  const dampings = [2, 6, 20];
  for (let i = 0; i < 3; i++) {
    const x = spring(target, 100, dampings[i]);   // same pull, three dampings
    circle(x, 60 + i * 90, 36);
  }
}
Hold the mouse: three circles are pulled right by the same spring with different damping. The top one bounces many times, the middle one a few, the bottom one not at all.

Related

  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
  • speed() Adds rate to a running total every second, so rotate(speed(TWO_PI)) turns once a second and the rate can change without the total ever jumping.
  • slope() How fast value is changing, per second: positive while it rises, negative while it falls, 0 while it holds still (TouchDesigner's Slope CHOP).
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 3039

trail()#

functionoav-p5
trail(value, [seconds], [name])

Also: OAVp5.chop.trail

The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).

A history to draw or measure: drawTrail(trail(micLevel, 3), 20, 20, width - 40, 100) plots the last three seconds of the microphone level, and Math.max(...trail(x, 2)) is a two-second peak. The values are sampled 60 times a second whatever the frame rate, and the gaps between two frames are interpolated, so 30 fps and 144 fps give the same shape.

Change seconds between frames and the history is resized, keeping the newest values. The array you got last frame is never changed afterwards.

Before you call it

Nothing to call or declare: write it in draw(). The first call loads @openav/chop, which takes a few frames.

Parameters

NameTypeDescription
valueNumber | BooleanThe number to record. A boolean counts as 0 or 1. Anything that is not a finite number is skipped.
secondsNumberHow many seconds of history to keep, from 0.05 to 600. optional default 2
nameString | NumberA name for this call's memory, written as the last argument. Optional. Without one, oav-p5 tells calls apart by their order in draw(); with one, the call is the same wherever it is made. optional read from arguments

Returns

Array<Number> About seconds * 60 + 1 numbers (121 for 2 seconds), a new array every call. The length is fixed from the first call and filled with the first value, so you can draw it from frame one. It is an empty array until @openav/chop has loaded, and for an unnamed call outside draw().

Notes

  • Unnamed calls find their memory by their order in draw(): the first lag() is the first lag, the second is the second. Put one inside an if, or in a loop whose length changes, and the order slips: a card says so, and from then on oav-p5 matches each call by the line it is written on. A name at the end is the sure way. Outside draw() an unnamed call has nothing to remember by: it returns what Returns says and a card says so.
  • OAVp5.chop.trail(…) is this function under another name. To see the line on the canvas use drawTrail(), to see it in the corner of the page use scope().

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const wave = 0.5 + 0.5 * sin(frameCount * 0.05);   // 0..1
  stroke(255);
  drawTrail(trail(wave, 3), 20, 100, width - 40, 100);   // the last 3 seconds
}
A sine wave drawn from its own history: it scrolls to the left as new values arrive at the right.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
}

function draw() {
  background(20);
  const t = trail(mouseX, 1);   // where the mouse was during the last second
  for (let i = 0; i < t.length; i += 6) {
    fill(255, 211, 107, 10 + (i / t.length) * 200);
    circle(t[i], mouseY, 24);
  }
}
A ghost trail behind the pointer: the older the copy, the fainter it is.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const t = trail(mouseX, 3);   // the last 3 seconds of the mouse position
  const left = Math.min(...t), right = Math.max(...t);
  noStroke();
  fill(255, 211, 107, 90);
  rect(left, 0, right - left, height);   // the band the mouse covered
  stroke(255);
  drawTrail(t, 0, 20, width, 60, 0, width);
}
A band shows how far left and right the mouse went in the last three seconds, with the trail itself drawn across the top.

Related

  • drawTrail() Draws a line through values on the canvas, so it is in the work and in a recording: the oldest at x, lo at the bottom of the box, hi at the top.
  • scope() Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
  • slope() How fast value is changing, per second: positive while it rises, negative while it falls, 0 while it holds still (TouchDesigner's Slope CHOP).
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 3050

trigger()#

functionoav-p5
trigger([name])

Also: OAVp5.chop.trigger

Fires an event of your own, by name, for env() and count() to react to.

Call it where the thing happens: function mousePressed() { trigger('boom'); }. Then read it anywhere in draw() with env('boom', 0.01, 0.6) (a pulse that fades) or count('boom') (how many times so far). It returns nothing.

It works anywhere, not only in draw(), and each call is one event, remembered with the time it happened. You can fire the built-in names by hand as well: trigger('kick') pulses env('kick') as a real kick would.

Parameters

NameTypeDescription
nameStringThe name of the event. Anything is fine except the built-in meanings: 'kick', 'snare', 'hat', 'beat', 'note', 'bar' and 'step' are also fired by the microphone, the keys and the drum machine. Left out: 'trigger'. optional default 'trigger'

Notes

  • OAVp5.chop.trigger(…) is this function under another name.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  trigger('boom');   // an event of our own
}

function draw() {
  background(20);
  const k = env('boom', 0.01, 0.6);
  circle(width / 2, height / 2, 60 + k * 220);
}
Click the canvas: each click fires "boom" and the circle pulses once.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  noStroke();
}

function mousePressed() { trigger('boom'); }
function keyPressed() { trigger('spark'); }

function draw() {
  background(20);
  fill(255, 90, 31);
  circle(120, 150, 40 + env('boom', 0.01, 0.6) * 160);
  fill(120, 190, 255);
  circle(280, 150, 40 + env('spark', 0.005, 0.2) * 160);
}
Click for the orange circle, press any key for the blue one: each reacts only to its own event.

Related

  • env() A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.
  • count() Counts how often on has turned on, or how many 'kick' (or any other event) have happened; with a limit it loops 0, 1, … limit − 1.
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 3032

OAVp5.chop#

objectoav-p5
OAVp5.chop

The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

Every function of this group is also here: OAVp5.chop.lag, .env, .trigger, .speed, .spring, .trail, .slope, .count, .scope and .drawTrail are the same functions as the plain names, for a sketch whose own speed or count keeps the plain name.

The rest is for looking inside: state says whether the module is loaded, module is the module itself, slots() lists every memory in use and reset() forgets them.

Notes

  • This is advanced: a sketch that only calls lag() and friends never needs it.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const x = OAVp5.chop.lag(mouseX, 0.4);   // the same as lag(mouseX, 0.4)
  fill(255);
  text('chop ' + OAVp5.chop.state + ', ' + OAVp5.chop.slots().length + ' memory in use', 20, 30);
  circle(x, height / 2, 40);
}
A circle chasing the mouse through OAVp5.chop.lag, with the module's state and the number of memories in use printed above it.

Related

  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
  • env() A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.
  • OAVp5.chop.state Where the @openav/chop module is: 'off', 'loading', 'on' or 'error'.
  • OAVp5.chop.slots() Lists every memory the CHOP functions are using right now: its kind, how it is found, and its latest value.
  • OAVp5.chop.reset() Forgets the memory named name (a lag, a speed total, a count …), or every memory when you give no name: the next call starts fresh.

p5/oav-p5.js, line 2861

OAVp5.chop.module#

variableoav-p5
OAVp5.chop.module

The @openav/chop module itself once it has loaded, otherwise null: the classes and functions the CHOP functions are built on.

For reaching below the p5-style calls: OAVp5.chop.module.DEFAULTS lists the defaults the functions use, and new OAVp5.chop.module.Lag({ up: 0.1 }) makes a lag you keep yourself. Check it is not null first.

Type
Object | null

null until the first CHOP function has been called and the module has arrived, then the module (Lag, Spring, Trail, envelope(), DEFAULTS and the rest).

Notes

  • This is advanced. The module is also published as @openav/chop for sketches and shows that do not go through oav-p5.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  lag(mouseX, 0.3, 0.3, 'warm-up');   // the first call loads the module
  const m = OAVp5.chop.module;
  fill(255);
  text(m ? 'spring defaults: ' + JSON.stringify(m.DEFAULTS.spring) : 'not loaded yet', 20, 30);
}
Prints the spring defaults from the module as soon as it has loaded.

Related

  • OAVp5.chop.state Where the @openav/chop module is: 'off', 'loading', 'on' or 'error'.
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 2842

OAVp5.chop.reset()#

functionoav-p5
OAVp5.chop.reset([name])

Forgets the memory named name (a lag, a speed total, a count …), or every memory when you give no name: the next call starts fresh.

The call that follows starts from nothing, as if it were the first one: a speed() total is 0 again, a lag() takes its input as it is, a count() is back at 0. Use it to restart a named effect, for example on a key press.

Calling it with no name also drops the unnamed memories. They are made again by order on the next frame, so nothing breaks, but every unnamed value starts over too. Event counts from trigger() are not memories of a call and are not touched.

Parameters

NameTypeDescription
nameStringThe name you gave a call as its last argument ('spin' in speed(1, 'spin')). The memory of that name is forgotten in every function that has one. Left out, every memory is forgotten, named or not. optional

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  OAVp5.chop.reset('spin');   // forget the named speed
}

function draw() {
  background(20);
  translate(width / 2, height / 2);
  rotate(speed(1, 'spin'));
  rect(-40, -40, 80, 80);
}
The square turns steadily. Click: it jumps back to its starting angle and carries on from there.

Related

  • OAVp5.chop.slots() Lists every memory the CHOP functions are using right now: its kind, how it is found, and its latest value.
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
  • speed() Adds rate to a running total every second, so rotate(speed(TWO_PI)) turns once a second and the rate can change without the total ever jumping.
  • count() Counts how often on has turned on, or how many 'kick' (or any other event) have happened; with a limit it loops 0, 1, … limit − 1.

p5/oav-p5.js, line 2855

OAVp5.chop.slots()#

functionoav-p5
OAVp5.chop.slots()

Lists every memory the CHOP functions are using right now: its kind, how it is found, and its latest value.

For finding out why two calls seem to share a memory, or whether a loop made hundreds: OAVp5.chop.slots().length is how many there are. It reads the state and changes nothing.

Returns

Array<Object> One { kind, key, mode, value } per memory. kind is the function ('lag', 'spring', …). key is '#1', '#2' for calls found by order, a name for named ones, or the line of code for calls matched by line. mode says which: 'order', 'name' or 'site'. value is its latest output (missing for a trail).

Notes

  • This is advanced. 🩺 and tests use it; a sketch rarely needs to.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textSize(14);
}

function draw() {
  background(20);
  lag(mouseX, 0.3);
  lag(mouseY, 0.3);
  spring(mouseX, 100, 6, 'ball');
  fill(255);
  const lines = OAVp5.chop.slots().map((s) => s.kind + ' ' + s.key + ' (' + s.mode + ')');
  text(lines.join('\n'), 20, 30);
}
Lists the memories the three calls are using: two lags found by order and one spring found by its name.

Related

  • OAVp5.chop.reset() Forgets the memory named name (a lag, a speed total, a count …), or every memory when you give no name: the next call starts fresh.
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).

p5/oav-p5.js, line 2844

OAVp5.chop.state#

variableoav-p5
OAVp5.chop.state

Where the @openav/chop module is: 'off', 'loading', 'on' or 'error'.

off until one of these functions is first called, loading while @openav/chop arrives (a few frames), on once it is in, error if it could not be loaded: a card says so and every function then passes its input through. trigger() on its own never loads it, because it only records that something happened.

Type
String
Range
'off' | 'loading' | 'on' | 'error'

Changes the first time a CHOP function is called and once more when the module has arrived. off before any call.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('chop: ' + OAVp5.chop.state, 20, 30);   // off, then loading, then on
  if (frameCount > 30) circle(lag(mouseX, 0.3), height / 2, 30);   // the first lag() loads the module
}
For half a second it says off. The first lag() call then loads the module: loading, then on.

Related

  • OAVp5.chop.module The @openav/chop module itself once it has loaded, otherwise null: the classes and functions the CHOP functions are built on.
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).

p5/oav-p5.js, line 2841

@openav/chop

approach()#

function@openav/chop
approach(current, target, seconds, dt)

One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.

The smallest piece of smoothing. Call it once per frame, value = approach(value, target, 0.2, dt), and value chases target: 63% of the way after seconds, 90% after 2.3 × seconds, 99% after 4.6 × seconds. It is exact for any dt, so a sketch at 30 fps and one at 144 fps draw the same curve.

seconds here is a time constant, which is shorter than the "seconds" of TouchDesigner's Lag CHOP. Lag takes the TouchDesigner kind and divides it by TD_LAG_RATIO before calling this function. @openav/mapping's smooth steps with approach and means the time constant too.

Before you call it

Nothing. It is pure arithmetic with no memory: you keep current yourself and pass in the seconds since the last call. For a value with its own memory use Lag.

Parameters

NameTypeDescription
currentNumberWhere the value is now.
targetNumberWhere it is heading.
secondsNumberThe time constant: after this many seconds the gap to target has closed by 63% (1 − 1/e). 0, a negative number or NaN jumps straight to target, whatever dt is.
dtNumberSeconds since the last step. 0, a negative number or NaN leaves current where it is.

Returns

Number The new value, between current and target.

Examples

js
import { approach } from '@openav/chop';

let value = 0;
const seen = [];
for (let frame = 0; frame < 6; frame++) {
  value = approach(value, 1, 0.1, 1 / 60);   // chase 1, time constant 0.1 s
  seen.push(value.toFixed(3));
}
document.getElementById('app').textContent = seen.join('  ');
module Six frames of a value chasing 1: it covers about 15% of the remaining gap each frame (0.154, 0.283, 0.393 …).
js
import { approach } from '@openav/chop';

// one second of time constant: the gap is 63% closed after 1 s, 90% after 2.3 s
const seconds = 1;
const after = (t) => approach(0, 1, seconds, t).toFixed(3);
document.getElementById('app').textContent =
  `after 1 s: ${after(1)}   after 2.3 s: ${after(2.3026)}   after 4.6 s: ${after(4.6052)}`;
module Prints 0.632, 0.900 and 0.990: the three numbers worth remembering.

Related

  • Lag Follows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.
  • TD_LAG_RATIO Math.LN10, about 2.303: how many time constants a lag needs to cover 90% of a jump, which is what TouchDesigner's Lag seconds mean.
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).

import { approach } from '@openav/chop'; packages/chop/index.js, line 38

Count#

class@openav/chop
new Count({ threshold = 0.5 } = {})

Counts how many times its input turned on (true, or a number above threshold); with a limit it loops 0 … limit − 1. The Count CHOP.

count.step(input, limit) counts a rising edge, the moment the input goes from off to on, not the time it stays on. Hold a button for ten seconds and that is one. With limit 4 the result walks 0 1 2 3 0 1 …, which is how a beat becomes "beat 1 of 4".

The counter keeps counting underneath even when you pass a limit, so you can change the limit later without losing your place. A value that is not a number counts as off.

In a p5 sketch use count(): count(beat, 4), or count('kick') to count the drum machine's kicks.

Before you call it

Nothing to load. Call step every frame, even while the input is off: it has to see the "off" to notice the next "on". It has no dt, because a count does not depend on time.

Parameters

NameTypeDescription
thresholdNumberA number counts as "on" when it is above this. A boolean true is 1, false is 0. optional default 0.5

Members

NameDescription
step(input, limit = 0)methodstep(input, limit?): look at the input and return the count, looped to 0 … limit − 1 if limit is above 0 (a fraction is rounded down).
reset(value = 0)methodreset(value?): set the counter to value (0 if left out; a fraction is cut to a whole number) and forget whether the input was on. Returns the Count.

Notes

  • Public fields: threshold, on (is the input on right now) and total (the count before looping).

Examples

js
import { Count } from '@openav/chop';

const count = new Count();
const inputs = [0, 1, 1, 0, 1, true, false, true];
document.getElementById('app').textContent =
  inputs.map((x) => count.step(x)).join('  ');
module Prints 0 1 1 1 2 2 2 3: it counts each time the input goes from off to on, and holding it on adds nothing.
js
import { Count } from '@openav/chop';

const count = new Count();
const beats = [];
for (let i = 0; i < 12; i++) beats.push(count.step(i % 2, 4));   // off, on, off, on …
document.getElementById('app').textContent = beats.join('  ');
module With a limit of 4 the count loops: 0 1 1 2 2 3 3 0 0 1 1 2.

Related

  • count() Counts how often on has turned on, or how many 'kick' (or any other event) have happened; with a limit it loops 0, 1, … limit − 1.
  • wrapCount() Loops a count to 0 … limit − 1: wrapCount(5, 4) is 1. A limit of 0 or less leaves the count as it is.
  • envelope() The level, 0 to peak, of an attack-and-decay envelope at time t, for events that happened at the given times; the Trigger CHOP.

import { Count } from '@openav/chop'; packages/chop/index.js, line 225

DEFAULTS#

constant@openav/chop
DEFAULTS = object, 5 keys

The default settings of every CHOP in the package, as one frozen object: lag, envelope, spring, trail and count.

For documentation and for interfaces that show the defaults, such as the placeholder text of a field. The classes and envelope() have these same numbers written in their own signatures; changing this object changes nothing about how they behave.

Type
Object

{ lag: { up: 0.1, down: 0.1 }, envelope: { attack: 0.01, decay: 0.3 }, spring: { stiffness: 100, damping: 6 }, trail: { seconds: 2, rate: 60 }, count: { threshold: 0.5 } }. Frozen: assigning to it does nothing.

Examples

js
import { DEFAULTS } from '@openav/chop';

document.getElementById('app').textContent = JSON.stringify(DEFAULTS.spring);
module Prints {"stiffness":100,"damping":6}: what a Spring does when you give it no settings.

Related

  • Lag Follows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.
  • Spring Follows its input like a mass on a spring: it overshoots, swings back and settles; the Spring CHOP with the input as position.
  • Trail Remembers the input over the last few seconds as a list of values, oldest first, at a fixed 60 values a second; the Trail CHOP.
  • Count Counts how many times its input turned on (true, or a number above threshold); with a limit it loops 0 … limit − 1. The Count CHOP.
  • envelope() The level, 0 to peak, of an attack-and-decay envelope at time t, for events that happened at the given times; the Trigger CHOP.

import { DEFAULTS } from '@openav/chop'; packages/chop/index.js, line 243

envelope()#

function@openav/chop
envelope(times, t, attack = 0.01, decay = 0.3, peak = 1)

The level, 0 to peak, of an attack-and-decay envelope at time t, for events that happened at the given times; the Trigger CHOP.

Keep a list of the times something happened (a kick, a key press, a cue) and ask for the level at the time you are drawing: scale(1 + envelope(kicks, now, 0.01, 0.3)). It rises in a straight line to 1 in attack seconds, then falls to 0 in decay seconds, quick at first and landing softly.

It is a pure function of the times and of t: ask twice, or at 30 fps and at 144 fps, and the same moment gives the same level; attack and decay may change every frame (a knob on the decay). A new event while the last one still sounds restarts from where the envelope is, never from 0, so fast hits pile up into a plateau instead of flickering. The attack counts from 0 to the peak, so a retrigger from halfway up takes half the attack.

In a p5 sketch you do not import it: env() and trigger() keep the event list for you and read the drum machine and the microphone.

Before you call it

Nothing to load. You record the event times yourself (for example kicks.push(performance.now() / 1000) when a kick arrives) and pass the same clock as t.

Parameters

NameTypeDescription
timesNumber[]When the events happened, in seconds on any clock you like, oldest first. An empty array (or none) gives 0. Events later than t have not happened yet and are ignored. Only the last few matter, so you can trim the list.
tNumberThe moment to read, on the same clock as times.
attackNumberSeconds to rise from 0 to the peak. 0 starts at the peak at once. optional default 0.01
decayNumberSeconds to fall back to 0 after the peak: it is exactly 0 from attack + decay seconds after an event. optional default 0.3
peakNumberThe top of the envelope. optional default 1

Returns

Number 0 to peak: peak just after an event, 0 once it has died away.

Examples

js
import { envelope } from '@openav/chop';

const kicks = [1];   // one kick, at 1 s
const times = [0.9, 1, 1.005, 1.01, 1.1, 1.2, 1.31];
document.getElementById('app').innerHTML = times
  .map((t) => `${t} s: ${envelope(kicks, t).toFixed(2)}`)
  .join('<br>');
module Before the kick 0.00, half way up the attack 0.50 at 1.005 s, the peak 1.00 at 1.01 s, then 0.49, 0.13 and 0.00 at 1.31 s.
js
import { envelope } from '@openav/chop';

const ctx = document.getElementById('c').getContext('2d');
const kicks = [];
let nextKick = 0;

function frame(ms) {
  const t = ms / 1000;
  if (t >= nextKick) { kicks.push(nextKick); nextKick += 0.5; }   // a kick every half second
  const level = envelope(kicks, t, 0.01, 0.3);
  ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
  ctx.fillStyle = '#fc4';
  ctx.beginPath(); ctx.arc(200, 150, 40 + level * 80, 0, Math.PI * 2); ctx.fill();
  requestAnimationFrame(frame);
}
requestAnimationFrame(frame);
module A disc that swells on every kick (twice a second) and shrinks back within 0.3 s.
js
import { envelope } from '@openav/chop';

// three hits 50 ms apart pile up; one lone hit at the same moment is much lower
const pile = envelope([0, 0.05, 0.1], 0.11);
const lone = envelope([0], 0.11);
document.getElementById('app').textContent =
  `pile-up: ${pile.toFixed(2)}   lone hit: ${lone.toFixed(2)}`;
module Prints "pile-up: 0.95 lone hit: 0.44": each new hit restarts from where the envelope is, so quick hits hold it high.

Related

  • envelopeStage() The level of one envelope elapsed seconds after its trigger: a straight rise to peak, then a fall to 0 in decay seconds.
  • env() A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.
  • Lag Follows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.
  • Count Counts how many times its input turned on (true, or a number above threshold); with a limit it loops 0 … limit − 1. The Count CHOP.

import { envelope } from '@openav/chop'; packages/chop/index.js, line 86

envelopeStage()#

function@openav/chop
envelopeStage(elapsed, attack, decay, from = 0, peak = 1)

The level of one envelope elapsed seconds after its trigger: a straight rise to peak, then a fall to 0 in decay seconds.

The building block of envelope(), which strings several of these together for a list of events. On its own it answers: "how high is a kick that happened elapsed seconds ago?" With the defaults of envelope() (attack 0.01, decay 0.3) it is 0.5 at 5 ms, 1 at 10 ms, 0.49 at 100 ms and 0 from 310 ms on.

The fall is quick at first and lands softly: that is the shape a kick drum swell looks right with. It reaches 0 in exactly decay seconds, not asymptotically.

Parameters

NameTypeDescription
elapsedNumberSeconds since the trigger. Below 0 (or NaN) gives from.
attackNumberSeconds to rise from 0 to peak. 0 or less starts at the peak at once.
decayNumberSeconds to fall from peak to 0, along (1 − u)² where u runs 0 to 1. 0 or less drops to 0 right after the attack.
fromNumberThe level at the moment of the trigger. The attack still counts from 0 to the peak, so starting from halfway up takes half the attack. optional default 0
peakNumberThe top of the envelope. optional default 1

Returns

Number The level, from 0 to peak.

Notes

  • Advanced. Most sketches never call it: use envelope(), or env() in p5.

Examples

js
import { envelopeStage } from '@openav/chop';

const row = [0, 0.005, 0.01, 0.1, 0.16, 0.31].map(
  (t) => `${t} s: ${envelopeStage(t, 0.01, 0.3).toFixed(2)}`
);
document.getElementById('app').textContent = row.join('   ');
module Prints the level at six moments after a trigger: 0.00, 0.50, 1.00, 0.49, 0.25, 0.00.

Related

  • envelope() The level, 0 to peak, of an attack-and-decay envelope at time t, for events that happened at the given times; the Trigger CHOP.
  • env() A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.
  • Lag Follows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.

import { envelopeStage } from '@openav/chop'; packages/chop/index.js, line 70

Lag#

class@openav/chop
new Lag({ up = 0.1, down = up, value = null } = {})

Follows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.

Make one Lag per value you want to smooth, then call step(input, dt) once per frame. new Lag({ up: 0.03, down: 0.4 }) is a VU meter: it opens almost at once and closes slowly. up and down are in TouchDesigner's seconds: with up: 1, a jump from 0 to 1 is at 0.9 one second later, at 30, 60 or 144 fps alike.

Each step is the exact answer for an input held at its new value over dt seconds, so the frame rate does not change the curve. A boolean counts as 0 or 1 (lag.step(mouseIsPressed, dt) is a soft switch). A value that is not a number, such as NaN, comes back untouched and leaves the memory as it was.

In a p5 sketch you do not import this: lag() keeps one Lag per call for you.

Before you call it

Nothing to load or start. Pass the seconds since the previous frame as dt (in a browser: the difference of two performance.now() / 1000). The first step only stores the input, so the output equals the input on frame one.

Parameters

NameTypeDescription
upNumberSeconds to follow a rise, while the input is above the output. TouchDesigner's seconds: about the time to cover 90% of a jump. 0 follows at once. optional default 0.1
downNumberSeconds to follow a fall, while the input is below the output. Leave it out and it is the same as up. optional default up
valueNumberA starting value. Leave it out and the first input is taken as it is, so nothing glides in from 0. optional default null

Members

NameDescription
step(input, dt, up = this.up, down = this.down)methodstep(input, dt, up?, down?): advance by dt seconds toward input and return the new output. up and down override the constructor's for this call only, so a knob can change them every frame. A non-number input is returned as it is.
reset(value = null)methodreset(value?): forget the memory. With no argument the next input is taken as it is again; with a number the output restarts from there. Returns the Lag, so calls chain.

Notes

  • Public fields: up, down and value, the last output (null until the first step). You can change up and down any time.
  • The time constant underneath is seconds / TD_LAG_RATIO (see TD_LAG_RATIO), not seconds. If you want the plain 63% kind, call approach().

Examples

js
import { Lag } from '@openav/chop';

const lag = new Lag({ up: 0.03, down: 0.4 });   // opens fast, closes slowly
const seen = [];
for (let frame = 0; frame < 12; frame++) {
  const input = frame >= 2 && frame < 6 ? 1 : 0;   // on for four frames
  seen.push(lag.step(input, 1 / 60).toFixed(2));
}
document.getElementById('app').textContent = seen.join('  ');
module The output shoots up within two frames when the input turns on, then sags back slowly after it turns off (0.00 0.00 0.72 0.92 0.98 0.99 0.90 0.82 …).
js
import { Lag } from '@openav/chop';

const ctx = document.getElementById('c').getContext('2d');
ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
ctx.lineWidth = 2;
const input = (t) => (t < 0.25 || t > 1.25 ? 0 : 1);   // a pulse, 1 s long

for (const [fps, color] of [[30, '#555'], [30, '#fa4'], [144, '#4af']]) {
  const lag = new Lag({ up: 0.1, down: 0.4 });
  ctx.strokeStyle = color; ctx.beginPath();
  for (let k = 0; k <= fps * 2; k++) {
    const t = k / fps;
    const y = color === '#555' ? input(t) : lag.step(input(t), k ? 1 / fps : 0);
    ctx.lineTo(20 + t * 180, 260 - y * 220);
  }
  ctx.stroke();
}
module A one-second pulse (grey) through a Lag at 30 fps (orange) and at 144 fps (blue): it rises in about 0.1 s, falls over about 0.4 s, and the two curves lie on top of each other.
js
import { Lag } from '@openav/chop';

const lag = new Lag();                       // up and down are both 0.1
lag.step(0.5, 0);                            // the first input is taken as it is
const nan = lag.step(NaN, 1 / 60);           // not a number: passed through, memory kept
const kept = lag.value;                      // still 0.5
const toward = lag.step(true, 1 / 60);       // a boolean counts as 1, so it moves up from 0.5
document.getElementById('app').textContent =
  `NaN in: ${nan}, memory still ${kept}, true in: ${toward.toFixed(2)}`;
module Prints "NaN in: NaN, memory still 0.5, true in: 0.66": a non-number leaves the memory alone, and a boolean counts as 0 or 1.

Related

  • approach() One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.
  • TD_LAG_RATIO Math.LN10, about 2.303: how many time constants a lag needs to cover 90% of a jump, which is what TouchDesigner's Lag seconds mean.
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).
  • Spring Follows its input like a mass on a spring: it overshoots, swings back and settles; the Spring CHOP with the input as position.
  • envelope() The level, 0 to peak, of an attack-and-decay envelope at time t, for events that happened at the given times; the Trigger CHOP.

import { Lag } from '@openav/chop'; packages/chop/index.js, line 49

Slope#

class@openav/chop
new Slope()

How fast the input is changing, in units per second; 0 on the first step. The Slope CHOP.

slope.step(value, dt) returns (value − previous value) / dt. A value rising by 3 every second gives 3 at any frame rate; a value that stands still gives 0; a falling one gives a negative number. Use it for "how hard did the knob just move": scale(1 + abs(slope.step(knob, dt))).

It has no smoothing: a single jump of the input is one big spike for one step. Put the result through a Lag if you want it to fade. A dt of 0 or less changes nothing and returns the last slope, and a value that is not a number returns the last slope too.

In a p5 sketch use slope().

Before you call it

Nothing to load. The first step only stores the value and returns 0, because there is no earlier value to compare with.

Members

NameDescription
step(input, dt)methodstep(input, dt): return the change per second since the last step.
reset()methodreset(): forget the previous value; the next step returns 0 again. Returns the Slope.

Notes

  • The last slope is the public field value.

Examples

js
import { Slope } from '@openav/chop';

const slope = new Slope();
const seen = [];
for (const [value, dt] of [[0, 1 / 60], [0.05, 1 / 60], [0.1, 1 / 60], [0.1, 1 / 60]]) {
  seen.push(slope.step(value, dt).toFixed(1));
}
document.getElementById('app').textContent = seen.join('  ');
module Prints 0.0 3.0 3.0 0.0: nothing on the first step, then 3 units a second while it rises, then 0 when it stops.

Related

  • slope() How fast value is changing, per second: positive while it rises, negative while it falls, 0 while it holds still (TouchDesigner's Slope CHOP).
  • Speed Adds rate every second to a running total: a rate in units per second becomes a position; the Speed CHOP.
  • Lag Follows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.

import { Slope } from '@openav/chop'; packages/chop/index.js, line 209

Speed#

class@openav/chop
new Speed({ value = 0 } = {})

Adds rate every second to a running total: a rate in units per second becomes a position; the Speed CHOP.

speed.step(rate, dt) adds rate * dt to the total and returns it. With rate 1 the total grows by 1 every second; rotate(speed.step(TWO_PI, dt)) is one turn a second. Feed it a rate that changes (a knob, micLevel) and the motion speeds up and slows down smoothly, without the jump you get from multiplying frameCount by it.

It is the integral of the rate, so it is the same at any frame rate. A boolean rate counts as 1 per second while true. A rate that is not a number, or a dt of 0 or less, adds nothing.

In a p5 sketch use speed(), which keeps one for each call.

Before you call it

Nothing to load. Pass the seconds since the previous frame as dt.

Parameters

NameTypeDescription
valueNumberThe starting total. Anything that is not a finite number starts at 0. optional default 0

Members

NameDescription
step(rate, dt)methodstep(rate, dt): add rate * dt to the total and return the total.
reset(value = 0)methodreset(value?): set the total back to value (0 if left out). Returns the Speed.

Notes

  • The total is the public field value.

Examples

js
import { Speed } from '@openav/chop';

const speed = new Speed();
const seen = [];
for (let i = 0; i < 4; i++) seen.push(speed.step(2, 0.5).toFixed(1));   // 2 units a second
document.getElementById('app').textContent = seen.join('  ');
module Prints 1.0 2.0 3.0 4.0: two units a second, half a second per step.
js
import { Speed } from '@openav/chop';

const ctx = document.getElementById('c').getContext('2d');
const spin = new Speed();
let last = performance.now();

function frame(now) {
  const dt = (now - last) / 1000; last = now;
  const angle = spin.step(Math.PI * 2, dt);        // one turn a second
  ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
  ctx.strokeStyle = '#fc4'; ctx.lineWidth = 6;
  ctx.beginPath(); ctx.moveTo(200, 150);
  ctx.lineTo(200 + Math.cos(angle) * 90, 150 + Math.sin(angle) * 90); ctx.stroke();
  requestAnimationFrame(frame);
}
requestAnimationFrame(frame);
module A hand that turns once a second, at any frame rate. Change the rate to a changing number (a level, a knob) and it speeds up and slows down smoothly.

Related

  • speed() Adds rate to a running total every second, so rotate(speed(TWO_PI)) turns once a second and the rate can change without the total ever jumping.
  • Slope How fast the input is changing, in units per second; 0 on the first step. The Slope CHOP.
  • Lag Follows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.

import { Speed } from '@openav/chop'; packages/chop/index.js, line 100

Spring#

class@openav/chop
new Spring({ stiffness = 100, damping = 6, value = null } = {})

Follows its input like a mass on a spring: it overshoots, swings back and settles; the Spring CHOP with the input as position.

Make one Spring per value, call step(target, dt) once per frame and draw what it returns. With the defaults (stiffness 100, damping 6), a jump of the input from 0 to 1 sends the output to 1.37 after a third of a second, and it is within 1% of 1 from about 1.4 s on. Raise the damping to 20 and it glides to the target without overshooting.

Each step is the exact answer from springStep(), so 30 fps and 144 fps give the same motion and a long frame cannot blow it up. The input is the position it is pulled toward; a boolean counts as 0 or 1; a value that is not a number is returned as it is and changes nothing.

In a p5 sketch use spring().

Before you call it

Nothing to load. Pass the seconds since the previous frame as dt. The first step only stores the input (the spring starts at rest on it).

Parameters

NameTypeDescription
stiffnessNumberHow hard the spring pulls, so how fast it swings. With nothing damping it, it swings about √stiffness / 2π times a second (100 is about 1.6). optional default 100
dampingNumberHow soon it settles. 6 with a stiffness of 100 bounces a few times; 2 * √stiffness (20 for 100) and above never bounces. optional default 6
valueNumberA starting position. Leave it out and the first input is taken as it is, so it does not swing in from 0. optional default null

Members

NameDescription
step(target, dt, stiffness = this.stiffness, damping = this.damping)methodstep(target, dt, stiffness?, damping?): advance the spring by dt seconds toward target and return the new position. stiffness and damping override the constructor's for this call.
reset(value = null)methodreset(value?): stop the spring. With no argument the next input is taken as it is; with a number it rests there. Returns the Spring.

Notes

  • Public fields: stiffness, damping, value (the position; null before the first step) and velocity.

Examples

js
import { Spring } from '@openav/chop';

const spring = new Spring();             // stiffness 100, damping 6
spring.step(0, 0);                        // rest at 0
const seen = [];
for (let frame = 0; frame < 24; frame += 3) {
  for (let i = 0; i < 3; i++) spring.step(1, 1 / 60);   // the input jumps to 1
  seen.push(spring.value.toFixed(2));
}
document.getElementById('app').textContent = seen.join('  ');
module Every third frame after the input jumps to 1: the value overshoots past 1 (to about 1.37), comes back and settles.
js
import { Spring } from '@openav/chop';

const ctx = document.getElementById('c').getContext('2d');
ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
ctx.lineWidth = 2;
const target = (t) => (t < 0.2 ? 0 : 1);   // the input jumps at 0.2 s

for (const [damping, color] of [[null, '#555'], [6, '#fa4'], [20, '#4af']]) {
  const s = new Spring({ stiffness: 100, damping: damping ?? 6 });
  ctx.strokeStyle = color; ctx.beginPath();
  for (let k = 0; k <= 180; k++) {
    const t = k / 60;
    const y = damping === null ? target(t) : s.step(target(t), k ? 1 / 60 : 0);
    ctx.lineTo(20 + t * 120, 250 - y * 150);
  }
  ctx.stroke();
}
module Two springs follow the same jump (grey): damping 6 (orange) overshoots and wobbles, damping 20 (blue) glides straight to the target.

Related

  • springStep() One exact step of a damped spring of mass 1 pulled toward target; returns [position, velocity].
  • spring() Follows value like a mass on a spring: it overshoots, bounces and settles (TouchDesigner's Spring CHOP).
  • Lag Follows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.
  • approach() One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.

import { Spring } from '@openav/chop'; packages/chop/index.js, line 149

springStep()#

function@openav/chop
springStep(x, v, target, stiffness, damping, dt)

One exact step of a damped spring of mass 1 pulled toward target; returns [position, velocity].

Solves x'' = −stiffness·(x − target) − damping·x' exactly over dt for a target that stays still during the step. It handles all three cases (under-damped: it bounces; critically damped; over-damped: it creeps) and a huge dt never blows up, which is why a sketch can fall behind for a whole second and the spring just lands on the target.

damping = 2 * Math.sqrt(stiffness) is the edge where it stops bouncing: with stiffness 100 that is a damping of 20. With no spring (stiffness 0) only the damping acts and the velocity just fades.

Parameters

NameTypeDescription
xNumberThe position now.
vNumberThe velocity now, in units per second.
targetNumberWhere the other end of the spring is held.
stiffnessNumberHow hard it pulls. Negative counts as 0.
dampingNumberHow much friction slows it. Negative counts as 0.
dtNumberSeconds to advance. 0 or less (or NaN) returns [x, v] unchanged.

Returns

[Number, Number] The new position and velocity.

Notes

  • Advanced. The Spring class holds the position and the velocity for you and calls this.

Examples

js
import { springStep } from '@openav/chop';

let x = 0, v = 0;
const seen = [];
for (let frame = 0; frame < 8; frame++) {
  [x, v] = springStep(x, v, 1, 100, 6, 1 / 60);   // pulled toward 1
  seen.push(x.toFixed(2));
}
document.getElementById('app').textContent = seen.join('  ');
module Eight frames of a spring released at 0 and pulled toward 1: it accelerates (0.01, 0.05, 0.11 …).
js
import { springStep } from '@openav/chop';

const [x, v] = springStep(0, 0, 1, 100, 6, 10);   // a ten-second step in one go
document.getElementById('app').textContent = `after 10 s: position ${x.toFixed(3)}, velocity ${v.toFixed(3)}`;
module A ten-second step lands on the target (position 1.000, velocity 0.000) instead of exploding.

Related

  • Spring Follows its input like a mass on a spring: it overshoots, swings back and settles; the Spring CHOP with the input as position.
  • spring() Follows value like a mass on a spring: it overshoots, bounces and settles (TouchDesigner's Spring CHOP).
  • approach() One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.

import { springStep } from '@openav/chop'; packages/chop/index.js, line 116

TD_LAG_RATIO#

constant@openav/chop
TD_LAG_RATIO = 2.302585092994046

Math.LN10, about 2.303: how many time constants a lag needs to cover 90% of a jump, which is what TouchDesigner's Lag seconds mean.

TouchDesigner's Lag CHOP quotes its seconds as roughly the time the output takes to follow 90% of a change. A one-pole lag reaches 90% after ln 10 time constants, so seconds / TD_LAG_RATIO is the time constant to hand to approach(). Lag does exactly that, which is why lag(x, 0.1) in a p5 sketch moves like a Lag CHOP set to 0.1.

You only need it when you mix the two kinds of seconds: a time constant tau is tdSeconds / TD_LAG_RATIO, and tdSeconds is tau * TD_LAG_RATIO. Attack and decay of an envelope() are plain lengths and are not converted.

Type
Number

Exactly Math.LN10, 2.302585092994046. A fixed number: it never changes.

Examples

js
import { approach, TD_LAG_RATIO } from '@openav/chop';

const tdSeconds = 0.5;                       // as you would type it into a TouchDesigner Lag CHOP
const tau = tdSeconds / TD_LAG_RATIO;        // the time constant approach() wants
const reached = approach(0, 1, tau, tdSeconds);
document.getElementById('app').textContent =
  `tau = ${tau.toFixed(3)} s, after ${tdSeconds} s the value is ${reached.toFixed(2)}`;
module Prints "tau = 0.217 s, after 0.5 s the value is 0.90": half a second of Lag CHOP seconds is 90% of the way.

Related

  • approach() One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.
  • Lag Follows its input with one speed while the input is above the output and another while it is below; the TouchDesigner Lag CHOP.
  • lag() Follows value smoothly instead of jumping: up in up seconds, down in down seconds (TouchDesigner's Lag CHOP).

import { TD_LAG_RATIO } from '@openav/chop'; packages/chop/index.js, line 29

Trail#

class@openav/chop
new Trail({ seconds = 2, rate = 60 } = {})

Remembers the input over the last few seconds as a list of values, oldest first, at a fixed 60 values a second; the Trail CHOP.

trail.step(value, dt) returns an array of seconds * rate past values followed by this step's input at the end, so the array is seconds * rate + 1 long (121 for the defaults). Draw it across the canvas as a line: that is a scope of the value.

The length is fixed from the first step, filled with the first value, so a sketch can draw the whole line from frame one. The values are interpolated between steps, so 30 fps and 144 fps give the same line. A gap longer than the whole window (a tab that was hidden) restarts it filled with the new value.

In a p5 sketch use trail() and drawTrail().

Before you call it

Nothing to load. Pass the seconds since the previous frame as dt. A value that is not a number leaves the history as it is and returns the current list.

Parameters

NameTypeDescription
secondsNumberHow many seconds of history to keep. 0 or less falls back to 2. optional default 2
rateNumberValues per second in the list. 0 or less falls back to 60. optional default 60

Members

NameDescription
sizegetterHow many past values are kept: seconds * rate, rounded and held between 2 and 36000. The list from step() is one longer.
ordered()methodordered(): the stored past values, oldest first, without the newest input (an empty array before the first step).
step(value, dt, seconds = this.seconds)methodstep(value, dt, seconds?): take in this frame's value and return the list. A different seconds resizes the window and keeps the newest values.
values()methodvalues(): the list again, oldest first, the newest input last; an empty array before the first step.
reset()methodreset(): forget the history; the next step fills the list with its value again. Returns the Trail.

Examples

js
import { Trail } from '@openav/chop';

const trail = new Trail({ seconds: 0.1 });   // 6 values a tenth of a second
let list;
for (let i = 0; i < 10; i++) list = trail.step(i, 1 / 60);
document.getElementById('app').textContent =
  `${list.length} values: ${list.join(' ')}`;
module Prints "7 values: 4 5 6 7 8 9 9": the last six steps of the input, then the newest input once more at the end.
js
import { Trail } from '@openav/chop';

const ctx = document.getElementById('c').getContext('2d');
const trail = new Trail({ seconds: 3 });
let last = performance.now();

function frame(now) {
  const dt = (now - last) / 1000; last = now;
  const list = trail.step(0.5 + 0.5 * Math.sin(now / 400), dt);   // a slow sine, 0..1
  ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
  ctx.strokeStyle = '#4af'; ctx.beginPath();
  list.forEach((v, i) => ctx.lineTo((i / (list.length - 1)) * 400, 260 - v * 220));
  ctx.stroke();
  requestAnimationFrame(frame);
}
requestAnimationFrame(frame);
module A scope of a sine wave that scrolls across the canvas: the newest value is at the right edge, with three seconds of history behind it (flat on the left until that much has passed).

Related

  • trail() The last few seconds of value as an array, 60 values a second, oldest first and newest last (TouchDesigner's Trail CHOP).
  • drawTrail() Draws a line through values on the canvas, so it is in the work and in a recording: the oldest at x, lo at the bottom of the box, hi at the top.
  • scope() Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
  • Slope How fast the input is changing, in units per second; 0 on the first step. The Slope CHOP.

import { Trail } from '@openav/chop'; packages/chop/index.js, line 169

wrapCount()#

function@openav/chop
wrapCount(total, limit)

Loops a count to 0 … limit − 1: wrapCount(5, 4) is 1. A limit of 0 or less leaves the count as it is.

The modulo that Count uses, made safe for negative numbers: wrapCount(-1, 4) is 3, where -1 % 4 is -1. It is useful on its own for anything that loops, such as a step number: wrapCount(frameCount, 16).

Parameters

NameTypeDescription
totalNumberThe count so far.
limitNumberHow many values to loop through. A fraction is rounded down. 0, a negative number, or no value at all returns total unchanged.

Returns

Number total looped into 0 … limit − 1 (never negative, even for a negative total).

Examples

js
import { wrapCount } from '@openav/chop';

const row = [0, 3, 4, 5, 9, -1].map((n) => wrapCount(n, 4));
document.getElementById('app').textContent =
  `0 3 4 5 9 -1 in a loop of 4: ${row.join(' ')}   no limit: ${wrapCount(5, 0)}`;
module Prints 0 3 0 1 1 3, then 5 when there is no limit.

Related

  • Count Counts how many times its input turned on (true, or a number above threshold); with a limit it loops 0 … limit − 1. The Count CHOP.
  • count() Counts how often on has turned on, or how many 'kick' (or any other event) have happened; with a limit it loops 0, 1, … limit − 1.

import { wrapCount } from '@openav/chop'; packages/chop/index.js, line 237

Recording#

startRecording(): the canvas and its sound as one video; @openav/record. 49

oav-p5

recordingIsOn#

variableoav-p5
recordingIsOn

Whether a recording is running right now.

Use it to keep things out of the video: if (!recordingIsOn) text('press r to record', 20, 30);, because everything on the canvas is recorded. Use it for a key that toggles: recordingIsOn ? stopRecording() : startRecording(). It changes when a take starts and when it ends, and reading it never starts anything. For the other stages (starting, saving) read OAVp5.record.state.

Type
Boolean

True while a take runs. It is false at the start, while the recorder is still loading, from the moment stopRecording() is called (the file is then being made) and after the take.

Notes

  • A sketch that declares its own variable called recordingIsOn keeps it: oav-p5 leaves the name alone and a card says to read OAVp5.record.state === 'on' instead.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textSize(16);
}

function keyPressed() {
  if (key === 'r') recordingIsOn ? stopRecording() : startRecording();
}

function draw() {
  background(20);
  fill(255, 120, 90);
  circle(200 + sin(frameCount * 0.05) * 120, 150, 50);
  if (!recordingIsOn) {   // the guide never reaches the video
    fill(255);
    text('r: start / stop recording', 20, 30);
  }
}
The guide text is on the canvas until you press r, and gone while recording, so the video has only the circle.

Related

  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • stopRecording() Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.
  • OAVp5.record.state Where the recording is: 'off', 'starting', 'on' or 'saving'.
  • soundIsOn Whether notes are being heard right now: true once startSound() has finished, false before and after.
  • drumsAreOn Whether the drum machine is playing right now.

p5/oav-p5.js, line 317

recordingSaved()#

eventoav-p5
function recordingSaved(info) {}

An event you define: it is called when a recording has been saved and checked, with an info object about the file.

Define function recordingSaved(info) { … } to know when the file is done: stopRecording() or the end of seconds leads to it. By then the file has already been handed to the browser to download. The same info is given to the promise of stopRecording() and to OAVp5.record.on('saved', fn), and stays in OAVp5.record.last.

info.url works as a link to the file (createA(info.url, 'play') or a <video>) until a minute after the next take is saved. warnings is what the card after the take says in words; an empty list means nothing looked wrong.

Parameters

NameTypeDescription
infoObjectname (the file name), format ('mp4' or 'webm'), type (the exact MIME type), seconds (length, to a tenth), bytes, width and height (even numbers for MP4), fps, audio (the video has a sound track), heard (something sounded), sources (which sounds were in it: 'sound', 'drums', 'mic', 'p5sound'), warnings (what looked wrong: 'recEmpty', 'recWebm', 'recNoAudio', 'recSilent', 'recBlack', usually an empty list), blob (the file) and url (an address for the blob).

Examples

jsOpenProcessing ↗
let saved = '';

function setup() {
  createCanvas(400, 300);
}

function keyPressed() {
  if (key === 'r') startRecording(3);
}

function recordingSaved(info) {
  saved = info.name + '  ' + info.format + '  ' + info.seconds + ' s  ' + round(info.bytes / 1024) + ' KB';
}

function draw() {
  background(20);
  circle(200 + sin(frameCount * 0.05) * 120, 150, 50);
  if (!recordingIsOn) {
    fill(255);
    text(saved || 'press r to record 3 seconds', 20, 30);
  }
}
Press r: three seconds later the file is downloaded and its name, format, length and size appear on the canvas.
jsOpenProcessing ↗
let note = '';

function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 120);
}

function keyPressed() {
  if (key === 'r') startRecording(4);
}

function recordingSaved(info) {
  note = info.heard ? 'with sound: ' + info.sources.join(', ') : 'silent: click the page before you record';
  if (info.warnings.length) note += '  (' + info.warnings.join(', ') + ')';
}

function draw() {
  background(20);
  fill(255);
  if (!recordingIsOn) text(note || 'click once, then press r', 20, 30);
}
Press r (after one click on the page): the text says whether the video has sound, from which sources, and lists anything that looked wrong.

Related

  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • stopRecording() Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.
  • OAVp5.record.last The info of the last recording that was saved (the same object recordingSaved got), or null before the first one.
  • OAVp5.record.on() Listens to the recording from anywhere: OAVp5.record.on('saved', fn) or on('start', fn), and it returns a function that stops listening.
  • recordingIsOn Whether a recording is running right now.

p5/oav-p5.js, line 1911

startRecording()#

functionoav-p5
startRecording([o])

Also: OAVp5.record.start

Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.

Start it from a key or a click: if (key === 'r') startRecording();. What goes into the video is the canvas alone, at its own pixel size (a retina screen doubles it), and everything your sketch sounds like: the instrument of startSound(), the drum machine, the pads, the microphone while it is on, and p5.sound 1.x, mixed into one track while the speakers play as before. The 🎹, the drum grid and the cards are part of the page, not of the canvas, so they are never in the video. A sound that comes on during the take joins it.

The file is MP4 (H.264 + AAC, which Instagram, YouTube and every editor take) where the browser can write it, and WebM where not; a card says Instagram will not take a WebM. It is named oav-20261015-183005.mp4, with name in the middle when you give one. When the take is done it is downloaded, and on a phone that can share files a Share button appears on the card. After stopRecording(), or when seconds has run out, recordingSaved(info) is called with the details. Without seconds a take stops by itself at the longest length (300 seconds unless the reclimit option says otherwise).

Without bitrate the quality follows the canvas: 0.16 bits per pixel per frame, kept between 4 and 16 megabits a second (24 above 30 frames a second). Calling startRecording() while a take is running starts nothing new. After every take a card says what was recorded and, when something looks wrong, what and why: nothing reached the sound track although a sound was on, WebM, a file that is almost empty, a black video of a canvas that was not black.

Before you call it

A browser that has MediaRecorder: where it is missing, a card says so and the promise resolves to false. The picture is recorded at once, but the page can only make sound after a click or key press on it, so a take that begins before the first one has nothing to hear at first. A canvas that shows an image from another site (no CORS) cannot be recorded, and a card says so.

Parameters

NameTypeDescription
oNumber | ObjectA number is the length in seconds: startRecording(10) stops by itself after ten. An object can hold seconds (the same, never more than the longest take, 300 by default), fps (frames a second, at most 60, 30 by default), mic (false keeps the microphone out of the video), name (a word for the file name), canvas (the canvas to record: a <canvas>, a p5 canvas or a createGraphics() buffer; the biggest p5 canvas on the page by default) and bitrate (bits a second). Leave it out to record until stopRecording(). optional

Returns

Promise<Boolean> Resolves to true once the recording is running (or when one already is). Resolves to false when the loading option record is off, when there is no canvas to record, when this browser has no MediaRecorder, or when it could not start; a card says why.

Notes

  • In the 🎹 panel, ⏺ starts and stops a take with no code, and 📷 saves the canvas as a PNG (see OAVp5.record.shot()). While a take runs the 🎹 button shows a red dot and the time. The loading options record (on, nomic or off), recfps, reclimit and title set the defaults.
  • The microphone is in the take while it is on: wear headphones, or the speakers are recorded twice. p5.sound for p5 2.x connects each sound straight to the speakers, so it cannot be recorded (a card says so); use p5 1.11, or startSound() and startDrums(). Sound that your sketch makes with its own Tone.js or Web Audio code is not recorded either.
  • Whatever you draw on the canvas is in the video, so show hints only while recordingIsOn is false.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function keyPressed() {
  if (key === 'r') startRecording(5);   // five seconds, then it stops and downloads
}

function draw() {
  background(20);
  circle(200 + sin(frameCount * 0.05) * 120, 150, 50);
  if (!recordingIsOn) {
    fill(255);
    text('press r to record 5 seconds', 20, 30);   // not drawn while recording
  }
}
Press r: after five seconds an MP4 of the moving circle is downloaded, without the hint text.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 110);
  startSound('synth/pluck');
}

function keyPressed() {
  if (key === 'r') {
    if (recordingIsOn) stopRecording();
    else startRecording({ name: 'my piece', fps: 60 });
  }
  if (key === 'a') midi.play(60, 0.8);
}

function kickDetected(level) {
  background(255 * level);
}

function draw() {
  background(20);
  if (!recordingIsOn) {
    fill(255);
    text('click once, then: a plays a note, r starts and stops a recording', 20, 30);
  }
}
Click once so the browser allows sound, then press r to start and r again to stop: the file holds the picture, the drums and the notes you played.
jsOpenProcessing ↗
let layer;

function setup() {
  createCanvas(400, 300);
  layer = createGraphics(200, 200);
}

function keyPressed() {
  if (key === 'g') startRecording({ canvas: layer, seconds: 3 });   // record the buffer, not the main canvas
}

function draw() {
  background(20);
  layer.background(40);
  layer.circle(100 + sin(frameCount * 0.1) * 60, 100, 40);
  image(layer, 100, 50);
  fill(255);
  text('press g to record the small square only (3 s)', 20, 30);
}
Press g: the video is 200 by 200, the buffer alone, not the 400 by 300 canvas.

Related

  • stopRecording() Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.
  • recordingIsOn Whether a recording is running right now.
  • recordingSaved() An event you define: it is called when a recording has been saved and checked, with an info object about the file.
  • OAVp5.record The object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.
  • OAVp5.record.shot() Saves the canvas alone as a PNG picture (not the 🎹, not the cards) and downloads it.
  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • drawRhythm() Draws the rhythm on the canvas as a grid, a circle or a spiral, with a playhead and the hits lighting up, so a recording has it too.

p5/oav-p5.js, line 2306

stopRecording()#

functionoav-p5
stopRecording()

Also: OAVp5.record.stop

Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.

It ends a take that has no seconds, or ends one early. recordingIsOn turns false at once, while the file is made (OAVp5.record.state is 'saving' until then), so the promise or recordingSaved() is how you know the file is ready. Both come after a fraction of a second, a little more for a long take.

Returns

Promise<Object | null> Resolves, once the file is saved and checked, to the same info that recordingSaved(info) gets. Resolves to null at once when nothing is recording, or when the recording was still starting (nothing was recorded then).

Examples

jsOpenProcessing ↗
let message = 'press r to record, r again to stop';

function setup() {
  createCanvas(400, 300);
}

function keyPressed() {
  if (key !== 'r') return;
  if (recordingIsOn) {
    stopRecording().then((info) => {
      if (info) message = 'saved ' + info.name;   // the file is ready
    });
  } else {
    startRecording();
  }
}

function draw() {
  background(20);
  circle(200 + sin(frameCount * 0.05) * 120, 150, 50);
  if (!recordingIsOn) {
    fill(255);
    text(message, 20, 30);
  }
}
Press r to start and r to stop: once the file is ready the text names it.

Related

  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • recordingIsOn Whether a recording is running right now.
  • recordingSaved() An event you define: it is called when a recording has been saved and checked, with an info object about the file.
  • OAVp5.record.state Where the recording is: 'off', 'starting', 'on' or 'saving'.

p5/oav-p5.js, line 2307

OAVp5.record#

objectoav-p5
OAVp5.record

The object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.

OAVp5.record (also reachable as midi.record) is what startRecording(), stopRecording() and recordingIsOn are built on. It holds the state of the recording, how many seconds it has run, the last file's info, the sources in the take, and the fps and limit in force. formats() says what the browser can write, shot() saves a PNG of the canvas and on() listens to 'start' and 'saved'. start and stop are the same as the two global functions.

Notes

  • For a sketch the global names are enough. This object is for tools, the console and debugging.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textSize(14);
}

function draw() {
  background(20);
  fill(255);
  const r = OAVp5.record;
  text('state: ' + r.state, 20, 30);
  text('seconds: ' + nf(r.seconds, 1, 1), 20, 60);
  text('limit: ' + r.limit + ' s at ' + r.fps + ' fps', 20, 90);
}
Three fields of the recording object; the state stays "off" until a take starts (press ⏺ in the 🎹 panel).

Related

  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • stopRecording() Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.
  • recordingIsOn Whether a recording is running right now.
  • OAVp5.sound The object behind the sound names: the state, the instrument, the error, the loading progress and the loaded parts of @openav/sound.
  • OAVp5.drums The object behind the drum names: the state of the machine, its counters, the line of rhythm, its history and the loaded machine.

p5/oav-p5.js, line 1906

OAVp5.record.formats()#

functionoav-p5
OAVp5.record.formats()

Says what this browser can record: { recorder, mp4, webm, mime }.

Use it before you offer a recording button, or to tell the person which file they will get: MP4 goes to Instagram and every editor, WebM does not. It asks the browser each time you call it.

Returns

Object recorder is true when the browser has MediaRecorder at all, mp4 and webm are true when it can write that format, and mime is the MIME type a recording asks for first (MP4 where possible), or '' when none works.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textSize(14);
}

function draw() {
  background(20);
  fill(255);
  const f = OAVp5.record.formats();
  text('recorder: ' + f.recorder, 20, 30);
  text('mp4: ' + f.mp4 + '   webm: ' + f.webm, 20, 60);
  text('asks for: ' + (f.mime || 'nothing'), 20, 90);
}
Shows what this browser can write: Chrome 126 and later and Safari say mp4 true.

Related

  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • OAVp5.record The object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.

p5/oav-p5.js, line 1895

OAVp5.record.fps#

variableoav-p5
OAVp5.record.fps

Also: OAVp5.record.limit

The frame rate (fps) and the longest take in seconds (limit) that the loading options recfps and reclimit set.

They are there to read: OAVp5.record.limit tells a sketch how long a take can be. Writing to them changes nothing, because a take reads the options, not these two. To record at another frame rate once, pass startRecording({ fps: 60 }); to change the limit, set the reclimit option before oav-p5 loads (window.OAV_P5 = { reclimit: 600 }).

Type
Number
Range
fps 1..60, limit 1 …

fps is 30 and limit is 300 unless the options say otherwise (fps is kept between 1 and 60).

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textSize(14);
}

function draw() {
  background(20);
  fill(255);
  text('a take is ' + OAVp5.record.fps + ' frames a second', 20, 30);
  text('and at most ' + OAVp5.record.limit + ' seconds (' + OAVp5.record.limit / 60 + ' minutes)', 20, 60);
}
Prints the two defaults: 30 frames a second and 300 seconds.

Related

  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • OAVp5.record The object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.

p5/oav-p5.js, line 1887

OAVp5.record.last#

variableoav-p5
OAVp5.record.last

The info of the last recording that was saved (the same object recordingSaved got), or null before the first one.

Handy when you did not keep info in recordingSaved(): show the last file name, or put OAVp5.record.last.url in a link. The address stays valid until a minute after the next take is saved.

Type
Object | null

Replaced each time a take is saved. See recordingSaved() for its fields.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textSize(14);
}

function keyPressed() {
  if (key === 'r') startRecording(2);
}

function draw() {
  background(20);
  fill(255);
  const last = OAVp5.record.last;
  text(last ? 'last file: ' + last.name + ' (' + last.seconds + ' s)' : 'nothing recorded yet, press r', 20, 30);
}
Press r: two seconds later the last file name appears; press again and it changes.

Related

  • recordingSaved() An event you define: it is called when a recording has been saved and checked, with an info object about the file.
  • OAVp5.record.seconds How long the running take has been going, in seconds, and the length of the last take after it.
  • stopRecording() Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.

p5/oav-p5.js, line 1887

OAVp5.record.on()#

functionoav-p5
OAVp5.record.on(name, fn)

Listens to the recording from anywhere: OAVp5.record.on('saved', fn) or on('start', fn), and it returns a function that stops listening.

recordingSaved() is a single function that you define. on() is for the cases where that does not fit: several listeners, a library, a listener you want to remove again. It is the same idea as OAVp5.drums.on() and midi.on(). An error thrown inside fn is printed and the other listeners still run.

Parameters

NameTypeDescription
nameString'start' (a take began) or 'saved' (a file is saved, like recordingSaved()). Any other name throws an error that lists the two.
fnFunctionFor 'start' it gets { width, height, fps, type, seconds } (seconds is 0 when no length was asked for). For 'saved' it gets the same info as recordingSaved().

Returns

Function A function: call it to stop listening.

Examples

jsOpenProcessing ↗
let status = 'press r to record 2 seconds';

function setup() {
  createCanvas(400, 300);
  OAVp5.record.on('start', (i) => { status = 'recording ' + i.width + 'x' + i.height + ' at ' + i.fps + ' fps'; });
  OAVp5.record.on('saved', (i) => { status = 'saved ' + i.name; });
}

function keyPressed() {
  if (key === 'r') startRecording(2);
}

function draw() {
  background(20);
  circle(200 + sin(frameCount * 0.05) * 120, 150, 50);
  if (!recordingIsOn) {
    fill(255);
    text(status, 20, 30);
  }
}
Press r: the text reports the start (size and frame rate) and then the saved file name, from two listeners.

Related

  • recordingSaved() An event you define: it is called when a recording has been saved and checked, with an info object about the file.
  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • OAVp5.drums.on() Listens to the drum machine from anywhere: OAVp5.drums.on('hit', fn) calls fn like drumHit does, and returns a function that stops listening.
  • OAVp5.record The object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.

p5/oav-p5.js, line 1897

OAVp5.record.seconds#

variableoav-p5
OAVp5.record.seconds

How long the running take has been going, in seconds, and the length of the last take after it.

Use it as a clock that starts with the take: const t = recordingIsOn ? OAVp5.record.seconds : 0; makes an animation that always begins at 0 in the video. It is the time since the recording began, not the time of the sketch, so it does not count the moment before startRecording().

Type
Number
Range
0 …

0 at the start, then refreshed four times a second while recording. When the take is saved it is that take's length, and the next take sets it back to 0.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function keyPressed() {
  if (key === 'r') startRecording(4);
}

function draw() {
  background(20);
  const t = recordingIsOn ? OAVp5.record.seconds : 0;   // 0 at the first frame of the video
  fill(255, 160, 80);
  circle(40 + t * 80, 150, 40);                         // slides right, 80 px a second
  if (!recordingIsOn) {
    fill(255);
    text('press r: the video starts with the circle at the left', 20, 30);
  }
}
Press r: the circle jumps to the left edge and slides right for four seconds, so the video always begins the same way.

Related

  • recordingIsOn Whether a recording is running right now.
  • OAVp5.record.state Where the recording is: 'off', 'starting', 'on' or 'saving'.
  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • OAVp5.record.last The info of the last recording that was saved (the same object recordingSaved got), or null before the first one.

p5/oav-p5.js, line 1887

OAVp5.record.shot()#

functionoav-p5
OAVp5.record.shot([o])

Saves the canvas alone as a PNG picture (not the 🎹, not the cards) and downloads it.

It is the button 📷 in the 🎹 panel as a function. The file is named oav-20261015-183005.png (with name in the middle when you give one) and is downloaded at once; on a phone that can share files a Share button appears on a card instead. The picture has the size of the canvas in pixels, so a retina screen gives twice the width and height. p5's own saveCanvas('name', 'png') does the same thing without the card.

Parameters

NameTypeDescription
oObjectOptions: canvas (the canvas to save: a <canvas>, a p5 canvas or a createGraphics() buffer; the biggest p5 canvas on the page by default) and name (a word for the file name). optional

Returns

Promise<Object | null> Resolves to { name, blob, url, width, height, bytes }, or to null when there is no canvas or it cannot be read (an image from another site on it, no CORS).

Notes

  • With the loading option record=off the 📷 button is gone and startRecording() does nothing, but this function still saves the PNG (tried).

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function keyPressed() {
  if (key === 'p') OAVp5.record.shot({ name: 'still' });
}

function draw() {
  background(20);
  circle(mouseX, mouseY, 60);
  fill(255);
  text('press p to save a PNG of the canvas', 20, 30);
}
Move the circle with the mouse and press p: a PNG of the canvas is downloaded.

Related

  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • OAVp5.record The object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.
  • recordingSaved() An event you define: it is called when a recording has been saved and checked, with an info object about the file.

p5/oav-p5.js, line 1893

OAVp5.record.sources#

variableoav-p5
OAVp5.record.sources

Which sounds the running take is hearing right now: a list of 'sound', 'drums', 'mic' and 'p5sound'; empty when no take runs.

It is what the take has connected, not whether something is audible at this moment. A source that is switched on halfway joins the list within a quarter of a second. After the take, recordingSaved() gives the final list in info.sources.

Type
Array<String>

A new list on every read. 'sound' is the instrument of startSound(), 'drums' the drum machine and the pads, 'mic' the microphone, 'p5sound' p5.sound 1.x.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('breakbeat', 120);
  startSound('synth/pluck');
  textSize(14);
}

function keyPressed() {
  if (key === 'r') recordingIsOn ? stopRecording() : startRecording();
}

function draw() {
  background(20);
  fill(255);
  text(recordingIsOn ? 'in the take: ' + OAVp5.record.sources.join(', ') : 'press r to start a take', 20, 30);
}
Press r: the text lists what the take has connected, here the drum machine and the instrument.

Related

  • recordingSaved() An event you define: it is called when a recording has been saved and checked, with an info object about the file.
  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • startMic() Turns the microphone on and returns a promise that says whether it worked.

p5/oav-p5.js, line 1903

OAVp5.record.state#

variableoav-p5
OAVp5.record.state

Where the recording is: 'off', 'starting', 'on' or 'saving'.

recordingIsOn is state === 'on'. Read the state when the other stages matter: a button can show "starting…" and "saving…" and refuse a new take while one is being saved.

Type
String
Range
off | starting | on | saving

Starts as 'off'. A take goes 'starting' (the recorder is loading, the first time), 'on' (recording), 'saving' (the file is being made) and back to 'off'.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(28);
}

function keyPressed() {
  if (key === 'r' && OAVp5.record.state === 'off') startRecording(3);
}

function draw() {
  background(20);
  circle(200 + sin(frameCount * 0.05) * 120, 220, 40);
  fill(255);
  text(OAVp5.record.state === 'off' ? 'press r' : OAVp5.record.state, width / 2, 70);
}
Press r: the word goes through starting, on and saving and comes back to the prompt; the key does nothing in between.

Related

  • recordingIsOn Whether a recording is running right now.
  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • stopRecording() Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.
  • OAVp5.record The object behind the recording names: the state, the time, the last file, the formats this browser can write and the screenshot function.

p5/oav-p5.js, line 1886

@openav/record

AUDIO_MIMES#

constant@openav/record
AUDIO_MIMES = array, 5 items

The audio MIME types the recorder tries for a sound-only file, best first: AAC in .m4a, then Opus.

AAC in an .m4a file comes first because it imports into every DAW and editor. This is the list a Recorder with audioOnly: true uses, through pickMime().

Type
String[]

A frozen list of 5 strings: audio/mp4;codecs=mp4a.40.2, audio/webm;codecs=opus, audio/ogg;codecs=opus, audio/mp4, audio/webm.

Examples

js
import { AUDIO_MIMES, pickMime, extFor } from '@openav/record';

const mime = pickMime((m) => MediaRecorder.isTypeSupported(m), { audioOnly: true });
console.log(AUDIO_MIMES[0]);        // audio/mp4;codecs=mp4a.40.2
console.log(mime, extFor(mime));    // the one this browser would use, and its file extension
module Logs the best candidate for a sound file, and the one this browser would pick with its extension.

Related

  • VIDEO_MIMES The video MIME types the recorder tries, best first: MP4 with H.264 and AAC, then WebM.
  • pickMime() Picks the first MIME type from a list that the browser can record, using the browser's own check.
  • extFor() Gives the file extension for a recorded MIME type: video/mp4 is 'mp4', audio/mp4 is 'm4a', anything unknown is 'bin'.
  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.

import { AUDIO_MIMES } from '@openav/record'; packages/record/mime.js, line 22

AudioTap#

class@openav/record
new AudioTap({ context = null, sources = [], captureDestination = false } = {})

Collects the sound of your work (and, if you add it, a microphone) into one recordable track, without sending anything to the speakers.

Recording the speakers with a microphone gives you the room. Tapping the Web Audio graph gives you the instrument exactly as it was synthesized, on the same clock as the canvas. An AudioTap has two buses and two outputs. The work bus is the sound of the piece; the extra bus is for things that belong in the video but not in the sound file, such as a microphone. tap.stream and tap.tracks are work and extras together (give them to the video Recorder); tap.workStream and tap.workTracks are the work alone (for the separate audio file). Both are a single track, on purpose: Chrome records only the first audio track of a stream.

The tap never routes anything to the speakers, so adding a microphone cannot feed back. To record a sketch you did not write (p5.sound, Tone.js, Sound), call captureDestination() before the work starts its sound: from then on every node that connects to a speaker destination is tapped too.

Before you call it

An AudioContext that is running. Browsers start audio only after a click, so call tap.resume() from a click handler. Without the click the tracks exist but carry silence.

Parameters

NameTypeDescription
contextAudioContextThe AudioContext to mix in, usually the work's own. One is made if you leave it out; it needs Web Audio to exist. optional default null
sourcesArrayThings to add to the work bus at once: AudioNodes, MediaStreams or MediaStreamTracks (see add()). optional default []
captureDestinationBooleanAlso call captureDestination() right away. optional default false

Members

NameDescription
streamgetterGetter. A MediaStream of the work and the extras mixed, one audio track. Give it to the video recorder.
workStreamgetterGetter. A MediaStream of the work alone, one audio track: the separate audio file.
tracksgetterGetter. The audio tracks of stream (one): new Recorder({ audioTracks: tap.tracks }).
workTracksgetterGetter. The audio tracks of workStream (one).
add(source, { bus = 'work', gain = 1 } = {})methodTaps a source: add(source, { bus = 'work', gain = 1 }). A node of this tap's context is connected directly, a node of another AudioContext is bridged through a MediaStream, and a MediaStream or MediaStreamTrack is brought in as a source node. bus: 'extra' reaches the video only. Returns a function that removes it. Throws a TypeError for anything else.
captureDestination()methodFrom now on, every node that connects to an AudioDestinationNode (the speakers), in any context, is tapped too, by a hook on AudioNode.prototype.connect. Install it before the work starts its sound: a node connected earlier is invisible. It can be called again without doubling. Returns a function that uninstalls it.
addMicrophone({ deviceId = '', gain = 1, echoCancellation = false, noiseSuppression = false, autoGainControl = false } = {})methodAdds a microphone (or any audio input) to the video's mix, not to the audio file. { deviceId, gain = 1, echoCancellation = false, noiseSuppression = false, autoGainControl = false }: those three are off by default because they eat piano attacks and room tone. Returns a promise of the MediaStream. A microphone added before is replaced. Needs the user's permission.
removeMicrophone({ stop = true } = {})methodTakes the microphone out of the mix and stops it, unless you pass { stop: false }.
level()methodThe peak level of the mix right now, 0..1: for a "sound is reaching the recording" meter.
resume()methodResumes a suspended context; call it from a click. Returns a promise.
dispose()methodLets go of everything the tap made, including the hook and the microphone. The context itself is left alone.

Examples

js
import { AudioTap } from '@openav/record';

const tap = new AudioTap();                              // makes its own AudioContext
const osc = new OscillatorNode(tap.context, { frequency: 440 });
const remove = tap.add(osc);                             // the work bus
osc.start();
await tap.resume();                                      // from a click in a real page
await new Promise((done) => setTimeout(done, 300));
console.log(tap.tracks.length, tap.workTracks.length);   // 1 1
console.log(tap.level() > 0.1);                          // true: sound is reaching the recording
remove();
module A 440 Hz tone is tapped and the meter confirms that sound reaches the recording. The tracks are ready to hand to a Recorder.
js
import { AudioTap } from '@openav/record';

const tap = new AudioTap();
const uninstall = tap.captureDestination();              // before the work starts its sound

const ctx = new AudioContext();                          // "someone else's" sketch, in its own context
const osc = new OscillatorNode(ctx, { frequency: 330 });
osc.connect(ctx.destination);                            // goes to the speakers, and into the tap
osc.start();
await new Promise((done) => setTimeout(done, 300));
console.log(tap.level() > 0.05);                         // true
uninstall();
tap.dispose();
module A sound made in another AudioContext, connected to the speakers the usual way, is picked up by the tap too: this is how you record a sketch you did not write.

Related

  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
  • mixTracks() Mixes several audio tracks into one, because Chrome records only the first audio track of a stream.
  • Compositor Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
  • Sound Makes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.

import { AudioTap } from '@openav/record'; packages/record/audio.js, line 44

cameraConstraints()#

function@openav/record
cameraConstraints({ deviceId = '', width = 1920, height = 1080, fps = 30, facingMode = '' } = {})

Builds the getUserMedia options for a camera: it asks for 1920 by 1080 at 30 fps, but only as an ideal, never as a demand.

Size and rate are ideal, never exact: a camera that cannot do 1920 by 1080 should open at what it can, not throw in front of an audience. It is pure, so you can look at what openCamera() will ask for. The stream has no audio: a microphone is added through AudioTap.

Parameters

NameTypeDescription
deviceIdStringA specific camera, from listCameras(). It is asked for as exact: if it is gone, opening fails. optional default ''
widthNumberThe width to ask for, as ideal. optional default 1920
heightNumberThe height to ask for, as ideal. optional default 1080
fpsNumberThe frame rate to ask for, as ideal. optional default 30
facingModeString'user' or 'environment' on a phone. Ignored when a deviceId is given. optional default ''

Returns

Object A MediaStreamConstraints: { video: { width: { ideal }, height: { ideal }, frameRate: { ideal }, deviceId?, facingMode? }, audio: false }.

Examples

js
import { cameraConstraints } from '@openav/record';

console.log(JSON.stringify(cameraConstraints()));
// {"video":{"width":{"ideal":1920},"height":{"ideal":1080},"frameRate":{"ideal":30}},"audio":false}
console.log(JSON.stringify(cameraConstraints({ deviceId: 'abc', width: 1280, height: 720, fps: 60 }).video));
// {"width":{"ideal":1280},"height":{"ideal":720},"frameRate":{"ideal":60},"deviceId":{"exact":"abc"}}
module The default request, and a request for one specific camera at 720p60.

Related

  • openCamera() Opens a camera and returns its MediaStream: ideally 1920 by 1080 at 30 fps, or whatever the camera can do.
  • listCameras() Lists the video inputs of this computer as { deviceId, groupId, label }.
  • Compositor Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.

import { cameraConstraints } from '@openav/record'; packages/record/camera.js, line 13

cleanValue()#

function@openav/record
cleanValue(v, depth = 0)

Makes a copy of a signal value that is safe to write as JSON: rounded numbers, plain arrays, no functions, nothing too deep.

The rules: numbers are rounded to 6 decimals, and NaN and infinities become null. Typed arrays become plain arrays. A bigint becomes a number. Objects and arrays are copied down to depth 4 and anything deeper becomes null. A function, symbol or undefined becomes null in an array and the key is dropped from an object. EventLog runs every value through it.

Parameters

NameTypeDescription
vanyThe value to clean.
depthNumberHow deep we already are. Leave it out; it is used by the function calling itself. optional default 0

Returns

any The cleaned copy.

Examples

js
import { cleanValue } from '@openav/record';

const out = cleanValue({
  a: 1.23456789, list: [1, NaN, () => 1], pos: new Float32Array([0.5, 0.25]), fn() {}, big: 10n,
});
console.log(JSON.stringify(out));
// {"a":1.234568,"list":[1,null,null],"pos":[0.5,0.25],"big":10}
module A messy object goes in and plain data comes out: the number is rounded, the function key is gone, the typed array became a list.

Related

  • EventLog Keeps a timestamped list of what happened during a take, such as notes and parameter changes, on the same clock as the recording.
  • normalizeTake() Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.

import { cleanValue } from '@openav/record'; packages/record/events.js, line 19

Compositor#

class@openav/record
new Compositor({ size = 'vertical-1080p', layout = 'stack', layoutOptions = {}, getWorkCanvas = null, background = '#000', mirrorCam = true, maxFps = 60, onDraw = null, canvas = null } = {})

Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.

A performance video is two things in one picture: your work and the performer. The Compositor makes that picture. It owns a canvas (comp.canvas) of exactly the size of the file you want, whatever your window is, so a vertical 4K video from a laptop really is 4K. It is the canvas you hand to a Recorder. To show it on the page as a preview, put comp.canvas in the DOM and shrink it with CSS.

comp.start() redraws every animation frame (and, because a hidden tab freezes animation frames, from a Web Worker timer too, so a recording does not freeze with the tab). comp.draw() draws one frame now: call it yourself right after your work renders if you have your own loop. comp.rects is the last layout, { work, cam } in output pixels, and comp.fps is the rate it has been drawing at.

The camera is optional and never owned: setCamera(stream) shows it and setCamera(null) lets go, and you stop the tracks yourself. The camera's area can show a panel instead, anything with draw(ctx, rect, now), for example a drawing of the controller being played. The area for the camera is reserved before the camera arrives, so the picture does not jump.

Parameters

NameTypeDescription
sizeString | Object | Number[]The size of the output, which is the size of the file. A preset id from PRESETS ('vertical-4k'), '1080x1920', [w, h] or { w, h }. Each side is rounded up to an even number. See resolveSize(). optional default 'vertical-1080p'
layoutStringWhere the work and the camera go: stack, pip, side or work (see LAYOUTS). optional default 'stack'
layoutOptionsObjectPassed on to layoutRects(): fit, camFit, camAnchor, minCam, split, pip. optional default {}
getWorkCanvasFunctionReturns the canvas of your work, or null. It is asked every frame, because a p5 sketch may replace its canvas and a WebGL work may resize it. The compositor never owns the work. optional default null
backgroundStringThe colour behind everything, any canvas colour string. optional default '#000'
mirrorCamBooleanShow the camera as a mirror, as performers expect. Keep it equal to the mirror of a hand tracker drawn on top. optional default true
maxFpsNumberDraw at most this often when running. A 120 Hz display does not need 120 composites a second. optional default 60
onDrawFunctiononDraw(ctx, rects, comp), called after the work and the camera are drawn, to put an overlay on top. A context state is saved and restored around it. An error in it is warned about once and does not stop the recording. optional default null
canvasHTMLCanvasElementDraw into this canvas instead of making a new one. Its width and height are set to the output size. optional default null

Members

NameDescription
sizegetterGetter. The output size { w, h }, which is the size of comp.canvas.
setSize(size)methodChanges the output size (the same values as the size option). Do it between takes: a recorder that is already running keeps the size it started with. Returns the compositor.
setLayout(layout, options)methodChanges the layout and, if you pass options, merges them into layoutOptions (the pip object is merged too). Throws for a name that is not in LAYOUTS. Returns the compositor.
videogetterGetter. The <video> element the camera plays in, or null when there is none. Use it for a separate monitor.
cameraSizegetterGetter. The camera's size { w, h } once it plays, otherwise null.
setCamera(src)methodShows a camera: a MediaStream (played in a hidden muted <video> made here), a <video> or a canvas, drawn as it is. null removes it, and its tracks keep running. Returns a promise of the camera size { w, h } once frames arrive, or null (no camera, or none within 5 seconds).
panelgetterGetter. The panel drawn in the camera's area, or null (the camera is shown).
setPanel(panel)methodDraws a panel where the camera goes: { draw(ctx, rect, now), aspect }. draw is called every frame with a context clipped to the area, rect is { x, y, w, h } in output pixels and aspect (width divided by height, default 16/9) shapes a picture-in-picture corner. setPanel(null) brings the camera back, which stayed attached. The compositor never disposes a panel. Throws if the panel has no draw.
draw()methodDraws one frame now and returns the rects { work, cam }. Call it right after your work renders if you run your own loop.
drawOnCamera(fn)methodDraws over the camera in the camera's own coordinates: fn(ctx, w, h) gets a context moved to where the whole camera frame lands and clipped to the visible camera area, which is what a hand or pose tracker's skeleton(ctx, w, h) expects. Call it from onDraw. Does nothing without a camera, or with a panel in its place.
start()methodRedraws every animation frame (at most maxFps a second), and from a Web Worker timer while the tab is hidden. Returns the compositor.
stop()methodStops redrawing. Returns the compositor.
dispose()methodStops and lets go of the camera (its tracks are yours to stop).

Notes

  • A WebGL work: the compositor copies its canvas in the same animation frame it was drawn in, which works in Chrome. If you see black, create the WebGL context with preserveDrawingBuffer: true, or call comp.draw() right after your render and do not call start().
  • Webcams usually stop at 1080p30, so a 4K stack layout enlarges the camera about 1.6 times. The work above it stays sharp because it is drawn at the output size, as long as the work's own canvas is at least as large as its area.
  • In a p5 sketch you do not need this: startRecording() records the canvas and its sound for you. The Compositor is for a page that records the work together with a camera.

Examples

js
import { Compositor } from '@openav/record';

const work = document.createElement('canvas');          // the work: any canvas
work.width = 600; work.height = 600;
const g = work.getContext('2d');
g.fillStyle = '#c33'; g.fillRect(0, 0, 600, 600);

const comp = new Compositor({ size: '540x960', layout: 'stack', getWorkCanvas: () => work });
const rects = comp.draw();                               // one frame, now
console.log(comp.size, rects.work.h, rects.cam.box.h);   // { w: 540, h: 960 } 540 420

comp.canvas.style.width = '135px';                       // a preview, scaled down with CSS
document.getElementById('app').append(comp.canvas);
module A square work is drawn into a 540 by 960 frame: the work takes the top 540 pixels and 420 are kept for the camera. The result appears on the page as a small preview.
js
import { Compositor, openCamera } from '@openav/record';

const work = document.getElementById('c');              // the 400 x 300 canvas of the page
const wg = work.getContext('2d');
const comp = new Compositor({ size: 'landscape-1080p', layout: 'pip', getWorkCanvas: () => work,
  layoutOptions: { pip: { corner: 'br', size: 0.25 } } });
comp.start();                                           // redraws every frame
const size = await comp.setCamera(await openCamera());  // the browser asks for the camera
console.log(size, comp.rects.cam.w);                    // { w: 1920, h: 1080 } 480 (what the camera gives may differ)

let x = 0;
(function animate() {                                   // the work moves, the composite follows
  wg.fillStyle = '#111'; wg.fillRect(0, 0, 400, 300);
  wg.fillStyle = '#ffd166'; wg.fillRect(x = (x + 4) % 380, 120, 20, 60);
  requestAnimationFrame(animate);
})();
comp.canvas.style.width = '480px';
document.getElementById('app').append(comp.canvas);
module The animated canvas fills a 1080p frame with the camera in a corner. Allow the camera when the browser asks; the composite appears below the work.
js
import { Compositor } from '@openav/record';

const work = document.getElementById('c');
const comp = new Compositor({
  size: [720, 720], layout: 'work', getWorkCanvas: () => work,
  onDraw(ctx, rects) {                                   // an overlay, on top of everything
    ctx.fillStyle = 'rgba(255, 255, 255, 0.85)';
    ctx.font = '32px sans-serif';
    ctx.fillText('take 1', rects.work.x + 20, rects.work.y + 48);
  },
});
work.getContext('2d').fillRect(0, 0, 400, 300);
comp.draw();
document.getElementById('app').append(comp.canvas);
comp.canvas.style.width = '240px';
module The work alone in a square frame, with a caption written over it by onDraw. The caption is part of the picture, so it is in the recording.

Related

  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
  • layoutRects() Works out where the work and the camera go in an output frame, in whole pixels, for one of the four layouts.
  • PRESETS The output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.
  • openCamera() Opens a camera and returns its MediaStream: ideally 1920 by 1080 at 30 fps, or whatever the camera can do.
  • AudioTap Collects the sound of your work (and, if you add it, a microphone) into one recordable track, without sending anything to the speakers.
  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.

import { Compositor } from '@openav/record'; packages/record/compositor.js, line 33

EventLog#

class@openav/record
new EventLog({ now = defaultNow, filter = null, throttleMs = 0 } = {})

Keeps a timestamped list of what happened during a take, such as notes and parameter changes, on the same clock as the recording.

A video keeps the picture and the sound. The log keeps the score of the take: every note, pad hit and (throttled) hand position, in milliseconds from the moment recording started. With it a take can be re-rendered at another size, re-synced to a better audio take, or turned into MIDI later. Start it with the recorder's start time so event times line up with the video's first frame: log.start(recorder.t0).

log.attach(signals) logs every signal a Signals bus publishes; pulses are always kept and continuous signals obey throttleMs. Or add events yourself with log.add(name, value). Values are cleaned into plain JSON data by cleanValue(). toJSON() gives { format: 'openav-eventlog', version: 1, startedAt, durationMs, count, events }, and EventLog.fromJSON(json) reads it back. toTake() turns the log into a MIDI take; TakeRecorder is a log that does exactly that on midi/*.

Properties: events is the list [{ t, name, value }] (t in ms, to a tenth), t0 the clock time of t = 0, startedAt the wall-clock time (ISO text) and durationMs is null until stop().

Parameters

NameTypeDescription
nowFunctionThe clock, in milliseconds. The default is the same clock as Recorder.t0. Pass your own in a test. optional default performance.now
filterFunctionfilter(name, value, meta), returns whether to keep an event. It applies to add() too. optional default null
throttleMsNumberFor continuous signals: keep at most one event per name in this many milliseconds. Pulses (note on, note off…) are never throttled. optional default 0

Members

NameDescription
runninggetterGetter. True from start() until stop().
start(at = this.now())methodStarts a take and clears the log. at is the clock time of t = 0 (default now): pass recorder.t0 to line up with a video. Returns the log.
stop(at = this.now())methodEnds the take and sets durationMs; later events are ignored. Returns the log.
add(name, value = null, { at = this.now(), pulse = true, meta } = {})methodLogs one event: add(name, value, { at, pulse, meta }), and returns it as { t, name, value }, or null when the log is not running, the filter refused it, it happened before t = 0, or it was throttled. pulse: false makes it subject to throttleMs; at is the clock time (default now).
mark(label, data = null)methodLogs a labelled marker such as "chorus" or "take 2 starts here" as an event named mark with { label, ...data }.
attach(signals)methodLogs every signal a Signals bus publishes (through signals.onAny). Returns a function that detaches again.
detach()methodDetaches from every bus it was attached to.
between(start, end)methodThe events with start <= t < end (milliseconds).
toJSON()methodThe log as plain data: { format: 'openav-eventlog', version: 1, startedAt, durationMs, count, events }.
toTake({ name = 'take', meta = {} } = {})methodThe log as a MIDI take { v: 1, name, createdAt, durationMs, events, meta } (see normalizeTake()). { name = 'take', meta = {} }. Every event is kept; filter to midi/* first, or use a TakeRecorder.
fromJSON(json)static methodStatic. Rebuilds a stopped log from toJSON() output or its text. Throws a TypeError if it is not an openav-eventlog.

Examples

js
import { EventLog } from '@openav/record';

let clock = 1000;                                   // a clock we move by hand, in ms
const log = new EventLog({ now: () => clock });
log.start();
clock = 1100; log.add('midi/note/on', { note: 60, vel: 0.8 });
clock = 1200; log.mark('chorus');
clock = 1500; log.stop();
console.log(log.events.map((e) => e.t + ' ' + e.name).join(' | '));   // 100 midi/note/on | 200 mark
console.log(log.durationMs, log.running, log.toJSON().count);          // 500 false 2
console.log(log.add('late'));                                          // null: the take is over
module A take of half a second on a clock moved by hand: a note at 100 ms, a marker at 200 ms. After stop() nothing more is accepted.
js
import { EventLog } from '@openav/record';
import { Signals } from '@openav/core';

const signals = new Signals();
const log = new EventLog({ throttleMs: 100 });        // knobs: at most one event per 100 ms
log.start();
const detach = log.attach(signals);
signals.pulse('midi/note/on', { note: 64, vel: 0.7 }); // a pulse: always kept
for (let i = 0; i < 10; i++) signals.set('param/hue', i / 10);   // continuous: throttled
detach();
console.log(log.events.length, log.events[0].name);    // 2 midi/note/on
console.log(EventLog.fromJSON(log.toJSON()).events.length);   // 2
module A bus is attached: the note pulse is kept, and ten changes of a continuous signal in a few milliseconds become one event. The log survives a round trip through JSON.

Related

  • TakeRecorder Records what you play: it listens to every midi/* signal on a bus from start() to stop() and gives you a take.
  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
  • cleanValue() Makes a copy of a signal value that is safe to write as JSON: rounded numbers, plain arrays, no functions, nothing too deep.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • toMidiFile() Writes a take as a Standard MIDI File (type 0) that opens in any DAW.

import { EventLog } from '@openav/record'; packages/record/events.js, line 38

extFor()#

function@openav/record
extFor(mime)

Gives the file extension for a recorded MIME type: video/mp4 is 'mp4', audio/mp4 is 'm4a', anything unknown is 'bin'.

A Recorder puts this in the ext of its result. Use it for a download name: 'take.' + extFor(blob.type).

Parameters

NameTypeDescription
mimeStringA MIME type, with or without codecs, in any letter case. Anything that is not text counts as empty.

Returns

String mp4, m4a (audio/mp4), webm, ogg, mkv (video/x-matroska) or bin.

Examples

js
import { extFor } from '@openav/record';

console.log(extFor('video/mp4;codecs=avc1.640033,mp4a.40.2'));   // mp4
console.log(extFor('audio/mp4'), extFor('audio/webm'));           // m4a webm
console.log(extFor('video/x-matroska'), extFor('text/plain'));    // mkv bin
module Five MIME types and the extension each one gets.

Related

  • pickMime() Picks the first MIME type from a list that the browser can record, using the browser's own check.
  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
  • VIDEO_MIMES The video MIME types the recorder tries, best first: MP4 with H.264 and AAC, then WebM.
  • AUDIO_MIMES The audio MIME types the recorder tries for a sound-only file, best first: AAC in .m4a, then Opus.

import { extFor } from '@openav/record'; packages/record/mime.js, line 57

framePoint()#

function@openav/record
framePoint(camRect, u, v)

Turns a point in the camera's own 0..1 coordinates, like a hand landmark, into a pixel in the output frame.

Landmarks from a hand or pose tracker come as fractions of the camera picture. To draw something on the output at the place a landmark is, you need pixels of the output, where the camera may be enlarged and cropped. framePoint() does that. It gives the same point whether or not part of the camera is cropped away, because it uses frame, the whole camera picture placed in the output. Use the same mirror in the tracker and in the compositor.

Parameters

NameTypeDescription
camRectObject | nullThe cam of layoutRects(), or of compositor.rects.
uNumberAcross the camera picture, 0 left to 1 right, in the displayed (mirrored if mirrorCam) orientation.
vNumberDown the camera picture, 0 top to 1 bottom.

Returns

Object | null { x, y } in output pixels, or null when there is no camera (or its size is not known yet).

Examples

js
import { layoutRects, framePoint } from '@openav/record';

const r = layoutRects('stack', { w: 1080, h: 1920 }, { workW: 1000, workH: 1000, camW: 1920, camH: 1080 });
console.log(framePoint(r.cam, 0.5, 0.5));    // { x: 540, y: 1500 }: the middle of the camera
console.log(framePoint(r.cam, 0, 0));        // { x: -206.67, y: 1080 }: the left edge is cropped away, off the frame
console.log(framePoint(null, 0.5, 0.5));     // null: no camera
module The centre of the camera lands in the middle of its area. The camera's left edge is outside the frame because the 16:9 picture is cropped to fit.

Related

  • layoutRects() Works out where the work and the camera go in an output frame, in whole pixels, for one of the four layouts.
  • Compositor Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.

import { framePoint } from '@openav/record'; packages/record/layout.js, line 219

fromMidiFile()#

function@openav/record
fromMidiFile(bytes, { name, families = 'both' } = {})

Reads a Standard MIDI File into a take whose events are the signals a keyboard playing that file would publish.

It reads type 0 and type 1 files (the tracks are merged), running status, tempo changes and SMPTE time division; system exclusive messages are skipped. A file that is not a MIDI file throws a TypeError (fromMidiFile: not a MIDI file (no MThd)), and one that ends early a RangeError. Play the result into a show with TakePlayer: it drives a work like someone playing the file on a keyboard.

Parameters

NameTypeDescription
bytesUint8Array | ArrayBuffer | Number[]The bytes of the .mid file, for example await file.arrayBuffer().
nameStringThe name of the take. Left out, it is the track name in the file, then 'take'. optional
families'both' | 'legacy' | 'channel'Which signal spellings to produce, as in signalsOfMidi(). The default is both, like hardware. optional default 'both'

Returns

Object A take. meta.bpm is the first tempo in the file (not set for SMPTE time).

Examples

js
import { toMidiFile, fromMidiFile } from '@openav/record';

const take = { v: 1, name: 'riff', durationMs: 1500, meta: { bpm: 120 }, events: [
  { t: 500, name: 'midi/note/on', value: { note: 60, vel: 0.8, ch: 1 } },
  { t: 900, name: 'midi/note/off', value: { note: 60, ch: 1 } },
] };
const back = fromMidiFile(toMidiFile(take));
console.log(back.name, back.durationMs, back.meta.bpm, back.events.length);   // riff 1500 120 4
console.log(fromMidiFile(toMidiFile(take), { families: 'legacy' }).events.map((e) => e.name));
// ['midi/note/on', 'midi/note/off']
try { fromMidiFile(new Uint8Array([1, 2, 3, 4, 5])); } catch (e) { console.log(e.message); }
// fromMidiFile: not a MIDI file (no MThd)
module A round trip: the two notes come back as four events (both spellings), or two with families: 'legacy'. Bytes that are not a MIDI file are refused.

Related

  • toMidiFile() Writes a take as a Standard MIDI File (type 0) that opens in any DAW.
  • signalsOfMidi() Turns one MIDI event into the signal events a keyboard playing it would publish, so a file can drive a work like live playing.
  • TakePlayer Plays a take back into a bus at the right times, so the work, its sound and its chord detector cannot tell it from you playing.
  • normalizeTake() Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.

import { fromMidiFile } from '@openav/record'; packages/record/smf.js, line 85

layoutRects()#

function@openav/record
layoutRects(layout, out, src = {}, opts = {})

Works out where the work and the camera go in an output frame, in whole pixels, for one of the four layouts.

It is pure arithmetic: no DOM, no clock. The Compositor calls it whenever a size changes. A 1080 by 1920 frame with a square work and stack splits into exactly 1080 by 1080 and 1080 by 840, with no seam row. The crop of a camera is in the displayed orientation, so with the mirror on, an anchor of x: 0.2 still means "keep the left of what you see".

stack puts the work across the full width at the top and the camera below. If that would leave the camera under minCam of the height (a square or landscape frame), the work gets split of the height and is letterboxed in it. side is the same turned sideways. pip fills the frame with the work and puts the camera in a corner as a rounded inset. work is the work alone.

Parameters

NameTypeDescription
layoutStringOne of LAYOUTS. Any other name throws a TypeError.
outObjectThe output size { w, h } in pixels. It must be positive, or this throws.
srcObjectThe sizes of the sources: { workW, workH, camW, camH }. A missing or zero value means unknown. An unknown work counts as square. With an unknown camera the camera's area is still reserved, with crop: null and frame: null, so the frame does not jump when the camera arrives. optional default {}
optsObjectOptions: fit ('contain' default, or 'cover', for the work), camFit ('cover' default, or 'contain'), camAnchor ({ x, y }, 0..1, default the middle: which part a cover-crop keeps, in what you see), minCam (0.2: the share of the height in stack, or of the width in side, that the camera must keep), split (0.6: the work's share when it would leave the camera less than minCam) and pip ({ corner: 'br' | 'bl' | 'tr' | 'tl', size: 0.3, margin: 0.03, radius: 0.08, aspect: 0 }: the inset's width as a share of the frame, its distance from the corner as a share of the short side, its corner radius as a share of the inset's short side, and its shape, 0 being the camera's own). optional default {}

Returns

Object { layout, out, work, cam }. work and cam are { x, y, w, h, crop, frame, box } in output pixels (all whole numbers), and cam also has radius. crop is the part of the source that is drawn (source pixels, may be fractional) or null for the whole source. frame is where the whole source frame would land, cropped parts falling outside the rect, which is what overlays need. box is the area it was placed in. cam is null for the work layout.

Examples

js
import { layoutRects } from '@openav/record';

const r = layoutRects('stack', { w: 1080, h: 1920 }, { workW: 1000, workH: 1000, camW: 1920, camH: 1080 });
console.log(r.work.x, r.work.y, r.work.w, r.work.h);   // 0 0 1080 1080
console.log(r.cam.x, r.cam.y, r.cam.w, r.cam.h);       // 0 1080 1080 840
console.log(r.cam.crop);                                 // the middle of the 16:9 camera that fills 1080x840
module A square work and a 16:9 camera in a vertical frame: the work is the top 1080 pixels, the camera fills the 840 below it with a middle crop.
js
import { layoutRects } from '@openav/record';

const pip = layoutRects('pip', { w: 1920, h: 1080 }, { workW: 1920, workH: 1080, camW: 1280, camH: 720 },
  { pip: { corner: 'tl', size: 0.25 } });
console.log(pip.cam.x, pip.cam.y, pip.cam.w, pip.cam.h, pip.cam.radius);   // 32 32 480 270 22

const noCam = layoutRects('stack', { w: 1080, h: 1920 }, { workW: 1000, workH: 1000 });
console.log(noCam.cam.h, noCam.cam.crop, noCam.cam.frame);                  // 840 null null: the area is reserved
try { layoutRects('grid', { w: 10, h: 10 }); } catch (e) { console.log(e.message); }
// layoutRects: unknown layout "grid" (stack, pip, side, work)
module A picture-in-picture in the top-left corner, the reserved camera area before a camera exists, and the error for a layout that does not exist.

Related

  • LAYOUTS The four layout names, in picker order: 'stack', 'pip', 'side', 'work'.
  • framePoint() Turns a point in the camera's own 0..1 coordinates, like a hand landmark, into a pixel in the output frame.
  • Compositor Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
  • resolveSize() Turns any way of naming a size, like 'vertical-4k', '1080x1920' or [w, h], into { w, h }, rounded up to even numbers.

import { layoutRects } from '@openav/record'; packages/record/layout.js, line 163

LAYOUTS#

constant@openav/record
LAYOUTS = array, 4 items

The four layout names, in picker order: 'stack', 'pip', 'side', 'work'.

stack: the work across the full width at the top, the camera fills the rest below (for vertical video: Reels, Shorts, TikTok). pip: the work fills the frame and the camera is a rounded inset in a corner (for square frames, or a work that needs all of the frame). side: the work at full height on the left, the camera fills the right (landscape). work: the work alone. layoutRects() says exactly where everything lands, and the layout option of the Compositor takes one of these names; any other name throws.

Type
String[]

['stack', 'pip', 'side', 'work'], a frozen-by-convention list (do not change it).

Examples

js
import { LAYOUTS, layoutRects } from '@openav/record';

console.log(LAYOUTS.join(' '));                                    // stack pip side work
for (const name of LAYOUTS) {
  const r = layoutRects(name, { w: 1080, h: 1920 }, { workW: 1000, workH: 1000, camW: 1920, camH: 1080 });
  console.log(name, r.work.w + 'x' + r.work.h, r.cam ? r.cam.w + 'x' + r.cam.h : 'no camera');
}
// stack 1080x1080 1080x840
// pip 1080x1080 324x182
// side 648x648 432x1920   (a vertical frame is too narrow for side: the work gets 60% of the width)
// work 1080x1080 no camera
module Runs the four layouts on one vertical frame with a square work and a 16:9 camera, and logs the size of each area.

Related

  • layoutRects() Works out where the work and the camera go in an output frame, in whole pixels, for one of the four layouts.
  • Compositor Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
  • PRESETS The output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.

import { LAYOUTS } from '@openav/record'; packages/record/layout.js, line 14

listCameras()#

function@openav/record
listCameras()

Lists the video inputs of this computer as { deviceId, groupId, label }.

Browsers hide the labels (and sometimes the ids) until the page has had camera permission once, so call it again after openCamera() to get the real names. Needs https or localhost: elsewhere there is no navigator.mediaDevices and the promise rejects.

Returns

Promise<Object[]> A promise of a list of { deviceId, groupId, label }. A device without a label is called Camera 1, Camera 2… by its position.

Examples

js
import { listCameras, openCamera } from '@openav/record';

console.log((await listCameras()).length > 0);        // true; the labels are generic until permission is given
const stream = await openCamera();                      // the browser asks for the camera
const cams = await listCameras();                       // now with their real names
console.log(cams.map((c) => c.label).join(', '));
stream.getTracks().forEach((t) => t.stop());
module Lists the cameras before and after permission is given; the second list has the real names. Stop the tracks when you are done.

Related

  • listMicrophones() Lists the audio inputs of this computer as { deviceId, groupId, label }.
  • openCamera() Opens a camera and returns its MediaStream: ideally 1920 by 1080 at 30 fps, or whatever the camera can do.
  • cameraConstraints() Builds the getUserMedia options for a camera: it asks for 1920 by 1080 at 30 fps, but only as an ideal, never as a demand.

import { listCameras } from '@openav/record'; packages/record/camera.js, line 36

listMicrophones()#

function@openav/record
listMicrophones()

Lists the audio inputs of this computer as { deviceId, groupId, label }.

The same as listCameras() for audio, with the same rule about labels: they are generic until the page has had microphone permission once. Pass a deviceId to AudioTap.addMicrophone() to use one.

Returns

Promise<Object[]> A promise of a list of { deviceId, groupId, label }. A device without a label is called Microphone 1, Microphone 2… by its position.

Examples

js
import { listMicrophones } from '@openav/record';

const mics = await listMicrophones();
console.log(mics.length > 0, Object.keys(mics[0]).join(','));   // true deviceId,groupId,label
module Lists the audio inputs and shows the shape of one item.

Related

  • listCameras() Lists the video inputs of this computer as { deviceId, groupId, label }.
  • AudioTap Collects the sound of your work (and, if you add it, a microphone) into one recordable track, without sending anything to the speakers.
  • openCamera() Opens a camera and returns its MediaStream: ideally 1920 by 1080 at 30 fps, or whatever the camera can do.

import { listMicrophones } from '@openav/record'; packages/record/camera.js, line 39

midiEventsOf()#

function@openav/record
midiEventsOf(take)

Reads a take as plain MIDI events (note on and off, controllers, pitch bend), picking one naming family so nothing is counted twice.

A take made on hardware holds the same note several times, under several names. This picks one family for the notes: midi/note/* if present, else midi/ch/*, else one device's midi/<device>/note/*, else raw midi/virtual bytes. Controllers and pitch bend come from midi/ch/* if present, else the classic names (as channel 1), else one device's. toMidiFile() and takeStats() are built on it.

Parameters

NameTypeDescription
takeObjectThe take.

Returns

Object[] A list in the shape of @openav/midi: { t, type: 'noteon', ch, note, vel }, noteoff, { t, type: 'cc', ch, cc, value }, { t, type: 'pitchbend', ch, value } (0..16383), chanat and polyat. vel and value are 0..127.

Examples

js
import { midiEventsOf } from '@openav/record';

const take = { v: 1, name: 'x', durationMs: 600, events: [
  { t: 100, name: 'midi/note/on', value: { note: 60, vel: 0.8, ch: 1 } },
  { t: 100, name: 'midi/ch/1/note/60', value: 0.8 },        // the same key, spelled the other way
  { t: 400, name: 'midi/note/off', value: { note: 60, ch: 1 } },
  { t: 400, name: 'midi/ch/1/note/60', value: 0 },
] };
console.log(midiEventsOf(take));
// [{ t: 100, type: 'noteon', ch: 1, note: 60, vel: 102 }, { t: 400, type: 'noteoff', ch: 1, note: 60, vel: 0 }]
module Four recorded events are one key pressed and released, spelled twice; the result has just the two MIDI events.

Related

  • toMidiFile() Writes a take as a Standard MIDI File (type 0) that opens in any DAW.
  • takeStats() Counts what is in a take: notes, events, length, the channels used, and the lowest and highest note.
  • noteOf() Tells whether a recorded event is a key going down or up, whatever its spelling, and which note on which channel.
  • signalsOfMidi() Turns one MIDI event into the signal events a keyboard playing it would publish, so a file can drive a work like live playing.

import { midiEventsOf } from '@openav/record'; packages/record/take.js, line 144

mixTracks()#

function@openav/record
mixTracks(tracks)

Mixes several audio tracks into one, because Chrome records only the first audio track of a stream.

A Recorder calls this itself when you give it more than one live audio track, and closes the mix on stop(). Call it yourself when you want the one track for something else. It makes an AudioContext, so the page needs to have had a click. For several sounds that come from Web Audio nodes, AudioTap is the better tool: it mixes them without a detour through streams.

Parameters

NameTypeDescription
tracksMediaStreamTrack[]The audio tracks to mix.

Returns

Object { track, close() }: the one mixed MediaStreamTrack, and a function that closes the AudioContext that mixes it.

Examples

js
import { AudioTap, mixTracks } from '@openav/record';

const tap = new AudioTap();
const mix = mixTracks([tap.tracks[0], tap.workTracks[0]]);   // two tracks in, one comes out
console.log(mix.track.kind, mix.track.readyState);            // audio live
mix.close();                                                  // closes the mixing context
tap.dispose();
module Two audio tracks go in and one live audio track comes out; close() shuts the mixer down.

Related

  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
  • AudioTap Collects the sound of your work (and, if you add it, a microphone) into one recordable track, without sending anything to the speakers.

import { mixTracks } from '@openav/record'; packages/record/recorder.js, line 23

normalizeTake()#

function@openav/record
normalizeTake(input)

Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.

A take is what an EventLog filtered to midi/* records, as plain data: t is milliseconds from the start, name is a signal name such as midi/note/on, value is its payload, and meta holds free fields like { bpm, device, notes }. normalizeTake() makes any input safe: events that are not objects, have no name or have a t that is not a number are dropped; t is rounded to a tenth of a millisecond and held at 0 or more; the events are sorted by t (events with equal times keep their order); name defaults to 'take'; durationMs is at least the time of the last event. A new object is returned.

Every function in this package that takes a take runs it through here first, so you can pass raw data.

Parameters

NameTypeDescription
inputObject | StringA take, or its JSON text. A string is parsed with JSON.parse. Anything that is not an object throws a TypeError.

Returns

Object A new take: { v: 1, name, createdAt, durationMs, events: [{ t, name, value }], meta }.

Examples

js
import { normalizeTake } from '@openav/record';

const take = normalizeTake({
  events: [{ t: 30, name: 'b' }, { t: 10, name: 'a', value: 1 }, { t: 'x', name: 'c' }, { name: 'd' }, null],
  durationMs: 5,
});
console.log(take.events.map((e) => e.t + ':' + e.name).join(' '));   // 10:a 30:b
console.log(take.durationMs, take.name, take.v);                     // 30 take 1
module Three of the five events are junk and go; the rest are sorted, and the length grows to cover the last event.

Related

  • validateTake() Checks a take and lists everything wrong with it as readable sentences. An empty list means it is valid.
  • TAKE_VERSION The version number of the take format, which is 1. Every take has v: 1.
  • trimSilence() Cuts the silence before the first key press of a take, so the first note comes at the moment you choose.
  • EventLog Keeps a timestamped list of what happened during a take, such as notes and parameter changes, on the same clock as the recording.
  • TakePlayer Plays a take back into a bus at the right times, so the work, its sound and its chord detector cannot tell it from you playing.

import { normalizeTake } from '@openav/record'; packages/record/take.js, line 30

noteOf()#

function@openav/record
noteOf(ev)

Tells whether a recorded event is a key going down or up, whatever its spelling, and which note on which channel.

A hardware keyboard publishes the same note under several names, and this function knows them all: midi/note/on and midi/note/off (family legacy, with value: { note, vel, ch }), midi/ch/<ch>/note/<note> (family channel, a number, 0 for off), midi/<device>/note/on|off (family slug:<device>) and raw bytes on midi/virtual (family virtual, { data: [status, note, vel] }). A key-down with velocity 0 in raw bytes counts as off. A channel of 0, as the on-screen piano says, counts as channel 1.

Parameters

NameTypeDescription
evObjectAn event { name, value } of a take.

Returns

Object | null { key, on, note, ch, family, vel }, or null if it is not a note event. key is '<ch>:<note>', on is true for a key-down, family says which spelling it came in, vel is 0..127.

Examples

js
import { noteOf } from '@openav/record';

console.log(noteOf({ name: 'midi/note/on', value: { note: 60, vel: 0.8, ch: 1 } }));
// { key: '1:60', on: true, note: 60, ch: 1, family: 'legacy', vel: 102 }
console.log(noteOf({ name: 'midi/ch/1/note/60', value: 0 }).on);                   // false
console.log(noteOf({ name: 'midi/virtual', value: { data: [0x90, 60, 100] } }).on);  // true
console.log(noteOf({ name: 'midi/cc/1', value: 1 }));                                // null
module The same key in three spellings, and a controller that is not a note.

Related

  • noteOffFor() Gives the events that release a sounding note, in the same spelling it was played in.
  • midiEventsOf() Reads a take as plain MIDI events (note on and off, controllers, pitch bend), picking one naming family so nothing is counted twice.
  • TakePlayer Plays a take back into a bus at the right times, so the work, its sound and its chord detector cannot tell it from you playing.
  • TakeRecorder Records what you play: it listens to every midi/* signal on a bus from start() to stop() and gives you a take.

import { noteOf } from '@openav/record'; packages/record/take.js, line 99

noteOffFor()#

function@openav/record
noteOffFor(n)

Gives the events that release a sounding note, in the same spelling it was played in.

This is how a TakePlayer and a TakeRecorder make sure no note is left sounding: for legacy it is midi/note/off, for channel it is midi/ch/<ch>/note/<note> with 0, for virtual it is a raw note-off byte triple, for a device family midi/<device>/note/off.

Parameters

NameTypeDescription
nObjectThe result of noteOf(): { note, ch, family }.

Returns

Object[] A list of { name, value } events (one, or none for an unknown family).

Examples

js
import { noteOf, noteOffFor } from '@openav/record';

const down = noteOf({ name: 'midi/ch/3/note/60', value: 0.5 });
console.log(noteOffFor(down));      // [{ name: 'midi/ch/3/note/60', value: 0 }]
const legacy = noteOf({ name: 'midi/note/on', value: { note: 60, vel: 0.8, ch: 1 } });
console.log(noteOffFor(legacy));    // [{ name: 'midi/note/off', value: { note: 60, ch: 1 } }]
module The release of the same key depends on how it was played: the per-channel spelling releases with a zero, the classic one with a note-off.

Related

  • noteOf() Tells whether a recorded event is a key going down or up, whatever its spelling, and which note on which channel.
  • TakePlayer Plays a take back into a bus at the right times, so the work, its sound and its chord detector cannot tell it from you playing.
  • TakeRecorder Records what you play: it listens to every midi/* signal on a bus from start() to stop() and gives you a take.

import { noteOffFor } from '@openav/record'; packages/record/take.js, line 126

openCamera()#

function@openav/record
openCamera(opts = {})

Opens a camera and returns its MediaStream: ideally 1920 by 1080 at 30 fps, or whatever the camera can do.

Give the stream to Compositor.setCamera(). The caller owns the stream: stop its tracks when you are done, the compositor never does. Webcams usually top out at 1080p30, so a 4K frame enlarges the camera; the work above it stays sharp. It needs https or localhost, and the user's permission.

Parameters

NameTypeDescription
optsObjectThe options of cameraConstraints(): { deviceId, width, height, fps, facingMode }. optional default {}

Returns

Promise<MediaStream> A promise of the stream, with one video track and no audio. It rejects with the browser's error when permission is refused or the camera is missing.

Examples

js
import { openCamera } from '@openav/record';

const stream = await openCamera();                // the browser asks for the camera
const video = document.createElement('video');
video.muted = true; video.srcObject = stream; await video.play();
console.log(video.videoWidth > 0);                // true once frames arrive
document.getElementById('app').append(video);
video.style.width = '240px';
module Allow the camera and a small live picture of it appears on the page.

Related

  • Compositor Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
  • cameraConstraints() Builds the getUserMedia options for a camera: it asks for 1920 by 1080 at 30 fps, but only as an ideal, never as a demand.
  • listCameras() Lists the video inputs of this computer as { deviceId, groupId, label }.

import { openCamera } from '@openav/record'; packages/record/camera.js, line 47

pickMime()#

function@openav/record
pickMime(isTypeSupported, { audioOnly = false, prefer = null, candidates = null } = {})

Picks the first MIME type from a list that the browser can record, using the browser's own check.

A Recorder calls this for you. It takes the browser's check as an argument so that it is pure and can be tried with a fake one, like the examples below.

Parameters

NameTypeDescription
isTypeSupportedFunctionThe browser's check, for example (m) => MediaRecorder.isTypeSupported(m). Not a function gives ''. A check that throws counts as "no" for that type.
audioOnlyBooleanUse AUDIO_MIMES instead of VIDEO_MIMES. optional default false
prefer'mp4' | 'webm'Move this container to the front, keeping the order inside each group. optional default null
candidatesString[]Your own list instead of the built-in one. optional default null

Returns

String The first supported type, or '' when none matches (then let MediaRecorder choose its default).

Examples

js
import { pickMime } from '@openav/record';

const pretend = (...ok) => (mime) => ok.includes(mime);      // a fake browser that supports only these
const both = pretend('video/webm;codecs=vp9,opus', 'video/mp4;codecs=avc1.640033,mp4a.40.2');
console.log(pickMime(both));                                  // video/mp4;codecs=avc1.640033,mp4a.40.2
console.log(pickMime(both, { prefer: 'webm' }));              // video/webm;codecs=vp9,opus
console.log(pickMime(pretend('audio/webm;codecs=opus'), { audioOnly: true }));   // audio/webm;codecs=opus
console.log(JSON.stringify(pickMime(() => false)));           // ""
module Four fake browsers: one that has both, the same asked to prefer WebM, one that only records Opus, and one that records nothing.

Related

  • VIDEO_MIMES The video MIME types the recorder tries, best first: MP4 with H.264 and AAC, then WebM.
  • AUDIO_MIMES The audio MIME types the recorder tries for a sound-only file, best first: AAC in .m4a, then Opus.
  • extFor() Gives the file extension for a recorded MIME type: video/mp4 is 'mp4', audio/mp4 is 'm4a', anything unknown is 'bin'.
  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.

import { pickMime } from '@openav/record'; packages/record/mime.js, line 42

presetById()#

function@openav/record
presetById(id)

Finds one size preset by its id: presetById('vertical-4k').

Use it to fill a size picker with PRESETS, or to read the suggested frame rate and layout of a size before you build a Compositor.

Parameters

NameTypeDescription
idStringThe preset id, for example 'square-1080'.

Returns

Object | null The preset { id, group, w, h, label, labelZh, layout, fps }, or null if there is no such id.

Examples

js
import { presetById } from '@openav/record';

const p = presetById('vertical-4k');
console.log(p.w, p.h, p.fps, p.layout);   // 2160 3840 30 stack
console.log(presetById('nope'));           // null
module Logs the numbers of one preset and shows that an unknown id gives null.

Related

  • PRESETS The output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.
  • resolveSize() Turns any way of naming a size, like 'vertical-4k', '1080x1920' or [w, h], into { w, h }, rounded up to even numbers.

import { presetById } from '@openav/record'; packages/record/layout.js, line 69

PRESETS#

constant@openav/record
PRESETS = array, 11 items

The output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.

text
id                 size        fps   layout
vertical-1080p     1080x1920   60    stack
vertical-4x5       1440x1800   60    stack
vertical-2.7k      1520x2704   30    stack
vertical-4k        2160x3840   30    stack
landscape-1080p    1920x1080   60    side
landscape-16x10    1920x1200   60    side
landscape-2.7k     2704x1520   30    side
landscape-4k       3840x2160   30    side
square-1080        1080x1080   60    pip
square-2048        2048x2048   30    pip
square-2160        2160x2160   30    pip

Every side is even, because H.264 and VP9 encode 4:2:0 colour, which needs even sizes. Use an id as the size of a Compositor; presetById() looks one up.

Type
Object[]

A frozen list of 11 presets. Each is { id, group, w, h, label, labelZh, layout, fps }: group is 'vertical', 'landscape' or 'square', label and labelZh are names for a picker, layout is the layout suggested for the shape and fps is recommendedFps().

Examples

js
import { PRESETS } from '@openav/record';

console.log(PRESETS.length, PRESETS[0].id);                      // 11 vertical-1080p
const big = PRESETS.filter((p) => p.w * p.h > 2.6e6).map((p) => p.id + ' @' + p.fps);
console.log(big.join(', '));
// vertical-2.7k @30, vertical-4k @30, landscape-2.7k @30, landscape-4k @30, square-2048 @30, square-2160 @30
module Lists the presets that are bigger than about 2.6 megapixels: all of them are recorded at 30 frames a second.

Related

  • presetById() Finds one size preset by its id: presetById('vertical-4k').
  • resolveSize() Turns any way of naming a size, like 'vertical-4k', '1080x1920' or [w, h], into { w, h }, rounded up to even numbers.
  • recommendedFps() The frame rate to record a frame size at: 60 up to about 2.6 megapixels, 30 above that.
  • Compositor Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.

import { PRESETS } from '@openav/record'; packages/record/layout.js, line 54

quantizeTake()#

function@openav/record
quantizeTake(take, { bpm, grid = 4, strength = 1, offsetMs = 0 } = {})

Snaps the key presses of a take to a rhythmic grid, so a loose performance lands on the beat.

Each key-down moves to the nearest grid line (60000 / bpm / grid milliseconds apart). The key-up of a note moves with it, so the length of every note is kept. Everything that is not a note (controllers, pitch bend) stays where it was. A note written in two spellings at the same time (midi/note/on and midi/ch/1/note/60) moves by the same amount in both.

Parameters

NameTypeDescription
takeObjectThe take.
bpmNumberThe tempo. Left out, it is take.meta.bpm, and then 120. optional
gridNumberGrid steps per beat: 4 is sixteenths, 2 is eighths, 1 is quarters. optional default 4
strengthNumberHow far to move, 0..1. 1 is all the way, 0.5 is half way to the grid, 0 changes nothing. optional default 1
offsetMsNumberShift the grid later by this many milliseconds, for a take that does not start on a beat. optional default 0

Returns

Object A new take.

Examples

js
import { quantizeTake } from '@openav/record';

const on = (t, note) => ({ t, name: 'midi/note/on', value: { note, vel: 0.8, ch: 1 } });
const off = (t, note) => ({ t, name: 'midi/note/off', value: { note, ch: 1 } });
const take = { v: 1, name: 'loose', durationMs: 1500, meta: { bpm: 120 },
  events: [on(510, 60), off(900, 60), on(1010, 64), off(1400, 64)] };

const q = quantizeTake(take, { grid: 4 });              // sixteenths at 120 bpm: a grid line every 125 ms
console.log(q.events.map((e) => e.t));                  // [500, 890, 1000, 1390]
console.log(quantizeTake(take, { grid: 4, strength: 0.5 }).events.map((e) => e.t));   // [505, 895, 1005, 1395]
module Two notes a little off the beat snap to 500 and 1000 ms, and each note-off moves with its note. With a strength of one half they only go halfway.

Related

  • trimSilence() Cuts the silence before the first key press of a take, so the first note comes at the moment you choose.
  • takeStats() Counts what is in a take: notes, events, length, the channels used, and the lowest and highest note.
  • toMidiFile() Writes a take as a Standard MIDI File (type 0) that opens in any DAW.
  • normalizeTake() Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.

import { quantizeTake } from '@openav/record'; packages/record/take.js, line 259

recommendedBitrate()#

function@openav/record
recommendedBitrate(w, h, fps = recommendedFps(w, h))

A video bitrate that keeps fast generative detail clean: 0.12 bits per pixel per frame, held between 4 and 80 Mb/s.

Particles and thin lines are the first things a low bitrate smears, so this is higher than a streaming bitrate on purpose: the file is a master that will be edited and encoded again. It is the default videoBitsPerSecond of a Recorder that records a canvas.

Parameters

NameTypeDescription
wNumberWidth in pixels.
hNumberHeight in pixels.
fpsNumberThe frame rate. Left out, it is recommendedFps() for the size. optional default recommendedFps(w, h)

Returns

Number Bits per second, rounded to 100 000. 1080 by 1920 at 60 fps is 14 900 000, 4K at 30 fps is 29 900 000.

Examples

js
import { recommendedBitrate } from '@openav/record';

const mbps = (w, h, fps) => (recommendedBitrate(w, h, fps) / 1e6).toFixed(1) + ' Mb/s';
console.log(mbps(1080, 1920));       // 14.9 Mb/s
console.log(mbps(2160, 3840));       // 29.9 Mb/s
console.log(mbps(320, 240));         // 4.0 Mb/s, the floor
console.log(mbps(3840, 2160, 60));   // 59.7 Mb/s
module The bitrate for a few sizes, in megabits per second: the floor of 4 holds for a tiny frame, and asking for 60 fps at 4K doubles the number.

Related

  • recommendedFps() The frame rate to record a frame size at: 60 up to about 2.6 megapixels, 30 above that.
  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.

import { recommendedBitrate } from '@openav/record'; packages/record/layout.js, line 37

recommendedFps()#

function@openav/record
recommendedFps(w, h)

The frame rate to record a frame size at: 60 up to about 2.6 megapixels, 30 above that.

60 fps holds up to about 1080 by 1920 (and 1440 by 1800, 1920 by 1200). A 4K canvas at 60 means 500 million pixels a second through the compositor and the encoder, which drops frames on a laptop in the middle of a performance, and 30 is what cameras shoot 4K at anyway. Pass the result as fps to a Recorder.

Parameters

NameTypeDescription
wNumberWidth in pixels.
hNumberHeight in pixels.

Returns

30 | 60 60 when w × h is at most 2 600 000 pixels, otherwise 30.

Examples

js
import { recommendedFps } from '@openav/record';

console.log(recommendedFps(1080, 1920));   // 60
console.log(recommendedFps(2160, 3840));   // 30
console.log(recommendedFps(1920, 1200));   // 60
module Logs the rate for a 1080p vertical frame, a 4K vertical frame and a 16:10 landscape frame.

Related

  • recommendedBitrate() A video bitrate that keeps fast generative detail clean: 0.12 bits per pixel per frame, held between 4 and 80 Mb/s.
  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.
  • PRESETS The output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.

import { recommendedFps } from '@openav/record'; packages/record/layout.js, line 25

Recorder#

class@openav/record
new Recorder({ canvas = null, stream = null, fps = 60, audioTracks = [], audioOnly = false, videoBitsPerSecond, audioBitsPerSecond = 192000, mimeType, prefer = null } = {})

Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.

new Recorder({ canvas, audioTracks }) then start() and, later, await stop(). The canvas track and the audio track go into the same MediaRecorder, which stamps both from one clock: nothing has to be lined up afterwards. stop() resolves, after the last chunk, with { mimeType, ext, bytes, chunks, durationMs, width, height, fps, audio, blob, error }: the blob is the file, ext the extension to give it (mp4, webm, m4a…), audio whether it has sound. Give the blob to URL.createObjectURL() for a download link.

Which file you get depends on the browser: it records MP4 (H.264 and AAC) when it can, WebM otherwise. A WebM from Chrome has no duration in its header; players cope, editors may want ffmpeg -i in.webm -c copy out.webm. The file is a master to edit, so the default bitrate is high.

To keep a long take out of the tab's memory, pass ondata(blob, seq) to start(): each chunk (every timeslice ms) is handed over in order, and a promise you return is awaited before the next chunk and before stop() resolves, so uploads stay ordered. With ondata the chunks are not kept and blob is null, unless you pass keep: true. To line a take up with an EventLog, start the log with recorder.t0, the performance.now() of the moment recording started.

Before you call it

A browser with MediaRecorder (Recorder.supported tells), and a canvas or a stream, or audioOnly: true: otherwise the constructor throws a TypeError. Mixing several audio tracks makes an AudioContext, which needs the page to have had a click. A camera or microphone in the stream needs the user's permission, as always.

Parameters

NameTypeDescription
canvasHTMLCanvasElementWhat to record, usually compositor.canvas. optional default null
streamMediaStreamA ready video stream to record instead of a canvas. optional default null
fpsNumberThe capture rate of the canvas. Use recommendedFps(w, h): 30 for 2.7K and 4K. optional default 60
audioTracksMediaStreamTrack[]The sound, for example tap.tracks of an AudioTap. Several tracks are mixed into one, because Chrome records only the first audio track of a stream. optional default []
audioOnlyBooleanRecord only the audio tracks: the sound as a file of its own. Then canvas and stream are not needed. optional default false
videoBitsPerSecondNumberThe video bitrate. The default is recommendedBitrate() of the canvas size and fps (12 Mb/s when there is no canvas). optional
audioBitsPerSecondNumberThe audio bitrate. optional default 192000
mimeTypeStringForce one MIME type. The default is the first one the browser supports from VIDEO_MIMES or AUDIO_MIMES, see pickMime(). optional
prefer'mp4' | 'webm'Try this container first when picking the MIME type. optional default null

Members

NameDescription
stategetterGetter. 'inactive', 'recording' or 'paused'.
recordinggetterGetter. Whether state is 'recording'.
elapsedgetterGetter. Milliseconds since start() while recording, the length of the take once stopped, 0 before.
bytesgetterGetter. Bytes handed over so far. Some browsers hand over the data only when you stop, so it can stay 0 during a short take.
start({ timeslice = 1000, ondata = null, keep } = {})methodStarts recording and returns the recorder. { timeslice = 1000, ondata, keep }: how often (ms) a chunk is handed over, a function for each chunk, and whether to keep the chunks for the blob. Starting a recorder that already records does nothing. Throws if the browser has no MediaRecorder.
stop()methodStops, waits for the last chunk and every ondata promise, stops the canvas track it made and returns a promise of the result object. Calling it again returns the same result. Throws if the recorder was never started.
supportedstatic propertyStatic getter. True when this browser can record a canvas: it has MediaRecorder and canvas.captureStream.

Examples

js
import { Recorder } from '@openav/record';

const canvas = document.getElementById('c');
const g = canvas.getContext('2d');
let x = 0;
(function animate() {                          // something moving to record
  g.fillStyle = '#111'; g.fillRect(0, 0, 400, 300);
  g.fillStyle = '#ffd166'; g.fillRect(x = (x + 5) % 380, 120, 20, 60);
  requestAnimationFrame(animate);
})();

const rec = new Recorder({ canvas, fps: 30 });
rec.start();
await new Promise((done) => setTimeout(done, 1000));   // record for one second
const take = await rec.stop();
console.log(take.ext, take.width, take.height, take.fps, take.audio, take.blob.size > 0);
// mp4 400 300 30 false true   (webm where the browser cannot write mp4)

const a = document.getElementById('app').appendChild(document.createElement('a'));
a.href = URL.createObjectURL(take.blob); a.download = 'take.' + take.ext; a.textContent = 'Download';
module Records one second of the moving square and logs what the file is. A Download link appears on the page.
js
import { Recorder, AudioTap } from '@openav/record';

const tap = new AudioTap();                                  // the sound that goes into the file
const osc = new OscillatorNode(tap.context, { frequency: 220 });
tap.add(osc); osc.start();
await tap.resume();                                          // from a click, in a real page

const video = new Recorder({ canvas: document.getElementById('c'), fps: 30, audioTracks: tap.tracks });
const sound = new Recorder({ audioOnly: true, audioTracks: tap.workTracks });   // the sound as its own file
video.start(); sound.start();
await new Promise((done) => setTimeout(done, 1000));
const [v, a] = await Promise.all([video.stop(), sound.stop()]);
console.log(v.ext, v.audio, a.ext, a.width);   // mp4 true m4a 0
module One second of video with a 220 Hz tone in it, and the same tone as a separate audio file. The picture and the sound of the video enter the same recorder.
js
import { Recorder } from '@openav/record';

const chunks = [];
const rec = new Recorder({ canvas: document.getElementById('c'), fps: 30 });
rec.start({
  timeslice: 250,
  ondata: async (blob, seq) => { chunks.push(blob.size); },   // send it somewhere here; the next chunk waits
});
await new Promise((done) => setTimeout(done, 1000));
const take = await rec.stop();
console.log(take.blob, take.chunks === chunks.length, take.bytes === chunks.reduce((a, b) => a + b, 0));   // null true true
module With ondata the chunks are handed to you one by one, in order, and the result carries no blob: this is how a long take stays out of the tab's memory.

To be confirmed

Whether data arrives while the recording is still running depends on the container and the browser. In headless Chromium, writing MP4, bytes stayed 0 for a one-second take with a timeslice of 200 and the chunks came at stop(). Other browsers and WebM were not tried.

Related

  • Compositor Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
  • AudioTap Collects the sound of your work (and, if you add it, a microphone) into one recordable track, without sending anything to the speakers.
  • EventLog Keeps a timestamped list of what happened during a take, such as notes and parameter changes, on the same clock as the recording.
  • pickMime() Picks the first MIME type from a list that the browser can record, using the browser's own check.
  • recommendedFps() The frame rate to record a frame size at: 60 up to about 2.6 megapixels, 30 above that.
  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • stopRecording() Stops the recording; the video is finished, downloaded, and handed to recordingSaved(info) and to the returned promise.

import { Recorder } from '@openav/record'; packages/record/recorder.js, line 32

resolveSize()#

function@openav/record
resolveSize(size)

Turns any way of naming a size, like 'vertical-4k', '1080x1920' or [w, h], into { w, h }, rounded up to even numbers.

It is what the Compositor uses for its size and setSize(). A size that cannot be read ('wat', a negative number, a missing side) throws a TypeError: record: unknown size "wat". Even sizes matter because H.264 and VP9 encode 4:2:0 colour.

Parameters

NameTypeDescription
sizeString | Number[] | ObjectA preset id from PRESETS, 'WxH' (also with × or *), [w, h] or { w, h }.

Returns

Object { w, h }, whole numbers, each rounded up to the next even number.

Examples

js
import { resolveSize } from '@openav/record';

console.log(resolveSize('vertical-4k'));      // { w: 2160, h: 3840 }
console.log(resolveSize('1081x1921'));        // { w: 1082, h: 1922 }
console.log(resolveSize([1000, 563]));        // { w: 1000, h: 564 }
try { resolveSize('wat'); } catch (e) { console.log(e.message); }   // record: unknown size "wat"
module Four ways of naming a size: a preset, text, a list, and a mistake that throws. Odd sides become even.

Related

  • PRESETS The output sizes the compositor knows by name, from vertical-1080p (1080 by 1920) to landscape-4k, each with its suggested layout and frame rate.
  • presetById() Finds one size preset by its id: presetById('vertical-4k').
  • Compositor Draws your work and the performer's camera into one canvas of an exact size, every frame: the canvas that a Recorder records.
  • layoutRects() Works out where the work and the camera go in an output frame, in whole pixels, for one of the four layouts.

import { resolveSize } from '@openav/record'; packages/record/layout.js, line 79

signalsOfMidi()#

function@openav/record
signalsOfMidi(ev, { families = 'both', device = null } = {})

Turns one MIDI event into the signal events a keyboard playing it would publish, so a file can drive a work like live playing.

It is what fromMidiFile() uses to build a take from a .mid file: each event of the file becomes the signals a keyboard would have published, so TakePlayer can play it into a show. Call it yourself when you have MIDI events from somewhere else and want them on a Signals bus.

Parameters

NameTypeDescription
evObjectA MIDI event { type, ch, note?, vel?, cc?, value? }. type is noteon, noteoff, cc, pitchbend (value 0..16383), chanat or polyat; velocities and values are 0..127.
families'both' | 'legacy' | 'channel'Which spellings to make: the classic names (midi/note/on, midi/cc/64, midi/bend), the per-channel ones (midi/ch/1/note/60, midi/ch/1/cc/64…), or both, as hardware does. optional default 'both'
deviceStringA device name to put in the classic note events. optional default null

Returns

Object[] A list of { name, value }. Note velocity becomes 0..1 (vel and velocity for the classic spelling), controllers 0..1, pitch bend -1..1. chanat and polyat exist only in the per-channel spelling.

Examples

js
import { signalsOfMidi } from '@openav/record';

const ev = { type: 'noteon', ch: 1, note: 60, vel: 100 };
console.log(signalsOfMidi(ev));
// [{ name: 'midi/note/on', value: { note: 60, vel: 0.787402, velocity: 100, ch: 1 } },
//  { name: 'midi/ch/1/note/60', value: 0.787402 }]
console.log(signalsOfMidi(ev, { families: 'channel' }));   // [{ name: 'midi/ch/1/note/60', value: 0.787402 }]
console.log(signalsOfMidi({ type: 'cc', ch: 1, cc: 64, value: 127 }, { families: 'legacy' }));
// [{ name: 'midi/cc/64', value: 1 }]
module One note-on becomes two signals (the two spellings), or one when you pick a family. The sustain pedal at full is a 1.

Related

  • fromMidiFile() Reads a Standard MIDI File into a take whose events are the signals a keyboard playing that file would publish.
  • midiEventsOf() Reads a take as plain MIDI events (note on and off, controllers, pitch bend), picking one naming family so nothing is counted twice.
  • TakePlayer Plays a take back into a bus at the right times, so the work, its sound and its chord detector cannot tell it from you playing.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).

import { signalsOfMidi } from '@openav/record'; packages/record/take.js, line 189

TAKE_VERSION#

constant@openav/record
TAKE_VERSION = 1

The version number of the take format, which is 1. Every take has v: 1.

A take is a plain object { v, name, createdAt, durationMs, events, meta }, see normalizeTake(). validateTake() complains when v is not this number, and normalizeTake() writes it.

Type
Number

1.

Examples

js
import { TAKE_VERSION, normalizeTake } from '@openav/record';

console.log(TAKE_VERSION);                       // 1
console.log(normalizeTake({ events: [] }).v);    // 1
module Logs the version, and that a take built from nothing gets it.

Related

  • normalizeTake() Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.
  • validateTake() Checks a take and lists everything wrong with it as readable sentences. An empty list means it is valid.

import { TAKE_VERSION } from '@openav/record'; packages/record/take.js, line 19

TakePlayer#

class@openav/record
new TakePlayer({ signals = null, take = null, loop = true, speed = 1, yieldToLive = false, resumeAfter = 0, isLive = defaultIsLive, chase = true, clock, maxStepMs = 250, onEvent = null, onState = null, now = defaultNow } = {})

Plays a take back into a bus at the right times, so the work, its sound and its chord detector cannot tell it from you playing.

Give it a bus and a take, call play(), and it publishes the take's events into the bus at their times: the same signals a keyboard would have, so a work's visuals, its sound (Sound) and its chord detector respond exactly as to your hands. That is what lets a work's preview play itself. Pass clock: 'manual' and call player.update(dt) from your loop for full control.

No stuck notes, ever: the player remembers every key it pressed, in the spelling it pressed it, and the sustain pedal, and lets go of them on pause, stop, seek and every loop wrap. With yieldToLive, playing a key yourself makes the take fall silent; with resumeAfter: 8 it comes back 8 seconds after you stop. Its state is 'playing', 'paused', 'yielded' or 'stopped', and position (milliseconds), loops, playing and yielded are plain properties.

Parameters

NameTypeDescription
signalsSignalsThe Signals bus to publish into. It also defines take/playing (0 or 1) and take/position (0..1). optional default null
takeObject | StringA take or its JSON. You can load() another one later. optional default null
loopBooleanStart again at the end. Off, it stops there. optional default true
speedNumberPlayback speed: 0.5 is half speed. optional default 1
yieldToLiveBooleanFall silent when someone plays live: a key-down that the player did not send (hardware, the on-screen piano, a phone) pauses the take and releases its notes. optional default false
resumeAfterNumberSeconds without live playing before a take that gave way resumes. 0 means it stays paused. optional default 0
isLiveFunctionisLive(name, value), what counts as live playing. The default is any key-down that did not come from the player; knobs do not count. optional default any key-down
chaseBooleanWhen you seek(), send again the last value of every controller before that point (the sustain pedal, knobs), so the take is in the state it would have been in. optional default true
clock'raf' | 'timer' | 'manual'Who drives the time: 'raf' (requestAnimationFrame, the default in a page), 'timer' (a 10 ms timer: tighter for sound and it keeps going, slower, in a background tab) or 'manual' (you call update(dt) or tick(); the default where there is no requestAnimationFrame). optional
maxStepMsNumberThe longest time one step may move forward. After a gap (a frozen tab) the take slows down instead of firing a burst of notes. optional default 250
onEventFunctiononEvent({ t, name, value }, player), for every event as it is played. optional default null
onStateFunctiononState(state, reason), when it changes: 'playing', 'paused', 'yielded' or 'stopped'. optional default null
nowFunctionThe clock, in milliseconds. optional default performance.now

Members

NameDescription
durationgetterGetter. The length of the loaded take, in milliseconds (0 without one).
progressgetterGetter. How far through the take, 0..1.
stategetterGetter. 'playing', 'yielded' (gave way to live playing), 'paused' (anywhere after the start, also at the end of a take that does not loop) or 'stopped' (at the start). The onState callback still says 'stopped' with the reason 'end' when a take that does not loop ends.
load(take)methodReplaces the take (stopping the current one and releasing its notes) and goes back to the start. null clears it. Returns the player.
play({ from } = {})methodPlays from where it is, from the start after the end, or from from milliseconds: play({ from }). Returns false when there is nothing to play (no take, or an empty one), otherwise true.
pause(reason)methodPauses where it is and releases the sounding notes. Returns the player.
stop()methodStops, rewinds to the start and releases the sounding notes. Returns the player.
seek(ms)methodJumps to ms milliseconds (held between 0 and the length), releasing the sounding notes, and with chase sends the controllers' values at that point. Returns the player.
update(dt)methodAdvances by dt seconds. For clock: 'manual': call it with the seconds since the last frame from your loop.
tick(nowMs = this.now())methodAdvances to the clock time nowMs (default now()): what the raf and timer clocks call. The first call only records the time.
dispose()methodStops, releases every note and stops listening to the bus.

Examples

js
import { TakePlayer } from '@openav/record';
import { Signals } from '@openav/core';

const take = { v: 1, name: 'riff', durationMs: 600, events: [
  { t: 100, name: 'midi/note/on', value: { note: 60, vel: 0.9, ch: 1 } },
  { t: 400, name: 'midi/note/off', value: { note: 60, ch: 1 } },
] };
const signals = new Signals();
const heard = [];
signals.on('midi/note/on', (v) => heard.push('on ' + v.note));
signals.on('midi/note/off', (v) => heard.push('off ' + v.note));

const player = new TakePlayer({ signals, take, loop: false, clock: 'manual' });
console.log(player.state, player.duration);                   // stopped 600
player.play();
for (let i = 0; i < 20; i++) player.update(0.05);              // one second, in 50 ms steps
console.log(heard.join(', '), player.state);                   // on 60, off 60 paused
module A two-event take is played by hand-cranked time into a bus: the listeners hear the note start and end, and because loop is off the player stays at the end of the take (state reads 'paused', position is 600).
js
import { TakePlayer } from '@openav/record';
import { Signals } from '@openav/core';

const take = { v: 1, name: 'loop', durationMs: 800, events: [
  { t: 0, name: 'midi/note/on', value: { note: 64, vel: 0.8, ch: 1 } },
  { t: 600, name: 'midi/note/off', value: { note: 64, ch: 1 } },
] };
const signals = new Signals();
const log = [];
signals.on('midi/note/on', (v) => log.push('note ' + v.note));
const player = new TakePlayer({ signals, take, yieldToLive: true, resumeAfter: 1, clock: 'manual',
  onState: (state, reason) => log.push(state + (reason ? ' (' + reason + ')' : '')) });
player.play();
player.update(0.1);
signals.pulse('midi/note/on', { note: 72, vel: 1, ch: 1 });   // someone plays live: the take gives way
log.push('live');
for (let i = 0; i < 25; i++) player.update(0.05);              // 1.25 s of silence: it comes back
console.log(log.join(' | '));
// playing | note 64 | note 72 | yielded (live) | live | playing (resume)
module A looping take gives way when a key is played live and returns one second after the playing stops. The log shows the order of events.

Related

  • TakeRecorder Records what you play: it listens to every midi/* signal on a bus from start() to stop() and gives you a take.
  • EventLog Keeps a timestamped list of what happened during a take, such as notes and parameter changes, on the same clock as the recording.
  • fromMidiFile() Reads a Standard MIDI File into a take whose events are the signals a keyboard playing that file would publish.
  • noteOffFor() Gives the events that release a sounding note, in the same spelling it was played in.
  • Sound Makes every midi/note/on and midi/note/off on a bus audible through a sound engine, and puts the engine's params on the bus as sound/*.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).

import { TakePlayer } from '@openav/record'; packages/record/take-player.js, line 105

TakeRecorder#

class@openav/record
new TakeRecorder({ signals, filter = /^midi\//, name = 'take', meta = {}, overdub = false, throttleMs = 0, now = defaultNow } = {})

Records what you play: it listens to every midi/* signal on a bus from start() to stop() and gives you a take.

A TakeRecorder is an EventLog that listens to midi/* on a bus. Press record: rec.start(). Play. Press stop: const take = rec.stop(). The take is plain data, so it can be saved as JSON, turned into a .mid (toMidiFile()) or played back (TakePlayer). It has every member of EventLog (mark(), between(), toJSON()…).

No stuck notes: a recorder stopped in the middle of a chord closes the keys that are still held (and a sustain pedal that is still down) in the take itself, at the stop time. The events are the signals as they were published, so a take from hardware holds both naming families (midi/note/on and midi/ch/1/note/60) and the device names, and playback publishes all of them. Pass recorder.t0 of a video Recorder to start() to line the take up with the video.

Parameters

NameTypeDescription
signalsSignalsThe Signals bus to listen to.
filterRegExp | FunctionWhich signals are the take: a pattern for the name, or filter(name, value, meta). optional default /^midi\//
nameStringThe name of the take. optional default 'take'
metaObjectFree fields copied into the take: { bpm, device, notes }. The device is found from the events if you leave it out. optional default {}
overdubBooleanAlso record what a TakePlayer is playing back right now. Off, a recorder ignores playback. optional default false
throttleMsNumberFor continuous signals (knobs, aftertouch): at most one event per name in this many milliseconds. optional default 0
nowFunctionThe clock, in milliseconds. optional default performance.now

Members

NameDescription
start(at = this.now())methodClears the take and starts listening. at is the clock time of t = 0 (default now), for example recorder.t0. Returns the recorder.
add(name, value = null, opts = {})methodAdds one event by hand, like EventLog.add() (see EventLog), and keeps track of which keys and pedals are held so stop() can release them.
stop(at = this.now())methodStops listening, releases the keys and pedal still down, and returns the take { v: 1, name, createdAt, durationMs, events, meta }.
toTake({ name = this.name, meta = this.meta } = {})methodThe take so far, as EventLog.toTake() does, with meta.device filled in from the events when it is missing.

Examples

js
import { TakeRecorder } from '@openav/record';
import { Signals } from '@openav/core';

const signals = new Signals();
let clock = 0;
const rec = new TakeRecorder({ signals, now: () => clock, meta: { bpm: 100 } });
rec.start();
clock = 100; signals.pulse('midi/note/on', { note: 60, vel: 0.9, ch: 1 });
clock = 400; signals.pulse('midi/note/off', { note: 60, ch: 1 });
clock = 450; signals.pulse('midi/note/on', { note: 62, vel: 0.5, ch: 1 });   // still held when we stop
clock = 600; signals.set('audio/rms', 0.2);                                  // not a midi/* signal
const take = rec.stop();
console.log(take.events.map((e) => e.t + ' ' + e.name).join(' | '));
// 100 midi/note/on | 400 midi/note/off | 450 midi/note/on | 600 midi/note/off
console.log(take.durationMs, take.meta);   // 600 { bpm: 100 }
module Records four events from a bus on a clock moved by hand. The last note was still held, so the take closes it at the stop time; the audio/rms signal is not part of the take.

Related

  • TakePlayer Plays a take back into a bus at the right times, so the work, its sound and its chord detector cannot tell it from you playing.
  • EventLog Keeps a timestamped list of what happened during a take, such as notes and parameter changes, on the same clock as the recording.
  • toMidiFile() Writes a take as a Standard MIDI File (type 0) that opens in any DAW.
  • trimSilence() Cuts the silence before the first key press of a take, so the first note comes at the moment you choose.
  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.

import { TakeRecorder } from '@openav/record'; packages/record/take-player.js, line 39

takeStats()#

function@openav/record
takeStats(take)

Counts what is in a take: notes, events, length, the channels used, and the lowest and highest note.

Good for showing what a take is before it is played or saved. Notes are counted through midiEventsOf(), which picks one naming family, so the count is the number of keys pressed.

Parameters

NameTypeDescription
takeObjectThe take.

Returns

Object { notes, events, durationMs, channels, low, high }. notes counts key-downs, once each even when the take holds two spellings of them. channels is a sorted list of MIDI channels. low and high are MIDI note numbers, or null for a take without notes.

Examples

js
import { takeStats } from '@openav/record';

const take = { v: 1, name: 'riff', durationMs: 1500, events: [
  { t: 500, name: 'midi/note/on', value: { note: 60, vel: 0.8, ch: 1 } },
  { t: 900, name: 'midi/note/off', value: { note: 60, ch: 1 } },
  { t: 1010, name: 'midi/note/on', value: { note: 64, vel: 0.5, ch: 1 } },
  { t: 1400, name: 'midi/note/off', value: { note: 64, ch: 1 } },
] };
console.log(takeStats(take));   // { notes: 2, events: 4, durationMs: 1500, channels: [1], low: 60, high: 64 }
module Two notes, four events, one channel, from C4 to E4.

Related

  • midiEventsOf() Reads a take as plain MIDI events (note on and off, controllers, pitch bend), picking one naming family so nothing is counted twice.
  • normalizeTake() Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.
  • noteOf() Tells whether a recorded event is a key going down or up, whatever its spelling, and which note on which channel.

import { takeStats } from '@openav/record'; packages/record/take.js, line 223

toMidiFile()#

function@openav/record
toMidiFile(take, { ppq = 960, bpm } = {})

Writes a take as a Standard MIDI File (type 0) that opens in any DAW.

One track holds note on and off with velocity, controllers (the sustain pedal is CC 64), pitch bend, and channel and poly aftertouch, each on its own channel; channel 10 stays the drums. The take's name becomes the track name, and a 4/4 time signature is written only so a DAW's grid looks sane. Times are kept to the millisecond whatever the tempo, so the file plays like the take. Only one naming family of the take is used, so nothing doubles (midiEventsOf()). To save it, wrap it in a Blob of type audio/midi.

Parameters

NameTypeDescription
takeObjectThe take.
ppqNumberTicks per quarter note. optional default 960
bpmNumberThe tempo written in the file. Left out, it is take.meta.bpm, and then 120. optional

Returns

Uint8Array The bytes of the .mid file.

Examples

js
import { toMidiFile, fromMidiFile } from '@openav/record';

const take = { v: 1, name: 'riff', durationMs: 1500, meta: { bpm: 120 }, events: [
  { t: 500, name: 'midi/note/on', value: { note: 60, vel: 0.8, ch: 1 } },
  { t: 900, name: 'midi/note/off', value: { note: 60, ch: 1 } },
] };
const bytes = toMidiFile(take);
console.log(bytes.constructor.name, String.fromCharCode(...bytes.slice(0, 4)));   // Uint8Array MThd
console.log(fromMidiFile(bytes).name);                                            // riff

const a = document.getElementById('app').appendChild(document.createElement('a'));
a.href = URL.createObjectURL(new Blob([bytes], { type: 'audio/midi' }));
a.download = 'riff.mid'; a.textContent = 'Download riff.mid';
module Writes a two-event take as a MIDI file, reads it back by its name, and offers it as a download link on the page.

Related

  • fromMidiFile() Reads a Standard MIDI File into a take whose events are the signals a keyboard playing that file would publish.
  • midiEventsOf() Reads a take as plain MIDI events (note on and off, controllers, pitch bend), picking one naming family so nothing is counted twice.
  • TakeRecorder Records what you play: it listens to every midi/* signal on a bus from start() to stop() and gives you a take.
  • quantizeTake() Snaps the key presses of a take to a rhythmic grid, so a loose performance lands on the beat.

import { toMidiFile } from '@openav/record'; packages/record/smf.js, line 46

trimSilence()#

function@openav/record
trimSilence(take, { keep = 0 } = {})

Cuts the silence before the first key press of a take, so the first note comes at the moment you choose.

Everything before the first key-down is cut away, but not lost: a controller value set during the silence, such as the sustain pedal or a knob, is kept and moved to the start (to t = 0 if it was before the cut), so the take still starts in the same state. If the take has no key-down at all, it is moved so its first event is at keep.

Parameters

NameTypeDescription
takeObjectThe take.
keepNumberMilliseconds of lead-in to keep before the first key-down. optional default 0

Returns

Object A new take. Everything moved earlier by the same amount, and durationMs is shorter by that amount. A take that starts at once comes back as it is.

Examples

js
import { trimSilence } from '@openav/record';

const take = { v: 1, name: 'riff', durationMs: 1500, events: [
  { t: 500, name: 'midi/note/on', value: { note: 60, vel: 0.8, ch: 1 } },
  { t: 900, name: 'midi/note/off', value: { note: 60, ch: 1 } },
] };
const t = trimSilence(take, { keep: 100 });
console.log(t.events.map((e) => e.t), t.durationMs);   // [100, 500] 1100
module A take that starts half a second late is moved earlier so its first note comes at 100 ms; its length shrinks by the 400 ms that were cut.

Related

  • quantizeTake() Snaps the key presses of a take to a rhythmic grid, so a loose performance lands on the beat.
  • normalizeTake() Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.
  • TakeRecorder Records what you play: it listens to every midi/* signal on a bus from start() to stop() and gives you a take.
  • takeStats() Counts what is in a take: notes, events, length, the channels used, and the lowest and highest note.

import { trimSilence } from '@openav/record'; packages/record/take.js, line 242

validateTake()#

function@openav/record
validateTake(take)

Checks a take and lists everything wrong with it as readable sentences. An empty list means it is valid.

It checks: v is TAKE_VERSION, name is a string, durationMs is a number from 0 up, events is an array, each event is an object with a signal name and a t of 0 or more, the events are in time order and none is after durationMs, and meta, if present, is an object. Use it on a take you read from a file; to repair one, use normalizeTake().

Parameters

NameTypeDescription
takeanyThe take to check.

Returns

String[] Sentences such as v must be 1 (got 2) or events[3] is out of order (t 100 < 200). [] when valid. It never throws.

Examples

js
import { validateTake } from '@openav/record';

const good = { v: 1, name: 'riff', durationMs: 1500, events: [{ t: 500, name: 'midi/note/on', value: { note: 60 } }] };
console.log(validateTake(good));                  // []
console.log(validateTake({ v: 2, events: [{ t: -1, name: '' }] }));
// ["v must be 1 (got 2)", "name must be a string", "durationMs must be a number ≥ 0",
//  "events[0].name must be a signal name", "events[0].t must be ms ≥ 0"]
console.log(validateTake(null));                  // ["not an object"]
module A valid take gives an empty list; a broken one lists every problem; something that is not even an object says so.

Related

  • normalizeTake() Turns anything take-shaped, or its JSON text, into a valid take: events sorted by time, bad events dropped, nothing negative.
  • TAKE_VERSION The version number of the take format, which is 1. Every take has v: 1.

import { validateTake } from '@openav/record'; packages/record/take.js, line 54

VIDEO_MIMES#

constant@openav/record
VIDEO_MIMES = array, 6 items

The video MIME types the recorder tries, best first: MP4 with H.264 and AAC, then WebM.

MP4 comes first because it opens everywhere a performance video goes next (an editor, Instagram, a phone's gallery) without a remux, and Chrome encodes H.264 in hardware. The first string asks for a profile (avc1.640033, High profile level 5.1, enough for 4K at 30 and 1080 by 1920 at 60) because Chrome says no to the generic avc1,mp4a but yes to the qualified one, so both are in the list for other browsers. Plain video/mp4 is for Safari. pickMime() walks this list.

Type
String[]

A frozen list of 6 strings: video/mp4;codecs=avc1.640033,mp4a.40.2, video/mp4;codecs=avc1,mp4a, video/webm;codecs=vp9,opus, video/webm;codecs=vp8,opus, video/webm, video/mp4.

Examples

js
import { VIDEO_MIMES, pickMime } from '@openav/record';

console.log(VIDEO_MIMES.length, VIDEO_MIMES[0]);   // 6 video/mp4;codecs=avc1.640033,mp4a.40.2
console.log(pickMime((m) => MediaRecorder.isTypeSupported(m)) || '(the browser chooses)');
module Logs the first candidate, and which one this browser would use.

Related

  • AUDIO_MIMES The audio MIME types the recorder tries for a sound-only file, best first: AAC in .m4a, then Opus.
  • pickMime() Picks the first MIME type from a list that the browser can record, using the browser's own check.
  • extFor() Gives the file extension for a recorded MIME type: video/mp4 is 'mp4', audio/mp4 is 'm4a', anything unknown is 'bin'.
  • Recorder Records a canvas, together with the sound you give it, into one video file with one MediaRecorder, so picture and sound stay in sync.

import { VIDEO_MIMES } from '@openav/record'; packages/record/mime.js, line 12

Loading options#

?oav_mic=click, window.OAV_P5 = { … }: how oav-p5 starts. 28

oav-p5

clock#

optionoav-p5
?oav_clock=follow | off

Whether an external MIDI clock (from a DAW, Strudel or a drum machine) moves drumPhase, drumBpm and the drum machine: follow or off.

With follow, an external MIDI clock (24 ticks a beat, plus start, stop and song position) is followed: drumPhase, beatPhase, drumBar, drumStep and drumBpm read it, and a drum machine that is up takes its tempo and its downbeat from it, starting and stopping with it. OAVp5.clock tells you what it hears. With nothing sending a clock, nothing changes.

off ignores every clock message: OAVp5.clock.info.state stays 'none'. Use it when something on the page or the machine sends a clock you do not want the sketch to follow.

Type
String
Values
follow | off
default
follow

Anything except off means follow.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { clock: … }; above the loader line wins, then ?oav_clock=… on the page address, then ?clock=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_clock=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { clock: 'off' };

function setup() {
  createCanvas(400, 300);
  OAVp5.input([0xFA]);                          // MIDI start
  setInterval(() => OAVp5.input([0xF8]), 20);   // clock ticks: 24 a beat, here 50 a second
}

function draw() {
  background(20);
  fill(255);
  text('clock option: ' + OAVp5.options.clock, 20, 30);
  text('external clock: ' + OAVp5.clock.info.state, 20, 60);   // none: the clock is ignored
}
The sketch sends itself a made-up MIDI clock, and with clock: 'off' it is ignored: the state stays "none". Delete the first line and it reads "running".

Related

  • OAVp5.clock What oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
  • beatPhase How far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • link Joins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.

p5/oav-p5.js, line 155

compact#

optionoav-p5
?oav_compact=on | off

Shows the 🎹 panel as a small box in the corner, a third of the window high, instead of a full-width bar: on or off.

Normally the panel opened by 🎹 is a bar across the whole window, up to half its height. With compact it is a box at most 560 pixels wide and a third of the window high, in the corner the 🎹 sits in, so the page beside it stays in view. It is the default on strudel.cc, where the code editor needs the room.

Type
String
Values
on | off
default
off (on in Strudel)

Only on turns it on. OAVp5.options.compact reads back as true or false.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { compact: … }; above the loader line wins, then ?oav_compact=… on the page address, then ?compact=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_compact=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { compact: 'on', panel: 'open' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('compact: ' + OAVp5.options.compact, 20, 30);   // true
}
The panel opens at once as a box in the bottom-right corner instead of a bar across the whole window.

Related

  • panel Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
  • strudel Whether oav-p5 treats the page as Strudel, the live-coding editor, and behaves politely beside it: auto, on or off.
  • corner Which side of the window the 🎹 button, its cards and the compact panel sit on: right or left.
  • dock Which edge of the window the 🎹 button and its cards sit on, bottom or top; set it and the panel starts on that edge too.

p5/oav-p5.js, line 152

corner#

optionoav-p5
?oav_corner=right | left

Which side of the window the 🎹 button, its cards and the compact panel sit on: right or left.

Moves the 🎹 to the left or right edge, with the cards that appear beside it. scope() draws on the opposite side, so the two never meet. On openprocessing.org the default is left, because OpenProcessing's own icon sits at the top right.

Type
String
Values
right | left
default
right (left on OpenProcessing)

Anything except left means right.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { corner: … }; above the loader line wins, then ?oav_corner=… on the page address, then ?corner=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_corner=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { corner: 'left' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('corner: ' + OAVp5.options.corner + ' (look for the 🎹)', 20, 30);
}
The 🎹 button sits in the bottom-left corner of the window instead of the bottom-right.

Related

  • dock Which edge of the window the 🎹 button and its cards sit on, bottom or top; set it and the panel starts on that edge too.
  • panel Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
  • compact Shows the 🎹 panel as a small box in the corner, a third of the window high, instead of a full-width bar: on or off.
  • scope() Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.

p5/oav-p5.js, line 141

debug#

optionoav-p5
?oav_debug=1 | 0

Prints [oav-p5 debug] lines in the console about clicks, the 🎹 button and loading, for finding out why the 🎹 does not respond.

Two seconds after the 🎹 is placed, a line says whether the 🎹 in the page is oav-p5's own copy and where oav-p5 loads the rest of OAV from. Every click then adds a line with the pointer position, what was clicked, and what is on top at that spot, which is how you find a page element sitting over the 🎹. It also says when the panel opens, when the page replaced the 🎹 and which name asked for the microphone.

Type
String
Values
1 | 0
default
'' (off)

Any value except an empty one and 0 turns it on.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { debug: … }; above the loader line wins, then ?oav_debug=… on the page address, then ?debug=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_debug=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { debug: 1 };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('open the console, then click the canvas', 20, 30);   // [oav-p5 debug] lines appear
}
Open the console: two seconds after load a debug line talks about the 🎹, and every click on the canvas adds one saying what was under the pointer.

Related

  • hints Whether oav-p5 shows its explaining cards beside the 🎹 (blocked microphone, no Web MIDI, an error in your callback): on or off.
  • panel Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
  • OAVp5.check() Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.

p5/oav-p5.js, line 158

dock#

optionoav-p5
?oav_dock=bottom | top

Which edge of the window the 🎹 button and its cards sit on, bottom or top; set it and the panel starts on that edge too.

Moves the 🎹, its cards and the panel to the top or bottom edge. On openprocessing.org the default is top, because OpenProcessing's console floats over the bottom of the sketch. The drum machine's card sits below a panel docked at the top.

The panel itself has two buttons for whoever is looking at it: Dock cycles bottom → top → floating (drag a floating panel by its bar), and Size cycles small → medium → large. This browser remembers the choice, and it wins over the option. When the sketch sets no dock, the panel starts at the bottom, also on openprocessing.org (from 2026-10-20: at the top it covered the sketch). The 🎹 moves out of the panel's way only when the two share an edge. Reloading the same page (Cmd+Enter on OpenProcessing) opens the panel again if it was open, and restarts drums started from its 🥁 on the same line and tempo.

Type
String
Values
bottom | top
default
bottom (top on OpenProcessing)

Anything except top means bottom.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { dock: … }; above the loader line wins, then ?oav_dock=… on the page address, then ?dock=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_dock=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { dock: 'top' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('dock: ' + OAVp5.options.dock + ' (look for the 🎹)', 20, 30);
}
The 🎹 button sits at the top-right of the window instead of the bottom-right.

Related

  • corner Which side of the window the 🎹 button, its cards and the compact panel sit on: right or left.
  • panel Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
  • compact Shows the 🎹 panel as a small box in the corner, a third of the window high, instead of a full-width bar: on or off.

p5/oav-p5.js, line 140

drumlatency#

optionoav-p5
?oav_drumlatency=auto | 0 to 500 (milliseconds)

How long it takes a drum hit to be heard after oav-p5 hands it to the audio clock, so pictures land with the sound: auto or milliseconds.

The drum machine hands each hit to the audio clock slightly ahead of time, and tells the sketch (drumHit(), kickDetected(), env('kick')) when you will hear it, which is the scheduled moment plus the output latency. auto uses the latency the browser reports. If the browser's figure is smaller than what your output really takes, the picture runs ahead of the sound, and giving the number yourself fixes that. The project's README names a Bluetooth speaker as the case and suggests trying 200.

Type
String | Number
Values
auto | 0 to 500 (milliseconds)
default
auto

A number is held between 0 and 500. Anything that is not a number means auto. OAVp5.options.drumlatency reads back in seconds (0.2 for 200).

Notes

  • Only the default drum machine (drums=lookahead) uses it: the frame-driven one has nothing scheduled ahead.
  • Set it before oav-p5 loads. window.OAV_P5 = { drumlatency: … }; above the loader line wins, then ?oav_drumlatency=… on the page address, then ?drumlatency=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_drumlatency=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { drumlatency: 200 };

function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, [~ sd]*2, hh*8', 108);
}

function draw() {
  background(20);
  fill(255);
  text('drumlatency: ' + OAVp5.options.drumlatency + ' s', 20, 30);
  circle(width / 2, height / 2, 50 + env('kick', 0.01, 0.3) * 200);   // told 0.2 s after the hit is scheduled
}
The circle swells on each kick, told 200 milliseconds after the hit was handed to the audio clock, to match a speaker that is that late. Click the canvas once if you hear nothing.

To be confirmed

No Bluetooth speaker was tried: the suggested 200 comes from the project's README, not from a measurement made for this page.

Related

  • drums How the drum machine is clocked: lookahead (hits scheduled ahead on the audio clock) or frame (moved by animation frames, 0.5's way).
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • drumHit() An event you define: it is called for every hit of the drum machine, with the drum that sounded and how hard.
  • env() A 0..1 pulse that jumps up after each event (a kick, a beat, a key, a press) and then fades out: TouchDesigner's Trigger CHOP.

p5/oav-p5.js, line 135

drums#

optionoav-p5
?oav_drums=lookahead | frame

How the drum machine is clocked: lookahead (hits scheduled ahead on the audio clock) or frame (moved by animation frames, 0.5's way).

With lookahead, a heartbeat in a Web Worker hands the hits to the audio clock a little ahead of time, so the beat stays steady when frames stutter or the tab is busy, and the sketch is told when each hit is heard. With frame, the machine is moved by the animation frames instead, as it was in oav-p5 0.5. It is the safety net: where the lookahead machine misbehaves, frame brings the old behaviour back, and the timing is then only as steady as the frame rate.

Type
String
Values
lookahead | frame
default
lookahead

Anything except frame means lookahead.

Notes

  • OAVp5.drums.clock and the 🩺 check-up say which clock is running.
  • Set it before oav-p5 loads. window.OAV_P5 = { drums: … }; above the loader line wins, then ?oav_drums=… on the page address, then ?drums=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_drums=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { drums: 'frame' };

function setup() {
  createCanvas(400, 300);
  startDrums('bd*4, [~ sd]*2, hh*8', 108);
}

function draw() {
  background(20);
  fill(255);
  text('drums: ' + OAVp5.options.drums + ' (clock: ' + OAVp5.drums.clock.clock + ')', 20, 30);
  circle(width / 2, height / 2, 50 + env('kick', 0.01, 0.3) * 200);
}
The drum machine is moved by the animation frames, and the text says so. Click the canvas once if you hear nothing.

Related

  • drumlatency How long it takes a drum hit to be heard after oav-p5 hands it to the audio clock, so pictures land with the sound: auto or milliseconds.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • OAVp5.check() Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.

p5/oav-p5.js, line 134

hardware#

optionoav-p5
?oav_hardware=auto | click | off

When oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.

With auto, oav-p5 asks for Web MIDI as soon as it loads, so a plugged-in controller plays the sketch without anyone pressing anything. The browser remembers your answer per site. With click, it asks only when the 🎹 panel is first opened. With off, it never asks by itself, and the on-screen controller and a phone still play the sketch.

OAVp5.connectHardware() asks whatever the option says, and OAVp5.hardware tells you the state: 'off' until it has been asked.

Type
String
Values
auto | click | off
default
auto (click in Strudel)

Only auto and click ask. Any other value acts like off.

Notes

  • On strudel.cc (or a page where Strudel's own functions are present) the default is click, because Strudel's midin() asks for MIDI itself. Setting hardware yourself wins.
  • Safari and iPad have no Web MIDI at all: when oav-p5 asks (or you call OAVp5.connectHardware()), the state there becomes 'unavailable'.
  • Set it before oav-p5 loads. window.OAV_P5 = { hardware: … }; above the loader line wins, then ?oav_hardware=… on the page address, then ?hardware=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_hardware=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { hardware: 'click' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('hardware option: ' + OAVp5.options.hardware, 20, 30);
  text('Web MIDI state: ' + OAVp5.hardware, 20, 60);   // stays "off" until you open the 🎹 panel
}
The page does not ask for Web MIDI when it loads, and the state stays "off". Press 🎹: that opens the panel and asks. Delete the first line and it asks at once.

Related

  • OAVp5.hardware Where Web MIDI stands: 'off', 'asking', 'connected', 'waiting', 'denied' or 'unavailable'.
  • OAVp5.connectHardware() Asks the browser for Web MIDI now, and returns a promise that says whether a real controller can be heard.
  • OAVp5.devices The names of the MIDI controllers plugged in, as the browser reports them: an array of text, empty when there is none.
  • panel Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
  • strudel Whether oav-p5 treats the page as Strudel, the live-coding editor, and behaves politely beside it: auto, on or off.

p5/oav-p5.js, line 126

hints#

optionoav-p5
?oav_hints=on | off

Whether oav-p5 shows its explaining cards beside the 🎹 (blocked microphone, no Web MIDI, an error in your callback): on or off.

When something goes wrong, a small card beside the 🎹 says what happened, why, and what to press, in the browser's language, once per page. hints: 'off' takes the cards off the page: the console still says the same thing and OAVp5.hints still lists what was said. Use it for a finished piece that goes on a wall or into a recording.

It also hides the corner oscilloscope of scope(), which is part of the same on-page help. scope=off hides only that.

Type
String
Values
on | off
default
on

Only off turns the cards off. Any other value, including the text false, leaves them on.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { hints: … }; above the loader line wins, then ?oav_hints=… on the page address, then ?hints=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_hints=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { hints: 'off' };

function setup() {
  createCanvas(400, 300);
  lag(1);   // an unnamed lag() outside draw(): oav-p5 has something to say about that
}

function draw() {
  background(20);
  fill(255);
  text('cards said: ' + OAVp5.hints.length + ' (none on the page)', 20, 30);
  text(OAVp5.hints.map((h) => h.what).join('\n'), 20, 60);
}
oav-p5 notices the lag() in setup() and would normally show a card. With hints off nothing appears on the page, but the console and the list in OAVp5.hints still have it.
jsOpenProcessing ↗
window.OAV_P5 = { hints: false };   // not 'off': the value is turned into the text "false"

function setup() {
  createCanvas(400, 300);
  lag(1);
}

function draw() {
  background(20);
  fill(255);
  text('the card is still beside the 🎹', 20, 30);
}
The gotcha: false is not 'off', so the card still appears.

Related

  • OAVp5.hints The list of cards oav-p5 has shown or would have shown so far (a blocked microphone, no Web MIDI, an error in your callback), each with what, why and what to press.
  • scope Whether scope() draws its oscilloscope in the corner of the page: on or off. hints=off turns it off too.
  • debug Prints [oav-p5 debug] lines in the console about clicks, the 🎹 button and loading, for finding out why the 🎹 does not respond.
  • OAVp5.check() Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.

p5/oav-p5.js, line 1076

optionoav-p5
?oav_link=a room name

Joins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.

Every oav-p5 page with the same room (another tab, another computer) gets what this page's programs play into the virtual MIDI output (notes, controllers and the clock), over the phone relay, and plays it at the moment it sounds here. On the other side, OAVp5.last.source is 'link'. Nothing it receives is sent on, so two pages cannot loop. The room is also the phones' room, so one QR code plays every linked page.

OAVp5.link.info tells you the room, whether it is connected and how many other pages were heard. The 🎹 grows a 🔗: green when another page is there, yellow while connecting or alone, red when the relay is gone. linkdelay adds a buffer.

Type
String
Values
a room name
default
'' (no room)

Only letters, digits, _ and - are kept, and only the first 32 characters.

Before it has a value

A network connection to the relay (relay, wss://remote.cheyuwu.com/relay unless you set another). Without it the 🔗 is red and nothing is shared.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { link: … }; above the loader line wins, then ?oav_link=… on the page address, then ?link=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_link=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

js
window.OAV_P5 = { link: 'my-room' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  const info = OAVp5.link.info;
  text('room: ' + OAVp5.options.link, 20, 30);
  text('state: ' + (info ? info.state + ', ' + info.peers + ' other page(s)' : 'not linked'), 20, 60);
}
static Open the same address in two tabs: each shows the room and, once connected, one other page. Anything one tab's program plays into OAV Virtual MIDI is heard by the other.

To be confirmed

The headless check cannot reach a relay, so this example is not run. Behaviour between two pages was read from the code and the project's README, not seen here.

Related

  • OAVp5.link What the link to other oav-p5 pages in the same room is doing: only OAVp5.link.info, and null unless a room is set.
  • OAVp5.link.info The state of the link: its room, state (connecting, open, closed), how many other pages are there, and the trip time. null when no room is set.
  • linkdelay Extra milliseconds of buffer on the notes a linked page receives, to smooth out an uneven network: 0 to 2000.
  • relay The address of the relay that phones and link rooms talk through: a WebSocket URL.
  • room The name of the room phones join to play this page: by default one made up per browser.

p5/oav-p5.js, line 156

linkdelay#

optionoav-p5
?oav_linkdelay=0 to 2000 (milliseconds)

Extra milliseconds of buffer on the notes a linked page receives, to smooth out an uneven network: 0 to 2000.

Only matters together with link: without a room nothing is received. Each note arriving from another page is played at the moment it sounded there, which already allows for the trip. linkdelay adds this much on top, so a note that took longer than usual is still on time. The price is that everything from the other page arrives that much later.

Type
Number
Values
0 to 2000 (milliseconds)
default
0

Held between 0 and 2000. Anything that is not a number is 0.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { linkdelay: … }; above the loader line wins, then ?oav_linkdelay=… on the page address, then ?linkdelay=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_linkdelay=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { linkdelay: 80 };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('linkdelay: ' + OAVp5.options.linkdelay + ' ms', 20, 30);
  text('link: ' + (OAVp5.link.info ? 'room ' + OAVp5.link.info.room : 'no room, so nothing to delay'), 20, 60);   // add link: 'my-room' to use it
}
Without a link room the number is only stored: the sketch says there is nothing to delay. Add link: 'my-room' to the options and it applies to every note that arrives.

Related

  • link Joins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
  • OAVp5.link.info The state of the link: its room, state (connecting, open, closed), how many other pages are there, and the trip time. null when no room is set.
  • relay The address of the relay that phones and link rooms talk through: a WebSocket URL.

p5/oav-p5.js, line 157

mic#

optionoav-p5
?oav_mic=auto | click | off

Who may turn the microphone on: auto (the first read of micLevel and friends), click (only 🎤 or startMic()) or off (nobody).

With auto, the first time a sketch reads micLevel, bass, mid, treble or beat, oav-p5 asks the browser for the microphone. With click, reading them does nothing: the microphone starts only when someone presses 🎤 in the 🎹 panel or the sketch calls startMic(), so the permission prompt arrives when you choose. With off, nothing opens the microphone, not even startMic(), which resolves false.

off is a promise to the page: a gallery piece that must never listen can say so. click is for pieces where a permission prompt out of nowhere would be rude, or where the sound only matters after an intro. The values of the microphone stay 0 while it is off.

Type
String
Values
auto | click | off
default
auto

Anything else means auto.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { mic: … }; above the loader line wins, then ?oav_mic=… on the page address, then ?mic=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_mic=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { mic: 'click' };

function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  startMic();   // now the browser asks
}

function draw() {
  background(20);
  circle(width / 2, height / 2, 50 + micLevel * 300);   // reading micLevel no longer asks
  fill(255);
  text('microphone: ' + OAVp5.mic.state + ' (click the canvas to start it)', 20, 30);
}
Reading micLevel does not ask for the microphone: the state stays "off" and the circle stays small. Click the canvas and the browser asks. Asks for the microphone
jsOpenProcessing ↗
window.OAV_P5 = { mic: 'off' };

let answer = 'click the canvas to try';

function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  startMic().then((ok) => { answer = 'startMic() resolved ' + ok; });
}

function draw() {
  background(20);
  fill(255);
  text('microphone: ' + OAVp5.mic.state, 20, 30);
  text(answer, 20, 60);   // startMic() resolved false
}
Nothing can open the microphone: the state stays "off" and startMic() resolves false.

To be confirmed

Not tried on openprocessing.org itself: that ?sketch_oav_mic=click on the page address reaches the sketch.

Related

  • micLevel How loud the microphone hears things, from 0 (a quiet room) to 1 (shouting).
  • startMic() Turns the microphone on and returns a promise that says whether it worked.
  • stopMic() Turns the microphone off and sets every microphone value back to 0.
  • OAVp5.mic.state Where the microphone is: 'off', 'asking', 'on', 'denied' or 'unavailable'.
  • micIsOn Whether the microphone is running right now.

p5/oav-p5.js, line 128

opc#

optionoav-p5
?oav_opc=on | off

Whether oav-p5 wires the controller to the variables of OP Configurator (OpenProcessing's sliders and buttons): on or off.

On OpenProcessing, OPC.slider('radius', 10) declares a variable with a slider. With opc on, oav-p5 then wires the controller to the OPC variables in order: knob 1 to the first slider, knob 2 to the second, the faders after the knobs, pad 1 to the first toggle or button. OPC's own panel moves with the knob. OAVp5.wiring() lists who drives what.

off leaves OPC's variables to their own sliders. A control the sketch reads itself (knob1 in draw()) is the sketch's anyway: the slider at that place stops following it.

Type
String
Values
on | off
default
on

Only off turns it off.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { opc: … }; above the loader line wins, then ?oav_opc=… on the page address, then ?opc=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_opc=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { opc: 'off' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('opc option: ' + OAVp5.options.opc, 20, 30);
  text('controls wired to OPC variables: ' + OAVp5.wiring().length, 20, 60);   // 0
}
Outside OpenProcessing there is no OPC, so the count is 0 either way. On OpenProcessing with sliders declared, it is 0 with opc: 'off' and the number of sliders and buttons without it.

To be confirmed

Not tried with OpenProcessing's real OPC library. The wiring was checked against a stand-in object with the same options shape, and the example here runs where no OPC exists, so it only shows the option arriving.

Related

  • OAVp5.wiring() Lists which controller knob, fader or pad drives which OpenProcessing OPC variable right now: [{ control: 'knob1', name: 'radius' }, …].
  • knob1 The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
  • knob() Returns the position of knob number n, from 0 to 1: the same as knob1 to knob8, for any n.
  • pad1 How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.

p5/oav-p5.js, line 127

panel#

optionoav-p5
?oav_panel=closed | open | off

Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.

closed shows just the small 🎹 button, and pressing it opens the panel: the on-screen controller, the buttons for 🎤, 🔊, 🥁, ⏺, 📷, 📱 and 🩺. open opens the panel as soon as the page loads, for a workshop where everyone needs the on-screen controller at once. off removes the 🎹 button altogether, for a piece in an exhibition: there is no button and OAVp5.panel.open() does nothing. The cards that explain a problem still appear in the corner, unless hints is off too.

Type
String
Values
closed | open | off
default
closed

Any value except open and off means closed.

Notes

  • With off everything the panel does is still there as code: startMic(), startSound(), startRecording() and the options keep working, and the console still says what goes wrong.
  • Set it before oav-p5 loads. window.OAV_P5 = { panel: … }; above the loader line wins, then ?oav_panel=… on the page address, then ?panel=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_panel=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { panel: 'open' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('panel: ' + OAVp5.options.panel, 20, 30);
}
The 🎹 panel is open as soon as the page loads: the on-screen controller and its buttons.
jsOpenProcessing ↗
window.OAV_P5 = { panel: 'off' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('panel: ' + OAVp5.options.panel + ' (no 🎹 on this page)', 20, 30);
}
There is no 🎹 button at all. The sketch still hears a real controller and everything else.

Related

  • OAVp5.panel The 🎹 panel as an object: open(), close() and toggle() show and hide it from code.
  • OAVp5.panel.open() Opens the 🎹 panel (open()), closes it (close()) or flips it (toggle()).
  • compact Shows the 🎹 panel as a small box in the corner, a third of the window high, instead of a full-width bar: on or off.
  • dock Which edge of the window the 🎹 button and its cards sit on, bottom or top; set it and the panel starts on that edge too.
  • corner Which side of the window the 🎹 button, its cards and the compact panel sit on: right or left.

p5/oav-p5.js, line 126

profile#

optionoav-p5
?oav_profile=a profile id

Which controller the on-screen 🎹 shows, by profile id such as akai-lpd8; by default it follows the one that is plugged in.

The on-screen controller of the 🎹 panel draws this device, and knob1, fader1 and pad1 mean the first knob, fader and pad printed on it. A sketch written with the LPD8 in mind, for instance, can ask for it with profile: 'akai-lpd8' so that whoever has no LPD8 still gets one on screen. The picker inside the panel changes it by hand.

If you do not set it, the panel starts as the MiniLab 3 and then follows a real controller as soon as one that OAV has a profile for is plugged in. If you do set it, that choice stays. An id that does not exist is reported in the console with the list of known ids, and the panel shows no controller.

Type
String
Values
a profile id
default
arturia-minilab3

Any id that @openav/midi has a profile for: arturia-minilab3, akai-lpd8, korg-nanokontrol2, generic-keys49 and the rest. When you set nothing, the panel follows the controller that is plugged in.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { profile: … }; above the loader line wins, then ?oav_profile=… on the page address, then ?profile=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_profile=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { profile: 'akai-lpd8', panel: 'open' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('profile: ' + OAVp5.options.profile, 20, 30);
  circle(width / 2, height / 2, 40 + knob1 * 240);   // knob 1 of the LPD8
}
The panel opens with an Akai LPD8 on screen: eight pads and eight knobs. Turn its first knob and the circle grows.

Related

  • knob1 The position of a controller knob, from 0 to 1: knob1 is the first knob, knob2 the second, up to knob8.
  • fader1 The position of a controller fader (slider), from 0 to 1: fader1 is the first fader, up to fader8.
  • pad1 How hard a drum pad is pressed, from 0 to 1: pad1 is the first pad, up to pad16; 0 when it is let go.
  • panel Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
  • OAVp5.input() Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.

p5/oav-p5.js, line 126

recfps#

optionoav-p5
?oav_recfps=1 to 60

The frame rate of recorded video, 1 to 60 frames a second.

How many frames a second the recorder takes from the canvas, for startRecording() and the ⏺ button. 30 suits most places Instagram and YouTube accept. 60 is smoother for fast motion and makes a bigger file. One take can ask for its own rate: startRecording({ fps: 60 }).

Type
Number
Values
1 to 60
default
30

Rounded to a whole number and held between 1 and 60. Anything that is not a number means 30.

Notes

  • It is read back as OAVp5.options.recfps and as OAVp5.record.fps.
  • Set it before oav-p5 loads. window.OAV_P5 = { recfps: … }; above the loader line wins, then ?oav_recfps=… on the page address, then ?recfps=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_recfps=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { recfps: 60 };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('recording frame rate: ' + OAVp5.record.fps, 20, 30);
}
Takes recorded from this page are 60 frames a second instead of 30.

Related

  • reclimit The longest one recording may last, in seconds: a take stops by itself there.
  • record Whether recording is available and whether it includes the microphone: on, nomic or off.
  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • OAVp5.record.fps The frame rate (fps) and the longest take in seconds (limit) that the loading options recfps and reclimit set.

p5/oav-p5.js, line 132

reclimit#

optionoav-p5
?oav_reclimit=1 or more (seconds)

The longest one recording may last, in seconds: a take stops by itself there.

A long take fills the tab's memory, so every recording stops by itself at this length, saves what it has and shows a card saying so. Raise it for a long piece, record in parts, or give one take its own length: startRecording(10) stops after ten seconds, and a number above the limit is cut down to the limit.

Type
Number
Values
1 or more (seconds)
default
300

At least 1. Anything that is not a number means 300 (five minutes).

Notes

  • It is read back as OAVp5.options.reclimit and as OAVp5.record.limit.
  • Set it before oav-p5 loads. window.OAV_P5 = { reclimit: … }; above the loader line wins, then ?oav_reclimit=… on the page address, then ?reclimit=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_reclimit=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { reclimit: 60 };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('longest take: ' + OAVp5.record.limit + ' seconds', 20, 30);
}
A recording from this page stops by itself after a minute.

Related

  • recfps The frame rate of recorded video, 1 to 60 frames a second.
  • record Whether recording is available and whether it includes the microphone: on, nomic or off.
  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • OAVp5.record.fps The frame rate (fps) and the longest take in seconds (limit) that the loading options recfps and reclimit set.

p5/oav-p5.js, line 133

record#

optionoav-p5
?oav_record=on | nomic | off

Whether recording is available and whether it includes the microphone: on, nomic or off.

With on, ⏺ and 📷 are in the 🎹 panel and startRecording() records the canvas together with the sound the sketch makes, the microphone included when it is on. nomic keeps the microphone out of the recording. With it in and no headphones, the speakers are recorded a second time through the microphone, so leave it out when you play sound out loud. off takes ⏺ and 📷 away and makes startRecording() do nothing.

off is a promise to the page, like mic=off: a piece that must not be captured can say so. One take can still choose its own: startRecording({ mic: false }).

Type
String
Values
on | nomic | off
default
on

Anything else means on.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { record: … }; above the loader line wins, then ?oav_record=… on the page address, then ?record=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_record=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { record: 'nomic' };

function setup() {
  createCanvas(400, 300);
}

function keyPressed() {
  if (key === 'r') startRecording(3);   // three seconds, without the microphone
}

function draw() {
  background(20 + 20 * sin(frameCount * 0.05));
  fill(255);
  text('press r to record 3 seconds (' + (recordingIsOn ? 'recording' : 'idle') + ')', 20, 30);
}
Press r: three seconds of the canvas are recorded and downloaded, with the microphone left out even if it is on.

Related

  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • recfps The frame rate of recorded video, 1 to 60 frames a second.
  • reclimit The longest one recording may last, in seconds: a take stops by itself there.
  • title A name that goes into the file name of recordings and images, as in oav-my-piece-20261015-183012.mp4.
  • recordingIsOn Whether a recording is running right now.

p5/oav-p5.js, line 131

relay#

optionoav-p5
?oav_relay=a ws:// or wss:// address

The address of the relay that phones and link rooms talk through: a WebSocket URL.

The 📱 button shows a QR code that sends a phone to the remote page, and the phone's MIDI tab plays your sketch through this relay. link rooms use it too. The default is the project's public relay. To keep your traffic on your own machine, run the relay from a checkout of the repository with node packages/relay/server.js and point relay at ws://localhost:7458/relay.

Type
String (URL)
Values
a ws:// or wss:// address
default
wss://remote.cheyuwu.com/relay

Used as given.

Notes

  • Nothing connects to it at load. It is used when you open the 📱 card (OAVp5.phone()) or set link.
  • Set it before oav-p5 loads. window.OAV_P5 = { relay: … }; above the loader line wins, then ?oav_relay=… on the page address, then ?relay=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_relay=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { relay: 'ws://localhost:7458/relay' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('relay: ' + OAVp5.options.relay, 20, 30);   // used when you open 📱 or set link
}
The sketch only prints the address in use: a phone or a linked page would now connect to a relay on your own computer.

Related

  • room The name of the room phones join to play this page: by default one made up per browser.
  • remote The address of the remote page that the 📱 QR code opens on a phone.
  • link Joins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
  • OAVp5.phone() Shows the 📱 card: a QR code that sends a phone to the remote page, whose MIDI tab then plays this sketch. Returns the room name.

p5/oav-p5.js, line 127

remote#

optionoav-p5
?oav_remote=an https:// address

The address of the remote page that the 📱 QR code opens on a phone.

The phone's page is where the on-screen controller on the phone lives. The QR code and the link under it are this address plus the relay, the room and the MIDI tab. Change it when you host the remote page yourself, for example on a local network where the phone cannot reach the public site.

Type
String (URL)
Values
an https:// address
default
https://openaudiovisual.com/packages/remote/

Used as given, with ?relay=…&room=…&tab=midi added.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { remote: … }; above the loader line wins, then ?oav_remote=… on the page address, then ?remote=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_remote=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { remote: 'http://192.168.0.10:8080/packages/remote/' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('remote page: ' + OAVp5.options.remote, 20, 30);   // the 📱 QR code points here
}
The sketch prints the remote page address in use. Press 🎹, then 📱: the QR code and the link under it point at that address.

Related

  • relay The address of the relay that phones and link rooms talk through: a WebSocket URL.
  • room The name of the room phones join to play this page: by default one made up per browser.
  • OAVp5.phone() Shows the 📱 card: a QR code that sends a phone to the remote page, whose MIDI tab then plays this sketch. Returns the room name.

p5/oav-p5.js, line 129

room#

optionoav-p5
?oav_room=a room name

The name of the room phones join to play this page: by default one made up per browser.

A phone and a page meet in a room on the relay. Left empty, oav-p5 picks a name like p5-k3x9a the first time and keeps it in this browser (localStorage), so the same QR code keeps working on reload. If link is set, its room is used. Set room yourself when you want a name people can type or remember, such as one per class.

A phone needs only the room name to join, so pick a name people cannot guess if that matters. OAVp5.phone() returns the room in use.

Type
String
Values
a room name
default
'' (random per browser)

Left empty, a name like p5-k3x9a is made up and kept in this browser. Only letters, digits, _ and - are kept, and only the first 32 characters. If nothing is left, the room is default.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { room: … }; above the loader line wins, then ?oav_room=… on the page address, then ?room=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_room=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { room: 'class-2026' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('room: ' + OAVp5.options.room, 20, 30);   // the phones' QR code uses this room
}
The sketch prints the room name. The 📱 card now shows class-2026 instead of a random name.

Related

  • relay The address of the relay that phones and link rooms talk through: a WebSocket URL.
  • link Joins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
  • remote The address of the remote page that the 📱 QR code opens on a phone.
  • OAVp5.phone() Shows the 📱 card: a QR code that sends a phone to the remote page, whose MIDI tab then plays this sketch. Returns the room name.

p5/oav-p5.js, line 127

scope#

optionoav-p5
?oav_scope=on | off

Whether scope() draws its oscilloscope in the corner of the page: on or off. hints=off turns it off too.

With off, scope() draws nothing in the corner and only returns its value, so a sketch can leave its scope() calls in and hide the plot for a show. drawTrail(), which draws on the canvas, is not affected.

Type
String
Values
on | off
default
on

Only off turns it off.

Notes

  • If the plot does not show up, check hints too: hints=off hides it as well.
  • Set it before oav-p5 loads. window.OAV_P5 = { scope: … }; above the loader line wins, then ?oav_scope=… on the page address, then ?scope=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_scope=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { scope: 'off' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  const wave = 0.5 + 0.5 * sin(frameCount * 0.05);
  circle(width / 2, height / 2, 40 + scope(wave, 'wave') * 200);   // returns wave, draws nothing
}
The circle still breathes, but there is no plot in the corner of the page. Delete the first line and it comes back.

Related

  • scope() Shows value in a small oscilloscope in the corner of the page and gives it back, so you can wrap any expression to watch it.
  • drawTrail() Draws a line through values on the canvas, so it is in the work and in a recording: the oldest at x, lo at the bottom of the box, hi at the top.
  • hints Whether oav-p5 shows its explaining cards beside the 🎹 (blocked microphone, no Web MIDI, an error in your callback): on or off.

p5/oav-p5.js, line 3069

shim#

optionoav-p5
?oav_shim=on | off

Whether oav-p5 replaces navigator.requestMIDIAccess so sketches with their own Web MIDI code also hear the 🎹 and phones: on or off.

With on, a sketch that calls navigator.requestMIDIAccess() itself (WebMidi.js does) gets the real ports and one more input called "OAV Virtual MIDI", which carries whatever the on-screen controller and the phone play. It also works in Safari and on iPad, where there is no Web MIDI: the sketch gets the virtual port alone, so a WebMidi.js sketch still runs there. Programs can play into its output as well (Strudel's .midi('OAV Virtual MIDI')).

off leaves the browser's own requestMIDIAccess alone. Use it when a page already has its own MIDI handling that must not see an extra port.

Type
String
Values
on | off
default
on

Only off turns it off.

Notes

  • Order matters: oav-p5 has to load before the code that asks for MIDI, which is why the loader line goes first. A page that opened MIDI earlier keeps the browser's own answer, and 🩺 says so.
  • Set it before oav-p5 loads. window.OAV_P5 = { shim: … }; above the loader line wins, then ?oav_shim=… on the page address, then ?shim=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_shim=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { shim: 'off' };

let names = 'asking…';

function setup() {
  createCanvas(400, 300);
  navigator.requestMIDIAccess().then(
    (access) => { names = [...access.inputs.values()].map((i) => i.name).join(', ') || 'no inputs'; },
    (error) => { names = 'refused: ' + error.name; }
  );
}

function draw() {
  background(20);
  fill(255);
  text('shim: ' + OAVp5.options.shim, 20, 30);
  text('MIDI inputs: ' + names, 20, 60);
}
With the shim off, the sketch gets only the browser's own answer (the real inputs, or a refusal). Delete the first line and "OAV Virtual MIDI" is in the list. The browser may ask you for MIDI permission.

Related

  • OAVp5.input() Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
  • hardware When oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.
  • midiMessage() You define this function; oav-p5 calls it for every MIDI message it receives, with the raw bytes and where they came from.
  • OAVp5.last The latest note, control change or pitch bend that arrived: its type, number, value, channel, where it came from and which control it is. null before any.

p5/oav-p5.js, line 127

sound#

optionoav-p5
?oav_sound=click | off | an instrument id

Whether notes are heard: click (only after startSound() or 🔊), off (never), or an instrument id that starts at the first click.

With click, nothing makes a sound until the sketch calls startSound() or someone presses 🔊 in the 🎹 panel. With an instrument id, every key, pad, the on-screen controller and a phone are heard from the first click on the page, with no code at all, so any MIDI sketch becomes playable by adding the option to the page address: ?oav_sound=piano. off is a promise to the page: nothing sounds, startSound() resolves false and the 🔊 button is hidden.

Browsers let a page play sound only after a click, and a MIDI key is not a click, so the first sound waits for one: a card says "sound is ready but silent" until then.

Type
String
Values
click | off | an instrument id
default
click

An instrument id such as piano, organ, strings, harp or music-box. on and auto start the default instrument (the piano). An id that does not exist plays the default synth (synth/pad) and the console lists the ids that exist.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { sound: … }; above the loader line wins, then ?oav_sound=… on the page address, then ?sound=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_sound=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { sound: 'organ' };

function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  OAVp5.play(60 + floor(random(12)), 0.8);   // a key, as a controller would play it
}

function draw() {
  background(20);
  fill(255);
  text('sound: ' + OAVp5.sound.state + ' (' + OAVp5.sound.instrument + ')', 20, 30);
  text('click the canvas to play a note', 20, 60);
}
No startSound() call: the organ is heard from your first click, which also plays a note. The state goes from "waiting" to "on".
jsOpenProcessing ↗
window.OAV_P5 = { sound: 'off' };

let answer = 'click the canvas to try';

function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  startSound().then((ok) => { answer = 'startSound() resolved ' + ok; });
}

function draw() {
  background(20);
  fill(255);
  text('sound: ' + OAVp5.sound.state, 20, 30);
  text(answer, 20, 60);   // startSound() resolved false
}
Nothing can make sound: the state stays "off", startSound() resolves false, and the 🔊 button is not in the panel.

Related

  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • stopSound() Turns sound off and lets every ringing note go; the instrument stays loaded, so startSound() brings it back at once.
  • soundIsOn Whether notes are being heard right now: true once startSound() has finished, false before and after.
  • tone Where Tone.js, the sound engine behind startSound(), is loaded from: an ES module address. Empty means jsDelivr.
  • OAVp5.play() Plays a key as if a controller did: midi.play(60, 0.8) presses middle C, midi.play(60, 0) lets it go.

p5/oav-p5.js, line 130

strudel#

optionoav-p5
?oav_strudel=auto | on | off

Whether oav-p5 treats the page as Strudel, the live-coding editor, and behaves politely beside it: auto, on or off.

Strudel is a live-coding editor, not a sketch, so a few things change when oav-p5 sees it: the names Strudel owns (note, velocity, beat) are left alone and one card says what to read instead, MIDI is asked for by Strudel's own midin() rather than at load (hardware becomes click), the 🎹 panel becomes the compact box, and the 🎹 sits above Strudel's side-panel buttons. oav-p5 only looks at the address and at the names on the page: nothing of Strudel is called.

on forces this behaviour on any page, off forces the plain one even on strudel.cc. Settings you give yourself (hardware, compact) still win over what Strudel mode would choose.

Type
String
Values
auto | on | off
default
auto

auto means on strudel.cc, or on a page where Strudel's own functions (note() and s() with stack() or samples()) are already there. Anything else means auto.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { strudel: … }; above the loader line wins, then ?oav_strudel=… on the page address, then ?strudel=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_strudel=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { strudel: 'on' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('hardware: ' + OAVp5.options.hardware, 20, 30);   // click: Strudel asks for MIDI itself
  text('compact panel: ' + OAVp5.options.compact, 20, 60);   // true
}
On an ordinary page, strudel on switches to Strudel's manners: Web MIDI is not asked for at load and the panel will be the compact box.

Related

  • hardware When oav-p5 asks the browser for Web MIDI, which a real controller needs: auto at load, click when the 🎹 panel opens, or off.
  • compact Shows the 🎹 panel as a small box in the corner, a third of the window high, instead of a full-width bar: on or off.
  • link Joins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
  • clock Whether an external MIDI clock (from a DAW, Strudel or a drum machine) moves drumPhase, drumBpm and the drum machine: follow or off.
  • OAVp5.clock What oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.

p5/oav-p5.js, line 148

telemetry#

optionoav-p5
?oav_telemetry=off | on

Whether the on-screen controller may send one anonymous page-load count to the project: off or on.

In oav-p5 it is off. With on, when the 🎹 panel builds the on-screen controller, @openav/midi sends one anonymous hit per page load to the project's Google Analytics, so the project can see where its controllers end up. The @openav/midi README (section Telemetry) lists exactly what it sends: the kind of embed, the profile, the page's origin (never the path, query or title) and the version. Nothing anyone plays is sent and no cookie is set.

Type
String
Values
off | on
default
off

Only on turns it on.

Notes

  • According to that README nothing is sent from localhost, local network addresses, file:// or openaudiovisual.com itself, or when the browser sends Do Not Track or Global Privacy Control.
  • Set it before oav-p5 loads. window.OAV_P5 = { telemetry: … }; above the loader line wins, then ?oav_telemetry=… on the page address, then ?telemetry=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_telemetry=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { telemetry: 'on' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('telemetry: ' + OAVp5.options.telemetry, 20, 30);   // nothing visible changes
}
Nothing visible changes. The option is passed to the on-screen controller when the 🎹 panel first opens.

To be confirmed

The hit itself was not observed here: the example runs on localhost, where nothing is sent. What it sends is as stated in the @openav/midi README.

Related

  • panel Whether the 🎹 panel starts shut, open, or does not exist: closed, open or off.
  • profile Which controller the on-screen 🎹 shows, by profile id such as akai-lpd8; by default it follows the one that is plugged in.

p5/oav-p5.js, line 129

title#

optionoav-p5
?oav_title=a name

A name that goes into the file name of recordings and images, as in oav-my-piece-20261015-183012.mp4.

Without it a recording is oav-20261015-183012.mp4. With title: 'my-piece' it is oav-my-piece-20261015-183012.mp4, so takes from several pieces sort themselves. One take can have its own name: startRecording({ name: 'take-2' }) replaces it.

Type
String
Values
a name
default
'' (no name)

Spaces and the characters a file name cannot hold become -, and only the first 40 characters are used.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { title: … }; above the loader line wins, then ?oav_title=… on the page address, then ?title=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_title=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { title: 'my-piece' };

let saved = 'press r to record 2 seconds';

function setup() {
  createCanvas(400, 300);
}

function keyPressed() {
  if (key === 'r') startRecording(2);
}

function recordingSaved(info) {
  saved = 'saved as ' + info.name;
}

function draw() {
  background(20);
  fill(255);
  text(saved, 20, 30);
}
Press r: two seconds are recorded and downloaded, and the sketch shows the file name, which starts with oav-my-piece-.

Related

  • startRecording() Records the canvas and the sound your sketch makes into one video file, and returns a promise that says whether the recording began.
  • recordingSaved() An event you define: it is called when a recording has been saved and checked, with an info object about the file.
  • record Whether recording is available and whether it includes the microphone: on, nomic or off.

p5/oav-p5.js, line 133

tone#

optionoav-p5
?oav_tone=an ES module URL

Where Tone.js, the sound engine behind startSound(), is loaded from: an ES module address. Empty means jsDelivr.

Sound (startSound(), the sound option, the 🔊 button) uses Tone.js 15, which oav-p5 loads the first time sound is wanted. By default it comes from jsDelivr. Point tone at your own copy when the page is behind a school firewall or an ad blocker that stops the CDN, or when you serve everything yourself. If it cannot be loaded, a card says which host was unreachable and what to try.

Type
String (URL)
Values
an ES module URL
default
'' (jsDelivr)

The address of Tone.js version 15 as an ES module. Left empty, it is https://cdn.jsdelivr.net/npm/tone@15.0.4/+esm, and OAVp5.options.tone reads ''.

Notes

  • Set it before oav-p5 loads. window.OAV_P5 = { tone: … }; above the loader line wins, then ?oav_tone=… on the page address, then ?tone=… on the script's own URL. Values in window.OAV_P5 are turned into text, so write 'off', not false. On OpenProcessing the page address takes ?sketch_oav_tone=… (the project's README says OpenProcessing forwards it to the sketch).

Examples

jsOpenProcessing ↗
window.OAV_P5 = { tone: 'https://cdn.jsdelivr.net/npm/tone@15.0.4/+esm' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('Tone.js from: ' + OAVp5.options.tone, 20, 30);   // the address here is the built-in one
}
Prints where Tone.js would come from. The address here is the built-in one, so nothing changes. Swap in your own copy to host it yourself.

Related

  • sound Whether notes are heard: click (only after startSound() or 🔊), off (never), or an instrument id that starts at the first click.
  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • soundIsOn Whether notes are being heard right now: true once startSound() has finished, false before and after.

p5/oav-p5.js, line 130

The OAVp5 object (advanced)#

Everything oav-p5 exposes under OAVp5 and midi: state, subscriptions, the check-up. 13

oav-p5

OAVp5#

objectoav-p5
OAVp5

Also: midi oavMidi

The object oav-p5 hangs everything on: MIDI state, the panel, the microphone, the clock, the check-up. midi and oavMidi are the same object.

It is there as soon as oav-p5 has loaded, in setup() and draw() alike. The plain names of the p5 style (knob1, note, micLevel) are the easy way in. OAVp5 is the way in when you want more: the raw MIDI state (OAVp5.note, OAVp5.cc, OAVp5.bend, OAVp5.held, OAVp5.last), subscriptions (OAVp5.on()), a way to send bytes (OAVp5.input()), the microphone's numbers (OAVp5.mic), the external clock (OAVp5.clock), the link (OAVp5.link), the values over time (OAVp5.chop) and the check-up (OAVp5.check()).

Three names, one object. OAVp5 is always there. oavMidi is always there too. midi is there unless the page already had a midi of its own, in which case it is left alone and the console says to read OAVp5 instead. OAVp5.options holds the loading options as oav-p5 read them: OAVp5.options.mic, and so on for each of the loading options (for example mic).

Before it has a value

Only the loader line. If it has not loaded, OAVp5 is not defined, and a sketch that reads it stops with "OAVp5 is not defined".

Notes

  • midi and oavMidi are the same object under other names.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('same object: ' + (OAVp5 === midi && OAVp5 === oavMidi), 20, 30);
  text('version: ' + OAVp5.version, 20, 60);
  text('MIDI hardware: ' + OAVp5.hardware, 20, 90);
  text('microphone: ' + OAVp5.mic.state, 20, 120);   // reading the state does not start it
  text('external clock: ' + OAVp5.clock.info.state, 20, 150);
}
A small status board: the three names are one object, then the version and the state of MIDI, the microphone and the clock.

Related

  • OAVp5.version The version of oav-p5 that loaded, as text such as '0.8.0'.
  • OAVp5.on() Listens to a MIDI event by name without defining a global function, and returns a function that stops listening.
  • OAVp5.input() Sends one MIDI message into oav-p5 by hand, as bytes: the whole sketch reacts as if a controller had sent it.
  • OAVp5.check() Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
  • OAVp5.mic The microphone under the plain names: its state, the numbers of micLevel and bass that never turn it on when read, the raw loudness, start, stop and events.
  • OAVp5.clock What oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
  • OAVp5.chop The CHOP functions under one roof, plus the state of the module behind them: OAVp5.chop.lag(…), .state, .slots(), .reset().

p5/oav-p5.js, line 110

OAVp5.check()#

functionoav-p5
OAVp5.check([opts])

Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.

The same thing the 🩺 button in the panel does. When something does not work for a student, OAVp5.check() in the console (or 🩺) gives a card that says what this browser did: whether the page is in an iframe, whether the microphone is on or blocked, whether Web MIDI exists and which devices it found, the last message, the sound and drum state, whether MP4 recording works, what the controller is wired to, which of the sketch's own names stepped in the way, and which cards have appeared. It has a copy button and a link to the site's full device check.

Parameters

NameTypeDescription
optsObject{ show: false } skips the card on the page and only returns the report (the text still goes to the console). Left out, the card is shown. optional

Returns

Object The report: { version, time, page, browser, microphone, midi, sound, drums, wiring, own, hints, external, recording, text }. text is the whole card as plain lines, the other fields are the same facts as data (microphone.state, midi.hardware, midi.devices …).

Notes

  • It is a snapshot taken when you call it, not a live view. Calling it again replaces the card.

Examples

jsOpenProcessing ↗
let report;

function setup() {
  createCanvas(400, 300);
  report = OAVp5.check({ show: false });   // the report, without the card on the page
}

function draw() {
  background(20);
  fill(255);
  textSize(10);
  text(report.text, 10, 10, width - 20, height - 20);
}
Prints the check-up text on the canvas: version, browser, microphone, MIDI and the rest, one line each.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function mousePressed() {
  OAVp5.check();   // the card appears over the page
}

function draw() {
  background(20);
  fill(255);
  text('click the canvas for the check-up card', 20, 30);
}
Click the canvas: the check-up card opens over the page, with a copy button, ready to screenshot.

Related

  • OAVp5.hints The list of cards oav-p5 has shown or would have shown so far (a blocked microphone, no Web MIDI, an error in your callback), each with what, why and what to press.
  • OAVp5.version The version of oav-p5 that loaded, as text such as '0.8.0'.
  • debug Prints [oav-p5 debug] lines in the console about clicks, the 🎹 button and loading, for finding out why the 🎹 does not respond.
  • OAVp5.mic.state Where the microphone is: 'off', 'asking', 'on', 'denied' or 'unavailable'.
  • OAVp5.hardware Where Web MIDI stands: 'off', 'asking', 'connected', 'waiting', 'denied' or 'unavailable'.

p5/oav-p5.js, line 2818

OAVp5.clock#

objectoav-p5
OAVp5.clock

What oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.

Some programs keep the time and everyone else follows: Strudel (midicmd("clock*96, <start stop>/8")), a DAW over a virtual cable, a drum machine on a MIDI cable. They send 24 clock messages per beat, plus start, continue, stop and song position. oav-p5 counts them: OAVp5.clock.info says whether a clock is running, OAVp5.clock.bpm its tempo and OAVp5.clock.phase how far through the bar and the beat it is now.

When a clock is moving, drumPhase, beatPhase, drumBar, drumStep and drumBpm read it, and a drum machine that is up follows it. ?oav_clock=off turns all of that off. Clock messages are not delivered to midiMessage(): they belong to the clock.

Notes

  • The first clock message loads the counting code (@openav/midi's MidiClock) from next to oav-p5.js, which takes a few frames. Until then the members read none, null and null.
  • This is advanced: most sketches only read drumPhase and let it follow a clock.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  OAVp5.input([0xFA]);                          // MIDI start
  setInterval(() => OAVp5.input([0xF8]), 20);   // clock ticks: 24 a beat, here 50 a second (125 BPM)
}

function draw() {
  background(20);
  fill(255);
  text('clock: ' + OAVp5.clock.info.state, 20, 30);
  text('tempo: ' + (OAVp5.clock.bpm ? nf(OAVp5.clock.bpm, 1, 1) : '-') + ' BPM', 20, 60);
  const p = OAVp5.clock.phase;
  if (p) circle(40 + p.beat * (width - 80), 150, 30);   // sweeps across once a beat
}
The sketch sends itself a made-up MIDI clock at about 125 BPM. A circle sweeps across once per beat, and the state and tempo are printed.

Related

  • OAVp5.clock.info A snapshot of the external clock: state (none, running, stopped or lost), bpm, whether the drums follow it, where it came from, and counts.
  • OAVp5.clock.bpm The tempo of the external MIDI clock in beats a minute, or null while there is none.
  • OAVp5.clock.phase Where the external clock is right now: { pos, bar, beat, bpm, running }, or null while there is no tempo.
  • clock Whether an external MIDI clock (from a DAW, Strudel or a drum machine) moves drumPhase, drumBpm and the drum machine: follow or off.
  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
  • beatPhase How far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.

p5/oav-p5.js, line 713

OAVp5.clock.bpm#

variableoav-p5
OAVp5.clock.bpm

The tempo of the external MIDI clock in beats a minute, or null while there is none.

The raw number, not rounded. Use it for anything that should run at the clock's tempo: OAVp5.clock.bpm / 60 is beats a second. If the drum machine follows the clock, drumBpm shows it rounded to a tenth.

Type
Number | null

Fitted over the last 24 clock ticks, so it settles within a moment and does not jump with one late tick. null before two ticks have arrived or when clock=off.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  OAVp5.input([0xFA]);                          // MIDI start
  setInterval(() => OAVp5.input([0xF8]), 20);   // clock ticks: 24 a beat, here 50 a second (125 BPM)
}

function draw() {
  background(20);
  fill(255);
  textSize(48);
  textAlign(CENTER, CENTER);
  const bpm = OAVp5.clock.bpm;
  text(bpm ? nf(bpm, 1, 1) + ' BPM' : 'no clock', width / 2, height / 2);
}
Prints the tempo of the made-up clock the sketch sends itself: about 125 BPM once a few ticks have arrived.

Related

  • OAVp5.clock What oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
  • OAVp5.clock.info A snapshot of the external clock: state (none, running, stopped or lost), bpm, whether the drums follow it, where it came from, and counts.
  • OAVp5.clock.phase Where the external clock is right now: { pos, bar, beat, bpm, running }, or null while there is no tempo.
  • drumBpm The tempo of the drum machine in beats per minute.

p5/oav-p5.js, line 716

OAVp5.clock.info#

objectoav-p5
OAVp5.clock.info

A snapshot of the external clock: state (none, running, stopped or lost), bpm, whether the drums follow it, where it came from, and counts.

state is 'none' until a clock message has come, 'running' while the clock is going, 'stopped' after a MIDI stop (the position holds and clocks may still arrive), and 'lost' when no clock message has arrived for 0.6 seconds. bpm is the tempo fitted over the last 24 ticks, rounded to two decimals, or null before two ticks. running is true while the transport runs, which a stream of clocks with no start message counts as too (implicit is true then).

following is true when the drums read this clock: it is running, has a tempo and clock=off is not set. source says where the messages came from: 'hardware', 'screen', 'phone', 'virtual-out' (a program playing into OAV Virtual MIDI) or 'link'. clocks, dupes, starts, stops and continues count what arrived; a start sent three times with one timestamp counts once and adds to dupes.

Type
Object

A new object each time you read it: { state, bpm, running, following, source, implicit, clocks, dupes, starts, stops, continues }. Before any clock message it is { state: 'none', bpm: null, running: false, following: false, source: null, clocks: 0 }.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  OAVp5.input([0xFA]);                          // MIDI start
  setInterval(() => OAVp5.input([0xF8]), 20);   // clock ticks: 24 a beat, here 50 a second (125 BPM)
}

function draw() {
  background(20);
  fill(255);
  const i = OAVp5.clock.info;
  text('state: ' + i.state + (i.running ? ' (running)' : ''), 20, 30);
  text('bpm: ' + i.bpm + ',  following: ' + i.following, 20, 60);
  text('source: ' + i.source + ',  clocks heard: ' + i.clocks, 20, 90);
}
Prints the clock's snapshot while the sketch sends itself a made-up clock: the state goes from "none" to "running" within a moment.

Related

  • OAVp5.clock What oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
  • OAVp5.clock.bpm The tempo of the external MIDI clock in beats a minute, or null while there is none.
  • OAVp5.clock.phase Where the external clock is right now: { pos, bar, beat, bpm, running }, or null while there is no tempo.
  • clock Whether an external MIDI clock (from a DAW, Strudel or a drum machine) moves drumPhase, drumBpm and the drum machine: follow or off.

p5/oav-p5.js, line 715

OAVp5.clock.phase#

variableoav-p5
OAVp5.clock.phase

Where the external clock is right now: { pos, bar, beat, bpm, running }, or null while there is no tempo.

A bar is four beats. bar and beat count up from 0 and fall back to 0 at the next downbeat, the same way drumPhase and beatPhase do. After a stop the position holds. A clock with no start message gets its downbeat from the first tick after a silence of half a second, so a phase is there anyway.

Type
Object | null

A new object each time you read it: pos is the number of bars since the start (0 is the downbeat of bar 1), bar and beat are 0..1 through the current bar and beat, bpm the tempo and running whether the transport is moving. null until a tempo is known.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  OAVp5.input([0xFA]);                          // MIDI start
  setInterval(() => OAVp5.input([0xF8]), 20);   // clock ticks: 24 a beat, here 50 a second (125 BPM)
}

function draw() {
  background(20);
  const p = OAVp5.clock.phase;
  if (!p) return;
  noStroke();
  fill(255, 211, 107);
  circle(40 + p.bar * (width - 80), 100, 30);   // once a bar (four beats)
  fill(120, 190, 255);
  circle(40 + p.beat * (width - 80), 200, 30);  // once a beat
}
Two circles sweep from left to right on the made-up clock: the blue one once per beat, the yellow one once per bar of four beats.

Related

  • OAVp5.clock What oav-p5 hears of an external MIDI clock: its state, tempo and where it is in the bar. Nothing until another program or device starts sending one.
  • OAVp5.clock.info A snapshot of the external clock: state (none, running, stopped or lost), bpm, whether the drums follow it, where it came from, and counts.
  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.
  • beatPhase How far through the current beat the drum machine is, smoothly from 0 up to just below 1, four times per bar.

p5/oav-p5.js, line 718

OAVp5.drums.clock#

objectoav-p5
OAVp5.drums.clock

How the drum machine keeps time: which clock moves it, how far ahead it schedules and how long the callbacks wait for the speakers.

This is the diagnostic behind the 🩺 check-up. When drum hits feel late or early, look here first: the handbook puts a Bluetooth speaker at 0.15 to 0.25 seconds of latency, and ?oav_drumlatency=200 sets it by hand. late, skipped, warning and why are further notes the check-up reads.

Type
Object
Range
{ clock, heartbeat, lookahead, latency, audio, extended, preDraw, … }

A new object on every read. clock is 'lookahead' (hits are handed to the audio clock ahead of time, the default) or 'frame' (moved by the frames, the drums option). heartbeat is 'worker' (a Web Worker wakes it every 25 ms), 'interval' (the fallback), 'frame' with the frame clock, or 'none' before the machine exists. lookahead is the seconds scheduled ahead and extended is true when a slow draw() made it look further ahead. latency is the seconds the callbacks wait for the output of the speakers, audio whether the kit's audio clock is running, and preDraw the p5 hook that delivers the callbacks right before draw().

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 120);
  textSize(14);
}

function draw() {
  background(20);
  fill(255);
  const c = OAVp5.drums.clock;
  text('clock: ' + c.clock + ' (' + c.heartbeat + ')', 20, 30);
  text('lookahead: ' + c.lookahead + ' s', 20, 60);
  text('latency: ' + c.latency + ' s', 20, 90);
  text('hook before draw: ' + (c.preDraw || 'none'), 20, 120);
}
The clock settings of the machine while it plays.

To be confirmed

What exactly late, skipped, warning and why count was not checked.

Related

  • OAVp5.drums The object behind the drum names: the state of the machine, its counters, the line of rhythm, its history and the loaded machine.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.
  • drumPhase How far through the current bar the drum machine is, smoothly from 0 up to just below 1, as it is heard.

p5/oav-p5.js, line 1451

OAVp5.drums.machine#

variableoav-p5
OAVp5.drums.machine

The drum machine from @openav/drums that the grid on the page belongs to, or null before it has loaded.

It is what mountDrums() of @openav/drums returns, with seq (the sequencer), engine (the kit), play(on), setPattern(p), setBpm(bpm), phase(), now(), info() and the rhythm functions. OAVp5.drums.machine.play(false) stops the sound and the counting but leaves the grid up: OAVp5.drums.state then says 'stopped'. It is an advanced handle; the names on this page cover what a sketch normally needs.

Type
Object | null

Set once, when startDrums() has loaded the machine. null before that and after a failed load.

Before it has a value

It is null until startDrums() (or 🥁) has loaded the machine. Check for null before you use it.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  startDrums('four on floor', 120);
  textAlign(CENTER, CENTER);
}

function mousePressed() {
  const m = OAVp5.drums.machine;
  if (m) m.play(!m.seq.playing);   // pause and resume, the grid stays
}

function draw() {
  background(OAVp5.drums.state === 'stopped' ? 80 : 20);
  fill(255);
  text('click to pause or resume (' + OAVp5.drums.state + ')', width / 2, height / 2);
}
A click pauses the machine and the grid stays on the page; the state reads "stopped" until the next click.

Related

  • OAVp5.drums The object behind the drum names: the state of the machine, its counters, the line of rhythm, its history and the loaded machine.
  • OAVp5.drums.state Where the drum machine is: 'off', 'loading', 'on', 'stopped' or 'error'.
  • startDrums() Starts a drum machine: a 16-step grid on the page that plays a preset, your own grid or one line of rhythm, and counts the bars for your sketch.

p5/oav-p5.js, line 1437

OAVp5.hints#

variableoav-p5
OAVp5.hints

The list of cards oav-p5 has shown or would have shown so far (a blocked microphone, no Web MIDI, an error in your callback), each with what, why and what to press.

Each situation is said once per page. The list holds them all, including the ones the page did not show because hints=off is set, so it is the place to look when a sketch does something odd and nothing is on screen: console.log(OAVp5.hints).

Type
Array<Object>

Starts empty and only grows. Each item is { id, kind, what, why, next }: id names the situation ('micDenied', 'chopOut:lag'), kind is 'error', 'warn' or 'info', and what, why, next are the three lines of the card in the browser's language.

Notes

  • A card that went away because the situation ended (the microphone came on) stays in the list.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  lag(1);   // an unnamed lag() outside draw(): oav-p5 has something to say
}

function draw() {
  background(20);
  fill(255);
  text(OAVp5.hints.length + ' card(s) so far', 20, 30);
  text(OAVp5.hints.map((h) => h.kind + ': ' + h.what).join('\n'), 20, 60);
}
Lists what oav-p5 has said so far: here, that lag() was called outside draw(). The same sentence is in the console.

Related

  • hints Whether oav-p5 shows its explaining cards beside the 🎹 (blocked microphone, no Web MIDI, an error in your callback): on or off.
  • OAVp5.check() Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.
  • debug Prints [oav-p5 debug] lines in the console about clicks, the 🎹 button and loading, for finding out why the 🎹 does not respond.

p5/oav-p5.js, line 1193

objectoav-p5
OAVp5.link

What the link to other oav-p5 pages in the same room is doing: only OAVp5.link.info, and null unless a room is set.

A link joins oav-p5 pages that share a room: what one page's programs play into OAV Virtual MIDI (notes, controllers and the clock) is heard by every other page in the room, at the moment it sounds there. You switch it on with ?oav_link=my-room, and OAVp5.link.info is where you look at how it is going. There is nothing else to call: the link is made and closed by the option.

Notes

  • This is advanced. It needs a network connection to the relay (relay).

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text(OAVp5.link.info ? 'linked' : 'not linked: add link: \'my-room\' to the options', 20, 30);
}
Says whether this page is linked. Without a link room it is not, and the sketch says so.

Related

  • OAVp5.link.info The state of the link: its room, state (connecting, open, closed), how many other pages are there, and the trip time. null when no room is set.
  • link Joins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
  • linkdelay Extra milliseconds of buffer on the notes a linked page receives, to smooth out an uneven network: 0 to 2000.
  • relay The address of the relay that phones and link rooms talk through: a WebSocket URL.

p5/oav-p5.js, line 854

OAVp5.link.info#

variableoav-p5
OAVp5.link.info

The state of the link: its room, state (connecting, open, closed), how many other pages are there, and the trip time. null when no room is set.

Read it in draw() for a status line, or look at the 🔗 on the 🎹, which turns green when another page is there, yellow while connecting or alone, and red when the relay is gone. The 🩺 check-up prints the same facts. A new object is made on every read.

Type
Object | null

null without a link room. With one: { room, state, peers, rtt, trip, late, sent, received, … }. state is 'connecting', 'open' or 'closed' (the relay is gone and it retries). peers is how many other pages were heard in the last 6 seconds. rtt is the shortest round trip to the relay in the last 30 seconds and trip the one-way trip estimated from it, both in milliseconds. late counts notes that arrived after the moment they should have sounded and were played at once. sent and received count messages. The object also has diff, lead, msgs, dupes and id, which explain the clock alignment (packages/relay/link.js).

Examples

js
window.OAV_P5 = { link: 'my-room' };

function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  const i = OAVp5.link.info;
  text('room: ' + i.room + ',  state: ' + i.state, 20, 30);
  text('other pages: ' + i.peers + ',  one-way trip: ' + (i.trip == null ? '-' : i.trip + ' ms'), 20, 60);
}
static Open the same address in two tabs: both say "open" and one other page, and show the trip time.
jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
}

function draw() {
  background(20);
  fill(255);
  text('link.info: ' + JSON.stringify(OAVp5.link.info), 20, 30);   // null: no room
}
Without a link room it prints null.

To be confirmed

The values between two real pages were not observed here: the headless check cannot reach a relay. The fields and their meanings are from the code (packages/relay/link.js) and the project's README.

Related

  • OAVp5.link What the link to other oav-p5 pages in the same room is doing: only OAVp5.link.info, and null unless a room is set.
  • link Joins a room: oav-p5 pages with the same room hear what each other's programs play, over the relay. A room name, or empty for none.
  • linkdelay Extra milliseconds of buffer on the notes a linked page receives, to smooth out an uneven network: 0 to 2000.
  • relay The address of the relay that phones and link rooms talk through: a WebSocket URL.

p5/oav-p5.js, line 856

OAVp5.sound.shell#

variableoav-p5
OAVp5.sound.shell

Also: OAVp5.sound.engine OAVp5.sound.module

The loaded parts of @openav/sound: the Sound shell, its Tone.js engine and the module itself. All three are null until sound has loaded.

OAVp5.sound.shell is the Sound instance of @openav/sound, OAVp5.sound.engine is the Tone.js engine it plays through (noteOn, noteOff, sustain, releaseAll, setInstrument) and OAVp5.sound.module is the whole module, for example OAVp5.sound.module.listInstruments(), which lists every instrument id. They are advanced handles: the sketch-level names (startSound(), soundIsOn) cover everything a sketch normally needs.

Type
Object | null

Set once, when startSound() has loaded @openav/sound (the engine and the shell at the same moment, the module a little before). null before that, and after a failed load.

Before it has a value

Call startSound() first and wait until OAVp5.sound.state is no longer 'loading', or check for null before you use them.

Examples

jsOpenProcessing ↗
let ids = [];

function setup() {
  createCanvas(400, 300);
  startSound('synth/pluck');
}

function draw() {
  background(20);
  fill(255);
  textSize(12);
  if (!ids.length && OAVp5.sound.module) ids = OAVp5.sound.module.listInstruments().map((d) => d.id);
  text(ids.length ? ids.join(', ') : 'loading…', 20, 20, width - 40, height - 40);
}
Lists every instrument id that startSound() accepts, as soon as the module has loaded.

Related

  • OAVp5.sound The object behind the sound names: the state, the instrument, the error, the loading progress and the loaded parts of @openav/sound.
  • startSound() Makes the keys you play heard through an instrument you name, and returns a promise that says whether sound is on.
  • OAVp5.sound.state Where sound is: 'off', 'waiting' (for a click), 'loading', 'on' or 'error'.

p5/oav-p5.js, line 1318

OAVp5.version#

variableoav-p5
OAVp5.version

The version of oav-p5 that loaded, as text such as '0.8.0'.

Useful in a bug report or to check which build a page got: openaudiovisual.com/p5/oav-p5.js is always the latest, while a jsDelivr address with a tag is frozen. The same number is on the first line of the 🩺 check-up.

Type
String

Set when oav-p5 loads and never changes.

Notes

  • If the file loads twice (the loader line and the OpenProcessing library list), the first one wins and the second does nothing.

Examples

jsOpenProcessing ↗
function setup() {
  createCanvas(400, 300);
  textAlign(CENTER, CENTER);
  textSize(32);
}

function draw() {
  background(20);
  fill(255);
  text('oav-p5 ' + OAVp5.version, width / 2, height / 2);
}
Shows the version of oav-p5 this page loaded.

Related

  • OAVp5 The object oav-p5 hangs everything on: MIDI state, the panel, the microphone, the clock, the check-up. midi and oavMidi are the same object.
  • OAVp5.check() Runs the 🩺 check-up: the version, page, browser, microphone, MIDI, sound, drums, recording, wiring and hints, as a card to screenshot and as an object.

p5/oav-p5.js, line 259

MIDI in and out (@openav/midi)#

Ports, parsing, profiles, clock, virtual MIDI. 42

@openav/midi

BEND_CENTER#

constant@openav/midi
BEND_CENTER = 8192

The resting value of a pitch-bend wheel: 8192, the middle of the 14-bit range 0..16383.

Use it when you build a pitch-bend message by hand: { type: 'pitchbend', ch: 1, value: BEND_CENTER } is a wheel at rest. Spring-loaded wheels in a profile (spring) go back to this value when released.

Type
Number
Range
8192

A constant. A pitch-bend value equal to it means "not bent" and becomes 0 in bendToUnit().

Examples

js
import { BEND_CENTER, encodeMessage } from '@openav/midi';

console.log(BEND_CENTER);                                          // 8192
console.log(encodeMessage({ type: 'pitchbend', ch: 1, value: BEND_CENTER }));
module Prints 8192, then the bytes of a pitch bend at rest.

Related

  • bendToUnit() Turns a 14-bit pitch-bend value (0..16383) into −1..1, with the centre (8192) exactly 0.
  • unitToBend() Turns a −1..1 number into a 14-bit pitch-bend value (0..16383), the inverse of bendToUnit().

import { BEND_CENTER } from '@openav/midi'; packages/midi/parse.js, line 15

bendToUnit()#

function@openav/midi
bendToUnit(v)

Turns a 14-bit pitch-bend value (0..16383) into −1..1, with the centre (8192) exactly 0.

MIDI centres pitch bend at 8192, so there are 8192 steps below the centre and 8191 above. Below the centre it divides by 8192 and above by 8191, so the bottom reaches exactly −1, the top exactly +1 and the rest position exactly 0. This is the number published as midi/bend and midi/ch/<ch>/bend.

Parameters

NameTypeDescription
vNumberA pitch-bend value, 0..16383 (what parseMessage() puts in value).

Returns

Number −1 at 0, 0 at 8192, +1 at 16383.

Examples

js
import { bendToUnit, unitToBend } from '@openav/midi';

console.log(bendToUnit(0), bendToUnit(8192), bendToUnit(16383));   // -1 0 1
console.log(bendToUnit(12288));                                    // about 0.5
console.log(unitToBend(bendToUnit(5000)));                         // 5000
module Prints −1, 0, 1, then about 0.5, then 5000: the round trip is exact.

Related

  • unitToBend() Turns a −1..1 number into a 14-bit pitch-bend value (0..16383), the inverse of bendToUnit().
  • BEND_CENTER The resting value of a pitch-bend wheel: 8192, the middle of the 14-bit range 0..16383.
  • genericSignals() Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.

import { bendToUnit } from '@openav/midi'; packages/midi/parse.js, line 92

CONTROL_TYPES#

constant@openav/midi
CONTROL_TYPES = array, 8 items

The kinds of control a profile may contain: knob, encoder, fader, pad, button, keys, wheel, strip.

What each sends: knob and fader a CC (0..1); encoder a CC that is either relative (position plus a /delta signal) or absolute; pad a note (velocity while held, with a /hit pulse) or a CC; button a note or CC, momentary or toggle; keys a range of notes; wheel and strip a CC or pitch bend, optionally bipolar and spring-loaded.

Type
string[]
Range
['knob', 'encoder', 'fader', 'pad', 'button', 'keys', 'wheel', 'strip']

A plain array. A control's type must be one of these or validateProfile() reports it.

Examples

js
import { CONTROL_TYPES, PROFILES } from '@openav/midi';

const count = {};
for (const p of PROFILES) for (const c of p.controls) count[c.type] = (count[c.type] || 0) + 1;
console.log(CONTROL_TYPES.map((t) => t + ': ' + (count[t] || 0)).join(', '));
module Prints how many controls of each type the shipped profiles hold, in the order of the list.

Related

  • validateProfile() Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
  • RESERVED_IDS Control ids a profile may not use, because they are signal suffixes: last, raw, delta, hit, on, off, pressure.
  • normalizeProfile() Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.

import { CONTROL_TYPES } from '@openav/midi'; packages/midi/profiles.js, line 27

controllerRoutes()#

function@openav/midi
controllerRoutes(profile, bindings, { short = profile.short, ...extra } = {})

Builds mapper routes for "this control drives that param": controllerRoutes(profile, { knob1: 'hue' }) routes the knob's signal to the param hue.

This is rule 1 of the project (continuous control goes through params): a world does not read midi/lpd8/k1 in update(), it declares a param and lets a route drive it. An id the profile does not have throws profile akai-lpd8 has no control "x", so a typo is loud.

Parameters

NameTypeDescription
profileObjectThe device's profile (raw or normalized): used to check the ids and to know which controls are bipolar.
bindingsObject{ controlId: target }, where target is a param name or a route object ({ target, inMin, inMax, … }). A pad written as { target: 'burst', on: 'hit' } listens to the /hit pulse instead of its velocity.
shortStringThe signal namespace. The default is the profile's short; give another when you run the device under a different one. optional default profile.short
extraObjectEvery other key of the third argument ({ inMin: 0, inMax: 0.5 }) is copied onto each route as a default; a key in the route object itself wins. optional

Returns

Object[] [{ source, target, … }], ready for mapper.addRoute(route). A bipolar control (a pitch wheel) gets inMin: -1, inMax: 1. Any other key in the third argument is copied onto every route.

Examples

js
import { controllerRoutes, profileById } from '@openav/midi';

const routes = controllerRoutes(profileById('lpd8'), {
  k1: 'hue',                                   // knob 1 drives the param "hue"
  k2: { target: 'size', inMin: 0, inMax: 1 },  // a route object, extra keys are kept
  pad1: { target: 'burst', on: 'hit' },        // fire on the hit, not on the velocity
});
for (const r of routes) console.log(r.source, '->', r.target);
module Prints midi/lpd8/k1 -> hue, midi/lpd8/k2 -> size and midi/lpd8/pad1/hit -> burst: the signal each control drives its param from.

Related

  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
  • profileSignals() Lists every signal name a profile can publish under its own namespace (midi/lpd8/… for the LPD8), for docs, tests and mapping screens.
  • linkControllers() Links the on-screen controllers to phones over the room relay: tells phones which device is on screen, echoes this machine's moves to them, and mirrors theirs here.

import { controllerRoutes } from '@openav/midi'; packages/midi/manager.js, line 159

createVirtualMIDIAccess()#

function@openav/midi
createVirtualMIDIAccess({ name = 'OAV Virtual MIDI', manufacturer = 'open-audiovisual', id = 'oav-virtual', sysex = false, onOutput = null, onClear = null } = {})

Builds a Web MIDI MIDIAccess that always has one more input and output of its own, so a sketch written for a MIDI keyboard works with none plugged in.

A 2018 sketch that calls navigator.requestMIDIAccess() (or WebMidi.js) and waits for a keyboard shows nothing on a laptop without one, and on an iPad has no Web MIDI at all. Hand it this access instead and the on-screen controller becomes a real-looking MIDI input: the sketch's own unmodified code receives midimessage events with the same bytes a keyboard sends.

The virtual OUTPUT is a real-looking port too: a program that plays out to it (Strudel's .midi('OAV Virtual MIDI'), a WebMidi.js sketch) calls send(bytes, timestamp), and onOutput hears both. Real ports can be merged in with addReal(realAccess): they come FIRST in inputs, the virtual one last, so a sketch that only listens to inputs[0] takes the hardware when it is there and the virtual controller when it is not. Real ports pass through untouched, so nothing is delivered twice.

Parameters

NameTypeDescription
nameStringThe name of the virtual port, as the sketch sees it. optional default 'OAV Virtual MIDI'
manufacturerStringThe manufacturer the virtual ports report. optional default 'open-audiovisual'
idStringThe port id prefix: the ports are <id>-in and <id>-out. optional default 'oav-virtual'
sysexBooleanWhat access.sysexEnabled reports. optional default false
onOutputFunction | null(bytes, timestamp) => … called when a program sends to the virtual OUTPUT. timestamp is when the bytes should sound (performance.now() milliseconds), or undefined for "now". optional default null
onClearFunction | nullCalled when the program calls output.clear() (Web MIDI: drop what is still waiting). optional default null

Returns

Object { access, input, output, emit(bytes), addReal(realAccess), real, requests, dispose() }. Give access to the program. emit() plays bytes into the virtual input.

Notes

  • access.inputs and access.outputs are Maps, as in Web MIDI. input.listening counts the listeners a program put on the port, and requests counts how often virtualRequestMIDIAccess() handed the access out.

Examples

js
import { createVirtualMIDIAccess } from '@openav/midi';

const v = createVirtualMIDIAccess({
  name: 'Demo MIDI',
  onOutput: (bytes, timestamp) => console.log('program played', bytes, timestamp),
});

const input = [...v.access.inputs.values()][0];
const output = [...v.access.outputs.values()][0];
console.log(input.name, '|', input.listening, 'listeners');

input.onmidimessage = (e) => console.log('program heard', [...e.data]);   // what a 2018 sketch does
v.emit([0x90, 60, 100]);              // the on-screen keyboard plays: program heard [144, 60, 100]
output.send([0x90, 62, 80]);          // the program plays out:      program played [144, 62, 80] undefined
module Prints the port name with 0 listeners, then program heard [144, 60, 100], then program played [144, 62, 80] undefined (no timestamp means "now").

To be confirmed

Not tried for this page: a real WebMidi.js or Strudel sketch talking to the virtual ports, and how the onOutput timestamps line up with a hardware port. The example calls the ports directly.

Related

  • virtualRequestMIDIAccess() Wraps navigator.requestMIDIAccess so every caller gets the virtual access: real ports join if they answer in time, and the virtual input stays without Web MIDI.
  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.

import { createVirtualMIDIAccess } from '@openav/midi'; packages/midi/virtual-access.js, line 139

DEFAULT_PROFILE#

constant@openav/midi
DEFAULT_PROFILE = "arturia-minilab3"

The profile id used when none is given: 'arturia-minilab3'.

parseEmbedOptions() fills profile with it. createController() and ControllerCanvas have the same id as their own default.

Type
String
Range
'arturia-minilab3'

A constant. <oav-controller> without a profile attribute and the iframe embed without ?profile= show this device.

Examples

js
import { DEFAULT_PROFILE, parseEmbedOptions } from '@openav/midi/embed';

console.log(DEFAULT_PROFILE);
console.log(parseEmbedOptions({}).profile === DEFAULT_PROFILE);   // true
module Prints arturia-minilab3, then true.

Related

  • parseEmbedOptions() Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').

import { DEFAULT_PROFILE } from '@openav/midi/embed'; packages/midi/embed.js, line 26

describe()#

function@openav/midi
describe(ev)

Returns a short readable line for a parsed MIDI event, for logs and learn screens: CC ch1 #74=64, Bend ch1 0.50.

The formats are Note On ch1 n60 v100 (two spaces after "On"), Note Off ch1 n60, CC ch1 #74=64, Bend ch1 0.50 (the bend as −1..1 with two decimals), Pressure ch1 50, Poly AT ch1 n60 33 and Program ch1 5. The Midi engine writes one of these to its log for each message it parses (clock is left out).

Parameters

NameTypeDescription
evObjectAn event from parseMessage().

Returns

String The line. null gives ?, and a type without a format (clock, sysex) gives its type name.

Notes

  • Easily confused with describeMessage() and describeDetail(), which write the shorter lines the <oav-controller> readout shows (CC 70 = 53). This one takes an event and is meant for logs.

Examples

js
import { parseMessage, describe } from '@openav/midi';

for (const bytes of [[0x90, 60, 100], [0xb0, 74, 64], [0xe0, 0, 96]]) {
  console.log(describe(parseMessage(bytes)));
}
module Prints Note On ch1 n60 v100, CC ch1 #74=64 and Bend ch1 0.50.

Related

  • parseMessage() Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
  • describeMessage() Writes one parsed MIDI message the way a person reads it: CC 70 = 53, Note 36 · vel 90, Bend 12288.
  • describeDetail() Writes the readout line for a control event payload: midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1.

import { describe } from '@openav/midi'; packages/midi/parse.js, line 195

EchoGuard#

class@openav/midi
new EchoGuard({ window = 40, max = 256 } = {})

Recognises our own MIDI messages coming back: remember what was sent, and an identical message arriving within a short window is an echo, not a performer.

A virtual cable or a DAW can send your message straight back on an input, and the engine would treat it as a performer. sent(bytes) records a message and isEcho(bytes) asks whether an arriving one is that message returning. Each sent message cancels at most ONE echo, so two real identical messages are not both eaten.

A note-on with velocity 0 counts as a note-off, in both directions, because a DAW may rewrite one as the other. System messages (clock and the like) are never recorded. Midi has one (midi.echo, with the echoWindow option) and drops the echoes it finds, counted in midi.echoDropped.

Parameters

NameTypeDescription
windowNumberMilliseconds an echo may take to come back. 0 turns the guard off. optional default 40
maxNumberHow many sent messages to remember. Only the newest max are kept. optional default 256

Members

NameDescription
sent(bytes, now = performance.now())methodRecords a message we just sent. Messages of 0xF0 and above are ignored, and nothing is recorded when window is 0.
isEcho(bytes, now = performance.now())methodGives true when this arriving message is a recorded one coming back within window ms, and uses that memory up. Older records are forgotten.
clear()methodForgets everything recorded.

Examples

js
import { EchoGuard } from '@openav/midi';

const guard = new EchoGuard({ window: 40 });
guard.sent([0x90, 60, 100], 1000);                 // we sent note-on 60 at t = 1000 ms

console.log(guard.isEcho([0x90, 60, 100], 1010));  // true: it came back 10 ms later
console.log(guard.isEcho([0x90, 60, 100], 1011));  // false: one send explains one echo
guard.sent([0x90, 62, 100], 2000);
console.log(guard.isEcho([0x90, 62, 100], 2100));  // false: 100 ms later is a performer
module Prints true, false, false: an echo is used up once, and a message that arrives too late is a real note.

Related

  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
  • isLive() Says whether a MIDI port can still be sent to: it exists and its state is not 'disconnected'.

import { EchoGuard } from '@openav/midi'; packages/midi/ports.js, line 72

encodeMessage()#

function@openav/midi
encodeMessage(ev)

Turns an event object back into MIDI bytes, the inverse of parseMessage(): { type: 'cc', ch: 1, cc: 74, value: 64 } becomes [0xb0, 74, 64].

encodeMessage({ type: 'noteon', ch: 10, note: 36, vel: 127 }) is [0x99, 36, 127]. The channel is held to 1..16 and every data byte is rounded and held to 0..127, so a bad value cannot produce an invalid message. A missing vel is 100 for noteon and 0 for noteoff.

A pitchbend value (0..16383) is split into two 7-bit bytes, low byte first, as MIDI requires. The package uses this for the on-screen controller and for LED feedback.

Parameters

NameTypeDescription
evObjectAn event: type is noteon, noteoff, polyat, cc, program, chanat or pitchbend, with the fields parseMessage() uses for that type. ch defaults to 1.

Returns

number[] | null The bytes, ready for output.send(). null for a type it cannot make (clock, sysex, anything unknown).

Examples

js
import { encodeMessage, parseMessage } from '@openav/midi';

const bytes = encodeMessage({ type: 'noteon', ch: 10, note: 36, vel: 127 });
console.log(bytes);                // [153, 36, 127]
console.log(parseMessage(bytes));  // back to the same event
module Prints [153, 36, 127] (0x99, 0x24, 0x7f), then the event it came from.
js
import { encodeMessage } from '@openav/midi';

console.log(encodeMessage({ type: 'pitchbend', ch: 1, value: 8192 }));  // centre: [224, 0, 64]
console.log(encodeMessage({ type: 'cc', ch: 1, cc: 74, value: 300 }));  // value held to 127
console.log(encodeMessage({ type: 'clock' }));                          // null
module Three results: the pitch-bend centre, a value held to 127, and null for a message type it cannot make.

Related

  • parseMessage() Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
  • describe() Returns a short readable line for a parsed MIDI event, for logs and learn screens: CC ch1 #74=64, Bend ch1 0.50.
  • unitToBend() Turns a −1..1 number into a 14-bit pitch-bend value (0..16383), the inverse of bendToUnit().
  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.

import { encodeMessage } from '@openav/midi'; packages/midi/parse.js, line 48

eventOfSignal()#

function@openav/midi
eventOfSignal(name, value, { channel = 1 } = {})

The inverse of genericSignals(): turns one generic MIDI signal (midi/ch/10/note/36, midi/cc/74) back into the MIDI event it came from.

It lets anything that only hears a Signals bus (a recorded take playing back, the on-screen piano, a phone) drive a controller model as if the bytes had arrived. Values come back on the 0..127 scale they were published from (value × 127, rounded), so the round trip through genericSignals() is exact.

A note signal with a value above 0 is a noteon (velocity at least 1), and 0 is a noteoff. The on-screen piano publishes channel 0, which is read as channel. A hardware message is published under both families, so a listener that wants each message once should use followMidiSignals() instead of calling this on every name.

Parameters

NameTypeDescription
nameStringA signal name. Only the generic names starting with midi/ are understood.
valueanyThe signal's value: a number 0..1 (−1..1 for bend), or the { note, vel, velocity, ch } payload of midi/note/on and midi/note/off.
channelNumberThe channel to use for the legacy names that carry none (midi/cc/<n>, midi/bend). optional default 1

Returns

{ family, ev } | null family is 'channel' or 'legacy', and ev is the event. null for a device name (midi/minilab3/…), midi/virtual and anything else.

Examples

js
import { eventOfSignal } from '@openav/midi';

console.log(JSON.stringify(eventOfSignal('midi/ch/10/note/36', 0.5)));
console.log(JSON.stringify(eventOfSignal('midi/cc/74', 1, { channel: 4 })));
console.log(eventOfSignal('midi/minilab3/knob1', 1));   // a device name: null
module Prints a noteon on channel 10 with velocity 64, a CC 74 on channel 4 with value 127, then null.

Related

  • genericSignals() Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
  • followMidiSignals() Listens to a Signals bus and hands you each MIDI message back exactly once, as an event, although hardware publishes it under two names.
  • ControllerCanvas Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.

import { eventOfSignal } from '@openav/midi'; packages/midi/parse.js, line 156

findPort()#

function@openav/midi
findPort(ports, query)

Finds a MIDI port by name: a /regex/flags, then the exact name, then a case-insensitive substring; null when nothing matches.

This is how the midi-out attribute, the hardware attribute's port name and createController()'s setOutput(query) read what you typed.

Parameters

NameTypeDescription
portsObject[]Ports with a name (an array, such as midi.outputs).
queryStringA name, a part of a name ('IAC', 'loopMIDI'), or '/^mini/'. Flags after the closing slash are used; without flags the regex is case-insensitive.

Returns

Object | null The first matching port. A broken regex finds nothing instead of throwing.

Examples

js
import { findPort } from '@openav/midi';

const ports = [{ name: 'IAC Driver Bus 1' }, { name: 'loopMIDI Port' }, { name: 'Minilab3 MIDI' }];
console.log(findPort(ports, 'iac').name);        // IAC Driver Bus 1  (substring, any case)
console.log(findPort(ports, '/^mini/').name);    // Minilab3 MIDI
console.log(findPort(ports, 'nanoKONTROL'));     // null
console.log(findPort(ports, '/[/'));             // null: a broken regex finds nothing
module Prints IAC Driver Bus 1, Minilab3 MIDI, null, null.

Related

  • pickOutput() Chooses which MIDI output to send to from a list: the port you named, else the first match of a preference list, else the first live port.
  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').

import { findPort } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 143

findProfile()#

function@openav/midi
findProfile(ref, profiles)

Finds a profile in a list by id or short name, or returns null; no validation, no loading.

Use it when you hold your own list, for example the profiles a picker offers (controller.controllers.profiles), and want the same id-or-short lookup the element uses.

Parameters

NameTypeDescription
refStringA profile id or short name.
profilesObject[]The list to search (required here, unlike profileById(), which uses the shipped ones).

Returns

Object | null The profile in the list.

Examples

js
import { PROFILES } from '@openav/midi';
import { findProfile } from '@openav/midi/embed';

console.log(findProfile('lpd8', PROFILES).id);         // akai-lpd8
console.log(findProfile('akai-lpd8', PROFILES).short); // lpd8
console.log(findProfile('nope', PROFILES));            // null
module Prints akai-lpd8, lpd8 and null.

Related

  • profileById() Looks up a shipped controller profile by its id ('akai-lpd8') or its short signal name ('lpd8'), or returns null.
  • resolveProfile() Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.

import { findProfile } from '@openav/midi/embed'; packages/midi/embed.js, line 105

genericSignals()#

function@openav/midi
genericSignals(ev, { device = null, legacy = true, channel = true } = {})

Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.

Two families, both published for every hardware port and every virtual controller. The short ones are what most worlds read: a noteon becomes the pulse midi/note/on with { note, vel (0..1), velocity (1..127), ch, device }, a noteoff becomes midi/note/off with { note, ch, device }, a cc becomes midi/cc/<n> (0..1), and a pitchbend becomes midi/bend (−1..1).

The per-channel ones are for two devices that share a CC number, or a work that cares about the channel: midi/ch/<ch>/note/<n> (velocity 0..1 while held, 0 on release), midi/ch/<ch>/cc/<n>, midi/ch/<ch>/bend, midi/ch/<ch>/pressure (channel aftertouch) and midi/ch/<ch>/poly/<n> (polyphonic aftertouch). Pressure and poly exist only in this family. program, clock and the other system messages produce nothing.

There is deliberately no midi/cc/<ch>/<n>: under prefix subscriptions it would collide with "CC 1 on any channel".

Parameters

NameTypeDescription
evObjectAn event from parseMessage(). null gives an empty list.
deviceString | nullWritten into the device field of the midi/note/on and midi/note/off pulses (the port slug). optional default null
legacyBooleanInclude the short names that ignore the channel (midi/note/on, midi/cc/<n>, midi/bend). optional default true
channelBooleanInclude the per-channel names (midi/ch/<ch>/…). optional default true

Returns

{ name, value, pulse?, min?, max? }[] One item per signal. Hand the list to publish().

Examples

js
import { genericSignals } from '@openav/midi';

const names = (ev) => genericSignals(ev).map((s) => s.name);
console.log(names({ type: 'cc', ch: 2, cc: 74, value: 64 }));
console.log(names({ type: 'noteon', ch: 10, note: 36, vel: 127 }));
console.log(names({ type: 'chanat', ch: 1, value: 80 }));
module Prints ['midi/cc/74', 'midi/ch/2/cc/74'], then the two note-on names, then the single per-channel pressure name.
js
import { genericSignals } from '@openav/midi';

const ev = { type: 'noteon', ch: 1, note: 60, vel: 100 };
console.log(genericSignals(ev, { device: 'my-keys' })[0].value);
console.log(genericSignals(ev, { legacy: false }).map((s) => s.name));
module The first prints the pulse payload { note: 60, vel: 0.787…, velocity: 100, ch: 1, device: 'my-keys' }, the second only the per-channel name.

Related

  • eventOfSignal() The inverse of genericSignals(): turns one generic MIDI signal (midi/ch/10/note/36, midi/cc/74) back into the MIDI event it came from.
  • publish() Publishes a list of { name, value, pulse?, min?, max? } into a Signals registry, declaring each signal the first time it is seen.
  • parseMessage() Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
  • followMidiSignals() Listens to a Signals bus and hands you each MIDI message back exactly once, as an event, although hardware publishes it under two names.

import { genericSignals } from '@openav/midi'; packages/midi/parse.js, line 106

groupsOf()#

function@openav/midi
groupsOf(p)

Groups a profile's controls by their group (knob, pad, fader …), returning { group: [control ids] }.

The groups follow the TouchDesigner tool's out CHOPs (out_knob, out_slider, out_pad …). A control gets its type as group unless the profile gives another. MidiController has a method of the same name that returns the current values instead of the ids.

Parameters

NameTypeDescription
pObjectA normalized profile, so every control has its group filled in.

Returns

Object An object of arrays of control ids.

Examples

js
import { groupsOf, normalizeProfile, profileById } from '@openav/midi';

const groups = groupsOf(normalizeProfile(profileById('lpd8')));
console.log(Object.keys(groups));   // [ 'pad', 'knob' ]
console.log(groups.knob);           // [ 'k1', 'k2', ... 'k8' ]
module Prints the group names and then the eight knob ids.

Related

  • profileSignals() Lists every signal name a profile can publish under its own namespace (midi/lpd8/… for the LPD8), for docs, tests and mapping screens.
  • normalizeProfile() Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.

import { groupsOf } from '@openav/midi'; packages/midi/profiles.js, line 206

indexProfile()#

function@openav/midi
indexProfile(p)

Builds the lookup from a MIDI message to the control it moves: "cc:1:74" maps to the knob listening for CC 74 on channel 1.

This is how MidiController decides which control a message belongs to. Use it to build your own router. Each value is a list because a message can move more than one control.

Parameters

NameTypeDescription
pObjectA normalized profile (each control needs its ch).

Returns

Map<string, { control, note? }[]> The keys are cc:<ch>:<cc>, note:<ch>:<note>, polyat:<ch>:<note>, pitchbend:<ch>, chanat:<ch>, program:<ch>. A keys control gets one entry per note of its range.

Examples

js
import { indexProfile, normalizeProfile, profileById } from '@openav/midi';

const index = indexProfile(normalizeProfile(profileById('lpd8')));
console.log(index.get('cc:1:70').map((hit) => hit.control.id));     // [ 'k1' ]
console.log(index.get('note:10:36').map((hit) => hit.control.id));  // [ 'pad1' ]
console.log(index.get('cc:1:99'));                                  // undefined
module Prints ['k1'] for CC 70, ['pad1'] for note 36 on channel 10, and undefined for a message the device does not send.

Related

  • normalizeProfile() Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.

import { indexProfile } from '@openav/midi'; packages/midi/profiles.js, line 136

isLive()#

function@openav/midi
isLive(p)

Says whether a MIDI port can still be sent to: it exists and its state is not 'disconnected'.

A port that was unplugged can still be an object in your hands. Sending to it gets nowhere, and the engine's try/catch swallows the error, so nothing on screen says why. The output rules (pickOutput(), reselectOutput()) therefore skip every port that is not live.

Parameters

NameTypeDescription
pMIDIPort | nullA port shaped like a MIDIOutput: it only needs a state.

Returns

Boolean false for null and for a disconnected port.

Examples

js
import { isLive } from '@openav/midi';

console.log(isLive({ name: 'A', state: 'connected' }));      // true
console.log(isLive({ name: 'B', state: 'disconnected' }));   // false
console.log(isLive(null));                                   // false
module Prints true, false, false.

Related

  • pickOutput() Chooses which MIDI output to send to from a list: the port you named, else the first match of a preference list, else the first live port.
  • reselectOutput() Decides what to send to after the port list changed or the current port died, and returns the one line to print about it.

import { isLive } from '@openav/midi'; packages/midi/ports.js, line 10

isProfileUrl()#

function@openav/midi
isProfileUrl(ref)

Tells a URL to a profile file apart from a profile id: true for ./pads.json, https://…, /p/x, false for 'akai-lpd8'.

It is the test resolveProfile() uses to decide between looking an id up and fetching a file. A bare name that happens to end in .json also counts as a URL.

Parameters

NameTypeDescription
refanyWhat the page or attribute gave. Anything that is not a string is false.

Returns

Boolean true when the string starts with http:, https:, blob: or data:, starts with /, ./ or ../, or ends in .json (optionally followed by ?… or #…).

Examples

js
import { isProfileUrl } from '@openav/midi/embed';

for (const ref of ['akai-lpd8', './pads.json', 'https://x.org/my.json', '/p/x', 'my.json']) {
  console.log(ref, isProfileUrl(ref));
}
module Prints false for the id and true for the other four.

Related

  • resolveProfile() Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.
  • loadProfile() Loads a controller profile by id, short name or URL (a JSON file anywhere CORS allows), checking any file or object you give it; returns a promise.

import { isProfileUrl } from '@openav/midi/embed'; packages/midi/embed.js, line 102

linkControllers()#

function@openav/midi
linkControllers(relay, signals, controllers)

Links the on-screen controllers to phones over the room relay: tells phones which device is on screen, echoes this machine's moves to them, and mirrors theirs here.

A phone's MIDI tab publishes the device's signals itself (the controller lives there), so this side only does three things. When you select another device it tells the phones with relay.config('midi', { profile }). When a knob, fader, encoder, wheel, strip or toggle button on THIS machine moves from hardware or the screen, it queues relay.feedback('midi/<short>/<id>', value) so the phone's knobs follow. When a phone plays (midi/virtual) it mirrors that on this screen, state and view only, because the phone already published the signals.

This is what @openav/remote's show-side host and the lab's midi module call. You need it only when you wire phones into a page yourself.

Parameters

NameTypeDescription
relayObjectA relay client with config(name, value) and feedback(signalName, value) (the show runner's RelayClient).
signalsSignalsThe bus the phones' midi/virtual messages arrive on.
controllersMidiControllersThe manager whose controllers are linked.

Returns

{ flush(), dispose() } flush() sends the queued feedback to the relay (call it at a steady pace, for example once per frame). dispose() stops listening.

Examples

js
import { Signals } from '@openav/core';
import { MidiControllers, PROFILES, linkControllers } from '@openav/midi';

const signals = new Signals();
const mcs = new MidiControllers({ profiles: PROFILES, signals, initial: 'akai-lpd8' });

// a stand-in for the relay client: it prints what would go to the phones
const relay = {
  config: (name, value) => console.log('config', name, value),
  feedback: (name, value) => console.log('feedback', name, value.toFixed(2)),
};
const link = linkControllers(relay, signals, mcs);

mcs.current.setValue('k1', 0.5);   // a knob moves on this machine
link.flush();                       // feedback midi/lpd8/k1 0.50
mcs.select('korg-nanokontrol2');   // config midi { profile: 'korg-nanokontrol2' }
link.dispose();
module Prints feedback midi/lpd8/k1 0.50 after the flush, then config midi { profile: 'korg-nanokontrol2' } when the device on screen changes.

Related

  • MidiControllers Ties hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
  • controllerRoutes() Builds mapper routes for "this control drives that param": controllerRoutes(profile, { knob1: 'hue' }) routes the knob's signal to the param hue.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.

import { linkControllers } from '@openav/midi'; packages/midi/link.js, line 11

loadProfile()#

function@openav/midi
loadProfile(ref, o = {})

Loads a controller profile by id, short name or URL (a JSON file anywhere CORS allows), checking any file or object you give it; returns a promise.

It is resolveProfile() with the shipped profiles and the page's base URL filled in. <oav-controller profile="./my-device.json"> calls it to read your file. A bad file says why instead of drawing nothing.

Parameters

NameTypeDescription
refString | ObjectAn id ('akai-lpd8'), a short name ('lpd8'), a URL or path to a profile JSON, or a profile object.
oObjectOptions passed on to resolveProfile(): fetch, base, and profiles (default: every shipped profile). base defaults to the page's document.baseURI, so a relative path is read from the page. optional default {}

Returns

Promise<Object> The raw profile. Rejects with every problem listed when a file or object is invalid, or with unknown MIDI profile "x" (known: …) when an id does not exist.

Examples

js
import { loadProfile } from '@openav/midi';

const lpd8 = await loadProfile('lpd8');
console.log(lpd8.id, lpd8.controls.length);

try { await loadProfile('no-such-box'); } catch (e) { console.log(e.message.slice(0, 70) + ' …'); }
module Prints akai-lpd8 16, then the beginning of the error for an id that does not exist (it goes on to list every known id).
js
import { loadProfile, profileById } from '@openav/midi';

// a profile file of your own; here a fake fetch stands in for the network
const mine = { ...structuredClone(profileById('lpd8')), id: 'my-lpd8', short: 'mylpd8', name: 'My LPD8' };
const fetch = async (url) => { console.log('fetching', url); return { ok: true, json: async () => mine }; };

const p = await loadProfile('./profiles/my-lpd8.json', { fetch, base: 'https://example.com/show/' });
console.log(p.name);
module Prints the full URL it fetched (https://example.com/show/profiles/my-lpd8.json), then My LPD8.

Related

  • resolveProfile() Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.
  • profileById() Looks up a shipped controller profile by its id ('akai-lpd8') or its short signal name ('lpd8'), or returns null.
  • validateProfile() Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').

import { loadProfile } from '@openav/midi'; packages/midi/index.js, line 261

matchProfile()#

function@openav/midi
matchProfile(portName, profiles)

Finds the device profile that owns a MIDI port name, such as 'Minilab3 MIDI', by testing each profile's match patterns.

Only device profiles are considered (a generic layout never matches by name). A profile matches when one of its match.ports regular expressions (case-insensitive) hits the name. The score starts at 1, gains 1 when a match.prefer pattern also hits, and loses 2 when a match.avoid pattern hits. The highest positive score wins, so a port a device only exposes for something else (Minilab3 DIN THRU, Minilab3 MCU/HUI) does not match.

This is how MidiControllers turns a plugged-in device into the right on-screen controller. When a device exposes several ports, pickPort() chooses which one to listen to.

Parameters

NameTypeDescription
portNameStringThe name of a MIDI input port.
profilesObject[]The profiles to search, usually PROFILES.

Returns

{ profile, score } | null The best match, or null when no device profile fits.

Examples

js
import { matchProfile, PROFILES } from '@openav/midi';

for (const name of ['LPD8 mk2', 'Minilab3 MIDI', 'Minilab3 DIN THRU', 'Some Unknown Synth']) {
  const m = matchProfile(name, PROFILES);
  console.log(name, '->', m ? m.profile.id + ' (score ' + m.score + ')' : null);
}
module Prints akai-lpd8 (score 1), arturia-minilab3 (score 2), then null for the DIN THRU port and for an unknown synth.

Related

  • pickPort() Of several ports one device exposes ("Minilab3 MIDI", "Minilab3 DIN THRU", "Minilab3 MCU/HUI"), picks the one to listen to.
  • MidiControllers Ties hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
  • PROFILES Every controller profile that ships with the package, as an array of plain JSON objects: real devices first, then the generic layouts.

import { matchProfile } from '@openav/midi'; packages/midi/profiles.js, line 163

Midi#

class@openav/midi
new Midi({ signals = null, filterOut = 'IAC', requestAccess = null, echoWindow = 40, prefer = [] } = {})

The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.

Create it, then await midi.enable(). It resolves true once the browser has given access and false when there is no Web MIDI (Safari, iPad) or the permission was refused; the reason is written to the log (onMessage). Nothing is listened to or sent before that.

In: every message from every listened input goes to the callbacks you added with listen(fn) (raw bytes, including clock), then is parsed and published as genericSignals() names. With two or more inputs it also publishes midi/<slug>/note/on, midi/<slug>/note/off, midi/<slug>/cc/<n> and midi/<slug>/bend, so devices do not collide. The slug comes from the port name (Minilab3 MIDI becomes minilab3-midi), so a device that drops and reconnects keeps its identity. Clock messages are not logged and publish nothing.

Out: send(bytes), noteOn(), noteOff() and cc() go to midi.out, the chosen output port, and are remembered for echo filtering. If that port is unplugged the engine moves to the next best live port (your choice by name, then prefer, then the first), moves back when your choice returns, and never lets an unrelated device steal the output. Every switch writes one line to the log and one console.warn, and calls the onOutput subscribers.

Hooks you can assign: onCC(cc, value01, ch), onNote(note, velocity01, on, ch), onMessage({ dir, text }) (the log), onSend(bytes) (sees everything sent, for a meter) and onDeviceChange(devices). Fields: enabled, inputs, outputs, out, access, echoDropped.

Before you call it

Call await midi.enable() first. Browsers with Web MIDI (Chrome, Edge, Firefox) ask the visitor for permission at that moment, so call it from a click when you can. Safari and iPad have no Web MIDI: enable() gives false.

Parameters

NameTypeDescription
signalsSignals | nullPublish every input message here as midi/… signals (see genericSignals()). Without it the engine still calls listen() callbacks and the on… hooks. optional default null
filterOutString | RegExp | nullInput port NAMES never to listen to, as a regular expression (a string is read as its source, case-insensitive). The default 'IAC' is the macOS bus you send into Ableton or TouchDesigner, so what you send never comes back as input. null listens to everything. optional default 'IAC'
requestAccessFunction | nullWhere the MIDIAccess comes from, called as requestAccess({ sysex: false }) and returning a promise. The default is navigator.requestMIDIAccess. Pass your own for a test, a shim, or a host that wraps Web MIDI for old sketches and hands the REAL one here. optional default null
echoWindowNumberMilliseconds. An input message identical to one just SENT within this window is our own echo and is dropped (counted in echoDropped). 0 turns it off. optional default 40
preferArray<String | RegExp>Output priority after the port you chose by name: the first entry that matches a live port wins (a string matches as a substring), else the first port. Example: ['IAC']. optional default []

Members

NameDescription
devices()methodThe inputs being listened to, for a picker: [{ slug, name, listening }]. Duplicate names get -2, -3 after the slug.
setListening(slug, on)methodMutes or unmutes one device without unplugging it, and tells the onDevices subscribers.
onDevices(fn)methodSubscribe to device-list changes (plug, unplug, mute): fn(devices). Returns a function that unsubscribes.
log(dir, text)methodWrites one line to the log by calling onMessage({ dir, text }). The engine uses dir 'in' and 'out'.
enable(preferredOut = '')methodAsks the browser for Web MIDI and wires the inputs. preferredOut is the output you want, by exact name and then by substring. Resolves true, or false when Web MIDI is missing or refused.
onOutput(fn)methodSubscribe to output changes (hot-plug, switch back, your own selectOutput): fn({ text, reason, next, out }). reason is found, fallback, back, none or select. Returns an unsubscribe function.
selectOutput(name)methodSends to the output with exactly this name from now on, and remembers it as YOUR choice: if it is unplugged and plugged in again, the output returns to it. Gives false when no output has that name.
outputFor(nameOrRe)methodThe output port a device answers on (for LED feedback): the one with the same name as its input, or the first whose name matches a RegExp. null when there is none.
listen(fn)methodSubscribe to the raw bytes of every listened input: fn(bytes, { slug, name, input }). Called before parsing, clock included. Returns an unsubscribe function.
send(bytes, { echo = true } = {})methodSends raw bytes to the output port, and to the onSend hook. Unless echo is false they are also remembered, so the same bytes coming back within echoWindow ms are dropped. Does nothing when there is no output.
noteOn(note, vel = 100, ch = 1)methodSends a note-on. note and vel are 0..127 and ch is 1..16.
noteOff(note, ch = 1)methodSends a note-off (velocity 0).
cc(cc, val, ch = 1)methodSends a control change.
dispose()methodLets go of every port and listener (an embed removed from the page). The MIDIAccess itself belongs to the browser and stays.
panic()methodSilences everything: all-notes-off, all-sound-off and sustain-off on all 16 channels, plus a note-off for every note. It goes through send(), so meters see the sweep.

Notes

  • createShow() creates one and enables it for you. createController() and <oav-controller> share one engine per page, so a page with five controllers asks for MIDI once.
  • The engine sends MIDI data only and plays no audio. To hear the notes you need a synth on the other end (a DAW on the IAC bus, a hardware synth), or @openav/sound.

Examples

js
import { Midi } from '@openav/midi';

// a stand-in for navigator.requestMIDIAccess, so this runs without a device
const output = { name: 'Demo out', state: 'connected', send: (bytes) => console.log('out', bytes) };
const access = { inputs: new Map(), outputs: new Map([['o', output]]) };

const midi = new Midi({ requestAccess: async () => access });
console.log(await midi.enable());   // true
midi.noteOn(60, 100);               // out [144, 60, 100]
midi.noteOff(60);                   // out [128, 60, 0]
module Prints true, then the two messages that reach the output port: note-on and note-off for middle C.
js
import { Signals } from '@openav/core';
import { Midi } from '@openav/midi';

const keys = { name: 'Demo Keys', state: 'connected', onmidimessage: null };
const access = { inputs: new Map([['k', keys]]), outputs: new Map() };

const signals = new Signals();
const midi = new Midi({ signals, requestAccess: async () => access });
await midi.enable();

signals.on('midi/cc/74', (v) => console.log('knob', v.toFixed(2)));
keys.onmidimessage({ data: Uint8Array.of(0xb0, 74, 127) });   // the keyboard "plays" CC 74 = 127
module Prints knob 1.00: a message arriving on the input port became the signal midi/cc/74.
js
import { Midi } from '@openav/midi';

const port = (name) => ({ name, state: 'connected', send: () => {} });
const iac = port('IAC Driver Bus 1'), synth = port('Synth');
const access = { inputs: new Map(), outputs: new Map([['a', iac], ['b', synth]]) };

const midi = new Midi({ requestAccess: async () => access });
midi.onOutput(({ reason, text }) => console.log(reason, '-', text));
await midi.enable('IAC');
console.log(midi.out.name);          // IAC Driver Bus 1

access.outputs.delete('a'); iac.state = 'disconnected';
access.onstatechange();              // what the browser does when a port is unplugged
console.log(midi.out.name);          // Synth
module Prints the chosen port, then a fallback line saying the output moved to "Synth" (the engine also writes it with console.warn), then Synth.
js
import { Midi } from '@openav/midi';

const midi = new Midi();
if (await midi.enable()) {
  console.log('inputs:', midi.devices().map((d) => d.name));
} else {
  console.log('no Web MIDI here, or the permission was refused');
}
static The real thing: the browser asks for MIDI permission, then the console lists your input devices (or says there is no Web MIDI).

To be confirmed

Not tried for this page: the real permission prompt and real MIDI hardware in each browser. The examples run against a stand-in for MIDIAccess, and the browser behaviour (Chrome, Edge and Firefox ask for permission, Safari and iPad have no Web MIDI) is as the package README describes it.

Related

  • EchoGuard Recognises our own MIDI messages coming back: remember what was sent, and an identical message arriving within a short window is an echo, not a performer.
  • pickOutput() Chooses which MIDI output to send to from a list: the port you named, else the first match of a preference list, else the first live port.
  • reselectOutput() Decides what to send to after the port list changed or the current port died, and returns the one line to print about it.
  • genericSignals() Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
  • MidiControllers Ties hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
  • ScoreMidi Turns a show's segment changes and cues into MIDI notes and CCs on one channel, so a DAW or lighting desk can follow the show.

import { Midi } from '@openav/midi'; packages/midi/index.js, line 45

MidiController#

class@openav/midi
new MidiController(profile, { signals = null, sink = null, short = null, onLed = null } = {})

One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.

The one idea: the on-screen controller and the hardware speak the SAME language, MIDI bytes. setValue(), press() and the other virtual hands encode the exact message the real control would send and feed it through ingest(), the same door the hardware uses. So a virtual knob and a real one can never drift apart, and what your work receives is identical whichever hand moved it. Pass the bytes of a real device to ingest(bytes, { source: 'hw' }).

The source of a message decides what is published, so every event is published exactly once. 'hw': the device lives here, and Midi already published the generic midi/… names, so only midi/<short>/… is added. 'ui': a virtual hand, so it publishes midi/<short>/…, the generic names, and midi/virtual (the raw bytes, for Web MIDI shims). 'mirror': someone else published it (a phone, feedback), so only the state and the views move.

Signals per control (values are normalised, raw keeps the MIDI number): midi/<short>/<id> (0..1, −1..1 for a bipolar wheel) and /raw; an encoder's /delta; a pad's /hit pulse { vel, velocity, note, ch } and /pressure; a keys control's /on and /off pulses and midi/<short>/n<note> while a key is held; and midi/<short>/last, a pulse saying what was touched last. profileSignals() lists them for a profile.

Each control keeps a state { value, raw, held, pressure, pos } in controller.state (read it with get(id)). controller.keys maps each held key's note to its velocity. controller.profile is the working copy (learning changes it), controller.base the normalized profile before any learning, controller.lastEvent the last message with its text, and controller.learning and controller.learned track learn mode.

Parameters

NameTypeDescription
profileObjectA raw or normalized profile (see PROFILES). It is copied, so changing the object later does nothing.
signalsSignals | nullPublish the controller's signals here. optional default null
sinkFunction | null(name, value, { pulse, min, max }) => … called for each signal instead of signals (a relay, a phone). When both are given, sink wins. optional default null
shortString | nullOverrides the signal namespace, so two controllers of the same device publish under different names. optional default null
onLedFunction | null(bytes) => … called with the MIDI that lights or dims a control's LED on the hardware, whenever its state changes. optional default null

Members

NameDescription
idgetterThe profile's id, such as 'akai-lpd8'.
control(id)methodReturns the control's definition from the profile (type, msg, ch, cc/note …), or null when there is no such id.
get(id)methodReturns the control's state { value, raw, held, pressure, pos }, or undefined for an unknown id.
groups()methodThe current values grouped like the TouchDesigner tool's out CHOPs: { knob: [v1, v2 …], pad: […] }. Keys controls are left out. (Unlike groupsOf(), which lists ids.)
onChange(fn)methodSubscribe to changes: fn(id, state, info) with info { source, ev, note, bytes }. When the mapping or learn state changes instead, it is called as fn(null, null, { meta: true }). Returns an unsubscribe function.
onMessage(fn)methodSubscribe to every message that passes through, once, after the state has moved: fn({ bytes, ev, source, ids }), where ids are the controls it moved (none for a message the profile does not know). An embed forwards source: 'ui' messages to a MIDI output here. Returns an unsubscribe function.
ingest(data, { source = 'hw' } = {})methodFeeds MIDI bytes in. source is 'hw' (default), 'ui' or 'mirror'. Returns the parsed event, or null. Clock, sysex and system messages are returned without touching any control.
setValue(id, v, { source = 'ui' } = {})methodMoves an absolute control (knob, fader, wheel, strip, absolute encoder) to v: 0..1, or −1..1 for a bipolar one. Returns the bytes it sent, or null when the value did not change. For a relative encoder it turns by the matching steps.
turn(id, steps, { source = 'ui' } = {})methodTurns an endless encoder by signed steps, sent in the encoding its profile names. Returns the last bytes sent.
press(id, vel = 1, note = null, { source = 'ui' } = {})methodPresses a pad, button or key with a velocity from 0 to 1 (a pad or key velocity is held to 1..127; a button always sends 127). A key needs its note. A toggle button flips. Returns the bytes.
release(id, note = null, { source = 'ui' } = {})methodLets go of a pad, button or key (a key needs its note). A toggle button ignores it. Returns the bytes.
pressure(id, v, note = null, { source = 'ui' } = {})methodSets a pad's or key's pressure, 0..1, as polyphonic or channel aftertouch, whichever its profile says. Does nothing for a control without aftertouch.
rest(id, { source = 'ui' } = {})methodSends a spring-loaded wheel or strip home: pitch bend back to the centre.
releaseAll({ source = 'ui' } = {})methodLets go of every key, pad and momentary button the virtual hands hold. Call it on blur, page hide or an octave change so no note hangs.
mirror(id, value)methodApplies a value that arrived from elsewhere (relay feedback) to the state and the views, without publishing anything.
ledBytes(c, st = this.state.get(c.id), note = null)methodReturns the MIDI bytes that light (or dim) that control's LED on the hardware, from its state, or null when the profile has no led for it.
ledSnapshot()methodThe bytes for every LED at its current state, to send once when the hardware connects (the idle colours).
learn(id)methodArms learn mode for one control: the next compatible hardware message is bound to it. Calling it again for the same id cancels. Returns the armed id or null.
bindMessage(id, ev)methodRebinds a control to the message ev, swapping with whichever control had it before.
exportMapping()methodThe learned bindings as JSON, { controlId: { msg, ch, cc, note, from, to } }, containing only what differs from the profile. Persist it per device (localStorage, a file).
applyMapping(m = {})methodApplies a mapping from exportMapping() and marks those controls as learned.
resetMapping()methodGoes back to the profile's own bindings and forgets what was learned.

Notes

Examples

js
import { MidiController, profileById } from '@openav/midi';

const lpd8 = new MidiController(profileById('lpd8'));
lpd8.onChange((id, st, info) => id && console.log(info.source, id, st.value.toFixed(2), st.raw));

lpd8.ingest([0xb0, 70, 127], { source: 'hw' });   // the real knob 1 moved to the top
lpd8.setValue('k2', 0.5);                          // a finger on the on-screen knob 2
lpd8.press('pad1', 0.8);                           // a finger on pad 1
module Prints three lines: hw k1 1.00 127, ui k2 0.50 64, ui pad1 0.80 102. The same model, three hands.
js
import { Signals } from '@openav/core';
import { MidiController, profileById } from '@openav/midi';

const signals = new Signals();
const mc = new MidiController(profileById('minilab3'), { signals });
signals.on('midi/minilab3/pad3/hit', (hit) => console.log('pad 3 hit', hit));

mc.press('pad3', 0.5);
console.log(signals.get('midi/minilab3/pad3'));       // 0.50...: velocity while held
mc.release('pad3');
console.log(signals.get('midi/minilab3/pad3'));       // 0
module Prints the /hit pulse { vel: 0.50…, velocity: 64, note: 38, ch: 10 }, then the pad's value while held and after release.
js
import { MidiController, profileById } from '@openav/midi';

// a generic layout learns whatever you move
const mc = new MidiController(profileById('generic-8k8p'));
mc.learn('k3');                                       // arm control k3
mc.ingest([0xb0, 50, 99], { source: 'hw' });          // the next CC from the hardware is bound to it
console.log(JSON.stringify(mc.exportMapping()));       // {"k3":{"msg":"cc","ch":1,"cc":50}}

const again = new MidiController(profileById('generic-8k8p'));
again.applyMapping(mc.exportMapping());
again.ingest([0xb0, 50, 10], { source: 'hw' });
console.log(again.get('k3').raw);                     // 10
module Prints the learned binding of k3, then 10: a second controller with the saved mapping reacts to the same CC.

Related

  • MidiControllers Ties hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
  • controllerRoutes() Builds mapper routes for "this control drives that param": controllerRoutes(profile, { knob1: 'hue' }) routes the knob's signal to the param hue.
  • profileSignals() Lists every signal name a profile can publish under its own namespace (midi/lpd8/… for the LPD8), for docs, tests and mapping screens.
  • parseMessage() Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.

import { MidiController } from '@openav/midi'; packages/midi/controller.js, line 36

MidiControllers#

class@openav/midi
new MidiControllers({ profiles, signals = null, sink = null, midi = null, initial = null, storage = safeStorage() } = {})

Ties hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.

There is one MidiController per profile, created the first time it is asked for (get(id)), so a virtual MiniLab and a real MiniLab are the SAME object: whichever hand moves it, the work receives the same midi/minilab3/… signals. current is the controller the panel shows.

With a midi attached it matches every input port to a profile by name (matchProfile(), one port per device through pickPort()), switches current to the first device it recognises, sends the LED colours to its output port, and routes each port's messages to that profile's controller. A port nobody recognises can be given to a profile by hand with assign(slug, profileId); that choice is remembered per port name.

It remembers in storage under openav.midi.map.<profile id> (learned mappings, restored when the controller is created) and openav.midi.assign (manual port assignments).

Parameters

NameTypeDescription
profilesObject[]The profiles to manage, usually PROFILES. Required: the first one is shown when initial is not given. optional
signalsSignals | nullPassed to every MidiController it creates. optional default null
sinkFunction | nullPassed on to every controller, as in MidiController. optional default null
midiMidi | nullAn enabled (or about-to-be-enabled) Midi to follow. Leave it out for no hardware, and call attach(midi) later. optional default null
initialString | nullThe profile id shown before any hardware arrives. optional default null
storageStorage | nullWhere learned mappings and manual port assignments are remembered: anything with getItem and setItem. The default is localStorage when the page has it, otherwise nothing is remembered. optional default safeStorage()

Members

NameDescription
attach(midi)methodStarts following a hardware Midi (now, or later: an embed connects only when asked). Switching to another engine detaches the first. Returns the manager.
detach()methodStops following the hardware. The on-screen controllers stay and are still playable.
register(profile)methodAdds (or replaces) a profile at runtime, for example one loaded from a URL or one moved to another channel with withChannel().
get(profileId)methodThe MidiController for a profile id, created on first use with any learned mapping restored. Throws unknown MIDI profile "x" for an id that was never registered.
select(profileId)methodMakes a profile the current one (the on-screen device) and tells the onChange subscribers with { type: 'select' }. Returns its controller.
onChange(fn)methodSubscribe to { type: 'select' } (the device on screen changed) and { type: 'ports' } (ports or their assignments changed): fn(event, manager). Returns an unsubscribe function.
devices()methodThe hardware ports and who they drive: [{ slug, name, listening, profileId, auto }]. auto is true when the match was made by name, false when you assigned it.
hardwareFor(profileId)methodThe name of the port feeding this profile right now, or null when no hardware drives it.
assign(slug, profileId, { persist = true } = {})methodRoutes a port to a profile by hand (an unknown device to a generic layout), or un-routes it with null. It then selects that profile. persist (default true) remembers the choice for this port name.
dispose()methodDetaches from the hardware and forgets the subscribers.

Examples

js
import { Signals } from '@openav/core';
import { MidiControllers, PROFILES } from '@openav/midi';

const signals = new Signals();
const mcs = new MidiControllers({ profiles: PROFILES, signals, initial: 'akai-lpd8' });

mcs.current.setValue('k1', 1);
console.log(signals.get('midi/lpd8/k1'));    // 1
mcs.select('korg-nanokontrol2');
console.log(mcs.current.id);                  // korg-nanokontrol2
module Prints 1 (a finger on the on-screen LPD8 knob became a signal), then korg-nanokontrol2 after switching the device on screen.
js
import { Signals } from '@openav/core';
import { Midi, MidiControllers, PROFILES } from '@openav/midi';

// a fake Minilab3 on a fake MIDIAccess
const keys = { name: 'Minilab3 MIDI', state: 'connected', onmidimessage: null };
const access = { inputs: new Map([['k', keys]]), outputs: new Map() };

const signals = new Signals();
const midi = new Midi({ signals, requestAccess: async () => access });
await midi.enable();

const mcs = new MidiControllers({ profiles: PROFILES, signals, midi });
console.log(mcs.current.id, '<-', mcs.hardwareFor('arturia-minilab3'));
keys.onmidimessage({ data: Uint8Array.of(0xb0, 74, 127) });         // the real knob 1
console.log(signals.get('midi/minilab3/knob1'));                     // 1
module Prints arturia-minilab3 <- Minilab3 MIDI (the port was recognised by name), then 1: a message from the port moved knob 1 of the on-screen MiniLab 3.

To be confirmed

LED feedback (the colours sent back to a device when it is recognised) was not tried on real hardware. Some controls in the shipped profiles are marked verified: false: their messages come from the manual but nobody has confirmed them on a real unit yet, and Learn rebinds them.

Related

  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
  • matchProfile() Finds the device profile that owns a MIDI port name, such as 'Minilab3 MIDI', by testing each profile's match patterns.
  • pickPort() Of several ports one device exposes ("Minilab3 MIDI", "Minilab3 DIN THRU", "Minilab3 MCU/HUI"), picks the one to listen to.
  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
  • mountMidiPanel() Mounts a controller panel: a toolbar (device picker, hardware status, port picker, Learn, last message) above a playable faceplate, docked at the bottom or full size.

import { MidiControllers } from '@openav/midi'; packages/midi/manager.js, line 17

normalizeProfile()#

function@openav/midi
normalizeProfile(raw)

Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.

It throws invalid MIDI profile "<id>": followed by every problem when the profile is not valid. Otherwise it fills in: channel (1), learn (true for a generic layout), stack (true), sources and match (empty lists), and per control label (its id), group (its type), ch (the profile's channel), an encoder's relative (offset64), a button's mode (momentary) and a pitch-bend wheel's bipolar and spring (both true). Sections get label, flow (row) and spans, and portrait is built from the sections, top to bottom, with each section's height as its weight when you did not write one.

MidiController, ControllerCanvas and the view all work on the normalized form, which is what controller.profile gives you.

Parameters

NameTypeDescription
rawObjectA raw profile. It must pass validateProfile().

Returns

Object A new profile object.

Examples

js
import { normalizeProfile, profileById } from '@openav/midi';

const raw = profileById('lpd8');
const p = normalizeProfile(raw);

console.log(raw.channel, '->', p.channel);                          // undefined -> 1
console.log(p.controls[0].id, p.controls[0].label, p.controls[0].group);
console.log('learn:', p.learn, '| portrait bands:', p.portrait.length);

try { normalizeProfile({ id: 'X' }); } catch (e) { console.log(e.message); }
module Prints the defaults it filled in, then the error message listing what is wrong with { id: 'X' }.

Related

  • validateProfile() Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
  • PROFILES Every controller profile that ships with the package, as an array of plain JSON objects: real devices first, then the generic layouts.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
  • indexProfile() Builds the lookup from a MIDI message to the control it moves: "cc:1:74" maps to the knob listening for CC 74 on channel 1.

import { normalizeProfile } from '@openav/midi'; packages/midi/profiles.js, line 117

parseMessage()#

function@openav/midi
parseMessage(data)

Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.

parseMessage([0x90, 60, 100]) is { type: 'noteon', ch: 1, note: 60, vel: 100 }. Channels are numbered 1 to 16, the way they are printed on a device (the status byte stores 0 to 15), and data bytes are cut to 0..127. Every other function in this package that handles MIDI speaks these event objects.

The type is noteon and noteoff (note, vel), polyat (polyphonic aftertouch: note, value), cc (cc, value), program (value), chanat (channel pressure: value) or pitchbend (value 0..16383, centre 8192). System messages come back as clock, start, continue, stop, sysex (with data, the whole message) or system (with the status byte).

A note-on with velocity 0 comes back as noteoff, because that is how many devices say "released". Bytes that start with a data byte give null: the browser has already resolved running status before it hands you the message.

Parameters

NameTypeDescription
dataArray | Uint8ArrayThe bytes of ONE message, the way Web MIDI hands them over (event.data): a status byte first, then up to two data bytes.

Returns

Object | null The event, or null when the input is empty, starts with a data byte (below 0x80) or is a channel status it has no case for.

Notes

  • It is pure: no DOM, no Web MIDI, no state. encodeMessage() is its inverse, and describe() turns the result into a line of text.

Examples

js
import { parseMessage } from '@openav/midi';

console.log(parseMessage([0x90, 60, 100]));   // note on, middle C
console.log(parseMessage([0x90, 60, 0]));     // velocity 0 reads as note off
console.log(parseMessage([0xe0, 0, 64]));     // pitch bend, centre
module Open the console: three event objects, a noteon, a noteoff and a pitchbend with value: 8192.
js
import { parseMessage } from '@openav/midi';

// what a Web MIDI port gives you: event.data is a Uint8Array
const ev = parseMessage(Uint8Array.of(0xb0, 74, 100));
if (ev && ev.type === 'cc') {
  console.log('CC', ev.cc, 'on channel', ev.ch, '=', (ev.value / 127).toFixed(2));
}
module Prints CC 74 on channel 1 = 0.79: a control-change value turned into the 0 to 1 range a sketch wants.

Related

  • encodeMessage() Turns an event object back into MIDI bytes, the inverse of parseMessage(): { type: 'cc', ch: 1, cc: 74, value: 64 } becomes [0xb0, 74, 64].
  • describe() Returns a short readable line for a parsed MIDI event, for logs and learn screens: CC ch1 #74=64, Bend ch1 0.50.
  • genericSignals() Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.

import { parseMessage } from '@openav/midi'; packages/midi/parse.js, line 18

pickOutput()#

function@openav/midi
pickOutput(outputs, { preferred = '', prefer = [] } = {})

Chooses which MIDI output to send to from a list: the port you named, else the first match of a preference list, else the first live port.

The order is: 1) the port named preferred, 2) the first port that matches each entry of prefer, in order, 3) the first port. Only live ports (see isLive()) are considered, so a disconnected port is never picked.

It is a pure function over plain objects, so you can test the rules without any hardware. The Midi engine applies them through reselectOutput().

Parameters

NameTypeDescription
outputsMIDIOutput[]The candidate ports (objects with a name and a state).
preferredStringThe port you asked for: tried by exact name, then as a substring. optional default ''
preferArray<String | RegExp>Fallback order. The first entry that matches a live port wins (a string is a substring). optional default []

Returns

MIDIOutput | null The chosen port, or null when no port is live.

Examples

js
import { pickOutput } from '@openav/midi';

const port = (name, state = 'connected') => ({ name, state });
const outs = [port('Synth A'), port('IAC Driver Bus 1'), port('Synth B')];

console.log(pickOutput(outs).name);                              // Synth A
console.log(pickOutput(outs, { preferred: 'IAC' }).name);        // IAC Driver Bus 1
console.log(pickOutput(outs, { prefer: [/synth b/i] }).name);    // Synth B
console.log(pickOutput([port('X', 'disconnected')]));            // null
module Prints Synth A, IAC Driver Bus 1, Synth B and null: the first port, your choice, the preference list, and nothing live.

Related

  • reselectOutput() Decides what to send to after the port list changed or the current port died, and returns the one line to print about it.
  • isLive() Says whether a MIDI port can still be sent to: it exists and its state is not 'disconnected'.
  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.

import { pickOutput } from '@openav/midi'; packages/midi/ports.js, line 18

pickPort()#

function@openav/midi
pickPort(ports, profile)

Of several ports one device exposes ("Minilab3 MIDI", "Minilab3 DIN THRU", "Minilab3 MCU/HUI"), picks the one to listen to.

A port scores 1 when its name matches one of the profile's match.ports (otherwise it is out), plus 1 for a prefer match and minus 2 for an avoid match. Only positive scores count and the highest wins. It is the second half of device detection after matchProfile().

Parameters

NameTypeDescription
portsObject[]Candidate ports: objects with a name.
profileObjectThe device's profile (its match.ports, prefer and avoid).

Returns

Object | null The best port, or null when none of them fits the profile.

Examples

js
import { pickPort, profileById } from '@openav/midi';

const ports = [{ name: 'Minilab3 DIN THRU' }, { name: 'Minilab3 MIDI' }, { name: 'Minilab3 MCU/HUI' }];
console.log(pickPort(ports, profileById('minilab3')).name);   // Minilab3 MIDI
console.log(pickPort([{ name: 'Other' }], profileById('minilab3')));   // null
module Prints Minilab3 MIDI, then null when no port belongs to the device.

Related

  • matchProfile() Finds the device profile that owns a MIDI port name, such as 'Minilab3 MIDI', by testing each profile's match patterns.
  • MidiControllers Ties hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.

import { pickPort } from '@openav/midi'; packages/midi/profiles.js, line 180

profileById()#

function@openav/midi
profileById(id)

Looks up a shipped controller profile by its id ('akai-lpd8') or its short signal name ('lpd8'), or returns null.

Only profiles from PROFILES are searched. A profile you loaded from a URL is not in there: use resolveProfile(), which also accepts ids. It does not normalize or validate: pass the result to normalizeProfile() when you need the filled-in form.

Parameters

NameTypeDescription
idStringA profile id or its short name (the part in midi/<short>/…).

Returns

Object | null The raw profile from PROFILES, the same object each time, or null.

Examples

js
import { profileById } from '@openav/midi';

const lpd8 = profileById('lpd8');                    // short name
console.log(lpd8.id, '|', lpd8.name, '|', lpd8.controls.length, 'controls');
console.log(profileById('akai-lpd8') === lpd8);      // true: the same profile by its id
console.log(profileById('no-such-box'));             // null
module Prints the LPD8's id, name and 16 controls, then true, then null.

Related

  • PROFILES Every controller profile that ships with the package, as an array of plain JSON objects: real devices first, then the generic layouts.
  • resolveProfile() Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.
  • normalizeProfile() Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
  • findProfile() Finds a profile in a list by id or short name, or returns null; no validation, no loading.

import { profileById } from '@openav/midi'; packages/midi/profiles/index.js, line 41

PROFILES#

constant@openav/midi
PROFILES = array, 27 items

Every controller profile that ships with the package, as an array of plain JSON objects: real devices first, then the generic layouts.

A profile describes one controller as data: what it looks like (the faceplate sections), every control on it (type and the exact MIDI message it sends), LED feedback where the device accepts it, how to recognise its port name, and the document each number came from. Each one is a JSON file in packages/midi/profiles/; the full field list is in the package README and the controllers page shows them all.

A device profile cites its sources and is matched to ports by name, so plugging the device in picks it automatically. A generic layout (generic-8k8p, generic-keys49 …) never auto-matches: you assign a device to it and it learns the controls in the order you move them.

Pick one with profileById() (by id or by the short name), and hand it to createController(), MidiController or MidiControllers. Do not edit these objects in place: copy first (structuredClone), as withChannel() does.

Type
Object[]

The order is the order a device picker shows. Each item is a raw profile (see normalizeProfile() for the filled-in form): id, short, name, kind ('device' or 'generic'), face, sections, controls and, for a device, match and sources.

Notes

  • A control marked verified: false (with a note saying why) has a message that comes from the manual or a guess and has not been confirmed on a real unit. The faceplate shows a small orange ? on its label, and Learn rebinds it in one move.

Examples

js
import { PROFILES } from '@openav/midi';

for (const p of PROFILES.slice(0, 4)) {
  console.log(p.id, '|', p.short, '|', p.kind, '|', p.controls.length, 'controls');
}
console.log(PROFILES.filter((p) => p.kind === 'generic').map((p) => p.id));
module Prints the first four devices (id, short name, kind, control count), then the ids of the generic layouts.

Related

  • profileById() Looks up a shipped controller profile by its id ('akai-lpd8') or its short signal name ('lpd8'), or returns null.
  • normalizeProfile() Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
  • validateProfile() Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
  • matchProfile() Finds the device profile that owns a MIDI port name, such as 'Minilab3 MIDI', by testing each profile's match patterns.

import { PROFILES } from '@openav/midi'; Also from @openav/midi/element packages/midi/profiles/index.js, line 35

profileSignals()#

function@openav/midi
profileSignals(p)

Lists every signal name a profile can publish under its own namespace (midi/lpd8/… for the LPD8), for docs, tests and mapping screens.

For each control: midi/<short>/<id> and /raw; an encoder adds /delta; a pad adds /hit (and /pressure when it has aftertouch). A keys control has only /on, /off and one midi/<short>/n<note> per key. These are the names a MidiController publishes, so a route from controllerRoutes() or a signals.on() can use them.

Parameters

NameTypeDescription
pObjectA profile (its short and controls are read).

Returns

string[] Signal names, starting with midi/<short>/last.

Examples

js
import { profileSignals, profileById } from '@openav/midi';

const names = profileSignals(profileById('lpd8'));
console.log(names.length, 'signals');
console.log(names.slice(0, 5));
module Prints how many signals the LPD8 can publish and the first five: midi/lpd8/last, then pad 1's value, raw value and hit, then pad 2.

Related

  • controllerRoutes() Builds mapper routes for "this control drives that param": controllerRoutes(profile, { knob1: 'hue' }) routes the knob's signal to the param hue.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
  • groupsOf() Groups a profile's controls by their group (knob, pad, fader …), returning { group: [control ids] }.

import { profileSignals } from '@openav/midi'; packages/midi/profiles.js, line 193

publish()#

function@openav/midi
publish(signals, list)

Publishes a list of { name, value, pulse?, min?, max? } into a Signals registry, declaring each signal the first time it is seen.

For a new continuous signal it defines the range from min and max (0 and 1 when missing) and source: 'midi'. A pulse is defined as kind: 'pulse'. Then it calls signals.pulse(name, value) or signals.set(name, value).

The Midi engine and MidiController already do this for you. Call it yourself when you read MIDI some other way, for example from a WebSocket, and want the same names in your show.

Parameters

NameTypeDescription
signalsSignalsThe registry from @openav/core. null makes the call do nothing.
listObject[]Usually the result of genericSignals(). pulse: true fires a pulse, anything else sets a value.

Examples

js
import { Signals } from '@openav/core';
import { parseMessage, genericSignals, publish } from '@openav/midi';

const signals = new Signals();
signals.on('midi/cc/74', (v) => console.log('cc 74 is now', v.toFixed(2)));

// bytes from anywhere: a WebSocket, a file, a test
publish(signals, genericSignals(parseMessage([0xb0, 74, 100])));
console.log(signals.get('midi/ch/1/cc/74'));
module Prints cc 74 is now 0.79, then the same value read from the per-channel name.

Related

  • genericSignals() Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
  • parseMessage() Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.
  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.

import { publish } from '@openav/midi'; packages/midi/parse.js, line 186

RELATIVE_MODES#

constant@openav/midi
RELATIVE_MODES = array, 4 items

The four ways an endless encoder can write a turn as a CC value: offset64, offset16, twos and signbit.

Use it to check or list the encodings, for example when you build a profile editor. validateProfile() reports an encoder whose relative is not in this list.

Type
string[]
Range
['offset64', 'offset16', 'twos', 'signbit']

A plain array. relativeDelta() and relativeValue() take one of these as mode, and a profile's relative field must be one of them (or false for an absolute encoder).

Examples

js
import { RELATIVE_MODES } from '@openav/midi';

console.log(RELATIVE_MODES);
module Prints the four mode names.

Related

  • relativeDelta() Turns the CC value an endless encoder sends into signed steps: with offset64, 65 is +1, 63 is −1 and 64 is no move.
  • relativeValue() The inverse of relativeDelta(): the CC value that says "turned by this many steps" in a given encoder encoding.
  • validateProfile() Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.

import { RELATIVE_MODES } from '@openav/midi'; packages/midi/parse.js, line 89

relativeDelta()#

function@openav/midi
relativeDelta(value, mode = 'offset64')

Turns the CC value an endless encoder sends into signed steps: with offset64, 65 is +1, 63 is −1 and 64 is no move.

An endless encoder has no position, so it sends how far it moved, and devices disagree on how to write it. offset64: 64 is no move, 65 is +1, 63 is −1. offset16: 16 is no move, 17 is +1, 15 is −1. twos: 1..63 are +1..+63 and 127..65 are −1..−63 (two's complement). signbit: 1..63 are positive, 65..127 are −1..−63 (sign and magnitude).

An unknown mode throws unknown relative mode "x". MidiController uses this for every encoder control whose profile says relative, and publishes the result as midi/<short>/<id>/delta.

Parameters

NameTypeDescription
valueNumberThe CC value, 0..127 (only the low 7 bits are read).
modeStringHow the device writes a turn: one of RELATIVE_MODES. A profile names it in the control's relative field. optional default 'offset64'

Returns

Number The signed step count, positive for clockwise.

Examples

js
import { relativeDelta } from '@openav/midi';

console.log(relativeDelta(65));            // 1  (offset64, the default)
console.log(relativeDelta(63));            // -1
console.log(relativeDelta(127, 'twos'));   // -1
console.log(relativeDelta(67, 'signbit')); // -3
module Prints 1, −1, −1 and −3: the same kind of turn written four ways.

Related

  • relativeValue() The inverse of relativeDelta(): the CC value that says "turned by this many steps" in a given encoder encoding.
  • RELATIVE_MODES The four ways an endless encoder can write a turn as a CC value: offset64, offset16, twos and signbit.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.

import { relativeDelta } from '@openav/midi'; packages/midi/parse.js, line 68

relativeValue()#

function@openav/midi
relativeValue(delta, mode = 'offset64')

The inverse of relativeDelta(): the CC value that says "turned by this many steps" in a given encoder encoding.

A virtual encoder sends the same encoding a hardware one does, so the on-screen controller and the real device look identical to everything downstream. relativeValue(-1) is 63 and relativeValue(3, 'twos') is 3. With offset16 the result is held at 0, so it can only go down to −16.

An unknown mode throws unknown relative mode "x".

Parameters

NameTypeDescription
deltaNumberSigned steps. Rounded and held to −63..63.
modeStringOne of RELATIVE_MODES. optional default 'offset64'

Returns

Number The CC value, 0..127.

Examples

js
import { relativeValue, relativeDelta, RELATIVE_MODES } from '@openav/midi';

for (const mode of RELATIVE_MODES) {
  const cc = relativeValue(-3, mode);
  console.log(mode, 'sends', cc, 'which reads back as', relativeDelta(cc, mode));
}
module Four lines: turning −3 steps is sent as 61, 13, 125 and 67, and each reads back as −3.

Related

  • relativeDelta() Turns the CC value an endless encoder sends into signed steps: with offset64, 65 is +1, 63 is −1 and 64 is no move.
  • RELATIVE_MODES The four ways an endless encoder can write a turn as a CC value: offset64, offset16, twos and signbit.

import { relativeValue } from '@openav/midi'; packages/midi/parse.js, line 79

reselectOutput()#

function@openav/midi
reselectOutput({ current, outputs, preferred = '', prefer = [] })

Decides what to send to after the port list changed or the current port died, and returns the one line to print about it.

It switches only when the current port is gone or disconnected (to the best live one, or to none), or when the port you named is back while a stand-in is in use (back to it). In every other case it stays: plugging in an unrelated device never steals the output. The text is plain ASCII, such as MIDI out: switched to "Synth" ("IAC Driver Bus 1" was unplugged).

This is the rule behind the Midi engine's output hot-plug. It never touches Web MIDI, so a test can call it with plain objects.

Parameters

NameTypeDescription
currentMIDIOutput | nullThe port in use now. optional
outputsMIDIOutput[]The ports there are now. optional
preferredStringThe port you chose by name (the one to go back to). optional default ''
preferArray<String | RegExp>Fallback order, as in pickOutput(). optional default []

Returns

{ next, changed, reason, text } next is the port to use. changed says whether it differs from current. reason is same, found, fallback, back or none. text is the line to log (an empty string when nothing changed).

Examples

js
import { reselectOutput } from '@openav/midi';

const iac = { name: 'IAC Driver Bus 1', state: 'connected' };
const synth = { name: 'Synth', state: 'connected' };

// the IAC bus is unplugged: only the synth is left
const r = reselectOutput({ current: iac, outputs: [synth], preferred: 'IAC Driver Bus 1' });
console.log(r.reason, r.changed, r.next.name);
console.log(r.text);

// the IAC bus comes back while the synth stands in: go back to your choice
const back = reselectOutput({ current: synth, outputs: [synth, iac], preferred: 'IAC Driver Bus 1' });
console.log(back.reason, back.next.name);
module Prints fallback true Synth and its sentence, then back IAC Driver Bus 1.

Related

  • pickOutput() Chooses which MIDI output to send to from a list: the port you named, else the first match of a preference list, else the first live port.
  • isLive() Says whether a MIDI port can still be sent to: it exists and its state is not 'disconnected'.
  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.

import { reselectOutput } from '@openav/midi'; packages/midi/ports.js, line 40

RESERVED_IDS#

constant@openav/midi
RESERVED_IDS = array, 7 items

Control ids a profile may not use, because they are signal suffixes: last, raw, delta, hit, on, off, pressure.

A profile's short has a separate reserved list (cc, note, bend, ch, virtual, raw, clock) because those are the generic midi/… families. Both are checked by validateProfile().

Type
string[]
Range
['last', 'raw', 'delta', 'hit', 'on', 'off', 'pressure']

A plain array. last is the signal midi/<short>/last, and the others are the suffixes a control's signals use (/raw, /delta, /hit, /on, /off, /pressure), so validateProfile() rejects them as control ids.

Examples

js
import { RESERVED_IDS, validateProfile, profileById } from '@openav/midi';

console.log(RESERVED_IDS);
const bad = structuredClone(profileById('lpd8'));
bad.controls[0].id = 'hit';
console.log(validateProfile(bad).filter((p) => p.includes('reserved')));
module Prints the list, then the problem raised when a control is renamed to hit.

Related

  • validateProfile() Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.
  • CONTROL_TYPES The kinds of control a profile may contain: knob, encoder, fader, pad, button, keys, wheel, strip.

import { RESERVED_IDS } from '@openav/midi'; packages/midi/profiles.js, line 30

resolveProfile()#

function@openav/midi
async resolveProfile(ref, { profiles = [], fetch = globalThis.fetch, base = undefined } = {})

Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.

Errors are written for a person: no MIDI profile given, unknown MIDI profile "x" (known: …), could not load MIDI profile <url> (404) and invalid MIDI profile (<where>): followed by the list from validateProfile().

Parameters

NameTypeDescription
refString | ObjectA profile object (checked and returned), an id or short name (looked up in profiles), or a URL (fetched). A URL is a string starting with http:, https:, blob:, data:, /, ./ or ../, or ending in .json.
profilesObject[]Where ids are looked up. The default is an empty list, so pass PROFILES (or use loadProfile(), which does). optional default []
fetchFunctionReplaces fetch, for tests or special loading. It must return { ok, status?, json() }. optional default globalThis.fetch
baseStringA base URL that a relative path is resolved against. optional default undefined

Returns

Promise<Object> The raw profile (not normalized).

Examples

js
import { resolveProfile, PROFILES, profileById } from '@openav/midi';

console.log((await resolveProfile('nano2', { profiles: PROFILES })).id);   // korg-nanokontrol2
console.log((await resolveProfile(profileById('lpd8'))).id);                 // an object is checked and returned

try { await resolveProfile({ id: 'Bad' }); } catch (e) { console.log(e.message.split('\n').slice(0, 3).join(' | ')); }
module Prints korg-nanokontrol2, akai-lpd8, then the error header and the first two problems of an invalid object.

Related

  • loadProfile() Loads a controller profile by id, short name or URL (a JSON file anywhere CORS allows), checking any file or object you give it; returns a promise.
  • findProfile() Finds a profile in a list by id or short name, or returns null; no validation, no loading.
  • isProfileUrl() Tells a URL to a profile file apart from a profile id: true for ./pads.json, https://…, /p/x, false for 'akai-lpd8'.
  • validateProfile() Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.

import { resolveProfile } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 111

ScoreMidi#

class@openav/midi
new ScoreMidi({ midi, channel = 15, segment = { note: 36 }, cue = { note: 84 }, length = 0.15, scrub = false, enabled = true, timer = (fn, ms) => setTimeout(fn, ms) } = {})

Turns a show's segment changes and cues into MIDI notes and CCs on one channel, so a DAW or lighting desk can follow the show.

The score knows when a segment starts and when a cue passes. An outside program cannot read the score, but it can read MIDI: launch a clip on a note, jump a scene on a CC. Create a ScoreMidi, then call out.segment({ index, cause }) and out.cue({ name, n }) (createShow({ score: { …, midi: { channel: 15 } } }) does this for you).

Markers go out only while the show plays (cause is play or release). A reset never sends, and seek and jump send only with scrub: true. Everything goes through the engine's own noteOn, noteOff and cc, so the MIDI OUT meter shows it and panic() can silence it. Off unless you create it.

Parameters

NameTypeDescription
midiMidiWhat to send through: a Midi (anything with noteOn, noteOff and cc). Required: without it the constructor throws. optional
channelNumberThe MIDI channel, 1..16 (anything else throws). optional default 15
segmentfalse | Object{ note, cc, velocity }: segment number i sends note note + i and/or CC cc with value i. velocity defaults to 100, note to 36 (give note: null to send only the CC). false sends no segment markers. optional default { note: 36 }
cuefalse | Object{ note, notes, cc, velocity }: every cue sends note (or its own note from the map notes, by cue name) and/or CC cc with the cue's number as value. note defaults to 84 (give note: null to send only the CC). false sends no cue markers. optional default { note: 84 }
lengthNumberSeconds before a marker note's note-off. A marker note is a trigger, not a held note. optional default 0.15
scrubBooleanAlso mark jumps the playhead made by hand (seek, jump). Off by default, so dragging the scrubber does not launch clips in the DAW. optional default false
enabledBooleanSet to false (or later, out.enabled = false) to send nothing. optional default true
timerFunction(fn, ms) => … used to schedule the note-offs. It is setTimeout; pass your own in a test. optional default (fn, ms) => setTimeout(fn, ms)

Members

NameDescription
segmentMessages(evt)methodReturns what a segment change would send, [{ type: 'note' | 'cc', number, value }], without sending it. An empty list when markers are off for this cause.
cueMessages(evt)methodReturns what a cue would send, in the same shape. evt is { name, n }.
segment(evt)methodSends the markers for a segment that began. evt is { index, cause }, cause being play, release, seek, jump or reset. Returns the messages it sent.
cue(evt)methodSends the markers for a cue that was passed. evt is { name, n }: the cue's name and its order in the cut. Returns the messages it sent.

Examples

js
import { ScoreMidi } from '@openav/midi';

// anything with noteOn / noteOff / cc will do; here it just prints
const midi = {
  noteOn: (note, vel, ch) => console.log('note on ', note, vel, 'ch', ch),
  noteOff: (note, ch) => console.log('note off', note, 'ch', ch),
  cc: (cc, val, ch) => console.log('cc', cc, '=', val, 'ch', ch),
};

const out = new ScoreMidi({
  midi, channel: 15,
  segment: { note: 36, cc: 20 },
  cue: { note: 84, notes: { 'dusk.glow': 90 }, cc: 21 },
});

out.segment({ index: 2, cause: 'play' });   // note 38 and CC 20 = 2
out.cue({ name: 'dusk.glow', n: 4 });       // note 90 and CC 21 = 4
out.segment({ index: 1, cause: 'seek' });   // nothing: scrub is off
module Prints note 38 and CC 20 = 2 for segment 2, note 90 and CC 21 = 4 for the cue, nothing for the seek, and the note-offs 0.15 s later.

Related

  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').

import { ScoreMidi } from '@openav/midi'; packages/midi/score-out.js, line 23

unitToBend()#

function@openav/midi
unitToBend(u)

Turns a −1..1 number into a 14-bit pitch-bend value (0..16383), the inverse of bendToUnit().

It does not hold u to −1..1, so clamp it first if it can overshoot. encodeMessage() clamps the final value into 0..16383 anyway, so a pitch-bend message built from an overshoot stays valid.

Parameters

NameTypeDescription
uNumberA position from −1 (fully down) to 1 (fully up). 0 is the centre.

Returns

Number The rounded 14-bit value: unitToBend(0) is 8192, unitToBend(1) is 16383.

Examples

js
import { unitToBend, encodeMessage } from '@openav/midi';

console.log(unitToBend(-1), unitToBend(0), unitToBend(1));   // 0 8192 16383
console.log(encodeMessage({ type: 'pitchbend', ch: 1, value: unitToBend(0.5) }));
module Prints 0 8192 16383, then the three bytes of a half-up pitch bend.

Related

  • bendToUnit() Turns a 14-bit pitch-bend value (0..16383) into −1..1, with the centre (8192) exactly 0.
  • BEND_CENTER The resting value of a pitch-bend wheel: 8192, the middle of the 14-bit range 0..16383.
  • encodeMessage() Turns an event object back into MIDI bytes, the inverse of parseMessage(): { type: 'cc', ch: 1, cc: 74, value: 64 } becomes [0xb0, 74, 64].

import { unitToBend } from '@openav/midi'; packages/midi/parse.js, line 93

validateProfile()#

function@openav/midi
validateProfile(p)

Checks a controller profile and returns every problem as a sentence; an empty array means it is valid.

It reports all problems at once, not just the first. It checks: id and short are lower-case [a-z0-9_-] and short is not a reserved family name (cc, note, bend, ch, virtual, raw, clock), name and kind exist, a device cites at least one source, match regexes compile, face has a size, every control has a unique non-reserved id (see RESERVED_IDS), a known type and msg, numbers in range and no two controls listening to the same message, encoders name their relative mode, and the sections (and portrait, if given) show every control exactly once, inside the faceplate and within their grid.

This is the check behind normalizeProfile() (it throws with this list), resolveProfile() and the test that validates every shipped profile. Run it on your own file before you try it in a page.

Parameters

NameTypeDescription
pObjectA raw profile object (for example parsed from your JSON file).

Returns

string[] The problems, in plain words. [] is valid.

Examples

js
import { validateProfile, profileById } from '@openav/midi';

console.log(validateProfile(profileById('lpd8')));   // [] : valid

const mine = {
  id: 'My Box', short: 'cc', name: 'My box', kind: 'device', face: { w: 10, h: 10 },
  controls: [{ id: 'k1', type: 'knob', msg: 'cc', cc: 200 }], sections: [],
};
for (const problem of validateProfile(mine)) console.log('-', problem);
module Prints [] for a shipped profile, then a list of what is wrong with the made-up one (bad id, reserved short name, no source, CC out of range, control not shown).

Related

  • normalizeProfile() Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.
  • resolveProfile() Turns a profile object, an id or short name, or a URL to a profile JSON into a profile; an object or a file that is not valid is rejected with every problem listed.
  • CONTROL_TYPES The kinds of control a profile may contain: knob, encoder, fader, pad, button, keys, wheel, strip.
  • RESERVED_IDS Control ids a profile may not use, because they are signal suffixes: last, raw, delta, hit, on, off, pressure.

import { validateProfile } from '@openav/midi'; packages/midi/profiles.js, line 38

virtualRequestMIDIAccess()#

function@openav/midi
virtualRequestMIDIAccess({ real = null, make, waitMs = 1200 })

Wraps navigator.requestMIDIAccess so every caller gets the virtual access: real ports join if they answer in time, and the virtual input stays without Web MIDI.

If the real request answers within waitMs (1200 ms by default), its ports are there from the start. If it answers later they hot-plug in as statechange events. If it is refused or missing (Safari, iPad), the virtual input is still there. A permission prompt nobody answers cannot hang the sketch: it gets the access after waitMs anyway.

Parameters

NameTypeDescription
realFunction | nullThe original requestMIDIAccess, already bound (navigator.requestMIDIAccess.bind(navigator)), or null when there is none. The real request still happens, permission prompt and all. optional default null
makeFunction(opts) => … returning the object of createVirtualMIDIAccess(). Required. Return the same one each time to share it. optional
waitMsNumberHow long to wait for the real request before answering anyway. optional default 1200

Returns

Function A function (opts?) => Promise<MIDIAccess> to put where navigator.requestMIDIAccess was.

Examples

js
import { createVirtualMIDIAccess, virtualRequestMIDIAccess } from '@openav/midi';

const virtual = createVirtualMIDIAccess({ name: 'Demo MIDI' });
const requestMIDIAccess = virtualRequestMIDIAccess({ real: null, make: () => virtual });

// this is what an old sketch does, unchanged
const access = await requestMIDIAccess();
console.log([...access.inputs.values()].map((p) => p.name));   // [ 'Demo MIDI' ]
console.log(access === virtual.access, virtual.requests);      // true 1
module Prints ['Demo MIDI'], then true 1: the caller got the virtual access, and it was asked for once.
js
import { createVirtualMIDIAccess, virtualRequestMIDIAccess } from '@openav/midi';

const virtual = createVirtualMIDIAccess();
navigator.requestMIDIAccess = virtualRequestMIDIAccess({
  real: navigator.requestMIDIAccess ? navigator.requestMIDIAccess.bind(navigator) : null,
  make: () => virtual,
});
// from now on every sketch on the page that asks for MIDI gets the virtual input too
static The real wiring: put it before the sketch runs and its old MIDI code needs no change.

Related

  • createVirtualMIDIAccess() Builds a Web MIDI MIDIAccess that always has one more input and output of its own, so a sketch written for a MIDI keyboard works with none plugged in.
  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.

import { virtualRequestMIDIAccess } from '@openav/midi'; packages/midi/virtual-access.js, line 203

withChannel()#

function@openav/midi
withChannel(profile, ch)

Returns a copy of a profile with every control moved to another MIDI channel, for a unit that is set to a channel other than the profile's.

Explicit LED channels (led.ch) stay as they are, since LED feedback may use a different channel from the control. This is what the channel option of createController() and the channel attribute of <oav-controller> call.

Parameters

NameTypeDescription
profileObjectA profile.
chNumberThe channel, 1..16. A falsy value (0, null) returns the same profile object untouched.

Returns

Object A deep copy, validated, with channel and every control's ch set. The original is not changed.

Examples

js
import { profileById } from '@openav/midi';
import { withChannel } from '@openav/midi/embed';

const lpd8 = profileById('lpd8');
const moved = withChannel(lpd8, 3);

console.log(lpd8.controls[0].ch, '->', moved.controls[0].ch);          // 10 -> 3
console.log(moved.controls.every((c) => c.ch === 3), moved.channel);   // true 3
module Prints 10 -> 3: the shipped profile keeps its channel, the copy has every control on channel 3.

Related

  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
  • normalizeProfile() Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.

import { withChannel } from '@openav/midi/embed'; packages/midi/embed.js, line 133

Controller faceplates and embedding#

<oav-controller>, the iframe embed, drawing a controller on a canvas. 39

@openav/midi

CANVAS_THEME#

constant@openav/midi
CANVAS_THEME = object, 16 keys

The default look of ControllerCanvas: background, body, text, idle colours, accent, glow and fonts. Pass any subset as the theme option.

Colours are CSS colour strings. bg: null lets the video or page underneath show through, and accent: '#7ea6ff' makes the panel monochrome-with-one-colour, which suits a branded video.

Type
Object

A frozen object (you cannot change it; pass theme instead). Keys: bg (the area around the device; null leaves what is underneath), body (the faceplate; null uses the profile's face.body), ink (big text), label (control labels), dim (faceplate and section lettering), idle (a pad at rest), button, cap (knob caps), well (fader slots and strip wells), screen, screenInk, screenDim, accent (one colour for everything that moves; null keeps each control's own), glow (0 flat to 1 like the on-screen controller) and the fonts font and mono.

Examples

js
import { CANVAS_THEME, ControllerCanvas } from '@openav/midi';

console.log(CANVAS_THEME.bg, CANVAS_THEME.accent, CANVAS_THEME.glow);

// a transparent, one-colour, flat panel
const panel = new ControllerCanvas({ profile: 'lpd8', theme: { bg: null, accent: '#ffb547', glow: 0 } });
console.log(panel.theme.bg, panel.theme.accent, panel.theme.glow, panel.theme.ink);
module Prints the defaults (#060709 null 1), then the panel's theme with your three keys replaced and everything else (such as ink) kept.

Related

  • ControllerCanvas Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
  • controllerGeometry() Computes where everything goes when a profile's faceplate is drawn into a pixel rectangle: the face, every section, every control cell, and the notes strip.

import { CANVAS_THEME } from '@openav/midi'; packages/midi/canvas.js, line 43

controlDetail()#

function@openav/midi
controlDetail(mc, id, st, info = {})

Builds the control event payload for one control change: id, value, raw, signal, the parsed message, its bytes and where it came from.

Always there: profile (the profile id), id, type, label, value, raw, signal, message, bytes, source and time (performance.now(), rounded). value is 0..1 (−1..1 for a bipolar wheel); a pad is its velocity while held and 0 on release; a button is 0 or 1; for keys it is that note's velocity. raw is the MIDI number (0..127, pitch bend 0..16383). signal is the name the rest of open-audiovisual uses (midi/lpd8/k1; for a key midi/minilab3/n60). message is the parsed MIDI ({ type: 'cc', ch: 1, cc: 70, value: 53 }) and bytes the raw bytes. source is 'ui' (a finger, a mouse, set()), 'hardware' (a real port, or bytes you ingest()) or 'mirror'.

Sometimes there: note and velocity (note messages, velocity 0 for a release), delta (a relative encoder: the signed steps), held (a pad or button), and pressure (a pad, when it is above 0). The control, noteon and noteoff events carry it as their detail.

Parameters

NameTypeDescription
mcMidiControllerThe controller the change happened on.
idStringThe control id.
stObjectIts state after the change (mc.get(id)).
infoObjectWhat onChange gave: { source, ev, note, bytes }. optional default {}

Returns

Object The payload, plain JSON that is safe to postMessage.

Examples

js
import { MidiController, profileById } from '@openav/midi';
import { controlDetail, describeDetail } from '@openav/midi/embed';

const mc = new MidiController(profileById('minilab3'));
mc.onChange((id, st, info) => {
  if (!id) return;
  const detail = controlDetail(mc, id, st, info);
  console.log(describeDetail(detail));
  console.log(detail.bytes, detail.source, detail.signal);
});

mc.setValue('knob1', 0.42);
module Prints the readout line midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1, then the bytes [176, 74, 53], ui and the signal name.

Related

  • describeDetail() Writes the readout line for a control event payload: midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1.
  • describeMessage() Writes one parsed MIDI message the way a person reads it: CC 70 = 53, Note 36 · vel 90, Bend 12288.
  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
  • ControllerHost The object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
  • toParent() Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.

import { controlDetail } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 160

ControllerCanvas#

class@openav/midi
new ControllerCanvas({ profile = 'arturia-minilab3', signals = null, controller = null, mapping = null, theme = {}, notes = 'auto', screen = true, padding = 0.05, now = defaultNow } = {})

Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.

The on-screen controller is DOM: perfect to play, impossible to record, because a page cannot capture its own DOM into a video. This draws the same faceplate, every section where it sits on the device and in the same colours, straight into a 2D context. It is vector and drawn at the output size, so a 4K frame is as sharp as a 1080 one, and it costs a few hundred path calls a frame. @openav/record's Compositor#setPanel takes one.

Call panel.draw(ctx, { x, y, w, h }) every frame, or panel.renderTo(canvas) to fill a canvas of its own. What a video needs that a screen does not: a touch has to stay visible for a few frames (30 fps is 33 ms; a staccato note can be shorter), so released keys and pads fade out instead of switching off, the control a hand just moved glows, and the keyboard follows the device's octave buttons. panel.aspect is the faceplate's width ÷ height, for reserving an area of that shape. With neither signals nor controller the panel just stays at rest.

Parameters

NameTypeDescription
profileString | ObjectProfile id, short name or object. An unknown id throws ControllerCanvas: unknown profile "x". optional default 'arturia-minilab3'
signalsSignals | nullFollow the generic MIDI names on this bus (midi/ch/<ch>/…, midi/note/on|off, midi/cc/<n>, midi/bend): live hardware, the on-screen controller, the QWERTY piano, phones and a recorded take playing back all move the panel. Each message counts once, though hardware publishes it under two names. optional default null
controllerMidiController | nullFollow this existing controller instead of the bus: exactly that device, with its learned mapping. A recorded take does not reach it. When given, signals is ignored. optional default null
mappingObject | nullLearned bindings to apply to the panel's own controller: the exportMapping() of the live device. optional default null
themeObjectAny subset of CANVAS_THEME: colours, glow, fonts. optional default {}
notesBoolean | 'auto'Write the held notes large under the faceplate. 'auto' does it when the area has room, true makes room, false never. optional default 'auto'
screenBooleanThe device's own screen (MiniLab 3, KeyStep) shows the last touch, as the hardware's does: Knob 3 · 87, C4 · 100. optional default true
paddingNumberSpace around the faceplate, as a share of the area's short side. optional default 0.05
nowFunctionThe clock in milliseconds that the fades use; default performance.now(). The now you pass to draw() must be on the same clock. optional default defaultNow

Members

NameDescription
profilegetterThe profile being drawn (normalized).
aspectgetterWidth ÷ height of the faceplate. Layouts use it to reserve an area of this shape (a picture-in-picture corner).
geometry(rect)methodReturns controllerGeometry() for that rect with this panel's padding and notes, cached while the rect stays the same.
reset()methodReleases every held key and pad, as insurance against a stuck key between takes. Nothing is published.
draw(ctx, rect, now = this.now())methodDraws into rect ({ x, y, w, h } in pixels) of a 2D context. Call it every frame. It saves and restores the context. now is on the same clock as the now option, in milliseconds.
renderTo(canvas, now = this.now())methodDraws into a canvas of its own, filling the whole canvas (an overlay, a thumbnail). Returns the canvas.
dispose()methodStops listening to the bus or controller.

Notes

  • It is a silent model: it publishes nothing back to the bus. panel.dispose() stops listening to the bus or controller (a controller you passed in is left as it was).

Examples

js
import { Signals } from '@openav/core';
import { ControllerCanvas, genericSignals, publish } from '@openav/midi';

const signals = new Signals();
const panel = new ControllerCanvas({ profile: 'minilab3', signals });

// a chord, a pad and a knob, published as the hardware would
const play = (ev) => publish(signals, genericSignals(ev));
play({ type: 'noteon', ch: 1, note: 60, vel: 100 });
play({ type: 'noteon', ch: 1, note: 64, vel: 100 });
play({ type: 'noteon', ch: 10, note: 36, vel: 127 });
play({ type: 'cc', ch: 1, cc: 74, value: 100 });

panel.renderTo(document.getElementById('c'));   // fills the 400 x 300 canvas
module The Arturia MiniLab 3 is drawn into the canvas with C4 and E4 lit, pad 1 lit, and knob 1 turned up. The device screen reads Knob 1 · 100.
js
import { Signals } from '@openav/core';
import { ControllerCanvas, genericSignals, publish } from '@openav/midi';

const signals = new Signals();
const panel = new ControllerCanvas({ profile: 'lpd8', signals, theme: { accent: '#7ea6ff' } });
const ctx = document.getElementById('c').getContext('2d');
const send = (ev) => publish(signals, genericSignals(ev));

let frame = 0;
function tick() {
  const pad = 36 + Math.floor(frame / 20) % 8;
  if (frame % 20 === 0) send({ type: 'noteon', ch: 10, note: pad, vel: 110 });   // hit a pad
  if (frame % 20 === 8) send({ type: 'noteoff', ch: 10, note: pad });             // let go: it fades out
  panel.draw(ctx, { x: 0, y: 0, w: 400, h: 300 });
  if (++frame < 120) requestAnimationFrame(tick);
}
tick();
module A different pad of the LPD8 lights every third of a second and then fades, all in one accent colour, as it would in a recorded video.

Related

  • controllerGeometry() Computes where everything goes when a profile's faceplate is drawn into a pixel rectangle: the face, every section, every control cell, and the notes strip.
  • CANVAS_THEME The default look of ControllerCanvas: background, body, text, idle colours, accent, glow and fonts. Pass any subset as the theme option.
  • followMidiSignals() Listens to a Signals bus and hands you each MIDI message back exactly once, as an event, although hardware publishes it under two names.
  • ControllerView The playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
  • eventOfSignal() The inverse of genericSignals(): turns one generic MIDI signal (midi/ch/10/note/36, midi/cc/74) back into the MIDI event it came from.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.

import { ControllerCanvas } from '@openav/midi'; packages/midi/canvas.js, line 257

controllerGeometry()#

function@openav/midi
controllerGeometry(profile, rect, { padding = 0.05, notes = 'auto' } = {})

Computes where everything goes when a profile's faceplate is drawn into a pixel rectangle: the face, every section, every control cell, and the notes strip.

The faceplate keeps its own proportions, centred in the rect. The sections are laid out the way the on-screen controller's CSS grid does (row or column flow, spans), so the canvas drawing and the DOM agree. It is pure, so it is the part to test, and the part to use when you want to draw something of your own on top of a control.

Parameters

NameTypeDescription
profileObjectA normalized profile (MidiController#profile).
rect{ x, y, w, h }The area, in pixels.
paddingNumberSpace around the faceplate, as a share of the rect's short side (0..1). optional default 0.05
notesBoolean | 'auto'Reserve a strip under the faceplate for the held notes: true always, false never, 'auto' when the rect has room. optional default 'auto'

Returns

{ u, face, notes, sections, controls, deco } u is the pixel size of one face unit. face is { x, y, w, h }. notes is the strip or null. sections is a list with id, label, the box and its grid. controls is a Map from control id to { id, type, section, x, y, w, h }. deco lists the decorations (screen, logo, lettering) in pixels.

Examples

js
import { controllerGeometry, normalizeProfile, profileById } from '@openav/midi';

const profile = normalizeProfile(profileById('lpd8'));
const g = controllerGeometry(profile, { x: 0, y: 0, w: 400, h: 300 });

console.log(g.face);                    // the faceplate, centred, in pixels
console.log(g.controls.get('pad1'));    // where pad 1 is
console.log(g.controls.size, 'controls', g.notes ? 'with a notes strip' : 'without a notes strip');
module Prints the faceplate rectangle (370 wide, centred), pad 1's cell in pixels, then 16 controls with a notes strip.

Related

  • ControllerCanvas Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
  • keyboardGeometry() Computes the rectangles of piano keys from one note to another inside a rect: white keys side by side, black keys over the seams at 60% of the height.
  • normalizeProfile() Validates a profile and returns a filled-in copy with defaults for channel, labels, groups, encoder mode and portrait layout; the input is not changed.

import { controllerGeometry } from '@openav/midi'; packages/midi/canvas.js, line 81

ControllerHost#

class@openav/midi
new ControllerHost(profile = 'arturia-minilab3', { profiles = PROFILES, signals = null, sink = null, channel = null, follow = false, midi = null, requestAccess = null, filterOut = 'IAC', storage = undefined, telemetry = true, embedKind = 'headless' } = {})

The object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.

Events, with ctl.on(name, fn) (returns an unsubscribe function): control (every knob, pad, key or button change), noteon and noteoff (the same payload, for pads, keys and note buttons), connect and disconnect ({ port, profile }: a hardware port started or stopped driving it), status, profilechange ({ profile, name }) and '*', which gets (name, detail) for all of them. The control payload is the one described under controlDetail(). ready and error exist only on the <oav-controller> element.

ctl.status is { hardware, input, output, outputState }. hardware is off, asking, waiting (listening, but the device is not plugged in), connected, unavailable (the browser has no Web MIDI) or denied (permission refused). outputState is off, asking, open, missing (no port matches), unavailable or denied. ctl.profileId, ctl.controller and ctl.profile say which device is on screen; ctl.controllers, ctl.view, ctl.midi and ctl.output are the objects underneath.

What a hand plays on the screen leaves through the output port as real MIDI; moves of your real hardware are not forwarded (your software already hears the device), and input with the same name as the chosen output (the loopMIDI echo) is muted.

Before you call it

connectHardware(), setOutput() and requestMidi() are the only calls that ask the browser for MIDI. Chrome, Edge and Firefox show a permission prompt, so call them from a click when you can. Without Web MIDI (Safari, iPad) the controller still plays and fires events, and status says unavailable.

Parameters

NameTypeDescription
profileString | ObjectProfile id, short name or object. optional default 'arturia-minilab3'
profilesObject[]The profiles a picker or follow may switch to. Default: every shipped one. optional default PROFILES
signalsSignals | nullAlso publish midi/<short>/… into a show. optional default null
sinkFunction | nullPublish through this function instead (a relay), as in MidiController. optional default null
channelNumber | nullMove every control to this MIDI channel (your unit is set to another one). See withChannel(). optional default null
followBooleanWhen a different known device is plugged in, show that one. optional default false
midiMidi | nullUse this engine (a show's) instead of the page-shared one. optional default null
requestAccessFunction | nullWhere the MIDIAccess comes from, as in Midi. Giving it makes the controller use its own engine. optional default null
filterOutString | RegExpThe input-port filter of Midi. Anything but the default 'IAC' also gives the controller its own engine. optional default 'IAC'
storageStorage | nullWhere learned mappings are kept, as in MidiControllers. Default: localStorage when available. optional default undefined
telemetryBooleanfalse sends no anonymous page-load hit. optional default true
embedKindStringWhich kind of embed this is for the hit: 'headless' (default), 'element' or 'iframe'. optional default 'headless'

Members

NameDescription
controllergetterThe live model (MidiController) of the device on screen.
profilegetterIts normalized profile: controls, faceplate, sources.
on(name, fn)methodSubscribes to control, noteon, noteoff, connect, disconnect, status, profilechange or '*'. fn(detail) (for '*', fn(name, detail)). Returns an unsubscribe function.
set(id, value, { note = null, quiet = false } = {})methodMoves a control like a hand: knobs and faders 0..1, bipolar wheels −1..1, pads and buttons above 0 press (with that velocity) and 0 release, keys set('keys', velocity, { note: 60 }). With quiet: true only the faceplate moves: no event and no MIDI out (two-way binding). Returns the bytes sent, or null when nothing changed. See setControl().
press(id, velocity = 1, note = null)methodPresses a pad, button or key.
release(id, note = null)methodLets go of it.
get(id)methodThe current value of one control (0..1, −1..1 for bipolar), or null for an unknown id.
values()method{ id: value } for every control (keys: 1 while any key is held). See valuesOf().
reset({ quiet = false } = {})methodLets go of everything and brings every control home (knobs and faders to 0, bipolar wheels to the centre), sending what a hand would. With quiet: true it only moves the faceplate.
releaseAll()methodLets go of every held pad and key, for blur or page hide, so the DAW gets its note-offs.
ingest(bytes, { source = 'hardware' } = {})methodMIDI arriving from anywhere (a WebSocket, another page, a port you manage): the faceplate follows. One message ([0xb0, 74, 64]) or a list of them. source is 'hardware' (default), 'ui' or 'quiet'. Returns the parsed event, or a list of them.
send(bytes)methodSends raw bytes out of the MIDI output port, if one is open. Returns true when sent, false otherwise.
mount(el, { layout = 'auto' } = {})methodDraws the playable faceplate into el; it fills el and follows its size, so give el a height. Returns the ControllerView. layout is auto, face or stack.
setLayout(layout)methodChanges the layout of the mounted view: auto, face or stack.
learn(on = !this.learning)methodLearn mode: tap a control on the screen, then move the one on your hardware (or ingest() it). Toggles by default; learn(true) and learn(false) set it. Returns the new state.
setProfile(ref)methodShows another device: an id, short name, URL or profile object. Held notes are released first. Resolves with the new controller and fires profilechange. Rejects with the reason when the profile is unknown or invalid.
connectHardware(port = null)methodFollows the real device: its port drives the on-screen one (asks the browser for Web MIDI). port is a name or /regex/ for routing a renamed or unknown device by hand. Resolves true when MIDI is available, false otherwise; check status.hardware for connected or waiting.
disconnectHardware()methodStops following the hardware (status.hardware becomes off). The page-shared MIDI engine is let go when nothing else on the page needs it.
requestMidi()methodAsks for Web MIDI without connecting anything, because an output picker needs the port list. Resolves with the output names.
outputs()methodThe output port names, once Web MIDI is on; an empty list before that.
setOutput(query)methodSends what is played here to a MIDI output: a name, a substring ('IAC', 'loopMIDI') or a /regex/, matched as in findPort(). null turns it off. Resolves with the port name, or null when there is none (status.outputState says why).
dispose()methodRemoves everything this controller added: listeners, the view, its share of the MIDI engine. Held notes are released first.

Examples

js
import { ControllerHost } from '@openav/midi';

// same as createController('lpd8'); the class is exported for instanceof and subclassing
const ctl = new ControllerHost('lpd8', { channel: 3 });
console.log(ctl instanceof ControllerHost, ctl.profileId, ctl.status.hardware);

console.log(ctl.set('k1', 1));                      // [178, 70, 127]: channel 3 is status byte 0xb2
await ctl.setProfile('nano2');                       // show another device
console.log(ctl.profileId, ctl.profile.controls.every((c) => c.ch === 3));
module Prints true akai-lpd8 off, then the bytes of knob 1 on channel 3, then the new device with the channel override kept.

Related

  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
  • ControllerView The playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
  • controlDetail() Builds the control event payload for one control change: id, value, raw, signal, the parsed message, its bytes and where it came from.

import { ControllerHost } from '@openav/midi'; packages/midi/index.js, line 277

ControllerView#

class@openav/midi
new ControllerView(container, controller, { mode = 'auto', minTouch = 38 } = {})

The playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.

Two shapes, chosen from the box it lives in. face: the device as it looks, sections where they sit on the real faceplate, scaled to fit (desktop, iPad, a phone held sideways). stack: the same controls re-gridded top to bottom for a tall box (a phone held upright), as the profile's portrait says. Nothing scrolls and nothing shrinks below a thumb. Square grid devices (Launchpad) stay face. mode fixes one; 'auto' follows the box.

Every control repaints from the controller's state, so a finger here and a knob on the desk look the same, and a move from hardware gets a brief glow ring so you can see your hand arrive. Touch: drag a knob (right or up raises it), drag or click a fader or strip, strike a pad (higher on the pad is softer, lower is harder), hold a button, play the keys (multi-touch works). The mouse wheel turns knobs and faders.

Keyboard: Tab reaches every control except the piano keys; the arrow keys, Page Up/Down, Home and End move knobs, faders and strips (Shift for fine steps); Space or Enter strikes pads and buttons, held while the key is held. Learn mode (setLearning(true)) turns every tap into "bind this control", and view.onLearnTap = (id) => … tells you which.

Before you call it

A browser DOM and a container with a height. createController(…).mount(el) is the usual way to get one.

Parameters

NameTypeDescription
containerHTMLElementThe element to fill. It needs a size: the view is 100% wide and tall, so give it a height. It can be inside a shadow root: the view's CSS goes into that root, not the page.
controllerMidiControllerThe model to show and drive.
modeString'auto', 'face' or 'stack' (below). optional default 'auto'
minTouchNumberPixels a control needs before face gives way to stack. optional default 38

Members

NameDescription
setController(c)methodSwitches the view to another MidiController and rebuilds it.
setLearning(on)methodTurns learn mode on or off: dashed outlines on every control, an orange outline on the armed one, orange labels on those already learned.
setMode(mode = 'auto')method'auto' (choose from the box), 'face' or 'stack'. Anything else means 'auto'.
dispose()methodStops listening, disconnects the resize observer and removes the view from the page.

Examples

js
import { ControllerView, MidiController, profileById } from '@openav/midi';

const app = document.getElementById('app');
app.style.height = '240px';

const lpd8 = new MidiController(profileById('lpd8'));
const view = new ControllerView(app, lpd8);   // fills #app

lpd8.press('pad1');                            // pad 1 lights up
setTimeout(() => lpd8.release('pad1'), 600);
module The AKAI LPD8 appears in the page and pad 1 lights for a moment. Try dragging a knob or tapping the pads.
js
import { ControllerView, MidiController, profileById } from '@openav/midi';

const app = document.getElementById('app');
app.style.height = '360px';
app.style.width = '220px';                    // a tall, narrow box

const view = new ControllerView(app, new MidiController(profileById('lpd8')));
console.log(view.mode);                        // 'stack': re-gridded for a tall box
view.setMode('face');
console.log(view.mode);                        // 'face': the device as it looks
module Prints stack, then face after forcing the layout: the same controls in the two shapes.

Related

  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
  • ControllerCanvas Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
  • mountMidiPanel() Mounts a controller panel: a toolbar (device picker, hardware status, port picker, Learn, last message) above a playable faceplate, docked at the bottom or full size.
  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
  • VIEW_CSS The stylesheet text of the on-screen controller (ControllerView), as a string.

import { ControllerView } from '@openav/midi'; packages/midi/view.js, line 177

createController()#

function@openav/midi
createController(profile = 'arturia-minilab3', o = {})

Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').

Four lines are enough: const ctl = createController('akai-lpd8'), ctl.mount(el) for the playable faceplate (optional, it works headless too), ctl.on('control', fn) to hear every knob, pad and key whether it came from the screen or the hardware, and ctl.connectHardware() to follow the real device. ctl.set(id, value) moves a control the way a hand would.

Controllers on one page share one Midi engine, so the browser asks for MIDI permission once. Nothing asks until you call connectHardware(), setOutput() or requestMidi(). Pass signals to publish midi/<short>/… into a show.

Parameters

NameTypeDescription
profileString | ObjectA profile id ('akai-lpd8'), a short name ('lpd8') or a profile object. For a URL, load it first with loadProfile(). An unknown id throws unknown MIDI profile "x" (known: …). optional default 'arturia-minilab3'
oObjectOptions, the same as the second argument of ControllerHost: profiles, signals, sink, channel, follow, midi, requestAccess, filterOut, storage, telemetry, embedKind. optional default {}

Returns

ControllerHost The controller. Its methods are listed under ControllerHost.

Notes

  • Like <oav-controller>, it sends one anonymous hit per page load (which kind of embed and which device; nothing anyone plays) to the project's analytics, and nothing from localhost or the LAN. Turn it off with createController(profile, { telemetry: false }). The README's Telemetry section has the details.

Examples

js
import { createController } from '@openav/midi';

const ctl = createController('akai-lpd8');
ctl.on('control', (e) => console.log(e.signal, e.value.toFixed(2), '<-', e.source));

ctl.set('k1', 0.5);        // a hand on knob 1
ctl.set('pad1', 1);        // press pad 1
ctl.set('pad1', 0);        // let go
console.log(ctl.values().k1, ctl.get('nope'));
module Prints three control events (knob 0.50, pad 1.00, pad 0.00, each from 'ui'), then the knob's value and null for an id that does not exist.
js
import { createController } from '@openav/midi';

// MIDI out to a DAW: a stand-in for the browser's Web MIDI shows what would be sent
const iac = { name: 'IAC Driver Bus 1', state: 'connected', send: (bytes) => console.log('to the DAW', bytes) };
const requestAccess = async () => ({ inputs: new Map(), outputs: new Map([['o', iac]]) });

const ctl = createController('akai-lpd8', { requestAccess });
console.log(await ctl.setOutput('IAC'));   // IAC Driver Bus 1
ctl.set('pad1', 1);                        // to the DAW [153, 36, 127]
ctl.set('pad1', 0);                        // to the DAW [137, 36, 0]
ctl.ingest([0x99, 37, 90]);                // hardware-style input: the faceplate follows, nothing is sent out
module Prints the port name, then the two messages a hand on the screen sends. The last line prints nothing: what arrives from hardware is not forwarded.
js
import { Signals } from '@openav/core';
import { createController } from '@openav/midi';

const signals = new Signals();
const ctl = createController('minilab3', { signals });

const app = document.getElementById('app');
app.style.height = '240px';
ctl.mount(app);                            // the playable faceplate fills #app

ctl.set('knob1', 0.8);                     // the knob on screen turns
console.log(signals.get('midi/minilab3/knob1'));
module The MiniLab 3 appears in the page with knob 1 turned up, and the console prints its signal value (0.80…). Play it with the mouse.

Related

  • ControllerHost The object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
  • loadProfile() Loads a controller profile by id, short name or URL (a JSON file anywhere CORS allows), checking any file or object you give it; returns a promise.
  • Midi The Web MIDI engine: asks the browser for access, listens to every input, publishes what arrives as signals, and sends to one output port.
  • ControllerView The playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.

import { createController } from '@openav/midi'; packages/midi/index.js, line 275

defineController()#

function@openav/midi
defineController(tag = 'oav-controller')

Registers the controller element under another tag name (importing @openav/midi/element already registers it as oav-controller).

A tag name can be registered only once per page, so use this when oav-controller clashes with another library, or when you want two differently-named copies. The default tag registers the exported OavController class itself; any other name registers a subclass of it.

Parameters

NameTypeDescription
tagStringThe new tag name. It must contain a hyphen, as every custom element does. optional default 'oav-controller'

Returns

Function | null The element class registered for that tag (an already-registered tag returns what is there). null when the page has no customElements.

Examples

js
import { defineController } from '@openav/midi/element';

defineController('my-pads');                          // now <my-pads> works too

const app = document.getElementById('app');
app.innerHTML = '<my-pads profile="generic-16p"></my-pads>';
const el = app.firstElementChild;
await el.ready;
console.log(el.tagName, el.profileData.id);           // MY-PADS generic-16p
module A 16-pad generic controller appears from a tag you named yourself; the console prints MY-PADS generic-16p.

Related

  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.

import { defineController } from '@openav/midi/element'; packages/midi/element.js, line 352

describeDetail()#

function@openav/midi
describeDetail(d)

Writes the readout line for a control event payload: midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1.

This is the text <oav-controller readout> shows under the faceplate. Use it for your own log or caption. A payload without a message shows only the signal and the value.

Parameters

NameTypeDescription
dObject | nullA payload from controlDetail() (or a control event's detail). null gives an empty string.

Returns

String The signal name, the value (two decimals, or a whole number from 100 up), then ←, the message as describeMessage() writes it and the channel.

Examples

js
import { createController } from '@openav/midi';
import { describeDetail } from '@openav/midi/embed';

const ctl = createController('minilab3');
ctl.on('control', (d) => console.log(describeDetail(d)));

ctl.set('knob1', 0.42);
ctl.set('pitch', 1);
ctl.set('pad1', 1);
module Prints midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1, then the same kind of line for the pitch bend and pad 1.

Related

  • controlDetail() Builds the control event payload for one control change: id, value, raw, signal, the parsed message, its bytes and where it came from.
  • describeMessage() Writes one parsed MIDI message the way a person reads it: CC 70 = 53, Note 36 · vel 90, Bend 12288.
  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.

import { describeDetail } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 199

describeMessage()#

function@openav/midi
describeMessage(m)

Writes one parsed MIDI message the way a person reads it: CC 70 = 53, Note 36 · vel 90, Bend 12288.

These are the shorter lines of the <oav-controller> readout. describe() writes the longer log lines (CC ch1 #70=53).

Parameters

NameTypeDescription
mObject | nullAn event from parseMessage(). null gives an empty string.

Returns

String CC 70 = 53, Note 36 · vel 90, Note 36 off, Bend 12288 (the raw 14-bit value), Pressure 50, Poly AT 36 = 12, Program 5; any other type gives its name.

Examples

js
import { parseMessage } from '@openav/midi';
import { describeMessage } from '@openav/midi/embed';

for (const bytes of [[0xb0, 70, 53], [0x99, 36, 90], [0x89, 36, 0], [0xe0, 0, 96]]) {
  console.log(describeMessage(parseMessage(bytes)));
}
module Prints CC 70 = 53, Note 36 · vel 90, Note 36 off and Bend 12288.

Related

  • describeDetail() Writes the readout line for a control event payload: midi/minilab3/knob1 = 0.42 ← CC 74 = 53 · ch 1.
  • describe() Returns a short readable line for a parsed MIDI event, for logs and learn screens: CC ch1 #74=64, Bend ch1 0.50.
  • parseMessage() Turns the bytes of one MIDI message into an event object such as { type: 'noteon', ch: 1, note: 60, vel: 100 }, or null for bytes it does not know.

import { describeMessage } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 185

EMBED_ATTRS#

constant@openav/midi
EMBED_ATTRS = array, 11 items

The attributes the oav-controller element reads, which are also the query parameters of the iframe page: profile, layout, hardware, midi-out and more.

The iframe page copies every one of these from its URL onto the element. The extra parameters bg, pad, frame, origin and lang belong to the iframe page and are read by parseEmbedOptions(). What each attribute means is under OavController.

Type
string[]
Range
['profile', 'layout', 'hardware', 'midi-out', 'channel', 'learn', 'picker', 'readout', 'follow', 'theme', 'telemetry']

A plain array, in this order. The element watches these (and lang) for changes.

Examples

js
import { EMBED_ATTRS } from '@openav/midi/embed';

console.log(EMBED_ATTRS.join(' '));
module Prints the eleven attribute names.

Related

  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
  • parseEmbedOptions() Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
  • EMBED_EVENTS The nine events the oav-controller element dispatches and the iframe posts to its page, from ready and control to profilechange and error.

import { EMBED_ATTRS } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 22

EMBED_EVENTS#

constant@openav/midi
EMBED_EVENTS = array, 9 items

The nine events the oav-controller element dispatches and the iframe posts to its page, from ready and control to profilechange and error.

The iframe also posts a tenth type, values, as the answer to the get command; it is not an element event, so it is not in this list. The payloads are described under OavController and controlDetail().

Type
string[]
Range
['ready', 'control', 'noteon', 'noteoff', 'connect', 'disconnect', 'status', 'profilechange', 'error']

A plain array, in this order. The element dispatches each as a CustomEvent that bubbles and is composed; the iframe page forwards each to window.parent with toParent().

Examples

js
import { EMBED_EVENTS } from '@openav/midi/embed';

console.log(EMBED_EVENTS.join(' '));
module Prints the nine event names.

Related

  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
  • toParent() Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
  • controlDetail() Builds the control event payload for one control change: id, value, raw, signal, the parsed message, its bytes and where it came from.
  • EMBED_ATTRS The attributes the oav-controller element reads, which are also the query parameters of the iframe page: profile, layout, hardware, midi-out and more.

import { EMBED_EVENTS } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 24

EMBED_LAYOUTS#

constant@openav/midi
EMBED_LAYOUTS = array, 3 items

The three layouts a controller can have: auto, face (the device as it looks) and stack (re-gridded for a tall box).

A profile can opt out of stack (stack: false); square grid devices such as the Launchpad always stay face. See preferredAspect().

Type
string[]
Range
['auto', 'face', 'stack']

A plain array. parseEmbedOptions() turns any other layout value into 'auto', and the layout command of the iframe accepts only these.

Examples

js
import { EMBED_LAYOUTS, parseEmbedOptions } from '@openav/midi/embed';

console.log(EMBED_LAYOUTS);
console.log(parseEmbedOptions({ layout: 'stack' }).layout);      // stack
console.log(parseEmbedOptions({ layout: 'sideways' }).layout);   // auto: not in the list
module Prints the three names, then stack, then auto.

Related

  • parseEmbedOptions() Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
  • preferredAspect() Works out the width ÷ height a controller wants in a box of a given width: its faceplate when every control stays finger-sized, a taller stack otherwise.
  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.

import { EMBED_LAYOUTS } from '@openav/midi/embed'; packages/midi/embed.js, line 25

embedChannel()#

function@openav/midi
embedChannel(v)

Reads the channel attribute: a whole number from 1 to 16, or null for anything else.

A null means "keep the profile's own channels". A number makes withChannel() move every control there.

Parameters

NameTypeDescription
vanyThe attribute value, a number or a string.

Returns

Number | null The channel, or null when it is missing, empty, not a whole number or outside 1..16.

Examples

js
import { embedChannel } from '@openav/midi/embed';

console.log(embedChannel('10'), embedChannel(3));                     // 10 3
console.log(embedChannel('0'), embedChannel('17'), embedChannel('x'), embedChannel(''));   // null null null null
module Prints 10 3, then four nulls.

Related

  • withChannel() Returns a copy of a profile with every control moved to another MIDI channel, for a unit that is set to a channel other than the profile's.
  • parseEmbedOptions() Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.

import { embedChannel } from '@openav/midi/embed'; packages/midi/embed.js, line 49

embedColor()#

function@openav/midi
embedColor(v)

Checks the bg colour of the iframe page and returns a safe value: transparent, a hex colour, or a plain colour name; anything else is null.

The iframe page puts the result into a CSS variable, so a value that could break out of CSS (red;}body{) is refused. It does not check that a name is a real colour: any 3 to 20 letters pass, and CSS ignores the ones it does not know.

Parameters

NameTypeDescription
vStringThe value from the query string.

Returns

String | null 'transparent'; '#' plus 3, 4, 6 or 8 hex digits (the # is optional in the input, so abc becomes #abc); or 3 to 20 letters, lower-cased ('RED' becomes 'red'); otherwise null.

Examples

js
import { embedColor } from '@openav/midi/embed';

console.log(embedColor('transparent'), embedColor('#0b0b0c'), embedColor('000'), embedColor('RED'));
console.log(embedColor('red;}body{'), embedColor('#12345'), embedColor(''));   // null null null
module Prints transparent #0b0b0c #000 red, then three nulls.

Related

  • parseEmbedOptions() Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.

import { embedColor } from '@openav/midi/embed'; packages/midi/embed.js, line 56

embedFlag()#

function@openav/midi
embedFlag(v)

Reads a boolean attribute: present means on, unless it says false, 0, off or no.

This is why <oav-controller readout> and readout="true" both turn the readout on and readout="off" turns it off. Any other word counts as on, so readout="ask" is true.

Parameters

NameTypeDescription
vanyThe attribute value: null/undefined/false (absent) or a string. A bare attribute is the empty string.

Returns

Boolean Absent is false; true is true; a string is true unless it is (case-insensitively, trimmed) false, 0, off or no.

Examples

js
import { embedFlag } from '@openav/midi/embed';

console.log(embedFlag(''), embedFlag('true'), embedFlag('ask'));    // true true true
console.log(embedFlag(null), embedFlag('off'), embedFlag('No'));    // false false false
module Prints three trues, then three falses.

Related

  • parseEmbedOptions() Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
  • embedMode() Reads hardware and midi-out: off, ask (a button, so the permission prompt waits for a click) or on, plus which port.

import { embedFlag } from '@openav/midi/embed'; packages/midi/embed.js, line 30

embedMode()#

function@openav/midi
embedMode(v, { bare = 'on' } = {})

Reads hardware and midi-out: off, ask (a button, so the permission prompt waits for a click) or on, plus which port.

Absent or false/0/off/no is off. ask is ask. on, true, yes, auto is on without a port. Any other text is on, and the text is the port name or /regex/ to look for (see findPort()): embedMode('IAC') is { mode: 'on', port: 'IAC' }.

Parameters

NameTypeDescription
vanyThe attribute value.
bareStringWhat a bare attribute (no value) means: 'on' for hardware (the default), 'ask' for midi-out, because there is no sensible default port to send to. optional default 'on'

Returns

{ mode, port } mode is 'off', 'ask' or 'on'. port is a string only when the value names one.

Examples

js
import { embedMode } from '@openav/midi/embed';

console.log(embedMode(null));                    // { mode: 'off', port: null }
console.log(embedMode(''));                      // { mode: 'on', port: null }  (a bare hardware attribute)
console.log(embedMode('', { bare: 'ask' }));     // { mode: 'ask', port: null } (a bare midi-out attribute)
console.log(embedMode('loopMIDI'));              // { mode: 'on', port: 'loopMIDI' }
module Prints four { mode, port } objects: off, on, ask, and on with a named port.

Related

  • parseEmbedOptions() Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
  • embedFlag() Reads a boolean attribute: present means on, unless it says false, 0, off or no.
  • findPort() Finds a MIDI port by name: a /regex/flags, then the exact name, then a case-insensitive substring; null when nothing matches.

import { embedMode } from '@openav/midi/embed'; packages/midi/embed.js, line 38

fadeLevel()#

function@openav/midi
fadeLevel(now, at, ms)

The brightness of a fading touch: 1 at the moment it happened, 0 after ms, eased so it still reads on a 30 fps frame.

ControllerCanvas fades released keys (180 ms), pads (450 ms), the glow of a touched control (650 ms), the last-touch text under the faceplate (1600 ms) and on the device screen (3200 ms) with it. A video frame is 33 ms at 30 fps, so a staccato note that was 20 ms long would be gone before the next frame without a fade.

Parameters

NameTypeDescription
nowNumberThe current time in milliseconds.
atNumber | nullWhen the touch happened, on the same clock. null or undefined gives 0.
msNumberHow long the fade lasts. Not above 0 gives 0.

Returns

Number From 1 down to 0. The curve is the square of the remaining time, so it falls quickly at first.

Examples

js
import { fadeLevel } from '@openav/midi';

for (const now of [100, 200, 300, 400, 500]) console.log(now, fadeLevel(now, 100, 400));
module Prints 1 at the moment of the touch, then 0.5625, 0.25, 0.0625 and 0 at the end of the fade.

Related

  • ControllerCanvas Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.

import { fadeLevel } from '@openav/midi'; packages/midi/canvas.js, line 64

followMidiSignals()#

function@openav/midi
followMidiSignals(signals, onEvent, { channel = 1, defer = (fn) => queueMicrotask(fn) } = {})

Listens to a Signals bus and hands you each MIDI message back exactly once, as an event, although hardware publishes it under two names.

Hardware and the on-screen controllers publish every message twice: the legacy name (midi/note/on) and then the per-channel one (midi/ch/1/note/60) in one call. The QWERTY piano and some takes publish only the legacy name. A per-channel name counts at once; a legacy name waits for a microtask and is dropped if its per-channel twin shows up first. So a relative encoder never turns twice, a legacy CC (which has no channel) never lands on the wrong channel, and order is kept.

ControllerCanvas uses it. Use it yourself to turn a bus back into MIDI events, for example to drive your own visual from a recorded take.

Parameters

NameTypeDescription
signalsSignalsThe bus. It needs onAny(cb).
onEventFunction(ev) => … called with an event of parseMessage() for each message.
channelNumberThe channel given to legacy CC and bend names, which carry none. optional default 1
deferFunction(fn) => … that schedules the flush of legacy-only names. Default queueMicrotask; replaceable in tests. optional default (fn) => queueMicrotask(fn)

Returns

{ flush(), dispose() } flush() delivers legacy-only messages that are still waiting. dispose() stops listening and drops them.

Examples

js
import { Signals } from '@openav/core';
import { followMidiSignals, genericSignals, publish } from '@openav/midi';

const signals = new Signals();
const follower = followMidiSignals(signals, (ev) => console.log(ev.type, ev.note ?? ev.cc, ev.ch));

// the hardware publishes a message under both names; the follower reports it once
publish(signals, genericSignals({ type: 'noteon', ch: 2, note: 60, vel: 100 }));
publish(signals, genericSignals({ type: 'cc', ch: 2, cc: 74, value: 64 }));

// the QWERTY piano publishes only the short name: it arrives after a microtask, or at flush()
signals.pulse('midi/note/on', { note: 64, vel: 0.8, ch: 0 });
follower.flush();
follower.dispose();
module Prints three lines: noteon 60 2, cc 74 2, then noteon 64 1 for the piano (channel 0 becomes the default channel 1).

Related

  • eventOfSignal() The inverse of genericSignals(): turns one generic MIDI signal (midi/ch/10/note/36, midi/cc/74) back into the MIDI event it came from.
  • genericSignals() Lists the device-independent signals one MIDI event becomes: midi/cc/74, midi/note/on, midi/bend, and their per-channel twins such as midi/ch/2/cc/74.
  • ControllerCanvas Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.

import { followMidiSignals } from '@openav/midi'; packages/midi/canvas.js, line 194

FRAME_COMMANDS#

constant@openav/midi
FRAME_COMMANDS = array, 8 items

The commands a page may send into an embed iframe: set, ingest, reset, profile, layout, get, press, release.

What each does: set { id, value, note?, quiet? } moves a control, press { id, velocity?, note? } and release { id, note? } hold and let go, ingest { bytes } feeds MIDI (one message or a list), reset brings everything home, profile { profile } switches the device, layout { layout } changes the layout, and get asks for every value (the iframe answers with a values message).

Type
string[]
Range
['set', 'ingest', 'reset', 'profile', 'layout', 'get', 'press', 'release']

A plain array. fromParent() ignores a message whose type is not one of these.

Examples

js
import { FRAME_COMMANDS, toFrame } from '@openav/midi/embed';

console.log(FRAME_COMMANDS.join(' '));
console.log(toFrame('set', { id: 'k1', value: 0.5 }));   // the message you post to the iframe
module Prints the eight commands, then the message { target: 'openav', v: 1, type: 'set', id: 'k1', value: 0.5 }.

Related

  • fromParent() Frame side: validates a command from the page and returns a clean command, or null (the message is ignored).
  • toFrame() Page side: builds a command to post into an embed iframe: { target: 'openav', v: 1, type, ...args }.
  • EMBED_EVENTS The nine events the oav-controller element dispatches and the iframe posts to its page, from ready and control to profilechange and error.

import { FRAME_COMMANDS } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 271

fromFrame()#

function@openav/midi
fromFrame(data)

Page side: reads a message that came from an embed iframe, returns { type, frame, detail, v }, or null when it is not one of ours.

Other scripts and extensions post to a page all the time, so always filter. Check event.origin too when the page cares who sent it: this function checks the shape, not the sender.

Parameters

NameTypeDescription
dataanyThe event.data of a message event.

Returns

{ type, frame, detail, v } | null null unless data is an object with source: 'openav' and a string type.

Examples

js
import { fromFrame, toParent } from '@openav/midi/embed';

// what a page does with every 'message' event; here the message is made by hand
const data = toParent('control', { id: 'k1', signal: 'midi/lpd8/k1', value: 1 }, { frame: 'pads' });
const msg = fromFrame(data);
if (msg && msg.type === 'control') console.log(msg.frame, msg.detail.signal, msg.detail.value);

console.log(fromFrame({ type: 'control' }), fromFrame('hello'));   // null null
module Prints pads midi/lpd8/k1 1, then null null for messages that are not ours.
js
window.addEventListener('message', (e) => {
  const msg = fromFrame(e.data);
  if (!msg) return;                                        // not from an openav iframe
  if (msg.frame === 'pads' && msg.type === 'control') {
    console.log(msg.detail.id, msg.detail.value);
  }
});
static The listener to put on the page that holds the iframe.

Related

  • toParent() Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
  • toFrame() Page side: builds a command to post into an embed iframe: { target: 'openav', v: 1, type, ...args }.
  • fromParent() Frame side: validates a command from the page and returns a clean command, or null (the message is ignored).
  • POST_SOURCE The word that marks a message as an open-audiovisual embed message: 'openav', sent as source by the iframe and as target by the page.

import { fromFrame } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 280

fromParent()#

function@openav/midi
fromParent(data)

Frame side: validates a command from the page and returns a clean command, or null (the message is ignored).

It is strict, because anything on the web can post to an iframe. The message needs target: 'openav' and a type from FRAME_COMMANDS. Control ids must be lower-case [a-z0-9_-]. value must be a finite number. note must be an integer 0..127 (else null). A press velocity is held to 0..1. ingest takes one message of 1 to 3 bytes whose first byte is a status byte (128 or more), or up to 256 such messages. profile takes an id or an https:// URL (not http://). layout must be in EMBED_LAYOUTS.

Parameters

NameTypeDescription
dataanyThe event.data of a message event.

Returns

Object | null A command such as { type: 'set', id: 'k1', value: 0.5, note: null, quiet: false }, or null.

Examples

js
import { fromParent, toFrame } from '@openav/midi/embed';

const show = (data) => console.log(JSON.stringify(fromParent(data)));

show(toFrame('set', { id: 'k1', value: '0.5' }));    // value is read as a number
show(toFrame('press', { id: 'pad1', velocity: 2 })); // velocity held to 1
show(toFrame('ingest', { bytes: [0xb0, 70, 64] }));

// all of these are ignored: null
show({ source: 'openav', type: 'set', id: 'k1', value: 1 });    // wrong key: source, not target
show(toFrame('set', { id: '<img>', value: 1 }));                // bad id
show(toFrame('profile', { profile: 'http://x.org/p.json' }));   // http, not https
show(toFrame('eval', { code: 'alert(1)' }));                    // not a command
module Prints three clean commands, then four nulls for messages the frame would ignore.

Related

  • toFrame() Page side: builds a command to post into an embed iframe: { target: 'openav', v: 1, type, ...args }.
  • fromFrame() Page side: reads a message that came from an embed iframe, returns { type, frame, detail, v }, or null when it is not one of ours.
  • toParent() Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
  • FRAME_COMMANDS The commands a page may send into an embed iframe: set, ingest, reset, profile, layout, get, press, release.

import { fromParent } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 291

keyboardGeometry()#

function@openav/midi
keyboardGeometry(from, to, rect)

Computes the rectangles of piano keys from one note to another inside a rect: white keys side by side, black keys over the seams at 60% of the height.

A range never starts on a black key: a black key with no white key before it is left out, the way the on-screen keyboard does. White keys share the width equally, so 48 to 72 is 15 white keys and 10 black ones.

Parameters

NameTypeDescription
fromNumberThe first MIDI note.
toNumberThe last MIDI note.
rect{ x, y, w, h }The area the keys fill.

Returns

{ note, white, x, y, w, h }[] One item per key, the white keys first and then the black ones (the draw order, so the black keys end up on top).

Examples

js
import { keyboardGeometry } from '@openav/midi';

const keys = keyboardGeometry(60, 72, { x: 0, y: 0, w: 400, h: 100 });
console.log(keys.filter((k) => k.white).length, 'white,', keys.filter((k) => !k.white).length, 'black');
console.log(keys[0].note, keys[0].x, keys[0].w);   // 60 (C4) is the first white key
module Prints 8 white, 5 black (one octave and the next C), then the first key's note and its position.

Related

  • ControllerCanvas Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
  • keyWindow() Tells which note a drawn keyboard should start at after a note outside its printed range arrives, so it follows the device's octave buttons.
  • controllerGeometry() Computes where everything goes when a profile's faceplate is drawn into a pixel rectangle: the face, every section, every control cell, and the notes strip.

import { keyboardGeometry } from '@openav/midi'; packages/midi/canvas.js, line 138

keyWindow()#

function@openav/midi
keyWindow(from, to, base, note)

Tells which note a drawn keyboard should start at after a note outside its printed range arrives, so it follows the device's octave buttons.

A keyboard control catches notes outside its printed range, because the device's Oct+/Oct− buttons shift what the keys send. When one arrives, the drawing moves and stays there, as the device does. A 48 to 72 MiniLab with Oct+ pressed (a note of 76) shows 60 to 84. A range narrower than an octave just moves to reach the note.

Parameters

NameTypeDescription
fromNumberThe first note of the printed range.
toNumberThe last note of the printed range.
baseNumberThe first note shown now.
noteNumberA note that just went down.

Returns

Number The new first note: base when the note fits, otherwise base moved by the fewest octaves that reach it, held to the MIDI range.

Examples

js
import { keyWindow } from '@openav/midi';

console.log(keyWindow(48, 72, 48, 60));    // 48: inside the window, unchanged
console.log(keyWindow(48, 72, 48, 76));    // 60: Oct+ on the MiniLab
console.log(keyWindow(48, 72, 48, 40));    // 36: Oct-
console.log(keyWindow(48, 72, 48, 127));   // 103: held to the MIDI range
module Prints 48, 60, 36 and 103.

Related

  • keyboardGeometry() Computes the rectangles of piano keys from one note to another inside a rect: white keys side by side, black keys over the seams at 60% of the height.
  • ControllerCanvas Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.

import { keyWindow } from '@openav/midi'; packages/midi/canvas.js, line 157

midiNoteName()#

function@openav/midi
midiNoteName(n)

Turns a MIDI note number into its name with the octave, middle C being C4: midiNoteName(60) is 'C4', midiNoteName(61) is 'C#4'.

This is the on-screen keyboard's convention. Some software (Ableton Live, Yamaha) calls 60 "C3": the number is the same, only the octave label differs. In a p5 sketch, noteName() does the same and also writes flats when you ask.

Parameters

NameTypeDescription
nNumberA MIDI note number, 0..127.

Returns

String Sharps only ('C#4', never 'Db4'). 0 is 'C-1' and 127 is 'G9'.

Examples

js
import { midiNoteName } from '@openav/midi';

console.log([60, 61, 69, 0, 127].map(midiNoteName));   // [ 'C4', 'C#4', 'A4', 'C-1', 'G9' ]
module Prints the five note names.

Related

  • noteName() Turns a MIDI number into a note name: noteName(61) is 'C#4'.
  • noteNumber() Turns a note name like 'C4' into its MIDI number, 0 to 127.
  • touchText() Writes what a device's screen says about a touch: Knob 1 · 87, C4 · 100, Pad 3 · 96, Pitch +0.42.

import { midiNoteName } from '@openav/midi'; packages/midi/canvas.js, line 40

mountMidiPanel()#

function@openav/midi
mountMidiPanel(parent, mcs, { mode = 'full', open = true, id = null, onToggle = null, closeButton = mode === 'dock', profiles = null, contained = false } = {})

Mounts a controller panel: a toolbar (device picker, hardware status, port picker, Learn, last message) above a playable faceplate, docked at the bottom or full size.

The bar shows which device is on screen (the picker switches it), a chip saying whether a hardware port drives it, a port picker when hardware is attached, a Learn button (click a control, then move the one on the hardware; a second button resets the mapping), and a tiny monitor that prints the last message, so you see the hardware talking before you see what it moves. Plugging in a known device switches the picker and the faceplate.

Parameters

NameTypeDescription
parentHTMLElementThe element to add the panel to. In full mode the panel is position: absolute; inset: 0, so give parent a position (relative) and a size.
mcsMidiControllersThe manager that holds the devices and the hardware.
modeString'full' fills parent. 'dock' is a drawer fixed to the bottom of the window, 40% of the window height by default, with a grip to drag its height (remembered in localStorage). optional default 'full'
openBooleanWhether it starts open. A closed dock builds its view only when opened. optional default true
idString | nullAn element id for the panel. optional default null
onToggleFunction | null(open) => … called when the panel opens or closes. optional default null
closeButtonBooleanShow a ✕ button. The default is on in dock mode only. optional default mode === 'dock'
profilesString[] | nullLimit the device picker to these profile ids. optional default null
containedBooleanDock inside parent (absolute) instead of the window (fixed). optional default false

Returns

{ el, view, isOpen, open(), close(), toggle(), dispose() } el is the panel element, view the ControllerView (once it exists), isOpen a boolean. open(), close() and toggle() return the same object; close() also releases held notes.

Notes

  • The labels on the toolbar (Learn, Reset, the hardware chip) are written in Chinese only, whatever the page language is. <oav-controller> has an English and a Chinese toolbar; this panel is the one the lab and the phone tab use.

Examples

js
import { MidiControllers, PROFILES, mountMidiPanel } from '@openav/midi';

const app = document.getElementById('app');
app.style.cssText = 'position: relative; height: 320px';

const mcs = new MidiControllers({ profiles: PROFILES, initial: 'akai-lpd8' });
const panel = mountMidiPanel(app, mcs, { mode: 'full' });

console.log(panel.isOpen, panel.view.constructor.name);   // true ControllerView
mcs.current.press('pad2');                                  // the pad lights up in the panel
setTimeout(() => mcs.current.release('pad2'), 500);
module A panel with the toolbar and the AKAI LPD8 appears in the page; pad 2 lights for half a second. Switch the device in the picker.

Related

  • MidiControllers Ties hardware ports to controller profiles: plug in a MiniLab 3 and the on-screen one appears and moves with your hands; unplug it and it stays playable.
  • ControllerView The playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
  • PANEL_CSS The stylesheet text of the controller panel (mountMidiPanel), as a string.
  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').

import { mountMidiPanel } from '@openav/midi'; packages/midi/panel.js, line 62

OavController#

class@openav/midi
new OavController()

The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.

Load @openav/midi/element (https://openaudiovisual.com/packages/midi/element.js) once and write <oav-controller profile="akai-lpd8" readout></oav-controller>. Importing the file registers the tag by itself. It is a shadow-DOM element, so the page's CSS cannot bend it and it cannot bend the page. It plays with a mouse, a finger, a pen and the keyboard, with or without Web MIDI: on Safari and iPad it still plays and fires events, and only the hardware and MIDI-out parts say they are unavailable.

Attributes, each also a property of the same name (midi-out is el.midiOut): profile (an id, a short name or a URL to a profile JSON; default arturia-minilab3), layout (auto, face as it looks, stack re-gridded for a tall box), hardware (off by default; bare or on follows the real device, ask shows a button so the permission prompt waits for a click, or a port name or /regex/), midi-out (off by default; bare or ask lets the visitor pick, or a port name, substring or /regex/ such as IAC or loopMIDI), channel (1 to 16: move every control to that channel), picker, readout (the last message under the faceplate), learn (a Learn button), follow (show whatever known device is plugged in), theme (dark or light: the toolbar only, the device stays the device), telemetry (off sends no anonymous hit) and lang (the toolbar language: zh… is Chinese, anything else English; it reads the nearest lang on the page).

Events are CustomEvents that bubble and cross the shadow root, so el.addEventListener('control', e => e.detail) works anywhere above it: ready { profile, name, controls }, control (see controlDetail()), noteon, noteoff, connect, disconnect, status, profilechange and error { message }. EMBED_EVENTS lists them. Methods: set(), get(), values(), reset(), press(), release(), ingest(), send(), on(), connectHardware(), disconnectHardware(), setOutput() and toggleLearn().

It fills its width, and its height follows the faceplate (a taller stack on a phone) unless you set one. CSS variables: --oav-bg, --oav-fg, --oav-accent, --oav-line, --oav-font, and --oav-ratio (width ÷ height). Parts: ::part(toolbar), ::part(body), ::part(readout).

Before you call it

Wait for the controller to exist: await el.ready (a promise that resolves with the element), or listen for the ready event. Before that, set(), press(), release() and ingest() return null and print one console warning, and get() also returns null. It never asks for MIDI permission unless hardware, midi-out or Learn is used.

Members

NameDescription
connectedCallback()methodStandard custom-element callback: when the element joins the page it builds its controller (in a microtask).
disconnectedCallback()methodStandard custom-element callback: when the element leaves the page it disposes its controller and releases held notes.
attributeChangedCallback(name, was, now)methodStandard custom-element callback. Changing profile, layout, hardware, midi-out, follow, readout or theme updates the live element. Changing channel or lang rebuilds it.
set(id, value, opts = {})methodMoves a control like a hand: knobs and faders 0..1, pads and buttons above 0 press and 0 release, keys set('keys', velocity, { note: 60 }). quiet: true moves only the faceplate: no event, no MIDI out. A number as the third argument is taken as note. Returns the bytes sent, or null.
get(id)methodThe current value of one control, or null when not ready or the id does not exist.
values()method{ id: value } for every control; {} before the element is ready.
reset(opts)methodLets go of everything and brings every control home. reset({ quiet: true }) moves only the faceplate.
press(id, velocity = 1, note = null)methodPresses a pad, button or key.
release(id, note = null)methodLets go of it.
ingest(bytes, opts)methodMIDI in from anywhere (a WebSocket, another tab): the faceplate follows and control fires with source: 'hardware'. One message or a list of messages.
send(bytes)methodRaw bytes to the MIDI output port, when midi-out is open. Returns true when sent.
on(name, fn)methodSubscribes to an event by name; fn(detail, event). Returns an unsubscribe function. Same as addEventListener, but you get the payload directly.
connectHardware()methodFollows the real device (asks the browser for Web MIDI), using the hardware attribute's port name if it has one. Resolves true or false.
disconnectHardware()methodStops following the hardware.
setOutput(query)methodSends what is played to a MIDI output: a name, substring or /regex/; null turns it off. Resolves with the port name or null.
toggleLearn(on)methodTurns Learn mode on or off (no argument toggles) and shows a hint in the readout. Turning it on also starts listening to hardware. Returns the new state.
coregetterThe headless controller underneath: the same API as createController() (null before ready).
controllergetterThe live model of the device on screen (MidiController).
controllersgetterEvery device this element can show (MidiControllers).
viewgetterThe faceplate (ControllerView).
midigetterThe Web MIDI engine (Midi), once hardware or MIDI out is on.
statusgetter{ hardware, input, output, outputState, profile }: see ControllerHost for the values.
profileDatagetterThe normalized profile on screen.
learngetterBoolean property for the learn attribute.
pickergetterBoolean property for the picker attribute.
readoutgetterBoolean property for the readout attribute.
followgetterBoolean property for the follow attribute.
profilegetterString property for the profile attribute. Setting it switches the device on screen.
layoutgetterString property for the layout attribute: auto, face or stack.
themegetterString property for the theme attribute: dark or light.
hardwaregetterString property for the hardware attribute.
midiOutgetterString property for the midi-out attribute.
channelgetterNumber property for the channel attribute: 1 to 16, or null when the attribute is missing or out of range.
observedAttributesstatic propertyThe attributes the element reacts to: EMBED_ATTRS plus lang.

Notes

  • For a place that only takes an iframe, the same element is served at /embed/controller/ (query parameters mirror the attributes). Hardware and MIDI out inside the frame need allow="midi" on the <iframe>. See parseEmbedOptions() and toParent().
  • If a page imports both @openav/midi and @openav/midi/element, the browser loads index.js twice, because the element imports its own copy under a ?v= stamp. Checks across the two copies fail: el.core instanceof ControllerHost is false. Talk to the element through its methods and events instead.
  • Like createController(), it sends one anonymous hit per page load, never from localhost or the LAN. telemetry="off" stops it.

Examples

js
// importing the file registers <oav-controller>
import '@openav/midi/element';

const app = document.getElementById('app');
app.innerHTML = '<oav-controller profile="akai-lpd8" readout></oav-controller>';
const el = app.firstElementChild;

await el.ready;                                    // the controller exists now
el.on('control', (d) => console.log(d.signal, d.value.toFixed(2), d.source));
el.set('k1', 0.5);                                 // turns knob 1 on the screen
el.ingest([0xb0, 71, 127]);                        // MIDI from "elsewhere": knob 2 follows
module An LPD8 appears with a message line under it. The console prints midi/lpd8/k1 0.50 ui, then midi/lpd8/k2 1.00 hardware. Drag the knobs or tap the pads.
js
import '@openav/midi/element';

const app = document.getElementById('app');
app.innerHTML = '<oav-controller profile="lpd8" picker style="max-width: 420px"></oav-controller>';
const el = app.firstElementChild;
await el.ready;

el.addEventListener('profilechange', (e) => console.log('now showing', e.detail.name));
el.profile = 'nano2';                              // the same as setAttribute('profile', 'nano2')
console.log(el.status.hardware, el.channel);       // off null
module The toolbar has a device picker. Setting el.profile switches the faceplate to the Korg nanoKONTROL2 and the console prints its name, then off null.
html
<script type="module" src="https://openaudiovisual.com/packages/midi/element.js"></script>

<!-- plays on its own; the real device and MIDI out stay off until you ask -->
<oav-controller profile="akai-lpd8" readout></oav-controller>

<!-- a button asks for the real device on click; a MIDI output named like IAC receives what you play -->
<oav-controller profile="arturia-minilab3" hardware="ask" midi-out="IAC" picker learn></oav-controller>
static The two lines you paste into any page. The first plays with the mouse; the second shows a Connect button and a Learn button.

To be confirmed

Not tried for this page: hardware and midi-out inside an iframe in each browser, and a real MIDI device behind the element. The allow="midi" requirement is as the package README states it.

Related

  • defineController() Registers the controller element under another tag name (importing @openav/midi/element already registers it as oav-controller).
  • createController() Makes one MIDI controller you can play, listen to, and point at hardware or a DAW, with no element and no framework: createController('akai-lpd8').
  • ControllerHost The object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
  • parseEmbedOptions() Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.
  • EMBED_ATTRS The attributes the oav-controller element reads, which are also the query parameters of the iframe page: profile, layout, hardware, midi-out and more.
  • EMBED_EVENTS The nine events the oav-controller element dispatches and the iframe posts to its page, from ready and control to profilechange and error.
  • controlDetail() Builds the control event payload for one control change: id, value, raw, signal, the parsed message, its bytes and where it came from.

import { OavController } from '@openav/midi/element'; packages/midi/element.js, line 82

PANEL_CSS#

constant@openav/midi
PANEL_CSS = "\n.mcp { position: absolute; inset: 0; display: flex; flex-direction: column; …"

The stylesheet text of the controller panel (mountMidiPanel), as a string.

Advanced: it is exported for pages that want to read or override the rules. You never need it to mount a panel.

Type
String

CSS for the .mcp… classes. mountMidiPanel() injects it by itself, once per document (a <style id="openav-midi-panel-css"> in the head).

Examples

js
import { PANEL_CSS } from '@openav/midi';

console.log(PANEL_CSS.includes('.mcp.dock'));   // the drawer rules
module Prints true: the panel's rules are readable CSS text.

Related

  • mountMidiPanel() Mounts a controller panel: a toolbar (device picker, hardware status, port picker, Learn, last message) above a playable faceplate, docked at the bottom or full size.
  • VIEW_CSS The stylesheet text of the on-screen controller (ControllerView), as a string.

import { PANEL_CSS } from '@openav/midi'; packages/midi/panel.js, line 15

parseEmbedOptions()#

function@openav/midi
parseEmbedOptions(src)

Reads the options of one embed from an element, a query string, an object or a getter function, and returns them cleaned up with defaults.

It is the one place that decides what the attributes and the iframe's query string mean, so the element and the iframe always agree. Defaults: profile is arturia-minilab3, layout auto, theme dark, hardware and midiOut off (so embedding never asks for MIDI permission), the booleans false, telemetry true, and channel, bg, frame, origin, lang are null.

Rules worth knowing: an unknown layout becomes auto; theme is light or dark; midi-out may also be written midiout or midiOut, and alone it means ask; frame may also be written id and is cut to 64 characters; origin is kept only as https://host or http://host with no path (anything else, such as javascript:, becomes null), lang is cut to 16 characters, and bg goes through embedColor().

Parameters

NameTypeDescription
srcElement | URLSearchParams | Object | FunctionWhere the options live: an element (read with getAttribute), a URLSearchParams (the iframe page passes its own), a plain object { profile: 'lpd8' }, or a function (name) => value | null.

Returns

Object { profile, layout, hardware, midiOut, channel, learn, picker, readout, follow, telemetry, theme, bg, frame, origin, lang }. hardware and midiOut are { mode, port } as in embedMode().

Examples

js
import { parseEmbedOptions } from '@openav/midi/embed';

const q = new URLSearchParams('profile=lpd8&layout=stack&hardware&midi-out=IAC&readout&bg=transparent&origin=https://example.com');
const o = parseEmbedOptions(q);
console.log(o.profile, o.layout, o.hardware, o.midiOut);
console.log(o.readout, o.bg, o.origin, o.channel);
module Prints lpd8 stack, hardware as { mode: 'on', port: null }, midiOut as { mode: 'on', port: 'IAC' }, then true transparent https://example.com null.
js
import { parseEmbedOptions } from '@openav/midi/embed';

console.log(JSON.stringify(parseEmbedOptions({})));                       // every default
const o = parseEmbedOptions({ 'midi-out': '', hardware: 'off' });
console.log(JSON.stringify(o.midiOut), JSON.stringify(o.hardware));      // midiOut asks, hardware stays off
console.log(parseEmbedOptions({ origin: 'javascript:alert(1)' }).origin);   // null
module Prints every default as JSON, then midiOut as {"mode":"ask","port":null} next to hardware as off, then null.

Related

  • embedMode() Reads hardware and midi-out: off, ask (a button, so the permission prompt waits for a click) or on, plus which port.
  • embedFlag() Reads a boolean attribute: present means on, unless it says false, 0, off or no.
  • embedChannel() Reads the channel attribute: a whole number from 1 to 16, or null for anything else.
  • embedColor() Checks the bg colour of the iframe page and returns a safe value: transparent, a hex colour, or a plain colour name; anything else is null.
  • EMBED_ATTRS The attributes the oav-controller element reads, which are also the query parameters of the iframe page: profile, layout, hardware, midi-out and more.
  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.

import { parseEmbedOptions } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 72

POST_SOURCE#

constant@openav/midi
POST_SOURCE = "openav"

The word that marks a message as an open-audiovisual embed message: 'openav', sent as source by the iframe and as target by the page.

toParent() and toFrame() write it for you, and fromFrame() and fromParent() check it.

Type
String
Range
'openav'

A constant. A message from the iframe has source: 'openav', and a command to the iframe has target: 'openav'. Anything else is ignored on both sides.

Examples

js
import { POST_SOURCE, toParent, toFrame } from '@openav/midi/embed';

console.log(POST_SOURCE);                              // openav
console.log(toParent('ready').source, toFrame('get').target);   // openav openav
module Prints openav, then openav openav.

Related

  • POST_VERSION The version of the iframe message format, 1, written as v in every message.
  • toParent() Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
  • toFrame() Page side: builds a command to post into an embed iframe: { target: 'openav', v: 1, type, ...args }.
  • fromFrame() Page side: reads a message that came from an embed iframe, returns { type, frame, detail, v }, or null when it is not one of ours.
  • fromParent() Frame side: validates a command from the page and returns a clean command, or null (the message is ignored).

import { POST_SOURCE } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 268

POST_VERSION#

constant@openav/midi
POST_VERSION = 1

The version of the iframe message format, 1, written as v in every message.

Messages from toParent() and toFrame() carry it as v: 1.

Type
Number
Range
1

A constant, so a page can tell when the format changes. fromFrame() gives v back (1 when a message lacks it).

Examples

js
import { POST_VERSION, toParent } from '@openav/midi/embed';

console.log(POST_VERSION, toParent('ready').v);   // 1 1
module Prints 1 1.

Related

  • POST_SOURCE The word that marks a message as an open-audiovisual embed message: 'openav', sent as source by the iframe and as target by the page.
  • toParent() Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
  • fromFrame() Page side: reads a message that came from an embed iframe, returns { type, frame, detail, v }, or null when it is not one of ours.

import { POST_VERSION } from '@openav/midi/embed'; packages/midi/embed.js, line 269

preferredAspect()#

function@openav/midi
preferredAspect(profile, width, { layout = 'auto', minTouch = 38 } = {})

Works out the width ÷ height a controller wants in a box of a given width: its faceplate when every control stays finger-sized, a taller stack otherwise.

<oav-controller> uses it to set its height, so a controller sizes itself from its width. In auto, the faceplate's own proportions are used while the smallest control stays at least minTouch pixels at that width; otherwise it becomes a stack, about 0.8 to 1.5 times as tall as it is wide depending on how many bands the profile's portrait has. Square grid devices (the Launchpad, stack: false) always keep their face.

Parameters

NameTypeDescription
profileObjectA normalized profile (face, sections, portrait, stack).
widthNumberThe box width in pixels.
layoutString'auto', 'face' or 'stack'. optional default 'auto'
minTouchNumberThe smallest control size in pixels that still counts as finger-sized. optional default 38

Returns

{ mode, aspect } mode is 'face' or 'stack', and aspect is the width ÷ height to give the box.

Examples

js
import { normalizeProfile, profileById } from '@openav/midi';
import { preferredAspect } from '@openav/midi/embed';

const lpd8 = normalizeProfile(profileById('lpd8'));
console.log(preferredAspect(lpd8, 1200));                      // { mode: 'face', aspect: 3.70... }
console.log(preferredAspect(lpd8, 340));                       // { mode: 'stack', aspect: 0.92... } : a phone
console.log(preferredAspect(lpd8, 340, { layout: 'face' }));   // forced back to the face

const launchpad = normalizeProfile(profileById('lpmini'));
console.log(preferredAspect(launchpad, 300));                  // { mode: 'face', aspect: 1 } : always square
module Prints four results: a wide face, a tall stack for a narrow box, the face when forced, and the square Launchpad.

Related

  • OavController The oav-controller custom element: a playable MIDI controller you paste into any page with one line, with no import map and no build.
  • EMBED_LAYOUTS The three layouts a controller can have: auto, face (the device as it looks) and stack (re-gridded for a tall box).
  • ControllerView The playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.

import { preferredAspect } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 251

restValue()#

function@openav/midi
restValue(c)

Says where a control rests: knobs and faders at their default (0), bipolar wheels at the centre, pads, buttons and keys at 0.

reset() on ControllerHost uses it to bring every control home. An endless encoder has no real position, so only its drawn position goes to the middle and no MIDI is sent for it.

Parameters

NameTypeDescription
cObjectA control definition from a normalized profile.

Returns

Number 0 for keys, pads, buttons and bipolar controls; otherwise the control's def, else 0.5 for an endless encoder and 0 for the rest.

Examples

js
import { normalizeProfile, profileById } from '@openav/midi';
import { restValue } from '@openav/midi/embed';

const p = normalizeProfile(profileById('minilab3'));
for (const id of ['knob1', 'pitch', 'enc', 'pad1', 'keys']) {
  console.log(id, restValue(p.controls.find((c) => c.id === id)));
}
module Prints 0 for the knob, the pitch strip (centre), the pad and the keys, and 0.5 for the endless encoder.

Related

  • setControl() Moves one control the way a hand would, whatever its type, and returns the MIDI bytes it sent: setControl(mc, 'k1', 0.5).
  • ControllerHost The object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.

import { restValue } from '@openav/midi/embed'; packages/midi/embed.js, line 238

setControl()#

function@openav/midi
setControl(mc, id, value, { note = null, source = 'ui' } = {})

Moves one control the way a hand would, whatever its type, and returns the MIDI bytes it sent: setControl(mc, 'k1', 0.5).

What value means depends on the type. Knob, fader, wheel, strip and encoder: 0..1 (−1..1 for a bipolar one). Pad and momentary button: above 0 presses (with that velocity, held to 1) and 0 releases. Toggle button: 0.5 or more is on. Keys: value is the velocity, above 0 presses that note and 0 releases it. This is what set() on ControllerHost and <oav-controller> call.

Parameters

NameTypeDescription
mcMidiControllerThe controller.
idStringThe control id. An unknown id throws <profile> has no control "x" (controls: …).
valueNumberThe value (below). Anything that is not a finite number throws set("id"): value must be a number.
noteNumber | nullFor a keys control: which note. The default is the first key of its range. optional default null
sourceString'ui' (default) publishes and reaches MIDI out. 'mirror' moves only the faceplate (two-way binding). optional default 'ui'

Returns

number[] | null The bytes sent, or null when nothing changed (for example releasing a pad that is not held).

Examples

js
import { MidiController, profileById } from '@openav/midi';
import { setControl } from '@openav/midi/embed';

const mc = new MidiController(profileById('lpd8'));
console.log(setControl(mc, 'k1', 0.5));      // [176, 70, 64]  : a knob
console.log(setControl(mc, 'pad1', 1));      // [153, 36, 127] : a pad pressed
console.log(setControl(mc, 'pad1', 0));      // [137, 36, 0]   : released
console.log(setControl(mc, 'pad1', 0));      // null           : nothing to release
module Prints the three messages the control types send, then null.

Related

  • valuesOf() Reads every control's current value from a controller as { id: number }; for keys it is 1 while any key is held.
  • MidiController One MIDI controller (a profile) as a live model: MIDI bytes go in, control values and signals come out, and a hand on screen sends the bytes the device would.
  • ControllerHost The object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.
  • restValue() Says where a control rests: knobs and faders at their default (0), bipolar wheels at the centre, pads, buttons and keys at 0.

import { setControl } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 221

toFrame()#

function@openav/midi
toFrame(type, args = {})

Page side: builds a command to post into an embed iframe: { target: 'openav', v: 1, type, ...args }.

It does not check the command: the iframe does, with fromParent(), and ignores what it does not understand. Post to the frame's real origin, not *, when you can.

Parameters

NameTypeDescription
typeStringThe command: one of FRAME_COMMANDS.
argsObjectThe command's fields ({ id, value } for set). Copied through JSON. optional default {}

Returns

Object A message ready for iframe.contentWindow.postMessage(msg, origin).

Examples

js
import { toFrame } from '@openav/midi/embed';

console.log(JSON.stringify(toFrame('set', { id: 'k1', value: 0.5 })));
console.log(JSON.stringify(toFrame('ingest', { bytes: [0xb0, 70, 64] })));
console.log(JSON.stringify(toFrame('profile', { profile: 'korg-nanokontrol2' })));
// iframe.contentWindow.postMessage(toFrame('get'), 'https://openaudiovisual.com');
module Prints three command messages; the last line (commented out) is how you would send one.

Related

  • fromParent() Frame side: validates a command from the page and returns a clean command, or null (the message is ignored).
  • toParent() Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.
  • FRAME_COMMANDS The commands a page may send into an embed iframe: set, ingest, reset, profile, layout, get, press, release.
  • POST_SOURCE The word that marks a message as an open-audiovisual embed message: 'openav', sent as source by the iframe and as target by the page.

import { toFrame } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 285

toParent()#

function@openav/midi
toParent(type, detail = null, { frame = null } = {})

Frame side: builds the message the iframe posts to its page for an event: { source: 'openav', v: 1, type, frame, detail }.

The iframe page calls parent.postMessage(toParent(name, detail, { frame }), target) for every event, where target is the ?origin= parameter or *. With ?origin=https://your.site the frame posts only to that origin and accepts commands only from it.

Parameters

NameTypeDescription
typeStringThe event name (see EMBED_EVENTS), or values.
detailanyThe event payload. It is copied through JSON, so the message is always postMessage-safe (undefined becomes null). optional default null
frameString | nullA name for this iframe (the ?frame= parameter), echoed in every message so a page with several frames can tell them apart. optional default null

Returns

Object The message { source: 'openav', v: 1, type, frame, detail }.

Examples

js
import { toParent } from '@openav/midi/embed';

const msg = toParent('control', { id: 'k1', value: 0.5 }, { frame: 'pads' });
console.log(JSON.stringify(msg));               // {"source":"openav","v":1,"type":"control","frame":"pads","detail":{"id":"k1","value":0.5}}
console.log(JSON.stringify(toParent('ready')));  // frame and detail are null
module Prints the full message for a control event, then a bare ready message with frame and detail as null.

Related

  • fromFrame() Page side: reads a message that came from an embed iframe, returns { type, frame, detail, v }, or null when it is not one of ours.
  • toFrame() Page side: builds a command to post into an embed iframe: { target: 'openav', v: 1, type, ...args }.
  • fromParent() Frame side: validates a command from the page and returns a clean command, or null (the message is ignored).
  • POST_SOURCE The word that marks a message as an open-audiovisual embed message: 'openav', sent as source by the iframe and as target by the page.
  • EMBED_EVENTS The nine events the oav-controller element dispatches and the iframe posts to its page, from ready and control to profilechange and error.

import { toParent } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 276

touchText()#

function@openav/midi
touchText(c, st, note = null)

Writes what a device's screen says about a touch: Knob 1 · 87, C4 · 100, Pad 3 · 96, Pitch +0.42.

ControllerCanvas draws this text on the device's screen (MiniLab 3, KeyStep) so a video shows what the hardware's own screen would. It is pure, so it is easy to reuse for a caption or a log.

Parameters

NameTypeDescription
cObjectThe control's definition (from the normalized profile). null gives an empty string.
stObjectIts state: { raw, value, held, pos }.
noteNumber | nullFor a keys control, the note that was touched. optional default null

Returns

String The line. A control whose label is only a number is named after its type (Knob 1). A pad shows its velocity only while held, a button shows on or off, a relative encoder shows its position as a percentage, and a bipolar wheel shows a signed value.

Examples

js
import { touchText, normalizeProfile, profileById } from '@openav/midi';

const p = normalizeProfile(profileById('minilab3'));
const c = (id) => p.controls.find((x) => x.id === id);

console.log(touchText(c('knob1'), { raw: 87 }));                // Knob 1 · 87
console.log(touchText(c('keys'), { raw: 100 }, 60));            // C4 · 100
console.log(touchText(c('pad3'), { held: true, raw: 96 }));     // Pad 3 · 96
console.log(touchText(c('pitch'), { value: -0.5 }));            // Pitch −0.50
module Prints the four lines shown in the comments.

Related

  • ControllerCanvas Draws a controller's faceplate into any 2D canvas context, every frame, at any size: the bottom of a performance video, an overlay, a thumbnail.
  • midiNoteName() Turns a MIDI note number into its name with the octave, middle C being C4: midiNoteName(60) is 'C4', midiNoteName(61) is 'C#4'.

import { touchText } from '@openav/midi'; packages/midi/canvas.js, line 167

valuesOf()#

function@openav/midi
valuesOf(mc)

Reads every control's current value from a controller as { id: number }; for keys it is 1 while any key is held.

This is what values() returns on ControllerHost and <oav-controller>, and what the iframe answers to the get command.

Parameters

NameTypeDescription
mcMidiControllerThe controller.

Returns

Object One number per control id, in the profile's order.

Examples

js
import { MidiController, profileById } from '@openav/midi';
import { valuesOf } from '@openav/midi/embed';

const mc = new MidiController(profileById('lpd8'));
mc.setValue('k1', 1);
mc.press('pad2', 0.5);
const values = valuesOf(mc);
console.log(Object.keys(values).length, 'controls');
console.log('k1 =', values.k1, '| pad2 =', values.pad2.toFixed(2), '| pad1 =', values.pad1);
module Prints 16 controls, then k1 = 1 | pad2 = 0.50 | pad1 = 0: one number for every control of the LPD8.

Related

  • setControl() Moves one control the way a hand would, whatever its type, and returns the MIDI bytes it sent: setControl(mc, 'k1', 0.5).
  • ControllerHost The object createController() returns: one controller with its events, its playable faceplate, its hardware connection and its MIDI output.

import { valuesOf } from '@openav/midi'; Also from @openav/midi/embed packages/midi/embed.js, line 206

VIEW_CSS#

constant@openav/midi
VIEW_CSS = "\n.mcv { position: relative; width: 100%; height: 100%; overflow: hidden; touc…"

The stylesheet text of the on-screen controller (ControllerView), as a string.

Advanced. You never need it to show a controller. It is exported for pages that build their own shadow DOM, a server-rendered preview or a theme that wants to read the rules. Colours use the CSS variables --oav-focus, --body, --c and --cap that the view sets per control.

Type
String

CSS for the .mcv… classes. The view injects it by itself, once per document, or once per shadow root when it lives inside one (a <style id="openav-midi-view-css">).

Examples

js
import { VIEW_CSS } from '@openav/midi';

console.log(typeof VIEW_CSS, VIEW_CSS.length > 1000);
console.log(VIEW_CSS.includes('.mcv-pad'));   // the rules for pads
module Prints string true, then true: it is plain CSS text you can read or inject yourself.

Related

  • ControllerView The playable on-screen faceplate of a controller: knobs, faders, pads and keys you drive with a mouse, a finger or the keyboard, repainted from the controller's state.
  • PANEL_CSS The stylesheet text of the controller panel (mountMidiPanel), as a string.

import { VIEW_CSS } from '@openav/midi'; packages/midi/view.js, line 32

Keyboard (@openav/keys)#

The on-screen piano and the QWERTY fallback. 4

@openav/keys

KeysPiano#

class@openav/keys
new KeysPiano(container, opts = {})

An on-screen piano you can play with the mouse, a finger or the computer keyboard, that calls a function for every note.

new KeysPiano(el, { onNote }) builds the keys inside el. Whatever plays a key (a click or a touch on it, sweeping across the keys with the button held, the computer keyboard, or your own call to press()) ends up in onNote(note, vel01, on): on is true for a key going down and false for one coming up, and vel01 is 0..1 (0 on the way up). The black and white pattern follows the real pitch, so a piano can start on any note. A key that is already down ignores a second press().

The computer keyboard is off until you set piano.captureEnabled = true, and it is up to you to pass the page's keydown and keyup events to handleKeyDown(e) and handleKeyUp(e) (see PIANO_KEYMAP for which letter is which note). mountKeys() does all of that for you.

Parameters

NameTypeDescription
containerHTMLElementThe element the keys are built in. It gets the class oav-keys and is emptied first.
optsObjectOptions: base (lowest key, default 48), octaves (default 2), semitones (the number of keys, default octaves × 12 + 1), velocity (0..127 for the mouse and the computer keyboard, default 100), onNote(note, vel01, on), onOctave(base), fill (size the keys in percent of the container, so it can be any width), keyWidth (px per white key when not filling, default 27), minBase (24) and maxBase (84), the range the octave keys can move base in. optional default {}

Members

NameDescription
noteName(n)methodThe name of a MIDI note with sharps: noteName(60) is 'C4', noteName(61) is 'C#4'.
setBase(n)methodMoves the piano to start on note n, held between minBase and maxBase. Sounding keys are released first, so no note sticks. onOctave(base) is called with the new value when it changed.
press(note, vel01)methodPresses a key as if it was played: press(60, 0.8). The key lights up and onNote(60, 0.8, true) is called. Pressing a key that is already down does nothing.
release(note)methodLets a key go: release(60) calls onNote(60, 0, false). A key that is not down does nothing.
releaseAll()methodReleases every key that is down, from the mouse, touch or computer keyboard. Call it when focus is lost so nothing sticks.
handleKeyDown(e)methodGive it a keydown event. Z and X move the piano an octave down and up, a letter in PIANO_KEYMAP presses its note (Shift adds 20 to the velocity), and the other letters are swallowed. Returns true when the event was used. A key held down (e.repeat) does not retrigger.
handleKeyUp(e)methodGive it a keyup event: it releases the note that key pressed. Returns true when the event was used.

Notes

  • handleKeyDown(e) returns true when the piano used the event, and then you should call e.preventDefault(). While capturing, every letter a–z counts as used, even those that play no note, so a stray letter cannot trigger a hotkey; digits, space and the arrows pass through. Held keys with Ctrl, Cmd or Alt are never used.

Examples

js
import { KeysPiano } from '@openav/keys';

const piano = new KeysPiano(document.getElementById('app'), {
  base: 60, octaves: 1,
  onNote: (note, vel, on) => console.log(on ? 'on' : 'off', piano.noteName(note), vel.toFixed(2)),
});
piano.press(64, 0.5);     // on E4 0.50
piano.release(64);        // off E4 0.00
module A one-octave piano from middle C appears on the page. Click its keys and the console logs each note by name.
js
import { KeysPiano } from '@openav/keys';

const piano = new KeysPiano(document.getElementById('app'), {
  fill: true,                                     // as wide as the container
  onNote: (note, vel, on) => { if (on) console.log('note', note); },
});
piano.captureEnabled = true;                      // the computer keyboard plays
window.addEventListener('keydown', (e) => { if (piano.handleKeyDown(e)) e.preventDefault(); });
window.addEventListener('keyup', (e) => piano.handleKeyUp(e));
window.addEventListener('blur', () => piano.releaseAll());
module Click the page, then press A, S, D, F… to play from C3, Z and X to change octave. The letter keys no longer reach the rest of the page.

Related

  • mountKeys() Puts a playable piano on the page in one call, with a keyboard switch and a self-playing performer, and publishes every note as midi/note/on and midi/note/off.
  • SimPlayer A hands-free performer: it presses and releases notes on its own, in a scale, so a demo can play itself.
  • PIANO_KEYMAP Which computer-keyboard letter plays which note: a is the base note, w a semitone higher, s two.
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.
  • noteName() Turns a MIDI number into a note name: noteName(61) is 'C#4'.

import { KeysPiano } from '@openav/keys'; packages/keys/index.js, line 48

mountKeys()#

function@openav/keys
mountKeys(container, { signals, chord = null, base = 48, octaves = 2, sim = true, capture = true } = {})

Puts a playable piano on the page in one call, with a keyboard switch and a self-playing performer, and publishes every note as midi/note/on and midi/note/off.

mountKeys() is the whole keyboard department in one line: a KeysPiano you can play with the mouse, a finger or the computer keyboard, and a SimPlayer that plays by itself. Above the piano are two checkboxes, both off to begin with: keyboard (the QWERTY keys play, Z and X move an octave, Shift plays harder) and simulate performance.

Everything is published as the standard midi/note/on and midi/note/off, the same names a real MIDI keyboard uses, so a world, a sound engine or a chord detector cannot tell a QWERTY performance from hardware. Call keys.update(dt) once per frame, or the simulated performer does not move.

Before you call it

A page with a DOM. The keyboard checkbox must be ticked before the computer keyboard plays, on purpose: while it is on, every letter key is taken by the piano, so hotkeys on letters stop working. A key listener goes on window until dispose().

Parameters

NameTypeDescription
containerHTMLElementThe element to put the piano in. It gets a small bar of checkboxes and the piano under it. dispose() empties it again.
signalsSignalsThe Signals bus to publish into. Every key down is midi/note/on { note, vel, ch: 0 } (vel 0..1) and every key up midi/note/off { note, ch: 0 }. Without a bus the piano still works and calls chord. optional
chordChordDetectorAnything with noteOn(note, vel01) and noteOff(note), usually a chord detector from @openav/chord, to be fed every note directly. optional default null
baseNumberThe MIDI note of the lowest key (48 is C3). The simulated performer plays an octave above it. optional default 48
octavesNumberHow many octaves the piano shows. It has octaves × 12 + 1 keys, so two octaves end on a C. optional default 2
simBooleanShow the "simulate performance" checkbox. The performer exists either way: keys.sim.toggle(true) starts it from code. optional default true
captureBooleanShow the "keyboard" checkbox that lets the computer keyboard play the piano. With false there is no checkbox and captureBox is null; you can still set keys.piano.captureEnabled = true. optional default true

Returns

Object { piano, sim, update(dt), dispose(), captureBox }: the KeysPiano, the SimPlayer, a function to call every frame with the seconds since the last one (it drives the performer), a function that takes everything down again, and the "keyboard" checkbox (or null).

Notes

  • In a p5 sketch you do not need this: the 🎹 panel of oav-p5 is a piano like this one, and notePressed() hears it. mountKeys() is for a page that has no oav-p5.

Examples

js
import { mountKeys } from '@openav/keys';
import { Signals } from '@openav/core';

const signals = new Signals();
signals.on('midi/note/on', ({ note, vel }) => console.log('on', note, vel.toFixed(2)));
signals.on('midi/note/off', ({ note }) => console.log('off', note));

const keys = mountKeys(document.getElementById('app'), { signals, base: 48 });
keys.piano.press(60, 0.8);     // as if C4 was clicked
keys.piano.release(60);
// on 60 0.80
// off 60
module A two-octave piano appears on the page with its two checkboxes; click a key and the console logs the same events the code below sends.
js
import { mountKeys } from '@openav/keys';
import { Signals } from '@openav/core';

const signals = new Signals();
const heard = [];
signals.on('midi/note/on', ({ note }) => heard.push(note));

const keys = mountKeys(document.getElementById('app'), { signals, octaves: 3, capture: false });
keys.sim.toggle(true);                          // the performer plays by itself
const t0 = performance.now();
let last = t0;
(function frame(now) {
  keys.update((now - last) / 1000); last = now;   // seconds since the last frame
  if (now - t0 < 3000) return requestAnimationFrame(frame);
  console.log(heard.length > 0, keys.captureBox);  // true null
  keys.dispose();                                   // the piano is taken down
})(t0);
module A three-octave piano plays itself for three seconds, its keys lighting up, and then it is taken down. capture: false leaves out the keyboard checkbox.

Related

  • KeysPiano An on-screen piano you can play with the mouse, a finger or the computer keyboard, that calls a function for every note.
  • SimPlayer A hands-free performer: it presses and releases notes on its own, in a scale, so a demo can play itself.
  • PIANO_KEYMAP Which computer-keyboard letter plays which note: a is the base note, w a semitone higher, s two.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • notePressed() You define this function; oav-p5 calls it each time a key (or a pad) goes down.

import { mountKeys } from '@openav/keys'; packages/keys/index.js, line 229

PIANO_KEYMAP#

constant@openav/keys
PIANO_KEYMAP = object, 17 keys

Which computer-keyboard letter plays which note: a is the base note, w a semitone higher, s two.

The layout the music software calls the DAW standard: the A S D F G H J K L ; row is the white keys and the W E T Y U O P row above it is the black keys, placed where they sit on a piano. With a piano that starts on C (base 48, 60…) it reads C, C♯, D, D♯, E, F, F♯, G, G♯, A, A♯, B, C, C♯, D, D♯, E. The keys are lowercase letters; KeysPiano.handleKeyDown(e) lowercases the key first. Z and X are not in the map: they move the octave.

Type
Object

{ a: 0, w: 1, s: 2, e: 3, d: 4, f: 5, t: 6, g: 7, y: 8, h: 9, u: 10, j: 11, k: 12, o: 13, l: 14, p: 15, ';': 16 }: key to semitones above the piano's base.

Examples

js
import { PIANO_KEYMAP } from '@openav/keys';

const base = 60;                                   // middle C
const noteFor = (key) => (key in PIANO_KEYMAP ? base + PIANO_KEYMAP[key] : null);
console.log(noteFor('a'), noteFor('w'), noteFor('k'), noteFor('q'));   // 60 61 72 null
console.log(Object.keys(PIANO_KEYMAP).join(''));                        // awsedftgyhujkolp;
module Looks up the note of a few letters on a piano that starts on middle C: k is the C an octave up, and q plays nothing.

Related

  • KeysPiano An on-screen piano you can play with the mouse, a finger or the computer keyboard, that calls a function for every note.
  • mountKeys() Puts a playable piano on the page in one call, with a keyboard switch and a self-playing performer, and publishes every note as midi/note/on and midi/note/off.

import { PIANO_KEYMAP } from '@openav/keys'; packages/keys/index.js, line 18

SimPlayer#

class@openav/keys
new SimPlayer({ press, release, base = 60, scale = [0, 2, 4, 7, 9], density = 1 } = {})

A hands-free performer: it presses and releases notes on its own, in a scale, so a demo can play itself.

It plays like someone noodling. Every gesture it picks at random: mostly a step up or down the scale, sometimes a leap, now and then a triad stacked in the scale, and real rests. Notes last 0.18 to 0.68 seconds (a triad twice as long), the next gesture comes 0.22 to 0.77 seconds later (a rest lasts 0.6 to 2 seconds), all at the default density of 1, and the notes are kept between 36 and 96. Because it presses and releases through your own two functions, whatever listens downstream cannot tell it from a person.

It starts off. Call toggle(true) and then update(dt) once per frame with the seconds since the last frame; it does nothing while it is off. toggle(false) releases every note it is holding, so none sticks. The notes are random (it uses Math.random, there is no seed).

Parameters

NameTypeDescription
pressFunctionpress(note, vel01): called when a note starts, with its MIDI number and a velocity between 0.45 and 0.85. Usually the press of a KeysPiano, so the keys light up.
releaseFunctionrelease(note): called when the note ends.
baseNumberThe MIDI note the scale starts on. optional default 60
scaleNumber[]The notes it may play, as semitones above base. The default is a pentatonic scale, which sounds fine whatever happens. optional default [0, 2, 4, 7, 9]
densityNumberHow busy it is. The gaps between notes are divided by it, so 2 plays about twice as often. optional default 1

Members

NameDescription
toggle(on = !this.enabled)methodTurns the performer on or off; with no argument it flips. Returns the new state. Turning it off releases every note it holds.
update(dt)methodAdvance by dt seconds: release the notes that are due and, when it is time, start the next gesture. Call it every frame. Does nothing while the performer is off.

Examples

js
import { SimPlayer } from '@openav/keys';

const played = [];
const sim = new SimPlayer({
  press: (note, vel) => played.push(note),
  release: () => {},
  base: 60, density: 2,
});
sim.toggle(true);
for (let i = 0; i < 600; i++) sim.update(1 / 60);   // ten seconds, in frames
console.log(played.length > 5, Math.min(...played) >= 36, Math.max(...played) <= 96);   // true true true
sim.toggle(false);
module Runs the performer for ten simulated seconds without any sound or screen and logs that it played a good number of notes, all inside 36 to 96.
js
import { KeysPiano, SimPlayer } from '@openav/keys';

const piano = new KeysPiano(document.getElementById('app'), { base: 48, octaves: 2 });
const sim = new SimPlayer({
  press: (n, v) => piano.press(n, v),        // through the piano: the keys light up
  release: (n) => piano.release(n),
  base: 60, scale: [0, 2, 3, 5, 7, 8, 10],   // a natural minor scale
});
sim.toggle(true);
const t0 = performance.now();
let last = t0;
(function frame(now) {
  sim.update((now - last) / 1000); last = now;
  if (now - t0 < 4000) requestAnimationFrame(frame); else sim.toggle(false);
})(t0);
module A piano plays itself in a minor scale for four seconds: the keys light up as the notes go down and up.

Related

  • KeysPiano An on-screen piano you can play with the mouse, a finger or the computer keyboard, that calls a function for every note.
  • mountKeys() Puts a playable piano on the page in one call, with a keyboard switch and a self-playing performer, and publishes every note as midi/note/on and midi/note/off.

import { SimPlayer } from '@openav/keys'; packages/keys/index.js, line 170

Leap Motion (@openav/leap)#

Hands as signals: frames, gestures, the bridge. 27

@openav/leap

boneBasis()#

function@openav/leap
boneBasis(prev, next, palmNormal, left)

An advanced helper: the orthonormal basis [x, y, z] of a bone, rebuilt from two joints and the palm normal the way LeapJS expects.

The sensor gives joints, not rotations, but LeapJS sketches read a rotation basis for every bone, so V6Converter uses this to fill in bases. You only need it if you write your own converter.

Parameters

NameTypeDescription
prevArray<Number>The joint at the base of the bone, [x, y, z] in millimetres.
nextArray<Number>The joint at the tip end of the bone.
palmNormalArray<Number>The palm normal; -palmNormal becomes the "back of the hand" axis (y).
leftBooleantrue for a left hand: the x axis is negated, so the determinant is negative, which is how LeapJS tells left from right.

Returns

Array Three rows [x, y, z], each a unit vector rounded to 3 decimals. z runs from the tip back to the base (LeapJS's Bone.direction() is -z).

Examples

js
import { boneBasis } from '@openav/leap';

// a bone pointing straight ahead (-z), palm facing down: the basis is the identity
console.log(boneBasis([0, 200, 0], [0, 200, -40], [0, -1, 0], false));
// the same bone on a left hand: x flips
console.log(boneBasis([0, 200, 0], [0, 200, -40], [0, -1, 0], true)[0]);
module The console prints the identity [ [ 1, 0, 0 ], [ 0, 1, 0 ], [ 0, 0, 1 ] ], then [ -1, 0, 0 ] for the left hand.

Related

  • palmBasis() An advanced helper: the orthonormal basis [x, y, z] of the palm, from its direction and normal, in the rows LeapJS reads as hand.r.
  • V6Converter Converts raw hand lines into LeapJS v6 frames, keeping the little state the protocol needs: time visible, smoothed positions and fingertip velocities.
  • rawToV6() One-off conversion of a raw frame to a LeapJS v6 frame, with a fresh converter each time (no history, so no velocities).

import { boneBasis } from '@openav/leap'; packages/leap/frame.js, line 48

DEFAULT_PORT#

constant@openav/leap
DEFAULT_PORT = 6437

The Leap bridge's default port: 6437, the port the original Leap service used, so old leap.js sketches find it unchanged.

The default of startBridge({ port }) and of --port. The environment variable LEAP_BRIDGE_PORT changes it for the command line. This is the port of the Leap bridge; the out-bridge has its own (DEFAULT_PORT of @openav/out, 6450).

Related

  • DEFAULT_URL The default address LeapInput connects to: ws://127.0.0.1:6437/v6.json, the bridge on this machine speaking the LeapJS protocol.

import { DEFAULT_PORT } from '@openav/leap/bridge'; packages/leap/bridge/leap-bridge.mjs, line 30

DEFAULT_URL#

constant@openav/leap
DEFAULT_URL = "ws://127.0.0.1:6437/v6.json"

The default address LeapInput connects to: ws://127.0.0.1:6437/v6.json, the bridge on this machine speaking the LeapJS protocol.

Port 6437 is the port the original Leap service used, so old leap.js sketches find the bridge without changing anything. …/raw on the same port serves the raw lines instead. In a show, ?leap=ws://host:port/v6.json in the page address points a page at another bridge, and ?leap=sim starts the simulator.

Related

  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.

import { DEFAULT_URL } from '@openav/leap'; packages/leap/index.js, line 36

deviceEvent()#

function@openav/leap
deviceEvent({ device = null, service = true } = {})

A LeapJS device event telling the client whether a device is attached and streaming, plus our own service flag.

LeapJS (protocol 5 and up) turns it into deviceAttached, deviceStreaming and streamingStarted. The bridge sends one on connect and on every status change. LeapInput maps it to its status: no service is 'no-service', a service without a device is 'no-device', and a streaming device is 'tracking'.

Parameters

NameTypeDescription
deviceString | nullThe device id, or null for no device. optional default null
serviceBooleanWhether the tracking service is running. LeapJS ignores this; LeapInput uses it to tell "no tracking service" from "no device". optional default true

Returns

Object { event: { type: 'deviceEvent', state: { attached, streaming, id, type: 'Peripheral', service } } }; id is 'none' without a device.

Examples

js
import { Signals } from '@openav/core';
import { LeapInput, deviceEvent } from '@openav/leap';

const signals = new Signals();
const leap = new LeapInput({ signals });
for (const event of [deviceEvent({ device: null, service: false }), deviceEvent({ device: null }), deviceEvent({ device: 'LE51' })]) {
  leap.ingest(event);
  console.log(leap.status, signals.get('leap/status'));
}
module Feeding three device events through a LeapInput: the console prints no-service 1, no-device 2, tracking 3.

Related

  • v6Header() The first message of a LeapJS v6 connection: { serviceVersion, version }, with version held to 1..6 because LeapJS throws above 6.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • STATUS The four levels of the leap/status signal by name: off 0, bridge 1, service 2, tracking 3.

import { deviceEvent } from '@openav/leap'; packages/leap/frame.js, line 197

FINGERS#

constant@openav/leap
FINGERS = array, 5 items

The five finger names in order, thumb index middle ring pinky: the order of hand.fingers and the names in the fingertip signals.

hand.fingers[0] is the thumb and hand.fingers[4] the pinky, so FINGERS[i] names hand.fingers[i]. The fingertip signals are leap/hand/<side>/<finger>/x|y|z with these names, for example leap/hand/right/index/y.

Related

  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • handsOf() Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
  • leapSignals() The list of every signal LeapInput publishes, with its range, as [[name, meta], …], ready to hand to signals.define().

import { FINGERS } from '@openav/leap'; packages/leap/frame.js, line 21

handAngles()#

function@openav/leap
handAngles(direction, normal)

Turns a hand's direction and palm normal into { pitch, yaw, roll } in radians, with the LeapJS conventions.

A flat hand, palm down, fingers away from you, is { pitch: 0, yaw: 0, roll: 0 }. The three angles are the leap/hand/<side>/pitch|yaw|roll signals, which LeapInput computes with this function from each hand.

Parameters

NameTypeDescription
directionArray<Number>The direction the fingers point, [x, y, z] (a unit vector).
normalArray<Number>The palm normal [x, y, z], pointing out of the palm (down when the palm faces the sensor).

Returns

Object { pitch, yaw, roll }, each −π..π: pitch is the fingers tipping up, yaw turning toward +x (the right), roll the palm turning over.

Notes

  • When the fingers point straight up there is no yaw or roll to measure, and the function answers π (3.14) for them. Expect that at the top of a gesture, and do not map those two signals when the hand is vertical.

Examples

js
import { handAngles } from '@openav/leap';

const round = (a) => ({ pitch: +a.pitch.toFixed(2), yaw: +a.yaw.toFixed(2), roll: +a.roll.toFixed(2) });
console.log(round(handAngles([0, 0, -1], [0, -1, 0])));           // flat, palm down: all 0
console.log(round(handAngles([0, 0.5, -0.866], [0, -0.866, -0.5])));   // fingers tipped up 30 degrees: pitch 0.52
console.log(round(handAngles([0.707, 0, -0.707], [0, -1, 0])));    // fingers turned 45 degrees to the right: yaw 0.79
console.log(round(handAngles([0, 0, -1], [0.5, -0.866, 0])));      // palm turned over 30 degrees: roll 0.52
module The console prints the angles for four hands: all zeros, then pitch: 0.52, yaw: 0.79 and roll: 0.52, one angle changing each time.

Related

  • normalizePoint() Maps a point in millimetres to 0..1 inside the interaction box, clamped by default: the same normalisation the leap/* signals use.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • palmBasis() An advanced helper: the orthonormal basis [x, y, z] of the palm, from its direction and normal, in the rows LeapJS reads as hand.r.

import { handAngles } from '@openav/leap'; packages/leap/frame.js, line 69

handBones()#

function@openav/leap
handBones(hand)

The bones to draw for a hand, as pairs of points: each finger's joints in turn, the line across the knuckles, and the forearm.

It only lists the pairs; you choose how to draw them. LeapInput.skeleton() calls it for you. Use it directly when you draw with p5, three.js or your own projection.

Parameters

NameTypeDescription
handObjectA hand from handsOf() or leap.hands.

Returns

Array An array of [pointA, pointB] pairs, each point [x, y, z] in millimetres. A hand with five 5-joint fingers, a wrist and an elbow gives 25.

Examples

js
import { INTERACTION_BOX, handBones, handsOf, mockFrame, normalizePoint } from '@openav/leap';

const ctx = document.getElementById('c').getContext('2d');
ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
ctx.strokeStyle = '#ffd166'; ctx.lineWidth = 2;
const at = (p) => { const n = normalizePoint(p, INTERACTION_BOX, false); return [n[0] * 400, (1 - n[1]) * 300]; };   // front view
for (const hand of handsOf(mockFrame(2.0))) {
  for (const [a, b] of handBones(hand)) {
    ctx.beginPath(); ctx.moveTo(...at(a)); ctx.lineTo(...at(b)); ctx.stroke();
  }
}
module Both mock hands are drawn on the canvas in a front view, bone by bone.

Related

  • handsOf() Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • normalizePoint() Maps a point in millimetres to 0..1 inside the interaction box, clamped by default: the same normalisation the leap/* signals use.

import { handBones } from '@openav/leap'; packages/leap/frame.js, line 248

HandGestures#

class@openav/leap
new HandGestures(thresholds = THRESHOLDS)

Turns pinch and grab strengths into clean start and end events, per hand plus a both-hands pinch, using hysteresis so a wobbling value fires once.

A pinch strength hovers around any single threshold while the fingers are "almost touching", and one threshold then fires start, end, start, end sixty times a second. So every gesture is a trigger with two thresholds: it starts above on and only ends below off. The defaults are a pinch starting above 0.86 and ending below 0.5, a both-hands pinch entering when both hands are above 0.8 and leaving when either drops below 0.5, and a grab (fist) above 0.8 and below 0.5. A hand that disappears ends all of its gestures.

LeapInput already runs one (leap.gestures) and turns its events into the leap/…/pinch-start pulses. Use HandGestures on its own when you have hands from somewhere else.

Parameters

NameTypeDescription
thresholdsObjectOverrides for the defaults in THRESHOLDS: { pinch: { on, off }, bothPinch: { on, off }, grab: { on, off } }. Keys you leave out keep their defaults. optional default THRESHOLDS

Members

NameDescription
update(hands)methodupdate(hands) takes [{ side: 'left' | 'right', pinch, grab, … }] and returns the events that just happened, in a stable order: { type: 'pinch-start' | 'pinch-end' | 'grab-start' | 'grab-end', side, hand }, then { type: 'both-pinch-start' | 'both-pinch-end', side: 'both', hands }. Call it once per frame.
stategetterWhat is held right now: { left: { pinch, grab }, right: { pinch, grab }, both }, all booleans.

Examples

js
import { HandGestures } from '@openav/leap';

const gestures = new HandGestures();
const frame = (pinch) => gestures.update([{ side: 'right', pinch, grab: 0 }]).map((e) => e.type);
console.log(frame(0.85));   // []               below 0.86: not yet
console.log(frame(0.9));    // [ 'pinch-start' ]
console.log(frame(0.6));    // []               still held: only below 0.5 ends it
console.log(frame(0.4));    // [ 'pinch-end' ]
module The console prints [], [ 'pinch-start' ], [], [ 'pinch-end' ]: the pinch starts above 0.86 and holds until it drops under 0.5.
js
import { HandGestures } from '@openav/leap';

const gestures = new HandGestures({ pinch: { on: 0.7, off: 0.3 } });   // easier to start, harder to let go
const hand = (side, pinch) => ({ side, pinch, grab: 0 });
const names = (events) => events.map((e) => e.type + ':' + e.side);
console.log(names(gestures.update([hand('right', 0.75), hand('left', 0.82)])));   // single pinches only: both need 0.8
console.log(names(gestures.update([hand('right', 0.9), hand('left', 0.9)])));     // now both hands pinch
console.log(names(gestures.update([])));                                          // both hands leave: everything ends
module The console prints [ 'pinch-start:left', 'pinch-start:right' ], then [ 'both-pinch-start:both' ], then the three ends when the hands leave.

Related

  • Hysteresis One Schmitt trigger: update(v) says 'start' when the value rises above on, and 'end' only when it falls below off.
  • THRESHOLDS The default hysteresis levels for the gestures: pinch 0.86 / 0.5, both-hands pinch 0.8 / 0.5, grab 0.8 / 0.5 (start above on, end below off).
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • mockFrame() One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.

import { HandGestures } from '@openav/leap'; packages/leap/gesture.js, line 39

handsOf()#

function@openav/leap
handsOf(msg)

Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].

This is the hand shape worlds draw from (leap.hands). A raw frame carries the fingers inline; a v6 frame carries them as pointables joined to their hand by handId, and handsOf() does that join. Both give the same hand, so code written against leap.hands does not care which URL the bridge was reached on.

Parameters

NameTypeDescription
msgObjectA parsed message: a raw frame (t: 'f') or a v6 frame (with hands and pointables).

Returns

Array The hands, each { id, side: 'left' | 'right', confidence, pinch, grab, palm, velocity, normal, direction, width, fingers, extended, elbow, wrist, timeVisible }. Positions are in millimetres; fingers is 5 arrays of joints (the last is the tip), extended 5 booleans.

Examples

js
import { mockFrame, rawToV6, handsOf } from '@openav/leap';

const raw = mockFrame(2.0);
const fromRaw = handsOf(raw)[0];
const fromV6 = handsOf(rawToV6(raw))[0];                 // the same frame as LeapJS would send it
console.log(fromRaw.side, fromRaw.palm, fromRaw.pinch);   // right [43.6, 233.2, 53.6] 1
console.log(fromV6.side, fromV6.palm, fromV6.pinch);      // the same hand
console.log(handsOf(null), handsOf({}));                  // [] []
module Both formats give the same hand: the console prints right [ 43.6, 233.2, 53.6 ] 1 twice, then [] [].

Related

  • handBones() The bones to draw for a hand, as pairs of points: each finger's joints in turn, the line across the knuckles, and the forearm.
  • rawToV6() One-off conversion of a raw frame to a LeapJS v6 frame, with a fresh converter each time (no history, so no velocities).
  • mockFrame() One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.

import { handsOf } from '@openav/leap'; packages/leap/frame.js, line 211

Hysteresis#

class@openav/leap
new Hysteresis(on, off)

One Schmitt trigger: update(v) says 'start' when the value rises above on, and 'end' only when it falls below off.

The gap between on and off is what stops a value that hovers near one threshold from flickering. Each call to update() returns 'start' (it just became active), 'end' (it just stopped) or null (nothing changed). A missing or NaN value while active counts as an end, so a lost sensor lets go. active tells you whether it is on now.

Parameters

NameTypeDescription
onNumberThe level the value must rise above to start.
offNumberThe level the value must fall below to end. It must be lower than on, or the constructor throws "Hysteresis: off must be below on".

Members

NameDescription
update(v)methodupdate(v) returns 'start', 'end' or null.
release()methodForce the end, for example when the hand is lost. Returns 'end' only if it was active, otherwise null.

Examples

js
import { Hysteresis } from '@openav/leap';

const trigger = new Hysteresis(0.8, 0.4);
const values = [0.5, 0.9, 0.6, 0.3, 0.5];
console.log(values.map((v) => trigger.update(v)));   // [ null, 'start', null, 'end', null ]
module The console prints [ null, 'start', null, 'end', null ]: 0.6 is below 0.8 but above 0.4, so it is still held.

Related

  • HandGestures Turns pinch and grab strengths into clean start and end events, per hand plus a both-hands pinch, using hysteresis so a wobbling value fires once.
  • THRESHOLDS The default hysteresis levels for the gestures: pinch 0.86 / 0.5, both-hands pinch 0.8 / 0.5, grab 0.8 / 0.5 (start above on, end below off).

import { Hysteresis } from '@openav/leap'; packages/leap/gesture.js, line 20

INTERACTION_BOX#

constant@openav/leap
INTERACTION_BOX = object, 2 keys

The classic Leap interaction box, { center: [0, 200, 0], size: [235.247, 235.247, 147.751] } in millimetres, that positions are normalised against.

A frozen object: the centre sits 200 mm above the sensor, and the box measures 235.247 mm along x, 235.247 mm along y and 147.751 mm along z (the sensor's own axes: x across, y up, z toward the player). normalizePoint() maps it to 0..1, and the console panel draws it as an outline so you can see when a hand leaves it. Pass your own box to new LeapInput({ box }) to change where 0 and 1 are.

Related

  • normalizePoint() Maps a point in millimetres to 0..1 inside the interaction box, clamped by default: the same normalisation the leap/* signals use.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • mountLeapPanel() Mounts a small panel that shows whether the Leap is alive (bridge, service, device, hands) and draws what it sees, with a button to simulate hands.

import { INTERACTION_BOX } from '@openav/leap'; packages/leap/frame.js, line 24

LeapInput#

class@openav/leap
new LeapInput({ signals = null, url = DEFAULT_URL, reconnect = true, fingers = true, box = INTERACTION_BOX, thresholds = THRESHOLDS, WebSocket: WS = null } = {})

Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.

Browsers cannot talk to the Leap Motion sensor directly, so a small program on the same machine, the bridge (startBridge(), ws://127.0.0.1:6437), reads it and serves the hands over a WebSocket. LeapInput connects to that bridge and does one job: it turns hands into normalised signals and never touches params or worlds. In a show, createShow({ modules: { leap: true } }) builds one and returns it as show.leap; the console's L1 panel shows which link is missing.

It publishes, for each side (left, right): leap/hand/<side>/present (0 or 1), x, y, z (the palm in the interaction box, 0..1, y up, z toward the player), pinch and grab (0..1), roll, pitch, yaw (−π..π radians), speed (0..1, 1 = 1 m/s), and <finger>/x|y|z for the five fingertips. Shared: leap/status (0 no bridge, 1 bridge but no tracking service, 2 service but no device, 3 tracking), leap/hands (0..2) and leap/both/distance (palm to palm, 1 = 40 cm). The full list with its ranges is what leapSignals() returns.

Gestures are pulses with a payload { side, x, y, z, strength }: leap/hand/<side>/pinch-start|pinch-end and grab-start|grab-end, and leap/both/pinch-start|pinch-end (payload { x, y, z, distance } at the midpoint). They use hysteresis, so a pinch hovering near the threshold fires once and not sixty times a second (see HandGestures). A hand that leaves the view lets go: its present, pinch, grab and speed go to 0 and its gestures end.

For continuous control, map a signal to a param ({ source: 'leap/hand/right/y', target: 'energy' }) so the timeline, a knob and a hand are interchangeable. Use leap.hands (joints in millimetres) only to draw the hand itself.

Before you call it

For real hands: the bridge must be running (node packages/leap/bridge/leap-bridge.mjs, or --mock for animated made-up hands) and you must call connect(). Nothing is published until the first frame arrives. No sensor and no bridge: feed it hands with ingest(), or call simulate() to turn the mouse into hands.

Parameters

NameTypeDescription
signalsSignalsThe Signals registry to publish into. Without one the input still works (hands, onFrame) but publishes nothing. optional default null
urlStringThe bridge address: the LeapJS protocol (the default, DEFAULT_URL) or …/raw for the raw line format. Any service that still speaks the LeapJS v6 protocol works too. optional default ws://127.0.0.1:6437/v6.json
reconnectBooleanWhether to try again after the connection drops, waiting 0.5 s, then 1 s, 2 s … up to 5 s between tries. optional default true
fingersBooleanPublish the fingertip signals (3 per finger, 30 across two hands). false leaves them out and keeps the signal list short. optional default true
boxObjectThe interaction box { center, size } in millimetres that positions are normalised against. The default is INTERACTION_BOX. optional default INTERACTION_BOX
thresholdsObjectOverrides for the gesture hysteresis, such as { pinch: { on: 0.9, off: 0.6 } } (see THRESHOLDS). Keys you leave out keep their defaults. optional default THRESHOLDS
WebSocketFunctionThe WebSocket constructor to use. Pass your own to test without a network. optional default globalThis.WebSocket

Members

NameDescription
connect()methodOpen the bridge connection; it keeps trying (with a growing delay) until disconnect(). Returns the input.
disconnect()methodClose the connection and stop retrying. The status becomes 'closed' and every hand is let go.
connectedgetterWhether the socket to the bridge is open right now.
hand(side)methodhand('left') or hand('right'): that side's hand object, or null when it is not in view.
normalize(p, clamp = true)methodA point in millimetres to 0..1 inside this input's interaction box (clamped unless clamp is false).
onFrame(cb)methodCall cb(message, hands) for every frame with hands data. Returns a function that unsubscribes.
onStatus(cb)methodCall cb(status, leap) whenever the status changes. Returns a function that unsubscribes.
ingest(msg)methodFeed one parsed message, exactly what the socket would deliver: a v6 header, a v6 frame, a device event, or a raw status or frame line. Public so that tests, recordings and the simulator take the very path real frames take.
simulate({ target = typeof window !== 'undefined' ? window : null, hz = 60 } = {})methodNo sensor? Mouse and touch become hands: move = right palm (x across, y up the screen), wheel = depth, press = pinch, right button or Alt+press = grab, hold Shift to drive the left hand, a second finger on a touch screen = left hand. { target = window, hz = 60 }. Returns stop(). Throws when there is no target to listen on.
simulatinggetterWhether the simulator is running.
stopSimulation()methodStop the simulator. The bridge's hands come back if it is connected.
project(p, w, h, { view = 'oblique', box = this.box } = {})methodA point in millimetres to canvas pixels: project(p, w, h, { view, box }). view is 'oblique' (default: the performer's eye, up the canvas = higher and farther, so a flat hand shows its fingers), 'front' (x across, y up) or 'top' (z down the canvas). Returns [x, y].
skeleton(ctx, w, h, { view = 'oblique', box = this.box, colorLeft = 'rgba(55,201,120,0.95)', colorRight = 'rgba(255,209,102,0.95)', lineWidth = 2, hands = this.hands } = {})methodDraw the hands onto a 2D canvas context: skeleton(ctx, w, h, { view, box, colorLeft, colorRight, lineWidth, hands }). The palm and fingertips are dots and a pinch gets a ring. Points outside the box still draw.
mountPanel(el, opts)methodMount the status and skeleton panel into an element; the same as mountLeapPanel(el, leap, opts).

Notes

  • Fields to read: status ('idle', 'connecting', 'bridge', 'no-service', 'no-device', 'tracking', 'simulated', 'closed'), hands (an array; each hand has id, side, palm, fingers (5 × 5 joints), pinch, grab, velocity, normal, direction … in millimetres), frame (the latest message as it came), frames, fps, device, serviceVersion, gestures and box.
  • While the simulator runs it owns the hands: real frames from the bridge are ignored, and what the bridge says is remembered and restored when the simulator stops.
  • The signals are normalised (y = 1 is up). leap.hands and the LeapJS frames keep the sensor's own millimetre units, with +z toward the player.
  • How the bridge talks, the formats and the troubleshooting table are in the package README.

Examples

js
import { Signals } from '@openav/core';
import { LeapInput, mockFrame } from '@openav/leap';

const signals = new Signals();
const leap = new LeapInput({ signals });
leap.ingest(mockFrame(2.0));                       // a made-up frame: the same path real frames take
console.log(leap.status, leap.hands.length);       // tracking 2
console.log(signals.get('leap/hand/right/pinch')); // 1
leap.skeleton(document.getElementById('c').getContext('2d'), 400, 300);
module The console prints tracking 2 and 1 (the mock right hand is pinching at second 2), and both hands are drawn on the canvas.
js
import { Signals } from '@openav/core';
import { LeapInput, mockFrame } from '@openav/leap';

const signals = new Signals();
const leap = new LeapInput({ signals });
signals.onAny((name, value, meta) => {
  if (meta.kind === 'pulse') console.log(name, value.strength.toFixed(2));
});
for (let i = 0; i <= 360; i++) leap.ingest(mockFrame(i / 60));   // six seconds of mock hands at 60 frames a second
console.log(signals.get('leap/hands'), signals.get('leap/status'));
module Six seconds of mock hands, with the pulses listed as they fire: the right hand pinches twice (leap/hand/right/pinch-start 0.86, then pinch-end 0.50, again) and the left fist closes once (leap/hand/left/grab-start 0.80). It ends with 2 3 (two hands, status 3 = tracking).
js
import { Signals } from '@openav/core';
import { LeapInput } from '@openav/leap';

const signals = new Signals();
const leap = new LeapInput({ signals });
const ctx = document.getElementById('c').getContext('2d');

leap.simulate({ target: window });                  // mouse and touch become hands
leap.onFrame(() => {
  ctx.clearRect(0, 0, 400, 300);
  leap.skeleton(ctx, 400, 300);
  document.getElementById('app').textContent = 'y ' + signals.get('leap/hand/right/y').toFixed(2) + '  pinch ' + signals.get('leap/hand/right/pinch').toFixed(2);
});
module Move the mouse over the page: a right hand follows it, and the text shows its height (y). Press the button to pinch, right-click to make a fist.
js
import { Signals } from '@openav/core';
import { LeapInput } from '@openav/leap';

const signals = new Signals();
const leap = new LeapInput({ signals }).connect();       // ws://127.0.0.1:6437/v6.json
leap.onStatus((status) => console.log('leap', status));  // connecting, bridge, tracking …
signals.on('leap/hand/right/pinch-start', ({ x, y }) => console.log('pinch at', x, y));
static With the bridge running and a hand over the sensor, the console follows the link (leap connecting, leap bridge, leap tracking) and prints a line for every pinch.

To be confirmed

Not tried with a real Leap Motion sensor here: the examples feed made-up hands (mockFrame) through the same ingest() path.

On a public https page, whether Chrome asks for permission before the page may reach 127.0.0.1 (the package README says it needs the "loopback-network" permission) was not tried here.

Related

  • leapSignals() The list of every signal LeapInput publishes, with its range, as [[name, meta], …], ready to hand to signals.define().
  • mockFrame() One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
  • HandGestures Turns pinch and grab strengths into clean start and end events, per hand plus a both-hands pinch, using hysteresis so a wobbling value fires once.
  • mountLeapPanel() Mounts a small panel that shows whether the Leap is alive (bridge, service, device, hands) and draws what it sees, with a button to simulate hands.
  • normalizePoint() Maps a point in millimetres to 0..1 inside the interaction box, clamped by default: the same normalisation the leap/* signals use.
  • handsOf() Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { LeapInput } from '@openav/leap'; packages/leap/index.js, line 64

leapSignals()#

function@openav/leap
leapSignals({ fingers = true } = {})

The list of every signal LeapInput publishes, with its range, as [[name, meta], …], ready to hand to signals.define().

new LeapInput({ signals }) already declares all of these, so you rarely call this yourself. It is useful when you want the list: to build a signal picker, to check that a route's source exists, or to declare the signals before the sensor is connected. The names it returns are the vocabulary of the Signals reference for Leap.

Parameters

NameTypeDescription
fingersBooleanInclude the fingertip signals. false leaves out the 30 of them. optional default true

Returns

Array The pairs [name, meta]: 63 with fingers, 33 without. Each meta has source: 'leap', and min and max (the roll, pitch and yaw ones in radians, −π..π) or kind: 'pulse'.

Examples

js
import { Signals } from '@openav/core';
import { leapSignals } from '@openav/leap';

console.log(leapSignals().length, leapSignals({ fingers: false }).length);   // 63 33
const signals = new Signals();
for (const [name, meta] of leapSignals({ fingers: false })) signals.define(name, meta);
console.log(signals.meta.get('leap/hand/right/roll').min);                   // -3.14159…
console.log(signals.meta.get('leap/hand/left/pinch-start').kind);            // pulse
module The console prints 63 33, then the roll range's lower end (−π) and pulse.

Related

  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • mockFrame() One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.

import { leapSignals } from '@openav/leap'; packages/leap/index.js, line 43

mockFrame()#

function@openav/leap
mockFrame(t, id = Math.round(t * 60), { hands = 2 } = {})

One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.

This is the bridge's --mock stream and the way to try Leap code without a sensor. The right hand pinches for about a second out of every 3, the left hand pinches once every 12 seconds, and at those moments (for example second 8) both are pinched, so the both-hands pinch fires too. The left fist comes and goes slowly. The output goes through exactly the same conversion and publishing as a real Leap Motion Controller.

Parameters

NameTypeDescription
tNumberTime in seconds. The result depends only on t, so it is the same every time.
idNumberThe frame number written into the frame. optional default Math.round(t * 60)
handsNumberHow many hands: 2 (right and left), 1 (the right only) or 0. optional default 2

Returns

Object A frame in the raw line format: { t: 'f', id, fps: 60, hands: [hand, …] }, the right hand (side: 'R') first. Feed it to LeapInput.ingest() or handsOf().

Examples

js
import { mockFrame } from '@openav/leap';

for (const t of [0.5, 2.0, 8.0]) {
  const frame = mockFrame(t);
  console.log(t, frame.hands.map((h) => h.side + ' pinch ' + h.pinch).join(' · '));
}
module The console prints 0.5 R pinch 0 · L pinch 0, 2 R pinch 1 · L pinch 0 and 8 R pinch 1 · L pinch 1 (the both-hands pinch).

Related

  • mockHand() Builds one made-up hand, with all five fingers and the arm, from a palm position plus pinch, grab, roll, yaw and pitch.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • handsOf() Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].

import { mockFrame } from '@openav/leap'; packages/leap/mock.js, line 104

mockHand()#

function@openav/leap
mockHand({ id = 1, side = 'R', palm = [0, 200, 0], pinch = 0, grab = 0, roll = 0, yaw = 0, pitch = 0, vel = [0, 0, 0], conf = 1, width = 85 } = {})

Builds one made-up hand, with all five fingers and the arm, from a palm position plus pinch, grab, roll, yaw and pitch.

It places a plausible hand in space: palm, knuckles, fingers curled by grab, thumb and index meeting as pinch goes to 1, wrist and elbow. LeapInput.simulate() builds its hands from the mouse with it, and mockFrame() with the animated ones. Use it to put a hand exactly where a test needs it.

Parameters

NameTypeDescription
idNumberThe hand's id. optional default 1
side'L' | 'R''R' for a right hand, 'L' for a left. optional default 'R'
palmArray<Number>The palm position [x, y, z] in millimetres (+x right, +y up, +z toward the player). optional default [0, 200, 0]
pinchNumberThumb to index, 0..1. At 1 the two fingertips meet. optional default 0
grabNumberHow closed the fist is, 0..1. optional default 0
rollNumberRadians; positive rolls toward the thumb side. optional default 0
yawNumberRadians; positive turns the fingers toward +x. optional default 0
pitchNumberRadians; positive tips the fingers up. optional default 0
velArray<Number>Palm velocity in millimetres per second. optional default [0, 0, 0]
confNumberTracking confidence, 0..1. optional default 1
widthNumberPalm width in millimetres. optional default 85

Returns

Object A hand in the raw line format: { id, side, conf, pinch, grab, palm, vel, n, dir, w, f, ext, arm }, where f is 5 fingers × 5 joints (thumb to pinky; the last joint of each is the tip). Wrap hands in { t: 'f', id, fps, hands: [...] } to make a frame.

Examples

js
import { mockHand } from '@openav/leap';

const open = mockHand({ palm: [0, 200, 0] });
const pinched = mockHand({ palm: [0, 200, 0], pinch: 1 });
const gap = (hand) => Math.hypot(...hand.f[0][4].map((v, i) => v - hand.f[1][4][i])).toFixed(0);
console.log('thumb to index tip, open:', gap(open), 'mm  pinched:', gap(pinched), 'mm');
module The console measures the distance between the thumb tip and the index tip: 100 mm for the open hand and 0 mm when pinched.

Related

  • mockFrame() One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • handsOf() Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
  • handBones() The bones to draw for a hand, as pairs of points: each finger's joints in turn, the line across the knuckles, and the forearm.

import { mockHand } from '@openav/leap'; packages/leap/mock.js, line 42

mountLeapPanel()#

function@openav/leap
mountLeapPanel(el, leap, { width = 300, simulate = true, simTarget } = {})

Mounts a small panel that shows whether the Leap is alive (bridge, service, device, hands) and draws what it sees, with a button to simulate hands.

One status line says which link is the broken one, in order: no bridge, bridge but no tracking service, service but no device, tracking (with the number of hands and the frame rate). Under it, a small front view draws the interaction box (a hand outside it is visibly outside, its signals sitting clamped at the edge) and the hand skeletons. It redraws only on frames and status changes, so nothing runs while the sensor is idle.

In a show you do not call it: the console's L1 · Input panel mounts it when you declare modules: { leap: true }.

Parameters

NameTypeDescription
elHTMLElementWhere to put the panel.
leapLeapInputThe LeapInput to show.
widthNumberThe width of the drawing in CSS pixels; the height follows (about 0.62 of the width). optional default 300
simulateBooleanShow the "simulate" button (mouse and touch become hands). optional default true
simTargetHTMLElement | WindowWhat the simulator listens on. Defaults to leap.simTarget, then the window. optional

Returns

Object { el, render(), destroy() }: the panel's root element, a function that redraws it, and one that removes it and unsubscribes.

Examples

js
import { LeapInput, mockFrame, mountLeapPanel } from '@openav/leap';

const leap = new LeapInput();
const panel = mountLeapPanel(document.getElementById('app'), leap, { width: 360 });
leap.ingest(mockFrame(2.0));          // a made-up frame stands in for the sensor
panel.render();
module A panel appears in the page: a green dot, the status line tracking · 2 hands · 60 fps · R:pinch (the right hand is pinching), a "simulate" button, and the two mock hands drawn in a small box.

Related

  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • mountConsole() Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.

import { mountLeapPanel } from '@openav/leap'; packages/leap/panel.js, line 28

normalizePoint()#

function@openav/leap
normalizePoint(p, box = INTERACTION_BOX, clamp = true)

Maps a point in millimetres to 0..1 inside the interaction box, clamped by default: the same normalisation the leap/* signals use.

The classic box is centred 200 mm above the sensor and about 235 × 235 × 148 mm (INTERACTION_BOX). A palm at the centre reads [0.5, 0.5, 0.5]; a palm far outside it reads 0 or 1 on that axis, which is why the signals "sit clamped at the edge" when a hand leaves the box. Pass clamp = false when you want to know how far outside it is.

Parameters

NameTypeDescription
pArray<Number>The point [x, y, z] in millimetres.
boxObjectThe box { center, size } in millimetres. optional default INTERACTION_BOX
clampBooleanHold each value to 0..1. false lets a point outside the box go beyond. optional default true

Returns

Array<Number> [x, y, z], each (p − center) / size + 0.5: 0.5 is the centre of the box, x grows to the right, y upward, z toward the player.

Examples

js
import { normalizePoint } from '@openav/leap';

console.log(normalizePoint([0, 200, 0]));                          // [ 0.5, 0.5, 0.5 ]: the centre
console.log(normalizePoint([500, 0, -500]));                       // [ 1, 0, 0 ]: far outside, clamped
console.log(normalizePoint([100, 320, 0]).map((v) => +v.toFixed(2)));   // [ 0.93, 1, 0.5 ]
module The console prints [ 0.5, 0.5, 0.5 ], [ 1, 0, 0 ] and [ 0.93, 1, 0.5 ].

Related

  • INTERACTION_BOX The classic Leap interaction box, { center: [0, 200, 0], size: [235.247, 235.247, 147.751] } in millimetres, that positions are normalised against.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • handAngles() Turns a hand's direction and palm normal into { pitch, yaw, roll } in radians, with the LeapJS conventions.

import { normalizePoint } from '@openav/leap'; packages/leap/frame.js, line 78

palmBasis()#

function@openav/leap
palmBasis(direction, normal, left)

An advanced helper: the orthonormal basis [x, y, z] of the palm, from its direction and normal, in the rows LeapJS reads as hand.r.

Like boneBasis(), it exists so that the LeapJS frames V6Converter writes have a r that hand.rotationAngle() can use. Worlds that draw hands read leap.hands and handAngles() instead.

Parameters

NameTypeDescription
directionArray<Number>The direction the fingers point.
normalArray<Number>The palm normal.
leftBooleantrue for a left hand: x is negated.

Returns

Array Three rows [x, y, z] (x sideways, y the back of the hand, z toward the wrist), rounded to 3 decimals.

Examples

js
import { palmBasis } from '@openav/leap';

console.log(palmBasis([0, 0, -1], [0, -1, 0], false));   // flat hand, palm down, fingers forward: the identity
module The console prints the identity [ [ 1, 0, 0 ], [ 0, 1, 0 ], [ 0, 0, 1 ] ].

Related

  • boneBasis() An advanced helper: the orthonormal basis [x, y, z] of a bone, rebuilt from two joints and the palm normal the way LeapJS expects.
  • handAngles() Turns a hand's direction and palm normal into { pitch, yaw, roll } in radians, with the LeapJS conventions.
  • V6Converter Converts raw hand lines into LeapJS v6 frames, keeping the little state the protocol needs: time visible, smoothed positions and fingertip velocities.

import { palmBasis } from '@openav/leap'; packages/leap/frame.js, line 60

PROTOCOL_VERSION#

constant@openav/leap
PROTOCOL_VERSION = 6

The highest LeapJS protocol version we speak: 6. LeapJS 1.1.1 and earlier throw "unrecognized version" above that.

v6Header() never reports more than this, and the bridge answers /v7.json as version 6. It is the default of the version argument there.

Related

  • v6Header() The first message of a LeapJS v6 connection: { serviceVersion, version }, with version held to 1..6 because LeapJS throws above 6.
  • protocolOfPath() Which protocol version a /vN.json path asks for, as a number, or null when the path is not one.
  • DEFAULT_URL The default address LeapInput connects to: ws://127.0.0.1:6437/v6.json, the bridge on this machine speaking the LeapJS protocol.

import { PROTOCOL_VERSION } from '@openav/leap'; packages/leap/frame.js, line 27

protocolOfPath()#

function@openav/leap
protocolOfPath(pathname)

Which protocol version a /vN.json path asks for, as a number, or null when the path is not one.

The bridge uses it to decide whether a WebSocket upgrade is a LeapJS client at all: /v6.json yes, /raw is handled separately, anything else is refused with a 404.

Parameters

NameTypeDescription
pathnameStringThe URL path, such as '/v6.json'.

Returns

Number | null The number in /v<number>.json, or null. It does not clip to 6: /v7.json gives 7 (the bridge then answers it as version 6).

Examples

js
import { protocolOfPath } from '@openav/leap';

console.log(protocolOfPath('/v6.json'), protocolOfPath('/v3.json'), protocolOfPath('/v7.json'));   // 6 3 7
console.log(protocolOfPath('/raw'), protocolOfPath('/v6.jsonx'), protocolOfPath(undefined));        // null null null
module The console prints 6 3 7, then null null null.

Related

  • v6Header() The first message of a LeapJS v6 connection: { serviceVersion, version }, with version held to 1..6 because LeapJS throws above 6.
  • PROTOCOL_VERSION The highest LeapJS protocol version we speak: 6. LeapJS 1.1.1 and earlier throw "unrecognized version" above that.

import { protocolOfPath } from '@openav/leap'; packages/leap/frame.js, line 202

rawToV6()#

function@openav/leap
rawToV6(raw, tUs, opts)

One-off conversion of a raw frame to a LeapJS v6 frame, with a fresh converter each time (no history, so no velocities).

Handy for tests and for looking at what a legacy sketch would receive. For a stream, keep one V6Converter instead, so that timeVisible and tipVelocity can accumulate.

Parameters

NameTypeDescription
rawObjectA raw frame { t: 'f', id, fps, hands }.
tUsNumberThe time in microseconds. Defaults to now. optional
optsObjectOptions for the V6Converter: { box, smoothing }. optional

Returns

Object The v6 frame. Every field the LeapJS Frame, Hand, Pointable, Finger and Bone constructors read is there.

Examples

js
import { mockFrame, rawToV6 } from '@openav/leap';

const frame = rawToV6(mockFrame(2.0, 120), 5_000_000);
console.log(Object.keys(frame).join(' '));
console.log(frame.hands.length, frame.pointables.length, frame.hands[0].type, frame.hands[0].pinchStrength);
module The console prints the frame's keys (id timestamp currentFrameRate hands pointables interactionBox gestures r s t), then 2 10 right 1.

Related

  • V6Converter Converts raw hand lines into LeapJS v6 frames, keeping the little state the protocol needs: time visible, smoothed positions and fingertip velocities.
  • handsOf() Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].
  • v6Header() The first message of a LeapJS v6 connection: { serviceVersion, version }, with version held to 1..6 because LeapJS throws above 6.
  • mockFrame() One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.

import { rawToV6 } from '@openav/leap'; packages/leap/frame.js, line 185

RESERVED_PORTS#

constant@openav/leap
RESERVED_PORTS = array, 1 items

The ports the Leap bridge refuses to use: [7457], which belongs to The Last Input and the backstage monitor.

startBridge({ port: 7457 }) throws "port 7457 belongs to The Last Input" immediately, so the bridge can never take the port the monitor listens on (node packages/monitor/server.js). The out-bridge refuses the same one.

Related

  • MonitorFeed Streams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.

import { RESERVED_PORTS } from '@openav/leap/bridge'; packages/leap/bridge/leap-bridge.mjs, line 31

startBridge()#

function@openav/leap
startBridge({ port = DEFAULT_PORT, host = '127.0.0.1', mock = false, hz = 60, bin = null, log = () => {} } = {})

Starts the Leap bridge on this machine: a loopback WebSocket and HTTP server that serves the sensor's hands to every page (Node only).

This is what node packages/leap/bridge/leap-bridge.mjs runs (add --mock for made-up hands, --port, --hz, --bin, --quiet). It serves ws://127.0.0.1:6437/v6.json (the LeapJS protocol, which every 2014 to 2021 browser sketch speaks), ws://…/raw (the raw lines), the same lines as server-sent events on GET /raw and /hands, and GET /health (source, service, device, fps, hands, clients). Any page origin may read it, as the original Leap service allowed.

Call it from your own Node program when you want the bridge inside it, for example in a test. startBridge() throws at once for port 7457 (it belongs to The Last Input and the monitor) and for a non-loopback host; a port already in use rejects the returned promise with a message that suggests curl http://127.0.0.1:6437/health.

Before you call it

Node 22 or later, and for real hands the Ultraleap Hand Tracking service plus Xcode's clang to build the reader on first start. With mock: true neither is needed.

Parameters

NameTypeDescription
portNumberThe port to listen on, DEFAULT_PORT unless you say otherwise. 0 picks any free one. The ports in RESERVED_PORTS (7457) are refused. optional default DEFAULT_PORT
hostStringThe address to bind. Only loopback (127.0.0.1, ::1, localhost) is allowed: hand data never leaves the machine. optional default '127.0.0.1'
mockBooleantrue serves two animated made-up hands (mockFrame()) instead of reading a sensor. optional default false
hzNumberThe most frames per second sent to each client (held to 1..120). A slow client skips frames; it never queues them. optional default 60
binString | nullThe path to the leap-stream reader program. By default the bridge compiles its C reader on first start (needs the Xcode command line tools) and caches it. optional default null
logFunctionCalled with a line of text for each event worth knowing (clients coming and going, status changes). optional default () => {}

Returns

Promise<Object> Resolves once listening, with { port, state, health(), close() }: the real port (useful with port: 0), the live state, a function returning the /health object, and a function that stops everything and returns a Promise.

Examples

js
import { startBridge } from '@openav/leap/bridge';

const bridge = await startBridge({ mock: true, port: 0 });          // made-up hands, any free port
const socket = new WebSocket('ws://127.0.0.1:' + bridge.port + '/v6.json');
socket.onmessage = async ({ data }) => {
  const message = JSON.parse(data);
  if (message.serviceVersion) console.log('handshake:', message);    // { serviceVersion: 'mock', version: 6, bridge: 'open-audiovisual' }
  if (message.hands) {                                               // a frame
    console.log('hands in the frame:', message.hands.length);        // 2
    socket.close();
    await bridge.close();
  }
};
static In Node: the first message is the handshake, handshake: { serviceVersion: 'mock', version: 6, bridge: 'open-audiovisual' }, then a frame arrives and the program prints hands in the frame: 2 and stops the bridge.

Related

  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • mockFrame() One frame of made-up, animated hands at time t: a right hand circling and pinching every 3 seconds, a left hand rising, falling and slowly making a fist.
  • V6Converter Converts raw hand lines into LeapJS v6 frames, keeping the little state the protocol needs: time visible, smoothed positions and fingertip velocities.
  • DEFAULT_URL The default address LeapInput connects to: ws://127.0.0.1:6437/v6.json, the bridge on this machine speaking the LeapJS protocol.

import { startBridge } from '@openav/leap/bridge'; packages/leap/bridge/leap-bridge.mjs, line 45

STATUS#

constant@openav/leap
STATUS = object, 4 keys

The four levels of the leap/status signal by name: off 0, bridge 1, service 2, tracking 3.

A frozen object { off: 0, bridge: 1, service: 2, tracking: 3 }: 0 no bridge, 1 the bridge is there but the tracking service is not, 2 the service is there but no device, 3 tracking. Use it to read the signal without magic numbers: signals.get('leap/status') === STATUS.tracking. Note it is the number published as leap/status; leap.status is a different thing, a string ('tracking', 'no-device' …), and the simulator counts as 3.

Notes

  • The code in LeapInput writes the numbers itself and does not read this object, so STATUS is a convenience for your own code. The strings map as: tracking and simulated to 3, no-device to 2, bridge and no-service to 1, the rest to 0.

Related

  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • deviceEvent() A LeapJS device event telling the client whether a device is attached and streaming, plus our own service flag.
  • mountLeapPanel() Mounts a small panel that shows whether the Leap is alive (bridge, service, device, hands) and draws what it sees, with a button to simulate hands.

import { STATUS } from '@openav/leap'; packages/leap/index.js, line 37

THRESHOLDS#

constant@openav/leap
THRESHOLDS = object, 3 keys

The default hysteresis levels for the gestures: pinch 0.86 / 0.5, both-hands pinch 0.8 / 0.5, grab 0.8 / 0.5 (start above on, end below off).

A frozen object { pinch: { on, off }, bothPinch: { on, off }, grab: { on, off } }. The numbers come from The Last Input, tuned on a Leap Motion Controller 2, so one bridge means one feel. Change them per input with new LeapInput({ thresholds: { pinch: { on: 0.9, off: 0.6 } } }) or new HandGestures(...); you cannot change this object itself.

Related

  • HandGestures Turns pinch and grab strengths into clean start and end events, per hand plus a both-hands pinch, using hysteresis so a wobbling value fires once.
  • Hysteresis One Schmitt trigger: update(v) says 'start' when the value rises above on, and 'end' only when it falls below off.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.

import { THRESHOLDS } from '@openav/leap'; packages/leap/gesture.js, line 13

V6Converter#

class@openav/leap
new V6Converter({ box = INTERACTION_BOX, smoothing = 0.5 } = {})

Converts raw hand lines into LeapJS v6 frames, keeping the little state the protocol needs: time visible, smoothed positions and fingertip velocities.

Every browser sketch written for the old Leap service (LeapJS, 2014 to 2021) speaks the v6 protocol, so the bridge serves it on /v6.json. This class does the conversion: one converter per stream. It is stateful where the protocol is: a hand's timeVisible grows while it stays in view and starts over when it leaves, stabilized… positions are a light low-pass, and tipVelocity is a finite difference between two frames.

LeapC gives joints, not bone rotations, so the bone bases are rebuilt from the joints and the palm normal (boneBasis()). Gestures are not synthesised: gestures is always [].

Parameters

NameTypeDescription
boxObjectThe interaction box written into every frame's interactionBox. optional default INTERACTION_BOX
smoothingNumber0..1: how strongly stabilizedPalmPosition and stabilizedTipPosition lag behind (0 follows exactly, close to 1 barely moves). optional default 0.5

Members

NameDescription
convert(raw, tUs = nowUs())methodconvert(raw, tUs = now): a raw frame { t: 'f', id, fps, hands } and its time in microseconds, to a v6 frame { id, timestamp, currentFrameRate, hands, pointables, interactionBox, gestures: [], r, s, t }. Pointable ids are handId × 10 + finger.

Examples

js
import { V6Converter, mockFrame } from '@openav/leap';

const converter = new V6Converter();
converter.convert(mockFrame(1.0), 1_000_000);                    // microseconds
const frame = converter.convert(mockFrame(1.1), 1_100_000);
console.log(frame.hands[0].timeVisible);                          // 0.1: seen for a tenth of a second
console.log(frame.pointables.length, frame.pointables[1].tipVelocity);   // 10 [ -21, 17, 28 ]  mm per second
module The console prints 0.1, then 10 pointables and the index fingertip's velocity [ -21, 17, 28 ] (millimetres per second between the two frames).

Related

  • rawToV6() One-off conversion of a raw frame to a LeapJS v6 frame, with a fresh converter each time (no history, so no velocities).
  • v6Header() The first message of a LeapJS v6 connection: { serviceVersion, version }, with version held to 1..6 because LeapJS throws above 6.
  • deviceEvent() A LeapJS device event telling the client whether a device is attached and streaming, plus our own service flag.
  • handsOf() Reads either wire format (raw line or LeapJS v6 frame) into the one hand shape the rest of the package uses; anything else gives [].

import { V6Converter } from '@openav/leap'; packages/leap/frame.js, line 91

v6Header()#

function@openav/leap
v6Header(serviceVersion = 'unknown', version = PROTOCOL_VERSION)

The first message of a LeapJS v6 connection: { serviceVersion, version }, with version held to 1..6 because LeapJS throws above 6.

The bridge sends this (plus bridge: 'open-audiovisual') as soon as a client connects to /v6.json, and LeapInput turns it into the status 'bridge'. Ask for 7 and you get 6.

Parameters

NameTypeDescription
serviceVersionStringThe tracking service version to report, such as '5.20.0'. optional default 'unknown'
versionNumberThe protocol version the client asked for. Held to 1..6. optional default PROTOCOL_VERSION

Returns

Object { serviceVersion: String, version: Number }.

Examples

js
import { v6Header } from '@openav/leap';

console.log(v6Header('5.20.0'));        // { serviceVersion: '5.20.0', version: 6 }
console.log(v6Header('x', 3));          // { serviceVersion: 'x', version: 3 }
console.log(v6Header('x', 9).version);  // 6: never above what LeapJS accepts
module The console prints the header for version 6, version 3, and 6 for a request of 9.

Related

  • deviceEvent() A LeapJS device event telling the client whether a device is attached and streaming, plus our own service flag.
  • PROTOCOL_VERSION The highest LeapJS protocol version we speak: 6. LeapJS 1.1.1 and earlier throw "unrecognized version" above that.
  • protocolOfPath() Which protocol version a /vN.json path asks for, as a number, or null when the path is not one.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.

import { v6Header } from '@openav/leap'; packages/leap/frame.js, line 188

Body and hand pose (@openav/pose)#

Camera in, landmarks and pinches out. 2

@openav/pose

HandTracker#

class@openav/pose
new HandTracker({ signals = null, mirror = true } = {})

Tracks up to two hands from the camera with 21 landmarks each and publishes palm position and thumb-to-finger pinch distances as hand/* signals.

The fine-control tracker: each hand gives two precise continuous controllers, thumb to index and thumb to middle finger, so two hands are four controllers you can pinch. The distances are normalised by the size of your palm (wrist to the base of the middle finger), so moving toward or away from the camera does not change a pinch: stage-robust. It runs on the device with MediaPipe Tasks Vision and downloads nothing until enable(). In a show, createShow({ modules: { hands: true } }) builds one (show.hands) and the L1 panel gets a 🖐 hands button.

For each of left and right it publishes hand/<side>/present (0 or 1), hand/<side>/x and /y (the palm, 0..1; y flipped so 1 is raised), hand/<side>/pinch/index and hand/<side>/pinch/middle (0..1, 0 = touching, 1 = wide open) and hand/<side>/spread (index fingertip to little fingertip, 0..1). A side with no hand in view gets present 0 and keeps its other values. Note that these names do not start with pose/, and are different from the body tracker's.

A pinch is a distance, so for "pinch closes" use a route with invert: { source: 'hand/right/pinch/index', target: 'tightness', invert: true, smooth: 0.1 }. If you want pinch start and end as events, LeapInput's gestures do that; HandTracker only gives the continuous distances.

Before you call it

Call await hands.enable(): it loads the MediaPipe library from jsDelivr and the hand model from Google's model storage (so it needs the network), asks for the camera (https or localhost, and the user must allow it) and starts the loop. Until then the signals sit at their starting values.

Parameters

NameTypeDescription
signalsSignals | nullThe Signals registry to publish into. Without one the tracker still runs and keeps hands, but publishes nothing. optional default null
mirrorBooleanTreat the camera as a mirror: x is flipped, and the left and right labels MediaPipe gives are swapped so the signals match the body. optional default true

Members

NameDescription
enable({ videoEl = null } = {})methodenable({ videoEl = null } = {}): ask for the camera (640 × 360), load the model (up to two hands) and start detecting. videoEl is a <video> to use; without one an off-screen video is created. Resolves true; rejects if the camera is refused or the library or model cannot load.
stop()methodStop detecting and release the camera. Signals keep their last values.
skeleton(ctx, w, h, { colorLeft = 'rgba(55,201,120,0.9)', colorRight = 'rgba(255,209,102,0.9)', lineWidth = 2 } = {})methodDraw the hands onto a 2D canvas context: skeleton(ctx, w, h, { colorLeft, colorRight, lineWidth = 2 }), the 21 landmarks joined into finger bones, with bigger dots at the thumb, index and middle fingertips (the pinch control points). Left is green and right is yellow by default.

Notes

  • Fields to read: hands ([{ handed: 'left' | 'right', lm }], lm the 21 landmarks { x, y, z } in the camera image, y downward and not mirrored), running, mirror, video.
  • Landmark indexes used: wrist 0, thumb tip 4, index tip 8, middle tip 12, little finger tip 20, base of the middle finger 9 (the palm position and the size reference).

Examples

js
import { Signals } from '@openav/core';
import { HandTracker } from '@openav/pose';

const signals = new Signals();
const hands = new HandTracker({ signals });
document.addEventListener('click', () => hands.enable(), { once: true });   // the camera prompt, then the model
signals.on('hand/right/pinch/index', (distance) => {
  if (distance < 0.1) console.log('right hand pinches thumb and index');
});
static Click the page and allow the camera; hold a hand up and pinch thumb to index finger: the console prints right hand pinches thumb and index while they touch.
js
import { HandTracker } from '@openav/pose';

const hands = new HandTracker();                      // never enabled: no camera, no download
const lm = Array.from({ length: 21 }, () => ({ x: 0.5, y: 0.5, z: 0 }));
lm[0] = { x: 0.5, y: 0.9, z: 0 };                     // wrist
const fingers = [[1, 0.38, 0.8, 0.28, 0.7, 0.2, 0.62, 0.14, 0.55],     // thumb: joints 1 to 4 as x, y pairs
  [5, 0.42, 0.55, 0.4, 0.42, 0.39, 0.32, 0.38, 0.22],                  // index
  [9, 0.5, 0.53, 0.5, 0.38, 0.5, 0.27, 0.5, 0.17],                     // middle
  [13, 0.58, 0.55, 0.6, 0.42, 0.61, 0.32, 0.62, 0.23],                 // ring
  [17, 0.64, 0.6, 0.67, 0.5, 0.69, 0.42, 0.7, 0.34]];                  // little
for (const [first, ...xy] of fingers) for (let i = 0; i < 4; i++) lm[first + i] = { x: xy[i * 2], y: xy[i * 2 + 1], z: 0 };
hands.hands = [{ handed: 'right', lm }];              // what enable() keeps filling in from the camera

const ctx = document.getElementById('c').getContext('2d');
ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
hands.skeleton(ctx, 400, 300, { lineWidth: 3 });
module An open right hand is drawn in yellow on the canvas from made-up landmarks, with bigger dots at the thumb, index and middle fingertips: the overlay skeleton() gives you over the camera picture.

To be confirmed

Not tried here with a live camera, so the real range of the pinch values (for example how close to 0 a firm pinch gets) is not measured.

Related

  • PoseTracker Tracks one body from the camera and publishes wrist positions, wrist speed, hand spread, height and lean as pose/* signals.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { HandTracker } from '@openav/pose'; packages/pose/index.js, line 158

PoseTracker#

class@openav/pose
new PoseTracker({ signals = null, mirror = true } = {})

Tracks one body from the camera and publishes wrist positions, wrist speed, hand spread, height and lean as pose/* signals.

This is the body tracker: one person, 33 landmarks (the BlazePose set), run by MediaPipe Tasks Vision on the device. No video leaves the machine, and nothing is downloaded until you call enable(). For fine finger control use HandTracker instead. In a show, createShow({ modules: { pose: true } }) builds one and the console's L1 panel gets a 🕺 body button that calls enable(); it is show.pose.

Signals, all normalised, with y flipped so that 1 means raised and 0 means the bottom of the frame (performance meaning beats pixel meaning): pose/present (0 or 1: someone is visible), pose/hand/left/x and /y, pose/hand/right/x and /y (wrist positions, 0..1), pose/hand/left/v and pose/hand/right/v (wrist speed, 0..1, clamped), pose/hands/spread (distance between the wrists, 0..1), pose/height (nose height, 0..1: crouching to standing) and pose/lean (shoulder-line tilt, −1..1). When nobody is in view only pose/present goes to 0; the others keep their last values.

Map these to params ({ source: 'pose/hand/right/y', target: 'energy', smooth: 0.1 }) so a raised hand does what a knob would. The detection runs on every animation frame while running.

Before you call it

Call await pose.enable(): it loads the MediaPipe library from jsDelivr and the model from Google's model storage (so it needs the network), asks the browser for the camera (the page needs https or localhost, and the user must allow it), and starts the loop. Until then every signal sits at its starting value.

Parameters

NameTypeDescription
signalsSignals | nullThe Signals registry to publish into. Without one the tracker still runs and keeps landmarks, but publishes nothing. optional default null
mirrorBooleanTreat the camera as a mirror: x is flipped, so moving toward the viewer's right makes x grow. Natural for performers. false keeps the camera's own left and right. optional default true

Members

NameDescription
enable({ videoEl = null } = {})methodenable({ videoEl = null } = {}) asks for the camera (640 × 360), loads the model and starts detecting. videoEl is a <video> element to use; without one an off-screen video is created. Resolves true; rejects if the camera is refused or the library or model cannot be loaded.
stop()methodStop detecting and release the camera. Signals keep their last values. enable() can start it again.
skeleton(ctx, w, h, { color = 'rgba(122,166,255,0.9)', lineWidth = 2 } = {})methodDraw the body onto a 2D canvas context: skeleton(ctx, w, h, { color, lineWidth = 2 }), the bones of the torso, arms and legs plus dots at the nose, shoulders, wrists and hips. It follows the same mirroring as the signals but not the y flip (it draws in screen space). Does nothing without landmarks. Call it every frame after drawing the background.

Notes

  • Fields to read: landmarks (the latest 33 raw points { x, y, z, visibility }, x and y 0..1 in the camera image, y downward and not mirrored; null when nobody is seen), running, mirror, video.
  • Landmark indexes (BlazePose): nose 0, shoulders 11 and 12, wrists 15 and 16, hips 23 and 24.
  • The model is MediaPipe's "lite" pose landmarker on the GPU, one pose at a time. If two people are in view, only one is tracked.

Examples

js
import { Signals } from '@openav/core';
import { PoseTracker } from '@openav/pose';

const signals = new Signals();
const pose = new PoseTracker({ signals });
document.addEventListener('click', async () => {
  await pose.enable();                                   // camera prompt, then the model downloads
  console.log('tracking');
}, { once: true });
setInterval(() => {
  console.log('raised hand', signals.get('pose/hand/right/y').toFixed(2), 'present', signals.get('pose/present'));
}, 500);
static Click the page and allow the camera: the console prints tracking, then every half second the height of the right wrist (1 = raised) and whether someone is in view.
js
import { PoseTracker } from '@openav/pose';

const pose = new PoseTracker();                       // never enabled: no camera, no download
const point = (x, y) => ({ x, y, z: 0, visibility: 1 });
const landmarks = Array.from({ length: 33 }, () => point(0.5, 0.5));
landmarks[0] = point(0.5, 0.2);                       // nose
landmarks[11] = point(0.58, 0.35); landmarks[12] = point(0.42, 0.35);   // shoulders
landmarks[13] = point(0.68, 0.45); landmarks[14] = point(0.32, 0.25);   // elbows
landmarks[15] = point(0.74, 0.58); landmarks[16] = point(0.24, 0.1);    // wrists: one down, one raised
landmarks[23] = point(0.55, 0.62); landmarks[24] = point(0.45, 0.62);   // hips
landmarks[25] = point(0.56, 0.8); landmarks[26] = point(0.44, 0.8);     // knees
landmarks[27] = point(0.56, 0.95); landmarks[28] = point(0.44, 0.95);   // ankles
landmarks[29] = point(0.57, 0.97); landmarks[30] = point(0.43, 0.97);   // heels
landmarks[31] = point(0.6, 0.98); landmarks[32] = point(0.4, 0.98);     // toes
pose.landmarks = landmarks;                           // what enable() keeps filling in from the camera

const ctx = document.getElementById('c').getContext('2d');
ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
pose.skeleton(ctx, 400, 300, { color: '#7aa6ff', lineWidth: 3 });
module A stick figure with one arm raised is drawn on the canvas from made-up landmarks: the overlay skeleton() draws over the camera picture, without needing a camera.
js
import { Signals, Params, Loop } from '@openav/core';
import { Mapper } from '@openav/mapping';
import { PoseTracker } from '@openav/pose';

const signals = new Signals();
const params = new Params([{ key: 'energy', min: 0, max: 1, def: 0.2 }]);
const mapper = new Mapper({ signals, params });
mapper.addRoute({ source: 'pose/hand/right/y', target: 'energy', smooth: 0.1 });   // a raised hand is energy
const pose = new PoseTracker({ signals });

document.addEventListener('click', () => pose.enable(), { once: true });
new Loop((dt) => {
  mapper.update(dt);
  const energy = params.resolve().energy;
  const ctx = document.getElementById('c').getContext('2d');
  ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
  ctx.fillStyle = '#7aa6ff'; ctx.fillRect(20, 280, 360 * energy, 10);
}).start();
static Click to start the camera, then raise your right hand: the bar at the bottom of the canvas grows with it, because the pose signal is mapped to a param.

To be confirmed

Not tried here with a live camera. In particular, which person's hand pose/hand/left/* follows in practice was not checked: the code takes landmark 16 for left and landmark 15 for right when mirror is true (the other way round when false), and the package docs say left is the viewer's left.

Whether the camera permission prompt needs a click first was not tested; the package README calls enable() from a button.

Related

  • HandTracker Tracks up to two hands from the camera with 21 landmarks each and publishes palm position and thumb-to-finger pinch distances as hand/* signals.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { PoseTracker } from '@openav/pose'; packages/pose/index.js, line 23

Chords (@openav/chord)#

Held notes in, chord quality and consonance out. 1

@openav/chord

ChordDetector#

class@openav/chord
new ChordDetector({ window = 80, onChord = null, signals = null } = {})

Groups notes struck close together into one gesture and analyses it: chord type, consonance, triad and cluster flags, published as chord/* signals.

This is not a music-theory library. It is a performance semantics layer: raw notes in, one analysed event per gesture out, with features that are made to drive a work. A single note is a seed, a chord a colony, a consonant chord a flourish, a dissonant one decay; rename them for your own piece. In a show, createShow({ modules: { chord: true } }) builds one ({ chord: { window: 120 } } tunes it) and connects it to midi/note/on and midi/note/off, so a MIDI keyboard, the on-screen piano and phones all feed it.

Each closed gesture publishes chord/consonance (−1..1), chord/count (0..10), chord/root (the lowest note) and the pulse chord/event, whose payload is the whole analysis: { notes, count, root, vel, pcs, consonance, isConsonant, isDissonant, isTriad, thirdsFraction, dissonanceLevel, chordType }. chordType is one of single, major, minor, sus2, sus4, dim, aug, maj7, min7, dom7, halfdim7, maj9, min9, dom9, six, min6, cluster (a very dissonant pile) or chord (anything else). Inversions resolve to the root-position name.

consonance is the average of an interval weight for every pair of notes (an octave or fifth is high, a semitone or tritone is negative). isConsonant is consonance > 0.25 and isDissonant is consonance < −0.15, both only for two notes or more. dissonanceLevel is 0 for none, 1 for mild (a storm warning: foreshadow it) and 2 for severe (three or more notes that are a true cluster: commit to the decay). isTriad is true for major, minor, sus2, sus4, dim and aug, and thirdsFraction is the share of neighbouring notes exactly 3 or 4 semitones apart, so "genuinely stacked thirds" can be told from "some consonant notes".

Before you call it

Feed it with noteOn(note, vel) (and noteOff(note)), or let createShow do it from midi/note/on|off. Nothing is published until window milliseconds after the last noteOn, so the result is asynchronous. analyze() is the exception: it is instant and needs no setup.

Parameters

NameTypeDescription
windowNumberHow many milliseconds after the last noteOn the gesture is closed and analysed. Notes struck within this time of each other are one chord. optional default 80
onChordFunction | nullCalled with the analysis object each time a gesture is closed. optional default null
signalsSignals | nullA Signals registry to publish chord/* into. The signals are declared when the detector is built. optional default null

Members

NameDescription
noteOn(note, vel = 0.8)methodA note was struck: noteOn(note, vel = 0.8), note a MIDI number and vel 0..1. Restarts the window timer; striking the same note twice in a gesture counts once, with the harder velocity.
noteOff(note)methodA note was released. It only updates the held map; a gesture is made of the notes struck inside the window, not of what is still held.
analyze(notes, vels = [])methodA pure function: analyze(notes, vels = []) takes MIDI note numbers (lowest first) and returns the analysis object at once, without timers or signals. vel is the mean of vels, or 0.8 without them.

Notes

  • Pass notes to analyze() lowest first: root is the first note, and the detector sorts for you but a direct call does not.
  • A major second like [60, 62] is typed cluster (no template fits and its consonance is below −0.3) even though its dissonanceLevel is only 1. Level 2 needs three or more notes.
  • The gesture window does not know about held notes: a note already down when you play two more does not join them.

Examples

js
import { ChordDetector } from '@openav/chord';

const chord = new ChordDetector();
const major = chord.analyze([60, 64, 67]);
console.log(major.chordType, major.isTriad, major.consonance, major.thirdsFraction);   // major true 0.75 1
const cluster = chord.analyze([60, 61, 62, 63]);
console.log(cluster.chordType, cluster.dissonanceLevel, cluster.isDissonant);          // cluster 2 true
module The console prints major true 0.75 1 for C-E-G, then cluster 2 true for four neighbouring semitones.
js
import { Signals } from '@openav/core';
import { ChordDetector } from '@openav/chord';

const signals = new Signals();
signals.define('midi/note/on', { kind: 'pulse' });
signals.define('midi/note/off', { kind: 'pulse' });
const chord = new ChordDetector({ signals, window: 30 });
signals.on('chord/event', (a) => console.log('chord/event:', a.chordType, a.count, 'notes'));
signals.on('midi/note/on', ({ note, vel }) => chord.noteOn(note, vel));      // what createShow wires for you
signals.on('midi/note/off', ({ note }) => chord.noteOff(note));

for (const note of [60, 64, 67]) signals.pulse('midi/note/on', { note, vel: 0.8 });   // struck together
setTimeout(() => console.log(signals.get('chord/consonance'), signals.get('chord/root')), 100);
module Three notes struck together become one event: the console prints chord/event: major 3 notes, then 0.75 60 (consonance and root).
js
import { ChordDetector } from '@openav/chord';

const chord = new ChordDetector();
const types = [[60, 64, 67], [64, 67, 72], [60, 64, 67, 70], [60, 63, 66], [60, 65, 67], [60, 67]];
for (const notes of types) console.log(JSON.stringify(notes), chord.analyze(notes).chordType);
module Inversions resolve to the root-position name: the console prints major, major (E-G-C is C major again), dom7, dim, sus4, and chord for the bare fifth, which matches no template.

Related

  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { ChordDetector } from '@openav/chord'; packages/chord/index.js, line 30

OSC and outputs#

Send signals to TouchDesigner, Max or a hardware bridge. 9

@openav/osc

OscOut#

class@openav/osc
new OscOut(url = 'http://127.0.0.1:7456')

Sends OSC messages from the browser to TouchDesigner, Max, Reaper or a spatialiser, batched once per frame, through a small local bridge.

A browser cannot send UDP, so OscOut posts batches of messages to a ~100-line Node program, packages/osc/bridges/osc-bridge.js, which sends them on as OSC packets. Start it with node packages/osc/bridges/osc-bridge.js 7456 192.168.1.50 3456 (HTTP port, target host, UDP port; the defaults are 7456 127.0.0.1 3456). Then send(address, args) as often as you like and flush() once per frame: only the latest message per address survives a frame, because a position stream does not need its in-between values.

Numbers go out as OSC floats (what Spat, Reaper and TouchDesigner expect), and anything else as a string. createShow() does not create an OscOut: make one, call enable(), flush it from onFrame, and if you like hand it to the worlds as show.stage.io.osc and to the Layers panel as show.app.osc (which then shows the OSC rate).

Before you call it

The bridge must be running, and you must await osc.enable() first: it probes GET /health and answers false if nobody is there. Until it succeeds flush() just throws the queue away. On a public https page, Chrome asks permission before the page may reach 127.0.0.1.

Parameters

NameTypeDescription
urlStringThe address of the OSC bridge (its HTTP port, not the UDP target). optional default 'http://127.0.0.1:7456'

Members

NameDescription
enable()methodProbe the bridge (GET /health) and, if it answers, switch on. Resolves true, or false when nothing is listening. Any HTTP answer counts as present. Calls onStatus(ok, info); info is the UDP target the bridge reports.
disable()methodSwitch off and throw away anything queued. Calls onStatus(false, 'disabled').
send(addr, args)methodQueue one message: send(address, args). args is an array, or a single value that is wrapped in one. Nothing leaves until flush(). Calls the onMsg(address, args) hook if you set one.
rate()methodMessages sent since the last call, which resets the count (so call it once a second for a rate). What the Layers panel shows.
flush()methodSend the queue as one POST, the latest message per address. Call once per frame. Async; it does nothing while a previous flush is still in flight, and throws the queue away when enabled is false.

Notes

  • Fields and hooks: enabled, udp (the target the bridge reported), onStatus = (ok, info) => {} and onMsg = (address, args) => {} (high frequency: use sparingly).
  • If the bridge disappears, the first failed flush calls onStatus(false, 'bridge lost'), and after more than 30 failed flushes in a row enabled turns false by itself, with no further message. Call enable() again to resume. The messages of a failed flush are lost.
  • Every number goes out as a 32-bit float, whole numbers included (the bridge's own header comment mentions integers, but its code only writes floats). If the receiver needs integers, convert there.

Examples

js
import { OscOut } from '@openav/osc';

const osc = new OscOut();                          // http://127.0.0.1:7456
console.log(await osc.enable());                   // true when the bridge answers
function frame(x, y, z) {
  osc.send('/source/3/xyz', [x, y, z]);            // queue as often as you like
  osc.flush();                                     // once per frame
}
frame(0.1, 0.5, 0.9);
static With node packages/osc/bridges/osc-bridge.js running, the console prints true and the bridge forwards /source/3/xyz as an OSC packet with three floats.
js
import { OscOut } from '@openav/osc';

// a stand-in for the bridge, so this runs anywhere
globalThis.fetch = async (url, options = {}) => {
  if (options.body) console.log('POST', url, options.body);
  return { json: async () => ({ ok: true, udp: '127.0.0.1:3456' }) };
};

const osc = new OscOut();
osc.onStatus = (ok, info) => console.log('status', ok, info);
await osc.enable();
osc.send('/source/1/xyz', [0.1, 0.2, 0.3]);
osc.send('/source/1/xyz', [0.4, 0.5, 0.6]);   // the same address: replaces the first
osc.send('/gain', 0.8);                       // a single value is wrapped in an array
await osc.flush();
console.log('rate', osc.rate());              // 2 messages went out
module Three send() calls become two messages: the console prints status true 127.0.0.1:3456, then the one POST with /source/1/xyz carrying [0.4,0.5,0.6] and /gain carrying [0.8], then rate 2.
js
import { OscOut } from '@openav/osc';

const osc = new OscOut('http://127.0.0.1:7499');    // an address where no bridge runs
osc.onStatus = (ok, info) => console.log('status', ok, info);
console.log(await osc.enable());                  // false: nobody answers
osc.send('/gain', 0.5);
await osc.flush();                                // nothing is sent while disabled: the queue is dropped
module With no bridge, enable() does not throw: the console prints status false bridge not running (node bridges/osc-bridge.js), then false. (The browser may also log one failed request.)

To be confirmed

Not tried here against TouchDesigner, Max or Reaper: the examples use a stand-in for the bridge, and what the bridge does with the packets is read from its code (floats for numbers, strings for the rest).

Related

  • OutClient The page side of the out-bridge: set(channel, value) sends a number to a named hardware channel, at most once per frame, and keeps the connection alive.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).

import { OscOut } from '@openav/osc'; packages/osc/index.js, line 15

@openav/out

DEFAULT_PORT#

constant@openav/out
DEFAULT_PORT = 6450

The out-bridge's default port: 6450.

The default of startBridge({ port }) and of --port; the environment variable OUT_BRIDGE_PORT changes it for the command line. It is the out-bridge's port: the Leap bridge has its own (DEFAULT_PORT of @openav/leap, 6437).

Related

  • OUT_DEFAULT_URL The address OutClient connects to by default: ws://127.0.0.1:6450/out, the out-bridge on this machine.

import { DEFAULT_PORT } from '@openav/out/bridge'; packages/out/bridge/out-bridge.mjs, line 27

DEFAULT_TIMEOUT_MS#

constant@openav/out
DEFAULT_TIMEOUT_MS = 500

The watchdog time when the config does not say: 500 milliseconds without a value, and a channel is put on its safe value.

It is the default of timeoutMs in the bridge config (the minimum allowed is 20). OutClient re-sends held values every third of whatever the bridge reports, so a still slider is not mistaken for a dead page. Lower it for hardware that must stop quickly, but keep it above the slowest frame your page may have.

Related

  • OutClient The page side of the out-bridge: set(channel, value) sends a number to a named hardware channel, at most once per frame, and keeps the connection alive.

import { DEFAULT_TIMEOUT_MS } from '@openav/out/bridge'; packages/out/bridge/out-bridge.mjs, line 29

globToRegExp()#

function@openav/out
globToRegExp(glob)

Turns a channel glob like light/* into the regular expression the bridge matches channel names with: only * is special, and it crosses slashes.

The match of a channel rule and the keys of safe in the bridge config are globs, and this is how both are read. Remember that * also crosses slashes: light/* fits light/0/r, and * alone fits every channel.

Parameters

NameTypeDescription
globStringThe pattern. * matches any run of characters, slashes included. Everything else, including ., ? and +, matches itself.

Returns

RegExp An anchored expression: globToRegExp('light/*') is /^light\/.*$/.

Examples

js
import { globToRegExp } from '@openav/out/bridge';

const lights = globToRegExp('light/*');
console.log(lights.test('light/0/r'), lights.test('motor/1'));   // true false
console.log(globToRegExp('*').test('any/channel/at/all'));       // true
console.log(globToRegExp('a.b').test('axb'));                    // false: the dot is only a dot
static In Node: the console prints true false, then true, then false.

Related

  • loadConfig() Reads the out-bridge config from a JSON file and returns it as an object (Node only); throws when the file cannot be read.

import { globToRegExp } from '@openav/out/bridge'; packages/out/bridge/out-bridge.mjs, line 37

loadConfig()#

function@openav/out
loadConfig(file)

Reads the out-bridge config from a JSON file and returns it as an object (Node only); throws when the file cannot be read.

A thin helper around fs.readFileSync and JSON.parse. A missing file or invalid JSON throws one error that names the file (its text is written in Chinese in the code). The command line uses it for --config out.json; in your own program you may just as well pass an object to startBridge({ config }).

Parameters

NameTypeDescription
fileStringThe path of the JSON file, such as out.json (copy packages/out/bridge/out.example.json to start).

Returns

Object The parsed config, unchecked: startBridge() is what validates it.

Examples

js
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { loadConfig } from '@openav/out/bridge';

const file = path.join(os.tmpdir(), 'oav-example-out.json');          // a scratch file, not your real config
fs.writeFileSync(file, JSON.stringify({ channels: [{ match: '*', sink: 'log' }], safe: { 'light/*': 0 } }));
const config = loadConfig(file);
console.log(config.channels.length, config.safe);      // 1 { 'light/*': 0 }
try { loadConfig('missing.json'); } catch (error) { console.log('cannot load:', error.message.includes('missing.json')); }   // true
static In Node: the console prints 1 { 'light/*': 0 }, then cannot load: true: the error names the missing file.

Related

  • globToRegExp() Turns a channel glob like light/* into the regular expression the bridge matches channel names with: only * is special, and it crosses slashes.

import { loadConfig } from '@openav/out/bridge'; packages/out/bridge/out-bridge.mjs, line 41

OUT_DEFAULT_URL#

constant@openav/out
OUT_DEFAULT_URL = "ws://127.0.0.1:6450/out"

The address OutClient connects to by default: ws://127.0.0.1:6450/out, the out-bridge on this machine.

Port 6450 is the bridge's DEFAULT_PORT and /out is the only WebSocket path it serves. In the lab, ?out=ws://host:port/out points a page at another bridge, ?out=sim records values without one and ?out=0 turns it off.

Related

  • OutClient The page side of the out-bridge: set(channel, value) sends a number to a named hardware channel, at most once per frame, and keeps the connection alive.

import { OUT_DEFAULT_URL } from '@openav/out'; packages/out/client.js, line 15

OutClient#

class@openav/out
new OutClient({ url = OUT_DEFAULT_URL, id = null, WebSocket: WS, schedule, maxBackoffMs = 5000, minBackoffMs = 250 } = {})

The page side of the out-bridge: set(channel, value) sends a number to a named hardware channel, at most once per frame, and keeps the connection alive.

Hardware needs a number, and a page has a number: out.set('light/0/r', 0.8). The channel name is yours (a-z A-Z 0-9 _ - . /, up to 128 characters) and the bridge (startBridge()) decides from its config where that channel goes: a WLED strip, a serial motor board, an OSC address. Values are plain numbers; the convention is 0..1. Several set() calls on one channel inside one frame leave as one value, the last. In a lab work lab.out is this class.

It keeps the connection alive on its own: it reconnects with a growing delay (250 ms doubling up to 5 s), resends everything it knows after a reconnect (a restarted bridge has forgotten it all), and while the socket is open it re-sends the held values every timeoutMs / 3 of the bridge's watchdog, so a slider that stands still is not mistaken for a frozen page. If the page really freezes, that heartbeat stops and the bridge puts the hardware on its safe values.

Before you call it

Call connect() (the bridge must be running: node packages/out/bridge/out-bridge.mjs --config out.json), or simulate() for no hardware at all. set() before the socket is open is not lost: it waits in last and goes out on open.

Parameters

NameTypeDescription
urlStringThe bridge's WebSocket address. The default is OUT_DEFAULT_URL, ws://127.0.0.1:6450/out. optional default OUT_DEFAULT_URL
idString | nullA name for this page (the work's id), sent to the bridge in its "hello" and shown in its log. optional default null
WebSocketFunctionThe WebSocket constructor to use. Defaults to the global one (also in Node 22). Pass your own to test without a network. optional
scheduleFunctionA function that runs a callback once, on the next frame. The default is requestAnimationFrame (or a 16 ms timer when the tab is hidden or there is no requestAnimationFrame). optional
maxBackoffMsNumberThe longest wait between reconnection attempts, in milliseconds. optional default 5000
minBackoffMsNumberThe first wait between reconnection attempts. It doubles each time up to maxBackoffMs. optional default 250

Members

NameDescription
onStatus(fn)methodCall fn(status) when the status changes. Returns a function that unsubscribes.
connect()methodOpen the socket and keep it open (reconnects with a capped, doubling delay). Returns the client, so new OutClient(…).connect() chains. Does nothing in simulate mode, or when there is no WebSocket constructor.
simulate()methodNo socket: values are only recorded in last (for sketches without hardware; the lab does this for ?out=sim). The status becomes 'sim'.
close()methodClose the socket and stop reconnecting. The status becomes 'closed'. The bridge sees the last client leave and puts every channel with a safe value back on it.
set(ch, v)methodset(channel, value): send one number. The value goes through Number(); a channel that is not a string or a value that is not finite is ignored. Recorded in last at once, sent on the next frame.
batch(obj)methodbatch({ channel: value, … }): the same as one set() per entry.

Notes

  • Fields to read: status ('closed', 'connecting', 'open', 'sim'), last ({ channel: last value given to set() }, also in simulate mode), sent (values actually put on the wire: less than the number of set() calls because of coalescing) and bridge (the bridge's last status message).
  • On the wire a single value is { "type": "set", "ch", "v" } and several are { "type": "batch", "items": [{ "ch", "v" }, …] }, after a { "type": "hello", "id" }.
  • On a public https page, Chrome asks permission before the page may reach 127.0.0.1; if it is refused, the status stays 'connecting'.

Examples

js
import { OutClient } from '@openav/out';

const out = new OutClient({ id: 'my-work' });       // ws://127.0.0.1:6450/out
out.onStatus((status) => console.log('out', status));   // connecting, open
out.connect();
out.set('light/0/r', 0.8);                          // waits for the socket, then goes out
function frame(t) {
  out.set('light/0/g', 0.5 + 0.5 * Math.sin(t / 1000));   // call as often as you like: one value a frame leaves
  requestAnimationFrame(frame);
}
requestAnimationFrame(frame);
static With the out-bridge running, the console prints out connecting, then out open, and the green channel breathes between 0 and 1.
js
import { OutClient } from '@openav/out';

class FakeSocket {                                  // stands in for the bridge's socket, so this runs anywhere
  constructor() { this.readyState = 1; setTimeout(() => this.onopen(), 0); }
  send(text) { console.log('wire:', text); }
  close() {}
}
const out = new OutClient({ id: 'demo', WebSocket: FakeSocket, schedule: (fn) => setTimeout(fn, 0) });
out.connect();
setTimeout(() => {
  out.set('light/0/r', 0.8);
  out.set('light/0/r', 0.9);                        // same channel, same frame: only 0.9 leaves
  out.batch({ 'light/0/g': 0.2, 'motor/1': 0.5 });
  setTimeout(() => { console.log('sent', out.sent, out.status); out.close(); }, 50);
}, 20);
module The console prints the hello, then one batch of three values (light/0/r is 0.9, not 0.8), then sent 3 open.
js
import { OutClient } from '@openav/out';

const out = new OutClient().simulate();              // no bridge, no hardware
out.set('light/0/r', 1);
out.set('light/0/r', 0.25);
out.set('typo', 'not a number');                     // ignored: not finite
console.log(out.status, out.last);                   // sim { 'light/0/r': 0.25 }
module Simulate mode records instead of sending: the console prints sim { 'light/0/r': 0.25 }; the bad value was ignored.

To be confirmed

Not tried here with real hardware (a WLED strip, a motor board): the examples use a stand-in socket and the bridge's log sink.

Related

  • OUT_DEFAULT_URL The address OutClient connects to by default: ws://127.0.0.1:6450/out, the out-bridge on this machine.
  • OscOut Sends OSC messages from the browser to TouchDesigner, Max, Reaper or a spatialiser, batched once per frame, through a small local bridge.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.

import { OutClient } from '@openav/out'; packages/out/client.js, line 17

RESERVED_PORTS#

constant@openav/out
RESERVED_PORTS = array, 1 items

The ports the out-bridge refuses to use: [7457], which belongs to The Last Input and the backstage monitor.

startBridge({ port: 7457 }) throws at once, so the bridge can never take the port the monitor listens on. The Leap bridge refuses the same one (RESERVED_PORTS of @openav/leap).

Related

  • MonitorFeed Streams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.

import { RESERVED_PORTS } from '@openav/out/bridge'; packages/out/bridge/out-bridge.mjs, line 28

startBridge()#

function@openav/out
startBridge({ port = DEFAULT_PORT, host = '127.0.0.1', config = null, watchdog = true, log = () => {}, print = (s) => console.log(s) } = {})

Starts the out-bridge: a loopback WebSocket server that routes channel values from the page to hardware sinks, with a watchdog that falls back to safe values (Node only).

This is what node packages/out/bridge/out-bridge.mjs --config out.json runs (also --port, --no-watchdog, --quiet; the environment variable OUT_BRIDGE_PORT). It serves ws://127.0.0.1:6450/out, which understands { type: 'hello', id }, { type: 'set', ch, v }, { type: 'batch', items } and { type: 'status' }, and answers with { type: 'status', … }; GET /health reports the sinks, the channel count, the clients and the time since the last value. Channel names are a-z A-Z 0-9 _ - . / up to 128 characters, values must be finite numbers, and there is room for 4096 channels.

The config routes channels: channels is a list of { match, sink, … } and the first rule whose match glob fits the channel name takes it (* means anything, slashes included); a channel no rule fits is counted and dropped. The sinks are log (stdout), udp-json (UDP, {"ch","v"}), osc (UDP, address "/" + channel, one float32), ddp (WLED over UDP 4048: light/<i>/r|g|b, 0..1, pixel index from 0, up to 10000 pixels) and serial (a device path, ch value lines or JSON, latest value only). safe maps a channel glob to its safe value, timeoutMs (default 500, at least 20) is the watchdog time, and allowOrigins lists the page origins that may connect (without it any page on this machine can).

The watchdog: a channel that has a safe value and gets no new value for timeoutMs is sent its safe value once (a new value re-arms it). When the last client disconnects, every channel the bridge has seen that has a safe value gets it at once, and so does a clean shutdown. It does not cover channels the bridge has never heard of, nor a bridge killed with SIGKILL.

It throws at once for port 7457 (it belongs to The Last Input and the monitor), for a host that is not loopback, and for a bad config (no channels, an unknown sink, a safe value that is not a number, a timeoutMs under 20). A port already in use rejects the returned promise.

Before you call it

Node 22 or later. The sinks need what they talk to: a WLED device for ddp, the device file for serial, a listener for osc.

Parameters

NameTypeDescription
portNumberThe port to listen on, DEFAULT_PORT unless you say otherwise. 0 picks any free one. The ports in RESERVED_PORTS (7457) are refused. optional default DEFAULT_PORT
hostStringThe address to bind. Only loopback is allowed. optional default '127.0.0.1'
configObject | nullThe routing: { channels: [{ match, sink, … }], safe: { glob: value }, timeoutMs, allowOrigins } (see the description). null sends every channel to the log sink. loadConfig(file) reads it from a JSON file. optional default null
watchdogBooleanfalse switches the watchdog off: no safe values are ever sent (the tests use it to prove the watchdog is what does the work). optional default true
logFunctionCalled with a line of text for diagnostics: clients coming and going, safe values sent. optional default () => {}
printFunctionWhat the log sink prints each value with ("channel value"). The default is stdout. optional default (s) => console.log(s)

Returns

Promise<Object> Resolves once listening, with { port, state, health(), close() }: the real port, the counters (received, rejected, unrouted, safeSent …), a function returning the /health object, and a function that puts every known channel on its safe value, stops the server and returns a Promise.

Notes

  • The error messages the bridge throws and sends back to a page are written in Chinese in the code, so a page may show them in Chinese whatever its language.

Examples

js
import { startBridge } from '@openav/out/bridge';
import { OutClient } from '@openav/out';

const bridge = await startBridge({
  port: 0,                                                       // any free port
  config: { channels: [{ match: '*', sink: 'log' }], safe: { 'light/*': 0 }, timeoutMs: 2000 },
  print: (line) => console.log('sink:', line),
});
const out = new OutClient({ url: 'ws://127.0.0.1:' + bridge.port + '/out', id: 'demo' }).connect();
setTimeout(() => out.set('light/0/r', 0.8), 100);                // reaches the sink
setTimeout(() => out.close(), 300);                              // the last client leaves: safe values go out
setTimeout(() => bridge.close(), 600);
static In Node: the console prints sink: light/0/r 0.8, then sink: light/0/r 0: when the page goes away the bridge puts the light channel on its safe value.

To be confirmed

The ddp (WLED) and serial sinks were not tried here with real hardware; what the description says about them comes from the package README and the code. The log sink and the watchdog were run.

Related

  • OutClient The page side of the out-bridge: set(channel, value) sends a number to a named hardware channel, at most once per frame, and keeps the connection alive.
  • loadConfig() Reads the out-bridge config from a JSON file and returns it as an object (Node only); throws when the file cannot be read.
  • globToRegExp() Turns a channel glob like light/* into the regular expression the bridge matches channel names with: only * is special, and it crosses slashes.
  • DEFAULT_TIMEOUT_MS The watchdog time when the config does not say: 500 milliseconds without a value, and a channel is put on its safe value.

import { startBridge } from '@openav/out/bridge'; packages/out/bridge/out-bridge.mjs, line 57

Mapping (@openav/mapping)#

Routes from signals to params, curves, smoothing, learn. 9

@openav/mapping

LearnWizard#

class@openav/mapping
new LearnWizard({ mapper, keys, onChange = null })

Binds a list of params one after another: it arms learn for the first, you turn a knob, it binds it and arms the next. Skip, back and stop steer it.

Binding twelve params one by one with the learn chip means twelve clicks and twelve guesses about what to touch next. The wizard walks the list for you: start() arms learn for keys[0], wizard.current says which param it is waiting for ('Turn a knob for: hue (1/12)'), you move a control, it is bound and the next one is armed. At the end learn is disarmed. The console's 🎛 map knobs in order button is this class.

It is pure logic on top of the mapper, with no screen of its own. Learn only ever adds routes, so running the wizard twice puts a second control on a param instead of losing the first. While it runs, it owns learn mode: call sync() every frame so that a click on some other learn chip cannot leave it waiting for nothing.

Parameters

NameTypeDescription
mapperMapperThe Mapper to bind through.
keysArray<String> | FunctionThe param keys to bind, in order, or a function that returns them (it is called when start() runs).
onChangeFunction | nullCalled with the wizard whenever what a banner would show changes: the current param, the position, finished or not. optional default null

Members

NameDescription
totalgetterHow many params are in this run.
currentgetterThe param key being waited for, or null when the wizard is not running.
start()methodRead the keys, arm the first one and begin. Returns false (and does nothing) when the list is empty.
skip()methodLeave the current param unbound and wait for the next one. Skipping the last one finishes the run.
back()methodGo back to the previous param. It keeps what it already has; a turned knob adds one more.
stop()methodEnd the run now and disarm learn. done stays false.
sync()methodCall every frame: if learn mode was switched off or pointed elsewhere while the wizard runs, it re-arms the current param.

Notes

  • Fields to read: active, done (the run reached the end), index, total, order, and added ([{ key, source }], what was bound in this run).
  • While it runs the wizard wraps the mapper's onLearn and puts yours back when it ends, so your own onLearn keeps being called.

Examples

js
import { Signals, Params } from '@openav/core';
import { Mapper, LearnWizard } from '@openav/mapping';

const signals = new Signals();
for (const name of ['knob/1', 'knob/2']) signals.define(name, { min: 0, max: 1 });
const params = new Params([{ key: 'hue', min: 0, max: 360, def: 0 }, { key: 'size', min: 0, max: 1, def: 0.2 }]);
const mapper = new Mapper({ signals, params });
const wizard = new LearnWizard({ mapper, keys: () => params.schema.map((p) => p.key) });

wizard.start();
console.log('waiting for', wizard.current);                       // hue
const turn = (name) => { signals.set(name, 0); signals.set(name, 0.9); };   // a knob that moves a lot
turn('knob/1'); console.log('waiting for', wizard.current);       // size
turn('knob/2'); console.log(wizard.done, wizard.added);
module The console prints waiting for hue, waiting for size, then true and the two bindings [ { key: 'hue', source: 'knob/1' }, { key: 'size', source: 'knob/2' } ].

Related

  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
  • mountConsole() Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.

import { LearnWizard } from '@openav/mapping'; packages/mapping/wizard.js, line 16

loadMapUrl()#

function@openav/mapping
async loadMapUrl(mapper, value, { base = (typeof location !== 'undefined' ? location.href : undefined), fetchImpl = (typeof fetch !== 'undefined' ? fetch : null), timeoutMs = 8000 } = {})

Fetches a map file from an address (what ?map= calls) and puts it on the mapper; it never throws and never blocks the show.

Fetches the address with credentials: 'omit', and importMap()s the text. Any failure (refused address, no fetch, the server answered 404, offline, a site that blocks cross-origin reads, a timeout, a file that is not a map) lands in the report and in mapper.report, where the console shows it, and the routes stay as they were. It is async, so createShow() does not wait for it and the show starts regardless.

A map on another site must be served with CORS (Access-Control-Allow-Origin: *); raw GitHub files and gists are. In a show you rarely call it: open the page with ?map=<url>, or pass map: 'maps/a.json' to createShow().

Parameters

NameTypeDescription
mapperMapperThe Mapper to import into.
valueStringThe address of the map file, checked by parseMapUrl() first.
baseStringThe page address that relative addresses resolve against. optional default location.href
fetchImplFunctionThe function that does the request. Replace it to test without a network. optional default fetch
timeoutMsNumberGive up after this many milliseconds. optional default 8000

Returns

Promise<Object> The report: what mapper.importMap() returned (source: 'url', address, added, skipped …), or { ok: false, source: 'url', address, error }. The same report is kept in mapper.report.

Examples

js
import { Signals, Params } from '@openav/core';
import { Mapper, loadMapUrl } from '@openav/mapping';

const signals = new Signals();
signals.define('audio/rms');
const params = new Params([{ key: 'bloom', min: 0, max: 100, def: 0 }]);
const mapper = new Mapper({ signals, params, profile: 'garden' });

const file = { format: 'openav.map', version: 1, world: 'garden', routes: [{ signal: 'audio/rms', param: 'bloom' }] };
const fetchImpl = async () => ({ ok: true, status: 200, text: async () => JSON.stringify(file) });   // no network needed
const report = await loadMapUrl(mapper, 'maps/stage-left.json', { base: 'https://openaudiovisual.com/x/', fetchImpl });
console.log(report.ok, report.address, report.added);
module With a stand-in for the network, the console prints true https://openaudiovisual.com/x/maps/stage-left.json 1: the route came in.
js
import { Signals, Params } from '@openav/core';
import { Mapper, loadMapUrl } from '@openav/mapping';

const mapper = new Mapper({ signals: new Signals(), params: new Params([]) });
const report = await loadMapUrl(mapper, 'http://example.com/map.json', { base: 'https://openaudiovisual.com/' });
console.log(report.ok, report.error);        // false 'a map from another site must be served over https'
console.log(mapper.report === report);       // true: the console would show it too
module A refused address is a report, not an exception: the console prints false a map from another site must be served over https, then true.

Related

  • parseMapUrl() Decides whether a ?map= address may be fetched: same origin, or an absolute https URL, and nothing else.
  • parseMap() Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { loadMapUrl } from '@openav/mapping'; packages/mapping/index.js, line 126

MAP_FORMAT#

constant@openav/mapping
MAP_FORMAT = "openav.map"

The text "openav.map": the value of format in every map file. A file with another format is refused.

A map file is JSON with "format": "openav.map", a version, a world name and a list of routes. parseMap() refuses anything whose format is not this string, so you cannot import some other tool's JSON by accident. The one exception is a bare array (what mapper.toJSON() returns), which is read as version 0.

Related

  • MAP_VERSION The newest map file version this build reads and writes: 1. A file with a higher version is refused with a message.
  • MAP_MAX_BYTES The longest map text parseMap() will read: 262144, which is 256 KB. Longer text is refused without being parsed.
  • parseMap() Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
  • serializeMap() Builds the portable map file object from a list of routes: { format, version, world, exported, routes }.

import { MAP_FORMAT } from '@openav/mapping'; packages/mapping/index.js, line 42

MAP_MAX_BYTES#

constant@openav/mapping
MAP_MAX_BYTES = 262144

The longest map text parseMap() will read: 262144, which is 256 KB. Longer text is refused without being parsed.

The limit is compared with the length of the text (characters), before any parsing, so a huge file cannot freeze the page. A map with more than 500 routes is refused as well, whatever its size.

Related

  • parseMap() Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
  • MAP_FORMAT The text "openav.map": the value of format in every map file. A file with another format is refused.
  • MAP_VERSION The newest map file version this build reads and writes: 1. A file with a higher version is refused with a message.

import { MAP_MAX_BYTES } from '@openav/mapping'; packages/mapping/index.js, line 44

MAP_VERSION#

constant@openav/mapping
MAP_VERSION = 1

The newest map file version this build reads and writes: 1. A file with a higher version is refused with a message.

serializeMap() writes this number and parseMap() accepts any version from 1 up to it. A newer file gets the message "this build of OAV reads up to version 1. Update OAV or re-export it." A file with no valid version is refused too. Only a bare array of routes counts as version 0.

Related

  • MAP_FORMAT The text "openav.map": the value of format in every map file. A file with another format is refused.
  • MAP_MAX_BYTES The longest map text parseMap() will read: 262144, which is 256 KB. Longer text is refused without being parsed.
  • parseMap() Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.

import { MAP_VERSION } from '@openav/mapping'; packages/mapping/index.js, line 43

Mapper#

class@openav/mapping
new Mapper({ signals, params, profile = 'default', onChange = null, onLearn = null })

Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.

The mapper is the only component that connects signals to params, and everything it knows is a list of routes, plain objects. mapper.addRoute({ source: 'midi/cc/74', target: 'bloom' }) makes the param bloom follow the signal midi/cc/74. One signal may drive several params, one param may listen to several signals (the last writer wins), and learn adds routes, it never replaces them. In a show, createShow() builds one, loads the saved routes (or adds the ones you declared in routes) and returns it as show.mapper.

A route needs only source and target; addRoute() fills in the rest and returns the stored route:

text
source, target      the signal name and the param key (required)
inMin, inMax        0, 1   the part of the signal that counts; outside it is clamped
outMin, outMax      the param's min and max AT THE MOMENT THE ROUTE IS ADDED (0 and 1 if the param does not exist yet)
curve               'linear' | 'exp' (x squared) | 'log' (square root) | 'smooth' (smoothstep); an unknown name is linear
invert              false   flip the value (1 - x) before the curve
smooth              0       glide time in seconds; 0 = instant
enabled             absent = on; false mutes the route
id                  assigned; used by removeRoute()

When a signal arrives, each enabled route with that source does five things in order: take the window ((value - inMin) / (inMax - inMin), clamped to 0..1), invert if asked, apply the curve, scale into outMin..outMax, and write. With smooth: 0 the param is overridden at once. With smooth > 0 the result becomes a target that update(dt) glides toward: after smooth seconds the param has covered about 63% of the way. If the target param is a pulse param, the route fires it instead: on every event of a pulse signal, or when a continuous source rises past 0.5.

Learn is generalised MIDI-learn: mapper.learn('bloom'), then move something. The first signal that moves by at least 5% of its declared range (a knob, a hand, a phone tilt), or the first pulse that fires (a key, a pad), becomes the source of a new route with the default settings. Calling learn('bloom') again cancels it. Because any pulse binds at once, stop the drum machine before you learn.

Before you call it

Create the signals and the params first, then the mapper. A smoothed route (smooth > 0) writes its param only inside update(dt), so call mapper.update(dt) every frame (createShow() does).

Parameters

NameTypeDescription
signalsSignalsThe Signals registry to listen to. The mapper subscribes to every signal at once, so a route works the moment it is added.
paramsParamsThe Params to write into. A route writes its target as an override.
profileStringThe name this mapper saves its routes under in localStorage (openav.map.<profile>), and the world name in an exported map file. createShow() uses the world's name. optional default 'default'
onChangeFunction | nullCalled with no arguments whenever the routes change or the import report changes (for a UI to redraw). optional default null
onLearnFunction | nullCalled as onLearn(route, signalName) when learn mode binds a signal. optional default null

Members

NameDescription
addRoute(r)methodAdd a route and return it, with the defaults filled in and an id assigned. outMin and outMax default to the param's range as it is right now.
removeRoute(id)methodRemove the route with this id.
routesFor(target)methodThe routes that write this param.
learn(target)methodToggle learn mode for a param: the next signal that moves (or the next pulse) becomes a new route to it. mapper.learnTarget is the armed key, or null.
update(dt)methodCall every frame with the seconds that passed: it advances the smoothed routes toward their targets.
toJSON()methodThe routes as a plain array (without the internal _last and _target), ready for JSON.stringify.
fromJSON(arr)methodReplace all routes with this array (the shape toJSON() returns). Defaults are not filled in; use importMap() for a file.
exportMap({ world = this.profile, now } = {})methodThe portable map file for the current routes: { format, version, world, exported, routes }. world defaults to the profile name.
importMap(input, { mode = 'replace', source = 'file', address = '', name = '' } = {})methodPut a map file on this mapper: the JSON text or an already parsed value, mode 'replace' (default) or 'add'. Routes whose param is missing, or whose signal this show cannot produce, are skipped and listed. An unreadable file changes nothing. Returns the report (also kept in mapper.report) and saves at once.
clear()methodRemove every route, and forget them: the saved profile is emptied too.
setReport(rep)methodRemember what the last import said and tell the UI (onChange). A failure shows up here, never as an exception.
save()methodSave the routes to localStorage under openav.map.<profile>. Every access is wrapped: blocked storage silently saves nothing.
load()methodLoad the saved routes; true when something was found.
dispose()methodStop listening to the signals.

Notes

  • A route writes its param as an override, and overrides are sticky: as soon as a mapped signal moves, the timeline stops driving that param until params.clearOverride(key).
  • A smoothed route keeps writing its last target every frame once its signal has arrived, so its param stays overridden even after clearOverride and even when the route is muted with enabled = false. Remove the route to give the param back to the timeline.
  • With createShow(), routes are added before the sound engine registers its sound/* params, so a route to sound/cutoff needs an explicit outMin: 100, outMax: 8000.
  • More, with the saved-profile rules: Mapping.

Examples

js
import { Signals, Params } from '@openav/core';
import { Mapper } from '@openav/mapping';

const signals = new Signals();
const params = new Params([{ key: 'bloom', min: 0, max: 100, def: 10 }]);
const mapper = new Mapper({ signals, params });

mapper.addRoute({ source: 'midi/cc/74', target: 'bloom', curve: 'exp' });
signals.set('midi/cc/74', 0.5);
console.log(params.resolve().bloom);   // 25: 0.5 through the exp curve is 0.25 of the range 0..100
module The console prints 25: the knob is halfway, the exp curve squares it to 0.25, and that is a quarter of the way along 0..100.
js
import { Signals, Params } from '@openav/core';
import { Mapper } from '@openav/mapping';

const signals = new Signals();
const params = new Params([{ key: 'hue', min: 0, max: 360, def: 0 }]);
const mapper = new Mapper({ signals, params });

mapper.addRoute({ source: 'audio/rms', target: 'hue', inMax: 0.3, smooth: 0.5 });
signals.set('audio/rms', 0.3);                       // a voice at 0.3 is the whole range
console.log(params.resolve().hue);                   // 0: nothing moved yet
mapper.update(0.5);                                  // half a second = one time constant
console.log(params.resolve().hue.toFixed(1));        // 227.6: about 63% of 360
module A smoothed route only moves inside update(dt): the console prints 0, then 227.6 after half a second (63% of the way to 360).
js
import { Signals, Params } from '@openav/core';
import { Mapper } from '@openav/mapping';

const signals = new Signals();
const params = new Params([{ key: 'bloom', min: 0, max: 100, def: 10 }]);
const mapper = new Mapper({ signals, params, onLearn: (route, name) => console.log('bound', name) });

signals.define('knob/a');
signals.set('knob/a', 0.5);
mapper.learn('bloom');                  // arm learn for the param
signals.set('knob/a', 0.51);            // moved 1%: not enough
signals.set('knob/a', 0.8);             // moved 30%: bound
console.log(mapper.routesFor('bloom').map((r) => r.source), mapper.learnTarget);
module Learn waits for a real movement: the console prints bound knob/a, then [ 'knob/a' ] null (learn disarms itself once it has bound a signal).
js
import { Signals, Params } from '@openav/core';
import { Mapper } from '@openav/mapping';

function rig(profile) {
  const signals = new Signals();
  signals.define('audio/rms');
  const params = new Params([{ key: 'bloom', min: 0, max: 100, def: 0 }]);
  return { signals, params, mapper: new Mapper({ signals, params, profile }) };
}
const a = rig('garden');
a.mapper.addRoute({ source: 'audio/rms', target: 'bloom', curve: 'log', inMax: 0.3 });
const file = a.mapper.exportMap();                    // the portable file
const report = rig('garden').mapper.importMap(file);  // on another machine or in another page
console.log(file.format, file.version, report.ok, report.added, report.skipped);
module A wiring travels as a file: the console prints openav.map 1 true 1 [].

Related

  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
  • LearnWizard Binds a list of params one after another: it arms learn for the first, you turn a knob, it binds it and arms the next. Skip, back and stop steer it.
  • parseMap() Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
  • serializeMap() Builds the portable map file object from a list of routes: { format, version, world, exported, routes }.
  • loadMapUrl() Fetches a map file from an address (what ?map= calls) and puts it on the mapper; it never throws and never blocks the show.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • approach() One step of a one-pole lag: moves current toward target, closing 63% of the gap after seconds.

import { Mapper } from '@openav/mapping'; packages/mapping/index.js, line 146

parseMap()#

function@openav/mapping
parseMap(input)

Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.

Use it to validate a file before you import it, or to show what is inside one. It refuses, with a plain-language error, text that is not JSON, text longer than MAP_MAX_BYTES, an object whose format is not MAP_FORMAT, a missing or newer version, no routes list, and more than 500 routes.

What it accepts is cleaned: a route without a string signal and param is dropped (and counted in dropped), unknown keys disappear, numbers must be finite, an unknown curve becomes 'linear', smooth is held to 0..60 seconds, and only invert: true inverts. A route is on unless it says enabled: false.

Parameters

NameTypeDescription
inputString | Object | ArrayThe JSON text of a map file, an already parsed map object, or a bare array of routes (version 0).

Returns

Object { ok: true, version, world, routes, dropped } or { ok: false, error }. Each route is { signal, param, curve, inMin, inMax, outMin?, outMax?, smooth, invert, enabled }; dropped counts the entries that were not routes.

Examples

js
import { parseMap } from '@openav/mapping';

const good = parseMap('{"format":"openav.map","version":1,"routes":[{"signal":"audio/rms","param":"bloom","curve":"wobble"},{"signal":7}]}');
console.log(good.ok, good.routes[0].curve, good.dropped);   // true linear 1
console.log(parseMap('not json'));                          // { ok: false, error: 'not valid JSON' }
console.log(parseMap('{"format":"other","version":1}'));   // refused: not an OAV map
module The console prints true linear 1 (the unknown curve became linear, the broken route was dropped), then the two refusals with their reasons.

Related

  • serializeMap() Builds the portable map file object from a list of routes: { format, version, world, exported, routes }.
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • MAP_FORMAT The text "openav.map": the value of format in every map file. A file with another format is refused.
  • MAP_VERSION The newest map file version this build reads and writes: 1. A file with a higher version is refused with a message.
  • MAP_MAX_BYTES The longest map text parseMap() will read: 262144, which is 256 KB. Longer text is refused without being parsed.
  • loadMapUrl() Fetches a map file from an address (what ?map= calls) and puts it on the mapper; it never throws and never blocks the show.

import { parseMap } from '@openav/mapping'; packages/mapping/index.js, line 73

parseMapUrl()#

function@openav/mapping
parseMapUrl(value, base)

Decides whether a ?map= address may be fetched: same origin, or an absolute https URL, and nothing else.

A map file named in the page address is somebody else's data, so the rules are strict. Accepted: an address on the same origin as base (any scheme, so plain http://localhost works for a page served from there), or an absolute https: address. Refused: an empty address, one longer than 2048 characters, one that is not a URL, one carrying a user name or password, http: to another host, and data:, javascript:, file:, blob: and every other scheme. loadMapUrl() runs this check before it fetches anything.

Parameters

NameTypeDescription
valueStringThe address as typed: absolute, or relative to base.
baseStringThe page address that relative addresses resolve against, and that "same origin" is judged by. Usually location.href.

Returns

Object { ok: true, url } with the resolved address, or { ok: false, error }.

Examples

js
import { parseMapUrl } from '@openav/mapping';

const base = 'https://openaudiovisual.com/examples/01-hello-particles/';
console.log(parseMapUrl('maps/stage-left.json', base));            // ok: same origin, resolved
console.log(parseMapUrl('https://example.com/my-map.json', base));  // ok: https
console.log(parseMapUrl('http://example.com/my-map.json', base));   // refused: http to another site
console.log(parseMapUrl('javascript:alert(1)', base));              // refused: not http or https
module The console prints two accepted results (the first with the full resolved address https://openaudiovisual.com/examples/01-hello-particles/maps/stage-left.json) and two refusals with their reasons.

Related

  • loadMapUrl() Fetches a map file from an address (what ?map= calls) and puts it on the mapper; it never throws and never blocks the show.
  • parseMap() Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.

import { parseMapUrl } from '@openav/mapping'; packages/mapping/index.js, line 109

serializeMap()#

function@openav/mapping
serializeMap(routes, { world = 'default', now = new Date() } = {})

Builds the portable map file object from a list of routes: { format, version, world, exported, routes }.

Routes in a map file are named by signal and param, never by id, so the file can travel between worlds. mapper.exportMap() calls this for you; use serializeMap() directly when you build routes by hand or want a fixed date. Missing inMin/inMax become 0 and 1, missing curve is 'linear', and the output range is 0..1 unless the route has outMin/outMax.

Parameters

NameTypeDescription
routesArrayRoutes in the mapper's own shape (what mapper.toJSON() returns: source, target, inMin, outMax …).
worldStringThe world name written into the file. Importing into another world drops the output ranges. optional default 'default'
nowDate | StringThe time written as exported (an ISO string). Pass a fixed date to get the same file twice. optional default new Date()

Returns

Object { format: 'openav.map', version: 1, world, exported, routes: [{ signal, param, curve, range: { in: [inMin, inMax], out: [outMin, outMax] }, smooth, invert }] }. A muted route also carries enabled: false.

Examples

js
import { serializeMap } from '@openav/mapping';

const file = serializeMap(
  [{ source: 'midi/cc/74', target: 'bloom', curve: 'exp', outMax: 100 }],
  { world: 'garden', now: new Date('2026-10-19T00:00:00Z') },
);
console.log(JSON.stringify(file));
module The console prints the whole file as one line of JSON, starting {"format":"openav.map","version":1,"world":"garden","exported":"2026-10-19T00:00:00.000Z","routes":[{"signal":"midi/cc/74","param":"bloom","curve":"exp","range":{"in":[0,1],"out":[0,100]} ….

Related

  • parseMap() Reads and cleans a map file (JSON text or a parsed value) without touching any mapper; it never throws.
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • MAP_FORMAT The text "openav.map": the value of format in every map file. A file with another format is refused.
  • MAP_VERSION The newest map file version this build reads and writes: 1. A file with a higher version is refused with a message.

import { serializeMap } from '@openav/mapping'; packages/mapping/index.js, line 54

Core: signals, params, stage, show#

The bus every package talks through, worlds, createShow(). 14

@openav/core

Bus#

class@openav/core
new Bus()

A minimal event emitter: on(event, cb), emit(event, payload), off(event, cb). No wildcards, no async.

new Bus() is a tiny publish and subscribe helper for your own code: subscribe to an event name with on(), fire it with emit(), and every callback gets the one payload you pass. Callbacks run in the order they subscribed, synchronously, inside emit().

It is not the thing shows talk through. Inputs and worlds talk through Signals (named values with a range) and Params; nothing else in the framework uses a Bus, and createShow() does not create one.

Members

NameDescription
on(event, cb)methodSubscribe: on(event, cb). Returns a function that unsubscribes.
off(event, cb)methodUnsubscribe the callback you passed to on().
emit(event, payload)methodFire an event with one payload: emit(event, payload). A callback that throws is caught and logged as [bus] <event> handler:; the rest still run.

Examples

js
import { Bus } from '@openav/core';

const bus = new Bus();
const off = bus.on('hit', (strength) => console.log('hit', strength));
bus.emit('hit', 0.8);   // hit 0.8
off();                  // unsubscribe
bus.emit('hit', 0.9);   // nothing: nobody is listening
module The console prints hit 0.8 once; after unsubscribing, the second emit is silent.

Related

  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.

import { Bus } from '@openav/core'; packages/core/src/bus.js, line 2

Loop#

class@openav/core
new Loop(onFrame, { maxDt = 0.1 } = {})

The frame clock: calls your function every animation frame with the seconds since the last one, and keeps ticking when the tab is hidden.

new Loop((dt) => { … }).start() runs your function once per requestAnimationFrame, with dt in seconds. It is the one clock a show has: the timeline advances, mapped signals settle, the world draws and the backstage feed is sent from inside it, in the order the architecture page lists. createShow() builds and starts one for you and returns it as show.loop.

Browsers freeze requestAnimationFrame in a hidden tab, but a performance must not stop because someone switched windows. So while the loop runs, a Web Worker beats every 33 ms; whenever no animation frame has arrived for 50 ms, the loop steps itself. A hidden tab therefore ticks at roughly 15 frames a second with a normal-sized dt, and show time stays real time. Where the page forbids workers (a strict content security policy) that fallback is simply absent.

loop.fps is the frame rate, rounded, refreshed about every half second (the very first value comes from the first frame alone); it is 0 before the first frame. loop.running says whether it is going.

Before you call it

Call start(): nothing ticks until you do. It needs a browser (requestAnimationFrame, performance); in Node there is no clock to run it.

Parameters

NameTypeDescription
onFrameFunctionCalled as onFrame(dt, now): dt is the seconds since the previous frame (never more than maxDt), now the time in milliseconds. An error thrown inside is caught and logged as [loop]; the clock keeps going.
maxDtNumberThe longest dt you will ever be handed, in seconds. A stall (a long garbage collection, a dragged window) is cut to this, so a world never has to cope with a one-second jump. optional default 0.1

Members

NameDescription
start()methodStart the clock. Calling it while it already runs does nothing.
stop()methodStop the clock and shut the hidden-tab worker down. start() begins again.

Notes

  • Do not read the real clock inside a world: take time from dt (or from the timeline), so a show at 30 fps and one at 144 fps do the same thing.

Examples

js
import { Loop } from '@openav/core';

let elapsed = 0;
const loop = new Loop((dt) => {
  elapsed += dt;
  document.getElementById('app').textContent = elapsed.toFixed(1) + ' s  ·  ' + loop.fps + ' fps';
  if (elapsed > 3) loop.stop();
});
loop.start();
module A clock counts up in the page and stops itself after three seconds.
js
import { Loop } from '@openav/core';

const ctx = document.getElementById('c').getContext('2d');
let x = 0;
const loop = new Loop((dt) => {
  x = (x + 120 * dt) % 400;          // 120 pixels a second, whatever the frame rate
  ctx.fillStyle = '#111'; ctx.fillRect(0, 0, 400, 300);
  ctx.fillStyle = '#7ea6ff'; ctx.fillRect(x, 130, 40, 40);
}, { maxDt: 0.05 });
loop.start();
module A square crosses the canvas at 120 pixels a second, the same speed at any frame rate because it moves by dt.

Related

  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • Timeline Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.

import { Loop } from '@openav/core'; packages/core/src/loop.js, line 7

Params#

class@openav/core
new Params(schema = [])

The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.

A param is the way a world takes continuous control: { key: 'bloom', min: 0, max: 1, def: 0.5 }. The world never asks who set it. Every frame each param resolves to one number: the override if there is one, otherwise the base value (usually the Timeline's automation at the current time), otherwise def. resolve() does exactly that and hands the world a { key: value } object.

An override is sticky. After params.override('bloom', 0.8) (what a slider, a MIDI knob or a mapped route does) the timeline stops moving bloom until clearOverride('bloom'). That matches the stage: when a person grabs a control, the score gives way. Overrides are clamped into min..max and snapped to step when they are written.

A param with pulse: true is a trigger, not a level: nobody reads a value from it. Fire it with firePulse(key) (a pad, a route from a pulse signal, a console button) and listen with onPulse(key, cb).

In a show, createShow() builds one Params and registers every world's schema into it with add(), then returns it as show.params.

Parameters

NameTypeDescription
schemaArrayThe param declarations: { key, label?, min, max, def, step?, pulse?, group? }. key is required; min, max and def give the range and the value when nothing else sets it. The array is kept, not copied. optional default []

Members

NameDescription
add(entries)methodRegister more params: add(entries). A key that already exists is skipped, so the first declaration wins. Returns the Params itself.
get(key)methodThe declaration of a param ({ key, min, max, def, … }), or undefined. It is not the current value: read that from resolve().
clamp(key, v)methodclamp(key, v) holds v inside the param's min..max and rounds it to a multiple of step. An unknown key returns v unchanged.
override(key, v)methodoverride(key, v) sets a sticky override, clamped like clamp(). A key that was never declared is ignored without a word.
clearOverride(key)methodGive one param back to the timeline.
clearAllOverrides()methodGive every param back to the timeline.
isOverridden(key)methodWhether a param currently has an override (the console shows these labels in yellow).
firePulse(key)methodFire a pulse param: every callback registered with onPulse(key, …) runs with the key.
onPulse(key, cb)methodonPulse(key, cb) runs cb(key) each time that pulse param fires. Returns a function that stops listening.
resolve(base = {})methodresolve(base = {}) gives { key: value } for every declared param: the override, else base[key] (for example timeline.state()), else def. A pulse param has no def, so its value is undefined.

Notes

  • step snaps with ordinary floating-point arithmetic: with step: 0.1, an override of 0.3 reads back as 0.30000000000000004. On a timeline a stepped param also holds each keyframe instead of interpolating.
  • resolve() does not clamp the base value, only overrides are clamped when written. Keep your automation inside min..max.
  • The fields a world may declare (label, options, surface, group …) are listed in Writing a world.

Examples

js
import { Params } from '@openav/core';

const params = new Params([{ key: 'bloom', min: 0, max: 1, def: 0.5 }]);
console.log(params.resolve().bloom);          // 0.5  (no automation, no override: def)
params.override('bloom', 0.8);                // what a slider or a mapped knob does
console.log(params.resolve({ bloom: 0.1 }).bloom);   // 0.8  (the override beats the base)
params.clearOverride('bloom');
console.log(params.resolve({ bloom: 0.1 }).bloom);   // 0.1  (the base is back)
module The console prints 0.5, 0.8, 0.1: def, then an override that beats the timeline, then the timeline again.
js
import { Params } from '@openav/core';

const params = new Params([
  { key: 'mode', min: 0, max: 3, def: 0, step: 1 },
  { key: 'burst', pulse: true },
]);
params.override('mode', 1.6);
console.log(params.resolve().mode);            // 2  (clamped and snapped to the step)
params.onPulse('burst', (key) => console.log('fired', key));
params.firePulse('burst');                     // fired burst
module A stepped param snaps (the console shows 2) and a pulse param fires its listener (fired burst).
js
import { Params } from '@openav/core';

const params = new Params([{ key: 'bloom', min: 0, max: 1, def: 0.5 }]);
console.log(params.get('bloom'));                       // the declaration, not a value
params.override('bloom', 5);
console.log(params.resolve().bloom);                    // 1  (clamped to max)
params.override('typo', 0.3);                           // unknown key: ignored
console.log(params.isOverridden('typo'));               // false
module The gotchas: get() returns the declaration { key, min, max, def }, an override is clamped to the range (1), and an unknown key is silently ignored (false).

Related

  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • Timeline Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • Stage Runs one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { Params } from '@openav/core'; packages/core/src/params.js, line 12

Signals#

class@openav/core
new Signals()

The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).

new Signals() makes an empty registry. In a show you rarely build it yourself: createShow() creates one and returns it as show.signals. A signal is a name plus a value. Input packages publish with set() and pulse(); everything else reads with get() and norm(), or subscribes with on() and onAny().

There are two kinds. A continuous signal (the default) always holds a current value, usually between 0 and 1: a knob, the loudness of the room, the height of a wrist. A pulse signal (kind: 'pulse') is an event: its value is the payload of the last firing and it is null until the first one: a key struck, a kick heard, a pinch started. Read a level whenever you like; subscribe to an event with on().

Names are paths with the source first: midi/cc/74, audio/rms, pose/hand/right/y, breath/pressure. Declare a signal with its range before you publish it: signals.define('breath/pressure', { min: 0, max: 1 }). set() on a name nobody declared defines it on the spot as a continuous signal from 0 to 1, so declare first whenever your numbers are in degrees, MIDI notes or any other range.

A knob, a raised hand and a chord are equal citizens here. Whoever reads a signal (the Mapper, a world that listens for events, the monitor) does not know and does not care what produced it.

Members

NameDescription
define(name, meta = {})methodDeclare a signal and its range: define(name, { kind, min, max, unit, description, source }). Safe to call again: later declarations merge into earlier ones. A continuous signal starts at min, a pulse at null. Returns the registry.
set(name, value)methodPublish a value: set(name, value). Subscribers run every time, even when the value did not change. An undeclared name is defined on the spot as continuous 0..1.
pulse(name, payload = 1)methodFire an event: pulse(name, payload = 1). It is set() under another name; declare the signal with kind: 'pulse' first, or it is registered as continuous.
get(name)methodThe last value of a signal, or undefined for a name that was never declared or set. A pulse that has not fired yet is null.
norm(name)methodThe value as 0..1 inside the declared min and max, clamped. An unknown name, or a signal whose min equals its max, gives 0.
on(name, cb)methodSubscribe to one signal: on(name, cb), where cb(value, name) runs on every set() or pulse(). Returns a function that unsubscribes.
onAny(cb)methodSubscribe to every signal: cb(name, value, meta) runs for each one. The mapper's learn mode and the monitor use it. Returns a function that unsubscribes.
list()methodEvery signal as [{ name, value, at, kind, min, max, unit, description, source }]; at is the time of the last set() (performance.now(), 0 if never). What the monitor and meters read.

Notes

  • signals.meta is a Map of name → declaration and signals.values a Map of name → last value. Read them if you must; write through define() and set() so subscribers hear about it.
  • A subscriber that throws is caught and logged as [signals] <name>; the other subscribers still run and set() does not throw.
  • The names every built-in package publishes are listed in the signals reference.

Examples

js
import { Signals } from '@openav/core';

const signals = new Signals();
signals.define('breath/pressure', { min: 0, max: 1 });
signals.on('breath/pressure', (v) => console.log('pressure', v));
signals.set('breath/pressure', 0.4);   // logs: pressure 0.4
module Open the console: it prints pressure 0.4, because the subscriber ran as soon as the value was set.
js
import { Signals } from '@openav/core';

const signals = new Signals();
signals.define('temp/room', { min: 10, max: 30, unit: '°C' });
signals.set('temp/room', 25);
console.log(signals.get('temp/room'), signals.norm('temp/room'));   // 25 0.75

signals.define('breath/sigh', { kind: 'pulse' });
signals.on('breath/sigh', (payload) => console.log('sigh', payload));
signals.pulse('breath/sigh', { depth: 0.8 });                       // sigh { depth: 0.8 }
module A level and an event side by side: the console shows 25 0.75, then sigh { depth: 0.8 }.
js
import { Signals } from '@openav/core';

const signals = new Signals();
signals.set('wind/speed', 12);              // never declared, so it is 0..1
console.log(signals.norm('wind/speed'));    // 1  (12 is far past the range)
signals.define('wind/speed', { min: 0, max: 20 });
console.log(signals.norm('wind/speed'));    // 0.6
module The gotcha: an undeclared signal is 0..1, so 12 reads as 1. Declare the range and the same value reads 0.6 (the console shows 1, then 0.6).

Related

  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
  • Loop The frame clock: calls your function every animation frame with the seconds since the last one, and keeps ticking when the tab is hidden.
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • Bus A minimal event emitter: on(event, cb), emit(event, payload), off(event, cb). No wildcards, no async.
  • ChordDetector Groups notes struck close together into one gesture and analyses it: chord type, consonance, triad and cluster flags, published as chord/* signals.

import { Signals } from '@openav/core'; packages/core/src/signals.js, line 19

@openav/show

createShow()#

function@openav/show
async createShow({ world = null, worlds = [], timeline: timelineCfg = { total: 120 }, routes = [], modules = {}, artwork = null, hint = '', profile = null, map = null, onFrame = null, mount = null, telemetry = true, score: scoreCfg = null, } = {})

Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

Every example is a world plus one createShow() call. It builds the Signals registry, the Params, the Stage with your world, the Timeline, the Mapper (with your routes), the inputs and outputs you declare in modules, the director's desk (mountConsole()), the backstage feed (MonitorFeed) and the frame Loop, starts the loop, and hands you every part. Nothing is hidden: show.signals, show.params, show.timeline, show.mapper, show.stage and the rest are the very objects the frame loop runs. When the framework grows a module, every show can have it just by declaring it.

The modules keys: midi (on by default, false turns it off, { controllers: true } or { controllers: { profile, profiles, open } } docks an on-screen MIDI controller), keys (the on-screen piano, QWERTY and the simulated performer: on by default, false turns them off, { base, octaves, sim, capture } tunes them), drums (true or { engine, autoEnableEngine }), audio (any truthy value: the microphone analyser, started from the 🎤 button), chord (true or { window }: a ChordDetector wired to every played note), hands (true: HandTracker), pose (true: PoseTracker), leap (true or { url, simulate, fingers, box }: LeapInput; ?leap=sim or ?leap=ws://… in the address overrides), remote (true or { room, … }: phones as controllers) and sound (true, an instrument id such as 'piano', or { instrument, remember, picker, engine }: instruments in the page, loaded only when declared). Parts that fetch things load lazily, so a show that does not declare them pays nothing.

The page shell is fixed once so every show is the same. Without mount, the page body becomes a two-column grid: the stage, and a 340 px side panel holding the piano and the console (an upright phone gets the stage on top and the panel below: see isStacked()). Every frame, in this order:

text
keys, drums, audio update   on-screen performers and the microphone
onFrame(dt, show)           your hook
timeline.advance(dt)        time moves
director.update(...)        with a score: segment modules enter, update, exit
mapper.update(dt)           smoothed routes settle
stage.frame(dt, state)      params resolve, world updates and renders
sound, remote               sound params reach the synth, values echo to phones
console.render(state)       the desk reflects reality
monitor.frame(snapshot)     the backstage hears about it

Before you call it

await it: it is async because it waits for the world's init() (and for lazily loaded modules). It needs a page (document, location, localStorage), and an import map or bundler that resolves every @openav/* package it imports (copy the one from examples/01-hello-particles/index.html). Browsers start audio only after a click, so sound and the microphone start from the console's buttons.

Parameters

NameTypeDescription
worldObject | nullThe world to perform: { name, params, init, update, render, dispose } (see Stage). It is activated first, and createShow waits for its init(). optional default null
worldsArrayMore worlds to register, after world. All of their params are registered up front; switch with show.stage.activate(name). optional default []
timelineObjectThe options of the Timeline: { total, automation, scenes, start, loop, onCue }. With a score, its scenes and total win. optional default { total: 120 }
routesArrayRoutes for the Mapper. If a saved profile exists it is loaded, and any declared route whose source → target pair is missing from it is still added. optional default []
modulesObjectThe inputs and outputs to switch on, by key (see the description): midi, keys, drums, audio, chord, hands, pose, leap, remote, sound. optional default {}
artworkObject | null{ title, artist, year, note }: a credit line at the bottom right of the stage and a <meta name="artwork"> tag in the head. optional default null
hintStringA line of HTML at the bottom left of the stage, for example 'play the piano · Space plays the timeline'. It is inserted as HTML, so never put untrusted text in it. optional default ''
profileString | false | nullThe name the mapper saves its routes under in localStorage (default: the first world's name, else 'show'). false ignores the saved routes when the show starts, so the routes come from your code on every load. optional default null
mapString | false | nullA map file to load on start: a same-origin or https address (see loadMapUrl()). Left out, ?map=<url> in the page address is used; false ignores ?map=. optional default null
onFrameFunction | nullCalled as onFrame(dt, show) every frame, after the on-screen performers and the microphone update and before the timeline advances. The place for assembly-level logic such as envelopes, or osc.flush(). optional default null
mountObject | null{ stage, side }: elements or selectors to render into instead of the generated full-page layout. optional default null
telemetryBooleanAllow one anonymous oav_show_start hit per page load. false sends nothing. optional default true
scoreObject | null{ cuts, cut, modules, api, onStatus, onCue, maxStep, midi }: the show's structure as segment lengths, performed by segment modules (see Score and Director). ?cut=<id> in the page address picks the version. optional default null

Returns

Promise<Object> The show: { signals, params, stage, timeline, mapper, midi, controllers, midiPanel, keys, drums, sound, audio, hands, pose, leap, chord, remote, score, director, scoreMidi, console, app, loop }. A part you did not ask for is null. With a score there is also show.show, the rehearsal controls. The same object is window.openav.

Notes

  • The mapper saves its routes under the profile name every 3 seconds, so a learned or edited route survives a reload and wins over a changed declaration in your code (the same source → target pair keeps its saved curve and smoothing). profile: false makes the routes come from code on every load, but the mapper still saves under the world's name from then on. To start clean, localStorage.removeItem('openav.map.<world name>').
  • Routes are added before the sound engine registers its sound/* params, so a declared route to sound/cutoff needs an explicit outMin: 100, outMax: 8000.
  • show.params.resolve() without an argument gives the def of every param. To read what the world sees this frame, pass the timeline's base state: show.params.resolve(show.timeline.state()).
  • Unless telemetry: false, one anonymous oav_show_start hit goes to Google Analytics per page load. Per the code it is silent on localhost, LAN addresses, file:// and openaudiovisual.com itself, and it honours Do Not Track and Global Privacy Control. Nothing played, no path, no cookie and no stable id is sent.
  • With a score, window.openav.show has table(), goto('dusk'), goto('dusk.glow'), goto(2), seek(T), cue(name), status(), next(), prev() and cuts, for rehearsal from the devtools console.

Examples

js
import { createShow } from '@openav/show';
import { createCanvas } from '@openav/stage';

const world = {
  name: 'dot',
  params: [{ key: 'size', label: 'Size', min: 0, max: 1, def: 0.3 }],
  init({ container }) { this.view = createCanvas(container); },
  update(dt, state) { this.size = state.size; },
  render() {
    const { ctx } = this.view, { w, h } = this.view.fit();
    ctx.fillStyle = '#000'; ctx.fillRect(0, 0, w, h);
    ctx.fillStyle = '#7ea6ff'; ctx.beginPath(); ctx.arc(w / 2, h / 2, this.size * h / 2, 0, Math.PI * 2); ctx.fill();
  },
  dispose() { this.view.dispose(); },
};

const show = await createShow({
  world,
  timeline: { total: 60, automation: { size: [[0, 0.1], [60, 0.9]] } },
  modules: { midi: false, keys: false },     // nothing to plug in
  telemetry: false,
});
show.timeline.play();                        // the dot grows over a minute
console.log(Object.keys(show).length, show.timeline.playing);
module A blue dot on the stage grows slowly while the timeline plays, with the desk (transport, layers, the size slider) on the right. The console prints 23 true: twenty-three parts, and the timeline is playing.
js
import { createShow } from '@openav/show';

const world = {
  name: 'glow',
  params: [{ key: 'size', min: 0, max: 1, def: 0.3 }, { key: 'burst', pulse: true }],
  init() {},
  update(dt, state) { this.last = state; },
};
const show = await createShow({
  world,
  routes: [{ source: 'midi/cc/1', target: 'size' }, { source: 'pad/1', target: 'burst' }],
  modules: { midi: false, keys: false },
  profile: false,                            // routes come from this code on every load
  telemetry: false,
});

show.params.onPulse('burst', () => console.log('burst!'));
show.signals.set('midi/cc/1', 0.9);          // what a knob would do
console.log(show.params.isOverridden('size'), show.params.resolve(show.timeline.state()).size);   // true 0.9
show.signals.set('pad/1', 0);
show.signals.set('pad/1', 1);                // a continuous source rising past 0.5 fires a pulse param
setTimeout(() => console.log('the world heard', world.last.size), 200);
module A signal drives a param through a route: the console prints true 0.9, then burst! as the pad crosses 0.5, then the world heard 0.9.
js
import { createShow } from '@openav/show';

const world = { name: 'sky', params: [{ key: 'sun', min: 0, max: 1, def: 0 }], init() {}, update() {} };
const dusk = {
  id: 'dusk',
  enter() { console.log('dusk enters'); },
  param: (key, value, ctx) => (key === 'sun' ? 1 - ctx.p : value),     // the segment bends the sun param
};
const show = await createShow({
  world,
  modules: { midi: false, keys: false },
  telemetry: false,
  profile: false,
  score: { cuts: [{ id: 'full', segments: [{ id: 'dawn', dur: 10 }, { id: 'dusk', dur: 10 }] }], modules: [dusk] },
});
console.log(show.score.total, show.timeline.total);     // 20 20: the timeline takes its length from the score
show.show.goto('dusk');                                  // rehearsal control: jump to a segment
setTimeout(() => console.log(show.show.status().segment.id, show.timeline.t), 300);
module With a score the timeline takes its length from it: the console prints 20 20, then dusk enters after the jump, and dusk 10.
js
import { createShow } from '@openav/show';

const show = await createShow({
  world,                                         // your world
  timeline: { total: 120, automation, scenes },
  routes: [{ source: 'audio/rms', target: 'energy', inMax: 0.3, smooth: 0.1 }],
  modules: { keys: { base: 48 }, chord: true, audio: true, sound: 'piano' },
  artwork: { title: 'Untitled', artist: 'You', year: 2026 },
  hint: 'play the piano · Space plays the timeline',
});
show.signals.on('chord/event', (a) => console.log(a.chordType));
static The whole set: an on-screen piano, chord analysis, the microphone and a grand piano that sounds after a click on the console's sound button. Play a chord and the console prints its type (major).

To be confirmed

The modules that need hardware, a permission or a download (microphone, camera, Leap, phones, the Tone.js sound engine, a MIDI device) were not run here; they are described from the code and the package docs. The examples switch midi and keys off so that they run anywhere.

Related

  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
  • Loop The frame clock: calls your function every animation frame with the seconds since the last one, and keeps ticking when the tab is hidden.
  • Stage Runs one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
  • Timeline Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • mountConsole() Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.
  • MonitorFeed Streams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.
  • Score A show written as segments with lengths, never as absolute seconds: it works out every start, end, cue and countdown, for every version (cut) of the show.
  • Director Runs a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
  • ChordDetector Groups notes struck close together into one gesture and analyses it: chord type, consonance, triad and cluster flags, published as chord/* signals.
  • LeapInput Reads Leap Motion hands from the local bridge and publishes them as leap/* signals, plus the joints (leap.hands) for drawing the hand itself.
  • isStacked() Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.

import { createShow } from '@openav/show'; packages/show/index.js, line 44

isStacked()#

function@openav/show
isStacked(width, height)

Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.

The shell of createShow() has one layout rule: wide windows get the stage and a 340 px side panel next to each other, and an upright phone or small tablet gets the stage on top and the panel below it, scrolling. The rule is the media query STACK_QUERY, and isStacked() is the very same decision as a plain function, so a test (or your own layout code) can use it without a browser. A phone on its side is never taller than wide, so it stays in the wide layout.

Parameters

NameTypeDescription
widthNumberThe window width in pixels.
heightNumberThe window height in pixels.

Returns

Boolean true when width <= 820 and height > width.

Examples

js
import { isStacked, stageHeight } from '@openav/show/layout';

console.log(isStacked(390, 844));     // true: a phone held upright
console.log(isStacked(844, 390));     // false: the same phone on its side
console.log(isStacked(820, 1180));    // true: a small tablet held upright
console.log(isStacked(1024, 1366));   // false: wider than 820, even though it is upright
console.log(stageHeight(844));        // 490: the stage keeps 58% of the window height
module The console prints true, false, true, false, then 490.

Related

  • stageHeight() The stage height in pixels for a window of this height when the layout is stacked: 58% of it, rounded.
  • STACK_QUERY The media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).
  • STACK_MAX_WIDTH The widest window that can get the stacked layout: 820 pixels (a small tablet held upright).
  • STAGE_SHARE How much of the window height the stage keeps when the layout is stacked: 58 (percent).
  • SHELL_CSS The CSS text createShow() puts in the page: the two-column grid, the 340 px side panel, the hint and credit styles, and the stacked layout.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { isStacked } from '@openav/show/layout'; packages/show/layout.js, line 16

SHELL_CSS#

constant@openav/show
SHELL_CSS = "\n  body { margin: 0; background: #000; height: 100vh; display: grid;\n    grid…"

The CSS text createShow() puts in the page: the two-column grid, the 340 px side panel, the hint and credit styles, and the stacked layout.

A string. createShow() adds it once as <style id="openav-shell-css"> (unless one with that id is already there). It sets the page body to a grid of the stage and a 340 px side panel, with the classes .oav-stage, .oav-side, .oav-keys-slot, .oav-desk, .oav-hint and .oav-credit, and the stacked layout from STACK_QUERY. Read it to see what the shell does to your page, or add a <style id="openav-shell-css"> of your own before calling createShow() to replace it entirely.

Examples

js
import { SHELL_CSS } from '@openav/show/layout';

console.log(SHELL_CSS.includes('.oav-stage'), SHELL_CSS.includes('340px'), SHELL_CSS.length > 500);   // true true true
console.log(SHELL_CSS.split('\n').slice(1, 4).join('\n'));      // the first rules: body is a grid
module The console prints true true true, then the first lines of the stylesheet (the body grid).

Related

  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • STACK_QUERY The media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).
  • isStacked() Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.

import { SHELL_CSS } from '@openav/show/layout'; packages/show/layout.js, line 25

STACK_MAX_WIDTH#

constant@openav/show
STACK_MAX_WIDTH = 820

The widest window that can get the stacked layout: 820 pixels (a small tablet held upright).

Used inside STACK_QUERY and isStacked(), so the CSS rule and the function cannot drift apart. Above 820 pixels the stage and the 340 px panel always sit side by side.

Related

  • STACK_QUERY The media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).
  • isStacked() Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
  • STAGE_SHARE How much of the window height the stage keeps when the layout is stacked: 58 (percent).

import { STACK_MAX_WIDTH } from '@openav/show/layout'; packages/show/layout.js, line 10

STACK_QUERY#

constant@openav/show
STACK_QUERY = "(max-width: 820px) and (orientation: portrait)"

The media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).

It is a string you can hand to matchMedia() to ask the browser the same question the shell's CSS asks: matchMedia(STACK_QUERY).matches. isStacked(width, height) is the same decision from numbers.

Examples

js
import { STACK_QUERY } from '@openav/show/layout';

const query = matchMedia(STACK_QUERY);
console.log(STACK_QUERY, query.matches);          // the window this runs in is not an upright phone: false
query.addEventListener('change', (e) => console.log('stacked now?', e.matches));
module The console prints the query and whether this window matches it (usually false on a desktop); the last line logs again whenever the window is resized across the rule.

Related

  • isStacked() Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
  • STACK_MAX_WIDTH The widest window that can get the stacked layout: 820 pixels (a small tablet held upright).
  • STAGE_SHARE How much of the window height the stage keeps when the layout is stacked: 58 (percent).
  • SHELL_CSS The CSS text createShow() puts in the page: the two-column grid, the 340 px side panel, the hint and credit styles, and the stacked layout.

import { STACK_QUERY } from '@openav/show/layout'; packages/show/layout.js, line 11

STAGE_SHARE#

constant@openav/show
STAGE_SHARE = 58

How much of the window height the stage keeps when the layout is stacked: 58 (percent).

The stage must stay the main thing on a phone, so it keeps 58% of the height and the console below it scrolls. stageHeight() turns this number into pixels.

Related

  • stageHeight() The stage height in pixels for a window of this height when the layout is stacked: 58% of it, rounded.
  • isStacked() Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
  • STACK_QUERY The media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).

import { STAGE_SHARE } from '@openav/show/layout'; packages/show/layout.js, line 13

stageHeight()#

function@openav/show
stageHeight(height)

The stage height in pixels for a window of this height when the layout is stacked: 58% of it, rounded.

When the shell stacks, the stage must stay the main thing, so it takes STAGE_SHARE percent of the window height (the CSS uses 58vh, and 58dvh where the browser has it) and the panel gets the rest. This is the same number as a function, for tests and your own layout code.

Parameters

NameTypeDescription
heightNumberThe window height in pixels.

Returns

Number A number: Math.round(height * 58 / 100).

Examples

js
import { stageHeight } from '@openav/show/layout';

for (const height of [667, 844, 1180]) console.log(height, stageHeight(height));   // 667 387, 844 490, 1180 684
module The console prints 667 387, 844 490 and 1180 684.

Related

  • isStacked() Whether a window of this size gets the stacked layout (stage on top, panel below): upright and no wider than 820 pixels.
  • STAGE_SHARE How much of the window height the stage keeps when the layout is stacked: 58 (percent).
  • STACK_QUERY The media query behind the stacked layout: (max-width: 820px) and (orientation: portrait).
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { stageHeight } from '@openav/show/layout'; packages/show/layout.js, line 21

@openav/stage

createCanvas()#

function@openav/stage
createCanvas(container, { alpha = false } = {})

Makes a 2D canvas that fills its container and keeps its size in step with it, at up to 2 times the device pixel ratio.

A convenience for worlds that draw with the 2D canvas API; a world may use it or ignore it. The canvas is display: block and as big as the container. fit() sets its pixel size to the container size times the device pixel ratio (never more than 2) and scales the context so you keep drawing in CSS pixels. Call fit() at the top of render(): it only does work when the size changed, and it returns { w, h } to draw with. A ResizeObserver also calls it when the container is resized.

Call dispose() in the world's dispose(): it stops the observer and removes the canvas.

Parameters

NameTypeDescription
containerHTMLElementThe element to put the canvas in (a world's container, usually).
alphaBooleanWhether the canvas may be transparent. The default false is faster and gives an opaque canvas. optional default false

Returns

Object { canvas, ctx, fit, dispose }: the <canvas>, its 2D context, fit() and dispose(). fit() returns { w, h }, the container size in CSS pixels.

Notes

  • If the container has no height (an empty <div> in normal flow is 0 pixels tall), the canvas is 0 × 0 and nothing shows. Give the container a size, or use it inside a Stage, whose layer fills the stage.

Examples

js
import { createCanvas } from '@openav/stage';

const container = document.getElementById('app');
container.style.cssText = 'width: 100%; height: 200px; background: #111;';
const view = createCanvas(container);

function render(time) {
  const { w, h } = view.fit();                         // the size in CSS pixels; cheap when nothing changed
  view.ctx.clearRect(0, 0, w, h);
  view.ctx.fillStyle = '#7ea6ff';
  view.ctx.fillRect(w / 2 - 30 + Math.sin(time / 500) * (w / 3), h / 2 - 30, 60, 60);
  requestAnimationFrame(render);
}
requestAnimationFrame(render);
module A blue square slides left and right across a canvas that fills a 200-pixel-high strip of the page, and stays sharp when the window is resized.

Related

  • Stage Runs one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.

import { createCanvas } from '@openav/stage'; packages/stage/index.js, line 75

Stage#

class@openav/stage
new Stage({ container, params, signals, io = {} })

Runs one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.

A world is any system with this shape, and every method is optional:

js
{
  name: 'garden',                                   // unique: the stage registers worlds by name
  params: [{ key: 'growth', min: 0, max: 1, def: 0.5 }],   // what it can be performed with
  init({ container, signals, params }) {},           // when it becomes active (may be async)
  update(dt, state, io) {},                          // every frame: state = the resolved params
  render() {},                                       // every frame, right after update
  dispose() {},                                      // when another world is activated
}

The stage owns the lifecycle and the param aggregation and deliberately not a renderer: draw with a 2D canvas, p5, three.js, SVG, the DOM, whatever you like. The shell never knows the work: switching works means swapping the world and its data, and the shell stays.

A world takes continuous control from state (the params) and may subscribe to signals for discrete events (a note struck, a chord resolved). If it reads signals.get('midi/cc/74') inside update(), it works with your controller today, but the timeline, a hand or another controller can no longer drive it. In a show, createShow() builds the stage from its world and worlds options and returns it as show.stage.

Before you call it

Register your worlds, then await stage.activate(name) once; until a world is active, frame() returns null and does nothing. Then call stage.frame(dt, baseState) every frame (a show's loop does).

Parameters

NameTypeDescription
containerHTMLElementThe element the worlds draw into. It becomes position: relative if it was static. The stage adds its own layer inside it, so overlays you put in the container (hints, start buttons) are not wiped when a world is switched. optional
paramsParamsThe Params the worlds' schemas are absorbed into, and that frame() resolves. optional
signalsSignalsThe Signals registry handed to every world, so it can subscribe to events. optional
ioObjectExtra output handles for the worlds' update(dt, state, io), such as { midi, osc }. signals is always there. optional default {}

Members

NameDescription
register(world)methodRegister a world by its name and add its params to the shared Params (a key that already exists keeps its first declaration). Returns the stage.
activate(name)methodSwitch to the world with this name. Async: it disposes the previous world (an error in its dispose() is logged, not thrown), empties the stage layer, then awaits the new world's init({ container, signals, params }). Does nothing if that world is already active; rejects with "unknown world" for a name that was never registered.
frame(dt, baseState)methodPer frame: frame(dt, baseState) resolves params.resolve(baseState) (override, else the base, else def), calls the active world's update(dt, state, io) and then render(), and returns the resolved state, or null with no active world. An exception in either call is caught and logged as [stage] update: or [stage] render:; the show keeps running.

Notes

  • Check the name before you activate: activate('nope') disposes the current world first and only then rejects, and the stage keeps pointing at the disposed world (active and activeName do not change), so the next frame() still calls its update() and render().
  • Fields to read: active (the active world object), activeName, worlds (a Map of name → world), container, layer (the element worlds are placed in) and io ({ signals, ...io }, what update receives; add handles later with stage.io.osc = osc).
  • An exception in init() is not caught: activate() rejects. Exceptions in update() and render() are caught. A world must release what it takes (GPU objects, DOM, subscriptions) in dispose(), because worlds get switched live.

Examples

js
import { Signals, Params } from '@openav/core';
import { Stage } from '@openav/stage';

const container = document.getElementById('app');
container.style.cssText = 'width: 400px; height: 120px;';          // give the stage some room

const stage = new Stage({ container, params: new Params(), signals: new Signals() });
stage.register({
  name: 'label',
  params: [{ key: 'size', min: 10, max: 60, def: 20 }],
  init({ container }) { this.el = document.createElement('div'); container.appendChild(this.el); },
  update(dt, state) { this.size = state.size; },
  render() { this.el.style.cssText = 'color: #7ea6ff; font: ' + this.size + 'px monospace'; this.el.textContent = 'size ' + this.size; },
  dispose() { this.el.remove(); },
});
await stage.activate('label');                      // init runs now
console.log(stage.frame(0.016, { size: 40 }));      // { size: 40 }: the base state, since nothing overrides it
console.log(stage.frame(0.016, {}));                // { size: 20 }: no base either, so def
module The console prints { size: 40 }, then { size: 20 }, and the page shows the text size 20 in blue: the world drew itself from the resolved params.
js
import { Signals, Params } from '@openav/core';
import { Stage } from '@openav/stage';

const signals = new Signals();
const stage = new Stage({ container: document.getElementById('app'), params: new Params(), signals });
const world = (name) => ({
  name, params: [{ key: 'calm', min: 0, max: 1, def: 0.5 }],
  init() { console.log(name, 'init'); },
  dispose() { console.log(name, 'dispose'); },
});
stage.register(world('calm')).register(world('storm'));
await stage.activate('calm');                  // calm init
await stage.activate('storm');                 // calm dispose, then storm init
await stage.activate('storm');                 // already active: nothing happens
console.log(stage.activeName, stage.params.schema.length);     // storm 1: both worlds share the one 'calm' param
await stage.activate('nope').catch((error) => console.log(error.message));   // unknown world: storm is disposed first, then this rejects
module The console prints calm init, calm dispose, storm init, then storm 1 (the shared param is registered once), then storm dispose and the error message for an unknown world: the failed switch has already disposed the world that was playing.

Related

  • createCanvas() Makes a 2D canvas that fills its container and keeps its size in step with it, at up to 2 times the device pixel ratio.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • webtoeWorld() Turns a WebToe node patch into a world: the patch runs in an iframe and every changed param is posted into it.
  • Timeline Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.

import { Stage } from '@openav/stage'; packages/stage/index.js, line 21

@openav/world-webtoe

webtoeWorld()#

function@openav/world-webtoe
webtoeWorld({ name = 'webtoe', app = 'https://webtoe.openaudiovisual.com/', project = null, params = [], extra = null, } = {})

Turns a WebToe node patch into a world: the patch runs in an iframe and every changed param is posted into it.

WebToe is the sister project: a web-native, node-based dataflow engine that speaks TouchDesigner. webtoeWorld() embeds its app in an iframe that fills the stage (with allow="camera; microphone"), and each frame it posts the resolved params into the iframe as { type: 'webtoe:ext', values }. Inside the patch, any parameter written as ext('name', fallback) follows the value called name, and uses fallback until one arrives.

The contract stays pure: the world reads params and never inputs. Which knob, hand or timeline drives speed is the mapper's business, as for any world. To send less, the world posts only when at least one value differs from the last post.

Before you call it

The page must be able to load the WebToe app in an iframe (the network, by default https://webtoe.openaudiovisual.com/), and the patch file must be reachable by the app. The world does its work when it is activated, so use it with createShow({ world }), or stage.register() and stage.activate().

Parameters

NameTypeDescription
nameStringThe world's name: what Stage registers it under and what you pass to stage.activate(). Give each patch its own. optional default 'webtoe'
appStringThe WebToe app to embed in the iframe. Change it to run your own copy of the app. optional default 'https://webtoe.openaudiovisual.com/'
projectString | nullThe address of a .webtoe.json patch file. It is handed to the app as ?project=<address>, and the app fetches it, so the file's server must allow the app's origin (CORS). Make the address absolute, for example new URL('./garden.webtoe.json', location.href).href. optional default null
paramsArrayThe param schema of the world: [{ key, min, max, def }, …]. The keys are the names the patch reads with ext('name', fallback). optional default []
extraFunction | nullextra(state, out), called every frame: add derived values to out (a copy of the state) that the patch can read too, such as out.hueRad = state.hue * Math.PI / 180. optional default null

Returns

Object A world { name, params, init, update, render, dispose } to pass to createShow({ world }) or stage.register().

Notes

  • The message goes to the iframe with the target origin *, so any page the iframe happens to be showing receives the values. If you point app at a site you do not trust, it can read your params.

Examples

js
import { webtoeWorld } from '@openav/world-webtoe';

// a stand-in for the WebToe app, so this needs no network: it just shows what it receives
const patch = '<body style="margin:0;background:#111;color:#7ea6ff;font:14px monospace"></body>'
  + '<script>onmessage = (e) => { document.body.textContent = JSON.stringify(e.data); };<\/script>';
const world = webtoeWorld({
  app: 'data:text/html,' + encodeURIComponent(patch),
  params: [{ key: 'speed', min: 0, max: 1, def: 0.5 }, { key: 'hue', min: 0, max: 360, def: 205 }],
  extra: (state, out) => { out.hueRad = Math.round(state.hue * Math.PI / 180 * 100) / 100; },
});
const container = document.getElementById('app');
container.style.cssText = 'position: relative; width: 400px; height: 120px;';
world.init({ container });
container.querySelector('iframe').onload = () => world.update(0.016, { speed: 0.8, hue: 180 });
module An iframe fills the container and, once it has loaded, shows the message the world posted: {"type":"webtoe:ext","values":{"speed":0.8,"hue":180,"hueRad":3.14}}.
js
import { createShow } from '@openav/show';
import { webtoeWorld } from '@openav/world-webtoe';

const world = webtoeWorld({
  name: 'garden-patch',
  project: new URL('./garden.webtoe.json', location.href).href,   // the app fetches it: CORS must allow it
  params: [
    { key: 'speed', min: 0, max: 1, def: 0.5 },
    { key: 'hue', min: 0, max: 360, def: 205 },
  ],
});
const show = await createShow({ world, routes: [{ source: 'midi/cc/1', target: 'speed' }] });
static The patch from garden.webtoe.json fills the stage, and turning MIDI knob 1 changes its speed through the mapper. Every ext('speed', …) in the patch follows.

To be confirmed

Not run against the real WebToe app here (the examples use a stand-in page, with no network). Because a value is only posted when it changed, a patch that finishes loading after the first frame may not receive values that never move again until one of them changes; whether that happens with the live app was not checked.

Related

  • Stage Runs one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.

import { webtoeWorld } from '@openav/world-webtoe'; packages/world-webtoe/index.js, line 20

Phones, relay and control surfaces#

@openav/remote, @openav/relay, @openav/surface. 40

@openav/relay

aliasOf()#

function@openav/relay
aliasOf(name, alias = 'any')

The "latest phone" name of a phone signal: aliasOf('phone/ab12/tilt/x') is 'phone/any/tilt/x'. null for anything else.

A route written in a World file cannot know the id of a stranger's phone. The alias fixes that: every phone signal is also filed under phone/any/…, which always holds the value from whichever phone sent last, so { source: 'phone/any/tilt/x', target: 'hue' } can be written before anyone has joined. bindSignals() and localSensors() both file the alias for you.

Parameters

NameTypeDescription
nameStringA signal name. Only phone/<id>/… has an alias.
aliasStringThe word that replaces the id. Leave it out for any. optional default 'any'

Returns

String | null The alias name, or null when name is not a phone signal or already uses the alias.

Examples

js
import { aliasOf } from '@openav/relay';

document.getElementById('app').innerHTML = [
  aliasOf('phone/ab12/tilt/x'),
  aliasOf('phone/any/tilt/x'),
  aliasOf('midi/note/on'),
  aliasOf('phone/ab12/knock', 'latest'),
].map(String).join('<br>');
module Prints phone/any/tilt/x, then null twice (already an alias; not a phone signal), then phone/latest/knock.

Related

  • bindSignals() Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
  • localSensors() For a show page that runs on the phone itself: the phone's own sensors go straight into a Signals as phone/local/…, with a phone/any/… copy.
  • fileSignal() Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.

import { aliasOf } from '@openav/relay'; packages/relay/signals.js, line 32

attachRelay()#

function@openav/relay
attachRelay(server, { path = '/relay', log = () => {}, heartbeatMs = 20000 } = {})

Attaches the room relay to a Node http.Server: WebSocket clients that connect to path are put in rooms and routed by role. Returns { hub, close() }.

This is the server half, in Node, with no dependencies (the WebSocket codec is in the package). node serve.js already does it for you at /relay. To use it in your own server: attachRelay(server) before or after server.listen(). Clients connect to ws://host/relay?role=controller|runner|monitor&room=<name>&id=<device>: a bad role becomes controller, a bad room name becomes default, and the id is cut to 64 characters.

Who may talk to whom: a controller's signal and batch go to the runner and the monitors; a runner's go to the monitors only; a runner's feedback and config go to the controllers and monitors; a monitor says nothing. A config is kept per key and replayed to controllers that join later, and the room forgets it when its runner leaves. Rooms never hear each other. A message over 256 KB is ignored and a frame over 1 MB closes the connection.

There is no authentication: anyone who can reach the server and knows a room name can join as a controller. Use a closed network, or a room name nobody will guess.

Before you call it

A Node program, imported from @openav/relay/server (a different entry point from @openav/relay, which is the browser side). Standalone: node packages/relay/server.js [port] (default 7458; /health answers with the room names; RELAY_HOST=127.0.0.1 binds to localhost behind a reverse proxy).

Parameters

NameTypeDescription
serverhttp.ServerA Node http.Server (or anything with the same upgrade event, such as an https server). The relay listens for its upgrade events.
pathStringThe URL path clients connect to. Any other path is refused. optional default '/relay'
logFunction(line) => {}, called with a line such as + controller ab12 @main when a client joins and - controller ab12 @main when it leaves. The default says nothing. optional default () => {}
heartbeatMsNumberHow often to ping every client, in milliseconds. A client that has not answered by the next ping is dropped, so a dead phone goes within one to two periods. optional default 20000

Returns

{ hub, close } hub is the RoomHub (hub.rooms is a Map of the rooms; hub.counts(room) gives { controllers, runners, monitors }). close() drops every connected client; it does not close your server.

Examples

js
import http from 'node:http';
import { attachRelay } from '@openav/relay/server';

const server = http.createServer((req, res) => { res.writeHead(404); res.end(); });
const relay = attachRelay(server, { log: (line) => console.log('[relay]', line) });
server.listen(7458, () => console.log('relay at ws://localhost:7458/relay'));

// later: relay.hub.counts('main') → { controllers: 1, runners: 1, monitors: 0 }
static A relay on port 7458 in about five lines. Open a show page with ?relay=ws://localhost:7458/relay and the terminal logs a line such as [relay] + runner show-k3x9 @default when it connects.

Related

  • RelayClient The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
  • bindSignals() Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
  • mountRemoteHost() The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.

import { attachRelay } from '@openav/relay/server'; packages/relay/server.js, line 20

bindSignals()#

function@openav/relay
bindSignals(client, signals, { alias = 'any', meta = signalMeta } = {})

Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.

Call it once on the show page, after making the runner's client: bindSignals(show, signals). It sets client.onSignal so that each signal that arrives goes through fileSignal(), and a phone signal is filed twice, under its own name (phone/ab12/tilt/x) and under the alias (phone/any/tilt/x). Routes written at author time use the alias.

A callback that was already in client.onSignal keeps running after the filing, so you can still watch the raw messages.

Parameters

NameTypeDescription
clientRelayClientThe runner's client (any object with an onSignal field works).
signalsSignalsThe registry to fill (from @openav/core).
aliasString | nullThe word used in the copy, phone/<alias>/…. null or an empty string turns the copy off. optional default 'any'
metaFunctionReplace signalMeta with your own declarations, as in fileSignal(). optional default signalMeta

Returns

Function Call it to unbind: client.onSignal goes back to what it was before.

Examples

js
import { bindSignals } from '@openav/relay';
import { Signals } from '@openav/core';

const signals = new Signals();
const client = {};                               // anything with an onSignal field; a RelayClient in real use
const unbind = bindSignals(client, signals);

// what the relay calls when a phone's message arrives
client.onSignal('phone/ab12/tilt/x', 0.4, {});
client.onSignal('phone/ab12/knock', { strength: 1 }, { pulse: true });
unbind();                                        // client.onSignal is back to what it was

document.getElementById('app').innerHTML = [
  'phone/any/tilt/x = ' + signals.get('phone/any/tilt/x'),
  'knock kind: ' + signals.meta.get('phone/any/knock').kind,
  'unbound: ' + (client.onSignal === undefined),
].join('<br>');
module Prints the tilt under the alias (0.4), the knock declared as a pulse, and "true" once unbind() has put onSignal back.
js
import { RelayClient, bindSignals } from '@openav/relay';
import { Signals } from '@openav/core';

const signals = new Signals();
const show = new RelayClient({ role: 'runner', room: 'main' });
const seen = [];
show.onSignal = (name) => seen.push(name);       // our own listener first
bindSignals(show, signals);                      // it still runs after the filing
show.onSignal('phone/ab12/tilt/y', 0.2, {});

document.getElementById('app').textContent = seen.join(', ') + ' | ' + signals.get('phone/any/tilt/y');
module A listener that was there before bindSignals() still hears every message: prints "phone/ab12/tilt/y | 0.2".

Related

  • RelayClient The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
  • fileSignal() Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
  • aliasOf() The "latest phone" name of a phone signal: aliasOf('phone/ab12/tilt/x') is 'phone/any/tilt/x'. null for anything else.
  • signalMeta() What the relay knows about a signal name: its range and unit, or that it is a pulse. Returns null for names it does not know.
  • mountRemoteHost() The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.

import { bindSignals } from '@openav/relay'; packages/relay/signals.js, line 65

deviceId()#

function@openav/relay
deviceId(key = 'lab.deviceId')

The short random id of this browser, kept in localStorage, that names this phone in its signals, as in phone/ab12/tilt/x.

The first call makes an id and stores it under key; later calls read it back. RelayClient uses it as its default id, so a phone keeps the same name across reloads and its signals stay phone/<same id>/….

If localStorage is blocked (some private windows), nothing can be stored and every call makes a new id.

Parameters

NameTypeDescription
keyStringThe localStorage key. The default is the one the lab uses, so /control and /remote on one phone count as one device. optional default 'lab.deviceId'

Returns

String A short id of about five letters and digits, such as k3x9a. The same one on every call, in the same browser.

Examples

js
import { deviceId } from '@openav/relay';

const first = deviceId();
const again = deviceId();
document.getElementById('app').textContent =
  `id ${first}, the same on the next call: ${first === again}`;
module Prints a short id and "true": the second call reads back what the first one stored.

Related

  • RelayClient The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
  • PhoneSensors Turns a phone's tilt, acceleration, rotation, knocks and front-camera light into named signals, in screen axes, and hands them to the io you give it.

import { deviceId } from '@openav/relay'; packages/relay/index.js, line 27

fileSignal()#

function@openav/relay
fileSignal(signals, name, value, { pulse = false, meta = signalMeta, source = '' } = {})

Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.

This is the receiving half of the wire format, and every receiver files a message the same way through it: bindSignals() for a show that has a relay, localSensors() for a show on a phone, and a host that runs its own socket. In order: if the registry has never heard the name, declare it with what signalMeta() says (a pulse is never mistaken for a knob, and meters get their range); fill in the missing velocity spelling with normalizeValue(); then signals.pulse(name, value) for a pulse or signals.set(name, value) for the rest.

Parameters

NameTypeDescription
signalsSignalsThe registry to file it in (from @openav/core).
nameStringThe signal name. It must be a non-empty string of at most 128 characters, or nothing is filed.
valueanyThe payload.
pulseBooleanTrue when the message was sent as an event. A name that signalMeta() knows to be a pulse (or was declared as one) is a pulse either way. optional default false
metaFunctionReplace signalMeta with your own (name, { pulse }) => metadata | null, to declare names the relay does not know. optional default signalMeta
sourceStringA label stored in the declared metadata (signals.meta.get(name).source), such as 'relay' or 'local'. optional default ''

Returns

Boolean true when it was filed, false for a bad name.

Examples

js
import { fileSignal } from '@openav/relay';
import { Signals } from '@openav/core';

const signals = new Signals();
fileSignal(signals, 'midi/note/on', { note: 64, velocity: 100 });   // a known pulse name
fileSignal(signals, 'phone/ab12/tilt/x', 0.3, { source: 'relay' });
const bad = fileSignal(signals, '', 1);

document.getElementById('app').innerHTML = [
  'note vel: ' + signals.get('midi/note/on').vel.toFixed(2),
  'note kind: ' + signals.meta.get('midi/note/on').kind,
  'tilt range: ' + signals.meta.get('phone/ab12/tilt/x').min + '..' + signals.meta.get('phone/ab12/tilt/x').max,
  'empty name filed: ' + bad,
].join('<br>');
module Prints a velocity of 0.79 (100 / 127) that the sender never wrote, the kind "pulse", the range -1..1 for tilt, and "false" for the empty name.

Related

  • bindSignals() Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
  • signalMeta() What the relay knows about a signal name: its range and unit, or that it is a pulse. Returns null for names it does not know.
  • normalizeValue() Fills in the missing velocity spelling of a note message: vel (0..1) or velocity (1..127). Every other message passes through unchanged.
  • localSensors() For a show page that runs on the phone itself: the phone's own sensors go straight into a Signals as phone/local/…, with a phone/any/… copy.
  • aliasOf() The "latest phone" name of a phone signal: aliasOf('phone/ab12/tilt/x') is 'phone/any/tilt/x'. null for anything else.

import { fileSignal } from '@openav/relay'; packages/relay/signals.js, line 55

normalizeValue()#

function@openav/relay
normalizeValue(name, value)

Fills in the missing velocity spelling of a note message: vel (0..1) or velocity (1..127). Every other message passes through unchanged.

@openav/midi publishes vel as 0..1, a hardware keyboard talks in velocity 1..127, and senders (an old phone page, a TouchOSC bridge, a sequencer) give one or the other. A sketch is written against one spelling, so the receiving end fills in the other: velocity becomes vel = velocity / 127, and vel becomes velocity = Math.round(vel * 127).

You rarely call it yourself: fileSignal() does it for every message it files.

Parameters

NameTypeDescription
nameStringThe signal name. Only midi/note/on, midi/note/off and the same with a device in the middle (midi/<device>/note/on) are touched.
valueanyThe payload. Only an object with vel or velocity is changed (a copy, never the original).

Returns

any The payload with both vel and velocity. If both were there already, the same object comes back untouched.

Examples

js
import { normalizeValue } from '@openav/relay';

const rows = [
  normalizeValue('midi/note/on', { note: 60, velocity: 127 }),
  normalizeValue('midi/note/on', { note: 60, vel: 0.5 }),
  normalizeValue('midi/minilab3/note/off', { note: 61, vel: 1 }),
  normalizeValue('surface/main/pad/hit', { velocity: 3 }),
];
document.getElementById('app').innerHTML = rows.map((r) => JSON.stringify(r)).join('<br>');
module The first two gain the spelling they lacked (vel 1 and velocity 64); the third works with a device in the name; the fourth is not a note and stays as it is.

Related

  • fileSignal() Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
  • bindSignals() Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.

import { normalizeValue } from '@openav/relay'; packages/relay/signals.js, line 43

RelayClient#

class@openav/relay
new RelayClient({ role = 'controller', room = 'default', id = deviceId(), url = null, batchHz = 30, resendMs = 1000, onStatus, onSignal, onFeedback, onConfig } = {})

The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.

Make one client per page, give it a role and a room, and call connect(). A phone calls phone.set('phone/me/tilt/x', 0.3) for values that change continuously and phone.send('midi/note/on', { note: 60 }, true) for events. The show passes its client to bindSignals(), and every phone signal lands in its Signals. The relay itself is attachRelay() on a Node server.

Continuous values are cheap on purpose: set() only stores the value (and does nothing if it is the same as the last), then every 1 / batchHz seconds the changed ones leave in one batch, and every resendMs all of them leave again. Events are the opposite: send() leaves at once and is never batched or repeated, because a note-on delivered twice is a bug.

It looks after the connection itself. When the socket closes it reconnects after 0.5 s, then 1 s, 2 s, 4 s and every 5 s, until you call close(). A ping goes out every 2 s and the answer becomes latency in milliseconds. status is 'connecting', 'open' or 'closed', and peers is how many controllers, runners and monitors the room has.

Before you call it

A relay server for it to reach, such as node serve.js (it hosts one at /relay) or node packages/relay/server.js. Until the socket is open, set() keeps the latest values and sends them when it opens, but send(), feedback() and config() messages are dropped, not queued.

Parameters

NameTypeDescription
role'controller' | 'runner' | 'monitor'Who this page is. A controller (a phone) sends signals to the runner and the monitors. A runner (the show) hears the controllers, and can send feedback and config back. A monitor only listens. The server treats any other word as controller. optional default 'controller'
roomStringThe room name. Pages in different rooms never hear each other. 1 to 32 characters of A–Z a–z 0–9 _ -; the server turns anything else into default. optional default 'default'
idStringThis device's id, the <id> in phone/<id>/…. The default is deviceId(), which is kept in localStorage. The server keeps its first 64 characters. optional default deviceId()
urlStringThe full WebSocket address, with the query already on it: wss://host/relay?role=controller&room=main&id=ab12. Nothing is added to it. Leave it out and the client uses ?relay=ws://… from the page's address, or else the page's own /relay (wss: on an https page), and adds role, room and id itself. optional default null
batchHzNumberHow many times a second the values you set() are flushed to the relay, as one batch message. optional default 30
resendMsNumberEvery this many milliseconds, send every stored value again, not only the changed ones. This is what lets a phone that joins late, or a show page that reloads, catch up. optional default 1000
onStatusFunction(client) => {}, called when the connection opens or closes, when a pong gives a new latency, and when the peer counts change. Read client.status, client.latency and client.peers inside it. optional
onSignalFunction(name, value, msg) => {}, for every signal that arrives (a runner or monitor hears the controllers; a monitor also hears the runner). A batch calls it once per item. msg.pulse is true for a pulse. bindSignals() fills this in for you. optional
onFeedbackFunction(name, value) => {}, on a controller: a value the runner sent with feedback(), to show in a meter or to move a fader. optional
onConfigFunction(key, data) => {}, on a controller: a standing config the runner sent with config(), such as the surface layout. A phone that joins late gets the stored ones right away. optional

Members

NameDescription
opengetterTrue while the socket is open (a getter). send() and config() do nothing when it is false.
connect()methodconnect(): open the socket and start the timers (flush, re-send, ping). Returns the client, so new RelayClient({…}).connect() works. Calling it again after close() starts over.
close()methodclose(): stop the timers and close the socket for good. It does not reconnect until you call connect() again.
send(name, value, pulse = false, extra)methodsend(name, value, pulse?, extra?): send one signal now. pulse true marks it as an event. extra is an object whose keys are added to the message. Dropped if the socket is not open.
set(name, value)methodset(name, value): a value that changes continuously. Only the latest value per name is kept; it goes out with the next flush and again every resendMs.
feedback(name, value)methodfeedback(name, value): runner only. A value for the controllers to display or to echo, flushed like set() and re-sent every resendMs. Nothing happens on a controller or a monitor.
config(key, data)methodconfig(key, data): runner only. Send a standing config at once, for example config('surface', { layout }). The relay keeps it per key for controllers that join later, and forgets it when the runner leaves.

Notes

  • Public fields: role, room, id, url, batchHz, resendMs, the callbacks, status, latency (milliseconds, null until the first pong), peers and sent (how many messages have gone out).
  • After a reconnect the runner has to send its config() again: the client does not remember it. mountRemoteHost() does that for you.

Examples

js
import { RelayClient, bindSignals } from '@openav/relay';
import { Signals } from '@openav/core';

// the show page: every phone signal lands in a Signals registry
const signals = new Signals();
const show = new RelayClient({ role: 'runner', room: 'main' }).connect();
bindSignals(show, signals);

// the phone page: a continuous value and an event
const phone = new RelayClient({ role: 'controller', room: 'main' }).connect();
phone.set('phone/me/tilt/x', 0.3);
phone.send('midi/note/on', { note: 60 }, true);
static The two ends of one room, as they would sit on two pages (the show and a phone): once the socket is open, the show's signals hold phone/me/tilt/x and, as an alias, phone/any/tilt/x.
js
import { RelayClient, bindSignals } from '@openav/relay';
import { Signals } from '@openav/core';

// A stand-in for the relay so this runs without a server: a controller's messages reach the runner,
// and the runner's feedback and config reach the controller (the same routing the real relay does).
const sockets = [];
globalThis.WebSocket = class {
  constructor(url) {
    this.role = new URL(url).searchParams.get('role');
    this.readyState = 1;
    sockets.push(this);
    setTimeout(() => this.onopen?.());
  }
  send(text) {
    const type = JSON.parse(text).type;
    const to = this.role === 'controller' ? 'runner' : (type === 'feedback' || type === 'config') ? 'controller' : null;
    for (const s of sockets) if (s.role === to) s.onmessage?.({ data: text });
  }
  close() {}
};

const signals = new Signals();
const show = new RelayClient({ role: 'runner', url: 'ws://relay/x?role=runner&room=demo&id=show' }).connect();
bindSignals(show, signals);
const got = [];
const phone = new RelayClient({
  role: 'controller', url: 'ws://relay/x?role=controller&room=demo&id=ab12',
  onFeedback: (name, value) => got.push('feedback ' + name + ' = ' + value),
  onConfig: (key, data) => got.push('config ' + key + ' = ' + JSON.stringify(data)),
}).connect();

phone.set('phone/ab12/tilt/x', 0.3);
show.feedback('surface/main/level', 0.4);
show.config('surface', { title: 'demo' });

setTimeout(() => {
  got.push('tilt in the show: ' + signals.get('phone/any/tilt/x'));
  document.getElementById('app').innerHTML = got.join('<br>');
  phone.close(); show.close();
}, 200);
module Prints the config and the feedback the phone got back from the show, then the phone's tilt as the show hears it (under its alias phone/any/tilt/x). Same calls as against a real relay.

To be confirmed

The reconnect delays (0.5 s, 1 s, 2 s, 4 s, then 5 s) are read from the code, not measured against a relay that goes down and comes back. The second example runs the client against a stand-in socket, not a real WebSocket server.

Related

  • bindSignals() Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
  • attachRelay() Attaches the room relay to a Node http.Server: WebSocket clients that connect to path are put in rooms and routed by role. Returns { hub, close() }.
  • deviceId() The short random id of this browser, kept in localStorage, that names this phone in its signals, as in phone/ab12/tilt/x.
  • mountRemoteHost() The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).

import { RelayClient } from '@openav/relay'; packages/relay/index.js, line 34

signalMeta()#

function@openav/relay
signalMeta(name, { pulse = false } = {})

What the relay knows about a signal name: its range and unit, or that it is a pulse. Returns null for names it does not know.

A Signals registry needs to know a name's range before a meter can scale it or a mapping can tell a knob from an event. fileSignal() calls this the first time it hears a name. What it knows: phone/<id>/tilt/* is −1..1, accel/* is −20..20 m/s², rot/* is −360..360 °/s, orient/* is 0..360 °, touch/count is 0..10, knock is a pulse, and every other phone/… name (touch, light) is 0..1. surface/… names are 0..1 (…/raw and …/delta are unscaled, {}). midi/note/on, midi/note/off, midi/virtual and the hit, on, off, last names of a controller are pulses, and midi/cc/* is 0..1.

Parameters

NameTypeDescription
nameStringA signal name such as phone/ab12/tilt/x, surface/main/hue or midi/note/on.
pulseBooleanPass { pulse: true } when the message was sent as an event: the answer is then always { kind: 'pulse' }. optional default false

Returns

Object | null Metadata for signals.define(): { min, max, unit? }, { kind: 'pulse' }, or {} for raw values. null when the name is not one it knows.

Examples

js
import { signalMeta } from '@openav/relay';

const names = ['phone/ab12/tilt/x', 'phone/ab12/accel/z', 'phone/ab12/knock', 'surface/main/hue', 'surface/main/hue/raw', 'midi/note/on', 'my/own/thing'];
document.getElementById('app').innerHTML = names
  .map((n) => `${n}: ${JSON.stringify(signalMeta(n))}`)
  .join('<br>');
module Prints what the relay knows about seven names, from {"min":-1,"max":1} for tilt to null for a name it has never heard of.

Related

  • fileSignal() Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
  • bindSignals() Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
  • aliasOf() The "latest phone" name of a phone signal: aliasOf('phone/ab12/tilt/x') is 'phone/any/tilt/x'. null for anything else.

import { signalMeta } from '@openav/relay'; packages/relay/signals.js, line 8

@openav/remote

attachTouchPad()#

function@openav/remote
attachTouchPad(pad, io, { onMove, onEnd } = {})

Makes an element a multi-touch pad: each finger publishes touch/x, touch/y and touch/down (0..1 inside the element), and touch/count says how many are down.

The first finger down is slot 0 and publishes touch/x, touch/y, touch/down; the second is slot 1 and publishes touch/1/x, touch/1/y, touch/1/down, and so on. A slot is given back when its finger lifts, so the lowest free number is used next. x runs 0 at the left edge to 1 at the right, y runs 0 at the top to 1 at the bottom, both held inside the element and rounded to 4 decimals. touch/down is 1 or 0, and touch/count is the number of fingers down.

It publishes touch/down 0 and touch/count 0 at once, so a receiver knows the pad is there before anyone touches it. It uses pointer events, so a mouse works too (one finger).

Parameters

NameTypeDescription
padHTMLElementThe element to listen on. Give it touch-action: none in CSS, or the browser may take a drag for a scroll and cancel the touch.
io{ set(name, value) }Where the values go, as for PhoneSensors: only set is used, and the names have no phone/<id>/ in front.
onMoveFunction(slot, x, y, down) => {}, when a finger touches and each time it moves. x and y are 0..1 and down is true. Use it to draw the finger. optional
onEndFunction(slot) => {}, when a finger lifts or the touch is cancelled. optional

Notes

  • The sense tab uses this on its touch pad, with io writing to a RelayClient under phone/<id>/.

Examples

js
import { attachTouchPad } from '@openav/remote';

const pad = document.createElement('div');
pad.style.cssText = 'width:200px;height:200px;background:#222;touch-action:none';
document.getElementById('app').append(pad);

const seen = [];
attachTouchPad(pad, { set: (name, value) => seen.push(name + '=' + value) });

// a finger lands a quarter of the way across, halfway down, then moves right
const r = pad.getBoundingClientRect();
const touch = (type, id, fx, fy) => pad.dispatchEvent(new PointerEvent(type, {
  pointerId: id, clientX: r.left + fx * r.width, clientY: r.top + fy * r.height, bubbles: true,
}));
touch('pointerdown', 1, 0.25, 0.5);
touch('pointermove', 1, 0.5, 0.5);
touch('pointerup', 1, 0.5, 0.5);
pad.after(Object.assign(document.createElement('p'), { innerHTML: seen.join('<br>') }));
module A 200 px square appears, and under it the values a finger would publish: touch/down=0, touch/count=0 at the start, then x, y, down and count as it lands, moves and lifts.
js
import { attachTouchPad } from '@openav/remote';

const pad = document.createElement('div');
pad.style.cssText = 'width:200px;height:100px;background:#222;touch-action:none';
document.getElementById('app').append(pad);

const seen = [];
attachTouchPad(pad, { set: (name, value) => { if (name.endsWith('/count') || name.endsWith('/down')) seen.push(name + '=' + value); } }, {
  onMove: (slot, x, y) => seen.push('finger ' + slot + ' at ' + x.toFixed(2) + ', ' + y.toFixed(2)),
  onEnd: (slot) => seen.push('finger ' + slot + ' lifted'),
});
const r = pad.getBoundingClientRect();
const touch = (type, id, fx, fy) => pad.dispatchEvent(new PointerEvent(type, {
  pointerId: id, clientX: r.left + fx * r.width, clientY: r.top + fy * r.height, bubbles: true,
}));
touch('pointerdown', 1, 0.1, 0.5);   // slot 0
touch('pointerdown', 2, 0.9, 0.5);   // slot 1: touch/1/…
touch('pointerup', 1, 0.1, 0.5);     // slot 0 is free again
touch('pointerdown', 3, 0.5, 0.5);   // takes slot 0
pad.after(Object.assign(document.createElement('p'), { innerHTML: seen.join('<br>') }));
module Two fingers get slots 0 and 1; after the first lifts, the next finger takes slot 0 again. The list shows the callbacks and the count.

Related

  • PhoneSensors Turns a phone's tilt, acceleration, rotation, knocks and front-camera light into named signals, in screen axes, and hands them to the io you give it.
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
  • RelayClient The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.

import { attachTouchPad } from '@openav/remote'; packages/remote/sensors.js, line 138

localSensors()#

function@openav/remote
localSensors(signals, { prefix = 'phone/local/', alias = 'any', source = 'local', hooks = {} } = {})

For a show page that runs on the phone itself: the phone's own sensors go straight into a Signals as phone/local/…, with a phone/any/… copy.

A remote phone reaches the show through a relay. When the show page is open on the phone itself, there is no one to relay to, so this reads the sensors in place. The names and thresholds are the same as a remote phone's (phone/local/tilt/x is −1..1, phone/local/knock is a pulse), declared the same way by fileSignal(), and mirrored to phone/any/…, so a route written for "the latest phone", phone/any/tilt/x, works whether the phone is in your hand as a controller or is the show.

Before you call it

Call .start() on the result from a tap (iOS asks for permission only inside a gesture, and only on https pages). Until then nothing is published. See PhoneSensors.

Parameters

NameTypeDescription
signalsSignalsThe registry to fill (from @openav/core).
prefixStringWhat goes in front of each sensor name. optional default 'phone/local/'
aliasString | nullThe word for the copy, phone/<alias>/…. null turns the copy off. optional default 'any'
sourceStringStored as the signals' source in signals.meta. optional default 'local'
hooksObjectThe hooks of PhoneSensors: onKnock, onSample, notify. optional default {}

Returns

PhoneSensors Nothing is listening yet: call .start() from a tap.

Examples

js
import { localSensors } from '@openav/remote';
import { Signals } from '@openav/core';

const signals = new Signals();
const own = localSensors(signals);
await own.start();   // on a phone: from a tap

window.dispatchEvent(new DeviceOrientationEvent('deviceorientation', { alpha: 0, beta: 22.5, gamma: -45 }));
own.stop();
document.getElementById('app').innerHTML = [
  'phone/local/tilt/x = ' + signals.get('phone/local/tilt/x'),
  'phone/any/tilt/y = ' + signals.get('phone/any/tilt/y'),
  'source: ' + signals.meta.get('phone/local/tilt/x').source,
].join('<br>');
module Prints the tilt under both names (-1 and 0.5) and the source "local". On a phone the events come from the sensors instead of from this made-up one.
js
import { localSensors } from '@openav/remote';
import { Signals } from '@openav/core';

const signals = new Signals();
const own = localSensors(signals, { prefix: 'phone/me/', alias: null });   // no phone/any copy
await own.start();
window.dispatchEvent(new DeviceOrientationEvent('deviceorientation', { alpha: 0, beta: 0, gamma: 9 }));
own.stop();
document.getElementById('app').textContent =
  'phone/me/tilt/x = ' + signals.get('phone/me/tilt/x') + ', copy exists: ' + signals.meta.has('phone/any/tilt/x');
module A different prefix and no alias: prints "phone/me/tilt/x = 0.2, copy exists: false".

Related

  • PhoneSensors Turns a phone's tilt, acceleration, rotation, knocks and front-camera light into named signals, in screen axes, and hands them to the io you give it.
  • fileSignal() Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.
  • aliasOf() The "latest phone" name of a phone signal: aliasOf('phone/ab12/tilt/x') is 'phone/any/tilt/x'. null for anything else.
  • bindSignals() Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.

import { localSensors } from '@openav/remote'; packages/remote/sensors.js, line 163

mountJoinCard()#

function@openav/remote
mountJoinCard(parent, host)

Puts a small card on the show page with a QR code and the URL a phone should open, and a line that says whether the relay is reachable and how many phones are in.

The URL is the phone page (packages/remote/ next to this file) with ?room=<room> on it, and with ?relay=… as well when the show uses the public relay or a relay of your own, so the phone ends up in the same room on the same relay. On localhost it asks node serve.js for this machine's LAN address at /__info and puts that in the URL, because a phone cannot open localhost. The QR code is drawn from the URL; if the URL is too long for the encoder, the code is left out and the URL stays.

The status line reads relay ok · 2 phones · room demo when the connection is open (and adds · public relay on the public one), relay offline — is it node serve.js? when a local relay cannot be reached, and can't reach the relay (…) — retrying for a shared one. On a phone held upright the card starts folded, because the stage is small; tap it to open or fold it.

Parameters

NameTypeDescription
parentHTMLElementThe element to add the card to. The card is position: absolute at its top left, so the parent should be positioned (position: relative) and big enough to show it.
hostObjectThe host from mountRemoteHost(). The card reads host.room, host.relayMode and host.relayUrl, and takes over host.onStatus to write the status line.

Returns

HTMLElement The card's element, already in parent. Remove it with .remove().

Notes

  • The words on the card are in English.

Examples

js
import { mountRemoteHost, mountJoinCard } from '@openav/remote/host';
import { Signals, Params } from '@openav/core';
import { Mapper } from '@openav/mapping';

const params = new Params([{ key: 'hue', min: 0, max: 360, def: 200 }]);
const signals = new Signals();
const mapper = new Mapper({ signals, params });
const host = mountRemoteHost({ signals, params, mapper, room: 'demo' });   // not connected

const card = mountJoinCard(document.getElementById('app'), host);
host.onStatus({ status: 'open', peers: { controllers: 2 } });   // pretend the relay answered
module A card appears at the top left with a QR code, the phone URL ending in ?room=demo, and the line "relay ok · 2 phones · room demo".

Related

  • mountRemoteHost() The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
  • RelayClient The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.

import { mountJoinCard } from '@openav/remote/host'; packages/remote/host.js, line 90

mountRemote()#

function@openav/remote
mountRemote(root, opts = {})

The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.

It is what a phone opens: lockViewport() so nothing scrolls or zooms, keepAwake() so the screen stays on, one RelayClient as a controller that every tab shares, and the tabs. Sense is a tilt ball, readouts, a multi-touch pad (attachTouchPad()) and the sensors (PhoneSensors) behind a start button. Keys is a piano that is one row in landscape and two stacked rows in portrait. Control is a Surface. MIDI is a virtual controller.

The control tab's layout comes from, in order: surfaceUrl or ?surface=, metaUrl or ?meta=, whatever layout the show publishes over the relay (the config surface), and finally a small generic panel so the page is never blank. The signals are phone/<id>/… (sense), midi/note/on|off and midi/cc/64 (keys), and surface/<page>/<id> (control). The app opens on ?tab=, else the tab used last in this browser, else sense.

Before you call it

A page of its own (it takes over the whole screen) and a relay to reach: the same origin's /relay unless relayUrl or ?relay= says otherwise. Without a show in the room the header says it is waiting for the work to open.

Parameters

NameTypeDescription
rootHTMLElementWhere to put the app. It fills the whole page (position: fixed), so pass document.body.
optsObject{ room, relayUrl, surfaceUrl, metaUrl }, each optional. room is the relay room (else ?room= in the address, else window.__REMOTE__.room, else default). relayUrl is a ws:// or wss:// relay (else ?relay=; only those two schemes are accepted, so a link cannot point the phone anywhere else). surfaceUrl is a layout JSON for the control tab and metaUrl a JSON with a params array, which the control tab turns into autoSurface(). The address can also say ?tab=sense|keys|control|midi. optional default {}

Returns

Object { relay, show(tab), surface, dispose() }: the RelayClient, a function that switches tab, the control tab's Surface (a getter, null until that tab has been opened), and dispose(), which closes the relay, unlocks the viewport, lets go of the wake lock and removes the app.

Notes

  • The words on the app's tabs and header are in Chinese.

Examples

js
import { mountRemote } from '@openav/remote';

// the whole phone app: this is what /packages/remote/ does
const app = mountRemote(document.body, { room: 'main' });

// later: app.show('keys') switches tab, app.dispose() takes everything down
static On a phone: the status bar with the connection and latency, and the four tabs (sense, keys, control and MIDI, labelled in Chinese) along the bottom.

To be confirmed

Not opened on a real phone for this entry: how the tabs behave is read from packages/remote/app.js and docs/remote.md.

Related

  • mountRemoteHost() The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
  • PhoneSensors Turns a phone's tilt, acceleration, rotation, knocks and front-camera light into named signals, in screen axes, and hands them to the io you give it.
  • attachTouchPad() Makes an element a multi-touch pad: each finger publishes touch/x, touch/y and touch/down (0..1 inside the element), and touch/count says how many are down.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
  • RelayClient The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.

import { mountRemote } from '@openav/remote'; packages/remote/app.js, line 105

mountRemoteHost()#

function@openav/remote
mountRemoteHost({ signals, params, mapper, world = null, room = 'default', auto = {}, surface = null, feedbackHz = 10, url = null, mode = null, controllers = null } = {})

The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.

createShow({ modules: { remote: true } }) calls this for you, so the world never has to know about phones. Calling it yourself does five things: it makes a RelayClient as the room's runner; files every phone signal into signals with bindSignals() (so phone/any/… always holds the latest phone); builds the layout from surface, world.surface or autoSurface() of the params; adds the surface/… → param routes to the mapper; and, when you call host.frame(dt, state) every frame, sends the resolved param values back about feedbackHz times a second so the phones' faders follow the timeline.

Making the host does not connect it: call host.connect() when you want the socket. On every (re)connection it publishes the layout as relay config surface, so a phone that joins later gets the right panel. host.layout, host.routes and host.bindings are there to look at straight away, and host.dispose() closes the socket.

Before you call it

A relay for connect() to reach: node serve.js hosts one at /relay, a static site uses the public relay (see pickRelay()), or pass url. Building the host needs none of that.

Parameters

NameTypeDescription
signalsSignalsThe registry that phone signals are filed in. optional
paramsParamsThe world's params. Its schema is what the control surface is built from. optional
mapperMapperThe mapper that gets the surface/… → param routes (a route it already has is not added twice). optional
worldObjectThe world: its params are used when params.schema is empty, its name becomes the layout's title, and its surface (a hand-written layout) is used instead of a generated one. optional default null
roomStringThe relay room to join. optional default 'default'
autoObjectOptions for autoSurface(): pairs, meters, style, perPage, smooth, title. optional default {}
surfaceObjectAn explicit layout. It wins over world.surface. optional default null
feedbackHzNumberHow many times a second host.frame() sends the param values back to the phones. optional default 10
urlStringA relay to use instead of this page's own /relay, such as wss://host/relay. role, room and id are added to it for you. The runner id is then show- plus four random characters, so two tabs of one show do not collide; without url it is show. optional default null
mode'local' | 'public' | 'custom'What pickRelay() decided. Only the words of the join card depend on it; the host reports it as host.relayMode. optional default null
controllersMidiControllersThe on-screen MIDI controllers of @openav/midi: the phone's MIDI tab then shows the same device, and what is played on either screen shows on the other. optional default null

Returns

Object { relay, layout, routes, bindings, publish, room, relayUrl, relayMode, onStatus, frame(dt, state), connect(), dispose() }. connect() returns the host again.

Examples

js
import { mountRemoteHost } from '@openav/remote/host';
import { Signals, Params } from '@openav/core';
import { Mapper } from '@openav/mapping';

const world = { name: 'Bloom', params: [
  { key: 'hue', min: 0, max: 360, def: 200 },
  { key: 'burst', pulse: true },
] };
const signals = new Signals();
const params = new Params(world.params);
const mapper = new Mapper({ signals, params });

const host = mountRemoteHost({ signals, params, mapper, world, room: 'demo' });   // not connected yet
document.getElementById('app').innerHTML = [
  'title: ' + host.layout.title,
  'widgets: ' + host.layout.pages[0].widgets.map((w) => w.id + ' (' + w.type + ')').join(', '),
  'routes: ' + host.routes.map((r) => r.source + ' > ' + r.target).join(', '),
  'in the mapper: ' + mapper.routes.length,
].join('<br>');
host.dispose();
module Prints what the host built from two params: a layout titled Bloom with a fader and a button, the two routes, and that the mapper now has them.
js
import { mountRemoteHost, mountJoinCard } from '@openav/remote/host';

// signals, params, mapper and world as above
const host = mountRemoteHost({ signals, params, mapper, world, room: 'main' }).connect();
mountJoinCard(document.body, host);   // the QR code and URL for phones

function frame(dt, state) {
  host.frame(dt, state);               // echo the params back to the phones
}
static The usual wiring on a show page: connect, show the join card, and call host.frame() from the frame loop. A phone that scans the card moves the faders.

Related

  • mountJoinCard() Puts a small card on the show page with a QR code and the URL a phone should open, and a line that says whether the relay is reachable and how many phones are in.
  • RelayClient The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
  • bindSignals() Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
  • autoSurface() Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
  • routesFromLayout() Writes the mapper routes for a hand-written layout: every widget that names a param in target gets a route from its signal to that param.
  • feedbackFor() Turns the resolved param state into the normalized values the phone's widgets should show right now: [{ name, value }].
  • attachRelay() Attaches the room relay to a Node http.Server: WebSocket clients that connect to path are put in rooms and routed by role. Returns { hub, close() }.
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.

import { mountRemoteHost } from '@openav/remote/host'; packages/remote/host.js, line 41

PhoneSensors#

class@openav/remote
new PhoneSensors(io, hooks = {})

Turns a phone's tilt, acceleration, rotation, knocks and front-camera light into named signals, in screen axes, and hands them to the io you give it.

This is the phone half of the sense tab, with no page in it: make one with an io, call start() from a tap, and signals flow. From the phone's own events it produces tilt/x and tilt/y (−1..1, which is ±45° from the resting pose, x to the right and y toward you, in screen axes, so a phone turned sideways still reports "tilt right is +x"), orient/alpha|beta|gamma (degrees), accel/x|y|z (m/s², gravity included), rot/alpha|beta|gamma (degrees per second), knock (a pulse { strength, delta, t }) and, if you turn the camera on, light (0..1).

A knock is a jolt: when the acceleration vector changes by more than threshold (10 m/s² by default) between two motion samples, and at least 130 ms after the last knock, io.send('knock', { strength, delta, t }, true) fires. delta is the size of the change, and strength runs from 0.15 just over the threshold up to 1. The phone also vibrates for 10 ms where it can. Hold the phone how you want to play it and call calibrate() to make that pose the zero of the tilt.

For the show page itself running on a phone, localSensors() wraps this class and files straight into a Signals.

Before you call it

Call start() from a tap or click, not on page load. iOS asks for motion permission only inside a user gesture, and it only gives motion sensors to pages on https://; touch and keys work over plain http://. start() always resolves true: look at denied for a refusal. If no sensor data arrives within 2.5 s, hooks.notify says so (a desktop, or an iPhone page on http). setCamera(true) also needs a gesture and HTTPS on iOS.

Parameters

NameTypeDescription
io{ set(name, value), send(name, value, pulse?) }Where the signals go. set gets the continuous ones, send the knocks (with pulse true). The names have no phone/<id>/ in front: tilt/x, accel/z. A RelayClient with a prefix, or a Signals registry, can sit behind it.
hooks{ onKnock?, onSample?, notify? }Optional callbacks. onKnock(strength) after each knock, onSample(name, value) for every continuous sample (to draw a readout), notify(message) with a one-line problem report such as a refused permission. The messages are written in Chinese. optional default {}

Members

NameDescription
start()methodstart(): async. Ask for the motion permission where the browser requires it, then listen to deviceorientation and devicemotion. Resolves true even when the answer was no (then denied is true and the listeners are attached anyway). Calling it again does nothing.
stop()methodstop(): stop listening and turn the camera off.
calibrate()methodcalibrate(): take the current tilt as the resting pose. Tilt reads 0 here, until the phone moves.
cycleThreshold()methodcycleThreshold(): make the knock threshold 2 m/s² higher and return it; past 24 it wraps to 4. For a "knock sensitivity" button.
setCamera(on)methodsetCamera(on): async. true opens the front camera and publishes light (the mean brightness of a 32 × 24 picture, 0..1) ten times a second; false stops it. Resolves true when done, false if the camera is not available (then hooks.notify says why).
screenTilt(beta, gamma, angle = 0)static methodPhoneSensors.screenTilt(beta, gamma, angle?): static. Turn the two raw tilt angles into screen axes for a screen angle of 0, 90, 180 or 270 degrees. Returns { x, y } in degrees.

Notes

  • Public fields: io, hooks, started, gotSensor (true once any sensor data has arrived), denied (iOS said no; it stays no until the Settings app is changed), rest and deg (tilt in degrees), threshold (the knock threshold, 10) and COOLDOWN (130 ms).

Examples

js
import { PhoneSensors } from '@openav/remote';

const seen = [];
const sensors = new PhoneSensors({
  set: (name, value) => seen.push(name + ' = ' + value),
  send: (name, value) => seen.push(name + ' ' + JSON.stringify(value)),
});
await sensors.start();   // on a phone: call this from a tap

// the event a phone fires; here a made-up one, tipped 22.5° toward you and 45° to the left
window.dispatchEvent(new DeviceOrientationEvent('deviceorientation', { alpha: 5, beta: 22.5, gamma: -45 }));
sensors.stop();
document.getElementById('app').innerHTML = seen.join('<br>');
module Prints tilt/x = -1 (45° left is the end of the range), tilt/y = 0.5, then the three raw orient angles.
js
import { PhoneSensors } from '@openav/remote';

const seen = [];
const sensors = new PhoneSensors(
  { set() {}, send: (name, v) => seen.push(name + ': strength ' + v.strength + ', delta ' + v.delta) },
  { onKnock: (s) => seen.push('onKnock(' + s + ')') },
);
await sensors.start();
await new Promise((r) => setTimeout(r, 200));   // the 130 ms knock cooldown counts from page load
navigator.vibrate = () => true;                 // a knock also buzzes the phone; a stand-in for a desktop page

const motion = (x) => window.dispatchEvent(new DeviceMotionEvent('devicemotion', {
  accelerationIncludingGravity: { x, y: 0, z: 9.8 },
}));
motion(0);    // at rest, gravity only
motion(15);   // a jolt: the acceleration changed by 15 m/s²
motion(15.5); // too small to count
sensors.stop();
seen.push('threshold now ' + sensors.cycleThreshold());
document.getElementById('app').innerHTML = seen.join('<br>');
module Prints one knock with strength 0.35 and delta 15 (and the onKnock call), then the new threshold 12 after cycleThreshold().
js
import { PhoneSensors } from '@openav/remote';

// the same tilt (beta 10, gamma 20) as the screen turns: x is always "to the right of the screen"
const rows = [0, 90, 180, 270].map((angle) => {
  const { x, y } = PhoneSensors.screenTilt(10, 20, angle);
  return 'screen angle ' + angle + ': x ' + x + ', y ' + y;
});
document.getElementById('app').innerHTML = rows.join('<br>');
module Prints x and y for the four screen angles: (20, 10), (10, -20), (-20, -10), (-10, 20).

To be confirmed

Not tried on a real iPhone or Android phone for this entry: the iOS permission flow, the HTTPS requirement and the 2.5 s "no data" message are read from the code and from docs/remote.md. The examples feed the class real DeviceOrientationEvent and DeviceMotionEvent objects in a desktop browser instead.

Related

  • localSensors() For a show page that runs on the phone itself: the phone's own sensors go straight into a Signals as phone/local/…, with a phone/any/… copy.
  • attachTouchPad() Makes an element a multi-touch pad: each finger publishes touch/x, touch/y and touch/down (0..1 inside the element), and touch/count says how many are down.
  • RelayClient The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
  • signalMeta() What the relay knows about a signal name: its range and unit, or that it is a pulse. Returns null for names it does not know.

import { PhoneSensors } from '@openav/remote'; packages/remote/sensors.js, line 24

@openav/surface

autoArrange()#

function@openav/surface
autoArrange(items, cols, fixed = [])

Packs widgets into a grid first-fit, row by row, and never overlaps; fixed placements are kept and block their cells. Returns the placements and the rows used.

Each item takes the first free spot found by scanning left to right, then top to bottom: what a person does laying out a controller. The grid grows downward as far as it must, so rows is a result, not a limit. resolvePage() uses it for widgets without a position and for the automatic reflow.

Parameters

NameTypeDescription
items{ id, w, h }[]What to place, in order. Every item needs a numeric w and h in cells, at least 1. A w wider than the grid is cut to cols.
colsNumberHow many columns the grid has.
fixed{ id, x, y, w, h }[]Placements that are already decided. They come first in the result and the other items go around them. optional default []

Returns

{ placements, rows } placements is a list of { id, x, y, w, h } (the fixed ones first), rows is how many rows they fill.

Notes

  • An item with no w (or a w that is not a number) never finds a spot and the function never returns. Give every item a w and an h; DEFAULT_SIZE has the natural size of each widget type.

Examples

js
import { autoArrange } from '@openav/surface/layout';

const { placements, rows } = autoArrange([
  { id: 'a', w: 4, h: 2 }, { id: 'b', w: 4, h: 2 }, { id: 'c', w: 8, h: 1 }, { id: 'd', w: 2, h: 2 },
], 8);
document.getElementById('app').innerHTML = placements
  .map((p) => p.id + ': x ' + p.x + ', y ' + p.y + ', ' + p.w + '×' + p.h).join('<br>') + '<br>rows: ' + rows;
module a and b share the first row, c takes a full row, d goes below: rows is 5.
js
import { autoArrange } from '@openav/surface/layout';

// a button that must stay at the top left; the new knob goes around it
const { placements } = autoArrange(
  [{ id: 'knob', w: 2, h: 1 }],
  4,
  [{ id: 'go', x: 0, y: 0, w: 2, h: 1 }],
);
document.getElementById('app').textContent = placements.map((p) => p.id + ' at ' + p.x + ',' + p.y).join('   ');
module Prints "go at 0,0 knob at 2,0": the fixed placement comes first and the knob takes the first free cells.

Related

  • resolvePage() Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.
  • stretchRows() Stretches a last row that stops short to the edge of the grid: five faders in four columns leave one lonely fader, and this makes it four columns wide.
  • DEFAULT_SIZE The natural size of each widget type, in grid cells, as { w, h }: a layout only has to give x, y, w, h when it cares.

import { autoArrange } from '@openav/surface'; Also from @openav/surface/layout packages/surface/layout.js, line 95

autoSurface()#

function@openav/surface
autoSurface(params = [], { perPage = 8, style = 'auto', pairs = [], meters = [], smooth = 0.04, title = '' } = {})

Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.

This is how a world that declares what it can be performed with gets a phone panel for free: const { layout, routes, bindings } = autoSurface(world.params), then routes.forEach((r) => mapper.addRoute(r)) and new Surface(el, { layout, sink }). Each param becomes the widget widgetTypeFor() picks. Each group of params gets its own page (a param without a group goes on main), long groups continue on main2, main3 …, and a pair from pairs becomes one XY pad with two routes, …/x and …/y.

Widget ids are the param keys (anything outside A–Z a–z 0–9 _ - becomes _), colours go round cyan, amber, magenta, lime, violet, coral, and a radio's options come from the param or are the numbers. A pulse param gets no binding, since there is no value to show. The routes already scale the normalized signal to the param's min and max. A world that comes with its own layout uses routesFromLayout() instead.

Parameters

NameTypeDescription
paramsObject[]The param declarations: { key, label?, min, max, def, step?, pulse?, group?, options?, surface? }. surface: { type, color, label } on a param overrides the choice. optional default []
perPageNumberHow much fits on a page, counted in weights: a fader or a meter weighs 1, an XY pad 4, anything else 2. The default 8 is eight faders, four knobs or two XY pads. A page that scrolls is a page that fails, so a full page starts the next. optional default 8
style'auto' | 'fader' | 'knob'Faders or knobs for the continuous params. 'auto' picks faders up to six, knobs beyond. optional default 'auto'
pairsstring[][]Pairs of param keys to drive with one XY pad: [['cx', 'cy']]. The first key is x and the second is y. optional default []
metersstring[]Param keys to also show as read-only meters, on a page called Meters. optional default []
smoothNumberSeconds of smoothing on the continuous routes, to hide the steps of a 30 Hz network (zipper noise). Pulses, toggles and radios are never smoothed. optional default 0.04
titleStringThe layout's title. optional default ''

Returns

{ layout, routes, bindings } layout is a layout JSON for Surface. routes is a list of mapper routes { source, target, inMin, inMax, curve, smooth, outMin, outMax }, from surface/<page>/<id> to each param. bindings is { param, name, page, widget, min, max, readonly? } for feedbackFor().

Examples

js
import { autoSurface } from '@openav/surface/auto';

const params = [
  { key: 'hue', min: 0, max: 360, def: 200 },
  { key: 'size', min: 1, max: 10, def: 4 },
  { key: 'mode', min: 0, max: 3, step: 1, def: 1 },
  { key: 'burst', pulse: true },
];
const { layout, routes } = autoSurface(params, { title: 'Bloom' });
document.getElementById('app').innerHTML = [
  layout.pages.map((p) => p.id + ': ' + p.widgets.map((w) => w.id + ' ' + w.type).join(', ')).join('<br>'),
  routes.map((r) => r.source + ' → ' + r.target + (r.smooth ? ' (smooth ' + r.smooth + ')' : '')).join('<br>'),
].join('<br><br>');
module Prints the one page main with a fader for hue and size, a radio for mode and a button for burst, then the four routes, smoothed by 0.04 s except the radio and the button.
js
import { autoSurface } from '@openav/surface/auto';

const params = [
  { key: 'cx', min: 0, max: 1, def: 0.5 }, { key: 'cy', min: 0, max: 1, def: 0.5 },
  { key: 'density', min: 0, max: 100, def: 20 },
  { key: 'glow', min: 0, max: 1, def: 0.3, group: 'look' },
];
const { layout, bindings } = autoSurface(params, { pairs: [['cx', 'cy']], meters: ['density'] });
document.getElementById('app').innerHTML = [
  layout.pages.map((p) => p.id + ' (' + p.title + '): ' + p.widgets.map((w) => w.id + ' ' + w.type).join(', ')).join('<br>'),
  'bindings: ' + bindings.map((b) => b.param + (b.readonly ? ' (read-only)' : '')).join(', '),
].join('<br><br>');
module cx and cy share one XY pad, density is a fader and also a read-only meter on its own page, and glow, in group look, gets a page of its own.
js
import { autoSurface } from '@openav/surface/auto';

// twenty params do not fit on one page: more than six continuous ones become knobs, four to a page
const params = Array.from({ length: 20 }, (_, i) => ({ key: 'p' + i, min: 0, max: 1, def: 0 }));
const { layout } = autoSurface(params);
document.getElementById('app').textContent = layout.pages
  .map((p) => p.id + ': ' + p.widgets.length + ' ' + p.widgets[0].type + 's').join('   ');
module Prints "main1: 4 knobs main2: 4 knobs …" up to main5: the page limit counts a knob as 2, so four fit in the default 8.

Related

  • widgetTypeFor() Picks the widget type for one param declaration: a button for a pulse, a toggle, a radio, a fader or a knob.
  • feedbackFor() Turns the resolved param state into the normalized values the phone's widgets should show right now: [{ name, value }].
  • routesFromLayout() Writes the mapper routes for a hand-written layout: every widget that names a param in target gets a route from its signal to that param.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
  • mountRemoteHost() The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
  • DEFAULT_SIZE The natural size of each widget type, in grid cells, as { w, h }: a layout only has to give x, y, w, h when it cares.

import { autoSurface } from '@openav/surface'; Also from @openav/surface/auto packages/surface/auto.js, line 54

canFullscreen()#

function@openav/surface
canFullscreen()

True when the page can ask for fullscreen: the browser has requestFullscreen (or the WebKit one). The code treats iPhone Safari as not having it.

Use it to decide whether to show a fullscreen button: if (canFullscreen()) button.hidden = false. It only checks that the function exists; whether a request succeeds is up to the browser, which wants it to come from a tap (toggleFullscreen()).

Returns

Boolean true or false. false without a document.

Examples

js
import { canFullscreen } from '@openav/surface';

document.getElementById('app').textContent = 'this browser can ask for fullscreen: ' + canFullscreen();
module Prints "true" in desktop Chrome, Firefox and Safari, and in Android Chrome.

To be confirmed

Not checked on an iPhone or iPad: that iPhone Safari has no fullscreen function is what the code's own comment says, not something measured here.

Related

  • toggleFullscreen() Enters fullscreen if the page is not in it, and leaves it if it is. Call it from a tap, because browsers refuse otherwise.
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.

import { canFullscreen } from '@openav/surface'; packages/surface/platform.js, line 35

COLOR_NAMES#

constant@openav/surface
COLOR_NAMES = array, 7 items

The seven colour names a widget's color may be: cyan, amber, magenta, lime, violet, coral, white.

{ id: 'hue', type: 'knob', color: 'amber' } makes the widget's fill, its value readout and its pressed outline amber. A widget without a colour uses --oav-accent of THEME_VARS. autoSurface() hands out the first six in turn (white is left for you). A name that is not in the list gets no colour of its own and falls back to the accent; validateLayout() does not check it.

Type
string[]

['cyan', 'amber', 'magenta', 'lime', 'violet', 'coral', 'white'].

Examples

js
import { COLOR_NAMES } from '@openav/surface';

document.getElementById('app').textContent = COLOR_NAMES.join(', ');
module Prints cyan, amber, magenta, lime, violet, coral, white.

Related

  • THEME_VARS The look of every control as 15 CSS custom properties on :root (--oav-bg, --oav-accent …): override them to re-skin the whole surface.
  • autoSurface() Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
  • WIDGET_TYPES The names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.

import { COLOR_NAMES } from '@openav/surface'; packages/surface/theme.js, line 30

DEFAULT_SIZE#

constant@openav/surface
DEFAULT_SIZE = object, 14 keys

The natural size of each widget type, in grid cells, as { w, h }: a layout only has to give x, y, w, h when it cares.

A widget without a w or h takes the size of its type, held to the number of columns of the grid. autoArrange() and resolvePage() use it, and so does autoSurface() when it counts how many widgets fit on a page. A type it does not know counts as 2×2.

Type
Object

Fourteen keys, one per type: fader 1×4, knob 2×2, encoder 2×2, button 2×1, toggle 2×1, xy 4×4, bank 4×4, radio 4×1, pads 4×4, keyboard 8×4, number 2×1, text 4×1, label 2×1, meter 1×4 (width × height).

Examples

js
import { DEFAULT_SIZE } from '@openav/surface';

const { w, h } = DEFAULT_SIZE.knob;
document.getElementById('app').textContent = 'a knob is ' + w + ' cells wide and ' + h + ' tall; a fader is ' +
  DEFAULT_SIZE.fader.w + ' by ' + DEFAULT_SIZE.fader.h;
module Prints "a knob is 2 cells wide and 2 tall; a fader is 1 by 4".

Related

  • WIDGET_TYPES The names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
  • autoArrange() Packs widgets into a grid first-fit, row by row, and never overlaps; fixed placements are kept and block their cells. Returns the placements and the rows used.
  • resolvePage() Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.

import { DEFAULT_SIZE } from '@openav/surface'; Also from @openav/surface/layout packages/surface/layout.js, line 31

feedbackFor()#

function@openav/surface
feedbackFor(state, bindings)

Turns the resolved param state into the normalized values the phone's widgets should show right now: [{ name, value }].

It is pure and cheap, so the show calls it about ten times a second and sends each result to the phones with relay.feedback(name, value); host.frame() of mountRemoteHost() does exactly that. On the phone, Surface's feedback() moves the widget, so a fader follows the timeline and other controllers.

Parameters

NameTypeDescription
stateObjectThe resolved param values, such as params.resolve(…): { hue: 90, size: 5.5 }.
bindingsObject[]The bindings that autoSurface() returned.

Returns

{ name, value }[] One entry per binding that has a number in state: name is the widget's signal, value is the param scaled to 0..1 and held in range. A param that is missing, not a number, or with min equal to max is skipped.

Examples

js
import { autoSurface, feedbackFor } from '@openav/surface/auto';

const params = [{ key: 'hue', min: 0, max: 360, def: 200 }, { key: 'size', min: 1, max: 10, def: 4 }];
const { bindings } = autoSurface(params);
const echo = feedbackFor({ hue: 90, size: 5.5 }, bindings);
document.getElementById('app').innerHTML = echo.map((f) => f.name + ' = ' + f.value).join('<br>');
module Prints surface/main/hue = 0.25 (90 of 0 to 360) and surface/main/size = 0.5 (5.5 of 1 to 10).

Related

  • autoSurface() Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
  • mountRemoteHost() The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
  • RelayClient The browser end of the room relay: sends signals to the other pages in a room and receives theirs, as a phone (controller), the show (runner) or a backstage monitor.

import { feedbackFor } from '@openav/surface'; Also from @openav/surface/auto packages/surface/auto.js, line 141

gridOrientation()#

function@openav/surface
gridOrientation(g)

Says whether a grid { cols, rows } is 'landscape' (at least as many columns as rows) or 'portrait'.

resolvePage() compares it with the orientation of the screen: a base grid of 8 by 4 is used as it is on a landscape screen and reflowed on a portrait one.

Parameters

NameTypeDescription
g{ cols, rows }The grid.

Returns

'landscape' | 'portrait' 'landscape' when g.cols >= g.rows, else 'portrait'.

Examples

js
import { gridOrientation } from '@openav/surface/layout';

document.getElementById('app').innerHTML = [{ cols: 8, rows: 4 }, { cols: 4, rows: 8 }, { cols: 4, rows: 4 }]
  .map((g) => g.cols + ' x ' + g.rows + ': ' + gridOrientation(g)).join('<br>');
module Prints landscape for 8 x 4, portrait for 4 x 8, and landscape for 4 x 4.

Related

  • orientationOf() Says whether a box is 'landscape' or 'portrait' from its width and height; a square counts as landscape.
  • resolvePage() Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.

import { gridOrientation } from '@openav/surface/layout'; packages/surface/layout.js, line 41

haptic()#

function@openav/surface
haptic(ms, enabled = true)

Vibrates the phone for ms milliseconds where the browser can (Android Chrome); silent everywhere else, and enabled false turns it off.

It calls navigator.vibrate(ms) when that exists and swallows any error. Browsers without it, such as Safari on iPhone, simply do nothing, and Chrome only vibrates after the person has touched the page. Surface calls it for buttons, toggles and radios, and its haptics: false option turns that off.

Parameters

NameTypeDescription
msNumberHow long to vibrate, in milliseconds. A button press in the surface uses 10 to 14.
enabledBooleanPass false to do nothing: a page can offer a "haptics" switch and hand it here. optional default true

Examples

js
import { haptic } from '@openav/surface';

document.querySelector('button').addEventListener('pointerdown', () => haptic(20));   // a short buzz on every press
static On an Android phone the button buzzes for 20 ms each time it is pressed; elsewhere nothing happens.
js
import { haptic } from '@openav/surface';

const log = [];
navigator.vibrate = (ms) => { log.push('vibrate ' + ms + ' ms'); return true; };   // a stand-in, so there is something to see

haptic(20);
haptic(12, false);    // switched off: nothing is called
haptic(30, true);
document.getElementById('app').innerHTML = log.join('<br>');
module Prints "vibrate 20 ms" and "vibrate 30 ms": the call with enabled false made no vibration.

To be confirmed

Not felt on a real Android phone for this entry. The example uses a stand-in for navigator.vibrate to show when it is called.

Related

  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
  • keepAwake() Asks the browser to keep the screen on (a screen wake lock) and asks again when the tab comes back. Returns { release() }.
  • lockViewport() Makes a page behave like an instrument panel instead of a web page: no scrolling, no pinch zoom, no rubber-banding. Returns a function that undoes most of it.

import { haptic } from '@openav/surface'; packages/surface/platform.js, line 43

injectTheme()#

function@openav/surface
injectTheme()

Writes the surface's stylesheet into the page, once: the 15 theme variables on :root and the rules for every widget. Safe to call again.

Surface and mountRemote() call it for you, so you need it only when you build a widget by yourself, or when you want the variables (var(--oav-accent)) for your own elements. It adds one <style id="openav-surface-css"> to the head, and does nothing if that already exists, or if there is no document. To change the look, set a variable after that: document.documentElement.style.setProperty('--oav-accent', '#ff5fb3').

Examples

js
import { injectTheme } from '@openav/surface';

const read = () => getComputedStyle(document.documentElement).getPropertyValue('--oav-accent').trim() || '(not set)';
const before = read();
injectTheme();
injectTheme();   // the second call does nothing
document.getElementById('app').innerHTML = [
  'accent before: ' + before,
  'accent after: ' + read(),
  'style tags: ' + document.querySelectorAll('#openav-surface-css').length,
].join('<br>');
module Prints "(not set)" before, #7ea6ff after, and exactly one style tag even though it was called twice.
js
import { Surface, injectTheme } from '@openav/surface';

injectTheme();
document.documentElement.style.setProperty('--oav-accent', '#ff5fb3');   // re-skin: pink instead of blue

const box = document.getElementById('app');
box.style.cssText = 'position:relative;width:260px;height:160px';
new Surface(box, {
  layout: { pages: [{ id: 'main', grid: { cols: 3, rows: 2 }, widgets: [
    { id: 'a', type: 'fader', x: 0, y: 0, w: 1, h: 2, def: 0.7 }, { id: 'b', type: 'knob', x: 1, y: 0, w: 2, h: 2, def: 0.4 },
  ] }] },
  sink() {},
});
module A fader and a knob drawn in pink: they have no colour of their own, so they take the accent that was overridden.

Related

  • THEME_VARS The look of every control as 15 CSS custom properties on :root (--oav-bg, --oav-accent …): override them to re-skin the whole surface.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
  • WIDGET_CLASSES The widget classes, one per type, that Surface builds from a layout: WIDGET_CLASSES.fader, .knob, .xy and so on (advanced).
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.

import { injectTheme } from '@openav/surface'; packages/surface/theme.js, line 164

keepAwake()#

function@openav/surface
keepAwake()

Asks the browser to keep the screen on (a screen wake lock) and asks again when the tab comes back. Returns { release() }.

A phone that is a controller dims and locks within a minute unless something holds the screen on. keepAwake() requests navigator.wakeLock.request('screen') right away if the page is visible. The operating system drops the lock whenever the page is hidden, so it asks again each time the page becomes visible. It is best effort and silent: a browser with no wake lock, or one that says no, just means the screen may still dim, and it never throws.

Before you call it

A page that is visible, on https or localhost (the wake lock only exists on secure pages). mountRemote() calls it for you.

Returns

{ release } release() stops asking and lets go of the lock.

Examples

js
import { keepAwake } from '@openav/surface';

const awake = keepAwake();            // the screen stays on while this page is visible

// when the show is over:
// awake.release();
static On a phone or a laptop the screen stops dimming while the page is open and visible.
js
import { keepAwake } from '@openav/surface';

const log = [];
// a stand-in for the browser's wake lock, so there is something to see on any machine
Object.defineProperty(navigator, 'wakeLock', { configurable: true, value: {
  request: async (type) => { log.push('request ' + type); return { release() { log.push('lock released'); } }; },
} });

const awake = keepAwake();
await new Promise((r) => setTimeout(r, 50));
awake.release();
log.push('release() called');
document.getElementById('app').innerHTML = log.join('<br>');
module Prints "request screen", then "lock released" and "release() called": one request when the page is visible, and release() lets the lock go.

To be confirmed

Not tried on a real iPhone or iPad: Safari may refuse a wake-lock request that is not made inside a tap, in which case the screen can still dim. The second example only shows the calls, with a stand-in for navigator.wakeLock.

Related

  • lockViewport() Makes a page behave like an instrument panel instead of a web page: no scrolling, no pinch zoom, no rubber-banding. Returns a function that undoes most of it.
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.

import { keepAwake } from '@openav/surface'; packages/surface/platform.js, line 25

lockViewport()#

function@openav/surface
lockViewport()

Makes a page behave like an instrument panel instead of a web page: no scrolling, no pinch zoom, no rubber-banding. Returns a function that undoes most of it.

It sets <meta name="viewport"> to width=device-width,initial-scale=1,maximum-scale=1,user-scalable=no,viewport-fit=cover (making the tag if there is none) and adds a style that gives html and body no margin, the full height, overflow: hidden, overscroll-behavior: none and touch-action: none. iOS ignores user-scalable=no, so it also cancels gesturestart and gesturechange (pinch zoom) and the default of every touchmove, except one that starts in an input or textarea, so a text box still works.

This is for a page that is a control panel, such as mountRemote(), which calls it. Do not call it on a page that has to scroll. Without a document (in Node) it returns a function that does nothing.

Returns

Function Call it to remove the stylesheet and the listeners. The viewport <meta> stays as it was set.

Examples

js
import { lockViewport } from '@openav/surface';

const unlock = lockViewport();
const meta = document.querySelector('meta[name=viewport]');
const locked = getComputedStyle(document.body).overflow;
unlock();
document.getElementById('app').innerHTML = [
  'viewport: ' + meta.content,
  'body overflow while locked: ' + locked,
  'body overflow after unlock: ' + getComputedStyle(document.body).overflow,
].join('<br>');
module Prints the viewport string, "hidden" while locked, and "visible" after unlock(): the stylesheet goes away, the meta tag stays.

Related

  • keepAwake() Asks the browser to keep the screen on (a screen wake lock) and asks again when the tab comes back. Returns { release() }.
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.

import { lockViewport } from '@openav/surface'; packages/surface/platform.js, line 7

normalizeLayout()#

function@openav/surface
normalizeLayout(raw)

Returns a copy of a layout with the defaults filled in: ids, titles, grids and the version. The input is not changed.

A page without an id becomes page1, page2 …; without a title it gets its id (or Page 1); without a grid it gets { cols: 8, rows: 6 }. A widget without an id becomes its type and its position in the list, fader1; an id already used on the page gets _ added until it is unique. Every other field is kept as it is. Surface normalizes the layout it is given, and so do signalNames() and the show side of the remote.

Parameters

NameTypeDescription
rawObjectThe layout JSON. undefined or an object without pages gives a layout with no pages.

Returns

Object { version: 1, title, pages } with every page and widget complete.

Notes

  • The default grid has 6 rows. autoSurface() writes its own grids of 8 by 4, so this default only matters for layouts that leave the grid out.

Examples

js
import { normalizeLayout } from '@openav/surface';

const layout = normalizeLayout({ pages: [{ widgets: [{ type: 'fader' }, { type: 'fader', id: 'fader1' }, { type: 'knob' }] }] });
const page = layout.pages[0];
document.getElementById('app').innerHTML = [
  'page: ' + page.id + ' "' + page.title + '", grid ' + page.grid.cols + 'x' + page.grid.rows,
  'widgets: ' + page.widgets.map((w) => w.id).join(', '),
].join('<br>');
module Prints "page: page1 "Page 1", grid 8x6" and the widget ids fader1, fader1_ (the second fader1 was already taken), knob3.

Related

  • validateLayout() Checks a layout and returns a list of readable problems; an empty list means it is fine.
  • signalNames() Lists every signal name a layout can publish, so a mapping screen, a doc or a test can show them without touching the panel.
  • resolvePage() Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.

import { normalizeLayout } from '@openav/surface'; Also from @openav/surface/layout packages/surface/layout.js, line 70

orientationOf()#

function@openav/surface
orientationOf(w, h)

Says whether a box is 'landscape' or 'portrait' from its width and height; a square counts as landscape.

Surface calls it with the size of its container, not the size of the screen, so a panel in a half-width pane can be portrait while the phone is landscape. Use gridOrientation() for the shape of a grid.

Parameters

NameTypeDescription
wNumberThe width, in any unit.
hNumberThe height, in the same unit.

Returns

'landscape' | 'portrait' 'landscape' when w >= h, else 'portrait'.

Examples

js
import { orientationOf } from '@openav/surface/layout';

document.getElementById('app').innerHTML = [[844, 390], [390, 844], [500, 500]]
  .map(([w, h]) => w + ' x ' + h + ': ' + orientationOf(w, h)).join('<br>');
module Prints landscape for 844 x 390, portrait for 390 x 844, and landscape for the square.

Related

  • gridOrientation() Says whether a grid { cols, rows } is 'landscape' (at least as many columns as rows) or 'portrait'.
  • resolvePage() Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.

import { orientationOf } from '@openav/surface/layout'; packages/surface/layout.js, line 40

planKeyboard()#

function@openav/surface
planKeyboard({ width, height, base = 48, minKeyPx = 38, maxWhites = 22, stackBelow = 2.2, forceRows = 0 })

Decides how many piano keys, in how many rows, fit a box: one wide row, or two stacked rows when the box is tall, so keys stay a playable size.

A piano is two and a half metres wide and a phone held upright is six centimetres, so shrinking the keys makes a toy. Instead the keyboard changes shape with its box. A wide box gets one row with as many octaves as fit at minKeyPx or wider. A tall box gets two rows, the upper one continuing the pitch range where the lower stops, within reach of a thumb. The rows end on E or B and the next one starts on the next white key, so no black key is left alone at the seam. Notes stop at 127. The keyboard widget of Surface calls this every time its box changes size.

Parameters

NameTypeDescription
widthNumberThe width of the box, in pixels. optional
heightNumberThe height of the box, in pixels. optional
baseNumberThe MIDI note of the lowest key. 48 is C3 (C4 is 60). optional default 48
minKeyPxNumberThe narrowest a white key may be, in pixels. At least 3 white keys are kept. optional default 38
maxWhitesNumberThe most white keys in a row. Two stacked rows are limited to 14 each. optional default 22
stackBelowNumberStack into two rows when width ÷ height is below this. optional default 2.2
forceRows1 | 21 or 2 to decide yourself; 0 lets the shape of the box decide. optional default 0

Returns

{ stacked, whites, keyPx, rows } stacked is true for two rows, whites the white keys in a row, keyPx the width of a white key. rows is a list of { base, semitones, whites }, the top row first (the higher notes), the way two keyboards are read: base is the MIDI note of the first key and semitones how many notes the row spans.

Examples

js
import { planKeyboard } from '@openav/surface';

const show = (label, plan) => label + ': ' + (plan.stacked ? 'two rows' : 'one row') + ', ' + plan.whites + ' white keys of ' +
  Math.round(plan.keyPx) + ' px, ' + plan.rows.map((r) => 'notes ' + r.base + '–' + (r.base + r.semitones - 1)).join(' / ');
document.getElementById('app').innerHTML = [
  show('phone on its side (844 x 330)', planKeyboard({ width: 844, height: 330 })),
  show('phone upright (390 x 640)', planKeyboard({ width: 390, height: 640 })),
].join('<br>');
module Prints one row of 21 white keys (notes 48 to 83) for the wide box, and two rows of 10 (notes 65 to 81 on top, 48 to 64 below) for the tall one.

Related

  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
  • WIDGET_TYPES The names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.

import { planKeyboard } from '@openav/surface'; packages/surface/kbplan.js, line 41

resolvePage()#

function@openav/surface
resolvePage(page, orientation)

Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.

A phone gets turned, so a layout has to survive it. The rules, in order: 1. If the page has a portrait or landscape entry for this orientation, use its grid and its place ({ widgetId: { x, y, w, h } }); widgets it does not mention keep their own position, or are packed into free cells (source is 'explicit'). 2. Else, if the base grid has the right shape (more columns than rows is landscape) and every widget has an x and a y, use the base ('base'). 3. Else reflow: the same widgets in reading order, packed again into half the columns (portrait, at least 2) or twice as many (landscape, up to 16), with a lone last row stretched to the edge by stretchRows() ('reflow').

Rows are a count, not pixels, so the result always fills the screen and nothing scrolls. A widget without w and h takes the size of its type from DEFAULT_SIZE. Surface calls this for every page each time the orientation changes.

Parameters

NameTypeDescription
pageObjectOne page of a layout: { grid, widgets, portrait?, landscape? }.
orientation'portrait' | 'landscape'Which way the screen is held.

Returns

{ grid, placements, source } grid is { cols, rows }, placements a list of { id, x, y, w, h }, source is 'explicit', 'base' or 'reflow'.

Examples

js
import { resolvePage } from '@openav/surface/layout';

// eight faders in a row: right for a landscape screen, too wide for a portrait one
const page = {
  grid: { cols: 8, rows: 4 },
  widgets: Array.from({ length: 8 }, (_, i) => ({ id: 'f' + i, type: 'fader', x: i, y: 0, w: 1, h: 4 })),
};
const out = ['landscape', 'portrait'].map((o) => {
  const r = resolvePage(page, o);
  return o + ': ' + r.source + ', ' + r.grid.cols + ' columns x ' + r.grid.rows + ' rows';
});
document.getElementById('app').innerHTML = out.join('<br>');
module Prints "landscape: base, 8 columns x 4 rows" and "portrait: reflow, 4 columns x 8 rows": turned upright, the eight faders become two rows of four.
js
import { resolvePage } from '@openav/surface/layout';

// a page that says where things go when the phone is upright
const page = {
  grid: { cols: 8, rows: 4 },
  widgets: [
    { id: 'a', type: 'fader', x: 0, y: 0, w: 1, h: 4 },
    { id: 'b', type: 'knob', x: 1, y: 0 },
    { id: 'c', type: 'button' },
  ],
  portrait: { grid: { cols: 4, rows: 6 }, place: { a: { x: 3, y: 0, w: 1, h: 6 } } },
};
const r = resolvePage(page, 'portrait');
document.getElementById('app').innerHTML = [r.source, ...r.placements.map((p) => p.id + ': ' + p.x + ',' + p.y + ' ' + p.w + '×' + p.h)].join('<br>');
module Prints "explicit": a goes where the portrait entry says, the knob and the button (which have no place) fill the free cells.

Related

  • autoArrange() Packs widgets into a grid first-fit, row by row, and never overlaps; fixed placements are kept and block their cells. Returns the placements and the rows used.
  • stretchRows() Stretches a last row that stops short to the edge of the grid: five faders in four columns leave one lonely fader, and this makes it four columns wide.
  • orientationOf() Says whether a box is 'landscape' or 'portrait' from its width and height; a square counts as landscape.
  • gridOrientation() Says whether a grid { cols, rows } is 'landscape' (at least as many columns as rows) or 'portrait'.
  • normalizeLayout() Returns a copy of a layout with the defaults filled in: ids, titles, grids and the version. The input is not changed.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.

import { resolvePage } from '@openav/surface'; Also from @openav/surface/layout packages/surface/layout.js, line 116

routesFromLayout()#

function@openav/surface
routesFromLayout(layout, params = [], { smooth = 0.04 } = {})

Writes the mapper routes for a hand-written layout: every widget that names a param in target gets a route from its signal to that param.

Give a widget a target and the layout file is enough to perform a world. target is a param key, or an array for the widgets with several signals: an xy sends x to the first and y to the second, a bank sends channel i to the i-th. Every other widget uses its first (or only) target. The routes already scale the 0..1 signal to the param's min and max. mountRemoteHost() uses this for a layout from world.surface or from its surface option.

Parameters

NameTypeDescription
layoutObjectThe layout JSON, whose widgets may have a target.
paramsObject[]The param declarations. A target that is not among them is skipped without a warning. optional default []
smoothNumberSeconds of smoothing on continuous routes. A widget's own smooth wins. Pulse params and button, toggle and radio widgets are never smoothed. optional default 0.04

Returns

Object[] Routes in the same shape autoSurface() makes: { source, target, inMin, inMax, curve, smooth, outMin, outMax }, with curve taken from the widget (linear if it has none).

Examples

js
import { routesFromLayout } from '@openav/surface/auto';

const params = [
  { key: 'size', min: 1, max: 10 }, { key: 'cx', min: 0, max: 1 }, { key: 'cy', min: 0, max: 1 }, { key: 'burst', pulse: true },
];
const layout = { pages: [{ id: 'p', widgets: [
  { id: 'size', type: 'fader', target: 'size' },
  { id: 'pos', type: 'xy', target: ['cx', 'cy'] },
  { id: 'go', type: 'button', target: 'burst' },
  { id: 'note', type: 'label' },
] }] };
document.getElementById('app').innerHTML = routesFromLayout(layout, params)
  .map((r) => r.source + ' → ' + r.target).join('<br>');
module Prints surface/p/size → size, surface/p/pos/x → cx, surface/p/pos/y → cy, surface/p/go → burst: the label has no target, so no route.

Related

  • autoSurface() Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
  • signalNames() Lists every signal name a layout can publish, so a mapping screen, a doc or a test can show them without touching the panel.
  • mountRemoteHost() The show side of the phone remote: joins the relay as the runner, files phone signals, builds the control panel from the params and echoes values back.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.

import { routesFromLayout } from '@openav/surface'; Also from @openav/surface/auto packages/surface/auto.js, line 156

signalNames()#

function@openav/surface
signalNames(layout)

Lists every signal name a layout can publish, so a mapping screen, a doc or a test can show them without touching the panel.

It follows the table in WIDGET_TYPES: an XY pad gives /x, /y, /down; a bank gives /1 to /count; pads give /hit and /0 to /<rows × cols − 1>; an encoder gives its name and /delta; a keyboard gives midi/note/on, midi/note/off, midi/cc/64 and midi/virtual; a label or a meter gives nothing. The …/raw names that a widget with a range also sends are not in the list.

Parameters

NameTypeDescription
layoutObjectThe layout JSON (it is normalized first).

Returns

string[] Unique names such as surface/main/hue, surface/main/pad/x, midi/note/on, in page and widget order.

Examples

js
import { signalNames } from '@openav/surface/layout';

const names = signalNames({ pages: [{ id: 'main', widgets: [
  { id: 'hue', type: 'knob' },
  { id: 'pos', type: 'xy' },
  { id: 'mix', type: 'bank', count: 2 },
  { id: 'title', type: 'label' },
] }] });
document.getElementById('app').innerHTML = names.join('<br>');
module Prints surface/main/hue, then pos/x, pos/y, pos/down, then mix/1 and mix/2: the label gives nothing.

Related

  • WIDGET_TYPES The names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
  • normalizeLayout() Returns a copy of a layout with the defaults filled in: ids, titles, grids and the version. The input is not changed.
  • routesFromLayout() Writes the mapper routes for a hand-written layout: every widget that names a param in target gets a route from its signal to that param.

import { signalNames } from '@openav/surface'; Also from @openav/surface/layout packages/surface/layout.js, line 172

signalSink()#

function@openav/surface
signalSink(signals)

Makes a sink for a Surface that writes straight into a Signals registry: for a panel on the same page as the show, with no phone and no relay.

A name it has not met is declared first: { kind: 'pulse' } when info.pulse is set, otherwise { min: 0, max: 1 }. Then a pulse fires with signals.pulse() and anything else is signals.set(). So a route in the mapper, { source: 'surface/main/hue', target: 'hue' }, works whether the finger is on this page or on a phone.

It does no scaling: the …/raw signal that a widget with a range sends is filed like any other name, with the range 0..1 it was declared with, so ignore those or give them their own declaration.

Parameters

NameTypeDescription
signalsSignalsThe registry to write into (from @openav/core).

Returns

Function A (name, value, info) => {} to pass as Surface's sink.

Examples

js
import { signalSink } from '@openav/surface';
import { Signals } from '@openav/core';

const signals = new Signals();
const sink = signalSink(signals);

sink('surface/main/level', 0.4, {});               // a continuous value
sink('surface/main/go', 1, { pulse: true });       // an event

document.getElementById('app').innerHTML = [
  'level = ' + signals.get('surface/main/level') + ', kind ' + signals.meta.get('surface/main/level').kind,
  'go = ' + signals.get('surface/main/go') + ', kind ' + signals.meta.get('surface/main/go').kind,
].join('<br>');
module Prints "level = 0.4, kind continuous" and "go = 1, kind pulse": the sink declared each name the first time it saw it.

Related

  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
  • bindSignals() Connects a runner's RelayClient to a Signals registry: every signal a phone sends is filed there, plus a phone/any/… copy. Returns an undo function.
  • fileSignal() Files one incoming signal into a Signals registry: declares the name the first time, fills in the velocity, then sets it or pulses it.

import { signalSink } from '@openav/surface'; packages/surface/index.js, line 34

stretchRows()#

function@openav/surface
stretchRows(placements, cols)

Stretches a last row that stops short to the edge of the grid: five faders in four columns leave one lonely fader, and this makes it four columns wide.

A row is stretched only when it is safe: its widgets are all the same height, side by side with no gaps from the left edge, and nothing else sits to their right. The extra columns are dealt out one at a time from left to right, so a short row of three in four columns gives the first widget two columns. A row that does not qualify is left alone, and nothing ever overlaps. resolvePage() calls it at the end of a reflow.

Parameters

NameTypeDescription
placements{ id, x, y, w, h }[]Placements, for example from autoArrange(). Not changed: copies come back.
colsNumberHow many columns the grid has.

Returns

Object[] The placements, with the short rows widened.

Notes

  • Advanced. You rarely call it yourself.

Examples

js
import { autoArrange, stretchRows } from '@openav/surface/layout';

const five = autoArrange(Array.from({ length: 5 }, (_, i) => ({ id: 'f' + i, w: 1, h: 4 })), 4).placements;
const stretched = stretchRows(five, 4);
const show = (list) => list.map((p) => p.id + ' w' + p.w).join(', ');
document.getElementById('app').innerHTML = 'before: ' + show(five) + '<br>after: ' + show(stretched);
module Prints f0 to f3 at width 1 both times, and f4 going from width 1 to width 4.

Related

  • autoArrange() Packs widgets into a grid first-fit, row by row, and never overlaps; fixed placements are kept and block their cells. Returns the placements and the rows used.
  • resolvePage() Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.

import { stretchRows } from '@openav/surface/layout'; packages/surface/layout.js, line 154

Surface#

class@openav/surface
new Surface(container, { layout, sink, haptics = true, tabs = true, orientation = null, onPage = null } = {})

A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.

Pass new Surface(container, { layout, sink }) and a panel appears: faders, knobs, an XY pad, pads, a keyboard, whatever the layout names, each with its live number. Widgets are dumb on purpose. Fingers go in, named normalized signals come out of sink, and the surface never knows what the work is. Run it on the show page with signalSink(), or on a phone with a sink that sends to the relay: the same layout works for both. A world's params can grow the whole layout by themselves with autoSurface().

The grid always fills the container and nothing scrolls. When the container is turned, the page is laid out again for the new orientation by resolvePage(): a portrait or landscape variant in the layout if there is one, else the base grid, else the same widgets packed again into half or double the columns. Touching a widget outlines it in its colour, and a double-tap on a fader or knob returns it to its default.

Runner to widget goes the other way: feedback(name, value) moves a fader to follow the timeline, or fills a meter. A widget under a finger ignores feedback while it is touched and for about half a second after, so it does not fight the hand (a button only while it is held).

Before you call it

A browser page (it builds DOM) and a container with a size. The panel injects its look once with injectTheme(). Nothing else to start: it works on the first touch. On a phone page, lockViewport() stops the page from scrolling and zooming under the fingers.

Parameters

NameTypeDescription
containerHTMLElementWhere the panel goes. The panel is position: absolute and fills it, so the container needs a position (position: relative) and a size, or the panel has none.
layoutObjectThe layout JSON (see validateLayout()): pages, a grid, widgets. A bad layout throws an error that lists every problem. optional
sinkFunction(name, value, info) => {}, called for every signal a touch makes. name is like surface/main/hue, value is normalized 0..1 (an event is info.pulse, a string for a text box). When a widget's range is not 0..1, info.raw holds the unscaled value, and a second call follows with the name …/raw. signalSink() writes into a Signals; a phone passes one that calls relay.set. optional
hapticsBooleanBuzz the phone briefly on a button, a toggle, a radio choice or a double-tap reset, where the browser can (Android Chrome). false turns it off. optional default true
tabsBooleanShow a tab strip when the layout has more than one page. false hides it, and you switch with setPage(). optional default true
orientation'portrait' | 'landscape'Force the orientation. Left out, it follows the container's shape (wider than tall is landscape) and changes when the container does. optional default null
onPageFunction(pageId) => {}, when the page changes, and once at the start with the first page. optional default null

Members

NameDescription
setLayout(raw)methodsetLayout(raw): rebuild the panel from a layout. Throws an Error that begins invalid surface layout: and lists every problem if validateLayout() finds any. Keeps the current page when the new layout has one with the same id. Returns the surface.
setPage(id)methodsetPage(id): show the page with this id, and call onPage.
feedback(name, value)methodfeedback(name, value): runner to widget. name is the full signal name such as surface/main/level, value is normalized 0..1. A name that no widget listens to is ignored.
widget(page, id)methodwidget(page, id): the widget object for a page id and a widget id, or undefined. Its element is .el.
disposeWidgets()methoddisposeWidgets(): remove every widget and its listeners but keep the panel. setLayout() calls it first.
dispose()methoddispose(): stop watching the container's size, remove the widgets and take the panel out of the page.

Notes

  • Public fields: layout (the normalized layout), root (the panel's element), orientation, pageId, widgets (a Map of "page/id" to widget), sink, haptics.
  • The constructor appends the panel to the container before it checks the layout, so a layout that throws there leaves an empty panel behind. Run validateLayout() first if the layout comes from a file or a person.

Examples

js
import { Surface } from '@openav/surface';

const box = document.getElementById('app');
box.style.cssText = 'position:relative;width:360px;height:220px';   // the panel fills its container

const out = document.createElement('pre');
box.after(out);
new Surface(box, {
  layout: { pages: [{ id: 'main', grid: { cols: 4, rows: 2 }, widgets: [
    { id: 'size', type: 'fader', label: 'Size', x: 0, y: 0, w: 1, h: 2, min: 1, max: 10, def: 4 },
    { id: 'pad', type: 'xy', x: 1, y: 0, w: 2, h: 2 },
    { id: 'go', type: 'button', label: 'Go', x: 3, y: 0, w: 1, h: 2 },
  ] }] },
  sink: (name, value, info) => {
    out.textContent = name + ' = ' + value + (info.raw !== undefined ? '  (raw ' + info.raw + ')' : '');
  },
});
module A dark panel with a fader, an XY pad and a button. Touch one and the line under it shows the signal it sent, for example surface/main/size = 0.75 (raw 7.75).
js
import { Surface } from '@openav/surface';

const box = document.getElementById('app');
box.style.cssText = 'position:relative;width:360px;height:220px';

const surface = new Surface(box, {
  layout: { pages: [{ id: 'main', grid: { cols: 4, rows: 2 }, widgets: [
    { id: 'level', type: 'meter', label: 'Level', x: 0, y: 0, w: 1, h: 2 },
    { id: 'echo', type: 'fader', label: 'Echo', x: 1, y: 0, w: 1, h: 2 },
  ] }] },
  sink() {},
});

function frame(ms) {
  const v = 0.5 + 0.5 * Math.sin(ms / 500);
  surface.feedback('surface/main/level', v);       // the meter fills
  surface.feedback('surface/main/echo', 1 - v);    // the fader follows
  requestAnimationFrame(frame);
}
requestAnimationFrame(frame);
module The meter rises and falls with a sine wave and the fader moves the opposite way: the show sending values back to the phone with feedback().
js
import { Surface } from '@openav/surface';

const box = document.getElementById('app');
box.style.cssText = 'position:relative;width:360px;height:220px';
const surface = new Surface(box, {
  layout: { pages: [{ id: 'a', widgets: [{ id: 'x', type: 'fader' }] }] },
  sink() {},
});

try {
  surface.setLayout({ pages: [{ id: 'a', widgets: [{ id: 'x', type: 'dial' }, { id: 'x', type: 'knob' }] }] });
} catch (e) {
  document.getElementById('app').after(Object.assign(document.createElement('pre'), { textContent: e.message }));
}
module A bad layout throws and lists every problem (an unknown type "dial", a duplicate id "x"); the panel that was already there stays as it was.

Related

  • signalSink() Makes a sink for a Surface that writes straight into a Signals registry: for a panel on the same page as the show, with no phone and no relay.
  • validateLayout() Checks a layout and returns a list of readable problems; an empty list means it is fine.
  • autoSurface() Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
  • resolvePage() Works out where every widget of one page goes for one orientation: the grid and a cell for each widget, and which rule produced it.
  • injectTheme() Writes the surface's stylesheet into the page, once: the 15 theme variables on :root and the rules for every widget. Safe to call again.
  • lockViewport() Makes a page behave like an instrument panel instead of a web page: no scrolling, no pinch zoom, no rubber-banding. Returns a function that undoes most of it.
  • WIDGET_TYPES The names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.

import { Surface } from '@openav/surface'; packages/surface/index.js, line 41

THEME_VARS#

constant@openav/surface
THEME_VARS = object, 15 keys

The look of every control as 15 CSS custom properties on :root (--oav-bg, --oav-accent …): override them to re-skin the whole surface.

The design is a set of rules rather than decoration, because a controller is used in the dark, fast and half-looking: dark cards on near-black, the only saturated thing on screen is a value, every touch target at least 44 px (--oav-touch-min), the number you are changing always visible, and a pressed widget outlines itself in its colour. Everything is a CSS variable, so a work re-skins the surface by overriding a few of them, never by editing widget code.

injectTheme() writes these values into the page once. Override after that, on document.documentElement or in your own stylesheet, with a rule that comes later on :root.

Type
Object

Property name to its default value: --oav-bg #0a0c11, --oav-surface #141922, --oav-surface-2 #1b2230, --oav-line #283044, --oav-text #e8ecf5, --oav-dim #8691a8, --oav-accent #7ea6ff, --oav-ok #3ddc84, --oav-warn #f5a524, --oav-bad #e5484d, --oav-radius 14px, --oav-gap 8px, --oav-touch-min 44px, --oav-font (the system font) and --oav-mono (a monospace font).

Examples

js
import { THEME_VARS } from '@openav/surface';

document.getElementById('app').innerHTML = ['--oav-bg', '--oav-accent', '--oav-radius', '--oav-touch-min']
  .map((k) => k + ': ' + THEME_VARS[k]).join('<br>');
module Prints the background #0a0c11, the accent #7ea6ff, the corner radius 14px and the 44px minimum touch size.

Related

  • injectTheme() Writes the surface's stylesheet into the page, once: the 15 theme variables on :root and the rules for every widget. Safe to call again.
  • COLOR_NAMES The seven colour names a widget's color may be: cyan, amber, magenta, lime, violet, coral, white.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.

import { THEME_VARS } from '@openav/surface'; packages/surface/theme.js, line 14

toggleFullscreen()#

function@openav/surface
toggleFullscreen()

Enters fullscreen if the page is not in it, and leaves it if it is. Call it from a tap, because browsers refuse otherwise.

It asks for the whole page (document.documentElement) with the browser's navigation bar hidden, and quietly does nothing on a browser without fullscreen (see canFullscreen()). The phone app puts it on a ⛶ button in the status bar.

Before you call it

A user gesture: call it from a click or pointerdown handler. Called on page load it is refused by the browser.

Examples

js
import { toggleFullscreen, canFullscreen } from '@openav/surface';

const button = document.createElement('button');
button.textContent = '⛶ fullscreen';
button.hidden = !canFullscreen();
button.onclick = toggleFullscreen;
document.body.append(button);
static A button that puts the page into fullscreen and out again, and is hidden where the browser has no fullscreen.

Related

  • canFullscreen() True when the page can ask for fullscreen: the browser has requestFullscreen (or the WebKit one). The code treats iPhone Safari as not having it.
  • mountRemote() The whole phone app in one call: a status bar and the tabs sense, keys, control and MIDI, talking to the relay as a controller. /packages/remote/ is exactly this.

import { toggleFullscreen } from '@openav/surface'; packages/surface/platform.js, line 36

validateLayout()#

function@openav/surface
validateLayout(layout)

Checks a layout and returns a list of readable problems; an empty list means it is fine.

It looks for: no object, or no non-empty pages array (it stops there); a page without an id, with a duplicate id or an id outside A–Z a–z 0–9 _ -; a widget whose type is not in WIDGET_TYPES, without an id, with a duplicate id on its page or an id outside those characters (the id becomes part of a signal name); and a min equal to its max. It reports every problem it finds, not only the first.

It does not look at positions, grids or targets: a widget that overlaps another, or a target that is not a param, is not a problem here. Surface's setLayout() runs it and throws with the list when it is not empty.

Parameters

NameTypeDescription
layoutObjectThe raw layout JSON.

Returns

string[] One sentence per problem, with where it is (pages[0].widgets[2]: unknown type "dial"). [] when the layout is fine.

Examples

js
import { validateLayout } from '@openav/surface';

const good = validateLayout({ pages: [{ id: 'main', widgets: [{ id: 'size', type: 'fader' }] }] });
const bad = validateLayout({ pages: [{ id: 'main page', widgets: [
  { id: 'a', type: 'dial' }, { id: 'a', type: 'fader' }, { type: 'knob', min: 1, max: 1 },
] }] });
document.getElementById('app').innerHTML = ['good: ' + JSON.stringify(good), ...bad].join('<br>');
module Prints "good: []", then five problems for the bad layout: the page id, the unknown type, the duplicate id, the missing id and min equal to max.

Related

  • normalizeLayout() Returns a copy of a layout with the defaults filled in: ids, titles, grids and the version. The input is not changed.
  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
  • WIDGET_TYPES The names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.

import { validateLayout } from '@openav/surface'; Also from @openav/surface/layout packages/surface/layout.js, line 44

WIDGET_CLASSES#

constant@openav/surface
WIDGET_CLASSES = object, 14 keys

The widget classes, one per type, that Surface builds from a layout: WIDGET_CLASSES.fader, .knob, .xy and so on (advanced).

Most pages never touch it: Surface looks each spec.type up here. It is exported for code that wants a single widget without a whole panel. spec is the widget's entry of the layout ({ id, type, label, min, max, … }) and ctx is { page, out(name, value, info), haptic(ms) }, where out is where the widget's signals go and page is the page id used in their names. A widget's element is .el; put it in the page yourself.

Adding your own key here does not make a new layout type: validateLayout() only accepts the names in WIDGET_TYPES.

Type
Object

An object with the same 14 keys as WIDGET_TYPES, each a class: new WIDGET_CLASSES[type](spec, ctx).

Notes

  • Advanced. The keyboard widget wraps KeysPiano from @openav/keys.

Examples

js
import { WIDGET_CLASSES } from '@openav/surface';

const sent = [];
const ctx = { page: 'solo', out: (name, value) => sent.push(name + ' = ' + value), haptic() {} };
const toggle = new WIDGET_CLASSES.toggle({ id: 'mute', type: 'toggle', label: 'Mute' }, ctx);
document.getElementById('app').append(toggle.el);

toggle.set(true);    // what a tap does
toggle.set(false);
document.getElementById('app').append(Object.assign(document.createElement('pre'), { textContent: sent.join('\n') }));
module A lone toggle on the page, and under it what it sent when it was switched on and off: surface/solo/mute = 1, then 0.

Related

  • Surface A TouchOSC-style control panel: draws the pages and widgets of a layout in a container, and sends each touch out as a named signal.
  • WIDGET_TYPES The names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.

import { WIDGET_CLASSES } from '@openav/surface'; packages/surface/widgets.js, line 510

WIDGET_TYPES#

constant@openav/surface
WIDGET_TYPES = array, 14 items

The names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.

Each widget publishes under surface/<page>/<id>, normalized 0..1; when its range is not 0..1 the unscaled value also goes out as …/raw. What each type sends:

text
fader, knob, number   <id>
button                 <id>  1 while held, 0 on release
toggle                 <id>  1 or 0
encoder                <id> (phase 0..1, wraps) and <id>/delta (turns)
xy                     <id>/x  <id>/y  <id>/down
bank                   <id>/1 … <id>/N        (count, default 8)
radio                  <id> = index / (n - 1), raw = index
pads                   <id>/hit {pad,row,col,vel}  and  <id>/<n> while held
keyboard               midi/note/on  midi/note/off  midi/cc/64
text                   <id>  a string, as a pulse
label, meter           nothing: they only receive feedback

Fields every widget accepts: id, type, label, min, max, def, step, unit, color (one of COLOR_NAMES) and, for hand-written layouts, target, the param key that routesFromLayout() turns into a route. Others by type: orient (v, h or auto) on a fader, spring: true on an XY pad, count and labels on a bank, options on a radio, rows, cols and labels on pads, pulse on a button.

Type
string[]

The keys of DEFAULT_SIZE, in that order. validateLayout() rejects any type that is not in it.

Examples

js
import { WIDGET_TYPES, DEFAULT_SIZE } from '@openav/surface';

document.getElementById('app').innerHTML = WIDGET_TYPES
  .map((t) => t + ' ' + DEFAULT_SIZE[t].w + '×' + DEFAULT_SIZE[t].h)
  .join('<br>');
module Lists the 14 types with their natural size in grid cells, from fader 1×4 to meter 1×4.

Related

  • DEFAULT_SIZE The natural size of each widget type, in grid cells, as { w, h }: a layout only has to give x, y, w, h when it cares.
  • WIDGET_CLASSES The widget classes, one per type, that Surface builds from a layout: WIDGET_CLASSES.fader, .knob, .xy and so on (advanced).
  • validateLayout() Checks a layout and returns a list of readable problems; an empty list means it is fine.
  • signalNames() Lists every signal name a layout can publish, so a mapping screen, a doc or a test can show them without touching the panel.
  • COLOR_NAMES The seven colour names a widget's color may be: cyan, amber, magenta, lime, violet, coral, white.

import { WIDGET_TYPES } from '@openav/surface'; Also from @openav/surface/layout packages/surface/layout.js, line 38

widgetTypeFor()#

function@openav/surface
widgetTypeFor(p, { continuousCount = 0, style = 'auto' } = {})

Picks the widget type for one param declaration: a button for a pulse, a toggle, a radio, a fader or a knob.

The rules, first match wins: a surface: { type } hint on the param; pulse is a button; step: 1 over a range of 1 (two values) is a toggle; step: 1 over a range of 2 to 6 (three to seven values) is a radio; style 'knob' or 'fader' as asked; otherwise a fader, or a knob when continuousCount is above 6. A missing min is 0 and a missing max is 1. autoSurface() calls it for every param.

Parameters

NameTypeDescription
pObjectA param: { key, min?, max?, step?, pulse?, surface? }.
continuousCountNumberHow many continuous params the panel has in all. More than 6 turns faders into knobs. optional default 0
style'auto' | 'fader' | 'knob'Force faders or knobs for the continuous params. optional default 'auto'

Returns

String One of the names in WIDGET_TYPES.

Examples

js
import { widgetTypeFor } from '@openav/surface/auto';

const params = [
  { key: 'hue', min: 0, max: 360 },
  { key: 'mode', min: 0, max: 3, step: 1 },
  { key: 'mirror', min: 0, max: 1, step: 1 },
  { key: 'burst', pulse: true },
];
const rows = params.map((p) => p.key + ' → ' + widgetTypeFor(p));
rows.push('hue with 9 continuous params → ' + widgetTypeFor(params[0], { continuousCount: 9 }));
document.getElementById('app').innerHTML = rows.join('<br>');
module Prints hue → fader, mode → radio, mirror → toggle, burst → button, and a knob for hue once the panel has nine continuous params.

Related

  • autoSurface() Turns a world's params into a control panel: a layout, the routes that drive the params, and the bindings for echoing values back.
  • WIDGET_TYPES The names of the 14 widget types a layout may use: fader, knob, encoder, button, toggle, xy, bank, radio, pads, keyboard, number, text, label, meter.

import { widgetTypeFor } from '@openav/surface'; Also from @openav/surface/auto packages/surface/auto.js, line 32

Console and monitor#

The director's console and the backstage monitor feed. 4

@openav/console

mountConsole()#

function@openav/console
mountConsole(root, app, opts = {})

Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.

The desk reads the same objects the loop runs and owns nothing. From top to bottom it has the transport (play, previous and next scene, reset, the clock) and a scrubber with a block for every scene (with a tick for every cue, and a dashed end for a hold, when the scenes have them); with a score, the Director panel; the Layers overview (L1 input, L2 mapping, L3 world, L4 output); L1 · Input (MIDI devices, 🎤 mic, 🖐 hands, 🕺 body, leap); L2 · Mapping (the routes with a mute box and ✕, 🎛 map knobs in order, ⬇ export, ⬆ import, clear); L3 · Params (a slider per param with ✕ to clear the override and a learn chip; a yellow label means overridden); L4 · sound when there is a sound engine; and the Signals meters with the MIDI log. Every panel header collapses its section.

Keyboard, on the whole window: Space play and pause (at a hold: release it), → and ← next and previous scene (→ at a hold releases it), R back to the start, T performance mode (a fullscreen teleprompter for the performer: scene title, note and acts, the clock, the next scene, a HOLD banner), F fullscreen, Esc stops the mapping wizard or leaves keyboard-piano capture. Typing in an input field, a held key and any combination with ⌘, Ctrl or Alt are ignored. While the on-screen piano captures the keyboard the letters (R, T, F) belong to the piano; Space, the arrows and Esc always work.

Before you call it

Call desk.render(state) every frame after stage.frame() (a show's loop does): the desk only redraws when you tell it to. state is the resolved params, params.resolve(timeline.state()).

Parameters

NameTypeDescription
rootHTMLElementThe element to build the desk in. It gets the class oav-console, and the desk's styles are added once to the head.
appObjectThe objects the desk reads: { timeline, params, mapper, signals, stage } are needed, and midi, controllers, keys, sound, audio, hands, pose, leap, chord, remote, drums, loop, osc, score, director are optional. A missing part hides its section. createShow() passes show.app.
optsObject{ layers: false } and { signals: false } leave out the Layers and Signals panels; { learnOrder: ['hue', 'size'] } sets the order of the 🎛 map knobs in order wizard (default: the params as declared, skipping hidden ones and any with wizard: false). optional default {}

Returns

Object { render(state), perf, director, wizard, dispose() }: render redraws the desk, perf is the performance mode (toggle(), active, render()), director the Director panel (or null), wizard the knob-mapping wizard, and dispose() removes the keyboard handler and empties the element.

Notes

  • The keyboard handler is on window, so two desks on one page both answer the keys. dispose() removes it. The performance mode's teleprompter is appended to document.body, not to root.
  • Touch a slider (or a mapped control) and that param leaves the timeline's hands until you press its ✕: the score gives way when a person grabs a control.

Examples

js
import { Signals, Params, Loop } from '@openav/core';
import { Mapper } from '@openav/mapping';
import { Timeline } from '@openav/timeline';
import { Stage } from '@openav/stage';
import { mountConsole } from '@openav/console';

const signals = new Signals();
const params = new Params([{ key: 'hue', label: 'Hue', min: 0, max: 360, def: 200 }, { key: 'size', min: 0, max: 1, def: 0.3 }]);
const mapper = new Mapper({ signals, params });
const timeline = new Timeline({ params, total: 60, automation: { hue: [[0, 200], [60, 320]] },
  scenes: [{ id: 'dawn', t: 0, title: 'Dawn' }, { id: 'noon', t: 30, title: 'Noon' }] });
const stage = new Stage({ container: document.createElement('div'), params, signals });

const desk = mountConsole(document.getElementById('app'), { timeline, params, mapper, signals, stage });
timeline.play();
new Loop((dt) => {
  timeline.advance(dt);
  desk.render(params.resolve(timeline.state()));      // once per frame
}).start();
module The desk appears in the page: a transport with a clock running, a scrubber with two scene blocks (Dawn, Noon), the layers overview and a slider each for Hue and Size. The Hue slider follows the timeline until you drag it.
js
import { Signals, Params } from '@openav/core';
import { Mapper } from '@openav/mapping';
import { Timeline } from '@openav/timeline';
import { Stage } from '@openav/stage';
import { mountConsole } from '@openav/console';

const signals = new Signals();
const params = new Params([{ key: 'hue', min: 0, max: 360, def: 200 }]);
const timeline = new Timeline({ params, total: 30 });
const app = { timeline, params, signals, mapper: new Mapper({ signals, params }), stage: new Stage({ container: document.createElement('div'), params, signals }) };

const desk = mountConsole(document.getElementById('app'), app, { layers: false, signals: false });
desk.render(params.resolve(timeline.state()));
console.log(desk.perf.active, desk.perf.toggle(), desk.perf.active);    // false true true: the T key, in code
desk.perf.toggle();                                                     // back to the desk
desk.dispose();                                                         // removes the key handler and empties the element
console.log(document.getElementById('app').children.length);            // 0
module A desk without the Layers and Signals panels: the console prints false true true as performance mode is switched on from code, then 0 after dispose() has emptied the element.

Related

  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • Timeline Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
  • Mapper Connects signals to params: it holds a list of routes (a window, a curve, smoothing) and writes the matching param whenever a signal arrives.
  • LearnWizard Binds a list of params one after another: it arms learn for the first, you turn a knob, it binds it and arms the next. Skip, back and stop steer it.
  • Director Runs a score's segment modules: enters each once, updates it every frame, exits it once, fires cues, and switches off a module that crashes without stopping the show.
  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
  • MonitorFeed Streams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.

import { mountConsole } from '@openav/console'; packages/console/index.js, line 39

@openav/monitor

defaultMonitorUrl()#

function@openav/monitor
defaultMonitorUrl(loc = globalThis.location)

Where the backstage relay is unless told otherwise: the page's own host on port 7457 as a ws:// address, and null on an https page or where there is no page.

It is the default url of MonitorFeed. Not on an https page: the browser blocks ws:// there (mixed content, logged as an error), and a page published over https (openaudiovisual.com, a tunnel) has no relay beside it. A show that runs one behind TLS passes { url: 'wss://…' } explicitly.

Parameters

NameTypeDescription
locObjectAnything with protocol and hostname, like location. Pass a fake one to test. optional default globalThis.location

Returns

String | null 'ws://<hostname>:7457', or null when loc is missing or its protocol is https:.

Examples

js
import { defaultMonitorUrl } from '@openav/monitor';

console.log(defaultMonitorUrl({ protocol: 'http:', hostname: '192.168.1.20' }));        // ws://192.168.1.20:7457
console.log(defaultMonitorUrl({ protocol: 'https:', hostname: 'openaudiovisual.com' })); // null
console.log(defaultMonitorUrl(null));                                                    // null: no page
module The console prints ws://192.168.1.20:7457, then null twice.

Related

  • MonitorFeed Streams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.
  • snapshotOf() Builds the usual snapshot of a show from its parts: clock, scene, world, frame rate, resolved params, overrides and every signal.

import { defaultMonitorUrl } from '@openav/monitor'; packages/monitor/index.js, line 17

MonitorFeed#

class@openav/monitor
new MonitorFeed({ url = defaultMonitorUrl(), hz = 15 } = {})

Streams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.

A show without a backstage is a rehearsal. Run the relay on the machine that serves the show (node packages/monitor/server.js, port 7457, which also serves the backstage page at / and GET /health), and a stage manager's phone or laptop on the same network opens http://<that machine>:7457/ to see the clock, the scene, the world, the frame rate, the overrides and every signal and param live, with a red "stage feed lost" banner when the stage stops sending for 4 seconds. The show page joins as ?role=stage; every other socket is a monitor. createShow() already creates and connects one, so you only build a MonitorFeed for a show you assembled by hand.

It is made to cost nothing when left on: with no relay it reconnects with a growing delay (about 5 seconds, growing by 1.6 each time up to 30), though the browser logs each refused connection, and that message is expected. frame() throttles itself, so call it every frame with a snapshot (see snapshotOf()).

Before you call it

Call connect() once (a show does), then frame(snapshot) every frame. frame() does nothing while connected is false. The relay must be reachable from the page: on an https page the browser blocks ws:// (mixed content), so give a wss:// address explicitly.

Parameters

NameTypeDescription
urlString | nullThe relay address, such as 'ws://192.168.1.20:7457' (?role=stage is added for you). The default comes from defaultMonitorUrl(): the page's own host on port 7457, and null on an https page. With null, connect() does nothing. optional default defaultMonitorUrl()
hzNumberThe most snapshots sent per second. optional default 15

Members

NameDescription
connect()methodOpen the socket. Does nothing without a url. If the connection drops or cannot be made, it tries again after a growing delay. The onStatus(ok) hook is called when it connects and when it drops.
frame(snapshot)methodSend a snapshot (anything that survives JSON.stringify; snapshotOf() builds the usual one). The message is { at: Date.now(), ...snapshot }. Throttled to hz: calls closer together than that are dropped, and nothing is sent while disconnected.

Notes

  • Fields to read and set: url (with ?role=stage appended, or null), hz, connected, ws, and the hook onStatus = (ok) => {}.
  • On a public https page a hand-assembled show that connects to ws://127.0.0.1:7457 meets Chrome's Local Network Access prompt (from Chrome 147 for WebSockets, per the package README): press Allow, or open the page from http://localhost.

Examples

js
import { MonitorFeed } from '@openav/monitor';

class FakeSocket {                                   // stands in for the relay, so this runs anywhere
  constructor(url) { console.log('connecting', url); setTimeout(() => this.onopen(), 0); }
  send(text) { console.log('sent', Object.keys(JSON.parse(text)).join(' ')); }
}
globalThis.WebSocket = FakeSocket;

const monitor = new MonitorFeed({ url: 'ws://192.168.1.20:7457', hz: 15 });
monitor.onStatus = (ok) => console.log('connected', ok);
monitor.connect();
setTimeout(() => {
  for (let i = 0; i < 5; i++) monitor.frame({ t: i, scene: 'Dawn' });     // five calls in the same instant
}, 20);
module The console prints connecting ws://192.168.1.20:7457?role=stage, connected true, and then only one sent at t scene: the other four calls came inside one throttle interval and were dropped.
js
import { Loop } from '@openav/core';
import { MonitorFeed, snapshotOf } from '@openav/monitor';

// a hand-assembled show: timeline, params, signals, stage and loop are yours
const monitor = new MonitorFeed({ url: 'ws://192.168.1.20:7457' });
monitor.connect();
const loop = new Loop((dt) => {
  timeline.advance(dt);
  const state = stage.frame(dt, timeline.state());
  monitor.frame(snapshotOf({ timeline, params, signals, stage, loop }, state));   // throttled for you
});
loop.start();
static With the relay running (node packages/monitor/server.js), a phone on the same network that opens http://<this machine>:7457/ sees the clock, the scene, the frame rate and every signal and param move.

To be confirmed

The relay itself (packages/monitor/server.js) and the backstage page were not run here; the examples use a stand-in socket.

Related

  • snapshotOf() Builds the usual snapshot of a show from its parts: clock, scene, world, frame rate, resolved params, overrides and every signal.
  • defaultMonitorUrl() Where the backstage relay is unless told otherwise: the page's own host on port 7457 as a ws:// address, and null on an https page or where there is no page.
  • createShow() Builds a whole show around a world in one call: signals, params, stage, timeline, mapper, the modules you switch on, the console, the backstage feed and the frame loop.
  • mountConsole() Mounts the director's desk into an element: transport and scrubber, layers overview, inputs, mapping, param sliders, signal meters and a fullscreen performance mode.

import { MonitorFeed } from '@openav/monitor'; packages/monitor/index.js, line 22

snapshotOf()#

function@openav/monitor
snapshotOf({ timeline, params, signals, stage, loop }, state)

Builds the usual snapshot of a show from its parts: clock, scene, world, frame rate, resolved params, overrides and every signal.

The data that MonitorFeed.frame() sends and the backstage page draws. Each part is optional and read with a safe fallback, so a hand-assembled show can pass only what it has. The snapshot is plain JSON, so you can also log it, store it for a rehearsal recording or show it in your own page.

Parameters

NameTypeDescription
timelineTimelineThe Timeline: gives t, playing, holding, total, sceneIndex and scene. optional
paramsParamsThe Params: gives the list of keys that are overridden. optional
signalsSignalsThe Signals: every signal as { n, v, k } (name, value, kind). optional
stageStageThe Stage: gives the active world's name. optional
loopLoopThe Loop: gives the frame rate. optional
stateObjectThe resolved param values of this frame (what stage.frame() returns). {} when you pass nothing.

Returns

Object { t, playing, holding, total, sceneIndex, scene, world, fps, state, overrides, signals }. scene is the current scene's title (or its id); holding is true while the show waits at a hold; signals is [{ n, v, k }] with numbers rounded to 3 decimals. A part you leave out gives zeros, empty strings and empty lists.

Examples

js
import { Signals, Params } from '@openav/core';
import { Timeline } from '@openav/timeline';
import { snapshotOf } from '@openav/monitor';

const signals = new Signals();
const params = new Params([{ key: 'hue', min: 0, max: 360, def: 200 }]);
const timeline = new Timeline({ params, total: 60, scenes: [{ id: 'dawn', t: 0, title: 'Dawn' }, { id: 'noon', t: 30 }] });
signals.set('midi/cc/74', 0.4567);
params.override('hue', 90);
timeline.seek(35);

const snapshot = snapshotOf({ timeline, params, signals, stage: { activeName: 'garden' }, loop: { fps: 60 } }, params.resolve(timeline.state()));
console.log(snapshot);
module The console prints the snapshot: t: 35, scene: 'noon' (a scene without a title shows its id), world: 'garden', fps: 60, state: { hue: 90 }, overrides: [ 'hue' ] and signals: [ { n: 'midi/cc/74', v: 0.457, k: 'continuous' } ].

Related

  • MonitorFeed Streams snapshots of the running show to the backstage relay over a WebSocket, at most hz times a second, and quietly keeps trying when there is no relay.
  • Timeline Answers "what is every param at time t?" from keyframes, and runs the transport around it: play, seek, scenes, holds, loops, cues and recording.
  • Params The values a world listens to: a list of declared params, each resolved every frame from the timeline, unless a hand, a knob or a route has overridden it.
  • Signals The registry every input writes into and everything else reads from: named values that are either a level (continuous) or an event (pulse).
  • Stage Runs one world at a time: registers worlds and their params, switches between them, and each frame resolves the params and calls the world's update and render.
  • Loop The frame clock: calls your function every animation frame with the seconds since the last one, and keeps ticking when the tab is hidden.

import { snapshotOf } from '@openav/monitor'; packages/monitor/index.js, line 59

Digital fabrication (@openav/fab)#

Meshes, STL / GLB / OBJ export, checks. 15

@openav/fab

candidateUrls()#

function@openav/fab
candidateUrls(kind, opts = {})

The list of URLs loadKernel() tries for a kernel, in order: your urls, then window.__LAB__.vendor, then the unpkg default.

The order lets a page choose where its kernels come from without touching the code that uses them: pass urls for one page, set window.__LAB__.vendor for a whole host (the lab does, to serve the kernels itself), and the unpkg default is the last resort. Calling it yourself is useful to see where a page would look.

Parameters

NameTypeDescription
kind'smlib' | 'manifold'Which kernel.
optsObjecturls: { smlib, manifold }, each a URL or a list of URLs. optional default {}

Returns

string[] The URLs, first to try first, without repeats.

Examples

js
import { candidateUrls } from '@openav/fab';

window.__LAB__ = { vendor: { smlib: 'https://lab.example/vendor/smlib.js' } };   // what a host may set
const rows = [
  'mine first: ' + candidateUrls('smlib', { urls: { smlib: 'https://mine.example/smlib.js' } }).join('  →  '),
  'manifold: ' + candidateUrls('manifold').join('  →  '),
];
delete window.__LAB__;
document.getElementById('app').innerHTML = rows.join('<br>');
module Prints three smlib URLs in order (yours, the host's, unpkg) and only the unpkg URL for manifold, which nobody configured.

Related

  • loadKernel() Imports a CAD kernel (smlibwasm or manifold-3d) from the first URL that works, and resolves to the ready module.
  • DEFAULT_URLS Where the CAD kernels are loaded from when nothing else says so: smlibwasm 0.3.0 and manifold-3d 3.5.4, both from unpkg.
  • Fab Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.

import { candidateUrls } from '@openav/fab'; packages/fab/kernel.js, line 10

checkMesh()#

function@openav/fab
checkMesh(input, opts = {})

Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.

Pure and synchronous, with no DOM. Duplicate vertices are merged first (within 1e-6 of the bounding-box diagonal), because without that every un-indexed mesh (three.js toNonIndexed(), an STL, any triangle soup) reads as open. In the report: triangles (after merging; a face that collapses is dropped), bbox as { min, max, size } in mm, volume in mm³ (always positive; a mesh wound inside out is measured the right way round), watertight (every edge used by exactly two faces, in opposite directions) with the counts openEdges, nonManifoldEdges and inconsistentEdges behind it, and grams: { PETG, PLA }, the weight of the volume as solid plastic (1.27 and 1.24 g/cm³).

overhang is { angle, area, ratio }: the area (mm²) of faces that look down steeper than angle allows, and its share of the whole surface. Faces that rest on the lowest 0.2 mm (the bed) do not count. minWall is { value, min, median, samples } or null: rays are shot into the part from sampled faces, and value is the 5th percentile of the thicknesses found, which is how thin most of the wall is. min is the single thinnest sample, which on a part with holes drilled at an angle is a knife edge and says little about the wall, so it has its own milder note.

warnings is a list of one sentence each, in Chinese: not watertight (with the three counts), overhang above overhangWarn, a wall thinner than minWall, a knife edge thinner than half of it, and a size beyond bed. An empty list means nothing to report. fab.check(x) is checkMesh(meshOf(x), opts).

Parameters

NameTypeDescription
inputTriMeshA mesh { vertices, triangles, units? }: vertices a Float32Array or Float64Array of x, y, z, x, y, z …, triangles a Uint32Array with three vertex numbers per face, counter-clockwise seen from outside, Z up, in millimetres. Use meshOf() to get one from something else.
optsObjectangle (45): the overhang angle in degrees. minWall (1.2): the thinnest wall allowed, in mm. bed ([320, 320, 325]): the size of the print bed in mm. wallSamples (2000): how many faces to measure the wall from; fewer is faster. overhangWarn (0.15): the share of overhanging area that earns a warning. weld (true): merge duplicate vertices first; false skips it. optional default {}

Returns

Object { units, triangles, bbox, volume, watertight, openEdges, nonManifoldEdges, inconsistentEdges, overhang, minWall, grams, warnings }. See the description.

Notes

  • grams is the weight of a solid part: it does not know about infill or wall count, so a sliced print weighs less.

Examples

js
import { checkMesh } from '@openav/fab';

const tetra = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
  units: 'mm',
};
const r = checkMesh(tetra);
document.getElementById('app').innerHTML = [
  'closed: ' + r.watertight + ' (open edges ' + r.openEdges + ')',
  'size: ' + r.bbox.size.join(' x ') + ' mm',
  'volume: ' + r.volume.toFixed(2) + ' mm³, PLA ' + r.grams.PLA + ' g',
  'thinnest wall (5th percentile): ' + r.minWall.value.toFixed(2) + ' mm',
].join('<br>');
module Prints closed: true, size 10 x 10 x 10 mm, volume 166.67 mm³, PLA 0.207 g and a wall of 3.33 mm.
js
import { checkMesh } from '@openav/fab';

// the same tetrahedron with its bottom face left out: a hole in the mesh
const open = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 1, 3, 0, 3, 2, 1, 2, 3]),
  units: 'mm',
};
const r = checkMesh(open, { bed: [5, 5, 5] });   // and a printer bed that is too small
document.getElementById('app').innerHTML = [
  'watertight: ' + r.watertight + ', open edges: ' + r.openEdges,
  'warnings: ' + r.warnings.length,
].join('<br>');
module Prints watertight: false with 3 open edges, and 2 warnings: the open mesh and the 10 mm part that does not fit a 5 mm bed.
js
import { checkMesh } from '@openav/fab';

// the same tetrahedron, but with every triangle listed again for each corner: 12 vertices, none shared (a "triangle soup")
const tetra = [0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10];
const faces = [[0, 2, 1], [0, 1, 3], [0, 3, 2], [1, 2, 3]];
const soup = {
  vertices: Float64Array.from(faces.flatMap((f) => f.flatMap((i) => tetra.slice(i * 3, i * 3 + 3)))),
  triangles: Uint32Array.from({ length: 12 }, (_, i) => i),
  units: 'mm',
};
document.getElementById('app').innerHTML = [
  'merged first (default): ' + checkMesh(soup).watertight,
  'weld: false: ' + checkMesh(soup, { weld: false }).watertight,
].join('<br>');
module Prints true with the default merge and false with weld: false: a soup of loose triangles looks open until its vertices are merged.

Related

  • Fab Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.
  • weldMesh() Merges vertices that sit closer together than a tolerance and drops the faces that collapse; returns a new mesh with shared vertices.
  • meshOf() Turns whatever you made into a triangle mesh { vertices, triangles, units }: a mesh, a three.js object, a mesh-like object or a smlibwasm shape.
  • toStlBytes() Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.

import { checkMesh } from '@openav/fab'; packages/fab/checks.js, line 236

DEFAULT_URLS#

constant@openav/fab
DEFAULT_URLS = object, 2 keys

Where the CAD kernels are loaded from when nothing else says so: smlibwasm 0.3.0 and manifold-3d 3.5.4, both from unpkg.

The last entry of candidateUrls(). Versions are pinned, so the kernels a page gets do not change under it. To use another copy, pass urls to Fab or loadKernel() rather than changing this object.

Type
Object

{ smlib: 'https://unpkg.com/@classcad/smlibwasm@0.3.0/dist/index.js', manifold: 'https://unpkg.com/manifold-3d@3.5.4/manifold.js' }.

Examples

js
import { DEFAULT_URLS } from '@openav/fab';

document.getElementById('app').innerHTML = Object.entries(DEFAULT_URLS).map(([kind, url]) => kind + ': ' + url).join('<br>');
module Prints the two unpkg addresses, with the versions in them.

Related

  • candidateUrls() The list of URLs loadKernel() tries for a kernel, in order: your urls, then window.__LAB__.vendor, then the unpkg default.
  • loadKernel() Imports a CAD kernel (smlibwasm or manifold-3d) from the first URL that works, and resolves to the ready module.
  • Fab Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.

import { DEFAULT_URLS } from '@openav/fab'; packages/fab/kernel.js, line 5

exportAs()#

function@openav/fab
async exportAs(x, fmt, opts = {}, kernel = null)

Exports a shape as STL, OBJ, GLB, STEP or DXF and resolves to { bytes, mime, ext, report }: the same as fab.export(), without the Fab class.

stl, obj and glb come from the mesh: toStlBytes(), toObjText(), toGlbBytes(). step and dxf come straight from the smlibwasm kernel (toStep, toDxf) and are refused for anything that is not a kernel shape, because a mesh has no curved surfaces to write; an error says so. A kernel solid's glb uses the kernel's toGlb when it has one. The report is made on the way, so a file and its check always agree.

Parameters

NameTypeDescription
xObjectThe shape: anything meshOf() accepts, and for step a smlibwasm Part, for dxf a smlibwasm Shape or Sketch.
fmtString'stl', 'obj', 'step', 'dxf' or 'glb', in any case. 'stp' means step and 'gltf' means glb. Another word throws an Error that lists the formats.
optsObjectThe options of meshOf() (scale, tessellator) and of checkMesh() (angle, minWall, bed …), and name, the object name of an OBJ file. optional default {}
kernel{ smlib }{ smlib: () => Promise<module> }, where to get the smlibwasm module for step, dxf, and the glb of a Part. fab.export() passes its own. optional default null

Returns

Promise<Object> bytes is a Uint8Array, mime is from MIME, ext is the extension without the dot ('step' even if you asked for 'stp'), and report is the checkMesh() report of the mesh (for step and dxf it can be null when the shape cannot be meshed, such as a 2D sketch).

Examples

js
import { exportAs } from '@openav/fab';

const tetra = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
};
const rows = [];
for (const fmt of ['STL', 'obj', 'gltf']) {
  const out = await exportAs(tetra, fmt);
  rows.push(fmt + ' → ' + out.ext + ', ' + out.mime + ', ' + out.bytes.length + ' bytes, watertight ' + out.report.watertight);
}
document.getElementById('app').innerHTML = rows.join('<br>');
module Prints three exports: STL as stl (284 bytes), obj, and the alias gltf as glb, each with the report's verdict.
js
import { exportAs } from '@openav/fab';

const tetra = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
};
let step, xyz;
try { await exportAs(tetra, 'step'); } catch (e) { step = e.message; }
try { await exportAs(tetra, 'xyz'); } catch (e) { xyz = e.message; }
document.getElementById('app').innerHTML = [step, xyz].join('<br>');
module Two refusals, each with a message in Chinese: a mesh cannot become STEP (it has no curved surfaces), and "xyz" is not a format.
js
import { exportAs } from '@openav/fab';

// a stand-in for the smlibwasm kernel: STEP only works for a kernel solid, and the kernel writes it
const solid = { faces: () => [], bbox: () => ({}), tessellate: () => ({ positions: [0, 0, 0, 10, 0, 0, 0, 10, 0], indices: [0, 1, 2] }) };
const kernel = { smlib: async () => ({ toStep: () => 'ISO-10303-21;' }) };
const out = await exportAs(solid, 'stp', {}, kernel);
document.getElementById('app').textContent = out.ext + ' (' + out.mime + '): ' + new TextDecoder().decode(out.bytes);
module Prints "step (model/step): ISO-10303-21;": the alias stp was understood and the bytes came from the kernel's toStep.

Related

  • Fab Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.
  • MIME The media type of each export format: MIME.stl is 'model/stl', and obj, step, dxf and glb the same way.
  • toStlBytes() Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.
  • toObjText() Writes a triangle mesh as OBJ text: welded vertices (v x y z), then faces (f a b c), one object, in millimetres.
  • toGlbBytes() Writes a triangle mesh as a minimal binary glTF (.glb): one mesh with welded positions and indices, in millimetres.
  • checkMesh() Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
  • loadKernel() Imports a CAD kernel (smlibwasm or manifold-3d) from the first URL that works, and resolves to the ready module.

import { exportAs } from '@openav/fab'; packages/fab/export.js, line 95

Fab#

class@openav/fab
new Fab(opts = {})

Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.

Make one Fab and hand it whatever you made: fab.check(x) returns a manufacturability report (is it closed, how much material, how much overhang, how thin is the thinnest wall, how many grams), and await fab.export(x, 'stl') returns the file as bytes. x can be a plain triangle mesh { vertices, triangles }, a three.js Mesh, Object3D or BufferGeometry (recognised by its shape, three.js is never imported), or a smlibwasm Part or Shape. meshOf() says exactly what is accepted.

Formats: stl, obj and glb work for any input; step only for a smlibwasm Part, and dxf only for a smlibwasm Shape or Sketch, because a mesh has no curved-surface information. The CAD kernels are not bundled: the first time one is needed, fab.smlib() or fab.manifold() imports it from a URL (unpkg by default) and remembers it.

fab.save() writes the file through fab.saver if you set one (the lab's dev server does), and otherwise as a browser download. Zero runtime dependencies: nothing is fetched until you ask for a kernel.

Before you call it

Nothing for meshes and three.js objects. A smlibwasm Part has to be tessellated by the kernel: call await fab.smlib() once before the synchronous check(), or use checkAsync(), meshOfAsync() and export(), which load it for you. Loading needs a network, or urls / window.__LAB__.vendor pointing somewhere that has the kernel.

Parameters

NameTypeDescription
optsObject{ scale, urls, importer, download, context }, all optional. scale is millimetres per unit for a three.js object (10 means one three.js unit is 10 mm). urls is { smlib, manifold }, a URL or a list of them, to load the CAD kernels from. importer(url) replaces the dynamic import() that loads a kernel. download(bytes, info) replaces the browser download that save() falls back to. context() returns { id, params } for save() to put in info. optional default {}

Members

NameDescription
smlib()methodsmlib(): a promise of the smlibwasm module, imported on the first call and remembered (the same promise comes back). A failed load is forgotten, so the next call tries again.
manifold()methodmanifold(): the same for manifold-3d. The WebAssembly is initialised for you (setup() is called), and the result is the ready-to-use module.
meshOf(x, opts = {})methodmeshOf(x, opts?): the same as meshOf(), with this Fab's scale and, once smlib() has loaded, the kernel's tessellator.
meshOfAsync(x, opts = {})methodmeshOfAsync(x, opts?): the same as meshOfAsync(), and it loads the smlib kernel when it needs it.
check(x, opts = {})methodcheck(x, opts?): synchronous. meshOf(x) then checkMesh(): returns the report. opts are the options of both (scale, angle, minWall, bed …).
checkAsync(x, opts = {})methodcheckAsync(x, opts?): the same through meshOfAsync, so a smlibwasm Part works without loading the kernel first. Returns a promise of the report.
export(x, fmt, opts = {})methodexport(x, fmt, opts?): a promise of { bytes, mime, ext, report }; see exportAs(). It uses this Fab's scale and loads the kernel for step, dxf and the glb of a Part.
save(x, fmt, { name, params, note, ...rest } = {})methodsave(x, fmt, { name, params, note, … }?): export, then hand the bytes to fab.saver(bytes, info) if it is a function; info is { id, fmt, ext, mime, name, params, note, report }. If the saver is missing, throws, or returns nothing, the file is downloaded instead, through opts.download or an <a download> click. Returns what the saver returned, or { downloaded: true, file }, plus report.

Notes

  • Public fields: opts and saver (set it to async (bytes, info) => ({ file, … }); it starts as null). The file name for save() is name, else the id from context(), else fab.

Examples

js
import { Fab } from '@openav/fab';

// a tetrahedron: three 10 mm edges meeting at the origin (x, y, z), counter-clockwise faces seen from outside
const mesh = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
  units: 'mm',
};
const report = new Fab().check(mesh);
document.getElementById('app').innerHTML = [
  'watertight: ' + report.watertight,
  'volume: ' + report.volume.toFixed(1) + ' mm³',
  'PETG: ' + report.grams.PETG + ' g',
  'overhang: ' + Math.round(report.overhang.ratio * 100) + '%',
  'warnings: ' + report.warnings.length,
].join('<br>');
module Prints watertight: true, volume 166.7 mm³ (10 × 10 × 10 ÷ 6), 0.212 g of PETG, no overhang and no warnings.
js
import { Fab } from '@openav/fab';

const mesh = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
  units: 'mm',
};
const fab = new Fab();
const stl = await fab.export(mesh, 'stl');
const obj = await fab.export(mesh, 'obj', { name: 'tetra' });
document.getElementById('app').innerHTML = [
  stl.ext + ' (' + stl.mime + '): ' + stl.bytes.length + ' bytes',
  obj.ext + ' (' + obj.mime + '): ' + new TextDecoder().decode(obj.bytes).split('\n').slice(0, 4).join(' | '),
].join('<br>');
module Prints the STL as 284 bytes (84 + 50 × 4) and the first lines of the OBJ: a comment, "o tetra" and the first two vertices.
js
import { Fab } from '@openav/fab';

const mesh = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
  units: 'mm',
};
const seen = [];
const fab = new Fab({ download: (bytes, info) => seen.push('download ' + info.name + '.' + info.ext + ' (' + bytes.length + ' bytes)') });

const first = await fab.save(mesh, 'stl', { name: 'tetra' });          // no saver: a download
fab.saver = async (bytes, info) => ({ file: info.name + '.' + info.ext, n: 1 });   // e.g. the lab's /api/fab
const second = await fab.save(mesh, 'obj', { name: 'tetra' });         // the saver wins
seen.push('first: ' + JSON.stringify({ downloaded: first.downloaded, file: first.file }));
seen.push('second: ' + JSON.stringify({ file: second.file, n: second.n }));
document.getElementById('app').innerHTML = seen.join('<br>');
module Prints the download of tetra.stl (the stand-in download replaces the browser's), then the saver's answer for tetra.obj.

To be confirmed

The STEP and DXF export, and the tessellation of a real smlibwasm Part, were not run for this entry: they need the real kernel from unpkg. The examples use a stand-in kernel to show the routing.

Related

  • checkMesh() Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
  • exportAs() Exports a shape as STL, OBJ, GLB, STEP or DXF and resolves to { bytes, mime, ext, report }: the same as fab.export(), without the Fab class.
  • meshOf() Turns whatever you made into a triangle mesh { vertices, triangles, units }: a mesh, a three.js object, a mesh-like object or a smlibwasm shape.
  • loadKernel() Imports a CAD kernel (smlibwasm or manifold-3d) from the first URL that works, and resolves to the ready module.
  • toStlBytes() Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.

import { Fab } from '@openav/fab'; packages/fab/index.js, line 16

fromMeshLike()#

function@openav/fab
fromMeshLike(m, scale = 1)

Turns a mesh-like object (positions and indices under any of the usual names) into a triangle mesh, multiplying the positions by scale.

This is what meshOf() uses for a kernel's tessellation result, whichever way the kernel spells it. It throws an Error (in Chinese) when m is empty or has no positions. It does not check that the faces point outward or that the mesh is closed: that is checkMesh().

Parameters

NameTypeDescription
mObjectAn object with the points under positions, vertices, points or position, and the faces under indices, triangles, faces, index or triVerts. Points may be flat numbers, nested arrays ([[x, y, z], …]) or objects ({ x, y, z }). Without faces, every three points are one triangle. m.units is kept (default 'mm').
scaleNumberMultiplies every position, for example millimetres per unit. optional default 1

Returns

TriMesh { vertices: Float64Array, triangles: Uint32Array, units }.

Examples

js
import { fromMeshLike } from '@openav/fab';

// no faces given: every three points are one triangle; positions in centimetres, scaled to millimetres
const mesh = fromMeshLike({ positions: [0, 0, 0, 1, 0, 0, 0, 1, 0] }, 10);
document.getElementById('app').textContent =
  'vertices: ' + [...mesh.vertices].join(', ') + ' | triangles: ' + [...mesh.triangles].join(', ') + ' | ' + mesh.units;
module Prints "vertices: 0, 0, 0, 10, 0, 0, 0, 10, 0 | triangles: 0, 1, 2 | mm".

Related

  • meshOf() Turns whatever you made into a triangle mesh { vertices, triangles, units }: a mesh, a three.js object, a mesh-like object or a smlibwasm shape.
  • meshOfAsync() Like meshOf(), but it waits for a tessellate() that returns a promise, and can load the smlibwasm kernel to tessellate a solid.
  • checkMesh() Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.

import { fromMeshLike } from '@openav/fab'; packages/fab/mesh.js, line 22

fromStlBytes()#

function@openav/fab
fromStlBytes(bytes)

Reads a binary STL file back into a triangle mesh: every face has its own three vertices, in 32-bit floats.

It reads what toStlBytes() writes, or any binary STL: the normals and the two attribute bytes are skipped, and the result is a triangle soup. Give it to checkMesh(), which merges the vertices itself, or to weldMesh(). The text (ASCII) form of STL, which starts with solid, is not supported and would read as garbage.

Parameters

NameTypeDescription
bytesUint8ArrayThe contents of a binary STL file. The face count is read from the header; the file's length is not checked against it.

Returns

TriMesh { vertices: Float32Array (9 per face), triangles: Uint32Array (0, 1, 2, 3, 4, 5 …), units: 'mm' }.

Examples

js
import { toStlBytes, fromStlBytes, checkMesh } from '@openav/fab';

const tetra = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
};
const back = fromStlBytes(toStlBytes(tetra));   // a round trip through the file format
document.getElementById('app').innerHTML = [
  'faces: ' + back.triangles.length / 3 + ', vertices: ' + back.vertices.length / 3,
  'still closed after the check merges them: ' + checkMesh(back).watertight,
].join('<br>');
module Prints 4 faces and 12 vertices (no sharing), and "true": the check welds them and finds the shape closed.

Related

  • toStlBytes() Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.
  • weldMesh() Merges vertices that sit closer together than a tolerance and drops the faces that collapse; returns a new mesh with shared vertices.
  • checkMesh() Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.

import { fromStlBytes } from '@openav/fab'; packages/fab/export.js, line 33

loadKernel()#

function@openav/fab
async loadKernel(kind, opts = {})

Imports a CAD kernel (smlibwasm or manifold-3d) from the first URL that works, and resolves to the ready module.

The kernels are not bundled, because open-audiovisual has no dependencies and no build step: the first time a page needs one, it is imported from a URL. The order is candidateUrls(): opts.urls[kind], then window.__LAB__.vendor[kind], then unpkg (DEFAULT_URLS). If every URL fails it rejects with an Error (in Chinese) that names each URL and why it failed. It does not remember anything: fab.smlib() and fab.manifold() keep the promise for you, so call those unless you want to manage it yourself.

Before you call it

A network, or urls pointing at a place that serves the kernel (the lab uses window.__LAB__.vendor), or an importer that returns one.

Parameters

NameTypeDescription
kind'smlib' | 'manifold'Which kernel: 'smlib' for smlibwasm (solid modelling, STEP and DXF) or 'manifold' for manifold-3d (fast mesh booleans).
optsObjecturls: { smlib, manifold }, a URL or a list of URLs for each kernel. importer: async (url) => module, to use instead of the dynamic import(). optional default {}

Returns

Promise<Object> The smlibwasm module as it came, or the manifold-3d module with its WebAssembly started (setup() called).

Examples

js
import { loadKernel } from '@openav/fab';

const tried = [];
// a stand-in importer so nothing is fetched: the first URL fails, the second one works
const importer = async (url) => {
  tried.push(url);
  if (url.includes('mirror-down')) throw new Error('503');
  return { name: 'manifold stand-in', default: async () => ({ setup() { tried.push('setup() called'); } }) };
};
const wasm = await loadKernel('manifold', { urls: { manifold: ['https://mirror-down.example/m.js', 'https://mirror.example/m.js'] }, importer });
document.getElementById('app').innerHTML = tried.join('<br>') + '<br>result has setup: ' + (typeof wasm.setup);
module Prints the two URLs in the order they were tried, then "setup() called": manifold's WebAssembly is started for you.
js
import { loadKernel } from '@openav/fab';

let message;
try {
  await loadKernel('smlib', { urls: { smlib: 'https://nowhere.example/a.js' }, importer: async () => { throw new Error('no network'); } });
} catch (e) { message = e.message; }
document.getElementById('app').textContent = message;
module When every URL fails the error names each one, yours first and then the unpkg default, with the reason (in a Chinese sentence).

Related

  • candidateUrls() The list of URLs loadKernel() tries for a kernel, in order: your urls, then window.__LAB__.vendor, then the unpkg default.
  • DEFAULT_URLS Where the CAD kernels are loaded from when nothing else says so: smlibwasm 0.3.0 and manifold-3d 3.5.4, both from unpkg.
  • Fab Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.
  • meshOfAsync() Like meshOf(), but it waits for a tessellate() that returns a promise, and can load the smlibwasm kernel to tessellate a solid.

import { loadKernel } from '@openav/fab'; packages/fab/kernel.js, line 17

meshOf()#

function@openav/fab
meshOf(x, opts = {})

Turns whatever you made into a triangle mesh { vertices, triangles, units }: a mesh, a three.js object, a mesh-like object or a smlibwasm shape.

Everything is recognised by its shape and nothing is imported, so three.js and smlibwasm are never loaded for you. A three.js object has its matrixWorld applied (a mirrored object keeps its faces facing outward), indexed or not, and Y up is turned into Z up as (x, y, z) → (x, −z, y), then everything is multiplied by scale. A missing input, or something it cannot read, throws an Error whose message (in Chinese) says what to do.

It is synchronous, so it cannot wait for a kernel: a smlibwasm solid without opts.tessellator throws a message that tells you to load the kernel first, and a tessellate() that returns a promise throws too. Use meshOfAsync() for those.

Parameters

NameTypeDescription
xObjectThe thing to convert. Accepted, tried in this order: a mesh { vertices, triangles } (returned as it is); a three.js Mesh, Object3D (every visible mesh below it) or BufferGeometry; anything with a tessellate() method; an object with positions / vertices / points and, optionally, indices / triangles / faces / index; a smlibwasm solid or shape (given opts.tessellator).
optsObjectscale: millimetres per unit for a three.js or mesh-like input (default 1). A mesh { vertices, triangles } is never scaled. tessellator(shape): a function that turns a smlibwasm shape into a mesh (the Fab class passes the kernel's once it is loaded). tessellate: an argument passed on to the shape's own tessellate(). optional default {}

Returns

TriMesh { vertices, triangles, units }: counter-clockwise faces seen from outside, Z up, millimetres.

Examples

js
import { meshOf } from '@openav/fab';

// what a three.js Mesh looks like to fab: a triangle in the XY plane, no index, an identity matrixWorld
const threeLike = {
  isMesh: true,
  geometry: { attributes: { position: { array: new Float32Array([0, 0, 0, 1, 0, 0, 0, 1, 0]), itemSize: 3, count: 3 } }, index: null },
  matrixWorld: { elements: [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1] },
};
const m = meshOf(threeLike, { scale: 10 });   // one three.js unit is 10 mm
document.getElementById('app').textContent = 'vertices: ' + [...m.vertices].join(', ') + ' (' + m.units + ')';
module Prints "vertices: 0, 0, 0, 10, 0, 0, 0, 0, 10 (mm)": scaled by 10, and three.js's up (y = 1) became fab's up (z = 10).
js
import { meshOf } from '@openav/fab';

// a mesh-like object: nested points and faces, as some libraries hand them out
const m = meshOf({ points: [[0, 0, 0], [20, 0, 0], [0, 20, 0]], faces: [[0, 1, 2]] });
// the vertices come back flat, as numbers: x, y, z, x, y, z …
document.getElementById('app').textContent = [...m.vertices].join(' ') + ' | faces: ' + [...m.triangles].join(' ');
module Prints the nine numbers of the three corners and the face "0 1 2": nested arrays are flattened.

Related

  • meshOfAsync() Like meshOf(), but it waits for a tessellate() that returns a promise, and can load the smlibwasm kernel to tessellate a solid.
  • fromMeshLike() Turns a mesh-like object (positions and indices under any of the usual names) into a triangle mesh, multiplying the positions by scale.
  • checkMesh() Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
  • Fab Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.

import { meshOf } from '@openav/fab'; packages/fab/mesh.js, line 83

meshOfAsync()#

function@openav/fab
async meshOfAsync(x, opts = {})

Like meshOf(), but it waits for a tessellate() that returns a promise, and can load the smlibwasm kernel to tessellate a solid.

Use it where the conversion may have to wait: a shape whose tessellate() is asynchronous, or a smlibwasm Part when the kernel is not loaded yet. With opts.kernel.smlib it awaits the module and uses its tessellate(part); an error says so if the module has none. Everything else is passed on to meshOf(). fab.meshOfAsync() fills in kernel for you.

Parameters

NameTypeDescription
xObjectThe thing to convert; see meshOf().
optsObjectThe options of meshOf(), and kernel: { smlib: () => Promise<module> }, where to get the smlibwasm module from when a solid has to be tessellated and no tessellator was given. optional default {}

Returns

Promise<TriMesh> The same mesh meshOf() would give.

Examples

js
import { meshOfAsync } from '@openav/fab';

// a shape whose tessellate() answers later, the way a kernel running in a worker would
const shape = { tessellate: async () => ({ positions: [0, 0, 0, 5, 0, 0, 0, 5, 0], indices: [0, 1, 2] }) };
const mesh = await meshOfAsync(shape);
document.getElementById('app').textContent = mesh.vertices.length / 3 + ' vertices, ' + mesh.triangles.length / 3 + ' face';
module Prints "3 vertices, 1 face": the promise was awaited and the result turned into a mesh.
js
import { meshOfAsync } from '@openav/fab';

// a stand-in for the smlibwasm module: a solid has no tessellate() of its own, the module's function cuts it
const part = { faces: () => [], bbox: () => ({}) };
const kernel = { smlib: async () => ({ tessellate: (p) => ({ vertices: [0, 0, 0, 5, 0, 0, 0, 5, 0], triangles: [0, 1, 2] }) }) };
const mesh = await meshOfAsync(part, { kernel });
document.getElementById('app').textContent = 'tessellated by the kernel: ' + mesh.triangles.length / 3 + ' face';
module Prints "tessellated by the kernel: 1 face": the module was fetched through kernel.smlib() and its tessellate did the work.

Related

  • meshOf() Turns whatever you made into a triangle mesh { vertices, triangles, units }: a mesh, a three.js object, a mesh-like object or a smlibwasm shape.
  • Fab Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.
  • loadKernel() Imports a CAD kernel (smlibwasm or manifold-3d) from the first URL that works, and resolves to the ready module.

import { meshOfAsync } from '@openav/fab'; packages/fab/mesh.js, line 102

MIME#

constant@openav/fab
MIME = object, 5 keys

The media type of each export format: MIME.stl is 'model/stl', and obj, step, dxf and glb the same way.

For the type of a Blob or the Content-Type of an upload when you handle the bytes yourself: new Blob([bytes], { type: MIME.stl }). exportAs() already puts the right one in its result.

Type
Object

{ stl: 'model/stl', obj: 'model/obj', step: 'model/step', dxf: 'image/vnd.dxf', glb: 'model/gltf-binary' }. Its keys are the formats exportAs() accepts.

Examples

js
import { MIME, toStlBytes } from '@openav/fab';

const tetra = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
};
const blob = new Blob([toStlBytes(tetra)], { type: MIME.stl });
document.getElementById('app').textContent = Object.keys(MIME).join(', ') + ' | blob: ' + blob.type + ', ' + blob.size + ' bytes';
module Prints the five formats and a blob of type model/stl, 284 bytes.

Related

  • exportAs() Exports a shape as STL, OBJ, GLB, STEP or DXF and resolves to { bytes, mime, ext, report }: the same as fab.export(), without the Fab class.
  • toStlBytes() Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.
  • Fab Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.

import { MIME } from '@openav/fab'; packages/fab/export.js, line 5

toGlbBytes()#

function@openav/fab
toGlbBytes(mesh)

Writes a triangle mesh as a minimal binary glTF (.glb): one mesh with welded positions and indices, in millimetres.

Vertices are welded first with weldMesh(). The file has POSITION and indices only: no normals, no materials, no colours, so a viewer shades it flat or works the normals out itself. The positions stay in millimetres and Z up, and asset.extras says { units: 'mm', up: 'Z' }. glTF itself assumes Y up, so a viewer that does not read the extras shows the part lying on its side.

await fab.export(part, 'glb') of a smlibwasm Part uses the kernel's own toGlb when it has one, and this function otherwise.

Parameters

NameTypeDescription
meshTriMeshA mesh { vertices, triangles }, in millimetres.

Returns

Uint8Array A glTF 2.0 binary: the glTF header, one JSON chunk, one binary chunk with 32-bit float positions and 32-bit indices. The length is a multiple of 4.

Examples

js
import { toGlbBytes } from '@openav/fab';

const tetra = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
};
const glb = toGlbBytes(tetra);
const dv = new DataView(glb.buffer);
const json = JSON.parse(new TextDecoder().decode(glb.subarray(20, 20 + dv.getUint32(12, true))));
document.getElementById('app').innerHTML = [
  'magic: ' + new TextDecoder().decode(glb.subarray(0, 4)) + ', ' + glb.length + ' bytes',
  'vertices: ' + json.accessors[0].count + ', indices: ' + json.accessors[1].count,
  'bounds: ' + json.accessors[0].max.join(' x '),
].join('<br>');
module Prints the magic "glTF" and the size, 4 vertices with 12 indices, and the bounds 10 x 10 x 10.

Related

  • toStlBytes() Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.
  • toObjText() Writes a triangle mesh as OBJ text: welded vertices (v x y z), then faces (f a b c), one object, in millimetres.
  • exportAs() Exports a shape as STL, OBJ, GLB, STEP or DXF and resolves to { bytes, mime, ext, report }: the same as fab.export(), without the Fab class.
  • weldMesh() Merges vertices that sit closer together than a tolerance and drops the faces that collapse; returns a new mesh with shared vertices.

import { toGlbBytes } from '@openav/fab'; packages/fab/export.js, line 54

toObjText()#

function@openav/fab
toObjText(mesh, name = 'fab')

Writes a triangle mesh as OBJ text: welded vertices (v x y z), then faces (f a b c), one object, in millimetres.

Vertices are welded first with weldMesh(), so a triangle soup comes out with shared vertices and a small file. There are no normals, no texture coordinates and no materials. await fab.export(x, 'obj', { name }) is the same thing with the conversions done for you.

Parameters

NameTypeDescription
meshTriMeshA mesh { vertices, triangles }, in millimetres.
nameStringThe object name on the o line. Spaces become underscores. optional default 'fab'

Returns

String The text: a comment line, o <name>, one v line per welded vertex (6 decimals at most) and one f line per face, vertex numbers counting from 1.

Examples

js
import { toObjText } from '@openav/fab';

const tetra = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
};
document.getElementById('app').innerHTML = toObjText(tetra, 'my part').trim().split('\n').join('<br>');
module Prints the comment, "o my_part", four v lines and four f lines: "f 1 3 2" is the first face.

Related

  • toStlBytes() Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.
  • toGlbBytes() Writes a triangle mesh as a minimal binary glTF (.glb): one mesh with welded positions and indices, in millimetres.
  • exportAs() Exports a shape as STL, OBJ, GLB, STEP or DXF and resolves to { bytes, mime, ext, report }: the same as fab.export(), without the Fab class.
  • weldMesh() Merges vertices that sit closer together than a tolerance and drops the faces that collapse; returns a new mesh with shared vertices.

import { toObjText } from '@openav/fab'; packages/fab/export.js, line 45

toStlBytes()#

function@openav/fab
toStlBytes(mesh)

Writes a triangle mesh as a binary STL file: 84 + 50 bytes per face, with a normal worked out for every face.

Every face becomes its own 50 bytes: STL has no shared vertices. The normal comes from the order of the corners (right-hand rule), so counter-clockwise faces seen from outside give outward normals. Positions are written as they are, as float32: nothing is converted to millimetres, so give it a mesh that is already in mm. await fab.export(x, 'stl') is the same thing with the conversions and the check done for you.

Parameters

NameTypeDescription
meshTriMeshA mesh { vertices, triangles }, in millimetres.

Returns

Uint8Array The file: an 80-byte header (the text open-audiovisual fab STL (mm)), the face count as a little-endian 32-bit integer, then per face the normal and three corners as 32-bit floats and two zero bytes.

Examples

js
import { toStlBytes } from '@openav/fab';

const tetra = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
};
const bytes = toStlBytes(tetra);
const faces = new DataView(bytes.buffer).getUint32(80, true);
document.getElementById('app').textContent =
  bytes.length + ' bytes = 84 + 50 × ' + faces + ' faces, header "' + new TextDecoder().decode(bytes.subarray(0, 29)) + '"';
module Prints 284 bytes = 84 + 50 × 4 faces, and the header text.
js
import { toStlBytes } from '@openav/fab';

const tetra = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
};
const bytes = toStlBytes(tetra);
const link = document.createElement('a');
link.href = URL.createObjectURL(new Blob([bytes], { type: 'model/stl' }));
link.download = 'tetra.stl';
link.textContent = 'download tetra.stl (' + bytes.length + ' bytes)';
document.getElementById('app').append(link);
module A link that downloads the file: open it in a slicer or a 3D viewer to see a small pyramid with a right-angle corner.

Related

  • fromStlBytes() Reads a binary STL file back into a triangle mesh: every face has its own three vertices, in 32-bit floats.
  • exportAs() Exports a shape as STL, OBJ, GLB, STEP or DXF and resolves to { bytes, mime, ext, report }: the same as fab.export(), without the Fab class.
  • toObjText() Writes a triangle mesh as OBJ text: welded vertices (v x y z), then faces (f a b c), one object, in millimetres.
  • toGlbBytes() Writes a triangle mesh as a minimal binary glTF (.glb): one mesh with welded positions and indices, in millimetres.
  • Fab Checks a shape for 3D printing and exports it as STL, OBJ, GLB, STEP or DXF; takes a mesh, a three.js object or a smlibwasm solid.

import { toStlBytes } from '@openav/fab'; packages/fab/export.js, line 9

weldMesh()#

function@openav/fab
weldMesh(mesh, tolerance)

Merges vertices that sit closer together than a tolerance and drops the faces that collapse; returns a new mesh with shared vertices.

A triangle soup, where every face has three vertices of its own (what an STL file or three.js toNonIndexed() gives), has no neighbours as far as topology goes: every edge looks open. Welding gives the faces shared vertices, so checkMesh() can see that they join. checkMesh(), toObjText() and toGlbBytes() call it themselves; toStlBytes() does not need it.

Parameters

NameTypeDescription
meshTriMeshA mesh { vertices, triangles }. It is not changed.
toleranceNumberVertices closer than this (in each of x, y and z) become one, in the mesh's units. Left out: 1e-6 times the diagonal of the bounding box. optional

Returns

TriMesh { vertices: Float64Array, triangles: Uint32Array, units } with each position once, and no face that has two corners on the same vertex.

Examples

js
import { weldMesh, fromStlBytes, toStlBytes } from '@openav/fab';

const tetra = {
  vertices: new Float64Array([0, 0, 0, 10, 0, 0, 0, 10, 0, 0, 0, 10]),
  triangles: new Uint32Array([0, 2, 1, 0, 1, 3, 0, 3, 2, 1, 2, 3]),
};
const soup = fromStlBytes(toStlBytes(tetra));   // reading an STL gives every face its own vertices
const welded = weldMesh(soup);
document.getElementById('app').textContent =
  'soup: ' + soup.vertices.length / 3 + ' vertices, welded: ' + welded.vertices.length / 3 + ' vertices, ' + welded.triangles.length / 3 + ' faces';
module Prints "soup: 12 vertices, welded: 4 vertices, 4 faces".

Related

  • checkMesh() Measures a triangle mesh for 3D printing: closed or not, volume, mass, overhang, thinnest wall, fit on the bed, with a plain warning for each problem.
  • fromStlBytes() Reads a binary STL file back into a triangle mesh: every face has its own three vertices, in 32-bit floats.
  • toObjText() Writes a triangle mesh as OBJ text: welded vertices (v x y z), then faces (f a b c), one object, in millimetres.
  • toGlbBytes() Writes a triangle mesh as a minimal binary glTF (.glb): one mesh with welded positions and indices, in millimetres.

import { weldMesh } from '@openav/fab'; packages/fab/checks.js, line 22

MCP tools for agents#

The tools of packages/mcp/server.js. 8

@openav/mcp

list_controllers#

tool@openav/mcp
list_controllers

Lists every MIDI controller profile that ships with the framework: id, name, maker, kind, control mix, how many controls are unverified, and its page URL.

No input: send "arguments": {}. It reads the profiles in packages/midi/profiles/, the same ones the controller pages and <oav-controller> use. Use it to find the id for read_profile, or to check that a controller is already covered before writing a new profile with scaffold_profile.

Returns

Object { count, controllers } as JSON text. Each controller is { id, short, name, maker, kind, controls, unverified, url }: id such as arturia-minilab3, short the signal namespace (midi/<short>/…), kind 'device' or 'generic', controls the mix as { knob: 8, pad: 8, keys: 25 } (a keyboard counts one per key), unverified how many of those controls still carry "verified": false, url like https://openaudiovisual.com/controllers/<id>/.

Examples

json
{
  "jsonrpc": "2.0",
  "id": 5,
  "method": "tools/call",
  "params": {
    "name": "list_controllers",
    "arguments": {}
  }
}
static The reply lists every shipped profile, starting with arturia-minilab3 (8 knobs, 4 faders, 8 pads, 25 keys …), each with its page URL.

Related

  • read_profile Reads one controller profile in full, plus a readable MIDI map: for each control its type, the exact message, the signal name and whether the number is verified.
  • validate_profile Checks a controller profile with validateProfile() and against the shipped ones: same id, same short name, or port patterns that claim the same device.
  • scaffold_profile Generates a valid skeleton profile for a new controller from maker, model and control counts. Every number is an unverified placeholder; nothing is written.

packages/mcp/server.js, line 77

list_examples#

tool@openav/mcp
list_examples

Lists the example works of this checkout: for each one its folder, its page title and the first comment line of its main.js.

The first call an agent makes in an unfamiliar checkout: what works already exist, so it can pick a donor for scaffold_world. It takes no input: send "arguments": {}.

Before it has a value

The server running: node packages/mcp/server.js, which Claude Code starts from the .mcp.json of the repository (server name openav). Other clients need the same command and args in their MCP settings. An MCP client sends initialize once before the first call. Over stdio every message is one line of JSON: the request bodies on this page are spread over several lines to be read, and a client writes them on one line for you.

Returns

Object[] [{ dir, title, summary }], one per folder in examples/, as JSON text. dir is like examples/01-hello-particles, title is the <title> of its index.html, summary is the first line of its main.js that starts with // (the slashes taken off). A file that is missing gives an empty string.

Examples

json
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "list_examples",
    "arguments": {}
  }
}
static Send this to the server over stdio: the reply carries one text item, a JSON list of the examples with their folders, titles and one-line summaries.

Related

  • scaffold_world Creates a new numbered example by copying a donor example's files; then the agent edits the World in main.js. This tool writes to disk.
  • read_doc Returns the text of one framework document: the agent briefing, a handbook chapter or the README, chosen by name.

packages/mcp/server.js, line 55

read_doc#

tool@openav/mcp
read_doc

Returns the text of one framework document: the agent briefing, a handbook chapter or the README, chosen by name.

Input: doc (required), one of architecture, writing-a-world, signals, show-control, score, roadmap, agents, readme, introduction, getting-started, concepts, inputs, mapping, controllers, remote, packages, agents-guide, troubleshooting, lab-integration. The tool's own description tells an agent to start with agents (the conventions and the two unbreakable rules), then writing-a-world.

Most names are docs/<name>.md. Two are not, and they are easy to mix up: agents is AGENTS.md at the root of the repository (the briefing), agents-guide is docs/agents.md (the handbook chapter), and readme is README.md.

Returns

String The Markdown of the file, as it is on disk, in one text item: not JSON. The whole file comes back, which can be tens of kilobytes.

Notes

  • A name that is not in the list, or no doc at all, is an error with a plain Node message (The "path" argument must be of type string. Received undefined), not a list of the valid names. A client that follows the input schema cannot send one: the schema has the names as an enum.

Examples

json
{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/call",
  "params": {
    "name": "read_doc",
    "arguments": {
      "doc": "agents"
    }
  }
}
static Asks for AGENTS.md: the reply is its Markdown text, which begins "# AGENTS.md — for AI coding agents working in this repo".

Related

  • list_examples Lists the example works of this checkout: for each one its folder, its page title and the first comment line of its main.js.
  • scaffold_world Creates a new numbered example by copying a donor example's files; then the agent edits the World in main.js. This tool writes to disk.

packages/mcp/server.js, line 60

read_profile#

tool@openav/mcp
read_profile

Reads one controller profile in full, plus a readable MIDI map: for each control its type, the exact message, the signal name and whether the number is verified.

Input: id (required), a profile id or its short, for example arturia-minilab3 or minilab3. Where a control says verified: false, its number is a placeholder or a guess that no source has confirmed yet. An unknown id is an error that lists every id it knows. list_controllers shows what is there.

Returns

Object { id, url, unverified, profile, mapping } as JSON text. profile is the JSON as shipped. mapping has one row per control: { control, type, message, signal, verified }, where message reads like CC 74, ch 1, note 36, ch 10 or notes 48–72 (25 keys), ch 1, and signal is midi/<short>/<id> (for a keyboard midi/<short>/<id>/on · /off · n<note>); a row may also carry led and note.

Examples

json
{
  "jsonrpc": "2.0",
  "id": 6,
  "method": "tools/call",
  "params": {
    "name": "read_profile",
    "arguments": {
      "id": "minilab3"
    }
  }
}
static Asks for the MiniLab 3 by its short name: the reply holds its full profile and a mapping whose first knob reads "CC …, ch 1" with the signal midi/minilab3/knob1.

Related

  • list_controllers Lists every MIDI controller profile that ships with the framework: id, name, maker, kind, control mix, how many controls are unverified, and its page URL.
  • validate_profile Checks a controller profile with validateProfile() and against the shipped ones: same id, same short name, or port patterns that claim the same device.
  • scaffold_profile Generates a valid skeleton profile for a new controller from maker, model and control counts. Every number is an unverified placeholder; nothing is written.

packages/mcp/server.js, line 82

run_checks#

tool@openav/mcp
run_checks

Runs the framework's whole test suite (node --test on every file in tests/), the same gate CI uses, and says whether it passed.

No input: send "arguments": {}. An agent calls it after every change to know whether the repository is still green. It runs node --test over every tests/*.test.js of the checkout the server lives in, waits for it to finish and gives up after 60 seconds.

Returns

Object When the run finishes cleanly: { pass, fail, ok } (numbers from the report, and ok true only when fail is 0), as JSON text. Otherwise { ok: false, output }, where output holds the lines of the report that say not ok or give the totals.

Notes

  • At the time of writing, in the checkout this was written from, the whole suite takes about 50 seconds and prints about 3.6 MB of report, while the tool reads at most 1 MiB of it. Past that the child process is stopped, and the tool answers { ok: false, output } with only the failing lines found in that first megabyte, even if every test passes. Until that changes, do not read ok: false with an empty output as a failing test: run node --test tests/*.test.js in a terminal.

Examples

json
{
  "jsonrpc": "2.0",
  "id": 4,
  "method": "tools/call",
  "params": {
    "name": "run_checks",
    "arguments": {}
  }
}
static Runs the suite: the reply is { pass, fail, ok } when it ends cleanly, or { ok: false, output } with the failing lines. Allow about a minute.

Related

  • scaffold_world Creates a new numbered example by copying a donor example's files; then the agent edits the World in main.js. This tool writes to disk.
  • validate_profile Checks a controller profile with validateProfile() and against the shipped ones: same id, same short name, or port patterns that claim the same device.

packages/mcp/server.js, line 117

scaffold_profile#

tool@openav/mcp
scaffold_profile

Generates a valid skeleton profile for a new controller from maker, model and control counts. Every number is an unverified placeholder; nothing is written.

Input, all required: maker (such as Arturia), model (such as MiniLab 3) and controls, a list of { type, count, from? } that describes the faceplate, for example [{"type":"knob","count":8},{"type":"pad","count":8},{"type":"keys","count":25,"from":48}]. type is one of knob, encoder, fader, pad, button, keys, wheel, strip; count is a whole number from 1 to 128; from is the lowest note of a keys section (default 48).

The skeleton passes validateProfile() and it is deliberately honest: every control has "verified": false, the source list holds one TODO entry, and the numbers are placeholders. Knobs, faders and strips get consecutive CC numbers from 20, pads get notes from 36 (skipping a keys range), buttons get momentary CCs, encoders get relative CCs, the first wheel is a pitch bend and any further wheel falls back to a CC. The id comes from maker and model (acme-pad-pro-16), short is the model without hyphens, and match.ports is a pattern made of the model's words.

Wrong input is an error with the reason: no maker or model, controls empty or not a list, an unknown type, a count outside 1..128, keys outside notes 0..127, or more controls than there are CC numbers (about 108) or pad notes. After that the agent sources every number from the maker's MIDI chart, runs validate_profile, and saves the file in packages/midi/profiles/; next_steps lists these.

Returns

Object { profile, problems, unverified, next_steps } as JSON text, plus warnings when the id or short name collides with a shipped profile. profile is the skeleton, problems is what validateProfile() says about it (empty), unverified is how many controls are placeholders (all of them), next_steps is what to do to turn it into a real profile.

Examples

json
{
  "jsonrpc": "2.0",
  "id": 8,
  "method": "tools/call",
  "params": {
    "name": "scaffold_profile",
    "arguments": {
      "maker": "Acme",
      "model": "Pad Pro 16",
      "controls": [
        { "type": "knob", "count": 8 },
        { "type": "pad", "count": 16 },
        { "type": "keys", "count": 25, "from": 48 }
      ]
    }
  }
}
static Asks for a skeleton for an invented "Acme Pad Pro 16": the reply has id acme-pad-pro-16, 25 control entries (the keyboard is one entry of 25 keys, so unverified is 49), no problems, and the next steps.

Related

  • validate_profile Checks a controller profile with validateProfile() and against the shipped ones: same id, same short name, or port patterns that claim the same device.
  • read_profile Reads one controller profile in full, plus a readable MIDI map: for each control its type, the exact message, the signal name and whether the number is verified.
  • list_controllers Lists every MIDI controller profile that ships with the framework: id, name, maker, kind, control mix, how many controls are unverified, and its page URL.

packages/mcp/server.js, line 92

scaffold_world#

tool@openav/mcp
scaffold_world

Creates a new numbered example by copying a donor example's files; then the agent edits the World in main.js. This tool writes to disk.

Input: slug (required), a kebab-case name made of lower-case letters, digits and hyphens, such as jellyfish-bloom; and donor (optional, default 01-hello-particles), the name of a folder in examples/ to copy. Good donors: 01-hello-particles (a minimal canvas), 02-chord-garden (driven by chords), 03-pose-field (the camera), 05-prebiotic-flake (p5 and sound). The new folder is examples/<number>-<slug>, where the number is the highest one already in examples/ plus 1, with two digits (28).

It copies every file of the donor folder (the donors have no sub-folders, and a sub-folder would stop the copy with an error), so the new example starts as a working copy. A slug with anything else in it, or an unknown donor, or a folder that already exists is an error (slug must be kebab-case, unknown donor: …, already exists: …).

Returns

Object { created, next_steps } as JSON text: created is the new folder (examples/28-jellyfish-bloom), next_steps is three lines: edit the World in main.js, edit the <title> and hint in index.html, then open it with node serve.js.

Notes

  • Send slug every time. It is marked required in the input schema, but the tool does not check that it is there: without it the folder is created with the name examples/<number>-undefined.
  • donor is used as a folder name without further checks, so give it a plain name from examples/.

Examples

json
{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "scaffold_world",
    "arguments": {
      "slug": "jellyfish-bloom",
      "donor": "02-chord-garden"
    }
  }
}
static Creates examples/<next number>-jellyfish-bloom from the chord garden; the reply says where, and what to edit next.

Related

  • list_examples Lists the example works of this checkout: for each one its folder, its page title and the first comment line of its main.js.
  • read_doc Returns the text of one framework document: the agent briefing, a handbook chapter or the README, chosen by name.
  • run_checks Runs the framework's whole test suite (node --test on every file in tests/), the same gate CI uses, and says whether it passed.

packages/mcp/server.js, line 65

validate_profile#

tool@openav/mcp
validate_profile

Checks a controller profile with validateProfile() and against the shipped ones: same id, same short name, or port patterns that claim the same device.

Input: profile (required), the profile as an object or as a JSON string. A string that is not valid JSON does not raise an error: the answer is { ok: false, problems: ['not valid JSON: …'], conflicts: [] }. The checks cover the id and short name (lower-case, and short must not be one of the reserved generic names), a name and a kind (device or generic), a source for every device profile, the port regular expressions, the faceplate size, every control (a known type and message, cc or note in 0..127, two controls never on the same message, an encoder's relative mode) and that every control is shown on the faceplate.

The conflicts are what the validator cannot see by itself: id and short are taken by a shipped profile (the short becomes the signal namespace midi/<short>/…), and port means a regular expression in match.ports is identical to a shipped one or matches its name, so the framework could not tell the two devices apart when one is plugged in. A profile for a new device passes when ok is true; a clone of a shipped one comes back with all three kinds.

Returns

Object { ok, problems, conflicts } as JSON text. problems is a list of sentences from validateProfile() (controls[0] (a): cc must be 0..127). conflicts is a list of { with, kind, detail }: with is the shipped profile, kind is 'id', 'short' or 'port'. ok is true only when both are empty.

Examples

json
{
  "jsonrpc": "2.0",
  "id": 7,
  "method": "tools/call",
  "params": {
    "name": "validate_profile",
    "arguments": {
      "profile": {
        "id": "Bad Id",
        "kind": "device",
        "controls": [
          {
            "id": "a",
            "type": "knob",
            "msg": "cc",
            "cc": 200
          }
        ]
      }
    }
  }
}
static Sends a profile with several mistakes: the reply has ok: false and a list of problems, among them the id that is not lower-case and "cc must be 0..127".

Related

  • scaffold_profile Generates a valid skeleton profile for a new controller from maker, model and control counts. Every number is an unverified placeholder; nothing is written.
  • read_profile Reads one controller profile in full, plus a readable MIDI map: for each control its type, the exact message, the signal name and whether the number is verified.
  • list_controllers Lists every MIDI controller profile that ships with the framework: id, name, maker, kind, control mix, how many controls are unverified, and its page URL.

packages/mcp/server.js, line 87

↑ Back to top