open-audiovisual
Search GitHub

Sister project · the engine

WebToe A node-based dataflow engine for real-time visuals, in the browser.

Patch operators together the way you would in TouchDesigner: a ramp through a transform into a composite, an LFO driving a parameter through an expression, a live preview on every node. WebToe is an original engine, built from scratch for the web on WebGL2 and WebGPU, and it opens your existing TouchDesigner projects.

WebToe is the engine, open-audiovisual is the show. A WebToe patch can be performed as an open-audiovisual world, and named signals map naturally onto CHOP channels.

The WebToe editor running the lfo garden example: three ramp and transform chains wired into composite, hsv adjust and out, a live preview on every node, the output in the viewer.
Example 03, lfo garden: three ramp chains rotated by LFOs through expressions, composited and hue-shifted. Every node previews live.

01 · The engine

Patch it live, in a browser tab

A network editor, a real-time GPU engine, and a parameter panel where any value can be an expression. No install, no build step.

of JavaScript for the whole app, with zero runtime dependencies
~90 KB
two GPU backends at parity, behind one pass contract
WebGL2 + WebGPU
of 28,698 nodes from 60 real TouchDesigner projects are runnable: the measured corpus coverage
62.3%
bundled projects that run out of the box, two of them raw .toe files saved by TouchDesigner in 2022
12

Numbers from the WebToe README, measured on real projects.

  • Patch live. Press Tab to create an operator, drag wires, step into containers. Every node shows a real-time preview, and one GPU compositor paints them all at full frame rate.
  • Expressions anywhere. Any parameter can be an expression: op('lfo1')['chan1'], parent().par.speed, time.seconds * 0.2.
  • A real-time GPU engine. A pull-based cook loop. TOPs run as GPU passes and CHOPs drive parameters: feedback, blur, compositing, displacement, edge detection, webcam and video input, and a 3D pipeline with geometry, cameras, lights and a render TOP.
  • Six operator families. TOP, CHOP, SOP, MAT, COMP and DAT. Projects save losslessly as .webtoe.json, a versioned format of its own.
Example 02, feedback trails, in the WebToe editor: rectangle, transform, composite, blur and out operators, with a feedback and level loop wired back into the composite.
Example 02, feedback trails. Move the mouse over the viewer and the rectangle leaves a trail through the feedback loop.

02 · With open-audiovisual

WebToe is the engine, open-audiovisual is the show

open-audiovisual brings the inputs, the mapping, the timeline and the backstage. WebToe draws. The adapter @openav/world-webtoe turns a WebToe patch into a world, so a knob, a chord or a hand can play it.

  1. 01 Signal

    midi/cc/1

    Something happens on stage: a key is struck, a knob turns. It arrives as a named signal.

  2. 02 Param

    midi/cc/1 → speed

    The mapper routes the signal to a param of the world, with a curve and smoothing. The timeline can drive the same param.

  3. 03 Message

    { type: 'webtoe:ext', values }

    The world embeds the WebToe app in an iframe. Whenever a param moves, it posts the new values into it.

  4. 04 Patch

    ext('speed', 0.5)

    Inside the patch, every parameter written with ext() follows. The second number is the fallback while nothing is connected.

The show: a world that is a WebToe patch

Example 04, shortened. The param keys are the names the patch listens for. The project file is handed to the app as ?project=.

examples/04-webtoe-stage/main.js
import { createShow } from '@openav/show';
import { webtoeWorld } from '@openav/world-webtoe';

await createShow({
  world: webtoeWorld({
    project: new URL('./garden.webtoe.json', location.href).href,
    // the keys are the names the patch listens for
    params: [
      { key: 'speed', min: 0, max: 1, def: 0.5 },
      { key: 'hue', min: 0, max: 360, def: 205 },
    ],
  }),
  routes: [{ source: 'midi/cc/1', target: 'speed' }],
});

The patch: parameters that listen

Two expressions from the example’s garden.webtoe.json. Put ext() into any parameter and it becomes playable.

garden.webtoe.json
// lfo1 · frequency
0.11 * (0.2 + ext('speed', 0.5) * 2.2)

// hsv1 · hueoffset
fract(ext('hue', 205) / 360 + time.seconds * 0.01)

Drive it from any page

These are plain postMessage calls, so any page that embeds WebToe can drive it. Values must be finite numbers. webtoe:load opens another project without reloading the frame.

postMessage
const webtoe = iframe.contentWindow;   // an iframe showing the WebToe app

// set the values every ext('name') in the patch reads
webtoe.postMessage({ type: 'webtoe:ext', values: { speed: 0.72 } }, '*');

// open another project: a .webtoe.json or .toe URL the app may fetch
webtoe.postMessage({ type: 'webtoe:load', url: nextProjectUrl }, '*');

03 · TouchDesigner projects

Drop a .toe and it opens

No install and no command-line step: WebToe decodes the TouchDesigner container right in the browser. Supported operators run live. Everything else becomes a stub that keeps its name, wires, layout, parameters and Python code, and an import report says what happened.

Validated against toeexpand

125 / 125
production project files, 3 KB to 151 MB, decoded to identical file sets, byte for byte (553,230 files)
17.7 s
to decode all 125 natively, against 266.5 s for TouchDesigner’s own toeexpand
4.2 s
for a 151 MB show file. A daily sketch takes a few milliseconds.
8 s
from drop to a running graph for a real 20 MB show file of 14,710 nodes

Try it with a raw 2022 project file ↗

The TouchDesigner import report in WebToe after importing a 213-node production project: 71 nodes runnable, 142 kept as stubs, 9 expressions translated.
The import report for a 213-node production project: what runs, what was kept as a stub, which expressions were translated.

The reference path: your own TouchDesigner

If native decoding fails, WebToe falls back to the official toeexpand tool that ships with every TouchDesigner install, run on your machine through a small bridge. The bridge listens on 127.0.0.1 only and has zero dependencies. Your project files never leave your computer.

terminal
npx webtoe        # the app, served locally with the bridge: drag your .toe in