Skip to content

Latest commit

 

History

History
1324 lines (1136 loc) · 72.5 KB

File metadata and controls

1324 lines (1136 loc) · 72.5 KB

Writing modules

Modules are self-contained folders under src/modules/<name>/ that plug into the scene, the flow node editor and the peer mesh. Music instruments, playable game prototypes, generators, custom controls — anything a user should be able to drop into a session and every connected peer can see and interact with.

The one rule that matters

A module runs on every peer. There is no server. Whatever your module does must end up identical on all clients, one of three ways:

  1. Deterministic from shared inputs — effects driven by node data (already replicated) and the synced clock. No extra work needed. Prefer this.
  2. Broadcast events — a discrete thing happened ("button pressed", "generate with seed 42"). Send a small message with api.send, apply the same change in the sender and in api.onMessage. Never re-broadcast from a receiver.
  3. State sync for late joiners — someone connects mid-session and needs your current state (score, generated layout, toggles). Provide registerStateSync; the handshake delivers it automatically.

Randomness must be seeded (send the seed, not the result — see the dungeon module). Math.random() in anything replicated is a desync.

Layout and registration

src/modules/
  index.js            <- list your module here (all peers need the same list)
  mymodule/
    module.js         <- export default { id, name, version, register(api) }
    SomethingNode.svelte  (optional custom node UI)
// src/modules/mymodule/module.js
export default {
	id: 'mymodule',        // stable, unique; used to route your messages
	name: 'My Module',
	version: '1.0.0',      // peers toast when versions differ
	register(api) {
		// wire everything here
	}
};

Peers exchange {id, version} lists when they connect and warn when a module is missing or a different version on the other side. That's advisory — the session still works, but replicated behavior of that module may differ.

The lifecycle: load, unload, reload (34 R6)

A module can be unloaded and loaded again while the app runs — switched off in the Modules manager (core modules too, since 1.20), removed, updated, dev-reloaded from its Dev URL, or unloaded by a scene switch. unloadModule(id) takes down everything the module registered, newest first, and register(api) can then run again with fresh code. Nothing needs a page reload, and nothing is left running for a module that is gone.

What is undone for you. Every registration you make through api is recorded in the module's lifecycle registry (one registry, keyed by your module id — src/lib/sdk/lifecycle.js) and undone at unload: node groups, effects, value nodes, node defs you seeded and the user did not edit, primitives, click/drop handlers, frame tasks, scene-clear hooks, interactive / system / listed groups (the scene-root group is removed AND its geometries, materials and textures are freed), the spawn override, message handlers and state sync, menus, toolboxes (closed) and their shortcuts, VR menu entries, key bindings, input listeners and claims, possess, the follow camera, VR panels, hit listeners, your music track, game levels / settings rows / help / restart hooks / the forced menu, LOD handles, quality / flow / game / settings / peer-variable listeners, unwrap / shader / post backends, post effects, audio device kinds, engine voices (api.audio.voice), transport events (api.audio.schedule), your copy of the mic stream, a recording you started, HUD rows / debug lines / actions / element kinds.

Every off() the api hands you still works, and calling it early also drops the registry entry — subscribe and unsubscribe as often as you like. A call that REPLACES (api.game.levels called on every unlock, setHelp, addSetting with the same id, a toolbox or VR menu entry re-registered under the same id) replaces its entry rather than stacking one per call.

What stays. What your module created in the SHARED scene (api.create, objects in objectsGroup, nodes from api.flow.addNodes, audio devices and cables, node data you wrote) is user content: it stays, replicated, like anything a person made. So do the shared game variables and your peer-variable row, and what you saved on this device with api.storage (a reinstall keeps progress; storage.clear() is your reset).

What only you know about — four tools:

// your own teardown: a DOM overlay you appended, a worker, a cache, a raw WebAudio graph
// you built on api.audio.context(). Runs at unload (newest first, so a hook added at the
// end of register() runs while everything before it is still in place). Returns cancel().
api.onUnload(() => overlay.remove());

// timers that die with the module (the window's signatures) + what is pending right now
const h = api.timers.setInterval(refresh, 500);
api.timers.clearInterval(h);
api.timers.setTimeout(fn, ms); api.timers.requestAnimationFrame(loop);
api.timers.pending(); // {timeouts, intervals, frames}

// an event listener removed at unload — window, document, the canvas, anything; returns off()
const off = api.listen(window, 'keydown', onKey, true);

// a scene-root object that is YOURS: removed at unload, its GPU resources freed (shared
// ones kept). Returns the object, so: api.scene().add(api.own(group)). Never for content
// inside objectsGroup (that is shared — it is left alone).
api.own(group);

Installed modules (zip / URL) get timers and window listeners tracked automatically. The entry is evaluated with its bare setTimeout / clearTimeout / setInterval / clearInterval / requestAnimationFrame / cancelAnimationFrame bound to tracked ones (a one-line prologue on line 1, so line numbers in stack traces are unchanged), and a listener your code adds to window or document is recorded too — at unload they are cleared and removed. Two edges: window.setTimeout(...) (or a listener on any other target) is not tracked — use api.timers / api.listen; and a module that declares one of those six names itself at top level is loaded without the prologue (its timers then untracked). A core module (in src/modules/) uses api.timers / api.listen explicitly.

Keep state inside register(). The browser keeps every imported module's TOP-LEVEL scope for the life of the page (an ES module is never unloaded), so module-level variables, caches and DOM survive an unload — the next load starts from them. State declared inside register() is fresh every time; module-level state you must reset belongs in api.onUnload. Each reload of an installed module also keeps the evaluated source itself, so a large module costs some heap per reload — fine for a development loop, not something to do every second. (Measured with module-lifecycle-repo, 3 cycles each: 19 of the 20 repo modules reload flat; Waves keeps one DOM node and ~0.3 MB of heap per reload.)

Adding an SDK surface (core contributors)

Every member of the api declares what it does to a module's lifecycle, beside its slice (src/lib/sdk/<name>.js):

export function sdkThing(ctx) {
	const { moduleId, onDispose, owned } = ctx;
	return {
		registerThing(spec) {
			thingRegistry.set(key, spec);
			// the undo, journaled with a KIND (the debug view names it); `{key}` makes a
			// re-registration under the same key REPLACE its entry
			onDispose(() => thingRegistry.delete(key), 'thing', { key: 'thing:' + key });
		},
		onThing(fn) {
			// a registration that hands back an off(): owned() returns an off that also
			// drops the journal entry
			return owned('thing.listener', subscribe(fn));
		},
		thingCount: () => thingRegistry.size
	};
}
sdkThing.surface = { registerThing: 'registers', onThing: 'registers', thingCount: 'read' };
  • Kinds: registers (leaves something behind: MUST journal its undo), action (does something now, leaves nothing of the module's), content (shared, replicated content that deliberately survives an unload), read, value (a non-function).
  • A registers member needs a fixture in tests/fixtures/sdkLifecycleFixtures.js: register it from a real module, read CORE's registry (never the journal), unload, read it gone. The vitest moduleLifecycle fails for an undeclared member, a stale declaration, a registers member with no fixture, and a fixture whose registration survives unloadModule; the browser suite module-lifecycle runs the members node cannot load.
  • Async registrations journal SYNCHRONOUSLY. A disposer recorded inside an import().then lands in the module's NEXT journal whenever an unload comes first — the registration then outlives the unload. Record it before the import and let a late registration undo itself: let off = null, gone = false; onDispose(() => { gone = true; off?.(); }, 'kind'); import('../x').then((m) => { off = m.register(...); if (gone) off(); });
  • A core subsystem acting FOR a module outside the api (kit entities, loaded models) records its undo with trackModuleResource(moduleId, kind, undo) from moduleSDK.js.
  • Debug: __stores.moduleSDK.registrationsOf(id) / allRegistrations() (live entries by kind), moduleScopeDebug() (an installed module's tracked timers + listeners), unloadModule(id) returns what it disposed, by kind.
  • Tests: vitest moduleLifecycle; e2e module-lifecycle (the browser contract, an installed probe module, and the leak test: each bundled module unloaded/loaded 3x with handlers, nodes, audio connections, timers, meshes, materials, GPU memory, DOM, JS listeners and the heap measured) and module-lifecycle-repo (every zip in the modules repo through 3 cycles).

API reference (v1)

Flow nodes

api.registerNodeGroup(
	{
		group: 'Modules', // palette/context-menu group header
		items: [
			{
				type: 'wave',                 // globally unique node type
				label: 'Wave (hello)',
				defaults: { amplitude: 0.4 }, // seeds node.data, replicated with the node
				params: [                     // optional: auto-generated controls
					{ key: 'amplitude', kind: 'range', min: 0, max: 1.5, step: 0.05 }
					// kind: 'range' (min/max/step), 'select' (options: [...]), 'toggle',
					// or 'text' (placeholder?, maxLength?) — a text param writes on COMMIT
					// (change/blur), never per keystroke: a node edit replicates the whole node
				]
			}
		]
	},
	{ wave: MyWaveNode } // optional: custom Svelte components per type
);

Items with params render with the built-in generic node UI (sliders/selects that write through setNodeData, which replicates). Custom components should follow the same pattern: render from data, write with setNodeData(id, {...}) from $lib/nodesHandler, never bind: to data.

Per-frame effects (node → Object Selector)

api.registerEffect('wave', (object, base, data, time) => {
	// base = the object's logical transform: {pos, rot, scale, visible}
	// The runtime restores base before each frame — apply offsets FROM base.
	object.rotation.z = base.rot[2] + Math.sin(time * (data.speed ?? 2)) * (data.amplitude ?? 0.4);
});

Effects run when an edge connects your node to an Object Selector node. time is wall-clock synced across peers (when "Sync animations" is on), so pure functions of (data, time) stay identical everywhere. Don't accumulate (rotation.z += ... breaks determinism) — always compute from base and time.

An effect takes an optional 5th argument {id, graphId} — its own node id and the graph it sits in, so one module can host many instances of the same node type. It is additive: a four-parameter effect is unchanged.

api.registerEffect(
	'wave',
	(object, base, data, time, ctx) => {
		// ctx.id = this node's id, ctx.graphId = 'scene' or the owning object's uuid
	},
	{ inputs: { who: 'object' } } // optional: typed named inputs, see below
);

Value nodes: your state into core nodes

A node that outputs a value, so module state can drive core nodes — a score into a HUD Text, a level into Map Range, a flag into a Gate.

api.registerValueNode(
	'score',
	(data, time, ctx) => Number(data.base ?? 0) * Number(data.mult ?? 1),
	{
		vtype: 'number',        // output socket type; also boolean/vector3/color/object/event
		inputs: { mult: 'number' } // typed named inputs, resolved BEFORE fn runs
	}
);

The evaluator MUST be a pure function of its arguments — the same rule Script nodes follow, and the one way to break a session silently. Values are never sent: every peer evaluates the node itself from the replicated node data and the shared clock. Read unreplicated local state here (a mouse position, a plain module Map, Math.random()) and every downstream consumer diverges per peer with no error anywhere. Keep mutable state in a replicated place — registerStateSync / api.send — and read that, or let the value ride the node's own data.

inputs matters more than it looks. Without a declaration every handle types as number, which refuses an Object Selector wire (object -> number is not a coercion) and renders no target socket on the card at all. data.<handle> is the wired value when wired and the node's own param when not. registerEffect accepts the same {inputs} option.

Firing your own events

// pulse every instance of the type...
api.fireNodeTrigger('levelcleared');
// ...or just the ones you mean
api.fireNodeTrigger('goal', (data, id) => data.team === 'blue');

Register the node with {vtype: 'event'} so it can be wired to a Counter or an Object Selector. This REPLICATES, exactly like fireObjectClick: the pulse rides the existing nodetrigger message from one peer's stamp, and every peer then computes the identical pulse. So call it on the peer where the event happened and not on all of them, or a Counter counts it once per peer. api.peerIds() and api.physics.isInitiator() are the usual ways to pick that peer.

R3a: pass {replicate: false} as the third argument to keep the pulse in this peer's own trigger log — the per-player mechanism (a per-player pickup hides only for its collector because the pulse never left their machine). Your effect/value node then reads its own pulse through ctx.trigger ({stamp, age} or null), which core has ALREADY folded through the round rules: stamp a perRound: true flag on your node's data and a round bump (or a return to the menu) retires the read with no round math of your own. That is the whole latch story — never rebuild it.

The game shell, per-player rows, and the graph (R3a)

api.game.roundCutoff();            // null (shell unused) | Infinity (menu/over) | startedAt
api.game.roundUnderway();          // true while playing/paused (and when the shell is unused)
api.game.playActive();             // THIS peer plays inside a running round
api.game.getVar('score', 0);       // the shared game variable (replicated singleton)
api.game.setVar('score', 7);

api.peerVars.setMine('laps', 3);   // MY replicated row — one writer per row, by construction
api.peerVars.mine('laps', 0);
api.peerVars.all('laps');          // [{id, name, value, me, rank}] — the leaderboard shape

api.playerPosition();              // [x, y, z] of the viewer — a touch trigger's read
api.selectObject(uuid);            // the manager-row click
api.selectedUuids();               // the selection SET (never the sticky primary)

api.flow.nodes('mytype');          // [{id, type, graphId, data}] across every graph
api.flow.edges();                  // [{id, source, target, sourceHandle, targetHandle, graphId}]
api.flow.nodeValue(latchId);       // a node's evaluated output (a core Latch's round-aware state)
api.flow.triggerStamp(nodeId);     // {stamp, age} | null — a node's own round-aware pulse
api.flow.setNodeData(nodeId, { respawn: 5 });   // replicated MERGE (the editor's nodedata path)
api.flow.addNodes({                // replicated, ONE undo entry for the batch
	nodes: [{ type: 'mytype', x: 60, y: 40, data: {} }, { type: 'objectselector', x: 280, y: 40, data: { selected: uuid } }],
	edges: [{ from: 0, to: 1 }]     // indices into nodes, or existing node id strings
});
api.flow.seedGraph({               // 36 (DEVX #10): the same, ABSENT-ONLY — a wired example
	key: 'example',                // graph on module load; nothing is added once any node is
	nodes: [/* … */], edges: []    // marked with this seed or is one of THIS module's types
});

api.hud.registerDebugLine(() => 'mymod: 3 left');   // a line on the debug HUD pill
api.hud.registerAction({ key: 'showleft', label: 'Show it', group: 'Data',
	role: 'drives', node: '', via: { node: 'mytype', data: {}, handle: 'value' } });

Graph reads are deterministic because the graph is replicated — treat them exactly like replicated state (the value-node rule). Two per-node data flags core honours on YOUR nodes: perRound (the trigger reads retire on a round bump) and whilePlaying (an effect node stands down outside play and the restore loop hands its object back — how a module hides an object without owning the give-back).

Primitives

api.registerPrimitive(
	'Flag',                                    // capitalized: /create Flag 2 1
	(w, h) => new THREE.PlaneGeometry(+w || 2, +h || 1),
	{ label: 'Flag', command: '/create Flag 2 1' } // spawn button on your manager card
);

The builder runs on every peer from the same /create command, so it must be deterministic in its arguments.

Interaction

api.registerClickHandler((object) => {
	if (object.userData.myButton) {
		press(object);                    // apply locally
		api.send({ op: 'press', uuid: object.uuid }); // tell peers
		return true;                      // consume the click (no selection)
	}
	return false;
});

Handlers see desktop clicks and VR trigger presses (the exact mesh hit, not the top-level group). Return false to let normal selection continue.

Modes (1.17). The editor has two click modes, Edit (every click selects) and Interact (play-style clicks and drags without starting the game), plus Play. A handler runs in Interact and Play by default, so an Edit click on your game piece selects it like any object. Say so when you want something else:

api.registerClickHandler(pressKey, { modes: ['interact', 'play'] }); // the default, spelled out
api.registerClickHandler(pickTool, { modes: ['edit'] });             // an editor TOOL

In a headset the same modes apply (1.17, 30b): Play in VR enters Interact, where the trigger clicks (module handlers, On Click) and selects nothing; in Edit it selects as before. Handlers also get ctx = {source, mode} — see Game feel below for the hold-and-sweep and {sweep: false}. api.editorMode() reads the mode on this screen ('edit' | 'interact') — for a module whose own pointer listeners run outside click routing (a drag), so it can stand down in Edit.

Listed in the object list (1.17). Content you build at the scene root shows in the object list's Module content section — a read-only row that frames it, hides it locally, or opens your toolbox. Give it a readable name:

api.registerListedGroup('piano-module', { label: 'Piano' });
api.registerFrameTask((time) => { /* runs every frame, synced time */ });

// click handlers only see the replicated objects root by default; if your
// module adds its own group at the scene root, register it for clicks:
api.registerInteractiveGroup('pong-module');

// an Explorer AUDIO/TEXT item dropped on one of your meshes (23-C2): take it or not.
// `hit` is the exact mesh under the drop, `item.hash` feeds api.audio.sample(hash).
api.registerDropHandler((hit, item, target) => {
	if (item.kind !== 'audio' || !hit.userData.pad) return false;
	api.audio.setParams(deviceOf(hit).uuid, { ['pad' + hit.userData.pad]: item.hash });
	return true;                     // consumed
});

Toolboxes: a real UI surface (A5)

const id = api.registerToolbox({
	id: 'settings',              // namespaced to mod-<moduleId>-settings
	title: 'Dungeon Kit',
	width: 240,
	shortcut: 'Ctrl+Shift+D',    // optional; also lists in Settings > Shortcuts
	playMode: false,             // default: hidden in Play mode
	sidebar: false,              // default true: also a row in the burger menu's Modules
	                             // section. false = viewport menu only (see below)
	mount(el) {
		const label = document.createElement('div');
		label.className = 'tbx-label';        // the shell styles this for you
		label.textContent = 'Rooms';
		const go = document.createElement('button');
		go.className = 'tbx-primary';
		go.textContent = 'Generate';
		go.onclick = () => regenerate();
		el.append(label, go);
		return () => { /* your cleanup */ };
	}
});

Write plain DOM into the node mount receives and you inherit the app's own tool-palette treatment: header drag with position persistence, the width grip, z-band focus, the bottom SHEET at <=640px, and the whole .tbx-* CSS contract (.tbx-label, .tbx-row, .tbx-btn, .tbx-seg, .tbx-primary, .tbx-check, .tbx-danger) with no CSS of your own. That is the point — before this seam, module controls could only live behind registerMenu, two clicks deep inside the Modules modal, which then had to be closed before the module's own overlay was usable.

The user opens it from the sidebar's Modules section and the viewport menu (both from one builder, so they cannot drift), plus your shortcut if you name one. It starts CLOSED: a palette that appears uninvited is the thing registerMenu was avoiding. onOpen/onClose fire on each transition; the header ✕ closes it.

Where the way IN belongs is your call. The burger menu's Modules section is the app's permanent chrome, so a row there is a standing claim on it — right for a tool a user reaches for constantly, heavy for a window that belongs to one workflow. Pass sidebar: false to keep the viewport-menu row and drop the permanent one, and open it from where your module already is:

const id = api.registerToolbox({ id: 'manager', title: 'Collectibles', sidebar: false, mount });
// a button on YOUR card in the Modules manager, beside Update/Remove
api.registerMenu('Open Collectibles', () => api.openToolbox(id));

api.openToolbox(id) / closeToolbox(id) / toggleToolbox(id) take the id registerToolbox returned. openToolbox also dismisses the Modules manager when it is open, because the manager is the one piece of chrome that can cover a toolbox — a card button that opens a window behind the dialog it was clicked in is the complaint this whole seam exists to answer. It is a no-op when the manager is closed.

mount returns its cleanup and a re-registration re-runs it, so dev-mode live reload rebuilds the contents in place. Disabling or removing the module force-closes and unregisters the toolbox — it never leaves a window behind a dead mount fn.

A toolbox is LOCAL: it is this viewer's window, and nothing about it replicates or is saved with the scene. What it changes must still go through the replicated paths (api.send, api.create, api.physics.set).

HUD elements: rows, and your own element kind (21-E7)

// fill a HUD List element (kind `list`) by id
api.hud.rows('leaderboard', ['1. Ada 12', '2. Grace 9']);
api.hud.clearRows('leaderboard');       // back to the element's authored rows

// the roster WITH NICKNAMES - what a leaderboard actually needs
api.peerNames();  // [{id, name, label, me}]  `label` falls back to 'peer abcd'

api.hud.rows is one of three doors onto the same store: the element's own authored rows (typed into the properties pane, one per line), a HUD Rows flow node (set / append / clear on a trigger edge), and this. A node or a module always wins over the authored rows, and clearing puts them back — the authored value is the fallback, not a second source of truth.

CALL IT ON EVERY PEER. Rows are never sent. Like a value node, this writes LOCAL state that every peer is expected to compute identically, so calling it on one peer shows the rows to one person. Drive it from your own registerStateSync state, or from something every peer already derives (api.peerNames() is exactly that — the roster is already replicated). Your rows are cleared at teardown, so disabling the module shows the authored rows again rather than freezing the last thing you pushed.

// YOUR OWN ELEMENT KIND: the registerToolbox contract, one layer in
const kind = api.registerHudElement('gauge', {
	label: 'Fuel gauge',            // palette + properties-pane heading
	icon: 'gauge',                  // a lucide name
	summary: 'A dial that reads a wired number.',
	defaultSize: { w: 150, h: 40 },
	defaults: { caption: 'Fuel' },  // your params' starting values
	fields: [                       // the pane renders these; same schema as core kinds
		{ key: 'caption', kind: 'text', label: 'caption' },
		{ key: 'redline', kind: 'number', label: 'redline', min: 0, max: 100, step: 1 }
	],
	mount(container, el, runtime) {
		const span = document.createElement('span');
		span.textContent = el.caption;
		container.append(span);
		// return a bare cleanup fn, OR this pair to avoid a rebuild on every change:
		return {
			update(nextEl, nextRuntime) { span.textContent = nextEl.caption; },
			destroy() { /* your cleanup */ }
		};
	}
});
// kind === 'mod-<moduleId>-gauge'

You write plain DOM and inherit the whole HUD system: the layer's z-tier, the 9-grid anchoring with pixel offsets, the properties pane, the palette, replication, undo and all four save paths. runtime is what a flow node is currently driving into the element ({text, value, min, max, rows, options, pulse}), so a HUD Text or HUD Bar node pointed at your element feeds it with no extra wiring.

The kind name is namespaced and gets written into a replicated, saved document. A peer without your module — or the same scene opened after it is gone — reaches an unknown kind, which the HUD PRESERVES verbatim and skips at render. So nobody's layout is destroyed, installing the module makes the element appear, and a disable is the same story rather than a special case. Everything is unregistered and unmounted from the teardown journal.

Messages and state

api.onMessage((data) => {          // {type:'module', moduleId, ...payload}
	if (data.op === 'press') press(findByUuid(data.uuid));
});
api.send({ op: 'press', uuid });   // broadcast to all peers

api.registerStateSync({
	getState: () => ({ pressed: [...pressedUuids] }),   // sent to late joiners
	applyState: (state) => state.pressed.forEach(markPressed)
});

Input (K-C)

// declare bindings so they LIST in Settings ▸ Shortcuts (display-only —
// you read the keys yourself via input()/onInput)
api.registerBindings([{ label: 'Drive forward', keys: 'W' }]);

api.registerFrameTask(() => {
	const { codes, axes } = api.input();  // codes: Set<'KeyW'...> (event.code),
	if (codes.has('KeyW')) drive(1);      // axes: {lx,ly,rx,ry} = VR stick axes
});
api.onInput((kind, code) => {});        // 'down'/'up' events; returns unsubscribe

// pause the HOST's use of an input scope while your module drives:
//   'keys'       — WASD camera fly + play-mode movement
//   'locomotion' — VR left-stick locomotion
//   'sticks'     — (1.19) BOTH VR sticks: left-stick move, right-stick snap/smooth turn + teleport
//                  (a held thing reeled on Y and scaled on X, the way an Edit grab does)
api.claimInput('keys');                 // ALWAYS release when your mode ends
api.releaseInput('keys');
const sticksOk = api.claimInput('sticks') === true; // true = this core knows the scope (older: undefined)

// reading a keydown YOURSELF (a toolbox's own handler)? resolve the key the way the
// editor does — `event.key` when it is an ASCII letter/digit, the physical
// `event.code` position otherwise — so it works on a Cyrillic/Greek/Hebrew layout:
window.addEventListener('keydown', (e) => {
	if (api.keyOf(e) === 'G') grab();       // 'G' on QWERTY, AZERTY, Dvorak AND ЙЦУКЕН
	if (api.letterOf(e) === 'w') drive();   // lowercase form; named keys ('Escape') unchanged
});

Pointer ray (190)

// where the user is POINTING, as a THREE.Raycaster in WORLD space: the desktop
// mouse over the viewport, or the VR pointer hand's ray. Fresh instance per
// call; null before the first pointer event. The drag recipe: click to pick,
// follow pointerRay() in a frame task, click to drop (works desktop + VR).
api.registerFrameTask(() => {
	const ray = api.pointerRay();
	if (!ray || !carried) return;
	const hit = ray.ray.intersectPlane(dragPlane, tempVec);
	if (hit) carried.position.copy(hit);
});

Free-cursor games (1.17)

A board, puzzle or instrument game can play with the real mouse cursor and no pointer lock: publish userData.play.cursor = 'free' on your scene-root group (or the scene's physics block sets play.cursor: 'free'). api.pointerRay() is then the cursor's ray in play; in a locked game it is the CROSSHAIR ray (the view's centre), never the stale mouse position the lock pinned.

Storage (1.17)

// JSON on THIS device, namespaced to your module (tp:mod:<id>:<key>), 256 KB per
// module; never replicated, never saved into a scene, survives disable/remove.
if (api.storage) {
	const progress = api.storage.get('progress', { unlocked: 1 });
	api.storage.set('progress', progress);   // -> true, or false over the quota
	api.storage.keys(); api.storage.bytes(); api.storage.remove('progress');
	api.storage.clear();                     // your "Reset progress"
}

Feature-detect it; on an older app write the SAME key yourself (localStorage['tp:mod:<id>:<key>'] = JSON.stringify(value)) so progress carries over.

Models: api.loadModel (1.20, roadmap 34 R7)

Load a .glb / .gltf through core's OWN loader — the scene's THREE (never bundle three's GLTFLoader into a module: it drags in a second three whose classes are not the scene's), with Draco, Meshopt and KTX2 decoders (the Basis transcoder is fetched only for a file that carries KHR_texture_basisu), parsed ONCE per URL and shared by every module that asks.

// a file your module PACKAGED (list it in manifest.json `files`) — or any URL
const enemy = await api.loadModel('assets/enemy.glb', { castShadow: false });
group.add(enemy.scene);                       // your own copy (another module's tweaks never reach it)
const next = enemy.instance({ ownMaterials: true }); // another copy: its OWN bones when skinned,
group.add(next);                                     // its own materials (a hit flash paints one)
const mixer = new api.THREE.AnimationMixer(next);
mixer.clipAction(enemy.animations.find((c) => c.name === 'walk')).play();
enemy.info;            // {meshes, triangles, materials, textures, skinned}
enemy.release(next);   // forget one copy now (out of the scene, LOD dropped, own materials freed)
enemy.dispose();       // forget them all — your module's unload does this for you

Options (all optional): lod — absent / 'auto' = automatic levels (31-perf, meshes over 3000 triangles; a URL that is a PACK item whose row lists lods uses those files instead), false = none, {ratios, distances, minTriangles} = tuned automatic levels, [{file, ratio}] = pre-built level FILES beside the model (contract P1: 'tree.lod1.glb' next to 'assets/tree.glb'), drawn as a LOD group wherever you put the copy. castShadow / receiveShadow = every mesh (absent = as the file says, which is off). collider: 'box'|'sphere'|'capsule'|'cylinder'|'cone'|'hull' = the physics shape a copy takes once it is SCENE content with physics (userData.colliderHint; your scene-root visuals have no colliders — publish a userData.play raster for walls). ownMaterials = every copy gets its own materials.

Lifetime: everything a module loaded is released when it is disabled or unloaded (a copy you put in objectsGroup stays — it is the scene's). A copy you drop WITHOUT release() (its wrapper removed) is noticed within ~2 s, its LOD entry dropped and only weakly held, so it is collected; put back, it is picked up again. A load still in flight when the module unloads rejects. Rejects with the reason when the file is missing or cannot be parsed — keep a fallback look. Feature-detect it (typeof api.loadModel === 'function'): 1.19 and older cores do not have it.

Performance: quality level, LOD, budgets (1.18, roadmap 31)

// The adaptive quality level on THIS device: 0 = best ... api.quality.max (every step taken).
// A headset session starts at 1 (shadows off) in auto mode; a game's Quality setting can pin it.
if (api.quality) {
	const apply = (level) => {
		particles.maxCount = level >= 3 ? 40 : 200;   // cut YOUR effects, not shared state
		torchLights.forEach((l, i) => (l.visible = i < (level >= 1 ? 2 : 6)));
	};
	apply(api.quality.level);
	const off = api.quality.onChange((level, { labels, vr }) => apply(level)); // released on disable
}

// Levels of detail for your own dense geometry (auto LOD already covers dense meshes in the
// shared scene and under the module world root). Built ONCE per asset in a worker
// (meshoptimizer), drawn by distance through a render-time swap: the mesh keeps its full
// geometry for picking, physics and your code. Distances are in the mesh's world RADII.
if (api.lod) {
	const lod = api.lod(enemyFigure, { ratios: [0.5, 0.2], distances: [6, 18] });
	lod.ready.then((n) => console.log(n, "meshes have levels")); // lod.remove() undoes it
	lod.force?.(2);     // 1.19: draw level 2 on THIS screen whatever the distance (null = auto)
	lod.levels?.();     // 1.19: the level each mesh drew last (0 = full)
}
mesh.userData.lod = false; // keep one mesh out of auto LOD

1.19 — LOD groups on scene objects. A replicated object may carry a LOD GROUP (userData.lod, edited in its Properties ▸ LOD section, or written by a pack item's lods): {mode: 'auto'|'forced', forced?, bias?, cull?, levels: [{source: 'self'|'pack'|'generated'| 'explorer'|'object', ref?, ratio?, screenSize, offset?, material?}]} with thresholds by SCREEN SIZE (the share of the viewport height the object covers). It is scene data (saved, replicated, undone) and drawn at render time like api.lod — the tree never holds a level. A module that builds scene objects can write one through the same path the panel uses (objectParameters {parameter: 'lod'} is the wire shape); module-only geometry keeps api.lod.

Both are LOCAL (a fact about this machine) — never let them change replicated state, or two peers on different hardware disagree about the game. Skinned meshes and morph targets get no levels (the simplifier cannot carry weights): pre-decimate those offline.

The Quest budget (roadmap 31, the planner's target for a scene in Interact/Play on a Quest 3): <= 150 draw calls, <= 300k triangles visible, <= 2 real-time lights with shadows off, no per-frame allocations in the hot path. Measure your game with the probe, the same way every time:

# dev server running; run it under the exclusive slot (frame times are the point)
APP_URL=https://theprototype.app:5173/ ~/.local/bin/e2e-slot --exclusive -- \
  node scripts/perf-games.cjs --label mine --only waves [--vr] [--profile]

It prints draw calls + triangles per frame (every render pass summed), geometries, textures and an MB estimate, lights (+ shadow casters), meshes / instanced / unculled, the LOD and quality state, and p50/p95/p99 frame ms at CPU throttle x4; --profile adds CPU self-time, the nearest app frame calling the heaviest functions, and allocation per function (garbage included). The things 1.18's round found, in the order they cost:

  • Never keep a THREE object inside Svelte $state (or pass one through an action parameter that is a state proxy): Svelte deep-reads a proxy, which walks the whole scene graph through parent — typed arrays included. Use $state.raw. This one line was 88% of a game's frame.
  • No whole-scene lookups per frame — scene.getObjectByName is a traversal; find once, keep the reference, re-find when it leaves the scene.
  • No allocation per frame: reuse vectors/arrays; a [a, b].map(...) in a frame task is garbage 60-90 times a second, and GC pauses are the stutter you feel in a headset.
  • Lights and shadow casters are draw calls: each shadow-casting light draws every caster again; transmission (glass) renders the scene an extra time per camera, per eye.

Functional pack items: doors, lids, levers — api.behavior (1.19, roadmap 33)

A pack item can be FUNCTIONAL: a door that opens on a click, a chest lid, a lever, a fan (the behavior field, see PACKS.md). Core runs them. Nothing plays in Edit; a click, a knock or walking up triggers them in Interact/Play; the state replicates; a door's collider follows its leaf. A module can drive them too:

if (api.behavior) {
	for (const item of api.behavior.list()) {        // [{uuid, type, trigger, open}]
		if (item.type === 'door' && !item.open) api.behavior.trigger(item.uuid, true); // open it
	}
	api.behavior.state(uuid);                          // {on, at, n} or null (never triggered)
	api.behavior.trigger(uuid);                         // toggle, like a player's click
}

trigger is REPLICATED (one behavior message; every peer poses the door from the same session-clock stamp), so call it on ONE peer: the authority, or the peer that saw the cause. It returns false when nothing changed (opening an open door). The state is runtime: it is never saved, and a peer in Edit shows the rest pose whatever it says. The item sounds are core game sounds: door, gate, slide, lever, lid (plus click), which api.playSound can play too.

Game feel: sound, music, haptics, effects, banners (1.17, roadmap 30b)

Everything here is LOCAL to the device it runs on — broadcast your own op (api.send) when every peer should hear, feel or see it — and all of it is feature-detected, so a module written against it still loads on an older app.

register(api) {
	api.registerClickHandler((object, ctx) => {
		if (!object.userData.ring) return false;
		const at = object.getWorldPosition(new api.THREE.Vector3()).toArray();
		api.playSound?.('success', at);
		api.effects?.burst(at, { kind: 'sparkle' });
		api.announce?.('Ring ' + object.userData.ring + ' reached', { sub: '+50' });
		api.hapticPattern?.('success');                // both hands; 'left' | 'right' for one
		return true;
	});
	api.game.onChange?.(() => (api.game.roundUnderway() ? api.music?.play('arcade') : api.music?.stop()));
}
  • api.playSound(name, position?) → true when a sound started. Twenty procedural game sounds, no assets: click pop whoosh success fail hit kick shoot laser explosion coin levelup goal whistle cheer step ring sparkle hurt portal, plus the ping chimes ding (the default), chime, pluck, bell. An unknown name is a quiet no-op (it used to play the ding). position spatialises it. Played at the player's "Game sounds" volume.

  • api.music.play(preset, {volume?}) / api.music.stop() — a looping procedural track under the effects, at the player's "Music" volume: arcade ambient dungeon stadium space puzzle studio. It is TEMPO-SYNCED to the session clock (two peers on one preset hear the same bar), plays only in Interact or Play (false from the editor) and stops by itself when the player goes back to editing. api.music.current() names what is playing; api.music.presets() lists them.

  • api.hapticPattern(name, hand?) — tap bump hit success fail rumble heartbeat on the VR controllers. api.haptic(intensity, ms, hand?) still exists. BOTH are silent in Edit mode (vibration is for playing), and core already plays the defaults for you in Interact/Play: a tap when the laser enters something clickable, a bump when a press lands, a hit on a grab, and a knock sized by how hard the hand hit.

  • api.effects.burst([x, y, z], {kind?, color?, count?}) — a short, pooled particle burst at the scene root: sparkle (default), confetti, smoke, sparks; count 1..96. Nothing is saved or replicated.

  • api.announce(text, {sub?, ms?, color?}) — a big centred banner ("GOAL!", "Level 3"): the desktop HUD, and in a headset a banner fixed in front of the player. A new one replaces the one showing; ms 300..15000 (default 1800).

  • api.setSpawn([x, y, z], yaw, {teleport?}) — where the player starts: y is the FEET, yaw is three's rotation.y (0 faces −Z). Entering Interact or Play puts the player there (a headset's rig lands its feet on it, a desktop its camera); without teleport it is a checkpoint and nobody moves until then. A module's spawn overrides the scene's play.spawn. api.respawnPlayer() sends the player back to it now.

Moving around in a game: locomotion, bounds, VR panels (1.18, roadmap 31)

In Interact and Play a VR player WALKS (collisions, gravity, a 0.3 m step, snap turn) and nothing else unless the scene's play block — or your module's userData.play on its scene-root group, field by field — allows more:

// feature-detect: an older core has no api.locomotion and would ignore the bounds
if (api.locomotion?.boundedTeleport) {
	group.userData.play = {
		locomotion: { teleport: true, worldGrab: false, fly: false },
		bounds: { min: [-12, -0.5, -12], max: [12, 0.4, 12] }, // in THIS group's local frame
		colliders: [{ min: [2, 0, -6], max: [2.3, 3, 6] }] // optional: solid boxes, same frame
	};
}
  • locomotion.teleport: true — the right stick's arc teleport, BOUNDED: the arc stops at the first surface it meets; the landing must face up (normal y ≥ 0.7) or be the floor plane, must lie inside bounds, and the straight line 1.1 m above your feet and the landing must not cross a collider box, a mesh, or (with a dungeon raster published) a wall cell. A refused landing draws the arc RED and releasing the stick does nothing; a valid one is green and puts your FEET on it. Without bounds the play area is the scene's content box pulled in 0.3 m. Edit mode's teleport is unchanged (it lands anywhere).
  • locomotion.worldGrab: true — the grips move, rotate and SCALE the world the way they do in Edit (two grips scale/rotate, the right grip alone pans) whenever a grip does NOT start on something the player may hold (a grip on a dynamic body under interaction: 'grab' still takes the body). For board games and instruments: Untangle, the Jam Room.
  • locomotion.fly: true — the left stick flies along the controller's aim, no gravity.
  • bounds: {min: [x, y, z], max: [x, y, z]} — a sibling of locomotion. The scene's play block gives it in scene coordinates; a module's userData.play.bounds is read in that group's local frame. Keep max.y just above the floor if box tops and ledges must not be landing spots.
  • vrControls.teleportVerdict(from, to, normalY?) (on the debug hook window.__stores.vrControls) answers {ok, reason} — ok steep outside off-floor wall-cell blocked — for your e2e; from/to are world FEET points [x, y, z]. vrControls.teleportPreview() reads the live arc ({engaged, bounded, valid, reason, target}).

Your own VR menu is a panel. A board, a level bar or buttons you draw in THREE for the headset should be registered with api.vrPanel?.(group) (returns the undo, also run when the module is disabled): it is then drawn OVER the scene — a floor, a base or a wall between the player and it can never hide its buttons — and the controller laser ends on it with its dot, even through whatever stands in front. Hit testing is unchanged. Core's own panels (the radial menu, the game board, the wrist card) already work this way, and the laser ends on your registerInteractiveGroup content too.

In VR, your game's HUD is in the player's hands. A screen with input: 'menu', a control on it, or bound to the menu / paused / over game state is drawn on a board ~1.2 m in front of the player that follows their head lazily; its buttons work with the laser and trigger and with a poke. The other screens (score, timer, level) read as short lines on a card on the left wrist and a strip across the top of the view. The board and the wrist card always carry an Edit mode button. Nothing to do on your side — author the HUD once and it works on both.

Hold the trigger and sweep (VR, Interact/Play). While the trigger is held, the controller TIP clicks each clickable it passes over — the one it is NEAREST to, so a glide along a keyboard plays one key at a time — once, and again after it leaves and comes back: piano keys, pads, toggles, HUD buttons; the laser does the same for things out of reach. "Clickable" (also what earns the hover tap) means: under a group you passed to registerInteractiveGroup, a part of an audio device (userData.device on an ancestor — each key/pad is its own control), an object an On Click node targets, or an object you mark userData.clickable = true. Your handler gets a second argument ctx = {source, mode}: source is 'click' (desktop), 'trigger' (the VR press itself) or 'sweep' (a later entry while held). A control that must not be swept (a knob you drag, a dot you carry) opts out:

api.registerClickHandler(pickDot, { sweep: false }); // presses still reach it; sweeps do not

The game shell: pause menu, levels, per-game settings (1.18, roadmap 31 K3)

Every game gets ONE pause menu from core — Resume · Restart · Levels · Settings · How to play · Main menu — on the desktop (Escape in Play, or the corner Menu button) and in VR (the left controller's X, or Menu on the game board / wrist card). You do not draw it; you feed it. A scene counts as a game when it has a state-bound HUD screen, a spawn, or a module publishing userData.play — or when your module registers levels. All of it is feature-detected (api.game.levels?.(…)), LOCAL, and torn down with your module.

Levels — the picker on the desktop and on the VR board. Re-call it whenever a level unlocks or earns stars; onPick never hears a locked level.

const refresh = () =>
	api.game.levels?.({
		list: LEVELS.map((l, i) => ({ id: l.id, label: 'Level ' + (i + 1), locked: i > unlocked, stars: best[l.id] ?? 0 })),
		current: currentLevel.id,
		onPick: (id) => loadLevel(id)
	});
refresh();
onLevelWon(() => { unlocked++; refresh(); });

Your own settings rows — shown under the core rows, persisted per game on this device (tp:game:<game>:<id>). type is 'toggle', 'choice' (options, optional optionLabels) or 'range' (min, max, step). A core id is refused.

api.game.addSetting?.({
	id: 'board', label: 'Board', type: 'choice',
	options: ['globe', '2d'], optionLabels: ['Globe', '2D board'], default: 'globe',
	onChange: (v) => setBoard(v)
});
setBoard(api.game.setting?.('board') ?? 'globe');            // the stored choice at load
// (1.19) a CHOICE that decides which levels you see: `onLevels: true` also draws it as tabs
// above the Levels page's grid (desktop + headset) — Untangle's Board [Globe] [2D board]
api.game.setSetting?.('board', '2d');                        // your in-game button writes the same row
api.game.onSettingsChange?.((values) => console.log(values.board, values.sfx));

The core rows every game has — music, musicVolume (0..100), sfx, sfxVolume, haptics, showFps, turning (default | snap | smooth | off), turnAngle (default | 15 | 30 | 45 | 90), vignette, quality (auto | low | medium | high). Core obeys them itself: the per-game volumes land on the audio BUSES (so api.playSound, api.music, flow sound nodes and the scene's track all follow), haptics, VR turning, the comfort vignette and the quality governor read them live. A module that plays audio through its OWN WebAudio graph should read them:

const gain = ctx.createGain();
const apply = () => (gain.gain.value = api.game.setting?.('sfx') === false ? 0 : (api.game.setting?.('sfxVolume') ?? 100) / 100);
apply();
api.game.onSettingsChange?.(apply);

How to play — your words first, then the device's controls (keyboard or controllers). Without it the menu shows the game's Games-tab description.

api.game.setHelp?.([
	'Drag the dots until no two lines cross.',
	'Finish a level to unlock the next — stars for fewer moves.'
]);

Restart — the menu resets the game shell (host / alone; others carry on as it is), respawns the player and runs your hook:

api.game.onRestart?.(() => { resetBoard(); spawnWave(1); });

Your own Menu button (a module HUD, a VR board of yours):

api.game.openMenu?.();          // only while playing a game; false otherwise
api.game.closeMenu?.();
if (api.game.menuOpen?.()) pauseMyTimers();

The game kit: rules, round, levels, score, pickups — api.kit (1.20, roadmap 34)

The logic every game used to re-build, done ONCE in core. Each piece is api.kit.<piece> AND a Kit: node group, both generated from one spec (src/lib/kit/<piece>.spec.js), so a graph author and a code author get the same behaviour and the same words.

The model — one writer. The kit's shared state is ONE document that only the authority peer changes (the physics initiator, else the lowest peer id — the peer Towers and Football already pick). Call an action on ANY peer: elsewhere it is a request the authority applies, exactly once. A kit node on a replicated pulse is asked by every peer with the same id, so one press is one change (the setvariable add double count cannot happen). Reads are replicated values; on<Event>(fn) fires on EVERY peer (play your sound there) and returns off; registrations (levels.define, pickups.register, rules.onGrabRequest) are torn down with your module. Late joiners get the document in the handshake and hear no history. A scene clear resets the kit.

const { round, levels, score, pickups, rules } = api.kit;
rules.set({ reach: 1.3, jump: 1.0 });                 // every grab path + the VR jump obey
rules.onGrabRequest((req) => { if (req.name === 'Star' && !starFree) req.refuse('Build to the ring first'); });
levels.define({ id: 'mygame', list: [{ id: '1', label: 'Easy', par: { time: 60 } }, { id: '2', label: 'Hard' }] });
round.configure(3, 120, 'lose', 2);                   // 3 s intro, 2 min limit, lose on time, 2 s outro
round.onGo(() => api.announce('Go!'));
pickups.register({ id: gem.uuid, score: 10, respawn: 8, grants: { time: 5 } });
pickups.onCollected(({ by }) => api.playSound('coin'));
round.onWon(() => levels.complete(true, score.total()));   // stars, unlocks, saved per device
levels.select('1'); round.start();
piece actions reads events
rules setReach(m) setJump(m) setBounds(min, max) clearRules() · set({reach, jump, bounds}) reach() jump() current() inside(p) clamp(p) checkGrab(req) refused (local) · onGrabRequest(fn) veto
round configure(intro, limit, 'lose'|'win', outro) start() restart() pause() resume() win(reason) lose(reason) extend(s) toMenu() phase() playing() elapsed() remaining() countdown() number() outcome() state() running() started go paused resumed won lost results menu
levels select(id) next() complete(won, score, time, level?, detail?) setMode(m) · define({id, list, unlock?, stars?, store?, merge?}) current() currentLabel() index() starsOf(id) unlocked(id) totalStars() mode() table() progress() resumeLevel() selected completed unlockedNext
score add(n, player?) set(n, player?) reset() · configure({autoReset}) useGame(id) total() mine() best() leader() of(id) leaderboard(n) results() scored newBest (local)
pickups collect(id, score?, respawn?) resetPickups() · register({id, score, respawn, radius, grants}) available(id) taken() left() takenBy(id) collected respawned allCollected
  • round drives core's game singleton (intro/playing → playing with a fresh core round, so perRound content resets; paused; won/lost → over with the reason as outcome; menu), so HUD showWhile screens and the pause menu keep working, and it ADOPTS a change it did not make (a Set Game State node pausing). restart() works while playing; the pause menu's Restart restarts a kit round by itself.
  • levels: progress is per DEVICE (every peer saves what it saw earned); unlocks read it merged with this session's results; setMode keeps the current level (progress belongs to the level, not the mode). The table feeds the pause menu's level picker — do not also call api.game.levels. store: {get, set} keeps your own save key (Towers keeps tp:mod:towers:progress).
  • score credits the asking peer unless a player is named (a shared pulse is asked by every peer — name the player, or use a per-player trigger, when it matters who). A new kit round zeroes it; the device best is checked when a round ends.
  • pickups: TOUCH is built in — a player walking into a registered pickup takes it (their own body, ~10x a second, in Interact/Play). A Kit: Pickups ▸ Collect pickup node with no object wired takes its own graph's object.
  • Prove your rules on the headless logic sim (tests/unit/sim/logicSim.js: N fake peers, a fake clock, the real wire validator) — createSim({peers: ['a', 'b']}), then sim.peer('a').kit.impls.round.start(); sim.advance(3000) — milliseconds, not e2e minutes.

Behaviours: game logic as a small file, with a live node view (1.20, roadmap 34 R3)

A behaviour is game logic written as ONE self-contained JavaScript file — the code half of "nodes vs code" (proposal §4): plain JS for what is a loop or a rule, a derived node view for seeing it run. Add a Behaviour node (palette: Logic) to the scene graph (or an object's graph — then this.object is that object) and write the file into it; Open view shows it.

export default behaviour({
	name: 'Waves spawner',
	params: { interval: { value: 3, min: 0, max: 30, step: 0.5, unit: 's' }, waves: 5 },
	state: { wave: 0, alive: 0 },                      // replicated: every peer reads it
	on: {
		go() { this.startWave(1); },                   // kit.round's "go"
		died({ entity }) {                             // kit.health's "died", typed payload
			if (!entity.is('robot')) return;
			if (--this.state.alive === 0) this.after(this.params.interval, 'startWave', this.state.wave + 1);
		}
	},
	startWave(n) {
		if (n > this.params.waves) return kit.round.win('All waves cleared');
		this.state.wave = n;
		this.state.alive = n + 1;
		kit.spawner.spawn({ kind: 'robot', template: this.find('Robot')?.uuid, count: n + 1, mover: { speed: 1.3 } });
	}
});
  • params — a literal (waves: 5) or {value, min, max, step, unit}; read as this.params.x. In the node view each is a KNOB: dragging previews on your device, releasing rewrites the literal in the file (one edit, one undo entry) and every peer reloads with it.
  • state — plain JSON. Handlers run on ONE peer, the kit's authority; after each one the state is sent to everybody (bhv, latest-wins). A late joiner gets it in the handshake; when the authority leaves, the next one carries on from it. A source edit keeps it.
  • on — event handlers: start (once per session), grabRequest({piece, distance, refuse}) (the kit.rules veto, asked on the GRABBING peer — read-only there), and every kit event as 'piece.event', pieceEvent or a short alias (roundStart, go, won, lost, died, damaged, spawned, emptied, scored, collected, stuck, grabRefused). An entity in a payload has is(glob) / hasTag(tag).
  • methods — any other function: this.name(...).
  • this — params, state, kit (also bare kit), after(s, 'method', ...args) (session clock; a method NAME survives the authority leaving, a closure does not), cancel(key), rand() / randInt(a, b) / pick(list) (seeded per event — never Math.random), now(), find(glob) / findAll(glob) (scene objects by name or tag: {uuid, name, pos, tags}), object, isAuthority(), me(), log(...). Bare helpers: dist(a, b), clamp, lerp.
  • The lint stops a file from loading (on every peer, with the line) when it uses what would make peers disagree or reach outside the game: Math.random, Date/Date.now/ performance.now, storage, the DOM/window/globalThis, network, eval/Function, bare timers, import. Loops run under the loop guard (a runaway handler throws, the game goes on).
  • Lifecycle (T2) — each behaviour is a module of its own (behaviour:<nodeId>): its kit listeners and the entities it spawned are tracked; deleting or stopping the node takes them away. Editing the source does NOT: the definition is swapped, the game keeps running.
  • The node view (derived, read-only): ⚡ events → ƒ handlers/methods → ▣ state, ⊕ kit calls (named exactly as the generated Kit: nodes), ⏱ timers, ✋ payload actions; ◆ params with knobs. Live values 10x a second, a glow on what just fired (on every peer), timer countdowns, errors with their line. Logic edits happen in the code (the view's Code panel).
  • Prove a behaviour on the logic sim: tests/unit/sim/behaviourSim.js — const sim = createBehaviourSim({peers: ['a', 'b']}); await sim.load('w', source); then drive the kit and assert sim.states('w'). Examples: static/behaviours/waves-spawner.js, static/behaviours/towers-reach.js.

Water: api.water (1.22, roadmap 36)

if (api.water) {
	// a tank in the shared scene (replicated like Create -> Water; kinds tank | pool | ocean | cylinder)
	const tank = await api.water.create({ kind: 'tank', preset: 'aquarium', size: [3, 1.5, 1.2], at: [0, 0.75, 0] });
	api.water.configure(tank, { look: { deepColor: '#0b3d4f' }, bubbles: { enabled: true, rate: 12 } });
	api.water.preset(tank, 'toxic');              // lava, swamp, toxic, ice, ocean, lake, river, pool, aquarium
	const hit = api.water.query([0, 0.5, 0]);       // {uuid, depth, surfaceY, flow} | null (waves included)
	api.water.disturb(tank, [0.2, 1.5, 0], 0.4, 0.8); // a LOCAL ripple — call it on every peer
	const off = api.water.onChange((volumes) => {}); // appear / edit / move / remove; off() or module unload
}

An object is water when userData.water holds a water blob (contract W1, src/lib/water/volumes.js): {version: 1, shape: 'box'|'plane'|'cylinder', level (local Y; null = top), density, linearDrag, angularDrag, flow: [x,y,z] (local m/s), preset, look, waves, bubbles}. configure merges a patch (look/waves/bubbles one level deep) and is replicated and undoable like an Inspector edit; on a dry object it takes a whole blob. query is deterministic (the waves run on the shared clock), so every peer's buoyancy agrees. disturb is visual only and NOT sent — drive it from shared physics on every peer. burst(uuid) fires the bubble burst for everyone (the moment rides the blob). Feature-detect it (api.water): 1.21 and older cores do not have it.

Physics (P-A)

All mutations are INITIATOR-ONLY — the peer that started the simulation steps the world (golden rule: authoritative, never mixed with deterministic). The blessed recipe for driven physics (pong's paddle pattern): every peer forwards its INPUT via api.send({op:'drive', ...}) at ~20Hz, and only the peer where api.physics.isInitiator() is true applies it.

api.physics.isInitiator();              // true while THIS peer runs the sim
api.physics.applyImpulse(uuid, [0, 5, 0]); // push a dynamic body (initiator-only)
api.physics.applyTorqueImpulse(uuid, [0, 2, 0]); // spin a dynamic body (world axes)
api.physics.setJointMotor(jointId, vel, maxForce); // drive a revolute joint
api.physics.joints();                   // Promise<the replicated joint defs>

The car module (now in the modules repo, modules/car/) is the worked example: replicated primitives + motorized revolute joints, click-to-claim (pong's paddle pattern), driver forwards {op:'drive', throttle, steer} at ~20Hz and only the initiator applies wheel motors. Driving + the chase camera engage only in Play mode with a running simulation (the module claims 'keys' and uses possess's startFollowCam while engaged; the claim itself works anytime).

Building in the shared scene (17-A)

// the replicated /create, handing back what appeared
const [uuid] = await api.create('/create Box 1 1 1');
await api.create('/create Box 0.6 0.6 0.6', { at: [x, y, z] });
api.moveObject(uuid, { pos, rot, scale });        // the editor's replicated move
api.physics.set(uuid, { mode: 'dynamic', mass: 30 });
api.physics.createJoint('revolute', a, b, 'x', { vel: 0, maxForce: 120 });
api.physics.running();   // a sim runs somewhere in the session
api.isPlaying();         // Play mode active
api.peerIds();           // roster - free a departed peer's state
// LOCAL only (a peer's module must never yank your viewpoint):
api.flyTo([x, y, z], [lx, ly, lz]);
api.playSound('pluck', [x, y, z]);
api.followCam(uuid); api.stopFollowCam();

Possess (K-D)

// drive any object with WASD/arrows or the VR left stick (tank controls),
// chase camera by default; Esc releases. Possessing SELECTS the object
// (selection = lock), suspends its flow effects, and records ONE undo entry
// on release. Movement replicates as plain throttled moves.
api.possess(api.selectedUuid(), { camera: 'chase' }); // 'chase'|'orbit'|'none'|'first'
api.releasePossess();
api.possessModes; // this build's camera modes — feature-detect 'first' here
                  // (an unknown camera value degrades silently on old builds)

// 17-A1 first person: eye at the object + eyeHeight; mouseLook = pointer lock
// (X turns the OBJECT so movement follows the look, Y pitches the camera;
// leaving the lock — Esc — releases the possession)
api.possess(uuid, { camera: 'first', eyeHeight: 1.7, mouseLook: true });

Audio devices (23-A5)

An instrument, an effect or a speaker is a DEVICE: an object carrying userData.device = {kind, params}, with a WebAudio subgraph the engine builds for it. Your module supplies the kind; core supplies the object, the replication, undo, saving, the cables and the clock.

const kind = await api.registerAudioDevice({
	kind: 'piano',                     // namespaced to mod-<moduleId>-piano
	label: 'Piano', icon: '🎹', group: 'Keys',
	ports: { in: [], out: [{ id: 'out', kind: 'audio' }] },   // 'audio' | 'cv' | 'midi'
	params: [{ key: 'level', kind: 'range', min: 0, max: 1, step: 0.01, default: 0.8 }],
	build(ctx, node, params) {         // ctx = the SHARED AudioContext; runs per object
		const out = ctx.createGain();
		out.gain.value = params.level;
		return { output: out, out, dispose() { out.disconnect(); } };
	},
	onParam(h, key, value) { if (key === 'level') h.out.gain.value = value; },
	onNote(h, { note, velocity, at }) {              // at = a WALL-CLOCK stamp
		const v = api.audio.voice({ freq: 440 * 2 ** ((note - 69) / 12), gain: velocity * 0.5, destination: h.out });
		const t = api.audio.timeFor(at);              // never Date.now() maths of your own
		v.start(t); v.stop(t + 0.5);
	},
	mesh: (THREE) => new THREE.Mesh(new THREE.BoxGeometry(1, 0.2, 0.4), new THREE.MeshStandardMaterial({ color: '#222' }))
});

api.audio.addDevice('piano', { position: [0, 1, 0] }); // also in the viewport Add menu > Devices
  • params render in the Inspector and the toolbox; every write replicates and undoes. Keep onParam cheap and build pure: a peer WITHOUT your module holds the same object as an inert placeholder with its document intact, and rebuilds it the moment your module registers.
  • Children of your mesh named port:<id> are where cables attach.
  • Sound goes NOWHERE until cabled: a speaker device connects its input to api.audio.bus('instruments'); api.audio.cable({from: {uuid, port}, to: {uuid, port}}) plugs one in.
api.audio.voice({ freq: 220, type: 'sawtooth', filter: { freq: 900 } }); // {output, start(t), stop(t), dispose()}
api.audio.voice({ buffer, loop: true, destination: 'instruments' });      // a sample voice
await api.audio.sample(hash);          // decoded AudioBuffer for an Explorer content hash,
                                       // pulled from a peer if missing (null after 30 s)
api.audio.bus('sfx');                  // a bus to connect to
api.audio.context();                   // the shared AudioContext — never make your own
api.audio.transport();                 // {bpm, beat, bar, step, phase, playing, loopBeats, swing}
api.audio.play(); api.audio.play(false); api.audio.setBpm(100); // the SHARED transport
const cancel = api.audio.schedule(0, ({ beat, at }) => hit(at), { every: 1 }); // a metronome:
                                       // called ~100 ms EARLY with the exact audio time;
                                       // start voices at `at`; cancelled at teardown
api.audio.note(uuid, { note: 64, velocity: 0.8 }); // replicated; every peer synthesizes it
api.audio.setParams(uuid, { level: 0.5 });          // one undo step, replicated
const before = api.audio.device(uuid);              // capture the document when a gesture STARTS
api.audio.previewParams(uuid, { level: 0.5 });      // a live gesture: replicated, NO undo entry -
                                       // throttle it while scrubbing, then commit once:
api.audio.setParams(uuid, { level: 0.5 }, { before }); // one undo entry that restores `before`
api.audio.captureMic();                // Promise<MediaStream>: the RAW mic, a separate capture
                                       // from voice chat (no AEC/NS/AGC, never gated by PTT)
const item = await api.audio.record({ maxSeconds: 8 }); // a take -> Explorer item (content-hashed,
                                       // shared to peers); null when refused BEFORE it starts
api.audio.stopRecording();             // end the take early
const dest = api.audio.context().createMediaStreamDestination(); // bounce a NODE instead:
myInputGain.connect(dest);             // a looper records what is patched into it
await api.audio.record({ maxSeconds: 8, stream: dest.stream, name: 'loop' });
// declare what a kind references by hash so the Scene manifest (and a .tpscene export)
// carries the bytes: `assets(params) -> [{hash, name}]` on the registerAudioDevice spec
api.audio.device(uuid);                             // {kind, params} or null

The rule behind schedule and onNote: anything scheduled on the transport must be a pure function of its arguments. Every peer runs the same pattern from the same startedAt; an impure callback desyncs silently, per peer, with no error anywhere.

Misc

api.registerMenu('Generate dungeon', () => generate()); // sidebar "Modules" section
api.registerVRMenuEntry({                               // sector in the VR radial menu
	id: 'spawn', group: 'root', label: 'Dungeon',         // group 'root' = base ring;
	action: () => generate(), closes: true                // other names make a sub-ring
});                                                     // (id gets prefixed moduleId:)
api.sceneAssets();  // Promise<[{group, name, kind, hash}]> — what the shared
                    // scene uses right now (audio/config/textures manifest)
api.scene();        // THREE.Scene
api.objectsGroup(); // the replicated objects root (add scene content here)
api.peerId();       // our peer id (undefined before the mesh is up)
api.toast('hi');    // toast in the corner
api.haptic(0.6, 60);          // buzz the VR controllers (no-op on desktop and in Edit);
api.haptic(0.6, 60, 'right'); // optional hand targets one controller (17-A1)
api.hapticPattern('success'); // a named pattern (1.17, see Game feel)
api.isVR();                   // true inside a VR session
api.vrHand('left');           // one hand's WORLD pose + buttons, or null:
                              // {position, quaternion, trigger, gripped, connected}
api.fireObjectClick(uuid);    // pulse On Click flow nodes targeting the object
                              // (replicated) — user graphs react to module events
api.now();          // the runtime clock in seconds — stamp replicated
                    // timestamps with this, never Date.now() directly

Objects your module adds to objectsGroup() are part of the shared scene: they get GLTF-synced to late joiners, appear in the object list, and can be moved/deleted by anyone. Objects that are derived state (e.g. a generated dungeon) are better added outside objectsGroup() and rebuilt from your module state — then they can't drift.

Walkthrough: the hello module

src/modules/hello/module.js is the smallest complete module: one node group

  • one effect, zero messages (fully deterministic). Open the flow editor, drag in Wave (hello) and an Object Selector, pick an object, connect them — the object rocks on every peer.

Walkthrough: build a door (button module)

The button module (src/modules/button/) shows the interactive pattern:

  1. Add a Button from the sidebar Modules group (or use any object).
  2. Add a Wall/Cube where the door should be.
  3. Open the flow editor: drag in Button trigger and an Object Selector.
  4. In the Button trigger node pick the button object; in the Object Selector pick the door object; connect trigger → selector.
  5. Click the button in the viewport (or pull the VR trigger on it): the door slides up by height on every peer. Toggle mode keeps it open, push mode springs back after 1.5 s.

How it replicates: the click writes {pressed, at: api.now()} into the node's data (replicated like any node edit); the slide is a pure function of (data, time) running on each peer — no motion messages at all. The registerClickHandler consumes the click so pressing doesn't select the button.

Walkthrough: the towers module (a game's rules as a core module)

src/modules/towers/ runs the Towers template's twelve levels. It shows how a game with real rules splits: the scene (the template) is the arena, the piece templates, the HUD and a small graph; the module is the rules. Worth copying:

  • The rules as a pure leaf (levels.js) — the level table, the star rule, the unlocks, the verdict stepper — tested with vitest, no browser.
  • One authority decides anything shared (the physics initiator, else the lowest peer id): it deals the pieces, judges the round and writes the result into game variables (api.game.setVar); every peer derives its HUD words from those through a value node wired into HUD Text's format (api.registerValueNode).
  • HUD buttons without a presser: a press is a replicated stamp on a hudbutton node, so every peer watches api.flow.triggerStamp and only the authority acts (first sight = history).
  • Progress on this device with api.storage (stars, unlocks — never replicated).
  • The shell's level picker, help and restart (api.game.levels, setHelp, onRestart), feature-detected so the module also runs on an app without them.
  • Since 1.20 it runs on the game kit (api.kit): its levels, unlock chain, stars and saved progress are kit.levels (with Towers' own store and star rule), its round is kit.round (Restart while playing), its tallest tower is the round's kit.score, and its reach + jump are kit.rules — what stays in the module is Towers itself.

Manager, dev mode & gallery (17-A2/A3)

Modules install, update, disable and remove live — unloadModule (33's deactivateModule) runs the module's lifecycle registry (see The lifecycle above), so nothing needs a page reload. Since 1.20 that includes CORE modules: their hooks outside the api (vrsleeve's VR hook registries) undo through api.onUnload.

Every user-module card carries a Dev URL row: Reload fetches fresh code (cache-busted), evaluates it FIRST, then tears down + re-registers — a broken body keeps the old instance running. Auto polls (~2s) and reloads on change. Peers toast the {id,version} mismatch while you iterate (correct — the dev peer differs). The Browse tab lists the community repo (theprototype-app/modules, index.json via jsDelivr — moduleGallery.js); installs go through installUrl with the entry's source folder, so Update and the dev reload keep working on gallery installs.

Checklist before you ship one

  • Everything a user can do through your module looks the same on a second connected browser (test with two windows).
  • A peer who connects after your module did something catches up (state sync or derivable-from-scene).
  • No Math.random() without a broadcast seed; no accumulation in effects.
  • Receiving a message never re-broadcasts it.
  • id unique, node types unique, version bumped on behavior changes.
  • Switch it off and on again in the Modules manager (or dev-reload it): no timer, listener, sound, overlay or scene object of the old instance is left, and the new one works from scratch (__stores.moduleSDK.registrationsOf(id) is {} while it is off).