Skip to main content
Family 02 · Interactions

Buttons, levers, dials and grabbable things.

WebXR Interactions adds interactive objects to a WebXR scene. The interaction logic has no 3D engine code in it. You add one adapter for the engine you already use, and that adapter feeds the shared core. It is based wholly on the Reality Toolkit's interaction framework for Unity, revised for the web.

Released, 0.1.0 on npmMITFour engine adapters
webxr-interactions
behaviours · targeting · gaze · events · feedback · descriptors
threejs-interactions
adapter
babylon-interactions
adapter
iwsdk-interactions
adapter
xrblocks-interactions
adapter, experimental
webxr-input
contracts · separate repository

How it is built

Every Reality Toolkit WebXR family has the same shape. The logic lives in an engine-free core that carries the tests. A thin adapter binds that core to one engine, depends on it, and re-exports it, so your app installs one package. Where families need to agree on a shape, they share plain contracts rather than importing each other.

Your app

Owns the content: meshes, panels, placement, physics and sound files. Installs one adapter per family and wires families together with subscriptions in the one place where both halves are in scope.

One adapter per engine

The only place an engine name appears. It reads the engine each frame, feeds the core, applies the core's results back, and reports anything the engine cannot do instead of failing silently.

The engine-free core

All the logic, as plain TypeScript with no engine import. An architecture test fails the build the moment one lands, which is what lets the same behaviour run unchanged on every engine, and be tested without a headset.

Shared contracts

One description of input, poses and pointers, published from its own repository with zero dependencies, so an adapter written once feeds both the Interactions and UI Extensions families.

Your appcontent, physics, the wiring between familiesOne adapterthreejs-, babylon-, iwsdk- or xrblocks-interactionsEngine-free corewebxr-interactions · behaviours, targeting, gaze, events, feedbackContractswebxr-input · one description of inputArrows only point down. Families never reference each other.

Where it sits in the stack

Four families, one rule: no package references a sibling family, not even a type. What crosses a boundary is a subscription your app makes, such as playing an Environment cue when an Interactions press fires.

The Service Framework for the web sits beside these four and keeps its own site: serviceframework.realitycollective.net. The layering rule explains the boundaries in full.

Why you would use it

01

Behaviours, ready made

Press with an optional latching mode, pulse, hinge, dial, slide, grab and toss-scoring for throw-and-catch games. Attach one to an object and subscribe to its events.

02

Targeting that does not flicker

For each hand or controller the core picks one target: the platform's own answer first, then a close touch, then a pointing ray, with a short hold on the boundary between two objects.

03

Capabilities that speak up

If the headset cannot do what a behaviour needs, the behaviour switches itself off and says so, rather than silently doing nothing.

Pick an adapter

Install exactly one. Each re-exports the core and the input contracts, so you never install those yourself.

three.js and raw WebXR

threejs-interactions

The default, standalone adapter. Reads raw WebXR, adds three.js hit-testing and object movement, and a desktop mouse fallback. Presence is opt-in through registerVisual.

defaultRead more →

Babylon.js

babylon-interactions

Reads a WebXRDefaultExperience: controllers, motion controller trigger and grip, hand joints. Structurally typed, so no @babylonjs/core import.

not yet run on a real appRead more →

Meta IWSDK

iwsdk-interactions

Passes IWSDK's own press and grab answers straight through, with native grab fulfilment and full presence control. Setup is one call: registerInteractions(world).

fullRead more →

Google XR Blocks

xrblocks-interactions

Maps the XR Blocks input pipeline onto the three.js machinery. XR Blocks has no haptics and no way to hide its own hand visuals, so neither is offered.

experimentalRead more →

Who does what

Read the ladder top to bottom. The middle column is the stage every engine goes through; the core owns the left, the adapter you install owns the right.

The core
Stage
The adapter
Asks the provider for snapshots each frame
Sample input
Implements InputProvider over raw WebXR, Babylon.js, IWSDK or XR Blocks
Platform hint, then touch, then ray, with a hold on the boundary
Pick a target
Hit-tests the scene and supplies the platform answer where it has one
Press, pulse, hinge, dial, slide, grab, toss-score
Run behaviours
Applies poses through its transform port
One event stream; never calls into app code
Emit events
Nothing to add
Requests a haptic pulse or a cue
Feedback
Fulfils haptics where the engine can and reports where it cannot

Five words you will meet

Interactablean object in your scene that can be interacted with, such as a button
Interactorthe thing doing the interacting: a hand, a controller, a ray or a mouse
Behaviourpress, pulse, hinge, dial, slide, grab or toss-score, attached to an interactable
Descriptora whole scene of stations as portable data
Feedbacka request for a haptic pulse or a sound; playing it is the app's job