@realitycollective/threejs-interactions
The three.js adapter for the Reality Collective Interaction Extensions. It reads raw WebXR directly, so no other framework is needed.
npm install @realitycollective/threejs-interactions three
It re-exports everything from @realitycollective/webxr-interactions, so this is the only interaction package your app needs.
What it binds
| Layer | Detail |
|---|---|
| Input | Raw WebXR, the browser's own XR API: controllers, hand joints, trigger and grip pressure, and haptic pulses |
| Hit-testing | three.js raycasting against your scene graph, through three-mesh-bvh when your app installs it |
| Movement | Moves and rotates three.js objects for grab, hinge, dial and slide |
| Desktop | A mouse fallback, so the same scene is testable without a headset |
| Presence | Show and hide the user's own hand and controller models, once you have registered them |
Presence
registerVisual(handedness, root) is on this adapter and on no other, because a standalone three.js app builds its own hand and controller models. The IWSDK and Babylon adapters find models the engine already built; here there is nothing to show or hide until you hand one over. So capabilities.presence is false until the first registerVisual call and true afterwards, and it changes back if you give the last one back with unregisterVisual. Registering notifies onCapabilitiesChanged like any other capability change.
provider.registerVisual("left", leftHandModel);
provider.registerVisual("right", rightHandModel);
provider.setPresenceVisible("left", false); // hide the left hand
Usage
import { createThreeInteractions } from "@realitycollective/threejs-interactions";
const interactions = createThreeInteractions({
xr: renderer.xr,
camera,
domElement: renderer.domElement,
});
interactions.register({ id: "button", behaviours: [{ kind: "press" }] }, buttonMesh);
interactions.runtime.onEvent((event) => console.log(event.type));
let last = performance.now();
renderer.setAnimationLoop(() => {
const now = performance.now();
const dt = Math.min(0.1, (now - last) / 1000);
last = now;
interactions.update(dt);
renderer.render(scene, camera);
});
Hit-testing detailed meshes
three.js tests every triangle of a mesh once a ray enters its bounding sphere, so a detailed model registered as an interactable costs the raycast close to a millisecond per ray on a desktop, and the runtime casts up to three rays a frame (one per controller, one for gaze). Two ways round it, and they combine.
Install three-mesh-bvh. The adapter declares it as an optional peer. Install it and add its prototype hooks once at startup; the hit tester then builds a bounds tree for every geometry it registers and asks the raycaster for the first hit only, so the same ray costs microseconds. IWSDK installs these hooks itself, so an IWSDK app has nothing to do.
import { BufferGeometry, Mesh } from "three";
import { acceleratedRaycast, computeBoundsTree, disposeBoundsTree } from "three-mesh-bvh";
BufferGeometry.prototype.computeBoundsTree = computeBoundsTree;
BufferGeometry.prototype.disposeBoundsTree = disposeBoundsTree;
Mesh.prototype.raycast = acceleratedRaycast;
The tree lives on the geometry for as long as the geometry does. The adapter never removes it, so a geometry shared between interactables is built once, and a geometry you discard takes its tree with it.
Register a collider proxy. Register a low-poly stand-in (a box, a sphere, a simplified hull) as the interactable object, and parent the detailed model to it or move both from the same transform. The proxy is what the ray and the poke test see; the model is what the player sees.
Poke targeting (hitProximity) is a sphere test on each registered object's world position and bounding-sphere radius, so it costs the same whatever the mesh.
Peer dependencies
three >= 0.170.0. The Reality Collective demos pin the super-three@0.181 fork that Meta's IWSDK mandates; stock three.js works equally well for this adapter.
three-mesh-bvh >= 0.9.14, optional. See the section above.
Live demo
The interaction playground - the full station set, mouse-capable on desktop, VR button for headsets: webxr-interactions.pages.dev
Documentation
See the repository README and the demos/playground client for a complete working scene.
License
MIT - see LICENSE.
Classes
| Class | Description |
|---|---|
| DialBehaviour | - |
| DwellState | - |
| Emitter | Minimal synchronous emitter (insertion-ordered, unsubscribe-safe). |
| GrabBehaviour | - |
| HingeBehaviour | - |
| InteractionRuntime | - |
| PressBehaviour | - |
| PulseBehaviour | - |
| SlideBehaviour | - |
| ThreeHitTester | Resolves which interactable a ray or a proximity probe touches. Implemented by the adapter (three.js Raycaster, engine BVH, …). A provider that pre-resolves targeting (IWSDK) may make this redundant - provider hints always take precedence over hit-tester results. |
| ThreeInteractions | - |
| ThreeTransformPort | The transform/effect surface of ONE interactable's scene object. Behaviours write through this; the adapter owns how writes land (Object3D fields, ECS components, …). |
| TossScoreBehaviour | - |
| VelocityTracker | - |
| WebXRInputProvider | - |
Interfaces
| Interface | Description |
|---|---|
| Behaviour | - |
| BehaviourContext | Everything a behaviour can reach during a tick or a routed moment. |
| BehaviourFactoryContext | - |
| DialConfig | - |
| DwellConfig | Gaze support - first-class, per the design decision that gaze serves two roles: |
| DwellTick | - |
| FeedbackAudioSink | Whatever the client uses to play a sound. Deliberately tiny: an id the client resolved to a buffer/sample of its own, and a gain. The core never loads, decodes or owns audio. |
| FeedbackIntent | - |
| GazeConfig | - |
| GrabConfig | - |
| HingeConfig | - |
| HitTester | Resolves which interactable a ray or a proximity probe touches. Implemented by the adapter (three.js Raycaster, engine BVH, …). A provider that pre-resolves targeting (IWSDK) may make this redundant - provider hints always take precedence over hit-tester results. |
| InteractableDescriptor | - |
| InteractableHit | - |
| InteractableState | - |
| InteractionDescriptor | - |
| InteractionEvent | - |
| InteractionRuntimeOptions | - |
| InteractorInfo | The slice of a live interactor a behaviour may consume. |
| PointerBridge | - |
| PressConfig | - |
| PulseConfig | - |
| RegisterPorts | - |
| SlideConfig | - |
| ThreeInteractionsOptions | - |
| TossScoreConfig | - |
| TransformPort | The transform/effect surface of ONE interactable's scene object. Behaviours write through this; the adapter owns how writes land (Object3D fields, ECS components, …). |
| VelocityTrackerOptions | - |
| WebXRProviderContext | - |
Type Aliases
| Type Alias | Description |
|---|---|
| BehaviourConfig | - |
| FeedbackCue | A named moment a client may want to sonify/hapticise. |
| FeedbackListener | - |
| GrabFulfilment | - |
| InteractionEventListener | - |
| InteractionEventType | - |
| TransformOwnership | Who owns the interactable's transform while this behaviour is active. |
Variables
| Variable | Description |
|---|---|
| DWELL_DEFAULTS | - |
| HEAD_GAZE_INTERACTOR_ID | Interactor id used when gaze is synthesized from the head pose. |
Functions
| Function | Description |
|---|---|
| centeredUnit | Map a centered hinge angle in `[- |
| clamp01 | Clamp to the unit interval. |
| clampDeadzone | Zero out a signal below threshold, leaving larger values untouched. |
| clampSym | Clamp a value to the symmetric range `[- |
| createBehaviour | - |
| createPointerBridge | Create a pointer stream from the runtime's sampled sources. match picks which source drives it (default: any source with a ray - first pressed wins and holds until release). |
| createThreeInteractions | - |
| decay | Exponential decay of value toward 0 over dt at ratePerSec. |
| easeOutBounce | Penner's ease-out-bounce on t ∈ [0,1]. |
| hingeAngleToHand | Hinge angle for a position-driven lever: given the hand's offset from the pivot resolved into the hinge plane - alongRest (toward the arm's rest direction) and alongSwing (the in-plane perpendicular) - the angle the arm should take to point at the hand, clamped to `± |
| holderPoint | Hand position a hand-driven behaviour should track, best available. |
| normalize01 | Normalize a measured displacement/angle against a signed range to 0..1. Sign-agnostic: a range of -π/2 maps -π/2 → 1. |
| passedThroughHoop | Did a tossed object pass through a hoop this frame? True when it crossed the hoop plane in the scoring direction (downward - prev above, curr at/below) while inside the ring (radial <= radius). |
| quatConjugate | - |
| quatFromAxisAngle | Quaternion from an axis (normalised) and an angle in radians. |
| quatMultiply | - |
| rayAngleTo | Angular offset (radians) between a ray direction and the direction to a point. |
| rayPointDistance | Closest-approach distance from a ray to a point, and the along-ray t. |
| registerBehaviourKind | - |
| registerDescriptor | Build a runtime's interactables from a portable descriptor. |
| resolveDwellConfig | - |
| routeAudioToSink | Client opt-in: play a sound for the feedback intents named in cueMap. Cues absent from the map are ignored, so an app sonifies only the moments it has sounds for. Gain is the intent's intensity scaled by gainScale, capped at 1. Returns the unsubscribe. |
| routeHapticsToProvider | Client opt-in: forward the haptic half of feedback intents to the input provider's pulse (which no-ops on sources without actuators - hands). Returns the unsubscribe. Audio remains entirely the client's concern. |
| springStep | One step of a damped harmonic oscillator (semi-implicit Euler). out[0] = next value, out[1] = next velocity. An under-damped spring (damping ratio ≈ 0.3-0.5) is what gives a pressed button its bounce. dt is clamped to 1/30 s so a long frame can't blow the integrator up. |
| vAdd | - |
| vApplyQuat | Rotate a vector by a quaternion. |
| vCross | - |
| vDistance | - |
| vDot | - |
| vLength | - |
| vNormalize | - |
| vScale | - |
| vSub | - |
| worldToLocal | Transform a world-space point into the local frame of pose. |