@realitycollective/iwsdk-uiextensions

Windowing, docking, layout regions and extra controls for Meta's Immersive Web SDK (@iwsdk/core).

This is the Meta IWSDK adapter - and the reference implementation - for the engine-free @realitycollective/webxr-uiextensions core, which it re-exports in full: one dependency gets IWSDK apps the whole surface. (A sibling @realitycollective/xrblocks-uiextensions adapter binds the same core to Google XR Blocks, experimentally.)

Reuse, not recreation. The IWSDK already ships an excellent spatial UI stack - UIKitML markup, @pmndrs/uikit rendering, Follower/ScreenSpace anchoring, grab/ray/poke interaction. This package adds the missing layer above it:

Feature	What you get
Windows	Title-bar chrome (pin / dock / minimize / close, each opt-in), focus & z-ordering, hide/show, a per-world WindowManager with typed events that is also the API for driving a window from code
Hand menus	A hand-locked window rides a hand (left, right or whichever is raised), anchored above the fingertips, at the wrist, or on the thumb or little-finger side, shown while the palm faces you; a vertical stack of buttons that sizes to its content, driving other windows through the manager
Dock states	world-locked (place in space) ⇄ body-follow (lazy follow) ⇄ head-locked, realised with the IWSDK's own Follower/ScreenSpace
Manipulation	Drag windows by the title bar with the far ray or a near grab (controller squeeze, hand pinch), powered by @pmndrs/handle, the same library behind IWSDK grabbing; billboard-while-dragging, drop-to-dock
Layout regions	Named regions (row / column / grid slots) windows snap into; regions can themselves follow the player
Controls	data-uix markup upgrades: stepper, toggle, expandable multi-line label, log/list view - plus everything UIKitML already has (buttons, inputs, textareas, images, and the horizon kit's Slider/Checkbox/…)

Everything is authored in plain UIKitML (HTML/CSS-like) - no new markup language, no custom renderer, no wrapper widgets around things the IWSDK already does.

Install

npm install @realitycollective/iwsdk-uiextensions
# peers: @iwsdk/core >=0.5.0 <0.6.0 and three >=0.170.0 (every IWSDK app already has both)

Quick start

import { World } from '@iwsdk/core';
import {
  registerUIExtensions,
  createUIWindow,
  createDockRegion,
  DockMode,
} from '@realitycollective/iwsdk-uiextensions';

const world = await World.create(container, {
  features: { spatialUI: true },
});

const windows = registerUIExtensions(world); // registers all systems, returns the WindowManager

createDockRegion(world, { id: 'wall', flow: 'column', position: [1.5, 1.8, -1.5] });

createUIWindow(world, {
  id: 'status',
  title: 'Player Status',
  config: './ui/status.uikitml', // UIKitML source; IWSDK 0.5 parses it at runtime
  dockMode: DockMode.BodyFollow,   // follows until the user pins it
  pinnable: true,                  // title-bar buttons are off unless asked for
  minimizable: true,
});

windows.events.on('closed', (w) => console.log(`${w.title} closed`));

// The manager is also how code changes a window - from a hand menu, say:
windows.hide('status');            // and show(), toggleHidden()
windows.togglePin('status');       // or setDockMode(id, DockMode.WorldLocked)
windows.dockTo('status', 'wall');  // undock(id), returnHome(id)
windows.setChrome('status', { close: true, dock: true });
windows.close('status');           // destroys the entity

Window markup

Windows are ordinary UIKitML panels; the chrome is discovered by well-known element ids (only the ids are contractual - restyle freely):

<div id="uix-window" class="my-window">
  <div id="uix-titlebar" class="my-titlebar">      <!-- drag surface -->
    <span id="uix-title" class="my-title">.</span>
    <div id="uix-pin">PIN</div>                    <!-- follow ⇄ placed; label auto-syncs to PIN/UNPIN -->
    <div id="uix-dock">DOCK</div>                  <!-- return to home (spawn region / placement) -->
    <div id="uix-minimize">MIN</div>
    <div id="uix-close">X</div>
  </div>
  <div id="uix-content">
    <!-- window body -->
  </div>
</div>

Use <div>s (not <button>s) for chrome buttons: with a component kit registered, lowercase <button> resolves to the kit's Button component, whose intrinsic sizing fights compact title-bar chrome.

Controls markup

Annotate any element with data-uix and the UIControlsSystem upgrades it - in any panel, not just windows:

<div data-uix="stepper" data-uix-id="health" data-uix-min="0" data-uix-max="100" data-uix-step="10">
  <button data-uix-role="decrement">-</button>
  <span data-uix-role="value">.</span>
  <button data-uix-role="increment">+</button>
</div>

import { panelControlsFor } from '@realitycollective/iwsdk-uiextensions';

const controls = panelControlsFor(document); // the panel's UIKitDocument
controls.stepper('health').events.on('change', (hp) => setHealth(hp));

UIKitML note: every dynamic-text element needs a literal placeholder child (<span data-uix-role="value">.</span>) or no Text node is created.

Every title-bar button is off by default. Keep all four in the markup, then enable the ones a window should have with closable, minimizable, pinnable and dockable at spawn, or later with windows.setChrome(id, { pin: true }). A disabled button is hidden and its click ignored; enabling one needs no rewiring.

Interaction model

• Drag the title bar with the ray (or mouse on desktop) to move a window; it billboards toward you while dragging and settles facing you when released. A press only becomes a drag after dragDelay seconds (default 0.3, per-window on UIWindow) - shorter presses stay clicks, and the chrome buttons swallow their presses entirely, so PIN/DOCK/MIN/X never fight the drag gesture.
• Near grab the title bar to pick the window up: squeeze with a controller, or pinch with a tracked hand while it is on the bar. A near grab is a deliberate gesture, so it drags at once with no hold delay. This does not need features.grabbing: UIDragSystem enables IWSDK's near grab pointer itself, lists every movable title bar as a target for it each frame, and forwards a hand pinch to it only while the hand is on a title bar, so pinching anywhere else still means what your app decided. Pass registerUIExtensions(world, { nearDrag: false }) to keep the ray as the only way to move windows.
• Drop a window inside a region's snap radius to dock it into the next slot; drag it out again to undock.
• Pin toggles body-follow ⇄ world-locked ("place in space").
• Dragging a following window implicitly places it - pin re-attaches it.
• Hide takes a window out of view and out of reach (no ray or poke can hit it) while keeping its dock mode, region slot and minimized state; show brings it back exactly where it was and in front. Minimize collapses the body but leaves the title bar drawn.
• Poke (near touch) is guarded. IWSDK's own touch pointer presses and releases on an unsigned distance to the panel, so a finger pushed through a panel and pulled back out fires two clicks, and a finger arriving from behind presses. UITouchGuardSystem drives the two touch pointers from the core's TouchPress state machine instead: a press only when the fingertip enters from the front (2 cm), a hold however deep it goes and whichever way it comes back, a release only on coming back out past 3 cm or on losing contact, and no second press until that release. Every poke target in the app gets this, IWSDK's own panels included. A press that starts on one button and ends over another clicks neither (the release lands where the finger is; IWSDK clicks only when both ends are the same element), and IWSDK's 800 ms click window still applies. Tune with registerUIExtensions(world, { touchGuard: { pressDistance, releaseDistance, allowFromBehind } }), or touchGuard: false for IWSDK's own behaviour.

Hand menus

Spawn a window with dockMode: DockMode.HandLocked and it becomes a hand menu in the manner of MRTK 2's: it rides a hand and shows while that palm is raised toward you. handMenu says how:

host.createWindow({
  id: 'menu',
  config: './ui/hand-menu.uikitml',
  dockMode: DockMode.HandLocked,
  handMenu: {
    hand: 'left',        // 'left' | 'right' | 'either' (whichever palm is raised)
    anchor: 'above',     // 'above' fingertips | 'inside' (thumb) | 'outside' | 'wrist'
    anchorDistance: 0.12, // meters from the palm
    offset: [0, 0, 0],   // extra hand-local nudge
    palmGate: true,      // show only while the palm faces you
    palmAngle: 60,       // how far off square the palm may be, degrees
  },
});

Every field is optional; the defaults are the values shown. windows.setHandMenu(id, { hand: 'right' }) changes them at runtime. The hand pose is the player rig's grip space, so a controller's grip or a tracked hand both work, and the panel always turns to face you. While the gate is shut the menu is neither drawn nor hittable, and hide() still wins over an open gate. Pinning or dragging a hand menu makes it an ordinary world-locked window where it was.

Use HAND_MENU_SNIPPET from the core as the markup starting point: the same uix-window / uix-content ids, no title bar, a vertical stack of buttons that sizes to its content. Examples/basic-window/ is a window whose manipulation buttons live on such a menu.

Driving windows from code

registerUIExtensions returns the WindowManager, and it is the one API app code needs to change a window - a hand menu, a keyboard shortcut, a voice command. Every call is applied by the systems, and the same calls work on the XR Blocks adapter:

Call	Effect
hide(id) / show(id) / toggleHidden(id)	Out of view and unhittable; back in place and in front
minimize(id) / restore(id) / toggleMinimized(id)	Collapse / expand the body
togglePin(id) / setDockMode(id, mode)	Follow the player or stay put
dockTo(id, regionId) / undock(id)	Into a region slot (world-locked) / out of it
returnHome(id)	Back to the spawn region, or the spawn placement and mode (what DOCK does)
setChrome(id, { pin, dock, minimize, close })	Enable or disable title-bar buttons at runtime
focus(id)	Bring to the front
close(id)	Destroy the window's entity

| setHandMenu(id, { hand, anchor, ... }) | Move a hand menu to the other hand or another anchor |

The record is always what the scene shows: a drag that docks a window, or a PIN click, is written back into windows.get(id), and every change emits a typed event (hidden, shown, regionChanged, returnHome, chromeChanged, handMenuChanged, alongside the existing ones) so a menu can keep its labels honest. See Examples/basic-window/.

See Examples/ (shipped in this package) and the deployable showcase client in the repository for complete, working demonstrations of every feature.

Windows and panel readiness

createUIWindow returns the ECS entity, which is what you want when you are going to add components to it. When you want the PANEL, use the scene host: createWindow gives you a handle that resolves itself.

import { createSceneHost, getPanelHandle } from '@realitycollective/iwsdk-uiextensions';

const host = createSceneHost(world);          // call after registerUIExtensions(world)

const status = host.createWindow({
  id: 'status',                                // optional - omit and you get uix-window-<n>
  title: 'Player Status',
  config: './ui/status.uikitml',
});

status.panel;                                  // undefined until IWSDK attaches the document
status.onReady((panel) => {                    // runs once, immediately if it is already there
  panel.getElementById('uix-title');
});
status.entity;                                 // still the entity, for ECS work

getPanelHandle(entity) does the same lookup for an entity you already hold, and returns undefined while the document is still loading.

Do not poll getPanelHandle on a timer. It is a single synchronous read, not a wait, and there is no deadline you can safely guess: the markup is fetched over the network and parsed over later frames, so a cold cache on a headset takes far longer than a warm one on a desktop. A poll that gives up early leaves a window that draws correctly and responds to nothing, with no error and no log line, which is close to undiagnosable from the outside. Every window has a readiness signal already, so use one.

Across a whole scene, subscribe to the host instead:

host.onPanelReady(({ id, panel, kind }) => {
  if (kind === 'panel') {
    // A bare PanelUI entity with no UIWindow: `id` is its config path.
    return;
  }
  wireMyWindow(id, panel);
});

onPanelReady covers windows spawned by createUIWindow too, not only by host.createWindow. So code that already holds factory entities does not have to change how it spawns them: create the host once, subscribe, and match on the id you passed to the factory.

const entity = createUIWindow(world, { id: 'status', config: './ui/status.uikitml' });

createSceneHost(world).onPanelReady(({ id, panel }) => {
  if (id === 'status') wireStatus(panel);
});

Pass an id to createUIWindow if you intend to match on one. Without it the window has no id to announce, so it arrives as kind: 'panel' with its config path as the id, and a listener filtering on kind === 'window' will silently never see it. host.createWindow differs here: it invents uix-window-<n> when you omit the id.

createSceneHost(world) returns the same host every time it is called for a world, so separate modules can each ask for it without coordinating or passing it around.

Bare panels are announced as well as managed windows, which is how devtools and hand-built PanelUI entities show up in the same stream. supportsStandalonePanels is false on this host: IWSDK owns panel lifecycles through the ECS, so createPanel() throws rather than half-working. Spawn a window instead.

Headless core

All decision logic (window manager, dock state machine, region slot math, drag math, control models) lives in @realitycollective/webxr-uiextensions - pure TypeScript with no engine imports, tested at 100% coverage. The ECS systems in this package are thin appliers of that core onto @iwsdk/core components.

Live demos

• Showcase: webxr-uiextensions.pages.dev
• Multiplatform lab: webxr-uix-lab.pages.dev

License

MIT © Reality Collective

Classes

Class	Description
Emitter	-
ExpandableLabelHandle	-
ExpandableModel	-
HoldToDrag	-
LogModel	-
LogViewHandle	-
PanelControls	-
RegionRegistry	-
StepperHandle	-
StepperModel	-
ToggleHandle	-
ToggleModel	Toggle model - pure boolean state for the data-uix="toggle" control.
TouchPress	-
UIControlsSystem	-
UIDockRegionSystem	-
UIDockSystem	-
UIDragSystem	-
UITouchGuardSystem	-
UIWindowSystem	-
WindowManager	-

Interfaces

Interface	Description
CreateDockRegionOptions	-
CreateWindowOptions	Options for createUIWindow.
DockRecipe	Engine-agnostic description of what a dock mode requires.
DockTransition	A transition plan: what to add and what to remove, in engine terms.
DragSession	-
ExpandableOptions	Expandable label model - pure truncation/expansion state for the data-uix="expandable-label" control (a multi-line label that collapses to a preview with an ellipsis and a "more/less" affordance).
HandMenuOptions	-
HandMenuPlacement	What the adapter applies this frame.
HandPoseSource	Supplies a hand's pose each frame as a WebXR GRIP space, the frame hand-menu.ts documents (-Z toward the thumb, +Y up the arm, the palm at -X on the right hand and +X on the left). A controller's grip and a tracked hand's gripSpace both are one; hand JOINT spaces are not, and must be converted. Returns undefined while that hand is not tracked; a hand menu on it is then hidden. An adapter without hands at all (a desktop) supplies no source and falls back to body-follow placement for hand-locked windows.
HoldUpdate	-
IwsdkSceneHost	The engine-agnostic surface an app needs to build a UI: spawn windows and regions from portable data, observe when panels become wireable, and reach the shared WindowManager.
IwsdkWindowHandle	A window spawned by IwsdkSceneHost.createWindow.
LogEntry	Log model - pure ring buffer + viewport for the data-uix="log-view" control (scrollable log/list windows).
LogModelOptions	-
OpenWindowOptions	-
PanelHandle	A live spatial panel created from compiled UIKitML JSON. The root is traversable with the core's walk/findRole helpers and the data-uix control upgraders - identical markup works on every adapter.
PanelHost	Creates spatial panels - the engine-specific half of UIKitML rendering.
PanelReadyEvent	A window whose panel has finished loading and is ready to be wired. Delivered by WindowHost.onPanelReady.
PointerInputSource	Delivers press-move-release for one interaction source (a controller ray, a hand pinch, a mouse). The core's hold-to-drag and drag-math consume these; the adapter decides what constitutes press/release.
RegionCandidate	-
RegionDefinition	-
RegisteredRegion	-
RegisterOptions	-
SceneDescriptor	A complete, engine-free scene definition.
SceneRegion	One dock region in a scene.
SceneTarget	What an adapter must provide for applyScene to build a scene. Both shipped adapters implement this; a new adapter only needs these two methods to gain full scene portability.
SceneWindow	One window in a scene.
StepperOptions	Stepper model - pure numeric state for the data-uix="stepper" control.
TouchPressOptions	Touch press - the press / hold / release state machine for a near (poke) pointer against a panel, pure logic with no engine imports.
TouchSample	What the adapter measured this frame. undefined means no contact at all.
TouchUpdate	The transitions the adapter acts on this frame.
UixElement	Structural view of a uikit element as the controls layer needs it.
WindowChrome	Which title-bar buttons are enabled. Keys match the chrome element ids in WINDOW_CHROME_IDS. A disabled button is hidden and its click ignored. Every button is OFF unless the app turns it on.
WindowHandle	A window an adapter spawned, before its panel necessarily exists.
WindowHost	The engine-agnostic surface an app needs to build a UI: spawn windows and regions from portable data, observe when panels become wireable, and reach the shared WindowManager.
WindowHostContractCase	One check a WindowHost implementation must pass. run returns silently on success and throws an Error describing the failure otherwise, so any test runner can host it.
WindowHostContractSetup	Everything a case needs to drive one adapter. Build a FRESH one per case: cases spawn windows of their own and do not clean up after themselves.
WindowManagerEvents	-
WindowOptionsBase	The window options every adapter understands.
WindowRecord	-

Type Aliases

Type Alias	Description
ControlHandle	-
DockModeValue	-
Hand	-
HandMenuAnchor	Where the panel sits relative to the palm.
HandPoses	This frame's tracked hands; a hand that is not tracked is absent.
HoldPhase	Hold-to-drag threshold - pure timing state for title-bar dragging.
Listener	Minimal typed event emitter.
PointerSample	One pointer/ray interaction stream, engine-normalised: a world-space origin (ray origin or touch point) and a normalised direction. It is the Input package's RayTuple, which is what lets a provider written against @realitycollective/webxr-input feed this contract unchanged.
RegionFlow	-
TouchPhase	-
Vec3	Layout regions - pure slot math for docking windows into named regions.

Variables

Variable	Description
DEFAULT_HAND_MENU	-
DEFAULT_REGION	-
DEFAULT_TOUCH_PRESS	-
DockMode	Dock state machine - pure logic, no engine imports.
ELLIPSIS	-
HAND_MENU_SNIPPET	Reference hand-menu markup: the same root and content ids, no title bar, a vertical stack of buttons that sizes to its content. Spawn it with dockMode: 'hand-locked' and it rides the hand; without a title bar there is nothing to drag, which is what a hand menu wants. Give each button an id and wire it to a WindowManager call.
HandAnchor	HandMenuOptions.anchor as an enum object for the component schema.
HandChoice	HandMenuOptions.hand as an enum object for the component schema.
NO_CHROME	-
RegionFlowType	Region flow options mirrored as an enum object for the component schema.
UIDockedTo	Present on a window entity while it is docked into a region.
UIDockRegion	A named layout region that docked windows snap to and are laid out within. Place the entity where the region should live (it can itself carry a Follower to make a body-locked region).
UIWindow	Marks a panel entity as a managed window with chrome, focus, docking and drag behaviour. Pair with PanelUI whose markup contains the window chrome elements (see WINDOW_CHROME_IDS / the Examples folder).
UIWindowState	Internal bookkeeping the systems keep on window entities - which dock mode has actually been applied to engine components, and whether chrome wiring has run for the current PanelDocument.
uixComponentSet	Pass to spatialUI.componentSets so <uix-*> elements parse on IWSDK 0.5. Without it, a panel using any control fails to parse and never attaches.
WINDOW_CHROME_IDS	Window chrome conventions.
WINDOW_CHROME_SNIPPET	-

Functions

Function	Description
anchorOffset	Hand-local offset for an anchor. The thumb is -Z and the fingertips -Y for both hands in the grip frame, so no mirroring is needed.
applyScene	Apply a descriptor to an adapter. Regions are created before windows so a window that spawns docked (region: 'x') always finds its region.
attrBoolean	-
attrNumber	-
attrString	-
beginDrag	Begin a drag: capture how far along the ray the grab landed and where the window origin sits relative to the grab point.
captureDrop	Which region captures a window dropped at point? The nearest region whose snapRadius contains the point and which still has capacity; undefined when the drop lands in open space (the window stays world-locked where it was released).
createDockRegion	-
createSceneHost	The host for a world: built on the first call, and the same instance every call after that. Call AFTER registerUIExtensions(world).
createUIWindow	-
dragPosition	Window position for the current ray. rayDirection must be normalized (pointer rays from the input layer already are).
evaluateHandMenu	The per-frame entry point an adapter calls for each hand-locked window.
faceViewer	Orientation whose local +Z (the face of a panel) points from position to viewer, upright against world +Y. Straight above or below the viewer there is no upright, so world +Z stands in for "up" there.
faceViewerYaw	Yaw (radians, three.js Y-up convention) that turns the window's -Z face toward viewer - the "PivotY" billboard used while dragging so a window never gets parked edge-on. Returns current when the viewer is directly above/below (degenerate on the Y axis).
findRole	-
findRoles	All descendants (including self) declared as <uix-{role}>.
getPanelHandle	The live panel on an entity, or undefined while IWSDK is still loading its document. Use it to reach a panel from an entity you already hold; onPanelReady stays the way to be TOLD when one appears.
handMenuPose	The menu's world pose for a hand: anchored in the hand frame, turned to face the viewer.
hasCapacity	-
isDockMode	-
isPalmFacing	-
minimizeLabelFor	What the minimize affordance should read for a window's current state - the label always names what the NEXT click does, matching pinLabelFor: - open → "MIN" (click collapses the content) - minimized → "MAX" (click restores it)
normalize	-
normalizeRegion	-
palmFacing	Cosine of the angle between the palm normal and the direction to the viewer: 1 is palm square on to the viewer, -1 is the back of the hand.
palmNormal	The palm normal in the hand frame: -X on the right hand, +X on the left.
panelControlsFor	Controls of an upgraded panel; undefined before UIControlsSystem ran.
pickHand	Pick the hand for this frame: the configured one when it is tracked, or for either the tracked hand whose palm faces the viewer most. The gate is applied here too, so a fixed hand that is tracked but turned away still yields undefined.
pinLabelFor	What the pin affordance should read for a window's current state: - dragging → "PIN" (the window is loose in your hand) - placed/world-locked → "UNPIN" (click releases it to follow) - following → "PIN" (click pins it where it is)
planTransition	Compute the minimal set of engine changes to move between dock modes. Returns undefined for a no-op (same mode).
quaternionFromBasis	Quaternion for the rotation whose columns are the given orthonormal axes.
recipeFor	-
regionRegistryFor	-
registerUIExtensions	-
resolveHandMenu	Fill a partial set of options from the defaults.
resolveTouchPress	-
rotate	Rotate a vector by a unit quaternion [x, y, z, w].
sampleOf	This frame's signed distance from the fingertip to the surface it is over, positive in front. The surface normal is the intersection's local normal (uikit panels report +Z) taken to world space; a target with no normal is treated as faced from the front, so a plain mesh keeps IWSDK's behaviour.
sceneConfigPaths	Every distinct panel config path in a scene (for preloading).
slotOffset	Local-space offset of slot index within a region.
tagOf	The custom-element tag this element was declared with, lowercased, or undefined for a plain built-in element.
togglePinned	The "pin" affordance on a window's title bar toggles between following the player and being placed in space. Head-locked and hand-locked windows unpin to world-locked too - pinning something rigidly to the user's face is never the toggle target you want, and a hand menu pinned in place is simply a window again.
upgradeExpandableLabel	-
upgradeLogView	-
upgradePanel	Upgrade every data-uix control under root and remember the result against documentKey (the panel's UIKitDocument). Idempotent per key.
upgradeStepper	-
upgradeToggle	-
validateScene	Validate a descriptor: unique window/region ids, and every region reference resolvable. Returns the problems found (empty = valid) rather than throwing, so callers can report them all at once.
walk	Depth-first traversal over uikit element children.
windowHostContractCases	The shared host conformance suite. An adapter's test file is a loop:
windowManagerFor	-