@three-ws/mocap

Motion capture as an API, turn a webcam or video into face, pose, and hand animation clips you can replay on any avatar.

Install · Quick start · API · How it works · three.ws

@three-ws/mocap is the official client for the three.ws mocap clip store - the library behind the in-browser mocap studio. Recording happens in the browser: MediaPipe Face Landmarker reads a webcam frame and emits 52 ARKit blendshapes plus a head-pose matrix per frame; the runtime smooths and packs them into a portable clip. This package is the durable half, it saves that clip, lists your library, makes clips public or for-sale, and hands any clip back so you can replay it on a different avatar. It wraps the auth'd /api/mocap/clips endpoints. It pairs with @three-ws/avatar and @three-ws/walk, those render the avatar, @three-ws/mocap drives its face.

#Why

Facial mocap in the browser is solved math, but the boring half, persisting a recording, versioning the wire format, sharing it, replaying it on a different rig: is where projects stall. A blendshape buffer in memory is gone the moment the tab closes. This package is that half, done once:

• Capture once, replay anywhere. A clip is rig-agnostic ARKit blendshapes, not baked keyframes, so a face you recorded on one avatar plays on any avatar that exposes ARKit morph targets.
• Versioned wire format. Every clip carries a format string (three.ws.face-mocap.v1). The runtime asserts on it before replay, a future v2 clip can't silently load on a v1 player and mangle the animation.
• Library, sharing, marketplace. Clips are private by default; flip one to public, attach a $THREE price, and it's listed. One field, no new plumbing.
• Cursor-paginated, owner-scoped. List your own library or browse the public pool with the same call.

#Install

npm install @three-ws/mocap

Zero runtime dependencies. Works in Node 18+ and the browser (uses fetch). To record a clip in the browser, use the three.ws mocap runtime; to render the avatar you replay it on, add @three-ws/avatar.

#Quick start

Save a recording the browser runtime handed you, then read it back:

import { saveClip, getClip } from '@three-ws/mocap';

// `recording` is the object FaceMocap.getRecording() returns:
//   { format: 'three.ws.face-mocap.v1', duration, frames: [{ t, shapes, mat }] }
const { id, slug } = await saveClip(recording, {
  name: 'Surprised reaction',
  tags: ['emote', 'reaction'],
}, { token: process.env.THREEWS_TOKEN });

const clip = await getClip(id);
console.log(clip.frameCount, 'frames,', clip.duration, 'seconds');
// → drop clip.frames into FaceMocap.play(clip) on any ARKit avatar

List your library, then browse the public pool:

import { listClips } from '@three-ws/mocap';

const mine   = await listClips({ token }, { kind: 'face' });
const shared = await listClips({}, { includePublic: true, limit: 100 });

#API

The SDK is a thin client over /api/mocap/clips. Every write needs a session cookie or a bearer token with the right scope; reads of public clips are open.

Writing with a session cookie also needs a CSRF token. Cookie-authenticated writes are double-submit protected: mint a token with GET /api/csrf-token and echo it in the X-CSRF-Token header on every POST/PATCH/DELETE, or the call answers 403 csrf_missing. Tokens are single-use, so mint one per write. Bearer tokens are exempt (a bearer is not auto-attached by a browser, so it is its own proof of intent), which makes { token } the simpler path outside the browser:

import { createMocap } from '@three-ws/mocap';

// Bearer: nothing else to do.
const mocap = createMocap({ token: process.env.THREEWS_TOKEN });

// Session cookie: wrap fetch and attach a fresh CSRF token per write.
const cookieClient = createMocap({
  headers: { cookie },
  fetch: async (url, init = {}) => {
    if ((init.method || 'GET') === 'GET') return fetch(url, init);
    const res = await fetch('https://three.ws/api/csrf-token', { headers: { cookie } });
    const { data } = await res.json();
    return fetch(url, { ...init, headers: { ...init.headers, 'x-csrf-token': data.token } });
  },
});

#saveClip(recording, meta, auth) → Promise<Clip>

Persist a browser recording. Wraps POST /api/mocap/clips. The recording is passed through as the clip field; meta supplies the library metadata.

recording: the object the capture runtime returns:

A Frame is { t: number, shapes: Record<string, number>, mat?: number[16] | null }

• t is seconds from clip start, shapes maps ARKit blendshape names to scores, mat is the optional 4×4 head-pose matrix.

meta

Returns the created Clip (201). kind (face / pose / hand / vmc) is derived from format server-side.

#getClip(idOrSlug, auth?) → Promise<Clip>

Fetch one clip with its full frames array, ready to replay. Wraps GET /api/mocap/clips/:idOrSlug. Public and unlisted clips need no auth; a private clip resolves only for its owner (otherwise 404, never a leak). Each non-owner fetch bumps play_count.

A slug is unique per owner, not globally, so a slug lookup resolves to your own clip when you are authenticated and otherwise to the oldest public clip carrying it. Pass the id when you need to name one exact clip.

#listClips(auth, opts?) → Promise<{ items, nextCursor }>

List clips without frames (metadata only: cheap, frames is null on every row). Wraps GET /api/mocap/clips. Cursor-paginated, newest first. Returns { items, nextCursor, raw }, where nextCursor is null on the last page and raw is the untouched wire envelope.

With no auth, you get the public pool only. With auth, you get your own library; add includePublic: true to fold in public clips too.

#updateClip(idOrSlug, patch, auth) → Promise<Clip>

Edit metadata. Wraps PATCH /api/mocap/clips/:id (owner only). Patchable: name, description, tags, visibility, avatarId, and price. Set price: { amount, currency } to list the clip for sale, or price: null to make it free again. Bearer tokens need the avatars:write scope.

#deleteClip(idOrSlug, auth) → Promise<{ ok: true }>

Soft-delete a clip you own. Wraps DELETE /api/mocap/clips/:id. The row is tombstoned (deleted_at), not destroyed. Bearer tokens need avatars:delete.

Clip shape (returned by every call; frames only on getClip). The SDK camelCases the wire row and keeps the untouched original on raw:

#How it works

Capture is client-side and free; persistence is this API. The clip is the contract between them: a portable, versioned blendshape buffer:

  webcam / video frame
         │
         ▼
  MediaPipe Face Landmarker      ← in the browser, three.ws mocap runtime
         │  52 ARKit blendshapes + 4×4 head-pose matrix
         ▼
  one-euro smoothing → record buffer
         │
         ▼
  getRecording() ──▶ { format, duration, frames:[{ t, shapes, mat }] }
         │
   saveClip()  │  POST /api/mocap/clips        ← @three-ws/mocap
         ▼
  ┌─────────────────────────────┐
  │  mocap_clips (Postgres)      │  owner-scoped, versioned, taggable, priceable
  └──────────────┬──────────────┘
   getClip()     │  GET /api/mocap/clips/:id
         ▼
  FaceMocap.play(clip) on ANY ARKit avatar   ← @three-ws/avatar / @three-ws/walk

Because a clip is rig-agnostic ARKit blendshapes: not baked bone keyframes - the same recording replays on any avatar that exposes ARKit morph targets. The format string carries the version; the runtime refuses a clip whose format it doesn't speak rather than mangling the animation.

Supported formats (the format string a recording must carry):

Frames are stored inline as JSONB up to 2 MB (≈ a multi-minute 30 Hz face capture). The runtime caps a single clip at 18,000 frames; longer captures should be split.

#Errors & edge cases

The endpoints return structured JSON errors { error: { code, message } }. The SDK surfaces them as a typed ThreeWsError carrying the code:

Every state is designed: a private clip you don't own returns 404, not its contents: non-owners can't even confirm it exists. Mirror that in your UI.

#Examples

Record in the browser, save to your library:

import { saveClip } from '@three-ws/mocap';
// faceMocap is the three.ws runtime recorder; recording is its output
const recording = faceMocap.getRecording();
const clip = await saveClip(recording, {
  name: 'Wink + smile',
  tags: ['emote'],
  visibility: 'unlisted',
}, { token });
console.log(`https://three.ws/mocap/${clip.slug}`);

Replay a public clip on a freshly forged avatar:

import { getClip } from '@three-ws/mocap';
import { forge } from '@three-ws/forge';

const avatar = await forge('a friendly robot, ARKit face');
const clip   = await getClip('surprised-reaction');
// drive avatar's ARKit morph targets from clip.frames in your render loop

List a tag-filtered gallery for a picker:

import { listClips } from '@three-ws/mocap';

let cursor;
const all = [];
do {
  const page = await listClips({}, { includePublic: true, limit: 100, cursor });
  all.push(...page.items);
  cursor = page.nextCursor;
} while (cursor);

#Related

• @three-ws/avatar, render the avatar a mocap clip drives.
• @three-ws/walk, a rigged avatar as a page companion you can face-animate.
• @three-ws/forge, generate the ARKit avatar to replay clips on.
• @three-ws/x402-fetch, settle payments for priced clips.

Built by three.ws · The only coin is $THREE