Handy Work
A module for doing efficient real-time pose detection from WebXR Hand Tracking using Web Workers to ensure it doesn't interfere with the main thread
The pose tracking module is Framework Agnostic, it doesn't rely on any particular library it should work just as well with THREE as BabylonJS or Play Canvas.
Poses can be found in the /poses/
folder and additional poses are welcome
There is also an AFrame module which handles pose tracking and displaying hand models.
Hand Pose Module Usage Example
import {
update as handyWorkUpdate,
loadPose
} from "../build/esm/handy-work.js";
loadPose('relax', '../poses/relax.handpose');
loadPose('fist', '../poses/fist.handpose');
loadPose('flat', '../poses/flat.handpose');
loadPose('point', '../poses/point.handpose');
// In RAF
handyWorkUpdate([controller1, controller2], referenceSpace, frame, callback);
Using via a CDN
If you use it via a CDN because it uses a Worker you need to use the standalone version which has the Worker encoded as a string.
import('https://cdn.jsdelivr.net/npm/handy-work@1.4.0/build/esm/handy-work.standalone.js')
.then(function ({
update,
loadPose,
dumpHands
}) {
const handyWorkUpdate = update;
const dumpHands = dumpHands;
const loadPose = loadPose;
loadPose('relax', POSE_FOLDER + 'relax.handpose');
loadPose('fist', POSE_FOLDER + 'fist.handpose');
loadPose('flat', POSE_FOLDER + 'flat.handpose');
loadPose('point', POSE_FOLDER + 'point.handpose');
loadPose('horns', POSE_FOLDER + 'horns.handpose');
loadPose('shaka', POSE_FOLDER + 'shaka.handpose');
loadPose('vulcan', POSE_FOLDER + 'vulcan.handpose');
}.bind(this));
handy-work
Documentation for using build/esm/handy-work.js or build/esm/handy-work.standalone.js
-
handy-work
-
.loadPose(name, url) ⇒
Promise.<void>
-
.getPose(name) ⇒
Promise.<Float32Array>
-
.setPose(name, pose) ⇒
Promise.<void>
- .resetHands()
- .dumpHands()
-
.generatePose(inputSources, referenceSpace, frame) ⇒
void
|Float32Array
- .update(inputSources, referenceSpace, frame, callback)
-
.loadPose(name, url) ⇒
Promise.<void>
handy-work.loadPose(name, url) ⇒ Loads a pose from the Web the recommended way of using this API
Kind: static method of handy-work
Returns: Promise.<void>
- Resolves once it has been completed
Param | Type | Description |
---|---|---|
name | string |
Name of the pose |
url | string |
URL of the pose to download and add to the registry |
Promise.<Float32Array>
handy-work.getPose(name) ⇒ Get a pose from the registry
Kind: static method of handy-work
Returns: Promise.<Float32Array>
- A copy of the pose from the registry in the worker
Param | Type | Description |
---|---|---|
name | string |
of the pose to fetch |
Promise.<void>
handy-work.setPose(name, pose) ⇒ Get a pose from the registry
Kind: static method of handy-work
Returns: Promise.<void>
- Resolves once the pose has been uploaded to the worker
Param | Type | Description |
---|---|---|
name | string |
of the pose to set |
pose | Float32Array |
Array buffer with the information about the current pose |
handy-work.resetHands()
Reset the current tracking performed automatically when the device returns from sleep
Kind: static method of handy-work
handy-work.dumpHands()
On the next frame save the current hand pose to a file
Kind: static method of handy-work
void
| Float32Array
handy-work.generatePose(inputSources, referenceSpace, frame) ⇒ Get a pose from the current hand position
Kind: static method of handy-work
Returns: void
| Float32Array
- The generated pose buffer for the pose.
Param | Type | Description |
---|---|---|
inputSources | XRInputSource |
Array of inputs you want to generate inputs for, requires a left AND right hand |
referenceSpace | XRReferenceSpace |
Current reference space |
frame | XRFrame |
Current active frame |
handy-work.update(inputSources, referenceSpace, frame, callback)
Kind: static method of handy-work
Param | Type | Description |
---|---|---|
inputSources | Array.<XRInputSource> |
The inputs you want to do pose tracking for |
referenceSpace | XRReferenceSpace |
The reference space for your scene |
frame | XRFrame |
The current active frame |
callback | function |
This gets called with an Array of Arrays with the poses and their distances |