Hire Me →
Home / Showcase / X-Ray Hand
PLAYABLE DEMO · WEBAR

X-Ray Hand

// REAL-TIME HAND TRACKING · MEDIAPIPE · THREE.JS · WEBGL
MediaPipeThree.jsTypeScriptWebARWebGLHealthcare
X-Ray Hand — an open palm held to the camera with its bones drawn inside it, the pose guide asking for fingers together and the hold countdown on one
X-Ray Hand — the metacarpals stage, with the second metacarpal tapped and lit green beside its name and description
X-Ray Hand — the phalanges stage, an index finger pointing with distal, middle and proximal labelled up the chain
X-Ray Hand — carpal tunnel mode, a pulsing ring at the wrist labelled pressure on the nerve
X-Ray Hand — the median nerve inflamed, tingling marked on index and middle while the little finger reads not here, ulnar nerve
X-Ray Hand — the closing stage, the little finger marked spared on the ulnar nerve beside the median territory and the test accuracy note
01 / 06
▶TRY X-RAY HAND← All ProjectsNeeds a camera · works on desktop and mobile
TRACKING
MediaPipe · WebAssembly
RENDERER
Three.js · WebGL
BUILD SIZE
3.5 MB with voiceover
SCOPE
Two guided modes

Hold your hand up to the camera and watch your own skeleton move with it, live in a browser tab. There is no app to install and no video ever leaves the device. Built to show what browser AR can do for patient education and HCP engagement without asking anyone to download anything.

The Idea

X-Ray Hand tracks your hand through the camera and draws real skeletal anatomy over it in real time. The bones sit inside your hand rather than on top of it, so your fingers pass in front of each other properly when you turn them. An X-ray slider controls how much of the camera feed shows through, and you can freeze a frame and share it.

There are two modes. Explore is a five-stage bone tour: spread your fingers, flatten your hand, make a fist, point, thumbs up. Each pose reveals that group of bones and names it, and you can tap any bone at any point to find out what it is. All 27 are in there.

Carpal tunnel mode is the one built for a clinical audience. You hold your hand up, it counts down from three, and the median and ulnar nerves appear with the tunnel resolving around them. Pressure rises and the median territory inflames: thumb, index, middle, and the thumb side of the ring finger. The little finger stays calm, because it runs on the ulnar nerve and that never enters the tunnel. You see the spared little finger rather than read about it.

The copy never tells anyone what they have. Your wrist is neutral while you hold your hand up, so nothing is actually compressing your nerve, and the demo says so on screen. It describes the mechanism in general terms and attributes Phalen's manoeuvre to clinicians, where it belongs. Get that wrong in front of a medical audience and the whole thing loses credibility.

Under The Hood

TRACKING
MediaPipe HandLandmarker running in WebAssembly, 21 landmarks per hand and up to two hands at once. One-Euro filters on every landmark with confidence gating, so the skeleton settles instead of jittering. An fps watchdog drops detail rather than dropping frames on a slower phone.
RIG
The 21 landmarks solve into 19 rigid bone segments per hand, each with a position, rotation and length. That is the only contract between tracking and rendering, so every layer reads the same transforms and any new layer deforms with the hand for free.
OCCLUSION
A weight-painted skinned mesh authored in Blender, 1.4k tris, rendered depth-only with a view-space bias. It is the thing that puts the bones inside the hand instead of floating over it. Depth-only ellipsoids cover the gap while the GLB loads, so it never looks broken on a slow connection.
ANATOMY
Median and ulnar nerve paths extracted from Z-Anatomy (CC BY-SA 4.0, derived from BodyParts3D) and retargeted into landmark space with up to four skinning influences per sample. It is real anatomy deforming with your hand, not an approximation drawn on top.
POSE DETECTION
Named poses return a continuous 0-1 match built from finger extension, splay and how upright the hand is, with the weakest requirement governing. That means the guide can name the one thing still wrong: "spread your fingers wider" rather than just refusing to advance.
AUDIO
ElevenLabs voiceover across 16 clips, split out of one master export and cut on measured silences so no clip loses a syllable. Stage timings are generated from the actual clip lengths, so the Next button never appears mid-sentence.
PRIVACY
Camera frames go to WebAssembly and WebGL and nowhere else. No upload, no server, no account, and it is stated on screen rather than buried in a policy. That matters in a pharma environment.

Where It Stands

Both modes are finished and run on a phone or a laptop. It needs a camera and an HTTPS page, and that is the whole install.

I built it as a demonstration piece rather than a product. It shows the pipeline I would use for patient education, an HCP engagement tool or a congress stand: real anatomy, tracking that runs on the device, and nothing for the visitor to download. The medical framing is deliberately conservative throughout, which is the part that usually gets skipped.

HOLD YOUR HAND UP.
Runs in the browser on a phone or a laptop. Allow the camera, hold your hand up, and everything stays on your device.
▶TRY X-RAY HAND
← Nanobot Nautilus ↩ Back to Showcase