Motion tracking demo by HTML5 Rocks
Last week, Kevin gave us the lowdown on the exciting new getUserMedia device API coming to browsers in the near future, partly thanks to the WebRTC specification (the set of JavaScript APIs for Real Time Communication). I can guarantee you we’ll be seeing more webcam and microphone-related experiments over the coming months!
Apart from the classic “Photo Booth” style apps we’re going to see lots of motion tracking and augmented reality experiments — and natively in the browser. Combine that with 3D graphics via WebGL and wow, imagine the possibilities…
So here are a couple of cutting-edge experiments that show off what will soon be possible. The first comes from the always awesome HTML5 Rocks website, run by Google. Check out this demo of motion tracking which uses a pre-recorded video instead of a webcam as the source. Or try out the real thing if you have Google Chrome Canary installed and have printed out the AR marker:
The main HTML5 Rocks article goes into more depth, but essentially it works by tracking a shape in 3D space via the webcam and aligning a 3D model with that space. Personally I’m very impressed with the smoothness, synchronisation and speed of the tracking, and I’m amazed this is even possible with little ol’ JavaScript.
The motion tracking is achieved using a JavaScript library called JSARToolkit (check out that page for more great demos). It’s a port of the Flash-based motion tracking library FLARToolkit, which is a port of the Java library NyARToolkit, which itself is a port of the C library ARToolkit! And I thought Box2D had been re-ported a lot!
A second experiment comes from Romuald Quantin (@soundstep), who has done many motion tracking apps before in Flash.
Romuald Quantin’s motion tracking demo
Check out his blog post and then play with his demo, which also uses a pre-recorded video. In this instance you can switch the image yourself so you can see that it’s dynamically placing the image onto the 3D plane in real time.
This demo doesn’t use JSARToolkit but uses 2D canvas and a clever perspective distortion trick, aka After Effects Corner Pin. This technique tracks the positions of the corners in the plane in the source video, and as the plane is moved around the target image is distorted using a perspective distortion library from Steven Wittens. An interesting technique with stunning results!
Update: Romuald has created a new experiment, this time with motion detection via the webcam. Check out his blog post and this demo video. Inspiring stuff!