Fizzy Cam is a fun little tech mashup from our very own CJS team member, Tim Holman. It uses webgl with three.js, custom webgl shaders, WebRTC and headtrackr for headtracking. Check it out and see yourself in a world of fun floaty squares!

Fizzy Cam
by Tim Holman

[UPDATE] Einar Öberg just sent more information – his company North Kingdom made this with the help of Dinahmoe. There’s also this very cute promo video – this project has very high production values!

Even if this latest game from Google’s Creative Lab (Ursine Diversion Division) didn’t showcase the kinds of things you can get up to with WebRTC and even if this game didn’t have some extremely addictive little tweaks, you’d still want to play Cube Slam simply because Bob the bear is so darned cute. Seriously.

The first few times you play, you might find yourself deliberately losing points just because Bob gets grumpy and a bit sad when you win a point. Lose a couple of times and he’s okay.

If you keep playing, though, you’ll find a nicely implemented variation of possibly the most classic of classic games, Pong. There are a enough power-ups, handicaps and obstacles to keep you playing for as long as you can stand making Bob the Bear sad. And that’s not even mentioning the fact that you can play two-player across the Internet allowing you to slam the cube into your opponents face via webcam. Throw in the fact that the sound is generated dynamically using the Web Audio API and there’s hardly a new API not covered.

Find out more at the Chrome Blog or play at cubeslam.com, made by North Kingdom

Last week, Kevin gave us the lowdown on the exciting new getUserMedia device API coming to browsers in the near future, partly thanks to the WebRTC specification (the set of JavaScript APIs for Real Time Communication). I can guarantee you we’ll be seeing more webcam and microphone-related experiments over the coming months!

Apart from the classic “Photo Booth” style apps we’re going to see lots of motion tracking and augmented reality experiments — and natively in the browser. Combine that with 3D graphics via WebGL and wow, imagine the possibilities…

So here are a couple of cutting-edge experiments that show off what will soon be possible. The first comes from the always awesome HTML5 Rocks website, run by Google. Check out this demo of motion tracking which uses a pre-recorded video instead of a webcam as the source. Or try out the real thing if you have Google Chrome Canary installed and have printed out the AR marker:

The main HTML5 Rocks article goes into more depth, but essentially it works by tracking a shape in 3D space via the webcam and aligning a 3D model with that space. Personally I’m very impressed with the smoothness, synchronisation and speed of the tracking, and I’m amazed this is even possible with little ol’ JavaScript.

The motion tracking is achieved using a JavaScript library called JSARToolkit (check out that page for more great demos). It’s a port of the Flash-based motion tracking library FLARToolkit, which is a port of the Java library NyARToolkit, which itself is a port of the C library ARToolkit! And I thought Box2D had been re-ported a lot!

A second experiment comes from Romuald Quantin (@soundstep), who has done many motion tracking apps before in Flash.

Romuald Quantin’s motion tracking demo

Check out his blog post and then play with his demo, which also uses a pre-recorded video. In this instance you can switch the image yourself so you can see that it’s dynamically placing the image onto the 3D plane in real time.

This demo doesn’t use JSARToolkit but uses 2D canvas and a clever perspective distortion trick, aka After Effects Corner Pin. This technique tracks the positions of the corners in the plane in the source video, and as the plane is moved around the target image is distorted using a perspective distortion library from Steven Wittens. An interesting technique with stunning results!

Update: Romuald has created a new experiment, this time with motion detection via the webcam. Check out his blog post and this demo video. Inspiring stuff!

Last month, Opera gave us a preview of their work on the getUserMedia method which allows users to interact with HTML5 applications via webcam and microphone. Now that Chrome has also started to support this feature on its development channel build (Chrome Canary), we can hope to reach a wider audience that can use this technology in the future.

To start, you will first need an install of Opera Labs or Chrome Canary (with MediaStream enabled). Update: The latest versions of Opera and Chrome now support this feature! Got it? Good. On with the code!

var n = navigator,
    is_webkit = false;
 
function onSuccess(stream) {
    var output = document.getElementById('output'), // a video element
    source;
 
    output.autoplay = true; // you can set this in your markup instead
 
    if (!is_webkit) {
        source = stream;
    }
    else {
        source = window.webkitURL.createObjectURL(stream);
    }
 
    output.src = source;
}
 
function onError() {
    // womp, womp =(
}
 
if (n.getUserMedia) {
    // opera users (hopefully everyone else at some point)
    n.getUserMedia({video: true, audio: true}, onSuccess, onError);
}
else if (n.webkitGetUserMedia) {
    // webkit users
    is_webkit = true;
    n.webkitGetUserMedia('video, audio', onSuccess, onError);
}
else {
    // moms, dads, grandmas, and grandpas
}

At the time of this writing, this method is not fully standardized. As a result, Opera and Webkit differ slightly. We create a flag, is_webkit, to check which browser we are in and handle these differences accordingly. We start by checking for support with what will (hopefully) become the standard method—getUserMedia, which is a property of the Navigator object:

if (n.getUserMedia) {
    // opera users (hopefully everyone else at some point)
    n.getUserMedia({video: true, audio: true}, onSuccess, onError);
}
else if (n.webkitGetUserMedia) {
    // webkit users
    is_webkit = true;
    n.webkitGetUserMedia('video, audio', onSuccess, onError);
}

The first parameter states which type of media we want access to. Opera takes an object, while WebKit, relying upon an older version of the spec, accepts a comma-delimited string. Update (May 17, 2012): Canary has been updated to also accept an object parameter. Update (August 16): Looks like this is a regression—Chrome takes a string once again. The final parameters in both cases are callbacks for successes and errors.

function onSuccess(stream) {
    var output = document.getElementById('output'), // a video element
        source;
 
    output.autoplay = true; // you can set this in your markup instead
 
    if (!is_webkit) {
        source = stream;
    }
    else {
        source = window.webkitURL.createObjectURL(stream);
    }
 
    output.src = source;
}

The success callback gives us access to a Stream object which, with an extra bit of feature-sniffing magic, we pass on to the src attribute of our video element. It is important that your video element has its autoplay attribute set, or you will only see the first frame of video.

Those wishing to future-proof their implementations would do best to check out the gUM Shield, which anticipates the use of browser prefixes for other browser implementations as they come out.

The potential of getUserMedia is larger than most people realize! While there are a few great demos currently out there, most of them are focused on personal use. This is a great start and it will be exciting to see how it plays out for games and chat applications. Did someone say native browser chat roulette?