Poor Man's Project Natal

Cap Area .NET
June 23, 2009
Poor Man’s Project Natal
Frank La Vigne
Lead Designer
Applied Information Sciences
http://www.franksworld.com
@Tableteer

What is Project Natal?

Video Demo
You have to see this in action to believe it.

The Future?

Why Should We Care?

Going Beyond GUI
NUI

Commercial Successes

Multi Touch
Opportunities on the Horizon
Surface

Steven Spielberg Has a Budget

We Don’t

Their Hardware

Our Hardware

Deconstructing Project Natal
Facial Recognition
Speech
Recognition
Motion
Recognition
Image Scanning

Let’s Get Experimenting

WPF
Speech Recognition
Speech Synthesis
Touchless SDK
Motion Capture
Gesturing
Replicating Project Natal in .NET

Open Source
Free
Developed by MSFT employee in his “spare time”
Uses any web cam*
Does not share any code withProject Natal
http://www.codeplex.com/touchless
Touchless SDK
* Web cam must support DirectX

Computing a “FacePrint”
Distance between the eyes
Width of the nose
Depth of the eye sockets
The shape of the cheekbones
The length of the jaw line
Facial Recognition
As of today, there is no SDK for facial recognition in .NET,
But there’s plenty of academic research, white papers and
inspiration, so have a go hero. ;)

Johnny Lee’s Wii Remote Projects

Touchless SDK Demo
Exploring the Touchless SDK

Speech Demo
Interacting just with your voice

The desktop metaphor is a relic.
You don’t need expensive hardware to get started.
You don’t have to wait for Project Natal to get your feet wet.
Getting this to work in all environments may be the biggest challenge for Project Natal’s engineers.
Take Aways