An introduction to what people are doing with Kinect. Given at Microsoft Social Computing Symposium 2011 at ITP in New York. http://scs.labforsocialcomputing.net/

1. What can you do
with a Kinect?
Matt Biddulph, Nokia

2. Kinect hacking is the latest in a long line of re-purposed consumer electronics. These days
it’s cheaper for companies to build devices by gluing together standard components than it is
to create proprietary hardware. Their economies of scale give hackers access to existing
hardware at new pricepoints.

5. The Kinect’s RGB camera captures a normal view of a scene

6. Its 3D infrared cameras capture a depth-map of the same scene - darker pixels are closer to
the camera

7. Rotate that depth-map in 3D and you can see how it creates all sorts of possibilities for
understanding a scene.

8. Software
XBox software performs:
depth mapping
skeletal analysis
gestures
face recognition
voice recognition with noise suppression
All these features are built into the closed XBox SDKs used by game developers. These are
NOT available to Kinect hackers.

9. Works with Processing, Cinder, Open
Frameworks etc
Full framerate camera + depth map
Microphone features not available yet
Lacks the XBox software features...
... but works well with OpenCV for
computer-vision image analysis
But OpenKinect is creating drivers, integrations and libraries for high-level systems such as
Processing to enable the software side of Kinect hacking.

10. “Kinect is making nothing
which wasn't already
technically possible, possible.”
http://memo.tv/kinect_why_it_matters

11. “It is just making it accessible,
not just in terms of price, but
also in terms of simplicity and
ease.”
http://memo.tv/kinect_why_it_matters

12. !2! 3,.-0-(-$0)$.)4,/1&#,4,0()#,5-$01)
!
?*#2A&*1*$"'&*G),$2H'%*-)$)"),$' ?*#2A&*1*$"'&*G),$2H'L,+#&'%)1*$2),$2'
C*G),$'EH'I#&J'&*%''''C*G),$'FH'K&)G7"'&*%' C*G),$'EH'MEN6'''C*G),$'FH'MFN6 '
!
"#$%%%!&'('!")**( !
$%&'(%)*+&',-+.,-)%,/)0.1,2+,3)4)526'(,'2)7%&'(%-)899:9999#;)99<;)9==)%,/)9=!>)
' !
!
?*#2A&*1*$"'&*G),$2H'&*L&*2*$"#"),$',:*&'L)9*+'-)*+%' ?*#2A&*1*$"'&*G),$2H'&*L&*2*$"#"),$',:*&'L)9*+'-)*+%'
!
!"#$%#&%'-)*+%',-':)*/'O#1*&#2'3456'378'9'546'3:88' ;)%*'-)*+%',-':)*/'O#1*&#2'3<=6'378'9'><6'3:88'
"#$%%%!&'('!")**( !
!
!
!
! "#$%&'($&)*
!+, -&.#/&*0(.#'1(2'1*&'3*"24'5('6*7*89:*$&.#/&1*;<<=<<<<,>*<<?>*<,=*&'3*<
' !
!
!
!"#$%&'()*(+,-,( .&('&/&'0&(#1&('231#(#$(456&(#&"172"58(58#&'5#2$7/(92#1$:#(;'2$'(7$#2"& ( ((((<5#5(=1&&#(>&0?()?@(
You can get higher framerates and more precise 3D imaging from industrial cameras, but
Kinect costs around $150 and is available on the high-street. It’s cheap enough to make silly
!
things with.
)*(+,-,( .&('&/&'0&(#1&('231#(#$(456&(#&"172"58(58#&'5#2$7/(92#1$:#(;'2$'(7$#2"& ( ((((<5#5(=1&&#(>&0?()?@(

13. Applications
UI control
Gestures and skeletal body tracking
Robotics
3D Scene Reconstruction
Visualisations and art
Realworld multitouch

14. Showreel

15. http://kinecthacks.net/kinect-background-deletion-and-depth-interaction/

16. http://kinecthacks.net/real-time-people-detection-and-tracking-with-multiple-kinect-
cameras

17. Signlanguage Recognition
http://kinecthacks.net/american-sign-language-recognition-using-kinect

18. http://kinecthacks.net/transform-yourself-into-a-japanese-superhero/

19. http://kinecthacks.net/body-dysmorphic-disorder

20. http://kinecthacks.net/kinect-invisibility

21. Robot vision
http://singularityhub.com/2010/11/17/hacked-irobot-uses-xbox-kinect-to-see-the-
world-obey-your-commands-video

22. http://kinecthacks.net/quadrotor-autonomous-flight-and-obstacle-avoidance

23. http://kinecthacks.net/kinect-piano