Hacking the Kinect with GAFFTA Day 2


Published on

Day 2 of my workshop on hacking the Kinect.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Hacking the Kinect with GAFFTA Day 2

  1. 1. OPEN FRAMEWORKS + KINECT PART II @GAFFTAThursday, March 7, 13
  2. 2. The Kinect for XBOX - released in November 2010Thursday, March 7, 13
  3. 3. why it matters http://memo.tv/archive/kinect_why_it_matters Affordable Depth Sensor Calibrated Depth + RGB Sensor Nothing new technically, just making it accessible to a MUCH larger audience.Thursday, March 7, 13
  4. 4. Microphone Array has advanced voice capabilities Motorized tilt helps it to adjust to multiple environments RGB camera for other data ( Facial Tracking, Color tracking )Thursday, March 7, 13
  5. 5. what does the kinect see ? http://www.youtube.com/watch?v=dTKlNGSH9Po Laser Projector casts a grid of infrared dots Infrared lens interprets the size of the dots as distance and turns it into a “depth map” This depth map makes it easy to separate parts of a scene Computer learning is based off of these depth maps and large data setsThursday, March 7, 13
  6. 6. Hack The Kinect Method #1 libfreenect is the driver / library ofxKinect is the OF addon https://github.com/ofTheo/ofxKinect Simplest wrapper for kinect. ( also the first ) Good at getting raw image + point cloud data + motor control Extremely useful if combined with openCV cross platform - PC / Mac / LinuxThursday, March 7, 13
  7. 7. What is OpenCV ? Open Source Computer Vision Library A library that allows for real-time image manipulation and feature extraction. Including : Gesture recognition Facial recognition 2D + 3D features Motion Tracking Stereopsis Originally developed by Intel Labs in 1999 for CPU intensive programs. Released to the public in 2006. Now developed by Willow GarageThursday, March 7, 13
  8. 8. OpenCV with a webcamThursday, March 7, 13
  9. 9. OpenCV with a webcam First there is a color image from a camera. That color image is converted to grayscale so that it can be processed it faster. ( less channels = less data ) The background has a grayscale “snapshot” taken A difference is applied to new frames after the background snapshot OpenCV ContourFinder returns: a shape with many points, a bounding box, and centroidThursday, March 7, 13
  10. 10. OpenCV with a kinectThursday, March 7, 13
  11. 11. OpenCV with a kinect There is already a grayscale depth feed, that can be used to isolate a range using OpenCV Threshold Running OpenCV contour finder on this thresholded gray image gets us OpenCV blobs. In addition to blob tracking we also get the 3d position of each point in space. Also known as a “point cloud”Thursday, March 7, 13
  12. 12. LET’S START CODING https://github.com/benMcChesney/OF_Kinect_TutorialsThursday, March 7, 13
  13. 13. ofxKinect + ofxUIThursday, March 7, 13