Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Connect with your
Wow!
Motorized Tilt
RGB Camera
3D Depth Sensors
Microphone Array
openkinect.org
www.openni.org
kinectforwindows.org
Kinect for Windows
That’s not USB
You’ll need this
Xbox Kinect
What you Need
SDK
Demo 0
Hello Tilt
BGR32 Format
width in pixels
heightinpixels
Stride = Bytes Per Row
= bytes per pixel * width
= 4 * width
4 Bytes Per pixel...
Demo 1
Hello Colorful World
Depth data
500mm 500mm 500mm 500mm
2000mm 2030mm 2010mm 2000mm
500mm500mm500mm2500mm
Depth data
Max resolution for raw depth data is 640 x 480
2 bytes per pixel of depth data (short)
11011110
pixel pixel pix...
Demo 2
Depth
Depth + Player Data
500mm 500mm 500mm 500mm
2000mm 2030mm 2010mm 2000mm
500mm500mm500mm2500mm
player 1
player 1
player 1pl...
Demo 3
Depth + Player Data
Skeletal Tracking
skeletonFrame.Skeletons
where skeleton.TrackingState == SkeletonTrackingState.Tracked
skeletonFrame.Skeletons
from skeleto...
hand_left
wrist_left
elbow_left
shoulder_left
hand_right
wrist_right
elbow_right
shoulder_right
hip_left
head
hip_right
kn...
Demo 3
Skeletal Tracking
Composites
+Color +
Composites
Color
Color
+
+
+
MapDepthFrameToColorFrame
MapDepthToColorImagePoint
MapDepthToSkeletonPoint
MapSkeletonPointT...
Demo 4
Green Screen
Kinect Audio
Echo Cancellation
Echo Cancellation
EchoCancellationMode: None, CancellationOnly, CancellationAndSuppression
int EchoCancellationSpeakerInde...
Noise Suppression
Noise Supression
bool NoiseSuppression – true, false
Automatic Gain Control
Automatic Gain Control
bool AutomaticGainControl – true, false
Beam Forming
Beam Forming
-0.875 radians
-50 degrees
0.875 radians
50 degrees
.0175 radians
10 degrees
Demo 5
Speech Recognition
Audio @!%!#!! Gotchas
Use Microsoft.Speech not System.Speech
Must run in an MTA
AEC SystemModes crash if no speaker output...
Demo 6
Beam Forming
Kinect SDK 1.5!
Seated Skeletal Tracking
Face Tracking Joint Rotation
Kinect Studio
Creative Commons Acknowledgments
• Flickr by Ian Lewis
• Flickr by major nelson
• Flickr by markodesign
• Flickr by stepha...
Connect with your Kinect
Connect with your Kinect
Connect with your Kinect
Connect with your Kinect
Connect with your Kinect
Connect with your Kinect
Connect with your Kinect
Connect with your Kinect
Connect with your Kinect
Upcoming SlideShare
Loading in …5
×

Connect with your Kinect

249 views

Published on

Slides from my 2012 talk at North Dallas .NET User Group on Kinect programming

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Connect with your Kinect

  1. 1. Connect with your
  2. 2. Wow!
  3. 3. Motorized Tilt RGB Camera 3D Depth Sensors Microphone Array
  4. 4. openkinect.org www.openni.org kinectforwindows.org
  5. 5. Kinect for Windows
  6. 6. That’s not USB You’ll need this Xbox Kinect
  7. 7. What you Need SDK
  8. 8. Demo 0 Hello Tilt
  9. 9. BGR32 Format width in pixels heightinpixels Stride = Bytes Per Row = bytes per pixel * width = 4 * width 4 Bytes Per pixel Byte Array:
  10. 10. Demo 1 Hello Colorful World
  11. 11. Depth data 500mm 500mm 500mm 500mm 2000mm 2030mm 2010mm 2000mm 500mm500mm500mm2500mm
  12. 12. Depth data Max resolution for raw depth data is 640 x 480 2 bytes per pixel of depth data (short) 11011110 pixel pixel pixel 00001 1110101000000 1110101000001 depthInMM = pixel >> DepthImageFrame.PlayerIndexBitmaskWidth 000 000 000
  13. 13. Demo 2 Depth
  14. 14. Depth + Player Data 500mm 500mm 500mm 500mm 2000mm 2030mm 2010mm 2000mm 500mm500mm500mm2500mm player 1 player 1 player 1player 1player 1 player 1 player 1 SkeletonStream.Enable(); var player = pixel & DepthImageFrame.PlayerIndexBitmask; 1101111000001010
  15. 15. Demo 3 Depth + Player Data
  16. 16. Skeletal Tracking
  17. 17. skeletonFrame.Skeletons where skeleton.TrackingState == SkeletonTrackingState.Tracked skeletonFrame.Skeletons from skeleton in skeletonFrame.Skeletons where skeleton.TrackingState == SkeletonTrackingState.PositionOnly
  18. 18. hand_left wrist_left elbow_left shoulder_left hand_right wrist_right elbow_right shoulder_right hip_left head hip_right knee_left knee_right ankle_rightankle_left foot_left foot_right hip center spine shoulder_center Skeleton.Joints[JointType] Joint.Position (X,Y,Z) Skeletons are mirrored X, Y, Z in meters Origin(0,0,0) = center of camera. +Z is out +X is to the right +Y is up
  19. 19. Demo 3 Skeletal Tracking
  20. 20. Composites +Color +
  21. 21. Composites Color Color + + + MapDepthFrameToColorFrame MapDepthToColorImagePoint MapDepthToSkeletonPoint MapSkeletonPointToColor MapSkeletonPointToDepth
  22. 22. Demo 4 Green Screen
  23. 23. Kinect Audio
  24. 24. Echo Cancellation
  25. 25. Echo Cancellation EchoCancellationMode: None, CancellationOnly, CancellationAndSuppression int EchoCancellationSpeakerIndex: Index of speaker to cancel
  26. 26. Noise Suppression
  27. 27. Noise Supression bool NoiseSuppression – true, false
  28. 28. Automatic Gain Control
  29. 29. Automatic Gain Control bool AutomaticGainControl – true, false
  30. 30. Beam Forming
  31. 31. Beam Forming -0.875 radians -50 degrees 0.875 radians 50 degrees .0175 radians 10 degrees
  32. 32. Demo 5 Speech Recognition
  33. 33. Audio @!%!#!! Gotchas Use Microsoft.Speech not System.Speech Must run in an MTA AEC SystemModes crash if no speaker output. Must plug Kinect into Wall DevicePairGuid / Qualitymetrics = crash Should run in ThreadMode Priority Highest?
  34. 34. Demo 6 Beam Forming
  35. 35. Kinect SDK 1.5! Seated Skeletal Tracking Face Tracking Joint Rotation Kinect Studio
  36. 36. Creative Commons Acknowledgments • Flickr by Ian Lewis • Flickr by major nelson • Flickr by markodesign • Flickr by stephantom • Flickr by Abstract Machine • Flickr By Aixo.co.kr • Flickr By Andresmh • Flickr by Antony Bennison • Flickr by Ars Electronica • Flickr By bm.iphone • Flickr by campuspartymexico • Flickr by Chris Jagers • Flickr By ci_polla Paola Sucato • Flickr by creepyhalloweenimages • Flickr by cvander • Flickr by IMAL.org • Flickr by Institut Douglas • Flickr by Joshua and AmberFlickr by Logon Sakai • Flickr by openexhibits • Flickr by Paul_Clarke • Flickr by petyr.rahl • Flickr by pt • Flickr by Stilldavid • Flickr by y3rdua • Flickr by yesyesnono • Flickr by yum9me • Flickr by Jeff Howard • Flickr by jhscreenwm • Flickr by Kyle McDonald • Flickr by Morten Rand-Hendriksen • Flickr by Stuart Herbert • Flickr by Visiophone • Flickr by orkomedix • Michael Zöllner and Stephan Huber at the University of Konstanz • Openclip by J_Alves • Openclip by jetxee • Openclip by molumen • Openclip by shokunin • Openclip by Jhnri • Openclip by Machovka • Adafruit image of Hector Martin Thanks!!

×