Get Moving!Reading Kinect data from Actionscript
Kinect   ๏ By Microsoft. Previously Natal.   ๏ Sold to be used with the XBox.   ๏ "Controller-free gaming and     entertai...
You don’t have to buy a XBox to do any of thestuff shown here. You can buy the camera on itsown for about NZ$200.
1   Sensors
Kinect Sensor(s)  “The Kinect sensor is a horizontal bar connected to a small base with  a motorized pivot and is designed...
Demo!(CocoaKinect)
Sensors                                                            Depth magic chip                                       ...
Sensors, #1 RGB                                Image chip                                                          480    ...
Sensors, #2 Depth๏ Project a pattern of dots with a near  infrared laser over the scene. A  detector establishes the paral...
Depth. Uh? Depth computed by comparing distance between             Image 1 (uniquely identifiable) dot in image 1 and imag...
Sensors, #3 Infrared                                          Image chip                                                  ...
Sensors, #4 Voice                                      16 bits                                     @ 16kHz          Not co...
2   Open Kinect
OpenKinect
BW + RGB images  ๏ By default, provides depth    information as an RGB    image. Each color represents    a layer of depth...
Demo!(Isometric Depth)
Demo  ๏ Modified as3 server   • video2rgba: rbg to rgba mapping     (prefixing with 0xFF for alpha).   • depth2heatmap : dep...
Resolution vs meters  ๏ The depth data provided by kinect follow    some logarithmic curve :     • High resolution in the ...
Info / Resources
3   Tuio Kinect
Hands tracking  ๏ TuioKinect tracks simple hand gestures    using the Kinect controller and sends    control data based on...
Demo!
Using TUIO from AS     https://github.com/silviopaganini/openKinect-as3-experiments                         Documentation,...
Demo!(Musical Hands)
4   Advanced
Body-Pose ๏ What MS uses for his games is Machine   Learning   “The idea was that we would teach the computer with lots of...
Skeleton                 http://www.openni.org/downloadfiles/            http://www.keyboardmods.com/2010/12/howto-        ...
Frame differencing           http://actionsnippet.com/?p=2820
Blob Detection
UX
That’s it, folks!
Upcoming SlideShare
Loading in …5
×

Getmoving as3kinect

5,282 views

Published on

Slides from the presentation made to the Flash/Flex User Group in Wellington.

Introduction to the Kinect sensors and how to read their data with actionscript.

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,282
On SlideShare
0
From Embeds
0
Number of Embeds
1,694
Actions
Shares
0
Downloads
111
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Getmoving as3kinect

    1. 1. Get Moving!Reading Kinect data from Actionscript
    2. 2. Kinect ๏ By Microsoft. Previously Natal. ๏ Sold to be used with the XBox. ๏ "Controller-free gaming and entertainment experience"
    3. 3. You don’t have to buy a XBox to do any of thestuff shown here. You can buy the camera on itsown for about NZ$200.
    4. 4. 1 Sensors
    5. 5. Kinect Sensor(s) “The Kinect sensor is a horizontal bar connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or below the video display. The device features an "RGB camera, depth sensor and multi-array microphone running proprietary software", which provide full-body 3D motion capture, facial recognition and voice recognition capabilities. The depth sensor consists of an infrared laser projector combined with a monochrome CMOS sensor, which captures video data in 3D under any ambient light conditions. The sensing range of the depth sensor is adjustable“ Source: http://en.wikipedia.org/wiki/Kinect
    6. 6. Demo!(CocoaKinect)
    7. 7. Sensors Depth magic chip (data stream) Infrared Laser Projector Image chip RGB Camera (video stream) Infrared Camera Multi-array mic Motorised tilt
    8. 8. Sensors, #1 RGB Image chip 480 @ 30 frames/sec ๏ The RBG camera can be used for recording 640 video and assist with Horizontal field of view: 57º Vertical field of view: 43º Physical tilt range: 27º tasks such as facial recognition.
    9. 9. Sensors, #2 Depth๏ Project a pattern of dots with a near infrared laser over the scene. A detector establishes the parallax shift Depth magic chip of the dot pattern for each pixel. 480 12 bits (Data Array) @ 30 frames/sec 640 Range: 1.2m -3.5 m
    10. 10. Depth. Uh? Depth computed by comparing distance between Image 1 (uniquely identifiable) dot in image 1 and image 2. The more distant, the more depth. Magic Algorithm generates the pattern of dots that will be sent by the Infrared projector. In a sense a virtual image. Image 2 Infrared camera Depth magic chip Stereo Image from: http://gigantico.squarespace.com/ 336554365346/2007/6/21/virtual-reality-part-1.html
    11. 11. Sensors, #3 Infrared Image chip 480 @ 30 frames/sec ๏ Images from the random dot pattern projected by 640 the near-infrared laser. ๏ Switch off other infrared sources (hallogen, sun light) to see them.
    12. 12. Sensors, #4 Voice 16 bits @ 16kHz Not covered in this talk
    13. 13. 2 Open Kinect
    14. 14. OpenKinect
    15. 15. BW + RGB images ๏ By default, provides depth information as an RGB image. Each color represents a layer of depth. Red = close, blue = far away ๏ Good for Flash. Flash is good with picture but doesn’t excel at looping through numbers.
    16. 16. Demo!(Isometric Depth)
    17. 17. Demo ๏ Modified as3 server • video2rgba: rbg to rgba mapping (prefixing with 0xFF for alpha). • depth2heatmap : depth to rgba mapping. • depth2grayscale : depth to grayscale mapping, a scale from 0...2048 is reduced to one of 0...255. •
    18. 18. Resolution vs meters ๏ The depth data provided by kinect follow some logarithmic curve : • High resolution in the middle, low res close and far away from the camera ๏ A linear estimate, in centimeters, can be computed with: 0.1236 * tan(rawDisparity / 2842.5 + 1.1863) http://openkinect.org/w/index.php?title=Imaging_Information
    19. 19. Info / Resources
    20. 20. 3 Tuio Kinect
    21. 21. Hands tracking ๏ TuioKinect tracks simple hand gestures using the Kinect controller and sends control data based on the TUIO protocol. This allows the rapid creation of gesture enabled applications
    22. 22. Demo!
    23. 23. Using TUIO from AS https://github.com/silviopaganini/openKinect-as3-experiments Documentation, Parameters Note: getting 0 for z or Z in current implementation. http://nuigroup.com/?ACT=28&fid=33&aid=703_FKSuKMcUkTAJrwBGk2Qv
    24. 24. Demo!(Musical Hands)
    25. 25. 4 Advanced
    26. 26. Body-Pose ๏ What MS uses for his games is Machine Learning “The idea was that we would teach the computer with lots of different people of lots of different shapes and sizes in lots of different poses and the computer will learn how to distinguish one part of your body from another part,” he said. Since the Kinect camera includes depth information, it can distinguish between big people a long way away and small people up close.” http://blogs.wsj.com/tech-europe/2010/11/08/key- kinect-technology-devised-in-cambridge-lab/ For publicly available methods: http://openkinect.org/wiki/Research_Material
    27. 27. Skeleton http://www.openni.org/downloadfiles/ http://www.keyboardmods.com/2010/12/howto- kinect-openninite-skeleton.html http://www.freenect.com/kinect-skeleton-detection- work-in-progress http://www.modmykinect.com/2010/11/29/skeleton- detection-part-2/
    28. 28. Frame differencing http://actionsnippet.com/?p=2820
    29. 29. Blob Detection
    30. 30. UX
    31. 31. That’s it, folks!

    ×