The document discusses the Kinect system, specifically its capabilities in capturing depth images and estimating body poses using infrared technology and time-of-flight cameras. It describes the integration of gesture recognition through pattern matching algorithms and the use of the Iterative Closest Point (ICP) algorithm for geometric alignment. Additionally, it outlines the initialization and animation processes for avatars in gaming applications based on player movements.

1. THE MAGIC OF KINECT
CAPSTONE PROJECT
BENJOE VIDAL

2. WHAT THE KINECT DOES ?
• GET THE DEPTH IMAGE
• ESTIMATE THE BODY POSE
• OUTPUT ( APPLICATION)

3. HOW KINECT WORKS
• IR PROJECTOR
• PROJECTED LIGHT PATTER (SPECLE PATTERN - TIME OF
FLIGHT)
• IR SENSOR
• STEREO ALGORITHM - CONVERT TO DEPTH IMAGE
• SEGMENTATION
• DEPTH IMAGE

5. STEREO ALGORITHM
•used by Primesense and kinect
•image based rendering to get
depth image

6. DEPTH IMAGE
• is constructed by analyzing a specle pattern of
infrared laser light.
• inferring body position is a two - stage process.
- First compute a depth map( using structured
light)

7. STRUCTURED LIGHT
•The kinect used structured light
and machine learning algorithm to
estimate the body pose..

8. HOW KINECT GET THE IMAGE DATA
• A time-of-flight camera emits light signals and then measures how
long it takes them to return. That needs to be accurate to
1/10,000,000,000 of a second, the camera is able to differentiate
light reflecting from objects in a room and the surrounding
environment. That provides an accurate depth estimation that
enables the shape of those objects to be computed.

9. HOW WE INTEGRATE THE GESTURE RECOGNITION.
• We used OMEK - GAT to record gesture.
• Convert to XML data
• Pattern maching algorithm
• ICP - Iterative Closest Point

12. PATTERN MATCHING
• pattern maching involves having a database of recorded images
produced by gestures.
• Using this database you can compare the current drawing(made with
the positions of the tracked joint) with all the recorded drawings in
the database.
• Pattern matching algorithm, determines if one of the templates
matches the current drawing, and if so, then the gesture is detected.

13. ITERATIVE CLOSEST POINT
• The ICP (Iterative Closest Point) algorithm is widely used for
geometric alignment of three-dimensional models when an initial
estimate of the relative pose is known. This capability has potential
application to real-time 3D model acquisition and model-based
tracking.

14. • Iterative closest point algorithm to solve the
simultaneous assignment of silhouette points to a
body part and alignment of the body part.

16. • Awake is used to initialize any variables or game state before the
game starts.
• Bool beckonAlive = BeckonManager.beckonInstance.IsInit(); // this
line initialize the beckon SDK
• OMKStatus rc; M_currentSkeleton = Factory.createSkeleton(out rc);
this line will create skeleton every point of joint of avatar. If the
OMKStatus is successfully initialized InitializeHeirarchy(); function
will call to create skeleton

18. • after the skeleton successfully attached to the
avatar it will animate the movement of the avatar
same as the movement of the player in the game