The Kinect sensor is an input device by Microsoft that uses cameras and microphones to track body movements and recognize gestures and voices. It consists of an RGB camera, depth sensor using infrared light, and 4-microphone array. The depth sensor uses structured light to measure distances by projecting a pattern and analyzing its distortion. Kinect can track up to 20 joints of the human body in real-time using skeletal tracking. It has applications in 3D scanning, sign language translation, augmented reality, robot control, and virtual fitting rooms due to its low-cost depth sensing capabilities.

1. KINECT SENSOR
Group members
Bhoomit Morkar 14103180
Sachin Keniya 14103170
Vaishnavi Shah 14103182
Ears and Eyes of your application

2. Content
 Introduction
• Basic Construction
• Basic Specifications
• Kinect’s Potential and applications
 Working of Kinect
• Classification of Depth Measurement Techniques
 Triangulation
 Principles of TOF Imaging
 Principle of structured light
• Skeletal Tracking
 Applications
 References

3. Introduction
• It was Designed by Microsoft for the Xbox 360 video game console and windows PC’s.
• While making the Kinect sensor , Microsoft’s aim was to enable interactions between
users and a game without the need to touch a controller.
• Kinect is an RGB-D sensor providing synchronized color and depth images.
• It’s greatest differentials is depth sensing technology that could be extended far beyond
gaming and at a much lower cost than traditional 3-D cameras.

4. Why Kinect is revolutionary ?
• Kinect has proven to be an incredible piece of technology.
• It is the first practical natural user interface for machines communicating with people.
• It is combining acoustical and visual sense.
• It is recognizing full body movements, even complicated ones.
• It finds application not only in games but also in the field of medical sciences and
robotics.
• After selling a total of 8 million units in its first 60 days, the Kinect holds the Guinness
World Record of being the "fastest selling consumer electronics device".

5. Basic Construction

6. Power
Comes
from the
Sum
• The Kinect Sensor is the outcome of several devices(or sensors)
working together.
• It is comprised of a color camera (also known as RGB), a depth sensor,
a 4-microphone array, and a motorized tilt.
• The depth sensor, in itself, consists of an infrared (IR) projector and an
IR camera.
• Due to that Kinect being a motion sensing input device, able to
capture, track, and interpret full-body movements(3D), and recognize
gestures and voices.

7. • The depth sensor is a monochrome complementary metaloxide semiconductor (CMOS)
sensor.
• One of Kinect’s greatest differentials is its relatively low cost depth sensor.
• The depth-sensing technology is licensed from the Israeli company PrimeSense. Although
the exact technology is not disclosed, it is based on the structured light principle.
• The IR projector is an IR laser that passes through a diffraction grating and turns into a set
of IR dots.
• The sensor has an RGB (red-green-blue) camera for color video.
• It has four microphones that enable your application to respond to verbal input, in
addition to responding to movement.

8. Basic Specifications
Hardware
• Depth resolution : 640x480 px
• RGB resolution : 1600x1200 px
• FrameRate : 60 FP
Software
• Depth;
30 FPS: 80x60 , 320x240 , 640x480
• Color;
12 FPS : 1280x980 RGB
15 FPS : Raw YUV 640x480
20 FPS : 640x480

9. Kinect’s Potential and applications
Depth sensor Skeletal tracking Microphone
array
Background removal
Object recognition
Multi-user
Easy Gesture Recognition
Sound source detection
Speech recognition

10. How it works
?

11. Audio input
Microphone array
Sound threshold
How does
Kinect hear ?
Track me !
Or
Speak louder .

12. what does it
see ?

13. Near mode depth
ranges
Default mode depth
ranges
Angle of vision
(RGB-D)
Area under Observation

14. Classification of Depth Measurement Techniques

15. Triangulation
• In trigonometry and geometry,
triangulation is the process of
determining the location of a
point by forming triangles to it
from known points.
• Here rather than measuring
distances to the point directly
as in trilateration; we use of
both angles and distance

16. Basic Mathematical model for Kinect

17. Principles of TOF Imaging
• Time-of-Flight (TOF) Imaging refers to the process of measuring the depth of a scene
by quantifying the changes that an emitted light signal encounters when it bounces
back from objects in a scene.

18. Pulsed ModulationContinuous Wave Modulation

19. Principle of structured light
• Project a known light pattern into the 3D scene, viewed by
cameras.
• Distortion of light pattern allows computing the 3D
structure.
• This pattern has to be unique per position in order to
recognize each point in the pattern.
• Kinect uses pseudo random pattern.

20. speckles
Based on distance and orientation
First Region
High accurate
depth.
(0.8 – 1.2 m)
Second Region
Medium accurate
depth.
(1.2 – 2.0 m)
Third Region
Low accurate
depth.
(2.0 – 3.0 m)

21. Depthobserved
pattern
Calibration
IR camera
observes
the scene
known
pattern
projected
3D map of
the
beginning
frame
New 3D
map of the
scene
Renew 3D
map of the
scene
Between
projector
and camera
&
x-direction
speckle shift

22. Precision
Spatial
x/y resolution
3mm @ 2m distance
Depth
Z resolution
1cm @ 2m distance
10 cm @ 4m distance
operation range
0.8m ~ 4m | 0.5m ~ 3m

23. Skeletal Tracking
In the beginning,it was just
hands;
• Head and hands,
• Added shoulder and elbows.
Today!
• It can track 20 joints of our body and
rans fast.

24. Process
Modes

25. Applications
• The Kinect fusion project allows a user to use an off-the-shelf Kinect camera as a 3D
scanner for producing high-quality 3D scans of small or large objects.
• Researchers in China, in collaboration with Microsoft Research, developed the Kinect
Sign Language Translator, a system that can translate sign language into spoken and
written language in near real time.
• The Holoflector is an augmented reality mirror. Behind the mirror is a large LCD
monitor and right above the mirror is the Kinect.
• Combining a Kinect, a few hacking tools and a 3D environment, Kinectasploit allows a
user to break into a computer’s security system using body gestures. The system works
almost like a video game. In one demo, a hacker ‘shoots’ at Wi-Fi targets to break into
them.

26. • UBI Interactive developed their software UBI which combines your Windows 8
computer to Kinect and projector to create an interactive touchscreen off any surface.
• In another case of copying from the movies, many users have hacked their Kinect to
allow users to control robots via body movement.
• We can pick out cloths and try them all out virtually with the help of Fitnect, an
interactive dressing room.

27. References
 http://mms.ecs.soton.ac.uk/2014/papers/2.pdf
 http://lshao.staff.shef.ac.uk/pub/KinectReview_TC2013.pdf
 https://www.microsoft.com/en-us/research/wp-
content/uploads/2016/02/Microsoft20Kinect20Sensor20and20Its20Effect20-20IEEE20MM202012.pdf
 http://mms.ecs.soton.ac.uk/2014/papers/2.pdf
 http://campar.in.tum.de/twiki/pub/Chair/TeachingSs11Kinect/2011-DSensors_LabCourse_Kinect.pdf
 http://www.slideshare.net/MatteoValoriani/introduction-to-kinect-update-v-18?qid=e70a04cc-282d-4c11-a0d0-
cff7f58c2435&v=&b=&from_search=9
 http://www.slideshare.net/rojizofrio/xbox-one-kinect?qid=e70a04cc-282d-4c11-a0d0-
cff7f58c2435&v=&b=&from_search=15

28. Videos
 https://youtu.be/uq9SEJxZiUg
 http://youtu.be/bRgEdqDiOuQ?t=10m16s
 http://www.google.com/patents/US20090
 http://campar.in.tum.de/twiki/pub/Cha... (p. 33)
 http://en.wikipedia.org/wiki/Range_im... (Stereo triangulation)
 https://www.youtube.com/watch?v=-RELJD0LBg8&spfreload=5
 https://youtu.be/uq9SEJxZiUg
 http://www.hongkiat.com/blog/innovative-uses-kinect/