Pranav Mistry, a PhD student at MIT Media Lab, developed the sixth sense technology. It is a wearable gestural interface that augments the physical world with digital information using hand gestures detected by a camera. The system includes a camera, projector, mirror, and smart phone to allow interactions like making calls, viewing maps, taking photos, and accessing information by pointing at physical objects or surfaces. While promising improved access to information, challenges include privacy and visibility of projections depending on lighting conditions.

1. Muhammad Hassan 1
12-Jan-13

2. Muhammad Hassan 2
12-Jan-13

3. Inventor of sixth sense
Pranav Mistry, of Indian origin, a PhD
student in the Fluid Interfaces Group
at the MIT Media Lab is the mastermind
behind the sixth sense technology.
Liyan change and Paula Aguilera which
are students at MIT also done important
role.
Muhammad Hassan 12-Jan-13 3

4. Introduction
• Sixth Sense is a wearable gestural interface that
augments the physical world around us with
digital information.
• Technology that plays with Human gestures to
make the world more interactive and workflow
much easier.
• Sixth sense is basically connection and relation
between physically things and digital things just
simple gesture with human body like hand and
foot gesture by using pocket projector, smart cell
phone, mirror, colour marker and camera.
Muhammad Hassan 12-Jan-13 4

5. Why sixth sense
• Because of sixth sense Physical world is
painted with the digital information.
• You can carry your digital world with you
wherever you go.
• You can start with any wall or any surface as
an interface even your palm.
Muhammad Hassan 12-Jan-13 5

6. Contents
1. Camera
2. Colour marker
3. Smart phone
4. Projector
5. Mirror
Muhammad Hassan 12-Jan-13 6

7. Working of contents
Camera:-
Camera captures the views of user’s hand gesture. It identifies the
images, gestures that user make himself. It also analyses the
digital pixels. Then camera sends all information to the smart
phone for processing. It is also called digital eye. It’s working as
same as our eyes. Our eyes capture objects and send to the brain
for processing.
The camera not only recognizes, but also registers the user’s
physical hand movements using computer vision based
technology. For the camera to recognize these hand gestures the
user is asked to wear “marker caps,” known as the visual tracking
fiducially, on the tips of his fingers.
Muhammad Hassan 12-Jan-13 7

8. Colour marker:-
Four types of colour marker red, yellow, green
and blue at the tip of user’s hand help the
webcam to recognize hand’s gesture. Every
action of colour markers interpreted into gesture.
Muhammad Hassan 12-Jan-13 8

9. Smart phone:-
Mobile translates colour marker’s action in form of
gesture that capture by cam at the tip of user. Mobile
behaves as a computer for processing data. Nokia n95
smart phone is used during making six senses (running
Symbian O.S s60 edition).
Muhammad Hassan 12-Jan-13 9

10. Projector:-
The main purpose of projector to view information that
processed through mobile on any surface like person.
Projectors itself have 3 hour limited battery. In future
day’s computer scientists are trying to lounged solar
projector. Pocket projector “pk 101 optoma” is used for
smart phone N95 during making sixth senses.
Muhammad Hassan 12-Jan-13 10

11. Mirror:-
Toward the downward direction on projector a mirror is
placed. The function of mirror is reflecting image on desire
surface.
Muhammad Hassan 12-Jan-13 11

12. How it works
•The hardware that makes Sixth Sense work is a pendant
like mobile wearable interface.
•It has a camera, a mirror and a projector and is
connected wirelessly to a Bluetooth smart phone that
can slip comfortably into one’s pocket.
•The camera
recognizes individuals, images, pictures, gestures one
makes with their hands.
•Information is sent to the Smartphone for processing.
The downward-facing projector projects the output
image on to the mirror.
•Mirror reflects image on to the desired surface.
• Thus, digital information is freed from its confines and
placed in the physical world.
Muhammad Hassan 12-Jan-13 12

13. Applications of sixth sense
Muhammad Hassan 12-Jan-13 13

14. Make a call:-
The amazing fun of this technology that you make a call without
taking out cell phone, you just place your hand before the projector.
A virtual keypad will appear on your palm; you dial numbers and
can make a call.
Time detail:-
If you have not wrist watch, you did not need to worry because
through sixth sense technology you are very close to time you
can make own watch you just have to draw a circle on your wrist
to get a virtual watch that gives your correct time.
Muhammad Hassan 12-Jan-13 14

15. Drawing Application:-
The drawing application allows the user you to draw on any surface
by tracking the fingertip movements of the user’s index finger. The
pictures that are drawn by the user can be stored and replaced on
any other surface. The user can also shuffle through various pictures
and drawing by using the hand gesture movements.
Viewing Map:-
With the help of a map application the user can call upon any
map of his/her choice and navigate through them by projecting
the map on to any surface. By using the thumb and index fingers
movements the user can zoom in, zoom out or pan the selected
map.
Muhammad Hassan 12-Jan-13 15

16. Taking Pictures:-
Another application of sixth sense devices is the implementation of a
gestural camera. This camera takes the photo of the location user is
looking at by detecting the framing gesture. After taking the desired
number of photos we can project them onto any surfaces and then
use gestures to sort through those photos and organize and resize
them.
Access book information:-
The system can project Amazon ratings on that book, as well as
reviews and other relevant information.
Muhammad Hassan 12-Jan-13 16

17. Access product information:-
Sixth Sense uses image recognition or marker technology to
recognize products. We pick up, and then feed us information on
those products.
Flight updates:-
The system will recognize your boarding pass and let you know
whether your flight is on time and if the gate has changed.
Muhammad Hassan 12-Jan-13 17

18. Information about People:-
With help of face recognition techniques the sixth sense devices
are able to provide information about the people when we meet
them. The sensor detects the face and checks the data base for
the relevant information. The system will then project the relevant
information about a person like what they do, where they work.
Muhammad Hassan 12-Jan-13 18

19. Software coding
Computer vision library is written in Symbian C++ (used
in Gesture tracking). The software for the sixth sense
prototype is designed on a Microsoft Windows platform
using C#, WPF (window presentation Foundation WPF is
the combine application of UI, 2D graphic, 3D
graphic, documents and multimedia). And open CV
(computer vision). Processing is happen in mobile using
computer algorithm. The lines of code are
approximately 50,000.
Muhammad Hassan 12-Jan-13 19

20. Advantages
•Portable (small and light weight which is able to
carry).
•Supports multi-touch and multi-use interaction.
• Connectedness between world and
information.
•Cost effective.
•Data access directly from machine in real time.
•Mind map the idea anywhere.
•It is an open source.
Muhammad Hassan 12-Jan-13 20

21. Disadvantages
•Hug disadvantage is no privacy while capturing picture
(etc) of someone even a person don’t knows you are
capturing his picture.
• It is not private enough just for user.
•When a device projecting on a hard surface, people
around you can see the projection.
•Projection is not better in a morning and bright areas.
Muhammad Hassan 12-Jan-13 21

22. When it comes in market
It is the most important issue when it comes in market.
People and the manufacture, which are ready to have
this product out in market, do not know when this will
be out.
Pranav Mistry said, “This prototype needs some serious
engineering and programming”.
Everyone is waiting hoping this will be available in
market once all the modifications are done.
Muhammad Hassan 12-Jan-13 22

23. Conclusion
•Sixth Sense recognizes the objects around
us, displaying information automatically and
letting us to access it in any way we need.
•The Sixth Sense prototype implements several
applications that demonstrate the
usefulness, viability and flexibility of the system.
•Allowing us to interact with this information via
natural hand gestures.
•The potential of becoming the ultimate
"transparent" user interface for accessing
information about everything around us.
Muhammad Hassan 12-Jan-13 23

24. Price:
The whole system price is 350$.
According to Pakistani currency is 34097
rupees.
Muhammad Hassan 12-Jan-13 24

25. Muhammad Hassan 12-Jan-13 25