The document describes the Sixth Sense technology, which allows users to access and interact with digital information by using hand gestures that are detected by a camera. The Sixth Sense consists of a camera, projector, mirror, and mobile device. It captures gestures and the visual scene, projects interfaces onto surfaces, and connects to the cloud for additional data. Examples are given of how it could be used to get book information, check travel statuses, take photos, and more. Requirements for building a Sixth Sense device and some advantages and disadvantages are also outlined.

2.  Sometimes when you get up in the morning you have a feeling that you are
going to have a really bad day.
 There may not be any logical explanation why you feel this way. You just
have this strange feeling and funnily enough you end up having a perfectly
lousy day.
 This intuitive feeling that you sometimes have that tells you that something
good or bad is going to happen, is your "sixth sense".
 We all have five senses — taste, hearing, sight, smell, and touch. This sixth
sense cannot be explained by the other five senses; it is something that cannot
be explained by logic. Here are a few examples………

4.  Sixth Sense Technology integrates digital information into the physical
world and its objects, making the entire world your computer.
 It can turn any surface into a touch-screen for computing, controlled by
simple hand gestures.
 Sixth Sense device is a mini-projector coupled with a camera and a cell
phone—which acts as the computer and your connection to the Cloud, all
the information stored on the web.

5.  So you want to make your own Sixth Sense Device. Now what?
Components
 Camera
 Projector
 Mirror
 Microphone
 Mobile computing device

7.  The camera is the key input device of the Sixth Sense system.
 The camera acts as a digital eye of the system. It basically
captures the scene the user is looking at.
 The video stream captured by the camera is passed to mobile
computing device which does the appropriate computer
vision computation. The major functions of the camera can be
listed as:

8.  Captures user’s hand movements and gestures .
 Captures the scene in front and objects the user is
interacting with .
 Takes a photo of the scene in front when the user
performs a ‘framing’ gesture .

9.  The projector is the key output device of the Sixth Sense system.
 The projector visually augments surfaces, walls and physical
objects the user is interacting with by projecting digital information
and graphical user interfaces. The mobile computing device
provides the projector with the content to be projected. The
projector unit used in prototype runs on a rechargeable battery. The
major functions of the projector can be listed as:
 Projects graphical user interface of the selected application onto
surfaces or walls in front
 Augments the physical objects the user interacting with by
projecting just-in-time and related information from the Internet

10.  The mirror reflects the projection coming out from the projector and
thus helps in projecting onto the desired locations on walls or surfaces.
 The user manually can change the tilt of the mirror to change the
location of the projection.
 For example in application where the user wants the projection to go
on the ground instead of the surface in front, he can change the tilt of
the mirror to change the projection.
 Thus, the mirror in the Sixth Sense helps in overcoming the limitation
of the limited projection space of the projector.

11.  The microphone is an optional component of the Sixth Sense.
 It is required when using a paper as a computing interface. When
the user wants to use a sheet of paper as an interactive surface, he
or she clips the microphone to the paper.
 The microphone attached this way captures the sound signals of
user’s touching the paper.
 This data is passed to computing device for processing. Later,
combined with the tracking information about user’s finger, the
system is able to identify precise touch events on the paper.
 Here, the sound signal captured by the microphone provides time
information whereas the camera performs tracking.

12.  The Sixth Sense system uses a mobile computing device in user’s
pocket as the processing device.
 The software program enabling all the features of the system runs
on this computing device.
 This device can be a mobile phone or a small laptop computer.
 The camera, the projector and the microphone are connected to this
device using wired or wireless connection.
 The mobile computing device is also connected to the Internet via
3G network or wireless connection.

13.  The hardware that makes Sixth Sense work is a pendant like mobile
wearable interface.
 It has a camera, a mirror and a projector and is connected wirelessly to a
Bluetooth smart phone that can slip comfortably into one’s pocket
 The camera recognizes individuals, images, pictures, gestures one makes
with their hands .
 Information is sent to the Smartphone for processing.
 The downward-facing projector projects the output image on to the
mirror.
 Mirror reflects image on to the desired surface.
 Thus, digital information is freed from its confines and placed in the
physical world.

15. Hardware Required
Mobile
 Smart phone is used.
 It has multitasking capability.
 Built-in camera provides execution of both:
Gesture tracking engine.
Gesture enabled application.

16.  Pocket projector is used.
 It augments near by surfaces.
 Suitable for mobile usage.

17.  Events & Gestures are detected using colored markers attached to fingers.
 The user can zoom in or out by moving his hands/fingers farther or nearer to each
other, respectively
 The user can draw on any surfaces using the movement of the finger used as a
pen.

18.  Applications are implemented using JAVA
2MICROedition.
 Computer vision library is written in Symbian c++
(used in Gesture tracking).
 The software for the sixth sense prototype is
developed on a Microsoft Windows platform using
C#, WPFand open CV.

19.  Make a call - If you want to make a call to anyone
and you find yourself with no such calling device, it
is possible to make a call with sixth sense
technology.

20.  A gesture of drawing a circle on the user’s wrist
projects an watch on the user’s hand.

23.  When you go to a book store to buy a new book, as
soon as u hold a book in your hand the sixth sense
device immediately projects a rating of the book
enabling you to choose the best one and get other
additional information..

24.  Not only a book any purchase you make this device can help
you. You hold up your boarding pass; the words DELAYED
20 MINUTES or ON TIME are beamed onto it.

26.  If you find beautiful scenery and fingers out at arm’s length
forming square. The system snaps a photo of what’s enclosed
by your fingers. Later, you can sort, resize and fiddle with
these photos by projecting them onto any wall and dragging
their images with your fingertips.

27.  Using the global positioning system, you can project maps
any where which provide information on your position
depending on your moves and find a way by using zoom out
and zoom in options.
 When this topic is explained to our former president DR.A.P.J
.ABDUL KALAM, he suggested a new use saying that a
person who is dumb can use speaker system to express his
thoughts using this technology.

29.  Attach the microphone available with the webcam to a paper.
Now the sound of touch is exactly getting a where a person is
touching the paper.

30. Advantages
 The gestures are very simple used for
pictures,knowing time, or making a phone call.
 We can use any surface for projection like wall
or paper or wrist.
 We can play games by using a simple piece of
paper.
 It is a great substitute for a laptop or computer.
 We can check our mails anywhere any time.

31. Disadvantages
 For example in taking picture you can use your
only fingers those have marker caps.
 Projection can be better use in night time and
dark areas rather than in morning and bright
areas.
 This is the issue because the vision of the user
can be damaged.
 The information about this device is last
updated in 2009 but since ever there are no
news about this device.

32. Thank
you…………