The document discusses the Sixth Sense technology, which aims to connect the digital world to the physical world. It describes the key components of the Sixth Sense device prototype, including a camera, projector, mirror, and colored markers on the fingers. The device processes gestures to project digital information onto physical surfaces. Some applications mentioned include using maps, taking photos, drawing, making calls, getting product/flight information, and interacting with objects. Future projects building on this technology, like mouseless computing and allowing multiple views on a single display, are also discussed.

1. 1
SIXTH SENSE TECHNOLOGY
Submitted By Renjith.R
Roll No 10BM60070
M.B.A 1st Year, VINOD GUPTA SCHOOL OF MANAGEMENT,IIT KHARAGPUR
Instructor: Dr Prithwis Mukerjee

2. 2
SIXTH SENSE TECHNOLOGY
Abstract
Every one of us are aware of the five basic senses – seeing, feeling, smelling, tasting and hearing.
These senses have evolved through millions of years. Whenever we encounter a new
object/experience our natural senses tries to analysis that experience and the information that is
obtained is used to modify our interaction with the environment. But in this new age of technologies
the most important information that helps one to make right decision is something that cannot be
perceived and analysed by our natural senses. That information is the data in the digital form, and it is
available to everyone through sources like internet. The sixth sense technology concept is an effort to
connect this data in the digital world in to the real world.
Introduction
Although miniaturized versions of computers help us to connect to the digital world even while we are
travelling there aren’t any device as of now which gives a direct link between the digital world and our
physical interaction with the real world. Usually the information’s are stored traditionally on a paper or
a digital storage device. Sixth sense technology helps to bridge this gap between tangible and non-
tangible world. Sixth Sense device is basically a wearable gestural interface that connects the
physical world around us with digital information and lets us use natural hand gestures to interact with
this information .The sixth sense technology was developed by Pranav Mistry, a PhD student in the
Fluid Interfaces Group at the MIT Media Lab .According to him the sixth sense technology has a Web
4.0 view of human and machine interactions. SixthSense integrates digital information into the
physical world and its objects, making the entire world your computer. It can turn any surface into a
touch-screen for computing, controlled by simple hand gestures. It is not a technology which is aimed
at changing human habits but causing computers and other machines to adapt to human needs .It
also supports multi user and multi touch provisions. Sixth Sense device is a mini-projector coupled
with a camera and a cell phone—which acts as the computer and your connection to the Cloud, all
the information stored on the web. The current prototype costs around $350. The Sixth Sense
prototype is used to implement several applications that have shown the usefulness, viability and
flexibility of the system.

3. 3
SIXTH SENSE DEVICES
'SixthSense' is a wearable gestural interface that augments the physical world around us with digital
information and lets us use natural hand gestures to interact with that information The hardware
components are coupled in a pendant like mobile wearable device. The SixthSense prototype is
comprised of a pocket projector, a mirror,colord marker and a camera. The camera, mirror and
projector is connected wirelessly to a blue tooth smart phone device that can easily fit into the user’s
pocket. A software then processes the data that is collected by the capturing device and produces
analysis.The software that is used in sixth sense device is open source type.
Camera
It captures the image of the object in view and track the user’s hand gesture. The camera recognizes
individuals, images, pictures, gestures that user makes with his hand. The camera then sends this
data to a smart phone for processing. Basically the camera forms a digital eye which connects to the
world of digital information.
Coloured Marker
There are colour markers placed at the tip of users finger. Marking the user’s fingers with red, yellow
green and blue coloured tape helps the webcam to recognize the hand gestures .The movements and
arrangement of these markers are interpreted into gestures that act as a interaction instruction for the
projected application interfaces.

4. 4
Mobile Component
The SixthSense device consists of a web enabled smart phone which process the data send by the
camera. The smart phone searches the web and interprets the hand gestures with help of the
coloured markers placed at the finger tips.
Projector
The information that is interpreted through the smart phone can be projected into any surface. The
projector projects the visual information enabling surfaces and physical objects to be used as
interfaces. The projector itself consists of a battery which have 3 hours of battery life .A tiny LED
projector displays the data sent from the smart phone on any surface in view- object, wall or person.
The downward facing projector projects the image on to a mirror.
Using palm for dialling a number
Mirror
The usage of a mirror is important as the projector dangles pointing downwards from the neck. The
mirror reflects the image on to a desire surface.Thus finally the digital image is freed from its confines
and placed in the physical world.

5. 5
Technologies that are related to Sixth Sense Devices
Augmented Reality
The augmented Reality is a visualization technology that allows the user to experience the virtual
experience added over real world in real time. With the help of advanced AR technology the
information about the surrounding real world of the user becomes interactive and digitally usable.
Artificial information about the environment and the objects in it can be stored and retrieved as an
information layer on top of the real world view. When we compare the spectrum between virtual
reality, which creates immersive, computer-generated environments, and the real world, augmented
reality is closer to the real world. Augmented reality adds graphics, sounds, haptic feedback and smell
to the natural world as it exists. Both video games and cell phones are driving the development of
augmented reality. The augmented systems will also superimpose graphics for every perspective
available and try adjust to every movement of the user's head and eyes. The three basic components
of an augmented reality system are the head-mounted display, tracking system and mobile computer
for the hardware. The main goal of this new technology is to merge these three components into a
highly portable unit much like a combination of a high tech Walkman and an ordinary pair or
eyeglasses. The head-mounted display used in augmented reality systems will enable the user to
view superimposed graphics and text created by the system. Another component of an augmented
reality system is its tracking and orientation system. This system pinpoints the user's location in
reference to his surroundings and additionally tracks the user's eye and head movements.
Augmented reality systems will need highly mobile computers. As of now many computers aren’t
there to satisfy to provide this option.

6. 6
Gesture Recognition
It is a technology which is aimed at interpreting human gestures with the help of mathematical
algorithms. Gesture recognition technique basically focuses on the emotion recognition from the face
and hand gesture recognition. Gender recognition technique enables humans to interact with
computers in a more direct way without using any external interfacing devices. It can provide a much
better alternative to text user interfaces and graphical user interface which requires the need of a
keyboard or mouse to interact with the computer. Interfaces which solely depends on the gestures
requires precise hand pose tracking. In the early versions of gesture recognition process special type
of hand gloves which provide information about hand position orientation and flux of the fingers. In the
SixthSense devices coloured bands are used for this purpose. Once hand pose has been captured
the gestures can be recognised using different technique’s. Neural network approaches or statistical
templates are the commonly used techniques used for the recognition purposes. This technique have
an high accuracy usually showing accuracy of more than 95%. Time dependent neural network will
also be used for real time recognition of the gestures.
Computer Vision
Computer vision is the technology in which machines are able to interpret/extract necessary
information from an image. Computer vision technology includes various fields like image processing,
image analysis and machine vision. It includes certain aspect of artificial intelligence techniques like
pattern recognition. The machines which implement computer vision techniques require image
sensors which detect electromagnetic radiation which are usually in the form of ultraviolet rays or light
rays. The computer vision find itself applicable in varies field of interest. One such field is bio medical
image processing. It’s also used in autonomous vehicle like SUV’s. The computer vision technique
basically includes four processes.
1. Recognition: One of the main task of computer vision technique is to determine whether the
particular object contain the useful data or not.
2. Motion Analysis: Motion analysis includes several tasks related to estimation of motion where
an image sequence is processed continuously to detect the velocity at each point of the
image or in the 3D scene.
3. Scene Reconstruction: Computer vision technique employs several methods to recreate a 3D
image from the available images of a scene.
4. Image Restoration: The main of aim of this step is to remove noise from an given image. The
simplest method involves using simple filters like low pass or median filters. In order to get
better quality images more complex methods like Inpainting are used.
Radio Frequency Identification
Radio frequency identification systems transmit the identity of an object wirelessly, using radio
magnetic waves. The main purpose of a radio frequency identification system is to enable the transfer
of a data via a portable device. The portable device is commonly known as tag. The data send by the
tag is received and processed by a reader according to the needs of the application. The data send
by the tag contains various information’s identification or location of the information, or specifics about

7. 7
the product that has been tagged, for example price, colour, date of purchase, etc. This technology
gained importance because of its ability to track moving object. A typical radio frequency tag consists
of a microchip attached to a radio antenna which is mounted on a substrate. To retrieve the data from
the tag a reader is needed. A typical radio frequency reader consists of two antennas that emit radio
waves and at the same are capable of accepting the signals from the tag. The reader then passes the
information that it has received to a computer device in digital form. The computer then interprets this
digital data and processes it. Radio frequency identification techniques are widely used in the fields
like asset tracking, supply chain management, manufacturing, payment systems etc. One of the major
advantages of devices using radio frequency technology over other similar devices is that RFID
devices need not be positioned precisely relative to the scanner. But till then there are certain areas of
concern for this technology. Some problem related to this technology is tag collision and reader
collision. Usually the reader collision occurs when the signals from two or more readers overlap, while
tag collision occurs when many tags are present in a small area.
ADVANTAGES OF SIXTH SENSE DEVICES
Portable
One of the main advantages of the sixth sense devices is its small size and portability. It can be easily
carried around without any difficulty. The prototype of the sixth sense is deigned in a such a way that
it gives more importance to the portability factor. All the devices are light in weight and the smart
phone can easily fit into the users pocket.
Support multi touch and multi user interaction
Multi touch and multi user interaction is another added feature of the sixth sense devices. Multi
sensing technique allows the user to interact with system with more than one than one finger at a
time.. Sixth sense devices also in-cooperates multi user functionality. This is typically useful for large
interaction scenarios such as interactive table tops and walls.
Cost effective
The cost incurred for the construction of the sixth sense proto type is quiet low. It was made from
parts collected together from common devices. And a typical sixth sense device cost up to $300. The
sixth sense devices have not been made in large scale for commercial purpose. Once that happens
it’s almost certain that the device will cost much lower than the current price.
Connectedness between real world and digital world
Forming a connection between the real world and the digital world was the main aim of the sixth
sense technology.
Data access directly from the machines in real time

8. 8
With help of a sixth sense device the user can easily access data from any machine at real time
speed. The user doesn’t require any machine-human interface to access the data. The data access
through recognition of hand gestures is much easier and user friendlier compared to the text user
interface or graphical user interface which requires keyboard or mouse.
Mind map the idea anywhere
With the advent of the sixth sense device, requirement of a platform or a screen to analyse and
interpret the data has become obsolete. We can project the information into any any surface and can
work and manage the data as per our convenience.
.
Open source software
The software that is used to interpret and analysis the data collected by the device is made open
source. This enables other developers to contribute to the development of the system
APPLICATION
The sixth sense technology finds a lot of application in the modern world. The sixth sense devices
bridge the gap by bringing the digital world into the real world and in that process allowing the users to
interact with the information without the help of any machine interfaces. Prototypes of the sixth sense
device have demonstrated viability, usefulness and flexibility of this new technology. According to the
words of its developers the extend of use of this new device is only limited by the imagination of
human beings. Some practical applications of the sixth sense technology is given below
Map Application
With the help of a map application the user can call upon any map of his/her choice and navigate
through them by projecting the map on to any surface .By using the thumb and index fingers
movements the user can zoom in, zoom out or pan the selected map.
Taking Pictures
Another application of sixth sense devices is the implementation of a gestural camera. This camera
takes the photo of the location user is looking at by detecting the framing gesture. After taking the
desired number of photos we can project them onto any surfaces and then use gestures to sort
through those photos and organize and resize them.

9. 9
The user taking making a pose gesture The camera takes the photo of the location
Drawing Application
The drawing application allows the user you to draw on any surface by tracking the fingertip
movements of the user’s index finger. The pictures that are drawn by the user can be stored and
replaced on any other surface. The user can also shuffle through various pictures and drawing by
using the hand gesture movements
Making Calls
We can make calls with help of sixth sense device .The sixth sense device is used to project the
keypad into your palm and the using that virtual keypad we can make calls to anyone.
Interacting with physical objects
The SixthSense system also helps to interact with physical objects we use in a better way. It
augments physical objects by projecting more information about these objects projected on them. For
example, a gesture of drawing a circle on the user’s wrist projects an analog watch on the users hand.
Similarly a newspaper can show live video news or dynamic information can be provided on a regular
piece of paper
Newspaper showing live feeds an analog watch projected into the wrist
Getting Information
Sixth sense devices can be used for getting various information relating to our everyday life by getting
in contact with objects
Product Information:
Sixth sense technology uses marker technology or image recognition techniques to recognize the
objects we pick in our hand and then provide information relating to that product.

10. 10
Book Information:
By holding and shuffling through the book pages, the sixth sense provides Amazon ratings on that
book, other reviews and other relevant things related to the book.
Flight Updates:
With the help of the sixth sense technology it is no longer required to log into any sites for checking
the status of the flights. The system will recognize your boarding pass and let you know whether the
flight is on time or not.
Information About People:
With help of face recognition techniques the sixth sense devices are able to provide information about
the people when we meet them. The sensor detects the face and checks the data base for the
relevant information. The system will then project the relevant information about a person like what
they do, where they work,
Boarding pass showing current status The information about a person being projected
WAY FORWARD
Upcoming Projects Making Use Of the Sixth Sense Technology
Mouse less:
Mouse less is a concept which is developed in order to avoid the usage of mouse in computers. It
helps to provide a familiarity of the physical mouse without actually requiring a physical mouse.
Even though the computer hardware technology has been developed to a greater extent, still there
has been no proper substitute for a physical mouse. This new invention aims to remove the
requirement of having a physical mouse altogether but still it provides the feeling of a physical

11. 11
mouse that users are familiar with. It basically consists of an Infrared laser beam and an Infrared
camera. Both laser and camera are embedded together with the computer. The laser beam module is
modified with a line cap and placed such that it creates a plane of IR laser just above the surface the
computer sits on. The user cups their hand, as if a physical mouse was present underneath, and the
laser beam lights up the hand which is in contact with the surface. The IR camera detects those bright
IR blobs using computer vision. The change in the position and arrangements of these blobs are
interpreted as mouse cursor movement and mouse clicks.
Third Eye:
Third Eye is a new technique that enables multiple viewers to see different things on a same display
screen at the same time.
With thirdEye, - We can have a public sign board where a Japanese tourist sees all the instructions in
Japanese and an American in English. - We don't need to have the split screen in games now. Each
player can see his/her personal view of the game on the TV screen. - Two people watching TV can
watch their favuorite channel on a single TV screen. - A public display can show secret messages or
patterns. - In the same movie theater, people can see different end of a suspense movie.
Inktuitive:
Even though there has been a tremendous development of computer aided designing tools, still paper
and pencil is the preferred by most of the designers ,especially in the earlier stages of designing
The project ‘inktuitive’ is aimed to combine the process of creation that is inherent in paper and pencil
with the power of computing that the digital design tools provide. It also extends the natural work-
practice of using physical paper by giving the pen the ability to control the design in physic al three-
dimensional space, freeing it from its tie to the paper. There always remains an intuition of pen and
paper ,but the lines that have been drawn are be captured and translated into shapes in the digital
world. The physical paper is augmented with overlaid digital strokes. The platform provides a novel

12. 12
interaction mechanism for drawing and designing using above the surface pen movements. In other
words ‘inktuitive’ can be considered as an intuitive physical design workspace that aims to bridge the
gap and bring together the conventional design tools such as paper and pencil with the power and
convenience of the digital tools for design.
TaPuma:
TaPuma is the short form of Tangible Public Map. It is another attempt to bring the digital world
nearer to the real world by exploring the techniques of sixth sense technology. Key word based
search mechanism has vastly influenced and modified the ways in which we search information’s.
The keyword based search allows the user to filter out the required data from the vast data that are
available on search engines such as yahoo and google. TaPuMa allows people to use their own
belongings, the objects they usually carry with them to access relevant and just-in-time information
and locations of places from a public map. The project also explores and analyzes the advantages
and challenges of this novel interaction mechanism, where real life objects serve as the interface for
information acquirement. The broad concept behind the project TaPuMa is Object Amelioration,
where the functions of everyday objects can be expanded by using their affordances or functionalities
in a variety of different contexts.
CONCLUSION
The sixth sense technology and devices are definitely the way forward. The sixth sense recognizes
the objects around us and displays the information relating to those objects in a real time
environment. The sixth sense technology allows the user to interact the information through hand
gestures. This is a quiet efficient way compared to the text and graphic based user interface. It has
the potential to form the transparent user interface for accessing the information around us. . Even
though the sixth sense devices are currently in development stages and have not been used in large
now, it’s quiet predictable that this technology have the potential to revolutionise the way in which we
interacts with the digital world. As Mr Pranav Mistry says “imagination of the human brain is the only
thing that could limit the application of the sixth sense technology”

13. 13
REFERNCES
 http://www.ted.com/talks/pranav_mistry_the_thrilling_potential_of_sixthsense_technology.htm
 http://www.commoncraft.com/augmented-reality-video
 http://www.arcane-technologies.com/en/
 http://en.wikipedia.org/wiki/Augmented_reality
 http://www.rfidjournal.com/
 http://www.ted.com/talks/pattie_maes_demos_the_sixth_sense.html
 http://timesofindia.indiatimes.com/Health__Science/Sixth_sense_gizmo_a_sci-
fi_reality/articleshow/4083978.cms
 http://www.gesturetek.com/