Sixth sense technology 04

SIXTH SENSE TECHNOLOGY
1. INTRODUCTION
Sixth Sense Technology is a mini-projector coupled with a camera and a cell phone
which acts as the computer and connected to the Cloud, all the information stored on
the web. Sixth Sense can also obey hand gestures. The camera recognizes objects
around a person instantly, with the micro-projector overlaying the information on any
surface, including the object itself or hand. Also can access or manipulate the
information using fingers.
Sixth Sense is a wearable gestural interface that augments the physical world around
us with digital information and lets us use natural hand gestures to interact with that
information. By using a camera and a tiny projector mounted in a pendant like
wearable device. Sixth Sense sees what you see and visually augments any surfaces or
objects we are interacting with. It projects information onto surfaces, walls, and
physical objects around us, and lets us interact with the projected information through
natural hand gestures, arm movements, or our interaction with the object itself. Sixth
Sense attempts to free information from its confines by seamlessly integrating it with
reality , and thus making the entire world your computer.[1]

Fig 1. Parts used in Sixth Sense Technology [1]

Akshay .N .Mahajan(120573116004)

Page 1

2. HOW SIXTH SENSE TECHNOLOGY WORKS
The technology in itself is nothing more than the combination of some
stunning technologies, but the idea of combining those technologies is really
great.
The technology is mainly based on hand gesture recognition, image capturing,
processing, and manipulation, etc.
The camera is used to recognize and track user‘s hand gestures and physical
objects using computer-vision based techniques, while the projector is used to
project visual information on walls or on any physical thing around us. Other
hardware includes mirror and colored caps to be used for fingers
The projector: projects visual information, enabling surfaces, walls and
physical objects around the wearer to be used as interfaces;
The camera and hands: recognizes and tracks the user's hand gestures and
physical objects using computer-vision based techniques.
The software program: processes the video stream data captured by the camera
and tracks the locations of the colored markers (visual tracking of colored
fingers) at the tip of the user‘s fingers using simple computer-vision
techniques. The movements and arrangements of these fiducials are
interpreted into gestures that act as interaction instructions for the projected
application interfaces.
The maximum number of tracked fingers is only constrained by the number of
unique fiducials, thus Sixth Sense also supports multi-touch and multi-user
interaction.


Page 2

3. ARCHITECTURE OF SIXTH SENSE TECHNOLOGY [3]

Fig 2: Architecture of Sixth Sense Technology[3]
Sixth Sense, a tool that connects the physical world with the world of data.
Digital prototype called the ―Sixth Sense‖, which wearable hi-tech devices
that can turn anything into a touch screen. Your wrist can become a watch and
your palm a phone. The device is combination of a wearable camera, a
projector and a mirror where the camera recognizes movements of the fingers
of your hand and translates those into commands.
The hardware components are coupled in a pendant like

wearable

device.

Both the projector and the camera are connected to the mobile computing
device in the user‘s pocket. The projector projects visual information enabling
surfaces, walls and physical objects around us to be used as interfaces; while
the camera recognizes and tracks user‘s hand gestures and physical objects
using computer vision based techniques. The software program processes the
video stream data captured by the camera and tracks the locations of the

Page 3

colored markers (visual tracking fiducials) at the tip of the user‘s fingers using
simple computer vision techniques.
The movements and arrangements of these fiducials are interpreted into
gestures that act as interaction instructions for the projected application
interfaces. The maximum number of tracked fingers is only constrained by the
number of unique fiducials, thus Sixth Sense also supports multi touch and
multi user interaction.

3.1 POCKET PROJECTOR [3]
Pocket projector is an emerging technology that applies the use of a projector in a
handheld device. It is also know as Handheld projector, pocket projector or mobile
projector or Pico projector. It is a response to the emergence of compact portable
devices such as mobile phones, personal digital assistants and digital cameras. Which
have sufficient storage capacity to handle presentation materials but little space to
accommodate an attached display screen. Handheld projectors involve miniaturized
hardware and software that can project digital images onto any nearby viewing
surface, such as a wall. This entire system is compacted into one very tiny chip. An
important design characteristic of a handheld projector is the ability to project a clear
image, regardless of the physical characteristics of the viewing surface.

3.2 PARTS OF POCKET PROJECTOR [3]
The system comprises four main parts:
The electronics
The laser light sources
The combiner optic, and the
Scanning mirrors.


Page 4

3.3 THE WORKING OF SIXTH SENSE DEVICE [4]

Fig.3 Working Of Sixth Sense [4]

First, the electronics system turns the image into an electronic signal.
Next the electronic signals drive laser light sources with different colors and
intensities down different paths.
In the combiner optic the different light paths are combined into one path
demonstrating a pallet of colors.
Finally, the mirrors copy the image pixel by pixel and can then project the
image.

3.4 APPLICATIONS [8]
Handheld projector can be used for similar applications than small conventional
projectors. Study applications that are specifically designed for handheld projectors
often using prototypes of mobile phones with an integrated projector.

Gaming: Handheld projectors, in particular projector phones, could offer new
possibilities for mobile gaming as demonstrated by the adaptation of the
Playstation3 game Little Big Planet.

Page 5

Hand gesture recognition: Size reduction of mobile devices is often limited
by the size of the used display. Apart from the display a complete phone can
be, for example, integrated in a headset. It has been demonstrated that Pico
projectors integrated in headsets could be used as interaction devices e.g.
using additional hand and finger tracking.
Picture browsing: Projector phones allow photographs to be shared with a
larger audience. The projected display allowed viewing of pictures by all
present which is currently not possible using a single mobile device.
[4]

3.5 MOBILE COMPUTING DEVICES
A mobile device (also known as cell phone device, handheld device, handheld
computer, "Palmtop" or simply handheld) is a pocket-sized computing device,
typically having a display screen with touch input or a miniature keyboard. In the case
of the personal digital assistant (PDA) the input and output are combined into a touchscreen interface. Smart phones and PDAs are popular amongst those who require the
assistance and convenience of a conventional computer, in environments where
carrying one would not be practical.[4]

3.5.1 TYPES OF MOBILE DEVICES [4]
Mobile devices have been designed for many applications and include:
Mobile computers.
Graphing calculator.
Handheld game consoles.
Digital still camera (DSC).
Digital video camera (DVC or digital camcorder).
Portable media player.
E-book reader.
Mobile phone.
Personal navigation devices(PNDs)


Page 6

3.5.2 USAGE
Handheld devices have become use in mobile field management situations to record
information in the field. They are used to achieve a variety of tasks for increasing
efficiency that include digitizing notes , sending and receiving invoices, assets
management, recording signatures, managing parts and scanning barcodes. Handheld
computers used at work have molded over time into a variety of form including smart
phones on the low end, handheld PDAs, Ultra Mobile PCs, tablet PCs .[4]

4. WEARABLE GESTURAL INTERFACE [5]
Information is traditionally confined to paper or digitally to a screen. In this paper,
we introduce WUW, a wearable gestural interface, which attempts to bring
information out into the tangible world. By using a tiny projector and a camera
mounted on a hat or coupled in a pendant like wearable device, WUW sees what the
user sees and visually augments surfaces or physical objects the user is interacting
with.WUW projects information onto surfaces, walls, and physical objects around us,
and lets the user interact with the projected information through natural hand gestures,
arm movements, or interaction with the object itself.

A wearable device with a projector that allows to augment reality with a pendant
projector. The device (via the web cam and coded finger-gloves) recognizes users
gestures while searching for information, taking photos, making a call and presents it
back to the user projected at any display.

The recent advent of novel sensing and display technologies has encouraged the
development of a variety of multi-touch and gesture based interactive systems. Such
systems have moved computing onto surfaces such as tables and walls and brought
the input and projection of digital interfaces into one-to-one correspondence such that
the user may interact directly with information using touch and natural hand gestures.
A parallel trend, the miniaturization of mobile computing devices permit ―anywhere‖
access to information, so we are always connected to the digital world. Unfortunately,
most gestural and multi-touch based interactive systems are not mobile and small

Page 7

mobile devices fail to provide the intuitive experience of full-sized gestural systems.
Moreover, information still resides on screens or dedicated projection surfaces. There
is no link between our interaction with these digital devices and interaction with the
physical world around us.

While it is impractical to modify all physical objects and surfaces into interactive
touch-screens, and to turn everything into network enabled devices, it is possible to
augment environment around us with visual information. When digital information is
projected onto physical objects, instinctively it makes sense to interact with it in
similar patterns as with physical objects: through hand gestures.

In this paper, we present WUW, a computer-vision based wearable and gestural
information interface that augments the physical world around us with digital
information and proposes natural hand gestures as the mechanism to interact with that
information. It is not the aim of this research to present an alternative to multi-touch
or gesture recognition technology, but rather to explore the novel free-hand gestural
interaction that WUW proposes.

4.1 What is WUW? [5]

WUW is a wearable gestural interface. It consists of a camera and a small projector
mounted on a hat or coupled in a pendant like mobile wearable device. The camera
sees what the user sees and the projector visually augments surfaces or physical
objects that the user is interacting with. WUW projects information onto the surfaces,
walls, and physical objects around the user, and lets the user interact with the
projected information through natural hand gestures, arm movements, or direct
manipulation of the object itself.

The tiny projector is connected to a laptop or mobile device and projects visual
information enabling surfaces, walls and physical objects around us to be used as
interfaces; while the camera tracks user hand gestures using simple computer vision
based techniques. The current WUW prototype implements several applications that
demonstrate the usefulness, viability and flexibility of the system. The map
application(see Fig.3) lets the user navigate a map displayed on a nearby surface

Page 8

using familiar hand gestures, letting the user zoom in, zoom out or pan using intuitive
hand movements. The drawing application (see Fig.4) lets the user draw on any
surface by tracking the fingertip movements of the user‘s index finger.

The WUW prototype also implements a gestural camera that takes photos of the scene
the user is looking at by detecting the ‗framing‘ gesture (see Fig.5). The user can stop
by any surface or wall and flick through the photos he/she has taken. The WUW
system also augments physical objects the user is interacting with by projecting more
information about these objects projected on them. For example, a newspaper can
show live video news or dynamic information can be provided on a regular piece of
paper (see Fig.6).The gesture of drawing a circle on the user‘s wrist projects an analog
watch (see Fig.4). The WUW system also supports multiple users collaborating and
interacting with the projected digital information using their hand movements and
gestures. [5]

Fig.4: Map Application [5]


Page 9


Fig.5: Drawing Application [5]

Fig.6: Framing Gesture [19]

Fig.7: Newspaper [5]

Page 10


Fig.8:Analog watch [5]

4.2 WUW prototype [5]
The WUW prototype is comprised of three main hardware components: a pocket
projector (3MMPro110), a camera (Logitech Quick Cam) and a laptop computer. The
software for the WUW prototype is developed on a Microsoft Windows platform
using C#. Both the projector and the camera are mounted on a hat and connected to
the laptop computer in the user‘s back pack. The prototype system uses plastic
colored-markers (e.g. caps of whiteboard markers) as the visual tracking fiducials.

Fig.9: WUW prototype system[5]


Page 11

The movements and arrangements of these color markers are interpreted into gestures
that act as interaction mechanism for the application interfaces projected by the
projector. The current prototype only tracks 4 fingers – index fingers and thumbs.
This is largely due to the relative importance of the index finger and thumb in natural
hand gestures, tracking these WUW is able to recognize a wide variety of gestures
and postures. The maximum number of tracked fingers is only constrained by the
number of unique fiducials, thus the WUW also supports multi-user interaction.

The prototype system also implements a few object augmentation applications where
the camera recognizes and tracks physical objects such as a newspaper, a coffee cup,
or a book and the projector projects relevant visual information on them. WUW uses
computer-vision techniques in order to track objects and align projected visual
information by matching pre-printed static color markers (or patterns) on these objects
with the markers projected by the projector.

4.3 CHALLANGES
There are many challenges associated with the accuracy and usefulness of gesture
recognition software. For image-based gesture recognition there are limitations on the
equipment used and image noise. Images or video may not be under consistent
lighting, or in the same location. Items in the background or distinct features of the
users may make recognition more difficult. The variety of implementations for imagebased gesture recognition may also cause issue for viability of the technology to
general usage. For example, an algorithm calibrated for one camera may not work for
a different camera. The amount of background noise also causes tracking and
recognition difficulties, especially when occlusions (partial and full) occur.
Furthermore, the distance from the camera, and the camera's resolution and quality,
also cause variations in recognition accuracy. [5]

5. HAND GESTURE RECOGNITION [6]
Hand gesture recognition is a topic in computer science and language technology with
the goal of interpreting human gestures via mathematical algorithms. Gestures can

Page 12

originate from any bodily motion or state but commonly originate from the face or
hand. Current focuses in the field include emotion recognition from the face and hand
gesture recognition. Many approaches have been made using cameras and computer
vision algorithms to interpret sign language. However, the identification and
recognition of posture, gait, proxemics, and human behaviors is also the subject of
gesture recognition techniques.
Hand gesture recognition can be seen as a way for computers to begin to understand
human body language, thus building a richer bridge between machines and humans
than primitive text user interfaces or even GUIs (graphical user interfaces), which still
limit the majority of input to keyboard and mouse.
Hand gesture recognition enables humans to interface with the machine (HMI) and
interact naturally without any mechanical devices. Using the concept of gesture
recognition, it is possible to point a finger at the computer screen so that the cursor
will move accordingly. This could potentially make conventional input devices such
as mouse, keyboards and even touch-screen redundant. It can be conducted with
techniques from computer vision and image processing. The literature includes
ongoing work in the computer vision field on capturing gestures or more general
human pose and movements by cameras connected to computer .Gesture recognition
can be conducted with techniques from computer vision and image processing. The
literature includes ongoing work in the computer vision field on capturing gestures or
more general human pose and movements by cameras connected to a computer.


Page 13


Fig.10: A child being sensed by a simple hand gesture recognition
detecting hand location and movement.[6]

5.1 USES [6]
Gesture recognition is useful for processing information from humans which is not
conveyed through speech or type. As well, there are various types of gestures which
can be identified by computers.
Sign language recognition: - Just as speech recognition can transcribe speech
to text, certain types of gesture recognition software can transcribe the
symbols represented through sign language into text.
For socially assistive robotics: - By using proper sensors (accelerometers and
gyros) worn on the body of a patient and by reading the values from those
sensors, robots can assist in patient rehabilitation. The best example can be
stroke rehabilitation.
Directional indication through pointing: - Pointing has a very specific
purpose in our society, to reference an object or location based on its position
relative to ourselves. The use of gesture recognition to determine where a
person is pointing is useful for identifying the context of statements or
instructions. This application is of particular interest in the field of robotics.
Control through facial gestures: - Controlling a computer through facial
gestures is a useful application of gesture recognition for users who may not


Page 14

physically be able to use a mouse or keyboard. Eye tracking in particular may
be of use for controlling cursor motion or focusing on elements of a display.
Alternative computer interfaces: - Foregoing the traditional keyboard and
mouse setup to interact with a computer, strong gesture recognition could
allow users to accomplish frequent or common tasks using hand or face
gestures to a camera.
Immersive game technology: - Gestures can be used to control interactions
within video games to try and make the game player's experience more
interactive or immersive.
Virtual controllers: - For systems where the act of finding or acquiring a
physical controller could require too much time, gestures can be used as an
alternative control mechanism. Controlling secondary devices in a car or
controlling a television set are examples of such usage.
Affective computing: - In affective computing, gesture recognition is used in
the process of identifying emotional expression through computer systems.
Remote control: - Through the use of gesture recognition, remote control with
the wave of a hand of various devices is possible. The signal must not only
indicate the desired response, but also which device to be controlled.

5.2 INPUT DEVICES
The ability to track a person's movements and determine what gestures they may be
performing can be achieved through various tools. Although there is a large amount of
research done in image/video based gesture recognition, there is some variation within
the tools and environments used between implementations.
Depth-aware cameras: - Using specialized cameras such as time-of-flight
cameras, one can generate a depth map of what is being seen through the
camera at a short range, and use this data to approximate a 3d representation
of what is being seen. These can be effective for detection of hand gestures
due to their short range capabilities. [6]
Stereo cameras: - Using two cameras whose relations to one another are
known, a 3d representation can be approximated by the output of the cameras.
To get the cameras' relations, one can use a positioning reference such as a
lexian-stripe or in framed emitters. [6]

Page 15

Controller-based gestures: - These controllers act as an extension of the
body so that when gestures are performed, some of their motion can be
conveniently captured by software. Mouse gestures are one such example,
where the motion of the mouse is correlated to a symbol being drawn by a
person's hand, as is the Wii Remote, which can study changes in acceleration
over time to represent gestures.[6]
Single camera: - A normal camera can be used for gesture recognition where
the resources/environment would not be convenient for other forms of imagebased recognition. Although not necessarily as effective as stereo or depth
aware cameras, using a single camera allows a greater possibility of
accessibility to a wider audience.[6]

5.3 MULTI TOUCH [10]
Multi-touch is an enhancement to touch screen technology, which provides the user
with the ability to apply multiple finger gestures simultaneously onto the electronic
visual display to send complex commands to the device. The term multi-touch is a
trade mark of Apple Inc.
Multi-touch has been implemented in several different ways, depending on the size
and type of interface. Both touch tables and touch walls project an image through
acrylic or glass, and then backlight the image with LED's. When a finger or an object
touches the surface, causing the light to scatter, the reflection is caught with sensors or
cameras that send the data to software which dictates response to the touch, depending
on the type of reflection measured. Touch surfaces can also be made pressuresensitive by the addition of a pressure-sensitive coating that flexes differently
depending on how firmly it is pressed, altering the reflection. Handheld technologies
use a panel that carries an electrical charge. When a finger touches the screen, the
touch disrupts the panel's electrical field. The disruption is registered and sent to the
software, which then initiates a response to the gesture.


Page 16

6. IMAGE CAPTURING [10]
Image Capture is an application program that enables users to upload pictures from
digital cameras or scanners which are either connected directly to the computer or the
network. It provides no organizational tools like Photo but is useful for collecting
pictures from a variety of sources with no need for drivers. It achieves this by
receiving the picture as it is and through a conversion process, downloads it onto your
computer. It uses QuickTime Image Capture. This way it doesn't need to understand
anything about the actual camera to be able to do that. It was first introduced in Mac
OS X version 10.0(cheetah).

Fig.11 The user taking making a pose gesture [19]

Fig.12 The camera takes the photo of the location [19]


Page 17

6.1 IMAGE CAPTURE API
Image Capture is also the name of the Application Programming Interface associated
with the Image Capture Application. Drivers or software built to use this interface can
also automatically interface many imaging systems and software that run on Mac OS
X. [19]

6.2 FIDUCIAL
A fiduciary marker or fiducial is an object used in the field of view of an imaging
system which appears in the image produced, for use as a point of reference or a
measure. It may be either something placed into or on the imaging subject, or a mark
or set of marks in the reticle of an optical instrument. In physics, 3D computer
graphics and photography fiducials are reference points: fixed points or lines within a
scene to which other objects can be related or against which objects can be measured.
Fiduciary markers are used in a wide range of medical imaging applications. Images
of the same subject produced with two different imaging systems may be correlated
by placing a fiduciary marker in the area imaged by both systems. In this case, a
marker which is visible in the images produced by both imaging modalities must be
used. [ 7]

7.0TECHNOLOGIE’S THAT ARE RELATED TO SIXTH SENSE
DEVICES
7.1 AUGMENTED REALITY [13]
The augmented Reality is a visualization technology that allows the user to experience
the virtual experience added over real world in real time.With the help of advanced
AR technology the information about the surrounding real world of the user becomes
interactive and digitally usable. Artificial information about the environment and the
objects in it can be stored and retrieved as an information layer on top of the real
world view. When we compare the spectrum between virtual reality, which creates
immersive, computer-generated environments, and the real world, augmented reality
is closer to the real world. Augmented reality adds graphics, sounds, hap tic feedback

Page 18

and smell to the natural world as it exists. Both video games and cell phones are
driving the development of augmented reality. The augmented systems will also
superimpose graphics for every perspective available and try adjust to every
movement of the user's head and eyes. The three basic components of an augmented
reality system are the head-mounted display, tracking system and mobile computer for
the hardware. The main goal of this new technology is to merge these three
components into a highly portable unit much like a combination of a high tech
Walkman and an ordinary pair or eyeglasses. The head-mounted display used in
augmented reality systems will enable the user to view superimposed graphics and
text created by the system. Another component of an augmented reality system is its
tracking and orientation system. This system pinpoints the user's location in reference
to his surroundings and additionally tracks the user's eye and head movements.
Augmented reality systems will need highly mobile computers. As of now many
computers aren‘t there to satisfy to provide this option.

7.2GESTURE RECOGNIZATION
It is a technology which is aimed at interpreting human gestures with the help of
mathematical algorithms. Gesture recognition technique basically focuses on the
emotion recognition from the face and hand gesture recognition. Gender recognition
technique enables humans to interact with computers in a more direct way without
using any external interfacing devices. It can provide a much better alternative to text
user interfaces and graphical user interface which requires the need of a keyboard or
mouse to interact with the computer. Interfaces which solely depends on the gestures
requires precise hand pose tracking. In the early versions of gesture recognition
process special type of hand gloves which provide information about hand position
orientation and flux of the fingers. In the Sixth Sense devices coloured bands are used
for this purpose. Once hand pose has been captured the gestures can be recognised
using different technique‘s. Neural network approaches or statistical templates are the
commonly used techniques used for the recognition purposes. This technique have an
high accuracy usually showing accuracy of more than 95%. Time dependent neural
network will also be used for real time recognition of the gestures.


Page 19

7.3 COMPUTER VISION [4]
Computer vision is the technology in which machines are able to interpret/extract
necessary information from an image. Computer vision technology includes various
fields like image processing, image analysis and machine vision. It includes certain
aspect of artificial intelligence techniques like pattern recognition. The machines
which implement computer vision techniques require image sensors which detect
electromagnetic radiation which are usually in the form of ultraviolet rays or light
rays. The computer vision find itself applicable in varies field of interest. One such
field is bio medical image processing. It‘s also used in autonomous vehicle like
SUV‘s. The computer vision technique basically includes four processes.
1. Recognition: One of the main task of computer vision technique is to determine
whether the particular object contain the useful data or not.
2. Motion Analysis: Motion analysis includes several tasks related to estimation of
motion where an image sequence is processed continuously to detect the velocity at
each point of the image or in the 3D scene.
3. Scene Reconstruction: Computer vision technique employs several methods to
recreate a 3D image from the available images of a scene.
4. Image Restoration: The main of aim of this step is to remove noise from an given
image. The simplest method involves using simple filters like low pass or median
filters. In order to get better quality images more complex methods like In painting
are used.

7.3.1 APPLICATION FOR COMPUTER VISION
One of the most prominent application fields is medical computer vision or
medical image processing. This area is characterized by the extraction of
information from image data for the purpose of making a medical diagnosis of
a patient. Generally, image data is in the form of microscopy images, X-ray
images, angiography images, ultrasonic images, and tomography images. An


Page 20

example of information which can be extracted from such image data is
detection of tumors, arteriosclerosis or other malign changes.
A second application area in computer vision is in industry, sometimes called
machine vision, where information is extracted for the purpose of supporting a
manufacturing process. One example is quality control where details or final
products are being automatically inspected in order to find defects. Another
example is measurement of position and orientation of details to be picked up
by a robot arm.
Military applications are probably one of the largest areas for computer vision.
The obvious examples are detection of enemy soldiers or vehicles and missile
guidance. More advanced systems for missile guidance send the missile to an
area rather than a specific target, and target selection is made when the missile
reaches the area based on locally acquired image data. In this case, automatic
processing of the data is used to reduce complexity and to fuse information
from multiple sensors to increase reliability.
One of the newer application areas is autonomous vehicles, which include
submersibles, land-based vehicles (small robots with wheels, cars or trucks),
aerial vehicles, and unmanned aerial vehicles (UAV).The level of autonomy
ranges from fully unmanned vehicles to vehicles where computer vision based
systems support a driver or a pilot in various situations. [4]

8. PRO’S OF SIXTH SENSE TECHNOLOGY [14]
Easy to Draw: A person can draw anything on paint with the help of his
fingers. So, why to use mouse for painting.
Capture photos with fingers: why to take camera on your holiday and no
tension for the photo space as this Sixth Sense computer will work like
your camera. It captures the photo, when you make a square with your
fingers, highlighting which one you want to frame.
Phone Call: You can call to your friend by typing the numbers on your
hand. It displays the keypad of the phone over your palm and the keys


Page 21

appear on the four finger. Use your another hand's finger to press the
keys.
Check the brand of the Product: It helps you to choose the best brand
product from the super market.
Read Books easily: Check out the ratings of the Book you are going to
buy, it checks the ratings from the internet. And another amazing thing is
that it reads the book for you.
Newspapers: Did you saw the moving pictures of the Newspaper in the
movie Harry Potter; it is quite similar to it. It Searches the most
appropriate video from the web by seeing the headlines or the caption of
the News report.
Check your Flight Status: You can check the status of the flight while
you are on Taxi. Just place the ticket in front of the projector and it checks
its status from the internet.

9. APPLICATIONS OF SIXTH SENSE TECHNOLOGY [19]
The Sixth Sense prototype implements several applications that demonstrate the
usefulness, viability and flexibility of the system.
The map application lets the user navigate a map displayed on a nearby
surface using hand gestures, similar to gestures supported by multi-touch
based systems, letting the user zoom in, zoom out or pan using intuitive hand
movements

Fig.13: Map Application [5]


Page 22


Fig.14: Drawing Application [5]

• The drawing application lets the user draw on any surface by tracking the fingertip
movements of the user‘s index finger. Sixth Sense also recognizes user‘s freehand
gestures (postures). For example, it implements a gestural camera that takes photos of
the scene the user is looking at by detecting the ‗framing‘ gesture. The user can stop
by any surface or wall and flick through the photos he/she has taken. [8]
Sixth Sense also lets the user draw icons or symbols in the air using the
movement of the index finger and recognizes those symbols as interaction
instructions. For example, drawing a magnifying glass symbol takes the user
to the map application or drawing a ‗@‘ symbol lets the user check his mail.
The Sixth Sense system also augments physical objects the user is interacting
with by projecting more information about these objects projected on them.
For example, a newspaper can show live video news or dynamic information
can be provided on a regular piece of paper. The gesture of drawing a circle on
the user‘s wrist projects an analog watch.


Page 23


Fig.15: Drawing a circle on the user‘s wrist projects an analog watch. [5]

Make a call by Extend hand on front of the projector and numbers will appear
for to click. Know the time by Draw a circle on wrist and a watch will appear.
take a photo by Just make a square with fingers, highlighting what want to
frame, and the system will make the photo—which can later organize with the
others using own hands over the air.
And the device has a huge number of applications, it is portable and easily to
carry as can wear it in neck. The drawing application lets user draw on any
surface by observing the movement of index finger. Mapping can also be done
anywhere with the features of zooming in or zooming out. The camera also
helps user to take pictures of the scene is viewing and later can arrange them
on any surface.[8]
Some of the more practical uses are reading a newspaper. Reading a
newspaper and viewing videos instead of the photos in the paper. Or live
sports updates while reading the newspaper.


Page 24


Fig.16: News paper [5]

The device can also tell arrival, departure or delay time of air plane on tickets.

For book lovers it is nothing less than a blessing. Open any book and find the
Amazon ratings of the book. To add to it, pick any page and the device gives
additional information on the text, comments and lot more add on feature.
However, instead of requiring you to be in front of a big screen like Tom
Cruise, Sixth Sense can do its magic and a lot more everywhere, even while
you are jumping hysteric over Oprah's sofa. The camera recognizes objects
around you instantly, with the micro-projector overlaying the information on
any surface, including the object itself or your hand. Then, you can access or
manipulate the information using your fingers.
Need to know the time? Draw a circle on your wrist and a watch will appear.
Want to take a photo? Just make a square with your fingers, highlighting what
you want to frame and the system will make the photo which you can later
organize with the others using your own hands over the air. But those are just
novelty applications.
Need to make a call? Extend your hand on front of the projector and numbers
will appear for you to


Page 25


Fig.17: Numbers on Hand to make a Call [17]
The true power of Sixth Sense lies on its potential to connect the real world
with the Internet, and overlaying the information on the world itself. Imagine
you are at the supermarket, thinking about what brand of soap is better. Or
maybe what food you should get for tonight's dinner. Just look at objects, hold
them on your hands, and Sixth Sense will show you if it's good or bad, or if it
fits your preferences or not.
Now take this to every aspect of your everyday life. You can be in a taxi going
to the airport, and just by taking out your boarding pass, Sixth Sense will grab
real time information about your flight and display it over the ticket. You
won't need to do any action. Just hold it in front of your and it will work.
[11]

The current prototype system costs approximate $350 to build. [12],[13]
The key here is that Sixth Sense recognizes the objects around you, displaying
information automatically and letting you access it in any way you want, in the
simplest way possible.
Clearly, this has the potential of becoming the ultimate "transparent" user interface for
accessing information about everything around us. If they can get rid of the colored
finger caps and it ever goes beyond the initial development phase, that is. But as it is


Page 26

now, it may change the way we interact with the real world and truly give everyone
complete awareness of the environment around us. [11]

10. WHY SIXTH SENSE TECHNOLOGY?
Humans take decisions after acquiring inputs from the senses. But the information we
collect aren‘t enough to result in the right decisions. But the information which could
help making a good decision is largely available on internet. Although the information
can be gathered by connecting devices like computers and mobiles but they are
restricted to the screen and there is no direct interaction between the tangible physical
world and intangible digital world. This sixth sense technology provides us with the
freedom of interacting with the digital world with hand gestures. This technology has
a wide application in the field of artificial intelligence. This methodology can aid in
synthesis of bots that will be able to interact with humans.

11. FUTURE ENHANCEMENT
Further progress has been made with the system by broadcasting their latest
demonstration online. The new Sixth Sense system uses a camera that is hung from
the user‘s neck, instead of mounted on your hat. Also hanging from the neck is a pico
projector and a mirror that will display onto any surface that the user is looking at.
When we first heard about this system, the user had to have a mini computer strapped
to their back that would always be connected to the internet and could find the
required information. The new system eliminates the backpack and instead connects
to your cell phone to retrieve information. The Sixth Sense is still in its development
stages.


Page 27

Mouse less:

Fig.18 Mouse less [17]

Mouse less is a concept which is developed in order to avoid the usage of mouse in
computers. It helps to provide a familiarity of the physical mouse without actually
requiring a physical mouse.

Even though the computer hardware technology has been developed to a greater
extent, still there has been no proper substitute for a physical mouse. This new
invention aims to remove the requirement of having a physical mouse altogether but
still it provides the feeling of a physical mouse that users are familiar with. It
basically consists of an Infrared laser beam and an Infrared camera. Both laser and
camera are embedded together with the computer. The laser beam module is modified
with a line cap and placed such that it creates a plane of IR laser just above the surface
the computer sits on. The user cups their hand, as if a physical mouse was present
underneath, and the laser beam lights up the hand which is in contact with the surface.
The IR camera detects those bright IR blobs using computer vision. The change in the
position and arrangements of these blobs are interpreted as mouse cursor movement
and mouse clicks.


Page 28

Third Eye:
Third Eye is a new technique that enables multiple viewers to see different things on
a same display screen at the same time.

Fig.19: Third Eye [17]

With third Eye, We can have a public sign board where a Japanese tourist sees all the
instructions in Japanese and an American in English. - We don't need to have the split
screen in games now. Each player can see his/her personal view of the game on the
TV screen. - Two people watching TV can watch their favorite channel on a single
TV screen. - A public display can show secret messages or patterns. - In the same
movie theater, people can see different end of a suspense movie.

Inktuitive:
Even though there has been a tremendous development of computer aided designing
tools, still paper and pencil is the preferred by most of the designers ,especially in the
earlier stages of designing .


Page 29


Fig.19: Inktuitive [17]
The project ‗inktuitive‘ is aimed to combine the process of creation that is inherent in
paper and pencil with the power of computing that the digital design tools provide. It
also extends the natural work-practice of using physical paper by giving the pen the
ability to control the design in physical three-dimensional space, freeing it from its tie
to the paper. There always remains an intuition of pen and paper ,but the lines that
have been drawn are be captured and translated into shapes in the digital world. The
physical paper is augmented with overlaid digital strokes. The platform provides a
novel interaction mechanism for drawing and designing using above the surface pen
movements. In other words ‗inktuitive‘ can be considered as an intuitive physical
design workspace that aims to bridge the gap and bring together the conventional
design tools such as paper and pencil with the power and convenience of the digital
tools for design.

TaPuma:
TaPuma is the short form of Tangible Public Map. It is another attempt to bring the
digital world nearer to the real world by exploring the techniques of sixth sense
technology. Key word based search mechanism has vastly influenced and modified
the ways in which we search information‘s. The keyword based search allows the user
to filter out the required data from the vast data that are available on search engines
such as yahoo and google. TaPuMa allows people to use their own belongings, the
objects they usually carry with them to access relevant and just-in-time information
and locations of places from a public map. The project also explores and analyzes the
advantages and challenges of this novel interaction mechanism, where real life objects

Page 30

serve as the interface for information acquirement. The broad concept behind the
project TaPuMa is Object Amelioration, where the functions of everyday objects can
be expanded by using their affordances or functionalities in a variety of different
contexts.


Page 31

CONCLUSION
Technology has transformed the world in which we live, changing how we spend our
time, how we understand ourselves, and how we interact with others. Technological
innovation results in social and economic change. Thus, Sixth Sense Technology will
lead to the development of present world. And it is the present world that promises to
restructure human life and activity.


Page 32

REFERENCES
1. http://pdfdatabase.com/sixth-sense-technology-in-technical.html
2. http://microreaders.blogspot.com/2009/11/sixth-sense-technology-shows-thatwe.html
3. http://ghallabhansali.com
4. http://fluid.media.mit.edu/assets/_pubs/wip177-mistry.pdf
5. http://techinfo.org/5983/Gesture_recognition
6. http://www.pranavmistry.com/projects/sixthsense
7. http://techhost.edu/tech/Fiduciary_marker
8. http://www.college-seminars.com/sixth-sense-technology
9. http://gizmodo.com/5167790/sixth-sense-technology-may-change-how-welook-at-the-world-forever
10. http://www.pranavmistry.com/projects/sixthsense/
11. http://er-divya-pai.blogspot.in/2013/01/pranav-mistry-innovator-of-sixthsense.html
12. http://www.ted.com/talks/pranav_mistry_the_thrilling_potential_of_sixthsense
_technology.htm
13. http://www.commoncraft.com/augmented-reality
14. http://www.arcane-technologies.com/en/
15. http://www.ted.com/talks/pattie_maes_demos_the_sixth_sense.html
16. http://timesofindia.indiatimes.com/Health__Science/Sixth_sense_gizmo_a_sci
-fi_reality/articleshow/4083978.cms
17. http://www.seminarpapers.net/sixth-sense.pdf


Page 33

Sixth sense technology 04

More Related Content

What's hot

Viewers also liked

Similar to Sixth sense technology 04

More from akki_hearts

Recently uploaded

Sixth sense technology 04