Sixth sense technology 04


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Sixth sense technology 04

  1. 1. SIXTH SENSE TECHNOLOGY 1. INTRODUCTION Sixth Sense Technology is a mini-projector coupled with a camera and a cell phone which acts as the computer and connected to the Cloud, all the information stored on the web. Sixth Sense can also obey hand gestures. The camera recognizes objects around a person instantly, with the micro-projector overlaying the information on any surface, including the object itself or hand. Also can access or manipulate the information using fingers. Sixth Sense is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information. By using a camera and a tiny projector mounted in a pendant like wearable device. Sixth Sense sees what you see and visually augments any surfaces or objects we are interacting with. It projects information onto surfaces, walls, and physical objects around us, and lets us interact with the projected information through natural hand gestures, arm movements, or our interaction with the object itself. Sixth Sense attempts to free information from its confines by seamlessly integrating it with reality , and thus making the entire world your computer.[1] Fig 1. Parts used in Sixth Sense Technology [1] Akshay .N .Mahajan(120573116004) Page 1
  2. 2. SIXTH SENSE TECHNOLOGY 2. HOW SIXTH SENSE TECHNOLOGY WORKS The technology in itself is nothing more than the combination of some stunning technologies, but the idea of combining those technologies is really great. The technology is mainly based on hand gesture recognition, image capturing, processing, and manipulation, etc. The camera is used to recognize and track user‘s hand gestures and physical objects using computer-vision based techniques, while the projector is used to project visual information on walls or on any physical thing around us. Other hardware includes mirror and colored caps to be used for fingers The projector: projects visual information, enabling surfaces, walls and physical objects around the wearer to be used as interfaces; The camera and hands: recognizes and tracks the user's hand gestures and physical objects using computer-vision based techniques. The software program: processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking of colored fingers) at the tip of the user‘s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus Sixth Sense also supports multi-touch and multi-user interaction. Akshay .N .Mahajan(120573116004) Page 2
  3. 3. SIXTH SENSE TECHNOLOGY 3. ARCHITECTURE OF SIXTH SENSE TECHNOLOGY [3] Fig 2: Architecture of Sixth Sense Technology[3] Sixth Sense, a tool that connects the physical world with the world of data. Digital prototype called the ―Sixth Sense‖, which wearable hi-tech devices that can turn anything into a touch screen. Your wrist can become a watch and your palm a phone. The device is combination of a wearable camera, a projector and a mirror where the camera recognizes movements of the fingers of your hand and translates those into commands. The hardware components are coupled in a pendant like wearable device. Both the projector and the camera are connected to the mobile computing device in the user‘s pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks user‘s hand gestures and physical objects using computer vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the Akshay .N .Mahajan(120573116004) Page 3
  4. 4. SIXTH SENSE TECHNOLOGY colored markers (visual tracking fiducials) at the tip of the user‘s fingers using simple computer vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus Sixth Sense also supports multi touch and multi user interaction. 3.1 POCKET PROJECTOR [3] Pocket projector is an emerging technology that applies the use of a projector in a handheld device. It is also know as Handheld projector, pocket projector or mobile projector or Pico projector. It is a response to the emergence of compact portable devices such as mobile phones, personal digital assistants and digital cameras. Which have sufficient storage capacity to handle presentation materials but little space to accommodate an attached display screen. Handheld projectors involve miniaturized hardware and software that can project digital images onto any nearby viewing surface, such as a wall. This entire system is compacted into one very tiny chip. An important design characteristic of a handheld projector is the ability to project a clear image, regardless of the physical characteristics of the viewing surface. 3.2 PARTS OF POCKET PROJECTOR [3] The system comprises four main parts: The electronics The laser light sources The combiner optic, and the Scanning mirrors. Akshay .N .Mahajan(120573116004) Page 4
  5. 5. SIXTH SENSE TECHNOLOGY 3.3 THE WORKING OF SIXTH SENSE DEVICE [4] Fig.3 Working Of Sixth Sense [4] First, the electronics system turns the image into an electronic signal. Next the electronic signals drive laser light sources with different colors and intensities down different paths. In the combiner optic the different light paths are combined into one path demonstrating a pallet of colors. Finally, the mirrors copy the image pixel by pixel and can then project the image. 3.4 APPLICATIONS [8] Handheld projector can be used for similar applications than small conventional projectors. Study applications that are specifically designed for handheld projectors often using prototypes of mobile phones with an integrated projector. Gaming: Handheld projectors, in particular projector phones, could offer new possibilities for mobile gaming as demonstrated by the adaptation of the Playstation3 game Little Big Planet. Akshay .N .Mahajan(120573116004) Page 5
  6. 6. SIXTH SENSE TECHNOLOGY Hand gesture recognition: Size reduction of mobile devices is often limited by the size of the used display. Apart from the display a complete phone can be, for example, integrated in a headset. It has been demonstrated that Pico projectors integrated in headsets could be used as interaction devices e.g. using additional hand and finger tracking. Picture browsing: Projector phones allow photographs to be shared with a larger audience. The projected display allowed viewing of pictures by all present which is currently not possible using a single mobile device. [4] 3.5 MOBILE COMPUTING DEVICES A mobile device (also known as cell phone device, handheld device, handheld computer, "Palmtop" or simply handheld) is a pocket-sized computing device, typically having a display screen with touch input or a miniature keyboard. In the case of the personal digital assistant (PDA) the input and output are combined into a touchscreen interface. Smart phones and PDAs are popular amongst those who require the assistance and convenience of a conventional computer, in environments where carrying one would not be practical.[4] 3.5.1 TYPES OF MOBILE DEVICES [4] Mobile devices have been designed for many applications and include: Mobile computers. Graphing calculator. Handheld game consoles. Digital still camera (DSC). Digital video camera (DVC or digital camcorder). Portable media player. E-book reader. Mobile phone. Personal navigation devices(PNDs) Akshay .N .Mahajan(120573116004) Page 6
  7. 7. SIXTH SENSE TECHNOLOGY 3.5.2 USAGE Handheld devices have become use in mobile field management situations to record information in the field. They are used to achieve a variety of tasks for increasing efficiency that include digitizing notes , sending and receiving invoices, assets management, recording signatures, managing parts and scanning barcodes. Handheld computers used at work have molded over time into a variety of form including smart phones on the low end, handheld PDAs, Ultra Mobile PCs, tablet PCs .[4] 4. WEARABLE GESTURAL INTERFACE [5] Information is traditionally confined to paper or digitally to a screen. In this paper, we introduce WUW, a wearable gestural interface, which attempts to bring information out into the tangible world. By using a tiny projector and a camera mounted on a hat or coupled in a pendant like wearable device, WUW sees what the user sees and visually augments surfaces or physical objects the user is interacting with.WUW projects information onto surfaces, walls, and physical objects around us, and lets the user interact with the projected information through natural hand gestures, arm movements, or interaction with the object itself. A wearable device with a projector that allows to augment reality with a pendant projector. The device (via the web cam and coded finger-gloves) recognizes users gestures while searching for information, taking photos, making a call and presents it back to the user projected at any display. The recent advent of novel sensing and display technologies has encouraged the development of a variety of multi-touch and gesture based interactive systems. Such systems have moved computing onto surfaces such as tables and walls and brought the input and projection of digital interfaces into one-to-one correspondence such that the user may interact directly with information using touch and natural hand gestures. A parallel trend, the miniaturization of mobile computing devices permit ―anywhere‖ access to information, so we are always connected to the digital world. Unfortunately, most gestural and multi-touch based interactive systems are not mobile and small Akshay .N .Mahajan(120573116004) Page 7
  8. 8. SIXTH SENSE TECHNOLOGY mobile devices fail to provide the intuitive experience of full-sized gestural systems. Moreover, information still resides on screens or dedicated projection surfaces. There is no link between our interaction with these digital devices and interaction with the physical world around us. While it is impractical to modify all physical objects and surfaces into interactive touch-screens, and to turn everything into network enabled devices, it is possible to augment environment around us with visual information. When digital information is projected onto physical objects, instinctively it makes sense to interact with it in similar patterns as with physical objects: through hand gestures. In this paper, we present WUW, a computer-vision based wearable and gestural information interface that augments the physical world around us with digital information and proposes natural hand gestures as the mechanism to interact with that information. It is not the aim of this research to present an alternative to multi-touch or gesture recognition technology, but rather to explore the novel free-hand gestural interaction that WUW proposes. 4.1 What is WUW? [5] WUW is a wearable gestural interface. It consists of a camera and a small projector mounted on a hat or coupled in a pendant like mobile wearable device. The camera sees what the user sees and the projector visually augments surfaces or physical objects that the user is interacting with. WUW projects information onto the surfaces, walls, and physical objects around the user, and lets the user interact with the projected information through natural hand gestures, arm movements, or direct manipulation of the object itself. The tiny projector is connected to a laptop or mobile device and projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera tracks user hand gestures using simple computer vision based techniques. The current WUW prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. The map application(see Fig.3) lets the user navigate a map displayed on a nearby surface Akshay .N .Mahajan(120573116004) Page 8
  9. 9. SIXTH SENSE TECHNOLOGY using familiar hand gestures, letting the user zoom in, zoom out or pan using intuitive hand movements. The drawing application (see Fig.4) lets the user draw on any surface by tracking the fingertip movements of the user‘s index finger. The WUW prototype also implements a gestural camera that takes photos of the scene the user is looking at by detecting the ‗framing‘ gesture (see Fig.5). The user can stop by any surface or wall and flick through the photos he/she has taken. The WUW system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper (see Fig.6).The gesture of drawing a circle on the user‘s wrist projects an analog watch (see Fig.4). The WUW system also supports multiple users collaborating and interacting with the projected digital information using their hand movements and gestures. [5] Fig.4: Map Application [5] Akshay .N .Mahajan(120573116004) Page 9
  10. 10. SIXTH SENSE TECHNOLOGY Fig.5: Drawing Application [5] Fig.6: Framing Gesture [19] Fig.7: Newspaper [5] Akshay .N .Mahajan(120573116004) Page 10
  11. 11. SIXTH SENSE TECHNOLOGY Fig.8:Analog watch [5] 4.2 WUW prototype [5] The WUW prototype is comprised of three main hardware components: a pocket projector (3MMPro110), a camera (Logitech Quick Cam) and a laptop computer. The software for the WUW prototype is developed on a Microsoft Windows platform using C#. Both the projector and the camera are mounted on a hat and connected to the laptop computer in the user‘s back pack. The prototype system uses plastic colored-markers (e.g. caps of whiteboard markers) as the visual tracking fiducials. Fig.9: WUW prototype system[5] Akshay .N .Mahajan(120573116004) Page 11
  12. 12. SIXTH SENSE TECHNOLOGY The movements and arrangements of these color markers are interpreted into gestures that act as interaction mechanism for the application interfaces projected by the projector. The current prototype only tracks 4 fingers – index fingers and thumbs. This is largely due to the relative importance of the index finger and thumb in natural hand gestures, tracking these WUW is able to recognize a wide variety of gestures and postures. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus the WUW also supports multi-user interaction. The prototype system also implements a few object augmentation applications where the camera recognizes and tracks physical objects such as a newspaper, a coffee cup, or a book and the projector projects relevant visual information on them. WUW uses computer-vision techniques in order to track objects and align projected visual information by matching pre-printed static color markers (or patterns) on these objects with the markers projected by the projector. 4.3 CHALLANGES There are many challenges associated with the accuracy and usefulness of gesture recognition software. For image-based gesture recognition there are limitations on the equipment used and image noise. Images or video may not be under consistent lighting, or in the same location. Items in the background or distinct features of the users may make recognition more difficult. The variety of implementations for imagebased gesture recognition may also cause issue for viability of the technology to general usage. For example, an algorithm calibrated for one camera may not work for a different camera. The amount of background noise also causes tracking and recognition difficulties, especially when occlusions (partial and full) occur. Furthermore, the distance from the camera, and the camera's resolution and quality, also cause variations in recognition accuracy. [5] 5. HAND GESTURE RECOGNITION [6] Hand gesture recognition is a topic in computer science and language technology with the goal of interpreting human gestures via mathematical algorithms. Gestures can Akshay .N .Mahajan(120573116004) Page 12
  13. 13. SIXTH SENSE TECHNOLOGY originate from any bodily motion or state but commonly originate from the face or hand. Current focuses in the field include emotion recognition from the face and hand gesture recognition. Many approaches have been made using cameras and computer vision algorithms to interpret sign language. However, the identification and recognition of posture, gait, proxemics, and human behaviors is also the subject of gesture recognition techniques. Hand gesture recognition can be seen as a way for computers to begin to understand human body language, thus building a richer bridge between machines and humans than primitive text user interfaces or even GUIs (graphical user interfaces), which still limit the majority of input to keyboard and mouse. Hand gesture recognition enables humans to interface with the machine (HMI) and interact naturally without any mechanical devices. Using the concept of gesture recognition, it is possible to point a finger at the computer screen so that the cursor will move accordingly. This could potentially make conventional input devices such as mouse, keyboards and even touch-screen redundant. It can be conducted with techniques from computer vision and image processing. The literature includes ongoing work in the computer vision field on capturing gestures or more general human pose and movements by cameras connected to computer .Gesture recognition can be conducted with techniques from computer vision and image processing. The literature includes ongoing work in the computer vision field on capturing gestures or more general human pose and movements by cameras connected to a computer. Akshay .N .Mahajan(120573116004) Page 13
  14. 14. SIXTH SENSE TECHNOLOGY Fig.10: A child being sensed by a simple hand gesture recognition detecting hand location and movement.[6] 5.1 USES [6] Gesture recognition is useful for processing information from humans which is not conveyed through speech or type. As well, there are various types of gestures which can be identified by computers. Sign language recognition: - Just as speech recognition can transcribe speech to text, certain types of gesture recognition software can transcribe the symbols represented through sign language into text. For socially assistive robotics: - By using proper sensors (accelerometers and gyros) worn on the body of a patient and by reading the values from those sensors, robots can assist in patient rehabilitation. The best example can be stroke rehabilitation. Directional indication through pointing: - Pointing has a very specific purpose in our society, to reference an object or location based on its position relative to ourselves. The use of gesture recognition to determine where a person is pointing is useful for identifying the context of statements or instructions. This application is of particular interest in the field of robotics. Control through facial gestures: - Controlling a computer through facial gestures is a useful application of gesture recognition for users who may not Akshay .N .Mahajan(120573116004) Page 14
  15. 15. SIXTH SENSE TECHNOLOGY physically be able to use a mouse or keyboard. Eye tracking in particular may be of use for controlling cursor motion or focusing on elements of a display. Alternative computer interfaces: - Foregoing the traditional keyboard and mouse setup to interact with a computer, strong gesture recognition could allow users to accomplish frequent or common tasks using hand or face gestures to a camera. Immersive game technology: - Gestures can be used to control interactions within video games to try and make the game player's experience more interactive or immersive. Virtual controllers: - For systems where the act of finding or acquiring a physical controller could require too much time, gestures can be used as an alternative control mechanism. Controlling secondary devices in a car or controlling a television set are examples of such usage. Affective computing: - In affective computing, gesture recognition is used in the process of identifying emotional expression through computer systems. Remote control: - Through the use of gesture recognition, remote control with the wave of a hand of various devices is possible. The signal must not only indicate the desired response, but also which device to be controlled. 5.2 INPUT DEVICES The ability to track a person's movements and determine what gestures they may be performing can be achieved through various tools. Although there is a large amount of research done in image/video based gesture recognition, there is some variation within the tools and environments used between implementations. Depth-aware cameras: - Using specialized cameras such as time-of-flight cameras, one can generate a depth map of what is being seen through the camera at a short range, and use this data to approximate a 3d representation of what is being seen. These can be effective for detection of hand gestures due to their short range capabilities. [6] Stereo cameras: - Using two cameras whose relations to one another are known, a 3d representation can be approximated by the output of the cameras. To get the cameras' relations, one can use a positioning reference such as a lexian-stripe or in framed emitters. [6] Akshay .N .Mahajan(120573116004) Page 15
  16. 16. SIXTH SENSE TECHNOLOGY Controller-based gestures: - These controllers act as an extension of the body so that when gestures are performed, some of their motion can be conveniently captured by software. Mouse gestures are one such example, where the motion of the mouse is correlated to a symbol being drawn by a person's hand, as is the Wii Remote, which can study changes in acceleration over time to represent gestures.[6] Single camera: - A normal camera can be used for gesture recognition where the resources/environment would not be convenient for other forms of imagebased recognition. Although not necessarily as effective as stereo or depth aware cameras, using a single camera allows a greater possibility of accessibility to a wider audience.[6] 5.3 MULTI TOUCH [10] Multi-touch is an enhancement to touch screen technology, which provides the user with the ability to apply multiple finger gestures simultaneously onto the electronic visual display to send complex commands to the device. The term multi-touch is a trade mark of Apple Inc. Multi-touch has been implemented in several different ways, depending on the size and type of interface. Both touch tables and touch walls project an image through acrylic or glass, and then backlight the image with LED's. When a finger or an object touches the surface, causing the light to scatter, the reflection is caught with sensors or cameras that send the data to software which dictates response to the touch, depending on the type of reflection measured. Touch surfaces can also be made pressuresensitive by the addition of a pressure-sensitive coating that flexes differently depending on how firmly it is pressed, altering the reflection. Handheld technologies use a panel that carries an electrical charge. When a finger touches the screen, the touch disrupts the panel's electrical field. The disruption is registered and sent to the software, which then initiates a response to the gesture. Akshay .N .Mahajan(120573116004) Page 16
  17. 17. SIXTH SENSE TECHNOLOGY 6. IMAGE CAPTURING [10] Image Capture is an application program that enables users to upload pictures from digital cameras or scanners which are either connected directly to the computer or the network. It provides no organizational tools like Photo but is useful for collecting pictures from a variety of sources with no need for drivers. It achieves this by receiving the picture as it is and through a conversion process, downloads it onto your computer. It uses QuickTime Image Capture. This way it doesn't need to understand anything about the actual camera to be able to do that. It was first introduced in Mac OS X version 10.0(cheetah). Fig.11 The user taking making a pose gesture [19] Fig.12 The camera takes the photo of the location [19] Akshay .N .Mahajan(120573116004) Page 17
  18. 18. SIXTH SENSE TECHNOLOGY 6.1 IMAGE CAPTURE API Image Capture is also the name of the Application Programming Interface associated with the Image Capture Application. Drivers or software built to use this interface can also automatically interface many imaging systems and software that run on Mac OS X. [19] 6.2 FIDUCIAL A fiduciary marker or fiducial is an object used in the field of view of an imaging system which appears in the image produced, for use as a point of reference or a measure. It may be either something placed into or on the imaging subject, or a mark or set of marks in the reticle of an optical instrument. In physics, 3D computer graphics and photography fiducials are reference points: fixed points or lines within a scene to which other objects can be related or against which objects can be measured. Fiduciary markers are used in a wide range of medical imaging applications. Images of the same subject produced with two different imaging systems may be correlated by placing a fiduciary marker in the area imaged by both systems. In this case, a marker which is visible in the images produced by both imaging modalities must be used. [ 7] 7.0TECHNOLOGIE’S THAT ARE RELATED TO SIXTH SENSE DEVICES 7.1 AUGMENTED REALITY [13] The augmented Reality is a visualization technology that allows the user to experience the virtual experience added over real world in real time.With the help of advanced AR technology the information about the surrounding real world of the user becomes interactive and digitally usable. Artificial information about the environment and the objects in it can be stored and retrieved as an information layer on top of the real world view. When we compare the spectrum between virtual reality, which creates immersive, computer-generated environments, and the real world, augmented reality is closer to the real world. Augmented reality adds graphics, sounds, hap tic feedback Akshay .N .Mahajan(120573116004) Page 18
  19. 19. SIXTH SENSE TECHNOLOGY and smell to the natural world as it exists. Both video games and cell phones are driving the development of augmented reality. The augmented systems will also superimpose graphics for every perspective available and try adjust to every movement of the user's head and eyes. The three basic components of an augmented reality system are the head-mounted display, tracking system and mobile computer for the hardware. The main goal of this new technology is to merge these three components into a highly portable unit much like a combination of a high tech Walkman and an ordinary pair or eyeglasses. The head-mounted display used in augmented reality systems will enable the user to view superimposed graphics and text created by the system. Another component of an augmented reality system is its tracking and orientation system. This system pinpoints the user's location in reference to his surroundings and additionally tracks the user's eye and head movements. Augmented reality systems will need highly mobile computers. As of now many computers aren‘t there to satisfy to provide this option. 7.2GESTURE RECOGNIZATION It is a technology which is aimed at interpreting human gestures with the help of mathematical algorithms. Gesture recognition technique basically focuses on the emotion recognition from the face and hand gesture recognition. Gender recognition technique enables humans to interact with computers in a more direct way without using any external interfacing devices. It can provide a much better alternative to text user interfaces and graphical user interface which requires the need of a keyboard or mouse to interact with the computer. Interfaces which solely depends on the gestures requires precise hand pose tracking. In the early versions of gesture recognition process special type of hand gloves which provide information about hand position orientation and flux of the fingers. In the Sixth Sense devices coloured bands are used for this purpose. Once hand pose has been captured the gestures can be recognised using different technique‘s. Neural network approaches or statistical templates are the commonly used techniques used for the recognition purposes. This technique have an high accuracy usually showing accuracy of more than 95%. Time dependent neural network will also be used for real time recognition of the gestures. Akshay .N .Mahajan(120573116004) Page 19
  20. 20. SIXTH SENSE TECHNOLOGY 7.3 COMPUTER VISION [4] Computer vision is the technology in which machines are able to interpret/extract necessary information from an image. Computer vision technology includes various fields like image processing, image analysis and machine vision. It includes certain aspect of artificial intelligence techniques like pattern recognition. The machines which implement computer vision techniques require image sensors which detect electromagnetic radiation which are usually in the form of ultraviolet rays or light rays. The computer vision find itself applicable in varies field of interest. One such field is bio medical image processing. It‘s also used in autonomous vehicle like SUV‘s. The computer vision technique basically includes four processes. 1. Recognition: One of the main task of computer vision technique is to determine whether the particular object contain the useful data or not. 2. Motion Analysis: Motion analysis includes several tasks related to estimation of motion where an image sequence is processed continuously to detect the velocity at each point of the image or in the 3D scene. 3. Scene Reconstruction: Computer vision technique employs several methods to recreate a 3D image from the available images of a scene. 4. Image Restoration: The main of aim of this step is to remove noise from an given image. The simplest method involves using simple filters like low pass or median filters. In order to get better quality images more complex methods like In painting are used. 7.3.1 APPLICATION FOR COMPUTER VISION One of the most prominent application fields is medical computer vision or medical image processing. This area is characterized by the extraction of information from image data for the purpose of making a medical diagnosis of a patient. Generally, image data is in the form of microscopy images, X-ray images, angiography images, ultrasonic images, and tomography images. An Akshay .N .Mahajan(120573116004) Page 20
  21. 21. SIXTH SENSE TECHNOLOGY example of information which can be extracted from such image data is detection of tumors, arteriosclerosis or other malign changes. A second application area in computer vision is in industry, sometimes called machine vision, where information is extracted for the purpose of supporting a manufacturing process. One example is quality control where details or final products are being automatically inspected in order to find defects. Another example is measurement of position and orientation of details to be picked up by a robot arm. Military applications are probably one of the largest areas for computer vision. The obvious examples are detection of enemy soldiers or vehicles and missile guidance. More advanced systems for missile guidance send the missile to an area rather than a specific target, and target selection is made when the missile reaches the area based on locally acquired image data. In this case, automatic processing of the data is used to reduce complexity and to fuse information from multiple sensors to increase reliability. One of the newer application areas is autonomous vehicles, which include submersibles, land-based vehicles (small robots with wheels, cars or trucks), aerial vehicles, and unmanned aerial vehicles (UAV).The level of autonomy ranges from fully unmanned vehicles to vehicles where computer vision based systems support a driver or a pilot in various situations. [4] 8. PRO’S OF SIXTH SENSE TECHNOLOGY [14] Easy to Draw: A person can draw anything on paint with the help of his fingers. So, why to use mouse for painting. Capture photos with fingers: why to take camera on your holiday and no tension for the photo space as this Sixth Sense computer will work like your camera. It captures the photo, when you make a square with your fingers, highlighting which one you want to frame. Phone Call: You can call to your friend by typing the numbers on your hand. It displays the keypad of the phone over your palm and the keys Akshay .N .Mahajan(120573116004) Page 21
  22. 22. SIXTH SENSE TECHNOLOGY appear on the four finger. Use your another hand's finger to press the keys. Check the brand of the Product: It helps you to choose the best brand product from the super market. Read Books easily: Check out the ratings of the Book you are going to buy, it checks the ratings from the internet. And another amazing thing is that it reads the book for you. Newspapers: Did you saw the moving pictures of the Newspaper in the movie Harry Potter; it is quite similar to it. It Searches the most appropriate video from the web by seeing the headlines or the caption of the News report. Check your Flight Status: You can check the status of the flight while you are on Taxi. Just place the ticket in front of the projector and it checks its status from the internet. 9. APPLICATIONS OF SIXTH SENSE TECHNOLOGY [19] The Sixth Sense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by multi-touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements Fig.13: Map Application [5] Akshay .N .Mahajan(120573116004) Page 22
  23. 23. SIXTH SENSE TECHNOLOGY Fig.14: Drawing Application [5] • The drawing application lets the user draw on any surface by tracking the fingertip movements of the user‘s index finger. Sixth Sense also recognizes user‘s freehand gestures (postures). For example, it implements a gestural camera that takes photos of the scene the user is looking at by detecting the ‗framing‘ gesture. The user can stop by any surface or wall and flick through the photos he/she has taken. [8] Sixth Sense also lets the user draw icons or symbols in the air using the movement of the index finger and recognizes those symbols as interaction instructions. For example, drawing a magnifying glass symbol takes the user to the map application or drawing a ‗@‘ symbol lets the user check his mail. The Sixth Sense system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. The gesture of drawing a circle on the user‘s wrist projects an analog watch. Akshay .N .Mahajan(120573116004) Page 23
  24. 24. SIXTH SENSE TECHNOLOGY Fig.15: Drawing a circle on the user‘s wrist projects an analog watch. [5] Make a call by Extend hand on front of the projector and numbers will appear for to click. Know the time by Draw a circle on wrist and a watch will appear. take a photo by Just make a square with fingers, highlighting what want to frame, and the system will make the photo—which can later organize with the others using own hands over the air. And the device has a huge number of applications, it is portable and easily to carry as can wear it in neck. The drawing application lets user draw on any surface by observing the movement of index finger. Mapping can also be done anywhere with the features of zooming in or zooming out. The camera also helps user to take pictures of the scene is viewing and later can arrange them on any surface.[8] Some of the more practical uses are reading a newspaper. Reading a newspaper and viewing videos instead of the photos in the paper. Or live sports updates while reading the newspaper. Akshay .N .Mahajan(120573116004) Page 24
  25. 25. SIXTH SENSE TECHNOLOGY Fig.16: News paper [5] The device can also tell arrival, departure or delay time of air plane on tickets. For book lovers it is nothing less than a blessing. Open any book and find the Amazon ratings of the book. To add to it, pick any page and the device gives additional information on the text, comments and lot more add on feature. However, instead of requiring you to be in front of a big screen like Tom Cruise, Sixth Sense can do its magic and a lot more everywhere, even while you are jumping hysteric over Oprah's sofa. The camera recognizes objects around you instantly, with the micro-projector overlaying the information on any surface, including the object itself or your hand. Then, you can access or manipulate the information using your fingers. Need to know the time? Draw a circle on your wrist and a watch will appear. Want to take a photo? Just make a square with your fingers, highlighting what you want to frame and the system will make the photo which you can later organize with the others using your own hands over the air. But those are just novelty applications. Need to make a call? Extend your hand on front of the projector and numbers will appear for you to Akshay .N .Mahajan(120573116004) Page 25
  26. 26. SIXTH SENSE TECHNOLOGY Fig.17: Numbers on Hand to make a Call [17] The true power of Sixth Sense lies on its potential to connect the real world with the Internet, and overlaying the information on the world itself. Imagine you are at the supermarket, thinking about what brand of soap is better. Or maybe what food you should get for tonight's dinner. Just look at objects, hold them on your hands, and Sixth Sense will show you if it's good or bad, or if it fits your preferences or not. Now take this to every aspect of your everyday life. You can be in a taxi going to the airport, and just by taking out your boarding pass, Sixth Sense will grab real time information about your flight and display it over the ticket. You won't need to do any action. Just hold it in front of your and it will work. [11] The current prototype system costs approximate $350 to build. [12],[13] The key here is that Sixth Sense recognizes the objects around you, displaying information automatically and letting you access it in any way you want, in the simplest way possible. Clearly, this has the potential of becoming the ultimate "transparent" user interface for accessing information about everything around us. If they can get rid of the colored finger caps and it ever goes beyond the initial development phase, that is. But as it is Akshay .N .Mahajan(120573116004) Page 26
  27. 27. SIXTH SENSE TECHNOLOGY now, it may change the way we interact with the real world and truly give everyone complete awareness of the environment around us. [11] 10. WHY SIXTH SENSE TECHNOLOGY? Humans take decisions after acquiring inputs from the senses. But the information we collect aren‘t enough to result in the right decisions. But the information which could help making a good decision is largely available on internet. Although the information can be gathered by connecting devices like computers and mobiles but they are restricted to the screen and there is no direct interaction between the tangible physical world and intangible digital world. This sixth sense technology provides us with the freedom of interacting with the digital world with hand gestures. This technology has a wide application in the field of artificial intelligence. This methodology can aid in synthesis of bots that will be able to interact with humans. 11. FUTURE ENHANCEMENT Further progress has been made with the system by broadcasting their latest demonstration online. The new Sixth Sense system uses a camera that is hung from the user‘s neck, instead of mounted on your hat. Also hanging from the neck is a pico projector and a mirror that will display onto any surface that the user is looking at. When we first heard about this system, the user had to have a mini computer strapped to their back that would always be connected to the internet and could find the required information. The new system eliminates the backpack and instead connects to your cell phone to retrieve information. The Sixth Sense is still in its development stages. Akshay .N .Mahajan(120573116004) Page 27
  28. 28. SIXTH SENSE TECHNOLOGY Mouse less: Fig.18 Mouse less [17] Mouse less is a concept which is developed in order to avoid the usage of mouse in computers. It helps to provide a familiarity of the physical mouse without actually requiring a physical mouse. Even though the computer hardware technology has been developed to a greater extent, still there has been no proper substitute for a physical mouse. This new invention aims to remove the requirement of having a physical mouse altogether but still it provides the feeling of a physical mouse that users are familiar with. It basically consists of an Infrared laser beam and an Infrared camera. Both laser and camera are embedded together with the computer. The laser beam module is modified with a line cap and placed such that it creates a plane of IR laser just above the surface the computer sits on. The user cups their hand, as if a physical mouse was present underneath, and the laser beam lights up the hand which is in contact with the surface. The IR camera detects those bright IR blobs using computer vision. The change in the position and arrangements of these blobs are interpreted as mouse cursor movement and mouse clicks. Akshay .N .Mahajan(120573116004) Page 28
  29. 29. SIXTH SENSE TECHNOLOGY Third Eye: Third Eye is a new technique that enables multiple viewers to see different things on a same display screen at the same time. Fig.19: Third Eye [17] With third Eye, We can have a public sign board where a Japanese tourist sees all the instructions in Japanese and an American in English. - We don't need to have the split screen in games now. Each player can see his/her personal view of the game on the TV screen. - Two people watching TV can watch their favorite channel on a single TV screen. - A public display can show secret messages or patterns. - In the same movie theater, people can see different end of a suspense movie. Inktuitive: Even though there has been a tremendous development of computer aided designing tools, still paper and pencil is the preferred by most of the designers ,especially in the earlier stages of designing . Akshay .N .Mahajan(120573116004) Page 29
  30. 30. SIXTH SENSE TECHNOLOGY Fig.19: Inktuitive [17] The project ‗inktuitive‘ is aimed to combine the process of creation that is inherent in paper and pencil with the power of computing that the digital design tools provide. It also extends the natural work-practice of using physical paper by giving the pen the ability to control the design in physical three-dimensional space, freeing it from its tie to the paper. There always remains an intuition of pen and paper ,but the lines that have been drawn are be captured and translated into shapes in the digital world. The physical paper is augmented with overlaid digital strokes. The platform provides a novel interaction mechanism for drawing and designing using above the surface pen movements. In other words ‗inktuitive‘ can be considered as an intuitive physical design workspace that aims to bridge the gap and bring together the conventional design tools such as paper and pencil with the power and convenience of the digital tools for design. TaPuma: TaPuma is the short form of Tangible Public Map. It is another attempt to bring the digital world nearer to the real world by exploring the techniques of sixth sense technology. Key word based search mechanism has vastly influenced and modified the ways in which we search information‘s. The keyword based search allows the user to filter out the required data from the vast data that are available on search engines such as yahoo and google. TaPuMa allows people to use their own belongings, the objects they usually carry with them to access relevant and just-in-time information and locations of places from a public map. The project also explores and analyzes the advantages and challenges of this novel interaction mechanism, where real life objects Akshay .N .Mahajan(120573116004) Page 30
  31. 31. SIXTH SENSE TECHNOLOGY serve as the interface for information acquirement. The broad concept behind the project TaPuMa is Object Amelioration, where the functions of everyday objects can be expanded by using their affordances or functionalities in a variety of different contexts. Akshay .N .Mahajan(120573116004) Page 31
  32. 32. SIXTH SENSE TECHNOLOGY CONCLUSION Technology has transformed the world in which we live, changing how we spend our time, how we understand ourselves, and how we interact with others. Technological innovation results in social and economic change. Thus, Sixth Sense Technology will lead to the development of present world. And it is the present world that promises to restructure human life and activity. Akshay .N .Mahajan(120573116004) Page 32
  33. 33. SIXTH SENSE TECHNOLOGY REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. _technology.htm 13. 14. 15. 16. -fi_reality/articleshow/4083978.cms 17. Akshay .N .Mahajan(120573116004) Page 33