IBM's Blue Eyes Technology Gives Computers Human Perceptual Abilities

BLUE EYES TECHNOLOGY
1
DEFINITION
Blue Eyes is a technology conducted by the research team of IBM at its
Almaden Research Center (ARC) in San Jose, California since 1997. Blue eyes
technology makes a computer to understand and sense human feelings and
behavior and also enables the computer to react according to the sensed emotional
levels. The aim of the blue eyes technology is to give human power or abilities to
a computer, so that the machine can naturally interact with human beings as we
interact with each other. All human beings have some perceptual capabilities, the
ability to understand each other’s emotional level or feelings from their facial
expressions. Blue eyes technology aims at creating a computer that have the
abilities to understand the perceptual powers of human being by recognizing their
facial expressions and react accordingly to them.
Imagine, a beautiful world, where humans collaborate with computers!! .The
computer can talk, listen or screech aloud!! .With the help ofspeech recognition
and facial recognition systems, computers gathers information from the users
and starts interacting with them according to their mood variations. Computer
recognizes your emotional levels by a simple touch on the mouse and it can
interact with us as an intimate partner. The machine feels your presence; verifies
your identity and starts interacting with you and even it will dial and call to your
home at any urgent situations. This all is happening with this “Blue Eyes”
technology.
The main objective of Blue eyes technology is to develop a computational
machine having sensory and perceptual ability like those of humans. The Blue
Eyes technology system is a combination of a set of hardware and software
systems.
HISTORY OF BLUE EYES TECHNOLOGY
The Blue Eyes technology aims at creating computational machines that have
perceptual and sensory ability like those of human beings. It uses non-obtrusive
sensing method, employing most modern video cameras and microphones to
identify the user’s actions through the use of imparted sensory abilities. The
machine can understand what a user wants, where he is looking at, and even
realize his physical or emotional states. The Blue Eyes Technology developed is
intended to be a complex solution for monitoring and recording the operator’s
conscious brain involvement as well as his/her physiological condition. This
shows yet another development in the field of Brain Computer Interface.

2
Blue Eyes system consists of a mobile measuring device called Data Acquisition
Unit (DAU) and a central analytical system called Central System Unit (CSU)
interconnected by Bluetooth. DAU collects information from the sensor and
sends it over the Bluetooth and delivers the messages sent from CSU to the
operator. CSU buffers incoming sensor data and provides visualization interface.
The basic idea behind Blue Eyes Technology is to give computer the human
power i.e.
▪ It uses non-obtrusive sensing method, employing most modern video
cameras and microphones to identify the user’s actions through the use of
imparted sensory abilities.
▪ The blue eyes system checks the physiological parameters like eye
movement, heart beat rate and blood oxygenation against abnormal and
undesirable values and triggers user-defined alarms when necessary.
▪ Blue eyes technology requires designing a personal area network linking all
the operators and the supervising system.
▪ As the operator using his sight and hearing, senses the state of the controlled
system, the supervising system will look after his physiological condition.
▪ The use of a miniature CMOS camera integrated into the eye movement
sensor will enable the system to calculate the point of gaze and observe what
the operator is actually looking at.
The Blue Eyes system has hardware with software loaded on it Blue Eyes
system can be applied in every working environment requiring permanent
operators attention for it. The hardware comprises of DAU and CSU.
Paul Ekman’s facial expression work gave the correlation between a person’s
emotional state and a person’s physiological measurements, which
described Facial Action Coding System (Ekman and Rosenberg, 1997). His
experiment involved participants attached to devices to record certain

3
measurements including pulse, galvanic skin response (GSR), temperature and
somatic movement.
A) Experiment:
The experiment involves devices attached to participants to record certain
measurements including pulse, galvanic skin response (GSR), temperature and
somatic movement. Six participants were trained to exhibit the facial expressions
of the six basic emotions, anger, fear, sadness, disgust, joy and surprise. The
physiological changes associated with affect were assessed and analyzed.
Because of our need to incorporate these measurements into a small, non-
intrusive form, we will explore taking these measurements from the hand, which
requires an emotional sensor such as a mouse.
B) Results:
The first analysis used multidimensional scaling (MDS) procedure to determine
the dimensionality of the data and suggested that the physiological similarities
and dissimilarities of the six emotional states fit within a four-dimensional model.
In the second analysis, discriminant function analysis was used to determine the
mathematic function that would distinguish the six emotional state which
suggested that all four physiological variables are sufficient to distinguish the six
states.
AIMS
1) To design smarter devices
2) To create devices with emotional intelligence
3) To create computational devices with perceptual abilities
The idea of giving computers personality or, more accurately, emotional
intelligence" may seem creepy, but the technologists say such machines would
offer important advantages.
De-spite their lightning speed and awesome powers of computation,
today's PCs are essentially deaf, dumb, and blind. They can't see you, they can't
hear you, and they certainly don't care a whit how you feel. Every computer user
knows the frustration of nonsensical error messages, buggy software, and abrupt
system crashes. We might berate the computer as if it was an unruly child, but, of
course, the machine can't respond. "It's ironic that people feel like dummies in
front of their computers, when in fact the computer is the dummy," says Rosalind
Picard, a computer science professor at the MIT Media Lab in Cambridge.
A computer endowed with emotional intelligence, on the other hand, could
recognize when its operator is feeling angry or frustrated and try to respond in an

4
appropriate fashion. Such a computer might slow down or replay a tutorial
program for a confused student, or recognize when a designer is burned out and
suggest he take a break. It could even play a recording of Beethoven's "Moonlight
Sonata" if it sensed anxiety or serve up a rousing Springsteen anthem if it detected
lethargy. The possible applications of "emotion technology" extend far beyond
the desktop.
A car equipped with an affective computing system could recognize when
a driver is feeling drowsy and ad-vise her to pull over, or it might sense when a
stressed-out motorist is about to explode and warn him to slow down and cool
off.
Human cognition depends primarily on the ability to perceive, interpret,
and integrate audio-visuals and sensoring information. Adding extraordinary
perceptual abilities to computers would enable computers to work together with
human beings as intimate partners.Researchers are attempting to add more
capabilities to computers that will allow them to interact like humans, recognize
human presents, talk, listen, or even guess their feelings.
NEED OF BLUE EYE TECHNOLOY
Artificial intelligence aims in creating intelligence to the ³Machines´. Intelligence
alone cannot make a machine perfect; even then it can only perform the
work it is designed for. But they never understood the perceptions or
emotions of Human beings. Blue Eye Technology now aims
at incorporating these perceptions and sensory abilities into the computational
machines. Machines will be more powerful ,effective and helpful if they even
have a little of these perceptions in them. In future, we might see in reality of a
computer playing your favourite movie, songs or photos to cheer you up. You
might just check your mails by asking your computer, or call your friends by
telling the computer to make the call for you. Those which were considered to be
a science fiction will be reality soon enough with the help of this technology.
Cars with emotional intelligence can reduce the number of accidents, by
judging the emotional state of the driver and help him driving safely.
The technology aims to enable these perceptions in machines by using
natural inputs like facial expressions. This technology has a power to
change the future of the world.
A computer on every desk´ - was the vision of Bill gates when he started
Microsoft. The vision has now become the almost reality of the present
world. As technologies evolve, old ones are replaced by the new ones.

5
Like Floppies were got ridden by CDs and Pen drives, Walkman which almost
dominated the market for a long time is dead now because of mp3
players and iPods. Similarly prospective future of Computers is Blue Eye
Technology. Since nineties, the world has seen a rapid change because of
computers. It has changed the way the world was working before and has brought
us to a scenario where no business can be called big and successful
without computers in them as apart of it. Animal survival depends on
highly developed sensory abilities. Likewise human cognition depends
on highly developed abilities to perceive, integrate and interpret visual, auditory
and touch information. If such perceptions are added to computers even in a small
fraction, definitely computers would become even more powerful than they are
today. Also they can help human beings in much more better manner. Blue Eye
aims at creating the perceptual and sensual abilities to computers. IBM scientists,
at Almaden Research Center (ARC) in San Jose, the main IBM research
laboratory, are trying to enable the computers to listen to our speech,
sense our gaze and read our body language since 1997.
Affective Computing
is computing that relates to, arises from, or deliberately influences
emotion or other affective phenomena.
Affective Computing research combines engineering and computer science with
psychology, cognitive science, neuroscience, sociology, education,
psychophysiology, value-cantered design, ethics, and more. The various steps
involved ineffective computing are
• Adding sensory abilities to computers
• Detecting the emotions of human being
• Respond accordingly
They are trying to enable these senses to computers with the help
of video cameras, speech recognition technology, gaze tracking
systems, facial geometry to detect expressions etc to gather key
information. This information is then analysed to determine the user's physical,
emotional, or informational state, which in turn can be used to help make
the user more productive by performing expected actions or by providing
expected information. The exciting science fiction of being able to tell your
computer what it should do is not far away from being reality with the
help of these technologies in place. Machines then can act according to the
commands received from the users by their senses rather than requiring user
to be present physically present before the system and enter the various
commands to execute the desired action. For example, ask the computer for a
search in Google, turn on music or turn off, make a phone call for you etc,
or even better computers can sense your emotional state and try reacting
accordingly like trying to play your favourite music when you are tensed or little
low etc. In the future, ordinary household devices -- such as televisions,

6
refrigerators, and ovens -- may be able to do their jobs when we look at
them and speak to them. Different ways to get inputs from the user can be:
• Gestures - Video monitoring
• Speech - ex. Artificial intelligence Speech Recognition systems
• Gaze Tracking - ex. MAGIC
• Facial expressions - ex. Facial geometry
• Touch - ex. Emotional mouse
Some of these examples can be discussed briefly to gather idea about
implementation of the technology in the machines.
In general, the blue eye technology aims at :
creating interactive computer.
computer acts as partner (and friend) to the user.
realizes his physical or emotional states.
gives computer human power.
provide technical means for monitoring and recording operator physiological
conditions
Design smarter devices .
Create devices with emotional intelligence .
Create computational devices with perceptual abilities.
BLUE EYE TECHNOLOGY:
The complex solution for human-operator monitoring:
Visual attention monitoring
Physiological condition
Operator’s position detection
Wireless data acquisition using Bluetooth
Real time user defined alarm triggering
Recorded data playback

7
Gesture recognition,
Facial recognition,
Eye tracking,
Speech recognition,
Doesn’t predict nor interfere with operator’s thoughts
Cannot force directly the operator to work
SYSTEM OVERVIEW:
Blue eye system provides technical means for monitoring and recording the
operator’s basic physiological parameters. The most important parameter is
saccadic activity ( Saccade is a rapid eye jump to a new location assigned by the
conscious attention process), which enables the system to monitor the status of
the operator’s visual attention along with head acceleration, which accompanies
large displacement of the visual axis (saccades larger than 15 degrees). Complex
industrial environment can create a danger of exposing the operator to toxic
substances, which can affect his cardiac, circulatory and pulmonary systems.
Thus, on the grounds of lethysmographic signal taken from the forehead skin
surface, the system computes heart beat rate and blood oxygenation. The blue eye
system checks above parameters against abnormal (e.g. a low level of blood
oxygenation or a high pulse rate) or undesirable (e.g. a longer period of lowered
visual attention) values and triggers user-defined alarms when necessary. Quite
often in an emergency situation operators speak to themselves expressing their
surprise or stating verbally the problem.
Therefore, the operator’s voice, physiological parameters and an overall view of
the operating room are recorded. This helps to reconstruct the course of operators’
work and provides data for long-term analysis. This system consists of a mobile
measuring device and a central analytical system. The mobile device is integrated
with Bluetooth module providing wireless interface between sensors worn by the
operator and the central unit. ID cards assigned to each of the operators and
adequate user profiles on the central unit side provide necessary data
personalization so different people can use a single mobile device.
DESIGN FEATURES:
It has a personal area network for linking all the operators and the supervising
system. It has two major units

8
DAU (data acquisition unit )
CSU (central system unit )
The basic block diagram is shown below:
DATA ACQUISITION UNIT:
The DAU consists of the following components
ATMEL 8952 microcontroller
BLUE TOOTH MODULE – supports synchronous voice data transmission
PCM CODEC -used to transmit operator’s voice and central system sound
feedback
UART -communication between Bluetooth module and microcontroller (115200
bps)
MAX232 -level shifter
ALPHAUNUMERIC LCD display
LED indicators
ID CARD interface
In creating the hardware part of the DAU a development board is built, which
enables the operator to mount, connect and test various peripheral devices
cooperating with the microcontroller. During the implementation of the DAU a
piece of software is needed to establish and test Bluetooth connections. Therefore
a tool called Blue Dentist is created. The tool provides support for controlling the
currently connected Bluetooth device. Its functions are:
local device management (resetting, reading local BD_ADDR, putting in
Inquiry/Page and Inquiry/Page scan modes, reading the list of locally supported
features and setting UART speed)
connection management (receiving and displaying Inquiry scan results,
establishing ACL links, adding SCO connections, performing link authorization
procedure, sending test data packets and disconnecting).
To test the possibilities and

9
performance of the remaining parts such as computer, camera and
database software, BlueCapture is created. The tool supports capturing video data
from various sources (USB web-cam, industrial camera) and
storing the data in the MS SQL Server database. Additionally, the application
performs sound recording. After filtering and removing insignificant fragments
(i.e. silence) the audio data is stored in the database. Finally, the program plays
the recorded audio-visual stream. The software is used to measure database
system performance and to optimize some of the SQL queries (e.g. replacing of
correlated SQL queries with cursor operations).
Also a simple tool for recording Jazz
Multisensory measurements is created. The program reads the data using a
parallel port and writes it to a file. To program the operator’s personal ID card we
use a standard parallel port, as the EPROMs and the port are both TTL-compliant.
A simple dialog-based application helps to accomplish the task.
WHY BLUE EYE TECHNOLOGY
Blue Eyes system provides technical means for monitoring and recording the
operator’s basic physiological parameters. The most important parameter is
saccadic activity1, which enables the system to monitor the status of the
operator’s visual attention along with head acceleration, which accompanies large
displacement of the visual axis (saccades larger than 15 degrees). Complex
industrial environment can create a danger of exposing the operator to toxic
substances, which can affect his cardiac, circulatory and pulmonary systems.
Thus, on the grounds of plethysmographic signal taken from the forehead skin
surface, the system computes heart beat rate and blood oxygenation.
The BlueEyes system checks above parameters against abnormal (e.g. a low level
of blood oxygenation or a high pulse rate) or undesirable (e.g. a longer period of
lowered visual attention) values and triggers user-defined alarms when necessary.
Quite often in an emergency situation operator speak to themselves expressing
their surprise or stating verbally the problem. Therefore, the operator’s voice,
physiological parameters and an overall view of the operating room are recorded.
This helps to reconstruct the course of operators’ work and provides data for long-
term analysis.

10
BlueEyes consists of a mobile measuring device and a central analytical system.
The mobile device is integrated with Bluetooth module providing wireless
interface between sensors worn by the operator and the central unit. ID cards
assigned to each of the operators and adequate user profiles on the central unit
side provide necessary data personalization so different people can use a single
mobile device (called hereafter DAU – Data Acquisition Unit). The overall
system diagram is shown in Figure 1. The tasks of the mobile Data Acquisition
Unit are to maintain Bluetooth connections, to get information from the sensor
and sending it over the wireless connection
EMOTION AND COMPUTING:
Rosalind Picard (1997) describes why emotions are important to the computing
community. There are two aspects of affective computing: giving the computer
the ability to detect emotions and giving the computer the ability to express
emotions. Not only are emotions crucial for rational decision making as Picard
describes, but emotion detection is an important step to an adaptive computer
system. An adaptive, smart computer system has been driving our efforts to detect
a person’s emotional state. An important element of incorporating emotion into
computing is for productivity for a computer user. A study (Dryer & Horowitz,
1997) has shown that people with personalities that are similar or complement
each other collaborate well. Dryer (1999) has also shown that people view their
computer as having a personality. For these reasons, it is important to develop
computers which can work well with its user.
By matching a person’s emotional state and the context of the expressed emotion,
over a period of time the person’s personality is being exhibited. Therefore, by
giving the computer a longitudinal understanding of the emotional state of its
user, the computer could adapt a working style which fits with its user’s
personality. The result of this collaboration could increase productivity for the
user. One way of gaining information from a user non-intrusively is by video.
Cameras have been used to detect a person’s emotional state (Johnson, 1999). We
have explored gaining information through touch. One obvious place to put

11
sensors is on the mouse. Through observing normal computer usage (creating and
editing documents and surfing the web), people spend approximately 1/3 of their
total computer time touching their input device. Because of the incredible amount
of time spent touching an input device, we will explore the possibility of detecting
emotion through touch.
ARTIFICIAL INTELLIGENT SPEECH RECOGNITION:
It is important to consider the environment in which the speech recognition
system has to work. The grammar used by the speaker and accepted by the
system, noise level, noise type, position of the microphone, and speed and manner
of the user’s speech are some factors that may affect the quality of speech
recognition .When you dial the telephone number of a big company, you are
likely to hear the sonorous voice of a cultured lady who responds to your call with
great courtesy saying “Welcome to company X. Please give me the extension
number you want”. You pronounce the extension number, your name, and the
name of person you want to contact. If the called person accepts the call, the
connection is given quickly. This is artificial intelligence where an automatic call-
handling system is used without employing any telephone operator.
THE TECHNOLOGY:
Artificial intelligence (AI) involves two basic ideas. First, it involves studying the
thought processes of human beings. Second, it deals with representing those
processes via machines (like computers, robots, etc). AI is behavior of a machine,
which, if performed by a human being, would be called intelligent. It makes
machines smarter and more useful, and is less expensive than natural intelligence.
Natural language processing (NLP) refers to artificial intelligence methods of
communicating with a computer in a natural language like English. The main
objective of a NLP program is to understand input and initiate action. The input
words are scanned and matched against internally stored known words.
Identification of a key word causes some action to be taken. In this way, one can
communicate with the computer in one’s language. No special commands or
computer language are required. There is no need to enter programs in a special
language for creating software.
SPEECH RECOGNITION:
The user speaks to the computer through a microphone, which, in used; a simple
system may contain a minimum of three filters. The more the number of filters
used, the higher the probability of accurate recognition. Presently, switched
capacitor digital filters are used because these can be custom-built in integrated
circuit form. These are smaller and cheaper than active filters using operational
amplifiers. The filter output is then fed to the ADC to translate the analogue signal

12
into digital word. The ADC samples the filter outputs many times a second. Each
sample represents different amplitude of the signal .Evenly spaced vertical lines
represent the amplitude of the audio filter output at the instant of sampling. Each
value is then converted to a binary number proportional to the amplitude of the
sample. A central processor unit (CPU) controls the input circuits that are fed by
the ADCS. A large RAM (random access memory) stores all the digital values in
a buffer area. This digital information, representing the spoken word, is now
accessed by the CPU to process it further. The normal speech has a frequency
range of 200 Hz to 7 kHz. Recognizing a telephone call is more difficult as it has
bandwidth limitation of 300 Hz to3.3 kHz.
As explained earlier, the spoken words are processed by the filters and ADCs.
The binary representation of each of these words becomes a template or standard,
against which the future words are compared. These templates are stored in the
memory. Once the storing process is completed, the system can go into its active
mode and is capable of identifying spoken words. As each word is spoken, it is
converted into binary equivalent and stored in RAM. The computer then starts
searching and compares the binary input pattern with the templates. t is to be
noted that even if the same speaker talks the same text, there are always slight
variations in amplitude or loudness of the signal, pitch, frequency difference, time
gap, etc. Due to this reason, there is never a perfect match between the template
and binary input word. The pattern matching process therefore uses statistical
techniques and is designed to look for the best fit.
The values of binary input words are subtracted from the corresponding values in
the templates. If both the values are same, the difference is zero and there is
perfect match. If not, the subtraction produces some difference or error. The
smaller the error, the better the match. When the best match occurs, the word is
identified and displayed on the screen or used in some other manner. The search
process takes a considerable amount of time, as the CPU has to make many
comparisons before recognition occurs. This necessitates use of very high-speed
processors. A large RAM is also required as even though a spoken word may last
only a few hundred milliseconds, but the same is translated into many thousands
of digital words. It is important to note that alignment of words and templates are
to be matched correctly in time, before computing the similarity score. This
process, termed as dynamic time warping, recognizes that different speakers
pronounce the same words at different speeds as well as elongate different parts
of the same word. This is important for the speaker-independent recognizers.
THE SIMPLE USER INTERST TRACKER (SUITOR):
Computers would have been much more powerful, had they gained perceptual
and sensory abilities of the living beings on the earth. What needs to be developed

13
is an intimate relationship between the computer and the humans. And the Simple
User Interest Tracker (SUITOR) is a revolutionary approach in this direction.
By observing the Webpage a netizen is browsing, the SUITOR can help by
fetching more information at his desktop. By simply noticing where the user’s
eyes focus on the computer screen, the SUITOR can be more precise in
determining his topic of interest. It can even deliver relevant information to a
handheld device. The success lies in how much the suitor can be intimate to the
user. IBM’s BlueEyes research project began with a simple question, according
to Myron Flickner, a manager in Almaden’s USER group: Can we exploit
nonverbal cues to create more effective user interfaces?
One such cue is gaze-the direction in which a person is looking. Flickner and his
colleagues have created some new techniques for tracking a person’s eyes and
have incorporated this gaze-tracking technology into two prototypes. One, called
SUITOR (Simple User Interest Tracker), fills a scrolling ticker on a computer
screen with information related to the user’s current task. SUITOR knows where
you are looking, what applications you are running, and what Web pages you may
be browsing. “If I’m reading a Web page about IBM, for instance,” says Paul
Maglio, the Almaden cognitive scientist who invented SUITOR, “the system
presents the latest stock price or business news stories that could affect IBM. If I
read the headline off the ticker, it pops up the story in a browser window. If I start
to read the story, it adds related stories to the ticker. That’s the whole idea of an
attentive system-one that attends to what you are doing, typing, reading, so that
it can attend to your information needs.”
PARTS OF A BLUE EYE SYSTEM:
The major parts in the Blue eye system are Data Acquisition Unit and Central
System Unit. The tasks of the mobile Data Acquisition Unit are to maintain
Bluetooth connections, to get information from the sensor and sending it over the
wireless connection, to deliver the alarm messages sent from the Central System
Unit to the operator and handle personalized ID cards. Central System Unit
maintains the other side of the Blue tooth connection, buffers incoming sensor
data, performs on-line data analysis, records the conclusions for further
exploration and provides visualization interface.
THE HARDWARE:
Data Acquisition Unit
Data Acquisition Unit is a mobile part of the Blue eyes system. Its main task is to
fetch the physiological data from the sensor and to send it to the central system
to be processed. To accomplish the task the device must manage wireless

14
Bluetooth connections (connection establishment, authentication and
termination). Personal ID cards and PIN codes provide operator’s authorization.
Figure Showing Jazz-multi Sensor
Communication with the operator is carried on using a simple 5-key keyboard, a
small LCD display and a beeper. When an exceptional situation is detected the
device uses them to notify the operator. Voice data is transferred using a small
headset, interfaced to the DAU with standard mini-jack plugs.
The Data Acquisition unit comprises several hardware modules figure showing
data
acquisition unit
· Atmel 89C52 microcontroller – system core
· Bluetooth module (based on ROK101008)
· HD44780 – small LCD display
· 24C16 – I2C EEPROM (on a removable ID card)
CENTRAL SYSTEM UNIT :
Central System Unit hardware is the second peer of the wireless connection. The
box contains a Bluetooth module (based on ROK101008) and a PCM codec for
voice data transmission. The module is interfaced to a PC using a parallel, serial
and USB cable.
The audio data is accessible through standard mini-jack sockets over view of
central system unit To program operator’s personal ID cards we developed a
simple programming device. The programmer is interfaced to a PC using serial
and PS/2 (power source) ports. Inside, there is Atmel 89C2051 microcontroller,
which handles UART transmission and I2C EEPROM (ID card) programming.
THE SOFTWARE:
Blue Eyes software’s main task is to look after working operators’ physiological
condition. To assure instant reaction on the operators’ condition change the
software performs real time buffering of the incoming data, real-time
physiological data analysis and alarm triggering.
The Blue Eyes software comprises several functional modules System core
facilitates the

15
transfers flow between other system modules (e.g. transfers raw data from the
Connection Manager to data analyzers, processed data from the data analyzers to
GUI controls, other data analyzers, data logger etc.).
The System Core fundamental are single-producer-multi-consumer thread safe
queues. Any number of consumers can register to receive the data supplied by a
producer. Every single consumer can register at any number of producers,
receiving therefore different types of data.
Naturally, every consumer may be a producer for other consumers. This approach
enables high system scalability – new data processing modules (i.e. filters, data
analyzers and loggers) can be easily added by simply registering as a costumer
Connection Manager is responsible for managing the wireless communication
between the mobile Data Acquisition Unit the central system. The Connection
Manager handles:
· communication with the CSU hardware
· searching for new devices in the covered range
· establishing Bluetooth connections
· connection authentication
· incoming data buffering
· sending alerts
Data Analysis module performs the analysis of the raw sensor data in order to
obtain information about the operator’s physiological condition. The separately
running Data Analysis module supervises each of the working operators.
The module consists of a number of smaller analysers extracting different types
of information. Each of the analysers registers at the appropriate Operator
Manager or another analyser as a data consumer and, acting as a producer,
provides the results of the analysis.
Visualization module provides a user interface for the supervisors. It enables
them to watch each of the working operator’s physiological condition along with
a preview of selected video source and related sound stream. All the incoming
alarm messages are instantly signalled to the supervisor. The Visualization
module can be set in an off-line mode, where all the data is fetched from the
database. Watching all the recorded physiological parameters, alarms, video and

16
audio data the supervisor is able to reconstruct the course of the selected
operator’s duty.
The physiological data is presented using a set of custom-built GUI controls:
· a pie-chart used to present a percentage of time the operator was actively
acquiring the visual information· A VU-meter showing the present value of a
parameter time series displaying a history of selected parameters’ value.
Mobile measuring device or Data Acquisition Unit(DAU) of Blue Eyes
technology:
The blue tooth module, which is integrated with the mobile device (DAU),
provides a wireless interface between the Central System Unit (CSU) and the user
or operator having the sensors. PIN codes and ID cards are assigned to the entire
operator’s for authentication purposes. The device uses a five-key keyboard,
beeper and LCD display for the interaction with the operators and if, any
unwanted situation occurs, the machine uses these devices to inform the
operators. The ‘voice’ information from the user is transferred with the help of a
headset, which is interfaced with the Data Acquisition Unit using a mini jack
plug. DAU incorporates various hardware modules like system-core Bluetooth
section, Atmel 89C52 microcontroller, EEPROM, Beeper, LCD display
(HD44780), LED indicator, voltage level monitors and 6 AA batteries.

17
Fig: Data Acquisition Unit (DAU) Components
Central system Unit (CSU) of Blue Eyes technology:
CSU is the next squint of wireless-network connection in the Blue Eyes
technology. The CSU mainly contains codec (PCM Codec commonly used for
voice information transmission) and a wireless blue tooth module. This CSU
section is integrated to a personal computer using USB, parallel and serial cable.
The mini-jack socket is used for audio data accessing. The program containing
the operators personal ID is amalgamating to the personal computer through the
serial and power ports. The microcontroller (Atmel- 89C2051) inside the unit
handles the I2C EEPROM- programming and UART transmission.

18
Fig: Central system Unit (CSU)Components
The Blue Eyes Technology and Its Basic Structure:
The objective of Blue Eyes technology is to design a computational machine
having sensory and perceptual abilities like human beings. Blue Eyes technology
uses most modern cameras, microphones and advanced non-obtrusive sensing
techniques to interact with humans and understand the emotions of human beings.
The machine has the ability to grasp the eye movement of the user, the needs of
the user and also can understand the emotional and physical states of a user in
front of the machine. The process of making a computer having sensing and
emotional capabilities is known as “Affective Computing”.
The steps involved for designing such type of computers are given below.
1. Process of giving sensing capacity.
2. Human Emotion detection or Affect Detection.
3. Respond appropriately and properly.

19
1. Process of giving sensing capacity:
Blue Eyes utilizes many sensor mechanisms, which is equivalent for the ears,
eyes and other sensory organs that human beings used to express emotions and
recognize each other. Blue Eyes uses voice recognition software, cameras and
biometric sensors to understand and respond to the emotional levels of humans.
The voice recognition software can perceive not only what is being spoken but
also the tone how it is said. High resolution cameras are used for tracking the
minute facial expressions, hand gestures and eye movements. Biometric sensors
are used for measuring and analyzing the muscle tension, body temperature,
blood pressure and other physiological gesture correlated with emotions.
2. Detecting human emotions/ Affect Detection:
In Blue Eyes technology, the machines have the ability to identify the minor
variations in the moods of human beings. Say a person may strike the keyboard
hastily or softly depends on his mood like happy or in angry. The Blue Eyes
enables the machines to identify these minor emotional variations of human
beings even by a single touch on the mouse or key board and the machines started
to react with the users according to this emotional levels. This is done with the
guidance of intelligent devices like “Emotion Mouse”. Along with this Emotion
Mouse, Simple User Interest Tracker (SUITOR) and Artificial Intelligent Speech
Recognition are equipped with the Blue Eyes technology to understand the speech
and identify the interest of the peoples at that instance of time.
For implementing the Affective Computing, we need Emotion Sensors.
Types of Emotion Sensors used in Blue Eyes Technology:
• For Hand - Emotion Mouse:
The major aim of Brain Computer Interface (BCI) is to develop a smart and
adaptive computer system. These types of project must include speech
recognition, eye tracking, facial recognition, gesture recognition etc. software and
hardware. Similarly in Blue Eyes technologies, we need to build a system have
the ability to identify all these perceptual abilities of human beings. In Blue Eyes,
the machines have the ability to identify the minor variations in the moods of
human beings. Say a person may strike the keyboard hastily or softly depends on
his mood like happy or in angry. The Blue Eyes technology enables the machines
to identify these minor emotional variations of human beings even by a single

20
touch on the mouse or key board and the machines started to react with the users
according to this emotional level. This is done with the guidance of intelligent
devices like “Emotion Mouse”. Actually this Emotion Mouse is an input device
to track the emotions of a user by a simple touch on it. The Emotion Mouse is
designed to evaluate and identify the user’s emotions such as fear, surprise, anger,
sadness, happiness, disgust etc. when he/she is interacting with computer. The
main objective of the Emotion Mouse is to gather the user’s physical and
physiological information by a simple touch.
• For Eye - Expression Glass:
Expression Glass is an alternative for the usually available machine vision face
or eye recognition methods. By analyzing pattern recognition methods and facial
muscle variations, the glass senses and identifies the expressions such as interest
or confusion of the user. The prototype used for this glass uses piezoelectric
sensors.

21
MAGIC Pointing:
The Eye gaze tracking methods explores a new way for handling ‘eye gaze’ for
man machine interfacing. The gaze tracking has been deliberated as an excellent
pointing method for giving input to computers. But many drawbacks exist with
this traditional eye gaze tracking methods. To overcome these difficulties an
alternative approach – termed as MAGIC - Manual and Gaze Input Cascaded –
is projected. In this approach, eye gaze pointing appears to the user as a manual
job, utilized for fine selection and manipulation processes. Even so, a large
amount of the cursor movement is removed by bending the cursor to the eye gaze
portion, which surrounds the target. The selection and pointing of the curser is
primarily controlled by manual means but also guided by a gaze tracking
mechanism and is commonly known as MAGIC Pointing. The main aim of
MAGIC pointing is to use ‘gaze’ to warp the previous position (home) of the
curser to the locality of the target, reasonably where the user was looking at, so
as to reduce the cursor motion amplitude required for target selection. When the
cursor position is identified, only a small movement is needed by the user to click
on the target by a manual input device that is to accomplish Manual Acquisition
with Gaze Initiated Cursor or Manual and Gaze Input Cascaded (MAGIC)
pointing. Two MAGIC Pointing methods – conservative and liberal –in terms of
cursor placement and target identification, were outlined, analyzed and executed
with an eye tracker unit.

22
Fig: The conservative MAGIC pointing Technique
Advantages of MAGIC Pointing Technique:
• Reduced physical effort compared with the traditional manual pointing
techniques.
• Greater spontaneity than traditional manual pointing.
• Greater accuracy.
• Faster speed of operation than manual pointing.

23
Drawbacks of MAGIC Pointing Technique:
The unconscious jittery movement that an eye continually makes and also the one
degree size of the fovea, eye gaze method is not accurately enough to perform UI
widgets such as slider handles, scrollbars and hyperlinks on the Graphic User
Interfaces. Sometimes the movement of the eye is spontaneously controlled while
sometimes it is uncontrollable. In MAGIC pointing many methods such as eye
blinking and continuous looking etc. are used for target selection procedures. But
sometime it is not working properly because if a user does not look at a particular
target continuously for a predetermined threshold value, the target will not be
selected. Thus there are more chances for false selections.
The Simple User Interest Tracker (SUITOR):
The Simple User Interest Tracker is revolutionary approaches towards the design
of machine having the ability to maintain an intimate relationship between the
humans and the computers. The SUTOR continuously analyzes the user that
where his eye focus on the personal computer screen. The SUITOR has the ability
to determining the topic of interest of the user and also according to this it can
able to deliver the appropriate data to a handheld device.
Artificial Intelligent Speech Recognition used in Blue Eyes Technology:
For implementing the Artificial Intelligent Speech Recognition system in Blue
Eyes technology, the working environment should be very important. The manner

24
of the user’s speech, grammar, noise type, noise level and the position of the
microphone are some important factors that may influence the features of speech
recognition system. In Artificial Intelligent Speech Recognition system, an
automatic call handling method is implemented without any telephone operator.
Two basic ideas are included in the Artificial intelligence (AI),
• Study the thought of human beings.
• Represents the thought process of human beings through robots, computers
etc.
Actually Artificial intelligence (AI) denotes the behavior of a computer or any
machines but it is carried out by the humans is called as ‘intelligent’. This AI
makes machines more power full, useful, and smarter and also it is less expensive
compared to natural intelligence. Natural language processing (NLP) makes
artificial intelligence systems to communicate English. The main goal of the
Natural language processing (NLP) is to understand the users input and react
according to these inputs. The input data or words are continuously scanned and
finds matches against inside stored known data or words. And after identifying
the key words, the corresponding actions are carried out by the machine. In this
way the Blue Eyes technology enables the users to communicate with the
machines with their own languages.
EMOTION SENSORY WORLD
Human emotion is a visible proof of effective state, personality, emotional state
and cognitive activity. There has been a lot of work done on blue eyes technology.
This paper presents number of techniques proposed to identify emotional state of
a person. According to Ekman, the neuro-part of the theory mention the partly
innate and biological program, called a facial affect program, which specifies the
relationships between various movements of the facial muscles and particular
emotions (happiness, sadness, anger, surprise). According to Ekman findings
during:
Happiness :- the eyes are relaxed ;
Anger :-The forehead are pulled down and inward; no sclera is shown in the eyes;
Sadness :- The brows are drawn together with the inner corners raised and the
outer corners lowered ; the eyes are glazed;

25
Surprise :-The eyebrows are raised and curved.
In this paper a new technique Emotion Sensory World of blue eyes technology
have been deals with the detection of emotions of human through the texture of
eye because eyes are window to the soul that they can tell much about person
internal state just by gazing into them, a camera will capture the image of a person
and focuses on the eye area by using texture filtering algorithm which is then
compared with the list of images place in data base .The correct image that
identifies the emotion of a person is shown on the window, after detecting the
emotion a song is played in order to normalize the mood of person.
METHODOLOGY
The methodology of Blue Eyes Technology is as follows:
Step 1: Get Snapshot
A video stream will start and when a person set focus on face and press "Enter"
then it will take a snapshot. Immediately returns one single image frame, from
the video input object . The frame of data returned is independent of the video
input object Frames Per Trigger property and has no impact on the value of the
Frames Available or Frames Acquired property. The object must be a one-byone
video input object. Frame is returned as an H-by-W-by-B matrix where H- Image
height, as specified in the ROI Position property W- Image width, as specified in
the ROI Position property B- Number of bands associated with obj, as specified
in the Number of Bands.
Step 2: Extract Eye Portion 1.
Detection of Face Parts:
(a) Input parameters: Detector: The detection object built by build Detector.
Thick (optional): Thickness of bounding box.
(b) Output parameters: It creates bounding box for face, eye, left eye, right eye,
mouth and nose, image with found face and these faces are stored as cell array
buildDetector build face parts detector object with threshold values for parts.

26
2. Shape Recognition and Edge Detection:
(a) After getting the eye part we match it with the existing images by classifying
it according to structure of eye and it texture we call it Shapes Classifier.
(b) Separates the eye part only from the box boundaries of face.
(c) Convert image from rgb to gray.
(d) Threshold the image Convert the image to colorless, in order to prepare for
boundary tracing using bw boundaries.
(e) Invert the Binary Image.
(f) Find the boundaries Concentrate only on the outside boundaries. Option
'noholes' will accelerate the processing by preventing bw boundaries from
searching for inner contours.
(g) Determine Shapes properties.
(h) Classify Shapes according to properties Wrinkles, flat, swelled, etc.
Step 3: Comparison with stored Images in data base
The shape classifier will then match the captured image with the data entries in
our database which in then converted to gray scale; the idea is to create a function
which will return the distinctness in range [0, 1] between two postures.This
means, we want to compare only a posture and on this basis the emotion of person
for given two images (a grey region). For example, if we pass 4 to my function,
the result will be 0 (because postures or emotions are not same and the result will
be 1 if same).
Step 4: Play song according to matched mood
The generated script take a list of sound files and create a database of these sounds
according to the emotion detection defined in database for each song, and then
subsequently take one or more audio files according to matched emotion of
previously created image database and plays it, List of sound files is analyzed and
written to a single database file. Various sound file formats are supported,
including wav, mp3 and aac.our database files are encoded with.wav extension.

27
The sound file in database can then be saved as a wav file using the WAVWRITE
function and later can be loaded using the WAVREAD function. The played
sound returns the sample rate (Fs) in Hertz and the number of bits per sample (n
bits) used to encode the data in the file.
FUTURE APPLICATIONS OF BLUE EYE TECHNOLOGY
Blue Eyes technology is an innovative approach and can be used in schools and
colleges or can also be used in the field of education to take sessions, in order to
check and observe students physical state and psychological state. Future
applications of Blue Eyes technology will be way ahead of other technologies.
The fields of advertisement and entertainment will be mostly benefited with this
as they have an important role in our day to day life.
Entertainment
With the help of blue eyes technology a person can rule and control the world of
entertainment through the eyes. One of the examples is video games. There are
many different techniques used for the implementation which helps us to move
our eyes from one side of the screen to another and then the game character will
look and make the same movements the same way as we suggest to them in result
to capture the eye moments of the user.
Techniques Used
The world of Entertainment uses eye tracking technology through Manual and
Gaze Input Cascaded Pointing which makes movements. The MAGIC pointing
is far better and advanced then the old traditional system. The screen
automatically captures around whatever user is looking at also, it gives a good
control over the game, and you overall start feeling so inside the game that you
will never want to use something else for video games. So, this will be a very best
option for the game players.
Outcome
By using such techniques user can achieve a lot more in the world of
entertainment. The outcome involves that all the future applications will be using
blue eyes technology in order to make our day to day life simpler and easier. In
the world of entertainment it would be much more fun to play the games with the
use of such technology which only need your emotions to play
Advertisement

28
In advertisement, it is not very far enough when blue eyes technology will make
the ability to use eye tracking as a powerful technology in our near future which
can even check the consumer’s attention as well as responses to marketing
messages. Basically, to know what people actually look at and which can help
advertisers to know that the ad presented is seen by the consumer or not. In this
way it is beneficial in the Advertisement world where the things will be easier
and simpler.
Techniques Used
Eye trackers uses eye gaze tool to identify what input is coming by knowing the
input it uses a camera that is small in size that gives infrared light off a user’s
eyes and the reflections of the light are followed in order to pinpoint where their
eyes are looking. Manual and Gaze Input Cascaded Pointing collects unique
visual information of user’s behavior. They also can tell whether user is reading
the ad or whether a user is searching for something else in the page. The eye gaze
usually maps user’s visualizations and tracks them accordingly.
Outcome
In the world of advertisement, blue eyes technology will play an important with
its techniques and devices. The field of advertisement will be benefitted by using
this technology as it can sense the consumer’s attention towards the advertisement
as well as user can know for what consumer is more interested in and what he is
looking for in the advertisement.
Besides this, a user can also operate the switch button for multiple devices such
as for TV, fan, and music playing system, and etc. this can be done by calling
them with different instructions which can be voice, audio or actions in order to
on or off simply by looking at them.
Security and control systems
can be controlled with the help of this technology. Human supervision is always
required at such areas. Human beings can get tired easily and distracted at times
and this can cause a huge devastation in such systems. So if blue eyes are
implemented in this area. The surveillance cameras can capture more
than just recording the video of particular area. They can recognize the
emotional state of the user and help security to focus on the persons whose
emotional conditions are out of normal and seemingly dangerous. Also
these cameras can provide data to the shopkeepers to track their
customers, giving them idea which section attracts people, which
advertisement displayed attracted more people, which made people feel

29
boredom etc. It can also used in the other control systems in banks, airplanes
etc. The security concerns of getting a wrong person with no harmful
intentions in the security systems should be avoided while still a proper alarming
system is in place with the help of it. They can prove to be better alarm systems.
Assisting Human Operators
by monitoring and recording the operators physical condition .In the
complex industries like chemical industry or nuclear sector where exists danger
in the environment of exposure to toxic substances or radiations,
monitoring helps the human operators. The operator has to raise the
alarm by himself by announcing the danger he is in but with the help of
these systems, the technology keeps an eye on the operator for normal
conditions. If any parameter is found in abnormal conditions like a raise in pulse
rate or low level of oxygenation then an alarm can be triggered
automatically.
Driving systems
can be controlled by blue eye technology and this can help in
reduction of the number of accidents happening in the world. When a sensor is
attached to the steering wheel, it can assess the emotional stability of the
driver and guide him in traffic conditions .For an example, when a driver is
very angry, not emotionally stable, stressed and the driver is increasing his speed
of the vehicle, then this technology can detect it and reduce the speed, or use auto-
drive mode stating the emotional instability of the driver and help him drive back

30
safely. Proper exceptions should be implemented for situations like emergency
or in case of ambulance to not make the technology look like a hindrance
to the driver. Also they can be used to alert the users who are drowsy or
sleepy during driving.
Medical supervision
Is possible with this technology. It can help the doctor in concentrating more on
his work than getting distracted for putting down all the observations he
is making. He can go through data of many patients and then just tell
the computer to make note of the observations he is making through voice
instructions. This helps especially in case of x-rays, scans etc. Monitoring of
patients can be done by detecting the physiological conditions of him/her like
blood pressure, pulse rate, oxygenation. The position of the person can be
determined like whether the person is sitting or standing or lying down.
Access to data
can be made by just giving a simple command to the computer rather
than executing a physical search for data. Stating the urgency of the situation,
the computer can be made to prioritize the tasks to be done. Like in a
situation where the user have to make an urgent call to some other
location which is a known location for both the sides, the technology
can assess the priorities of the call on both the sides and could make the
call in a faster pace. Other use might be where we can access net by just
seeing the computer on the link we would like to open in a search, where
the search can be initiated by voice command. With more ease to access
the data the risk of intrusion in the systems increase. Proper biometric
authentication systems should be in place to protect the sensitive
data. Acquiring data through Bluetooth. The data is available to anyone
now in wireless mode and can be collected also in easy manner.
In future, this can be implemented in House hold devices. The technology can
make everyday life more comfortable. We can switch on or off Television, music
player, a/c, fan, washing machine, oven, coffee machine etc by just having one
look at them or by instructing them through voice commands. Imagine the
computer showing you the list of events for the day and displaying happy
pictures and mails to you every morning to cheer you up.
TYPES OF USERS
Users belong to three categories:

31
• Operators
• Supervisors
• System administrators
Operator:
•Operator is a person whose physiological parameters are supervised.
•The operator wears the DAU.
•The only functions offers to the operator are Authorization to the system and
receiving alarm alerts.
•Authorization: Operator has to enter his personal PIN into DAU, if PIN is
accepted, authorization is said to be complete.
•Receiving Alerts: This function supplies the operator with the most important
alerts about his and his co-workers’ condition and mobile device state.
Supervisor:
•He is the person responsible for analysing operators’ condition and performance.
•The supervisor receives tools for inspecting present values of the parameters
(on-line browsing)as well as browsing the results of the long-term analysis (off-
line browsing).
System Administrator:
•He is the user that maintains the system.
•The administrator is delivered tools for adding new operators to the database.
Defining alarm conditions.
•Configuring logging tools.
•Creating new analyser modules.
ADVANTAGES OF BLUE EYE TECHNOLOGY
• Biometric data cannot be falsified (although it can be misread)
• Biometric data cannot be changed.
• Biometrics can be more efficient with certain applications (i.e. statistical
surveys)
• Prevention from dangerous incidents
• Minimization of ecological consequences financial loss a threat to a
human life Blue Eyes system provides technical means for monitoring and
recording.
Human-operator’s physiological condition. The key features of the system are:

32
• visual attention monitoring (eye motility analysis)
• physiological condition monitoring (pulse rate, blood oxygenation)
• operator's position detection (standing, lying)
• wireless data acquisition using Bluetooth technology
• real-time user-defined alarm triggering
• physiological data, operator's voice and overall view of the control
• room recording
• recorded data playback
•
DISADVANTAGES:
All present biometric systems have FAR (false-acceptance-rate) and a FRR
(false-recognition-rate). In basic security application both these rates have been
much studied and error has been reduced, but not eliminated. High FAR biometric
security systems will generally have a higher FRR. When the object being
scanned is stationary or near stationary and can scanned multiple times this is
more of nuisance, but when the object being scanned is moving the system loses
effectiveness. But this is not a priority when scanning is done intentionally as
with employee entry or clock-in applications.
In biometric systems which collecting information on consumers, a low
FFR is essential. When the system measures certain quantities to model consumer
behaviour it does so generally. Unlike with a security entry system, varying input
can equal the same output. A Single negative in a binary sequence representing a
set of Input quantities would never cause a positive output. Thus FAR is less of a
concern. When is necessary is for more data to be collected so corporate surveys
of consumer behaviour can be accurate. This is why statistical applications of
biometrics are engineered for low FFR. (This is based on a biometric statistical
survey where data is only collected if one or more of a set of outputs is positive.)
Systems that identify objects in motion fall in between. These systems need
a balance of FAR and FFR. Taking negative input as positive has to be minimized
because they look for an exact match with data already in the system. Equally,
taking positive input as negative also has to be minimized because the time to
scan is limited. These systems are more difficult to optimize because they depend
on finding the equilibrium between FAR and FRR.
TOOLS USED
During the implementation of the
DAU there was a need of a piece
of software to establish and
testBluetooth connections. T
herefore, a tool had been cre
ated called Blue Dentist.
The tool provides support for
controlling the currently
connected Bluetooth device. Its
functions are: local device

33
management and connection
management. To test the
possibilities and performance of
the remaining parts of the
Project Kit (computer, camera
and database software) Blue Capture
had been created. The tool supports
capturing video data from various
sources (USB web-cam,
industrial camera) and storing
the data in the MS SQL
Server database. Additionally
, the application performs so
und recording. After filterin
g and removinginsignificant
fragments (i.e. silence) the audio data
is stored in the database. Finally, the
program plays the recorded audio-
visual stream. The software was used
to measure database system
performance and to optimize some
of the SQL queries. Since all the
components of the application
have been tested thoroughly they
were reused in the final software,
which additionally reduced testing
time. A simple tool was
created for recording Jazz Multi
sensor measurements. The
program reads the data using a
parallel port and writes it to a file.
ARTIFICIAL INTELLIGENCE SPEECH RECOGNITION
It is important to consider the environment in which the speech recognition
system has to work. The grammar used by the speaker and accepted by the
system, noise level, noise type, position of the microphone, and speed and manner
of the user’s speech are some factors that may affect the quality of speech
recognition .When you dial the telephone number of a big company, you are
likely to hear the sonorous voice of a cultured lady who responds to your call with
great courtesy saying “Welcome to company X. Please give me the extension
number you want”. You pronounce the extension number, your name, and the
name of person you want to contact. If the called person accepts the call, the

34
connection is given quickly. This is artificial intelligence where an automatic call-
handling system is used without employing any telephone operator.
THE TECHNOLOGY
Artificial intelligence (AI) involves two basic ideas. First, it involves studying the
thought processes of human beings. Second, it deals with representing those
processes via machines (like computers, robots, etc). AI is behavior of a machine,
which, if performed by a human being, would be called intelligent. It makes
machines smarter and more useful, and is less expensive than natural intelligence.
Natural language processing (NLP) refers to artificial intelligence methods of
communicating with a computer in a natural language like English. The main
objective of a NLP program is to understand input and initiate action. The input
words are scanned and matched against internally stored known words.
Identification of a key word causes some action to be taken. In this way, one can
communicate with the computer in one’s language. No special commands or
computer language are required. There is no need to enter programs in a special
language for creating software.
The blue eyes technology depends upon the interaction between a device and the
human being, as we exchange the information with everyone and interact with
everyone likewise this applied science can interact with the human beings in a
similar manner. An example for explaining about the blue eyes technology is as
follows, at the time of your going to bed your mobile device automatically says
you the good night and we have such amazing features in the blue eyes
technology.
The general and basic idea of this technology is very simple and it is that the
human beings are going to make a computer that has the power of sensitivity and
the sensitivity is the power of human beings too. In real the blue eyes technology
works on the sensors, Bluetooth, and the central unit; the total system of the blue
eyes applied science consists of two main parts and they are as follows:
▪ Data acquisition unit (DAU)
▪ Central system unit (CSU)
Types of Emotional Sensors:
The emotional sensors are of three type sand they are as follows:
▪ Emotional sensors for hands
▪ Emotional sensors for eyes
▪ Emotional sensors for voice
1. For hands: The devices used as emotional sensors for the hands are as
follows:
▪ Emotion mouse
▪ Sentic mouse

35
1. Emotion mouse: The emotion mouse is implemented on the genuine
mouse and it includes a set of sensors along with the infrared detectors and
temperature sensitive chips. The data obtained from emotion mouse is
about the behavior and physiological information, the behavior includes
the actions like a movement of the mouse, frequency of clicking the button
and the pressure of fingers when a person clicks the button. The
physiological information includes the heart rate, a temperature of the skin,
skin electricity electromyography activity.
2. Sentic mouse: The genetic mouse is an experiment which is inspired by
the work of Peter J.Lang, Dr. Manfred Clynes and Ward Winton, Lois
Putnam and Robert Kraus.
2. For eyes: The devices used as emotional sensors for the hands are as
follows:
▪ The expression glasses
▪ The magic pointing
▪ The eye tracking
The expression glasses are a wearable machine that permits the user to see a
graphical view of the wearer’s facial expression.
3. For voice: The voice is also used as an emotional sensor in the blue eyes
technology.
SPEECH RECOGNITION
The user speaks to the computer through a microphone, which, in used; a simple
system may contain a minimum of three filters. The more the number of filters
used, the higher the probability of accurate recognition. Presently, switched
capacitor digital filters are used because these can be custom-built in integrated
circuit form. These are smaller and cheaper than active filters using operational
amplifiers. The filter output is then fed to the ADC to translate the analogue signal
into digital word. The ADC samples the filter outputs many times a second. Each
sample represents different amplitude of the signal .Evenly spaced vertical lines
represent the amplitude of the audio filter output at the instant of sampling. Each
value is then converted to a binary number proportional to the amplitude of the
sample. A central processor unit (CPU) controls the input circuits that are fed by
the ADCS. A large RAM (random access memory) stores all the digital values in
a buffer area. This digital information, representing the spoken word, is now
accessed by the CPU to process it further. The normal speech has a frequency
range of 200 Hz to 7 kHz. Recognizing a telephone call is more difficult as it has
bandwidth limitation of 300 Hz to3.3 kHz.
As explained earlier, the spoken words are processed by the filters and
ADCs. The binary representation of each of these words becomes a template or
standard, against which the future words are compared. These templates are

36
stored in the memory. Once the storing process is completed, the system can go
into its active mode and is capable of identifying spoken words. As each word is
spoken, it is converted into binary equivalent and stored in RAM. The computer
then starts searching and compares the binary input pattern with the templates. t
is to be noted that even if the same speaker talks the same text, there are always
slight variations in amplitude or loudness of the signal, pitch, frequency
difference, time gap, etc. Due to this reason, there is never a perfect match
between the template and binary input word. The pattern matching process
therefore uses statistical techniques and is designed to look for the best fit.
The values of binary input words are subtracted from the corresponding
values in the templates. If both the values are same, the difference is zero and
there is perfect match. If not, the subtraction produces some difference or error.
The smaller the error, the better the match. When the best match occurs, the word
is identified and displayed on the screen or used in some other manner. The search
process takes a considerable amount of time, as the CPU has to make many
comparisons before recognition occurs. This necessitates use of very high-speed
processors. A large RAM is also required as even though a spoken word may last
only a few hundred milliseconds, but the same is translated into many thousands
of digital words. It is important to note that alignment of words and templates are
to be matched correctly in time, before computing the similarity score. This
process, termed as dynamic time warping, recognizes that different speakers
pronounce the same words at different speeds as well as elongate different parts
of the same word. This is important for the speaker-independent recognizers.
EXPERIMENT
Empirical studies are relatively rare in eye tracking-based interaction research,
although they are particularly needed in this field. Human behavior and processes
at the perceptual motor level often do not conform to conscious-level reasoning.
One usually cannot correctly describe how to make a turn on a bicycle.
Hypotheses on novel interaction techniques can only be validated by empirical
data.
However, it is also particularly difficult to conduct empirical research on gaze-
based interaction techniques, due to the complexity of eye movement and the lack
of reliability in eye tracking equipment. Satisfactory results only come when
"everything is going right." When results are not as expected, It is difficult to find
the true reason among many possible reasons: Is it because a subject's particular
eye property fooled the eye tracker Was there a calibration error Or random noise
in the imaging system Or is the hypothesis in fact invalid We are still at a very
early stage of exploring the MAGIC pointing techniques. More refined or even
very different techniques may be designed in the future. We are by no means

37
ready to conduct the definitive empirical studies on MAGIC pointing. However,
we also feel that it is important to subject our work to empirical evaluations early
so that quantitative observations can be made and fed back to the iterative design-
evaluation-design cycle. We therefore decided to conduct a small-scale pilot
study to take an initial peek at the use of MAGIC pointing, however unrefined.
EXPERIMENTAL DESIGN
The two MAGIC pointing techniques described earlier were put to test using a
set of parameters such as the filter's temporal and spatial thresholds, the minimum
cursor warping distance, and the amount of "intelligent bias" (subjectively
selected by the authors without extensive user testing). Ultimately the MAGIC
pointing techniques should be evaluated with an array of manual input devices,
against both pure manual and pure gaze-operated pointing methods. Since this is
an early pilot study, we decided to limit ourselves to one manual input device. A
standard mouse was first considered to be the manual input device in the
experiment. However, it was soon realized not to be the most suitable device for
MAGIC pointing, especially when a user decides to use the push-upwards
strategy with the intelligent offset. Because in such a case the user always moves
in one direction, the mouse tends to be moved off the pad, forcing the user adjust
the mouse position, often during a pointing trial. We hence decided to use a
miniature isometric pointing stick (IBM Track Point IV commercially used in the
IBM ThinkPad 600 and 770 series notebook computers). Another device suitable
for MAGIC pointing is a touchpad: the user can choose one convenient gesture
and to take advantage of the intelligent offset. The experimental task was
essentially a Fits' pointing task. Subjects were asked to point and click at targets
appearing in random order. If the subject clicked off-target, a miss was logged
but the trial continued until a target was clicked. An extra trial was added to make
up for the missed trial. Only trials with no misses were collected for time
performance analyses. Subjects were asked to complete the task as quickly as
possible and as accurately as possible. To serve as a motivator, a $20 cash prize
was set for the subject with the shortest mean session completion time with any
technique.
The task was presented on a 20 inch CRT color monitor, with a 15 by 11
inch viewable area set at resolution of 1280 by 1024 pixels. Subjects sat from the
screen at a distance of 25 inches. The following factors were manipulated in the
experiments: two target sizes: 20 pixels (0.23 in or 0.53 degree of viewing angle
at 25 in distance) and 60 pixels in diameter (0.7 in, 1.61degree)three target
distances: 200 pixels (2.34 in, 5.37 degree), 500 pixels (5.85 in, 13.37 degree),
and 800 pixels (9.38 in, 21.24degree)three pointing directions: horizontal, vertical
and diagonal. A withinsubject design was used. Each subject performed the task
with all three techniques:

38
(1) Standard, pure manual pointing with no gaze tracking (No Gaze);
(2) The conservative MAGIC pointing method with intelligent offset
(Gaze);
(3) The liberal MAGIC pointing method (Gaze2)
CONCLUSION
The nineties witnessed quantum leaps interface designing for improved man
machine interactions. The BLUE EYES technology ensures a convenient way of
simplifying the life by providing more delicate and user-friendly facilities in
computing devices. Now that we have proven the method, the next step is to
improve the hardware. Instead of using cumbersome modules to gather
information about the user, it will be better to use smaller and less intrusive units.
The day is not far when this technology will push its way into your house hold,
making you more lazy. It may even reach your hand held mobile device. Any way
this is only a technological forecast.
The report present two results of emotional sensory world. First,
observation reveals the fact that different eye colors and their results in change in
emotions. It changes without giving any information on shape and actual detected
emotion. It is used to favourably recognize four different emotions of eyes. This
developed methodology can be widespread to other activities. Second result were
achieved for converging in good emotions using a mixture of features, shapes,
colors based on eye points. After this favourable capturing of eye spots, it will
help to tell about the state of a person and also helps to cheer up by playing songs
or other sources. The motive of this research proves to be a source of economic
growth over all.
FUTURE WORK
This study work can be extended to home appliances where it can perform
various tasks within home premises through blue eye technology. Further as a
world is digitizing and we are moving towards robotic world, several human
activities can be shrunk with emotion sensory world tool. The tool or system is
fitted in robot with eye emotions which detects what is the demand and the action
can be taken by robot accordingly.

IBM's Blue Eyes Technology Gives Computers Human Perceptual Abilities

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to IBM's Blue Eyes Technology Gives Computers Human Perceptual Abilities

Similar to IBM's Blue Eyes Technology Gives Computers Human Perceptual Abilities (20)

More from komal jain

More from komal jain (6)

Recently uploaded

Recently uploaded (20)

IBM's Blue Eyes Technology Gives Computers Human Perceptual Abilities