It is based on emotion computing, FACS,eye tracking.

1. By :
Priyanka Datta(Roll No.-23)
Nilanjana Sarkar(Roll No.- 18)
Stream-CSE (A)
Batch:2013-2017
The EmotionSensory Worldidentifying human emotions defining new cognitive approaches to
the vast technologicalworld.

2. Contents
• INTRODUCTION
• REASEARCH & INVENTION
• SYSTEM OVERVIEWS
• EMOTION COMPUTING
• RESULTS OF EMOTION COMPUTING
• EMOTION SENSORS
• APPLICATIONS
• THE IBM ALMADEN EYE TRACKER
• MEASURED & GAZE INPUT CASCADED MAGIG
POINTING
• IMPLEMENTATION of MAGIC POINTING
• ARTIFICIAL INTELLIGENT SPEECH RECOGNITION
• Conclusion
• References

3. Introduction
Human cognition depends primarily on the ability to perceive, interpret, and integrate
audio-visuals and sensoring information. Adding extraordinary perceptual abilities to
computers would enable computers to work together with human beings as intimate
partners. Researchers are attempting to add more capabilities to computers that will
allow them to interact like humans, recognize human presence, talk, listen, or even
guess their feelings.
The BLUE EYES technology aims at creating computational machines that have
perceptual and sensory ability like those of human beings. It uses sensing method,
employing most modern video cameras and microphones to identify the user’s
actions through the use of imparted sensory abilities. The machine can understand
what a user wants, where he is looking at, and even realize his physical or emotional
states.

4. Research & Invention
(Historical Background)
Paul Ekman’s facial expression work gave the correlation
between a person’s emotional state & psychological state
which described the Facial Action Coding System
(Ekman and Rosenberg,1997).
Blue Eyes is a technology conducted by the Research
Team of IBM at its Almaden Research Institute(ARC)
,Sanjose , California since 1997.
Blue Eyes Technology aims at creating a computer that
have the abilities to understand the perceptual powers by
recognizing their facial expressions and react accordingly to
them.
One of his experiments involved participants attached
to devices to record certain measurements including
pulse, galvanic skin response (GSR), temperature,

5. What is FACS?
FACS-Facial Action Coding System.
Paul Ekman and W.V Friesen developed the original FACS in 1970s.
It is a guide to categorize the facial expressions.
It is based on the muscular contractions which produce our facial expressions.

6. THE TERM: Blue Eyes
(Why this technology is referred to as Blue Eyes?)
Blue in terms of Bluetooth which enable reliable wireless
communication .
Eyes because the eye movement enable us to obtain a lot of
interesting and important information.
People spend approximately 1/3 of their total computer time
touching input device. Physiological data is obtained and
emotional state is determined during the process.

7. System Overview
Blue Eyes system consists of a mobile measuring device and a central
analytical system.
The mobile device is integrated with Bluetooth module providing wireless
interface between sensors worn by the operator and the central unit.
ID cards assigned to each of the operators and adequate user profiles on
the central unit side provide necessary data personalization's so the
system consists of:
Mobile measuring device (DAU)
Central System Unit(CSU)
Data Acquisition Unit is a mobile
part of the Blue eyes system. Its main
task is to fetch the physiological data from the
sensor and to send it to the central system to
be processed. To accomplish the task the device must manage wireless
Bluetooth connections (connection establishment, authentication and
termination).
Central System Unit hardware is the second peer of the wireless
connection. The module is interfaced to a PC using parallel, serial and

8. Emotion Computing
Emotion Computing also known as Affective Computing is the study and
development of systems and devices that can recognize, interpret, process and
simulate human affects.
It is an in disciplinary field spanning computer science, psychology and
cognitive science.
Rosalind Picard ‘s 1995 paper gave birth to a new technology “Affective
Computing” to computer science field .
Emotion Computing aims to bridge the gap between emotions and
computational technology.
There are two aspects of emotion computing: giving the computer the ability
to detect emotions and to express emotions.
 Emotion detection is an important step to an adaptive computer system.
Some of the important algorithms which allows a computer system to identify
quick and accurate emotion are: Linear Discriminant Classifiers (LDC),k-nearest
neighbour(k-NN), Gaussian mixture model (GMM),etc.

9. TYPES OF EMOTION SENSORS
Emotion Sensors can be for hand, eyes and voice.
•For Hand:
•Emotion Mouse
•Sentic Mouse
•For Eyes:
• Expression Glasses
•Magic Pointing
•Eye Tracking
•For Voice:
•Artificial intelligence speech recognition
Emotion Mouse:
• One method for gaining user information through touch is via a computer input
device, the mouse.
• This allows the user to relate the cardiac rhythm, the body temperature, electrical
conductivity of the skin and other physiological attributes with the mood. This has led
to the creation of the “Emotion Mouse”.
Sentic Mouse:It is a modified computer mouse that includes a directional pressure sensor
for aiding in recognition of emotional valence.
continued to next slide......

10. TYPE OF EMOTION SENSORS
 Expression Glasses :A wearable device which allows any viewer to
visualize the confusion and interest levels of the wearer.
The Sentic Mouse is an experiment inspired
by the work of Peter J. Lang, Ward Winton,
Lois Putnam, Robert Kraus and Dr. Manfred
Clynes.

11. IBM Almaden Eye Tracker
Eye tracker was first developed at IBM Almaden research institute.
Working of Eye Tracker(in brief) :
Its first task is to detect the pupil and it is
done using infra-red rays.
Once the pupil has been detected, the corneal
reflection (the glint reflected from the surface of
the cornea due to one of the light sources) is
determined from the dark pupil image.
 The reflection is then used to estimate the user's point of gaze in terms of the screen
coordinates where the user is looking at.
The following picture on the next slide would make this above explanation more vivid.
continued to next slide..

13. Applications
 Speech recognition system ->it allows the users to do multiple
works simultaneously.
Radiologist scanning hundreds of X-rays, ultra sonograms, CT scans
and simultaneously dictating conclusions to a speech recognition
system.
Visual attention monitoring (eye motility analysis)
 Physiological condition monitoring (pulse rate, blood
oxygenation)9. Operator's position detection (standing, lying).
--------------------------------------------------------------------------------------------

14. MAGIC POINTING(Introduction)
 Reduce the cursor movement needed for
target selection.
Based on Eye tracking.
This work explores a new direction in
utilizing eye gaze for computer input.
 Click on the target with a regular eye
tracker.

15. MAGIC POINTING(Types)
LIBERAL CONSERVATIVE
To wrap the cursor
to every new object
user looks at.
 Does not wrap
the cursor target
until the manual
input device has
been actuated.

16. IMPLEMENTATION OF MAGIC
POINTING
 MANUAL INPUT DEVICE ( such as mouse ).
 EYE TRACKING SYSTEM (connected by
serial port).

17. ARTIFICIAL INTELLIGENT SPEECH
RECOGNITION
 Input words are scanned and matched with
the internally stored words.
 Speech recognition is done using
microphone.
 Filtered and fed to ADC and stored in RAM.

18. APPLICATIONS OF AISR
 Can be used in military operations ( e.g.
Voice control).
 Pilots can give command and information
to the computers by speaking into
microphones.
 Voice recognition system can be connected
to word processor.

19. SUITOR
( THE SIMPLE USER INTEREST
TRACKER )
 Helps in fetching more information at his
desktop.
 Fills a scrolling ticker on a computer screen
with information related to user’s current
task.
 e.g. If I start to read a story, pops up the
story in the browser window.

20. ADVANTAGES
 Reduces manual stress and fatigue.
 Reduces human effort.
 Prevention from dangerous incidents.

21. DISADVANTAGES
 Cannot force the operator to do work.
 Target not selected if not looked at for a
set threshold.
 Target selected if stared at without user
intension.

22. FUTURE AND CURRENT
DEVELOPMENTS
 EMOTIONAL MOUSE.
 MAGIC POINTING.
 SUITOR
 In future may be ordinary house hold devices
may do their jobs by looking at them or speaking
to them.( e.g. Ovens, televisions etc).

23. CONCLUSION
 Provides more delicate and user friendly in
computing devices.
 Gap between the electronic and physical
world is reduced.
 The computer can run on implicit
commands instead of the explicit commands.

24. References
http://www.research.ibm.com
http://www.media.mit.edu
http://www.thehumansolution.com
http://www.academia.edu
https://en.wikipedia.org/wiki/Affective_computing
http://youtube.com/
 https://en.wikipedia.org/wiki/Magic_Pointing