BlueEyes is a system that monitors human operators through eye tracking and other physiological sensors. It detects eye movements and brain activity to determine the operator's level of attention and triggers alerts if attention is low. The system aims to prevent human errors by continuously monitoring operators in critical environments like power plants, hospitals, and transportation. It collects data through a wearable sensor and sends it wirelessly to a central computer for analysis and visualization of the operator's state.

1. BlueEyes
Human
Operator
Monitoring
System
•What is
Blue Eyes ?
•Motivation
•Physiological
foundations
•System
Blue Eyes
design
•System
implementation
•Future
improvements
•Conclusion
Technologies

2. BlueEyes
Human
Operator
What is BlueEyes ?
Monitoring
System
•What is
Blue Eyes ?
•Motivation
• The complex solution for human-operator
•Physiological monitoring:
foundations
•System – conscious brain involvement monitoring
design
•System – triggering user-defined actions
implementation
•Future – data recording
improvements
•Conclusion
What is BlueEyes not?
• Doesn’t predict nor interfere with
operator’s thoughts
• Cannot force directly the operator to work

3. BlueEyes
Human
Operator
Who is BlueEyes for ?
Monitoring
System
•What is
Blue Eyes ?
•Motivation
• Generic control rooms
•Physiological
foundations
– Power station
•System
design
– Flight control centers
•System
implementation
– Operating theatres – anesthesiologists
•Future
improvements
•Conclusion • Common application
– A simpler system version for drivers

4. BlueEyes
Human
Operator
Motivation
Monitoring
System
•What is
Blue Eyes ?
•Motivation
• Human error - a frequent reason of
•Physiological catastrophes and ecological disasters
foundations
•System – temporal indisposition
design
•System – weariness
implementation
•Future – oversight
improvements
•Conclusion • Monitoring of conscious
brain involvement
• Capable sensors available
• Lack of complex solution exploiting wireless
link technology, database systems and AI
algorithms

5. BlueEyes
Human
Operator
Wires, wires, wires...
Monitoring
System
•What is
Blue Eyes ?
•Motivation
•Physiological
foundations A serious limitation of:
•System
design
•System
implementation – Mobility
•Future
improvements – Comfort
•Conclusion
– Safety

6. BlueEyes
Human
Operator
Personal Area Network !
Monitoring
System
•What is
Blue Eyes ?
•Motivation
•Physiological
foundations Solution:
•System
design
•System A wireless Personal Area
implementation
•Future Network
improvements
•Conclusion

7. BlueEyes
Human
Operator
BlueEyes - benefits
Monitoring
System
•What is
Blue Eyes ?
•Motivation
• Prevention from dangerous incidents
•Physiological
foundations
• Minimization of
•System
design
– ecological consequences
•System
implementation
– financial loss
•Future – a threat to a human life
improvements
•Conclusion
• The reconstruction
of the course of
operator’s work

8. BlueEyes
Human
Operator
Physiological foundations
Monitoring
System
•What is
Blue Eyes ?
•Motivation
•Physiological Human brain
foundations
•System
design
•System Preconscious brain Conscious brain
implementation
•Future
improvements
• illogical • manages the
•Conclusion • “automatic” knowledge
• logical reasoning
• makes decisions

9. BlueEyes
Human
Operator
Conscious brain - operation modes
Monitoring
System
•What is
Blue Eyes ?
•Motivation
•Physiological
foundations
•System
design
•System
implementation
•Future
improvements
•Conclusion
Exploratory mode Planning mode

10. BlueEyes
Human
Operator
Eye movements
Monitoring
System
•What is
Blue Eyes ?
•Motivation
• Retina - central vision (macula)
•Physiological
foundations
• Central vision and
•System the conscious brain
design
•System
implementation
•Future
Saccade - a rapid
improvements eye jump to a new
•Conclusion
location assigned
by the conscious
attention process.
Saccades presence ⇔ Exploratory mode

11. BlueEyes
Human
Operator
The eye movement sensor
Monitoring
System
•What is
Blue Eyes ?
•Motivation
•Physiological
foundations
• Eye position measuring - direct infrared
•System oculography
design
•System • Two axial
implementation
•Future
accelerometer
improvements
•Conclusion
• Ambient light
sensor
Jazz Multisensor

12. BlueEyes
Human
Operator
Design objectives
Monitoring
System
•Motivation
•What is
Blue Eyes ?
• A mobile data acquisition device
•Physiological
foundations
• A reliable real-time buffering, processing
•System and recording
design
•Objectives • A clear visualization interface
•System
overview • Mechanisms for creating custom data
•System analyzers
implementation
•Future • To ensure a possibility of distributing the
improvements
•Conclusion
processing among a number of computers

13. BlueEyes
Human
Operator
System overview
Monitoring
System
•Motivation
•What is
Blue Eyes ?
Data Acquisition Unit Central System Unit
•Physiological
foundations
Atmel 89C52 Connection Visualization
•System Bluetooth Bluetooth
design microcontroller Manager Module
•Objectives
Jazz Data
•System Data Analysis
Multisensor Logger
overview
•System
implementation
•Future
improvements Bluetooth technology provides
•Conclusion
means for creating a Personal
Area Network linking the
operators and the central
system.

14. BlueEyes
Human
Operator
Implementation - DAU
Monitoring
System
•Motivation
•What is
Blue Eyes ?
•Physiological
foundations
•System
design
•System
implementation
•DAU
•CSU
•Future
improvements
•Conclusion

15. BlueEyes
Human
Operator
Implementation - DAU
Monitoring
System
•Motivation
•What is
Blue Eyes ?
• Bluetooth module inside
•Physiological
foundations
• Jazz sensor
•System interface
design
•System • Microphone and
implementation
earphone sockets
•DAU
•CSU
• 5-key keyboard
• Two lines, 8
•Future
improvements characters LCD
•Conclusion
• Powered from
6 AA batteries

16. BlueEyes
Human
Operator
DAU - components
Monitoring
System
•Motivation
•What is
Blue Eyes ?
•Physiological
foundations microphone MC 145483
•System
earphone PCM codec
design
•System Jazz Multisensor Bluetooth
implementation
module
•DAU Beeper Atmel 89C52
UART
•CSU microcontroller
LCD display
UART MAX232
•Future LED indicators
improvements
•Conclusion
BlueEyes
Data Acquisition Unit
ID card Simple
interface keyboard
ID card

17. BlueEyes
Human
Operator
DAU - features
Monitoring
System
•Motivation
•What is Data Acquisition Unit
Blue Eyes ? • Lightweight
•Physiological
foundations • Runs on batteries - low
Atmel 89C52
•System
design microcontroller
Bluetooth power consumption
•System
implementation Jazz • Easy to use - does not
Multisensor disturb the operator
•DAU
working
•CSU
• ID cards for operator
•Future
improvements authorization
•Conclusion
• Voice transmission using
hardware PCM codec

18. BlueEyes
Human
Operator
Implementation - CSU
Monitoring
System
•Motivation
•What is
• Standard MDI user
Blue Eyes ? interface
•Physiological
foundations • Custom-built
•System
design
Bluetooth
•System communication
implementation
routines
•DAU • Intel USB camera
•CSU
• ID card programmer
•Future
improvements
• Developed using Visual C++ 6.0
•Conclusion • Deployment environment:
Windows 2000,
MS SQL Server

19. BlueEyes
Human
Operator
CSU - saccade detector
Monitoring
System
•Motivation
•What is • Saccade detector and conscious brain
Blue Eyes ?
•Physiological involvement level calculation
foundations
Eye movement velocity
•System Raw eye 250 saccade
design 200
movement data 150
•System 100
Velocity
50
implementation 0
-50
-100
-150
•DAU -200
-250
•CSU Differentiation 0 8000 16000 24000 32000
Time [ms]
•Future Visual attention level
improvements 60
Attention level (Lva)
•Conclusion 50
40 Conscious
30 brain
20
involvement
10
0
level
0 8000 16000 24000 32000
Time [ms]

20. BlueEyes
Human
Operator
CSU - Visualization Module
Monitoring
System
•Motivation
•What is • Raw and processed data visualization
Blue Eyes ?
•Physiological using:
foundations
•System – VU-meters
design
•System
implementation
– pie-charts
– time series
•DAU
•CSU
•Future
improvements • audio and video
•Conclusion
data playback

21. BlueEyes
Human
Operator
CSU - Connection Manager
Monitoring
System
•Motivation
•What is Operator
Blue Eyes ? Data Separated
•Physiological Manager data streams
foundations
•System Raw Data
design Bluetooth Operator Buffer
•System Manager
implementation Alert buffer
Alerts
•DAU
•CSU .. Bluetooth
.
Connection ...
Manager
•Future
improvements .. ...
Raw Data
•Conclusion Operator
.
Bluetooth Manager
Buffer
Alert buffer Alerts

22. BlueEyes
Human
Operator
CSU - features
Monitoring
System
•Motivation
•What is Central System Unit
Blue Eyes ? • Connection
•Physiological
foundations
management
Connection Visualization
•System
design
• Data processing Bluetooth Manager Module
•System
implementation
• Visualization Data Analysis
Data
Logger
•DAU • Data recording
•CSU
•Future • Access
improvements
•Conclusion verification
• System maintenance

23. BlueEyes
Human
Operator
CSU - Data Analysis
Monitoring
System
•Motivation
•What is • Custom data analyzers - decision tree
Blue Eyes ?
•Physiological based classifiers
foundations
•System
design Example: Operator gesture recognition
•System (simplified)
implementation
Vertical Acceleration variance
•DAU value high
•CSU
Horizontal Y N
•Future Acceleration variance Horizontal Acceleration
improvements high variance high
•Conclusion N Y N Y
“YES” No answer “NO”

24. BlueEyes
Human
Operator
CSU - Data Analysis
Monitoring
System
•Motivation
•What is • Custom data analyzers - decision tree
Blue Eyes ?
•Physiological
based classifiers
foundations
•System • Automatic decision tree coefficients
design
•System
generation - C4.5 based algorithm
implementation
Assigned
•DAU Data samples category
•CSU
“Yes”
•Future No
improvements
... answer
...
•Conclusion Decision tree
“Yes” coefficients
“No”
No
answer

25. BlueEyes
Human
Operator
Data security
Monitoring
System
•Motivation
•What is • Only registered mobile devices
Blue Eyes ?
•Physiological can connect to the system
foundations
•System
design
• Bluetooth connection authentication
•System
implementation • Bluetooth connection encryption
•DAU
•CSU
• Access rights restrictions
• Personal and physiological data
•Future
improvements
encryption
•Conclusion

26. BlueEyes
Human
Operator
Future improvements
Monitoring
System
•Motivation
•What is • DAU
Blue Eyes ?
•Physiological – small CMOS camera to monitor the
foundations
operator’s point of gaze
•System
design
– single PCB (SMD technlogy)
•System
implementation
– low voltage ICs - LiIO batteries power
•Future
improvements
•Conclusion
• CSU
– data mining algorithms
– advanced database encryption using e.g.
AES algorithm

27. BlueEyes
Human
Operator
Conclusion
Monitoring
System
•Motivation
•What is
Blue Eyes ?
• In the near future ,ordinary
•Physiological
foundations
household devices- such as television
•System
design
, refrigerators ,ovens may be able to
•System do their jobs when we look at them
implementation
•Future and speak to them.
improvements
•Conclusion
• Future applications of blue eye
technology is limitless

28. BlueEyes
Human
Operator
Monitoring
Thank you for your attention
System
•Motivation
•What is
Blue Eyes ?
•Physiological
Any Queries….???
foundations
•System
Blue Eyes
design
•System
implementation
Technology
•Future
improvements
•Conclusion