This document summarizes the Opengazer project, which uses a webcam to track users' gestures, head movements, and facial features to allow communication through a laptop. The system includes a gesture switch that detects eyebrow raises or smiles to select items, a ticker audio method for text entry that can cope with noise, and a head tracker that moves the mouse based on translations of the user's head. The overall goal is to help people with disabilities like locked-in syndrome communicate automatically.

1. Opengazer: Gesture switch, Ticker & Head tracking
Emli-Mari Nel, Patrick Welche
Department of Physics
University of Cambridge
Overview Ticker
● Process data from webcam to communicate: ● Each voice reads a part of the alphabet.
● For example, Cheerful Charlie reads “m”- “q”.
* £30, USB connection ● Say “yes” when you hear the letter you want to write.
* Automatic gain control ● Select a word in dictionary based on probability.
* 640x480 @ 30fps ● Update click distribution.
* Easy to mount ● High speed:
● Multiple voices/channels & tolerance to noise.
● High noise tolerance:
● Generate switch events from facial gestures: ● Click distribution
● Soft decisions on letter selections (probabilities)
→Laptop→YES→Text-entry method ● Random alphabet sequence.
● Ticker:
● An audio text-entry method for single-switch users
● Can cope with noise: Imperfect switch, user delays
“a” ? →YES (smile) →Increase probability of “a”
●
Head translations→Mouse position→Dasher:
Head tracker
Head tracker
● Based on Viola-Jones face detector.
●Components of larger system called Opengazer ● Use rectangular features.
(gaze tracking) which is not complete yet. ● Find most probable location where face is.
● Help severely disabled users, e.g., with locked-in
● Control mouse based on head translations.
syndrome, to communicate automatically.
Gesture switch
Training:
Make gesture three times
Testing:
Automatically detect gesture → switch event.
Results:
Robust eye-brow/smile detector:
(outperforms EMG switch),
Currently making robust for eye-motions.
Method:
● Motion-based features.
● Quantify gesture: Sequence of images, where Acknowledgments
some pixels are more probable to be on than off. We would like to thank David MacKay for his guidance, Mick
●
Example: Donegan for his assistance to visit users who can benefit from
● Face rectangle, 60x60. this research, and the Gatsby Charitable Foundation for
● Red (more probable),
previously funding this project.
● Eye brow raise = yes.
References
[1] http://www.inference.phy.cam.ac.uk/opengazer/
[2] http://www.inference.phy.cam.ac.uk/dasher/
www
www.aegis-project.eu