6. Imaged-Based Human Computer Interfaces for People with Severe Disabilities
Abstract
This paper presents an implementation of a low-cost image-based
communication aid system which allows people with disabilities to use their
limited voluntary motions to communicate with family and friends, access
computers, and control TV and air conditioner, etc. The system consists of only
one low-cost web camera and a personal computer.
The communication aid dichotomizes daily living necessities into 7 groups
(e.g., Voiced Messages, Typing, Home Appliance Control, Help, A/V
Entertainments, Web Surfing, Messages). In addition to the seven selections,
Suspend and Exit, are another two available options. The system provides users
with three kinds of operational interfaces to operate the communication aid. First
of all, people who still are able to control their arms can operate the system via
the movements of their arms. Secondly, for people who still can voluntarily open
and close their mouths, the system can be operated via the open and the closure
of the mouth. Thirdly, for people with severe disabilities, an extreme disability
such as severe cerebral palsy or amyotrophic lateral sclerosis, the system can be
operated via eye blinks. People with different kinds of disabilities can choose an
appropriate operational interface from the aforementioned three interfaces to
operate the communication aid.
Experimental results show that the performance of the proposed
communication aid is very encouraging.
Keywords: Human Computer Interface, Communication Aid, Eye-Blink
Communication, Amyotrophic Lateral Sclerosis.
ii
31. M −1
∑ h( x ) × xi i
x =
* i =0
M −1 (1)
∑ h( xi )
i =0
N −1
∑ h( y ) × yi i
y =
* i =0
N −1 (2)
∑ h( y )
i =0
i
M ×N x* y*
xi yi h( xi )
量 h( yi ) 量
論
14
14
離
19
32. 離 離不 離
離 錄參
m× n 亮度 (template)
離
離
離
利 來
度來
度
度
度
∑∑ (t (m, n) − t )(s(m, n) − s )
m m
γ= m n
(3)
(∑∑ t (m, n) − t )(∑∑ s (m, n) − s
m m )
m n m n
t ( m, n ) 亮度 tm 亮度
20
33. s ( m, n ) 亮度 sm 亮度
m n
度 15
綠
(a) (b)
15
15(b) 都 度
都
度
度
兩 類
例
16 藍 藍
度
21
34. 16
來 度
度 度
度
度
論
0.75
γ up ,up + γ up ,bottom + γ up ,left + γ up ,right
γ Avg _ up = (4)
4
γ bottom,up + γ bottom,bottom + γ bottom,left + γ bottom,right
γ Avg _ bottom = (5)
4
γ left ,up + γ left ,bottom + γ left ,left + γ left ,right
γ Avg _ left = (6)
4
γ right ,up + γ right ,bottom + γ right ,left + γ right ,right
γ Avg _ right = (7)
4
γ max = MAX (γ Avg _ up , γ Avg _ bottom , γ Avg _ left , γ Avg _ right ) (8)
γ up ,bottom
度 γ Avg _ up 度 γ max
度
22
70. 參
[1] J. Y. Bouguet, “Pyramidal Implementation of the Lucas Kanade Feature
Tracker Description of the Algorithm,” Intel Corporation Microprocessor
Research Labs.
[2] M. J. Black, D. J. Fleet, and Y. Yacoob, “A Framework for Modeling
Appearance Change in Image Sequences,” in International Conference on
Computer Vision, 1998, pp. 660-667.
[3] M. Betke, J. Gips, and P. Fleming, “The Camera Mouse: Visual Tracking
of Body Feature to Provide Computer Access for People with Severe
Disabilities,” IEEE Trans. on Neural Systems and Rehabilitation
Engineering, vol. 10, no. 1, pp. 1-10, 2002.
[4] T. N. Bhaskar, F. T. Keat, S. Ranganath, and Y. V. Venkatesh, “Blink
Detection and Eye Tracking for Eye Localization,” in Conference on
Convergent Technologies for Asia-Pacific Region, 2003, vol. 2, pp.
821-824.
[5] Y. L. Chen, F. T. Tang, W. H. Chang, M. K. Wong, Y. Y. Shih, and T. S.
Kuo, “The New Design of an Infrared-Controlled Human-Computer
Interface for the Disabled,” IEEE Trans. on Rehabilitation Engineering,
vol. 7, pp. 474-481, Dec. 1999.
[6] E. B. Delabarre, “A Method of Recording Eye-Movements,” The American
Journal of Psychology, vol. 9, no. 4, pp. 572-574, July 1898.
[7] L. S. Di and A. Bulgarelli, “A Simple and Efficient Connected Components
Labeling Algorithm,” in IEEE Proc. of Image Analysis and Processing,
Sep. 1999, pp. 322-327.
[8] D. G. Evans, R. Drew, and P. Blenkhorn, “Controlling Mouse Pointer
Position Using an Infrared Head-Operated Joystick,” IEEE Trans. on
Rehabilitation Engineering, vol. 8, no. 1, pp. 107-117, 2000.
[9] T. Fawcett, “An Introduction to ROC Analysis,” Pattern Recognition
58
71. Letters, vol. 27, no. 8, pp. 861-874, June 2006.
[10] M. F. Funada, S. P. Ninomija, S. Suzuki, K. Idogawa, Y. Yam, and H. Ide,
“On an Image Processing of Eye Blinking to Monitor Awakening Levels of
Human Beings,” in 18th Annual International Conference of the
Engineering in Medicine and Biology Society, 1996, vol. 3, pp. 966-967.
[11] K. Grauman, M. Betke, J. Gips, and G. R. Bradski, “Communication via
Eye Blinks – Detection and Duration Analysis in Real Time,” in Proc. of
CVPR 2001, vol. 1, pp. I-1010-1017.
[12] J. Gang and E. Sung, “Study on Eye Gaze Estimation,” IEEE Trans. on
Systems, Man and Cybernetics, Part B, vol. 32, no. 3, pp. 332-350, June
2002.
[13] R. Heishman and Z. Duric, “Using Image Flow to Detect Eye Blinks in
Color Videos,” in IEEE Workshop on Applications of Computer Vision,
2007.
[14] R. L. Hsu, M. A. Mottaleb, and A. K. Jain, “Face Detection in Color
Images,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.
24, pp. 696-706, 2002.
[15] D. Kumar and E. Poole, “Classification of EOG for Human Computer
Interface,” in Conference in the Second Joint EMBS/BMES, Oct. 2002, vol.
1, pp. 64-67.
[16] C. S. Lin, C. C. Huan, C. N. Chan, M. S. Yeh, and C. C. Chiu, “Design of a
Computer Game Using an Eye-Tracking Device for Eye's Activity
Rehabilitation,” Optics and Lasers in Engineering, pp. 91-108, July 2004.
[17] H. Lim and V. K. Singh, “Design of Healthcare System for Disable Person
Using Eye Blinking,” in Proc. of Fourth Annual ACIS International
Conference on Computer and Information Science, 2005, pp. 551-555.
[18] C. H. Morimoto, D. Koons, A. Amit, M. Flickner, and S. Zhai, “Keeping an
Eye for HCI,” in Processing of XII Brazilian Symp. Computer Graphics
and Image, 1999, pp. 171-176.
59
72. [19] T. Miyakawa, H. Takano, and K. Nakamura, “Development of Non-contact
Real-time Blink Detection System for Doze Alarm,” in Proc. of SICE
Annual Conference, 2004, vol. 2, pp. 1626-1631.
[20] G. Norris and E. Wilson, “The Eye Mouse, An Eye Communication
Device,” in IEEE Proc. of Bioengineering, May 1997, pp. 66–67.
[21] I. Park, J.-H. Ahn, and H. Byun, “Efficient Measurement of Eye Blinking
under Various Illumination Conditions for Drowsiness Detection Systems,”
in Proc. of the 18th International Conference on Pattern Recognition, 2006,
vol. 1, pp. 383-386.
[22] K. S. Park and K. T. Lee, “Eye-Controlled Human Computer Interface
Using the Line-of-Sight and the Intentional Blink,” Computer Engineering,
vol. 30, no. 3, pp. 463-473, 1996.
[23] R. B. Reilly and M. J. O’Malley, “Adaptive Noncontact Gesture-Based
System for Augmentative Communication,” IEEE Trans. on Rehabilitation
Engineering, vol. 7, no. 2, pp. 174-182, 1999.
[24] J. A. Stern, “The Eye Blink: Affective and Cognitive Influence,” in Anxiety:
Recent Developments in Cognitive, Psychophysiological, and Health
Research, D. G. Forgays, T. Sosnowski, and K. Wrzesniewski, Eds.
Washington: Hemisphere Publishing, 1992, ch. 8, pp. 109-128.
[25] M. C. Su, S. Y. Su, and G. D. Chen, “A Low Cost Vision-Based
Human-Computer Interface for People with Severe Disabilities,”
Biomedical Engineering-Applications, Basis, Communications, vol. 17, no.
6, pp. 10-18, 2005.
[26] Y. Tomita, Y. Igarashi, S. Honda, and N. Matsuo, “Electro-Oculography
Mouse for Amyotrophic Lateral Sclerosis Patients,” in IEEE Conference
Engineering in Medicine and Biology Society, Nov. 1996, vol. 5, pp.
1780-1781.
[27] O. Takami, N. Irie, C. Kang, T. Ishmatsu, and T. Ochiai, “Computer
Interface to Use Head Movement for Handicapped People,” in Proc. of
60
73. IEEE TENCON’96, Digital, Signal Processing Applications, 1996, vol. 1,
pp. 468-472.
[28] T. Westeyn, P. Pesti, K.-H. Park, and T. Starner, “Biometric Identification
Using Song-Based Blink Pattern,” in Proc. of 11th International Conference
on Human-Computer Interaction, 2005, vol. 3.
[29] L. Young and D. Sheena, “Survey of Eye Movement Recording Methods,”
Behav. Res. Meth. Instrum, vol. 7, no. 5, pp. 397-429, 1975.
[30] Z. Zhu and Q. Ji, “Eye Gaze Tracking under Natural Head Movements,” in
IEEE Computer Society Conference on Computer Vision and Pattern
Recognition, June 2005, vol. 1, pp. 918-923.
[31] LC Technologies, INC. EYEGAZE SYSTEMS. [Online]. Available:
http://www.eyegaze.com/INDEX.htm July 2008 [date accessed]
[32] Texas Instruments, System-On-Chip for 2.4 GHz ZigBee. [Online].
Available: http://focus.ti.com/lit/ds/symlink/cc2431.pdf July 2008 [date
accessed]
[33] 李 理參數量 論
年六
[34] 立 數 理 林
年
[35] 論
立 年
[36] 論
六年
[37] 論
立 年六
[38] PC Camera 論
年
61
