Scott MacKenzie at BayCHI: Evaluating Eye Tracking Systems for Computer Data Entry

3,282 views

Published on

The human eye, with the assistance of an eye tracking apparatus, may serve as an input controller to a computer system. Much like point-select operations with a mouse, the eye can "look-select", and thereby activate items such as buttons, icons, links, or text. Evaluating the eye working in concert with an eye tracking system requires a methodology that uniquely addresses the characteristics of both the eye and the eye tracking apparatus. Among the interactions considered are eye typing and mouse emulation. Eye typing involves using the eye to interact with an on-screen keyboard to generate text messages. Mouse emulation involves using the eye for conventional point-select operations in a graphical user interface. In this case, the methodologies for evaluating pointing devices (e.g., Fitts' law and ISO 9241-9) are applicable but must be tailored to the unique characteristics of the eye, such as saccadic movement. This presentation surveys and reviews these and other issues in evaluating eye-tracking systems for computer input.

Scott MacKenzie is associate professor of Computer Science and Engineering at York University, Toronto, Canada. His research is in human-computer interaction with an emphasis on human performance measurement and modeling, experimental methods and evaluation, interaction devices and techniques, alphanumeric entry, language modeling, and mobile computing. He has more than 100 peer-reviewed publications in the field of Human-Computer Interaction, including more than 30 from the ACM's annual SIGCHI conference. He has given numerous invited talks over the past 20 years.

  • Be the first to comment

Scott MacKenzie at BayCHI: Evaluating Eye Tracking Systems for Computer Data Entry

  1. 1. Evaluating Eye Tracking Systems for Computer Data Entry I. Scott MacKenzie York University http://www.yorku.ca/mack/
  2. 2. Evaluating Eye Tracking Systems for Computer Data Entry I. Scott MacKenzie York University http://www.yorku.ca/mack/
  3. 3. Overview <ul><li>Eye tracking challenges </li></ul><ul><li>Looking </li></ul><ul><li>Selecting </li></ul><ul><li>Evaluating </li></ul><ul><li>Case studies </li></ul>
  4. 4. Eye Tracking Challenges <ul><li>“ It’s not about the bike” (Lance Armstrong) </li></ul><ul><li>“ It’s not about the tracker” </li></ul><ul><li>Evaluate the interface, the interaction, not the eye tracker </li></ul><ul><li>Easier said than done (like speech or handwriting recognition) </li></ul><ul><li>Human-machine interaction </li></ul>
  5. 5. Human-Computer Interface Using the eye as the input control
  6. 6. “ Double Duty”
  7. 7. An Eye Tracking Session at CHI Input control Visual search Visual search Visual search
  8. 8. Two-step Input Control <ul><li>Mouse  point-select </li></ul><ul><li>Eye  look-select </li></ul><ul><li>Where is the user looking? </li></ul><ul><li>Select it! </li></ul>
  9. 9. Overview <ul><li>Eye tracking challenges </li></ul><ul><li>Looking </li></ul><ul><li>Selecting </li></ul><ul><li>Evaluating </li></ul><ul><li>Case studies </li></ul>
  10. 10. The Eye
  11. 11. The Eye
  12. 12. The Eye
  13. 13. Where Is The User Looking? Pointing  = 0.17  1000 ft 3 ft
  14. 14. Where Is The User Looking? 30 in 0.25 in  = 0.48 
  15. 15. Where Is My Thumb?
  16. 16. Where Is My Thumb? 0.6 in 25 in  = 1.2 
  17. 17. Double Trouble! 1.2 in 25 in  = 2.3 
  18. 18. Fovea <ul><li>Area of the eye responsible for sharp central vision </li></ul><ul><li>Necessary for reading, driving, sewing, etc. </li></ul><ul><li>1% of retina, but 50% of visual cortex in brain </li></ul><ul><li>The fovea sees “twice the width of a thumbnail at arms length” (about 2-3 degrees) </li></ul>Eye tracking relevance: The fovea image represents a “region of uncertainty”
  19. 19. Where is the User Looking?
  20. 20. Demo
  21. 21. Two-step Eye Input Control <ul><li>Mouse  point-select </li></ul><ul><li>Eye  look-select </li></ul><ul><li>Where is the user looking? </li></ul><ul><li>Select it! </li></ul>
  22. 22. Overview <ul><li>Eye tracking challenges </li></ul><ul><li>Looking </li></ul><ul><li>Selecting </li></ul><ul><li>Evaluating </li></ul><ul><li>Case studies </li></ul>
  23. 23. “ Smart” Target Acquisition <ul><li>Target acquisition complicated by </li></ul><ul><ul><li>Fovea’s “region of uncertainty” </li></ul></ul><ul><ul><li>Eye jitter </li></ul></ul><ul><ul><li>Identifying a fixation from raw data </li></ul></ul><ul><ul><li>No natural selection method for eye tracking </li></ul></ul><ul><li>Selection methods </li></ul><ul><ul><li>Dwell time (250 ms to 1000 ms) </li></ul></ul><ul><ul><li>Separate key press </li></ul></ul><ul><ul><li>Blink, wink, frown, etc. </li></ul></ul><ul><li>E.g., “smart” target acquisition techniques </li></ul>
  24. 24. Dwell-time Progress Indicators 1 1 Hansen, J. P., Hansen, D. W., & Johansen, A. S. (2001). Bringing gaze-based interaction back to basics. Proceedings of HCI International 2001 , 325-328. Mahwah, NJ: Erlbaum.
  25. 25. Shrinking Letter Progress Indicator 1 1 Majaranta, P., MacKenzie, I. S., Aula, A., & Räiha, K.-J. (2003). Auditory and visual feedback during eye typing. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems - CHI 2003 , 766-767. New York: ACM.
  26. 26. Grab And Hold Algorithm 1 1 Miniotas, D., Špakov, O., & MacKenzie, I. S. (2004). Eye gaze interaction with expanding targets. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems - CHI 2004 , 1255-1258. New York: ACM.
  27. 27. Next-letter Prediction 1 1 MacKenzie, I. S., & Zhang, X. (2008). Eye typing using word and letter prediction and a fixation algorithm. Proceedings of the ACM Symposium on Eye Tracking Research and Applications -- ETRA 2008 , 55-58. New York: ACM.
  28. 28. Speech-assisted Selection 1 1 Zhang, Q., Imamiya, A., Go, K., & Mao, X. (2004). Resolving ambiguities of a gaze and speech interface, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2004 (pp. 85-92): New York: ACM.
  29. 29. Speech-assisted Selection 1 1 Miniotas, D., Spakov, O., & MacKenzie, I. S. (2006). Speech-augmented eye gaze interaction with small closely-spaced targets. Proceedings of the Symposium on Eye Tracking Research and Applications - ETRA 2006 , 67-72. New York: ACM.
  30. 30. Snap-on Selection 1 1 Tien, G., & Atkins, M. S. (2008). Improving hands-free menu selection using eye gaze glances and fixations. Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2008 , 47-50. New York: ACM.
  31. 31. Snap-clutch Selection 1 1 Istance, H., Bates, R., Hyrskykari, A., & Vickers, S. (2008). Snap clutch: A moded approach to solving the Midas touch problem. Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2008 , 221-228. New York: ACM.
  32. 32. Fisheye Lens Selection 1 1 Ashmore, M., Duchowski, A. T., & Shoemaker, G. (2005). Efficient eye pointing with a fisheye lens . Proceedings of Graphics Interface 2005 , 203-210. Toronto: Canadian Information Processing Society.
  33. 33. Cursor Warping 1 1 Zhai, S., Morimoto, C., & Ihde, S. (1999). Manual gaze input cascaded (MAGIC) pointing . Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI '99 , 248-253. New York: ACM.
  34. 34. Early-raw-smoothed-late Selection 1 1 Kumar, M., Klingner, J., Winograd, T., & Paepcke, A. (2008). Improving the accuracy of gaze input for interaction. Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2008 , 65-68. New York: ACM.
  35. 35. Gesture-based Selection 1 1 Mollenbach, E., Hansen, J. P., Lillholm, M., & Gale, A. G. (2009). Single stroke gaze gestures. Extended Abstracts of the ACM Conference on Human Factors in Computing Systems - CHI 2009 , 4555-4560. New York: ACM.
  36. 36. Nearest Neighbor Selection 1 1 Sibert, L. E., & Jacob, R. J. K. (2000). Evaluation of eye gaze interaction. Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI 2000 , 281-288. New York: ACM.
  37. 37. Big Targets 1 1 Hansen, J. P., Johansen, A. S., Hansen, D. W., Itoh, K., & Mashino, S. (2003). Command without a click: Dwell time typing by mouse and gaze selection. Proceedings of INTERACT 2003 , 121-128. Berlin: Springer. Demo
  38. 38. Overview <ul><li>Eye tracking challenges </li></ul><ul><li>Looking </li></ul><ul><li>Selecting </li></ul><ul><li>Evaluating </li></ul><ul><li>Case studies </li></ul>
  39. 39. Focus on the Interaction <ul><li>Eye tracking research tends to focus on the technology (hardware, software, algorithms) </li></ul><ul><li>Focus on the interaction </li></ul><ul><li>With eye tracking, easier said than done </li></ul><ul><li>Eye tracking technology is finicky (perhaps you have a better word) </li></ul><ul><li>Remember the human-computer interface (5 th slide) </li></ul>
  40. 40. Evaluation Paradigm <ul><li>Experimental in nature </li></ul><ul><li>Follow “the Scientific Method” </li></ul><ul><ul><li>Human participants </li></ul></ul><ul><ul><li>Independent variables (factors and levels) </li></ul></ul><ul><ul><li>Dependent variables (measure behaviours) </li></ul></ul><ul><ul><li>Counterbalancing </li></ul></ul><ul><ul><li>Hypothesis testing (analysis of variance) </li></ul></ul><ul><ul><li>Etc. </li></ul></ul>
  41. 41. Independent Variables (eye tracking)
  42. 42. Dependent Variables (eye tracking)
  43. 43. Read Text Events (RTE)
  44. 44. Eye Tracking Path Characteristics Are there observable, measurable behaviours here – that can be used as dependent variables?
  45. 45. Case Studies <ul><li>Case Study #1 – Eye typing </li></ul><ul><li>Case Study #2 – BlinkWrite </li></ul>
  46. 46. Based on…
  47. 47. Background <ul><li>The problem </li></ul><ul><ul><li>Eye typing – using the eye to enter text into an application by fixating on keys on an on-screen keyboard </li></ul></ul><ul><li>The challenge </li></ul><ul><ul><li>Improve the speed and accuracy of input </li></ul></ul><ul><li>Ideas </li></ul><ul><ul><li>Word prediction, letter prediction </li></ul></ul><ul><ul><li>Use linguistic information to correct for fixation error </li></ul></ul>
  48. 48. Fixation Error <ul><li>User fixation point Computed fixation point </li></ul> Q W E R T Y U A S D F G H J Z X C V B N M I O P K L Desired input: EYE TRACKIN_ + + User fixates here Computed fixation point
  49. 49. Fixation Algorithm <ul><li>Use linguistic knowledge to choose the “correct” button </li></ul>Q W E R T Y U A S D F G H J Z X C V B N M I O P K L Desired input: EYE TRACKIN_ + + User fixates here Computed fixation point
  50. 50. Fixation Algorithm Model <ul><li>Parameters include </li></ul><ul><ul><li>tolerantDrift </li></ul></ul><ul><ul><li>Weighting method </li></ul></ul><ul><ul><li>Word stem size (not discussed here) </li></ul></ul>
  51. 51. tolerantDrift <ul><li>Definition: </li></ul><ul><ul><li>Radius around the computed fixation point within which alternative buttons are considered for “eye-over” highlighting </li></ul></ul><ul><li>In modeling this, we assume the user is fixating on the correct button but that the computed fixation point is off by n pixels </li></ul><ul><li>Our model uses n = 55, 60, 65, 70, 75, 80, 85, and 90 pixels </li></ul><ul><li>For each button within the tolerantDrift region, a word stem is built </li></ul><ul><li>The button to receive “eye-over” highlighting is then computed using a weighting method for the candidate word stems </li></ul>
  52. 52. tolerantDrift Example
  53. 53. Weighting Method <ul><li>0 None (always choose the closest button) </li></ul><ul><li>1 Weighting = word stem frequency divided by the squared distance to the center of the button </li></ul><ul><li>2 Weighting = word stem frequency divided by the distance to the center of the button </li></ul><ul><li>3 Weighting = word stem frequency </li></ul>
  54. 54. Model Results Note: Small buttons (diameter = 76 pixels)
  55. 55. Model Results (including word stem size) Weighting method 0 (each bar is the average of the error rates computed over the range of tolerantDrift) Weighting method 1 tolerantDrift = 70 pixels
  56. 56. Method <ul><li>Participants </li></ul><ul><ul><li>10 (8 male, 2 female; 19-34 years of age) </li></ul></ul><ul><ul><li>All with normal vision </li></ul></ul><ul><li>Apparatus </li></ul><ul><ul><li>Arrington Research ViewPoint </li></ul></ul><ul><ul><li>30 Hz sampling </li></ul></ul><ul><ul><li>0.25°-1.0° visual arc accuracy (~10-40 pixels ) </li></ul></ul><ul><ul><li>C amera focused on a participant’s dominant eye </li></ul></ul><ul><ul><li>19&quot; 1280×1024 LCD at a distance of ~60 cm </li></ul></ul><ul><ul><li>Selection via key press </li></ul></ul>
  57. 57. Setup
  58. 58. Independent Variables (IV)
  59. 59. Button Size, Word Prediction
  60. 60. Letter Prediction
  61. 61. Button Feedback Normal Letter Predict Eye- over Select (+audible “click”)
  62. 62. Task <ul><li>Phrases presented for input </li></ul><ul><li>Example: </li></ul>Error (“beep”) Next character to enter Timeout (after six seconds)
  63. 63. Dependent Variables (DV) <ul><li>Speed and accuracy </li></ul><ul><li>Others (not mentioned here) </li></ul>
  64. 64. Data Collection <ul><li>10 participants × </li></ul><ul><li>2 prediction modes × </li></ul><ul><li>2 fixation algorithms × </li></ul><ul><li>2 button sizes × </li></ul><ul><li>10 sentences per condition = 800 phrases </li></ul>
  65. 65. Results <ul><li>Entry speed </li></ul><ul><ul><li>Grand mean: 11.7 wpm </li></ul></ul><ul><li>Error rate </li></ul><ul><ul><li>Grand mean: 11.8% </li></ul></ul>Method Entry speed EyeWrite 4.87 wpm Dasher (1 st session) 2.49 wpm Dasher (10 th session) 17.26 wpm Eye-S 6.8 wpm (2 experts) 2008 Comparison with other results from ETRA 2008
  66. 66. Entry Speed by Button Size Difference NOT statistically significant ( F 1,15 = 1.162, p > .05 )
  67. 67. Entry Speed by Prediction Mode Difference (marginally) significant ( F 1,15 = 5.531 , p < .05 )
  68. 68. Entry Speed by Fixation Algorithm Difference NOT statistically significant ( F 1,15 = 0.878 , ns)
  69. 69. Conclusions <ul><li>Letter prediction as good as word prediction (in our implementation) </li></ul><ul><li>F ixation algorithm </li></ul><ul><ul><li>Slight improvement </li></ul></ul><ul><ul><li>S ensitive to the correctness of the first letters in a word </li></ul></ul><ul><ul><li>Reduced e rror rates, but only in letter prediction mode </li></ul></ul><ul><li>Experiment was too big (two IVs max!) </li></ul><ul><li>Too much variability in the data (eye tracker issues) </li></ul>
  70. 70. Case Studies <ul><li>Case Study #1 – Eye typing </li></ul><ul><li>Case Study #2 – BlinkWrite </li></ul>Skip
  71. 71. Based on…
  72. 72. Background <ul><li>The problem </li></ul><ul><ul><li>Create and evaluate a single-switch text entry method </li></ul></ul><ul><ul><li>Switch input using eye blinks </li></ul></ul><ul><li>The application </li></ul><ul><ul><li>Severely disable users (e.g., locked-in syndrome) </li></ul></ul><ul><li>The general idea </li></ul><ul><ul><li>SAK: Scanning Ambiguous Keyboard 1 </li></ul></ul>1 MacKenzie, I. S. (2009). The one-key challenge: Searching for an efficient one-key text entry method . Proceedings of the ACM Conference on Computers and Assessibility - ASSETS 2009 , 91-98. New York: ACM.
  73. 73. BlinkWrite Application Demo
  74. 74. Three Types of Blinks <ul><li>Involuntary (0-200 ms) </li></ul><ul><li>Key select (200-500 ms) </li></ul><ul><li>Delete (>500 ms) </li></ul>
  75. 75. Method <ul><li>Participants </li></ul><ul><ul><li>12 (8 male, 4 female; 23-31 years of age) </li></ul></ul><ul><ul><li>All with normal vision </li></ul></ul><ul><li>Apparatus </li></ul><ul><ul><li>EyeTech TM3 </li></ul></ul><ul><ul><li>Windows XP notebook </li></ul></ul><ul><ul><li>15” LCD </li></ul></ul><ul><ul><li>(next slide) </li></ul></ul><ul><li>Task </li></ul><ul><ul><li>Enter phrases of text </li></ul></ul><ul><ul><li>Corrections allowed using “long blink” </li></ul></ul>
  76. 76. Setup
  77. 77. Independent Variable <ul><li>Scanning Interval </li></ul><ul><ul><li>700 ms, 850 ms, 1000 ms </li></ul></ul><ul><ul><li>Five phrases each </li></ul></ul><ul><ul><li>Counterbalanced </li></ul></ul>
  78. 78. Dependent Variables <ul><li>Entry speed (wpm) </li></ul><ul><li>Error rate (%) </li></ul><ul><li>Deletes per phrase </li></ul>
  79. 79. Results – Entry Speed Differences NOT statistically significant ( F 2,22 = 2.05, p > .05 )
  80. 80. Results – Error Rate Differences NOT statistically significant ( F 2,22 = 1.89, p > .05 )
  81. 81. Results – Deletes Per Phrase
  82. 82. Video Demo
  83. 83. References 1. Ashmore, M., Duchowski, A. T., and Shoemaker, G., Efficient eye pointing with a fisheye lens, Proceedings of Graphics Interface 2005, (Toronto: Canadian Information Processing Society, 2005), 203-210. 2. Hansen, J. P., Hansen, D. W., and Johansen, A. S., Bringing gaze-based interaction back to basics, Proceedings of HCI International 2001, (Mahwah, NJ: Erlbaum, 2001), 325-328. 3. Hansen, J. P., Tørning, K., Johansen, A. S., Itoh, K., and Aoki, H., Gaze typing compared with input by head and hand, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2004, (New York: ACM, 2004), 131-138. 4. Itoh, K., Aoki, H., and Hansen, J. P., A comparative usability study of two Japanese gaze typing systems, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2006, (New York:, 2006), 59-66. 5. Majaranta, P., MacKenzie, I. S., Aula, A., and Räiha, K.-J., Effects of feedback and dwell time on eye typing speed and accuracy, Universal Access in the Information Society (UAIS), 5, 2006, 199-208. 6. Pan, B., Hembrooke, H. A., Gay, G. K., Granka, L. A., Feusner, M. K., and Newman, J. K., The determinants of web page viewing behavior: An eye-tracking study, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2004, (New York: ACM, 2004), 147-154. 7. Sodhi, M., Reimer, B., Cohen, J. L., Vastenburg, E., Kaars, R., and Kirschenbaum, S., Off road driver eye movement tracking using head-mounted devices, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2002, (New York: ACM, 2002), 61-68. 8. Takahashi, K., Nakayama, M., and Shimizu, Y., The response of eye movement and pupil size to audio instruction while viewing a moving target, Proceedings of the ACM Symposium on Eye Tracking Research and Applications - ETRA 2000, (New York: ACM, 2000), 131-138. 9. Zhai, S., Morimoto, C., and Ihde, S., Manual gaze input cascaded (MAGIC) pointing, Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI '99, (New York: ACM, 1999), 248-253. 10. Zhang, X. and MacKenzie, I. S., Evaluating eye tracking with ISO 9241 -- Part 9, Proceedings of HCI International 2007, (Heidelberg: Springer, 2007), 779-788. 11. Zhang, X., Ren, X., and Zha, H., Improving eye cursor's stability for eye pointing tasks,,, Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI 2008. Florence, Italy: New York:ACM, 2008, 525-534.
  84. 84. The End (almost) but first…
  85. 85. Cheap Advertising <ul><li>I’m giving a Course at CHI 2010 (Atlanta in April) </li></ul><ul><li>Please consider attending, or </li></ul><ul><li>Invite me to give the course (or other talks) at your company </li></ul><ul><li>I’m also available tomorrow to visit, meet, present, demo, discuss, etc. (departing Friday a.m.) </li></ul>Empirical Research Methods for Human-Computer Interaction Put your logo here Put your logo here
  86. 86. Thank You – Questions?

×