More Related Content
Similar to PupilWare Petra 2015
Similar to PupilWare Petra 2015 (20)
More from Eric Larson
More from Eric Larson (20)
PupilWare Petra 2015
- 1. eric c. larson | eclarson.com PupilWare Assistant Professor Computer Science and Engineering towards pervasive cognitive load measurement using commodity devices
- 4. Pupillometry Primer macro changes: light reflex drug impairment concussions
- 5. sclerairispupil pupil: regulate light autonomic nervous system physiological response to stress, arousal Pupillometry Primer
- 6. Subtle Pupillary Response 1940 1950 1960 1970 1980 1990 2000 2010 2015 1943 Blood alcohol proportional to pupil size 1947 First scientific studies of light reflex 1950 Pupil size decreases with age 1959 Pupil size projector invented, by Hess and Polt 1960 Pupil size and visual stimuli 1961-1969 Indicator of preference, fatigue 1966 Pupil size & memory 1967 Pupil size & processing difficulty 1968 Pupil size maintained during thought 1983 First digital device for capturing pupil size 1989 Apparatus for desktop imaging of pupil 1995 First handheld digital pupillometer 2005-2012 Validation of eye trackers for pupil measurements 2008 Framework for measuring digital interruption cost
- 7. 1940 1950 1943 Blood alcohol proportional to pupil size 1947 First scientific studies of light reflex 1950 Pupil size decreases with age Skoglund, 1943 Thompson 1947 & Delauney 1949 Birren, 1950
- 8. 1950 19 udies x 1950 Pupil size decreases with age 1959 Pupil size projector invented, by Hess and Polt Hess and Polk, 1959-1960 12 mm 4 mm 4.2 mm
- 9. 1960 59 Pupil size projector ented, by Hess and lt 1960 Pupil size and visual stimuli preference, f Hess and Polk, 1959-1960 Hess and Polk, 1960
- 10. 1960 1970 ctor d 1960 Pupil size and visual stimuli 1961-1969 Indicator of preference, fatigue • schizophrenia and neurotic diagnosis (Rubin 1964) • sexual orientation (Hess 1965) • political preference (Hess 1965) • pleasant taste (Hess 1965) • infant object recognition and preferential looking (Fitzgerald et al.1967) • musical pitch interpretation (Kahneman and Beatty 1967) • fatigue, alertness, and sleep deprivation (Lowenstein and Loewenfeld 1964, Bartlett et al. 1967) Consensus: -pupil dilation indicated something pleasing -not a strict measure of emotion -related but not proportional to arousal
- 11. 1966 Pupil size & memory 1967 Pupil size & processing difficulty 1968 Pupil size maintained during thought Kahneman and Beatty, 1966 Hess and Polk, 1967 Bradshaw, 1968 Eventually the processing of the brain and use of working memory came to be understood as cognitive load
- 12. 1966 Pupil size & memory 1967 Pupil size & processing difficulty 1968 Pupil size maintained during thought Kahneman and Beatty, 1966 Hess and Polk, 1967 Bradshaw, 1968 Eventually the processing of the brain and use of working memory came to be understood as cognitive load klingner, 2012 Digit Span Task
- 13. 1960 1970 1980 ctor d 1960 Pupil size and visual stimuli 1961-1969 Indicator of preference, fatigue 1966 Pupil size & memory 1967 Pupil size & processing difficulty 1968 Pupil size maintained during thought 1983 First digital device for capturing pupil size
- 14. 1980 1990 1983 First digital device for capturing pupil size 1989 Apparatus for desktop imaging of pupil Jones and Smith, 1983 Carter, 1989
- 15. 2000 1995 First handheld digital pupillometer mea 2 Carter, 1995 $10,000 USD $4,000 USD
- 16. 2000 2010 2005-2012 Validation of eye trackers for pupil measurements 2008 Framework for Klingner et al., 2005-2012 $500+ USD
- 17. 2010 20152008 Framework for measuring digital interruption cost Iqbal and Bailey, 2008 Use of cognitive load for real time user awareness is becoming more of a possibility
- 18. can cognitive load be measured ubiquitously?
- 19. the webcam ~15 x 15 pixels dilations are mostly sub-pixel highly affected by noise and lighting HD 1280x720
- 20. Study Can baseline cognitive load be measured, compared to a gold standard pupillometer? Can subtle variations be measured, compared to a remote eye tracker? Age: (mean=23, range=19-38) Sex: 7 Male, 5 Female Color: Lighter Eyes: 7, Brown: 5 Vision: Corrected-to-normal, 2 12 participants
- 21. Experimenter Laptop Control View Participant Camera iPad Application Survey & Self Report Remote Gaze Tracker Focal point Calipers measurement Pupillometer Measurement DesignStudy
- 22. 5 Digits 6 Digits 7 Digits 8 Digits 9 Digits x 5 iterations over 200 iterations total 2 7 9 3 13 Digit Span Task
- 26. darkness threshold morphology means of gradient
- 27. iris bounding box histogram equalization median filtering
- 29. modified starburst algorithm find strong edges ~45x45
- 30. eliminate measurements with RANSAC fit ellipse to points calculate distance between pupil centers
- 33. Post Processing 0 10 20 30 40 50 60 15 16 14 17 13 seconds ellipsediameter,inpixels
- 34. Post Processing 0 10 20 30 15 16 14 13 seconds ellipsediameter,inpixels average
- 35. Post Processing 0 10 20 30 15 16 14 13 seconds ellipsediameter,inpixels 12 11 10 average median smooth
- 36. Post Processing 0 10 20 30 seconds 5.0 5.2 4.8 4.6 ellipsediameter,inmm 4.4 4.2 4.0 5.4 average median smooth millimeters
- 37. average median smooth millimeters Light Eyes Dark Eyes Reflection 3 participants removed from analysis
- 38. Baseline Results 3.00 4.00 5.00 6.00 7.00 P1 P2 P3 P4 P5 P6 P7 P8 P9 Pupillometer PupilWare GZ-C Ŧ = GZ-C and PupilWare different (p<0.05) Ŧ Ŧ * = different from pupillometer (p<0.05) * * * * * * * * * * baselinepupilsize,millimeters
- 39. Digit Span Tasks Time 20% 10% 0% -10% -20% 5s 10s 15s 20s 25s 30s Gaze Tracker PupilWare PercentagePupilDilation Baseline Period Listening Period Speaking Period Relaxation Period mean percent difference < 4% 90th percentile < 9% difference
- 40. 20% 15% 10% 5% 0% 0s 5s 10s 15s 20s 25s 30s 0s 5s 10s 15s 20s 25s 30s Aggregate Results PercentagePupilDilation Time Area = Interquartile Range GazeTracker PupilWareBaseline 8 digit 5 digit 6 digit 7 digit Kahneman and Beatty, 1968
- 41. Per Trial Correlation 1.0 0.8 0.6 0.4 0.2 0.0 -0.2 -0.4 P1 P2 P3 P4 P5 P6 P7 P8 P9 RankOrderCorrelation GazeTrackervs.PupilWare Participant
- 42. Conclusions and Future Research +about as accurate as gaze tracking +within 0.5mm of pupillometer -dark eyes, reflection -complex stimuli -privacy -off axis eye and head pose -accounting for light reflex i.e., screen brightness changes ubiquitous?
- 43. markers of pain sympathetic nerve damage head injury context aware computing attention Future Work and Applications fatigue and sleep deprivation comprehension and cognitive disability
- 44. PupilWare towards pervasive cognitive load measurement using commodity devices Thank You!
- 45. eric c. larson | eclarson.com Assistant Professor Computer Science and Engineering eclarson.com eclarson@smu.edu @ec_larson PupilWare towards pervasive cognitive load measurement using commodity devices Suku NairSohail Rafiqi Chatchai Wangwiwattana Ephrem Fernandez Jasmine Kim
- 47. Back Up Graphs
- 48. Back Up Graphs