Presentation given at FTW, June 2010.

1. In-Car Speech User Interfaces and their Effects on Driver Cognitive Load Andrew Kun

2. Students and staff @ UNH Oskar Palinko Zeljko Medenica Nemanja Memarovic Jon Oppelaar Alex Shyrokov Puneet Lakhanpal

3. Outline Ubicomp and cars Studies: Police radio SUI characteristics Glove PTT Navigation Conclusion

4. Navigation

5. Music, phones

6. Police cars Toys vs. tools Vanguard of in-car ubicomp

7. Project54

8. Studies Goal: Evaluation tool Improved HCI

9. Cognitive load Measures: Performance Driving Visual attention Physiological Pupil diameter Heart rate Skin conductance Subjective NASA TLX

10. Outline Ubicomp and cars Studies: Police radio SUI characteristics Glove PTT Navigation Conclusion

12. Seeingmachines eye tracker tinyurl.com/eyetracker

13. Visual attention Fixations (> 100 ms): number, timing, etc. Percent dwell time: looking at road

14. Outline Ubicomp and cars Studies: Police radio SUI characteristics Glove PTT Navigation Conclusion

15. Police radio study: SUI vs. hardware

18. Outline Ubicomp and cars Studies: Police radio SUI characteristics Glove PTT Navigation Conclusion

19. SUI characteristics study Secondary task: speech control of radio 2 x 2 x 2 design: SR accuracy: high/low PTT button: yes/no – ambient recognition Dialog repair strategy: mis-/non-understanding

21. Rec. accuracy influences steering

22. Low acc. + PTT influences lane position

23. Outline Ubicomp and cars Studies: Police radio SUI characteristics Glove PTT Navigation Conclusion

24. Glove PTT study Motivation: PTT vs. driving performance Secondary task: speech control of radio 2 x 3 x 3 design: SR accuracy: high/low PTT activation: push-hold-release/push-release/no push PTT button: ambient/fixed/glove

25. Glove PTT study

26. Fixed PTT operation

27. Fixed PTT – glances!

28. Glove PTT operation

29. Glove PTT – few glances

30. Glove PTT transcription

31. Hand position on wheel [degrees]

32. Outline Ubicomp and cars Studies: Police radio SUI characteristics Glove PTT Navigation Conclusion

33. Navigation study Problems: In-car devices vs.driving performance and visual attention Driving performance/visual attention vs. probability of accident

34. Navigation study With vs. without map: Time looking at the road? Effect on driving performance? Voice instructionfor both 8 male subjects

35. Driving simulator

36. 2 Driving scenario 1 2 3 3 1

37. Visual attention Fixations (> 100 ms): number, timing, etc. Percent dwell time: looking at road

38. Driving performance Variance: Lane position Steering wheel angle Velocity Collisions

39. Effect of visual attention? Average variances (13 segments) Cross-correlation peaks: PDT decrease -> variance increase?

41. Camera setup Front camera Eye tracker camera Rear camera

42. Standard PND

43. Voice-only PND

44. Questions Influence on visual attention? If yes, effect on driving performance?

45. PDT on the outside world

46. PDT at world vs. distance

47. PDT at screen vs. distance

48. Questions Influence on visual attention? If yes, effect on driving performance?

49. Cross-correlation

50. Cross-correlation, gazes > 200 ms

51. Fixations at standard PND screen

52. Navigation study conclusions Display: (visual attention)↘ (visual attention)↘ -> (variances↗) (long gazes at PND) -> (variances↗) Display viewing: 60% of gazes last >200ms Collision risk: need testing But…

53. I prefer to have a PND screen…

54. Outline Ubicomp and cars Studies: Police radio SUI characteristics Glove PTT Navigation Conclusion

55. Design implications SUI may improve performance. SUI details matter (SR accuracy, PTT). Must earn users trust. Complicated displays may reduce visual attention even more. Small screen? Voice-only may be just fine.

56. Next? “You’re on the right track.” Complex environments? Other possibilities e.g. augmented reality. Safety?

57. Acknowledgements US DOJ (NIJ, COPS) NSF Microsoft Research TellMe