1. blindSight: Eyes-free mobile phone interaction Kevin Li , University of California, San Diego Patrick Baudisch , Microsoft Research Ken Hinckley , Microsoft Research

2. calendar preview “ Monday 9am” “ tic, tic, sssssh” “ How ab out Monday morning?” “ Yeah, looks like I’m free after 10” blindSight

4. why?

5. PCs… PC screens have the users’ undivided attention  design for the visual channel

6. eyePhone

7. environment

8. visual impairment

9. screen-less device

10. can’t see screen

13. Lots of information is stored on mobile phones…

14. … the interfaces are visual

15. # of participants “ I need to access as part of a phone conversation:” survey

17. Please listen carefully as our options have changed…

18. related work

21. blindSight’s auditory feedback

22. audio is heard only by the user, not by the person at the other end

23. rationale  people can recover from audio interruptions as long as interruption is short human-human conversation contains redundancy can we use this redundancy to inject auditory feedback from the device?

25. rules 1. feedback only on-demand hear voice note mute speaker phone hear task list add contact record voice find contact cal endar hear emails hear text message home

28. rules 2. brevity 2 1 3 8 delete 4 9 7 6 5 abc def tuv ghi wxyz pqrs mno jkl next play find contact type 6 “ 200 hits” type 2 “ 12 hits” type 7 “ Marion”

29. rules 3. non-speech previews of composites week week day day 3 hours 3 hours ½ hour ½ hour block ½h preview day + – – – + _ whereAmI go today preview 3 hours ++ calendar (what if the content is a long list, such as appointments for a day?)

32. rules 4. decomposition week week day day 3 hours 3 hours ½ hour ½ hour next preview day + – – – + _ whereAmI go today preview 3 hours (what if the content is a long list, such as appointments for a day?)

33. rules 5. interruptability user interface runs as a separate thread

34. rules 6. minimize modes mon wed tue sat thu fri sun action delete save pick day start time end time … 1 3 2 6 4 5 9 7 8 action delete save 1 3 2 6 4 5 9 7 8 action delete save modes

35. rules 6. minimize modes (  avoid wizards) week week day day 3 hours 3 hours ½ hour ½ hour block ½h preview day + – – – + _ whereAmI go today preview 3 hours ++ + +

36. home hear voice note mute speaker phone hear task list add contact record voice find contact cal endar hear emails hear text message

37. add contact 2 1 3 8 save 4 9 7 6 0 5 delete

38. iterator 1 3 2 6 4 5 9 7 8 action action delete save menu patterns

39. calendar week week day day 3 hours 3 hours ½ hour ½ hour block ½h preview day + – – – + _ whereAmI go today preview 3 hours ++ + +

43. hardware

47. space

48. epoxy dots enlarged spaces

49. error Flip Ear Visual

51. 1 3 2 6 4 5 9 7 8 # * 0 .

52. error Flip Ear Visual

53. blindSight evaluation

54. interfaces Smartphone 2003 (sighted) BlindSight (eyes-free) vs.

55. task while “driving” idle (1) schedule appointments and (2) add contacts

56. Overall preference results 0 1 2 3 4 5 6 7 8 Was not missing information Knew position in the menu Knew what day/time I was at Felt in control of the conversation Better for setting meeting times Prefer if driving and talking Prefer Overall blindSight Smartphone

57. 1. brevity is good, but use in moderation clarification of navigation overrides brevity 2. predictable/modeless user interface is key 3. auditory feedback goes a long way even during phone call (disclaimer: need to study how it interferes with activities… driving) lessons

58. environment visual impairment can’t see screen screen-less device next:

59. eyePhone

60. eyePhone eyes Free Phone ?

61. blindSight: Eyes-free mobile phone interaction Kevin Li , University of California, San Diego Patrick Baudisch , Microsoft Research Ken Hinckley , Microsoft Research

63. extra slides

67. iterator 1 3 2 6 4 5 9 7 8 action action delete save menu patterns

68. home hear voice note mute speaker phone hear task list add contact record voice find contact cal endar hear emails hear text message

69. add contact 2 1 3 8 save 4 9 7 6 0 5 delete

70. iterator 1 3 2 6 4 5 9 7 8 action action delete save menu patterns

71. calendar week week day day 3 hours 3 hours ½ hour ½ hour block ½h preview day + – – – + _ whereAmI go today preview 3 hours ++ + +

72. hear voice note mute speaker phone hear task list add contact record voice find contact cal endar hear emails hear text message 2 1 3 8 save 4 9 7 6 0 5 delete week week day day 3 hours 3 hours ½ hour ½ hour block ½h preview day + – – – + _ whereAmI go today preview 3 hours ++ 2 1 3 8 delete 4 9 7 6 5 abc def tuv ghi wxyz pqrs mno jkl next play add contact find contact calendar home type type folder folder n items n items item item play + – – – + _ + preview + email, tasks, voice, SMS home help hold bottom left for hold bottom right for menu

73. … is a phenomenon in which people who are perceptually blind in a certain area of their visual field demonstrate some visual awareness, without any qualitative experience blindSight... [wikipedia]

74. don’t mode me in blind sight 10 design rules to allow eyes-free use and flow tactile features

75. phones… … are in in a mobile situation If they requires visual attention, users will fail at their current activity interference with social activities drive off the road…

77. vs. baseline interfaces

78. task times