Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1
Shengdong Zhao
Department of Computer Science
University of Toronto
July 9, 2008
earPod: Efficient, Hierarchical,
Eyes-f...
Outline
• Research problem
• Previous research
• Research questions
• Research strategy and focus
• Prototype
• Empirical ...
Research problems and objectives
• Motivating problem
– How can users interact with information devices in mobile
environm...
Previous research
• Visual menus
– Linear menus (e.g., Sear and Schneiderman ’94; Cockburn and Gin ‘06, etc.)
– Radial men...
Research questions
• How to design usable eyes-free menus?
– Designing interfaces with reasonable levels of
efficiency and...
Research strategy
• Define research problem
• Review literature
• Define general design concept
• Perform iteratively desi...
Selected input and output
modalities
• Touch input
– Combines relative and absolute pointing (Hinckley ‘01)
– Can be robus...
earPod prototype
• A touch-sensitive device with software to
perform eyes-free, hierarchical menu selection
• earPod proto...
earPod Prototype
9
earPod interaction
10
Novice Intermediate
Intermediate Expert
Video - browsing
11
Video - fast browsing
12
Video - very fast browsing
13
Video - expert use
14
Prototype design
• Efficiency
– By-pass audio
– Use interruptible audio feedback
– Use non-speech audio
• Sense of control...
Empirical studies
Motivating questions:
1. How does earPod compare with a popular visual menu
selection technique used by ...
Evaluation issues
• Modality
– Visual
– Audio
– Dual
• Menu style
– Linear
– Radial
17
Menu Style
Radial (absolute) Linear...
4 empirical studies
• 4 empirical studies were conducted at UofT in 2006-08
• Experiment 1: earPod vs. iPod (audio radial ...
Experimental 1 design
• Goal: compare earPod with iPod-like linear
menu
• 60-90 minute session, within-subject, counter-
b...
Results: response time
Block number Block number
20
Results: response time
Block number Block number
21
Results: response time
Block number Block number
22
Experiment 1 - results and
discussion
• earPod has comparable performance with
iPod-like linear menu
• earPod user perform...
Experiment 2 design
• Goal: systematic investigation of modality of
feedback and menu style
• 60-minute, within subject, c...
25
Response time
Audio
radial
Audio
linear
Visual
radial
Visual
linear
Audio visual
radial
Audio visual
linear
Experiment 2 - results and
discussion
• Accuracy
– Comparable performance among all techniques
• Speed
– visual radial ~ d...
Experiment 3: longitudinal study
Goal: learning of earPod and related techniques
5 one-hour sessions in a week, between-su...
Accuracy
28
0.88
0.9
0.92
0.94
0.96
0.98
1
1 2 3 4 5
Audio linear Audio radial Visual linear Visual radial
Day
Response time
29
0
1
2
3
4
5
6
1 2 3 4 5
Audio linear Audio radial Visual linear Visual radial
Day
Gliding to tapping transition
30
Items explored Tapping percentage
0
1
2
3
4
5
6
7
1 2 3 4 5
Audio linear Audio radial Vis...
Experiment 3 - results and
discussion
• Novice performance is determined by
modality, expert performance is determined
by ...
Experiment 4: dual-task study
Goal:
Investigate the properties of using earPod and
the related techniques as a secondary t...
Experiment 4: dual-task study
• Dual-task
– Primary task: simulated driving
– Secondary task: menu selection
• External va...
Experiment 4 design
• Conditions
– Desktop
• 12 participants X 6 techniques X 8 items of 1 menu configurations X (3
+ 1) b...
Experiment 4 setup
35
Response time: modality
36
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
1 2 3
Desktop
Driving
Audio Visual Dual
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
1 2
Desktop
Driving
Response time: menu style
37
PRadial Linear
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Audio Visual Dual
LateralVelocity(m/s) Lateral velocity
38
0
10
20
30
40
50
60
70
80
Audio Visual Dual
FollowingDistance(m) Following distance
39
Experiment 4 - results and
discussion
• Audio techniques work better with a visually
demanding primary task
– Comparable p...
Summary of empirical studies
• earPod has comparable performance with
visual linear menus
• Transition from novice to expe...
Contributions
• Developed a novel interaction technique
called earPod
• Conducted empirical evaluations on earPod
and rela...
Future work
• Explore eyes-free menu techniques for long
and dynamic lists
• Investigate the role of compression and
dicho...
Questions
44
45
The End
Upcoming SlideShare
Loading in …5
×

of

earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 1 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 2 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 3 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 4 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 5 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 6 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 7 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 8 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 9 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 10 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 11 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 12 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 13 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 14 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 15 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 16 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 17 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 18 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 19 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 20 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 21 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 22 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 23 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 24 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 25 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 26 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 27 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 28 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 29 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 30 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 31 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 32 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 33 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 34 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 35 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 36 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 37 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 38 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 39 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 40 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 41 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 42 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 43 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 44 earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback Slide 45
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0 Likes

Share

Download to read offline

earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback

Download to read offline

Authors: Shengdong Zhao, Pierre Dragicevic, Mark Chignell, Ravin Balakrishnan, Patrick Baudisch

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

earPod: Eyes-free Menu Selection using Touch Input and Reactive Audio Feedback

  1. 1. 1 Shengdong Zhao Department of Computer Science University of Toronto July 9, 2008 earPod: Efficient, Hierarchical, Eyes-free Menu Selection
  2. 2. Outline • Research problem • Previous research • Research questions • Research strategy and focus • Prototype • Empirical studies • Contributions • Future work 2
  3. 3. Research problems and objectives • Motivating problem – How can users interact with information devices in mobile environments when their visual attention is taken up by other tasks (e.g navigation through the environment) • Objective – To develop interactive techniques that support user control and feedback without relying on the visual modality • Scope – This research focuses on menu selection because this is an elemental task (Foley et al. ‘84) 3
  4. 4. Previous research • Visual menus – Linear menus (e.g., Sear and Schneiderman ’94; Cockburn and Gin ‘06, etc.) – Radial menus (e.g., Callahan et al. ‘88; Kurtenbach ’93, etc.) • Auditory applications – Accessibility for the blind (e.g., Edwards ‘89; Mynatt and Weber ‘94, etc.) – Mobile auditory interfaces (e.g., Roy and Schmandt ‘96; Pirhonen et al. ’02, etc.) • Auditory menus – Interactive Voice Response (IVR) systems (e.g., Marics and Engelbeck 1997; Resnick and Virzi ’92, etc.) – Head gesture auditory pie menu (Brewster et al. ‘03) 4
  5. 5. Research questions • How to design usable eyes-free menus? – Designing interfaces with reasonable levels of efficiency and ease of use – Facilitating a smooth novice to expert transition • How to work around the problems with serial and temporal audio feedback? – Facilitating scan and compare of menu items – Providing users with a sense of control and responsiveness 5
  6. 6. Research strategy • Define research problem • Review literature • Define general design concept • Perform iteratively design and pilot studies for prototype • Conduct empirical studies to explore prototype properties • Derive design recommendations based on empirical results 6
  7. 7. Selected input and output modalities • Touch input – Combines relative and absolute pointing (Hinckley ‘01) – Can be robustly operated in motion (Buxton et al. ‘85) – Can be combined with other input devices (Hinckley and Sinclair ‘99; Rekimoto et al. ‘03) • Auditory feedback – The other primary remote sense (Gaver ‘97) – The only other sense with a natural language* 7
  8. 8. earPod prototype • A touch-sensitive device with software to perform eyes-free, hierarchical menu selection • earPod prototype used commercial touchpad augmented with custom-built plastic cover, with cable to computer for voice output • Built in University of Toronto in 2006 with Pierre Dragicevic 8
  9. 9. earPod Prototype 9
  10. 10. earPod interaction 10 Novice Intermediate Intermediate Expert
  11. 11. Video - browsing 11
  12. 12. Video - fast browsing 12
  13. 13. Video - very fast browsing 13
  14. 14. Video - expert use 14
  15. 15. Prototype design • Efficiency – By-pass audio – Use interruptible audio feedback – Use non-speech audio • Sense of control – Synchronous communication (reactive audio feedback) • Simplicity and power – Gliding and tapping • Easy to learn – Use spatial audio to reinforce mental model – Self-discoverable transition 15
  16. 16. Empirical studies Motivating questions: 1. How does earPod compare with a popular visual menu selection technique used by iPod? 2. How does earPod compare with a number of competitor techniques with different modalities and menu styles? 3. What are learning behaviors for earPod and the related techniques? 4. How does earPod and the related techniques perform with a visually demanding primary task? 16
  17. 17. Evaluation issues • Modality – Visual – Audio – Dual • Menu style – Linear – Radial 17 Menu Style Radial (absolute) Linear (relative) Modality Audio Audio radial Audio linear Visual Visual radial Visual linear Dual Audio visual radial Audio visual linear
  18. 18. 4 empirical studies • 4 empirical studies were conducted at UofT in 2006-08 • Experiment 1: earPod vs. iPod (audio radial vs. visual linear) • Experiment 2: 3 x 2 study (3 modalities x 2 menu styles) • Experiment 3: 2 x 2 longitudinal study (audio linear, visual linear, audio radial, visual radial) • Experiment 4: 3 x 2 dual-task study (3 modalities x 2 menu styles) 18
  19. 19. Experimental 1 design • Goal: compare earPod with iPod-like linear menu • 60-90 minute session, within-subject, counter- balanced • Setup: 12 participants x 2 techniques (audio and visual) x (40+80) items for the 2 menu configurations (8 and 8x8) x 4 blocks = 11,520 menu selections in total. 19
  20. 20. Results: response time Block number Block number 20
  21. 21. Results: response time Block number Block number 21
  22. 22. Results: response time Block number Block number 22
  23. 23. Experiment 1 - results and discussion • earPod has comparable performance with iPod-like linear menu • earPod user performance improves more rapidly than that of iPod-like linear menu • With practice, earPod can outperform iPod- like linear menu 23
  24. 24. Experiment 2 design • Goal: systematic investigation of modality of feedback and menu style • 60-minute, within subject, counter-balanced • Setup 12 participants x 6 techniques x 8 menu items x 13 blocks (12 blocks + 1 practice block) = 7488 menu selections in total 24
  25. 25. 25 Response time Audio radial Audio linear Visual radial Visual linear Audio visual radial Audio visual linear
  26. 26. Experiment 2 - results and discussion • Accuracy – Comparable performance among all techniques • Speed – visual radial ~ dual radial < audio radial ~ visual linear ~ dual linear < audio linear • Modality – Visual ~ dual < audio • Menu style – Radial < linear 26
  27. 27. Experiment 3: longitudinal study Goal: learning of earPod and related techniques 5 one-hour sessions in a week, between-subject Setup: 2 participants (per technique) x 4 techniques x 64 items for 1 menu configuration (8 x 8) x 7 blocks per day x 5 days = 17920 menu selections in total 27
  28. 28. Accuracy 28 0.88 0.9 0.92 0.94 0.96 0.98 1 1 2 3 4 5 Audio linear Audio radial Visual linear Visual radial Day
  29. 29. Response time 29 0 1 2 3 4 5 6 1 2 3 4 5 Audio linear Audio radial Visual linear Visual radial Day
  30. 30. Gliding to tapping transition 30 Items explored Tapping percentage 0 1 2 3 4 5 6 7 1 2 3 4 5 Audio linear Audio radial Visual linear Visual radial 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 2 3 4 5 Audio radial Visual radial Day Day
  31. 31. Experiment 3 - results and discussion • Novice performance is determined by modality, expert performance is determined by menu style • Rapid learning is observed for earPod • Users can perform tapping for 70-80% of the 64 menu items from day 3 onwards • Audio linear has comparable performance with visual linear from day 3 onwards 31
  32. 32. Experiment 4: dual-task study Goal: Investigate the properties of using earPod and the related techniques as a secondary task in a dual-task setting and their impact on the visually demanding primary task 32
  33. 33. Experiment 4: dual-task study • Dual-task – Primary task: simulated driving – Secondary task: menu selection • External validity: similar settings have been successfully used by Salvucci et al. ‘01, ‘02, ‘05 33
  34. 34. Experiment 4 design • Conditions – Desktop • 12 participants X 6 techniques X 8 items of 1 menu configurations X (3 + 1) blocks = 2304 menu selections – Driving • 12 participants X 6 techniques X 8 items of 1 menu configurations X (1 + 1) blocks = 1152 menu selections • Measures – Desktop • Speed & accuracy of menu selection – Driving • Speed & accuracy of menu selection (secondary task) • Driving performance (primary task) 34
  35. 35. Experiment 4 setup 35
  36. 36. Response time: modality 36 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 1 2 3 Desktop Driving Audio Visual Dual
  37. 37. 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 1 2 Desktop Driving Response time: menu style 37 PRadial Linear
  38. 38. 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Audio Visual Dual LateralVelocity(m/s) Lateral velocity 38
  39. 39. 0 10 20 30 40 50 60 70 80 Audio Visual Dual FollowingDistance(m) Following distance 39
  40. 40. Experiment 4 - results and discussion • Audio techniques work better with a visually demanding primary task – Comparable performance with visual techniques, less impact on primary tasks • Although dual-channel techniques have similar performance with audio techniques, users prefer single channel audio feedback 40
  41. 41. Summary of empirical studies • earPod has comparable performance with visual linear menus • Transition from novice to expert for earPod is fast • Audio techniques work better with a visually demanding primary task • Users only pay attention to the appropriate channel when are presented with both channels of feedback 41
  42. 42. Contributions • Developed a novel interaction technique called earPod • Conducted empirical evaluations on earPod and related techniques • Derived design recommendation for incorporating earPod and related techniques to mobile devices 42
  43. 43. Future work • Explore eyes-free menu techniques for long and dynamic lists • Investigate the role of compression and dichotic listening • Extend earPod to other mobile scenarios (e.g., walking, running, etc.) • Develop eyes-free interaction techniques for other tasks and applications (e.g., drag & drop, text entry, etc.) 43
  44. 44. Questions 44
  45. 45. 45 The End

Authors: Shengdong Zhao, Pierre Dragicevic, Mark Chignell, Ravin Balakrishnan, Patrick Baudisch

Views

Total views

64

On Slideshare

0

From embeds

0

Number of embeds

29

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×