Matching Attentional Draw with Utility in Interruption

2,490 views
2,426 views

Published on

This research examines a design guideline that aims to increase the positive perception of interruptions. The guideline advocates matching the amount of attention attracted by an interruption’s notification method (attentional draw) to the utility of the interruption content. Our first experiment examined a set of 10 visual notification signals in terms of their detection times and established a set of three significantly different signals along the spectrum of attentional draw. Our second experiment investigated matching these different signals to interruption content with different levels of utility. Results indicate that the matching strategy decreases annoyance and increases perception of benefit compared to a strategy that uses the same signal regardless of interruption utility, with no significant impact on workload or performance. Design implications arising from the second experiment as well as recommendations for future work are discussed.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,490
On SlideShare
0
From Embeds
0
Number of Embeds
48
Actions
Shares
0
Downloads
90
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Matching Attentional Draw with Utility in Interruption

    1. 1. Matching Attentional Draw with Utility in Interruption Jennifer Gluck, Andrea Bunt, Joanna McGrenere University of British Columbia CHI 2007 April 30, 2007
    2. 2. Matching Attentional Draw with Utility in Interruption? <ul><li>Attentional Draw (AD): </li></ul><ul><ul><li>How quickly attention is drawn to a notification </li></ul></ul><ul><li>Utility: </li></ul><ul><ul><li>Usefulness of the interruption content </li></ul></ul><ul><li>Matching: </li></ul><ul><ul><li>Low AD ↔ Low Utility </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><ul><li>High AD ↔ High Utility </li></ul></ul>
    3. 3. Interruption <ul><li>When something causes a break in our actions, activities, or concentration. </li></ul>
    4. 4. Negative Effects <ul><li>Lowered primary task performance </li></ul><ul><ul><li>Cutrell, Czerwinski, & Horvitz 2001 </li></ul></ul><ul><ul><li>Gillie & Broadbent 1989 </li></ul></ul><ul><li>Increased anxiety, annoyance, perceived </li></ul><ul><li>difficulty in completing primary task </li></ul><ul><ul><li>Bailey, Konstan, & Carlis, 2001 </li></ul></ul>
    5. 5. Positive Aspects <ul><li>Technologies that interrupt are popular </li></ul><ul><li>suggests there is value to interruption </li></ul><ul><li>Other systems also have the potential to promote beneficial interruption… </li></ul>
    6. 6. <ul><li>Assist users by making suggestions that can help users complete a task </li></ul><ul><ul><li>FlexExcel (Thomas & Krogseoter, 1993) </li></ul></ul><ul><ul><li>Adaptive Bar (Debevc, Meyer, Donlagic & Svecko 1996) </li></ul></ul>Mixed-Initiative Systems How can we design interruption to emphasize positive aspects?
    7. 7. Design Guideline <ul><li>Make the amount of attention attracted by a notification relative to the usefulness of the interruption content </li></ul><ul><li>Obermayer & Nugent, 2000 </li></ul><ul><li>McFarlane & Latorella, 2002 </li></ul>
    8. 8. What did we do? <ul><li>Empirical investigation of the effects </li></ul><ul><li>of matching attentional draw (AD) of </li></ul><ul><li>notification and interruption utility </li></ul>
    9. 9. Attentional Draw (AD) <ul><li>Empirical investigation of the effects </li></ul><ul><li>of matching attentional draw (AD) of </li></ul><ul><li>notification and interruption utility </li></ul><ul><li>How quickly your attention is drawn to the notification signal </li></ul>attentional draw
    10. 10. Utility <ul><li>Empirical investigation of the effects </li></ul><ul><li>of matching attentional draw (AD) of </li></ul><ul><li>notification and interruption utility </li></ul><ul><li>How useful, important, or urgent the interruption content is to the recipient </li></ul><ul><li>“ How helpful the interruption is in terms of performing a primary task” </li></ul>utility
    11. 11. Findings <ul><li>Reduces negative effects </li></ul><ul><li>Facilitates positive perception of interruption </li></ul>Empirical investigation of the effects of matching attentional draw (AD) of notification and interruption utility
    12. 12. Research Overview <ul><li>Study 1: Find some signals </li></ul><ul><li>Established a set of significantly different notification signals in terms of their attentional draw (AD) </li></ul><ul><li>Study 2: Match the signals to utility </li></ul><ul><li>Investigated matching AD and utility to see if this strategy can promote positive interruption </li></ul>
    13. 13. Primary Task
    14. 20. <ul><li>Goal: </li></ul><ul><li>Establish a set of visual notification signals with significantly different detection times </li></ul><ul><li>Approach: Concurrent detection task </li></ul><ul><ul><li>Play Memory game </li></ul></ul><ul><ul><li>Detect notification signals </li></ul></ul>Study 1
    15. 21. Notification Signals <ul><li>10 signals : </li></ul><ul><ul><li>Transformations applied to a base icon </li></ul></ul><ul><li>Properties explored </li></ul><ul><ul><li>Colour </li></ul></ul><ul><ul><li>Motion </li></ul></ul><ul><ul><li>Location </li></ul></ul>
    16. 22. Signals Time (seconds)
    17. 23. Study 1 Design <ul><li>Within-subjects </li></ul><ul><li>12 subjects </li></ul><ul><li>10 signals </li></ul><ul><li>120 trials per subject </li></ul>
    18. 24. Study 1 Results: Detection Times High AD signal: Follow Med AD signal: Slow Zoom Low AD signal: Flag
    19. 25. <ul><li>Goal: </li></ul><ul><li>Investigate the effects of matching attentional draw and utility </li></ul><ul><ul><li>In terms of: </li></ul></ul><ul><ul><li>Annoyance </li></ul></ul><ul><ul><li>Perceived benefit </li></ul></ul><ul><ul><li>Performance </li></ul></ul>Study 2
    20. 26. Interrupting Task <ul><li>Context-sensitive hints </li></ul><ul><li>3 hint utility levels: </li></ul><ul><ul><li>very helpful </li></ul></ul><ul><ul><li>somewhat helpful </li></ul></ul><ul><ul><li>not helpful </li></ul></ul><ul><li>Emulated a mixed-initiative system </li></ul>
    21. 28. 3 Game Conditions <ul><li>Match </li></ul><ul><ul><li>Match AD to utility </li></ul></ul><ul><li>Static </li></ul><ul><ul><li>Same AD for all </li></ul></ul><ul><ul><li>utilities </li></ul></ul><ul><li>Control </li></ul><ul><ul><li>No interruption </li></ul></ul>Low utility hint ↔ Low AD signal Med utility hint ↔ Med AD signal High utility hint ↔ High AD signal   Low utility hint Med utility hint ↔ Med AD signal High utility hint
    22. 29. Study 2 Design <ul><li>Conditions: Match, Static, Control </li></ul><ul><li>Within-subjects design </li></ul><ul><li>24 subjects </li></ul><ul><li>15 interruptions per condition </li></ul>
    23. 30. Study 2 Measures & Procedure <ul><li>Procedure: </li></ul><ul><ul><li>Training, {Condition, Survey} x 3, Interview </li></ul></ul><ul><li>Measures: </li></ul><ul><ul><li>Annoyance </li></ul></ul><ul><ul><li>Perceived benefit </li></ul></ul><ul><ul><li>Fatigue </li></ul></ul><ul><ul><li>Performance (# of matches) </li></ul></ul><ul><ul><li>Preference </li></ul></ul>
    24. 31. Study 2 Hypotheses <ul><li>H1 : Interruption annoyance is lower in the Match </li></ul><ul><li>condition than in the Static condition </li></ul><ul><li>H2 : Perceived benefit is higher in the Match </li></ul><ul><li>condition than in the Static condition </li></ul><ul><li>H3 : Performance is higher in the Match condition </li></ul><ul><li>than in all other conditions </li></ul>
    25. 32. Results: Annoyance & Benefit <ul><ul><li>Annoyance lower in Match ( p = .034) </li></ul></ul><ul><ul><li>Benefit higher in Match ( p = .037) </li></ul></ul>
    26. 33. Condition Preference
    27. 34. Performance Results (# of matches) <ul><li>No main effect of condition ( p = .454) </li></ul><ul><li>Other effects present </li></ul><ul><ul><li>Condition x Presentation, Fatigue, Learning, </li></ul></ul><ul><li> Performance results unclear </li></ul>
    28. 35. Study 2: Summary of Results <ul><li>H1 : Interruption annoyance is lower in </li></ul><ul><li>the Match condition than in the Static condition </li></ul><ul><li>H2 : Perceived benefit is higher in the </li></ul><ul><li>Match condition than in the Static condition </li></ul><ul><li>H3 : Performance is higher in the Match condition </li></ul><ul><li>than in all other conditions </li></ul>  
    29. 36. Future Research <ul><li>Computational appraisal of utility </li></ul><ul><li>Notification signals </li></ul><ul><li>Scope of utility </li></ul><ul><ul><li>Primary task relevance: </li></ul></ul><ul><ul><ul><li>Re-examine performance results </li></ul></ul></ul><ul><ul><li>Personal relevance: </li></ul></ul><ul><ul><ul><li>Confirm annoyance and benefit results </li></ul></ul></ul>
    30. 37. Take Home Message <ul><li>Matching AD and utility </li></ul><ul><ul><li>Reduces annoyance and </li></ul></ul><ul><ul><li>Increases perception of benefit </li></ul></ul><ul><li>UI design implications? </li></ul>
    31. 38. Acknowledgements <ul><li>Many thanks to: </li></ul><ul><li>Natural Science and Engineering Research Council of Canada (NSERC) </li></ul><ul><li>Lyn Bartram </li></ul>
    32. 39. <ul><li>Matching AD and utility </li></ul><ul><ul><li>Reduces annoyance and </li></ul></ul><ul><ul><li>Increases perception of benefit </li></ul></ul>Take Home Message Questions?
    33. 40. Additional Information
    34. 41. Interview Results <ul><li>Comprehension of AD-utility relationship </li></ul><ul><ul><li>Low 70% </li></ul></ul><ul><ul><li>Med 40% </li></ul></ul><ul><ul><li>High 45% </li></ul></ul><ul><ul><li>All 40% </li></ul></ul><ul><ul><li>None 25% </li></ul></ul><ul><li>40% utilized this knowledge to ignore low-utility hints </li></ul>
    35. 42. Related Work <ul><li>Coordination </li></ul><ul><ul><li>McFarlane 2002 </li></ul></ul><ul><li>Timing of Onset </li></ul><ul><ul><li>Fogarty et al. 2004 </li></ul></ul><ul><ul><li>Bailey et al. 2001 </li></ul></ul><ul><li>Peripheral Awareness </li></ul><ul><ul><li>Maglio & Campbell 2000 </li></ul></ul><ul><ul><li>McCrickard et al. 2003 </li></ul></ul><ul><li>Notification </li></ul><ul><ul><li>Bartram, Ware, Calvert 2003 </li></ul></ul><ul><ul><li>Horvitz, Kadie, Paek, Hovel 2003 </li></ul></ul><ul><li>Relevance </li></ul><ul><ul><li>Czerwinski, Cutrell, Horvitz 2000 </li></ul></ul><ul><li>Utility </li></ul><ul><ul><li>Horvitz, Koch, Apacible 2004 </li></ul></ul><ul><ul><li>Horvitz, Jacobs, Hovel 1999 </li></ul></ul><ul><ul><li>Avrahami & Hudson 2005 </li></ul></ul><ul><ul><li>Gievska & Sibert 2005 </li></ul></ul>
    36. 43. Study 1: Simple Editor Task
    37. 44. Study 1 Design <ul><li>Within-subjects: 2 (task) x 10 (signal) x 3 (block) </li></ul><ul><li>Block Design </li></ul><ul><ul><li>2 replications per block </li></ul></ul><ul><ul><li>3 blocks per primary task (6 total) </li></ul></ul><ul><ul><li>120 trials per subject </li></ul></ul><ul><li>Measures </li></ul><ul><ul><li>Detection time </li></ul></ul><ul><li>12 subjects, 90 minute experiment </li></ul>
    38. 45. Study 1: Structure of a Trial
    39. 46. Study 1 Results <ul><li>Significant main effect of signal ( p < .001) </li></ul><ul><li>3-way significant comparison between Flag, Slow Zoom, and Follow </li></ul>
    40. 47. Study 1: Pairwise Comparisons
    41. 48. Study 1: Timeouts Main effect of signal (F(2.542,27.961) = 3.630, p = .031, η 2 = .248)
    42. 49. Study 2 Design <ul><li>Conditions: Match, Static, Control </li></ul><ul><li>Within-subjects design </li></ul><ul><li>5 replications in 17-min block (i.e. 15 interruptions per condition) </li></ul><ul><li>65-second average interruption frequency </li></ul><ul><li>24 subjects </li></ul><ul><li>1 hour experiment </li></ul>
    43. 50. Study 2: Interruption Structure
    44. 51. Missed-Hints Dialog
    45. 52. Study 2 General Annoyance No main effect of condition ( F (2,24) = 2.788, p = .079, η 2 = .166, power = .429)
    46. 53. Study 2: Presentation Order Effect F (5,14) = 2.720, p = .064, η 2 = .493
    47. 54. Study 2 Interaction Effect F (10,28) = 2..35, p = .068, η 2 = .421
    48. 55. Study 2 Hint Duration Main effect of utility ( F (2,36) = 6.839, p = .003, η 2 = .275)
    49. 56. Study 2 Workload .143 .115 2.335 Physical Demand .027 .687 .381 Frustration .088 .276 1.347 Perceived Performance .008 .889 .118 Effort .071 .357 1.069 Temporal Demand .004 .945 .057 Mental Demand η 2 p F(2,28) NASA-TLX Factor
    50. 57. NASA Task Load Index <ul><li>Mental Demand </li></ul><ul><li>How much mental and perceptual activity was required to play the game and attend to the hints? (e.g., thinking, remembering, looking, searching, deciding, etc.)? </li></ul><ul><li>Physical Demand </li></ul><ul><li>How much physical activity was required to play the game? (e.g. moving the mouse, clicking the mouse button, etc.) </li></ul><ul><li>Temporal Demand </li></ul><ul><li>How much time pressure did you feel due to the rate or pace at which the tasks or task elements occurred? </li></ul>
    51. 58. NASA Task Load Index (2) <ul><li>Effort </li></ul><ul><li>How hard did you have to work (mentally and physically) to accomplish your level of performance? </li></ul><ul><li>Perceived Performance </li></ul><ul><li>How successful do you think you were in accomplishing the goals of the task set by the experimenter (or yourself)? </li></ul><ul><li>Frustration </li></ul><ul><li>How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent did you feel during the task? </li></ul>
    52. 59. Performance Results <ul><li>No main effect of condition ( p = .454) </li></ul><ul><li>Main effect: presentation order ( p = .064, η 2 = .493) </li></ul><ul><ul><li>Interaction effect: condition & presentation order ( p = .068, η 2 = .421) </li></ul></ul><ul><li>Learning effect ( p = .053) </li></ul><ul><li>Fatigue effect ( p = .009) </li></ul><ul><li>Impossible to interpret performance results </li></ul>

    ×