Matching Attentional Draw with Utility in Interruption

Slide 1: Matching Attentional Draw with Utility in Interruption Jennifer Gluck, Andrea Bunt, Joanna McGrenere University of British Columbia CHI 2007 April 30, 2007 Imager Laboratory 1 for Graphics, Visualization and HCI

Slide 2: Matching Attentional Draw with Utility in Interruption? • Attentional Draw (AD):  How quickly attention is drawn to a notification • Utility:  Usefulness of the interruption content • Matching:  Low AD ↔ Low Utility …  High AD ↔ High Utility Imager Laboratory 2 for Graphics, Visualization and HCI

Slide 3: Interruption When something causes a break in our actions, activities, or concentration. Imager Laboratory 3 for Graphics, Visualization and HCI

Slide 4: Negative Effects Lowered primary task performance  Cutrell, Czerwinski, & Horvitz 2001  Gillie & Broadbent 1989 Increased anxiety, annoyance, perceived difficulty in completing primary task  Bailey, Konstan, & Carlis, 2001 Imager Laboratory 4 for Graphics, Visualization and HCI

Slide 5: Positive Aspects Technologies that interrupt are popular  suggests there is value to interruption Other systems also have the potential to promote beneficial interruption… Imager Laboratory 5 for Graphics, Visualization and HCI

Slide 6: Mixed-Initiative Systems Assist users by making suggestions that can help users complete a task  FlexExcel (Thomas & Krogseoter, 1993)  Adaptive Bar (Debevc, Meyer, Donlagic & Svecko 1996) How can we design interruption to emphasize positive aspects? Imager Laboratory 6 for Graphics, Visualization and HCI

Slide 7: Design Guideline Make the amount of attention attracted by a notification relative to the usefulness of the interruption content Obermayer & Nugent, 2000 McFarlane & Latorella, 2002 Imager Laboratory 7 for Graphics, Visualization and HCI

Slide 8: What did we do? Empirical investigation of the effects of matching attentional draw (AD) of notification and interruption utility Imager Laboratory 8 for Graphics, Visualization and HCI

Slide 9: Attentional Draw (AD) Empirical investigation of the effects of matching attentional draw (AD) of attentional draw notification and interruption utility How quickly your attention is drawn to the notification signal Imager Laboratory 9 for Graphics, Visualization and HCI

Slide 10: Utility Empirical investigation of the effects of matching attentional draw (AD) of notification and interruption utility utility How useful, important, or urgent the interruption content is to the recipient “How helpful the interruption is in terms of performing a primary task” Imager Laboratory 10 for Graphics, Visualization and HCI

Slide 11: Findings •Empirical investigation of the effects Reduces negative effects •of matching positive perception of of Facilitates attentional draw (AD) interruption notification and interruption utility Imager Laboratory 11 for Graphics, Visualization and HCI

Slide 12: Research Overview Study 1: Find some signals Established a set of significantly different notification signals in terms of their attentional draw (AD) Study 2: Match the signals to utility Investigated matching AD and utility to see if this strategy can promote positive interruption Imager Laboratory 12 for Graphics, Visualization and HCI

Slide 13: Primary Task Imager Laboratory 13 for Graphics, Visualization and HCI

Slide 14: Imager Laboratory 14 for Graphics, Visualization and HCI

Slide 20: Study 1 Goal: Establish a set of visual notification signals with significantly different detection times Approach: Concurrent detection task  Play Memory game  Detect notification signals Imager Laboratory 20 for Graphics, Visualization and HCI

Slide 21: Notification Signals 10 signals:  Transformations applied to a base icon Properties explored  Colour  Motion  Location Imager Laboratory 21 for Graphics, Visualization and HCI

Slide 22: Signals Imager Laboratory 22 for Graphics, Visualization and HCI Time (seconds)

Slide 23: Study 1 Design • Within-subjects • 12 subjects • 10 signals • 120 trials per subject Imager Laboratory 23 for Graphics, Visualization and HCI

Slide 24: Study 1 Results: Detection Times Low AD signal: Flag Med AD signal: Slow Zoom High AD signal: Follow Imager Laboratory 24 for Graphics, Visualization and HCI

Slide 25: Study 2 Goal: Investigate the effects of matching attentional draw and utility In terms of:  Annoyance  Perceived benefit  Performance Imager Laboratory 25 for Graphics, Visualization and HCI

Slide 26: Interrupting Task Context-sensitive hints • 3 hint utility levels:  very helpful  somewhat helpful  not helpful • Emulated a mixed-initiative system Imager Laboratory 26 for Graphics, Visualization and HCI

Slide 27: QuickTime™ and a TechSmith EnSharpen decompressor are needed to see this picture. Imager Laboratory 27 for Graphics, Visualization and HCI

Slide 28: 3 Game Conditions 1. Match Low utility hint ↔ Low AD signal • Match AD to utility  Med utility hint ↔ Med AD signal High utility hint ↔ High AD signal 2. Static Low utility hint • Same AD for all  utilities Med utility hint ↔ Med AD signal High utility hint 3. Control • No interruption Imager Laboratory 28 for Graphics, Visualization and HCI

Slide 29: Study 2 Design • Conditions: Match, Static, Control • Within-subjects design • 24 subjects • 15 interruptions per condition Imager Laboratory 29 for Graphics, Visualization and HCI

Slide 30: Study 2 Measures & Procedure Procedure:  Training, {Condition, Survey} x 3, Interview Measures:  Annoyance  Perceived benefit  Fatigue  Performance (# of matches)  Preference Imager Laboratory 30 for Graphics, Visualization and HCI

Slide 31: Study 2 Hypotheses H1: Interruption annoyance is lower in the Match condition than in the Static condition H2: Perceived benefit is higher in the Match condition than in the Static condition H3: Performance is higher in the Match condition than in all other conditions Imager Laboratory 31 for Graphics, Visualization and HCI

Slide 32: Results: Annoyance & Benefit  Annoyance lower in Match (p = .034)  Benefit higher in Match (p = .037) 100 Match 80 Static 60 40 20 0 Interruption Perceived Annoyance Benefit Imager Laboratory 32 for Graphics, Visualization and HCI

Slide 33: Condition Preference 100% 80% 75% % of Participants 60% 40% 20% 15% 10% 0% Match Static Control Imager Laboratory 33 for Graphics, Visualization and HCI

Slide 34: Performance Results (# of matches) • No main effect of condition (p = .454) • Other effects present  Condition x Presentation, Fatigue, Learning,  Performance results unclear Imager Laboratory 34 for Graphics, Visualization and HCI

Slide 35: Study 2: Summary of Results  H1: Interruption annoyancein the Static condition is lower in the Match condition than  H2: Perceived benefit isinhigher in thecondition Match condition than the Static  H3: Performance is higher in the Match condition than in all other conditions Imager Laboratory 35 for Graphics, Visualization and HCI

Slide 36: Future Research • Computational appraisal of utility • Notification signals • Scope of utility Primary task relevance: • Re-examine performance results Personal relevance: • Confirm annoyance and benefit results Imager Laboratory 36 for Graphics, Visualization and HCI

Slide 37: Take Home Message Matching AD and utility  Reduces annoyance and  Increases perception of benefit • UI design implications? Imager Laboratory 37 for Graphics, Visualization and HCI

Slide 38: Acknowledgements Many thanks to: • Natural Science and Engineering Research Council of Canada (NSERC) • Lyn Bartram Imager Laboratory 38 for Graphics, Visualization and HCI

Slide 39: Take Home Message Matching AD and utility  Reduces annoyance and  Increases perception of benefit Questions? Imager Laboratory 39 for Graphics, Visualization and HCI

Slide 40: Additional Information Imager Laboratory 40 for Graphics, Visualization and HCI

Slide 41: Interview Results Comprehension of AD-utility relationship Low 70% Med 40% High 45% All 40% None 25% 40% utilized this knowledge to ignore low- utility hints Imager Laboratory 41 for Graphics, Visualization and HCI

Slide 42: Related Work Coordination Notification  McFarlane 2002  Bartram, Ware, Calvert 2003  Horvitz, Kadie, Paek, Hovel 2003 Timing of Onset Relevance  Fogarty et al. 2004  Bailey et al. 2001  Czerwinski, Cutrell, Horvitz 2000 Peripheral Utility Awareness  Horvitz, Koch, Apacible 2004  Maglio & Campbell 2000  Horvitz, Jacobs, Hovel 1999  McCrickard et al. 2003  Avrahami & Hudson 2005  Gievska & Sibert 2005 Imager Laboratory 42 for Graphics, Visualization and HCI

Slide 43: Study 1: Simple Editor Task Imager Laboratory 43 for Graphics, Visualization and HCI

Slide 44: Study 1 Design Within-subjects: 2 (task) x 10 (signal) x 3 (block) Block Design  2 replications per block  3 blocks per primary task (6 total)  120 trials per subject Measures  Detection time 12 subjects, 90 minute experiment Imager Laboratory 44 for Graphics, Visualization and HCI

Slide 45: Study 1: Structure of a Trial Imager Laboratory 45 for Graphics, Visualization and HCI

Slide 46: Study 1 Results • Significant main effect of signal (p < .001) • 3-way significant comparison between Flag, Slow Zoom, and Follow 10 10 9.19 9.19 8 8 Time (seconds) Time (seconds) 6.02 6.02 6 6 4.83 4.83 4.57 4.57 3.39 3.39 4 4 1.96 1.96 1.88 1.88 1.74 1.741.49 1.49 2 2 1.19 1.19 0 0 Flag Flag Colour Colour GrowOscillate Grow Oscillate Slow Slow Slow Bounce FastFast Fast Fast Slow Bounce Follow Follow Zoom Zoom Blink Blink Zoom ZoomBlink Blink Imager Laboratory 46 for Graphics, Visualization and HCI

Slide 47: Study 1: Pairwise Comparisons Imager Laboratory 47 for Graphics, Visualization and HCI

Slide 48: Study 1: Timeouts Main effect of signal (F(2.542,27.961) = 3.630, p = .031, η2 = .248) Imager Laboratory 48 for Graphics, Visualization and HCI

Slide 49: Study 2 Design • Conditions: Match, Static, Control • Within-subjects design • 5 replications in 17-min block (i.e. 15 interruptions per condition) • 65-second average interruption frequency • 24 subjects • 1 hour experiment Imager Laboratory 49 for Graphics, Visualization and HCI

Slide 50: Study 2: Interruption Structure Imager Laboratory 50 for Graphics, Visualization and HCI

Slide 51: Missed-Hints Dialog Imager Laboratory 51 for Graphics, Visualization and HCI

Slide 52: Study 2 General Annoyance No main effect of condition (F(2,24) = 2.788, p = .079, η2 = .166, power = .429) Imager Laboratory 52 for Graphics, Visualization and HCI

Slide 53: Study 2: Presentation Order Effect F(5,14) = 2.720, p = .064, η2 = .493 Imager Laboratory 53 for Graphics, Visualization and HCI

Slide 54: Study 2 Interaction Effect F(10,28) = 2..35, p = .068, η2 = .421 Imager Laboratory 54 for Graphics, Visualization and HCI

Slide 55: Study 2 Hint Duration Main effect of utility (F(2,36) = 6.839, p = .003, η2 = .275) Imager Laboratory 55 for Graphics, Visualization and HCI

Slide 56: Study 2 Workload NASA-TLX Factor F(2,28) p η2 Mental Demand .057 .945 .004 Physical Demand 2.335 .115 .143 Temporal Demand 1.069 .357 .071 Effort .118 .889 .008 Perceived Performance 1.347 .276 .088 Frustration .381 .687 .027 Imager Laboratory 56 for Graphics, Visualization and HCI

Slide 57: NASA Task Load Index Mental Demand How much mental and perceptual activity was required to play the game and attend to the hints? (e.g., thinking, remembering, looking, searching, deciding, etc.)? Physical Demand How much physical activity was required to play the game? (e.g. moving the mouse, clicking the mouse button, etc.) Temporal Demand How much time pressure did you feel due to the rate or pace at which the tasks or task elements occurred? Imager Laboratory 57 for Graphics, Visualization and HCI

Slide 58: NASA Task Load Index (2) Effort How hard did you have to work (mentally and physically) to accomplish your level of performance? Perceived Performance How successful do you think you were in accomplishing the goals of the task set by the experimenter (or yourself)? Frustration How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent did you feel during the task? Imager Laboratory 58 for Graphics, Visualization and HCI

Slide 59: Performance Results • No main effect of condition (p = .454) • Main effect: presentation order (p = .064, η2 = .493)  Interaction effect: condition & presentation order (p = .068, η2 = .421) • Learning effect (p = .053) • Fatigue effect (p = .009) Impossible to interpret performance results Imager Laboratory 59 for Graphics, Visualization and HCI

Slideshare.net (beta)