Slide 1: EVALUATING VISUAL CUES FOR WINDOW SWITCHING ON LARGE SCREENS Raphael Hoffmann (University of Washington) Patrick Baudisch (Microsoft Research) Daniel S. Weld (University of Washington) 1

Slide 2: WHICH IS THE ACTIVE WINDOW? 2

Slide 3: ON LARGE SCREENS CHANGES MAY GO UNNOTICED 3

Slide 4: NOTICING CHANGE  Flashing of window moving to foreground  Flashing of window frame  Visual Search 4

Slide 5: A HIGHLY EFFECTIVE CUE? 5

Slide 6: QUESTION Which notification technique is highly effective and has little side effects? 6

Slide 7: DESIGN GOALS 1 Minimize visual search time; Do not slow down users who already know where the target is 2 Minimize impact on other screen content 3 Use sparse visual effects to minimize fatigue/annoyance 7

Slide 8: VISUAL CUES frames trails 8

Slide 9: BLINKINGFRAME 9

Slide 10: REDFRAME 10

Slide 11: BUBBLEFRAME 11

Slide 12: SHADOWFRAME 12

Slide 13: MASK 13

Slide 14: CENTERBEAM 14

Slide 15: CENTERSPLASH 15

Slide 16: WINDOWSPLASH 16

Slide 17: WINDOWBEAM 17

Slide 18: OUTLINE • Motivation • Related Work & Visual Cues for Window Switching • Studies • Conclusion 18

Slide 19: PERCEPTUAL UNDERPINNINGS high acuity vision peripheral vision 19

Slide 20: PERCEPTUAL UNDERPINNINGS 20

Slide 21: FEATURE INTEGRATION THEORY (Treisman, Gelade; CP 1980) 1. Pre-attentive Search 2. Attentive Search 21

Slide 22: COLOR Feature Integration Theory (Treisman, Gelade; CP 1980) 22

Slide 23: MOTION Feature Integration Theory (Treisman, Gelade; CP 1980) 23

Slide 24: SHAPE Feature Integration Theory (Treisman, Gelade; CP 1980) 24

Slide 25: SHADING + SHADOWS Feature Integration Theory (Treisman, Gelade; CP 1980) 25

Slide 26: NOISY BACKGROUND 26

Slide 27: COLOR VS. SHAPE Feature Integration Theory (Treisman, Gelade; CP 1980) 27

Slide 28: COLOR VS. SHAPE Feature Integration Theory (Treisman, Gelade; CP 1980) > 28

Slide 29: HETEROGENOUS DISTRACTORS Feature Integration Theory (Treisman, Gelade; CP 1980) 29

Slide 30: HOMOGENOUS DISTRACTORS Feature Integration Theory (Treisman, Gelade; CP 1980) 30

Slide 31: WHAT STANDS OUT IN OUR UI? 31

Slide 32: POPOUT PRISM Suh, Woodruff, Rosenholtz, Glass; CHI 2002 32

Slide 33: HALO Baudisch, Rosenholtz; CHI 2003 + 33

Slide 34: SPOTLIGHT Khan, Matejka, Fitzmaurice, Kurtenbach; CHI 2005 + 34

Slide 35: PHOSPHOR Baudisch, Tan, Collomb, Robbins, Hinckley, Agrawala, Zhao, Ramos; UIST 2006 DRAG-AND-POP Baudisch, Cutrell, Robbins, Czerwinski, Tandler, Bederson, Zierlinger; Interact 2003 35

Slide 36: TRAILS • Work even when target outside field of view • Reduce 2d search to 1d search 36

Slide 37: IMPLEMENTATION • Program written in C# using WPF • Cues can be selected through a configuration menu • System integrated with Alt-TAB mechanism in Windows XP 37

Slide 38: OUTLINE • Motivation • Related Work & Visual Cues for Window Switching • Studies • Conclusion 38

Slide 39: WHICH CUE SUPPORTS USERS BEST? <remind>: • Efficiency in finding target • Awareness of other screen content • Avoiding annoyance 39

Slide 40: SEARCH SPACE 2 x 2 x 2 x 2 x 2 x ( ) 4+1 = 160 control combinations 40

Slide 41: SEARCH SPACE 1 + 1 + 1 + 1 + 1 + ( ) = 10 (combinations) 4+1 STUDY 1 41

Slide 42: SEARCH SPACE 1 + 1 + 1 + 1 + 1 + ( ) 4+1 42

Slide 43: SEARCH SPACE 1 + 1 + 1 + =6 combinations 1 STUDY 2 + 1 + 1 43

Slide 44: STUDY 1: CUES IN ISOLATION 44

Slide 45: TASK • Switch window focus by hitting space bar. • Locate newly active window and type number (title bar read ‘Window - #’). • If number correct, system removed it from title bar and replaced it with a new number on next switch 45

Slide 46: TASK • In 20% of the cases when windows were switched, a popup window with a letter appeared for 350ms at another location. • Participants were asked to type the letter, if they saw the popup. • Participants rated annoyance on Likert scale; said what they prefered 46

Slide 47: MEASURED … • Efficiency in finding target task time • Awareness of other screen content proportion of overlooked popups • Annoyance subjective rating on 7pt Likert scale 47

Slide 48: STUDY DESIGN • Within-subjects design, 10 conditions, 62 repetitions, 10 participants • Background content: 80 windows – 1 /3 accessories, 1/3 web pages, 1/3 office docs • Mixed Model analysis, ANOVA on ranks • Sequential Bonferroni correction 48

Slide 49: APPARATUS • 24’’ + 30’’ + 24’’ displays • 25’’ distance to center monitor • Used leash to prevent moving backwards 49

Slide 50: RESULTS – OVERVIEW Task Time Task Time (seconds ± SEM) 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash Popup Misses Popup Detection Error (N SEM) ± 12 10 8 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 5 Annoyance Annoyance (rating ± SEM) 4 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 50

Slide 51: RESULTS – OVERVIEW Task Time Task Time (seconds ± SEM) 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash Popup Misses Popup Detection Error (N SEM) ± 12 10 8 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 5 Annoyance Annoyance (rating ± SEM) 4 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 51

Slide 52: FRAMES – TASK TIME sec 3 2 1 dominant dominant 0 Control BlinkingFrame RedFrame BubbleFrame ShadowFrame 52

Slide 53: FRAMES – ANNOYANCE 6 4 2 dominant dominant 0 Control BlinkingFrame RedFrame BubbleFrame ShadowFrame 53

Slide 54: PARTICIPANT’S FAVORITE FRAME Task Time Task Time (seconds ± SEM) 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash Popup Misses Popup Detection Error (N SEM) ± 12 10 8 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 5 Annoyance Annoyance (rating ± SEM) 4 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 54

Slide 55: RESULTS – OVERVIEW Task Time Task Time (seconds ± SEM) 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash Popup Misses Popup Detection Error (N SEM) ± 12 10 8 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 5 Annoyance Annoyance (rating ± SEM) 4 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 55

Slide 56: MASK – TASK TIME sec 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 56

Slide 57: MASK – POPUP MISSES 12 10 8 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 57

Slide 58: MASK – ANNOYANCE 5 4 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 58

Slide 59: RESULTS – OVERVIEW Task Time Task Time (seconds ± SEM) 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash Popup Misses Popup Detection Error (N SEM) ± 12 10 8 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 5 Annoyance Annoyance (rating ± SEM) 4 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 59

Slide 60: TRAILS – TASK TIME 3 2 1 asymm. asymm. 0 Control CenterBeam WindowBeam CenterSplash WindowSplash 60

Slide 61: PARTICIPANT’S FAVORITE TRAIL Task Time Task Time (seconds ± SEM) 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash Popup Misses Popup Detection Error (N SEM) ± 12 10 8 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 5 Annoyance Annoyance (rating ± SEM) 4 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 61

Slide 62: RESULTS – OVERVIEW Task Time Task Time (seconds ± SEM) 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash Popup Misses Popup Detection Error (N SEM) ± 12 10 8 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 5 Annoyance Annoyance (rating ± SEM) 4 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Dark- Center- Center- Window- Window- Frame Frame Frame Frame Mask Beam Splash Beam Splash 62

Slide 63: TRAILS/FRAMES – TASK TIME targets close 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Center- Center- Window- Window- Frame Frame Frame Frame Beam Splash Beam Splash 63

Slide 64: TRAILS/FRAMES – TASK TIME targets far 3 2 1 0 Control Blinking- Red- Bubble- Shadow- Center- Center- Window- Window- Frame Frame Frame Frame Beam Splash Beam Splash 64

Slide 65: TRAILS/FRAMES – POPUP MISS. 12 10 8 Control Blinking- Red- Bubble- Shadow- Center- Center- Window- Window- Frame Frame Frame Frame Beam Splash Beam Splash 65

Slide 66: OPEN QUESTIONS • Does combining frames and trails make better cues? + > ? • Does adding additional frame features help? + + > + ? • Will the mask improve with lower opacity (and a frame)? > ? 66

Slide 67: STUDY 2: CUE COMBINATIONS synergy or interference? 67

Slide 68: MASK + BUBBLE 68

Slide 69: CENTERSPLASH + REDFRAME 69

Slide 70: CS + RED + BUBBLE 70

Slide 71: CS + RED + BUBBLE + SHADOW 71

Slide 72: MASK + FRAMES – TASK TIME sec 1.5 Task Time 1.0 0.5 0.0 CenterSplash CenterSplash CenterSplash CenterSplash + RedFrame + RedFrame + RedFrame + BubbleFrame + BubbleFrame + ShadowFrame 72

Slide 73: MASK + BUBBLE Task Time sec 3 2 1 0 Control DarkMask CenterSplash CenterSplash CenterSplash CenterSplash + BubbleFrame + RedFrame + RedFrame + RedFrame + BubbleFrame + BubbleFrame + ShadowFrame Annoyance 6 4 2 0 Control DarkMask CenterSplash CenterSplash CenterSplash CenterSplash + BubbleFrame + RedFrame + RedFrame + RedFrame + BubbleFrame + BubbleFrame + ShadowFrame 73

Slide 74: PARTICIPANTS’ FAVORITE CUE 74

Slide 75: OUTLINE • Motivation • Related Work & Visual Cues for Window Switching • Studies • Conclusion 75

Slide 76: IMPLICATIONS FOR DESIGN • Visual sparseness: Even low-opacity masks interfere w/ ability to detect target by its onset. • Adapt cue by distance: When targets are far, trails perform better than frames. • Use Asymmetric trails: Trails must stand out from rectilinear screen content. • Use subtle features: Even low-opacity frames pop out; dominant features outperform others, but are more annoying. • Use combinations: Combinations of frames and trails perform better than cues in isolation. 76

Slide 77: THANK YOU! Raphael Hoffmann Computer Science & Engineering University of Washington raphaelh@cs.washington.edu Patrick Baudisch Microsoft Research Redmond, Washington baudisch@microsoft.com Daniel S. Weld Computer Science & Engineering University of Washington weld@cs.washington.edu This material is based upon work supported by the National Science Foundation under grant IIS-0307906, by the Office of Naval Research under grant N00014-06-1-0147, SRI International under CALO grant 03-000225 and the Washington Research Foundation / TJ Cable Professorship. 77

Slide 78: TRAILS – POPUP MISSES 12 10 asymm. asymm. 8 Control CenterBeam WindowBeam CenterSplash WindowSplash 78