Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AwToolkit: Attention-Aware User Interface Widgets

683 views

Published on

AVI 2014 Best Paper: http://dx.doi.org/10.1145/2598153.2598160
See patent: Visual focus-aware techniques for visualizing display changes - https://patents.google.com/patent/US20150227334A1
Increasing screen real-estate allows for the development of applications where a single user can manage a large amount of data and related tasks through a distributed user interface. However, such users can easily become overloaded and become unaware of display changes as they alternate their attention towards different displays. We propose AwToolkit, a novel widget set for developers that supports users in maintaining awareness in multi-display systems. The AwToolkit widgets automatically determine which display a user is looking at and provide users with notifications with different levels of subtlety to make the user aware of any unattended display changes. The toolkit uses four notification levels (unnoticeable, subtle, intrusive and disruptive), ranging from an almost imperceptible visual change to a clear and visually salient change. We describe AwToolkit’s six widgets, which have been designed for C# developers, and the design of a user study with an application oriented towards healthcare environments. The evaluation results reveal a marked increase in user awareness in comparison to the same application implemented without AwToolkit.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

AwToolkit: Attention-Aware User Interface Widgets

  1. 1. AwToolkit: Attention-Aware User Interface Widgets Attention-Aware User Interface Widgets Juan E. Garrido, Victor M. R. Penichet, Maria D. Lozano Computer Science Research Institute, University of Castilla-La Mancha Aaron Quigley, Per Ola Kristensson SACHI, School of Computer Science, University of St Andrews
  2. 2. 2AVI’14: AwToolkit Slide of 20 The problem
  3. 3. 3AVI’14: AwToolkit Slide of 20 The opportunity • Determine which display a user is attending to • Determine what changes are occurring on a non- attended display and where • Determine a means to notify a user of these changes • either on the display they are currently attending to or • in-situ, when they return their gaze to a previously unattended display • Determine the granularity to notify of change • Determine if this has an impact on performance.
  4. 4. 4AVI’14: AwToolkit Slide of 20 Overview • Related work • AwToolkit concept and features in widgets • Spectrum of change notification • Properties • AWPanel and change notification techniques • AwButton, AwLabel, AwTextBox, AwComboBox and AwPlayer • Evaluation: User’s View • Evaluation: Developer’s View • Conclusions and future work
  5. 5. 5AVI’14: AwToolkit Slide of 20 Related work MAUI Toolkit, CSCW 2004 Mnemonic rendering, UIST 2006 DiffDisplays, IUI 2014TeleEye, CollaborateCom 2008
  6. 6. 6AVI’14: AwToolkit Slide of 20 DiffDisplays: Gaze dependent techniques for visualizing display changes (IUI 2013) Jakub Dostal, Per Ola Kristensson, and Aaron Quigley. 2013. Subtle gaze-dependent techniques for visualising display changes in multi-display environments. In Proceedings of the 2013 international conference on Intelligent user interfaces (IUI '13). ACM, New York, NY, USA, 137-148.
  7. 7. 7AVI’14: AwToolkit Slide of 20 DiffDisplays Jakub Dostal, Per Ola Kristensson, and Aaron Quigley. 2013. Subtle gaze-dependent techniques for visualising display changes in multi-display environments. In Proceedings of the 2013 international conference on Intelligent user interfaces (IUI '13). ACM, New York, NY, USA, 137-148.
  8. 8. 8AVI’14: AwToolkit Slide of 20 DiffDisplays PixMap Aura WindowMap
  9. 9. 9AVI’14: AwToolkit Slide of 20 DiffDisplays
  10. 10. 10AVI’14: AwToolkit Slide of 20 SpiderEyes QuickTime™ and a H.264 decompressor are needed to see this picture. Dostal, J., Hinrichs, U., Kristensson, P.O. and Quigley, A. 2014. SpiderEyes: designing attention- and proximity-aware collaborative interfaces for wall-sized displays. In Proceedings of the 19th ACM International Conference on Intelligent User Interfaces (IUI 2014).
  11. 11. 11AVI’14: AwToolkit Slide of 20 Overview • Related work • AwToolkit concept and features in widgets • Spectrum of change notification • Properties • AWPanel and change notification techniques • AwButton, AwLabel, AwTextBox, AwComboBox and AwPlayer • Evaluation: User’s View • Evaluation: Developer’s View • Conclusions and future work
  12. 12. 12AVI’14: AwToolkit Slide of 20 Widgets http://www.eclipse.org/swt/widgets/
  13. 13. 13AVI’14: AwToolkit Slide of 20 AWToolkit concept • Goal: generate change notifications upon return of gaze to display • Approach: Gaze-aware user interface components (widgets) • subtle change within widgets • Three categories: entire screen, part of screen, mirroring techniques • AwToolkit C# toolkit for developers for gaze sensitive user interface controls • Examples: panel, button, label, textbox, comboBox and media player
  14. 14. 14AVI’14: AwToolkit Slide of 20 AWToolkit features • Toolkit for detection and reaction • Objective: preserve look and feel of existing components as in, for example, Visual Studio • Detection based on the Diff Displays system (widgets connected to server) • Two gaze aware states • Properties of each widget allow the developer to modify the visual appearance and behaviour ie. the degree of saliency in change notification.
  15. 15. 15AVI’14: AwToolkit Slide of 20 Spectrum of change notification • Disruptive: • the user is alerted and the current activity is disrupted • Intrusive: • the user is alerted and the goal is clearly to avert their gaze to the location of the displayed change notification • Subtle: • the user is alerted with less obvious visual cues. The goal is to draw their gaze to the display change notification • Unnoticeable: • the user is alerted in such a subtle manner, that they may not even notice the display change notification.
  16. 16. 16AVI’14: AwToolkit Slide of 20 Properties • Developer can specify • Choice of display change notifications • Appearance of graphic elements • Length of time for change notification • Or use AwDefaultTechnique to automate level selection • default selection of visual change elements • based on user study discussed later • Three methods for localisation of widget content change • Widget screenshot, widget text, state change.
  17. 17. 17AVI’14: AwToolkit Slide of 20 AWPanel • Based on standard panel control • Typically used to organise application content • Polygon with content and margin (a continuous line) • Widget detects changes by comparing screen-shots over time (on a pixel level) • This allows AWPanel to be content agnostic
  18. 18. 18AVI’14: AwToolkit Slide of 20 3 Change notification techniques • Modifying the widget margin: colour, line stippling pattern, corner marks • Choice of these vs standard margin can range from unnoticeable to intrusive • Quadrant view: panel conceived as four regions tied to the four corners (change and where) • Book mode: dependent on the number of changes series of navigable screen shots. 18
  19. 19. 19AVI’14: AwToolkit Slide of 20 AwButton, AwLabel, AwTextBox, AwComboBox and AwPlayer • Modifying the widget margin: colour or line stippling pattern • Highlight changes within the active area of the widget • AwPlayer offers a disruptive notification connected to a media player stopping or restarting a video depending on a users gaze.
  20. 20. 20AVI’14: AwToolkit Slide of 20 Ubi4health tasks to be performed alert control
  21. 21. 21AVI’14: AwToolkit Slide of 20 Evaluation: User’s View • Evaluation was performed with 12 participants (5m, 7f: age 21-48, avg 36) with Ubi4health • a healthcare application with a three-screen configuration • used to manage essential tasks in residential care homes • typical use case: left screen (tasks to be performed), centre screen (alert control), right screen (staff management) • two versions of Ubi4health • One with AwPanel, AwButton, AwLabel, AwTextBox and AwComboBox (and one without). QuickTime™ and a AVC Coding decompressor are needed to see this picture.
  22. 22. 22AVI’14: AwToolkit Slide of 20 Evaluation: Tasks • Scenario: a user responsible for the completion of pending tasks, staff organisation and the immediate attention of emergencies in a real residential care home • Task 1: start of shift, task progression monitoring, resident alert monitoring, moving to staff allocation (as tasks not performed in time) • Task 2: staff organisation (single screen focus 15 sec), move to alter monitoring • Task 3: staff reorganisation while monitoring emergencies (25 sec), return to pending tasks • Task 4: task monitoring (15 sec), move to staff organisation.
  23. 23. 23AVI’14: AwToolkit Slide of 20 Evaluation: Results • Within-subjects design with toolkit as the independent variable, accuracy as the dependent variable • Conditions made it difficult for the participants to compare the current state of a screen with a previous one • Improvement in tasks 1, 3 and 4 (p < 0.0001; RM-ANOVA, Bonferroni corrected) • Task 2, Ubi4health already has a clear visual change on the screen • (1,3,4) without AwToolkit: 11-23% accuracy With 83-100% accuracy.
  24. 24. 24AVI’14: AwToolkit Slide of 20 Evaluation: Developer’s View • 12 developers (good knowledge of C# and Visual Studio) • Task 1: import the library with AwToolkit to be able to see and use its items in the toolbar • Task 2: insert AwButton and AwPanel into an application and define how widgets behave when user is not looking • Task 3: insert a 10-seconds timer to be run when Aw- Button is pressed • Task 4: disable the button after the 10-second timeout and introduce any change in the AwPanel • Avg accuracy 100% on tasks 1 and 2, 83% on task 3 and 92% on task 4.
  25. 25. 25AVI’14: AwToolkit Slide of 20 Conclusions and future work • A novel toolkit for developers to support notifying users of changes that appear in the user interface when users are not looking at the display • Awareness of changes at the widget level at varying degrees of subtly • Future: • extensions of the toolkit to further interface components • exploration of mirroring and multi-screen notifications • exploration of spectrum of change in multi-display environments including mobile interface elements.
  26. 26. Thank you aquigley@st-andrews.ac.uk

×