1a8 parrallel ui rendering

322 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
322
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

1a8 parrallel ui rendering

  1. 1. Parallel User Interface Rendering ~Accessibility for Custom Widgets Kris Van Hees & Jan Engelen Katholieke Universiteit Leuven Department of Electrical Engineering ESAT - SCD - DocArch Kris Van Hees & Jan Engelen 1/16
  2. 2. Agenda● Introduction● Parallel User Interface Rendering● Widgets● Cosmetic widgets● Conclusions● Future... Kris Van Hees & Jan Engelen 2/16
  3. 3. Introduction● GUIs in all aspects of life: computers, MP3 players, home appliances, mobile phones, ...● Inherent visual interaction model poses a complication for users with disabilities (especially blindness)● Mixing graphical toolkits within a single environment complicates matters even more (Linux: Qt, GTK, Athena, …)● Custom widgets: often not supported in assistive technology without fancy scripting (if available) Kris Van Hees & Jan Engelen 3/16
  4. 4. Introduction (cont...)● Abstracting the user interface offers advantages: – Unified user interaction semantics across toolkits and modalities – Retaining flexibility in rendering for multiple modalities – Assistive technology as 1st generation rendering● Parallel User Interface Rendering (PUIR) provides for equivalent concurrent renderings of the UI Kris Van Hees & Jan Engelen 4/16
  5. 5. Parallel User Interface Rendering● User interface defined in abstract form (XML file) – Effectively defines data and how to present data● Abstract UI description interpreted by an AUI engine – Handles user interaction semantics● UI presentation rendered by modality-specific agent(s)● Rendering agent depends on system specific toolkits● Rendering agents provide presentations concurrently – Provide both visual and non-visual rendering Kris Van Hees & Jan Engelen 5/16
  6. 6. PUIR: The AUI engine● Interprets an abstract UI description (XML file)● Implements user interaction semantics● Encapsulates data in the UI● Provides focus management● Dispatches events to the application and rendering agents● Can be called from application and rendering agents● Has no knowledge of implementation details of application and rendering agents Kris Van Hees & Jan Engelen 6/16
  7. 7. PUIR: GUI rendering agent● Java Swing based in-process rendering agent● Complex changes to bypass user interaction in widgets● Reduced to provide merely the presentation of widgets● Some widgets operate slightly differently● Pointer device UI interaction interpreted in the rendering agent: – Necessary because position is tied to GUI context – Semantic operations mapped to AUI operations Kris Van Hees & Jan Engelen 7/16
  8. 8. PUIR: Non-visual rendering agent● Remote rendering agent● Basic speech output presentation as proof-of-concept● Proxy objects for transparency and caching● Equivalent with GUI rendering agent: – Receives the same events – Synchronized with other rendering agents Kris Van Hees & Jan Engelen 8/16
  9. 9. PUIR: Input processing● Keyboard: – Textual input – Navigation, text selection, editing, … – Processed directly by AUI (for proof-of-concept)● Mouse: – Processed by rendering agent – Position to be interpreted within graphical context – Semantic user interaction delegated to AUI engine Kris Van Hees & Jan Engelen 9/16
  10. 10. PUIR: Input processing (cont...)● Braille keyboard: – Coded textual input translated into text – Text processed as if entered on regular keyboard – Limited cursor addressable positional input – Positional input interpreted in context – Semantic user interaction delegated to AUI engine Kris Van Hees & Jan Engelen 10/16
  11. 11. Widgets● Two entities per widget: – Abstract widget: ● Operates entirely within the AUI engine ● Implements user interaction semantics ● Encapsulates data – Widget presentation: ● Usually one per rendering agent ● Implements rendering in output modality Kris Van Hees & Jan Engelen 11/16
  12. 12. Widgets (cont...)● Design: – Define UI in abstract UI description – Typically by means of design tool – Rendering specific information can be included Kris Van Hees & Jan Engelen 12/16
  13. 13. Widgets (cont...)● Runtime construction: – Instantiation of UI based on AUI description – For every widget: ● Find built-in abstract widget class ● If none, search in alternate locations ● If none, UI cannot be instantiated Kris Van Hees & Jan Engelen 13/16
  14. 14. Widgets (cont...)● Rendering: – One-to-one correspondence between abstract widget class and presentation class – For every widget: ● Find built-in widget presentation class ● If none, search in alternate locations ● If none and cosmetic widget, use core alternate (more on this 2 slides from now...) ● If none, use default place-holder widget Kris Van Hees & Jan Engelen 14/16
  15. 15. Widgets (cont...)● Rendering (cont...): – Rendering agent knows what widgets are supported – Rendering agent can alert user appropriately – Rendering agent can query information from AUI about the unsupported widget – Accidental user interaction is guarded against Kris Van Hees & Jan Engelen 15/16
  16. 16. Cosmetic widgets● Widgets with a very specific presentation● Typically derivative of an existing core widget● User interaction semantics remain the same● AUI description specifies equivalence between cosmetic widget and core widget● Core widget can act as default presentation when no custom presentation class is found Kris Van Hees & Jan Engelen 16/16
  17. 17. Cosmetic widgets (cont...)● Left-to-right: – Four regular buttons – VU-meter style widget: equivalent with numeric label – Four knobs: equivalent with discrete slider – Four indicator lights: equivalent with label – Four regular buttons Kris Van Hees & Jan Engelen 17/16
  18. 18. Conclusions● Parallel User Interface Rendering – Powerful technique for Design-for-All – Coherence between concurrent equivalent renderings – Generic: not just for non-visual renderings – Robust handling of custom widgets – Support for custom widgets independent from applications and specific uses – Can be used in automated application testing as well Kris Van Hees & Jan Engelen 18/16
  19. 19. Future...● Completion of proof-of-concept● More testing● Public release● Default presentation of custom widgets merely based on abstract widget information● Remote operation of applications At all stages, feedback from real users is be crucial! Kris Van Hees & Jan Engelen 19/16
  20. 20. Kris Van Hees & Jan Engelen Katholieke Universiteit LeuvenDepartment of Electrical Engineering ESAT – SCD – DocArch Kasteelpark Arenberg 10 B-3001 Heverlee Belgiumkris@alchar.org, jan@docarch.be Kris Van Hees & Jan Engelen 20/16

×