• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
1a8 parrallel ui rendering
 

1a8 parrallel ui rendering

on

  • 338 views

 

Statistics

Views

Total Views
338
Views on SlideShare
338
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    1a8 parrallel ui rendering 1a8 parrallel ui rendering Presentation Transcript

    • Parallel User Interface Rendering ~Accessibility for Custom Widgets Kris Van Hees & Jan Engelen Katholieke Universiteit Leuven Department of Electrical Engineering ESAT - SCD - DocArch Kris Van Hees & Jan Engelen 1/16
    • Agenda● Introduction● Parallel User Interface Rendering● Widgets● Cosmetic widgets● Conclusions● Future... Kris Van Hees & Jan Engelen 2/16
    • Introduction● GUIs in all aspects of life: computers, MP3 players, home appliances, mobile phones, ...● Inherent visual interaction model poses a complication for users with disabilities (especially blindness)● Mixing graphical toolkits within a single environment complicates matters even more (Linux: Qt, GTK, Athena, …)● Custom widgets: often not supported in assistive technology without fancy scripting (if available) Kris Van Hees & Jan Engelen 3/16
    • Introduction (cont...)● Abstracting the user interface offers advantages: – Unified user interaction semantics across toolkits and modalities – Retaining flexibility in rendering for multiple modalities – Assistive technology as 1st generation rendering● Parallel User Interface Rendering (PUIR) provides for equivalent concurrent renderings of the UI Kris Van Hees & Jan Engelen 4/16
    • Parallel User Interface Rendering● User interface defined in abstract form (XML file) – Effectively defines data and how to present data● Abstract UI description interpreted by an AUI engine – Handles user interaction semantics● UI presentation rendered by modality-specific agent(s)● Rendering agent depends on system specific toolkits● Rendering agents provide presentations concurrently – Provide both visual and non-visual rendering Kris Van Hees & Jan Engelen 5/16
    • PUIR: The AUI engine● Interprets an abstract UI description (XML file)● Implements user interaction semantics● Encapsulates data in the UI● Provides focus management● Dispatches events to the application and rendering agents● Can be called from application and rendering agents● Has no knowledge of implementation details of application and rendering agents Kris Van Hees & Jan Engelen 6/16
    • PUIR: GUI rendering agent● Java Swing based in-process rendering agent● Complex changes to bypass user interaction in widgets● Reduced to provide merely the presentation of widgets● Some widgets operate slightly differently● Pointer device UI interaction interpreted in the rendering agent: – Necessary because position is tied to GUI context – Semantic operations mapped to AUI operations Kris Van Hees & Jan Engelen 7/16
    • PUIR: Non-visual rendering agent● Remote rendering agent● Basic speech output presentation as proof-of-concept● Proxy objects for transparency and caching● Equivalent with GUI rendering agent: – Receives the same events – Synchronized with other rendering agents Kris Van Hees & Jan Engelen 8/16
    • PUIR: Input processing● Keyboard: – Textual input – Navigation, text selection, editing, … – Processed directly by AUI (for proof-of-concept)● Mouse: – Processed by rendering agent – Position to be interpreted within graphical context – Semantic user interaction delegated to AUI engine Kris Van Hees & Jan Engelen 9/16
    • PUIR: Input processing (cont...)● Braille keyboard: – Coded textual input translated into text – Text processed as if entered on regular keyboard – Limited cursor addressable positional input – Positional input interpreted in context – Semantic user interaction delegated to AUI engine Kris Van Hees & Jan Engelen 10/16
    • Widgets● Two entities per widget: – Abstract widget: ● Operates entirely within the AUI engine ● Implements user interaction semantics ● Encapsulates data – Widget presentation: ● Usually one per rendering agent ● Implements rendering in output modality Kris Van Hees & Jan Engelen 11/16
    • Widgets (cont...)● Design: – Define UI in abstract UI description – Typically by means of design tool – Rendering specific information can be included Kris Van Hees & Jan Engelen 12/16
    • Widgets (cont...)● Runtime construction: – Instantiation of UI based on AUI description – For every widget: ● Find built-in abstract widget class ● If none, search in alternate locations ● If none, UI cannot be instantiated Kris Van Hees & Jan Engelen 13/16
    • Widgets (cont...)● Rendering: – One-to-one correspondence between abstract widget class and presentation class – For every widget: ● Find built-in widget presentation class ● If none, search in alternate locations ● If none and cosmetic widget, use core alternate (more on this 2 slides from now...) ● If none, use default place-holder widget Kris Van Hees & Jan Engelen 14/16
    • Widgets (cont...)● Rendering (cont...): – Rendering agent knows what widgets are supported – Rendering agent can alert user appropriately – Rendering agent can query information from AUI about the unsupported widget – Accidental user interaction is guarded against Kris Van Hees & Jan Engelen 15/16
    • Cosmetic widgets● Widgets with a very specific presentation● Typically derivative of an existing core widget● User interaction semantics remain the same● AUI description specifies equivalence between cosmetic widget and core widget● Core widget can act as default presentation when no custom presentation class is found Kris Van Hees & Jan Engelen 16/16
    • Cosmetic widgets (cont...)● Left-to-right: – Four regular buttons – VU-meter style widget: equivalent with numeric label – Four knobs: equivalent with discrete slider – Four indicator lights: equivalent with label – Four regular buttons Kris Van Hees & Jan Engelen 17/16
    • Conclusions● Parallel User Interface Rendering – Powerful technique for Design-for-All – Coherence between concurrent equivalent renderings – Generic: not just for non-visual renderings – Robust handling of custom widgets – Support for custom widgets independent from applications and specific uses – Can be used in automated application testing as well Kris Van Hees & Jan Engelen 18/16
    • Future...● Completion of proof-of-concept● More testing● Public release● Default presentation of custom widgets merely based on abstract widget information● Remote operation of applications At all stages, feedback from real users is be crucial! Kris Van Hees & Jan Engelen 19/16
    • Kris Van Hees & Jan Engelen Katholieke Universiteit LeuvenDepartment of Electrical Engineering ESAT – SCD – DocArch Kasteelpark Arenberg 10 B-3001 Heverlee Belgiumkris@alchar.org, jan@docarch.be Kris Van Hees & Jan Engelen 20/16