Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, and Challenges Jean Vanderdonckt Université catholique de Louvain (UCL) Louvain School of Management (LSM) Information Systems Unit (ISYS) Belgian Laboratory of Computer-Human Interaction (BCHI) http://www.isys.ucl.ac.be/bchi Place des Doyens, 1 – B-1348 Louvain-la-Neuve (Belgium)

Talk’s outline What is model-based UI design? What is a UI model? What do we need to get a quality UI model? How many models do we need to characterize a UI? From model-based UI design to model-driven UI engineering Three dimensions What are the models? Which language? What is the methodological approach to manipulate them? What is the tool support? Promises Successes vs failures Challenges What can we do today together?

What is model-based UI design?

What is a UI model?

What do we need to get a quality UI model?

How many models do we need to characterize a UI?

From model-based UI design to model-driven UI engineering

Three dimensions

What are the models? Which language?

What is the methodological approach to manipulate them?

What is the tool support?

Promises

Successes vs failures

Challenges

What can we do today together?

What is the today’s situation regarding user interfaces? Technological aspects of user interfaces progress significantly faster than Software engineering aspects It takes time to develop a user interface with a new device, a new interaction technique It takes more time to develop a toolkit It takes even more time to rely on a model-driven approach Usability engineering aspects New user interfaces are shipped with usability problems because Little or no experience No past, no empirical evidence Empirical experiments require a lot of resources if done carefully [Dragicevic et al.,2004]

Technological aspects of user interfaces progress significantly faster than

Software engineering aspects

It takes time to develop a user interface with a new device, a new interaction technique

It takes more time to develop a toolkit

It takes even more time to rely on a model-driven approach

Usability engineering aspects

New user interfaces are shipped with usability problems because

Little or no experience

No past, no empirical evidence

Empirical experiments require a lot of resources if done carefully

What is model-based UI design? Classical approch to developing UIs with a UI builder (IDE) Describe it, prototype it, code it with trial and errors, repeat

Classical approch to developing UIs with a UI builder (IDE)

Describe it, prototype it, code it with trial and errors, repeat

What is model-based UI design? UI builders offer only partial solution Desired Interface Table with dynamic data Gantt chart Direct manipulation of widgets UI Builder Solution Window Menu bar Palette (icons) Scrollbars Other widgets: drop-down list, etc.

UI builders offer only partial solution

Table with dynamic data

Gantt chart

Direct manipulation of widgets

Window

Menu bar

Palette (icons)

Scrollbars

Other widgets: drop-down list, etc.

What is model-based UI design? UI builders cannot produce their own UI (no bootstrapping)

UI builders cannot produce their own UI (no bootstrapping)

What is model-based UI design? Classical approch to developing UIs Advantages UI is graphical by nature A UI can be Rapidly prototyped Easily modified Visually demonstrated Shortcomings No structured method for layout and programming Selection of widget can be inappropriate Layout of widget can be tedious, repetitive Problem of the spaghetti of callbacks Any UI modification can lead to unstructured design

Classical approch to developing UIs

Advantages

UI is graphical by nature

A UI can be

Rapidly prototyped

Easily modified

Visually demonstrated

Shortcomings

No structured method for layout and programming

Selection of widget can be inappropriate

Layout of widget can be tedious, repetitive

Problem of the spaghetti of callbacks

Any UI modification can lead to unstructured design

What is model-based UI design? Model-based UI design consists of Describing a UI in a UI model Relying on this UI model to drive the UI development lifecycle While pursuing the following goals: To provide comprehensive UI development environments To deliver robust development lifecycle support To improve portability To integrate usability with UI development A structure in 3 dimensions Models UsiXML Language Step-wise method Software tools support involves described in Development methodology

Model-based UI design consists of

Describing a UI in a UI model

Relying on this UI model to drive the UI development lifecycle

While pursuing the following goals:

To provide comprehensive UI development environments

To deliver robust development lifecycle support

To improve portability

To integrate usability with UI development

A structure in 3 dimensions

What is model-based UI design? What is a model? Webster’s definitions: « One who is employed to display clothes or other merchandise » « A set of plans for a building » (good analogy) « A system of postulates, data, and inferences presented as mathematical description of an entity or state of affairs » Our definition A structured set of plans for developing a UI A system of postulates, data, and inferences presented as a UI description (or specification) that drives the UI development lifecycle

What is a model?

Webster’s definitions:

« One who is employed to display clothes or other merchandise »

« A set of plans for a building » (good analogy)

« A system of postulates, data, and inferences presented as mathematical description of an entity or state of affairs »

Our definition

A structured set of plans for developing a UI

A system of postulates, data, and inferences presented as a UI description (or specification) that drives the UI development lifecycle

What is model-based UI design? What is a model? A simple example for understanding purposes: a system login Finding the right model abstractions is NOT Replicating code in another way Capturing all physical properties

What is a model?

A simple example for understanding purposes: a system login

Finding the right model abstractions is NOT

Replicating code in another way

Capturing all physical properties

What is model-based UI design? What is a model? Finding the right model abstractions IS Turning important aspects into UI design options E.g. LabelAlignement instead of absolute coordinates Deriving final properties from these options E.g., compute coordinates from LabelAlignement = {left, right, aligned, centred, justified} LabelAlignement = left ButtonPlacement = TotalEquil

What is a model?

Finding the right model abstractions IS

Turning important aspects into UI design options

E.g. LabelAlignement instead of absolute coordinates

Deriving final properties from these options

E.g., compute coordinates from LabelAlignement = {left, right, aligned, centred, justified}

What is model-based UI design? What is a model? Finding the right model abstractions IS Describing UI widgets independently from technology Why? Identification label Input text Push button

What is a model?

Finding the right model abstractions IS

Describing UI widgets independently from technology

Why?

What is model-based UI design? What is a model? Because different evolutions time Platform User Environment Language

What is a model?

Because different evolutions

What is model-based UI design? What is a model? Because different evolutions

What is a model?

Because different evolutions

What is model-based UI design? What is a model? Because same widget appears in many different technologies Because many different ways to reach the same goal (e.g., selector) Check Box (Windows) BoxArray (Garnet) XmToggleButton (OSF-Motif)

What is a model?

Because same widget appears in many different technologies

Because many different ways to reach the same goal (e.g., selector)

What is model-based UI design? What is a model? In order to derive UIs systematically GrafiXML

What is a model?

In order to derive UIs systematically

What is model-based UI design? What is a model? In order to derive UIs systematically GrafiXML

What is a model?

In order to derive UIs systematically

What is model-based UI design? What is a model? In order to derive UIs systematically Edit the UsiXML Show attributes A click on the tree highlights the corresponding UsiXML GrafiXML

What is a model?

In order to derive UIs systematically

What is model-based UI design? What do we need to get a quality UI model? Abstractions are always limited, but extensible Plateau effect: Do not introduce too many, otherwise too complex (<> simple) Do not introduce too few, otherwise too simplistic (<> simple) C1: Need to ensure quality properties of a model Completeness Consistency Correction Expressiveness … CUI Model CIO graphicalCIO graphicalContainer graphicalIndividualComponent

What do we need to get a quality UI model?

Abstractions are always limited, but extensible Plateau effect:

Do not introduce too many, otherwise too complex (<> simple)

Do not introduce too few, otherwise too simplistic (<> simple)

C1: Need to ensure quality properties of a model

Completeness

Consistency

Correction

Expressiveness

…

What do we have so far? With this first set of abstractions, we go from Final UI (FUI) to Concrete UI (CUI) Consequently, a CUI is independent from Any language: (X)HTML, Java, Visual Basic, C++ Any computing platform: Windows, Linux, MacOS Concrete user Interface S Final user Interface S

With this first set of abstractions, we go from Final UI (FUI) to Concrete UI (CUI)

Consequently, a CUI is independent from

Any language: (X)HTML, Java, Visual Basic, C++

Any computing platform: Windows, Linux, MacOS

And now what? UIs are not only graphical, but could be vocal, tactile, 3D, haptic, an multimodal So, there is a need for more abstraction Abstract Container Abstract Individual Component with Output Abs. Ind. Comp. with Input Abs. Ind. Comp. with Control

UIs are not only graphical, but could be vocal, tactile, 3D, haptic, an multimodal

So, there is a need for more abstraction

What do we have so far? With this second set of abstractions, we go from CUI to Abstract UI (AUI) Consequently, a AUI is independent from Any interaction modality Concrete user Interface S Final user Interface S Abstract user Interface S

With this second set of abstractions, we go from CUI to Abstract UI (AUI)

Consequently, a AUI is independent from

Any interaction modality

And now what? UIs, even if they are abstract, are structured according to the target system as opposed to be user-centered So, there is a need for an abstraction that is computing independent Task and domain models (UML Class Diagram) System Login Enter information Check U+P []>> Enter Username Enter Password ||| Check existence Verify correspondence |||

UIs, even if they are abstract, are structured according to the target system as opposed to be user-centered

So, there is a need for an abstraction that is computing independent

Task and domain models (UML Class Diagram)

What do we have so far? With this third set of abstractions, we go from AUI to Task and Domain (T+D) Consequently, a T+D is independent from any implementation Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S

With this third set of abstractions, we go from AUI to Task and Domain (T+D)

Consequently, a T+D is independent from any implementation

Do we need more models? Users want to determine their path on each platform [Forrester Research,2003]

Users want to determine their path on each platform

Do we need more models? Demo FlashiXML

Do we need more models? Multiplicity of contexts of use TV is multi-media family device #1 Family Device Booking notification Everywhere connectivity for simple data exchange Travelling Travel booking site Powerful interface for complex operations Working Multimedia Travel programme Sporting Experience Role Location

Multiplicity of contexts of use

Do we need more models? Property of plasticity = adaptation to the context of use while satisfying predefined usability properties of interest [Grolaux et al.,2002] Demo FlexClock

Property of plasticity = adaptation to the context of use while satisfying predefined usability properties of interest

So what we have now is Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use User S [Calvary et al.,2003] In GrafiXML Platform S Environment S

Multi-platform for Emergency Three platforms Pocket PC Desktop PC Wall Screens

Three platforms

Pocket PC

Desktop PC

Wall Screens

Multi-platform for Emergency Model and method Design the reference screen first Refine the others screens later By applying graceful degradation By applying transformation techniques

Model and method

Design the reference screen first

Refine the others screens later

By applying graceful degradation

By applying transformation techniques

Graceful degradation [Florins & Vanderdonckt,2004]

Transformation rules

Transformation rules Add all >> Add > << Remove all < Remove >> > << < > < Group box Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 1 Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7

Transformation rules

What do we have now Cameleon Reference Framework Final user Interface T Concrete user Interface T Task and Domain T Abstract user Interface T T=Target context of use Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use http://www.plasticity.org UsiXML unsupported model UsiXML supported model User S User T [Calvary et al.,2003] Environment T Reification Abstraction Reflexion Translation Platform S Environment S Platform T

Cameleon Reference Framework

Other challenges raised C2: Need to cover Semantics: metamodel as a UML Class Diagram Syntax: UsiXML language Stylistics: C3: Minimum amount of models C4: Maximum amount of models Root insert search criteria view results aux show item details show list items select item insert criteria submit edit edit criteria submit view results show view details view items show select

C2: Need to cover

Semantics: metamodel as a UML Class Diagram

Syntax: UsiXML language

Stylistics:

C3: Minimum amount of models

C4: Maximum amount of models

Towards model-driven engineering Possible definition MDA is an OMG initiative that proposes to define a set of non-proprietary standards that will specify interoperable technologies with which to realize model-driven development with automated transformations. Not all of these technologies will directly concern the transformation involved in MDA. MDA does not necessarily rely on the UML, but, as a specialized kind of MDD (Model Driven Development), MDA necessarily involves the use of model(s) in development, which entails that at least one modeling language must be used. Any modeling language used in MDA must be described in terms of the MOF language to enable the metadata to be understood in a standard manner, which is a precondition for any activity to perform automated transformation .

Possible definition

MDA is an OMG initiative that proposes to define a set of non-proprietary standards that will specify interoperable technologies with which to realize model-driven development with automated transformations. Not all of these technologies will directly concern the transformation involved in MDA. MDA does not necessarily rely on the UML, but, as a specialized kind of MDD (Model Driven Development), MDA necessarily involves the use of model(s) in development, which entails that at least one modeling language must be used. Any modeling language used in MDA must be described in terms of the MOF language to enable the metadata to be understood in a standard manner, which is a precondition for any activity to perform automated transformation .

Towards model-driven engineering C5: Keep decision decision as SDLC is evolving Support for annotation-based design

C5: Keep decision decision as SDLC is evolving

Support for annotation-based design

Example of preference-based design Goal = to input the temperature of a human body Solution n°1: edit box Solution n°2: list box Solution n°3: drop down list Solution n°4: field with scroll bar Solution n°5: thermometer

Goal = to input the temperature of a human body

Solution n°1: edit box

Solution n°2: list box

Solution n°3: drop down list

Solution n°4: field with scroll bar

Solution n°5: thermometer

How to argue for preference? Use the QOC notation Question? [McLean & Belotti,1998] Criteria 1 Criteria 2 Criteria j Criteria m Option 1 Option 2 Option i Option n = negatively affects = positively affects

Use the QOC notation

Preference example Problem Solution 1 Solution 2 (RE1) (RE2) suggests suggests avoids contradicts respects = for = against (A1) = drop down list requires less space than radio buttons (A2) = some possible values become obsolete when they are infrequently used (A3) = drop down list shows better then current value than the possible values (A4) = radio buttons are faster and easier to manipulate than drop down list (A5) = radio buttons are recommended in Microsoft Windows style guide (A6) = radio buttons do not allow users to change the values drop down list radio buttons (A4) is contradicted by (A2) and (A3) : the widget should be more used for output than input. (A6) is contradicted by (A3) : it is better to present all possible values than only one at a time (A5) is an autority argument , and can fall in front of (A1)+(A2)+(A3) [Vanderdonckt,1997] (A1) (A2) (A3) (A4) (A5) (A6) Standard compatibility (e.g. Windows) Cognitive respect Minimal actions Dialog representativity Prompting Clarity contradicts

Theory of argumentation [Toulmin,1975] [Perelman, 1978] Design problem Design option parameter (parameter, value) Is described by Is solved by Accepted solution Set of admissible solutions Solution 1 Solution 2 Solution n Counter-argument (rebuttal, counter-claim) Argument (ground, typically data) can be rejected because of Can be accepted thanks to Is justified by Design claim Warrant Is reinforced by Qualifier satisfied unsatisfied Is related to corresponds to Is justified by Is qualified by

Towards model-driven engineering C6: Need to support (de)composition Demo FormiXML

C6: Need to support (de)composition

Towards Model-Driven Engineering C7: Support for multi-path development Different types of engineering Forward engineering Reverse engineering Lateral engineering Diagonal engineering (with or without shortcuts) Different approaches Top-down Bottom-up Middle-out Different development paths Example: Round-trip engineering = composition of Reification CUI -> FUI Reflexion FUI -> FUI Abstraction FUI -> CUI Reflexion CUI -> CUI

C7: Support for multi-path development

Different types of engineering

Forward engineering

Reverse engineering

Lateral engineering

Diagonal engineering (with or without shortcuts)

Different approaches

Top-down

Bottom-up

Middle-out

Different development paths

Example: Round-trip engineering = composition of

Reification CUI -> FUI

Reflexion FUI -> FUI

Abstraction FUI -> CUI

Reflexion CUI -> CUI

Revisitation [Bouillon,2006]

Towards Model-Driven Engineering C8: Support for multi-fidelity Demo SketchiXML

C8: Support for multi-fidelity

Towards model-driven engineering Other challenges (C13 -> C16) High ceiling Low threshold Wide walls Capabilities Resources (time, experience,…) 100% 50% Ceiling Threshold First generation Second generation MDA CASE tools Interface builders and integrated environments Resource win for applications supported by MDA-compliant tools Resource win for applications supported by first-generation

Other challenges (C13 -> C16)

High ceiling

Low threshold

Wide walls

Towards model-driven engineering Other challenges (C13 -> C16) High ceiling Low threshold Wide walls Capabilities Resources (time, experience,…) 100% 50% Ceiling Threshold First generation Second generation Third generation Integrated Development Environments UI types Walls

Other challenges (C13 -> C16)

High ceiling

Low threshold

Wide walls

Towards model-driven engineering C17: Effort incrementality and C19: rendering engines UsiXML interpreter for Graphical user interfaces: XHTML, XUL, Java, Flash, Tcl-Tk Vocal user interfaces: VoiceXML Multimodal user interfaces: X+V Haptic user interfaces: HaptiXML Support for Distributed user interfaces Mixed-reality user interfaces

C17: Effort incrementality and C19: rendering engines

UsiXML interpreter for

Graphical user interfaces: XHTML, XUL, Java, Flash, Tcl-Tk

Vocal user interfaces: VoiceXML

Multimodal user interfaces: X+V

Haptic user interfaces: HaptiXML

Support for

Distributed user interfaces

Mixed-reality user interfaces

Developed by Donatien Grolaux Problem: how to design a UI that takes care of multiple displays? Solution: Migration = DEMIPLAT DistriXML = software architecture for migrating UIs from one platform to another at run-time DistriXML [Grolaux & Vanderdonckt,2005] Pencil Palette Painting Painting tool

Problem: how to design a UI that takes care of multiple displays?

Solution: Migration = DEMIPLAT

DistriXML = software architecture for migrating UIs from one platform to another at run-time

Demonstration DistriXML [Grolaux & Vanderdonckt,2005]

Demonstration using two displays from two different computers DistriXML

Example using a Pocket PC DistriXML

UI Migration: DEMIPLAT De tach DistriXML

De tach

UI Migration: DEMIPLAT De tach - Mi grate DistriXML

De tach - Mi grate

UI Migration: DEMIPLAT De tach - Mi grate - Pl astify DistriXML

De tach - Mi grate - Pl astify

UI Migration: DEMIPLAT De tach - Mi grate - Pl astify - At tach DistriXML

De tach - Mi grate - Pl astify - At tach

This is not a floating toolbar Process DistriXML

This is migration ! Computer B Process Process Computer A DistriXML

Towards model-driven engineering C18: More dynamic aspects Mixed reality Minority report

C18: More dynamic aspects

Mixed reality

Conclusion What can be really supported? [Skezely,1996] 0 2000 4000 6000 8000 10000 12000 14000 16000 Word KB KB Parser Query Network ops. UI states Presentation Actions Update Interactions Application logic User interface Generated code Models

What can be really supported?

Conclusion Successes: Work for very-well defined domains Really fosters collaboration Supports flexibility, maintainability (Belgium) Failures Does not work for very complex, dynamic UIs Takes a lot of resources for Models (some) Method (more) Tool (even more) Support for mnemonics and shortcuts

Successes:

Work for very-well defined domains

Really fosters collaboration

Supports flexibility, maintainability (Belgium)

Failures

Does not work for very complex, dynamic UIs

Takes a lot of resources for

Models (some)

Method (more)

Tool (even more)

How to participate? Join the UsiXML Consortium today! Anybody can be a member: observer, participant, developer Send an e-mail to [email_address] and/or « Register » on the web site: for this purpose, provide Your firstname, last name, affiliation, desired level of membership, and motivations Think of your own research in terms of models, e.g., Instead of evaluating a Graphical UI for a certain platform, conduct its evaluation on the corresponding CUI model: it is more general, you will gain more audience Instead of developing a separate tool, make it UsiXML-compliant: you will gain more audience, the results are more largely applicable Output UsiXML session at conferences UsiXML CD-ROM, etc.

Join the UsiXML Consortium today!

Anybody can be a member: observer, participant, developer

Send an e-mail to [email_address] and/or « Register » on the web site: for this purpose, provide

Your firstname, last name, affiliation, desired level of membership, and motivations

Think of your own research in terms of models, e.g.,

Instead of evaluating a Graphical UI for a certain platform, conduct its evaluation on the corresponding CUI model: it is more general, you will gain more audience

Instead of developing a separate tool, make it UsiXML-compliant: you will gain more audience, the results are more largely applicable

Output

UsiXML session at conferences

UsiXML CD-ROM, etc.

How to participate? Develop web services for UsiXML Open positions in UsiXML project (from April 2009) PhD students Internships for MSc students

Develop web services for UsiXML

Open positions in UsiXML project (from April 2009)

PhD students

Internships for MSc students

Thank you very much for your attention For more information and downloading, http://www.isys.ucl.ac.be/bchi http://www.usixml.org User Interface eXtensible Markup Language http://www.similar.cc European network on Multimodal UIs