Your SlideShare is downloading. ×
0
Model-Driven Engineering of User Interfaces Jean Vanderdonckt Université catholique de Louvain (UCL) Louvain School of Man...
Course outline <ul><li>Day 1:  Why  do we need to model the user interface? </li></ul><ul><ul><li>H1: Motivations and intr...
Course outline <ul><li>Day 3:  What  do we need to model to cover UI aspects? (Part 2) </li></ul><ul><ul><li>H1: Concrete ...
Course outline <ul><li>Day 5:  How  do we assess the UI modelled? </li></ul><ul><ul><li>H1: Automated evaluation of guidel...
What is the situation today? <ul><li>Technological aspects of user interfaces progress significantly faster than </li></ul...
Adapted from [Palanque,2002] Target Applications, Domains  Notations and tools User Interface Interaction Techniques 2006 ...
What’s the situation of today? <ul><li>Interactive Software evolution: context of use =(U,P,E) </li></ul>time Day 1: Why? ...
Software evolution Day 1: Why?
Diversity of users <ul><li>Traditional users </li></ul><ul><li>People with disabilities </li></ul><ul><ul><li>Visual, moto...
Variety of tasks <ul><li>Users want to determine their path on each platform </li></ul>Day 1: Why? [Forrester Research,2003]
Heterogeneousness of platforms Day 1: Why?
Multiplicity of contexts of use Day 1: Why? TV is multi-media family device #1 Family Device Booking notification Everywhe...
Consequence <ul><li>Proliferation of developments </li></ul>Day 1: Why? [Abrams et al.,2001] UI #12 UI #11 UI #10 UI #9 Ap...
Is this web site usable? Day 1: Why?
Ergonomic criteria <ul><li>Criteria for assessing the usability of any UI </li></ul><ul><ul><li>A priori and/or a posterio...
Navigation <ul><li>Use a reactive image as a navigation map </li></ul>Day 1: Why? No map without any relation Metaphoric map
Navigation <ul><li>Include navigation clues </li></ul>Day 1: Why? Navigation bar Landmark Global vs local diagram
Navigation <ul><li>Navigation clues should be consistent </li></ul>Day 1: Why? Inconsistencies between the menu and naviga...
Navigation <ul><li>No more &quot;Clic here&quot; </li></ul>Day 1: Why? Clic  here  to go to the Publications page Clic  he...
Navigation <ul><li>Do not use &quot;Return to...&quot;  links </li></ul>Day 1: Why? Aller à :... Button Avoid : come back,...
Presentation <ul><li>Information wich are semantically related should be presented together </li></ul>Day 1: Why? Related ...
Presentation <ul><li>Example of a web page format (before) </li></ul>Day 1: Why? Display zone of browser navigation button...
Presentation <ul><li>How to format? Position of home page link, internal links </li></ul>Day 1: Why? [Shaikh & Lenz,2006]
Presentation <ul><li>How to format? Position of search engine, advertisements, about us </li></ul>Day 1: Why? [Shaikh & Le...
Presentation <ul><li>Example of a web page format (after) </li></ul>Day 1: Why? Logo to home page Site menus zone Informat...
Simple choice Mixed domain: Know domain: Day 1: Why? [Vanderdonckt & Faulkner,2002] Unknown domain  : Amount of possible v...
Choix multiple Mixed domain: Known domain: Day 1: Why? [Vanderdonckt & Faulkner,2002] Domaine inconnu : Amount of possible...
Graphics <ul><li>Use graphics for headers </li></ul>Day 1: Why? Use moderately artistic fonts for headers Keep hear in a s...
Multimedia elements <ul><li>The background should be appropriate for the task </li></ul>Use various horizon lines dependin...
Multimedia elements <ul><li>Low horizon line </li></ul>Day 1: Why?
Multimedia elements <ul><li>High horizon line </li></ul>Day 1: Why?
Multimedia elements <ul><li>Median horizon line </li></ul>Day 1: Why?
Multimedia elements <ul><li>Diagonal horizon line </li></ul>Day 1: Why?
Multimedia elements <ul><li>Sharp horizon line </li></ul>Day 1: Why?
Multimedia elements <ul><li>Do prefer low-intensity, light backgrounds </li></ul>Day 1: Why? No white on black
Multimedia elements <ul><li>Do not use backgrounds with textures </li></ul>Day 1: Why? Pale backgrounds, submit and test P...
<ul><li>Keep the good color combinations </li></ul>Multimedia elements Day 1: Why? [Murch,1987] Blue (94%)  Black (63%) Re...
<ul><li>Keep the good color combinations </li></ul>Multimedia elements Day 1: Why? [Murch,1987] Black (69%) Blue (63%)  Re...
<ul><li>Avoid the bad color combinations </li></ul>Multimedia elements Day 1: Why? [Murch,1987] Yellow (100%) Cyan (94%)  ...
<ul><li>Avoid the bad color combinations </li></ul>Multimedia elements Day 1: Why? [Murch,1987] Yellow (95%) Cyan (75%)  B...
Multimedia elements <ul><li>Consider font variations induced by different platforms </li></ul>Day 1: Why? Fixed size font ...
IFIP Quality Properties for UIs <ul><li>Ten properties in 3 categories </li></ul><ul><li>Category 1: information presentat...
IFIP Quality Properties for UIs <ul><li>Multiplicity of devices and representations </li></ul>Day 1: Why? [Stephanidis,2001]
Derivation panel with preview CTTE Operator Operator parameters Design comment resulting from Applying guidelines Design P...
Plastic UI <ul><li>Example: the Virtual keyboard </li></ul>Day 1: Why? [Grolaux et al.,2001]
Plastic UI <ul><li>Plastic UI: adaptable bounded value </li></ul>Day 1: Why? [Vanderdonckt et al.,2005]
Plastic User interface <ul><li>Property of plasticity = adaptation to the context of use while satisfying predefined usabi...
How to address this issue? <ul><li>Capture essence of concepts through models </li></ul><ul><ul><li>Separation of concerns...
Typical models
What do we want to get? Final user Interface T Concrete user Interface T Task and  Domain T Abstract user Interface T T=Ta...
Software engineering interpretation <ul><li>Different types of engineering </li></ul><ul><ul><li>Forward engineering </li>...
IEEE Reverse Engineering Taxonomy Reference:  Elliot J. Chikofsky , James H. Cross II, Reverse Engineering and Design Reco...
Revisitation [Bouillon,2006]
Multi-Path Development of UI <ul><li>Multi-path development qualifies a methodology that support the realization of multip...
Our goals <ul><li>Objectives </li></ul><ul><ul><li>Description of interactive systems </li></ul></ul><ul><ul><ul><li>Using...
UsiXML <ul><li>UsiXML = </li></ul><ul><ul><li>USer Interface exTensible Markup Language </li></ul></ul><ul><ul><li>XML-com...
UsiXML =  User Interface eXtensible Markup Language <ul><li>What is UsiXML? </li></ul><ul><ul><li>It is a XML-compliant Us...
Adapted from [Palanque,2002] Target Applications, Domains  Notations and tools User Interface Interaction Technique 2006 T...
The problem <ul><li>To develop user interfaces (UIs) simultaneously for multiple contexts of use </li></ul><ul><li>A conte...
The problem <ul><li>Why is it difficult? </li></ul><ul><ul><li>For the designer: </li></ul></ul><ul><ul><ul><li>Multiplici...
The problem <ul><li>Why should it be systematic? </li></ul><ul><ul><li>Some consider it noble to have a method </li></ul><...
Mono-platform UI <ul><li>Traditional approach </li></ul><ul><ul><li>Visual development </li></ul></ul>
Mono-platform UI <ul><li>Advantages of traditional approach </li></ul><ul><ul><li>UI is graphical by nature </li></ul></ul...
Mono-platform UI <ul><li>Shortcomings of traditional approach </li></ul><ul><ul><li>No structured method for developing a ...
Mono-platform UI <ul><li>Shortcomings of traditional approach </li></ul><ul><ul><li>Limited use of UI builder </li></ul></...
Mono-platform UI <ul><li>Interface builders cannot build their own UI </li></ul>[Szekely,1996]
Mono-platform UI <ul><li>Any development method (or methodology) is decomposed into 3 axes: </li></ul><ul><ul><li>Models :...
Mono-platform UI <ul><li>Goal: to integrate all three facets </li></ul>Interface 1 Method Model 1 Model … Model  n Models ...
MDE based on UsiXML Model to Model Platform Independent Model (PIM) Platform Specific Model (PSM) Model to Code Source cod...
CIM Step 1: Task model
New Abstraction: the user’s task <ul><li>Task = set of actions carried out by a user in a given context to reach a goal </...
New Abstraction: the user’s task <ul><li>Task definition = action + object </li></ul><ul><ul><li>Action types </li></ul></...
New Abstraction: the task meta-model [Limbourg,2004]
CIM Step 2: domain model [Limbourg,2004]
CIM Step 3: Task-domain mappings IdealXML [Limbourg,2004]
New Abstraction: the abstract UI <ul><li>Different CIOs can be used for the same purpose, but with different interaction m...
Abstraction: the abstract UI
Abstraction: the abstract UI <ul><li>Notation: based on L. Constantine’s notation for canonical abstract prototypes </li><...
IdealXML [Montero,2005]
Example of AUI produced
Mapping the models [Montero,2005] triggers (tg): {  ,  } x updates (up):   x observes (ob):   x isExecutedIn (ex):   x man...
Mapping the models <ul><li>Mapping the models with a mapping model (!!) </li></ul>
Your turn now! <ul><li>Virtual Polling System </li></ul><ul><ul><li>Characterization of a participant </li></ul></ul><ul><...
Example solution (first variant)
Example solution (first variant)
Task and domain models [Montero,2005]
Typed Model-to-Model Transformation Uses language Meta-Meta-Model Graph Structure  is instance of Meta-Model Our Meta-Mode...
Expression of models as graphs <ul><li>All transformations are in UsiXML </li></ul><ul><ul><li>Each model = instance of me...
Definition of a production <ul><li>Find an occurrence of LHS in G (this occurrence is called a match). If several occurren...
Transformation system [Limbourg,2004] G Host USIXM specification G’ Resultant USIXM specification LHS RHS Matches - Co-Mat...
PIM step: task+domain to AUI <ul><li>Abstract UI (AUI) = UI independent of any interaction modality </li></ul><ul><li>Defi...
PIM step: task+domain to AUI Identification of AUI structure Spatio-Temporal Arrangement of AIOs Selection of AIC Definiti...
How to read a graph transformation? Node type Node (Attribute,value) Edge type Edge (Attribute,value) Node Edge
Rule 1 LHS RHS ::= Ø
Rule 2 LHS RHS ::=
Rule 3 LHS RHS ::=
Rule 4 LHS RHS ::= NAC
Rule 5 LHS RHS ::= NAC1 NAC2
Rule 6 LHS RHS ::=
Rule 7  LHS RHS ::= PAC “ X < 3000”
Rule 8  LHS RHS ::= PAC “ X > 3000”  NAC
Rule 9 LHS RHS ::=
Rule 10 PAC LHS RHS ::= “ X > Y”
PIM sub-step 1: Definition of AUI structure (AC)
PIM sub-step 1: Definition of AUI structure (AC)
Facet type
 
AC Participate to OpinionPoll AC  Answer Poll AC  Answer Questionnaire AC  Answer Question AIC (Input/Single Selection) Se...
PIM sub-step 2: definition of AIC types  AC = Abstract Container AIC = Abstract Individual Component CIC = Concrete Intera...
PIM sub-step 3:  Definition of spatio-temporal arrangement
PSM Step: AUI to CUI <ul><li>Concrete UI (CUI) = UI independent of toolkit </li></ul><ul><li>Definition of CUI structure <...
PSM Step: AUI to CUI Reification of AC into CC Arrangement of CICs Selection of CIC Concrete Dialog Control Definition STE...
PSM sub-step 3: definition of navigation An example of a complex rule
PSM: Concrete User Interface
Example: Platform adaptation widget substitution (1) <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot; standalone=&...
Example: widget substitution (2)
Example: widget substitution (3) The UsiXML graph before applying any rule
Example: widget substitution (4) LHS RHS NAC Rule 1: Create a new comboBox with the same  id  and  name  as the name of th...
Example: widget substitution (5) Rule 1: Create a new comboBox with the same  id  and  name  as the name of the group of r...
Example: widget substitution (6) LHS RHS ::= Rule 2: Convert every radio button within the group “x” into an item for the ...
Example: widget substitution (7) Rule 2: Convert every radioButton within the group “x” into an item for the comboBox “x”,...
Example: widget substitution (8) <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot; standalone=&quot;yes&quot;?> < c...
Example: widget substitution (9)
What do we have so far? Concrete user Interface S Final user Interface S Task and  Domain S Abstract user Interface S S=So...
Multiple development paths
Mapping the Methodological World onto the Transformation World A development library stores (in usiXML textual format) pat...
Multiple development paths Rule n Transformation  System 1 Rule 1 Rule 2 … Rule n Transformation  System 2 Rule 1 Rule 2 …...
TransformiXML API/GUI <ul><li>API = set of transformations  </li></ul>
From T&D to AUI <ul><li>TransformiXML </li></ul>TransformiXML [Bouillon et al.,2005]
Final user interface <ul><li>Two forms of UI rendering </li></ul><ul><ul><li>Interpretation </li></ul></ul><ul><ul><ul><li...
What is a Feature Model? C F1 C F1 F2 C F1 F2 Dependencies 1. 2. Optional Exclusive Alternate [Schlee & Vanderdonckt,2004]
Example of a Feature Model Alternate [Schlee & Vanderdonckt,2004]
Interpretation of a feature model C F1 F2 F3 F2a F2b F3a F3b C F1 F2 F3 F2a F2b F3a F3b 0 1 1 1 1 0 0 UsiXML Specification...
Generative Programming void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange  (T*, Pl* ppl = NULL  ); voi...
Developed by Benjamin Michotte <ul><li>GrafiXML consists of a user interface builder for designing a graphical user interf...
Design Tab Design window Components toolbar Components options
Localisation of UIs <ul><li>GrafiXML allows the user to create multi-language GUI </li></ul>Support for mnemonics and shor...
Preview <ul><li>At any time, you can preview the UI in the language you want </li></ul>
XML Editor <ul><li>GrafiXML contains a XML editor which shows the UsiXML specification of your work </li></ul><ul><li>You ...
Plugins <ul><li>GrafiXML works with plugins </li></ul><ul><ul><li>Install / remove using the plug-in manager </li></ul></u...
What do we have so far? Concrete user Interface S Final user Interface S Task and  Domain S Abstract user Interface S S=So...
Reverse engineering <ul><li>From FUI to CUI </li></ul><ul><li>From CUI to AUI, task & domain </li></ul>
From FUI to CUI <ul><li>Do you have the source code? </li></ul><ul><ul><li>Markup languages (e.g., HTML): static code anal...
From compiled code [Marion,2005]
The calculator <ul><li>Begin VB.Form Calculator BorderStyle = 1 'Fixed Single Caption = &quot;Calculator&quot; ClientHeigh...
From CUI to AUI, task & domain <ul><li>Re-engineering = reverse + forward </li></ul><ul><li>Possible re-interpretation by ...
Example of Derivation Rules <ul><li>Examples: </li></ul><ul><ul><li>For a textbox in HTML/WML </li></ul></ul><ul><li>   x...
Derivation rules categories <ul><li>Rules can be classified into different categories, outlining the common issues in a RE...
RE of Web Pages: ReversiXML <ul><ul><ul><li>Written in PHP5 </li></ul></ul></ul><ul><ul><ul><li>On-line RE </li></ul></ul>...
RE of Phone Interfaces <ul><li>Major working hypotheses </li></ul><ul><ul><li>Static RE of WML 1.1 and voiceXML 2.0 up to ...
RE of WML Phone Interfaces <ul><li>Example </li></ul><ul><ul><li>Prototype using XSLT + XPATH </li></ul></ul><ul><ul><li>L...
Results of the Evaluation <ul><li>Case studies illustrating various HTML RE techniques </li></ul><ul><li>Study of reengine...
Results of the Evaluation <ul><li>With Teresa </li></ul><ul><ul><li>RE applied without retargeting </li></ul></ul><ul><ul>...
Results of the Evaluation <ul><li>With QtkiXML </li></ul>Original UI Reengineered UI
What do we have so far? Concrete user Interface S Final user Interface S Task and  Domain S Abstract user Interface S S=So...
What do we want to get? Final user Interface T Concrete user Interface T Task and  Domain T Abstract user Interface T T=Ta...
Rules for Graceful Degradation <ul><li>Constitution of an original corpus of rules </li></ul><ul><li>Typology of rules, fo...
Rules at the Concrete UI level <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul>...
<ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul></ul><ul><ul><li>Modification r...
Rules at the Concrete UI level <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul>...
Rules at the Concrete UI level <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul>...
Rules at the Concrete UI level <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul>...
Rules at the Concrete UI level <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul>...
Rules at the Abstract UI level <ul><li>Spitting rules </li></ul><ul><li>Consist in breaking the initial UI into chunks </l...
Rules at the Abstract UI level <ul><li>Important because: </li></ul><ul><ul><li>Difficult and significant step: generates ...
Rules at the Abstract UI level Source interface (in the graphical editor GrafiXML) (b) Execution of the splitting rule (a)...
Rules at the Abstract UI level <ul><li>Application of the rule using task level information </li></ul>[Florins,2006]
Rules at the Tasks&Concepts level <ul><li>Task deletion </li></ul><ul><li>Information summarization </li></ul><ul><li>… </...
GD Plug-in (4) <ul><li>Sections of rules </li></ul>[Florins et al.,2006]
GD Plug-in (4) <ul><li>Sections of rules </li></ul>
GD Plug-in (4) <ul><li>Sections of rules </li></ul>
GD Plug-in (5) <ul><li>Rules selection / parameters </li></ul>
GD Plug-in (6) <ul><li>Results </li></ul>
Multi-platform for Emergency [Amouh et al.,2005]
Multi-platform for Emergency <ul><li>Three platforms </li></ul><ul><ul><li>Pocket PC </li></ul></ul><ul><ul><li>Desktop PC...
Multi-platform for Emergency <ul><li>Model and method </li></ul><ul><ul><li>Design the reference screen first </li></ul></...
Graceful degradation [Florins & Vanderdonckt,2004]
Transformation rules
Transformation rules Add all >> Add > << Remove all < Remove >> > << < > < Group box Item 1 Item 2 Item 3 Item 4 Item 5 It...
Transformation rules
Considering the context of use in designing user interfaces <ul><li>Context of use = triple </li></ul><ul><ul><li>User </l...
Context model
Adapted from [Palanque,2002] Target Applications, Domains  Notations and tools User Interface Interaction Technique 2006 T...
Virtualisation of UIs <ul><li>Going from 2D to 3D </li></ul><ul><ul><li>Mapping widgets in 2D to 3D </li></ul></ul>[Vander...
Developed by Donatien Grolaux <ul><li>Problem: how to design a UI that takes care of multiple displays? </li></ul><ul><li>...
Why take care of multiple displays? [Czerwinsky,2005] Effects of Display Size on Task Times 0 20 40 60 80 100 120 140 160 ...
Why take care of multiple displays? The tasks were easy to perform 0 1 2 3 4 5 Small Large Display Size Average Rating (1=...
Why take care of multiple displays? [Czerwinsky,2005]
Demonstration DistriXML [Grolaux & Vanderdonckt,2005]
Demonstration using two displays from two different computers DistriXML
Example using a Pocket PC DistriXML
UI Migration: DEMIPLAT <ul><li>De tach </li></ul>DistriXML
UI Migration: DEMIPLAT <ul><li>De tach -  Mi grate </li></ul>DistriXML
UI Migration: DEMIPLAT <ul><li>De tach -  Mi grate -  Pl astify </li></ul>DistriXML
UI Migration: DEMIPLAT <ul><li>De tach -  Mi grate -  Pl astify -  At tach </li></ul>DistriXML
This is not a floating toolbar Process DistriXML
This is migration ! Computer B Process Process Computer A DistriXML
User model <ul><li>User population = hierarchy of user stereotypes </li></ul><ul><li>User stereotype = set of parameters <...
Why user preference is important? <ul><li>End users tend to manipulate more efficiently the user interface they prefer </l...
Example of preference-based design <ul><li>Goal = to input the temperature of a human body </li></ul><ul><ul><li>Solution ...
Users <ul><li>Properties of the user also constrain interaction. </li></ul><ul><ul><li>Expertise </li></ul></ul><ul><ul><u...
Modeling a range of designs <ul><li>UI modeling languages must model a range of design possibilities to account for differ...
A Spectrum of Preference Relations <ul><li>We describe a spectrum of preference relations </li></ul><ul><ul><li>Less flexi...
The Spectrum <ul><li>Concrete Preferences </li></ul><ul><ul><li>Bindings </li></ul></ul><ul><ul><li>Simple Preference </li...
Modeling preferences in UsiXML <ul><li>UsiXML = User Interface eXtensible Markup Language (http://www.usixml.org) </li></u...
Bindings <ul><li>Bindings are simple one-to-one mappings </li></ul><ul><ul><li>Task A is implemented by interactor X </li>...
Binding example <ul><li>User U1 prefers presentation element P1 </li></ul><PREFERENCE ID=&quot;G1&quot;> <CONDITIONS> <REL...
Simple Preferences <ul><li>Simple Preferences are many-to-one mappings </li></ul><ul><ul><li>For user U, task A is impleme...
Simple preference example <ul><li>User U1 prefers presentation element P1 for representing domain model DM1 </li></ul><PRE...
Bindings <ul><li>Bindings are simple one-to-one mappings </li></ul><ul><ul><li>Task A is implemented by interactor X </li>...
Binding example <ul><li>User U1 prefers presentation element P1 </li></ul><PREFERENCE ID=&quot;G1&quot;> <CONDITIONS> <REL...
Ordered Preferences <ul><li>An ordered preference is a type of many-to-many mapping </li></ul><ul><ul><li>For user U, task...
Ordered preference example <ul><li>User U1 prefers presentation element P1 to presentation element P2, for representing do...
Summary of Concrete Preferences <ul><li>All of the previous are concrete preferences </li></ul><ul><ul><li>They model the ...
Abstract Preferences <ul><li>Abstract preferences do not model the target directly </li></ul><ul><li>They model  character...
Example of Abstract Preference <ul><li>User U prefers the interactor </li></ul><ul><ul><li>Occupying the least screen spac...
Abstract preference example <ul><li>User U1 prefers the presentation element that occupies the least screen space </li></u...
Multiple Preferential Criteria <ul><li>Preferential criteria may have “priorities” </li></ul><ul><ul><li>This specifies th...
Design Guidelines <ul><li>Abstract preferences consider criteria that relate to the targets </li></ul><ul><li>Design guide...
Chaining Abstract Preferences <ul><li>Design guidelines can have other preference relations as their targets: </li></ul><u...
Chaining Design Guidelines <ul><li>Design guidelines can also target other design guidelines </li></ul><ul><li>This can cr...
How to argue for preference? <ul><li>Use the QOC notation </li></ul>Question? [McLean & Belotti,1998] Criteria 1 Criteria ...
Preference example Problem Solution 1 Solution 2 (RE1) (RE2) suggests suggests avoids contradicts respects = for = against...
Theory or argumentation [Toulmin,1975] [Perelman, 1978] Design problem Design option parameter (parameter, value) Is descr...
Mixed reality <ul><li>Mixed reality = real world + digital world </li></ul>
Mixed reality for games [Alterface,2006]
Mixed reality in surgery <ul><li>No representation for </li></ul><ul><ul><li>Tasks in real or virtual world </li></ul></ul...
Composing the MIS Real CIO Patient head Virtual CIOs Anatomic organs Virtual CIOs Annotations Primary object  of interest
Annotation placement strategy  <ul><li>No overlapping annotations for readability </li></ul><ul><li>Highest priority of vi...
Context-aware Multimodal Annotation Rendering (a) Background context (b) Background/Real  Object context (c) Virtual Objec...
Context model in mixed reality <ul><li>A  context model  describes all the entities that may influence carrying out the in...
Plasticity analysis and usability evaluation Plasticity Domains  in the MIS Plasticity thresholds and context  boundary va...
What do we have now? Final user Interface T Concrete user Interface T Task and  Domain T Abstract user Interface T T=Targe...
Conclusion <ul><li>User model is described by parameters </li></ul><ul><ul><li>Context independent  </li></ul></ul><ul><ul...
Evaluation [Skezely,1996] 0 2000 4000 6000 8000 10000 12000 14000 16000 Word KB KB Parser Query Network ops. UI states Pre...
Coverage <ul><li>How to improve our MDA-based method? </li></ul><ul><ul><li>High ceiling </li></ul></ul><ul><ul><li>Low th...
What’s the User Interface language? <ul><li>Today </li></ul><ul><ul><li>Programming languages: Visual Basic, Java, C++,… <...
Conclusion: the big picture of MDA UsiXML model: Abstract user interface UsiXML model: Concrete user interface Rendering F...
The Invisible UI All Applications Adapted from [Palanque,2002] Target Applications, Domains  Notations and tools User Inte...
Thank you very much for your attention For more information and downloading, http://www.isys.ucl.ac.be/bchi   http://www.u...
Upcoming SlideShare
Loading in...5
×

Model-driven engineering of user interfaces

3,551

Published on

This presentation contains the slides of the Doctoral Course given at University of Valencia (Spain) regarding model-driven engineering of user interfaces based on UsiXML (User Interface eXtensible Markup-Language, www.usixml.org), November 2006.

Published in: Economy & Finance, Technology
2 Comments
8 Likes
Statistics
Notes
No Downloads
Views
Total Views
3,551
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
327
Comments
2
Likes
8
Embeds 0
No embeds

No notes for slide
  • Transcript of "Model-driven engineering of user interfaces"

    1. 1. Model-Driven Engineering of User Interfaces Jean Vanderdonckt Université catholique de Louvain (UCL) Louvain School of Management (LSM) Information Systems Unit (ISYS) Belgian Laboratory of Computer-Human Interaction (BCHI) http://www.isys.ucl.ac.be/bchi
    2. 2. Course outline <ul><li>Day 1: Why do we need to model the user interface? </li></ul><ul><ul><li>H1: Motivations and introduction to usability evaluation of user interfaces </li></ul></ul><ul><ul><ul><li>Case studies and evaluation </li></ul></ul></ul><ul><ul><li>H2: Concepts used for usability evaluation for every UI </li></ul></ul><ul><ul><ul><li>Usability guideline, ergonomic criteria, IFIP properties </li></ul></ul></ul><ul><li>Day 2: What do we need to model to cover UI aspects? (Part 1) </li></ul><ul><ul><li>H1: The Cameleon reference framework for multi-target UIs </li></ul></ul><ul><ul><ul><li>Underlying metamodel </li></ul></ul></ul><ul><ul><ul><li>Terminology and revisitation of IEEE Taxonomy of RE </li></ul></ul></ul><ul><ul><ul><li>Task modelling, domain modelling in IdealXML </li></ul></ul></ul><ul><ul><li>H2: Abstract UI </li></ul></ul><ul><ul><ul><li>Mapping model </li></ul></ul></ul><ul><ul><ul><li>Model-to-model transformation in IdealXML </li></ul></ul></ul>
    3. 3. Course outline <ul><li>Day 3: What do we need to model to cover UI aspects? (Part 2) </li></ul><ul><ul><li>H1: Concrete UI </li></ul></ul><ul><ul><ul><li>Model-to-model transformation in TransformiXML </li></ul></ul></ul><ul><ul><ul><li>Model-to-code generation in GrafiXML </li></ul></ul></ul><ul><ul><ul><li>Graph grammars and other techniques </li></ul></ul></ul><ul><ul><li>H2: Final UI </li></ul></ul><ul><ul><ul><li>UI rendering: by interpretation, by compilation </li></ul></ul></ul><ul><ul><ul><li>UI rendering in multiple computing platforms </li></ul></ul></ul><ul><li>Day 4: When do we need to model what to cover UI aspects? </li></ul><ul><ul><li>H1: Multi-path development of UIs </li></ul></ul><ul><ul><ul><li>Forward, reverse, lateral engineering </li></ul></ul></ul><ul><ul><ul><li>UI adaptation: graceful degradation </li></ul></ul></ul><ul><ul><li>H2: Multiple targets </li></ul></ul><ul><ul><ul><li>Multiple users, platforms, environments </li></ul></ul></ul>
    4. 4. Course outline <ul><li>Day 5: How do we assess the UI modelled? </li></ul><ul><ul><li>H1: Automated evaluation of guidelines </li></ul></ul><ul><ul><ul><li>Evolution of code static analysis </li></ul></ul></ul><ul><ul><ul><li>Evaluation for web and non-web applications </li></ul></ul></ul><ul><ul><li>H2: Towards genuine model-checking of UIs </li></ul></ul><ul><ul><ul><li>Usability advisor: natural language evaluation </li></ul></ul></ul><ul><ul><ul><li>Evaluation of usability guidelines, design rules, properties of interest </li></ul></ul></ul><ul><ul><ul><li>Final conclusion: evolution of MDA for UIs </li></ul></ul></ul><ul><ul><ul><ul><li>Low threshold </li></ul></ul></ul></ul><ul><ul><ul><ul><li>High ceiling </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Wide walls </li></ul></ul></ul></ul>
    5. 5. What is the situation today? <ul><li>Technological aspects of user interfaces progress significantly faster than </li></ul><ul><ul><li>Software engineering aspects </li></ul></ul><ul><ul><ul><li>It takes time to develop a user interface with a new device, a new interaction technique </li></ul></ul></ul><ul><ul><ul><li>It takes more time to develop a toolkit </li></ul></ul></ul><ul><ul><ul><li>It takes even more time to rely on a model-driven approach </li></ul></ul></ul><ul><ul><li>Usability engineering aspects </li></ul></ul><ul><ul><ul><li>New user interfaces are shipped with usability problems because </li></ul></ul></ul><ul><ul><ul><ul><li>Little or no experience </li></ul></ul></ul></ul><ul><ul><ul><ul><li>No past, no empirical evidence </li></ul></ul></ul></ul><ul><ul><ul><li>Empirical experiments require a lot of resource if done carefully </li></ul></ul></ul>Day 1: Why? [Dragicevic et al.,2004]
    6. 6. Adapted from [Palanque,2002] Target Applications, Domains Notations and tools User Interface Interaction Techniques 2006 TODAY No Interaction Technique Automated, batch systems Nothing Character UIs Business applications Screen definitions Graphical User Interfaces Information systems Entity-relationship Attribute model State-transition diagrams
    7. 7. What’s the situation of today? <ul><li>Interactive Software evolution: context of use =(U,P,E) </li></ul>time Day 1: Why? Platform User Environment Language
    8. 8. Software evolution Day 1: Why?
    9. 9. Diversity of users <ul><li>Traditional users </li></ul><ul><li>People with disabilities </li></ul><ul><ul><li>Visual, motor, cognitive </li></ul></ul>Day 1: Why?
    10. 10. Variety of tasks <ul><li>Users want to determine their path on each platform </li></ul>Day 1: Why? [Forrester Research,2003]
    11. 11. Heterogeneousness of platforms Day 1: Why?
    12. 12. Multiplicity of contexts of use Day 1: Why? TV is multi-media family device #1 Family Device Booking notification Everywhere connectivity for simple data exchange Travelling Travel booking site Powerful interface for complex operations Working Multimedia Travel programme Sporting Experience Role Location
    13. 13. Consequence <ul><li>Proliferation of developments </li></ul>Day 1: Why? [Abrams et al.,2001] UI #12 UI #11 UI #10 UI #9 Application 3 UI #8 UI #7 UI #6 UI #5 Application 2 UI #4 UI #3 UI #2 UI #1 Application 1 Platform #4 Platform #3 Platform #2 Platform #1 Application 1 Application 2 Application 3 UI model #1 UI model #2 UI model #3 Platform #1 Platform #2 Platform #3 Platform #4 Platform model #1 Platform model #2 Platform model #3 Platform model #4
    14. 14. Is this web site usable? Day 1: Why?
    15. 15. Ergonomic criteria <ul><li>Criteria for assessing the usability of any UI </li></ul><ul><ul><li>A priori and/or a posteriori </li></ul></ul><ul><ul><li>At design and/or evaluation time </li></ul></ul><ul><li>Structured into 8 first-level criteria </li></ul><ul><ul><li>Compatibility </li></ul></ul><ul><ul><li>Consistency </li></ul></ul><ul><ul><li>Work load </li></ul></ul><ul><ul><li>Adaptation </li></ul></ul><ul><ul><li>Dialog control </li></ul></ul><ul><ul><li>Guidance </li></ul></ul><ul><ul><li>Error management </li></ul></ul><ul><li>Implicit order: sorted by importance </li></ul>Day 1: Why? [Scapin & Bastien,1997] [Vanderdonckt,1995]
    16. 16. Navigation <ul><li>Use a reactive image as a navigation map </li></ul>Day 1: Why? No map without any relation Metaphoric map
    17. 17. Navigation <ul><li>Include navigation clues </li></ul>Day 1: Why? Navigation bar Landmark Global vs local diagram
    18. 18. Navigation <ul><li>Navigation clues should be consistent </li></ul>Day 1: Why? Inconsistencies between the menu and navigation bar Use the same clues at the same location for the same purpose
    19. 19. Navigation <ul><li>No more &quot;Clic here&quot; </li></ul>Day 1: Why? Clic here to go to the Publications page Clic here to go to the LSM page Clic here to go to the UCL page Go to : UCL / LSM / Publications
    20. 20. Navigation <ul><li>Do not use &quot;Return to...&quot;  links </li></ul>Day 1: Why? Aller à :... Button Avoid : come back, return, redo, cancel, undo, redirect Home page Page A Page B Page C Page D Page E First site Page 1 Page 2 Page 3 Page 4 Page 5 Second site
    21. 21. Presentation <ul><li>Information wich are semantically related should be presented together </li></ul>Day 1: Why? Related label Related label
    22. 22. Presentation <ul><li>Example of a web page format (before) </li></ul>Day 1: Why? Display zone of browser navigation buttons Browser status bar Area for accessing to main commands, langages, map, about, contact User category picture Site menus zone Informational contents Display zone for external links and external applications Organisation logo area Area for accessing to local commands Structure indicator
    23. 23. Presentation <ul><li>How to format? Position of home page link, internal links </li></ul>Day 1: Why? [Shaikh & Lenz,2006]
    24. 24. Presentation <ul><li>How to format? Position of search engine, advertisements, about us </li></ul>Day 1: Why? [Shaikh & Lenz,2006]
    25. 25. Presentation <ul><li>Example of a web page format (after) </li></ul>Day 1: Why? Logo to home page Site menus zone Informational contents External links You are here: structure indictor NL -FR-DL Title, sub-title Contact - Mentions légales – Vie privée – Médiateur - Accessibilité Log in – Site map - Help Search Update – Printer-friendly v.
    26. 26. Simple choice Mixed domain: Know domain: Day 1: Why? [Vanderdonckt & Faulkner,2002] Unknown domain : Amount of possible values  [2,3] Amount of possible values  [8,50] Amount of possible values  [4,7] Amount of possible values  ]50,+  [ Amount of possible values  [2,3] Amount of possible values  [4,7] Amount of possible values  [8,50] Amount of possible values  ]50,+  [
    27. 27. Choix multiple Mixed domain: Known domain: Day 1: Why? [Vanderdonckt & Faulkner,2002] Domaine inconnu : Amount of possible values  [4,7] Amount of possible values  [8,50] Amount of possible values  ]50,+  [ Amount of possible values  [2,3] Amount of possible values  [4,7] Amount of possible values  [8,50] Amount of possible values  ]50,+  [ Amount of possible values  [2,3]
    28. 28. Graphics <ul><li>Use graphics for headers </li></ul>Day 1: Why? Use moderately artistic fonts for headers Keep hear in a separate layer Use anti-aliasing
    29. 29. Multimedia elements <ul><li>The background should be appropriate for the task </li></ul>Use various horizon lines depending on the context Day 1: Why? Horizon line Example Meaning Low Brings attention to the sky, the abstract, high values High Stresses immediate results, the concrete Median Suggests balance, equilibrium, peace Diagonal Creates some instability, a feeling of moving Sharp Suggests dynamic aspects, changing, aggressivity
    30. 30. Multimedia elements <ul><li>Low horizon line </li></ul>Day 1: Why?
    31. 31. Multimedia elements <ul><li>High horizon line </li></ul>Day 1: Why?
    32. 32. Multimedia elements <ul><li>Median horizon line </li></ul>Day 1: Why?
    33. 33. Multimedia elements <ul><li>Diagonal horizon line </li></ul>Day 1: Why?
    34. 34. Multimedia elements <ul><li>Sharp horizon line </li></ul>Day 1: Why?
    35. 35. Multimedia elements <ul><li>Do prefer low-intensity, light backgrounds </li></ul>Day 1: Why? No white on black
    36. 36. Multimedia elements <ul><li>Do not use backgrounds with textures </li></ul>Day 1: Why? Pale backgrounds, submit and test Pages with automatically built background
    37. 37. <ul><li>Keep the good color combinations </li></ul>Multimedia elements Day 1: Why? [Murch,1987] Blue (94%) Black (63%) Red (25%) White (75%) Yellow (63%) Yellow (75%) White (56%) Black (44%) Black (100%) Blue (56%) Red (25%) White (75%) Yellow (63%) Cyan (25%) Blue (69%) Black (56%) Red (37%) Black (63%) White (56%) Blue (44%) Background Thin lines and text White Black Red Green Blue Cyan Magenta Yellow Background Thin lines and text White Black Red Green Blue Cyan Magenta Yellow Red (63%) Blue (63%) Black (56%)
    38. 38. <ul><li>Keep the good color combinations </li></ul>Multimedia elements Day 1: Why? [Murch,1987] Black (69%) Blue (63%) Red (31%) Yellow (69%) White (50%) Green (25%) Black (50%) Yellow (44%) White (44%) Black (69%) Red (63%) Blue (31%) Yellow (38%) Magenta (31%) Black (31%) Red (56%) Blue (50%) Black (44%) Blue (50%) Black (44%) Yellow (25%) Red (75%) Blue (63%) Black (50%) Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow Black (69%) Blue (63%) Red (31%) Yellow (69%) White (50%) Green (25%) Black (50%) Yellow (44%) White (44%) Black (69%) Red (63%) Blue (31%) Yellow (38%) Magenta (31%) Black (31%) Red (56%) Blue (50%) Black (44%) Blue (50%) Black (44%) Yellow (25%) Red (75%) Blue (63%) Black (50%) Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow
    39. 39. <ul><li>Avoid the bad color combinations </li></ul>Multimedia elements Day 1: Why? [Murch,1987] Yellow (100%) Cyan (94%) Blue (87%) Red (37%) Magenta (25%) Magenta (81%) Blue (44%) Green (25%) Cyan (81%) Magenta (50%) Yellow (37%) Green (62%) Red (37%) Black (37%) Green (81%) Yellow (75%) White (31%) Green (75%) Red (56%) Cyan (44%) White (81%) Cyan (81%) Background Thin lines and text White Black Red Green Blue Cyan Magenta Yellow Yellow (100%) Cyan (94%) Blue (87%) Red (37%) Magenta (25%) Magenta (81%) Blue (44%) Green (25%) Cyan (81%) Magenta (50%) Yellow (37%) Green (62%) Red (37%) Black (37%) Green (81%) Yellow (75%) White (31%) Green (75%) Red (56%) Cyan (44%) White (81%) Cyan (81%) Yellow (100%) Cyan (94%) Blue (87%) Red (37%) Magenta (25%) Magenta (81%) Blue (44%) Green (25%) Cyan (81%) Magenta (50%) Yellow (37%) Green (62%) Red (37%) Black (37%) Green (81%) Yellow (75%) White (31%) Green (75%) Red (56%) Cyan (44%) White (81%) Cyan (81%) Background Thin lines and text White Black Red Green Blue Cyan Magenta Yellow Background Thin lines and text White Black Red Green Blue Cyan Magenta Yellow
    40. 40. <ul><li>Avoid the bad color combinations </li></ul>Multimedia elements Day 1: Why? [Murch,1987] Yellow (95%) Cyan (75%) Blue (81%) Magenta (31%) Magenta (69%) Blue (50%) Green (37%) Cyan (81%) Magenta (44%) Yellow (44%) Green (44%) Red (31%) Black (31%) Yellow (69%) Green (62%) White (56%) Cyan (81%) Green (69%) Red (44%) White (81%) Cyan (56%) Green (25%) Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow Yellow (95%) Cyan (75%) Blue (81%) Magenta (31%) Magenta (69%) Blue (50%) Green (37%) Cyan (81%) Magenta (44%) Yellow (44%) Green (44%) Red (31%) Black (31%) Yellow (69%) Green (62%) White (56%) Cyan (81%) Green (69%) Red (44%) White (81%) Cyan (56%) Green (25%) Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow
    41. 41. Multimedia elements <ul><li>Consider font variations induced by different platforms </li></ul>Day 1: Why? Fixed size font Label on top of field Same fonts Two browsers
    42. 42. IFIP Quality Properties for UIs <ul><li>Ten properties in 3 categories </li></ul><ul><li>Category 1: information presentation </li></ul><ul><ul><li>Multiplicity of devices: keyboard, mouse, tablet,… </li></ul></ul><ul><ul><li>Multiplicity of representations: alternate representations both in input and output, honesty </li></ul></ul><ul><ul><li>Input/output reusability (use output produced by one action as input for another) </li></ul></ul><ul><li>Category 2: ordering of task planning </li></ul><ul><ul><li>Multiplicity of user roles </li></ul></ul><ul><ul><li>Multiplicity of execution paths: users should be able to be engaged in different tasks simultaneously </li></ul></ul><ul><ul><li>Non-preemptiveness: degree of freedom for the user to decide what’s next </li></ul></ul><ul><ul><li>Reachability: possibility to navigate in the system (undo, redo) </li></ul></ul><ul><ul><li>Observability vs Browsability </li></ul></ul><ul><li>Category 3: dialog adaptation </li></ul><ul><ul><li>Reconfigurability: system ability to support user personalization </li></ul></ul><ul><ul><li>Adaptivity: system ability to support automated adaptation </li></ul></ul><ul><ul><li>Migrability: system ability to transfer responsibility from one user to another, among users, among users and systems </li></ul></ul><ul><ul><li>Plasticity: system ability to adapt to the context of use while preserving predefined usability properties </li></ul></ul><ul><li>Dix’s Principle of Effort Maximility </li></ul><ul><ul><li>A complex action should be easy to do, but complex to undo </li></ul></ul>Day 1: Why? [Gram & Cockton,1986]
    43. 43. IFIP Quality Properties for UIs <ul><li>Multiplicity of devices and representations </li></ul>Day 1: Why? [Stephanidis,2001]
    44. 44. Derivation panel with preview CTTE Operator Operator parameters Design comment resulting from Applying guidelines Design Preview that dynamically changes according to parameters Legends Day 1: Why? [Vanderdonckt et al.,2003]
    45. 45. Plastic UI <ul><li>Example: the Virtual keyboard </li></ul>Day 1: Why? [Grolaux et al.,2001]
    46. 46. Plastic UI <ul><li>Plastic UI: adaptable bounded value </li></ul>Day 1: Why? [Vanderdonckt et al.,2005]
    47. 47. Plastic User interface <ul><li>Property of plasticity = adaptation to the context of use while satisfying predefined usability properties of interest </li></ul>Day 1: Why? [Grolaux et al.,2002]
    48. 48. How to address this issue? <ul><li>Capture essence of concepts through models </li></ul><ul><ul><li>Separation of concerns, Correlability </li></ul></ul><ul><ul><li>Parsability, editability </li></ul></ul><ul><ul><li>If possible, human readability </li></ul></ul><ul><li>Typical models </li></ul>Day 2: What? [Calvary et al.,2003]
    49. 49. Typical models
    50. 50. What do we want to get? Final user Interface T Concrete user Interface T Task and Domain T Abstract user Interface T T=Target context of use Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use http://www.plasticity.org UsiXML unsupported model UsiXML supported model User S User T Day 2: What? [Calvary et al.,2003] Environment T Reification Abstraction Reflexion Translation Platform S Environment S Platform T
    51. 51. Software engineering interpretation <ul><li>Different types of engineering </li></ul><ul><ul><li>Forward engineering </li></ul></ul><ul><ul><li>Reverse engineering </li></ul></ul><ul><ul><li>Lateral engineering </li></ul></ul><ul><ul><li>Diagonal engineering (with or without shortcuts) </li></ul></ul><ul><li>Different approaches </li></ul><ul><ul><li>Top-down </li></ul></ul><ul><ul><li>Bottom-up </li></ul></ul><ul><ul><li>Middle-out </li></ul></ul><ul><li>Different development paths </li></ul><ul><ul><li>Example: Round-trip engineering = composition of </li></ul></ul><ul><ul><ul><li>Reification CUI -> FUI </li></ul></ul></ul><ul><ul><ul><li>Reflexion FUI -> FUI </li></ul></ul></ul><ul><ul><ul><li>Abstraction FUI -> CUI </li></ul></ul></ul><ul><ul><ul><li>Reflexion CUI -> CUI </li></ul></ul></ul>
    52. 52. IEEE Reverse Engineering Taxonomy Reference: Elliot J. Chikofsky , James H. Cross II, Reverse Engineering and Design Recovery: A Taxonomy, IEEE Software, v.7 n.1, p.13-17, January 1990: http://labs.cs.utt.ro/labs/acs/html/resources/ReengineeringTaxonomy.pdf <ul><li>Reengineering </li></ul><ul><li>&quot;the examination of a subject system to reconstitute it in a new form and the subsequent implementation of the new form.“ </li></ul><ul><li>Restructuring </li></ul><ul><li>&quot;a transformation from one form of representation to another at the same relative level of abstraction.&quot; The new representation is meant to preserve the semantics and external behavior of the original. </li></ul><ul><li>Redocumentation </li></ul><ul><li>&quot;a form of restructuring where the resulting semantically-equivalent representation is an alternate view intended for a human audience.“ </li></ul><ul><li>Design recovery </li></ul><ul><li>&quot;a subset of reverse engineering in which domain knowledge, external information, and deduction or fuzzy reasoning are added to the observations of the subject system.&quot; The objective of design recovery is to identify meaningful higher-level abstractions beyond those obtained directly by examining the system itself </li></ul>[Chikofsky & Cross,1990]
    53. 53. Revisitation [Bouillon,2006]
    54. 54. Multi-Path Development of UI <ul><li>Multi-path development qualifies a methodology that support the realization of multiples types of development paths withing a single framework </li></ul>Forward engineering Reverse Engineering Adaptation Any Relevant Composition [Limbourg,2004] Development step Development step Development path Development path * 1 Development Scenario Development Scenario * * Development step Development step Development path Development path * 1 Development Scenario Development Scenario * *
    55. 55. Our goals <ul><li>Objectives </li></ul><ul><ul><li>Description of interactive systems </li></ul></ul><ul><ul><ul><li>Using pre existing domain specific meta-models </li></ul></ul></ul><ul><ul><ul><li>Used both at design and run time </li></ul></ul></ul><ul><ul><li>Capitalization </li></ul></ul><ul><ul><ul><li>Properties </li></ul></ul></ul><ul><ul><ul><li>Transformations </li></ul></ul></ul>Interactive system Model of the IS Model of the IS ‘ Interactive system ‘ Checks of properties Transformation Models, instances of Meta-Models described in MOF (even for properties and transformations…)
    56. 56. UsiXML <ul><li>UsiXML = </li></ul><ul><ul><li>USer Interface exTensible Markup Language </li></ul></ul><ul><ul><li>XML-compliant specification language for user interfaces suitable for any interface </li></ul></ul><ul><ul><ul><li>Web </li></ul></ul></ul><ul><ul><ul><li>Java </li></ul></ul></ul><ul><ul><ul><li>Windows-based applications </li></ul></ul></ul><ul><ul><ul><li>Multimodal applications, 3D applications </li></ul></ul></ul><ul><ul><ul><li>Virtual, mixed reality applications </li></ul></ul></ul><ul><ul><li>http://www.usixml.org </li></ul></ul><ul><ul><li>Join the UsiXML Consortium by registering on line </li></ul></ul><ul><ul><li>Download the CD image from http://www.usixml.org/index.php?download= UsiXML RelOne.iso </li></ul></ul>
    57. 57. UsiXML = User Interface eXtensible Markup Language <ul><li>What is UsiXML? </li></ul><ul><ul><li>It is a XML-compliant User Interface Description Language </li></ul></ul><ul><ul><li>Publicly available from http://www.usixml.org </li></ul></ul><ul><ul><li>Free to use, open for access, easy to expand </li></ul></ul><ul><ul><li>Definition of the language </li></ul></ul>UML Class Diagrams UsiXML Reference manual XSD XML Schema Descriptions UsiXML Models
    58. 58. Adapted from [Palanque,2002] Target Applications, Domains Notations and tools User Interface Interaction Technique 2006 TODAY 2007 No Interaction Technique Automated, batch systems Nothing Character UIs Business applications Screen definitions Graphical User Interfaces Information systems Entity-relationship Attribute model State-transition diagrams Multi-platform User Interfaces Web, Java applications Task model, context model, UML ,…
    59. 59. The problem <ul><li>To develop user interfaces (UIs) simultaneously for multiple contexts of use </li></ul><ul><li>A context of use = triple </li></ul><ul><ul><li>User </li></ul></ul><ul><ul><li>Computing platform </li></ul></ul><ul><ul><li>Surrounding environment </li></ul></ul><ul><ul><ul><li>Organisation </li></ul></ul></ul><ul><ul><ul><li>Socio-psychological factors </li></ul></ul></ul>[Calvary et al.,2003]
    60. 60. The problem <ul><li>Why is it difficult? </li></ul><ul><ul><li>For the designer: </li></ul></ul><ul><ul><ul><li>Multiplicity of contexts of use </li></ul></ul></ul><ul><ul><ul><li>No systematic design method </li></ul></ul></ul><ul><ul><li>For the user: </li></ul></ul><ul><ul><ul><li>Poor usability of resulting UI </li></ul></ul></ul><ul><ul><ul><li>UI not adapted to the genuine context of use </li></ul></ul></ul><ul><ul><li>For the developer: </li></ul></ul><ul><ul><ul><li>Increase of development and maintenance costs </li></ul></ul></ul><ul><ul><ul><li>Increasing development complexity </li></ul></ul></ul><ul><ul><ul><li>Little of no factoring out of common elements </li></ul></ul></ul>
    61. 61. The problem <ul><li>Why should it be systematic? </li></ul><ul><ul><li>Some consider it noble to have a method </li></ul></ul><ul><ul><li>Other consider it noble not to have a method </li></ul></ul><ul><ul><li>Not to have a method is bad </li></ul></ul><ul><ul><li>But to stop entirely at any method is worse still </li></ul></ul><ul><ul><li>One should at first observe rules severely, then change them in an intelligent way </li></ul></ul><ul><ul><li>The aim of possessing method is to seem finally as if one had no method </li></ul></ul>[Lao Tch’ai Tche, many many years ago]
    62. 62. Mono-platform UI <ul><li>Traditional approach </li></ul><ul><ul><li>Visual development </li></ul></ul>
    63. 63. Mono-platform UI <ul><li>Advantages of traditional approach </li></ul><ul><ul><li>UI is graphical by nature </li></ul></ul><ul><ul><li>A UI prototype can be </li></ul></ul><ul><ul><ul><li>Rapidly produced </li></ul></ul></ul><ul><ul><ul><li>Easily modified </li></ul></ul></ul><ul><ul><ul><li>Visually demonstrated </li></ul></ul></ul>
    64. 64. Mono-platform UI <ul><li>Shortcomings of traditional approach </li></ul><ul><ul><li>No structured method for developing a UI </li></ul></ul><ul><ul><li>All the knowledge remain implicit </li></ul></ul><ul><ul><li>Modification can lead to unstructured design </li></ul></ul><ul><ul><li>Little or no tool support for iterative design </li></ul></ul><ul><ul><ul><li>Selection of widgets can be inappropriate </li></ul></ul></ul><ul><ul><ul><li>UI Layout can be tedious, repetitive </li></ul></ul></ul><ul><ul><ul><li>Problem of the spaghetti of callbacks </li></ul></ul></ul><ul><ul><li>Early prototyping is hard to achieve and time consuming </li></ul></ul><ul><ul><li>Limited reusability (throw away) </li></ul></ul>
    65. 65. Mono-platform UI <ul><li>Shortcomings of traditional approach </li></ul><ul><ul><li>Limited use of UI builder </li></ul></ul>[Szekely,1996] <ul><li>Table with dynamic data </li></ul><ul><li>Gantt chart </li></ul><ul><li>Direct manipulation </li></ul>Desired interface Obtained interface <ul><li>Menu bar and pull-down menus </li></ul><ul><li>Toolbar </li></ul><ul><li>Scrollbars </li></ul>
    66. 66. Mono-platform UI <ul><li>Interface builders cannot build their own UI </li></ul>[Szekely,1996]
    67. 67. Mono-platform UI <ul><li>Any development method (or methodology) is decomposed into 3 axes: </li></ul><ul><ul><li>Models : explicitly capture knowledge about UI and Interactive Applications with appropriate abstractions </li></ul></ul><ul><ul><li>Method : structures the definition and use of underlying models in a stage-wise approach </li></ul></ul><ul><ul><li>Supporting tools : support the use of the method by providing tools for models and their related operations. Ideally, one model should be supported by at least one tool </li></ul></ul>[Bodart,1989]
    68. 68. Mono-platform UI <ul><li>Goal: to integrate all three facets </li></ul>Interface 1 Method Model 1 Model … Model n Models Model Model Model Tools
    69. 69. MDE based on UsiXML Model to Model Platform Independent Model (PIM) Platform Specific Model (PSM) Model to Code Source code MDA Components Techniques proposed based on UsiXML Computing Independent Model (CIM) Model to Model UsiXML model: Abstract user interface UsiXML model: Concrete user interface Rendering Final user interface UsiXML models: task, domain Graph transformations Graph transformations
    70. 70. CIM Step 1: Task model
    71. 71. New Abstraction: the user’s task <ul><li>Task = set of actions carried out by a user in a given context to reach a goal </li></ul><ul><li>Logical decomposition of task into sub-tasks </li></ul><ul><li>Temporal ordering: LOTOS operators (in CTTE) </li></ul><ul><ul><li>T1 >> T2 Enabling </li></ul></ul><ul><ul><li>T1[ ]>>T2 Enabling + information passing </li></ul></ul><ul><ul><li>T1 |> T2 Suspend/resume </li></ul></ul><ul><ul><li>T1 [ ] T2 Non-deterministic choice </li></ul></ul><ul><ul><li>T1  T2 Deterministic choice </li></ul></ul><ul><ul><li>T1 [> T2 Disabling (e.g. Form submit) </li></ul></ul><ul><ul><li>T1 |=| T2 Independence (any order, but finished) </li></ul></ul><ul><ul><li>T1* Iteration </li></ul></ul><ul><ul><li>T1{n} Finite iteration </li></ul></ul><ul><ul><li>T1 ||| T2 Concurrency </li></ul></ul><ul><ul><li>T1 |[x]| T2 Concurrency + information passing </li></ul></ul><ul><ul><li>[T] Optional </li></ul></ul><ul><ul><li>T Recursion </li></ul></ul>[Markopoulos,1992]
    72. 72. New Abstraction: the user’s task <ul><li>Task definition = action + object </li></ul><ul><ul><li>Action types </li></ul></ul><ul><ul><ul><li>CRUD pattern: create, read, update, delete </li></ul></ul></ul><ul><ul><ul><li>Select, control,… </li></ul></ul></ul><ul><ul><ul><li>Acquire, render, modify, publish, compute, derive,… </li></ul></ul></ul><ul><ul><li>Object types: </li></ul></ul><ul><ul><ul><li>Element, list, table, collection, compound,… </li></ul></ul></ul>
    73. 73. New Abstraction: the task meta-model [Limbourg,2004]
    74. 74. CIM Step 2: domain model [Limbourg,2004]
    75. 75. CIM Step 3: Task-domain mappings IdealXML [Limbourg,2004]
    76. 76. New Abstraction: the abstract UI <ul><li>Different CIOs can be used for the same purpose, but with different interaction modalities </li></ul><ul><li>Definition </li></ul><ul><ul><li>Abstract Container = set of Abstract Individual Component </li></ul></ul><ul><ul><li>AIC = abstraction of CIOs of the same type, but independently of any interaction modality </li></ul></ul><ul><ul><li>Abstract User Interface (AUI) = decomposition into AC+AIC </li></ul></ul>[Vanderdonckt & Bodart,1993]
    77. 77. Abstraction: the abstract UI
    78. 78. Abstraction: the abstract UI <ul><li>Notation: based on L. Constantine’s notation for canonical abstract prototypes </li></ul>IdealXML [Montero et al.,2005] [Constantine,2003] Abs.container Abs. component Input Output Navigation Control Select
    79. 79. IdealXML [Montero,2005]
    80. 80. Example of AUI produced
    81. 81. Mapping the models [Montero,2005] triggers (tg): { , } x updates (up): x observes (ob): x isExecutedIn (ex): x manipulates (ma): { , } x These mappings can be established:
    82. 82. Mapping the models <ul><li>Mapping the models with a mapping model (!!) </li></ul>
    83. 83. Your turn now! <ul><li>Virtual Polling System </li></ul><ul><ul><li>Characterization of a participant </li></ul></ul><ul><ul><ul><li>Age </li></ul></ul></ul><ul><ul><ul><li>Gender </li></ul></ul></ul><ul><ul><ul><li>Region </li></ul></ul></ul><ul><ul><ul><li>Country </li></ul></ul></ul><ul><ul><li>Polling </li></ul></ul><ul><ul><ul><li>Series of questions & answers </li></ul></ul></ul><ul><ul><ul><li>Each answer may have </li></ul></ul></ul><ul><ul><ul><ul><li>Simple choice </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Multiple choices </li></ul></ul></ul></ul><ul><ul><ul><li>More capabilities… </li></ul></ul></ul>
    84. 84. Example solution (first variant)
    85. 85. Example solution (first variant)
    86. 86. Task and domain models [Montero,2005]
    87. 87. Typed Model-to-Model Transformation Uses language Meta-Meta-Model Graph Structure is instance of Meta-Model Our Meta-Model Meta-Model Subset 1 e.g., Task+Domain Model is instance of Meta-Model Subset 2 e.g., Concrete UI Model is instance of Initial UI Model e.g., MyTaskAndDomainModel Resultant UI Model e.g., MyConcreteUIModel Transformation Rule Our transformation catalog [Limbourg,2004]
    88. 88. Expression of models as graphs <ul><li>All transformations are in UsiXML </li></ul><ul><ul><li>Each model = instance of meta-model </li></ul></ul><ul><ul><li>Each model = graph as instance of graph type </li></ul></ul><ul><ul><li>Each model transformation = </li></ul></ul><ul><ul><ul><li>graph transformation </li></ul></ul></ul><ul><ul><ul><li>Set of productions </li></ul></ul></ul>
    89. 89. Definition of a production <ul><li>Find an occurrence of LHS in G (this occurrence is called a match). If several occurrences exist, choose one non-deterministically. </li></ul><ul><li>Check preconditions of both type PAC and NAC. If not verified, then skip. </li></ul><ul><li>Remove the part of G which corresponds to (LHS – K), where K is the morphism specified between LHS and RHS. </li></ul><ul><li>Add RHS – K into G – (LHS – K) as it is given by the corresponding relation between RHS – K and K </li></ul><ul><li>Check postconditions of both type PAC (and notably that the resulting graph is properly typed) and NAC. If not verified, then undo the transformation rule. </li></ul>LHS G RHS G’ r m m* r* LHS G RHS G’ r m m* r*
    90. 90. Transformation system [Limbourg,2004] G Host USIXM specification G’ Resultant USIXM specification LHS RHS Matches - Co-Matches Is Transformed Into Is Transformed Into Transformation Rule 1 Transformation Rule 2 … Transformation Rule N Transformation System NAC Not Matches +
    91. 91. PIM step: task+domain to AUI <ul><li>Abstract UI (AUI) = UI independent of any interaction modality </li></ul><ul><li>Definition of AUI structure in terms of Abtract Containers (AC) </li></ul><ul><ul><li>Which tasks should be logically grouped? </li></ul></ul><ul><li>Definition of Abstract Individual Components (AIC) types </li></ul><ul><ul><li>Which « functionnality » should assume AICs and what data do they manipulate ? </li></ul></ul><ul><li>Definition of spatio-temporal arrangement </li></ul><ul><ul><li>How should AIC be arranged in space and time ? </li></ul></ul><ul><li>Definition of dialog control </li></ul><ul><ul><li>What is the valid flow of action on AICs ? </li></ul></ul>UsiXML model: Abstract user interface UsiXML models: task, domain Graph transformations
    92. 92. PIM step: task+domain to AUI Identification of AUI structure Spatio-Temporal Arrangement of AIOs Selection of AIC Definition of Abstract Dialog Control STEP : From Task & Domain to AUI SUB-STEPS Derivation of AUI to Domain Relationships
    93. 93. How to read a graph transformation? Node type Node (Attribute,value) Edge type Edge (Attribute,value) Node Edge
    94. 94. Rule 1 LHS RHS ::= Ø
    95. 95. Rule 2 LHS RHS ::=
    96. 96. Rule 3 LHS RHS ::=
    97. 97. Rule 4 LHS RHS ::= NAC
    98. 98. Rule 5 LHS RHS ::= NAC1 NAC2
    99. 99. Rule 6 LHS RHS ::=
    100. 100. Rule 7 LHS RHS ::= PAC “ X < 3000”
    101. 101. Rule 8 LHS RHS ::= PAC “ X > 3000” NAC
    102. 102. Rule 9 LHS RHS ::=
    103. 103. Rule 10 PAC LHS RHS ::= “ X > Y”
    104. 104. PIM sub-step 1: Definition of AUI structure (AC)
    105. 105. PIM sub-step 1: Definition of AUI structure (AC)
    106. 106. Facet type
    107. 108. AC Participate to OpinionPoll AC Answer Poll AC Answer Questionnaire AC Answer Question AIC (Input/Single Selection) Select Proposition AIC (Output) See Statistics AIC (Input/Single Selection ) Chose Poll AIC (Control) Validate Question AIC (Output) Provide Personal Data AIC (Output) Read Question AIC (Input/Single Selection) Chose Questionnaire
    108. 109. PIM sub-step 2: definition of AIC types AC = Abstract Container AIC = Abstract Individual Component CIC = Concrete Interaction Component
    109. 110. PIM sub-step 3: Definition of spatio-temporal arrangement
    110. 111. PSM Step: AUI to CUI <ul><li>Concrete UI (CUI) = UI independent of toolkit </li></ul><ul><li>Definition of CUI structure </li></ul><ul><ul><li>Which AIC is a window? </li></ul></ul><ul><li>Definition of Concrete Interaction Component (CIC) type </li></ul><ul><ul><li>Which « widget » should represent which AIC ? </li></ul></ul><ul><li>Definition of placement </li></ul><ul><ul><li>What layout can be specified between CICs,… </li></ul></ul><ul><li>Definition of navigation </li></ul><ul><ul><li>Which container can be started or closed from which container? </li></ul></ul><ul><li>Definition of dialog control </li></ul><ul><ul><li>What is the valid flow of action on AIOs </li></ul></ul>UsiXML model: Abstract user interface UsiXML model: Concrete user interface UsiXML models: task, domain Graph transformations Graph transformations
    111. 112. PSM Step: AUI to CUI Reification of AC into CC Arrangement of CICs Selection of CIC Concrete Dialog Control Definition STEP : From AUI to CUI SUB-STEPS Definition of Navigation Derivation of CUI to Domain Relationships
    112. 113. PSM sub-step 3: definition of navigation An example of a complex rule
    113. 114. PSM: Concrete User Interface
    114. 115. Example: Platform adaptation widget substitution (1) <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot; standalone=&quot;yes&quot;?> < cuiModel name =&quot; MyModel &quot;> < version modifDate =&quot; 2004-03-24T17:09:17.402+01:00 &quot; xmlns =&quot;&quot;> 7 </ version > < authorName xmlns =&quot;&quot;> Youri </ authorName > < window height =&quot; 500 &quot; width =&quot; 600 &quot; name =&quot; Formulaire (2/5) &quot; id =&quot; window_1 &quot;> < box relativeHeight =&quot; 100 &quot; name =&quot; box1_0 &quot; id =&quot; box1_0 &quot;> < box type =&quot; vert &quot; name =&quot; boxTodo &quot; id =&quot; boxTodo &quot;> ... < box type =&quot; horiz &quot; name =&quot; box_2_2_2_1 &quot; id =&quot; box_2_2_2_1 &quot;> < textComponent defaultContent =&quot; Sexe &quot; isBold =&quot; true &quot; id =&quot; label_2 &quot;/> < radioButton groupName =“ grupo01 &quot; defaultContent =&quot; Femme &quot; defaultState =&quot; false &quot; id =&quot; radiobutton_0 &quot;/> < radioButton groupName =&quot; grupo01 &quot; defaultContent =&quot; Homme &quot; defaultState =&quot; true &quot; id =&quot; radiobutton_1 &quot;/> </ box > ... </ box > </ box > </ window > </ cuiModel > Excerpt for a UsiXML CUI specification
    115. 116. Example: widget substitution (2)
    116. 117. Example: widget substitution (3) The UsiXML graph before applying any rule
    117. 118. Example: widget substitution (4) LHS RHS NAC Rule 1: Create a new comboBox with the same id and name as the name of the group of radioButtons.
    118. 119. Example: widget substitution (5) Rule 1: Create a new comboBox with the same id and name as the name of the group of radioButtons. The UsiXML graph after applying the first rule
    119. 120. Example: widget substitution (6) LHS RHS ::= Rule 2: Convert every radio button within the group “x” into an item for the comboBox “x” that we have just created thanks to Rule 1
    120. 121. Example: widget substitution (7) Rule 2: Convert every radioButton within the group “x” into an item for the comboBox “x”, we have just created. The UsiXML graph after applying the second rule: radio buttons disappeared
    121. 122. Example: widget substitution (8) <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot; standalone=&quot;yes&quot;?> < cuiModel name =&quot; MyModel &quot;> < version modifDate =&quot; 2004-03-24T17:09:17.402+01:00 &quot; xmlns =&quot;&quot;> 7 </ version > < authorName xmlns =&quot;&quot;> Youri </ authorName > < window height =&quot; 500 &quot; width =&quot; 600 &quot; name =&quot; Formulaire (2/5) &quot; id =&quot; window_1 &quot;> < box relativeHeight =&quot; 100 &quot; name =&quot; box1_0 &quot; id =&quot; box1_0 &quot;> < box type =&quot; vert &quot; name =&quot; boxTodo &quot; id =&quot; boxTodo &quot;> ... < box type =&quot; horiz &quot; name =&quot; box_2_2_2_1 &quot; id =&quot; box_2_2_2_1 &quot;> < textComponent defaultContent =&quot; Sexe &quot; isBold =&quot; true &quot; id =&quot; label_2 &quot;/> < comboBox id =&quot; comboBox001 &quot; name =&quot; label_3 &quot; isDropDown =&quot; true &quot;> < item id =&quot; radiobutton_0 &quot; name =&quot; radiobutton_0 &quot; defaultContent =&quot; Femme &quot;/> < item id =&quot; radiobutton_1 &quot; name =&quot; radiobutton_1 &quot; defaultContent =&quot; Homme &quot;/> </ comboBox > ... </ box > </ box > </ window > </ cuiModel > Excerpt from the final transformated UsiXML specification
    122. 123. Example: widget substitution (9)
    123. 124. What do we have so far? Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use UsiXML unsupported model UsiXML supported model User S Reification Platform S Environment S
    124. 125. Multiple development paths
    125. 126. Mapping the Methodological World onto the Transformation World A development library stores (in usiXML textual format) paths, steps and sub-steps definition and their associated transformation systems and transformation rules Development step Development step Development sub - step Development sub - step Development path Development path Transformation System Transformation System Transformation Rule Transformation Rule isComposedOf isRealizedBy isComposedOf isComposedOf * 1 * 1 1 * 1 0..1 Development step Development step Development sub - step Development sub - step Development path Development path Transformation System Transformation System Transformation Rule Transformation Rule isComposedOf isRealizedBy isComposedOf isComposedOf * 1 * 1 1 * 1 0..1 Methodological World Development step Development step Development sub - step Development sub - step Development path Development path Transformation System Transformation System Transformation Rule Transformation Rule isComposedOf isRealizedBy isComposedOf isComposedOf * 1 * 1 1 * 1 0..1 Transformation World [Limbourg,2004]
    126. 127. Multiple development paths Rule n Transformation System 1 Rule 1 Rule 2 … Rule n Transformation System 2 Rule 1 Rule 2 … Rule n Transformation System ... Rule 1 Rule 2 … Rule n Transformation System n Rule 1 Rule 2 … : when source terminates apply target : execute development step Development Step α
    127. 128. TransformiXML API/GUI <ul><li>API = set of transformations </li></ul>
    128. 129. From T&D to AUI <ul><li>TransformiXML </li></ul>TransformiXML [Bouillon et al.,2005]
    129. 130. Final user interface <ul><li>Two forms of UI rendering </li></ul><ul><ul><li>Interpretation </li></ul></ul><ul><ul><ul><li>By run-time static analysis and direct rendering (InterpiXML & FormiXML) </li></ul></ul></ul><ul><ul><li>Code generation </li></ul></ul><ul><ul><ul><li>By program synthesis (GrafiXML) </li></ul></ul></ul><ul><ul><ul><li>By generative programming (Angie) </li></ul></ul></ul><ul><ul><ul><ul><li>Feature model </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Components assembling </li></ul></ul></ul></ul>UsiXML model: Abstract user interface UsiXML model: Concrete user interface Rendering Final user interface UsiXML models: task, domain Graph transformations Graph transformations Generative programming
    130. 131. What is a Feature Model? C F1 C F1 F2 C F1 F2 Dependencies 1. 2. Optional Exclusive Alternate [Schlee & Vanderdonckt,2004]
    131. 132. Example of a Feature Model Alternate [Schlee & Vanderdonckt,2004]
    132. 133. Interpretation of a feature model C F1 F2 F3 F2a F2b F3a F3b C F1 F2 F3 F2a F2b F3a F3b 0 1 1 1 1 0 0 UsiXML Specifications <C> <F1>0</F1> <F2>1 <F2a>1</F2a> <F2b>0</F2b> </F2> <F3>1 <F3a>0</F3a> <F3b>1</F3b> </F3> </C> ABA ANGIE-Part [Schlee & Vanderdonckt,2004]
    133. 134. Generative Programming void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void nlinFilterCommon (const int& nID, Mask*, Pl*); int pot (const unsigned int& ) const; ImgProcessT<T>& channelXor (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); ImgProcessT<T>& chann elMix ImgProcessT<T> & channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); V oid channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void nlinFilterCommon (const int& nID, Mask*, Pl*); int pot (const unsigned int& ) const; ImgProcessT<T>& channelXor (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); public: ImgProcessT(const char* const psz = &quot;&quot;, Pl* ppl = NULL); ImgProcessT(const int&, const int& nWidth = 20, const int& nHeight = 20, const int& nFillColor = 0, ProgressLine* ppl = NULL ); ImgProcessT(const ImgProcessT<T>&); virtual ~ImgProcessT(); virtual char* getClassName() const; virtual char getClassID () const; ImgProcessT<T>& operator+=(const ImgProcessT<T>&); ImgProcessT<T>& operator-=(const ImgProcessT<T>&); ImgProcessT<T>& operator^=(const ImgProcessT<T>&); ImgProcessT<T> operator+ (const ImgProcessT<T>&); ImgProcessT<T> operator- (const ImgProcessT<T>&); ImgProcessT<T> operator^ (const ImgProcessT<T>&); ImgProcessT<T>& channelNLinFilter(const int&, Mask*, T*, Pl* ppl=NULL); mgProcessT<T>& operator+=(const ImgProcessT<T>&); ImgProcessT<T>& operator-=(const ImgProcessT<T>&); ImgProcessT<T>& operator^=(const ImgProcessT<T>&); ImgProcessT<T> operator+ (const ImgProcessT<T>&); ImgProcessT<T> operator- (const ImgProcessT<T>&); ImgProcessT<T> operator^ (const ImgProcessT<T>&); ImgProcessT<T>& channelNLinFilter(const int&, Mask*, T*, Pl* ppl=NULL); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void nlinFilterCommon (const int& nID, Mask*, Pl*); int pot (const unsigned int& ) const; ImgProcessT<T>& channelXor (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); USIXML Specifications ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL) ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL) Components void areaOperation(const int&, const int&, const int&, const int&, ImgProcessT<T>& (ImgProcessT<T>::*pimgpr)(const int&, Pl*), const int& nAngle, Pl* ppl ); void areaOperation(const int&, const int&, const int&, const int&, ImgProcessT<T>& (ImgProcessT<T>::*pimgpr)(Pl*), Pl* ppl ); void areaOperation(const int&, const int&, const int&, const int&, ImgProcessT<T>& (ImgProcessT<T>::*pimgpr)(const int&, const int&, const int&, const int& ), const int&, const int&, const int&, const int& ); void channelShowHistogram( int*, const int&, const int&, const int&, const int&, const int&, const int&, const int& n = 0 ) const; void channelFft ( T*, T*, T*, const unsigned int&, const unsigned int&, Pl* ppl = NULL ); void channelIfft ( T*, T*, T*, const unsigned int&, const unsigned int&, Pl* ppl = NULL ); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void nlinFilterCommon (const int& nID, Mask*, Pl*); int pot (const unsigned int& ) const; ImgProcessT<T>& channelXor (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); public: ImgProcessT(const char* const psz = &quot;&quot;, Pl* ppl = NULL); ImgProcessT(const int&, const int& nWidth = 20, const int& nHeight = 20, const int& nFillColor = 0, ProgressLine* ppl = NULL ); ImgProcessT(const ImgProcessT<T>&); virtual ~ImgProcessT(); virtual char* getClassName() const; virtual char getClassID () const; ImgProcessT<T>& operator+=(const ImgProcessT<T>&); I Final code ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); public: ImgProcessT(const char* const psz = &quot;&quot;, Pl* ppl = NULL); ImgProcessT(const int&, const int& nWidth = 20, const int& nHeight = 20, const int& nFillColor = 0, ProgressLine* ppl = NULL ); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); [Schlee & Vanderdonckt,2004]
    134. 135. Developed by Benjamin Michotte <ul><li>GrafiXML consists of a user interface builder for designing a graphical user interface (GUI) independently of the platform and save it in UsiXML format language. </li></ul><ul><li>Features </li></ul><ul><li>Exports GUI in </li></ul><ul><ul><li>Java source (using Swing) </li></ul></ul><ul><ul><li>XHTML 1.0 Strict </li></ul></ul><ul><ul><li>XUL </li></ul></ul><ul><li>Works on Windows, Linux and MacOs X </li></ul><ul><li>Available in English, French and Spanish </li></ul><ul><li>Based on plugins </li></ul><ul><ul><li>Export </li></ul></ul><ul><ul><li>Import </li></ul></ul><ul><ul><li>Project management </li></ul></ul><ul><ul><li>Graceful degradation of user interfaces </li></ul></ul><ul><li>Allows creating context models </li></ul>[Michotte,2004]
    135. 136. Design Tab Design window Components toolbar Components options
    136. 137. Localisation of UIs <ul><li>GrafiXML allows the user to create multi-language GUI </li></ul>Support for mnemonics and shortcuts
    137. 138. Preview <ul><li>At any time, you can preview the UI in the language you want </li></ul>
    138. 139. XML Editor <ul><li>GrafiXML contains a XML editor which shows the UsiXML specification of your work </li></ul><ul><li>You can edit yourself some part of the XML </li></ul>Edit the UsiXML Show attributes A click on the tree highlights the corresponding UsiXML
    139. 140. Plugins <ul><li>GrafiXML works with plugins </li></ul><ul><ul><li>Install / remove using the plug-in manager </li></ul></ul><ul><ul><li>Updated if needed using one click </li></ul></ul>Click on « add » to open the downloader Choose the plugins you want install And install them Double-click on a plugin And look at the plugin informations
    140. 141. What do we have so far? Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use UsiXML unsupported model UsiXML supported model User S Reification Platform S Environment S
    141. 142. Reverse engineering <ul><li>From FUI to CUI </li></ul><ul><li>From CUI to AUI, task & domain </li></ul>
    142. 143. From FUI to CUI <ul><li>Do you have the source code? </li></ul><ul><ul><li>Markup languages (e.g., HTML): static code analysis (ReversiXML) </li></ul></ul><ul><li>Do you have the compiled code? </li></ul><ul><ul><li>Programming languages (e.g., compiled C): resource file extraction and static code analysis </li></ul></ul><ul><li>Do you have the running application? </li></ul><ul><ul><li>Video analysis: computer vision </li></ul></ul><ul><li>Do you have only the documentation? </li></ul><ul><ul><li>Image analysis: image processing </li></ul></ul>[Bouillon & Vanderdonckt,2004]
    143. 144. From compiled code [Marion,2005]
    144. 145. The calculator <ul><li>Begin VB.Form Calculator BorderStyle = 1 'Fixed Single Caption = &quot;Calculator&quot; ClientHeight = 2970 ClientLeft = 2580 ClientTop = 1485 ClientWidth = 3270 ClipControls = 0 'False BeginProperty Font name = &quot;System&quot; charset = 1 weight = 700 size = 9.75 underline = 0 'False italic = 0 'False strikethrough = 0 'False EndProperty Height = 3375 Icon = &quot;CALC.frx&quot;:0000 Left = 2520 LinkMode = 1 'Source LinkTopic = &quot;Form1&quot; MaxButton = 0 'False ScaleHeight = 2970 ScaleWidth = 3270 Top = 1140 Width = 3390 Begin VB.CommandButton Number Caption = &quot;7&quot; Height = 480 Index = 7 Left = 120 </li></ul><Window id=“1” name=“1” isVisible=“true” isEnabled=“true” defaultBorderTitle=“Calculator” borderWidth=“1”height=“358” width=“309”> <Box id=“2” name=“2” type =“verticall” isVisible=“true” isEnabled=“true”> <button isEnabled=“true” … USIXML [Marion,2005]
    145. 146. From CUI to AUI, task & domain <ul><li>Re-engineering = reverse + forward </li></ul><ul><li>Possible re-interpretation by QtkiXML </li></ul>
    146. 147. Example of Derivation Rules <ul><li>Examples: </li></ul><ul><ul><li>For a textbox in HTML/WML </li></ul></ul><ul><li> x  Ts : x = input ٨ (x.type=“text” ٧ x.type=“password” ٧ x.type=NULL)  Addnode (“textComponent”, idtext) </li></ul><ul><li> where idtext= NodeAmount(Tt) </li></ul><ul><ul><li>For a table in HTML/WML </li></ul></ul><ul><li> x  Ts : x = table ٨ x.border>0 </li></ul><ul><li>  Addnode (“table”, idtable) where idtable = NodeAmount(Tt) </li></ul><ul><li>  x  Ts : x = table ٨ x.border=0  Addnode (“box”, idbox) where idbox = NodeAmount(Tt) </li></ul><ul><li> AddAttribute (idbox, “type”, “vertical”) </li></ul>Input Name=in1 Maxlength=15 Value=login Type=text Element B TextComponent Name=in1 Maxlength=15 defaultContent=login Id=in1 Element Y [Bouillon,2006]
    147. 148. Derivation rules categories <ul><li>Rules can be classified into different categories, outlining the common issues in a RE process for various source UIs. </li></ul><ul><ul><li>Element recovery </li></ul></ul><ul><ul><li>Attribute detection </li></ul></ul><ul><ul><li>Layout / temporal ordering analysis </li></ul></ul><ul><ul><li>Dialog recuperation </li></ul></ul><ul><ul><li>Hierarchy recovery </li></ul></ul><ul><ul><li>Multi-model transformations </li></ul></ul><ul><ul><li>Retargeting operations </li></ul></ul> x  to Tt i ,y  to Tt 0 : x = window ٨ (y=box ٧ y=window) ٨ x.filename =y.targetfile  CloneNode(x.id, idnew, Tt0) where idnew =∑ node  Tt 0 ٨ RemoveNode(x, x.id) ٨ RemoveArc(ParentNode(x).id, x.id) ٨ z=root(Tt i ) ٨ Remove Node(z,z.id) ٨ AddArc(y.id, idnew)  x  to Tt i ,y  to Tt 0 : x = vocalGroup ٨ y=VocalGroup ٨ x.filename =y.insertFile  CloneNode(x.id, idnew, Tt0) where idnew =∑ node  Tt0 ٨ RemoveNode(x, x.id) ٨ RemoveArc(ParentNode(x).id, x.id) ٨ z=root(Tt i ) ٨ Remove Node(z,z.id) ٨ AddArc(y.id, idnew) Multi-model transformations Element 1 Element 2 Element 3 Horizontal box Horizontal box Vertical box Element 1 Element 2 Line Break Element 3  x  Tt: x=bix ٨ ((rigthSibling(x)!=table ٧ rigthSibling(x)!=bix ٧ rigthSibling(x)!=cell ٧ rigthSibling(x)!=box) ٨ rigthSibling(x)!=NULL)  CloneNode (rightSibling(x).id, idnew) ٨   RemoveNode (rightSibling(x), rightSibling(x).id) ٨ RemoveArc (ParentNode(rightSibling(x)), rightSibling(x)) ٨ AddArc(x.id, idnew) where idnew =∑ node  Tt Layout Analysis
    148. 149. RE of Web Pages: ReversiXML <ul><ul><ul><li>Written in PHP5 </li></ul></ul></ul><ul><ul><ul><li>On-line RE </li></ul></ul></ul><ul><ul><ul><li>About 12,000 LOC </li></ul></ul></ul><ul><ul><ul><li>Set of libraries </li></ul></ul></ul>
    149. 150. RE of Phone Interfaces <ul><li>Major working hypotheses </li></ul><ul><ul><li>Static RE of WML 1.1 and voiceXML 2.0 up to the CUI in USIXML 1.4.6 </li></ul></ul><ul><ul><li>Recovery of the layout and hierarchy/temporal ordering. </li></ul></ul><ul><ul><li>Dialog-Navigation analyzed (but not scripts) </li></ul></ul><ul><ul><li>No retargeting operations </li></ul></ul><ul><li>Similar method to HTML applied for WML & VoiceXML reverse engineering </li></ul><ul><ul><li>Description of languages meta-models and derivation rules. </li></ul></ul>
    150. 151. RE of WML Phone Interfaces <ul><li>Example </li></ul><ul><ul><li>Prototype using XSLT + XPATH </li></ul></ul><ul><ul><li>Local process allowing no design alternatives </li></ul></ul>WML <p> Yahoo! ID: <input name=&quot;login&quot; value=&quot;&quot; format=&quot;*m&quot; />Password: <input type=&quot;password&quot; name= &quot;passwd&quot; value=&quot;&quot; format=&quot;*m&quot; /> <anchor title=&quot;OK&quot;>Submit <go method=&quot;post&quot; href=&quot;/raw?&quot;> USIXML <textComponent id= &quot;textComponent_1&quot; glueHorizontal =&quot;left“ isVisible=&quot;true&quot; isEnabled= &quot;true&quot; defaultContent=&quot;Yahoo! ID: <textComponent id= &quot;textComponent_2&quot; glueHorizontal= &quot;left&quot; defaultContent=&quot;&quot; isEditable= &quot;true&quot; isVisible=&quot;true&quot;/> [Cui,2005]
    151. 152. Results of the Evaluation <ul><li>Case studies illustrating various HTML RE techniques </li></ul><ul><li>Study of reengineering of the Sedan-Bouillon Website thanks to two FE tools: Teresa and QtkXML </li></ul>
    152. 153. Results of the Evaluation <ul><li>With Teresa </li></ul><ul><ul><li>RE applied without retargeting </li></ul></ul><ul><ul><li>USIXML CUI model used as input in Teresa that performs translations for another context of use </li></ul></ul><ul><ul><li>Produces the Web site designed for Pocket PC (XHTML) </li></ul></ul>
    153. 154. Results of the Evaluation <ul><li>With QtkiXML </li></ul>Original UI Reengineered UI
    154. 155. What do we have so far? Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use UsiXML unsupported model UsiXML supported model User S Let’s go for adaptation [Calvary et al.,2003] Reification Abstraction Platform S Environment S
    155. 156. What do we want to get? Final user Interface T Concrete user Interface T Task and Domain T Abstract user Interface T T=Target context of use Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use UsiXML unsupported model UsiXML supported model User S User T [Calvary et al.,2003] Environment T Reification Abstraction Reflexion Translation Platform S Environment S Platform T
    156. 157. Rules for Graceful Degradation <ul><li>Constitution of an original corpus of rules </li></ul><ul><li>Typology of rules, following the CAMELEON framework: </li></ul><ul><ul><li>(Rules at the Final User Interface level) </li></ul></ul><ul><ul><li>Rules at the Concrete User Interface level </li></ul></ul><ul><ul><li>Rules at the Abstract User Interface level </li></ul></ul><ul><ul><li>Rules at the Tasks & Concepts level </li></ul></ul><ul><li>Structured description of these rules </li></ul>[Florins,2006]
    157. 158. Rules at the Concrete UI level <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul></ul><ul><ul><li>Modification rules </li></ul></ul><ul><ul><li>Substitution rules </li></ul></ul><ul><ul><li>Removal rules </li></ul></ul><ul><li>Transformation of graphical relationships </li></ul><ul><ul><li>Reorientation rules </li></ul></ul><ul><ul><li>Moving rules </li></ul></ul>[Florins,2006]
    158. 159. <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul></ul><ul><ul><li>Modification rules </li></ul></ul><ul><ul><li>Substitution rules </li></ul></ul><ul><ul><li>Removal rules </li></ul></ul><ul><li>Transformation of graphical relationships </li></ul><ul><ul><li>Reorientation rules </li></ul></ul><ul><ul><li>Moving rules </li></ul></ul>Rules at the Concrete UI level [Florins,2006]
    159. 160. Rules at the Concrete UI level <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul></ul><ul><ul><li>Modification rules </li></ul></ul><ul><ul><li>Substitution rules </li></ul></ul><ul><ul><li>Removal rules </li></ul></ul><ul><li>Transformation of graphical relationships </li></ul><ul><ul><li>Reorientation rules </li></ul></ul><ul><ul><li>Moving rules </li></ul></ul>[Florins,2006]
    160. 161. Rules at the Concrete UI level <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul></ul><ul><ul><li>Modification rules </li></ul></ul><ul><ul><li>Substitution rules </li></ul></ul><ul><ul><li>Removal rules </li></ul></ul><ul><li>Transformation of graphical relationships </li></ul><ul><ul><li>Reorientation rules </li></ul></ul><ul><ul><li>Moving rules </li></ul></ul>[Florins,2006]
    161. 162. Rules at the Concrete UI level <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul></ul><ul><ul><li>Modification rules </li></ul></ul><ul><ul><li>Substitution rules </li></ul></ul><ul><ul><li>Removal rules </li></ul></ul><ul><li>Transformation of graphical relationships </li></ul><ul><ul><li>Reorientation rules </li></ul></ul><ul><ul><li>Moving rules </li></ul></ul>[Florins,2006]
    162. 163. Rules at the Concrete UI level <ul><li>Transformation of graphical objects </li></ul><ul><ul><li>Resizing rules </li></ul></ul><ul><ul><li>Modification rules </li></ul></ul><ul><ul><li>Substitution rules </li></ul></ul><ul><ul><li>Removal rules </li></ul></ul><ul><li>Transformation of graphical relationships </li></ul><ul><ul><li>Reorientation rules </li></ul></ul><ul><ul><li>Moving rules </li></ul></ul>[Florins,2006]
    163. 164. Rules at the Abstract UI level <ul><li>Spitting rules </li></ul><ul><li>Consist in breaking the initial UI into chunks </li></ul><ul><li>+ adding transitions </li></ul>[Florins,2006]
    164. 165. Rules at the Abstract UI level <ul><li>Important because: </li></ul><ul><ul><li>Difficult and significant step: generates important changes into the very structure of the UI </li></ul></ul><ul><ul><li>Appreciated by users </li></ul></ul><ul><li>Supporting algorithms developed during the thesis. </li></ul><ul><li>Originality: involve UI description at several abstraction levels </li></ul><ul><ul><li>Can be rely on the sole CUI level </li></ul></ul><ul><ul><li>Can exploit information from the AUI / task models. </li></ul></ul>[Florins,2006]
    165. 166. Rules at the Abstract UI level Source interface (in the graphical editor GrafiXML) (b) Execution of the splitting rule (a) box box box <ul><li>Application of the rule using CUI level information </li></ul>[Florins,2006]
    166. 167. Rules at the Abstract UI level <ul><li>Application of the rule using task level information </li></ul>[Florins,2006]
    167. 168. Rules at the Tasks&Concepts level <ul><li>Task deletion </li></ul><ul><li>Information summarization </li></ul><ul><li>… </li></ul>
    168. 169. GD Plug-in (4) <ul><li>Sections of rules </li></ul>[Florins et al.,2006]
    169. 170. GD Plug-in (4) <ul><li>Sections of rules </li></ul>
    170. 171. GD Plug-in (4) <ul><li>Sections of rules </li></ul>
    171. 172. GD Plug-in (5) <ul><li>Rules selection / parameters </li></ul>
    172. 173. GD Plug-in (6) <ul><li>Results </li></ul>
    173. 174. Multi-platform for Emergency [Amouh et al.,2005]
    174. 175. Multi-platform for Emergency <ul><li>Three platforms </li></ul><ul><ul><li>Pocket PC </li></ul></ul><ul><ul><li>Desktop PC </li></ul></ul><ul><ul><li>Wall Screens </li></ul></ul>
    175. 176. Multi-platform for Emergency <ul><li>Model and method </li></ul><ul><ul><li>Design the reference screen first </li></ul></ul><ul><ul><li>Refine the others screens later </li></ul></ul><ul><ul><ul><li>By applying graceful degradation </li></ul></ul></ul><ul><ul><ul><li>By applying transformation techniques </li></ul></ul></ul>
    176. 177. Graceful degradation [Florins & Vanderdonckt,2004]
    177. 178. Transformation rules
    178. 179. Transformation rules Add all >> Add > << Remove all < Remove >> > << < > < Group box Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 1 Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7
    179. 180. Transformation rules
    180. 181. Considering the context of use in designing user interfaces <ul><li>Context of use = triple </li></ul><ul><ul><li>User </li></ul></ul><ul><ul><li>Platform </li></ul></ul><ul><ul><li>Environment </li></ul></ul><ul><li>Each time one of these facets change, the context changes </li></ul>Generation for multiple platforms [Plomp & Keranen,2004]
    181. 182. Context model
    182. 183. Adapted from [Palanque,2002] Target Applications, Domains Notations and tools User Interface Interaction Technique 2006 TODAY 2007 No Interaction Technique Automated, batch systems Nothing Character UIs Business applications Screen definitions Graphical User Interfaces Information systems Entity-relationship Attribute model State-transition diagrams Multi-platform User Interfaces Web, Java applications Task model, context model, UML ,… Virtual Reality User Interfaces 3D Applications Scene model
    183. 184. Virtualisation of UIs <ul><li>Going from 2D to 3D </li></ul><ul><ul><li>Mapping widgets in 2D to 3D </li></ul></ul>[Vanderdonckt et al., 2004]
    184. 185. Developed by Donatien Grolaux <ul><li>Problem: how to design a UI that takes care of multiple displays? </li></ul><ul><li>Solution: Migration = DEMIPLAT </li></ul><ul><li>DistriXML = software architecture for migrating UIs from one platform to another at run-time </li></ul>DistriXML [Grolaux & Vanderdonckt,2005] Pencil Palette Painting Painting tool
    185. 186. Why take care of multiple displays? [Czerwinsky,2005] Effects of Display Size on Task Times 0 20 40 60 80 100 120 140 160 DISPLAY Average Task Time (Seconds) Small Large
    186. 187. Why take care of multiple displays? The tasks were easy to perform 0 1 2 3 4 5 Small Large Display Size Average Rating (1=Disagree, 5=Agree) [Czerwinsky,2005]
    187. 188. Why take care of multiple displays? [Czerwinsky,2005]
    188. 189. Demonstration DistriXML [Grolaux & Vanderdonckt,2005]
    189. 190. Demonstration using two displays from two different computers DistriXML
    190. 191. Example using a Pocket PC DistriXML
    191. 192. UI Migration: DEMIPLAT <ul><li>De tach </li></ul>DistriXML
    192. 193. UI Migration: DEMIPLAT <ul><li>De tach - Mi grate </li></ul>DistriXML
    193. 194. UI Migration: DEMIPLAT <ul><li>De tach - Mi grate - Pl astify </li></ul>DistriXML
    194. 195. UI Migration: DEMIPLAT <ul><li>De tach - Mi grate - Pl astify - At tach </li></ul>DistriXML
    195. 196. This is not a floating toolbar Process DistriXML
    196. 197. This is migration ! Computer B Process Process Computer A DistriXML
    197. 198. User model <ul><li>User population = hierarchy of user stereotypes </li></ul><ul><li>User stereotype = set of parameters </li></ul><ul><ul><li>Context independent </li></ul></ul><ul><ul><ul><li>Age, gender, motor skills, general preference,… </li></ul></ul></ul><ul><ul><li>Context dependent </li></ul></ul><ul><ul><ul><li>Preference for certain contents </li></ul></ul></ul><ul><ul><ul><li>Interaction history </li></ul></ul></ul><ul><ul><ul><li>Can be initiated by the corresponding context independent parameter </li></ul></ul></ul><ul><ul><ul><li>Can overwrite this parameter </li></ul></ul></ul>
    198. 199. Why user preference is important? <ul><li>End users tend to manipulate more efficiently the user interface they prefer </li></ul><ul><ul><li>When end users notice significant differences in their own performance and when they consider performance as important, then preference and performance are highly correlated. </li></ul></ul><ul><ul><li>Preference is more important than performance </li></ul></ul>[Johnsgard,1995] If information density is low If information density is high Simple choice Multiple choice Input with exte n- sion Predefined sele c- tio n
    199. 200. Example of preference-based design <ul><li>Goal = to input the temperature of a human body </li></ul><ul><ul><li>Solution n°1: edit box </li></ul></ul><ul><ul><li>Solution n°2: list box </li></ul></ul><ul><ul><li>Solution n°3: drop down list </li></ul></ul><ul><ul><li>Solution n°4: field with scroll bar </li></ul></ul><ul><ul><li>Solution n°5: thermometer </li></ul></ul>
    200. 201. Users <ul><li>Properties of the user also constrain interaction. </li></ul><ul><ul><li>Expertise </li></ul></ul><ul><ul><ul><li>Does the user understand this interaction technique? </li></ul></ul></ul><ul><ul><li>Privileges </li></ul></ul><ul><ul><ul><li>What tasks is the user allowed to perform? </li></ul></ul></ul><ul><ul><li>Preferences </li></ul></ul><ul><ul><ul><li>What tasks is the user likely to perform? </li></ul></ul></ul>
    201. 202. Modeling a range of designs <ul><li>UI modeling languages must model a range of design possibilities to account for different contexts </li></ul><ul><li>This could be accomplished by modeling preferences instead of decisions </li></ul>
    202. 203. A Spectrum of Preference Relations <ul><li>We describe a spectrum of preference relations </li></ul><ul><ul><li>Less flexible relations are easier to implement but less powerful </li></ul></ul><ul><ul><li>More flexible relations can model highly adaptive UIs, but are difficult to implement </li></ul></ul>[Eisenstein,2001]
    203. 204. The Spectrum <ul><li>Concrete Preferences </li></ul><ul><ul><li>Bindings </li></ul></ul><ul><ul><li>Simple Preference </li></ul></ul><ul><ul><li>Ordered Preference </li></ul></ul><ul><li>Abstract Preference </li></ul><ul><li>Design guideline </li></ul>
    204. 205. Modeling preferences in UsiXML <ul><li>UsiXML = User Interface eXtensible Markup Language (http://www.usixml.org) </li></ul>User1 Domain2 Domain3 Presen- tation4 Presen- tation5 User2
    205. 206. Bindings <ul><li>Bindings are simple one-to-one mappings </li></ul><ul><ul><li>Task A is implemented by interactor X </li></ul></ul><ul><ul><li>A is the condition </li></ul></ul><ul><ul><li>X is the target </li></ul></ul><ul><li>Bindings are our point of departure, </li></ul><ul><ul><li>Most current modeling formalisms support only bindings </li></ul></ul>
    206. 207. Binding example <ul><li>User U1 prefers presentation element P1 </li></ul><PREFERENCE ID=&quot;G1&quot;> <CONDITIONS> <RELATION_STATEMENT DEFINITION=&quot;condition&quot; REFERENCE=&quot;U1&quot;/> </CONDITIONS> <TARGETS> <RELATION_STATEMENT DEFINITION=&quot;target&quot; REFERENCE=&quot;P1&quot;/> </TARGETS> </PREFERENCE>
    207. 208. Simple Preferences <ul><li>Simple Preferences are many-to-one mappings </li></ul><ul><ul><li>For user U, task A is implemented using interactor X </li></ul></ul><ul><ul><li>Conditions: U, A </li></ul></ul><ul><ul><li>Target: X </li></ul></ul>
    208. 209. Simple preference example <ul><li>User U1 prefers presentation element P1 for representing domain model DM1 </li></ul><PREFERENCE ID=&quot;G1&quot;> <CONDITIONS> <RELATION_STATEMENT DEFINITION=&quot;condition&quot; REFERENCE=&quot;U1&quot;/> <RELATION_STATEMENT DEFINITION=&quot;condition&quot; REFERENCE=&quot;DM1&quot;/> </CONDITIONS> <TARGETS> <RELATION_STATEMENT DEFINITION=&quot;target&quot; REFERENCE=&quot;P1&quot;/> </TARGETS> </PREFERENCE>
    209. 210. Bindings <ul><li>Bindings are simple one-to-one mappings </li></ul><ul><ul><li>Task A is implemented by interactor X </li></ul></ul><ul><ul><li>A is the condition </li></ul></ul><ul><ul><li>X is the target </li></ul></ul><ul><li>Bindings are our point of departure, </li></ul><ul><ul><li>Most current modeling formalisms support only bindings </li></ul></ul>
    210. 211. Binding example <ul><li>User U1 prefers presentation element P1 </li></ul><PREFERENCE ID=&quot;G1&quot;> <CONDITIONS> <RELATION_STATEMENT DEFINITION=&quot;condition&quot; REFERENCE=&quot;U1&quot;/> </CONDITIONS> <TARGETS> <RELATION_STATEMENT DEFINITION=&quot;target&quot; REFERENCE=&quot;P1&quot;/> </TARGETS> </PREFERENCE>
    211. 212. Ordered Preferences <ul><li>An ordered preference is a type of many-to-many mapping </li></ul><ul><ul><li>For user U, task A should be implemented by one of the following interactors: X1, X2, or X3 </li></ul></ul><ul><ul><ul><li>Multiple targets, in order of preference: X1, X2, X3 </li></ul></ul></ul><ul><ul><ul><li>Targets can be assigned “priorities” </li></ul></ul></ul><ul><ul><ul><ul><li>This specifies the “level of preference” </li></ul></ul></ul></ul><ul><ul><ul><ul><li>E.g.: X1 = 100, X2 = 95, X3 = 20 </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Means choose either X1 or X2, it doesn’t matter much </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>E.g.: X1 = 100, X2 = 35, X3 = 35 </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Try hard to choose X1 </li></ul></ul></ul></ul></ul>
    212. 213. Ordered preference example <ul><li>User U1 prefers presentation element P1 to presentation element P2, for representing domain model DM1 </li></ul><PREFERENCE ID=&quot;G1&quot;> <CONDITIONS> <RELATION_STATEMENT DEFINITION=&quot;condition&quot; REFERENCE=&quot;U1&quot;/> <RELATION_STATEMENT DEFINITION=&quot;condition&quot; REFERENCE=&quot;DM1&quot;/> </CONDITIONS> <TARGETS> <RELATION_STATEMENT DEFINITION=&quot;target&quot; REFERENCE=&quot;P1&quot;> < ATTRIBUTE_STATEMENT DEFINITION=&quot;priority&quot;> 100 </ATTRIBUTE_STATEMENT> </RELATION_STATEMENT> <RELATION_STATEMENT DEFINITION=&quot;target&quot; REFERENCE=&quot;P2&quot;> <ATTRIBUTE_STATEMENT DEFINITION=&quot;priority&quot;> 50 </ATTRIBUTE_STATEMENT> </RELATION_STATEMENT> </TARGETS> </PREFERENCE>
    213. 214. Summary of Concrete Preferences <ul><li>All of the previous are concrete preferences </li></ul><ul><ul><li>They model the target directly </li></ul></ul><ul><ul><li>Can be combined with a model of the contextual constraints: </li></ul></ul><ul><ul><ul><li>Example: For task A, </li></ul></ul></ul><ul><ul><ul><ul><li>interactors X1, X2, and X3 are preferred, in order </li></ul></ul></ul></ul><ul><ul><ul><ul><li>But X1 cannot be implemented on current device </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Therefore, X2 is used </li></ul></ul></ul></ul>
    214. 215. Abstract Preferences <ul><li>Abstract preferences do not model the target directly </li></ul><ul><li>They model characteristics of the target </li></ul><ul><li>Abstract preferences have criteria for making mappings </li></ul><ul><ul><li>Preferential, stating which is preferred </li></ul></ul><ul><ul><li>Logical, stating which is allowed </li></ul></ul>
    215. 216. Example of Abstract Preference <ul><li>User U prefers the interactor </li></ul><ul><ul><li>Occupying the least screen space </li></ul></ul><ul><ul><ul><li>This is a preferential criteria </li></ul></ul></ul><ul><ul><ul><li>The behavior is to minimize the value of the criteria </li></ul></ul></ul><ul><ul><li>But not less than 20 pixels wide </li></ul></ul><ul><ul><ul><li>This is a logical criteria </li></ul></ul></ul>
    216. 217. Abstract preference example <ul><li>User U1 prefers the presentation element that occupies the least screen space </li></ul><PREFERENCE ID=&quot;G1&quot;> <CONDITIONS> <RELATION_STATEMENT DEFINITION=&quot;condition&quot; REFERENCE=&quot;U1&quot;/> </CONDITIONS> <TARGETS> <RELATION_STATEMENT DEFINITION=&quot;target&quot; REFERENCE=&quot;P1&quot;/> <RELATION_STATEMENT DEFINITION=&quot;target&quot; REFERENCE=&quot;P2&quot;/> </TARGETS> <CRITERIA> <RELATION_STATEMENT DEFINITION=&quot;criteria&quot; REFERENCE=&quot;screen space&quot;> <ATTRIBUTE_STATEMENT DEFINITION=&quot;criteria type&quot;> preferential </ATTRIBUTE_STATEMENT> <ATTRIBUTE_STATEMENT DEFINITION=&quot;criteria behavior&quot;> minimum </ATTRIBUTE_S TATEMENT> </RELATION_STATEMENT> </CRITERIA> </PREFERENCE>
    217. 218. Multiple Preferential Criteria <ul><li>Preferential criteria may have “priorities” </li></ul><ul><ul><li>This specifies their relative importance </li></ul></ul><ul><ul><li>E.g.: “Prefer presentation elements that require few clicks and are small, but the criteria of having few clicks is most important” </li></ul></ul><ul><ul><li>Here, the “number of clicks” criteria has a higher priority than the “size” criteria </li></ul></ul>
    218. 219. Design Guidelines <ul><li>Abstract preferences consider criteria that relate to the targets </li></ul><ul><li>Design guidelines consider criteria of any object in the UI model </li></ul><ul><li>Frequently take the form of If-Then statements </li></ul><ul><li>Example: “If the experience level of User U is less than “ expert”, then map to interactor X1 ” </li></ul>
    219. 220. Chaining Abstract Preferences <ul><li>Design guidelines can have other preference relations as their targets: </li></ul><ul><ul><li>“ If User U is not an expert, then select the navigational structure with the least cognitive complexity.” </li></ul></ul><ul><ul><li>In this case, the target of the design guideline is an abstract preference </li></ul></ul><ul><ul><ul><li>The abstract preference has a single, preferential criteria: </li></ul></ul></ul><ul><ul><ul><ul><li>Minimize cognitive complexity </li></ul></ul></ul></ul>
    220. 221. Chaining Design Guidelines <ul><li>Design guidelines can also target other design guidelines </li></ul><ul><li>This can create complex logical formulae: </li></ul><ul><ul><li>“ If User U is an expert, and the platform is a PDA, then choose one of X1, X2, or X3, whichever is smallest” </li></ul></ul><ul><li>Such formulae can model highly adaptive design phenomena </li></ul>
    221. 222. How to argue for preference? <ul><li>Use the QOC notation </li></ul>Question? [McLean & Belotti,1998] Criteria 1 Criteria 2 Criteria j Criteria m Option 1 Option 2 Option i Option n = negatively affects = positively affects
    222. 223. Preference example Problem Solution 1 Solution 2 (RE1) (RE2) suggests suggests avoids contradicts respects = for = against (A1) = drop down list requires less space than radio buttons (A2) = some possible values become obsolete when they are infrequently used (A3) = drop down list shows better then current value than the possible values (A4) = radio buttons are faster and easier to manipulate than drop down list (A5) = radio buttons are recommended in Microsoft Windows style guide (A6) = radio buttons do not allow users to change the values drop down list radio buttons (A4) is contradicted by (A2) and (A3) : the widget should be more used for output than input. (A6) is contradicted by (A3) : it is better to present all possible values than only one at a time (A5) is an autority argument , and can fall in front of (A1)+(A2)+(A3) [Vanderdonckt,1997] (A1) (A2) (A3) (A4) (A5) (A6) Standard compatibility (e.g. Windows) Cognitive respect Minimal actions Dialog representativity Prompting Clarity contradicts
    223. 224. Theory or argumentation [Toulmin,1975] [Perelman, 1978] Design problem Design option parameter (parameter, value) Is described by Is solved by Accepted solution Set of admissible solutions Solution 1 Solution 2 Solution n Counter-argument (rebuttal, counter-claim) Argument (ground, typically data) can be rejected because of Can be accepted thanks to Is justified by Design claim Warrant Is reinforced by Qualifier satisfied unsatisfied Is related to corresponds to Is justified by Is qualified by
    224. 225. Mixed reality <ul><li>Mixed reality = real world + digital world </li></ul>
    225. 226. Mixed reality for games [Alterface,2006]
    226. 227. Mixed reality in surgery <ul><li>No representation for </li></ul><ul><ul><li>Tasks in real or virtual world </li></ul></ul><ul><ul><li>Tasks independent of user interaction e.g. tracking object of interest </li></ul></ul><ul><ul><li>Various contexts of use in Mixed Reality Systems i.e. changing point-of-view </li></ul></ul>Adaptive IS composition based on focus of interest [Trevisan et al.,2004]
    227. 228. Composing the MIS Real CIO Patient head Virtual CIOs Anatomic organs Virtual CIOs Annotations Primary object of interest
    228. 229. Annotation placement strategy <ul><li>No overlapping annotations for readability </li></ul><ul><li>Highest priority of virtual object (3) </li></ul><ul><li>Medium priority of real object (2) </li></ul><ul><li>Low priority of background (1) </li></ul><ul><li>Annotations placed from peripheral to central zones </li></ul><ul><li>Other objects as part of Background </li></ul>
    229. 230. Context-aware Multimodal Annotation Rendering (a) Background context (b) Background/Real Object context (c) Virtual Object context <ul><li>- what information to display </li></ul><ul><li>annotation positions </li></ul><ul><li>modality switch </li></ul>User focus controls adaptive presentation management
    230. 231. Context model in mixed reality <ul><li>A context model describes all the entities that may influence carrying out the interactive task of user with the intended UI </li></ul><ul><ul><li>Environment model - s uch attributes may be physical (e.g., lighting conditions), psychological (e.g., level of stress), and organization (e.g., location and role definition in the organization chart). </li></ul></ul><ul><ul><li>Platform model – couple of software/hardware, such atributes may be screen resolution/size </li></ul></ul><ul><ul><li>User model – stereotypes, sub-stereotypes (profiles) </li></ul></ul>
    231. 232. Plasticity analysis and usability evaluation Plasticity Domains in the MIS Plasticity thresholds and context boundary values Inter-context continuity Perceptive, cognitive, functional intra-context continuity and inter-context continuity Intra-context / inter-context frequency and frequency switch Refined Metrics for Plastic MR systems
    232. 233. What do we have now? Final user Interface T Concrete user Interface T Task and Domain T Abstract user Interface T T=Target context of use Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use UsiXML unsupported model UsiXML supported model User S User T Environment T Reification Abstraction Reflexion Translation Platform S Environment S Platform T
    233. 234. Conclusion <ul><li>User model is described by parameters </li></ul><ul><ul><li>Context independent </li></ul></ul><ul><ul><ul><li>Age, gender, motor skills, general preference,… </li></ul></ul></ul><ul><ul><li>Context dependent </li></ul></ul><ul><ul><ul><li>Preference for certain contents </li></ul></ul></ul><ul><ul><ul><li>Interaction history </li></ul></ul></ul><ul><ul><ul><li>Can be initiated by the corresponding context independent parameter </li></ul></ul></ul><ul><ul><ul><li>Can overwrite this parameter </li></ul></ul></ul><ul><li>Preference specification </li></ul><ul><ul><li>Is stored in user model </li></ul></ul><ul><ul><li>Can be processed by « critiquing tools » based on argumentation, QOC notation </li></ul></ul>
    234. 235. Evaluation [Skezely,1996] 0 2000 4000 6000 8000 10000 12000 14000 16000 Word KB KB Parser Query Network ops. UI states Presentation Actions Update Interactions Application logic User interface Generated code Models
    235. 236. Coverage <ul><li>How to improve our MDA-based method? </li></ul><ul><ul><li>High ceiling </li></ul></ul><ul><ul><li>Low threshold </li></ul></ul><ul><ul><li>Wide walls </li></ul></ul>Capabilities Resources (time, experience,…) 100% 50% Ceiling Threshold First generation Second generation MDA CASE tools Interface builders and integrated environments Resource win for applications supported by MDA-compliant tools Resource win for applications supported by first-generation
    236. 237. What’s the User Interface language? <ul><li>Today </li></ul><ul><ul><li>Programming languages: Visual Basic, Java, C++,… </li></ul></ul><ul><ul><li>Markup languages: HTML, XUL,… </li></ul></ul><ul><li>Tomorrow </li></ul><ul><ul><li>Multimodal interfaces: XHTML + VoiceXML = X+V </li></ul></ul><ul><ul><li>Vectorial interfaces: SVG, LZX </li></ul></ul><ul><ul><li>Virtual reality interfaces: VRML97, XVRML </li></ul></ul><ul><ul><li>3D user interfaces: X3D, MEL </li></ul></ul><ul><ul><li>Tabletop interface: OpenGL </li></ul></ul><ul><ul><li>Many new modalities are arriving </li></ul></ul><ul><li>Conclusion </li></ul><ul><ul><li>It is impossible to evolve/survive without any modelling approach, even for user interfaces </li></ul></ul><ul><ul><li>It is required to progress also on the design process </li></ul></ul>
    237. 238. Conclusion: the big picture of MDA UsiXML model: Abstract user interface UsiXML model: Concrete user interface Rendering Final user interface UsiXML models: task, domain Generative programming Graph transformations Graph transformations Derivation rules IdealXML ReversiXML FlashiXML QtkXML JaviXML VisualiXML TransformiXML GrafiXML VisiXML SketchiXML FormiXML KnowiXML MethodiXML
    238. 239. The Invisible UI All Applications Adapted from [Palanque,2002] Target Applications, Domains Notations and tools User Interface Interaction Technique No more programming: only models 2020 2006 TODAY 2008 2007 No Interaction Technique Automated, batch systems Nothing Character UIs Business applications Screen definitions Graphical User Interfaces Information systems Entity-relationship Attribute model State-transition diagrams Multi-platform User Interfaces Web, Java applications Task model, context model, UML ,… Virtual Reality User Interfaces 3D Applications Scene model Mixed Reality User Interfaces Command and control systems, games World models Tangible UIs Embodied UIs Physical models Embarked syst
    239. 240. Thank you very much for your attention For more information and downloading, http://www.isys.ucl.ac.be/bchi http://www.usixml.org User Interface eXtensible Markup Language http://www.similar.cc European network on Multimodal UIs Special thanks to all members of the team!
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×