SlideShare a Scribd company logo

Model-driven engineering of multimodal user interfaces

•Download as PPT, PDF•
•
Jean Vanderdonckt
Jean Vanderdonckt

This presentation explains to use model-driven engineering techniques for producing multimodal user interfaces for the Web.

BusinessTechnology
1 of 42
Model-Driven Engineering of User Interfaces for Multimodal Web Applications Jean Vanderdonckt Université catholique de Louvain (UCL) Louvain School of Management (LSM) Information Systems Unit (ISYS) Belgian Laboratory of Computer-Human Interaction (BCHI) http://www.isys.ucl.ac.be/bchi
What is a multimodal user interface? ,[object Object],[object Object],[object Object]
What is a multimodal user interface? ,[object Object],[Tison,2003]
What is a multimodal user interface? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[Bodnar,2004]
What is a multimodal user interface? ,[object Object],[object Object],[object Object],[object Object],[Bellik,2002]
Why are multimodal user interfaces important? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[IntuiLab,2006]
Why are multimodal user interfaces challenging? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Concepts: The CARE Properties ,[object Object],[object Object],[object Object],[object Object],[object Object],Put that there [Bolt,1979]
What are the appropriate abstractions? ,[object Object],[object Object],[object Object],[object Object]
Why a design space? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Multimodal Web Applications ,[object Object],[object Object],[object Object],[object Object],[W3C,2006]
Solution space ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[Stanciulescu et al.,2005]
Design options for graphical web UIs Design options for graphical UIs Sub-task presentation Sub-task navigation Concretization of navigation & control Navigation concretization Control concretization
Design options for graphical web UIs ,[object Object],Sub-task presentation separated combined one by one combined box tabbed dialog box float window many at once float window all in one areas with separators group boxes bulleted list numbered list Abstract level Concrete level
Sub-task presentation – example ,[object Object],[object Object],T11 T12 T13 T1 AC11 AC12 AC13 AC1 T11 T12 T13 T1 AC11 AC12 AC13 NAC LHS RHS NAC LHS RHS
Design options for graphical web UIs ,[object Object],Navigation concretization [Stanciulescu,2006] placement cardinality local global simple multiple
Sub-task navigation - example ,[object Object],[object Object],AC1 T11 T12 T13 T1 AC11 AC12 AC13 AIC111 AIC121 AIC131 AC1 T11 T12 T13 T1 AC11 AC12 AC13 AIC11 NAC LHS RHS NAC LHS RHS
Design options for graphical web UIs ,[object Object],Sub-task navigation asynchronous sequential (synchronous)
Design options for graphical web UIs ,[object Object],Concretization of navigation & control separated combined
Design options for graphical web UIs ,[object Object],Control concretizations placement cardinality local global multiple simple
Design options for multimodal web UIs ,[object Object],[object Object],[object Object],Design options for multimodal UIs [Scapin & Bastien,1997] [Vanderdonckt,1995] Prompt Grouping / Distinction of items Immediate feedback Guidance
Design options for multimodal web UIs ,[object Object],[object Object],[object Object],[object Object]
Design options for multimodal web UIs ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Design options for multimodal web UIs ,[object Object],[object Object],[object Object],[object Object]
Design options for multimodal web UIs ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Multimodal example ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[Stanciulescu,2006] Prompt: multimodal Please say your name (vocal + graphical) 1 Input: multimodal Juan Gonzalez 2 (vocal + graphical) Juan Gonzalez Guidance for input: iconic Guidance for feedback: iconic Feedback:multimodal Your answer was: Juan Gonzalez 3 (graphical + vocal)
Vocal Concrete Interaction Objects ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Vocal Concrete Interaction Objects ,[object Object]
Multimodal Concrete Interaction Objects ,[object Object],outputText + inputText vocalPrompt + vocalInput outputText + inputText + vocalInput + vocalFeedback + guidance groupBox + radioButtons vocalMenu + vocalItems + vocalInput groupBox + radioButtons + vocalMenu + vocalItems + vocalInput + guidance Graphical Vocal Multimodal
Concrete UI Relationships ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Tool support ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Multimodal Concrete Interaction Objects ,[object Object],[object Object]
Specification of transformation ,[object Object],[object Object],[object Object]
How transformation rules work… ,[object Object],[object Object],[object Object],[Limbourg,2004] Transformation rule Initial Model Resulting Model
UsiXML syntax of transformation rules NAC NAC RHS mappings
Transformation rule NAC: LHS: RHS: Multimodal prompt Guidance for input Multimodal input Guidance for output Vocal feedback
Case studies ,[object Object]
Conclusion ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Conclusion ,[object Object],[object Object],[object Object],[object Object],[object Object],[Stanciulescu & Vanderdonckt,2006] [Beaudouin-Lafon,2004]
Future work ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
References ,[object Object],[object Object],[object Object]
Thank you very much for your attention For more information and downloading, http://www.isys.ucl.ac.be/bchi http://www.usixml.org User Interface eXtensible Markup Language http://www.similar.cc European network on Multimodal UIs Special thanks to all members of the team!

Recommended

Model-driven engineering of user interfaces
Model-driven engineering of user interfacesModel-driven engineering of user interfaces
Model-driven engineering of user interfaces
Jean Vanderdonckt
 
User Interface Design
User Interface DesignUser Interface Design
User Interface Design
Gil Pasiona
 
SECh1516
SECh1516SECh1516
SECh1516
Joe Christensen
 
User interface-design
User interface-designUser interface-design
User interface-design
DarkHorse Technologies Pvt Ltd
 
Nota sendiri hci-HCI
Nota sendiri hci-HCINota sendiri hci-HCI
Nota sendiri hci-HCI
Shafy Fify
 
User Interface Design in Software Engineering SE15
User Interface Design in Software Engineering SE15User Interface Design in Software Engineering SE15
User Interface Design in Software Engineering SE15
koolkampus
 
HCI 3e - Ch 11: User support
HCI 3e - Ch 11: User supportHCI 3e - Ch 11: User support
HCI 3e - Ch 11: User support
Alan Dix
 
Unit iii(part c - user interface design)
Unit iii(part c - user interface design)Unit iii(part c - user interface design)
Unit iii(part c - user interface design)
BALAJI A
 

Recommended

Model-driven engineering of user interfaces
Model-driven engineering of user interfacesModel-driven engineering of user interfaces
Model-driven engineering of user interfaces
Jean Vanderdonckt
 
User Interface Design
User Interface DesignUser Interface Design
User Interface Design
Gil Pasiona
 
SECh1516
SECh1516SECh1516
SECh1516
Joe Christensen
 
User interface-design
User interface-designUser interface-design
User interface-design
DarkHorse Technologies Pvt Ltd
 
Nota sendiri hci-HCI
Nota sendiri hci-HCINota sendiri hci-HCI
Nota sendiri hci-HCI
Shafy Fify
 
User Interface Design in Software Engineering SE15
User Interface Design in Software Engineering SE15User Interface Design in Software Engineering SE15
User Interface Design in Software Engineering SE15
koolkampus
 
HCI 3e - Ch 11: User support
HCI 3e - Ch 11: User supportHCI 3e - Ch 11: User support
HCI 3e - Ch 11: User support
Alan Dix
 
Unit iii(part c - user interface design)
Unit iii(part c - user interface design)Unit iii(part c - user interface design)
Unit iii(part c - user interface design)
BALAJI A
 
Chapter 12 user interface design
Chapter 12 user interface designChapter 12 user interface design
Chapter 12 user interface design
SHREEHARI WADAWADAGI
 
USER INTERFACE DESIGN PPT
USER INTERFACE DESIGN PPTUSER INTERFACE DESIGN PPT
USER INTERFACE DESIGN PPT
vicci4041
 
user support system in HCI
user support system in HCIuser support system in HCI
user support system in HCI
Usman Mukhtar
 
Seii unit5 ui_design
Seii unit5 ui_designSeii unit5 ui_design
Seii unit5 ui_design
Ahmad sohail Kakar
 
HCI 3e - Ch 7: Design rules
HCI 3e - Ch 7: Design rulesHCI 3e - Ch 7: Design rules
HCI 3e - Ch 7: Design rules
Alan Dix
 
Ch09-User Interface Design
Ch09-User Interface DesignCh09-User Interface Design
Ch09-User Interface Design
Fajar Baskoro
 
Unit 7 performing user interface design
Unit 7 performing user interface designUnit 7 performing user interface design
Unit 7 performing user interface design
Preeti Mishra
 
User Interface Design
User Interface DesignUser Interface Design
User Interface Design
JReifman
 
16 user interfacedesign
16 user interfacedesign16 user interfacedesign
16 user interfacedesign
randhirlpu
 
Mobile UI Design – User Centered Design and UI Best Practices
Mobile UI Design – User Centered Design and UI Best PracticesMobile UI Design – User Centered Design and UI Best Practices
Mobile UI Design – User Centered Design and UI Best Practices
OXD
 
User interface design(sommerville) bangalore university
User interface design(sommerville) bangalore universityUser interface design(sommerville) bangalore university
User interface design(sommerville) bangalore university
Jaisha Shankar
 
CSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 DocumentationCSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 Documentation
Salocin Dot TEN
 
Graphical user interface of web form
Graphical user interface of web formGraphical user interface of web form
Graphical user interface of web form
mentorrbuddy
 
golden rules of user interface design
golden rules of user interface designgolden rules of user interface design
golden rules of user interface design
gadige harshini
 
HCI 3e - Ch 6: HCI in the software process
HCI 3e - Ch 6: HCI in the software processHCI 3e - Ch 6: HCI in the software process
HCI 3e - Ch 6: HCI in the software process
Alan Dix
 
Software Usability Implications in Requirements and Design
Software Usability Implications in Requirements and DesignSoftware Usability Implications in Requirements and Design
Software Usability Implications in Requirements and Design
Natalia Juristo
 
Pressman ch-11-component-level-design
Pressman ch-11-component-level-designPressman ch-11-component-level-design
Pressman ch-11-component-level-design
Oliver Cheng
 
User interface design
User interface designUser interface design
User interface design
Slideshare
 
Android design patterns in mobile application development presentation
Android design patterns in mobile application development presentationAndroid design patterns in mobile application development presentation
Android design patterns in mobile application development presentation
Michail Grigoropoulos
 
Methodology for the Development of Vocal User Interfaces
Methodology for the Development of Vocal User InterfacesMethodology for the Development of Vocal User Interfaces
Methodology for the Development of Vocal User Interfaces
Jean Vanderdonckt
 
2015-04-29 research seminar
2015-04-29 research seminar2015-04-29 research seminar
2015-04-29 research seminar
ifi8106tlu
 
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Jean Vanderdonckt
 

More Related Content

What's hot

USER INTERFACE DESIGN PPT
USER INTERFACE DESIGN PPTUSER INTERFACE DESIGN PPT
USER INTERFACE DESIGN PPT
vicci4041
 
16 user interfacedesign
16 user interfacedesign16 user interfacedesign
16 user interfacedesign
randhirlpu
 
Graphical user interface of web form
Graphical user interface of web formGraphical user interface of web form
Graphical user interface of web form
mentorrbuddy
 
Software Usability Implications in Requirements and Design
Software Usability Implications in Requirements and DesignSoftware Usability Implications in Requirements and Design
Software Usability Implications in Requirements and Design
Natalia Juristo
 
Pressman ch-11-component-level-design
Pressman ch-11-component-level-designPressman ch-11-component-level-design
Pressman ch-11-component-level-design
Oliver Cheng
 
User interface design
User interface designUser interface design
User interface design
Slideshare
 

What's hot (19)

Chapter 12 user interface design
Chapter 12 user interface designChapter 12 user interface design
Chapter 12 user interface design
 
USER INTERFACE DESIGN PPT
USER INTERFACE DESIGN PPTUSER INTERFACE DESIGN PPT
USER INTERFACE DESIGN PPT
 
user support system in HCI
user support system in HCIuser support system in HCI
user support system in HCI
 
Seii unit5 ui_design
Seii unit5 ui_designSeii unit5 ui_design
Seii unit5 ui_design
 
HCI 3e - Ch 7: Design rules
HCI 3e - Ch 7: Design rulesHCI 3e - Ch 7: Design rules
HCI 3e - Ch 7: Design rules
 
Ch09-User Interface Design
Ch09-User Interface DesignCh09-User Interface Design
Ch09-User Interface Design
 
Unit 7 performing user interface design
Unit 7 performing user interface designUnit 7 performing user interface design
Unit 7 performing user interface design
 
User Interface Design
User Interface DesignUser Interface Design
User Interface Design
 
16 user interfacedesign
16 user interfacedesign16 user interfacedesign
16 user interfacedesign
 
Mobile UI Design – User Centered Design and UI Best Practices
Mobile UI Design – User Centered Design and UI Best PracticesMobile UI Design – User Centered Design and UI Best Practices
Mobile UI Design – User Centered Design and UI Best Practices
 
User interface design(sommerville) bangalore university
User interface design(sommerville) bangalore universityUser interface design(sommerville) bangalore university
User interface design(sommerville) bangalore university
 
CSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 DocumentationCSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 Documentation
 
Graphical user interface of web form
Graphical user interface of web formGraphical user interface of web form
Graphical user interface of web form
 
golden rules of user interface design
golden rules of user interface designgolden rules of user interface design
golden rules of user interface design
 
HCI 3e - Ch 6: HCI in the software process
HCI 3e - Ch 6: HCI in the software processHCI 3e - Ch 6: HCI in the software process
HCI 3e - Ch 6: HCI in the software process
 
Software Usability Implications in Requirements and Design
Software Usability Implications in Requirements and DesignSoftware Usability Implications in Requirements and Design
Software Usability Implications in Requirements and Design
 
Pressman ch-11-component-level-design
Pressman ch-11-component-level-designPressman ch-11-component-level-design
Pressman ch-11-component-level-design
 
User interface design
User interface designUser interface design
User interface design
 
Android design patterns in mobile application development presentation
Android design patterns in mobile application development presentationAndroid design patterns in mobile application development presentation
Android design patterns in mobile application development presentation
 

Similar to Model-driven engineering of multimodal user interfaces

Methodology for the Development of Vocal User Interfaces
Methodology for the Development of Vocal User InterfacesMethodology for the Development of Vocal User Interfaces
Methodology for the Development of Vocal User Interfaces
Jean Vanderdonckt
 
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Jean Vanderdonckt
 
Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...
Jean Vanderdonckt
 
Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...
Serenoa Project
 
Designing and evaluating web sites using universal design principles (hands on)
Designing and evaluating web sites using universal design principles (hands on)Designing and evaluating web sites using universal design principles (hands on)
Designing and evaluating web sites using universal design principles (hands on)
Howard Kramer
 
Device Independence
Device IndependenceDevice Independence
Device Independence
bjornh
 
A Methodology for the Development of Vocal User Interfaces
A Methodology for the Development of Vocal User InterfacesA Methodology for the Development of Vocal User Interfaces
A Methodology for the Development of Vocal User Interfaces
David Céspedes-Hernández
 
What Is Interaction Design
What Is Interaction DesignWhat Is Interaction Design
What Is Interaction Design
Graeme Smith
 
Designidm C L P Revised
Designidm C L P RevisedDesignidm C L P Revised
Designidm C L P Revised
guest7ff51a
 
Model-Based Approaches to Reengineering Web Pages
Model-Based Approaches to Reengineering Web PagesModel-Based Approaches to Reengineering Web Pages
Model-Based Approaches to Reengineering Web Pages
Jean Vanderdonckt
 
Information and Instructional Technologies
Information and Instructional TechnologiesInformation and Instructional Technologies
Information and Instructional Technologies
Alaa Sadik
 

Similar to Model-driven engineering of multimodal user interfaces (20)

Methodology for the Development of Vocal User Interfaces
Methodology for the Development of Vocal User InterfacesMethodology for the Development of Vocal User Interfaces
Methodology for the Development of Vocal User Interfaces
 
2015-04-29 research seminar
2015-04-29 research seminar2015-04-29 research seminar
2015-04-29 research seminar
 
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
 
Ni week 2018_ux_lab_viewcandothat
Ni week 2018_ux_lab_viewcandothatNi week 2018_ux_lab_viewcandothat
Ni week 2018_ux_lab_viewcandothat
 
Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...
 
Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...
 
Designing and evaluating web sites using universal design principles (hands on)
Designing and evaluating web sites using universal design principles (hands on)Designing and evaluating web sites using universal design principles (hands on)
Designing and evaluating web sites using universal design principles (hands on)
 
Device Independence
Device IndependenceDevice Independence
Device Independence
 
A Methodology for the Development of Vocal User Interfaces
A Methodology for the Development of Vocal User InterfacesA Methodology for the Development of Vocal User Interfaces
A Methodology for the Development of Vocal User Interfaces
 
What Is Interaction Design
What Is Interaction DesignWhat Is Interaction Design
What Is Interaction Design
 
Designidm C L P Revised
Designidm C L P RevisedDesignidm C L P Revised
Designidm C L P Revised
 
Ch11
Ch11Ch11
Ch11
 
Human-Computer Interaction: Adaptation, Distribution, Evaluation
Human-Computer Interaction: Adaptation, Distribution, EvaluationHuman-Computer Interaction: Adaptation, Distribution, Evaluation
Human-Computer Interaction: Adaptation, Distribution, Evaluation
 
Model-Based Approaches to Reengineering Web Pages
Model-Based Approaches to Reengineering Web PagesModel-Based Approaches to Reengineering Web Pages
Model-Based Approaches to Reengineering Web Pages
 
User Interface Composition with UsiXML
User Interface Composition with UsiXMLUser Interface Composition with UsiXML
User Interface Composition with UsiXML
 
When User Interface Patterns Become Mobile
When User Interface Patterns Become MobileWhen User Interface Patterns Become Mobile
When User Interface Patterns Become Mobile
 
UX Practices for dApps on Tezos
UX Practices for dApps on TezosUX Practices for dApps on Tezos
UX Practices for dApps on Tezos
 
ACCESSIBLE project concept and achievements
ACCESSIBLE project concept and achievementsACCESSIBLE project concept and achievements
ACCESSIBLE project concept and achievements
 
Information and Instructional Technologies
Information and Instructional TechnologiesInformation and Instructional Technologies
Information and Instructional Technologies
 
Summ11 useinterx
Summ11 useinterxSumm11 useinterx
Summ11 useinterx
 

More from Jean Vanderdonckt

Engineering the Transition of Interactive Collaborative Software from Cloud C...
Engineering the Transition of Interactive Collaborative Software from Cloud C...Engineering the Transition of Interactive Collaborative Software from Cloud C...
Engineering the Transition of Interactive Collaborative Software from Cloud C...
Jean Vanderdonckt
 
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
Jean Vanderdonckt
 
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
Jean Vanderdonckt
 
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
Jean Vanderdonckt
 
Engineering Slidable User Interfaces with Slime
Engineering Slidable User Interfaces with SlimeEngineering Slidable User Interfaces with Slime
Engineering Slidable User Interfaces with Slime
Jean Vanderdonckt
 
User-centred Development of a Clinical Decision-support System for Breast Can...
User-centred Development of a Clinical Decision-support System for Breast Can...User-centred Development of a Clinical Decision-support System for Breast Can...
User-centred Development of a Clinical Decision-support System for Breast Can...
Jean Vanderdonckt
 
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Jean Vanderdonckt
 
Attach Me, Detach Me, Assemble Me like you Work
Attach Me, Detach Me, Assemble Me like you WorkAttach Me, Detach Me, Assemble Me like you Work
Attach Me, Detach Me, Assemble Me like you Work
Jean Vanderdonckt
 
The Impact of Comfortable Viewing Positions on Smart TV Gestures
The Impact of Comfortable Viewing Positions on Smart TV GesturesThe Impact of Comfortable Viewing Positions on Smart TV Gestures
The Impact of Comfortable Viewing Positions on Smart TV Gestures
Jean Vanderdonckt
 
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper BodyHead and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
Jean Vanderdonckt
 
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for SmartphonesG-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
Jean Vanderdonckt
 
Vector-based, Structure Preserving Stroke Gesture Recognition
Vector-based, Structure Preserving Stroke Gesture RecognitionVector-based, Structure Preserving Stroke Gesture Recognition
Vector-based, Structure Preserving Stroke Gesture Recognition
Jean Vanderdonckt
 
An ontology for reasoning on body-based gestures
An ontology for reasoning on body-based gestures An ontology for reasoning on body-based gestures
An ontology for reasoning on body-based gestures
Jean Vanderdonckt
 

More from Jean Vanderdonckt (20)

To the end of our possibilities with Adaptive User Interfaces
To the end of our possibilities with Adaptive User InterfacesTo the end of our possibilities with Adaptive User Interfaces
To the end of our possibilities with Adaptive User Interfaces
 
Engineering the Transition of Interactive Collaborative Software from Cloud C...
Engineering the Transition of Interactive Collaborative Software from Cloud C...Engineering the Transition of Interactive Collaborative Software from Cloud C...
Engineering the Transition of Interactive Collaborative Software from Cloud C...
 
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
 
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
 
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
 
Gesture-based information systems: from DesignOps to DevOps
Gesture-based information systems: from DesignOps to DevOpsGesture-based information systems: from DesignOps to DevOps
Gesture-based information systems: from DesignOps to DevOps
 
Engineering Slidable User Interfaces with Slime
Engineering Slidable User Interfaces with SlimeEngineering Slidable User Interfaces with Slime
Engineering Slidable User Interfaces with Slime
 
Evaluating Gestural Interaction: Models, Methods, and Measures
Evaluating Gestural Interaction: Models, Methods, and MeasuresEvaluating Gestural Interaction: Models, Methods, and Measures
Evaluating Gestural Interaction: Models, Methods, and Measures
 
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
 
Designing Gestural Interaction: Challenges and Pitfalls
Designing Gestural Interaction: Challenges and PitfallsDesigning Gestural Interaction: Challenges and Pitfalls
Designing Gestural Interaction: Challenges and Pitfalls
 
Fundamentals of Gestural Interaction
Fundamentals of Gestural InteractionFundamentals of Gestural Interaction
Fundamentals of Gestural Interaction
 
Gestural Interaction, Is it Really Natural?
Gestural Interaction, Is it Really Natural?Gestural Interaction, Is it Really Natural?
Gestural Interaction, Is it Really Natural?
 
User-centred Development of a Clinical Decision-support System for Breast Can...
User-centred Development of a Clinical Decision-support System for Breast Can...User-centred Development of a Clinical Decision-support System for Breast Can...
User-centred Development of a Clinical Decision-support System for Breast Can...
 
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
 
Attach Me, Detach Me, Assemble Me like you Work
Attach Me, Detach Me, Assemble Me like you WorkAttach Me, Detach Me, Assemble Me like you Work
Attach Me, Detach Me, Assemble Me like you Work
 
The Impact of Comfortable Viewing Positions on Smart TV Gestures
The Impact of Comfortable Viewing Positions on Smart TV GesturesThe Impact of Comfortable Viewing Positions on Smart TV Gestures
The Impact of Comfortable Viewing Positions on Smart TV Gestures
 
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper BodyHead and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
 
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for SmartphonesG-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
 
Vector-based, Structure Preserving Stroke Gesture Recognition
Vector-based, Structure Preserving Stroke Gesture RecognitionVector-based, Structure Preserving Stroke Gesture Recognition
Vector-based, Structure Preserving Stroke Gesture Recognition
 
An ontology for reasoning on body-based gestures
An ontology for reasoning on body-based gestures An ontology for reasoning on body-based gestures
An ontology for reasoning on body-based gestures
 

Recently uploaded

The Abortion pills for sale in Qatar@Doha [+27737758557] []Deira Dubai Kuwait
The Abortion pills for sale in Qatar@Doha [+27737758557] []Deira Dubai KuwaitThe Abortion pills for sale in Qatar@Doha [+27737758557] []Deira Dubai Kuwait
The Abortion pills for sale in Qatar@Doha [+27737758557] []Deira Dubai Kuwait
daisycvs
 
Uneak White's Personal Brand Exploration Presentation
Uneak White's Personal Brand Exploration PresentationUneak White's Personal Brand Exploration Presentation
Uneak White's Personal Brand Exploration Presentation
uneakwhite
 
Only Cash On Delivery Call Girls In Sikandarpur Gurgaon ❤️8448577510 ⊹Escorts...
Only Cash On Delivery Call Girls In Sikandarpur Gurgaon ❤️8448577510 ⊹Escorts...Only Cash On Delivery Call Girls In Sikandarpur Gurgaon ❤️8448577510 ⊹Escorts...
Only Cash On Delivery Call Girls In Sikandarpur Gurgaon ❤️8448577510 ⊹Escorts...
lizamodels9
 
PARK STREET 💋 Call Girl 9827461493 Call Girls in Escort service book now
PARK STREET 💋 Call Girl 9827461493 Call Girls in Escort service book nowPARK STREET 💋 Call Girl 9827461493 Call Girls in Escort service book now
PARK STREET 💋 Call Girl 9827461493 Call Girls in Escort service book now
kapoorjyoti4444
 
Challenges and Opportunities: A Qualitative Study on Tax Compliance in Pakistan
Challenges and Opportunities: A Qualitative Study on Tax Compliance in PakistanChallenges and Opportunities: A Qualitative Study on Tax Compliance in Pakistan
Challenges and Opportunities: A Qualitative Study on Tax Compliance in Pakistan
vineshkumarsajnani12
 
Al Mizhar Dubai Escorts +971561403006 Escorts Service In Al Mizhar
Al Mizhar Dubai Escorts +971561403006 Escorts Service In Al MizharAl Mizhar Dubai Escorts +971561403006 Escorts Service In Al Mizhar
Al Mizhar Dubai Escorts +971561403006 Escorts Service In Al Mizhar
allensay1
 

Recently uploaded (20)

The Abortion pills for sale in Qatar@Doha [+27737758557] []Deira Dubai Kuwait
The Abortion pills for sale in Qatar@Doha [+27737758557] []Deira Dubai KuwaitThe Abortion pills for sale in Qatar@Doha [+27737758557] []Deira Dubai Kuwait
The Abortion pills for sale in Qatar@Doha [+27737758557] []Deira Dubai Kuwait
 
Uneak White's Personal Brand Exploration Presentation
Uneak White's Personal Brand Exploration PresentationUneak White's Personal Brand Exploration Presentation
Uneak White's Personal Brand Exploration Presentation
 
Berhampur 70918*19311 CALL GIRLS IN ESCORT SERVICE WE ARE PROVIDING
Berhampur 70918*19311 CALL GIRLS IN ESCORT SERVICE WE ARE PROVIDINGBerhampur 70918*19311 CALL GIRLS IN ESCORT SERVICE WE ARE PROVIDING
Berhampur 70918*19311 CALL GIRLS IN ESCORT SERVICE WE ARE PROVIDING
 
Only Cash On Delivery Call Girls In Sikandarpur Gurgaon ❤️8448577510 ⊹Escorts...
Only Cash On Delivery Call Girls In Sikandarpur Gurgaon ❤️8448577510 ⊹Escorts...Only Cash On Delivery Call Girls In Sikandarpur Gurgaon ❤️8448577510 ⊹Escorts...
Only Cash On Delivery Call Girls In Sikandarpur Gurgaon ❤️8448577510 ⊹Escorts...
 
WheelTug Short Pitch Deck 2024 | Byond Insights
WheelTug Short Pitch Deck 2024 | Byond InsightsWheelTug Short Pitch Deck 2024 | Byond Insights
WheelTug Short Pitch Deck 2024 | Byond Insights
 
Cuttack Call Girl Just Call 8084732287 Top Class Call Girl Service Available
Cuttack Call Girl Just Call 8084732287 Top Class Call Girl Service AvailableCuttack Call Girl Just Call 8084732287 Top Class Call Girl Service Available
Cuttack Call Girl Just Call 8084732287 Top Class Call Girl Service Available
 
Bangalore Call Girl Just Call♥️ 8084732287 ♥️Top Class Call Girl Service Avai...
Bangalore Call Girl Just Call♥️ 8084732287 ♥️Top Class Call Girl Service Avai...Bangalore Call Girl Just Call♥️ 8084732287 ♥️Top Class Call Girl Service Avai...
Bangalore Call Girl Just Call♥️ 8084732287 ♥️Top Class Call Girl Service Avai...
 
PARK STREET 💋 Call Girl 9827461493 Call Girls in Escort service book now
PARK STREET 💋 Call Girl 9827461493 Call Girls in Escort service book nowPARK STREET 💋 Call Girl 9827461493 Call Girls in Escort service book now
PARK STREET 💋 Call Girl 9827461493 Call Girls in Escort service book now
 
QSM Chap 10 Service Culture in Tourism and Hospitality Industry.pptx
QSM Chap 10 Service Culture in Tourism and Hospitality Industry.pptxQSM Chap 10 Service Culture in Tourism and Hospitality Industry.pptx
QSM Chap 10 Service Culture in Tourism and Hospitality Industry.pptx
 
Nanded Call Girl Just Call 8084732287 Top Class Call Girl Service Available
Nanded Call Girl Just Call 8084732287 Top Class Call Girl Service AvailableNanded Call Girl Just Call 8084732287 Top Class Call Girl Service Available
Nanded Call Girl Just Call 8084732287 Top Class Call Girl Service Available
 
Horngren’s Cost Accounting A Managerial Emphasis, Canadian 9th edition soluti...
Horngren’s Cost Accounting A Managerial Emphasis, Canadian 9th edition soluti...Horngren’s Cost Accounting A Managerial Emphasis, Canadian 9th edition soluti...
Horngren’s Cost Accounting A Managerial Emphasis, Canadian 9th edition soluti...
 
Puri CALL GIRL ❤️8084732287❤️ CALL GIRLS IN ESCORT SERVICE WE ARW PROVIDING
Puri CALL GIRL ❤️8084732287❤️ CALL GIRLS IN ESCORT SERVICE WE ARW PROVIDINGPuri CALL GIRL ❤️8084732287❤️ CALL GIRLS IN ESCORT SERVICE WE ARW PROVIDING
Puri CALL GIRL ❤️8084732287❤️ CALL GIRLS IN ESCORT SERVICE WE ARW PROVIDING
 
Buy gmail accounts.pdf buy Old Gmail Accounts
Buy gmail accounts.pdf buy Old Gmail AccountsBuy gmail accounts.pdf buy Old Gmail Accounts
Buy gmail accounts.pdf buy Old Gmail Accounts
 
PHX May 2024 Corporate Presentation Final
PHX May 2024 Corporate Presentation FinalPHX May 2024 Corporate Presentation Final
PHX May 2024 Corporate Presentation Final
 
SEO Case Study: How I Increased SEO Traffic & Ranking by 50-60% in 6 Months
SEO Case Study: How I Increased SEO Traffic & Ranking by 50-60% in 6 MonthsSEO Case Study: How I Increased SEO Traffic & Ranking by 50-60% in 6 Months
SEO Case Study: How I Increased SEO Traffic & Ranking by 50-60% in 6 Months
 
Falcon Invoice Discounting: The best investment platform in india for investors
Falcon Invoice Discounting: The best investment platform in india for investorsFalcon Invoice Discounting: The best investment platform in india for investors
Falcon Invoice Discounting: The best investment platform in india for investors
 
Challenges and Opportunities: A Qualitative Study on Tax Compliance in Pakistan
Challenges and Opportunities: A Qualitative Study on Tax Compliance in PakistanChallenges and Opportunities: A Qualitative Study on Tax Compliance in Pakistan
Challenges and Opportunities: A Qualitative Study on Tax Compliance in Pakistan
 
Berhampur CALL GIRL❤7091819311❤CALL GIRLS IN ESCORT SERVICE WE ARE PROVIDING
Berhampur CALL GIRL❤7091819311❤CALL GIRLS IN ESCORT SERVICE WE ARE PROVIDINGBerhampur CALL GIRL❤7091819311❤CALL GIRLS IN ESCORT SERVICE WE ARE PROVIDING
Berhampur CALL GIRL❤7091819311❤CALL GIRLS IN ESCORT SERVICE WE ARE PROVIDING
 
Phases of Negotiation .pptx
Phases of Negotiation .pptx Phases of Negotiation .pptx
Phases of Negotiation .pptx
 
Al Mizhar Dubai Escorts +971561403006 Escorts Service In Al Mizhar
Al Mizhar Dubai Escorts +971561403006 Escorts Service In Al MizharAl Mizhar Dubai Escorts +971561403006 Escorts Service In Al Mizhar
Al Mizhar Dubai Escorts +971561403006 Escorts Service In Al Mizhar
 

Model-driven engineering of multimodal user interfaces

  • 1. Model-Driven Engineering of User Interfaces for Multimodal Web Applications Jean Vanderdonckt Université catholique de Louvain (UCL) Louvain School of Management (LSM) Information Systems Unit (ISYS) Belgian Laboratory of Computer-Human Interaction (BCHI) http://www.isys.ucl.ac.be/bchi
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13. Design options for graphical web UIs Design options for graphical UIs Sub-task presentation Sub-task navigation Concretization of navigation & control Navigation concretization Control concretization
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35. UsiXML syntax of transformation rules NAC NAC RHS mappings
  • 36. Transformation rule NAC: LHS: RHS: Multimodal prompt Guidance for input Multimodal input Guidance for output Vocal feedback
  • 37.
  • 38.
  • 39.
  • 40.
  • 41.
  • 42. Thank you very much for your attention For more information and downloading, http://www.isys.ucl.ac.be/bchi http://www.usixml.org User Interface eXtensible Markup Language http://www.similar.cc European network on Multimodal UIs Special thanks to all members of the team!