Challenges in using MDE for HMI                           Multi-Paradigm Modeling                            of HMI Applic...
Challenges in using MDE for HMI  HMI modelingOutline : HMI modeling                                  HMI modeling  Tom Men...
Challenges in using MDE for HMI  HMI modelingExample of a HMI application           3D visual drawing tool           Hapti...
Challenges in using MDE for HMI  HMI modelingMulti-Paradigm Modeling for HMI      Goal      Use MDE techniques to advance ...
Challenges in using MDE for HMI  HMI modelingTask modeling      Example : ConcurTaskTrees task model for a digital home  T...
Challenges in using MDE for HMI  HMI modelingBehavioural modeling      Example : Statecharts model of hand behaviour [DM11...
Challenges in using MDE for HMI  HMI modelingBehavioural modeling      Example : Statecharts mode of UI component behaviou...
Challenges in using MDE for HMI  HMI modelingBehavioural modeling      Example : Statecharts model of graphical scene beha...
Challenges in using MDE for HMI  HMI modelingBehavioural modeling      Example : Petri net model of double click behaviour...
Challenges in using MDE for HMI  HMI modelingPhysical modeling      Goal      Modeling physical behaviour or interaction o...
Challenges in using MDE for HMI  Research ChallengesOutline : Research Challenges                            Research Chal...
Challenges in using MDE for HMI  Research ChallengesResearch Challenges      Top-down      Come up with domain-specific mod...
Challenges in using MDE for HMI  Research ChallengesFormal reasoning - Example      Work by Combéfis and Pecheur to analyse...
Challenges in using MDE for HMI  Research ChallengesFormal reasoning - Example       Work by Combéfis and Pecheur to analys...
Challenges in using MDE for HMI  Research ChallengesDomain-Specific Modeling for HMI      DSML      How to define a DSML tha...
Challenges in using MDE for HMI  Research ChallengesDomain-Specific Modeling for HMI      Domain-specific model : specific to...
Challenges in using MDE for HMI  Research Challenges      Visual formalisms      Are used for formal reasoning and analysi...
Challenges in using MDE for HMI  Research ChallengesVisual Formalisms      Use most appropriate formalism, at most suitabl...
Challenges in using MDE for HMI  Research ChallengesVisual Formalisms      Which semantics to associate to each formalism ...
Challenges in using MDE for HMI  Research ChallengesMore challenges             How to facilitate evolution ?             ...
Challenges in using MDE for HMI  Research ChallengesThank you                                  Thank you for your attentio...
Challenges in using MDE for HMI  Research ChallengesBibliography I            Sébastien Combéfis and Charles Pecheur.      ...
Challenges in using MDE for HMI  Research ChallengesBibliography II            David Navarre, Philippe A. Palanque, Jean-F...
Upcoming SlideShare
Loading in …5
×

Multi-Paradigm Modeling of HMI Applications

1,555 views

Published on

This presentation about the use of multi-paradigm modeling for human-machine interaction was given at the Universiteit Antwerpen as part of a research seminar of the Belgian PAI research project MoVES (Modeling, Verification and Evolution of Software).
It presents an overview of the planned research in the context of the PhD of Romuald Deshayes, under supervision by Tom Mens at the Software Engineering Lab of the University of Mons, Belgium.

Published in: Education, Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,555
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
29
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Multi-Paradigm Modeling of HMI Applications

  1. 1. Challenges in using MDE for HMI Multi-Paradigm Modeling of HMI Applications Tom Mens Romuald Deshayes Software Engineering Lab MoVES event, Universiteit Antwerpen, 20 December 2011 Tom Mens – UMONS 1 / 23
  2. 2. Challenges in using MDE for HMI HMI modelingOutline : HMI modeling HMI modeling Tom Mens – UMONS 2 / 23
  3. 3. Challenges in using MDE for HMI HMI modelingExample of a HMI application 3D visual drawing tool Haptic (gestural) user interface (using hands) based on Kinect Creation and manipulation of 3D objects, using OpenGL Complex UI behaviour represented using statecharts www.youtube.com/ watch?v=lVcqzWTnpMY Tom Mens – UMONS 3 / 23
  4. 4. Challenges in using MDE for HMI HMI modelingMulti-Paradigm Modeling for HMI Goal Use MDE techniques to advance the state-of-the-art in HMI (HCI) Multi-paradigm modeling. Existing approaches focus on : Task modeling : understand user tasks to improve UIs (cf. Jean Vanderdonckt - UCL) Presentation modeling : focus on visual appearance of the application (e.g., OpenGL) User modeling : focus on the user characteristics Behavioural modeling : model how the application reacts to users actions (e.g. event-based) Physical modeling : physical behaviour of real or virtual objects ... Tom Mens – UMONS 4 / 23
  5. 5. Challenges in using MDE for HMI HMI modelingTask modeling Example : ConcurTaskTrees task model for a digital home Tom Mens – UMONS 5 / 23
  6. 6. Challenges in using MDE for HMI HMI modelingBehavioural modeling Example : Statecharts model of hand behaviour [DM11] Tom Mens – UMONS 6 / 23
  7. 7. Challenges in using MDE for HMI HMI modelingBehavioural modeling Example : Statecharts mode of UI component behaviour [DM11] Tom Mens – UMONS 7 / 23
  8. 8. Challenges in using MDE for HMI HMI modelingBehavioural modeling Example : Statecharts model of graphical scene behaviour [DM11] Tom Mens – UMONS 8 / 23
  9. 9. Challenges in using MDE for HMI HMI modelingBehavioural modeling Example : Petri net model of double click behaviour (Palanque - Toulouse) [NPLB09] Tom Mens – UMONS 9 / 23
  10. 10. Challenges in using MDE for HMI HMI modelingPhysical modeling Goal Modeling physical behaviour or interaction of virtual objects E.g., falling, rolling, colliding, bouncing, breaking, . . . Requires continuous models (e.g., differential equations) Challenge Combining discrete and continuous models Possible solution : hybrid automata Discrete automata containing differential equations in the states Tom Mens – UMONS 10 / 23
  11. 11. Challenges in using MDE for HMI Research ChallengesOutline : Research Challenges Research Challenges Tom Mens – UMONS 11 / 23
  12. 12. Challenges in using MDE for HMI Research ChallengesResearch Challenges Top-down Come up with domain-specific modeling languages (DSML) that combine multiple views on HMI modeling. Bottom-up Enable formal reasoning over HMI models Detect unintended behavioural problems (nondeterminism, non-termination, ...) Detect HMI vulnerabilities, inconsistencies, incompleteness Facilitate code generation ... Tom Mens – UMONS 12 / 23
  13. 13. Challenges in using MDE for HMI Research ChallengesFormal reasoning - Example Work by Combéfis and Pecheur to analyse HMI vulnerabilities [CP09] Using labeled-transition systems (LTS) semantics to generate and analyse users’ mental models Example : LTS of user’s mental model of a lamp, compared to the real system model. Tom Mens – UMONS 13 / 23
  14. 14. Challenges in using MDE for HMI Research ChallengesFormal reasoning - Example Work by Combéfis and Pecheur to analyse HMI vulnerabilities Example : accident with Therac-25 medical system due to a incorrect operator-machine interaction Fig. 7. Statechart model of the Therac-25 medical system [16]. The enter command is represented with and the up command is represented with ↑. Tom Mens – UMONS 14 / 23
  15. 15. Challenges in using MDE for HMI Research ChallengesDomain-Specific Modeling for HMI DSML How to define a DSML that combines the multiple modeling languages/paradigms used for HMI ? Tom Mens – UMONS 15 / 23
  16. 16. Challenges in using MDE for HMI Research ChallengesDomain-Specific Modeling for HMI Domain-specific model : specific to application domain E.g., robotics, gaming, home automation, virtual reality Platform model : describes platform specificities modality (vocal, tactile, WIMP) input devices (hardware sensors, keyboard, mouse, Kinect) output (screen, VR goggles, voice) ... Transformation model : weaves both models into platform-specific model, using model transformation technology Tom Mens – UMONS 16 / 23
  17. 17. Challenges in using MDE for HMI Research Challenges Visual formalisms Are used for formal reasoning and analysis over the models Questions : Which formalisms to use for which purpose ? How to combine results from different formalisms ? How to feed back results into DSML ? Remark : model transformations are models too, amenable to formal analysis Tom Mens – UMONS 17 / 23
  18. 18. Challenges in using MDE for HMI Research ChallengesVisual Formalisms Use most appropriate formalism, at most suitable abstraction level, for a given purpose Possible candidates Statecharts : for event-driven, deterministic, reactive systems Petri nets : for analysing concurrency-related issues Labelled Transition Systems Hybrid automata (combining discrete and continuous models) Logic-based or constraint-based formalisms Graph transformation (e.g., for model transformation) Tom Mens – UMONS 18 / 23
  19. 19. Challenges in using MDE for HMI Research ChallengesVisual Formalisms Which semantics to associate to each formalism ? Which tools to use for formal reasoning ? Graph transformation tools (e.g., AGG, Fujaba, GrGen) Model manipulation tools (ATL, EMF, Epsilon, AtoMPM, TCore, . . . ) HyTech tool (for hybrid automata) Model checkers SAT solvers ... Tom Mens – UMONS 19 / 23
  20. 20. Challenges in using MDE for HMI Research ChallengesMore challenges How to facilitate evolution ? How to generate code from models ? How to develop generic framework (independent of particular DSM or platform model) ? Where and how to use graph transformation ? Killer app ? Augmented reality, gaming, home automation, robotics ? Tom Mens – UMONS 20 / 23
  21. 21. Challenges in using MDE for HMI Research ChallengesThank you Thank you for your attention ! Questions, suggestions ? Tom Mens – UMONS 21 / 23
  22. 22. Challenges in using MDE for HMI Research ChallengesBibliography I Sébastien Combéfis and Charles Pecheur. A bisimulation-based approach to the analysis of human-computer interaction. In EICS, pages 101–110, 2009. Romuald Deshayes and Tom Mens. Statechart modelling of interactive gesture-based applications. In CoDeisMoto Workshop, co-located with Interact 2011, 2011. Tom Mens – UMONS 22 / 23
  23. 23. Challenges in using MDE for HMI Research ChallengesBibliography II David Navarre, Philippe A. Palanque, Jean-François Ladry, and Eric Barboni. Icos : A model-based user interface description technique dedicated to interactive systems addressing usability, reliability and scalability. ACM Trans. Comput.-Hum. Interact., 16(4) :56, 2009. Tom Mens – UMONS 23 / 23

×