Model-based engineering of multi-platform, synchronous & collaborative UIs


Published on

The paper describes an engineering method for building user interfaces for ubiquitous environments. The method comprises of several extensions in the UsiXML family
of modes as well as design and runtime support so as to enable multi-platform, synchronous and collaborative interactions. We demonstrate key concepts of the method and their application by elaborating a scenario of collaborative co-play
of the ‘tic-tac-toe’ game. The specific use case features synchronous co-engagement in game play by remote users (players or observers) using desktop PCs or Android devices.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Model-based engineering of multi-platform, synchronous & collaborative UIs

  1. 1. Model-based engineering of multi-platform, George Vellisa Dimitrios Kotsalisa synchronous & Demosthenes Akoumianakisa Jean Vanderdoncktbcollaborative UIs aDepartment of Applied Informatics & Multimedia, Technological Education Institution of Crete, Greece bUniversité catholique de Louvain, Extending UsiXML for Louvain Interaction Laboratory ( Louvain School of Management, Belgiumpolymorphic user interface specification
  2. 2. Introduction• User interface challenges for software engineers – Programming intensive – Broad range of interaction components and widgets – Platform specificity (GUI objects, Web widgets, etc) – Lack of interoperability due to platform specificity• (Some) Model-based engineering claims – UI engineering = Model specification – Focus on design rather than implementation – Linking (rather than direct calls) to platform-specific libraries
  3. 3. Introduction• User interface challenges for software engineers – UI development time takes a large portion of time System feasibility Validation Software plans and requirements UI Design occupied 45% of the total Validation Product design amount of time devoted to each activity Verification Detailed design Mean = 44% Verification Code Unit test 50% Integration Product verification Implementation System test Operations and maintenance 37% Revalidation
  4. 4. Retrospective assessment• Claims proven useful – Model specifications can provide an engineering basis for UI engineering – Run-time linking alleviates the complexity of programming• Claims proven possible – Multi-platform execution – Relatively straightforward Uis (e.g., form-based UI)
  5. 5. Retrospective assessment (Cont.)• Several initiatives – One of them is UsiXML which is used in this work• Reference Architectures – CAMELEON Reference framework • Recommended by W3C Working Group on Model-based User Interface Design • UsiXML is compliant with this framework• Plethora of methods with some focus on collaboration – FlowiXML, AMENITES, CIAM, TOUCHE
  6. 6. Retrospective assessment (cont.)• Challenges still pending attention – non-standard and customized widgets? – synchronous and collaborative UIs? – new affordances such as social awareness in distributed collaborative settings? – radically different interaction vocabularies? – design and run-time support tools for all the above?
  7. 7. Research design• Theory development – Build abstraction to capture interaction components as assemblages of capabilities (or affordances) – Substantiate such capabilities (or affordances) to polymporphic instantiation schemes – Specify polymorphic properties in a model-based fashion• Engineering framework – Extend UsiXML as needed to support the new specification – Provide dedicated design tools and run-time components – Demonstrate the concept’s validity through pilot scenarios
  8. 8. Why abstraction?• Consider various generations of ‘buttons’ – GUI buttons (in the 90s)• The (primitive) assemblage of button = {two-state device to exercise control over functionality}
  9. 9. Why abstraction? • A continuum of abstractions (Cameleon Ref. Framework)Task & Task Method triggered: download fileConcepts Classes Object: computer file Modality-independent AIO type Control AIOAbstract UserInterface (AUI) Modality-dependent Software control AIO Physical control AIO AIO Platform- independent Graphical 2D push button Graphical 3D push button Physical push buttonConcrete User CIO typeInterface (CUI) Platform- HTML pushbutton Windows OSF/Motif VRML97/X3D Software Function independent push button XmButton button key key CIO Code <input type=submit value=“Download" name=btnG>Final UserInterface (FUI) Rendering Download Down Download Loa d
  10. 10. ‘Adding’ social scent• Scented button (cf. Willett et al., 2007)• The (augmented) assemblage of ‘scented’ button = { Two-state device to exercise control } + { Collectivity (i.e., who else is in the same state of mind)}
  11. 11. ‘Adding’ expression of intention• ‘Like’ buttons (in the Web 2.0)• The (new) assemblage of the ‘like’ button = {Two-state devices to exercise control} + {Expression of opinion (i.e. I like this)}
  12. 12. Putting them all together• ‘Like’ buttons with social scent• The (combined) assemblage = {Two-state device to exercise control} + {Expression of opinion (i.e. I like this)} + {Collectivity (i.e., who else is in the same state of mind)}
  13. 13. Generalizing the concept• Why limit our creativity to conventional widgets? – elastic buttons and their interactive behaviour Note the embedded affordances
  14. 14. Adding to complexity• What happens when an elastic button is to be instantiated across devices and platforms
  15. 15. Motivating question• How can we … – model digital assemblages in interaction components? – instantiate them as the need arises and depending on context / platform capabilities – integrate them into specifications as first class elements / properties – compile specs that are general enough
  16. 16. (One) Answer: Polymorphic instantiation• A scheme that relies on implementation agnostic (i.e., abstract) specifications of UIs – i.e. an abstract button can be instantiated as • a conventional GUI button when implemented with the ‘primitive’ assemblage • a scented button when implemented with the ‘augmented’ assemblage • a ‘like’ button when implemented with the ‘combined’ assemblage• At run-time and once user and usage context parameters are discovered, the implementation agnostic spec is translated to context-specific interaction vocabularies using dedicated tools
  17. 17. Polymorphic Widget Specifications Basic concept• Introduce new widgets and their properties as first- class design elements WSL Libraries Resources Widget Archive
  18. 18. The role of MBUI engineering• Specify complex capabilities & affordances in a model-based fashion – Devise required properties – Define model elements – Create specification – Mapping models
  19. 19. The role of MBUI engineering the case of a socially aware button
  20. 20. A new set of models• Extend UsiXML as needed to support the new specification – New set of models • Widget Resource Model • Behavior model – Additional models for collaborative work • Squad model • Consistency model • Abstraction model • Session model
  21. 21. A new design tool
  22. 22. A new run-time environment UI-1 UI-N Design Environment UI Design Tool deployUI 6.attachStates 5.buildUI Widget Spec Tool Platform 4.getArchives Runtime Environment Server Widget Archives Repository P.S WSL Resources 1.initPS Dependencies Collaboration Plugin Collaboration Plugin UI-Model Repository Social Social Abstraction 2.join Social Social Abstraction Social Abstraction Abstraction Notification Queue Social Abstraction Abstraction Proxy Proxy Classes Proxy Proxy Classes Proxy Classes Classes Proxy Classes Classes Session ManagerapplyChanges Web-Services LayerdistributeChanges Polling Polling trackChanges Axis2-FrameworkClient-1 Server Side Framework (SSF) Client-N
  23. 23. UIDL’2011 (simple) desktop scenario
  24. 24. MDSD’12 multiplatform scenario (in the paper)Player-1:Desktop/PC Player-2: Android/mobile Viewer-1: Desktop/PC
  25. 25. Post MDSD’12 scenario• Collaborative assembly of vacation packages• User roles – Business partner (desktop) – Administrator (desktop or mobile – Platforms (desktop/swing, mobile/Android) – Context (synchronous and collaborative)• Complex and custom widgets (i.e., elastic buttons)• Synchronous collaboration
  26. 26. The desktop caseadministrator’s view of vacation package / swing
  27. 27. The desktop casebusiness partners’ view of vacation package / swing
  28. 28. The mobile caseadministrator’s view of vacation package - Android Due to space constraints certain functions become automatically adapted to vocal interaction
  29. 29. The designer’s role?Designing for certain affordances 1 2 3 4 5 6
  30. 30. The run-time instantiation the specified affordances enabled 1 2 3 4 5 6 2
  31. 31. Future work & acknowledgements• Current and on-going work – Advanced toolkits (i.e., visualization) – Run-time adaptivity and UI plasticity in distributed and ubiquitous settings – ‘Social’ games• Acknowledgements – The work is supported by ARCHIMEDES III – Thanks to Université catholique de Louvain (UCL) for supporting the first two authors’ PhD dissertations • Due to be submitted in 2013