Model Based Development For 3 D User Interfaces


Published on

A method to develop
3D User Interfaces following a model-based approach

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Model Based Development For 3 D User Interfaces

  1. 1. A model-based development for 3D User Interfaces Juan Manuel Gonzalez Calleros Université catholique de Louvain, School of Management (IAG) Information Systems Unit (ISYS) Louvain-la-Neuve, 13th June , 2006
  2. 2. Contents <ul><li>Introduction </li></ul><ul><li>Related Work </li></ul><ul><li>Model-based development for 3D User Interfaces </li></ul><ul><li>Case Study </li></ul><ul><li>Conclusion </li></ul>
  3. 3. Introduction - Problem <ul><li>Current methods: </li></ul><ul><ul><li>rarely provide the design knowledge </li></ul></ul><ul><ul><li>are more focusing on the implementation </li></ul></ul><ul><li>Available tools for 3D UIs are: Toolkits, interface builders, rendering engines </li></ul>
  4. 4. Introduction – proposal <ul><li>We propose: </li></ul><ul><ul><li>a user-centered approach </li></ul></ul><ul><ul><li>model-driven architecture </li></ul></ul><ul><ul><li>separation of concerns </li></ul></ul><ul><li>Our goal: </li></ul><ul><li>“to demonstrate the feasibility of a MDE-compliant method that is user-centered as opposed to contents-centric for developing 3D UIs” </li></ul>
  5. 5. Contents <ul><li>Introduction </li></ul><ul><li>Related Work </li></ul><ul><li>Model-based development for 3D User Interfaces </li></ul><ul><li>Case Study </li></ul><ul><li>Conclusion </li></ul>
  6. 6. Related work – 3D UIs generation <ul><li>Programmatic approach: OpenGL, Direct3D, C++, MCNL, VRML, X3D, … </li></ul><ul><li>Toolkit programming: Maya, Max3D, Crazy Eddie's GUI System, Anark, Alice </li></ul><ul><li>Render engines: Microsoft task gallery, infinite 3D Cube, Sphere XP, looking glass, Clara browser, 3DNA </li></ul><ul><li>Model based: ontoWEb (VR-Wise), ConGenIVE (VRIXML), InTML, Contigra </li></ul>
  7. 7. Related work – Programmatic Approach, VRML
  8. 8. Related work – Toolkit Approach, Anark
  9. 9. Related work – Render Engines, 3DNA
  10. 10. Related work – Model-based approach, CoGenIVE-VRIXML <ul><li>< GroupItem > </li></ul><ul><li>< UISlider paramID =&quot; 10 &quot;> </li></ul><ul><li>< Value min =&quot; 0 &quot; max =&quot; 255 &quot;/> </li></ul><ul><li>< Tickstyle orientation =&quot; horizontal “ position =&quot; both &quot; </li></ul><ul><li> frequency =&quot; 16 &quot;/> </li></ul><ul><li></ UISlider > </li></ul><ul><li>< Position > </li></ul><ul><li>< X > 1.0 </ X >< Y > 1.5 </ Y > </li></ul><ul><li></ Position > </li></ul><ul><li></ GroupItem > </li></ul>
  11. 11. Related work – Observations <ul><li>Methodological diversity </li></ul><ul><li>Not user centered approaches </li></ul><ul><li>Single entry and single output </li></ul><ul><li>Transformations are hidden </li></ul>
  12. 12. Related work – Shortcomings <ul><li>Lack of a methodology for developing 3D UIs </li></ul><ul><li>Lack of user task models </li></ul><ul><li>Lack of models independent to the final implementations </li></ul><ul><li>Lack of a toolkit for such work </li></ul><ul><li>Lack of genuine 3D UIs </li></ul>
  13. 13. Contents <ul><li>Introduction </li></ul><ul><li>Related work </li></ul><ul><li>Model-based development for 3D User Interfaces </li></ul><ul><li>Case Study </li></ul><ul><li>Conclusion </li></ul>
  14. 14. Model-based development of 3DUIs requirements <ul><li>Expressivity </li></ul><ul><li>Machine processable </li></ul><ul><li>Human readable </li></ul><ul><li>Methodological explicitness, flexibility, separation of concerns </li></ul>
  15. 15. We rely on… <ul><li>Cameleon Framework </li></ul><ul><li>UsiXML </li></ul><ul><li>UML class diagrams </li></ul><ul><li>Graph transformations </li></ul>
  16. 16. Method Outline
  17. 17. Development paths
  18. 18. Graph transformation tool support AGG
  19. 19. Method Step1 – Task and concepts description
  20. 20. Method Step1 - User Action Task Extension The action of identifying any user action Perceive The action of mediate user actions Mediate The action to communicate the user to the system or vice versa Communicate The existence of two different states of an item Toggle An action of modifying an item Modify Specifies the creation an item Create Specifies some sort of indication Indicate Specifies the end to an action Terminate Specifies the triggers of an action Trigger Definition actionType
  21. 21. Method Step1– 3D Task Patterns <ul><li>A pattern is not a finished design that can be transformed directly into code </li></ul><ul><li>It is a description or template for how to solve a problem that can be used in many different situations </li></ul><ul><li>We propose some patterns for common 3D user tasks </li></ul>
  22. 22. Method Step1 – 3D Task Patterns <ul><li>Navigation pattern </li></ul><ul><ul><li>Travel </li></ul></ul><ul><ul><li>Wayfinding </li></ul></ul>
  23. 23. Method Step1– 3D Task Patterns <ul><li>Navigation pattern </li></ul><ul><ul><li>Travel pattern </li></ul></ul><ul><ul><li>Wayfinding </li></ul></ul>
  24. 24. Method Step1– 3D Task Patterns <ul><li>Navigation pattern </li></ul><ul><ul><li>Travel </li></ul></ul><ul><ul><li>Wayfinding pattern </li></ul></ul>
  25. 25. Method Step1– Domain Model
  26. 26. Method Step1 – Software tool
  27. 27. Method Step2 – Abstract Model <ul><li>An Abstract Individual Component ( AIC ) is an abstraction that allows the description of interaction objects in a way that is independent of the modality </li></ul><ul><li>An Abstract Container (AC) is an entity allowing a logical grouping of other abstract containers or abstract individual components. </li></ul>
  28. 28. Method Step2 – Tool support
  29. 29. Method Step3 –To obtain a Concrete specification
  30. 30. Method Step3 –Taxonomy of 3D UIs
  31. 31. Method Step3 -Taxonomy of 3D UIs <ul><li>Pure Reality </li></ul><ul><li>Augmented reality </li></ul><ul><li>Augmented Virtuality </li></ul><ul><li>Virtual 3D GUI </li></ul><ul><li>Digital 3D GUI </li></ul><ul><li>3D rendering of 2D GUI </li></ul><ul><li>2D GUI </li></ul>
  32. 32. Method Step3 - Lesson learnt from Taxonomy <ul><li>Source of inspiration </li></ul><ul><li>Abstract characteristics </li></ul><ul><li>Design options </li></ul><ul><li>Gaps to fulfill </li></ul><ul><li>Concrete User Interface Model </li></ul>
  33. 33. Method Step3 - Tool support - Alice
  34. 34. Method Step3 - Tool support - Maya
  35. 35. Method Step3 - Tool support - VUIToolkit
  36. 36. Method Step4 – Final User Interface <ul><li>VRML </li></ul><ul><li>X3D </li></ul><ul><li>Java 3D </li></ul>
  37. 37. Contents <ul><li>Introduction </li></ul><ul><li>Related Work </li></ul><ul><li>Model-based development for 3D User Interfaces </li></ul><ul><li>Case Study </li></ul><ul><li>Conclusion </li></ul>
  38. 38. Case Study – Task and domain definition
  39. 39. Case Study – Concrete Definition
  40. 40. Case Study – Concrete Model B
  41. 41. Case Study – From Abstract to concrete Model x>2 RHS := LHS NAC NAC
  42. 42. Case Study – From Abstract to concrete Model A trigger Feedback Navigation “ Navigation” A trigger Feedback Control “ Send Questionnaire” A dropdown list, a group of option buttons Data type, domain characteristics Select attribute value + repetitive (selection values not known) “ Select Answer” An output text Attribute, data type, domain characteristics Output (value unknown) “ Show Questionnaire”
  43. 43. Mock-up of the Concrete User Interface
  44. 44. Case Study – From Abstract to concrete Model <ul><li>< Group > </li></ul><ul><li>< CubeFace id =&quot; C1 &quot;> </li></ul><ul><li>< SphereTrigger defaultContent =&quot; Start &quot; radious =&quot; 1.5 &quot; solid =&quot; True &quot; isVisible =&quot; true &quot; </li></ul><ul><li>isEnabled =&quot; true &quot;> </li></ul><ul><li>< Transform scale =&quot; 8.23 8.23 8.23 &quot; translation =&quot; 0.27 12.14 18.30 &quot;/> </li></ul><ul><li>< TouchSensor id =&quot; TS1 &quot; enabled =&quot; True &quot;/> </li></ul><ul><li>< Appearance name =&quot; ButtonAppe &quot; id =&quot; App1 &quot;> </li></ul><ul><li>< Material </li></ul><ul><li>diffuseColor =&quot; 0.8 0.8 0.0 &quot; specularColor =&quot; 0.11 0.11 0.11 &quot; </li></ul><ul><li>emissiveColor =&quot; 0.0 0.0 0.0 &quot; shininess =&quot; 0.3 &quot;/> </li></ul><ul><li></ Appearance > </li></ul><ul><li></ SphereTrigger > </li></ul><ul><li></ Group > </li></ul>
  45. 45. Case Study – Editing the FUI in Maya
  46. 46. Case Study – Editing the FUI in Alice
  47. 47. Case Study – FUI generated with Alice
  48. 48. Case Study – FUI generated with the VUIToolkit
  49. 49. Case Study – FUI generated with the Maya
  50. 50. Case Study – FUI generated with the Maya
  51. 51. Use of the virtual polling system Case Study in a virtual world
  52. 52. Contents <ul><li>Introduction </li></ul><ul><li>Related Work </li></ul><ul><li>Model-based development for 3D User Interfaces </li></ul><ul><li>Case Study </li></ul><ul><li>Conclusion </li></ul>
  53. 53. Conclusion –Model-based approaches Java, XHTML, Flash, HTML, Voice XML* Java 3D*, X3D*, VRML*. T  Do, T  AUI, AUI  CUI, CUI  AUI, T  CUI, (T, Do,AUI,CUI)  C, CUI ≈ -> FUI T, Do, C, AUI, CUI Our methodology X3D, Behavior3D, Audio3D (CUI, Di) -> FUI CUI, Di Contigra T ≈-> ( Di, CUI), ( Di, CUI) ≈ -> FUI Di, CUI InTML C++ ( T, Di, CUI )  FUI T, Di, CUI CoGenIVE VRML, X3D CUI -> FUI CUI VR-Wise FUI target languages *Indicates that it is done manually Inter Model Transformation  Bidirectional derivation -> Derivation link ≈-> Manual Derivation ≈  Manual Bidirectional der. FUI = Final User Interface Models t= task, Do = Domain Di = dialog AUI=abstract presentation CUI=concrete user interface U = user, C = context.
  54. 54. Conclusion <ul><li>The method based on graph transformation has been introduced, defined and illustrated </li></ul><ul><li>We proposed a method for 3D UI development whose life cycle was divided into four steps ranging from the most abstract to the most concretes according to the principles of Model-Driven Engineering. </li></ul><ul><li>The feasibility of the approach is much depending on the amount and the quality of the design rules that are encoded in UsiXML. </li></ul>
  55. 55. Conclusion <ul><li>Future work. Exploring more design options and encode them in UsiXML </li></ul><ul><ul><li>Models </li></ul></ul><ul><ul><li>Transformational rules </li></ul></ul><ul><ul><li>Software tools </li></ul></ul><ul><ul><li>More FUI </li></ul></ul><ul><li>Consolidate concepts </li></ul>
  56. 56. Thank you <ul><li>Juan Manuel Gonzalez Calleros </li></ul>