How to Survive Multi-Device User Interface Design with UIML

2,243 views

Published on

Talk given at the DSP Valley seminar "Exploring GUI design for embedded systems" together with Jan Meskens on June 9, 2009.

See also: http://www.dspvalley.com/upload/event/site/5270/index.html

Abstract:

The User Interface Markup Language has been around for 10 years now and it is still evolving. The language did not reach a large audience yet and there are few publicly known cases where the language is used. Nevertheless, UIML is probably one of the most generic and flexible user interface markup languages available, mainly because it is in fact a meta-language that is not limited to a particular domain, device, platform or widget set. The design of the language makes it an ideal candidate to support future evolutions in user interface techniques and guarantees the survival of user interface designs over time.

At the dawn of the 4.0 version of the UIML specification—which will be released soon—this talk provides an overview of the language, the novelties of the 4.0 specification and some tools that we have built and are available today.

We also provide an overview of our work to integrate UIML in a multi-device user interface design process that allows designers to create user interfaces for multiple devices without having to deal with an abstract user interface specification.

Finally, we will highlight our experiences with designing tailored user interfaces for embedded platforms with UIML, and discuss the pros and cons of using a high-level user interface description language on devices with limited resources.

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,243
On SlideShare
0
From Embeds
0
Number of Embeds
28
Actions
Shares
0
Downloads
55
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

How to Survive Multi-Device User Interface Design with UIML

  1. 1. How to Survive Multi-Device User Interface Development with UIML Jo Vermeulen and Jan Meskens {jo.vermeulen,jan.meskens}@uhasselt.be Hasselt University – tUL – IBBT Expertise Centre for Digital Media Belgium
  2. 2. A little bit about us …
  3. 3. A little bit about us … About 80 researchers … Specializing in …
  4. 4. A little bit about us … About 80 researchers … Specializing in …
  5. 5. The UIML Crew Kris Luyten Jo Vermeulen Jan Meskens Professor PhD student PhD student Since 2002 Since 2004 Since 2006 • PhD about XML UIDLs • extended Uiml.net • extended Uiml.net • developed Uiml.net • MSc thesis • developed Gummy • several research projects • 2 applied research proj. • BSc & MSc thesis • > 25 related publications • 6 related publications • basic research project • numerous students • 6 students • 5 related publications • 1 student
  6. 6. What is UIML?
  7. 7. A description of a user interface
  8. 8. UIDLs in practice
  9. 9. UIML is different UIML is different
  10. 10. UIML is different • High level • Generic – No specific tags (e.g. <button>) • Meta: build your own User Interface Language – cfr. XML
  11. 11. What does UIML look like?
  12. 12. What does UIML look like? <uiml> <interface> <structure> <part>...</part> </structure> <style> <property>...</property> </style> <behavior> <rule>...</rule> </behavior> </interface> <peers>...</peers> </uiml>
  13. 13. What does UIML look like?
  14. 14. A bit of history
  15. 15. UIML: a pioneering UIDL 1997: UIML is born. Professor Marc Abrams (Virginia Tech, USA) • 20 years experience with building GUIs • Frustrated with then-current methods • UIML 1.0 spec and Java renderer • Founded Harmonia to provide tools & services for UIML
  16. 16. UIML: a pioneering UIDL 1998-2000: updated specification & tool support. • 1998 – PhD dissertation by Constantinos Phanouriou. – Corresponding UIML 2.0 spec • 1999: – Windows CE renderer – Demo at WWW6 conference / W3C DI workshop • 2000: – HTML, WML, VoiceXML renderers
  17. 17. UIML: a pioneering UIDL 2001-2004: UIML comes of age • 2001 – UIML 3.0 spec – LiquidUI tool (Harmonia) – UIML Europe 2001 conference (Paris, France) • 2002: – UIML Technical Committee starts • 2003: – Kris Luyten creates open source Uiml.net renderer • 2004: – UIML 3.1 specification
  18. 18. The company behind UIML Harmonia, Inc. • contracts with US DoD • e.g. UIs for missile control
  19. 19. UIML as abstraction
  20. 20. UIML as abstraction UIML UI toolkits Logic OS Hardware
  21. 21. Same device, different toolkit UIML UIML Toolkit 1 … Toolkit 2 … … … … … Device A Device A
  22. 22. Different device/OS/toolkit UIML UIML … … … … … … … … Device A Device B
  23. 23. Multi-device/toolkit examples
  24. 24. Extra component necessary UIML UIML renderer UI toolkits Logic OS Hardware
  25. 25. UIML Renderer • Runtime environment • Allows for: – rapid prototyping – dynamic changes at runtime
  26. 26. Our UIML Renderers • .NET – Works on most .NET-enabled platforms: • Mono (*NIX) • Microsoft .NET • Compact .NET • Java – Runs on various Java VMs and APIs, e.g. • Java ME • IBM J9 • MHP Xlet
  27. 27. UIML research/teaching
  28. 28. Dynamic mobile guide UIs Context-aware UIs Adapt according to the context-of-use (e.g. location, user profile) Dynamic changes at runtime
  29. 29. OSGi + UIML UIML Activator Bundle controls rendering core and connects with other OSGI components
  30. 30. Distributed Applications • MSc thesis Neal Robben • Distributed user interfaces with Uiml.net • Synchronization between different UI instances
  31. 31. UIML and distributed Applications
  32. 32. UIML and X + V • MSc thesis Rob Van Roey • X+V backend for UIML • multimodal user interfaces – GUI interaction (XHTML) – speech (VoiceXML)
  33. 33. UIML and X + V
  34. 34. UIML on CarMelody platform • Thesis Pieter Gevers (industrial engineering) • Prototyping UI with Uiml.net that invokes hardware APIs of Philips car radio • Philips PNX0106 running Montavista Linux
  35. 35. UIML on CarMelody platform
  36. 36. Disadvantages of UIML • Lacks mature/available tools • Lacks community • Lacks documentation • Lacks proof that it “just works” in industrial/commercial settings
  37. 37. What about performance?
  38. 38. Smart optimization • Rendering certainly slower • But on the other hand, … • The UI description provides more data, allowing for smarter optimizations
  39. 39. Case study • iDTV application, several linked UIML “pages” • Painfully slow (7 seconds to render a page) • Serious performance boost after optimization using pre-caching and MPEG-2 frames
  40. 40. Performance comparison 8 7 7 6 5 4 Unoptimized 3 Pre-caching 2 1 0,35 0 Rendering time per page (seconds)
  41. 41. Intermediate UIML
  42. 42. Motivation • UIML language is platform independent • Renderers have to be created – For different computing platforms – Different programming languages
  43. 43. UIML for Java UIML Swing AWT UIML Java Backend renderer Java Swing Java AWT Logic • Containment • UI type conversion OS Hardware
  44. 44. UIML for .NET UIML GTK# SWF UIML .NET Backend renderer GTK# SWF Logic OS Hardware
  45. 45. High maintenance and development cost UIML UIML Toolkit UIML .NET UIML Java Toolkit Backend renderer renderer Backend Toolkit Logic Logic Toolkit OS OS Hardware Hardware
  46. 46. Flexible Intermediate rendering UIML UIML Toolkit UIML .NET Toolkit UIML Java iRenderer iRenderer Backend renderer Backend renderer Toolkit Logic Toolkit Logic OS OS Hardware Hardware
  47. 47. Intermediate Renderer UIML iUIML Concrete Widget Structure Properties Callbacks Style Concrete Widget Concrete Widget Intermediate Properties Properties Behavior Renderer Callbacks Callbacks Concrete Widget Concrete Widget Properties Properties Vocabulary Callbacks Callbacks
  48. 48. Example UIML <d-class id=« frame » maps-to=« System.Windows.Forms.GroupBox »> … </d-class>
  49. 49. Corresponding iUIML
  50. 50. Intermediate UIML • JSON language (http://json.org) • All UIML abstractions are resolved – Concrete widget names – Concrete property names – Concrete event names • iUIML is easy to translate into a running UI • We built an Adobe Flex renderer in 3 days…
  51. 51. Adobe Flex Renderer
  52. 52. Design Tools
  53. 53. Motivation • Learning UIML takes time – Especially for non-technical designers • Polishing a UI in UIML is difficult – Requires switching between edit and run mode
  54. 54. Motivation • Creating one UI for a range of devices is challenging • Automatic transformations often not aesthetically pleasing • Manual adjustment is needed
  55. 55. Design Tool: Gummy
  56. 56. Gummy • Platform independent GUI builder • Features – Design GUIs graphically – Export GUIs as UIML – Transform GUIs from one platform to another (based on UIML) – Load a platform-specific design workspace dynamically
  57. 57. Dynamic Design Workspace Loading
  58. 58. Gummy Architecture Desktop renderer <uiml> Desktop Proxy </uiml> PDA proxy PDA renderer W2P DVB-MHP proxy Desktop DVB-MHP renderer
  59. 59. Problem in Mobile UI Design Target Device Designer Device
  60. 60. Spacing and Positioning • Foto
  61. 61. Touch Screen Characteristics Touch Screen Sensitivity? Occluded Areas?
  62. 62. Interaction Techniques and Emulators
  63. 63. Our Solution: Gummy-Live “Mirror design changes to the mobile target device in real time”
  64. 64. Gummy-Live
  65. 65. Conclusion Renderer Tool Support Designer Support

×