How to Survive Multi-Device User Interface Design with UIML
Upcoming SlideShare
Loading in...5

How to Survive Multi-Device User Interface Design with UIML



Talk given at the DSP Valley seminar "Exploring GUI design for embedded systems" together with Jan Meskens on June 9, 2009. ...

Talk given at the DSP Valley seminar "Exploring GUI design for embedded systems" together with Jan Meskens on June 9, 2009.

See also:


The User Interface Markup Language has been around for 10 years now and it is still evolving. The language did not reach a large audience yet and there are few publicly known cases where the language is used. Nevertheless, UIML is probably one of the most generic and flexible user interface markup languages available, mainly because it is in fact a meta-language that is not limited to a particular domain, device, platform or widget set. The design of the language makes it an ideal candidate to support future evolutions in user interface techniques and guarantees the survival of user interface designs over time.

At the dawn of the 4.0 version of the UIML specification—which will be released soon—this talk provides an overview of the language, the novelties of the 4.0 specification and some tools that we have built and are available today.

We also provide an overview of our work to integrate UIML in a multi-device user interface design process that allows designers to create user interfaces for multiple devices without having to deal with an abstract user interface specification.

Finally, we will highlight our experiences with designing tailored user interfaces for embedded platforms with UIML, and discuss the pros and cons of using a high-level user interface description language on devices with limited resources.



Total Views
Views on SlideShare
Embed Views



2 Embeds 23 21 2



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

How to Survive Multi-Device User Interface Design with UIML How to Survive Multi-Device User Interface Design with UIML Presentation Transcript

  • How to Survive Multi-Device User Interface Development with UIML Jo Vermeulen and Jan Meskens {jo.vermeulen,jan.meskens} Hasselt University – tUL – IBBT Expertise Centre for Digital Media Belgium
  • A little bit about us …
  • A little bit about us … About 80 researchers … Specializing in …
  • A little bit about us … About 80 researchers … Specializing in …
  • The UIML Crew Kris Luyten Jo Vermeulen Jan Meskens Professor PhD student PhD student Since 2002 Since 2004 Since 2006 • PhD about XML UIDLs • extended • extended • developed • MSc thesis • developed Gummy • several research projects • 2 applied research proj. • BSc & MSc thesis • > 25 related publications • 6 related publications • basic research project • numerous students • 6 students • 5 related publications • 1 student
  • What is UIML?
  • A description of a user interface
  • UIDLs in practice
  • UIML is different UIML is different
  • UIML is different • High level • Generic – No specific tags (e.g. <button>) • Meta: build your own User Interface Language – cfr. XML
  • What does UIML look like?
  • What does UIML look like? <uiml> <interface> <structure> <part>...</part> </structure> <style> <property>...</property> </style> <behavior> <rule>...</rule> </behavior> </interface> <peers>...</peers> </uiml>
  • What does UIML look like?
  • A bit of history
  • UIML: a pioneering UIDL 1997: UIML is born. Professor Marc Abrams (Virginia Tech, USA) • 20 years experience with building GUIs • Frustrated with then-current methods • UIML 1.0 spec and Java renderer • Founded Harmonia to provide tools & services for UIML
  • UIML: a pioneering UIDL 1998-2000: updated specification & tool support. • 1998 – PhD dissertation by Constantinos Phanouriou. – Corresponding UIML 2.0 spec • 1999: – Windows CE renderer – Demo at WWW6 conference / W3C DI workshop • 2000: – HTML, WML, VoiceXML renderers
  • UIML: a pioneering UIDL 2001-2004: UIML comes of age • 2001 – UIML 3.0 spec – LiquidUI tool (Harmonia) – UIML Europe 2001 conference (Paris, France) • 2002: – UIML Technical Committee starts • 2003: – Kris Luyten creates open source renderer • 2004: – UIML 3.1 specification
  • The company behind UIML Harmonia, Inc. • contracts with US DoD • e.g. UIs for missile control
  • UIML as abstraction
  • UIML as abstraction UIML UI toolkits Logic OS Hardware
  • Same device, different toolkit UIML UIML Toolkit 1 … Toolkit 2 … … … … … Device A Device A
  • Different device/OS/toolkit UIML UIML … … … … … … … … Device A Device B
  • Multi-device/toolkit examples
  • Extra component necessary UIML UIML renderer UI toolkits Logic OS Hardware
  • UIML Renderer • Runtime environment • Allows for: – rapid prototyping – dynamic changes at runtime
  • Our UIML Renderers • .NET – Works on most .NET-enabled platforms: • Mono (*NIX) • Microsoft .NET • Compact .NET • Java – Runs on various Java VMs and APIs, e.g. • Java ME • IBM J9 • MHP Xlet
  • UIML research/teaching
  • Dynamic mobile guide UIs Context-aware UIs Adapt according to the context-of-use (e.g. location, user profile) Dynamic changes at runtime
  • OSGi + UIML UIML Activator Bundle controls rendering core and connects with other OSGI components
  • Distributed Applications • MSc thesis Neal Robben • Distributed user interfaces with • Synchronization between different UI instances
  • UIML and distributed Applications
  • UIML and X + V • MSc thesis Rob Van Roey • X+V backend for UIML • multimodal user interfaces – GUI interaction (XHTML) – speech (VoiceXML)
  • UIML and X + V
  • UIML on CarMelody platform • Thesis Pieter Gevers (industrial engineering) • Prototyping UI with that invokes hardware APIs of Philips car radio • Philips PNX0106 running Montavista Linux
  • UIML on CarMelody platform
  • Disadvantages of UIML • Lacks mature/available tools • Lacks community • Lacks documentation • Lacks proof that it “just works” in industrial/commercial settings
  • What about performance?
  • Smart optimization • Rendering certainly slower • But on the other hand, … • The UI description provides more data, allowing for smarter optimizations
  • Case study • iDTV application, several linked UIML “pages” • Painfully slow (7 seconds to render a page) • Serious performance boost after optimization using pre-caching and MPEG-2 frames
  • Performance comparison 8 7 7 6 5 4 Unoptimized 3 Pre-caching 2 1 0,35 0 Rendering time per page (seconds)
  • Intermediate UIML
  • Motivation • UIML language is platform independent • Renderers have to be created – For different computing platforms – Different programming languages
  • UIML for Java UIML Swing AWT UIML Java Backend renderer Java Swing Java AWT Logic • Containment • UI type conversion OS Hardware
  • UIML for .NET UIML GTK# SWF UIML .NET Backend renderer GTK# SWF Logic OS Hardware
  • High maintenance and development cost UIML UIML Toolkit UIML .NET UIML Java Toolkit Backend renderer renderer Backend Toolkit Logic Logic Toolkit OS OS Hardware Hardware
  • Flexible Intermediate rendering UIML UIML Toolkit UIML .NET Toolkit UIML Java iRenderer iRenderer Backend renderer Backend renderer Toolkit Logic Toolkit Logic OS OS Hardware Hardware
  • Intermediate Renderer UIML iUIML Concrete Widget Structure Properties Callbacks Style Concrete Widget Concrete Widget Intermediate Properties Properties Behavior Renderer Callbacks Callbacks Concrete Widget Concrete Widget Properties Properties Vocabulary Callbacks Callbacks
  • Example UIML <d-class id=« frame » maps-to=« System.Windows.Forms.GroupBox »> … </d-class>
  • Corresponding iUIML
  • Intermediate UIML • JSON language ( • All UIML abstractions are resolved – Concrete widget names – Concrete property names – Concrete event names • iUIML is easy to translate into a running UI • We built an Adobe Flex renderer in 3 days…
  • Adobe Flex Renderer
  • Design Tools
  • Motivation • Learning UIML takes time – Especially for non-technical designers • Polishing a UI in UIML is difficult – Requires switching between edit and run mode
  • Motivation • Creating one UI for a range of devices is challenging • Automatic transformations often not aesthetically pleasing • Manual adjustment is needed
  • Design Tool: Gummy
  • Gummy • Platform independent GUI builder • Features – Design GUIs graphically – Export GUIs as UIML – Transform GUIs from one platform to another (based on UIML) – Load a platform-specific design workspace dynamically
  • Dynamic Design Workspace Loading
  • Gummy Architecture Desktop renderer <uiml> Desktop Proxy </uiml> PDA proxy PDA renderer W2P DVB-MHP proxy Desktop DVB-MHP renderer
  • Problem in Mobile UI Design Target Device Designer Device
  • Spacing and Positioning • Foto
  • Touch Screen Characteristics Touch Screen Sensitivity? Occluded Areas?
  • Interaction Techniques and Emulators
  • Our Solution: Gummy-Live “Mirror design changes to the mobile target device in real time”
  • Gummy-Live
  • Conclusion Renderer Tool Support Designer Support