Model-Based User Interfaces

1. Intelligent User
Interfaces
ICS2208
vanessa.camilleri@um.edu.mt
Dr Vanessa Camilleri
Department of AI,
University of Malta

2. Topic 2: Overview
• Why UI design?
• What is the Model-based Interface Development?
• UI models: task, abstract, concrete, final
• Benefits of MBID
• UMs & Intelligent Interfaces
2

3. Why adaptive UIs?
• What is UI design?
• The figure shows hierarchy of properties of
adaptive UIs
3
source: Akiki, Bandara & Yu (2015)

4. Model-based UI
• Models capture world knowledge
• Doesn’t rely on code but on higher level
specifications from which UI can be derived
• UI models based on CAMELEON reference
framework: task, abstract, concrete, final
4

5. Model-based Interface
Development
5
The Cameleon Reference Framework
Source: Meixner, Cavalry & Coutaz, 2014.
http://www.w3.org/TR/2014/NOTE-mbui-intro-20140107/

6. Model-based Interface
Development
6

7. Benefits of MBID
• Benefits from using a step-wise development
lifecycle:
• reducing gap between requirements &
implementation;
• coordinating the involvement of multiple
stakeholders;
• Producing well structured systems;
7

8. Benefits of MBID
• Benefits from using explicit abstract models:
• planning an adequate level of abstraction;
• improving communication by explicit models;
• supporting UI quality;
8

9. Benefits of MBID
• Benefits from exploring alternative designs:
• supporting creation and creativity;
• enabling the production and comparison of
alternative designs for multiple contexts of use
while preserving quality (e.g., consistency);
9

10. Benefits of MBID
• Benefits resulting from code generation:
• enhancing development productivity;
• capturing and reusing expert knowledge
throughout UI development life cycle;
• reducing errors;
10

11. Benefits of MBID
• Benefits from using models at runtime:
• defining and enacting method for UI
development process;
• knowledge about creation of modeling
languages;
• usage of frameworks and tools;
11

12. Benefits of MBID
• Maintenance of modelling language and
transformations:
• systematic and explicit definition of meta-models
and transformations;
• maintenance of modelling languages and code
generators;
• R e u s e o f m o d e l s , m e t a - m o d e l s , a n d
transformations;
12

13. Use Cases
• UC1: Car rental; The car rental example consists of a
scenario in which the interactive system permits users
to rent a car. In this sense, various contextual
information can be used to adapt application aspects,
and to properly display the list of cars to rent, enabling
users to make choices and to accomplish the main task
• context 1: at home using desktop pc
• context 2: smart phone (noisy environment/user
walking)

14. Car rental
Screenshot of Context 1
Screenshot of Context 2
Models involved Car rental process

15. Use Cases
• UC2: Digital Home; Digital home refers to a
residence with devices that are connected through
a computer network. A digital home has a network
of consumer electronics, mobile, and PC devices
that cooperate transparently.
• context 1: at home using desktop pc
• context 2: smart phone (noisy environment/user
walking)

16. Digital Home
Screenshot of Context 1
Screenshot of Context 2
The task model for the digital home

17. Use Cases
• UC3: Omitting minimalistic UIs through the use of an
universal interaction device in production environments;
The SmartFactoryKL (see Figure UC3.1) is an arbitrarily
modifiable and expandable (flexible) intelligent production
environment, connecting components from multiple
manufacturers (networked), enabling its components to
perform context-related tasks autonomously (self-organising),
and emphasising user-friendliness (user-oriented).
• Context 1: Physical environment is that of a production
environment (industrial factory), platform is a tablet PC (+
modules or field devices from different vendors), one
single user

18. SmartFactory KL
Screenshot of Final UI Models in SmartFactory KL

19. Use Cases
• UC4: Story Editor; The production of e-learning tools for deaf
people meets with several difficulties, connected to the need
to resort only to the visual channel, without saturating it, and to
the adoption of different cognitive strategies developed within
the community - the use of a user interface abstract model in
the development of the interactive story editor, to be used by
tutors and teachers to organise the course material and path,
and which generates interactive pages for the students.
• Context 1: The generation of a course by a teacher,
possibly involving in the process tutors who will then have
to assist the learners. Learners and tutors will then interact
with the generated web pages

20. Story Editor
Workflow and generated page for the story
Models in Story Editor

21. Use Cases
• UC5: Augmented Reality Widgets; Post-WIMP interactors
running inside a web application: A mixed reality furniture
online shop that can be controlled by gestures and
supports inter-reality migration of interactors. In the web
furniture shop, a customer can choose between different
furniture and fill up a shopping cart. By a drag-and-drop
gesture using both hands one can drag furniture interactors
virtually out of the monitor displaying the web shop and
drop them into an augmented reality to see if the furniture
really matches in space and color to the user’s
environment. An augmented reality frame that surrounds
the shopping cart allows to seamlessly switch between
realities while crossing it during the drag and drop gesture.

22. AR Furniture shop
Shopping cart Web application
Two handed
gestures to drag
and drop furniture

23. Use Cases
• UC6: Interactive music sheet using head gestures;
When learning to play a musical instrument, or when
playing one, a music sheet is used to give guidance
as to how to perform the musical piece. However, as
songs become longer and more intricate they may
span across several sheets, forcing the player to stop
playing to turn the page. Although this may become
easier as one becomes more experienced with the
instrument, it is a barrier for inexperienced players
that can be tackled easily using a different mode to
turn the pages other than your hand.

24. Interactive music
Models used for the Widgets Use Cases
Head tracking interactor

25. Generic Requirements for
MBID
1. Completeness
2. Consistency
3. Separation of concerns
4. Extensibility
5. Concision
6. Correlability

26. Model-based UI tools
• ConcurTaskTrees Environment (CTTE),http://giove.isti.cnr.it/tools/
CTTE/home
• Task/Design time/Java application
• Model-based lAnguage foR Interactive Applications Environment
(MARIAE), http://giove.isti.cnr.it/tools/MARIAE/home
• Task, AUI, CUI, FUI/Design time/Java application
• USICOMP: an extensible model-driven composer
• http://www.usixml.org/en/garcia-frey-a-ce-ret-e-dupuy-chessa-s-
calvary-g-gabillon-y-usicomp-an-extensible-model-driven-
composer.html?IDC=465&IDD=2464

27. Model-based Tools:
Research Gaps
• End users
• UI designers
• Technical

28. UMs & Intelligent Interfaces
• Complexities of intelligent interfaces include:
• knowledge representation
• architectures for knowledge-based systems
• human factors

29. UMs & Intelligent Interfaces
• There are three key benefits that a model of an
intelligent interface offers:
1. it establishes the knowledge requirements of
the interface,
2. it prescribes the functionality provided by the
system, and
3. it defines the concept of an intelligent interface