Toolkit for Distributed User Interfaces. Invited conference given at University of Elche (Elche, November 23rd, 2010)

1. Towards a toolkit for
Distributed User Interfaces (DUIs):
Think everything distributed
Jean Vanderdonckt
Louvain Interaction Lab
Université catholique de Louvain
Jean.Vanderdonckt@uclouvain.be

2. Invited conference, University of Elche (Elche, 23 November 2010)
What is the situation today?
One user performing one task on one domain
in one context of use
• The predominant paradigm: single…
• Monitor, device
• Computing platform
• User
• Environment
• Context of use
• Domain and task
2
one many

3. Invited conference, University of Elche (Elche, 23 November 2010)
Let us distribute Uis, but
one dimension at a time
3
• The predominant paradigm: single…
• Monitor, device
• Computing platform
• User
• Environment
• Context of use
• Domain and task
one many

4. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-monitor/device
• Why take care of multiple monitors?
– Device side
• Monitor surface is
increasing, while price
is decreasing
– User side
• Users prefer more display surface
• User’s productivity increases from 10% to 30%
(although some usability problems are raised)
[Harris,2002]
Projected LCD Pricing 2002-2005
$437
$378
$327 $283
$699
$597
$510
$436
$1,089
$905
$752
$625
$0
$200
$400
$600
$800
$1,000
$1,200
2002 2003 2004 2005
$US
15" -13.5%
17" -14.6%
18" -16.9%

5. Invited conference, University of Elche (Elche, 23 November 2010)
Mutiple PCs and Displays
0%
10%
20%
30%
40%
50%
60%
70%
80%
None Multiple monitors
attached to
multiple
computers.
Laptop and
desktop monitor
connected
together.
Dualmon or higher
Config
PercentRespondants
All
Multi-monitor/device
• Why take care of multiple monitors?
– User side
[Harris,2002]

6. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-monitor/device
• Why take care of multiple monitors?
Effects of Display Size on Task Times
0
20
40
60
80
100
120
140
160
DISPLAY
AverageTaskTime(Seconds)
Small
Large
[Czerwinsky,2005]

7. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-monitor/device
• Why take care of multiple monitors?
[Czerwinsky,2005]
The tasks were easy to perform
0
1
2
3
4
5
Small Large
Display Size
AverageRating(1=Disagree,
5=Agree)

8. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-monitor/device
• Why take care of multiple monitors?
[Czerwinsky,2005]
I was satisfied with the ease of windows
layout
0
1
2
3
4
5
Display Size
AverageRating
(1=Disagree,
5=Agree)
Small
Large

9. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-monitor/device
• Distribution of UIs across monitors (hardware)
– Coupling different monitors to the same
workstation
 Hard wired, manual
solution
 Opportunistic
 Lack of reusability

10. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-monitor/device
• Distribution of UIs across monitors (hardware)
– Coupling different monitors: control rooms
© 2010 www.symbio.pro

11. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-monitor/device
• Distribution of UIs across monitors (software)
– Partitioning of a window across several screens to
obtain a DUI

12. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-monitor/device
• Extending a Desktop with another Monitor
Ref_1 SCREEN_1 SCREEN_2
1 2
4 5
3
6
8 9
1 2
4 5
3
6
8 97
2
4
8
5
5
5 6
Without anti-
aliasing. No pixel
information is lost.
But proportions
are. Rendering
quality is quite
bad.
1 1-2 32-3 With anti-aliasing.
Pixel information
are lost (e.g. pixel
2). Proportions are
quite ok.
Rendering quality
is good
1-4 3-6
4-7 6-9
7 97-8 8-9
7
[Demeure et al.,2004]

13. Invited conference, University of Elche (Elche, 23 November 2010)
One dimension at a time
• The predominant paradigm: single…
– Monitor, device
– Computing platform
– User
– Environment
– Context of use
– Domain and task
13
one many

14. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform
1970 1980 1990 2000 2010 2020
10
100
1000
10000
(Amount of end users)
(Amount of interactive systems)
50
500
5000
50000
1 system for 100
users
1 system for 1
user
2 or 3 systems
for 1 user
10 to 50 systems
for 1 user
Source: Forrester Research, 2002

15. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform
• Problem: how to design a UI that takes care
of multiple computing platforms?
• Two examples:
– The Painter's palette [Grolaux et al., 2005]
– Migration [Molina et al., 2006]

16. Invited conference, University of Elche (Elche, 23 November 2010)
The Painter’s Palette
+ =

17. Invited conference, University of Elche (Elche, 23 November 2010)
Digital Painter’s Palette

18. Invited conference, University of Elche (Elche, 23 November 2010)
Useful area is wasted !

19. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform
• Problem: how to design a UI that takes care
of multiple computing platforms?
Pencil
Palette
Painting
Painting
tool
[Grolaux & Vanderdonckt,2005]

20. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform
[Grolaux & Vanderdonckt,2005]

21. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform

22. Invited conference, University of Elche (Elche, 23 November 2010)
Example using a Pocket PC

23. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform

24. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform
• DEMIPLAT Principle for migration: Detach

25. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform
• DEMIPLAT Principle for migration: Detach - Migrate

26. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform
• DEMIPLAT Principle for migration: Detach - Migrate - Plastify

27. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform
• DEMIPLAT Principle for migration:
Detach - Migrate - Plastify - Attach

28. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-platform
• This is not a floating bar
Process

29. Invited conference, University of Elche (Elche, 23 November 2010)
Computer B
Process
Multi-platform
Process
Computer A

30. Invited conference, University of Elche (Elche, 23 November 2010)
MigriXML Case study
• The user’s environment is a small office, with
five different platforms

31. Invited conference, University of Elche (Elche, 23 November 2010)
One dimension at a time
• The predominant paradigm: single…
– Monitor, device
– Computing platform
– User
– Environment
– Context of use
– Domain and task
31
one many

32. Invited conference, University of Elche (Elche, 23 November 2010)
Multiple users
• Design guideline #1: provide multiple ways to carry
out a task by different users
– Interactive systems should better support multiple users
in multi-tasking because group definition changes
Extended from [Mandviwalla & Olfman, 2000]

33. Invited conference, University of Elche (Elche, 23 November 2010)
Multiple users
• Design guideline #2: support multiple work
methods
– A same task could involve different sub-tasks depending
on the context
• Design guideline #3:support group evolution
– GR1 et GR2 changent en fonction du temps
[Mandviwalla & Olfman, 2000]

34. Invited conference, University of Elche (Elche, 23 November 2010)
One dimension at a time
• The predominant paradigm: single…
– Monitor, device
– Computing platform
– User
– Environment
– Context of use
– Domain and task
34
one many

35. Invited conference, University of Elche (Elche, 23 November 2010)
Multi-context
• The Sedan-Bouillon Tourist Application with
Plasticity
• The Slides Manager
Title
Content
N
A
V
I
G
A
T
I
[Demeure et al.,2004]

36. Invited conference, University of Elche (Elche, 23 November 2010)
One dimension at a time
• The predominant paradigm: single…
– Monitor, device
– Computing platform
– User
– Environment
– Context of use
– Domain and task
36
one many

37. Invited conference, University of Elche (Elche, 23 November 2010)
What we have for multi-tasking
• Ext/Tk toolkit

38. Invited conference, University of Elche (Elche, 23 November 2010)
DUI development
• Current shortcomings
– #1. No genuine distribution

39. Invited conference, University of Elche (Elche, 23 November 2010)
DUI development
• Current shortcomings
– #2. Distribution is predefined, opportunistic or
optimized for some task
Desktop ComputerTablet PC
Designer's Outpost
[Klemmer et al., 2002]

40. Invited conference, University of Elche (Elche, 23 November 2010)
DUI development
• Current
shortcomings
– #3.
Distribution
is at the
container
level (high
granularity)
[Czerwinsky et al., 2000]

41. Invited conference, University of Elche (Elche, 23 November 2010)
DUI development
• Current shortcomings
– #4. Distribution is not replicable
[Vandervelpen et al., 2000]

42. Invited conference, University of Elche (Elche, 23 November 2010)
DUI development
• Towards a Toolkit of distribution primitives
(more…)
– DISPLAY
– COPY
– MOVE
– REPLACE
– MERGE
– SWITCH
– DISTRIBUTE
– …

43. Invited conference, University of Elche (Elche, 23 November 2010)
DUI development
• Toolkit of distribution primitives: examples
– COPY button_1 TO shared_display
Meta-UI [Coutaz 2006] Source UI Shared_display

44. Invited conference, University of Elche (Elche, 23 November 2010)
DUI development
• Toolkit of distribution primitives: examples
– REMOVE outputText FROM "window_comp_0"
Source UI Target UI

45. Invited conference, University of Elche (Elche, 23 November 2010)
DUI development
• Toolkit of distribution primitives: examples
– MERGE "window_comp_0", "window_comp_1"
TO "window_comp_2"
Source UI Target UI

46. Invited conference, University of Elche (Elche, 23 November 2010)
Conclusion
• Toolkit for DUI distribution primitives
– Pros
• Distribution is regulated by primitives in a scenario
• Distribution is model-based, at any level of
granularity, is replicable, etc.
– Cons
• Still need to develop a model-based approach
• What is the best distribution graph?
– Distribution scene
– Concrete User Interface Model
• More distribution algorithms are required (optim.)

47. Invited conference, University of Elche (Elche, 23 November 2010)
For more information and downloading,
http://www.isys.ucl.ac.be/bchi
User Interface eXtensible Markup Language
http://www.usixml.org
FP7 Selfman project
http://www.ist-selfman.org/
Thank you very much!
FP7 Serenoa project
http://www.serenoa-fp7.eu