1. The Cost of Display Switching: A
Comparison of Mobile, Large Display
and Hybrid UI Configurations
Umer Rashid
SACHI Research Group,
University of St Andrews

2. Mobile Devices

3. Original Use of Mobile Phones

4. Current Use of Mobile Phones

5. Current Use of Mobile Phones

6. Mobile Phone Proliferation
• More than 5 billion mobile phones worldwide
– 3 x personal computers (BBC News, 09.07.2010)
• More Mobiles than Humans
– 10 billion mobile phones by 2016 (BBC News,
15.02.2012)

7. Limitations of Mobile Devices

8. Mobile Coupling with Large Displays

9. Research Questions
• Impact of distribution of UIs across displays
– Task Performance
– Subjective Workload
– Preference Rankings
– Overhead of attention switching

10. Visual Search Tasks
• Map
– Locating a place with a specific criterion
• Text
– Locating a verbatim decalaration among web-
page descriptors
• Photo
– Locating a face image in a collection

11. Data Size
• Small
– Fits on the mobile display
• Large
– Fits on the large display
– Requires navigation on the mobile device

12. UI Configurations
• Mobile-only
– Input & Ouptut on Mobile
• Large Display
– Input on Mobile
– Output on Large Display
• Hybrid
– Input & Output on Mobile
– Output on Large Display

13. Map UI

14. Text UI

15. Photo UI

16. Participants (N=26)
• Age (19-33)
• Normal or corrected-to-normal vision
• Recruited through email advertisement
• Session duration: 1 hour
• Compensation: £5

17. Map Search: Completion Times
*
20
Completion Time (s)
*
* *
15 Mobile
10
Hybrid
5
Large Display
0
Small Large

18. Map Search: Subjective Workload
Factor χ² (2) p Mobile Hybrid Large D.
Physical Demand 11.1 <0.01* 2.98 2.58 2.27
Mental Demand 5.01 >0.05 2.75 2.19 2.27
Temporal Demand 9.32 <0.01* 2.9 2.33 2.27
Performance 7.27 <0.05* 7.85 8.42 8.75
Effort 6.93 <0.05* 3.36 2.92 2.48
Frustration 8.44 <0.05* 3.17 2.38 2.07

19. Map Search: Preference Rankings
Best Worst
Mobile 2 7 17
Hybrid 12 11 3
Large Display 12 8 6

20. Cost of Gaze Shifts (GS)
30 30
Small Large
Completion Time (s)
25 25
20 20
15 15
10 10
5 5
0 0
0 2 4 6 0 2 4 6
Average # of Gaze Shifts (per trial) Average # of Gaze Shifts (per trial)
CT(seconds) = 1.8*#GS+ 8.21, CT(seconds) = 1.8*#GS+ 9.53,
R2 = 0.214 R2 = 0.385

21. Text Search: Completion Time
50
Completion Time (s) * *
40 Mobile
30
20
Hybrid
10
Large Display
0

22. Text Search: Subjective Workload
Large
Factor χ² (2) p Mobile Hybrid D.
Physical Demand 9.08 <0.05* 5.03 4.85 3.9
Mental Demand
Temporal 7.25 <0.05* 1.8 2.22 1.45
Demand 4.95 >0.05 3.85 3.37 3.18
Performance 4.82 >0.05 6.12 6.68 7.18
Effort 4.35 >0.05 4.5 3.9 3.25
Frustration 0.95 >0.05 2.93 2.38 2.92

23. Text Search: Preference Rankings
Best Worst
Mobile 8 10 4
Hybrid 2 6 14
Large Display 12 6 4

24. Photo Search: Completion Time
60
Completion Time (s)
* *
50
40
Mobile
30
20 Hybrid
10
0 Large Display
Small Large

25. Photo Search: Subjective Workload
Large
Factor χ² (2) p Mobile Hybrid D.
Physical Demand 4.48 >0.05 4.14 4.39 3.71
Mental Demand
Temporal 4.73 >0.05 2.75 3.25 2.66
Demand 4.45 >0.05 3.86 3.68 2.98
Performance 6.21 <0.05* 8.34 7.45 8.41
Effort 4.86 >0.05 4.34 4.43 3.66
Frustration 1.69 >0.05 3.07 3.18 2.64

26. Photo Search: Preference Rankings
Best Worst
Mobile 4 6 10
Hybrid 3 9 8
Large Display 13 5 2

27. Lessons for Practitioners
• Large Display improves task performance,
subjective workload and preference rankings,
for mobile tasks.
• Distributing visual output causes overhead for
visual search tasks.
• Tasks that require navigation of continuous
space (e.g., map search) benefit most from
large displays.

28. Future Directions
• Variations of Hybrid UIs
• Multi-modal Interaction
• Multiple displays for group collaboration

29. Role in SACHI Projects
• LADDI
– Large Augmented Digital Displays for Interactive
Experience of Historical Sites
• SMART
– Scotland’s Museums Augmented Reality Tourism
• THRIVE
– Trading Histories Revisited, Information
Visualization Environment

30. Inter-disciplinary Collaboration
• School of Pscyhology
– Perception (Research Group)
– Human Cognition (Research Group)
• School of History
– THRIVE project
• School of Physics & Astronomy
– Photonics Innovation Centre

31. More Information
• Umer Rashid,
– PhD Student,
– School of Computer Science,
– University of St Andrews
– ur9@st-andrews.ac.uk