The document discusses tool support for designing, managing, and optimizing multi-device user interfaces. It introduces the Gummy design environment, which allows creating high-fidelity prototypes across devices. It also discusses techniques for managing multiple prototypes, such as automatic transformations and linked editing. Finally, it covers live UI design techniques like the Design Mirror for iteratively optimizing designs by intertwining testing and design.

1. Expertise centre for Digital Media (EDM) - UHasselt 26/04/2011 Tool Support for Designing, Managing and Optimizing Multi-Device User Interfaces Jan Meskens jan.meskens@uhasselt.be Advisor: prof. dr. Karin Coninx Co-Advisor: prof. dr. Kris Luyten

2. 2 People use Multiple Devices Tablet PCs Mobile Phones Interactive TV

3. 3 Existing Multi-Device UI Design approaches Models Manually Model Based

5. Duplicated design effort

6. Models are often complex

7. Automatic generation can lead to unexpected resultsModels Myers, 2000 Grigoreanu, 2009 Lin, 2005

8. Designers need Multi-Device Design Tools to... 1 2 3 Createrealistic high fidelity prototypes Managemultiple device-specific prototypes IterativelyOptimizedesigns for the targeted devices 5

9. 1 2 3 Createrealistic high fidelity prototypes Managemultiple device-specific prototypes IterativelyOptimizedesigns for the targeted devices 6

11. What You See Is What You Get (WYSIWYG)

13. 9 Gummy design environment Device-specific workspaces Presented at AVI’08 and AVI’10

14. 10 Gummy design environment Toolbox Presented at AVI’08 and AVI’10

15. 11 Gummy design environment Canvas Presented at AVI’08 and AVI’10

16. 12 Gummy design environment Properties panel Presented at AVI’08 and AVI’10

17. 13 Gummycan target many platforms Mobile Web Desktop Presented at AVI’08 and AVI’10

19. Hiding the underlying UI models

20. Providing visual design feedback (WYSIWYG)14 Gummy architecture: requirements Presented at AVI’08 and AVI’10

21. 15 Gummy architecture: MVC Model Controller Screenshot Screenshot Platform Plug-In Design Change Render View XMPP Pierce et al., 2008 Presented at AVI’08 and AVI’10

22. 16 Gummy’s underlying Model A B C Root Panel Properties ... Panel Layout Information Rendering Information Button Rounded Range Presented at AVI’08 and AVI’10

23. 17 Example UI: Video Explorer Desktop (.NET) Mobile (Android) Web (Flex) Keyframes Video Player

25. Hard to keep prototypes consistentDesktop (.NET) Mobile (Android)

26. 1 2 3 Createrealistic high fidelity prototypes Managemultiple device-specific prototypes IterativelyOptimizedesigns for the targeted devices 19

28. Keep prototypes consistentDesktop (.NET) Grigoreanu, 2009 Lin, 2005 Mobile (Android)

29. 21 Managing Techniques Automatic Transformations + Linked Editing Design tool Macros(D-Macs)

30. 22 Managing Techniques Automatic Transformations + Linked Editing Design tool Macros(D-Macs)

31. Automatic Transformations + Linked Editing Copy Paste Edit Contents Presented at AVI’10 23

32. 24

33. 25

35. Designers cannot create their own transformations

36. Difficult to integrate new componentsSemantic UI Network Vermeulen et al., 2007 Demeure et al., 2006

37. 27 Managing Techniques Automatic Transformations + Linked Editing Design tool Macros(D-Macs)

38. 28 D-Macs approach Design Record Replay Presented at UIST’10

39. 29 D-Macs design action recorder Presented at UIST’10

40. 30 Example UI: News Video Explorer Desktop (.NET) Mobile (Android) Presented at UIST’10

41. 31

42. 32 Replay design actions using a mixed initiative approach Manual action Computer action Leshed et al, 2008 Li et al, 2010 Presented at UIST’10

43. 33

44. 34 Replay can fail Success: Add item to collection Add item to collection Switch to android Switch to android Dependency Select android.widget.gallery Select android.widget.button Failure: Presented at UIST’10

45. 35

46. 36 D-Macs action sharing D-macs repository Add recording Reuse recording http://dmacs Designer A Designer B Presented at UIST’10

47. 37 Online Action Repository Query Repository View Recording

48. 1 2 3 Createrealistic high fidelity prototypes Managemultiple device-specific prototypes IterativelyOptimizedesigns for the targeted devices 38

49. Iteratively Optimizing User Interfaces 39 Design Test Rethink

51. Testing is often postponed to the end of the design process

52. Design changes are more expensiveDesign Test Rethink

53. 41 Live UI Design: Intertwine test and design Hartmann et al., 2006 Bila et al., 2007 Pagetailor d.tools

54. 42 Multi-device Live UI Design Techniques Design Mirror Continuous Design Pointer Design Toolglass

55. 43 Multi-device Live UI Design Techniques Design Mirror Continuous Design Pointer Design Toolglass

56. 44 Design Mirror Device A B C Real Time Visualization Device A Design Environment Presented at VL/HCC’09

57. 45

59. Observations:

60. Changing widget sizes,

61. Replacing widgets,

62. Fine-tuning colors

64. 48 Continuous Design Pointer Device A B C Device A Design Environment Presented at EICS’09

65. 49

66. 50 Multi-device Live UI Design Techniques Design Mirror Continuous Design Pointer Design Toolglass

67. 51 Design Toolglass Device A B C Transparent Canvas Design Environment Emulator of Device A Presented at AVI’10

68. 52 Design Toolglass Windows Mobile Emulator Canvas Canvas Android Emulator Windows Mobile Android Presented at AVI’10

69. 1 2 3 Createrealistic high fidelity prototypes Managemultiple device-specific prototypes IterativelyOptimizedesigns for the targeted devices 53

70. 54 Conclusions

72. Models are used in the background

74. Keep prototypes consistent

75. Automation techniques:

76. Automatic Transformations + Linked Editing

78. Stimulate design iterations

79. Design techniques:

80. Design Mirror

81. Continuous Design Pointer

83. Questions? 1 2 3 Createrealistic high fidelity prototypes Managemultiple device-specific prototypes IterativelyOptimizedesigns for the targeted devices 59