FrontEnd2014 Zurich The world consists of processes, not things. Time does not stand still for anyone. But user interfaces tend to present things as they are, in a frozen moment in time. In this talk we will look at different ways to incorporate time, process and a narrative structure into front-ends.

1. Time to Think about Time
Leander Conradie
Frontend 2014
Zurich, 28/08.2014
1

2. What is UX
Computers, the industrial revolution, the development of
tools in general serve one purpose:
To help people perform tasks more efficiently
!
2

3. What is UX
This talk is about moving towards that goal
Examine how the history of a users actions and history of
the objects she manipulates can help use make for a better
user experience
!
3

4. Object vs Process
4

5. The World Around Us
State of a system
Position of each part
But.. this is not enough
No concept of time
5

6. The World Around Us
Momentum
Position
6

7. The World Around Us
Momentum
Position and velocity
7

8. The World according to Newton
Objects stay in motion unless a
force is applied to them
8

9. The World according to Einstein
The speed of light, not
position, is the reference point
of space-time
9

10. The World around Us
Movement is the base case
Everything changes all the time
10

11. The World Around Us
Movement is the base case
Movement is a change of state
over time
11

12. Process
Every object that we deal with is in the state it is because
of a process that preceded the current time.
!
!
12

13. Take home point #1
The universe consists of
processes, not things
13

14. What has this got to do with UX?
14

15. Software Artefacts
UI interfaces a user with software artefacts
Digital representations of real world objects
As the real world behaves, the software should behave
15

16. Software Artefacts
But the change is often missing
State of a system is usually presented as it is now
What you see is a frozen moment in time
16

17. Software Artefacts
The external objects that digital artefacts represent have a
narrative, a story, a history behind them
This is reflected in the software artefact
Software artefacts are in a state because of a process that
preceded the current time
17

18. Software Artefacts
These processes are often neglected
Allowing the user to see and perhaps manipulate the
history can improve the user experience
18

19. The User
Users perform actions over time to perform a task
Knowledge of the past can help maintain flow and guide
the user in this task
This history is often neglected
19

20. Take home point #2
Users actions and software
objects are the result of process
20

21. User Tasks and Process History
21
Allow users to examine and
manipulate the history of the
objects they deal with
Use the history of user actions to
improve the flow of the task at
hand

22. Why is Object History Interesting?
22

23. User Tasks and Process History
23
Allow users to examine and
manipulate the history of the
objects they deal with
Use the history of user actions to
improve the flow of the task at
hand

24. “Nostalgia - it's delicate, but
potent. Teddy told me that in
Greek, nostalgia literally means
the pain from an old wound. It's
a twinge in your heart far more
powerful than memory alone.”
!
Don Draper, Mad Men, ‘The Wheel’
24

25. Nostalgia leads to old junk
Not too useful in a business app
But people can sometimes trigger their
memories by looking at events in the
past
25

26. UI Design
UX is about representing a digital model to a user
We must take care to make sure we work the the user’s
model, not the one of the computer
26

27. User Perception of Object Model
Might be thinking about event order
Probably thinking about causality
27

28. User Perception of Object Model
What we really want is to see causality
We can seldomly represent causality directly
We can only represent events
But the events give us a guide to the causes
28

29. Why vs What
Looking into the past helps us answer the question of why
an object is in the state it is.
!
!
29

30. Take home point #3
This is not about construction steps
This is about why things became way
they are
30

31. Why is User Action History Interesting?
31

32. User Tasks and Process History
32
Allow users to examine and
manipulate the history of the
objects they deal with
Use the history of user actions to
improve the flow of the task at
hand

33. Flow
Users perform a sequence of tasks, in time, towards a goal
Flow is a desirable property of UX
Flow consists of smoothly going from one task to the other
Lack of flow is like a sequence of stumbling blocks
33

34. Flow
“Users shouldn’t feel like they have to follow—or worse,
memorize—a strict set of instructions to achieve the goal.
Instead, what you’re striving for is a feeling of naturalness,
an inevitability, a flow.”
Dan Saffer, Microinteractions
34

35. Flow
We can use history to improve flow
35

36. Take home point #4
Flow is good
Flow is about keeping momentum
36

37. The Object Model
37

38. Remember Everything
Sequence of events that constructed the database as it is
Allow the user to look at and reconstruct the past
38

39. Manage History
Let users view history
Let users commit specific versions in time
Let users create alternate histories
39

40. Take home point #5
The database state is just a cached
version of the event log
40

41. Representation
41

42. Representation of Object History
Time adds another dimension
Screens are only 2D (Still)
42

43. Representation of Object History
We can display the events in the past
We can display the objects as they were
43

44. Past Events: Log File
User control? Rarely
Sacrifice a small animal to the
information priest in Nerdvana
44

45. Past Events: Event Log In UI
DIsplay Events in the UI
Better than a log file
45

46. Past Events: Time Axis as Vertical Axis
Facebook Timeline
46

47. Objects as they were: Time Axis as Control
Entire Object Onscreen
Time Machine
47

48. Objects as they were: Time Axis as Control
Entire Object Onscreen
Xcode Git
48

49. Animation
Display Time Sequence as
Animation
Look at Gource for video of Git
49

50. User Interaction
50

51. Temporal Coherence
Users tend to perform the same task after each other
51

52. User Flow
Maintain flow
Flow is about momentum
Momentum is a change of state in time
We can use temporal coherence to maintain flow
52

53. Predicting the Future
Knowing where the user is coming from is not so important
Use it to predict where user wants to go
53

54. Keyboard Navigation with Delete
Typical microinteraction
Go through list with arrows
Delete items
54

55. Keyboard Navigation with Delete
Typical microinteraction
Go through list with arrows
Delete items
55
Item 1
Item 2
Item3
Item 4
Item 5

56. Keyboard Navigation with Delete
Press the down arrow
56
Item 1
Item 2
Item 3
Item 4
Item 5

57. Keyboard Navigation with Delete
Press delete
Where is the selection now?
Item 3 or 5?
57
Item 1
Item 2
Item 3
?
Item 5 ?
Item 6

58. Keyboard Navigation with Delete
Programs use some default
Does not consider the past
actions of the user
58
Item 1
Item 2
Item 3
?
Item 5 ?
Item 6

59. Keyboard Navigation with Delete
A common default:
Item after deleted item
That would be Item 5
59
Item 1
Item 2
Item 3
Item 5
Item 6

60. Keyboard Navigation with Delete
Press Delete again
We delete the one the user
expects
60
Item 1
Item 2
Item 3
Item 6
Item 7

61. Keyboard Navigation with Delete
Let’s do this again
61
Item 1
Item 2
Item3
Item 4
Item 5

62. Keyboard Navigation with Delete
Press the up arrow this time
62
Item 1
Item 2
Item 3
Item 4
Item 5

63. Keyboard Navigation with Delete
Press delete
Selection of next logical one
Item 3 then
63
Item 1
Item 3
Item 4
Item 5
Item 6

64. Keyboard Navigation with Delete
If we press delete now, we will
do the wrong thing
This sort of interaction is very
automatic
64
Item 1
Item 3
Item 4
Item 5
Item 6

65. Keyboard Navigation with Delete
We should see where the user
came from in the past
Move in the same direction
We should have picked Item 1
65
Item 1
Item 3
Item 4
Item 5
Item 6

66. Keyboard Navigation with Delete
Keep the flow going
Remember the past
Determine direction of
movement
66

67. Keyboard Navigation with Delete
This is a small microinteraction
Bad Microinteractions can be
stumbling blocks
67

68. File Open Dialog Box
Remember the last Open/
Save Folder
Don’t use a default
68

69. Disclosure Boxes
User selects a document
69
John. Q. Doe
Contact
Uploads
Notes
Mail History

70. Disclosure Boxes
User selects a document
User opens a disclosure box
70
John. Q. Doe
Contact
Uploads
File 1
File 2
File 3

71. Disclosure Boxes
User selects a new document
71
Suzy. P. Deer
Contact
Uploads
File a
File b

72. Disclosure Boxes
The same box is open
Do not start with all closed
Use immediate past action
72
Suzy. P. Deer
Contact
Uploads
File a
File b

73. Predicting the Future
Use the immediate past to predict where user wants to go
Sometimes we know beforehand where we want to go
73

74. Business Process
Sometimes we have knowledge
of the desired future
74

75. Business Process
Sometimes we have knowledge
of the desired future
Use that to guide the user
75

76. Business Process
If you know the next step,
make it visible
76
John. Q. Doe
Todo: Review Contract
Contact
Uploads
Notes

77. Take home point #6
Assume that the user might be doing in
the immediate future what she was
doing in the immediate past
77

78. Take home point #7
Maintain flow by predicting or
knowing the future
78

79. The Object Model
79

80. The Current Way of Doing Things
The File Menu
Used to transfer document between
storages
80

81. The Current Way of Doing Things
Load/Save has a background in archaic
storage technology
81

82. Classical Object Lifecycle
Load
82
Main Memory
Fast DRAM
HDD
Slow Magnetic

83. Classical Object Lifecycle
Load…Edit
83
Main Memory
Fast DRAM
HDD
Slow Magnetic

84. Classical Object Lifecycle
Load…Edit…Save
84
Main Memory
Fast DRAM
HDD
Slow Magnetic

85. Classical Object Lifecycle
During edit we have Undo/Redo
The only way to go back in history
85
Step 1
Step 2
Step 3
Step 4
Step 5

86. Why Are We Doing This?
Computers have also changed over time
Current UI paradigms are still mired in the past
In this case, we might want to forget history
!
86

87. In the Old Days
Computers were very unreliable
‘Save often’ had more to do with saving
your work from the buggy OS
87

88. Now
Reliability is much improved
88

89. In the Old Days
Memory hierarchy
Reflected in the document lifecycle
Manually move things between memories
Memory transfer is slow
89
Main Memory
Fast DRAM
HDD
Slow Magnetic
Backup
Optical/Tape

90. Now
Programs are distributed between cloud
and local
Memory transfer is much faster
It is not really clear where data resides
Manually moving it around does not make
much sense
90
User

91. What do we need?
There is a serious impedance mismatch between current
paradigms and the reality of current technology
Classic document transfer is becoming obsolete
We should look at a new paradigm for document lifecycle
91

92. Save Button Quo Vadis?
Saving should not transfer data between memories
92

93. What comes after Save/Load?
Action to set a checkpoint in history
Action to manage history
93

94. Document Model
Current State
History
Perhaps also alternate histories
94

95. Managing History
Continuous updates
Explicit Snapshots in time
95

96. History Granularity
Save Points?
Each Action?
Each Keystroke?
96

97. Current Examples
Version Control Systems
Photoshop History Brush
Text Editor
97

98. Version Control
Time does not have to be linear
Change an undo stack into a tree
Distributed VCS show the way
98
Step 5
Step 4
Step 3
Step 2
Step 1
Alternate Step 4
Alternate Step 3

99. Version Control
Histories could in principle be
merged
Different Users or Features
This is an unsolved problem for
hierarchical structures
99
Step 4
Step 3
Step 2
Alternate Step 4
Alternate Step 3
Step 5
Merged

100. Photoshop History Brush
Checkpoints on an action
100

101. Photoshop History Brush
Stores the document on an action
Allows previous versions as a source
101

102. Photoshop History Brush
Stores the document on an action
Allows previous versions as a source
102

103. Text Editor
My dream: An editor with a eternal undo
Closing a tab should not lose the undo stack
Neither should quitting the editor
An editor continuously saving
103

104. Take home point #8
Manually moving data is the past
Managing history is the future
104

105. Pros and Cons of Amnesia
105

106. Why would we want to forget?
Privacy
Confidential info
Unneeded data
Storage limitations
106

107. What would we want for forget?
Short term memory to maintain flow
Test data
How long should we keep undo?
107

108. Take home point #9
Check how long and what you want
to remember
108

109. Why Not?
109

110. Why Not?
Storage Cost
User Inertia
Implementation Complexity
110

111. “The cost for 128 kilobytes
of memory will fall below
$100 in the near future.”
Creative Computing magazine, December 1981
111
Storage Cost ($/MB)
10000,0
100,0
1,0
0,0
0,0
1985 1992 2000 2007 2014

112. Inertia
Apple tried to change the File/Save
paradigm in newer versions of MacOS/X
The Save As… command is changed
Automatic versioning
This is generally not well received
112

113. Complexity
Storing a history is more complicated
113

114. Complexity: Differentials
Using differences causes errors to percolate
Makes deletes much harder
114

115. Take home point #10
Lack of storage is not an excuse
Users might not like the change
115

116. Implementation
116

117. Implementation
Store Snapshots
Store Events
Store Diffs
Each of these would reflect differently in the UI
117

118. In the back end
Logging
Datomic
Temporal SQL
Snapshotting with Distributed Version Control
118

119. Summary
119

120. Take home point #1
The universe consists of
processes, not things
120

121. Take home point #2
Users actions and software
objects are the result of process
121

122. Take home point #3
This is not about construction steps.
This is about why things became way
they are
122

123. Take home point #4
Flow is good
Flow is about momentum
Momentum is about change over time
123

124. Take home point #5
The database state is just a cached
version of the event log
124

125. Take home point #6
Assume that the user might be doing in
the immediate future what she was
doing in the immediate past
125

126. Take home point #7
Maintain flow by predicting or
knowing the future
126

127. Take home point #8
Manually moving data is the past
Managing history is the future
127

128. Take home point #9
Check how long and what you want
to remember
128

129. Take home point #10
Lack of storage is not an excuse
129