1. Krist Wongsuphasawat / @kristw
6 THINGS TO EXPECT
WHEN YOU ARE
VISUALIZING

2. 6 THINGS TO EXPECT
WHEN YOU ARE
VISUALIZING
Krist Wongsuphasawat / @kristw

3. Computer Engineer
Bangkok, Thailand
Chulalongkorn University
Krist Wongsuphasawat / @kristw

4. Programming + Soccer
Computer Engineer
Bangkok, Thailand
Krist Wongsuphasawat / @kristw

5. Programming + Soccer
Computer Engineer
Bangkok, Thailand
Krist Wongsuphasawat / @kristw

6. (P.S. These are actually not my robots, but our competitors’.)
Krist Wongsuphasawat / @kristw
Computer Engineer
Bangkok, Thailand

7. Krist Wongsuphasawat / @kristw
Computer Engineer
Bangkok, Thailand
PhD in Computer Science
Information Visualization
Univ. of Maryland

8. Krist Wongsuphasawat / @kristw
Computer Engineer
Bangkok, Thailand
IBM
Microsoft
PhD in Computer Science
Information Visualization
Univ. of Maryland

9. PhD in Computer Science
Information Visualization
Univ. of Maryland
IBM
Microsoft
Data Visualization Scientist
Twitter
Krist Wongsuphasawat / @kristw
Computer Engineer
Bangkok, Thailand

10. #interactive visualizations
Open-source projects
Visual Analytics Tools

11. DATA =ME+ VIS

12. Data, I’m ready!

13. Data, I’m ready!
Here I come!

14. WHAT TO EXPECT?

15. 1. EXPECT TO FIND THE REAL NEED

16. INPUT (DATA)
What clients think they have

17. INPUT (DATA)
What clients think they have What they usually have

18. YOU
What clients think you are

19. YOU
What clients think you are What they will get

20. OUTPUT (VIS)
What clients ask for

21. OUTPUT (VIS)
What clients ask for What they really need

22. COMMUNICATE

23. GOALS
Present data
Communicate information effectively
Analyze data
Exploratory data analysis
Tools to analyze data
Reusable tools for exploration
Enjoy
Combination of above

24. GOALS
Present data
Communicate information effectively
Analyze data
Exploratory data analysis
Tools to analyze data
Reusable tools for exploration
Enjoy
Combination of above
Who are the audience?
What do you want to tell?
What are the questions?
Who will use this?
What would they use this for?
Who are the audience?

25. I need this. Take this.

26. I need this. Here you are.
I need this. Take this.

27. & COMPROMISE

28. 2. EXPECT TO CLEAN DATA

29. 2. EXPECT TO CLEAN DATA A LOT

30. 70-80% of time cleaning data
“DATA JANITOR”

31. Collect + Clean + Transform
DATA WRANGLING

32. WHY DOES IT TAKE SO MUCH TIME?

33. 2.1 Many sources and data format

34. DATA SOURCES
Open data
Publicly available
Internal data
Private, owned by clients’ organization
Self-collected data
Manual, site scraping, etc.
Combine the above

35. DATA FORMAT
Standalone files
txt, csv, tsv, json, Google Docs, …, pdf*
Databases
doesn’t necessary mean they are organized
API
better quality with more overhead
Website
Big data*

36. NEED TO…
Change format
e.g. tsv => json
Combine data
Resolve multiple sources of truth

37. 2.2 Data transformation is needed.

38. EXAMPLES
Convert latitude/longitude into zip code
Change country code from 3-letter (USA) to 2-letter (US)
Correct time of day based on users’ timezone
etc.

39. 2.3 Data collection issues

40. EXAMPLES
Typos
Incorrect values
Incorrect timestamps
Missing data

41. 2.4 Definition of “clean” data

42. IS THIS CLEAN?
USER RESTAURANT RATING
========================
A MCDONALD’S 3
B MCDONALDS 3
C MCDONALD 4
D MCDONALDS 5
E IHOP 4
F SUBWAY 4

43. IS THIS CLEAN?
USER RESTAURANT RATING
========================
A MCDONALD’S 3
B MCDONALDS 3
C MCDONALD 4
D MCDONALDS 5
E IHOP 4
F SUBWAY 4
How many reviews are there?
Clean.
How many restaurants are there?
Not clean.
McDonald, McDonald’s, McDonalds

44. 2.5 Bigger data, bigger problems

45. HAVING ALL TWEETS
How people think I feel.

46. How people think I feel. How I really feel.
HAVING ALL TWEETS

47. Hadoop Cluster
GETTING BIG DATA
Data Storage

48. Scalding (slow)
GETTING BIG DATA
Hadoop Cluster
Data Storage
Tool

49. Scalding (slow)
GETTING BIG DATA
Hadoop Cluster
Data Storage
Tool
Your laptop Smaller dataset

50. Hadoop Cluster
Scalding (slow)
Data Storage
Tool
Final dataset
Tool node.js / python / excel (fast)
Your laptop
GETTING BIG DATA
Smaller dataset

51. CHALLENGES
Slow
Long processing time (hours)
Get relevant Tweets
hashtag: #oscars
keywords: “moonlight” (movie name)
Too big
Need to aggregate & reduce size
Harder to spot problems

52. RAMSAY & RAMSEY

53. 2.6 New issues can show up any time.

55. RECOMMENDATIONS
Always think that you will have to do it again
document the process, automation
Reusable scripts
break a gigantic do-it-all function into smaller ones
Reusable data
keep for future project

56. 3. EXPECT TRIALS AND ERRORS

57. It’s gonna be legen-

58. Celebrate your trials
#D3BrokeAndMadeArt

59. When your vis starts working

60. “Necessity is the mother of invention.”
— English Proverb

61. “Necessity is the mother of invention.”
— English Proverb
DEADLINE

62. EXAMPLE PROJECTS

63. PROJECT 1:
GAME OF THRONES
#INTERACTIVE

64. INTERACTIVE.TWITTER.COM

65. WHAT TO EXPECT
timely
Deadline is strict. Also can be unexpected events.
wide audience
easy to explain and understand, multi-device support
one-off project
scope
analyze data to find stories and find best way to present them

66. from fans’ conversations
Reveal the talking points
of every episode of

67. Problem is coming.
CHAPTER I

68. Problem
Want to know what the audience
talk about a TV show
from Tweets

69. HBO’s Game of Thrones
Based on a book series “A Song of Ice and Fire”
Medieval Fantasy. Knights, magic and dragons.

70. Brief Story

71. A King dies.
A lot of contenders wage a war
to reclaim the throne.

72. Minor characters with no claim to the throne
set their own plans in action to gain power
when all the major characters end up killing each other.

73. Brave/Honest/Honorable characters die.
Intelligent but shady characters
and characters who know nothing
continue to live.

74. While humans are busy killing each other,
ice zombies “White walkers” are invading from the North.
The only group who seems to care about this
is neutral group called the Night’s Watch.

75. HBO’s Game of Thrones
Based on a book series “A Song of Ice and Fire”
Medieval Fantasy. Knights, magic and dragons.
Many characters.
Anybody can die.
6 seasons (60 episodes) so far
Multiple storylines in each episode

76. Problem
Want to know what the audience
talk about a TV show
from Tweets

77. Ideas
Common words
Too much noise

78. Ideas
Common words
Too much noise
Characters
How o!en each character were mentioned?

79. I demand a trial by prototyping.
CHAPTER II

80. Prototyping
Pull sample data
from Twitter API
Entity recognition and counting
naive approach

81. List of names
Daenerys Targaryen,Khaleesi
Jon Snow
Sansa Stark
Tyrion Lannister
Arya Stark
Cersei Lannister
Khal Drogo
Gregor Clegane,Mountain
Margaery Tyrell
Joffrey Baratheon
Bran Stark
Theon Greyjoy
Jaime Lannister
Brienne
Eddard Stark,Ned Stark
Ramsay Bolton
Sandor Clegane,Hound
Ygritte
Stannis Baratheon
Petyr Baelish,Little Finger
Robb Stark
Bronn
Varys
Catelyn Stark
Oberyn Martell
Daario Naharis
Davos Seaworth
Jorah Mormont
Melisandre
Myrcella Baratheon
Tywin Lannister
Tommen Baratheon
Grey Worm
Tyene Sand
Rickon Stark
Missandei
Roose Bolton
Robert Baratheon
Jojen Reed
Jeor Mormont
Tormund Giantsbane
Lysa Arryn
Yara Greyjoy,Asha Greyjoy
Samwell Tarly,Sam
Hodor
Victarion Greyjoy
High Sparrow
Dragon
Winter
Dothraki

82. Sample Tweet

83. Sample Tweet

84. Sample data
Character Count
Hodor 10000
Jon Snow 5000
Daenerys 4000
Bran Stark 3000
… …
*These numbers are made up for presentation, not real data.

85. When you play the game of vis,
you iterate or you die.
CHAPTER III

86. Where to go from here?

87. + episodes
The Guardian & Google Trends
http://www.theguardian.com/news/datablog/ng-interactive/2016/apr/22/game-of-thrones-the-most-googled-characters-episode-by-episode

88. + emotion

89. + connections

90. + connections

91. Gain insights from a single episode
emotion & connections

92. Sample data
Character Count
Jon Snow+Sansa 1000
Tormund+Brienne 500
Bran Stark+Hodor 300
… …
Character Count
Hodor 10000
Jon Snow 5000
Daenerys 4000
… …
INDIVIDUALS CONNECTIONS
+ top emojis + top emojis
*These numbers are made up for presentation, not real data.

93. Graph
NODES LINKS
+ top emojis + top emojis
Character Count
Jon Snow+Sansa 1000
Tormund+Brienne 500
Bran Stark+Hodor 300
… …
Character Count
Hodor 1000
Jon Snow 500
Daenerys 400
… …
*These numbers are made up for presentation, not real data.

94. Network Visualization
Node-link diagram
Force-directed layout
http://blockbuilder.org/kristw/762b680690e4b2b2666dfec15838a384

95. Issue: Hairball

96. Issue: Occlusions

97. Tried: Fixed positions

98. + Collision Detection
http://blockbuilder.org/kristw/2850f65d6329c5fef6d5c9118f1de6e6

99. + Community Detection
https://github.com/upphiminn/jLouvain

100. + Collision Detection (with clusters)
https://bl.ocks.org/mbostock/7881887

101. Tormund + Brienne

102. Issue: Convex hull
http://bl.ocks.org/mbostock/4341699

103. x & y only, no radius

104. Example

105. Fix it

106. Fix it

107. Let’s get other episodes.

108. Hadoop remembers.
CHAPTER IV

109. More data
Hadoop
Rewrite the scripts in Scalding
to get archived data

110. How much data do we need?
Whole week?
5 days?
2 days?
A day?
etc.

111. How much data do we need?

112. Transitions

113. Changing episode

114. A#er switching episode
1. Store old positions for existing characters.
2. Assign positions for new characters.

115. Community transition
t=0 t=1

116. Smoother
t=0 t=1t=0.5 t=0.51

117. Colors
Default: D3 category10
Distinct but nothing about the context
Custom palette
Colors related to the groups/houses.
Black = Night’s Watch
Blue = North
Red = Daenerys
Gold = Lannister
…

118. Hold the vis.
CHAPTER V

119. The vis is not enough.

121. Legend

122. Navigation

123. Top 3

124. Adjust threshold

125. Recap

126. Filtered Recap
Tooltip

127. Demo
https://interactive.twitter.com/game-of-thrones

128. Mobile Support

129. A visualizer always evaluates his work.
CHAPTER VI

130. Self & Peer
Does it solve the problem?

131. Google Analytics
Pageviews
Visitors
Actions
Referrals
Sites/Social

132. Feedback

133. Feedback

134. PROJECT 2:
VISUAL ANALYTICS TOOLS
FOR LOGGING

135. WHAT TO EXPECT
richer, more features
to support exploration of complex data
more technical audience
product managers, engineers, data scientists
accuracy
designed for dynamic input
long-term projects

136. Data sources
Output
explore
analyze
present
get
*
*

137. Data sources
Output
explore
analyze
present
get
*
*
ad-hoc scripts

138. Data sources
Output
explore
analyze
present
get
*
*
ad-hoc scripts tools for exploration

139. USER ACTIVITY LOGS

140. UsersUseTwitter

141. UsersUse
Product Managers
Curious
Twitter

142. UsersUse
Curious
Engineers
Log data
in Hadoop
Write Twitter
Instrument
Product Managers

143. WHAT ARE BEING LOGGED?
tweet
Activities

144. WHAT ARE BEING LOGGED?
tweet from home timeline on twitter.com
tweet from search page on iPhone
Activities

145. WHAT ARE BEING LOGGED?
tweet from home timeline on twitter.com
tweet from search page on iPhone
sign up
log in
retweet
etc.
Activities

146. ORGANIZE?

147. LOG EVENT A.K.A. “CLIENT EVENT”
[Lee et al. 2012]

148. LOG EVENT A.K.A. “CLIENT EVENT”
client : page : section : component : element : action
web : home : timeline : tweet_box : button : tweet
1) User ID
2) Timestamp
3) Event name
4) Event detail
[Lee et al. 2012]

149. LOG DATA

150. UsersUse
Curious
Engineers
Log data
in Hadoop
Twitter
Instrument
Write
Product Managers
bigger than
Tweet data

151. UsersUse
Curious
Engineers
Log data
in Hadoop
Data Scientists
Ask
Twitter
Instrument
Write
Product Managers

152. UsersUse
Curious
Engineers
Log data
in Hadoop
Data Scientists
Find
Ask
Twitter
Instrument
Write
Product Managers

153. LOG DATA

154. UsersUse
Curious
Engineers
Log data
in Hadoop
Data Scientists
Find, Clean
Ask
Twitter
Instrument
Write
Product Managers

155. UsersUse
Curious
Engineers
Log data
in Hadoop
Data Scientists
Find, Clean
Ask
Monitor
Twitter
Instrument
Write
Product Managers

156. UsersUse
Curious
Engineers
Log data
in Hadoop
Data Scientists
Find, Clean, Analyze
Ask
Monitor
Twitter
Instrument
Write
Product Managers

157. Log data
EngineersData Scientists
Usersin Hadoop
Find, Clean, Analyze
Use
Monitor
Ask
Curious
1 2
Twitter
Instrument
Write
Product Managers

158. client page section component element action
Event
50,000+ event types

159. client page section component element action
Event
50,000+ event types
one graph / event
x 50,000

160. DESIGN

161. See
Client event collection
Engineers & Data Scientists

162. See
Client event collection
Engineers & Data Scientists
narrow down
Interactions
search box => filter

163. See
HOW TO VISUALIZE?
narrow down
Client event collection
Engineers & Data Scientists
Interactions
search box => filter

164. See
Client event collection
Engineers & Data Scientists
client : page : section : component : element : action
HOW TO VISUALIZE?
narrow down
Interactions
search box => filter

165. CLIENT EVENT HIERARCHY
iphone home -
- - impression
tweet tweet click
iphone:home:-:-:-:impression
iphone:home:-:tweet:tweet:click

166. DETECT CHANGES
iphone home -
- - impression
tweet tweet click
iphone home -
- - impression
tweet tweet click
TODAY
7 DAYS AGO
compared to

167. CALCULATE CHANGES
+5% +5% +5%
+10% +10% +10%
-5% -5% -5%
DIFF

168. DISPLAY CHANGES
iphone home -
- - impression
tweet tweet click
Map of the Market [Wattenberg 1999], StemView [Guerra-Gomez et al. 2013]

169. DISPLAY CHANGES
home -
- - impression
tweet tweet click
iphone

170. Demo Demo Demo
Demo / Scribe Radar

171. Twitter for Banana

173. PROJECT 3:
VISUAL ANALYTICS TOOLS
FOR EXPERIMENTATION

174. A/B TESTING

175. RUN AN EXPERIMENT
Develop feature
Track metrics
1. No. of Tweets read
2. No. of Tweets sent
3. No. of Users
4. …
Set bucket size
How many users?

176. RETROSPECTIVE ANALYSIS
Data scientist analyzed 100+ past experiments.
Many useful insights.
- We could move metric A by X% on average.
- Experiment 18 moved metric A the most
- Which team was able to move metric A successfully?
- etc.

177. RETROSPECTIVE ANALYSIS
Data scientist analyzed 100+ past experiments.
Many useful insights.
- We could move metric A by X% on average.
- Experiment 18 moved metric A the most
- Which team was able to move metric A successfully?
- etc.
Amount of knowledge transfer = slide deck + wiki page.
Reproduce for recent experiments? Manually.

178. RETROSPECTIVE ANALYSIS
Data scientist analyzed 100+ past experiments.
Many useful insights.
- We could move metric A by X% on average.
- Experiment 18 moved metric A the most
- Which team was able to move metric A successfully?
- etc.
Amount of knowledge transfer = slide deck + wiki page.
Reproduce for recent experiments? Manually.
Make results more accessible
and convenient to use.

179. RETROSPECTIVE ANALYSIS
Data scientist analyzed 100+ past experiments.
Many useful insights.
- We could move metric A by X% on average.
- Experiment 18 moved metric A the most
- Which team was able to move metric A successfully?
- etc.
Amount of knowledge transfer = slide deck + wiki page.
Reproduce for recent experiments? Manually.
Make results more accessible
and convenient to use.
Automatic

180. Metric Mover
I like to move it, move it
Krist Wongsuphasawat, Joseph Liu,
Matthew Schreiner, Andy Schlaikjer, Lucile Lu and Busheng Lou

181. Set OKRs
Process
# of posts

182. Implement a feature
Set OKRs
Process
Setup experiment
# of posts
# of posts

183. Implement a feature
Set OKRs
Interpret results
Process
Run experiment
+1.0%
Setup experiment
# of posts
# of posts

184. Implement a feature
Set OKRs
Interpret results
Process
Run experiment
+1.0%
Setup experiment
How easy/hard it is to move this metric?
How much change to aim for?
Challenges
# of posts
# of posts

185. Implement a feature
Set OKRs
Interpret results
Process
Run experiment
+1.0%
How much to expect from one experiment?
What were the successful features?
Who had experience with this?Setup experiment
How easy/hard it is to move this metric?
How much change to aim for?
Challenges
# of posts
# of posts

186. Implement a feature
Set OKRs
Interpret results
Process
Run experiment
+1.0%
How much to expect from one experiment?
What were the successful features?
Who had experience with this?Setup experiment
How easy/hard it is to move this metric?
How much change to aim for?
How good is this?
Challenges
# of posts
# of posts

187. Past experiments
Metric Mover

188. Exp. 1
Exp. 2
Exp. 3
Exp. 4
Metric: No. of Posts

189. Exp. 1
Exp. 2
Exp. 3
Exp. 4
Metric: No. of Posts
Control buckets

190. Exp. 1
Exp. 2
Exp. 3
Exp. 4
Metric: No. of Posts

191. Exp. 1
Exp. 2
Exp. 3
Exp. 4
Metric: No. of Posts
Insignificant buckets

192. Exp. 1
Exp. 2
Exp. 3
Exp. 4
Metric: No. of Posts

193. Metric: No. of Posts

194. Metric: No. of Posts
% change
0-1%-2% 2%1%

195. Metric: No. of Posts
% change
0-1%-2% 2%1%
|scaled impact|
100,000,000
1,000,000
10,000
100

196. Users who watch cat GIFs Users who like cat GIFs Users who post cat GIFs
**These are fake data.**

197. WORKFLOW
Identify needs
Design and prototype
Make it work for sample dataset
Refine, generalize and productionize
Make it work for other cases
Document and release
Maintain and support
Keep it running, Feature requests & Bugs fix

198. What separates good and great work
4. EXPECT TIME FOR REFINEMENT

199. REFINE & POLISH
UX / UI
+ Mobile Support
Color
Animation / Transition
Performance
Loading time, Data file size
“The little of visualisation design” by Andy Kirk
http://www.visualisingdata.com/2016/03/little-visualisation-design/

200. “The first 90% of the code
accounts for the first 90% of the development time.
The remaining 10% of the code
accounts for the other 90% of the development time.”
— Tom Cargill, Bell Labs

201. or find ways to get some
5. EXPECT FEEDBACK

202. “Feedback is the breakfast of champion.”
— Ken Blanchard

203. FEEDBACK
During development
Feedback sessions with clients/potential users
After release
Logging
User study
Forum, User group
Office hours

204. 6. EXPECT TO IMPROVE

205. HOW TO BE BETTER?
Time is limited.
Learn from the past
Expand skills
Get help / Grow the team
Improve tooling
Solve a problem once and for all
Automate repetitive tasks

206. http://twitter.github.io/labella.js
Demo / Labella.js

207. https://github.com/twitter/d3kit
Demo / d3Kit
http://www.slideshare.net/kristw/d3kit

208. yeoman.io
Demo / Yeoman

209. SUMMARY

210. EXPECT…
1. to find the real need
2. to clean data a lot
3. trials and errors
4. time for refinement
5. feedback
6. to improve
Krist Wongsuphasawat / @kristw
kristw.yellowpigz.com

211. THANK YOU

212. QUESTIONS?

213. My colleagues at Twitter for their collaboration
and support in these projects;
and my wife for taking care of the baby
while I make these slides.
ACKNOWLEDGEMENT

214. RESOURCES
Images
Banana phone http://goo.gl/GmcMPq
Bar chart https://goo.gl/1G1GBg
Boss https://goo.gl/gcY8Kw
Champions League http://goo.gl/DjtNKE
Database http://goo.gl/5N7zZz
Fishing shark http://goo.gl/2fp4zW
Frustrated programmer https://goo.gl/ZLDNny
Globe visualization http://goo.gl/UiGMMj
Harry Potter http://goo.gl/Q9Cy64
Holding phone http://goo.gl/It2TzH
Jon Snow https://goo.gl/CACWxE
Jon Snow lightsaber https://goo.gl/CJt1Tn
Kiwi orange http://goo.gl/ejQ73y
Kiwi http://goo.gl/9yk7o5
Library https://goo.gl/HVeE6h
Library earthquake http://goo.gl/rBqBrs
Minion http://goo.gl/I19Ijg
Nemo https://goo.gl/m0pmzC
Orange & Apple http://goo.gl/NG6RIL
Pile of paper http://goo.gl/mGLQTx
Scrooge McDuck https://goo.gl/aKv8D7
Trash pile http://goo.gl/OsFfo3
Watercolor Map by Stamen Design
Yes GIF https://goo.gl/agvlAE