Data Visualization at Twitter

Krist Wongsuphasawat
Krist WongsuphasawatData Visualization
visualization 
at Twitter 
data 
Krist Wongsuphasawat / @kristw
Krist Wongsuphasawat / @kristw
Bangkok, Thailand 
Krist Wongsuphasawat / @kristw
Computer Engineer 
Bangkok, Thailand 
Chulalongkorn University 
Krist Wongsuphasawat / @kristw
Computer Engineer 
Bangkok, Thailand 
Programming + Soccer 
Krist Wongsuphasawat / @kristw
Computer Engineer 
Bangkok, Thailand 
Programming + Soccer 
Krist Wongsuphasawat / @kristw
Computer Engineer 
Bangkok, Thailand 
Programming + Soccer 
Krist Wongsuphasawat / @kristw
Computer Engineer 
Bangkok, Thailand 
M.S. in Computer Science 
Univ. of Maryland 
Krist Wongsuphasawat / @kristw
Computer Engineer 
Bangkok, Thailand 
PhD in Computer Science 
Univ. of Maryland 
Information Visualization 
Krist Wongsuphasawat / @kristw
Computer Engineer 
Bangkok, Thailand 
PhD in Computer Science 
Univ. of Maryland 
Information Visualization 
Krist Wongsuphasawat / @kristw 
IBM 
Microsoft
Computer Engineer 
Bangkok, Thailand 
PhD in Computer Science 
Univ. of Maryland 
Information Visualization 
Krist Wongsuphasawat / @kristw 
IBM 
Microsoft 
Sr. Data Visualization Scientist 
Twitter
data visualization 
at Twitter
data visualization 
at Twitter
visualization 
data 
at Twitter
vis 
data 
at Twitter
data 
at Twitter 
“Tweets”
data 
at Twitter 
“Tweets” 
#events 
TV Shows New Year 
Earthquake 
Oscars 
Protest 
Super Bowl 
World Cup Election 
Breaking news 
…
data 
at Twitter 
“Tweets” 
#events 
TV Shows New Year 
Earthquake 
Oscars 
Protest 
Super Bowl 
World Cup Election 
Breaking news 
… 
#curiosity 
Sleep pattern 
Human behavior 
Language …
data 
at Twitter 
“Tweets” 
#events 
TV Shows New Year 
Earthquake 
Oscars 
Protest 
Super Bowl 
World Cup Election 
Breaking news 
… 
#curiosity 
Sleep pattern 
Human behavior 
Language … 
What could we learn from the Tweets?
vis 
data 
at Twitter 
“Tweets” 
Tell stories about an event, 
Pursue curiosity or inspiration 
Goal:
vis 
data 
at Twitter 
“Tweets” 
Tell stories about an event, 
Pursue curiosity or inspiration 
(with deadline) 
Goal:
Challenge accepted
vis 
data 
at Twitter 
“Tweets” 
Get data 
1
easy?
Having all Tweets 
How people think I feel.
Having all Tweets 
How people think I feel. How I really feel.
Challenges 
• Too much data 
• Want only relevant Tweets 
• hashtag: #BRA 
• keywords: “goal” 
• Need to aggregate & reduce size 
• Long processing time (hours)
Hadoop Cluster 
Vertica 
Pig / Scalding (slow) SQL 
Data Storage 
Tool 
Workflow
Hadoop Cluster 
Vertica 
Pig / Scalding (slow) SQL 
Data Storage 
Tool 
Workflow
Workflow 
Hadoop Cluster 
Vertica 
Pig / Scalding (slow) SQL 
Data Storage 
Tool 
Your laptop Smaller dataset
Hadoop Cluster 
Vertica 
Pig / Scalding (slow) SQL 
Data Storage 
Tool 
Tool node.js / python / excel (fast) 
Final dataset 
Your laptop 
Workflow 
Smaller dataset
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize
Visualize 
• Peek into data 
• Check data & test ideas 
• Decide how to visualize 
• Guided by data type 
• Choose tools 
• Start building
Visualize 
• Peek into data 
• Check data & test ideas 
• Decide how to visualize 
• Guided by data type 
• Choose tools 
• Start building 
R d3 
Tableau Yeoman
(+ media) 
photos, videos 
data 
What? 
TEXT 
Where? When? 
GEO TIME
Visualize Data 
What? 
TEXT 
Where? When? 
GEO TIME
Visualize Data 
What? 
TEXT 
Where? When? 
GEO TIME
Time Tweets/second
Time Tweets/second
Time Tweets/second + Annotation 
http://www.flickr.com/photos/twitteroffice/5681263084/
Visualize Data 
What? 
TEXT 
Where? When? 
GEO TIME
Geo 
Heatmap 
Low density 
High density
Geo 
San Francisco 
flickr.com/photos/twitteroffice/8798020541 
Low density 
High density
Geo 
San Francisco 
Rebuild the world 
based on 
tweet volumes 
twitter.github.io/interactive/andes/
Visualize Data 
What? 
TEXT 
Where? When? 
GEO TIME
Text 
www.wordle.net 
Some experiments 
during World Cup
Text 
www.wordle.net 
Word cloud of Tweets right after the 1st goal
Text Word cloud of Tweets right after the 1st goal 
It was an “own” goal. 
www.wordle.net
Text WordTree [Wattenberg & Viégas 2008] 
www.jasondavies.com/wordtree 
www.jasondavies.com/wordtree
Visualize Data 
What? 
TEXT 
Where? When? 
GEO TIME
Time + Geo 
Japan Earthquake 2011 
blog.twitter.com/2011/global-pulse 
youtu.be/SybWjN9pKQk
Data Visualization at Twitter
Data Visualization at Twitter
Time + Geo Tweet pattern [Rios & Lin 2012] 
Night 
Late night 
Daytime 
Night 
Late night 
Daytime
Time + Geo Tweet pattern [Rios & Lin 2012] 
Night 
Late night 
Daytime 
Night 
Late night 
Daytime
Time + Geo Tweet pattern [Rios & Lin 2012] 
Night 
Late night 
Daytime 
Night 
Late night 
Daytime
Time + Geo Tweet pattern [Rios & Lin 2012] 
Night 
Late night 
Daytime 
Night 
Late night 
Daytime
Visualize Data 
What? 
TEXT 
Where? When? 
GEO TIME
Geo + Text Real-time Tweet map
Geo + Text Real-time Tweet map
Geo + Text Real-time Tweet map 
most 
frequent 
term
Geo + Text Real-time Tweet map 
Gmail was down 
Jan 24, 2014
Geo + Text Real-time Tweet map 
Nelson Mandela 
passed away 
Dec 5, 2013
Visualize Data 
What? 
TEXT 
Where? When? 
GEO TIME
Time + Text 
UEFA Champions League 
Biggest tournament for European soccer clubs 
Many Tweets during the matches
UEFA Champions League 
Team 1 Team 2 
Time + Text 
Dortmund Bayern Munich
UEFA Champions League 
Team 1 Team 2 
Time + Text 
Dortmund Bayern Munich
UEFA Champions League 
Team 1 Team 2 
Time + Text 
Dortmund Bayern Munich
UEFA Champions League 
Team 1 Team 2 
Dortmund Bayern Munich 
Count Tweets mentioning 
the teams every minute 
Time + Text
Time + Text UEFA Champions League
Time + Text UEFA Champions League 
+ “goal” count 
+ context
+ “offside” 
Time + Text UEFA Champions League
+ players 
Time + Text UEFA Champions League
Competition Tree 
vs vs 
A B C D 
vs 
A C 
C
Competition Tree 
vs vs 
A B C D 
vs + 
A C 
C
Competition Tree 
vs vs 
A B C D 
vs + = 
A C 
C
Visualize Data 
What? 
TEXT 
Where? When? 
GEO TIME
Time + Text + Geo State of the Union 
twitter.github.io/interactive/sotu2014
Time + Text + Geo State of the Union 
1) timeline + topic from Tweets 
4) Density map of 
Tweets about 
selected topic 
3) Volume of Tweets 
by topics 
during selected 
part of the SOTU 
2) context 
(speech) 
twitter.github.io/interactive/sotu2014
Time + Text World Cup 2014
Time + Text + Geo World Cup 2014
Visualize Data 
What? 
TEXT 
Where? When? 
GEO TIME
Visualize Data 
What? 
TEXT 
+ 
Where? When? 
GEO TIME 
Non-Twitter data 
CONTEXT
Time + Text New Year 2014
Time + Text New Year 2014
Time + Text + Geo (c) New Year 2014 
twitter.github.io/interactive/newyear2014/
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize 
Evaluate 
3
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize 
Evaluate 
3 
Iterate!
Evaluation 
• Self 
• Peer feedback 
• Non team members / Potential audience
vis 
data 
at Twitter 
Get data 
1 
2 
Visualize 
Evaluate 
3
vis 
data 
at Twitter 
Get data 
1 
2 
Visualize 
Evaluate 
3 
big data => small data
vis 
data 
at Twitter 
Get data 
1 
2 
Visualize 
Evaluate 
3 
big data => small data 
What? Where? When?
big data => small data self, peer, external 
vis 
data 
at Twitter 
Get data 
1 
2 
Visualize 
Evaluate 
3 
What? Where? When?
big data => small data self, peer, external 
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize 
Evaluate 
3 
What? Where? When?
big data => small data self, peer, external 
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize 
Evaluate 
3 
What? Where? When? 
• users 
• followers graph 
• logs 
• etc. 
! 
• derived data: language, sentiment
big data => small data self, peer, external 
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize 
Evaluate 
3 
What? Where? When? 
• users Who? … 
• followers graph 
• logs 
• etc. 
! 
• derived data: language, sentiment
big data => small data self, peer, external 
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize 
Evaluate 
3 
What? Where? When? 
• users Who? … 
• followers graph 
• logs 
• etc. 
! 
• derived data: language, sentiment 
(with deadline)
big data => small data self, peer, external 
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize 
Evaluate 
3 
What? Where? When? 
• users Who? … 
• followers graph 
• logs 
• etc. 
(with deadline) 
! 
• derived data: language, sentiment @kristw / https://interactive.twitter.com
big data => small data self, peer, external 
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize 
Evaluate 
3 
What? Where? When? 
• users Who? … 
• followers graph 
• logs 
• etc. 
(with deadline) 
! 
• derived data: language, sentiment @kristw / https://interactive.twitter.com 
+ visualizations by @philogb, @miguelrios & @trebor
Questions?
big data => small data self, peer, external 
vis 
data 
at Twitter 
“Tweets” 
Get data 
1 
2 
Visualize 
Evaluate 
3 
What? Where? When? 
• users Who? … 
• followers graph 
• logs 
• etc. 
(with deadline) 
@kristw / https://interactive.twitter.com 
+ visualizations by @philogb, @miguelrios & @trebor
Thank you
1 of 103

Recommended

Adventure in Data: A tour of visualization projects at Twitter by
Adventure in Data: A tour of visualization projects at TwitterAdventure in Data: A tour of visualization projects at Twitter
Adventure in Data: A tour of visualization projects at TwitterKrist Wongsuphasawat
1.1K views175 slides
Making Sense of Millions of Thoughts: Finding Patterns in the Tweets by
Making Sense of Millions of Thoughts: Finding Patterns in the TweetsMaking Sense of Millions of Thoughts: Finding Patterns in the Tweets
Making Sense of Millions of Thoughts: Finding Patterns in the TweetsKrist Wongsuphasawat
1.7K views106 slides
Using Visualizations to Monitor Changes and Harvest Insights from a Global-sc... by
Using Visualizations to Monitor Changes and Harvest Insights from a Global-sc...Using Visualizations to Monitor Changes and Harvest Insights from a Global-sc...
Using Visualizations to Monitor Changes and Harvest Insights from a Global-sc...Krist Wongsuphasawat
8.5K views109 slides
What to expect when you are visualizing by
What to expect when you are visualizingWhat to expect when you are visualizing
What to expect when you are visualizingKrist Wongsuphasawat
1.7K views187 slides
Logs & Visualizations at Twitter by
Logs & Visualizations at TwitterLogs & Visualizations at Twitter
Logs & Visualizations at TwitterKrist Wongsuphasawat
1.8K views142 slides
Data Visualization: A Quick Tour for Data Science Enthusiasts by
Data Visualization: A Quick Tour for Data Science EnthusiastsData Visualization: A Quick Tour for Data Science Enthusiasts
Data Visualization: A Quick Tour for Data Science EnthusiastsKrist Wongsuphasawat
50.6K views95 slides

More Related Content

What's hot

6 things to expect when you are visualizing by
6 things to expect when you are visualizing6 things to expect when you are visualizing
6 things to expect when you are visualizingKrist Wongsuphasawat
2.3K views214 slides
Apache Spark 101 [in 50 min] by
Apache Spark 101 [in 50 min]Apache Spark 101 [in 50 min]
Apache Spark 101 [in 50 min]Pawel Szulc
4.9K views185 slides
Python for Big Data Analytics by
Python for Big Data AnalyticsPython for Big Data Analytics
Python for Big Data AnalyticsEdureka!
37.1K views20 slides
Twitter analysis by
Twitter analysisTwitter analysis
Twitter analysisHimanshu Yadav
235 views14 slides
6 things to expect when you are visualizing (2020 Edition) by
6 things to expect when you are visualizing (2020 Edition)6 things to expect when you are visualizing (2020 Edition)
6 things to expect when you are visualizing (2020 Edition)Krist Wongsuphasawat
475 views203 slides

What's hot(20)

Apache Spark 101 [in 50 min] by Pawel Szulc
Apache Spark 101 [in 50 min]Apache Spark 101 [in 50 min]
Apache Spark 101 [in 50 min]
Pawel Szulc4.9K views
Python for Big Data Analytics by Edureka!
Python for Big Data AnalyticsPython for Big Data Analytics
Python for Big Data Analytics
Edureka!37.1K views
6 things to expect when you are visualizing (2020 Edition) by Krist Wongsuphasawat
6 things to expect when you are visualizing (2020 Edition)6 things to expect when you are visualizing (2020 Edition)
6 things to expect when you are visualizing (2020 Edition)
Power of Python with Big Data by Edureka!
Power of Python with Big DataPower of Python with Big Data
Power of Python with Big Data
Edureka!1.6K views
Open Source Search Tools for www2010 conferencesourcesearchtoolswww20100426dA... by Ted Drake
Open Source Search Tools for www2010 conferencesourcesearchtoolswww20100426dA...Open Source Search Tools for www2010 conferencesourcesearchtoolswww20100426dA...
Open Source Search Tools for www2010 conferencesourcesearchtoolswww20100426dA...
Ted Drake132.6K views
UBC STAT545 2014 Cm001 intro to-course by Jennifer Bryan
UBC STAT545 2014 Cm001 intro to-courseUBC STAT545 2014 Cm001 intro to-course
UBC STAT545 2014 Cm001 intro to-course
Jennifer Bryan5.4K views
Integrating and Interpreting Social Data from Heterogeneous Sources by Matthew Rowe
Integrating and Interpreting Social Data from Heterogeneous SourcesIntegrating and Interpreting Social Data from Heterogeneous Sources
Integrating and Interpreting Social Data from Heterogeneous Sources
Matthew Rowe598 views
Effective and efficient google searching power point tutorial by Jaclyn Lee Parrott
Effective and efficient google searching power point tutorialEffective and efficient google searching power point tutorial
Effective and efficient google searching power point tutorial
Open Source Community Metrics for FOSDEM by Dawn Foster
Open Source Community Metrics for FOSDEMOpen Source Community Metrics for FOSDEM
Open Source Community Metrics for FOSDEM
Dawn Foster1.8K views
Hadoop and Neo4j: A Winning Combination for Bioinformatics by osintegrators
Hadoop and Neo4j: A Winning Combination for BioinformaticsHadoop and Neo4j: A Winning Combination for Bioinformatics
Hadoop and Neo4j: A Winning Combination for Bioinformatics
osintegrators1.2K views

Viewers also liked

From Data to Visualization, what happens in between? by
From Data to Visualization, what happens in between?From Data to Visualization, what happens in between?
From Data to Visualization, what happens in between?Krist Wongsuphasawat
48K views78 slides
Principles of Data Visualization by
Principles of Data VisualizationPrinciples of Data Visualization
Principles of Data VisualizationEamonn Maguire
5.7K views169 slides
Visual Design with Data by
Visual Design with DataVisual Design with Data
Visual Design with DataSeth Familian
2.9M views76 slides
Data Visualization 101: How to Design Charts and Graphs by
Data Visualization 101: How to Design Charts and GraphsData Visualization 101: How to Design Charts and Graphs
Data Visualization 101: How to Design Charts and GraphsVisage
62.8K views25 slides
JESS3 Social Media Data Visualization by
JESS3 Social Media Data VisualizationJESS3 Social Media Data Visualization
JESS3 Social Media Data VisualizationJesse Thomas
2.4K views119 slides
Analytics for startups by
Analytics for startupsAnalytics for startups
Analytics for startupsKissmetrics on SlideShare
6.7K views48 slides

Viewers also liked(20)

From Data to Visualization, what happens in between? by Krist Wongsuphasawat
From Data to Visualization, what happens in between?From Data to Visualization, what happens in between?
From Data to Visualization, what happens in between?
Principles of Data Visualization by Eamonn Maguire
Principles of Data VisualizationPrinciples of Data Visualization
Principles of Data Visualization
Eamonn Maguire5.7K views
Visual Design with Data by Seth Familian
Visual Design with DataVisual Design with Data
Visual Design with Data
Seth Familian2.9M views
Data Visualization 101: How to Design Charts and Graphs by Visage
Data Visualization 101: How to Design Charts and GraphsData Visualization 101: How to Design Charts and Graphs
Data Visualization 101: How to Design Charts and Graphs
Visage62.8K views
JESS3 Social Media Data Visualization by Jesse Thomas
JESS3 Social Media Data VisualizationJESS3 Social Media Data Visualization
JESS3 Social Media Data Visualization
Jesse Thomas2.4K views
Data 2.0 - Harnessing New Data Visualization Tools CIL 2008 by Darlene Fichter
Data 2.0 - Harnessing New Data Visualization Tools CIL 2008Data 2.0 - Harnessing New Data Visualization Tools CIL 2008
Data 2.0 - Harnessing New Data Visualization Tools CIL 2008
Darlene Fichter3.4K views
Can Digital Data help predict the results of the US elections? by Laurence Borel
Can Digital Data help predict the results of the US elections? Can Digital Data help predict the results of the US elections?
Can Digital Data help predict the results of the US elections?
Laurence Borel596 views
Tip from IBM Connect 2014: Socialytics = Social Business, Big Social Data and... by SocialBiz UserGroup
Tip from IBM Connect 2014: Socialytics = Social Business, Big Social Data and...Tip from IBM Connect 2014: Socialytics = Social Business, Big Social Data and...
Tip from IBM Connect 2014: Socialytics = Social Business, Big Social Data and...
SocialBiz UserGroup2.2K views
Searching lexis nexis in power search mode by Joyce Johnston
Searching lexis nexis in power search modeSearching lexis nexis in power search mode
Searching lexis nexis in power search mode
Joyce Johnston772 views
Analyzing social conversation: a guide to data mining and data visualization by Tempero UK
Analyzing social conversation: a guide to data mining and data visualization Analyzing social conversation: a guide to data mining and data visualization
Analyzing social conversation: a guide to data mining and data visualization
Tempero UK831 views
Business Models in the Data Economy: A Case Study from the Business Partner D... by Boris Otto
Business Models in the Data Economy: A Case Study from the Business Partner D...Business Models in the Data Economy: A Case Study from the Business Partner D...
Business Models in the Data Economy: A Case Study from the Business Partner D...
Boris Otto1.5K views
Data Driven PR: 8 Steps to Building Media Attention with Research by WalkerSands
Data Driven PR: 8 Steps to Building Media Attention with ResearchData Driven PR: 8 Steps to Building Media Attention with Research
Data Driven PR: 8 Steps to Building Media Attention with Research
WalkerSands646 views
Digital Winners 2013: Aleksander stensby by Telenor Group
Digital Winners 2013: Aleksander stensbyDigital Winners 2013: Aleksander stensby
Digital Winners 2013: Aleksander stensby
Telenor Group405 views
Unleashing Twitter Data for Fun and Insight by Matthew Russell
Unleashing Twitter Data for Fun and InsightUnleashing Twitter Data for Fun and Insight
Unleashing Twitter Data for Fun and Insight
Matthew Russell2.2K views
Designing a Better Planet with Big Data and Sensor Networks (for Intelligent ... by Rainer Sternfeld
Designing a Better Planet with Big Data and Sensor Networks (for Intelligent ...Designing a Better Planet with Big Data and Sensor Networks (for Intelligent ...
Designing a Better Planet with Big Data and Sensor Networks (for Intelligent ...
Rainer Sternfeld4.2K views
Text Analytics: Yesterday, Today and Tomorrow by Tony Russell-Rose
Text Analytics: Yesterday, Today and TomorrowText Analytics: Yesterday, Today and Tomorrow
Text Analytics: Yesterday, Today and Tomorrow
Tony Russell-Rose4.4K views

Similar to Data Visualization at Twitter

Unleashing twitter data for fun and insight by
Unleashing twitter data for fun and insightUnleashing twitter data for fun and insight
Unleashing twitter data for fun and insightDigital Reasoning
1.3K views132 slides
What I tell myself before visualizing by
What I tell myself before visualizingWhat I tell myself before visualizing
What I tell myself before visualizingKrist Wongsuphasawat
26 views138 slides
Working With Facebook, Twitter, et al. - Social Media Camp by
Working With Facebook, Twitter, et al. - Social Media CampWorking With Facebook, Twitter, et al. - Social Media Camp
Working With Facebook, Twitter, et al. - Social Media CampMike Anderson
334 views25 slides
Fundamentals for the New Era PR Pro with Sarah Evans by
Fundamentals for the New Era PR Pro with Sarah EvansFundamentals for the New Era PR Pro with Sarah Evans
Fundamentals for the New Era PR Pro with Sarah EvansCision
2.3K views35 slides
Linking Feral Event Data: IWMW 2009 Case Study by
Linking Feral Event Data: IWMW 2009 Case StudyLinking Feral Event Data: IWMW 2009 Case Study
Linking Feral Event Data: IWMW 2009 Case Studylisbk
665 views26 slides

Similar to Data Visualization at Twitter(20)

Unleashing twitter data for fun and insight by Digital Reasoning
Unleashing twitter data for fun and insightUnleashing twitter data for fun and insight
Unleashing twitter data for fun and insight
Digital Reasoning1.3K views
Working With Facebook, Twitter, et al. - Social Media Camp by Mike Anderson
Working With Facebook, Twitter, et al. - Social Media CampWorking With Facebook, Twitter, et al. - Social Media Camp
Working With Facebook, Twitter, et al. - Social Media Camp
Mike Anderson334 views
Fundamentals for the New Era PR Pro with Sarah Evans by Cision
Fundamentals for the New Era PR Pro with Sarah EvansFundamentals for the New Era PR Pro with Sarah Evans
Fundamentals for the New Era PR Pro with Sarah Evans
Cision2.3K views
Linking Feral Event Data: IWMW 2009 Case Study by lisbk
Linking Feral Event Data: IWMW 2009 Case StudyLinking Feral Event Data: IWMW 2009 Case Study
Linking Feral Event Data: IWMW 2009 Case Study
lisbk665 views
Surfacing Real-World Event Content on Twitter by Hila Becker
Surfacing Real-World Event Content on TwitterSurfacing Real-World Event Content on Twitter
Surfacing Real-World Event Content on Twitter
Hila Becker894 views
John Conroy by blogtalk
John ConroyJohn Conroy
John Conroy
blogtalk433 views
#interactives at Twitter by philogb
#interactives at Twitter#interactives at Twitter
#interactives at Twitter
philogb2.3K views
#interactives at Twitter by philogb
#interactives at Twitter#interactives at Twitter
#interactives at Twitter
philogb869 views
Twitter recruiting McGill Sept 2013 by Philip Youssef
Twitter recruiting McGill Sept 2013Twitter recruiting McGill Sept 2013
Twitter recruiting McGill Sept 2013
Philip Youssef6.4K views
TapIntoTwitter NYC June 2019 - Twitter Developer Labs by Twitter Developers
TapIntoTwitter NYC June 2019 - Twitter Developer LabsTapIntoTwitter NYC June 2019 - Twitter Developer Labs
TapIntoTwitter NYC June 2019 - Twitter Developer Labs
Twitter Developers332 views
Twitter - Measuring Reach and Engagement by Sarah Baughman
Twitter - Measuring Reach and EngagementTwitter - Measuring Reach and Engagement
Twitter - Measuring Reach and Engagement
Sarah Baughman966 views
Data Engineering 101: Building your first data product by Jonathan Dinu PyDat... by PyData
Data Engineering 101: Building your first data product by Jonathan Dinu PyDat...Data Engineering 101: Building your first data product by Jonathan Dinu PyDat...
Data Engineering 101: Building your first data product by Jonathan Dinu PyDat...
PyData4.7K views
Conducting Twitter Reserch by Kim Holmberg
Conducting Twitter ReserchConducting Twitter Reserch
Conducting Twitter Reserch
Kim Holmberg2.7K views
Apps For Journalists by aschweigert
Apps For JournalistsApps For Journalists
Apps For Journalists
aschweigert631 views

More from Krist Wongsuphasawat

Navigating the Wide World of Data Visualization Libraries by
Navigating the Wide World of Data Visualization LibrariesNavigating the Wide World of Data Visualization Libraries
Navigating the Wide World of Data Visualization LibrariesKrist Wongsuphasawat
2K views72 slides
Encodable: Configurable Grammar for Visualization Components by
Encodable: Configurable Grammar for Visualization ComponentsEncodable: Configurable Grammar for Visualization Components
Encodable: Configurable Grammar for Visualization ComponentsKrist Wongsuphasawat
451 views79 slides
ร้อยเรื่องราวจากข้อมูล / Storytelling with Data by
ร้อยเรื่องราวจากข้อมูล / Storytelling with Dataร้อยเรื่องราวจากข้อมูล / Storytelling with Data
ร้อยเรื่องราวจากข้อมูล / Storytelling with DataKrist Wongsuphasawat
630 views126 slides
Reveal the talking points of every episode of Game of Thrones from fans' conv... by
Reveal the talking points of every episode of Game of Thrones from fans' conv...Reveal the talking points of every episode of Game of Thrones from fans' conv...
Reveal the talking points of every episode of Game of Thrones from fans' conv...Krist Wongsuphasawat
1.1K views94 slides
d3Kit by
d3Kitd3Kit
d3KitKrist Wongsuphasawat
3.1K views20 slides
A Narrative Display for Sports Tournament Recap by
A Narrative Display for Sports Tournament RecapA Narrative Display for Sports Tournament Recap
A Narrative Display for Sports Tournament RecapKrist Wongsuphasawat
1.8K views16 slides

More from Krist Wongsuphasawat(18)

Navigating the Wide World of Data Visualization Libraries by Krist Wongsuphasawat
Navigating the Wide World of Data Visualization LibrariesNavigating the Wide World of Data Visualization Libraries
Navigating the Wide World of Data Visualization Libraries
Encodable: Configurable Grammar for Visualization Components by Krist Wongsuphasawat
Encodable: Configurable Grammar for Visualization ComponentsEncodable: Configurable Grammar for Visualization Components
Encodable: Configurable Grammar for Visualization Components
ร้อยเรื่องราวจากข้อมูล / Storytelling with Data by Krist Wongsuphasawat
ร้อยเรื่องราวจากข้อมูล / Storytelling with Dataร้อยเรื่องราวจากข้อมูล / Storytelling with Data
ร้อยเรื่องราวจากข้อมูล / Storytelling with Data
Reveal the talking points of every episode of Game of Thrones from fans' conv... by Krist Wongsuphasawat
Reveal the talking points of every episode of Game of Thrones from fans' conv...Reveal the talking points of every episode of Game of Thrones from fans' conv...
Reveal the talking points of every episode of Game of Thrones from fans' conv...
Krist Wongsuphasawat's Dissertation Proposal Slides: Interactive Exploration ... by Krist Wongsuphasawat
Krist Wongsuphasawat's Dissertation Proposal Slides: Interactive Exploration ...Krist Wongsuphasawat's Dissertation Proposal Slides: Interactive Exploration ...
Krist Wongsuphasawat's Dissertation Proposal Slides: Interactive Exploration ...
Outflow: Exploring Flow, Factors and Outcome of Temporal Event Sequences by Krist Wongsuphasawat
Outflow: Exploring Flow, Factors and Outcome of Temporal Event SequencesOutflow: Exploring Flow, Factors and Outcome of Temporal Event Sequences
Outflow: Exploring Flow, Factors and Outcome of Temporal Event Sequences
Information Visualization for Knowledge Discovery by Krist Wongsuphasawat
Information Visualization for Knowledge DiscoveryInformation Visualization for Knowledge Discovery
Information Visualization for Knowledge Discovery
Krist Wongsuphasawat's Dissertation Defense: Interactive Exploration of Tempo... by Krist Wongsuphasawat
Krist Wongsuphasawat's Dissertation Defense: Interactive Exploration of Tempo...Krist Wongsuphasawat's Dissertation Defense: Interactive Exploration of Tempo...
Krist Wongsuphasawat's Dissertation Defense: Interactive Exploration of Tempo...
LifeFlow: Understanding Millions of Event Sequences in a Million Pixels by Krist Wongsuphasawat
LifeFlow: Understanding Millions of Event Sequences in a Million PixelsLifeFlow: Understanding Millions of Event Sequences in a Million Pixels
LifeFlow: Understanding Millions of Event Sequences in a Million Pixels
Krist Wongsuphasawat11.2K views
Information Visualization for Knowledge Discovery: An Introduction by Krist Wongsuphasawat
Information Visualization for Knowledge Discovery: An IntroductionInformation Visualization for Knowledge Discovery: An Introduction
Information Visualization for Knowledge Discovery: An Introduction
Finding Comparable Temporal Categorical Records: A Similarity Measure with an... by Krist Wongsuphasawat
Finding Comparable Temporal Categorical Records: A Similarity Measure with an...Finding Comparable Temporal Categorical Records: A Similarity Measure with an...
Finding Comparable Temporal Categorical Records: A Similarity Measure with an...
Outflow: Visualizing Patients Flow by Symptoms & Outcome by Krist Wongsuphasawat
Outflow: Visualizing Patients Flow by Symptoms & OutcomeOutflow: Visualizing Patients Flow by Symptoms & Outcome
Outflow: Visualizing Patients Flow by Symptoms & Outcome

Recently uploaded

PTicketInput.pdf by
PTicketInput.pdfPTicketInput.pdf
PTicketInput.pdfstuartmcphersonflipm
376 views1 slide
JConWorld_ Continuous SQL with Kafka and Flink by
JConWorld_ Continuous SQL with Kafka and FlinkJConWorld_ Continuous SQL with Kafka and Flink
JConWorld_ Continuous SQL with Kafka and FlinkTimothy Spann
100 views36 slides
Data structure and algorithm. by
Data structure and algorithm. Data structure and algorithm.
Data structure and algorithm. Abdul salam
18 views24 slides
Organic Shopping in Google Analytics 4.pdf by
Organic Shopping in Google Analytics 4.pdfOrganic Shopping in Google Analytics 4.pdf
Organic Shopping in Google Analytics 4.pdfGA4 Tutorials
10 views13 slides
Building Real-Time Travel Alerts by
Building Real-Time Travel AlertsBuilding Real-Time Travel Alerts
Building Real-Time Travel AlertsTimothy Spann
109 views48 slides
Launch of the Knowledge Exchange Platform - Romina Boarini - 21 November 2023 by
Launch of the Knowledge Exchange Platform - Romina Boarini - 21 November 2023Launch of the Knowledge Exchange Platform - Romina Boarini - 21 November 2023
Launch of the Knowledge Exchange Platform - Romina Boarini - 21 November 2023StatsCommunications
55 views5 slides

Recently uploaded(20)

JConWorld_ Continuous SQL with Kafka and Flink by Timothy Spann
JConWorld_ Continuous SQL with Kafka and FlinkJConWorld_ Continuous SQL with Kafka and Flink
JConWorld_ Continuous SQL with Kafka and Flink
Timothy Spann100 views
Data structure and algorithm. by Abdul salam
Data structure and algorithm. Data structure and algorithm.
Data structure and algorithm.
Abdul salam 18 views
Organic Shopping in Google Analytics 4.pdf by GA4 Tutorials
Organic Shopping in Google Analytics 4.pdfOrganic Shopping in Google Analytics 4.pdf
Organic Shopping in Google Analytics 4.pdf
GA4 Tutorials10 views
Building Real-Time Travel Alerts by Timothy Spann
Building Real-Time Travel AlertsBuilding Real-Time Travel Alerts
Building Real-Time Travel Alerts
Timothy Spann109 views
Launch of the Knowledge Exchange Platform - Romina Boarini - 21 November 2023 by StatsCommunications
Launch of the Knowledge Exchange Platform - Romina Boarini - 21 November 2023Launch of the Knowledge Exchange Platform - Romina Boarini - 21 November 2023
Launch of the Knowledge Exchange Platform - Romina Boarini - 21 November 2023
Cross-network in Google Analytics 4.pdf by GA4 Tutorials
Cross-network in Google Analytics 4.pdfCross-network in Google Analytics 4.pdf
Cross-network in Google Analytics 4.pdf
GA4 Tutorials6 views
Introduction to Microsoft Fabric.pdf by ishaniuudeshika
Introduction to Microsoft Fabric.pdfIntroduction to Microsoft Fabric.pdf
Introduction to Microsoft Fabric.pdf
ishaniuudeshika24 views
Supercharging your Data with Azure AI Search and Azure OpenAI by Peter Gallagher
Supercharging your Data with Azure AI Search and Azure OpenAISupercharging your Data with Azure AI Search and Azure OpenAI
Supercharging your Data with Azure AI Search and Azure OpenAI
Peter Gallagher37 views
Survey on Factuality in LLM's.pptx by NeethaSherra1
Survey on Factuality in LLM's.pptxSurvey on Factuality in LLM's.pptx
Survey on Factuality in LLM's.pptx
NeethaSherra15 views
Understanding Hallucinations in LLMs - 2023 09 29.pptx by Greg Makowski
Understanding Hallucinations in LLMs - 2023 09 29.pptxUnderstanding Hallucinations in LLMs - 2023 09 29.pptx
Understanding Hallucinations in LLMs - 2023 09 29.pptx
Greg Makowski13 views
Chapter 3b- Process Communication (1) (1)(1) (1).pptx by ayeshabaig2004
Chapter 3b- Process Communication (1) (1)(1) (1).pptxChapter 3b- Process Communication (1) (1)(1) (1).pptx
Chapter 3b- Process Communication (1) (1)(1) (1).pptx
ayeshabaig20045 views
Short Story Assignment by Kelly Nguyen by kellynguyen01
Short Story Assignment by Kelly NguyenShort Story Assignment by Kelly Nguyen
Short Story Assignment by Kelly Nguyen
kellynguyen0118 views
3196 The Case of The East River by ErickANDRADE90
3196 The Case of The East River3196 The Case of The East River
3196 The Case of The East River
ErickANDRADE9011 views
Advanced_Recommendation_Systems_Presentation.pptx by neeharikasingh29
Advanced_Recommendation_Systems_Presentation.pptxAdvanced_Recommendation_Systems_Presentation.pptx
Advanced_Recommendation_Systems_Presentation.pptx

Data Visualization at Twitter

  • 1. visualization at Twitter data Krist Wongsuphasawat / @kristw
  • 3. Bangkok, Thailand Krist Wongsuphasawat / @kristw
  • 4. Computer Engineer Bangkok, Thailand Chulalongkorn University Krist Wongsuphasawat / @kristw
  • 5. Computer Engineer Bangkok, Thailand Programming + Soccer Krist Wongsuphasawat / @kristw
  • 6. Computer Engineer Bangkok, Thailand Programming + Soccer Krist Wongsuphasawat / @kristw
  • 7. Computer Engineer Bangkok, Thailand Programming + Soccer Krist Wongsuphasawat / @kristw
  • 8. Computer Engineer Bangkok, Thailand M.S. in Computer Science Univ. of Maryland Krist Wongsuphasawat / @kristw
  • 9. Computer Engineer Bangkok, Thailand PhD in Computer Science Univ. of Maryland Information Visualization Krist Wongsuphasawat / @kristw
  • 10. Computer Engineer Bangkok, Thailand PhD in Computer Science Univ. of Maryland Information Visualization Krist Wongsuphasawat / @kristw IBM Microsoft
  • 11. Computer Engineer Bangkok, Thailand PhD in Computer Science Univ. of Maryland Information Visualization Krist Wongsuphasawat / @kristw IBM Microsoft Sr. Data Visualization Scientist Twitter
  • 15. vis data at Twitter
  • 16. data at Twitter “Tweets”
  • 17. data at Twitter “Tweets” #events TV Shows New Year Earthquake Oscars Protest Super Bowl World Cup Election Breaking news …
  • 18. data at Twitter “Tweets” #events TV Shows New Year Earthquake Oscars Protest Super Bowl World Cup Election Breaking news … #curiosity Sleep pattern Human behavior Language …
  • 19. data at Twitter “Tweets” #events TV Shows New Year Earthquake Oscars Protest Super Bowl World Cup Election Breaking news … #curiosity Sleep pattern Human behavior Language … What could we learn from the Tweets?
  • 20. vis data at Twitter “Tweets” Tell stories about an event, Pursue curiosity or inspiration Goal:
  • 21. vis data at Twitter “Tweets” Tell stories about an event, Pursue curiosity or inspiration (with deadline) Goal:
  • 23. vis data at Twitter “Tweets” Get data 1
  • 24. easy?
  • 25. Having all Tweets How people think I feel.
  • 26. Having all Tweets How people think I feel. How I really feel.
  • 27. Challenges • Too much data • Want only relevant Tweets • hashtag: #BRA • keywords: “goal” • Need to aggregate & reduce size • Long processing time (hours)
  • 28. Hadoop Cluster Vertica Pig / Scalding (slow) SQL Data Storage Tool Workflow
  • 29. Hadoop Cluster Vertica Pig / Scalding (slow) SQL Data Storage Tool Workflow
  • 30. Workflow Hadoop Cluster Vertica Pig / Scalding (slow) SQL Data Storage Tool Your laptop Smaller dataset
  • 31. Hadoop Cluster Vertica Pig / Scalding (slow) SQL Data Storage Tool Tool node.js / python / excel (fast) Final dataset Your laptop Workflow Smaller dataset
  • 32. vis data at Twitter “Tweets” Get data 1 2 Visualize
  • 33. Visualize • Peek into data • Check data & test ideas • Decide how to visualize • Guided by data type • Choose tools • Start building
  • 34. Visualize • Peek into data • Check data & test ideas • Decide how to visualize • Guided by data type • Choose tools • Start building R d3 Tableau Yeoman
  • 35. (+ media) photos, videos data What? TEXT Where? When? GEO TIME
  • 36. Visualize Data What? TEXT Where? When? GEO TIME
  • 37. Visualize Data What? TEXT Where? When? GEO TIME
  • 40. Time Tweets/second + Annotation http://www.flickr.com/photos/twitteroffice/5681263084/
  • 41. Visualize Data What? TEXT Where? When? GEO TIME
  • 42. Geo Heatmap Low density High density
  • 43. Geo San Francisco flickr.com/photos/twitteroffice/8798020541 Low density High density
  • 44. Geo San Francisco Rebuild the world based on tweet volumes twitter.github.io/interactive/andes/
  • 45. Visualize Data What? TEXT Where? When? GEO TIME
  • 46. Text www.wordle.net Some experiments during World Cup
  • 47. Text www.wordle.net Word cloud of Tweets right after the 1st goal
  • 48. Text Word cloud of Tweets right after the 1st goal It was an “own” goal. www.wordle.net
  • 49. Text WordTree [Wattenberg & Viégas 2008] www.jasondavies.com/wordtree www.jasondavies.com/wordtree
  • 50. Visualize Data What? TEXT Where? When? GEO TIME
  • 51. Time + Geo Japan Earthquake 2011 blog.twitter.com/2011/global-pulse youtu.be/SybWjN9pKQk
  • 54. Time + Geo Tweet pattern [Rios & Lin 2012] Night Late night Daytime Night Late night Daytime
  • 55. Time + Geo Tweet pattern [Rios & Lin 2012] Night Late night Daytime Night Late night Daytime
  • 56. Time + Geo Tweet pattern [Rios & Lin 2012] Night Late night Daytime Night Late night Daytime
  • 57. Time + Geo Tweet pattern [Rios & Lin 2012] Night Late night Daytime Night Late night Daytime
  • 58. Visualize Data What? TEXT Where? When? GEO TIME
  • 59. Geo + Text Real-time Tweet map
  • 60. Geo + Text Real-time Tweet map
  • 61. Geo + Text Real-time Tweet map most frequent term
  • 62. Geo + Text Real-time Tweet map Gmail was down Jan 24, 2014
  • 63. Geo + Text Real-time Tweet map Nelson Mandela passed away Dec 5, 2013
  • 64. Visualize Data What? TEXT Where? When? GEO TIME
  • 65. Time + Text UEFA Champions League Biggest tournament for European soccer clubs Many Tweets during the matches
  • 66. UEFA Champions League Team 1 Team 2 Time + Text Dortmund Bayern Munich
  • 67. UEFA Champions League Team 1 Team 2 Time + Text Dortmund Bayern Munich
  • 68. UEFA Champions League Team 1 Team 2 Time + Text Dortmund Bayern Munich
  • 69. UEFA Champions League Team 1 Team 2 Dortmund Bayern Munich Count Tweets mentioning the teams every minute Time + Text
  • 70. Time + Text UEFA Champions League
  • 71. Time + Text UEFA Champions League + “goal” count + context
  • 72. + “offside” Time + Text UEFA Champions League
  • 73. + players Time + Text UEFA Champions League
  • 74. Competition Tree vs vs A B C D vs A C C
  • 75. Competition Tree vs vs A B C D vs + A C C
  • 76. Competition Tree vs vs A B C D vs + = A C C
  • 77. Visualize Data What? TEXT Where? When? GEO TIME
  • 78. Time + Text + Geo State of the Union twitter.github.io/interactive/sotu2014
  • 79. Time + Text + Geo State of the Union 1) timeline + topic from Tweets 4) Density map of Tweets about selected topic 3) Volume of Tweets by topics during selected part of the SOTU 2) context (speech) twitter.github.io/interactive/sotu2014
  • 80. Time + Text World Cup 2014
  • 81. Time + Text + Geo World Cup 2014
  • 82. Visualize Data What? TEXT Where? When? GEO TIME
  • 83. Visualize Data What? TEXT + Where? When? GEO TIME Non-Twitter data CONTEXT
  • 84. Time + Text New Year 2014
  • 85. Time + Text New Year 2014
  • 86. Time + Text + Geo (c) New Year 2014 twitter.github.io/interactive/newyear2014/
  • 87. vis data at Twitter “Tweets” Get data 1 2 Visualize
  • 88. vis data at Twitter “Tweets” Get data 1 2 Visualize Evaluate 3
  • 89. vis data at Twitter “Tweets” Get data 1 2 Visualize Evaluate 3 Iterate!
  • 90. Evaluation • Self • Peer feedback • Non team members / Potential audience
  • 91. vis data at Twitter Get data 1 2 Visualize Evaluate 3
  • 92. vis data at Twitter Get data 1 2 Visualize Evaluate 3 big data => small data
  • 93. vis data at Twitter Get data 1 2 Visualize Evaluate 3 big data => small data What? Where? When?
  • 94. big data => small data self, peer, external vis data at Twitter Get data 1 2 Visualize Evaluate 3 What? Where? When?
  • 95. big data => small data self, peer, external vis data at Twitter “Tweets” Get data 1 2 Visualize Evaluate 3 What? Where? When?
  • 96. big data => small data self, peer, external vis data at Twitter “Tweets” Get data 1 2 Visualize Evaluate 3 What? Where? When? • users • followers graph • logs • etc. ! • derived data: language, sentiment
  • 97. big data => small data self, peer, external vis data at Twitter “Tweets” Get data 1 2 Visualize Evaluate 3 What? Where? When? • users Who? … • followers graph • logs • etc. ! • derived data: language, sentiment
  • 98. big data => small data self, peer, external vis data at Twitter “Tweets” Get data 1 2 Visualize Evaluate 3 What? Where? When? • users Who? … • followers graph • logs • etc. ! • derived data: language, sentiment (with deadline)
  • 99. big data => small data self, peer, external vis data at Twitter “Tweets” Get data 1 2 Visualize Evaluate 3 What? Where? When? • users Who? … • followers graph • logs • etc. (with deadline) ! • derived data: language, sentiment @kristw / https://interactive.twitter.com
  • 100. big data => small data self, peer, external vis data at Twitter “Tweets” Get data 1 2 Visualize Evaluate 3 What? Where? When? • users Who? … • followers graph • logs • etc. (with deadline) ! • derived data: language, sentiment @kristw / https://interactive.twitter.com + visualizations by @philogb, @miguelrios & @trebor
  • 102. big data => small data self, peer, external vis data at Twitter “Tweets” Get data 1 2 Visualize Evaluate 3 What? Where? When? • users Who? … • followers graph • logs • etc. (with deadline) @kristw / https://interactive.twitter.com + visualizations by @philogb, @miguelrios & @trebor