Applications of Twitter Data Extraction For Business and Research John Conroy [email_address]
This talk… <ul><li>Twitter 101 </li></ul><ul><li>Twitter data: Open, Plentiful, Real-time </li></ul><ul><li>Twitter Size, ...
Twitter 101 <ul><li>Post short messages </li></ul><ul><li>Follow Other Users </li></ul><ul><li>Messages (tweets) can conta...
Twitter 101
Twitter 101 Retweets, at/replies…
Twitter Data is Open, Plentiful, Real-time <ul><li>Open attitude to data: most users’ tweets are public (>90%) </li></ul><...
<ul><li>Iran protests ‘09: </li></ul><ul><ul><li>retweets </li></ul></ul>
Twitter Size, Growth <ul><li>Size: 105m users by April ’10 </li></ul><ul><li>2.1m new users per week </li></ul><ul><li>600...
Twitter User-Profile <ul><li>Even Male-Female split </li></ul><ul><li>Brand knowledge now ubiquitous </li></ul><ul><li>1/6...
Twitter users: Age http://www.edisonresearch.com/twitter_usage_2010.php Edison research (U.S.-oriented research) Twitter U...
Acquiring Twitter Data <ul><li>Twitter Search </li></ul><ul><ul><li>http://search.twitter.com </li></ul></ul><ul><ul><li>F...
Acquiring Twitter Data- Twitter Search
Acquiring Twitter Data- Twitter Search
<ul><li>Things to do with Twitter Search </li></ul>Acquiring Twitter Data- Twitter Search <ul><li>Find business opportunit...
Acquiring Data from Twitter APIs <ul><li>REST api –  find out about users – how many friends, how often they tweet, get la...
What can we do with this data? <ul><li>Model the social graph of sub groups: find most-influential users (retweets, replie...
<ul><li>Irish users: time since last tweet (c.23k users) </li></ul>Acquiring Data from Twitter APIs
<ul><li>Most replied-to by Irish users </li></ul><ul><li>Feb-March ’10 </li></ul><ul><li>93k replies from 23k users </li><...
Predictive Modelling <ul><li>Business Intelligence </li></ul><ul><li>Non-Twitter example: satellite images of Wal-Mart car...
Using Twitter for Predictive Modelling <ul><li>Eg 1: holiday destinations  </li></ul>Acquiring Data from Twitter APIs
<ul><li>Eg 2: Movie pre-launch “Buzz” & marketing budget (not real figures!) </li></ul>Acquiring Data from Twitter APIs – ...
Brand Sentiment Analysis <ul><li>Sentiment analysis of Super Bowl commercials 2010  Conroy and Griffith, 2010 </li></ul><u...
Sentiment Analysis of Superbowl Commercials: Results Acquiring Data from Twitter APIs Note: initial manual verification of...
What else can we do with this data? Acquiring Data from Twitter APIs
Limitations of Twitter Data <ul><li>Twitter < Facebook for “knowing your customer” </li></ul><ul><ul><li>Facebook has demo...
Upcoming SlideShare
Loading in …5
×

John Conroy

761 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
761
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

John Conroy

  1. 1. Applications of Twitter Data Extraction For Business and Research John Conroy [email_address]
  2. 2. This talk… <ul><li>Twitter 101 </li></ul><ul><li>Twitter data: Open, Plentiful, Real-time </li></ul><ul><li>Twitter Size, Growth & User Profile </li></ul><ul><li>Acquiring Twitter Data (the easy & the hard way) </li></ul><ul><li>Applications of Twitter data analysis </li></ul><ul><ul><li>Business </li></ul></ul><ul><ul><li>Research </li></ul></ul><ul><li>Limitations of Twitter data – demographics and spam </li></ul>
  3. 3. Twitter 101 <ul><li>Post short messages </li></ul><ul><li>Follow Other Users </li></ul><ul><li>Messages (tweets) can contain hyperlinks </li></ul><ul><li>i.e. subscribe to see their tweets when you log in </li></ul><ul><li>1 </li></ul>
  4. 4. Twitter 101
  5. 5. Twitter 101 Retweets, at/replies…
  6. 6. Twitter Data is Open, Plentiful, Real-time <ul><li>Open attitude to data: most users’ tweets are public (>90%) </li></ul><ul><ul><ul><ul><li>Channels: API, Twitter Search </li></ul></ul></ul></ul><ul><li>Data is plentiful: ~100m tweets per day by Nov. ’10 </li></ul><ul><li>Data is real-time: </li></ul><ul><ul><ul><ul><li>140 char posts + retweets = wildfire dissemination of news & viral content </li></ul></ul></ul></ul>
  7. 7. <ul><li>Iran protests ‘09: </li></ul><ul><ul><li>retweets </li></ul></ul>
  8. 8. Twitter Size, Growth <ul><li>Size: 105m users by April ’10 </li></ul><ul><li>2.1m new users per week </li></ul><ul><li>600m search queries/day </li></ul><ul><ul><ul><ul><ul><li>Williams(CEO), Chirp, April ‘10 </li></ul></ul></ul></ul></ul><ul><li>User growth: 155% p.a. </li></ul><ul><li>Daily tweets growing at 550% p.a </li></ul><ul><li>~100m tweets per day by Nov. ‘10 </li></ul><ul><ul><ul><ul><ul><li>Conroy/Griffith June ’10 (6 months data) </li></ul></ul></ul></ul></ul>
  9. 9. Twitter User-Profile <ul><li>Even Male-Female split </li></ul><ul><li>Brand knowledge now ubiquitous </li></ul><ul><li>1/6 as many users as Facebook </li></ul><ul><li>Age: 1/3 between 25-34 years old </li></ul><ul><li>Better educated, earn more </li></ul>http://www.edisonresearch.com/twitter_usage_2010.php Edison research (U.S.-oriented research)
  10. 10. Twitter users: Age http://www.edisonresearch.com/twitter_usage_2010.php Edison research (U.S.-oriented research) Twitter Users Profile
  11. 11. Acquiring Twitter Data <ul><li>Twitter Search </li></ul><ul><ul><li>http://search.twitter.com </li></ul></ul><ul><ul><li>For anybody </li></ul></ul><ul><li>The Easy Way </li></ul><ul><li>The Hard Way </li></ul><ul><li>Twitter APIs </li></ul><ul><ul><li>REST, Search, Streaming APIs </li></ul></ul><ul><ul><li>Code (Python/PHP/Java etc…) </li></ul></ul>
  12. 12. Acquiring Twitter Data- Twitter Search
  13. 13. Acquiring Twitter Data- Twitter Search
  14. 14. <ul><li>Things to do with Twitter Search </li></ul>Acquiring Twitter Data- Twitter Search <ul><li>Find business opportunities </li></ul><ul><li>Intel on competitors </li></ul><ul><li>Community-building: Answer “Does anyone know…?” queries in your segment </li></ul><ul><li>Find gripes/compliments on your service </li></ul><ul><li>Find anything else people are saying about you </li></ul><ul><li>… etc… </li></ul>
  15. 15. Acquiring Data from Twitter APIs <ul><li>REST api – find out about users – how many friends, how often they tweet, get last N tweets, are they active etc. </li></ul><ul><li>SEARCH api – programmatic access to Twitter search </li></ul><ul><li>STREAMING api – ‘firehose’ of tweets from everyone </li></ul>
  16. 16. What can we do with this data? <ul><li>Model the social graph of sub groups: find most-influential users (retweets, replies, follower/friend quotient) </li></ul><ul><li>Eg Modelling the Irish Twittersphere (Conroy, Griffith, 2010) </li></ul><ul><ul><ul><li>Find the ‘true’ social graph described by conversations, find authoritative users, broadcasters </li></ul></ul></ul><ul><ul><ul><li>User engagement metrics (how often they tweet etc.) </li></ul></ul></ul><ul><ul><ul><li>Find similar users based on graph theory </li></ul></ul></ul><ul><ul><ul><li>Study viral news propagation through this sub-group </li></ul></ul></ul><ul><ul><ul><li>Find super-users (with a view to engaging them) </li></ul></ul></ul>Acquiring Data from Twitter APIs
  17. 17. <ul><li>Irish users: time since last tweet (c.23k users) </li></ul>Acquiring Data from Twitter APIs
  18. 18. <ul><li>Most replied-to by Irish users </li></ul><ul><li>Feb-March ’10 </li></ul><ul><li>93k replies from 23k users </li></ul><ul><li>Also c.7k retweets (?) </li></ul>Acquiring Data from Twitter APIs
  19. 19. Predictive Modelling <ul><li>Business Intelligence </li></ul><ul><li>Non-Twitter example: satellite images of Wal-Mart car-parks to predict earnings – smart but expensive! </li></ul>Acquiring Data from Twitter APIs
  20. 20. Using Twitter for Predictive Modelling <ul><li>Eg 1: holiday destinations </li></ul>Acquiring Data from Twitter APIs
  21. 21. <ul><li>Eg 2: Movie pre-launch “Buzz” & marketing budget (not real figures!) </li></ul>Acquiring Data from Twitter APIs – Predictive Modelling
  22. 22. Brand Sentiment Analysis <ul><li>Sentiment analysis of Super Bowl commercials 2010 Conroy and Griffith, 2010 </li></ul><ul><ul><li>300k tweets collected during the game </li></ul></ul><ul><ul><li>Probabilistic classification models & machine learning </li></ul></ul><ul><ul><ul><ul><li>Naïve Bayes, Maximum Entropy, (S.V.M.) </li></ul></ul></ul></ul><ul><ul><li>Try to find out which were the most popular commercials </li></ul></ul><ul><ul><li>Hard!! Human language is complex… </li></ul></ul>Acquiring Data from Twitter APIs
  23. 23. Sentiment Analysis of Superbowl Commercials: Results Acquiring Data from Twitter APIs Note: initial manual verification of these results shows disappointing results… the research continues 
  24. 24. What else can we do with this data? Acquiring Data from Twitter APIs
  25. 25. Limitations of Twitter Data <ul><li>Twitter < Facebook for “knowing your customer” </li></ul><ul><ul><li>Facebook has demographics- age, sex etc </li></ul></ul><ul><li>Demographic skewed towards 25-34 yr olds & tech-savvy- not ubiquitous </li></ul><ul><li>Spam: The game can be rigged </li></ul>

×