This document discusses applications of analyzing Twitter data for business and research purposes. It provides an overview of Twitter, including its size, growth, and user demographics. It then discusses how to acquire Twitter data through the Twitter search feature or Twitter APIs. Finally, it outlines several potential applications of Twitter data analysis, including social network modeling, predictive modeling for business, brand sentiment analysis, and limitations around demographics and spam.

1. Applications of Twitter Data Extraction For Business and Research John Conroy [email_address]

2. This talk… Twitter 101 Twitter data: Open, Plentiful, Real-time Twitter Size, Growth & User Profile Acquiring Twitter Data (the easy & the hard way) Applications of Twitter data analysis Business Research Limitations of Twitter data – demographics and spam

3. Twitter 101 Post short messages Follow Other Users Messages (tweets) can contain hyperlinks i.e. subscribe to see their tweets when you log in 1

4. Twitter 101

5. Twitter 101 Retweets, at/replies…

6. Twitter Data is Open, Plentiful, Real-time Open attitude to data: most users’ tweets are public (>90%) Channels: API, Twitter Search Data is plentiful: ~100m tweets per day by Nov. ’10 Data is real-time: 140 char posts + retweets = wildfire dissemination of news & viral content

7. Iran protests ‘09: retweets

8. Twitter Size, Growth Size: 105m users by April ’10 2.1m new users per week 600m search queries/day Williams(CEO), Chirp, April ‘10 User growth: 155% p.a. Daily tweets growing at 550% p.a ~100m tweets per day by Nov. ‘10 Conroy/Griffith June ’10 (6 months data)

9. Twitter User-Profile Even Male-Female split Brand knowledge now ubiquitous 1/6 as many users as Facebook Age: 1/3 between 25-34 years old Better educated, earn more http://www.edisonresearch.com/twitter_usage_2010.php Edison research (U.S.-oriented research)

10. Twitter users: Age http://www.edisonresearch.com/twitter_usage_2010.php Edison research (U.S.-oriented research) Twitter Users Profile

11. Acquiring Twitter Data Twitter Search http://search.twitter.com For anybody The Easy Way The Hard Way Twitter APIs REST, Search, Streaming APIs Code (Python/PHP/Java etc…)

12. Acquiring Twitter Data- Twitter Search

13. Acquiring Twitter Data- Twitter Search

14. Things to do with Twitter Search Acquiring Twitter Data- Twitter Search Find business opportunities Intel on competitors Community-building: Answer “Does anyone know…?” queries in your segment Find gripes/compliments on your service Find anything else people are saying about you … etc…

15. Acquiring Data from Twitter APIs REST api – find out about users – how many friends, how often they tweet, get last N tweets, are they active etc. SEARCH api – programmatic access to Twitter search STREAMING api – ‘firehose’ of tweets from everyone

16. What can we do with this data? Model the social graph of sub groups: find most-influential users (retweets, replies, follower/friend quotient) Eg Modelling the Irish Twittersphere (Conroy, Griffith, 2010) Find the ‘true’ social graph described by conversations, find authoritative users, broadcasters User engagement metrics (how often they tweet etc.) Find similar users based on graph theory Study viral news propagation through this sub-group Find super-users (with a view to engaging them) Acquiring Data from Twitter APIs

17. Irish users: time since last tweet (c.23k users) Acquiring Data from Twitter APIs

18. Most replied-to by Irish users Feb-March ’10 93k replies from 23k users Also c.7k retweets (?) Acquiring Data from Twitter APIs

19. Predictive Modelling Business Intelligence Non-Twitter example: satellite images of Wal-Mart car-parks to predict earnings – smart but expensive! Acquiring Data from Twitter APIs

20. Using Twitter for Predictive Modelling Eg 1: holiday destinations Acquiring Data from Twitter APIs

21. Eg 2: Movie pre-launch “Buzz” & marketing budget (not real figures!) Acquiring Data from Twitter APIs – Predictive Modelling

22. Brand Sentiment Analysis Sentiment analysis of Super Bowl commercials 2010 Conroy and Griffith, 2010 300k tweets collected during the game Probabilistic classification models & machine learning Naïve Bayes, Maximum Entropy, (S.V.M.) Try to find out which were the most popular commercials Hard!! Human language is complex… Acquiring Data from Twitter APIs

23. Sentiment Analysis of Superbowl Commercials: Results Acquiring Data from Twitter APIs Note: initial manual verification of these results shows disappointing results… the research continues 

24. What else can we do with this data? Acquiring Data from Twitter APIs

25. Limitations of Twitter Data Twitter < Facebook for “knowing your customer” Facebook has demographics- age, sex etc Demographic skewed towards 25-34 yr olds & tech-savvy- not ubiquitous Spam: The game can be rigged