Are Twitter Users Equal in           Predicting Elections?A Study of User Groups in Predicting 2012 U.S. Republican Presid...
There is a surge of interest in building systems that harness thepower of social data to predict election results.        ...
One problem seems to be ignored: Are social media users equal   in predicting elections?  They may be from different count...
o We Studied different groups of                       social media users who engage in                       the discussi...
User Categorization                   2. Tweet Mode 3. Content Type                                                       ...
1    More than half of the users posted only one tweet. Only 8% of the    users posted more than 10 tweets.     A small ...
According to users preference on generating their tweets, i.e., tweet mode, we   classified the users as original tweet-do...
We use target-specific sentiment analysis techniques to classify each tweet as   positive or negative – whether the expres...
We collected a set of Twitter users with known political preference from Twellow(http://www.twellow.com/categories/politic...
 We utilized the background knowledge from LinkedGeoData to identify thestates from user location information. If the us...
Predicting a Users Vote• Basic idea: for which candidate the user shows the most support   – Frequent mentions            ...
Prediction ResultsWe examine the predictive power of different user groups in predicting theresults of Super Tuesday races...
7    The prediction accuracy:     Engagement Degree: High > Low or Very Low    Tweet Mode: Original Tweet-Prone >Retweet...
Revealing the challenge of                                                              Retweets may not necessarily8   id...
Our findingsTwitter users are not “equal”  in predicting elections!  The likely voters’ opinions matter more.  Some users’...
More Work need to be                                 done…                                 • Identifying likely/actual vot...
It is actually about tracking public opinion.          PollingorSocial Media Analysis?                 1. Sample size     ...
1   Sample Size         Polling                                       Social Media Analysis    Thousands of people        ...
2             Representative of the Target Population                       Polling                                       ...
3      Accurate measure of opinions            Polling                                       Social Media Analysis Ask pe...
4       Timeliness             Polling                                       Social Media Analysis    Not be able to track...
Social Media Analysis – Promising but VeryChallenging                                                          Extracting...
Our Twitris+ System kept tracking      people’s opinion on 2012 U.S.Presidential Election in real time and this  is what w...
/t                                        The screenshots of Twitris+ were taken on Nov. 6th 6 PM ESTSubjective Informatio...
Twitris+: http://twitris.knoesis.org/                                                                            Select ev...
 A key innovation in sentiment analysis, employed in Twitris+, is topic specific sentimentanalysis -- to associate sentim...
Twitris+ Insights in 2012 Presidential DebatesHow was Obama doing in the first debate?          Are Twitter Users Equal in...
How was Obama doing in the second debate?                                Red Color: Negative Topics                       ...
Obama vsRomney in the third debate                                                                                    Obam...
Thank you !                        More about this study:         http://wiki.knoesis.org/index.php/ElectionPrediction    ...
Upcoming SlideShare
Loading in...5
×

Are Twitter Users Equal in Predicting Elections? Insights from Republican Primaries and 2012 General Election

1,200

Published on

Lu Chen, Wenbo Wang, Amit Sheth. Are Twitter Users Equal in Predicting Elections? A Study of User Groups in Predicting 2012 U.S. Republican Presidential Primaries. The 4th International Conference on Social Informatics (SocInfo2012), December 5-8, 2012, Lausanne, Switzerland.
http://knoesis.org/library/resource.php?id=1787

Published in: Education
1 Comment
3 Likes
Statistics
Notes
No Downloads
Views
Total Views
1,200
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
16
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide
  • Paper at: http://knoesis.org/library/resource.php?id=1787
  • Tweet volume alone may not be a reliable predictor, since a small group of users can produce a large amount of tweets. E.g., political campaign, promotion tweets
  • Some of the Twellow preferences are self declared
  • There is very strong correlation between the number of Twitter users/tweets from each state and the population of each state. Usually the Pearson's correlation coefficient between 0.9 to 1.0 indicates Very strong correlation.
  • Categorized by engagement degree: the high engagement users achieved better prediction results. It may be due to two reasons. (1) high engagement users posted more tweets. It is more reliable to make the prediction using more tweets. (2) more engaged users were more involved in the election event, and were more likely to vote.Categorized by tweet mode: the original tweet prone users achieved better prediction results. It might suggest the difficulty of identifying users' voting intent from retweets.Categorized by content type: No significant difference is found between two groupsCategorized by political preference: the right-leaning user group achieved significantly better results than left-leaning group.
  • Add
  • Are Twitter Users Equal in Predicting Elections? Insights from Republican Primaries and 2012 General Election

    1. 1. Are Twitter Users Equal in Predicting Elections?A Study of User Groups in Predicting 2012 U.S. Republican Presidential Primaries (with additional insights into the 2012 General Election) Lu Chen Wenbo Wang Amit Sheth chen@knoesis.org wenbo@knoesis.org amit@knoesis.org Ohio Center of Excellent in Knowledge-enabled Computing (Kno.e.sis) Wright State University, Dayton, OH, USA Lu Chen, Wenbo Wang, Amit Sheth. Are Twitter Users Equal in Predicting Elections? A Study of User Groups in Predicting 2012 U.S. Republican Presidential Primaries. The 4th International Conference on Social Informatics 1 (SocInfo2012), December 5-8, 2012, Lausanne, Switzerland.
    2. 2. There is a surge of interest in building systems that harness thepower of social data to predict election results. # of Facebook users Twitter users’ talking about each # of Facebook Positive/negative candidate; who is talking “likes” & Twitter opinions about about which candidate : “follower” each candidate age, gender, state Tweets from @BarackObama and Real time semantic @MittRomney organized analysis of topic,by engagement on Twitter opinion, emotion, and popularity about each candidate Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 2
    3. 3. One problem seems to be ignored: Are social media users equal in predicting elections? They may be from different countries and states. They may be have different political beliefs. They may be of different ages. They may engage in the elections in different ways and with different levels of involvement. …… They may be … different in predicting elections…? WHOSE opinion really matters? Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 3
    4. 4. o We Studied different groups of social media users who engage in the discussions of 2012 U.S. Republican Presidential Primaries, and compare the predictive power among these user groups.Data: Using Twitter Streaming API, we collected tweets that contain the words“gingrich”, “romney”, “ronpaul”, or “santorum” from 01/10/2012 to 03/05/2012 (SuperTuesday was 03/06/2012). The dataset comprises 6,008,062tweets from 933,343users. Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 4
    5. 5. User Categorization 2. Tweet Mode 3. Content Type 4. Political Preference1. EngagementDegree Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 5
    6. 6. 1 More than half of the users posted only one tweet. Only 8% of the users posted more than 10 tweets.  A small group of users (0.23%) can produce a large amount of tweets (23.73%) – Is tweet volume a reliable predictor?2 The usage of hashtags and URLs reflects the users intent to attract peoples attention on the topic they discuss. The more engaged users show stronger such intent and are more involved in the election event. Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 6
    7. 7. According to users preference on generating their tweets, i.e., tweet mode, we classified the users as original tweet-dominant, original tweet-prone, balanced, retweet-prone and retweet-dominant. 3EngagementDegree  The original tweet-dominant group accounts for the biggest proportion of users in every user engagement group.  A significant number of users (34.71% of all the users) belong to the retweet -dominant group, whose voting intent might be more difficult to detect. Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 7
    8. 8. We use target-specific sentiment analysis techniques to classify each tweet as positive or negative – whether the expressed opinion about a specific candidate is positive or negative. The users are categorized based on whether they post more information or more opinion. 4EngagementDegree  More engaged users tend to post a mixture of content, with similar proportion of opinion and information, or larger proportion of information. Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 8
    9. 9. We collected a set of Twitter users with known political preference from Twellow(http://www.twellow.com/categories/politics). Based on the assumption that a user tendsto follow others who share the same political preference as his/hers, we identified theleft-leaning and right-leaning users utilizing their following/follower relations. Wetested this method using a datasets of 3341 users, and it showed an accuracy of 0.9243. 5  Right-leaning users were (as expected) more involved in republican primaries in several ways: more users, more tweets, more original tweets, higher usage of hashtags and URLs. Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 9
    10. 10.  We utilized the background knowledge from LinkedGeoData to identify thestates from user location information. If the users state could not be inferred from his/her location in the profile, weutilized the geographic locations of his/her tweets. A user was recognized as froma state if his/her tweets were from that state. 6 The Pearsons r for the correlation between the number of users/tweets and the population is 0.9459/0.9667 (p<.0001). Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 10
    11. 11. Predicting a Users Vote• Basic idea: for which candidate the user shows the most support – Frequent mentions The user More mentions, – Positive sentiment posted opinion higher score about cMore positive/less The usernegative opinions, mentioned c but higher score did not post Nm(c): the number of tweets mentioning the candidate c opinion about c Npos(c): the number of positive tweets about candidate c Nneg(c): the number of negative tweets about candidate c (0 < < 1): smoothing parameter (0 < < 1): discounting the score when the user does not express any opinion towards c. Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 11
    12. 12. Prediction ResultsWe examine the predictive power of different user groups in predicting theresults of Super Tuesday races in 10 states.To predict the election results in a state, we used only the collection ofusers who are identified from that state. We examined four time windows -- 7 days, 14 days, 28 days and 56 days prior to the primary day. In a specific time window, a users vote was assessed using only the set of tweets he/she created during this time. The results were evaluated in two ways: (1) the accuracy of predicting winners, and (2) the error rate between the predicted percentage of votes and the actual percentage of votes for each candidate. Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 12
    13. 13. 7 The prediction accuracy:  Engagement Degree: High > Low or Very Low Tweet Mode: Original Tweet-Prone >Retweet-Prone  Content Type: In a draw  Political Preference: Right-Leaning >> Left Leaning Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 13
    14. 14. Revealing the challenge of Retweets may not necessarily8 identifying the vote intent of “silent reflect users attitude. majority” The right-leaning user group provides the most accurate prediction result. In the best case (56-day time window), it correctly predict the winners in 8 out of 10 states with an average Prediction of user’s vote based on prediction error of 0.1. more opinion tweets is not necessarily more accurate than the To some extent, it demonstrates the prediction using more information importance of identifying likely voters tweets in electoral prediction. Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 14
    15. 15. Our findingsTwitter users are not “equal” in predicting elections! The likely voters’ opinions matter more. Some users’ opinions are more difficult to identify because of their lower levels of engagement or the implicit ways to express opinions. Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 15
    16. 16. More Work need to be done… • Identifying likely/actual voters • Improving sentiment analysis techniques • Investigating possible data biases (e.g., spam tweets and political campaign tweets) and how they might affect the results and more …Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 16
    17. 17. It is actually about tracking public opinion. PollingorSocial Media Analysis? 1. Sample size 2. Representative of the target population 3. Accurate measure of opinions 4. Timeliness Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 17
    18. 18. 1 Sample Size Polling Social Media Analysis Thousands of people Millions of people Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 18
    19. 19. 2 Representative of the Target Population Polling Social Media Analysis About 95% of US homes can bereached by landline telephone andcell phone.  About 60% of American adults Sampling the target population use social networking sites.randomly. Difficult to do random sampling. Weighting the sample to census Limited demographic dataestimates for demographic (although with some work, can becharacteristics (gender, race, age, improved).educational attainment, andregion).[1] Can Social Media Be Used for Political Polling? http://www.radian6.com/blog/2012/07/can-social-media-be-used-for-political-polling/ Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 19
    20. 20. 3 Accurate measure of opinions Polling Social Media Analysis Ask people what they think Who will you vote for?  Look at what people talk about and extract their opinions ……  Not as accurate as Polling Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 20
    21. 21. 4 Timeliness Polling Social Media Analysis Not be able to track people’s opinion in real time What is happening now Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 21
    22. 22. Social Media Analysis – Promising but VeryChallenging  Extracting demographic  Increasing number of social information media users  Identifying the target population  Convenient and comfortable whose opinion matter, e.g. the way to express opinions likely voters in electoral prediction  The analysis can be done in real  Discriminate personal opinion time from the voice of mainstream media and political campaign  Lower cost  More accurate sentiment A great complement (if not analysis/opinion mining, substitute) for polling especially the identification of opinions about a specific object Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 22
    23. 23. Our Twitris+ System kept tracking people’s opinion on 2012 U.S.Presidential Election in real time and this is what we saw on the Election Day … Subjective Information Extraction, Lu Chen 23
    24. 24. /t The screenshots of Twitris+ were taken on Nov. 6th 6 PM ESTSubjective Information Extraction, Lu Chen 24
    25. 25. Twitris+: http://twitris.knoesis.org/ Select event Multi-faceted Analysis Select date N-gram summaries Related tweets Reference news Wikipedia articles Subjective Information Extraction, Lu Chen 25
    26. 26.  A key innovation in sentiment analysis, employed in Twitris+, is topic specific sentimentanalysis -- to associate sentiment with an entity. The same sentiment phrases may beassigned different polarities associated with different entities.Twitris+ tracks sentiment trend about different entities, and identifies topics/events thatcontribute to sentiment changes. The result is updated every hour. Sentiment change about BarackObama Analysis can be performed at location (eg, by state) or issue Positive/negative topics based level (eg, that contribute to such economy, tax, Sentiment change about change social issues – Mitt Romney women, …) Individual tweets related to chosen topic Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 26
    27. 27. Twitris+ Insights in 2012 Presidential DebatesHow was Obama doing in the first debate? Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 27
    28. 28. How was Obama doing in the second debate? Red Color: Negative Topics Green Color: Positive Topics Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 28
    29. 29. Obama vsRomney in the third debate Obama RomneyYou can find a lot more –Eg analysis from network, demographic,emotion, temporal, …perspectives athttp://twitris.knoesis.org Are Twitter Users Equal in Predicting Elections? Lu Chen, Wenbo Wang, Amit Sheth 29
    30. 30. Thank you ! More about this study: http://wiki.knoesis.org/index.php/ElectionPrediction Kno.e.sis Center: http://knoesis.wright.edu/ Twitris+: http://twitris.knoesis.org/ Semantics driven Analysis of Social Media: http://knoesis.org/research/semweb/projects/socialmedia Subjective Information Extraction, Lu Chen 30
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×