Cody Buntain
@codybuntain
cbuntain@cs.umd.edu
HCIL
University of Maryland
Erin McGrath, Gary LaFree
{ecmcgrath,lafree}@umd...
Surveys are mature,
ubiquitous, and
powerful instruments
2 - Introduction
Social media platforms
produce huge data
sets and let users
respond to events
instantly
3 - Introduction
0
0.4
0.8
1.2
1.6...
4 - Introduction
5 - Introduction
Social	media	isn’t		
representa1ve!	
Surveys	are	be7er!
Social	media	is	
cheap,	big,	and		
easy!
The Argu...
What is the point
of this talk?
6 - Introduction
- Introduction7
The Point
To compare social
media and survey data
Compared across:
• Costs
• Data Types
• Relevance
• Vali...
- Introduction8
Grounding the Point
9 - Case Study
What makes the Boston Marathon
Bombing a good case study?
The Bombing from Multiple
Perspectives
• Three-wave survey on public
perceptions of law enforcement
• The bombing happened...
Why Twitter?
11 - Case Study
12 - Results
What are the strengths and weaknesses
of surveys and social media?
Four Axes of Comparison
• Data types
• Relevance
• Cost
• Validity
13 - Results
Comparing Data Types
14 - Results
Survey Data
Social Media

Data
• Attitudinal data
• Behavioral data
• Demographic data
Comparing Data Types
15 - Results
Survey Data
Social Media

Data
• Attitudinal data
• Behavioral data
• Demographic data
Comparing Data Types
16 - Results
Survey Data
Social Media

Data
• Attitudinal data
• Behavioral data
• Demographic data
Comparing Data Types
17 - Results
Comparing Data Types
18 - Results
Boston PD Follower Counts
Boston-Related Twitter Activity
Comparing Data Types
19 - Results
0	
0.05	
0.1	
0.15	
0.2	
0.25	
0.3	
0.35	
04-01	04-03	04-05	04-07	04-09	04-11	04-13	04-1...
Comparing Data Types
20 - Results
Self-reported “willingness”
Sentiment Towards Police in the US
Takeaway:
Social media and survey data provide
different types of data
- Results
Comparing Data Types
21
With some overlap...
Four Axes of Comparison
• Data types
• Relevance
• Cost
• Validity
22 - Results
Comparing Relevance
23 - Results
• Temporal Relevance
• Topical Relevance
• Geographic Relevance
Comparing Relevance
24 - Results
Temporal Relevance
2
m
onths
M
arch
‘13
April‘13
M
ay
‘13
June
‘13
July
‘13
August‘13
Comparing Relevance
25 - Results
Topical Relevance
April1
‘13
April8
‘13
April15
‘13
April22
‘13
April29
‘13
M
ay
6
‘13
“p...
Comparing Relevance
26 - Results
Geographic Relevance
Distribution of Twitter Users Target Country for Survey
Comparing Relevance
27 - Results
Geographic Relevance
Distribution of Twitter Users Target Country for Survey
???
Takeaway:
Social media data is more timely than survey
data, but surveys can better target topical and
geographic response...
Four Axes of Comparison
• Data types
• Relevance
• Cost
• Validity
29 - Results
- Results
Comparing Cost
30
- Results
Comparing Cost
31
$300/month
$11/month
+ $3,000/month
$3,311/month
Average
one-time fee:
$70,000
- Results
Comparing Cost
32
Takeaway:
Storing large sets of social media data,
contracting with a data reseller, and processing
it at scale is signific...
Four Axes of Comparison
• Data types
• Relevance
• Cost
• Validity
34 - Results
- Results
Comparing Validity
35
- Results
Comparing Validity
36
Takeaway:
The typical definition of validity is difficult (and
perhaps undesirable) to apply to social media
data.
- Results...
What is the point
of this talk?
38 - Conclusions
Complements and
Synergies
• Use both social media and
surveys where possible
• Social media may provide
first approximation...
Cody Buntain
@codybuntain
cbuntain@cs.umd.edu
HCIL
University of Maryland
Comparing Social Media and Traditional
Surveys A...
Upcoming SlideShare
Loading in …5
×

#Microposts16 - Comparing Social Media and Traditional Surveys around the Boston Marathon Bombing

188 views

Published on

A presentation of our work in comparing social science work on social media and using traditional surveys. Our major takeaway from this lessons-learned work is that social media is timely, big, and cheap, and nationally representative surveys provide higher quality data at higher cost.

Published in: Social Media
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
188
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

#Microposts16 - Comparing Social Media and Traditional Surveys around the Boston Marathon Bombing

  1. 1. Cody Buntain @codybuntain cbuntain@cs.umd.edu HCIL University of Maryland Erin McGrath, Gary LaFree {ecmcgrath,lafree}@umd.edu START Center, UMD Jennifer Golbeck golbeck@cs.umd.edu University of Maryland Comparing Social Media and Traditional Surveys Around the Boston Marathon Bombing 1 #Microposts2016 11 April 2016 Montreal, Quebec CA
  2. 2. Surveys are mature, ubiquitous, and powerful instruments 2 - Introduction
  3. 3. Social media platforms produce huge data sets and let users respond to events instantly 3 - Introduction 0 0.4 0.8 1.2 1.6 Facebook*C hina U SA Tw itter* Population(Billions) Instagram Twitter Facebook 1 100 10,000 1,000,000 100,000,000 New posts Since I’ve been talking
  4. 4. 4 - Introduction
  5. 5. 5 - Introduction Social media isn’t representa1ve! Surveys are be7er! Social media is cheap, big, and easy! The Argument
  6. 6. What is the point of this talk? 6 - Introduction
  7. 7. - Introduction7 The Point To compare social media and survey data Compared across: • Costs • Data Types • Relevance • Validity
  8. 8. - Introduction8 Grounding the Point
  9. 9. 9 - Case Study What makes the Boston Marathon Bombing a good case study?
  10. 10. The Bombing from Multiple Perspectives • Three-wave survey on public perceptions of law enforcement • The bombing happened to occur between waves • Post hoc survey on information seeking/sharing during crises • Included respondents exposed to Boston Marathon coverage • Analysis of public response to the bombing on Twitter • 134 million tweets from April 2013 10 - Case Study
  11. 11. Why Twitter? 11 - Case Study
  12. 12. 12 - Results What are the strengths and weaknesses of surveys and social media?
  13. 13. Four Axes of Comparison • Data types • Relevance • Cost • Validity 13 - Results
  14. 14. Comparing Data Types 14 - Results Survey Data Social Media
 Data • Attitudinal data • Behavioral data • Demographic data
  15. 15. Comparing Data Types 15 - Results Survey Data Social Media
 Data • Attitudinal data • Behavioral data • Demographic data
  16. 16. Comparing Data Types 16 - Results Survey Data Social Media
 Data • Attitudinal data • Behavioral data • Demographic data
  17. 17. Comparing Data Types 17 - Results
  18. 18. Comparing Data Types 18 - Results Boston PD Follower Counts Boston-Related Twitter Activity
  19. 19. Comparing Data Types 19 - Results 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 04-01 04-03 04-05 04-07 04-09 04-11 04-13 04-15 04-17 04-19 04-21 04-23 04-25 04-27 04-29 Percentage of Tweets Containing Emo=on Date Anger Disgust Fear Happiness Sadness Surprise Emotions in Boston-related Tweets from the US
  20. 20. Comparing Data Types 20 - Results Self-reported “willingness” Sentiment Towards Police in the US
  21. 21. Takeaway: Social media and survey data provide different types of data - Results Comparing Data Types 21 With some overlap And both are useful for different purposes.
  22. 22. Four Axes of Comparison • Data types • Relevance • Cost • Validity 22 - Results
  23. 23. Comparing Relevance 23 - Results • Temporal Relevance • Topical Relevance • Geographic Relevance
  24. 24. Comparing Relevance 24 - Results Temporal Relevance 2 m onths M arch ‘13 April‘13 M ay ‘13 June ‘13 July ‘13 August‘13
  25. 25. Comparing Relevance 25 - Results Topical Relevance April1 ‘13 April8 ‘13 April15 ‘13 April22 ‘13 April29 ‘13 M ay 6 ‘13 “police”in Tw itter #delhirape Protests
  26. 26. Comparing Relevance 26 - Results Geographic Relevance Distribution of Twitter Users Target Country for Survey
  27. 27. Comparing Relevance 27 - Results Geographic Relevance Distribution of Twitter Users Target Country for Survey ???
  28. 28. Takeaway: Social media data is more timely than survey data, but surveys can better target topical and geographic responses for higher quality. - Results Comparing Relevance 28
  29. 29. Four Axes of Comparison • Data types • Relevance • Cost • Validity 29 - Results
  30. 30. - Results Comparing Cost 30
  31. 31. - Results Comparing Cost 31 $300/month $11/month + $3,000/month $3,311/month Average one-time fee: $70,000
  32. 32. - Results Comparing Cost 32
  33. 33. Takeaway: Storing large sets of social media data, contracting with a data reseller, and processing it at scale is significantly cheaper than running nationally representative surveys. - Results Comparing Cost 33
  34. 34. Four Axes of Comparison • Data types • Relevance • Cost • Validity 34 - Results
  35. 35. - Results Comparing Validity 35
  36. 36. - Results Comparing Validity 36
  37. 37. Takeaway: The typical definition of validity is difficult (and perhaps undesirable) to apply to social media data. - Results Comparing Validity 37
  38. 38. What is the point of this talk? 38 - Conclusions
  39. 39. Complements and Synergies • Use both social media and surveys where possible • Social media may provide first approximation when time and cost are factors • Survey data can be of higher quality, more targeted, and can ask deeper questions 39 - Conclusions
  40. 40. Cody Buntain @codybuntain cbuntain@cs.umd.edu HCIL University of Maryland Comparing Social Media and Traditional Surveys Around the Boston Marathon Bombing 40 #Microposts2016 11 April 2016 Montreal, Quebec CA Thank you! Questions?

×