SlideShare a Scribd company logo
stanford hci group
Quantifying the
Invisible Audience
in Social Networks
Eytan Bakshy, Moira Burke, Brian Karrer
Facebook Data Science Team
Michael Bernstein
Stanford Computer Science Department
Sharing on a social
network is like
giving a talk from
behind a curtain.
Sharing on a social
network is like
giving a talk from
behind a curtain.
2
Quantify the difference between
users’ estimated and actual audience
Quantify the difference between
users’ estimated and actual audience
Measure audience size uncertainty
for 220,000 Facebook users
Our perception of audience size
affects our behavior
We guide our audience’s impression of us
[Goffman 1959]
We manage the boundaries of when to engage
[Altman 1975]
On social media, we speak to the audience that we
expect is listening
[Marwick and boyd 2011, Viégas 1999]
Our perception of audience size
affects our behavior
We guide our audience’s impression of us
[Goffman 1959]
We manage the boundaries of when to engage
[Altman 1975]
On social media, we speak to the audience that we
expect is listening
[Marwick and boyd 2011, Viégas 1999]
What if our audience size estimates are
inaccurate?
Perceived audience
vs. reality
- survey
- folk theories of audience
- desired audience size
Predictability of
audience size
- using friend count
- using feedback
Perceived audience
vs. reality
- survey
- folk theories of audience
- desired audience size
Predictability of
audience size
- using friend count
- using feedback
Method
Data
220,000 U.S. Facebook users who share with
friends-only privacy
Collected audience information for their status
updates and link shares over 30 days
150,000,000 viewer-story pairs
Method
Audience size measurement
Javascript tracking whether a
story remained in the browser
viewport for at least 900ms
Method
Audience size measurement
Javascript tracking whether a
story remained in the browser
viewport for at least 900ms
Method
Audience size measurement
Javascript tracking whether a
story remained in the browser
viewport for at least 900ms
900ms
Method
Audience size measurement
Javascript tracking whether a
story remained in the browser
viewport for at least 900ms
Method
Audience size measurement
Javascript tracking whether a
story remained in the browser
viewport for at least 900ms
Not a direct measure of
attention: users remember
~70% of posts they see
[Counts and Fisher 2011]
Method
Survey
Recruited users with recent content (2-90 days ago)
via a request at the top of news feed
N=589; 61% female; mean age 33
Audience size survey
Show participants their most recent story
“How many people do you think saw it?”
“Describe how you came up with that number.”
“How many people do you wish saw this content?”
Method
Analysis
Compare participants’ actual audience size to
their estimated audience size
Method
Analysis
Compare participants’ actual audience size to
their estimated audience size
Consider your own most recent status update:
What percentage of your social network do you
think saw it?
Users underestimate by 4x
Results
Users underestimate by 4x
Results
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Actual audience (% of friends)
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Actual audience (% of friends)
Perceivedaudience(%offriends)
Users underestimate by 4x
Results
Users underestimate by 4x
Results
Accurate
estimations along
the diagonal
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Actual audience (% of friends)
Perceivedaudience(%offriends)
Users underestimate by 4x
Results
Accurate
estimations along
the diagonal
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Actual audience (% of friends)
Perceivedaudience(%offriends)
overestimates
underestimates
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Actual audience (% of friends)
Perceivedaudience(%offriends)
Users underestimate by 4x
Results
Estimated
20 friends = 6% of
network
Actual
78 friends = 24%
of network
R2 = 0.04
overestimates
underestimates
Folk theories of audience
Results
Inductive coding on participants’ reasons for how
estimating their audience (Fleiss’s Kappa = 0.72)
Folk theories of audience
Results
Inductive coding on participants’ reasons for how
estimating their audience (Fleiss’s Kappa = 0.72)
Random guess 23%
Folk theories of audience
Results
Inductive coding on participants’ reasons for how
estimating their audience (Fleiss’s Kappa = 0.72)
Random guess 23%
Feedback — likes and comments 21%
“I figured about half of the people who see it
will ‘like’ it, or comment on it”
Folk theories of audience
Results
Inductive coding on participants’ reasons for how
estimating their audience (Fleiss’s Kappa = 0.72)
Random guess 23%
Feedback — likes and comments 21%
Fraction of friend count 15%
“Maybe a third of my friends saw it.”
Folk theories of audience
Results
Inductive coding on participants’ reasons for how
estimating their audience (Fleiss’s Kappa = 0.72)
Random guess 23%
Feedback — likes and comments 21%
Fraction of friend count 15%
Login timing 9%
“Not a lot of people stay up late at night”
Folk theories of audience
Results
Inductive coding on participants’ reasons for how
estimating their audience (Fleiss’s Kappa = 0.72)
Random guess 23%
Feedback — likes and comments 21%
Fraction of friend count 15%
Login timing 9%
Friends seen active on the site 5%
Number of close friends and family 3%
Who might be interested in the topic 2%
Other 10%
Folk theories of audience
Results
No folk theory was more accurate than a
random guess
Random guess 23%
Feedback — likes and comments 21%
Fraction of friend count 15%
Login timing 9%
Friends seen active on the site 5%
Number of close friends and family 3%
Who might be interested in the topic 2%
Other 10%
Users want larger audiences
Results
same more far morefewer
far fewer
“How many people do you wish saw this content?”
50% 25% 22%
Users want larger audiences
Results
Roughly half want a larger audience...
but they already have it.
same more far morefewer
far fewer
“How many people do you wish saw this content?”
50% 25% 22%
Users underestimate their
audience by 4x
Common folk theories use
feedback and friend count
Users want larger audiences,
but already have them
Perceived audience
vs. reality
- survey
- folk theories of audience
- desired audience size
Predictability of
audience size
- using friend count
- using feedback
Perceived audience
vs. reality
- survey
- folk theories of audience
- desired audience size
Predictability of
audience size
- using friend count
- using feedback
Can we predict a post’s
audience using public
signals?
using the full 220,000 user and 150,000,000 view dataset
35% of friends see median post
Results
More friends means higher variability in audience
0.00
0.01
0.02
0.03
0 50 100 150 200
Number of friends who saw post
Density
50th percentile by friend count (266)
0.00
0.01
0.02
0.03
0 50 100 150 200
Number of friends who saw post
Density
35% of friends see median post
Results
More friends means higher variability in audience
25th percentile by friend count (138)
35% of friends see median post
Results
More friends means higher variability in audience
0.00
0.01
0.02
0.03
0 50 100 150 200
Number of friends who saw post
Density
75th percentile by friend count (484)
Audience size is highly variable
Results
Highly variable: 50th percentile range is
20% of friends
0%
20%
40%
60%
80%
100%
lllllllllllllllllllllllllllllllll
0 200 400 600 800
Number of friends
Percentoffriendswhosawpost
Audience size is highly variable
Results
Highly variable: 50th percentile range is
20% of friends
50th percentile range
90th percentile range0%
20%
40%
60%
80%
100%
lllllllllllllllllllllllllllllllll
0 200 400 600 800
Number of friends
Percentoffriendswhosawpost
Audience size prediction
OLS regression
Model predictors R2 Mean absolute
error
Friend count 0.12 8% of friend
count
Feedback 0.13 8%
Friend count and
feedback
0.27 7%
0%
20%
40%
60%
80%
100%
l
l
l l l l l l l l l l l l l l l l l l
0 5 10 15
Unique friends liking the post
Percentoffriendswhosawpost
Feedback is not predictive
Results
Rapid audience growth until the post receives
feedback from five unique friends
Posts with no likes or comments have especially
large variance: 90th percentile is 2%–55%
Model predictors R2 Mean absolute
error
Friend count 0.12 8% of friend
count
Feedback 0.13 8% of friend
count
Friend count and
feedback
0.27 7% of friend
count
Audience size prediction
OLS regression
Model predictors R2 Mean absolute
error
Friend count 0.12 8% of friend
count
Feedback 0.13 8% of friend
count
Friend count and
feedback
0.27 7% of friend
count
Audience size prediction
OLS regression
Model predictors R2 Mean absolute
error
Friend count 0.12 8% of friend
count
Feedback 0.13 8% of friend
count
Friend count and
feedback
0.27 7% of friend
count
Even with access to all user-visible signals,
audience size is still unpredictable.
Audience size prediction
OLS regression
How predictable is a user’s
cumulative audience?
Consider the audience for all of a user’s posts over
30 days instead of a single post
50% of the users in our sample produced five or
more pieces of content during the month
0%
20%
40%
60%
80%
100%
l
l
l
llllllllllllllllllllllllllllll
0 200 400 600 800
Number of friends
Cumulativeaudience(%offriends)
61% of friends see at least one
of the user’s posts each month
However, actual audience size is still highly variable
Discussion
Fundamental mismatch between
perceived and actual audience
How might a 4x underestimate be impacting user
behavior?
Type of content shared, sharing volume, motivation
Ambiguous whether a more socially transparent
design would be desirable
Fundamental mismatch between
perceived and actual audience
How might a 4x underestimate be impacting user
behavior?
Type of content shared, sharing volume, motivation
Ambiguous whether a more socially transparent
design would be desirable
Why underestimate audience size?
The wishful thinking hypothesis: more comfortable
to blame a noisy distribution channel than to blame
yourself for writing bad content
Why underestimate audience size?
The wishful thinking hypothesis: more comfortable
to blame a noisy distribution channel than to blame
yourself for writing bad content
What role might be played by...
The availability heuristic?
Algorithmic feed filtering?
Your invisible audience is
larger than you probably think.
Your invisible audience is
larger than you probably think.
Users underestimate audience size by 4x
Median reach is 35% per post and 61% per month
Many want larger audiences but already have them
stanford hci group
Quantifying the
Invisible Audience
in Social Networks
Eytan Bakshy, Moira Burke, Brian Karrer
Facebook Data Science Team
Michael Bernstein
Stanford Computer Science Department

More Related Content

Viewers also liked

Managing Privacy and Context Collapse in the Facebook Age
Managing Privacy and Context Collapse in the Facebook AgeManaging Privacy and Context Collapse in the Facebook Age
Managing Privacy and Context Collapse in the Facebook Age
Jessica Vitak
 
Jessica Vitak, "When Contexts Collapse: Managing Self-Presentation Across Soc...
Jessica Vitak, "When Contexts Collapse: Managing Self-Presentation Across Soc...Jessica Vitak, "When Contexts Collapse: Managing Self-Presentation Across Soc...
Jessica Vitak, "When Contexts Collapse: Managing Self-Presentation Across Soc...
summersocialwebshop
 
The Glass House: Anonymity Vs. Transparency on the Internet
The Glass House: Anonymity Vs. Transparency on the InternetThe Glass House: Anonymity Vs. Transparency on the Internet
The Glass House: Anonymity Vs. Transparency on the Internet
Center for Sustainable Journalism at Kennesaw State University
 
Context collapse on social media: implications for interpersonal and marketin...
Context collapse on social media: implications for interpersonal and marketin...Context collapse on social media: implications for interpersonal and marketin...
Context collapse on social media: implications for interpersonal and marketin...
ACSM @ VU University Amsterdam
 
Social Media - Introduzione al Corso [a.a. 2014-2015] - UniTo
Social Media - Introduzione al Corso [a.a. 2014-2015] - UniToSocial Media - Introduzione al Corso [a.a. 2014-2015] - UniTo
Social Media - Introduzione al Corso [a.a. 2014-2015] - UniTo
Agnese Vellar
 
Comunicare nell'era dei social media
Comunicare nell'era dei social mediaComunicare nell'era dei social media
Comunicare nell'era dei social media
Agnese Vellar
 
Context Collapse on Social Networking Sites
Context Collapse on Social Networking SitesContext Collapse on Social Networking Sites
Context Collapse on Social Networking Sites
Stefanie Duguay
 
Theorizing the Future of Computer-Mediated Communication: The Changing Role o...
Theorizing the Future of Computer-Mediated Communication: The Changing Role o...Theorizing the Future of Computer-Mediated Communication: The Changing Role o...
Theorizing the Future of Computer-Mediated Communication: The Changing Role o...
Jessica Vitak
 
Digital Footprint
Digital FootprintDigital Footprint
Digital Footprint
Robin Low
 
Using Social Smarts to Engage Students on Social Media
Using Social Smarts to Engage Students on Social MediaUsing Social Smarts to Engage Students on Social Media
Using Social Smarts to Engage Students on Social Media
Paul Brown
 

Viewers also liked (10)

Managing Privacy and Context Collapse in the Facebook Age
Managing Privacy and Context Collapse in the Facebook AgeManaging Privacy and Context Collapse in the Facebook Age
Managing Privacy and Context Collapse in the Facebook Age
 
Jessica Vitak, "When Contexts Collapse: Managing Self-Presentation Across Soc...
Jessica Vitak, "When Contexts Collapse: Managing Self-Presentation Across Soc...Jessica Vitak, "When Contexts Collapse: Managing Self-Presentation Across Soc...
Jessica Vitak, "When Contexts Collapse: Managing Self-Presentation Across Soc...
 
The Glass House: Anonymity Vs. Transparency on the Internet
The Glass House: Anonymity Vs. Transparency on the InternetThe Glass House: Anonymity Vs. Transparency on the Internet
The Glass House: Anonymity Vs. Transparency on the Internet
 
Context collapse on social media: implications for interpersonal and marketin...
Context collapse on social media: implications for interpersonal and marketin...Context collapse on social media: implications for interpersonal and marketin...
Context collapse on social media: implications for interpersonal and marketin...
 
Social Media - Introduzione al Corso [a.a. 2014-2015] - UniTo
Social Media - Introduzione al Corso [a.a. 2014-2015] - UniToSocial Media - Introduzione al Corso [a.a. 2014-2015] - UniTo
Social Media - Introduzione al Corso [a.a. 2014-2015] - UniTo
 
Comunicare nell'era dei social media
Comunicare nell'era dei social mediaComunicare nell'era dei social media
Comunicare nell'era dei social media
 
Context Collapse on Social Networking Sites
Context Collapse on Social Networking SitesContext Collapse on Social Networking Sites
Context Collapse on Social Networking Sites
 
Theorizing the Future of Computer-Mediated Communication: The Changing Role o...
Theorizing the Future of Computer-Mediated Communication: The Changing Role o...Theorizing the Future of Computer-Mediated Communication: The Changing Role o...
Theorizing the Future of Computer-Mediated Communication: The Changing Role o...
 
Digital Footprint
Digital FootprintDigital Footprint
Digital Footprint
 
Using Social Smarts to Engage Students on Social Media
Using Social Smarts to Engage Students on Social MediaUsing Social Smarts to Engage Students on Social Media
Using Social Smarts to Engage Students on Social Media
 

Similar to Quantifying the Invisible Audience in Social Networks

0915 omma metrics jim sterne
0915 omma metrics jim sterne0915 omma metrics jim sterne
0915 omma metrics jim sterne
MediaPost
 
0915 omma metrics jim sterne
0915 omma metrics jim sterne0915 omma metrics jim sterne
0915 omma metrics jim sterne
MediaPost
 
A look at facebook insights
A look at facebook insightsA look at facebook insights
A look at facebook insights
Friday Explorer
 
Surfing a Web of Trust: Reputation and Reciprocity on CouchSurfing.com
Surfing a Web of Trust: Reputation and Reciprocity on CouchSurfing.comSurfing a Web of Trust: Reputation and Reciprocity on CouchSurfing.com
Surfing a Web of Trust: Reputation and Reciprocity on CouchSurfing.com
debra24
 
Social Recommender Systems
Social Recommender SystemsSocial Recommender Systems
Social Recommender Systems
guest77b0cd12
 
IAC Facebook ROI Matters
IAC Facebook ROI MattersIAC Facebook ROI Matters
IAC Facebook ROI Matters
Peter Podolinsky
 
Algorithms and the technology of personalisation final
Algorithms and the technology of personalisation finalAlgorithms and the technology of personalisation final
Algorithms and the technology of personalisation final
Colin Strong
 
Influencer poll webinar 1 7slideshare
Influencer poll webinar 1 7slideshareInfluencer poll webinar 1 7slideshare
Influencer poll webinar 1 7slideshare
Cision
 
The Only Metric That Matters by a Partner at Greylock Partners
The Only Metric That Matters by a Partner at Greylock PartnersThe Only Metric That Matters by a Partner at Greylock Partners
The Only Metric That Matters by a Partner at Greylock Partners
Product School
 
October Wine & Web: How To Think About Your Social Media Metrics
October Wine & Web: How To Think About Your Social Media MetricsOctober Wine & Web: How To Think About Your Social Media Metrics
October Wine & Web: How To Think About Your Social Media Metrics
Orbit Media Studios
 
New Ways to Fund Media
New Ways to Fund MediaNew Ways to Fund Media
New Ways to Fund Media
Sharon Chan
 
Branchout 2017 - Day 2 Session - Josh Elman
Branchout 2017 - Day 2 Session - Josh ElmanBranchout 2017 - Day 2 Session - Josh Elman
Branchout 2017 - Day 2 Session - Josh Elman
Branch
 
Leveraging Social Media
Leveraging Social MediaLeveraging Social Media
Leveraging Social Media
Rebecca Krause-Hardie
 
spoiled media Media Kit (2016)
spoiled media Media Kit (2016)spoiled media Media Kit (2016)
spoiled media Media Kit (2016)
Pavel Novel
 
Increasing customer loyalty with social media, 2 13
Increasing customer loyalty with social media, 2 13Increasing customer loyalty with social media, 2 13
Increasing customer loyalty with social media, 2 13
Game Day Communications
 
Real-time retailing webinar - 21st November
Real-time retailing webinar - 21st NovemberReal-time retailing webinar - 21st November
Real-time retailing webinar - 21st November
Qubit
 
How to Strategise for Social Media
How to Strategise for Social MediaHow to Strategise for Social Media
How to Strategise for Social Media
Benita Bhatia Dua
 
Get Better Content with Analytics and User Testing
Get Better Content with Analytics and User TestingGet Better Content with Analytics and User Testing
Get Better Content with Analytics and User Testing
Michael Powers
 
HatlingFlint KNOW Series
HatlingFlint KNOW SeriesHatlingFlint KNOW Series
HatlingFlint KNOW Series
Flint Group
 
Presentation Social Media With A Purpose
Presentation  Social Media With A PurposePresentation  Social Media With A Purpose
Presentation Social Media With A Purpose
Community Foundation of Sarasota County
 

Similar to Quantifying the Invisible Audience in Social Networks (20)

0915 omma metrics jim sterne
0915 omma metrics jim sterne0915 omma metrics jim sterne
0915 omma metrics jim sterne
 
0915 omma metrics jim sterne
0915 omma metrics jim sterne0915 omma metrics jim sterne
0915 omma metrics jim sterne
 
A look at facebook insights
A look at facebook insightsA look at facebook insights
A look at facebook insights
 
Surfing a Web of Trust: Reputation and Reciprocity on CouchSurfing.com
Surfing a Web of Trust: Reputation and Reciprocity on CouchSurfing.comSurfing a Web of Trust: Reputation and Reciprocity on CouchSurfing.com
Surfing a Web of Trust: Reputation and Reciprocity on CouchSurfing.com
 
Social Recommender Systems
Social Recommender SystemsSocial Recommender Systems
Social Recommender Systems
 
IAC Facebook ROI Matters
IAC Facebook ROI MattersIAC Facebook ROI Matters
IAC Facebook ROI Matters
 
Algorithms and the technology of personalisation final
Algorithms and the technology of personalisation finalAlgorithms and the technology of personalisation final
Algorithms and the technology of personalisation final
 
Influencer poll webinar 1 7slideshare
Influencer poll webinar 1 7slideshareInfluencer poll webinar 1 7slideshare
Influencer poll webinar 1 7slideshare
 
The Only Metric That Matters by a Partner at Greylock Partners
The Only Metric That Matters by a Partner at Greylock PartnersThe Only Metric That Matters by a Partner at Greylock Partners
The Only Metric That Matters by a Partner at Greylock Partners
 
October Wine & Web: How To Think About Your Social Media Metrics
October Wine & Web: How To Think About Your Social Media MetricsOctober Wine & Web: How To Think About Your Social Media Metrics
October Wine & Web: How To Think About Your Social Media Metrics
 
New Ways to Fund Media
New Ways to Fund MediaNew Ways to Fund Media
New Ways to Fund Media
 
Branchout 2017 - Day 2 Session - Josh Elman
Branchout 2017 - Day 2 Session - Josh ElmanBranchout 2017 - Day 2 Session - Josh Elman
Branchout 2017 - Day 2 Session - Josh Elman
 
Leveraging Social Media
Leveraging Social MediaLeveraging Social Media
Leveraging Social Media
 
spoiled media Media Kit (2016)
spoiled media Media Kit (2016)spoiled media Media Kit (2016)
spoiled media Media Kit (2016)
 
Increasing customer loyalty with social media, 2 13
Increasing customer loyalty with social media, 2 13Increasing customer loyalty with social media, 2 13
Increasing customer loyalty with social media, 2 13
 
Real-time retailing webinar - 21st November
Real-time retailing webinar - 21st NovemberReal-time retailing webinar - 21st November
Real-time retailing webinar - 21st November
 
How to Strategise for Social Media
How to Strategise for Social MediaHow to Strategise for Social Media
How to Strategise for Social Media
 
Get Better Content with Analytics and User Testing
Get Better Content with Analytics and User TestingGet Better Content with Analytics and User Testing
Get Better Content with Analytics and User Testing
 
HatlingFlint KNOW Series
HatlingFlint KNOW SeriesHatlingFlint KNOW Series
HatlingFlint KNOW Series
 
Presentation Social Media With A Purpose
Presentation  Social Media With A PurposePresentation  Social Media With A Purpose
Presentation Social Media With A Purpose
 

More from Michael Bernstein

The Future of Crowd Work
The Future of Crowd WorkThe Future of Crowd Work
The Future of Crowd Work
Michael Bernstein
 
Direct Answers for Search Queries in the Long Tail
Direct Answers for Search Queries in the Long TailDirect Answers for Search Queries in the Long Tail
Direct Answers for Search Queries in the Long Tail
Michael Bernstein
 
Analytic Methods for Optimizing Realtime Crowdsourcing
Analytic Methods for Optimizing Realtime CrowdsourcingAnalytic Methods for Optimizing Realtime Crowdsourcing
Analytic Methods for Optimizing Realtime Crowdsourcing
Michael Bernstein
 
4chan and /b/: An Analysis of Anonymity and Ephemerality in a Large Online Co...
4chan and /b/: An Analysis of Anonymity and Ephemerality in a Large Online Co...4chan and /b/: An Analysis of Anonymity and Ephemerality in a Large Online Co...
4chan and /b/: An Analysis of Anonymity and Ephemerality in a Large Online Co...
Michael Bernstein
 
RepliCHI: Graduate Student Perspectives
RepliCHI: Graduate Student PerspectivesRepliCHI: Graduate Student Perspectives
RepliCHI: Graduate Student Perspectives
Michael Bernstein
 
RepliCHI: Graduate Student Perspectives
RepliCHI: Graduate Student PerspectivesRepliCHI: Graduate Student Perspectives
RepliCHI: Graduate Student Perspectives
Michael Bernstein
 
The Trouble with Social Computing Systems Research
The Trouble with Social Computing Systems ResearchThe Trouble with Social Computing Systems Research
The Trouble with Social Computing Systems Research
Michael Bernstein
 
Soylent: A Word Processor with a Crowd Inside
Soylent: A Word Processor with a Crowd InsideSoylent: A Word Processor with a Crowd Inside
Soylent: A Word Processor with a Crowd Inside
Michael Bernstein
 
Eddi: Interactive Topic-Based Browsing of Social Status Streams
Eddi: Interactive Topic-Based Browsing of Social Status StreamsEddi: Interactive Topic-Based Browsing of Social Status Streams
Eddi: Interactive Topic-Based Browsing of Social Status Streams
Michael Bernstein
 
HarambeeNet: Data by the people, for the people
HarambeeNet: Data by the people, for the peopleHarambeeNet: Data by the people, for the people
HarambeeNet: Data by the people, for the people
Michael Bernstein
 
FeedMe: Enhancing Directed Content Sharing on the Web
FeedMe: Enhancing Directed Content Sharing on the WebFeedMe: Enhancing Directed Content Sharing on the Web
FeedMe: Enhancing Directed Content Sharing on the Web
Michael Bernstein
 

More from Michael Bernstein (11)

The Future of Crowd Work
The Future of Crowd WorkThe Future of Crowd Work
The Future of Crowd Work
 
Direct Answers for Search Queries in the Long Tail
Direct Answers for Search Queries in the Long TailDirect Answers for Search Queries in the Long Tail
Direct Answers for Search Queries in the Long Tail
 
Analytic Methods for Optimizing Realtime Crowdsourcing
Analytic Methods for Optimizing Realtime CrowdsourcingAnalytic Methods for Optimizing Realtime Crowdsourcing
Analytic Methods for Optimizing Realtime Crowdsourcing
 
4chan and /b/: An Analysis of Anonymity and Ephemerality in a Large Online Co...
4chan and /b/: An Analysis of Anonymity and Ephemerality in a Large Online Co...4chan and /b/: An Analysis of Anonymity and Ephemerality in a Large Online Co...
4chan and /b/: An Analysis of Anonymity and Ephemerality in a Large Online Co...
 
RepliCHI: Graduate Student Perspectives
RepliCHI: Graduate Student PerspectivesRepliCHI: Graduate Student Perspectives
RepliCHI: Graduate Student Perspectives
 
RepliCHI: Graduate Student Perspectives
RepliCHI: Graduate Student PerspectivesRepliCHI: Graduate Student Perspectives
RepliCHI: Graduate Student Perspectives
 
The Trouble with Social Computing Systems Research
The Trouble with Social Computing Systems ResearchThe Trouble with Social Computing Systems Research
The Trouble with Social Computing Systems Research
 
Soylent: A Word Processor with a Crowd Inside
Soylent: A Word Processor with a Crowd InsideSoylent: A Word Processor with a Crowd Inside
Soylent: A Word Processor with a Crowd Inside
 
Eddi: Interactive Topic-Based Browsing of Social Status Streams
Eddi: Interactive Topic-Based Browsing of Social Status StreamsEddi: Interactive Topic-Based Browsing of Social Status Streams
Eddi: Interactive Topic-Based Browsing of Social Status Streams
 
HarambeeNet: Data by the people, for the people
HarambeeNet: Data by the people, for the peopleHarambeeNet: Data by the people, for the people
HarambeeNet: Data by the people, for the people
 
FeedMe: Enhancing Directed Content Sharing on the Web
FeedMe: Enhancing Directed Content Sharing on the WebFeedMe: Enhancing Directed Content Sharing on the Web
FeedMe: Enhancing Directed Content Sharing on the Web
 

Recently uploaded

Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
christinelarrosa
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
Chart Kalyan
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
Javier Junquera
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
Fwdays
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
DianaGray10
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
DanBrown980551
 
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
UiPathCommunity
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
ScyllaDB
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
Fwdays
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
Mydbops
 
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeckPoznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
FilipTomaszewski5
 

Recently uploaded (20)

Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
Northern Engraving | Modern Metal Trim, Nameplates and Appliance Panels
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfHow to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdf
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)GNSS spoofing via SDR (Criptored Talks 2024)
GNSS spoofing via SDR (Criptored Talks 2024)
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
 
What is an RPA CoE? Session 2 – CoE Roles
What is an RPA CoE?  Session 2 – CoE RolesWhat is an RPA CoE?  Session 2 – CoE Roles
What is an RPA CoE? Session 2 – CoE Roles
 
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
LF Energy Webinar: Carbon Data Specifications: Mechanisms to Improve Data Acc...
 
Session 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdfSession 1 - Intro to Robotic Process Automation.pdf
Session 1 - Intro to Robotic Process Automation.pdf
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
 
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeckPoznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
Poznań ACE event - 19.06.2024 Team 24 Wrapup slidedeck
 

Quantifying the Invisible Audience in Social Networks

  • 1. stanford hci group Quantifying the Invisible Audience in Social Networks Eytan Bakshy, Moira Burke, Brian Karrer Facebook Data Science Team Michael Bernstein Stanford Computer Science Department
  • 2. Sharing on a social network is like giving a talk from behind a curtain.
  • 3. Sharing on a social network is like giving a talk from behind a curtain. 2
  • 4.
  • 5. Quantify the difference between users’ estimated and actual audience
  • 6. Quantify the difference between users’ estimated and actual audience Measure audience size uncertainty for 220,000 Facebook users
  • 7. Our perception of audience size affects our behavior We guide our audience’s impression of us [Goffman 1959] We manage the boundaries of when to engage [Altman 1975] On social media, we speak to the audience that we expect is listening [Marwick and boyd 2011, Viégas 1999]
  • 8. Our perception of audience size affects our behavior We guide our audience’s impression of us [Goffman 1959] We manage the boundaries of when to engage [Altman 1975] On social media, we speak to the audience that we expect is listening [Marwick and boyd 2011, Viégas 1999] What if our audience size estimates are inaccurate?
  • 9. Perceived audience vs. reality - survey - folk theories of audience - desired audience size Predictability of audience size - using friend count - using feedback
  • 10. Perceived audience vs. reality - survey - folk theories of audience - desired audience size Predictability of audience size - using friend count - using feedback
  • 11. Method Data 220,000 U.S. Facebook users who share with friends-only privacy Collected audience information for their status updates and link shares over 30 days 150,000,000 viewer-story pairs
  • 12. Method Audience size measurement Javascript tracking whether a story remained in the browser viewport for at least 900ms
  • 13. Method Audience size measurement Javascript tracking whether a story remained in the browser viewport for at least 900ms
  • 14. Method Audience size measurement Javascript tracking whether a story remained in the browser viewport for at least 900ms 900ms
  • 15. Method Audience size measurement Javascript tracking whether a story remained in the browser viewport for at least 900ms
  • 16. Method Audience size measurement Javascript tracking whether a story remained in the browser viewport for at least 900ms Not a direct measure of attention: users remember ~70% of posts they see [Counts and Fisher 2011]
  • 17. Method Survey Recruited users with recent content (2-90 days ago) via a request at the top of news feed N=589; 61% female; mean age 33 Audience size survey Show participants their most recent story “How many people do you think saw it?” “Describe how you came up with that number.” “How many people do you wish saw this content?”
  • 18. Method Analysis Compare participants’ actual audience size to their estimated audience size
  • 19. Method Analysis Compare participants’ actual audience size to their estimated audience size Consider your own most recent status update: What percentage of your social network do you think saw it?
  • 21. Users underestimate by 4x Results 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Actual audience (% of friends)
  • 22. 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Actual audience (% of friends) Perceivedaudience(%offriends) Users underestimate by 4x Results
  • 23. Users underestimate by 4x Results Accurate estimations along the diagonal 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Actual audience (% of friends) Perceivedaudience(%offriends)
  • 24. Users underestimate by 4x Results Accurate estimations along the diagonal 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Actual audience (% of friends) Perceivedaudience(%offriends) overestimates underestimates
  • 25. 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Actual audience (% of friends) Perceivedaudience(%offriends) Users underestimate by 4x Results Estimated 20 friends = 6% of network Actual 78 friends = 24% of network R2 = 0.04 overestimates underestimates
  • 26. Folk theories of audience Results Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72)
  • 27. Folk theories of audience Results Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72) Random guess 23%
  • 28. Folk theories of audience Results Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72) Random guess 23% Feedback — likes and comments 21% “I figured about half of the people who see it will ‘like’ it, or comment on it”
  • 29. Folk theories of audience Results Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72) Random guess 23% Feedback — likes and comments 21% Fraction of friend count 15% “Maybe a third of my friends saw it.”
  • 30. Folk theories of audience Results Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72) Random guess 23% Feedback — likes and comments 21% Fraction of friend count 15% Login timing 9% “Not a lot of people stay up late at night”
  • 31. Folk theories of audience Results Inductive coding on participants’ reasons for how estimating their audience (Fleiss’s Kappa = 0.72) Random guess 23% Feedback — likes and comments 21% Fraction of friend count 15% Login timing 9% Friends seen active on the site 5% Number of close friends and family 3% Who might be interested in the topic 2% Other 10%
  • 32. Folk theories of audience Results No folk theory was more accurate than a random guess Random guess 23% Feedback — likes and comments 21% Fraction of friend count 15% Login timing 9% Friends seen active on the site 5% Number of close friends and family 3% Who might be interested in the topic 2% Other 10%
  • 33. Users want larger audiences Results same more far morefewer far fewer “How many people do you wish saw this content?” 50% 25% 22%
  • 34. Users want larger audiences Results Roughly half want a larger audience... but they already have it. same more far morefewer far fewer “How many people do you wish saw this content?” 50% 25% 22%
  • 35. Users underestimate their audience by 4x Common folk theories use feedback and friend count Users want larger audiences, but already have them
  • 36. Perceived audience vs. reality - survey - folk theories of audience - desired audience size Predictability of audience size - using friend count - using feedback
  • 37. Perceived audience vs. reality - survey - folk theories of audience - desired audience size Predictability of audience size - using friend count - using feedback
  • 38. Can we predict a post’s audience using public signals? using the full 220,000 user and 150,000,000 view dataset
  • 39. 35% of friends see median post Results More friends means higher variability in audience 0.00 0.01 0.02 0.03 0 50 100 150 200 Number of friends who saw post Density 50th percentile by friend count (266)
  • 40. 0.00 0.01 0.02 0.03 0 50 100 150 200 Number of friends who saw post Density 35% of friends see median post Results More friends means higher variability in audience 25th percentile by friend count (138)
  • 41. 35% of friends see median post Results More friends means higher variability in audience 0.00 0.01 0.02 0.03 0 50 100 150 200 Number of friends who saw post Density 75th percentile by friend count (484)
  • 42. Audience size is highly variable Results Highly variable: 50th percentile range is 20% of friends 0% 20% 40% 60% 80% 100% lllllllllllllllllllllllllllllllll 0 200 400 600 800 Number of friends Percentoffriendswhosawpost
  • 43. Audience size is highly variable Results Highly variable: 50th percentile range is 20% of friends 50th percentile range 90th percentile range0% 20% 40% 60% 80% 100% lllllllllllllllllllllllllllllllll 0 200 400 600 800 Number of friends Percentoffriendswhosawpost
  • 44. Audience size prediction OLS regression Model predictors R2 Mean absolute error Friend count 0.12 8% of friend count Feedback 0.13 8% Friend count and feedback 0.27 7%
  • 45. 0% 20% 40% 60% 80% 100% l l l l l l l l l l l l l l l l l l l l 0 5 10 15 Unique friends liking the post Percentoffriendswhosawpost Feedback is not predictive Results Rapid audience growth until the post receives feedback from five unique friends Posts with no likes or comments have especially large variance: 90th percentile is 2%–55%
  • 46. Model predictors R2 Mean absolute error Friend count 0.12 8% of friend count Feedback 0.13 8% of friend count Friend count and feedback 0.27 7% of friend count Audience size prediction OLS regression
  • 47. Model predictors R2 Mean absolute error Friend count 0.12 8% of friend count Feedback 0.13 8% of friend count Friend count and feedback 0.27 7% of friend count Audience size prediction OLS regression
  • 48. Model predictors R2 Mean absolute error Friend count 0.12 8% of friend count Feedback 0.13 8% of friend count Friend count and feedback 0.27 7% of friend count Even with access to all user-visible signals, audience size is still unpredictable. Audience size prediction OLS regression
  • 49. How predictable is a user’s cumulative audience? Consider the audience for all of a user’s posts over 30 days instead of a single post 50% of the users in our sample produced five or more pieces of content during the month
  • 50. 0% 20% 40% 60% 80% 100% l l l llllllllllllllllllllllllllllll 0 200 400 600 800 Number of friends Cumulativeaudience(%offriends) 61% of friends see at least one of the user’s posts each month However, actual audience size is still highly variable
  • 52. Fundamental mismatch between perceived and actual audience How might a 4x underestimate be impacting user behavior? Type of content shared, sharing volume, motivation Ambiguous whether a more socially transparent design would be desirable
  • 53. Fundamental mismatch between perceived and actual audience How might a 4x underestimate be impacting user behavior? Type of content shared, sharing volume, motivation Ambiguous whether a more socially transparent design would be desirable
  • 54. Why underestimate audience size? The wishful thinking hypothesis: more comfortable to blame a noisy distribution channel than to blame yourself for writing bad content
  • 55. Why underestimate audience size? The wishful thinking hypothesis: more comfortable to blame a noisy distribution channel than to blame yourself for writing bad content What role might be played by... The availability heuristic? Algorithmic feed filtering?
  • 56. Your invisible audience is larger than you probably think.
  • 57. Your invisible audience is larger than you probably think. Users underestimate audience size by 4x Median reach is 35% per post and 61% per month Many want larger audiences but already have them
  • 58. stanford hci group Quantifying the Invisible Audience in Social Networks Eytan Bakshy, Moira Burke, Brian Karrer Facebook Data Science Team Michael Bernstein Stanford Computer Science Department