stanford hci groupQuantifying theInvisible Audiencein Social NetworksEytan Bakshy, Moira Burke, Brian KarrerFacebook Data ...
Sharing on a socialnetwork is likegiving a talk frombehind a curtain.
Sharing on a socialnetwork is likegiving a talk frombehind a curtain.2
Quantify the difference betweenusers’ estimated and actual audience
Quantify the difference betweenusers’ estimated and actual audienceMeasure audience size uncertaintyfor 220,000 Facebook us...
Our perception of audience sizeaffects our behaviorWe guide our audience’s impression of us[Goffman 1959]We manage the bound...
Our perception of audience sizeaffects our behaviorWe guide our audience’s impression of us[Goffman 1959]We manage the bound...
Perceived audiencevs. reality- survey- folk theories of audience- desired audience sizePredictability ofaudience size- usi...
Perceived audiencevs. reality- survey- folk theories of audience- desired audience sizePredictability ofaudience size- usi...
MethodData220,000 U.S. Facebook users who share withfriends-only privacyCollected audience information for their statusupd...
MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900ms
MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900ms
MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900ms900ms
MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900ms
MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900msNot a ...
MethodSurveyRecruited users with recent content (2-90 days ago)via a request at the top of news feedN=589; 61% female; mea...
MethodAnalysisCompare participants’ actual audience size totheir estimated audience size
MethodAnalysisCompare participants’ actual audience size totheir estimated audience sizeConsider your own most recent stat...
Users underestimate by 4xResults
Users underestimate by 4xResults0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)
0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)Perceivedaudience(%offriends)Users underestimate by...
Users underestimate by 4xResultsAccurateestimations alongthe diagonal0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audie...
Users underestimate by 4xResultsAccurateestimations alongthe diagonal0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audie...
0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)Perceivedaudience(%offriends)Users underestimate by...
Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa...
Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa...
Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa...
Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa...
Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa...
Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa...
Folk theories of audienceResultsNo folk theory was more accurate than arandom guessRandom guess 23%Feedback — likes and co...
Users want larger audiencesResultssame more far morefewerfar fewer“How many people do you wish saw this content?”50% 25% 22%
Users want larger audiencesResultsRoughly half want a larger audience...but they already have it.same more far morefewerfa...
Users underestimate theiraudience by 4xCommon folk theories usefeedback and friend countUsers want larger audiences,but al...
Perceived audiencevs. reality- survey- folk theories of audience- desired audience sizePredictability ofaudience size- usi...
Perceived audiencevs. reality- survey- folk theories of audience- desired audience sizePredictability ofaudience size- usi...
Can we predict a post’saudience using publicsignals?using the full 220,000 user and 150,000,000 view dataset
35% of friends see median postResultsMore friends means higher variability in audience0.000.010.020.030 50 100 150 200Numb...
0.000.010.020.030 50 100 150 200Number of friends who saw postDensity35% of friends see median postResultsMore friends mea...
35% of friends see median postResultsMore friends means higher variability in audience0.000.010.020.030 50 100 150 200Numb...
Audience size is highly variableResultsHighly variable: 50th percentile range is20% of friends0%20%40%60%80%100%llllllllll...
Audience size is highly variableResultsHighly variable: 50th percentile range is20% of friends50th percentile range90th pe...
Audience size predictionOLS regressionModel predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0....
0%20%40%60%80%100%lll l l l l l l l l l l l l l l l l l0 5 10 15Unique friends liking the postPercentoffriendswhosawpostFe...
Model predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8% of friendcountFriend count andfe...
Model predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8% of friendcountFriend count andfe...
Model predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8% of friendcountFriend count andfe...
How predictable is a user’scumulative audience?Consider the audience for all of a user’s posts over30 days instead of a si...
0%20%40%60%80%100%lllllllllllllllllllllllllllllllll0 200 400 600 800Number of friendsCumulativeaudience(%offriends)61% of ...
Discussion
Fundamental mismatch betweenperceived and actual audienceHow might a 4x underestimate be impacting userbehavior?Type of co...
Fundamental mismatch betweenperceived and actual audienceHow might a 4x underestimate be impacting userbehavior?Type of co...
Why underestimate audience size?The wishful thinking hypothesis: more comfortableto blame a noisy distribution channel tha...
Why underestimate audience size?The wishful thinking hypothesis: more comfortableto blame a noisy distribution channel tha...
Your invisible audience islarger than you probably think.
Your invisible audience islarger than you probably think.Users underestimate audience size by 4xMedian reach is 35% per po...
stanford hci groupQuantifying theInvisible Audiencein Social NetworksEytan Bakshy, Moira Burke, Brian KarrerFacebook Data ...
Quantifying the Invisible Audience in Social Networks
Upcoming SlideShare
Loading in...5
×

Quantifying the Invisible Audience in Social Networks

5,119

Published on

Presented at CHI 2013

When you share content in an online social network, who is listening? Users have scarce information about who actually sees their content, making their audience seem invisible and difficult to estimate. However, understanding this invisible audience can impact both science and design, since perceived audiences influence content production and self-presentation online. In this paper, we combine survey and large-scale log data to examine how well users’ perceptions of their audience match their actual audience on Facebook. We find that social media users consistently underestimate their audience size for their posts, guessing that their audience is just 27% of its true size. Qualitative coding of survey responses reveals folk theories that attempt to reverse-engineer audience size using feedback and friend count, though none of these approaches are particularly accurate. We analyze audience
logs for 222,000 Facebook users’ posts over the course of one month and find that publicly visible signals — friend count, likes, and comments — vary widely and do not strongly indicate the audience of a single post. Despite the variation, users typically reach 61% of their friends each month. Together, our results begin to reveal the invisible undercurrents of audience attention and behavior in online social networks.

Published in: Technology

Quantifying the Invisible Audience in Social Networks

  1. 1. stanford hci groupQuantifying theInvisible Audiencein Social NetworksEytan Bakshy, Moira Burke, Brian KarrerFacebook Data Science TeamMichael BernsteinStanford Computer Science Department
  2. 2. Sharing on a socialnetwork is likegiving a talk frombehind a curtain.
  3. 3. Sharing on a socialnetwork is likegiving a talk frombehind a curtain.2
  4. 4. Quantify the difference betweenusers’ estimated and actual audience
  5. 5. Quantify the difference betweenusers’ estimated and actual audienceMeasure audience size uncertaintyfor 220,000 Facebook users
  6. 6. Our perception of audience sizeaffects our behaviorWe guide our audience’s impression of us[Goffman 1959]We manage the boundaries of when to engage[Altman 1975]On social media, we speak to the audience that weexpect is listening[Marwick and boyd 2011, Viégas 1999]
  7. 7. Our perception of audience sizeaffects our behaviorWe guide our audience’s impression of us[Goffman 1959]We manage the boundaries of when to engage[Altman 1975]On social media, we speak to the audience that weexpect is listening[Marwick and boyd 2011, Viégas 1999]What if our audience size estimates areinaccurate?
  8. 8. Perceived audiencevs. reality- survey- folk theories of audience- desired audience sizePredictability ofaudience size- using friend count- using feedback
  9. 9. Perceived audiencevs. reality- survey- folk theories of audience- desired audience sizePredictability ofaudience size- using friend count- using feedback
  10. 10. MethodData220,000 U.S. Facebook users who share withfriends-only privacyCollected audience information for their statusupdates and link shares over 30 days150,000,000 viewer-story pairs
  11. 11. MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900ms
  12. 12. MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900ms
  13. 13. MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900ms900ms
  14. 14. MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900ms
  15. 15. MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900msNot a direct measure ofattention: users remember~70% of posts they see[Counts and Fisher 2011]
  16. 16. MethodSurveyRecruited users with recent content (2-90 days ago)via a request at the top of news feedN=589; 61% female; mean age 33Audience size surveyShow participants their most recent story“How many people do you think saw it?”“Describe how you came up with that number.”“How many people do you wish saw this content?”
  17. 17. MethodAnalysisCompare participants’ actual audience size totheir estimated audience size
  18. 18. MethodAnalysisCompare participants’ actual audience size totheir estimated audience sizeConsider your own most recent status update:What percentage of your social network do youthink saw it?
  19. 19. Users underestimate by 4xResults
  20. 20. Users underestimate by 4xResults0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)
  21. 21. 0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)Perceivedaudience(%offriends)Users underestimate by 4xResults
  22. 22. Users underestimate by 4xResultsAccurateestimations alongthe diagonal0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)Perceivedaudience(%offriends)
  23. 23. Users underestimate by 4xResultsAccurateestimations alongthe diagonal0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)Perceivedaudience(%offriends)overestimatesunderestimates
  24. 24. 0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)Perceivedaudience(%offriends)Users underestimate by 4xResultsEstimated20 friends = 6% ofnetworkActual78 friends = 24%of networkR2 = 0.04overestimatesunderestimates
  25. 25. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)
  26. 26. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)Random guess 23%
  27. 27. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)Random guess 23%Feedback — likes and comments 21%“I figured about half of the people who see itwill ‘like’ it, or comment on it”
  28. 28. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)Random guess 23%Feedback — likes and comments 21%Fraction of friend count 15%“Maybe a third of my friends saw it.”
  29. 29. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)Random guess 23%Feedback — likes and comments 21%Fraction of friend count 15%Login timing 9%“Not a lot of people stay up late at night”
  30. 30. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)Random guess 23%Feedback — likes and comments 21%Fraction of friend count 15%Login timing 9%Friends seen active on the site 5%Number of close friends and family 3%Who might be interested in the topic 2%Other 10%
  31. 31. Folk theories of audienceResultsNo folk theory was more accurate than arandom guessRandom guess 23%Feedback — likes and comments 21%Fraction of friend count 15%Login timing 9%Friends seen active on the site 5%Number of close friends and family 3%Who might be interested in the topic 2%Other 10%
  32. 32. Users want larger audiencesResultssame more far morefewerfar fewer“How many people do you wish saw this content?”50% 25% 22%
  33. 33. Users want larger audiencesResultsRoughly half want a larger audience...but they already have it.same more far morefewerfar fewer“How many people do you wish saw this content?”50% 25% 22%
  34. 34. Users underestimate theiraudience by 4xCommon folk theories usefeedback and friend countUsers want larger audiences,but already have them
  35. 35. Perceived audiencevs. reality- survey- folk theories of audience- desired audience sizePredictability ofaudience size- using friend count- using feedback
  36. 36. Perceived audiencevs. reality- survey- folk theories of audience- desired audience sizePredictability ofaudience size- using friend count- using feedback
  37. 37. Can we predict a post’saudience using publicsignals?using the full 220,000 user and 150,000,000 view dataset
  38. 38. 35% of friends see median postResultsMore friends means higher variability in audience0.000.010.020.030 50 100 150 200Number of friends who saw postDensity50th percentile by friend count (266)
  39. 39. 0.000.010.020.030 50 100 150 200Number of friends who saw postDensity35% of friends see median postResultsMore friends means higher variability in audience25th percentile by friend count (138)
  40. 40. 35% of friends see median postResultsMore friends means higher variability in audience0.000.010.020.030 50 100 150 200Number of friends who saw postDensity75th percentile by friend count (484)
  41. 41. Audience size is highly variableResultsHighly variable: 50th percentile range is20% of friends0%20%40%60%80%100%lllllllllllllllllllllllllllllllll0 200 400 600 800Number of friendsPercentoffriendswhosawpost
  42. 42. Audience size is highly variableResultsHighly variable: 50th percentile range is20% of friends50th percentile range90th percentile range0%20%40%60%80%100%lllllllllllllllllllllllllllllllll0 200 400 600 800Number of friendsPercentoffriendswhosawpost
  43. 43. Audience size predictionOLS regressionModel predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8%Friend count andfeedback0.27 7%
  44. 44. 0%20%40%60%80%100%lll l l l l l l l l l l l l l l l l l0 5 10 15Unique friends liking the postPercentoffriendswhosawpostFeedback is not predictiveResultsRapid audience growth until the post receivesfeedback from five unique friendsPosts with no likes or comments have especiallylarge variance: 90th percentile is 2%–55%
  45. 45. Model predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8% of friendcountFriend count andfeedback0.27 7% of friendcountAudience size predictionOLS regression
  46. 46. Model predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8% of friendcountFriend count andfeedback0.27 7% of friendcountAudience size predictionOLS regression
  47. 47. Model predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8% of friendcountFriend count andfeedback0.27 7% of friendcountEven with access to all user-visible signals,audience size is still unpredictable.Audience size predictionOLS regression
  48. 48. How predictable is a user’scumulative audience?Consider the audience for all of a user’s posts over30 days instead of a single post50% of the users in our sample produced five ormore pieces of content during the month
  49. 49. 0%20%40%60%80%100%lllllllllllllllllllllllllllllllll0 200 400 600 800Number of friendsCumulativeaudience(%offriends)61% of friends see at least oneof the user’s posts each monthHowever, actual audience size is still highly variable
  50. 50. Discussion
  51. 51. Fundamental mismatch betweenperceived and actual audienceHow might a 4x underestimate be impacting userbehavior?Type of content shared, sharing volume, motivationAmbiguous whether a more socially transparentdesign would be desirable
  52. 52. Fundamental mismatch betweenperceived and actual audienceHow might a 4x underestimate be impacting userbehavior?Type of content shared, sharing volume, motivationAmbiguous whether a more socially transparentdesign would be desirable
  53. 53. Why underestimate audience size?The wishful thinking hypothesis: more comfortableto blame a noisy distribution channel than to blameyourself for writing bad content
  54. 54. Why underestimate audience size?The wishful thinking hypothesis: more comfortableto blame a noisy distribution channel than to blameyourself for writing bad contentWhat role might be played by...The availability heuristic?Algorithmic feed filtering?
  55. 55. Your invisible audience islarger than you probably think.
  56. 56. Your invisible audience islarger than you probably think.Users underestimate audience size by 4xMedian reach is 35% per post and 61% per monthMany want larger audiences but already have them
  57. 57. stanford hci groupQuantifying theInvisible Audiencein Social NetworksEytan Bakshy, Moira Burke, Brian KarrerFacebook Data Science TeamMichael BernsteinStanford Computer Science Department
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×