Quantifying the Invisible Audience in Social Networks

1. stanford hci groupQuantifying theInvisible Audiencein Social NetworksEytan Bakshy, Moira Burke, Brian KarrerFacebook Data Science TeamMichael BernsteinStanford Computer Science Department

2. Sharing on a socialnetwork is likegiving a talk frombehind a curtain.

3. Sharing on a socialnetwork is likegiving a talk frombehind a curtain.2

4. Quantify the diﬀerence betweenusers’ estimated and actual audience

5. Quantify the diﬀerence betweenusers’ estimated and actual audienceMeasure audience size uncertaintyfor 220,000 Facebook users

6. Our perception of audience sizeaﬀects our behaviorWe guide our audience’s impression of us[Goﬀman 1959]We manage the boundaries of when to engage[Altman 1975]On social media, we speak to the audience that weexpect is listening[Marwick and boyd 2011, Viégas 1999]

7. Our perception of audience sizeaﬀects our behaviorWe guide our audience’s impression of us[Goﬀman 1959]We manage the boundaries of when to engage[Altman 1975]On social media, we speak to the audience that weexpect is listening[Marwick and boyd 2011, Viégas 1999]What if our audience size estimates areinaccurate?

8. Perceived audiencevs. reality- survey- folk theories of audience- desired audience sizePredictability ofaudience size- using friend count- using feedback

10. MethodData220,000 U.S. Facebook users who share withfriends-only privacyCollected audience information for their statusupdates and link shares over 30 days150,000,000 viewer-story pairs

11. MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900ms

13. MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900ms900ms

15. MethodAudience size measurementJavascript tracking whether astory remained in the browserviewport for at least 900msNot a direct measure ofattention: users remember~70% of posts they see[Counts and Fisher 2011]

16. MethodSurveyRecruited users with recent content (2-90 days ago)via a request at the top of news feedN=589; 61% female; mean age 33Audience size surveyShow participants their most recent story“How many people do you think saw it?”“Describe how you came up with that number.”“How many people do you wish saw this content?”

17. MethodAnalysisCompare participants’ actual audience size totheir estimated audience size

18. MethodAnalysisCompare participants’ actual audience size totheir estimated audience sizeConsider your own most recent status update:What percentage of your social network do youthink saw it?

19. Users underestimate by 4xResults

20. Users underestimate by 4xResults0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)

21. 0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)Perceivedaudience(%offriends)Users underestimate by 4xResults

22. Users underestimate by 4xResultsAccurateestimations alongthe diagonal0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)Perceivedaudience(%offriends)

23. Users underestimate by 4xResultsAccurateestimations alongthe diagonal0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)Perceivedaudience(%offriends)overestimatesunderestimates

24. 0%20%40%60%80%100%0% 20% 40% 60% 80% 100%Actual audience (% of friends)Perceivedaudience(%offriends)Users underestimate by 4xResultsEstimated20 friends = 6% ofnetworkActual78 friends = 24%of networkR2 = 0.04overestimatesunderestimates

25. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)

26. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)Random guess 23%

27. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)Random guess 23%Feedback — likes and comments 21%“I ﬁgured about half of the people who see itwill ‘like’ it, or comment on it”

28. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)Random guess 23%Feedback — likes and comments 21%Fraction of friend count 15%“Maybe a third of my friends saw it.”

29. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)Random guess 23%Feedback — likes and comments 21%Fraction of friend count 15%Login timing 9%“Not a lot of people stay up late at night”

30. Folk theories of audienceResultsInductive coding on participants’ reasons for howestimating their audience (Fleiss’s Kappa = 0.72)Random guess 23%Feedback — likes and comments 21%Fraction of friend count 15%Login timing 9%Friends seen active on the site 5%Number of close friends and family 3%Who might be interested in the topic 2%Other 10%

31. Folk theories of audienceResultsNo folk theory was more accurate than arandom guessRandom guess 23%Feedback — likes and comments 21%Fraction of friend count 15%Login timing 9%Friends seen active on the site 5%Number of close friends and family 3%Who might be interested in the topic 2%Other 10%

32. Users want larger audiencesResultssame more far morefewerfar fewer“How many people do you wish saw this content?”50% 25% 22%

33. Users want larger audiencesResultsRoughly half want a larger audience...but they already have it.same more far morefewerfar fewer“How many people do you wish saw this content?”50% 25% 22%

34. Users underestimate theiraudience by 4xCommon folk theories usefeedback and friend countUsers want larger audiences,but already have them

37. Can we predict a post’saudience using publicsignals?using the full 220,000 user and 150,000,000 view dataset

38. 35% of friends see median postResultsMore friends means higher variability in audience0.000.010.020.030 50 100 150 200Number of friends who saw postDensity50th percentile by friend count (266)

39. 0.000.010.020.030 50 100 150 200Number of friends who saw postDensity35% of friends see median postResultsMore friends means higher variability in audience25th percentile by friend count (138)

40. 35% of friends see median postResultsMore friends means higher variability in audience0.000.010.020.030 50 100 150 200Number of friends who saw postDensity75th percentile by friend count (484)

41. Audience size is highly variableResultsHighly variable: 50th percentile range is20% of friends0%20%40%60%80%100%lllllllllllllllllllllllllllllllll0 200 400 600 800Number of friendsPercentoffriendswhosawpost

42. Audience size is highly variableResultsHighly variable: 50th percentile range is20% of friends50th percentile range90th percentile range0%20%40%60%80%100%lllllllllllllllllllllllllllllllll0 200 400 600 800Number of friendsPercentoffriendswhosawpost

43. Audience size predictionOLS regressionModel predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8%Friend count andfeedback0.27 7%

44. 0%20%40%60%80%100%lll l l l l l l l l l l l l l l l l l0 5 10 15Unique friends liking the postPercentoffriendswhosawpostFeedback is not predictiveResultsRapid audience growth until the post receivesfeedback from ﬁve unique friendsPosts with no likes or comments have especiallylarge variance: 90th percentile is 2%–55%

45. Model predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8% of friendcountFriend count andfeedback0.27 7% of friendcountAudience size predictionOLS regression

46. Model predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8% of friendcountFriend count andfeedback0.27 7% of friendcountAudience size predictionOLS regression

47. Model predictors R2 Mean absoluteerrorFriend count 0.12 8% of friendcountFeedback 0.13 8% of friendcountFriend count andfeedback0.27 7% of friendcountEven with access to all user-visible signals,audience size is still unpredictable.Audience size predictionOLS regression

48. How predictable is a user’scumulative audience?Consider the audience for all of a user’s posts over30 days instead of a single post50% of the users in our sample produced ﬁve ormore pieces of content during the month

49. 0%20%40%60%80%100%lllllllllllllllllllllllllllllllll0 200 400 600 800Number of friendsCumulativeaudience(%offriends)61% of friends see at least oneof the user’s posts each monthHowever, actual audience size is still highly variable

50. Discussion

51. Fundamental mismatch betweenperceived and actual audienceHow might a 4x underestimate be impacting userbehavior?Type of content shared, sharing volume, motivationAmbiguous whether a more socially transparentdesign would be desirable

52. Fundamental mismatch betweenperceived and actual audienceHow might a 4x underestimate be impacting userbehavior?Type of content shared, sharing volume, motivationAmbiguous whether a more socially transparentdesign would be desirable

53. Why underestimate audience size?The wishful thinking hypothesis: more comfortableto blame a noisy distribution channel than to blameyourself for writing bad content

54. Why underestimate audience size?The wishful thinking hypothesis: more comfortableto blame a noisy distribution channel than to blameyourself for writing bad contentWhat role might be played by...The availability heuristic?Algorithmic feed ﬁltering?

55. Your invisible audience islarger than you probably think.

56. Your invisible audience islarger than you probably think.Users underestimate audience size by 4xMedian reach is 35% per post and 61% per monthMany want larger audiences but already have them

57. stanford hci groupQuantifying theInvisible Audiencein Social NetworksEytan Bakshy, Moira Burke, Brian KarrerFacebook Data Science TeamMichael BernsteinStanford Computer Science Department

Quantifying the Invisible Audience in Social Networks

Michael Bernstein

Transcript