Successfully reported this slideshow.

Peer Ratings in Massive Online Social Networks

2

Share

1 of 24
1 of 24

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Peer Ratings in Massive Online Social Networks

  1. 1. Peer Ratings in Massive Online Social Networks Dmitry Zinoviev Suffolk University Boston Sunbelt-2012
  2. 2. SUNBELT-2012 / SUFFOLK UNIVERSITY 2 To Like or not to Like? ● Like/dislike ratings provide instant peer feedback in massive online social networks (MOSNs) ● Objects subject to ratings: posts; photographs; comments ● Questions: ➢ How many stops should rating scales have? ➢ How do MOSN users perceive the ratings? ➢ How do MOSN users react to the ratings?
  3. 3. SUNBELT-2012 / SUFFOLK UNIVERSITY 3 Rating Types ● Single-valued (Facebook ”Like”, Google+ ”+1”); do not allow negative attitudes ● Binary a.k.a. ”Thumbs Up/Down” (Pandora, Yahoo Answers); do not allow fuzzy answers ● Multivalued (Yahoo, Amazon); may be hard to use if there are many stops; these ratings are of particular interest to us
  4. 4. SUNBELT-2012 / SUFFOLK UNIVERSITY 4 “Odnoklassniki.Ru”—Our Testing Range ● Largest Russian-language social network (100 mln users, 31 mln daily visits) ● Most users have more than one personal picture and reasonably credible demographics (age/gender) ● 6-way ratings (“1” through “5+”*) applicable to personal pictures ● Anyone can leave comments to any personal picture of to the profile in general ● The system records and displays all profile visits (and visitors) *The grade of “5+” requires a symbolic payment from the grader
  5. 5. SUNBELT-2012 / SUFFOLK UNIVERSITY 5 A Profile, as Seen by a Visitor My PageMy Page Personal PhotographsPersonal Photographs DemographicsDemographics FriendsFriends Profile PictureProfile Picture
  6. 6. SUNBELT-2012 / SUFFOLK UNIVERSITY 6 Picture Rating Interface PicturePicture Picture CommentsPicture Comments DemographicsDemographics This is a ”5+” pictureThis is a ”5+” picture Your ratingYour rating DescriptionDescription
  7. 7. SUNBELT-2012 / SUFFOLK UNIVERSITY 7 Method ● Select a random avatar Q from the three avatars on the right (30/Female, 40/Male, 30/Male) ● Select a random currently logged user and record the user's age A (≥18) and gender G ● Select a random grade Rin between “1” and “5+” and post it ● Record the user's response: ➢ did the user visited the test profile? (V) ➢ did the user post a comment? (C) ➢ what is the response grade Rout ? ● Apologize to the user for a low grade. Posted grades and comments can be easily removed without any traces
  8. 8. SUNBELT-2012 / SUFFOLK UNIVERSITY 8 Descriptive Statistics ● 10,799 observations ● 3,600 observations per avatar ● 1,800 observations per grade ● Distribution by gender: ➢ 54% female ➢ 46% male ● Mean age 34 Russia in 2009
  9. 9. SUNBELT-2012 / SUFFOLK UNIVERSITY 9 Empirical Parametrized Mapping {Rout , V, C} = f (Rin ; Q, A, G ) Free variable Parameters
  10. 10. SUNBELT-2012 / SUFFOLK UNIVERSITY 10 Response Details with Trend Lines Responders, by kind and by age:
  11. 11. SUNBELT-2012 / SUFFOLK UNIVERSITY 11 Response Statistics All Responders Of them: Visitors Graders Rate M 47.0% 39.7% 19.7% 4.8% F 42.4% 31.7% 19.5% 5.0% M -0.1% 0.0% -0.4% -0.1% F 0.5% 0.6% -0.1% 0.1% Commenters Age Trend (%/year) ● Visits are the most common responses, followed by grades, followed by comments ● Male subjects visit the experimenter's profile more often ● Older female visitors are more active than younger ● Younger male graders are more active than older ● Grading/commenting rates are gender-agnostic
  12. 12. SUNBELT-2012 / SUFFOLK UNIVERSITY 12 OutGrade Distribution ● The grades of “5” and “5+” are grouped (“5+” is special because of its associated price) ● Response grades have a sharp bi-modal distribution ● Only grades “1” and “5” matter!
  13. 13. SUNBELT-2012 / SUFFOLK UNIVERSITY 13 OutGrades vs InGrades ● Response grades are sharply distributed around “1” and “5” for any stimulus grade (the red lines show best-fit bi-exponential distributions) ● Only two stops on the scale are necessary!
  14. 14. SUNBELT-2012 / SUFFOLK UNIVERSITY 14 RG=R+G ● Response Grade = Reciprocity + Generosity ➢ Reciprocity: Social norm of in-kind responses to the behavior of others (a.k.a. “eye for an eye”)  Responding with the same grade ➢ Generosity: Habit of giving freely without expecting anything in return  Responding with a better of worse grade (positive vs negative generosity) ● How popular are these mechanisms?
  15. 15. SUNBELT-2012 / SUFFOLK UNIVERSITY 15 Reciprocity (1) ● Define reciprocity as the “fraction of reciprocally equal grades out of all grades”: ● Calculate reciprocity for each avatar and for each age-gender group ● Calculate linear best-fit estimate ● Notation: ➢ Blue lines for male subjects, red lines for female subjects ➢ Solid lines for the 40/M avatar, dashed lines for 30/M, dotted lines for 30/F Rec= N Rin=Rout N
  16. 16. SUNBELT-2012 / SUFFOLK UNIVERSITY 16 Reciprocity (2) ● Older subjects are less reciprocating ● Male subjects are on average less reciprocating ● Younger avatars generate more reciprocity from the subjects of the same gender
  17. 17. SUNBELT-2012 / SUFFOLK UNIVERSITY 17 Generosity (1) ● Define generosity as the average value of the difference between the stimulus grade and the response grade: ● It can be positive and negative ● Calculate generosity for each avatar and for each age-gender group ● Calculate linear best-fit estimate Gen=∑ i Rout−Ri n N
  18. 18. SUNBELT-2012 / SUFFOLK UNIVERSITY 18 Generosity (2) ● Older subjects are more generous ● Older avatar generates less generosity ● Subjects are less generous to the avatars of the same gender ● Younger avatars generate less generosity from the subjects of the same gender
  19. 19. SUNBELT-2012 / SUFFOLK UNIVERSITY 19 Benevolence (1) ● Let negative comments have the value of -1, positive comments—the value of 1, and neutral comments—the value of 0. Define benevolence as the average value all comments ● Calculate benevolence for each avatar and for each age-gender group ● Calculate linear best-fit estimate
  20. 20. SUNBELT-2012 / SUFFOLK UNIVERSITY 20 Benevolence (2) ● Older subjects leave better comments ● Female subjects leave better comments ● Younger avatars get worse comments from the subjects of the same gender
  21. 21. SUNBELT-2012 / SUFFOLK UNIVERSITY 21 Grading/Commenting Overview (1) Generosity G Reciprocity R Benevolence B ● Most behaviors are avatar-specific; however, the dependency on age is universal
  22. 22. SUNBELT-2012 / SUFFOLK UNIVERSITY 22 Grading/Commenting Overview (2) ● Generosity and benevolence grow with age ● Generosity and benevolence are weaker for same-gender avatar-subject pairs ● Reciprocity slowly declines with age ● Reciprocity is stronger for same-gender avatar-subject pairs ● All three values can be approximated using quadratic functions
  23. 23. SUNBELT-2012 / SUFFOLK UNIVERSITY 23 Latent Hostility ● Age 18–22: subjects are all negative ● Age 22–36: subjects combine positive generosity and negative benevolence: they post higher grades but leave negative comments ➢ Hypothesis: Higher response grades make subjects feel good, but comments reveal true feelings ● Age 36–80 subjects are all positive 18–22 36–80 22–36
  24. 24. SUNBELT-2012 / SUFFOLK UNIVERSITY 24 THANK YOU!

×