Peer Ratings in Massive Online Social Networks


Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Peer Ratings in Massive Online Social Networks

  1. 1. Peer Ratings in Massive Online SocialNetworksDmitry ZinovievSuffolk UniversityBostonSunbelt-2012
  2. 2. SUNBELT-2012 / SUFFOLK UNIVERSITY 2To Like or not to Like?● Like/dislike ratings provide instant peer feedback in massiveonline social networks (MOSNs)● Objects subject to ratings: posts; photographs; comments● Questions:➢ How many stops should rating scales have?➢ How do MOSN users perceive the ratings?➢ How do MOSN users react to the ratings?
  3. 3. SUNBELT-2012 / SUFFOLK UNIVERSITY 3Rating Types● Single-valued (Facebook ”Like”, Google+ ”+1”); donot allow negative attitudes● Binary a.k.a. ”Thumbs Up/Down” (Pandora, YahooAnswers); do not allow fuzzy answers● Multivalued (Yahoo, Amazon); may be hard to useif there are many stops; these ratings are ofparticular interest to us
  4. 4. SUNBELT-2012 / SUFFOLK UNIVERSITY 4“Odnoklassniki.Ru”—Our Testing Range● Largest Russian-language social network (100 mln users, 31 mlndaily visits)● Most users have more than one personal picture and reasonablycredible demographics (age/gender)● 6-way ratings (“1” through “5+”*) applicable to personal pictures● Anyone can leave comments to any personal picture of to the profilein general● The system records and displays all profile visits (and visitors)*The grade of “5+” requires a symbolic payment from the grader
  5. 5. SUNBELT-2012 / SUFFOLK UNIVERSITY 5A Profile, as Seen by a VisitorMy PageMy PagePersonal PhotographsPersonal PhotographsDemographicsDemographicsFriendsFriendsProfile PictureProfile Picture
  6. 6. SUNBELT-2012 / SUFFOLK UNIVERSITY 6Picture Rating InterfacePicturePicturePicture CommentsPicture CommentsDemographicsDemographicsThis is a ”5+” pictureThis is a ”5+” pictureYour ratingYour ratingDescriptionDescription
  7. 7. SUNBELT-2012 / SUFFOLK UNIVERSITY 7Method● Select a random avatar Q from the three avatars on the right(30/Female, 40/Male, 30/Male)● Select a random currently logged user and record the users ageA (≥18) and gender G● Select a random grade Rinbetween “1” and “5+” and post it● Record the users response:➢ did the user visited the test profile? (V)➢ did the user post a comment? (C)➢ what is the response grade Rout?●Apologize to the user for a low grade. Posted grades andcomments can be easily removed without any traces
  8. 8. SUNBELT-2012 / SUFFOLK UNIVERSITY 8Descriptive Statistics● 10,799 observations● 3,600 observations per avatar● 1,800 observations per grade● Distribution by gender:➢ 54% female➢ 46% male● Mean age 34Russia in 2009
  9. 9. SUNBELT-2012 / SUFFOLK UNIVERSITY 9Empirical Parametrized Mapping{Rout, V, C} = f (Rin; Q, A, G )Free variableParameters
  10. 10. SUNBELT-2012 / SUFFOLK UNIVERSITY 10Response Details with Trend LinesResponders, by kind and by age:
  11. 11. SUNBELT-2012 / SUFFOLK UNIVERSITY 11Response StatisticsAll RespondersOf them:Visitors GradersRateM 47.0% 39.7% 19.7% 4.8%F 42.4% 31.7% 19.5% 5.0%M -0.1% 0.0% -0.4% -0.1%F 0.5% 0.6% -0.1% 0.1%CommentersAge Trend(%/year)● Visits are the most common responses, followed by grades, followedby comments● Male subjects visit the experimenters profile more often● Older female visitors are more active than younger● Younger male graders are more active than older● Grading/commenting rates are gender-agnostic
  12. 12. SUNBELT-2012 / SUFFOLK UNIVERSITY 12OutGrade Distribution● The grades of “5” and “5+” are grouped (“5+” is special because of itsassociated price)● Response grades have a sharp bi-modal distribution● Only grades “1” and “5” matter!
  13. 13. SUNBELT-2012 / SUFFOLK UNIVERSITY 13OutGrades vs InGrades● Response grades are sharply distributed around “1” and “5” for anystimulus grade (the red lines show best-fit bi-exponentialdistributions)● Only two stops on the scale are necessary!
  14. 14. SUNBELT-2012 / SUFFOLK UNIVERSITY 14RG=R+G● Response Grade = Reciprocity + Generosity➢ Reciprocity: Social norm of in-kind responses to the behavior ofothers (a.k.a. “eye for an eye”) Responding with the same grade➢ Generosity: Habit of giving freely without expecting anything inreturn Responding with a better of worse grade (positivevs negative generosity)● How popular are these mechanisms?
  15. 15. SUNBELT-2012 / SUFFOLK UNIVERSITY 15Reciprocity (1)● Define reciprocity as the “fraction of reciprocally equal grades out ofall grades”:● Calculate reciprocity for each avatar and for each age-gender group● Calculate linear best-fit estimate● Notation:➢ Blue lines for male subjects, red lines for female subjects➢ Solid lines for the 40/M avatar, dashed lines for 30/M, dotted linesfor 30/FRec=N Rin=RoutN
  16. 16. SUNBELT-2012 / SUFFOLK UNIVERSITY 16Reciprocity (2)● Older subjectsare lessreciprocating● Male subjects areon average lessreciprocating● Younger avatarsgenerate morereciprocity fromthe subjects ofthe same gender
  17. 17. SUNBELT-2012 / SUFFOLK UNIVERSITY 17Generosity (1)● Define generosity as the average value of the difference between thestimulus grade and the response grade:● It can be positive and negative● Calculate generosity for each avatar and for each age-gender group● Calculate linear best-fit estimateGen=∑iRout−Ri nN
  18. 18. SUNBELT-2012 / SUFFOLK UNIVERSITY 18Generosity (2)● Older subjectsare moregenerous● Older avatargenerates lessgenerosity● Subjects are lessgenerous to theavatars of thesame gender● Younger avatarsgenerate lessgenerosity fromthe subjects ofthe same gender
  19. 19. SUNBELT-2012 / SUFFOLK UNIVERSITY 19Benevolence (1)● Let negative comments have the value of -1, positive comments—thevalue of 1, and neutral comments—the value of 0. Definebenevolence as the average value all comments● Calculate benevolence for each avatar and for each age-gendergroup● Calculate linear best-fit estimate
  20. 20. SUNBELT-2012 / SUFFOLK UNIVERSITY 20Benevolence (2)● Older subjectsleave bettercomments● Female subjectsleave bettercomments● Younger avatarsget worsecomments fromthe subjects ofthe same gender
  21. 21. SUNBELT-2012 / SUFFOLK UNIVERSITY 21Grading/Commenting Overview (1)Generosity GReciprocity RBenevolence B● Most behaviors are avatar-specific; however, the dependency on ageis universal
  22. 22. SUNBELT-2012 / SUFFOLK UNIVERSITY 22Grading/Commenting Overview (2)● Generosity and benevolence grow with age● Generosity and benevolence are weaker for same-genderavatar-subject pairs● Reciprocity slowly declines with age● Reciprocity is stronger for same-gender avatar-subject pairs● All three values can be approximated using quadratic functions
  23. 23. SUNBELT-2012 / SUFFOLK UNIVERSITY 23Latent Hostility● Age 18–22: subjects areall negative● Age 22–36: subjectscombine positivegenerosity and negativebenevolence: they posthigher grades but leavenegative comments➢ Hypothesis: Higherresponse grades makesubjects feel good, butcomments reveal truefeelings● Age 36–80 subjects are allpositive18–2236–8022–36