Your SlideShare is downloading. ×
The Many Faces of Reputation: Towards a discipline of Web 2.0  reputation system design
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

The Many Faces of Reputation: Towards a discipline of Web 2.0 reputation system design

1,565

Published on

Published in: Education, Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,565
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
85
Comments
0
Likes
5
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. The Many Faces of Reputation: Towards a discipline of Web 2.0 reputation system design Prof. Chris Dellarocas Robert H. Smith School of Business University of Maryland cdell@umd.edu Copyright © C. Dellarocas 2008-09 Suddenly everything Web 2.0 became so sexy… Copyright © C. Dellarocas 2008-09 1
  • 2. Except, perhaps, reputation systems… RepuNomics How Online Reputation Enables Markets Sustains Communities This and Creates New Business book Opportunities does not exist Copyright © C. Dellarocas 2008-09 But reputation systems are everywhere eCommerce eBay Amazon Epinions Web 2.0 Yelp Slashdot Crowdsourcing Yahoo! Answers iStockPhoto Gaming Xbox Live Copyright © C. Dellarocas 2008-09 2
  • 3. Reputation Systems are the Unsung Heroes of the Web Copyright © C. Dellarocas 2008-09 Outline of this talk Reputation Systems Serve Business Objectives Four areas where practice runs ahead of research Our challenge: Developing a rigorous discipline of real-life reputation system design Copyright © C. Dellarocas 2008-09 3
  • 4. Some Definitions Reputation = a summary of one’s past actions defined within the context of a specific community Reputation system = a system that mediates and automates the process of assessing one’s reputation Keeps track of a user’s actions Aggregates and displays summary statistics Copyright © C. Dellarocas 2008-09 Bird’s eye view of a reputation system (Inter) actions Inputs Aggregation Outputs Users Artifacts COMMUNITY Copyright © C. Dellarocas 2008-09 4
  • 5. Reputation Systems serve a variety of Business Objectives Copyright © C. Dellarocas 2008-09 Why do communities need reputation systems? Trust Encourage “good” and discourage “bad” behavior Quality Provide incentives for quality contributions Recognize best contributors Matching and Filtering Assist users in finding suitable partners Reduce information overload Participation and loyalty Give user reasons to join and stay in your community Copyright © C. Dellarocas 2008-09 5
  • 6. Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 6
  • 7. Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 7
  • 8. Copyright © C. Dellarocas 2008-09 Four examples Trust Quality Filtering Loyalty eBay +++ + + ++ Amazon ++ +++ + ++ Yelp ++ +++ +++ +++ Yahoo!Answers ++ ++ + +++ Copyright © C. Dellarocas 2008-09 8
  • 9. The design space of Reputation Systems is very rich Copyright © C. Dellarocas 2008-09 Four dimensions of reputation system design where practice runs ahead of research What inputs should be solicited 1. What outputs should be presented 2. How transparent should the rules be 3. How should reputation evolve over time 4. Copyright © C. Dellarocas 2008-09 9
  • 10. What inputs should be solicited? What internal actions to keep track of What external feedback to solicit Ratings of artifacts vs. ratings of people Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 10
  • 11. Determining inputs Perform an audit of your system List all actions a user can take Assess how each action relates to each of the four classes of objectives Track/report actions that: you want to encourage/discourage provide the most information related to one or more of your key objectives Copyright © C. Dellarocas 2008-09 A story of a site that got it wrong: Consumating.com Copyright © C. Dellarocas 2008-09 11
  • 12. An interesting question: Should you allow ratings of users…. … separately from the artifacts they create? Pluses: Makes a person feel like a person… which might increase site loyalty Minuses: Can distract from quality of one’s work Might encourage personal attacks Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 12
  • 13. Four dimensions of reputation system design where practice runs ahead of research What inputs should be solicited 1. How outputs should be presented 2. How transparent should the rules be 3. How should reputation evolve over time 4. Copyright © C. Dellarocas 2008-09 What outputs should be presented? Simple statistics Star ratings Score Named levels Achievement badges Social network Copyright © C. Dellarocas 2008-09 13
  • 14. Copyright © C. Dellarocas 2008-09 Simple Statistics Copyright © C. Dellarocas 2008-09 14
  • 15. Statistics Score Copyright © C. Dellarocas 2008-09 Social Network Copyright © C. Dellarocas 2008-09 15
  • 16. Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 16
  • 17. Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 17
  • 18. Copyright © C. Dellarocas 2008-09 To what extent should the system allow comparison across users? Display of relative performance statistics Top N lists “Leaderboards” Copyright © C. Dellarocas 2008-09 18
  • 19. Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 19
  • 20. Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 20
  • 21. Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 21
  • 22. To compare or not to compare? Pluses Increases incentives to contribute Enhances filtering role of reputation Minuses Instills a culture of competition Obsession with rankings might lead to manipulative, disruptive behavior Low ranked users might be discouraged and exit Copyright © C. Dellarocas 2008-09 A cautionary tale Digg.com Copyright © C. Dellarocas 2008-09 22
  • 23. How you display reputation information affects the spirit of your community COMPETITIVE CORDIAL Named Ranking Statistical Levels Point-based score Leaderboards evidence Badges Reputation system design must be consistent with the community’s overall culture Copyright © C. Dellarocas 2008-09 Some ideas Implement multiple ways of ranking users No single measure that people obsess on Allows users with different qualities to feel good Particularly relevant in systems where matching and filtering is an important objective Copyright © C. Dellarocas 2008-09 23
  • 24. Copyright © C. Dellarocas 2008-09 Four dimensions of reputation system design where practice runs ahead of research What inputs should be solicited 1. What outputs should be presented 2. How transparent should the rules be 3. How should reputation evolve over time 4. Copyright © C. Dellarocas 2008-09 24
  • 25. How transparent should be the aggregation rules? Transparency breeds trust … but also facilitates gaming Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 25
  • 26. Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 26
  • 27. Copyright © C. Dellarocas 2008-09 Four dimensions of reputation system design where practice runs ahead of research What inputs should be solicited 1. What outputs should be presented 2. How transparent should the rules be 3. How should reputation evolve over time 4. Copyright © C. Dellarocas 2008-09 27
  • 28. How should reputation evolve over time? Accumulate and grow Decay and refresh Tradeoffs Accumulation breeds complacency Accumulation discourages new entrants BUT Accumulation promotes loyalty and lock-in Copyright © C. Dellarocas 2008-09 Accumulation Decay and Refresh Copyright © C. Dellarocas 2008-09 28
  • 29. A case study: Yelp Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 29
  • 30. Copyright © C. Dellarocas 2008-09 Copyright © C. Dellarocas 2008-09 30
  • 31. Copyright © C. Dellarocas 2008-09 The challenge: Mapping Business Objectives to Design Copyright © C. Dellarocas 2008-09 31
  • 32. Observation #1: Everything should be driven from a clear understanding of objectives Trust building/Community Policing Incentives for Quality Matching and Filtering Site Loyalty It is important for the designer to be very clear with respect to how these goals are prioritized It is important to understand how every aspect of design affects each of these goals Copyright © C. Dellarocas 2008-09 Observation #2: The design of a reputation system can affect a community’s culture COMPETITIVE CORDIAL Named Ranking Statistical Levels Point-based score Leaderboards evidence Badges Reputation system design must be consistent with the community’s overall culture Copyright © C. Dellarocas 2008-09 32
  • 33. Observation #3: Intangible aspects of reputation matter Users care about their reputation beyond the tangible benefits it confers to them Reputation as pure status Status is zero-sum Users who are not recognized might get upset So, introduction of a reputation system might make some users worse off and cause them to leave Copyright © C. Dellarocas 2008-09 Observation #4: A reputation system can be a site’s competitive weapon A way to lock-in users to our site Lock-in properties of reputation system can sometimes be in conflict with its other objectives Understanding the design of competitors’ reputation mechanisms matters Copyright © C. Dellarocas 2008-09 33
  • 34. In summary Much of what we know about reputation system design in Web 2.0 communities is still anecdotal We need research-driven guidelines on how to design such systems A lot of open areas where theoretical and experimental work needs to be done Copyright © C. Dellarocas 2008-09 In the meantime… Reputation systems designers will be the poor lonesome cowboys of the Web Not necessarily such a bad place to be (hey, it can get you to places like Gargonza) Copyright © C. Dellarocas 2008-09 34

×