Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Social (Reputation) System Design:
Opportunities and Challenges
A Practitioners’ View

F Randall Farmer
Rosenfeld Media
(C...
Setting Context:
Who am I to speak to eBay?
Definitions
Why Web Reputation Systems?
Two words:

Information Overload
Searching, organizing, filtering, advertizing—all
built on to...
Challenges and Fallacies
But, if web reputation systems are so important, why do
so many of them suck?
– “Any content repu...
Contexts for Reputation
Individual Context
Community Context
Corporate Context
Industrial Context
Challenges:

Reputation is Individual
Personal Meaning & Utility of Ratings
What does this mean?

Like
What does it actually do?
Karma is Hard
•
•
•
•
•
•
•
•

Karma is useful for building trust between users, and between a user and the site
Karma can...
The FICO Fallacy: Universal Karma
Credit Score Factors
On-time Payments
Capacity Used
Length of Credit
History
Types of Cr...
Public Karma is Identity (and Personal)

Scott McCloud, Understanding Comics, pp. 38.
Words are Weapons: Public Karma
“We built a point system into Consumating
because we thought giving direct feedback to
peo...
However…
Challenges:

Reputation is a Community
Communities Have Temperaments
Collaborative

Caring

Competitive

Cordial

Combative

ALTRUISTIC

COMMERCIAL
EGOCENTRIC
The Virtuous Circle
CREATE CONTENT FOR

THE
COMMUNITY

RANKS
IT FOR

CONTRIBUTORS
THE
SITE

REWARDS AND INCENTIVIZES
The Vicious Spiral
POOR
INCENTIVES
WRONG
FOCUS

POINTS!

ACTIVITY OVER QUALITY
ILLBEHAVIOR

FACTIONS

GRINDING, POINT WHOR...
Tragedy of the Commons
Challenges:

Reputation is Corporate
The Corporate Virtuous Circle (eBay)
CREATE LISTINGS,
SELL ITEMS
BUYERS

BUY,
EVALUATE
SELLERS

SELLERS
EBAY
(THE CORPORAT...
Yelp: Multi-sided Market…
Yelp: Multi-sided Market with Many
Contexts
eBay as Multi-Sided Market:
Drop-shippers and Feedback

When is this true? 100 – 5 < 0

What is a Negative Feedback worth?...
Google’s Virtuous Circle
CREATE WEB PAGES FOR
CONSUMERS
(SEARCHERS &
LINKERS)

PROVIDE
INPUTS
TO RANK

PUBLISHERS
SEARCH R...
Challenges:

Reputation is an Industry
Challenge: Systems that model real life
Honestly.com? Not so much.
Fallacy: Karma as influence & currency
Our Ultimate Challenge:

Short Institutional Memory
From Fuzzy to Focused
A++++++ audience, would visit again…
Randy Farmer
@frandallfarmer
Randy.Farmer@pobox.com
http://about.me/frandallfarmer
Upcoming SlideShare
Loading in …5
×

Reputation Systems Talk at Ebay

906 views

Published on

A variation of my Trust Symposium Keynote given at eBay, where I do poke a bit at the venerable "Feedback" system.

  • Be the first to comment

Reputation Systems Talk at Ebay

  1. 1. Social (Reputation) System Design: Opportunities and Challenges A Practitioners’ View F Randall Farmer Rosenfeld Media (Coauthor: Bryce Glass) @eBay, November 14th, 2013
  2. 2. Setting Context: Who am I to speak to eBay?
  3. 3. Definitions
  4. 4. Why Web Reputation Systems? Two words: Information Overload Searching, organizing, filtering, advertizing—all built on top of web reputation systems—is driving an economy worth $ trillions.
  5. 5. Challenges and Fallacies But, if web reputation systems are so important, why do so many of them suck? – “Any content reputation system is better than none.” – “Content quality is a rich-man’s problem.” – “The other guys have thumbs-up/down, we should too.” – “What cool tools do we have on the shelf?” Time-to-market trumps correctness, testing, and even research. Reputation is not, primarily, a technical problem.
  6. 6. Contexts for Reputation Individual Context Community Context Corporate Context Industrial Context
  7. 7. Challenges: Reputation is Individual
  8. 8. Personal Meaning & Utility of Ratings What does this mean? Like What does it actually do?
  9. 9. Karma is Hard • • • • • • • • Karma is useful for building trust between users, and between a user and the site Karma can be an incentive for participation and contributions Karma is contextual and has limited utility globally. Karma comes in several flavors - Participation, Quality and Robust Karma is complex (via indirect evaluations), and formulation is often opaque Personal karma is displayed only to the owner, good for measuring progress Corporate karma is used by the site to find the very best and very worst users Public karma is displayed to other users, which makes it the hardest to get right – It should be used sparingly – It is hard to understand, isn't expected, and easily confused with content ratings. – It shouldn't be have a socially negative value. – It encourages competition in some users, and may discourage others
  10. 10. The FICO Fallacy: Universal Karma Credit Score Factors On-time Payments Capacity Used Length of Credit History Types of Credit Used Past Credit Applications
  11. 11. Public Karma is Identity (and Personal) Scott McCloud, Understanding Comics, pp. 38.
  12. 12. Words are Weapons: Public Karma “We built a point system into Consumating because we thought giving direct feedback to people about their conduct on the site would encourage them to be nice to one another— you get a thumbs up when you are nice (treat!), and a thumbs down when you are a douche (electric shock!)” Ben Brown, Internet Rockstar
  13. 13. However…
  14. 14. Challenges: Reputation is a Community
  15. 15. Communities Have Temperaments Collaborative Caring Competitive Cordial Combative ALTRUISTIC COMMERCIAL EGOCENTRIC
  16. 16. The Virtuous Circle CREATE CONTENT FOR THE COMMUNITY RANKS IT FOR CONTRIBUTORS THE SITE REWARDS AND INCENTIVIZES
  17. 17. The Vicious Spiral POOR INCENTIVES WRONG FOCUS POINTS! ACTIVITY OVER QUALITY ILLBEHAVIOR FACTIONS GRINDING, POINT WHORING BEST USERS LEAVE DEPARTURES BAD NORMS “HOW IS BABBY FORMED!?”
  18. 18. Tragedy of the Commons
  19. 19. Challenges: Reputation is Corporate
  20. 20. The Corporate Virtuous Circle (eBay) CREATE LISTINGS, SELL ITEMS BUYERS BUY, EVALUATE SELLERS SELLERS EBAY (THE CORPORATION) REWARDS AND INCENTIVIZES
  21. 21. Yelp: Multi-sided Market…
  22. 22. Yelp: Multi-sided Market with Many Contexts
  23. 23. eBay as Multi-Sided Market: Drop-shippers and Feedback When is this true? 100 – 5 < 0 What is a Negative Feedback worth? (in U.S. Dollars)
  24. 24. Google’s Virtuous Circle CREATE WEB PAGES FOR CONSUMERS (SEARCHERS & LINKERS) PROVIDE INPUTS TO RANK PUBLISHERS SEARCH RESULTS • SALES • ADS • ATTENTION • WHATEVER… (VIA PAGERANK) PROVIDES TRAFFIC BACK TO ADVERTISING
  25. 25. Challenges: Reputation is an Industry
  26. 26. Challenge: Systems that model real life
  27. 27. Honestly.com? Not so much.
  28. 28. Fallacy: Karma as influence & currency
  29. 29. Our Ultimate Challenge: Short Institutional Memory
  30. 30. From Fuzzy to Focused
  31. 31. A++++++ audience, would visit again… Randy Farmer @frandallfarmer Randy.Farmer@pobox.com http://about.me/frandallfarmer

×