The Value of Reputation on
eBay: A Controlled Experiment
         Andrew Berry
           11/25/08
Internet Market
• No outside instrument of reputation
• Temptation for sellers to misrepresent
  products is great
• Tempt...
Internet Reputation Systems
• Necessary to substitute for traditional
  seller reputation mechanisms
• Inform buyers wheth...
Internet and Reputation
• Information can be tallied costlessly on a
  continuous basis
• Written assessments are easily a...
Prior Studies
• Observational studies of a set of items
  whose sellers had varying reputations
• Studies correlate reputa...
Observational Studies
• Can only examine reputation in markets
  for standardized goods
• Plagued by omitted variable bias...
Prior Work
• Shows that reputation affects:
  – Probability of a sale
  – Price
  – Probability that bidders enter an auct...
Confounds with Observational
             Studies
• Private email communications
  – Can influence buyer willingness to bi...
Advantages to Field Experiments
• Automatically controls for confounds
• Ability to investigate reputation for non-
  stan...
eBay Reputation System
• To leave feedback a transaction must
  have occurred
• Buyer and seller can rate each other
• Opp...
eBay Reputation System
• A buyer can click the net score in order to
  see a detailed breakdown
• Scroll to see individual...
General Page
Feedback Scores and Stars
• The Feedback score is the number in
  parentheses next to a member’s user ID
• Next to the Fee...
Feedback Profile
Key Areas
• Positive Feedback Ratings
  – The percentage of positive ratings left by
    members in the last 12 months.
  ...
Feedback Profile
Key Areas
• Recent Feedback Ratings
  – The total number of positive, neutral, and
    negative Feedback ratings the membe...
Feedback Profile
Key Areas
• Detailed Seller Ratings
  – provide more details about this member’s
    performance as a seller
  – Five star...
Feedback Profile
Key Areas
• All Feedback
  – Provides feedback from all transactions
  – Detailed user comments from transaction
    histo...
eBay Reputation Trends
• Half of the buyers provide positive
  feedback
• This positive feedback is similar to saying
  “t...
Halftime
• Thought Questions:
  – 1. What do you think are biggest factors that
    account for so much positive feedback ...
Experimental Setup
• 8 eBay identities
  – STRONG
     • Net score of 2000 with one negative feedback
  – NEW
     • 7 new...
Experimental Setup
• Vintage postcards sold
  – Asymmetry between seller and buyer about
    condition
  – No established ...
Experimental Setup
• To prevent customers from identifying the
  experiment:
  – Lots listed in a category that has thousa...
Second Experiment
• Tested the effects of negative feedback
• 3 week experiment
• Purchased lots from three of the new
  s...
Second Experiment
• Negative feedback was displayed at the
  top of the comments page
• 35 more matched pair lots
Hypotheses
• Hypothesis 1:
  – Buyers will view an established seller as less
    risky and pay more
• Hypothesis 2:
  – N...
Imperfect Observation
• Neither STRONG nor NEW sell
  – Gives little information
• Either STRONG or NEW sells
  – Provides...
Slight Detour
• Censored Normal Regression Models
  – Arise when the variable of interest is
    observable in certain con...
Slight Detour
• Why don’t we just use data where both
  sell?
  – Reduce the sample size too much
  – Truncation Bias
    ...
Results
• Sign Test
  – If STRONG sells but NEW does not, the sign is
    positive
  – If both sell, the observed differen...
Results
• Censored normal estimation
  – Parametric Estimate
  – Used lots where either or both sellers sold
• Estimated m...
Results
• Second experiment shows negatives in a brief
  reputation don’t necessarily hurt revenues
• NEW sellers without ...
Threats to Validity
• Experiment 1
  – Differences in listing quality
  – Repeat customers and private reputation
  – Mult...
Discussion
• Validity of results?
• Is the percentage of negative feedback
  more valuable?
  – Dewally and Edgerington (2...
Discussion
• Given the results, what moral hazards
  does this pose for the structure of the
  eBay market?
Presentation
Upcoming SlideShare
Loading in …5
×

Presentation

203
-1

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
203
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Presentation

  1. 1. The Value of Reputation on eBay: A Controlled Experiment Andrew Berry 11/25/08
  2. 2. Internet Market • No outside instrument of reputation • Temptation for sellers to misrepresent products is great • Temptation to sloth – Example: Ship slowly after receiving payment • Buyers are forced to assume risk – Should lower the price buyers are willing to pay
  3. 3. Internet Reputation Systems • Necessary to substitute for traditional seller reputation mechanisms • Inform buyers whether potential trading partners are trustworthy • Deter opportunistic behavior – Past actions affect future business – Open record of transaction history
  4. 4. Internet and Reputation • Information can be tallied costlessly on a continuous basis • Written assessments are easily assembled • Information can be costlessly transmitted across many customers
  5. 5. Prior Studies • Observational studies of a set of items whose sellers had varying reputations • Studies correlate reputations with auction outcomes • Most studies found that buyers paid more to sellers with better reputations
  6. 6. Observational Studies • Can only examine reputation in markets for standardized goods • Plagued by omitted variable bias – Discussion: What factors could lead to OMVB?
  7. 7. Prior Work • Shows that reputation affects: – Probability of a sale – Price – Probability that bidders enter an auction – Number of bids in the auction – Assessment of a seller’s trustworthiness
  8. 8. Confounds with Observational Studies • Private email communications – Can influence buyer willingness to bid high – Unobservable to researcher • Layout aesthetics • More experience may mean higher quality
  9. 9. Advantages to Field Experiments • Automatically controls for confounds • Ability to investigate reputation for non- standard goods with unavailable book value
  10. 10. eBay Reputation System • To leave feedback a transaction must have occurred • Buyer and seller can rate each other • Opportunity for a one-line text comment • Rated individuals can respond to comments that they feel are unfair
  11. 11. eBay Reputation System • A buyer can click the net score in order to see a detailed breakdown • Scroll to see individual comments • Users may change identity by registering again • No search mechanism to find negatives
  12. 12. General Page
  13. 13. Feedback Scores and Stars • The Feedback score is the number in parentheses next to a member’s user ID • Next to the Feedback score, you may also see a star – A Feedback score of at least 10 earns you a yellow star – The higher the Feedback score, the more positive ratings a member has received – As your Feedback score increases, your star will change color accordingly, all the way to a silver shooting star for a score above 1,000,000
  14. 14. Feedback Profile
  15. 15. Key Areas • Positive Feedback Ratings – The percentage of positive ratings left by members in the last 12 months. – This is calculated by dividing the number of positive ratings by the total number of ratings (positive + neutral + negative).
  16. 16. Feedback Profile
  17. 17. Key Areas • Recent Feedback Ratings – The total number of positive, neutral, and negative Feedback ratings the member has received in the last 1, 6, and 12 months
  18. 18. Feedback Profile
  19. 19. Key Areas • Detailed Seller Ratings – provide more details about this member’s performance as a seller – Five stars is the highest rating, and one star is the lowest – These ratings do not count toward the overall Feedback score and they are anonymous – Sellers cannot trace detailed seller ratings back to the buyer who left them
  20. 20. Feedback Profile
  21. 21. Key Areas • All Feedback – Provides feedback from all transactions – Detailed user comments from transaction history
  22. 22. eBay Reputation Trends • Half of the buyers provide positive feedback • This positive feedback is similar to saying “thank you” in everyday discourse • Sellers receive negative feedback only 1% of the time • Buyers receive negative feedback only 2% of the time
  23. 23. Halftime • Thought Questions: – 1. What do you think are biggest factors that account for so much positive feedback and so little negative feedback? Is the reputation system that good or is there something else at play? – 2. You’ve seen the eBay interface. Is there too much information to digest? What do buyers and sellers actually look at?
  24. 24. Experimental Setup • 8 eBay identities – STRONG • Net score of 2000 with one negative feedback – NEW • 7 new eBay identities with no feedback • Matched 200 items sold by STRONG with one of the new sellers
  25. 25. Experimental Setup • Vintage postcards sold – Asymmetry between seller and buyer about condition – No established book value to guide buyers • 12 week experiment • 5 new sellers presented 20 lots each for sale • 2 sellers presented 50 lots each
  26. 26. Experimental Setup • To prevent customers from identifying the experiment: – Lots listed in a category that has thousands of lots for sale – New sellers had slightly different format for listings – Each half of each matched pair was listed at different times
  27. 27. Second Experiment • Tested the effects of negative feedback • 3 week experiment • Purchased lots from three of the new sellers to give negative feedback • Two categories of negative comments – Item did not match description – Item was in worse condition than listed
  28. 28. Second Experiment • Negative feedback was displayed at the top of the comments page • 35 more matched pair lots
  29. 29. Hypotheses • Hypothesis 1: – Buyers will view an established seller as less risky and pay more • Hypothesis 2: – New sellers with negative feedback will reap lower profits than those without negative feedback • Thoughts on these hypotheses or the experimental setup?
  30. 30. Imperfect Observation • Neither STRONG nor NEW sell – Gives little information • Either STRONG or NEW sells – Provides a lower or upper bound on the ratio of a buyer’s willingness to pay • Both STRONG and NEW sell – Ideal situation
  31. 31. Slight Detour • Censored Normal Regression Models – Arise when the variable of interest is observable in certain conditions – OLS is biased when the variable is unobservable – Use these models when the independent variable is known, but the dependent variable is not • Allows us to include data where either NEW or STRONG sold
  32. 32. Slight Detour • Why don’t we just use data where both sell? – Reduce the sample size too much – Truncation Bias • New sellers sold fewer lots – Observations of sold lots for NEW reflect more extreme points than for STRONG
  33. 33. Results • Sign Test – If STRONG sells but NEW does not, the sign is positive – If both sell, the observed difference is used • One sided sign test approaches significance • Probability of sale was not independent of two sellers • STRONG sold 63% of time • NEW sellers sold 56% of the time
  34. 34. Results • Censored normal estimation – Parametric Estimate – Used lots where either or both sellers sold • Estimated mean difference is significant – P = .044 • Suggests buyers are willing to pay 8.1% more for lots sold by STRONG
  35. 35. Results • Second experiment shows negatives in a brief reputation don’t necessarily hurt revenues • NEW sellers without negatives sold 16 of 35 lots • NEW sellers with negatives sold 14 of 35 lots • No significant differences • Sellers without negatives often received lower prices when they did sell – Favored sellers without negatives 9 times – Favored sellers with negatives 11 times
  36. 36. Threats to Validity • Experiment 1 – Differences in listing quality – Repeat customers and private reputation – Multiple purchases • Experiment 2 – Small sample size – Profile Design – Timing of negative feedback
  37. 37. Discussion • Validity of results? • Is the percentage of negative feedback more valuable? – Dewally and Edgerington (2006) • Do buyers click through profiles or merely rely on overall score? • How do we test if the market is over or underestimating reputation?
  38. 38. Discussion • Given the results, what moral hazards does this pose for the structure of the eBay market?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×