Bayesian truth-serum
•Download as PPTX, PDF•
0 likes•450 views
1. Bayesian Truth Serum (BTS) is a survey method that incentivizes truthful responses by having respondents provide their own answers to questions as well as estimates of what percentage of others would answer. 2. BTS scores responses based on how surprisingly common a respondent's answer is compared to their estimated distribution of others' answers. Answers that are more common than estimated receive higher scores, incentivizing truthful responses. 3. The premise of BTS is that individuals are generally best at estimating what beliefs and experiences are most common among others. By revealing discrepancies between personal responses and population estimates, BTS can identify untruthful responses.
More Related Content
Similar to Bayesian truth-serum
Similar to Bayesian truth-serum (20)
Recently uploaded
Recently uploaded (20)
Bayesian truth-serum
- 1. Introduction to Bayesian Truth Serum (BTS) presented by Fuming Shih
- 2. Ask for true opinion? • Will you buy Samsung Galaxy S3 when it comes out? (Yes/No) • Will you vote in the next presidential election? – (definitely/probably/probably not/definitely not) • Have you had more than 20 sexual partners over the past year (Yes/No)
- 3. What is BTS? • Survey scoring method that provides truth- telling incentives for respondents answering multiple-choice questions • Respondents to supply not only their own answers, but also percentage estimates of others’ answers. • The formula then assigns high scores to answers that are surprisingly common A Bayesian Truth Serum for Subjective Data by Drazen Prelec Science 15 October 2004: Vol. 306 no. 5695 pp. 462-466
- 4. BTS simplified • “The premise behind this approach is the following. If people truly hold a particular belief, they are more likely to think that others agree or have had similar experiences.” • you are your best estimator – or your estimation reveals you – posterior probability You your estimation the unknown world (distribution of different opinions)
- 6. How it works reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 7. Calculate BTS Score reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 8. The Information score: measures surprisingly common ex. log(0.15/0.05) reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 9. prediction score measures prediction accuracy equals zero for a perfect prediction reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 10. Conclusion First • The best strategy for the respondent is to tell the truth Your preference “wins” to the extent that it is more popular than collectively estimated reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 11. The intuitive argument for m=2 reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 12. and I happen to like Red reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 13. This is my best estimate of the Red share (e.g., 50%) reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 14. Bayesian reasoning implies that someone who likes White will estimate a smaller share for Red reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 15. The average predicted share for Red will fall somewhere between these two estimates reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 16. Hence, if I like Red I should believe that the share for Red will be underestimated or ‘surprisingly popular’ My prediction of the average Red share estimate reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 17. The argument holds even if I know that my preferences are unusual reference: http://internetconferences.net/ipsi/files/CroatiaPrelecVIPSI.pdf
- 18. Application? • Honest signals subjective preferences – BTS draws more truth opinions from the users – reality mining captures the objective ground truths • Are there relations between these two? – I feel stressful when multiple people around me – I feel depressed when I am alone • A improvement on psychological-social probe – developing an opinion probe on funf-framework – capture preferences and context at the same time
Editor's Notes
- The algorithm assigns more points to responses to answers that are "surprisingly common", that is, answers that are more common that collectively predicted. For example, let's say you are being asked about which political candidate you support. A candidate who is chosen (in the first question) by 10% of the respondents, but only predicted as being chosen (the second question) by 5% of the respondents is a surprisingly common answer. This technique gets more true opinions because it is believed that people systematically believe that their own views are unique, and hence will underestimate the degree to which other people will predict their own true views.
- example:X_bar = 0.15Y_bar = 0.05 information score = log(3)The Information score measures whether ananswer is surprisingly common
- example:X_bar = 0.15Y_bar = 0.05 information score = log(3)The Information score measures whether ananswer is surprisingly common