Thomas Sandholm, Hang Ung, Christina Aperjis, Bernardo Huberman<br />Hewlett-Packard, HP Labs,  Social Computing Lab<br />...
Why Vote?<br /> How do we get more people to contribute their opinions?<br />
500<br />30<br />6<br />Users<br />EXPERIMENTS<br />LOCATIONS<br />Paris<br />Chicago<br />Athens<br />Palo Alto<br />Mumb...
Many people consume content – FEW LEAVE OPINIONS<br />May 2008<br />300M+<br />April 2010<br />8M<br />April 2010<br />27M...
RATING Budgets and RANKING<br />Restaurants<br />Restaurants<br />$3<br />Pizza<br />Bob:$10<br />$4<br />Pizza<br />http:...
RATING REWARDS<br />Alice<br />$5<br />http://dominos.com<br />Bob<br />$4<br />http://dominos.com<br />A<br />$3<br />htt...
CLICK TO RATING Ratio<br />July 2010<br />4K<br />May 2008<br />300M+<br />April 2010<br />8M<br />April 2010<br />27M<br ...
RECOMMENDATION SUCCESS<br />Success = proportion of query sessions ending with clicks/ratings<br />
System Coverage<br />
Mechanical Turk Experiments<br />Setup-5 Surveys<br />-10 URLs<br />-6 Locations<br />-3 Continents<br />-500 Users<br />S...
Mechanism Results<br />Kendall Tau Rank Correlation (higher better)<br />RMSE (lower better)<br />
Bonus effect on participation<br />Probability of signals<br />Number of surveys taken<br />
Lessons LEARNED<br />Amazon Mechanical Turk<br />Workers not random geographic sample<br />Sensitive to task complexity<br...
FUTURE WORK<br />Projects<br />AfricaMap: map annotation in remote parts for disaster relief UNOSAT/Uni. Geneva<br />Mobil...
Papers at www.hpl.hp.com/research/scl<br />Live system at www.hpgloe.com<br />Social Computing Lab<br />THANK YOU<br />
Backup<br />
System Architecture<br />
Scalability<br />
HP Gloe: STATUS<br />~4k* users on Android, iPhone, BlackBerry, WebOS, Web…<br />~7m* recommendations at http://hpgloe.com...
Lessons LEARNED CONTINUED<br />Gloe system<br />Geohash location partitioning simple and efficient<br />HTTP(S) GET/JSON(P...
Upcoming SlideShare
Loading in...5
×

Global Budgets for Local Recommendations

730

Published on

RecSys 2010 Talk on Paper with the same name.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
730
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Global Budgets for Local Recommendations

  1. 1. Thomas Sandholm, Hang Ung, Christina Aperjis, Bernardo Huberman<br />Hewlett-Packard, HP Labs, Social Computing Lab<br />RecSys, Barcelona September 27, 2010<br />Global budgets for local recommendations<br />
  2. 2. Why Vote?<br /> How do we get more people to contribute their opinions?<br />
  3. 3.
  4. 4. 500<br />30<br />6<br />Users<br />EXPERIMENTS<br />LOCATIONS<br />Paris<br />Chicago<br />Athens<br />Palo Alto<br />Mumbai<br />Bangalore<br />
  5. 5. Many people consume content – FEW LEAVE OPINIONS<br />May 2008<br />300M+<br />April 2010<br />8M<br />April 2010<br />27M<br />Sample taken<br />Users<br />
  6. 6. RATING Budgets and RANKING<br />Restaurants<br />Restaurants<br />$3<br />Pizza<br />Bob:$10<br />$4<br />Pizza<br />http://roundtable.com<br />http://dominos.com<br />Alice: $11<br />Restaurants<br />$6<br />Burgers<br />http://burgerking.com<br />Restaurants<br />$8<br />Diners<br />http://applebees.com<br />Top Channels<br />Friend Channels<br />$8 Applebee’s<br />$6 Burger King<br />$4 Dominos<br />$3RoundTable<br />$8 Diners<br />$7 Pizza<br />$6 Burgers<br />$6 Burgers<br />$4 Pizza<br />John<br />
  7. 7. RATING REWARDS<br />Alice<br />$5<br />http://dominos.com<br />Bob<br />$4<br />http://dominos.com<br />A<br />$3<br />http://roundtable.com<br />B<br />$3<br />http://roundtable.com<br />http://roundtable.com<br />$3<br />John<br />Top Rewards<br />(reward factor 10)$40Bob Dominos @ A<br />$30AliceRoundTable @ B<br />
  8. 8. CLICK TO RATING Ratio<br />July 2010<br />4K<br />May 2008<br />300M+<br />April 2010<br />8M<br />April 2010<br />27M<br />Sample taken<br />Users<br />
  9. 9. RECOMMENDATION SUCCESS<br />Success = proportion of query sessions ending with clicks/ratings<br />
  10. 10. System Coverage<br />
  11. 11. Mechanical Turk Experiments<br />Setup-5 Surveys<br />-10 URLs<br />-6 Locations<br />-3 Continents<br />-500 Users<br />Surveys<br />-1-5 Star<br />-Budget<br />-Star Bonus<br />-Budget Bonus<br />-Gloe<br />
  12. 12. Mechanism Results<br />Kendall Tau Rank Correlation (higher better)<br />RMSE (lower better)<br />
  13. 13. Bonus effect on participation<br />Probability of signals<br />Number of surveys taken<br />
  14. 14. Lessons LEARNED<br />Amazon Mechanical Turk<br />Workers not random geographic sample<br />Sensitive to task complexity<br />Respond well to small incentives<br />Budget Mechanism<br />Higher quality recommendations with incentives<br />Social/Economic/Status value extract more opinions<br />Tuned based on usage, e.g. reward factor<br />
  15. 15. FUTURE WORK<br />Projects<br />AfricaMap: map annotation in remote parts for disaster relief UNOSAT/Uni. Geneva<br />Mobile print provider recommendations via HP ePrint<br />Research<br />GSP Auction for commercial bidding<br />LMSR Market for recommendation arbitrage<br />Enhance reward mechanism to both encourage and identify high quality contributions<br />
  16. 16. Papers at www.hpl.hp.com/research/scl<br />Live system at www.hpgloe.com<br />Social Computing Lab<br />THANK YOU<br />
  17. 17. Backup<br />
  18. 18. System Architecture<br />
  19. 19. Scalability<br />
  20. 20. HP Gloe: STATUS<br />~4k* users on Android, iPhone, BlackBerry, WebOS, Web…<br />~7m* recommendations at http://hpgloe.com<br />*Sept 2010<br />
  21. 21. Lessons LEARNED CONTINUED<br />Gloe system<br />Geohash location partitioning simple and efficient<br />HTTP(S) GET/JSON(P) has served us well on all platforms<br />MySQL & Sharded architecture flexible and fast<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×