Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Global Budgets for Local Recommendations


Published on

RecSys 2010 Talk on Paper with the same name.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Global Budgets for Local Recommendations

  1. 1. Thomas Sandholm, Hang Ung, Christina Aperjis, Bernardo Huberman<br />Hewlett-Packard, HP Labs, Social Computing Lab<br />RecSys, Barcelona September 27, 2010<br />Global budgets for local recommendations<br />
  2. 2. Why Vote?<br /> How do we get more people to contribute their opinions?<br />
  3. 3.
  4. 4. 500<br />30<br />6<br />Users<br />EXPERIMENTS<br />LOCATIONS<br />Paris<br />Chicago<br />Athens<br />Palo Alto<br />Mumbai<br />Bangalore<br />
  5. 5. Many people consume content – FEW LEAVE OPINIONS<br />May 2008<br />300M+<br />April 2010<br />8M<br />April 2010<br />27M<br />Sample taken<br />Users<br />
  6. 6. RATING Budgets and RANKING<br />Restaurants<br />Restaurants<br />$3<br />Pizza<br />Bob:$10<br />$4<br />Pizza<br /><br /><br />Alice: $11<br />Restaurants<br />$6<br />Burgers<br /><br />Restaurants<br />$8<br />Diners<br /><br />Top Channels<br />Friend Channels<br />$8 Applebee’s<br />$6 Burger King<br />$4 Dominos<br />$3RoundTable<br />$8 Diners<br />$7 Pizza<br />$6 Burgers<br />$6 Burgers<br />$4 Pizza<br />John<br />
  7. 7. RATING REWARDS<br />Alice<br />$5<br /><br />Bob<br />$4<br /><br />A<br />$3<br /><br />B<br />$3<br /><br /><br />$3<br />John<br />Top Rewards<br />(reward factor 10)$40Bob Dominos @ A<br />$30AliceRoundTable @ B<br />
  8. 8. CLICK TO RATING Ratio<br />July 2010<br />4K<br />May 2008<br />300M+<br />April 2010<br />8M<br />April 2010<br />27M<br />Sample taken<br />Users<br />
  9. 9. RECOMMENDATION SUCCESS<br />Success = proportion of query sessions ending with clicks/ratings<br />
  10. 10. System Coverage<br />
  11. 11. Mechanical Turk Experiments<br />Setup-5 Surveys<br />-10 URLs<br />-6 Locations<br />-3 Continents<br />-500 Users<br />Surveys<br />-1-5 Star<br />-Budget<br />-Star Bonus<br />-Budget Bonus<br />-Gloe<br />
  12. 12. Mechanism Results<br />Kendall Tau Rank Correlation (higher better)<br />RMSE (lower better)<br />
  13. 13. Bonus effect on participation<br />Probability of signals<br />Number of surveys taken<br />
  14. 14. Lessons LEARNED<br />Amazon Mechanical Turk<br />Workers not random geographic sample<br />Sensitive to task complexity<br />Respond well to small incentives<br />Budget Mechanism<br />Higher quality recommendations with incentives<br />Social/Economic/Status value extract more opinions<br />Tuned based on usage, e.g. reward factor<br />
  15. 15. FUTURE WORK<br />Projects<br />AfricaMap: map annotation in remote parts for disaster relief UNOSAT/Uni. Geneva<br />Mobile print provider recommendations via HP ePrint<br />Research<br />GSP Auction for commercial bidding<br />LMSR Market for recommendation arbitrage<br />Enhance reward mechanism to both encourage and identify high quality contributions<br />
  16. 16. Papers at<br />Live system at<br />Social Computing Lab<br />THANK YOU<br />
  17. 17. Backup<br />
  18. 18. System Architecture<br />
  19. 19. Scalability<br />
  20. 20. HP Gloe: STATUS<br />~4k* users on Android, iPhone, BlackBerry, WebOS, Web…<br />~7m* recommendations at<br />*Sept 2010<br />
  21. 21. Lessons LEARNED CONTINUED<br />Gloe system<br />Geohash location partitioning simple and efficient<br />HTTP(S) GET/JSON(P) has served us well on all platforms<br />MySQL & Sharded architecture flexible and fast<br />