Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)

1,855 views

Published on

Talk given at Recsys Challenge Workshop in Dublin (@ ACM Recsys 2012), on 13.09.2012.

Published in: Technology
  • Be the first to comment

From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)

  1. 1. From a toolkit ofrecommendation algorithmsinto a real business:the Gravity R&D experience13.09.2012.
  2. 2. The kick-start2 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  3. 3. Facing with real needs What we had What clients wanted • rating prediction algorithms • recommendations that • coded in various languages bring revenue • blending mechanism • robustness • accuracy oriented • low response time • easy integration • reporting3 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  4. 4. What we do? users content of service provider recommender4 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  5. 5. Explicit vs implicit feedback No ratings but interactions sparse vs. dense matrix requires different learning5 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  6. 6. Increase revenue: A/B tests against the original solution internally6 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  7. 7. Robustness Management LAN SNMP Nagios Monitoring HP OpenView Aggregator HTTP HTTP Platform OSS/BSS / SQL / SQL IMPRESS IMPRESS SOAP Application Server #1 Application Server #2 IMPRESS Frontend web server #1 Backend LAN Reco LAN HTTP Load Balancer HTTP(S) Firewall SQL SQL CSV over FTP TV Service LAN IMPRESS Frontend web server #2 Database #1 Database #2Reporting Subsystem End users7 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  8. 8. Time requirements • Response time: few ms (max 200) • Training time: maximum few hours • regular retraining • incremental training • Newsletters: • nightly batch run8 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  9. 9. Productization IMPRESS RECO AD•APT for for for IPTV, CATV and satellite e-commerce ad networks and ad server providers Recommends Recommends Recommends Personally Personally Relevant Relevant Personally Relevant products & services ads Linear TV, VOD, catch-up TV and more Gravity personalization platform9 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  10. 10. The 5% question – Importance of UI Francisco Martin (Strands): „the algorithm is only 5% in the success of the recommender system” • placement  below or above the fold  scrolling  easy to recognize  floating in • title  not misleading  explanation like • widget  carrousel  static10 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  11. 11. Recommendation scenario Item2Item recommendation logic: the ad’s profile will be matched to the profile model of available ads11 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  12. 12. Marketing channels Changing the order of two boxes: 25% CTR increase12 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  13. 13. Cannibalization • Goal: increase user engagement • Measurements • average visit length • average page views • Effect of accurate recommendations: • use of listing page ↓ • use of item page ↑ • Overall page view: remains the same • Secondary measurements • Contacting • CTR increase13 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  14. 14. Evolution: increased user engagement • not a cold start problem • parameter optimization and user engagement14 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  15. 15. KPIs – may change during testing15 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  16. 16. Complete personalization: coupon-world • Newsletter (daily + occassionally) • Ranking all offers on the website • top1 item • category preferences • user metadata (gender, age, …) • user category preferences (seldom given) • item metadata • context • customer vs. vendor16 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  17. 17. Business rules – driving/overriding ranking17 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  18. 18. Contexts18 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  19. 19. Context at TV program recommendation • TV (EPG program & video-on-demand)  explicit and implicit identification of the user in the household  time-dependent recommendation19 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  20. 20. (offline) Some results (online) Improvement using season iTALS iTALSx Dataset Recall@20 MAP@20 Recall@20 MAP@20 Grocery 64,31% 137,96% 89,99% 199,82% TV1 14,77% 43,80% 28,66% 85,33% TV2 -7,94% 10,69% 7,77% 14,15% LastFM 96,10% 116,54% 40,98% 254,62% Improvement using Seq iTALS iTALSx Dataset Recall@20 MAP@20 Recall@20 MAP@20 Grocery 84,48% 104,13% 108,83% 122,24% TV1 36,15% 55,07% 26,14% 29,93%20 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  21. 21. Anecdotes • Item2item recommendations – bookstore • Placebo effect • buyer vs. seller21 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  22. 22. Conclusion • Offline and online testing • From simple to sophisticated • Many more potential fields of application22 From a toolkit of recommendation algorithms into a real business 13.09.2012.

×