From a toolkit ofrecommendation algorithmsinto a real business:the Gravity R&D experience13.09.2012.
The kick-start2   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Facing with real needs    What we had                                                  What clients wanted    • rating pre...
What we do?          users                                                                       content of service       ...
Explicit vs implicit feedback    No ratings but interactions    sparse vs. dense matrix    requires different learning5   ...
Increase revenue: A/B tests    against the original solution    internally6   From a toolkit of recommendation algorithms ...
Robustness                                                                                                  Management LAN...
Time requirements    • Response time: few ms (max 200)    • Training time: maximum few hours      • regular retraining    ...
Productization              IMPRESS                                     RECO                       AD•APT             for ...
The 5% question – Importance of UI     Francisco Martin (Strands): „the algorithm is only 5% in the success of     the rec...
Recommendation scenario                                                                                          Item2Item...
Marketing channels        Changing the order of two boxes: 25% CTR increase12   From a toolkit of recommendation algorithm...
Cannibalization     • Goal: increase user engagement     • Measurements       • average visit length       • average page ...
Evolution: increased user engagement     • not a cold start problem     • parameter optimization and user engagement14   F...
KPIs – may change during testing15   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Complete personalization: coupon-world     • Newsletter (daily +       occassionally)     • Ranking all offers on the webs...
Business rules – driving/overriding ranking17   From a toolkit of recommendation algorithms into a real business   13.09.2...
Contexts18   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Context at TV program recommendation     • TV (EPG program & video-on-demand)        explicit and implicit identification...
(offline)     Some results (online)                                  Improvement using season                             ...
Anecdotes     • Item2item recommendations – bookstore     • Placebo effect     • buyer vs. seller21   From a toolkit of re...
Conclusion     • Offline and online testing     • From simple to sophisticated     • Many more potential fields of applica...
Upcoming SlideShare
Loading in …5
×

From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)

1,717 views

Published on

Talk given at Recsys Challenge Workshop in Dublin (@ ACM Recsys 2012), on 13.09.2012.

Published in: Technology
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,717
On SlideShare
0
From Embeds
0
Number of Embeds
20
Actions
Shares
0
Downloads
59
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)

  1. 1. From a toolkit ofrecommendation algorithmsinto a real business:the Gravity R&D experience13.09.2012.
  2. 2. The kick-start2 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  3. 3. Facing with real needs What we had What clients wanted • rating prediction algorithms • recommendations that • coded in various languages bring revenue • blending mechanism • robustness • accuracy oriented • low response time • easy integration • reporting3 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  4. 4. What we do? users content of service provider recommender4 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  5. 5. Explicit vs implicit feedback No ratings but interactions sparse vs. dense matrix requires different learning5 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  6. 6. Increase revenue: A/B tests against the original solution internally6 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  7. 7. Robustness Management LAN SNMP Nagios Monitoring HP OpenView Aggregator HTTP HTTP Platform OSS/BSS / SQL / SQL IMPRESS IMPRESS SOAP Application Server #1 Application Server #2 IMPRESS Frontend web server #1 Backend LAN Reco LAN HTTP Load Balancer HTTP(S) Firewall SQL SQL CSV over FTP TV Service LAN IMPRESS Frontend web server #2 Database #1 Database #2Reporting Subsystem End users7 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  8. 8. Time requirements • Response time: few ms (max 200) • Training time: maximum few hours • regular retraining • incremental training • Newsletters: • nightly batch run8 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  9. 9. Productization IMPRESS RECO AD•APT for for for IPTV, CATV and satellite e-commerce ad networks and ad server providers Recommends Recommends Recommends Personally Personally Relevant Relevant Personally Relevant products & services ads Linear TV, VOD, catch-up TV and more Gravity personalization platform9 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  10. 10. The 5% question – Importance of UI Francisco Martin (Strands): „the algorithm is only 5% in the success of the recommender system” • placement  below or above the fold  scrolling  easy to recognize  floating in • title  not misleading  explanation like • widget  carrousel  static10 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  11. 11. Recommendation scenario Item2Item recommendation logic: the ad’s profile will be matched to the profile model of available ads11 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  12. 12. Marketing channels Changing the order of two boxes: 25% CTR increase12 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  13. 13. Cannibalization • Goal: increase user engagement • Measurements • average visit length • average page views • Effect of accurate recommendations: • use of listing page ↓ • use of item page ↑ • Overall page view: remains the same • Secondary measurements • Contacting • CTR increase13 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  14. 14. Evolution: increased user engagement • not a cold start problem • parameter optimization and user engagement14 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  15. 15. KPIs – may change during testing15 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  16. 16. Complete personalization: coupon-world • Newsletter (daily + occassionally) • Ranking all offers on the website • top1 item • category preferences • user metadata (gender, age, …) • user category preferences (seldom given) • item metadata • context • customer vs. vendor16 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  17. 17. Business rules – driving/overriding ranking17 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  18. 18. Contexts18 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  19. 19. Context at TV program recommendation • TV (EPG program & video-on-demand)  explicit and implicit identification of the user in the household  time-dependent recommendation19 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  20. 20. (offline) Some results (online) Improvement using season iTALS iTALSx Dataset Recall@20 MAP@20 Recall@20 MAP@20 Grocery 64,31% 137,96% 89,99% 199,82% TV1 14,77% 43,80% 28,66% 85,33% TV2 -7,94% 10,69% 7,77% 14,15% LastFM 96,10% 116,54% 40,98% 254,62% Improvement using Seq iTALS iTALSx Dataset Recall@20 MAP@20 Recall@20 MAP@20 Grocery 84,48% 104,13% 108,83% 122,24% TV1 36,15% 55,07% 26,14% 29,93%20 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  21. 21. Anecdotes • Item2item recommendations – bookstore • Placebo effect • buyer vs. seller21 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  22. 22. Conclusion • Offline and online testing • From simple to sophisticated • Many more potential fields of application22 From a toolkit of recommendation algorithms into a real business 13.09.2012.

×