1.
From a toolkit ofrecommendation algorithmsinto a real business:the Gravity R&D experience13.09.2012.

2.
The kick-start2 From a toolkit of recommendation algorithms into a real business 13.09.2012.

3.
Facing with real needs What we had What clients wanted • rating prediction algorithms • recommendations that • coded in various languages bring revenue • blending mechanism • robustness • accuracy oriented • low response time • easy integration • reporting3 From a toolkit of recommendation algorithms into a real business 13.09.2012.

4.
What we do? users content of service provider recommender4 From a toolkit of recommendation algorithms into a real business 13.09.2012.

5.
Explicit vs implicit feedback No ratings but interactions sparse vs. dense matrix requires different learning5 From a toolkit of recommendation algorithms into a real business 13.09.2012.

6.
Increase revenue: A/B tests against the original solution internally6 From a toolkit of recommendation algorithms into a real business 13.09.2012.

7.
Robustness Management LAN SNMP Nagios Monitoring HP OpenView Aggregator HTTP HTTP Platform OSS/BSS / SQL / SQL IMPRESS IMPRESS SOAP Application Server #1 Application Server #2 IMPRESS Frontend web server #1 Backend LAN Reco LAN HTTP Load Balancer HTTP(S) Firewall SQL SQL CSV over FTP TV Service LAN IMPRESS Frontend web server #2 Database #1 Database #2Reporting Subsystem End users7 From a toolkit of recommendation algorithms into a real business 13.09.2012.

8.
Time requirements • Response time: few ms (max 200) • Training time: maximum few hours • regular retraining • incremental training • Newsletters: • nightly batch run8 From a toolkit of recommendation algorithms into a real business 13.09.2012.

9.
Productization IMPRESS RECO AD•APT for for for IPTV, CATV and satellite e-commerce ad networks and ad server providers Recommends Recommends Recommends Personally Personally Relevant Relevant Personally Relevant products & services ads Linear TV, VOD, catch-up TV and more Gravity personalization platform9 From a toolkit of recommendation algorithms into a real business 13.09.2012.

10.
The 5% question – Importance of UI Francisco Martin (Strands): „the algorithm is only 5% in the success of the recommender system” • placement  below or above the fold  scrolling  easy to recognize  floating in • title  not misleading  explanation like • widget  carrousel  static10 From a toolkit of recommendation algorithms into a real business 13.09.2012.

11.
Recommendation scenario Item2Item recommendation logic: the ad’s profile will be matched to the profile model of available ads11 From a toolkit of recommendation algorithms into a real business 13.09.2012.

12.
Marketing channels Changing the order of two boxes: 25% CTR increase12 From a toolkit of recommendation algorithms into a real business 13.09.2012.

13.
Cannibalization • Goal: increase user engagement • Measurements • average visit length • average page views • Effect of accurate recommendations: • use of listing page ↓ • use of item page ↑ • Overall page view: remains the same • Secondary measurements • Contacting • CTR increase13 From a toolkit of recommendation algorithms into a real business 13.09.2012.

14.
Evolution: increased user engagement • not a cold start problem • parameter optimization and user engagement14 From a toolkit of recommendation algorithms into a real business 13.09.2012.

15.
KPIs – may change during testing15 From a toolkit of recommendation algorithms into a real business 13.09.2012.

16.
Complete personalization: coupon-world • Newsletter (daily + occassionally) • Ranking all offers on the website • top1 item • category preferences • user metadata (gender, age, …) • user category preferences (seldom given) • item metadata • context • customer vs. vendor16 From a toolkit of recommendation algorithms into a real business 13.09.2012.

17.
Business rules – driving/overriding ranking17 From a toolkit of recommendation algorithms into a real business 13.09.2012.

18.
Contexts18 From a toolkit of recommendation algorithms into a real business 13.09.2012.

19.
Context at TV program recommendation • TV (EPG program & video-on-demand)  explicit and implicit identification of the user in the household  time-dependent recommendation19 From a toolkit of recommendation algorithms into a real business 13.09.2012.

20.
(offline) Some results (online) Improvement using season iTALS iTALSx Dataset Recall@20 MAP@20 Recall@20 MAP@20 Grocery 64,31% 137,96% 89,99% 199,82% TV1 14,77% 43,80% 28,66% 85,33% TV2 -7,94% 10,69% 7,77% 14,15% LastFM 96,10% 116,54% 40,98% 254,62% Improvement using Seq iTALS iTALSx Dataset Recall@20 MAP@20 Recall@20 MAP@20 Grocery 84,48% 104,13% 108,83% 122,24% TV1 36,15% 55,07% 26,14% 29,93%20 From a toolkit of recommendation algorithms into a real business 13.09.2012.

21.
Anecdotes • Item2item recommendations – bookstore • Placebo effect • buyer vs. seller21 From a toolkit of recommendation algorithms into a real business 13.09.2012.

22.
Conclusion • Offline and online testing • From simple to sophisticated • Many more potential fields of application22 From a toolkit of recommendation algorithms into a real business 13.09.2012.