1.
From a toolkit of
recommendation algorithms
into a real business:
the Gravity R&D experience
13.09.2012.

2.
The kick-start
2 From a toolkit of recommendation algorithms into a real business 13.09.2012.

3.
Facing with real needs
What we had What clients wanted
• rating prediction algorithms • recommendations that
• coded in various languages bring revenue
• blending mechanism • robustness
• accuracy oriented • low response time
• easy integration
• reporting
3 From a toolkit of recommendation algorithms into a real business 13.09.2012.

4.
What we do?
users
content of service
provider
recommender
4 From a toolkit of recommendation algorithms into a real business 13.09.2012.

5.
Explicit vs implicit feedback
No ratings but interactions
sparse vs. dense matrix
requires different learning
5 From a toolkit of recommendation algorithms into a real business 13.09.2012.

6.
Increase revenue: A/B tests
against the original solution
internally
6 From a toolkit of recommendation algorithms into a real business 13.09.2012.

7.
Robustness
Management LAN
SNMP
Nagios Monitoring HP OpenView
Aggregator
HTTP HTTP
Platform OSS/BSS / SQL / SQL
IMPRESS IMPRESS
SOAP Application Server #1 Application Server #2
IMPRESS Frontend
web server #1
Backend LAN Reco LAN HTTP Load Balancer HTTP(S)
Firewall SQL SQL
CSV over FTP
TV Service LAN
IMPRESS Frontend
web server #2
Database #1 Database #2
Reporting Subsystem
End users
7 From a toolkit of recommendation algorithms into a real business 13.09.2012.

8.
Time requirements
• Response time: few ms (max 200)
• Training time: maximum few hours
• regular retraining
• incremental training
• Newsletters:
• nightly batch run
8 From a toolkit of recommendation algorithms into a real business 13.09.2012.

9.
Productization
IMPRESS RECO AD•APT
for for for
IPTV, CATV and satellite e-commerce ad networks and ad
server providers
Recommends Recommends Recommends Personally
Personally Relevant Relevant
Personally Relevant
products & services ads
Linear TV, VOD,
catch-up TV and more
Gravity personalization platform
9 From a toolkit of recommendation algorithms into a real business 13.09.2012.

10.
The 5% question – Importance of UI
Francisco Martin (Strands): „the algorithm is only 5% in the success of
the recommender system”
• placement
 below or above the fold
 scrolling
 easy to recognize
 floating in
• title
 not misleading
 explanation like
• widget
 carrousel
 static
10 From a toolkit of recommendation algorithms into a real business 13.09.2012.

11.
Recommendation scenario
Item2Item
recommendation
logic: the ad’s
profile will be
matched to the
profile model of
available ads
11 From a toolkit of recommendation algorithms into a real business 13.09.2012.

12.
Marketing channels
Changing the order of two boxes: 25% CTR increase
12 From a toolkit of recommendation algorithms into a real business 13.09.2012.

13.
Cannibalization
• Goal: increase user engagement
• Measurements
• average visit length
• average page views
• Effect of accurate recommendations:
• use of listing page ↓
• use of item page ↑
• Overall page view: remains the same
• Secondary measurements
• Contacting
• CTR increase
13 From a toolkit of recommendation algorithms into a real business 13.09.2012.

14.
Evolution: increased user engagement
• not a cold start problem
• parameter optimization and user engagement
14 From a toolkit of recommendation algorithms into a real business 13.09.2012.

15.
KPIs – may change during testing
15 From a toolkit of recommendation algorithms into a real business 13.09.2012.

16.
Complete personalization: coupon-world
• Newsletter (daily +
occassionally)
• Ranking all offers on the website
• top1 item
• category preferences
• user metadata (gender, age, …)
• user category preferences
(seldom given)
• item metadata
• context
• customer vs. vendor
16 From a toolkit of recommendation algorithms into a real business 13.09.2012.

17.
Business rules – driving/overriding ranking
17 From a toolkit of recommendation algorithms into a real business 13.09.2012.

18.
Contexts
18 From a toolkit of recommendation algorithms into a real business 13.09.2012.

19.
Context at TV program recommendation
• TV (EPG program & video-on-demand)
 explicit and implicit identification of the user in the household
 time-dependent recommendation
19 From a toolkit of recommendation algorithms into a real business 13.09.2012.

20.
(offline)
Some results (online)
Improvement using season
iTALS iTALSx
Dataset Recall@20 MAP@20 Recall@20 MAP@20
Grocery 64,31% 137,96% 89,99% 199,82%
TV1 14,77% 43,80% 28,66% 85,33%
TV2 -7,94% 10,69% 7,77% 14,15%
LastFM 96,10% 116,54% 40,98% 254,62%
Improvement using Seq
iTALS iTALSx
Dataset Recall@20 MAP@20 Recall@20 MAP@20
Grocery 84,48% 104,13% 108,83% 122,24%
TV1 36,15% 55,07% 26,14% 29,93%
20 From a toolkit of recommendation algorithms into a real business 13.09.2012.

21.
Anecdotes
• Item2item recommendations – bookstore
• Placebo effect
• buyer vs. seller
21 From a toolkit of recommendation algorithms into a real business 13.09.2012.

22.
Conclusion
• Offline and online testing
• From simple to sophisticated
• Many more potential fields of application
22 From a toolkit of recommendation algorithms into a real business 13.09.2012.