SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 14 day free trial to unlock unlimited reading.
From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)
From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)
1.
From a toolkit of
recommendation algorithms
into a real business:
the Gravity R&D experience
13.09.2012.
2.
The kick-start
2 From a toolkit of recommendation algorithms into a real business 13.09.2012.
3.
Facing with real needs
What we had What clients wanted
• rating prediction algorithms • recommendations that
• coded in various languages bring revenue
• blending mechanism • robustness
• accuracy oriented • low response time
• easy integration
• reporting
3 From a toolkit of recommendation algorithms into a real business 13.09.2012.
4.
What we do?
users
content of service
provider
recommender
4 From a toolkit of recommendation algorithms into a real business 13.09.2012.
5.
Explicit vs implicit feedback
No ratings but interactions
sparse vs. dense matrix
requires different learning
5 From a toolkit of recommendation algorithms into a real business 13.09.2012.
6.
Increase revenue: A/B tests
against the original solution
internally
6 From a toolkit of recommendation algorithms into a real business 13.09.2012.
7.
Robustness
Management LAN
SNMP
Nagios Monitoring HP OpenView
Aggregator
HTTP HTTP
Platform OSS/BSS / SQL / SQL
IMPRESS IMPRESS
SOAP Application Server #1 Application Server #2
IMPRESS Frontend
web server #1
Backend LAN Reco LAN HTTP Load Balancer HTTP(S)
Firewall SQL SQL
CSV over FTP
TV Service LAN
IMPRESS Frontend
web server #2
Database #1 Database #2
Reporting Subsystem
End users
7 From a toolkit of recommendation algorithms into a real business 13.09.2012.
8.
Time requirements
• Response time: few ms (max 200)
• Training time: maximum few hours
• regular retraining
• incremental training
• Newsletters:
• nightly batch run
8 From a toolkit of recommendation algorithms into a real business 13.09.2012.
9.
Productization
IMPRESS RECO AD•APT
for for for
IPTV, CATV and satellite e-commerce ad networks and ad
server providers
Recommends Recommends Recommends Personally
Personally Relevant Relevant
Personally Relevant
products & services ads
Linear TV, VOD,
catch-up TV and more
Gravity personalization platform
9 From a toolkit of recommendation algorithms into a real business 13.09.2012.
10.
The 5% question – Importance of UI
Francisco Martin (Strands): „the algorithm is only 5% in the success of
the recommender system”
• placement
below or above the fold
scrolling
easy to recognize
floating in
• title
not misleading
explanation like
• widget
carrousel
static
10 From a toolkit of recommendation algorithms into a real business 13.09.2012.
11.
Recommendation scenario
Item2Item
recommendation
logic: the ad’s
profile will be
matched to the
profile model of
available ads
11 From a toolkit of recommendation algorithms into a real business 13.09.2012.
12.
Marketing channels
Changing the order of two boxes: 25% CTR increase
12 From a toolkit of recommendation algorithms into a real business 13.09.2012.
13.
Cannibalization
• Goal: increase user engagement
• Measurements
• average visit length
• average page views
• Effect of accurate recommendations:
• use of listing page ↓
• use of item page ↑
• Overall page view: remains the same
• Secondary measurements
• Contacting
• CTR increase
13 From a toolkit of recommendation algorithms into a real business 13.09.2012.
14.
Evolution: increased user engagement
• not a cold start problem
• parameter optimization and user engagement
14 From a toolkit of recommendation algorithms into a real business 13.09.2012.
15.
KPIs – may change during testing
15 From a toolkit of recommendation algorithms into a real business 13.09.2012.
16.
Complete personalization: coupon-world
• Newsletter (daily +
occassionally)
• Ranking all offers on the website
• top1 item
• category preferences
• user metadata (gender, age, …)
• user category preferences
(seldom given)
• item metadata
• context
• customer vs. vendor
16 From a toolkit of recommendation algorithms into a real business 13.09.2012.
17.
Business rules – driving/overriding ranking
17 From a toolkit of recommendation algorithms into a real business 13.09.2012.
18.
Contexts
18 From a toolkit of recommendation algorithms into a real business 13.09.2012.
19.
Context at TV program recommendation
• TV (EPG program & video-on-demand)
explicit and implicit identification of the user in the household
time-dependent recommendation
19 From a toolkit of recommendation algorithms into a real business 13.09.2012.
20.
(offline)
Some results (online)
Improvement using season
iTALS iTALSx
Dataset Recall@20 MAP@20 Recall@20 MAP@20
Grocery 64,31% 137,96% 89,99% 199,82%
TV1 14,77% 43,80% 28,66% 85,33%
TV2 -7,94% 10,69% 7,77% 14,15%
LastFM 96,10% 116,54% 40,98% 254,62%
Improvement using Seq
iTALS iTALSx
Dataset Recall@20 MAP@20 Recall@20 MAP@20
Grocery 84,48% 104,13% 108,83% 122,24%
TV1 36,15% 55,07% 26,14% 29,93%
20 From a toolkit of recommendation algorithms into a real business 13.09.2012.
21.
Anecdotes
• Item2item recommendations – bookstore
• Placebo effect
• buyer vs. seller
21 From a toolkit of recommendation algorithms into a real business 13.09.2012.
22.
Conclusion
• Offline and online testing
• From simple to sophisticated
• Many more potential fields of application
22 From a toolkit of recommendation algorithms into a real business 13.09.2012.