From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)

Domonkos Tikk
Domonkos TikkCEO at Gravity R&D
From a toolkit of
recommendation algorithms
into a real business:
the Gravity R&D experience




13.09.2012.
The kick-start




2   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Facing with real needs

    What we had                                                  What clients wanted
    • rating prediction algorithms • recommendations that
    • coded in various languages     bring revenue
    • blending mechanism           • robustness
    • accuracy oriented            • low response time
                                   • easy integration
                                   • reporting




3   From a toolkit of recommendation algorithms into a real business   13.09.2012.
What we do?




          users


                                                                       content of service
                                                                           provider
                               recommender
4   From a toolkit of recommendation algorithms into a real business    13.09.2012.
Explicit vs implicit feedback

    No ratings but interactions




    sparse vs. dense matrix



    requires different learning

5   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Increase revenue: A/B tests

    against the original solution




    internally




6   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Robustness


                                                                                                  Management LAN

                                                                                    SNMP
                                                                                                                          Nagios Monitoring     HP OpenView
                                                                                                                             Aggregator


                                                              HTTP                         HTTP
    Platform OSS/BSS                                          / SQL                        / SQL
                                              IMPRESS                   IMPRESS
        SOAP                            Application Server #1     Application Server #2
                                                                                                       IMPRESS Frontend
                                                                                                         web server #1
          Backend LAN                                      Reco LAN                        HTTP                                 Load Balancer   HTTP(S)


                             Firewall                   SQL             SQL
        CSV over FTP
                                                                                                                                    TV Service LAN
                                                                                                      IMPRESS Frontend
                                                                                                        web server #2

                                                   Database #1        Database #2
Reporting Subsystem




                                                                                                                   End users


7    From a toolkit of recommendation algorithms into a real business                             13.09.2012.
Time requirements

    • Response time: few ms (max 200)
    • Training time: maximum few hours
      • regular retraining
      • incremental training
    • Newsletters:
      • nightly batch run




8   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Productization



              IMPRESS                                     RECO                       AD•APT
             for                                          for                            for
    IPTV, CATV and satellite                          e-commerce                 ad networks and ad
                                                                                  server providers


         Recommends                                Recommends                Recommends Personally
                                                 Personally Relevant              Relevant
      Personally Relevant
                                                products & services                 ads
        Linear TV, VOD,
     catch-up TV and more



                                Gravity personalization platform

9   From a toolkit of recommendation algorithms into a real business   13.09.2012.
The 5% question – Importance of UI

     Francisco Martin (Strands): „the algorithm is only 5% in the success of
     the recommender system”
     • placement
         below or above the fold
         scrolling
         easy to recognize
         floating in
     • title
         not misleading
         explanation like
     • widget
         carrousel
         static

10   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Recommendation scenario


                                                                                          Item2Item
                                                                                      recommendation
                                                                                        logic: the ad’s
                                                                                         profile will be
                                                                                       matched to the
                                                                                       profile model of
                                                                                         available ads




11   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Marketing channels




        Changing the order of two boxes: 25% CTR increase

12   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Cannibalization

     • Goal: increase user engagement
     • Measurements
       • average visit length
       • average page views
     • Effect of accurate recommendations:
       • use of listing page ↓
       • use of item page ↑
     • Overall page view: remains the same
     • Secondary measurements
       • Contacting
       • CTR increase




13   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Evolution: increased user engagement




     • not a cold start problem
     • parameter optimization and user engagement




14   From a toolkit of recommendation algorithms into a real business   13.09.2012.
KPIs – may change during testing




15   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Complete personalization: coupon-world

     • Newsletter (daily +
       occassionally)
     • Ranking all offers on the website
        • top1 item
        • category preferences



                                                                  • user metadata (gender, age, …)
                                                                  • user category preferences
                                                                    (seldom given)
                                                                  • item metadata
                                                                  • context

                                                                  • customer vs. vendor

16   From a toolkit of recommendation algorithms into a real business     13.09.2012.
Business rules – driving/overriding ranking




17   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Contexts




18   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Context at TV program recommendation

     • TV (EPG program & video-on-demand)
        explicit and implicit identification of the user in the household
        time-dependent recommendation




19   From a toolkit of recommendation algorithms into a real business   13.09.2012.
(offline)
     Some results (online)

                                  Improvement using season
                                  iTALS              iTALSx
                   Dataset Recall@20 MAP@20 Recall@20 MAP@20
                  Grocery     64,31% 137,96%     89,99% 199,82%
                  TV1         14,77% 43,80%      28,66% 85,33%
                  TV2         -7,94% 10,69%       7,77% 14,15%
                  LastFM      96,10% 116,54%     40,98% 254,62%

                                    Improvement using Seq
                                  iTALS               iTALSx
                   Dataset Recall@20 MAP@20 Recall@20 MAP@20
                  Grocery     84,48% 104,13% 108,83% 122,24%
                  TV1         36,15% 55,07%       26,14% 29,93%

20   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Anecdotes

     • Item2item recommendations – bookstore


     • Placebo effect


     • buyer vs. seller


21   From a toolkit of recommendation algorithms into a real business   13.09.2012.
Conclusion

     • Offline and online testing


     • From simple to sophisticated


     • Many more potential fields of application



22   From a toolkit of recommendation algorithms into a real business   13.09.2012.
1 of 22

Recommended

Challenges Encountered by Scaling Up Recommendation Services at Gravity R&D by
Challenges Encountered by Scaling Up Recommendation Services at Gravity R&DChallenges Encountered by Scaling Up Recommendation Services at Gravity R&D
Challenges Encountered by Scaling Up Recommendation Services at Gravity R&DDomonkos Tikk
1.7K views20 slides
Cloud Wars: Performance Benchmarking AWS, GCP and Azure by
Cloud Wars: Performance Benchmarking AWS, GCP and Azure Cloud Wars: Performance Benchmarking AWS, GCP and Azure
Cloud Wars: Performance Benchmarking AWS, GCP and Azure ThousandEyes
1.1K views38 slides
How to See and Resolve Office 365 Performance Challenges by
How to See and Resolve Office 365 Performance Challenges How to See and Resolve Office 365 Performance Challenges
How to See and Resolve Office 365 Performance Challenges ThousandEyes
652 views23 slides
Become the Master of Your DNS by
Become the Master of Your DNSBecome the Master of Your DNS
Become the Master of Your DNSThousandEyes
184 views32 slides
7 Common Questions About a Cloud Management Platform by
7 Common Questions About a Cloud Management Platform7 Common Questions About a Cloud Management Platform
7 Common Questions About a Cloud Management PlatformRightScale
4.9K views14 slides
AWS re:Invent 2016: Setting the Stage for Instant Success: Getting the Most O... by
AWS re:Invent 2016: Setting the Stage for Instant Success: Getting the Most O...AWS re:Invent 2016: Setting the Stage for Instant Success: Getting the Most O...
AWS re:Invent 2016: Setting the Stage for Instant Success: Getting the Most O...Amazon Web Services
691 views36 slides

More Related Content

What's hot

ASTQB washington-sept-2015 by
ASTQB washington-sept-2015ASTQB washington-sept-2015
ASTQB washington-sept-2015Dan Boutin
115 views49 slides
Creating a Culture of Cost Management in Your Organization – J.R. Storment, C... by
Creating a Culture of Cost Management in Your Organization – J.R. Storment, C...Creating a Culture of Cost Management in Your Organization – J.R. Storment, C...
Creating a Culture of Cost Management in Your Organization – J.R. Storment, C...Amazon Web Services
1.7K views111 slides
(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix by
(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix
(ISM309) Efficient Innovation:High-Velocity Cost Management at NetflixAmazon Web Services
2K views28 slides
12 Ways to Manage Cloud Costs and Optimize Cloud Spend by
12 Ways to Manage Cloud Costs and Optimize Cloud Spend12 Ways to Manage Cloud Costs and Optimize Cloud Spend
12 Ways to Manage Cloud Costs and Optimize Cloud SpendRightScale
2.1K views52 slides
GO For A Cloud Certification (AWS) by
GO For A Cloud Certification (AWS)GO For A Cloud Certification (AWS)
GO For A Cloud Certification (AWS)Dhaval Nagar
92 views32 slides
Cw13 aws by tamer abdul radi-cloud9ners by
Cw13 aws by tamer abdul radi-cloud9nersCw13 aws by tamer abdul radi-cloud9ners
Cw13 aws by tamer abdul radi-cloud9nersTheInevitableCloud
351 views35 slides

What's hot(20)

ASTQB washington-sept-2015 by Dan Boutin
ASTQB washington-sept-2015ASTQB washington-sept-2015
ASTQB washington-sept-2015
Dan Boutin115 views
Creating a Culture of Cost Management in Your Organization – J.R. Storment, C... by Amazon Web Services
Creating a Culture of Cost Management in Your Organization – J.R. Storment, C...Creating a Culture of Cost Management in Your Organization – J.R. Storment, C...
Creating a Culture of Cost Management in Your Organization – J.R. Storment, C...
Amazon Web Services1.7K views
(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix by Amazon Web Services
(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix
(ISM309) Efficient Innovation:High-Velocity Cost Management at Netflix
12 Ways to Manage Cloud Costs and Optimize Cloud Spend by RightScale
12 Ways to Manage Cloud Costs and Optimize Cloud Spend12 Ways to Manage Cloud Costs and Optimize Cloud Spend
12 Ways to Manage Cloud Costs and Optimize Cloud Spend
RightScale2.1K views
GO For A Cloud Certification (AWS) by Dhaval Nagar
GO For A Cloud Certification (AWS)GO For A Cloud Certification (AWS)
GO For A Cloud Certification (AWS)
Dhaval Nagar92 views
AWS re:Invent 2016: Saving at Scale with Reserved Instances (ENT307) by Amazon Web Services
AWS re:Invent 2016: Saving at Scale with Reserved Instances (ENT307)AWS re:Invent 2016: Saving at Scale with Reserved Instances (ENT307)
AWS re:Invent 2016: Saving at Scale with Reserved Instances (ENT307)
Amazon Web Services1.2K views
AtlasCamp 2014: Stash State of the Union by Atlassian
AtlasCamp 2014: Stash State of the UnionAtlasCamp 2014: Stash State of the Union
AtlasCamp 2014: Stash State of the Union
Atlassian719 views
Cut AWS Costs: Using Spot Instances for More Than Batch by RightScale
Cut AWS Costs: Using Spot Instances for More Than BatchCut AWS Costs: Using Spot Instances for More Than Batch
Cut AWS Costs: Using Spot Instances for More Than Batch
RightScale469 views
Cloud computing: cost reduction by Hesham Shabana
Cloud computing: cost reductionCloud computing: cost reduction
Cloud computing: cost reduction
Hesham Shabana1.4K views
„OWASP Top Ten in Latvia“ by Agris Krusts from IT Centrs SIA at Security focu... by DevClub_lv
„OWASP Top Ten in Latvia“ by Agris Krusts from IT Centrs SIA at Security focu...„OWASP Top Ten in Latvia“ by Agris Krusts from IT Centrs SIA at Security focu...
„OWASP Top Ten in Latvia“ by Agris Krusts from IT Centrs SIA at Security focu...
DevClub_lv3.2K views
AWS Cloud Cost Optimization by Yogesh Sharma
AWS Cloud Cost OptimizationAWS Cloud Cost Optimization
AWS Cloud Cost Optimization
Yogesh Sharma206 views
Keep Cloud Transformation on Track: Nine Best Practices to Avoid or Break Thr... by Amazon Web Services
Keep Cloud Transformation on Track: Nine Best Practices to Avoid or Break Thr...Keep Cloud Transformation on Track: Nine Best Practices to Avoid or Break Thr...
Keep Cloud Transformation on Track: Nine Best Practices to Avoid or Break Thr...
Hybrid Cloud Orchestration: How SuperChoice Does It by RightScale
Hybrid Cloud Orchestration: How SuperChoice Does ItHybrid Cloud Orchestration: How SuperChoice Does It
Hybrid Cloud Orchestration: How SuperChoice Does It
RightScale694 views
AWS re:Invent 2019 Recap - Expert Virtual Panel - Agile/DevOps/Team Tooling by Cprime
AWS re:Invent 2019 Recap - Expert Virtual Panel - Agile/DevOps/Team ToolingAWS re:Invent 2019 Recap - Expert Virtual Panel - Agile/DevOps/Team Tooling
AWS re:Invent 2019 Recap - Expert Virtual Panel - Agile/DevOps/Team Tooling
Cprime160 views
AWS re:Invent 2016: Identifying Your Migration Options: the 6 Rs (ENT311) by Amazon Web Services
AWS re:Invent 2016: Identifying Your Migration Options: the 6 Rs (ENT311)AWS re:Invent 2016: Identifying Your Migration Options: the 6 Rs (ENT311)
AWS re:Invent 2016: Identifying Your Migration Options: the 6 Rs (ENT311)
Amazon Web Services9.9K views
Manage and Optimize Cloud Spend with RightScale Optima by RightScale
Manage and Optimize Cloud Spend with RightScale OptimaManage and Optimize Cloud Spend with RightScale Optima
Manage and Optimize Cloud Spend with RightScale Optima
RightScale396 views
Pivoting to Cloud: How an MSP Brokers Cloud Services by RightScale
Pivoting to Cloud: How an MSP Brokers Cloud Services Pivoting to Cloud: How an MSP Brokers Cloud Services
Pivoting to Cloud: How an MSP Brokers Cloud Services
RightScale961 views

Viewers also liked

Gravity rd corporate introduction - nlp matiné 2014 by
Gravity rd corporate introduction  - nlp matiné 2014Gravity rd corporate introduction  - nlp matiné 2014
Gravity rd corporate introduction - nlp matiné 2014Zoltan Varju
1.4K views4 slides
Gravity personalizaton intro by
Gravity personalizaton introGravity personalizaton intro
Gravity personalizaton introEszter Nagy
573 views12 slides
Xây dựng mạng lưới tài năng trẻ trong sáng tạo – khởi nghiệp by
Xây dựng mạng lưới tài năng trẻ trong sáng tạo – khởi nghiệpXây dựng mạng lưới tài năng trẻ trong sáng tạo – khởi nghiệp
Xây dựng mạng lưới tài năng trẻ trong sáng tạo – khởi nghiệpTri Dung, Tran
513 views25 slides
Entrepreneurship & Innovation: Dual-core Engine by
Entrepreneurship & Innovation: Dual-core EngineEntrepreneurship & Innovation: Dual-core Engine
Entrepreneurship & Innovation: Dual-core EngineTri Dung, Tran
428 views27 slides
The rise of Recommendation Engines by
The rise of Recommendation EnginesThe rise of Recommendation Engines
The rise of Recommendation Engineslamnk
346 views30 slides
Lessons learnt at building recommendation services at industry scale by
Lessons learnt at building recommendation services at industry scaleLessons learnt at building recommendation services at industry scale
Lessons learnt at building recommendation services at industry scaleDomonkos Tikk
3K views80 slides

Viewers also liked(6)

Gravity rd corporate introduction - nlp matiné 2014 by Zoltan Varju
Gravity rd corporate introduction  - nlp matiné 2014Gravity rd corporate introduction  - nlp matiné 2014
Gravity rd corporate introduction - nlp matiné 2014
Zoltan Varju1.4K views
Gravity personalizaton intro by Eszter Nagy
Gravity personalizaton introGravity personalizaton intro
Gravity personalizaton intro
Eszter Nagy573 views
Xây dựng mạng lưới tài năng trẻ trong sáng tạo – khởi nghiệp by Tri Dung, Tran
Xây dựng mạng lưới tài năng trẻ trong sáng tạo – khởi nghiệpXây dựng mạng lưới tài năng trẻ trong sáng tạo – khởi nghiệp
Xây dựng mạng lưới tài năng trẻ trong sáng tạo – khởi nghiệp
Tri Dung, Tran513 views
Entrepreneurship & Innovation: Dual-core Engine by Tri Dung, Tran
Entrepreneurship & Innovation: Dual-core EngineEntrepreneurship & Innovation: Dual-core Engine
Entrepreneurship & Innovation: Dual-core Engine
Tri Dung, Tran428 views
The rise of Recommendation Engines by lamnk
The rise of Recommendation EnginesThe rise of Recommendation Engines
The rise of Recommendation Engines
lamnk346 views
Lessons learnt at building recommendation services at industry scale by Domonkos Tikk
Lessons learnt at building recommendation services at industry scaleLessons learnt at building recommendation services at industry scale
Lessons learnt at building recommendation services at industry scale
Domonkos Tikk3K views

Similar to From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)

IBM Pulse 2013 session - DevOps for Mobile Apps by
IBM Pulse 2013 session - DevOps for Mobile AppsIBM Pulse 2013 session - DevOps for Mobile Apps
IBM Pulse 2013 session - DevOps for Mobile AppsSanjeev Sharma
3.2K views47 slides
Whitepaper: Volume Testing Thick Clients and Databases by
Whitepaper:  Volume Testing Thick Clients and DatabasesWhitepaper:  Volume Testing Thick Clients and Databases
Whitepaper: Volume Testing Thick Clients and DatabasesRTTS
1.1K views9 slides
3158 - Cloud Infrastructure & It Optimization - Application Performance Manag... by
3158 - Cloud Infrastructure & It Optimization - Application Performance Manag...3158 - Cloud Infrastructure & It Optimization - Application Performance Manag...
3158 - Cloud Infrastructure & It Optimization - Application Performance Manag...Sandeep Chellingi
861 views21 slides
apidays Helsinki & North 2023 - How can data-driven DevRel help identify gaps... by
apidays Helsinki & North 2023 - How can data-driven DevRel help identify gaps...apidays Helsinki & North 2023 - How can data-driven DevRel help identify gaps...
apidays Helsinki & North 2023 - How can data-driven DevRel help identify gaps...apidays
11 views48 slides
Evaluating Php As A Technology Platform For Soa Implementations by
 Evaluating Php As A Technology Platform For Soa Implementations Evaluating Php As A Technology Platform For Soa Implementations
Evaluating Php As A Technology Platform For Soa ImplementationsVedanta Barooah
1.2K views44 slides
Running a World Class SaaS Organization by
Running a World Class SaaS OrganizationRunning a World Class SaaS Organization
Running a World Class SaaS OrganizationFlexera
2.1K views20 slides

Similar to From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)(20)

IBM Pulse 2013 session - DevOps for Mobile Apps by Sanjeev Sharma
IBM Pulse 2013 session - DevOps for Mobile AppsIBM Pulse 2013 session - DevOps for Mobile Apps
IBM Pulse 2013 session - DevOps for Mobile Apps
Sanjeev Sharma3.2K views
Whitepaper: Volume Testing Thick Clients and Databases by RTTS
Whitepaper:  Volume Testing Thick Clients and DatabasesWhitepaper:  Volume Testing Thick Clients and Databases
Whitepaper: Volume Testing Thick Clients and Databases
RTTS1.1K views
3158 - Cloud Infrastructure & It Optimization - Application Performance Manag... by Sandeep Chellingi
3158 - Cloud Infrastructure & It Optimization - Application Performance Manag...3158 - Cloud Infrastructure & It Optimization - Application Performance Manag...
3158 - Cloud Infrastructure & It Optimization - Application Performance Manag...
Sandeep Chellingi861 views
apidays Helsinki & North 2023 - How can data-driven DevRel help identify gaps... by apidays
apidays Helsinki & North 2023 - How can data-driven DevRel help identify gaps...apidays Helsinki & North 2023 - How can data-driven DevRel help identify gaps...
apidays Helsinki & North 2023 - How can data-driven DevRel help identify gaps...
apidays11 views
Evaluating Php As A Technology Platform For Soa Implementations by Vedanta Barooah
 Evaluating Php As A Technology Platform For Soa Implementations Evaluating Php As A Technology Platform For Soa Implementations
Evaluating Php As A Technology Platform For Soa Implementations
Vedanta Barooah1.2K views
Running a World Class SaaS Organization by Flexera
Running a World Class SaaS OrganizationRunning a World Class SaaS Organization
Running a World Class SaaS Organization
Flexera2.1K views
APM Talk by MongoDB
APM TalkAPM Talk
APM Talk
MongoDB1.2K views
Marketcom PowerPoint by gwilliams92
Marketcom PowerPointMarketcom PowerPoint
Marketcom PowerPoint
gwilliams92141 views
Are Your Applications Delivering What Your End-Users Expect? by Compuware APM
Are Your Applications Delivering What Your End-Users Expect?Are Your Applications Delivering What Your End-Users Expect?
Are Your Applications Delivering What Your End-Users Expect?
Compuware APM560 views
Keynote - The Benefits of an Open Service Oriented Architecture in the Enterpr... by mfrancis
Keynote - The Benefits of an Open Service Oriented Architecture in the Enterpr...Keynote - The Benefits of an Open Service Oriented Architecture in the Enterpr...
Keynote - The Benefits of an Open Service Oriented Architecture in the Enterpr...
mfrancis679 views
Practical guide to building public APIs by Reda Hmeid MBCS
Practical guide to building public APIsPractical guide to building public APIs
Practical guide to building public APIs
Reda Hmeid MBCS463 views
Openly Replacing ERPs with Sugar | SugarCon 2011 by SugarCRM
Openly Replacing ERPs with Sugar | SugarCon 2011Openly Replacing ERPs with Sugar | SugarCon 2011
Openly Replacing ERPs with Sugar | SugarCon 2011
SugarCRM351 views
DevOps vs. ShadowOps (Pulse 2013) by Michael Elder
DevOps vs. ShadowOps (Pulse 2013)DevOps vs. ShadowOps (Pulse 2013)
DevOps vs. ShadowOps (Pulse 2013)
Michael Elder2.9K views
Introduction to Event-Driven Architecture by Solace
Introduction to Event-Driven Architecture Introduction to Event-Driven Architecture
Introduction to Event-Driven Architecture
Solace1.7K views
Service Management excellence with operational intelligence by HP Enterprise Italia
Service Management excellence with operational intelligenceService Management excellence with operational intelligence
Service Management excellence with operational intelligence
Hewlett Packard Enterprise View on Going Big with API Management - Applicatio... by CA Technologies
Hewlett Packard Enterprise View on Going Big with API Management - Applicatio...Hewlett Packard Enterprise View on Going Big with API Management - Applicatio...
Hewlett Packard Enterprise View on Going Big with API Management - Applicatio...
CA Technologies4.6K views
APIs for biz dev 2.0 - Which business model? by 3scale
APIs for biz dev 2.0 - Which business model?APIs for biz dev 2.0 - Which business model?
APIs for biz dev 2.0 - Which business model?
3scale7.7K views

More from Domonkos Tikk

Recommenders on video sharing portals - business and algorithmic aspects by
Recommenders on video sharing portals - business and algorithmic aspectsRecommenders on video sharing portals - business and algorithmic aspects
Recommenders on video sharing portals - business and algorithmic aspectsDomonkos Tikk
917 views16 slides
Neighbor methods vs matrix factorization - case studies of real-life recommen... by
Neighbor methods vs matrix factorization - case studies of real-life recommen...Neighbor methods vs matrix factorization - case studies of real-life recommen...
Neighbor methods vs matrix factorization - case studies of real-life recommen...Domonkos Tikk
5.8K views35 slides
General factorization framework for context-aware recommendations by
General factorization framework for context-aware recommendationsGeneral factorization framework for context-aware recommendations
General factorization framework for context-aware recommendationsDomonkos Tikk
360 views1 slide
Tartalomgazdagítás (content enrichment) by
Tartalomgazdagítás (content enrichment) Tartalomgazdagítás (content enrichment)
Tartalomgazdagítás (content enrichment) Domonkos Tikk
674 views33 slides
Idomaar crowd rec_reference_fw by
Idomaar crowd rec_reference_fwIdomaar crowd rec_reference_fw
Idomaar crowd rec_reference_fwDomonkos Tikk
1.2K views15 slides
Big Data in Online Classifieds by
Big Data in Online ClassifiedsBig Data in Online Classifieds
Big Data in Online ClassifiedsDomonkos Tikk
2.1K views13 slides

More from Domonkos Tikk(10)

Recommenders on video sharing portals - business and algorithmic aspects by Domonkos Tikk
Recommenders on video sharing portals - business and algorithmic aspectsRecommenders on video sharing portals - business and algorithmic aspects
Recommenders on video sharing portals - business and algorithmic aspects
Domonkos Tikk917 views
Neighbor methods vs matrix factorization - case studies of real-life recommen... by Domonkos Tikk
Neighbor methods vs matrix factorization - case studies of real-life recommen...Neighbor methods vs matrix factorization - case studies of real-life recommen...
Neighbor methods vs matrix factorization - case studies of real-life recommen...
Domonkos Tikk5.8K views
General factorization framework for context-aware recommendations by Domonkos Tikk
General factorization framework for context-aware recommendationsGeneral factorization framework for context-aware recommendations
General factorization framework for context-aware recommendations
Domonkos Tikk360 views
Tartalomgazdagítás (content enrichment) by Domonkos Tikk
Tartalomgazdagítás (content enrichment) Tartalomgazdagítás (content enrichment)
Tartalomgazdagítás (content enrichment)
Domonkos Tikk674 views
Idomaar crowd rec_reference_fw by Domonkos Tikk
Idomaar crowd rec_reference_fwIdomaar crowd rec_reference_fw
Idomaar crowd rec_reference_fw
Domonkos Tikk1.2K views
Big Data in Online Classifieds by Domonkos Tikk
Big Data in Online ClassifiedsBig Data in Online Classifieds
Big Data in Online Classifieds
Domonkos Tikk2.1K views
Context-aware similarities within the factorization framework - presented at ... by Domonkos Tikk
Context-aware similarities within the factorization framework - presented at ...Context-aware similarities within the factorization framework - presented at ...
Context-aware similarities within the factorization framework - presented at ...
Domonkos Tikk524 views
Slides from CARR 2012 WS - Enhancing Matrix Factorization Through Initializat... by Domonkos Tikk
Slides from CARR 2012 WS - Enhancing Matrix Factorization Through Initializat...Slides from CARR 2012 WS - Enhancing Matrix Factorization Through Initializat...
Slides from CARR 2012 WS - Enhancing Matrix Factorization Through Initializat...
Domonkos Tikk902 views
Fast ALS-Based Tensor Factorization for Context-Aware Recommendation from Imp... by Domonkos Tikk
Fast ALS-Based Tensor Factorization for Context-Aware Recommendation from Imp...Fast ALS-Based Tensor Factorization for Context-Aware Recommendation from Imp...
Fast ALS-Based Tensor Factorization for Context-Aware Recommendation from Imp...
Domonkos Tikk1.4K views
Recommender Systems Evaluation: A 3D Benchmark - presented at RUE 2012 worksh... by Domonkos Tikk
Recommender Systems Evaluation: A 3D Benchmark - presented at RUE 2012 worksh...Recommender Systems Evaluation: A 3D Benchmark - presented at RUE 2012 worksh...
Recommender Systems Evaluation: A 3D Benchmark - presented at RUE 2012 worksh...
Domonkos Tikk1.2K views

Recently uploaded

Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum... by
Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...
Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...NUS-ISS
28 views35 slides
Micron CXL product and architecture update by
Micron CXL product and architecture updateMicron CXL product and architecture update
Micron CXL product and architecture updateCXL Forum
27 views7 slides
Samsung: CMM-H Tiered Memory Solution with Built-in DRAM by
Samsung: CMM-H Tiered Memory Solution with Built-in DRAMSamsung: CMM-H Tiered Memory Solution with Built-in DRAM
Samsung: CMM-H Tiered Memory Solution with Built-in DRAMCXL Forum
105 views7 slides
Combining Orchestration and Choreography for a Clean Architecture by
Combining Orchestration and Choreography for a Clean ArchitectureCombining Orchestration and Choreography for a Clean Architecture
Combining Orchestration and Choreography for a Clean ArchitectureThomasHeinrichs1
68 views24 slides
Transcript: The Details of Description Techniques tips and tangents on altern... by
Transcript: The Details of Description Techniques tips and tangents on altern...Transcript: The Details of Description Techniques tips and tangents on altern...
Transcript: The Details of Description Techniques tips and tangents on altern...BookNet Canada
119 views15 slides
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV by
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTVSplunk
86 views20 slides

Recently uploaded(20)

Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum... by NUS-ISS
Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...
Beyond the Hype: What Generative AI Means for the Future of Work - Damien Cum...
NUS-ISS28 views
Micron CXL product and architecture update by CXL Forum
Micron CXL product and architecture updateMicron CXL product and architecture update
Micron CXL product and architecture update
CXL Forum27 views
Samsung: CMM-H Tiered Memory Solution with Built-in DRAM by CXL Forum
Samsung: CMM-H Tiered Memory Solution with Built-in DRAMSamsung: CMM-H Tiered Memory Solution with Built-in DRAM
Samsung: CMM-H Tiered Memory Solution with Built-in DRAM
CXL Forum105 views
Combining Orchestration and Choreography for a Clean Architecture by ThomasHeinrichs1
Combining Orchestration and Choreography for a Clean ArchitectureCombining Orchestration and Choreography for a Clean Architecture
Combining Orchestration and Choreography for a Clean Architecture
ThomasHeinrichs168 views
Transcript: The Details of Description Techniques tips and tangents on altern... by BookNet Canada
Transcript: The Details of Description Techniques tips and tangents on altern...Transcript: The Details of Description Techniques tips and tangents on altern...
Transcript: The Details of Description Techniques tips and tangents on altern...
BookNet Canada119 views
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV by Splunk
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
Splunk86 views
Future of Learning - Khoong Chan Meng by NUS-ISS
Future of Learning - Khoong Chan MengFuture of Learning - Khoong Chan Meng
Future of Learning - Khoong Chan Meng
NUS-ISS31 views
Web Dev - 1 PPT.pdf by gdsczhcet
Web Dev - 1 PPT.pdfWeb Dev - 1 PPT.pdf
Web Dev - 1 PPT.pdf
gdsczhcet52 views
Understanding GenAI/LLM and What is Google Offering - Felix Goh by NUS-ISS
Understanding GenAI/LLM and What is Google Offering - Felix GohUnderstanding GenAI/LLM and What is Google Offering - Felix Goh
Understanding GenAI/LLM and What is Google Offering - Felix Goh
NUS-ISS39 views
AMD: 4th Generation EPYC CXL Demo by CXL Forum
AMD: 4th Generation EPYC CXL DemoAMD: 4th Generation EPYC CXL Demo
AMD: 4th Generation EPYC CXL Demo
CXL Forum126 views
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur by Fwdays
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur
"Thriving Culture in a Product Company — Practical Story", Volodymyr Tsukur
Fwdays40 views
Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen... by NUS-ISS
Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen...Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen...
Upskilling the Evolving Workforce with Digital Fluency for Tomorrow's Challen...
NUS-ISS23 views
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy by Fwdays
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy
"Role of a CTO in software outsourcing company", Yuriy Nakonechnyy
Fwdays40 views
PharoJS - Zürich Smalltalk Group Meetup November 2023 by Noury Bouraqadi
PharoJS - Zürich Smalltalk Group Meetup November 2023PharoJS - Zürich Smalltalk Group Meetup November 2023
PharoJS - Zürich Smalltalk Group Meetup November 2023
Noury Bouraqadi113 views
MemVerge: Gismo (Global IO-free Shared Memory Objects) by CXL Forum
MemVerge: Gismo (Global IO-free Shared Memory Objects)MemVerge: Gismo (Global IO-free Shared Memory Objects)
MemVerge: Gismo (Global IO-free Shared Memory Objects)
CXL Forum112 views
AI: mind, matter, meaning, metaphors, being, becoming, life values by Twain Liu 刘秋艳
AI: mind, matter, meaning, metaphors, being, becoming, life valuesAI: mind, matter, meaning, metaphors, being, becoming, life values
AI: mind, matter, meaning, metaphors, being, becoming, life values
JCon Live 2023 - Lice coding some integration problems by Bernd Ruecker
JCon Live 2023 - Lice coding some integration problemsJCon Live 2023 - Lice coding some integration problems
JCon Live 2023 - Lice coding some integration problems
Bernd Ruecker67 views
Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa... by The Digital Insurer
Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...
Webinar : Competing for tomorrow’s leaders – How MENA insurers can win the wa...

From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience (talk given at Recsys 2012)

  • 1. From a toolkit of recommendation algorithms into a real business: the Gravity R&D experience 13.09.2012.
  • 2. The kick-start 2 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 3. Facing with real needs What we had What clients wanted • rating prediction algorithms • recommendations that • coded in various languages bring revenue • blending mechanism • robustness • accuracy oriented • low response time • easy integration • reporting 3 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 4. What we do? users content of service provider recommender 4 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 5. Explicit vs implicit feedback No ratings but interactions sparse vs. dense matrix requires different learning 5 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 6. Increase revenue: A/B tests against the original solution internally 6 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 7. Robustness Management LAN SNMP Nagios Monitoring HP OpenView Aggregator HTTP HTTP Platform OSS/BSS / SQL / SQL IMPRESS IMPRESS SOAP Application Server #1 Application Server #2 IMPRESS Frontend web server #1 Backend LAN Reco LAN HTTP Load Balancer HTTP(S) Firewall SQL SQL CSV over FTP TV Service LAN IMPRESS Frontend web server #2 Database #1 Database #2 Reporting Subsystem End users 7 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 8. Time requirements • Response time: few ms (max 200) • Training time: maximum few hours • regular retraining • incremental training • Newsletters: • nightly batch run 8 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 9. Productization IMPRESS RECO AD•APT for for for IPTV, CATV and satellite e-commerce ad networks and ad server providers Recommends Recommends Recommends Personally Personally Relevant Relevant Personally Relevant products & services ads Linear TV, VOD, catch-up TV and more Gravity personalization platform 9 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 10. The 5% question – Importance of UI Francisco Martin (Strands): „the algorithm is only 5% in the success of the recommender system” • placement  below or above the fold  scrolling  easy to recognize  floating in • title  not misleading  explanation like • widget  carrousel  static 10 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 11. Recommendation scenario Item2Item recommendation logic: the ad’s profile will be matched to the profile model of available ads 11 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 12. Marketing channels Changing the order of two boxes: 25% CTR increase 12 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 13. Cannibalization • Goal: increase user engagement • Measurements • average visit length • average page views • Effect of accurate recommendations: • use of listing page ↓ • use of item page ↑ • Overall page view: remains the same • Secondary measurements • Contacting • CTR increase 13 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 14. Evolution: increased user engagement • not a cold start problem • parameter optimization and user engagement 14 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 15. KPIs – may change during testing 15 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 16. Complete personalization: coupon-world • Newsletter (daily + occassionally) • Ranking all offers on the website • top1 item • category preferences • user metadata (gender, age, …) • user category preferences (seldom given) • item metadata • context • customer vs. vendor 16 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 17. Business rules – driving/overriding ranking 17 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 18. Contexts 18 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 19. Context at TV program recommendation • TV (EPG program & video-on-demand)  explicit and implicit identification of the user in the household  time-dependent recommendation 19 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 20. (offline) Some results (online) Improvement using season iTALS iTALSx Dataset Recall@20 MAP@20 Recall@20 MAP@20 Grocery 64,31% 137,96% 89,99% 199,82% TV1 14,77% 43,80% 28,66% 85,33% TV2 -7,94% 10,69% 7,77% 14,15% LastFM 96,10% 116,54% 40,98% 254,62% Improvement using Seq iTALS iTALSx Dataset Recall@20 MAP@20 Recall@20 MAP@20 Grocery 84,48% 104,13% 108,83% 122,24% TV1 36,15% 55,07% 26,14% 29,93% 20 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 21. Anecdotes • Item2item recommendations – bookstore • Placebo effect • buyer vs. seller 21 From a toolkit of recommendation algorithms into a real business 13.09.2012.
  • 22. Conclusion • Offline and online testing • From simple to sophisticated • Many more potential fields of application 22 From a toolkit of recommendation algorithms into a real business 13.09.2012.