SlideShare a Scribd company logo
Copyright © 2015 Criteo
New machine learning challenges at Criteo
Olivier Koch
Engineering Program Manager, Criteo
Rythm Meetup
June 15, 2016
Copyright © 2015 Criteo
Banners… what else?
2
Advertiser Publisher
Copyright © 2015 Criteo
Machine learning applications at Criteo
• Bidding (2nd price auctions)
• Product recommendation
• Banner look and feel selection
Copyright © 2015 Criteo
Machine learning at Criteo
• Supervised learning using standard regression methods / optimization algorithms (SGD, L-BFGS)
• Distribution on Hadoop (MapReduce, Spark)
• 3B displays / day
• 40 PB of data -- 15,000 servers
• 7 data centers worldwide
Copyright © 2015 Criteo
Data sparsity
10 000 displays
lead to
50 clicks
lead to
1 sale
Copyright © 2015 Criteo
Now what?
Copyright © 2015 Criteo
Challenges in online advertising
• We have an impact on users
• A user is seen more than 20 times a day in average
• Every bid has an influence on our competitors
• We want to provide a better online advertising experience
• Personalized
• Cross-device
• Long tail (new users, new products)
Copyright © 2015 Criteo
Machine learning challenges
• Optimal bidding strategies under uncertainty -- reinforcement learning, policy learning
• Probabilistic match of devices
• Classification/prediction of time series
• Long tail (users, products) -- transfer learning, factorization
• Offline metrics – counterfactual analysis
Copyright © 2015 Criteo
The good news
• New generations of algorithms
• NLP (word embeddings), reinforcement learning, policy learning, deep networks
• Releases of ML infrastructures
• Caffe on Spark, TensorFlow, Torch, PhotonML, GPUs inside clusters
→ strong traction in the academic/industrial community
Copyright © 2015 Criteo
The good news (c’ed)
• A lot of data is available
• Interactions with banners : clicks
• Interactions with products/advertisers : sales, baskets, home views, listings, visit history
→ faster decision-making in AB test, feature engineering of ML models
• New data is coming : mobile, cross-device, (offline)
→ we need to make sense of it
Copyright © 2015 Criteo
Conclusions
• Machine learning applies well to online advertising at scale
• Yet we still need to improve the users’ experience significantly
• The community is pushing new algorithms and new infrastructures forward
• Lots of new data is coming : we need to make sense of it
Copyright © 2015 Criteo
Thanks! Questions?
o.koch@criteo.com

More Related Content

What's hot

Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
MLconf
 
Criteo Couchbase live 2015
Criteo Couchbase live 2015Criteo Couchbase live 2015
Criteo Couchbase live 2015
Nicolasgmail.com Helleringer
 
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
Digiday
 
Sis mon 1315 sponsored lunch criteo
Sis mon 1315 sponsored lunch criteoSis mon 1315 sponsored lunch criteo
Sis mon 1315 sponsored lunch criteo
MediaPost
 
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
Digiday
 
WebSpectator Presentation deck
WebSpectator Presentation deckWebSpectator Presentation deck
WebSpectator Presentation deck
rkemmer
 
WS `Publisher Deck
WS `Publisher DeckWS `Publisher Deck
WS `Publisher Deck
David Facter
 
Criteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
Criteo's Ad Week 2012 presentation - Big Data and the Value of ClickersCriteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
Criteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
Criteo
 
CPEx Audience – Adform PAB
CPEx Audience – Adform PABCPEx Audience – Adform PAB
CPEx Audience – Adform PAB
Matěj Novák
 
Criteo - NOAH13 London
Criteo - NOAH13 LondonCriteo - NOAH13 London
Criteo - NOAH13 London
NOAH Advisors
 
3 Minute Introduction
3 Minute Introduction3 Minute Introduction
3 Minute Introduction
Julian Tol
 
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation SlidesthinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
thinkLA
 
Essent - Using AI@Scale to drive a seamless customer experience
Essent - Using AI@Scale to drive a seamless customer experienceEssent - Using AI@Scale to drive a seamless customer experience
Essent - Using AI@Scale to drive a seamless customer experience
BBP
 
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation SlidesthinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
thinkLA
 
IAB Europe Virtual Programmatic Day H1 2018 Presentation
IAB Europe Virtual Programmatic Day H1 2018 PresentationIAB Europe Virtual Programmatic Day H1 2018 Presentation
IAB Europe Virtual Programmatic Day H1 2018 Presentation
IAB Europe
 
Using Programmatic to Reach People in the Moments that Matter
Using Programmatic to Reach People in the Moments that MatterUsing Programmatic to Reach People in the Moments that Matter
Using Programmatic to Reach People in the Moments that Matter
Incubeta NMPi
 
The History of Advertising Technology
The History of Advertising TechnologyThe History of Advertising Technology
The History of Advertising Technology
Clearcode
 
Snowplow at Sigfig
Snowplow at SigfigSnowplow at Sigfig
Snowplow at Sigfig
yalisassoon
 
Programmatic Advertising: How To Join In On the Fun
Programmatic Advertising: How To Join In On the FunProgrammatic Advertising: How To Join In On the Fun
Programmatic Advertising: How To Join In On the Fun
Hanapin Marketing
 
What You Need To Know For A Successful Start on DoubleClick Bid Manager
What You Need To Know For A Successful Start on DoubleClick Bid ManagerWhat You Need To Know For A Successful Start on DoubleClick Bid Manager
What You Need To Know For A Successful Start on DoubleClick Bid Manager
Hanapin Marketing
 

What's hot (20)

Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
Amanda Casari, Senior Data Scientist, Concur at MLconf SEA - 5/20/16
 
Criteo Couchbase live 2015
Criteo Couchbase live 2015Criteo Couchbase live 2015
Criteo Couchbase live 2015
 
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
Back to the Future: Bringing Performance Targeting to Mobile Devices from DRS...
 
Sis mon 1315 sponsored lunch criteo
Sis mon 1315 sponsored lunch criteoSis mon 1315 sponsored lunch criteo
Sis mon 1315 sponsored lunch criteo
 
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
Aligning with Buyers to Maximize Mobile Revenue presentation by Criteo at DPS...
 
WebSpectator Presentation deck
WebSpectator Presentation deckWebSpectator Presentation deck
WebSpectator Presentation deck
 
WS `Publisher Deck
WS `Publisher DeckWS `Publisher Deck
WS `Publisher Deck
 
Criteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
Criteo's Ad Week 2012 presentation - Big Data and the Value of ClickersCriteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
Criteo's Ad Week 2012 presentation - Big Data and the Value of Clickers
 
CPEx Audience – Adform PAB
CPEx Audience – Adform PABCPEx Audience – Adform PAB
CPEx Audience – Adform PAB
 
Criteo - NOAH13 London
Criteo - NOAH13 LondonCriteo - NOAH13 London
Criteo - NOAH13 London
 
3 Minute Introduction
3 Minute Introduction3 Minute Introduction
3 Minute Introduction
 
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation SlidesthinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
thinkLA Programmatic Summit 2014 Aimee Gerry Presentation Slides
 
Essent - Using AI@Scale to drive a seamless customer experience
Essent - Using AI@Scale to drive a seamless customer experienceEssent - Using AI@Scale to drive a seamless customer experience
Essent - Using AI@Scale to drive a seamless customer experience
 
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation SlidesthinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
thinkLA Programmatic Summit 2014 Carl Kalapesi Presentation Slides
 
IAB Europe Virtual Programmatic Day H1 2018 Presentation
IAB Europe Virtual Programmatic Day H1 2018 PresentationIAB Europe Virtual Programmatic Day H1 2018 Presentation
IAB Europe Virtual Programmatic Day H1 2018 Presentation
 
Using Programmatic to Reach People in the Moments that Matter
Using Programmatic to Reach People in the Moments that MatterUsing Programmatic to Reach People in the Moments that Matter
Using Programmatic to Reach People in the Moments that Matter
 
The History of Advertising Technology
The History of Advertising TechnologyThe History of Advertising Technology
The History of Advertising Technology
 
Snowplow at Sigfig
Snowplow at SigfigSnowplow at Sigfig
Snowplow at Sigfig
 
Programmatic Advertising: How To Join In On the Fun
Programmatic Advertising: How To Join In On the FunProgrammatic Advertising: How To Join In On the Fun
Programmatic Advertising: How To Join In On the Fun
 
What You Need To Know For A Successful Start on DoubleClick Bid Manager
What You Need To Know For A Successful Start on DoubleClick Bid ManagerWhat You Need To Know For A Successful Start on DoubleClick Bid Manager
What You Need To Know For A Successful Start on DoubleClick Bid Manager
 

Viewers also liked

C# development workflow @ criteo
C# development workflow @ criteoC# development workflow @ criteo
C# development workflow @ criteo
Ibrahim Abubakari
 
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
Paris Monitoring
 
Infographic: How Professional Services Automation can transform your Company!
Infographic: How Professional Services Automation can transform your Company!Infographic: How Professional Services Automation can transform your Company!
Infographic: How Professional Services Automation can transform your Company!
Changepoint
 
Saintjo Two AV4
Saintjo Two AV4Saintjo Two AV4
Saintjo Two AV4
Bernard Alexandre
 
Etude Criteo - la valeur réelle des internautes qui cliquent - 26 Juin 2012
Etude Criteo - la valeur réelle des internautes qui cliquent  - 26 Juin 2012Etude Criteo - la valeur réelle des internautes qui cliquent  - 26 Juin 2012
Etude Criteo - la valeur réelle des internautes qui cliquent - 26 Juin 2012Romain Fonnier
 
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
MongoDB
 
Google for Mobile: Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
Google for Mobile:  Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛Google for Mobile:  Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
Google for Mobile: Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
Google Cloud Platform - Japan
 
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
Amazon Web Services
 
RecsysFR: Criteo presentation
RecsysFR: Criteo presentationRecsysFR: Criteo presentation
RecsysFR: Criteo presentation
recsysfr
 
Google for Mobile: コンテナで作るモバイル バックエンド - 福田 潔
Google for Mobile:  コンテナで作るモバイル バックエンド - 福田 潔Google for Mobile:  コンテナで作るモバイル バックエンド - 福田 潔
Google for Mobile: コンテナで作るモバイル バックエンド - 福田 潔
Google Cloud Platform - Japan
 
Hadoop summit-ams-2014-04-03
Hadoop summit-ams-2014-04-03Hadoop summit-ams-2014-04-03
Hadoop summit-ams-2014-04-03
SDanzanvilliersCriteo
 
RecSys 2015: Large-scale real-time product recommendation at Criteo
RecSys 2015: Large-scale real-time product recommendation at CriteoRecSys 2015: Large-scale real-time product recommendation at Criteo
RecSys 2015: Large-scale real-time product recommendation at Criteo
Romain Lerallut
 
Response prediction for display advertising - WSDM 2014
Response prediction for display advertising - WSDM 2014Response prediction for display advertising - WSDM 2014
Response prediction for display advertising - WSDM 2014
Olivier Chapelle
 
Taga arthur d little
Taga arthur d littleTaga arthur d little
Taga arthur d little
southmos
 
A.T. Kearney: Positioning for the Telematics Tipping Point
A.T. Kearney: Positioning for the Telematics Tipping PointA.T. Kearney: Positioning for the Telematics Tipping Point
A.T. Kearney: Positioning for the Telematics Tipping Point
bengillTU
 
Roland berger best_practices_in_new_product_development_20130419
Roland berger best_practices_in_new_product_development_20130419Roland berger best_practices_in_new_product_development_20130419
Roland berger best_practices_in_new_product_development_20130419
Alberto Garcia Romera
 
2016 투자동향보고서
2016 투자동향보고서2016 투자동향보고서
2016 투자동향보고서
Platum
 
McKinsey: Understanding shifts in consumer behavior
McKinsey: Understanding shifts in consumer behaviorMcKinsey: Understanding shifts in consumer behavior
McKinsey: Understanding shifts in consumer behavior
McKinsey on Marketing & Sales
 

Viewers also liked (18)

C# development workflow @ criteo
C# development workflow @ criteoC# development workflow @ criteo
C# development workflow @ criteo
 
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
Bizarre... vous avez dit bizarre - Paris Monitoring meetup #2
 
Infographic: How Professional Services Automation can transform your Company!
Infographic: How Professional Services Automation can transform your Company!Infographic: How Professional Services Automation can transform your Company!
Infographic: How Professional Services Automation can transform your Company!
 
Saintjo Two AV4
Saintjo Two AV4Saintjo Two AV4
Saintjo Two AV4
 
Etude Criteo - la valeur réelle des internautes qui cliquent - 26 Juin 2012
Etude Criteo - la valeur réelle des internautes qui cliquent  - 26 Juin 2012Etude Criteo - la valeur réelle des internautes qui cliquent  - 26 Juin 2012
Etude Criteo - la valeur réelle des internautes qui cliquent - 26 Juin 2012
 
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
Morning with MongoDB Paris 2012 - Cas d'usages courant en entreprise. Présent...
 
Google for Mobile: Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
Google for Mobile:  Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛Google for Mobile:  Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
Google for Mobile: Google スケールで構築する! ゲームインフラと分析環境 - 橋口 剛
 
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
AWS Summit Paris - Track 1 - Session 2 - Designez vos architectures pour plus...
 
RecsysFR: Criteo presentation
RecsysFR: Criteo presentationRecsysFR: Criteo presentation
RecsysFR: Criteo presentation
 
Google for Mobile: コンテナで作るモバイル バックエンド - 福田 潔
Google for Mobile:  コンテナで作るモバイル バックエンド - 福田 潔Google for Mobile:  コンテナで作るモバイル バックエンド - 福田 潔
Google for Mobile: コンテナで作るモバイル バックエンド - 福田 潔
 
Hadoop summit-ams-2014-04-03
Hadoop summit-ams-2014-04-03Hadoop summit-ams-2014-04-03
Hadoop summit-ams-2014-04-03
 
RecSys 2015: Large-scale real-time product recommendation at Criteo
RecSys 2015: Large-scale real-time product recommendation at CriteoRecSys 2015: Large-scale real-time product recommendation at Criteo
RecSys 2015: Large-scale real-time product recommendation at Criteo
 
Response prediction for display advertising - WSDM 2014
Response prediction for display advertising - WSDM 2014Response prediction for display advertising - WSDM 2014
Response prediction for display advertising - WSDM 2014
 
Taga arthur d little
Taga arthur d littleTaga arthur d little
Taga arthur d little
 
A.T. Kearney: Positioning for the Telematics Tipping Point
A.T. Kearney: Positioning for the Telematics Tipping PointA.T. Kearney: Positioning for the Telematics Tipping Point
A.T. Kearney: Positioning for the Telematics Tipping Point
 
Roland berger best_practices_in_new_product_development_20130419
Roland berger best_practices_in_new_product_development_20130419Roland berger best_practices_in_new_product_development_20130419
Roland berger best_practices_in_new_product_development_20130419
 
2016 투자동향보고서
2016 투자동향보고서2016 투자동향보고서
2016 투자동향보고서
 
McKinsey: Understanding shifts in consumer behavior
McKinsey: Understanding shifts in consumer behaviorMcKinsey: Understanding shifts in consumer behavior
McKinsey: Understanding shifts in consumer behavior
 

Similar to New machine learning challenges at Criteo

Recommendation at scale
Recommendation at scaleRecommendation at scale
Recommendation at scale
simondolle
 
Kritter introduction - advertiser
Kritter   introduction - advertiserKritter   introduction - advertiser
Kritter introduction - advertiser
Krittercorporate
 
Kritter introduction - technology player
Kritter   introduction - technology playerKritter   introduction - technology player
Kritter introduction - technology player
Krittercorporate
 
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
Dion Hinchcliffe
 
Mumbrella - Programmatic For Marketers
Mumbrella - Programmatic For MarketersMumbrella - Programmatic For Marketers
Mumbrella - Programmatic For Marketers
Louder
 
Big Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to ActionBig Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to Action
Murtaza Doctor
 
100 day plan - Technology Vision Australian Perspective
100 day plan - Technology Vision Australian Perspective100 day plan - Technology Vision Australian Perspective
100 day plan - Technology Vision Australian Perspective
accenture
 
Vital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
Vital Trends in Digital and Social in 2015 and Beyond by Dion HinchcliffeVital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
Vital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
7Summits
 
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
Dion Hinchcliffe
 
Webinar: Retargeting to the Max
Webinar: Retargeting to the MaxWebinar: Retargeting to the Max
Webinar: Retargeting to the Max
Katana Media
 
Embitel's Motto on Customer Experience and Changes they Bring in Digital Expe...
Embitel's Motto on Customer Experience and Changes they Bring in Digital Expe...Embitel's Motto on Customer Experience and Changes they Bring in Digital Expe...
Embitel's Motto on Customer Experience and Changes they Bring in Digital Expe...
Embitel Technologies (I) PVT LTD
 
Selling Display to Your SMB Clients
Selling Display to Your SMB ClientsSelling Display to Your SMB Clients
Selling Display to Your SMB Clients
Acquisio
 
Selling display display to your smb clients final
Selling display display to your smb clients   finalSelling display display to your smb clients   final
Selling display display to your smb clients final
Acquisio
 
UK Integration WebSphere User Group - MultiSpeed IT
UK Integration WebSphere User Group - MultiSpeed ITUK Integration WebSphere User Group - MultiSpeed IT
UK Integration WebSphere User Group - MultiSpeed IT
AndyHumphreys
 
Machine Learning for Computational Advertising
Machine Learning for Computational AdvertisingMachine Learning for Computational Advertising
Machine Learning for Computational Advertising
Paris Women in Machine Learning and Data Science
 
Migrating thinking not just applications to the cloud
Migrating thinking not just applications to the cloudMigrating thinking not just applications to the cloud
Migrating thinking not just applications to the cloud
Cloudsoft
 
Simon Dollé_Large-scale Real-time recommendation at Criteo
Simon Dollé_Large-scale Real-time recommendation at Criteo Simon Dollé_Large-scale Real-time recommendation at Criteo
Simon Dollé_Large-scale Real-time recommendation at Criteo
Dataconomy Media
 
Narrative Mind Week 8 H4D Stanford 2016
Narrative Mind Week 8 H4D Stanford 2016Narrative Mind Week 8 H4D Stanford 2016
Narrative Mind Week 8 H4D Stanford 2016
Stanford University
 
Capgemini Insights and Data
Capgemini Insights and Data Capgemini Insights and Data
Capgemini Insights and Data
DataWorks Summit/Hadoop Summit
 
Point roll webinar_your brand everywhere_december 2011_external
Point roll webinar_your brand everywhere_december 2011_externalPoint roll webinar_your brand everywhere_december 2011_external
Point roll webinar_your brand everywhere_december 2011_external
Michael Zarcone
 

Similar to New machine learning challenges at Criteo (20)

Recommendation at scale
Recommendation at scaleRecommendation at scale
Recommendation at scale
 
Kritter introduction - advertiser
Kritter   introduction - advertiserKritter   introduction - advertiser
Kritter introduction - advertiser
 
Kritter introduction - technology player
Kritter   introduction - technology playerKritter   introduction - technology player
Kritter introduction - technology player
 
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
Vital Trends in Digital Experience and Transformation in 2016 | Dreamforce 20...
 
Mumbrella - Programmatic For Marketers
Mumbrella - Programmatic For MarketersMumbrella - Programmatic For Marketers
Mumbrella - Programmatic For Marketers
 
Big Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to ActionBig Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to Action
 
100 day plan - Technology Vision Australian Perspective
100 day plan - Technology Vision Australian Perspective100 day plan - Technology Vision Australian Perspective
100 day plan - Technology Vision Australian Perspective
 
Vital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
Vital Trends in Digital and Social in 2015 and Beyond by Dion HinchcliffeVital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
Vital Trends in Digital and Social in 2015 and Beyond by Dion Hinchcliffe
 
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
Vital Trends in Digital and Social in 2015 and Beyond | Dreamforce 2015 by Di...
 
Webinar: Retargeting to the Max
Webinar: Retargeting to the MaxWebinar: Retargeting to the Max
Webinar: Retargeting to the Max
 
Embitel's Motto on Customer Experience and Changes they Bring in Digital Expe...
Embitel's Motto on Customer Experience and Changes they Bring in Digital Expe...Embitel's Motto on Customer Experience and Changes they Bring in Digital Expe...
Embitel's Motto on Customer Experience and Changes they Bring in Digital Expe...
 
Selling Display to Your SMB Clients
Selling Display to Your SMB ClientsSelling Display to Your SMB Clients
Selling Display to Your SMB Clients
 
Selling display display to your smb clients final
Selling display display to your smb clients   finalSelling display display to your smb clients   final
Selling display display to your smb clients final
 
UK Integration WebSphere User Group - MultiSpeed IT
UK Integration WebSphere User Group - MultiSpeed ITUK Integration WebSphere User Group - MultiSpeed IT
UK Integration WebSphere User Group - MultiSpeed IT
 
Machine Learning for Computational Advertising
Machine Learning for Computational AdvertisingMachine Learning for Computational Advertising
Machine Learning for Computational Advertising
 
Migrating thinking not just applications to the cloud
Migrating thinking not just applications to the cloudMigrating thinking not just applications to the cloud
Migrating thinking not just applications to the cloud
 
Simon Dollé_Large-scale Real-time recommendation at Criteo
Simon Dollé_Large-scale Real-time recommendation at Criteo Simon Dollé_Large-scale Real-time recommendation at Criteo
Simon Dollé_Large-scale Real-time recommendation at Criteo
 
Narrative Mind Week 8 H4D Stanford 2016
Narrative Mind Week 8 H4D Stanford 2016Narrative Mind Week 8 H4D Stanford 2016
Narrative Mind Week 8 H4D Stanford 2016
 
Capgemini Insights and Data
Capgemini Insights and Data Capgemini Insights and Data
Capgemini Insights and Data
 
Point roll webinar_your brand everywhere_december 2011_external
Point roll webinar_your brand everywhere_december 2011_externalPoint roll webinar_your brand everywhere_december 2011_external
Point roll webinar_your brand everywhere_december 2011_external
 

Recently uploaded

Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
Neo4j
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
Neo4j
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
Edge AI and Vision Alliance
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 

Recently uploaded (20)

Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Leveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and StandardsLeveraging the Graph for Clinical Trials and Standards
Leveraging the Graph for Clinical Trials and Standards
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge GraphGraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
GraphRAG for LifeSciences Hands-On with the Clinical Knowledge Graph
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
“How Axelera AI Uses Digital Compute-in-memory to Deliver Fast and Energy-eff...
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 

New machine learning challenges at Criteo

  • 1. Copyright © 2015 Criteo New machine learning challenges at Criteo Olivier Koch Engineering Program Manager, Criteo Rythm Meetup June 15, 2016
  • 2. Copyright © 2015 Criteo Banners… what else? 2 Advertiser Publisher
  • 3. Copyright © 2015 Criteo Machine learning applications at Criteo • Bidding (2nd price auctions) • Product recommendation • Banner look and feel selection
  • 4. Copyright © 2015 Criteo Machine learning at Criteo • Supervised learning using standard regression methods / optimization algorithms (SGD, L-BFGS) • Distribution on Hadoop (MapReduce, Spark) • 3B displays / day • 40 PB of data -- 15,000 servers • 7 data centers worldwide
  • 5. Copyright © 2015 Criteo Data sparsity 10 000 displays lead to 50 clicks lead to 1 sale
  • 6. Copyright © 2015 Criteo Now what?
  • 7. Copyright © 2015 Criteo Challenges in online advertising • We have an impact on users • A user is seen more than 20 times a day in average • Every bid has an influence on our competitors • We want to provide a better online advertising experience • Personalized • Cross-device • Long tail (new users, new products)
  • 8. Copyright © 2015 Criteo Machine learning challenges • Optimal bidding strategies under uncertainty -- reinforcement learning, policy learning • Probabilistic match of devices • Classification/prediction of time series • Long tail (users, products) -- transfer learning, factorization • Offline metrics – counterfactual analysis
  • 9. Copyright © 2015 Criteo The good news • New generations of algorithms • NLP (word embeddings), reinforcement learning, policy learning, deep networks • Releases of ML infrastructures • Caffe on Spark, TensorFlow, Torch, PhotonML, GPUs inside clusters → strong traction in the academic/industrial community
  • 10. Copyright © 2015 Criteo The good news (c’ed) • A lot of data is available • Interactions with banners : clicks • Interactions with products/advertisers : sales, baskets, home views, listings, visit history → faster decision-making in AB test, feature engineering of ML models • New data is coming : mobile, cross-device, (offline) → we need to make sense of it
  • 11. Copyright © 2015 Criteo Conclusions • Machine learning applies well to online advertising at scale • Yet we still need to improve the users’ experience significantly • The community is pushing new algorithms and new infrastructures forward • Lots of new data is coming : we need to make sense of it
  • 12. Copyright © 2015 Criteo Thanks! Questions? o.koch@criteo.com