SlideShare a Scribd company logo
RecSys FR: 4th Session
6th October 2016
Etienne Sanson – manager R&D Engine
About Criteo
1
3 | Copyright © 2016 Criteo
Our mission
TARGET THE
RIGHT USER
AT THE
RIGHT TIME
WITH THE RIGHT
MESSAGE
4 | Copyright © 2016 Criteo
Key Figures
16 000
PUBLISHERS
90%
RETENTION RATE2
+130
COUNTRIES
LISTED ON THE
NASDAQ
SINCE
OCTOBER 2013
R&D REPRESENTS 21%
OF THE WORKFORCE
2000
EMPLOYEES
21
BILLIONS $3
11 000
ADVERTISERS
1.19 bn€1
31
OFFICES
1: REVENUE IN 2015
2: ANNUAL RATE 2015
3: $ OF TURNOVER GENERATED TO OUR CLIENTS - TURNOVER POST-CLICK WW FROM JANUARY TO DECEMBER
2015
5 | Copyright © 2016 Criteo
Revenue Growth
2009 2010 2011 2012 2013
22M$
86M$
199M$
349M$
589M$
988M$
2014
1,3MM$
2015
6 | Copyright © 2016 Criteo
GENERAL CONCEPT
Users visit an
advertiser’s website
1
Criteo identifies the users
(via cookies)
2
Users leave the advertiser’s website
& browse publisher on the Internet
3
Criteo identifies users on
these pages
(via cookie)
4
Criteo displays an advertising
banner, personalized for
each user
5
Click through directly
to the advertiser’s
page
6
@
Retargeting principles
7 | Copyright © 2016 Criteo
Infrastructure Key Figures
Sunnyvale
2 PoP
500 kVA
1 559 Servers
New York
2 PoP
930 kVA
2 625 Servers
Hong Kong
2 PoP
472 kVA
2185 Servers
Paris
4 Pop
1 800 kVA
3 625 Servers
Amsterdam
2 PoP
+2 500 kVA
3 609 Servers
Tokyo
2 PoP
455 kVA
2 564 Servers
Shanghai
1 PoP
200 kVA
931 Servers
World Wide
15 PoP
6,8 MVA
17 098 Servers
> 55Gbps
+ 2.5M req/s
Hosting Global Partners :
About ML@Criteo
2
9 | Copyright © 2016 Criteo
Our challenges – Product recommendation
• Select the best ~10 products to show to a user
• >1B users
• Product catalog contains ~1M items, up to 1B
• Time constraints: 20ms
• Combination of offline/online processing steps
• CF
• Product embeddings (word2vec -> prod2vec)
• CNN for image features
What products should
we recommend?
10 | Copyright © 2016 Criteo
Our challenges - Bidding
How much should we
bid for this display?
What is the best
campaign to display?
My company
BUY! BUY! BUY!
BUY!
• Select the best campaign to display and evaluate its
value in a few ms
• Large scale regression models
• >1B daily displays (but few positive examples!)
• >1M parameters
• Distributed optimization (SGD, L-BFGS)
• Feature Engineering
• Transfer learning, FFM, Policy learning
• Marketplace, game theory, auction theory
11 | Copyright © 2016 Criteo
Our challenges – X-device
• Build a huge graph (Billions of nodes/edges):
• Nodes = devices
• Edge = the 2 devices belong to the same user
• How to connect 2 devices?
• How to know the ground truth?
• How to keep it stable?
• At scale & taking care about privacy
Who is the user
behind the device?
12 | Copyright © 2016 Criteo
Our challenges – Testing
• We test everything!
• Offline tests / AB Tests
• Infrastructure to perform large-scale tests
• >100K offline tests / year
• >1K AB Tests / year
• Dedicated teams
• Technical / Business Metrics
• Randomization
• Counterfactual evaluation
Thank you!
…and we’re hiring!

More Related Content

Viewers also liked

Injecting semantic links into a graph-based recommender system
Injecting semantic links into a graph-based recommender systemInjecting semantic links into a graph-based recommender system
Injecting semantic links into a graph-based recommender system
recsysfr
 
What can bring library metadata to the web? Trust, links and love
What can bring library metadata to the web? Trust, links and loveWhat can bring library metadata to the web? Trust, links and love
What can bring library metadata to the web? Trust, links and love
recsysfr
 
Dictionary Learning for Massive Matrix Factorization
Dictionary Learning for Massive Matrix FactorizationDictionary Learning for Massive Matrix Factorization
Dictionary Learning for Massive Matrix Factorization
recsysfr
 
Predictive quality metrics @ tinyclues - Artem Kozhevnikov - Tinyclues
Predictive quality metrics @ tinyclues - Artem Kozhevnikov - TinycluesPredictive quality metrics @ tinyclues - Artem Kozhevnikov - Tinyclues
Predictive quality metrics @ tinyclues - Artem Kozhevnikov - Tinyclues
recsysfr
 
Recommendation @ Meetic
Recommendation @ MeeticRecommendation @ Meetic
Recommendation @ Meetic
recsysfr
 
Meta-Prod2Vec: Simple Product Embeddings with Side-Information
Meta-Prod2Vec: Simple Product Embeddings with Side-InformationMeta-Prod2Vec: Simple Product Embeddings with Side-Information
Meta-Prod2Vec: Simple Product Embeddings with Side-Information
recsysfr
 
Sequential Learning in the Position-Based Model
Sequential Learning in the Position-Based ModelSequential Learning in the Position-Based Model
Sequential Learning in the Position-Based Model
recsysfr
 
CONTENT2VEC: a Joint Architecture to use Product Image and Text for the task ...
CONTENT2VEC: a Joint Architecture to use Product Image and Text for the task ...CONTENT2VEC: a Joint Architecture to use Product Image and Text for the task ...
CONTENT2VEC: a Joint Architecture to use Product Image and Text for the task ...
recsysfr
 
Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanusso...
Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanusso...Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanusso...
Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanusso...
recsysfr
 

Viewers also liked (9)

Injecting semantic links into a graph-based recommender system
Injecting semantic links into a graph-based recommender systemInjecting semantic links into a graph-based recommender system
Injecting semantic links into a graph-based recommender system
 
What can bring library metadata to the web? Trust, links and love
What can bring library metadata to the web? Trust, links and loveWhat can bring library metadata to the web? Trust, links and love
What can bring library metadata to the web? Trust, links and love
 
Dictionary Learning for Massive Matrix Factorization
Dictionary Learning for Massive Matrix FactorizationDictionary Learning for Massive Matrix Factorization
Dictionary Learning for Massive Matrix Factorization
 
Predictive quality metrics @ tinyclues - Artem Kozhevnikov - Tinyclues
Predictive quality metrics @ tinyclues - Artem Kozhevnikov - TinycluesPredictive quality metrics @ tinyclues - Artem Kozhevnikov - Tinyclues
Predictive quality metrics @ tinyclues - Artem Kozhevnikov - Tinyclues
 
Recommendation @ Meetic
Recommendation @ MeeticRecommendation @ Meetic
Recommendation @ Meetic
 
Meta-Prod2Vec: Simple Product Embeddings with Side-Information
Meta-Prod2Vec: Simple Product Embeddings with Side-InformationMeta-Prod2Vec: Simple Product Embeddings with Side-Information
Meta-Prod2Vec: Simple Product Embeddings with Side-Information
 
Sequential Learning in the Position-Based Model
Sequential Learning in the Position-Based ModelSequential Learning in the Position-Based Model
Sequential Learning in the Position-Based Model
 
CONTENT2VEC: a Joint Architecture to use Product Image and Text for the task ...
CONTENT2VEC: a Joint Architecture to use Product Image and Text for the task ...CONTENT2VEC: a Joint Architecture to use Product Image and Text for the task ...
CONTENT2VEC: a Joint Architecture to use Product Image and Text for the task ...
 
Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanusso...
Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanusso...Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanusso...
Highlights on most interesting RecSys papers - Elena Smirnova, Lowik Chanusso...
 

Similar to RecsysFR: Criteo presentation

Selling Display to Your SMB Clients
Selling Display to Your SMB ClientsSelling Display to Your SMB Clients
Selling Display to Your SMB Clients
Acquisio
 
Selling display display to your smb clients final
Selling display display to your smb clients   finalSelling display display to your smb clients   final
Selling display display to your smb clients final
Acquisio
 
Thomvest Advertising Technology overview - Sept 2014
Thomvest Advertising Technology overview - Sept 2014Thomvest Advertising Technology overview - Sept 2014
Thomvest Advertising Technology overview - Sept 2014
andrewtweed1
 
Tag Management : A Key Component of an International Digital Strategy
Tag Management : A Key Component of an International Digital StrategyTag Management : A Key Component of an International Digital Strategy
Tag Management : A Key Component of an International Digital Strategy
Ensighten
 
Seedtag Media Kit 2024 Faaor UAE and KSA
Seedtag Media Kit 2024 Faaor UAE and KSASeedtag Media Kit 2024 Faaor UAE and KSA
Seedtag Media Kit 2024 Faaor UAE and KSA
AdilHussain801079
 
Oct-2016_AdAsia Credentials
Oct-2016_AdAsia CredentialsOct-2016_AdAsia Credentials
Oct-2016_AdAsia CredentialsDang Pham
 
Machine Learning for Computational Advertising
Machine Learning for Computational AdvertisingMachine Learning for Computational Advertising
Machine Learning for Computational Advertising
Paris Women in Machine Learning and Data Science
 
EBU (Geneva, April 2016)
EBU (Geneva, April 2016)EBU (Geneva, April 2016)
EBU (Geneva, April 2016)
Nice People At Work
 
Big Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to ActionBig Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to Action
Murtaza Doctor
 
From Analytics to Intelligence
From Analytics to Intelligence From Analytics to Intelligence
From Analytics to Intelligence
Catherine Mylinh
 
SCREENlens - People Tracking with Quividi
SCREENlens - People Tracking with QuividiSCREENlens - People Tracking with Quividi
SCREENlens - People Tracking with Quividi
pilot Screentime GmbH
 
The Importance of a Data-Driven Dynamic Creative Strategy
The Importance of a Data-Driven Dynamic Creative StrategyThe Importance of a Data-Driven Dynamic Creative Strategy
The Importance of a Data-Driven Dynamic Creative Strategy
In Marketing We Trust
 
Smaato - NOAH16 Berlin
Smaato - NOAH16 BerlinSmaato - NOAH16 Berlin
Smaato - NOAH16 Berlin
NOAH Advisors
 
S4M - NOAH15 London
S4M - NOAH15 LondonS4M - NOAH15 London
S4M - NOAH15 London
NOAH Advisors
 
Media Management Workshop by OpenText and Skillz Middle East
Media Management Workshop by OpenText and Skillz Middle EastMedia Management Workshop by OpenText and Skillz Middle East
Media Management Workshop by OpenText and Skillz Middle East
Dieter Hovorka
 
Zinnov Zones - Media & Technology 2016
Zinnov Zones - Media & Technology 2016Zinnov Zones - Media & Technology 2016
Zinnov Zones - Media & Technology 2016
Zinnov
 
Odoo Strategy and Roadmap
Odoo Strategy and RoadmapOdoo Strategy and Roadmap
Odoo Strategy and Roadmap
Odoo
 
Keynotes Odoo - 2014 Opendays by Fabien (CEO @ Odoo)
Keynotes Odoo - 2014 Opendays by Fabien (CEO @ Odoo) Keynotes Odoo - 2014 Opendays by Fabien (CEO @ Odoo)
Keynotes Odoo - 2014 Opendays by Fabien (CEO @ Odoo)
Mustufa Rangwala
 
The Very Best Intranets & Digital Workplace from the 2016 Intranet Global Forum
The Very Best Intranets & Digital Workplace from the 2016 Intranet Global ForumThe Very Best Intranets & Digital Workplace from the 2016 Intranet Global Forum
The Very Best Intranets & Digital Workplace from the 2016 Intranet Global Forum
Prescient Digital Media
 
If you are not paying for it, you are the product: How much do advertisers p...
 If you are not paying for it, you are the product: How much do advertisers p... If you are not paying for it, you are the product: How much do advertisers p...
If you are not paying for it, you are the product: How much do advertisers p...
Panagiotis Papadopoulos
 

Similar to RecsysFR: Criteo presentation (20)

Selling Display to Your SMB Clients
Selling Display to Your SMB ClientsSelling Display to Your SMB Clients
Selling Display to Your SMB Clients
 
Selling display display to your smb clients final
Selling display display to your smb clients   finalSelling display display to your smb clients   final
Selling display display to your smb clients final
 
Thomvest Advertising Technology overview - Sept 2014
Thomvest Advertising Technology overview - Sept 2014Thomvest Advertising Technology overview - Sept 2014
Thomvest Advertising Technology overview - Sept 2014
 
Tag Management : A Key Component of an International Digital Strategy
Tag Management : A Key Component of an International Digital StrategyTag Management : A Key Component of an International Digital Strategy
Tag Management : A Key Component of an International Digital Strategy
 
Seedtag Media Kit 2024 Faaor UAE and KSA
Seedtag Media Kit 2024 Faaor UAE and KSASeedtag Media Kit 2024 Faaor UAE and KSA
Seedtag Media Kit 2024 Faaor UAE and KSA
 
Oct-2016_AdAsia Credentials
Oct-2016_AdAsia CredentialsOct-2016_AdAsia Credentials
Oct-2016_AdAsia Credentials
 
Machine Learning for Computational Advertising
Machine Learning for Computational AdvertisingMachine Learning for Computational Advertising
Machine Learning for Computational Advertising
 
EBU (Geneva, April 2016)
EBU (Geneva, April 2016)EBU (Geneva, April 2016)
EBU (Geneva, April 2016)
 
Big Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to ActionBig Data at Tube: Events to Insights to Action
Big Data at Tube: Events to Insights to Action
 
From Analytics to Intelligence
From Analytics to Intelligence From Analytics to Intelligence
From Analytics to Intelligence
 
SCREENlens - People Tracking with Quividi
SCREENlens - People Tracking with QuividiSCREENlens - People Tracking with Quividi
SCREENlens - People Tracking with Quividi
 
The Importance of a Data-Driven Dynamic Creative Strategy
The Importance of a Data-Driven Dynamic Creative StrategyThe Importance of a Data-Driven Dynamic Creative Strategy
The Importance of a Data-Driven Dynamic Creative Strategy
 
Smaato - NOAH16 Berlin
Smaato - NOAH16 BerlinSmaato - NOAH16 Berlin
Smaato - NOAH16 Berlin
 
S4M - NOAH15 London
S4M - NOAH15 LondonS4M - NOAH15 London
S4M - NOAH15 London
 
Media Management Workshop by OpenText and Skillz Middle East
Media Management Workshop by OpenText and Skillz Middle EastMedia Management Workshop by OpenText and Skillz Middle East
Media Management Workshop by OpenText and Skillz Middle East
 
Zinnov Zones - Media & Technology 2016
Zinnov Zones - Media & Technology 2016Zinnov Zones - Media & Technology 2016
Zinnov Zones - Media & Technology 2016
 
Odoo Strategy and Roadmap
Odoo Strategy and RoadmapOdoo Strategy and Roadmap
Odoo Strategy and Roadmap
 
Keynotes Odoo - 2014 Opendays by Fabien (CEO @ Odoo)
Keynotes Odoo - 2014 Opendays by Fabien (CEO @ Odoo) Keynotes Odoo - 2014 Opendays by Fabien (CEO @ Odoo)
Keynotes Odoo - 2014 Opendays by Fabien (CEO @ Odoo)
 
The Very Best Intranets & Digital Workplace from the 2016 Intranet Global Forum
The Very Best Intranets & Digital Workplace from the 2016 Intranet Global ForumThe Very Best Intranets & Digital Workplace from the 2016 Intranet Global Forum
The Very Best Intranets & Digital Workplace from the 2016 Intranet Global Forum
 
If you are not paying for it, you are the product: How much do advertisers p...
 If you are not paying for it, you are the product: How much do advertisers p... If you are not paying for it, you are the product: How much do advertisers p...
If you are not paying for it, you are the product: How much do advertisers p...
 

More from recsysfr

Multi Task DPP for Basket Completion by Romain WARLOP, Fifty Five
Multi Task DPP for Basket Completion by Romain WARLOP, Fifty FiveMulti Task DPP for Basket Completion by Romain WARLOP, Fifty Five
Multi Task DPP for Basket Completion by Romain WARLOP, Fifty Five
recsysfr
 
Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...
Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...
Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...
recsysfr
 
An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...
An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...
An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...
recsysfr
 
Recommendations @ Rakuten Group
Recommendations @ Rakuten GroupRecommendations @ Rakuten Group
Recommendations @ Rakuten Group
recsysfr
 
Data-Driven Recommender Systems
Data-Driven Recommender SystemsData-Driven Recommender Systems
Data-Driven Recommender Systems
recsysfr
 
Recommender systems
Recommender systemsRecommender systems
Recommender systems
recsysfr
 
Recommendation @Deezer
Recommendation @DeezerRecommendation @Deezer
Recommendation @Deezer
recsysfr
 
Flexible recommender systems based on graphs
Flexible recommender systems based on graphsFlexible recommender systems based on graphs
Flexible recommender systems based on graphs
recsysfr
 
Using Neural Networks to predict user ratings
Using Neural Networks to predict user ratingsUsing Neural Networks to predict user ratings
Using Neural Networks to predict user ratings
recsysfr
 
Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?
Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?
Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?
recsysfr
 
Recommendation @ PriceMinister-Rakuten - Road to personalization
Recommendation @ PriceMinister-Rakuten - Road to personalizationRecommendation @ PriceMinister-Rakuten - Road to personalization
Recommendation @ PriceMinister-Rakuten - Road to personalization
recsysfr
 
Rakuten Institute of Technology Paris
Rakuten Institute of Technology ParisRakuten Institute of Technology Paris
Rakuten Institute of Technology Paris
recsysfr
 
Tailor-made personalization and recommendation - Sailendra
Tailor-made personalization and recommendation - SailendraTailor-made personalization and recommendation - Sailendra
Tailor-made personalization and recommendation - Sailendra
recsysfr
 
New tools from the bandit literature to improve A/B Testing
New tools from the bandit literature to improve A/B TestingNew tools from the bandit literature to improve A/B Testing
New tools from the bandit literature to improve A/B Testing
recsysfr
 
Story of the algorithms behind Deezer Flow
Story of the algorithms behind Deezer FlowStory of the algorithms behind Deezer Flow
Story of the algorithms behind Deezer Flow
recsysfr
 

More from recsysfr (15)

Multi Task DPP for Basket Completion by Romain WARLOP, Fifty Five
Multi Task DPP for Basket Completion by Romain WARLOP, Fifty FiveMulti Task DPP for Basket Completion by Romain WARLOP, Fifty Five
Multi Task DPP for Basket Completion by Romain WARLOP, Fifty Five
 
Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...
Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...
Building a recommender system with Annoy and Word2Vec by Cristian PEREZ, Kern...
 
An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...
An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...
An Homophily-based Approach for Fast Post Recommendation in Microblogging Sys...
 
Recommendations @ Rakuten Group
Recommendations @ Rakuten GroupRecommendations @ Rakuten Group
Recommendations @ Rakuten Group
 
Data-Driven Recommender Systems
Data-Driven Recommender SystemsData-Driven Recommender Systems
Data-Driven Recommender Systems
 
Recommender systems
Recommender systemsRecommender systems
Recommender systems
 
Recommendation @Deezer
Recommendation @DeezerRecommendation @Deezer
Recommendation @Deezer
 
Flexible recommender systems based on graphs
Flexible recommender systems based on graphsFlexible recommender systems based on graphs
Flexible recommender systems based on graphs
 
Using Neural Networks to predict user ratings
Using Neural Networks to predict user ratingsUsing Neural Networks to predict user ratings
Using Neural Networks to predict user ratings
 
Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?
Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?
Preference Elicitation in Mangaki: Is Your Taste Kinda Weird?
 
Recommendation @ PriceMinister-Rakuten - Road to personalization
Recommendation @ PriceMinister-Rakuten - Road to personalizationRecommendation @ PriceMinister-Rakuten - Road to personalization
Recommendation @ PriceMinister-Rakuten - Road to personalization
 
Rakuten Institute of Technology Paris
Rakuten Institute of Technology ParisRakuten Institute of Technology Paris
Rakuten Institute of Technology Paris
 
Tailor-made personalization and recommendation - Sailendra
Tailor-made personalization and recommendation - SailendraTailor-made personalization and recommendation - Sailendra
Tailor-made personalization and recommendation - Sailendra
 
New tools from the bandit literature to improve A/B Testing
New tools from the bandit literature to improve A/B TestingNew tools from the bandit literature to improve A/B Testing
New tools from the bandit literature to improve A/B Testing
 
Story of the algorithms behind Deezer Flow
Story of the algorithms behind Deezer FlowStory of the algorithms behind Deezer Flow
Story of the algorithms behind Deezer Flow
 

Recently uploaded

History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shopHistory+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
laozhuseo02
 
1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...
JeyaPerumal1
 
Internet-Security-Safeguarding-Your-Digital-World (1).pptx
Internet-Security-Safeguarding-Your-Digital-World (1).pptxInternet-Security-Safeguarding-Your-Digital-World (1).pptx
Internet-Security-Safeguarding-Your-Digital-World (1).pptx
VivekSinghShekhawat2
 
test test test test testtest test testtest test testtest test testtest test ...
test test  test test testtest test testtest test testtest test testtest test ...test test  test test testtest test testtest test testtest test testtest test ...
test test test test testtest test testtest test testtest test testtest test ...
Arif0071
 
BASIC C++ lecture NOTE C++ lecture 3.pptx
BASIC C++ lecture NOTE C++ lecture 3.pptxBASIC C++ lecture NOTE C++ lecture 3.pptx
BASIC C++ lecture NOTE C++ lecture 3.pptx
natyesu
 
This 7-second Brain Wave Ritual Attracts Money To You.!
This 7-second Brain Wave Ritual Attracts Money To You.!This 7-second Brain Wave Ritual Attracts Money To You.!
This 7-second Brain Wave Ritual Attracts Money To You.!
nirahealhty
 
The+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptxThe+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptx
laozhuseo02
 
How to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptxHow to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptx
Gal Baras
 
guildmasters guide to ravnica Dungeons & Dragons 5...
guildmasters guide to ravnica Dungeons & Dragons 5...guildmasters guide to ravnica Dungeons & Dragons 5...
guildmasters guide to ravnica Dungeons & Dragons 5...
Rogerio Filho
 
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesMulti-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Sanjeev Rampal
 
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
3ipehhoa
 
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
eutxy
 
Latest trends in computer networking.pptx
Latest trends in computer networking.pptxLatest trends in computer networking.pptx
Latest trends in computer networking.pptx
JungkooksNonexistent
 
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
3ipehhoa
 
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
APNIC
 
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
keoku
 
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
3ipehhoa
 
JAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
JAVIER LASA-EXPERIENCIA digital 1986-2024.pdfJAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
JAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
Javier Lasa
 
Comptia N+ Standard Networking lesson guide
Comptia N+ Standard Networking lesson guideComptia N+ Standard Networking lesson guide
Comptia N+ Standard Networking lesson guide
GTProductions1
 
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptxBridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
Brad Spiegel Macon GA
 

Recently uploaded (20)

History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shopHistory+of+E-commerce+Development+in+China-www.cfye-commerce.shop
History+of+E-commerce+Development+in+China-www.cfye-commerce.shop
 
1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...1.Wireless Communication System_Wireless communication is a broad term that i...
1.Wireless Communication System_Wireless communication is a broad term that i...
 
Internet-Security-Safeguarding-Your-Digital-World (1).pptx
Internet-Security-Safeguarding-Your-Digital-World (1).pptxInternet-Security-Safeguarding-Your-Digital-World (1).pptx
Internet-Security-Safeguarding-Your-Digital-World (1).pptx
 
test test test test testtest test testtest test testtest test testtest test ...
test test  test test testtest test testtest test testtest test testtest test ...test test  test test testtest test testtest test testtest test testtest test ...
test test test test testtest test testtest test testtest test testtest test ...
 
BASIC C++ lecture NOTE C++ lecture 3.pptx
BASIC C++ lecture NOTE C++ lecture 3.pptxBASIC C++ lecture NOTE C++ lecture 3.pptx
BASIC C++ lecture NOTE C++ lecture 3.pptx
 
This 7-second Brain Wave Ritual Attracts Money To You.!
This 7-second Brain Wave Ritual Attracts Money To You.!This 7-second Brain Wave Ritual Attracts Money To You.!
This 7-second Brain Wave Ritual Attracts Money To You.!
 
The+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptxThe+Prospects+of+E-Commerce+in+China.pptx
The+Prospects+of+E-Commerce+in+China.pptx
 
How to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptxHow to Use Contact Form 7 Like a Pro.pptx
How to Use Contact Form 7 Like a Pro.pptx
 
guildmasters guide to ravnica Dungeons & Dragons 5...
guildmasters guide to ravnica Dungeons & Dragons 5...guildmasters guide to ravnica Dungeons & Dragons 5...
guildmasters guide to ravnica Dungeons & Dragons 5...
 
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesMulti-cluster Kubernetes Networking- Patterns, Projects and Guidelines
Multi-cluster Kubernetes Networking- Patterns, Projects and Guidelines
 
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
1比1复刻(bath毕业证书)英国巴斯大学毕业证学位证原版一模一样
 
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
一比一原版(LBS毕业证)伦敦商学院毕业证成绩单专业办理
 
Latest trends in computer networking.pptx
Latest trends in computer networking.pptxLatest trends in computer networking.pptx
Latest trends in computer networking.pptx
 
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
原版仿制(uob毕业证书)英国伯明翰大学毕业证本科学历证书原版一模一样
 
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024
 
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
一比一原版(SLU毕业证)圣路易斯大学毕业证成绩单专业办理
 
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
急速办(bedfordhire毕业证书)英国贝德福特大学毕业证成绩单原版一模一样
 
JAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
JAVIER LASA-EXPERIENCIA digital 1986-2024.pdfJAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
JAVIER LASA-EXPERIENCIA digital 1986-2024.pdf
 
Comptia N+ Standard Networking lesson guide
Comptia N+ Standard Networking lesson guideComptia N+ Standard Networking lesson guide
Comptia N+ Standard Networking lesson guide
 
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptxBridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptx
 

RecsysFR: Criteo presentation

  • 1. RecSys FR: 4th Session 6th October 2016 Etienne Sanson – manager R&D Engine
  • 3. 3 | Copyright © 2016 Criteo Our mission TARGET THE RIGHT USER AT THE RIGHT TIME WITH THE RIGHT MESSAGE
  • 4. 4 | Copyright © 2016 Criteo Key Figures 16 000 PUBLISHERS 90% RETENTION RATE2 +130 COUNTRIES LISTED ON THE NASDAQ SINCE OCTOBER 2013 R&D REPRESENTS 21% OF THE WORKFORCE 2000 EMPLOYEES 21 BILLIONS $3 11 000 ADVERTISERS 1.19 bn€1 31 OFFICES 1: REVENUE IN 2015 2: ANNUAL RATE 2015 3: $ OF TURNOVER GENERATED TO OUR CLIENTS - TURNOVER POST-CLICK WW FROM JANUARY TO DECEMBER 2015
  • 5. 5 | Copyright © 2016 Criteo Revenue Growth 2009 2010 2011 2012 2013 22M$ 86M$ 199M$ 349M$ 589M$ 988M$ 2014 1,3MM$ 2015
  • 6. 6 | Copyright © 2016 Criteo GENERAL CONCEPT Users visit an advertiser’s website 1 Criteo identifies the users (via cookies) 2 Users leave the advertiser’s website & browse publisher on the Internet 3 Criteo identifies users on these pages (via cookie) 4 Criteo displays an advertising banner, personalized for each user 5 Click through directly to the advertiser’s page 6 @ Retargeting principles
  • 7. 7 | Copyright © 2016 Criteo Infrastructure Key Figures Sunnyvale 2 PoP 500 kVA 1 559 Servers New York 2 PoP 930 kVA 2 625 Servers Hong Kong 2 PoP 472 kVA 2185 Servers Paris 4 Pop 1 800 kVA 3 625 Servers Amsterdam 2 PoP +2 500 kVA 3 609 Servers Tokyo 2 PoP 455 kVA 2 564 Servers Shanghai 1 PoP 200 kVA 931 Servers World Wide 15 PoP 6,8 MVA 17 098 Servers > 55Gbps + 2.5M req/s Hosting Global Partners :
  • 9. 9 | Copyright © 2016 Criteo Our challenges – Product recommendation • Select the best ~10 products to show to a user • >1B users • Product catalog contains ~1M items, up to 1B • Time constraints: 20ms • Combination of offline/online processing steps • CF • Product embeddings (word2vec -> prod2vec) • CNN for image features What products should we recommend?
  • 10. 10 | Copyright © 2016 Criteo Our challenges - Bidding How much should we bid for this display? What is the best campaign to display? My company BUY! BUY! BUY! BUY! • Select the best campaign to display and evaluate its value in a few ms • Large scale regression models • >1B daily displays (but few positive examples!) • >1M parameters • Distributed optimization (SGD, L-BFGS) • Feature Engineering • Transfer learning, FFM, Policy learning • Marketplace, game theory, auction theory
  • 11. 11 | Copyright © 2016 Criteo Our challenges – X-device • Build a huge graph (Billions of nodes/edges): • Nodes = devices • Edge = the 2 devices belong to the same user • How to connect 2 devices? • How to know the ground truth? • How to keep it stable? • At scale & taking care about privacy Who is the user behind the device?
  • 12. 12 | Copyright © 2016 Criteo Our challenges – Testing • We test everything! • Offline tests / AB Tests • Infrastructure to perform large-scale tests • >100K offline tests / year • >1K AB Tests / year • Dedicated teams • Technical / Business Metrics • Randomization • Counterfactual evaluation

Editor's Notes

  1. How to connect? We have some data that allows for deterministic match, but we also have to build a probabilistic match
  2. Hashing trick, one-hot encoding, distributed optimization Tradeoff between freshness of model and historical data Irma
  3. How to connect? We have some data that allows for deterministic match, but we also have to build a probabilistic match
  4. trade-off fast-cheap / expensive-truth