1.
RecSys FR: 4th Session
6th October 2016
Etienne Sanson – manager R&D Engine

2.
About Criteo
1

3.
3 | Copyright © 2016 Criteo
Our mission
TARGET THE
RIGHT USER
AT THE
RIGHT TIME
WITH THE RIGHT
MESSAGE

4.
4 | Copyright © 2016 Criteo
Key Figures
16 000
PUBLISHERS
90%
RETENTION RATE2
+130
COUNTRIES
LISTED ON THE
NASDAQ
SINCE
OCTOBER 2013
R&D REPRESENTS 21%
OF THE WORKFORCE
2000
EMPLOYEES
21
BILLIONS $3
11 000
ADVERTISERS
1.19 bn€1
31
OFFICES
1: REVENUE IN 2015
2: ANNUAL RATE 2015
3: $ OF TURNOVER GENERATED TO OUR CLIENTS - TURNOVER POST-CLICK WW FROM JANUARY TO DECEMBER
2015

5.
5 | Copyright © 2016 Criteo
Revenue Growth
2009 2010 2011 2012 2013
22M$
86M$
199M$
349M$
589M$
988M$
2014
1,3MM$
2015

6.
6 | Copyright © 2016 Criteo
GENERAL CONCEPT
Users visit an
advertiser’s website
1
Criteo identifies the users
(via cookies)
2
Users leave the advertiser’s website
& browse publisher on the Internet
3
Criteo identifies users on
these pages
(via cookie)
4
Criteo displays an advertising
banner, personalized for
each user
5
Click through directly
to the advertiser’s
page
6
@
Retargeting principles

7.
7 | Copyright © 2016 Criteo
Infrastructure Key Figures
Sunnyvale
2 PoP
500 kVA
1 559 Servers
New York
2 PoP
930 kVA
2 625 Servers
Hong Kong
2 PoP
472 kVA
2185 Servers
Paris
4 Pop
1 800 kVA
3 625 Servers
Amsterdam
2 PoP
+2 500 kVA
3 609 Servers
Tokyo
2 PoP
455 kVA
2 564 Servers
Shanghai
1 PoP
200 kVA
931 Servers
World Wide
15 PoP
6,8 MVA
17 098 Servers
> 55Gbps
+ 2.5M req/s
Hosting Global Partners :

8.
About ML@Criteo
2

9.
9 | Copyright © 2016 Criteo
Our challenges – Product recommendation
• Select the best ~10 products to show to a user
• >1B users
• Product catalog contains ~1M items, up to 1B
• Time constraints: 20ms
• Combination of offline/online processing steps
• CF
• Product embeddings (word2vec -> prod2vec)
• CNN for image features
What products should
we recommend?

10.
10 | Copyright © 2016 Criteo
Our challenges - Bidding
How much should we
bid for this display?
What is the best
campaign to display?
My company
BUY! BUY! BUY!
BUY!
• Select the best campaign to display and evaluate its
value in a few ms
• Large scale regression models
• >1B daily displays (but few positive examples!)
• >1M parameters
• Distributed optimization (SGD, L-BFGS)
• Feature Engineering
• Transfer learning, FFM, Policy learning
• Marketplace, game theory, auction theory

11.
11 | Copyright © 2016 Criteo
Our challenges – X-device
• Build a huge graph (Billions of nodes/edges):
• Nodes = devices
• Edge = the 2 devices belong to the same user
• How to connect 2 devices?
• How to know the ground truth?
• How to keep it stable?
• At scale & taking care about privacy
Who is the user
behind the device?

12.
12 | Copyright © 2016 Criteo
Our challenges – Testing
• We test everything!
• Offline tests / AB Tests
• Infrastructure to perform large-scale tests
• >100K offline tests / year
• >1K AB Tests / year
• Dedicated teams
• Technical / Business Metrics
• Randomization
• Counterfactual evaluation

13.
Thank you!
…and we’re hiring!