Slides from Berlin buzzwords 2015: Behavioral retargeting consists of displaying online advertisements that are personalized according to each user’s browsing history. At this point, the selection of the products to display in the banner needs to be fast and accurate. At Criteo, we built a recommender system which is able to choose a dozen of relevant products from over two billion products in a few milliseconds. In this talk, we will expose the problems we faced whilst building this system and how we solved them thanks to a mix of online and offline computations.

1. Copyright © 2015 Criteo
Recommendation at scale
Simon Dollé
Berlin Buzzwords, June 2nd, 2015

2. Copyright © 2015 Criteo
2

3. Copyright © 2015 Criteo
We buy
• Inventory ! (ad spaces)
• Billions of times a day
• All over the Internet
• For 95% of the population
Where is the need for tech ?
We sell
• Clicks !
• (that convert)
• (that convert a lot)
We take the risk
You pay only for what you get

4. Copyright © 2015 Criteo
4
Physical infrastructure
6 Data centers on 3 continents operated and conceived in-house
~ 12000 servers, largest Hadoop cluster in Europe
More than 35 PB of storage Big Data
Traffic
800 k HTTP requests / sec (peak activity)
29000 impressions /sec (peak activity)
Less than 10 ms to process an RTB request

5. Copyright © 2015 Criteo
Data Sources
Ad display dataUser behavior dataCatalog data

6. Copyright © 2015 Criteo

7. Copyright © 2015 Criteo

8. Copyright © 2015 Criteo

9. Copyright © 2015 Criteo

10. Copyright © 2015 Criteo

11. Copyright © 2015 Criteo

12. Copyright © 2015 Criteo

13. Copyright © 2015 Criteo
Offline
• Similarities computed on browsing data
• Based on coevents
• Computed on Hadoop cluster
• Map reduce jobs, pig
• Takes around 12 hours
• Tradeoff performance, speed

14. Copyright © 2015 Criteo

15. Copyright © 2015 Criteo
0.04 0.02 0.02 0.05

16. Copyright © 2015 Criteo
0.04 0.020.020.05

17. Copyright © 2015 Criteo
0.04 0.020.020.05

18. Copyright © 2015 Criteo
ML model
• Logistic regression models
• Features
Product-specific User-specific User-product interactions Display-specific

19. Copyright © 2015 Criteo
Online optimizations
• Algorithmic
• Use simpler ML model
• Quickly discard candidates
• Technical
• Memcache + local cache
• Async I/O

20. Copyright © 2015 Criteo
Upcoming challenges
• Long(er)-term user profiles
• More and better product information (images, semantic, NLP)
• Instant-update of similarities
• (because batch computation is soooo last year)

21. Copyright © 2015 Criteo
Fancy a try ?
On your own:
With us !
http://labs.criteo.com/jobs/
• Our 1st public dataset is online: http://bit.ly/1vgw2XC
• 4GB display and click data, Kaggle challenge in 2014
• NEW : 1TB dataset released a few weeks ago: http://bit.ly/1PyH4Vq
• Hosted on Microsoft Azure, just waiting for you

22. Copyright © 2015 Criteo
Questions?

23. Copyright © 2015 Criteo
Thank you !
s.dolle@criteo.com