1. Copyright © 2015 Criteo
Product Recommendation Beyond Collaborative
Filtering - welcome to the Twilight Zone!
Olivier Koch, Machine Learning Lead
o.koch@criteo.com
@olivkoch
RecSys Meetup London - Sept 20, 2017

2. Copyright © 2015 Criteo
Outline
• What we do
• Lessons learned from building collaborative filtering at scale
• Now what?

3. Copyright © 2015 Criteo
What we do
• Buy ad space on publishers’ websites.
• Build banners showing products that users will like / want to buy.
• Get paid if users click / buy the product.

4. 3 billion ads/day
5 billion products
100 ms

5. 15 data centers
24 000 servers
2000-node hadoop cluster

6. Copyright © 2015 Criteo

7. Copyright © 2015 Criteo
Machine learning is at the core of Criteo
• How much should we bid?
• What look & feel should we use?
• Which products should we recommend?

8. Copyright © 2015 Criteo
Recommending products from user timelines
Clicks / Sales / Views / Baskets events (on advertisers’ websites)
Click / no-click / attributed sales events (on publishers’ websites)
User
= ?
… for billions of users and products

9. Bob saw orange shoes
Some candidate products
Historical
Similar
Complementary
Most viewed

10. Lessons learned from building
a CF system at scale

11. Start simple (counters)
Expect a long tail
Randomize
Yes, your Spark jobs will break
Log everything (and more)
Revisit your evaluation metrics
Revisit your features
Check your attribution model
Beating CF is really hard

12. CF is great…
intuitive
simple to implement
scales relatively well
captures many implicit signals via wisdom of the crowd

13. But CF has limitations too…
does not scale that well actually (quadratic in user timelines)
does not capture temporal signals
does not solve cold start
does not address exploration in the long tail

14. Deep learning to the rescue
Fusing image, text and CF (Content2vec)
Specializing Joint Representations for the task of Product Recommendation, Thomas Nedelec,
Elena Smirnova, Flavian Vasile, RecSys 2017 DL Workshop, arXiv:1706.07625
Contextual RNNs
Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks, Elena
Smirnova, Flavian Vasile, RecSys 2017 DL Workshop, arXiv:1706.07684

15. Deep learning to the rescue
Hierarchical recurrent neural networks
Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks by
Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, Paolo Cremonesi, RecSys 2017
Compressed embeddings
Getting Deep Recommenders Fit: Bloom Embeddings for Sparse Binary Input/Output Networks
by Joan Serrà and Alexandros Karatzoglou, RecSys 2017
3D Convolutional Networks for Session-based Recommendation with Content Features, Trinh
Xuan Tuan and Tu Minh Phuong, RecSys 2017

16. Deep learning to the rescue
Can we build neural network architectures that will make our recommendations
more relevant?
Can we leverage temporal information and product metadata?
At scale.

17. Attribution and incrementality
The true objective of recommendation is to predict and show ads that cause new
(incremental) sales
Large-scale Validation of Counterfactual Learning Methods: A Test-Bed. Damien Lefortier,
Xiaotao Gu, Adith Swaminathan, Thorsten Joachims and Maarten de Rijke, arXiv:1612.00367,
NIPS What If 2016
http://research.criteo.com/dataset-release-evaluation-counterfactual-algorithms/

18. Copyright © 2014 Criteo
Join us!
We are reverse-engineering the brain
of 1B+ shoppers worldwide!
http://labs.criteo.com/rd-jobs/