Recommender systems need to optimize a delicate balance between exploring new recommendations and exploiting informative ones. Like most fields in computer science, recommender systems have been widely affected by the recent advancements in Machine Learning, and more specifically Deep Learning. Up until this point recommender systems employed mostly bayesian prediction algorithms and utilized the inherent measures of uncertainty to optimize the crucial exploration/exploitation balance. While DNNs obtain state of the art results, they present specific challenges for recommender systems due to their lack of standardized uncertainty measures needed for most exploration strategies. In this talk we’ll cover how we overcome this challenge at Taboola and how we use uncertainty measures to our advantage when serving content recommendations billions of times a day.

1. Inbar Naor / Data Scientist, Taboola April 2018
Don't believe everything your
network tells you
Uncertainty in deep learning

2. Outline
• Why do we need uncertainty?
• Uncertainty Types
• Uncertainty in Recommendation Systems
• Bayesian Learning 101
• Data Uncertainty
• Model Uncertainty
DON'T BELIEVE EVERYTHING YOUR NETWORK TELLS YOU

3. Why Do We Need Uncertainty?
● High risk applications
● Out of data samples
● Improving the model:
○ mistakes with high confidence
○ true labels with low confidence

4. Capturing Model Blind spots is Crucial
UNCERTAINTY IN DEEP LEARNING

5. Why Do We Need Uncertainty?
● High risk applications
● Out of data samples
● Improving the model:
○ mistakes with high confidence
○ true labels with low confidence

6. SLIDE | 6

7. Why Do We Need Uncertainty?
● High risk applications
● Out of data samples
● Improving the model:
○ mistakes with high confidence
○ true labels with low confidence

8. Model Uncertainty
UNCERTAINTY IN DEEP LEARNING
More Data Please!

9. Data Uncertainty
UNCERTAINTY IN DEEP LEARNING
More Data Won’t Help – I want to
know how good my predictions are

10. Uncertainty in
Recommendation
Systems

11. Content Discovery
A DEEP LEARNING APPROACH TO DISCOVERY
Context
Metadata
User Historical Data
Rank N
recommendations by
CTR * CPC
~1M Possible
Recommendations Region-based
Location
Information

12. Machine Learning + Discovery = Hard
A DEEP LEARNING APPROACH TO DISCOVERY
Rank N
recommendations by
CTR * CPC
Implicit
Feedback
Data is Very
Sparse
No Human
Baseline
World is
Ever-changing
“Walmart cameras captured these awesome photos”“Walmart cameras captured these hilarious photos”
“15 rarely seen WW2 Photos Discovered”

13. Why Go Deep for Discovery?
• Cold start is a huge issue
• Many hard sub problems
– Language modeling
– Image classification
– User Profiling
• There are many complex relations
A DEEP LEARNING APPROACH TO DISCOVERY

14. Exploration/Exploitation in Recommender Systems
UNCERTAINTY IN DEEP LEARNING
Best Performing
Recommendations
Add new Information
Fight selection biasSearch for new stars
Exploration at random doesn’t work

15. Bayesian Learning
● Probability is a measure of belief
● Prior distribution represents
our belief about the world
● Likelihood
● Posterior
Introduction

16. Capturing Data Uncertainty
Know what you don’t know

17. Likelihood as loss
CAPTURING DATA UNCERTAINTY
h1
h2
… hn
LSTM LSTM LSTM LSTM
WordWordWord …
y

18. Likelihood as loss
CAPTURING DATA UNCERTAINTY
(*) In classification we can
assume a binomial
distribution with Beta prior
h1
h2
… hn
LSTM LSTM LSTM LSTM
WordWordWord …
Mu Std

19. Mixture Density Network
CAPTURING DATA UNCERTAINTY
Christopher M. Bishop, Mixture density networks (1994)

20. Capturing Data Uncertainty
CAPTURING DATA UNCERTAINTY

21. Data Uncertainty and Training Error
CAPTURING DATA UNCERTAINTY

22. Data Uncertainty and OOV
CAPTURING DATA UNCERTAINTY
h1
h2
… hn
LSTM LSTM LSTM LSTM
Word
OOV
Word1
Word …
Mu Std
OOV
Word2
Probably
higher
variance

23. Data Uncertainty and OOV
CAPTURING DATA UNCERTAINTY

24. Capturing Model
Uncertainty

25. Bayesian Learning
● Probability is a measure of belief
● Prior distribution represents
our belief about the world
● Likelihood
● Posterior
Introduction

26. Neural Network - A Probabilistic Perspective
Introduction
Neural Network is a probabilistic model
We learn W using MLE:
Adding regularization using MAP:

27. Bayesian Neural Networks
Model Uncertainty
Weights uncertainty
Prior:
Posterior:
Prediction:
is often intractable

28. Bayesian Posterior Inference
Model Uncertainty
Variational Inference Sampling Methods
High bias - low variance High variance - low bias

29. Dropouts – a regularization technique
CAPTURING MODEL UNCERTAINTY

30. Dropout variational inference as Bayesian Approximation
• Monte-Carlo Dropouts as Bayesian Approximation
• A little bit like bagging
CAPTURING MODEL UNCERTAINTY
gal & ghahramani, Bayesian convolutional neural networks with bernoulli approximate variational inference (2016)

31. Uncertainty as a function of amount of data
CAPTURING MODEL UNCERTAINTY
Image: Yarin Gal, “what my deep model doesn’t know”

32. Uncertainty as a function of amount of data
CAPTURING MODEL UNCERTAINTY

33. Uncertainty as a function of amount of data
CAPTURING MODEL UNCERTAINTY

34. Uncertainty in exploration
● Upper Confidence Bound - adding sigma to the score
● Thompson Sampling - sample from the distribution

35. Summary
• Two types of uncertainty: model and data
• Mixture Density Networks
• Captures the true variance of your prediction
• Monte-Carlo Dropouts Variational Inference
• Sheds light on where the model lacks data
• Why not both?
• Also interesting: uncertainty due to measurement noise
DON'T BELIEVE EVERYTHING YOUR NETWORK TELLS YOU
Model Uncertainty Data Uncertainty

36. Thank You
Feel free to contact with questions!
Inbar Naor - inbar.naor1@gmail.com
Taboola Engineering Blog
Unsupervised - A Podcast about Data Science in Israel

37. Bibliography
● What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? Kendall & Gal, 2017
● Dropout Inference In Bayesian Neural Networks with Alpha-Divergence. Li & Gal, ICML 2017
● On Modern Deep Learning and Variational Inference. Gal & Ghahramani, Advances in Approximate Bayesian Inference
workshop, NIPS 2015
● Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference. Gal & Ghahramani, ICLR 2016
● Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. Gal & Ghahramani, ICML 2016
● Dropout as a Bayesian Approximation: Insights and Applications. Gal & Ghahramani, Deep Learning Workshop, ICML
2015
● Weight Uncertainty in Neural Networks . Blundell, Cornebise, Kavakcuoglu, Wierstra 2015
● Training Deep Learning Neural Networks Based on Unreliable Labels. Bekker & Goldberger, ICASSP 2016
● Keeping Neural Networks Simple by Minimizing the description length of weights. Hinton & van Camp, Proceedings of
COLT- 1993
● Practical Variational Inference for Neural Networks. Graves, NIPS 2011
● Learning Weight Uncertainty with Stochastic Gradient MCMC for Shape Classification. Chunyuan et al. CVPR 2016