The document discusses Restricted Boltzmann Machines (RBMs), highlighting their role as energy-based neural networks that function as generative models for probability distributions. It contrasts RBMs with autoencoders and covers their structure, training processes like Gibbs sampling and contrastive divergence, and applications such as collaborative filtering. Additionally, it explains the use of latent factors in data prediction with RBM models.

2. Restricted Boltzmann Machine

3. Agenda
❖ History of RBM
❖ Difference between RBM & Autoencoders
❖ Introduction to RBMs
❖ Energy Based Model & Probabilistic Model
❖ Training of RBMs
❖ Example: Collaborative Filtering

4. Restricted Boltzmann Machine

5. Restricted Boltzmann Machine

6. Restricted Boltzmann Machines (RBMs) are neural
networks that belong to Energy Based Models. These
are parameterized generative models representing a
probability distribution.
Introduction to RBMs
edureka!

7. Restricted Boltzmann MachineAutoencoders
Autoencoders vs RBM

8. Visible Layer Hidden Layer
Layers of RBMs
edureka!

9. Visible Layer Hidden Layer
Layers of RBMs

10. activation f((weight w * input x) + bias b ) = output a

11. Working of RBMs
X – input
W – weight
a – activation function
w
Visible Layer Hidden Layer
x
+ b = a
x
x
x
+ b = a
+ b = a
edureka!

12. Reconstruction of RBM
edureka!

13. RBM: Energy Based Model

14. RBM: Energy Based Model
Visible Layer
Hidden Layer

15. RBM: Probabilistic Model

16. RBM: Probabilistic Model
Partition Function

17. RBM: Probabilistic Model
Partition Function

18. RBM: Probabilistic Model

19. RBM:
Probabilistic
Model
MNIST’s handwritten numerals
Headshots found in Labelled Faces
edureka!

20. RBM Training

21. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
1
2
RBM Training
Gibbs Sampling
Contrastive Divergence

22. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
1
2
RBM Training
Gibbs Sampling
Contrastive Divergence

23. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
1
2
RBM Training
Gibbs Sampling
Contrastive Divergence

24. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
1
2
RBM Training
Gibbs Sampling
Contrastive Divergence

25. RBM: Training to Prediction
Step 1
Step 3
Step 4Train the network on
the data of all users
During inference time
take the training data
of a specific user
Use this data to
obtain the activations
of hidden neurons
Use the hidden neuron
values to get the activations
of input neurons
The new values of input
neurons show the rating
the user would give
Step 2
Step 5
edureka!

26. RBM: Example

27. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
1
2
RBM: Collaborative Filtering
Recognizing Latent Factors in
The Data
Using Latent Factors for
Prediction
0
P = 0.838
0
P = 0.354
1
P = 0.838
1 0 0 1 0 -1
Drama Fantasy Science Fiction
Hidden nodes
Visible nodes
Lord of the Rings The Matrix Fight Club Harry Potter Titanic The Hobbit

28. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
1
2
Recognizing Latent Factors in
The Data
Using Latent Factors for
Prediction
00 1
1
P=0.985
0
P=0.171
0
P=0.290
1
P=0.857
0
P=0.214
-1
P=0.910
Drama Fantasy Science Fiction
Hidden nodes
Visible nodes
Lord of the Rings The Matrix Fight Club Harry Potter Titanic The Hobbit
RBM: Collaborative Filtering