This document discusses Restricted Boltzmann Machines (RBMs), which are unsupervised deep learning algorithms that can learn probability distributions over inputs. RBMs have two layers of nodes - visible units that receive input data and hidden units that learn representations of the input. They are trained to reconstruct their input by adjusting weights between nodes through forward and backward passes. RBMs can be stacked to form Deep Belief Networks, where the hidden units of one RBM become the visible units of the next. This allows higher layers to learn more abstract representations useful for tasks like classification. Training involves adjusting weights to minimize reconstruction error between original and reconstructed inputs.

1. (Restricted Boltzmann
Machines )
by
Hussein Ali
&
Ahmed majeed

2.  Outlining
 Introduction
 RBM characteristics
 Mechanism of RBM
 Deep Believe Learning
 Mechanism of Training

3. (Restricted Boltzmann
Machines )
 It is named after the Boltzmann
distribution in statistical mechanics,
which is used in their sampling
function.
 It has been highly promoted by Jeffrey
Hinton, Terry Sejnowski, and Yan
Lacon in the cognitive science
communities and in machine learning.

4. Introduction
 RBMs : They are random neural
networks that can learn from a
probability distribution over a set of
inputs.
 This deep learning algorithm is used
for dimension reduction, classification,
regression, collaborative filtering,
feature learning, and subject
modeling.
 Type of RBM is:

5. RBM characteristics
 Boltzmann Machines is an unsupervised DL
model in which every node is connected to
every other node.
 Boltzmann Machine is not a deterministic DL
model but a stochastic or generative DL model.
 Consists of tow layers:
◦ (Visible units)
 The visual units are those that receive information
from the 'environment‘
 those nodes which we can and do measure.
◦ (Hidden units)
◦ those nodes which we cannot or do not
measure.

6. RBM characteristics
 One interesting aspect of the RBM's
approach is that the data does not need to
be labeled. This is very important in the
real world, such as images, videos,
sounds, and sensor data. Instead of
having people manually label the data and
introduce errors, the RBM automatically
sorts the data by properly adjusting the
weights and biases.
 The RBM can extract the important
features and reconstruct the inputs.

7. Why it’s Restrction
 In a full Boltzmann machine, each node
is connected to every other node and
hence the connections
grow exponentially. This is the reason
we use RBMs.
 The restrictions in the node connections
in RBMs are as follows –
◦ Hidden nodes cannot be connected to one
another.
◦ Visible nodes also cannot be connected to
one another.

9. Mechanism of RBM
 In the forward path, the RBM takes the
input and encodes it into a set of
numbers that encode the input.
 In the backtrack, the RBM takes these
numbers and encodes them again to
form the reconstructed input.
 Through repeated forward and backward
passes (training), the RBM is trained to
reconstruct the input data.
 In the visiblel layer, the reconstructed
data is compared with the original input
to determine the quality of the result

10.  Using some randomly assigned initial weights, RBM calculates
the hidden nodes, which in turn use the same weights to
reconstruct the input nodes.
 Each hidden node is constructed from all the visible nodes and
each visible node is reconstructed from all the hidden node and
hence, the input is different from the reconstructed input, though
the weights are the same.
 The process continues until the reconstructed input matches the
previous input.
 The process is said to be converged at this stage.
RBM adjusts its weights by this
method

11. Deep Believe Learning (DBL)
 The algorithm has feature extraction
and input reconstruction. But how does
it help us solve the problem of fading
color gamut?
 Jeff designed the BDL and was
subsequently assigned to Image
Recognition at Google, where it is now
a large-scale project under
development.
 The training method is what
distinguishes this algorithm.

12. Mechanism of training
 The first RBM is trained to reconstruct its input
as accurately as possible
 The hidden layer of the first RBM is treated as
a visible layer of the second layer, and the
second RBM is trained using the output of the
first RBM.
 This process is repeated until every layer in
the network has been trained
 In other types of neural networks, early layers
detect simple patterns and later layers
recombine them. As in the face recognition
example, the first layers detect edges in the
image, and the observing layers use these

13. Mechanism of training
 To finish the training, we need to
introduce pattern labels and adjust the
network using supervised learning. To do
this we need a very small set of
categorized samples.
 The change in weights and biases is so
small that it results in a slight change in
the network's perception of the patterns.
 Labeled data is a small set relative to the
original data set, which is useful in the
real world.

14. Thank you