Recommended
More Related Content
What's hot
What's hot (20)
Similar to Deep belief network.pptx
Similar to Deep belief network.pptx (20)
Recently uploaded
Recently uploaded (20)
Deep belief network.pptx
- 1. Deep Belief Network Sushil Acharya s356528
- 2. (Restricted Boltzmann machine ) RBM • RBMs are shallow neural network nets. • Unsupervised. • In the forward pass (V H), extract meaningful features • Reconstruct the input during backward pass (H V) . • To train RBM, several forward and backward pass are required. • At the visible layer, the reconstruction is compared against the original input to determine the quality of the result.
- 3. RBM continued…. • A trained RBM can reveal which features are the most important ones when detecting patterns. • RBMs use a stochastic approach to learning the underlying structure of data, whereas autoencoders, for example, use a deterministic approach. • RBM automatically sorts through the data, and by properly adjusting the weights and biases, an RBM is able to extract the important features and reconstruct
- 4. DBN • A kind of deep learning network formed by stacking several RBMs. • It is a probabilistic generative model. • There is connection between the layers and but not between units within each layer. • DBN can be divided into two parts. Multiple layers of (RBMs) and feed forward backpropagation network.
- 5. Purpose of DBN Problem with backpropagation • It requires labeled training data. • It is very slow in networks with multiple hidden layers. • It can get stuck in poor local optima. How DBN solve this problem? • By using an extra step called pre-training. • Pre training is done before backpropagation.
- 6. How to train DBN
- 7. How to Train DBN
- 8. Use of DBN Although they are rarely used nowadays…. • Deep-belief networks are used to recognize, cluster and generate images, video sequences and motion-capture data. • Used to learn feature representations and several related tasks.
- 9. Questions ??
- 10. Thank you very much
Editor's Notes
- To understand the Deep belief network properly, we should have some understanding of Restricted Boltzmann machine or RBM. RBMs are shallow neural networks that learn to reconstruct data by themselves in an unsupervised fashion. This is considered shallow because they have only two layers called visible layer (or input layer) and hidden layer. It is unsupervised, meaning, it can extract meaningful information from unlabeled data. Training of such network is done in two steps. They are forward pass and backward pass. In the forward pass, RBM can extract meaningful features from input and translate them to numbers. These numbers can be translated back to the visible layer to reconstruct the input during backward pass. To train RBM completely, several forward and backward pass are required. Finally, to determine the quality of the result, the reconstruction is compared against the original input at the visible layer or input layer.
- As I already explain, RBM is unsupervised, and it deals with unlabeled data. So, it has capacity to extract most important features from the input data. The concept of RBM is similar to autoencoder. One of the difference is that RBM uses probabilistic approach because their learning is unsupervised and autoencoders use deterministic approach because their learning is supervised. Of course, there are so many differences between autoencoders and RBM, I am not going to explain everything now. And like other neural networks, to extract the important features from the data, value of weight and bias should be adjusted properly. In case of RBM, it automatically adjusts weight and bias based on input data.
- Now we are good to go through Deep belief network or DBN. A deep belief network is a kind of deep learning network formed by stacking several RBMs. We can see in the picture, by stacking several RBMs, a DBN has been constructed. This is the reason why we should know RBM before digging in to DBN. DBN is a probabilistic generative model in which there is connection between the layers but not between units within each layer. So, each units of layers learns independently, If the units are connected, then learning of one neuron could be affected by another. But here, units are not connected with each other, so their learning process is independent. and this is the strongest point of DBN. DBN can be divided into two parts. The first one are multiple layers of restricted Boltzmann machines (RBMs). The second one is feed forward backpropagation network.
- One problem with traditional multilayer perceptron (MLP) or artificial neural network (ANN) is that the backpropagation requires labelled data and it often lean to local minima, and there are problem with vanishing gradient and exploding gradients during gradient descent process. Which restrict the network to update the weight properly and consequently the network converge to local minima not the global minima. Deep belief network solve this problem by using an extra step called pre-training. And pre training is done before backpropagation which reduces the error rate and decrease the probability to converge into local minima.
- To train DBN, contrastive divergence-based method is used. It has two phases, positive and negative phases. In the positive phase, we update the weights of hidden unit and in the negative phase, we update the weights of the visible units. As I already explain, the training of RBM is probabilistic. Probability can be calculated by using sigmoid of the function. A and B are Bias associated with visible layer and hidden layer.
- And with the help of positive and negative probabilities, we can finally update the weight of the edge. This process continuous until the desired value of weight is achieved.
- DBN has won the Netflix challenges for movie recommender system . Meaning it is already evident that they are effective neural networks. In spite of this facts, DBN are not that popular nowadays. They are rarely used. Usually, DBN can be used to recognize, cluster and generate images, video sequences and motion-capture data. They are also used to learn feature representations and several related tasks.