Recurrent neural networks (RNNs) are a type of artificial neural network that can process sequential data of varying lengths. Unlike traditional neural networks, RNNs maintain an internal state that allows them to exhibit dynamic temporal behavior. RNNs take the output from the previous step and feed it as input to the current step, making the network dependent on information from earlier steps. This makes RNNs well-suited for applications like text generation, machine translation, image captioning, and more. RNNs can remember information for long periods of time but are difficult to train due to issues like vanishing gradients.

1. RECURRENT NEURAL NETWORK
SYED ANNUS ALI SHAH F2016266257

2. RECURRENT NEURAL NETWORKS
 Recurrent Neural Network is the type of Artificial neural network which takes the output of the
previous step and feed as the input of current step. Unlike traditional Neural Networks the all the new
outputs in the network are dependent to the previously generated output. It is mostly used to generate
random sentences for example: to predict the next word of a sentence, the previous words are required
and there is a need to remember the previously generated words. And this issue is solved by the Hidden
layer of RNN. It will not be wrong to say that the Hidden state/layer is the most important feature of
RNN , which is used to memorize some info about sequence.

3. IMPORTANCE
 The Importance of Recurrent Neural Network is that for every instance you do not think from the
scratch again and again, You can use prior understanding or outputs to get the understanding of the
new instance. Which a Traditional ANN is unable to do.
 With RNN, Machine is able to think more likely as a Human.

4. APPLICATIONS
Image Captioning
Poetry Generator
Chatbot

5. WORKING
 The Working of RNN can be understood by the following example:
 Suppose there is a deeper network with one input layer, three hidden layers and one output layer. Then
like other neural networks, each hidden layer will have its own set of weights and biases, let’s say, for
hidden layer 1 the weights and biases are (w1, b1), (w2, b2) for second hidden layer and (w3, b3) for third
hidden layer. This means that each of these layers are independent of each other, i.e. they do not
memorize the previous outputs.

6. WORKING (CONT.)
Now the RNN will do the following:
 RNN converts the independent activations into dependent activations by providing the same weights and
biases to all the layers, thus reducing the complexity of increasing parameters and memorizing each
previous outputs by giving each output as input to the next hidden layer.
 Hence these three layers can be joined together such that the weights and bias of all the hidden layers is
the same, into a single recurrent layer.

7. FORMULAS
 To calculate Current State:
 ht = f(ht-1,xt)
 Where :
 ht -> current state
 ht-1 -> previous state
 xt -> input state

8. FORMULAS (CONT.)
 Formula for applying Activation function(tanh):
 Where
 Whh -> weight at recurrent neuron
 Wxh -> weight at input neuron

9. FORMULAS (CONT.)
 Formula for calculating output:
 Where
 Yt -> output
 Why -> weight at output layer

10. TRAINING
 Training through RNN
 A single time step of the input is provided to the network.
 Then calculate its current state using set of current input and the previous state.
 The current ht becomes ht-1 for the next time step.
 One can go as many time steps according to the problem and join the information from all the previous
states.
 Once all the time steps are completed the final current state is used to calculate the output.
 The output is then compared to the actual output i.e the target output and the error is generated.
 The error is then back-propagated to the network to update the weights and hence the network (RNN) is
trained.

11. ADVANTAGES AND DISADVANTAGES
 Advantages of Recurrent Neural Network
 An RNN remembers each and every information through time. It is useful in time series prediction only
because of the feature to remember previous inputs as well. This is called Long Short Term Memory.
 Recurrent neural network are even used with convolutional layers to extend the effective pixel
neighborhood.
 Disadvantages of Recurrent Neural Network
 Gradient vanishing and exploding problems.
 Training an RNN is a very difficult task.
 It cannot process very long sequences if using tanh or relu as an activation function.

12. REFRENCES.
 Medium
 https://medium.com/explore-artificial-intelligence/an-introduction-to-recurrent-neural-networks-72c97bf0912
 Wikipedia
 Https://en.wikipedia.org/wiki/Recurrent_neural_network
 Geeks for Geeks
 https://www.geeksforgeeks.org/introduction-to-recurrent-neural-network/

13. Any Questions ???