This document discusses recurrent neural networks and their ability to learn sequences through an internal memory component. It covers different recurrent architectures like RNNs, GRUs, and LSTMs. Recurrent nets can be used for applications involving sequences and prediction like generating text, forecasting, image captioning, and predictive maintenance in IoT. Their ability to model temporal data makes them well-suited for problems involving videos, sensors and predicting future events or states.

1. Recurrent Nets and
Sensors
Idosi Korea 2015 - ujava.org

2. Recurrent nets
Credit: Andrej
Karpathy

3. Recurrent Architecture
Credit:
http://blog.josephwilk.net/ima
ges/blog/2012/10/

4. Recurrent vs normal feed forward
Normal feed forward meant for basic tasks
such as classification
No memory component
Recurrent can be used to learn sequences
Different backpropagation

5. BPTT vs normal Backprop
Extra weight matrices represent a sort of
“memory component”
Learns to minimize sequences rather than just
one classification

6. Different Recurrent Architectures
Recurrent Nets (normal)
Gated recurrent units
Long Short term Memory
Attention models

7. Gated Recurrent Nets
http://image.slidesharec
dn.com/generalsequenc
elearningwithrecurrentn
euralnetworksfornextml-
150217161745-
conversion-
gate01/95/general-
sequence-learning-
using-recurrent-neural-
networks-19-
638.jpg?cb=143075083
8

8. LSTM
Credit:
http://blog.otoro.net/
wp-
content/uploads/sites
/2/2015/05/LSTM.pn
g

9. What can you do with them?
http://karpathy.github.io/2015/05/21/rnn-
effectiveness/
Generate shakespeare
Compilable code
Forecasting

10. Other applications
Caption Generation for images
Real time object tracking based on the output
of a conv net
Image segmentation

11. Internet of Things
Anomaly Detection
Predictive maintenance
Targeted advertising (sensors + video)

12. All about predicting the future
Recurrent net sequence learning allows for us
to predict anything with a time axis
Think about videos and forecasting movement
over time

13. Thanks!
help@skymind.io
@agibsonccc
https://deeplearning4j.org/