The document discusses teaching recurrent neural networks (RNNs) using TensorFlow, covering theory and practical examples such as learning sine waves, addition, and handwriting. It highlights the architecture for implementing RNNs, including initializing RNN cells and defining hyperparameters. Additionally, it mentions the use of R and Shiny for creating a visual interface to interact with TensorFlow models.

1. Teaching
Recurrent Neural Networks
using Tensorflow
Rajiv Shah
twitter: rajcs4
rshah@pobox.com
www.rajivshah.com

4. Teaching Recurrent Neural Networks
using Tensorflow
• How they are used
• Theory - http://playground.tensorflow.org/
• Learn a sine wave - http://tinyurl.com/raj-sine
• Learn to add - http://tinyurl.com/raj-add
• Learn handwriting - https://github.com/hardmaru/
write-rnn-tensorflow

10. Theory

12. Recurrent Neural Networks:
Theory

13. Unrolling an RNN

15. RNN

16. LSTM

18. Examples of RNNs

19. Little more theory . . . but fun

20. Tensorflow Playground
• http://playground.tensorflow.org/

23. Sine Wave

25. High Level Architecture
• Function to get batches
• Determined type of RNN cells and initialize them
• Decide on cost function and hyper parameters
(dropout, learning rate, optimizer, batch size, sequence
length)
• Run session

26. Go to the python notebook

27. Friendly Tensorflow - Using
R / Shiny
• Wrapped calls to tensorflow using the rPython
language
• Shiny provided a nice visual interface and the ability to
quickly change different sorts of inputs / models
• https://github.com/rajshah4/tensorflow_shiny

28. Adding

29. Teaching Kids to Add

31. Teaching to Add
(array([[7],
[4],
[1],
[8],
[0],
[3],
[1],
[5],
[6],
[9],
[2],
[0],
[0],
[0],
[0]]), 46.0)

32. Go to the python notebook

33. Learning Handwriting

35. Links
• http://www.cs.toronto.edu/~graves/handwriting.html
• https://github.com/hardmaru/write-rnn-tensorflow

36. Take Aways
• RNNs are sequential learners and state of the art
• Understand why deep learning is powerful
• See how a RNN learns to model a sine wave, add,
learn handwriting . . .

37. Teaching
Recurrent Neural Networks
using Tensorflow
Rajiv Shah
twitter: rajcs4
rshah@pobox.com
www.rajivshah.com