This document provides an overview of Theano tutorial part 2, including brief recaps of symbolic variables, functions, and computational graphs. It then summarizes various machine learning models like multivariate logistic regression, multilayer perceptrons, 1D and 2D convolution, max pooling, convolutional neural networks. It also mentions recurrent neural networks and the scan function in Theano for symbolic loops. References are provided for further reading on convolution networks and RNNs.

1. Theano tutorial
part 2
AACIMP 2015
Sergii Gavrylov

2. Overview
● Brief recap
● Multivariate logistic regression
● Multilayer perceptron
● Convolution
● Convolutional neural network
● scan
● Recurrent neural network

3. Brief recap
● Symbolic variables
● Functions
● Shared variables / updates
● Gradients
● Substitution

4. Computational graph
X
+
Z
Y
● Code generation
● Symbolic differentiation

5. Multivariate logistic
regression
0
1
2
3
softmax(x * W)

6. Multilayer perceptron
0
1
2
3
relu(x * W) softmax(x * W)

7. 1D Convolution

8. 1D Convolution

9. 1D Convolution

10. 1D Convolution

11. 2D Convolution
filter =

12. Max pooling
cs231n.github.io/convolutional-networks

13. ConvPoolLayer

14. Convolutional NN
cs231n.github.io/convolutional-networks
deeplearning.net/tutorial/lenet.html

15. Convolutional NN
deeplearning.net/tutorial/lenet.html

16. scan
(Symbolic loop in theano)

17. Recurrent neural network
www.iro.umontreal.ca/~bengioy/dlbook/rnn.html

18. “Vanilla” RNN
www.iro.umontreal.ca/~bengioy/dlbook/rnn.html

19. Conclusion
● Theano has a lot of useful building blocks (convolution, scan).
● Theano supports both cpu and gpu backends.