This document discusses neural networks and deep learning. It provides an agenda that covers what neural networks are, how to train a neural network, unsupervised feature learning, building a handwritten digits classifier, and tips and tricks. It describes how neural networks are inspired by the human brain and are best suited for human-like tasks such as speech and object recognition. It also outlines the processes of feedforwarding, backpropagation, autoencoders, and stacked autoencoders. Recommended links for further learning are also included.

1. Neural Networks and
Deep Learning
Anastasiia Kornilova

2. Agenda
● What is Neural Networks and Deep Learning
● How to train Neural Network
● Unsupervised Feature Learning
● Building Handwritten Digits Classifier
● Tips and Tricks

3. NN and Deep Learning
Inspired by human brain
Best suitable for human brain tasks:
● speech recognition
● object recognition

4. How brain works?

6. How neural networks work?

7. Activation functions:

9. Feedforwarding

10. Error function

11. Stochastic Gradient Descent

12. Backpropagation

13. Backpopagation
1. Perform a feedforward pass, computing the activations for layers L2, L3, and so on up to the output layer
.
2. For each output unit i in layer nl (the output layer), set
3. For
For each node i in layer l, set
1. Compute the desired partial derivatives, which are given as:

14. Autoencoder

15. Stacked autoencoder

16. NN for digits recognition
1) Data visualization
2) Train autoencoder
3) Build classifier
4 ) Enjoy!

17. Optimization tips
● Linear algebra libraries
● Minibatch
● More optimizations methods (activation
functions, dropout, dropconnect,
automatization learning rates)
● GPU computing

18. Links
● Machine Learning by Andrew Ng
● Unsupervised Feature Learning and Deep
Learning tutorial
● Neural Networks for Machine Learning by
Geoffrey Hinton
● Neural Networks and Deep Learning - free
online book
● Pylearn2 - framework for deep learning