Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Deep Learning Tutorial
Similar to Deep Learning Tutorial (20)
Recently uploaded
Recently uploaded (20)
Deep Learning Tutorial
- 2. ● What is deep learning (DL)? ● In short, DL is about works driven by super deep neural networks. Overview of Deep Learning
- 3. Overview of Deep LearningFus e Creat e Prisma Paints Chainer Understand
- 4. Meaning of Deep Learning ● What is the necessity of having such a giant (millions of parameters)? ● Solve non-well-defined problem ● Handle ambiguous features ● Perhaps, the giant is not so scary?
- 5. Meaning of Deep Learning ● Solve non-well-defined problem How to describe this mapping? 32 * 32
- 6. Meaning of Deep Learning ● Handle ambiguous features
- 7. Meaning of Deep Learning ● Handle ambiguous features
- 8. Meaning of Deep Learning ● Perhaps, the giant is not so scary? ● There are many excellent open source frameworks. ● Caffe/Tensorflow/MXnet/Torch <- My favorite ● End-to-end fashion doesn’t involve much stuff. ● The state-of-the-art model can be implemented in just 30 lines! ● Model can easily be generalized to other cases. ● Design once, deploy everywhere.
- 9. Prerequisites ● Math ● Calculus (derivate / maxima / maxium) ● Linear algebra ( basic vector / matrix operation) ● Coding ● Have taken any course about C/C++/Java ● Python basics
- 10. The First Step — Linear Regression Given data (x1, y1), (x2, y2), … (xn, yn), to find a line which minimize the overall distance Example: You now are going to rent a apartment and worrying about leasing a overpriced one. Therefore you write a web crawl to collect apartment information nearby. Linear regression can help you to find proper parameters to estimate apartment price.
- 11. Linear Regression Now write it in matrix form Data (x1, y1), (x2, y2), … (xn, yn)
- 13. Linear Regression Mathematical approach Code Monkeys’ approach WTF ? Repeat n times gradient = get_gradient(W) W = W - learning_rate * gradient
- 14. Linear Regression Gradient Descent Repeat n times gradient = get_gradient(W) W = W - learning_rate * gradient Why it works?
- 16. Linear Regression Repeat n times gradient = get_gradient(W) W = W - lr * gradient
- 17. Limits of Linear Regression ● How to deal with big data? ● Stochastic Gradient Descent (A small batch per loop) ● How to choose a proper learning rate(step size)? ● Cross-Validation ● Advanced algorithm — Adagrad ● What if stuck at local optimal? ● A good initialization ● Advanced algorithm — Momentum, Nestrov
- 18. Limits of Linear Regression ● How to deal with non-linear data?
- 19. Limits of Linear Regression ● Normalization ● Normalize all data into [-1, 1] range ● Activation ● Bound update range by threshold
- 20. Linear Regression ● Activation ● Bound update range by threshold
- 21. Linear Regression E.O.F — Linear Regression Part
- 22. Neural Network ● The history of Neural Network ● Created in late 1940s by a psychologist (Hebbian Learning). ● Short resurgence in 1975 powered by back propagation. ● Extremely popular after 2012 (The AlexNet outperforms any other traditional models)
- 23. Multi-Layer Perceptron ● Neural Unit
- 24. Multi-Layer Perceptron ● Stacking (Hidden) Layers
- 25. Multi-Layer Perceptron ● Activation brings non-linearity
- 26. Propagation and Back Propagation W_ij : the weight of edge from previous Jth node to Ith node a_k : the value of Kth node z_k : the value after activation of Kth node
- 28. Propagation and Back Propagation If we have computed all deltas, then derivates on each edge can be easily computed by chain rule.
- 29. Propagation and Back Propagation If we have computed all deltas, then derivates on each edge can be easily computed by chain rule.
- 30. Going Deeper! Deep Neural Network ● Why previous neural network cannot go deeper? ● Too much parameters! ● Gradient vanish ● Backward of tons of parameters ● take weeks to train ● easily to overfit ● stuck at local optimal
- 31. Deep Neural Network ● Image Processing Basics — convolution
- 32. Deep Neural Network ● Image Processing Basics — convolution
- 33. Deep Neural Network ● Convolutional operation ● Different kernel matrices brings different results ● Can be view as a powerful preprocess tool ● But how to choose a proper kernel matrix? ● Let the matrix learn parameters by itself ! ● Convolutional layers!
- 34. Deep Neural Network ● Convolutional Layer
- 36. Deep Neural Network ● AlexNet — Champion of ImageNet competition 2012
- 38. Coding ● PyTorch Tutorial ● Don’t waste time on choosing framework! Just pick one, and start coding!