An Introduction to Deep Learning (March 2018)
•
2 likes•27,341 views
This document provides an introduction to deep learning, including common network architectures and use cases. It discusses neural networks and how they are trained using techniques like stochastic gradient descent and backpropagation. Common network types are described, such as convolutional neural networks (CNNs) for image data and long short term memory networks (LSTMs) for sequence data. Applications like object detection, segmentation, machine translation and generative adversarial networks are overviewed. The document concludes by describing Amazon SageMaker for building and deploying machine learning models at scale.
More Related Content
What's hot
What's hot (20)
Similar to An Introduction to Deep Learning (March 2018)
Similar to An Introduction to Deep Learning (March 2018) (20)
More from Julien SIMON
More from Julien SIMON (20)
Recently uploaded
Recently uploaded (20)
An Introduction to Deep Learning (March 2018)
- 1. An Introduction to Deep Learning Julien Simon, AI Evangelist, EMEA @julsimon https://medium.com/@julsimon
- 2. Agenda • An introduction to Deep Learning • Common network architectures and use cases • Demos • Resources
- 3. An introduction to Deep Learning
- 4. Activation functions The neuron i=1 l xi ∗ wi = u ”Multiply and Accumulate” Source: Wikipedia
- 5. x = x11, x12, …. x1I x21, x22, …. x2I … … … xm1, xm2, …. xmI I features m samples y = 2 0 … 4 m labels, N2 categories 0,0,1,0,0,…,0 1,0,0,0,0,…,0 … 0,0,0,0,1,…,0 One-hot encoding Neural networks
- 6. x = x11, x12, …. x1I x21, x22, …. x2I … … … xm1, xm2, …. xmI I features m samples y = 2 0 … 4 m labels, N2 categories Total number of predictions Accuracy = Number of correct predictions 0,0,1,0,0,…,0 1,0,0,0,0,…,0 … 0,0,0,0,1,…,0 One-hot encoding Neural networks
- 7. Neural networks Initially, the network will not predict correctly f(X1) = Y’1 A loss function measures the difference between the real label Y1 and the predicted label Y’1 error = loss(Y1, Y’1) For a batch of samples: 𝑖=1 𝑏𝑎𝑡𝑐ℎ 𝑠𝑖𝑧𝑒 loss(Yi, Y’i) = batch error The purpose of the training process is to minimize error by gradually adjusting weights
- 8. Training Training data set Training Trained neural network Batch size Learning rate Number of epochs Hyper parameters Backpropagation
- 9. Stochastic Gradient Descent (SGD) Imagine you stand on top of a mountain with skis strapped to your feet. You want to get down to the valley as quickly as possible, but there is fog and you can only see your immediate surroundings. How can you get down the mountain as quickly as possible? You look around and identify the steepest path down, go down that path for a bit, again look around and find the new steepest path, go down that path, and repeat—this is exactly what gradient descent does. Tim Dettmers University of Lugano 2015 https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-history-training/ The « step size » is called the learning rate z=f(x,y)
- 10. Optimizers
- 11. Validation Validation data set Trained neural network Validation accuracy Prediction at the end of each epoch
- 12. Early stopping Training accuracy Loss function Accuracy 100% Epochs Validation accuracy Loss Best epoch OVERFITTING « Deep Learning ultimately is about finding a minimum that generalizes well, with bonus points for finding one fast and reliably », Sebastian Ruder
- 13. Common network architectures and use cases
- 14. Convolutional Neural Networks (CNN) Le Cun, 1998: handwritten digit recognition, 32x32 pixels Convolution and pooling reduce dimensionality https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/
- 15. Object Detection https://github.com/precedenceguo/mx-rcnn https://github.com/zhreshold/mxnet-yolo
- 17. Text Detection and Recognition https://github.com/Bartzi/stn-ocr
- 19. Real-Time Pose Estimation https://github.com/dragonfly90/mxnet_Realtime_Multi-Person_Pose_Estimation
- 20. Long Short Term Memory Networks (LSTM) • A LSTM neuron computes the output based on the input and a previous state • LSTM networks have memory • They’re great at predicting sequences, e.g. machine translation
- 22. GAN: Welcome to the (un)real world, Neo Generating new ”celebrity” faces https://github.com/tkarras/progressive_growing_of_gans From semantic map to 2048x1024 picture https://tcwang0509.github.io/pix2pixHD/
- 23. Wait! There’s more! Models can also generate text from text, text from images, text from video, images from text, sound from video, 3D models from 2D images, etc.
- 25. Highly-optimized machine learning algorithms BuildPre-built notebook instances Train Amazon SageMaker One-click training for ML, DL, and custom algorithms Easier training with hyperparameter optimization
- 26. One-click training for ML, DL, and custom algorithms Easier training with hyperparameter optimization Highly-optimized machine learning algorithms Deployment without engineering effort Fully-managed hosting at scale BuildPre-built notebook instances Deploy Train Amazon SageMaker
- 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Resources https://aws.amazon.com/machine-learning https://aws.amazon.com/sagemaker https://aws.amazon.com/blogs/ai https://gluon.mxnet.io An overview of Amazon SageMaker https://www.youtube.com/watch?v=ym7NEYEx9x4
- 29. Thank you! Julien Simon, AI Evangelist, EMEA @julsimon https://medium.com/@julsimon