Download as PDF, PPTX
Uploaded byJulien SIMON
An introduction to Deep Learning
The document introduces deep learning, explaining its concepts and the distinction between artificial intelligence, machine learning, and deep learning. It discusses common myths about AI, provides insights into deep learning in action, and outlines the basics of neural networks and the training process. Additionally, it highlights resources and tools available for implementing deep learning on platforms like AWS and NVIDIA.
More Related Content
Similar to An introduction to Deep Learning
An introduction to Deep Learning
- 1. An introduction to DeepLearning Julien Simon, AI Evangelist, EMEA @julsimon
- 2. What to expect •Artificial Intelligence, Machine Learning, Deep Learning • The 5 myths of AI • Deep Learning in action • Basics of Deep Learning • NVIDIA Volta V100 and AWS P3 • Q&A
- 3. • Artificial Intelligence: design software applications which exhibit human-like behavior, e.g. speech, natural language processing, reasoning or intuition • Machine Learning: teach machines to learn without being explicitly programmed •Deep Learning: using neural networks, teach machines to learn from data where features cannot be explicitly expressed
- 4. The 5 Mythsof AI
- 5. Myth #1 -AI is the flavour of the month
- 6. John McCarthy (1927-2011) 1956- Coined the term “Artificial Intelligence” 1958 - Invented LISP 1971 - Received the Turing Award Fact #1 - AI is 60 years old Marvin Minsky (1927-2016) 1959 - Co-founded the MIT AI Lab 1968 - Advised Kubrick on “2001: A Space Odyssey” 1969 - Received the Turing Award
- 7. Myth #2 -AI is dark magic aka « You’re not smart enough »
- 8. Fact #2 -AI is math, code and chips A bit of Science, a lot of Engineering
- 9. Myth #3 –The “cognitive” unicorn
- 10. Myth #3 –The “cognitive” unicorn
- 11. Fact #3: AIis a wide range of techniques and tools • Machine Learning • Natural Language Processing • Speech • Vision • Expert Systems • And more
- 12. Myth #4 -AI is for esoteric use cases
- 13. Fact #4: AIshines on intuitive problems
- 14. Amazon AI isbased on Deep Learning
- 15. Myth #5 -AI is not production-ready
- 16. Fact #5: AImeans business
- 17. Deep Learning inaction
- 18. Selected customers runningAI on AWS
- 19. Basics of DeepLearning
- 20. Activation functions The neuron !xi ∗ wi & '() = u ”Multiply and Accumulate” Source: Wikipedia
- 21. © 2017, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. x = x11, x12, …. x1I x21, x22, …. x2I … … … xm1, xm2, …. xmI I features m samples y = y1 y2 … ym m labels, N2 outputs 0,0,1,0,0,…,0 1,0,0,0,0,…,0 … 0,0,0,0,1,…,0 One-hot encoding Neural networks
- 22. © 2017, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. x = x11, x12, …. x1I x21, x22, …. x2I … … … xm1, xm2, …. xmI I features m samples y = y1 y2 … ym m labels, N2 categories Total number of predictions Accuracy = Number of correct predictions 0,0,1,0,0,…,0 1,0,0,0,0,…,0 … 0,0,0,0,1,…,0 One-hot encoding Neural networks
- 23. © 2017, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Neural networks Initially, the network will not predict correctly f(X1) = Y’1 A loss function measures the difference between the real label Y1 and the predicted label Y’1 error = loss(Y1, Y’1) For a batch of samples: ! loss(Yi, Y’i) ,-./0 2345 3() = batch error The purpose of the training process is to minimize error by gradually adjusting weights
- 24. © 2017, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Training Training data set Training Trained neural network Batch size Learning rate Number of epochs Hyper parameters Backpropagation
- 25. Stochastic Gradient Descent(SGD) Imagine you stand on top of a mountain with skis strapped to your feet. You want to get down to the valley as quickly as possible, but there is fog and you can only see your immediate surroundings. How can you get down the mountain as quickly as possible? You look around and identify the steepest path down, go down that path for a bit, again look around and find the new steepest path, go down that path, and repeat—this is exactly what gradient descent does. Tim Dettmers University of Lugano 2015 https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-history-training/ The « step size » is called the learning rate z=f(x,y)
- 26. © 2017, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Validation Validation data set Trained neural network Validation accuracy Prediction at the end of each epoch Save the model at the end of each epoch
- 27. © 2017, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Early stopping Training accuracy Loss function Accuracy 100% Epochs Validation accuracy Loss Best checkpoint OVERFITTING
- 28. © 2017, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Deep Learning in practice AWS Deep Learning AMI MXNet Torch CTKKeras TheanoCaffe TensorFlow Amazon EC2 AnacondaIntel MKL CUDA+cuDNN Python 2+3 Caffe2 • One-click launch • Single node or distributed • CPU, GPU, FPGA • NVIDIA & Intel libraries • Anaconda Data Science Platform • Python w/ AI/ML/DL libraries
- 29. NVIDIA Volta andAWS P3
- 30. © 2017, AmazonWeb Services, Inc. or its Affiliates. All rights reserved. Resources https://aws.amazon.com/machine-learning https://aws.amazon.com/blogs/ai https://www.nvidia.com/en-us/deep-learning-ai/ https://www.nvidia.fr/dli https://www.nvidia.fr/data-center/volta-gpu-architecture/ https://aws.amazon.com/ec2/instance-types/p3/ https://medium.com/@julsimon
- 31. Thank you! Julien Simon,AI Evangelist, EMEA @julsimon