Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to Machine Learning, Deep Learning and MXNet

1,343 views

Published on

Speaker: Osemeke Isibor, Solutions Architect, AWS
Deep learning continues to push the state of the art in domains such as computer vision, natural language understanding and recommendation engines. One of the key reasons for this progress is the availability of highly flexible and developer friendly deep learning frameworks. In this workshop, we will provide an overview of deep learning focusing on getting started with the TensorFlow and Keras frameworks on AWS.

  • Be the first to comment

Introduction to Machine Learning, Deep Learning and MXNet

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Osemeke Isibor, Solutions Architect. iosemeke@amazon.com 31st October 2017 Introduction to Machine Learning, Deep Learning and MXNet
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda • Introduction • Machine Learning • Deep Learning • MXNet • Conclusion
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Introduction Artificial Intelligence Machine Learning Deep Learning
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Timeline Of Intelligent Machines 1950 1952 1957 1979 1986 1997 2011 2012 2014 2016 The Learning Machine (Alan Turing) Machine Playing Checker (Author Samuel) Perceptron (Frank Rosenblatt) Stanford Cart Backpropagation (D. Rumelhart, G. Hinton, R. Williams) Deep Blue Beats Kasparov Watson Wins Jeopardy DeepMind Wins GoGoogle NN recognizing cat in Youtube Facebook DeepFace, Amazon Echo
  5. 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is Machine Learning?
  6. 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Marketing Offer On A New Product
  7. 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Option 1- Build A Rule Engine Age Gender Purchase Date Items 30 M 3/1/2017 Toy 40 M 1/3/2017 Books …. …… ….. ….. Input Output Age Gender Purchase Date Items 30 M 3/1/2017 Toy …. …… ….. ….. Rule 1: 15 <age< 30 Rule 2: Bought Toy=Y, Last Purchase<30 days Rule 3: Gender = ‘M’, Bought Toy =‘Y’ Rule 4: …….. Rule 5: …….. Human Programmer
  8. 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Problem with Hand Designed Rules Adaptability Scalability Closed Loop
  9. 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Option 2 - Learn The Business Rules From Data Age Gender Purchase Date Items 30 M 3/20/2017 Toy * 40 M 1/3/2017 Books …. …… ….. ….. Learning Algorithm Model Output Historical Purchase Data (Training Data) Prediction Age Gender Items 35 F 39 M Toy Input - New Unseen Data
  10. 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. We Call This Approach Machine Learning
  11. 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Why Use Machine Learning? • Use ML when you can’t code it • Complex tasks where deterministic solution don’t suffice • E.g. Recognizing speech/images • Use ML when you can’t scale it • Replace repetitive tasks needing human like expertise • E.g Recommendations, spam, fraud detection, machine translation. • Use ML when you have to adapt/personalize • E.g. Recommendation and personalization • Use ML when you can’t track it • E.g. Automated driving, fraud detection.
  12. 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Types Of Machine Learning
  13. 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Supervised Learning It is a cat. No, it’s a Labrador.
  14. 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Supervised Learning – How Machine Learn Human intervention and validation required e.g. Photo classification and tagging Input Label Machine Learning Algorithm Labrador Prediction Cat Training Data ? Label Labrador Adjust Model
  15. 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Unsupervised Learning No human intervention required (e.g. Customer segmentation) Input Machine Learning Algorithm Prediction
  16. 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Model Training
  17. 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Model Training – Split training data All Labeled Dataset Training Data 70% 30%
  18. 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Model Training – Training w/ training data All Labeled Dataset Training Data 70% 30% Training Trial Model
  19. 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Model Training – Split the test data All Labeled Dataset Training Data 70% 30% Training Trial Model Test Data
  20. 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Model Training – Model evaluation All Labeled Dataset Training Data 70% 30% Training Test Data Evaluation Result Trial Model
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Model Training - Performance Measurement All Labeled Dataset Training Data 70% 30% Training Test Data Evaluation Result Trial Model Accuracy
  22. 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Deep Learning
  23. 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What is Deep Learning? • Deep Learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks. • Data is passed through multiple non-linear transformations to generate a prediction • Objective: Learn the parameters of the transformations that minimize a cost function
  24. 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Performance Data Performance Traditional Machine Learning Algorithms Deep Learning Algorithms
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Sample Deep Learning Use Cases ASR/NLU Language Translation Self Driving Cars Playing Go Financial Risk Medical Diagnosis
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Algorithms Data Programming Models GPUs & Acceleration The Advent of Deep Learning image understanding natural language processing speech recognition autonomy
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Artificial Neuron/Perceptron Input: Vector of training data x Output: Linear function of input Nonlinearity: Transform output into desired range of value Training Learn the weights and bias b by minimize loss f(x) = 𝜎 (⟨w, x⟩ + b) X0 X1 X2 Xn … w0 w1 w2 wn OutputInputs ⟨w, x⟩ 𝜎 Neuron b
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Human Brain Neuron Inputs Output
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Neural Network X0 Xn Neuron 0 Neuron n Neuron 0 Neuron n Output Neuron Input Layer Hidden Layer 1 Hidden Layer 2 Output Layer …… …… …… w1 0 w1 1 w1 2 w1 3
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Neural Network – Forward Propagation X0 Xn Neuron 0 Neuron n Neuron 0 Neuron n Output Neuron Input Layer Hidden Layer 1 Hidden Layer 2 Output Layer …… …… …… Input 5 w1 0 w1 1 w1 2 w1 3
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Neural Network – Backpropagation X0 Xn Neuron 0 Neuron n Neuron 0 Neuron n Output Neuron Input Layer Hidden Layer 1 Hidden Layer 2 Output Layer …… …… …… 5 Label 4 ? Input Error/Loss Error/LossError/Loss Error/Loss Error/Loss w1 0 w1 1 w1 2 w1 3
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Neural Network – Backpropagation X0 Xn Neuron 0 Neuron n Neuron 0 Neuron n Output Neuron Input Layer Hidden Layer 1 Hidden Layer 2 Output Layer …… …… …… 5 Label 4 ? Error/Loss Input Error/LossError/Loss Error/Loss Error/Loss W1’ 0 W1’ 1 W1’ 2 W1’ 3 Update weights
  33. 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Classification Computer Vision – Deep Learning Approach Raw Image Pixels Edge Detection Object Parts Detection (Combination of edges) Object Model Detection Object Prediction Feature Extraction
  34. 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 你 好 吗 “How” “Are” “You” Encoder Decoder 0.643 0.875 0.345 . . Input One Word At a Time Model ModelEncoded Vector Output One Word At a Time Memory of previous word influences next result Memory of previous word influences next result Language Translation – Deep Learning Approach
  35. 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Demo – http://amzn.to/takeselfie
  36. 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MXNet
  37. 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Flexible - Supports both imperative and symbolic programming • Portable - Runs on CPUs or GPUs, on clusters, servers, desktops, or mobile phones • Multiple Languages - C++, Python, R, Scala, Julia, Matlab, Javascript, and Perl • Distributed on Cloud - Supports distributed training on multiple CPU/GPU machines • Performance Optimized - Optimized C++ backend engine parallelizes both I/O and computation • Broad Model Support - CNN, RNN/LSTM
  38. 38. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MXNet Architecture BLAS Dep Engine Comm ND Array Symbolic Expr Binder KV Store User Facing Modules System Modules …CPU GPU Android iOS Hardware & OS C++ Python R Julia… Language Interface
  39. 39. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MXNet API Components • NDArray – Provides imperative tensor operations • Symbol – Provides neural network graph and auto- differentiation • RNN Cell – Tools for building RNN symbolic graph • Module – Provides interface for performing computation with Symbol • Data Loading – Provides iterators for reading data • Metric - Evaluation metric to evaluate performance of trained model
  40. 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Model training flow in MXNet Data Loading Input Data (text, image, sound, …) Data Iterator (NDArrayIter, CSVIter, …) Symbol, RNN Cell Module Data Loading and Processing Network Graph & Error Function Learned Model (Network Graph, Parameters) Model Training Optimizer (sgd, adam, …) Context (cpu,gpu) Metric (acc, mse, … ) Device Selection Optimizer Selection Metric Selection
  41. 41. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Distributed Deep Learning – Data Parallelism Data Shards Model Replica Parameter Server W ’ = W - 𝛼∆W W ∆W W ∆W ∆WW Data Shards Data Shards Model Replica Model Replica
  42. 42. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Distributed Deep Learning – Model Parallelism
  43. 43. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Conclusion: Get Started https://aws.amazon.com/amazon-ai/what-is-ai/
  44. 44. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Osemeke Isibor, Solutions Architect iosemeke@amazon.com

×