Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deploying Enterprise Deep Learning Masterclass Preview - Enterprise Deep Learning

180 views

Published on

Day 2 - New York

Published in: Software
  • Be the first to comment

  • Be the first to like this

Deploying Enterprise Deep Learning Masterclass Preview - Enterprise Deep Learning

  1. 1. Deploying Enterprise Deep Learning Sam Putnam, Enterprise Deep Learning, LLC July 27, 2017 Day 2
  2. 2. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Deep Learning Made This Video https://www.extremetech.com/g00/extreme/215170-artificial-neural-networks-are-changing-the-world-what-are-they?i10c.referrer=https%3A%2F%2Fwww.google.com%2F Preview - Slide Available at deeplearningconf.com
  3. 3. Sam Putnam @edeeplearning Deploying Enterprise Deep Learning And This Song! Preview - Slide Available at deeplearningconf.com
  4. 4. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning And Predicted These Housing Prices! https://github.com/EnterpriseDeepLearning/housing-prices-wide-and-deep/blob/master/real_estate.ipynb http://www.mirror.co.uk/news/uk-news/experts-predict-house-prices-could-10724816 Run the Code -> This Afternoon
  5. 5. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning OK, so what is Deep Learning? http://p.migdal.pl/imgs/2017-04-30-learning-deep-learning/deep_learning_meme_keras.png Preview - Slide Available at deeplearningconf.com
  6. 6. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Deep Learning is Neural Networks http://p.migdal.pl/imgs/2017-04-30-learning-deep-learning/deep_learning_meme_keras.png Preview - Slide Available at deeplearningconf.com
  7. 7. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Well, Artificial Neural Networks. But they are based off of how the brain works. https://www.extremetech.com/g00/extreme/215170-artificial-neural-networks-are-changing-the-world-what-are-they?i10c.referrer=https%3A%2F%2Fwww.google.com%2F Preview - Slide Available at deeplearningconf.com
  8. 8. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning This is a Neuron: Electrical Signals Are Sent By Axons and Received By Dendrites http://www.newworldencyclopedia.org/entry/Dendrite Preview - Slide Available at deeplearningconf.com
  9. 9. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning This is a Single Artificial Neuron With Two Inputs and Two Weights Preview - Slide Available at deeplearningconf.com
  10. 10. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning The Artificial Neuron is Also Called a “Perceptron”. It “Perceives” Inputs. Preview - Slide Available at deeplearningconf.com
  11. 11. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning b is the bias, and to activate the neuron:
  12. 12. Sam Putnam @edeeplearning Deploying Enterprise Deep Learning Click Play, Visualize Neuron’s Performance http://playground.tensorflow.org/
  13. 13. Sam Putnam @edeeplearning Deploying Enterprise Deep Learning Trust Me, When you add more inputs, your decision boundary becomes a “plane” Preview - Slide Available at deeplearningconf.com
  14. 14. When you’re talking about Deep Learning you’re talking about Artificial Neural Networks of Neurons @edeeplearning Preview - Slide Available at deeplearningconf.com
  15. 15. In particular, Deep Learning = Deep Artificial Neural Networks of Neurons @edeeplearning Preview - Slide Available at deeplearningconf.com
  16. 16. This it it - Just more than one layer of neurons between the input and output! @edeeplearninghttp://blog.christianperone.com/2015/08/convolutional-neural-networks-and-feature-extraction-with-python/ Preview - Slide Available at deeplearningconf.com
  17. 17. You Tried The Line, Now Add A Hidden Layer To Capture More Complex Data Separations @edeeplearninghttp://blog.christianperone.com/2015/08/convolutional-neural-networks-and-feature-extraction-with-python/
  18. 18. This is No Line You Are Tweaking! This Is a More Complex Function @edeeplearninghttp://blog.christianperone.com/2015/08/convolutional-neural-networks-and-feature-extraction-with-python/ Preview - Slide Available at deeplearningconf.com
  19. 19. More Neurons + A Deeper Network = More Sophisticated Representations @edeeplearninghttps://cloud.google.com/blog/big-data/2016/07/understanding-neural-networks-with-tensorflow-playground
  20. 20. Look at the last hidden layer - it is doing a pretty good job separating the data @edeeplearninghttps://cloud.google.com/blog/big-data/2016/07/understanding-neural-networks-with-tensorflow-playground Preview - Slide Available at deeplearningconf.com
  21. 21. This Architecture is called a Feedforward Neural Network @edeeplearninghttps://cloud.google.com/blog/big-data/2016/07/understanding-neural-networks-with-tensorflow-playground Preview - Slide Available at deeplearningconf.com
  22. 22. There’s a Lot Going on Here. Do You Really Want to Change Those Weights One at a Time? @edeeplearninghttps://cloud.google.com/blog/big-data/2016/07/understanding-neural-networks-with-tensorflow-playground Preview - Slide Available at deeplearningconf.com
  23. 23. Solution: Gradient Descent - Look How the Lowest Loss (0) Is Sought Out on the Right @edeeplearninghttps://cloud.google.com/blog/big-data/2016/07/understanding-neural-networks-with-tensorflow-playground
  24. 24. What other Architectures Are There? Well, what about for Time Series Data @edeeplearninghttps://deeplearning4j.org/lstm.html Preview - Slide Available at deeplearningconf.com
  25. 25. Recurrent Neural Networks are Networks with Loops in Them @edeeplearninghttp://colah.github.io/posts/2015-08-Understanding-LSTMs/ Preview - Slide Available at deeplearningconf.com
  26. 26. Information is passed from one step of the network to the next @edeeplearninghttp://colah.github.io/posts/2015-08-Understanding-LSTMs/ Preview - Slide Available at deeplearningconf.com
  27. 27. Recurrent Neural Networks (RNNs) Contain A Single Layer (Yellow Box) @edeeplearninghttp://colah.github.io/posts/2015-08-Understanding-LSTMs/
  28. 28. Recurrent Neural Networks (RNNs) cannot handle long term dependencies @edeeplearninghttp://colah.github.io/posts/2015-08-Understanding-LSTMs/ Preview - Slide Available at deeplearningconf.com
  29. 29. LSTM Recurrent Neural Networks (RNNs) Contain Four Layers (Yellow Box) @edeeplearninghttp://colah.github.io/posts/2015-08-Understanding-LSTMs/
  30. 30. Long Short Term Memory Networks (LSTMs) Can Handle Long Term Dependencies @edeeplearninghttp://colah.github.io/posts/2015-08-Understanding-LSTMs/ Preview - Slide Available at deeplearningconf.com
  31. 31. @edeeplearning Sigmoid Activation Function Scales Your Input Between 0 and 1 http://neuralnetworksanddeeplearning.com/chap4.html Preview - Slide Available at deeplearningconf.com
  32. 32. Sigmoid Activation Function Scales Your Input Between 0 and 1 @edeeplearning http://neuralnetworksanddeeplearning.com/chap4.html Preview - Slide Available at deeplearningconf.com
  33. 33. Tanh Activation Function Scales Your Input Between -1 and 1 @edeeplearning http://neuralnetworksanddeeplearning.com/chap4.html Preview - Slide Available at deeplearningconf.com
  34. 34. RELU Activation Function Cuts off Negative Inputs @edeeplearning http://neuralnetworksanddeeplearning.com/chap4.html Preview - Slide Available at deeplearningconf.com
  35. 35. What About Image Data? For That, Convolutional Neural Networks (Right) @edeeplearninghttp://cs231n.github.io/convolutional-networks/
  36. 36. Convolution Just Means Picking Out Representative Elements from a Bunch of Pixels (F represents all the below) @edeeplearning Preview - Slide Available at deeplearningconf.com
  37. 37. This is What Convolution Looks Like, a Little Boring, But Look How It Reduces the Size of the Data (3x3 on the Right!) @edeeplearning Preview - Slide Available at deeplearningconf.com
  38. 38. So Convolution Is a Mathematical Operation (Using a Weight Filter, Let’s Look!), but Even Simpler is Max Pooling, just take the Biggest Pixel! @edeeplearning
  39. 39. A Pre-Trained Image Recognition Neural Network Looks Like This (Input At Left) - Let’s Zoom In @edeeplearninghttps://arxiv.org/pdf/1409.4842.pdf Preview - Slide Available at deeplearningconf.com
  40. 40. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Deep Learning in Production Considerations Help me find the source for this nice diagram? Lost it :(
  41. 41. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Solution - Use What I Know and Am Allowed To Use, i.e. Kaggle, AWS, TF, Jupyter Help me find the source for this nice diagram? Lost it :( Preview - Slide Available at deeplearningconf.com
  42. 42. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning What I Did - A Real Estate Agent Told Me About His Problem & Pointed Me To The Data https://www.fortunebuilders.com/how-to-become-a-real-estate-agent/ Preview - Slide Available at deeplearningconf.com
  43. 43. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning I Took a Holdout Set of the Data and Left the Rest for Train/Validate https://www.fortunebuilders.com/how-to-become-a-real-estate-agent/
  44. 44. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Started with Feature Selection and then Linear Regression in Good old Excel, WITH the subject matter expert (the real estate agent). Did Pretty Well. https://www.youtube.com/watch?v=SQkpLMLoqww Preview - Slide Available at deeplearningconf.com
  45. 45. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Preview - Slide Available at deeplearningconf.com How Well? 18 percent Mean Error and 26 percent Median Error before Feature Selection using Weighted Importance, Did some thinking:
  46. 46. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Feature Importance, for Feature Selection, Often Use After Deep Learning Step, to Gain Interpretability
  47. 47. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning https://www.kaggle.com/samdeeplearning/naive-subsample-5-10-city/output GBM (Boosted Decision Tree) - with Hyperparameter Tweaking - Yields Strong Model for this Problem
  48. 48. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning https://www.kaggle.com/samdeeplearning/naive-subsample-0-25-xgb/notebook Preview - Slide Available at deeplearningconf.com Starts to Degrade Upon Subsampling only 1/4 of the Trees
  49. 49. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning
  50. 50. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Want To See If Neural Network Can Improve Performance
  51. 51. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Building off of a preexisting housing prices regression model https://github.com/EnterpriseDeepLearning/housing-prices-wide-and-deep/blob/master/real_estate.ipynb Preview - Slide Available at deeplearningconf.com
  52. 52. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Set Input Layer (n_hidden_1 is input layer here) to number of features (27) https://github.com/EnterpriseDeepLearning/housing-prices-wide-and-deep/blob/master/real_estate.ipynb Preview - Slide Available at deeplearningconf.com
  53. 53. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Combining the features right off the bat by using a wide first hidden layer (200 nodes) https://github.com/EnterpriseDeepLearning/housing-prices-wide-and-deep/blob/master/real_estate.ipynb Preview - Slide Available at deeplearningconf.com
  54. 54. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Set number of classes to 1 for regression problem https://github.com/EnterpriseDeepLearning/housing-prices-wide-and-deep/blob/master/real_estate.ipynb Preview - Slide Available at deeplearningconf.com
  55. 55. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Multilayer Perceptron = Feedforward Perceptron We Looked At Before https://github.com/EnterpriseDeepLearning/housing-prices-wide-and-deep/blob/master/real_estate.ipynb Preview - Slide Available at deeplearningconf.com
  56. 56. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning RELU - As Mentioned in Executing Strategies Yesterday, not using negative features/weights https://github.com/EnterpriseDeepLearning/housing-prices-wide-and-deep/blob/master/real_estate.ipynb
  57. 57. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Training - Can tell performance needs tweaking, pretty good on houses 2-6 https://github.com/EnterpriseDeepLearning/housing-prices-wide-and-deep/blob/master/real_estate.ipynb 1 2 34 5 6 Estimate on LeftPrediction on Left
  58. 58. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Let’s Dive Into the TensorFlow Housing Code https://www.kaggle.com/samdeeplearning/deep-neural-network-for-starters-r/edit Preview - Slide Available at deeplearningconf.com
  59. 59. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning What Next? Go Deep Learning! https://www.youtube.com/watch?v=cSKfRcEDGUs&list=PLOU2XLYxmsIIuiBfYad6rFYQU_jL2ryal&index=6
  60. 60. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning 1) Identify an architecture for your problem https://www.youtube.com/watch?v=cSKfRcEDGUs&list=PLOU2XLYxmsIIuiBfYad6rFYQU_jL2ryal&index=6
  61. 61. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning 2) Identify a Framework, Be Willing to Try a New Framework https://blog.algorithmia.com/deploying-deep-learning-cloud-services/
  62. 62. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning 3) Build or tailor a model for your application, Don’t be Afraid to Reference Academic Code https://www.youtube.com/watch?v=cSKfRcEDGUs&list=PLOU2XLYxmsIIuiBfYad6rFYQU_jL2ryal&index=6
  63. 63. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning 4)Tweak the Parameters for your Model https://www.youtube.com/watch?v=cSKfRcEDGUs&list=PLOU2XLYxmsIIuiBfYad6rFYQU_jL2ryal&index=6
  64. 64. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning 5)Train on a (probably, Nvidia, maybe Intel) GPU if need be (for image data, you need it) https://www.youtube.com/watch?v=2NrgPdGSXhE
  65. 65. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning 6) Validate, test, check in the real world. Iterate! https://medium.com/towards-data-science/train-test-split-and-cross-validation-in-python-80b61beca4b6 Preview - Slide Available at deeplearningconf.com
  66. 66. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Weekend trial: Josh Gordon’s Machine Learning Recipes. If the code doesn’t work for you (I am 99% sure it will), let me know! https://www.youtube.com/watch?v=cSKfRcEDGUs&list=PLOU2XLYxmsIIuiBfYad6rFYQU_jL2ryal&index=6 Preview - Slide Available at deeplearningconf.com
  67. 67. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Like R? Try this notebook on my Kaggle https://www.kaggle.com/samdeeplearning/deep-neural-network-for-starters-r/edit
  68. 68. Sam Putnam @edeeplearning July 27 Deploying Enterprise Deep Learning Submit your predictions, even! (disclaimer: middle of the pack result) https://www.kaggle.com/samdeeplearning/deep-neural-network-for-starters-r/edit Preview - Slide Available at deeplearningconf.com
  69. 69. Sam Putnam Thank you to Google and others who have published diagrams and photos. Slides are for today only. @edeeplearning Questions/Comments: Sam@EDeepLearning.com Thank you July 27 Deploying Enterprise Deep Learning

×