Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Intro to Deep Learning w/ Clojure

1,195 views

Published on

This is the code for a talk I gave to the Clojure PDX meetup group on Intro to Deep Learning w/ Clojure using ThinkTopic/cortex.
The talk examined the question "Given that Python dominates ML/DL; Is there a valid use case for Clojure?" and came to the conclusion is that deep learning is not a silver bullet but a power tool that can

- handle much of the feature engineering,
- handles complex (non linear) problems
- and has many interesting advancements coming quickly
Code and other info are available at https://E-String.com/articles/intro-deep-learning-clojure/

Published in: Technology
  • Be the first to comment

Intro to Deep Learning w/ Clojure

  1. 1. Intro to Deep Learning w/ Clojure It’s Difficult to Make Predictions. Especially About the Future. @JulioBarros Consultant E-String.com @JulioBarros http://E-String.com 1
  2. 2. Hypothesis Given that Python dominates ML/DL; Is there a valid use case for Clojure? TL;DR --It depends @JulioBarros http://E-String.com 2
  3. 3. I Hope So ...and I'm not the only one. @JulioBarros http://E-String.com 3
  4. 4. Hype or Reality? @JulioBarros http://E-String.com 4
  5. 5. "It is a renaissance, it is a golden age," "MachinelearningandAIisahorizontalenablinglayer.Itwillempowerand improveeverybusiness,everygovernmentorganization,everyphilanthropy —basicallythere'snoinstitutionintheworldthatcannotbeimprovedwith machinelearning." —Bezos @JulioBarros http://E-String.com 5
  6. 6. Every industry can expect to be transformed by Artificial Intelligence @JulioBarros http://E-String.com 6
  7. 7. Image Classification Justin Johnson,Andrej Karpathy,Li Fei-Fei-Stanford @JulioBarros http://E-String.com 7
  8. 8. My dog Rex @JulioBarros http://E-String.com 8
  9. 9. Object Detection @JulioBarros http://E-String.com 9
  10. 10. Image Captioning @JulioBarros http://E-String.com 10
  11. 11. Dense Captioning @JulioBarros http://E-String.com 11
  12. 12. Healthcare Near or better than human level performance. @JulioBarros http://E-String.com 12
  13. 13. Business — Law & Finance — Text,audio,image,video understanding — Churn prediction,customer segmentation — Product recommendations — Manufacturing,maintenance and control — Many many more @JulioBarros http://E-String.com 13
  14. 14. Artificial Intelligence Artificial Intelligence (AI)-the study of"intelligent agents".Reasoning, knowledge representation,planning,robotics,etc. — Artificial Narrow Intelligence (ANI) — Artificial General Intelligence (AGI) — Artificial Superintelligence (ASI) @JulioBarros http://E-String.com 14
  15. 15. Machine Learning Machine Learning (ML)-Programs that learn from the data and make predictions. @JulioBarros http://E-String.com 15
  16. 16. Types of prediction Regression-continuous values Classification-discreet values/labels @JulioBarros http://E-String.com 16
  17. 17. Types of ML Algorithms Supervised-trained on labeled data (regression or classification). Unsupervised-trained on unlabeled data (clustering,segmentation). Reinforcement-learns based on outcomes/results of actions. @JulioBarros http://E-String.com 17
  18. 18. Deep Learning Deep Learning (DL)-ML/AIusingartificialneuralnetworks(ANNs) You might be thinking... What?Isn'tthatjustaneuralnet? Didn'tweshowthosedon'twork. Twice. @JulioBarros http://E-String.com 18
  19. 19. Third time is a charm Capabilities of neural nets have changed dramatically due to advancements in: — Data — Algorithms — Hardware @JulioBarros http://E-String.com 19
  20. 20. Neurons: Biologically inspired 1942 McCulloch and Pitts 1957 Rosenblatt (A (+ b (apply + (map * x w))) @JulioBarros http://E-String.com 20
  21. 21. Activation Function Introduces non linearity — Historicaly: Identity,Step,Tanh,Sigmoid — Currently: Rectified linear Unit (relu),Softmax @JulioBarros http://E-String.com 21
  22. 22. Universal Approximation Theorem (1989) ...a feed-forward network with a single hidden layer containing a finite number of neurons can approximate continuous functions... @JulioBarros http://E-String.com 22
  23. 23. Deep Neural Nets A net with more than one hidden layer. @JulioBarros http://E-String.com 23
  24. 24. VGG16 @JulioBarros http://E-String.com 24
  25. 25. GoogleLeNet @JulioBarros http://E-String.com 25
  26. 26. Training 1) Initialize weights randomly 2) Make prediction 3) Measure error (loss) 4) Adjust weights in the right direction 5) GOTO 2 Repeat over and over and over again with your training data @JulioBarros http://E-String.com 26
  27. 27. Backpropagation Finding the right direction. @JulioBarros http://E-String.com 27
  28. 28. Backpropagation To know the right direction calculate the gradient of the loss function with respect to each weight.Multiply by the error and a learning rate to get the delta. Don'tworry.Thelibrariesdoitforyou. @JulioBarros http://E-String.com 28
  29. 29. Millions of Knobs Parameters @JulioBarros http://E-String.com 29
  30. 30. A Simple Neural Net in Cortex (def network-architecture [(layers/input 11 1 1 :id :x) (layers/linear 64) (layers/relu) (layers/linear 1 :id :y)]) @JulioBarros http://E-String.com 30
  31. 31. Wine Quality Data 11 features,1 target column @JulioBarros http://E-String.com 31
  32. 32. UCI Machine Learning Repository https://archive.ics.uci.edu/ml/datasets/Wine+Quality Source: Paulo Cortez, University of Minho, Guimarães, Portugal, http://www3.dsi.uminho.pt/pcortez A.Cerdeira, F.Almeida, T.Matos and J.Reis, Viticulture Commission of the Vinho Verde Region(CVRVV), Porto, Portugal @2009 @JulioBarros http://E-String.com 32
  33. 33. Normalized / Scaled data (Standardized) @JulioBarros http://E-String.com 33
  34. 34. Demo https://github.com/thinktopic/cortex https://github.com/JulioBarros/clj-dl-demo @JulioBarros http://E-String.com 34
  35. 35. How Do We Work With Images Well,images are just numbers/data. Though numbers close to each other are more related. @JulioBarros http://E-String.com 35
  36. 36. Convolutional Layers Similar to correlations from signal processing or filters from photoshop. A small NxN filter is slid over and convolved/correlated with the image. Learns to find features. Then lower level features are combined into higher level features. @JulioBarros http://E-String.com 36
  37. 37. Common Types of Layers — input / output — fully connected (dense) — activation-relu,softmax — convolutional — maxpool — flatten — drop out @JulioBarros http://E-String.com 37
  38. 38. Types of ANN 1. Dense Neural Net (DNN) 2. Convolutional Neural Net (CNN) 3. Recurrent Neural Net (RNN) 4. Everything else @JulioBarros http://E-String.com 38
  39. 39. Programming Abstractions Level Python CLJ/JVM DSL/API Keras,Lasagne,TF-Slim Cortex,??? Computation Graph, Backprop,Autograd Tensorflow,Theano, Torch,Pytorch,Caffe, MXNet,CNTK Cortex,dl4j,Java bindings Matrix Math CUDA (cuDNN),Eigen3 Neandertal, core.matrix,vectorz-clj @JulioBarros http://E-String.com 39
  40. 40. Production Considerations — Target hardware environment — GPU(s) — Powerful multicore server — Mobile device — Embedded — Desired latency / scalability — Data pipeline — Software engineering practices @JulioBarros http://E-String.com 40
  41. 41. Don't Underestimate the Last Two We need to consider running and maintaining the models in production. @JulioBarros http://E-String.com 41
  42. 42. Approaches — Python based environment — Clojure based environment — Generate C++ binaries — API calls to third party — API calls to microservices @JulioBarros http://E-String.com 42
  43. 43. Challenges with DL — Needs lots of data.Labeled data is expensive. — Lacks explain-ability — Performance requirements-training and inference — Max performance unclear — Best architecture unclear @JulioBarros http://E-String.com 43
  44. 44. Benefits of DL — Handles much of the feature engineering — Handles complex (non linear) problems — Advancements coming quickly @JulioBarros http://E-String.com 44
  45. 45. Recommendations Do not be intimidated by ANNs or the math. Start with Keras (and Tensorflow or Theano) tutorials (or maybe Pytorch). Later choose language/framework as needs dictate. @JulioBarros http://E-String.com 45
  46. 46. Resources Andrew Ng's Coursera Course and Fast.AI Mooc Deep Learning Book-Goodfellow,Bengio and Courville Meetups -Portland-Data-Science-Group -Portland-Machine-Learning-Meetup -Portland-Deep-Learning @JulioBarros http://E-String.com 46
  47. 47. Thank you! Questions? Julio@E-String.com @JulioBarros @JulioBarros http://E-String.com 47

×