Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introduction to Deep Learning with Will Constable

Deep Residual Nets, Activity recognition in videos, and Q&A systems using neon and the Nervana Cloud

Will Constable will start with an introduction to the field of Deep Learning, neon and the Nervana Cloud. The presentation will be followed by an interactive workshop using neon. neon is an open-source Python based Deep Learning framework that has been built from the ground up for speed, scalability and ease of use.

  • Be the first to comment

  • Be the first to like this

Introduction to Deep Learning with Will Constable

  1. 1. Proprietary and confidential. Do not distribute. Deep Learning, neon, and the Nervana Cloud Will Constable March 3 2016 MAKING MACHINES SMARTER.™
  2. 2. Agenda 2 Neon Nervana Cloud Intro to Deep Learning Demo
  3. 3. deep speech 2 3
  4. 4. what is deep learning? 4 Before deep learning: • Input → designed features → output • Input → designed features → SVM → output • Input → learned features → SVM → output • Input → levels of learned features → output
  5. 5. what is deep learning? 5 (Zeiler and Fergus, 2013)
  6. 6. what is deep learning? 6 ++ +
  7. 7. Proprietary and confidential. Do not distribute. Deep learning techniques imagnet error rate 7 Source: ImageNet 1: ImageNet top 5 error rate Error rate1 human performance
  8. 8. what is deep learning? 8 No free lunch: • lots of data • model design needs intuition • slow experiments
  9. 9. agenda 9 Neon Nervana Cloud Intro to Deep Learning Demo
  10. 10. model Convolution BatchNorm ReLu Activation training workflow 10 Fully Connected Pooling Softmax cost optimizer Train Set Validation Set
  11. 11. model zoo 11 • github.com/nervanazoo/NervanaModelZoo • model files, parameters GoogLeNetAlexnet VGG Deep Residual Net bAbI Q&A imdb Sentiment Analysis Video Activity Detection Deep Reinforcement Learning LSTM Image Captioning Fast-RCNN Object Localization AllCNN
  12. 12. caffe compatibility 12 • github.com/NervanaSystems/caffe2neon.git • Convert caffe model to neon format > ./decaffeinate.py caffe_model.prototxt caffe_params -o neon_model
  13. 13. backend interface 13 from neon.backends import gen_backend be = gen_backend(backend=‘gpu') # backend creation x0 = be.ones((2, 2), name=‘x0') # buffer allocation x1 = be.ones((2, 2), name=‘x1') f_val = be.empty((2, 2)) f = x0 + x1 # op-tree creation f_val[:] = f # execution • numpy-like • consistent cpu/gpu interface • deferred execution op-tree
  14. 14. full stack optimization 14 • multithreaded data loader • hand-assembled kernels • compound kernel generator
  15. 15. release 15 neon 1.3 github.com/nervanaSystems/neon
  16. 16. winograd convolution 16 • 2x faster convolutionAlgorithmicSpeedup 0 0.5 1 1.5 2 winograd cuDNN v4 Efficient Convolution
  17. 17. Agenda 17 Neon Nervana Cloud Intro to Deep Learning Demo
  18. 18. nervana cloud 18 Import Build Train Deploy
  19. 19. nervana cloud 19 neon deep learning framework S3Nervana HardwareAWS S3TitanX Cluster
  20. 20. web UI 20
  21. 21. ncloud cli 21
  22. 22. Proprietary and confidential. Do not distribute. Summary 22 • neon 1.3 (download today) • cloud - contact us (info@nervanasys.com) • We’re hiring! (nervanasys.com/careers) will@nervanasys.com
  23. 23. Agenda 23 Neon Nervana Cloud Intro to Deep Learning Demo

×