Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Deep Learning Deep Dive & Workshop
on Convolutional and Recurrent
Neural Networks
Urs Köster
San Diego Deep Learning Meetu...
Outline
2
• Deep Learning
• Nervana
• Neon
• Convolutional Network Demo (meetup.nervanasys.com)
• Recurrent Network Demo (...
INTRO TO DEEP LEARNING
3
4
Scene Parsing
*Yann LeCun https://www.youtube.com/watch?v=ZJMtDRbqH40
5
Speech Translation
*Skype https://www.youtube.com/watch?v=eu9kMIeS0wQ
6
Understanding Images
*Karpathy http://cs.stanford.edu/people/karpathy/deepimagesent/
7
What is deep learning?
Historical perspective:
• Input → designed features → output
• Input → designed features → SVM → ...
8
What is deep learning?
A method for extracting features at
multiple levels of abstraction
• Features are discovered from...
9
What is deep learning?
No free lunch:
• lots of data
• flexible models
• powerful priors
10
Imagenet ILSVRC Challenge
Error rate1
Source: ImageNet
1: ImageNet top 5 error rate
0%
10%
20%
30%
2010 2011 2012 2013 ...
NERVANA PLATFORM
11
12
nervana platform for deep learning
neon deep
learning
framework
train deploy
nervana
cloud
explore
GPUs
CPUs
nervana en...
13
Deep learning as a core technology
DL
Photos
Maps
Voice
Search
Self-driving
car
Ad
Targeting
Machine
Translation
‘Googl...
• Architecture optimized for
algorithm
14
Core technology
• Unprecedented compute density
• Scalable distributed architect...
15
Markets
Pharma Oil&Gas AgricultureMedical
$
Finance Internet Govt
NEON
16
neon: nervana python deep learning library
17
• User-friendly, extensible, abstracts parallelism & data caching
• Support ...
18
Image classification (VGG-D) speed comparison
Speed for one full forward/backward
pass on VGG model D
Imagespersecond
0...
1 Soumith Chintala, github.com/soumith/convnet-benchmarks
Benchmarks for convnets1
19
Benchmarks compiled by Facebook. Sma...
20
End-to-end optimized
• GPU Kernels:
Written in SASS
Assembler, near full
utilization for most
layers
• Data Loader: neo...
Proprietary and confidential. Do not distribute.
21
Running locally:
% python rnn.py # or neon rnn.yaml
Running in nervana ...
22
HANDS ON!
• Option 1: Interactive ipython notebook in your
browser without installing anything (you should have
paper s...
Contact
23
urs@nervanasys.com
github.com/NervanaSystems/neon
Urs Köster - Convolutional and Recurrent Neural Networks
Upcoming SlideShare
Loading in …5
×

Urs Köster - Convolutional and Recurrent Neural Networks

Speaker: Urs Köster, PhD

Urs will join us to dive deep into the field of Deep Learning and focus on Convolutional and Recurrent Neural Networks. The talk will be followed by a workshop highlighting neon™, an open source python based deep learning framework that has been built from the ground up for speed and ease of use.

  • Login to see the comments

Urs Köster - Convolutional and Recurrent Neural Networks

  1. 1. Deep Learning Deep Dive & Workshop on Convolutional and Recurrent Neural Networks Urs Köster San Diego Deep Learning Meetup December 2, 2015
  2. 2. Outline 2 • Deep Learning • Nervana • Neon • Convolutional Network Demo (meetup.nervanasys.com) • Recurrent Network Demo (meetup.nervanasys.com)
  3. 3. INTRO TO DEEP LEARNING 3
  4. 4. 4 Scene Parsing *Yann LeCun https://www.youtube.com/watch?v=ZJMtDRbqH40
  5. 5. 5 Speech Translation *Skype https://www.youtube.com/watch?v=eu9kMIeS0wQ
  6. 6. 6 Understanding Images *Karpathy http://cs.stanford.edu/people/karpathy/deepimagesent/
  7. 7. 7 What is deep learning? Historical perspective: • Input → designed features → output • Input → designed features → SVM → output • Input → learned features → SVM → output • Input → levels of learned features → output
  8. 8. 8 What is deep learning? A method for extracting features at multiple levels of abstraction • Features are discovered from data • Performance improves with more data • Network can express complex transformations • High degree of representational power
  9. 9. 9 What is deep learning? No free lunch: • lots of data • flexible models • powerful priors
  10. 10. 10 Imagenet ILSVRC Challenge Error rate1 Source: ImageNet 1: ImageNet top 5 error rate 0% 10% 20% 30% 2010 2011 2012 2013 2014 2015 Deep learning techniques human performance
  11. 11. NERVANA PLATFORM 11
  12. 12. 12 nervana platform for deep learning neon deep learning framework train deploy nervana cloud explore GPUs CPUs nervana engine AWS VM S3 S3 Web VM VM VM VM VM S3
  13. 13. 13 Deep learning as a core technology DL Photos Maps Voice Search Self-driving car Ad Targeting Machine Translation ‘Google Brain’ model DL Image classification Image localization Speech recognition Video indexing Sentiment analysis Machine Translation Nervana Platform
  14. 14. • Architecture optimized for algorithm 14 Core technology • Unprecedented compute density • Scalable distributed architecture • Learning and inference
  15. 15. 15 Markets Pharma Oil&Gas AgricultureMedical $ Finance Internet Govt
  16. 16. NEON 16
  17. 17. neon: nervana python deep learning library 17 • User-friendly, extensible, abstracts parallelism & data caching • Support for many deep learning models • Interface to nervana cloud • Multiple backends • nervana engine • GPU (optimized on assembler level) • CPU cluster Open source (Apache 2.0) on github.com/nervanaSystems/neon
  18. 18. 18 Image classification (VGG-D) speed comparison Speed for one full forward/backward pass on VGG model D Imagespersecond 0 20 40 60 80 100 120 Neon Caffe • Neon trains networks about 2x faster! • Caffe uses CuDNN v3 (NVidia’s own optimized library) • Same holds for other models (GoogLeNet, AlexNet) • And other Frameworks (Torch7, TensorFlow)
  19. 19. 1 Soumith Chintala, github.com/soumith/convnet-benchmarks Benchmarks for convnets1 19 Benchmarks compiled by Facebook. Smaller is better.
  20. 20. 20 End-to-end optimized • GPU Kernels: Written in SASS Assembler, near full utilization for most layers • Data Loader: neon never blocks waiting for data Library Wrapper DataLoader DataLoader DecodeThreads start IOThreads destroy thread pool stop next ... next create thread pool create thread pool destroy thread pool read macrobatch file decode decode decode macrobatch buffers minibatch buffers (pinned) raw file buffers Control Codes Dual issue instr. Fused fp32 multiply add Load from shared Barrier sync Set barrier
  21. 21. Proprietary and confidential. Do not distribute. 21 Running locally: % python rnn.py # or neon rnn.yaml Running in nervana cloud: % ncloud submit rnn.py # or rnn.yaml % ncloud show <model_id> % ncloud list % ncloud deploy <model_id> % ncloud predict <model_id> <data> # or use REST api
  22. 22. 22 HANDS ON! • Option 1: Interactive ipython notebook in your browser without installing anything (you should have paper slip with your URL and password) • Option 2: Use your linux or mac laptop to download neon and try our examples. Get the notebooks and data from meetup.nervanasys.com
  23. 23. Contact 23 urs@nervanasys.com github.com/NervanaSystems/neon

×