Intel Nervana, profile picture
Uploaded byIntel Nervana
PDF, PPTX1,462 views

Introduction to Deep Learning and neon at Galvanize

The document provides an introduction to deep learning and the Nervana framework. It discusses the speaker's background and Intel's Artificial Intelligence Products Group. It then covers machine learning concepts, a brief history of deep learning, neural network architectures, training procedures, and examples of computer vision applications for deep learning like image classification. Use cases for recurrent neural networks and long short-term memory networks are also mentioned.

Technology◦
Related topics:
Deep Learning•

Embed presentation

Download as PDF, PPTX
Proprietary and confidential. Do not distribute. Introduction to Deep Learning and Neon MAKING MACHINES SMARTER.™ Kyle H. Ambert, PhD
 Senior Data Scientist May 25 , 2017th @TheKyleAmbert
Nervana Systems Proprietary About me & Intel’s Artificial Intelligence Products Group (AIPG) +
Nervana Systems Proprietary About me & Intel’s Artificial Intelligence Products Group (AIPG) +
Nervana Systems Proprietary About me & Intel’s Artificial Intelligence Products Group (AIPG) +
Nervana Systems Proprietary About me & Intel’s Artificial Intelligence Products Group (AIPG) +
Nervana Systems Proprietary About me & Intel’s Artificial Intelligence Products Group (AIPG) +
Nervana Systems Proprietary About me & Intel’s Artificial Intelligence Products Group (AIPG) + Together, we create production deep learning solutions in multiple domains, while advancing the field of applied analytics and optimization.
Nervana Systems Proprietary 8 Intel’s Interest in Analytics To provide the infrastructure for the fastest time-to-insight To create tools that enable scientists to think about their research, rather than their process To enable users to ask bigger questions Bigger Data Better Hardware Smarter Algorithms Image: 1000 KB / picture Audio: 5000 KB / song Video: 5,000,000 KB / movie Transistor density doubles every 18 months Cost / GB in 1995: $1000.00 Cost / GB in 2015: $0.03 Advances in neural networks leading to better accuracy in training models Great solutions require great hardware!
Nervana Systems Proprietary LIBRARIES Intel® MKL Intel® MKL-DNN FRAMEWORKS Intel® DAAL HARDWARE Memory/Storage FabricCompute Intel Distribution MORE UNLEASHING POTENTIAL FULL SOLUTIONS PLATFORMS/TOOLS BIGDL Intel® Nervana™ Deep Learning Platform Intel® Nervana™ Cloud Intel® Nervana™ Graph
Nervana Systems Proprietary 10 This Evening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
Nervana Systems Proprietary 11 This Evening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
Nervana Systems Proprietary 12 AI? Machine Learning? Deep Learning?
Machine learning is the development, and application of, algorithms that can learn from data in an automated, semi-automated, or supervised setting. Deep LearningStatistical Learning Algorithms where multiple layers of neurons learn successively complex representations of input data CNN RNN DFF RBM LSTM Algorithms which leverage statistical methods for estimating functions from examples Naïve Bayes SVM GLM Tree- based kNN Training: building a mathematical model based on input data Classification (scoring): using a trained model to make predictions about new data
Machine learning is the development, and application of, algorithms that can learn from data in an automated, semi-automated, or supervised setting. Deep LearningStatistical Learning Algorithms where multiple layers of neurons learn successively complex representations of input data CNN RNN DFF RBM LSTM Algorithms which leverage statistical methods for estimating functions from examples Naïve Bayes SVM GLM Tree- based kNN Training: building a mathematical model based on input data Classification (scoring): using a trained model to make predictions about new data
Machine learning is the development, and application of, algorithms that can learn from data in an automated, semi-automated, or supervised setting. Deep LearningStatistical Learning Algorithms where multiple layers of neurons learn successively complex representations of input data CNN RNN DFF RBM LSTM Algorithms which leverage statistical methods for estimating functions from examples Naïve Bayes SVM GLM Tree- based kNN Training: building a mathematical model based on input data Classification (scoring): using a trained model to make predictions about new data Ingest Data Engineer
Features Structure
 Model Clean Data Visualize Query/ Analyze TrainM odel Deploy
Nervana Systems Proprietary 16 This Evening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
Nervana Systems Proprietary 17 A Quite Brief History of Deep Learning • 1960s: Neural networks used for binary classification • 1970s: Neural networks popularity dries after not delivering on the hype • 1980s: Backpropagation is used to train deep networks • 1990s: Neural networks take the back seat to support vector machines due to the nice theoretical properties and guarantee bounds • 2010s: Access to large datasets and more computation allowed deep networks to return and have state-of-the-art results in speech, vision, and natural language processing • 1949: The Organization of Behavior is published (Hebb!) (Minsky) Today: Deep Learning is a fast-moving area of academic and applied analytics! There are many opportunities for new discoveries! (Vapnik) (Hinton)
Nervana Systems Proprietary 18 ML v. DL: Practical Differences   SVM Random Forest Naïve Bayes Decision Trees Logistic Regression Ensemble methods     Harrison
Nervana Systems Proprietary 19 End-to-End Deep learning ~60 million parameters Harrison  
Nervana Systems Proprietary 20 Workflows in Machine Learning ⟹ The same rules apply for deep learning! ➝ Preprocessing data ➝ Feature extraction ➝ Parsimony in model selection ⟹ How we go about some of this does change…
Nervana Systems Proprietary 21 End-to-End Deep learning: Data Considerations
Nervana Systems Proprietary 22 End-to-End Deep learning: Data Considerations
Nervana Systems Proprietary 23 End-to-End Deep learning: Data Considerations X X X XX X Labels: Harrison? Transformations! More data is always better!
Nervana Systems Proprietary Deep Learning: Networks of Artificial Neurons       Output of unit Activation Function Linear weights Bias unit Input from unit j                  ⟹ With an explosion of moving parts, being able to understand and keep track of what sort of model is being built becomes even more important!
Nervana Systems Proprietary Practical example: recognition of handwritten digits MNIST dataset 70,000 images (28x28 pixels) Goal: classify images into a digit 0-9 N = 28 x 28 pixels = 784 input units N = 10 output units (one for each digit) Each unit i encodes the probability of the input image of being of the digit i N = 100 hidden units (user-defined parameter) Input Hidden Output
Nervana Systems Proprietary Training procedure Input Hidden Output 1. Randomly seed weights 2. Forward-pass 3. Cost 4. Backward-pass 5. Update weights
Nervana Systems Proprietary Forward pass 0.0 0.1 0.0 0.3 0.1 0.1 0.0 0.0 0.4 0.0 Output (10x1) Input Hidden Output 28x28
Nervana Systems Proprietary Cost 0.0 0.1 0.0 0.3 0.1 0.1 0.0 0.0 0.4 0.0 Output (10x1) 28x28 Input Hidden Output 0 0 0 1 0 0 0 0 0 0 Ground Truth Cost function  
Nervana Systems Proprietary Backward pass 0.0 0.1 0.0 0.3 0.1 0.1 0.0 0.0 0.4 0.0 Output (10x1) Input Hidden Output 0 0 0 1 0 0 0 0 0 0 Ground Truth Cost function    ∆Wi→j
Nervana Systems Proprietary Back-propagation Input Hidden Output   compute
Nervana Systems Proprietary Back-propagation Input Hidden Output    
Nervana Systems Proprietary Back-propagation Input Hidden Output     =       a ! = max ((,0) a !′(()
Nervana Systems Proprietary Back-propagation Input Hidden Output        
Nervana Systems Proprietary Training fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop  
Nervana Systems Proprietary Gradient descent fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   Update weights via:   Learning rate
Nervana Systems Proprietary Stochastic (minibatch) Gradient descent fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   minibatch #1 weight update minibatch #2 weight update
Nervana Systems Proprietary Stochastic (minibatch) Gradient descent Epoch 0 Epoch 1 Sample numbers: • Learning rate ~0.001 • Batch sizes of 32-128 • 50-90 epochs
Nervana Systems Proprietary Why Does This Work at All? Krizhevsky, 2012 60 million parameters 120 million parameters Taigman, 2014
Nervana Systems Proprietary 39 This Evening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
Nervana Systems Proprietary Nervana in 30 seconds. Possibly less. 40 neon deep learning framework train deployexplore nervana engine 2-3x speedup on Titan X GPUs cloudn
Nervana Systems Proprietary neon framework
Nervana Systems Proprietary nervana cloud Web Interface Command Line
Nervana Systems Proprietary 43 This Evening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
Nervana Systems Proprietary Ge(i)t Neon! 1. git clone https://github.com/NervanaSystems/neon.git 2. pip install {h5py, pyaml, virtualenv} 3. brew install {opencv|opencv3} 4. make {python2|python3} 5. . .venv/bin/activate 6. examples/mnist_mlp.py 7. deactivate ⟹ https://goo.gl/jZgfNg Documentation!
Nervana Systems Proprietary Deep learning ingredients Dataset Model/Layers Activation OptimizerCost  
Nervana Systems Proprietary neon overview Backend NervanaGPU, NervanaCPU, NervanaMGPU Datasets MNIST, CIFAR-10, Imagenet 1K, PASCAL VOC, Mini-Places2, IMDB, Penn Treebank, Shakespeare Text, bAbI, Hutter-prize, UCF101, flickr8k, flickr30k, COCO Initializers Constant, Uniform, Gaussian, Glorot Uniform, Xavier, Kaiming, IdentityInit, Orthonormal Optimizers Gradient Descent with Momentum, RMSProp, AdaDelta, Adam, Adagrad,MultiOptimizer Activations Rectified Linear, Softmax, Tanh, Logistic, Identity, ExpLin Layers Linear, Convolution, Pooling, Deconvolution, Dropout, Recurrent,Long Short- Term Memory, Gated Recurrent Unit, BatchNorm, LookupTable,Local Response Normalizat ion, Bidirectional-RNN, Bidirectional-LSTM Costs Binary Cross Entropy, Multiclass Cross Entropy, Sum of Squares Error Metrics Misclassification (Top1, TopK), LogLoss, Accuracy, PrecisionRecall, ObjectDetection
Nervana Systems Proprietary Curated Models 47 • https://github.com/NervanaSystems/ModelZoo • Pre-trained weights and models SegNet Deep Speech 2 Skip-thought Autoencoders Deep Dream
Nervana Systems Proprietary Neon workflow 1. Generate backend 2. Load data 3. Specify model architecture 4. Define training parameters 5. Train model 6. Evaluate
Nervana Systems Proprietary Interacting with Neon 1. Via command line 2. In a virtual environment 3. In an ipython/jupyter notebook 4. ncloud
Nervana Systems Proprietary Nervana Cloud
Nervana Systems Proprietary Nervana Cloud
Nervana Systems Proprietary Nervana Cloud
Nervana Systems Proprietary 53 This Evening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
Nervana Systems Proprietary 54
Nervana Systems Proprietary
Nervana Systems Proprietary •Layers: convolution, rectified linear units, pooling, dropout, softmax •Popular with 2D + depth (+ time) inputs •Gray or RBG images •Videos •Synthetic aperture radar •Spectrogram (speech)
Nervana Systems Proprietary •Layers: convolution, rectified linear units, pooling, dropout, softmax •Use multiple copies of the same feature on the input (correlation) •Use several features (aka kernels, filters) •Reduces number of weights compared to fully connected
Nervana Systems Proprietary •Layers: convolution, rectified linear units (ReLu), pooling, dropout, softmax •It is fast – no normalization or exponential computations •Induces sparsity in the hidden units  
Nervana Systems Proprietary •Layers: convolution, rectified linear units, pooling, dropout, softmax •Downsampling •Reduces the number of parameters •Provides some translation invariance
Nervana Systems Proprietary •Layers: convolution, rectified linear units, pooling, dropout, softmax •Reduces overfitting – Prevents co-adaptation on training data
Nervana Systems Proprietary •Layers: convolution, rectified linear units, pooling, dropout, softmax •aka “normalized exponential function” •Normalizes vector to a probability distribution 
Nervana Systems Proprietary Code!
Nervana Systems Proprietary 63 DEEP LEARNING USE CASES! Long Short-Term Memory (LSTM)
Nervana Systems Proprietary Why Recurrent Neural Networks? Input Hidden Output • Temporal dependencies • Variable sequence length • Independence • Fixed Length
Nervana Systems Proprietary Recurrent neuron                
Nervana Systems Proprietary RNN: what is it good for? 0.1 -0.4 0.6 1 0 0 0 0.1 0.7 0.1 0.1 -0.3 0.6 1.6 1 0 0 0 0.1 0.3 0.4 0.2 0.7 -0.4 -0.4 1 0 0 0 0.3 0.0 0.6 0.1 0.1 -0.8 0.1 1 0 0 0 0.0 0.0 0.2 0.8 “h” “e” “l” “l” “e” “l” “l” “o”   Learned a language model!
Nervana Systems Proprietary RNN: what is it good for? 0.1 -0.4 0.6 1 0 0 0 0.1 0.7 0.1 0.1 -0.3 0.6 1.6 1 0 0 0 0.1 0.3 0.4 0.2 0.7 -0.4 -0.4 1 0 0 0 0.4 0.0 0.5 0.1 0.1 -0.8 0.1 1 0 0 0 0.0 0.0 0.2 0.8 “cash” “flow” “is” “high” “flow” “is” “high” “today”   Learned a language model! “low” “high”
Nervana Systems Proprietary RNN: what is it good for? 0.1 -0.4 0.6 1 0 0 0 -0.3 0.6 1.6 0 1 0 0 0.7 -0.4 -0.4 0 0 1 0 0.1 -0.8 0.1 0 0 0 1 “this” “movie” “was” “bad” NEGATIVE “and” “long” <eos> 0.1 -0.8 0.1 1 0 0 0 0.7 -0.4 -0.4 1 0 0 0 -0.3 0.6 1.6 0 1 0 0 0.2 0.8
Nervana Systems Proprietary RNN: what is it good for? 0.1 -0.4 0.6 1 0 0 0 -0.3 0.6 1.6 0 1 0 0 0.7 -0.4 -0.4 0 0 1 0 0.1 -0.8 0.1 “neon” “is” “amazing” 0.1 -0.8 0.1 0.7 -0.4 -0.4 -0.3 0.6 1.6 0.1 0.7 0.1 0.1 0.1 0.3 0.4 0.2 0.3 0.0 0.6 0.1 0.0 0.0 0.2 0.8 “neon” “est” “incroyable” “!” 0.1 -0.4 0.6 1 0 0 0 -0.3 0.6 1.6 0 1 0 0 0.7 -0.4 -0.4 0 0 1 0 0.1 -0.8 0.1 “neon” “is” “amazing” 0.1 -0.8 0.1 0.7 -0.4 -0.4 -0.3 0.6 1.6 0.1 0.7 0.1 0.1 0.1 0.3 0.4 0.2 0.3 0.0 0.6 0.1 0.0 0.0 0.2 0.8 “neon”“est”“incroyable”“!”
Nervana Systems Proprietary Long-Short Term Memory (LSTM)           1 1   1 Manipulate memory cell: 1. “forget” (flush the memory) 2. “input” (add to memory) 3. “output” (get from memory)
Nervana Systems Proprietary Example – Sentiment analysis with LSTM “Okay, sorry, but I loved this movie. I just love the whole 80’s genre of these kind of movies, because you don’t see many like this...” -~CupidGrl~ POSITIVE The plot/writing is completely unrealistic and just dumb at times. Bond is dressed up in a white tux on an overnight train ride? eh, OK. But then they just show up at the villain’s compound like nothing bad is going to happen to them. How stupid is this Bond? NEGATIVE
Nervana Systems Proprietary Preprocessing “Okay, sorry, but I loved this movie. I just love the whole 80’s genre of these kind of movies, because you don’t see many like this...” -~CupidGrl~ [5, 4, 940, 107, 14, 672, 1790, 333, 47, 11, 7890, …,1] Out-of-Vocab (e.g. CupidGrl) • Limit vocab size to 20,000 words • Truncate each example to 128 words [from the left] • Pad examples up to 128 whitespace
Nervana Systems Proprietary Model d=128 embedding layer LSTM LSTM LSTM LSTM N=2 [5, 4, 940, 107, 14, 672, 1790, 333, 47, 11, 7890, …,1]   POS NEG N=64 LSTM AffineRecurrentSum  
Nervana Systems Proprietary Data flow d=128 embedding layer LSTM (2, 1) POS NEG LSTM Affine      LSTM LSTM LSTM         RecurrentSum     n=64
Nervana Systems Proprietary Data flow in batches with neon d=128 embedding layer LSTM (2, bsz) [5, 4, 940, 107, 14, 672, 1790, 333, 47, 11, 7890,…, 1]   POS NEG LSTM Affine        LSTM LSTM LSTM         RecurrentSum     n=64
Nervana Systems Proprietary Code! LSTM
Nervana Systems Proprietary More Code! LSTM
Nervana Systems Proprietary In Summary… 1. Deep learning methods are powerful and versatile 2. It’s important to understand how DL relates to traditional ML methods 3. The barrier of entry to using DL in practice is lowered with the neon framework on the Nervana ecosystem kyle.h.ambert@intel.com @TheKyleAmbert

More Related Content

PDF
Startup.Ml: Using neon for NLP and Localization Applications
byIntel Nervana
 
PPTX
Deep Learning for Robotics
byIntel Nervana
 
PDF
Rethinking computation: A processor architecture for machine intelligence
byIntel Nervana
 
PDF
Introduction to deep learning @ Startup.ML by Andres Rodriguez
byIntel Nervana
 
PDF
Introduction to Deep Learning with Will Constable
byIntel Nervana
 
PDF
Deep Learning at Scale
byIntel Nervana
 
PPTX
Intel Nervana Artificial Intelligence Meetup 11/30/16
byIntel Nervana
 
PDF
ODSC West
byIntel Nervana
 
Startup.Ml: Using neon for NLP and Localization Applications
byIntel Nervana
 
Deep Learning for Robotics
byIntel Nervana
 
Rethinking computation: A processor architecture for machine intelligence
byIntel Nervana
 
Introduction to deep learning @ Startup.ML by Andres Rodriguez
byIntel Nervana
 
Introduction to Deep Learning with Will Constable
byIntel Nervana
 
Deep Learning at Scale
byIntel Nervana
 
Intel Nervana Artificial Intelligence Meetup 11/30/16
byIntel Nervana
 
ODSC West
byIntel Nervana
 

What's hot

PDF
Intel Nervana Artificial Intelligence Meetup 1/31/17
byIntel Nervana
 
PPTX
Deep Learning Made Easy with Deep Features
byTuri, Inc.
 
PPTX
Squeezing Deep Learning Into Mobile Phones
byAnirudh Koul
 
PPTX
Amazon Deep Learning
byAmanda Mackay (she/her)
 
PDF
Large Scale Deep Learning with TensorFlow
byJen Aman
 
PDF
Urs Köster Presenting at RE-Work DL Summit in Boston
byIntel Nervana
 
PPTX
Deep learning on mobile
byAnirudh Koul
 
PDF
Deep Learning as a Cat/Dog Detector
byRoelof Pieters
 
PDF
Urs Köster - Convolutional and Recurrent Neural Networks
byIntel Nervana
 
PDF
Recent developments in Deep Learning
byBrahim HAMADICHAREF
 
PPTX
Deep Learning Frameworks Using Spark on YARN by Vartika Singh
byData Con LA
 
PDF
NVIDIA 深度學習教育機構 (DLI): Approaches to object detection
byNVIDIA Taiwan
 
PDF
Language translation with Deep Learning (RNN) with TensorFlow
byS N
 
PPTX
AI powered emotion recognition: From Inception to Production - Global AI Conf...
byApache MXNet
 
PPTX
Android and Deep Learning
byOswald Campesato
 
PDF
Improving Hardware Efficiency for DNN Applications
byChester Chen
 
PDF
Deep Learning Primer: A First-Principles Approach
byMaurizio Calo Caligaris
 
PDF
Nervana and the Future of Computing
byIntel Nervana
 
PPTX
Deep learning on mobile - 2019 Practitioner's Guide
byAnirudh Koul
 
Intel Nervana Artificial Intelligence Meetup 1/31/17
byIntel Nervana
 
Deep Learning Made Easy with Deep Features
byTuri, Inc.
 
Squeezing Deep Learning Into Mobile Phones
byAnirudh Koul
 
Amazon Deep Learning
byAmanda Mackay (she/her)
 
Large Scale Deep Learning with TensorFlow
byJen Aman
 
Urs Köster Presenting at RE-Work DL Summit in Boston
byIntel Nervana
 
Deep learning on mobile
byAnirudh Koul
 
Deep Learning as a Cat/Dog Detector
byRoelof Pieters
 
Urs Köster - Convolutional and Recurrent Neural Networks
byIntel Nervana
 
Recent developments in Deep Learning
byBrahim HAMADICHAREF
 
Deep Learning Frameworks Using Spark on YARN by Vartika Singh
byData Con LA
 
NVIDIA 深度學習教育機構 (DLI): Approaches to object detection
byNVIDIA Taiwan
 
Language translation with Deep Learning (RNN) with TensorFlow
byS N
 
AI powered emotion recognition: From Inception to Production - Global AI Conf...
byApache MXNet
 
Android and Deep Learning
byOswald Campesato
 
Improving Hardware Efficiency for DNN Applications
byChester Chen
 
Deep Learning Primer: A First-Principles Approach
byMaurizio Calo Caligaris
 
Nervana and the Future of Computing
byIntel Nervana
 
Deep learning on mobile - 2019 Practitioner's Guide
byAnirudh Koul
 

Viewers also liked

PPTX
Text categorization
byKU Leuven
 
PPTX
Sentiment analysis using naive bayes classifier
byDev Sahu
 
PPT
Text categorization
byPhuong Nguyen
 
PDF
A comparison of Lexicon-based approaches for Sentiment Analysis of microblog ...
byCataldo Musto
 
PPTX
Rule based approach to sentiment analysis at romip’11 slides
byDmitry Kan
 
PDF
(Deep) Neural Networks在 NLP 和 Text Mining 总结
by君 廖
 
PPTX
Tutorial on Opinion Mining and Sentiment Analysis
byYun Hao
 
PDF
CS571: Gradient Descent
byJinho Choi
 
PDF
CS571: Sentiment Analysis
byJinho Choi
 
Text categorization
byKU Leuven
 
Sentiment analysis using naive bayes classifier
byDev Sahu
 
Text categorization
byPhuong Nguyen
 
A comparison of Lexicon-based approaches for Sentiment Analysis of microblog ...
byCataldo Musto
 
Rule based approach to sentiment analysis at romip’11 slides
byDmitry Kan
 
(Deep) Neural Networks在 NLP 和 Text Mining 总结
by君 廖
 
Tutorial on Opinion Mining and Sentiment Analysis
byYun Hao
 
CS571: Gradient Descent
byJinho Choi
 
CS571: Sentiment Analysis
byJinho Choi
 

Similar to Introduction to Deep Learning and neon at Galvanize

PDF
Deep Learning - The Past, Present and Future of Artificial Intelligence
byLukas Masuch
 
PPTX
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
bySimplilearn
 
PPTX
Deep Learning Tutorial
byAmr Rashed
 
PPTX
Deep learning tutorial 9/2019
byAmr Rashed
 
PDF
Introduction to Deep Learning: Concepts, Architectures, and Applications
byAmr Rashed
 
PDF
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
byPoo Kuan Hoong
 
PPTX
Session_2_Introduction_to_Deep_Learning.pptx
byaljeboorymuhammed
 
PPTX
Build a Neural Network for ITSM with TensorFlow
byEntrepreneur / Startup
 
PPTX
A simple presentation for deep learning.
bymahfuzur32785
 
PPTX
Computer Design Concepts for Machine Learning
byFacultad de Informática UCM
 
PPT
DEEP LEARNING PPT aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
byRRamya22
 
PDF
State-Of-The Art Machine Learning Algorithms and How They Are Affected By Nea...
byinside-BigData.com
 
PDF
An Introduction to Deep Learning
byPoo Kuan Hoong
 
PDF
Deep Learning Demystified
byAffine Analytics
 
PDF
Deep Learning & Tensor flow: An Intro
bySiby Jose Plathottam
 
PPTX
Introduction to Deep learning
byMassimiliano Patacchiola
 
PDF
Big Data Malaysia - A Primer on Deep Learning
byPoo Kuan Hoong
 
PDF
Practical deepllearningv1
byarthi v
 
PPTX
Deep Learning Jump Start
byMichele Toni
 
PPTX
Rise of AI through DL
byRehan Guha
 
Deep Learning - The Past, Present and Future of Artificial Intelligence
byLukas Masuch
 
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
bySimplilearn
 
Deep Learning Tutorial
byAmr Rashed
 
Deep learning tutorial 9/2019
byAmr Rashed
 
Introduction to Deep Learning: Concepts, Architectures, and Applications
byAmr Rashed
 
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
byPoo Kuan Hoong
 
Session_2_Introduction_to_Deep_Learning.pptx
byaljeboorymuhammed
 
Build a Neural Network for ITSM with TensorFlow
byEntrepreneur / Startup
 
A simple presentation for deep learning.
bymahfuzur32785
 
Computer Design Concepts for Machine Learning
byFacultad de Informática UCM
 
DEEP LEARNING PPT aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
byRRamya22
 
State-Of-The Art Machine Learning Algorithms and How They Are Affected By Nea...
byinside-BigData.com
 
An Introduction to Deep Learning
byPoo Kuan Hoong
 
Deep Learning Demystified
byAffine Analytics
 
Deep Learning & Tensor flow: An Intro
bySiby Jose Plathottam
 
Introduction to Deep learning
byMassimiliano Patacchiola
 
Big Data Malaysia - A Primer on Deep Learning
byPoo Kuan Hoong
 
Practical deepllearningv1
byarthi v
 
Deep Learning Jump Start
byMichele Toni
 
Rise of AI through DL
byRehan Guha
 

More from Intel Nervana

PDF
Using neon for pattern recognition in audio data
byIntel Nervana
 
PDF
Object Detection and Recognition
byIntel Nervana
 
PDF
Women in AI kickoff
byIntel Nervana
 
PDF
An Analysis of Convolution for Inference
byIntel Nervana
 
PDF
RE-Work Deep Learning Summit - September 2016
byIntel Nervana
 
PDF
Anil Thomas - Object recognition
byIntel Nervana
 
PDF
Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...
byIntel Nervana
 
PDF
High-Performance GPU Programming for Deep Learning
byIntel Nervana
 
PDF
Video Activity Recognition and NLP Q&A Model Example
byIntel Nervana
 
Using neon for pattern recognition in audio data
byIntel Nervana
 
Object Detection and Recognition
byIntel Nervana
 
Women in AI kickoff
byIntel Nervana
 
An Analysis of Convolution for Inference
byIntel Nervana
 
RE-Work Deep Learning Summit - September 2016
byIntel Nervana
 
Anil Thomas - Object recognition
byIntel Nervana
 
Andres Rodriguez at AI Frontiers: Catalyzing Deep Learning's Impact in the En...
byIntel Nervana
 
High-Performance GPU Programming for Deep Learning
byIntel Nervana
 
Video Activity Recognition and NLP Q&A Model Example
byIntel Nervana
 

Recently uploaded

PDF
Digit Expo 2025 - EICC Edinburgh 27th November
byRay Bugg
 
PPTX
Combining Induction and Transduction for Abstract Reasoning.pptx
byallen578468
 
PDF
Incident Response Planning with a Foundation Model
byKim Hammar
 
PDF
Agentic Automation for Testers – A Hands-On Deep Dive
byUiPathCommunity
 
PDF
TrustArc Webinar - Looking Ahead: The 2026 Privacy Landscape
byTrustArc
 
PDF
Session 1 - Solving Semi-Structured Documents with Document Understanding
byDianaGray10
 
PPTX
THIS IS CYBER SECURITY NOTES USED IN CLASS ON VARIOUS TOPICS USED IN CYBERSEC...
byMelissaGblinwon
 
PDF
Guided Substation Engineering with CoMPAS: Simplifying IEC 61850
byDanBrown980551
 
PDF
The major tech developments for 2026 by Pluralsight, a research and training ...
byChris Skinner
 
PPTX
Basics of Identity Access Management In mordern Infrastructure
byPrinceXavier18
 
PPTX
The Landscape of Computer Software Development Companies
byYash Singh
 
PPTX
Ethics in AI - Artificial Intelligence Fundamentals.pptx
byemenyiblessing
 
PDF
Igniting the Future: Copilot trends, agentic transformation and product roadm...
byUni Systems S.M.S.A.
 
PDF
AI and Computer Architecture: 200 Years Together
byDmitry Zinoviev
 
PDF
Generative AI in 2026: Hype, Bubble, Winter or The Real Deal?
byDr. Tathagat Varma
 
PDF
How Mobile Apps Are Shaping the Future of Digital Innovation
byWilliam Taylor
 
PDF
TPPmark2025 Kenta Inoue's answer 12/04/2025
byKenta Inoue
 
PDF
Top Cybersecurity Threats 2025 Guide by Sureshdas
byPrintersassist
 
PDF
Ransomware_Resilience_Strategic_Playbook.pdf
byAfonso Henrique Rodrigues Alves
 
PDF
8 LLM Surveys https://tinyurl.com/bdz8e6fp
byBob Marcus
 
Digit Expo 2025 - EICC Edinburgh 27th November
byRay Bugg
 
Combining Induction and Transduction for Abstract Reasoning.pptx
byallen578468
 
Incident Response Planning with a Foundation Model
byKim Hammar
 
Agentic Automation for Testers – A Hands-On Deep Dive
byUiPathCommunity
 
TrustArc Webinar - Looking Ahead: The 2026 Privacy Landscape
byTrustArc
 
Session 1 - Solving Semi-Structured Documents with Document Understanding
byDianaGray10
 
THIS IS CYBER SECURITY NOTES USED IN CLASS ON VARIOUS TOPICS USED IN CYBERSEC...
byMelissaGblinwon
 
Guided Substation Engineering with CoMPAS: Simplifying IEC 61850
byDanBrown980551
 
The major tech developments for 2026 by Pluralsight, a research and training ...
byChris Skinner
 
Basics of Identity Access Management In mordern Infrastructure
byPrinceXavier18
 
The Landscape of Computer Software Development Companies
byYash Singh
 
Ethics in AI - Artificial Intelligence Fundamentals.pptx
byemenyiblessing
 
Igniting the Future: Copilot trends, agentic transformation and product roadm...
byUni Systems S.M.S.A.
 
AI and Computer Architecture: 200 Years Together
byDmitry Zinoviev
 
Generative AI in 2026: Hype, Bubble, Winter or The Real Deal?
byDr. Tathagat Varma
 
How Mobile Apps Are Shaping the Future of Digital Innovation
byWilliam Taylor
 
TPPmark2025 Kenta Inoue's answer 12/04/2025
byKenta Inoue
 
Top Cybersecurity Threats 2025 Guide by Sureshdas
byPrintersassist
 
Ransomware_Resilience_Strategic_Playbook.pdf
byAfonso Henrique Rodrigues Alves
 
8 LLM Surveys https://tinyurl.com/bdz8e6fp
byBob Marcus
 

Introduction to Deep Learning and neon at Galvanize

  • 1.
    Proprietary and confidential.Do not distribute. Introduction to Deep Learning and Neon MAKING MACHINES SMARTER.™ Kyle H. Ambert, PhD
 Senior Data Scientist May 25 , 2017th @TheKyleAmbert
  • 2.
    Nervana Systems Proprietary Aboutme & Intel’s Artificial Intelligence Products Group (AIPG) +
  • 3.
    Nervana Systems Proprietary Aboutme & Intel’s Artificial Intelligence Products Group (AIPG) +
  • 4.
    Nervana Systems Proprietary Aboutme & Intel’s Artificial Intelligence Products Group (AIPG) +
  • 5.
    Nervana Systems Proprietary Aboutme & Intel’s Artificial Intelligence Products Group (AIPG) +
  • 6.
    Nervana Systems Proprietary Aboutme & Intel’s Artificial Intelligence Products Group (AIPG) +
  • 7.
    Nervana Systems Proprietary Aboutme & Intel’s Artificial Intelligence Products Group (AIPG) + Together, we create production deep learning solutions in multiple domains, while advancing the field of applied analytics and optimization.
  • 8.
    Nervana Systems Proprietary 8 Intel’sInterest in Analytics To provide the infrastructure for the fastest time-to-insight To create tools that enable scientists to think about their research, rather than their process To enable users to ask bigger questions Bigger Data Better Hardware Smarter Algorithms Image: 1000 KB / picture Audio: 5000 KB / song Video: 5,000,000 KB / movie Transistor density doubles every 18 months Cost / GB in 1995: $1000.00 Cost / GB in 2015: $0.03 Advances in neural networks leading to better accuracy in training models Great solutions require great hardware!
  • 9.
    Nervana Systems Proprietary LIBRARIESIntel® MKL Intel® MKL-DNN FRAMEWORKS Intel® DAAL HARDWARE Memory/Storage FabricCompute Intel Distribution MORE UNLEASHING POTENTIAL FULL SOLUTIONS PLATFORMS/TOOLS BIGDL Intel® Nervana™ Deep Learning Platform Intel® Nervana™ Cloud Intel® Nervana™ Graph
  • 10.
    Nervana Systems Proprietary 10 ThisEvening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
  • 11.
    Nervana Systems Proprietary 11 ThisEvening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
  • 12.
    Nervana Systems Proprietary 12 AI?Machine Learning? Deep Learning?
  • 13.
    Machine learning isthe development, and application of, algorithms that can learn from data in an automated, semi-automated, or supervised setting. Deep LearningStatistical Learning Algorithms where multiple layers of neurons learn successively complex representations of input data CNN RNN DFF RBM LSTM Algorithms which leverage statistical methods for estimating functions from examples Naïve Bayes SVM GLM Tree- based kNN Training: building a mathematical model based on input data Classification (scoring): using a trained model to make predictions about new data
  • 14.
    Machine learning isthe development, and application of, algorithms that can learn from data in an automated, semi-automated, or supervised setting. Deep LearningStatistical Learning Algorithms where multiple layers of neurons learn successively complex representations of input data CNN RNN DFF RBM LSTM Algorithms which leverage statistical methods for estimating functions from examples Naïve Bayes SVM GLM Tree- based kNN Training: building a mathematical model based on input data Classification (scoring): using a trained model to make predictions about new data
  • 15.
    Machine learning isthe development, and application of, algorithms that can learn from data in an automated, semi-automated, or supervised setting. Deep LearningStatistical Learning Algorithms where multiple layers of neurons learn successively complex representations of input data CNN RNN DFF RBM LSTM Algorithms which leverage statistical methods for estimating functions from examples Naïve Bayes SVM GLM Tree- based kNN Training: building a mathematical model based on input data Classification (scoring): using a trained model to make predictions about new data Ingest Data Engineer
Features Structure
 Model Clean Data Visualize Query/ Analyze TrainM odel Deploy
  • 16.
    Nervana Systems Proprietary 16 ThisEvening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
  • 17.
    Nervana Systems Proprietary 17 AQuite Brief History of Deep Learning • 1960s: Neural networks used for binary classification • 1970s: Neural networks popularity dries after not delivering on the hype • 1980s: Backpropagation is used to train deep networks • 1990s: Neural networks take the back seat to support vector machines due to the nice theoretical properties and guarantee bounds • 2010s: Access to large datasets and more computation allowed deep networks to return and have state-of-the-art results in speech, vision, and natural language processing • 1949: The Organization of Behavior is published (Hebb!) (Minsky) Today: Deep Learning is a fast-moving area of academic and applied analytics! There are many opportunities for new discoveries! (Vapnik) (Hinton)
  • 18.
    Nervana Systems Proprietary 18 MLv. DL: Practical Differences   SVM Random Forest Naïve Bayes Decision Trees Logistic Regression Ensemble methods     Harrison
  • 19.
    Nervana Systems Proprietary 19 End-to-EndDeep learning ~60 million parameters Harrison  
  • 20.
    Nervana Systems Proprietary 20 Workflowsin Machine Learning ⟹ The same rules apply for deep learning! ➝ Preprocessing data ➝ Feature extraction ➝ Parsimony in model selection ⟹ How we go about some of this does change…
  • 21.
    Nervana Systems Proprietary 21 End-to-EndDeep learning: Data Considerations
  • 22.
    Nervana Systems Proprietary 22 End-to-EndDeep learning: Data Considerations
  • 23.
    Nervana Systems Proprietary 23 End-to-EndDeep learning: Data Considerations X X X XX X Labels: Harrison? Transformations! More data is always better!
  • 24.
    Nervana Systems Proprietary DeepLearning: Networks of Artificial Neurons       Output of unit Activation Function Linear weights Bias unit Input from unit j                  ⟹ With an explosion of moving parts, being able to understand and keep track of what sort of model is being built becomes even more important!
  • 25.
    Nervana Systems Proprietary Practicalexample: recognition of handwritten digits MNIST dataset 70,000 images (28x28 pixels) Goal: classify images into a digit 0-9 N = 28 x 28 pixels = 784 input units N = 10 output units (one for each digit) Each unit i encodes the probability of the input image of being of the digit i N = 100 hidden units (user-defined parameter) Input Hidden Output
  • 26.
    Nervana Systems Proprietary Trainingprocedure Input Hidden Output 1. Randomly seed weights 2. Forward-pass 3. Cost 4. Backward-pass 5. Update weights
  • 27.
    Nervana Systems Proprietary Forwardpass 0.0 0.1 0.0 0.3 0.1 0.1 0.0 0.0 0.4 0.0 Output (10x1) Input Hidden Output 28x28
  • 28.
    Nervana Systems Proprietary Cost 0.0 0.1 0.0 0.3 0.1 0.1 0.0 0.0 0.4 0.0 Output(10x1) 28x28 Input Hidden Output 0 0 0 1 0 0 0 0 0 0 Ground Truth Cost function  
  • 29.
    Nervana Systems Proprietary Backwardpass 0.0 0.1 0.0 0.3 0.1 0.1 0.0 0.0 0.4 0.0 Output (10x1) Input Hidden Output 0 0 0 1 0 0 0 0 0 0 Ground Truth Cost function    ∆Wi→j
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
    Nervana Systems Proprietary Training fpropcost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop  
  • 35.
    Nervana Systems Proprietary Gradientdescent fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   Update weights via:   Learning rate
  • 36.
    Nervana Systems Proprietary Stochastic(minibatch) Gradient descent fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   fprop cost bprop   minibatch #1 weight update minibatch #2 weight update
  • 37.
    Nervana Systems Proprietary Stochastic(minibatch) Gradient descent Epoch 0 Epoch 1 Sample numbers: • Learning rate ~0.001 • Batch sizes of 32-128 • 50-90 epochs
  • 38.
    Nervana Systems Proprietary WhyDoes This Work at All? Krizhevsky, 2012 60 million parameters 120 million parameters Taigman, 2014
  • 39.
    Nervana Systems Proprietary 39 ThisEvening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
  • 40.
    Nervana Systems Proprietary Nervanain 30 seconds. Possibly less. 40 neon deep learning framework train deployexplore nervana engine 2-3x speedup on Titan X GPUs cloudn
  • 41.
  • 42.
    Nervana Systems Proprietary nervanacloud Web Interface Command Line
  • 43.
    Nervana Systems Proprietary 43 ThisEvening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
  • 44.
    Nervana Systems Proprietary Ge(i)tNeon! 1. git clone https://github.com/NervanaSystems/neon.git 2. pip install {h5py, pyaml, virtualenv} 3. brew install {opencv|opencv3} 4. make {python2|python3} 5. . .venv/bin/activate 6. examples/mnist_mlp.py 7. deactivate ⟹ https://goo.gl/jZgfNg Documentation!
  • 45.
    Nervana Systems Proprietary Deeplearning ingredients Dataset Model/Layers Activation OptimizerCost  
  • 46.
    Nervana Systems Proprietary neonoverview Backend NervanaGPU, NervanaCPU, NervanaMGPU Datasets MNIST, CIFAR-10, Imagenet 1K, PASCAL VOC, Mini-Places2, IMDB, Penn Treebank, Shakespeare Text, bAbI, Hutter-prize, UCF101, flickr8k, flickr30k, COCO Initializers Constant, Uniform, Gaussian, Glorot Uniform, Xavier, Kaiming, IdentityInit, Orthonormal Optimizers Gradient Descent with Momentum, RMSProp, AdaDelta, Adam, Adagrad,MultiOptimizer Activations Rectified Linear, Softmax, Tanh, Logistic, Identity, ExpLin Layers Linear, Convolution, Pooling, Deconvolution, Dropout, Recurrent,Long Short- Term Memory, Gated Recurrent Unit, BatchNorm, LookupTable,Local Response Normalizat ion, Bidirectional-RNN, Bidirectional-LSTM Costs Binary Cross Entropy, Multiclass Cross Entropy, Sum of Squares Error Metrics Misclassification (Top1, TopK), LogLoss, Accuracy, PrecisionRecall, ObjectDetection
  • 47.
    Nervana Systems Proprietary CuratedModels 47 • https://github.com/NervanaSystems/ModelZoo • Pre-trained weights and models SegNet Deep Speech 2 Skip-thought Autoencoders Deep Dream
  • 48.
    Nervana Systems Proprietary Neonworkflow 1. Generate backend 2. Load data 3. Specify model architecture 4. Define training parameters 5. Train model 6. Evaluate
  • 49.
    Nervana Systems Proprietary Interactingwith Neon 1. Via command line 2. In a virtual environment 3. In an ipython/jupyter notebook 4. ncloud
  • 50.
  • 51.
  • 52.
  • 53.
    Nervana Systems Proprietary 53 ThisEvening 1. Machine Learning and Data Science 2. Introduction to Deep Learning 3. Nervana! 4. Neon 5. Deep Learning Use Cases
  • 54.
  • 55.
  • 56.
    Nervana Systems Proprietary •Layers:convolution, rectified linear units, pooling, dropout, softmax •Popular with 2D + depth (+ time) inputs •Gray or RBG images •Videos •Synthetic aperture radar •Spectrogram (speech)
  • 57.
    Nervana Systems Proprietary •Layers:convolution, rectified linear units, pooling, dropout, softmax •Use multiple copies of the same feature on the input (correlation) •Use several features (aka kernels, filters) •Reduces number of weights compared to fully connected
  • 58.
    Nervana Systems Proprietary •Layers:convolution, rectified linear units (ReLu), pooling, dropout, softmax •It is fast – no normalization or exponential computations •Induces sparsity in the hidden units  
  • 59.
    Nervana Systems Proprietary •Layers:convolution, rectified linear units, pooling, dropout, softmax •Downsampling •Reduces the number of parameters •Provides some translation invariance
  • 60.
    Nervana Systems Proprietary •Layers:convolution, rectified linear units, pooling, dropout, softmax •Reduces overfitting – Prevents co-adaptation on training data
  • 61.
    Nervana Systems Proprietary •Layers:convolution, rectified linear units, pooling, dropout, softmax •aka “normalized exponential function” •Normalizes vector to a probability distribution 
  • 62.
  • 63.
    Nervana Systems Proprietary 63 DEEPLEARNING USE CASES! Long Short-Term Memory (LSTM)
  • 64.
    Nervana Systems Proprietary WhyRecurrent Neural Networks? Input Hidden Output • Temporal dependencies • Variable sequence length • Independence • Fixed Length
  • 65.
    Nervana Systems Proprietary Recurrentneuron                
  • 66.
    Nervana Systems Proprietary RNN:what is it good for? 0.1 -0.4 0.6 1 0 0 0 0.1 0.7 0.1 0.1 -0.3 0.6 1.6 1 0 0 0 0.1 0.3 0.4 0.2 0.7 -0.4 -0.4 1 0 0 0 0.3 0.0 0.6 0.1 0.1 -0.8 0.1 1 0 0 0 0.0 0.0 0.2 0.8 “h” “e” “l” “l” “e” “l” “l” “o”   Learned a language model!
  • 67.
    Nervana Systems Proprietary RNN:what is it good for? 0.1 -0.4 0.6 1 0 0 0 0.1 0.7 0.1 0.1 -0.3 0.6 1.6 1 0 0 0 0.1 0.3 0.4 0.2 0.7 -0.4 -0.4 1 0 0 0 0.4 0.0 0.5 0.1 0.1 -0.8 0.1 1 0 0 0 0.0 0.0 0.2 0.8 “cash” “flow” “is” “high” “flow” “is” “high” “today”   Learned a language model! “low” “high”
  • 68.
    Nervana Systems Proprietary RNN:what is it good for? 0.1 -0.4 0.6 1 0 0 0 -0.3 0.6 1.6 0 1 0 0 0.7 -0.4 -0.4 0 0 1 0 0.1 -0.8 0.1 0 0 0 1 “this” “movie” “was” “bad” NEGATIVE “and” “long” <eos> 0.1 -0.8 0.1 1 0 0 0 0.7 -0.4 -0.4 1 0 0 0 -0.3 0.6 1.6 0 1 0 0 0.2 0.8
  • 69.
    Nervana Systems Proprietary RNN:what is it good for? 0.1 -0.4 0.6 1 0 0 0 -0.3 0.6 1.6 0 1 0 0 0.7 -0.4 -0.4 0 0 1 0 0.1 -0.8 0.1 “neon” “is” “amazing” 0.1 -0.8 0.1 0.7 -0.4 -0.4 -0.3 0.6 1.6 0.1 0.7 0.1 0.1 0.1 0.3 0.4 0.2 0.3 0.0 0.6 0.1 0.0 0.0 0.2 0.8 “neon” “est” “incroyable” “!” 0.1 -0.4 0.6 1 0 0 0 -0.3 0.6 1.6 0 1 0 0 0.7 -0.4 -0.4 0 0 1 0 0.1 -0.8 0.1 “neon” “is” “amazing” 0.1 -0.8 0.1 0.7 -0.4 -0.4 -0.3 0.6 1.6 0.1 0.7 0.1 0.1 0.1 0.3 0.4 0.2 0.3 0.0 0.6 0.1 0.0 0.0 0.2 0.8 “neon”“est”“incroyable”“!”
  • 70.
    Nervana Systems Proprietary Long-ShortTerm Memory (LSTM)           1 1   1 Manipulate memory cell: 1. “forget” (flush the memory) 2. “input” (add to memory) 3. “output” (get from memory)
  • 71.
    Nervana Systems Proprietary Example– Sentiment analysis with LSTM “Okay, sorry, but I loved this movie. I just love the whole 80’s genre of these kind of movies, because you don’t see many like this...” -~CupidGrl~ POSITIVE The plot/writing is completely unrealistic and just dumb at times. Bond is dressed up in a white tux on an overnight train ride? eh, OK. But then they just show up at the villain’s compound like nothing bad is going to happen to them. How stupid is this Bond? NEGATIVE
  • 72.
    Nervana Systems Proprietary Preprocessing “Okay,sorry, but I loved this movie. I just love the whole 80’s genre of these kind of movies, because you don’t see many like this...” -~CupidGrl~ [5, 4, 940, 107, 14, 672, 1790, 333, 47, 11, 7890, …,1] Out-of-Vocab (e.g. CupidGrl) • Limit vocab size to 20,000 words • Truncate each example to 128 words [from the left] • Pad examples up to 128 whitespace
  • 73.
    Nervana Systems Proprietary Model d=128 embeddinglayer LSTM LSTM LSTM LSTM N=2 [5, 4, 940, 107, 14, 672, 1790, 333, 47, 11, 7890, …,1]   POS NEG N=64 LSTM AffineRecurrentSum  
  • 74.
    Nervana Systems Proprietary Dataflow d=128 embedding layer LSTM (2, 1) POS NEG LSTM Affine      LSTM LSTM LSTM         RecurrentSum     n=64
  • 75.
    Nervana Systems Proprietary Dataflow in batches with neon d=128 embedding layer LSTM (2, bsz) [5, 4, 940, 107, 14, 672, 1790, 333, 47, 11, 7890,…, 1]   POS NEG LSTM Affine        LSTM LSTM LSTM         RecurrentSum     n=64
  • 76.
  • 77.
  • 78.
    Nervana Systems Proprietary InSummary… 1. Deep learning methods are powerful and versatile 2. It’s important to understand how DL relates to traditional ML methods 3. The barrier of entry to using DL in practice is lowered with the neon framework on the Nervana ecosystem kyle.h.ambert@intel.com @TheKyleAmbert