SlideShare a Scribd company logo
From multiplication to 
convolutional networks 
How do ML with Theano
Today’s Talk 
● A motivating problem 
● Understanding a model based framework 
● Theano 
○ Linear Regression 
○ Logistic Regression 
○ Net 
○ Modern Net 
○ Convolutional Net
Follow along 
Tutorial code at: 
https://github.com/Newmu/Theano-Tutorials 
Data at: 
http://yann.lecun.com/exdb/mnist/ 
Slides at: 
http://goo.gl/vuBQfe
A motivating problem 
How do we program a computer to recognize a picture of a 
handwritten digit as a 0-9? 
What could we do?
A dataset - MNIST 
What if we have 60,000 of these images and their label? 
X = images 
Y = labels 
X = (60000 x 784) #matrix (list of lists) 
Y = (60000) #vector (list) 
Given X as input, predict Y
An idea 
For each image, find the “most similar” image and guess 
that as the label.
An idea 
For each image, find the “most similar” image and guess 
that as the label. 
KNearestNeighbors ~95% accuracy
Trying things 
Make some functions computing relevant information for 
solving the problem
What we can code 
Make some functions computing relevant information for 
solving the problem 
feature engineering
What we can code 
Hard coded rules are brittle and often aren’t obvious or 
apparent for many problems.
Model 
A Machine Learning Framework 
8 
Inputs Computation Outputs
from arXiv:1409.4842v1 [cs.CV] 17 Sep 2014 
A … model? - GoogLeNet
3 mult 
Input Computation Output 
A very simple model 
by x 12
Theano intro
Theano intro 
imports
Theano intro 
imports 
theano symbolic variable initialization
Theano intro 
imports 
theano symbolic variable initialization 
our model
Theano intro 
imports 
theano symbolic variable initialization 
our model 
compiling to a python function
Theano intro 
imports 
theano symbolic variable initialization 
our model 
compiling to a python function 
usage
Theano
Theano 
imports
Theano 
imports 
training data generation
Theano 
imports 
training data generation 
symbolic variable initialization
Theano 
imports 
training data generation 
symbolic variable initialization 
our model
Theano 
imports 
training data generation 
symbolic variable initialization 
our model 
model parameter initialization
Theano 
imports 
training data generation 
symbolic variable initialization 
our model 
model parameter initialization 
metric to be optimized by model
Theano 
imports 
training data generation 
symbolic variable initialization 
our model 
model parameter initialization 
metric to be optimized by model 
learning signal for parameter(s)
Theano 
imports 
training data generation 
symbolic variable initialization 
our model 
model parameter initialization 
metric to be optimized by model 
learning signal for parameter(s) 
how to change parameter based on learning signal
Theano 
imports 
training data generation 
symbolic variable initialization 
our model 
model parameter initialization 
metric to be optimized by model 
learning signal for parameter(s) 
how to change parameter based on learning signal 
compiling to a python function
Theano 
imports 
training data generation 
symbolic variable initialization 
our model 
model parameter initialization 
metric to be optimized by model 
learning signal for parameter(s) 
how to change parameter based on learning signal 
compiling to a python function 
iterate through data 100 times and train model 
on each example of input, output pairs
Theano doing its thing
Zero One Two Three Four Five Six Seven Eight Nine 
0.1 
0. 0. 0.1 0. 0. 0. 0. 0.7 0.1 
Logistic Regression 
softmax(X) 
T.dot(X, w)
Back to Theano
Back to Theano 
convert to correct dtype
Back to Theano 
convert to correct dtype 
initialize model parameters
Back to Theano 
convert to correct dtype 
initialize model parameters 
our model in matrix format
Back to Theano 
convert to correct dtype 
initialize model parameters 
our model in matrix format 
loading data matrices
Back to Theano 
convert to correct dtype 
initialize model parameters 
our model in matrix format 
loading data matrices 
now matrix types
Back to Theano 
convert to correct dtype 
initialize model parameters 
our model in matrix format 
loading data matrices 
now matrix types 
probability outputs and maxima predictions
Back to Theano 
convert to correct dtype 
initialize model parameters 
our model in matrix format 
loading data matrices 
now matrix types 
probability outputs and maxima predictions 
classification metric to optimize
Back to Theano 
convert to correct dtype 
initialize model parameters 
our model in matrix format 
loading data matrices 
now matrix types 
probability outputs and maxima predictions 
classification metric to optimize 
compile prediction function
Back to Theano 
convert to correct dtype 
initialize model parameters 
our model in matrix format 
loading data matrices 
now matrix types 
probability outputs and maxima predictions 
classification metric to optimize 
compile prediction function 
train on mini-batches of 128 
examples
0 1 2 3 4 5 6 7 8 9 
What it learns
0 1 2 3 4 5 6 7 8 9 
What it learns 
Test Accuracy: 92.5%
Zero One Two Three Four Five Six Seven Eight Nine 
0.0 
0. 0. 0.1 0. 0. 0. 0. 0.9 0. 
y = softmax(T.dot(h, wo)) 
h = T.nnet.sigmoid(T.dot(X, wh)) 
An “old” net (circa 2000)
A “old” net in Theano
A “old” net in Theano 
generalize to compute gradient descent on 
all model parameters
2D moons dataset 
courtesy of scikit-learn 
Understanding SGD
A “old” net in Theano 
generalize to compute gradient descent on 
all model parameters 
2 layers of computation 
input -> hidden (sigmoid) 
hidden -> output (softmax)
Understanding Sigmoid Units
A “old” net in Theano 
generalize to compute gradient descent on 
all model parameters 
2 layers of computation 
input -> hidden (sigmoid) 
hidden -> output (softmax) 
initialize both weight matrices
A “old” net in Theano 
generalize to compute gradient descent on 
all model parameters 
2 layers of computation 
input -> hidden (sigmoid) 
hidden -> output (softmax) 
initialize both weight matrices 
updated version of updates
What an “old” net learns 
Test Accuracy: 98.4%
Zero One Two Three Four Five Six Seven Eight Nine 
0.0 
0. 0. 0.1 0. 0. 0. 0. 0.9 0. 
y = softmax(T.dot(h2, wo)) 
h2 = rectify(T.dot(h, wh)) 
h = rectify(T.dot(X, wh)) 
Noise 
Noise 
Noise 
(or augmentation) 
A “modern” net - 2012+
A “modern” net in Theano
rectifier 
A “modern” net in Theano
Understanding rectifier units
rectifier 
numerically stable softmax 
A “modern” net in Theano
rectifier 
numerically stable softmax 
a running average of the magnitude of the gradient 
A “modern” net in Theano
rectifier 
numerically stable softmax 
a running average of the magnitude of the gradient 
scale the gradient based on running average 
A “modern” net in Theano
2D moons dataset 
courtesy of scikit-learn 
Understanding RMSprop
rectifier 
numerically stable softmax 
a running average of the magnitude of the gradient 
scale the gradient based on running average 
A “modern” net in Theano 
randomly drop values and scale rest
rectifier 
numerically stable softmax 
a running average of the magnitude of the gradient 
scale the gradient based on running average 
A “modern” net in Theano 
randomly drop values and scale rest 
Noise injected into model 
rectifiers now used 
2 hidden layers
What a “modern” net learns 
Test Accuracy: 99.0%
Quantifying the difference
What a “modern” net is doing
from deeplearning.net 
Convolutional Networks
A convolutional network in Theano
a “block” of computation conv -> activate -> pool -> noise 
A convolutional network in Theano
a “block” of computation conv -> activate -> pool -> noise 
convert from 4tensor to normal matrix 
A convolutional network in Theano
a “block” of computation conv -> activate -> pool -> noise 
convert from 4tensor to normal matrix 
reshape into conv 4tensor (b, c, 0, 1) format 
A convolutional network in Theano
a “block” of computation conv -> activate -> pool -> noise 
convert from 4tensor to normal matrix 
reshape into conv 4tensor (b, c, 0, 1) format 
now 4tensor for conv instead of matrix 
A convolutional network in Theano
a “block” of computation conv -> activate -> pool -> noise 
convert from 4tensor to normal matrix 
reshape into conv 4tensor (b, c, 0, 1) format 
now 4tensor for conv instead of matrix 
conv weights (n_kernels, n_channels, kernel_w, kerbel_h) 
A convolutional network in Theano
a “block” of computation conv -> activate -> pool -> noise 
convert from 4tensor to normal matrix 
reshape into conv 4tensor (b, c, 0, 1) format 
now 4tensor for conv instead of matrix 
conv weights (n_kernels, n_channels, kernel_w, kerbel_h) 
highest conv layer has 128 filters and a 3x3 grid of responses 
A convolutional network in Theano
a “block” of computation conv -> activate -> pool -> noise 
convert from 4tensor to normal matrix 
reshape into conv 4tensor (b, c, 0, 1) format 
now 4tensor for conv instead of matrix 
conv weights (n_kernels, n_channels, kernel_w, kerbel_h) 
highest conv layer has 128 filters and a 3x3 grid of responses 
A convolutional network in Theano 
noise during training
a “block” of computation conv -> activate -> pool -> noise 
convert from 4tensor to normal matrix 
reshape into conv 4tensor (b, c, 0, 1) format 
now 4tensor for conv instead of matrix 
conv weights (n_kernels, n_channels, kernel_w, kerbel_h) 
highest conv layer has 128 filters and a 3x3 grid of responses 
A convolutional network in Theano 
noise during training 
no noise for prediction
Test Accuracy: 99.5% 
What a convolutional network learns
Takeaways 
● A few tricks are needed to get good results 
○ Noise important for regularization 
○ Rectifiers for faster, better, learning 
○ Don’t use SGD - lots of cheap simple improvements 
● Models need room to compute. 
● If your data has structure, your model should 
respect it.
Resources 
● More in-depth theano tutorials 
○ http://www.deeplearning.net/tutorial/ 
● Theano docs 
○ http://www.deeplearning.net/software/theano/library/ 
● Community 
○ http://www.reddit.com/r/machinelearning
A plug 
Keep up to date with indico: 
https://indico1.typeform.com/to/DgN5SP
Questions?

More Related Content

What's hot

Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)
Oswald Campesato
 
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
MLconf
 
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
Altoros
 
Introduction to Neural Networks in Tensorflow
Introduction to Neural Networks in TensorflowIntroduction to Neural Networks in Tensorflow
Introduction to Neural Networks in Tensorflow
Nicholas McClure
 
Language translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlowLanguage translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlow
S N
 
H2O Deep Learning at Next.ML
H2O Deep Learning at Next.MLH2O Deep Learning at Next.ML
H2O Deep Learning at Next.ML
Sri Ambati
 
How to win data science competitions with Deep Learning
How to win data science competitions with Deep LearningHow to win data science competitions with Deep Learning
How to win data science competitions with Deep Learning
Sri Ambati
 
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres RodriguezIntroduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Intel Nervana
 
AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)
AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)
AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)
Amazon Web Services
 
H2O Distributed Deep Learning by Arno Candel 071614
H2O Distributed Deep Learning by Arno Candel 071614H2O Distributed Deep Learning by Arno Candel 071614
H2O Distributed Deep Learning by Arno Candel 071614
Sri Ambati
 
PyTorch for Deep Learning Practitioners
PyTorch for Deep Learning PractitionersPyTorch for Deep Learning Practitioners
PyTorch for Deep Learning Practitioners
Bayu Aldi Yansyah
 
Learning stochastic neural networks with Chainer
Learning stochastic neural networks with ChainerLearning stochastic neural networks with Chainer
Learning stochastic neural networks with Chainer
Seiya Tokui
 
Statistical Machine Learning for Text Classification with scikit-learn and NLTK
Statistical Machine Learning for Text Classification with scikit-learn and NLTKStatistical Machine Learning for Text Classification with scikit-learn and NLTK
Statistical Machine Learning for Text Classification with scikit-learn and NLTK
Olivier Grisel
 
Rajat Monga at AI Frontiers: Deep Learning with TensorFlow
Rajat Monga at AI Frontiers: Deep Learning with TensorFlowRajat Monga at AI Frontiers: Deep Learning with TensorFlow
Rajat Monga at AI Frontiers: Deep Learning with TensorFlow
AI Frontiers
 
H2O Open Source Deep Learning, Arno Candel 03-20-14
H2O Open Source Deep Learning, Arno Candel 03-20-14H2O Open Source Deep Learning, Arno Candel 03-20-14
H2O Open Source Deep Learning, Arno Candel 03-20-14
Sri Ambati
 
Distributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflowDistributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflow
Emanuel Di Nardo
 
Scalable Deep Learning Using Apache MXNet
Scalable Deep Learning Using Apache MXNetScalable Deep Learning Using Apache MXNet
Scalable Deep Learning Using Apache MXNet
Amazon Web Services
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to Chainer
Shunta Saito
 
TensorFlow Tutorial | Deep Learning With TensorFlow | TensorFlow Tutorial For...
TensorFlow Tutorial | Deep Learning With TensorFlow | TensorFlow Tutorial For...TensorFlow Tutorial | Deep Learning With TensorFlow | TensorFlow Tutorial For...
TensorFlow Tutorial | Deep Learning With TensorFlow | TensorFlow Tutorial For...
Simplilearn
 
Deep learning for molecules, introduction to chainer chemistry
Deep learning for molecules, introduction to chainer chemistryDeep learning for molecules, introduction to chainer chemistry
Deep learning for molecules, introduction to chainer chemistry
Kenta Oono
 

What's hot (20)

Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)Diving into Deep Learning (Silicon Valley Code Camp 2017)
Diving into Deep Learning (Silicon Valley Code Camp 2017)
 
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
Alex Smola, Director of Machine Learning, AWS/Amazon, at MLconf SF 2016
 
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
Deep Learning with TensorFlow: Understanding Tensors, Computations Graphs, Im...
 
Introduction to Neural Networks in Tensorflow
Introduction to Neural Networks in TensorflowIntroduction to Neural Networks in Tensorflow
Introduction to Neural Networks in Tensorflow
 
Language translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlowLanguage translation with Deep Learning (RNN) with TensorFlow
Language translation with Deep Learning (RNN) with TensorFlow
 
H2O Deep Learning at Next.ML
H2O Deep Learning at Next.MLH2O Deep Learning at Next.ML
H2O Deep Learning at Next.ML
 
How to win data science competitions with Deep Learning
How to win data science competitions with Deep LearningHow to win data science competitions with Deep Learning
How to win data science competitions with Deep Learning
 
Introduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres RodriguezIntroduction to deep learning @ Startup.ML by Andres Rodriguez
Introduction to deep learning @ Startup.ML by Andres Rodriguez
 
AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)
AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)
AWS re:Invent 2016: Using MXNet for Recommendation Modeling at Scale (MAC306)
 
H2O Distributed Deep Learning by Arno Candel 071614
H2O Distributed Deep Learning by Arno Candel 071614H2O Distributed Deep Learning by Arno Candel 071614
H2O Distributed Deep Learning by Arno Candel 071614
 
PyTorch for Deep Learning Practitioners
PyTorch for Deep Learning PractitionersPyTorch for Deep Learning Practitioners
PyTorch for Deep Learning Practitioners
 
Learning stochastic neural networks with Chainer
Learning stochastic neural networks with ChainerLearning stochastic neural networks with Chainer
Learning stochastic neural networks with Chainer
 
Statistical Machine Learning for Text Classification with scikit-learn and NLTK
Statistical Machine Learning for Text Classification with scikit-learn and NLTKStatistical Machine Learning for Text Classification with scikit-learn and NLTK
Statistical Machine Learning for Text Classification with scikit-learn and NLTK
 
Rajat Monga at AI Frontiers: Deep Learning with TensorFlow
Rajat Monga at AI Frontiers: Deep Learning with TensorFlowRajat Monga at AI Frontiers: Deep Learning with TensorFlow
Rajat Monga at AI Frontiers: Deep Learning with TensorFlow
 
H2O Open Source Deep Learning, Arno Candel 03-20-14
H2O Open Source Deep Learning, Arno Candel 03-20-14H2O Open Source Deep Learning, Arno Candel 03-20-14
H2O Open Source Deep Learning, Arno Candel 03-20-14
 
Distributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflowDistributed implementation of a lstm on spark and tensorflow
Distributed implementation of a lstm on spark and tensorflow
 
Scalable Deep Learning Using Apache MXNet
Scalable Deep Learning Using Apache MXNetScalable Deep Learning Using Apache MXNet
Scalable Deep Learning Using Apache MXNet
 
Introduction to Chainer
Introduction to ChainerIntroduction to Chainer
Introduction to Chainer
 
TensorFlow Tutorial | Deep Learning With TensorFlow | TensorFlow Tutorial For...
TensorFlow Tutorial | Deep Learning With TensorFlow | TensorFlow Tutorial For...TensorFlow Tutorial | Deep Learning With TensorFlow | TensorFlow Tutorial For...
TensorFlow Tutorial | Deep Learning With TensorFlow | TensorFlow Tutorial For...
 
Deep learning for molecules, introduction to chainer chemistry
Deep learning for molecules, introduction to chainer chemistryDeep learning for molecules, introduction to chainer chemistry
Deep learning for molecules, introduction to chainer chemistry
 

Viewers also liked

Best Practices in Maintenance and Reliability
Best Practices in Maintenance and ReliabilityBest Practices in Maintenance and Reliability
Best Practices in Maintenance and Reliability
Ricky Smith CMRP, CMRT
 
The Ultimate Guide to Creating Visually Appealing Content
The Ultimate Guide to Creating Visually Appealing ContentThe Ultimate Guide to Creating Visually Appealing Content
The Ultimate Guide to Creating Visually Appealing Content
Neil Patel
 
Thai tech startup ecosystem report 2017
Thai tech startup ecosystem report 2017Thai tech startup ecosystem report 2017
Thai tech startup ecosystem report 2017
Techsauce Media
 
Mri brain anatomy Dr Muhammad Bin Zulfiqar
Mri brain anatomy Dr Muhammad Bin ZulfiqarMri brain anatomy Dr Muhammad Bin Zulfiqar
Mri brain anatomy Dr Muhammad Bin Zulfiqar
Dr. Muhammad Bin Zulfiqar
 
What I Carry: 10 Tools for Success
What I Carry: 10 Tools for SuccessWhat I Carry: 10 Tools for Success
What I Carry: 10 Tools for Success
Jonathon Colman
 
Autonomic nervous system Physiology
Autonomic nervous system PhysiologyAutonomic nervous system Physiology
Autonomic nervous system Physiology
Raghu Veer
 
Dear NSA, let me take care of your slides.
Dear NSA, let me take care of your slides.Dear NSA, let me take care of your slides.
Dear NSA, let me take care of your slides.
Emiland
 
What Makes Great Infographics
What Makes Great InfographicsWhat Makes Great Infographics
What Makes Great Infographics
SlideShare
 
Masters of SlideShare
Masters of SlideShareMasters of SlideShare
Masters of SlideShare
Kapost
 
You Suck At PowerPoint!
You Suck At PowerPoint!You Suck At PowerPoint!
You Suck At PowerPoint!
Jesse Desjardins - @jessedee
 
10 Ways to Win at SlideShare SEO & Presentation Optimization
10 Ways to Win at SlideShare SEO & Presentation Optimization10 Ways to Win at SlideShare SEO & Presentation Optimization
10 Ways to Win at SlideShare SEO & Presentation Optimization
Oneupweb
 
Deep Dive into SharePoint Topologies and Server Architecture for SharePoint 2013
Deep Dive into SharePoint Topologies and Server Architecture for SharePoint 2013Deep Dive into SharePoint Topologies and Server Architecture for SharePoint 2013
Deep Dive into SharePoint Topologies and Server Architecture for SharePoint 2013
K.Mohamed Faizal
 
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
SlideShare
 
Engaging Learners with Technology
Engaging Learners with TechnologyEngaging Learners with Technology
Engaging Learners with Technology
Dean Shareski
 
2015 Upload Campaigns Calendar - SlideShare
2015 Upload Campaigns Calendar - SlideShare2015 Upload Campaigns Calendar - SlideShare
2015 Upload Campaigns Calendar - SlideShare
SlideShare
 
Operating Systems - File Management
Operating Systems -  File ManagementOperating Systems -  File Management
Operating Systems - File Management
Damian T. Gordon
 
Actionable Customer Development
Actionable Customer DevelopmentActionable Customer Development
Actionable Customer Development
Andreas Klinger
 
STOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
STOP! VIEW THIS! 10-Step Checklist When Uploading to SlideshareSTOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
STOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
Empowered Presentations
 
How To Get More From SlideShare - Super-Simple Tips For Content Marketing
How To Get More From SlideShare - Super-Simple Tips For Content MarketingHow To Get More From SlideShare - Super-Simple Tips For Content Marketing
How To Get More From SlideShare - Super-Simple Tips For Content Marketing
Content Marketing Institute
 
How to Make Awesome SlideShares: Tips & Tricks
How to Make Awesome SlideShares: Tips & TricksHow to Make Awesome SlideShares: Tips & Tricks
How to Make Awesome SlideShares: Tips & Tricks
SlideShare
 

Viewers also liked (20)

Best Practices in Maintenance and Reliability
Best Practices in Maintenance and ReliabilityBest Practices in Maintenance and Reliability
Best Practices in Maintenance and Reliability
 
The Ultimate Guide to Creating Visually Appealing Content
The Ultimate Guide to Creating Visually Appealing ContentThe Ultimate Guide to Creating Visually Appealing Content
The Ultimate Guide to Creating Visually Appealing Content
 
Thai tech startup ecosystem report 2017
Thai tech startup ecosystem report 2017Thai tech startup ecosystem report 2017
Thai tech startup ecosystem report 2017
 
Mri brain anatomy Dr Muhammad Bin Zulfiqar
Mri brain anatomy Dr Muhammad Bin ZulfiqarMri brain anatomy Dr Muhammad Bin Zulfiqar
Mri brain anatomy Dr Muhammad Bin Zulfiqar
 
What I Carry: 10 Tools for Success
What I Carry: 10 Tools for SuccessWhat I Carry: 10 Tools for Success
What I Carry: 10 Tools for Success
 
Autonomic nervous system Physiology
Autonomic nervous system PhysiologyAutonomic nervous system Physiology
Autonomic nervous system Physiology
 
Dear NSA, let me take care of your slides.
Dear NSA, let me take care of your slides.Dear NSA, let me take care of your slides.
Dear NSA, let me take care of your slides.
 
What Makes Great Infographics
What Makes Great InfographicsWhat Makes Great Infographics
What Makes Great Infographics
 
Masters of SlideShare
Masters of SlideShareMasters of SlideShare
Masters of SlideShare
 
You Suck At PowerPoint!
You Suck At PowerPoint!You Suck At PowerPoint!
You Suck At PowerPoint!
 
10 Ways to Win at SlideShare SEO & Presentation Optimization
10 Ways to Win at SlideShare SEO & Presentation Optimization10 Ways to Win at SlideShare SEO & Presentation Optimization
10 Ways to Win at SlideShare SEO & Presentation Optimization
 
Deep Dive into SharePoint Topologies and Server Architecture for SharePoint 2013
Deep Dive into SharePoint Topologies and Server Architecture for SharePoint 2013Deep Dive into SharePoint Topologies and Server Architecture for SharePoint 2013
Deep Dive into SharePoint Topologies and Server Architecture for SharePoint 2013
 
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
A Guide to SlideShare Analytics - Excerpts from Hubspot's Step by Step Guide ...
 
Engaging Learners with Technology
Engaging Learners with TechnologyEngaging Learners with Technology
Engaging Learners with Technology
 
2015 Upload Campaigns Calendar - SlideShare
2015 Upload Campaigns Calendar - SlideShare2015 Upload Campaigns Calendar - SlideShare
2015 Upload Campaigns Calendar - SlideShare
 
Operating Systems - File Management
Operating Systems -  File ManagementOperating Systems -  File Management
Operating Systems - File Management
 
Actionable Customer Development
Actionable Customer DevelopmentActionable Customer Development
Actionable Customer Development
 
STOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
STOP! VIEW THIS! 10-Step Checklist When Uploading to SlideshareSTOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
STOP! VIEW THIS! 10-Step Checklist When Uploading to Slideshare
 
How To Get More From SlideShare - Super-Simple Tips For Content Marketing
How To Get More From SlideShare - Super-Simple Tips For Content MarketingHow To Get More From SlideShare - Super-Simple Tips For Content Marketing
How To Get More From SlideShare - Super-Simple Tips For Content Marketing
 
How to Make Awesome SlideShares: Tips & Tricks
How to Make Awesome SlideShares: Tips & TricksHow to Make Awesome SlideShares: Tips & Tricks
How to Make Awesome SlideShares: Tips & Tricks
 

Similar to Introduction to Deep Learning with Python

Josh Patterson MLconf slides
Josh Patterson MLconf slidesJosh Patterson MLconf slides
Josh Patterson MLconf slides
MLconf
 
Power ai tensorflowworkloadtutorial-20171117
Power ai tensorflowworkloadtutorial-20171117Power ai tensorflowworkloadtutorial-20171117
Power ai tensorflowworkloadtutorial-20171117
Ganesan Narayanasamy
 
Introduction to Deep Learning and neon at Galvanize
Introduction to Deep Learning and neon at GalvanizeIntroduction to Deep Learning and neon at Galvanize
Introduction to Deep Learning and neon at Galvanize
Intel Nervana
 
Neural networks with python
Neural networks with pythonNeural networks with python
Neural networks with python
Simone Piunno
 
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
R Cheat Sheet
R Cheat SheetR Cheat Sheet
R Cheat Sheet
Dr. Volkan OBAN
 
Next Generation Programming in R
Next Generation Programming in RNext Generation Programming in R
Next Generation Programming in R
Florian Uhlitz
 
Lesson 2 data preprocessing
Lesson 2   data preprocessingLesson 2   data preprocessing
Lesson 2 data preprocessing
AbdurRazzaqe1
 
Neural networks using tensor flow in amazon deep learning server
Neural networks using tensor flow in amazon deep learning serverNeural networks using tensor flow in amazon deep learning server
Neural networks using tensor flow in amazon deep learning server
Ramco Institute of Technology, Rajapalayam, Tamilnadu, India
 
Machine Learning and Go. Go!
Machine Learning and Go. Go!Machine Learning and Go. Go!
Machine Learning and Go. Go!
Diana Ortega
 
FivaTech
FivaTechFivaTech
FivaTech
marxliouville
 
1212 regular meeting
1212 regular meeting1212 regular meeting
1212 regular meeting
marxliouville
 
maxbox starter60 machine learning
maxbox starter60 machine learningmaxbox starter60 machine learning
maxbox starter60 machine learning
Max Kleiner
 
The ABC of Implementing Supervised Machine Learning with Python.pptx
The ABC of Implementing Supervised Machine Learning with Python.pptxThe ABC of Implementing Supervised Machine Learning with Python.pptx
The ABC of Implementing Supervised Machine Learning with Python.pptx
Ruby Shrestha
 
Machine learning
Machine learningMachine learning
Machine learning
Andrea Iacono
 
Android and Deep Learning
Android and Deep LearningAndroid and Deep Learning
Android and Deep Learning
Oswald Campesato
 
Image classification using cnn
Image classification using cnnImage classification using cnn
Image classification using cnn
Debarko De
 
Forelasning4
Forelasning4Forelasning4
Forelasning4
Memo Love
 
Angular and Deep Learning
Angular and Deep LearningAngular and Deep Learning
Angular and Deep Learning
Oswald Campesato
 
MLConf 2013: Metronome and Parallel Iterative Algorithms on YARN
MLConf 2013: Metronome and Parallel Iterative Algorithms on YARNMLConf 2013: Metronome and Parallel Iterative Algorithms on YARN
MLConf 2013: Metronome and Parallel Iterative Algorithms on YARN
Josh Patterson
 

Similar to Introduction to Deep Learning with Python (20)

Josh Patterson MLconf slides
Josh Patterson MLconf slidesJosh Patterson MLconf slides
Josh Patterson MLconf slides
 
Power ai tensorflowworkloadtutorial-20171117
Power ai tensorflowworkloadtutorial-20171117Power ai tensorflowworkloadtutorial-20171117
Power ai tensorflowworkloadtutorial-20171117
 
Introduction to Deep Learning and neon at Galvanize
Introduction to Deep Learning and neon at GalvanizeIntroduction to Deep Learning and neon at Galvanize
Introduction to Deep Learning and neon at Galvanize
 
Neural networks with python
Neural networks with pythonNeural networks with python
Neural networks with python
 
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
 
R Cheat Sheet
R Cheat SheetR Cheat Sheet
R Cheat Sheet
 
Next Generation Programming in R
Next Generation Programming in RNext Generation Programming in R
Next Generation Programming in R
 
Lesson 2 data preprocessing
Lesson 2   data preprocessingLesson 2   data preprocessing
Lesson 2 data preprocessing
 
Neural networks using tensor flow in amazon deep learning server
Neural networks using tensor flow in amazon deep learning serverNeural networks using tensor flow in amazon deep learning server
Neural networks using tensor flow in amazon deep learning server
 
Machine Learning and Go. Go!
Machine Learning and Go. Go!Machine Learning and Go. Go!
Machine Learning and Go. Go!
 
FivaTech
FivaTechFivaTech
FivaTech
 
1212 regular meeting
1212 regular meeting1212 regular meeting
1212 regular meeting
 
maxbox starter60 machine learning
maxbox starter60 machine learningmaxbox starter60 machine learning
maxbox starter60 machine learning
 
The ABC of Implementing Supervised Machine Learning with Python.pptx
The ABC of Implementing Supervised Machine Learning with Python.pptxThe ABC of Implementing Supervised Machine Learning with Python.pptx
The ABC of Implementing Supervised Machine Learning with Python.pptx
 
Machine learning
Machine learningMachine learning
Machine learning
 
Android and Deep Learning
Android and Deep LearningAndroid and Deep Learning
Android and Deep Learning
 
Image classification using cnn
Image classification using cnnImage classification using cnn
Image classification using cnn
 
Forelasning4
Forelasning4Forelasning4
Forelasning4
 
Angular and Deep Learning
Angular and Deep LearningAngular and Deep Learning
Angular and Deep Learning
 
MLConf 2013: Metronome and Parallel Iterative Algorithms on YARN
MLConf 2013: Metronome and Parallel Iterative Algorithms on YARNMLConf 2013: Metronome and Parallel Iterative Algorithms on YARN
MLConf 2013: Metronome and Parallel Iterative Algorithms on YARN
 

More from indico data

Small Data for Big Problems: Practical Transfer Learning for NLP
Small Data for Big Problems: Practical Transfer Learning for NLPSmall Data for Big Problems: Practical Transfer Learning for NLP
Small Data for Big Problems: Practical Transfer Learning for NLP
indico data
 
Getting to AI ROI: Finding Value in Your Unstructured Content
Getting to AI ROI: Finding Value in Your Unstructured ContentGetting to AI ROI: Finding Value in Your Unstructured Content
Getting to AI ROI: Finding Value in Your Unstructured Content
indico data
 
Everything You Wanted to Know About Optimization
Everything You Wanted to Know About OptimizationEverything You Wanted to Know About Optimization
Everything You Wanted to Know About Optimization
indico data
 
ODSC East: Effective Transfer Learning for NLP
ODSC East: Effective Transfer Learning for NLPODSC East: Effective Transfer Learning for NLP
ODSC East: Effective Transfer Learning for NLP
indico data
 
TensorFlow in Practice
TensorFlow in PracticeTensorFlow in Practice
TensorFlow in Practice
indico data
 
The Unreasonable Benefits of Deep Learning
The Unreasonable Benefits of Deep LearningThe Unreasonable Benefits of Deep Learning
The Unreasonable Benefits of Deep Learning
indico data
 
How Machine Learning is Shaping Digital Marketing
How Machine Learning is Shaping Digital MarketingHow Machine Learning is Shaping Digital Marketing
How Machine Learning is Shaping Digital Marketing
indico data
 
Deep Advances in Generative Modeling
Deep Advances in Generative ModelingDeep Advances in Generative Modeling
Deep Advances in Generative Modeling
indico data
 
Machine Learning for Non-technical People
Machine Learning for Non-technical PeopleMachine Learning for Non-technical People
Machine Learning for Non-technical People
indico data
 
Getting started with indico APIs [Python]
Getting started with indico APIs [Python]Getting started with indico APIs [Python]
Getting started with indico APIs [Python]
indico data
 

More from indico data (10)

Small Data for Big Problems: Practical Transfer Learning for NLP
Small Data for Big Problems: Practical Transfer Learning for NLPSmall Data for Big Problems: Practical Transfer Learning for NLP
Small Data for Big Problems: Practical Transfer Learning for NLP
 
Getting to AI ROI: Finding Value in Your Unstructured Content
Getting to AI ROI: Finding Value in Your Unstructured ContentGetting to AI ROI: Finding Value in Your Unstructured Content
Getting to AI ROI: Finding Value in Your Unstructured Content
 
Everything You Wanted to Know About Optimization
Everything You Wanted to Know About OptimizationEverything You Wanted to Know About Optimization
Everything You Wanted to Know About Optimization
 
ODSC East: Effective Transfer Learning for NLP
ODSC East: Effective Transfer Learning for NLPODSC East: Effective Transfer Learning for NLP
ODSC East: Effective Transfer Learning for NLP
 
TensorFlow in Practice
TensorFlow in PracticeTensorFlow in Practice
TensorFlow in Practice
 
The Unreasonable Benefits of Deep Learning
The Unreasonable Benefits of Deep LearningThe Unreasonable Benefits of Deep Learning
The Unreasonable Benefits of Deep Learning
 
How Machine Learning is Shaping Digital Marketing
How Machine Learning is Shaping Digital MarketingHow Machine Learning is Shaping Digital Marketing
How Machine Learning is Shaping Digital Marketing
 
Deep Advances in Generative Modeling
Deep Advances in Generative ModelingDeep Advances in Generative Modeling
Deep Advances in Generative Modeling
 
Machine Learning for Non-technical People
Machine Learning for Non-technical PeopleMachine Learning for Non-technical People
Machine Learning for Non-technical People
 
Getting started with indico APIs [Python]
Getting started with indico APIs [Python]Getting started with indico APIs [Python]
Getting started with indico APIs [Python]
 

Recently uploaded

一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
nuttdpt
 
Intelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicineIntelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicine
AndrzejJarynowski
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
vikram sood
 
End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024
Lars Albertsson
 
Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
Roger Valdez
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
manishkhaire30
 
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
v7oacc3l
 
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
Timothy Spann
 
Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...
Bill641377
 
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
bopyb
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
sameer shah
 
Experts live - Improving user adoption with AI
Experts live - Improving user adoption with AIExperts live - Improving user adoption with AI
Experts live - Improving user adoption with AI
jitskeb
 
Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......
Sachin Paul
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
nyfuhyz
 
The Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series DatabaseThe Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series Database
javier ramirez
 
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
xclpvhuk
 
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdfUdemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Fernanda Palhano
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
apvysm8
 
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performanceAnalysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
roli9797
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
ihavuls
 

Recently uploaded (20)

一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
 
Intelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicineIntelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicine
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
 
End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024
 
Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
 
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
 
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
 
Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...
 
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
 
Experts live - Improving user adoption with AI
Experts live - Improving user adoption with AIExperts live - Improving user adoption with AI
Experts live - Improving user adoption with AI
 
Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
 
The Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series DatabaseThe Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series Database
 
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
 
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdfUdemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
 
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performanceAnalysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
 

Introduction to Deep Learning with Python

  • 1. From multiplication to convolutional networks How do ML with Theano
  • 2. Today’s Talk ● A motivating problem ● Understanding a model based framework ● Theano ○ Linear Regression ○ Logistic Regression ○ Net ○ Modern Net ○ Convolutional Net
  • 3. Follow along Tutorial code at: https://github.com/Newmu/Theano-Tutorials Data at: http://yann.lecun.com/exdb/mnist/ Slides at: http://goo.gl/vuBQfe
  • 4. A motivating problem How do we program a computer to recognize a picture of a handwritten digit as a 0-9? What could we do?
  • 5. A dataset - MNIST What if we have 60,000 of these images and their label? X = images Y = labels X = (60000 x 784) #matrix (list of lists) Y = (60000) #vector (list) Given X as input, predict Y
  • 6. An idea For each image, find the “most similar” image and guess that as the label.
  • 7. An idea For each image, find the “most similar” image and guess that as the label. KNearestNeighbors ~95% accuracy
  • 8. Trying things Make some functions computing relevant information for solving the problem
  • 9. What we can code Make some functions computing relevant information for solving the problem feature engineering
  • 10. What we can code Hard coded rules are brittle and often aren’t obvious or apparent for many problems.
  • 11. Model A Machine Learning Framework 8 Inputs Computation Outputs
  • 12. from arXiv:1409.4842v1 [cs.CV] 17 Sep 2014 A … model? - GoogLeNet
  • 13. 3 mult Input Computation Output A very simple model by x 12
  • 16. Theano intro imports theano symbolic variable initialization
  • 17. Theano intro imports theano symbolic variable initialization our model
  • 18. Theano intro imports theano symbolic variable initialization our model compiling to a python function
  • 19. Theano intro imports theano symbolic variable initialization our model compiling to a python function usage
  • 22. Theano imports training data generation
  • 23. Theano imports training data generation symbolic variable initialization
  • 24. Theano imports training data generation symbolic variable initialization our model
  • 25. Theano imports training data generation symbolic variable initialization our model model parameter initialization
  • 26. Theano imports training data generation symbolic variable initialization our model model parameter initialization metric to be optimized by model
  • 27. Theano imports training data generation symbolic variable initialization our model model parameter initialization metric to be optimized by model learning signal for parameter(s)
  • 28. Theano imports training data generation symbolic variable initialization our model model parameter initialization metric to be optimized by model learning signal for parameter(s) how to change parameter based on learning signal
  • 29. Theano imports training data generation symbolic variable initialization our model model parameter initialization metric to be optimized by model learning signal for parameter(s) how to change parameter based on learning signal compiling to a python function
  • 30. Theano imports training data generation symbolic variable initialization our model model parameter initialization metric to be optimized by model learning signal for parameter(s) how to change parameter based on learning signal compiling to a python function iterate through data 100 times and train model on each example of input, output pairs
  • 32. Zero One Two Three Four Five Six Seven Eight Nine 0.1 0. 0. 0.1 0. 0. 0. 0. 0.7 0.1 Logistic Regression softmax(X) T.dot(X, w)
  • 34. Back to Theano convert to correct dtype
  • 35. Back to Theano convert to correct dtype initialize model parameters
  • 36. Back to Theano convert to correct dtype initialize model parameters our model in matrix format
  • 37. Back to Theano convert to correct dtype initialize model parameters our model in matrix format loading data matrices
  • 38. Back to Theano convert to correct dtype initialize model parameters our model in matrix format loading data matrices now matrix types
  • 39. Back to Theano convert to correct dtype initialize model parameters our model in matrix format loading data matrices now matrix types probability outputs and maxima predictions
  • 40. Back to Theano convert to correct dtype initialize model parameters our model in matrix format loading data matrices now matrix types probability outputs and maxima predictions classification metric to optimize
  • 41. Back to Theano convert to correct dtype initialize model parameters our model in matrix format loading data matrices now matrix types probability outputs and maxima predictions classification metric to optimize compile prediction function
  • 42. Back to Theano convert to correct dtype initialize model parameters our model in matrix format loading data matrices now matrix types probability outputs and maxima predictions classification metric to optimize compile prediction function train on mini-batches of 128 examples
  • 43. 0 1 2 3 4 5 6 7 8 9 What it learns
  • 44. 0 1 2 3 4 5 6 7 8 9 What it learns Test Accuracy: 92.5%
  • 45. Zero One Two Three Four Five Six Seven Eight Nine 0.0 0. 0. 0.1 0. 0. 0. 0. 0.9 0. y = softmax(T.dot(h, wo)) h = T.nnet.sigmoid(T.dot(X, wh)) An “old” net (circa 2000)
  • 46. A “old” net in Theano
  • 47. A “old” net in Theano generalize to compute gradient descent on all model parameters
  • 48. 2D moons dataset courtesy of scikit-learn Understanding SGD
  • 49. A “old” net in Theano generalize to compute gradient descent on all model parameters 2 layers of computation input -> hidden (sigmoid) hidden -> output (softmax)
  • 51. A “old” net in Theano generalize to compute gradient descent on all model parameters 2 layers of computation input -> hidden (sigmoid) hidden -> output (softmax) initialize both weight matrices
  • 52. A “old” net in Theano generalize to compute gradient descent on all model parameters 2 layers of computation input -> hidden (sigmoid) hidden -> output (softmax) initialize both weight matrices updated version of updates
  • 53. What an “old” net learns Test Accuracy: 98.4%
  • 54. Zero One Two Three Four Five Six Seven Eight Nine 0.0 0. 0. 0.1 0. 0. 0. 0. 0.9 0. y = softmax(T.dot(h2, wo)) h2 = rectify(T.dot(h, wh)) h = rectify(T.dot(X, wh)) Noise Noise Noise (or augmentation) A “modern” net - 2012+
  • 55. A “modern” net in Theano
  • 56. rectifier A “modern” net in Theano
  • 58. rectifier numerically stable softmax A “modern” net in Theano
  • 59. rectifier numerically stable softmax a running average of the magnitude of the gradient A “modern” net in Theano
  • 60. rectifier numerically stable softmax a running average of the magnitude of the gradient scale the gradient based on running average A “modern” net in Theano
  • 61. 2D moons dataset courtesy of scikit-learn Understanding RMSprop
  • 62. rectifier numerically stable softmax a running average of the magnitude of the gradient scale the gradient based on running average A “modern” net in Theano randomly drop values and scale rest
  • 63. rectifier numerically stable softmax a running average of the magnitude of the gradient scale the gradient based on running average A “modern” net in Theano randomly drop values and scale rest Noise injected into model rectifiers now used 2 hidden layers
  • 64. What a “modern” net learns Test Accuracy: 99.0%
  • 66. What a “modern” net is doing
  • 69. a “block” of computation conv -> activate -> pool -> noise A convolutional network in Theano
  • 70. a “block” of computation conv -> activate -> pool -> noise convert from 4tensor to normal matrix A convolutional network in Theano
  • 71. a “block” of computation conv -> activate -> pool -> noise convert from 4tensor to normal matrix reshape into conv 4tensor (b, c, 0, 1) format A convolutional network in Theano
  • 72. a “block” of computation conv -> activate -> pool -> noise convert from 4tensor to normal matrix reshape into conv 4tensor (b, c, 0, 1) format now 4tensor for conv instead of matrix A convolutional network in Theano
  • 73. a “block” of computation conv -> activate -> pool -> noise convert from 4tensor to normal matrix reshape into conv 4tensor (b, c, 0, 1) format now 4tensor for conv instead of matrix conv weights (n_kernels, n_channels, kernel_w, kerbel_h) A convolutional network in Theano
  • 74. a “block” of computation conv -> activate -> pool -> noise convert from 4tensor to normal matrix reshape into conv 4tensor (b, c, 0, 1) format now 4tensor for conv instead of matrix conv weights (n_kernels, n_channels, kernel_w, kerbel_h) highest conv layer has 128 filters and a 3x3 grid of responses A convolutional network in Theano
  • 75. a “block” of computation conv -> activate -> pool -> noise convert from 4tensor to normal matrix reshape into conv 4tensor (b, c, 0, 1) format now 4tensor for conv instead of matrix conv weights (n_kernels, n_channels, kernel_w, kerbel_h) highest conv layer has 128 filters and a 3x3 grid of responses A convolutional network in Theano noise during training
  • 76. a “block” of computation conv -> activate -> pool -> noise convert from 4tensor to normal matrix reshape into conv 4tensor (b, c, 0, 1) format now 4tensor for conv instead of matrix conv weights (n_kernels, n_channels, kernel_w, kerbel_h) highest conv layer has 128 filters and a 3x3 grid of responses A convolutional network in Theano noise during training no noise for prediction
  • 77. Test Accuracy: 99.5% What a convolutional network learns
  • 78. Takeaways ● A few tricks are needed to get good results ○ Noise important for regularization ○ Rectifiers for faster, better, learning ○ Don’t use SGD - lots of cheap simple improvements ● Models need room to compute. ● If your data has structure, your model should respect it.
  • 79. Resources ● More in-depth theano tutorials ○ http://www.deeplearning.net/tutorial/ ● Theano docs ○ http://www.deeplearning.net/software/theano/library/ ● Community ○ http://www.reddit.com/r/machinelearning
  • 80. A plug Keep up to date with indico: https://indico1.typeform.com/to/DgN5SP