SlideShare a Scribd company logo
7 steps for highly effective
deep neural networks
Natalino Busa - Head of Data Science
2 Natalino Busa - @natbusa
O’Reilly Author and Speaker
Teradata EMEA Practice Lead on Open Source Technologies
Teradata Principal Data Scientist
ING Group Enterprise Architect: Cybersecurity, Marketing, Fintech
Cognitive Finance Group Advisory Board Member
Philips Senior Researcher, Data Architect
Linkedin and Twitter:
@natbusa
3
Natalino Busa - @natbusa
X: independent variable
Y: dependent variable
Linear Regression:
How to best fit a line to some data
4
Natalino Busa - @natbusa
input : 784 numbers
output: 10 classes
28x28 pixels
Classification: Handwritten digits
5
Natalino Busa - @natbusa
Sharing the (Not so) Secret Lore:
Keras & Tensorflow
Some smaller projects:
Tflearn, Tensorlayers
http://keras.io/
6
Natalino Busa - @natbusa
1: Single Layer Perceptron
“dendrites”
Axon’s
response
Activation function
7
Natalino Busa - @natbusa
More activation functions:
https://en.wikipedia.org/wiki/Activation_function
8
Natalino Busa - @natbusa
Single Layer Neural Network
Takes: n-input features: Map them to a soft “binary” space
∑
x1
x2
xn
f
1: Single Layer Perceptron (binary classifier)
9
Natalino Busa - @natbusa
1: Single Layer Perceptron (multi-class classifier)
Values between 0 and 1
Sum of all outcomes = 1
It produces an estimate of a probability!
From soft binary space to predicting probabilities:
Take n inputs, Divide by the sum of the predicted values
∑
x1
x2
xn
f
∑ f
softmax ‘1’: 95%
‘8’: 5%
10
Natalino Busa - @natbusa
1: Single Layer Perceptron
Minimize costs:
The cost function depends on:
- Parameters of the model
- How the model “composes”
Goal :
Reduce the mean probability error
modify the parameters to reduce the error! Vintage math from last century
11
Natalino Busa - @natbusa
Supervised Learning
Stack layers of perceptrons
- Feed Forward Network
- Scoring goes
from input to output
- Back propagate the error
from output to input
SOFTMAX
Input parameters
classes (estimated probabilities)
Feed-forwardfunctions
Cost function
supervised : actual output
BackPropagateErrors
12
Natalino Busa - @natbusa
Let’s go!
13
Natalino Busa - @natbusa
1. Single Layer Perceptron
14
Natalino Busa - @natbusa
1. Single Layer Perceptron
15
Natalino Busa - @natbusa
1. Single Layer Perceptron
16
Natalino Busa - @natbusa
1. Single Layer Perceptron
17
Natalino Busa - @natbusa
Tensorboard
18
Natalino Busa - @natbusa
Tensorboard
19
Natalino Busa - @natbusa
Tensorboard
20
Natalino Busa - @natbusa
2. Multi Layer Perceptron
21
Natalino Busa - @natbusa
2. Multi Layer Perceptron
22
Natalino Busa - @natbusa
2. Multi Layer Perceptron
23
Natalino Busa - @natbusa
2. Multi Layer Perceptron
24
Natalino Busa - @natbusa
3. Convolution
From Krizehvsky et al. (2012)
25
Natalino Busa - @natbusa
3. Convolution
diagrams:
By Aphex34 - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=45659236
By Aphex34 - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=45673581
CC0, https://en.wikipedia.org/w/index.php?curid=48817276
convolution Max pooling RELU / ELU
26
Natalino Busa - @natbusa
3. Convolution
27
Natalino Busa - @natbusa
3. Convolution
28
Natalino Busa - @natbusa
3. Batch Normalization
29
Natalino Busa - @natbusa
3. Batch Normalization ( example for MLP)
30
Natalino Busa - @natbusa
3. Batch Normalization ( example for MLP)
31
Natalino Busa - @natbusa
3. Batch Normalization ( example for MLP)
Activation function:
SIGMOID
32
Natalino Busa - @natbusa
3. Batch Normalization ( example for MLP)
Activation function:
RELU
33
Natalino Busa - @natbusa
4. Regularization: Prevent overfitting in ANNs
- Batch Normalization
- RELU/ELU
- RESIDUAL / SKIP Networks
- DROP LAYER
- REDUCE PRECISION (HUFFMAN ENCODING)
In general ANN are parameters rich, constraining the
parameter space usually produces better results and speed
up the learning
34
Natalino Busa - @natbusa
5. Inception architectures
Cannot be stacked!
35
Natalino Busa - @natbusa
5. Inception architectures
36
Natalino Busa - @natbusa
5. Inception architectures
compression
Avoid
dimensions
explosion
37
Natalino Busa - @natbusa
5. Inception architectures
38
Natalino Busa - @natbusa
5. Inception architectures (top architecture)
39
Natalino Busa - @natbusa
6. Residual Networks
https://culurciello.github.io/tech/2016/06/04/nets.html
40
Natalino Busa - @natbusa
6. Residual Networks
41
Natalino Busa - @natbusa
6. Residual + Inception Networks
42
Natalino Busa - @natbusa
7. LSTM on Images
43
Natalino Busa - @natbusa
7. LSTM on Images
44
Natalino Busa - @natbusa
7. LSTM on Images
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
45
Natalino Busa - @natbusa
7. LSTM on Images
46
Natalino Busa - @natbusa
7. LSTM on Images
47
Natalino Busa - @natbusa
7. LSTM on ConvNets (bonus slide)
48
Natalino Busa - @natbusa
All the codez :)
https://github.com/natbusa/deepnumbers
49
Natalino Busa - @natbusa
Meta- References
… just a few articles, but extremely dense in content.
A must read!
https://keras.io/
http://karpathy.github.io/neuralnets/
https://culurciello.github.io/tech/2016/06/04/nets.html
http://colah.github.io/posts/2015-08-Understanding-LSTMs/
https://gab41.lab41.org/batch-normalization-what-the-hey-d480039a9e3b

More Related Content

Similar to 7 steps for highly effective deep neural networks

Scaling PyData Up and Out
Scaling PyData Up and OutScaling PyData Up and Out
Scaling PyData Up and Out
Travis Oliphant
 
Fast and Scalable Python
Fast and Scalable PythonFast and Scalable Python
Fast and Scalable Python
Travis Oliphant
 
Scientific Python
Scientific PythonScientific Python
Scientific Python
Eueung Mulyana
 
Puppet 3: Present and Future Tense
Puppet 3: Present and Future TensePuppet 3: Present and Future Tense
Puppet 3: Present and Future Tense
Puppet
 
Puppet 3: Present and Future Tense
Puppet 3: Present and Future TensePuppet 3: Present and Future Tense
Puppet 3: Present and Future Tense
Eric Sorenson
 
Understanding Jupyter notebooks using bioinformatics examples
Understanding Jupyter notebooks using bioinformatics examplesUnderstanding Jupyter notebooks using bioinformatics examples
Understanding Jupyter notebooks using bioinformatics examples
Lynn Langit
 
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
Myungyon Kim
 
Real-Time Anomoly Detection with Spark MLib, Akka and Cassandra by Natalino Busa
Real-Time Anomoly Detection with Spark MLib, Akka and Cassandra by Natalino BusaReal-Time Anomoly Detection with Spark MLib, Akka and Cassandra by Natalino Busa
Real-Time Anomoly Detection with Spark MLib, Akka and Cassandra by Natalino Busa
Spark Summit
 
2951085 dzone-2016guidetobigdata
2951085 dzone-2016guidetobigdata2951085 dzone-2016guidetobigdata
2951085 dzone-2016guidetobigdata
balu kvm
 
Introduction to Neural Networks in Tensorflow
Introduction to Neural Networks in TensorflowIntroduction to Neural Networks in Tensorflow
Introduction to Neural Networks in Tensorflow
Nicholas McClure
 
Python coroutine
Python coroutinePython coroutine
Python coroutine
경섭 심
 
Espresso: LinkedIn's Distributed Data Serving Platform (Talk)
Espresso: LinkedIn's Distributed Data Serving Platform (Talk)Espresso: LinkedIn's Distributed Data Serving Platform (Talk)
Espresso: LinkedIn's Distributed Data Serving Platform (Talk)
Amy W. Tang
 
Using Deep Learning in Production Pipelines to Predict Consumers’ Interest wi...
Using Deep Learning in Production Pipelines to Predict Consumers’ Interest wi...Using Deep Learning in Production Pipelines to Predict Consumers’ Interest wi...
Using Deep Learning in Production Pipelines to Predict Consumers’ Interest wi...
Databricks
 
Large scale social networks analysis joclad 2013
Large scale social networks analysis   joclad 2013Large scale social networks analysis   joclad 2013
Large scale social networks analysis joclad 2013
Rui Sarmento
 
Machine Learning with Spark
Machine Learning with SparkMachine Learning with Spark
Machine Learning with Spark
elephantscale
 
Startup.Ml: Using neon for NLP and Localization Applications
Startup.Ml: Using neon for NLP and Localization Applications Startup.Ml: Using neon for NLP and Localization Applications
Startup.Ml: Using neon for NLP and Localization Applications
Intel Nervana
 
A Platform for Accelerating Machine Learning Applications
 A Platform for Accelerating Machine Learning Applications A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning Applications
NVIDIA Taiwan
 
ドキュメントを作りたくなってしまう魔法のツールSphinx
ドキュメントを作りたくなってしまう魔法のツールSphinxドキュメントを作りたくなってしまう魔法のツールSphinx
ドキュメントを作りたくなってしまう魔法のツールSphinx
Takayuki Shimizukawa
 
11_Saloni Malhotra_SummerTraining_PPT.pptx
11_Saloni Malhotra_SummerTraining_PPT.pptx11_Saloni Malhotra_SummerTraining_PPT.pptx
11_Saloni Malhotra_SummerTraining_PPT.pptx
SaloniMalhotra23
 
Basic of python for data analysis
Basic of python for data analysisBasic of python for data analysis
Basic of python for data analysis
Pramod Toraskar
 

Similar to 7 steps for highly effective deep neural networks (20)

Scaling PyData Up and Out
Scaling PyData Up and OutScaling PyData Up and Out
Scaling PyData Up and Out
 
Fast and Scalable Python
Fast and Scalable PythonFast and Scalable Python
Fast and Scalable Python
 
Scientific Python
Scientific PythonScientific Python
Scientific Python
 
Puppet 3: Present and Future Tense
Puppet 3: Present and Future TensePuppet 3: Present and Future Tense
Puppet 3: Present and Future Tense
 
Puppet 3: Present and Future Tense
Puppet 3: Present and Future TensePuppet 3: Present and Future Tense
Puppet 3: Present and Future Tense
 
Understanding Jupyter notebooks using bioinformatics examples
Understanding Jupyter notebooks using bioinformatics examplesUnderstanding Jupyter notebooks using bioinformatics examples
Understanding Jupyter notebooks using bioinformatics examples
 
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...
 
Real-Time Anomoly Detection with Spark MLib, Akka and Cassandra by Natalino Busa
Real-Time Anomoly Detection with Spark MLib, Akka and Cassandra by Natalino BusaReal-Time Anomoly Detection with Spark MLib, Akka and Cassandra by Natalino Busa
Real-Time Anomoly Detection with Spark MLib, Akka and Cassandra by Natalino Busa
 
2951085 dzone-2016guidetobigdata
2951085 dzone-2016guidetobigdata2951085 dzone-2016guidetobigdata
2951085 dzone-2016guidetobigdata
 
Introduction to Neural Networks in Tensorflow
Introduction to Neural Networks in TensorflowIntroduction to Neural Networks in Tensorflow
Introduction to Neural Networks in Tensorflow
 
Python coroutine
Python coroutinePython coroutine
Python coroutine
 
Espresso: LinkedIn's Distributed Data Serving Platform (Talk)
Espresso: LinkedIn's Distributed Data Serving Platform (Talk)Espresso: LinkedIn's Distributed Data Serving Platform (Talk)
Espresso: LinkedIn's Distributed Data Serving Platform (Talk)
 
Using Deep Learning in Production Pipelines to Predict Consumers’ Interest wi...
Using Deep Learning in Production Pipelines to Predict Consumers’ Interest wi...Using Deep Learning in Production Pipelines to Predict Consumers’ Interest wi...
Using Deep Learning in Production Pipelines to Predict Consumers’ Interest wi...
 
Large scale social networks analysis joclad 2013
Large scale social networks analysis   joclad 2013Large scale social networks analysis   joclad 2013
Large scale social networks analysis joclad 2013
 
Machine Learning with Spark
Machine Learning with SparkMachine Learning with Spark
Machine Learning with Spark
 
Startup.Ml: Using neon for NLP and Localization Applications
Startup.Ml: Using neon for NLP and Localization Applications Startup.Ml: Using neon for NLP and Localization Applications
Startup.Ml: Using neon for NLP and Localization Applications
 
A Platform for Accelerating Machine Learning Applications
 A Platform for Accelerating Machine Learning Applications A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning Applications
 
ドキュメントを作りたくなってしまう魔法のツールSphinx
ドキュメントを作りたくなってしまう魔法のツールSphinxドキュメントを作りたくなってしまう魔法のツールSphinx
ドキュメントを作りたくなってしまう魔法のツールSphinx
 
11_Saloni Malhotra_SummerTraining_PPT.pptx
11_Saloni Malhotra_SummerTraining_PPT.pptx11_Saloni Malhotra_SummerTraining_PPT.pptx
11_Saloni Malhotra_SummerTraining_PPT.pptx
 
Basic of python for data analysis
Basic of python for data analysisBasic of python for data analysis
Basic of python for data analysis
 

More from Natalino Busa

Data Production Pipelines: Legacy, practices, and innovation
Data Production Pipelines: Legacy, practices, and innovationData Production Pipelines: Legacy, practices, and innovation
Data Production Pipelines: Legacy, practices, and innovation
Natalino Busa
 
[Ai in finance] AI in regulatory compliance, risk management, and auditing
[Ai in finance] AI in regulatory compliance, risk management, and auditing[Ai in finance] AI in regulatory compliance, risk management, and auditing
[Ai in finance] AI in regulatory compliance, risk management, and auditing
Natalino Busa
 
Strata London 16: sightseeing, venues, and friends
Strata  London 16: sightseeing, venues, and friendsStrata  London 16: sightseeing, venues, and friends
Strata London 16: sightseeing, venues, and friends
Natalino Busa
 
Data in Action
Data in ActionData in Action
Data in Action
Natalino Busa
 
Real-Time Anomaly Detection with Spark MLlib, Akka and Cassandra
Real-Time Anomaly Detection  with Spark MLlib, Akka and  CassandraReal-Time Anomaly Detection  with Spark MLlib, Akka and  Cassandra
Real-Time Anomaly Detection with Spark MLlib, Akka and Cassandra
Natalino Busa
 
The evolution of data analytics
The evolution of data analyticsThe evolution of data analytics
The evolution of data analytics
Natalino Busa
 
Towards Real-Time banking API's: Introducing Coral, a web api for realtime st...
Towards Real-Time banking API's: Introducing Coral, a web api for realtime st...Towards Real-Time banking API's: Introducing Coral, a web api for realtime st...
Towards Real-Time banking API's: Introducing Coral, a web api for realtime st...
Natalino Busa
 
Streaming Api Design with Akka, Scala and Spray
Streaming Api Design with Akka, Scala and SprayStreaming Api Design with Akka, Scala and Spray
Streaming Api Design with Akka, Scala and Spray
Natalino Busa
 
Hadoop + Cassandra: Fast queries on data lakes, and wikipedia search tutorial.
Hadoop + Cassandra: Fast queries on data lakes, and  wikipedia search tutorial.Hadoop + Cassandra: Fast queries on data lakes, and  wikipedia search tutorial.
Hadoop + Cassandra: Fast queries on data lakes, and wikipedia search tutorial.
Natalino Busa
 
Big data solutions for advanced marketing analytics
Big data solutions for advanced marketing analyticsBig data solutions for advanced marketing analytics
Big data solutions for advanced marketing analytics
Natalino Busa
 
Awesome Banking API's
Awesome Banking API'sAwesome Banking API's
Awesome Banking API's
Natalino Busa
 
Yo. big data. understanding data science in the era of big data.
Yo. big data. understanding data science in the era of big data.Yo. big data. understanding data science in the era of big data.
Yo. big data. understanding data science in the era of big data.
Natalino Busa
 
Big and fast a quest for relevant and real-time analytics
Big and fast a quest for relevant and real-time analyticsBig and fast a quest for relevant and real-time analytics
Big and fast a quest for relevant and real-time analytics
Natalino Busa
 
Big Data and APIs - a recon tour on how to successfully do Big Data analytics
Big Data and APIs - a recon tour on how to successfully do Big Data analyticsBig Data and APIs - a recon tour on how to successfully do Big Data analytics
Big Data and APIs - a recon tour on how to successfully do Big Data analytics
Natalino Busa
 
Strata 2014: Data science and big data trending topics
Strata 2014: Data science and big data trending topicsStrata 2014: Data science and big data trending topics
Strata 2014: Data science and big data trending topics
Natalino Busa
 
Streaming computing: architectures, and tchnologies
Streaming computing: architectures, and tchnologiesStreaming computing: architectures, and tchnologies
Streaming computing: architectures, and tchnologies
Natalino Busa
 
Big data landscape
Big data landscapeBig data landscape
Big data landscape
Natalino Busa
 

More from Natalino Busa (17)

Data Production Pipelines: Legacy, practices, and innovation
Data Production Pipelines: Legacy, practices, and innovationData Production Pipelines: Legacy, practices, and innovation
Data Production Pipelines: Legacy, practices, and innovation
 
[Ai in finance] AI in regulatory compliance, risk management, and auditing
[Ai in finance] AI in regulatory compliance, risk management, and auditing[Ai in finance] AI in regulatory compliance, risk management, and auditing
[Ai in finance] AI in regulatory compliance, risk management, and auditing
 
Strata London 16: sightseeing, venues, and friends
Strata  London 16: sightseeing, venues, and friendsStrata  London 16: sightseeing, venues, and friends
Strata London 16: sightseeing, venues, and friends
 
Data in Action
Data in ActionData in Action
Data in Action
 
Real-Time Anomaly Detection with Spark MLlib, Akka and Cassandra
Real-Time Anomaly Detection  with Spark MLlib, Akka and  CassandraReal-Time Anomaly Detection  with Spark MLlib, Akka and  Cassandra
Real-Time Anomaly Detection with Spark MLlib, Akka and Cassandra
 
The evolution of data analytics
The evolution of data analyticsThe evolution of data analytics
The evolution of data analytics
 
Towards Real-Time banking API's: Introducing Coral, a web api for realtime st...
Towards Real-Time banking API's: Introducing Coral, a web api for realtime st...Towards Real-Time banking API's: Introducing Coral, a web api for realtime st...
Towards Real-Time banking API's: Introducing Coral, a web api for realtime st...
 
Streaming Api Design with Akka, Scala and Spray
Streaming Api Design with Akka, Scala and SprayStreaming Api Design with Akka, Scala and Spray
Streaming Api Design with Akka, Scala and Spray
 
Hadoop + Cassandra: Fast queries on data lakes, and wikipedia search tutorial.
Hadoop + Cassandra: Fast queries on data lakes, and  wikipedia search tutorial.Hadoop + Cassandra: Fast queries on data lakes, and  wikipedia search tutorial.
Hadoop + Cassandra: Fast queries on data lakes, and wikipedia search tutorial.
 
Big data solutions for advanced marketing analytics
Big data solutions for advanced marketing analyticsBig data solutions for advanced marketing analytics
Big data solutions for advanced marketing analytics
 
Awesome Banking API's
Awesome Banking API'sAwesome Banking API's
Awesome Banking API's
 
Yo. big data. understanding data science in the era of big data.
Yo. big data. understanding data science in the era of big data.Yo. big data. understanding data science in the era of big data.
Yo. big data. understanding data science in the era of big data.
 
Big and fast a quest for relevant and real-time analytics
Big and fast a quest for relevant and real-time analyticsBig and fast a quest for relevant and real-time analytics
Big and fast a quest for relevant and real-time analytics
 
Big Data and APIs - a recon tour on how to successfully do Big Data analytics
Big Data and APIs - a recon tour on how to successfully do Big Data analyticsBig Data and APIs - a recon tour on how to successfully do Big Data analytics
Big Data and APIs - a recon tour on how to successfully do Big Data analytics
 
Strata 2014: Data science and big data trending topics
Strata 2014: Data science and big data trending topicsStrata 2014: Data science and big data trending topics
Strata 2014: Data science and big data trending topics
 
Streaming computing: architectures, and tchnologies
Streaming computing: architectures, and tchnologiesStreaming computing: architectures, and tchnologies
Streaming computing: architectures, and tchnologies
 
Big data landscape
Big data landscapeBig data landscape
Big data landscape
 

Recently uploaded

一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
bmucuha
 
Module 1 ppt BIG DATA ANALYTICS_NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS_NOTES FOR MCAModule 1 ppt BIG DATA ANALYTICS_NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS_NOTES FOR MCA
yuvarajkumar334
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
eudsoh
 
一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理
keesa2
 
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
osoyvvf
 
Build applications with generative AI on Google Cloud
Build applications with generative AI on Google CloudBuild applications with generative AI on Google Cloud
Build applications with generative AI on Google Cloud
Márton Kodok
 
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
oaxefes
 
社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .
NABLAS株式会社
 
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
9gr6pty
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
Vietnam Cotton & Spinning Association
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
Vietnam Cotton & Spinning Association
 
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
aguty
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
ElizabethGarrettChri
 
A gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented GenerationA gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented Generation
dataschool1
 
Digital Marketing Performance Marketing Sample .pdf
Digital Marketing Performance Marketing  Sample .pdfDigital Marketing Performance Marketing  Sample .pdf
Digital Marketing Performance Marketing Sample .pdf
Vineet
 
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
ML-PPT-UNIT-2 Generative Classifiers Discriminative ClassifiersML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
MastanaihnaiduYasam
 
Cell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docxCell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docx
vasanthatpuram
 
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
ytypuem
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
ihavuls
 
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
actyx
 

Recently uploaded (20)

一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理一比一原版(UO毕业证)渥太华大学毕业证如何办理
一比一原版(UO毕业证)渥太华大学毕业证如何办理
 
Module 1 ppt BIG DATA ANALYTICS_NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS_NOTES FOR MCAModule 1 ppt BIG DATA ANALYTICS_NOTES FOR MCA
Module 1 ppt BIG DATA ANALYTICS_NOTES FOR MCA
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
 
一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理
 
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
一比一原版(uom毕业证书)曼彻斯特大学毕业证如何办理
 
Build applications with generative AI on Google Cloud
Build applications with generative AI on Google CloudBuild applications with generative AI on Google Cloud
Build applications with generative AI on Google Cloud
 
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
一比一原版卡尔加里大学毕业证(uc毕业证)如何办理
 
社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .社内勉強会資料_Hallucination of LLMs               .
社内勉強会資料_Hallucination of LLMs               .
 
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
一比一原版(uob毕业证书)伯明翰大学毕业证如何办理
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics March 2024
 
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
一比一原版澳洲西澳大学毕业证(uwa毕业证书)如何办理
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
 
A gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented GenerationA gentle exploration of Retrieval Augmented Generation
A gentle exploration of Retrieval Augmented Generation
 
Digital Marketing Performance Marketing Sample .pdf
Digital Marketing Performance Marketing  Sample .pdfDigital Marketing Performance Marketing  Sample .pdf
Digital Marketing Performance Marketing Sample .pdf
 
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
ML-PPT-UNIT-2 Generative Classifiers Discriminative ClassifiersML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
ML-PPT-UNIT-2 Generative Classifiers Discriminative Classifiers
 
Cell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docxCell The Unit of Life for NEET Multiple Choice Questions.docx
Cell The Unit of Life for NEET Multiple Choice Questions.docx
 
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
 
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
 

7 steps for highly effective deep neural networks

  • 1. 7 steps for highly effective deep neural networks Natalino Busa - Head of Data Science
  • 2. 2 Natalino Busa - @natbusa O’Reilly Author and Speaker Teradata EMEA Practice Lead on Open Source Technologies Teradata Principal Data Scientist ING Group Enterprise Architect: Cybersecurity, Marketing, Fintech Cognitive Finance Group Advisory Board Member Philips Senior Researcher, Data Architect Linkedin and Twitter: @natbusa
  • 3. 3 Natalino Busa - @natbusa X: independent variable Y: dependent variable Linear Regression: How to best fit a line to some data
  • 4. 4 Natalino Busa - @natbusa input : 784 numbers output: 10 classes 28x28 pixels Classification: Handwritten digits
  • 5. 5 Natalino Busa - @natbusa Sharing the (Not so) Secret Lore: Keras & Tensorflow Some smaller projects: Tflearn, Tensorlayers http://keras.io/
  • 6. 6 Natalino Busa - @natbusa 1: Single Layer Perceptron “dendrites” Axon’s response Activation function
  • 7. 7 Natalino Busa - @natbusa More activation functions: https://en.wikipedia.org/wiki/Activation_function
  • 8. 8 Natalino Busa - @natbusa Single Layer Neural Network Takes: n-input features: Map them to a soft “binary” space ∑ x1 x2 xn f 1: Single Layer Perceptron (binary classifier)
  • 9. 9 Natalino Busa - @natbusa 1: Single Layer Perceptron (multi-class classifier) Values between 0 and 1 Sum of all outcomes = 1 It produces an estimate of a probability! From soft binary space to predicting probabilities: Take n inputs, Divide by the sum of the predicted values ∑ x1 x2 xn f ∑ f softmax ‘1’: 95% ‘8’: 5%
  • 10. 10 Natalino Busa - @natbusa 1: Single Layer Perceptron Minimize costs: The cost function depends on: - Parameters of the model - How the model “composes” Goal : Reduce the mean probability error modify the parameters to reduce the error! Vintage math from last century
  • 11. 11 Natalino Busa - @natbusa Supervised Learning Stack layers of perceptrons - Feed Forward Network - Scoring goes from input to output - Back propagate the error from output to input SOFTMAX Input parameters classes (estimated probabilities) Feed-forwardfunctions Cost function supervised : actual output BackPropagateErrors
  • 12. 12 Natalino Busa - @natbusa Let’s go!
  • 13. 13 Natalino Busa - @natbusa 1. Single Layer Perceptron
  • 14. 14 Natalino Busa - @natbusa 1. Single Layer Perceptron
  • 15. 15 Natalino Busa - @natbusa 1. Single Layer Perceptron
  • 16. 16 Natalino Busa - @natbusa 1. Single Layer Perceptron
  • 17. 17 Natalino Busa - @natbusa Tensorboard
  • 18. 18 Natalino Busa - @natbusa Tensorboard
  • 19. 19 Natalino Busa - @natbusa Tensorboard
  • 20. 20 Natalino Busa - @natbusa 2. Multi Layer Perceptron
  • 21. 21 Natalino Busa - @natbusa 2. Multi Layer Perceptron
  • 22. 22 Natalino Busa - @natbusa 2. Multi Layer Perceptron
  • 23. 23 Natalino Busa - @natbusa 2. Multi Layer Perceptron
  • 24. 24 Natalino Busa - @natbusa 3. Convolution From Krizehvsky et al. (2012)
  • 25. 25 Natalino Busa - @natbusa 3. Convolution diagrams: By Aphex34 - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=45659236 By Aphex34 - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=45673581 CC0, https://en.wikipedia.org/w/index.php?curid=48817276 convolution Max pooling RELU / ELU
  • 26. 26 Natalino Busa - @natbusa 3. Convolution
  • 27. 27 Natalino Busa - @natbusa 3. Convolution
  • 28. 28 Natalino Busa - @natbusa 3. Batch Normalization
  • 29. 29 Natalino Busa - @natbusa 3. Batch Normalization ( example for MLP)
  • 30. 30 Natalino Busa - @natbusa 3. Batch Normalization ( example for MLP)
  • 31. 31 Natalino Busa - @natbusa 3. Batch Normalization ( example for MLP) Activation function: SIGMOID
  • 32. 32 Natalino Busa - @natbusa 3. Batch Normalization ( example for MLP) Activation function: RELU
  • 33. 33 Natalino Busa - @natbusa 4. Regularization: Prevent overfitting in ANNs - Batch Normalization - RELU/ELU - RESIDUAL / SKIP Networks - DROP LAYER - REDUCE PRECISION (HUFFMAN ENCODING) In general ANN are parameters rich, constraining the parameter space usually produces better results and speed up the learning
  • 34. 34 Natalino Busa - @natbusa 5. Inception architectures Cannot be stacked!
  • 35. 35 Natalino Busa - @natbusa 5. Inception architectures
  • 36. 36 Natalino Busa - @natbusa 5. Inception architectures compression Avoid dimensions explosion
  • 37. 37 Natalino Busa - @natbusa 5. Inception architectures
  • 38. 38 Natalino Busa - @natbusa 5. Inception architectures (top architecture)
  • 39. 39 Natalino Busa - @natbusa 6. Residual Networks https://culurciello.github.io/tech/2016/06/04/nets.html
  • 40. 40 Natalino Busa - @natbusa 6. Residual Networks
  • 41. 41 Natalino Busa - @natbusa 6. Residual + Inception Networks
  • 42. 42 Natalino Busa - @natbusa 7. LSTM on Images
  • 43. 43 Natalino Busa - @natbusa 7. LSTM on Images
  • 44. 44 Natalino Busa - @natbusa 7. LSTM on Images http://colah.github.io/posts/2015-08-Understanding-LSTMs/
  • 45. 45 Natalino Busa - @natbusa 7. LSTM on Images
  • 46. 46 Natalino Busa - @natbusa 7. LSTM on Images
  • 47. 47 Natalino Busa - @natbusa 7. LSTM on ConvNets (bonus slide)
  • 48. 48 Natalino Busa - @natbusa All the codez :) https://github.com/natbusa/deepnumbers
  • 49. 49 Natalino Busa - @natbusa Meta- References … just a few articles, but extremely dense in content. A must read! https://keras.io/ http://karpathy.github.io/neuralnets/ https://culurciello.github.io/tech/2016/06/04/nets.html http://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://gab41.lab41.org/batch-normalization-what-the-hey-d480039a9e3b