SlideShare a Scribd company logo
Deep Learning
Jin Sakuma
Neural Network
● Neural network is what in the deep learning
● Designed to simulate human brain
● Consist of several layers and data will be passed
one by one
Input Layer
Hidden Layer
Hidden Layer
Output Layer
Data Input
Data Output
Perceptron
● In one perceptron, output value is computed by
● Where xi is the input, wi is weight and f is activation function
y
x1
x2
x3
f z
One Layer
● For input vector x, output of one layer is computed by
y = Wx+b
z = f(y)
Where W is weight of the layer, b is bias, f is activation function.
● W and b are parameters because we will modify
them to optimize the model
Activation Function
● Without activation function, multiple layers would be meaningless
● Good activation function is non-linear, differentiable, monotonically increasing.
● Logistic function
● Hyperbolic Tangent
● Retified linear function (ReLU)
For Various Problems
● The activation function of the output layer depends on the type of problem
● For regression:
○ Activation function: Identity function
○ Length of output vector: Any
● For binary classification:
○ Activation function: Logistic function
○ Length of output vector: 1
● For multi-class classification:
Example Task
● Task setting:
○ Given picture of hand-written numbers 0-9, we want to tell which number
it is
○ Training dataset consist of a lot of pictures and all picture is labeled with
correct answer.
● Analyze task:
○ Problem type: Multi-class classification
○ Activation function of output layer: Softmax function
Learn: Minimize Error
● Consider error function E(xi, di; W, b) which represents how off the model is
from the true value for the ith picture. Here xi is the vector representation of
picture.
● In our example we use
Here, dik is 1 only if xi is actually picture of number k and otherwise 0.
● We want to modify W and b to minimize error function.
Learn: Gradient Descent
We use Gradient Descent to modify parameters W and b.
● Consider yourself in mountain and willing to reach the top of mountain, but you
lost map and you can’t look distance because of smog. How do you reach the
top?
➔Move to direction that bring you to the highest.
● The vector that indicate the direction to move is called gradient
● Since we want to minimize (instead of maximize), we update parameters by
subtracting gradient.
Learn: Backpropagation
● Neural network with multiple layers are too complex to
simply compute gradients.
● This problem was one bottleneck in early stage of
development of deep learning.
● Backpropagation compute gradient from output layer
to input layer (A lot of chain rules).
Input Layer
Hidden Layer
Hidden Layer
Output Layer
Data Input
Data Output
Why Deep Learning?
● Deep learning has tons of parameters (things that we can change to optimize
the model)
➔Better accuracy
➔Hard to optimize
➔Need a lot of data
● Flexible model
➔Can be used to different types of problems
➔Easy to modify models for various situations
Experiment
● Run python script for the example task
● Input vector is given as vector of length 784
● Configurations are
○ 2 hidden layers
○ 1000 perceptrons for each layers
○ ReLU function for activation function
○ Softmax function for activation function for the output layer
○ Batch size: 100
Result
● For this experiment, I used example code of chainer
● Execution time: about 45min
● Final Validation Loss: 0.107
● Final Validation Accuracy: 0.98
Use case of Deep Learning
● Convolutional Neural Network: Deep learning specific to picture data
○ Object identification
○ Face recognition
● Recurrent Neural Network: Deep learning for sequential data
○ Speech recognition
○ For text
● DQN: Combination of deep learning and Q learning
○ Alpha Go uses DQN and won top level Go player

More Related Content

What's hot

Accelerated Logistic Regression on GPU(s)
Accelerated Logistic Regression on GPU(s)Accelerated Logistic Regression on GPU(s)
Accelerated Logistic Regression on GPU(s)
RAHUL BHOJWANI
 
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Universitat Politècnica de Catalunya
 
Transfer Learning and Domain Adaptation (DLAI D5L2 2017 UPC Deep Learning for...
Transfer Learning and Domain Adaptation (DLAI D5L2 2017 UPC Deep Learning for...Transfer Learning and Domain Adaptation (DLAI D5L2 2017 UPC Deep Learning for...
Transfer Learning and Domain Adaptation (DLAI D5L2 2017 UPC Deep Learning for...
Universitat Politècnica de Catalunya
 
Optimization for Deep Networks (D2L1 2017 UPC Deep Learning for Computer Vision)
Optimization for Deep Networks (D2L1 2017 UPC Deep Learning for Computer Vision)Optimization for Deep Networks (D2L1 2017 UPC Deep Learning for Computer Vision)
Optimization for Deep Networks (D2L1 2017 UPC Deep Learning for Computer Vision)
Universitat Politècnica de Catalunya
 
Set Transfomer: A Framework for Attention-based Permutaion-Invariant Neural N...
Set Transfomer: A Framework for Attention-based Permutaion-Invariant Neural N...Set Transfomer: A Framework for Attention-based Permutaion-Invariant Neural N...
Set Transfomer: A Framework for Attention-based Permutaion-Invariant Neural N...
Thien Q. Tran
 
Deep Learning for Computer Vision: Deep Networks (UPC 2016)
Deep Learning for Computer Vision: Deep Networks (UPC 2016)Deep Learning for Computer Vision: Deep Networks (UPC 2016)
Deep Learning for Computer Vision: Deep Networks (UPC 2016)
Universitat Politècnica de Catalunya
 
Optimizing Deep Networks (D1L6 Insight@DCU Machine Learning Workshop 2017)
Optimizing Deep Networks (D1L6 Insight@DCU Machine Learning Workshop 2017)Optimizing Deep Networks (D1L6 Insight@DCU Machine Learning Workshop 2017)
Optimizing Deep Networks (D1L6 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
Semantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite ImagerySemantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite Imagery
RAHUL BHOJWANI
 
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic...
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic...Quantization and Training of Neural Networks for Efficient Integer-Arithmetic...
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic...
Ryo Takahashi
 
Transfer Learning and Domain Adaptation - Ramon Morros - UPC Barcelona 2018
Transfer Learning and Domain Adaptation - Ramon Morros - UPC Barcelona 2018Transfer Learning and Domain Adaptation - Ramon Morros - UPC Barcelona 2018
Transfer Learning and Domain Adaptation - Ramon Morros - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
Universitat Politècnica de Catalunya
 
Centernet
CenternetCenternet
Centernet
Arithmer Inc.
 
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Universitat Politècnica de Catalunya
 
Training Deep Networks with Backprop (D1L4 Insight@DCU Machine Learning Works...
Training Deep Networks with Backprop (D1L4 Insight@DCU Machine Learning Works...Training Deep Networks with Backprop (D1L4 Insight@DCU Machine Learning Works...
Training Deep Networks with Backprop (D1L4 Insight@DCU Machine Learning Works...
Universitat Politècnica de Catalunya
 
Deep Neural Networks (D1L2 Insight@DCU Machine Learning Workshop 2017)
Deep Neural Networks (D1L2 Insight@DCU Machine Learning Workshop 2017)Deep Neural Networks (D1L2 Insight@DCU Machine Learning Workshop 2017)
Deep Neural Networks (D1L2 Insight@DCU Machine Learning Workshop 2017)
Universitat Politècnica de Catalunya
 
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
Skip RNN: Learning to Skip State Updates in Recurrent Neural NetworksSkip RNN: Learning to Skip State Updates in Recurrent Neural Networks
Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
Universitat Politècnica de Catalunya
 
Deep Learning for Computer Vision: Backward Propagation (UPC 2016)
Deep Learning for Computer Vision: Backward Propagation (UPC 2016)Deep Learning for Computer Vision: Backward Propagation (UPC 2016)
Deep Learning for Computer Vision: Backward Propagation (UPC 2016)
Universitat Politècnica de Catalunya
 
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
Universitat Politècnica de Catalunya
 

What's hot (20)

Accelerated Logistic Regression on GPU(s)
Accelerated Logistic Regression on GPU(s)Accelerated Logistic Regression on GPU(s)
Accelerated Logistic Regression on GPU(s)
 
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
 
Transfer Learning and Domain Adaptation (DLAI D5L2 2017 UPC Deep Learning for...
Transfer Learning and Domain Adaptation (DLAI D5L2 2017 UPC Deep Learning for...Transfer Learning and Domain Adaptation (DLAI D5L2 2017 UPC Deep Learning for...
Transfer Learning and Domain Adaptation (DLAI D5L2 2017 UPC Deep Learning for...
 
Optimization for Deep Networks (D2L1 2017 UPC Deep Learning for Computer Vision)
Optimization for Deep Networks (D2L1 2017 UPC Deep Learning for Computer Vision)Optimization for Deep Networks (D2L1 2017 UPC Deep Learning for Computer Vision)
Optimization for Deep Networks (D2L1 2017 UPC Deep Learning for Computer Vision)
 
Set Transfomer: A Framework for Attention-based Permutaion-Invariant Neural N...
Set Transfomer: A Framework for Attention-based Permutaion-Invariant Neural N...Set Transfomer: A Framework for Attention-based Permutaion-Invariant Neural N...
Set Transfomer: A Framework for Attention-based Permutaion-Invariant Neural N...
 
Deep Learning for Computer Vision: Deep Networks (UPC 2016)
Deep Learning for Computer Vision: Deep Networks (UPC 2016)Deep Learning for Computer Vision: Deep Networks (UPC 2016)
Deep Learning for Computer Vision: Deep Networks (UPC 2016)
 
Optimizing Deep Networks (D1L6 Insight@DCU Machine Learning Workshop 2017)
Optimizing Deep Networks (D1L6 Insight@DCU Machine Learning Workshop 2017)Optimizing Deep Networks (D1L6 Insight@DCU Machine Learning Workshop 2017)
Optimizing Deep Networks (D1L6 Insight@DCU Machine Learning Workshop 2017)
 
Semantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite ImagerySemantic Segmentation on Satellite Imagery
Semantic Segmentation on Satellite Imagery
 
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
 
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic...
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic...Quantization and Training of Neural Networks for Efficient Integer-Arithmetic...
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic...
 
Transfer Learning and Domain Adaptation - Ramon Morros - UPC Barcelona 2018
Transfer Learning and Domain Adaptation - Ramon Morros - UPC Barcelona 2018Transfer Learning and Domain Adaptation - Ramon Morros - UPC Barcelona 2018
Transfer Learning and Domain Adaptation - Ramon Morros - UPC Barcelona 2018
 
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
Convolutional Neural Networks (D1L3 2017 UPC Deep Learning for Computer Vision)
 
Centernet
CenternetCenternet
Centernet
 
Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)Deep Learning for Computer Vision: Attention Models (UPC 2016)
Deep Learning for Computer Vision: Attention Models (UPC 2016)
 
Training Deep Networks with Backprop (D1L4 Insight@DCU Machine Learning Works...
Training Deep Networks with Backprop (D1L4 Insight@DCU Machine Learning Works...Training Deep Networks with Backprop (D1L4 Insight@DCU Machine Learning Works...
Training Deep Networks with Backprop (D1L4 Insight@DCU Machine Learning Works...
 
Deep Neural Networks (D1L2 Insight@DCU Machine Learning Workshop 2017)
Deep Neural Networks (D1L2 Insight@DCU Machine Learning Workshop 2017)Deep Neural Networks (D1L2 Insight@DCU Machine Learning Workshop 2017)
Deep Neural Networks (D1L2 Insight@DCU Machine Learning Workshop 2017)
 
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018Backpropagation - Elisa Sayrol - UPC Barcelona 2018
Backpropagation - Elisa Sayrol - UPC Barcelona 2018
 
Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
Skip RNN: Learning to Skip State Updates in Recurrent Neural NetworksSkip RNN: Learning to Skip State Updates in Recurrent Neural Networks
Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
 
Deep Learning for Computer Vision: Backward Propagation (UPC 2016)
Deep Learning for Computer Vision: Backward Propagation (UPC 2016)Deep Learning for Computer Vision: Backward Propagation (UPC 2016)
Deep Learning for Computer Vision: Backward Propagation (UPC 2016)
 
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
Generative Models and Adversarial Training (D2L3 Insight@DCU Machine Learning...
 

Viewers also liked

No 2º Período foram desenvolvidos trabalhos subordinados aos temas "Moda" e "...
No 2º Período foram desenvolvidos trabalhos subordinados aos temas "Moda" e "...No 2º Período foram desenvolvidos trabalhos subordinados aos temas "Moda" e "...
No 2º Período foram desenvolvidos trabalhos subordinados aos temas "Moda" e "...guest486a60
 
Portafolio fotocopias yenny
Portafolio fotocopias yennyPortafolio fotocopias yenny
Portafolio fotocopias yenny
copiasyenny
 
WMD
WMDWMD
South korea tablet listening
South korea tablet listeningSouth korea tablet listening
South korea tablet listeningMeagan Kaiser
 
Prwr625 bizplan menu
Prwr625 bizplan menuPrwr625 bizplan menu
Prwr625 bizplan menu
keythjones
 
8 aula 6 república velha
8   aula 6 república velha8   aula 6 república velha
8 aula 6 república velhaprofdu
 
шинагулова даметай+одежда для дома+создать магазин одежды для дома
шинагулова даметай+одежда для дома+создать магазин одежды для домашинагулова даметай+одежда для дома+создать магазин одежды для дома
шинагулова даметай+одежда для дома+создать магазин одежды для дома
Даметай Шинагулова
 
20151107 - The Influences on Motivation in Online Educational Environments
20151107 - The Influences on Motivation in Online Educational Environments20151107 - The Influences on Motivation in Online Educational Environments
20151107 - The Influences on Motivation in Online Educational EnvironmentsWilliam Harding
 
Europa e Europeus - Celtas
Europa e Europeus - CeltasEuropa e Europeus - Celtas
Europa e Europeus - Celtas
Carlos Ribeiro Medeiros
 
Stock and Share Market Technical Analysis
Stock and Share Market Technical AnalysisStock and Share Market Technical Analysis
Stock and Share Market Technical Analysis
Dhanashri Academy
 
The back of the napkin Presentation
The back of the napkin PresentationThe back of the napkin Presentation
The back of the napkin Presentation
VM87
 

Viewers also liked (14)

No 2º Período foram desenvolvidos trabalhos subordinados aos temas "Moda" e "...
No 2º Período foram desenvolvidos trabalhos subordinados aos temas "Moda" e "...No 2º Período foram desenvolvidos trabalhos subordinados aos temas "Moda" e "...
No 2º Período foram desenvolvidos trabalhos subordinados aos temas "Moda" e "...
 
Portafolio fotocopias yenny
Portafolio fotocopias yennyPortafolio fotocopias yenny
Portafolio fotocopias yenny
 
WMD
WMDWMD
WMD
 
South korea tablet listening
South korea tablet listeningSouth korea tablet listening
South korea tablet listening
 
Trucha
TruchaTrucha
Trucha
 
Prwr625 bizplan menu
Prwr625 bizplan menuPrwr625 bizplan menu
Prwr625 bizplan menu
 
8 aula 6 república velha
8   aula 6 república velha8   aula 6 república velha
8 aula 6 república velha
 
Mono lds2012
Mono lds2012Mono lds2012
Mono lds2012
 
GOVT2305SyllabusHCC
GOVT2305SyllabusHCCGOVT2305SyllabusHCC
GOVT2305SyllabusHCC
 
шинагулова даметай+одежда для дома+создать магазин одежды для дома
шинагулова даметай+одежда для дома+создать магазин одежды для домашинагулова даметай+одежда для дома+создать магазин одежды для дома
шинагулова даметай+одежда для дома+создать магазин одежды для дома
 
20151107 - The Influences on Motivation in Online Educational Environments
20151107 - The Influences on Motivation in Online Educational Environments20151107 - The Influences on Motivation in Online Educational Environments
20151107 - The Influences on Motivation in Online Educational Environments
 
Europa e Europeus - Celtas
Europa e Europeus - CeltasEuropa e Europeus - Celtas
Europa e Europeus - Celtas
 
Stock and Share Market Technical Analysis
Stock and Share Market Technical AnalysisStock and Share Market Technical Analysis
Stock and Share Market Technical Analysis
 
The back of the napkin Presentation
The back of the napkin PresentationThe back of the napkin Presentation
The back of the napkin Presentation
 

Similar to Deep learning

Deep Learning Module 2A Training MLP.pptx
Deep Learning Module 2A Training MLP.pptxDeep Learning Module 2A Training MLP.pptx
Deep Learning Module 2A Training MLP.pptx
vipul6601
 
Deep Learning Tutorial
Deep Learning Tutorial Deep Learning Tutorial
Deep Learning Tutorial
Ligeng Zhu
 
Machine Learning With Neural Networks
Machine Learning  With Neural NetworksMachine Learning  With Neural Networks
Machine Learning With Neural Networks
Knoldus Inc.
 
Neural networks
Neural networksNeural networks
Neural networks
Prakhar Mishra
 
Fully Homomorphic Encryption (1).pptx
Fully Homomorphic Encryption (1).pptxFully Homomorphic Encryption (1).pptx
Fully Homomorphic Encryption (1).pptx
ssuser1716c81
 
Introduction to Machine Learning with Spark
Introduction to Machine Learning with SparkIntroduction to Machine Learning with Spark
Introduction to Machine Learning with Spark
datamantra
 
Practical ML
Practical MLPractical ML
Practical ML
Antonio Pitasi
 
物件偵測與辨識技術
物件偵測與辨識技術物件偵測與辨識技術
物件偵測與辨識技術
CHENHuiMei
 
Neural network basic and introduction of Deep learning
Neural network basic and introduction of Deep learningNeural network basic and introduction of Deep learning
Neural network basic and introduction of Deep learning
Tapas Majumdar
 
Intro to TensorFlow and PyTorch Workshop at Tubular Labs
Intro to TensorFlow and PyTorch Workshop at Tubular LabsIntro to TensorFlow and PyTorch Workshop at Tubular Labs
Intro to TensorFlow and PyTorch Workshop at Tubular Labs
Kendall
 
Introduction to Applied Machine Learning
Introduction to Applied Machine LearningIntroduction to Applied Machine Learning
Introduction to Applied Machine Learning
SheilaJimenezMorejon
 
Fundamental of deep learning
Fundamental of deep learningFundamental of deep learning
Fundamental of deep learning
Stanley Wang
 
Computer Vision.pptx
Computer Vision.pptxComputer Vision.pptx
Computer Vision.pptx
GDSCIIITDHARWAD
 
Introduction to Neural Netwoks
Introduction to Neural Netwoks Introduction to Neural Netwoks
Introduction to Neural Netwoks
Abdallah Bashir
 
Eye deep
Eye deepEye deep
Eye deep
sveitser
 
V2.0 open power ai virtual university deep learning and ai introduction
V2.0 open power ai virtual university   deep learning and ai introductionV2.0 open power ai virtual university   deep learning and ai introduction
V2.0 open power ai virtual university deep learning and ai introduction
Ganesan Narayanasamy
 
Netflix machine learning
Netflix machine learningNetflix machine learning
Netflix machine learning
Amer Ather
 
19 - Neural Networks I.pptx
19 - Neural Networks I.pptx19 - Neural Networks I.pptx
19 - Neural Networks I.pptx
EmanAl15
 
Unit 1
Unit 1Unit 1
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
 

Similar to Deep learning (20)

Deep Learning Module 2A Training MLP.pptx
Deep Learning Module 2A Training MLP.pptxDeep Learning Module 2A Training MLP.pptx
Deep Learning Module 2A Training MLP.pptx
 
Deep Learning Tutorial
Deep Learning Tutorial Deep Learning Tutorial
Deep Learning Tutorial
 
Machine Learning With Neural Networks
Machine Learning  With Neural NetworksMachine Learning  With Neural Networks
Machine Learning With Neural Networks
 
Neural networks
Neural networksNeural networks
Neural networks
 
Fully Homomorphic Encryption (1).pptx
Fully Homomorphic Encryption (1).pptxFully Homomorphic Encryption (1).pptx
Fully Homomorphic Encryption (1).pptx
 
Introduction to Machine Learning with Spark
Introduction to Machine Learning with SparkIntroduction to Machine Learning with Spark
Introduction to Machine Learning with Spark
 
Practical ML
Practical MLPractical ML
Practical ML
 
物件偵測與辨識技術
物件偵測與辨識技術物件偵測與辨識技術
物件偵測與辨識技術
 
Neural network basic and introduction of Deep learning
Neural network basic and introduction of Deep learningNeural network basic and introduction of Deep learning
Neural network basic and introduction of Deep learning
 
Intro to TensorFlow and PyTorch Workshop at Tubular Labs
Intro to TensorFlow and PyTorch Workshop at Tubular LabsIntro to TensorFlow and PyTorch Workshop at Tubular Labs
Intro to TensorFlow and PyTorch Workshop at Tubular Labs
 
Introduction to Applied Machine Learning
Introduction to Applied Machine LearningIntroduction to Applied Machine Learning
Introduction to Applied Machine Learning
 
Fundamental of deep learning
Fundamental of deep learningFundamental of deep learning
Fundamental of deep learning
 
Computer Vision.pptx
Computer Vision.pptxComputer Vision.pptx
Computer Vision.pptx
 
Introduction to Neural Netwoks
Introduction to Neural Netwoks Introduction to Neural Netwoks
Introduction to Neural Netwoks
 
Eye deep
Eye deepEye deep
Eye deep
 
V2.0 open power ai virtual university deep learning and ai introduction
V2.0 open power ai virtual university   deep learning and ai introductionV2.0 open power ai virtual university   deep learning and ai introduction
V2.0 open power ai virtual university deep learning and ai introduction
 
Netflix machine learning
Netflix machine learningNetflix machine learning
Netflix machine learning
 
19 - Neural Networks I.pptx
19 - Neural Networks I.pptx19 - Neural Networks I.pptx
19 - Neural Networks I.pptx
 
Unit 1
Unit 1Unit 1
Unit 1
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
 

Recently uploaded

How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
BookNet Canada
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
RTTS
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
Sri Ambati
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
Alison B. Lowndes
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 

Recently uploaded (20)

How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdfFIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 

Deep learning

  • 2. Neural Network ● Neural network is what in the deep learning ● Designed to simulate human brain ● Consist of several layers and data will be passed one by one Input Layer Hidden Layer Hidden Layer Output Layer Data Input Data Output
  • 3. Perceptron ● In one perceptron, output value is computed by ● Where xi is the input, wi is weight and f is activation function y x1 x2 x3 f z
  • 4. One Layer ● For input vector x, output of one layer is computed by y = Wx+b z = f(y) Where W is weight of the layer, b is bias, f is activation function. ● W and b are parameters because we will modify them to optimize the model
  • 5. Activation Function ● Without activation function, multiple layers would be meaningless ● Good activation function is non-linear, differentiable, monotonically increasing. ● Logistic function ● Hyperbolic Tangent ● Retified linear function (ReLU)
  • 6. For Various Problems ● The activation function of the output layer depends on the type of problem ● For regression: ○ Activation function: Identity function ○ Length of output vector: Any ● For binary classification: ○ Activation function: Logistic function ○ Length of output vector: 1 ● For multi-class classification:
  • 7. Example Task ● Task setting: ○ Given picture of hand-written numbers 0-9, we want to tell which number it is ○ Training dataset consist of a lot of pictures and all picture is labeled with correct answer. ● Analyze task: ○ Problem type: Multi-class classification ○ Activation function of output layer: Softmax function
  • 8. Learn: Minimize Error ● Consider error function E(xi, di; W, b) which represents how off the model is from the true value for the ith picture. Here xi is the vector representation of picture. ● In our example we use Here, dik is 1 only if xi is actually picture of number k and otherwise 0. ● We want to modify W and b to minimize error function.
  • 9. Learn: Gradient Descent We use Gradient Descent to modify parameters W and b. ● Consider yourself in mountain and willing to reach the top of mountain, but you lost map and you can’t look distance because of smog. How do you reach the top? ➔Move to direction that bring you to the highest. ● The vector that indicate the direction to move is called gradient ● Since we want to minimize (instead of maximize), we update parameters by subtracting gradient.
  • 10. Learn: Backpropagation ● Neural network with multiple layers are too complex to simply compute gradients. ● This problem was one bottleneck in early stage of development of deep learning. ● Backpropagation compute gradient from output layer to input layer (A lot of chain rules). Input Layer Hidden Layer Hidden Layer Output Layer Data Input Data Output
  • 11. Why Deep Learning? ● Deep learning has tons of parameters (things that we can change to optimize the model) ➔Better accuracy ➔Hard to optimize ➔Need a lot of data ● Flexible model ➔Can be used to different types of problems ➔Easy to modify models for various situations
  • 12. Experiment ● Run python script for the example task ● Input vector is given as vector of length 784 ● Configurations are ○ 2 hidden layers ○ 1000 perceptrons for each layers ○ ReLU function for activation function ○ Softmax function for activation function for the output layer ○ Batch size: 100
  • 13. Result ● For this experiment, I used example code of chainer ● Execution time: about 45min ● Final Validation Loss: 0.107 ● Final Validation Accuracy: 0.98
  • 14. Use case of Deep Learning ● Convolutional Neural Network: Deep learning specific to picture data ○ Object identification ○ Face recognition ● Recurrent Neural Network: Deep learning for sequential data ○ Speech recognition ○ For text ● DQN: Combination of deep learning and Q learning ○ Alpha Go uses DQN and won top level Go player

Editor's Notes

  1. Deep learningの全体的な話をする
  2. 目の例を使って、人間の脳をもとに考えられたことを説明 一つ一つの層が何をしているかをざっくりと説明して次のスライドにつなげる
  3. 一つの神経細胞内で何が起こるかの説明。activation functionに関しては、次に説明するからってことにする
  4. 一つの層の線形性について
  5. Activation Functionの条件についてちゃんと話す
  6. ここまででdeep learningの全体像については一通り終わり。次に行く前にまとめをする