Neural network and deep learning Devfest17

Introduction to Neural
Network and Deep
Learning
ISSAM A. AL-ZINATI
OUTREACH & TECHNICAL ADVISOR
UCAS TECHNOLOGY INCUBATOR
ISSAM A. AL-ZINATI - UCASTI 1

Artificial Intelligence vs Machine Learning
Artificial Intelligence is the replication of human intelligence in
computers.
Machine Learning refers to the ability of a machine to learn using
large data sets instead of hard coded rules.

Supervised learning vs unsupervised
learning
Supervised Learning involves using labelled data sets that have
inputs and expected outputs.
Unsupervised Learning is the task of machine learning using data
sets with no specified structure.

What is Deep Learning
Is a Neural Network

Is a Neural Network Neuron
Can run small specific
mathematical task

Is a Neural Network Neuron
Can run small specific
mathematical task
Edge
Connects Neurons
Holds weights to adjust inputs

Is a Neural Network
With More Layers

Is a Neural Network
With More Layers
And More Neurons

Deep Learning is a machine learning method. It allows us to train an
AI to predict outputs, given a set of inputs. Both supervised and
unsupervised learning can be used to train the AI.

How it wok – The Magic

How it work – No Magic
Deep Neural network is not magic. But it is very good at finding patterns.
“The hierarchy of concepts allows the computer to learn complicated concepts
by building them out of simpler ones. If we draw a graph showing how these
concepts are built on top of each other, the graph is deep, with many layers. For
this reason, we call this approach to AI deep learning”, Ian Goodfellow.
Deep Learning is Hierarchical Feature Learning.

How human brain works exactly?

How perceptron as an artificial neuron
works - Forward neural Network?

What is weight in Neural Network?
Weight refers to the strength of connection between nodes. Unsigned value
(without +, -) of weight depends on how nodes have power to connect to each
other.
It can be positive or negative. Positive means it is more likely to transmit data
and having strong connection among neurons while negative is vice versa. At the
initialize point we select weight randomly but for having reasonable result it is
better to normalize input data as follow, X is input data:

What is Activation Function role in
Neural Network?
Activation function is (although a bit) equivalent to polarization and stabilizing.

How backward propagation works?
In backward propagation because we need optimum value so we differentiate from sigmoid
function and go inversely from right to left, in order to finding new values for weights.
(1) output_node′ = Sigmoid′ (hidden_sigma) * margin
(2) weight_2 ′ = (output_node′ / hidden_node) + weight_2
(3) hidden_node ′ = (output_node′ / weight_2) * Sigmoid′ (input_sigma)
(4) weight_1 ′ = (hidden_node′ / input_node) + weight_1
(5) Again we repeat steps 1 to 5 with new weights and comparison value from current margin
errors and previous margin errors if current error is less than previous one, so it shows us that
we are in right direction.
(6) We iterate step 1 to 10 until margin error is near to our “Y”.

How backward propagation works?

Why Now

Why Now- Scale
Data

Why Now- Scale
Data
Small Meduim Large
Performance Based on Data Size
Performance
The more data
you feed the
model, the
better results
you get

Why Now- Scale
Model Size & GPU

Why Now- Scale
Model Size & GPU
Small Meduim Large
Performance Based on Model Size
Performance
Bigger model could
achieve better
results.
GPUs help to train
those models in
much faster, 20X!!

Why Now– vs Others
What about other kind of machine learning algorithms, i.e. SVM, DT, Boosting, ….
Would they do better if they got more data and power?

Why Now– vs Others
Small Data Medium Data Large Data
Performance of NN VS Others
Based on Model Size and Data Amount
Others Small NN Medium NN Large NN

Why Now– End-To-End
Usual machine learning approach contains a pipeline of stages that are
responsible of feature extraction.
Each stage passes a set of engineered features which help model to better
understand the case it works on.
This approach is complex and prone to errors.

Data (Audio)
Speech Recognition Pipeline
Audio
Features
Phonemes
Language
Model
Transcript

Data (Audio)
Speech Recognition - DL
Audio
Features Phonemes Language
Model
Transcript

Data (Audio)
Speech Recognition - DL
Audio
Features Phonemes Language
Model
Transcript
The Magic

Deep Learning Models
General
Model
FC
Sequence
Model
RNN
LSTM
Image
Model
CNN
Other
Models
Unsupervised
RL

Deep Learning Models
General
Model
FC
Sequence
Model
RNN
LSTM
Image
Model
CNN
Other
Models
Unsupervised
RL
Hot Research Topic

Advanced Deep Learning Models –
VGGNET - ResNet
Achieves 7.3% on ImageNet-2014 classification Challenge, come in the first
place.
It Used
120 million
parameters.

Google Inception V3
Achieves 5.64% on ImageNet-2015 classification Challenge, come in the second place.

Google Inception V3
Based on ConvNet concept with the addition
of inception module.
Using a network with a
computational cost of 5 billion
multiply-adds per inference and
with using less than 25 million
parameters.

Deep Learning Applications – Deep Voice
Baidu Research presents Deep Voice, a production-quality text-to-speech system
constructed entirely from deep neural networks.
Ground Truth
Generated Voice

Deep Learning Applications – Image
Captioning
Multimodal Recurrent Neural Architecture generates sentence descriptions from
images. Source.
"man in black shirt is playing guitar." "two young girls are playing with lego toy."

Deep Learning Applications – Generating
Videos
This approach was driven by using Adversarial Network to
1) Generate Videos
2) Conditional Video Generation based on Static Images
Source

Applying Deep Learning – Frameworks
Low Level

Applying Deep Learning – Frameworks
Low Level
High Level

Thanks for listening 

Neural network and deep learning Devfest17

More Related Content

What's hot

Similar to Neural network and deep learning Devfest17

Recently uploaded

Neural network and deep learning Devfest17