Introduction to Neural network and deep learning

1. Introduction to Neural
Network and Deep
Learning
ISSAM A. AL-ZINATI
OUTREACH & TECHNICAL ADVISOR
UCAS TECHNOLOGY INCUBATOR
ISSAM A. AL-ZINATI - UCASTI 1

2. ISSAM A. AL-ZINATI - UCASTI 2

3. Artificial Intelligence vs Machine Learning
Artificial Intelligence is the replication of human intelligence in
computers.
Machine Learning refers to the ability of a machine to learn using
large data sets instead of hard coded rules.
ISSAM A. AL-ZINATI - UCASTI 3

4. Supervised learning vs unsupervised
learning
Supervised Learning involves using labelled data sets that have
inputs and expected outputs.
Unsupervised Learning is the task of machine learning using data
sets with no specified structure.
ISSAM A. AL-ZINATI - UCASTI 4

5. What is Deep Learning
ISSAM A. AL-ZINATI - UCASTI 5
Is a Neural Network

6. What is Deep Learning
ISSAM A. AL-ZINATI - UCASTI 6
Is a Neural Network Neuron
Can run small specific
mathematical task

7. What is Deep Learning
ISSAM A. AL-ZINATI - UCASTI 7
Is a Neural Network Neuron
Can run small specific
mathematical task
Edge
Connects Neurons
Holds weights to adjust inputs

8. What is Deep Learning
ISSAM A. AL-ZINATI - UCASTI 8
Is a Neural Network
With More Layers

9. What is Deep Learning
ISSAM A. AL-ZINATI - UCASTI 9
Is a Neural Network
With More Layers
And More Neurons

10. What is Deep Learning
Deep Learning is a machine learning method. It allows us to train an
AI to predict outputs, given a set of inputs. Both supervised and
unsupervised learning can be used to train the AI.
ISSAM A. AL-ZINATI - UCASTI 10

11. How it wok – The Magic
ISSAM A. AL-ZINATI - UCASTI 11

12. How it work – No Magic
Deep Neural network is not magic. But it is very good at finding patterns.
“The hierarchy of concepts allows the computer to learn complicated concepts
by building them out of simpler ones. If we draw a graph showing how these
concepts are built on top of each other, the graph is deep, with many layers. For
this reason, we call this approach to AI deep learning”, Ian Goodfellow.
Deep Learning is Hierarchical Feature Learning.
ISSAM A. AL-ZINATI - UCASTI 12

13. How human brain works exactly?
ISSAM A. AL-ZINATI - UCASTI 13

14. How human brain works exactly?
ISSAM A. AL-ZINATI - UCASTI 14

15. How perceptron as an artificial neuron
works - Forward neural Network?
ISSAM A. AL-ZINATI - UCASTI 15

16. How perceptron as an artificial neuron
works - Forward neural Network?
ISSAM A. AL-ZINATI - UCASTI 16

17. What is weight in Neural Network?
Weight refers to the strength of connection between nodes. Unsigned value
(without +, -) of weight depends on how nodes have power to connect to each
other.
It can be positive or negative. Positive means it is more likely to transmit data
and having strong connection among neurons while negative is vice versa. At the
initialize point we select weight randomly but for having reasonable result it is
better to normalize input data as follow, X is input data:
ISSAM A. AL-ZINATI - UCASTI 17

18. What is Activation Function role in
Neural Network?
Activation function is (although a bit) equivalent to polarization and stabilizing.
ISSAM A. AL-ZINATI - UCASTI 18

19. How backward propagation works?
In backward propagation because we need optimum value so we differentiate from sigmoid
function and go inversely from right to left, in order to finding new values for weights.
(1) output_node′ = Sigmoid′ (hidden_sigma) * margin
(2) weight_2 ′ = (output_node′ / hidden_node) + weight_2
(3) hidden_node ′ = (output_node′ / weight_2) * Sigmoid′ (input_sigma)
(4) weight_1 ′ = (hidden_node′ / input_node) + weight_1
(5) Again we repeat steps 1 to 5 with new weights and comparison value from current margin
errors and previous margin errors if current error is less than previous one, so it shows us that
we are in right direction.
(6) We iterate step 1 to 10 until margin error is near to our “Y”.
ISSAM A. AL-ZINATI - UCASTI 19

20. How backward propagation works?
ISSAM A. AL-ZINATI - UCASTI 20

21. Why Now
ISSAM A. AL-ZINATI - UCASTI 21

22. Why Now- Scale
ISSAM A. AL-ZINATI - UCASTI 22
Data

23. Why Now- Scale
ISSAM A. AL-ZINATI - UCASTI 23
Data
Small Meduim Large
Performance Based on Data Size
Performance
The more data
you feed the
model, the
better results
you get

24. Why Now- Scale
ISSAM A. AL-ZINATI - UCASTI 24
Model Size & GPU

25. Why Now- Scale
ISSAM A. AL-ZINATI - UCASTI 25
Model Size & GPU
Small Meduim Large
Performance Based on Model Size
Performance
Bigger model could
achieve better
results.
GPUs help to train
those models in
much faster, 20X!!

26. Why Now– vs Others
ISSAM A. AL-ZINATI - UCASTI 26
What about other kind of machine learning algorithms, i.e. SVM, DT, Boosting, ….
Would they do better if they got more data and power?

27. Why Now– vs Others
Small Data Medium Data Large Data
Performance of NN VS Others
Based on Model Size and Data Amount
Others Small NN Medium NN Large NN
ISSAM A. AL-ZINATI - UCASTI 27

28. Why Now– End-To-End
ISSAM A. AL-ZINATI - UCASTI 28
Usual machine learning approach contains a pipeline of stages that are
responsible of feature extraction.
Each stage passes a set of engineered features which help model to better
understand the case it works on.
This approach is complex and prone to errors.

29. Why Now– End-To-End
ISSAM A. AL-ZINATI - UCASTI 29
Data (Audio)
Speech Recognition Pipeline
Audio
Features
Phonemes
Language
Model
Transcript

30. Why Now– End-To-End
ISSAM A. AL-ZINATI - UCASTI 30
Data (Audio)
Speech Recognition - DL
Audio
Features Phonemes Language
Model
Transcript

31. Why Now– End-To-End
ISSAM A. AL-ZINATI - UCASTI 31
Data (Audio)
Speech Recognition - DL
Audio
Features Phonemes Language
Model
Transcript
The Magic

32. Deep Learning Models
ISSAM A. AL-ZINATI - UCASTI 32
General
Model
FC
Sequence
Model
RNN
LSTM
Image
Model
CNN
Other
Models
Unsupervised
RL

33. Deep Learning Models
ISSAM A. AL-ZINATI - UCASTI 33
General
Model
FC
Sequence
Model
RNN
LSTM
Image
Model
CNN
Other
Models
Unsupervised
RL
Hot Research Topic

34. Advanced Deep Learning Models –
VGGNET - ResNet
Achieves 7.3% on ImageNet-2014 classification Challenge, come in the first
place.
It Used
120 million
parameters.
ISSAM A. AL-ZINATI - UCASTI 34

35. Advanced Deep Learning Models –
Google Inception V3
Achieves 5.64% on ImageNet-2015 classification Challenge, come in the second place.
ISSAM A. AL-ZINATI - UCASTI 35

36. Advanced Deep Learning Models –
Google Inception V3
Based on ConvNet concept with the addition
of inception module.
ISSAM A. AL-ZINATI - UCASTI 36
Using a network with a
computational cost of 5 billion
multiply-adds per inference and
with using less than 25 million
parameters.

37. Deep Learning Applications – Deep Voice
Baidu Research presents Deep Voice, a production-quality text-to-speech system
constructed entirely from deep neural networks.
Ground Truth
Generated Voice
ISSAM A. AL-ZINATI - UCASTI 37

38. Deep Learning Applications – Image
Captioning
Multimodal Recurrent Neural Architecture generates sentence descriptions from
images. Source.
ISSAM A. AL-ZINATI - UCASTI 38
"man in black shirt is playing guitar." "two young girls are playing with lego toy."

39. Deep Learning Applications – Generating
Videos
ISSAM A. AL-ZINATI - UCASTI 39
This approach was driven by using Adversarial Network to
1) Generate Videos
2) Conditional Video Generation based on Static Images
Source

40. Applying Deep Learning – Frameworks
Low Level
ISSAM A. AL-ZINATI - UCASTI 40

41. Applying Deep Learning – Frameworks
Low Level
ISSAM A. AL-ZINATI - UCASTI 41
High Level

42. ISSAM A. AL-ZINATI - UCASTI 42
Thanks for listening 