Deep learning is introduced along with its applications and key players in the field. The document discusses the problem space of inputs and outputs for deep learning systems. It describes what deep learning is, providing definitions and explaining the rise of neural networks. Key deep learning architectures like convolutional neural networks are overviewed along with a brief history and motivations for deep learning.

1. Introduction to Deep
Learning
Introduced by : Dr. Amr Rashed
Lecturer, Department of Computer Engineering,
College of Computers and Information Technology, Taif University
Ph.D., Electronics & Communication Engineering Faculty of Engineering,
Mansoura University

5. Deep Learning : Applications

6. The Rise of Deep Learning

13. Big Players :Superstar Researcher
Jeoferry Hilton : University of Toronto
&Google
Yann Lecun : New Yourk University
&Facebook
Andrew Ng : Stanford & Baidu
Yoshua Bengio : University of Montreal
Jurgen schmidhuber : Swiss AI Lab &
NNAISENSE

14. Big Players : Companies

25. The Problem Space : inputs & outputs
0 Depending on the resolution and size of the image, it will see a 32 x
32 x 3 array of numbers
0 These numbers, while meaningless to us when we perform image
classification, are the only inputs available to the computer.
0 The idea is that you give the computer this array of numbers and it
will output numbers that describe the probability of the image
being a certain class (.80 for cat, .15 for dog, .05 for bird, etc).

26. The Problem Space :What We Want the Computer to Do
0 To be able to differentiate between all the images it’s given
and figure out the unique features that make a dog a dog or
that make a cat a cat.
0 This is the process that goes on in our minds
subconsciously as well.
0 When we look at a picture of a dog, we can classify it as
such if the picture has identifiable features such as paws or
4 legs.
0 In a similar way, the computer is able perform image
classification by looking for low level features such as
edges and curves, and then building up to more abstract
concepts through a series of convolutional layers.

28. What is Deep Learning(DL)
Part or a powerful class of the machine learning field of
learning representations of data. Exceptional effective at
learning patterns.
Utilize learning algorithms that derive meaning out of data
by using hierarchy of multiple layers that mimic the neural
networks of our brain.
If you provide the system tons of information , it begins to
understand it and respond in useful ways.
Stacked “Neural Network”. Is usually indicates “Deep Neural Network”.

29. What is Deep Learning (DL)
• Collection of simple, trainable mathematical functions.
• Learning methods which have deep architecture.
• It often allows end-to-end learning.
• It automatically finds intermediate representation. Thus, it can
be regarded as a representation learning.

30. A Brief History
0 2012 was the first year that neural nets grew to prominence
as Alex Krizhevsky used them to win that year’s ImageNet
competition (basically, the annual Olympics of computer
vision), dropping the classification error record from 26% to
15%, an astounding improvement at the time.

31. Motivations
NIPS(Computational Neuroscience
Conference) Growth
Amount of Data vs Performance

32. Why Deep Learning and Why Now?

35. The Perceptron
The structural building block of deep learning

36. Commonalities with real brains:
● Each neuron is connected to a small subset of other neurons.
● Based on what it sees, it decides what it wants to say.
● Neurons learn to cooperate to accomplish the task.
• The vental pathway in the visual cortex has multiple stages
• There exist a lot of intermediate representations
Deep Learning Basics :Neuron

37. Deep Learning Basics :Neuron
An artificial neuron contains a nonlinear activation function and has several
incoming and outgoing weighted connections.
Neurons are trained filters and detect specific features or patterns (e.g.
edge, nose) by receiving weighted input, transforming it with the activation
function and passing it to the outgoing connections

47. Supervised Learning : Learning with a labeled
training set
Example : e-mail spam detector with training set of
already labeled emails
Unsupervised Learning :Discovering patterns in
unlabeled data
Example :cluster similar documents based on the text
content
Reinforcement learning : learning based on feedback
or reward.
Example :learn to play chess by wining or losing
Machine Learning –Basics
Learning Approach

52. Problem Types

53. Deep Neural Network : Architecture types
Feed-Forward
• Convolutional neural networks
• De-convolutional networks
Bi-Directional
• Deep Boltzmann Machines
• Stacked auto-encoders
Sequence -Based
• RNNs
• LSTMs

54. Types of Networks used for Deep Learning
Convolutional Neural Networks(Convnet,CNN)
Recurrent Neural Networks(RNN)
Long Short Term Memory (LSTM) networks
Deep/Restricted Boltzmann Machines (RBM)
Deep Q-networks
Deep Belief Networks(DBN)
Deep Stacking Networks

55. Convolution Neural Networks(CNN):Basic Idea
0 This idea was expanded upon by a fascinating experiment by
Hubel and Wiesel in 1962 where they showed that some
individual neuronal cells in the brain responded (or fired)
only in the presence of edges of a certain orientation.
0 For example, some neurons fired when exposed to vertical
edges and some when shown horizontal or diagonal edges.
Hubel and Wiesel found out that all of these neurons were
organized in a columnar architecture and that together, they
were able to produce visual perception.
0 This idea of specialized components inside of a system
having specific tasks (the neuronal cells in the visual cortex
looking for specific characteristics) is one that machines use
as well, and is the basis behind CNNs.

57. CNN Layers
1 • Convolution Layer
2 • Batch Normalization Layer
3 • RELU Layer
4 • Local Response Normalization Layer
5 • Max and AVG Pooling
6 • Dropout Layer
7 • Fully Connected Layer (FC)
8 • Output Layer (SOFT MAX ,Regression )

58. Convolution Neural Networks(CNN)
• Convolutional neural networks learn a complex representation of visual data
using vast amounts of data .they are inspired by human visual system and learn
multiple layers of transformations , which are applied on top of each other to
extract progressively more sophisticated representation of the input .
DEFENITION
• Inspired by the visual cortex and Pioneered by Yann Lecun (NYU).
• CNN have multiple types of layers ,the first of which is the Convolutional
layer.
Notes

59. Test pretrained model in Matlab
and Python

63. Dataset storage, and search platforms
• https://www.kaggle.com/datasets
• https://datasetsearch.research.google.com/
• https://github.com/
• https://ieee-dataport.org/
• medical-imaging-datasets
• https://data.worldbank.org/
• https://data.world/
• https://data.un.org/
• https://archive.ics.uci.edu/ml/datasets/

64. Best Python tutorials:
• Books:
• Python Crash Course A Hands-On, Project-Based Introduction to
Programming by E R I C M A T T H E S
• Videos :
• Python Beginners Tutorial – ‫بالعربي‬
• ‫الحوراني‬ ‫حسام‬ ‫بايثون‬ ‫قناة‬
• Mastering Python
• 1- Python programming ‫اساسيات‬ ‫بايثون‬ ‫برمجة‬
• ‫إم‬ ‫جامعة‬ ‫من‬ ‫والبرمجة‬ ‫الكومبيوتر‬ ‫علم‬ ‫في‬ ‫مقدمة‬
‫آي‬
‫تي‬

65. Best AI, ML, and DL tutorials:
• Books and websites:
• Deep Learning book by Ian Goodfellow (MIT press)
• “Neural Networks and Deep Learning” by Michael Nielsen
• TensorFlow
• Machine Learning Mastery by Jason Brownlee
• Towards data science
• Data science community
• http://introtodeeplearning.com(MIT)
• https://www.pyimagesearch.com/
• Videos:
• Machine Learning Andrew Ng Courses (coursera)
• ‫للجميع‬ ‫االصطناعي‬ ‫(الذكاء‬Andrew ng) with Arabic subtitle
• Courses and Specializations (all AI, ML, and DL courses)
• Machine Learning Course - CS 156 (Prof. Yasser abo Moustafa, Caltech university)
• Introduction to data science
• ‫االصطناعي‬ ‫الذكاء‬ ‫قناة‬
-
‫الحوراني‬ ‫حسام‬
• 01 machine learning ‫اآللة‬ ‫تعليم‬
,
‫األول‬ ‫القسم‬
:
‫مقدمة‬

66. For more information
• TensorFlow and Deep Learning without a PhD, Part 1 (Google Cloud Next '17)
• TensorFlow and Deep Learning without a PhD, Part 2 (Google Cloud Next ‘17)
• Welcome (Deep Learning Specialization C1W1L01)
• A friendly introduction to Deep Learning and Neural Networks
• Introduction to Deep Learning: Machine Learning vs. Deep Learning
• Introduction to Deep Learning: What Are Convolutional Neural Networks?
• How Convolutional Neural Networks work
• How Deep Neural Networks Work
• Deep Learning In 5 Minutes | What Is Deep Learning? | Deep Learning Explained Simply |
Simplilearn
• Deep Learning Crash Course for Beginners

67. Cont.
• What is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutorial |
Simplilearn
• MIT Introduction to Deep Learning | 6.S191(2021)
• MIT 6.S191 Lecture 1: Intro to Deep Learning (2017)
• Machine Learning Foundations: Ep #1 - What is ML?
• The 7 steps of machine learning
• But what is a neural network? | Chapter 1, Deep learning
• Deep Learning State of the Art (2020)
• What is Artificial Intelligence? In 5 minutes.
• A friendly introduction to Convolutional Neural Networks and Image Recognition

68. Contact Information
• Dr. Amr E. Rashed