SlideShare a Scribd company logo
BUMIC + DSC React Series
Darcy
03/09/2021
Introduction to Applied
Machine Learning
Applications of Deep Learning
1. Cool things using deep learning
a. Computer Vision
i. Tesla recognizing items on a street
b. Text generation
i. OpenAI GPT3 can solve almost any
language task in a few examples
c. Reinforcement Learning
i. Can play Atari games, Board games,
Real Time Strategy games
ii. Robotic control
d. Many more...
2
Learning from data
3
We have some data D
4
D
X
Y
Make an assumption about D
5
D
X
Y
What is learning?
6
The approximation of some unknown function f based on some data D.
D
X
Y
How do we set the parameters?
How do we know what assumptions to
make?
Intro to Deep Learning
What is Deep Learning
8
Artificial
Intelligence
Machine
Learning
Deep
Learning
Deep learning is a subset of machine learning
What is Deep Learning
9
Deep learning learns from data using a class of functions known
as Neural Networks
A neural network maps an input to an output
Biological Neuron vs. Artificial Neuron
Andrej Karpathy
What is a Neural Network?
Steps to Train a NN
Push example through the network to get a predicted output
Forward propagation
Input
Number of
Bedrooms
Number of
Bathrooms
Square
Feet
Hidden 1 Hidden 2 Hidden 3
Output
Price of House
Calculate difference between predicted output and actual data
Compute the cost
Output
Price of House
D
X
Y
Calculate difference between predicted output and actual data
Compute the cost
Output
Price of House
Where i is the ith training example and m is the number of training examples
Push back the derivative of the error and apply to each weight, such that next
time it will result in a lower error
Backward propagation - “Update”
https://hmkcode.github.io/ai/backpropagation-step-by-step/
Convolutional Neural Networks
Image Data
18
● Images are commonly represented in code as a 3D
array of pixels. Here, we notice 3 represents RGB
values
● In vanilla neural networks, we would simply flatten
this 3D array into a 3072 length vector. However,
by doing this, we lose spatial correlation between
pixels close to other pixels
Image Data
19
● In 2012 a paper called AlexNet out competed
state of the art image classification models
through the usage of kernels (also called filters)
Kernel
20
● Kernel: a small matrix used for
feature detection on an image
○ Also called a filter
● Usage
○ Superimpose the kernel over a section
of an image
○ Do element-wise multiplication
between the weights in the kernel and
the values in the image
○ Record the sum of the multiplications
Example Convolution
21
Example: Multiply the 5x5 image by a
3x3 kernel with weights:
1 0 1
0 1 0
1 0 1
The output?
Sum of weight times
part of image to a
single number.
Kernel example
22
6 3 2
4 3 1
3 5 5
0 1 0
1 2 1
0 1 0
* = 19
Section of
an image
Kernel
6*0 3*1 2*0
4*1 3*2 1*1
3*0 5*1 5*0
=
sum
Kernel
3 3 1
4 6 5
3 5 2
Kernel example (cont.)
23
Section of
an image
This image section contains the same values as before, but they have
been rearranged, resulting in a greater activation with this kernel
0 1 0
1 2 1
0 1 0
* = 23
Section of
an image
3*0 3*1 1*0
4*1 6*2 5*1
3*0 5*1 2*0
=
sum
Example Convolution
24
● Note that the output is
smaller than the input
● This can be prevented
by using padding
around the edges of the
image.
Padding
25
● Before padding:
○ 7x7 input, 3x3 filter creating a 5x5 sized
output
● After padding:
○ 9x9 input, 3x3 filter creating a 7x7 sized
output which maintains the same size as
our input
● Edges and corners aren’t as accurate
but in practice this works well
enough
7
9
Stride
26
● Here, the kernel is moving
one pixel at a time
(“stride” = 1)
● The kernel can move by
more than one pixel at a
time
● Size = (N - F) / Stride + 1
Stride
27
● Increasing stride
decreases the size of the
output
● Here, stride = 2
● (N - F) / Stride + 1
(7 - 3) / 2 + 1 = 3
Dimensionality Practice
28
● What would be the output size of a
5x5x3 filter with a 32x32x3 image
and a stride of 1?
● (N - F) / Stride + 1
Dimensionality Practice
29
● (32 - 5) / 1 + 1 = 28
● Now let’s say we had a stride of 2,
○ (32 - 5) / 2 + 1 = 14.5
○ Fractional size means the filter hangs off
the input
○ We wouldn’t use this stride value
consequently
Intentionally Shrinking Output Size
30
● Now, let’s say you want to shrink your outputs (which are inputs to the
next layer) to reduce operations.
● You can do this by either increasing the stride
○ (N - F)/Stride + 1
● Alternatively, you can use a pooling layer
Conv Layer Output
31
● Use multiple kernels for multiple activation
maps
● In this example, we have 6 activation maps
each created through a different filter with
its own set of weights and biases
Additional Layers
Pooling Layers
33
● Limitation of output of Convolutional Layers:
○ Record the precise position of features in the input
○ Small movements in the position of the feature in the input image will result in a
different feature map
● Solution: Pooling Layers
○ Lower resolution version of input is created with large and important structure
elements preserved
○ Reduces the computational cost by reducing the number of parameters to learn
Max Pooling
34
Input (4 x 4) Output (2 x 2)
Extracts the sharpest features of an image, making it more general
Average Pooling
35
Input (4 x 4) Output (2 x 2)
Takes average feature of an image, minimize overfitting
Dropout
1. First, what is overfitting?
a. Overfitting is when the neural network corresponds too closely to the dataset, and cannot
be generalized. This tends to happen when a model is excessively complex relative to the
data
b. Conversely, underfitting is when the network cannot capture the underlying trend of the
dataset which may happen if your network is not complicated enough.
36
Dropout - How can we solve overfitting?
1. Training phase
a. Each weight has a probability p that they will be multiplied by zero (dropped). This
probability is often set to 0.5, which is considered to be close to optimal for a wide range of
networks and tasks
b. This has the effect of removing random connections between activations effectively creating
a new network/outlook on the data per each train set
37
Dropout - How can we solve overfitting?
2. Post Train
a. After training weights will be abnormally high as they were adjusted assuming only (1-p)
percent of the weights would be summed together and used.
b. To fix this we normalize weights to lower the expectation of each weight. We do this by
scaling each weight by 1/p
c. “This makes sure that for each unit, the expected output from it under random dropout will
be the same as the output during pretraining.” ~Dropout: A Simple Way to Prevent Neural
Networks from Overfitting
i. http://www.jmlr.org/papers/volume15/srivastava14a/srivastava14a.pdf
38
Convolutional Neural Network
39
So what does our network look like?
40
●

More Related Content

What's hot

Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Simplilearn
 
Benchmark comparison of Large Language Models
Benchmark comparison of Large Language ModelsBenchmark comparison of Large Language Models
Benchmark comparison of Large Language Models
Matej Varga
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
Oleg Mygryn
 
neural networks
neural networksneural networks
neural networks
Ruchi Sharma
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
Christian Perone
 
Intro to LLMs
Intro to LLMsIntro to LLMs
Intro to LLMs
Loic Merckel
 
1.Introduction.ppt
1.Introduction.ppt1.Introduction.ppt
1.Introduction.ppt
BharatSingh848748
 
Learning Deep Learning
Learning Deep LearningLearning Deep Learning
Learning Deep Learning
simaokasonse
 
Computer vision introduction
Computer vision  introduction Computer vision  introduction
Computer vision introduction
Wael Badawy
 
AI - Fuzzy Logic Systems
AI - Fuzzy Logic SystemsAI - Fuzzy Logic Systems
AI - Fuzzy Logic Systems
Learnbay Datascience
 
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
Simplilearn
 
Hopfield Networks
Hopfield NetworksHopfield Networks
Hopfield Networks
Kanchana Rani G
 
Soft computing
Soft computingSoft computing
Soft computing
ganeshpaul6
 
Neural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronNeural Networks: Multilayer Perceptron
Neural Networks: Multilayer Perceptron
Mostafa G. M. Mostafa
 
Fuzzy Logic Seminar with Implementation
Fuzzy Logic Seminar with ImplementationFuzzy Logic Seminar with Implementation
Fuzzy Logic Seminar with Implementation
Bhaumik Parmar
 
Fuzzy control and its applications
Fuzzy control and its applicationsFuzzy control and its applications
Fuzzy control and its applications
jeevithaelangovan
 
Regularization in deep learning
Regularization in deep learningRegularization in deep learning
Regularization in deep learning
Kien Le
 
Machine Learning and Real-World Applications
Machine Learning and Real-World ApplicationsMachine Learning and Real-World Applications
Machine Learning and Real-World Applications
MachinePulse
 
Optimizers
OptimizersOptimizers
Optimizers
Il Gu Yi
 
Sequence to Sequence Learning with Neural Networks
Sequence to Sequence Learning with Neural NetworksSequence to Sequence Learning with Neural Networks
Sequence to Sequence Learning with Neural Networks
Nguyen Quang
 

What's hot (20)

Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
 
Benchmark comparison of Large Language Models
Benchmark comparison of Large Language ModelsBenchmark comparison of Large Language Models
Benchmark comparison of Large Language Models
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
 
neural networks
neural networksneural networks
neural networks
 
Deep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural NetworksDeep Learning - Convolutional Neural Networks
Deep Learning - Convolutional Neural Networks
 
Intro to LLMs
Intro to LLMsIntro to LLMs
Intro to LLMs
 
1.Introduction.ppt
1.Introduction.ppt1.Introduction.ppt
1.Introduction.ppt
 
Learning Deep Learning
Learning Deep LearningLearning Deep Learning
Learning Deep Learning
 
Computer vision introduction
Computer vision  introduction Computer vision  introduction
Computer vision introduction
 
AI - Fuzzy Logic Systems
AI - Fuzzy Logic SystemsAI - Fuzzy Logic Systems
AI - Fuzzy Logic Systems
 
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...
 
Hopfield Networks
Hopfield NetworksHopfield Networks
Hopfield Networks
 
Soft computing
Soft computingSoft computing
Soft computing
 
Neural Networks: Multilayer Perceptron
Neural Networks: Multilayer PerceptronNeural Networks: Multilayer Perceptron
Neural Networks: Multilayer Perceptron
 
Fuzzy Logic Seminar with Implementation
Fuzzy Logic Seminar with ImplementationFuzzy Logic Seminar with Implementation
Fuzzy Logic Seminar with Implementation
 
Fuzzy control and its applications
Fuzzy control and its applicationsFuzzy control and its applications
Fuzzy control and its applications
 
Regularization in deep learning
Regularization in deep learningRegularization in deep learning
Regularization in deep learning
 
Machine Learning and Real-World Applications
Machine Learning and Real-World ApplicationsMachine Learning and Real-World Applications
Machine Learning and Real-World Applications
 
Optimizers
OptimizersOptimizers
Optimizers
 
Sequence to Sequence Learning with Neural Networks
Sequence to Sequence Learning with Neural NetworksSequence to Sequence Learning with Neural Networks
Sequence to Sequence Learning with Neural Networks
 

Similar to Introduction to Applied Machine Learning

Eye deep
Eye deepEye deep
Eye deep
sveitser
 
00463517b1e90c1e63000000
00463517b1e90c1e6300000000463517b1e90c1e63000000
00463517b1e90c1e63000000
Ivonne Liu
 
03 image transformations_i
03 image transformations_i03 image transformations_i
03 image transformations_i
ankit_ppt
 
BMVA summer school MATLAB programming tutorial
BMVA summer school MATLAB programming tutorialBMVA summer school MATLAB programming tutorial
BMVA summer school MATLAB programming tutorial
potaters
 
Keras on tensorflow in R & Python
Keras on tensorflow in R & PythonKeras on tensorflow in R & Python
Keras on tensorflow in R & Python
Longhow Lam
 
Generative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural NetworksGenerative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural Networks
Denis Dus
 
Scala and Deep Learning
Scala and Deep LearningScala and Deep Learning
Scala and Deep Learning
Oswald Campesato
 
Image De-Noising Using Deep Neural Network
Image De-Noising Using Deep Neural NetworkImage De-Noising Using Deep Neural Network
Image De-Noising Using Deep Neural Network
aciijournal
 
Image De-Noising Using Deep Neural Network
Image De-Noising Using Deep Neural NetworkImage De-Noising Using Deep Neural Network
Image De-Noising Using Deep Neural Network
aciijournal
 
IMAGE DE-NOISING USING DEEP NEURAL NETWORK
IMAGE DE-NOISING USING DEEP NEURAL NETWORKIMAGE DE-NOISING USING DEEP NEURAL NETWORK
IMAGE DE-NOISING USING DEEP NEURAL NETWORK
aciijournal
 
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Universitat Politècnica de Catalunya
 
Deep learning (2)
Deep learning (2)Deep learning (2)
Deep learning (2)
Muhanad Al-khalisy
 
Log polar coordinates
Log polar coordinatesLog polar coordinates
Log polar coordinates
Oğul Göçmen
 
28 01-2021-05
28 01-2021-0528 01-2021-05
28 01-2021-05
AdemarAlves7
 
C++ and Deep Learning
C++ and Deep LearningC++ and Deep Learning
C++ and Deep Learning
Oswald Campesato
 
chap4_ann (5).pptx
chap4_ann (5).pptxchap4_ann (5).pptx
chap4_ann (5).pptx
SajjadAbdullah4
 
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Accelerated Logistic Regression on GPU(s)
Accelerated Logistic Regression on GPU(s)Accelerated Logistic Regression on GPU(s)
Accelerated Logistic Regression on GPU(s)
RAHUL BHOJWANI
 
Image Classification using Deep Learning
Image Classification using Deep LearningImage Classification using Deep Learning
Image Classification using Deep Learning
IRJET Journal
 
Neural Networks - How do they work?
Neural Networks - How do they work?Neural Networks - How do they work?
Neural Networks - How do they work?
Accubits Technologies
 

Similar to Introduction to Applied Machine Learning (20)

Eye deep
Eye deepEye deep
Eye deep
 
00463517b1e90c1e63000000
00463517b1e90c1e6300000000463517b1e90c1e63000000
00463517b1e90c1e63000000
 
03 image transformations_i
03 image transformations_i03 image transformations_i
03 image transformations_i
 
BMVA summer school MATLAB programming tutorial
BMVA summer school MATLAB programming tutorialBMVA summer school MATLAB programming tutorial
BMVA summer school MATLAB programming tutorial
 
Keras on tensorflow in R & Python
Keras on tensorflow in R & PythonKeras on tensorflow in R & Python
Keras on tensorflow in R & Python
 
Generative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural NetworksGenerative modeling with Convolutional Neural Networks
Generative modeling with Convolutional Neural Networks
 
Scala and Deep Learning
Scala and Deep LearningScala and Deep Learning
Scala and Deep Learning
 
Image De-Noising Using Deep Neural Network
Image De-Noising Using Deep Neural NetworkImage De-Noising Using Deep Neural Network
Image De-Noising Using Deep Neural Network
 
Image De-Noising Using Deep Neural Network
Image De-Noising Using Deep Neural NetworkImage De-Noising Using Deep Neural Network
Image De-Noising Using Deep Neural Network
 
IMAGE DE-NOISING USING DEEP NEURAL NETWORK
IMAGE DE-NOISING USING DEEP NEURAL NETWORKIMAGE DE-NOISING USING DEEP NEURAL NETWORK
IMAGE DE-NOISING USING DEEP NEURAL NETWORK
 
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
Multilayer Perceptron (DLAI D1L2 2017 UPC Deep Learning for Artificial Intell...
 
Deep learning (2)
Deep learning (2)Deep learning (2)
Deep learning (2)
 
Log polar coordinates
Log polar coordinatesLog polar coordinates
Log polar coordinates
 
28 01-2021-05
28 01-2021-0528 01-2021-05
28 01-2021-05
 
C++ and Deep Learning
C++ and Deep LearningC++ and Deep Learning
C++ and Deep Learning
 
chap4_ann (5).pptx
chap4_ann (5).pptxchap4_ann (5).pptx
chap4_ann (5).pptx
 
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
Multilayer Perceptron - Elisa Sayrol - UPC Barcelona 2018
 
Accelerated Logistic Regression on GPU(s)
Accelerated Logistic Regression on GPU(s)Accelerated Logistic Regression on GPU(s)
Accelerated Logistic Regression on GPU(s)
 
Image Classification using Deep Learning
Image Classification using Deep LearningImage Classification using Deep Learning
Image Classification using Deep Learning
 
Neural Networks - How do they work?
Neural Networks - How do they work?Neural Networks - How do they work?
Neural Networks - How do they work?
 

More from SheilaJimenezMorejon

Add a backend and deploy!
Add a backend and deploy!Add a backend and deploy!
Add a backend and deploy!
SheilaJimenezMorejon
 
Build a chatroom!
Build a chatroom!Build a chatroom!
Build a chatroom!
SheilaJimenezMorejon
 
Intro to React
Intro to ReactIntro to React
Intro to React
SheilaJimenezMorejon
 
Building Beautiful Flutter Apps
Building Beautiful Flutter AppsBuilding Beautiful Flutter Apps
Building Beautiful Flutter Apps
SheilaJimenezMorejon
 
Git & GitHub WorkShop
Git & GitHub WorkShopGit & GitHub WorkShop
Git & GitHub WorkShop
SheilaJimenezMorejon
 
Information session
Information sessionInformation session
Information session
SheilaJimenezMorejon
 
Flutter introduction
Flutter introductionFlutter introduction
Flutter introduction
SheilaJimenezMorejon
 

More from SheilaJimenezMorejon (7)

Add a backend and deploy!
Add a backend and deploy!Add a backend and deploy!
Add a backend and deploy!
 
Build a chatroom!
Build a chatroom!Build a chatroom!
Build a chatroom!
 
Intro to React
Intro to ReactIntro to React
Intro to React
 
Building Beautiful Flutter Apps
Building Beautiful Flutter AppsBuilding Beautiful Flutter Apps
Building Beautiful Flutter Apps
 
Git & GitHub WorkShop
Git & GitHub WorkShopGit & GitHub WorkShop
Git & GitHub WorkShop
 
Information session
Information sessionInformation session
Information session
 
Flutter introduction
Flutter introductionFlutter introduction
Flutter introduction
 

Recently uploaded

"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
Fwdays
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's TipsGetting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
ScyllaDB
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
BibashShahi
 
Discover the Unseen: Tailored Recommendation of Unwatched Content
Discover the Unseen: Tailored Recommendation of Unwatched ContentDiscover the Unseen: Tailored Recommendation of Unwatched Content
Discover the Unseen: Tailored Recommendation of Unwatched Content
ScyllaDB
 
AWS Certified Solutions Architect Associate (SAA-C03)
AWS Certified Solutions Architect Associate (SAA-C03)AWS Certified Solutions Architect Associate (SAA-C03)
AWS Certified Solutions Architect Associate (SAA-C03)
HarpalGohil4
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
Fwdays
 
Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!
Ortus Solutions, Corp
 
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdfLee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
leebarnesutopia
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
DianaGray10
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
Ajin Abraham
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
Containers & AI - Beauty and the Beast!?!
Containers & AI - Beauty and the Beast!?!Containers & AI - Beauty and the Beast!?!
Containers & AI - Beauty and the Beast!?!
Tobias Schneck
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
Enterprise Knowledge
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
ScyllaDB
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Ukraine
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
Fwdays
 

Recently uploaded (20)

"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin..."$10 thousand per minute of downtime: architecture, queues, streaming and fin...
"$10 thousand per minute of downtime: architecture, queues, streaming and fin...
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's TipsGetting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
Getting the Most Out of ScyllaDB Monitoring: ShareChat's Tips
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
Principle of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptxPrinciple of conventional tomography-Bibash Shahi ppt..pptx
Principle of conventional tomography-Bibash Shahi ppt..pptx
 
Discover the Unseen: Tailored Recommendation of Unwatched Content
Discover the Unseen: Tailored Recommendation of Unwatched ContentDiscover the Unseen: Tailored Recommendation of Unwatched Content
Discover the Unseen: Tailored Recommendation of Unwatched Content
 
AWS Certified Solutions Architect Associate (SAA-C03)
AWS Certified Solutions Architect Associate (SAA-C03)AWS Certified Solutions Architect Associate (SAA-C03)
AWS Certified Solutions Architect Associate (SAA-C03)
 
"What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w..."What does it really mean for your system to be available, or how to define w...
"What does it really mean for your system to be available, or how to define w...
 
Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!Introducing BoxLang : A new JVM language for productivity and modularity!
Introducing BoxLang : A new JVM language for productivity and modularity!
 
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdfLee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
Lee Barnes - Path to Becoming an Effective Test Automation Engineer.pdf
 
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectorsConnector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
Connector Corner: Seamlessly power UiPath Apps, GenAI with prebuilt connectors
 
AppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSFAppSec PNW: Android and iOS Application Security with MobSF
AppSec PNW: Android and iOS Application Security with MobSF
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
Containers & AI - Beauty and the Beast!?!
Containers & AI - Beauty and the Beast!?!Containers & AI - Beauty and the Beast!?!
Containers & AI - Beauty and the Beast!?!
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
Demystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through StorytellingDemystifying Knowledge Management through Storytelling
Demystifying Knowledge Management through Storytelling
 
A Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's ArchitectureA Deep Dive into ScyllaDB's Architecture
A Deep Dive into ScyllaDB's Architecture
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
GlobalLogic Java Community Webinar #18 “How to Improve Web Application Perfor...
 
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba"NATO Hackathon Winner: AI-Powered Drug Search",  Taras Kloba
"NATO Hackathon Winner: AI-Powered Drug Search", Taras Kloba
 

Introduction to Applied Machine Learning

  • 1. BUMIC + DSC React Series Darcy 03/09/2021 Introduction to Applied Machine Learning
  • 2. Applications of Deep Learning 1. Cool things using deep learning a. Computer Vision i. Tesla recognizing items on a street b. Text generation i. OpenAI GPT3 can solve almost any language task in a few examples c. Reinforcement Learning i. Can play Atari games, Board games, Real Time Strategy games ii. Robotic control d. Many more... 2
  • 4. We have some data D 4 D X Y
  • 5. Make an assumption about D 5 D X Y
  • 6. What is learning? 6 The approximation of some unknown function f based on some data D. D X Y How do we set the parameters? How do we know what assumptions to make?
  • 7. Intro to Deep Learning
  • 8. What is Deep Learning 8 Artificial Intelligence Machine Learning Deep Learning Deep learning is a subset of machine learning
  • 9. What is Deep Learning 9 Deep learning learns from data using a class of functions known as Neural Networks A neural network maps an input to an output
  • 10. Biological Neuron vs. Artificial Neuron Andrej Karpathy
  • 11. What is a Neural Network?
  • 13. Push example through the network to get a predicted output Forward propagation Input Number of Bedrooms Number of Bathrooms Square Feet Hidden 1 Hidden 2 Hidden 3 Output Price of House
  • 14. Calculate difference between predicted output and actual data Compute the cost Output Price of House D X Y
  • 15. Calculate difference between predicted output and actual data Compute the cost Output Price of House Where i is the ith training example and m is the number of training examples
  • 16. Push back the derivative of the error and apply to each weight, such that next time it will result in a lower error Backward propagation - “Update” https://hmkcode.github.io/ai/backpropagation-step-by-step/
  • 18. Image Data 18 ● Images are commonly represented in code as a 3D array of pixels. Here, we notice 3 represents RGB values ● In vanilla neural networks, we would simply flatten this 3D array into a 3072 length vector. However, by doing this, we lose spatial correlation between pixels close to other pixels
  • 19. Image Data 19 ● In 2012 a paper called AlexNet out competed state of the art image classification models through the usage of kernels (also called filters)
  • 20. Kernel 20 ● Kernel: a small matrix used for feature detection on an image ○ Also called a filter ● Usage ○ Superimpose the kernel over a section of an image ○ Do element-wise multiplication between the weights in the kernel and the values in the image ○ Record the sum of the multiplications
  • 21. Example Convolution 21 Example: Multiply the 5x5 image by a 3x3 kernel with weights: 1 0 1 0 1 0 1 0 1 The output? Sum of weight times part of image to a single number.
  • 22. Kernel example 22 6 3 2 4 3 1 3 5 5 0 1 0 1 2 1 0 1 0 * = 19 Section of an image Kernel 6*0 3*1 2*0 4*1 3*2 1*1 3*0 5*1 5*0 = sum
  • 23. Kernel 3 3 1 4 6 5 3 5 2 Kernel example (cont.) 23 Section of an image This image section contains the same values as before, but they have been rearranged, resulting in a greater activation with this kernel 0 1 0 1 2 1 0 1 0 * = 23 Section of an image 3*0 3*1 1*0 4*1 6*2 5*1 3*0 5*1 2*0 = sum
  • 24. Example Convolution 24 ● Note that the output is smaller than the input ● This can be prevented by using padding around the edges of the image.
  • 25. Padding 25 ● Before padding: ○ 7x7 input, 3x3 filter creating a 5x5 sized output ● After padding: ○ 9x9 input, 3x3 filter creating a 7x7 sized output which maintains the same size as our input ● Edges and corners aren’t as accurate but in practice this works well enough 7 9
  • 26. Stride 26 ● Here, the kernel is moving one pixel at a time (“stride” = 1) ● The kernel can move by more than one pixel at a time ● Size = (N - F) / Stride + 1
  • 27. Stride 27 ● Increasing stride decreases the size of the output ● Here, stride = 2 ● (N - F) / Stride + 1 (7 - 3) / 2 + 1 = 3
  • 28. Dimensionality Practice 28 ● What would be the output size of a 5x5x3 filter with a 32x32x3 image and a stride of 1? ● (N - F) / Stride + 1
  • 29. Dimensionality Practice 29 ● (32 - 5) / 1 + 1 = 28 ● Now let’s say we had a stride of 2, ○ (32 - 5) / 2 + 1 = 14.5 ○ Fractional size means the filter hangs off the input ○ We wouldn’t use this stride value consequently
  • 30. Intentionally Shrinking Output Size 30 ● Now, let’s say you want to shrink your outputs (which are inputs to the next layer) to reduce operations. ● You can do this by either increasing the stride ○ (N - F)/Stride + 1 ● Alternatively, you can use a pooling layer
  • 31. Conv Layer Output 31 ● Use multiple kernels for multiple activation maps ● In this example, we have 6 activation maps each created through a different filter with its own set of weights and biases
  • 33. Pooling Layers 33 ● Limitation of output of Convolutional Layers: ○ Record the precise position of features in the input ○ Small movements in the position of the feature in the input image will result in a different feature map ● Solution: Pooling Layers ○ Lower resolution version of input is created with large and important structure elements preserved ○ Reduces the computational cost by reducing the number of parameters to learn
  • 34. Max Pooling 34 Input (4 x 4) Output (2 x 2) Extracts the sharpest features of an image, making it more general
  • 35. Average Pooling 35 Input (4 x 4) Output (2 x 2) Takes average feature of an image, minimize overfitting
  • 36. Dropout 1. First, what is overfitting? a. Overfitting is when the neural network corresponds too closely to the dataset, and cannot be generalized. This tends to happen when a model is excessively complex relative to the data b. Conversely, underfitting is when the network cannot capture the underlying trend of the dataset which may happen if your network is not complicated enough. 36
  • 37. Dropout - How can we solve overfitting? 1. Training phase a. Each weight has a probability p that they will be multiplied by zero (dropped). This probability is often set to 0.5, which is considered to be close to optimal for a wide range of networks and tasks b. This has the effect of removing random connections between activations effectively creating a new network/outlook on the data per each train set 37
  • 38. Dropout - How can we solve overfitting? 2. Post Train a. After training weights will be abnormally high as they were adjusted assuming only (1-p) percent of the weights would be summed together and used. b. To fix this we normalize weights to lower the expectation of each weight. We do this by scaling each weight by 1/p c. “This makes sure that for each unit, the expected output from it under random dropout will be the same as the output during pretraining.” ~Dropout: A Simple Way to Prevent Neural Networks from Overfitting i. http://www.jmlr.org/papers/volume15/srivastava14a/srivastava14a.pdf 38
  • 40. So what does our network look like? 40 ●