SlideShare a Scribd company logo
1 of 53
Download to read offline
Deep Learning
A Visual Introduction
Interest
Google NGRAM & Google Trends
Hype or Reality?
Quotes
I have worked all my life in Machine Learning, and I’ve never seen one
algorithm knock over benchmarks like Deep Learning
– Andrew Ng (Stanford & Baidu)
Deep Learning is an algorithm which has no theoretical limitations
of what it can learn; the more data you give and the more
computational time you provide, the better it is – Geoffrey Hinton (Google)
Human-level artificial intelligence has the potential to help humanity
thrive more than any invention that has come before it – Dileep George
(Co-Founder Vicarious)
For a very long time it will be a complementary tool that human
scientists and human experts can use to help them with the things
that humans are not naturally good – Demis Hassabis (Co-Founder DeepMind)
Hype or Reality?
Deep Learning at Google
Hype or Reality?
NIPS (Computational Neuroscience Conference) Growth
What is Artificial Intelligence?
Artificial
Intelligence
Output:
Movement
Text
Input:
Sensors
Data
Machine Learning - Basics
Introduction
Machine Learning is a type of Artificial Intelligence that provides
computers with the ability to learn without being explicitly
programmed.
Machine Learning
Algorithm
Learned Model
Data
Prediction
Labeled Data
Training
Prediction
Provides various techniques that can learn from and make predictions on data
Machine Learning - Basics
Learning Approaches
Supervised Learning: Learning with a labeled training set
Example: email spam detector with training set of already labeled emails
Unsupervised Learning: Discovering patterns in unlabeled data
Example: cluster similar documents based on the text content
Reinforcement Learning: learning based on feedback or reward
Example: learn to play chess by winning or losing
Machine Learning - Basics
Problem Types
Regression
(supervised – predictive)
Classification
(supervised – predictive)
Anomaly Detection
(unsupervised– descriptive)
Clustering
(unsupervised – descriptive)
Machine Learning - Basics
Algorithms Comparison - Classification
What is Deep Learning?
Part of the machine learning field of learning representations of
data. Exceptional effective at learning patterns.
Utilizes learning algorithms that derive meaning out of data by using
a hierarchy of multiple layers that mimic the neural networks of our
brain.
If you provide the system tons of information, it begins to
understand it and respond in useful ways.
Inspired by the Brain
The first hierarchy of neurons
that receives information in the
visual cortex are sensitive to
specific edges while brain regions
further down the visual pipeline
are sensitive to more complex
structures such as faces.
Our brain has lots of neurons connected together and the strength of
the connections between neurons represents long term knowledge.
One learning algorithm hypothesis: all significant mental algorithms
are learned except for the learning and reward machinery itself.1
Why Deep Learning?
Applications
Speech
Recognition
Computer
Vision
Natural Language
Processing
A brief History
A long time ago…
1974 Backpropagation
awkward silence (AI Winter)
1995
SVM reigns
Convolution Neural Networks for
Handwritten Recognition
1998
2006
Restricted
Boltzmann
Machine
1958 Perceptron
1969
Perceptron criticized
Google Brain Project on
16k Cores
2012
2012
AlexNet wins
ImageNet
A brief History
The Big Bang aka “One net to rule them all”
ImageNet: The “computer vision World Cup”
A brief History
The Big Bang aka “One net to rule them all”
Deep Learning in Speech Recognition
What changed?
Old wine in new bottles
Big Data
(Digitalization)
Computation
(Moore’s Law, GPUs)
Algorithmic
Progress
The Big Players
Superstar Researchers
Geoffrey Hinton: University of Toronto & Google
Yann LeCun: New York University & Facebook
Andrew Ng: Stanford & Baidu
Yoshua Bengio: University of Montreal
Jürgen Schmidhuber: Swiss AI Lab & NNAISENSE
The Big Players
Companies
The Big Players
Startups
Acquired
DNNresearch
Deep Learning - Basics
No more feature engineering
Feature
Engineering
Traditional
Learning
AlgorithmInput Data
Costs lots of time
Deep
Learning
AlgorithmInput Data
Deep Learning - Basics
Architecture
A deep neural network consists of a hierarchy of layers, whereby each layer
transforms the input data into more abstract representations (e.g. edge ->
nose -> face). The output layer combines those features to make predictions.
Deep Learning - Basics
What did it learn?
Edges Nose, Eye… Faces
Deep Learning - Basics
Artificial Neural Networks
Consists of one input, one output and multiple fully-connected hidden layers in-
between. Each layer is represented as a series of neurons and progressively extracts
higher and higher-level features of the input until the final layer essentially makes a
decision about what the input shows. The more layers the network has, the higher-
level features it will learn.
Deep Learning - Basics
The Neuron
An artificial neuron contains a nonlinear activation function and has
several incoming and outgoing weighted connections.
Neurons are trained to filter and detect specific features or patterns
(e.g. edge, nose) by receiving weighted input, transforming it with
the activation function und passing it to the outgoing connections.
Deep Learning - Basics
Non-linear Activation Function
Non-linearity is needed to learn complex (non-linear)
representations of data, otherwise the NN would be just
a linear function.
Most deep networks use ReLU -
max(0,x) - nowadays for hidden
layers, since it trains much faster, is
more expressive than logistic
function and prevents the gradient
vanishing problem.
Deep Learning - Basics
The Training Process
Forward it trough
the network to get
predictionsSample labeled data
Backpropagate
the errors
Update the
connection weights
Learns by generating an error signal that measures the difference between the
predictions of the network and the desired values and then using this error signal
to change the weights (or parameters) so that predictions get more accurate.
Deep Learning - Basics
Gradient Descent
Gradient Descent finds the (local) the minimum of the cost function (used to
calculate the output error) and is used to adjust the weights.
Deep Learning - Basics
Data transformation in other dimensions
A neural network is transforming the data into other dimensions to solve
the specified problem.
Deep Learning - Basics
Deep Autoencoders
Composed of two symmetrical
deep-belief networks. The encoding
network learns to compresses the
input to a condensed vector
(dimensionality reduction). The
decoding network can be used to
reconstruct the data.
Topic Modeling: Document in a collection is converted to a Bag-of-
Words and transformed to a compressed feature vector using an
autoencoder. The distance from every other document-vector can be
measured and nearby document-vectors fall under the same topic.
Deep Learning - Basics
Convolutional Neural Nets (CNN)
Convolutional Neural Networks learn a complex representation of visual data
using vast amounts of data. They are inspired by the human visual system and
learn multiple layers of transformations, which are applied on top of each other
to extract a progressively more sophisticated representation of the input.
Every layer of a CNN takes a 3D volume of numbers and outputs a 3D volume of
numbers. E.g. Image is a 224*224*3 (RGB) cube and will be transformed to
1*1000 vector of probabilities.
Deep Learning - Basics
Convolutional Neural Nets (CNN)
Convolution layer is a feature detector that automagically learns to filter out not
needed information from an input by using convolution kernel.
Pooling layers compute the max or average value of a particular feature over a
region of the input data (downsizing of input images). Also helps to detect objects
in some unusual places and reduces memory size.
Deep Learning - Basics
Recurrent Neural Nets (RNN)
general computers which can learn algorithms to map input sequences to
output sequences
RNNs are general computers which can learn algorithms to map input
sequences to output sequences (flexible-sized vectors). The output
vector’s contents are influenced by the entire history of inputs.
general computers which can learn algorithms to map input sequences to
output sequences
State-of-the-art results in time series prediction, adaptive robotics,
handwriting recognition, image classification, speech recognition,
stock market prediction, and other sequence learning problems.
Everything can be processed sequentially.
Deep Learning - Basics
Long Short-Term Memory RNN (LSTM)
A Long Short-Term Memory (LSTM) network is a
particular type of recurrent network that works
slightly better in practice, owing to its more
powerful update equation and some appealing
back propagation dynamics.
general computers which can learn algorithms to map input sequences to
output sequences
The LSTM units give the network memory cells with read, write
and reset operations. During training, the network can learn when
it should remember data and when it should throw it away.
general computers which can learn algorithms to map input sequences to
output sequences
Well-suited to learn from experience to classify, process
and predict time series when there are very long time lags of
unknown size between important events.
Deep Learning - Basics
Recurrent Neural Nets (RNN) – Generating Text
To train the RNN, insert characters sequentially and
predict the probabilities of the next letter.
Backpropagate error and update RNN’s weights to
increase the confidence of the correct letter (green)
and decrease the confidence of all other letters (red).
Trained on structured Wikipedia markdown. Network learns to spell English words
completely from scratch and copy general syntactic structures.
Deep Learning - Basics
Recurrent Neural Nets (RNN) – Generating Text
To generate text, we feed a character into the trained RNN and get a distribution
over what characters are likely to come next (red = likely). We sample from this
distribution, and feed it right back in to get the next letter.
This highlighted neuron gets very excited (green = excited, blue = not excited) when
the RNN is inside the [[ ]] markdown environment and turns off outside of it.
The RNN is likely using this neuron to remember if it is inside a URL or not.
Deep Learning - Basics
Image Captioning – Combining CNN and RNN
Neural Image Caption
Generator generates fitting
natural-language captions
only based on the pixels by
combining a vision CNN and
a language-generating RNN.
A man flying through the air
while riding a skateboard
Two pizzas sitting on top of
a stove top oven
A close up of a child holding
a stuffed animal
Deep Learning - Basics
Natural Language Processing – Embeddings
Embeddings are used to turn textual data (words, sentences, paragraphs) into high-
dimensional vector representations and group them together with semantically
similar data in a vectorspace. Thereby, computer can detect similarities
mathematically.
Deep Learning - Basics
Natural Language Processing – Word2Vec
Word2Vec is an unsupervised learning algorithm for obtaining vector
representations for words. These vectors were trained for a specific domain on
a very large textual data set. GloVe is a better performing alternative.
It detects similarities mathematically by grouping the vectors of similar words together.
All it needs is words co-occurance in the given corpus.
Deep Learning - Basics
Natural Language Processing – Word2Vec
Woman – Man ≈ Aunt - Uncle
King - Male + Female ≈ Queen
Human - Animal ≈ Ethics
Deep Learning - Basics
Natural Language Processing – Thought Vectors
Thought vectors is a way of embedding thoughts in vector space. Their
features will represent how each thought relates to other thoughts.
By reading every document on the web, computers might be able to
reason like humans do by mimicking the thoughts expressed in content.
A neural machine translation is trained on bilingual text
using a encoder and decoder RNN. For translation, the input
sentence is transformed into a thought vector. This vector is
used to reconstruct the given thought in another language.
Deep Learning - Basics
DeepMind Deep Q-Learning
Deep Q-Learning (DQN) is a model-free approach to reinforcement learning
using deep networks in environments with discrete action choices
Deep Learning - Basics
DeepMind Deep Q-Learning
Outperforms humans in over 30 Atari games just by receiving the pixels on the
screen with the goal to maximize the score (Reinforcement Learning)
Policy distillation: Extracts the learned state (policy) of a
reinforcement learning agent (teacher) and trains a new
network (student) that performs at the expert level while
being dramatically smaller and more efficient.
Single-task policy distillation Multi-task policy distillation
Deep Learning - Basics
DeepMind Deep Q-Learning
Deep Learning - Basics
Usage Requirements
Large data set with good quality (input-output mappings)
Measurable and describable goals (define the cost)
Enough computing power (AWS GPU Instance)
Excels in tasks where the basic unit (pixel, word) has very little meaning
in itself, but the combination of such units has a useful meaning
Deep Learning - Tools
Its all Open Source
Deep Learning - Tools
Computing is affordable
AWS EC2 GPU Spot Instance: g2.2xlarge - $0.0782 per Hour
The DIGITS DevBox combines the
world’s best hardware (4 GPUs),
software, and systems engineering
for deep learning in a powerful
solution that can fit under your
desk. Cost: $15k
Outlook
NVIDIA Pascal
NVIDIA’s Pascal GPU architecture will accelerate
deep learning applications up to 10X beyond the
speed of its current-generation Maxwell processors.
Outlook
Artificial Quantum Intelligence
Quantum Artificial Intelligence Lab is a joint initiative of NASA and Google to
study how quantum computing might advance machine learning. This type of
computing may provide the most creative and parallelized problem-solving
process under the known laws of physics.
Quantum computers handle what are called quantum bits
or qubits that can readily have a value of one or zero or
anything in between.
Quantum computing represents a paradigm shift, a radical
change in the way we do computing and at a scale that has
unimaginable power – Eric Ladizinsky (Co-founder D-Wave)
Outlook
Neuromorphic Chips
IBM TrueNorth is a brain-inspired computer chip that implements
networks of integrate-and-fire spiking artificial neurons and uses
only a tiny 70 mw of power –orders of magnitude less energy
than traditional chips. The system is designed to be able to run
deep-learning algorithms.
1 million
Programmable
Neurons
256 million
Programmable
Synapses
4096
Neurosynaptic
Cores
Significant advances in deep reinforcement and unsupervised
learning
Bigger and more complex architectures based on various
interchangeable modules/techniques
Deeper models that can learn from much fewer training cases
Harder problems such as video understanding and natural language
processing will be successfully tackled by deep learning algorithms
Outlook
Deep Learning
Takeaways
Machines that learn to represent the world from experience.
Deep Learning is no magic! Just statistics in a black box, but
exceptional effective at learning patterns.
We haven’t figured out creativity and human-empathy.
Transitioning from research to consumer products. Will make the
tools you use every day work better, faster and smarter.
Lukas Masuch
@lukasmasuch
+lukasmasuch

More Related Content

What's hot

Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowSri Ambati
 
Deep Learning Explained
Deep Learning ExplainedDeep Learning Explained
Deep Learning ExplainedMelanie Swan
 
Neural networks and deep learning
Neural networks and deep learningNeural networks and deep learning
Neural networks and deep learningJörgen Sandig
 
Image classification using CNN
Image classification using CNNImage classification using CNN
Image classification using CNNNoura Hussein
 
An Introduction to Deep Learning
An Introduction to Deep LearningAn Introduction to Deep Learning
An Introduction to Deep LearningPoo Kuan Hoong
 
Deep Learning Tutorial
Deep Learning TutorialDeep Learning Tutorial
Deep Learning TutorialAmr Rashed
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep LearningOleg Mygryn
 
Convolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsConvolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsKasun Chinthaka Piyarathna
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Gaurav Mittal
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep LearningOswald Campesato
 
Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningMohamed Loey
 
Introduction of Deep Learning
Introduction of Deep LearningIntroduction of Deep Learning
Introduction of Deep LearningMyungjin Lee
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networksSi Haem
 
Deep learning for real life applications
Deep learning for real life applicationsDeep learning for real life applications
Deep learning for real life applicationsAnas Arram, Ph.D
 
Deep learning with tensorflow
Deep learning with tensorflowDeep learning with tensorflow
Deep learning with tensorflowCharmi Chokshi
 

What's hot (20)

Introduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlowIntroduction to Deep Learning, Keras, and TensorFlow
Introduction to Deep Learning, Keras, and TensorFlow
 
Deep Learning Explained
Deep Learning ExplainedDeep Learning Explained
Deep Learning Explained
 
Neural networks and deep learning
Neural networks and deep learningNeural networks and deep learning
Neural networks and deep learning
 
Image classification using CNN
Image classification using CNNImage classification using CNN
Image classification using CNN
 
An Introduction to Deep Learning
An Introduction to Deep LearningAn Introduction to Deep Learning
An Introduction to Deep Learning
 
Deep learning
Deep learning Deep learning
Deep learning
 
Deep Learning Tutorial
Deep Learning TutorialDeep Learning Tutorial
Deep Learning Tutorial
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
 
Convolutional Neural Network and Its Applications
Convolutional Neural Network and Its ApplicationsConvolutional Neural Network and Its Applications
Convolutional Neural Network and Its Applications
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learning
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
 
Deep learning
Deep learningDeep learning
Deep learning
 
Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep Learning
 
Introduction of Deep Learning
Introduction of Deep LearningIntroduction of Deep Learning
Introduction of Deep Learning
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networks
 
Cnn
CnnCnn
Cnn
 
Deep learning for real life applications
Deep learning for real life applicationsDeep learning for real life applications
Deep learning for real life applications
 
Deep learning with tensorflow
Deep learning with tensorflowDeep learning with tensorflow
Deep learning with tensorflow
 
Deep learning presentation
Deep learning presentationDeep learning presentation
Deep learning presentation
 

Viewers also liked

Utilizing social data to connect brands to celebrities
Utilizing social data to connect brands to celebritiesUtilizing social data to connect brands to celebrities
Utilizing social data to connect brands to celebritiesExtract Data Conference
 
Deep learning - Conceptual understanding and applications
Deep learning - Conceptual understanding and applicationsDeep learning - Conceptual understanding and applications
Deep learning - Conceptual understanding and applicationsBuhwan Jeong
 
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
Python for Image Understanding: Deep Learning with Convolutional Neural NetsPython for Image Understanding: Deep Learning with Convolutional Neural Nets
Python for Image Understanding: Deep Learning with Convolutional Neural NetsRoelof Pieters
 
Large Scale Deep Learning with TensorFlow
Large Scale Deep Learning with TensorFlow Large Scale Deep Learning with TensorFlow
Large Scale Deep Learning with TensorFlow Jen Aman
 
Deep Learningと画像認識   ~歴史・理論・実践~
Deep Learningと画像認識 ~歴史・理論・実践~Deep Learningと画像認識 ~歴史・理論・実践~
Deep Learningと画像認識   ~歴史・理論・実践~nlab_utokyo
 

Viewers also liked (9)

Andrew Ng, Chief Scientist at Baidu
Andrew Ng, Chief Scientist at BaiduAndrew Ng, Chief Scientist at Baidu
Andrew Ng, Chief Scientist at Baidu
 
Utilizing social data to connect brands to celebrities
Utilizing social data to connect brands to celebritiesUtilizing social data to connect brands to celebrities
Utilizing social data to connect brands to celebrities
 
Introduction to Deep Learning
Introduction to Deep Learning Introduction to Deep Learning
Introduction to Deep Learning
 
Deep learning
Deep learningDeep learning
Deep learning
 
Deep learning - Conceptual understanding and applications
Deep learning - Conceptual understanding and applicationsDeep learning - Conceptual understanding and applications
Deep learning - Conceptual understanding and applications
 
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
Python for Image Understanding: Deep Learning with Convolutional Neural NetsPython for Image Understanding: Deep Learning with Convolutional Neural Nets
Python for Image Understanding: Deep Learning with Convolutional Neural Nets
 
Large Scale Deep Learning with TensorFlow
Large Scale Deep Learning with TensorFlow Large Scale Deep Learning with TensorFlow
Large Scale Deep Learning with TensorFlow
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
 
Deep Learningと画像認識   ~歴史・理論・実践~
Deep Learningと画像認識 ~歴史・理論・実践~Deep Learningと画像認識 ~歴史・理論・実践~
Deep Learningと画像認識   ~歴史・理論・実践~
 

Similar to Deep learning - A Visual Introduction

Deep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial IntelligenceDeep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial IntelligenceLukas Masuch
 
Top 10 deep learning algorithms you should know in
Top 10 deep learning algorithms you should know inTop 10 deep learning algorithms you should know in
Top 10 deep learning algorithms you should know inAmanKumarSingh97
 
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014lebsoftshore
 
Deep learning tutorial 9/2019
Deep learning tutorial 9/2019Deep learning tutorial 9/2019
Deep learning tutorial 9/2019Amr Rashed
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learningAmr Rashed
 
AI_07_Deep Learning.pptx
AI_07_Deep Learning.pptxAI_07_Deep Learning.pptx
AI_07_Deep Learning.pptxYousef Aburawi
 
BASIC CONCEPT OF DEEP LEARNING.pptx
BASIC CONCEPT OF DEEP LEARNING.pptxBASIC CONCEPT OF DEEP LEARNING.pptx
BASIC CONCEPT OF DEEP LEARNING.pptxRiteshPandey184067
 
What is deep learning and how does it work?
What is deep learning and how does it work?What is deep learning and how does it work?
What is deep learning and how does it work?Eligo Creative Services
 
Deep learning Techniques JNTU R20 UNIT 2
Deep learning Techniques JNTU R20 UNIT 2Deep learning Techniques JNTU R20 UNIT 2
Deep learning Techniques JNTU R20 UNIT 2EXAMCELLH4
 
FreddyAyalaTorchDomineering
FreddyAyalaTorchDomineeringFreddyAyalaTorchDomineering
FreddyAyalaTorchDomineeringFAYALA1987
 
Big Data Intelligence: from Correlation Discovery to Causal Reasoning
Big Data Intelligence: from Correlation Discovery to Causal Reasoning Big Data Intelligence: from Correlation Discovery to Causal Reasoning
Big Data Intelligence: from Correlation Discovery to Causal Reasoning Wanjin Yu
 
let's dive to deep learning
let's dive to deep learninglet's dive to deep learning
let's dive to deep learningMohamed Essam
 
Understanding Deep Learning & Parameter Tuning with MXnet, H2o Package in R
Understanding Deep Learning & Parameter Tuning with MXnet, H2o Package in RUnderstanding Deep Learning & Parameter Tuning with MXnet, H2o Package in R
Understanding Deep Learning & Parameter Tuning with MXnet, H2o Package in RManish Saraswat
 
IRJET- Survey on Text Error Detection using Deep Learning
IRJET-  	  Survey on Text Error Detection using Deep LearningIRJET-  	  Survey on Text Error Detection using Deep Learning
IRJET- Survey on Text Error Detection using Deep LearningIRJET Journal
 

Similar to Deep learning - A Visual Introduction (20)

Deep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial IntelligenceDeep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial Intelligence
 
Deep Learning
Deep LearningDeep Learning
Deep Learning
 
Top 10 deep learning algorithms you should know in
Top 10 deep learning algorithms you should know inTop 10 deep learning algorithms you should know in
Top 10 deep learning algorithms you should know in
 
Deep learning
Deep learningDeep learning
Deep learning
 
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
 
Deep learning tutorial 9/2019
Deep learning tutorial 9/2019Deep learning tutorial 9/2019
Deep learning tutorial 9/2019
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
 
Deep learning
Deep learning Deep learning
Deep learning
 
AI_07_Deep Learning.pptx
AI_07_Deep Learning.pptxAI_07_Deep Learning.pptx
AI_07_Deep Learning.pptx
 
BASIC CONCEPT OF DEEP LEARNING.pptx
BASIC CONCEPT OF DEEP LEARNING.pptxBASIC CONCEPT OF DEEP LEARNING.pptx
BASIC CONCEPT OF DEEP LEARNING.pptx
 
What is deep learning and how does it work?
What is deep learning and how does it work?What is deep learning and how does it work?
What is deep learning and how does it work?
 
Deep learning.pptx
Deep learning.pptxDeep learning.pptx
Deep learning.pptx
 
Deep learning Techniques JNTU R20 UNIT 2
Deep learning Techniques JNTU R20 UNIT 2Deep learning Techniques JNTU R20 UNIT 2
Deep learning Techniques JNTU R20 UNIT 2
 
FreddyAyalaTorchDomineering
FreddyAyalaTorchDomineeringFreddyAyalaTorchDomineering
FreddyAyalaTorchDomineering
 
Big Data Intelligence: from Correlation Discovery to Causal Reasoning
Big Data Intelligence: from Correlation Discovery to Causal Reasoning Big Data Intelligence: from Correlation Discovery to Causal Reasoning
Big Data Intelligence: from Correlation Discovery to Causal Reasoning
 
let's dive to deep learning
let's dive to deep learninglet's dive to deep learning
let's dive to deep learning
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learning
 
Deep Learning Demystified
Deep Learning DemystifiedDeep Learning Demystified
Deep Learning Demystified
 
Understanding Deep Learning & Parameter Tuning with MXnet, H2o Package in R
Understanding Deep Learning & Parameter Tuning with MXnet, H2o Package in RUnderstanding Deep Learning & Parameter Tuning with MXnet, H2o Package in R
Understanding Deep Learning & Parameter Tuning with MXnet, H2o Package in R
 
IRJET- Survey on Text Error Detection using Deep Learning
IRJET-  	  Survey on Text Error Detection using Deep LearningIRJET-  	  Survey on Text Error Detection using Deep Learning
IRJET- Survey on Text Error Detection using Deep Learning
 

More from Lukas Masuch

Enterprise Knowledge Graph
Enterprise Knowledge GraphEnterprise Knowledge Graph
Enterprise Knowledge GraphLukas Masuch
 
Growth Hacking 101
Growth Hacking 101Growth Hacking 101
Growth Hacking 101Lukas Masuch
 
Trend detection and analysis on Twitter
Trend detection and analysis on TwitterTrend detection and analysis on Twitter
Trend detection and analysis on TwitterLukas Masuch
 
City Layer - Pitch
City Layer - PitchCity Layer - Pitch
City Layer - PitchLukas Masuch
 
SparkX - Enterprise Crowdfunding
SparkX - Enterprise CrowdfundingSparkX - Enterprise Crowdfunding
SparkX - Enterprise CrowdfundingLukas Masuch
 
Google Cloud Platform - Building a scalable mobile application
Google Cloud Platform - Building a scalable mobile applicationGoogle Cloud Platform - Building a scalable mobile application
Google Cloud Platform - Building a scalable mobile applicationLukas Masuch
 
Powerpoint for Dummies
Powerpoint for DummiesPowerpoint for Dummies
Powerpoint for DummiesLukas Masuch
 

More from Lukas Masuch (7)

Enterprise Knowledge Graph
Enterprise Knowledge GraphEnterprise Knowledge Graph
Enterprise Knowledge Graph
 
Growth Hacking 101
Growth Hacking 101Growth Hacking 101
Growth Hacking 101
 
Trend detection and analysis on Twitter
Trend detection and analysis on TwitterTrend detection and analysis on Twitter
Trend detection and analysis on Twitter
 
City Layer - Pitch
City Layer - PitchCity Layer - Pitch
City Layer - Pitch
 
SparkX - Enterprise Crowdfunding
SparkX - Enterprise CrowdfundingSparkX - Enterprise Crowdfunding
SparkX - Enterprise Crowdfunding
 
Google Cloud Platform - Building a scalable mobile application
Google Cloud Platform - Building a scalable mobile applicationGoogle Cloud Platform - Building a scalable mobile application
Google Cloud Platform - Building a scalable mobile application
 
Powerpoint for Dummies
Powerpoint for DummiesPowerpoint for Dummies
Powerpoint for Dummies
 

Recently uploaded

Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Karmanjay Verma
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 

Recently uploaded (20)

Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#Microservices, Docker deploy and Microservices source code in C#
Microservices, Docker deploy and Microservices source code in C#
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 

Deep learning - A Visual Introduction

  • 1. Deep Learning A Visual Introduction
  • 2. Interest Google NGRAM & Google Trends
  • 3. Hype or Reality? Quotes I have worked all my life in Machine Learning, and I’ve never seen one algorithm knock over benchmarks like Deep Learning – Andrew Ng (Stanford & Baidu) Deep Learning is an algorithm which has no theoretical limitations of what it can learn; the more data you give and the more computational time you provide, the better it is – Geoffrey Hinton (Google) Human-level artificial intelligence has the potential to help humanity thrive more than any invention that has come before it – Dileep George (Co-Founder Vicarious) For a very long time it will be a complementary tool that human scientists and human experts can use to help them with the things that humans are not naturally good – Demis Hassabis (Co-Founder DeepMind)
  • 4. Hype or Reality? Deep Learning at Google
  • 5. Hype or Reality? NIPS (Computational Neuroscience Conference) Growth
  • 6. What is Artificial Intelligence? Artificial Intelligence Output: Movement Text Input: Sensors Data
  • 7. Machine Learning - Basics Introduction Machine Learning is a type of Artificial Intelligence that provides computers with the ability to learn without being explicitly programmed. Machine Learning Algorithm Learned Model Data Prediction Labeled Data Training Prediction Provides various techniques that can learn from and make predictions on data
  • 8. Machine Learning - Basics Learning Approaches Supervised Learning: Learning with a labeled training set Example: email spam detector with training set of already labeled emails Unsupervised Learning: Discovering patterns in unlabeled data Example: cluster similar documents based on the text content Reinforcement Learning: learning based on feedback or reward Example: learn to play chess by winning or losing
  • 9. Machine Learning - Basics Problem Types Regression (supervised – predictive) Classification (supervised – predictive) Anomaly Detection (unsupervised– descriptive) Clustering (unsupervised – descriptive)
  • 10. Machine Learning - Basics Algorithms Comparison - Classification
  • 11. What is Deep Learning? Part of the machine learning field of learning representations of data. Exceptional effective at learning patterns. Utilizes learning algorithms that derive meaning out of data by using a hierarchy of multiple layers that mimic the neural networks of our brain. If you provide the system tons of information, it begins to understand it and respond in useful ways.
  • 12. Inspired by the Brain The first hierarchy of neurons that receives information in the visual cortex are sensitive to specific edges while brain regions further down the visual pipeline are sensitive to more complex structures such as faces. Our brain has lots of neurons connected together and the strength of the connections between neurons represents long term knowledge. One learning algorithm hypothesis: all significant mental algorithms are learned except for the learning and reward machinery itself.1
  • 14. A brief History A long time ago… 1974 Backpropagation awkward silence (AI Winter) 1995 SVM reigns Convolution Neural Networks for Handwritten Recognition 1998 2006 Restricted Boltzmann Machine 1958 Perceptron 1969 Perceptron criticized Google Brain Project on 16k Cores 2012 2012 AlexNet wins ImageNet
  • 15. A brief History The Big Bang aka “One net to rule them all” ImageNet: The “computer vision World Cup”
  • 16. A brief History The Big Bang aka “One net to rule them all” Deep Learning in Speech Recognition
  • 17. What changed? Old wine in new bottles Big Data (Digitalization) Computation (Moore’s Law, GPUs) Algorithmic Progress
  • 18. The Big Players Superstar Researchers Geoffrey Hinton: University of Toronto & Google Yann LeCun: New York University & Facebook Andrew Ng: Stanford & Baidu Yoshua Bengio: University of Montreal Jürgen Schmidhuber: Swiss AI Lab & NNAISENSE
  • 21. Deep Learning - Basics No more feature engineering Feature Engineering Traditional Learning AlgorithmInput Data Costs lots of time Deep Learning AlgorithmInput Data
  • 22. Deep Learning - Basics Architecture A deep neural network consists of a hierarchy of layers, whereby each layer transforms the input data into more abstract representations (e.g. edge -> nose -> face). The output layer combines those features to make predictions.
  • 23. Deep Learning - Basics What did it learn? Edges Nose, Eye… Faces
  • 24. Deep Learning - Basics Artificial Neural Networks Consists of one input, one output and multiple fully-connected hidden layers in- between. Each layer is represented as a series of neurons and progressively extracts higher and higher-level features of the input until the final layer essentially makes a decision about what the input shows. The more layers the network has, the higher- level features it will learn.
  • 25. Deep Learning - Basics The Neuron An artificial neuron contains a nonlinear activation function and has several incoming and outgoing weighted connections. Neurons are trained to filter and detect specific features or patterns (e.g. edge, nose) by receiving weighted input, transforming it with the activation function und passing it to the outgoing connections.
  • 26. Deep Learning - Basics Non-linear Activation Function Non-linearity is needed to learn complex (non-linear) representations of data, otherwise the NN would be just a linear function. Most deep networks use ReLU - max(0,x) - nowadays for hidden layers, since it trains much faster, is more expressive than logistic function and prevents the gradient vanishing problem.
  • 27. Deep Learning - Basics The Training Process Forward it trough the network to get predictionsSample labeled data Backpropagate the errors Update the connection weights Learns by generating an error signal that measures the difference between the predictions of the network and the desired values and then using this error signal to change the weights (or parameters) so that predictions get more accurate.
  • 28. Deep Learning - Basics Gradient Descent Gradient Descent finds the (local) the minimum of the cost function (used to calculate the output error) and is used to adjust the weights.
  • 29. Deep Learning - Basics Data transformation in other dimensions A neural network is transforming the data into other dimensions to solve the specified problem.
  • 30. Deep Learning - Basics Deep Autoencoders Composed of two symmetrical deep-belief networks. The encoding network learns to compresses the input to a condensed vector (dimensionality reduction). The decoding network can be used to reconstruct the data. Topic Modeling: Document in a collection is converted to a Bag-of- Words and transformed to a compressed feature vector using an autoencoder. The distance from every other document-vector can be measured and nearby document-vectors fall under the same topic.
  • 31. Deep Learning - Basics Convolutional Neural Nets (CNN) Convolutional Neural Networks learn a complex representation of visual data using vast amounts of data. They are inspired by the human visual system and learn multiple layers of transformations, which are applied on top of each other to extract a progressively more sophisticated representation of the input. Every layer of a CNN takes a 3D volume of numbers and outputs a 3D volume of numbers. E.g. Image is a 224*224*3 (RGB) cube and will be transformed to 1*1000 vector of probabilities.
  • 32. Deep Learning - Basics Convolutional Neural Nets (CNN) Convolution layer is a feature detector that automagically learns to filter out not needed information from an input by using convolution kernel. Pooling layers compute the max or average value of a particular feature over a region of the input data (downsizing of input images). Also helps to detect objects in some unusual places and reduces memory size.
  • 33. Deep Learning - Basics Recurrent Neural Nets (RNN) general computers which can learn algorithms to map input sequences to output sequences RNNs are general computers which can learn algorithms to map input sequences to output sequences (flexible-sized vectors). The output vector’s contents are influenced by the entire history of inputs. general computers which can learn algorithms to map input sequences to output sequences State-of-the-art results in time series prediction, adaptive robotics, handwriting recognition, image classification, speech recognition, stock market prediction, and other sequence learning problems. Everything can be processed sequentially.
  • 34. Deep Learning - Basics Long Short-Term Memory RNN (LSTM) A Long Short-Term Memory (LSTM) network is a particular type of recurrent network that works slightly better in practice, owing to its more powerful update equation and some appealing back propagation dynamics. general computers which can learn algorithms to map input sequences to output sequences The LSTM units give the network memory cells with read, write and reset operations. During training, the network can learn when it should remember data and when it should throw it away. general computers which can learn algorithms to map input sequences to output sequences Well-suited to learn from experience to classify, process and predict time series when there are very long time lags of unknown size between important events.
  • 35. Deep Learning - Basics Recurrent Neural Nets (RNN) – Generating Text To train the RNN, insert characters sequentially and predict the probabilities of the next letter. Backpropagate error and update RNN’s weights to increase the confidence of the correct letter (green) and decrease the confidence of all other letters (red). Trained on structured Wikipedia markdown. Network learns to spell English words completely from scratch and copy general syntactic structures.
  • 36. Deep Learning - Basics Recurrent Neural Nets (RNN) – Generating Text To generate text, we feed a character into the trained RNN and get a distribution over what characters are likely to come next (red = likely). We sample from this distribution, and feed it right back in to get the next letter. This highlighted neuron gets very excited (green = excited, blue = not excited) when the RNN is inside the [[ ]] markdown environment and turns off outside of it. The RNN is likely using this neuron to remember if it is inside a URL or not.
  • 37. Deep Learning - Basics Image Captioning – Combining CNN and RNN Neural Image Caption Generator generates fitting natural-language captions only based on the pixels by combining a vision CNN and a language-generating RNN. A man flying through the air while riding a skateboard Two pizzas sitting on top of a stove top oven A close up of a child holding a stuffed animal
  • 38. Deep Learning - Basics Natural Language Processing – Embeddings Embeddings are used to turn textual data (words, sentences, paragraphs) into high- dimensional vector representations and group them together with semantically similar data in a vectorspace. Thereby, computer can detect similarities mathematically.
  • 39. Deep Learning - Basics Natural Language Processing – Word2Vec Word2Vec is an unsupervised learning algorithm for obtaining vector representations for words. These vectors were trained for a specific domain on a very large textual data set. GloVe is a better performing alternative. It detects similarities mathematically by grouping the vectors of similar words together. All it needs is words co-occurance in the given corpus.
  • 40. Deep Learning - Basics Natural Language Processing – Word2Vec Woman – Man ≈ Aunt - Uncle King - Male + Female ≈ Queen Human - Animal ≈ Ethics
  • 41. Deep Learning - Basics Natural Language Processing – Thought Vectors Thought vectors is a way of embedding thoughts in vector space. Their features will represent how each thought relates to other thoughts. By reading every document on the web, computers might be able to reason like humans do by mimicking the thoughts expressed in content. A neural machine translation is trained on bilingual text using a encoder and decoder RNN. For translation, the input sentence is transformed into a thought vector. This vector is used to reconstruct the given thought in another language.
  • 42. Deep Learning - Basics DeepMind Deep Q-Learning Deep Q-Learning (DQN) is a model-free approach to reinforcement learning using deep networks in environments with discrete action choices
  • 43. Deep Learning - Basics DeepMind Deep Q-Learning Outperforms humans in over 30 Atari games just by receiving the pixels on the screen with the goal to maximize the score (Reinforcement Learning)
  • 44. Policy distillation: Extracts the learned state (policy) of a reinforcement learning agent (teacher) and trains a new network (student) that performs at the expert level while being dramatically smaller and more efficient. Single-task policy distillation Multi-task policy distillation Deep Learning - Basics DeepMind Deep Q-Learning
  • 45. Deep Learning - Basics Usage Requirements Large data set with good quality (input-output mappings) Measurable and describable goals (define the cost) Enough computing power (AWS GPU Instance) Excels in tasks where the basic unit (pixel, word) has very little meaning in itself, but the combination of such units has a useful meaning
  • 46. Deep Learning - Tools Its all Open Source
  • 47. Deep Learning - Tools Computing is affordable AWS EC2 GPU Spot Instance: g2.2xlarge - $0.0782 per Hour The DIGITS DevBox combines the world’s best hardware (4 GPUs), software, and systems engineering for deep learning in a powerful solution that can fit under your desk. Cost: $15k
  • 48. Outlook NVIDIA Pascal NVIDIA’s Pascal GPU architecture will accelerate deep learning applications up to 10X beyond the speed of its current-generation Maxwell processors.
  • 49. Outlook Artificial Quantum Intelligence Quantum Artificial Intelligence Lab is a joint initiative of NASA and Google to study how quantum computing might advance machine learning. This type of computing may provide the most creative and parallelized problem-solving process under the known laws of physics. Quantum computers handle what are called quantum bits or qubits that can readily have a value of one or zero or anything in between. Quantum computing represents a paradigm shift, a radical change in the way we do computing and at a scale that has unimaginable power – Eric Ladizinsky (Co-founder D-Wave)
  • 50. Outlook Neuromorphic Chips IBM TrueNorth is a brain-inspired computer chip that implements networks of integrate-and-fire spiking artificial neurons and uses only a tiny 70 mw of power –orders of magnitude less energy than traditional chips. The system is designed to be able to run deep-learning algorithms. 1 million Programmable Neurons 256 million Programmable Synapses 4096 Neurosynaptic Cores
  • 51. Significant advances in deep reinforcement and unsupervised learning Bigger and more complex architectures based on various interchangeable modules/techniques Deeper models that can learn from much fewer training cases Harder problems such as video understanding and natural language processing will be successfully tackled by deep learning algorithms Outlook Deep Learning
  • 52. Takeaways Machines that learn to represent the world from experience. Deep Learning is no magic! Just statistics in a black box, but exceptional effective at learning patterns. We haven’t figured out creativity and human-empathy. Transitioning from research to consumer products. Will make the tools you use every day work better, faster and smarter.