SlideShare a Scribd company logo
Neural Networks
and Deep Learning
Tijmen Blankevoort
Scyfer

Prof dr. Max Welling

Drs. Jorgen Sandig

Msc. Taco Cohen
Deep Learning
All purpose machine learning
Using Neural Networks:
- Using large amounts of data
- Learning very complex problems
- Automatically learning features
A new era of machine learning
Deep learning wins all competitions
- IJCNN 2011 Traffic Sign Recognition Competition
- ISBI 2012 Segmentation of neuronal structures in EM
stacks challenge
- ICDAR 2011 Chinese
handwriting recognition
Applications
A lot of state of the art systems use deep learning to some
extent:
- IBMs Watson: Jeopardy contest 2011
- Google’s self-driving car
- Google Glasses
- Facebook face recognition
- Facebook user modelling
Mostly image and sound recognition tasks (difficult)
Google Brain (2011)
- 10 million youtube/imagenet images
- 1 billion parameters
- 16.000 processors
- Largely unsupervised!
- 20.000 categories
- 15.8% accuracy
Bigger, better
Deep Learning:
- The scope of what
computers can learn has
greatly been increased
- Interaction with the real
world
Biological Inspiration

Neuron
Neuron computer model
Activation function

Sigmoid activation function
Neuron computer model

Perceptron - 1957 Rosenblatt
Easy functions with a neuron
Linking neurons and training
- Initialize randomly
- Sequentially give it data.
- See what the difference is between
network output and actual output.
- Update the weights according to this error.
- End result: give a model input, and it
produces a proper output.
Quest for the weights. The weights are the
model!
The Perceptron (1958)

“A machine which senses, recognizes, remembers, and responds like the human mind”
“Remarkable machine… [was] capable of what amounts to thought” - The New Yorker
Criticism and downfall (1969)
- Perceptrons are painfully limited. They can not even learn a
simple XOR function!
- No feasible way of learning networks with multiple layers
- Interest in neural networks close to fully disappeared
Renewed interest (90’s)
- Learning multiple layers
- “Back propagation”
- Can theoretically learn any
function!
But…
Very slow and inefficient
- Machine learning attention
towards SVMs, random forests
etc.
Deep learing (2006)
- Quest: Mimic human brain representations
- Large networks
- Lots of data

Problem:
Simple back propagation fails
on large networks.
Deep learning (2006)
- Exactly same networks as
before, just BIGGER

- Combination of three factors:
- (Big data)
- Better algorithms
- Parallel computing (GPU)
Better algorithms

Restricted Boltzmann machine
Pre-training: Learn the representation by parts!
Very strong unsupervised learning
After pre-training, use back propagation
Parallel (GPU) power
- Every set of weights can be stored as a matrix (w_ij)
- GPUs are made to do common parallel problems fast!
- All similar calculations done at the same time, huge performance boost.
- CPU parallelizing
Future of Deep Learning
- Currently an explosion of developments
- Hessian-Free networks (2010)
- Long Short Term Memory (2011)
- Large Convolutional nets, max-pooling (2011)
- Nesterov’s Gradient Descent (2013)
- Currently state of the art but...
- No way of doing logical inference (extrapolation)
- No easy integration of abstract knowledge
- Hypothetic space bias might not conform with reality
When to apply Deep Learning
- Generally, vision and sound
recognition, but...
- Works great for any other problem too!
- A lot of data / features
- Don’t want to make your own features
- State of the art results
How to apply Deep Learning
Deep learning is very difficult!
- No easy plug and play software
- Far too many different networks/options/additions
- Mathematics and programming very challenging
- Research is fast paced
- Learning a network is both an art and a science
My advice:
Cooperation university <=> business
How to apply Deep Learning
- For most current business problems, no need for
expensive hardware. e.g. we use a laptop

More Related Content

What's hot

Deep learning - A Visual Introduction
Deep learning - A Visual IntroductionDeep learning - A Visual Introduction
Deep learning - A Visual Introduction
Lukas Masuch
 
Deep learning
Deep learningDeep learning
Deep learning
Ratnakar Pandey
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
Gaurav Mittal
 
Deep learning
Deep learningDeep learning
Deep learning
Mohamed Loey
 
Deep learning
Deep learningDeep learning
Deep learning
Pratap Dangeti
 
1.Introduction to deep learning
1.Introduction to deep learning1.Introduction to deep learning
1.Introduction to deep learning
KONGU ENGINEERING COLLEGE
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network
Yan Xu
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networks
Si Haem
 
Deep Learning: Application & Opportunity
Deep Learning: Application & OpportunityDeep Learning: Application & Opportunity
Deep Learning: Application & Opportunity
iTrain
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
Oswald Campesato
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learning
Massimiliano Ruocco
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Simplilearn
 
Machine Learning vs. Deep Learning
Machine Learning vs. Deep LearningMachine Learning vs. Deep Learning
Machine Learning vs. Deep Learning
Belatrix Software
 
Deep Learning Explained
Deep Learning ExplainedDeep Learning Explained
Deep Learning Explained
Melanie Swan
 
Convolution Neural Network (CNN)
Convolution Neural Network (CNN)Convolution Neural Network (CNN)
Convolution Neural Network (CNN)
Suraj Aavula
 
Deep learning presentation
Deep learning presentationDeep learning presentation
Deep learning presentation
Tunde Ajose-Ismail
 
An Introduction to Deep Learning
An Introduction to Deep LearningAn Introduction to Deep Learning
An Introduction to Deep Learning
Poo Kuan Hoong
 
The Deep Learning Glossary
The Deep Learning GlossaryThe Deep Learning Glossary
The Deep Learning Glossary
NVIDIA
 
Deep Learning
Deep LearningDeep Learning
Deep Learning
MoctardOLOULADE
 
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete DeckAI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
SlideTeam
 

What's hot (20)

Deep learning - A Visual Introduction
Deep learning - A Visual IntroductionDeep learning - A Visual Introduction
Deep learning - A Visual Introduction
 
Deep learning
Deep learningDeep learning
Deep learning
 
Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)Convolutional Neural Networks (CNN)
Convolutional Neural Networks (CNN)
 
Deep learning
Deep learningDeep learning
Deep learning
 
Deep learning
Deep learningDeep learning
Deep learning
 
1.Introduction to deep learning
1.Introduction to deep learning1.Introduction to deep learning
1.Introduction to deep learning
 
Convolutional neural network
Convolutional neural network Convolutional neural network
Convolutional neural network
 
Deep neural networks
Deep neural networksDeep neural networks
Deep neural networks
 
Deep Learning: Application & Opportunity
Deep Learning: Application & OpportunityDeep Learning: Application & Opportunity
Deep Learning: Application & Opportunity
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
 
Introduction to Deep learning
Introduction to Deep learningIntroduction to Deep learning
Introduction to Deep learning
 
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
Convolutional Neural Network - CNN | How CNN Works | Deep Learning Course | S...
 
Machine Learning vs. Deep Learning
Machine Learning vs. Deep LearningMachine Learning vs. Deep Learning
Machine Learning vs. Deep Learning
 
Deep Learning Explained
Deep Learning ExplainedDeep Learning Explained
Deep Learning Explained
 
Convolution Neural Network (CNN)
Convolution Neural Network (CNN)Convolution Neural Network (CNN)
Convolution Neural Network (CNN)
 
Deep learning presentation
Deep learning presentationDeep learning presentation
Deep learning presentation
 
An Introduction to Deep Learning
An Introduction to Deep LearningAn Introduction to Deep Learning
An Introduction to Deep Learning
 
The Deep Learning Glossary
The Deep Learning GlossaryThe Deep Learning Glossary
The Deep Learning Glossary
 
Deep Learning
Deep LearningDeep Learning
Deep Learning
 
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete DeckAI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
AI Vs ML Vs DL PowerPoint Presentation Slide Templates Complete Deck
 

Viewers also liked

Learning To Rank data2day 2017
Learning To Rank data2day 2017Learning To Rank data2day 2017
Learning To Rank data2day 2017
Stefan Kühn
 
Machine learning with scikitlearn
Machine learning with scikitlearnMachine learning with scikitlearn
Machine learning with scikitlearn
Pratap Dangeti
 
Deep Learning: a birds eye view
Deep Learning: a birds eye viewDeep Learning: a birds eye view
Deep Learning: a birds eye view
Roelof Pieters
 
Artificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep LearningArtificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep Learning
Sujit Pal
 
What Is Conversational Marketing?
What Is Conversational Marketing?What Is Conversational Marketing?
What Is Conversational Marketing?
Drift
 
Deep Learning for NLP: An Introduction to Neural Word Embeddings
Deep Learning for NLP: An Introduction to Neural Word EmbeddingsDeep Learning for NLP: An Introduction to Neural Word Embeddings
Deep Learning for NLP: An Introduction to Neural Word Embeddings
Roelof Pieters
 

Viewers also liked (6)

Learning To Rank data2day 2017
Learning To Rank data2day 2017Learning To Rank data2day 2017
Learning To Rank data2day 2017
 
Machine learning with scikitlearn
Machine learning with scikitlearnMachine learning with scikitlearn
Machine learning with scikitlearn
 
Deep Learning: a birds eye view
Deep Learning: a birds eye viewDeep Learning: a birds eye view
Deep Learning: a birds eye view
 
Artificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep LearningArtificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep Learning
 
What Is Conversational Marketing?
What Is Conversational Marketing?What Is Conversational Marketing?
What Is Conversational Marketing?
 
Deep Learning for NLP: An Introduction to Neural Word Embeddings
Deep Learning for NLP: An Introduction to Neural Word EmbeddingsDeep Learning for NLP: An Introduction to Neural Word Embeddings
Deep Learning for NLP: An Introduction to Neural Word Embeddings
 

Similar to Neural networks and deep learning

Fascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange TomorrowFascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange Tomorrow
Julien SIMON
 
Y conf talk - Andrej Karpathy
Y conf talk - Andrej KarpathyY conf talk - Andrej Karpathy
Y conf talk - Andrej Karpathy
Sze Siong Teo
 
Deep learning & Humanity's Grand Challenges
Deep learning & Humanity's Grand ChallengesDeep learning & Humanity's Grand Challenges
Deep learning & Humanity's Grand Challenges
The Wisdom Daily
 
Deep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial IntelligenceDeep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial Intelligence
Lukas Masuch
 
Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...
Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...
Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...
John Mathon
 
Deep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and HypeDeep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and Hype
Siby Jose Plathottam
 
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2O
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2ODeep Water - Bringing Tensorflow, Caffe, Mxnet to H2O
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2O
Sri Ambati
 
A Platform for Accelerating Machine Learning Applications
 A Platform for Accelerating Machine Learning Applications A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning Applications
NVIDIA Taiwan
 
Artificial intelligence and Neural Network
Artificial intelligence and Neural NetworkArtificial intelligence and Neural Network
Artificial intelligence and Neural Network
Abdullah Saghir Ahmad
 
"Methods for Understanding How Deep Neural Networks Work," a Presentation fro...
"Methods for Understanding How Deep Neural Networks Work," a Presentation fro..."Methods for Understanding How Deep Neural Networks Work," a Presentation fro...
"Methods for Understanding How Deep Neural Networks Work," a Presentation fro...
Edge AI and Vision Alliance
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
Amr Rashed
 
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
lebsoftshore
 
Deep learning introduction
Deep learning introductionDeep learning introduction
Deep learning introduction
giangbui0816
 
From web 2 to web 3
From web 2 to web 3From web 2 to web 3
From web 2 to web 3
Asher Idan
 
Fascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange TomorrowFascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange Tomorrow
Julien SIMON
 
Deep Learning with GPU Instances - Pop-up Loft TLV 2017
Deep Learning with GPU Instances - Pop-up Loft TLV 2017Deep Learning with GPU Instances - Pop-up Loft TLV 2017
Deep Learning with GPU Instances - Pop-up Loft TLV 2017
Amazon Web Services
 
Introduction to the Artificial Intelligence and Computer Vision revolution
Introduction to the Artificial Intelligence and Computer Vision revolutionIntroduction to the Artificial Intelligence and Computer Vision revolution
Introduction to the Artificial Intelligence and Computer Vision revolution
Darian Frajberg
 
Deep learning tutorial 9/2019
Deep learning tutorial 9/2019Deep learning tutorial 9/2019
Deep learning tutorial 9/2019
Amr Rashed
 
Intro to artificial intelligence
Intro to artificial intelligence Intro to artificial intelligence
Intro to artificial intelligence
ankit yadav
 

Similar to Neural networks and deep learning (20)

Fascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange TomorrowFascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange Tomorrow
 
Y conf talk - Andrej Karpathy
Y conf talk - Andrej KarpathyY conf talk - Andrej Karpathy
Y conf talk - Andrej Karpathy
 
Deep learning & Humanity's Grand Challenges
Deep learning & Humanity's Grand ChallengesDeep learning & Humanity's Grand Challenges
Deep learning & Humanity's Grand Challenges
 
Deep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial IntelligenceDeep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial Intelligence
 
Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...
Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...
Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...
 
Deep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and HypeDeep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and Hype
 
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2O
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2ODeep Water - Bringing Tensorflow, Caffe, Mxnet to H2O
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2O
 
Null
NullNull
Null
 
A Platform for Accelerating Machine Learning Applications
 A Platform for Accelerating Machine Learning Applications A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning Applications
 
Artificial intelligence and Neural Network
Artificial intelligence and Neural NetworkArtificial intelligence and Neural Network
Artificial intelligence and Neural Network
 
"Methods for Understanding How Deep Neural Networks Work," a Presentation fro...
"Methods for Understanding How Deep Neural Networks Work," a Presentation fro..."Methods for Understanding How Deep Neural Networks Work," a Presentation fro...
"Methods for Understanding How Deep Neural Networks Work," a Presentation fro...
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
 
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
 
Deep learning introduction
Deep learning introductionDeep learning introduction
Deep learning introduction
 
From web 2 to web 3
From web 2 to web 3From web 2 to web 3
From web 2 to web 3
 
Fascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange TomorrowFascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange Tomorrow
 
Deep Learning with GPU Instances - Pop-up Loft TLV 2017
Deep Learning with GPU Instances - Pop-up Loft TLV 2017Deep Learning with GPU Instances - Pop-up Loft TLV 2017
Deep Learning with GPU Instances - Pop-up Loft TLV 2017
 
Introduction to the Artificial Intelligence and Computer Vision revolution
Introduction to the Artificial Intelligence and Computer Vision revolutionIntroduction to the Artificial Intelligence and Computer Vision revolution
Introduction to the Artificial Intelligence and Computer Vision revolution
 
Deep learning tutorial 9/2019
Deep learning tutorial 9/2019Deep learning tutorial 9/2019
Deep learning tutorial 9/2019
 
Intro to artificial intelligence
Intro to artificial intelligence Intro to artificial intelligence
Intro to artificial intelligence
 

Recently uploaded

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
DianaGray10
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Product School
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
Paul Groth
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
Elena Simperl
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Product School
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Product School
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
Alan Dix
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Product School
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Product School
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
Abida Shariff
 

Recently uploaded (20)

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMsTo Graph or Not to Graph Knowledge Graph Architectures and LLMs
To Graph or Not to Graph Knowledge Graph Architectures and LLMs
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
Knowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and backKnowledge engineering: from people to machines and back
Knowledge engineering: from people to machines and back
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...Designing Great Products: The Power of Design and Leadership by Chief Designe...
Designing Great Products: The Power of Design and Leadership by Chief Designe...
 
Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...Mission to Decommission: Importance of Decommissioning Products to Increase E...
Mission to Decommission: Importance of Decommissioning Products to Increase E...
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI supportEpistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
 
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...
 
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 

Neural networks and deep learning

  • 1. Neural Networks and Deep Learning Tijmen Blankevoort
  • 2. Scyfer Prof dr. Max Welling Drs. Jorgen Sandig Msc. Taco Cohen
  • 3. Deep Learning All purpose machine learning Using Neural Networks: - Using large amounts of data - Learning very complex problems - Automatically learning features A new era of machine learning
  • 4. Deep learning wins all competitions - IJCNN 2011 Traffic Sign Recognition Competition - ISBI 2012 Segmentation of neuronal structures in EM stacks challenge - ICDAR 2011 Chinese handwriting recognition
  • 5. Applications A lot of state of the art systems use deep learning to some extent: - IBMs Watson: Jeopardy contest 2011 - Google’s self-driving car - Google Glasses - Facebook face recognition - Facebook user modelling Mostly image and sound recognition tasks (difficult)
  • 6. Google Brain (2011) - 10 million youtube/imagenet images - 1 billion parameters - 16.000 processors - Largely unsupervised! - 20.000 categories - 15.8% accuracy
  • 7. Bigger, better Deep Learning: - The scope of what computers can learn has greatly been increased - Interaction with the real world
  • 11. Neuron computer model Perceptron - 1957 Rosenblatt
  • 13. Linking neurons and training - Initialize randomly - Sequentially give it data. - See what the difference is between network output and actual output. - Update the weights according to this error. - End result: give a model input, and it produces a proper output. Quest for the weights. The weights are the model!
  • 14. The Perceptron (1958) “A machine which senses, recognizes, remembers, and responds like the human mind” “Remarkable machine… [was] capable of what amounts to thought” - The New Yorker
  • 15. Criticism and downfall (1969) - Perceptrons are painfully limited. They can not even learn a simple XOR function! - No feasible way of learning networks with multiple layers - Interest in neural networks close to fully disappeared
  • 16. Renewed interest (90’s) - Learning multiple layers - “Back propagation” - Can theoretically learn any function! But… Very slow and inefficient - Machine learning attention towards SVMs, random forests etc.
  • 17. Deep learing (2006) - Quest: Mimic human brain representations - Large networks - Lots of data Problem: Simple back propagation fails on large networks.
  • 18. Deep learning (2006) - Exactly same networks as before, just BIGGER - Combination of three factors: - (Big data) - Better algorithms - Parallel computing (GPU)
  • 19. Better algorithms Restricted Boltzmann machine Pre-training: Learn the representation by parts! Very strong unsupervised learning After pre-training, use back propagation
  • 20. Parallel (GPU) power - Every set of weights can be stored as a matrix (w_ij) - GPUs are made to do common parallel problems fast! - All similar calculations done at the same time, huge performance boost. - CPU parallelizing
  • 21. Future of Deep Learning - Currently an explosion of developments - Hessian-Free networks (2010) - Long Short Term Memory (2011) - Large Convolutional nets, max-pooling (2011) - Nesterov’s Gradient Descent (2013) - Currently state of the art but... - No way of doing logical inference (extrapolation) - No easy integration of abstract knowledge - Hypothetic space bias might not conform with reality
  • 22. When to apply Deep Learning - Generally, vision and sound recognition, but... - Works great for any other problem too! - A lot of data / features - Don’t want to make your own features - State of the art results
  • 23. How to apply Deep Learning Deep learning is very difficult! - No easy plug and play software - Far too many different networks/options/additions - Mathematics and programming very challenging - Research is fast paced - Learning a network is both an art and a science My advice: Cooperation university <=> business
  • 24. How to apply Deep Learning - For most current business problems, no need for expensive hardware. e.g. we use a laptop