This document provides an overview of deep learning and its applications. It discusses how deep learning uses neural networks with many layers to learn representations of data, such as images and text, in an automated way. For computer vision, deep learning has made major improvements in tasks like object recognition, surpassing human-level performance. Deep learning has also been applied successfully to natural language processing tasks like learning word embeddings. The document suggests deep learning is an important development for achieving more broadly intelligent artificial systems.
What is "deep learning" and why is it suddenly so popular? In this talk I explore how Deep Learning provides a convenient framework for expressing learning problems and using GPUs to solve them efficiently.
It’s long ago, approx. 30 years, since AI was not only a topic for Science-Fiction writers, but also a major research field surrounded with huge hopes and investments. But the over-inflated expectations ended in a subsequent crash and followed by a period of absent funding and interest – the so-called AI winter. However, the last 3 years changed everything – again. Deep learning, a machine learning technique inspired by the human brain, successfully crushed one benchmark after another and tech companies, like Google, Facebook and Microsoft, started to invest billions in AI research. “The pace of progress in artificial general intelligence is incredible fast” (Elon Musk – CEO Tesla & SpaceX) leading to an AI that “would be either the best or the worst thing ever to happen to humanity” (Stephen Hawking – Physicist).
What sparked this new Hype? How is Deep Learning different from previous approaches? Are the advancing AI technologies really a threat for humanity? Let’s look behind the curtain and unravel the reality. This talk will explore why Sundar Pichai (CEO Google) recently announced that “machine learning is a core transformative way by which Google is rethinking everything they are doing” and explain why "Deep Learning is probably one of the most exciting things that is happening in the computer industry” (Jen-Hsun Huang – CEO NVIDIA).
Either a new AI “winter is coming” (Ned Stark – House Stark) or this new wave of innovation might turn out as the “last invention humans ever need to make” (Nick Bostrom – AI Philosoph). Or maybe it’s just another great technology helping humans to achieve more.
What is "deep learning" and why is it suddenly so popular? In this talk I explore how Deep Learning provides a convenient framework for expressing learning problems and using GPUs to solve them efficiently.
It’s long ago, approx. 30 years, since AI was not only a topic for Science-Fiction writers, but also a major research field surrounded with huge hopes and investments. But the over-inflated expectations ended in a subsequent crash and followed by a period of absent funding and interest – the so-called AI winter. However, the last 3 years changed everything – again. Deep learning, a machine learning technique inspired by the human brain, successfully crushed one benchmark after another and tech companies, like Google, Facebook and Microsoft, started to invest billions in AI research. “The pace of progress in artificial general intelligence is incredible fast” (Elon Musk – CEO Tesla & SpaceX) leading to an AI that “would be either the best or the worst thing ever to happen to humanity” (Stephen Hawking – Physicist).
What sparked this new Hype? How is Deep Learning different from previous approaches? Are the advancing AI technologies really a threat for humanity? Let’s look behind the curtain and unravel the reality. This talk will explore why Sundar Pichai (CEO Google) recently announced that “machine learning is a core transformative way by which Google is rethinking everything they are doing” and explain why "Deep Learning is probably one of the most exciting things that is happening in the computer industry” (Jen-Hsun Huang – CEO NVIDIA).
Either a new AI “winter is coming” (Ned Stark – House Stark) or this new wave of innovation might turn out as the “last invention humans ever need to make” (Nick Bostrom – AI Philosoph). Or maybe it’s just another great technology helping humans to achieve more.
Deep Learning for Natural Language ProcessingJonathan Mugan
Deep Learning represents a significant advance in artificial intelligence because it enables computers to represent concepts using vectors instead of symbols. Representing concepts using vectors is particularly useful in natural language processing, and this talk will elucidate those benefits and provide an understandable introduction to the technologies that make up deep learning. The talk will outline ways to get started in deep learning, and it will conclude with a discussion of the gaps that remain between our current technologies and true computer understanding.
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...Simplilearn
This Deep Learning Presentation will help you in understanding what is Deep learning, why do we need Deep learning, applications of Deep Learning along with a detailed explanation on Neural Networks and how these Neural Networks work. Deep learning is inspired by the integral function of the human brain specific to artificial neural networks. These networks, which represent the decision-making process of the brain, use complex algorithms that process data in a non-linear way, learning in an unsupervised manner to make choices based on the input. This Deep Learning tutorial is ideal for professionals with beginners to intermediate levels of experience. Now, let us dive deep into this topic and understand what Deep learning actually is.
Below topics are explained in this Deep Learning Presentation:
1. What is Deep Learning?
2. Why do we need Deep Learning?
3. Applications of Deep Learning
4. What is Neural Network?
5. Activation Functions
6. Working of Neural Network
Simplilearn’s Deep Learning course will transform you into an expert in deep learning techniques using TensorFlow, the open-source software library designed to conduct machine learning & deep neural network research. With our deep learning course, you’ll master deep learning and TensorFlow concepts, learn to implement algorithms, build artificial neural networks and traverse layers of data abstraction to understand the power of data and prepare you for your new role as deep learning scientist.
Why Deep Learning?
It is one of the most popular software platforms used for deep learning and contains powerful tools to help you build and implement artificial neural networks.
Advancements in deep learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in deep learning models, learn to operate TensorFlow to manage neural networks and interpret the results.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms.
There is booming demand for skilled deep learning engineers across a wide range of industries, making this deep learning course with TensorFlow training well-suited for professionals at the intermediate to advanced level of experience. We recommend this deep learning online course particularly for the following professionals:
1. Software engineers
2. Data scientists
3. Data analysts
4. Statisticians with an interest in deep learning
Deep Learning with Python: Getting started and getting from ideas to insights in minutes.
PyData Seattle 2015
Alex Korbonits (@korbonits)
This presentation was given July 25, 2015 at the PyData Seattle conference hosted by PyData and NumFocus.
Talk given at PYCON Stockholm 2015
Intro to Deep Learning + taking pretrained imagenet network, extracting features, and RBM on top = 97 Accuracy after 1 hour (!) of training (in top 10% of kaggle cat vs dog competition)
Deep learning: the future of recommendationsBalázs Hidasi
An informative talk about deep learning and its potential uses in recommender systems. Presented at the Budapest Startup Safary, 21 April, 2016.
The breakthroughs of the last decade in neural network research and the quick increasing of computational power resulted in the revival of deep neural networks and the field focusing on their training: deep learning. Deep learning methods have succeeded in complex tasks where other machine learning methods have failed, such as computer vision and natural language processing. Recently deep learning has began to gain ground in recommender systems as well. This talk introduces deep learning and its applications, with emphasis on how deep learning methods can solve long standing recommendation problems.
“Automatically learning multiple levels of representations of the underlying distribution of the data to be modelled”
Deep learning algorithms have shown superior learning and classification performance.
In areas such as transfer learning, speech and handwritten character recognition, face recognition among others.
(I have referred many articles and experimental results provided by Stanford University)
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...Simplilearn
This Deep Learning presentation will help you in understanding what is Deep Learning, why do we need Deep learning, what is neural network, applications of Deep Learning, what is perceptron, implementing logic gates using perceptron, types of neural networks. At the end of the video, you will get introduced to TensorFlow along with a usecase implementation on recognizing hand-written digits. Deep Learning is inspired by the integral function of the human brain specific to artificial neural networks. These networks, which represent the decision-making process of the brain, use complex algorithms that process data in a non-linear way, learning in an unsupervised manner to make choices based on the input. Deep Learning, on the other hand, uses advanced computing power and special type of neural networks and applies them to large amounts of data to learn, understand, and identify complicated patterns. W will also understand neural networks and how they work in this Deep Learning tutorial video. This Deep Learning tutorial is ideal for professionals with beginner to intermediate level of experience. Now, let us dive deep into this topic and understand what Deep Learning actually is.
Below topics are explained in this Deep Learning presentation:
1. What is Deep Learning?
2. Why do we need Deep Learning?
3. What is Neural network?
4. What is Perceptron?
5. Implementing logic gates using Perceptron
6. Types of Neural networks
7. Applications of Deep Learning
8. Working of Neural network
9. Introduction to TensorFlow
10. Use case implementation using TensorFlow
Simplilearn’s Deep Learning course will transform you into an expert in deep learning techniques using TensorFlow, the open-source software library designed to conduct machine learning & deep neural network research. With our deep learning course, you’ll master deep learning and TensorFlow concepts, learn to implement algorithms, build artificial neural networks and traverse layers of data abstraction to understand the power of data and prepare you for your new role as deep learning scientist.
Why Deep Learning?
It is one of the most popular software platforms used for deep learning and contains powerful tools to help you build and implement artificial neural networks.
Advancements in deep learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change.
There is booming demand for skilled deep learning engineers across a wide range of industries, making this deep learning course with TensorFlow training well-suited for professionals at the intermediate to advanced level of experience. We recommend this deep learning online course particularly for the following professionals:
1. Software engineers
2. Data scientists
3. Data analysts
4. Statisticians with an interest in deep learning
Zaikun Xu from the Università della Svizzera Italiana presented this deck at the 2016 Switzerland HPC Conference.
“In the past decade, deep learning as a life-changing technology, has gained a huge success on various tasks, including image recognition, speech recognition, machine translation, etc. Pio- neered by several research groups, Geoffrey Hinton (U Toronto), Yoshua Benjio (U Montreal), Yann LeCun(NYU), Juergen Schmiduhuber (IDSIA, Switzerland), Deep learning is a renaissance of neural network in the Big data era.
Neural network is a learning algorithm that consists of input layer, hidden layers and output layers, where each circle represents a neural and the each arrow connection associates with a weight. The way neural network learns is based on how different between the output of output layer and the ground truth, following by calculating the gradients of this discrepancy w.r.b to the weights and adjust the weight accordingly. Ideally, it will find weights that maps input X to target y with error as lower as possible.”
Watch the video presentation: http://insidehpc.com/2016/03/deep-learning/
See more talks in the Swiss Conference Video Gallery: http://insidehpc.com/2016-swiss-hpc-conference/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Best Deep Learning Post from LinkedIn Group
Datasets for Deep Learning (Slide share)
http://www.slideshare.net/pirahansiah/computer-vision-deep-learning-opencv
Deep Learning for Video Analysis – part 1 (DeepStream SDK NVIDIA TensorRT || NVIDIA GPU Inference Engine (GIE))
https://www.linkedin.com/lite/external-redirect?url=http%3A%2F%2Fwww%2Eslideshare%2Enet%2Fpirahansiah%2Fdeep-learning-for-video-analysis-part-1-deepstream-sdk-nvidia-tensorrt-nvidia-gpu-inference-engine-gie&urlHash=F2e8
How to install Digits 5.1 on Ubuntu 14
https://www.linkedin.com/redir/redirect?url=http%3A%2F%2Fwww%2Eslideshare%2Enet%2Fpirahansiah%2Fhow-to-install-digits-51-on-ubuntu-14&urlhash=8V-j&_t=tracking_anet
example of using deep learning in opencv 3.1
http://docs.opencv.org/3.1.0/d5/de7/tutorial_dnn_googlenet.html
Layers in Deep Learning & Caffe layers (model architecture )
https://www.linkedin.com/lite/external-redirect?url=http%3A%2F%2Fwww%2Eslideshare%2Enet%2Fpirahansiah%2Flayers-in-deep-learningcaffe-layers-model-architecture&urlHash=BsHf
Install, Compile, Setup, Setting OpenCV 3.2, Visual C++ 2015, Win 64bit,
https://www.linkedin.com/lite/external-redirect?url=http%3A%2F%2Fwww%2Eslideshare%2Enet%2Fpirahansiah%2Finstall-compile-setup-setting-opencv-32-visual-c-2015-win-64bit&urlHash=dC7R
Autonomous driving vehicles (Computer Vision, Deep Learning)
https://www.linkedin.com/pulse/autonomous-driving-vehicles-computer-vision-deep-farshid-pirahansiah-1?trk=mp-author-card
Compile OpenCV 3.2 with Visual Studio 2017 (C++) on Windows 10 x64 bit and test with TensorFlow
https://www.youtube.com/watch?v=mdeP8SdvSJw
TensorFlow in OpenCV 3.2 Visual Studio 2017 (C++ deep learning) application
https://www.youtube.com/watch?v=QK1kLTfS97c
Deep Learning for Natural Language ProcessingJonathan Mugan
Deep Learning represents a significant advance in artificial intelligence because it enables computers to represent concepts using vectors instead of symbols. Representing concepts using vectors is particularly useful in natural language processing, and this talk will elucidate those benefits and provide an understandable introduction to the technologies that make up deep learning. The talk will outline ways to get started in deep learning, and it will conclude with a discussion of the gaps that remain between our current technologies and true computer understanding.
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...Simplilearn
This Deep Learning Presentation will help you in understanding what is Deep learning, why do we need Deep learning, applications of Deep Learning along with a detailed explanation on Neural Networks and how these Neural Networks work. Deep learning is inspired by the integral function of the human brain specific to artificial neural networks. These networks, which represent the decision-making process of the brain, use complex algorithms that process data in a non-linear way, learning in an unsupervised manner to make choices based on the input. This Deep Learning tutorial is ideal for professionals with beginners to intermediate levels of experience. Now, let us dive deep into this topic and understand what Deep learning actually is.
Below topics are explained in this Deep Learning Presentation:
1. What is Deep Learning?
2. Why do we need Deep Learning?
3. Applications of Deep Learning
4. What is Neural Network?
5. Activation Functions
6. Working of Neural Network
Simplilearn’s Deep Learning course will transform you into an expert in deep learning techniques using TensorFlow, the open-source software library designed to conduct machine learning & deep neural network research. With our deep learning course, you’ll master deep learning and TensorFlow concepts, learn to implement algorithms, build artificial neural networks and traverse layers of data abstraction to understand the power of data and prepare you for your new role as deep learning scientist.
Why Deep Learning?
It is one of the most popular software platforms used for deep learning and contains powerful tools to help you build and implement artificial neural networks.
Advancements in deep learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in deep learning models, learn to operate TensorFlow to manage neural networks and interpret the results.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms.
There is booming demand for skilled deep learning engineers across a wide range of industries, making this deep learning course with TensorFlow training well-suited for professionals at the intermediate to advanced level of experience. We recommend this deep learning online course particularly for the following professionals:
1. Software engineers
2. Data scientists
3. Data analysts
4. Statisticians with an interest in deep learning
Deep Learning with Python: Getting started and getting from ideas to insights in minutes.
PyData Seattle 2015
Alex Korbonits (@korbonits)
This presentation was given July 25, 2015 at the PyData Seattle conference hosted by PyData and NumFocus.
Talk given at PYCON Stockholm 2015
Intro to Deep Learning + taking pretrained imagenet network, extracting features, and RBM on top = 97 Accuracy after 1 hour (!) of training (in top 10% of kaggle cat vs dog competition)
Deep learning: the future of recommendationsBalázs Hidasi
An informative talk about deep learning and its potential uses in recommender systems. Presented at the Budapest Startup Safary, 21 April, 2016.
The breakthroughs of the last decade in neural network research and the quick increasing of computational power resulted in the revival of deep neural networks and the field focusing on their training: deep learning. Deep learning methods have succeeded in complex tasks where other machine learning methods have failed, such as computer vision and natural language processing. Recently deep learning has began to gain ground in recommender systems as well. This talk introduces deep learning and its applications, with emphasis on how deep learning methods can solve long standing recommendation problems.
“Automatically learning multiple levels of representations of the underlying distribution of the data to be modelled”
Deep learning algorithms have shown superior learning and classification performance.
In areas such as transfer learning, speech and handwritten character recognition, face recognition among others.
(I have referred many articles and experimental results provided by Stanford University)
Deep Learning Tutorial | Deep Learning TensorFlow | Deep Learning With Neural...Simplilearn
This Deep Learning presentation will help you in understanding what is Deep Learning, why do we need Deep learning, what is neural network, applications of Deep Learning, what is perceptron, implementing logic gates using perceptron, types of neural networks. At the end of the video, you will get introduced to TensorFlow along with a usecase implementation on recognizing hand-written digits. Deep Learning is inspired by the integral function of the human brain specific to artificial neural networks. These networks, which represent the decision-making process of the brain, use complex algorithms that process data in a non-linear way, learning in an unsupervised manner to make choices based on the input. Deep Learning, on the other hand, uses advanced computing power and special type of neural networks and applies them to large amounts of data to learn, understand, and identify complicated patterns. W will also understand neural networks and how they work in this Deep Learning tutorial video. This Deep Learning tutorial is ideal for professionals with beginner to intermediate level of experience. Now, let us dive deep into this topic and understand what Deep Learning actually is.
Below topics are explained in this Deep Learning presentation:
1. What is Deep Learning?
2. Why do we need Deep Learning?
3. What is Neural network?
4. What is Perceptron?
5. Implementing logic gates using Perceptron
6. Types of Neural networks
7. Applications of Deep Learning
8. Working of Neural network
9. Introduction to TensorFlow
10. Use case implementation using TensorFlow
Simplilearn’s Deep Learning course will transform you into an expert in deep learning techniques using TensorFlow, the open-source software library designed to conduct machine learning & deep neural network research. With our deep learning course, you’ll master deep learning and TensorFlow concepts, learn to implement algorithms, build artificial neural networks and traverse layers of data abstraction to understand the power of data and prepare you for your new role as deep learning scientist.
Why Deep Learning?
It is one of the most popular software platforms used for deep learning and contains powerful tools to help you build and implement artificial neural networks.
Advancements in deep learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change.
There is booming demand for skilled deep learning engineers across a wide range of industries, making this deep learning course with TensorFlow training well-suited for professionals at the intermediate to advanced level of experience. We recommend this deep learning online course particularly for the following professionals:
1. Software engineers
2. Data scientists
3. Data analysts
4. Statisticians with an interest in deep learning
Zaikun Xu from the Università della Svizzera Italiana presented this deck at the 2016 Switzerland HPC Conference.
“In the past decade, deep learning as a life-changing technology, has gained a huge success on various tasks, including image recognition, speech recognition, machine translation, etc. Pio- neered by several research groups, Geoffrey Hinton (U Toronto), Yoshua Benjio (U Montreal), Yann LeCun(NYU), Juergen Schmiduhuber (IDSIA, Switzerland), Deep learning is a renaissance of neural network in the Big data era.
Neural network is a learning algorithm that consists of input layer, hidden layers and output layers, where each circle represents a neural and the each arrow connection associates with a weight. The way neural network learns is based on how different between the output of output layer and the ground truth, following by calculating the gradients of this discrepancy w.r.b to the weights and adjust the weight accordingly. Ideally, it will find weights that maps input X to target y with error as lower as possible.”
Watch the video presentation: http://insidehpc.com/2016/03/deep-learning/
See more talks in the Swiss Conference Video Gallery: http://insidehpc.com/2016-swiss-hpc-conference/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Best Deep Learning Post from LinkedIn Group
Datasets for Deep Learning (Slide share)
http://www.slideshare.net/pirahansiah/computer-vision-deep-learning-opencv
Deep Learning for Video Analysis – part 1 (DeepStream SDK NVIDIA TensorRT || NVIDIA GPU Inference Engine (GIE))
https://www.linkedin.com/lite/external-redirect?url=http%3A%2F%2Fwww%2Eslideshare%2Enet%2Fpirahansiah%2Fdeep-learning-for-video-analysis-part-1-deepstream-sdk-nvidia-tensorrt-nvidia-gpu-inference-engine-gie&urlHash=F2e8
How to install Digits 5.1 on Ubuntu 14
https://www.linkedin.com/redir/redirect?url=http%3A%2F%2Fwww%2Eslideshare%2Enet%2Fpirahansiah%2Fhow-to-install-digits-51-on-ubuntu-14&urlhash=8V-j&_t=tracking_anet
example of using deep learning in opencv 3.1
http://docs.opencv.org/3.1.0/d5/de7/tutorial_dnn_googlenet.html
Layers in Deep Learning & Caffe layers (model architecture )
https://www.linkedin.com/lite/external-redirect?url=http%3A%2F%2Fwww%2Eslideshare%2Enet%2Fpirahansiah%2Flayers-in-deep-learningcaffe-layers-model-architecture&urlHash=BsHf
Install, Compile, Setup, Setting OpenCV 3.2, Visual C++ 2015, Win 64bit,
https://www.linkedin.com/lite/external-redirect?url=http%3A%2F%2Fwww%2Eslideshare%2Enet%2Fpirahansiah%2Finstall-compile-setup-setting-opencv-32-visual-c-2015-win-64bit&urlHash=dC7R
Autonomous driving vehicles (Computer Vision, Deep Learning)
https://www.linkedin.com/pulse/autonomous-driving-vehicles-computer-vision-deep-farshid-pirahansiah-1?trk=mp-author-card
Compile OpenCV 3.2 with Visual Studio 2017 (C++) on Windows 10 x64 bit and test with TensorFlow
https://www.youtube.com/watch?v=mdeP8SdvSJw
TensorFlow in OpenCV 3.2 Visual Studio 2017 (C++ deep learning) application
https://www.youtube.com/watch?v=QK1kLTfS97c
Deep Learning - The Past, Present and Future of Artificial IntelligenceLukas Masuch
In the last couple of years, deep learning techniques have transformed the world of artificial intelligence. One by one, the abilities and techniques that humans once imagined were uniquely our own have begun to fall to the onslaught of ever more powerful machines. Deep neural networks are now better than humans at tasks such as face recognition and object recognition. They’ve mastered the ancient game of Go and thrashed the best human players. “The pace of progress in artificial general intelligence is incredible fast” (Elon Musk – CEO Tesla & SpaceX) leading to an AI that “would be either the best or the worst thing ever to happen to humanity” (Stephen Hawking – Physicist).
What sparked this new hype? How is Deep Learning different from previous approaches? Let’s look behind the curtain and unravel the reality. This talk will introduce the core concept of deep learning, explore why Sundar Pichai (CEO Google) recently announced that “machine learning is a core transformative way by which Google is rethinking everything they are doing” and explain why “deep learning is probably one of the most exciting things that is happening in the computer industry“ (Jen-Hsun Huang – CEO NVIDIA).
Transform your Business with AI, Deep Learning and Machine LearningSri Ambati
Video: https://www.youtube.com/watch?v=R3IXd1iwqjc
Meetup: http://www.meetup.com/SF-Bay-ACM/events/231709894/
In this talk, Arno Candel presents a brief history of AI and how Deep Learning and Machine Learning techniques are transforming our everyday lives. Arno will introduce H2O, a scalable open-source machine learning platform, and show live demos on how to train sophisticated machine learning models on large distributed datasets. He will show how data scientists and application developers can use the Flow GUI, R, Python, Java, Scala, JavaScript and JSON to build smarter applications, and how to take them to production. He will present customer use cases from verticals including insurance, fraud, churn, fintech, and marketing.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Some resources how to navigate in the hardware space in order to build your own workstation for training deep learning models.
Alternative download link: https://www.dropbox.com/s/o7cwla30xtf9r74/deepLearning_buildComputer.pdf?dl=0
Suggestions:
1) For best quality, download the PDF before viewing.
2) Open at least two windows: One for the Youtube video, one for the screencast (link below), and optionally one for the slides themselves.
3) The Youtube video is shown on the first page of the slide deck, for slides, just skip to page 2.
Screencast: http://youtu.be/VoL7JKJmr2I
Video recording: http://youtu.be/CJRvb8zxRdE (Thanks to Al Friedrich!)
In this talk, we take Deep Learning to task with real world data puzzles to solve.
Data:
- Higgs binary classification dataset (10M rows, 29 cols)
- MNIST 10-class dataset
- Weather categorical dataset
- eBay text classification dataset (8500 cols, 500k rows, 467 classes)
- ECG heartbeat anomaly detection
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Scalable Data Science and Deep Learning with H2O
In this session, we introduce the H2O data science platform. We will explain its scalable in-memory architecture and design principles and focus on the implementation of distributed deep learning in H2O. Advanced features such as adaptive learning rates, various forms of regularization, automatic data transformations, checkpointing, grid-search, cross-validation and auto-tuning turn multi-layer neural networks of the past into powerful, easy-to-use predictive analytics tools accessible to everyone. We will present a broad range of use cases and live demos that include world-record deep learning models, anomaly detection tools and approaches for Kaggle data science competitions. We also demonstrate the applicability of H2O in enterprise environments for real-world customer production use cases.
By the end of the hands-on-session, attendees will have learned to perform end-to-end data science workflows with H2O using both the easy-to-use web interface and the flexible R interface. We will cover data ingest, basic feature engineering, feature selection, hyperparameter optimization with N-fold cross-validation, multi-model scoring and taking models into production. We will train supervised and unsupervised methods on realistic datasets. With best-of-breed machine learning algorithms such as elastic net, random forest, gradient boosting and deep learning, you will be able to create your own smart applications.
A local installation of RStudio is recommended for this session.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
Distributed Deep Learning At Scale On Apache Spark With BigDLYulia Tell
Intel recently released BigDL, an open source distributed deep Learning framework for Apache Spark (https://github.com/intel-analytics/BigDL). It brings native support for deep learning functionalities to Spark, provides orders of magnitude speedup than out-of-box open source DL frameworks (e.g., Caffe/Torch/TensorFlow) with respect to single node Xeon performance, and efficiently scales out deep learning workloads based on the Spark architecture. In addition, it also allows data scientists to perform distributed deep learning analysis on big data using the familiar tools including python, notebook, etc.
In this talk, we will give an introduction to BigDL, show how Big Data users and data scientist can leverage BigDL for their deep learning (such as image recognition, object detection, NLP, etc.) analysis on large amounts of data in a distributed fashion, which allows them to use their Big Data (e.g., Apache Hadoop and Spark) cluster as the unified data analytics platform for data storage, data processing and mining, feature engineering, traditional (non-deep) machine learning, and deep learning workloads.
Deep Qualia: Philosophy of Statistics, Deep Learning, and Blockchain
Deep learning: What is it, why is it important, and what do I need to know?
The aim of this talk is to discuss deep learning as an advanced computational method and its philosophical implications. Computing is a fundamental model by which we are understanding more about ourselves and the world. We think that reality is composed of patterns, which can be detected by machine learning methods.
Deep learning is a complexity optimization technique in which algorithms learn from data by modeling high-level abstractions and assigning probabilities to nodes as they characterize the system and make predictions. An important challenge in deep learning is that these methods work in certain domains (image, speech, and text recognition), but we do not have a good explanation for why, which impedes a wider application of these solutions.
Another recent advance in computational methods is blockchain technology which allows the secure transfer of assets and information, and the automated coordination of operations via a trackable remunerative ledger and smart contracts (automatically-executing Internet-based programs).
This talk looks at how deep learning technology, particularly as coupled with blockchain systems, might be used to produce a new kind of global computing platform. The goal is for blockchain deep learning systems to address higher-dimensional computing challenges that require learning and dynamic response in domains such as economics and financial risk, epidemiology, social modeling, public health (cancer, aging), dark matter, atomic reactions, network-modeling (transportation, energy, smart cities), artificial intelligence, and consciousness.
Yinyin Liu presents at SD Robotics Meetup on November 8th, 2016. Deep learning has made great success in image understanding, speech, text recognition and natural language processing. Deep Learning also has tremendous potential to tackle the challenges in robotic vision, and sensorimotor learning in a robotic learning environment. In this talk, we will talk about how current and future deep learning technologies can be applied for robotic applications.
최근 IT업계에서는 딥러닝에 대한 큰 활약이 이슈화 되고 있습니다.
이와 같은 트렌드에 따라가고 싶지만 선형대수 등의 수학적 배경지식 습득부터 시작하여, 딥러닝의 원리와 주요 개념들을 이해 후에 실험을 시도하기에는 많은 시간과 노력이 필요합니다.
그러나 기존의 유용한 딥러닝 오픈소스를 활용한다면 어렵지 않게 딥러닝을 맛볼 수 있습니다.
본 발표는 수학적인 설명을 최대한 배제하고, 오픈소스 툴인 theano, pylearn2를 활용한 예제에 대해 설명하려고 합니다. 추가로 필요할 코드들도 소개하려고 합니다.
그리고 word2vec 를 활용하여, 자연어 처리에 딥러닝을 적용하는 사례를 다루려고 합니다.
주제가 학문적이고 이론적이기 때문에 발표가 부담되지만, 최대한 개념적으로 설명하여 실험을 쉽게 따라 할 수 있도록 돕고자 합니다.
오픈소스 툴들의 문서화가 잘 되어있지만, 저 또한 처음 접했을 때는 어려움이 있었기 때문에 딥러닝을 시작해보려는 분들에게 도움이 될 듯합니다.
컴퓨터가 딥러닝하는 동안 틈틈이 이론공부 하시면 좋겠네요.
From Natural Language Processing to Artificial IntelligenceJonathan Mugan
Overview of natural language processing (NLP) from both symbolic and deep learning perspectives. Covers tf-idf, sentiment analysis, LDA, WordNet, FrameNet, word2vec, and recurrent neural networks (RNNs).
Understanding Deep Learning & Parameter Tuning with MXnet, H2o Package in RManish Saraswat
Simple guide which explains deep learning and neural network with hands on experience in R using MXnet and H2o package. It also explains gradient descent and backpropagation algorithm.
Complete tutorial: http://blog.hackerearth.com/understanding-deep-learning-parameter-tuning-with-mxnet-h2o-package-r
A Threshold Logic Unit (TLU) is a mathematical function conceived as a crude model, or abstraction of biological neurons. Threshold logic units are the constitutive units in an artificial neural network. In this paper a positive clock-edge triggered T flip-flop is designed using Perceptron Learning Algorithm, which is a basic design algorithm of threshold logic units. Then this T flip-flop is used to design a two-bit up-counter that goes through the states 0, 1, 2, 3, 0, 1… Ultimately, the goal is to show how to design simple logic units based on threshold logic based perceptron concepts.
With massive amounts of computational power, machines can now recognize objects and translate speech in real time. Thanks to Deep Learning, Artificial Intelligence is now getting smart. Deep Learning models attempt to mimic the activity of the neocortex. It is understood that the activity of these layers of neurons is what constitutes a brain to be able to "think". These models learn to recognize patterns in digital representations of data in a very similar sense to humans. In this survey report, we introduce the most important concepts of Deep Learning along with the state of the art models that are now widely adopted in commercial products.
a paper review. This presentation introduces Abductive Commonsense Reasoning which is the published paper in ICLR 2020. In this paper, the authors use commonsense to generate plausible hypotheses. They generate new data set 'ART' and propose new models for 'aNLI', 'aNLG' using BERT, and GPT.
Implementation Of Back-Propagation Neural Network For Isolated Bangla Speech ...ijistjournal
This paper is concerned with the development of Back-propagation Neural Network for Bangla Speech Recognition. In this paper, ten bangla digits were recorded from ten speakers and have been recognized. The features of these speech digits were extracted by the method of Mel Frequency Cepstral Coefficient (MFCC) analysis. The mfcc features of five speakers were used to train the network with Back propagation algorithm. The mfcc features of ten bangla digit speeches, from 0 to 9, of another five speakers were used to test the system. All the methods and algorithms used in this research were implemented using the features of Turbo C and C++ languages. From our investigation it is seen that the developed system can successfully encode and analyze the mfcc features of the speech signal to recognition. The developed system achieved recognition rate about 96.332% for known speakers (i.e., speaker dependent) and 92% for unknown speakers (i.e., speaker independent).
Implementation Of Back-Propagation Neural Network For Isolated Bangla Speech ...ijistjournal
This paper is concerned with the development of Back-propagation Neural Network for Bangla Speech Recognition. In this paper, ten bangla digits were recorded from ten speakers and have been recognized.The features of these speech digits were extracted by the method of Mel Frequency Cepstral Coefficient(MFCC) analysis. The mfcc features of five speakers were used to train the network with Back propagation algorithm. The mfcc features of ten bangla digit speeches, from 0 to 9, of another five speakers were used to test the system. All the methods and algorithms used in this research were implemented using the features of Turbo C and
C++ languages. From our investigation it is seen that the developed system can successfully encode and analyze the mfcc features of the speech signal to recognition. The developed system achieved recognition rate about 96.332% for known speakers (i.e., speaker dependent) and 92% for unknown speakers (i.e., speaker independent).
This is an introduction to deep learning presented to Plymouth University students. In the introduction it is explained how a neural network works. In the practical section it is shown how to use Tensorflow for building simple models. Finally the case studies, how to use deep learning in real world applications.
(date is wrong, sorry, it was January 28th, 2023)
So many intelligent robots have come and gone, failing to become a commercial success. We’ve lost Aibo, Romo, Jibo, Baxter, Scout, and even Alexa is reducing staff. I posit that they failed because you can’t talk to them, not really. AI has recently made substantial progress, speech recognition now actually works, and we have neural networks such as ChatGPT that produce astounding natural language. But you can’t just throw a neural network into a robot because there isn’t anybody home in those networks—they are just purposeless mimicry agents with no understanding of what they are saying. This lack of understanding means that they can’t be relied upon because the mistakes they make are ones that no human would make, not even a child. Like a child first learning about the world, a robot needs a mental model of the current real-world situation and must use that model to understand what you say and generate a meaningful response. This model must be composable to represent the infinite possibilities of our world, and to keep up in a conversation, it must be built on the fly using human-like, immediate learning. This talk will cover what is required to build that mental model so that robots can begin to understand the world as well as human children do. Intelligent robots won’t be a commercial success based on their form—they aren’t as cute and cuddly as cats and dogs—but robots can be much more if we can talk to them.
Moving Your Machine Learning Models to Production with TensorFlow ExtendedJonathan Mugan
ML is great fun, but now we want it to solve real problems. To do this, we need a way of keeping track of all of our data and models, and we need to know when our models fail and why. This talk will cover how to move ML to production with TensorFlow Extended (TFX). TFX is used by Google internally for machine-learning model development and deployment, and it has recently been made public. TFX consists of multiple pipeline elements and associated components, and this talk will cover them all, but three elements are particularly interesting: TensorFlow Data Validation, TensorFlow Model Analysis, and the What-If Tool.
The TensorFlow Data Validation library analyses incoming data and computes distributions over the feature values. This can show us which features many not be useful, maybe because they always have the same value, or which features may contain bugs. TensorFlow Model Analysis allows us to understand how well our data performs on different slices of the data. For example, we may find that our predictive models are more accurate for events that happen on Tuesdays, and such knowledge can be used to help us better understand our data and our business. The What-If Tool is as an interactive tool that allows you to change data and see what the model would say if a particular record had a particular feature value. It lets you probe your model, and it can automatically find the closest record with a different predicted label, which allows you to learn what the model is homing in on. Machine learning is growing up.
Generating Natural-Language Text with Neural NetworksJonathan Mugan
Automatic text generation enables computers to summarize text, to have conversations in customer-service and other settings, and to customize content based on the characteristics and goals of the human interlocutor. Using neural networks to automatically generate text is appealing because they can be trained through examples with no need to manually specify what should be said when. In this talk, we will provide an overview of the existing algorithms used in neural text generation, such as sequence2sequence models, reinforcement learning, variational methods, and generative adversarial networks. We will also discuss existing work that specifies how the content of generated text can be determined by manipulating a latent code. The talk will conclude with a discussion of current challenges and shortcomings of neural text generation.
There are lots of frameworks for building chatbots, but those abstractions can obscure understanding and hinder application development. In this talk, we will cover building chatbots from the ground up in Python. This can be done with either classic NLP or deep learning. We will cover both approaches, but this talk will focus on how one can build a chatbot using spaCy, pattern matching, and context-free grammars.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf
What Deep Learning Means for Artificial Intelligence
1. What Deep Learning Means
for Artificial Intelligence
Jonathan Mugan
Austin Data Geeks
March 11, 2015
@jmugan, www.jonathanmugan.com
2. AI through the lens of System 1 and System 2
Psychologist Daniel Kahneman in Thinking Fast and Slow describes
humans as having two modes of thought: System 1 and System 2.
System 1: Fast and Parallel System 2: Slow and Serial
AI systems in these domains are useful
but limited. Called GOFAI (Good, Old-
Fashioned Artificial Intelligence).
1. Search and planning
2. Logic
3. Rule-based systems
AI systems in these domains have
been lacking.
1. Serial computers too slow
2. Lack of training data
3. Didn’t have the right algorithms
Subconscious: E.g., face recognition or
speech understanding.
Conscious: E.g., when listening to a
conversation or making PowerPoint slides.
We underestimated how hard it would
be to implement. E.g., we thought
computer vision would be easy.
We assumed it was the most difficult.
E.g., we thought chess was hard.
3. AI through the lens of System 1 and System 2
Psychologist Daniel Kahneman in Thinking Fast and Slow describes
humans as having two modes of thought: System 1 and System 2.
System 1: Fast and Parallel System 2: Slow and Serial
AI systems in these domains are useful
but limited. Called GOFAI (Good, Old-
Fashioned Artificial Intelligence).
1. Search and planning
2. Logic
3. Rule-based systems
Subconscious: E.g., face recognition or
speech understanding.
Conscious: E.g., when listening to a
conversation or making PowerPoint slides.
We underestimated how hard it would
be to implement. E.g., we thought
computer vision would be easy.
We assumed it was the most difficult.
E.g., we thought chess was hard.
This has changed
1. We now have GPUs and
distributed computing
2. We have Big Data
3. We have new algorithms [Bengio et
al., 2003; Hinton et al., 2006; Ranzato et
al., 2006]
4. Deep learning begins with a little function
sum 𝑥 =
𝑖=1
𝑛
𝑤𝑖 𝑥𝑖 = 𝑤 𝑇
𝑥
𝑤𝑒𝑖𝑔ℎ𝑡1 × 𝑖𝑛𝑝𝑢𝑡1
𝑤𝑒𝑖𝑔ℎ𝑡2 × 𝑖𝑛𝑝𝑢𝑡2
𝑤𝑒𝑖𝑔ℎ𝑡3 × 𝑖𝑛𝑝𝑢𝑡3+
𝑠𝑢𝑚
It all starts with a humble linear function called a perceptron.
Perceptron:
If 𝑠𝑢𝑚 > 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑: output 1
Else: output 0
In math, with 𝑥 being an input
vector and 𝑤 being a weight
vector.
Example: The inputs can be your data. Question: Should I buy this car?
0.2 × 𝑔𝑎𝑠 𝑚𝑖𝑙𝑎𝑔𝑒
0.3 × ℎ𝑜𝑟𝑠𝑒 𝑝𝑜𝑤𝑒𝑟
0.5 × 𝑛𝑢𝑚 𝑐𝑢𝑝 ℎ𝑜𝑙𝑑𝑒𝑟𝑠+
𝑠𝑢𝑚
If 𝑠𝑢𝑚 > 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑: buy car
Else: walk
5. These little functions are chained together
Deep learning comes from chaining a bunch of these little
functions together. Chained together, they are called
neurons.
𝜎 𝑠𝑢𝑚 𝑥 + 𝑏 where 𝜎 𝑥 =
1
1+
1
𝑒
To create a neuron, we add a nonlinearity to the perceptron to get
extra representational power when we chain them together.
Our nonlinear perceptron is
sometimes called a sigmoid.
The value 𝑏 just offsets the sigmoid so the
center is at 0.
Plot of a sigmoid
7. Three-layered neural network
A bunch of neurons chained together is called a neural network.
Layer 2: hidden layer. Called
this because it is neither input
nor output.
[16.2, 17.3, −52.3, 11.1]
Layer 3: output. E.g., cat
or not a cat; buy the car or
walk.
Layer 1: input data. Can
be pixel values or the number
of cup holders.
This network has three layers.
(Some edges lighter
to avoid clutter.)
8. Training with supervised learning
Supervised Learning: You show the network a bunch of things
with a labels saying what they are, and you want the network to
learn to classify future things without labels.
Example: here are some
pictures of cats. Tell me
which of these other pictures
are of cats.
To train the network, want to
find the weights that
correctly classify all of the
training examples. You hope
it will work on the testing
examples.
Done with an algorithm
called Backpropagation
[Rumelhart et al., 1986].
[16.2, 17.3, −52.3, 11.1]
9. Deep learning is adding more layers
There is no exact
definition of
what constitutes
“deep learning.”
The number of
weights (parameters)
is generally large.
Some networks have
millions of parameters
that are learned.
[16.2, 17.3, −52.3, 11.1]
(Some edges omitted
to avoid clutter.)
10. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
11. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
12. Recall our standard architecture
Layer 2: hidden layer. Called
this because it is neither input
nor output.
Layer 3: output. E.g., cat
or not a cat; buy the car or
walk.
Layer 1: input data. Can
be pixel values or the number
of cup holders.
[16.2, 17.3, −52.3, 11.1]
Is this a cat?
13. Neural nets with multiple outputs
Okay, but what kind of cat is it?
𝑃(𝑥)
[16.2, 17.3, −52.3, 11.1]
𝑃 𝑜𝑖 =
𝑒 𝑠𝑢𝑚(𝑥 𝑖)+𝑏 𝑖
𝑗 𝑒 𝑠𝑢𝑚 𝑥 𝑗 +𝑏 𝑗
𝑃(𝑥)𝑃(𝑥) 𝑃(𝑥) 𝑃(𝑥)
Introduce a new node
called a softmax.
Probability a
house cat
Probability a
lion
Probability a
panther
Probability a
bobcat
Where 𝑗 varies over all the
other nodes at that layer.
Just normalize the output 𝑜𝑖
over the sum of the other
outputs.
14. Learning word vectors
13.2, 5.4, −3.1 [−12.1, 13.1, 0.1] [7.2, 3.2,-1.9]
the man ran
From the sentence, “The man ran fast.”
𝑃(𝑥)𝑃(𝑥) 𝑃(𝑥) 𝑃(𝑥)
Probability
of “fast”
Probability
of “slow”
Probability
of “taco”
Probability
of “bobcat”
Learns a vector for each word based on the “meaning” in the sentence by
trying to predict the next word [Bengio et al., 2003].
Computationally
expensive because you
need a softmax
node for each word in
the vocabulary.
Recent work models the top
layer using a binary tree
[Mikolov et al., 2013].
These numbers updated
along with the weights
and become the vector
representations of the
words.
15. Comparing vector and symbolic representations
Vector representation
taco = [17.32, 82.9, −4.6, 7.2]
Symbolic representation
taco = 𝑡𝑎𝑐𝑜
• Vectors have a similarity score.
• A taco is not a burrito but similar.
• Symbols can be the same or not.
• A taco is just as different from a
burrito as a Toyota.
• Vectors have internal structure
[Mikolov et al., 2013].
• Italy – Rome = France – Paris
• King – Queen = Man – Woman
• Symbols have no structure.
• Symbols are arbitrarily assigned.
• Meaning relative to other symbols.
• Vectors are grounded in
experience.
• Meaning relative to predictions.
• Ability to learn representations
makes agents less brittle.
16. Learning vectors of longer text
[33.1, 7.5, 11.2] 13.2, 5.4, −3.1 [−12.1, 13.1, 0.1] [7.2, 3.2,-1.9]
<current paragraph> the man ran
From the sentence, “The man ran fast.”
𝑃(𝑥)𝑃(𝑥) 𝑃(𝑥) 𝑃(𝑥)
Probability
of “fast”
Probability
of “slow”
Probability
of “taco”
Probability
of “bobcat”
The “meaning” of a
paragraph is encoded in
a vector that allows you
to predict the next word
in the paragraph using
already learned word
vectors [Le and Mikolov,
2014].
17. Learning to parse
The dog went to the store.
NP NP
PP
VP
PPGiven the semantic
representation of
words, we can do
structure learning.
The system can
learn to group
things that go
together.
18. Learning to parse
[1.1,1.3] [8.1, 2.1] [2.6, 8.4] [2.0, 7.1] [8.9, 2.4] [7.1, 2.2]
[1.1, 1.3] [1.1, 1.3]
[7.3, 9.8]
[3.2, 8.8]
[9.1, 0.2]Given the semantic
representation of
words we can do
structure learning.
Socher et al. [2010]
learned word
vectors and then
used the Penn Tree
Bank to train a
parser that
encodes semantic
meaning.
The dog went to the store.
19. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
20. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
21. Vision is hard
• Even harder for 3D
objects.
• You move a bit, and
everything changes.
72 ⋯ 91
⋮ ⋱ ⋮
16 ⋯ 40
How a computer sees an image.
Example from MNIST
handwritten digit dataset
[LeCun and Cortes, 1998].
Vision is hard because images are big matrices of numbers.
22. Breakthrough: Unsupervised Model
• Big breakthrough in 2006 by Hinton et al.
• Use a network with symmetric weights called a restricted Boltzmann machine.
• Stochastic binary neuron.
• Probabilistically outputs 0
(turns off) or 1 (turns on)
based on the weight of the
inputs from on units.
0 1
𝑃 𝑠𝑖 = 1 =
1
1 +
1
𝑒 𝑠𝑢𝑚(𝑠 𝑖)
• Limit connections to be from one
layer to the next.
• Fast because decisions are made
locally.
• Trained in an unsupervised way
to reproduce the data.
0 1 0 1 0 1
0 1 0 1 0 1
23. Stack up the layers to make a deep network
Input data
Hidden
The output of each layer
becomes the input to the
next layer [Hinton et al.,
2006].
See video starting at
second 45
https://www.coursera.org
/course/neuralnets
0 1 0 1 0 1
0 1 0 1 0 1
Input data
Hidden
0 1 0 1 0 1
0 1 0 1 0 1
Hidden layer becomes
input data of next layer.
24. Computer vision, scaling up
Unsupervised learning was scaled up
by Honglak Lee et al. [2009] to learn
high-level visual features.
[Lee et al., 2009]
Further scaled up by Quoc Le et al.
[2012].
• Used 1,000 machines (16,000
cores) running for 3 days to train 1
billion weights by watching
YouTube videos.
• The network learned to identify
cats.
• The network wasn’t told to look
for cats, it naturally learned that
cats were integral to online
viewing.
• Video on the topic at NYT
http://www.nytimes.com/2012/06/26/tec
hnology/in-a-big-network-of-computers-
evidence-of-machine-learning.html
25. Why is this significant?
To have a grounded understanding
of its environment, an agent must
be able to acquire representations
through experience [Pierce et al.,
1997; Mugan et al., 2012].
Without a grounded understanding,
the agent is limited to what was
programmed in.
We saw that unsupervised learning
could be used to learn the meanings of words, grounded in the
experience of reading.
Using these deep Boltzmann machines, machines can learn
to see the world through experience.
26. Limit connections and duplicate parameters
Convolutional neural networks build in
a kind of feature invariance. They take
advantage the layout of the pixels.
[16.2, 17.3, −52.3, 11.1]
With the layers and
topology, our networks are
starting to look a little like
the visual cortex. Although,
we still don’t fully
understand the visual
cortex.
Different areas
of the image go
to different
parts of the
neural network,
and weights are
shared.
27. Recent deep vision networks
ImageNet http://www.image-net.org/ is a huge collection of images
corresponding to the nouns of the WordNet hierarchy. There are hundreds to
thousands of images per noun.
2012 – Deep Learning begins to dominate image recognition
Krizhevsky et al. [2012] got 16% error on recognizing objects, when
before the best error was 26%. They used a convolutional neural
network.
2015 – Deep Learning surpasses human level performance
He et al. [2015] surpassed human level performance on recognizing
images of objects.* Computers seem to have an advantage when the
classes of objects are fine grained, such as multiple species of dogs.
Closing note on computer vision: Hinton points out that modern networks can just
work with top down (supervised learning) if the network is small enough relative to
the amount of the training data; but the goal of AI is broad-based understanding,
and there will likely never be enough labeled training data for general intelligence.
*But deep learning can be easily fooled [Nguyen et al., 2014]. Enlightening video at https://www.youtube.com/watch?v=M2IebCN9Ht4.
28. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
29. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
30. A stamping in of behavior
When we think of doing things, we think of conscious planning
with System 2.
Imagine trying to get to Seattle.
• Get to the airport. How? Take a taxi. How? Call a taxi. How?
Find my phone.
• Some behaviors arise more from a
a gradual stamping in [Thorndike,
1898].
• Became the study of Behaviorism
[Skinner, 1953] (see Skinner box
on the right).
• Formulated into artificial
intelligence as Reinforcement
Learning [Sutton and Barto, 1998]. By Andreas1 (Adapted from Image:Boite skinner.jpg) [GFDL
(http://www.gnu.org/copyleft/fdl.html) or CC-BY-SA-3.0
(http://creativecommons.org/licenses/by-sa/3.0/)], via
Wikimedia Commons
A “Skinner box”
31. Beginning with random exploration
In reinforcement learning, the agent begins by
randomly exploring until it reaches its goal.
32. • When it reaches the goal, credit is propagated back to its previous
states.
• The agent learns the function 𝑄 𝜋(𝑠, 𝑎), which gives the cumulative
expected discounted reward of being in state 𝑠 and taking action
𝑎 and acting according to policy 𝜋 thereafter.
Reaching the goal
33. Eventually, the agent learns the value of being in each state and
taking each action and can therefore always do the best thing in
each state.
Learning the behavior
34. Playing Atari with Deep Learning
Input, last four frames, where each frame
is downsampled to 84 by 84 pixels.
[Mnih et al., 2013]
represent the
state-action value
function 𝑄 𝑠, 𝑎 as
a convolutional
neural network.
𝑃(𝑥)𝑃(𝑥) 𝑃(𝑥) 𝑃(𝑥)
Value of
moving left
Value of
moving right
Value of
shooting
Value of
reloading
In [Mnih et al., 2013],
this is actually three
hidden layers.
See some videos at
http://mashable.com/2015/02
/25/computer-wins-at-atari-
games/
35. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
36. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
37. We must go deeper for a robust System 1
Imagine a dude standing
on a table. How would a
computer know that if
you move the table you
also move the dude?
Likewise, how could a
computer know that it
only rains outside?
Or, as Marvin Minsky asks, how could a computer learn
that you can pull a box with a string but not push it?
38. We must go deeper for a robust System 1
Imagine a dude standing
on a table. How would a
computer know that if
you move the table you
also move the dude?
Likewise, how could a
computer know that it
only rains outside?
Or, as Marvin Minsky asks, how could a computer learn
that you can pull a box with a string but not push it?
You couldn’t possibly explain
all of these situations to a
computer. There’s just too
many variations.
A robot can learn through
experience, but it must be
able to efficiently generalize
that experience.
39. Abstract thinking through image schemas
Humans efficiently generalize experience using abstractions called
image schemas [Johnson, 1987]. Image schemas map experience
to conceptual structure.
Developmental psychologist Jean Mandler argues that some
image schemas are formed before children begin to talk, and that
language is eventually built onto this base set of schemas [2004].
• Consider what it means for an object to contain another object,
such as for a box to contain a ball.
• The container constrains the movement of the object inside.
If the container is moved, the contained object moves.
• These constraints are represented by the container image
schema.
• Other image schemas from Mark Johnson’s book, The Body in the Mind:
path, counterforce, restraint, removal, enablement, attraction, link,
cycle, near-far, scale, part-whole, full-empty, matching, surface, object,
and collection.
40. Abstract thinking through metaphors
Love is a journey. Love is war.
Lakoff and Johnson [1980] argue that we understand abstract
concepts through metaphors to physical experience.
Neural networks will likely require advances in both architecture
and size to reach this level of abstraction.
For example, a container can be more than
a way of understanding physical
constraints, it can be a metaphor used to
understand the abstract concept of what
an argument is. You could say that
someone’s argument doesn’t hold water, or
you could say that it is empty, or you could
say that the argument has holes in it.
41. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
42. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
43. Some code to get you started
Google Word2Vec: they looked at (3 billion documents) and
created 300 long vectors.
https://code.google.com/p/word2vec/
Gensim has an implementation of the learning algorithm in
Python.
http://radimrehurek.com/gensim/models/word2vec.html
Theano is a general-purpose deep learning implementation with
great documentation and tutorials.
http://deeplearning.net/software/theano/
44. Best learning resources
Best place to start. Hinton’s Coursera Course. Get it right from
the horse’s mouth. He explains things well.
https://www.coursera.org/course/neuralnets
Online textbook in preparation for deep learning from Yoshua
Bengio and friends. Clear and understandable.
http://www.iro.umontreal.ca/~bengioy/dlbook/
Introduction to programming deep learning with Python and
Theano. It's clear, detailed, and entertaining. 1-hour talk.
http://www.rosebt.com/blog/introduction-to-deep-learning-
with-python
45. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
46. Talk Outline
• Introduction
• Deep learning and natural language processing
• Deep learning and computer vision
• Deep learning and robot actions
• What deep learning still can’t do
• Practical ways you can get started
• Conclusion
• References
47. What deep learning means for artificial intelligence
Deep learning can allow a robot to autonomously
learn a representation through experience with the
world.
When its representation is grounded in experience,
a robot can be autonomous without having to rely
on the intentionality of the human designer.
If we can continue to scale up deep learning to
represent ever higher levels of abstraction, our
robots may view the world in an alien way, but
they will be independently intelligent.
48. References
1. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. A neural probabilistic
language model. The Journal of Machine Learning Research, 3:1137–1155, 2003.
2. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers:
Surpassing human-level performance on imagenet classification. arXiv preprint
arXiv:1502.01852, 2015.
3. Geoffrey Hinton, Simon Osindero, and Yee-Whye Teh. A fast learning algorithm for deep
belief nets. Neural computation, 18(7):1527–1554, 2006.
4. M. Johnson. The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason.
University of Chicago Press, Chicago, Illinois, USA, 1987.
5. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep
convolutional neural networks. In Advances in neural information processing systems, pages
1097–1105, 2012.
6. G. Lakoff and M. Johnson. Metaphors We Live By. University of Chicago Press, Chicago, 1980.
7. Quoc Le and Tomas Mikolov. Distributed representations of sentences and documents. In
Proceedings of the 31st International Conference on Machine Learning (ICML-14), pages
1188–1196, 2014.
8. Q.V. Le, M.A. Ranzato, R. Monga, M. Devin, K. Chen, G.S. Corrado, J. Dean, and A. Ng. Building
high-level features using large scale unsupervised learning. In International Conference on
Machine Learning (ICML), 2012.
9. Yann LeCun and Corinna Cortes. The mnist database of handwritten digits, 1998.
49. References (continued)
10. Honglak Lee, Roger Grosse, Rajesh Ranganath, and Andrew Y Ng. Convolutional deep belief
networks for scalable unsupervised learning of hierarchical representations. In Proceedings
of the 26th Annual International Conference on Machine Learning, pages 609–616. ACM,
2009.
11. J. Mandler. The Foundations of Mind, Origins of Conceptual Thought. Oxford University
Press, New York, New York, USA, 2004.
12. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed
representations of words and phrases and their compositionality. In Advances in Neural
Information Processing Systems, pages 3111–3119, 2013.
13. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan
Wierstra, and Martin Riedmiller. Playing Atari with deep reinforcement learning. arXiv
preprint arXiv:1312.5602, 2013.
14. J. Mugan and B. Kuipers. Autonomous learning of high-level states and actions in
continuous environments. IEEE Trans. Autonomous Mental Development, 4(1):70–86,
2012.
15. Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High
confidence predictions for unrecognizable images. arXiv preprint arXiv:1412.1897, 2014.
16. D. M. Pierce and B. J. Kuipers. Map learning with uninterpreted sensors and effectors.
92:169–227, 1997.
17. Marc’Aurelio Ranzato, Christopher Poultney, Sumit Chopra, and Yann LeCun. Efficient
learning of sparse representations with an energy-based model. In Advances in neural
information processing systems, pages 1137–1144, 2006.
50. References (continued)
18. David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Parallel Distributed
Processing: Explorations in the Microstructure of Cognition, Volume 2, chapter Learning
representations by error propagation, pages 3–18–362. 1986.
19. Burrhus Frederic Skinner. Science and human behavior. Simon and Schuster, 1953.
20. Richard Socher, Christopher D Manning, and Andrew Y Ng. Learning continuous phrase
representations and syntactic parsing with recursive neural networks. In Proceedings of the
NIPS-2010 Deep Learning and Unsupervised Feature Learning Workshop, pages 1–9, 2010.
21. R. S. Sutton and A. G. Barto. Reinforcement Learning. MIT Press, Cambridge MA, 1998.
22. E.L. Thorndike. Animal intelligence: an experimental study of the associative processes in
animals. 1898.