This document summarizes a study of deep learning models and Bayesian statistics. It discusses the history of artificial intelligence and machine learning before introducing restricted Boltzmann machines, deep belief networks, and Bayesian statistics. It describes experiments applying restricted Boltzmann machines to classify movies and generate images, and using a deep belief network to classify images from multiple datasets with 100% accuracy. The conclusion states that deep learning has advanced artificial intelligence by allowing algorithms to perform multiple tasks and taken us closer to the original goal of general artificial intelligence.
Corinna Cortes, Head of Research, Google, at MLconf NYC 2017MLconf
Corinna Cortes is a Danish computer scientist known for her contributions to machine learning. She is currently the Head of Google Research, New York. Cortes is a recipient of the Paris Kanellakis Theory and Practice Award for her work on theoretical foundations of support vector machines.
Cortes received her M.S. degree in physics from Copenhagen University in 1989. In the same year she joined AT&T Bell Labs as a researcher and remained there for about ten years. She received her Ph.D. in computer science from the University of Rochester in 1993. Cortes currently serves as the Head of Google Research, New York. She is an Editorial Board member of the journal Machine Learning.
Cortes’ research covers a wide range of topics in machine learning, including support vector machines and data mining. In 2008, she jointly with Vladimir Vapnik received the Paris Kanellakis Theory and Practice Award for the development of a highly effective algorithm for supervised learning known as support vector machines (SVM). Today, SVM is one of the most frequently used algorithms in machine learning, which is used in many practical applications, including medical diagnosis and weather forecasting.
Abstract Summary:
Harnessing Neural Networks:
Deep learning has demonstrated impressive performance gain in many machine learning applications. However, unveiling and realizing these performance gains is not always straightforward. Discovering the right network architecture is critical for accuracy and often requires a human in the loop. Some network architectures occasionally produce spurious outputs, and the outputs have to be restricted to meet the needs of an application. Finally, realizing the performance gain in a production system can be difficult because of extensive inference times.
In this talk we discuss methods for making neural networks efficient in production systems. We also discuss an efficient method for automatically learning the network architecture, called AdaNet. We provide theoretical arguments for the algorithm and present experimental evidence for its effectiveness.
LOG MESSAGE ANOMALY DETECTION WITH OVERSAMPLINGijaia
Imbalanced data is a significant challenge in classification with machine learning algorithms. This is particularly important with log message data as negative logs are sparse so this data is typically imbalanced. In this paper, a model to generate text log messages is proposed which employs a SeqGAN network. An Autoencoder is used for feature extraction and anomaly detection is done using a GRU network. The proposed model is evaluated with three imbalanced log data sets, namely BGL, OpenStack, and Thunderbird. Results are presented which show that appropriate oversampling and data balancing
improves anomaly detection accuracy.
Using Deep Learning to Find Similar DressesHJ van Veen
Report by Luís Mey ( https://www.linkedin.com/in/lu%C3%ADs-gustavo-bernardo-mey-97b38927/ ) on Udacity Machine Learning Course - Final Project: Use Deep Learning to Find Similar Dresses.
Deep Learning with Python: Getting started and getting from ideas to insights in minutes.
PyData Seattle 2015
Alex Korbonits (@korbonits)
This presentation was given July 25, 2015 at the PyData Seattle conference hosted by PyData and NumFocus.
Corinna Cortes, Head of Research, Google, at MLconf NYC 2017MLconf
Corinna Cortes is a Danish computer scientist known for her contributions to machine learning. She is currently the Head of Google Research, New York. Cortes is a recipient of the Paris Kanellakis Theory and Practice Award for her work on theoretical foundations of support vector machines.
Cortes received her M.S. degree in physics from Copenhagen University in 1989. In the same year she joined AT&T Bell Labs as a researcher and remained there for about ten years. She received her Ph.D. in computer science from the University of Rochester in 1993. Cortes currently serves as the Head of Google Research, New York. She is an Editorial Board member of the journal Machine Learning.
Cortes’ research covers a wide range of topics in machine learning, including support vector machines and data mining. In 2008, she jointly with Vladimir Vapnik received the Paris Kanellakis Theory and Practice Award for the development of a highly effective algorithm for supervised learning known as support vector machines (SVM). Today, SVM is one of the most frequently used algorithms in machine learning, which is used in many practical applications, including medical diagnosis and weather forecasting.
Abstract Summary:
Harnessing Neural Networks:
Deep learning has demonstrated impressive performance gain in many machine learning applications. However, unveiling and realizing these performance gains is not always straightforward. Discovering the right network architecture is critical for accuracy and often requires a human in the loop. Some network architectures occasionally produce spurious outputs, and the outputs have to be restricted to meet the needs of an application. Finally, realizing the performance gain in a production system can be difficult because of extensive inference times.
In this talk we discuss methods for making neural networks efficient in production systems. We also discuss an efficient method for automatically learning the network architecture, called AdaNet. We provide theoretical arguments for the algorithm and present experimental evidence for its effectiveness.
LOG MESSAGE ANOMALY DETECTION WITH OVERSAMPLINGijaia
Imbalanced data is a significant challenge in classification with machine learning algorithms. This is particularly important with log message data as negative logs are sparse so this data is typically imbalanced. In this paper, a model to generate text log messages is proposed which employs a SeqGAN network. An Autoencoder is used for feature extraction and anomaly detection is done using a GRU network. The proposed model is evaluated with three imbalanced log data sets, namely BGL, OpenStack, and Thunderbird. Results are presented which show that appropriate oversampling and data balancing
improves anomaly detection accuracy.
Using Deep Learning to Find Similar DressesHJ van Veen
Report by Luís Mey ( https://www.linkedin.com/in/lu%C3%ADs-gustavo-bernardo-mey-97b38927/ ) on Udacity Machine Learning Course - Final Project: Use Deep Learning to Find Similar Dresses.
Deep Learning with Python: Getting started and getting from ideas to insights in minutes.
PyData Seattle 2015
Alex Korbonits (@korbonits)
This presentation was given July 25, 2015 at the PyData Seattle conference hosted by PyData and NumFocus.
Deep Learning: Chapter 11 Practical MethodologyJason Tsai
Lecture for Deep Learning 101 study group to be held on June 9th, 2017.
Reference book: https://www.deeplearningbook.org/
Past video archives: https://goo.gl/hxermB
Initiated by Taiwan AI Group (https://www.facebook.com/groups/Taiwan.AI.Group/)
Generative Adversarial Networks and Their ApplicationsArtifacia
This is the presentation from our AI Meet Jan 2017 on GANs and its applications.
You can join Artifacia AI Meet Bangalore Group: https://www.meetup.com/Artifacia-AI-Meet/
Generative Adversarial Networks is an advanced topic and requires a prior basic understanding of CNNs. Here is some pre-reading material for you.
- https://arxiv.org/pdf/1406.2661v1.pdf
- https://arxiv.org/pdf/1701.00160v1.pdf
Slides from the presentation given at M^3 conference: http://www.mcubed.london/
The idea is to use 3 statements to describe and start to work with the TensorFlow library.
Capitalico / Chart Pattern Matching in Financial Trading Using RNNAlpaca
Capitalico is a web/mobile platform that utilizes deep learning to help financial traders build automated trading system by understanding their trading charts. In this talk I show many of the techniques we developed to achieve the best performance and accuracy in deep learning for sequence pattern matching.
It’s long ago, approx. 30 years, since AI was not only a topic for Science-Fiction writers, but also a major research field surrounded with huge hopes and investments. But the over-inflated expectations ended in a subsequent crash and followed by a period of absent funding and interest – the so-called AI winter. However, the last 3 years changed everything – again. Deep learning, a machine learning technique inspired by the human brain, successfully crushed one benchmark after another and tech companies, like Google, Facebook and Microsoft, started to invest billions in AI research. “The pace of progress in artificial general intelligence is incredible fast” (Elon Musk – CEO Tesla & SpaceX) leading to an AI that “would be either the best or the worst thing ever to happen to humanity” (Stephen Hawking – Physicist).
What sparked this new Hype? How is Deep Learning different from previous approaches? Are the advancing AI technologies really a threat for humanity? Let’s look behind the curtain and unravel the reality. This talk will explore why Sundar Pichai (CEO Google) recently announced that “machine learning is a core transformative way by which Google is rethinking everything they are doing” and explain why "Deep Learning is probably one of the most exciting things that is happening in the computer industry” (Jen-Hsun Huang – CEO NVIDIA).
Either a new AI “winter is coming” (Ned Stark – House Stark) or this new wave of innovation might turn out as the “last invention humans ever need to make” (Nick Bostrom – AI Philosoph). Or maybe it’s just another great technology helping humans to achieve more.
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...Myungyon Kim
Deep learning and Tensorflow implementation
2016.11.16
<Cotents>
Feature Engineering
Deep Neural Network
Tensorflow
Tensorflow Implementation
Future works
References
This slides deals with several things about deep learning.
ex) History of Deep learning, Several difficulties and breakthroughs. Things related to deep learning such as activation functions, perceptrons, Backpropagation, pre-train, drop-out, Convolutional Neural Network (CNN), Simple implementation of Tensor Flow, Python, and so on.
딥러닝, 기계학습, 머신러닝, 텐서플로우, 파이썬
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...Simplilearn
This presentation about Deep Learning is designed for beginners who want to learn Deep Learning from scratch. We will look at where Deep Learning is applied and what exactly this term means. We'll see how Deep Learning, Machine Learning, and AI are different and why Deep Learning even came into the picture. We will then proceed to look at Neural Networks, which are the core of Deep Learning. Before we move into the working of Neural Networks, we'll cover activation and cost functions. The video will also introduce you to the most popular Deep Learning platforms. We wrap it up with a demo in TensorFlow to predict if a person receives a salary above or below 50k. Now, let us get started and understand Deep Learning in detail.
Below topics are explained in this Deep Learning presentation:
1. Applications of Deep Learning
2. What is Deep Learning
3. Why is Deep Learning important
4. What are Neural Networks
5. Activation function
6. Cost function
7. How do Neural Networks work
8. Deep Learning platforms
9. Introduction to TensorFlow
10. Use case implementation using TensorFlow
Why Deep Learning?
It is one of the most popular software platforms used for Deep Learning and contains powerful tools to help you build and implement artificial Neural Networks.
Advancements in Deep Learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in Deep Learning models, learn to operate TensorFlow to manage Neural Networks and interpret the results. According to payscale.com, the median salary for engineers with Deep Learning skills tops $120,000 per year.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to:
1. Understand the concepts of TensorFlow, its main functions, operations and the execution pipeline
2. Implement Deep Learning algorithms, understand Neural Networks and traverse the layers of data abstraction which will empower you to understand data like never before
3. Master and comprehend advanced topics such as convolutional Neural Networks, Recurrent Neural Networks, training deep networks and high-level interfaces
4. Build Deep Learning models in TensorFlow and interpret the results
5. Understand the language and fundamental concepts of Artificial Neural Networks
6. Troubleshoot and improve Deep Learning models
Learn more at https://www.simplilearn.com/deep-learning-course-with-tensorflow-training
Scalable Data Science and Deep Learning with H2O
In this session, we introduce the H2O data science platform. We will explain its scalable in-memory architecture and design principles and focus on the implementation of distributed deep learning in H2O. Advanced features such as adaptive learning rates, various forms of regularization, automatic data transformations, checkpointing, grid-search, cross-validation and auto-tuning turn multi-layer neural networks of the past into powerful, easy-to-use predictive analytics tools accessible to everyone. We will present a broad range of use cases and live demos that include world-record deep learning models, anomaly detection tools and approaches for Kaggle data science competitions. We also demonstrate the applicability of H2O in enterprise environments for real-world customer production use cases.
By the end of the hands-on-session, attendees will have learned to perform end-to-end data science workflows with H2O using both the easy-to-use web interface and the flexible R interface. We will cover data ingest, basic feature engineering, feature selection, hyperparameter optimization with N-fold cross-validation, multi-model scoring and taking models into production. We will train supervised and unsupervised methods on realistic datasets. With best-of-breed machine learning algorithms such as elastic net, random forest, gradient boosting and deep learning, you will be able to create your own smart applications.
A local installation of RStudio is recommended for this session.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
What Deep Learning Means for Artificial IntelligenceJonathan Mugan
Describes deep learning as applied to natural language processing, computer vision, and robot actions. Also discusses what deep learning still can't do.
Josh Patterson, Principal at Patterson Consulting: Introduction to Parallel Iterative Machine Learning Algorithms on Hadoop’s NextGeneration YARN Framework
H2O Distributed Deep Learning by Arno Candel 071614Sri Ambati
Deep Learning R Vignette Documentation: https://github.com/0xdata/h2o/tree/master/docs/deeplearning/
Deep Learning has been dominating recent machine learning competitions with better predictions. Unlike the neural networks of the past, modern Deep Learning methods have cracked the code for training stability and generalization. Deep Learning is not only the leader in image and speech recognition tasks, but is also emerging as the algorithm of choice in traditional business analytics.
This talk introduces Deep Learning and implementation concepts in the open-source H2O in-memory prediction engine. Designed for the solution of enterprise-scale problems on distributed compute clusters, it offers advanced features such as adaptive learning rate, dropout regularization and optimization for class imbalance. World record performance on the classic MNIST dataset, best-in-class accuracy for eBay text classification and others showcase the power of this game changing technology. A whole new ecosystem of Intelligent Applications is emerging with Deep Learning at its core.
About the Speaker: Arno Candel
Prior to joining 0xdata as Physicist & Hacker, Arno was a founding Senior MTS at Skytree where he designed and implemented high-performance machine learning algorithms. He has over a decade of experience in HPC with C++/MPI and had access to the world's largest supercomputers as a Staff Scientist at SLAC National Accelerator Laboratory where he participated in US DOE scientific computing initiatives. While at SLAC, he authored the first curvilinear finite-element simulation code for space-charge dominated relativistic free electrons and scaled it to thousands of compute nodes.
He also led a collaboration with CERN to model the electromagnetic performance of CLIC, a ginormous e+e- collider and potential successor of LHC. Arno has authored dozens of scientific papers and was a sought-after academic conference speaker. He holds a PhD and Masters summa cum laude in Physics from ETH Zurich.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
The field of Artificial Intelligence (AI) has been revitalized in this decade, primarily due to the large-scale application of Deep Learning (DL) and other Machine Learning (ML) algorithms. This has been most evident in applications like computer vision, natural language processing, and game bots. However, extraordinary successes within a short period of time have also had the unintended consequence of causing a sharp difference of opinion in research and industrial communities regarding the capabilities and limitations of deep learning. A few questions you might have heard being asked (or asked yourself) include:
a. We don’t know how Deep Neural Networks make decisions, so can we trust them?
b. Can Deep Learning deal with highly non-linear continuous systems with millions of variables?
c. Can Deep Learning solve the Artificial General Intelligence problem?
The goal of this seminar is to provide a 1000-feet view of Deep Learning and hopefully answer the questions above. The seminar will touch upon the evolution, current state of the art, and peculiarities of Deep Learning, and share thoughts on using Deep Learning as a tool for developing power system solutions.
Deep Learning: Chapter 11 Practical MethodologyJason Tsai
Lecture for Deep Learning 101 study group to be held on June 9th, 2017.
Reference book: https://www.deeplearningbook.org/
Past video archives: https://goo.gl/hxermB
Initiated by Taiwan AI Group (https://www.facebook.com/groups/Taiwan.AI.Group/)
Generative Adversarial Networks and Their ApplicationsArtifacia
This is the presentation from our AI Meet Jan 2017 on GANs and its applications.
You can join Artifacia AI Meet Bangalore Group: https://www.meetup.com/Artifacia-AI-Meet/
Generative Adversarial Networks is an advanced topic and requires a prior basic understanding of CNNs. Here is some pre-reading material for you.
- https://arxiv.org/pdf/1406.2661v1.pdf
- https://arxiv.org/pdf/1701.00160v1.pdf
Slides from the presentation given at M^3 conference: http://www.mcubed.london/
The idea is to use 3 statements to describe and start to work with the TensorFlow library.
Capitalico / Chart Pattern Matching in Financial Trading Using RNNAlpaca
Capitalico is a web/mobile platform that utilizes deep learning to help financial traders build automated trading system by understanding their trading charts. In this talk I show many of the techniques we developed to achieve the best performance and accuracy in deep learning for sequence pattern matching.
It’s long ago, approx. 30 years, since AI was not only a topic for Science-Fiction writers, but also a major research field surrounded with huge hopes and investments. But the over-inflated expectations ended in a subsequent crash and followed by a period of absent funding and interest – the so-called AI winter. However, the last 3 years changed everything – again. Deep learning, a machine learning technique inspired by the human brain, successfully crushed one benchmark after another and tech companies, like Google, Facebook and Microsoft, started to invest billions in AI research. “The pace of progress in artificial general intelligence is incredible fast” (Elon Musk – CEO Tesla & SpaceX) leading to an AI that “would be either the best or the worst thing ever to happen to humanity” (Stephen Hawking – Physicist).
What sparked this new Hype? How is Deep Learning different from previous approaches? Are the advancing AI technologies really a threat for humanity? Let’s look behind the curtain and unravel the reality. This talk will explore why Sundar Pichai (CEO Google) recently announced that “machine learning is a core transformative way by which Google is rethinking everything they are doing” and explain why "Deep Learning is probably one of the most exciting things that is happening in the computer industry” (Jen-Hsun Huang – CEO NVIDIA).
Either a new AI “winter is coming” (Ned Stark – House Stark) or this new wave of innovation might turn out as the “last invention humans ever need to make” (Nick Bostrom – AI Philosoph). Or maybe it’s just another great technology helping humans to achieve more.
Deep Learning and Tensorflow Implementation(딥러닝, 텐서플로우, 파이썬, CNN)_Myungyon Ki...Myungyon Kim
Deep learning and Tensorflow implementation
2016.11.16
<Cotents>
Feature Engineering
Deep Neural Network
Tensorflow
Tensorflow Implementation
Future works
References
This slides deals with several things about deep learning.
ex) History of Deep learning, Several difficulties and breakthroughs. Things related to deep learning such as activation functions, perceptrons, Backpropagation, pre-train, drop-out, Convolutional Neural Network (CNN), Simple implementation of Tensor Flow, Python, and so on.
딥러닝, 기계학습, 머신러닝, 텐서플로우, 파이썬
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...Simplilearn
This presentation about Deep Learning is designed for beginners who want to learn Deep Learning from scratch. We will look at where Deep Learning is applied and what exactly this term means. We'll see how Deep Learning, Machine Learning, and AI are different and why Deep Learning even came into the picture. We will then proceed to look at Neural Networks, which are the core of Deep Learning. Before we move into the working of Neural Networks, we'll cover activation and cost functions. The video will also introduce you to the most popular Deep Learning platforms. We wrap it up with a demo in TensorFlow to predict if a person receives a salary above or below 50k. Now, let us get started and understand Deep Learning in detail.
Below topics are explained in this Deep Learning presentation:
1. Applications of Deep Learning
2. What is Deep Learning
3. Why is Deep Learning important
4. What are Neural Networks
5. Activation function
6. Cost function
7. How do Neural Networks work
8. Deep Learning platforms
9. Introduction to TensorFlow
10. Use case implementation using TensorFlow
Why Deep Learning?
It is one of the most popular software platforms used for Deep Learning and contains powerful tools to help you build and implement artificial Neural Networks.
Advancements in Deep Learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in Deep Learning models, learn to operate TensorFlow to manage Neural Networks and interpret the results. According to payscale.com, the median salary for engineers with Deep Learning skills tops $120,000 per year.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to:
1. Understand the concepts of TensorFlow, its main functions, operations and the execution pipeline
2. Implement Deep Learning algorithms, understand Neural Networks and traverse the layers of data abstraction which will empower you to understand data like never before
3. Master and comprehend advanced topics such as convolutional Neural Networks, Recurrent Neural Networks, training deep networks and high-level interfaces
4. Build Deep Learning models in TensorFlow and interpret the results
5. Understand the language and fundamental concepts of Artificial Neural Networks
6. Troubleshoot and improve Deep Learning models
Learn more at https://www.simplilearn.com/deep-learning-course-with-tensorflow-training
Scalable Data Science and Deep Learning with H2O
In this session, we introduce the H2O data science platform. We will explain its scalable in-memory architecture and design principles and focus on the implementation of distributed deep learning in H2O. Advanced features such as adaptive learning rates, various forms of regularization, automatic data transformations, checkpointing, grid-search, cross-validation and auto-tuning turn multi-layer neural networks of the past into powerful, easy-to-use predictive analytics tools accessible to everyone. We will present a broad range of use cases and live demos that include world-record deep learning models, anomaly detection tools and approaches for Kaggle data science competitions. We also demonstrate the applicability of H2O in enterprise environments for real-world customer production use cases.
By the end of the hands-on-session, attendees will have learned to perform end-to-end data science workflows with H2O using both the easy-to-use web interface and the flexible R interface. We will cover data ingest, basic feature engineering, feature selection, hyperparameter optimization with N-fold cross-validation, multi-model scoring and taking models into production. We will train supervised and unsupervised methods on realistic datasets. With best-of-breed machine learning algorithms such as elastic net, random forest, gradient boosting and deep learning, you will be able to create your own smart applications.
A local installation of RStudio is recommended for this session.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
What Deep Learning Means for Artificial IntelligenceJonathan Mugan
Describes deep learning as applied to natural language processing, computer vision, and robot actions. Also discusses what deep learning still can't do.
Josh Patterson, Principal at Patterson Consulting: Introduction to Parallel Iterative Machine Learning Algorithms on Hadoop’s NextGeneration YARN Framework
H2O Distributed Deep Learning by Arno Candel 071614Sri Ambati
Deep Learning R Vignette Documentation: https://github.com/0xdata/h2o/tree/master/docs/deeplearning/
Deep Learning has been dominating recent machine learning competitions with better predictions. Unlike the neural networks of the past, modern Deep Learning methods have cracked the code for training stability and generalization. Deep Learning is not only the leader in image and speech recognition tasks, but is also emerging as the algorithm of choice in traditional business analytics.
This talk introduces Deep Learning and implementation concepts in the open-source H2O in-memory prediction engine. Designed for the solution of enterprise-scale problems on distributed compute clusters, it offers advanced features such as adaptive learning rate, dropout regularization and optimization for class imbalance. World record performance on the classic MNIST dataset, best-in-class accuracy for eBay text classification and others showcase the power of this game changing technology. A whole new ecosystem of Intelligent Applications is emerging with Deep Learning at its core.
About the Speaker: Arno Candel
Prior to joining 0xdata as Physicist & Hacker, Arno was a founding Senior MTS at Skytree where he designed and implemented high-performance machine learning algorithms. He has over a decade of experience in HPC with C++/MPI and had access to the world's largest supercomputers as a Staff Scientist at SLAC National Accelerator Laboratory where he participated in US DOE scientific computing initiatives. While at SLAC, he authored the first curvilinear finite-element simulation code for space-charge dominated relativistic free electrons and scaled it to thousands of compute nodes.
He also led a collaboration with CERN to model the electromagnetic performance of CLIC, a ginormous e+e- collider and potential successor of LHC. Arno has authored dozens of scientific papers and was a sought-after academic conference speaker. He holds a PhD and Masters summa cum laude in Physics from ETH Zurich.
- Powered by the open source machine learning software H2O.ai. Contributors welcome at: https://github.com/h2oai
- To view videos on H2O open source machine learning software, go to: https://www.youtube.com/user/0xdata
The field of Artificial Intelligence (AI) has been revitalized in this decade, primarily due to the large-scale application of Deep Learning (DL) and other Machine Learning (ML) algorithms. This has been most evident in applications like computer vision, natural language processing, and game bots. However, extraordinary successes within a short period of time have also had the unintended consequence of causing a sharp difference of opinion in research and industrial communities regarding the capabilities and limitations of deep learning. A few questions you might have heard being asked (or asked yourself) include:
a. We don’t know how Deep Neural Networks make decisions, so can we trust them?
b. Can Deep Learning deal with highly non-linear continuous systems with millions of variables?
c. Can Deep Learning solve the Artificial General Intelligence problem?
The goal of this seminar is to provide a 1000-feet view of Deep Learning and hopefully answer the questions above. The seminar will touch upon the evolution, current state of the art, and peculiarities of Deep Learning, and share thoughts on using Deep Learning as a tool for developing power system solutions.
Machine learning_ Replicating Human BrainNishant Jain
Slides will make you realize how humans makes decision and following the same pattern how Machines are trained to learn and make decisions. Slides gives an overview of all the steps involved in designing an efficient decision making machine.
This talk was presented in Startup Master Class 2017 - http://aaiitkblr.org/smc/ 2017 @ Christ College Bangalore. Hosted by IIT Kanpur Alumni Association and co-presented by IIT KGP Alumni Association, IITACB, PanIIT, IIMA and IIMB alumni.
My co-presenter was Biswa Gourav Singh. And contributor was Navin Manaswi.
http://dataconomy.com/2017/04/history-neural-networks/ - timeline for neural networks
Separating Hype from Reality in Deep Learning with Sameer FarooquiDatabricks
Deep Learning is all the rage these days, but where does the reality of what Deep Learning can do end and the media hype begin? In this talk, I will dispel common myths about Deep Learning that are not necessarily true and help you decide whether you should practically use Deep Learning in your software stack.
I’ll begin with a technical overview of common neural network architectures like CNNs, RNNs, GANs and their common use cases like computer vision, language understanding or unsupervised machine learning. Then I’ll separate the hype from reality around questions like:
• When should you prefer traditional ML systems like scikit learn or Spark.ML instead of Deep Learning?
• Do you no longer need to do careful feature extraction and standardization if using Deep Learning?
• Do you really need terabytes of data when training neural networks or can you ‘steal’ pre-trained lower layers from public models by using transfer learning?
• How do you decide which activation function (like ReLU, leaky ReLU, ELU, etc) or optimizer (like Momentum, AdaGrad, RMSProp, Adam, etc) to use in your neural network?
• Should you randomly initialize the weights in your network or use more advanced strategies like Xavier or He initialization?
• How easy is it to overfit/overtrain a neural network and what are the common techniques to ovoid overfitting (like l1/l2 regularization, dropout and early stopping)?
The goal of this report is the presentation of our biometry and security course’s project: Face recognition for Labeled Faces in the Wild dataset using Convolutional Neural Network technology with Graphlab Framework.
Speaker: Pierre Richemond, Data Science Institute of Imperial College
Title: Cutting edge generative models: Applications and implications
Abstract: This talk will examine recent developments in deep learning content generation at scale. Whether it be images or text, the latest methods have now reached a level of quality making it hard to discriminate between human- and AI-generated content. We will review recent examples of such generative models, and put their significance in a broader context, in light of such powerful tools’ potential for dual use.
Bio: Pierre is currently researching his PhD in deep reinforcement learning at the Data Science Institute of Imperial College. He also teaches Deep Learning at the Graduate School, and helps to run the Deep Learning Network and organises thematic reading groups. His background is in mathematics - he has studied electrical engineering at ENST, probability theory and stochastic processes at Universite Paris VI - Ecole Polytechnique, and business management at HEC.
Deep learning algorithms have drawn the attention of researchers working in the field of computer vision, speech
recognition, malware detection, pattern recognition and natural language processing. In this paper, we present an overview of
deep learning techniques like Convolutional neural network, deep belief network, Autoencoder, Restricted Boltzmann machine
and recurrent neural network. With this, current work of deep learning algorithms on malware detection is shown with the
help of literature survey. Suggestions for future research are given with full justification. We also showed the experimental
analysis in order to show the importance of deep learning techniques.
Vertex has invested in companies across geographies addressing different industry applications leveraging AI to transform their service offerings. Read more on the trends and waves of AI developments observed.
For the full video of this presentation, please visit:
https://www.edge-ai-vision.com/2021/02/introducing-machine-learning-and-how-to-teach-machines-to-see-a-presentation-from-tryolabs/
Facundo Parodi, Research and Machine Learning Engineer at Tryolabs, presents the “Introduction to Machine Learning and How to Teach Machines to See” tutorial at the September 2020 Embedded Vision Summit.
What is machine learning? How can machines distinguish a cat from a dog in an image? What’s the magic behind convolutional neural networks? These are some of the questions Parodi answers in this introductory talk on machine learning in computer vision.
Parodi introduces machine learning and explores the different types of problems it can solve. He explains the main components of practical machine learning, from data gathering and training to deployment. He then focuses on deep learning as an important machine learning technique and provides an introduction to convolutional neural networks and how they can be used to solve image classification problems. Parodi will also touches on recent advancements in deep learning and how they have revolutionized the entire field of computer vision.
Artificial Intelligence, Machine Learning and Deep Learning with CNN
ProjectReport
1. 1
A Study of Deep Learning Models and
Bayesian Statistics
By: Pritish Yuvraj
Summer Research Fellow
Indian Statistical Institute, Kolkata
Guide: Prof. Rajat Kumar De
Machine Intelligence Unit
Indian Statistical Institute, Kolkata
2. 2
A Study of Deep Learning Models and
Bayesian Statistics
By: Pritish Yuvraj
Summer Research Fellow
Indian Statistical Institute, Kolkata
Guide: Prof. Rajat Kumar De
Machine Intelligence Unit
Indian Statistical Institute, Kolkata
3. 3
Table of Contents
Serial No: Content Page No:
1) History 3 - 4
2)
Restricted Boltzmann
Machine
5 - 11
3) Deep Belief Network 12 - 18
4) Bayesian Statistics 19
5) Conclusion 20
4. 4
1. History
● The ultimate aim for Artificial Intelligence is to reach a pinnacle where machines can
accomplish tasks which are pernicious, byzantine and arduous for human beings to
perform. Towards this direction, researchers are working and have invented multiple
algorithms. With a colossal amount of money and manpower dedicated to the
improvement of the field of Artificial Intelligence, the field is improving very fast.
● The history of Artificial Intelligence backs to 1940 when Philosopher Pamela
McCorduck, attempted to describe the process of human thinking as the mechanical
manipulation of symbols. Time passed, algorithms with the basis of statistics, which
could "theoretically" solve real life problems. Unfortunately, these problems could not
be applicable to real life situations. These Artificially Intelligent programs failed to
undertake the importance of Environment, hence failed in real life. This was paved the
advent of Machine Learning.
● A subfield of Artificial Intelligence evolved from studies of Pattern Recognition and
Computational Learning Theory. In 1959, Arthur Samuel defined machine learning as
a "Field of study that gives computers the ability to learn without being explicitly
programmed". Machine Learning was able to overcome the deficiency of AI. Unlike the
previous AI applications, ML incorporated a lot of Data.
5. 5
A notion was created that more the data from pragmatic sources the better will your program run. The world
knew about the importance of "Data". A relevant question here asked could be, why did Machine Learning
do well? It's because it was being trained on real datasets recorded from sensors, cameras, and recording
devices. Statistical Analysis of these data gave us a good insight into the convoluted pattern stored in data
and hence Machine Learning Researchers were able to mimic the learned information into a real life
application. Some examples where Machine Learning is being actively used is Spam Filtering, Search
Engines, Computer Vision, etc.
● At the present moment, a mammoth amount of data has already been collected and the process is still
continuing. Here comes the glitch, these data are mostly unsupervised. Data Mining and other techniques
perform the task of Unsupervised Learning better than Machine Learning. These stirred up an environment
to delve into a complex subfield of Machine Learning, "Deep Learning". Deep Learning was present since
1940's but never utilized as:
● 1) Data was not Sufficient
● 2) Computation power was high
● 3) No immediate necessity
●
● In this project, a study of Deep Learning Algorithms was conducted by implementing the algorithms in C++,
using object oriented approach and results are shared. The following four algorithms were implemented:
● 1) Restricted Boltzmann Machine
● 2) Deep Belief Network
● 3) Recurrent Neural Network
● 4) Convoluted Neural Network
6. 6
2.1 Restricted Boltzmann Machine
● Found prominance after
efforts from Geoffrey Hinton
(University of Toranto).
● Unsupervised or
Supervised Depending on
Application
● Applications in Dimensional
Reduction, Classification,
Collaborative Filtering,
Feature Learning and Topic
Modelling.
7. 7
2.1 RBM: Mathematical Formulas
Energy Configurations:
Probability Dist (where
Z is PartitionFunction):
Probabiltiy V given H:
Probability H given V:
Weight Update:
8. 8
2.2 RBM:Training
● Take training Sample (v), Compute probabilites of hidden units.
● Sample a hidden activation vector (h) from the above probability Dist.
● Compute “Outer Product” of (v) and (h) (called Positive Gradient p(x)).
● From (h), sample reconstruction (v') of visible units, resample hidden activations (h') from this
probability Dist. (Gibbs Sampling)
● Compute “Outer Product” of (v') and (h'). (called Negative Gradient q(x))
● Update the Weights based upon differences between Positive Gradient and Negative Gradient.
● Aim of KL Divergence is to maximize common area from function p(x) and q(x).
● Or in other ways the probability of positive gradient and the probability of negative gradient are converged.
● The better the convergence the better we have predicted probability dist. Of input to prob. Dist of hidden Layer.
9. 9
Experiment 1: RBM
● Example taken from Edwin Chen blog.
● Results are found after implementing RBM in C+
+. Codes available with github under profile of
“Pritish Yuvraj”.
● RBM conducts the experiment in Unsupervised
way. It isn't fed with the comments. So based
only on the inputs it needs create some sort of
pattern. Only hint it has is that it needs to create 2
separate group based on the given inputs.
Harray Potter Avatar LOTR Gladiator Titanic Glitter Comments
Alice 1 1 1 0 0 0 Big SF/Fantasy
Fan
Bob 1 0 1 0 0 0 SF/Fantasy fan,
but not Avatar
Carol 1 1 1 0 0 0 Big SG/fantasy
fab
David 0 0 1 1 1 0 Big Oscar
Winners Fan
Eric 0 0 1 1 1 0 Oscar Fan Except
Titanic
Fred 0 0 1 1 1 0 Big Oscar Winner
Fan
10. 10
Conclusion 1: RBM
Hidden Layer
1
Hidden Layer
2
Harry Potter -7.70958 6.260625
Avatar -13.7941 3.09608
LOTR3 8.89752 4.491787
Gladiator 7.87261 -6.69001
Titanic 7.87356 -6.73726
Glitter -8.50164 -5.07191
●
Result Inference:
● 1) Harry Potter and Avatar form one group
(Science Fiction/ Fantasy Movies).
● 2) Gladiator and Titanic form another group
(Oscar Winners).
● 3) LOTR3 and Glitter don't belong clearly to
anyone of the two groups.
● Experiment Conducted with 6 visible
layers (Inputs) and 2 hidden layers
(Output).
● No of Epoch = 50000
● Final Error after iteration of all the
epochs = 5.683 * 10-6 .
11. 11
Experiment 2: RBM
● Applied Restricted Boltzmann Machine
Algorithm on Datasets available by Yale
University. The database is called “Yale
Face Database”.
● References: P. Belhumeur, J. Hespanha, D.
Kriegman, ÒEigenfaces vs. Fisherfaces:
Recognition Using Class Specific Linear
Projection,Ó IEEE Transactions on Pattern
Analysis and Machine Intelligence, July
1997, pp. 711-720.
● The next slide will show the conlcusion of
the experiment conducted. We try to
generate the image based on whatever the
RBM has learned. The number of epochs
and Hidden Layers used wil are mentioned
on the next slide.
13. 13
3. Deep Belief Network
History:
● Observed by Yee-Whye Teh, a student
of Geoffrey Hinton.
● 1st Effective Deep Learning Algorithm.
About the Model:
● Generative Graphical Model.
● Can be used in Unsupervised /
Supervised way.
● Composition of RBM's in stack.
● Supervised Deep Belief Network can
be used for Classification.
15. 15
3.2 DBN: Algorithm
● 1) Train RBM on inputs (X) to obtain weight matrix (W).
● 2)Transform (X) by the RBM to produce new data (X').
● 3) Repeat this procedure with X <- X' for the next pair of layers
● 4) Stop before the top 2 layers.
● 5) Fine-tune the top 2 layers for Supervised Learning.
3.3 Fine-Tuning
Implemented using RBM stack layers and Logistic
Classifier in Fine-Tuning Stage. Implementation of this
program is available on github under the profile of “Pritish
Yuvraj”
Methods for Fine-tuning:
● Feed Forward Network
● Support Vector Machine
● Logistic Classifier
16. 16
3.4 SoftMax Function
● The purpose of SoftMax Function is to
convert the Output into probability it
belongs to a certain group.
(Classification Problems).
● Eg.
– [0.1, 0.0001, 0.00002] becomes
– [0.35558, 0.3220, 0.3219]
– Or, [35%, 32%, 32%] (approx).
17. 17
Experiment 3: DBN
● The objective of the experiment was to
determine the accuracy of Deep Belief
Network wrt to other Machine Learning
Algorithms.
● The problem we picked was to classify
images.
● To conduct the experiment, two image
databases were merged to create an
artificial database.
– LFW database (Labeled Wild Face taken from
Department of Computer Science, University
of Massachussets, Amhrest)
– Flowers database (Department of Computer
Science, University of Oxford)
.
18. 18
3. 5 DBN Architecture for the Experiment
Preprocessing of Images.
● First the images were preprocessed
Black/White from RBM.
● Pixels were made unifrom to 320 * 240 pixels.
● No of Input Neurons: 77760
● 2 Hidden Layers with 100 neurons
each.
● Output Layer with 2 neurons,
classifying either a flower or a human
in the image.
● The Fine-Tuning was achieved
using Logistic Regression Classifier.
● Probablity of an image belonging to a
group was decided based on
SoftMax Funciton.
19. 19
3. 6 DBN: Results
Algorithm Accuracy (in %):
Stochastic Gradient
Descent
96.20%
Random Forest Tree
Classifier
98.26%
Support Vector
Machine
98.3%
Deep Belief Network 100%
Detailed Report on Deep Belief Network
performance:
Number of Epochs
(of Entire
Architecture):
Accuracy
10 74.91%
50 95.38%
100 100%
20. 20
4. Bayesian statistics●
Three approaches to Probability
– Axiomatic
●
Probability by definition and properties
– Relative Frequency
●
Repeated trials
– Degree of belief (subjective)
●
Personal measure of uncertainty
●
Problems
– The chance that a meteor strikes earth is 1%
– The probability of rain today is 30%
– The chance of getting an A on the exam is 50%
4.1 Bayes Theorem for Statistics
●
Let θ represent parameter(s)
●
Let X represent data
●
Left-hand side is a function of θ
●
Denominator on right-hand side does not depend on θ
●
Posterior distribution: Likelihood x Prior distribution
●
Posterior dist’n = Constant x Likelihood x Prior dist’n
●
Equation can be understood at the level of densities
●
Goal: Explore the posterior distribution of θ
( | ) ( | ) ( ) / ( )f X f X f f X
( | ) ( | ) ( )f X f X f
21. 21
5. Conclusion
●
The project was on much more on the practical aspects of Deep Learning and
extracting the statistical knowledge required for that.
●
Deep Learning is a new emerging field of Artificial Intelligence which has brought
us one step closer to real vision of AI.
●
RBM is very important as most of the data present in the world are unsupervised.
The same goes with Deep Belief Network, We can use a lot of unsupervised data
to train the intial stack of RBM's on DBN and then use fine-tuning method for some
small supervised dataset.
●
DBN is more accurate compared to SVM and other Machine Learning algorithms
as can be deduced by the results of the experiment conducted in DBN section.
●
Unlike Machine Learning where an algorithm does particularly a single task, Deep
Learning Algorithms can perform multiple tasks. Like the same DBN can be used in
Classification of an Image or Classification of Text.
●
This was the orignal dream of Artificial Intelligence, from which we are still very far
but Deep Learning has taken us one step closer to it.