This presentation is for introducing google DeepMind's DeepDPG algorithm to my colleagues.
I tried my best to make it easy to be understood...
Comment is always welcome :)
hiddenmaze91.blogspot.com
1118_Seminar_Continuous_Deep Q-Learning with Model based accelerationHye-min Ahn
The material that I've used to present the paper
"Continuous Deep Q-Learning with Model-based Acceleration", S.Gu, T.Lillicrap, I.Sutskever, S.Levine, 2016 ICML
Mathematics online: some common algorithmsMark Moriarty
Brief overview of some basic algorithms used online and across data-mining, and a word on where to learn them. Prepared specially for UCC Boole Prize 2012.
(141205) Masters_Thesis_Defense_Sundong_KimSundong Kim
Masters thesis defense presentation slide
Topic : Maximizing Influence over a Target user through Friend Recommendation
Presenter : Sundong Kim @ KAIST IsysE department
Keywords : Social network, Friend recommendation, Incremental Algorithm, Maximizing influence
This presentation is for introducing google DeepMind's DeepDPG algorithm to my colleagues.
I tried my best to make it easy to be understood...
Comment is always welcome :)
hiddenmaze91.blogspot.com
1118_Seminar_Continuous_Deep Q-Learning with Model based accelerationHye-min Ahn
The material that I've used to present the paper
"Continuous Deep Q-Learning with Model-based Acceleration", S.Gu, T.Lillicrap, I.Sutskever, S.Levine, 2016 ICML
Mathematics online: some common algorithmsMark Moriarty
Brief overview of some basic algorithms used online and across data-mining, and a word on where to learn them. Prepared specially for UCC Boole Prize 2012.
(141205) Masters_Thesis_Defense_Sundong_KimSundong Kim
Masters thesis defense presentation slide
Topic : Maximizing Influence over a Target user through Friend Recommendation
Presenter : Sundong Kim @ KAIST IsysE department
Keywords : Social network, Friend recommendation, Incremental Algorithm, Maximizing influence
DMTM Lecture 13 Representative based clusteringPier Luca Lanzi
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Survey for recursive neural networks. Including recursive neural network (RNN), recursive autoencoder (RAE), unfolding RAE & dynamic pooling, matrix-vector RNN (MV-RNN), and recursive neural tensor network (RNTN), published by Socher et al.
K Means Clustering Algorithm | K Means Example in Python | Machine Learning A...Edureka!
** Python Training for Data Science: https://www.edureka.co/python **
This Edureka Machine Learning tutorial (Machine Learning Tutorial with Python Blog: https://goo.gl/fe7ykh ) series presents another video on "K-Means Clustering Algorithm". Within the video you will learn the concepts of K-Means clustering and its implementation using python. Below are the topics covered in today's session:
1. What is Clustering?
2. Types of Clustering
3. What is K-Means Clustering?
4. How does a K-Means Algorithm works?
5. K-Means Clustering Using Python
Machine Learning Tutorial Playlist: https://goo.gl/UxjTxm
Slides from the 2016/2017 edition of the Video game Design and Programming course at the Politecnico di Milano. More information at http://www.polimigamecollective.org Some of the video games developed by the students during the course are available at https://polimi-game-collective.itch.io
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
We propose a distributed deep learning model to learn control policies directly from high-dimensional sensory input using reinforcement learning (RL). We adapt the DistBelief software framework to efficiently train the deep RL agents using the Apache Spark cluster computing framework.
Maximum Entropy Reinforcement Learning (Stochastic Control)Dongmin Lee
I reviewed the following papers.
- T. Haarnoja, et al., “Reinforcement Learning with Deep Energy-Based Policies", ICML 2017
- T. Haarnoja, et al., “Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor", ICML 2018
- T. Haarnoja, et al., “Soft Actor-Critic Algorithms and Applications", arXiv preprint 2018
Thank you.
K Means Clustering Algorithm | K Means Clustering Example | Machine Learning ...Simplilearn
This K-Means clustering algorithm presentation will take you through the machine learning introduction, types of clustering algorithms, k-means clustering, how does K-Means clustering work and at least explains K-Means clustering by taking a real life use case. This Machine Learning algorithm tutorial video is ideal for beginners to learn how K-Means clustering work.
Below topics are covered in this K-Means Clustering Algorithm presentation:
1. Types of Machine Learning?
2. What is K-Means Clustering?
3. Applications of K-Means Clustering
4. Common distance measure
5. How does K-Means Clustering work?
6. K-Means Clustering Algorithm
7. Demo: k-Means Clustering
8. Use case: Color compression
- - - - - - - -
About Simplilearn Machine Learning course:
A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all people’s digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars.This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning.
- - - - - - -
Why learn Machine Learning?
Machine Learning is taking over the world- and with that, there is a growing need among companies for professionals to know the ins and outs of Machine Learning
The Machine Learning market size is expected to grow from USD 1.03 Billion in 2016 to USD 8.81 Billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period.
- - - - - -
What skills will you learn from this Machine Learning course?
By the end of this Machine Learning course, you will be able to:
1. Master the concepts of supervised, unsupervised and reinforcement learning concepts and modeling.
2. Gain practical mastery over principles, algorithms, and applications of Machine Learning through a hands-on approach which includes working on 28 projects and one capstone project.
3. Acquire thorough knowledge of the mathematical and heuristic aspects of Machine Learning.
4. Understand the concepts and operation of support vector machines, kernel SVM, naive bayes, decision tree classifier, random forest classifier, logistic regression, K-nearest neighbors, K-means clustering and more.
5. Be able to model a wide variety of robust Machine Learning algorithms including deep learning, clustering, and recommendation systems
- - - - - - -
DMTM Lecture 13 Representative based clusteringPier Luca Lanzi
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
Survey for recursive neural networks. Including recursive neural network (RNN), recursive autoencoder (RAE), unfolding RAE & dynamic pooling, matrix-vector RNN (MV-RNN), and recursive neural tensor network (RNTN), published by Socher et al.
K Means Clustering Algorithm | K Means Example in Python | Machine Learning A...Edureka!
** Python Training for Data Science: https://www.edureka.co/python **
This Edureka Machine Learning tutorial (Machine Learning Tutorial with Python Blog: https://goo.gl/fe7ykh ) series presents another video on "K-Means Clustering Algorithm". Within the video you will learn the concepts of K-Means clustering and its implementation using python. Below are the topics covered in today's session:
1. What is Clustering?
2. Types of Clustering
3. What is K-Means Clustering?
4. How does a K-Means Algorithm works?
5. K-Means Clustering Using Python
Machine Learning Tutorial Playlist: https://goo.gl/UxjTxm
Slides from the 2016/2017 edition of the Video game Design and Programming course at the Politecnico di Milano. More information at http://www.polimigamecollective.org Some of the video games developed by the students during the course are available at https://polimi-game-collective.itch.io
Slides for the 2016/2017 edition of the Data Mining and Text Mining Course at the Politecnico di Milano. The course is also part of the joint program with the University of Illinois at Chicago.
We propose a distributed deep learning model to learn control policies directly from high-dimensional sensory input using reinforcement learning (RL). We adapt the DistBelief software framework to efficiently train the deep RL agents using the Apache Spark cluster computing framework.
Maximum Entropy Reinforcement Learning (Stochastic Control)Dongmin Lee
I reviewed the following papers.
- T. Haarnoja, et al., “Reinforcement Learning with Deep Energy-Based Policies", ICML 2017
- T. Haarnoja, et al., “Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor", ICML 2018
- T. Haarnoja, et al., “Soft Actor-Critic Algorithms and Applications", arXiv preprint 2018
Thank you.
K Means Clustering Algorithm | K Means Clustering Example | Machine Learning ...Simplilearn
This K-Means clustering algorithm presentation will take you through the machine learning introduction, types of clustering algorithms, k-means clustering, how does K-Means clustering work and at least explains K-Means clustering by taking a real life use case. This Machine Learning algorithm tutorial video is ideal for beginners to learn how K-Means clustering work.
Below topics are covered in this K-Means Clustering Algorithm presentation:
1. Types of Machine Learning?
2. What is K-Means Clustering?
3. Applications of K-Means Clustering
4. Common distance measure
5. How does K-Means Clustering work?
6. K-Means Clustering Algorithm
7. Demo: k-Means Clustering
8. Use case: Color compression
- - - - - - - -
About Simplilearn Machine Learning course:
A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all people’s digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars.This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning.
- - - - - - -
Why learn Machine Learning?
Machine Learning is taking over the world- and with that, there is a growing need among companies for professionals to know the ins and outs of Machine Learning
The Machine Learning market size is expected to grow from USD 1.03 Billion in 2016 to USD 8.81 Billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period.
- - - - - -
What skills will you learn from this Machine Learning course?
By the end of this Machine Learning course, you will be able to:
1. Master the concepts of supervised, unsupervised and reinforcement learning concepts and modeling.
2. Gain practical mastery over principles, algorithms, and applications of Machine Learning through a hands-on approach which includes working on 28 projects and one capstone project.
3. Acquire thorough knowledge of the mathematical and heuristic aspects of Machine Learning.
4. Understand the concepts and operation of support vector machines, kernel SVM, naive bayes, decision tree classifier, random forest classifier, logistic regression, K-nearest neighbors, K-means clustering and more.
5. Be able to model a wide variety of robust Machine Learning algorithms including deep learning, clustering, and recommendation systems
- - - - - - -
Please read through these slides. Let me know if you have any questions. Once complete, please sign off with your IEP Team Leader.
Please disregard slides 13 and 23.
Ufinity knows the financial department. That is why we can give advise on the right financial tools, processes and implementation. We want the right solution for your specific business.
Scott Clark, Co-Founder and CEO, SigOpt at MLconf SF 2016MLconf
Using Bayesian Optimization to Tune Machine Learning Models: In this talk we briefly introduce Bayesian Global Optimization as an efficient way to optimize machine learning model parameters, especially when evaluating different parameters is time-consuming or expensive. We will motivate the problem and give example applications.
We will also talk about our development of a robust benchmark suite for our algorithms including test selection, metric design, infrastructure architecture, visualization, and comparison to other standard and open source methods. We will discuss how this evaluation framework empowers our research engineers to confidently and quickly make changes to our core optimization engine.
We will end with an in-depth example of using these methods to tune the features and hyperparameters of a real world problem and give several real world applications.
A presentation about NGBoost (Natural Gradient Boosting) which I presented in the Information Theory and Probabilistic Programming course at the University of Oklahoma.
Extra Lecture - Support Vector Machines (SVM), a lecture in subject module St...Maninda Edirisooriya
Support Vector Machines are one of the main tool in classical Machine Learning toolbox. This was one of the lectures of a full course I taught in University of Moratuwa, Sri Lanka on 2023 second half of the year.
발표자: 곽동현(서울대 박사과정, 현 NAVER Clova)
강화학습(Reinforcement learning)의 개요 및 최근 Deep learning 기반의 RL 트렌드를 소개합니다.
발표영상:
http://tv.naver.com/v/2024376
https://youtu.be/dw0sHzE1oAc
A machine learning method for efficient design optimization in nano-optics JCMwave
The slideshow contains a brief explanation of Gaussian process regression and Bayesian optimization. For two optimization problems, benchmarks against other local gradient-based and global heuristic optimization methods are included. They show, that Bayesian optimization can identify better designs in exceptionally short computation times.
3. Introduction
What is Hyperparameter?
• a parameter of a prior distribution, before some evidence is
taken into account
• it is a measure of the probability distribution over probabilities
• external model mechanics
• in our context of deep learning using neural networks,
hyperparameters can be:
• Number of layers
• Number of hidden units
• Weight
• Kernel parameter
| 3
4. x1
y
training data
x2
xn
x1:n -> hyperparameter values y -> test error
Goal :
• For a given setting of hypereparameter values the
system gives test error generated upon training
• And out task is to find the optimal setting of the
hyperparameter for which the test error is minimum
Plotting the problem :
.
.
.
.
.
testerror
hyperparameter
• There is an underlying objective function which
connects these dots
• We want to estimate it and find minima of that
function
• We don’t have any knowledge about the form of the
function we want to optimize
Problem Specification
| 4
5. Naïve Grid Search :
Basic idea : Make a list of all possible combination of
hyperparameter values and do an exhaustive search for best setting
• there can be large number of combinations
• running each setting takes a very long time
Simple Approach
Sampling Random Combination :
Basic idea : Find those hyperparameter which have less impact on
output. Eliminate redundant run of the system by pruning some
combination where these parameters change
• still we are left with large combination of hyperparameter setting
• we need an approach that directs us towards an optimal setting
| 5
6. Bayesian Optimization
Motivation:
• Lack of knowledge about concrete form of the objective function
• Few observation data
• We only need to rely on priors to best estimate the objective function
• Under these setting, Bayesian Optimization appears to be a powerful strategy
• It allows us to model the objective function and get better estimate with each observation
A Bayesian approach works in two steps:
1. Developing a prior function, basically a probabilistic modelling of our current beliefs
2. Developing an acquisition function
Step 1:
• With few initial runs of the system with different hyperparameter, we accumulate observations as
• D = {x1:t; y1:t)}
• x1:t are t different settings of hyperparameter
• y1:t are t different test errors
• P(f) is our prior estimate of the objective function
• When we observe new evaluation, we can compute our posterior probability
• P(f|D) ~ P(D|f) * P(f)
• We use the famous Bayes Theorem to update our belief about the objective function
| 6
7. Bayesian Optimization
Motivation for step 2:
• Our next point of evaluation should be such where our
• estimate about objective function is highly uncertain
• improvement w.r.t current best error is maximum
• We can achieve this via a utility function which considers above constraint and returns the next point of evaluation
Utility function
current best error
prior function
x (t+1)
(next point of observation, t+1 th observation)
Explores the areas
of high uncertainty
Exploits the areas of
max improvement
| 7
8. Gaussian Priors
| 8
Motivation:
• We hold an underlying assumption that the objective function is smooth i.e. for a small change in input the change in
output is small
• It should be continuous
• GP priors can be one of the approach to formalize our prior as they hold these assumptions well
• It says that there is a Gaussian connecting these dots
• There can many Gaussians doing so, we want to approximate those using a similarity measure between given points
Formalizing our GP prior: • Given the data, D, we want to model f’s with multivariate Gaussian
k(xi, xj) = exp –norm(xi - xj)sq = 0, if xi and xj are far
1, if xi and xj are close
f1
f2
f3
~ GP
0
0
0 ,
k11 k12 k13
k21 k22 k23
k31 k32 k33
G =
• K is the co-variance matric, defined by us depending on similarity of
data points
• So, our prior is just a simulation of G
.
.
.
testerror
hyperparameter
f(x)
x
x1 x2 x3
f3
f2
f1
9. Gaussian Priors
| 9
• The Gaussian, G, which we derived are multivariate Gaussian of functions
• Now suppose we want to predict the test error at a particular value of x (say test
data is x*)
• We can assume f* comes from a Gaussian distribution such that,
f* ~ GP( 0, k(x*, x*))
• Now since x* can be assumed to come from the same set of training data,
f* is jointly Gaussian with G defined earlier and given by,
• Thus the whole problem can now be assumed to have cut in multivariate
Gaussian at x*, we get another Gaussian defined in a plane and finding the
mean of that Gaussian
.
.
.
testerror
hyperparameter
f(x)
x
x1 x2 x3
f3
f2
f1
x*
f*
meu*
sigma*
.
f1
f2
f3
f*
~ GP Mue, KG’ = Mue = Mean Vector
K= Covariance Matrix
10. • Cutting a multivariate Gaussian is now basically a problem of conditional distribution
• We can find the mean and variance of this Gaussian by the Multi Variate Gaussian Theorem
• By this theorem we can find the conditional mean and variance from a given joint distribution
• The estimate of the objective function is combining these mean at different cuts
• The shaded region gives the confidence interval of the mean
Gaussian Priors
| 10
11. Acquisition function
Intuition :
• Confidence interval (variance) is a measure of uncertainty
at a point
• So we intend to find the point where the variance is
maximum
• With a constraint that the mean at that point should be
less than the best known error at that state
Formalizing by Probability of Improvement :
PI(x) = P(f(x) <= f(x+) + $)
PI(x) = Probability of improvement at a point x
f(x) = test error at that point
f(x+) = best current test error
$ = parameter that controls uncertainty
| 11
12. Gaussian Posterior
• Now once we know the next setting of hyperparameter, we can evaluate our system at that point
• In this way, we get to know a new evidence
• Knowing this we can update our belief and calculate the posterior function
P(f_new |D, x(t+1))
• We can repeat this whole cycle until we find the setting of hyperparameter for which tests error is very less
Benefits of Bayesian approach
• Effective where objective function is open, not closed form
• When problem is non-convex, which we don’t know
• We need few evaluations of objective function
| 12
Bayesian Optimization Algorithm
13. References
| 13
Practical Bayesian Optimization of Machine Learning Algorithms
Algorithms for Hyper-Parameter Optimization
A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User
Modeling and Hierarchical Reinforcement Learning
Introduction to Gaussian Processes