SlideShare a Scribd company logo
https://www.slideshare.net/MingukKang/vae-ppt
VAE 알아보기
KL-divergence ..
정보이론 ..
Manifold ..
Variational inference 몰라요.
Inference 알고리즘 몰라요.
https://www.slideshare.net/HwanheeKim2/ndc2017-vae-75419284
http://kvfrans.com/variational-autoencoders-explained/
https://www.slideshare.net/ssuser06e0c5/variational-autoencoder-76552518
http://kvfrans.com/variational-autoencoders-explained/
이전 글에서 나는 머신러닝이 주어진 데이터를 가장 잘 설명하는 ‘함수’를 찾는 알고리즘을 디자인하는 것이라 설명했다.
그러나 머신러닝은 확률의 관점에서도 설명이 가능하다. 머신러닝을 probability density를 찾는 과정으로 생각하는 것
이다. 즉, 함수를 가정하는 것이 아니라 확률 분포를 가정하고, 적절한 확률 분포의 parameter를 유추하는 과정으로 생
각하는 것이다.
주어진 데이터가 gaussian distribution으로 drawn되었다고 가정하고, 데이터와 현상을 가장 잘 설명하는 mean과
covariance를 찾는 과정과 비슷한 것이라고 생각하면 된다. 즉, 앞에서 설명한 방식은 function parameter를 찾는
방식이라면, 이제는 probability density function parameter를 찾는 것으로 바뀌는 것이다.
http://sanghyukchun.github.io/58/
https://www.slideshare.net/MingukKang/vae-ppt
http://nolsigan.com/blog/what-is-variational-autoencoder/
https://www.slideshare.net/MingukKang/vae-ppt
http://nolsigan.com/blog/what-is-variational-autoencoder/
BAYES
http://nolsigan.com/blog/what-is-variational-autoencoder/
http://nolsigan.com/blog/what-is-variational-autoencoder/
모수는 p
https://en.wikipedia.org/wiki/Bernoulli_distribution
p(x|z)
Variational Inference for Machine Learning
Shakir Mohamed / Google Deepmind
http://nolsigan.com/blog/what-is-variational-autoencoder/
Posterior
Evidence
Likelihood Prior
Posterior
Evidence
Likelihood Prior
모든 가능한 Z에 대해,
p(x,z)와 p(z)를 구해야 함
각각은 observation(likelihood), 현상에 대한 사전정보 (prior), 주어진 데이터에 대한 현상의 확률 (posterior)을 의미한다.
http://nolsigan.com/blog/what-is-variational-autoencoder/
우리의	variational posterior q(z∣x) 가		
진짜	posterior p(z∣x) 를	얼마나	잘	근사하는지	어떻게	알	수	있을까?
확률과 정보량
확률에 대한 정보량
이산확률분포에 대한 평균정보량
Variational Inference / norman3.github.io
Variational Inference / norman3.github.io
P, Q의 평균정보량 P의 평균정보량
Variational Inference / norman3.github.io
하나의 이벤트 집합에서
P, Q 두개의 확률분포에 대한
평균정보량을
크로스 엔트로피로 함.
p, q의 크로스 엔트로피 p의 엔트로피
https://en.wikipedia.org/wiki/Cross_entropy
우리의	variational posterior q(z∣x) 가		
진짜	posterior p(z∣x) 를	얼마나	잘	근사하는지	어떻게	알	수	있을까?
근사추론의 방법
p(z|x)~q_lambda(z|x)
https://ratsgo.github.io/generative%20model/2017/12/19/vi/
http://nolsigan.com/blog/what-is-variational-autoencoder/
Tutorial on Variational Autoencoders / CARL DOERSCH 

arXiv:1606.05908v2
Variational Inference / David M. Blei
우리는 q와 그에 대한 lambda를 바꾸고 다루고 있는데,
Variational Inference / David M. Blei
http://nolsigan.com/blog/what-is-variational-autoencoder/
좋은 posterior 근사를 얻는 방법으로..
minibatch..
뉴럴넷
https://www.slideshare.net/MingukKang/vae-ppt
https://www.slideshare.net/MingukKang/vae-ppt
https://www.slideshare.net/MingukKang/vae-ppt
Tutorial on Variational Autoencoders / CARL DOERSCH 

arXiv:1606.05908v2
MLE 로 parameter를 추정하는데,
앞서본 것처럼 ELBO 최대화해서.. 추정
(b)를 계산할 때,
실제 코드에 사용하는 식
http://pyro.ai/examples/vae.html
Notes on Variational Autoencoders
Variational Inference for Machine Learning
Shakir Mohamed / Google Deepmind
Non-independent : Markovian
https://stackoverflow.com/questions/13058379/example-for-non-iid-data
https://www.quora.com/What-is-non-stationary-data?utm_medium=organic
E[𝑓]= Sigma( 𝑝(𝑥)𝑓(𝑥) )
E[𝑓]= Integral( 𝑝(𝑥)𝑓(𝑥)𝑑𝑥 )

More Related Content

What's hot

Divide and Conquer
Divide and ConquerDivide and Conquer
Divide and Conquer
Melaku Bayih Demessie
 
Stock Price Prediction PPT
Stock Price Prediction  PPTStock Price Prediction  PPT
Stock Price Prediction PPT
PrashantGanji4
 
Ensemble Learning and Random Forests
Ensemble Learning and Random ForestsEnsemble Learning and Random Forests
Ensemble Learning and Random Forests
CloudxLab
 
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
Simplilearn
 
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016
Taehoon Kim
 
딥 러닝 자연어 처리를 학습을 위한 파워포인트. (Deep Learning for Natural Language Processing)
딥 러닝 자연어 처리를 학습을 위한 파워포인트. (Deep Learning for Natural Language Processing)딥 러닝 자연어 처리를 학습을 위한 파워포인트. (Deep Learning for Natural Language Processing)
딥 러닝 자연어 처리를 학습을 위한 파워포인트. (Deep Learning for Natural Language Processing)
WON JOON YOO
 
Lstm
LstmLstm
Speaker Recognition using Gaussian Mixture Model
Speaker Recognition using Gaussian Mixture Model Speaker Recognition using Gaussian Mixture Model
Speaker Recognition using Gaussian Mixture Model
Saurab Dulal
 
SVM
SVMSVM
What is Machine Learning | Introduction to Machine Learning | Machine Learnin...
What is Machine Learning | Introduction to Machine Learning | Machine Learnin...What is Machine Learning | Introduction to Machine Learning | Machine Learnin...
What is Machine Learning | Introduction to Machine Learning | Machine Learnin...
Simplilearn
 
Stock Market Prediction using Machine Learning
Stock Market Prediction using Machine LearningStock Market Prediction using Machine Learning
Stock Market Prediction using Machine Learning
Aravind Balaji
 
Introduction to Machine learning with Python
Introduction to Machine learning with PythonIntroduction to Machine learning with Python
Introduction to Machine learning with Python
Chariza Pladin
 
Support Vector Machines
Support Vector MachinesSupport Vector Machines
Support Vector Machines
CloudxLab
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural Networks
Databricks
 
Machine Learning and Real-World Applications
Machine Learning and Real-World ApplicationsMachine Learning and Real-World Applications
Machine Learning and Real-World Applications
MachinePulse
 
project ppt.pdf
project ppt.pdfproject ppt.pdf
project ppt.pdf
sumayyabeegam1
 
Credit card fraud dection
Credit card fraud dectionCredit card fraud dection
Credit card fraud dection
Sonali Birajadar
 
Linear Regression Analysis | Linear Regression in Python | Machine Learning A...
Linear Regression Analysis | Linear Regression in Python | Machine Learning A...Linear Regression Analysis | Linear Regression in Python | Machine Learning A...
Linear Regression Analysis | Linear Regression in Python | Machine Learning A...
Simplilearn
 
Neural networks
Neural networksNeural networks
Neural networks
Basil John
 

What's hot (20)

Divide and Conquer
Divide and ConquerDivide and Conquer
Divide and Conquer
 
Stock Price Prediction PPT
Stock Price Prediction  PPTStock Price Prediction  PPT
Stock Price Prediction PPT
 
Ensemble Learning and Random Forests
Ensemble Learning and Random ForestsEnsemble Learning and Random Forests
Ensemble Learning and Random Forests
 
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
Support Vector Machine - How Support Vector Machine works | SVM in Machine Le...
 
Logic agent
Logic agentLogic agent
Logic agent
 
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016
딥러닝과 강화 학습으로 나보다 잘하는 쿠키런 AI 구현하기 DEVIEW 2016
 
딥 러닝 자연어 처리를 학습을 위한 파워포인트. (Deep Learning for Natural Language Processing)
딥 러닝 자연어 처리를 학습을 위한 파워포인트. (Deep Learning for Natural Language Processing)딥 러닝 자연어 처리를 학습을 위한 파워포인트. (Deep Learning for Natural Language Processing)
딥 러닝 자연어 처리를 학습을 위한 파워포인트. (Deep Learning for Natural Language Processing)
 
Lstm
LstmLstm
Lstm
 
Speaker Recognition using Gaussian Mixture Model
Speaker Recognition using Gaussian Mixture Model Speaker Recognition using Gaussian Mixture Model
Speaker Recognition using Gaussian Mixture Model
 
SVM
SVMSVM
SVM
 
What is Machine Learning | Introduction to Machine Learning | Machine Learnin...
What is Machine Learning | Introduction to Machine Learning | Machine Learnin...What is Machine Learning | Introduction to Machine Learning | Machine Learnin...
What is Machine Learning | Introduction to Machine Learning | Machine Learnin...
 
Stock Market Prediction using Machine Learning
Stock Market Prediction using Machine LearningStock Market Prediction using Machine Learning
Stock Market Prediction using Machine Learning
 
Introduction to Machine learning with Python
Introduction to Machine learning with PythonIntroduction to Machine learning with Python
Introduction to Machine learning with Python
 
Support Vector Machines
Support Vector MachinesSupport Vector Machines
Support Vector Machines
 
Training Neural Networks
Training Neural NetworksTraining Neural Networks
Training Neural Networks
 
Machine Learning and Real-World Applications
Machine Learning and Real-World ApplicationsMachine Learning and Real-World Applications
Machine Learning and Real-World Applications
 
project ppt.pdf
project ppt.pdfproject ppt.pdf
project ppt.pdf
 
Credit card fraud dection
Credit card fraud dectionCredit card fraud dection
Credit card fraud dection
 
Linear Regression Analysis | Linear Regression in Python | Machine Learning A...
Linear Regression Analysis | Linear Regression in Python | Machine Learning A...Linear Regression Analysis | Linear Regression in Python | Machine Learning A...
Linear Regression Analysis | Linear Regression in Python | Machine Learning A...
 
Neural networks
Neural networksNeural networks
Neural networks
 

More from 민석 김

pt
ptpt
off-policy methods with approximation
off-policy methods with approximationoff-policy methods with approximation
off-policy methods with approximation
민석 김
 
Multi armed bandit
Multi armed banditMulti armed bandit
Multi armed bandit
민석 김
 
NN and PDF
NN and PDFNN and PDF
NN and PDF
민석 김
 
Kanerva machine
Kanerva machineKanerva machine
Kanerva machine
민석 김
 
Shouting at gwanghwamun
Shouting at gwanghwamunShouting at gwanghwamun
Shouting at gwanghwamun
민석 김
 
ML 60'~80' new paradigm 1
ML 60'~80' new paradigm 1ML 60'~80' new paradigm 1
ML 60'~80' new paradigm 1
민석 김
 
Internet speed 인터넷 속도를 측정해보자
Internet speed 인터넷 속도를 측정해보자Internet speed 인터넷 속도를 측정해보자
Internet speed 인터넷 속도를 측정해보자
민석 김
 
벽 생성기 Wall generator
벽 생성기 Wall generator 벽 생성기 Wall generator
벽 생성기 Wall generator
민석 김
 
복소수와 오일러 공식
복소수와 오일러 공식복소수와 오일러 공식
복소수와 오일러 공식
민석 김
 
Bayesian nets 발표 3
Bayesian nets 발표 3Bayesian nets 발표 3
Bayesian nets 발표 3
민석 김
 
Bayesian nets 발표 1
Bayesian nets 발표 1Bayesian nets 발표 1
Bayesian nets 발표 1
민석 김
 
Bayesian nets 발표 2
Bayesian nets 발표 2Bayesian nets 발표 2
Bayesian nets 발표 2
민석 김
 
AI 인공지능이란 단어 읽기
AI 인공지능이란 단어 읽기AI 인공지능이란 단어 읽기
AI 인공지능이란 단어 읽기
민석 김
 
Hopfield network 처음부터 공부해보기
Hopfield network 처음부터 공부해보기Hopfield network 처음부터 공부해보기
Hopfield network 처음부터 공부해보기
민석 김
 

More from 민석 김 (15)

pt
ptpt
pt
 
off-policy methods with approximation
off-policy methods with approximationoff-policy methods with approximation
off-policy methods with approximation
 
Multi armed bandit
Multi armed banditMulti armed bandit
Multi armed bandit
 
NN and PDF
NN and PDFNN and PDF
NN and PDF
 
Kanerva machine
Kanerva machineKanerva machine
Kanerva machine
 
Shouting at gwanghwamun
Shouting at gwanghwamunShouting at gwanghwamun
Shouting at gwanghwamun
 
ML 60'~80' new paradigm 1
ML 60'~80' new paradigm 1ML 60'~80' new paradigm 1
ML 60'~80' new paradigm 1
 
Internet speed 인터넷 속도를 측정해보자
Internet speed 인터넷 속도를 측정해보자Internet speed 인터넷 속도를 측정해보자
Internet speed 인터넷 속도를 측정해보자
 
벽 생성기 Wall generator
벽 생성기 Wall generator 벽 생성기 Wall generator
벽 생성기 Wall generator
 
복소수와 오일러 공식
복소수와 오일러 공식복소수와 오일러 공식
복소수와 오일러 공식
 
Bayesian nets 발표 3
Bayesian nets 발표 3Bayesian nets 발표 3
Bayesian nets 발표 3
 
Bayesian nets 발표 1
Bayesian nets 발표 1Bayesian nets 발표 1
Bayesian nets 발표 1
 
Bayesian nets 발표 2
Bayesian nets 발표 2Bayesian nets 발표 2
Bayesian nets 발표 2
 
AI 인공지능이란 단어 읽기
AI 인공지능이란 단어 읽기AI 인공지능이란 단어 읽기
AI 인공지능이란 단어 읽기
 
Hopfield network 처음부터 공부해보기
Hopfield network 처음부터 공부해보기Hopfield network 처음부터 공부해보기
Hopfield network 처음부터 공부해보기
 

VAE 처음부터 알아보기