SlideShare a Scribd company logo
1
Shohei Taniguchi, Matsuo Lab (M1)
• (EBM) (?)
• 2
- Flow Contrastive Estimation of Energy-Based Models
‣ 2
- Your Classifier is Secretly an Energy Based Model and You Should Treat it
Like One
‣
2
Outline
1. Energy Based Model (EBM)
- EBM
‣ Contrastive Divergence Learning (CD )
‣ Noice Contrastive Estimation ( )
2. EBM
- Restricted Boltzmann Machine (RBM)
3. Flow Contrastive Estimation of Energy-Based Models
4. Your Classifier is Secretly an Energy Based Model and You Should Treat it
Like One 3
Energy Based Model
4
EBM
•
-
x pθ (x) x
Eθ (x)
pθ (x) =
exp (−Eθ (x))
Z (θ) (
Z (θ) =
∫
exp (−Eθ (x)) dx
)
Z (θ)
5
EBM
• ( )
-
‣
‣ NCE
- (?)
•
- HMC
‣ MCMC
6
EBM
•
( )
-
➡
- EBM
log pθ (x)
Z (θ)
7
Contrastive Divergence Learning (CD )
• SGD
-
‣
‣ MCMC
∂ log pθ (x)
∂θ
∂ log pθ (x)
∂θ
= 𝔼pθ(x) [
∂Eθ (x)
∂θ ]
− 𝔼pdata(x) [
∂Eθ (x)
∂θ ]
pθ (x) 8
CD
•
- MCMC
- MCMC
➡
pθ (x)
9
Noise Contrastive Estimation (NCE, )
•
-
‣ ( )
‣
‣ GAN ( )
Z (θ) c
log pθ (x) = − Eθ (x) − c
θ
c Z (θ)
J (θ) = 𝔼pdata(x)
[
log
pθ(x)
pθ(x) + q(x)]
+ 𝔼q(x)
[
log
q(x)
pθ(x) + q(x) ]
q (x)
10
NCE
•
-
①
②
③
- ①, ② ③
‣
EBM
q (x)
q (x)
pdata (x)
11
EBM
12
EBM
•
• 2
- :
e.g. Autoencoder, Denoising AE
- EBM : 2
e.g. Restricted Boltzmann Machine, Deep Boltzmann Machine
13
Restricted Boltzmann Machine (RBM)
• 2
•
( 2 )
• CD
• Deep Boltzmann Machine
hi
P (hi = 1|v) = σ (v⊤
W:,i + bi)
E(v, h) = − b⊤
v − c⊤
h − v⊤
Wh
h
p (v) =
∑
i
p(v, h)
(
p(v, h) =
1
Z
exp(−E(v, h))
)
hi
14
RBM
RBM EBM
• RBM DBM EBM
ReLU
• VAE, GAN
•
•
➡
15
EBM
EBM (RBM )
• 2
•
EBM
• NN
(NN 1
)
•
16
E (v) = NN (v)
= w(n)
(
⋯φ (W(2)
φ (W(1)
v + b(1)
) + b(2)
))
+ b(n)
E (v, h(1)
, h(2)
, h(3)
)
= − v⊤
W(1)
h(1)
− h(1)⊤
W(2)
h(2)
− h(2)⊤
W(3)
h(3)
EBM
• Implicit Generation and Modeling with Energy-Based Models (NeurIPS
2019)
- EBM
- CD
-
-
17
32x32 Imagenet
Flow Contrastive Estimation of Energy-Based Models
18
•
- Ruiqi Gao, Erik Nijkamp, Diederik P. Kingma, Zhen Xu, Andrew M.
Dai, Ying Nian Wu
• NeurIPS 2019 Bayesian Deep Learning Workshop
• Kingma
• NCE EBM flow
• 19
(flow)
Noise Contrastive Estimation ( )
•
-
‣ ( )
‣
‣ GAN ( )
Z (θ) c
log pθ (x) = − Eθ (x) − c
θ
c Z (θ)
J (θ) = 𝔼pdata(x)
[
log
pθ(x)
pθ(x) + q(x)]
+ 𝔼q(x)
[
log
q(x)
pθ(x) + q(x) ]
q (x)
20
NCE ( )
•
-
①
②
③
- ①, ② ③
‣
EBM
q (x)
q (x)
pdata (x)
21
Flow Contrastive Estimation (FCE)
• flow
- flow
https://www.slideshare.net/DeepLearningJP2016/dlflowbased-deep-
generative-models
• flow
FCE NCE EBM
- EBM flow
q (x)
qα (x)
V(θ, α) = 𝔼pdata(x)
[
log
pθ(x)
pθ(x) + qα(x)]
+ 𝔼p(z)
[
log
qα (gα(z))
pθ (gα(z)) + qα (gα(z)) ]
22
FCE flow
• GAN
EBM
flow
• EBM flow
V(θ, α) = 𝔼pdata(x)
[
log
pθ(x)
pθ(x) + qα(x) ]
+ 𝔼p(z)
[
log
qα (gα(z))
pθ (gα(z)) + qα (gα(z)) ]
pθ(x)
pθ(x) + qα(x)
x
qα (gα(z))
pθ (gα(z)) + qα (gα(z))
gα(z)
23
= JSDV
• EBM EBM
flow
➡ GAN
• GAN EBM
Jensen-Shannon Divergence (JSD)V
JSD (qα∥pdata) = KL (pdata∥ (pdata + qα)/2) + KL (qα∥ (pdata + qα)/2)
24
FCE
• EBM flow
- flow
- EBM MCMC
➡
‣ EBM flow
25
1 2D
• 1
- Glow-MLE: Glow
- Glow-FCE: FCE Glow
- EBM-FCE: FCE EBM
• FCE EBM 1
26
1 2D
• EBM
• Glow FCE (trained)
FCE (rand)
27
2
FCE Glow
FID
28
FCE
• NCE EBM flow
Flow Contrastive Estimation (FCE)
• flow EBM
• flow GAN generator (JSD)
discriminator generator
GAN
• EBM
29
Your Classifier is Secretly an Energy Based Model and You
Should Treat it Like One
30
•
- Will Grathwohl, Kuan-Chieh Wang, Jörn-Henrik Jacobsen, David
Duvenaud, Mohammad Norouzi, Kevin Swersky
• ICLR 2020 accepted (8, 8, 6)
•
•
•
x y
p (y|x) p (x)
31
Joint Energy based Model (JEM)
• softmax
•
pθ(y|x) =
exp (fθ(x)[y])
∑y′
exp (fθ(x)[y′])
x y
pθ(x, y) =
exp (fθ(x)[y])
Z(θ)
, Z (θ) =
∫ ∑
y′
exp (fθ(x)[y′]) dx
32
Joint Energy based Model (JEM)
•
• 2
• 1
CD ( NCE )
log pθ(x, y) = log pθ(x) + log pθ(y|x)
x Eθ (x)
Eθ(x) = − LogSumExpy (fθ(x)[y]) = − log
∑
y
exp (fθ(x)[y])
33
JEM
•
- class-conditional
•
-
34
•
• class-conditional
35
CIFAR10
JEM
• CD EBM
-
- MCMC
‣
‣ FCE (?)
36
•
• RBM
• EBM FCE EBM
- JEM FCE
• NCE
• EBM (?)
37

More Related Content

What's hot

[DL輪読会]When Does Label Smoothing Help?
[DL輪読会]When Does Label Smoothing Help?[DL輪読会]When Does Label Smoothing Help?
[DL輪読会]When Does Label Smoothing Help?
Deep Learning JP
 
【DL輪読会】Flow Matching for Generative Modeling
【DL輪読会】Flow Matching for Generative Modeling【DL輪読会】Flow Matching for Generative Modeling
【DL輪読会】Flow Matching for Generative Modeling
Deep Learning JP
 
【DL輪読会】ViT + Self Supervised Learningまとめ
【DL輪読会】ViT + Self Supervised Learningまとめ【DL輪読会】ViT + Self Supervised Learningまとめ
【DL輪読会】ViT + Self Supervised Learningまとめ
Deep Learning JP
 
【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion Models
【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion Models【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion Models
【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion Models
Deep Learning JP
 
GAN(と強化学習との関係)
GAN(と強化学習との関係)GAN(と強化学習との関係)
GAN(と強化学習との関係)
Masahiro Suzuki
 
【メタサーベイ】Video Transformer
 【メタサーベイ】Video Transformer 【メタサーベイ】Video Transformer
【メタサーベイ】Video Transformer
cvpaper. challenge
 
SSII2022 [TS1] Transformerの最前線〜 畳込みニューラルネットワークの先へ 〜
SSII2022 [TS1] Transformerの最前線〜 畳込みニューラルネットワークの先へ 〜SSII2022 [TS1] Transformerの最前線〜 畳込みニューラルネットワークの先へ 〜
SSII2022 [TS1] Transformerの最前線〜 畳込みニューラルネットワークの先へ 〜
SSII
 
近年のHierarchical Vision Transformer
近年のHierarchical Vision Transformer近年のHierarchical Vision Transformer
近年のHierarchical Vision Transformer
Yusuke Uchida
 
「世界モデル」と関連研究について
「世界モデル」と関連研究について「世界モデル」と関連研究について
「世界モデル」と関連研究について
Masahiro Suzuki
 
[DL輪読会]Flow-based Deep Generative Models
[DL輪読会]Flow-based Deep Generative Models[DL輪読会]Flow-based Deep Generative Models
[DL輪読会]Flow-based Deep Generative Models
Deep Learning JP
 
[DL輪読会]相互情報量最大化による表現学習
[DL輪読会]相互情報量最大化による表現学習[DL輪読会]相互情報量最大化による表現学習
[DL輪読会]相互情報量最大化による表現学習
Deep Learning JP
 
【チュートリアル】コンピュータビジョンによる動画認識
【チュートリアル】コンピュータビジョンによる動画認識【チュートリアル】コンピュータビジョンによる動画認識
【チュートリアル】コンピュータビジョンによる動画認識
Hirokatsu Kataoka
 
[DL輪読会]NVAE: A Deep Hierarchical Variational Autoencoder
[DL輪読会]NVAE: A Deep Hierarchical Variational Autoencoder[DL輪読会]NVAE: A Deep Hierarchical Variational Autoencoder
[DL輪読会]NVAE: A Deep Hierarchical Variational Autoencoder
Deep Learning JP
 
【メタサーベイ】数式ドリブン教師あり学習
【メタサーベイ】数式ドリブン教師あり学習【メタサーベイ】数式ドリブン教師あり学習
【メタサーベイ】数式ドリブン教師あり学習
cvpaper. challenge
 
Transformerを多層にする際の勾配消失問題と解決法について
Transformerを多層にする際の勾配消失問題と解決法についてTransformerを多層にする際の勾配消失問題と解決法について
Transformerを多層にする際の勾配消失問題と解決法について
Sho Takase
 
【DL輪読会】Perceiver io a general architecture for structured inputs & outputs
【DL輪読会】Perceiver io  a general architecture for structured inputs & outputs 【DL輪読会】Perceiver io  a general architecture for structured inputs & outputs
【DL輪読会】Perceiver io a general architecture for structured inputs & outputs
Deep Learning JP
 
グラフィカルモデル入門
グラフィカルモデル入門グラフィカルモデル入門
グラフィカルモデル入門Kawamoto_Kazuhiko
 
[DL輪読会]Learning Transferable Visual Models From Natural Language Supervision
[DL輪読会]Learning Transferable Visual Models From Natural Language Supervision[DL輪読会]Learning Transferable Visual Models From Natural Language Supervision
[DL輪読会]Learning Transferable Visual Models From Natural Language Supervision
Deep Learning JP
 
SSII2021 [TS2] 深層強化学習 〜 強化学習の基礎から応用まで 〜
SSII2021 [TS2] 深層強化学習 〜 強化学習の基礎から応用まで 〜SSII2021 [TS2] 深層強化学習 〜 強化学習の基礎から応用まで 〜
SSII2021 [TS2] 深層強化学習 〜 強化学習の基礎から応用まで 〜
SSII
 
Transformer メタサーベイ
Transformer メタサーベイTransformer メタサーベイ
Transformer メタサーベイ
cvpaper. challenge
 

What's hot (20)

[DL輪読会]When Does Label Smoothing Help?
[DL輪読会]When Does Label Smoothing Help?[DL輪読会]When Does Label Smoothing Help?
[DL輪読会]When Does Label Smoothing Help?
 
【DL輪読会】Flow Matching for Generative Modeling
【DL輪読会】Flow Matching for Generative Modeling【DL輪読会】Flow Matching for Generative Modeling
【DL輪読会】Flow Matching for Generative Modeling
 
【DL輪読会】ViT + Self Supervised Learningまとめ
【DL輪読会】ViT + Self Supervised Learningまとめ【DL輪読会】ViT + Self Supervised Learningまとめ
【DL輪読会】ViT + Self Supervised Learningまとめ
 
【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion Models
【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion Models【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion Models
【DL輪読会】High-Resolution Image Synthesis with Latent Diffusion Models
 
GAN(と強化学習との関係)
GAN(と強化学習との関係)GAN(と強化学習との関係)
GAN(と強化学習との関係)
 
【メタサーベイ】Video Transformer
 【メタサーベイ】Video Transformer 【メタサーベイ】Video Transformer
【メタサーベイ】Video Transformer
 
SSII2022 [TS1] Transformerの最前線〜 畳込みニューラルネットワークの先へ 〜
SSII2022 [TS1] Transformerの最前線〜 畳込みニューラルネットワークの先へ 〜SSII2022 [TS1] Transformerの最前線〜 畳込みニューラルネットワークの先へ 〜
SSII2022 [TS1] Transformerの最前線〜 畳込みニューラルネットワークの先へ 〜
 
近年のHierarchical Vision Transformer
近年のHierarchical Vision Transformer近年のHierarchical Vision Transformer
近年のHierarchical Vision Transformer
 
「世界モデル」と関連研究について
「世界モデル」と関連研究について「世界モデル」と関連研究について
「世界モデル」と関連研究について
 
[DL輪読会]Flow-based Deep Generative Models
[DL輪読会]Flow-based Deep Generative Models[DL輪読会]Flow-based Deep Generative Models
[DL輪読会]Flow-based Deep Generative Models
 
[DL輪読会]相互情報量最大化による表現学習
[DL輪読会]相互情報量最大化による表現学習[DL輪読会]相互情報量最大化による表現学習
[DL輪読会]相互情報量最大化による表現学習
 
【チュートリアル】コンピュータビジョンによる動画認識
【チュートリアル】コンピュータビジョンによる動画認識【チュートリアル】コンピュータビジョンによる動画認識
【チュートリアル】コンピュータビジョンによる動画認識
 
[DL輪読会]NVAE: A Deep Hierarchical Variational Autoencoder
[DL輪読会]NVAE: A Deep Hierarchical Variational Autoencoder[DL輪読会]NVAE: A Deep Hierarchical Variational Autoencoder
[DL輪読会]NVAE: A Deep Hierarchical Variational Autoencoder
 
【メタサーベイ】数式ドリブン教師あり学習
【メタサーベイ】数式ドリブン教師あり学習【メタサーベイ】数式ドリブン教師あり学習
【メタサーベイ】数式ドリブン教師あり学習
 
Transformerを多層にする際の勾配消失問題と解決法について
Transformerを多層にする際の勾配消失問題と解決法についてTransformerを多層にする際の勾配消失問題と解決法について
Transformerを多層にする際の勾配消失問題と解決法について
 
【DL輪読会】Perceiver io a general architecture for structured inputs & outputs
【DL輪読会】Perceiver io  a general architecture for structured inputs & outputs 【DL輪読会】Perceiver io  a general architecture for structured inputs & outputs
【DL輪読会】Perceiver io a general architecture for structured inputs & outputs
 
グラフィカルモデル入門
グラフィカルモデル入門グラフィカルモデル入門
グラフィカルモデル入門
 
[DL輪読会]Learning Transferable Visual Models From Natural Language Supervision
[DL輪読会]Learning Transferable Visual Models From Natural Language Supervision[DL輪読会]Learning Transferable Visual Models From Natural Language Supervision
[DL輪読会]Learning Transferable Visual Models From Natural Language Supervision
 
SSII2021 [TS2] 深層強化学習 〜 強化学習の基礎から応用まで 〜
SSII2021 [TS2] 深層強化学習 〜 強化学習の基礎から応用まで 〜SSII2021 [TS2] 深層強化学習 〜 強化学習の基礎から応用まで 〜
SSII2021 [TS2] 深層強化学習 〜 強化学習の基礎から応用まで 〜
 
Transformer メタサーベイ
Transformer メタサーベイTransformer メタサーベイ
Transformer メタサーベイ
 

Similar to [DL輪読会]近年のエネルギーベースモデルの進展

Phase diagram at finite T & Mu in strong coupling limit of lattice QCD
Phase diagram at finite T & Mu in strong coupling limit of lattice QCDPhase diagram at finite T & Mu in strong coupling limit of lattice QCD
Phase diagram at finite T & Mu in strong coupling limit of lattice QCD
Benjamin Jaedon Choi
 
Modeling the Dynamics of SGD by Stochastic Differential Equation
Modeling the Dynamics of SGD by Stochastic Differential EquationModeling the Dynamics of SGD by Stochastic Differential Equation
Modeling the Dynamics of SGD by Stochastic Differential Equation
Mark Chang
 
【DL輪読会】Unbiased Gradient Estimation for Marginal Log-likelihood
【DL輪読会】Unbiased Gradient Estimation for Marginal Log-likelihood【DL輪読会】Unbiased Gradient Estimation for Marginal Log-likelihood
【DL輪読会】Unbiased Gradient Estimation for Marginal Log-likelihood
Deep Learning JP
 
Stochastic Alternating Direction Method of Multipliers
Stochastic Alternating Direction Method of MultipliersStochastic Alternating Direction Method of Multipliers
Stochastic Alternating Direction Method of Multipliers
Taiji Suzuki
 
Using blurred images to assess damage in bridge structures?
Using blurred images to assess damage in bridge structures?Using blurred images to assess damage in bridge structures?
Using blurred images to assess damage in bridge structures?
Alessandro Palmeri
 
RF Module Design - [Chapter 1] From Basics to RF Transceivers
RF Module Design - [Chapter 1] From Basics to RF TransceiversRF Module Design - [Chapter 1] From Basics to RF Transceivers
RF Module Design - [Chapter 1] From Basics to RF Transceivers
Simen Li
 
Multiband Transceivers - [Chapter 1]
Multiband Transceivers - [Chapter 1] Multiband Transceivers - [Chapter 1]
Multiband Transceivers - [Chapter 1]
Simen Li
 
Modeling the Dynamics of SGD by Stochastic Differential Equation
Modeling the Dynamics of SGD by Stochastic Differential EquationModeling the Dynamics of SGD by Stochastic Differential Equation
Modeling the Dynamics of SGD by Stochastic Differential Equation
Mark Chang
 
Pseudo Random Number Generators
Pseudo Random Number GeneratorsPseudo Random Number Generators
Pseudo Random Number Generators
Darshini Parikh
 
射頻電子 - [第一章] 知識回顧與通訊系統簡介
射頻電子 - [第一章] 知識回顧與通訊系統簡介射頻電子 - [第一章] 知識回顧與通訊系統簡介
射頻電子 - [第一章] 知識回顧與通訊系統簡介
Simen Li
 
Control as Inference (強化学習とベイズ統計)
Control as Inference (強化学習とベイズ統計)Control as Inference (強化学習とベイズ統計)
Control as Inference (強化学習とベイズ統計)
Shohei Taniguchi
 
Formations Near The Libration Points: Design Strategies Using Natural And Non...
Formations Near The Libration Points: Design Strategies Using Natural And Non...Formations Near The Libration Points: Design Strategies Using Natural And Non...
Formations Near The Libration Points: Design Strategies Using Natural And Non...
Belinda Marchand
 
Improving EV Lateral Dynamics Control Using Infinity Norm Approach with Close...
Improving EV Lateral Dynamics Control Using Infinity Norm Approach with Close...Improving EV Lateral Dynamics Control Using Infinity Norm Approach with Close...
Improving EV Lateral Dynamics Control Using Infinity Norm Approach with Close...
Valerio Salvucci
 
Doubly Accelerated Stochastic Variance Reduced Gradient Methods for Regulariz...
Doubly Accelerated Stochastic Variance Reduced Gradient Methods for Regulariz...Doubly Accelerated Stochastic Variance Reduced Gradient Methods for Regulariz...
Doubly Accelerated Stochastic Variance Reduced Gradient Methods for Regulariz...
Tomoya Murata
 
Ph ddefence
Ph ddefencePh ddefence
A Note on the Derivation of the Variational Inference Updates for DILN
A Note on the Derivation of the Variational Inference Updates for DILNA Note on the Derivation of the Variational Inference Updates for DILN
A Note on the Derivation of the Variational Inference Updates for DILN
Tomonari Masada
 
digital logic design Chapter 2 boolean_algebra_&_logic_gates
digital logic design Chapter 2 boolean_algebra_&_logic_gatesdigital logic design Chapter 2 boolean_algebra_&_logic_gates
digital logic design Chapter 2 boolean_algebra_&_logic_gates
Imran Waris
 
グラム行列のスパース近似を用いた生成的モーメントマッチングネットに基づく音声合成の検討
グラム行列のスパース近似を用いた生成的モーメントマッチングネットに基づく音声合成の検討グラム行列のスパース近似を用いた生成的モーメントマッチングネットに基づく音声合成の検討
グラム行列のスパース近似を用いた生成的モーメントマッチングネットに基づく音声合成の検討
Tomoki Koriyama
 
Introduction To Lisp
Introduction To LispIntroduction To Lisp
Introduction To Lisp
kyleburton
 
Agilent ADS 模擬手冊 [實習2] 放大器設計
Agilent ADS 模擬手冊 [實習2]  放大器設計Agilent ADS 模擬手冊 [實習2]  放大器設計
Agilent ADS 模擬手冊 [實習2] 放大器設計
Simen Li
 

Similar to [DL輪読会]近年のエネルギーベースモデルの進展 (20)

Phase diagram at finite T & Mu in strong coupling limit of lattice QCD
Phase diagram at finite T & Mu in strong coupling limit of lattice QCDPhase diagram at finite T & Mu in strong coupling limit of lattice QCD
Phase diagram at finite T & Mu in strong coupling limit of lattice QCD
 
Modeling the Dynamics of SGD by Stochastic Differential Equation
Modeling the Dynamics of SGD by Stochastic Differential EquationModeling the Dynamics of SGD by Stochastic Differential Equation
Modeling the Dynamics of SGD by Stochastic Differential Equation
 
【DL輪読会】Unbiased Gradient Estimation for Marginal Log-likelihood
【DL輪読会】Unbiased Gradient Estimation for Marginal Log-likelihood【DL輪読会】Unbiased Gradient Estimation for Marginal Log-likelihood
【DL輪読会】Unbiased Gradient Estimation for Marginal Log-likelihood
 
Stochastic Alternating Direction Method of Multipliers
Stochastic Alternating Direction Method of MultipliersStochastic Alternating Direction Method of Multipliers
Stochastic Alternating Direction Method of Multipliers
 
Using blurred images to assess damage in bridge structures?
Using blurred images to assess damage in bridge structures?Using blurred images to assess damage in bridge structures?
Using blurred images to assess damage in bridge structures?
 
RF Module Design - [Chapter 1] From Basics to RF Transceivers
RF Module Design - [Chapter 1] From Basics to RF TransceiversRF Module Design - [Chapter 1] From Basics to RF Transceivers
RF Module Design - [Chapter 1] From Basics to RF Transceivers
 
Multiband Transceivers - [Chapter 1]
Multiband Transceivers - [Chapter 1] Multiband Transceivers - [Chapter 1]
Multiband Transceivers - [Chapter 1]
 
Modeling the Dynamics of SGD by Stochastic Differential Equation
Modeling the Dynamics of SGD by Stochastic Differential EquationModeling the Dynamics of SGD by Stochastic Differential Equation
Modeling the Dynamics of SGD by Stochastic Differential Equation
 
Pseudo Random Number Generators
Pseudo Random Number GeneratorsPseudo Random Number Generators
Pseudo Random Number Generators
 
射頻電子 - [第一章] 知識回顧與通訊系統簡介
射頻電子 - [第一章] 知識回顧與通訊系統簡介射頻電子 - [第一章] 知識回顧與通訊系統簡介
射頻電子 - [第一章] 知識回顧與通訊系統簡介
 
Control as Inference (強化学習とベイズ統計)
Control as Inference (強化学習とベイズ統計)Control as Inference (強化学習とベイズ統計)
Control as Inference (強化学習とベイズ統計)
 
Formations Near The Libration Points: Design Strategies Using Natural And Non...
Formations Near The Libration Points: Design Strategies Using Natural And Non...Formations Near The Libration Points: Design Strategies Using Natural And Non...
Formations Near The Libration Points: Design Strategies Using Natural And Non...
 
Improving EV Lateral Dynamics Control Using Infinity Norm Approach with Close...
Improving EV Lateral Dynamics Control Using Infinity Norm Approach with Close...Improving EV Lateral Dynamics Control Using Infinity Norm Approach with Close...
Improving EV Lateral Dynamics Control Using Infinity Norm Approach with Close...
 
Doubly Accelerated Stochastic Variance Reduced Gradient Methods for Regulariz...
Doubly Accelerated Stochastic Variance Reduced Gradient Methods for Regulariz...Doubly Accelerated Stochastic Variance Reduced Gradient Methods for Regulariz...
Doubly Accelerated Stochastic Variance Reduced Gradient Methods for Regulariz...
 
Ph ddefence
Ph ddefencePh ddefence
Ph ddefence
 
A Note on the Derivation of the Variational Inference Updates for DILN
A Note on the Derivation of the Variational Inference Updates for DILNA Note on the Derivation of the Variational Inference Updates for DILN
A Note on the Derivation of the Variational Inference Updates for DILN
 
digital logic design Chapter 2 boolean_algebra_&_logic_gates
digital logic design Chapter 2 boolean_algebra_&_logic_gatesdigital logic design Chapter 2 boolean_algebra_&_logic_gates
digital logic design Chapter 2 boolean_algebra_&_logic_gates
 
グラム行列のスパース近似を用いた生成的モーメントマッチングネットに基づく音声合成の検討
グラム行列のスパース近似を用いた生成的モーメントマッチングネットに基づく音声合成の検討グラム行列のスパース近似を用いた生成的モーメントマッチングネットに基づく音声合成の検討
グラム行列のスパース近似を用いた生成的モーメントマッチングネットに基づく音声合成の検討
 
Introduction To Lisp
Introduction To LispIntroduction To Lisp
Introduction To Lisp
 
Agilent ADS 模擬手冊 [實習2] 放大器設計
Agilent ADS 模擬手冊 [實習2]  放大器設計Agilent ADS 模擬手冊 [實習2]  放大器設計
Agilent ADS 模擬手冊 [實習2] 放大器設計
 

More from Deep Learning JP

【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
Deep Learning JP
 
【DL輪読会】事前学習用データセットについて
【DL輪読会】事前学習用データセットについて【DL輪読会】事前学習用データセットについて
【DL輪読会】事前学習用データセットについて
Deep Learning JP
 
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
Deep Learning JP
 
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
Deep Learning JP
 
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
Deep Learning JP
 
【DL輪読会】マルチモーダル LLM
【DL輪読会】マルチモーダル LLM【DL輪読会】マルチモーダル LLM
【DL輪読会】マルチモーダル LLM
Deep Learning JP
 
【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
 【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo... 【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
Deep Learning JP
 
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
Deep Learning JP
 
【DL輪読会】Can Neural Network Memorization Be Localized?
【DL輪読会】Can Neural Network Memorization Be Localized?【DL輪読会】Can Neural Network Memorization Be Localized?
【DL輪読会】Can Neural Network Memorization Be Localized?
Deep Learning JP
 
【DL輪読会】Hopfield network 関連研究について
【DL輪読会】Hopfield network 関連研究について【DL輪読会】Hopfield network 関連研究について
【DL輪読会】Hopfield network 関連研究について
Deep Learning JP
 
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
Deep Learning JP
 
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
Deep Learning JP
 
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
Deep Learning JP
 
【DL輪読会】"Language Instructed Reinforcement Learning for Human-AI Coordination "
【DL輪読会】"Language Instructed Reinforcement Learning  for Human-AI Coordination "【DL輪読会】"Language Instructed Reinforcement Learning  for Human-AI Coordination "
【DL輪読会】"Language Instructed Reinforcement Learning for Human-AI Coordination "
Deep Learning JP
 
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
Deep Learning JP
 
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
Deep Learning JP
 
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
Deep Learning JP
 
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
Deep Learning JP
 
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
Deep Learning JP
 
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
Deep Learning JP
 

More from Deep Learning JP (20)

【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
 
【DL輪読会】事前学習用データセットについて
【DL輪読会】事前学習用データセットについて【DL輪読会】事前学習用データセットについて
【DL輪読会】事前学習用データセットについて
 
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
 
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
 
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
 
【DL輪読会】マルチモーダル LLM
【DL輪読会】マルチモーダル LLM【DL輪読会】マルチモーダル LLM
【DL輪読会】マルチモーダル LLM
 
【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
 【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo... 【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
 
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
 
【DL輪読会】Can Neural Network Memorization Be Localized?
【DL輪読会】Can Neural Network Memorization Be Localized?【DL輪読会】Can Neural Network Memorization Be Localized?
【DL輪読会】Can Neural Network Memorization Be Localized?
 
【DL輪読会】Hopfield network 関連研究について
【DL輪読会】Hopfield network 関連研究について【DL輪読会】Hopfield network 関連研究について
【DL輪読会】Hopfield network 関連研究について
 
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
 
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
 
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
 
【DL輪読会】"Language Instructed Reinforcement Learning for Human-AI Coordination "
【DL輪読会】"Language Instructed Reinforcement Learning  for Human-AI Coordination "【DL輪読会】"Language Instructed Reinforcement Learning  for Human-AI Coordination "
【DL輪読会】"Language Instructed Reinforcement Learning for Human-AI Coordination "
 
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
 
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
 
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
 
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
 
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
 
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
 

Recently uploaded

RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
KAMESHS29
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofszkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
Alex Pruden
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
sonjaschweigert1
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
TIPNGVN2
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
GridMate - End to end testing is a critical piece to ensure quality and avoid...
GridMate - End to end testing is a critical piece to ensure quality and avoid...GridMate - End to end testing is a critical piece to ensure quality and avoid...
GridMate - End to end testing is a critical piece to ensure quality and avoid...
ThomasParaiso2
 

Recently uploaded (20)

RESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for studentsRESUME BUILDER APPLICATION Project for students
RESUME BUILDER APPLICATION Project for students
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofszkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
Data structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdfData structures and Algorithms in Python.pdf
Data structures and Algorithms in Python.pdf
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
GridMate - End to end testing is a critical piece to ensure quality and avoid...
GridMate - End to end testing is a critical piece to ensure quality and avoid...GridMate - End to end testing is a critical piece to ensure quality and avoid...
GridMate - End to end testing is a critical piece to ensure quality and avoid...
 

[DL輪読会]近年のエネルギーベースモデルの進展

  • 2. • (EBM) (?) • 2 - Flow Contrastive Estimation of Energy-Based Models ‣ 2 - Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One ‣ 2
  • 3. Outline 1. Energy Based Model (EBM) - EBM ‣ Contrastive Divergence Learning (CD ) ‣ Noice Contrastive Estimation ( ) 2. EBM - Restricted Boltzmann Machine (RBM) 3. Flow Contrastive Estimation of Energy-Based Models 4. Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One 3
  • 5. EBM • - x pθ (x) x Eθ (x) pθ (x) = exp (−Eθ (x)) Z (θ) ( Z (θ) = ∫ exp (−Eθ (x)) dx ) Z (θ) 5
  • 6. EBM • ( ) - ‣ ‣ NCE - (?) • - HMC ‣ MCMC 6
  • 7. EBM • ( ) - ➡ - EBM log pθ (x) Z (θ) 7
  • 8. Contrastive Divergence Learning (CD ) • SGD - ‣ ‣ MCMC ∂ log pθ (x) ∂θ ∂ log pθ (x) ∂θ = 𝔼pθ(x) [ ∂Eθ (x) ∂θ ] − 𝔼pdata(x) [ ∂Eθ (x) ∂θ ] pθ (x) 8
  • 10. Noise Contrastive Estimation (NCE, ) • - ‣ ( ) ‣ ‣ GAN ( ) Z (θ) c log pθ (x) = − Eθ (x) − c θ c Z (θ) J (θ) = 𝔼pdata(x) [ log pθ(x) pθ(x) + q(x)] + 𝔼q(x) [ log q(x) pθ(x) + q(x) ] q (x) 10
  • 11. NCE • - ① ② ③ - ①, ② ③ ‣ EBM q (x) q (x) pdata (x) 11
  • 13. EBM • • 2 - : e.g. Autoencoder, Denoising AE - EBM : 2 e.g. Restricted Boltzmann Machine, Deep Boltzmann Machine 13
  • 14. Restricted Boltzmann Machine (RBM) • 2 • ( 2 ) • CD • Deep Boltzmann Machine hi P (hi = 1|v) = σ (v⊤ W:,i + bi) E(v, h) = − b⊤ v − c⊤ h − v⊤ Wh h p (v) = ∑ i p(v, h) ( p(v, h) = 1 Z exp(−E(v, h)) ) hi 14 RBM
  • 15. RBM EBM • RBM DBM EBM ReLU • VAE, GAN • • ➡ 15
  • 16. EBM EBM (RBM ) • 2 • EBM • NN (NN 1 ) • 16 E (v) = NN (v) = w(n) ( ⋯φ (W(2) φ (W(1) v + b(1) ) + b(2) )) + b(n) E (v, h(1) , h(2) , h(3) ) = − v⊤ W(1) h(1) − h(1)⊤ W(2) h(2) − h(2)⊤ W(3) h(3)
  • 17. EBM • Implicit Generation and Modeling with Energy-Based Models (NeurIPS 2019) - EBM - CD - - 17 32x32 Imagenet
  • 18. Flow Contrastive Estimation of Energy-Based Models 18
  • 19. • - Ruiqi Gao, Erik Nijkamp, Diederik P. Kingma, Zhen Xu, Andrew M. Dai, Ying Nian Wu • NeurIPS 2019 Bayesian Deep Learning Workshop • Kingma • NCE EBM flow • 19 (flow)
  • 20. Noise Contrastive Estimation ( ) • - ‣ ( ) ‣ ‣ GAN ( ) Z (θ) c log pθ (x) = − Eθ (x) − c θ c Z (θ) J (θ) = 𝔼pdata(x) [ log pθ(x) pθ(x) + q(x)] + 𝔼q(x) [ log q(x) pθ(x) + q(x) ] q (x) 20
  • 21. NCE ( ) • - ① ② ③ - ①, ② ③ ‣ EBM q (x) q (x) pdata (x) 21
  • 22. Flow Contrastive Estimation (FCE) • flow - flow https://www.slideshare.net/DeepLearningJP2016/dlflowbased-deep- generative-models • flow FCE NCE EBM - EBM flow q (x) qα (x) V(θ, α) = 𝔼pdata(x) [ log pθ(x) pθ(x) + qα(x)] + 𝔼p(z) [ log qα (gα(z)) pθ (gα(z)) + qα (gα(z)) ] 22
  • 23. FCE flow • GAN EBM flow • EBM flow V(θ, α) = 𝔼pdata(x) [ log pθ(x) pθ(x) + qα(x) ] + 𝔼p(z) [ log qα (gα(z)) pθ (gα(z)) + qα (gα(z)) ] pθ(x) pθ(x) + qα(x) x qα (gα(z)) pθ (gα(z)) + qα (gα(z)) gα(z) 23
  • 24. = JSDV • EBM EBM flow ➡ GAN • GAN EBM Jensen-Shannon Divergence (JSD)V JSD (qα∥pdata) = KL (pdata∥ (pdata + qα)/2) + KL (qα∥ (pdata + qα)/2) 24
  • 25. FCE • EBM flow - flow - EBM MCMC ➡ ‣ EBM flow 25
  • 26. 1 2D • 1 - Glow-MLE: Glow - Glow-FCE: FCE Glow - EBM-FCE: FCE EBM • FCE EBM 1 26
  • 27. 1 2D • EBM • Glow FCE (trained) FCE (rand) 27
  • 29. FCE • NCE EBM flow Flow Contrastive Estimation (FCE) • flow EBM • flow GAN generator (JSD) discriminator generator GAN • EBM 29
  • 30. Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One 30
  • 31. • - Will Grathwohl, Kuan-Chieh Wang, Jörn-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, Kevin Swersky • ICLR 2020 accepted (8, 8, 6) • • • x y p (y|x) p (x) 31
  • 32. Joint Energy based Model (JEM) • softmax • pθ(y|x) = exp (fθ(x)[y]) ∑y′ exp (fθ(x)[y′]) x y pθ(x, y) = exp (fθ(x)[y]) Z(θ) , Z (θ) = ∫ ∑ y′ exp (fθ(x)[y′]) dx 32
  • 33. Joint Energy based Model (JEM) • • 2 • 1 CD ( NCE ) log pθ(x, y) = log pθ(x) + log pθ(y|x) x Eθ (x) Eθ(x) = − LogSumExpy (fθ(x)[y]) = − log ∑ y exp (fθ(x)[y]) 33
  • 36. JEM • CD EBM - - MCMC ‣ ‣ FCE (?) 36
  • 37. • • RBM • EBM FCE EBM - JEM FCE • NCE • EBM (?) 37