1. The document discusses energy-based models (EBMs) and how they can be applied to classifiers. It introduces noise contrastive estimation and flow contrastive estimation as methods to train EBMs.
2. One paper presented trains energy-based models using flow contrastive estimation by passing data through a flow-based generator. This allows implicit modeling with EBMs.
3. Another paper argues that classifiers can be viewed as joint energy-based models over inputs and outputs, and should be treated as such. It introduces a method to train classifiers as EBMs using contrastive divergence.
1. The document discusses energy-based models (EBMs) and how they can be applied to classifiers. It introduces noise contrastive estimation and flow contrastive estimation as methods to train EBMs.
2. One paper presented trains energy-based models using flow contrastive estimation by passing data through a flow-based generator. This allows implicit modeling with EBMs.
3. Another paper argues that classifiers can be viewed as joint energy-based models over inputs and outputs, and should be treated as such. It introduces a method to train classifiers as EBMs using contrastive divergence.
This document discusses generative adversarial networks (GANs) and their relationship to reinforcement learning. It begins with an introduction to GANs, explaining how they can generate images without explicitly defining a probability distribution by using an adversarial training process. The second half discusses how GANs are related to actor-critic models and inverse reinforcement learning in reinforcement learning. It explains how GANs can be viewed as training a generator to fool a discriminator, similar to how policies are trained in reinforcement learning.
This document discusses generative adversarial networks (GANs) and their relationship to reinforcement learning. It begins with an introduction to GANs, explaining how they can generate images without explicitly defining a probability distribution by using an adversarial training process. The second half discusses how GANs are related to actor-critic models and inverse reinforcement learning in reinforcement learning. It explains how GANs can be viewed as training a generator to fool a discriminator, similar to how policies are trained in reinforcement learning.
1. Adversarial Training
to avoid overfitting
NBME top#2 Solution and Discussion
https://www.kaggle.com/competitions/nbme-score-
clinical-patient-notes/discussion/323085
Feedback top#1で活用されたが,NBMEでは
“Although its CV score was quite higher than the one I selected above, both its public LB score and private LB score were lower.
It seems that my way of doing pseudo labeling was better. It may be that being quite new to these techniques I didn't tune
them correctly. I will try them in future competitions for sure.”
とあるので汎用性についてはさらなる実装と議論が必要か。
2. Adversarial Training
Inputs Perturbation
into “Local” worst-case
Weights Perturbation
into “Global” worst-case
Gradient-based Adversary Not gradient-based
Need Labels Not need Labels
FGM, SiFT
VAT, TRADES,
SMART
MART
AWP