This document summarizes two papers presented at NIPS 2018 on anomaly detection and out-of-distribution detection. The first paper proposes a simple unified framework using geometric transformations and Dirichlet density estimation to detect anomalies and adversarial examples. The second paper introduces a method that uses an ensemble of neural networks to detect out-of-distribution samples and adversarial attacks with state-of-the-art performance on CIFAR-10, SVHN and FGSM attacks. It also explores applications to class-incremental learning.

1. (NIPS 2018)

2. NIPS2018
n Deep Anomaly Detection Using Geometric Transformations
n deep learning
n A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks
n

3. :
n One-class classification:
( 1)
n Multi-class classification:
( 2)
( )
( )

4. :
n
n
n One Class SVM

5. :
n (PCA, Robust-PCA, deep autoencoders, ADGAN…)
n
n One Class SVM
L2

6. :
n
n (KDE, Robust-KDE, DSEBM…)
n One Class SVM

7. :
n
n
n One Class SVM (SVDD, Deep SVDD...)

8. : AUROC
n AUROC(area under an ROC curve)
http://www.randpy.tokyo/entry/roc_auc
AUROC

9. : Deep Anomaly Detection Using Geometric Transformations
n NIPS 2018 accepted
n : Izhak Golan, Ran El-Yaniv
n :
n ( flip )
n
n AUROC OC-SVM, DAGMM, DSEBM, ADGAN SOTA
!"($) !&($) !'($) !(($)
( )
( )

10. :
n
n
n
n or
n
λ

11. :
n step1:
n step2:
n step3:

12. : step1
n k
n identity transformation
n
n x
cross-entropy
deep k-class
!"
72(=2x3x3x4)

13. : step2 Dirichlet Normality Score
n softmax y(x) (Dirichlet ) α
n α
n x
x

14. : step3
n k f
n α y
n !"($)
!"($)

15. :
step1:
step2: Dirichlet
α
step3:

16. :
n One-Class SVM (OC-SVM)
n RAW-OC-SVM
n CAE-OC-SVM
n Deep One-Class Classification (E2E-OC-SVM)
n ICML2018
n OC-SVM(SVDD) deep
n Deep structured energy-based models (DSEBM)
n ICML 2016
n
n Deep Autoencoding Gaussian Mixture Model (DAGMM)
n ICLR 2018
n
n Anomaly Detection with a Generative Adversarial Network (ADGAN)
n AnoGAN(IPMI 2017)
n GAN

17. :
n CIFAR-10
n 10 6000 32x32
n CIFAR-100
n 100 600 32x32
n 20
n Fashion-MNIST
n 10 7000 28x28
n CatsVsDogs
n ASIRRA
n 2 12500 360x400 64x64

18. : CIFAR-10
SOTA

19. : CIFAR-100
SOTA

20. : Fashion-MNIST CatsVsDogs
SOTA

21. n
n
n GAN SOTA
n
n ( ?)
n

22. : A Simple Unified Framework for Detecting Out-of-
Distribution Samples and Adversarial Attacks
n NIPS 2018 accepted
n : Kimin Lee, Kibok Lee, Honglak Lee, Jinwoo Shin
n :
n
n adversarial attack SOTA

23. :
n step1: NN
n step2: NN
n step3:

24. : step1
n

25. : step2
n NN ( )
n

26. : step3
n
n
n input preprocessing:
n feature ensemble:
n L
n validation data

27. : Class-Incremental learning
n Class-Incremental learning
n
n NN
n
n OOD ≒
n

28. : ablation study
n
n CIFAR10: SVHN:
n :
n ODIN:
SOTA

29. : SOTA
OOD
FGSM

30. : SOTA
n
n
n FGSM
n
n kernel density (KD) + predictive uncertainty (PU):
n LID: kNN local intrinsic dimensionality (LID)

31. : Class-incremental learning
n
n 1: CIFAR-100
n 2: CIFAR-100 ImageNet
n
n Softmax:
n Euclidean:

32. n
n ( ) NN
n
n
n :
few-shot