SlideShare a Scribd company logo
1 of 25
딥러닝 논문 읽기모임
이미지처리팀
Adversarial Robustness through
Local Linearization
Authors : Chongli Qin*, James Martens, Sven Gowal, Dilip Krishnan,
Alhussein Fawzi, Soham De, Robert Stanforth, Pushmeet Kohli,
Deepmind
Krishnamurthy (Dj) Dvijotham
Google
Dec/20/2020
NeurIPS 2019
김병현 안종식 홍은기 허다은
딥러닝
논문
읽기모임
What is Adversarial Attack?
2
 Intriguing properties of neural networks (Szegedy et al., 2014)
Neural networks are velnurable to visually imperceptible adversarial perturbations
Classified as
Ostrich,
Speaker
Mantis
Dog
Correctly
Predicted
Difference
Btwn Left/Right
딥러닝
논문
읽기모임
What is Adversarial Attack?
3
 Intriguing properties of neural networks (Szegedy et al., 2014)
Neural networks are velnurable to visually imperceptible adversarial perturbations
Classified as
Ostrich,
Speaker
Mantis
Dog
Correctly
Predicted
Difference
Btwn Left/Right
딥러닝
논문
읽기모임
Difficulties in Adversarial Training
 Previous Works on Adversarial Training
Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks
Thermometer encoding: One hot way to resist adversarial examples
Stochastic Activation Pruning for Robust Adversarial Defense
Leveraging generative models to understand and defend against adversarial examples.
PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial
Examples
Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
Robustness via curvature regularization, and vice versa
 Growing model complexity and high input dimensionality
 make adversarial training prohibitive
4
VGG ResNet
More complex Higher dimension
MNIST
(28×28)
ImageNet
(256×256 )
딥러닝
논문
읽기모임
Difficulties in Adversarial Training
 Multiple iterations for computing adversarial perturbations
5
Define Model
Training Data Load
for i in range(training step):
for j in range(perturbation step):
train adversarial perturbation
(requires too much training cost)
training model against perturbations
Psuedo Code for Adversairal Trainining i.e. Non-linear loss surface
 Requires large step
to find an adversarial perturbation
Gradient Obfuscation
(Broken Gradient)
If perturbation steps are reduced,
A trained model becomes less robust
딥러닝
논문
읽기모임
How to Prevent Gradient Obfuscation
 Local Linearity Regularizer (LLR)
Make Loss Surface Linear around Training Example
• Much more likely to avoid Gradient Obfuscation
 Requires small number of steps
6
w/o LLR with LLR
딥러닝
논문
읽기모임
Contribution
 We show that training with LLR is significantly faster than adversarial
training, allowing us to train a robust ImageNet model with a 5× speed up when
training on 128 TPUv3 cores.
 We show that LLR trained models exhibit higher robustness relative to
adversarially trained models when evaluated under strong attacks. Adversarially trained
models can exhibit a decrease in accuracy of 6% when increasing the attack strength
at test time for CIFAR-10, whereas LLR shows only a decrease of 2%.
 We achieve new state of the art results for adversarial accuracy against
untargeted white-box attack for ImageNet (with = 4/2552 ): 47%. Furthermore, we
match state of the art results for CIFAR 10 (with = 8/255): 52.81%.
 We perform a large scale evaluation of existing methods for adversarially robust
training under consistent, strong, white-box attacks. For this we recreate
several baseline models from the literature, training them both for CIFAR-10 and
ImageNet (where possible).
7
딥러닝
논문
읽기모임
Adversarial Traininig
 Classification Function (Model)
Notiation
𝒙 : input, 𝜽 : weigts, 𝑪 : logits for classes
i.e. Classification model with Softmax Output
 Adversarial Training
more robust to adversarial attack
•  Return Same Result
• Perturbations Set (where : 𝜖 : magnitude)
8
w/o attack under attack
딥러닝
논문
읽기모임
Adversarial Traininig
 General Model Training
Empirical Risk Minimization (ERM)
Standard Cross-Entropy Loss
 Adversarial Trainig
Inner Maximization using Projected Gradient Descent
9
Constraint :
딥러닝
논문
읽기모임
Adversarial Traininig
 General Model Training
Empirical Risk Minimization (ERM)
Standard Cross-Entropy Loss
 Adversarial Trainig
Inner Maximization using Projected Gradient Descent
10
Constraint :
Define Model
Training Data Load
for i in range(training step):
for j in range(perturbation step):
train adversarial perturbation
training model against perturbations
Important to achieve
fast/good(…) training
딥러닝
논문
읽기모임
Motivating the Local Linearity Regularizer
 Taylor Expansion
 Relatively Linear Loss Surface will be..
well-predicted by 1st order Talyor Expansion
11
1st order Taylor Expansion
a a
Error is
trivial
Error is
significant
Linear Loss Surface Non-Linear Loss Surfcae
딥러닝
논문
읽기모임
Motivating the Local Linearity Regularizer
 Local Linearity Measure
To measure local linearity of loss surface
12
True loss when
perturbation applied
1st Taylor Expansion
indicator of how linear the loss surface
Maximum among perturbation set
Local Linearity Measure
딥러닝
논문
읽기모임
Local Linearity Regularizer
 Empirical Observations on Adversarial Training
13
for i in range(training step):
for j in range(perturbation step):
train adversarial perturbation
training model against perturbations
딥러닝
논문
읽기모임
Local Linearity Upper Bounds Adversarial Loss
 Local Linearity Upper Bounds Adversarial Loss
14
Difference between
Loss under attack and w/o attack
1st Taylor expansion step
+ Local linearity measure
딥러닝
논문
읽기모임
Local Linearity Regularization (LLR)
 Local Linearity Regularization (LLR)
15
Upper bound equation
Loss under attack can be
replaced by gamma and
1st Taylor expansion term
Adversarial training
딥러닝
논문
읽기모임
Local Linearity Regularizer - Algorithm
16
딥러닝
논문
읽기모임
Experiments and Results
 Loss Functions for Adversarial Attack
1. Random-Targeted
• i.e. Cat-to-Dog (Randomly Selected, Not Change while training)
2. Untargeted
• i.e. Cat-to-Dog or Mantis … Speaker
(Highest Logit excluding true lable, Change while training)
3. Multi-targeted
• i.e. Cat-to-Dog and Mantis … Speaker
(Target All Class except true lable, Not Change while training)
Nominal : Trained with Perturbation but tested without attack
 Metric
Attack Success Rate : Does the model return targeted class?
Adversarial Accuracy : Accuracy under Adversarial Attack
17
1
2
3
딥러닝
논문
읽기모임
Experiments and Results
 CIFAR-10
18
Sign of Gradient :
One of perturbation optimizer
2 3
딥러닝
논문
읽기모임
Experiments and Results
 ImageNet
LLR showed notably higher score when radius is 4/255
LLR showed lower score when radius is 16/255
19
2 1
딥러닝
논문
읽기모임
Experiments and Results
 Accuracy Degradation
20
딥러닝
논문
읽기모임
Images created with perturbation radius 16/255
21
 The perturbations are VISIBLE when radius is 16/255
Normal Under Attack
Perturbations are visible
딥러닝
논문
읽기모임
Experiments and Results
 Resistance to Gradient Obfuscation
22
딥러닝
논문
읽기모임
Ablation Studies
23
딥러닝
논문
읽기모임
Discussions
 Gradient Obfuscation : Broken Gradient
Shattered Gradients
• nonexistent or incorrect gradient (non-differentiable)
Stochastic Gradients
• randomized defenses
Exploding & Vanishing Gradients
• feeding the output of one computation as the input of the next
Even though the authors mentioned the original paper found
gradient obfuscation, they did not deal with any defensive
experiments on attack methods proposed in the paper.
24
Athalye, Anish, Nicholas Carlini, and David Wagner. "Obfuscated gradients give a false sense of security:
Circumventing defenses to adversarial examples." arXiv preprint arXiv:1802.00420 (2018).
딥러닝
논문
읽기모임
Discussions
 The paper showed how LLR contributes to a robust
adversarial training by experimental results but the
theoretical explanation is weak.
 It seems that the notations in the paper do not include
full-information for re-implementation of the proposed
method.
i.e.
25
Athalye, Anish, Nicholas Carlini, and David Wagner. "Obfuscated gradients give a false sense of security:
Circumventing defenses to adversarial examples." arXiv preprint arXiv:1802.00420 (2018).

More Related Content

What's hot

GAN with Mathematics
GAN with MathematicsGAN with Mathematics
GAN with MathematicsHyeongmin Lee
 
You only look once: Unified, real-time object detection (UPC Reading Group)
You only look once: Unified, real-time object detection (UPC Reading Group)You only look once: Unified, real-time object detection (UPC Reading Group)
You only look once: Unified, real-time object detection (UPC Reading Group)Universitat Politècnica de Catalunya
 
Why Batch Normalization Works so Well
Why Batch Normalization Works so WellWhy Batch Normalization Works so Well
Why Batch Normalization Works so WellChun-Ming Chang
 
Overview of tree algorithms from decision tree to xgboost
Overview of tree algorithms from decision tree to xgboostOverview of tree algorithms from decision tree to xgboost
Overview of tree algorithms from decision tree to xgboostTakami Sato
 
画像認識と深層学習
画像認識と深層学習画像認識と深層学習
画像認識と深層学習Yusuke Uchida
 
SIGNATE オフロードコンペ 精度認識部門 3rd Place Solution
SIGNATE オフロードコンペ 精度認識部門 3rd Place SolutionSIGNATE オフロードコンペ 精度認識部門 3rd Place Solution
SIGNATE オフロードコンペ 精度認識部門 3rd Place SolutionYusuke Uchida
 
Tokyo r15 異常検知入門
Tokyo r15 異常検知入門Tokyo r15 異常検知入門
Tokyo r15 異常検知入門Yohei Sato
 
Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationDat Nguyen
 
PRML 2.3節 - ガウス分布
PRML 2.3節 - ガウス分布PRML 2.3節 - ガウス分布
PRML 2.3節 - ガウス分布Yuki Soma
 
FEPチュートリアル2021 講義3 「潜在変数が連続値、生成モデルが正規分布の場合」の改良版
FEPチュートリアル2021 講義3 「潜在変数が連続値、生成モデルが正規分布の場合」の改良版FEPチュートリアル2021 講義3 「潜在変数が連続値、生成モデルが正規分布の場合」の改良版
FEPチュートリアル2021 講義3 「潜在変数が連続値、生成モデルが正規分布の場合」の改良版Masatoshi Yoshida
 
A brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsA brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsShunta Saito
 
Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)Terry Taewoong Um
 
コンピュテーショナルフォトグラフティの基礎
コンピュテーショナルフォトグラフティの基礎コンピュテーショナルフォトグラフティの基礎
コンピュテーショナルフォトグラフティの基礎Norishige Fukushima
 
[DL輪読会]Learning an Embedding Space for Transferable Robot Skills
[DL輪読会]Learning an Embedding Space for Transferable Robot Skills[DL輪読会]Learning an Embedding Space for Transferable Robot Skills
[DL輪読会]Learning an Embedding Space for Transferable Robot SkillsDeep Learning JP
 
論文紹介 Amortized bayesian meta learning
論文紹介 Amortized bayesian meta learning論文紹介 Amortized bayesian meta learning
論文紹介 Amortized bayesian meta learningXiangze
 
You Only Look Once: Unified, Real-Time Object Detection
You Only Look Once: Unified, Real-Time Object DetectionYou Only Look Once: Unified, Real-Time Object Detection
You Only Look Once: Unified, Real-Time Object DetectionDADAJONJURAKUZIEV
 
【論文紹介】Deep Mimic: Example-Guided Deep Reinforcement Learning of Physics-Based...
【論文紹介】Deep Mimic: Example-Guided Deep Reinforcement Learning of Physics-Based...【論文紹介】Deep Mimic: Example-Guided Deep Reinforcement Learning of Physics-Based...
【論文紹介】Deep Mimic: Example-Guided Deep Reinforcement Learning of Physics-Based...Tomoyuki Hioki
 
レイトレ空間構造入門
レイトレ空間構造入門レイトレ空間構造入門
レイトレ空間構造入門Toru Matsuoka
 
[기초개념] Graph Convolutional Network (GCN)
[기초개념] Graph Convolutional Network (GCN)[기초개념] Graph Convolutional Network (GCN)
[기초개념] Graph Convolutional Network (GCN)Donghyeon Kim
 

What's hot (20)

GAN with Mathematics
GAN with MathematicsGAN with Mathematics
GAN with Mathematics
 
You only look once: Unified, real-time object detection (UPC Reading Group)
You only look once: Unified, real-time object detection (UPC Reading Group)You only look once: Unified, real-time object detection (UPC Reading Group)
You only look once: Unified, real-time object detection (UPC Reading Group)
 
Why Batch Normalization Works so Well
Why Batch Normalization Works so WellWhy Batch Normalization Works so Well
Why Batch Normalization Works so Well
 
Overview of tree algorithms from decision tree to xgboost
Overview of tree algorithms from decision tree to xgboostOverview of tree algorithms from decision tree to xgboost
Overview of tree algorithms from decision tree to xgboost
 
画像認識と深層学習
画像認識と深層学習画像認識と深層学習
画像認識と深層学習
 
SIGNATE オフロードコンペ 精度認識部門 3rd Place Solution
SIGNATE オフロードコンペ 精度認識部門 3rd Place SolutionSIGNATE オフロードコンペ 精度認識部門 3rd Place Solution
SIGNATE オフロードコンペ 精度認識部門 3rd Place Solution
 
Tokyo r15 異常検知入門
Tokyo r15 異常検知入門Tokyo r15 異常検知入門
Tokyo r15 異常検知入門
 
Mask-RCNN for Instance Segmentation
Mask-RCNN for Instance SegmentationMask-RCNN for Instance Segmentation
Mask-RCNN for Instance Segmentation
 
PRML 2.3節 - ガウス分布
PRML 2.3節 - ガウス分布PRML 2.3節 - ガウス分布
PRML 2.3節 - ガウス分布
 
FEPチュートリアル2021 講義3 「潜在変数が連続値、生成モデルが正規分布の場合」の改良版
FEPチュートリアル2021 講義3 「潜在変数が連続値、生成モデルが正規分布の場合」の改良版FEPチュートリアル2021 講義3 「潜在変数が連続値、生成モデルが正規分布の場合」の改良版
FEPチュートリアル2021 講義3 「潜在変数が連続値、生成モデルが正規分布の場合」の改良版
 
A brief introduction to recent segmentation methods
A brief introduction to recent segmentation methodsA brief introduction to recent segmentation methods
A brief introduction to recent segmentation methods
 
Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)Deformable Convolutional Network (2017)
Deformable Convolutional Network (2017)
 
コンピュテーショナルフォトグラフティの基礎
コンピュテーショナルフォトグラフティの基礎コンピュテーショナルフォトグラフティの基礎
コンピュテーショナルフォトグラフティの基礎
 
Tabnet presentation
Tabnet presentationTabnet presentation
Tabnet presentation
 
[DL輪読会]Learning an Embedding Space for Transferable Robot Skills
[DL輪読会]Learning an Embedding Space for Transferable Robot Skills[DL輪読会]Learning an Embedding Space for Transferable Robot Skills
[DL輪読会]Learning an Embedding Space for Transferable Robot Skills
 
論文紹介 Amortized bayesian meta learning
論文紹介 Amortized bayesian meta learning論文紹介 Amortized bayesian meta learning
論文紹介 Amortized bayesian meta learning
 
You Only Look Once: Unified, Real-Time Object Detection
You Only Look Once: Unified, Real-Time Object DetectionYou Only Look Once: Unified, Real-Time Object Detection
You Only Look Once: Unified, Real-Time Object Detection
 
【論文紹介】Deep Mimic: Example-Guided Deep Reinforcement Learning of Physics-Based...
【論文紹介】Deep Mimic: Example-Guided Deep Reinforcement Learning of Physics-Based...【論文紹介】Deep Mimic: Example-Guided Deep Reinforcement Learning of Physics-Based...
【論文紹介】Deep Mimic: Example-Guided Deep Reinforcement Learning of Physics-Based...
 
レイトレ空間構造入門
レイトレ空間構造入門レイトレ空間構造入門
レイトレ空間構造入門
 
[기초개념] Graph Convolutional Network (GCN)
[기초개념] Graph Convolutional Network (GCN)[기초개념] Graph Convolutional Network (GCN)
[기초개념] Graph Convolutional Network (GCN)
 

Similar to adversarial robustness through local linearization

On the Resilience of Deep Learning for reduced-voltage FPGAs
On the Resilience of Deep Learning for reduced-voltage FPGAsOn the Resilience of Deep Learning for reduced-voltage FPGAs
On the Resilience of Deep Learning for reduced-voltage FPGAsLEGATO project
 
Adversarial Training is all you Need.pptx
Adversarial Training is all you Need.pptxAdversarial Training is all you Need.pptx
Adversarial Training is all you Need.pptxPrerana Khatiwada
 
Adversarial Attacks and Defenses in Deep Learning.pdf
Adversarial Attacks and Defenses in Deep Learning.pdfAdversarial Attacks and Defenses in Deep Learning.pdf
Adversarial Attacks and Defenses in Deep Learning.pdfMichelleHoogenhout
 
Black-Box attacks against Neural Networks - technical project presentation
Black-Box attacks against Neural Networks - technical project presentationBlack-Box attacks against Neural Networks - technical project presentation
Black-Box attacks against Neural Networks - technical project presentationRoberto Falconi
 
Survey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning ModelsSurvey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning ModelsIRJET Journal
 
Iclr2020: Compression based bound for non-compressed network: unified general...
Iclr2020: Compression based bound for non-compressed network: unified general...Iclr2020: Compression based bound for non-compressed network: unified general...
Iclr2020: Compression based bound for non-compressed network: unified general...Taiji Suzuki
 
Robustness of compressed CNNs
Robustness of compressed CNNsRobustness of compressed CNNs
Robustness of compressed CNNsKaushalya Madhawa
 
Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)MeetupDataScienceRoma
 
NS-CUK Seminar :J.H.Lee, "Review on "Similarity Preserving Adversarial Graph ...
NS-CUK Seminar :J.H.Lee, "Review on "Similarity Preserving Adversarial Graph ...NS-CUK Seminar :J.H.Lee, "Review on "Similarity Preserving Adversarial Graph ...
NS-CUK Seminar :J.H.Lee, "Review on "Similarity Preserving Adversarial Graph ...ssuser4b1f48
 
phd-defense
phd-defensephd-defense
phd-defenseHan Xiao
 
Research of adversarial example on a deep neural network
Research of adversarial example on a deep neural networkResearch of adversarial example on a deep neural network
Research of adversarial example on a deep neural networkNAVER Engineering
 
Regression vs Deep Neural net vs SVM
Regression vs Deep Neural net vs SVMRegression vs Deep Neural net vs SVM
Regression vs Deep Neural net vs SVMRatul Alahy
 
Improving Hardware Efficiency for DNN Applications
Improving Hardware Efficiency for DNN ApplicationsImproving Hardware Efficiency for DNN Applications
Improving Hardware Efficiency for DNN ApplicationsChester Chen
 
Decoupling Direction and NormによるAdversarial Exampleの生成
Decoupling Direction and NormによるAdversarial Exampleの生成Decoupling Direction and NormによるAdversarial Exampleの生成
Decoupling Direction and NormによるAdversarial Exampleの生成Motokawa Tetsuya
 
Deceiving Autonomous Cars with Toxic Signs
Deceiving Autonomous Cars with Toxic SignsDeceiving Autonomous Cars with Toxic Signs
Deceiving Autonomous Cars with Toxic SignsLeonardoSalvucci1
 
Edge-based Discovery of Training Data for Machine Learning
Edge-based Discovery of Training Data for Machine LearningEdge-based Discovery of Training Data for Machine Learning
Edge-based Discovery of Training Data for Machine LearningZiqiang Feng
 
BOOSTING ADVERSARIAL ATTACKS WITH MOMENTUM - Tianyu Pang and Chao Du, THU - D...
BOOSTING ADVERSARIAL ATTACKS WITH MOMENTUM - Tianyu Pang and Chao Du, THU - D...BOOSTING ADVERSARIAL ATTACKS WITH MOMENTUM - Tianyu Pang and Chao Du, THU - D...
BOOSTING ADVERSARIAL ATTACKS WITH MOMENTUM - Tianyu Pang and Chao Du, THU - D...GeekPwn Keen
 

Similar to adversarial robustness through local linearization (20)

On the Resilience of Deep Learning for reduced-voltage FPGAs
On the Resilience of Deep Learning for reduced-voltage FPGAsOn the Resilience of Deep Learning for reduced-voltage FPGAs
On the Resilience of Deep Learning for reduced-voltage FPGAs
 
Adversarial Training is all you Need.pptx
Adversarial Training is all you Need.pptxAdversarial Training is all you Need.pptx
Adversarial Training is all you Need.pptx
 
Adversarial Attacks and Defenses in Deep Learning.pdf
Adversarial Attacks and Defenses in Deep Learning.pdfAdversarial Attacks and Defenses in Deep Learning.pdf
Adversarial Attacks and Defenses in Deep Learning.pdf
 
Black-Box attacks against Neural Networks - technical project presentation
Black-Box attacks against Neural Networks - technical project presentationBlack-Box attacks against Neural Networks - technical project presentation
Black-Box attacks against Neural Networks - technical project presentation
 
Survey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning ModelsSurvey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning Models
 
Iclr2020: Compression based bound for non-compressed network: unified general...
Iclr2020: Compression based bound for non-compressed network: unified general...Iclr2020: Compression based bound for non-compressed network: unified general...
Iclr2020: Compression based bound for non-compressed network: unified general...
 
Robustness of compressed CNNs
Robustness of compressed CNNsRobustness of compressed CNNs
Robustness of compressed CNNs
 
Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)
 
NS-CUK Seminar :J.H.Lee, "Review on "Similarity Preserving Adversarial Graph ...
NS-CUK Seminar :J.H.Lee, "Review on "Similarity Preserving Adversarial Graph ...NS-CUK Seminar :J.H.Lee, "Review on "Similarity Preserving Adversarial Graph ...
NS-CUK Seminar :J.H.Lee, "Review on "Similarity Preserving Adversarial Graph ...
 
Security of Machine Learning
Security of Machine LearningSecurity of Machine Learning
Security of Machine Learning
 
phd-defense
phd-defensephd-defense
phd-defense
 
Research of adversarial example on a deep neural network
Research of adversarial example on a deep neural networkResearch of adversarial example on a deep neural network
Research of adversarial example on a deep neural network
 
Regression vs Deep Neural net vs SVM
Regression vs Deep Neural net vs SVMRegression vs Deep Neural net vs SVM
Regression vs Deep Neural net vs SVM
 
Improving Hardware Efficiency for DNN Applications
Improving Hardware Efficiency for DNN ApplicationsImproving Hardware Efficiency for DNN Applications
Improving Hardware Efficiency for DNN Applications
 
Adversarial ml
Adversarial mlAdversarial ml
Adversarial ml
 
Decoupling Direction and NormによるAdversarial Exampleの生成
Decoupling Direction and NormによるAdversarial Exampleの生成Decoupling Direction and NormによるAdversarial Exampleの生成
Decoupling Direction and NormによるAdversarial Exampleの生成
 
2019 Triangle Machine Learning Day - Mitigating Evasion Attacks to Deep Neura...
2019 Triangle Machine Learning Day - Mitigating Evasion Attacks to Deep Neura...2019 Triangle Machine Learning Day - Mitigating Evasion Attacks to Deep Neura...
2019 Triangle Machine Learning Day - Mitigating Evasion Attacks to Deep Neura...
 
Deceiving Autonomous Cars with Toxic Signs
Deceiving Autonomous Cars with Toxic SignsDeceiving Autonomous Cars with Toxic Signs
Deceiving Autonomous Cars with Toxic Signs
 
Edge-based Discovery of Training Data for Machine Learning
Edge-based Discovery of Training Data for Machine LearningEdge-based Discovery of Training Data for Machine Learning
Edge-based Discovery of Training Data for Machine Learning
 
BOOSTING ADVERSARIAL ATTACKS WITH MOMENTUM - Tianyu Pang and Chao Du, THU - D...
BOOSTING ADVERSARIAL ATTACKS WITH MOMENTUM - Tianyu Pang and Chao Du, THU - D...BOOSTING ADVERSARIAL ATTACKS WITH MOMENTUM - Tianyu Pang and Chao Du, THU - D...
BOOSTING ADVERSARIAL ATTACKS WITH MOMENTUM - Tianyu Pang and Chao Du, THU - D...
 

More from taeseon ryu

OpineSum Entailment-based self-training for abstractive opinion summarization...
OpineSum Entailment-based self-training for abstractive opinion summarization...OpineSum Entailment-based self-training for abstractive opinion summarization...
OpineSum Entailment-based self-training for abstractive opinion summarization...taeseon ryu
 
3D Gaussian Splatting
3D Gaussian Splatting3D Gaussian Splatting
3D Gaussian Splattingtaeseon ryu
 
Hyperbolic Image Embedding.pptx
Hyperbolic  Image Embedding.pptxHyperbolic  Image Embedding.pptx
Hyperbolic Image Embedding.pptxtaeseon ryu
 
MCSE_Multimodal Contrastive Learning of Sentence Embeddings_변현정
MCSE_Multimodal Contrastive Learning of Sentence Embeddings_변현정MCSE_Multimodal Contrastive Learning of Sentence Embeddings_변현정
MCSE_Multimodal Contrastive Learning of Sentence Embeddings_변현정taeseon ryu
 
LLaMA Open and Efficient Foundation Language Models - 230528.pdf
LLaMA Open and Efficient Foundation Language Models - 230528.pdfLLaMA Open and Efficient Foundation Language Models - 230528.pdf
LLaMA Open and Efficient Foundation Language Models - 230528.pdftaeseon ryu
 
Dataset Distillation by Matching Training Trajectories
Dataset Distillation by Matching Training Trajectories Dataset Distillation by Matching Training Trajectories
Dataset Distillation by Matching Training Trajectories taeseon ryu
 
Packed Levitated Marker for Entity and Relation Extraction
Packed Levitated Marker for Entity and Relation ExtractionPacked Levitated Marker for Entity and Relation Extraction
Packed Levitated Marker for Entity and Relation Extractiontaeseon ryu
 
MOReL: Model-Based Offline Reinforcement Learning
MOReL: Model-Based Offline Reinforcement LearningMOReL: Model-Based Offline Reinforcement Learning
MOReL: Model-Based Offline Reinforcement Learningtaeseon ryu
 
Scaling Instruction-Finetuned Language Models
Scaling Instruction-Finetuned Language ModelsScaling Instruction-Finetuned Language Models
Scaling Instruction-Finetuned Language Modelstaeseon ryu
 
Visual prompt tuning
Visual prompt tuningVisual prompt tuning
Visual prompt tuningtaeseon ryu
 
variBAD, A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning.pdf
variBAD, A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning.pdfvariBAD, A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning.pdf
variBAD, A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning.pdftaeseon ryu
 
Reinforced Genetic Algorithm Learning For Optimizing Computation Graphs.pdf
Reinforced Genetic Algorithm Learning For Optimizing Computation Graphs.pdfReinforced Genetic Algorithm Learning For Optimizing Computation Graphs.pdf
Reinforced Genetic Algorithm Learning For Optimizing Computation Graphs.pdftaeseon ryu
 
The Forward-Forward Algorithm
The Forward-Forward AlgorithmThe Forward-Forward Algorithm
The Forward-Forward Algorithmtaeseon ryu
 
Towards Robust and Reproducible Active Learning using Neural Networks
Towards Robust and Reproducible Active Learning using Neural NetworksTowards Robust and Reproducible Active Learning using Neural Networks
Towards Robust and Reproducible Active Learning using Neural Networkstaeseon ryu
 
BRIO: Bringing Order to Abstractive Summarization
BRIO: Bringing Order to Abstractive SummarizationBRIO: Bringing Order to Abstractive Summarization
BRIO: Bringing Order to Abstractive Summarizationtaeseon ryu
 

More from taeseon ryu (20)

VoxelNet
VoxelNetVoxelNet
VoxelNet
 
OpineSum Entailment-based self-training for abstractive opinion summarization...
OpineSum Entailment-based self-training for abstractive opinion summarization...OpineSum Entailment-based self-training for abstractive opinion summarization...
OpineSum Entailment-based self-training for abstractive opinion summarization...
 
3D Gaussian Splatting
3D Gaussian Splatting3D Gaussian Splatting
3D Gaussian Splatting
 
JetsonTX2 Python
 JetsonTX2 Python  JetsonTX2 Python
JetsonTX2 Python
 
Hyperbolic Image Embedding.pptx
Hyperbolic  Image Embedding.pptxHyperbolic  Image Embedding.pptx
Hyperbolic Image Embedding.pptx
 
MCSE_Multimodal Contrastive Learning of Sentence Embeddings_변현정
MCSE_Multimodal Contrastive Learning of Sentence Embeddings_변현정MCSE_Multimodal Contrastive Learning of Sentence Embeddings_변현정
MCSE_Multimodal Contrastive Learning of Sentence Embeddings_변현정
 
LLaMA Open and Efficient Foundation Language Models - 230528.pdf
LLaMA Open and Efficient Foundation Language Models - 230528.pdfLLaMA Open and Efficient Foundation Language Models - 230528.pdf
LLaMA Open and Efficient Foundation Language Models - 230528.pdf
 
YOLO V6
YOLO V6YOLO V6
YOLO V6
 
Dataset Distillation by Matching Training Trajectories
Dataset Distillation by Matching Training Trajectories Dataset Distillation by Matching Training Trajectories
Dataset Distillation by Matching Training Trajectories
 
RL_UpsideDown
RL_UpsideDownRL_UpsideDown
RL_UpsideDown
 
Packed Levitated Marker for Entity and Relation Extraction
Packed Levitated Marker for Entity and Relation ExtractionPacked Levitated Marker for Entity and Relation Extraction
Packed Levitated Marker for Entity and Relation Extraction
 
MOReL: Model-Based Offline Reinforcement Learning
MOReL: Model-Based Offline Reinforcement LearningMOReL: Model-Based Offline Reinforcement Learning
MOReL: Model-Based Offline Reinforcement Learning
 
Scaling Instruction-Finetuned Language Models
Scaling Instruction-Finetuned Language ModelsScaling Instruction-Finetuned Language Models
Scaling Instruction-Finetuned Language Models
 
Visual prompt tuning
Visual prompt tuningVisual prompt tuning
Visual prompt tuning
 
mPLUG
mPLUGmPLUG
mPLUG
 
variBAD, A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning.pdf
variBAD, A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning.pdfvariBAD, A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning.pdf
variBAD, A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning.pdf
 
Reinforced Genetic Algorithm Learning For Optimizing Computation Graphs.pdf
Reinforced Genetic Algorithm Learning For Optimizing Computation Graphs.pdfReinforced Genetic Algorithm Learning For Optimizing Computation Graphs.pdf
Reinforced Genetic Algorithm Learning For Optimizing Computation Graphs.pdf
 
The Forward-Forward Algorithm
The Forward-Forward AlgorithmThe Forward-Forward Algorithm
The Forward-Forward Algorithm
 
Towards Robust and Reproducible Active Learning using Neural Networks
Towards Robust and Reproducible Active Learning using Neural NetworksTowards Robust and Reproducible Active Learning using Neural Networks
Towards Robust and Reproducible Active Learning using Neural Networks
 
BRIO: Bringing Order to Abstractive Summarization
BRIO: Bringing Order to Abstractive SummarizationBRIO: Bringing Order to Abstractive Summarization
BRIO: Bringing Order to Abstractive Summarization
 

Recently uploaded

TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesBernd Ruecker
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesKari Kakkonen
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructureitnewsafrica
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfpanagenda
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 

Recently uploaded (20)

TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
QCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architecturesQCon London: Mastering long-running processes in modern architectures
QCon London: Mastering long-running processes in modern architectures
 
Testing tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examplesTesting tools and AI - ideas what to try with some tool examples
Testing tools and AI - ideas what to try with some tool examples
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdfSo einfach geht modernes Roaming fuer Notes und Nomad.pdf
So einfach geht modernes Roaming fuer Notes und Nomad.pdf
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 

adversarial robustness through local linearization

  • 1. 딥러닝 논문 읽기모임 이미지처리팀 Adversarial Robustness through Local Linearization Authors : Chongli Qin*, James Martens, Sven Gowal, Dilip Krishnan, Alhussein Fawzi, Soham De, Robert Stanforth, Pushmeet Kohli, Deepmind Krishnamurthy (Dj) Dvijotham Google Dec/20/2020 NeurIPS 2019 김병현 안종식 홍은기 허다은
  • 2. 딥러닝 논문 읽기모임 What is Adversarial Attack? 2  Intriguing properties of neural networks (Szegedy et al., 2014) Neural networks are velnurable to visually imperceptible adversarial perturbations Classified as Ostrich, Speaker Mantis Dog Correctly Predicted Difference Btwn Left/Right
  • 3. 딥러닝 논문 읽기모임 What is Adversarial Attack? 3  Intriguing properties of neural networks (Szegedy et al., 2014) Neural networks are velnurable to visually imperceptible adversarial perturbations Classified as Ostrich, Speaker Mantis Dog Correctly Predicted Difference Btwn Left/Right
  • 4. 딥러닝 논문 읽기모임 Difficulties in Adversarial Training  Previous Works on Adversarial Training Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks Thermometer encoding: One hot way to resist adversarial examples Stochastic Activation Pruning for Robust Adversarial Defense Leveraging generative models to understand and defend against adversarial examples. PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality Robustness via curvature regularization, and vice versa  Growing model complexity and high input dimensionality  make adversarial training prohibitive 4 VGG ResNet More complex Higher dimension MNIST (28×28) ImageNet (256×256 )
  • 5. 딥러닝 논문 읽기모임 Difficulties in Adversarial Training  Multiple iterations for computing adversarial perturbations 5 Define Model Training Data Load for i in range(training step): for j in range(perturbation step): train adversarial perturbation (requires too much training cost) training model against perturbations Psuedo Code for Adversairal Trainining i.e. Non-linear loss surface  Requires large step to find an adversarial perturbation Gradient Obfuscation (Broken Gradient) If perturbation steps are reduced, A trained model becomes less robust
  • 6. 딥러닝 논문 읽기모임 How to Prevent Gradient Obfuscation  Local Linearity Regularizer (LLR) Make Loss Surface Linear around Training Example • Much more likely to avoid Gradient Obfuscation  Requires small number of steps 6 w/o LLR with LLR
  • 7. 딥러닝 논문 읽기모임 Contribution  We show that training with LLR is significantly faster than adversarial training, allowing us to train a robust ImageNet model with a 5× speed up when training on 128 TPUv3 cores.  We show that LLR trained models exhibit higher robustness relative to adversarially trained models when evaluated under strong attacks. Adversarially trained models can exhibit a decrease in accuracy of 6% when increasing the attack strength at test time for CIFAR-10, whereas LLR shows only a decrease of 2%.  We achieve new state of the art results for adversarial accuracy against untargeted white-box attack for ImageNet (with = 4/2552 ): 47%. Furthermore, we match state of the art results for CIFAR 10 (with = 8/255): 52.81%.  We perform a large scale evaluation of existing methods for adversarially robust training under consistent, strong, white-box attacks. For this we recreate several baseline models from the literature, training them both for CIFAR-10 and ImageNet (where possible). 7
  • 8. 딥러닝 논문 읽기모임 Adversarial Traininig  Classification Function (Model) Notiation 𝒙 : input, 𝜽 : weigts, 𝑪 : logits for classes i.e. Classification model with Softmax Output  Adversarial Training more robust to adversarial attack •  Return Same Result • Perturbations Set (where : 𝜖 : magnitude) 8 w/o attack under attack
  • 9. 딥러닝 논문 읽기모임 Adversarial Traininig  General Model Training Empirical Risk Minimization (ERM) Standard Cross-Entropy Loss  Adversarial Trainig Inner Maximization using Projected Gradient Descent 9 Constraint :
  • 10. 딥러닝 논문 읽기모임 Adversarial Traininig  General Model Training Empirical Risk Minimization (ERM) Standard Cross-Entropy Loss  Adversarial Trainig Inner Maximization using Projected Gradient Descent 10 Constraint : Define Model Training Data Load for i in range(training step): for j in range(perturbation step): train adversarial perturbation training model against perturbations Important to achieve fast/good(…) training
  • 11. 딥러닝 논문 읽기모임 Motivating the Local Linearity Regularizer  Taylor Expansion  Relatively Linear Loss Surface will be.. well-predicted by 1st order Talyor Expansion 11 1st order Taylor Expansion a a Error is trivial Error is significant Linear Loss Surface Non-Linear Loss Surfcae
  • 12. 딥러닝 논문 읽기모임 Motivating the Local Linearity Regularizer  Local Linearity Measure To measure local linearity of loss surface 12 True loss when perturbation applied 1st Taylor Expansion indicator of how linear the loss surface Maximum among perturbation set Local Linearity Measure
  • 13. 딥러닝 논문 읽기모임 Local Linearity Regularizer  Empirical Observations on Adversarial Training 13 for i in range(training step): for j in range(perturbation step): train adversarial perturbation training model against perturbations
  • 14. 딥러닝 논문 읽기모임 Local Linearity Upper Bounds Adversarial Loss  Local Linearity Upper Bounds Adversarial Loss 14 Difference between Loss under attack and w/o attack 1st Taylor expansion step + Local linearity measure
  • 15. 딥러닝 논문 읽기모임 Local Linearity Regularization (LLR)  Local Linearity Regularization (LLR) 15 Upper bound equation Loss under attack can be replaced by gamma and 1st Taylor expansion term Adversarial training
  • 17. 딥러닝 논문 읽기모임 Experiments and Results  Loss Functions for Adversarial Attack 1. Random-Targeted • i.e. Cat-to-Dog (Randomly Selected, Not Change while training) 2. Untargeted • i.e. Cat-to-Dog or Mantis … Speaker (Highest Logit excluding true lable, Change while training) 3. Multi-targeted • i.e. Cat-to-Dog and Mantis … Speaker (Target All Class except true lable, Not Change while training) Nominal : Trained with Perturbation but tested without attack  Metric Attack Success Rate : Does the model return targeted class? Adversarial Accuracy : Accuracy under Adversarial Attack 17 1 2 3
  • 18. 딥러닝 논문 읽기모임 Experiments and Results  CIFAR-10 18 Sign of Gradient : One of perturbation optimizer 2 3
  • 19. 딥러닝 논문 읽기모임 Experiments and Results  ImageNet LLR showed notably higher score when radius is 4/255 LLR showed lower score when radius is 16/255 19 2 1
  • 21. 딥러닝 논문 읽기모임 Images created with perturbation radius 16/255 21  The perturbations are VISIBLE when radius is 16/255 Normal Under Attack Perturbations are visible
  • 22. 딥러닝 논문 읽기모임 Experiments and Results  Resistance to Gradient Obfuscation 22
  • 24. 딥러닝 논문 읽기모임 Discussions  Gradient Obfuscation : Broken Gradient Shattered Gradients • nonexistent or incorrect gradient (non-differentiable) Stochastic Gradients • randomized defenses Exploding & Vanishing Gradients • feeding the output of one computation as the input of the next Even though the authors mentioned the original paper found gradient obfuscation, they did not deal with any defensive experiments on attack methods proposed in the paper. 24 Athalye, Anish, Nicholas Carlini, and David Wagner. "Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples." arXiv preprint arXiv:1802.00420 (2018).
  • 25. 딥러닝 논문 읽기모임 Discussions  The paper showed how LLR contributes to a robust adversarial training by experimental results but the theoretical explanation is weak.  It seems that the notations in the paper do not include full-information for re-implementation of the proposed method. i.e. 25 Athalye, Anish, Nicholas Carlini, and David Wagner. "Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples." arXiv preprint arXiv:1802.00420 (2018).

Editor's Notes

  1. 안녕하세요 이번주 이미지처리팀 발표를 맡은 김병현입니다. 발표에 시작하기에 앞서 발표 준비 과정에서 도움을 주신 이미지팀 안종식, 홍은기, 허다은 님께 감사드립니다. 오늘 이미지처리팀에서 발표할 논문은 Adversarial Robustness through Local Linearization입니다. 2019년 NeurIPS에서 DeepMind 팀이 발표한 논문이고 Adversarial Attack에 대한 공격을 방어하기 위한 방법론을 제시하기 위한 논문입니다. 오늘 발표는 먼저 논문의 Introduction, 논문의 전체 내용 그리고 discussion 순으로 발표를 진행할 예정입니다. 발표 중 궁금한 부분이 있으시면 언제든지 질문 부탁드립니다.
  2. 먼저 본 논문은 Adversarial Attack에 관한 일반적인 설명으로 시작됩니다. Adversarial attack은 세게디가 2014년에 발표한 intriguing properties of neural networks라는 논문에서 소개되었는데요. 딥러닝 모델에 입력에 사람 눈에 보이지 않는 노이즈를 더해주면 딥러닝 모델이 오작동하도록 유도할 수 있다는 내용입니다. 이 노이즈를 adversarial perturbation이라 부르는데요. 이 노이즈는 보통 딥러닝 모델 학습 과정에 사용되는 gradient에 근거하여 만들어집니다. 아래 그림에 예시가 나타나 있는데요. 왼쪽에 있는 스피커 사마귀 개 이미지에 가운데 있는 사람이 인식하기 어려운 형태의 노이즈를 더해주면 오른쪽에 있는 이미지들은 여전히 사람이 보기에는 스피커 사마귀 개 이지만
  3. 타조로 분류를 하도록 유도할 수 있습니다. 열심히 학습시켜놓은 딥러닝 모델이 오작동하면 만드느라 고생한 개발자도 마음이 아프지만 사용시 오작동을 일으키면 사고와 같은 큰 문제를 유발할 수 있기 때문에 이를 방어하기 위한 여러가 방법론들이 연구되고 있습니다.
  4. 이러한 adversarial attac을 방어하기 위한 학습을 adversarial training 이라 하는데요. 이 adversarial training 을 막기 위한 여러가지 방법론들이 제기가 되었습니다. 오늘 자세히는 다루지 않겠지만 ppt에 리스트를 작성해두었으니 참고하시면 좋을 것 같습니다. 그런데 이러한 방법론들을 적용하고 확장시키는데 따르는 어려움이 있는데요. 이들은 바로 모델의 복잡도 증가와 입력 데이터의 크기입니다. 아무래도 왼쪽의 VGG와 같은 단순한 형태의 모델보다 ResNet과 같이 복잡한 모델이 학습이 더 어려울 것이고, MNIST 같은 저차원의 데이터보다는 ImageNet과 같이 상대적으로 고차원의 데이터에 대한 adversarial training이 어려울 것입니다.
  5. 저자들을 이 모델의 복잡성 증가와 입력 데이터의 차원 증가가 구체적으로 어떤 어려움을 유발하는지 기술했습니다. 왼쪽에 adversarial training 의 Psuedo 코드 예시를 작성해 두었는데요. 보시면 학습을 위한 loop 안에 adversarial perturbation을 학습하는 loop가 들어가 있습니다. 매 학습 step 마다 perturbation 학습을 한 번 완주해야하기 때문에 학습이 길어지는데요. 이 학습이 모델의 복잡성 증가와 입력 데이터 차원 증가에 비례하기 때문에 학습 시간 증가를 유발합니다. 그래서 이 perturbation 학습 자체를 적게 가져가는 방법론들이 있는데, 그렇게 되면 직관적으로 생각해도 상대적으로 강한 공격에 취약할 수 밖에 없다는 것을 알 수 있습니다. 저자들은 perturbation 학습을 적게 가져갈 수 없는 이유로 Gradient obfuscation을 꼽았습니다. Gradient Obfuscation은 모델이 adversarial training 중 비정상적인 gradient를 보이는 형태로 학습되는 것을 말합니다. 본 논문에서는 Gradient Obfuscation의 한 가지 예로 모델이 자신의 loss surface 를 비선형화 시켜 perturbation 을 찾는 시간을 지연시키는 현상을 꼽고 있습니다. perturbation 이 발생 가능한 지점이 비선형화된 loss surface 어딘가에 숨어 있으면, 이 지점을 어떻게든 찾아서 adversarial training에 포함시켜야하기 때문에 학습이 더욱 길어진다고 말합니다. 그래서 이러한 gradient obfuscation의 발생을 막아주면, 적은 iteration의 perturbation training 만으로 충분히 adversarial attac에 robust한 모델을 학습시킬 수 있다고 말합니다.
  6. 그래서 본 논문의 저자들은 이렇게 비선형 loss surface 의 발생을 줄여 perturbation 학습 시간을 줄이기 위한 방법으로 Local linearity regularizer를 제안하였습니다. 이는 일반적인 training loss를 linear하게 제어하기위한 regularizer라고 생각하시면 됩니다. 이렇게 loss surface 가 linear 하게 제어되면 , gradient obfuscation의 발생 가능성이 낮아지고 작은 step의 perturbation training 만으로 robust한 성능을 달성할 수 있다고 주장합니다.
  7. LLR을 적용한 결과 슬라이드에 보시는 것과 같은 성과를 얻을 수 있었다고 말합니다. 간단하게 요약해 드리면 아주 빠른 속도로 adversarial training을 수행할 수 있었고, 물론 기존 방법론들보다 더 높은 정확도를 달성했다고 합니다. 실험은 ImageNet과 CIFAR-10 데이터셋에서 수행되었습니다. 이 실험 자체가 가지는 신뢰도를 높이기 위해서 adversarial attack을 다른 모델들과 유사한 환경에서 비교적 강한 공격을 가했다고 합니다. 여기 white box라는 표현이 나오는데 이는 모델이 어떤 상태인지 아는 상태에서 수행하는 공격으로 그렇지 못한 상태에서 진행하는 black box attack 보다 더 강한 공격성능을 가지는 것으로 알려져 있습니다.
  8. 지금 슬라이드부터는 앞에서 설명드린 내용들이 조금 더 자세하게 서술됩니다. 먼저 본 논문에서는 Adversarial training을 수식으로 정의하면서 시작합니다. 맨 위의 수식은 일반적인 classification model을 정의하고 있습니다. 먼저 classification model을 입력값 x 와 가중치 theta를 받아서 클래스를 나타내는 실수의 집합을 반환하는 함수로 정의하고 있습니다. 이 수식을 이용하면 아래 예시에 나타난 Softmax Output 도 나타낼 수 있습니다. Adversarial training은 아래 수식에 정의되어 있는데요. 좀 전에 보여드린 classification model의 입력값에 순수한 x가 들어가든, x + delta 즉 perturbation이 섞인 입력이 들어가든 최대한 동일한 결과를 반환하는 모델을 만드는 것을 목표로 합니다. 여기서 공격에 사용되는 perturbation delta는 특정한 크기 이상을 가지지 못하도록 제한됩니다. 만약 이 부분이 너무 커지면 peturbation이 눈에 보이는 수준이 되기 때문에 이런 제한을 걸어서 perturbation 이 눈에 보이지 않도록 유도합니다.
  9. Adversarial training에 대한 개념적인 목표에 대해서 다루었고 이제 그 목표를 달성하기 위한 최적화 과정에 대해서 설명드리겠습니다. 일반적인 classification model의 최적화 문제는 오차에 의해서 계산된 loss를 이용하여 Empirical Risk Minimization 문제를 풀도록 되어 있습니다. Empirical Risk Minimization의 뜻은 어떤 모델이 사용될 환경을 100% 예측할 수 없기 때문에, 주어진 특정 데이터 셋에서 최적화를 수행하는 방식입니다. 즉 완벽한 정해를 찾기 어렵기 때문에 어느정도 근사된 모델을 사용한다고 보시면 좋을 것 같습니다. 본 논문에서는 adversarial training 의 loss로 standard cross entropy loss 를 채택하고 있습니다. Adversarial training 은 일반적인 학습과 달리 ERM 내부에 maximization을 포함하고 있는데요. 이를 본 논문에서 inner maximization 이라고 부릅니다. 여기서 Inner maximization 은 loss가 커지는 방향으로 delta를 업데이트합니다. 이 delta를 최적화하는 방법으로 본 논문에서는 projected gradient descent 가 활용됩니다. Gradient descent 에서 특정 constraint 가 제한하는 집합으로 변수를 project 하기 때문에 project gradient descent 라고 표현합니다. 여기서 constraint 는 앞에서 보여드렸던 delta 크기의 집합으로 보시면 될 것 같습니다.
  10. Inner maximization에 의한 학습과정은 제가 앞에서 보여드렸던 Psuedo code 에도 나타나 있습니다. 일반적인 모델이 취하는 학습 과정 사이에 perturbation training 이 들어가는 것이죠. 그래서 perturbation trainin이 얼마나 길어지냐에 따라서 학습시간이 굉장히 달라지게 됩니다. 본 논문에서는 이 perturbation training하는 방법을 잘 설계해서 전체 학습 과정을 줄이고자 목표하고 있습니다. 예를 들어 학습을 50 epoch로 잡았을 때, perturbation 학습이 매번 20 epoch씩 이루어지면 총 1000epoch만큼 학습이 이루어져야 합니다. Perturbation 학습이 증가하면 전체적인 학습의 관점에서 보면 큰 폭으로 학습 소요 시간이 길어지게 되는데요. adversarial training에서 이 perturbation training의 학습 속도와 좋은 품질을 달성하는게 관건이라고 보시면 될 것 같습니다.
  11. 본 논문에서는 adversarial training의 computational cost를 줄이기 위한 방법론으로 학습 중의 loss surface 를 최대한 평평하게, linear하게 유지하는 방법을 제안하는데요. 그래서 먼저 loss surface가 얼마나 평평한지 측정하기 위한 방법으로 taylor expansio을 이용한 식을 제안합니다. 만약 loss surface 가 linear 하다면 taylor 1차 근사에 의해서 잘 예측이 될 것이라는 가정입니다. 왼쪽 아래 그림을 보시면 2차함수 그래프가 있는데요. 만약 loss가 이렇게 linear한 형태를 가지고 있으면 특정 지점에서 taylor 1차 근사를 했을 때, 그 오차가 그림에서 보시는 것처럼 작을 것입니다. 오른쪽에는 다차함수의 그래프가 있습니다. 만약 loss surface가 이런 형태를 띄고 있으면 특정 지점에서 taylor 1차 근사 값이 실제 값과 차이가 클 것입니다. 이 오차 값에 근거해서 loss surface 의 linear 한 정도를 예측한다고 보시면 될 것 같습니다.
  12. 이 아이디어에서 제안이 된 식이 여기 local linearity measure 입니다. 먼저 위에 g 함수를 보시면, 저자들은 이 g 함수를 실제 x+delta에 의하여 계산된 true loss 값과 1차 테일러 근사값의 차이로 구성하였습니다. 그래서 이 차이를 loss surfac의 linear 한 정도를 나타내는 값으로 특정 짓습니다. 그리고 전체 delta set 중에서 가장 큰 차이를 보이게 하는 perturbation 에 의한 g 함수 값을 local linearity measure 라고 정의합니다. 뒤에서는 감마값이라고 부르겠습니다.
  13. 저자들은 이 감마값을 이용해서 학습 중에 loss surface 의 비선형성이 어떻게 변화하는지 관측했습니다. 붉은색 박스로 표시된 부분이 perturbation step이고, 파란색 박스로 표시된 부분이 모델 학습 step 입니다. 이 실험은 perturbation step 을 특정 수로 고정하고 모델의 학습을 진행했을 때 모델이 어떤 loss surface의 형태를 보이는지 관측한 것입니다. 그래프에서 확인하실 수 있듯 perturbation step을 적게 설정하였을 때 모델의 loss surface가 굉장히 비선형적으로 발현되는 것을 보실 수 있습니다. 감마 값이 굉장히 커지는 것입니다. 1 step일 때는 non linearity를 나타내는 감마값이 6-12 정도에 분포하는데, 4 step 정도로 설정하면 그 값이 0.1 정도로 떨어지는 것을 확인하실 수 있습니다. 이 실험을 통해서 저자들은 adversarial training 학습 초기에 나타나는 loss 의 비선형성을 제어해주는 것이 빠르고 고품질(?)의 학습에 기여할 것이다라고 주장합니다.
  14. 실험 결과에 근거하여 저자들은 adversarial trainin을 위한 새로운 loss functio을 제안하려고 하는데요. 여기 보시면 upper bounds adversarial loss라는 표현이 있습니다. 이는 adversarial loss, 즉 perturbation이 섞인 입력에 의해서 계산되는 loss의 범위를 규정할 수 있다면, 그 범위를 규정하는 term 들을 이용하여 새로운 loss를 정의할 수 있다는 것입니다. 그래서 위쪽 식을 보시면 공격에 의해 나타나는 로스와 / 공격이 없을 때 나타나는 로스의 차이는 위에서 나타났던 테일러 1차 근사의 항과 감마 값의 합으로 계산할 수 있다고 합니다. 아래는 논문 부록에 있는 증명을 제가 가져온 것인데요. 공격이 있을 때와 없을 때의 loss 차이에 테일러 1차 근사 값을 더해주고 빼주면 첫번째 줄이 됩니다. 여기서 테일러 1차 근사값 부분을 제외하면 앞에서 정의한 g 함수가 됩니다. 그리고 감마 값은 이 g 함수의 최대값이기 때문에 최종적으로 공격이 있을 때와 없을 때의 loss 차이는 테일러 1차 근사값과 감마값의 합보다 작다라는 결론이 나옵니다.
  15. 앞에 제안된 바운더리를 나타내는 식에서 공격이 없을 때 loss를 식의 오른쪽으로 넘겨주면 오른쪽에 식은 공격이 없을 때의 식 더하기 테일러 1차 근사에 의한 값 더하기 감마 값이 됩니다. 여기서 공격이 있을 때의 로스가 공격이 없을 때의 로스보다 크다는 가정이 들어갑니다. 여기서 얻어진 공격이 있을 때 loss가 최종적으로는 공격이 없을 때의 로스 + 이 두 regularizer 텀의 합보다 작기 때문에 이 식을 아래 adversarial training 에 필요한 inner maximization 수식에 대체하여 집어 넣을 수 있게 됩니다. 거기에 람다와 누라는 hyper parameter를 곱해서 LLR을 이용한 loss functio을 완성한다고 보시면 될 것 같습니다.