SlideShare a Scribd company logo
Adversarial Attacks and
Defenses in Deep Learning
Kui Ren, Tiahnhang Zheng, Zhan Qin, Xue Liu (2020) Engineering
On April 23, 2013, Syrian hackers compromised the Associated Press
Twitter feed and tweeted, “Breaking: Two Explosions in the White House
and Barack Obama is injured”.
In response to the tweet, the Dow Jones Industrial Average dropped by
$136 billion dollars (although this drop was reversed 3 minutes later).
What are adversarial attacks and why should you
care?
● Any attempt to fool a deep learning model with deceptive input
● Especially researched in image recognition, but can also be applied to audio, text or tabular data
● When building models, we mostly focus on classification effectiveness/ minimizing error. Relatively
little work on model security and robustness.
● Imperceptible amounts of non-random noise can fool neural networks!
● Some of these attacks are 100% effective in fooling normal neural networks!
Gong & Poellabauer, 2018. Comp Sci & Engineering
What I’ll talk about
● Threat-models
● Some background terminology
● Notable adversarial models
● Notable adversarial defences
● Trends and remaining challenges
● Code
Level of threat
● White-box: full knowledge of model architecture and parameters
● Gray-box: knowledge limited to features, model type
● Black-box: no/minimal knowledge of the model, can only use output
All non-adversarially trained models are susceptible, even to black box models
Adversarially trained models are still susceptible to white box models
Background
Adversarial loss: J(θ, x, y), θ = model weights
An adversarial sample x’ : D(x,x’) < η (predefined distance constraint, perturbation)
● Idea: find the minimum difference or perturbation f(x’) ≠ y’
Adversarial samples should be indistinguishable from
benign samples
Distance metrics:
● L₂ distance: What is the squared difference between adversarial and benign image?
● L∞ distance: Maximum element-wise difference between adversarial and benign image
(for each pixel, take the absolute value difference between X and Z, and return the
largest such distance found over all pixels)
Notable adversarial models
Limited-memory BFGS
Grid search / line search to find optimal hyperparameter
Carlini and Wagner (C&W) attack
Set of optimization-based attacks that generate L₀, L₂ and L∞ norm measured adversarial samples, with some
restrictions (kappa) to make sure a valid image is produced
100% attack success on ‘normal’ neural networks trained on MNIST, CIFAR-10, ImageNet
Compromised defensive models
Notable adversarial models
DeepFool
“Iterative linearization of the
classifier to generate minimal
perturbations that are sufficient
to change classification labels”
Computes perturbations more
reliably
Moosavi-Dezfooli et al., https://arxiv.org/pdf/1511.04599.pdf
Notable adversarial models
Universal adversarial attack
● Is there a universal perturbation that will work on most
samples?
● L-BFGS- based
● Effective in attacking NN like CaffeNet, GoogleNet, VGG and
Resnet
● Fooling rate > 53%
Adversarial patch
Only certain areas are perturbed
Text & Audio Models
1% audio perturbation can change 50 words in
text transcription!
Attacks are robust to MP3 compression, but get
lost when played over speakers
https://nicholas.carlini.com/code/audio_adversarial_examples/
Strategies for text attacks generally include
deleting, inserting, and modifying
characters/words
Adversarial defenses fall into 5 categories
1. Training on adversarial samples
2. Randomization
3. Adding noise
4. Removing noise
5. Mathematically provable defenses
Defang: Randomize input or features
● Randomly padding and resizing input; image transformations with randomness
● Add random noise layer before each convolutional layer in training and test sets (RSE)
● Random feature pruning at each layer
Detect: Denoise the input or features
● Conventional input rectification
○ ‘Squeeze’ image → if output is very different from input, then likely adversarial
● GAN-based
○ Use GAN to learn benign data distribution
○ Generate a benign projection for the adversarial sample
● Autoencoder-based
○ Detector & reformer
○ Use an autocoder to compress input and learn manifold of benign samples
○ Detector compares each sample to learnt manifold
○ Reformer rectifies adversarial samples
Detect: Denoise the input or features
● High-level representation guided denoiser (HGD)
○ Trains a denoising u-net using a feature-level loss function to minimize feature
differences between benign and adversarial samples
○ Won first place in black-box defenses, 2017
○ Even so, certain (white-box) attacks can reduce effectiveness to 0%
Provable (certificated) defenses
● Defenses that have theoretical backing to have a certain accuracy against attacks
● Range of defenses include KNN and Bayesian-based defenses
● Consistency-based defenses:
○ Perturbations also affect the area around them
○ > 90 detection rate
● Very computationally intensive
Trends in adversarial research
● Design stronger attacks to probe for weaknesses
● Real-world attack capabilities
● Certificated defenses - but currently not scalable
“A problem is that an attack can only target one category of defenses, but defenses are required to … be effective
against all possible attack methods”
● Analyzing model robustness - mostly done on KKN and linear classifiers
Unresolved challenges
● Causality
● Does a general robust decision boundary exist that could be learnt by (certain) neural
networks?
● Effectiveness vs efficiency
○ Adversarial training is effective, but requires a lot of data and compute
○ Randomization and denoising strategies very efficient, but not as effective as claimed
Discussion
In what other ways are models not robust?
Is model robustness/ security applicable to what you do / to our students?
Model fairness has been a hot topic lately, but robustness/ security seems to lag behind - what do you
think needs to change for adversarial training to be widely implemented?
What are your thoughts on the paper in general?
Try it yourself
Benchmark machine learning systems' vulnerability to adversarial examples:
https://github.com/cleverhans-lab/cleverhans
Blog: cleverhans.io

More Related Content

What's hot

Adversarial Attacks on Deep Learning Systems
Adversarial Attacks on Deep Learning SystemsAdversarial Attacks on Deep Learning Systems
Adversarial Attacks on Deep Learning Systems
mohammad zaher airout
 
Adversarial machine learning for av software
Adversarial machine learning for av softwareAdversarial machine learning for av software
Adversarial machine learning for av software
junseok seo
 
Research of adversarial example on a deep neural network
Research of adversarial example on a deep neural networkResearch of adversarial example on a deep neural network
Research of adversarial example on a deep neural network
NAVER Engineering
 
Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)
MeetupDataScienceRoma
 
Learning Deep Learning
Learning Deep LearningLearning Deep Learning
Learning Deep Learning
simaokasonse
 
Federated Learning
Federated LearningFederated Learning
Federated Learning
DataWorks Summit
 
WILD PATTERNS - Introduction to Adversarial Machine Learning - ITASEC 2019
WILD PATTERNS - Introduction to Adversarial Machine Learning - ITASEC 2019WILD PATTERNS - Introduction to Adversarial Machine Learning - ITASEC 2019
WILD PATTERNS - Introduction to Adversarial Machine Learning - ITASEC 2019
Pluribus One
 
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
Dataconomy Media
 
Robust Filtering Schemes for Machine Learning Systems to Defend Adversarial A...
Robust Filtering Schemes for Machine Learning Systems to Defend Adversarial A...Robust Filtering Schemes for Machine Learning Systems to Defend Adversarial A...
Robust Filtering Schemes for Machine Learning Systems to Defend Adversarial A...
Kishor Datta Gupta
 
Using Machine Learning in Networks Intrusion Detection Systems
Using Machine Learning in Networks Intrusion Detection SystemsUsing Machine Learning in Networks Intrusion Detection Systems
Using Machine Learning in Networks Intrusion Detection Systems
Omar Shaya
 
Explainable Machine Learning (Explainable ML)
Explainable Machine Learning (Explainable ML)Explainable Machine Learning (Explainable ML)
Explainable Machine Learning (Explainable ML)
Hayim Makabee
 
Explainable AI
Explainable AIExplainable AI
Explainable AI
Dinesh V
 
Poisoning attacks on Federated Learning based IoT Intrusion Detection System
Poisoning attacks on Federated Learning based IoT Intrusion Detection SystemPoisoning attacks on Federated Learning based IoT Intrusion Detection System
Poisoning attacks on Federated Learning based IoT Intrusion Detection System
Sai Kiran Kadam
 
Intrusion Detection with Neural Networks
Intrusion Detection with Neural NetworksIntrusion Detection with Neural Networks
Intrusion Detection with Neural Networks
antoniomorancardenas
 
Deep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingDeep Learning for Natural Language Processing
Deep Learning for Natural Language Processing
Devashish Shanker
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
Yunjey Choi
 
Computational Learning Theory
Computational Learning TheoryComputational Learning Theory
Computational Learning Theorybutest
 
Adversarial machine learning
Adversarial machine learningAdversarial machine learning
Adversarial machine learning
Rama Chetan
 
GAN - Theory and Applications
GAN - Theory and ApplicationsGAN - Theory and Applications
GAN - Theory and Applications
Emanuele Ghelfi
 
Memebership inference attacks against machine learning models
Memebership inference attacks against machine learning modelsMemebership inference attacks against machine learning models
Memebership inference attacks against machine learning models
Rrubaa Panchendrarajan
 

What's hot (20)

Adversarial Attacks on Deep Learning Systems
Adversarial Attacks on Deep Learning SystemsAdversarial Attacks on Deep Learning Systems
Adversarial Attacks on Deep Learning Systems
 
Adversarial machine learning for av software
Adversarial machine learning for av softwareAdversarial machine learning for av software
Adversarial machine learning for av software
 
Research of adversarial example on a deep neural network
Research of adversarial example on a deep neural networkResearch of adversarial example on a deep neural network
Research of adversarial example on a deep neural network
 
Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)
 
Learning Deep Learning
Learning Deep LearningLearning Deep Learning
Learning Deep Learning
 
Federated Learning
Federated LearningFederated Learning
Federated Learning
 
WILD PATTERNS - Introduction to Adversarial Machine Learning - ITASEC 2019
WILD PATTERNS - Introduction to Adversarial Machine Learning - ITASEC 2019WILD PATTERNS - Introduction to Adversarial Machine Learning - ITASEC 2019
WILD PATTERNS - Introduction to Adversarial Machine Learning - ITASEC 2019
 
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
Big Data Helsinki v 3 | "Federated Learning and Privacy-preserving AI" - Oguz...
 
Robust Filtering Schemes for Machine Learning Systems to Defend Adversarial A...
Robust Filtering Schemes for Machine Learning Systems to Defend Adversarial A...Robust Filtering Schemes for Machine Learning Systems to Defend Adversarial A...
Robust Filtering Schemes for Machine Learning Systems to Defend Adversarial A...
 
Using Machine Learning in Networks Intrusion Detection Systems
Using Machine Learning in Networks Intrusion Detection SystemsUsing Machine Learning in Networks Intrusion Detection Systems
Using Machine Learning in Networks Intrusion Detection Systems
 
Explainable Machine Learning (Explainable ML)
Explainable Machine Learning (Explainable ML)Explainable Machine Learning (Explainable ML)
Explainable Machine Learning (Explainable ML)
 
Explainable AI
Explainable AIExplainable AI
Explainable AI
 
Poisoning attacks on Federated Learning based IoT Intrusion Detection System
Poisoning attacks on Federated Learning based IoT Intrusion Detection SystemPoisoning attacks on Federated Learning based IoT Intrusion Detection System
Poisoning attacks on Federated Learning based IoT Intrusion Detection System
 
Intrusion Detection with Neural Networks
Intrusion Detection with Neural NetworksIntrusion Detection with Neural Networks
Intrusion Detection with Neural Networks
 
Deep Learning for Natural Language Processing
Deep Learning for Natural Language ProcessingDeep Learning for Natural Language Processing
Deep Learning for Natural Language Processing
 
Generative adversarial networks
Generative adversarial networksGenerative adversarial networks
Generative adversarial networks
 
Computational Learning Theory
Computational Learning TheoryComputational Learning Theory
Computational Learning Theory
 
Adversarial machine learning
Adversarial machine learningAdversarial machine learning
Adversarial machine learning
 
GAN - Theory and Applications
GAN - Theory and ApplicationsGAN - Theory and Applications
GAN - Theory and Applications
 
Memebership inference attacks against machine learning models
Memebership inference attacks against machine learning modelsMemebership inference attacks against machine learning models
Memebership inference attacks against machine learning models
 

Similar to Adversarial Attacks and Defenses in Deep Learning.pdf

Applicability issues of Evasion-Based Adversarial Attacks and Mitigation Tech...
Applicability issues of Evasion-Based Adversarial Attacks and Mitigation Tech...Applicability issues of Evasion-Based Adversarial Attacks and Mitigation Tech...
Applicability issues of Evasion-Based Adversarial Attacks and Mitigation Tech...
Kishor Datta Gupta
 
Group 10 - DNN Presentation for UOM.pptx
Group 10 - DNN Presentation for UOM.pptxGroup 10 - DNN Presentation for UOM.pptx
Group 10 - DNN Presentation for UOM.pptx
DanNiles4
 
Adversarial Training is all you Need.pptx
Adversarial Training is all you Need.pptxAdversarial Training is all you Need.pptx
Adversarial Training is all you Need.pptx
Prerana Khatiwada
 
Battista Biggio @ S+SSPR2014, Joensuu, Finland -- Poisoning Complete-Linkage ...
Battista Biggio @ S+SSPR2014, Joensuu, Finland -- Poisoning Complete-Linkage ...Battista Biggio @ S+SSPR2014, Joensuu, Finland -- Poisoning Complete-Linkage ...
Battista Biggio @ S+SSPR2014, Joensuu, Finland -- Poisoning Complete-Linkage ...Pluribus One
 
[Paper Review] Audio adversarial examples
[Paper Review] Audio adversarial examples[Paper Review] Audio adversarial examples
[Paper Review] Audio adversarial examples
JaeSung Bae
 
PRACTICAL ADVERSARIAL ATTACKS AGAINST CHALLENGING MODELS ENVIRONMENTS - Moust...
PRACTICAL ADVERSARIAL ATTACKS AGAINST CHALLENGING MODELS ENVIRONMENTS - Moust...PRACTICAL ADVERSARIAL ATTACKS AGAINST CHALLENGING MODELS ENVIRONMENTS - Moust...
PRACTICAL ADVERSARIAL ATTACKS AGAINST CHALLENGING MODELS ENVIRONMENTS - Moust...
GeekPwn Keen
 
Bringing Red vs. Blue to Machine Learning
Bringing Red vs. Blue to Machine LearningBringing Red vs. Blue to Machine Learning
Bringing Red vs. Blue to Machine Learning
Bobby Filar
 
slides_security_and_privacy_in_machine_learning.pptx
slides_security_and_privacy_in_machine_learning.pptxslides_security_and_privacy_in_machine_learning.pptx
slides_security_and_privacy_in_machine_learning.pptx
ssuserabf73f
 
Survey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning ModelsSurvey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning Models
IRJET Journal
 
Black-Box attacks against Neural Networks - technical project presentation
Black-Box attacks against Neural Networks - technical project presentationBlack-Box attacks against Neural Networks - technical project presentation
Black-Box attacks against Neural Networks - technical project presentation
Roberto Falconi
 
DEF CON 27 - workshop - YACIN NADJI - hands on adverserial machine learning
DEF CON 27 - workshop - YACIN NADJI - hands on adverserial machine learningDEF CON 27 - workshop - YACIN NADJI - hands on adverserial machine learning
DEF CON 27 - workshop - YACIN NADJI - hands on adverserial machine learning
Felipe Prado
 
AI Cybersecurity: Pros & Cons. AI is reshaping cybersecurity
AI Cybersecurity: Pros & Cons. AI is reshaping cybersecurityAI Cybersecurity: Pros & Cons. AI is reshaping cybersecurity
AI Cybersecurity: Pros & Cons. AI is reshaping cybersecurity
Tasnim Alasali
 
Adversarial ML - Part 2.pdf
Adversarial ML - Part 2.pdfAdversarial ML - Part 2.pdf
Adversarial ML - Part 2.pdf
KSChidanandKumarJSSS
 
Machine Duping 101: Pwning Deep Learning Systems
Machine Duping 101: Pwning Deep Learning SystemsMachine Duping 101: Pwning Deep Learning Systems
Machine Duping 101: Pwning Deep Learning Systems
Clarence Chio
 
Adversarial robustness using clever hans
Adversarial robustness using clever hansAdversarial robustness using clever hans
Adversarial robustness using clever hans
ShrutiMishra164
 
Attacks on Victim Model! A Defense Strategy
Attacks on Victim Model! A Defense StrategyAttacks on Victim Model! A Defense Strategy
Attacks on Victim Model! A Defense Strategy
Sivaranjanikumar1
 
Robustness of compressed CNNs
Robustness of compressed CNNsRobustness of compressed CNNs
Robustness of compressed CNNs
Kaushalya Madhawa
 
Deceiving Autonomous Cars with Toxic Signs
Deceiving Autonomous Cars with Toxic SignsDeceiving Autonomous Cars with Toxic Signs
Deceiving Autonomous Cars with Toxic Signs
LeonardoSalvucci1
 
Who is responsible for adversarial defense
Who is responsible for adversarial defenseWho is responsible for adversarial defense
Who is responsible for adversarial defense
Kishor Datta Gupta
 
Securing Neural Networks
Securing Neural NetworksSecuring Neural Networks
Securing Neural Networks
Tahseen Shabab
 

Similar to Adversarial Attacks and Defenses in Deep Learning.pdf (20)

Applicability issues of Evasion-Based Adversarial Attacks and Mitigation Tech...
Applicability issues of Evasion-Based Adversarial Attacks and Mitigation Tech...Applicability issues of Evasion-Based Adversarial Attacks and Mitigation Tech...
Applicability issues of Evasion-Based Adversarial Attacks and Mitigation Tech...
 
Group 10 - DNN Presentation for UOM.pptx
Group 10 - DNN Presentation for UOM.pptxGroup 10 - DNN Presentation for UOM.pptx
Group 10 - DNN Presentation for UOM.pptx
 
Adversarial Training is all you Need.pptx
Adversarial Training is all you Need.pptxAdversarial Training is all you Need.pptx
Adversarial Training is all you Need.pptx
 
Battista Biggio @ S+SSPR2014, Joensuu, Finland -- Poisoning Complete-Linkage ...
Battista Biggio @ S+SSPR2014, Joensuu, Finland -- Poisoning Complete-Linkage ...Battista Biggio @ S+SSPR2014, Joensuu, Finland -- Poisoning Complete-Linkage ...
Battista Biggio @ S+SSPR2014, Joensuu, Finland -- Poisoning Complete-Linkage ...
 
[Paper Review] Audio adversarial examples
[Paper Review] Audio adversarial examples[Paper Review] Audio adversarial examples
[Paper Review] Audio adversarial examples
 
PRACTICAL ADVERSARIAL ATTACKS AGAINST CHALLENGING MODELS ENVIRONMENTS - Moust...
PRACTICAL ADVERSARIAL ATTACKS AGAINST CHALLENGING MODELS ENVIRONMENTS - Moust...PRACTICAL ADVERSARIAL ATTACKS AGAINST CHALLENGING MODELS ENVIRONMENTS - Moust...
PRACTICAL ADVERSARIAL ATTACKS AGAINST CHALLENGING MODELS ENVIRONMENTS - Moust...
 
Bringing Red vs. Blue to Machine Learning
Bringing Red vs. Blue to Machine LearningBringing Red vs. Blue to Machine Learning
Bringing Red vs. Blue to Machine Learning
 
slides_security_and_privacy_in_machine_learning.pptx
slides_security_and_privacy_in_machine_learning.pptxslides_security_and_privacy_in_machine_learning.pptx
slides_security_and_privacy_in_machine_learning.pptx
 
Survey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning ModelsSurvey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning Models
 
Black-Box attacks against Neural Networks - technical project presentation
Black-Box attacks against Neural Networks - technical project presentationBlack-Box attacks against Neural Networks - technical project presentation
Black-Box attacks against Neural Networks - technical project presentation
 
DEF CON 27 - workshop - YACIN NADJI - hands on adverserial machine learning
DEF CON 27 - workshop - YACIN NADJI - hands on adverserial machine learningDEF CON 27 - workshop - YACIN NADJI - hands on adverserial machine learning
DEF CON 27 - workshop - YACIN NADJI - hands on adverserial machine learning
 
AI Cybersecurity: Pros & Cons. AI is reshaping cybersecurity
AI Cybersecurity: Pros & Cons. AI is reshaping cybersecurityAI Cybersecurity: Pros & Cons. AI is reshaping cybersecurity
AI Cybersecurity: Pros & Cons. AI is reshaping cybersecurity
 
Adversarial ML - Part 2.pdf
Adversarial ML - Part 2.pdfAdversarial ML - Part 2.pdf
Adversarial ML - Part 2.pdf
 
Machine Duping 101: Pwning Deep Learning Systems
Machine Duping 101: Pwning Deep Learning SystemsMachine Duping 101: Pwning Deep Learning Systems
Machine Duping 101: Pwning Deep Learning Systems
 
Adversarial robustness using clever hans
Adversarial robustness using clever hansAdversarial robustness using clever hans
Adversarial robustness using clever hans
 
Attacks on Victim Model! A Defense Strategy
Attacks on Victim Model! A Defense StrategyAttacks on Victim Model! A Defense Strategy
Attacks on Victim Model! A Defense Strategy
 
Robustness of compressed CNNs
Robustness of compressed CNNsRobustness of compressed CNNs
Robustness of compressed CNNs
 
Deceiving Autonomous Cars with Toxic Signs
Deceiving Autonomous Cars with Toxic SignsDeceiving Autonomous Cars with Toxic Signs
Deceiving Autonomous Cars with Toxic Signs
 
Who is responsible for adversarial defense
Who is responsible for adversarial defenseWho is responsible for adversarial defense
Who is responsible for adversarial defense
 
Securing Neural Networks
Securing Neural NetworksSecuring Neural Networks
Securing Neural Networks
 

Recently uploaded

DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
Abida Shariff
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
OnBoard
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
Frank van Harmelen
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Inflectra
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
Product School
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Jeffrey Haguewood
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
Product School
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Prayukth K V
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User Group
CatarinaPereira64715
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 

Recently uploaded (20)

DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdfFIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptxIOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx
 
FIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdfFIDO Alliance Osaka Seminar: Overview.pdf
FIDO Alliance Osaka Seminar: Overview.pdf
 
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdfLeading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*Neuro-symbolic is not enough, we need neuro-*semantic*
Neuro-symbolic is not enough, we need neuro-*semantic*
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...How world-class product teams are winning in the AI era by CEO and Founder, P...
How world-class product teams are winning in the AI era by CEO and Founder, P...
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
From Daily Decisions to Bottom Line: Connecting Product Work to Revenue by VP...
 
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 previewState of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
ODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User GroupODC, Data Fabric and Architecture User Group
ODC, Data Fabric and Architecture User Group
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 

Adversarial Attacks and Defenses in Deep Learning.pdf

  • 1. Adversarial Attacks and Defenses in Deep Learning Kui Ren, Tiahnhang Zheng, Zhan Qin, Xue Liu (2020) Engineering
  • 2. On April 23, 2013, Syrian hackers compromised the Associated Press Twitter feed and tweeted, “Breaking: Two Explosions in the White House and Barack Obama is injured”. In response to the tweet, the Dow Jones Industrial Average dropped by $136 billion dollars (although this drop was reversed 3 minutes later).
  • 3. What are adversarial attacks and why should you care? ● Any attempt to fool a deep learning model with deceptive input ● Especially researched in image recognition, but can also be applied to audio, text or tabular data ● When building models, we mostly focus on classification effectiveness/ minimizing error. Relatively little work on model security and robustness. ● Imperceptible amounts of non-random noise can fool neural networks! ● Some of these attacks are 100% effective in fooling normal neural networks!
  • 4. Gong & Poellabauer, 2018. Comp Sci & Engineering
  • 5.
  • 6.
  • 7. What I’ll talk about ● Threat-models ● Some background terminology ● Notable adversarial models ● Notable adversarial defences ● Trends and remaining challenges ● Code
  • 8. Level of threat ● White-box: full knowledge of model architecture and parameters ● Gray-box: knowledge limited to features, model type ● Black-box: no/minimal knowledge of the model, can only use output All non-adversarially trained models are susceptible, even to black box models Adversarially trained models are still susceptible to white box models
  • 9. Background Adversarial loss: J(θ, x, y), θ = model weights An adversarial sample x’ : D(x,x’) < η (predefined distance constraint, perturbation) ● Idea: find the minimum difference or perturbation f(x’) ≠ y’
  • 10. Adversarial samples should be indistinguishable from benign samples Distance metrics: ● L₂ distance: What is the squared difference between adversarial and benign image? ● L∞ distance: Maximum element-wise difference between adversarial and benign image (for each pixel, take the absolute value difference between X and Z, and return the largest such distance found over all pixels)
  • 11. Notable adversarial models Limited-memory BFGS Grid search / line search to find optimal hyperparameter Carlini and Wagner (C&W) attack Set of optimization-based attacks that generate L₀, L₂ and L∞ norm measured adversarial samples, with some restrictions (kappa) to make sure a valid image is produced 100% attack success on ‘normal’ neural networks trained on MNIST, CIFAR-10, ImageNet Compromised defensive models
  • 12. Notable adversarial models DeepFool “Iterative linearization of the classifier to generate minimal perturbations that are sufficient to change classification labels” Computes perturbations more reliably Moosavi-Dezfooli et al., https://arxiv.org/pdf/1511.04599.pdf
  • 13.
  • 14. Notable adversarial models Universal adversarial attack ● Is there a universal perturbation that will work on most samples? ● L-BFGS- based ● Effective in attacking NN like CaffeNet, GoogleNet, VGG and Resnet ● Fooling rate > 53%
  • 15. Adversarial patch Only certain areas are perturbed
  • 16. Text & Audio Models 1% audio perturbation can change 50 words in text transcription! Attacks are robust to MP3 compression, but get lost when played over speakers https://nicholas.carlini.com/code/audio_adversarial_examples/ Strategies for text attacks generally include deleting, inserting, and modifying characters/words
  • 17. Adversarial defenses fall into 5 categories 1. Training on adversarial samples 2. Randomization 3. Adding noise 4. Removing noise 5. Mathematically provable defenses
  • 18. Defang: Randomize input or features ● Randomly padding and resizing input; image transformations with randomness
  • 19. ● Add random noise layer before each convolutional layer in training and test sets (RSE) ● Random feature pruning at each layer
  • 20. Detect: Denoise the input or features ● Conventional input rectification ○ ‘Squeeze’ image → if output is very different from input, then likely adversarial ● GAN-based ○ Use GAN to learn benign data distribution ○ Generate a benign projection for the adversarial sample ● Autoencoder-based ○ Detector & reformer ○ Use an autocoder to compress input and learn manifold of benign samples ○ Detector compares each sample to learnt manifold ○ Reformer rectifies adversarial samples
  • 21. Detect: Denoise the input or features ● High-level representation guided denoiser (HGD) ○ Trains a denoising u-net using a feature-level loss function to minimize feature differences between benign and adversarial samples ○ Won first place in black-box defenses, 2017 ○ Even so, certain (white-box) attacks can reduce effectiveness to 0%
  • 22. Provable (certificated) defenses ● Defenses that have theoretical backing to have a certain accuracy against attacks ● Range of defenses include KNN and Bayesian-based defenses ● Consistency-based defenses: ○ Perturbations also affect the area around them ○ > 90 detection rate ● Very computationally intensive
  • 23. Trends in adversarial research ● Design stronger attacks to probe for weaknesses ● Real-world attack capabilities ● Certificated defenses - but currently not scalable “A problem is that an attack can only target one category of defenses, but defenses are required to … be effective against all possible attack methods” ● Analyzing model robustness - mostly done on KKN and linear classifiers
  • 24. Unresolved challenges ● Causality ● Does a general robust decision boundary exist that could be learnt by (certain) neural networks? ● Effectiveness vs efficiency ○ Adversarial training is effective, but requires a lot of data and compute ○ Randomization and denoising strategies very efficient, but not as effective as claimed
  • 25. Discussion In what other ways are models not robust? Is model robustness/ security applicable to what you do / to our students? Model fairness has been a hot topic lately, but robustness/ security seems to lag behind - what do you think needs to change for adversarial training to be widely implemented? What are your thoughts on the paper in general?
  • 26. Try it yourself Benchmark machine learning systems' vulnerability to adversarial examples: https://github.com/cleverhans-lab/cleverhans Blog: cleverhans.io