Downloaded 96 times
Uploaded byPriyanka Aash
Security and Privacy of Machine Learning
The document presents insights into machine learning's intersection with security, highlighting malware detection, privacy of training data, and various attack vectors such as adversarial examples and model theft. It outlines the significance of differential privacy, training set poisoning, and the challenges faced by current defenses based on unrealistic threat models. The presentation also emphasizes the need for robust methodologies when deploying machine learning models, particularly those that handle sensitive data.
In this document
Powered by AI
Session introduces Ian Goodfellow discussing machine learning security and privacy, outlining concepts, and emphasizing the collaborative nature of the work.
Describes components of machine learning pipeline, including training data, algorithms, learned parameters, testing inputs, and outputs.
Focus on training data privacy, differential privacy definitions, and implications of training set poisoning.
Discusses adversarial examples, model theft, and transfer attacks, showing the vulnerabilities in models that allow malicious exploitation.
Explores cross-model transfer, the implications of transfer attacks on human cognition, and their relevance in the physical world.
Contrasts adversarial training with certified defenses, discussing their effectiveness and assumptions of threat models.
Introduces the Clever Hans effect in algorithms and encourages engagement and involvement in the field.
Highlights the importance of considering data sensitivity, adversarial potential, and the practical limitations of current defenses.
More Related Content
Similar to Security and Privacy of Machine Learning
Security and Privacy of Machine Learning
- 1. SESSION ID: #RSAC Ian Goodfellow SECURITYAND PRIVACY OF MACHINE LEARNING Staff Research Scientist Google Brain @goodfellow_ian
- 2. (Goodfellow 2018) #RSAC Machine Learningand Security 2 Machine Learning for Security Malware detection Intrusion detection … Security against Attacks that use Machine Learning Password guessing Fake reviews …
- 3. (Goodfellow 2018) #RSAC Security ofMachine Learning 3
- 4. (Goodfellow 2018) #RSAC An overviewof a field 4 This presentation summarizes the work of many people, not just my own / my collaborators Download the slides for this link to extensive references The presentation focuses on the concepts, not the history or the inventors
- 5. (Goodfellow 2018) #RSAC Machine LearningPipeline 5 Training data Learning algorithm Learned parameters Test input Test output
- 6. (Goodfellow 2018) #RSAC Privacy ofTraining Data 6
- 7. (Goodfellow 2018) #RSAC Defining (ε,δ)-Differential Privacy 7 (Abadi 2017)
- 8. (Goodfellow 2018) #RSAC Private Aggregationof Teacher Ensembles 8 (Papernot et al 2016)
- 9. (Goodfellow 2018) #RSAC Training SetPoisoning 9
- 10. (Goodfellow 2018) #RSAC ImageNet Poisoning 10 (Kohand Liang 2017)
- 11.
- 12.
- 13.
- 14. (Goodfellow 2018) #RSAC Deep Diveon Adversarial Examples 14 ...solving CAPTCHAS and reading addresses... ...recognizing objects and faces…. (Szegedy et al, 2014) (Goodfellow et al, 2013) (Taigmen et al, 2013) (Goodfellow et al, 2013) and other tasks... Since 2013, deep neural networks have matched human performance at...
- 15.
- 16. (Goodfellow 2018) #RSAC Turning objectsinto airplanes 16
- 17. (Goodfellow 2018) #RSAC Attacking alinear model 17
- 18. (Goodfellow 2018) #RSAC Wrong almosteverywhere 18
- 19.
- 20. (Goodfellow 2018) #RSAC Transfer acrosslearning algorithms 20 (Papernot 2016)
- 21. (Goodfellow 2018) #RSAC Transfer attack 21 Trainyour own model Target model with unknown weights, machine learning algorithm, training set; maybe non- differentiable Substitute model mimicking target model with known, differentiable function Adversarial examples Adversarial crafting against substitute Deploy adversarial examples against the target; transferability property results in them succeeding
- 22. (Goodfellow 2018) #RSAC Enhancing Transferwith Ensembles 22 (Liu et al, 2016)
- 23. (Goodfellow 2018) #RSAC Transfer tothe Human Brain 23 (Elsayed et al, 2018)
- 24. (Goodfellow 2018) #RSAC Transfer tothe Physical World 24 (Kurakin et al, 2016)
- 25.
- 26. (Goodfellow 2018) #RSAC Adversarial Trainingvs Certified Defenses 26 Adversarial Training: Train on adversarial examples This minimizes a lower bound on the true worst-case error Achieves a high amount of (empirically tested) robustness on small to medium datasets Certified defenses Minimize an upper bound on true worst-case error Robustness is guaranteed, but amount of robustness is small Verification of models that weren’t trained to be easy to verify is hard
- 27. (Goodfellow 2018) #RSAC Limitations ofdefenses 27 Even certified defenses so far assume unrealistic threat model Typical model: attacker can change input within some norm ball Real attacks will be stranger, hard to characterize ahead of time (Brown et al., 2017)
- 28. (Goodfellow 2018) #RSAC Clever Hans 28 (“CleverHans, Clever Algorithms,” Bob Sturm)
- 29.
- 30. (Goodfellow 2018) #RSAC Apply WhatYou Have Learned 30 Publishing an ML model or a prediction API? Is the training data sensitive? -> train with differential privacy Consider how an attacker could cause damage by fooling your model Current defenses are not practical Rely on situations with no incentive to cause harm / limited amount of potential harm