Recommended
More Related Content
What's hot
What's hot (20)
Similar to Towards Secure and Interpretable AI: Scalable Methods, Interactive Visualizations, Practical Tools
Similar to Towards Secure and Interpretable AI: Scalable Methods, Interactive Visualizations, Practical Tools (20)
Recently uploaded
Recently uploaded (20)
Towards Secure and Interpretable AI: Scalable Methods, Interactive Visualizations, Practical Tools
- 1. INTERPRETABLE AI TOWARDS SECURE & SCALABLE METHODS, INTERACTIVE VISUALIZATIONS, PRACTICAL TOOLS Polo Chau Associate Professor Associate Director, MS Analytics Georgia Tech poloclub.github.io
- 2. AI HI+ HUMAN INTELLIGENCE ARTIFICIAL INTELLIGENCE Scalable interactive tools to make sense of complex large-scale datasets and models Polo Club of Data Science
- 3. Human-Centered AI Cyber Security Social Good & HealthLarge Graph Mining & Visualization Polo Club of Data Science poloclub.github.io Adversarial ML
- 4. Why focus on them? How are they related? Today’s Main Topics SecureAI InterpretableAI
- 5. New York Times, 2018 AI now used in safety-critical applications. Important to study threats & countermeasures. SecureAI
- 6. AI Security Problems Are Everywhere Smart toaster does exist!
- 7. Source: Cisco Source: US Department of Homeland Security Increased > 10-fold # incidents reported by U.S. federal agencies AI Security is becoming increasingly important
- 8. ? How do we know if a defense for AI is working?
- 9. AI models often used as black-box
- 10. InterpretableAI
- 11. RESEARCH PROBLEM Via scalable, interactive, usable interfaces to help people understand complex, large-scale ML systems. InterpretableAI
- 12. SecureAI InterpretableAI Do-it-yourself Adversarial ML Interactive Learning (Education) Understand Industry ModelsAttack & Defense (DNN) ADAGIO MLsploit ActiVis GAN Lab Research landscape Survey ShapeShifter SHIELD
- 13. Study ML vulnerabilities and develop secure AI for high-stakes problems Our Goal
- 14. Attack & Defense of Deep Neural Networks Do-it-yourself Adversarial ML ShapeShifter - Physical Adversarial Attack SHIELD - Real-time Defense for Images ADAGIO - Experimentation with Real-time Defense for Audio MLsploit - Interactive Experimentation with Adversarial ML Secure AI
- 15. First Targeted Physical Adversarial Attack for Object Detection Shang-Tse Chen Intel Cory Cornelius Jason Martin Polo Chau IntelGeorgia Tech Georgia Tech ShapeShifter ECML-PKDD 2018
- 16. Image Classification output a single label, e.g., “car”
- 17. Object Detection recognize and localize multiple objects!
- 18. Deep Neural Networks are vulnerable
- 19. Classified as Stop Sign Deep Neural Networks are vulnerable
- 20. Benign Image Classified as Stop Sign Misclassified as Max Speed 100 😈 Adversarial Perturbation Deep Neural Networks are vulnerable But most attacks have impractical threat model
- 22. Capture Preprocess Recognition 😈 Digital Attack Physically Realizable Adversarial Attack Attacker has no access to internal pipeline Manipulate Physical Environment = More Realistic, Targeted Attack Autonomous car system
- 23. Stop Sign ! Person Printed Adversarial Stop SignReal Stop Sign
- 24. Prior Work on Physical Attacks Glasses that fool a face classifier [Sharif et al. CCS’16] 3D objects that fool an image classifier [Athalye et al. ICML’18] Stickers that fool a traffic sign classifier [Evtimov et al. CVPR’18] They all focus on attacking image classifiers
- 25. Attack Object Detectors: Naïve Approach Lu et al. [1] show the current technique cannot fool state-of-the-art object detectors like Faster R-CNN and YOLO [1] Standard detectors aren’t (currently) fooled by physical adversarial stop signs. Lu et al., arXiv ‘17
- 26. Brief Overview of Faster R-CNN A state-of-the-art Object Detector Model
- 27. Brief Overview of Faster R-CNN Stage 1: Generate region proposals Stage 2: Refined localization and classification feature map sharing
- 28. Challenges of Physically Attacking Faster R-CNN 1. Multiple region proposals 2. Distances, angles, lightings
- 29. Our Solution: Fool Multiple Region Proposals Minimize: sum of classification losses Only perturb RED area Human eye is less sensitive to changes in darker color ≈ + deviation loss
- 30. Our Solution: Robust to Real-World Distortions Adapt Expectation over Transformation [Athalye et al, ICML’18] Optimize over different backgrounds, scales, rotations, lightings
- 32. ShapeShifter Motivates DARPA Program GARD (Defense for AI) Highlights ShapeShifter as the state-of-the-art physical attack https://www.darpa.mil/attachments/GARD_ProposersDay.pdf
- 33. [Open-sourced] Fast, Practical Defense for Image Classification Nilaksh Das Madhuri Shangbogue Shang-Tse Chen Fred Hohman Li Chen Michael Kounavis Siwei Li Polo Chau SHIELD KDD’18 Audience Appreciation Award (runner-up) KDD’19 LEMINCS Cory Cornelius
- 34. Adversarial Machine Learning Landscape Attack Defense Our Focus: Fast & Practical (digital)
- 36. SHIELD leverages JPEG compression SHIELD’s SLQ applies JPEG compression of a random quality to each 8 x 8 block of the image JPEQ Quality 80 JPEQ Quality 60 JPEQ Quality 40 JPEQ Quality 20 * larger blocks shown for presentation
- 37. SHIELD is multi-pronged approach that incorporates - Stochastic Local Quantization - Model Vaccination (re-training) - Ensembling to mitigate adversarial attacks
- 38. Results with ResNet-50 v2 (on ImageNet validation set) higher is better increasing attack perturbation
- 39. tested on 50,000 images from the ImageNet validation set
- 40. ADAGIO incorporates compression as defense, which blocks the gradient to the attacker. Adversarial Attack on Speech-to-Text
- 41. ADAGIO Interactive Experimentation with Adversarial Attack & Defense for Audio 🗣 Upload your own audio sample ⚔ Perform audio adversarial attack 🔰 Apply compression to defend ▶ Play audio, listen for differences ADAGIO = Attack & Defense for Audio in a Gadget with Interactive Operations [PKDD18]
- 43. Attack & Defense of Deep Neural Networks Do-it-yourself Adversarial ML ShapeShifter - Physical Adversarial Attack SHIELD - Real-time Defense for Images ADAGIO - Experimentation with Real-time Defense for Audio MLsploit - Interactive Experimentation with Adversarial ML Secure AI
- 44. A Framework for Interactive Experimentation with Adversarial Machine Learning Research github.com/mlsploit Contributors from Intel Science and Technology Center for Adversary-Resilient Security Analytics: Nilaksh Das, Siwei Li, Chanil Jeon, Jinho Jung*, Shang-Tse Chen*, Carter Yagemann*, Evan Downing*, Haekyu Park, Evan Yang, Li Chen, Michael Kounavis, Ravi Sahita, David Durham, Scott Buck, Polo Chau, Taesoo Kim, Wenke Lee (*equal contribution) [BlackHat Asia ’19, KDD’19 Showcase]
- 45. MLsploit ★ Research modules for adversarial ML ✴ Enables comparison of attacks and defenses ★ Interactive experimentation with ML research ★ Researchers can easily integrate novel research into an intuitive and seamless user interface
- 46. MLsploit ★ AVPass (leaking and bypassing Android malware detection systems) ★ ELF (bypassing Linux malware detection with API perturbation) ★ PE (create and attack ML models for detecting Windows PE malware) ★ Intel®-Software Guard Extensions (privacy preserving adversarial ML as a service) ★ SHIELD (attack and defend state-of-the-art image classification models) ✴ Attacks: FGSM, DeepFool, Carlini-Wagner ✴ Defenses: SLQ, JPEG, Median Filter, TV-Bregman
- 49. ONE-STEP INSTALLATION docker-compose up --build
- 52. SecureAI InterpretableAI Do-it-yourself Adversarial ML Interactive Learning (Education) Understand Industry ModelsAttack & Defense (DNN) ADAGIO MLsploit ActiVis GAN Lab Research landscape Survey ShapeShifter SHIELD
- 53. Practitioners very interested in why and how AI works Figure from Washington Post
- 54. OUR KEY IDEA Scalable Interactive visualization as a medium for connecting users with ML models
- 55. Why interactive visualization? Machine learning aims to find patterns from data. Visualization amplifies human cognition to find patterns.
- 56. Why interactive visualization? By interacting with visualization, users can incrementally make sense of AI models.
- 57. Understanding Industry-Scale Models Interactive Learning of Complex Models ActiVis - Activation analysis by subsets GAN Lab - Experimentation with GANs Interpretable AI via Visual Analytics Research Landscape Survey
- 58. IEEE VIS 2017 ActiVis Scaling Visualization to Industry-scale Models & Data Deployed by Minsuk Kahng Pierre Andrews Aditya Kalro Polo Chau Georgia Tech Georgia TechFacebook Facebook
- 59. why practitioners want visualization? Facebook data scientists need visualization tools to interpret complex models
- 60. Those down totags: #mycat, #cute date: 10/1/2017 location: 33.7, 88.4 Practical Design Challenges DIVERSE FEATURESDEEP/WIDE MODELS LARGE DATASETS image, text, numerical, categorical, … 1,000+ operations/layers 1 billion+ instances Enjoying nice weather with kiki❤
- 61. UNDERSTANDING USERS’ NEEDS Participatory design sessions with 15+ researchers, engineers & data scientists at Facebook over 11 months
- 62. Visualizing activation of industry-scale deep neural nets, deployed by Facebook ActiVis
- 63. Location How to visualize many model parameters? Challenge #1 INPUT OUTPUTMODEL Person Location 81% 8% Where is Mercedes-Benz Stadium located? Number 11% particularly useful many layers Observation: No need to show everything
- 64. Model Overview to Activation Details 64 ActiVis Key Ideas #1
- 65. How to analyze many data instances? Challenge #2 SUBSET-LEVELINSTANCE-LEVEL Complementary Useful for debugging Useful for large datasets Observation: Two Analytics Patterns How model behaves at higher-level categorization (e.g., by topic)? How model responds to individual instances?
- 66. Unified Analysis for Instances & Subsets ActiVis Key Ideas (2)
- 67. Scaling Up ActiVis for Facebook 1. User-guided Instance Sampling 2. Selective Pre-computation of Layers 3. Matrix Computation for Billion-Scale Instances ActiVis Key Ideas (3)
- 68. Deployed on FBLearner used by >25% of engineering team Facebook’s ML platform
- 69. IEEE VIS 2019
- 71. 71
- 72. Understanding Industry-Scale Models Interactive Learning of Complex Models ActiVis - Activation analysis by subsets GAN Lab - Experimentation with GANs Interpretable AI via Visual Analytics Research Landscape Survey
- 73. GAN Lab Understanding Complex Deep Generative Models using Interactive Visual Experimentation Georgia Tech Google Minsuk Kahng Nikhil Thorat Polo Chau Fernanda Viégas Martin Wattenberg Georgia Tech Google Google PAIR | People + AI Research Initiative
- 74. Visualization for ML Education
- 75. Modern deep models are complex 75
- 76. Generative Adversarial Networks (GANs) 76 “the most interesting idea in the last 10 years in ML” - Yann LeCun Face images generated by BEGAN [Berthelot et al., 2017]
- 77. Generative Adversarial Networks (GANs) Hard to understand and train even for experts 77 Discriminator Generator
- 78. Why GANs are hard? A GAN uses two competing neural networks 78 Discriminator spots fake Police spots fake bills Generator synthesizes outputs Counterfeiter makes fake bills
- 79. GAN Lab Research Challenges 1. Conceptual understanding of GANs 2. Interactive model training 3. Easily accessible by anyone Can we design an interactive tool for GANs?
- 80. What type of data to visualize? 2D distribution, instead of high-dimensional images 80 Discriminator (Police) Generator (Counterfeiter)
- 81. What type of data to visualize? 2D distribution, instead of high-dimensional images 81 Discriminator (Police) Generator (Counterfeiter) 1. To focus on GAN’s main concepts 2. To easily visualize data distribution Why 2D data points?
- 83. Discriminator (Police) How to visually explain the generator? 83 Generator (Counterfeiter)
- 84. How to visually explain the generator? map an input point into a new position random Manifold ? 84 Generator (Counterfeiter)
- 85. How to visualize the discriminator? 85 Generator (Counterfeiter) Discriminator (Police)
- 86. How to visualize the discriminator? 2D heatmap, to represent binary classification 86 Data points in this region are likely real. Data points are likely fake.
- 87. VER. 0.5 87 realdata fakedata generator discriminator + + +
- 88. VER. 0.5 88 Hard to develop mental models for GANs
- 90. 90
- 91. GAN Lab broadens education access 91 Conventional Deep Learning Visualization in JavaScript in Python with GPU Model Training Visualization $$$
- 92. Everything done in browser, powered by TensorFlow.js GAN Lab broadens education access 92 Accelerated by WebGL in JavaScript Visualization also in JavaScript Model Training
- 93. GAN Lab is Live! 93 30K visitors, 135 countries 1.9K Likes 800+ Retweets Try at bit.ly/gan-lab
- 94. Understanding Industry-Scale Models Interactive Learning of Complex Models ActiVis - Activation analysis by subsets GAN Lab - Experimentation with GANs Interpretable AI via Visual Analytics Research Landscape Survey
- 95. Visual Analytics in Deep Learning An Interrogative Survey for the Next Frontiers Fred Hohman Georgia Tech TVCG 2018 Minsuk Kahng Georgia Tech Polo Chau Georgia Tech Robert Pienta Symantec
- 96. 96
- 97. Key Takeaways 1. Most tools aimed at expert users 2. Instance-based analysis 3. Inherently interdisciplinary 4. Lacks actionability 5. Evaluation is hard 6. State-of-the-art models not robust bit.ly/va-dl-survey
- 98. SecureAI InterpretableAI Do-it-yourself Adversarial ML ML Education Understand Industry ModelsAttack & Defense (DNN) ADAGIO MLsploit ActiVis GAN Lab Research landscape Survey ShapeShifter SHIELD
- 99. INTERPRETABLE AI TOWARDS SECURE & SCALABLE METHODS, INTERACTIVE VISUALIZATIONS, PRACTICAL TOOLS Polo Chau Georgia Tech poloclub.github.io Thanks!