Successfully reported this slideshow.
Your SlideShare is downloading. ×

ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models

ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models

Presentation slides for the following paper presented at IEEE VIS 2017 (http://ieeevis.org).
Minsuk Kahng, Pierre Y. Andrews, Aditya Kalro, and Duen Horng (Polo) Chau. ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models. IEEE Transactions on Visualization and Computer Graphics, Vol. 24, No. 1 (VAST 2017).

Presentation slides for the following paper presented at IEEE VIS 2017 (http://ieeevis.org).
Minsuk Kahng, Pierre Y. Andrews, Aditya Kalro, and Duen Horng (Polo) Chau. ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models. IEEE Transactions on Visualization and Computer Graphics, Vol. 24, No. 1 (VAST 2017).

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models

  1. 1. ActiVis Visual Exploration of Industry-Scale Deep Neural Network Models Minsuk Kahng Pierre Andrews Aditya Kalro Polo Chau Georgia Tech Georgia TechFacebook Facebook *
  2. 2. Deep Learning is Powerful 2Image credit:http://www.nvidia.com/object/drive-px.html, https://venturebeat.com/2016/04/14/, https://finance.yahoo.com/news/
  3. 3. Understanding Deep Learning is Challenging 3 Cat Dog Cat Dog INPUT OUTPUTMODEL Image credit: https://www.kaggle.com/c/dogs-vs-cats/ Incorrect
  4. 4. Cat Dog Visualizing Instance Activation 4 Which neurons are highly activated for a given input? INPUT OUTPUTMODEL Cat Dog 78% 22% [Yosinski et al., 2015; Harley, 2015; Karpathy, 2016; Liu et al., 2017]
  5. 5. tags: #mycat, #cute date: 10/1/2017 location: 33.7, 88.4 Practical Challenges in Industry 5 DIVERSE INPUT TYPESCOMPLEX MODELS LARGE DATASETS image, text, numerical, categorical, … many nodes in graph-structure 1 billion+ instances 1. 2. 3. Develop ActiVis for Facebook-scale models and dataGOAL: Enjoying nice weather with kiki❤️
  6. 6. Understanding ActiVis Users’ Needs Participatory design sessions over 11 months, with 15+ Facebook researchers, engineers, & data scientists 6
  7. 7. Complex Model Architectures UNDERSTANDING ACTIVIS USERS’ NEEDS (1/3) 7 Numerous deep & wide models are used. input output Separate architecture from activation detailsGOAL:
  8. 8. Huge Datasets with Diverse Features UNDERSTANDING ACTIVIS USERS’ NEEDS (2/3) 1 billion+ instances 8 1,000+ multi-type features (e.g., image, text, numerical, categorical) tags: #mycat, #cute date: 10/1/2017 location: 33.7, 88.4 Enjoying nice weather with kiki❤️ Use multiple approaches for scalability GOAL: Make use of diverse features
  9. 9. Two Key Analytics Patterns UNDERSTANDING ACTIVIS USERS’ NEEDS (3/3) How model responds to individual instances? (This instance highly activates neurons #2, 5, 11.) How model behaves at higher-level categorization (e.g., by topic)? (A subset of instances about “sports” highly activates neurons #3, 7, 11.) Useful for debugging Useful for large datasets SUBSET-LEVELINSTANCE-LEVEL 9 [Kahng et al., 2016; Krause et al., 2016] Complementary [Kulesza et al., 2015; Amershi et al., 2015] Support both analyticsGOAL:
  10. 10. ActiVis Design Goals (Recap) 10 Two analytics patterns Complex models Huge datasets 1. 2. 3. Unified analysis for instances & subsets Model overview as entry point Multiple approaches for scalability
  11. 11. ActiVis Visualizing activation of industry-scale deep neural nets Deployed on Facebook ML Platform 11
  12. 12. 12 demo: “Where is Phoenix located?” → location “What is the diameter of a golf ball?” → numeric Exploring text classification results ActiVis Demo
  13. 13. Demo 13
  14. 14. • Unified analysis for instances & subsets • Model architecture to activation details • Scaling to industry-scale data & models 14 ActiVis Key Ideas (Demo Recap)
  15. 15. User-guided Instance Sampling 15 SCALING TO LARGE DATA & MODELS (1/2) e.g., “What does VAST mean?” should be in ABBR class. Users either want representative samples or maintain “test cases”.
  16. 16. Selective Precomputation for Important Nodes 16 SCALING TO LARGE DATA & MODELS (2/2) Often only a few nodes are helpful and model developers know them. precomputed
  17. 17. Deployed on FBLearner 17 Model developers add 3 API calls to enable ActiVis. Facebook ML platform, used by 25% of their engineers Click to launch ActiVis
  18. 18. Case Studies 3 Facebook participants. All work with text classification models. Each session 60 minutes long. 18
  19. 19. Spot1. -checking models with “test cases” “Where is … located?” → location Graph architecture view as entry point2. Debugging hints from activation patterns3. 19 Key Observations from Case Studies
  20. 20. • Discover interesting subsets interactively • Support input-dependent models (e.g., RNN) • Provide direct guidance for performance improvement 20 Future Research Directions
  21. 21. ActiVis Visual Exploration of Industry-Scale Deep Neural Network Models ✓ Deployed on Facebook’s ML platform ✓ Support subset-level analysis ✓ Model architecture to activation details We thank Facebook Applied Machine Learning Group, especially Yangqing Jia, Andrew Tulloch, Liang Xiong, and Zhao Tan and NSF Graduate Research Fellowship Program. Pierre Andrews Aditya Kalro Polo Chau Georgia Tech Facebook Facebook Minsuk Kahng Georgia Tech PhD student http://minsuk.com

×