Interpreting complex machine learning models can be difficult. Given an interpretation, its meaningfulness and reliability are hard to evaluate. Even more, depending on the purpose (debugging, ...), a technique in the literature may be more appropriate than others. How to choose the best approach in the landscape of the existing techniques?
This talk is organized as a virtual "walk" through different techniques for interpreting machine learning, and particularly deep learning. Moving from the inside out, we will first cover techniques (such as gradient ascent and deconvolution) for interpreting the internal state of the model, namely its neurons, channels and layer activations. We will then focus on the model behavior from the outside. The model output, for instance, can be explained by attributing the final decisions to subsets of input pixels (as in saliency, occlusion and class activation maps) or to higher-level concepts, such as object size, scale and texture. Concept-based attribution, in particular, has been our research focus over the last years, allowing us to explain deep learning in simple terms to clinicians. For this, digital pathology and retinopathy were our main application domains. In addition, concept-based interpretability helped us explain internal CNN mechanisms such as the encoding of scale and memorization of input-label pairs.
The importance of model fairness and interpretability in AI systemsFrancesca Lazzeri, PhD
Machine learning model fairness and interpretability are critical for data scientists, researchers and developers to explain their models and understand the value and accuracy of their findings. Interpretability is also important to debug machine learning models and make informed decisions about how to improve them.
In this session, Francesca will go over a few methods and tools that enable you to "unpack” machine learning models, gain insights into how and why they produce specific results, assess your AI systems fairness and mitigate any observed fairness issues.
Using open-source fairness and interpretability packages, attendees will learn how to:
- Explain model prediction by generating feature importance values for the entire model and/or individual data points.
- Achieve model interpretability on real-world datasets at scale, during training and inference.
- Use an interactive visualization dashboard to discover patterns in data and explanations at training time.
- Leverage additional interactive visualizations to assess which groups of users might be negatively impacted by a model and compare multiple models in terms of their fairness and performance.
This document discusses improving the interpretability of RASA NLU models through machine learning techniques. It introduces interpretable machine learning and how tools like ScatterText and LIME can be used to analyze RASA NLU training data and models. These techniques help identify confusing intents, common words between intents, and explain model predictions. The goal is to troubleshoot models and refine training data to improve natural language understanding.
Part of the ongoing effort with Skater for enabling better Model Interpretation for Deep Neural Network models presented at the AI Conference.
https://conferences.oreilly.com/artificial-intelligence/ai-ny/public/schedule/detail/65118
Human in the loop: Bayesian Rules Enabling Explainable AIPramit Choudhary
The document provides an overview of a presentation on enabling explainable artificial intelligence through Bayesian rule lists. Some key points:
- The presentation will cover challenges with model opacity, defining interpretability, and how Bayesian rule lists can be used to build naturally interpretable models through rule extraction.
- Bayesian rule lists work well for tabular datasets and generate human-understandable "if-then-else" rules. They aim to optimize over pre-mined frequent patterns to construct an ordered set of conditional statements.
- There is often a tension between model performance and interpretability. Bayesian rule lists can achieve accuracy comparable to more opaque models like random forests on benchmark datasets while maintaining interpretability.
1. The document discusses model interpretation and techniques for interpreting machine learning models, especially deep neural networks.
2. It describes what model interpretation is, its importance and benefits, and provides examples of interpretability algorithms like dimensionality reduction, manifold learning, and visualization techniques.
3. The document aims to help make machine learning models more transparent and understandable to humans in order to build trust and improve model evaluation, debugging and feature engineering.
Interpretable machine learning : Methods for understanding complex modelsManojit Nandi
1. Interpretability helps understand complex machine learning models by explaining their outcomes based on inputs. Higher predictive accuracy often reduces interpretability.
2. Methods like LIME and SHAP attribute model outcomes to input features through local surrogate models and game theory.
3. Recourse analysis identifies actions individuals could take to improve outcomes from automated decisions.
Applied Artificial Intelligence Unit 4 Semester 3 MSc IT Part 2 Mumbai Univer...Madhav Mishra
The document discusses various topics related to evolutionary computation and artificial intelligence, including:
- Evolutionary computation concepts like genetic algorithms, genetic programming, evolutionary programming, and swarm intelligence approaches like ant colony optimization and particle swarm optimization.
- The use of intelligent agents in artificial intelligence and differences between single and multi-agent systems.
- Soft computing techniques involving fuzzy logic, machine learning, probabilistic reasoning and other approaches.
- Specific concepts discussed in more depth include genetic algorithms, genetic programming, swarm intelligence, ant colony optimization, and metaheuristics.
This was presented at the London Artificial Intelligence & Deep Learning Meetup.
https://www.meetup.com/London-Artificial-Intelligence-Deep-Learning/events/245251725/
Enjoy the recording: https://youtu.be/CY3t11vuuOM.
- - -
Kasia discussed complexities of interpreting black-box algorithms and how these may affect some industries. She presented the most popular methods of interpreting Machine Learning classifiers, for example, feature importance or partial dependence plots and Bayesian networks. Finally, she introduced Local Interpretable Model-Agnostic Explanations (LIME) framework for explaining predictions of black-box learners – including text- and image-based models - using breast cancer data as a specific case scenario.
Kasia Kulma is a Data Scientist at Aviva with a soft spot for R. She obtained a PhD (Uppsala University, Sweden) in evolutionary biology in 2013 and has been working on all things data ever since. For example, she has built recommender systems, customer segmentations, predictive models and now she is leading an NLP project at the UK’s leading insurer. In spare time she tries to relax by hiking & camping, but if that doesn’t work ;) she co-organizes R-Ladies meetups and writes a data science blog R-tastic (https://kkulma.github.io/).
https://www.linkedin.com/in/kasia-kulma-phd-7695b923/
The importance of model fairness and interpretability in AI systemsFrancesca Lazzeri, PhD
Machine learning model fairness and interpretability are critical for data scientists, researchers and developers to explain their models and understand the value and accuracy of their findings. Interpretability is also important to debug machine learning models and make informed decisions about how to improve them.
In this session, Francesca will go over a few methods and tools that enable you to "unpack” machine learning models, gain insights into how and why they produce specific results, assess your AI systems fairness and mitigate any observed fairness issues.
Using open-source fairness and interpretability packages, attendees will learn how to:
- Explain model prediction by generating feature importance values for the entire model and/or individual data points.
- Achieve model interpretability on real-world datasets at scale, during training and inference.
- Use an interactive visualization dashboard to discover patterns in data and explanations at training time.
- Leverage additional interactive visualizations to assess which groups of users might be negatively impacted by a model and compare multiple models in terms of their fairness and performance.
This document discusses improving the interpretability of RASA NLU models through machine learning techniques. It introduces interpretable machine learning and how tools like ScatterText and LIME can be used to analyze RASA NLU training data and models. These techniques help identify confusing intents, common words between intents, and explain model predictions. The goal is to troubleshoot models and refine training data to improve natural language understanding.
Part of the ongoing effort with Skater for enabling better Model Interpretation for Deep Neural Network models presented at the AI Conference.
https://conferences.oreilly.com/artificial-intelligence/ai-ny/public/schedule/detail/65118
Human in the loop: Bayesian Rules Enabling Explainable AIPramit Choudhary
The document provides an overview of a presentation on enabling explainable artificial intelligence through Bayesian rule lists. Some key points:
- The presentation will cover challenges with model opacity, defining interpretability, and how Bayesian rule lists can be used to build naturally interpretable models through rule extraction.
- Bayesian rule lists work well for tabular datasets and generate human-understandable "if-then-else" rules. They aim to optimize over pre-mined frequent patterns to construct an ordered set of conditional statements.
- There is often a tension between model performance and interpretability. Bayesian rule lists can achieve accuracy comparable to more opaque models like random forests on benchmark datasets while maintaining interpretability.
1. The document discusses model interpretation and techniques for interpreting machine learning models, especially deep neural networks.
2. It describes what model interpretation is, its importance and benefits, and provides examples of interpretability algorithms like dimensionality reduction, manifold learning, and visualization techniques.
3. The document aims to help make machine learning models more transparent and understandable to humans in order to build trust and improve model evaluation, debugging and feature engineering.
Interpretable machine learning : Methods for understanding complex modelsManojit Nandi
1. Interpretability helps understand complex machine learning models by explaining their outcomes based on inputs. Higher predictive accuracy often reduces interpretability.
2. Methods like LIME and SHAP attribute model outcomes to input features through local surrogate models and game theory.
3. Recourse analysis identifies actions individuals could take to improve outcomes from automated decisions.
Applied Artificial Intelligence Unit 4 Semester 3 MSc IT Part 2 Mumbai Univer...Madhav Mishra
The document discusses various topics related to evolutionary computation and artificial intelligence, including:
- Evolutionary computation concepts like genetic algorithms, genetic programming, evolutionary programming, and swarm intelligence approaches like ant colony optimization and particle swarm optimization.
- The use of intelligent agents in artificial intelligence and differences between single and multi-agent systems.
- Soft computing techniques involving fuzzy logic, machine learning, probabilistic reasoning and other approaches.
- Specific concepts discussed in more depth include genetic algorithms, genetic programming, swarm intelligence, ant colony optimization, and metaheuristics.
This was presented at the London Artificial Intelligence & Deep Learning Meetup.
https://www.meetup.com/London-Artificial-Intelligence-Deep-Learning/events/245251725/
Enjoy the recording: https://youtu.be/CY3t11vuuOM.
- - -
Kasia discussed complexities of interpreting black-box algorithms and how these may affect some industries. She presented the most popular methods of interpreting Machine Learning classifiers, for example, feature importance or partial dependence plots and Bayesian networks. Finally, she introduced Local Interpretable Model-Agnostic Explanations (LIME) framework for explaining predictions of black-box learners – including text- and image-based models - using breast cancer data as a specific case scenario.
Kasia Kulma is a Data Scientist at Aviva with a soft spot for R. She obtained a PhD (Uppsala University, Sweden) in evolutionary biology in 2013 and has been working on all things data ever since. For example, she has built recommender systems, customer segmentations, predictive models and now she is leading an NLP project at the UK’s leading insurer. In spare time she tries to relax by hiking & camping, but if that doesn’t work ;) she co-organizes R-Ladies meetups and writes a data science blog R-tastic (https://kkulma.github.io/).
https://www.linkedin.com/in/kasia-kulma-phd-7695b923/
This document provides an overview of machine learning applications in natural language processing and text classification. It discusses common machine learning tasks like part-of-speech tagging, named entity extraction, and text classification. Popular machine learning algorithms for classification are described, including k-nearest neighbors, Rocchio classification, support vector machines, bagging, and boosting. The document argues that machine learning can be used to solve complex real-world problems and that text processing is one area with many potential applications of these techniques.
Introduction to machine learning and model building using linear regressionGirish Gore
An basic introduction of Machine learning and a kick start to model building process using Linear Regression. Covers fundamentals of Data Science field called Machine Learning covering the fundamental topic of supervised learning method called linear regression. Importantly it covers this using R language and throws light on how to interpret linear regression results of a model. Interpretation of results , tuning and accuracy metrics like RMSE Root Mean Squared Error are covered here.
Applied Artificial Intelligence Unit 3 Semester 3 MSc IT Part 2 Mumbai Univer...Madhav Mishra
The document discusses machine learning paradigms including supervised learning, unsupervised learning, clustering, artificial neural networks, and more. It then discusses how supervised machine learning works using labeled training data for tasks like classification and regression. Unsupervised learning is described as using unlabeled data to find patterns and group data. Semi-supervised learning uses some labeled and some unlabeled data. Reinforcement learning provides rewards or punishments to achieve goals. Inductive learning infers functions from examples to make predictions for new examples.
This document introduces Factorization Machines, a general model that can mimic many successful factorization models. Factorization Machines allow feature vectors to be easily input and enjoy benefits of factorizing interactions between variables. The model has properties like expressiveness, multi-linearity, and scalable complexity. It relates to models like matrix factorization, tensor factorization, SVD++, and nearest neighbor models. Experiments show Factorization Machines outperform other models on rating prediction, context-aware recommendation, and tag recommendation tasks.
This document summarizes a conference paper published at ICLR 2020 that proposes a method called Plug and Play Language Models (PPLM) for controlled text generation using pretrained language models. PPLM allows controlling attributes of generated text like topic or sentiment without retraining the language model by combining it with simple attribute classifiers that guide the text generation process. The paper presents PPLM as a simple alternative to retraining language models that is more efficient and practical for controlled text generation.
해당 자료는 풀잎스쿨 18기 중 "설명가능한 인공지능 기획!" 진행 중 Counterfactual Explanation 세션에 대해서 정리한 자료입니다.
논문, Youtube 및 하기 자료를 바탕으로 정리되었습니다.
https://christophm.github.io/interpretable-ml-book/
The document examines using a nearest neighbor algorithm to rate men's suits based on color combinations. It trained the algorithm on 135 outfits rated as good, mediocre, or bad. It then tested the algorithm on 30 outfits rated by a human. When trained on 135 outfits, the algorithm incorrectly rated 36.7% of test outfits. When trained on only 68 outfits, it incorrectly rated 50% of test outfits, showing larger training data improves accuracy. It also tested using HSL color representation instead of RGB with similar results.
This document provides an introduction to machine learning. It discusses how children learn through explanations from parents, examples, and reinforcement learning. It then defines machine learning as programs that improve in performance on tasks through experience processing. The document outlines typical machine learning tasks including supervised learning, unsupervised learning, and reinforcement learning. It provides examples of each type of learning and discusses evaluation methods for supervised learning models.
Explainable AI - making ML and DL models more interpretableAditya Bhattacharya
The document discusses explainable AI (XAI) and making machine learning and deep learning models more interpretable. It covers the necessity and principles of XAI, popular model-agnostic XAI methods for ML and DL models, frameworks like LIME, SHAP, ELI5 and SKATER, and research questions around evolving XAI to be understandable by non-experts. The key topics covered are model-agnostic XAI, surrogate models, influence methods, visualizations and evaluating descriptive accuracy of explanations.
Deciphering AI - Unlocking the Black Box of AIML with State-of-the-Art Techno...Analytics India Magazine
Most organizations understand the predictive power and the potential gains from AIML, but AI and ML are still now a black box technology for them. While deep learning and neural networks can provide excellent inputs to businesses, leaders are challenged to use them because of the complete blind faith required to ‘trust’ AI. In this talk we will use the latest technological developments from researchers, the US defense department, and the industry to unbox the black box and provide businesses a clear understanding of the policy levers that they can pull, why, and by how much, to make effective decisions?
The document summarizes key concepts in machine learning, including defining learning, types of learning (induction vs discovery, guided learning vs learning from raw data, etc.), generalisation and specialisation, and some simple learning algorithms like Find-S and the candidate elimination algorithm. It discusses how learning can be viewed as searching a generalisation hierarchy to find a hypothesis that covers the examples. The candidate elimination algorithm maintains the version space - the set of hypotheses consistent with the training examples - by updating the general and specific boundaries as new examples are processed.
A short presentation for beginners on Introduction of Machine Learning, What it is, how it works, what all are the popular Machine Learning techniques and learning models (supervised, unsupervised, semi-supervised, reinforcement learning) and how they works with various Industry use-cases and popular examples.
Applied Artificial Intelligence Unit 5 Semester 3 MSc IT Part 2 Mumbai Univer...Madhav Mishra
The document discusses various topics in natural language processing and knowledge representation techniques, including conceptual dependency theory, script structures, the CYC theory, case grammars, and the semantic web. It provides information on each topic through a series of slides by Madhav Mishra, describing things like the components of scripts, features and examples of CYC knowledge base, how semantic web uses XML, RDF and ontologies, and an overview of case grammars and their use of functional relationships between nouns and verbs.
This was presented to software developers with the goal of introducing them to basic machine learning workflow, code snippets, possibilities and state-of-the-art in NLP and give some clues on where to get started.
Machine Learning for Dummies (without mathematics)ActiveEon
It presents an introduction and the basic concepts of machine learning without mathematics. This is a short presentation for beginners in machine learning.
1. The document discusses machine learning and provides an overview of the seven steps of machine learning including gathering data, preparing data, choosing a model, training the model, evaluating the model, tuning hyperparameters, and making predictions.
2. It describes tips for data preparation such as exploring data for trends and issues, formatting data consistently, and handling missing values, outliers, and imbalanced data.
3. Techniques for outlier removal are discussed including clustering-based, nearest-neighbor based, density-based, graphical, and statistical approaches. Limitations and challenges of outlier removal are noted.
Machine Learning: Generative and Discriminative Modelsbutest
The document discusses machine learning models, specifically generative and discriminative models. It provides examples of generative models like Naive Bayes classifiers and hidden Markov models. Discriminative models discussed include logistic regression and conditional random fields. The document contrasts how generative models estimate class-conditional probabilities while discriminative models directly estimate posterior probabilities. It also compares how hidden Markov models model sequential data generatively while conditional random fields model sequential data discriminatively.
Nautral Langauge Processing - Basics / Non Technical Dhruv Gohil
This document provides an overview of natural language processing (NLP) and discusses several NLP applications. It introduces NLP and how it helps computers understand human language through examples like Apple's Siri and Google Now. It then summarizes popular NLP toolkits and describes applications including text summarization, information extraction, sentiment analysis, and dialog systems. The document concludes by discussing NLP system development, testing, and evaluation.
1) The document summarizes a symposium on interpretable AI, discussing various methods for making AI systems more transparent and understandable to humans.
2) It outlines different categories of interpretability techniques including understanding data distributions, mechanistic interpretability of models, and concept-based explanations of outcomes.
3) Moving forward, the document suggests expanding interpretability research into non-tabular high dimensional data, incorporating causality, and using interpretability methods for data and knowledge discovery.
invited talk in the ExUM workshop in the UMAP 2022 conference
abstract:
Explainability has become an important topic both in Data Science and AI in general and in recommender systems in particular, as algorithms have become much less inherently explainable. However, explainability has different interpretations and goals in different fields. For example, interpretability and explanainability tools in machine learning are predominantly developed for Data Scientists to understand and scrutinize their models. Current tools are therefore often quite technical and not very ‘user-friendly’. I will illustrate this with our recent work on improving the explainability of model-agnostic tools such as LIME and SHAP. Another stream of research on explainability in the HCI and XAI fields focuses more on users’ needs for explainability, such as contrastive and selective explanations and explanations that fit with the mental models and beliefs of the user. However, how to satisfy those needs is still an open question. Based on recent work in interactive AI and machine learning, I will propose that explainability goes together with interactivity, and will illustrate this with examples from our own work in music genre exploration, that combines visualizations and interactive tools to help users understand and tune our exploration model.
This document provides an overview of machine learning applications in natural language processing and text classification. It discusses common machine learning tasks like part-of-speech tagging, named entity extraction, and text classification. Popular machine learning algorithms for classification are described, including k-nearest neighbors, Rocchio classification, support vector machines, bagging, and boosting. The document argues that machine learning can be used to solve complex real-world problems and that text processing is one area with many potential applications of these techniques.
Introduction to machine learning and model building using linear regressionGirish Gore
An basic introduction of Machine learning and a kick start to model building process using Linear Regression. Covers fundamentals of Data Science field called Machine Learning covering the fundamental topic of supervised learning method called linear regression. Importantly it covers this using R language and throws light on how to interpret linear regression results of a model. Interpretation of results , tuning and accuracy metrics like RMSE Root Mean Squared Error are covered here.
Applied Artificial Intelligence Unit 3 Semester 3 MSc IT Part 2 Mumbai Univer...Madhav Mishra
The document discusses machine learning paradigms including supervised learning, unsupervised learning, clustering, artificial neural networks, and more. It then discusses how supervised machine learning works using labeled training data for tasks like classification and regression. Unsupervised learning is described as using unlabeled data to find patterns and group data. Semi-supervised learning uses some labeled and some unlabeled data. Reinforcement learning provides rewards or punishments to achieve goals. Inductive learning infers functions from examples to make predictions for new examples.
This document introduces Factorization Machines, a general model that can mimic many successful factorization models. Factorization Machines allow feature vectors to be easily input and enjoy benefits of factorizing interactions between variables. The model has properties like expressiveness, multi-linearity, and scalable complexity. It relates to models like matrix factorization, tensor factorization, SVD++, and nearest neighbor models. Experiments show Factorization Machines outperform other models on rating prediction, context-aware recommendation, and tag recommendation tasks.
This document summarizes a conference paper published at ICLR 2020 that proposes a method called Plug and Play Language Models (PPLM) for controlled text generation using pretrained language models. PPLM allows controlling attributes of generated text like topic or sentiment without retraining the language model by combining it with simple attribute classifiers that guide the text generation process. The paper presents PPLM as a simple alternative to retraining language models that is more efficient and practical for controlled text generation.
해당 자료는 풀잎스쿨 18기 중 "설명가능한 인공지능 기획!" 진행 중 Counterfactual Explanation 세션에 대해서 정리한 자료입니다.
논문, Youtube 및 하기 자료를 바탕으로 정리되었습니다.
https://christophm.github.io/interpretable-ml-book/
The document examines using a nearest neighbor algorithm to rate men's suits based on color combinations. It trained the algorithm on 135 outfits rated as good, mediocre, or bad. It then tested the algorithm on 30 outfits rated by a human. When trained on 135 outfits, the algorithm incorrectly rated 36.7% of test outfits. When trained on only 68 outfits, it incorrectly rated 50% of test outfits, showing larger training data improves accuracy. It also tested using HSL color representation instead of RGB with similar results.
This document provides an introduction to machine learning. It discusses how children learn through explanations from parents, examples, and reinforcement learning. It then defines machine learning as programs that improve in performance on tasks through experience processing. The document outlines typical machine learning tasks including supervised learning, unsupervised learning, and reinforcement learning. It provides examples of each type of learning and discusses evaluation methods for supervised learning models.
Explainable AI - making ML and DL models more interpretableAditya Bhattacharya
The document discusses explainable AI (XAI) and making machine learning and deep learning models more interpretable. It covers the necessity and principles of XAI, popular model-agnostic XAI methods for ML and DL models, frameworks like LIME, SHAP, ELI5 and SKATER, and research questions around evolving XAI to be understandable by non-experts. The key topics covered are model-agnostic XAI, surrogate models, influence methods, visualizations and evaluating descriptive accuracy of explanations.
Deciphering AI - Unlocking the Black Box of AIML with State-of-the-Art Techno...Analytics India Magazine
Most organizations understand the predictive power and the potential gains from AIML, but AI and ML are still now a black box technology for them. While deep learning and neural networks can provide excellent inputs to businesses, leaders are challenged to use them because of the complete blind faith required to ‘trust’ AI. In this talk we will use the latest technological developments from researchers, the US defense department, and the industry to unbox the black box and provide businesses a clear understanding of the policy levers that they can pull, why, and by how much, to make effective decisions?
The document summarizes key concepts in machine learning, including defining learning, types of learning (induction vs discovery, guided learning vs learning from raw data, etc.), generalisation and specialisation, and some simple learning algorithms like Find-S and the candidate elimination algorithm. It discusses how learning can be viewed as searching a generalisation hierarchy to find a hypothesis that covers the examples. The candidate elimination algorithm maintains the version space - the set of hypotheses consistent with the training examples - by updating the general and specific boundaries as new examples are processed.
A short presentation for beginners on Introduction of Machine Learning, What it is, how it works, what all are the popular Machine Learning techniques and learning models (supervised, unsupervised, semi-supervised, reinforcement learning) and how they works with various Industry use-cases and popular examples.
Applied Artificial Intelligence Unit 5 Semester 3 MSc IT Part 2 Mumbai Univer...Madhav Mishra
The document discusses various topics in natural language processing and knowledge representation techniques, including conceptual dependency theory, script structures, the CYC theory, case grammars, and the semantic web. It provides information on each topic through a series of slides by Madhav Mishra, describing things like the components of scripts, features and examples of CYC knowledge base, how semantic web uses XML, RDF and ontologies, and an overview of case grammars and their use of functional relationships between nouns and verbs.
This was presented to software developers with the goal of introducing them to basic machine learning workflow, code snippets, possibilities and state-of-the-art in NLP and give some clues on where to get started.
Machine Learning for Dummies (without mathematics)ActiveEon
It presents an introduction and the basic concepts of machine learning without mathematics. This is a short presentation for beginners in machine learning.
1. The document discusses machine learning and provides an overview of the seven steps of machine learning including gathering data, preparing data, choosing a model, training the model, evaluating the model, tuning hyperparameters, and making predictions.
2. It describes tips for data preparation such as exploring data for trends and issues, formatting data consistently, and handling missing values, outliers, and imbalanced data.
3. Techniques for outlier removal are discussed including clustering-based, nearest-neighbor based, density-based, graphical, and statistical approaches. Limitations and challenges of outlier removal are noted.
Machine Learning: Generative and Discriminative Modelsbutest
The document discusses machine learning models, specifically generative and discriminative models. It provides examples of generative models like Naive Bayes classifiers and hidden Markov models. Discriminative models discussed include logistic regression and conditional random fields. The document contrasts how generative models estimate class-conditional probabilities while discriminative models directly estimate posterior probabilities. It also compares how hidden Markov models model sequential data generatively while conditional random fields model sequential data discriminatively.
Nautral Langauge Processing - Basics / Non Technical Dhruv Gohil
This document provides an overview of natural language processing (NLP) and discusses several NLP applications. It introduces NLP and how it helps computers understand human language through examples like Apple's Siri and Google Now. It then summarizes popular NLP toolkits and describes applications including text summarization, information extraction, sentiment analysis, and dialog systems. The document concludes by discussing NLP system development, testing, and evaluation.
1) The document summarizes a symposium on interpretable AI, discussing various methods for making AI systems more transparent and understandable to humans.
2) It outlines different categories of interpretability techniques including understanding data distributions, mechanistic interpretability of models, and concept-based explanations of outcomes.
3) Moving forward, the document suggests expanding interpretability research into non-tabular high dimensional data, incorporating causality, and using interpretability methods for data and knowledge discovery.
invited talk in the ExUM workshop in the UMAP 2022 conference
abstract:
Explainability has become an important topic both in Data Science and AI in general and in recommender systems in particular, as algorithms have become much less inherently explainable. However, explainability has different interpretations and goals in different fields. For example, interpretability and explanainability tools in machine learning are predominantly developed for Data Scientists to understand and scrutinize their models. Current tools are therefore often quite technical and not very ‘user-friendly’. I will illustrate this with our recent work on improving the explainability of model-agnostic tools such as LIME and SHAP. Another stream of research on explainability in the HCI and XAI fields focuses more on users’ needs for explainability, such as contrastive and selective explanations and explanations that fit with the mental models and beliefs of the user. However, how to satisfy those needs is still an open question. Based on recent work in interactive AI and machine learning, I will propose that explainability goes together with interactivity, and will illustrate this with examples from our own work in music genre exploration, that combines visualizations and interactive tools to help users understand and tune our exploration model.
Personalized Retweet Prediction in TwitterLiangjie Hong
This document proposes a method to predict which tweets a user is likely to retweet from their friends on Twitter. It discusses related work on generic and personalized tweet prediction. The proposed method uses factorization machines with a weighted approximate ranking pairwise loss function to model users' historical retweeting behaviors through collaborative filtering and content features. Experiments on a dataset of 0.7M users and their tweets show the proposed method outperforms baselines that use matrix factorization and other techniques. Topic modeling is also applied to identify topics in tweets.
A Categorisation of Post-hoc Explanations for Predictive ModelsJane Dane
This work is highly influenced by work previously completed by Zachary Lipton in Mythos of Model Interpretability. Essentially we are arguing that as long as there's no consensus and formal standardisation of what people mean by interpretability it will prevent us from having a pragmatic and influential progress in this direction. At the time of the presentation there was no consesus on validation metrics, datasets or methodologies to evaluate and compare interpretability methods in the literature. We highly emphasised the need of an axiomatic and formal approach relating to earlier efforts in interpretability in fuzzy systems in order to enforce the healthy habit of thinking about formal definitions and standardisations.
GDG Cloud Southlake #17: Meg Dickey-Kurdziolek: Explainable AI is for EveryoneJames Anderson
If Artificial Intelligence (AI) is a black-box, how can a human comprehend and trust the results of Machine Learning (ML) alogrithms? Explainable AI (XAI) tries to shed light into that AI black-box so humans can trust what is going on. Our speaker Meg Dickey-Kurdziolek is currently a UX Researcher for Google Cloud AI and Industry Solutions, where she focuses her research on Explainable AI and Model Understanding. Recording of the presentation: https://youtu.be/6N2DNN_HDWU
Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...AI Frontiers
In this tutorial I will introduce recent work in applying weak supervision and reinforcement learning to Questions Answering (QA) systems. Specifically we discuss the semantic parsing task for which natural language queries are converted to computation steps on knowledge graphs or data tables and produce the expected answers. State-of-the-art results can be achieved by novel memory structure for sequence models and improvements in reinforcement learning algorithms. Related code and experiment setup can be found at https://github.com/crazydonkey200/neural-symbolic-machines. Related paper: https://openreview.net/pdf?id=SyK00v5xx.
Building AI Applications using Knowledge GraphsAndre Freitas
This document provides an overview of building AI applications using knowledge graphs. It discusses the goals of the tutorial, which are to provide a broad view of multiple perspectives on knowledge graphs and show how knowledge graphs can form the foundation for building AI systems. The tutorial focuses on contemporary and emerging perspectives through exemplar approaches and infrastructures, rather than providing an exhaustive survey. It also notes that the tutorial is not a standard academic tutorial and takes a big picture view rather than being a comprehensive survey.
Bridging the gap between AI and UI - DSI Vienna - full versionLiad Magen
This is a summary of the latest research on model interpretability, including Recurrent neural networks (RNN) for Natural Language Processing (NLP) in terms of what's in an RNN.
In addition, it contains suggestion to improve machine learning based user interface, to engage users and encourage them to contribute data to adapt the models to them.
The document summarizes an agenda for a session on evaluation frameworks. The session will include presentations on experiences with evaluation, the Group Concept Mapping method, and an initial version of the evaluation framework with examples of criteria. There will also be discussions on suitable evaluation criteria, methods, and experts to involve in developing the framework. The objectives are to increase awareness of the evaluation task and collect suitable evaluation indicators.
Presenting the landscape of AI/ML in 2023 by introducing a quick summary of the last 10 years of its progress, current situation, and looking at things happening behind the scene.
Feature Fusion and Classifier Ensemble Technique for Robust Face RecognitionCSCJournals
Face recognition is an important part of the broader biometric security systems research. In the past, researchers have explored either the Feature Space or the Classifier Space at a time to achieve efficient face recognition. In this work, both the Feature Space optimization as well as the Classifier Space optimization have been used to achieve improved results. The efficient technique of Feature Fusion in the Feature Space and Classifier Ensemble technique in the Classifier Space have been used to achieve robust and efficient face recognition. In the Feature Space, the Discrete Wavelet Transform (DWT) and the Histogram of Oriented Gradient (HOG) features have been extracted from face images and these have been used for classification purposes after Feature Fusion using the Principal Component Analysis (PCA). In the Classifier Space, a Classifier Ensemble has been used, utilizing the bagging technique for ensemble training, instead of a single classifier for efficient classification. Proper selections of various parameters of the DWT, HOG features and the Classification Ensemble have been considered to achieve optimum performance. The proposed classification technique has been applied to the AT&T (ORL) and Yale benchmark face recognition databases, and we have achieved excellent results of 99.78% and 97.72% classification accuracy respectively. The proposed Feature Fusion and Classifier Ensemble technique has been subjected to sensitivity analysis and it has been found to be robust under reduced spatial resolution conditions.
Automating Software Development Using Artificial Intelligence (AI)Jeremy Bradbury
In recent years, traditional software development activities have been enhanced through the use of Artificial Intelligence (AI) techniques including genetic algorithms, machine learning and deep learning. The use cases for AI in software development have ranged from developer recommendations to complete automation of software developer activities. To demonstrate the breadth of application, I will present several recent examples of how AI can be leveraged to automate software development. First, I will present an approach to predicting future code changes in GitHub projects using historical data and machine learning. Next, I will present our framework for repairing multi-threaded software bugs using genetic algorithms. I will conclude with a broad discussion of the impact AI is having on software development.
Artificial Intelligence power point presentation documentDavid Raj Kanthi
This document provides a certificate for a seminar report on the topic of artificial intelligence. It was completed by a student in partial fulfillment of an M.C.A. degree program in 2016-2017. The document includes an acknowledgment, declaration, abstract, and index sections that provide information about the student, guide, and overall content covered in the seminar report on artificial intelligence.
This is slides used at Arithmer seminar given by Dr. Masaaki Uesaka at Arithmer inc.
It is a summary of recent methods for quality assurance of machine learning model.
Arithmer Seminar is weekly held, where professionals from within our company give lectures on their respective expertise.
Arithmer株式会社は東京大学大学院数理科学研究科発の数学の会社です。私達は現代数学を応用して、様々な分野のソリューションに、新しい高度AIシステムを導入しています。AIをいかに上手に使って仕事を効率化するか、そして人々の役に立つ結果を生み出すのか、それを考えるのが私たちの仕事です。
Arithmer began at the University of Tokyo Graduate School of Mathematical Sciences. Today, our research of modern mathematics and AI systems has the capability of providing solutions when dealing with tough complex issues. At Arithmer we believe it is our job to realize the functions of AI through improving work efficiency and producing more useful results for society.
Incorporating Word Embeddings into Open Directory Project Based Large-Scale C...Korea University
1. The document proposes two novel joint models that incorporate word embeddings into an Open Directory Project (ODP)-based classification framework to improve large-scale text classification.
2. The models generate category vectors that represent the semantics of ODP categories using both the explicit ODP taxonomy and implicit word2vec representations.
3. Experiments on real-world datasets demonstrate the models outperform state-of-the-art baselines, validating the efficacy of jointly leveraging explicit and implicit representations for large-scale text classification.
GDG Community Day 2023 - Interpretable ML in productionSARADINDU SENGUPTA
Validating an ML model with train-test accuracy metrics offers an initial understanding of viability but generating consistent inferencing with contextual business goals requires understanding how the deployed model works in different nature and how they will behave in case of soft data drift.
In this talk, I will try to go through different explainability methods and how to employ them and how the choice of type of models affects or affects the interpretability in production inferencing.
THE IMPACT OF USING VISUAL PROGRAMMING ENVIRONMENT TOWARDS COLLEGE STUDENTS’ ...ijma
ABSTRACT
This study aimed to identify the impact of using a visual programming environment on college students’ achievement and understanding when learning computer programming. In this quasi-experimental study, 91 students were divided systematically into an experimental group (53 students) and a control group (38 students). The experimental group were exposed with a visual programming environment while the control group were using an ordinary text-based programming environment. Data was collected using pre-test and post-test, then analysed using paired t-test, independent sample t-test and thematic content analysis. A significant increase in the students’ achievement was recorded during the paired t-test for both groups. However, there is no significant difference in the students’ achievement between the groups. Surprisingly, the thematic analysis showed that students’ understanding in the experimental group were improved relatively better than in the control group. Thus, we conclude that visual programming environment have better impact to the students’ understanding.
Jakub Langr (University of Oxford) - Overview of Generative Adversarial Netwo...Codiax
This document provides an overview of Generative Adversarial Networks (GANs) in 3 sections. It begins by briefly discussing supervised and unsupervised machine learning. It then explains that GANs use two neural networks, a generator and discriminator, that compete against each other in a game theoretic setup. The generator learns to produce more realistic samples while the discriminator learns to better distinguish real and fake samples. Popular GAN architectures like CycleGAN and BigGAN are also summarized.
Deep Learning & NLP: Graphs to the Rescue!Roelof Pieters
This document provides an overview of deep learning and natural language processing techniques. It begins with a history of machine learning and how deep learning advanced beyond early neural networks using methods like backpropagation. Deep learning methods like convolutional neural networks and word embeddings are discussed in the context of natural language processing tasks. Finally, the document proposes some graph-based approaches to combining deep learning with NLP, such as encoding language structures in graphs or using finite state graphs trained with genetic algorithms.
Rsqrd AI: Recent Advances in Explainable Machine Learning ResearchSanjana Chowdhury
In this talk, Bernease Herman speaks about recent explainable ML research
Presented on 06/06/2019
**These slides are from a talk given at Rsqrd AI. Learn more at rsqrdai.org**
Open Channel Flow: fluid flow with a free surfaceIndrajeet sahu
Open Channel Flow: This topic focuses on fluid flow with a free surface, such as in rivers, canals, and drainage ditches. Key concepts include the classification of flow types (steady vs. unsteady, uniform vs. non-uniform), hydraulic radius, flow resistance, Manning's equation, critical flow conditions, and energy and momentum principles. It also covers flow measurement techniques, gradually varied flow analysis, and the design of open channels. Understanding these principles is vital for effective water resource management and engineering applications.
We have designed & manufacture the Lubi Valves LBF series type of Butterfly Valves for General Utility Water applications as well as for HVAC applications.
Impartiality as per ISO /IEC 17025:2017 StandardMuhammadJazib15
This document provides basic guidelines for imparitallity requirement of ISO 17025. It defines in detial how it is met and wiudhwdih jdhsjdhwudjwkdbjwkdddddddddddkkkkkkkkkkkkkkkkkkkkkkkwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwioiiiiiiiiiiiii uwwwwwwwwwwwwwwwwhe wiqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq gbbbbbbbbbbbbb owdjjjjjjjjjjjjjjjjjjjj widhi owqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq uwdhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhwqiiiiiiiiiiiiiiiiiiiiiiiiiiiiw0pooooojjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjjj whhhhhhhhhhh wheeeeeeee wihieiiiiii wihe
e qqqqqqqqqqeuwiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiqw dddddddddd cccccccccccccccv s w c r
cdf cb bicbsad ishd d qwkbdwiur e wetwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww w
dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffw
uuuuhhhhhhhhhhhhhhhhhhhhhhhhe qiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii iqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc ccccccccccccccccccccccccccccccccccc bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbu uuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuum
m
m mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm m i
g i dijsd sjdnsjd ndjajsdnnsa adjdnawddddddddddddd uw
Sachpazis_Consolidation Settlement Calculation Program-The Python Code and th...Dr.Costas Sachpazis
Consolidation Settlement Calculation Program-The Python Code
By Professor Dr. Costas Sachpazis, Civil Engineer & Geologist
This program calculates the consolidation settlement for a foundation based on soil layer properties and foundation data. It allows users to input multiple soil layers and foundation characteristics to determine the total settlement.
Determination of Equivalent Circuit parameters and performance characteristic...pvpriya2
Includes the testing of induction motor to draw the circle diagram of induction motor with step wise procedure and calculation for the same. Also explains the working and application of Induction generator
Sri Guru Hargobind Ji - Bandi Chor Guru.pdfBalvir Singh
Sri Guru Hargobind Ji (19 June 1595 - 3 March 1644) is revered as the Sixth Nanak.
• On 25 May 1606 Guru Arjan nominated his son Sri Hargobind Ji as his successor. Shortly
afterwards, Guru Arjan was arrested, tortured and killed by order of the Mogul Emperor
Jahangir.
• Guru Hargobind's succession ceremony took place on 24 June 1606. He was barely
eleven years old when he became 6th Guru.
• As ordered by Guru Arjan Dev Ji, he put on two swords, one indicated his spiritual
authority (PIRI) and the other, his temporal authority (MIRI). He thus for the first time
initiated military tradition in the Sikh faith to resist religious persecution, protect
people’s freedom and independence to practice religion by choice. He transformed
Sikhs to be Saints and Soldier.
• He had a long tenure as Guru, lasting 37 years, 9 months and 3 days
Supermarket Management System Project Report.pdfKamal Acharya
Supermarket management is a stand-alone J2EE using Eclipse Juno program.
This project contains all the necessary required information about maintaining
the supermarket billing system.
The core idea of this project to minimize the paper work and centralize the
data. Here all the communication is taken in secure manner. That is, in this
application the information will be stored in client itself. For further security the
data base is stored in the back-end oracle and so no intruders can access it.
Blood finder application project report (1).pdfKamal Acharya
Blood Finder is an emergency time app where a user can search for the blood banks as
well as the registered blood donors around Mumbai. This application also provide an
opportunity for the user of this application to become a registered donor for this user have
to enroll for the donor request from the application itself. If the admin wish to make user
a registered donor, with some of the formalities with the organization it can be done.
Specialization of this application is that the user will not have to register on sign-in for
searching the blood banks and blood donors it can be just done by installing the
application to the mobile.
The purpose of making this application is to save the user’s time for searching blood of
needed blood group during the time of the emergency.
This is an android application developed in Java and XML with the connectivity of
SQLite database. This application will provide most of basic functionality required for an
emergency time application. All the details of Blood banks and Blood donors are stored
in the database i.e. SQLite.
This application allowed the user to get all the information regarding blood banks and
blood donors such as Name, Number, Address, Blood Group, rather than searching it on
the different websites and wasting the precious time. This application is effective and
user friendly.
A high-Speed Communication System is based on the Design of a Bi-NoC Router, ...DharmaBanothu
The Network on Chip (NoC) has emerged as an effective
solution for intercommunication infrastructure within System on
Chip (SoC) designs, overcoming the limitations of traditional
methods that face significant bottlenecks. However, the complexity
of NoC design presents numerous challenges related to
performance metrics such as scalability, latency, power
consumption, and signal integrity. This project addresses the
issues within the router's memory unit and proposes an enhanced
memory structure. To achieve efficient data transfer, FIFO buffers
are implemented in distributed RAM and virtual channels for
FPGA-based NoC. The project introduces advanced FIFO-based
memory units within the NoC router, assessing their performance
in a Bi-directional NoC (Bi-NoC) configuration. The primary
objective is to reduce the router's workload while enhancing the
FIFO internal structure. To further improve data transfer speed,
a Bi-NoC with a self-configurable intercommunication channel is
suggested. Simulation and synthesis results demonstrate
guaranteed throughput, predictable latency, and equitable
network access, showing significant improvement over previous
designs
5. 5
“Interpretability is defined as the ability to explain or
to present in understandable terms to a human*.”
* not all humans are familiar with Machine Learning
[Kim et al., 2018]
6. 6
“The goal of interpretability is to describe the internals of
a system in a way that is understandable to humans*.”
* not all humans are familiar with Machine Learning
[Giplin et al., 2019]
8. Interpretability as a human-centric “translation” problem
8
Explanation in the model
representation space
(input pixels, activations)
Explanation in the human
representation space
(a visualization, a concept, a
sentence, an important
factor)
[Kim et al., 2018]
10. Why do we need interpretablity?
10
Trained CNN
It’s a cat
11. Why do we need interpretablity?
11
Trained CNN
It’s a cat
YAY!
12. Why do we need interpretablity?
12
Trained CNN
It’s a cat
If you want to know more about networks easily fooled:
[Szegedy et al., 2013], [Nguyen et al., 2015],
[Papernot et al., 2016], [Moosavi-Dezfooli et al., 2017]
13. Why do we need interpretablity?
13
Trained CNN
It’s a cat
Oh…but…why?
If you want to know more about networks easily fooled:
[Szegedy et al., 2013], [Nguyen et al., 2015],
[Papernot et al., 2016], [Moosavi-Dezfooli et al., 2017]
14. Why do we need interpretablity?
14
Trained CNN
It’s a cat
…
If you want to know more about networks easily fooled:
[Szegedy et al., 2013], [Nguyen et al., 2015],
[Papernot et al., 2016], [Moosavi-Dezfooli et al., 2017]
15. Why questions
[Giplin et al., 2019]
Why is the model working?
Why is it not?
Why is the output like this?
Why is it not something else?
Why should we trust
the model?
Explain
Defend
actions
Gain trust …
develop better models!
Why do we need interpretablity?
15
17. HEALTH ROBOTICS ASSISTED DRIVING LAW SOCIAL SCIENCES
High-risk applications demanding also for accountability,
transparency, fairness, trust [FAccT conference]
FINANCE
Where do we need interpretability?
17
18. Where it is not needed
Why is it not needed?
One motivation does not cover it all …
privacy robustness
[Kim B, Hooker S.]
already well
studied problem
Where is it needed?
Why is it needed?
Safety, science, debugging.
aligning objectives
18
19. How do we achieve interpretablity?
19
Interpretability is challenging
and trending
21. Our goal today is the how
21
gain a clearer understanding
KNOWING WHAT TO
APPLY & WHERE
22. 22
1. Inherently interpretable models
a. (Generalized) linear regression
b. Decision trees and rules
2. Interpreting complex models
a. From inside (opening the black box)
b. From outside (black-box)
3. What else? Use interpretability to develop
better models.
4. Q&A
Outline
Inside
Out
I will not talk about
dimensionality reduction
24. 24
Linear Regression
Output is a weighted sum of each feature
A linear increase of a feature is translated into a proportional effected in the outcome
No interactions between features
25. 25
Generalized Linear Regression
Family: normal
Link: identity
Family: binomial
Link: logit
Interpretation comes mostly from assumption on the data generation process
Complexity ~ generalization
We can replace with arbitrary distributions according to the data generation process
[Caruana et al. 2015]Generalized Additive Models
26. 26
Decision trees and rules
Car’s model
Ah, it’s a Tesla
Autonomy >= 350?
Yes
Is it electric?
Yes
Trackable and explainable decisions.
Good for data interactions!
Only categorical features
Step functions…sharp!
Changes in data lead to
different tree
Complexity ~ depth [Kim B.]
27. 27
Decision trees and rules
IF
PERSON CAPACITY < 2 &&
PRICE = ‘high’ &&
ELECTRIC = False &&
COMPANY LOGO = ‘horse’
THEN
car is a FERRARI
Intrinsic explanation
Sparse
Efficient
Not for regression
Only categorical features
Complexity ~ #rules
[Kim B.]
29. Local Global
Model-specific Post-hoc
29
Helpful terminology
vs
= true for a specific instance
= true for an entire set of inputs
(ex. a class)
= model built-in analysis = applicable to any model
[Lipton, 2016]
30. InsideMMD [Kim et al., 2016].
IF [Koh et al., 2017]
Database search
Geometric approaches
Visualization
Deconv [Zeiler et al., 2013],
AM [Erhan et al., 2009],
Dissection [Bau et al., 2017]
SVCCA [Raghu et al., 2017]
Surrogates
2. Interpretability of deep learning inside out
30
Out Attribution…
To features
To concepts
LIME [Ribeiro et al., 2016].
SHAP [Lundberg et al., 2016]
Saliency [Simonyan et al., 2013],
CAM [Zhou et al., 2016],
LRP [Binder et al., 2016]
TCAV [Kim et al., 2018],
RCVs [Graziani et al., 2018]
32. Database search
32
What examples explain the data or the model?
Prototype
= representative of all data
Criticism
= under-represented bit
Maximum Mean
Discrepancy
[Kim et al., 2016]
Global
Post-hoc
33. Influential Instances
33
Deleting one of these would strongly affect learning
Influence Functions
[Koh et al., 2017]
Best paper award!
Post-hoc
Global
35. 35
What is a neuron, a channel or a layer looking for?
Deconvolution:
inverting convolution
operations
[Zeiler et al., 2013]
Figure credits: stanford CS230 (2018 Youtube)
Post-hoc
Local Deconvolutions
36. 36
What is a neuron, a channel or a layer looking for?
Gradient Ascent
[Erhan et al., 2009],
[Olah et al. 2019]
This activation is maximized
Lucid toolbox
Post-hoc
Global Gradient Ascent
37. Network Dissection
37
What is a neuron, a channel or a layer looking for?
Network Dissection
[Bau et al., 2017]
~1K concepts:
Post-hoc
Global
Early training finds concepts, late training improves them
color, texture,
material, object, scene
Set of segmented
regions for all
concepts
38. 38
Singular Vector Canonical Correlation Analysis
Can we compress what a layer has learned? [Raghu et al., 2017]
Responses of this
layer to all data
Singular Value Decomposition
& Canonical Correlation Analysis
Allows comparisons of layers, archiectures and insights on training dynamics
What did it look like and what can we
do here?
Post-hoc
Global
39. InsideMMD [Kim et al., 2016].
IF [Koh et al., 2017]
Database search
Geometric approaches
Visualization
Deconv [Zeiler et al., 2013],
AM [Erhan et al., 2009],
Dissection [Bau et al., 2017]
SVCCA [Raghu et al., 2017]
Surrogates
2. Interpretability of deep learning inside out
39
Out Attribution…
To features
To concepts
LIME [Ribeiro et al., 2016].
SHAP [Lundberg et al., 2016]
Saliency [Simonyan et al., 2013],
CAM [Zhou et al., 2016],
LRP [Binder et al., 2016]
TCAV [Kim et al., 2018],
RCVs [Graziani et al., 2018]
40. Surrogate models
40
Replacement is an interpretable model trained on the data and the black-box predictions
Complex decision function
Linear surrogate (R2 0.7)
Flexibility by different surrogates
Very approximative ….
Post-hocGlobal
41. Local Interpretable Model-agnostic Explanations (LIME)
41
Replacement is an interpretable model trained on the data and the black-box predictions
to explain each prediction individually
Local linear surrogate
Flexible, universal (post-hoc)
Size of local neighborhood undefined
[Ribeiro et al., 2017]
42. Local Interpretable Model-agnostic Explanations (LIME)
42
Replacement is an interpretable model trained on the data and the black-box predictions
to explain each prediction individually
Local linear surrogate
Flexible, universal (post-hoc)
Size of local neighborhood undefined
[Ribeiro et al., 2017]
43. Local Interpretable Model-agnostic Explanations (LIME)
43
Replacement is an interpretable model trained on the data and the black-box predictions
to explain each prediction individually
Local linear surrogate
Flexible, universal (post-hoc)
Size of local neighborhood undefined
[Ribeiro et al., 2017]Sampling of local instances not very robust
44. Local Interpretable Model-agnostic Explanations (LIME)
44
Replacement is an interpretable model trained on the data and the black-box predictions
to explain each prediction individually
Local linear surrogate
Flexible, universal (post-hoc)
Size of local neighborhood undefined
Sampling of local instances not very robust
45. Local Interpretable Model-agnostic Explanations (LIME)
45
Replacement is an interpretable model trained on the data and the black-box predictions
to explain each prediction individually
Local linear surrogate
Flexible, universal (post-hoc)
Size of local neighborhood undefined
[Ribeiro et al., 2017]Sampling of local instances not very robust
46. SHapley Additive exPlanations
46
A game theoretic approach of competing features
[Lundberg et al., 2017]
Attributes to each input feature the change in the expected model prediction when conditioning on that feature.
Unifying framework, direct for categorical features
All rights reserved. No reuse allowed without permission.
(which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.
The copyright holder for this preprint.http://dx.doi.org/10.1101/206540doi:bioRxiv preprint first posted online Oct. 21, 2017;
Only qualitative on images, difficult abstraction
49. Saliency**
49 Slide credits: Hooker S,
** Saliency was recently shown to be very unstable and failing to be reliable in edge cases such
as in randomized networks or compared to random attribution maps (see Remove and Retrain, ROAR)
50. Class Activation Mapping
50
Importance of the image
regions is given by the
projection of the weights
of the output layer on the
last layer’s convolutional
feature maps
[Zhou et al., 2015]
Greatly succesfull technique for its transparency and directness
Only qualitative evaluation
Little focus on multiple instances of the same object
Post-hocLocal
52. Concept Activation Vectors
52
?
Vector of “striped” texture
∂output
∂vector
Directional
derivative
[Kim et al., 2018]
We collect examples of a
concept, i.e. “striped
texture”
We take the internal
activations (unrolled)
Classification of
“striped” vs
“random”
What about the relevance of “striped” texture in the classification of a zebra?
Generalized
saliency
Post-hocLocal Global
53. 53
Out
What if the concept is non-binary?
Such as tumor extension, patient age, color, ..
55. Regression Concept Vectors
55
Segmentation
(manual or
automatic)
Handcrafted
features, texture
descriptors, shape,
size, …
Take the internal
activations (aggregation)
Linear regression of
measures
∂output
∂vector
Directional
derivative
Generalized
saliency
[Graziani et al., 2018]
Best paper award, iMIMIC, MICCAI 2018!
Vector of “size”
Post-hocLocal Global
56. Application to health
56
1 Modeling of visual concepts
2 CNN explanation
Nuclei
Pleomorphism
Tubular formation
Mitotic count
Enlarged nuclei
Vesicular
appearance
Multiple nucleoli
Segmentation
size
Image
texture
descriptors
contrast
ASM
correlation
area
Nottingham
grading
guidelines
relevant for
positive class
image
tumor
probability
contrast
area
ASM
correlation
relevant for
negative class
black-box
state-of-art
model
contrast
ASM
correlation
area
high
low
57. Application to health
57
curvature mean
raw segmented
Individual relevancepn = 0.22
ppre = 0.70
pplus = 0.08
GT: normal; prediction: normal
cti median
cti meancurvature median
avg point diameter mean avg segment diameter median
raw segmented
pn = 0.99
ppre = 0.009
pplus = 0.0
1.082
1.168
0.118
0.447
5.24
-1 10
1.030
1.045
0.040
0.095
3.775
Retinopathy of prematurity
[Graziani et al., 2019]
[Yeche et al., 2019]
Radiomics
Image credits: Yeche et al. springer
59. Applications to computer vision
59
Image blue-ness
Color and texture were used to
reduce this loss!
[Graziani et al., 2019]
Interpreting intentionally flawed models
60. Our goal today is the how
60
gain a clearer understanding
WHAT TO
APPLY & WHERE?
61. 61
WHAT TO APPLY & WHERE?
What do you need most?
In deep learning
Understand
each
component
User-friendly
explanations
Visual explanations on
single input
Global
understanding
Individually
Interactions
Make
comparisons
Surrogate
models
From dataset of
conceptual examples
Measuring attributes on
imagesRCVs CAVs
Gradient
Ascent
Geometry-
based
Before decision
layer
Class
Activation
Maps
Layerwise
Layerwise
Relevance
64. 64
Can we use interpretability for better control and development?
It’s a cat
It has pointy ears…
And mustache!
Cats DO NOT have penguin legs….
Robustness to
adversarial
65. 65
Can we use interpretability for better control and development?
Interpretability
analysis
Prior knowledge
User’s feedback
Additional targets for
our model output
Desired
features
Multi-task
Learning
Undesired
Features
Adversarial
Learning
66. 66
TAKE AWAY
Pic credits: bannerengineering.com
Cartography to navigate
interpretability (slide 121)
Growing conferences, workshops:
Some interesting people and projects:
FAccT, Tutorial on Interpretable Machine Learning,
NeurIPS Interpretable ML, ICML Interpretable ML,
DL summer schools, AISTATS, CVPR, ICLR, ECCV,
ICCV, ECML, KDD, Workshop on Intepreting and
Explaining Visual AI Models, Tutorial on
Interpretable & transporent deep learning, WHI 2020
(virtual this year)… and many others!
B. Zhou (Torralba, MIT), B. Kim (Google Brain), G. Montavon (heatmapping.org), DARPA’s XAI, Ruth C.
Fong (Harvard & Microsoft), Finale Doshi-Velez (Harvard), A. Weller (Cambridge), S. Lundberg
(Microsoft), DARPA’s XAI Explainable Artificial Intelligence…and many others!
ML interpretability is human-centric, multi-faceted and should be tailored on a precise scope.
67. Thank you!
67
linear probing interpretationinternal representation
classification [1] regression (ours)
TCAV [1]
Br (ours [3,4,5])
UBS [2]
post-hoc interpretability
Text
DL model
feedback
through
concepts
Modeling of prior knowledge
area
contrast
Update of objective function
Handcrafted ML features
ML interpretability for healthcare: our vision
mara.graziani@hevs.ch
@mormontre
@maragraziani