Uploaded byMLAI2
PDF, PPTX405 views

Cost-effective Interactive Attention Learning with Neural Attention Process

The document presents an interactive learning framework aimed at improving model interpretability in deep neural networks through human interaction and feedback. It addresses challenges such as incorrect interpretations and the high costs of retraining models, proposing cost-effective approaches for annotation and instance selection. The framework utilizes neural attention processes for efficient learning and evaluation across various datasets, including electronic health records and real estate transactions.

Embed presentation

Download as PDF, PPTX
Jay Heo1, Junhyeon Park1, Hyewon Jeong1, Kwang joon Kim2, Juho Lee3, Eunho Yang1 3, Sung Ju Hwang1 3 Cost-Effective Interactive Attention Learning with Neural Attention Processes KAIST1, Yonsei University College of Medicine2, AITRICS3
Model Interpretability Main Network InferenceInput Data Training The complex nature of deep neural networks has led to the recent surge of interests in interpretable models which provide model interpretations.
Model Interpretability Main Network InferenceInput Data Model Interpretation Interpretation tool Inference The complex nature of deep neural networks has led to the recent surge of interests in interpretable models which provide model interpretations.
Model Interpretability The complex nature of deep neural networks has led to the recent surge of interests in interpretable models which provide model interpretations. Main Network InferenceInput Data Model Interpretation Interpretation tool Inference Provide explanations for model’s decision.
Challenge: Incorrect & Unreliable Interpretation Not all machine-generated interpretations are correct or human-understandable. • Correctness and reliability of a learning model heavily depends on quality and quantity of training data. • Neural networks tend to learn non-robust features that help with predictions, but are not human-perceptible. Model Interpretation Quality of training data Quantity of training data
Challenge: Incorrect & Unreliable Interpretation Not all machine-generated interpretations are correct or human-understandable. • Correctness and reliability of a learning model heavily depends on quality and quantity of training data. • Neural networks tend to learn non-robust features that help with predictions, but are not human-perceptible. Model Interpretation Quality of training data Quantity of training data Whether a model learn too many non-robust features during training?
Challenge: Incorrect & Unreliable Interpretation Not all machine-generated interpretations are correct or human-understandable. • Correctness and reliability of a learning model heavily depends on quality and quantity of training data. • Neural networks tend to learn non-robust features that help with predictions, but are not human-perceptible. Model Interpretation 1. Is it correct? 2. Is it understandable enough to trust? Quality of training data Quantity of training data Whether a model learn too many non-robust features during training?
Interactive Learning Framework Propose an interactive learning framework which iteratively update the model by interacting with the human supervisors who adjust the provided interpretations. • Actively use human supervisors as a channel for human-model communications. Attentional Networks Human Annotator Learning Model Physician Interpretation (Attentions) Model’s decision 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention
Interactive Learning Framework Propose an interactive learning framework which iteratively update the model by interacting with the human supervisors who adjust the provided interpretations. • Actively use human supervisors as a channel for human-model communications. Attentional Networks Human Annotator Learning Model Physician Interpretation (Attentions) Model’s decision Annotation AnnotateRetrain 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention
Challenge: Model Retraining Cost To reflect human feedback, the model needs to be retrained, which is costly. • Retraining the model with scarce human feedback may result in the model overfitting Physician Learning Model Annotated Examples 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention Interpretation
Challenge: Model Retraining Cost To reflect human feedback, the model needs to be retrained, which is costly. • Retraining the model with scarce human feedback may result in the model overfitting Physician Learning Model Annotated Examples 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention Interpretation Scarce feedback
Challenge: Model Retraining Cost To reflect human feedback, the model needs to be retrained, which is costly. • Retraining the model with scarce human feedback may result in the model overfitting Physician Learning Model Annotated Examples 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention Interpretation Scarce feedback Retraining
Challenge: Model Retraining Cost To reflect human feedback, the model needs to be retrained, which is costly. Physician Learning Model Annotated Examples 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention Interpretation Scarce feedback Retraining ! Overfitting • Retraining the model with scarce human feedback may result in the model overfitting
Challenge: Expensive Human Supervision Cost Obtaining human feedback on datasets with large numbers of training instances and features is extremely costly. • Obtaining feedback on already correct or previously corrected interpretations is wasteful. Annotator Annotate 𝑴 𝒂𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 (𝒕) ∈{0, 1} Big data
Challenge: Expensive Human Supervision Cost Obtaining human feedback on datasets with large numbers of training instances and features is extremely costly. • Obtaining feedback on already correct or previously corrected interpretations is wasteful. Annotator Annotate Big data Annotate
Challenge: Expensive Human Supervision Cost Obtaining human feedback on datasets with large numbers of training instances and features is extremely costly. • Obtaining feedback on already correct or previously corrected interpretations is wasteful. Annotator Annotate Big data Costly Annotate !
Interactive Attention Learning Framework Domain experts interactively evaluate learned attentions and provide feedbacks to obtain models that generate human-intuitive interpretations. Learning Model Physician Attention Mechanism LDL Respiration Cholesterol Creatine BMI Diabetes Heart Failure Hypertension Deliver attention Annotate attention Attention0.80.60.3 : Attention 𝑴 𝒂𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 (𝒕) ∈ {0, 1} Influence Function MC Dropout Annotation Mask Deep Interpretation Tool Correlation & Causal relationship Analysis Manipulate Explain Neural Attention Processes Counterfactual Estimation Counterfactual Estimation a c b Granger Causality
Interactive Attention Learning Framework Domain experts interactively evaluate learned attentions and provide feedbacks to obtain models that generate human-intuitive interpretations. Learning Model Physician Attention Mechanism 1. Neural Attention Processes LDL Respiration Cholesterol Creatine BMI Diabetes Heart Failure Hypertension Deliver attention Annotate attention Attention0.80.60.3 : Attention 𝑴 𝒂𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 (𝒕) ∈ {0, 1} Influence Function MC Dropout Annotation Mask Deep Interpretation Tool Correlation & Causal relationship Analysis Manipulate Explain Neural Attention Processes Counterfactual Estimation Counterfactual Estimation a c b Granger Causality
Interactive Attention Learning Framework Domain experts interactively evaluate learned attentions and provide feedbacks to obtain models that generate human-intuitive interpretations. Learning Model Physician Attention Mechanism LDL Respiration Cholesterol Creatine BMI Diabetes Heart Failure Hypertension Deliver attention Annotate attention Attention0.80.60.3 : Attention 𝑴 𝒂𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 (𝒕) ∈ {0, 1} Influence Function MC Dropout Annotation Mask Deep Interpretation Tool Correlation & Causal relationship Analysis Manipulate Explain Neural Attention Processes Counterfactual Estimation 1. Neural Attention Processes 2. Cost-effective Reranking Counterfactual Estimation a c b Granger Causality
Neural Attention Processes (NAP) NAP naturally reflects the information from the annotation summarization z via amor tized inference. Domain Expert Annotation Neural Attention Processes • NAP learns to summarize delivered annotations to a latent vector, and gives th e summarization as an additional input to the attention generating network.
Neural Attention Processes (NAP) NAP naturally reflects the information from the annotation summarization z via amor tized inference. Domain Expert Annotation Neural Attention Processes • NAP learns to summarize delivered annotations to a latent vector, and gives th e summarization as an additional input to the attention generating network.
Neural Attention Processes (NAP) ! Attention Context Points "! !! "! #! !! New Observations !!"# "!"# !!"# NAP minimizes retraining cost by incorporating new labeled instances without retrai ning and overfitting. First Round (s=1) • NAP doesn’t require retraining for further new observations, in that NAP automatically adapt to them at the cost of a forward pass through a network 𝑔!.
Neural Attention Processes (NAP) ! Attention Context Points "! !! "! #! !! New Observations !!"# "!"# !!"# NAP minimizes retraining cost by incorporating new labeled instances without retrai ning and overfitting. First Round (s=1) • NAP doesn’t require retraining for further new observations, in that NAP automatically adapt to them at the cost of a forward pass through a network 𝑔!.
Neural Attention Processes (NAP) ! Attention Context Points "! !! "! #! !! New Observations !!"# "!"# !!"# NAP minimizes retraining cost by incorporating new labeled instances without retrai ning and overfitting. First Round (s=1) • NAP doesn’t require retraining for further new observations, in that NAP automatically adapt to them at the cost of a forward pass through a network 𝑔!.
Neural Attention Processes (NAP) ! Attention Context Points "! !! "! #! !! New Observations !!"# "!"# !!"# NAP minimizes retraining cost by incorporating new labeled instances without retrai ning and overfitting. First Round (s=1) • NAP doesn’t require retraining for further new observations, in that NAP automatically adapt to them at the cost of a forward pass through a network 𝑔!.
Neural Attention Processes (NAP) NAP minimizes retraining cost by incorporating new labeled instances without retrain ing and overfitting. • NAP is trained in a meta-learning fashion for few-shot function estimation, where it is trained to predict the attention mask of other labeled samples, given a randomly selected labeled set as context. ! Attention "! !! New Observations !!"# "!"# !!"# Context Points !! "! #!!!"$ "!"$ !!"$ Further Rounds (s=2,3,4)
Neural Attention Processes (NAP) NAP minimizes retraining cost by incorporating new labeled instances without retrain ing and overfitting. • NAP is trained in a meta-learning fashion for few-shot function estimation, where it is trained to predict the attention mask of other labeled samples, given a randomly selected labeled set as context. ! Attention "! !! New Observations !!"# "!"# !!"# Context Points !! "! #!!!"$ "!"$ !!"$ Further Rounds (s=2,3,4)
Neural Attention Processes (NAP) NAP minimizes retraining cost by incorporating new labeled instances without retrain ing and overfitting. • NAP is trained in a meta-learning fashion for few-shot function estimation, where it is trained to predict the attention mask of other labeled samples, given a randomly selected labeled set as context. ! Attention "! !! New Observations !!"# "!"# !!"# Context Points !! "! #!!!"$ "!"$ !!"$ Further Rounds (s=2,3,4)
Cost-Effective Instance & Features Reranking (CER) Address the expensive human labeling cost by reranking the instances, features, and timesteps (for time-series data) by their negative impacts. Attentional Network Domain Expert Instance-wise and Feature-wise Reranking
Cost-Effective Instance & Features Reranking (CER) Address the expensive human labeling cost by reranking the instances, features, and timesteps (for time-series data) by their negative impacts. Attentional Network Domain Expert Instance-wise and Feature-wise Reranking 𝑷 𝑲 … … …TrainTrainValid Valid Estimate 𝑰(𝒖!) / 𝐕𝐚𝐫(𝒖!) Instance-level Reranking Select Re-rank & select
Cost-Effective Instance & Features Reranking (CER) Address the expensive human labeling cost by reranking the instances, features, and timesteps (for time-series data) by their negative impacts. Attentional Network Domain Expert Instance-wise and Feature-wise Reranking 𝑷 𝑲 … … …TrainTrainValid Valid Estimate 𝑰(𝒖!) / 𝐕𝐚𝐫(𝒖!) Instance-level Reranking Train Feature-level Reranking Select Re-rank & select Estimate 𝑰(𝒖",$ (&) ) / 𝐕𝐚𝒓 (𝒖",$ (&) ) / 𝜓(𝒖",$ (&) ) Feature … … Feature 𝑭 Re-rank & select
CER: 1. Influence Score Use the influence function (Koh & Liang, 2017) to approximate the impact of individual training points on the model prediction. Fish Dog Dog “Dog” Training Training data Test Input [Koh and Liang 17] Understanding Black-box Predictions via Influence Functions, ICML 2017
CER: 2. Uncertainty Score Measure the negative impacts using the uncertainty which can be measured by Mon te-Carlo sampling (Gal & Ghahramani, 2016). Uncertainty-aware Attention Mechanism • Less expensive approach to measure the negative impacts. • Assume that instances having high-predictive uncertainties are potential candidate to be corrected. Uncertainty [Jay Heo*, Haebeom Lee*, Saehun Kim,, Juho Lee, Gwangjun Kim , Eunho Yang, Sung Joo Hwang] Uncertainty-aware Attention Mechanism for Reliable prediction and Interpretation, Neurips 2018.
CER: 2. Uncertainty Score Measure the negative impacts using the uncertainty which can be measured by Mon te-Carlo sampling (Gal & Ghahramani, 2016). Measure instance-wise & feature-wise uncertainty 0.80.60.3 Low uncertainty High uncertainty High uncertainty µ σ, )(N SpO2 Pulse Respiration • Less expensive approach to measure the negative impacts. • Assume that instances having high-predictive uncertainties are potential candidate to be corrected. Uncertainty-aware Attention Mechanism [Jay Heo*, Haebeom Lee*, Saehun Kim,, Juho Lee, Gwangjun Kim , Eunho Yang, Sung Joo Hwang] Uncertainty-aware Attention Mechanism for Reliable prediction and Interpretation, Neurips 2018.
CER: 3. Counterfactual Score How would the prediction change if we ignore a certain feature by manually turning on/off the corresponding attention value? • Not need for retraining since we can simply set its attention value to zero. • Used to rerank the features with regards to their importance. Counterfactual Estimation Interface
CER: 3. Counterfactual Score How would the prediction change if we ignore a certain feature by manually turning on/off the corresponding attention value? • Not need for retraining since we can simply set its attention value to zero. • Used to rerank the features with regards to their importance. Counterfactual Estimation Interface
CER: 3. Counterfactual Score How would the prediction change if we ignore a certain feature by manually turning on/off the corresponding attention value? • Not need for retraining since we can simply set its attention value to zero. • Used to rerank the features with regards to their importance. Counterfactual Estimation Interface
CER: 3. Counterfactual Score How would the prediction change if we ignore a certain feature by manually turning on/off the corresponding attention value? • Not need for retraining since we can simply set its attention value to zero. • Used to rerank the features with regards to their importance. Counterfactual Estimation Interface
Experimental setting – Datasets Use electronic health records, real estate sales transaction records, and exercise squat posture correction records for classification and regression tasks. 1. EHR Datasets 1) Cerebral Infarction 2) Cardio Vascular Disease 3) Heart Failure Binary Classification task
Experimental setting – Datasets Use electronic health records, real estate sales transaction records, and exercise squat posture correction records for classification and regression tasks. 1. EHR Datasets 1) Cerebral Infarction 2) Cardio Vascular Disease 3) Heart Failure 2. Real-estate dataset 1) Housing price forecasting Binary Classification task Regression task
Experimental setting – Datasets Use electronic health records, real estate sales transaction records, and exercise squat posture correction records for classification and regression tasks. 1. EHR Datasets 1) Cerebral Infarction 2) Cardio Vascular Disease 3) Heart Failure 2. Real-estate dataset 3. Squat Posture set 1) Housing price forecasting 1) Squat posture correction Binary Classification task Regression task Multi-label Classification task
Attention Evaluation Interface – Risk Prediction Domain experts evaluate the delivered attention 𝛼 𝑎𝑛𝑑 𝜷 via attention annotation mask 𝑴 𝒕 𝜶 ∈ 0, 1 $×& and 𝑴 𝒕 𝜷 ∈ 0, 1 $×(×& . Attention Annotation Platform (Risk Prediction Task with Counterfactual Estimation)
Real estate in NYC Domain experts evaluate the delivered attention 𝛼 𝑎𝑛𝑑 𝜷 via attention annotation mask 𝑴 𝒕 𝜶 ∈ 0, 1 $×& and 𝑴 𝒕 𝜷 ∈ 0, 1 $×(×& . Attention Annotation Platform (Real estate Price Forecasting in New York City)
Action Posture Correction Task Domain experts evaluate the delivered attention 𝛼 𝑎𝑛𝑑 𝜷 via attention annotation mask 𝑴 𝒕 𝜶 ∈ 0, 1 $×& and 𝑴 𝒕 𝜷 ∈ 0, 1 $×(×& . Attention Annotation Platform (Action Posture Correction Task)
Experiment Results Conducted experiments on three risk prediction tasks, one fitness squat, and one real estate forecasting task. EHR Fitness Squat Real Estate ForecastingHeart Failure Cerebral Infarction CVD One-time Training RETAIN 0.6069 ± 0.01 0.6394 ± 0.02 0.6018 ± 0.02 0.8425 ± 0.03 0.2136 ± 0.01 Random-RETAIN 0.5952 ± 0.02 0.6256 ± 0.02 0.5885 ± 0.01 0.8221 ± 0.05 0.2140 ± 0.01 IF-RETAIN 0.6134 ± 0.03 0.6422 ± 0.02 0.5882 ± 0.02 0.8363 ± 0.03 0.2049 ± 0.01 Random Re-ranking Random-UA 0.6231 ± 0.03 0.6491 ± 0.01 0.6112 ± 0.02 0.8521 ± 0.02 0.2222 ± 0.02 Random-NAP 0.6414 ± 0.01 0.6674 ± 0.02 0.6284 ± 0.01 0.8525 ± 0.01 0.2061 ± 0.01 IAL (Cost-effective) AILA 0.6363 ± 0.03 0.6602 ± 0.03 0.6193 ± 0.02 0.8425 ± 0.01 0.2119 ± 0.01 IAL-NAP 0.6612 ± 0.02 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.01 0.1835 ± 0.01
Experiment Results Conducted experiments on three risk prediction tasks, one fitness squat, and one real estate forecasting task. EHR Fitness Squat Real Estate ForecastingHeart Failure Cerebral Infarction CVD One-time Training RETAIN 0.6069 ± 0.01 0.6394 ± 0.02 0.6018 ± 0.02 0.8425 ± 0.03 0.2136 ± 0.01 Random-RETAIN 0.5952 ± 0.02 0.6256 ± 0.02 0.5885 ± 0.01 0.8221 ± 0.05 0.2140 ± 0.01 IF-RETAIN 0.6134 ± 0.03 0.6422 ± 0.02 0.5882 ± 0.02 0.8363 ± 0.03 0.2049 ± 0.01 Random Re-ranking Random-UA 0.6231 ± 0.03 0.6491 ± 0.01 0.6112 ± 0.02 0.8521 ± 0.02 0.2222 ± 0.02 Random-NAP 0.6414 ± 0.01 0.6674 ± 0.02 0.6284 ± 0.01 0.8525 ± 0.01 0.2061 ± 0.01 IAL (Cost-effective) AILA 0.6363 ± 0.03 0.6602 ± 0.03 0.6193 ± 0.02 0.8425 ± 0.01 0.2119 ± 0.01 IAL-NAP 0.6612 ± 0.02 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.01 0.1835 ± 0.01 • Random-UA, which is retrained with human attention-level supervision on randomly selected. samples, performs worse than Random-NAP.
Experiment Results Conducted experiments on three risk prediction tasks, one fitness squat, and one real estate forecasting task. EHR Fitness Squat Real Estate ForecastingHeart Failure Cerebral Infarction CVD One-time Training RETAIN 0.6069 ± 0.01 0.6394 ± 0.02 0.6018 ± 0.02 0.8425 ± 0.03 0.2136 ± 0.01 Random-RETAIN 0.5952 ± 0.02 0.6256 ± 0.02 0.5885 ± 0.01 0.8221 ± 0.05 0.2140 ± 0.01 IF-RETAIN 0.6134 ± 0.03 0.6422 ± 0.02 0.5882 ± 0.02 0.8363 ± 0.03 0.2049 ± 0.01 Random Re-ranking Random-UA 0.6231 ± 0.03 0.6491 ± 0.01 0.6112 ± 0.02 0.8521 ± 0.02 0.2222 ± 0.02 Random-NAP 0.6414 ± 0.01 0.6674 ± 0.02 0.6284 ± 0.01 0.8525 ± 0.01 0.2061 ± 0.01 IAL (Cost-effective) AILA 0.6363 ± 0.03 0.6602 ± 0.03 0.6193 ± 0.02 0.8425 ± 0.01 0.2119 ± 0.01 IAL-NAP 0.6612 ± 0.02 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.01 0.1835 ± 0.01 • Random-UA, which is retrained with human attention-level supervision on randomly selected. samples, performs worse than Random-NAP. • IAL-NAP significantly outperforms Random-NAP, showing that the effect of attention annotation cannot have much effect on the model when the instances are randomly selected.
Experiment Results Results of Ablation study with proposed IAL-NAP combinations for instance- and feat ure-level reranking on all tasks. IAL-NAP Variants EHR Fitness Squat Real Estate ForecastingInstance-level Feature-level Heart Failure Cerebral Infarction CVD Influence Function Uncertainty 0.6563 ± 0.01 0.6821 ± 0.02 0.6308 ± 0.02 0.8712 ± 0.01 0.1921 ± 0.01 Influence Function Influence Function 0.6514 ± 0.02 0.6825 ± 0.01 0.6329 ± 0.03 0.8632 ± 0.01 0.1865 ± 0.02 Influence Function Counterfactual 0.6592 ± 0.02 0.6921 ± 0.03 0.6379 ± 0.02 0.8682 ± 0.01 0.1863 ± 0.02 Uncertainty Counterfactual 0.6612 ± 0.01 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.02 0.1835 ± 0.02 • For instance-level scoring, influence and uncertainty scores work similarly Ablation study with IAL-NAP combinations
Experiment Results Results of Ablation study with proposed IAL-NAP combinations for instance- and feat ure-level reranking on all tasks. IAL-NAP Variants EHR Fitness Squat Real Estate ForecastingInstance-level Feature-level Heart Failure Cerebral Infarction CVD Influence Function Uncertainty 0.6563 ± 0.01 0.6821 ± 0.02 0.6308 ± 0.02 0.8712 ± 0.01 0.1921 ± 0.01 Influence Function Influence Function 0.6514 ± 0.02 0.6825 ± 0.01 0.6329 ± 0.03 0.8632 ± 0.01 0.1865 ± 0.02 Influence Function Counterfactual 0.6592 ± 0.02 0.6921 ± 0.03 0.6379 ± 0.02 0.8682 ± 0.01 0.1863 ± 0.02 Uncertainty Counterfactual 0.6612 ± 0.01 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.02 0.1835 ± 0.02 • For instance-level scoring, influence and uncertainty scores work similarly, while the counterfactual score was the most effective for feature-wise reranking. Ablation study with IAL-NAP combinations
Experiment Results Results of Ablation study with proposed IAL-NAP combinations for instance- and feat ure-level reranking on all tasks. IAL-NAP Variants EHR Fitness Squat Real Estate ForecastingInstance-level Feature-level Heart Failure Cerebral Infarction CVD Influence Function Uncertainty 0.6563 ± 0.01 0.6821 ± 0.02 0.6308 ± 0.02 0.8712 ± 0.01 0.1921 ± 0.01 Influence Function Influence Function 0.6514 ± 0.02 0.6825 ± 0.01 0.6329 ± 0.03 0.8632 ± 0.01 0.1865 ± 0.02 Influence Function Counterfactual 0.6592 ± 0.02 0.6921 ± 0.03 0.6379 ± 0.02 0.8682 ± 0.01 0.1863 ± 0.02 Uncertainty Counterfactual 0.6612 ± 0.01 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.02 0.1835 ± 0.02 • For instance-level scoring, influence and uncertainty scores work similarly, while the counterfactual score was the most effective for feature-wise reranking. • The combination of uncertainty-counterfactual is the most cost-effective solution since it avoids ex pensive computation of the Hessians. Ablation study with IAL-NAP combinations
Effect of Neural Attention Processes Retraining time to retrain examples of human annotation and Mean Response Time of human annotations on the risk prediction tasks. (a) Heart Failure (b) Cerebral Infarction (c) CVD
Effect of Neural Attention Processes Retraining time to retrain examples of human annotation and Mean Response Time of human annotations on the risk prediction tasks. (a) Heart Failure (b) Cerebral Infarction (c) CVD
Effect of Neural Attention Processes Retraining time to retrain examples of human annotation and Mean Response Time of human annotations on the risk prediction tasks. (a) Heart Failure (b) Cerebral Infarction (c) CVD
Effect of Cost-Effective Reranking Change of accuracy with 100 annotations across four rounds (S) between IAL-NAP (bl ue) vs Random-NAP (red). (b) Cerebral Infarction(a) Heart Failure (c) CVD (d) Squat • IAL-NAP uses a smaller number of annotated examples (100 examples) than Rando m-NAP (400 examples) to improve the model with the comparable accuracy (auc: 0. 6414).
Qualitative Analysis – Risk Prediction Further analyze the contribution of each feature for a CVD patient (label=1). A patient records for Cardio Vascular Disease • At s=3, IAL allocated more attention weights on the important feature (Smoking), which the initial training model missed to attend. • Age • Smoking : Whether smoke or not • SysBP : Systolic Blood Pressure • HDL : High-density Lipoprotein • LDL : Low-density Lipoprotein (a) Pretrained (b) s=1 (c) s=2
Qualitative Analysis – Risk Prediction Further analyze the contribution of each feature for a CVD patient (label=1). A patient records for Cardio Vascular Disease • At s=3, IAL allocated more attention weights on the important feature (Smoking), which the initial training model missed to attend. à Clinicians guided the model to learn it since smoking is a key factor to access CVD. • Age • Smoking : Whether smoke or not • SysBP : Systolic Blood Pressure • HDL : High-density Lipoprotein • LDL : Low-density Lipoprotein (a) Pretrained (b) s=1 (c) s=2
Summary Propose a novel interactive learning framework which iteratively updates the model by interacting with the human supervisor via the generated attentions. • Unlike conventional active learning, IAL allows the human annotators to “actively” interpret, manipulate the model’s behavior, and see its effect. IAL allows for online learning without retrai ning the main network, by training a “separate” attention generator.
Summary Propose a novel interactive learning framework which iteratively updates the model by interacting with the human supervisor via the generated attentions. • Unlike conventional active learning, IAL allows the human annotators to “actively” interpret, manipulate the model’s behavior, and see its effect. IAL allows for online learning without retrai ning the main network, by training a “separate” attention generator. • Neural Attention Processes is a novel attention mechanism which can generate attention on unla beled instances given few labeled samples, and can incorporate new labeled instances without retraining and overfitting.
Summary Propose a novel interactive learning framework which iteratively updates the model by interacting with the human supervisor via the generated attentions. • Unlike conventional active learning, IAL allows the human annotators to “actively” interpret, manipulate the model’s behavior, and see its effect. IAL allows for online learning without retrai ning the main network, by training a “separate” attention generator. • Neural Attention Processes is a novel attention mechanism which can generate attention on unla beled instances given few labeled samples, and can incorporate new labeled instances without retraining and overfitting. • Our reranking strategy re-ranks the instance and features, which substantially reduces the annot ation cost and time for high-dimensional inputs.
Thanks

More Related Content

PPTX
neural network
bySTUDENT
 
PDF
Task Adaptive Neural Network Search with Meta-Contrastive Learning
byMLAI2
 
PDF
Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distrib...
byMLAI2
 
PDF
Neural network image recognition
byOleksii Sekundant
 
PDF
Pattern Recognition using Artificial Neural Network
byEditor IJCATR
 
PPTX
Neural network
bySaddam Hussain
 
PPTX
Hyperparameter Tuning
byJon Lederman
 
PDF
Neural Computing
byJehoshaphat Abu
 
neural network
bySTUDENT
 
Task Adaptive Neural Network Search with Meta-Contrastive Learning
byMLAI2
 
Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distrib...
byMLAI2
 
Neural network image recognition
byOleksii Sekundant
 
Pattern Recognition using Artificial Neural Network
byEditor IJCATR
 
Neural network
bySaddam Hussain
 
Hyperparameter Tuning
byJon Lederman
 
Neural Computing
byJehoshaphat Abu
 

What's hot

PDF
Deep Dive into Hyperparameter Tuning
byShubhmay Potdar
 
PDF
Artificial neural network
byFaria Priya
 
PPT
Artificial Neural Networks - ANN
byMohamed Talaat
 
PDF
Neural network
byMuhammad Aleem Siddiqui
 
PPTX
ANN load forecasting
byDr Ashok Tiwari
 
PPTX
IROS 2017 Slides
bySungjoon Choi
 
PDF
Issues while working with Multi Layered Perceptron and Deep Neural Nets
byDEBJYOTI PAUL
 
PDF
Human uncertainty makes classification more robust, ICCV 2019 Review
byLEE HOSEONG
 
PPT
Slides ppt
bybutest
 
PDF
Advance deep learning
byaliaKhan71
 
PPTX
Neural networks
byRizwan Rizzu
 
PDF
Artificial Neural Network Paper Presentation
byguestac67362
 
PPTX
Genetic algorithm for hyperparameter tuning
byDr. Jyoti Obia
 
PDF
Data Science, Machine Learning and Neural Networks
byBICA Labs
 
PPTX
Artificial neural network
byGauravPandey319
 
PPTX
Deep Learning Enabled Question Answering System to Automate Corporate Helpdesk
bySaurabh Saxena
 
PDF
Applications in Machine Learning
byJoel Graff
 
PPTX
Artificial Neural Network(Artificial intelligence)
byspartacus131211
 
PPTX
Regression and Artificial Neural Network in R
byDr. Vaibhav Kumar
 
PPTX
Forecasting of Sales using Neural network techniques
byHitesh Dua
 
Deep Dive into Hyperparameter Tuning
byShubhmay Potdar
 
Artificial neural network
byFaria Priya
 
Artificial Neural Networks - ANN
byMohamed Talaat
 
Neural network
byMuhammad Aleem Siddiqui
 
ANN load forecasting
byDr Ashok Tiwari
 
IROS 2017 Slides
bySungjoon Choi
 
Issues while working with Multi Layered Perceptron and Deep Neural Nets
byDEBJYOTI PAUL
 
Human uncertainty makes classification more robust, ICCV 2019 Review
byLEE HOSEONG
 
Slides ppt
bybutest
 
Advance deep learning
byaliaKhan71
 
Neural networks
byRizwan Rizzu
 
Artificial Neural Network Paper Presentation
byguestac67362
 
Genetic algorithm for hyperparameter tuning
byDr. Jyoti Obia
 
Data Science, Machine Learning and Neural Networks
byBICA Labs
 
Artificial neural network
byGauravPandey319
 
Deep Learning Enabled Question Answering System to Automate Corporate Helpdesk
bySaurabh Saxena
 
Applications in Machine Learning
byJoel Graff
 
Artificial Neural Network(Artificial intelligence)
byspartacus131211
 
Regression and Artificial Neural Network in R
byDr. Vaibhav Kumar
 
Forecasting of Sales using Neural network techniques
byHitesh Dua
 

Similar to Cost-effective Interactive Attention Learning with Neural Attention Process

PDF
Julien Simon - Deep Dive - Accelerating Models with Better Attention Layers
byJulien SIMON
 
PDF
Deep Learning for Computer Vision: Attention Models (UPC 2016)
byUniversitat Politècnica de Catalunya
 
PPTX
transformer25aug nlp 2025 transformer .pptx
byProFa3
 
PDF
Melanie Warrick, Deep Learning Engineer, Skymind.io at MLconf SF - 11/13/15
byMLconf
 
PDF
Attention is All You Need (Transformer)
byJeong-Gwan Lee
 
PDF
Transformers in AI: Revolutionizing Natural Language Processing
bystudyandinnovation
 
PDF
Transformers: Revolutionizing NLP with Self-Attention
bystudyandinnovation
 
PPTX
Survey of Attention mechanism
bySwatiNarkhede1
 
PPTX
[Paper Reading] Attention is All You Need
byDaiki Tanaka
 
PDF
Context-Aware and Time-Aware Attention-Based Model for Disease Risk Predictio...
byOKOKPROJECTS
 
PDF
Transformer based approaches for visual representation learning
byRyohei Suzuki
 
PPTX
Deep Attention Model for Triage of Emergency Department Patients - Djordje Gl...
byInstitute of Contemporary Sciences
 
PDF
Attention is all you need (UPC Reading Group 2018, by Santi Pascual)
byUniversitat Politècnica de Catalunya
 
PDF
Deep learning and applications in non-cognitive domains II
byDeakin University
 
PDF
descriptipn Artificial intelliginet and
byssuser612109
 
PDF
Theoretical Neuroscience and Deep Learning Theory
byMLReview
 
PPTX
[AIoTLab]attention mechanism.pptx
byTuCaoMinh2
 
PDF
Hybrid deep learning model using recurrent neural network and gated recurrent...
byIJECEIAES
 
PDF
AI for Neuroscience and Neuroscience for AI
byMLconf
 
PPTX
240318_JW_labseminar[Attention Is All You Need].pptx
bythanhdowork
 
Julien Simon - Deep Dive - Accelerating Models with Better Attention Layers
byJulien SIMON
 
Deep Learning for Computer Vision: Attention Models (UPC 2016)
byUniversitat Politècnica de Catalunya
 
transformer25aug nlp 2025 transformer .pptx
byProFa3
 
Melanie Warrick, Deep Learning Engineer, Skymind.io at MLconf SF - 11/13/15
byMLconf
 
Attention is All You Need (Transformer)
byJeong-Gwan Lee
 
Transformers in AI: Revolutionizing Natural Language Processing
bystudyandinnovation
 
Transformers: Revolutionizing NLP with Self-Attention
bystudyandinnovation
 
Survey of Attention mechanism
bySwatiNarkhede1
 
[Paper Reading] Attention is All You Need
byDaiki Tanaka
 
Context-Aware and Time-Aware Attention-Based Model for Disease Risk Predictio...
byOKOKPROJECTS
 
Transformer based approaches for visual representation learning
byRyohei Suzuki
 
Deep Attention Model for Triage of Emergency Department Patients - Djordje Gl...
byInstitute of Contemporary Sciences
 
Attention is all you need (UPC Reading Group 2018, by Santi Pascual)
byUniversitat Politècnica de Catalunya
 
Deep learning and applications in non-cognitive domains II
byDeakin University
 
descriptipn Artificial intelliginet and
byssuser612109
 
Theoretical Neuroscience and Deep Learning Theory
byMLReview
 
[AIoTLab]attention mechanism.pptx
byTuCaoMinh2
 
Hybrid deep learning model using recurrent neural network and gated recurrent...
byIJECEIAES
 
AI for Neuroscience and Neuroscience for AI
byMLconf
 
240318_JW_labseminar[Attention Is All You Need].pptx
bythanhdowork
 

More from MLAI2

PDF
Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Unce...
byMLAI2
 
PDF
Online Hyperparameter Meta-Learning with Hypergradient Distillation
byMLAI2
 
PDF
Online Coreset Selection for Rehearsal-based Continual Learning
byMLAI2
 
PDF
Representational Continuity for Unsupervised Continual Learning
byMLAI2
 
PDF
Sequential Reptile_Inter-Task Gradient Alignment for Multilingual Learning
byMLAI2
 
PDF
Skill-Based Meta-Reinforcement Learning
byMLAI2
 
PDF
Edge Representation Learning with Hypergraphs
byMLAI2
 
PDF
Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Genera...
byMLAI2
 
PDF
Mini-Batch Consistent Slot Set Encoder For Scalable Set Encoding
byMLAI2
 
PDF
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint L...
byMLAI2
 
PDF
Meta-GMVAE: Mixture of Gaussian VAE for Unsupervised Meta-Learning
byMLAI2
 
PDF
Accurate Learning of Graph Representations with Graph Multiset Pooling
byMLAI2
 
PDF
Contrastive Learning with Adversarial Perturbations for Conditional Text Gene...
byMLAI2
 
PDF
Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Le...
byMLAI2
 
PDF
MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures
byMLAI2
 
PDF
Adversarial Self-Supervised Contrastive Learning
byMLAI2
 
PDF
Learning to Extrapolate Knowledge: Transductive Few-shot Out-of-Graph Link Pr...
byMLAI2
 
PDF
Neural Mask Generator : Learning to Generate Adaptive Word Maskings for Langu...
byMLAI2
 
PDF
Adversarial Neural Pruning with Latent Vulnerability Suppression
byMLAI2
 
PDF
Generating Diverse and Consistent QA pairs from Contexts with Information-Max...
byMLAI2
 
Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Unce...
byMLAI2
 
Online Hyperparameter Meta-Learning with Hypergradient Distillation
byMLAI2
 
Online Coreset Selection for Rehearsal-based Continual Learning
byMLAI2
 
Representational Continuity for Unsupervised Continual Learning
byMLAI2
 
Sequential Reptile_Inter-Task Gradient Alignment for Multilingual Learning
byMLAI2
 
Skill-Based Meta-Reinforcement Learning
byMLAI2
 
Edge Representation Learning with Hypergraphs
byMLAI2
 
Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Genera...
byMLAI2
 
Mini-Batch Consistent Slot Set Encoder For Scalable Set Encoding
byMLAI2
 
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint L...
byMLAI2
 
Meta-GMVAE: Mixture of Gaussian VAE for Unsupervised Meta-Learning
byMLAI2
 
Accurate Learning of Graph Representations with Graph Multiset Pooling
byMLAI2
 
Contrastive Learning with Adversarial Perturbations for Conditional Text Gene...
byMLAI2
 
Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Le...
byMLAI2
 
MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures
byMLAI2
 
Adversarial Self-Supervised Contrastive Learning
byMLAI2
 
Learning to Extrapolate Knowledge: Transductive Few-shot Out-of-Graph Link Pr...
byMLAI2
 
Neural Mask Generator : Learning to Generate Adaptive Word Maskings for Langu...
byMLAI2
 
Adversarial Neural Pruning with Latent Vulnerability Suppression
byMLAI2
 
Generating Diverse and Consistent QA pairs from Contexts with Information-Max...
byMLAI2
 

Recently uploaded

PDF
Post-Hackathon-Learnings-Maximizing-Impact-Beyond-the-Event.pdf
byishantyadav1111
 
PDF
What Is the Azure AI Foundry and Why Does It Matter for Enterprises?
byjohncarterjn
 
PDF
Pneumatic Pressure Pump PPP01 for Calibration & Testing
bySERRAX TECHNOLOGIES LLP
 
PDF
Transcript: What Thema can do: Leveraging metadata to support the discoverabi...
byBookNet Canada
 
PPTX
Introduction to Industrial-Arts Grade 8 ppt Lesson 1
byFSBTLEDNathanVince
 
PDF
Adapt.com Seed Fundraising Deck | The AI Computer for Business
byashley459565
 
PDF
Parental Control App for Phones_ The Complete 2026 Guide for Safer, Smarter P...
byRyan Cooper
 
PPTX
Introduction to Artificial Intelligence (AI).pptx
byammar414783
 
PDF
What Thema can do: Leveraging metadata to support the discoverability of Firs...
byBookNet Canada
 
PDF
OpenCharacter AI Reviews: Features, Plans, and Best Alternative
byOpenCharacter AI
 
PDF
ICT500 - CRITICAL AND CREATIVE THINKING FOR INFORMATION TECHNOLOGY SOLUTIONS:...
by2024432452
 
PDF
Computer-Based Training (CBT) The Backbone of Modern Technical & Defence Trai...
bycomputerbasedtrainin1
 
PDF
Website Redesign Vs Website Refresh, What’s Right for You in 2026?
byAnuj Kumar Singh
 
PPTX
The Lex Wire Precedent: A Technical Standard for Machine-Mediated Authority ...
byJeff Howell
 
PDF
UiPath Automation Developer Associate Training Series 2026 - Session 1
byDianaGray10
 
PDF
Title Installing Windows, Linux, MacOS.pdf
byGeraldAbadajos
 
PPTX
Jisc Equipment Data Service Update Workshop 3 December 2025
byJisc
 
PPTX
TechSprint WinterHack — Top 10 Teams Pitching Session we are excited for pit...
bybajpaitusharoffon678
 
PDF
The Enterprise Web3 Landscape - a view from Kaleido based on the past 10 year...
byPeter Broadhurst
 
PDF
Taxonomy Alignment for SDG Tagging - ASHA Case Study
byEnterprise Knowledge
 
Post-Hackathon-Learnings-Maximizing-Impact-Beyond-the-Event.pdf
byishantyadav1111
 
What Is the Azure AI Foundry and Why Does It Matter for Enterprises?
byjohncarterjn
 
Pneumatic Pressure Pump PPP01 for Calibration & Testing
bySERRAX TECHNOLOGIES LLP
 
Transcript: What Thema can do: Leveraging metadata to support the discoverabi...
byBookNet Canada
 
Introduction to Industrial-Arts Grade 8 ppt Lesson 1
byFSBTLEDNathanVince
 
Adapt.com Seed Fundraising Deck | The AI Computer for Business
byashley459565
 
Parental Control App for Phones_ The Complete 2026 Guide for Safer, Smarter P...
byRyan Cooper
 
Introduction to Artificial Intelligence (AI).pptx
byammar414783
 
What Thema can do: Leveraging metadata to support the discoverability of Firs...
byBookNet Canada
 
OpenCharacter AI Reviews: Features, Plans, and Best Alternative
byOpenCharacter AI
 
ICT500 - CRITICAL AND CREATIVE THINKING FOR INFORMATION TECHNOLOGY SOLUTIONS:...
by2024432452
 
Computer-Based Training (CBT) The Backbone of Modern Technical & Defence Trai...
bycomputerbasedtrainin1
 
Website Redesign Vs Website Refresh, What’s Right for You in 2026?
byAnuj Kumar Singh
 
The Lex Wire Precedent: A Technical Standard for Machine-Mediated Authority ...
byJeff Howell
 
UiPath Automation Developer Associate Training Series 2026 - Session 1
byDianaGray10
 
Title Installing Windows, Linux, MacOS.pdf
byGeraldAbadajos
 
Jisc Equipment Data Service Update Workshop 3 December 2025
byJisc
 
TechSprint WinterHack — Top 10 Teams Pitching Session we are excited for pit...
bybajpaitusharoffon678
 
The Enterprise Web3 Landscape - a view from Kaleido based on the past 10 year...
byPeter Broadhurst
 
Taxonomy Alignment for SDG Tagging - ASHA Case Study
byEnterprise Knowledge
 

Cost-effective Interactive Attention Learning with Neural Attention Process

  • 1.
    Jay Heo1, JunhyeonPark1, Hyewon Jeong1, Kwang joon Kim2, Juho Lee3, Eunho Yang1 3, Sung Ju Hwang1 3 Cost-Effective Interactive Attention Learning with Neural Attention Processes KAIST1, Yonsei University College of Medicine2, AITRICS3
  • 2.
    Model Interpretability Main NetworkInferenceInput Data Training The complex nature of deep neural networks has led to the recent surge of interests in interpretable models which provide model interpretations.
  • 3.
    Model Interpretability Main NetworkInferenceInput Data Model Interpretation Interpretation tool Inference The complex nature of deep neural networks has led to the recent surge of interests in interpretable models which provide model interpretations.
  • 4.
    Model Interpretability The complexnature of deep neural networks has led to the recent surge of interests in interpretable models which provide model interpretations. Main Network InferenceInput Data Model Interpretation Interpretation tool Inference Provide explanations for model’s decision.
  • 5.
    Challenge: Incorrect &Unreliable Interpretation Not all machine-generated interpretations are correct or human-understandable. • Correctness and reliability of a learning model heavily depends on quality and quantity of training data. • Neural networks tend to learn non-robust features that help with predictions, but are not human-perceptible. Model Interpretation Quality of training data Quantity of training data
  • 6.
    Challenge: Incorrect &Unreliable Interpretation Not all machine-generated interpretations are correct or human-understandable. • Correctness and reliability of a learning model heavily depends on quality and quantity of training data. • Neural networks tend to learn non-robust features that help with predictions, but are not human-perceptible. Model Interpretation Quality of training data Quantity of training data Whether a model learn too many non-robust features during training?
  • 7.
    Challenge: Incorrect &Unreliable Interpretation Not all machine-generated interpretations are correct or human-understandable. • Correctness and reliability of a learning model heavily depends on quality and quantity of training data. • Neural networks tend to learn non-robust features that help with predictions, but are not human-perceptible. Model Interpretation 1. Is it correct? 2. Is it understandable enough to trust? Quality of training data Quantity of training data Whether a model learn too many non-robust features during training?
  • 8.
    Interactive Learning Framework Proposean interactive learning framework which iteratively update the model by interacting with the human supervisors who adjust the provided interpretations. • Actively use human supervisors as a channel for human-model communications. Attentional Networks Human Annotator Learning Model Physician Interpretation (Attentions) Model’s decision 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention
  • 9.
    Interactive Learning Framework Proposean interactive learning framework which iteratively update the model by interacting with the human supervisors who adjust the provided interpretations. • Actively use human supervisors as a channel for human-model communications. Attentional Networks Human Annotator Learning Model Physician Interpretation (Attentions) Model’s decision Annotation AnnotateRetrain 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention
  • 10.
    Challenge: Model RetrainingCost To reflect human feedback, the model needs to be retrained, which is costly. • Retraining the model with scarce human feedback may result in the model overfitting Physician Learning Model Annotated Examples 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention Interpretation
  • 11.
    Challenge: Model RetrainingCost To reflect human feedback, the model needs to be retrained, which is costly. • Retraining the model with scarce human feedback may result in the model overfitting Physician Learning Model Annotated Examples 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention Interpretation Scarce feedback
  • 12.
    Challenge: Model RetrainingCost To reflect human feedback, the model needs to be retrained, which is costly. • Retraining the model with scarce human feedback may result in the model overfitting Physician Learning Model Annotated Examples 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention Interpretation Scarce feedback Retraining
  • 13.
    Challenge: Model RetrainingCost To reflect human feedback, the model needs to be retrained, which is costly. Physician Learning Model Annotated Examples 0.80.60.3 Low uncertainty High uncertainty High uncertainty : Attention Interpretation Scarce feedback Retraining ! Overfitting • Retraining the model with scarce human feedback may result in the model overfitting
  • 14.
    Challenge: Expensive HumanSupervision Cost Obtaining human feedback on datasets with large numbers of training instances and features is extremely costly. • Obtaining feedback on already correct or previously corrected interpretations is wasteful. Annotator Annotate 𝑴 𝒂𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 (𝒕) ∈{0, 1} Big data
  • 15.
    Challenge: Expensive HumanSupervision Cost Obtaining human feedback on datasets with large numbers of training instances and features is extremely costly. • Obtaining feedback on already correct or previously corrected interpretations is wasteful. Annotator Annotate Big data Annotate
  • 16.
    Challenge: Expensive HumanSupervision Cost Obtaining human feedback on datasets with large numbers of training instances and features is extremely costly. • Obtaining feedback on already correct or previously corrected interpretations is wasteful. Annotator Annotate Big data Costly Annotate !
  • 17.
    Interactive Attention LearningFramework Domain experts interactively evaluate learned attentions and provide feedbacks to obtain models that generate human-intuitive interpretations. Learning Model Physician Attention Mechanism LDL Respiration Cholesterol Creatine BMI Diabetes Heart Failure Hypertension Deliver attention Annotate attention Attention0.80.60.3 : Attention 𝑴 𝒂𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 (𝒕) ∈ {0, 1} Influence Function MC Dropout Annotation Mask Deep Interpretation Tool Correlation & Causal relationship Analysis Manipulate Explain Neural Attention Processes Counterfactual Estimation Counterfactual Estimation a c b Granger Causality
  • 18.
    Interactive Attention LearningFramework Domain experts interactively evaluate learned attentions and provide feedbacks to obtain models that generate human-intuitive interpretations. Learning Model Physician Attention Mechanism 1. Neural Attention Processes LDL Respiration Cholesterol Creatine BMI Diabetes Heart Failure Hypertension Deliver attention Annotate attention Attention0.80.60.3 : Attention 𝑴 𝒂𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 (𝒕) ∈ {0, 1} Influence Function MC Dropout Annotation Mask Deep Interpretation Tool Correlation & Causal relationship Analysis Manipulate Explain Neural Attention Processes Counterfactual Estimation Counterfactual Estimation a c b Granger Causality
  • 19.
    Interactive Attention LearningFramework Domain experts interactively evaluate learned attentions and provide feedbacks to obtain models that generate human-intuitive interpretations. Learning Model Physician Attention Mechanism LDL Respiration Cholesterol Creatine BMI Diabetes Heart Failure Hypertension Deliver attention Annotate attention Attention0.80.60.3 : Attention 𝑴 𝒂𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 (𝒕) ∈ {0, 1} Influence Function MC Dropout Annotation Mask Deep Interpretation Tool Correlation & Causal relationship Analysis Manipulate Explain Neural Attention Processes Counterfactual Estimation 1. Neural Attention Processes 2. Cost-effective Reranking Counterfactual Estimation a c b Granger Causality
  • 20.
    Neural Attention Processes(NAP) NAP naturally reflects the information from the annotation summarization z via amor tized inference. Domain Expert Annotation Neural Attention Processes • NAP learns to summarize delivered annotations to a latent vector, and gives th e summarization as an additional input to the attention generating network.
  • 21.
    Neural Attention Processes(NAP) NAP naturally reflects the information from the annotation summarization z via amor tized inference. Domain Expert Annotation Neural Attention Processes • NAP learns to summarize delivered annotations to a latent vector, and gives th e summarization as an additional input to the attention generating network.
  • 22.
    Neural Attention Processes(NAP) ! Attention Context Points "! !! "! #! !! New Observations !!"# "!"# !!"# NAP minimizes retraining cost by incorporating new labeled instances without retrai ning and overfitting. First Round (s=1) • NAP doesn’t require retraining for further new observations, in that NAP automatically adapt to them at the cost of a forward pass through a network 𝑔!.
  • 23.
    Neural Attention Processes(NAP) ! Attention Context Points "! !! "! #! !! New Observations !!"# "!"# !!"# NAP minimizes retraining cost by incorporating new labeled instances without retrai ning and overfitting. First Round (s=1) • NAP doesn’t require retraining for further new observations, in that NAP automatically adapt to them at the cost of a forward pass through a network 𝑔!.
  • 24.
    Neural Attention Processes(NAP) ! Attention Context Points "! !! "! #! !! New Observations !!"# "!"# !!"# NAP minimizes retraining cost by incorporating new labeled instances without retrai ning and overfitting. First Round (s=1) • NAP doesn’t require retraining for further new observations, in that NAP automatically adapt to them at the cost of a forward pass through a network 𝑔!.
  • 25.
    Neural Attention Processes(NAP) ! Attention Context Points "! !! "! #! !! New Observations !!"# "!"# !!"# NAP minimizes retraining cost by incorporating new labeled instances without retrai ning and overfitting. First Round (s=1) • NAP doesn’t require retraining for further new observations, in that NAP automatically adapt to them at the cost of a forward pass through a network 𝑔!.
  • 26.
    Neural Attention Processes(NAP) NAP minimizes retraining cost by incorporating new labeled instances without retrain ing and overfitting. • NAP is trained in a meta-learning fashion for few-shot function estimation, where it is trained to predict the attention mask of other labeled samples, given a randomly selected labeled set as context. ! Attention "! !! New Observations !!"# "!"# !!"# Context Points !! "! #!!!"$ "!"$ !!"$ Further Rounds (s=2,3,4)
  • 27.
    Neural Attention Processes(NAP) NAP minimizes retraining cost by incorporating new labeled instances without retrain ing and overfitting. • NAP is trained in a meta-learning fashion for few-shot function estimation, where it is trained to predict the attention mask of other labeled samples, given a randomly selected labeled set as context. ! Attention "! !! New Observations !!"# "!"# !!"# Context Points !! "! #!!!"$ "!"$ !!"$ Further Rounds (s=2,3,4)
  • 28.
    Neural Attention Processes(NAP) NAP minimizes retraining cost by incorporating new labeled instances without retrain ing and overfitting. • NAP is trained in a meta-learning fashion for few-shot function estimation, where it is trained to predict the attention mask of other labeled samples, given a randomly selected labeled set as context. ! Attention "! !! New Observations !!"# "!"# !!"# Context Points !! "! #!!!"$ "!"$ !!"$ Further Rounds (s=2,3,4)
  • 29.
    Cost-Effective Instance &Features Reranking (CER) Address the expensive human labeling cost by reranking the instances, features, and timesteps (for time-series data) by their negative impacts. Attentional Network Domain Expert Instance-wise and Feature-wise Reranking
  • 30.
    Cost-Effective Instance &Features Reranking (CER) Address the expensive human labeling cost by reranking the instances, features, and timesteps (for time-series data) by their negative impacts. Attentional Network Domain Expert Instance-wise and Feature-wise Reranking 𝑷 𝑲 … … …TrainTrainValid Valid Estimate 𝑰(𝒖!) / 𝐕𝐚𝐫(𝒖!) Instance-level Reranking Select Re-rank & select
  • 31.
    Cost-Effective Instance &Features Reranking (CER) Address the expensive human labeling cost by reranking the instances, features, and timesteps (for time-series data) by their negative impacts. Attentional Network Domain Expert Instance-wise and Feature-wise Reranking 𝑷 𝑲 … … …TrainTrainValid Valid Estimate 𝑰(𝒖!) / 𝐕𝐚𝐫(𝒖!) Instance-level Reranking Train Feature-level Reranking Select Re-rank & select Estimate 𝑰(𝒖",$ (&) ) / 𝐕𝐚𝒓 (𝒖",$ (&) ) / 𝜓(𝒖",$ (&) ) Feature … … Feature 𝑭 Re-rank & select
  • 32.
    CER: 1. InfluenceScore Use the influence function (Koh & Liang, 2017) to approximate the impact of individual training points on the model prediction. Fish Dog Dog “Dog” Training Training data Test Input [Koh and Liang 17] Understanding Black-box Predictions via Influence Functions, ICML 2017
  • 33.
    CER: 2. UncertaintyScore Measure the negative impacts using the uncertainty which can be measured by Mon te-Carlo sampling (Gal & Ghahramani, 2016). Uncertainty-aware Attention Mechanism • Less expensive approach to measure the negative impacts. • Assume that instances having high-predictive uncertainties are potential candidate to be corrected. Uncertainty [Jay Heo*, Haebeom Lee*, Saehun Kim,, Juho Lee, Gwangjun Kim , Eunho Yang, Sung Joo Hwang] Uncertainty-aware Attention Mechanism for Reliable prediction and Interpretation, Neurips 2018.
  • 34.
    CER: 2. UncertaintyScore Measure the negative impacts using the uncertainty which can be measured by Mon te-Carlo sampling (Gal & Ghahramani, 2016). Measure instance-wise & feature-wise uncertainty 0.80.60.3 Low uncertainty High uncertainty High uncertainty µ σ, )(N SpO2 Pulse Respiration • Less expensive approach to measure the negative impacts. • Assume that instances having high-predictive uncertainties are potential candidate to be corrected. Uncertainty-aware Attention Mechanism [Jay Heo*, Haebeom Lee*, Saehun Kim,, Juho Lee, Gwangjun Kim , Eunho Yang, Sung Joo Hwang] Uncertainty-aware Attention Mechanism for Reliable prediction and Interpretation, Neurips 2018.
  • 35.
    CER: 3. CounterfactualScore How would the prediction change if we ignore a certain feature by manually turning on/off the corresponding attention value? • Not need for retraining since we can simply set its attention value to zero. • Used to rerank the features with regards to their importance. Counterfactual Estimation Interface
  • 36.
    CER: 3. CounterfactualScore How would the prediction change if we ignore a certain feature by manually turning on/off the corresponding attention value? • Not need for retraining since we can simply set its attention value to zero. • Used to rerank the features with regards to their importance. Counterfactual Estimation Interface
  • 37.
    CER: 3. CounterfactualScore How would the prediction change if we ignore a certain feature by manually turning on/off the corresponding attention value? • Not need for retraining since we can simply set its attention value to zero. • Used to rerank the features with regards to their importance. Counterfactual Estimation Interface
  • 38.
    CER: 3. CounterfactualScore How would the prediction change if we ignore a certain feature by manually turning on/off the corresponding attention value? • Not need for retraining since we can simply set its attention value to zero. • Used to rerank the features with regards to their importance. Counterfactual Estimation Interface
  • 39.
    Experimental setting –Datasets Use electronic health records, real estate sales transaction records, and exercise squat posture correction records for classification and regression tasks. 1. EHR Datasets 1) Cerebral Infarction 2) Cardio Vascular Disease 3) Heart Failure Binary Classification task
  • 40.
    Experimental setting –Datasets Use electronic health records, real estate sales transaction records, and exercise squat posture correction records for classification and regression tasks. 1. EHR Datasets 1) Cerebral Infarction 2) Cardio Vascular Disease 3) Heart Failure 2. Real-estate dataset 1) Housing price forecasting Binary Classification task Regression task
  • 41.
    Experimental setting –Datasets Use electronic health records, real estate sales transaction records, and exercise squat posture correction records for classification and regression tasks. 1. EHR Datasets 1) Cerebral Infarction 2) Cardio Vascular Disease 3) Heart Failure 2. Real-estate dataset 3. Squat Posture set 1) Housing price forecasting 1) Squat posture correction Binary Classification task Regression task Multi-label Classification task
  • 42.
    Attention Evaluation Interface– Risk Prediction Domain experts evaluate the delivered attention 𝛼 𝑎𝑛𝑑 𝜷 via attention annotation mask 𝑴 𝒕 𝜶 ∈ 0, 1 $×& and 𝑴 𝒕 𝜷 ∈ 0, 1 $×(×& . Attention Annotation Platform (Risk Prediction Task with Counterfactual Estimation)
  • 43.
    Real estate inNYC Domain experts evaluate the delivered attention 𝛼 𝑎𝑛𝑑 𝜷 via attention annotation mask 𝑴 𝒕 𝜶 ∈ 0, 1 $×& and 𝑴 𝒕 𝜷 ∈ 0, 1 $×(×& . Attention Annotation Platform (Real estate Price Forecasting in New York City)
  • 44.
    Action Posture CorrectionTask Domain experts evaluate the delivered attention 𝛼 𝑎𝑛𝑑 𝜷 via attention annotation mask 𝑴 𝒕 𝜶 ∈ 0, 1 $×& and 𝑴 𝒕 𝜷 ∈ 0, 1 $×(×& . Attention Annotation Platform (Action Posture Correction Task)
  • 45.
    Experiment Results Conducted experimentson three risk prediction tasks, one fitness squat, and one real estate forecasting task. EHR Fitness Squat Real Estate ForecastingHeart Failure Cerebral Infarction CVD One-time Training RETAIN 0.6069 ± 0.01 0.6394 ± 0.02 0.6018 ± 0.02 0.8425 ± 0.03 0.2136 ± 0.01 Random-RETAIN 0.5952 ± 0.02 0.6256 ± 0.02 0.5885 ± 0.01 0.8221 ± 0.05 0.2140 ± 0.01 IF-RETAIN 0.6134 ± 0.03 0.6422 ± 0.02 0.5882 ± 0.02 0.8363 ± 0.03 0.2049 ± 0.01 Random Re-ranking Random-UA 0.6231 ± 0.03 0.6491 ± 0.01 0.6112 ± 0.02 0.8521 ± 0.02 0.2222 ± 0.02 Random-NAP 0.6414 ± 0.01 0.6674 ± 0.02 0.6284 ± 0.01 0.8525 ± 0.01 0.2061 ± 0.01 IAL (Cost-effective) AILA 0.6363 ± 0.03 0.6602 ± 0.03 0.6193 ± 0.02 0.8425 ± 0.01 0.2119 ± 0.01 IAL-NAP 0.6612 ± 0.02 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.01 0.1835 ± 0.01
  • 46.
    Experiment Results Conducted experimentson three risk prediction tasks, one fitness squat, and one real estate forecasting task. EHR Fitness Squat Real Estate ForecastingHeart Failure Cerebral Infarction CVD One-time Training RETAIN 0.6069 ± 0.01 0.6394 ± 0.02 0.6018 ± 0.02 0.8425 ± 0.03 0.2136 ± 0.01 Random-RETAIN 0.5952 ± 0.02 0.6256 ± 0.02 0.5885 ± 0.01 0.8221 ± 0.05 0.2140 ± 0.01 IF-RETAIN 0.6134 ± 0.03 0.6422 ± 0.02 0.5882 ± 0.02 0.8363 ± 0.03 0.2049 ± 0.01 Random Re-ranking Random-UA 0.6231 ± 0.03 0.6491 ± 0.01 0.6112 ± 0.02 0.8521 ± 0.02 0.2222 ± 0.02 Random-NAP 0.6414 ± 0.01 0.6674 ± 0.02 0.6284 ± 0.01 0.8525 ± 0.01 0.2061 ± 0.01 IAL (Cost-effective) AILA 0.6363 ± 0.03 0.6602 ± 0.03 0.6193 ± 0.02 0.8425 ± 0.01 0.2119 ± 0.01 IAL-NAP 0.6612 ± 0.02 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.01 0.1835 ± 0.01 • Random-UA, which is retrained with human attention-level supervision on randomly selected. samples, performs worse than Random-NAP.
  • 47.
    Experiment Results Conducted experimentson three risk prediction tasks, one fitness squat, and one real estate forecasting task. EHR Fitness Squat Real Estate ForecastingHeart Failure Cerebral Infarction CVD One-time Training RETAIN 0.6069 ± 0.01 0.6394 ± 0.02 0.6018 ± 0.02 0.8425 ± 0.03 0.2136 ± 0.01 Random-RETAIN 0.5952 ± 0.02 0.6256 ± 0.02 0.5885 ± 0.01 0.8221 ± 0.05 0.2140 ± 0.01 IF-RETAIN 0.6134 ± 0.03 0.6422 ± 0.02 0.5882 ± 0.02 0.8363 ± 0.03 0.2049 ± 0.01 Random Re-ranking Random-UA 0.6231 ± 0.03 0.6491 ± 0.01 0.6112 ± 0.02 0.8521 ± 0.02 0.2222 ± 0.02 Random-NAP 0.6414 ± 0.01 0.6674 ± 0.02 0.6284 ± 0.01 0.8525 ± 0.01 0.2061 ± 0.01 IAL (Cost-effective) AILA 0.6363 ± 0.03 0.6602 ± 0.03 0.6193 ± 0.02 0.8425 ± 0.01 0.2119 ± 0.01 IAL-NAP 0.6612 ± 0.02 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.01 0.1835 ± 0.01 • Random-UA, which is retrained with human attention-level supervision on randomly selected. samples, performs worse than Random-NAP. • IAL-NAP significantly outperforms Random-NAP, showing that the effect of attention annotation cannot have much effect on the model when the instances are randomly selected.
  • 48.
    Experiment Results Results ofAblation study with proposed IAL-NAP combinations for instance- and feat ure-level reranking on all tasks. IAL-NAP Variants EHR Fitness Squat Real Estate ForecastingInstance-level Feature-level Heart Failure Cerebral Infarction CVD Influence Function Uncertainty 0.6563 ± 0.01 0.6821 ± 0.02 0.6308 ± 0.02 0.8712 ± 0.01 0.1921 ± 0.01 Influence Function Influence Function 0.6514 ± 0.02 0.6825 ± 0.01 0.6329 ± 0.03 0.8632 ± 0.01 0.1865 ± 0.02 Influence Function Counterfactual 0.6592 ± 0.02 0.6921 ± 0.03 0.6379 ± 0.02 0.8682 ± 0.01 0.1863 ± 0.02 Uncertainty Counterfactual 0.6612 ± 0.01 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.02 0.1835 ± 0.02 • For instance-level scoring, influence and uncertainty scores work similarly Ablation study with IAL-NAP combinations
  • 49.
    Experiment Results Results ofAblation study with proposed IAL-NAP combinations for instance- and feat ure-level reranking on all tasks. IAL-NAP Variants EHR Fitness Squat Real Estate ForecastingInstance-level Feature-level Heart Failure Cerebral Infarction CVD Influence Function Uncertainty 0.6563 ± 0.01 0.6821 ± 0.02 0.6308 ± 0.02 0.8712 ± 0.01 0.1921 ± 0.01 Influence Function Influence Function 0.6514 ± 0.02 0.6825 ± 0.01 0.6329 ± 0.03 0.8632 ± 0.01 0.1865 ± 0.02 Influence Function Counterfactual 0.6592 ± 0.02 0.6921 ± 0.03 0.6379 ± 0.02 0.8682 ± 0.01 0.1863 ± 0.02 Uncertainty Counterfactual 0.6612 ± 0.01 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.02 0.1835 ± 0.02 • For instance-level scoring, influence and uncertainty scores work similarly, while the counterfactual score was the most effective for feature-wise reranking. Ablation study with IAL-NAP combinations
  • 50.
    Experiment Results Results ofAblation study with proposed IAL-NAP combinations for instance- and feat ure-level reranking on all tasks. IAL-NAP Variants EHR Fitness Squat Real Estate ForecastingInstance-level Feature-level Heart Failure Cerebral Infarction CVD Influence Function Uncertainty 0.6563 ± 0.01 0.6821 ± 0.02 0.6308 ± 0.02 0.8712 ± 0.01 0.1921 ± 0.01 Influence Function Influence Function 0.6514 ± 0.02 0.6825 ± 0.01 0.6329 ± 0.03 0.8632 ± 0.01 0.1865 ± 0.02 Influence Function Counterfactual 0.6592 ± 0.02 0.6921 ± 0.03 0.6379 ± 0.02 0.8682 ± 0.01 0.1863 ± 0.02 Uncertainty Counterfactual 0.6612 ± 0.01 0.6892 ± 0.03 0.6371 ± 0.02 0.8689 ± 0.02 0.1835 ± 0.02 • For instance-level scoring, influence and uncertainty scores work similarly, while the counterfactual score was the most effective for feature-wise reranking. • The combination of uncertainty-counterfactual is the most cost-effective solution since it avoids ex pensive computation of the Hessians. Ablation study with IAL-NAP combinations
  • 51.
    Effect of NeuralAttention Processes Retraining time to retrain examples of human annotation and Mean Response Time of human annotations on the risk prediction tasks. (a) Heart Failure (b) Cerebral Infarction (c) CVD
  • 52.
    Effect of NeuralAttention Processes Retraining time to retrain examples of human annotation and Mean Response Time of human annotations on the risk prediction tasks. (a) Heart Failure (b) Cerebral Infarction (c) CVD
  • 53.
    Effect of NeuralAttention Processes Retraining time to retrain examples of human annotation and Mean Response Time of human annotations on the risk prediction tasks. (a) Heart Failure (b) Cerebral Infarction (c) CVD
  • 54.
    Effect of Cost-EffectiveReranking Change of accuracy with 100 annotations across four rounds (S) between IAL-NAP (bl ue) vs Random-NAP (red). (b) Cerebral Infarction(a) Heart Failure (c) CVD (d) Squat • IAL-NAP uses a smaller number of annotated examples (100 examples) than Rando m-NAP (400 examples) to improve the model with the comparable accuracy (auc: 0. 6414).
  • 55.
    Qualitative Analysis –Risk Prediction Further analyze the contribution of each feature for a CVD patient (label=1). A patient records for Cardio Vascular Disease • At s=3, IAL allocated more attention weights on the important feature (Smoking), which the initial training model missed to attend. • Age • Smoking : Whether smoke or not • SysBP : Systolic Blood Pressure • HDL : High-density Lipoprotein • LDL : Low-density Lipoprotein (a) Pretrained (b) s=1 (c) s=2
  • 56.
    Qualitative Analysis –Risk Prediction Further analyze the contribution of each feature for a CVD patient (label=1). A patient records for Cardio Vascular Disease • At s=3, IAL allocated more attention weights on the important feature (Smoking), which the initial training model missed to attend. à Clinicians guided the model to learn it since smoking is a key factor to access CVD. • Age • Smoking : Whether smoke or not • SysBP : Systolic Blood Pressure • HDL : High-density Lipoprotein • LDL : Low-density Lipoprotein (a) Pretrained (b) s=1 (c) s=2
  • 57.
    Summary Propose a novelinteractive learning framework which iteratively updates the model by interacting with the human supervisor via the generated attentions. • Unlike conventional active learning, IAL allows the human annotators to “actively” interpret, manipulate the model’s behavior, and see its effect. IAL allows for online learning without retrai ning the main network, by training a “separate” attention generator.
  • 58.
    Summary Propose a novelinteractive learning framework which iteratively updates the model by interacting with the human supervisor via the generated attentions. • Unlike conventional active learning, IAL allows the human annotators to “actively” interpret, manipulate the model’s behavior, and see its effect. IAL allows for online learning without retrai ning the main network, by training a “separate” attention generator. • Neural Attention Processes is a novel attention mechanism which can generate attention on unla beled instances given few labeled samples, and can incorporate new labeled instances without retraining and overfitting.
  • 59.
    Summary Propose a novelinteractive learning framework which iteratively updates the model by interacting with the human supervisor via the generated attentions. • Unlike conventional active learning, IAL allows the human annotators to “actively” interpret, manipulate the model’s behavior, and see its effect. IAL allows for online learning without retrai ning the main network, by training a “separate” attention generator. • Neural Attention Processes is a novel attention mechanism which can generate attention on unla beled instances given few labeled samples, and can incorporate new labeled instances without retraining and overfitting. • Our reranking strategy re-ranks the instance and features, which substantially reduces the annot ation cost and time for high-dimensional inputs.
  • 60.