Interpreting and Steering AI Explanations Via Interactive Visualizations

Qianwen WANG, PhD
Postdoctoral Fellow
Department of Biomedical Informatics
Harvard Medical School
FromAI Models toAIApplications
Interpreting and Steering AI Explanations
with InteractiveVisualizations
Powerful AI Model
Usable and Useful AI Applications
1

2
About Me
Tenure-Track Assistant Professor
Aug 2023, CSE@UMN
Ph.D study at HKUST
with Prof. Huamin Qu
2015-2020
2015
2020
PostDoc at Harvard
with Prof. Nils Gehlenborg
2020-2023
Oxford
https://qianwen.info
•Awardee of the Harvard DSI Postdoctoral
Research Fund
•Abstract Chair for ISMB BioVis, Poster Chair for
PacificVis
•Program Committee members of IEEE VIS,
ACM IUI, ChinaVis.
•Honorable mention award from IEEE VIS 2022
•Best paper award from IMLH@ICML 2021
•Best abstract awards from BioVis ISMB 2021
and 2022
B.Eng at XJTU
2011-2015

3
We are Hiring
The Department of Computer Science and Engineering
University of Minnesota, Twin Cites (UMN)
I am seeking highly motivated students, RAs, interns, and visitors to be part of our
dynamic team at UMN CSE. Feel free to drop me an email if you are interested!
Twin cites, Minnesota
• Boasts 29 Nobel Laureates and 3 Pulitzer Prize winners
among its alumni.
• 44th in Academic Ranking of World University, 2022
• The CSE department is recognized for housing numerous
esteemed scholars, including Tian He, Vipin Kumar,
Joseph Konstan, etc
• One of the largest metropolitan areas in
the Midwestern US
• Land of 10,000 lakes
• Good public transportation, a thriving arts
scene, and teams in all four major
professional sports (NBA, NFL, MLB, NHL)

How can domain users apply AI to complete
desired tasks easily and efficiently
How can domain users apply AI to complete
desired tasks easily and efficiently
Adapt from Langer et al. 2021. What Do We Want From Explainable Artificial Intelligence
AI and Human
AI User
Developer
Deployer
Aﬀected
Parties
Regulator
AI Application
5

Even if we were to make no further progress in the next decade,
deploying existing AI algorithms to every applicable problem would be a
game changer for most industries.
— Francois Chollet
6
The Importance of AI Application
Medical
Diagnosis
Drug Design Personalised
Medicine
Prognosis
Prediction
Healthcare
Chatbot

Epic’s AI algorithms are delivering
inaccurate information on
seriously ill patients
MIKE REDDY FOR STAT
https://www.statnews.com/2021/07/26/epic-hospital-algorithms-sepsis-investigation/?
utm_source=researcher_app&utm_medium=referral&utm_campaign=RESR_MRKT_Rese
archer_inbound
https://www.fiercehealthcare.com/practices/nearly-half-u-s-doctors-
say-they-are-anxious-about-using-ai-powered-software-survey
It is hard to achieve, especially in biomedical applications
7
The Challenges in AI Application

Why it is hard to achieve
The capabilities of AI The needs from users
8
AI Application

The gap needs
to be filled!
Abstract benchmark tasks
The capabilities of AI The needs from users
Complicated domain-specific tasks
9
Why it is hard to achieve
AI Application

Powerful AI Model
Usable and Useful AI Applications
10
Interactive Visualization
Explainable AI
Filling the Gap

Stages of the
Human Communication
Uijt!qbujfou!tipvme!cf!ejbhoptfe!
xjui!Ejtfbtf!B!
◥Cfdbvtf!pg!uif!sftvmut!pg!
uftu!N!!boe!!uftu!O◤
◥Pi-!sjhiu-!uif!sftvmu!pg!
uftu!N!joejdbuft!uibu１/◤
◥Cvu!uif!sftvmu!pg!uftu!O!
dbo!cf!mjolfe!up!bopuifs!
ejtfbtf!◤
Nbz!J!lopx!zpvs!
tvhhftujpot@
11
Receive
Interpret
Feedback

InteractiveVisualizations + Explainable / Interactive AI
AI Applications
12
Receive
Interpret
Feedback

InteractiveVisualizations + Explainable / Interactive AI
13
Receive
Interpret
Feedback
AI Applications

Enable users to provide feedback and
steer AI models for the desired tasks
14
Help users interpret AI and generate
meaningful and actionable insights
Present users explanations about the AI
for the desired tasks
Receive
Interpret
Feedback
My Studies

Drug Repurposing
Biomedical Knowledge Graph
Single-Cell Transcriptomics
Genomics
Genomics, Pathology
Cancer Genomics, Patent Cohort
15
• DNN Genealogy (TVCG 19)
• DiscriLens (VIS 20)
• GNNLens (TVCG 21)
• ATMSeer (CHI 19)
• HypoML (VIS 20)
• ThreadStates (VIS 21)
• GenoRec (VIS 22)
• DrugExplorer🏅(IMLH@ICML 21, VIS 22)
• Polyphony🏅(VIS 22, Biovis ISMB 22)
• Drava (CHI 2023)
Receive
Interpret
Feedback
My Studies

Drug Repurposing
Biomedical Knowledge Graph
Single-Cell Transcriptomics
Genomics
Genomics, Pathology
Cancer Genomics, Patent Cohort
16
• DNN Genealogy (TVCG 19)
• DiscriLens (VIS 20)
• GNNLens (TVCG 21)
• ATMSeer (CHI 19)
• HypoML (VIS 20)
• ThreadStates (VIS 21)
• GenoRec (VIS 22)
• DrugExplorer🏅(IMLH@ICML 21, VIS 22)
• Polyphony🏅(VIS 22, Biovis ISMB 22)
Receive
Interpret
Feedback
My Studies

Receiving Explanations
DO NOT Guarantee Insights!
Receive
Interpret

Debugging Tests for Model Explanations,
NeurIPs 2020, Julius Adebayo, Michael Muelly, Ilaria Liccardi, Been Kim
Human subjects fail to identify defective models
using attribution-based explanations, but instead
rely, primarily, on model predictions.
Attribution-based Explanations
18
Receiving Explanations
DO NOT Guarantee Insights!

R
Natural Images
19

Natural Images
Anshul Kundaje, Stanford University
Deep learning approaches to decode the human genome
Regulatory Genomic
Insights from Explanations,
it depends…

21
How to Select and Present aVisual
Explanation that can lead to
actionable insights?

Designing InteractiveVisualizations
for User-Centric XAI
Payal Chandak
Kexin Huang
Nils Gehlenborg Marinka Zitnik
HARVARD-MIT
Qianwen Wang
Best Paper Award
IMLH@ICML 2021
Best Paper Honorable Mention
IEEE VIS Conference 2022
IEEE Transactions on Visualization and Computer Graphics
A Study on GNN-based Drug Repurposing
22

Nodes: drugs, diseases, proteins, etc
Edges: known relations among these nodes
Graph Neural Networks (GNNs)
for Drug Repurposing
23

13-15 YEARS
$2-3 BILLION
Develop a new drug from scratch and
get it to the market
< 1/2 time
~ 1/4 cost
Drug Repurposing
Identify new therapeutic uses of existing drugs
Hair loss
Hypertension
24
Graph Neural Networks (GNNs)
for Drug Repurposing

AI Algorithm:
Explain a GNN model
AI Application:
Explain a GNN model used for
drug repurposing to Domain Users
Hao Yuan et al. 2022 25

Wang, Danding, et al. "Designing theory-driven user-centric
explainable AI." Proceedings of the 2019 CHI conference on
human factors in computing systems. 2019.
Liao, Q. Vera, Daniel Gruen, and Sarah Miller. "Questioning the
AI: informing design practices for explainable AI user
experiences." Proceedings of the 2020 CHI Conference on
Human Factors in Computing Systems. 2020.
For General Users, not domain specific
For General Interfaces, little discussion about visualisation design
26

An Extended Nested Model
!"#$%& '()*$&$+%"&
,-.+/$0+%"&
1%.2$*%3$+%"&
!"#$%&'()%#*
!"#$!%&&$'()$*")
+,$)-.&/0/*%#0
,*4"/%+5#
+,%#"&-./
!"1
!"1$0#*
!"/*$%2
"#!$*#$
!"/*$)&()
012'13$.#-&45
6%/7%#-/$'
13$.#-&45
()"$%'8./&%9&
!")0$/03$!")4)$
!%&&$)-.&/0/*%#0($
5)$'()3
012':."3
!"1$)-.&/0/*%#0(
/4)$0))3)3
;.5"-/'
<9,3"/"&-./
"#!$3#$3#6/%0
'()4($)-.&/%0$
*#$*")%4$.))4(
:#"/=3"#-&>
&#7/&8$94#'.8$
#4$9&#5/&
?.#5"&
)-/6.&)8$
/**4%5'*%#08$
7&/'()
0#3)8
./*"8
('594/."
9)0)4/*)$+,$
)-.&/0/*%#0(
:%('/&%;)$/03$
'.3/*)$+,$
)-.&/0/*%#0(
./*"<5/()3
)-.&/0/*%#0
!"/*$)&()
!"#$%&'#()"
*"')+(",
!"1
1 2 2 drug gene/protein cellular_com.. gene/protein disease
2 drug drug disease disease disease
1 drug disease drug disease disease
Agalsidase beta chylomicron ret... Alipogene tipar... lysosomal acid l... Wolman disease
indication indication indication includes
1 drug disease gene/protein disease disease
1 drug disease gene/protein disease
Agalsidase beta
Avelumab
Idursulfase
Galsulfase
&#7/&
)-.&/0/*%#0
94#'.
)-.&/0/*%#0
XAI Design Considerations
27

CommonVisual Presentations
of GNN Explanations
29
Alzheimer
BCHE
ACHE
BCL2
BAX
endometriu...
Organopho...
Chlorpyrifos
Chlorpyrifos
Neurotrans...
nucleus
1-BENZYL...
Methylphos...
Ranitidine
Estrogen-de...
The NLRP1 ...
Activation o...
Estrogen-de...
Ibuprofen
Release of a...
TP53 Regul...
Transcriptio...
nucleus
Repaglinide
hypogonad...
ABCC8
PPARG
CYP2C8
Hypogonadi...
Betamethas...
Testosterone
monobutyl ...
Regulation ...
ATP sensitiv...
ATP
endometriu...
Diethylhexy...
Air Pollutants
Dibutyl Pht...
nucleus
nucleoplasm
alpha-Linol...
Vemurafenib
Diethylstilb...
Ibuprofen
Neighbor Nodes
a Subgraph
b
disease gene/protein anatomy gene/protein drug
Alzheimer BCHE endometrium e... ABCC8 Repaglinide
disease_protein absent absent drug targets
disease gene/protein cellular_com.. gene/protein drug
Alzheimer ACHE nucleus PPARG Repaglinide
disease_protein interact with interact with drug targets
BAX
disease_protein interact with interact with drug targets
disease gene/protein drug gene/protein drug
Alzheimer BCL2 Ibuprofen CYP2C8 Repaglinide
disease_protein drug targets drug targets drug targets
disease gene/protein drug
Alzheimer PPARG Repaglinide
disease_protein drug targets
Alzheimer nucleus PPARG Repaglinide
Paths
c
Meta Path
Path

of GNN Explanations
30

of GNN Explanations
31

of GNN Explanations
32

of GNN Explanations
more similar
less similar
33

ALS Ritonavir
NR1|2
Local Explanation: individual semantic paths in the
knowledge graph that reflects biomedical mechanisms
disease drug
gene/protein
Group Explanation: a meta-path that indicate a
sequence of node/relation types
disease A drug P
gene n
disease A drug O
gene j
disease A drug O
gene m
disease B drug P
gene n
disease C drug Q
gene j
Granularity of Explanations
34

A predicted drug
A Meta Path
Organize and compare path-based explanations at different levels of granularity
Meta Matrix
35
35

User Study
42
12 medical professionals who have worked related fields for more than five years,
five clinical researchers,
five practicing physicians,
two medical school students who used to work as pharmacists.
7 males, 5 females, the mean (SD) age was 34.25 (6.12) years
“important problem”
“Super helpful”
“Exactly why I would prescribe an off-label medication for chronic pain”

0.667
0.542
0.542
0.792
0.0 0.2 0.4 0.6 0.8 1.0
path subgraph node baseline
Accuracy
58.308
92.150
92.688
18.358
0 20 40 60 80 100 120
Time(second)
3.542
3.167
2.688
2.375
1.0 2.0 3.0 4.0 5.0
Confidence
F(3,33)=3.39
p<.05
F(3,33)=6.58
p<.05
F(3,33)=24.73
p<.05
more
accurate
less
accurate
quicker slower
more
confident
less
confident
b
a
c d Significant difference
Users are able to perform
tasks more accurately,
confidently, and quickly.
Format of explanations
User Study
43

0.667
0.542
0.542
0.792
0.0 0.2 0.4 0.6 0.8 1.0
path subgraph node baseline
Accuracy
58.308
92.150
92.688
18.358
0 20 40 60 80 100 120
Time(second)
3.542
3.167
2.688
2.375
1.0 2.0 3.0 4.0 5.0
Confidence
F(3,33)=3.39
p<.05
F(3,33)=6.58
p<.05
F(3,33)=24.73
p<.05
more
accurate
less
accurate
quicker slower
more
confident
less
confident
b
a
c d Significant difference
User Study
A poorly-designed visual
explanation is not necessarily
better than a non-explanation
baseline
44
Format of explanations

45
Best Paper Award
IMLH@ICML 2021
Best Paper Honorable Mention
IEEE VIS Conference 2022
c Path Explanation
E
s
c
i
t
a
l
o
p
r
a
m
D
e
s
v
e
n
l
a
f
a
x
i
n
e
F
l
u
o
x
e
t
i
n
e
M
i
r
t
a
z
a
p
i
n
e
C
l
o
z
a
p
i
n
e
C
l
o
m
i
p
r
a
m
i
n
e
I
s
o
c
a
r
d
b
o
x
a
z
i
d
11 2 5 13 2
13 disease
gene/protein
molecular_function
drug
1 1 1 2 1
2
disease
gene/protein
drug
unipolar depres...
HTR7
Clozapine associated
targets
〃
HTR2C
〃
associated
targets
〃
〃
Clomipramine associated
targets
1
disease
gene/protein
pathway
drug
20 17 20 20 15 14 11 disease
gene/protein
anatomy
drug
Users can compare the
explanations of different
selected drugs
Users can hide ( ), unhide ( ), collapse ( ), or expand ( )
a group of explanation paths based on the meta-path
Drug Embedding
b
gene/protein
gene/protein
gene/protein
C3
C4
C2
Ditto mark (〃) indicates this
node is the same as the node
in the above path
a Control Panel
Select drugs
through lasso or click
M
o
c
l
o
b
e
m
i
d
e
A
g
o
m
e
l
a
t
i
n
e
33
11
28
2 3
10
4
3
1
2 1
T
r
i
m
i
p
r
a
m
i
n
e
N
e
f
a
z
o
d
o
n
e
T
r
a
z
o
d
o
n
e
22 5 5
27 11 7
11 10 8
2 1 1
1
N
o
r
t
r
i
p
t
y
l
i
n
e
E
s
c
i
t
a
l
o
p
r
a
m
29 23
23 27
12 13
1 1
1 2
C
l
o
m
i
p
r
a
m
i
n
e
5
8
3
1
1
C1 MetaMatrix provides an overview
of all predicted drugs in terms
of meta paths
C5
Ranked by scores or grouped
based on embeddings
DrugExplorer
How can user feedback steer AI?
>1,400 users from 64 countries
in the first month
http://txgnn.org

Steer AI
46
Can AI Explanations enable users to
Interpret and Steer AI at the same time?
Human Knowledge
about Classes
Human Knowledge
about Concepts
• Polyphony (VIS 22, Biovis ISMB 22)

47
Human Knowledge
about Classes

Steer
!"#$%&'()"&
$*()+%$(&$
#,%'(
#*%-(!
.(%$/-(#
Interpret Anchor
0''(,$
1(2('$
0!!
1(3&(
$-%"&"&4
!%$%
$(#$"&4)
!%$%
! " #
$
$
Model
Fine-tuning
Update
PolyPhony:
An Interactive Transfer Learning Framework
for Single-Cell Data Analysis
Mark Keller
Furui Cheng
Nils Gehlenborg
Huamin Qu
Qianwen Wang
Best Long Abstract Award
BioVis COSI, Conference on Intelligent System for Molecular Biology (ISMB)
IEEE VIS 2022
IEEE Transactions on Visualization and Computer Graphics
48

Training Dataset
AI Algorithm:
Classification
49

AI Tool:
Single Cell Annotation
50
Tissue
Cell-Type
Mapping

AI Algorithm:
Classification
AI Tool:
Single Cell Annotation
Cannot be directly applied
51
Cell Types
Labelled data
Unlabelled new data
Labelled data
Unlabelled new data
AI may not tell Technical Variations
(i.e., batch effect) from Biological
Variations (i.e., different cell types)

Human inputs are needed!
Cell Types
52
AI Tool for Single Cell Annotation
Automatic
Annotation
Manual
Validation
Workflow

AI Tool for Single Cell Annotation
How about asking users to manually label
some items?
Power to the People: The Role of Humans in Interactive Machine Learning
Saleema Amershi et al. 2014, AI Magazine
Previous studies show that
• Users do not want to be treated as an
oracle that simply label individual items
• Transparency about the AI system will help
users provide accurate feedback
53
Cell Types

Anchor
analogous cell populations across datasets
• Interpret AI in a way that is consistent with user
workflow and mental model
• Steer AI by integrating human knowledge
Interactive Anchors
Enable Simultaneous AI Interpretation and Steering
54
Cell Types

Anchor
analogous cell populations across datasets
• Interpret AI in a way that is consistent with user
workflow and mental model
• Steer AI by integrating human knowledge
Interactive Anchors
Enable Simultaneous AI Interpretation and Steering
55
Cell Types

Steer
!"#$%&'()"&
$*()+%$(&$
#,%'(
#*%-(!
.(%$/-(#
Interpret Anchor
0''(,$
1(2('$
0!!
1(3&(
$-%"&"&4
!%$%
$(#$"&4)
!%$%
! " #
$
$
Model
Fine-tuning
Update
Generate Anchor
0
0
Polyphony
56

Generate Anchor Recommendations
Harmony (Korsunsky et al., Nature Methods, 2019)
One Anchor
Clusters
57
Polyphony
Similarity Matrix

Polyphony
Interpret Anchors
A
B C
Steer
!"#$%&'()"&
$*()+%$(&$
#,%'(
#*%-(!
.(%$/-(#
Interpret Anchor
0''(,$
1(2('$
0!!
1(3&(
$-%"&"&4
!%$%
$(#$"&4)
!%$%
! " #
$
$
Model
Fine-tuning
Update
Reference
Query
58

Polyphony
Steer Anchors: Edit
59

Use Cases
Before Refinement
The reference dataset
• a plate-based protocol
• contains 7,290 cells from 32 donors
• annotated with eleven cell types
The query dataset:
• generated using a droplet-based protocol
• contains 8,391 cells from 4 donors
• Has the same cell types as the reference
Pancreas Dataset
60
After Refinement
Six postdoc researchers and one assistant professor in single-cell analysis.
“intuitive and easy to use”
“more than just giving me an answer”
“I can fix undesired outcomes”

62
Best Long Abstract Award
BioVis COSI, Conference on Intelligent System for Molecular Biology (ISMB)
Computational biology
Data Visualization
PolyPhony:
An Interactive Transfer Learning Framework
for Single-Cell Data Analysis

Knowledge can be more complicated than Classes
63
In Polyphony,
items form Clear Clusters based on their
overall similarity after Dimension Reduction

In Polyphony,
items form Clear Clusters based on their
overall similarity after Dimension Reduction
What if there is no clear clusters?
What if the users are interested in certain
aspect rather than the overall similarity?
64

Steer AI
65
Human Knowledge
about Classes
Human Knowledge
about Concepts

DRAVA
for theVisual Exploration of Small Multiples
Aligning Human Concepts with Machine Learning Latent Dimensions
Qianwen Wang Nils Gehlenborg
Sehi L’Yi
BioVis COSI, Conference on Intelligent
System for Molecular Biology (ISMB’22)
ACM CHI Conference on Human Factors in
Computing Systems (CHI’23)
66
Harvard Data Science Initiative
Postdoctoral Fellow Research Fund

Concepts
One Class
(Zebra)
67
Multiple Concepts
(Horse, stripe, grass)

Concepts
68
One Class
(Zebra)
Multiple Concepts

Concepts
69
One Class
(Zebra)
Multiple Concepts
Pink-
Purple
Tissue
Density

Extract Concepts from AI
70
Been Kim et al. TCAV, 2018
Zhenge Zhao et al. Concept Extract, 2021

Extract Concepts from AI
encoder decoder
input: x latent
vector: z
!"
output: x
#"
• Learn concepts without labels
• Show what a concept looks like
Disentangled Representation Learning
!"#$%#&'("
71
UMAP

AI Algorithm:
Disentangled
Representation Learning
AI Tool:
Concept-driven
Visual Exploration
encoder decoder
input: x latent
vector: z
!"
output: x
#"
72

!"#$%&'(&)#*+&,-./% 01%12/*1&31)&
,-./%&4%5)"16$1
latent
dimensions
synthesized
images
data
items
7%*12821*&95%:18*;
modify
items
modify
groups
split merge
investigated dim
assistant
dim
assign
distribution
locate items
of interest
reveal
association
a b c
assign
DRAVA
A Three-Step Workflow
73
TADs in Genomic Contact Matrices

Each row represents a latent dimension (a potential concept)
Ranked by a saliency score
Interpret Concepts
via synthesised images
74

nested structure
thickness of diagonal
asymmetric structure
Interpret Concepts
75

nested structure
thickness of diagonal
asymmetric structure
Interpret Concepts
76

Data Items
Partial Compression See-Through Item
Label
Group
Label
a c
label 1 label2
b
1D grouping
2D grouping
Representative Average
label1
22
Interpret Concepts
via data items
77
Lekschas et al.
Pining.js. InfoVIS 2020

The model confuses the
diagonal thickness with the
nested structure
Interpret Concepts
via data items
78

User Reﬁnement
Refine Concepts
79
!"#$%&'( )#*$#
a b
a
selected
metric
+,"-%#
c
!"#$%&!$'()*+
,)-+!(&.)/
!0#$%&!$&12-
+(2%&23 !4#$%&!$-21(&,. 5"#$-)%26-2('2 50#$.+7&1 54#$,8!/'2$'()*+ ,"#$7),!7 ,0#$'7)5!7
drag & drop
lasso
Depile All
Extract
Browse Separately
Magnify
Depile All
Extract
Browse Separately
Magnify
drag &
drop
extract

Input
feature
map
Encoder
conv
block
conv
block
flatten
FC
reparametrize
…
conv
block
Concept A
Decoder
Transposed
conv
block
resize
FC
…
Transposed
conv
block
a c
b
z
conv
block
global
maxpool
softmax
Concept Adaptor
Decoder
Transposed
conv
block
resize
FC
…
Transposed
conv
block
c
b
80
Refine Concepts
Model Structure

Bangs
Smiling
Scale
N=5%
N=2%
N=1%
Ours Baseline
81
Refine Concepts
Evaluation

Concept-drivenVisual Exploration
Image patches of
breast cancer specimens
82

Strong correlation with the
presence of cancer cells

Filter items to investigate conﬁdent
false-negative predictions

They are more likely fatty tissues
surrounded by cancer cells
These items are mostly loose purple
tissues

89
!"##$%&'%(#"!)
*&+,#"!",(
)&+,#"!",(
#-"."(/
0%1'&1(/.%
+$2+.%&3&+"(4
!"##$%&'%(#"!)
Interpret and Steer Concept-based Explanations
More accurate concepts
More interpretable and flexible visual exploration
DRAVA Harvard Data Science Initiative
Postdoctoral Fellow Research Fund
A NSF-sponsored workshop on interactive visualization
and analysis of high-dimensional scientific data
https://qianwen.info/DRAVA/#/use-cases

Summary
Usable and Useful AI Applications through interactive visual explanations
that imitate the communication process between humans
90
Enable users to provide feedback and
steer AI for the desired tasks
Present users explanations about AI

Summary
User-Centric
Design Considerations
Data
Format
Human
Knowledge
Solutions
Graph
Biological
Mechanisms
Increased speed, accuracy,
and confidence in validating
predictions
High-
dimensional
Vectors
Classes
Improved user satisfaction
level & annotation accuracy
Concepts
Reveal and fix concept
mismatches that are hidden
in previous methods
Steer
!"#$%&'()"&
$*()+%$(&$
#,%'(
#*%-(!
.(%$/-(#
Interpret Anchor
0''(,$
1(2('$
0!!
1(3&(
$-%"&"&4
!%$%
$(#$"&4)
!%$%
! " #
$
$
Model
Fine-tuning
Update
Partial Compression See-Through
Item
Label
Group
Label
label 1
label2
1D grouping
2D grouping
label1
22
Path Explanation
E
s
c
i
t
a
l
o
p
r
a
m
D
e
s
v
e
n
l
a
f
a
x
i
n
e
F
l
u
o
x
e
t
i
n
e
M
i
r
t
a
z
a
p
i
n
e
C
l
o
z
a
p
i
n
e
C
l
o
m
i
p
r
a
m
i
n
e
I
s
o
c
a
r
d
b
o
x
a
z
i
d
11 2 5 13 2
13 disease
gene/protein
molecular_function
drug
1 1 1 2 1
2
disease
gene/protein
drug
unipolar depres...
HTR7
targets
〃
HTR2C
〃
associated
targets
〃
〃
targets
1
disease
gene/protein
pathway
drug
20 17 20 20 15 14 11 disease
gene/protein
anatomy
drug
selected drugs
gene/protein
gene/protein
gene/protein
2
3
1
in the above path
91

Summary
User-Centric
Design Considerations
Data
Format
Human
Knowledge
Solutions
Graph
Biological
Mechanisms
predictions
High-
dimensional
Vectors
Classes
Concepts
in previous methods
Steer
!"#$%&'()"&
$*()+%$(&$
#,%'(
#*%-(!
.(%$/-(#
Interpret Anchor
0''(,$
1(2('$
0!!
1(3&(
$-%"&"&4
!%$%
$(#$"&4)
!%$%
! " #
$
$
Model
Fine-tuning
Update
Item
Label
Group
Label
label 1
label2
1D grouping
2D grouping
label1
22
Path Explanation
E
s
c
i
t
a
l
o
p
r
a
m
D
e
s
v
e
n
l
a
f
a
x
i
n
e
F
l
u
o
x
e
t
i
n
e
M
i
r
t
a
z
a
p
i
n
e
C
l
o
z
a
p
i
n
e
C
l
o
m
i
p
r
a
m
i
n
e
I
s
o
c
a
r
d
b
o
x
a
z
i
d
11 2 5 13 2
13 disease
gene/protein
molecular_function
drug
1 1 1 2 1
2
disease
gene/protein
drug
unipolar depres...
HTR7
targets
〃
HTR2C
〃
associated
targets
〃
〃
targets
1
disease
gene/protein
pathway
drug
20 17 20 20 15 14 11 disease
gene/protein
anatomy
drug
selected drugs
gene/protein
gene/protein
gene/protein
2
3
1
in the above path
92

Beyond Publications
Open-Source Real-World Users Media Coverage
93
Users of ML4VIS
Over 3,000 visits
across 40+ countries
Users of Gosling
Over 15,000 NPM
downloads across 120+
countries

Research Agenda
Previous Studies
94
Design Frameworks and Guidelines
Narvis (VIS 18)
ML4VIS (TVCG 2021)
User-Centric XAI🏅(VIS 22)
SineStream (VIS 20)
Visualization Techniques/Tools for
Human-AI Collaboration
HypoML (VIS 20) DiscriLens (VIS 20)
ATMSeer (CHI 19) GNNLens (TVCG 21)
DNN Genealogy (TVCG 19)
ThreadStates (VIS 21) DrugExplorer🏅(VIS 22)
Polyphony🏅(VIS 22, Biovis ISMB 22)
PapARVis (CHI 20)
Biomedical Applications
OncoThreads (Bioinformatics 21)
Polyphony🏅(VIS 22, Biovis ISMB 22)
Gosling🏅(VIS 21, Biovis ISMB 21)
GenoRec (VIS 22) ThreadStates (VIS 21)
DrugExplorer🏅(VIS 22, Biovis ISMB 21)
Future Research Agenda
Towards
Multimodality
https://storyset.com/illustration/data-points/rafiki
8
txt
Safeguards for
Interaction
https://storyset.com/illustration/warning/rafiki
Hypothesis for
New Biological
Knowledge
https://storyset.com/illustration/stem-cell-research/rafiki

Research Agenda
95
Human-AI Teaming:
Safeguards for Interaction
https://storyset.com/illustration/warning/
AI is imperfect, so are humans
• When and Why do users give biased/inconsistent
feedback for AI explanations?
• How to help users provide informed and accurate
feedback?
• How to detect biased/inconsistent user feedback in
Human-AI collaboration?

Research Agenda
96
Human-AI Teaming:
Safeguards for Interaction
https://storyset.com/illustration/warning/
AI is imperfect, so are humans
• When and Why do users give biased/
inconsistent feedback for AI
explanations?
• How to help users provide informed
and accurate feedback?
• How to detect biased/inconsistent
user feedback in Human-AI
collaboration?
My previous study:
Framework of designing interactive
visual explanations
Testbeds
Drava, Polyphony, DrugExplorer,
TheadStates, etc
Item
Label
Group
Label
label 1
label2
1D grouping
2D grouping
label1
22
Path Explanation
E
s
c
i
t
a
l
o
p
r
a
m
D
e
s
v
e
n
l
a
f
a
x
i
n
e
F
l
u
o
x
e
t
i
n
e
M
i
r
t
a
z
a
p
i
n
e
C
l
o
z
a
p
i
n
e
C
l
o
m
i
p
r
a
m
i
n
e
I
s
o
c
a
r
d
b
o
x
a
z
i
d
11 2 5 13 2
13 disease
gene/protein
molecular_function
drug
1 1 1 2 1
2
disease
gene/protein
drug
unipolar depres...
HTR7
targets
〃
HTR2C
〃
associated
targets
〃
〃
targets
1
disease
gene/protein
pathway
drug
20 17 20 20 15 14 11 disease
gene/protein
anatomy
drug
selected drugs
gene/protein
gene/protein
gene/protein
2
3
1
in the above path

Towards Relatable Explainable AI with the Perceptual Process, Zhang et al. 2022
Research Agenda
97
Human-AI Communication:
Towards Multimodality
8
txt
Multimodality enables
• Comprehensive Analysis: highlight patterns that might
be missed by using a single modality.
• Effective Communication: improve visual representations
through the integration of other modalities (e.g., text).

Research Agenda
98
Human-AI Communication:
Towards Multimodality
8
txt
Multimodality enables
• Comprehensive Analysis: highlight patterns that might
be missed by using a single modality.
• Effective Communication: improve visual representations
through the integration of other modalities (e.g., text).
Pathology Radiology
Genomic
Data
Electronic
Medical Record
Diagnosis and
Treatment

Research Agenda
99
AI for Science:
Hypothesis Generation
and Validation
https://storyset.com/illustration/stem-cell-research/rafiki
Learning new knowledge from AI
• How to evaluate AI explanations when there is
no ground truth?
• How to systematically generate hypotheses
from AI explanations?
Consistent with the off-label prescription decisions made
by clinicians in a large healthcare system (1,272,085
patients, 480 diseases, and 1,290 drugs)

Acknowledgement
Icons & Images

Thanks!
Data
Format
Human
Knowledge
Solutions
Graph
Biological
Mechanisms
AI predictions
DrugExplorer🏅
(VIS 22, IMLH@ICML 21)
High-
dimensional
Vectors
Classes
Polyphony🏅
(VIS 22, Biovis ISMB 22)
Concepts
in previous methods
Drava
(ACM CHI 23)
Steer
!"#$%&'()"&
$*()+%$(&$
#,%'(
#*%-(!
.(%$/-(#
Interpret Anchor
0''(,$
1(2('$
0!!
1(3&(
$-%"&"&4
!%$%
$(#$"&4)
!%$%
! " #
$
$
Model
Fine-tuning
Update
Item
Label
Group
Label
label 1
label2
1D grouping
2D grouping
label1
22
Path Explanation
E
s
c
i
t
a
l
o
p
r
a
m
D
e
s
v
e
n
l
a
f
a
x
i
n
e
F
l
u
o
x
e
t
i
n
e
M
i
r
t
a
z
a
p
i
n
e
C
l
o
z
a
p
i
n
e
C
l
o
m
i
p
r
a
m
i
n
e
I
s
o
c
a
r
d
b
o
x
a
z
i
d
11 2 5 13 2
13 disease
gene/protein
molecular_function
drug
1 1 1 2 1
2
disease
gene/protein
drug
unipolar depres...
HTR7
targets
〃
HTR2C
〃
associated
targets
〃
〃
targets
1
disease
gene/protein
pathway
drug
20 17 20 20 15 14 11 disease
gene/protein
anatomy
drug
selected drugs
gene/protein
gene/protein
gene/protein
2
3
1
in the above path
Present users
explanations
Help users
generate insights
Enable users to
steer AI
Usable and Useful AI Applications through interactive visual
explanations that imitate the communication process between humans
https://qianwen.info
qianwen@umn.edu

Interpreting and Steering AI Explanations Via Interactive Visualizations

Recommended

Recommended

More Related Content

Similar to Interpreting and Steering AI Explanations Via Interactive Visualizations

Similar to Interpreting and Steering AI Explanations Via Interactive Visualizations (20)

Recently uploaded

Recently uploaded (20)

Interpreting and Steering AI Explanations Via Interactive Visualizations