Successfully reported this slideshow.
Your SlideShare is downloading. ×

Bridge the Capabilities of AI with the Needs of Human Users

Bridge the Capabilities of AI with the Needs of Human Users

Download to read offline

Current machine learning (ML) methods are usually developed via a data-centric approach regardless of the usage context and the end users.
While such a one-size-fits-all strategy ensures these algorithms are generic and applicable to a variety of domain problems, it also poses usability challenges for domain users in interpreting model results, obtaining actionable insights, and collaborating with AI in decision-making and knowledge discovery.

To facilitate the application of ML and improve its usability, interactive visualization should be considered as an indispensable component. Apart from its efficiency in organizing and communicating information, interactive visualization can be disentangled from the algorithmic aspects to incorporate the perspective of users and the characteristics of the applied domains.
In this talk, we demonstrate the important role of visual interfaces in the successful application of AI via real-world case studies. We summarize design guidelines for bridging the capabilities of AI with the needs of domain users.

Current machine learning (ML) methods are usually developed via a data-centric approach regardless of the usage context and the end users.
While such a one-size-fits-all strategy ensures these algorithms are generic and applicable to a variety of domain problems, it also poses usability challenges for domain users in interpreting model results, obtaining actionable insights, and collaborating with AI in decision-making and knowledge discovery.

To facilitate the application of ML and improve its usability, interactive visualization should be considered as an indispensable component. Apart from its efficiency in organizing and communicating information, interactive visualization can be disentangled from the algorithmic aspects to incorporate the perspective of users and the characteristics of the applied domains.
In this talk, we demonstrate the important role of visual interfaces in the successful application of AI via real-world case studies. We summarize design guidelines for bridging the capabilities of AI with the needs of domain users.

More Related Content

Bridge the Capabilities of AI with the Needs of Human Users

  1. 1. Qianwen WANG InteractiveVisualizationforUsableAI Bridgethe Capabilities of AI with theNeedsofHumanUsers Users AI 1
  2. 2. Langer et al. 2021. What Do We Want From Explainable Artificial Intelligence AIandHuman AI System 2
  3. 3. Langer et al. 2021. What Do We Want From Explainable Artificial Intelligence AIandHuman AI System 3
  4. 4. AIandHuman AI System Usable whether domain users can use AI to complete desired tasks easily and efficiently Langer et al. 2021. What Do We Want From Explainable Artificial Intelligence 4
  5. 5. AIandHuman Usable What makes Usable AI AI More accurate, stable, and faithful algorithms Users Interface Jointly consider the capabilities of AI, the needs of users, and the characteristics of the usage context 5
  6. 6. AIandHuman Usable What makes Usable AI 6 AI
  7. 7. AIandHuman Usable It is promising Even if we were to make no further progress in the next decade, deploying existing ML algorithms to every applicable problem would be a game changer for most industries. — Francois Chollet 7
  8. 8. AIandHuman Usable It is promising, but difficult 8 Epic’s AI algorithms are delivering inaccurate information on seriously ill patients MIKE REDDY FOR STAT https://www.statnews.com/2021/07/26/epic-hospital-algorithms-sepsis-investigation/? utm_source=researcher_app&utm_medium=referral&utm_campaign=RESR_MRKT_Researcher_inbound https://www.fiercehealthcare.com/practices/nearly-half-u-s-doctors-say-they-are-anxious-about- using-ai-powered-software-survey
  9. 9. AIandHuman Usable Why Usable AI is hard Users AI Abstract benchmark tasks Complicated domain-specific tasks Treatment suggestion age disease history symptoms ….. Low level of domain expertise Which domain-related information should be provided by the AI model? Low level of AI expertise How to ask about domain-related information from the AI model? 9 Algorithm-centric User-centric
  10. 10. 10 U d U d Stages of the Listening Process Receive Understand and Remember Evaluate and Feedback
  11. 11. Relevant information about the AI are revealed to the users for the desired tasks 11 Receive
  12. 12. Users and AI can achieve a consensus about the desired tasks 12 Relevant information about the AI are revealed to the users for the desired tasks Understand and Remember
  13. 13. Users can provide feedback to AI about the desired tasks 13 Users and AI can achieve a consensus about the desired tasks Relevant information about the AI are revealed to the users for the desired tasks Evaluate and Feedback
  14. 14. achieve a consensus about the desired tasks refine AI for the desired tasks UnderstandAIintermsofperformance Visual Genealogy of Deep Neural Networks Qianwen Wang1, Jun Yuan2, Shuxin Chen2, Hang Su2, Huamin Qu1, and Shixia Liu2 Tshinghua University ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning Qianwen Wang, Yao Ming, Zhihua Jin, Qiaomu Shen, Dongyu Liu, Micah J. Smith, Kalyan Veeramachaneni, Huamin Qu 14 Relevant information about the AI are revealed to the users for the desired tasks IEEE transactions on visualization and computer graphics 26 (11), 3340-3352 Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
  15. 15. achieve a consensus about the desired tasks refine AI for the desired tasks UnderstandAIintermsoffairness To help users better understand AI for the desired tasks 15 Tshinghua University 1 2 Qianwen Wang1 Zhenhua Xu1 Huamin Qu1 Shixia Liu2 Zhutian Chen1 Yong Wang1 IEEE InfoVis 2020
  16. 16. VisualAnalysisofDiscrimination inMachineLearning Tshinghua University 1. 2. Qianwen Wang1 Zhenhua Xu1 Huamin Qu1 Shixia Liu2 Zhutian Chen1 Yong Wang1 16
  17. 17. What is a fair prediction? 17
  18. 18. A College Admission Example 50%>42% Seems unfair? accepted females accepted males rejected 18
  19. 19. A College Admission Example accepted females accepted males rejected Low score High score 33.3%>26.7% 75%>65% 19
  20. 20. A College Admission Example 20%=20% 40%=40% 60%=60% 80%=80% Low score High score CS EE CS EE accepted females accepted males rejected 20
  21. 21. Two individuals who are similar with respect to a task are treated equally 21
  22. 22. feder-gov <8 >65 Yes 0.8 Yes 1.0 work_class education hours/week income<50 k Conf M F Discriminatory Itemset 22
  23. 23. Discriminatory Itemset 23
  24. 24. Challenges in Analysis work_class loc-gov Education:8-12 hours/week >65 relation: own-child house: rent capital-gain: <2000 marital: divorced work_class private Education:<12 hours/week:25-35 relation: not-in-familiy house: own work_class private Education:<12 hours/week:25-35 capital-gain: 2000-3000 house: own Long and Complex Definition work_class private Education:<12 hours/week:25-35 capital-gain: 2000-3000 marital: divorced Intertwining Relationship 24
  25. 25. Long and Complex Definition 25
  26. 26. Long and Complex Definition 23< raised hands < 50 Attribute Matrix Itemset Attribute 26
  27. 27. Intertwining Relationships work_class private Education:<12 hours/week:25-35 relation: not-in-familiy house: own work_class private Education:<12 hours/week:25-35 capital-gain: 2000-3000 house: own work_class private Education:<12 hours/week:25-35 capital-gain: 2000-3000 marital: divorced RippleSet 27
  28. 28. Designing RippleSet An item Items set A ∈   An item Items set A ∈   28
  29. 29. An item Items set A ∈   (C D)(AUBUE) ∩ (A B C D)E ∩ ∩ ∩ (A B C)(DUE) ∩ ∩ (A B E)(CUD) ∩ ∩ (B C E)(AUD) ∩ ∩ Designing RippleSet 29
  30. 30. An item Items set A ∈   (C D)(AUBUE) ∩ (A B C D)E ∩ ∩ ∩ (A B C)(DUE) ∩ ∩ (A B E)(CUD) ∩ ∩ (B C E)(AUD) ∩ ∩ ABC ABE BCE ABCD CD Designing RippleSet 30
  31. 31. An item Items set A ∈   (C D)(AUBUE) ∩ (A B C D)E ∩ ∩ ∩ (A B C)(DUE) ∩ ∩ (A B E)(CUD) ∩ ∩ (B C E)(AUD) ∩ ∩ ABC ABE BCE ABCD CD Items belonging to the same set are put together D D Weighted DAG Circle packing algorithm Designing RippleSet 31
  32. 32. 32
  33. 33. Model Information Design visualizations 33
  34. 34. AI Explanations XAI Algorithm Design visualizations for a specific XAI 34
  35. 35. AI Explanation XAI Algorithm Design visualizations for a specific XAI Rule-based Explanations Ming et al. "Rulematrix: Visualizing and understanding classifiers with rules." IEEE transactions on visualization and computer graphics 25.1 (2018): 342-352. Counterfactual Explanations Chen et al. "DECE: decision explorer with counterfactual explanations for machine learning models." IEEE transactions on visualization and computer graphics 27.2 (2021): 1438-1447. Attribution-based Explanations Hohman et al. "Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations." IEEE VAST 2019 Generaltask:ShowmeAIexplanations 35
  36. 36. Forcertaintasks,areaIlexplanations equallyusableforhumanusers? 36
  37. 37. AIexplanationsarenotalwaysusable Debugging Tests for Model Explanations, NeurIPs 2020, Julius Adebayo, Michael Muelly, Ilaria Liccardi, Been Kim Are Explanations Helpful: A Comparative Study of the Effects of Explanations in AI- Assisted Decision-Making IUI 2021, Xinru Wang, Ming Yin 37 Human subjects fail to identify defective models using attribution-based explanations, but instead rely, primarily, on model predictions. The explanation that is considered to resemble how human explain decisions (i.e., counterfactual explanation) does not improve calibrated trust.
  38. 38. UsableAI,itdepends Not always work for images, but can reveal very important insights for regulatory genomic What works or does not is extremely domain-specific! 38 Anshul Kundaje, Stanford University Deep learning approaches to decode the human genome
  39. 39. refine AI for the desired tasks 39 Users and AI can achieve a consensus about the desired tasks Relevant information about the AI are revealed to the users for the desired tasks
  40. 40. ExtendingtheNestedModelforUser-CentricXAI: ADesignStudyonGNN-basedDrugRepurposing Qianwen Wang Kexin Huang Payal Chandak Nils Gehlenborg Marinka Zitnik HARVARD-MIT HEALTH SCIENCES AND TECHNOLOGY 40 IEEE VIS 2022
  41. 41. Generaltask: ExplainaGNNmodel 41 Generaltask: ExplainaGNNmodel usedfor drugrepurposing Somemethodscanfail
  42. 42. GNNforDrugRepurposing Nodes: drugs, diseases, proteins, etc Edges: known relations among these nodes GNN Human Experts ? 42
  43. 43. Model for Visualization Design and Validation 43 ExtendingtheNestedModelforUser-CentricXAI
  44. 44. Explanation Ontology:  A Model of Explanations for User-Centered AI Designing Theory-Driven User-Centric Explainable AI 
  45. 45. !"#$%& '()*$&$+%"& ,-.+/$0+%"& 1%.2$*%3$+%"& !"#$%&'()%#* !"#$!%&&$'()$*") +,$)-.&/0/*%#0 ,*4"/%+5# +,%#"&-./ !"1 !"1$0#* !"/*$%2 "#!$*#$ !"/*$)&() 012'13$.#-&45 6%/7%#-/$' 13$.#-&45 ()"$%'8./&%9& !")0$/03$!")4)$ !%&&$)-.&/0/*%#0($ 5)$'()3 012':."3 !"1$)-.&/0/*%#0( /4)$0))3)3 ;.5"-/' <9,3"/"&-./ "#!$3#$3#6/%0 '()4($)-.&/%0$ *#$*")%4$.))4( :#"/=3"#-&> &#7/&8$94#'.8$ #4$9&#5/& ?.#5"& )-/6.&)8$ /**4%5'*%#08$ 7&/'() 0#3)8 ./*"8 ('594/." 9)0)4/*)$+,$ )-.&/0/*%#0( :%('/&%;)$/03$ '.3/*)$+,$ )-.&/0/*%#0( ./*"<5/()3 )-.&/0/*%#0 !"/*$)&() !"#$%&'#()" *"')+(", !"1 1 2 2 drug gene/protein cellular_com.. gene/protein disease 2 drug drug disease disease disease 1 drug disease drug disease disease Agalsidase beta chylomicron ret... Alipogene tipar... lysosomal acid l... Wolman disease indication indication indication includes 1 drug disease gene/protein disease disease 1 drug disease gene/protein disease Agalsidase beta Avelumab Idursulfase Galsulfase &#7/& )-.&/0/*%#0 94#'. )-.&/0/*%#0 45
  46. 46. Common Visual Presentations of GNN Explanations 46
  47. 47. Common Visual Presentations of GNN Explanations 47
  48. 48. Common Visual Presentations of GNN Explanations more similar less similar 48
  49. 49. c Path Explanation E s c i t a l o p r a m D e s v e n l a f a x i n e F l u o x e t i n e M i r t a z a p i n e C l o z a p i n e C l o m i p r a m i n e I s o c a r d b o x a z i d 11 2 5 13 2 13 disease gene/protein molecular_function drug 1 1 1 2 1 2 disease gene/protein drug unipolar depres... HTR7 Clozapine associated targets 〃 HTR2C 〃 associated targets 〃 〃 Clomipramine associated targets 1 disease gene/protein pathway drug 20 17 20 20 15 14 11 disease gene/protein anatomy drug Users can compare the explanations of different selected drugs Users can hide ( ), unhide ( ), collapse ( ), or expand ( ) a group of explanation paths based on the meta-path Drug Embedding b gene/protein gene/protein gene/protein C3 C4 C2 Ditto mark (〃) indicates this node is the same as the node in the above path a Control Panel Select drugs through lasso or click M o c l o b e m i d e A g o m e l a t i n e 33 11 28 2 3 10 4 3 1 2 1 T r i m i p r a m i n e N e f a z o d o n e T r a z o d o n e 22 5 5 27 11 7 11 10 8 2 1 1 1 N o r t r i p t y l i n e E s c i t a l o p r a m 29 23 23 27 12 13 1 1 1 2 C l o m i p r a m i n e 5 8 3 1 1 C1 MetaMatrix provides an overview of all predicted drugs in terms of meta paths C5 Ranked by scores or grouped based on embeddings 49
  50. 50. 50
  51. 51. 51
  52. 52. 52
  53. 53. antidiabetic drugs 53
  54. 54. expand/collapse hide/unhide 54
  55. 55. 55
  56. 56. The visual representation of explanations matters! User study with 12 domain experts (physicians, senior medical students, medical researchers) 56
  57. 57. 0.667 0.542 0.542 0.792 0.0 0.2 0.4 0.6 0.8 1.0 path subgraph node baseline Accuracy 58.308 92.150 92.688 18.358 0 20 40 60 80 100 120 Time(second) 3.542 3.167 2.688 2.375 1.0 2.0 3.0 4.0 5.0 Confidence F(3,33)=3.39 p<.05 F(3,33)=6.58 p<.05 F(3,33)=24.73 p<.05 more accurate less accurate quicker slower more confident less confident b a c d Significant difference The visual representation of explanations matters! An inappropriate explanation is not necessarily better than no explanation 57
  58. 58. Observations • Explanations help assess predication qualities and reveal model flaws While users commented most explanations “make sense”, they also pointed out that some explanations are more like “correlation”than “causation”. We should provide testable rather than“always-look-correct”explanations • Users desire to fix the model flaws revealed by AI • Users want to provide more than just data labels Instead of correcting one specific prediction, users desire to demonstrate high-level drug action mechanism to the AI model 58 Receive Understand and Remember Evaluate and Feedback
  59. 59. 59 Users can provide feedback to AI about the desired tasks Users and AI can achieve a consensus about the desired tasks Relevant information about the AI are revealed to the users for the desired tasks
  60. 60. Isthereamechanismthatenables bothuser-centricexplanationand effectivefeedbackatthesametime? 60
  61. 61. Furui Cheng, Mark S Keller, Huamin Qu, Nils Gehlenborg, Qianwen Wang Polyphony An Interactive Transfer Learning Framework for Single-Cell Data Analysis 61 IEEE VIS 2022
  62. 62. GeneralTask:Classification 62 Training Dataset AI
  63. 63. GeneralTask: Classification 63 DomainTask: CellTypeAnnotation Cell Type Annotation Cannot be directly applied
  64. 64. Task:CellTypeAnnotation Assume that are similar to each other Can be very different 64 Training Test Labelled data Unlabelled new data
  65. 65. Task:CellTypeAnnotation 65 The prediction can be inaccurate! We need to • Provide Explanations so that users can know when predictions are inaccurate • Enable Feedback so that users can refine the wrong predictions
  66. 66. Anchor analogous cell populations across datasets • An AI explanation that is consistent with user workflow and mental model • An feedback mechanism that can be used to refine AI performance 66 InteractiveAnchors: ProvideExplanation&EnableFeedback
  67. 67. InteractiveVisualizationofAnchors Three Aspects Reference + Query center of reference cell set center of query cell set gene expression distance A B C 67
  68. 68. Polyphony Framework • Anchor recommendation • Reference building with Harmony 1 Harmony (Korsunsky et al., Nature Methods, 2019) 68
  69. 69. • Anchor recommendation • Reference building with Harmony • Query cell assignment 1 anchor Polyphony Framework 69
  70. 70. • Anchor recommendation • User feedback 2 1 Polyphony Interface Polyphony Framework 70
  71. 71. • Anchor recommendation • User feedback • Model fine-tuning 3 2 1 Polyphony Framework 71
  72. 72. Polyphony Framework • Anchor recommendation • User feedback • Model fine-tuning • Embedding updating 4 3 2 1 72
  73. 73. Use Cases After Refinement Before Refinement The reference dataset • a plate-based protocol • contains 7,290 cells from 32 donors • annotated with eleven cell types The query dataset: • generated using a droplet-based protocol • contains 8,391 cells from 4 donors • Has the same cell types as the reference 73
  74. 74. 74
  75. 75. 75
  76. 76. Evaluation Use case 76
  77. 77. Evaluation Use case After checking the marker genes, the user confirms that these cells belong to the pDC cell type. 77
  78. 78. Evaluation Simulation Study • Results: model performance after running four iterations (50*4 epochs) 78
  79. 79. The visualization should be carefully designed by considering not only the AI aspect but also the needs and mental models of the users An interactive visualization is need to provide relevant information to the users for achieving the domain tasks The visualization should provide both user-centric explanations and effective feedback mechanism Summary 79 1 2 3
  80. 80. Thanks! BridgetheCapabilitiesof AI withthe Needsof Human Users https://qianwen.info/ qianwen_wang@hms.harvard.edu Users AI InteractiveVisualizationforUsableAI 80

×