Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

機械学習モデルの判断根拠の説明

18,213 views

Published on

第20回ステアラボ人工知能セミナー
https://stair.connpass.com/event/109983/

【講演動画】
https://youtu.be/Fgza_C6KphU

【講演タイトル】
機械学習モデルの判断根拠の説明

【講演概要】
本講演では、機械学習モデルの判断根拠を提示するための説明法について紹介する。高精度な認識・識別が可能な機械学習モデルは一般に非常に複雑な構造をしており、どのような基準で判断が下されているかを人間が窺い知ることは困難である。このようなモデルのブラックボックス性を解消するために、近年様々なモデルの説明法が研究・提案されてきている。本講演の前半ではまず近年の代表的な研究について紹介する。後半では、発表者の最近の研究として「ランダムフォレストの簡略化」と「モデル列挙」について紹介する。

Published in: Technology

機械学習モデルの判断根拠の説明

  1. 1. 1 20 2018/12/21
  2. 2. n • ~2013.3, PhD@ , • 2013.4~2016.3, @IBM • 2016.4~2017.8, @ ERATO, NII • 2017.9~, @ , n • ECML’11 AISTATS’15,17 • AISTATS’18 AAAI’17,18 ongoing 2
  3. 3. • • • • AISTATS’18 • AAAI’17,18 3
  4. 4. AI n • n • 4 …
  5. 5. AI n • n • 5 XX XX
  6. 6. AI n • AI AI AI AI n • AI • n 6
  7. 7. n • • • AI • • 7
  8. 8. 8
  9. 9. EU GDPR n GDPR-22 1. The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her. 2. Paragraph 1 shall not apply if the decision: is necessary for entering into, or performance of, a contract between the data subject and a data controller; is authorised by Union or Member State law to which the controller is subject and which also lays down suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests; or is based on the data subject’s explicit consent. 3. In the cases referred to in points (a) and (c) of paragraph 2, the data controller shall implement suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests, at least the right to obtain human intervention on the part of the controller, to express his or her point of view and to contest the decision. 4. Decisions referred to in paragraph 2 shall not be based on special categories of personal data referred to in Article 9(2)1), unless point (a) or (g) of Article 9(2) applies and suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests are in place. 9
  10. 10. n 2016 • ICML, NIPS n • , , Vol.33, No.3, pages 366--369, 2018. • , Qiita 10
  11. 11. n AI 11 Peeking inside the black-box: A survey on Explainable Artificial Intelligence (XAI) https://ieeexplore.ieee.org/document/8466590/ 7 SCOPUS, IEEExplore, ACM Digital Library, Google Scholar, Citeseer Library, ScienceDirect, arXiv “intelligible”, “interpretable”, “transparency”, “black box”, “understandable”, “comprehensible”, “explainable” AI “Artificial Intelligence”, “Intelligent system”, “Machine learning”, “deep learning”, “classifier” , “decision tree”
  12. 12. • • • • AISTATS’18 • AAAI’17,18 12
  13. 13. n • n • 1. • 2. • 3. • 4. • … 13
  14. 14. n n / • n • • n “ ” • 14
  15. 15. 1. • 2. • 3. • 15
  16. 16. n n Why Should I Trust You?: Explaining the Predictions of Any Classifier, KDD'16 [Python LIME; R LIME] n A Unified Approach to Interpreting Model Predictions, NIPS'17 [Python SHAP] n Anchors: High-Precision Model-Agnostic Explanations, AAAI'18 [Python Anchor] n Understanding Black-box Predictions via Influence Functions, ICML’17 [Python influence-release] 16
  17. 17. n n Born Again Trees n Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach, AISTATS'18 [Python defragTrees] 17
  18. 18. n n [Python+Tensorflow saliency; DeepExplain] • Striving for Simplicity: The All Convolutional Net (GuidedBackprop) • On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation (Epsilon-LRP) • Axiomatic Attribution for Deep Networks (IntegratedGrad) • SmoothGrad: Removing Noise by Adding Noise (SmoothGrad) • Learning Important Features Through Propagating Activation Differences (DeepLIFT) 18
  19. 19. 19
  20. 20. n n Why Should I Trust You?: Explaining the Predictions of Any Classifier, KDD'16 [Python LIME; R LIME] n A Unified Approach to Interpreting Model Predictions, NIPS'17 [Python SHAP] n Anchors: High-Precision Model-Agnostic Explanations, AAAI'18 [Python Anchor] n Understanding Black-box Predictions via Influence Functions, ICML’17 [Python influence-release] 20
  21. 21. n n • LIME, SHAP, Anchor n • influence 21
  22. 22. LIME n Why Should I Trust You?: Explaining the Predictions of Any Classifier, KDD'16 [Python LIME; R LIME] • • 22
  23. 23. LIME 23
  24. 24. LIME n 2 • One-Hot • n LIME • Adult One-Hot 24
  25. 25. LIME n ! !′ • LIME !′ !# ∈ {0, 1}* + !, # + n -(!) !0 • - ! ≈ 2 !# ≔ 40 + 46!# for !# ∈ NeighborOf(!0 # ) • 4 4, 25
  26. 26. LIME n !(#) #% • ! # ≈ ' #( ≔ *% + *,#( for #( ∈ NeighborOf(#% ( ) n * • min : ∑ <,<> ∈? @AB C ! C − ' C( E s.t. * % ≤ G * % @AB C #% H #% 26
  27. 27. LIME n n • vs • → LIME 27
  28. 28. SHAP n A Unified Approach to Interpreting Model Predictions, NIPS'17 [Python SHAP] • • LIME 28
  29. 29. SHAP n 1 n 2 29
  30. 30. SHAP n SHAP LIME • Adult One-Hot n . “ ” SHAP “ ” 30
  31. 31. SHAP n SHAP = LIME • ! "# = %& + %("# = %& + ∑* %*"* # 3 n 1. + " = ! "# • 31
  32. 32. SHAP n SHAP = LIME • ! "# = %& + %("# = %& + ∑* %*"* # 3 n 2. "* # = 0 ⇒ %* = 0 • 32
  33. 33. SHAP n SHAP = LIME • ! "# = %& + %("# = %& + ∑* %*"* # 3 n 3. + " = +, "# + +′ +, # "# − +, # "# ∖ 0 ≥ +, "# − +, "# ∖ 0 ⇒ %* +# ≥ %* + • + +′ "* # +′ + +# "* # + 33
  34. 34. SHAP n SHAP = LIME • ! "# = %& + %("# = %& + ∑* %*"* # SHAP n 3 • %* = ∑+⊆-. + ! 01 + 12 ! 0! (4- 5 − 4-(5 ∖ 8)) • Shapley Value 34
  35. 35. SHAP n LIME • min $ ∑&∈( ) * +, * − . * / • ) * = 123 1 456678 & & (1 2|&|) n • Linear SHAP: • Tree SHAP: • Deep SHAP: DeepLIFT 35
  36. 36. Anchor n Anchors: High-Precision Model-Agnostic Explanations, AAAI'18 [Python Anchor] • 36
  37. 37. Anchor 37
  38. 38. Anchor n Anchor LIME • Adult One-Hot n • Anchor + 38
  39. 39. Anchor n “ + • 39
  40. 40. Anchor n ! • " ! ! " = 1 n %(⋅ |!) ! n ! =Anchor • Anchor ! * " + ,- * ! 1. / 0. 1 ≥ +, ! " = 1 Anchor ! " 40
  41. 41. Anchor n !" # $ 1& ' (& ) ≥ + • 1 − - Pr !" # $ 1& ' (& ) ≥ + ≥ 1 − - 41
  42. 42. Anchor n ! = ! • ! • max % &' ( [!(+)] s.t. Pr &' + ! 11 ( 21 3 ≥ 5 ≥ 1 − 7 n 1. n 2. • Pr &' + ! 11 ( 21 3 ≥ 5 1 − 7 ! 42
  43. 43. n n • LIME, SHAP, Anchor n • influence 43
  44. 44. influence n Understanding Black-box Predictions via Influence Functions, ICML’17 [Python influence-release] n ("′, %′) " 44
  45. 45. influence n ("′, %′) " n % = (("; *+), *+ *+ = argmin 2∈4 5 67(8,9)∈: ;(<; +) *+=6> = argmin 2∈4 5 6∈: ?@A 6B6> ;(<; +) n <> = ("′, %′) *+ • C+=6> − C+ *+=6> − *+ <> 45 <> = ("′, %′)
  46. 46. influence n !" = (%′, (′) *+ • ,+-." − ,+ *+-." − *+ !" n • !" = (%′, (′) ,+-." • n influence • ,+-." − ,+ ,+-." − ,+ ≈ − 1 2 345 -6 78(!"; ,+) 46
  47. 47. influence n Data Poisoning • • 47
  48. 48. influence n • 48 This looks like that: deep learning for interpretable image recognition, arxiv: 1806.10574.
  49. 49. n n • LIME • SHAP • Anchor n • influence 49
  50. 50. n • Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking, KDD’17 • 50 30 200 20 200 30 300 ⭕ ⭕ … …
  51. 51. n • Generating Visual Explanations, ECCV‘16 • 51
  52. 52. 52
  53. 53. n n Born Again Trees n Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach, AISTATS'18 [Python defragTrees] 53
  54. 54. BATrees n Born Again Trees • • n • ! ! • 54
  55. 55. n • BATrees • defragTrees RandomForest 55
  56. 56. 56
  57. 57. n n [Python+Tensorflow saliency; DeepExplain] • Striving for Simplicity: The All Convolutional Net (GuidedBackprop) • On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation (Epsilon-LRP) • Axiomatic Attribution for Deep Networks (IntegratedGrad) • SmoothGrad: Removing Noise by Adding Noise (SmoothGrad) • Learning Important Features Through Propagating Activation Differences (DeepLIFT) 57
  58. 58. n • 58 DNN
  59. 59. n • 59 DNN
  60. 60. n • 60 DNN
  61. 61. n DNN 61
  62. 62. n • → • → 62
  63. 63. n ! = # $ n $ n [Simonyan et al., arXiv’14] $% &' ( &() • → → *+ , *,- → • → → &' ( &() → 63
  64. 64. n [Simonyan et al., arXiv’14] !" #$ % #%& n • GuidedBP [Springenberg et al., arXiv’14] back propagation • LRP [Bach et al., PloS ONE’15] • IntegratedGrad [Sundararajan et al., arXiv’17] • SmoothGrad [Smilkov et al., arXiv’17] • DeepLIFT [Shrikumar et al., ICML’17] 64
  65. 65. n n [Python+Tensorflow saliency; DeepExplain] • Striving for Simplicity: The All Convolutional Net (GuidedBackprop) • On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation (Epsilon-LRP) • Axiomatic Attribution for Deep Networks (IntegratedGrad) • SmoothGrad: Removing Noise by Adding Noise (SmoothGrad) • Learning Important Features Through Propagating Activation Differences (DeepLIFT) 65
  66. 66. • • • • AISTATS’18 • AAAI’17,18 66
  67. 67. n AISTATS’18 • n AAAI’17,18 • 67
  68. 68. defragTrees n Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach, AISTATS'18 [Python defragTrees] • • n 69 when Relationship ≠ Not-in-family, Wife Capital Gain < 7370 when Relationship ≠ Not-in-family Capital Gain >= 7370 when Relationship ≠ Not-in-family, Unmarried Capital Gain < 5095 Capital Loss < 2114 when Relationship = Not-in-family Country ≠ China, Peru Capital Gain < 5095 when Relationship ≠ Not-in-family Country ≠ China Capital Gain < 5095 when Relationship ≠ Not-in-family Capital Gain >= 7370 … …
  69. 69. n n • 70 y = XOR(x1 < 0.5, x2 < 0.5) + ✏
  70. 70. n n 71 2017 The State of Data Science & Machine LearningYour Year on Kaggle: Most Memorable Community Stats from 2016
  71. 71. n n 72 R
  72. 72. n • • n 73
  73. 73. defragTrees n 1. • !" #, % &) n 2. !"( #, % )) • !"( #, % )) ≈ !"(#, %|&) ) ≪ & !"( #, % )) n • ) • Factorized Asymptotic Bayesian (FAB) Inference 74 & )
  74. 74. n n • • • 75 D Synthetic 2 1000 1000 Spambase 57 1000 1000 MiniBooNE 50 5000 5000 Magic 11 5000 5000 Higgs 28 5000 5000 Energy 8 384 384
  75. 75. n 76
  76. 76. n 77
  77. 77. n • n • • 78
  78. 78. n AISTATS’18 • n AAAI’17,18 • 79
  79. 79. n Enumerate Lasso Solu/ons for Feature Selec/on, AAAI’17 [Python LassoVariants]. n Approximate and Exact Enumera/on of Rule Models, AAAI'18.
  80. 80. → NO! n • 82
  81. 81. n • → • n • 83
  82. 82. n • n • → 84
  83. 83. n • n • → 85
  84. 84. n • n • → 86
  85. 85. n • n • 87
  86. 86. • •
  87. 87. Lasso Given: !", $" ∈ ℝ'×ℝ ) = 1, 2, … , . Find: / ∈ ℝ' s.t. !" 0 / ≈ $ () = 1, 2, … , .) / n • • Lasso ℓ5 /∗ = argmin = 1 2 >/ − $ @ + B / 5 • Lasso /∗ supp(/∗) = {) ∶ /" ∗ ≠ 0}
  88. 88. Lasso n • → Lasso n Lasso • → n
  89. 89. Lasso n ! ⊆ {$%, $', … , $)} Lasso Lasso ! = min 3 % ' 45 − 7 ' + 9 5 % s.t. supp 5 ⊆ ! Lasso ! Lasso ! < supp 5 = ! • 9 • 9 • $%, $', $= , $%, $', $> , $%, $>, $? , $%, $' , … Lasso !
  90. 90. Lawler !-best 1. " # 2. $ ∈ # " $ "& = " ∖ {$} Lasso("′) #′ (#&, "′) 3. 4.
  91. 91. Lawler !-best 1. " # 2. $ ∈ & ' $ '( = ' ∖ {$} Lasso('′) &′ (&(, '′) 3. 4. & = 56, 57, 58 ' = 56, 57, 59, 58, 5: &6 = 56, 57, 58 '6 = 56, 57, 59, 58, 5:
  92. 92. Lawler !-best 1. " # 2. $ ∈ & ' $ '( = ' ∖ {$} Lasso("′) #′ (#(, "′) 3. 4. "5 ( = 67, 68, 69, 6: "7 ( = 65, 68, 69, 6: "8 ( = 65, 67, 68, 6: "5 = 65, 67, 68, 69, 6: #5 = 65, 67, 69 "5 = 65, 67, 68, 69, 6:
  93. 93. Lawler !-best 1. " # 2. $ ∈ & " ' "( = " ∖ {'} -.//0(2′) &′ (&(, 2′) 3. 4. (#6 (= 78, 79, 7: , "6 ()"6 ( = 78, 7;, 79, 7: "8 ( = 76, 7;, 79, 7: "; ( = 76, 78, 7;, 7: "6 = 76, 78, 7;, 79, 7: #6 = 76, 78, 79 "6 = 76, 78, 7;, 79, 7:
  94. 94. Lawler !-best 1. " # 2. $ ∈ & " ' "( = " ∖ {'} -.//0(2′) &′ (&(, 2′) 3. 4. (#6 (= 78, 79, 7: , "6 ()"6 ( = 78, 7;, 79, 7: "8 ( = 76, 7;, 79, 7: "; ( = 76, 78, 7;, 7: "6 = 76, 78, 7;, 79, 7: (#8 (= 76, 7;, 79 , "8 () #6 = 76, 78, 79 "6 = 76, 78, 7;, 79, 7:
  95. 95. Lawler !-best 1. " # 2. $ ∈ & " ' "( = " ∖ {'} -.//0(2′) &′ (&(, 2′) 3. 4. (#6 (= 78, 79, 7: , "6 ()"6 ( = 78, 7;, 79, 7: "8 ( = 76, 7;, 79, 7: "; ( = 76, 78, 7;, 7: "6 = 76, 78, 7;, 79, 7: (#8 (= 76, 7;, 79 , "8 () (#; (= 76, 78, 7: , "; ()#6 = 76, 78, 79 "6 = 76, 78, 7;, 79, 7:
  96. 96. Lawler !-best 1. " # 2. $ ∈ # " $ "& = " ∖ {$} +,--.("′) #′ (#&, "′) 3. 4. (#3 &= 45, 46, 47 , "3 &) (#5 &= 43, 48, 46 , "5 &) (#8 &= 43, 45, 47 , "8 &) #5 = 45, 46, 47 "5 = 45, 48, 46, 47 #3 = 43, 45, 46 "3 = 43, 45, 48, 46, 47
  97. 97. Lawler !-best 1. " # 2. $ ∈ & ' $ '( = ' ∖ {$} Lasso("′) #′ (#(, "′) 3. 4. #5 = 65, 67, 68 "5 = 65, 67, 69, 68, 6: (#7 (= 65, 69, 68 , "7 () (#9 (= 65, 67, 6: , "9 () #7 = 67, 68, 6: "7 = 67, 69, 68, 6: "8 ( = 69, 68, 6: ": ( = 67, 69, 6: "; ( = 67, 69, 68 "7 = 67, 69, 68, 6:
  98. 98. Lasso % % n • % • % Lasso
  99. 99. 1. n Thaliana gene expression data (Atwell et al. ’10): • ! ∈ ℝ$%&%'( 2 • ) ∈ ℝ • 134
  100. 100. 2. n 20 Newsgroups Data (Lang’95); ibm vs mac • ! ∈ ℝ$$%&' tf-idf • ( ∈ {ibm, mac} 2 • 1168 → bios drive ibm ide drive ibm dos os, drive ibm controller drive ibm quadra, centris 040, clock mac windows, bios, controller disk, drive ibm bios, help, controller disk, drive ibm centris, pc 610 mac
  101. 101. n • n Lasso • Lawler !-best n • •
  102. 102. • • • • AISTATS’18 • AAAI’17,18 106
  103. 103. n • • 107
  104. 104. n n Sanity Checks for Saliency Maps, NeurIPS’18. • • Sanity Check 108 “ ” Guided-BP
  105. 105. n • • • n • Please Stop Explaining Black Box Models for High-Stakes Decisions, arXiv:1811.10154 109
  106. 106. n • • n • 110

×