Phenotyping is essential in medical research, as it provides a better understanding of healthcare problems owing to the fact that clinical phenotypes identify subsets of patients with common characteristics. Subgroup discovery appears to be a promising machine learning approach because it provides a framework with which to search for interesting subgroups according to the relations between the individual characteristics and a target value. Each single pattern extracted by SD algorithms is human-readable. However, its complexity (the number of attributes involved) and the high number of subgroups obtained make the overall model difficult to understand. In this work, we propose a method with which to explain SD, designed for the clinical context. We have employed a two-step process in order to obtain SD model-agnostic explanations based on a decision tree surrogate model. The complexity involved in evaluating explainable methods led us to adopt a multiple strategy. We first show how explanations are built, and test a selection of state-of-the-art SD algorithms and gold-standard datasets. We then illustrate the suitability of the method in a clinical use case for an antimicrobial resistance problem. Finally, we study the utility of the method by surveying a small group in order to validate it from a human-centric perspective

1. Enrique Valero-Leal 1, M. Campos2,3, J. M. Juarez2
1 Technical University of Madrid
2 AIKE research group (INTICO), University of Murcia
3 IMIB-ARRIXACA Murcian Biomedical Research Institute
Simple explanations to summarise
Subgroup Discovery outcomes: a case of
study concerning patient phenotyping
Sept 19 2022 X-KDD workshop, Grenoble
Funded by Spanish Ministry of Science, Innovation and Universities
under the CONFAINCE project (Ref:PID2021-122194OB-I00 ), and by
the European Fund for Regional Development (EFRD, FEDER).

2. 2
Simple explanations to summarise Subgroup Discovery: patient phenotyping
FULL PAPER DOWNLOADABLE AT:
https://kdd.isti.cnr.it/xkdd2022/papers/XKDD_2022_paper_9989.pdf
Enrique Valero-Leal, M. Campos, J. M. Juarez. Simple
explanations to summarise Subgroup Discovery:
patient phenotyping. Proceedings of the
International Workshop on
eXplainable Knowledge Discovery in Data Mining
XKDD 2022. Lecture Notes in Computer Science.
Springer Series. 2022
These slides summarise the conference
paper presented at XKDD 2022 workshop @ECML-PKDD:
This research was funded by
under the CONFAINCE project
(Ref:PID2021-122194OB-I00 )

3. • OUR RESEARCH GOAL
Generate trustworthy medical hypotheses for
patient phenotyping.
Subgroup Discovery algorithms approach
Medical-friendly explanations
SubgroupExplainer
3
Simple explanations to summarise Subgroup Discovery: patient phenotyping

4. • OUTLINE:
1. Clinical problem & research goal
2. Subgroup discovery
3. Contribution: SubgroupExplainer
4. Experiments
5. Conclusions
4
✓
Simple explanations to summarise Subgroup Discovery: patient phenotyping

5. • SUBGROUP DISCOVERY
Clustering != Subgroup discovery
5
Simple explanations to summarise Subgroup Discovery: patient phenotyping
Picture from: S. Ventura and J. M. Luna (2018). Supervised Descriptive Pattern Mining. Springer books.

6. • SUBGROUP DISCOVERY: DEFINITIONS
6
Simple explanations to summarise Subgroup Discovery: patient phenotyping
𝑫𝒂𝒕𝒂𝒔𝒆𝒕: 𝐷 = 𝐼, 𝐴 𝐴 = 𝑎!, 𝑎", … , 𝑎#
𝑺𝒆𝒍𝒆𝒄𝒕𝒐𝒓 𝑠$%: 𝐼 → 𝐵𝑜𝑜𝑙𝑒𝑎𝑛 𝑠$% 𝑖 = 𝑇 ⟺ 𝑠𝑐 𝑖𝑠 𝑓𝑢𝑙𝑓𝑖𝑙𝑙𝑒𝑑 𝑏𝑦 𝑖
𝑷𝒂𝒕𝒕𝒆𝒓𝒏: 𝑃 = 𝑠!, 𝑠", … , 𝑠& 𝑃 𝑖𝑛 𝑐𝑜𝑛𝑗𝑢𝑛𝑐𝑡𝑖𝑣𝑒 𝑓𝑜𝑟𝑚
𝑺𝒖𝒃𝒈𝒓𝒐𝒖𝒑: 𝑆𝐺 = 𝑃, 𝑠$' 𝑆𝐺 ∘ = ∀𝑖 ∈ 𝐼 |𝑠$% 𝑖 = 𝑇, ∀𝑠𝑐 ∈ 𝑃
SG= if (age>35, culture=Enteroc.Faecium) THEN suscept=Resistant
𝑸𝒖𝒂𝒍𝒊𝒕𝒚 𝒇𝒖𝒏𝒄𝒕𝒊𝒐𝒏: 𝑞𝑓: 𝑃, 𝐷 → ℝ
SD algorithms:
Frequent pattern mining: SD-MAP, Dp-Subroup, BSD, etc.
Beam search: SD, CN2-SD, SD4TS (heuristics)

7. • OUTLINE:
1. Clinical problem & research goal
2. Subgroup discovery
3. Contribution: SubgroupExplainer
4. Experiments
5. Conclusions
7
✓
Simple explanations to summarise Subgroup Discovery: patient phenotyping
✓

8. • CONTRIBUTION
– Overcome difficulties explaining SD to clinicians
– Simple explanations to increase trust in SD
8
Simple explanations to summarise Subgroup Discovery: patient phenotyping

9. • CONTRIBUTION
– SubgroupExplainer
• XAI characteristics
–SD model-agnostic
–Global explanations
–Surrogated model
–Tree-like explanations
9
Simple explanations to summarise Subgroup Discovery: patient phenotyping

10. • CONTRIBUTION
– SubgroupExplainer
• Tree-like explanations, why?
10
Simple explanations to summarise Subgroup Discovery: patient phenotyping

11. • CONTRIBUTION
– SubgroupExplainer: simple explanations
11
Simple explanations to summarise Subgroup Discovery: patient phenotyping
DATASET
#attributes: 15
#instances:1049
BlackBox
SD
Algorithm
SG #1
SG #2
SG#19
SG# 20
...
#20
#5,2 #11
#19 #9,2 #7,14
#17
#7 #10
#3
#2,7
AT1 AT2 . . . AT15
1 26 . . . A
. . . . . . . . . B
64 12 . . . A
Step 1
DB
labelling
A1 … A14 L SUBGROUPS
1 … . . . #17,#2, #5
. . . . . . #20
64 . . . #11, #7, #2
Step 2:
SURROGATE
Explainer
building
SUBGROUPS:
SG#1: At1>2,A3=6=>AT15=A
SG#2: At1=4,At2=5, At4=21=>AT15=B
SG#3: At1<11,At2=5=>AT15=A
SG#4: At2>40,At3=5, At4=21=>AT15=C
. . .
SG#20: At2=3,At3<7,AT4<22 =>AT15=A
Labelled Dataset
SD EXPLANATION
Target Attribute
PROPOSAL ADVANTAGES:

12. • OUTLINE:
1. Clinical problem & research goal
2. Subgroup discovery
3. Contribution: SubgroupExplainer
4. Experiments
5. Conclusions
12
✓
Simple explanations to summarise Subgroup Discovery: patient phenotyping
✓
✓

13. • EXPERIMENTS
1. Computational properties and scalability
2. Clinical reproducible use case
3. Human subjective study
13
Simple explanations to summarise Subgroup Discovery: patient phenotyping

14. • EXPERIMENTS: Computational properties and scalability
14
Simple explanations to summarise Subgroup Discovery: patient phenotyping
Ssg: subgroups
S: all selectors from Ssg
Su : unique selectors from S
card: mean cardinality |S|/|Ssg|
T: number vertex of tree
purity: proportion correctly class

15. • EXPERIMENTS
– Clinical reproducible use case
MIMIC III dataset (60,000/1280 admissions)
15
Simple explanations to summarise Subgroup Discovery: patient phenotyping
CN2-SD CART+WRAcc

16. • EXPERIMENTS
– Human subjective study
18 participants surveyed
ML and unfamiliar AI
Task oriented: SD & Trees.
Subjective opinion
16
Simple explanations to summarise Subgroup Discovery: patient phenotyping

17. • OUTLINE:
1. Clinical problem & research goal
2. Subgroup discovery
3. Contribution: SubgroupExplainer
4. Experiments
5. Conclusions
17
✓
Simple explanations to summarise Subgroup Discovery: patient phenotyping
✓
✓
✓

18. • CONCLUSIONS
– Subgroup explainer:
SD problem pioneer
SD-agnostic, global, tree-like surrogate explanations.
Designed for phenotyping problems.
– Compactness: distil myriad of subgroups
– Comparative method: multiple SD outcomes
– Secondary use: surrogated model.
18
Simple explanations to summarise Subgroup Discovery: patient phenotyping

19. Simple explanations to summarise Subgroup Discovery
outcomes: a case of study concerning patient phenotyping
Contact:
Jose M. Juarez
jmjuarez@um.es
Simple explanations to summarise Subgroup Discovery: patient phenotyping
Subgroup Explainer:
Phenotyping method
Compact, comparative, secondary use

20. 20
Simple explanations to summarise Subgroup Discovery: patient phenotyping
FULL PAPER DOWNLOADABLE AT:
https://kdd.isti.cnr.it/xkdd2022/papers/XKDD_2022_paper_9989.pdf
Enrique Valero-Leal, M. Campos, J. M. Juarez. Simple
explanations to summarise Subgroup Discovery:
patient phenotyping. Proceedings of the
International Workshop on
eXplainable Knowledge Discovery in Data Mining
XKDD 2022. Lecture Notes in Computer Science.
Springer Series. 2022
These slides summarise the conference
paper presented at XKDD 2022 workshop @ECML-PKDD: