Eisai-2

1. Qike Li, A. Grant Schissler, Vincent Gardeux, Ikbel Achour, Colleen Kenost,
Joanne Berghout, Haiquan Li, Hao Helen Zhang, Yves A. Lussier
N-of-1-pathways MixEnrich: advancing precision
medicine via single-subject analysis in discovering
dynamic changes of transcriptomes

2. Outline
• Background
• Methods: N-of-1-pathways MixEnrich
• Results:
• Simulation Study
• Validation Case Study
• Limitations
• Take home message
We developed a new and effective method to identify
dysregulated pathways within a single patient.

3. Problem statement
Population
Case Control
Average / Common
Gene signature / pathway signature
Control / Case
Paired Samples
Individual
Common
signature
Individual
signature
Transcriptome
Analysis Transcriptome
Analysis
DEG+Enrichment1
GSEA2
1Beißbarth, T. and Speed, T. Bioinformatics 2004; 2Subramanian A. et. al PNAS 2005

4. Applications of Single-subject Analysis
• Personalized drug responses
• Personalized disease mechanisms
• Rare diseases

5. Bridging the GAP
Dynamic mRNA Changes
Control Case
Methods developed so far
• Focus on static transcriptome profile
• Dynamic changes of transcriptomes
Individual
Pathway dysregulation

6. Bridging the GAP
N-of-1-pathways Wilcoxon1
N-of-1-pathways MD2
Deviation from EqualityMethods developed so far
• Focus on static transcriptome profile
• Dynamic changes of transcriptomes
1Gardeux, V., …, Lussier, Y., JAMIA 2013; 2Schissler, A., …, Lussier, Y., Bioinformatics 2015
Control (Log2 Expression)
Sample 2
Sample1
Case(Log2Expression)

7. Bridging the GAP
Methods developed so far
• Focus on static transcriptome profile
• Dynamic changes of transcriptomes
• Ignore the background noise
Deviation from Equality
1Gardeux, V., …, Lussier, Y., JAMIA 2013; 2Schissler, A., …, Lussier, Y., Bioinformatics 2015
Control (Log2 Expression)
Sample 2
Sample1
Case(Log2Expression)
N-of-1-pathways Wilcoxon1
N-of-1-pathways MD2

8. Bridging the GAP
Methods developed so far
• Focus on static transcriptome profile
• Dynamic changes of transcriptomes
• Ignore the background noise
• Unidirectional dynamic change
Control (Log2 Expression)
Sample 2
Sample1
Case(Log2Expression)
Deviation from Equality
N-of-1-pathways Wilcoxon1
N-of-1-pathways MD2

9. Bridging the GAP
Our Approach
A competitive model to discover
uni- and bi-directional dysregulated
pathways
Baseline Case
Dynamic mRNA Changes
Individual

10. Bridging the GAP
Our Approach
A competitive model to discover
uni- and bi-directional dysregulated
pathways
Our Method
Mixture model clustering
followed by gene-set
enrichment test
(MixEnrich)
Baseline Case
Dynamic mRNA Changes
Individual

11. Outline
• Background
• Methods: N-of-1-pathways MixEnrich
• Results:
• Simulation Study
• Validation Case Study
• Limitations
• Take home message
We developed a new and effective method to identify
dysregulated pathways within a single patient.

12. Two Samples of Transcriptome

13. DEG Discovery: Mixture Model

14. DEG Discovery: Mixture Model
• The cluster membership of each genei is a latent variable that follows Bernoulli
distribution.

15. DEG Discovery: Mixture Model
• The cluster membership of each genei is a latent variable that follows Bernoulli
distribution.
• Given the cluster membership, the |log2FC| of genei follows the following distribution,

16. DEG Discovery: Mixture Model
• The cluster membership of each genei is a latent variable that follows Bernoulli
distribution.
• Given the cluster membership, the |log2FC| of genei follows the following distribution,
• The likelihood that a genei belongs one cluster or the other is assessed by the
posterior probability using Bayes rules.

17. Gene Set Enrichment

18. Gene Set Enrichment
Contingency table for Fisher’s Exact Test
dysregulated genes unaltered genes
genes in target pathway d M – d
genes not in target pathway D - d N - M - D + d

19. Simulation Parameters
Parameter Description of the parameter Values tested
p.S Number of mRNAs randomly chosen in the target pathway {5, 10, [15, 490] by step 25, 500}
p.dPct Percentage of dysregulated mRNAs in the target pathway {(0, 1] by step 0.05}
p.FC Fold change of mRNAs in the target pathway {1.3, 1.5, 2}
p.upPct Percentage of up-regulated mRNAs among dysregulated
mRNAs in the target pathways
{0, 0.1, 0.2, 0.3, 0.4, 0.5}
bg.FC Fold change of dysregulated background mRNAs {0, 1.3, 1.5, 2}
bg.dPct Percentage of dysregulated mRNAs as noise in the
background
{0, 0.01, 0.05, 0.1, 0.2}
• 107,640 scenarios of pathway dysregulation were investigated

20. Outline
• Background
• Methods: N-of-1-pathways MixEnrich
• Results:
• Simulation Study
• Validation Case Study
• Limitations
• Take home message
We developed a new and effective method to identify
dysregulated pathways within a single patient.

21. Exemplar ROC curves

22. Exemplar ROC curves

23. Overall Performance Comparison

24. Background Fold-Change
MixEnrich MD
●
●●
●
●●
●●
●●●
●●●●●●●●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●●●
●●●●●●●
●
●
●
●
●
●
●●
●●●●●
●
●
●●
●
●
●
●
●●●●●●●●
●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●●●
●●●
●●●●
●
●
●●
●
●●●
●●●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●●●●●
●
●
●
●
●●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●●
●●●●●●●●●
●●
●●
●
●
●
●
●●●●●●●●
●
●
●
●
●
●
●●
●●●●●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●●
●●●●●
●●●
●
●
●
●
●
●
●
●
●●
●●●●●●
●
●
●●
●
●
●●●●●●●●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●
●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●●
●●●●●
●●●●
●●
●●
●
●
●
●
●●●●●●●●●
●
●
●
●
●
●
●●
●
●●●●●●●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●●●●●●●●●
●●
●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●●
●
●
●
●
●●●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●●
●●●●●●●●●
●
●
●
●
●
●
●●
●●
●●●●●●
●
●
●●
●
●
●●
●
●●●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●●●
●●●
●●●●●
●●●●
●
●
●●
●
●●●
●●●●●●●●
●
●
●
●
●
●
●●
●
●●●●●●
●
●
●
●
●
●●●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●●●●●●●●
●
●
●●
●
●●
●
●●
●●●●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●●●●
●
●
●
●
●●●●
●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●●●●●●
●●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●
●●
●●●●●
●
●
●
●
●
●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●●●
●●●●●●
●
●
●
●
●
●●●
●●●●●●●●
●
●
●
●
●
●
●●
●
●●●●●●
●
●
●
●
●
●●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●●●●●
●●●●
●
●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●●●●●●●●
●
●
●●
●
●●●●
●
●
●
●
●●●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●●●●
●
●
●●
●
●
●
●
●●
●
●●●●
●
●
●●
●
●
●●
●
●●●●●●
●
●
●
●
●
●●●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●●●●
●●●●
●
●
●●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●●●●●●●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●
●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●●●
●●●●●●●●●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●●
●●●●●●●●
●
●
●
●
●
●●●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●●●●●●●●
●●
●
●
●
●
●
●
●●
●●●●●●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●●●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●●
●●●●●
●●●
●
●
●
●
●
●●●
●●
●●●●●
●
●
●
●
●
●
●●
●
●
●●●●●●
●
●
●
●
●
●●●●●
●
●
●
●
●●●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●●●●●●●●
●●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●
●
●
●
●●
●●●●●●●●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●●●●●
●●●
●
●
●
●
●
●
●●
●●
●●●●●●●●
●
●
●●
●
●
●
●
●
●●●●●●●
●
●
●
●
●
●●●●●●●
●
●
●
●
●●●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●●●●●
●●●●
●●
●
●
●
●
●●
●●
●●●●●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●●●●
●
●
●
●
●●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●●
●●●●●
●●
●●
●
●
●
●
●
●
●●
●
●●●●●●●●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●●●●●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●●
●
●●●●●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●●●
●
●
●●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●●●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●●●●
●●
●
●
●●
●
●
●
●
●●
●
●●●●●●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●
●●
●●
●
●
●
●●●
●
●
●
●●●●
●
●
●
●
●
●
●●
●
●●●●●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●●●●●●●●
●
●
●
●
●
●
●●
●●
●●●●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●●●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●●●●●
●●
●
●
●
●●●
●
●
●●●●●●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●●●
●
●●
●●
●●●●
●
●
●
●
●
●
●
●
●●●
●●
●●
●●●
●●
●●
●
●
●●
●
●●
●●●
●
●●●
●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●●●●
●●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●
●
●●●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●●
●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●●●
0.00
0.25
0.50
0.75
1.00
0 0.010.05 0.1 0.2 0 0.010.05 0.1 0.2
Percentage of background noise (bg.dPct)
AUC

25. Up-regulation Percentage
MixEnrich MD
●
●●
●
●●
●●
●●●
●●●●●●●●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●●●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●●
●●●●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●●
●●●●●●●●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●●●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●●
●●●●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●●
●●●●●●●●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●●●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●●
●●●●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●●
●●●●●●●●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●●●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●●
●●●●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●●
●●●●●●●●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●●●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●●
●●●●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●●●
●●●●●●●●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●●●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●●
●●●●
●
●
●●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
0.4
0.6
0.8
1.0
0 0.1 0.2 0.3 0.4 0.5 0 0.1 0.2 0.3 0.4 0.5
Up−regulation Percentage (p.upPct)
AUC

26. Outline
• Background
• Methods: N-of-1-pathways MixEnrich
• Results:
• Simulation Study
• Validation Case Study (matched tumor and normal samples
of 45 head and neck cancer patients1)
• Limitations
• Take home message
We developed a new and effective method to identify
dysregulated pathways within a single patient.
1Downloaded from TCGA

27. Study design
45 head and neck cancer patients
30 patients
for building cohort-
expectation standard using
DESeq1+Enrichment
15 patients
for evaluating the performance
of the methods
1Anders, S, and Huber, W. Genome biology 11.10 (2010): 1.
Run Single-subject
methods on each patient
Randomly sample 50
distinct subsets

28. Validation Case Study—Head and Neck Squamous Cell Carcinoma

29. Outline
• Background
• Methods: N-of-1-pathways MixEnrich
• Results:
• Simulation Study
• Validation Case Study
• Limitations
• Take home message
We developed a new and effective method to identify
dysregulated pathways within a single patient.

30. Comparison of Transcriptome Analyses
DEG Enrich /
GSEA
N-of-1-pathways
MixEnrich
Cohort-level
pathway testing
N-of-1-pathways
Wilcoxon / MD
N-of-1
(1) Bidirectional
dysregulation
(2) Adjust for
background noise
(3) Analyzes
paired samples
  
 

  

31. Towards Precision Medicine
Individual
Genetic Makeup
N-of-1-pathways MixEnrich:
Personal Transcriptome Profile
Same Phenotype
Trait or Disease

32. Towards Precision Medicine
Individual
Genetic Makeup
N-of-1-pathways MixEnrich:
Personal Transcriptome Profile
Same Phenotype
Trait or Disease
Personalized drug responses

33. Towards Precision Medicine
Individual
Genetic Makeup
N-of-1-pathways MixEnrich:
Personal Transcriptome Profile
Same Phenotype
Trait or Disease
Personalized drug responses
Personalized disease mechanism

34. Towards Precision Medicine
Individual
Genetic Makeup
N-of-1-pathways MixEnrich:
Personal Transcriptome Profile
Same Phenotype
Trait or Disease
Personalized drug responses
Personalized disease mechanism
Rare diseases

35. Limitations
• Expectation Maximization (EM) algorithm is guaranteed to
converge to global optimum
• Future study: Nonparametric clustering or Ensemble clustering
• Not suitable for testing a small panel of genes (e.g. # genes <
200).

36. Software
N-of-1-pathways MixEnrich
N-of-1-pathways Software available
as R package and Shiny app
www.lussiergroup.org

37. N-of-1-pathways MixEnrich Team & Acknowledgements
Ikbel Achour, PhD
Haiquan Li, PhD
Vincent Gardeux, PhD
Joanne Berghout, PhD
Yves A. Lussier, MD
Qike Li, MS A. Grant Schissler, MS Colleen Kenost, EdD
@lussiergroup
@UA_CB2
#PrecisionMedicine
Helen Zhang, PhD
Acknowledgements: This study was supported in part by The University of Arizona Cancer
Center, The University of Arizona BIO5 Institute, The University of Arizona Center for Biomedical
Informatics and Biostatistics, and the University of Arizona Health Sciences Center.

38. Questions?

40. Pathway Dysregulation Percentage
MixEnrich MD
●●
●●●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●●
●
●
●
●
●
●
●●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●●●
●●
●
●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●●
●●
●●●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●●●
●
●
●●●●●
●
●●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●●
●
●●
●●●●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●
●●●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●
●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●●
●
●
●
●●
●
●●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●●
●●●
●
●
●●●●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●●●●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●●
●●
●
●●●●●
●
●●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●●
●●●
●
●●
●●●
●
●●
●●●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●●
●
●●
●●
●●●
●●●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●●●●●●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●
●
●●●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●●
●●
●
●●
●●●
●
●
●
●●●
●
●
●
●●●●
●
●●
●
●●
●
●●
●
●●●●
●
●●
●
●●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●●
●
●●
●●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●●●●
●●
●
●●
●●●
●
●
●●●●●●●
●
●●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●●●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●●●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●
●●●
●
●
●
●
●
●●●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●●
●●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●●
●
●●●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●●●
●
●
●
●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●●
●
●●●●●●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●●●●
●●●
●
●●●●●
●●●●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●●
●●
●
●●●●●●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●●
●
●●●●
●
●●●●●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●●●
●
●●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●●●
●●●●
●
●●●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●●
●●
●
●●
●
●●●
●
●●●●●●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●●
●●
●●●
●●
●
●●
●●
●
●●●●
●
●●●●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●●
●●
●
●
●
●●●
●●
●
●
●
●●●●●●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●●
●●●●●●
●
●
●●●●●●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●●●
●●●●●
●
●●●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●●●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●●
●●
●
●
●
●●●
●●
●
●●●
●●●●●●●●●●
●
●●
●
●●
●
●●
●
●●
●
●●
●●●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●●●
●
●●
●
●●
●
●●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●
●●●●●●●
●●●●●●●
●
●●
●
●
●●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●●
●
●●●
●●●●●●●●●●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●●
●
●●●●●●●●●●●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●●
●●
●
●
●●
●●●
●●●●●●●
●●●●●●●●●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●●
●●●●●●●●●●●●●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●
●●●
●●●●●●●●●●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●●●●●●●●
●●●●●●●●●●●●●
●
●●
●
●●
●
●●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●●●●●
●●●
●●●●●●●●●●●●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●●●
●
●
●●●●●●●●
●●●●●●●●●●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●●●●●●●●●●●●●●●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●●●
●
●●
●●●
●●●●●●●●●●●●●●●●●●●●
●
●●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●●●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●●
●
●●●
●
●●
●
●●●●
●
●●
●
●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●●●
●
●●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●●●●●●●●●●●●●●●●●●●●●●
●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●●
●
●●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●●●
●
●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●●●
●
●
●●
●
●●●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●●
●
●●●
●
●
●
●●
●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●●
●
●●
●
●●●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●
●
●●●
●
●●●
●
●●
●
●
●
●
●●
●
●●
●
●
●●
●
●●●
●
●
●
●
●●
●
●●
●
●
●●●●
●
●●
●●
●●●
●
●●
●
●●
●
●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●●
●
●●●
●
●●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●●
●
●●●
●
●●●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●●●
●
●●●●●
●●
●●●●●●●●●●●●●●●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●●●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●●●●●●
●●●
●
●
●●
●
●●●●●●
●
●●
●
●
●●
●●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●
●●●
●●●
●
●●●
●
●●●●
●
●
●●
●
●●
●
●●
●
●
●●●●●●●●
●
●●
●
●●
●
●
●
●●●●●●
●●●
●
●●●
●
●●●●●●
●●●●●●
●●●
●
●
●
●●
●●
●
●●
●
●●●●●●
●●
●
●
●
●
●
●
●
●
●●
●●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●
●
●●●
●
●
●
●
●●●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●●●
●
●●●●●●
●
●●●●●●●
●
●●●
●●●
●●
●
●●●●●
●
●●●●●●●●●●●
●
●
●
●●●●●
●
●●●
●
●●
●
●●
●
●●
●●
●●
●
●
●
●●
●
●●●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●
●●
●
●●●●
●
●
●
●●●●●
●
●●
●
●
●●
●
●●
●
●●
●
●
●●●●●●●●●●●●
●
●
●
●●●
●●●
●
●
●
●
●●●●
●
●●
●
●●●●●●
●●●●●
●
●●●●
●
●●●●●
●
●
●
●●●●●
●
●●
●
●
●
●
●
●
●
●
●●●
●●●
●●●●●●●●●●●●
●
●●●●
●
●
●
●●●●●
●
●●
●●
●
●●
●
●●
●
●
●
●
●●
●
●●●
●●●●●●●●●
●
●●●●●●●●
●
●●●
●●●●●
●
●●
●
●
●●
●
●●●●●●●●
●●●●
●
●●
●●●●
●
●●
●
●●
●
●●●
●
●●
●●
●
●
●
●●
●
●●
●●
●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●
●●●●●
●
●●
●●●●
●
●
●●
●
●●
●
●
●●
●
●●
●
●●●●●●
●
●●●●●●
●
●●●
●●●●
●
●●
●
●●●
●
●●
●●
●
●
●●●●●●●●
●●●●
●
●●
●●●●
●
●
●
●●●
●●
●
●●
●●
●
●
●
●●
●
●●
●●
●
●●●●●●●●●●●●●●●●●●●
●
●●●●
●
●
●●●
●
●●
●
●●●●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●●●●●●
●
●●●●
●
●●●●●●●
●
●●
●
●●
●
●●●
●●●●●●●
●●●●
●
●●
●
●●●●●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●
●●
●
●●
●
●●●●●
●
●●
●
●●
●
●●
●
●
●●●●●●●●●
●●●
●●●●
●
●
●
●●●
●
●●
●
●●●
●
●●●●
●●●●●●●●
●●●
●
●●
●
●
●●
●
●●
●
●●●●●
●
●
●●
●
●
●
●
●
●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●
●●●●
●
●
●●●
●
●●●
●
●●
●
●●
●
●●
●
●●
●
●
●●●●●●
●
●●●●●●●
●
●
●
●●●●
●●●
●
●
●●●●●●
●
●●●●●●●●●●●●●
●
●●
●●●●●●
●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●
●
●
●●●
●
●
●
●●
●
●●●●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●●●●●●●
●●●●●●
●
●
●●●
●
●●
●
●●
●
●
●●●●
●
●●
●
●●●●●●●
●●●●●
●
●●
●●●●●●
●
●●●
●
●●●●●
●
●●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●●
●●●
●
●●
●●●●
●
●●
●
●●●●
●
●●
●
●●
●
●●●
●●●●●●●
●
●●●●●●
●●
●●●●●
●
●●
●
●
●
●
●●
●
●●●●●●●●●●
●●
●●●●
●
●●
●
●●●
●●
●
●●●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●
●●●
●
●
●●
●●●●
●
●
●●
●
●●
●
●●●
●
●●
●
●●●●●●
●
●●●●●
●
●●
●●
●
●●●
●
●●
●
●●●●
●●
●
●
●●●●●●●●●●●●
●
●
●
●●
●
●●
●
●●
●
●●●
●
●●
●
●
●
●
●
●
●●●
●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●
●●●
●
●●
●
●●●●●
●
●
●
●●
●
●●
●
●●
●
●
●●
●●●●●
●●●●
●
●●●
●●●●●●
●●
●
●●
●
●
●
●
●● ●
●●●●●●●
●
●●
●
●
●
●●●
●
●●●●●●
●
●●●●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●
●
●●●
●
●
●●
●
●●
●
●●
●
●●●
●
●●
●
●●●
●
●
●
●
●●●●●●●
●
●●
●
●●
●
●
●
●●●●
●
●●
●
●●
●
●●
●●
●
●
●●●●●●●
●●●●
●
●●
●
●●
●
●●●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●●
●●●●●●●●●●●●●●●●●●●
●
●●●●●●●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●
●●●
●
●●●●●●●●
●
●●●●●●
●
●
●
●
●
●
●
●
●●●●
●●●
●
●●
●●●●●●●
●
●●●●
●
●
●
●●●●●●●●
●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●
●
●
●●●●
●
●●●
●
●●
●
●
●●●
●
●●
●
●●
●
●●●
●●●●●
●
●
●●
●●●
●
●
●
●●
●
●●●●
●
●
●●●●
●
●●
●
●
●●●●●●●
●
●●●●
●
●●
●
●●●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●
●
●●
●
●●●●●●
●
●●●
●
●
●
●
●
●●
●
●●●
●
●
●
●●●
●●●●●●
●
●●●●●●●
●●
●●
●
●
●●
●
●●
●
●●●
●●●●●●●
●
●●●●
●
●
●
●●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●
●
●●●●●
●
●●
●●
●●
●
●●●●
●
●●
●
●●
●
●
●
●●●
●●●●
●
●●●●●
●
●●●
●
●
●
●
●
●
●●●
●
●●
●
●●●
●●●●●●●
●●●●
●
●●
●
●●●●●●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●
●
●●●●●●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●●
●●●●●●●
●●●
●
●
●●●
●
●
●
●
●
●●
●
●●●●
●
●●●
●●●●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●●●●●●●●●●●●●
●
●●●●●●
●
●●
●●●
●●●
●
●
●
●●
●
●●
●
●●
●
●
●●
●●●●●
●
●
●
●●●
●
●
●
●●●●●
●●
●
●
●
●●●
●●
●
●●●●●●
●
●●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●●●●●●●●●●●●●●●●
●
●●●●●●●●●
●
●●●
●●●●
●
●●
●
●
●
●
●
●
●●●
●
●●●
●
●●●●●●●●●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●●●
●
●●●●●●
●
●●●●
●●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●
●
●●●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●
●
●●
●
●●●●●●●
●●
●●
●●●●●
●
●●●●
●
●●●●●●●●
●●●●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●
●
●●●●
●
●
●●
●●
●●●
●
●
●
●
●●
●
●●●
●
●
●●●●●●●
●
●●●●●●
●
●
●
●●●
●
●●●
●
●
●
●●
●●●●●●
●
●●●●●
●
●
●
●●●
●●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●
●
●●
●●
●●
●●●●
●
●●
●
●●●
●
●
●
●●●●●●
●
●●●
●
●●
●●
●●
●
●●
●
●●
●
●●
●
●●●●●●●●
●
●●●●●●●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●●●●●●●●●●●●●●●●●
●●●●●
●
●●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●●●●
●
●●
●●
●●
●●
●●
●
●●●
●
●
●
●●
●
●●●●●●●
●●●●
●
●
●
●●●
●●
●●
●
●
●●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●
●
●●●●●●
●
●
●●
●
●●
●
●●●
●
●●
●
●●●
●
●●
●
●●
●
●●
●●●●●●●
●●●
●
●●●
●
●
●●●
●
●●
●
●●
●
●
●
●●●
●●●●●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●
●
●●●●●●
●
●●
●
●
●
●
●
●
●●●●
●
●●
●
●●●●
●
●●
●
●●●●●●●
●
●
●●●●
●
●
●●
●
●●●
●●●
●
●●
●
●●●●●●●
●
●●●●
●
●●
●
●●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●
●
●●●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●●●
●●●●●
●●●●●
●
●
●
●●●
●●
●
●
●●
●●●
●
●●●
●●●●●●●
●
●●●●●
●
●●
●
●●●●●
●
●
●●●●●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●
●
●●●●●
●
●●
●
●●
●
●●●
●
●●
●
●●●
●
●
●
●●
●
●●●●
●
●●●●
●
●
●●
●
●●●
●
●
●
●●
●
●●
●●●●●●●
●
●●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●
●
●●●●●●●
●
●
●●
●●
●
●●●●
●
●
●
●●
●
●●
●
●●●
●●●●●
●●●●
●
●●
●
●
●●
●
●
●●
●
●
●
●●●
●●●●●●●
●
●●●●●
●
●
●
●●●●●●
●
●●●
●
●●●
●
●
●
●
●
●
●●
●●●●●●●●●●●●●●●
●
●●●
●●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●●●●●
●●
●
●●
●●
●●●
●
●
●
●●
●
●
●
● ●●●●●●●●●●●●●
●
●●
●
●●●●●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●
●
●●●●
●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●●●●●
●
●●●●●●●●●●●●●●●
●
●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●
●●●●●●
●●●●●●●●●●●●●●●●●
●
●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●
●
●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●
● ●●●●●●●●●●
●
●●●
●
●●●●●●●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●
●
●●●
●
●●●●●●●●●●●●●●●●●●●●
●
●●●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●
●●
●
●●●●●●●●●●●●●●
●
●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●●●
●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●● ●●●●●●●●●●●●
●
●●●●
●
●●●●●●●●●●
●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
●
●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●
●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●
●
●●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●
●●
●
●
●
●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●
●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●●●●●
●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●
●
●●●●
●●
●
●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●● ●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●
●
●●●●●●●●●●●●●●●●●●
●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●
●
●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●
●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●
●
●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●
●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●
●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●
●●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●
●●
●
●●
●●
●●
●
●
●
●●
●
●
●
●●●
●●●●●
●●●
●
●
●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●
●●●●●
●
●
●●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●
●●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●●●●●
●●
●
●
●
●●●●●
●●
●
●
●●
●●●●●●●●
●
●●●
●●●●●
●●
●
●●●●●
●
●
●●●
●●●●●●●●●●
●
●●●●●
●●●
●●●●●●●●●●●●●●●●●
0.00
0.25
0.50
0.75
1.00
0.2 0.4 0.6 0.8 1 0.2 0.4 0.6 0.8 1
Percentage of dysregulated genes
in the target pathway (p.dPct)
AUC