This document discusses different methods for microbial typing and comparing the results of different typing methods. It introduces Simpson's Index of Diversity, Adjusted Rand coefficient, and Adjusted Wallace coefficient as quantitative methods for comparing partitions or groupings obtained from different typing methods. These coefficients, along with their confidence intervals, provide a standardized way to determine which methods produce similar or concordant results and which typing markers or combinations of markers best discriminate between microbial strains.

1. João André Carriço, PhD
Microbiology Institute/Institute for Molecular Medicine
Faculty of Medicine, University of Lisbon
Portugal
Mayo Clinic, Rochester MS, 2 November 2015
CHOOSING THE RIGHT MICROBIAL
TYPING METHOD: A QUANTITATIVE
APPROACH
http://im.fm.ul.pt
http://imm.fm.ul.pt
http://www.joaocarrico.info
Twitter: @jacarrico

2. MICROBIAL TYPING
“Crude classifications and False generalizations
are the curse of organized life”
George Bernard Shaw (1856 – 1950)
What is microbial typing:
discriminating strains within a species/subspecies,
for the identification of clones/lineages of interest

3. APPLICATIONS OF MICROBIAL TYPING
Bacterial
Population
Genetics
Pathogenesis
and
Natural
History of
Infection
Surveillance of
Infectious
Diseases
Outbreak
Investigation
and Control

4. MICROBIAL TYPING METHODS
Genotypic – gel based
Pulsed Field Gel Electrophoresis
(PFGE )
Phenotypic :
ATB Resistance profiles
Serotyping
Genotypic - Sequence based
Multilocus Sequence Typing (MLST)
Multilocus VNTR Analysis (MLVA)
Single Locus methods :
emm typing (Group B ,C & G Strep)
Spa typing (S aureus)

5. TRADITIONAL TYPING AND NGS
Chronicle of a Death Foretold
http://en.wikipedia.org/wiki/File:ChronicleOfADeathForetold.JPG
Next Generation Sequencing:
- Gene-by-gene:
wgMLST, cgMLST, MLST
- SNP approaches:
comparison with reference
strains
- Ability to recover most of the
present sequence based
typing information in a single
experimental procedure

6. TYPE /CLONE/LINEAGE AND SUBTYPE CLASSIFICATIONS
Street market, Florence, Italy

7. Serotype :
Serogroup
Serotype
emm typing:
emm type (95% similarity to prototype)
emm subtypes (specific sequence)
Different typing method results are
different partitions of a dataset
Spa typing:
Spa type
BURP complex
PFGE :
PFGE Type (cut-off 80% DICE/UPGMA)
PFGE Subtype (cut-off 80% DICE/UPGMA)
MLST /MLVA/ cgMLST/SNP profile:
Sequence Type
Clonal Complex (eBURST/goeBURST) / cut-off on Minnimun Spanning Trees
TYPE /CLONE/LINEAGE AND SUBTYPE CLASSIFICATIONS

8. CHOOSING THE RIGHT METHOD
Struelens, M.J. et al, 1996. Clinical microbiology and infection, 2(1), pp.2–11.
Performance criteria:
Typeability
Reproducibility
Stability
Discriminatory power
Epidemiological concordance
Typing System concordance
Convinience Criteria:
Flexibility
Speed
Cost
Goal

9. COMPARING TYPING METHODS
Weissman S J et al. Appl. Environ. Microbiol. 2012;78:1353-1360
ConcatenatedMLSTlocus
flmHsequences
The Hard way….

10. NEED FOR QUANTIFICATION AND STATISTICS
When you can measure what you are talking
about and express it in numbers you know
something about it. When you cannot
measure it, when you cannot express it,
your knowledge is of a meagre and
unsatisfactory kind.
- Lord Kelvin 1861

11. COMPARING PARTITIONS FRAMEWORK
Use of three Coefficients :
1) Simpson’s Index of Diversity
2) Adjusted Rand
3) Adjusted Wallace
And the respective 95% confidence intervals

12. COMPARINGPARTITIONS WEBSITE
http://www.comparingpartitions.info

13. Copy/Paste from Excel
COMPARINGPARTITIONS WEBSITE

14. MEASURING DIVERSITY: SID
Simpson’s Index of Diversity
This index indicates the probability of two strains sampled
randomly from a population belonging to two different types
Since it is a probability varies between 0 – 1.
Highly discriminatory methods are desired…
..but are they always needed?
Confidence intervals were defined for SID and should be used.
Simpson, 1948
Hunter and Gaston, 1988
Grundmann et al ,2001

15. Comparing SID’s 95% CIs
Null Hypothesis: The values under comparison are the same

16. COMPARING METHODS RESULTS
PFGEClusters1
s2
s3
s4
s5
s6
s7
SameSequenceType?
Same PFGE cluster?
Y
N
Y N
aa b
c d
For each pair of isolates:
SequenceType

17. ADJUSTED RAND
Overall concordance of two methods taking into account that
the agreement between results could arise by chance alone.
Bi-directional agreement measure
Confidence intervals by jackknife pseudo-values method.

18. CHANCE AGREEMENT ILLUSTRATION
Two possible random rearrangements…

19. CHANCE AGREEMENT:
RAND VS ADJUSTED RAND
Without correction by
chance agreement

20. CHANCE AGREEMENT:
RAND VS ADJUSTED RAND
With correction by
chance agreement

21. ADJUSTED WALLACE
Probability that if two strains share the same classification by a
Method A they also share the same classification by Method B,
corrected by chance agreement
Analytical confidence intervals.
Jackknife pseudo values confidence intervals

22. COMPARING AR AND AW 95% CI
Null Hypothesis: The values under comparison are the same

23. OTHER APPLICATIONS FOR SID,AR AND AW
• Determination of the best set of markers for typing
purposes : given dozens to hundreds or thousands of
possible loci or SNPs is there a subset with enough
discrimination to produce the same results as other
typing method?
http://www.cidmpublichealth.org/pages/ausetts.html.

24. OTHER APPLICATIONS FOR SID,AR AND AW
• Determination of the best set of markers /typing methods for
typing purposes for predicting a specific outcome or any
associated metadata. Examples:
• Using AW to determine the which typing method better
predicts a clinical outcome or prognosis.
• Using AW to determine association between alleles and
Clonal Complexes (Weissman S J et al. Appl. Environ. Microbiol.
2012;78:1353-1360)
• Determining association between alleles or types and
geographical location of sampling
• Determining the best combination of locus to predict a
clinical outcome in order to design a fast RT-PCR method

25. CONCLUSIONS
•The larger the sample size the more accurate can be the
conclusions. Usually >50 is enough but >100 is better to get
statistically significant results. (Severiano PlosOne 2011)
•Always use SID, Adjusted Rand and Adjusted Wallace to have
an overall idea how the methods relate
•Confidence intervals give more information than the point
estimates because they intrinsically take the sample size into
consideration
•Don’t use coefficients that not corrected by chance agreement
when comparing typing methods

26. ACKNOWLEDGEMENTS
Mário Ramirez
Francisco Pinto
Ana Severiano
Mramirez Lab / UMMI Members
Funding from Fundação para a Ciência e Tecnologia
EU 7th Framework programme
www.comparingpartitions.info

27. http://www.escmid.org/immem11
Registration and Abstract submission open
Plenary Sessions
•Small scale microbial epidemiology - outbreaks
•Large scale microbial epidemiology - surveillance
•Bioinformatics for genome-based microbial
epidemiology
•Population genetics: pathogen emergence
•Population dynamics: transmission networks and
surveillance
•Molecular epidemiology for global health and One
Health
Parallel Sessions
•Food and environmental pathogens
•Microbial forensics
•Viral evolution and surveillance
•Fungal and yeast pathogens
•Novel diagnostics and typing methodologies
•Antimicrobial resistance: from mechanisms to impact in
healthcare-associated infections
•Microbial phylogenetic inference
•Making sense of genomic data: bioinformatics tools

28. TO KNOW MORE:
For examples of usage see the list of references in:
http://www.comparingpartitions.info/index.php?link=References