Evaluation of
Molecular Typing Techniques
Thomas Weniger
Department of Periodontology
University of Münster, Germany
tweniger@uni-muenster.de
Typing:
Phenotypic and/or genetic
analysis of bacterial isolates,
below the species/subspecies
level, performed in order to
generate strain/clone-specific
fingerprints […]
van Belkum et al., (2007). CMI, 13:1
PFGE MLST
flaAothers
C. jejuni: Different typing methods
Questions Suitable methods
Discriminatory
power
Time
span
Outbreak investigations
Short-term/local surveillance
Control of hygiene measures
PFGE, RFLP,
AFLP, RA-PCR,
VNTR, SLST,
micro-array
high
weeks -
month
Long-term/global
epidemiological studies
Population genetics
Analysis of population-based
interventions, e.g. vaccination
(MLEE), MLST,
micro-array,
in part SLST
low years
6-AM 4
Typing questions & suitable methods
Typing system
performance
Convenience
A
A
B
Stability
A
A
Reproducibility
B
A
A
B
C
D
E
E
Typing method 1
Discriminatory
power
A
B
C
D
E
E
Typing method 1
a
b
c
c
b
b
Typing method 2
DI: 0.933 [0.805 – 1.0] DI: 0.733 [0.53 – 0.936]
Discriminatory
power
AAA
B
C
Epidemiological Concordance
Typing system
performance
Convenience
 Costs
 Rapidity
 Ease of use
Convenience
Typing system concordance
- Rand‘s index
- adjusted Rand‘s index
- Wallace coefficients
Carrico et al. (2006) JCM 44: 2524
PFGE versus MLST MLST
same different
PFGE
same a b
different c d
Rand (1971) J. Am. Stat. Assoc. 66
Hubert & Arabie (1985) J. Classification 2
Wallace (1983) J. Am. Stat. Assoc. 78
A
B
C
D
E
E
a
b
c
c
b
b
Typing method 1 Typing method 2
Isolate 1
Isolate 2
Isolate 3
Isolate 4
Isolate 5
Isolate 6
Typing system concordance
Example
• 42 Campylobacter
isolates
• 2 outbreaks
• 12 sporadic cases
• Typing by PFGE,
MLST, flaA, and flaB
sequencing.
Mellmann et al. (2004). JCM 42: 4840
Example
Mellmann et al. (2004). JCM 42: 4840
Example DI
Method No. of types
No. of most
frequent type
Index of diversity
(95% CI)
PFGE 19 8 0.944 (0.909-0.979)
MLST ST 14 12 0.886 (0.825-0.948)
flaA 13 6 0.920 (0.895-0.944)
flaB 12 8 0.902 (0.871-0.934)
Concordances
Gold-standard: PFGE
Rand‘s
coefficent
Adjusted
Rand‘s
coefficient:
Wallace‘s
coefficient W1
MLST ST 0.94 0.615 0.979
flaA 0.948 0.588 0.75
flaB 0.958 0.706 1.0
Result:
“In conclusion, PFGE remains the most discriminatory
typing method for Campylobacter. However, flaB typing is
a rapid, reproducible, discriminatory, and stable screening
tool. ”
Calculation helper
http://www.ridom.de/epicompare/
Ridom EpiCompare (free software)
Calculation helper
http://darwin.phyloviz.net/ComparingPartitions/index.php?link=Home
Comparing partitions (free webpage)
Summary
Evaluation of
Molecular Typing Techniques
Image Credits
Biohazard by Szczur http://www.flickr.com/photos/szczur/54101979/
Tips by PlaxcoLab: http://www.flickr.com/photos/34857812@N04/3235836272/
Blackboard by ©Miss cicicola.Z:
http://www.flickr.com/photos/cicicola-zy/1430894423/
aboutpixel.de / postkasten...damals © mpdrei master
aboutpixel.de / laptop © Peter Ehmann
aboutpixel.de / Reagiernder Bürgermeister von Berlin. © Andi Streidl
aboutpixel.de / notizen 04 © Bernd Boscolo
aboutpixel.de / Kleingeld © daylight
aboutpixel.de / Busstation © Konstantin Gastmann
aboutpixel.de / Pinwand © Alexander Kreher
Bildquelle: aboutpixel.de / Blutplatte © Ute Pelz
we*ge / photocase.com
Some images from www.morguefile.com
Thomas Weniger
University of Münster, Germany
tweniger@uni-muenster.de

Evaluation of Moleculare Typing Techniques

Editor's Notes

  • #3 Typing: Analysis of bacterial isolates below species level, performed to generate strain-specific fingerprints. To put it simply, typing applies distinct labels to bacterial isolates. Fingerprint: A specific pattern displayed by an isolate on application of one or more typing methods. see van Belkum et al., (2007). CMI, 13:1
  • #4 Example of genotyping methods for Campylobacter jejuni: -Pulsed field gel electrophoresis (PFGE) -Multi Locus Sequence Typing (MLST) -flaA typing (Single Locus Sequence Typing, SLST) How can the performance of different typing method be measured? Note that different investigations may have different requirements, therefore there is no ideal, universally applicable bacterial typing method.
  • #5 An example slide that compares different typing methods. Author of slide: Alexander Mellmann
  • #6 Evaluation and validation can be done by various criteria. These can be divided into performance and convenience criteria.
  • #7 Typability: a method’s ability to assign a type to all isolates tested by it. - Typability should be high (100%) - Can be lower for phenotyping methods, e.g. serotyping (schemes may not cover all genotypes)
  • #8 An isolate’s fingerprint should - not change rapidly - based on the epidemiological context - results should remain stable during laboratory storage and subculture - multiple subcultures of same isolate > store under same conditions for different periods > same results? - stable during study period - depending on species - not to be confused with reproducibility
  • #9 - The ability of a typing method to assign - the same type to an isolate tested on independent occasions, - separated in time and/or - Place - Reproducibility may be influenced by - the protocol used - stringency of its application. - analysis and interpretation of results. - Reproducibility has intra-laboratory and inter-laboratory dimensions. - should be high (100%) -> Use standardized protocols and adequate personnel training to ensure a reliable method
  • #10 - a method’s ability to assign a different type to two unrelated strains sampled randomly from the population of a given species. - should be calculated using a test population that includes epidemiologically unrelated strains Discriminatory power can be expressed as a probability using Simpson‘s index of diversity - Simpson’s index of diversity should be 95% to 100% (according to van Belkum) - Do critical assessment of the confidence interval see van Belkum et al., (2007). CMI, 13:1
  • #11 Example DI for typing method 1 is 0.933 [0.805 – 1.0] compared to typing method 2: DI: 0.733 [0.53 – 0.936] => method 1 is more discriminatory
  • #12 How well does the typing method fit into the epidemiological context? The results of a typing method should reflect and agree with the available epidemiological information For example, epidemiologically related isolates derived from single-strain outbreaks should be assigned to identical or related types.
  • #13 A good typing method should assess a marker that - should be testable in every isolate, i.e., it should provide universal typeability of all isolates. - remains stable during the study period - should be reproducible, independently of the operator, place and time - usefully discriminates among isolates - This discrimination should be concordant with the epidemiological picture
  • #14 Cost: - initial equipment (e.g. sequencer) - costs of servicing - cost of consumable reagents - staff costs (work time, training time) Rapidity: - time required to get from the bacterial isolates to the final typing results - faster if method can be applied directly on clinical material Ease of use: - technical simplicity - suitability for processing large numbers of isolates - ease of interpreting the results - amenability to computerised analysis - ability to incorporate typing results in electronic databases
  • #15 Compare the result of two typing systems. All these coefficients are calculated based on how pairs of isolates are grouped in both compared typing methods see Carrico et al. (2006) JCM 44: 2524
  • #16 Rand’s Coefficient - Rands’s coefficient: proportion of agreement for both matches (a) and Mismatches - Rand’s coefficient is not zero for random data sets -> Adjusted rand addresses this issue and corrects for change agreement. see Rand (1971) J. Am. Stat. Assoc. 66 Hubert & Arabie (1985) J. Classification 2
  • #17 Wallace coefficients. Wallace’s coefficients W1 and W2 - Assume that a typing method is the standard. - Probability that a pair of points which are in the same cluster under P are also in the same cluster under P’ (and vice versa) - Provide an estimate of how much new information is obtained from another typing method. - High values indicate that partitions defined by a given method could have been predicted from the results of another method, suggesting that the use of both methodologies is redundant: see Wallace (1983) J. Am. Stat. Assoc. 78
  • #18 Adjusted Rand‘s is low, and W1 is high-> method 2 adds no new information
  • #19 Be careful with missing data! Do NOT use “no category”, “NT” or negative values. Use empty space instead. Otherwise, the software might think that “no category” or “NT” is an own category.
  • #20 Example from Mellmann et al. 2004 see Mellmann et al. (2004). JCM 42: 4840