The PGMY L1 consensus primer pair combined with the line blot assay allows the detection of 27 genital human papillomavirus (HPV) genotypes. We conducted an intralaboratory and interlaboratory agreement study to assess the accuracy and reproducibility of PCR for HPV DNA detection and typing using the PGMY primers and typing amplicons with the line blot (PGMY-LB) assay. A test panel of 109 samples consisting of 29 HPV-negative (10 buffer controls and 19 genital samples) and 80 HPV-positive samples (60 genital samples and 20 controls with small or large amounts of HPV DNA plasmids) were tested blindly in triplicate by three laboratories. Intralaboratory agreement ranged from 86 to 98% for HPV DNA detection. PGMY-LB assay results for samples with a low copy number of HPV DNA were less reproducible. The rate of intralaboratory agreement excluding negative results for HPV typing ranged from 78 to 96%. Interlaboratory reliability for HPV DNA positivity and HPV typing was very good, with levels of agreement of >95% and kappa values of >0.87. Again, low-copy-number samples were more prone to generating discrepant results. The accuracy varied from 91 to 100% for HPV DNA positivity and from 90 to 100% for HPV typing. HPV testing can thus be accomplished reliably with PCR by using a standardized written protocol and quality-controlled reagents. The use of validated HPV DNA detection and typing assays demonstrating excellent interlaboratory agreement will allow investigators to better compare results between epidemiological studies.

1. JOURNAL OF CLINICAL MICROBIOLOGY, Mar. 2003, p. 1080–1086 Vol. 41, No. 3
0095-1137/03/$08.00 0 DOI: 10.1128/JCM.41.3.1080–1086.2003
Copyright © 2003, American Society for Microbiology. All Rights Reserved.
International Proficiency Study of a Consensus L1 PCR Assay for the
Detection and Typing of Human Papillomavirus DNA: Evaluation
of Accuracy and Intralaboratory and Interlaboratory Agreement
Janet R. Kornegay,1 Michel Roger,2,3 Philip O. Davies,4 Amanda P. Shepard,1 Nayana A. Guerrero,5
Belen Lloveras,5 Darren Evans,4 and Francois Coutlee2,3*
¸ ´
Roche Molecular Systems Inc., Alameda, California1; Departement de Microbiologie et Infectiologie, Hopital Notre-Dame du
´ ˆ
Centre Hospitalier de l’Universite de Montreal,2 and Departement de Microbiologie et Immunologie, Universite de
´ ´ ´ ´
Montreal,3 Montreal, Quebec, Canada; BMI, London, United Kingdom4; and Department of Pathology,
´ ´ ´
CSU Bellvitge/Lab. Recerca Translacional, Institut Catala d’Oncologia,
Barcelona, Spain5
Received 24 May 2002/Returned for modification 26 August 2002/Accepted 9 December 2002
The PGMY L1 consensus primer pair combined with the line blot assay allows the detection of 27 genital
human papillomavirus (HPV) genotypes. We conducted an intralaboratory and interlaboratory agreement
study to assess the accuracy and reproducibility of PCR for HPV DNA detection and typing using the PGMY
primers and typing amplicons with the line blot (PGMY-LB) assay. A test panel of 109 samples consisting of
29 HPV-negative (10 buffer controls and 19 genital samples) and 80 HPV-positive samples (60 genital samples
and 20 controls with small or large amounts of HPV DNA plasmids) were tested blindly in triplicate by three
laboratories. Intralaboratory agreement ranged from 86 to 98% for HPV DNA detection. PGMY-LB assay
results for samples with a low copy number of HPV DNA were less reproducible. The rate of intralaboratory
agreement excluding negative results for HPV typing ranged from 78 to 96%. Interlaboratory reliability for
HPV DNA positivity and HPV typing was very good, with levels of agreement of >95% and kappa values of
>0.87. Again, low-copy-number samples were more prone to generating discrepant results. The accuracy varied
from 91 to 100% for HPV DNA positivity and from 90 to 100% for HPV typing. HPV testing can thus be
accomplished reliably with PCR by using a standardized written protocol and quality-controlled reagents. The
use of validated HPV DNA detection and typing assays demonstrating excellent interlaboratory agreement will
allow investigators to better compare results between epidemiological studies.
Human papillomavirus (HPV) infection is a strong indepen- a nonisotopic reverse hybridization assay, the line blot (LB)
dent predictor for squamous intraepithelial lesions (SIL) and assay (4, 13). Recently, a colorimetric microtiter plate-based
invasive cancer of the uterine cervix (10, 34). Women with enzyme immunoassay was also reported for screening of the
persistent HPV infection are at highest risk for development of broad spectrum of HPV amplified by the PGMY primers using
cervical SIL or evolution from low-grade SIL to higher-grade a generic probe mix (21).
disease of the uterine cervix (8, 24). The causal association The proficiency of microbiology laboratory testing is gener-
between HPV and cervical cancer was demonstrated by epide- ally monitored by proficiency-testing programs that also allow
miological investigations that used PCR, the most sensitive the determination of test variability between laboratories (32).
tool for HPV detection and typing, in cells collected from the Implementation of proficiency testing panels is essential for
uterine cervix (5). unregulated molecular diagnostic tests. Proficiency panels have
Because of the genetic polymorphism of HPV, consensus been developed for the molecular diagnosis of several infec-
PCR assays have been utilized to amplify in one reaction the tious agents (25, 32) and in research settings to monitor the
majority of known, as well as novel, anogenital HPV geno- performance of molecular virology laboratories (17). Thus far,
types. The most widely used primer sets, the MY09/MY11/ no proficiency panel has been constructed and made available
HMB01, GP5 /GP6 , PGMY09/PGMY11, and SPF1/SPF2 to the general research community for HPV testing. The va-
consensus primers, target conserved sequences in the HPV L1 lidity of HPV DNA detection and typing with PCR assays has
gene (1, 7, 12, 15, 18, 20, 23). To render the consensus PCR not been thoroughly assessed.
assays more feasible for large-scale testing and clinical appli- Epidemiological studies and vaccine clinical trials require
cation, convenient assays for the detection and typing of HPV the reliable and reproducible identification of genital HPV
have been developed for all primer sets. HPV amplicons gen- infection. Several studies have evaluated the intermethod vari-
erated by PGMY primers can be easily detected and typed by ation of HPV DNA detection (2, 11, 14, 19, 22, 26, 28, 30, 33).
However, few studies have evaluated the intralaboratory and
interlaboratory reproducibility of L1 consensus PCR assays
* Corresponding author. Mailing address: Departement de Micro-
´ although these assays have been widely used (6, 16, 19). The
biologie et Infectiologie, Hopital Notre-Dame du Centre Hospitalier
ˆ
de l’Universite de Montreal, 1560 Sherbrooke est, Montreal, Quebec
´ ´ ´ ´
latter studies were conducted with in-house reagents and pro-
H2L 4M1, Canada. Phone: (514) 890-8000, ext. 25162. Fax: (514) tocols. Although the consensus L1 PCR assays are not com-
412-7512. E-mail: francois.coutlee@ssss.gouv.qc.ca. mercially available, standardized reagents and protocols for
1080

2. VOL. 41, 2003 INTERLABORATORY STUDY OF A CONSENSUS L1 PCR FOR HPV 1081
the PGMY-LB assay are available from Roche Molecular Sys- but unfamiliar with the PGMY-LB assay. Standardized reagents comprising the
tems for research purposes. The use of standardized and re- PCR master mix, LB strips, and reagents were sent to each participating center
along with a written standard operating procedure for the PGMY-LB assay.
producible protocols for HPV detection and typing could fa- Each participating center was asked to follow the protocol without modification.
cilitate the comparison of results between studies on HPV Three 20- l aliquots from each panel sample were coded and distributed on
infection. dry ice by Roche Molecular Systems to each of the three independent labora-
In order to assess the accuracy and reproducibility of the tories. In the test panel, the HPV-negative or buffer controls were distributed
randomly among the HPV-positive specimens. Five microliters of each sample
PGMY-LB assay, three laboratories were invited to participate
was used for HPV testing with the PGMY-LB assay. Each laboratory tested, in
in a collaborative study to compare their abilities to detect the different PCR runs, each of the 109 samples three times, blinded to the results
presence of and to genotype HPV DNA in blinded specimens. obtained from Roche Molecular Systems, from the other laboratories, and from
We report here the estimate of intralaboratory and interlabo- previous runs. Laboratories A and B also tested PGMY amplicons with the
ratory reproducibility of the PGMY-LB assay for HPV DNA generic probe microplate assay. The study testing was conducted between June
and August 2000.
detection and typing on 109 specimens tested in triplicate by PGMY-LB assay. HPV DNA was amplified in each center under standard
three independent laboratories under standardized conditions. conditions with the L1 consensus HPV PGMY09/PGMY11 primer set, as pre-
This information is useful in view of the wider use of the viously described (12). The amplification mixture contained 4 mM MgCl2, 50
PGMY-LB assay by numerous research groups, as well as its mM KCl, 7.5 U of AmpliTaq Gold DNA polymerase (Applied Biosystems), 200
potential application in diagnostic laboratories for HPV detec- M concentrations each of dATP, dCTP, and dGTP, 600 M dUTP, 100 pmol
of each biotinylated PGMY primer pool, and 5 pmol each of the 5 -biotinylated
tion and typing. -globin primers GH20 and PC04. HPV was amplified with the ultrasensitive
profile that consisted of activation of AmpliTaq Gold at 95°C for 9 min, dena-
turation for 1 min at 95°C, annealing for 1 min at 55°C, and extension at 72°C for
MATERIALS AND METHODS 1 min for a total of 40 cycles. Amplification was followed by a 5-min terminal
Selection of samples for the test panel. Roche Molecular Systems established extension step at 72°C. Laboratory A used a Biometra Uno II thermocycler,
the test panel of samples with the PGMY-LB assay. The quality control set laboratory B used a TC 9600 thermocycler, and laboratory C used an MJ Re-
included 30 synthetic samples: negative buffer samples devoid of HPV DNA search PTC-1 thermocycler. Measures to avoid false-positive reactions due to
sequences (n 10), samples containing 0.063 g of human genomic DNA per 5 contamination were strictly adhered to. HPV genotyping was performed with the
l (catalog no. 1691112; Roche Molecular Biochemicals, Indianapolis, Ind.) reverse LB detection system as previously described (13). PCR products were
spiked with 4,000 copies of HPV-16 DNA plasmid per 5 l (n 5), human DNA denatured in 0.4 N NaOH and hybridized to an immobilized probe array con-
spiked with 400 copies of HPV-16 DNA plasmid per 5 l (n 5), human DNA taining probes for 27 HPV genotypes (types 6, 11, 16, 18, 26, 31, 33, 35, 39, 40,
spiked with 4,000 copies of HPV-45 DNA plasmid per 5 l (n 5), and human 42, 45, 51, 52, 53, 54, 55, 56, 57, 58, 59, 66, 68, 73, 82, 83, and 84). Positive
DNA spiked with 400 copies of HPV-45 DNA plasmid per 5 l (n 5). The hybridization was detected by streptavidin-horseradish peroxidase-mediated
panel also included 79 anonymous cervical samples collected with cytobrushes color precipitation on the membrane at the probe line.
from 79 women attending clinics at the BMI hospitals in London. The latter Generic probe microplate assay. Generic probe detection reactions were per-
samples came from a set of 90 samples that had been screened initially with the formed by using reagents from a PCR-ELISA DIG detection kit (Roche Mo-
PGMY-LB assay at the BMI Health Services Laboratory and stored frozen. lecular Biochemicals) as described previously (21). Twenty microliters of dena-
Roche Molecular Systems retested with the PGMY-LB assay the 90 clinical tured PCR products was added to the streptavidin-coated microtiter wells,
samples in duplicate to ensure the presence of HPV genotypes detected initially. followed by the addition of 200 l of hybridization buffer provided by Roche and
Confirmation of initial PCR results was not obtained for 11 of the 90 samples (8 20 l of the denatured generic probe pool. Roche synthesized the digoxigenin-
generated discordant results between PCR runs, 2 had been mislabeled, and 1 labeled generic probe pool by amplification of DNA from HPV types 11, 16, 18,
was only tested once at Roche), leaving 79 clinical samples (19 HPV negative and and 51 as described previously (21). Hybridization was performed at 37°C for 1 h.
60 HPV positive) in the panel. Following color development, absorbance was measured at 405 nm and the
Of the 109 samples included in the proficiency-testing panel, 29 were HPV- background, defined as the average value of blank cells containing no PCR
negative specimens (10 buffer controls and 19 genital samples) and 80 samples product, was subtracted from all values. A specimen was considered positive if
contained HPV DNA. The samples containing HPV DNA included 47 genital the corrected A405 was greater than 0.5, negative if the value was less than 0.2,
samples with one HPV type, 10 synthetic samples with low HPV DNA copy and indeterminate in the range between 0.2 and 0.499.
numbers, 10 synthetic samples with high HPV DNA copy numbers, 3 synthetic Statistical methods. To ensure an independent evaluation of the reproduc-
samples containing multiple HPV types, and 10 genital samples containing, in ibility of the assay, statistical analyses were performed by a scientist (F.C.)
addition to the genotype(s) consistently detected in repeated PGMY-LB assay without financial interests in Roche Molecular Systems. Results from the three
runs, at least one type that was not consistently detected. The latter 10 samples laboratories were imported into a common database for comparison (Microsoft
were tested four times by Roche Molecular Systems: an additional type was Excel). Results obtained by Roche Molecular Systems (the reference laboratory)
detected in one out of four runs (two samples), two out of four runs (seven were considered the “gold standard” (HPV DNA reference standard). Agree-
samples), or three out of four runs (two samples). One sample contained two ment for overall HPV positivity (HPV DNA positive irrespective of types iden-
HPV types that were inconsistently detected in the four runs. HPV genotypes tified) and for type-specific positivity was calculated as the percentage of runs
that were inconsistently detected in these 10 samples included types 42 (in three with identical results for the presence or absence of HPV. Cohen’s unweighted
samples), 31 and 58 (in two samples each), and 16, 35, 33, and 84 (in one sample kappa statistic was calculated to adjust for chance agreement between sites or
each). between sites and HPV DNA reference standard results (9). In general, a kappa
In addition to the 10 samples containing HPV-16 plasmids and 10 samples value of 0.75 indicates excellent agreement beyond chance, a kappa value
containing HPV-45 plasmids, there were a variety of HPV genotypes represented between 0.40 and 0.75 indicates fair to good agreement, and a kappa value of
among the 60 HPV-positive samples. They included types 16 (in 10 samples), 18 0.40 represents poor agreement.
and 51 (in 5 samples each), 31, 45, 52, 59, and 66 (in 4 samples each), 33, 39, 42, The reproducibility of repeated PCR assays (intralaboratory reproducibility)
and 54 (in 3 samples each), 56, 68, 83, and 6 (in 2 samples each), and 35, 53, 58, was calculated for each site by comparing results from triplicate runs and calcu-
82, 84, and 73 (in 1 sample each). Of the three samples with multiple types, one lating the crude percent agreement and the percent agreement considering only
contained types 16 and 31, one contained types 16, 18, 52, and 73, and another HPV-positive results. The intralaboratory reproducibility for the presence or
one contained types 16, 35, 42, and 82. The results from the detection of -globin absence of each of 27 genotypes was first calculated by using crude typing results.
were not considered and compared in this work. Since 10 samples contained HPV types that were not consistently detected
Study design. Three laboratories with different levels of experience with mo- between runs by the reference laboratory, agreement was recalculated by not
lecular diagnostic methods participated in the study. Laboratory A had experi- considering the presence or absence of these additional types (see above).
ence in human genetic testing but was unfamiliar with molecular virology tests. Interlaboratory agreement (interlaboratory reproducibility) was assessed by
Laboratory B was a diagnostic molecular microbiology laboratory specifically pairwise comparisons of test results from the three laboratories calculating the
trained to perform the PGMY-LB assay. Laboratory C was a diagnostic cytopa- crude percent agreement and the kappa statistic. Results obtained by the refer-
thology laboratory familiar with commercialized molecular diagnostic techniques ence laboratory were not considered in these comparisons. To avoid combining

3. 1082 KORNEGAY ET AL. J. CLIN. MICROBIOL.
TABLE 1. Intralaboratory reproducibility of PGMY-LB for HPV DNA detection and HPV typing on triplicate testing of
109 samples by three laboratories
No. of assays with following results in runs 1/2/3a: Agreement (%)b
Test and laboratory HPV
/ / / / / / / / All results
results
HPV DNA detectionc
A 65 11 4 29 86.2 81.3
B 77 3 0 29 97.3 96.3
C 78 2 0 29 98.2 97.5
HPV typingd
A 69 17 (14)e 6 (5)e 2,858 99.2 (99.4)e 75.0 (78.4)e
B 90 5 (3)e 3 (1)e 2,845 99.7 (99.9)e 91.8 (95.8)e
C 84 8 (2)e 6 (5)e 2,845 99.5 (99.8)e 85.7 (92.3)e
a
Triplicate testing of 109 samples was done by each laboratory.
b
Two measurements of percent agreement were calculated: one in which HPV-positive and HPV-negative results were considered (all results) and one in which
specimens that had negative results in all triplicates were excluded.
c
Results for the detection of HPV DNA irrespective of types detected.
d
Typing results for 27 genotypes were considered.
e
In parentheses are numbers of assays not taking into consideration types that were inconsistently detected by the reference laboratory.
intralaboratory and interlaboratory variability in the comparisons of the three of discordant triplicates between laboratories B and C did not
laboratories, two strategies of analysis were considered. Laboratories were first reach statistical significance (P 0.65; Pearson chi-square
compared by considering the HPV DNA and typing results obtained on the
initial run for each sample in each laboratory. This strategy allows evaluation of
test).
interlaboratory reproducibility when samples are only tested once, as is often the An intralaboratory agreement analysis stratified by the
case. Laboratories were also compared by using a consensus definition of HPV amount of HPV DNA introduced in the sample for laboratory
results: the final HPV result for a specimen was that obtained from at least two A that had the highest number of discordant triplicates re-
out of three runs for the presence of HPV DNA and HPV typing. This definition
vealed that the presence of a low copy number of HPV DNA
favors a higher degree of concordance between laboratories but takes into
account triplicate results for each sample. had an influence on reproducibility. In this laboratory, 7
Accuracy of HPV DNA detection and HPV typing was assessed by pairwise (70.0%) of the 10 samples containing small amounts of HPV
comparisons in contingency tables of each laboratory with the HPV DNA ref- DNA generated discordant triplicates for HPV DNA detec-
erence standard results, using the two-sided McNemar chi-square analysis for tion, as opposed to 8 of the 70 samples with clear signals in the
matched-pair data. Agreement was determined first by considering results of the
first aliquot for each sample and then by considering the consensus HPV results
PGMY-LB assay or high copy numbers (P 0.002; Fisher’s
for each sample as explained above. Since the reference laboratory did not exact test).
consistently detect a type in 10 samples, agreement was recalculated by not In order to estimate the intralaboratory agreement for HPV
considering the presence or absence of these additional types as a discordant DNA typing, we compared the results of triplicate testing of
result. Results from laboratories A and B for the generic probe microplate assay
109 samples for 27 genotypes (2,943 type-specific results per
were compared for HPV DNA positivity with results obtained with the LB assay.
Indeterminate results were excluded from the calculation of agreement. center) (Table 1). Although the number of discordant tripli-
cates was greater for typing results than HPV DNA detection
in all laboratories, agreement reached at least 99% when all
RESULTS
specimens were considered (Table 1). The high number of
Results from triplicate runs with the PGMY-LB assay were HPV type-specific negative results explains the greater intral-
compared in each laboratory to first assess the reproducibility aboratory agreement obtained with HPV typing results than
of HPV DNA detection irrespective of the type(s) detected. with generic HPV DNA detection despite a greater number of
The intralaboratory agreement for HPV DNA detection of the discordant triplicates. When concordant negative triplicates
PGMY-LB assay was high for each site (Table 1). The agree- were excluded from the analysis, the level of agreement de-
ment between triplicates for each center was lower when only creased as shown in Table 1. We then considered in our eval-
the HPV-positive results were considered. Two laboratories uation the fact that 10 samples contained HPV types that were
had intralaboratory levels of agreement of 97% for HPV not consistently detected between runs by the reference labo-
DNA detection when all results were considered. A significant ratory. In laboratory C, 7 (50%) of the 14 discordant triplicates
difference was found in the number of discordant triplicates for contained HPV types not uniformly detected by the reference
HPV DNA detection among the three participating laborato- laboratory, thus leaving only 7 truly discordant triplicates for
ries (Table 1): 15 (13.8%), 3 (2.8%), and 2 (1.8%) of 109 HPV typing (values in parentheses in Table 1). In laboratory B,
samples generated discordant triplicates for HPV DNA in four (50%) of the eight discordant triplicates contained HPV
laboratories A, B, and C, respectively (P 0.005; Pearson types not uniformly detected by the reference laboratory, leav-
chi-square test). When laboratories were compared two by ing only four truly discordant triplicates for HPV typing. In
two, significant differences were found, after the Bonferroni laboratory A, 4 (17.3%) of the 23 discordant triplicates con-
correction for a total of three comparisons, in the proportion tained HPV types not uniformly detected by the reference
of discordant triplicates between laboratories A and B and laboratory, leaving 19 truly discordant triplicates for HPV typ-
between laboratories A and C (P 0.009 and 0.003, respec- ing. When only truly discordant triplicates were considered,
tively; Pearson chi-square test). The difference in the number the intralaboratory agreement for HPV typing increased

4. VOL. 41, 2003 INTERLABORATORY STUDY OF A CONSENSUS L1 PCR FOR HPV 1083
TABLE 2. Interlaboratory agreement of PGMY-LB results for TABLE 3. Interlaboratory agreement of PGMY-LB results for
HPV DNA detection and HPV typing obtained on initial testing of HPV DNA detection and HPV typing obtained in three laboratories
109 samples by three laboratories on triplicate testings of 109 samples using the consensus definition
for HPV results
No. of concordant
results % Agreement No. of concordant
Test and Kappa
(no. identical/ results % Agreement
laboratory pair HPV HPV value Test and Kappa
no. tested) (no. identical/
(n 80) (n 29) laboratory pair HPV HPV value
no. tested)
(n 80) (n 29)
HPV DNA
detection HPV DNA
A-B 73 29 94 (102/109) 0.85 detection
B-C 79 29 99 (108/109) 0.98 A-B 76 29 96 (105/109) 0.91
A-C 72 29 93 (101/109) 0.83 B-C 80 29 100 (109/109) 1.00
A-C 76 29 96 (105/109) 0.91
HPV typing
A-B 68 29 95 (97/109) 0.75 HPV typing
B-C 71 29 93 (101/109) 0.81 A-B 75 29 95 (104/109) 0.87
A-C 69 29 90 (98/109) 0.78 B-C 80 29 100 (109/109) 1.00
A-C 76 29 96 (105/109) 0.91
slightly when all results were included in the calculation of
agreement and ranged from 78.4 to 95.8% when only HPV- values of 0.87, including perfect kappa values for HPV DNA
positive results were used (Table 1). detection and HPV typing between laboratories B and C. Of
When the numbers of truly discordant triplicates for HPV the four samples testing negative in laboratory A but positive
DNA typing were compared between participating sites, sig- in the two other laboratories, two contained low copy numbers
nificant differences were found, after the Bonferroni correction of HPV-16 or -45. Low-copy-number samples were more
for a total of three comparisons, between laboratories A and B prone to generate discordant results in laboratory A, since 2
and between laboratories A and C (P 0.006 and 0.054, (20%) of 10 low-copy-number samples versus 2 (3%) of 70
respectively; Pearson chi-square test). Considering typing re- other HPV-positive samples were misclassified by laboratory A
sults from laboratory A that showed the greatest intralabora- (P 0.07; Fisher’s exact test).
tory variability, 8 of 10 samples with low copy numbers of HPV The accuracy of the PGMY-LB assay was measured by com-
DNA generated discordant triplicates, as opposed to 15 of 70 paring results obtained from each laboratory with those ob-
samples with clear signals or high HPV copy numbers (P tained at Roche Molecular Systems (HPV DNA reference
0.005; Fisher’s exact test). This relationship was not found for standard) for 109 samples. Additional HPV types inconsis-
laboratories B and C, which had a higher level of agreement tently detected by Roche in multiple testing in 10 samples were
between triplicates (data not shown). Excluding specimens not considered in this analysis. For the 29 HPV-negative sam-
with low copy numbers of HPV types 45 and 16, discordant ples, only 2 (0.8%) of 261 assays scored positive for HPV
triplicates were distributed equally across several types. Repro- DNA. One specimen that tested falsely positive for HPV-51
ducibility was perfect for the three samples with multiple HPV was preceded by an HPV-51-positive sample, while HPV-31
type infections for two laboratories (data not shown). One was responsible for the other false-positive result. Since sam-
laboratory failed to detect an HPV type in one of the triplicates ples were tested in triplicate, results from the first run and from
for two of the latter samples (data not shown). a consensus definition of HPV positivity were compared sep-
Interlaboratory reproducibility of the PGMY-LB assay was arately to the HPV DNA reference standard results, as shown
first assessed by considering only the first PCR run in each in Table 4. The percentages of agreement between each labo-
laboratory. Laboratory pairwise comparisons are shown in Ta- ratory and the HPV DNA reference standard varied from 91 to
ble 2 for HPV DNA detection and typing. Agreement for HPV 100% for HPV DNA positivity and from 90 to 100% for HPV
DNA positivity between laboratory pairs A and B, B and C, typing. Laboratory B correctly identified all HPV-positive sam-
and A and C reached 94% (102 of 109 samples, kappa value of ples using results from the first assay or the consensus defini-
0.85), 99% (108 of 109 samples, kappa value of 0.98), and 93% tion of HPV positivity. HPV types were identified correctly in
(101 of 109 samples, kappa value of 0.83), respectively. Agree- all samples when the consensus definition of HPV positivity
ment for HPV typing results between laboratory pairs A and B, was used and in 79 (98.8%) samples when the first run only was
B and C, and A and C reached 94% (102 of 109 samples, kappa considered. Similarly, laboratory C correctly identified 99% of
value of 0.81), 97% (106 of 109 samples, kappa value of 0.88), HPV-positive samples and correctly identified HPV types for
and 93% (101 of 109 samples, kappa value of 0.80), respec- nearly all samples except three (3.8%) and one (1.3%) when
tively. the first run and the consensus definition of HPV positivity
Interlaboratory reproducibility of the PGMY-LB assay was were used, respectively. When the first sample was considered,
then assessed by using a consensus definition of HPV positivity laboratory A failed to identify 7 (9%) of 80 HPV-positive
(two of three runs with identical results), as explained in Ma- samples and failed in HPV typing for 8 (10%) for 80 HPV-
terials and Methods. Laboratory pairwise comparisons are positive samples. Four (57.1%) of the seven HPV-positive
shown in Table 3 for HPV DNA detection and typing. Inter- samples that were misclassified by laboratory A in the first run
laboratory reliability for HPV DNA positivity and HPV typing contained low copy numbers of HPV DNA (data not shown).
was excellent, with levels of agreement of 95% and kappa The agreement for HPV-positive samples, which can be con-

5. 1084 KORNEGAY ET AL. J. CLIN. MICROBIOL.
TABLE 4. Accuracy of HPV DNA detection and typing results obtained with PGMY-LB for 109 samples by three laboratories
No. of correct PGMY-LB results (% accuracy)
Laboratory and HPV
No. of samples HPV DNA detection HPV typing
standard
1st assay Consensus 1st assay Consensus
A
HPV positive 80 73 (91) 76 (95) 72 (90) 76 (95)
HPV negative 29 29 (100) 29 (100) 29 (100) 29 (100)
All samples 109 102 (94) 105 (96) 101 (93) 105 (96)
B
HPV positive 80 80 (100) 80 (100) 79 (99) 80 (100)
HPV negative 29 29 (100) 29 (100) 29 (100) 29 (100)
All samples 109 109 (100) 109 (100) 108 (99) 109 (100)
C
HPV positive 80 79 (99) 80 (100) 77 (96) 79 (99)
HPV negative 29 29 (100) 29 (100) 29 (100) 29 (100)
All samples 109 108 (99) 109 (100) 106 (97) 108 (99)
sidered the sensitivity of the PGMY-LB assay compared to the required higher concentrations of -globin primers in the PCR
HPV DNA reference standard, ranged from 95 to 100% for master mix (data not shown). It was demonstrated previously
HPV DNA detection and typing using the consensus definition that coamplification of -globin with HPV DNA can result in
of HPV results and from 90 to 100% using results from the first competition and impede PGMY-LB assay performance, ex-
test (Table 4). Samples containing multiple HPV types or with plaining in part the discordant results obtained with low-copy-
high copy numbers of HPV DNA were all appropriately clas- number synthetic samples (3, 4). The importance of utilizing
sified by each site (data not shown). the same instruments and parameters set by validation studies
In laboratories A and B, amplicons generated by PGMY of a test is emphasized by these results. Each of the three
primers were tested with the LB assay and also with the generic laboratories used a different make and model of thermocycler,
probe microplate assay. Since two laboratories tested 109 sam- while the PGMY-LB assay protocol was developed and opti-
ples in triplicate, 654 assay results could be compared. Inde- mized for only one, the TC 9600 from Perkin-Elmer. As an
terminate results for the generic probe microplate assay were alternative explanation, specimen heterogeneity may have ac-
obtained for 59 samples (30 HPV-positive samples and 29 counted for the difference in performance between laborato-
HPV-negative samples). When these indeterminate results ries.
were excluded, 420 assays were HPV positive by both tests, 149 We demonstrated on synthetic samples that specimens with
were negative by both tests, 11 were HPV positive with the small amounts of HPV DNA were prone to generate discor-
generic probe microplate assay only, and 15 were HPV-posi- dant results between triplicates. Since synthetic samples do not
tive with the LB assay only. The agreement between the tests contain the amplification modifier found in clinical specimens,
for detection of HPV amplicons was 95.6% (569 of 595 re- we also included in the proficiency panel anonymous cytobrush
sults), for an excellent kappa value of 0.89. samples. Ten clinical samples contained at least one HPV type
not consistently detected by the reference laboratory. For all of
these samples, the type detected intermittently by the refer-
DISCUSSION
ence laboratory was also detected by at least one of the par-
Interlaboratory agreement is a concern for any HPV DNA ticipating laboratories. This phenomenon could result from the
detection method that has not undergone extensive compara- presence of a low copy number of HPV DNA or inhibitors and
tive field testing. The HPV detection and typing method also illustrates the difficulty of using clinical samples in a proficiency
has to be accurate. We evaluated the intralaboratory and in- test.
terlaboratory reproducibility of the PGMY-LB assay per- The intralaboratory agreement was excellent for all three
formed by three laboratories with different levels of experience laboratories. It increased when we did not consider variable
with PCR testing. Our study demonstrates the high reliability HPV results obtained in the certification panel by the refer-
of the PGMY-LB assay when a laboratory is well trained to ence laboratory. Some groups have reported greater reproduc-
perform the assay. Laboratories without experience with in- ibility of PCR for HPV testing (22, 27). In those studies, du-
house PCR tests can also use the PGMY-LB assay, since lab- plicates instead of triplicates were tested, only one laboratory
oratory C generated reproducible and accurate results. The was evaluated, and the panel included more HPV-negative
use of standardized reagents and clearly written protocols was samples than did our study. Our level of intralaboratory agree-
instrumental in obtaining these excellent results with the ment for HPV DNA detection and typing corresponds to a
PGMY-LB assay. The lowest intralaboratory, interlaboratory, previous evaluation by Daniel et al. (6). They also reported
and accuracy levels were found in a laboratory with experience that the reproducibility of HPV typing with MY09/MY11 in-
with PCR testing. That laboratory had determined in previous creased in the presence of greater HPV loads. We found that
experiments that the PGMY-LB assay with their thermocycler for the laboratory with several discordant triplicates, low-viral-

6. VOL. 41, 2003 INTERLABORATORY STUDY OF A CONSENSUS L1 PCR FOR HPV 1085
load synthetic samples were prone to generating discordant ACKNOWLEDGMENTS
results. As reported by others, we found a lower agreement for We thank Diane Gaudreault and France Dion for testing samples
HPV typing than for HPV DNA detection (6, 19). We calcu- with the PGMY-LB assay, Pierre Forest for training participants, and
lated not only overall agreement but also agreement with re- BMI Health Services for the provision of clinical samples. Roche
gard only to HPV-positive results because of the large number Molecular Systems supplied reagents for the PGMY-LB assay.
F.C. and M.R. hold a clinical career award supported by the Fonds
of concordant HPV-negative assays that artificially increased de Recherche en Sante du Quebec.
´ ´
the level of agreement for HPV typing. As previously reported,
agreement was strong for samples with multiple HPV types, REFERENCES
although we only analyzed three such specimens (6, 19). 1. Bauer, H. M., Y. Ting, C. E. Greer, J. C. Chambers, C. J. Tashiro, J. Chi-
Two studies reported the interlaboratory agreement of PCR mera, A. Reingold, and M. M. Manos. 1991. Genital human papillomavirus
infection in female university students as determined by a PCR-based
for HPV detection using the MY09/MY11 primer pair (16, 22). method. JAMA 265:472–477.
In one study, 33 samples were tested in two laboratories, while 2. Cope, J. U., A. Hildesheim, M. H. Schiffman, M. M. Manos, A. T. Lorincz,
in the other study, 70 anal samples were analyzed in two lab- R. D. Burk, A. G. Glass, C. Greer, J. Buckland, K. Helgesen, D. R. Scott,
M. E. Sherman, R. J. Kurman, and K. L. Liaw. 1997. Comparison of the
oratories. Our evaluation of agreement was consistent with the Hybrid Capture tube test and PCR for detection of human papillomavirus
estimates from these previous reports for HPV DNA positivity DNA in cervical specimens. J. Clin. Microbiol. 35:2262–2265.
3. Coutlee, F., P. Gravitt, J. Kornegay, C. Hankins, H. Richardson, N.
´
(83 and 96%) and HPV typing (90 and 88%). Lapointe, H. Voyer, The Canadian Women’s HIV study Group, and E.
The accuracy of the PGMY-LB assay was excellent. How- Franco. 2002. Use of PGMY primers in L1 consensus PCR improves detec-
ever, specimens may better agree with a reference standard tion of human papillomavirus DNA in genital samples. J. Clin. Microbiol.
40:902–907.
when the same test is used to validate and test the panel. The 4. Coutlee, F., P. Gravitt, H. Richardson, C. Hankins, E. Franco, N. Lapointe,
´
low rate of false-positive results in the 29 HPV-negative sam- H. Voyer, and The Canadian Women’s HIV Study Group. 1999. Nonisotopic
ples suggests a low rate of contamination in our study. One of detection and typing of human papillomavirus DNA in genital samples by
the line blot assay. J. Clin. Microbiol. 37:1852–1857.
the false-positive specimens was preceded by an HPV-51-pos- 5. Coutlee, F., M. H. Mayrand, D. Provencher, and E. Franco. 1997. The future
´
itive sample. Contamination may have occurred after amplifi- of HPV testing in clinical laboratories and applied virology research. Clin.
Diagn. Virol. 8:123–141.
cation during the hybridization of amplicons with the LBs, 6. Daniel, R. W., L. Ahdieh, D. Hayden, S. Cu-Uvin, and K. V. Shah. 2000.
since the same amplicons hybridized with the generic probe Intra-laboratory reproducibility of human papillomavirus identification in
tested negative. Another study evaluated the accuracy of the cervical specimens by a polymerase chain reaction-based assay. J. Clin. Virol.
19:187–193.
GP5 /GP6 L1 consensus primer pair and reported excellent 7. de Roda, H., J. M. Walboomers, A. J. van den Brule, C. J. Meijer, and P. J.
accuracy for 50 samples tested in four laboratories, ranging Snijders. 1995. The use of general primers GP5 and GP6 elongated at their
from 92 to 100% for detection of HPV DNA, a rate similar to 3 ends with adjacent highly conserved sequences improves human papillo-
mavirus detection by PCR. J. Gen. Virol. 76:1057–1062.
our results (19). That study also reported that most discrepant 8. Ferenczy, A., and E. L. Franco. 2002. Persistent human papillomavirus in-
results were false-negative results from HPV-positive samples fection and cervical neoplasia. Lancet Oncol. 3:11–16.
9. Fleiss, J. L. 1981. Statistical methods for rates and proportions. John Wiley
(19). The reproducibility of the PGMY-LB assay in our study and Sons Inc., New York, N.Y.
was also similar to that of the FDA-approved hybrid capture 10. Franco, E. L. F. 1996. Epidemiology of anogenital warts and cancer, p.
assay from Digene (29). The accuracy of hybrid capture in 597–623. In R. Reid and A. T. Lorincz (ed.), Human papillomavirus I. W. B.
Saunders, Philadelphia, Pa.
detecting high-risk HPV types ranged for three laboratories 11. Gravitt, P., A. Hakenewerth, and J. Stoerker. 1991. A direct comparison of
from 88 to 92%, and interlaboratory agreement reached 90%. methods proposed for use in widespread screening of human papillomavirus
Nonetheless, our results should be interpreted cautiously, infection. Mol. Cell. Probes 5:65–72.
12. Gravitt, P., C. L. Peyton, T. Q. Alessi, C. Wheeler, F. Coutlee, A. Hildesheim,
´
with the following caveats. Few samples with multiple type M. Schiffman, D. R. Scott, and R. J. Apple. 2000. Improved amplification of
infections were included in our evaluation. These samples have genital human papillomaviruses. J. Clin. Microbiol. 38:357–361.
13. Gravitt, P., C. L. Peyton, R. J. Apple, and C. Wheeler. 1998. Genotyping of
been demonstrated to generate intralaboratory variability in 27 human papillomavirus types by using L1 consensus PCR products by
one study (6). Future evaluations of the PGMY-LB assay single-hybridization, reverse line blot detection method. J. Clin. Microbiol.
should include multiple type infections with some types in 36:3020–3027.
14. Guerrero, E., R. W. Daniel, F. X. Bosch, X. Castellsague, N. Munoz, M. Gili,
´
small amounts. Our clinical samples were obtained without P. Viladu, C. Navarro, M. L. Zubiri, N. Ascunce, L. C. Gonzales, L. Tafur, I.
knowledge of cytopathological diagnoses. Assay reproducibil- Izaraugaza, and K. V. Shah. 1992. Comparison of Virapap, Southern blot
ity and accuracy may differ when samples from different pa- hybridization, and polymerase chain reaction methods for human papillo-
mavirus identification in an epidemiological investigation of cervical cancer.
tient populations are selected, especially since viral load is J. Clin. Microbiol. 30:2951–2959.
related to cervical disease status and assay reproducibility (31). 15. Hildesheim, A., M. H. Schiffman, P. E. Gravitt, A. G. Glass, C. E. Greer, T.
Zhang, D. R. Scott, B. B. Rush, P. Lawler, M. E. Sherman, R. J. Kurman,
We demonstrated here that there is a very strong interlabo- and M. M. Manos. 1994. Persistence of type-specific human papillomavirus
ratory agreement when the PGMY-LB assay is used. HPV infection among cytologically normal women. J. Infect. Dis. 169:235–240.
testing can thus be reliably accomplished with PCR by using a 16. Hsing, A. W., R. D. Burk, K. L. Liaw, C. J. Chen, T. Zhang, M. Schiffman,
C. E. Greer, S. L. You, C. Y. Hsieh, T. W. Huang, T. C. Wu, T. J. O’Leary,
validated and clearly written protocol with quality-controlled J. D. Seidman, and M. M. Manos. 1996. Interlaboratory agreement in a
reagents. Although the assay procedure is well standardized, polymerase chain reaction-based human papillomavirus DNA assay. Cancer
an initial certification panel to ensure the reliability of results Epidemiol. Biomed. Prev. 5:483–484.
17. Jackson, J. B., J. Drew, H. J. Lin, P. Otto, J. W. Bremer, F. B. Hollinger,
would be desirable before a laboratory performs the S. M. Wolinski, The ACTG PCR Working Group, and The ACTG PCR
PGMY-LB assay on a regular basis. Our study stresses the Virology Laboratories. 1993. Establishment of a quality assurance program
for human immunodeficiency virus type 1 DNA polymerase chain reaction
importance of using standardized reagents and protocols and assays by the AIDS clinical trials group. J. Clin. Microbiol. 31:3123–3128.
the establishment of proficiency panels for the comparability of 18. Jacobs, M. V., P. J. F. Snijders, A. J. C. van den Brule, T. J. M. Helmerhorst,
results between studies. More studies on performance stan- C. Meijer, and J. Walboomers. 1997. A general primer GP5 /GP6 -medi-
ated PCR-enzyme immunoassay method for rapid detection of 14 high-risk
dards that include samples with multiple type infections, with and 6 low-risk human papillomavirus genotypes in cervical scrapings. J. Clin.
inhibitors and low viral load infections, need to be performed. Microbiol. 35:791–795.

7. 1086 KORNEGAY ET AL. J. CLIN. MICROBIOL.
19. Jacobs, M. V., P. J. F. Snijders, F. J. Voorhorst, J. Dillner, O. Forslund, B. 26. Peyton, C. L., M. Schiffman, A. T. Lorincz, W. C. Hunt, I. Mielzynska, C.
Johansson, M. V. K. Doeberitz, C. J. L. M. Meijer, T. Meyer, I. Nindl, H. Bratti, S. Eaton, A. Hildesheim, L. A. Morera, A. C. Rodriguez, R. Herrero,
Pfister, E. Stockfleth, A. Strand, G. Wadell, and J. M. M. Walboomers. 1999. M. E. Sherman, and C. M. Wheeler. 1998. Comparison of PCR- and hybrid
Reliable high risk HPV DNA testing by polymerase chain reaction: an capture-based human papillomavirus detection systems using multiple cer-
intermethod and intramethod comparison. J. Clin. Pathol. 52:790. vical specimen collection strategies. J. Clin. Microbiol. 36:3248–3254.
20. Kleter, B., L. J. van Doorn, L. Schrauwen, A. Molijn, S. Sastrowijoto, J. ter 27. Qu, W., G. Jiang, Y. Cruz, C. J. Chang, G. Y. Ho, R. S. Klein, and R. D. Burk.
Schegget, J. Lindeman, B. ter Harmsel, M. Burger, and W. Quint. 1999. 1997. PCR detection of human papillomavirus: comparison between MY09/
Development and clinical evaluation of a highly sensitive PCR-reverse hy- MY11 and GP5 /GP6 primer systems. J. Clin. Microbiol. 35:1304–1310.
bridization line probe assay for detection and identification of anogenital 28. Schiffman, M. H., H. M. Bauer, and A. T. Lorincz. 1991. Comparison of
human papillomavirus. J. Clin. Microbiol. 37:2508–2517. Southern blot hybridization and polymerase chain reaction methods for the
21. Kornegay, J. R., A. P. Shepard, C. Hankins, E. Franco, N. Lapointe, H. detection of human papillomavirus DNA. J. Clin. Microbiol. 29:573–577.
Richardson, The Canadian Women’s HIV Study Group, and F. Coutlee. ´ 29. Schiffman, M. H., N. B. Kiviat, R. D. Burk, K. V. Shah, R. W. Daniel, R.
2001. Nonisotopic detection of human papillomavirus DNA in clinical spec- Lewis, J. Kuypers, M. M. Manos, D. R. Scott, M. E. Sherman, R. J. Kurman,
imens using a consensus PCR and a generic probe mix in an enzyme-linked M. H. Stoler, A. G. Glass, B. B. Rush, I. Mielzynska, and A. T. Lorincz. 1995.
immunoassay format. J. Clin. Microbiol. 39:3530–3536. Accuracy and interlaboratory reliability of human papillomavirus DNA test-
22. Kuypers, J. M., C. W. Critchlow, P. E. Gravitt, D. A. Vernon, J. B. Sayer, ing by hybrid capture. J. Clin. Microbiol. 33:545–550.
M. M. Manos, and N. B. Kiviat. 1993. Comparison of dot filter hybridization, 30. Smits, H. L., L. J. M. Bollen, S. P. Tjong-A-Hung, J. Vonk, J. van der Vendel,
Southern transfer hybridization, and polymerase chain reaction amplification F. J. W. Ten Kate, J. A. Kaan, B. W. Mol, and J. ter Schegget. 1995.
for diagnosis of anal human papillomavirus infection. J. Clin. Microbiol. Intermethod variation in detection of human papillomavirus DNA in cervical
31:1003–1006. smears. J. Clin. Microbiol. 33:2631–2636.
23. Manos, M. M., Y. Ting, D. K. Wright, A. J. Lewis, T. Broker, and S. M. 31. Swan, D. C., R. A. Tucker, G. Tortolero-Luna, M. F. Mitchell, L. Wideroff,
Wolinski. 1989. Use of the polymerase chain reaction amplification for the E. R. Unger, R. A. Nisenbaum, W. C. Reeves, and J. P. Icenogle. 1999.
detection of genital human papillomaviruses. Cancer Cells 7:209–215. Human papillomavirus (HPV) DNA copy number is dependent on grade of
24. Nobbenhuis, M. A., J. M. Walboomers, T. J. Helmerhorst, L. Rozendaal, cervical disease and HPV type. J. Clin. Microbiol. 37:1030–1034.
A. J. Remmink, E. K. Risse, H. C. van der Linden, F. J. Voorhorst, P. 32. Tang, Y. W., and D. H. Persing. 1999. Molecular detection and identification
Kenemans, and C. J. Meijer. 1999. Relation of human papillomavirus status of microorganisms, p. 215–244. In P. R. Murray, E. J. Baron, M. A. Pfaller,
to cervical lesions and consequences for cervical-cancer screening: a pro- F. C. Tenover, and R. H. Yolken (ed.), Manual of clinical microbiology, 7th
spective study. Lancet 354:20–25. ed. American Society for Microbiology, Washington, D.C.
25. Noordhoek, G. T., A. H. Kolk, G. Bjune, D. Catty, J. W. Dale, P. E. Fine, P. 33. Vernon, S. D., E. R. Unger, and D. Williams. 2000. Comparison of human
Godfrey-Faussett, S. N. Cho, T. Shinnick, and S. B. Svenson. 1994. Sensi- papillomavirus detection and typing by cycle sequencing, line blotting, and
tivity and specificity of PCR for detection of Mycobacterium tuberculosis: a Hybrid Capture. J. Clin. Microbiol. 38:651–655.
blind comparison study among seven laboratories. J. Clin. Microbiol. 32:277– 34. Villa, L. L. 1997. Human papillomaviruses and cervical cancer. Adv. Cancer
284. Res. 71:321–341.