Dr. Sherif Omar
Dr. Zaid Al-hamadani PGY3 Family Medicine Resident
Dr. Omnia Darweesh PGY2 Family Medicine Resident
Basic structure for a diagnostic study?
Through the following steps
• Series of patients (of interest)
• New test
• Reference (Gold) standard
• Compare the results of the index test with the
reference standard, blinded
Critical appraisal process for the
diagnostic test
Three easy steps
A/ Are the results valid?
B/ What are the results?
C/ Will they help me care of my patients?
1. Was there a clear question for the study to address?
A question should include information about:
– the population
– the test
– the setting
– the outcomes
A/ Are the results of the study valid?
Population: Sera from 787 consecutive patients tested at the VCT
centre in September-October 2012 were included in the analysis.
Test (Intervention): Rapid tests versus ELISA for HIV screening
Setting: Voluntary Counselling and Testing Facility of a tertiary
care centre in North India.
Outcomes: Our study highlights that RDTs fare poorly compared
to ELISA as screening assays
2. Was there a comparison with an appropriate reference
standard?
HINT: Is this reference test(s) the best available indicator in the
circumstances?
-All 787 sera were tested by RDTs (test in question) and all of
them also tested by and compared to the standard screening
method (ELISA).
-Results were also confirmed by doing Western Blot
(confirmatory gold standard method).
3. Did all patients get the diagnostic test and the reference
standard?
Consider:
– Were both received regardless of the Results of the test of
interest?
– Check the 2 x 2 table
Yes all the population received the two tests in comparison.
4. Could the results of the test of interest have been influenced
by the results of the reference standard?
Consider:
– Was there blinding?
– Were the tests performed independently?
(Review bias)
-The evaluation was conducted in a blinded fashion with the RDTs
and ELISA performed by different technical personnel and the status
of the sera as per the RDT based algorithm not revealed to the
personnel performing the ELISA.
5. Is the disease status of the tested population
clearly described?
Consider:
– Presenting symptoms
– Disease stage or severity
– Co-morbidity
– Differential diagnoses
(Spectrum bias)
-As the aim of the diagnostic tests is screening of HIV,
which is done in an asymptomatic population, so the
answer to these questions is “No”.
-HIV is a silent infection for many years, and thus the aim
of screening is towards people representing the general
population, for example:
•All people coming to live in Qatar regardless of age, sex,
race or ethnicity
•Pre-employment screening tests
•Pre-marital screening tests
•Military enrollment
6. Were the methods for performing the test described in
sufficient detail?
HINT: Was a protocol followed?
-The study itself was performed in three phases:
-The staff were trained and pretest informed consent was taken.
Phase one:
The first RDT (SD Bioline) applied for all sera, then the results were
labeled as reactive or non reactive. Then the reactive tests were
tested further by the two other RDTs.
Phase two:
All sera then were tested by ELISA (standard screening test).
Phase three:
All the positive tests from ELISA and RDTs were compared with
Western Blot for confirmation of results.
7. What are the results?
Consider:
– Are the sensitivity and specificity and/or likelihood ratios
presented?
– Are the results presented in such a way that we can work
them out?
B/ If so, what are the results?
-Thirty-six serum samples were reactive by the first test.
-On subsequent evaluation of all the 787 samples by Microlisa-
HIV, 40 HIV reactive samples were identified (all confirmed as
positive by western blot), 9 of which had been reported as
nonreactive by SD Bioline HIV-1/2 3.0.
-Thus the first RDT had missed 9 (22.5%) HIV reactive samples
(also confirmed to be positive by western blot) and its
sensitivity on comparison with ELISA was 77.5%.
-In addition, SD Bioline HIV-1/2 3.0 registered 5 false positive
results (negative by ELISA and western blot) giving a specificity
of 99.3%.
8. How sure are we about these results?
Consider:
– Could they have occurred by chance?
– Are there confidence limits?
– What are they?
-No Confidence Limit mentioned in the study
9. Can the results be applied to your patients/ the population of
interest?
HINT: Do you think you patients / population are so different from
those in the study that the results cannot be applied? Such as age,
sex, ethnicity and spectrum bias.
C/ Will the results help me and my
patients/population?
10. Can the test be applied to your patient or population of
interest?
Consider:
– Think of resources and opportunity costs
– Level and availability of expertise required to interpret the
tests
– Current practice and availability of services
11. Were all outcomes important to the individual or population
considered?
Consider:
– Will the knowledge of the test result improve patient
wellbeing
– Will the knowledge of the test result lead to a change in
patient management?
12. What would be the impact of using this test on your
patients/population?

Critical appraisal diagnostic

  • 1.
    Dr. Sherif Omar Dr.Zaid Al-hamadani PGY3 Family Medicine Resident Dr. Omnia Darweesh PGY2 Family Medicine Resident
  • 2.
    Basic structure fora diagnostic study? Through the following steps • Series of patients (of interest) • New test • Reference (Gold) standard • Compare the results of the index test with the reference standard, blinded
  • 3.
    Critical appraisal processfor the diagnostic test Three easy steps A/ Are the results valid? B/ What are the results? C/ Will they help me care of my patients?
  • 4.
    1. Was therea clear question for the study to address? A question should include information about: – the population – the test – the setting – the outcomes A/ Are the results of the study valid?
  • 5.
    Population: Sera from787 consecutive patients tested at the VCT centre in September-October 2012 were included in the analysis. Test (Intervention): Rapid tests versus ELISA for HIV screening Setting: Voluntary Counselling and Testing Facility of a tertiary care centre in North India. Outcomes: Our study highlights that RDTs fare poorly compared to ELISA as screening assays
  • 7.
    2. Was therea comparison with an appropriate reference standard? HINT: Is this reference test(s) the best available indicator in the circumstances?
  • 8.
    -All 787 serawere tested by RDTs (test in question) and all of them also tested by and compared to the standard screening method (ELISA). -Results were also confirmed by doing Western Blot (confirmatory gold standard method).
  • 9.
    3. Did allpatients get the diagnostic test and the reference standard? Consider: – Were both received regardless of the Results of the test of interest? – Check the 2 x 2 table
  • 10.
    Yes all thepopulation received the two tests in comparison.
  • 12.
    4. Could theresults of the test of interest have been influenced by the results of the reference standard? Consider: – Was there blinding? – Were the tests performed independently? (Review bias)
  • 13.
    -The evaluation wasconducted in a blinded fashion with the RDTs and ELISA performed by different technical personnel and the status of the sera as per the RDT based algorithm not revealed to the personnel performing the ELISA.
  • 14.
    5. Is thedisease status of the tested population clearly described? Consider: – Presenting symptoms – Disease stage or severity – Co-morbidity – Differential diagnoses (Spectrum bias)
  • 15.
    -As the aimof the diagnostic tests is screening of HIV, which is done in an asymptomatic population, so the answer to these questions is “No”. -HIV is a silent infection for many years, and thus the aim of screening is towards people representing the general population, for example: •All people coming to live in Qatar regardless of age, sex, race or ethnicity •Pre-employment screening tests •Pre-marital screening tests •Military enrollment
  • 16.
    6. Were themethods for performing the test described in sufficient detail? HINT: Was a protocol followed?
  • 17.
    -The study itselfwas performed in three phases: -The staff were trained and pretest informed consent was taken. Phase one: The first RDT (SD Bioline) applied for all sera, then the results were labeled as reactive or non reactive. Then the reactive tests were tested further by the two other RDTs.
  • 18.
    Phase two: All serathen were tested by ELISA (standard screening test). Phase three: All the positive tests from ELISA and RDTs were compared with Western Blot for confirmation of results.
  • 19.
    7. What arethe results? Consider: – Are the sensitivity and specificity and/or likelihood ratios presented? – Are the results presented in such a way that we can work them out? B/ If so, what are the results?
  • 20.
    -Thirty-six serum sampleswere reactive by the first test. -On subsequent evaluation of all the 787 samples by Microlisa- HIV, 40 HIV reactive samples were identified (all confirmed as positive by western blot), 9 of which had been reported as nonreactive by SD Bioline HIV-1/2 3.0. -Thus the first RDT had missed 9 (22.5%) HIV reactive samples (also confirmed to be positive by western blot) and its sensitivity on comparison with ELISA was 77.5%. -In addition, SD Bioline HIV-1/2 3.0 registered 5 false positive results (negative by ELISA and western blot) giving a specificity of 99.3%.
  • 22.
    8. How sureare we about these results? Consider: – Could they have occurred by chance? – Are there confidence limits? – What are they?
  • 23.
    -No Confidence Limitmentioned in the study
  • 24.
    9. Can theresults be applied to your patients/ the population of interest? HINT: Do you think you patients / population are so different from those in the study that the results cannot be applied? Such as age, sex, ethnicity and spectrum bias. C/ Will the results help me and my patients/population?
  • 25.
    10. Can thetest be applied to your patient or population of interest? Consider: – Think of resources and opportunity costs – Level and availability of expertise required to interpret the tests – Current practice and availability of services
  • 26.
    11. Were alloutcomes important to the individual or population considered? Consider: – Will the knowledge of the test result improve patient wellbeing – Will the knowledge of the test result lead to a change in patient management?
  • 27.
    12. What wouldbe the impact of using this test on your patients/population?

Editor's Notes

  • #6 Voluntary Counseling and Testing (VCT) for HIV usually involves two counseling sessions: one prior to taking the test known as "pre-test counseling" and one following the HIV test when the results are given, often referred to as "post-test counseling". Counseling focuses on the infection (HIV), the disease (AIDS), the test, and positive behavior change. VCT has become popular in many parts of Africa as a way for a person to learn their HIV status. VCT centers and counselors often use rapid HIV tests that require a drop of blood or some cells from the inside of one's cheek; the tests are cheap, require minimal training, and provide accurate results in about 15 minutes.
  • #24 This discordance may possibly be due to low antibody titres especially in recent infections where the levels may well be below the detection limit of RDTs but are picked up by the more sensitive enzyme Immunoassay In our study, the false positives with RDTs are definitely not due to cross-reactivity since all these samples were nonreactive by ELISA. While the cause of these false positives is not exactly known, the common reasons could be technical errors, mislabelling of samples, problems with components of the test devices, and subjective and ambiguous interpretation of faint bands as positive when the test sample is actually negative