ASSESSING THE
ACCURACY OF A
CLINICAL OUTBREAK Jost, C.,
Priyono, W.,
Bett, B., Poole,
DEFINITION FOR HIGHLY J., Schoonman,
L., McLaws, M.,
Unger, F.,
PATHOGENIC AVIAN Mariner, J.
INFLUENZA (HPAI)

OUTLINE
Background
Key
concepts
Methods
Results
Discussion

T YPE A H5N1 HPAI
Emerged in
Guangdong Province,
China
First detected in Hong
Kong, 1997
Spread throughout
Asia, Europe, the
Middle East and
Africa
(Sims, 2007; Morris and Jackson, 2006)

H5N1 HPAI IN INDONESIA
First H5N1
outbreak 2003
First reported to
OIE in Feb 2004
Confirmed in 31
of 33 provinces
Photo credit: Peter Roeder
(Azar et al., 2010)

CLINICAL CASE DEFINITION
A set of clinical criteria (symptoms, other epidemiological
parameter s) for detecting diseased individuals or populations
Indonesia MoA PDSR HPAI sur veillance
Operational Research in Indonesia for Highly Pathogenic Avian
Influenza (ORIHPAI)
Early Detection, Response and Sur veillance for Avian Influenza in
Africa (EDRSAIA)
H7N7 Netherlands backyard chickens (Elbers, Koch and Bouma, 2005)
Increased mortality or swollen head were the most sensitive clinical
indicators (sensitivity 100%, specificity 20-32%)
Addition of cyanosis increased the specificity of a diganosis
(sensitivity 65-100%, specificity 68-80%)

DIAGNOSTIC ACCURACY
Accuracy: Propor tion of subjects
correctly identified by the test
Sensitivity (Se): Probability that a
test correctly identifies infected Infected Not
subjects as positive Infected
Specificity ( Sp ): Probability that Test True False
a test correctly identifies NON - Positive Positive Positive
infected subjects as negative Test False True
Negative Negative Negative
(PV+):
Predictive value positive (PV+) :
test-
Propor tion of test - positive
subjects that are truly infected
Predictive value negative (PV - ):
test-
Propor tion of test - negative
subjects that are truly not
infected (Zepeda, pers. com.)

METHODS
Sample Frame
16 districts West and Central Java
Level of assessment RT (lowest geopolitical unit in Indonesia)
Routine veterinary officer visits or in response to disease
report
Vaccinated and unvaccinated areas

METHODS
Outbreak case definition for HPAI in backyard and small-scale
commercial chicken flocks in Indonesia
Step 1: High mortality rate (>80%), poultry dying within 12 hours of
the onset of clinical signs = Sudden Death
Step 2:
Per acute death (or death within 4 hours), involvement of more than one
household, with or without blue discoloration of the head and body = HPAI
As above but death in >4 hours, with or without recovery of some birds =
ND
Population:
Sick Chickens
Level 1 Level 1
Sudden Death Not Sudden Death
Level 2 Level 2 Level 2
HPAI-compatible ND-compatible Unknown

METHODS
Biological Sampling
Tracheal samples from up to 25 affected chickens
Standard World Health Organization (WHO) H5N1 sampling protocols
Pooled 5 swabs per universal transport medium (UTM) tube
containing antibiotics
Maintained at 4°C until delivered to lab
Stored at -80°C in lab
Analysis (Wates DIC)
Virus isolation, amplification and typing using Indonesia H5N1 and
ND specific antibodies (Pusvetma)
Up to 4 passages
Outbreak considered positive if at least 1 tube positive

RESULTS
N = 297 outbreaks
Average 18.6+7.8 outbreaks investigated per district
11 .4% from vaccinated areas, 88.6% unvaccinated areas
Sudden death diagnosed in 70.4% of outbreaks
77.0% were further diagnosed a HPAI-compatible
14.8% as VVND-compatible
8.1% as unknown

RESULTS
Frequentist calculations
Sudden death: Se 76.4%+5.9%, Sp 41.8+9.8%, PV+ 72.7+6.0%
HPAI-compatible: Se 71.2+7.3%, Sp 62.3+7.7%, PV+ 64.6+7.4%
Series diagnoses
HPAI-compatible: Se 54.4+8.1%, Sp 78.0+6.6%
VVND-compatible: Se 10.0+7.5%, Sp 94.3+3.0%

RESULTS
Bayesian analysis HPAI (no gold standard)
Case definition HPAI-compatible: Se 85.1+4.3%, Sp 78.2+4.5%
PV+75.1+6.0%
H5N1 virus isolation and typing: Se 88.1+4.4%, Sp 91.0+3.1%, PV+
88.6+4.1%
True prevalence H5N1 43.8+4.6%
Bayesian analysis VVND (no gold standard)
Case definition VVND-compatible: Se 69.1+10.6%, Sp 89.2+2.0%,
PV+ 22.5+7.7%
ND virus isolation and typing: Se 87.6+4.9%, Sp 90.9+1.9%, PV+
30.8+10.7%
True prevalence ND 4.5+1.7

DISCUSSION
prevalence, PV+, PV-
Predictive
value %
Prevalence %
PV+, Sp PV-, Se
PV + PV -
Specificity % Sensitivity % (Zepeda, pers. com.)

DISCUSSION
Bayesian analysis likely provides a more realistic estimation
of the accuracy of the level 2 clinical outbreak case
definitions
Gold standard?
Sensitivities of sudden death and HPAI-compatible diagnoses
within acceptable ranges reported in the literature for clinical
case definitions, but…
For a study of the impact of control measures on disease
incidence, higher sensitivities would have been more
desirable

DISCUSSION
For surveillance:
Use a test with high Se and high PV- to:
Reduce the number of false negatives
Avoid the introduction of a disease
Use a test with high Sp and high PV+ to:
Confirm a diagnosis
Avoid the unnecessary slaughter of animals
Therefore, our HPAI case definition:
Best used to avoid the introduction of a disease

DISCUSSION
Test in parallel when:
PV-
You wish to increase Se and PV -
All tests must be negative
Test in series when:
You wish to increase Sp and PV+
All tests must be positive
O ur HPAI case definition:
Was applied in series (levels 1 and 2)
Increased the specificity of the diagnosis
Sensitivity could be increased by removing level 1 of the diagnosis

LESSONS LEARNT
Diagnostic tools that rely on antigen detection
are only useful in active cases
For acute viral infections of short duration
(HPAI), studies based on antibody detection
require prohibitively large sample sizes
Because very few chickens survive and go on to
develop antibodies
Therefore, case definitions are an important
diagnostic tool for surveillance and research
of acute viral infections

LESSONS LEARNT
Case definitions should be designed with the objective in
mind
100% accuracy does not exist!
Estimating the diagnostic accuracy of the case definition used
is important for interpreting study results

ACKNOWLEDGEMENTS
Indonesia MoA
Participating district VS
Wates DIC
ILRI