This document discusses issues and strategies in evaluating interventions against animal disease outbreaks after the fact. It notes that ex-post evaluation faces four key challenges: defining the scenario if no intervention occurred, accounting for all losses under that scenario, handling data uncertainty, and determining the intervention's impact given other influencing factors. It provides examples showing how stochastic simulation can address these challenges by generating distributions of key outputs and conducting sensitivity analysis. Overall, the document emphasizes the importance of integrated epidemiological and economic modeling combined with qualitative analysis to plausibly attribute changes in a disease's trajectory to the evaluated intervention.

1. Issues and strategies in ex-post evaluation of intervention
against animal disease outbreaks and spreads
Mohamadou Fadiga & Hikuepi Katjiuongua
Mainstreaming Livestock Value Chain Conference.
5-6 Nov. 2013. Accra, Ghana

2. Impacts of animal diseases
• Animal disease outbreaks: can be devastating
Direct costs
- Death of animals
-
Lower productivity – slow growth, reduced efficiency of input use
- Control costs
Indirect costs
-
Reduced access to markets
Long term macroeconomic effects
Effects of price changes on supply chain actors
Spillover effects such as effects on tourism
2

3. Impacts of animal disease
• Highly pathogenic avian influenza: Nigeria (2006/2008) 1.5 million
birds lost (747,000 culled)
• Foot and mouth disease: Botswana - trade ban led to $ 33 million
(USD) losses at processing level alone
• Tick & tick borne diseases: undermine livestock productivity
• Impacts differ by production system, coping & risk management
capacity of VC actors, state of veterinary service delivery
• Require costly interventions: culling, quarantine & movement
restrictions, and vaccinations…
• Important to evaluate animal disease intervention
3

4. Issues in ex post evaluation of animal disease outbreaks
Four key issues in ex post economic assessment of
intervention against animal disease outbreaks
Defining the counterfactual scenario
Accounting for the losses (under counterfactual)
Handling data uncertainty and specificity of
epidemiological data
Dealing with the issues of attribution
4

5. Issues in ex post evaluation of animal disease outbreaks
• Governments are generally risk averse with respect to animal
diseases
• Design intervention interventions to minimize losses in
expected social welfare
EW
i
p ( i )WD
i
[1 p( i )]WF
i
r( i )
• or some expected damage function
ED
i
DC ( i ) IC
i
• Measuring these losses accurately requires integrated
epidemiology and economic models
5

6. On the counterfactual scenario
• Probability of outbreak is a composite risk estimate
– A product of or risk of introduction, risk of spread, and mortality risk
– Defining the counterfactual risk is akin to answering the question about what
would have been the trajectory of the disease in the absence of intervention
– Use a combination of experimental, historical, participatory data on the
disease, etc…
•
Use total death relative to population at risk to derive mortality risk
• For risk of spread
– Develop an SI (susceptible-infected)model
– Data on transmission rate, incubation period, infectious period, and lapse
between depopulation and restocking are used to solve the differential
equations that illustrate the SI model
6

7. On the counterfactual scenario
Prevalence
50%
Endemic state
40%
30%
St
Ct
It
Rt
1 1
1
0
0
St 1
Ct 1
1
0
1 1
1
0
1 1
It 1
Rt 1
0
0
0
0
1
20%
Unstable epidemic
10%
0%
0
100
200
300
400
Day
Scenario
Spread
Risk Estimates
Mortality
Spread
Mortality
Composite
Burn-out
Low
0.13
0.01
0.0013
Burn-out
High
0.13
0.02
0.0026
Endemic
Low
0.27
0.01
0.0027
Endemic
High
0.27
0.02
0.0054
7

8. Use of counterfactuals and example from HPAI in Nigeria
Calculate expected welfare or
expected damage without the
intervention at various scenarios
Net Social Welfare Gain over 2006-2010
US $ Million
Analyse the net effect of risk
reduction on social welfare
110
88
66
Find the socially optimal level of risk
that justifies intervention
44
22
0
0%
Evaluate if eradication makes sense
economically
20%
40%
60%
80%
100%
Risk Reduction Level
8

9. Stochastic approach
• Characteristics of the data
– Underlying uncertainty (multiple sources, measurement errors, etc...)
– Disease transmission is stochastic
– Not all susceptible subjects in contact with infectious ones would
catch the disease
– Necessity to test the validity of results through sensitivity analysis
• Stochastic simulation addresses all these points at once
– Use the collected and/or derived data on spread and mortality to
simulate their distribution
– Solve for the key output variables using random draws from the
distribution of risk parameters
– Generate a distribution of the key output variables
– Conduct a probabilistic sensitivity analysis on the key output variables
9

10. Stochastic simulation
Examples of Stochastic Results
Simulated Risks of HPAI
Welfare
Simulated Risk of Spread
18%
Losses
Frequency
15%
12%
Disease Cost
Mean
147.44
144.97
St. Deviation
291.93
115.99
Lower 95%
121.85
134.80
Upper 95%
173.03
155.14
9%
6%
3%
0%
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
Risk of Spread
Simulated Mortality Risk
Sensitivity analysis on welfare losses
53%
1.0
Probability
Frequency
45%
38%
30%
23%
15%
8%
0.8
0.6
0.4
0.2
0%
0.00
0.01
0.03
0.04
0.05
0.07
Mortality Risk
0.08
0.09
0.0
0
500
1,000
1,500
Social Welfare Effects
2,000
10

11. Issues of attribution in intervention
• Ideal Scenario: randomization and use RCTs
• But scope for randomization nearly zero in case animal disease
outbreak – responding to a crisis. Alternative approaches exist but
depend on unit of analysis
• Other factors influence disease spread and the effectiveness of the
intervention (e.g. feed, water availability, agent behavior etc.)
• These make attribution of the intervention in a probabilistic sense
difficult
11

12. Issues of attribution in intervention
• Necessity to use qualitative analysis in addition to quantitative assessment
targeting indicators through key informant interview about what could
have happened in the absence of the disease
• Use participatory epidemiology to capture indigenous knowledge about
the disease effects
• Pay attention to the timing of intervention and careful documentation of
all inputs used in the intervention and their sources
• Use past information about losses that coincided with the disease
outbreak to gain insights how the intervention may have potentially
affected the disease dynamics
• Overall: With that one could plausibly attribute observed change in
disease trajectory to the intervention that was carried out
12

13. Accounting for losses
• Customary to focus on total cost (both direct and indirect)
• But when calculating total cost it is important to focus on
avoidable losses – can overestimate of cost savings
• In other words not all risk will be removed as a result of the
intervention
• Doing that would yield more reasonable incremental benefits
and incremental net benefits that accrued because of the
intervention
13

14. Thank you
14