Health Impact AssessmentA triumph over common sense? Centre for Primary Health Care and Equity Ben Harris-Roxas Consultant, Harris-Roxas Health Conjoint Lecturer, University of New South Wales, Sydney Health Section Co-Chair, International Association for Impact Assessment
A detailed paper on this talk is available fromwww.harrisroxashealth.com/2012/10/korea2012
Background• We work in increasingly resource constrained systems and all interventions are expected to demonstrate their usefulness – this includes HIA• The use of HIA may seem like “common sense”. It’s already being used; it has already demonstrating its usefulness. We have good examples.
Background• But common sense isn’t good enough. We need to convincingly demonstrate HIA’s effectiveness.• This effectiveness can be thought of narrowly, in terms of changes to decisions and implementation, and evidence of recommendations being adopted.• It’s also important to look at HIA’s ability to change ways of working, understanding and learning.
This presentation• Draws on a before-and-after study (prospective multiple case study): • Two similar equity focused HIAs of service plans (obesity prevention and management strategies) • One HIA was conducted, one was screened out • 23 interviews, conducted before and after the HIAs, document analysis
The effectiveness of HIA• The effectiveness of HIA is complex and affected by many factors• Colleagues at UNSW and I developed a conceptual framework that attempts to look at the broad range of factors that can be affected through HIAs, but also the factors that affect them
Source: Harris-Roxas B, Harris E (2012) The Impact and Effectiveness of Health Impact Assessment: A conceptual framework, Environmental Impact Assessment Review: accepted, in press. doi:10.1016/j.eiar.2012.09.003
The need to look at perceptions of effectiveness • It was obvious to us from previous research that perceptions of an HIA matter. • Perceptions determined the extent to which people attributed changes to an HIA. • Perceptions can give rise to conflict.Source: Harris-Roxas B, Harris P, Harris E, Kemp L (2011) A Rapid Equity Focused Health Impact Assessment of a Policy Implementation Plan: An Australian case study and impact evaluation, International Journal for Equity in Health, 10(6), doi:10.1186/1475-9276-10-6.. http://www.equityhealthj.com/content/10/1/6
Example: What is the purpose of an HIA? “I think people felt when recommendations came in, that they saw as a critique, or not that they were a critique, because different... They were like ‘Oh, but it wasn’t a proper plan anyway, it was just, you know, we were just trying to get the money, and that was our goal at that time, just get the money, and we said we’d do this, but not sure if we really will’.”
Example: What is the purpose of an HIA? “We didn’t have a shared understanding of why we were undertaking it. Our purposes were probably different[...], and maybe that’s where they don’t work, but if you have two differing purposes, unless you can fully appreciate what those two different purposes are, maybe it doesn’t work out as well as it could... I think there was a feeling that, well, we could get something out of [the EFHIA]. There were probably two rationales for why it would be useful. One is that we could get some, a critique if you like, or some feedback about, through an equity lens, on the strategies that we had proposed. And the second one was that it would perhaps serve a process of helping people who are more engaged in the consultation process.”
Example: What is the purpose of an HIA? “In a way, it was about improving the quality of the document, it was actually quite important to be able to debate some of the issues.”
Example: We would have known it anyway(the recommendations are common sense) “There were also some things in *the EFHIA report+ that, I guess, implied, that we wouldn’t consider, some issues that I think can be dealt with in careful planning, and careful implementation, and the intention, as I said before, if the [ABHI implementation plan+ was really about ‘this is the flavour of where we’re going with this’ we’re going to have to obviously have greater implementation plans around each of these strategies, we’ve only got sixty pages to do it in.”
Example: We would have known it anyway(the recommendations are common sense) “There are quite dichotomous views about what people believe about HIAs. Some people believe there is a place [for HIAs], blah, blah, blah and they’re fantastic. Other people believe [these issues are addressed as] part of a good planning process, and there’s some there are in between those two.”
Are common sense and HIA synonymous?• Not really! What seems obvious in hindsight wasn’t really at the time – this is known as hindsight bias.• I found several examples of hindsight bias in this study, where people expressed disappointment that the HIA didn’t result in radically new information, even though that isn’t what they hoped for in the interviews before the HIA (more detailed examples in the paper)
Conceptual challenges in evaluating HIASo, given: 1. perception of HIA’s effectiveness matters, and 2. we know hindsight bias applies to perceptions of HIA;how should we address this problem when evaluating HIA? (Keep in mind almost all evaluations of HIAs have relied on retrospective/historical accounts)
Challenge 1: Narrative fallacy• We create narratives (stories) to explain events.• Stories are created after the event, often to explain what happened, e.g. “this is why the HIA failed/succeeded”.• We tend to create stories to suit we want events to be understood, and this leads to narrative fallacy.• We cant rely on stories to explain what actually happened (stories often sound like they’re about different events!)
Challenge 2: Creeping determinism• Sounds complicated but is essentially the tendency to discount information and learning because “we knew it all along” or “the HIA was always going to end up not making a difference”.• It highlights the need to have clear descriptions about the purpose and desired outcomes of an HIA before it starts.
What’s required: Prospective evaluation• We lack counterfactuals – the “what if?” cases• Comparative prospective evaluation/natural experiments of similar proposals when HIAs were done • won’t allow perfect comparisons • but will allow some comparisons.• Data also needs to be collected before, during and after HIAs to attempt to address the problems of narrative fallacy and creeping determinism.
What’s required: HIAs of health proposals• There’s an assumption health addresses health well in planning but not much evidence of this.• HIAs with a focus on distributional impact and equity have been useful in getting the health sector to think about the impacts of health services and programs in a different way.• Capacity gets built in the health sector, not just to do HIAs but to set ToRs, review completed HIAs and advocate for HIA’s use (and work intersectorally)
What’s required: HIAs of health proposals• This study was only possible of strong relationships and credibility in the health sector. It would have been more difficult working with other sectors.• If health agencies/professionals want to promote Health in All Policies they will also need to demonstrate they’re serious about Health in Health Policies.
Conclusions• HIA has the ability to alter not only plans and documents but also perceptions and understandings, often in ways that are only poorly accounted for by participants in retrospect (looking backwards).• Impact assessment, at its best, provides useful recommendations that are timely and linked to implementation.• The recommendations can seem obvious!
Conclusions• Narrative fallacy and creeping determinism can make it seem like we knew this information all along.• Common sense is often anything but common, but it can seem that way in retrospect.• HIA may be a way to triumph over common sense.
These slides are available atwww.slideshare.net/benharrisroxas firstname.lastname@example.org @ben_hr or @hiablog healthimpactassessment.blogspot.com linkedin.com/in/benharrisroxas