Lecture - Rigour and ethics

1. Rigour and ethics
IHPSR Presentation 8
www.hpsa-africa.org
@hpsa_africa
www.slideshare.net/hpsa_africa
Introduction to Health Policy and
Systems Research

2. Four key steps in HPSR
1. Identify research focus
(problem/concern/ opportunity) and
question
2. Design study
3. Ensure quality and rigour
4. Apply ethical principles

3. HPSR rigour
• Health policies and systems are politically and
socially constructed – issues of focus in HPSR are
complex phenomena
• HPSR demands:
– An active process of questioning and checking during the
inquiry
– A constant process of conceptualising and reconceptualising
– Crafting interpretive judgements
– Researcher reflexivity

4. ‘Researcher as instrument’
• Enquiring mind
• Good listening
• Adaptiveness and flexibility
• Grasp of the issues
• Lack of bias (reflexivity)

5. Researcher reflexivity
Robson, 2002
• Identify your personal issues in relation to topic
• Clarify your value system
• Identify areas of possible role conflict
• Identify gatekeepers and how they will influence
you
• Identify where you are not neutral

6. Study validity/trustworthiness
Fixed designs
• Are the findings valid?
• Are the findings
statistically
generalisable?
Flexible designs
• Are the findings
plausible?
• Do the findings provide
theoretical insights that
have a sufficient degree
of generality to be
projected to other
contexts? (analytic
generalisability)

7. Rigour in fixed design
• Validity
– does the study measure what is intended?
– internal validity: cause-effect relationships
• Threats to validity
– bias: selection, measurement, confounding
• Reliability
– participant
– observer
• Generalisability (external validity)
– representative sample
– valid and reliable methods

8. Rigour in RCTs
(randomised control trials)
• Pre-specified study outcomes
• Control groups
• Random allocation to intervention and control
(randomisation)
• Intervention fidelity
• Contamination and co-intervention
• Blinding
• CONSORT guidelines

9. Challenges to RCTs in HPSR
• Sampling units in health systems research (costs!)
• Control groups not always possible
• Complex adaptive systems
– intervention process not under researcher control
– context specificity
• Improved performance requires combined (complex)
interventions
• Cause-effect relationships not easy to establish
• Often have to construct models of complex systems and human
behaviour based on theoretical or practical assumptions

10. Strategies to enhance rigour in
fixed HPSR design
• Focus on plausibility rather than probability
• Mixed methods and triangulation
• Sensitivity analyses in modelling
• Drawing on theory
• Statistical techniques
– multi-level analysis
– structural equation modelling
– quantitative modelling of system dynamics (causal
loops)

11. Rigour in flexible design
Robson, 2002
Threats lie in:
– description
– interpretation
– theory (rival explanations)
Bias from:
– reactivity (e.g. Hawthorne effect)
– respondent (e.g. social desirability bias)
– researcher (e.g. preconceived ideas)

12. Strategies to enhance rigour in
flexible HPSR design
• Use theory (at different stages)
• Use literature and a careful protocol
• Clear exposition of methods
• Multiple sources of data collection
• Triangulation (data, observer, methodological, theory)
• Prolonged involvement
• Negative/’deviant’ case analysis
• Counterfactuals (rival explanations)
• Member checking (respondent validation)
• Peer debriefing/support
• Reflexivity

13. Case study
Validity in case study work requires
paying attention to the trustworthiness of
the interpretive analysis and the resulting
generalisable claim

14. Critical case study issues
• Case selection and sampling
• Data collection approach and procedures
– prolonged engagement with cases
• Analytic procedures:
– respondent validation (member checking)
– triangulation across data sets and with theory
– negative case analysis
– peer debriefing and support
• Clear report of methods of data collection and analysis
(audit trail)

15. Analytic/theoretical
generalisation
• Develop ‘theoretical’ insights’ (generalisable
claims) by:
– building or testing theory and/or
– comparative analysis across multiple cases
• These insights are universal enough to have
relevance in other settings

16. Four ‘tests’ Yin, 2009 (adapted with Robson, 2002)
Case study tactic Phase of research
Confirmability
(the usual
flexible
strategies)
Literature review, identify key concepts
Multiple sources evidence
Establish chain of evidence
Have key informants review draft (member
checking)
Research design
Data collection
Data collection
Write up
Credibility
(the usual
flexible
strategies)
Do pattern matching
Do explanation building
Do rival explanations
Use logic models
Triangulation
Negative cases
Data analysis
Transferability Use theory in single case studies
Use replication logic in multiple case studies
Research design
Dependability Use case study protocol
Develop case study data base (audit trail)
Data collection

17. Interpretive analysis
in HPSR
• Remember the question/s you are asking
• Analysis begins during data collection
– looking for patterns and possible explanations
• Always need to reduce and display data
– narratives, examples of broader phenomena,
diagrams, figures, pictures, tables

18. • Ask questions of the data:
– Is explanation plausible? Can you find
evidence confirming it? Can a finding be
replicated in another data set?
Be reflective!
• Contextualise in analysis
– think about and present relevant features of
the specific social and physical setting,
including historical factors, that support
explanation: identify from theory and in
analysis

19. Role of theory in ensuring
HPSR rigour?
To support generalisable claims:
• Testing hypotheses generated from theory
with empirical evidence
• Triangulation with theory
• Generating theory as a generalisable claim

20. Four key steps in HPSR
1. Identify research focus
(problem/concern/ opportunity) and
question
2. Design study
3. Ensure quality and rigour
4. Apply ethical principles

21. Principles of bio-medical
research ethics
• Are these relevant to HPSR?
How?/Why not?
– Scientific validity and collaborative
partnerships
– Social value and risk benefit ratios
– Informed consent and respect for
participants and communities
– Independent review
Emanuel et al. 2004

22. What practices of real world research are
(most) important in HPSR and why?
Robson, 2002
• Involving people without consent
• Coercing them to participate
• Withholding information about true nature of research
• Otherwise deceiving participants
• Inducing participants to commit acts diminishing of their self-
esteem
• Violating rights of self-determination
• Exposing participants to physical or mental stress
• Invading privacy
• Withholding benefits from some participants
• Not treating participants fairly or with respect

23. Examples of HPSR ethical
challenges
• Being informed about or observing unethical
behaviour of providers, managers, community
members
• How to present findings in ways that balance
confidentiality and give voice to those not usually
heard
• How to act in ways that have potential to stimulate
change in (rather than exacerbate) unequal
relationships between people

24. Other HPSR issues
• Who determines the questions?
• Gaining consent and relative power:
– from policy makers to marginalised groups
• Anonymity and small sample sizes
• Acting ethically towards field workers
• Recognising researchers’ privilege and power
• Generating social value: From substantive relevance
to where? Where does the researcher’s role end?

25. Copyright
Funding
You are free:
To Share – to copy, distribute and transmit the work
To Remix – to adapt the work
Under the following conditions:
Attribution You must attribute the work in the manner
specified by the author or licensor (but not in any way that
suggests that they endorse you or your use of the work).
Non-commercial You may not use this work for commercial
purposes.
Share Alike If you alter, transform, or build upon this work,
you may distribute the resulting work but only under the same
or similar license to this one.
Other conditions
For any reuse or distribution, you must make clear to
others the license terms of this work.
Nothing in this license impairs or restricts the authors’
moral rights.
Nothing in this license impairs or restricts the rights of
authors whose work is referenced in this document.
Cited works used in this document must be cited following
usual academic conventions.
Citation of this work must follow normal academic
conventions. Suggested citation:
Introduction to Health Policy and Systems Research,
course presentation, Presentation 8. Copyright
CHEPSAA (Consortium for Health Policy & Systems
Analysis in Africa) 2014, www.hpsa-africa.org
www.slideshare.net/hpsa_africa
This document is an output from a project funded by the European Commission (EC) FP7-Africa (Grant no.
265482). The views expressed are not necessarily those of the EC.

26. The CHEPSAA partners
University of Dar Es Salaam
Institute of Development Studies
University of the Witwatersrand
Centre for Health Policy
University of Ghana
School of Public Health, Department of
Health Policy, Planning and Management
University of Leeds
Nuffield Centre for International Health and
Development
University of Nigeria Enugu
Health Policy Research Group & the
Department of Health Administration and
Management
London School of Hygiene and
Tropical Medicine
Health Economics and Systems Analysis
Group, Depart of Global Health & Dev.
Great Lakes University of Kisumu
Tropical Institute of Community Health and
Development
Karolinska Institutet
Health Systems and Policy Group,
Department of Public Health Sciences
University of Cape Town
Health Policy and Systems Programme,
Health Economics Unit
Swiss Tropical and Public Health
Institute
Health Systems Research Group
University of the Western Cape
School of Public Health