Intro to realist research
•
2 likes•382 views
This document provides an introduction to realist evaluation and synthesis methodology. Realist methodology seeks to understand how and why programs work by exploring the interactions between contexts, mechanisms, and outcomes. It uses a theory-driven approach to evaluate complex social programs and assess how outcomes may differ depending on contextual factors. The key aspects of realist evaluation explained in the document are that it focuses on developing explanatory program theories, explores causal pathways and mechanisms, and aims to provide context-sensitive explanations rather than generalized verdicts. Undertaking realist research is described as an iterative process of refining theories based on mixed evidence.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Intro to realist research
Similar to Intro to realist research (20)
Recently uploaded
Recently uploaded (20)
Intro to realist research
- 1. An Introduction to Realist Evaluation & Synthesis Siobhan Connolly s.connolly@ed.ac.uk
- 2. What is Realist Methodology? • A theory-driven approach to evaluation and synthesis • Seeks to understand and explain the context and underlying mechanisms. • Used to assess complex ‘social world’ evidence using programme theories. Programmes Policy Implementation Services Interventions * For differences between groups an their expierences the method will need to be adapted.
- 3. The Realist Approach Differing emphasis and what it offers • Clarity on programme designs. • Clarity on how programmes are run. • Causal pathways leading to outcomes • Explores contextual determinants and compares them • Theorises mechanisms for empirical testing either as part of the project or at a later date • Defines, conceptualises and scopes out intended and unintended outcomes. Final Product = Not a verdict, it’s an explanatory model on how programmes are operating.
- 4. Realist paradigm(s) Critical Realism • Critical Realism is a philosophy that has ontological and epistemic roots. • Roy Bhaskar - radical critique of the philosophies of western science – social and natural. • Created a ‘realist’ theory of science (not to be confused with old-school realism). • Built upon by others esp. Margaret Archer who looked at the role of social structures and time in effecting change or causing stagnation. • Other notables – Andrew Sayer & Andrew Collier Methodology • Realist Methodology is a family • Pawson & Tilley 1997 and further developed by Ray Pawson. • Other notables: Geoff Wong, Trisha Greenlaugh, Nick Emmel and Justin Jogash. • Influenced by critical realism and others – see Pawson (2006) A Realist Manifesto.
- 5. Realist evaluation and synthesis What’s the difference? Realist Evaluation Realist Synthesis (primary analysis) (secondary analysis) Data collection, interpretation & analysis Mixed Methods Pawson & Tilley (1997) Realistic Evaluation SAGE, London. Aka a ‘realist review’ Synthesises the literature/evidence Pawson (2006) Evidence-based policy. A realist perspective. SAGE, London.
- 6. Realist Principles Generalisability works for natural science The concept of generalisability heavily entrenched in RCT’s aka the ‘gold standard’ of evidence based medicine. Lets get realist – Context matters in social science Too often experimental evaluators take positive results and say ‘It works’ and has ‘validity’. Scale it up and roll it out. Failing to ask ‘does it work?’ why? It ignores context to generalise Variation is real, even in RCTs. Core Realist Principle What works for whom in what circumstances and how? Embrace Mixed Methods Realist research uses and iterates between evidence and theory to build upon it and refine it.
- 7. The Hierarchy of Evidence Reinterpreted
- 8. Realist Inquiry Positivism Experimentation Quasi-Experimentation Theory testing Deduction Quantitative Realism The best of both Mixed Methods Refines theory Tests theory Constructivism Theory building Inductive Qualitative Narrative analysis Interprets differing perspectives
- 9. Realist Inquiry & Retroduction • Retroduction is the cornerstone of realist methodology • “Retroduction is meant to overcome the deficiencies of induction and deduction to offer causal explanations…” SAGE encylopedia of social science research methods • Iterative analysis – deduce/infer from the evidence and induct/formulate explanations that go behind observable patterns to discover what produces them.
- 10. Retroduction in Action Empirical Data Narrative Analysis Observed Patterns Paradigmatic conflict Hidden Patterns
- 11. C + M = O • Realist research uses the configuration C+M = O • Contexts + Mechanisms = Outcomes • Provides a casual model for evaluating and synthesising complex evidence • Transparency is vital
- 12. A Simplified Schemata of CMO Configurations CONTEXT MECHANISM OUTCOME Environment Reactions & Interactions to Resources Effect
- 13. What is meant by context? • Cultural norms & values • History • Economic and financial conditions • Geographic • Socio-political • Existing programmes, policies, procedures etc. • Previous programme outcomes • Anything ‘real’ within the social environment
- 14. What is a Mechanism? • Generative force (causal power) that leads to outcomes • Mechanisms can be active or inactive ‘triggered’ or ‘insidious’ • Related to reasoning, decision-making, responses, reactions and engagement with resources Why? Because in social systems, people affect change not interventions • Mechanisms operate on different levels of reality • Mechanisms inform but are not synonymous with program strategy
- 15. Mechanisms • What ‘turns on’ or ‘turns off’ in the minds of participants when asked to engage with a programme or intervention?
- 16. Complexity and the Strata’s of Reality
- 17. Take a closer look – a lot of context Mechanisms are interactions, reactions And responses toward the levels of reality *Observed or *Hidden = real
- 18. A Retroduction Metaphor OBSERVABLE & MESURABLE C+M HIDDEN C+M Abstract concepts Narrative
- 19. A Retroduction Metaphor OBSERVABLE & MESURABLE C+M HIDDEN C+M Abstract concepts Narrative Latent Mechanism Critical realist concept Enablers, Constrainers and Potentialities Potentialities can inhibit or exert influence They are real but inactive
- 20. Outcomes • Intended or unintended • Have an effect i.e. increase/decrease/improve/enhance/no change etc. • Can be quantified or qualitative • Should be able to explain: What about this programme is or is not working?
- 21. Context Effects Outcomes • Contextual factors can impede mechanisms that effect the outcome • Contextual factors can work in a particular situations influencing the mechanisms and outcomes that can be altered under different circumstances. Context Context _______________ M1 O1 Informs the next phase M2 02 Include a transparent timeline to show The ripple effect Through the phases Jogosh et.al., 2015
- 22. Realist Research & Pre-Existing Conditions • Some interventions and programmes have to grapple with pre-existing conditions. • Initial programme theories will need to make the soil more fertile and create the context for the intervention and forge partnerships for further work. • Should be viewed as a refined starting point. • The growth phase will be further realist evaluation and synthesis.
- 23. Realist Research is Infinite • Realist research provides new evidence informed theory that has been refined. • It will need to be developed and refined further. • A realist project is one link in the long ongoing chain of evaluation. • Advocates for further testing both theoretically and empirically. • It does not claim to be generalizable • Provides context sensitive information that allows for change.
- 24. Undertaking a Realist Project Write it up Disseminate Present Publish Present it Drawing on both theory and evidence and from evaluation and synthesis refine the initial programme theories using CMO configurations. Start with a table Configure theory and evidence together into CMO’s Gather evidence based on the programme theories Register the protocol 2. Develop Protocols & Gather Evidence Establish the programme theories Previous experience Retroductive thinking Generate a hypothesis or a casual picture of what’s happening 1. Programme Theories
- 25. Recommended Reading • Pawson R., & Tilley N. (1997) Realist Evaluation. SAGE, London. • Pawson R. (2006) Evidence-based policy. A realist perspective. SAGE, London. • Pawson R. (2013) A Realist Manifesto: SAGE London. • Jagosh, J., Bush, P., Salsberg, J., Macaulay, A., Greenhalgh, T., Wong, G., . . . Pluye, P. (2015). A realist evaluation of community-based participatory research: Partnership synergy, trust building and related ripple effects. BMC Public Health, 15(1), 725. • Andrew Collier (2014) Critical Realism: An introduction to Roy Bhaskars Philosophy. Verso, London.
- 26. Useful Resources • Join the realist research community mailing list at https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=RAMESES
Editor's Notes
- This can be done in isolation or they can overlap. It depends on the aims of the project.
- No rules on what comes first. Some start with data collection and synthesis the litrarture afterward
- There is a whole debate to be had about the hierarchy of evidence because there is issues with RCTs. And public health is starting to offset some of them with realist research.
- Hovis = The best of both – Realism straddles both paradigms
- Revealing hidden patterns that are at play in any given intervention or programme.
- Challenges: Environment is massive! How do we define it? Environment is also interpretive Resources and Reactions require a detailed look at implementation chains and the stakeholders involved. Outcome – Context and outcomes can be confusing! We can test how context has an impact.
- The concept of enablers, constrainers and latent mechanisms comes from critical realism. Its not in existence but has the potential to be,
- Context is not the same as factors – factors are too vague.
- The anatomy syllabus project is a really good example of this.