Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Evaluate Complex Interventions

1,362 views

Published on

How to Evaluate Complex Interventions

Published in: Education
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

How to Evaluate Complex Interventions

  1. 1. How to evaluate complex interventions Astrid Brousselle, Ph.D. Canada research Chair in Evaluation and Health System Improvement (crc-easy.ca) Full professor, community health sciences departement Researcher, Centre de recherche de l’Hôpital Charles-LeMoyne Université de Sherbrooke Toronto, 2015, Program Science Symposium
  2. 2. What is evaluation? - Determining the merit, worth and value (judgement, several criteria) - Of an intervention, or of its components (resources, process, results) - Systematic inquiry - Final objective: improvement of the intervention, social betterment, accountability, social agenda, action Merriam-Webster: “to judge the value or condition of (someone or something) in a careful and thoughtful way”
  3. 3. Normative evaluation Evaluative research Assessment of the results Assessment of the process Strategic analysis Effect analysis Economic eval. Production analysis Implementation analysis Logic analysis SEVERAL QUESTIONS Assessment of the structure The problem Objectives Activities/service s Resources Effects Context
  4. 4. Lots of models
  5. 5. from Shadish Cook and Leviton (1991) PracticePractice Values KnowledgeUtilization Social programming Problem
  6. 6. Pawson & Tilley Source: Alkin MC and Christie CA (2004) An evaluation theory tree. In: Alkin MC (ed.) Evaluation Roots. Tracing Theorists’ Views and Influences. Thousand Oaks, CA: SAGE, 12–65. ADAPTED from Shadish Cook and Leviton (1991)
  7. 7. The utilization paradise The lobbying zone The knowledge- driven swamp Evaluation models and use CONTANDRIOPOULOS, D., BROUSSELLE, A. (2012) “Evaluation models and evaluation use?” Evaluation: The International Journal of Theory Research and Practice, vol. 18: 61-77.
  8. 8. Complex Intervention Complex interventions are characterized by : • Uncertainty, unpredictability • Interdependence among a large number of actors • Adapting and evolving actors • Emergent outcomes created by the connections or relationships in the system • Non-linearity between inputs and outputs (Shiell, Hawe, & Gold, 2008, cited in Zimmerman et al. 2012 )
  9. 9. Analysing complex interventions Reduction trap / complexity trap Embracing complexity: • Accepting paradoxes (not contradictions) and dealing with them • Analyzing the relations, the “in between” • Accepting uncertainty • Using multiple perspectives (Zimmerman et al. 2012 ) A statement that appears to be self-contradictory or silly but may include a latent truth. A statement that appears to be self-contradictory or silly but may include a latent truth.
  10. 10. Evaluation answers to interventions’ complexity Logic analysis Realistic evaluation Contribution analysis Economic value of programs
  11. 11. Logic analysis Source: Rey et al. 2012 Crucial characteristics of the intervention Effects Desired effects Other effects Critical conditions Direct logic analysis Alternatives (interventions) Effects Other effects Critical conditions Reverse logic analysis Desired effects
  12. 12. Realistic evaluation “What works, for whom, under which circumstances?” Theory Hypotheses Observations Program specification Mechanisms Contexts Outcomes
  13. 13. Contribution analysis Source: Mayne, 2012
  14. 14. Economic value TCHOUAKET, E., BROUSSELLE, A., FANSI, A., DIONNE, P-A., BERTRAND, E., FORTIN, C. (2013) The economic value of Quebec's water fluoridation program, Journal of Public Health, 26 juin 2013, open access: springerlink.com
  15. 15. What these approaches share Centrality of the “theory of change” Importance of the context Integration of various perspectives Plausible interpretation (uncertainty) Valid and rigorous research process Creativity http://www.jr-art.net
  16. 16. References Alkin MC and Christie CA (2004) An evaluation theory tree. In: Alkin MC (ed.) Evaluation Roots. Tracing Theorists’ Views and Influences. Thousand Oaks, CA: SAGE, 12–65. Champagne F., Contandriopoulos A-P, Brousselle A., Hartz Z., Denis J-L. (2011-2e édition) « L'évaluation dans le domaine de la santé : concepts et méthodes », dans BROUSSELLE A., Champagne F., Contandriopoulos A P., Hartz Z. (Eds) L'Évaluation :‑ Concepts et méthodes, Presses de l’Université de Montréal : 49-70. Contandriopoulos, D., Brousselle, A. (2012) “Evaluation models and evaluation use?” Evaluation: The International Journal of Theory Research and Practice, vol. 18: 61-77. Mayne J. Contribution analysis: Coming of age? Evaluation 2012; 18(3): 270-280. Dubois N, Lloyd S, Houle J, Mercier C, Brousselle A, Rey L. Discussion: Practice-based evaluation as a response to address intervention complexity. Canadian Journal of Program Evaluation 2012; 26(3): 105-113. Pawson, R., & Tilley, N. (2005). Realistic evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 362-367). Thousand Oaks, CA: Sage Publications. Rey L., Brousselle , A., Dedobbeleer, N. (2012) «Logic analysis: testing program theory to better evaluate complex interventions», dans Houle, J., Dubois, N., Lloyd, S., Mercier C., Hartz, Z., Brousselle, A. (Ed.) (2012) « L'évaluation des interventions complexes », numéro spécial, Revue Canadienne d'Évaluation de Programme/ Canadian Journal of Program Evaluation, Volume 26(3): 61-89. Shadish W.R., Cook T.D., Leviton L.C. (1991) «Good Theory for Social Program Evaluation», in Shadish W.R., Cook T.D., Leviton L.C., Foundations of program evaluation. Theories of practice, Sage Publications: 36-67. Shiell, A., Hawe, P., & Gold, L. (2008). Complex interventions or complex systems? Implications for health economic evaluation. BMJ, 336(7656), 1281–1283. doi:10.1136/bmj.39569.510521.AD Tchouaket, E., Brousselle , A., Fansi, A., Dionne, P-A., Bertrand, E., Fortin, C. (2013) The economic value of Quebec's water fluoridation program, Journal of Public Health, 26 juin 2013, open access: springerlink.com Zimmerman, B., Dubois, N., Houle, N., Lloyd, S., Mercier, C., Brousselle, A. , Rey, L. (2012) «How does complexity impact evaluation?: An Introduction to the Special Issue», numéro spécial, Revue Canadienne d'Évaluation de Programme/ Canadian Journal of Program Evaluation, Volume 26(3):v-xx. Astrid.brousselle@usherbrooke.ca crc-easy.ca

×