Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Epidemiology in Action

34 views

Published on

HSCI 330

Published in: Health & Medicine
  • Be the first to comment

  • Be the first to like this

Epidemiology in Action

  1. 1. EPIDEMIOLOGY IN ACTIONA PRIMER ON FIELD WORK AND IMPLEMENTATION
  2. 2. OUTLINE FOR TODAY’S LECTURE • Field Investigation • Case Study of the Oswego Potluck • Stages of Prevention • Knowledge Translation • Dissemination and Implementation • Models, Theories, and Frameworks • Program Evaluation • Sustainability, Scale-up, and Spread
  3. 3. CASE STUDY
  4. 4. FIELD INVESTIGATION Field investigations involving acute problems may differ from conventional epidemiologic studies in three important ways 1. Field investigations often do not start with a clear hypothesis. 2. Acute problems involve an immediate need to protect the public and resolve the concern. 3. Field epidemiologists must decide when the available information is sufficient to take appropriate action.
  5. 5. STEPS IN FIELD INVESTIGATIONS 1. Establish the existence of an epidemic (or outbreak) 2. Confirm the diagnosis 3. Establish criteria for case identification 4. Search for missing cases 5. Count cases 6. Orient data according to person, place, and time 7. Classify the epidemic 8. Determine who is at risk of becoming a case 9. Analyze data 10.Formulate hypotheses 11.Test hypotheses 12.Develop reports and inform those who need to know 13.Execute control and prevention measures 14.Administration and planning activities
  6. 6. 1. Establish the existence of an epidemic (or outbreak) • Attack rates are appropriate statistics for investigating disease outbreaks because they describe rapidly occurring new cases of disease in a well-defined population over a limited time period • Attack rates are usually calculated by person characteristics (e.g., age, sex, race/ethnicity, and occupation) in order to identify high-risk groups
  7. 7. 2. Confirm the Diagnosis • Assessment of the clinical findings should be done to assure correctness and reliability of the findings • Clinical diagnosis by appropriately trained individuals • Laboratory diagnosis
  8. 8. 3. Establish Criteria for Case Identification • Standard clinical criteria (what) • Loose case definition vs. strict case definition • A case may be further characterized by • Who • When • Where
  9. 9. 4. Search for Missing Cases Investigation may include Physicians Clinics Health maintenance organizations Hospital emergency rooms Public health clinics Migrant health clinics Related facilities Asymptomatic persons or mild cases and their contacts should be evaluated Suspected cases vs. probable cases
  10. 10. 5. Count Cases • Exposure status and disease frequency need to be determined and compared with the appropriate at-risk population
  11. 11. 6. Orient data according to person, place, and time • Person • Inherent characteristics or people (age, race/ethnicity, sex) • Acquired characteristics (immunity or marital status) • Activities (occupation, leisure, use of medications) • Conditions (socioeconomic state, access to health care) • Place • Residence,birthplace, place of employment, school district, hospital unit, country, state, county, street address, map coordinates, etc. • Census tract • Time • Epidemic curve
  12. 12. 7. Classify the Epidemic • Common source • Propagated • Mixed
  13. 13. 8. Determine who is at risk of becoming a case • Clinical, medical, and lab findings need to be confirmed, evaluated, and analyzed for all cases to substantiate the diagnosis • Classify cases by exposure status
  14. 14. 9. Analyze Data • The epidemiologist gathers, compiles, tabulates, analyzes, and interprets the findings • Analysis often involves statistical methods: • Frequency tables • Bivariate analyses • Multiple regression
  15. 15. 10. Formulate Hypotheses • For example, in a food-borne outbreak, hypotheses should be developed for the following • Infection source, vehicle • Suspect foods • Transmission mode • Pathogen type (based on clinical symptoms, incubation periods) • Time factors in the outbreak and course of the disease • Place factors in the outbreak • Person characteristics and factors in the outbreak • Outside sources of the infection • Transmission of the disease outside of the study population • Exposed, unexposed, well, and ill cases/individuals
  16. 16. 11. Test Hypotheses • Statistical tests should be employed to evaluate hypotheses • T-test • Chi-square test • F-test • If established facts or information are lacking to substantiate a hypothesis, more information should be gathered or research hypothesis should be rejected
  17. 17. 12. Develop reports & inform those who need to know • Narrative of the investigation and review of the course of the epidemic • Tables, graphs, charts, or any useful and helpful illustrations are presented, as well as any pertinent epidemiologic data, tests, lab reports, information, and characteristics • Addresses the information presented under hypothesis
  18. 18. 13. Execute control and prevention measures • Immunization programs • Risk factor prevention • Behavior change programs
  19. 19. 14. Administration and planning activities • Organization • Coordination • Communication • Planning • Funding • Allocation
  20. 20. CASE STUDY: OSWEGO
  21. 21. Background On April 19, 1940, the local health officer in the village of Lycoming, Oswego County, New York, reported the occurrence of an outbreak of acute gastrointestinal illness to the District Health Officer in Syracuse. Dr. A. M. Rubin, epidemiologist-in-training, was assigned to conduct an investigation. When Dr. Rubin arrived in the field, he learned from the health officer that all persons known to be ill had attended a church supper held on the previous evening, April 18. Family members who did not attend the church supper did not become ill. Accordingly, Dr. Rubin focused the investigation on the supper. He completed interviews with 75 of the 80 persons known to have attended, collecting information about the occurrence and time of onset of symptoms, and foods consumed. Of the 75 persons interviewed, 46 persons reported gastrointestinal illness. 1. Establish the existence of an epidemic (or outbreak). - Would you call this outbreak an epidemic?
  22. 22. Review the steps of an outbreak investigation • The advantage of a list such as this, or an alternative version of this list, is to make sure that important steps are not missed while conducting a field investigation • However, the order of these steps is not fixed (e.g., it may be appropriate to implement control measures immediately) • In addition, the steps are often dynamic; that is, case definitions, line listings, hypotheses, and so on, can and sometimes should change with further information
  23. 23. Clinical Description • The onset of illness in all cases was acute, characterized chiefly by nausea, vomiting, diarrhea, and abdominal pain. • None of the ill persons reported having an elevated temperature; all recovered within 24 to 30 hours. • Approximately 20% of the ill persons visited physicians. No fecal specimens were obtained for bacteriologic examination.
  24. 24. Description of the Supper The supper was held in the basement of the village church. Foods were contributed by numerous members of the congregation. The supper began at 6:00 p.m. and continued until 11:00 p.m. Food was spread out on a table and consumed over a period of several hours. The approximate time of eating supper was collected for only about half the persons who had gastrointestinal illness. Data regarding onset of illness and food eaten or water drunk by each of the 75 persons interviewed are provided on Canvas: Canvas > Files > Oswego > Oswego Line Listing.pdf Are there any cases for which the times of onset are inconsistent with the general experience? How might they be explained?
  25. 25. Case Inconsistencies • Subject #52 was an 8-year-old boy who ate early (11:00 a.m.). • The boy may have eaten early and the vehicle already prepared and contaminated by 11:00 a.m. • Subject #16 was a 32-year-old woman. Perhaps she had a longer incubation period or ate later. • Other possibilities are that she had an unrelated illness, the information was incorrect, there was a data coding error, or she was a secondary case. Now, determine the median and range of the incubation period.
  26. 26. Determine the median and range of the incubation period • The median incubation period is 4 hours • Range: Minimum = 3 hours and maximum = 7 hours. The range is 4 hours. How does the information on incubation period, combined with the data on clinical symptoms, help in the differential diagnosis of the illness?
  27. 27. Differential Diagnosis • Each food-borne illness has a characteristic incubation period, specific symptoms, and foods with which it is most likely associated. • Refer to the compendium of food and waterborne illness on Canvas: Canvas > Files > Oswego > Compendium of food and water borne illness What pathogens were the most likely cause of illness given the symptoms, food types, and incubation period?
  28. 28. Differential Diagnosis • The observed incubation period is too long for heavy metals • The observed incubation period is too short for botulism (usually 12-48 hours) and most poisonous foods. • Furthermore the menu helps rule out ciguatoxin. • Likely not clostridium perfringens, bacillus cereus, vibrio parahemolyicus, salmonella, norovirus, rotavirus, E. coli enterotoxigenic, E.coli enteroinvasive, Listeria monocytogenes, Vibrio cholerae, Shigella, E. Coli 0157, Yersinia enterocolitica, cyclospora caytanensis, cryptosporidium parvum, or giardia lamblia as there was no presentation of fever. • There was no rice on the menu, so Bacillus cereus is unlikely. • Straphylococcus aureus is most likely as the incubation period (between 2 – 4 hours) fits our window, there is usually no vefer but lots of vomiting and diarrhea). Furthermore sliced ham, means, custards and creams are common.
  29. 29. Testing Hypotheses Seeing that we are dealing with a foodbourne illness (likely Straphylococcus aureus ), which food is the most likely cause of the disease? Baked Ham Cakes . Vanilla Ice Cream Chocolate Ice Cream Fruit Salad
  30. 30. Hypothesis Testing The proportion of people who ate each food item is the second most important factor to consider. In this case, three ill people denied eating the vanilla ice cream. They may have not remembered eating the vanilla ice cream, there could be multiple vehicles, or there could have been cross-contamination between foods by way of dishes, spoons, and servers. ;6.5 )183/(3 )1143/(43 (RR)    RatioRisk 10.16,95.1 813 18/3 1134 11/43 1.96ln5.6expCI(5.6)95%                      The vanilla ice cream had the highest risk ratio.
  31. 31. Outline further investigations • Detailed review of source, ingredients, preparation, and storage of incriminated food • Try to explain cases with atypical time of onset • Laboratory examination • Determine if secondary spread in family members occurred • Additional calculations (e.g., age- or gender-specific attack rates) In the mean time, what control measures would you suggest?
  32. 32. What control measures would you suggest? • Prevent consumption of remaining vanilla ice cream • Prevent recurrence of similar events in the future by educating food handlers • Ascertain whether a commercial product is involved • Eliminate any contaminated sources of food Why was it important to work up this outbreak?
  33. 33. IMPORTANCE OF FIELD INVESTIGATIONS • Rule out contamination of a commercial product. If a commercial product is involved, immediate intervention may prevent a substantial number of further cases. • Prevent future outbreaks by identifying infected food handler, specific gaps in education or food handling techniques • Public health officials need to respond to such problems in a timely manner to maintain a cooperative relationship with local health departments, private physicians, and the community • An epidemiologic explanation of the cause of the outbreak may allay community fears and concerns • The outbreak investigation may provide opportunities for investigators to answer questions about the agent, host, environment, incubation period, and so on
  34. 34. Fitting Outbreak Investigation? 1. Prepare for field work – Not discussed 2. Establish the existence of an epidemic or outbreak – Done 3. Confirm the diagnosis – Diagnosis based on signs and symptoms. No fecal specimens were obtained for bacteriologic examination. 4. Establish criteria for case identification – No case definition was given in the case study 5. Search for missing cases – Not discussed 6. Count cases – Done 7. Orient the data according to person, place, and time – Not done, with the exception that time was characterized by the epidemic curve 8. Classify the epidemic – Done: point source 9. Determine who is at risk of becoming a case – Done: those at the church super
  35. 35. Fitting Outbreak Investigation? 10. Analyze the data – Done: risk ratios involving the different foods eaten 11. Formulate hypotheses – Not formally done 12. Test hypotheses – Not formally done 13. Develop reports and inform those who need to know – A report was prepared and disseminated (i.e., see Part IV of the case study) 14. Maintain surveillance to monitor trends and execute control and prevention measures • On May 19, all remaining ice cream prepared by the Petrie sisters from raw milk from the Petrie farm was condemned. The method of contamination of ice cream is not clearly understood. Whether positive Staphylococcus nose and throat cultures occurring in the Petrie family was involved is unclear. 15. Carry out administration and planning activities – Not specified in the case study
  36. 36. STAGES OF PREVENTION
  37. 37. STAGES OF PREVENTION Primordial Prevention Populations Primary Prevention At-Risk Population Secondary Prevention Early/Asymptomatic Tertiary Prevention Diseased Prevent the emergence of risk factors which contribute to disease. e.g., teach kids to resist smoking. Prevent risk factors from contributing to illness e.g., help kids stop smoking. Screen for illnesses and stop progression of the disease. e.g., conduct cancer screenings Treat illness to prevent death or further disability. e.g., surgical intervention and chemotherapy
  38. 38. STAGES OF PREVENTION Active Prevention • Requires behavior change on part of subject • Wearing protective devices • Health promotion • Lifestyle changes • Community health education • Ensuring healthy conditions at home, school, and workplace Passive Primary Prevention • Does not require behavior change • Vitamin-fortified foods • Fluoridation of public water supplies
  39. 39. CASCADES OF CARE • Screening • Diagnosis • Linkage to care • Monitoring and Treatment • Adherence to treatment • If curable, avoid reinfection
  40. 40. KNOWLEDGE TRANSLATION
  41. 41. WHERE CAN YOU LEARN MORE ABOUT KT? • Ian Graham, Sharon Straus and Jacqueline Tetroe have produced a "primer" on knowledge translation (KT). • This book is a compilation of chapters, written by experts in a wide range of KT topics. • This is a useful reference for any future KT work
  42. 42. KNOWLEDGE TRANSLATION • Dynamic and iterative process includes synthesis, dissemination, exchange and ethically sound application of knowledge • Improves health services and products, and strengthen the health care system • Takes place within a complex system of interaction Source: Canadian Institutes of Health Research (CIHR). www.cihr-irsc.gc.ca/e/29418.html
  43. 43. PURPOSE OF KNOWLEDGE TRANSLATION • Despite the considerable resources devoted to health research, a consistent finding in the literature is that moving evidence into practice and policy is slow and haphazard • Patient care may not be evidence-based • Public – individuals and communities – cannot act on best evidence • Health policies lack sufficient, high quality, synthesized evidence • Organizations and health systems miss opportunities to improve
  44. 44. IMPORTANCE OF KNOWLEDGE TRANSLATION • 1/3 of patients don’t get treatments of proven effectiveness • 1/4 of patients get care that’s not required or is potentially harmful • Up to 3/4 of patients do not get the information they need for decision making • Up to 1/2 of physicians do not get the evidence they need for decision making Straus, S.E., Tetroe, J., Graham, I.D., & Leung, E. (2010). Knowledge to action: What it is and what it isn't. http://www.cihr-irsc.gc.ca/e/41928.html
  45. 45. IMPORTANCE OF KNOWLEDGE TRANSLATION Source: Balas E, Boren S. Managing Clinical Knowledge for Health Care Improvement. In: van Bemmel JH, McCray AT, eds. Yearbook of Medical Informatics. Stuttgart: Schattauer Verlagsgesellschaft mbH, 2000:65–70; 17 Year Delay
  46. 46. IMPORTANCE OF KNOWLEDGE TRANSLATION Brownson, Ross, et al. (2012) Dissemination and Implementation Research in Health. New York: Oxford University Press.
  47. 47. IMPORTANCE OF KNOWLEDGE TRANSLATION Source: “Why are Implementation Teams Important?” http://implementation.fpg.unc.edu/book/export/html/193 3 Year Delay with Knowledge Translation
  48. 48. DISSEMINATION & IMPLEMENTATION
  49. 49. THE EVIDENCE PYRAMID Expert Opinion Ecological Studies Case Studies Cross-Sectional Surveys Case Control Studies Cohort Studies Randomized Controlled Trials Meta-Analyses & Reviews Epidemiological Studies StrengthofCausalEvidence
  50. 50. IMPLEMENTATION SCIENCE Knowledge Translation relies on evidence for… Practice (i.e., what we are implementing) Implementation (i.e., how we are implementing it)
  51. 51. Practice Implementation Science DISSEMINATION VERSUS IMPLEMENTATION Dissemination Dissemination Practice Purposive distribution of information and intervention materials to a specific audience. The intent is to spread information. (NIH) Implementation Practice The use of strategies to adopt and integrate evidence-based interventions and change practice within specific settings. (NIH) Dissemination Science The scientific study of processes and variables that determine and/or influence the spread/sharing of knowledge to various stakeholders. Implementation Science The scientific study ofthe methods to promote the uptake of research findings in clinical, organizational, or policy contexts. (Implementation Science journal) Sources: Dissemination and Implementation Science. (n.d.). National Institutes of Health. https://www.nlm.nih.gov/hsrinfo/implementation_science.html; Implementation Science. http://www.springer.com/public+health/health+promotion+%26+disease+prevention/journal/13012
  52. 52. RETURN ON INVESTMENT Sources: Dissemination and Implementation Science. (n.d.). National Institutes of Health. https://www.nlm.nih.gov/hsrinfo/implementation_science.html; Implementation Science. http://www.springer.com/public+health/health+promotion+%26+disease+prevention/journal/13012 Practice ImplementationDissemination Dissemination Practice 1-2% change in outcomes Implementation Practice ~ 10% change in outcomes
  53. 53. ACTIVITY In your small group, read the scenarios provided below and identify if the scenarios are describing Dissemination Practices, Dissemination Science, Implementation Science, or Implementation practice: • Developing and distributing policy briefs and research summaries to policy makers. • Comparing two different website layouts to determine which format patients find more useable. • A group wants to evaluate the most effective strategies to train new staff to effectively use the screening tools related to their role. • Health facilities were provided with a set of tools, training, and support to decrease staff practices that were associated with hospital acquired infections. • Assessing which tool helps counselors have better adherence to cognitive behavioural therapy. • A profession association collaborated with provincial agencies to make sure healthcare professionals represented by the association had received copies of their new clinical guidelines.
  54. 54. ACTIVITY – ANSWER KEY In your small group, read the scenarios provided below and identify if the scenarios are describing Dissemination Practices, Dissemination Science, Implementation Science, or Implementation practice: • Developing and distributing policy briefs and research summaries to policy makers. • Comparing two different website layouts to determine which format patients find more useable. • A group wants to evaluate the most effective strategies to train new staff to effectively use the screening tools related to their role. • Health facilities were provided with a set of tools, training, and support to decrease staff practices that were associated with hospital acquired infections. • Assessing which tool helps counselors have better adherence to cognitive behavioural therapy. • A profession association collaborated with provincial agencies to make sure healthcare professionals represented by the association had received copies of their new clinical guidelines.
  55. 55. MODELS, THEORIES, AND FRAMEWORKS
  56. 56. Models, Theories and Frameworks MTF Description Reference KTA . Knowledge to Action model is a process model that defines and describes Knowledge Translation processes and outlines strategies for building Knowledge Translation capacity Graham, D., et al. (2006). Lost in translation: Time for a map?. Journal of continuing education in the health professions, 26 (1): 13-24. COM-B Capability, Opportunity, Motivation – Behaviour is a behavior change theory. Michie, S. et al. (2011). The behaviuor change wheel: A new method for characterizing and designing behaviour change interventions. Implementation Science, 6:42. TDF Theoretical Domains Framework is a framework that was developed to make behavior change theories more accessible in relation to implementation practices. Cane, J., et al. (2012). Validation of the theoretical domains framework for use in behaviour change and implementation research. Implementation Science, 7(1). CFIR Consolidated Framework for Implementation Damschroder, L., et al. (2009). Fostering Research is a framework that can be used implementation of health services research findings into practice: a to understand the implementation context. consolidated framework for advancing implementation science. Implementation Science, 4:50.
  57. 57. Models, Theories and Frameworks MTF Description Reference QIF . Quality Implementation Framework is a framework that provides a synthesized overview of critical steps of implementation that can be used as a guide for practice and research. Wandersman, et al. (2012). The Quality Implementation Framework: ASynthesis of Critical Steps in the Implementation Process. American Journal of Community Psychology, 50(3-4):462-80. doi: 10.1007/s10464-012-9522-x. ISF Interactive Systems Framework is a framework that outlines the people/organizations/contexts needed to move effective prevention into the field. Wandersman, A., et al. (2008).Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American journal of community psychology, 41(3-4): 171- 181. RE-AIM A framework for evaluating public health interventions that assesses 5 dimensions: reach, efficacy, adoption, implementation, and maintenance. These dimensions occur at multiple levels (e.g., individual, clinic or organization, community) and interact to determine the public health or population- based impact of a program or policy. © Copyrighted by St. Michael’s Hospital2017. Glasgow RE, Vogt TM, Boles SM: Evaluating the public health impact of health promotion interventions: the RE- AIM framework. Am J PublicHealth. 1999, 89: 1322-1327. 10.2105/AJPH.89.9.1322.
  58. 58. KNOWLEDGE-TO-ACTION (KTA) FRAMEWORK Source: Graham ID et al. JCHEP 2006;26:13-24. • Knowledge generation and the implementation of existing and new solutions is an intricate cyclical process that has been summarized by Graham et al. as the "knowledge-to-action" framework. • This is widely used in the Canadian healthcare context. • Many other models fit within this overall “meta-framework.” • For example we might use the COM-B to assess barriers and facilitators…
  59. 59. COM-B Source: Michie S, Atkins L, West R. The behaviour change wheel: a guide to designing interventions. Great Britain: Silverback Pub; 2014. COM-B MODEL OF BEHAVIOUR CHANGE Capability Opportunity Motivation Behaviour • Capability • Physical • Physical Skills • Psychological • The capacity to engage in necessary though processes, comprehension, and reasoning.
  60. 60. COM-B Source: Michie S, Atkins L, West R. The behaviour change wheel: a guide to designing interventions. Great Britain: Silverback Pub; 2014. COM-B MODEL OF BEHAVIOUR CHANGE Capability Opportunity Motivation Behaviour • Motivation • Reflective • Evaluations and Plans • Automatic • Innate disposition, emotion, impulse, and learned behaviour.
  61. 61. COM-B Source: Michie S, Atkins L, West R. The behaviour change wheel: a guide to designing interventions. Great Britain: Silverback Pub; 2014. COM-B MODEL OF BEHAVIOUR CHANGE Capability Opportunity Motivation Behaviour • Opportunity • Physical • Opportunity afforded by environment • Social • Opportunity afforded to by culture and social environment.
  62. 62. ACTIVITY • The behavioural problem: sedentary lifestyles in adults living in deprived urban areas • In pairs/groups spend 10mins discussing which of the six COM-B components might be barriers or facilitators to increasing physical activity in in adults living in deprived urban areas • Feedback to the rest of the group • Remember! At this stage we are thinking about what needs to change, not how to change it (yet!)
  63. 63. The Behaviour Change Wheel Intervention Functions • Restrictions • Environmental Restructuring • Modelling • Enablement • Training • Coercion • Incentivization • Persuasion • Education
  64. 64. The Behaviour Change Wheel Policy Areas • Environmental and Social Planning • Guidelines • Fiscal measures • Regulation • Service Provision • Legislation • Communication and Marketing
  65. 65. EXAMPLE STRATEGIES AND THEIR INTERVENTION TARGETS Implementation Strategies Definition Target Level Intervention Function Conduct educational meetings Hold meetings involving program targets (e.g., providers, administrators, other organizational stakeholders, and community, patient/consumer, and family stakeholders) to improve knowledge about the ideal practice. Patient, Provider Education Work with educational institutions Encourage educational institutions to train providers in the ideal practice. System Training Model and simulate change Have experts/leaders/respected colleagues model or simulate the ideal practice. Organization, Provider Modelling Prepare patients/consum ers to be active participants Prepare patients/consumers to be active in their care - e.g., to ask questions about the ideal practice, and evidence behind the ideal practice. Patient, Provider Enablement Alter payments to health workers Change ways in which providers are paid for providing the ideal practice. System Enablement
  66. 66. EXAMPLE STRATEGIES AND THEIR INTERVENTION TARGETS Implementation Strategies Definition Target Level Intervention Function Change record systems Change records systems to allow better capturing of patient information and assessment of implementation or clinical outcomes related to the ideal practice; for example electronic patient records, or systems for recalling patients for follow-up or prevention e.g., immunization. System, Organization Environmental restructuring Change service sites Change the setting where the ideal practice is provided; for e.g., home vs. healthcare facility, inpatient vs outpatient, specialized vs. non specialized facility, walk in clinics, medical day hospital, mobile units. System, Organization Environmental restructuring Alter incentive/ allowance structures Work to incentivize or disincentivize the adoption and implementation of the ideal practice. System, Organization Incentivization Change accreditation or membership requirements Strive to alter accreditation standards so that they require or encourage use of the ideal practice. Work to alter membership organization requirements so that those who want to affiliate with the organization are encouraged or required to use the ideal practice. System, Organization Incentivization
  67. 67. EXAMPLE STRATEGIES AND THEIR INTERVENTION TARGETS Implementation Strategies Definition Target Level Intervention Function Conduct local consensus discussions Engage local providers and other stakeholders in discussions about whether the chosen problem is important and whether the selected practice to address it is appropriate; e.g., agreeing on a clinical protocol to manage a patient group or adapting a guideline for a local health system. Organization, Provider, Patient Persuasion Audit and provide feedback Collect and summarize performance data related to the ideal practice over a specified time period and give it to providers and administrators to monitor, evaluate, and modify behavior. Provider Persuasion Create or change credentialing and/or licensure standards Create or change credentialing and/or licensure standards related to the ideal practice. System Restriction Develop/alter scope of practice standards Develop evidence-based policies that regulate what health professionals are able to do in their role, or alter existing scope of practice standards to include the ideal practice. System Restriction
  68. 68. WHICH INTERNVENTION TO USE? Example: individual level barriers and facilitators to implementing prostate cancer screening guideline • “I’m afraid of missing a diagnosis of prostate cancer if I don’t screen.” • “It’s hard to have a conversation with patients about harms and benefits.” • “Patients want to be screened, and it’s easier to just say yes to them and do it.” • “I used the ‘Dots Poster’ for prostate cancer and that really helped give me the confidence I needed have a good conversation about harms and benefits with patients.”
  69. 69. WHICH INTERNVENTION TO USE? Example: individual level barriers and facilitators to implementing prostate cancer screening guideline • “The guideline recommendations don’t align with the standard practices within our organization” • “We don’t have the available resources to follow some of the guideline recommendations.” • “The guideline recommendations need to be mandated by our Health Authority before we can implement these recommendations.”
  70. 70. EVIDENCE OF EFFECTIVENESS Implementation intervention Evidence of effectiveness on outcomes (reviews with AMSTAR scores ≥ 8); Magnitude ofeffect Educational materials Effective for improving target care outcomes (13 reviews;);2.0% improvement in care (IQR 0.0% to +11.0%)1 Reminders Effective for improving care and prescribing outcomes (14 reviews);11.2% (IQR +6.5% to +19.5%) 3 Audit and feedback Effective for improving care and prescribing outcomes when used aloneor with other implementation strategies (8 reviews); 4.3% (IQR +0.5%to +16%) 2 Identify and prepare champions Effective for improving care outcomes when used alone or with other implementation strategies (2 reviews); N/A On-going training Effective for improving care and prescribing behaviour when used aloneor in combination with audit and feedback (18 reviews);N/A Patient-mediated interventions Effective for improving care and behaviour outcomes (3 reviews); N/A 1. Giguère et al. Printed educational materials: effects on professional practice and healthcare outcomes. The Cochrane Library. 2012 Jan1. 2. Ivers et al. No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implementation Science.2014 Jan 17;9(1):14. 3. Arditi et al. Computer‐generated reminders delivered on paper to healthcare professionals; effects on professional practice and health care outcomes.The
  71. 71. EVIDENCE FOR EFFECTIVENESS
  72. 72. PROGRAM EVALUATION
  73. 73. Where does ‘evaluation’ fit in the KTA process model? Process Outcome Impact Formative
  74. 74. Observation Implementer Self-Report Administrative Documents Participant Self-Reports WHERE TO LOOK?
  75. 75. What is a formative evaluation? Source: Introduction to program evaluation for public health programs: Centers for Disease Control and Prevention, 2011. http://www.cdc.gov/eval/guide/cdcevalmanual.pdf What it shows: • Needs • How target audience receives information • Ideal champions/ spokespeople • Challenges in using program content Why it is useful: • Making sure there’s a need • Clarification • Make revisions • Maximizes the likelihood that the program will succeed When to use it: • Program development or revisions • Examples: • Needs assessment • Usability testing • Implementation planning • Stakeholder engagement
  76. 76. What is a process evaluation? Source: Introduction to program evaluation for public health programs: Centers for Disease Control and Prevention, 2011. http://www.cdc.gov/eval/guide/cdcevalmanual.pdf What does it show? • Activities conducted • Characteristics of activities conducted • (e.g., how many people are participating in the program and how many people are not) Why is it useful? • Identifies challenges in reaching the target population • Allows programs to evaluate how well their plans, procedures, activities, and materials are working and to take timely, corrective actions When is it done? • As soon as the program begins
  77. 77. What is an outcome evaluation? Source: Introduction to program evaluation for public health programs: Centers for Disease Control and Prevention, 2011. http://www.cdc.gov/eval/guide/cdcevalmanual.pdf What it shows: • The degree to which a program is meeting its goals • For example, changes in knowledge, skills, attitudes, beliefs, behaviour, patient/other outcomes Why it is useful: • To make adaptations • Is program moving toward achieving goals When to use it: • After the program has made contact with at least one person or one group of people in the target population
  78. 78. What is an impact evaluation? Source: Introduction to program evaluation for public health programs: Centers for Disease Control and Prevention, 2011. http://www.cdc.gov/eval/guide/cdcevalmanual.pdf What it shows: • The degree to which the program has met its ultimate goals Why it is useful: • Allows programs to learn from their successes and failures and to incorporate what they have learned into their next project • Provides evidence of success for use in future requests for funding When to use it: • For ongoing programs • For one-time programs
  79. 79. Different frameworks can be used to guide evaluation Source: Introduction to program evaluation for public health programs: Centers for Disease Control and Prevention, 2011. Evaluation steps/components to consider • RE-AIM (Glasgow, R. E. et al., 1999) • PRECEDE-PROCEED (Green, L. 1968) Implementation factors to assess in your evaluation plan • CFIR (Damschroder, et al., 2009) • PARiHS (Harvey & McCormack,1998)
  80. 80. RE-AIM is a commonly used evaluation framework Source: Introduction to program evaluation for public health programs: Centers for Disease Control and Prevention, 2011. • Number, proportion, or representativeness of individuals participating Reach • Impact of the intervention on outcomesEffectiveness • Number, proportion, or representativeness of settingsAdoption • How well the intervention was implementedImplementation • Sustainability of the individual changes, program delivery, and outcomes Maintenance http://www.cdc.gov/eval/guide/cdcevalmanual.pdf; Glasgow, 1999
  81. 81. How do we evaluate Reach? Source: National Cancer Institute (2013). Research-Tested Intervention Programs (RTIPs). https://rtips.cancer.gov/rtips/reAim.do Measure the number, proportion, or representativeness of individuals who were recipients of the ideal practice • # of potential participants approached • # of participants deemed ineligible to participate • # of participants that actually participated • % of all eligible invited participants who accepted participation • Characteristics of participants compared with nonparticipants
  82. 82. How do we evaluate Effectiveness? Source: National Cancer Institute (2013). Research-Tested Intervention Programs (RTIPs). https://rtips.cancer.gov/rtips/reAim.do Measure impact of the ETP on outcomes • Did the ETP make a difference? • How much of a difference (effect size or the magnitude of change)? • Did some program recipients benefit more than others (subgroup analysis)?
  83. 83. How do we evaluate Adoption? Source: National Cancer Institute (2013). Research-Tested Intervention Programs (RTIPs). https://rtips.cancer.gov/rtips/reAim.do Measure number, proportion, or representativeness of settings/sites that decided to adopt the ETP • # of settings in a given population qualified to host the ETP • # of settings that were interested in participating • # of settings that were not appropriate for the study • # of settings that met criteria and chose to participate • % of the total number of available settings that actually participate • Characteristics of participating settings compared with nonparticipating settings
  84. 84. How do we evaluate Implementation? Source: Durlak & DuPre (2008). AM Community Psychol; 41(3-4):327-50. doi: 10.1007/s10464-008-9165-0 Measure how well the strategies were implemented (i.e., implementation quality) • Dosage • Adherence/ Fidelity • Adaptation • Quality of delivery • Participant responsiveness • Program reach
  85. 85. WHAT ARE YOU EVALUATING? When evaluating the implementation of an evidence based project project, you are concerned with “implementation quality.” 1. Dosage 2. Adherence/ Fidelity 3. Adaptation 4. Quality of delivery 5. Participant responsiveness 6. Reach Source: Durlak & DuPre (2008). AM Community Psychol; 41(3-4):327-50. doi: 10.1007/s10464-008-9165-0
  86. 86. How do we measure implementation quality? How much of the original was delivered? 1. Dosage Source: Durlak & DuPre (2008). AM Community Psychol; 41(3-4):327-50. doi: 10.1007/s10464-008-9165-0
  87. 87. How do we measure implementation quality? 2. Adherence/ Fidelity How close was the implementer to the originally intended form? Source: Durlak & DuPre (2008). AM Community Psychol; 41(3-4):327-50. doi: 10.1007/s10464-008-9165-0
  88. 88. How do we measure implementation quality? 3. Adaptation What changes were made during the implementation process? Source: Durlak & DuPre (2008). AM Community Psychol; 41(3-4):327-50. doi: 10.1007/s10464-008-9165-0
  89. 89. How do we measure implementation quality? 4. Quality of delivery What was the quality of delivery (affective quality)? Source: Durlak & DuPre (2008). AM Community Psychol; 41(3-4):327-50. doi: 10.1007/s10464-008-9165-0
  90. 90. How do we measure implementation quality? How interested and receptive were participants? 5. Participant responsiveness Source: Durlak & DuPre (2008). AM Community Psychol; 41(3-4):327-50. doi: 10.1007/s10464-008-9165-0
  91. 91. How do we measure implementation quality? 6. Reach Who is being reached (how representative they are of the target audience)? Source: Durlak & DuPre (2008). AM Community Psychol; 41(3-4):327-50. doi: 10.1007/s10464-008-9165-0
  92. 92. SUSTAINABILITY, SCALE-UP, AND SPREAD
  93. 93. © Copyrighted by St. Michael’s Hospital2017. The materials are intended for non-commercial use only. No part of the materials may be used for commercial purposes without the written permission of the What happens after you have successfully implemented? Sustainability Intermediary copyright owner.
  94. 94. Source: Graham ID et al. JCHEP 2006;26:13-24. Where is sustainability on the KTA process model?
  95. 95. Why does sustainability matter?
  96. 96. Why plan for sustainability? o Planning for sustainability makes it more likely that you will sustain the intervention o Not sustaining the implementation intervention decreases the chances that the intervention will be adopted in the future Source: Flagan & Flay (2009). Health Educ Behav;36(1):9-23. doi: 10.1177/1090198106291376; Johnson, Hays, Center, & Daley (2004) Building capacity and sustainable prevention innovations: a sustainability planning model. Evaluation and Program Planning; 27:135-49. doi: 10.1016/j.evalprogplan.2004.01.002; Bumbarger & Perkins (2008). Journal of Children’s Services; 3:55–64
  97. 97. RELATED TERMS TO SUSTAINABILITY Davies B. In Knowledge Translation in Health Care, second edition. Wiley, In Press
  98. 98. SUSTAINABILITY What do we know about sustainability? • Literature on sustainability remains theoretical, with little guidance on how to sustain • 4 systematic reviews • 11.5% (n=24) of the articles provided a definition of sustainability • Few models of sustainability have been rigorously tested Source: Stirman et al. (2012). Implementation Science; 7:17. doi: 10.1186/1748-5908-7-17; Gruen et al. (2008). Lancet; 372(9649):1579-89. doi: 10.1016/S0140-6736(08)61659-1; Ament et al. (2015). BMJ Open; 5(12). doi: 10.1136/bmjopen-2015-00807; Tricco et al., (2016). Implementation Science; 11:55. doi: 10.1186/s13012-016-0421-7
  99. 99. DEFINING SUSTAINABILITY 1. After a defined period of time 2. a program or implementation strategies continue to be delivered; 3. behavior change aligned with 4. ideal practice is maintained; 5. the implementation strategies (2) and ideal practice (3) may evolve or adapt, 6. while continuing to produce benefits for individuals/systems. Source: Moore, Mascarenhas, Bain, & Straus (2017). Implementation Science; 12(1):110. doi: 10.1186/s13012-017-0637-1.
  100. 100. SUSTAINABILITY 1. After a defined period of time • Refers to a specific period of time after implementation • Defined by implementers • Time period depends on the ideal practice or program and the outcomes of interest Source: Moore, Mascarenhas, Bain, & Straus (2017). Implementation Science; 12(1):110. doi: 10.1186/s13012-017-0637-1.
  101. 101. SUSTAINABILITY 2. A program or implementation strategies continue to be delivered • An organization or community continues to deliver implementation strategies Source: Moore, Mascarenhas, Bain, & Straus (2017). Implementation Science; 12(1):110. doi: 10.1186/s13012-017-0637-1.
  102. 102. SUSTAINABILITY 3. Behavior change aligned with ideal practice is maintained • New ways of working become the norm • An individual continues to follow the recommendations of the evidence-based program, guideline, or practice Source: Moore, Mascarenhas, Bain, & Straus (2017). Implementation Science; 12(1):110. doi: 10.1186/s13012-017-0637-1.
  103. 103. SUSTAINABILITY 4. Implementation strategies and/or ideal practice may evolve or adapt • The implementation strategies may evolve in response to changes inside or outside of the host organization or community • The evidence-based practice may evolve in response to changes in the host organization, a community, or new evidence. Source: Moore, Mascarenhas, Bain, & Straus (2017). Implementation Science; 12(1):110. doi: 10.1186/s13012-017-0637-1.
  104. 104. SUSTAINABILITY 5. While continuing to produce benefits for individuals/systems • The program or ideal practice continues to: • Show a positive impact on health outcomes • Reduce costs for organization/community • Display advantages over previous practices or programs • Display advantages over new practices or programs Source: Moore, Mascarenhas, Bain, & Straus (2017). Implementation Science; 12(1):110. doi: 10.1186/s13012-017-0637-1.
  105. 105. SUSTAINABILITY How do you measure sustainability (maintenance)? • Reach o Continue to reach the intended target audience? • Effectiveness o Does the program continue to produce positive effects? oAre these effects maintained over time? • Adoption o Are sites continuing to implement the program? • Implementation o Are sites continuing to implement with high quality?
  106. 106. WHAT IS “SPREAD”? Diffusion: “The process by which an innovation is communicated through certain channels over time among the members of a social system” Source: Rogers (2003). Diffusion of innovations. New York: Free Press.
  107. 107. HOW DO YOU SPREAD? What are examples of strategies that can be used to spread an ETP? Develop educational materials • Develop and format manuals, toolkits, and other supporting materials in ways that make it easier for stakeholders to learn about the ETP and how to implement the ETP Through a dissemination organization • Identify or start a separate organization that is responsible for disseminating the ETP. It could be a for-profit or non-profit organization
  108. 108. WHAT IS “SCALE-UP”? Deliberate efforts to increase the impact of innovations successfully tested in pilot or experimental projects so as to benefit more people and to foster policy and program development on a lasting basis. Source: Simmons, Ruth, Fajans, Peters & Ghiron, Laura (2007). Scaling up health service delivery: from pilot innovations to policies and programmes / edited by Ruth Simmons, Peter Fajans, and Laura Ghiron. Geneva : World Health Organization. http://www.who.int/iris/handle/10665/43794
  109. 109. HOW DO YOU SCALE UP? Institute for Healthcare improvement (IHI) Framework for Going to Full Scale Can be used to scale up health interventions. Source: Barker, Reid, & Schall (2016). Implementation Science; 11(1):12. doi: 10.1186/s13012-016-0374-x
  110. 110. SPREAD VERSUS SCALE-UP Spread Horizontal diffusion or actively disseminating best practice and knowledge and implementing each intervention in every available care setting Source: Institute for HealthcareImprovement (2008). Scale up Vertical diffusion or deliberate, systematic approaches to increasing the coverage, range, and sustainability of services Source: Eaton et al. (2011). Lancet;378(9802):1592- 603). doi: 10.1016/S0140-6736(11)60891-X

×