The document discusses problems with current approaches to quality improvement (QI) research and implementation science. It argues that QI research often lacks consideration of context and implementation processes. Multilevel and mixed methods designs are proposed to better account for these factors. Complexity science principles are also discussed as relevant to understanding implementation within organizational systems. The document calls for building more capacity for implementation research through multidisciplinary teams, long-term studies, and involvement of end users.

1. The Science and Practice of Implementation : Are We headed Down the Right Path? 10th Biennial Regenstrief Conference Emerging Perspectives on Transformational Change in Healthcare Systems October 2 – October 4, 2007. Jeff Alexander, Ph.D. Department of Health Management and Policy University of Michigan School of Public Health

2. Goals Problems with Quality Improvement Research- why it’s not contributing to systems change State of the art- Implementation Research- good, bad, and ugly Modest proposals for advancing the science of implementation and the usefulness of QI research

3. State of the Art – QI Research Single organizational samples Opportunistic not systematic Short duration studies No replication of studies No explicit consideration of context No explicit consideration of implementation

4. Bottom line: We don’t know what works, when it works, or where it works

5. BEYOND THE LINEAR MODEL Basic Research Clinical Trial (Efficacy) Treatment Development Effectiveness Trial Treatment Deployment

6. Problems with the Linear Model of Implementation Little on causal pathways & nested interconnected structures and activities Little influence of OT & OB in QI studies RCT thinking: control context away

7. RCTs as the Gold Standard? Great for testing efficacy of molar interventions Not so great for assessing: Process related phenomena Complex interactions among program components Contextual effects implementation

8. Background Health care provided in organizational context behavior of clinicians influenced by the organizations in which they work recognition of the interconnections among components of organizations (clinical teams function within hospitals, interact with other clinical teams, support systems - embeddedness

9. Implementation-State of the Art Emerging lists of “best practices” Continued assumption, encouraged by funding streams, of linear development of interventions Anecdotal information on implementation Some efforts to produce models, theories to test implementation

10. Implementation: the influence of content, context, and process Implementation Content Process Opinion leaders, change champion Systemic processes (e.g., supervisory practices, quality improvement) Organizational learning Triability Innovation type Evidence interpretation and packaging Internal: Organizational culture Organizational structure Practice setting characteristics Local stakeholders (e.g., attitudes and behaviors) Resources External: Networks regulation Economic (e.g., reimbursement) Competition Context

11. Klein and Sora Model Management Support : Management communicates a rationale and priority Implementation Effectiveness : Consistency and quality of innovation use Innovation-Values Fit : The perceived fit between the end-user's values and the innovation Champion(s) : Champion(s) promotes the innovation with targeted org members and/or management Financial Resource Availability : Resources are made available to support implementation policies and practices Implementation Climate : The innovation is perceived as an organizational priority by targeted end users Implementation Policies and Practices : Formal organizational actions ensure user skills, create incentives and/or identify and address barriers to use

12. Why Context Matters Context may affect implementation directly Context may moderate the relationship between an innovation and outcomes of interest Context may establish the external validity of both implementation and QI research

13. Problems with Implementation Context Measurement and Analyses assigning the same group value to all members of a group aggregating individual outcomes to the group level Separate analyses of organizational and individual phenomena

14. Advantages of Multilevel Designs statistically efficient estimates of regression coefficients Use of clustering information provides correct standard errors, confidence intervals and significance tests Allows for uneven assessments and different program tenures (for longitudinal studies)

15. Advantages of MLD Measurement at any of the levels of a hierarchy enables examination of whether differences in average outcomes between organizations are explained by factors such as organizational practices/structures , or other characteristics of individual patients or providers

17. Potential applications of MLD Effects of organizational infrastructure on implementation in micro teams Effects of org. culture on individual provider attitudes and behavior (e.g. physician use of clinical guidelines) Translational research Effects of micro-team structure and process on patient outcomes

18. Issues with Multilevel Analysis Data requirements Statistical power Analysis and interpretation issues

19. Multi-method Designs Quantitative-Qualitative RCT-case study Process study-outcome study Sustainability- long term studies

20. Life Cycle of Quality Improvement

21. Science of Complexity Assumptions regarding interactions among components or “agents” of the system Heterogeneity- agents differ in important characteristics (e.g. preferences) Dynamic-agents change, how system changes are non linear, chaotic Feedback- change often results from feedback that agents receive from their own behavior

22. Complexity Science Organization- agents organized into groups or hierarchies that influence how system evolves over time Emergent behavior- what results from the actions and interaction of individual agents

23. Engaged Scholarship Theory Solution Model Reality Problem Formulation Theory Building Research Design Problem Solving Describe Problem/Issue - visit & study it - map & diagnose it Formulate the Question - from users’ perspective? Criterion - Relevance Answers & Arguments - plausible alternatives - clarify context - identify key variations - cross levels of abstraction Criterion - Validity Obtain the Evidence - case/field/experimental study - unit selection & sampling - measurement & observation - data analysis Criterion - Truth Application & Implementation - knowledge for what? who? - for science & profession - apply findings to problem - develop implementation plan Criterion - Impact

24. Capacity Building for Implementation Research Data Funding Multi-disciplinary teams Make implementation part of the intervention Bring in users of intervention/innovation Long term studies Basic research on implementation

25. Other questions What aspects of care are modular and what aspects are inherently interdependent? To what extent can one element of a system be altered without consequences to other elements? Should intervention content and context dictate implementation process?