Alexandermultilevel Analysis4

The Science and Practice of Implementation : Are We headed Down the Right Path? 10th Biennial Regenstrief Conference Emerging Perspectives on Transformational Change in Healthcare Systems October 2 – October 4, 2007. Jeff Alexander, Ph.D. Department of Health Management and Policy University of Michigan School of Public Health

Goals
Problems with Quality Improvement Research- why it’s not contributing to systems change
State of the art- Implementation Research- good, bad, and ugly
Modest proposals for advancing the science of implementation and the usefulness of QI research

State of the Art – QI Research
Single organizational samples
Opportunistic not systematic
Short duration studies
No replication of studies
No explicit consideration of context
No explicit consideration of implementation

Bottom line:
We don’t know what works, when it works, or where it works

BEYOND THE LINEAR MODEL Basic Research Clinical Trial (Efficacy) Treatment Development Effectiveness Trial Treatment Deployment

Problems with the Linear Model of Implementation
Little on causal pathways & nested interconnected structures and activities
Little influence of OT & OB in QI studies
RCT thinking: control context away

RCTs as the Gold Standard?
Great for testing efficacy of molar interventions
Not so great for assessing:
Process related phenomena
Complex interactions among program components
Contextual effects
implementation

Background
Health care provided in organizational context
behavior of clinicians influenced by the organizations in which they work
recognition of the interconnections among components of organizations (clinical teams function within hospitals, interact with other clinical teams, support systems - embeddedness

Implementation-State of the Art
Emerging lists of “best practices”
Continued assumption, encouraged by funding streams, of linear development of interventions
Anecdotal information on implementation
Some efforts to produce models, theories to test implementation

Implementation: the influence of content, context, and process Implementation Content
Process
Opinion leaders, change champion
Systemic processes (e.g., supervisory practices, quality improvement)
Organizational learning
Triability
Innovation type
Evidence interpretation and packaging
Internal:
Organizational culture
Organizational structure
Practice setting characteristics
Local stakeholders (e.g., attitudes and behaviors)
Resources
External:
Networks
regulation
Economic (e.g., reimbursement)
Competition
Context

Klein and Sora Model Management Support : Management communicates a rationale and priority Implementation Effectiveness : Consistency and quality of innovation use Innovation-Values Fit : The perceived fit between the end-user's values and the innovation Champion(s) : Champion(s) promotes the innovation with targeted org members and/or management Financial Resource Availability : Resources are made available to support implementation policies and practices Implementation Climate : The innovation is perceived as an organizational priority by targeted end users Implementation Policies and Practices : Formal organizational actions ensure user skills, create incentives and/or identify and address barriers to use

Why Context Matters
Context may affect implementation directly
Context may moderate the relationship between an innovation and outcomes of interest
Context may establish the external validity of both implementation and QI research

Problems with Implementation Context Measurement and Analyses
assigning the same group value to all members of a group
aggregating individual outcomes to the group level
Separate analyses of organizational and individual phenomena

Advantages of Multilevel Designs
statistically efficient estimates of regression coefficients
Use of clustering information provides correct standard errors, confidence intervals and significance tests
Allows for uneven assessments and different program tenures (for longitudinal studies)

Advantages of MLD
Measurement at any of the levels of a hierarchy enables examination of whether differences in average outcomes between organizations are explained by factors such as organizational practices/structures , or other characteristics of individual patients or providers

Potential applications of MLD
Effects of organizational infrastructure on implementation in micro teams
Effects of org. culture on individual provider attitudes and behavior (e.g. physician use of clinical guidelines)
Translational research
Effects of micro-team structure and process on patient outcomes

Issues with Multilevel Analysis
Data requirements
Statistical power
Analysis and interpretation issues

Multi-method Designs
Quantitative-Qualitative
RCT-case study
Process study-outcome study
Sustainability- long term studies

Life Cycle of Quality Improvement

Science of Complexity
Assumptions regarding interactions among
components or “agents” of the system
Heterogeneity- agents differ in important characteristics (e.g. preferences)
Dynamic-agents change, how system changes are non linear, chaotic
Feedback- change often results from feedback that agents receive from their own behavior

Complexity Science
Organization- agents organized into groups or hierarchies that influence how system evolves over time
Emergent behavior- what results from the actions and interaction of individual agents

Engaged Scholarship Theory Solution Model Reality Problem Formulation Theory Building Research Design Problem Solving Describe Problem/Issue - visit & study it - map & diagnose it Formulate the Question - from users’ perspective? Criterion - Relevance
Answers & Arguments
- plausible alternatives
- clarify context
- identify key variations
- cross levels of abstraction
Criterion - Validity
Obtain the Evidence - case/field/experimental study - unit selection & sampling - measurement & observation - data analysis Criterion - Truth Application & Implementation - knowledge for what? who? - for science & profession - apply findings to problem - develop implementation plan Criterion - Impact

Capacity Building for Implementation Research
Data
Funding
Multi-disciplinary teams
Make implementation part of the intervention
Bring in users of intervention/innovation
Long term studies
Basic research on implementation

Other questions
What aspects of care are modular and what aspects are inherently interdependent?
To what extent can one element of a system be altered without consequences to other elements?
Should intervention content and context dictate implementation process?

Alexandermultilevel Analysis4

by ShawnHoke on Oct 06, 2007

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 1

Statistics

Alexandermultilevel Analysis4 — Presentation Transcript