Training in Implementation: Actionable Research Approaches (TIARA): Module 4, Practical Considerations

1. Implementation Science and
Practice
Real World Considerations
Anne Sales
Department of Learning Health Sciences
University of Michigan Medical School

2. Is it research? Or is it QI?

4. https://www.hhs.gov/ohrp/regul
ations-and-policy/decision-
charts/index.html#c1

5. Key points
• Is the activity a systematic
investigation designed to
develop or contribute to
generalizable knowledge?
• Does the research involve
obtaining information about
living individuals?
• Does the research involve
intervention or interaction with
the individuals?
• Is the information individually
identifiable?

6. Some additional considerations
• Is the work research-funded?
• If yes, then rules related to the
funding may apply
• But it may still depend
• If no, it may still depend
• Are there any legal agreements
specifying that activities may
only be used for quality
improvement?
• If yes, then it probably is QI
• But it may still depend

7. Important considerations
• Quality improvement
• Evidence: sufficient to support
practice change?
• Clinical staff: are local staff who are
already employed the people doing
the work?
• Methods: are they flexible and
adaptable?
• Population: current patients?
• Consent: is current clinical consent
sufficient?
• Benefits: current patients now?
• Risk: greater than current standard of
care?
• Research
• Evidence: discovery of new evidence,
or implementation of existing?
• Clinical staff: research staff?
• Methods: are they focused on
internal validity?
• Population: selected sample based on
strict inclusion/exclusion criteria?
• Consent: is specific research consent
required?
• Benefits: future patients?
• Risk: greater than current standard of
care?

8. An example
Transition to Geoff’s slides

9. A few more considerations
Considering science– issues around theory use, development, and testing

10. Utility of theory
• We use theory all the time
• Mostly without being explicit
• Explicit theory is more useful than implicit theory
• Why do you think something will work?
• Think about causal logic

11. Two types of feedback report

12. Implicit theory
• Designed for someone sitting at
their desk
• Need time to work through each line
of the report
• Cognitive burden is an important
consideration
• Requires understanding of the
elements of each report
• Comprehension requires detailed
knowledge of business
• Confidence that the data come from
reliable sources
• Report assumes that the recipient
understands and trusts the data
sources
• Designed for someone busy doing
clinical work
• Should be assimilable at a glance
• Cognitive burden varies by the amount
of time available
• Simple elements
• Draws the eye to important
information
• Description of data sources allows the
person to investigate if they want
to/have data access
• Describing the source gives contextual
information to the recipient

13. Overarching theory of feedback
• Underlying theoretical proposition:
• If people know their performance is not “as good as” the performance of similar other
people, they will act to improve it Individuals respond to knowledge of performance
• Their response depends on a number of factors
• Understanding the size of the gap
• Understanding trends over time
• Comparing their performance to others
• Knowing the source of the information in the report
• Cognitive ability to process the information
• Time
• Energy
• Attention
• Perception of their environment and social context– how is performance perceived as poor treated?
• Who the report comes from
• Graphic display vs. (or in addition to) text

14. Two types of feedback report

15. Building the science
• Consider causal mechanisms
• What is the mechanism by which we get the responses we get?
• What mediates the expected mechanisms?
• How can we increase the reliability and replicability of the implementation interventions
we use?
• What are the important features of context?
• Major difference between QI and implementation science
• QI is embedded in context, rarely studies it
• IS attempts to study the effect of context
• Although we are still unclear about what we mean

16. A few comments on the state of the science
of implementation
• Current efforts to engage and articulate theory
• Lewis et al.– “theory fragments” to link strategies with mechanisms
• Davidoff et al.– call for articulating theory explicitly
• Note: a basic scientist would never conduct an experiment without explicitly stating their
hypotheses– which arise from the underlying theory
• Gabbay and LeMay– building theory from observation
• Theory can be somewhat surprising when it comes from deep knowledge and practice
observation
• Why do we use theory so little?
• Core underlying disciplines seem atheoretical
• Medicine, health services research
• Operate extensively with very implicit, mostly unstated, theory
• An obsession with whether something works
• Very little attention to why it does or does not work

17. Pulling this all together
• Not everything has to be about science
• We need practitioners of implementation informed by science
• Tools and approaches within quality improvement are amenable to adaptation and
use of tools developed through implementation science
• But we need better science to keep moving forward on discovery
• We are currently pretty “stuck” on science in this field
• Lots of opportunities and frontiers
• Need committed, consistent scientists who develop and attend to theory
• Even when the labs are complex and difficult to build and maintain
• Even when the push is to “just do it”
• We’re not actually running a race
• We’re trying to understand the complex relationships between runners and the conditions in which
they run races

18. Both are important