Mulaz Bustani Regenstrief Conference Slides


Published on

Selecting An Intervention In A Complex Adaptive Health Care System & Evaluating Its Impact

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mulaz Bustani Regenstrief Conference Slides

  1. 1. Selecting An Intervention In A Complex Adaptive Health Care System & Evaluating Its Impact Malaz Boustani, MD, MPH Indianapolis Discovery Network for Dementia Indiana University center for Aging Research Regenstrief Institute, Inc
  2. 2. The US Health Care System Report Card <ul><li>Sub-optimal quality </li></ul><ul><ul><li>Dementia recognition:< 40% </li></ul></ul><ul><ul><li>FDA approved Trx: < 10% </li></ul></ul><ul><ul><li>Inappropriate Trx: > 25% </li></ul></ul><ul><li>Compromised safety </li></ul><ul><ul><li>90,000 death / yr </li></ul></ul><ul><ul><li>1,000,000 injury / yr </li></ul></ul><ul><li>Waste </li></ul><ul><ul><li>30% - 40% health expenditure (overuse, underuse, misuse, duplication) </li></ul></ul><ul><li>Discovery to Delivery: </li></ul><ul><ul><li>17 yrs </li></ul></ul><ul><ul><li>$800 millions </li></ul></ul><ul><li>Generalizability of Research Findings: </li></ul><ul><ul><li>< 1% of real world patients enrolled in AD studies </li></ul></ul>Boustani, 2007; IOM, 2001, Westfall, 2007; Faison et al, 2003;
  3. 3. Independent Agent E Other Local CASs Stress Feedback Independent Agent D Independent Agent C Independent Agent B Independent Agent A Changing Environment Connection The health care CAS Emergent Performance
  4. 4. Complex Adaptive Health Care System <ul><li>An open, dynamic, flexible, adaptive, and complex network </li></ul><ul><li>Complex due to </li></ul><ul><ul><li>Numerous interconnected, semi-autonomous, competing, and collaborating members </li></ul></ul><ul><li>Adaptive due to </li></ul><ul><ul><li>Its capability of learning from its prior experience </li></ul></ul><ul><ul><li>Its flexibility to change its members connecting patterns to fit better with its surrounding environment </li></ul></ul>Holden, 2005; Litaker et al. 2006 Feb.
  5. 5. Complex Adaptive Health Care System <ul><li>Emergent behaviors </li></ul><ul><ul><li>NOT predetermined ones </li></ul></ul><ul><li>Self-organized controls </li></ul><ul><ul><li>NOT central controls </li></ul></ul>Holden, 2005; Litaker et al. 2006 Feb.
  6. 6. The Paradox of CAS <ul><li>Each CAS is unique in its </li></ul><ul><ul><li>Members diversity </li></ul></ul><ul><ul><li>Members interactions </li></ul></ul><ul><ul><li>Surrounding environment </li></ul></ul><ul><ul><li>Previous history </li></ul></ul><ul><ul><li>Evolving and learning process. </li></ul></ul><ul><li>Health care research centers generate scientific data from group-based studies </li></ul><ul><ul><li>Then regulators/guideline developers ask each unique CAS to implement the findings. </li></ul></ul>Holden, 2005; Litaker et al. 2006 Feb; Stoebel et al, 2005.
  7. 7. The Paradox of CAS <ul><li>The performance of a CAS fluctuates over time </li></ul><ul><li>The connecting patterns of its members are dynamic and change over time in respond to the stress or the fitness requirements of its surrounding environment. </li></ul>Holden, 2005; Litaker et al. 2006 Feb; Stoebel et al, 2005.
  8. 8. How Can I Assess CAUSALITY? Can I Predict? <ul><li>How can I identify or determine that a system is a CAS? </li></ul><ul><li>How can a system with no bosses, no mandates and no plans produce repeatedly complicated structures and perform difficult tasks? </li></ul><ul><li>Can I build a complex adaptive system model to study? What can I do with a simulator? </li></ul><ul><li>What data do I have & how can I analyze data in CAS? </li></ul><ul><li>How can I manage the problems of uncertainty, variability, and variable interdependency in a CAS? </li></ul><ul><li>Is replication essential in science? </li></ul>
  9. 9. <ul><li>How can I select a change in a CAS? </li></ul><ul><li>How can I evaluate the success or the impact of my proposed intervention or change in a CAS? </li></ul><ul><li>How are experiments performed in a complex adaptive system? </li></ul><ul><li>Is a randomized controlled trial the best accurate study design to evaluate an intervention in a CAHS? </li></ul>How Can I Assess CAUSALITY? Can I Predict?
  10. 10. Selecting an Intervention
  11. 11. The 9 Principles of Selecting a Change in a Complex Health Care System <ul><li>View your system through the lens of complexity. </li></ul><ul><li>Build a good enough vision with minimum specifications </li></ul><ul><li>Lead from the edge by balancing data and intuition, planning and acting, safety and risk. </li></ul><ul><li>Foster the &quot;right&quot; degree of information flow, diversity and difference, connection among agents, power differential, and anxiety. </li></ul><ul><li>Uncover and work with paradox and tension. </li></ul>Paul E Plsek. 1997.
  12. 12. The 9 Principles of Selecting a Change in a Complex Health Care System <ul><li>Go for multiple actions at the fringes, let direction arise. </li></ul><ul><li>Listen to the informal relationships, gossip, rumor, and hallway conversations that contribute significantly to the individuals’ perceptions about their surrounding environment and their subsequent actions. </li></ul><ul><li>Allow complex systems to emerge out of the links among simple systems that work well and are capable of operating independently. </li></ul><ul><li>Build a community of members who collaborate, create, learn and compete </li></ul>Paul E Plsek. 1997.
  13. 13. Reflective Adaptive Process <ul><li>Build a local “Think-Tank: the RAP Team” responsible of introducing an acceptable, locally matched, flexible and effective change in its CAS. </li></ul><ul><li>External or internal facilitators who encourage the RAP Team to select, adopt or create local processes to </li></ul><ul><ul><li>solve the CAS problem </li></ul></ul><ul><ul><li>enhance the CAS performance </li></ul></ul><ul><ul><li>guide the CAS respond to its surrounding environment </li></ul></ul><ul><li>RAP is the second generation of CQI </li></ul>Stroebel et al.2005.
  14. 14. The RAP Principles <ul><li>Vision, mission, and shared values are fundamental in guiding ongoing change processes in a CAS </li></ul><ul><li>Creating time and space for learning and reflection is necessary for a CAS to adapt to and plan change </li></ul><ul><li>Tension and discomfort are essential and normal during CAS change </li></ul><ul><li>Improvement teams should include a variety of system’s agents with different perspectives of the system and its environment </li></ul><ul><li>System change requires supportive leadership that is actively involved in the change process, ensuring full participation from all members and protecting time for reflection </li></ul>McDaniel et al, 2003; Stroebel et al.2005; Boustani et al 2007.
  15. 15. Evaluation of Interventions in CAS <ul><li>Selecting the content and the delivery of the intervention </li></ul><ul><ul><li>A- The content based on SER of past research or guidelines </li></ul></ul><ul><ul><li>B- Develop a RAP team to </li></ul></ul><ul><ul><ul><li>Localize the content </li></ul></ul></ul><ul><ul><ul><li>Localize / Invent the delivery process </li></ul></ul></ul><ul><ul><ul><li>Monitor the delivery process </li></ul></ul></ul><ul><ul><ul><li>Monitor the agents interactions </li></ul></ul></ul><ul><ul><ul><li>Detect emergent behaviors </li></ul></ul></ul>
  16. 16. Evaluating the Impact of the Selected Interventions
  17. 17. Methodological Quality “ Extent to which a study’s design, conduct, and analysis has minimized selection, measurement , and confounding biases” West et al, 2002
  18. 18. The “Safeguards” Evidence <ul><li>Random assignment against </li></ul><ul><ul><li>both unconscious and deliberate human influence on the assignment of subjects to different groups </li></ul></ul><ul><ul><li>unknown confounders </li></ul></ul><ul><li>Blind outcome assessment ensures </li></ul><ul><ul><li>treatment and analysis of outcomes are not colored by prejudice </li></ul></ul>Guyatt et al, 1993; West et al, 2002
  19. 19. The Hierarchical Strength of Evidence in Treatment Decisions <ul><li>N-of-1 RCT </li></ul><ul><li>SER of RCT </li></ul><ul><li>Single RCT </li></ul><ul><li>SER of observational studies </li></ul><ul><li>Single observational study </li></ul><ul><li>Physiologic studies </li></ul><ul><li>Unsystematic clinical observations </li></ul>Guyatt et al, 1993; West et al, 2002
  20. 20. <ul><li>Design: </li></ul><ul><ul><li>N-OF-1 RCT </li></ul></ul><ul><li>System outcomes </li></ul><ul><ul><li>Overall Cost </li></ul></ul><ul><ul><li>Overall Safety </li></ul></ul><ul><ul><li>Overall Quality </li></ul></ul><ul><ul><li>Overall Patient Satisfaction </li></ul></ul><ul><ul><li>Overall Staff Satisfaction </li></ul></ul><ul><li>Intervention specific outcomes </li></ul>Proposed Evaluation Process
  21. 21. Applying N-OF-1 RCT in CAS <ul><li>A CAS receives pairs of evaluation periods </li></ul><ul><ul><li>Each pair including one period of active intervention and another of usual care in random order. </li></ul></ul><ul><li>CAS members are kept masked to allocation </li></ul><ul><li>Outcomes are monitored blindly and continuously </li></ul>
  22. 22. N-OF-1 RCT in CAS 0----------1------------2-----------3-----------4----------5-----------6------------7-----------8-------------9-----------10------------11------------12 Time Outcome Data Outcome Data Outcome Data Outcome Data Outcome Data Outcome Data Outcome Data R to Int R to Int R to Int R to UC R to UC R to UC
  23. 23. Issues of –OF-1 RCT <ul><li>Selecting the Intervention: </li></ul><ul><ul><li>Is there enough data to suggest the efficacy of the intervention? </li></ul></ul><ul><ul><li>Did the RAP team modify the intervention delivery and content to match the local resources? </li></ul></ul><ul><li>Determining the rapidity of onset of the intervention effect: </li></ul><ul><ul><li>How quickly does the intervention begin to act and cease acting? </li></ul></ul><ul><li>Optimizing dose: </li></ul><ul><ul><li>What is the best dose? </li></ul></ul><ul><ul><li>Does it differ between systems? </li></ul></ul>Guyatt et al, 1986; Price & Evans, 2002
  24. 24. Issues of –OF-1 RCT <ul><li>4. Selecting the outcomes: </li></ul><ul><ul><li>the overall common system outcomes not related specifically to the intervention. </li></ul></ul><ul><ul><li>the outcomes that the intervention effect directly. </li></ul></ul><ul><ul><li>the process that mediates the effect of the interventions such as the relationship and the new connection between the CAS members. </li></ul></ul><ul><li>5. Interpreting the Results of N-of-1 RCTs: </li></ul><ul><ul><li>What a priori criteria should be established for classifying a trial as definitely negative or indefinite? </li></ul></ul>Guyatt et al, 1986; Price & Evans, 2002
  25. 25. The Requirement for N-of-1 RCT: <ul><li>The problem that requires intervention needs to be chronic. </li></ul><ul><li>The problem that requires intervention needs to be stable. </li></ul><ul><li>The effect of the new intervention begins to acts within days to weeks at most. </li></ul><ul><li>The effect of the new change ceases acting after discontinuing within days to weeks at most. </li></ul>Guyatt et al, 1986; Price & Evans, 2002
  26. 26. The Science of CAS <ul><li>“ A general feeling of unease when contemplating complex systems dynamics. Its devotees are practicing fact-free science. A fact for them is, at best, the outcome of a computer simulation; it is rarely a fact about the world.” </li></ul><ul><ul><ul><ul><ul><li>(Smith 1995, p. 30). </li></ul></ul></ul></ul></ul>
  27. 27. The Science of CAS <ul><li>“ Prediction is our best means of distinguishing science from pseudo-science” </li></ul><ul><ul><ul><ul><ul><li>Karl Popper </li></ul></ul></ul></ul></ul><ul><li>“ The history of 20th-century science should also give complexologists pause. Complexity is simply the latest in a long line of highly mathematical ‘theories of almost everything’ that have gripped the imaginations of scientists in this century” </li></ul><ul><ul><ul><ul><ul><li>(Horgan 1995, p. 104). </li></ul></ul></ul></ul></ul>