Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Measuring and Improving Decision QualityKaren Sepucha, PhDHealth Decision Science CenterMassachusetts General Hospitalksep...
Disclosure   Dr Sepucha receives research and salary    support from Informed Medical Decisions    Foundation   Dr. Sepu...
Agenda   What is a good decision?   How to measure “decision quality”?       Knowledge       Matching treatment to goa...
Case study: Mr. M’s Story 71yo man referred to orthopedics, worsening  right hip pain over past 2 years, x-rays confirm  ...
Mr. M’s Letter         5
High quality, patient-centered care       NQF       National Quality Forum                                Core Themes:    ...
Agenda   What is a good decision?   How to measure “decision quality”?       Knowledge       Matching treatment to goa...
Measuring Decision Quality                                                To provide evidence that                        ...
Who made the decision about treatment of your breast cancer?                          “they didn’t say to me, “Well, we co...
Survey development processITEM GENERATION  Literature review  Focus groups and   interviews                          DRA...
Field tests across decisions   Surgical decisions (n=1,221)       Breast cancer surgery (n=237, n=445) and Reconstructio...
Measuring    knowledge   Key facts   Mix of gist and    quantitative
Knowledge scores – discriminant validity         Usual                                                        58%         ...
Do treatments match patients’ goals?       Key outcome in Cochrane systematic review of        patient decision aids     ...
Measuringgoals   Achieve or avoid   Discriminate among    options   Challenge of timing    assessment
Calculating a match     Logistic regression model (treatment      received) with goals as independent      predictors   ...
Validity: How well does model reflect patients’ preferences?  Treatment preference                         Non surgical op...
Do patients get treatments that    match their goals? (n=383)                                                             ...
Is there a “Decision Quality” score?   Composite score       Requires benchmark for considering patients        “informe...
Agenda   What is a good decision?   How to measure “decision quality”?       Knowledge       Matching treatment to goa...
What’s the purpose of measurement?    Research             Clinical practice       AccountabilityBasicTranslClinical   C...
Mr. M’s story, continued   2 years later, pain worsened and night time    pain came back   Went back to surgeon and had ...
Summary   Decision quality definition: extent to which patients are informed    and receive treatments match their goals...
Upcoming SlideShare
Loading in …5
×

Measuring and Improving Decision Quality

1,379 views

Published on

Karen Sepucha, PhD, describes what a good decision is, how we measure decision quality and how the decision quality instrument might be used.

This presentation was part of a Shared Decision Making Month webinar -- What Makes a Good Medical Decision? Defining and Implementing Decision Quality Measures.

Published in: Health & Medicine
  • Be the first to comment

Measuring and Improving Decision Quality

  1. 1. Measuring and Improving Decision QualityKaren Sepucha, PhDHealth Decision Science CenterMassachusetts General Hospitalksepucha@partners.org http://www.massgeneral.org/decisionsciences/
  2. 2. Disclosure Dr Sepucha receives research and salary support from Informed Medical Decisions Foundation Dr. Sepucha is on the advisory board for Vital Decisions, LLC
  3. 3. Agenda What is a good decision? How to measure “decision quality”?  Knowledge  Matching treatment to goals How might the survey be used?
  4. 4. Case study: Mr. M’s Story 71yo man referred to orthopedics, worsening right hip pain over past 2 years, x-rays confirm damage Orthopedic surgeon’s note: “I went over in some detail different treatment options. He very much wishes to proceed with right total hip replacement.” Talked with family and friends, saw PCP for pre- op evaluation 4
  5. 5. Mr. M’s Letter 5
  6. 6. High quality, patient-centered care NQF National Quality Forum Core Themes:  fully informed  treatments reflect patients’ want, needs and preferences  play a key role in making healthcare decisions
  7. 7. Agenda What is a good decision? How to measure “decision quality”?  Knowledge  Matching treatment to goals How might the survey be used?
  8. 8. Measuring Decision Quality To provide evidence that - The patient understands key facts. -The treatment received is consistent with the patient’s personal goals. -The patient was meaningfully involved in decision makingSepucha et al. 2004 Health Affairs; Elwyn BMJ 2006
  9. 9. Who made the decision about treatment of your breast cancer? “they didn’t say to me, “Well, we could remove the breast, we could do this, we could do that.” They just said, “This is what we’re going to do.” And that Mainly the doctor was it—I wasn’t in on the decision.” “She was compassionate, … [and] gave me the data that I needed ... We talked statistics and sizes and measurementsX Both equally and things that helped me..with my decision.” “I made the decision. I’m very happy with the lumpectomy because that’s what I wanted to do from the beginning. They Mainly you [my doctors] didn’t disagree. They didn’t agree. They just said, “Okay.” They understood.”
  10. 10. Survey development processITEM GENERATION Literature review Focus groups and interviews DRAFT INSTRUMENT Candidate facts and goals • Draft items Patient and provider • Cognitive importance ratings interviews (~n=5) (~n=20) • Medical and literacy FINAL INSTRUMENT review • Formal evaluation, • Field testing large, diverse samples • Benchmarks and standards for reporting
  11. 11. Field tests across decisions Surgical decisions (n=1,221)  Breast cancer surgery (n=237, n=445) and Reconstruction (n=84)  Knee and hip osteoarthritis (n=382; n=127)  Herniated disc (n=183) Cancer screening (n=338)  Colon cancer screening (n=338) Medication decisions (n=1,243)  Menopause (n=401)  Depression (n=404)  Breast cancer systemic therapy (n=358) Underserved populations (n=289)  Colon cancer screening, African American (n=191)  Breast surgery Spanish language, HIspanic (n=98)
  12. 12. Measuring knowledge Key facts Mix of gist and quantitative
  13. 13. Knowledge scores – discriminant validity Usual 58% DVD 69%Healthy control 41% Patients 53% Providers 77% 0 20 40 60 80 100 Sepucha KR, et al. Spine 2012; Sepucha K et al. BMC Musculoskelet Disord 2011 Jul 5;12(1):149; Lee C, et al. J Am Coll Surg 2012 Jan;214(1):1-10.
  14. 14. Do treatments match patients’ goals?  Key outcome in Cochrane systematic review of patient decision aids  2009 update: 3 studies reported  2011 update: 13 studies reported  Systematic review of concordance methods (Sepucha and Ozanne 2010)  Variability in definitions  Variability in calculationsStacey et al. Cochrane Database of Systematic Reviews. 2011, Issue 10. Art. No.: CD001431; Sepucha K and Ozanne E.Patient Educ Couns 2010 Jan;78(1):12-23. .
  15. 15. Measuringgoals Achieve or avoid Discriminate among options Challenge of timing assessment
  16. 16. Calculating a match Logistic regression model (treatment received) with goals as independent predictors Model returns predicted probability of having surgery based on patients’ goals Considered “match” if probability ≥0.5 and had surgery or if <0.5 and didn’tSource: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMCMusculoskelet Disord 2011 Jul 5;12(1):149.
  17. 17. Validity: How well does model reflect patients’ preferences? Treatment preference Non surgical options 40% Unsure 59% Surgery 74% Treatment Preference Model predicted probability of surgery 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Model predicted probability, discriminates among those with different stated treatment preferences, p<0.001 for all comparisonsSource: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMCMusculoskelet Disord 2011 Jul 5;12(1):149.
  18. 18. Do patients get treatments that match their goals? (n=383) Had Had non surgical Surgery treatment Model predicted Surgery 49% 13% Model predicted 12% 25% Non surgical  Those who matched had lower regret and more confidenceSource: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMC 18Musculoskelet Disord 2011 Jul 5;12(1):149.
  19. 19. Is there a “Decision Quality” score? Composite score  Requires benchmark for considering patients “informed” (mean of group that watched decision aid)  Variable across topics, populations Risk adjustment (e.g. literacy)
  20. 20. Agenda What is a good decision? How to measure “decision quality”?  Knowledge  Matching treatment to goals How might the survey be used?
  21. 21. What’s the purpose of measurement? Research Clinical practice AccountabilityBasicTranslClinical Care is implemented in Performance various settings measured and compared Detailed Actionable Benchmarks Theory Feasible Cost/Feasible Controlled Acceptable Risk adjustment
  22. 22. Mr. M’s story, continued 2 years later, pain worsened and night time pain came back Went back to surgeon and had replacement surgery Good relief of pain, good function, no regrets
  23. 23. Summary Decision quality definition: extent to which patients are informed and receive treatments match their goals Well tested survey instruments exist for common topics Potential uses span research, clinical care, accountability  Research: evaluate different decision support protocols  Diagnostic screen: identify knowledge gaps and goals in advance of visit  Accountability: documentation required to proceed with elective surgery

×