Measuring and Improving Decision Quality

  • 453 views
Uploaded on

Karen Sepucha, PhD, describes what a good decision is, how we measure decision quality and how the decision quality instrument might be used. …

Karen Sepucha, PhD, describes what a good decision is, how we measure decision quality and how the decision quality instrument might be used.

This presentation was part of a Shared Decision Making Month webinar -- What Makes a Good Medical Decision? Defining and Implementing Decision Quality Measures.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
453
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • We laid out the proposal for for measuring decision quality in a Health affairs article a few years ago. Main concerns were response that focus on guidelines or “setting right rate” that ignored warranted sources fo variation. Instead we wanted to figure out whether the right treatment is being matched with the right patient, to do that you need treatment rates alone are not enough
  • Three main phases to measure development First item generation – for us that means identifying the key facts and values that are salient for the decisions. TO do this we review clinical evidence for situation, review literature on decision making experiences, run focus groups and patients and providers to learn about their experiences, distill set of candidate facts and values, Those are then rated by samples of patients and providers for importance to select those facts and values that will be included in draft questionnaire. Items in the draft are run through cognitive testing to make sure patients comprehend the questions and responses and that their answers are refelcting what we hope to learn. In addition field testing of the instrument at this phase can help provide some evidence of acceptability (repsonse rates, missing items) test retest and some preliminary validity. Further refinements and formal testing with large diverse samples, result in a final instrument that is ready for widespread use. To date we are in the second phase.

Transcript

  • 1. Measuring and Improving Decision QualityKaren Sepucha, PhDHealth Decision Science CenterMassachusetts General Hospitalksepucha@partners.org http://www.massgeneral.org/decisionsciences/
  • 2. Disclosure Dr Sepucha receives research and salary support from Informed Medical Decisions Foundation Dr. Sepucha is on the advisory board for Vital Decisions, LLC
  • 3. Agenda What is a good decision? How to measure “decision quality”?  Knowledge  Matching treatment to goals How might the survey be used?
  • 4. Case study: Mr. M’s Story 71yo man referred to orthopedics, worsening right hip pain over past 2 years, x-rays confirm damage Orthopedic surgeon’s note: “I went over in some detail different treatment options. He very much wishes to proceed with right total hip replacement.” Talked with family and friends, saw PCP for pre- op evaluation 4
  • 5. Mr. M’s Letter 5
  • 6. High quality, patient-centered care NQF National Quality Forum Core Themes:  fully informed  treatments reflect patients’ want, needs and preferences  play a key role in making healthcare decisions
  • 7. Agenda What is a good decision? How to measure “decision quality”?  Knowledge  Matching treatment to goals How might the survey be used?
  • 8. Measuring Decision Quality To provide evidence that - The patient understands key facts. -The treatment received is consistent with the patient’s personal goals. -The patient was meaningfully involved in decision makingSepucha et al. 2004 Health Affairs; Elwyn BMJ 2006
  • 9. Who made the decision about treatment of your breast cancer? “they didn’t say to me, “Well, we could remove the breast, we could do this, we could do that.” They just said, “This is what we’re going to do.” And that Mainly the doctor was it—I wasn’t in on the decision.” “She was compassionate, … [and] gave me the data that I needed ... We talked statistics and sizes and measurementsX Both equally and things that helped me..with my decision.” “I made the decision. I’m very happy with the lumpectomy because that’s what I wanted to do from the beginning. They Mainly you [my doctors] didn’t disagree. They didn’t agree. They just said, “Okay.” They understood.”
  • 10. Survey development processITEM GENERATION Literature review Focus groups and interviews DRAFT INSTRUMENT Candidate facts and goals • Draft items Patient and provider • Cognitive importance ratings interviews (~n=5) (~n=20) • Medical and literacy FINAL INSTRUMENT review • Formal evaluation, • Field testing large, diverse samples • Benchmarks and standards for reporting
  • 11. Field tests across decisions Surgical decisions (n=1,221)  Breast cancer surgery (n=237, n=445) and Reconstruction (n=84)  Knee and hip osteoarthritis (n=382; n=127)  Herniated disc (n=183) Cancer screening (n=338)  Colon cancer screening (n=338) Medication decisions (n=1,243)  Menopause (n=401)  Depression (n=404)  Breast cancer systemic therapy (n=358) Underserved populations (n=289)  Colon cancer screening, African American (n=191)  Breast surgery Spanish language, HIspanic (n=98)
  • 12. Measuring knowledge Key facts Mix of gist and quantitative
  • 13. Knowledge scores – discriminant validity Usual 58% DVD 69%Healthy control 41% Patients 53% Providers 77% 0 20 40 60 80 100 Sepucha KR, et al. Spine 2012; Sepucha K et al. BMC Musculoskelet Disord 2011 Jul 5;12(1):149; Lee C, et al. J Am Coll Surg 2012 Jan;214(1):1-10.
  • 14. Do treatments match patients’ goals?  Key outcome in Cochrane systematic review of patient decision aids  2009 update: 3 studies reported  2011 update: 13 studies reported  Systematic review of concordance methods (Sepucha and Ozanne 2010)  Variability in definitions  Variability in calculationsStacey et al. Cochrane Database of Systematic Reviews. 2011, Issue 10. Art. No.: CD001431; Sepucha K and Ozanne E.Patient Educ Couns 2010 Jan;78(1):12-23. .
  • 15. Measuringgoals Achieve or avoid Discriminate among options Challenge of timing assessment
  • 16. Calculating a match Logistic regression model (treatment received) with goals as independent predictors Model returns predicted probability of having surgery based on patients’ goals Considered “match” if probability ≥0.5 and had surgery or if <0.5 and didn’tSource: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMCMusculoskelet Disord 2011 Jul 5;12(1):149.
  • 17. Validity: How well does model reflect patients’ preferences? Treatment preference Non surgical options 40% Unsure 59% Surgery 74% Treatment Preference Model predicted probability of surgery 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Model predicted probability, discriminates among those with different stated treatment preferences, p<0.001 for all comparisonsSource: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMCMusculoskelet Disord 2011 Jul 5;12(1):149.
  • 18. Do patients get treatments that match their goals? (n=383) Had Had non surgical Surgery treatment Model predicted Surgery 49% 13% Model predicted 12% 25% Non surgical  Those who matched had lower regret and more confidenceSource: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMC 18Musculoskelet Disord 2011 Jul 5;12(1):149.
  • 19. Is there a “Decision Quality” score? Composite score  Requires benchmark for considering patients “informed” (mean of group that watched decision aid)  Variable across topics, populations Risk adjustment (e.g. literacy)
  • 20. Agenda What is a good decision? How to measure “decision quality”?  Knowledge  Matching treatment to goals How might the survey be used?
  • 21. What’s the purpose of measurement? Research Clinical practice AccountabilityBasicTranslClinical Care is implemented in Performance various settings measured and compared Detailed Actionable Benchmarks Theory Feasible Cost/Feasible Controlled Acceptable Risk adjustment
  • 22. Mr. M’s story, continued 2 years later, pain worsened and night time pain came back Went back to surgeon and had replacement surgery Good relief of pain, good function, no regrets
  • 23. Summary Decision quality definition: extent to which patients are informed and receive treatments match their goals Well tested survey instruments exist for common topics Potential uses span research, clinical care, accountability  Research: evaluate different decision support protocols  Diagnostic screen: identify knowledge gaps and goals in advance of visit  Accountability: documentation required to proceed with elective surgery