The document discusses measuring and improving decision quality in healthcare, focusing on how well patients understand treatment options and whether these options align with their personal goals. It presents a case study of a patient and outlines the importance of informed, patient-centered care, including the development and validation of survey instruments to assess decision quality. The ultimate goal is to enhance research, clinical practice, and accountability by ensuring patients are knowledgeable and involved in their treatment decisions.

1. Measuring and Improving
Decision Quality
Karen Sepucha, PhD
Health Decision Science Center
Massachusetts General Hospital
ksepucha@partners.org
http://www.massgeneral.org/decisionsciences/

2. Disclosure
 Dr Sepucha receives research and salary
support from Informed Medical Decisions
Foundation
 Dr. Sepucha is on the advisory board for Vital
Decisions, LLC

3. Agenda
 What is a good decision?
 How to measure “decision quality”?
 Knowledge
 Matching treatment to goals
 How might the survey be used?

4. Case study: Mr. M’s Story
 71yo man referred to orthopedics, worsening
right hip pain over past 2 years, x-rays confirm
damage
 Orthopedic surgeon’s note: “I went over in some
detail different treatment options. He very much
wishes to proceed with right total hip
replacement.”
 Talked with family and friends, saw PCP for pre-
op evaluation
4

5. Mr. M’s Letter
5

6. High quality, patient-centered care
NQF
National Quality Forum
Core Themes:
 fully informed
 treatments reflect patients’
want, needs and preferences
 play a key role in making
healthcare decisions

7. Agenda
 What is a good decision?
 How to measure “decision quality”?
 Knowledge
 Matching treatment to goals
 How might the survey be used?

8. Measuring Decision Quality
To provide evidence that
- The patient understands key
facts.
-The treatment received is
consistent with the patient’s
personal goals.
-The patient was meaningfully
involved in decision making
Sepucha et al. 2004 Health Affairs; Elwyn BMJ 2006

9. Who made the decision about treatment of
your breast cancer?
“they didn’t say to me, “Well, we could
remove the breast, we could do this,
we could do that.” They just said, “This
is what we’re going to do.” And that
 Mainly the doctor
was it—I wasn’t in on the decision.”
“She was compassionate, … [and] gave
me the data that I needed ... We talked
statistics and sizes and measurements
X Both equally

and things that helped me..with my
decision.”
“I made the decision. I’m very happy with
the lumpectomy because that’s what I
wanted to do from the beginning. They
 Mainly you [my doctors] didn’t disagree. They didn’t
agree. They just said, “Okay.” They
understood.”

10. Survey development process
ITEM GENERATION
 Literature review
 Focus groups and
interviews
DRAFT INSTRUMENT
 Candidate facts and
goals • Draft items
 Patient and provider • Cognitive
importance ratings interviews (~n=5)
(~n=20)
• Medical and literacy FINAL INSTRUMENT
review • Formal evaluation,
• Field testing large, diverse
samples
• Benchmarks and
standards for
reporting

11. Field tests across decisions
 Surgical decisions (n=1,221)
 Breast cancer surgery (n=237, n=445) and Reconstruction (n=84)
 Knee and hip osteoarthritis (n=382; n=127)
 Herniated disc (n=183)
 Cancer screening (n=338)
 Colon cancer screening (n=338)
 Medication decisions (n=1,243)
 Menopause (n=401)
 Depression (n=404)
 Breast cancer systemic therapy (n=358)
 Underserved populations (n=289)
 Colon cancer screening, African American (n=191)
 Breast surgery Spanish language, HIspanic (n=98)

12. Measuring
knowledge
 Key facts
 Mix of gist and
quantitative

13. Knowledge scores – discriminant validity
Usual 58%
DVD 69%
Healthy control 41%
Patients 53%
Providers 77%
0 20 40 60 80 100
Sepucha KR, et al. Spine 2012; Sepucha K et al. BMC Musculoskelet Disord 2011 Jul 5;12(1):149; Lee C, et
al. J Am Coll Surg 2012 Jan;214(1):1-10.

14. Do treatments match patients’ goals?
 Key outcome in Cochrane systematic review of
patient decision aids
 2009 update: 3 studies reported
 2011 update: 13 studies reported
 Systematic review of concordance methods
(Sepucha and Ozanne 2010)
 Variability in definitions
 Variability in calculations
Stacey et al. Cochrane Database of Systematic Reviews. 2011, Issue 10. Art. No.: CD001431; Sepucha K and Ozanne E.
Patient Educ Couns 2010 Jan;78(1):12-23. .

15. Measuring
goals
 Achieve or avoid
 Discriminate among
options
 Challenge of timing
assessment

16. Calculating a match
 Logistic regression model (treatment
received) with goals as independent
predictors
 Model returns predicted probability of having
surgery based on patients’ goals
 Considered “match” if probability ≥0.5 and
had surgery or if <0.5 and didn’t
Source: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMC
Musculoskelet Disord 2011 Jul 5;12(1):149.

17. Validity: How well does model reflect
patients’ preferences?
Treatment preference
Non surgical options 40%
Unsure 59%
Surgery 74%
Treatment Preference
Model predicted probability of surgery
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
Model predicted probability, discriminates among those with
different stated treatment preferences, p<0.001 for all comparisons
Source: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMC
Musculoskelet Disord 2011 Jul 5;12(1):149.

18. Do patients get treatments that
match their goals? (n=383)
Had
Had non surgical
Surgery
treatment
Model predicted
Surgery 49% 13%
Model predicted
12% 25%
Non surgical
 Those who matched had lower regret and more confidence
Source: Sepucha K et al. Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation. BMC
18
Musculoskelet Disord 2011 Jul 5;12(1):149.

19. Is there a “Decision Quality” score?
 Composite score
 Requires benchmark for considering patients
“informed” (mean of group that watched
decision aid)
 Variable across topics, populations
 Risk adjustment (e.g. literacy)

20. Agenda
 What is a good decision?
 How to measure “decision quality”?
 Knowledge
 Matching treatment to goals
 How might the survey be used?

21. What’s the purpose of measurement?
Research Clinical practice Accountability
BasicTranslClinical Care is implemented in Performance
various settings measured and
compared
Detailed Actionable Benchmarks
Theory Feasible Cost/Feasible
Controlled Acceptable Risk adjustment

22. Mr. M’s story, continued
 2 years later, pain worsened and night time
pain came back
 Went back to surgeon and had replacement
surgery
 Good relief of pain, good function, no regrets

23. Summary
 Decision quality definition: extent to which patients are informed
and receive treatments match their goals
 Well tested survey instruments exist for common topics
 Potential uses span research, clinical care, accountability
 Research: evaluate different decision support protocols
 Diagnostic screen: identify knowledge gaps and goals in advance
of visit
 Accountability: documentation required to proceed with elective
surgery