This document discusses various topics related to outcomes research, including comparative effectiveness research, multilevel data analysis, investigating change over time, and estimating treatment effects from observational data. Comparative effectiveness research directly compares existing healthcare interventions to determine their benefits and harms. Multilevel analysis is helpful for comparing patient outcomes across hospitals while accounting for risk factors. Propensity score adjustment and other statistical techniques can be used to estimate causal treatment effects from observational data and reduce selection bias. Bayesian statistics are increasingly being used in areas like early-phase cancer trials.

1. Week 3
Educational
Product
BY YANA PUCKETT, MD

2. Introduction
 Comparative Effectiveness Research
 Multilevel Data in Outcomes Research
 Investigating Change Over Time
 Estimating Effect of Intervention from Observational Data

3. Comparative Effectiveness
Research
 Comparative effectiveness research (CER) is the direct comparison
of existing healthcare interventions to determine which work best for
which patients and which pose the greatest benefits and harms,
and which are cost effective.
 A defining objective of CER is to provide information to help
patients, consumers, clinicians, and payers make more informed
clinical and health policy decisions.
 Comparing two different treatments, technologies, pharmacologic
drugs on their effectiveness.
 Highly needed in this age of evidence-based medicine.
 The American Recovery and Reinvestment Act of 2009 allocated a
$1.1 billion “down payment” to support comparative effectiveness
research (CER) (4).

4. Comparative Effectiveness
Research and RCTs
 RCTs, while great, are becoming extremely difficult to approve,
design, and carry out.
 RCTs take years to complete and very few of them while clinical
comparative questions continue to arise.
 Medicine is evolving, new technology is built quickly and RCTs have
no way of keeping up with that.
 Funding is limited and RCTs are extremely expensive to carry out.
 RCTS often exclude patients on strict parameters, thus diminishing
application of findings/results to the population that is targeted.
 Bayesian Statistics may be the solution (4).

5. References
1. Risk Adjustment for Measuring Health Care Outcomes, 4th Ed. By
Iezzoni, L (Ed.) Publisher: Health Administration Press ISBN:
9781567934373.
2. Cho (2003). Using multilevel analysis in patient and organizational
outcomes research. Nursing Research, 52(1), 61-65.
3. Applied Longitudinal Data Analysis: Modeling Change and Event
Occurrence. New York: Oxford University Press. pp. 3-15 in Singer &
Willet (2003).
4. Luce B, Kramer J, Schwartz J, et al. Rethinking Randomized Clinical
Trials for Comparative Effectiveness Research: The Need for
Transformational Change. Annals Of Internal Medicine[serial
online]. August 4, 2009;151(3):206-W.45. Available from: Academic
Search Complete, Ipswich, MA. Accessed June 17, 2015.

6. Multilevel Data in Outcomes
Research
 Randomized Controlled Trials (RCTs)not always feasible or practical.
 RCTs expensive and require years to complete.
 Most clinical questions and health outcomes assessed through
observational data.
 Multivariable model accounts for various baseline differences in risk
and confounders.
 Has become extremely popular in research.

7. Multilevel Analysis (Hierarchical
Modeling)
 Analytic model that measures variables at different levels of
hierarchy.
 Helpful for comparing patient outcomes across hospitals because
can adjust for risk without manipulating risk factors at hospital level.
 Allows simultaneous examination of group-level and individual level
variables over individual level outcome.

8. Multivariable Models for Estimating
Effects of Interventions
 Continuous Outcomes: estimates effect of an intervention on a
continuous outcome via linear regression. Ex: estimating effect of
enrolling in an MCO and how it influences a persons’ health care
expenditures over a year.
 Dichotomous Outcomes: uses logistic regression to assess treatment
effectiveness. Ex: being alive 30 days after hospital admission.
 Time to Event Outcomes: Death is usually the outcome assessed,
survival modeling, proportional hazards modeling or Kaplan-Meyer
Statistics. Ex: Cancer treatment and survival outcomes.

9. Investigating Change Over Time
 Requires:
 Good multilevel longitudinal data that describes how something
changes over time.
 Sensible metric for time that is reliable and valid.
 Continuous outcome that changes systematically over time such as test
scores, self-assessments, psychological measurements.

10. Propensity Score Adjustment
 A propensity score is the probability of a unit being assigned to a
particular treatment given a set of observed covariates.
 Statistical analysis of observational data that accounts for
confounders when comparing treatment results.
 Attempts to reduce bias due to confounding variables that could
be found by simply comparing outcomes among units.
 Attempts to mimic randomization by creating a sample of units that
received the treatment that is comparable on all observed
covariates.
 Decreases selection bias.

11. Estimating Effect of Intervention
from Observational Data
 In randomized studies, association=causation, but can we say the same
for observational data? Generally not.
 Two analytical approaches to compute causal effects from
observational data: standardisation and inverse probability weighting.
 Standardisation: There are two methods of standardisation, direct and
indirect. Standardisation allows a single index of comparative mortality
to be derived, in a way that permits comparison of mortality measures
that are free of the effects of the underlying age distributions of the
populations under observation.
 Inverse Probability Weighting: statistical technique for calculating
statistics standardized to a population different from that in which the
data was collected. Ex: study designs with a disparate sampling
population and population of target inference (target population) are
common in application.

12. Bayesian Statistics
 Use has been very popular in recent years (4).
 Early-phase cancer trials are commonly performed using Bayesian
designs (4).
 Statistical modeling that deals basically determines the likelihood of
something happening based on probabilities given by a set of data
points.