A talk by Brian Sites at the 2017 meeting of the Scandinavian Society of Anaestesiology and Intensive Care Medicine. All available content from SSAI2017: https://scanfoam.org/ssai2017/

1. Spin, lies, and truths:
observational data in
medicine and
anesthesiology
Brian D. Sites, MD MS

2. No Conflicts

3. Special Thanks to Gil Welch, MD from
The Dartmouth Institute
“It’s all about the denominator”

4. Objective
• Cite one really scary example where
observational data got it wrong and
people were injured.

5. In vitro
Animal research
Opinions, editorials
Case reports, series
Cross-sectional
Case control
Cohort
RCT
SR
Increasing
strength of
evidence

6. It’s everywhere
• Cohort Studies and case-control studies
• QA database reviews/clinical registries
• Cross-sectional surveys (Patient
surveys)
• Medicare claims data
• EMR feeds
• Cost-analysis data

8. One month before the second
Obama election, the
unemployment rate unexpectedly
dropped from
8.2% 7.7%

9. “Excluding those not looking
for work, the unemployment
rate dropped to its lowest....”
82 people
unemployed
1000 people in
population
= 8.2%
77 people
unemployed
1000 people in
population
= 7.7%
5 people
leave work
force

10. Hormonal Replacement
Therapy (HRT)

11. “Therapy for menopause
syndrome should not only relieve
the psychic instability attendent
the condition, but the vasomotor
instability....The patient is not
alone in her devotion to this
natural estrogen. Doctors,
husbands, and family all like what
it does for the patient, the wife,
and the homemaker.”
Originally for relief of
menopause symptoms
Circa 1950s
JAMA

12. Hormonal Replacement
Therapy (HRT)
• Expanded into a multi-billion dollar industry
• Use shifted from TREATMENT of
menopausal symptoms to PREVENTION of
dementia, cardiovascular disease, and hip
fractures
• Millions and Millions of women WITHOUT
disease consumed hormonal replacement
therapy

13. HRT for the prevention of dementia: The
Cache County study
Recruitment
of elderly
patients
without
dementia and
assessed
for HRT
Hormone replacement and the incidence of Alzheimer disease. JAMA
2002;288:2123
1066 exposed
to HRT
800 not exposed
to HRT
26 get
dementia
58 get
dementia
(Classic Prospective Cohort)

14. Risk of Dementia with Hormones = 26/1066
Risk of Dementia with no Hormones = 58/800
Relative Risk of
Dementia (RR)
26/1086
58/800
= = 0.33
That’s a 67%
reduction in risk

16. Association (p<.05) DOES
NOT prove Causation!
Drinking Depression
Drinkers (compared to non drinkers)
have a RR of 1.5 for depression, p value
< 0.0001
Depression Drinking

17. Hill criteria supporting
causality
• Magnitude of the effect
• Dose response
• Biological plausibility
• Consistency
• Temporality
• Experiment
“Oh my”
“More drug more effect”
“Whiff test”
“I’ve seen similar”
Drinkers didn’t have depression
A real study

18. Story grows for
10,000,000 womenDozens and Dozens of
articles emerged suggesting
that HRT reduced a women’s
chance of developing heart
disease!

19. “Most, but not all, studies of hormone replacement
therapy in postmenopausal women show around a
50% reduction in risk of a coronary event in women
using unopposed oral estrogen.” JAMA 1991
“There is extensive and consistent observational
evidence that estrogen use reduces risks for CHD about
35%.” Ann Inter Med 1991
“...a recent update from the Nurse’s Health Study
again confirmed a 40% to 60% reduction in
cardiovascular events in women taking HRT.”
Circulation 2001

20. JAMA 1997 Fancy
Computer Model
Predicting Years
Gained!

21. “Hormonal replacement therapy should
increase life expectancy for nearly all
postmenopausal women, with some gains
exceeding 3 years….”

22. Women’s Health Initiative
Women
50-79
Postmenopaus
al
16,608
Risks and Benefits of Estrogen Plus Progestin in Healthy Postmenopausal
Women JAMA 2002 288: 321-333.
HRT
No HRT
Classic RCT

23. No Change in
death
A Few less
-Hip fx
-Colon Ca
A Few More
-MIs
-Strokes
-DVTs
-Breast Ca
No change in
-Sleep disturbance
-Depression
-Sexual
Satisfaction
-Energy
5.2 years

25. Don’t Forget About the Original Fuel: Dementia
Post-
Menopausal
women free of
dementia
Estrogen Plus Progestin and the Incidence of Dementia in Postmenopausal
Women JAMA 2003 289: 2651-2662.
2229
HRT
2303
No HRT
Classic RCT

26. Risk of Dementia with Hormones = 40/2229
Risk of Dementia with no Hormones = 21/2303
Relative Risk of
Dementia (RR)
=
Estrogen Plus Progestin and the Incidence of Dementia in Postmenopausal
Women JAMA 2003 289: 2651-2662.
2.05, 95% CI 1.2-3.5

27. Hormone Replacement & Alzheimer's Disease
Cache County Study – A Prospective Cohort
Study
0.33 0.66Adjustment Unmeasured
Confounders 2.05
Cache County RCT

28. • neurotrophic and neuroprotective influences!
• enhancement of synaptic plasticity!
• effects on several neurotransmitter systems!
• reduction in β-amyloid formation from its precursor protein!
• and modulation of regional cerebral glucose metabolism!
• the pattern of brain activation during memory processing!
• increases regional cerebral blood flow and glucose metabolism
• modulates brain activity in specific brain regions affected in the
Humans:
Animals:
Looking back: “The Discussion”

29. “The reversal of established medical
practice is common and occurs across all
classes of medical practice. This
investigation sheds light on low-value
practices and patterns of medical
research.”
of the 363 gold
standards, 146 (40.2%)
were reversed!

30. Anesthesia is in there
too!
• Two articles contraindicated routine use
of PA catheters
• 2 articles found worse outcomes with
recommended glycemic standards
• Stenting of stable CAD was reversed
• No go for routine arthroscopy for OA

31. Too reliant on observational (flawed) data

32. Why would we ever want
observational data?
• Rare outcomes
• Harmful treatments
• No experimental control
possible (QA data)
• Hypothesis generating
• Validation of expert opinion
and RCT

33. What should target Hgb A1C be?
7% or less
7% to 8%
6% to 7%
7% or less
6.5%
or less
Around 7%

34. Ann Intern Med. 2007;147(6):417-422
The AGREE appraisal instrument asks 23 questions in 6
domains: scope and purpose, stakeholder involvement,
rigor of development, clarity and presentation,
applicability, and editorial independence
“To prevent microvascular complications of
diabetes, the goal for glycemic control should
be as low as is feasible without undue risk for
adverse events or an unacceptable burden on
patients. A hemoglobin A1c level less than 7%
based on individualized assessment is a
reasonable goal”

35. Is “good control”
actually safe?
More than 47,000 patients getting intensified diabetes
therapy in the UK were studied from a National Health
Database
Lancet 2010; 375: 481-89

36. Relative
Risk of
Death
Mean Hemoglobin A1c
following intensification
Is “good control” actually safe?

37. 783 Intensive
Insulin Therapy
783
conventional
2229
HRT
These findings translated
into real world morbidity
and mortality
More death
More life!!

38. Root cause via a systemic review suggested that the
Van Den Berghe trial likely succeeded because of
“In their over enthusiams to implement the IIT, the
researchers, clinicians went overboard with their own half-
baked protocols.”
Editorial: Anaesthesia 2015; 2.2:77-81.
• Pumps for delivery
• ICU nurse training
• How glucose technically measured
• Frequency of sampling
• Patients were always fasting

39. Summary
• Be skeptical
• Unmeasured
confounders are real
• Fancy biological
mechanisms are not
proof of anything
• Bar should be set very
high for healthy patients
You can use observational data and
“PRAGMATIC” approaches to validate
traditional RCTs in your health system!

40. Thanks!