Learning objectives: Recognize that many of the questions you receive will come from patients that just read an article on a news site or social media Understand that the vast majority of them qualify as B.S. (bad science) rather than G.S. (good science) Know how to evaluate the article to determine which category applies quickly

1. G.S. or B.S.?
Ron Lyon, MS, RPH

2. Learning Outcomes
 Recognize that many of the questions you receive will
come from patients that just read an article on a news
site or social media
 Understand that the vast majority of them qualify as B.S.
(bad science) rather than G.S. (good science)
 Know how to quickly evaluate the article to determine
which category applies

3. First clue: “May”
essentially the same as
“May not”

4. Title, source, and URL of page you are
evaluating
Title, source, and URL of
page you are evaluating
“Vegan Diet May Improve Diabetes-Related
Insulin Sensitivity: Study”
NDTV: New Delhi Television Limited
https://www.ndtv.com/food/vegan-diet-may-
improve-diabetes-related-insulin-sensitivity-study-
1813413
TinyURL: http://tinyurl.com/yacvt3nb

5. News article authors usually lack
credentials to evaluate clinical studies
Look up the author in Google
and list author’s level of subject
knowledge
 Subject
matter expert
(SME)
 Journalist or
other  Unknown

6. Did they make it easy to evaluate their
conclusions?
Links to primary literature?
Do they work? Find the
primary article and complete
the following Abstract
Review section.
 Yes  No
According to a latest study, overweight adults with no
history of diabetes were shown to have marginally better
insulin sensitivity and beta cell function after eating a plant-
based diet as compared to an animal-based one. The study
was published in the journal Nutrients.

7. I don’t like to have to find the article myself
List the full citation for the
primary article
Kahleova, H.; Tura, A.; Hill, M.; Holubkov, R.; Barnard,
N.D. A Plant-Based Dietary Intervention Improves Beta-
Cell Function and Insulin Resistance in Overweight
Adults: A 16-Week Randomized Clinical
Trial. Nutrients2018, 10, 189.
http://www.mdpi.com/2072-6643/10/2/189

8. How old is the primary
literature compared to
the article publication
date?
News article written seven (7) days after
publication.

9. Would the authors financially benefit from a
positive result?
Rate potential bias  Authors
have major
financial
interest in
outcome
 Authors
received grants
to conduct
research
(expected level
of bias)
 Research
conducted with
no financial
interest in the
study
medication
Acknowledgments
This work was funded by the Physicians Committee for Responsible Medicine.

10. Does the study hold interest for your
practice setting?
P Problem Describe the key issue:
Can a vegan diet reduce the risk of developing diabetes in
overweight patients?
Briefly describe whether this study could apply to your practice
setting:
Very likely, we would deal with patients at risk of developing diabetes.
We should always be considering lifestyle changes in relation to
medication therapy.

11. Was the study population representative of
your patients?
P Patients
or
population
Describe whether the study patients will be similar the
patients in your practice setting:
We would commonly see ”overweight adults with no history of
diabetes” in our practice.

12. The new treatment being studied is
considered the “intervention”
I Intervention
(study
medication)
Is the medication available and feasible for use in your
organization? Y/N
In this case, the intervention would be the vegan diet, not a new
medication.
If not, briefly describe why.

13. Ideally, the “comparison” should reflect the current
recommended guideline to address the condition
C
Comparison
(comparator
medication,
treatment or
placebo)
Does the active comparator (or placebo) realistically
reflect current best practices? Y/N List the comparator
used:
No diet changes.
Ideally, this study would compare outcomes against other dietary
approaches. The results, if meaningful, or statistically believable
would be interesting but wouldn’t tell me which diet to recommend.

14. Were the differences in outcomes
meaningful?
O Outcome Describe whether and how patients will benefit from this
medication compared to the current standard treatment:
The Homeostasis Model Assessment (HOMA-IR) index was used
to assess insulin resistance while fasting.
First, I needed to learn about HOMA-IR. While it varies by age
and gender, a HOMA-IR of greater than 2 would indicate
metabolic syndrome.
This study showed a 1.0 decrease in HOMA-IR for the vegan diet
versus usual diet.

15. Was the study large enough?
N Number
of subjects
List number of subjects on study medication:_38____
Comparator:_37___
Was that enough to make you believe the results? Y/N
Since they used objective, specific laboratory measures I tended to
believe they included enough patients. But, the statistical analysis will
tell the real story.

16. If they only show the relative difference,
then I wonder if they’re hiding something
In this case, the authors stated the absolute difference
(-1.0) and not the relative difference. That’s unusual,
but a very good sign.

17. I almost never state the relative difference
If the starting HOMA-IR for both groups was 2.0,
here’s the absolute versus relative difference
calculations:
 Intervention group dropped from 2.0 to 0.9 = -1.1
 Comparison group dropped from 2.0 to 1.9 = -0.1
Absolute difference= -1.1 minus -0.1 = -1.0
Relative difference = (-1.1 divided by -0.1) then
subtract 1 = 10 times difference
Converted to a percentage = 1,000% difference

18. My favorite statistic to cite
Number needed to treat (NNT) or Number needed to
harm (NNH)
 But can only be used when measuring a specific
goal (e.g., death or not, HBA1c less than 6.0,
diastolic blood pressure < 90 mm Hg).
 Let’s pretend for an example:
 20 of 38 (53%) vegan diet patients reached HOMA-IR
goal of < 1.0
 2 of 37 (5%) usual diet patients reached that goal
 Absolute difference = 53% - 5% = 48%
 NNT= 1/(absolute % difference) = 1/48% = 2.1
 Stated as “If we switch 2 patients to a vegan diet, then 1
patient will reach the HOMA-IR goal of less than 1.0.”

19. Confidence intervals in the abstract usually
demonstrate confidence in the study
 List the confidence interval for the primary endpoint:
HOMA-IR index fell significantly (p < 0.001) in the
intervention group (treatment effect −1.0 (95% CI, −1.2 to
−0.8))
 p < 0.001: “. . . there is less than a 0.1% chance that this
result was due to chance.”
 It also means there were enough patients in the study.
 95% CI: “. . . 95% sure that the results in the full
population would fall between . . .”
 -1.2 to -0.8: “ . . . a decrease in HOMA-IR values of 0.8
to 1.2.”

20. Do the confidence intervals overlap? Y/N
Death rate
Drug A: 10%
Drug B: 10%
Risk/odds ratio =
10%/10% =1.0

21. Precise versus imprecise
More precise

22. Relative
change
Odds ratio
1.0
95% CI, 1.10 to 2.0
95% CI, 0.20 to 0.80
95% CI, 0.70 to 1.50
Comparing death rates of Medication A to Medication B
95% confident that Medication A
has a 10% to 100% higher death
rate than Medication B
95% confident that Medication A has a
20% to 80% lower death rate than
Medication B
95% confident that Medication A has a
30% lower OR a 50% higher death rate
than Medication B

23. Indicate your opinion on the news article
___ Good science ___ Bad science

24. Rewrite the title of the article to best
reflect the “truth”
 Original:
Vegan Diet May Improve Diabetes-Related Insulin
Sensitivity: Study
 Revised:
Vegan Diet May Improve Diabetes-Related Insulin
Sensitivity in overweight, non-diabetic patients:
Study