Progress with a Price Tag attempts to provide a concise summary of evaluating evidence and mitigating biases. It discusses how truth exists in detail rather than on the surface. Evaluating evidence requires considering trial methodology, results, and applicability. Statistical significance does not always mean clinical significance. Applying PICO helps determine if evidence applies. Surrogate outcomes like viral presence may not reflect patient benefits. Overall it emphasizes critically evaluating evidence claims and considering multiple perspectives to find detailed truth.

1. Progress with a Price Tag
Trying to survive misinformation amidst a
pandemic
Centre for Inquiry Canada Annual General Meeting – March 11, 2021
Zack Dumont
Manager of Clinical Pharmacy Services – SHA Regina Pharmacy Department
Clinical Pharmacist – RxFiles Academic Detailing

2. Acknowledgements
• CFIC – for the invitation to present
22 March 2021 2

3. A poll
• Type your answers in
the chatbox
–The dress is:
• A) Black and blue
• B) Gold and white
• C) Other
22 March 2021 3
Udland M. One year ago, the internet lost its mind over 'The Dress' that might be blue and black or white and gold — And now some people see the colors differently.
Business Insider February 2016. Available from: https://www.businessinsider.com/is-the-dress-white-and-gold-black-and-blue-2015-2.

4. Faculty/Presenter Disclosure
Faculty: Zack Dumont
• Current or past relationships with
commercial interests
– Advisory Board/Speakers Bureau: nil
– Funding (Grants/Honoraria): nil
– Research/Clinical Trials: nil
– Speaker/Consulting Fees: nil
– Other: nil
• Speaking Fees for current program
– I have received no speaker’s fee for this
learning activity
• This presentation has not received
financial support from any organization
• This presentation has not received in-
kind support from any organization
22 March 2021 4

5. “Your personal experiences make up maybe 0.00000001% of
what’s happened in the world but maybe 80% of how you think
the world works... We’re all biased to our own personal history.”.”
-Morgan Housel
22 March 2021 5

6. Mitigating Potential Bias
• Not applicable
– Really?
22 March 2021 6

7. Disclaimers
• I have biases
– I am aware of some
– I am not aware of others
– You will learn some of mine
– Do you have any?
“We are misled by the ease with
which our minds fall into the ruts of
one or two experiences.”
-Sir William Osler
22 March 2021 7

8. We all have biases, and…
"There are things in medicine that
make perfect sense... but are
wrong“
-James S. Forrester, MD
• Vitamin E to prevent heart disease
• Estrogen to prevent heart disease
• Raising HDL (torcetrapib)
• Taking pills to lower homocysteine
• Lowering A1c (rosiglitazone)
• Lowering BP (doxazosin)
Forrester JS, Shah PK 1997 Lipid lowering versus revascularization. An idea whose time (for testing) has come. Circulation 96:1360–1362
22 March 2021 8

9. Formulary
RN
Experience
Colleagues
Practice Guidelines
Pharm Industry
Patient
Treatment?
Practice site
Pharmacist
CME
Influences on a treatment decision
22 March 2021 9
Prescriber

10. Introducing: COVID-19
• Hydroxychloroquine  rheumatoid arthritis
• Remdesivir  Ebola
• Colchicine  acute gout treatment
• Azithromycin  antibiotic
• Ivermectin  anthelmintic (eg, worms, lice, etc)
• Tocilizumab  rheumatoid arthritis
22 March 2021 10
• Many interventions
have been proposed
(samples right)
• Theoretical basis
proceeds proof
– May make history
tomorrow
– Makes for a headline
today
– …more on this…

11. Objectives
Learning Objectives
• Participants will be able to:
– Explain rationale for critical thinking when
reviewing literature
If you haven’t already guess… one last
disclaimer
• I’m a pharmacist and not a behavioural
psychologist… will do my best to stay in
my lane
– This will be about drugs  (but I think you
can apply it to any intervention)
22 March 2021 11

12. On “misinformation”
What it is
• Most common definitions
approximating:
– “Incorrect and/or misleading information”
• Plenty of overlap, yet some definitions
differ
– Eg, Some suggest an underlying intent to
mislead
What it is NOT
• Truth
22 March 2021 12

13. On “Truth”
• …and perspective
22 March 2021 13
On the Surface
In the Detail

14. In medications and in life:
truth is in the detail
22 March 2021 14
• Approximate truth by learning the detail
• …then try sharing it with someone who
• Is really smart and successful and confident, OR
• Is already convinced they see a circle (or a square), OR
• Is too busy to look and just wants you to tell them if it
there’s a circle (or square) there, OR
• Likes to say it’s a square and see if you can convince them
it’s not
– and… you have minutes, seconds, or less
• Facilitating someone’s journey from square/circle
 to the truth
– As much as this is about truth, it’s about empathy
On the Surface
In the Detail

15. QUICK ASIDE: Truth?
• “Scientific objectivity”
– Possible explanations from philosophy
• Faithfulness to facts
• Absence of normative commitments and
the value-free ideal
• Freedom from personal biases
• A feature of scientific communities and
their practices
– Enduring philosophical debate
• We won’t really solve this today…
22 March 2021 15
Reiss J, Sprenger I. "Scientific Objectivity", The Stanford Encyclopedia of Philosophy (Winter 2020 Edition), Edward N Zalta (ed). Available from:
https://plato.stanford.edu/entries/scientific-objectivity/.
Truth
Possible
Observations
Truth exists
amongst
many
possible
observations

16. In medications and in life:
truth is in the detail
22 March 2021 16
Downden B. ”Fallacies", The Internet Encyclopedia of Philosophy, ISSN 2161-0002. Available from: https://www.iep.utm.edu/.
Reiss J, Sprenger I. "Scientific Objectivity", The Stanford Encyclopedia of Philosophy (Winter 2020 Edition), Edward N Zalta (ed). Available from: https://plato.stanford.edu/entries/scientific-
objectivity/.
• Even if we discover and/or deliver
TRUTH, there remain barriers to
rational decisions
– Rational Choice Theory
• If an individual prefers A to B, they value A higher
than B
– Desires do not have to align with any objective
measure of “goodness”
– Rationalization
• A fallacy in decision-making – “inauthentically offer
reasons to support our claim”
On the Surface
In the Detail

17. Truth = Evidence? • Globe & Mail
– Risk appears increased mainly
• In patients with pre-existing risk factors
• In more potent or higher doses of statins
• Reality
– If treating 255 patients for 4 years
– 1 more case of diabetes
– 5.4 fewer vascular events
• Caveat… pooling high risk and low risk
– High CV risk  most benefit from statin
– Low CV risk  less benefit from statin
– Therefore, when CV benefit is low, diabetes risk may
supersede
» Details…
• Details…
• Details…
22 March 2021 17
• Evidence: Can it be
trusted?

18. Truth = Evidence?
• What do we know?
– Interpretation is complex
– Not well-served by a headline
22 March 2021 18
• Evidence: Can it be
trusted?

19. Truth = Evidence?
22 March 2021 19
• Evidence: Can it be
trusted?
– Three over-arching
questions:
• 1) Quality of trial
methodology?
– Risk of bias
• 2) What are the results?
• 3) How do we apply these
results to us/our
patients?

20. Truth = Evidence?
• Three main considerations
– 1) Randomization
• How?
• Allocation concealment?
– 2) Blinding
• Who did they blind?
– 3) Follow-up
• Technical questions
– Intention-to-treat? Per protocol?
• Questions anyone can ask
– Did a lot of people drop-out of the study?
– Did they stop the trial early for “benefit”?
22 March 2021 20
• Evidence: Can it be
trusted?
– Three over-arching
questions:
• 1) Quality of trial
methodology?
– Risk of bias
• 2) What are the results?
• 3) How do we apply these
results to us/our
patients?

21. Example: remdesivir
• Claim: “clinical improvement was
observed in 36 of 53 patients (68%)”
• Problem: randomization violation
– Did some patients get better in the
study while on the medication?
• Yes, absolutely
– Would they have gotten better without
the medication?
• We have no idea! Ie, there was no control
group
22 March 2021 21
Grein J, Ohmagari N, Shin D. Compassionate Use of Remdesivir for Patients with Severe Covid-19. NEJM April 10, 2020. Available from:
https://www.nejm.org/doi/full/10.1056/NEJMoa2007016.

22. Truth = Evidence?
• Two main considerations
– 1) How large or small is the effect?
• Statistical significance vs clinical
significance
• What about benefits relative to harms?
– 2) How precise or trustworthy is the
effect?
• Confidence intervals
• Line of no difference
22 March 2021 22
• Evidence: Can it be
trusted?
– Three over-arching
questions:
• 1) Quality of trial
methodology?
– Risk of bias
• 2) What are the results?
• 3) How do we apply these
results to us/our
patients?

23. Example: tocilizumab
• Claim: “Improves survival”
– 596 (29%) of the 2022 patients allocated
tocilizumab died
– 694 (33%) of the 2094 patients allocated to
usual care died
– Relative risk = 86%
• 95% confidence interval [CI] 0·77-0·96; p=0·007)
– Relative risk reduction = 14%
• Problem: statistical significance inflates
benefit
– Absolute risk reduction = 33% - 29% = 4%
• Admittedly, very promising because of the mortality
reduction, but will be over-hyped
22 March 2021 23
RECOVERY pre-print. Available from: https://www.medrxiv.org/content/10.1101/2021.02.11.21249258v1

24. Truth = Evidence?
• Use PICO Tool
–Patients/Populations – Are these
even the patients you see? Were they
too healthy?
–Intervention – Is this intervention
even possible?
–Comparator – Did they compare it to
the gold standard? Eg, drug, dose, route,
interval, duration
–Outcomes – Do/should these
outcomes even matter to the public?
22 March 2021 24
• Evidence: Can it be
trusted?
– Three over-arching
questions:
• 1) Quality of trial
methodology?
– Risk of bias
• 2) What are the results?
• 3) How do we apply these
results to us/our
patients?

25. Example: molnupiravir
• Claim: “Stops COVID in its tracks”
– Eg, “By day 3, 28% of patients in the placebo
arm had SARS-CoV-2 in their nasopharynx,
compared to 20.4% of patients receiving any
dose of molnupiravir.”
• Problem: Surrogate outcomes!
– Should we really care about viral presence?
• It’s not irrelevant, but we demand proof of benefit to
patients!
22 March 2021 25
Boerner H. Five-Day Course of Oral Antiviral Appears to Stop SARS-CoV-2 in Its Tracks. March 08, 2021. Available from:
https://www.medscape.com/viewarticle/947061?src=WNL_dne_210310_mscpedit&uac=102683EX&impID=3239028&faf=1#vp_1.

26. Conclusions
22 March 2021 26

27. • Just because we have biases
doesn’t mean we can be
complacent about them
“I was taught that the way of
progress was neither swift nor
easy.”
-Marie Curie
22 March 2021 27

28. Conclusions
Progress with a price tag
• Scientific progress like we’ve never
seen
– Some interventions will stick
• Masks
• Tocilizumab?
• Sickness, suffering, and death like
we’ve never seen
Trying to survive misinformation amidst
a pandemic
• Evidence: Can it be trusted?
– Three over-arching questions:
• 1) Quality of trial methodology?
– Risk of bias
• 2) What are the results?
• 3) How do we apply these results to us/our
patients?
22 March 2021 28

29. “It is the mark of an educated mind to be able to entertain a
thought without accepting it.”
-Aristotle
22 March 2021 29

30. Objectives
Learning Objectives
• Participants will be able to:
– Explain rationale for critical thinking when
reviewing literature
If you haven’t already guess… one last
disclaimer
• I’m a pharmacist and not a behavioural
psychologist… will do my best to stay in
my lane
– This will be about drugs 
22 March 2021 30

31. References
• Udland M. One year ago, the internet lost its mind over 'The Dress' that might be blue and black or white and gold — And now some people see the colors differently. Business Insider
February 2016. Available from: https://www.businessinsider.com/is-the-dress-white-and-gold-black-and-blue-2015-2.
• Lau A. “Implementation,” The First and Most Important Concept You Should Know in Implementation Science. Active Aging Research Team @ UBC November 2019. Available from:
https://medium.com/@activeaging_research/implementation-the-first-and-most-important-concept-you-should-know-in-implementation-science-3d15ee2ace8d.
• Balas EA, Boren SA. (2000). Managing clinical knowledge for health care improvement. In: Bemmel J, McCray AT, editors. Yearbook of Medical Informatics 2000: Patient-Centered
Systems. Stuttgart, Germany: Schattauer Verlagsgesellschaft mbH; 2000:65-70.
• Kennedy AG, Regier L, Fischer M. Educating community clinicians using principles of academic detailing in an evolving landscape. American Journal of Health-System Pharmacy
November 2020. Available from: https://doi.org/10.1093/ajhp/zxaa351.
• Reiss J, Sprenger I. "Scientific Objectivity", The Stanford Encyclopedia of Philosophy (Winter 2020 Edition), Edward N Zalta (ed). Available from:
https://plato.stanford.edu/entries/scientific-objectivity/.
• Guyatt, Gordon; Guyatt, Gordon. Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice, 3E (Users Guides to the Medical Literature) (p. 72). McGraw-Hill
Education.
• Downden B. ”Fallacies", The Internet Encyclopedia of Philosophy, ISSN 2161-0002. Available from: https://www.iep.utm.edu/.
• Reasons J. Human error: models and management. BMJ March 2000. Volume 320, pages 768-770. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1117770/.
• Elliott MH, Skydel JJ, Dhruva SS, Ross JS, Wallach JD. Characteristics and Reporting of Number Needed to Treat, Number Needed to Harm, and Absolute Risk Reduction in Controlled
Clinical Trials, 2001-2019. JAMA Internal Medicine November 2020. E1-E3.
• Mostofsky E, Dunn JA, Hernández-Díaz S, Mittleman MA. Patient and Physician Preferences for Reporting Research Findings. Fam Med June 2019; 51(6):502-508. Available from:
https://journals.stfm.org/familymedicine/2019/june/mostofsky-2018-0384/.
• Minkow D. The Evidence-Based Medicine Pyramid! Students 4 Best Evidence April 2014. Available from: https://s4be.cochrane.org/blog/2014/04/29/the-evidence-based-medicine-
pyramid/.
• Grein J, Ohmagari N, Shin D. Compassionate Use of Remdesivir for Patients with Severe Covid-19. NEJM April 10, 2020. Available from:
https://www.nejm.org/doi/full/10.1056/NEJMoa2007016.
• RECOVERY pre-print. Available from: https://www.medrxiv.org/content/10.1101/2021.02.11.21249258v1
• Boerner H. Five-Day Course of Oral Antiviral Appears to Stop SARS-CoV-2 in Its Tracks. March 08, 2021. Available from:
https://www.medscape.com/viewarticle/947061?src=WNL_dne_210310_mscpedit&uac=102683EX&impID=3239028&faf=1#vp_1.
22 March 2021 31

32. Questions?
Now
• Please share any
– Questions
– Comments
– Musings
Later
• Contact me
– Email: zackdumont@me.com
– Twitter: @ZackDumontYQR
– LinkedIn: /ZackDumont
22 March 2021 32

33. Audience Q&A
• Zack’s answer
– Well, where to begin? He received/took quite a few medications throughout this
pandemic (eg, hydroxychloroquine). If speaking specifically about his
hospitalization, however, we might be able to pin it down to a couple/several:
dexamethasone, remdesivir, and REGN-COV2 monoclonal antibody.
– Dexamethasone is a corticosteroid that’s been around for quite some time. It’s
not magic, but of all the interventions, it’ll probably be one sticks. It’s quite
effective at knocking out inflammation. Why doesn’t it get much attention? It’s
been around long enough that it’s been genericized and, therefore, there aren’t
shareholders standing to make much gain.
– Remdesivir, or Veklury®, is the anti-malarial that was re-purposed to supposedly
work for COVID. It’s showed a bit of benefit in studies, but also some neutral and
negative findings. Why this can happen with studies: the drug probably doesn’t
do much, if anything, but if you study it enough times it will on occasion – due
purely to random chance – show a benefit in one study. One study might be all
you need to market your medication.
– The REGN-COV2 monoclonal antibody (Regeneron®) will probably stick around,
too. It’s not quite this simple, but one possible explanation of how it works: while
a vaccine is given to trick your body into thinking an infective agent is present
and then developing endogenous (from within) antibodies, this medication is a
short-cut and just gives you the antibodies (usually copied from someone who’s
had COVID)
22 March 2021 33
• Question
– What drug was given
to Trump?

34. Audience Q&A
• Zack’s answer
– My gut instinct says ‘no’, but I don’t like relying
on my “feelings”, so if it was (created in a lab)
all I can say is that there could be massive
implications. But, from a politically agnostic
perspective, it doesn’t matter. Pardon my
curtness. It doesn’t matter if it did or not
because this could have happened – and
eventually would have happened – naturally. In
addition, the idea shares characteristics with
conspiracy theories, which are almost always
untrue purely because they would be too
difficult to orchestrate.
22 March 2021 34
• Question
– What is your view on
the potential that
COVID came from a
lab?

35. Audience Q&A
• Zack’s answer
– Great question. In short, I think it’s a net problem. But there
are pros and cons. Most formularies are responsible for a
particular geographical area or population; so, a formulary can
be a bit more nimble and responsive to a specific population.
An overly simplistic example: in an area where there’s a higher
elderly population, they might dedicate more resources to
covering medications for seniors. The threat that probably
immediately comes to mind is that transferability is
immediately compromised; so, as soon as someone moves to
another jurisdiction without coverage then they’re scrambling
to change therapies. At the same time, a big single formulary is
possible, and just needs to be developed with higher levels of
detail. It should be sought-after over the long-term. We’re
fortunate in Canada to have CADTH (the Canadian Agency for
Drugs and Technology in Health) to help coordinate the many
formularies. Patience is required, no doubt. It’s been around
for a few decades and the ecosystem it works within is still very
very far away from realizing the power and economy of scale
from having a single formulary.
22 March 2021 35
• Question
– How much of a
problem is it having
so many formularies?

36. Audience Q&A
• Zack’s answer
– You’ve probably heard (Spider-Man’s) Uncle Ben’s saying: with great power
comes great responsibility? Well… that doesn’t fit here… at all. Instead, this
situation is probably best quoted with something like: with low expectations
comes low exertion. Bottom-line is that research doesn’t have to be conducted
well because it has nearly the same impact independent of its rigour. A study can
and will be marketed easily, even if poorly-conducted. A study can and will face
change intolerance, even if well-conducted. It’s such a paradox; everything
changes, yet everything stays the same. For every person that adopts drugs too
early and based on poor research, there’s nearly one-to-one someone who
refuses to budge even if the research is great. So, there’s perhaps not much
impetus.
– The second, and perhaps more charitable reason: good intentions (you can
probably think of a saying about good intentions, too ;) ). But, to be fair, educated
people are under intense pressure from patients and society to fix things. To fix
everything… and to fix it yesterday. This causes interventions to progress through
the research stages too quickly: a theory is tested in a lab, and before it even gets
a chance to get outside, someone learns about the theory and starts trialing it in
certain patients, and before it gets a chance to prove whether or not it works in
patients, someone finds out about it and is writing it into a guidelines, maybe
simply because they know an expert who believes it… and so on, and so on.
– There is a number of other reasons that we just can’t get into: publication quotas
for academics, decentralized study conduct, uncoordinated endeavours,
competing interests, politics, stigma, etc, etc, etc
22 March 2021 36
• Question
– How do so many
poor studies get run,
despite that
researchers should
be educated in
proper study design?