Critical Appraisal Overview

Critical Appraisal Master class
Tues 10th May

Critical Appraisal: WTF?
(well that’s fab)

Introductions
Elspeth Hill
MRes Medical Sciences
PhD Education Student
Journal Reviewer
Cochrane Author
Nervous wreck
James Durrand
BSc Pathology
MB ChB (nearly!)
Academic Foundation Post
RSM Rep
Massive geek
You?

Outline
Evidence-based medicine
Principles of critical appraisal
Jargon busting
Clinical studies
Lab stuff
Qualitative work

Evidence-Based Medicine
What is it?

Structure of EBM

Critical Appraisal
Should I believe this paper?
Clinical
Should I treat my patients differently based on this paper?
Education
Should we include this in the medical school curriculum?
Lab Science
Should we divert resources to investigate this further?

3 Keys of Appraisal
Validity
Applicability
Impact

Validity
Validity:
How far has this paper actually answered the question they set out to answer?
Question:
Does endovascular emergency AAA repair reduce mortality?
Answer:
Endovascular emergency AAA repair is associated with reduced blood loss.

Applicability
Applicability
Does the evidence from this paper apply to my patients?
Trial Title
Cleverstatin reduces cardiac events in male patients with type II diabetes.
A randomized controlled trial
Population
1000 patients in Helsinki, Finland
Should I change my Wigan
patients to the more expensive
cleverstatin?

Impact
Impact
How much of a difference would this make?
Trial Title
High flow oxygen increases mortality in acute myocardial infarction. A randomised controlled trial.
Recommendation
Patients should no longer be given maximal oxygen therapy in acute MI.
I am the SHO in A&E: a man presents with crushing chest pain
– am I really going to withhold oxygen?

How to approach a paper!

Case Study

First impressions
What journal?
BMJ vs The Antarctic Journal of Homeopathic Resuscitation
Impact factors
Conflict of Interests
Track Record
Abstract?

Research Question
What is the research question at the centre ofthis paper?
Is it an important question?
Did I learn anything from reading this paper?

Introduction
. Outlines and references preceding work and ‘the story so far’.
. Points out weaknesses and gaps in preceding work.
. Justifies need and usefulness of the study.
. Puts the paper in context.

Methods

What kind of a study is it?

Expert opinion
Thoughts of one person who has seen many cases

Case Reports/Series
What happened with one patient?
What happened with a few patients?

Case-control studies
Risk Factors
Time

Cohort studies
Time

Confounding factors
C
Smoking
Chewing Gum
Lung Cancer
A
B

Randomized controlled trials (RCTs)
...Randomised…
– minimisebias by randomly assigning subjects to groups
...Controlled…
– compare your intervention to a control, where the only difference is the intervention
…Trial…
– you don’t know what’s going to happen

The Risk of Bias
‘Systematic differences between the two groups’
Selection bias
Treatment bias
Observer bias

Minimising bias
Blinding
Double blinding
Triple blinding
Losing will to live?

Results
Groups at baseline
Intention to treat
Selective outcome reporting
Subgroup analyses
Side effects

Discussion
What does this mean?
How was the data interpreted?
What does this mean for practice?
Does it fit with other studies?
Have they acknowledged/justified the flaws?
Conclusion
Is this a reasonable conclusion?
Does it answer the research question?

Lab Research
What is the research question?
Description – observation
Clarification – mechanism
Very similar to clinical studies, but watch out for relevance
cell biology vs animal studies vs human tissuevshuman trials
Does it follow from previous research – theoretical justification

n numbers, selective reporting
are the results meaningful – does it matter if the CRP is high if all the animals died?
good modeling – young animals do not represent aged obese smoking Salford residents
adequacy of controls – is that really the only difference
outcome assessment
are the controls normal
are the interventions different, how was this assessed
is there another explanation
how does it fit with existing research
do their conclusions support extrapolations

Qualitative/Education
Scientific methods - leaves on trees
Did they need to use words?
Well-problematized topic
Clear research questions – relevance to the real world?
Appropriate methodology - RCT in education
Clear recipe methods
Limitations and conclusions
Big problem
So many qualitative methods,
but 90% is description,
very little is justification,
even less clarification

Take home messages
take nothing at face value
healthy skepticism
anyone can do it!

Further Reading
Books:
How to read a paper. Greenhalgh T. BMJ books. Blackwell publishing
The doctors guide to critical appraisal. Gosall N, Gosall G. Pastest books
Bad Science. Ben Goldacre
Papers:
Why Most Published Research Findings Are False. John Ioannidis
Courses:
Research methods and critical appraisal-
www.rsm.ac.uk (events)

Critical Appraisal Overview

by Fastbleep

Accessibility

Categories

Upload Details

Usage Rights

Statistics