This document provides an introduction to critical appraisal of literature. It discusses the importance of critically evaluating research to separate reliable evidence from unreliable evidence. It outlines the process of critical appraisal, including asking a focused question, finding relevant evidence, and using appraisal tools to systematically examine research quality, validity, and relevance. The document also introduces some key statistical concepts used in research, such as p-values, confidence intervals, risk reduction, and number needed to treat. The goal of critical appraisal is to make informed decisions about integrating research findings into clinical practice and policy.
Introduction to meta-analysis (1612_MA_workshop)Ahmed Negida
Chapter 1: Introduction to Meta-analysis
- From the 1612 MA Workshop that will be held on 11th, December, 2016 at Dokki, Giza, Egypt
- Workshop instructor: Mr. Ahmed Negida, MBBCh candidate
Study designs, Epidemiological study design, Types of studiesDr Lipilekha Patnaik
Study design, Epidemiological study designA study design is a specific plan or protocol
for conducting the study, which allows the investigator to translate the conceptual hypothesis into an operational one.
This presentation, presented to senior thesis students at UC Berkeley, reviews the uses of qualitative research methods such as ethnography in public health, walking students through methods, sampling, ensuring rigor, and analysis with CAQDAS software such as Atlas.ti
Introduction to meta-analysis (1612_MA_workshop)Ahmed Negida
Chapter 1: Introduction to Meta-analysis
- From the 1612 MA Workshop that will be held on 11th, December, 2016 at Dokki, Giza, Egypt
- Workshop instructor: Mr. Ahmed Negida, MBBCh candidate
Study designs, Epidemiological study design, Types of studiesDr Lipilekha Patnaik
Study design, Epidemiological study designA study design is a specific plan or protocol
for conducting the study, which allows the investigator to translate the conceptual hypothesis into an operational one.
This presentation, presented to senior thesis students at UC Berkeley, reviews the uses of qualitative research methods such as ethnography in public health, walking students through methods, sampling, ensuring rigor, and analysis with CAQDAS software such as Atlas.ti
Basics of Systematic Review and Meta-analysis: Part 3Rizwan S A
A 4 part lecture series on the basics of Systematic Review and Meta-analysis, Part 3 discusses the software needed and analytical techniques used for this purpose.
Critical appraisal is the process of carefully and systematically analyze the research paper to judge its trustworthiness, its value and relevance in a particular context. (Amanda Burls 2009)
A critical review must identify the strengths and limitations in a research paper and this should be carried out in a systematic manner.
The Critical Appraisal helps in developing the necessary skills to make sense of scientific evidence, based on validity, results and relevance.
Basics of Systematic Review and Meta-analysis: Part 3Rizwan S A
A 4 part lecture series on the basics of Systematic Review and Meta-analysis, Part 3 discusses the software needed and analytical techniques used for this purpose.
Critical appraisal is the process of carefully and systematically analyze the research paper to judge its trustworthiness, its value and relevance in a particular context. (Amanda Burls 2009)
A critical review must identify the strengths and limitations in a research paper and this should be carried out in a systematic manner.
The Critical Appraisal helps in developing the necessary skills to make sense of scientific evidence, based on validity, results and relevance.
Judith Goldberg MedicReS World Congress 2014MedicReS
Statistics in Clinical and Translational Research in Drug Development - Judith D. Goldberg, Sc.D. Professor
Division of Biostatistics New York University School of Medicine
Critical appraisal of evidence/journal clubdassoumitradr
journal club: A Randomized Double-Blind Study of Risperidone
and Olanzapine in the Treatment of Schizophrenia
or Schizoaffective Disorder(Am J Psychiatry 2001; 158:765–774)
Study of the distribution and determinants of
health-related states or events in specified populations and the application of this study to control health problems.
John M. Last, Dictionary of Epidemiology
Evidence based decision making in periodonticsHardi Gandhi
INTRODUCTION TO EVIDENCE BASED DENTISTRY
EVIDENCE BASED PERIODONTOLOGY
NEED, PRINCIPLES, GOALS AND ADVANTAGES OF EBDM
SKILLS NEEDED FOR EBDM
ASSESING THE EVIDENCE
INCORPORATING INTO THE PRACTICE
Chapter 2
Study Designs
Learning Objectives
• List and define the components of a good
study design
• Compare and contrast observational and
experimental study designs
• Summarize the advantages and disadvantages
of alternative study designs
Learning Objectives
• Describe the key features of a randomized
controlled trial
• Identify the study designs used in public health
and medical studies
Study Designs
• Observational Studies
– Case-series study
– Cross-sectional (prevalence) survey
– Case-control study
– Cohort study
• Experimental Studies
– Randomized Controlled (Clinical) Trial
Inferences
• Observational studies – inferences limited to descriptions
and associations; with carefully designed analysis can
make stronger inferences (statistical adjustment)
• Experimental studies – cause and effect
In ALL studies – need careful definition of disease
(outcome) and exposure (risk factor)
Which Design is Best
• Depends on the study question
• What is current knowledge on topic
• How common is disease (and risk factors)
• How long would study take, what are costs
• Ethical issues
Case Report/Case Series
• Observational study
• Case report: Detailed report of specific
features of case
• Case series: Systematic review of common
features of a small number of cases
• Advantage: Cost-efficient
• Disadvantages: No comparison group, no
specific research question
Case-Series
• Simplest design – description of interesting
observations in a small number of individuals
• Usually case-series do not involve control patients
(i.e., patients free of disease)
• Usually lead to generation of hypotheses for more
formal testing
• Criticisms: not planned – no research hypotheses
Case-Series
• Gottleib (1981) studied 5 young homosexual
men with rare form of pneumonia and other
unusual infections
• Initial report was followed by more series (26
cases in NY and CA; “cluster” in southern CA;
34 cases among Haitians, etc.)
• Condition termed AIDS in 1982
Cross-Sectional Survey
• Observational study conducted at a point in
time
• Advantages: Cost-efficient, easy to implement,
ethical
• Disadvantages: No temporal information, non-
response bias
Cross-Sectional Survey
• Is there an association between diabetes and
cardiovascular disease (CVD)?
Patients
with
Diabetes
Patients without
Diabetes
Patients with
CVD
Prospective Cohort Study
• Observational study involving a group (cohort)
of individuals who meet inclusion criteria
followed prospectively in time for risk factor
and outcome information
• Advantages: Can assess temporal relationships
• Disadvantages: Need large numbers for rare
outcomes, confounding
Cohort Study
• Is there an association between hypertension and
cardiovascular disease?
CVD
Hypertension
No CVD
Cohort
CVD
No Hypertension
No CVD
Study Start Time
Cohort Studies
• Identify a group of individuals that meet
inclusion crit ...
Biostatistics is one of the most unavoidable area in the modern day practice of evidence based medicine . In the ppt , trying to give a glimpse on how a clinician should approach Biostatistics
Sharpen existing tools or get a new toolbox? Contemporary cluster initiatives...Orkestra
UIIN Conference, Madrid, 27-29 May 2024
James Wilson, Orkestra and Deusto Business School
Emily Wise, Lund University
Madeline Smith, The Glasgow School of Art
Have you ever wondered how search works while visiting an e-commerce site, internal website, or searching through other types of online resources? Look no further than this informative session on the ways that taxonomies help end-users navigate the internet! Hear from taxonomists and other information professionals who have first-hand experience creating and working with taxonomies that aid in navigation, search, and discovery across a range of disciplines.
This presentation by Morris Kleiner (University of Minnesota), was made during the discussion “Competition and Regulation in Professions and Occupations” held at the Working Party No. 2 on Competition and Regulation on 10 June 2024. More papers and presentations on the topic can be found out at oe.cd/crps.
This presentation was uploaded with the author’s consent.
0x01 - Newton's Third Law: Static vs. Dynamic AbusersOWASP Beja
f you offer a service on the web, odds are that someone will abuse it. Be it an API, a SaaS, a PaaS, or even a static website, someone somewhere will try to figure out a way to use it to their own needs. In this talk we'll compare measures that are effective against static attackers and how to battle a dynamic attacker who adapts to your counter-measures.
About the Speaker
===============
Diogo Sousa, Engineering Manager @ Canonical
An opinionated individual with an interest in cryptography and its intersection with secure software development.
Acorn Recovery: Restore IT infra within minutesIP ServerOne
Introducing Acorn Recovery as a Service, a simple, fast, and secure managed disaster recovery (DRaaS) by IP ServerOne. A DR solution that helps restore your IT infra within minutes.
This presentation, created by Syed Faiz ul Hassan, explores the profound influence of media on public perception and behavior. It delves into the evolution of media from oral traditions to modern digital and social media platforms. Key topics include the role of media in information propagation, socialization, crisis awareness, globalization, and education. The presentation also examines media influence through agenda setting, propaganda, and manipulative techniques used by advertisers and marketers. Furthermore, it highlights the impact of surveillance enabled by media technologies on personal behavior and preferences. Through this comprehensive overview, the presentation aims to shed light on how media shapes collective consciousness and public opinion.
2. Objectives
• Why we need to evaluate literature
• How to form a question that will result in a
targeted literature search
• Critical appraisal of the evidence – using
appraisal tools/checklists
• Thoughts on statistics
3. What is Critical Appraisal
“Critical appraisal is the process of carefully and systematically
examining research to judge its trustworthiness, and its value
and relevance in a particular context”
Burls (2009) What is critical appraisal?
http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/What_is_critical_appraisal.pdf
More than just reading and summarising a paper.
4. Critical Appraisal is:
• Balanced assessment of strengths of research against its weaknesses
• Assessment of research process and results
• Consideration of quantitative and qualitative aspects of research
• To be undertaken by all health professionals as part of their work
as just one part of the decision making process
5. Critical Appraisal is not:
• Negative dismissal of a piece of research
• Assessment of results alone
• Based entirely on detailed statistical analysis
• To be undertaken by expert researchers only
6. Why bother?
• Published evidence is not always reliable –
a way of separating “wheat from the chaff”
• Only an estimated 2% is judged clinically relevant
• Patients are better informed and increasingly present
having read material on the web or in the media
7. Challenges:
• Can be time-consuming
• It does not always give you an easy answer – there are still
judgements and decisions to be made
• It may not give the expected or hoped for result
• It can be dispiriting if it highlights a lack of good evidence
– need determination to persist with an area of interest
when access to good research in the area is limited
• Maths and statistics …..!
8. When to do Critical Appraisal
Ask the
question
Find the
evidence
Appraise and
interpret the
evidence
Act on appropriate
evidence
Evaluate and
reflect
9. Step 1: Asking the question
A well structured question is the key to finding evidence
Contains 4 parts (usually) … PICO
PICO
P = patient or problem
How would you describe a group of patients similar to yours?
I = intervention or treatment
What is the main action you are considering?
C = comparison or alternative
What is/are the other option(s) available?
O = outcome
What do you/the patient want to happen?
10. Step 2: Finding the evidence
IDENTIFICATION OF THE BEST AND
MOST RELEVANT RESOURCE(S) TO
SEARCH SHOULD START FROM THE
TOP OF THE EVIDENCE HIERARCHY
Systematic
Reviews
Expert Opinion
Randomised
Controlled
Trials
11. Anatomy of a paper
Look for:
Abstract - abbreviated form of the study
Introduction - problem being studied and explains why the research is being
carried out
Methods - information concerning the experiment design, including study
sample, treatment allocation, outcome measures
Results - clear presentation of the experiments results, including any adverse
effects or variables that may have affected the outcome
Discussion/conclusion - gives the author a chance to give their viewpoint
and explain any clinical importance of the experiment
12. Appraisal Tools
• Several different appraisal tools are available
• Check sheets of question
• Different tools for different types of research
• Helps make appraisal quick, simple, effective and consistent
• Can help get rid of personal bias
• Help identify if results are significant and valid
13. Implications of the research:
Consider:
• Does it produce new knowledge?
• How applicable is the research to my own patients?
Perhaps the ultimate question …
• Should you act on the findings and change practice?
16. Some numbers and statistics
• Presentation of statistics
• Sample Size
• P Values
• Confidence Intervals
• Risk Reduction
• Number Needed to Treat
Disclaimer:
(Most) Librarians are not statisticians
This session is not about in depth-
statistics but about how to interpret
statistics when reading the paper
17. Presentation of Statistics
• The same statistics can be presented in numerous ways
• The way in which statistics are presented can affect how we
interpret the results
• Authors may use this when writing papers to suggest a
particular view of the results
18. Shock Horror!
Daily Mail Online
Why eating just one
sausage a day increases
your risk of cancer by 20%
http://www.dailymail.co.uk/health/article-550729/Why-eating-just-
sausage-day-raises-cancer-risk-20-cent.html
19. Critically Appraised
Why eating just one sausage a day raises your
cancer risk by 20 per cent
• Specifics – This refers to Bowel Cancer
• Baseline – the usual risk for bowel cancer in a lifetime is:
- 1 in 18 for women (5.6%)
- 1 in 20 for men (5%)
• The risk with a “sausage a day” is not 25% (5% + 20%) or 1 in 4
• The risk with a “sausage a day” increases by 20% of the
original risk (20% of 5% = 1%)
So the new risk if you eat a sausage is a far less dramatic:
- 1 in 15 for women (6.6%)
- 1 in 17 for men (6%)
20. Which would you prefer to fund?
The following are four different rehabilitation programmes for heart attack
victims:
Programme A – which reduced the rate of deaths by 20%
Programme B – which produced an absolute reduction in deaths of 3%
Programme C – which increased patients’ survival rate from 84% to 87%
Programme D – which meant that 31 people needed to enter the
programme to avoid one death
21. Which would you prefer to fund?
The following are four different rehabilitation programmes for heart attack
victims:
Programme A – which reduced the rate of deaths by 20%
Programme B – which produced an absolute reduction in deaths of 3%
Programme C – which increased patients’ survival rate from 84% to 87%
Programme D – which meant that 31 people needed to enter the
programme to avoid one death
They all refer to the same study!
(Taken from: How to read a paper / Trisha Greenhalgh, 2010, 4th ed.)
22. Sample Size
“A trial should be big enough to have a high chance of detecting…
a worthwhile effect if it exists, and thus to be reasonably sure that
no benefit exists if it is not found in the trial”
Practical Statistics for Medical Research/ Doug Altman, 1991
• A sample size may be given in the paper
• Don’t worry about how they have calculated it but……
• If they do provide a sample size check if they actually achieve
the stated number in their recruiting figures
• If they do not provide the sample size look at the numbers
included in the trial as low numbers can mean a less reliable study
23. P Values
Are useful in deciding if a result could be just by chance:
0 1
Impossible that result is by chance Certainty that result is by chance
(Good) (Bad)
P= 0.001 Very unlikely 1 in 1000
P= 0.05 Fairly unlikely 1 in 20
P= 0.5 Fairly likely 1 in 2
P = 0.75 Very likely 3 in 4
A P value of <0.05 is generally considered statistically significant
If P <0.05 then the result in question is likely to be more than just chance
24. Confidence Intervals #1
“A confidence interval….allows you to estimate for both positive trials and
negative ones whether the strength of the evidence is strong or weak.”
Greenhalgh, Trisha (2014) How to read a paper
If the same trial was run many, many times you might not expect the exact
same results every time – but the results should lie within a specific range.
With a 95% CI we are saying that 95% of the time the results lie within a
specific range.
Trial result
Range of results of 95%
Exceptions to the rule (outliers)
(The other 5%)
25. Confidence Intervals
Commute Example
John’s journey home
Average = 50 minutes
Can often take up to 55 minutes
Can be as quick as 45 minutes
Very occasionally (5% of time or less) it can take over an hour
(eg: traffic accidents etc.) or can be quicker if he travels at off peak times
John’s journey time could be expressed as: 50 min (95% C.I. 45 min – 55 min)
Avishai Teicher via the PikiWiki - Israel free image collection project
26. Confidence Intervals #2
The range of the confidence interval (CI) is important.
The narrower it is the more accurate and precise the results
The broader it is the wider the possibility and range of likely results
Referring back to our commute example:
It is more useful to tell someone it may take between 45 and 55 minutes to travel somewhere than it would
be to say it could take between 30 minutes and 2 hours. It would help them to plan better.
Precision is usually preferable.
27. Risk Reduction
Absolute Risk Reduction (ARR)
If an intervention decreases risk of an event occurring from
2/100 to 1/100
1% absolute risk reduction
Relative Risk Reduction (RRR)
If an intervention decreases risk of an event occurring from
2/100 to 1/100
50% relative risk reduction
28. Number Needed to Treat (NNT)
For every n patients treated with the intervention, one extra
patient would be expected to benefit
i.e. how many people do you need to treat to be certain that one
person would benefit
Example: If NNT = 10 then on average for every ten patients
treated we would expect one of them to benefit
29. Number Needed to Treat (NNT)
Extreme examples:
For the use of antibiotics Cimetidine, Ranitidine & Omeprazole to
eradicate Heliobacter Pylori from the stomach
NNT = 1.3
For the use of cholesterol lowering Statins in the primary
prevention of vascular events like stroke and heart disease
NNT = 1000
30. Useful Calculations
Absolute Risk Reduction (ARR) = Control Rate – Experimental Rate
Relative Risk Reduction (RRR) = Absolute Risk Reduction / Control Rate
Number Needed to Treat (NNT) = 100 / Absolute Risk Reduction
Control Rate = Chance of outcome in control group (%)
Experimental Rate= Chance of outcome in experimental group (%)
31. Example Exercise
Infected with flu? Yes No Total
New Trial Drug 38 73 111
Placebo (control) 79 35 114
Total 117 108 225
Outcome: Preventing influenza
Control Rate = Chance of outcome in control group (%)
(79/114) x 100 = 69%
Experimental Rate= Chance of outcome in experimental group (%)
(38/111) x 100 = 34%
32. Example Exercise (continued)
Outcome: Preventing influenza
Absolute Risk Reduction (ARR) = Control Rate – Experimental Rate
69 - 34 = 35%
Relative Risk Reduction (RRR) = Absolute Risk Reduction /Control Rate
35 / 69 = 0.5%
Number Needed to Treat (NNT) = 100 / Absolute Risk Reduction
100/35 = 2.85 round up to 3
Control Rate = 69%
Experimental Rate=34%
33. Practice Paper Part 2
Look at the checklist and try to respond to points 7 - 11
• What is the sample size and do they achieve it?
• Are there any P values given in the results and what do these suggest?
Looking at the primary outcome measure
• What is the absolute risk reduction ?
• What is the relative risk reduction ?
• What is the number needed to treat?
35. Further Reading
Greenhalgh, Trisha (2014) How to read a paper: the basics of evidence-
based medicine. Oxford: Wiley Blackwell. 5th ed.
Crombie, Iain K (1996) Pocket guide to critical appraisal.
London: BMJ.
Straus, Sharon (ed.) (2005) Evidence-based medicine: how to practice and
teach EBM. Edinburgh: Elsevier Churchill Livingstone. 3rd ed.
Ajetunmobi, Olajide (2002) Making sense of critical appraisal.
London: Arnold.
36. Conclusion
We have examined:
• Why we need to evaluate literature
• Critical appraisal of the evidence –
using appraisal tools / checklists
• Statistics introduction
37. Contact Details
Dominic Gilroy
Library & Knowledge Service Manager
Leeds & York Partnerships NHS FT
dominic.gilroy1@nhs.net
0113 85 55658
Leeds Libraries for Health
www.leedslibraries.nhs.uk