CRITICAL APPRAISAL OF A
JOURNAL ARTICLE
VIPUL SHUKLA
JR II
PHARMACOLOGY
Critical appraisal is the process of carefully and systematically
examining research to judge its trustworthiness, its value and
relevance in a particular context.
It is essential to
• Combat information overload;
• Identify papers that are clinically relevant;
• Continuing Professional Development (CPD)
CASP(Critical Appraisal Skill Programme)
● It began in 1993, Oxford, UK to help health care decisions makers to
systematically assess the trustworthiness, relevance and results of published
of published papers.
● Similar critical appraisal programmes started in; Spain in 1997 and further
networks developed in Romania, Hungary, Japan, Poland, India and Norway.
● It provides critical appraisal checklist for different types of study designs.
PICO METHOD
Tool to assess the problem addressed in the article
• Problem-If the research has a focused question (problem).
• Intervention- Appropriate and clearly stated management
strategy.
• Comparison- A suitable control or alternative.
• Outcome - Desired results or patient related consequences
have been identified (outcomes).
The Critical Appraisal starts by double checking
the following main sections:
1. Overview of the paper
2. Abstract
3. Introduction/Background
4. Method and Material
5. Results
6. Discussion
7. References
Overview of the paper
• Journal quality ( by Impact factor, Cite score, Rejection rate,
peer review process in journal acceptance etc.)
• Article title ( Does it state key trial objectives)
• The author and their Institutions (General reputation of
organisation like APA,IEEE are highly regarded journals)
• Year of article published & if any new evidence has been
added since this publication.
• Conflict of interest, if any should be mentioned (regarding
funding, issues of a research grant)
2. ABSTRACT
• Abstract is a quick way of getting to know the article, its
purpose, major procedures/methods, main findings, and
conclusions.
• Aim of the study ( well and clearly written)
• Materials and Methods (study design and type of groups,
type of randomization process, sample size, gender, age, and
procedure rendered to each group and measuring tool(s)
should be mentioned)
• Results (variables with their statistical analysis and
significance)
• Conclusion (must clearly answer the question of interest)
3. Introduction/Background
• It include references to earlier work related to the area under
discussion and express the importance and limitations of what is
previously acknowledged.
• Study must address a significant or relevant problem within
healthcare.
• Why this study is considered necessary ?
• What is the purpose of this study ?
• What has been already achieved and how does this study be at
variance ?
• Does the scientific approach outline the advantages along with
possible drawbacks associated with the intervention ?
4.Method and Material
A. Study design
• Study design of the research is fundamental to the usefulness of the
study.
• Suboptimal study design can incorporate bias into the study.
• Two types of study are conducted in medical science
- Qualitative studies
- Quantitative studies
• Qualitative studies explore and understand people's beliefs,
experiences, attitudes, behaviour and interactions.
• They generate non-numerical data.
Examples of qualitative studies:
• Document - study of documentary accounts of events, such as
meetings.
• Passive observation - systematic watching of behaviour and talk in
natural occurring settings.
• Participant observation - observation in which the researcher also
occupies a role or part in the setting, in addition to observing.
• In depth interview - face to face conversation with the purpose of
exploring issues or topics in detail.
• Focus group - method of group interview which explicitly includes
and uses the group interaction to generate data.
QUANTITATIVE STUDIES
• Quantitative studies generate numerical data or data that can be
converted into numbers.
• Case report - report on a single patient.
• Case series - report on a series of patients.
• Case control study - identifies patients with a particular outcome
(cases) and without outcome(control),(useful in causation research)
• Cohort study - identifies two groups (cohorts) of patients one which
received the exposure of interest, and one which did not. Follows these
cohorts forward for the outcome of interest (useful in causation as well
as prognosis research).
Contd…
• Randomized Controlled Trial (RCT) - a clinical trial in which
participants are randomly allocated to a test treatment and a control;
involves concurrent enrolment and follow-up of both groups,( gold
standard in testing the efficacy of an intervention)
• Systematic review - identifies and critically appraises all research on
a specific topic, and combines valid studies; increasingly important in
evidence based medicine; different from review article (which is a
summary of more than one paper on a specific topic, and which may or
may not be comprehensive).
• Meta-analysis - a systematic review that uses quantitative methods
to summarise the results.
Educational Forum
• Publishes international research on education, encouraging the
advancement & transformation of teaching and learning for
educators.
• The main purpose of an educational forum is to share new
information & strategies.
• It provides a mean for people to increase their knowledge about
subject under consideration.
CLINICAL QUESTION
1. Aetiology/Causation
2. Therapy
3. Prognosis
4. Diagnosis
5. Cost effectiveness
INVESTIGATION METHOD
1. RCT/Case control/Cohort study
2. RCT/Systemic Review/MetaAnalysis
3. Cohort Study
4. Cohort/Case control study
5. Economical Analysis
B. Participants/Sample Population
• The sample should be the representative of the target population.
• Eligibility criteria should be mentioned clearly.
• It should be stated that where and how the samples were recruited.
• Sample size should be calculated rightly and it should be adequate to
detect statistical and clinical significant results.
• Participants should be allocated randomly with in the study.
• Level of blinding(single/double or triple blinding) should be
mentioned.
BIAS
• Bias is used to describe an error at any stage of the study that was not
due to chance.
• Bias leads to results in which there are a systematic deviation from
the truth.
• Good research design and appropriate sample population are
prerequisites to minimize any bias within a study.
• It should be noted that if the study is adequately powered to produce
statistically significant results, i.e., p-values quoted are <0.05.
C. STUDY FACTORS
• All the relevant study factors should be included in the study.
• All the factors should be measured using appropriate tools.
Confounding Factors:
- A confounder has a triangular relationship with both the exposure
and the outcome.
- Confounders are important source of bias so they should be
examined and controlled.
D. Statistical method
• Statistical methods should be appropriate to compare participants or
draw conclusion.
• Confidence intervals or p-values should be mentioned.
• p value - refers to the probability that any particular outcome would
have arisen by chance.
• A p-value of less than 1 in 20 (p<0.05) is statistically significant. When
p-value is less than significance level, which is usually 0.05, we often
reject the null hypothesis and the result is considered to be
statistically significant
5. RESULTS
• Correct statistical analysis of results is crucial to the reliability of the
conclusions drawn from the research paper.
• Statistical tests and result should answer the research question.
• Results should be presented in a way to help in health policy
decisions.
6.Discussion
• Questions posed in the study should be adequately addressed.
• The conclusions should be justified by the data.
• Shortcomings of the study should be addressed and constructive
suggestions given for future research.
• Conclusion should be convincing.
7. References
• It is a method used to demonstrate to readers that you have
conducted a thorough and appropriate literature search, and reading.
• It enhances the presentation of your work and shows that your
writing is based on knowledge and informed by appropriate academic
reading.
• Types ; Harvard system (k/a Author-Date system), British Standard
(Numeric) system used in social sciences, technology and natural
sciences.
• APA Style & Chicago style used in social sciences arts and humanities.
• Vancouver style system is most commonly used in medicine and
natural sciences.
BIBLIOGRAPHIC ELEMENT PUNCTUATION MARK
RCT
(Is the study valid for RCT)
1. Did the study address a clearly
focused research question?
2. Was the assignment of
participants to interventions
randomised?
Was the study methodologically
sound
3. Were the participants ‘blind’ to
intervention they were given?
• Were the investigators ‘blind’ to
the intervention they were giving to
participants?
• Were the people
assessing/analysing outcome/s
‘blinded’?
4. Were the study groups similar at
the start of the randomised
controlled trial?
5. Apart from the experimental
intervention, did each study group
receive the same level of care.
SYSTEMIC REVIEW
(Are the results of review valid)
1. Did the review address a clearly focused question?
2. Did the authors look for the right type of papers ?
3. Do you think all the important, relevant studies were
included?
4. Did the review’s authors do enough to assess quality of the
included studies?
RCT
( What are the results)
6. Were the effects of intervention
reported comprehensively?
7. Was the precision of the
estimate of the intervention or
treatment effect reported?
8. Do the benefits of the
experimental intervention
outweigh the harms and costs?
SYSTEMIC REVIEW
( What are the results)
5. What are the overall results of
the review?
6. How precise are the results?
RCT
( Will the results help locally)
9. Can the results be applied to
your local population/in your
context?
10. Would the experimental
intervention provide greater value
to the people in your care than
any of the existing interventions?
SYSTEMIC REVIEW
(Will the results help locally)
7. Can the results be applied to
the local population?
8. Were all important outcomes
considered?
9. Are the benefits worth the
harms and costs?
CONCLUSION
THANKYOU

critical appraisal ppt.pptx

  • 1.
    CRITICAL APPRAISAL OFA JOURNAL ARTICLE VIPUL SHUKLA JR II PHARMACOLOGY
  • 2.
    Critical appraisal isthe process of carefully and systematically examining research to judge its trustworthiness, its value and relevance in a particular context. It is essential to • Combat information overload; • Identify papers that are clinically relevant; • Continuing Professional Development (CPD)
  • 3.
    CASP(Critical Appraisal SkillProgramme) ● It began in 1993, Oxford, UK to help health care decisions makers to systematically assess the trustworthiness, relevance and results of published of published papers. ● Similar critical appraisal programmes started in; Spain in 1997 and further networks developed in Romania, Hungary, Japan, Poland, India and Norway. ● It provides critical appraisal checklist for different types of study designs.
  • 4.
    PICO METHOD Tool toassess the problem addressed in the article • Problem-If the research has a focused question (problem). • Intervention- Appropriate and clearly stated management strategy. • Comparison- A suitable control or alternative. • Outcome - Desired results or patient related consequences have been identified (outcomes).
  • 5.
    The Critical Appraisalstarts by double checking the following main sections: 1. Overview of the paper 2. Abstract 3. Introduction/Background 4. Method and Material 5. Results 6. Discussion 7. References
  • 6.
    Overview of thepaper • Journal quality ( by Impact factor, Cite score, Rejection rate, peer review process in journal acceptance etc.) • Article title ( Does it state key trial objectives) • The author and their Institutions (General reputation of organisation like APA,IEEE are highly regarded journals) • Year of article published & if any new evidence has been added since this publication. • Conflict of interest, if any should be mentioned (regarding funding, issues of a research grant)
  • 7.
    2. ABSTRACT • Abstractis a quick way of getting to know the article, its purpose, major procedures/methods, main findings, and conclusions. • Aim of the study ( well and clearly written) • Materials and Methods (study design and type of groups, type of randomization process, sample size, gender, age, and procedure rendered to each group and measuring tool(s) should be mentioned) • Results (variables with their statistical analysis and significance) • Conclusion (must clearly answer the question of interest)
  • 8.
    3. Introduction/Background • Itinclude references to earlier work related to the area under discussion and express the importance and limitations of what is previously acknowledged. • Study must address a significant or relevant problem within healthcare. • Why this study is considered necessary ? • What is the purpose of this study ? • What has been already achieved and how does this study be at variance ? • Does the scientific approach outline the advantages along with possible drawbacks associated with the intervention ?
  • 9.
    4.Method and Material A.Study design • Study design of the research is fundamental to the usefulness of the study. • Suboptimal study design can incorporate bias into the study. • Two types of study are conducted in medical science - Qualitative studies - Quantitative studies • Qualitative studies explore and understand people's beliefs, experiences, attitudes, behaviour and interactions. • They generate non-numerical data.
  • 10.
    Examples of qualitativestudies: • Document - study of documentary accounts of events, such as meetings. • Passive observation - systematic watching of behaviour and talk in natural occurring settings. • Participant observation - observation in which the researcher also occupies a role or part in the setting, in addition to observing. • In depth interview - face to face conversation with the purpose of exploring issues or topics in detail. • Focus group - method of group interview which explicitly includes and uses the group interaction to generate data.
  • 11.
    QUANTITATIVE STUDIES • Quantitativestudies generate numerical data or data that can be converted into numbers. • Case report - report on a single patient. • Case series - report on a series of patients. • Case control study - identifies patients with a particular outcome (cases) and without outcome(control),(useful in causation research) • Cohort study - identifies two groups (cohorts) of patients one which received the exposure of interest, and one which did not. Follows these cohorts forward for the outcome of interest (useful in causation as well as prognosis research).
  • 12.
    Contd… • Randomized ControlledTrial (RCT) - a clinical trial in which participants are randomly allocated to a test treatment and a control; involves concurrent enrolment and follow-up of both groups,( gold standard in testing the efficacy of an intervention) • Systematic review - identifies and critically appraises all research on a specific topic, and combines valid studies; increasingly important in evidence based medicine; different from review article (which is a summary of more than one paper on a specific topic, and which may or may not be comprehensive). • Meta-analysis - a systematic review that uses quantitative methods to summarise the results.
  • 13.
    Educational Forum • Publishesinternational research on education, encouraging the advancement & transformation of teaching and learning for educators. • The main purpose of an educational forum is to share new information & strategies. • It provides a mean for people to increase their knowledge about subject under consideration.
  • 14.
    CLINICAL QUESTION 1. Aetiology/Causation 2.Therapy 3. Prognosis 4. Diagnosis 5. Cost effectiveness INVESTIGATION METHOD 1. RCT/Case control/Cohort study 2. RCT/Systemic Review/MetaAnalysis 3. Cohort Study 4. Cohort/Case control study 5. Economical Analysis
  • 15.
    B. Participants/Sample Population •The sample should be the representative of the target population. • Eligibility criteria should be mentioned clearly. • It should be stated that where and how the samples were recruited. • Sample size should be calculated rightly and it should be adequate to detect statistical and clinical significant results. • Participants should be allocated randomly with in the study. • Level of blinding(single/double or triple blinding) should be mentioned.
  • 16.
    BIAS • Bias isused to describe an error at any stage of the study that was not due to chance. • Bias leads to results in which there are a systematic deviation from the truth. • Good research design and appropriate sample population are prerequisites to minimize any bias within a study. • It should be noted that if the study is adequately powered to produce statistically significant results, i.e., p-values quoted are <0.05.
  • 17.
    C. STUDY FACTORS •All the relevant study factors should be included in the study. • All the factors should be measured using appropriate tools. Confounding Factors: - A confounder has a triangular relationship with both the exposure and the outcome. - Confounders are important source of bias so they should be examined and controlled.
  • 18.
    D. Statistical method •Statistical methods should be appropriate to compare participants or draw conclusion. • Confidence intervals or p-values should be mentioned. • p value - refers to the probability that any particular outcome would have arisen by chance. • A p-value of less than 1 in 20 (p<0.05) is statistically significant. When p-value is less than significance level, which is usually 0.05, we often reject the null hypothesis and the result is considered to be statistically significant
  • 19.
    5. RESULTS • Correctstatistical analysis of results is crucial to the reliability of the conclusions drawn from the research paper. • Statistical tests and result should answer the research question. • Results should be presented in a way to help in health policy decisions.
  • 20.
    6.Discussion • Questions posedin the study should be adequately addressed. • The conclusions should be justified by the data. • Shortcomings of the study should be addressed and constructive suggestions given for future research. • Conclusion should be convincing.
  • 21.
    7. References • Itis a method used to demonstrate to readers that you have conducted a thorough and appropriate literature search, and reading. • It enhances the presentation of your work and shows that your writing is based on knowledge and informed by appropriate academic reading. • Types ; Harvard system (k/a Author-Date system), British Standard (Numeric) system used in social sciences, technology and natural sciences. • APA Style & Chicago style used in social sciences arts and humanities. • Vancouver style system is most commonly used in medicine and natural sciences.
  • 22.
  • 23.
    RCT (Is the studyvalid for RCT) 1. Did the study address a clearly focused research question? 2. Was the assignment of participants to interventions randomised? Was the study methodologically sound 3. Were the participants ‘blind’ to intervention they were given? • Were the investigators ‘blind’ to the intervention they were giving to participants? • Were the people assessing/analysing outcome/s ‘blinded’? 4. Were the study groups similar at the start of the randomised controlled trial? 5. Apart from the experimental intervention, did each study group receive the same level of care.
  • 24.
    SYSTEMIC REVIEW (Are theresults of review valid) 1. Did the review address a clearly focused question? 2. Did the authors look for the right type of papers ? 3. Do you think all the important, relevant studies were included? 4. Did the review’s authors do enough to assess quality of the included studies?
  • 25.
    RCT ( What arethe results) 6. Were the effects of intervention reported comprehensively? 7. Was the precision of the estimate of the intervention or treatment effect reported? 8. Do the benefits of the experimental intervention outweigh the harms and costs? SYSTEMIC REVIEW ( What are the results) 5. What are the overall results of the review? 6. How precise are the results?
  • 26.
    RCT ( Will theresults help locally) 9. Can the results be applied to your local population/in your context? 10. Would the experimental intervention provide greater value to the people in your care than any of the existing interventions? SYSTEMIC REVIEW (Will the results help locally) 7. Can the results be applied to the local population? 8. Were all important outcomes considered? 9. Are the benefits worth the harms and costs?
  • 27.
  • 28.