QJM Advance Access published October 8, 2008
Q J Med

Teaching critical appraisal and statisti...
R.S. Moharari et al.

Table 1 Comparison of the scores of the pretest and posttest multiple choice questions to assess the...
Critical appraisal and statistics

indicated that teaching the principles of critical            References
appraisal may ...
Upcoming SlideShare
Loading in...5



Published on

Published in: Health & Medicine, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. QJM Advance Access published October 8, 2008 Q J Med doi:10.1093/qjmed/hcn131 Teaching critical appraisal and statistics in anesthesia journal club R.S. MOHARARI1, E. RAHIMI1, A. NAJAFI1, P. KHASHAYAR2, M.R. KHAJAVI1 and A.P. MEYSAMIE3 From the 1Department of Anesthesiology, 2Research and Development Center and 3Department of Community Medicine, Tehran University of Medical Sciences, Sina Hospital, Tehran, Iran Received 21 April 2008 and in revised form 9 September 2008 Summary Background: This study was designed to assess evaluating a randomized controlled trial paper the effectiveness of a series of journal clubs held using the CONSORT checklist before and after the for anesthesiology residents in promoting their course. awareness of research methods and statistical Results: Residents’ awareness in the application of analysis, as well as their skills in critical thinking information improved (P = 0.012), as well as and appraisal. research methodology (combined study design and Material and methods: Twenty-four journal club application of information, P = 0.017). Their ability sessions were held between September 2006 and in critical appraisal did also significantly rise at the August 2007 for 16 residents of anesthesiology. end of the course (P < 0.001). A 31 multiple-choice question (MCQ) was taken as Conclusion: Journal clubs can enable residents to pretest and posttest to evaluate the participants’ develop the knowledge, expertise and enthusiasm level of awareness in research methodology and needed to undertake research plans and can also statistical analysis. Their competence in critical enhance their ability in critical thinking and thinking and appraisal was also evaluated by scientific reading. Introduction Residency training represents an ideal time when Attendance was closely monitored. In each session, clinicians may learn and practice the necessary a single article relevant to anesthesiology and skills for searching, evaluating and applying medical critical care medicine was discussed with respect knowledge. The journal club may play an important to statistical analyses and research methodology role in this regard.1 The present study was designed (type of study, biases and methods). Suitable articles to assess the effectiveness of a regular journal club were identified by Medline and EMBase English in teaching anaesthetic trainees research methodol- searches and were given to the participants 4 days ogy and critical appraisal skills. beforehand. At the beginning of each session, a senior resident presented a 10-min summary of the article including its objectives and findings. Then, a senior clinician focused in detail on a single Material and methods methodological issue according to a prepared Twenty-four journal-club sessions (biweekly for 1 h) syllabus (principles of statistics and research; con- were held between September 2006 and August cepts of risk, exposure, odds and bias; block 2007 for residents of anesthesiology (years 1–3). randomization; case–control studies, cohort studies, Address correspondence to M.R. Khajavi, MD, Sina Hospital, Tehran-11367-46911, Iran. email: mohammadreza.khajavi@gmail.com ! The Author 2008. Published by Oxford University Press on behalf of the Association of Physicians. All rights reserved. For Permissions, please email: journals.permissions@oxfordjournals.org Page 1 of 3
  2. 2. R.S. Moharari et al. Table 1 Comparison of the scores of the pretest and posttest multiple choice questions to assess the improvement of the participants’ level of awareness Pretesta Posttesta Mean differencea 95% CI P-value Methodology 6.8 Æ 2.5 8.8 Æ 3.9 2.0 Æ 3.3 0.4 to 3.7 0.017 Study design 4.8 Æ 2.0 5.2 Æ 2.2 0.4 Æ 2.0 À0.6 to 1.5 0.367 Application of information 2.0 Æ 1.2 3.6 Æ 2.4 1.6 Æ 2.4 0.4 to 2.8 0.012 Statistical analysis 3.8 Æ 2.2 3.8 Æ 2.3 0.0 Æ 2.0 À0.9 to 0.9 1.000 Total 10.6 Æ 4.0 12.6 Æ 5.5 2.0 Æ 4.0 0.0 to 4.0 0.045 a Mean Æ SD. randomized controlled trials and use of the Table 2 The CONSORT checklist showing the mean CONSORT checklist). score in terms of each item after the course In the early sessions, theoretical issues were discussed in a didactic style. As the knowledge of Item Posttest the residents improved, the sessions became mainly student-centered. To assess the overall improvement 1 Title and Abstract 0.78 Æ 0.9 in the participants’ awareness level, an identical 2 Background 1.56 Æ 0.6 pretest and posttest including 31 multiple-choice 3 Participants 0.89 Æ 0.7 (MCQ) questions was administered (Chronbach-a 4 Intervention 1.00 Æ 0.8 5 Objectives 1.06 Æ 0.7 score = 0.7); their mean values were compared using 6 Outcomes 1.17 Æ 0.7 paired samples test. Subgroup analyses were also 7 Sample size 1.61 Æ 0.5 performed for research methodology (22 MCQs, 11 8 Randomization—sequence generation 1.11 Æ 0.5 for study design and 11 for application of informa- 9 Randomization—allocation concealment 1.17 Æ 0.4 tion) and statistical analysis (nine MCQs), using 10 Randomization—implementation 0.67 Æ 0.9 Student’s t-test. In addition, to evaluate the partici- 11 Blinding 0.67 Æ 0.7 pants’ competence in critical thinking and appraisal, 12 Statistic analysis 1.17 Æ 0.8 they were asked to appraise a single randomized 13 Participant flow 1.06 Æ 0.8 clinical trial (RCT) paper, using the CONSORT 14 Recruitment 1.50 Æ 0.8 checklist, before and after the course. Two profes- 15 Baseline data 0.89 Æ 0.9 sors were also asked to do the same test to provide 16 Numbers analyzed 1.72 Æ 0.6 17 Outcomes and estimation 1.72 Æ 0.6 the reference answers; in case of any discrepancy, a 18 Ancillary analyses 1.50 Æ 0.9 third professor was asked to do the test. Participants 19 Adverse events 1.11 Æ 0.9 were instructed to give points to each of the 22 20 Interpretation 1.72 Æ 0.7 individual items in the CONSORT checklist 22 Generalizability 1.50 Æ 0.7 between 0 (not mentioned in the article at all) and 22 Overall evidence 1.50 Æ 0.6 1 (discussed in the article). Then, the scores of individual items on every answer sheet were summed to yield a total score for its respondent. declared a lack of knowledge in this area. However, The mean values of these total scores before and their posttest appraisal total score (mean Æ SD, after the course were compared using Paired 27.06 Æ 2.98) showed significant improvement Samples Test. (Table 2). However, when we segregated the total 22 items of the CONSORT checklist into 14 well-discussed items (19.56 Æ 1.82) and 8 poorly- Results discussed items (7.50 Æ 1.58), there appeared a There was a significant improvement in the partici- significant difference between the two compart- pants overall awareness level (Table 1) and this was ments (mean difference = 12.06; 95% CI clearly for ‘application of information’ but not for 10.90–13.21; P < 0.001 using unpaired t-test). ‘study design’ or ‘statistical analysis’. After combin- ing study design and application of data (method- ology), the study revealed a significant difference Discussion between the pre and posttest scores (P = 0.017). The potential use of a journal club in improving The pretest score on critical thinking and apprai- reading habits1 and awareness of biostatistics2 has sal was deliberately considered zero as all residents already been described. Other authors have also Page 2 of 3
  3. 3. Critical appraisal and statistics indicated that teaching the principles of critical References appraisal may also be achieved by this means.3–5 1. Linzer M, Brown JT, Delong ER, Siegel WC. Impact of Another review of six studies confirmed the journal medical journal club on house-staff reading habits, knowl- club’s significant role in promoting critical appraisal edge and critical appraisal skills: a randomized, control trial. skills6 but this view is not widely held.7 J Am Med Assoc 1988; 260:2537–41. Our study similarly suggested an improvement in 2. Cheatam ML. A structured curriculum for improved resident critical appraisal competency and in epidemiologic education in statistics. Am Surg 2000; 66:585. awareness level of the residents. We concur with 3. Kellum JA, Rieker JP, Power M, Powner DJ. Teaching critical Kuhn et al.8 that the unavailability of trained faculty appraisal during critical care fellowship training: A founda- members in the very areas, lack of sufficient time tion for evidence-based critical care medicine. Crit care Med 2000; 28:3067–70. and budget shortages are the factors which can limit the effectiveness of the journal club. We also found 4. Seeling CB. Affecting residents’ literature reading attitudes, that the participants were unaware of their lack of behaviors and knowledge through a journal club interven- tion. J Gen Intern Med 1991; 6:330–4. knowledge of epidemiology and critical appraisal skills, a situation that improved by the end of the 5. Cramer JS, Mahoney MC. Introducing evidence based medicine to the journal club, using a structured pre a post sessions. test: a cohort study. BMC Med Educ 2001; 1:6. In summary, we conclude that the journal club 6. Ebbert JO, Montori VM, Schultz HJ. The journal club in post represents a simple but effective means of effectively graduate medical education: a systematic review. Med Teach engaging residents in developing their research and 2001; 23:455–61. critical appraisal skills. We recommend that further 7. Bennett KJ, Sackett DL, Haynes RB, Neufeld VR, Tugwell P, research we control groups and larger sample sizes Roberts R. A controlled trial of teaching critical appraisal of should be performed in order to fully exploit the the clinical literature to medical students. J Am Med Assoc teaching potential of the journal club. 1987; 257:2451–4. 8. Kuhn GJ, Wyer PC, Cordell WH, Rowe BH. A Survey to determine the prevalence and characteristics of training in Acknowledgement evidence – based medicine in emergency medicine. J Emer Med 2005; 28:353–9. We acknowledge the support of the Research and Development Center of Sina Hospital. Page 3 of 3