1. Residents’ Education
Developing an Integrated Evidence-Based
Medicine Curriculum for Family Medicine
Residency at the University of Alberta
G. Michael Allan, MD, Christina Korownyk, MD, Amy Tan, MD, Hugh Hindle, MD,
Lina Kung, PhD, MD, and Donna Manca, MD
Abstract
There is general consensus in the committee drafted goals and objectives, practice; and a journal club to support
academic community that evidence- the primary of which were to assist peer learning and growth of rapid
based medicine (EBM) teaching is residents to (1) become competent self- appraisal skills. Issues including time use,
essential. Unfortunately, many directed, lifelong learners with skills to costs, and change management are
postgraduate programs have significant effectively and efficiently keep up to discussed. Ongoing evaluation of the
weakness in their EBM programs. The date, and 2) develop EBM skills to solve curriculum and its components is a
Family Medicine Residency committee at problems encountered in daily practice. principal factor of the design, allowing
the University of Alberta felt their EBM New curriculum components, each critical review and adaptation of the
curriculum would benefit from critical evidence based, were introduced in 2005 curriculum. The first two years of the
review and revision. An EBM Curriculum and include a family medicine EBM curriculum have yielded positive feedback
Committee was created to evaluate workshop to establish basic EBM
from faculty and statistically significant
previous components and develop new knowledge; a Web-based Family
improvement in multiple areas of residents’
strategies as needed. Input from Medicine Desktop promoting easier
opinions of the curriculum and comfort
stakeholders including faculty and access to evidence-based Internet
with evidence-based practice.
residents was sought, and evidence resources; a brief evidence-based
regarding the teaching and practical assessment of the research project Acad Med. 2008; 83:581–587.
application of EBM was gathered. The enhancing integration of EBM into daily
There is general consensus in the curricula often fail2 and that teaching management, and details of each newly
academic community that evidence- critical appraisal yields small changes in introduced component, including costs
based medicine (EBM) teaching is knowledge but no change in application.3 and evaluation. Finally, we review and
essential.1 Unfortunately, research has Although classroom education improves discuss the evidence for this curriculum
shown that critical-appraisal-based knowledge, integrated EBM curricula and its components.
improve skills, attitudes, and behavior4
and result in incorporation of EBM in
patient care.5 Furthermore, research Background
Dr. Allan is assistant professor, Department of
Family Medicine, University of Alberta, Edmonton,
shows that practicing clinicians do not The University of Alberta Family
Alberta, Canada; and research fellow, Institute of perform critical appraisals to answer their Medicine Residency Program is a two-
Health Economics, Edmonton, Alberta, Canada. daily questions6 but choose preappraised, year program with approximately 60
Dr. Korownyk is assistant professor, Department summarized sites.6,7 An integrated EBM residents in each year (currently, 40
of Family Medicine, University of Alberta, Edmonton, curriculum designed to improve urban-area-based residents, 12 rural-
Alberta, Canada. knowledge, skills, attitudes, and area-based residents, 2 military-
Dr. Tan is clinical lecturer, Department of Family incorporation into clinical practice might sponsored residents, 6 Alberta
Medicine, University of Alberta, Edmonton, Alberta, include EBM-teaching methods of proven International Medical Graduate Program
Canada.
success such as workshops,8,9 journal residents, and 1 to 3 externally sponsored
Dr. Hindle is rural academic development clubs,10,11 daily generation of clinical residents). There are five clinical teaching
coordinator, Rural Physician Action Plan, Edmonton, questions,12 summarized evidence-based units in Edmonton and two rural centers.
Alberta, Canada; and a family physician, Sylvan Lake,
Alberta, Canada. resources, and Web-based activities.13 In their first year, the residents have a
Unfortunately, there is very little four-month family medicine rotation
Dr. Kung is a family physician, Calgary, Alberta,
Canada.
information available in the literature (three months for rural-area-based
describing integrated, practical EBM residents) at one of the teaching units
Dr. Manca is assistant professor, Department of
Family Medicine, University of Alberta, Edmonton,
curricula, particularly in family medicine. and have access to a desktop computer
Alberta, Canada; and clinical director, Alberta Family while there.
Practice Research Network, Edmonton, Alberta, We describe an integrated EBM
Canada. curriculum for a family medicine At a monthly residency program
Correspondence should be addressed to Dr. Allan, residency that was implemented at the committee meeting in 2004, residents
Department of Family Medicine, University of University of Alberta in 2005. In this raised concerns that components of the
Alberta, 901 College Plaza, Edmonton, Alberta,
Canada, T6G 2C8; telephone: (780) 472-5038; fax: article, we outline the background of the family medicine residency curriculum
(780) 4 72-5192; e-mail: (michael.allan@ualberta.ca). curriculum, the process of change involving EBM were labor intensive but
Academic Medicine, Vol. 83, No. 6 / June 2008 581

2. Residents’ Education
offered minimal benefit. There was
consensus among residents and List 2
faculty that the EBM curriculum had Learning Objectives of the University of Alberta Family Medicine Resident
deficiencies. The EBM curricular Evidence-Based Medicine (EBM) Curriculum
components at that time consisted
Knowledge component
of a critical appraisal project, a librarian/
1. Understand the rationale and benefits of EBM.
informatics session, a journal club, and a
2. Provide a strong foundation in the basic principles of EBM, including
practice audit and quality-improvement
● Recognizing and formulating clinical questions;
project. An EBM curriculum (EBMC)
committee was formed to investigate the ● Finding and accessing information (refining questions and searching the most suitable
resource);
concerns and develop a new curriculum.
● Interpreting information (appropriately evaluating information in a quick, reliable format);
EBMC committee members and the chair
and
were nominated and approved by
● Applying information.
members of the residency program
committee. 3. Provide and identify online resources, educational tools, and web-links.
Skill component
The EBMC committee consisted of two 1. Learn to identify problems/questions encountered in practice and seek solutions by rapidly
urban faculty members, one rural faculty answering clinical questions with best evidence.
member, two urban residents, and one 2. Build skills and comfort in knowledge transfer (primarily through oral presentation skills and
rural resident. The committee chair discussion with patients).
completed a detailed literature search of Attitude component
EBM teaching, the application of EBM to 1. Facilitate appreciation and enthusiasm for
practice, and answering questions in ● The judicious use of current evidence to optimize patient care, and
clinical practice, and the summarized ● Maintenance of skills and knowledge through life-long self-directed learning.
findings were made available to the
committee. Guided by the information
gathered in the literature search, the little in gaining the skills to regularly second-year resident to review and
committee developed goals and answer clinical questions. present in a didactic format. Because
objectives (Lists 1 and 2) and then topics did not stem from resident
The librarian/informatics session, a interest, there was little motivation for
reviewed components of the curriculum yearly, hourlong, large-group didactic
and concerns of residents and faculty. resident involvement.
session including an assigned literature
Review of previous curriculum searching project, needed modification. The practice and quality-improvement
The EBMC committee determined that, project, a one-time, second-year project,
Once during the first year, each resident in this form, the session was minimally required each resident to review clinic
searched the literature and selected an effective in teaching residents the skills charts to determine whether practice
article he or she felt best addressed one of necessary to formulate clinical questions patterns followed established practice
their clinical questions. Each resident and identify resources applicable to guidelines for specific clinical concerns.
then critically appraised the article and family medicine. The EBMC committee thought the
presented the findings to the faculty and practice and quality-improvement
residents at his or her teaching site. This The journal club, a monthly, one-hour, project offered good insight into practice
critical appraisal project was eliminated small-group session during academic patterns and factors influencing best
because it was time consuming, not half-day (AHD), was unpopular. Articles practice.
relevant to daily practice, and offered were selected by faculty and provided to a
On considering the curriculum overall,
many residents felt they lacked adequate
List 1 training in EBM, and a survey of resident
Goals of the University of Alberta Family Medicine Resident Evidence-Based opinion supported this sentiment.14 In
Medicine (EBM) Curriculum addition, some informally expressed
concern that some preceptors may lack
Primary goals evidence-based practice skills.
1. Train competent self-directed life-long learners with the skills to effectively and efficiently keep
up-to-date.
2. Cultivate residents’ EBM skills to enable them to solve problems encountered in daily practice. New Curriculum Components
Secondary goals The EBMC committee developed a
1. Review EBM components within the present curriculum to number of suggestions that were based
● Eliminate or modify components that are evaluated negatively or perform poorly; on discussion, identified goals and
● Enhance other areas of the curriculum to promote EBM; and objectives, and literature-based evidence
● Add new or additional education time only to achieve the gains in life-long learning skills. available about EBM practices. From the
2. Promote the development of skills in residents to critique educational opportunities to suggestions, four main curricular
maximize learning and recognize low-quality biased activities. components (workshop, Family
3. Promote and encourage residents seeking additional skills, training or experience in EBM, Medicine Desktop, brief evidence-based
knowledge transfer, and/or research. assessment of the research [BEAR], and
582 Academic Medicine, Vol. 83, No. 6 / June 2008

3. Residents’ Education
journal club) were developed and event rates, absolute risk, number needed The yearly cost for the Family Medicine
implemented in 2005. We describe these to treat, and likelihood ratios. Desktop is $5,000 (Canadian dollars) for
in detail below. After the primary all residents and full-time faculty in the
components were drafted, the EBMC The yearly cost for this session is $6,000 department. The site is monitored for
committee developed a comprehensive (Canadian dollars), including facilities, frequency and duration of use by CCHE.
evaluation strategy to monitor and catering (two breakfasts and two Faculty committee members review these
research each component. Faculty lunches), materials, and gifts. At the data annually. These data will be used to
members of the EBMC committee had beginning of the workshop, before any determine whether residents’ use of
multiple meetings to secure funding for teaching, residents complete (1) a survey Family Medicine Desktop-based EBM
the various components, arrange and to determine their self-assessed skills, past resources changes throughout residency
develop the Family Medicine Desktop training, and objectives in EBM, and 2) a or during particular periods (such as
component (which required more initial preworkshop quiz of 15 questions, immediately after the workshop or while
time and resources than the other derived in part from two previous EBM completing BEARs).
components), and seek input from tests.9,20 At the conclusion of the
stakeholders (particularly those whose conference, at the end of their last small-
group session, residents complete (1) a BEAR
educational activities were changing, such
as the librarians). Once these aspects were postworkshop evaluation of the The BEAR project helps residents identify
finalized, the committee began the conference, and (2) a postworkshop quiz daily clinical questions, focus their search
process of promoting change in the (same as the preworkshop quiz). Of note, for answers, get answers rapidly, and
department. the residents were not given answers for reflect on the utility of those answers.
the preworkshop quiz, and small-group During their family medicine rotation in
Family medicine EBM workshop instructors were not aware of the content the first year, residents record questions
Because studies have demonstrated of the quiz. Analysis (in process) of the arising from their clinical practice and
improved knowledge with workshops multiyear results will allow validation of complete one BEAR assignment monthly.
and courses in EBM,8,9,15,16 the the tool for assessing EBM knowledge. The BEAR form (Appendix 1) guides
committee called for a formal EBM residents through the exercise with a
Family Medicine Desktop target time of less than one hour per
workshop to take place early in the
residency. The two-day workshop for The Family Medicine Desktop is an question. Suggested resources include
first-year residents is broken up into four Internet resource for each resident to guidelines, filtered resources (e.g., EBM
half-days, each consisting of a one-hour reduce search time, identify information and ACP Journal Club), summary sites
lecture followed by a two-hour, small- resources, and develop skills as EBM (e.g., Bandolier and UpToDate), review
group session. Three of the lectures are users. It uses a graphically displayed front sites (e.g., Cochrane), and research
devoted to appraising articles (one each page with links to relevant resources as studies (including RCT, Cohort,
of therapy, diagnosis, and meta-analysis), well as tabs to pages of various themes. Qualitative). Faculty challenge residents
and one is an overview of family The front page (browser) includes rapid to look at different questions (diagnostic,
practice research. All lectures are given links to Cochrane, PubMed, Google, ACP therapeutic, harm, practice management,
by University of Alberta faculty, but the Journal Club, Evidence-Based Medicine, etc.) and experiment with a variety of
first three topics are taught from the Clinical Evidence, Bandolier, UpToDate, resources. Each BEAR is presented in
principles of the Users’ Guides to the InfoPOEMS, Therapeutics Initiative, and approximately 15 minutes or less during
Medical Literature.17 In three small- BestBETs. There are tabs to Guides rounds and is scored simply as pass for
group sessions, 8 to 10 residents, led by (Users’ Guides to the Medical Literature), completion. To help generate discussion in
a faculty member and a second-year Profession (links to the Web pages of rounds, the presentations include the case,
resident facilitator, appraise previously differing professional bodies), Activities the clinical question, resources examined,
unseen papers on therapy, diagnosis, and (residency program-specific features like and practice reflection. Family medicine
meta-analysis. In the other small-group BEAR presentations, journal club, and rotations in first year at the University of
session, faculty librarians lead residents in previous faculty presentations), Clinical Alberta are generally 16 weeks (12 in the
the computer lab through question (guidelines and local practice tools) and rural program), meaning four (or three in
development, efficient Medline search e-library (an extensive list of Internet rural) BEARs per resident.
techniques, and identification of medical resources). The Family Medicine
summary resources (available on the Desktop is created and managed by the Integrating EBM exercises into clinical
Family Medicine Desktop, which is Canadian Centre for Health Evidence work improves residents’ adaptation
detailed below). Librarian faculty (CCHE), but it is amended and updated to evidence-based practice.4,21,22
participation has proven to be an by members of the faculty. The Family Furthermore, the use of preappraised and
important part of EBM education.18,19 Medicine Desktop can be accessed from summarized evidence is required if
any computer with Internet access. clinicians are to achieve evidence-based
The workshop also includes a brief Because all residents have access to a practice6,7,23,24 and become “evidence
lecture on why EBM is important, a desktop computer during their three- or users.”25 Lastly, the element of reflection, in
tutorial on basic statistical concepts, and four-month first-year family medicine both completing the BEAR form and
a short, “game show”-type quiz with rotation, the Family Medicine Desktop during presentation with group discussion,
prizes during the last session. The component is available to all residents at enhances the likelihood of residents
statistical session includes working some point, regardless of whether they integrating EBM into practice and changing
through control event rates, experimental have access to the Internet outside work. behavior for the long term.26
Academic Medicine, Vol. 83, No. 6 / June 2008 583

4. Residents’ Education
There is no yearly cost for the BEAR sources, the original, complete article is hour reviewing and discussing the
curricular component. The residents’ brought to AHD for review. suggested changes. The planned changes
faculty advisor reviews each BEAR. were also presented to all urban residents
Resident evaluation of the BEAR project There is no yearly cost for the journal at a program-wide AHD. The proposal
is collected as part of the annual survey of club component. Resident evaluation of was presented and discussed at the
the EBM curriculum. Full review of the the journal club occurs at each AHD and departmental geographic full-time faculty
BEAR projects is in process to examine as part of the annual survey of the EBM meeting.
the type of questions asked in clinical curriculum.
practice, resources used (summary sites, After feedback from both residents and
original research, guidelines, etc.), Additional components faculty, the EBMC committee decided to
answers obtained, and practice impact. Other components of the curriculum offer a two-day faculty development
Activity records from the Family include teaching on industry–physician workshop in evidence-based practice for
Medicine Desktop and direct observation interaction, presentation skills, and a faculty members. The workshop was a
of residents will be correlated with quarterly review of the literature. For the small-group seminar with didactic,
resource use reported in the BEARs to quarterly review, a faculty member interactive, and computer learning time.
determine the reliability of these data. presents a five-minute EBM teaching To allow for maximum attendance, it ran
The results of this work will be reported point, a summary of five to seven family- on two separate occasions before the start
in a subsequent publication. medicine-relevant articles from the last of the new curriculum. The importance
year, and an introduction to an evidence- of faculty development and role
Resident journal club (academic based practice tool (e.g., links to modeling to promote resident learning in
half-day) calculators for stroke risk in atrial EBM is well known.29,30
Each month, the small-group AHD has fibrillation). In addition, the previously
an established theme (e.g., neurology). In mentioned practice audit and quality- One month before the start of the new
preparation for the upcoming AHD, a improvement project continues curriculum, the final description was
second-year resident generates a case and unchanged. distributed to all staff and faculty. A
clinical question related to the monthly faculty member from the EBMC
topic and completes a literature search. Although organization differed from past committee visited each teaching site and
He or she then reviews the abstracts of curriculum elements, the overall time use met with faculty and staff to review any
the resulting articles and selects two to three for faculty and residents was cost neutral, last questions. Finally, two months after
papers to best address the question, without apart from two clinical days to attend the the start of the curriculum, e-mail was
critically appraising the papers. Whereas family medicine EBM workshop. distributed to each site director and site
the preparation of the case and question coordinating staff, and another follow-up
should take no more than 30 minutes, the There is no yearly cost for the additional visit was offered. Although a few brief
search and selection of two to three papers components. Residents complete concerns were reviewed over e-mail,
could take up to two hours. evaluations after each teaching session. A none of the site directors or the site
year-end summative evaluation/survey coordinating staff felt a follow-up visit
At the AHD, the designated resident obtains residents’ opinions regarding the was necessary. Initial conception to
presents the case, question, and four primary components and the implementation took nine months.
summarized search in approximately five curriculum overall. It also includes
minutes. Papers are then distributed, and attitude questions regarding evidence-
based practice in general. This Outcomes
the residents break into two to three
subgroups, based on the number of information is used to assess the The annual EBM curriculum evaluation
papers to be reviewed. Each subgroup curriculum and to modify areas found to has now been completed by two groups
reviews one of the papers as a team. Each be inadequate. of residents (2005–2006 and 2006 –2007)
subgroup has 30 to 40 minutes to browse since implementation, and the results
and appraise the article as a team to were compared with those from the year
Change Management before implementation (2004 –2005). The
develop consensus. After that, each
subgroup shares the information with A number of steps were taken to ensure evaluation was distributed to residents
the entire group. The total time is that faculty, staff, and residents had the attending an AHD in the last quarter of
approximately 60 minutes. Peer group opportunity to learn and discuss the academic year and was emailed once
learning and teaching has been described concerns as well as generate feedback for to all those reporting as absent from the
in previous EBM education models.27,28 the EBMC committee to improve the AHD. The survey lacked any personal
new curriculum. Urban and rural identifiable features. This research was
After the first year of the new curriculum, residents’ opinions were surveyed approved by the health research ethics
residents indicated that the articles regarding EBM in general and the board of the University of Alberta.
chosen by colleagues were frequently low previous curriculum specifically.14 The
in quality and not relevant. As a result, draft proposal was distributed to all The combined response rate for the three
articles are now selected from those members of the residency program years was 60% (181 returned out of a
abstracted or listed in the other articles committee approximately one month possible 304). There was significant
noted section of ACP Journal Club or in before a planned review. Faculty, staff, improvement in all areas of comfort with
the Evidence-Based Medicine journal. and resident representatives of the evidence-based practice and opinions
Whereas the article is picked from these residency program committee spent one regarding the curriculum (Table 1). Three
584 Academic Medicine, Vol. 83, No. 6 / June 2008

5. Residents’ Education
Table 1
Comparison of Resident Opinion Before and After Implementation of the
Integrated Evidence-Based Medicine (EBM) Curriculum in the University of
Alberta Family Medicine Residency in 2005
Resident mean Resident mean Two-tailed
Question* score before (n) score after (n) t test†
I feel comfortable accessing the literature. 2.95 (60) 3.97 (115) P Ͻ .0001
...................................................................................................................................................................................................................................................................................................................
I feel comfortable appraising articles (all types). 2.51 (60) 3.69 (114) P Ͻ .0001
...................................................................................................................................................................................................................................................................................................................
How comfortable are you with your ability to recognize 3.51 (61) 4.12 (115) P Ͻ .0001
and refine clinical questions?
...................................................................................................................................................................................................................................................................................................................
How comfortable are you with your basic computer skills? 4.07 (61) 4.53 (115) P ϭ .0005
...................................................................................................................................................................................................................................................................................................................
How comfortable are you with your knowledge and 3.46 (61) 4.17 (114) P Ͻ .0001
comfort with accessing the literature?
...................................................................................................................................................................................................................................................................................................................
How comfortable are you with your knowledge and 3.11 (61) 3.87 (115) P Ͻ .0001
comfort in appraising the literature?
...................................................................................................................................................................................................................................................................................................................
How comfortable are you with the time necessary to 2.80 (61) 3.37 (115) P ϭ .0003
answer clinical questions?
...................................................................................................................................................................................................................................................................................................................
Overall, I feel the EBM curriculum is well organized and 2.86 (59) 3.90 (114) P Ͻ .0001
integrated.
...................................................................................................................................................................................................................................................................................................................
Overall, I feel the EBM curriculum has improved my 2.76 (58) 3.86 (114) P Ͻ .0001
comfort with evidence-based practice.
...................................................................................................................................................................................................................................................................................................................
The practice of EBM results in better patient care. 3.88 (60) 4.17 (115) P ϭ .0110
...................................................................................................................................................................................................................................................................................................................
I have a positive attitude to the current promotion of EBM. 3.77 (60) 4.17 (115) P ϭ .0004
...................................................................................................................................................................................................................................................................................................................
Most of my colleagues have a positive attitude 3.56 (59) 3.81 (115) P ϭ .0451
to the current promotion of EBM.
...................................................................................................................................................................................................................................................................................................................
How often do you use EBM in clinical practice? 3.37 (60) 3.53 (114) P ϭ .1547
* Residents rated their opinions on a five-point Likert scale.
†
All t tests compare mean of the year before implementation to the mean of the two years since implementation.
of four questions of general opinion (e.g., ACP Journal Club or the Evidence-Based amount of time (1–10 minutes) to
The practice of EBM results in better patient Medicine journal. This change effectively answering clinical questions.6,33,34 Our
care) showed improvement as well, albeit to addressed this concern for the residents. curriculum focused on this issue. The
a lesser degree. Family Medicine Desktop introduced
Evaluations of other specific components residents to the variety of EBM resources,
In addition to the formal evaluation, (such as the workshop and the BEARs) with an emphasis on preappraised,
resident and faculty feedback was sought are too large and complex to present summarized sites (i.e., BestBETs, ACP
at meetings and during presentations. here, but they will be published in Journal Club) that will serve as the tools
Faculty have been strong supporters of subsequent articles. for their future as evidence users.6,23,25 By
the new curriculum and were particularly
collecting these in one location and
satisfied with the BEARs. Informal
Discussion accessing any with a single click, residents
comments from faculty exemplified a
can search different sites rapidly.
view that the BEARs were more This new integrated EBM curriculum is
applicable to daily practice as compared based on goals and objectives felt to be
with the previous critical appraisal The workshop and journal club do not
important to residents and faculty, and it
projects. Residents reported satisfaction include prereading papers, because it is
incorporates the best available evidence
with the workshop and the didactic not realistic to read papers entirely and
in teaching EBM. Comparison of the
literature reviews of the AHD. Some then spend another hour discussing
goals and objectives with components of
residents struggled with the simplicity of them. Additionally, journal clubs often
the curriculum reveals significant overlap.
the BEAR project and, at times, put much Surveys of other EBM curricula find that have trouble with some participants not
more effort than would be possible for components such as faculty development, reading articles.10 We focus on helping
questions in practice. They requested a accessible electronic information resources, residents gain practical skills to review
tutorial on the BEAR, and this will be and evaluation mechanism are frequently and appraise papers in a rapid,
integrated into the EBM workshop. As missing,31 and some authors have provided abbreviated fashion so that they may
mentioned earlier, although residents suggestions for future EBM curricula.21,22 efficiently obtain answers to questions.
thought the journal club had improved, Our curriculum targeted these areas and Finally, the BEAR is the key feature to
they felt that many papers chosen by their more. help residents become time-efficient
colleagues were poor in quality and evidence users because it promotes
relevance. The EBMC committee decided Time is perhaps the most significant focusing their clinical questions,
that residents would select journal club barrier to evidence-based practice,7,32 and accessing information from summarized
articles from those abstracted or noted in most clinicians will dedicate only a small and preappraised resources, and
Academic Medicine, Vol. 83, No. 6 / June 2008 585

6. Residents’ Education
reflecting on the level of influence the with improvement in skills and evidence based medicine improve knowledge
results have on daily practice. knowledge.38,39 Unfortunately, linking and skills? Validation of Berlin questionnaire
and before and after study of courses in
self-reported improvements after EBM evidence based medicine. BMJ. 2002;325:
Knowledge is often cited as a barrier to educational interventions with persistent 1338 –1341.
evidence-based practice,35,36 and, clearly, behavioral changes does not seem to have 10 Elnicki DM, Halperin AK, Shockcor WT,
there is a place for learning the basic been studied. Therefore, although it Aronoff SC. Multidisciplinary evidence-based
principles of EBM, even for evidence might be reasonable to assume that medicine journal clubs: Curriculum design
users.25 Interpreting the quality of and participants’ reactions. Am J Med Sci.
residents’ skills and knowledge have 1999;317:243–246.
preappraised, summarized evidence improved with this curriculum, actual 11 Ebbert JO, Victor MM, Schultz HJ. The
requires a basic understanding of EBM long-term behavioral changes remain journal club in postgraduate medical
terms and statistics. Furthermore, uncertain. education: A systematic review. Med Teach.
occasionally preappraised sources will 2001;23:455– 461.
not have the required information.25 The In summary, this integrated EBM 12 Ozuah PO, Orbe J, Sharif I. Ambulatory
rounds: A venue for evidence-based
workshop provides these foundations, curriculum provides a foundation of medicine. Acad Med. 2002;77:740 –741.
and the journal club builds the knowledge, supplies easy access to 13 Crowley SD, Owens TA, Schardt CM, et al. A
knowledge and skills. resources, promotes point-of-care and Web-based compendium of clinical questions
team learning, and develops applicable and medical evidence to educate internal
Access to resources is also frequently cited skills for lifelong learning while achieving medicine residents. Acad Med. 2003;78:270 –
as a barrier to evidence-based practice.7,31,36 274.
faculty goals of being evidence based, low 14 Allan GM, Manca DP, Szafran O, Korownyk
The Family Medicine Desktop provides cost, and time sensitive. C. EBM a challenge for international medical
easily accessible evidence-based resources, graduates. Fam Med. 2007;39:612.
and the BEAR promotes the use of the 15 Ghali WA, Saitz R, Eskew AH, Gupta M,
resources and helps users gain familiarity Acknowledgments Quan H, Hershman WY. Successful teaching
with these preappraised, filtered, and The authors would like to thank Olga Szafran in evidence-based medicine. Med Educ. 2000;
with her assistance in the process of securing 34:18 –22.
summarized sites. 16 Smith CA, Ganschow PS, Reilly BM, et al.
ethical approval for this project and the
evaluation work, as well as Victoria Ma for her Teaching residents evidence-based medicine
Evaluation is recommended as essential skills: A controlled trial of effectiveness and
to EBM-teaching programs,31 and the assistance in the compilation of the evaluation
assessment of durability. J Gen Intern Med.
data. The authors would also like to thank Dr.
EBMC committee felt it integral to each Paul Humphries and the residency program
2000;15:710 –715.
component and the curriculum in 17 Guyatt G, Rennie D, eds. Users’ Guides to the
committee for their support in moving this Medical Literature. Chicago, Ill: American
general. Evaluation allows faculty to project forward. Medical Association Press; 2002.
adapt the curriculum according to 18 Finkel ML, Brown HA, Gerber LM, Supino
changing needs or outcomes (such as PG. Teaching evidence-based medicine to
Family Medicine Desktop use). Although References medical students. Med Teach. 2003;25:202–
rigorous research of EBM teaching is 1 Whitcomb ME. Why we must teach 204.
evidence-based medicine. Acad Med. 2005; 19 Dorsch JL, Jacobson S, Scherrer CS. Teaching
ideal,37 we needed to introduce a new EBM teachers: A team approach. Med Ref
80:1–2.
curriculum within a limited time. Also, 2 Green ML. Graduate medical education Serv Q. 2003;22:107–114.
funding prohibited a trial with a training in clinical epidemiology, critical 20 Ross R, Verdieck A. Introducing an evidence-
comparator group, the examination of appraisal, and evidence-based medicine: A based medicine curriculum into a family
patient outcomes, and accounting for critical review of curricula. Acad Med. 1999; practice residency—Is it effective? Acad Med.
74:686 – 694. 2003;78:412– 417.
confounders. Previous authors used or 21 Del Mar C, Glasziou P, Mayer D. Teaching
3 Norman GR, Shannon SI. Effectiveness of
encouraged testing knowledge, assessing instruction in critical appraisal (evidence- evidence based medicine. BMJ. 2004;329:
attitudes/opinions, and evaluating based medicine) skills: A critical appraisal. 989 –990.
learners around specific curriculum CMAJ. 1998;158:177–181. 22 Dobbie AE, Schneider FD, Anderson AD,
4 Coomarasamy A, Khan KS. What is the Littlefield J. What evidence supports teaching
components.20,22 They provide excellent
evidence that postgraduate teaching in evidence-based medicine? Acad Med. 2000;
assessment of the curriculum while evidence based medicine changes anything? 75:1184 –1185.
offering potentially useful research A systematic review. BMJ. 2004;329:1017– 23 Smith R. What clinical information do
information. Additionally, we are 1019. doctors need? BMJ. 1996;313:1062–1068.
monitoring Family Medicine Desktop use 5 Korenstein D, Dunn A, McGinn T. Mixing 24 Shaughnessy AF, Slawson DC. Are we
it up: Integrating evidence-based medicine providing doctors with the training and
and directly observing residents in clinics tools for life-long learning? BMJ. 1999;319:
and patient care. Acad Med. 2002;77:741–
(before and after the workshop) to 742. 1280.
determine the types of questions asked in 6 Ely JW, Osheroff JA, Ebell MH, et al. Analysis 25 Guyatt GH, O Meade M, Jaeschke RZ, Cook
clinical practice and the utility of of questions asked by family doctors DJ, Haynes RB. Practitioners of evidence
evidence-based resources in answering regarding patient care. BMJ. 1999;319:358 – based care. BMJ. 2000;320:954 –955.
361. 26 Robertson K. Reflection in professional
their questions. We will compare these 7 McColl A, Smith H, White P, Field J. General practice and education. Aust Fam Phys. 2005;
results with the questions and resources practitioners’ perceptions of the route to 34:781–783.
used in BEAR assignments. evidence based medicine: A questionnaire 27 Hunt DP, Haidet P, Coverdale JH, Richards
survey. BMJ. 1998;316:361–365. B. The effect of using team learning in an
In this article, we report a statistically 8 Schoenfeld P, Cruess D, Peterson W. Effect of evidence-based medicine course for
significant improvement in resident an evidence-based medicine seminar on medical students. Teach Learn Med. 2003;
participants’ interpretations of clinical trials: 15:131–139.
attitude and self-reported abilities. Self- A pilot study. Acad Med. 2000;75:1212–1214. 28 Edwards K, Wolf PK, Hetlzer T. Pediatric
reported improvements after EBM 9 Fritsche L, Greenhalgh T, Falck-Ytter Y, residents as learners and teachers of evidence-
educational intervention can correlate Neumayer HH, Kunz R. Do short courses in based medicine. Acad Med. 2002;77:748.
586 Academic Medicine, Vol. 83, No. 6 / June 2008

7. Residents’ Education
29 Bhandari M, Montori V, Devereaux PJ, medicine: A representative national survey. medicine in Saudi Arabia. J Cont Educ
Dosanjh S, Sprague S, Guyatt GH. Challenges Intern Med J. 2005;35:9 –17. Health. 2004;24:163–170.
to the practice of evidence-based medicine 33 Ramos K, Linscheid R, Schafer S. Real-time 37 Hatala R, Guyatt G. Evaluating the teaching
during residents’ surgical training: A information-seeking behavior of residency of evidence-based medicine. JAMA. 2002;288:
qualitative study using grounded theory. physicians. Fam Med. 2003;35:257–260. 1110 –1112.
Acad Med. 2003;78:1183–1190. 34 Schilling LM, Steimer JF, Lundahl K, 38 Green ML, Ellis PJ. Impact of an evidence-
30 Lam WW, Fielding R, Johnston JM, Tin KY, Anderson RJ. Residents’ patient-specific based medicine curriculum based on adult
Leung GM. Identifying barriers to the clinical questions: Opportunities for learning theory. J Gen Intern Med. 1997;12:
adoption of evidence-based medicine practice evidence-based learning. Acad Med. 2005;80: 742–750.
in clinical clerks: A longitudinal focus group 51–56. 39 Stark R, Helenius IM, Schimming LM,
study. Med Educ. 2004;38:987–997. 35 McAlister FA, Graham I, Karr GW, Laupacis Takahara N, Kronish I, Korenstein D. Real-
31 Green ML. Evidence-based medicine training A. Evidence-based medicine and the time EBM: From bed board to keyboard and
in internal medicine residency programs. practicing clinician. J Gen Intern Med. 1999; back. J Gen Intern Med. 2007;22:1656 –1660.
J Gen Intern Med. 2000;15:129 –133. 14:236 –242. Available at: (http://www.springerlink.com/
32 Toulkidis V, Donnelly NJ, Ward JE. Engaging 36 Al-Almaie SM, Al-Baghli N. Barriers facing content/27p764108v1404q7/fulltext.html).
Australian physicians in evidence-based physicians practicing evidence-based Accessed February 27, 2008.
Appendix 1
Brief Evidence-Based Assessment of the Research (BEAR) Form from the
University of Alberta Family Medicine Resident Evidence-Based Medicine
Curriculum
BEAR Work-Sheet
Name of Resident: Date:
Question: _______________________________________________________________
_______________________________________________________________________
Search: (Check all that apply)
Pubmed/Ovid/Medline:  Filtered Resources:  Summary/Review Sites: 
College/Society/Guidelines:  Other:  ( Describe: ___________________ )
Number of Resources Reviewed: ____
Resources (Top 3)
#1 Resource: Abstract  Paper  Filtered Article  Summary 
Review/Meta-Analysis  College/Society/Guideline Paper  Other Research 
- Abbreviated Citation: _________________________________________
- Strengths:___________________________________________________
- Weaknesses:_________________________________________________
Take-Home Message:______________________________________________________
#2 Resource: Abstract  Paper  Filtered Article  Summary 
Review/Meta-Analysis  College/Society/Guideline Paper  Other Research 
- Abbreviated Citation: _________________________________________
- Strengths:___________________________________________________
- Weaknesses:_________________________________________________
Take-Home Message: _____________________________________________________
#3 Resource: Abstract  Paper  Filtered Article  Summary 
Review/Meta-Analysis  College/Society/Guideline Paper  Other Research 
- Abbreviated Citation: _________________________________________
- Strengths:___________________________________________________
- Weaknesses:_________________________________________________
Take-Home Message: _____________________________________________________
Bottom-line: ____________________________________________________________
________________________________________________________________________
________________________________________________________________________
Practice (These findings had a):
Large Change  Small Change  Reassured  No Help 
Academic Medicine, Vol. 83, No. 6 / June 2008 587