SlideShare a Scribd company logo
1 of 7
Download to read offline
This article was downloaded by: [UPSTATE Medical University Health Sciences Library]
On: 10 February 2015, At: 04:36
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK
Teaching and Learning in Medicine: An International
Journal
Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/htlm20
Long-Term Gain After Team-Based Learning Experience
in a Pediatric Clerkship
Kavita S. Warrier
a
, Jocelyn H. Schiller
a
, Nicole R. Frei
a
, Hilary M. Haftel
a
& Jennifer G.
Christner
a
a
Department of Pediatrics , University of Michigan Medical School , Ann Arbor , Michigan ,
USA
Published online: 10 Oct 2013.
To cite this article: Kavita S. Warrier , Jocelyn H. Schiller , Nicole R. Frei , Hilary M. Haftel & Jennifer G. Christner (2013)
Long-Term Gain After Team-Based Learning Experience in a Pediatric Clerkship, Teaching and Learning in Medicine: An
International Journal, 25:4, 300-305, DOI: 10.1080/10401334.2013.827975
To link to this article: http://dx.doi.org/10.1080/10401334.2013.827975
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
Teaching and Learning in Medicine, 25(4), 300–305
Copyright C 2013, Taylor & Francis Group, LLC
ISSN: 1040-1334 print / 1532-8015 online
DOI: 10.1080/10401334.2013.827975
Long-Term Gain After Team-Based Learning Experience
in a Pediatric Clerkship
Kavita S. Warrier, Jocelyn H. Schiller, Nicole R. Frei, Hilary M. Haftel,
and Jennifer G. Christner
Department of Pediatrics, University of Michigan Medical School, Ann Arbor, Michigan, USA
Background: Team-based learning (TBL) increases student en-
gagement, value of teamwork, and performance on standardized
evaluations. Purpose: The authors implemented a 3rd-year pedi-
atric TBL curriculum, evaluating its effect on satisfaction, engage-
ment, value of teamwork, and short-term and long-term academic
performance. Method: Students evaluated the TBL curriculum
and core lectures through satisfaction, engagement and value of
team surveys. Scores on short-term and long-term examinations
were compared to historical data. Results: The first implementa-
tion year, students were less likely to enjoy TBL sessions com-
pared to lectures. The 2nd year, this difference lessened. Through
both years, students reported dramatic increases in classroom en-
gagement during TBL compared to lecture. Students developed a
greater value for teams after participating in TBL. Short-term and
long-term examination scores improved significantly. Conclusions:
Both short-term and long-term performance improved with imple-
mentation of TBL, emphasizing the benefits of a curriculum that
allows students to critically engage with material.
Keywords team based learning, medical student, pediatrics, active
learning
A teacher-centric perspective has traditionally been the most
common orientation to teaching in medical education. Little
evidence supports dissemination-only strategies if the goal is
to improve professional practice.1
A growing body of medical
education research has demonstrated that active learning is more
effective than passive learning.2,3
A review of educational theory suggests that learners actively
contribute to the educational process, practice skills accompa-
nied by constructive feedback from teachers and peers, and re-
late learning to understanding and solving real-life problems.4
Indeed, the Liaison Committee for Medical Education has incor-
We thank Drs. Mary Hernandez and Laura Hopson for their assis-
tance in data collection. We also acknowledge Drs. Kenneth Pituch,
Terrance Murphy, Patricia Keefer, Kerry Mychaliska, and Francis
McBee-Orzulak for their help developing and implementing the TBL
curriculum.
Correspondence may be sent to Jocelyn H. Schiller, 12-525 Mott,
1540 East Hospital Drive, SPC 4280, Ann Arbor, MI 48109-4280,
USA. E-mail: johuang@umich.edu
porated active learning into its expected standards.5
Although
education reform emphasizes altering the pedagogical approach
to curriculum, such as switching from lecture format to case
studies,6
generational issues also need consideration. A recent
meta-analysis on generational changes indicates that students
born after 1980 prefer learning by doing rather than by listening
to lectures. This generation of students is also more likely to be
overconfident and thus may require more feedback.7
Team-based learning (TBL) is an innovative instructional
method that fosters active learning and has been adopted by
some medical schools, particularly in the preclinical years.8–13
TBL utilizes active small-group discussion facilitated by an in-
structor. Teams solve clinical problems and receive immediate
feedback on performance. Student performance is graded, with
an assessment of individual knowledge, evaluation of group
performance, and peer feedback making up the incentive struc-
ture of the method. TBL has been shown to improve learning
outcomes and examination scores,11,14–18
communication pro-
cesses, teamwork skills, and the value students place on these
skills,10.11,13,15
and student engagement in class settings.11,19–21
This correlates both with student satisfaction and student
achievement, especially when the subject matter is difficult.
TBL in the clinical setting has been previously described
for psychiatry and internal medicine clerkships11,14,22,23
but uti-
lizing TBL to deliver core curriculum in a pediatric clerkship
has not been reported. Long-term clinical knowledge gains have
also not been reported. In our clerkships’ transition from didac-
tic lectures to TBL, we had four objectives: (a) compare student
evaluations of TBL teaching sessions to core didactic lectures,
(b) analyze the students’ perspectives of the value of teams
before and after experiencing TBL, (c) analyze classroom en-
gagement during TBL, and (d) compare academic performance
between students prior to and after implementation of TBL ses-
sions, including long-term gains.
METHODS
Educational Intervention
At the University of Michigan Medical School, all 3rd-year
medical (M3) students rotate through an 8-week pediatric clerk-
ship with approximately 25 to 32 students in each cohort. In the
300
Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015
LONG-TERM GAINS AFTER TBL IN PEDIATRIC CLERKSHIP 301
past, 21 hours were set aside for core pediatric teaching in the
form of didactic lectures. Beginning with the 2009–10 academic
year (2009 AY), six 90-minute TBL sessions replaced 12 of the
core teaching hours.
The 10 pediatric faculty members involved in the develop-
ment, implementation, and facilitation of TBL modules attended
a TBL workshop in 2008. Three core faculty members also at-
tended the TBL Collaborative Annual Conference. Module top-
ics were chosen by reviewing institutional objectives and the
national Council on Medical Student Education in Pediatrics
Curriculum to identify core pediatric topics.24
The modules
were piloted and reviewed prior to implementation and have
since been published on MedEd Portal.25–30
Because the students had no prior TBL experience, they were
given information about TBL during the clerkship orientation.
Reading assignments were provided for each module. The in-
structors randomly divided students into teams of five to six at
the first TBL session. The students remained in the same teams
for all six TBL sessions. Each block of six TBL sessions was
facilitated by a team of two faculty members to allow students
to develop rapport with the faculty and to provide a variety of
faculty perspectives.
Each session consisted of readiness assurance tests (RATs)
taken individually (IRATs) and as a group (GRATs), followed by
application exercises (see Table 1). Students were not allowed
to use reading materials during the RATs. Immediate feedback
was provided during the GRAT with scratch-off answer sheets
denoting the correct answer with a star. Faculty facilitators re-
viewed the answers to the RATs and clarified key points before
students started the application exercises. For 24 hours after
the session, any team was allowed to appeal RAT questions by
showing a credible source that supported an alternative answer.
The application exercises consisted of more complex clinical
scenarios, which required synthesis of information, often focus-
ing on diagnosis or treatment. Most questions had one “most
right” answer with other “right” answer choices. Each team dis-
cussed the questions and came to a consensus on the answers.
After the intra-team discussion was complete, the teams came
together for inter-team dialogue. Each team simultaneously dis-
played their answer; the facilitators then required teams to ex-
plain their reasoning and defend their answer choice.
TABLE 1
Time spent on Readiness Assurance Tests (RAT) and
application questions
Exercise Time Spent No. and Type of Questions
Individual RAT 15 min 9 to 12 multiple choice
questions based on readings
Group RAT 30 min Same as IRAT
Application 45 min 2 to 5 complex clinical
scenarios
At the final session, students gave written peer feedback
adapted from the Koles method,31
which was reviewed by the
associate clerkship director and assigned a grade. Twenty-five
percent of the TBL grade was based on the IRAT and 65% of
the TBL grade was based on the GRAT; the other 10% of the
TBL grade came from peer feedback. These distributions were
assigned by the course director. The TBL grade was worth 5%
of the total clerkship grade, which was in line with amounts
assigned to other clerkship assessments such as written notes,
assignments, and professionalism.
Objectives and Data Collection
During the 2009 AY, we gathered student evaluations of the
core didactic lectures and TBL sessions. These evaluations were
not anonymous. We also collected data on student engagement
during the didactic and TBL sessions using the Classroom En-
gagement Survey, a tool from Baylor College of Medicine that
asked questions relating to engagement in the lecture or TBL
session.11,32
Students’ perception of the value of teamwork was
measured at the beginning and end of the clerkship using a nine-
item “Value of Teams” survey developed at Baylor College of
Medicine.11,33
All items were scored on a 5-point Likert scale
anchored by 1 (strongly disagree) and 5 (strongly agree).
Academic performance was assessed with three measures.
National Board of Medical Examiners (NBME) Pediatric Shelf
scores for the 2009 AY and 2010 AY were compared to scores
from the 4 years prior. We also analyzed scores on our M4 Com-
prehensive Clinical Assessment (CCA), a high-stakes Objective
Structured Clinical Examination all 4th-year students must pass.
The CCA included several stations, one of which entailed ob-
taining a pediatric history from a standardized patient parent.
Students are randomly assigned to either the pediatric fever or
asthma standardized patient parent. Scores from this pediatric
station were compared between the students who completed the
clerkship in AY 2009 to students from the 4 years prior. Our
last performance measure was an examination taken during the
required M4 Emergency Medicine (EM) clerkship. AY 2009
student scores on two clinical questions about fever in neonates
and toddlers were compared with those from the 2 years prior.
The CCA stations and EM examination questions were chosen
because both fever and asthma were previously taught during
the M3 year by didactic lecture, but in 2009 AY became TBL
topics. These M4 CCA assessments were done between 1 and
10 months after the M3 pediatric clerkship and the EM exam
was taken between 1 and 22 months after completion of the M3
pediatric clerkship. Other than the implementation of TBL, no
other changes were made to the curriculum during these study
years.
This study was reviewed by the University of Michigan In-
stitutional Review Board and granted exempt status.
Statistical Analysis
Mean response to each survey item for overall evaluation,
student classroom engagement, and value of teams were
Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015
302 K. S. WARRIER ET AL.
calculated and compared by independent samples t tests. The
Value of Teams Survey analyzes two different learner perceived
dimensions of learning in groups, the Value of Working with
Peers and the Value of Group Work,34
which were calculated
and compared by independent samples t test. Independent
samples t tests were used to compare means for NBME scores,
CCA performance, and EM exam scores. Cohen’s d was
calculated to show effect size. Data were analyzed using SAS
9.2 Statistical Software.
RESULTS
One hundred seventy-nine students completed the pediatric
clerkship in 2009 AY and 160 students completed the clerkship
in 2010 AY.
Satisfaction
Students reported lower satisfaction with team-based learn-
ing as compared to lectures for almost all features in both years
(see Table 2). However, the mean differences in satisfaction be-
tween lecture and TBL dropped in the 2nd year of study. The
differences between these means were statistically significant
for six of the seven questions in the 1st year, and five of the
seven questions in the 2nd year. The only change between years
was the question of whether the topic was of importance.
Value of Teams
For all items on the Value of Team survey, students demon-
strated a greater appreciation for teamwork after the completion
of the term, but this was only statistically significant for three of
the items: “Working in teams in class is productive and efficient,”
“Group decisions are often better than individual decisions,” and
“Solving problems in groups leads to better decisions than solv-
ing problems alone” (see Table 3). Although the other six items
were not statistically significant, all items trended in a positive
direction after students experienced TBL. The Value of Teams
Survey analyzes two different learner perceived dimensions of
learning in groups; there was no significant difference in the
overall Value of Working with Peers. There was, however, a sig-
nificant difference in student assessment of the Value of Group
Work (0.24; p <.01, d = .38) after completion of the team-based
learning curriculum.
Classroom Engagement
Students reported higher levels of personal contribution and
participation, as well as fellow student involvement, in TBL
TABLE 2
Student satisfaction and classroom engagement: Comparing lecture to TBL sessions
2010 2011
Lecture Lecture
Question Ma
TBL Mb
Diff p d Mc
TBL Md
Diff p d
Standard Evaluation
Achieved Objectives 4.60 4.24 0.36 <.01 0.41 4.63 4.45 0.18 <.01 0.24
Session Effective 4.55 4.28 0.26 <.01 0.32 4.57 4.46 0.11 <.01 0.13
Topic Important 4.65 4.57 0.08 <.01 0.12 4.67 4.65 0.02 .43 0.04
Level of Discussion Appropriate 4.46 4.43 0.03 .48 0.03 4.50 4.56 0.06 .16 0.06
Organized 4.64 4.31 0.33 <.01 0.42 4.67 4.46 0.21 <.01 0.30
Session Appropriate 4.61 4.44 0.17 <.01 0.23 4.66 4.57 0.09 <.01 0.12
Engagement
I Contributed 3.04 4.34 1.30 <.01 0.94 3.36 4.37 1.01 <.01 0.75
I Did Not Have Fun 1.62 1.95
∗
n = 1,030
0.33 <.01 0.32 1.53 1.70 0.17 <.01 0.19
I Had Fun 4.03 3.90 0.13 <.01 0.12 4.10 4.17 0.07 .14 0.07
I Paid Attention 4.45
∗
n = 878
4.43 0.02 .63 0.02 4.45 4.42 0.03 .40 0.04
I Would Like More Like This 4.12 3.94 0.18 <.01 0.18 4.20 4.12 0.08 .10 0.08
Most Students Involved 3.85 4.33 0.48 <.01 0.45 4.00 4.39 0.39 <.01 0.22
Most Students Did Not Pay Attention 1.73 1.84 0.11 .03 0.10 1.69 1.70 0.01 .84 0.01
I Participated 3.52 4.45 0.93 <.01 0.62 3.77 4.47 0.70 <.01 0.58
Average 3.96
∗
n = 878
4.20
∗
n = 1,030
0.24 <.01 0.34 4.08 4.32 0.24 <.01 0.32
Note. TBL = team-based learning.
a
N = 879. b
N = 1,033. c
N = 943. d
N = 916.
Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015
LONG-TERM GAINS AFTER TBL IN PEDIATRIC CLERKSHIP 303
TABLE 3
Value of teams survey
Pre-TBL Post-TBL
Item Question Meana
Meanb
p
1 The ability to collaborate with my peers will be necessary if
I am to be successful as a student.
4.61 4.65 .61
2 It is a waste of time to work in groups. 1.80 1.69 .20
3c
The ability to work with my peers is a valuable skill. 4.70 4.72 .71
4 Collaborating with my peers will help me be a better student. 4.40 4.52 .07
5 Solving problems in a group is an effective way to practice
what I have learned.
4.19 4.25 .50
6 Solving problems in a group is an effective way to learn. 4.11 4.21 .25
7c
Working in teams in class is productive and efficient. 3.54 3.78 .02
8 Group decisions are often better than individual decisions. 3.91 4.26 <.01
9c
Solving problems in groups leads to better decisions than
solving problems alone.
3.99 4.31 <.01
Note. TBL = team-based learning.
a
n = 171. b
n = 170. c
Post-TBL n = 169.
sessions as compared to lecture, noted by the higher ratings for
TBL in the items “I contributed to class,” “Most students were
involved in class,” and “I participated in class” (see Table 2).
However, similar to the data from the evaluations, students re-
ported higher agreement with statements such as “I did not have
fun” for TBL sessions and did not want more sessions like TBL.
Again, similar to the data from the evaluations, we noted im-
provements in the 2nd year with students enjoying TBL more
the 2nd year. Overall, the cumulative Classroom Engagement
score showed a significant increase between TBL sessions as
compared to lecture in both years.
The effect size for these mean differences was notable partic-
ularly for student agreement with statements about their personal
contribution and participation across both years.
Academic Performance
NBME SHELF Exam. For 4 years prior to the initiation of
TBL, the mean on the Pediatric Shelf was 79.46 (n = 614; see
Table 4). The mean on the Pediatric Shelf for the 2 years after
the initiation of TBL was 82.50 (n = 311). This is an increase
of 3.04 points ( p <.0001, d = .36). Shelf data were unavailable
for 31 students in 2009 AY.
M4 CCA. For the 4 years prior to initiation of TBL, the
student mean on the CCA Pediatric Station was 73.02 (n = 668;
see Table 4). The student mean on the CCA for the 1st year after
TBL was introduced was 75.55 (n = 149). The increase of 2.53
points was statistically significant (p = .0109, d = .24). Data for
22 students were unavailable, as several students took a leave of
absence between M3 and M4 year to pursue academic interests.
TABLE 4
Measures of academic performance
Test M Score SD Difference p Cohen’s d
NBME Shelf Examination
Pre-TBLa
79.46 8.14 3.04 <.01 0.36
Post-TBLb
82.50 8.72
Clinical Skills Examination
Pre-TBLc
73.05 12.42 2.89 <.01 0.24
Post-TBLd
75.94 11.00
EM Score (Toddler)
Pre-TBLe
2.21 1.04 0.28 <.01 0.28
Post-TBLf
2.49 0.84
EM Score (Neonate)
Pre-TBLe
6.00 1.51 0.45 <.01 0.32
Post-TBLf
6.45 1.17
Note. NBME = National Board of Medical Examiners; TBL = team-based learning; EM = Emergency Medicine.
a
n = 614. b
n = 311. c
n = 668. d
n = 149. e
n = 281. f
n = 151.
Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015
304 K. S. WARRIER ET AL.
EM Clinical Skills Exam. The student mean score on the
question addressing fever in toddlers increased by 0.28 (p =
.0032) after the introduction of TBL sessions, a 9% increase in
the score out of a maximum score of 3 points. The student mean
score on the question addressing fever in neonates increased
by 0.45 (p = .0006, d = .32), an increase of 6.4% out of the
maximum score of 7 points.
DISCUSSION
To the best of our knowledge, this is the first study demon-
strating that 3rd-year students participating in TBL had sus-
tained improved in examination performance. Active discussion
in the context of clinical scenarios may lead to longer retention
of knowledge than that learned through rote memorization. Our
findings support our hypothesis that TBL has a positive im-
pact on students’ learning as demonstrated by improved test
scores the following academic year. These results are likely
multifactorial. Students must read preparatory materials prior
to the TBL session, which may increase the amount of reading
during the clerkship on subject matter that is directly linked to
clerkship and institutional learning objectives, improving their
preparation for examinations. The peer interactions during TBL
sessions require students to choose best answers within teams
and educate each other, which likely enhances personal under-
standing of the material.17
The increased accountability to the
group due to peer evaluation may have encouraged students to
engage in more independent study.17
Across both years of study, students reported an increase in
classroom engagement during TBL sessions as compared to lec-
tures. This may also help explain the improvement in academic
performance. As students critically examine clinical questions,
they become invested in learning why certain choices are most
appropriate, which likely invests them in learning.17
Because
faculty can assess student knowledge from the IRAT and group
discussion, faculty can then direct the ensuing discussion toward
clarifying difficult concepts that were not well understood.
Despite the positive effect on learning, students did not rate
TBL as highly as didactic lectures. This confirms the results of
another study which showed that although learners were more
engaged during TBL, the perceived value of didactic sessions
was higher.21
In that study, however, student performance was
the same between TBL and didactic lectures, whereas our stu-
dents had improved performance after TBL. We chose to keep
only the most highly rated didactic lectures in our curriculum,
which may have influenced our findings. In addition, our stu-
dents have minimal exposure to TBL; introduction of a novel
instructional method that requires more work on the part of
the student may have affected student satisfaction. Orienting
the students to TBL may be important in fostering acceptance.
Overall, we are encouraged that the evaluations of TBL im-
proved in the 2nd year. Another study has also demonstrated
improvement in student perceptions of the merit of TBL teach-
ing over a 3-year time span.35
Although faculty data were not
formally collected, our faculty report increased comfort with
the material and teaching style over time, which is supported
by a qualitative study by Thompson et al. reporting that faculty
needed time to become comfortable with the TBL method and
faculty expertise was vital to successful TBL implementation.36
Faculty indicated that initial use of TBL was of limited success
but that repeated use of TBL was essential to success.13
Our
experience reminds us that creating the conditions for effective
change require time, experience, and a shifting of attitudes for
both students and faculty.
The students involved in this study were primarily students
from “Generation Me” who often require support when try-
ing something new. It has been noted that this generation likes
to know exactly what to do to get good grades and finds am-
biguous instruction stressful.7
Our students may need additional
preparation in order to accept TBL.23
We have created a more
extensive orientation to TBL illustrating not only the process
but the perceived benefits of this type of instruction, hoping to
give students clearer expectations and improve their view of the
process. The fact that medical decision making can yield mul-
tiple correct yet different paths is also an intrinsic part of TBL,
and one that may be difficult for junior students to accept. Even
experienced clinicians may be uncomfortable with diagnostic
uncertainty, despite it being part of clinical medicine. Although
it may be uncomfortable for students—and may lead to low
evaluations—it does not change the fact that this is important
exposure for students and may be something they value in the
long run—if not now.
Our results support the findings of previous studies indicating
that TBL can teach students to value teamwork. Students were
more likely to appreciate using teamwork in class and, more
important, see the value that comes from shared decision mak-
ing. This promotes a collaborative decision-making process, an
important value in this age of patient- and family-centered care.
More research will be necessary to determine whether expo-
sure to team learning impacts future views of teamwork in the
clinical setting or ultimately impacts clinical care.
This study has some limitations. Because TBL is a graded
component of the 3rd-year curriculum, we were unable to cre-
ate a control group for comparison; the entire class participated.
This necessitated the use of historical controls for the analy-
sis of academic performance. The yearly differences between
classes in the medical school could explain the variation in Shelf,
CCA, and ED clinical examination scores. Our mean student
Shelf scores prior to the intervention, however, were constant.
Demographic data that influence standardized test-taking ability
were not included in the statistical analysis and could influence
the scores. We have only 1 year of ED clinical exam data, as the
examination was changed after that year, so we are unable to see
if this improvement was sustained in the next academic year. All
students were required to turn in the surveys, and so response or
nonresponse bias was not a factor; however, the student surveys
were not anonymous, which could have affected the students’
report on all measures.
Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015
LONG-TERM GAINS AFTER TBL IN PEDIATRIC CLERKSHIP 305
CONCLUSIONS
This study adds to the growing body of evidence suggest-
ing the effectiveness of TBL in achievement of knowledge ob-
jectives for medical students and is the first to demonstrate
knowledge gains extending into the next academic year. TBL
addressed the needs of this generation of learners to be more
actively involved in their education while allowing the fac-
ulty member to provide “real-time” feedback as students work
through clinical cases. Faculty need time to adapt to a differ-
ent pedagogical method, but once practiced, the gains in ac-
tive learning, teamwork, and increase in long-term performance
seem well worth the investment.
REFERENCES
1. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets:
A systematic review of 102 trials of interventions to improve profes-
sional practice. Canadian Medical Association Journal 1995;153:1423–
31.
2. Richardson D, Birge B. Teaching physiology by combining passive (peda-
gogical) and Active (andragogical) methods. Advances in Physiology Edu-
cation 1995;13:S66–74.
3. Graffam B. Active learning in medical education: Strategies for beginning
implementation. Medical Teacher 2007;29:38–42.
4. Kaufman DM. Applying educational theory in practice. British Medical
Journal. 2003;326:213–6.
5. Liaison Committee on Medical Education. Functions and Struc-
ture of a Medical School. 2008. Available at: http://www.lcme.org/
functions2008jun.pdf. Accessed December 7, 2009.
6. Armstrong EG, Mackey M, Spear SJ. Medical education as a process man-
agement problem. Academic Medicine 2004;79:721–8.
7. Twenge JM. Generational changes and their impact in the classroom: Teach-
ing Generation Me. Medical Education 2009;43:398–405.
8. Seidel CL, Richards BF. Application of team learning in a medical physi-
ology course. Academic Medicine 2001;76:533–4.
9. Haidet P, O’Malley KJ, Richards B. An initial experience with ‘team learn-
ing’ in medical education. Academic Medicine 2003;77:40–4.
10. Hunt DP, Haidet P, Coverdale JH, Richards BF. The effect of using team
learning in an evidence-based medicine course for medical students. Teach-
ing & Learning in Medicine 2003;15:131–9.
11. Levine RE, O’Boyle M, Haidet P, Lynn DJ, Stone MM, Wolf DV, et al.
Transforming a clinical clerkship with team learning. Teaching & Learning
in Medicine 2004;16:270–5.
12. Dunaway GA. Adaption of team learning to an introductory graduate phar-
macology course. Teaching & Learning in Medicine 2005;17:56–62.
13. Thompson BM, Schneider VF, McMahon K, Levine R, Perkowski L,
Richards BF. Team learning at ten medical schools: Two years later. Medi-
cal Education 2007;41:250–7.
14. Tan NCK, Kandiah N, Chan YH, Umapathi T, Lee SH, Tan K. A con-
trolled study of team-based learning for undergraduate clinical neurology
education. BMC Medical Education 2011;11:91.
15. Zgheib NK, Simaan JA, Sabra R. Using team-based learning to teach phar-
macology to second year medical students improves student performance.
Medical Teacher 2010;32:130–5.
16. Touchet BK, Coon KA. A pilot use of team-based learning in psychiatry
resident psychodynamic psychotherapy education. Academic Psychiatry
2005;29:293–6.
17. Koles PG, Stolfi A, Borges NJ, Nelson S, Parmelee DX. The impact of
team-based learning on medical students’ academic performance. Academic
Medicine 2010;85:1739–42.
18. Searle NS, Haidet P, Kelly A, Schneider VF, Seidel CL, Richards BF.
Team learning in medical education: Initial experiences at ten institutions.
Academic Medicine 2003;78:S55–8.
19. Vasan NS, DeFouw D. Team learning in a medical gross anatomy course.
Medical Education 2005;39:524.
20. Kelly PA, Haidet P, Schneider V, Searle N, Seidel CL, Richards BF. A
comparison on in-class learner engagement across lecture, problem-based
learning and team learning using the STROBE classroom observation tool.
Teaching & Learning in Medicine 2005;17:112–8.
21. Haidet P, Morgan RO, O’Malley K, Moran BJ, Richards BF. A controlled
trial of active versus passive learning strategies in a large group setting.
Advances in Health Sciences Education 2004;9:15–27.
22. Ravindranath D, Gay TL, Riba MB. Trainees as teachers in team-based
learning. Academic Psychiatry 2010;34:294–7.
23. Thomas PA, Bowen CW. A controlled trial of team-based learning in an
ambulatory medicine clerkship for medical students. Teaching & Learning
in Medicine 2011;23:31–6.
24. Council on Medical Student Education in Pediatrics. Curriculum
competencies and objectives. 2005. Available at: http://comsep.org/
educationalresources/currobjectives.cfm. Accessed December 15, 2009.
25. Warrier K, Schmidt J, Schiller J, Murphy T, Frei N. Pediatric Fever Team-
Based Learning Module. MedEdPORTAL; 2010. Available from: http://
services.aamc.org/30/mededportal/servlet/s/segment/mededportal/?subid=
8356
26. Schiller J, Frei N, Keefer P, Warrier K, McBee Orzulak F. Pediatric Anemia
Team-Based Learning Module. MedEdPORTAL; 2010. Available from:
http://services.aamc.org/30/mededportal/servlet/s/segment/mededportal/?
subid=8332
27. Warrier K, Mychaliska K, Schiller J, Mcbee-Orzulak F, Frei N.
Pediatric Nutrition Team-Based Learning Module. MedEdPORTAL;
2010. Available from: http://services.aamc.org/30/mededportal/servlet/s/
segment/mededportal/?subid=8358
28. Keefer P, Frei N, Schiller J, Murphy T. Pediatric Abdominal Pain and Mass
Team-Based Learning Module. MedEdPORTAL; 2010. Available from:
http://services.aamc.org/30/mededportal/servlet/s/segment/mededportal/?
subid=8371
29. McBee Orzulak F, Frei N, Murphy T, Keefer P, Schiller J. Pediatric Asthma
Team-Based Learning Module. MedEdPORTAL; 2011. Available from:
http://services.aamc.org/30/mededportal/servlet/s/segment/mededportal/?
subid=8352
30. Frei N, Pituch K, Keefer P, Schiller J. Pediatric Pneumonia Team-
Based Learning Module. MedEdPORTAL; 2011. Available from: http://
services.aamc.org/mededportal/servlet/s/segment/mededportal/?subid =
8353
31. Levine RE. Peer evaluation in team-based learning. In LK Michaelson, DX
Parmelee, KK McMahon, & RE Levine (Eds.), Team-based learning for
health professions education (pp. 107–14). Sterling, UK: Stylus, 2008.
32. Classroom engagement survey. In Team learning in medical education
workshop handbook. Houston, TX: Baylor College of Medicine, Office of
Curriculum, 2001.
33. Value of Teams Survey. In Team learning in medical education workshop
handbook. Houston, TX: Baylor College of Medicine, Office of Curriculum,
2001.
34. Guide to using the Value of Teams Survey. In Team learning in medical
education workshop handbook. Houston, TX: Baylor College of Medicine,
Office of Curriculum, 2001.
35. Davidson LK. A 3-year experience implementing blended TBL: Active
instructional methods can shift student attidues to learning. Medical Teacher
2011;33:750–3.
36. Thompson BM, Schneider VF, Haidet P, Perkowski LC, Richards BF.
Factors influencing implementation of team-based learning in health science
education. Academic Medicine 2007;82:S53–6.
Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015

More Related Content

What's hot

To Err Is Human
To Err Is HumanTo Err Is Human
To Err Is Humanjlocklear
 
Faculty Development Strategy is An Essential Element for Quality Teaching
Faculty Development Strategy is An Essential Element for Quality TeachingFaculty Development Strategy is An Essential Element for Quality Teaching
Faculty Development Strategy is An Essential Element for Quality TeachingMatiaAhmed
 
Studentsevaluationresults
StudentsevaluationresultsStudentsevaluationresults
StudentsevaluationresultsYisehak Tura
 
A Study to Evaluate the Effectiveness of Motivational Intervention on Anxiety...
A Study to Evaluate the Effectiveness of Motivational Intervention on Anxiety...A Study to Evaluate the Effectiveness of Motivational Intervention on Anxiety...
A Study to Evaluate the Effectiveness of Motivational Intervention on Anxiety...iosrjce
 
20171023 The Train-the-Trainer Model 1-1 Taiwan-USA Experience -- Educating ...
20171023 The Train-the-Trainer Model 1-1 Taiwan-USA Experience -- Educating ...20171023 The Train-the-Trainer Model 1-1 Taiwan-USA Experience -- Educating ...
20171023 The Train-the-Trainer Model 1-1 Taiwan-USA Experience -- Educating ...R.O.C.Ministry of Health and Welfare
 
Assessment of the Existing School Health Promotion Program in a Selected Educ...
Assessment of the Existing School Health Promotion Program in a Selected Educ...Assessment of the Existing School Health Promotion Program in a Selected Educ...
Assessment of the Existing School Health Promotion Program in a Selected Educ...AnuragSingh1049
 
Journal of Advanced Academics-2015-Missett-96-111
Journal of Advanced Academics-2015-Missett-96-111Journal of Advanced Academics-2015-Missett-96-111
Journal of Advanced Academics-2015-Missett-96-111Lisa Foster
 
Pilot Study Publication (in press)
Pilot Study Publication (in press)Pilot Study Publication (in press)
Pilot Study Publication (in press)naomi tutticci
 
The effectiveness of co curricular activities on academic achievements of sec...
The effectiveness of co curricular activities on academic achievements of sec...The effectiveness of co curricular activities on academic achievements of sec...
The effectiveness of co curricular activities on academic achievements of sec...Alexander Decker
 
Orla davy universitycollegedublin_revised
Orla davy universitycollegedublin_revisedOrla davy universitycollegedublin_revised
Orla davy universitycollegedublin_revisedBlackboardEMEA
 
EFFECTIVENESS OF CO-OPERATIVE LEARNING METHOD IN LEARNING OF MATHEMATICS AMON...
EFFECTIVENESS OF CO-OPERATIVE LEARNING METHOD IN LEARNING OF MATHEMATICS AMON...EFFECTIVENESS OF CO-OPERATIVE LEARNING METHOD IN LEARNING OF MATHEMATICS AMON...
EFFECTIVENESS OF CO-OPERATIVE LEARNING METHOD IN LEARNING OF MATHEMATICS AMON...Thiyagu K
 
11.the effectiveness of co curricular activities on academic achievements of ...
11.the effectiveness of co curricular activities on academic achievements of ...11.the effectiveness of co curricular activities on academic achievements of ...
11.the effectiveness of co curricular activities on academic achievements of ...Alexander Decker
 
Rethinking feedback practices: Keynote Med Ed Conference Taiwan 18Oct 2014
Rethinking feedback practices: Keynote Med Ed Conference Taiwan 18Oct 2014Rethinking feedback practices: Keynote Med Ed Conference Taiwan 18Oct 2014
Rethinking feedback practices: Keynote Med Ed Conference Taiwan 18Oct 2014r_ajjawi
 
Final Paper
Final PaperFinal Paper
Final Paperlanel09
 
How to get the most out of assessment through feedback
How to get the most out of assessment through feedbackHow to get the most out of assessment through feedback
How to get the most out of assessment through feedbackr_ajjawi
 
Evidence-Based Educational Interventions
Evidence-Based Educational InterventionsEvidence-Based Educational Interventions
Evidence-Based Educational Interventionsnoblex1
 
Pnas 2014-freeman-1319030111
Pnas 2014-freeman-1319030111Pnas 2014-freeman-1319030111
Pnas 2014-freeman-1319030111telcalit2
 
Difference between problem based learning and traditional way of teaching 1
Difference between problem based learning and traditional way of teaching 1Difference between problem based learning and traditional way of teaching 1
Difference between problem based learning and traditional way of teaching 1Rocky kagiso
 

What's hot (20)

To Err Is Human
To Err Is HumanTo Err Is Human
To Err Is Human
 
Faculty Development Strategy is An Essential Element for Quality Teaching
Faculty Development Strategy is An Essential Element for Quality TeachingFaculty Development Strategy is An Essential Element for Quality Teaching
Faculty Development Strategy is An Essential Element for Quality Teaching
 
Studentsevaluationresults
StudentsevaluationresultsStudentsevaluationresults
Studentsevaluationresults
 
A Study to Evaluate the Effectiveness of Motivational Intervention on Anxiety...
A Study to Evaluate the Effectiveness of Motivational Intervention on Anxiety...A Study to Evaluate the Effectiveness of Motivational Intervention on Anxiety...
A Study to Evaluate the Effectiveness of Motivational Intervention on Anxiety...
 
20171023 The Train-the-Trainer Model 1-1 Taiwan-USA Experience -- Educating ...
20171023 The Train-the-Trainer Model 1-1 Taiwan-USA Experience -- Educating ...20171023 The Train-the-Trainer Model 1-1 Taiwan-USA Experience -- Educating ...
20171023 The Train-the-Trainer Model 1-1 Taiwan-USA Experience -- Educating ...
 
Assessment of the Existing School Health Promotion Program in a Selected Educ...
Assessment of the Existing School Health Promotion Program in a Selected Educ...Assessment of the Existing School Health Promotion Program in a Selected Educ...
Assessment of the Existing School Health Promotion Program in a Selected Educ...
 
Evidence based ppt
Evidence based pptEvidence based ppt
Evidence based ppt
 
Journal of Advanced Academics-2015-Missett-96-111
Journal of Advanced Academics-2015-Missett-96-111Journal of Advanced Academics-2015-Missett-96-111
Journal of Advanced Academics-2015-Missett-96-111
 
Pilot Study Publication (in press)
Pilot Study Publication (in press)Pilot Study Publication (in press)
Pilot Study Publication (in press)
 
The effectiveness of co curricular activities on academic achievements of sec...
The effectiveness of co curricular activities on academic achievements of sec...The effectiveness of co curricular activities on academic achievements of sec...
The effectiveness of co curricular activities on academic achievements of sec...
 
Orla davy universitycollegedublin_revised
Orla davy universitycollegedublin_revisedOrla davy universitycollegedublin_revised
Orla davy universitycollegedublin_revised
 
EFFECTIVENESS OF CO-OPERATIVE LEARNING METHOD IN LEARNING OF MATHEMATICS AMON...
EFFECTIVENESS OF CO-OPERATIVE LEARNING METHOD IN LEARNING OF MATHEMATICS AMON...EFFECTIVENESS OF CO-OPERATIVE LEARNING METHOD IN LEARNING OF MATHEMATICS AMON...
EFFECTIVENESS OF CO-OPERATIVE LEARNING METHOD IN LEARNING OF MATHEMATICS AMON...
 
11.the effectiveness of co curricular activities on academic achievements of ...
11.the effectiveness of co curricular activities on academic achievements of ...11.the effectiveness of co curricular activities on academic achievements of ...
11.the effectiveness of co curricular activities on academic achievements of ...
 
Rethinking feedback practices: Keynote Med Ed Conference Taiwan 18Oct 2014
Rethinking feedback practices: Keynote Med Ed Conference Taiwan 18Oct 2014Rethinking feedback practices: Keynote Med Ed Conference Taiwan 18Oct 2014
Rethinking feedback practices: Keynote Med Ed Conference Taiwan 18Oct 2014
 
Simulated Learning Environments
Simulated Learning EnvironmentsSimulated Learning Environments
Simulated Learning Environments
 
Final Paper
Final PaperFinal Paper
Final Paper
 
How to get the most out of assessment through feedback
How to get the most out of assessment through feedbackHow to get the most out of assessment through feedback
How to get the most out of assessment through feedback
 
Evidence-Based Educational Interventions
Evidence-Based Educational InterventionsEvidence-Based Educational Interventions
Evidence-Based Educational Interventions
 
Pnas 2014-freeman-1319030111
Pnas 2014-freeman-1319030111Pnas 2014-freeman-1319030111
Pnas 2014-freeman-1319030111
 
Difference between problem based learning and traditional way of teaching 1
Difference between problem based learning and traditional way of teaching 1Difference between problem based learning and traditional way of teaching 1
Difference between problem based learning and traditional way of teaching 1
 

Viewers also liked

Christner.LessonsLearnedFromPedResidents.ClinicalPediatrics.2006
Christner.LessonsLearnedFromPedResidents.ClinicalPediatrics.2006Christner.LessonsLearnedFromPedResidents.ClinicalPediatrics.2006
Christner.LessonsLearnedFromPedResidents.ClinicalPediatrics.2006Jenny Christner
 
Christner.WhatParentsWantFromEmailsWithTheirPediatrician.PtEducAndCounseling....
Christner.WhatParentsWantFromEmailsWithTheirPediatrician.PtEducAndCounseling....Christner.WhatParentsWantFromEmailsWithTheirPediatrician.PtEducAndCounseling....
Christner.WhatParentsWantFromEmailsWithTheirPediatrician.PtEducAndCounseling....Jenny Christner
 
Cc bond marketing
Cc bond marketingCc bond marketing
Cc bond marketingccbondgroup
 
Christner.PlayingWithChildLifeFULL.ChildLifeCouncilBulletin.2011
Christner.PlayingWithChildLifeFULL.ChildLifeCouncilBulletin.2011Christner.PlayingWithChildLifeFULL.ChildLifeCouncilBulletin.2011
Christner.PlayingWithChildLifeFULL.ChildLifeCouncilBulletin.2011Jenny Christner
 
Host Analytics & Inside Info's Unflagging Commitment To Australia With New Da...
Host Analytics & Inside Info's Unflagging Commitment To Australia With New Da...Host Analytics & Inside Info's Unflagging Commitment To Australia With New Da...
Host Analytics & Inside Info's Unflagging Commitment To Australia With New Da...Inside Info Pty Ltd
 
Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAsse...
Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAsse...Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAsse...
Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAsse...Jenny Christner
 
Christner.LessonsLearnedFromCommunitySiteAdministrators.AmbulatoryPediatrics....
Christner.LessonsLearnedFromCommunitySiteAdministrators.AmbulatoryPediatrics....Christner.LessonsLearnedFromCommunitySiteAdministrators.AmbulatoryPediatrics....
Christner.LessonsLearnedFromCommunitySiteAdministrators.AmbulatoryPediatrics....Jenny Christner
 
Thuốc chữa bệnh làm đàn ông giảm ham muốn
Thuốc chữa bệnh làm đàn ông giảm ham muốnThuốc chữa bệnh làm đàn ông giảm ham muốn
Thuốc chữa bệnh làm đàn ông giảm ham muốnbess229
 

Viewers also liked (11)

Real smm 2016
Real smm 2016Real smm 2016
Real smm 2016
 
Christner.LessonsLearnedFromPedResidents.ClinicalPediatrics.2006
Christner.LessonsLearnedFromPedResidents.ClinicalPediatrics.2006Christner.LessonsLearnedFromPedResidents.ClinicalPediatrics.2006
Christner.LessonsLearnedFromPedResidents.ClinicalPediatrics.2006
 
Christner.WhatParentsWantFromEmailsWithTheirPediatrician.PtEducAndCounseling....
Christner.WhatParentsWantFromEmailsWithTheirPediatrician.PtEducAndCounseling....Christner.WhatParentsWantFromEmailsWithTheirPediatrician.PtEducAndCounseling....
Christner.WhatParentsWantFromEmailsWithTheirPediatrician.PtEducAndCounseling....
 
Cc bond marketing
Cc bond marketingCc bond marketing
Cc bond marketing
 
Inside Info Pty Ltd
Inside Info Pty LtdInside Info Pty Ltd
Inside Info Pty Ltd
 
Christner.PlayingWithChildLifeFULL.ChildLifeCouncilBulletin.2011
Christner.PlayingWithChildLifeFULL.ChildLifeCouncilBulletin.2011Christner.PlayingWithChildLifeFULL.ChildLifeCouncilBulletin.2011
Christner.PlayingWithChildLifeFULL.ChildLifeCouncilBulletin.2011
 
тест практика речи 3 курс 6 сем
тест практика речи 3 курс 6 семтест практика речи 3 курс 6 сем
тест практика речи 3 курс 6 сем
 
Host Analytics & Inside Info's Unflagging Commitment To Australia With New Da...
Host Analytics & Inside Info's Unflagging Commitment To Australia With New Da...Host Analytics & Inside Info's Unflagging Commitment To Australia With New Da...
Host Analytics & Inside Info's Unflagging Commitment To Australia With New Da...
 
Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAsse...
Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAsse...Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAsse...
Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAsse...
 
Christner.LessonsLearnedFromCommunitySiteAdministrators.AmbulatoryPediatrics....
Christner.LessonsLearnedFromCommunitySiteAdministrators.AmbulatoryPediatrics....Christner.LessonsLearnedFromCommunitySiteAdministrators.AmbulatoryPediatrics....
Christner.LessonsLearnedFromCommunitySiteAdministrators.AmbulatoryPediatrics....
 
Thuốc chữa bệnh làm đàn ông giảm ham muốn
Thuốc chữa bệnh làm đàn ông giảm ham muốnThuốc chữa bệnh làm đàn ông giảm ham muốn
Thuốc chữa bệnh làm đàn ông giảm ham muốn
 

Similar to Christner.Long-TermGainAfterTeam-BasedLearningExp.TeachingAndLearningInMedicine.2013

DPARK Ed D
DPARK Ed DDPARK Ed D
DPARK Ed DDeb Park
 
Team-based learning (TBL) as a teaching modality in Pharmacology
Team-based learning (TBL) as a teaching modality in PharmacologyTeam-based learning (TBL) as a teaching modality in Pharmacology
Team-based learning (TBL) as a teaching modality in Pharmacologyiosrjce
 
Spotting and Supporting Eating Disorders in School - Recommendations from Sch...
Spotting and Supporting Eating Disorders in School - Recommendations from Sch...Spotting and Supporting Eating Disorders in School - Recommendations from Sch...
Spotting and Supporting Eating Disorders in School - Recommendations from Sch...Pooky Knightsmith
 
Grad Cert Final Assignment Presentation
Grad Cert Final Assignment PresentationGrad Cert Final Assignment Presentation
Grad Cert Final Assignment PresentationDanielle Hitch
 
Athletic Training Students And Preceptors Perceptions Of Active Learning Ti...
Athletic Training Students  And Preceptors  Perceptions Of Active Learning Ti...Athletic Training Students  And Preceptors  Perceptions Of Active Learning Ti...
Athletic Training Students And Preceptors Perceptions Of Active Learning Ti...Audrey Britton
 
Quality nursing education
Quality nursing educationQuality nursing education
Quality nursing educationBhushan Joshi
 
Accessing professional development activities: a survey of health sciences ac...
Accessing professional development activities: a survey of health sciences ac...Accessing professional development activities: a survey of health sciences ac...
Accessing professional development activities: a survey of health sciences ac...Brenda Leibowitz
 
PBL in ESP
PBL in ESPPBL in ESP
PBL in ESPmizah16
 
Portfolio My class is NURSING RESEARCH 28358.pdf
Portfolio My class is NURSING RESEARCH 28358.pdfPortfolio My class is NURSING RESEARCH 28358.pdf
Portfolio My class is NURSING RESEARCH 28358.pdfsdfghj21
 
PICO QUESTION2PICO QUESTION3Pico QuestionNRS.docx
PICO QUESTION2PICO QUESTION3Pico QuestionNRS.docxPICO QUESTION2PICO QUESTION3Pico QuestionNRS.docx
PICO QUESTION2PICO QUESTION3Pico QuestionNRS.docxrandymartin91030
 
2017 CoP conference program distribution
2017 CoP conference program distribution2017 CoP conference program distribution
2017 CoP conference program distributionncmi_meharry
 
Busniness logics
Busniness logicsBusniness logics
Busniness logicsfiza209
 
research articles in the fields of special education andor intern.docx
research articles in the fields of special education andor intern.docxresearch articles in the fields of special education andor intern.docx
research articles in the fields of special education andor intern.docxrgladys1
 
Expert Validation of a Teamwork Assessment Rubric PARRATT FAHY ET AL
Expert Validation of a Teamwork Assessment Rubric PARRATT FAHY ET ALExpert Validation of a Teamwork Assessment Rubric PARRATT FAHY ET AL
Expert Validation of a Teamwork Assessment Rubric PARRATT FAHY ET ALJenny Parratt
 
Each student is required to complete and turn in the evolution o
Each student is required to complete and turn in the evolution oEach student is required to complete and turn in the evolution o
Each student is required to complete and turn in the evolution oAlyciaGold776
 
Clinical Nursing Education Redesign
Clinical Nursing Education Redesign Clinical Nursing Education Redesign
Clinical Nursing Education Redesign niederha
 
Addressing Student Problem Behavior--Part III Creating Positive Behavioral I...
Addressing Student Problem Behavior--Part III  Creating Positive Behavioral I...Addressing Student Problem Behavior--Part III  Creating Positive Behavioral I...
Addressing Student Problem Behavior--Part III Creating Positive Behavioral I...Scott Donald
 

Similar to Christner.Long-TermGainAfterTeam-BasedLearningExp.TeachingAndLearningInMedicine.2013 (20)

rdme-10-10.pdf
rdme-10-10.pdfrdme-10-10.pdf
rdme-10-10.pdf
 
DPARK Ed D
DPARK Ed DDPARK Ed D
DPARK Ed D
 
Team-based learning (TBL) as a teaching modality in Pharmacology
Team-based learning (TBL) as a teaching modality in PharmacologyTeam-based learning (TBL) as a teaching modality in Pharmacology
Team-based learning (TBL) as a teaching modality in Pharmacology
 
Spotting and Supporting Eating Disorders in School - Recommendations from Sch...
Spotting and Supporting Eating Disorders in School - Recommendations from Sch...Spotting and Supporting Eating Disorders in School - Recommendations from Sch...
Spotting and Supporting Eating Disorders in School - Recommendations from Sch...
 
Strategies for integrating health literacy into entry level ot cu
Strategies for integrating health literacy into entry level ot cuStrategies for integrating health literacy into entry level ot cu
Strategies for integrating health literacy into entry level ot cu
 
Grad Cert Final Assignment Presentation
Grad Cert Final Assignment PresentationGrad Cert Final Assignment Presentation
Grad Cert Final Assignment Presentation
 
Athletic Training Students And Preceptors Perceptions Of Active Learning Ti...
Athletic Training Students  And Preceptors  Perceptions Of Active Learning Ti...Athletic Training Students  And Preceptors  Perceptions Of Active Learning Ti...
Athletic Training Students And Preceptors Perceptions Of Active Learning Ti...
 
Quality nursing education
Quality nursing educationQuality nursing education
Quality nursing education
 
Accessing professional development activities: a survey of health sciences ac...
Accessing professional development activities: a survey of health sciences ac...Accessing professional development activities: a survey of health sciences ac...
Accessing professional development activities: a survey of health sciences ac...
 
PBL in ESP
PBL in ESPPBL in ESP
PBL in ESP
 
Jackson2011
Jackson2011Jackson2011
Jackson2011
 
Portfolio My class is NURSING RESEARCH 28358.pdf
Portfolio My class is NURSING RESEARCH 28358.pdfPortfolio My class is NURSING RESEARCH 28358.pdf
Portfolio My class is NURSING RESEARCH 28358.pdf
 
PICO QUESTION2PICO QUESTION3Pico QuestionNRS.docx
PICO QUESTION2PICO QUESTION3Pico QuestionNRS.docxPICO QUESTION2PICO QUESTION3Pico QuestionNRS.docx
PICO QUESTION2PICO QUESTION3Pico QuestionNRS.docx
 
2017 CoP conference program distribution
2017 CoP conference program distribution2017 CoP conference program distribution
2017 CoP conference program distribution
 
Busniness logics
Busniness logicsBusniness logics
Busniness logics
 
research articles in the fields of special education andor intern.docx
research articles in the fields of special education andor intern.docxresearch articles in the fields of special education andor intern.docx
research articles in the fields of special education andor intern.docx
 
Expert Validation of a Teamwork Assessment Rubric PARRATT FAHY ET AL
Expert Validation of a Teamwork Assessment Rubric PARRATT FAHY ET ALExpert Validation of a Teamwork Assessment Rubric PARRATT FAHY ET AL
Expert Validation of a Teamwork Assessment Rubric PARRATT FAHY ET AL
 
Each student is required to complete and turn in the evolution o
Each student is required to complete and turn in the evolution oEach student is required to complete and turn in the evolution o
Each student is required to complete and turn in the evolution o
 
Clinical Nursing Education Redesign
Clinical Nursing Education Redesign Clinical Nursing Education Redesign
Clinical Nursing Education Redesign
 
Addressing Student Problem Behavior--Part III Creating Positive Behavioral I...
Addressing Student Problem Behavior--Part III  Creating Positive Behavioral I...Addressing Student Problem Behavior--Part III  Creating Positive Behavioral I...
Addressing Student Problem Behavior--Part III Creating Positive Behavioral I...
 

Christner.Long-TermGainAfterTeam-BasedLearningExp.TeachingAndLearningInMedicine.2013

  • 1. This article was downloaded by: [UPSTATE Medical University Health Sciences Library] On: 10 February 2015, At: 04:36 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Teaching and Learning in Medicine: An International Journal Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/htlm20 Long-Term Gain After Team-Based Learning Experience in a Pediatric Clerkship Kavita S. Warrier a , Jocelyn H. Schiller a , Nicole R. Frei a , Hilary M. Haftel a & Jennifer G. Christner a a Department of Pediatrics , University of Michigan Medical School , Ann Arbor , Michigan , USA Published online: 10 Oct 2013. To cite this article: Kavita S. Warrier , Jocelyn H. Schiller , Nicole R. Frei , Hilary M. Haftel & Jennifer G. Christner (2013) Long-Term Gain After Team-Based Learning Experience in a Pediatric Clerkship, Teaching and Learning in Medicine: An International Journal, 25:4, 300-305, DOI: 10.1080/10401334.2013.827975 To link to this article: http://dx.doi.org/10.1080/10401334.2013.827975 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions
  • 2. Teaching and Learning in Medicine, 25(4), 300–305 Copyright C 2013, Taylor & Francis Group, LLC ISSN: 1040-1334 print / 1532-8015 online DOI: 10.1080/10401334.2013.827975 Long-Term Gain After Team-Based Learning Experience in a Pediatric Clerkship Kavita S. Warrier, Jocelyn H. Schiller, Nicole R. Frei, Hilary M. Haftel, and Jennifer G. Christner Department of Pediatrics, University of Michigan Medical School, Ann Arbor, Michigan, USA Background: Team-based learning (TBL) increases student en- gagement, value of teamwork, and performance on standardized evaluations. Purpose: The authors implemented a 3rd-year pedi- atric TBL curriculum, evaluating its effect on satisfaction, engage- ment, value of teamwork, and short-term and long-term academic performance. Method: Students evaluated the TBL curriculum and core lectures through satisfaction, engagement and value of team surveys. Scores on short-term and long-term examinations were compared to historical data. Results: The first implementa- tion year, students were less likely to enjoy TBL sessions com- pared to lectures. The 2nd year, this difference lessened. Through both years, students reported dramatic increases in classroom en- gagement during TBL compared to lecture. Students developed a greater value for teams after participating in TBL. Short-term and long-term examination scores improved significantly. Conclusions: Both short-term and long-term performance improved with imple- mentation of TBL, emphasizing the benefits of a curriculum that allows students to critically engage with material. Keywords team based learning, medical student, pediatrics, active learning A teacher-centric perspective has traditionally been the most common orientation to teaching in medical education. Little evidence supports dissemination-only strategies if the goal is to improve professional practice.1 A growing body of medical education research has demonstrated that active learning is more effective than passive learning.2,3 A review of educational theory suggests that learners actively contribute to the educational process, practice skills accompa- nied by constructive feedback from teachers and peers, and re- late learning to understanding and solving real-life problems.4 Indeed, the Liaison Committee for Medical Education has incor- We thank Drs. Mary Hernandez and Laura Hopson for their assis- tance in data collection. We also acknowledge Drs. Kenneth Pituch, Terrance Murphy, Patricia Keefer, Kerry Mychaliska, and Francis McBee-Orzulak for their help developing and implementing the TBL curriculum. Correspondence may be sent to Jocelyn H. Schiller, 12-525 Mott, 1540 East Hospital Drive, SPC 4280, Ann Arbor, MI 48109-4280, USA. E-mail: johuang@umich.edu porated active learning into its expected standards.5 Although education reform emphasizes altering the pedagogical approach to curriculum, such as switching from lecture format to case studies,6 generational issues also need consideration. A recent meta-analysis on generational changes indicates that students born after 1980 prefer learning by doing rather than by listening to lectures. This generation of students is also more likely to be overconfident and thus may require more feedback.7 Team-based learning (TBL) is an innovative instructional method that fosters active learning and has been adopted by some medical schools, particularly in the preclinical years.8–13 TBL utilizes active small-group discussion facilitated by an in- structor. Teams solve clinical problems and receive immediate feedback on performance. Student performance is graded, with an assessment of individual knowledge, evaluation of group performance, and peer feedback making up the incentive struc- ture of the method. TBL has been shown to improve learning outcomes and examination scores,11,14–18 communication pro- cesses, teamwork skills, and the value students place on these skills,10.11,13,15 and student engagement in class settings.11,19–21 This correlates both with student satisfaction and student achievement, especially when the subject matter is difficult. TBL in the clinical setting has been previously described for psychiatry and internal medicine clerkships11,14,22,23 but uti- lizing TBL to deliver core curriculum in a pediatric clerkship has not been reported. Long-term clinical knowledge gains have also not been reported. In our clerkships’ transition from didac- tic lectures to TBL, we had four objectives: (a) compare student evaluations of TBL teaching sessions to core didactic lectures, (b) analyze the students’ perspectives of the value of teams before and after experiencing TBL, (c) analyze classroom en- gagement during TBL, and (d) compare academic performance between students prior to and after implementation of TBL ses- sions, including long-term gains. METHODS Educational Intervention At the University of Michigan Medical School, all 3rd-year medical (M3) students rotate through an 8-week pediatric clerk- ship with approximately 25 to 32 students in each cohort. In the 300 Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015
  • 3. LONG-TERM GAINS AFTER TBL IN PEDIATRIC CLERKSHIP 301 past, 21 hours were set aside for core pediatric teaching in the form of didactic lectures. Beginning with the 2009–10 academic year (2009 AY), six 90-minute TBL sessions replaced 12 of the core teaching hours. The 10 pediatric faculty members involved in the develop- ment, implementation, and facilitation of TBL modules attended a TBL workshop in 2008. Three core faculty members also at- tended the TBL Collaborative Annual Conference. Module top- ics were chosen by reviewing institutional objectives and the national Council on Medical Student Education in Pediatrics Curriculum to identify core pediatric topics.24 The modules were piloted and reviewed prior to implementation and have since been published on MedEd Portal.25–30 Because the students had no prior TBL experience, they were given information about TBL during the clerkship orientation. Reading assignments were provided for each module. The in- structors randomly divided students into teams of five to six at the first TBL session. The students remained in the same teams for all six TBL sessions. Each block of six TBL sessions was facilitated by a team of two faculty members to allow students to develop rapport with the faculty and to provide a variety of faculty perspectives. Each session consisted of readiness assurance tests (RATs) taken individually (IRATs) and as a group (GRATs), followed by application exercises (see Table 1). Students were not allowed to use reading materials during the RATs. Immediate feedback was provided during the GRAT with scratch-off answer sheets denoting the correct answer with a star. Faculty facilitators re- viewed the answers to the RATs and clarified key points before students started the application exercises. For 24 hours after the session, any team was allowed to appeal RAT questions by showing a credible source that supported an alternative answer. The application exercises consisted of more complex clinical scenarios, which required synthesis of information, often focus- ing on diagnosis or treatment. Most questions had one “most right” answer with other “right” answer choices. Each team dis- cussed the questions and came to a consensus on the answers. After the intra-team discussion was complete, the teams came together for inter-team dialogue. Each team simultaneously dis- played their answer; the facilitators then required teams to ex- plain their reasoning and defend their answer choice. TABLE 1 Time spent on Readiness Assurance Tests (RAT) and application questions Exercise Time Spent No. and Type of Questions Individual RAT 15 min 9 to 12 multiple choice questions based on readings Group RAT 30 min Same as IRAT Application 45 min 2 to 5 complex clinical scenarios At the final session, students gave written peer feedback adapted from the Koles method,31 which was reviewed by the associate clerkship director and assigned a grade. Twenty-five percent of the TBL grade was based on the IRAT and 65% of the TBL grade was based on the GRAT; the other 10% of the TBL grade came from peer feedback. These distributions were assigned by the course director. The TBL grade was worth 5% of the total clerkship grade, which was in line with amounts assigned to other clerkship assessments such as written notes, assignments, and professionalism. Objectives and Data Collection During the 2009 AY, we gathered student evaluations of the core didactic lectures and TBL sessions. These evaluations were not anonymous. We also collected data on student engagement during the didactic and TBL sessions using the Classroom En- gagement Survey, a tool from Baylor College of Medicine that asked questions relating to engagement in the lecture or TBL session.11,32 Students’ perception of the value of teamwork was measured at the beginning and end of the clerkship using a nine- item “Value of Teams” survey developed at Baylor College of Medicine.11,33 All items were scored on a 5-point Likert scale anchored by 1 (strongly disagree) and 5 (strongly agree). Academic performance was assessed with three measures. National Board of Medical Examiners (NBME) Pediatric Shelf scores for the 2009 AY and 2010 AY were compared to scores from the 4 years prior. We also analyzed scores on our M4 Com- prehensive Clinical Assessment (CCA), a high-stakes Objective Structured Clinical Examination all 4th-year students must pass. The CCA included several stations, one of which entailed ob- taining a pediatric history from a standardized patient parent. Students are randomly assigned to either the pediatric fever or asthma standardized patient parent. Scores from this pediatric station were compared between the students who completed the clerkship in AY 2009 to students from the 4 years prior. Our last performance measure was an examination taken during the required M4 Emergency Medicine (EM) clerkship. AY 2009 student scores on two clinical questions about fever in neonates and toddlers were compared with those from the 2 years prior. The CCA stations and EM examination questions were chosen because both fever and asthma were previously taught during the M3 year by didactic lecture, but in 2009 AY became TBL topics. These M4 CCA assessments were done between 1 and 10 months after the M3 pediatric clerkship and the EM exam was taken between 1 and 22 months after completion of the M3 pediatric clerkship. Other than the implementation of TBL, no other changes were made to the curriculum during these study years. This study was reviewed by the University of Michigan In- stitutional Review Board and granted exempt status. Statistical Analysis Mean response to each survey item for overall evaluation, student classroom engagement, and value of teams were Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015
  • 4. 302 K. S. WARRIER ET AL. calculated and compared by independent samples t tests. The Value of Teams Survey analyzes two different learner perceived dimensions of learning in groups, the Value of Working with Peers and the Value of Group Work,34 which were calculated and compared by independent samples t test. Independent samples t tests were used to compare means for NBME scores, CCA performance, and EM exam scores. Cohen’s d was calculated to show effect size. Data were analyzed using SAS 9.2 Statistical Software. RESULTS One hundred seventy-nine students completed the pediatric clerkship in 2009 AY and 160 students completed the clerkship in 2010 AY. Satisfaction Students reported lower satisfaction with team-based learn- ing as compared to lectures for almost all features in both years (see Table 2). However, the mean differences in satisfaction be- tween lecture and TBL dropped in the 2nd year of study. The differences between these means were statistically significant for six of the seven questions in the 1st year, and five of the seven questions in the 2nd year. The only change between years was the question of whether the topic was of importance. Value of Teams For all items on the Value of Team survey, students demon- strated a greater appreciation for teamwork after the completion of the term, but this was only statistically significant for three of the items: “Working in teams in class is productive and efficient,” “Group decisions are often better than individual decisions,” and “Solving problems in groups leads to better decisions than solv- ing problems alone” (see Table 3). Although the other six items were not statistically significant, all items trended in a positive direction after students experienced TBL. The Value of Teams Survey analyzes two different learner perceived dimensions of learning in groups; there was no significant difference in the overall Value of Working with Peers. There was, however, a sig- nificant difference in student assessment of the Value of Group Work (0.24; p <.01, d = .38) after completion of the team-based learning curriculum. Classroom Engagement Students reported higher levels of personal contribution and participation, as well as fellow student involvement, in TBL TABLE 2 Student satisfaction and classroom engagement: Comparing lecture to TBL sessions 2010 2011 Lecture Lecture Question Ma TBL Mb Diff p d Mc TBL Md Diff p d Standard Evaluation Achieved Objectives 4.60 4.24 0.36 <.01 0.41 4.63 4.45 0.18 <.01 0.24 Session Effective 4.55 4.28 0.26 <.01 0.32 4.57 4.46 0.11 <.01 0.13 Topic Important 4.65 4.57 0.08 <.01 0.12 4.67 4.65 0.02 .43 0.04 Level of Discussion Appropriate 4.46 4.43 0.03 .48 0.03 4.50 4.56 0.06 .16 0.06 Organized 4.64 4.31 0.33 <.01 0.42 4.67 4.46 0.21 <.01 0.30 Session Appropriate 4.61 4.44 0.17 <.01 0.23 4.66 4.57 0.09 <.01 0.12 Engagement I Contributed 3.04 4.34 1.30 <.01 0.94 3.36 4.37 1.01 <.01 0.75 I Did Not Have Fun 1.62 1.95 ∗ n = 1,030 0.33 <.01 0.32 1.53 1.70 0.17 <.01 0.19 I Had Fun 4.03 3.90 0.13 <.01 0.12 4.10 4.17 0.07 .14 0.07 I Paid Attention 4.45 ∗ n = 878 4.43 0.02 .63 0.02 4.45 4.42 0.03 .40 0.04 I Would Like More Like This 4.12 3.94 0.18 <.01 0.18 4.20 4.12 0.08 .10 0.08 Most Students Involved 3.85 4.33 0.48 <.01 0.45 4.00 4.39 0.39 <.01 0.22 Most Students Did Not Pay Attention 1.73 1.84 0.11 .03 0.10 1.69 1.70 0.01 .84 0.01 I Participated 3.52 4.45 0.93 <.01 0.62 3.77 4.47 0.70 <.01 0.58 Average 3.96 ∗ n = 878 4.20 ∗ n = 1,030 0.24 <.01 0.34 4.08 4.32 0.24 <.01 0.32 Note. TBL = team-based learning. a N = 879. b N = 1,033. c N = 943. d N = 916. Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015
  • 5. LONG-TERM GAINS AFTER TBL IN PEDIATRIC CLERKSHIP 303 TABLE 3 Value of teams survey Pre-TBL Post-TBL Item Question Meana Meanb p 1 The ability to collaborate with my peers will be necessary if I am to be successful as a student. 4.61 4.65 .61 2 It is a waste of time to work in groups. 1.80 1.69 .20 3c The ability to work with my peers is a valuable skill. 4.70 4.72 .71 4 Collaborating with my peers will help me be a better student. 4.40 4.52 .07 5 Solving problems in a group is an effective way to practice what I have learned. 4.19 4.25 .50 6 Solving problems in a group is an effective way to learn. 4.11 4.21 .25 7c Working in teams in class is productive and efficient. 3.54 3.78 .02 8 Group decisions are often better than individual decisions. 3.91 4.26 <.01 9c Solving problems in groups leads to better decisions than solving problems alone. 3.99 4.31 <.01 Note. TBL = team-based learning. a n = 171. b n = 170. c Post-TBL n = 169. sessions as compared to lecture, noted by the higher ratings for TBL in the items “I contributed to class,” “Most students were involved in class,” and “I participated in class” (see Table 2). However, similar to the data from the evaluations, students re- ported higher agreement with statements such as “I did not have fun” for TBL sessions and did not want more sessions like TBL. Again, similar to the data from the evaluations, we noted im- provements in the 2nd year with students enjoying TBL more the 2nd year. Overall, the cumulative Classroom Engagement score showed a significant increase between TBL sessions as compared to lecture in both years. The effect size for these mean differences was notable partic- ularly for student agreement with statements about their personal contribution and participation across both years. Academic Performance NBME SHELF Exam. For 4 years prior to the initiation of TBL, the mean on the Pediatric Shelf was 79.46 (n = 614; see Table 4). The mean on the Pediatric Shelf for the 2 years after the initiation of TBL was 82.50 (n = 311). This is an increase of 3.04 points ( p <.0001, d = .36). Shelf data were unavailable for 31 students in 2009 AY. M4 CCA. For the 4 years prior to initiation of TBL, the student mean on the CCA Pediatric Station was 73.02 (n = 668; see Table 4). The student mean on the CCA for the 1st year after TBL was introduced was 75.55 (n = 149). The increase of 2.53 points was statistically significant (p = .0109, d = .24). Data for 22 students were unavailable, as several students took a leave of absence between M3 and M4 year to pursue academic interests. TABLE 4 Measures of academic performance Test M Score SD Difference p Cohen’s d NBME Shelf Examination Pre-TBLa 79.46 8.14 3.04 <.01 0.36 Post-TBLb 82.50 8.72 Clinical Skills Examination Pre-TBLc 73.05 12.42 2.89 <.01 0.24 Post-TBLd 75.94 11.00 EM Score (Toddler) Pre-TBLe 2.21 1.04 0.28 <.01 0.28 Post-TBLf 2.49 0.84 EM Score (Neonate) Pre-TBLe 6.00 1.51 0.45 <.01 0.32 Post-TBLf 6.45 1.17 Note. NBME = National Board of Medical Examiners; TBL = team-based learning; EM = Emergency Medicine. a n = 614. b n = 311. c n = 668. d n = 149. e n = 281. f n = 151. Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015
  • 6. 304 K. S. WARRIER ET AL. EM Clinical Skills Exam. The student mean score on the question addressing fever in toddlers increased by 0.28 (p = .0032) after the introduction of TBL sessions, a 9% increase in the score out of a maximum score of 3 points. The student mean score on the question addressing fever in neonates increased by 0.45 (p = .0006, d = .32), an increase of 6.4% out of the maximum score of 7 points. DISCUSSION To the best of our knowledge, this is the first study demon- strating that 3rd-year students participating in TBL had sus- tained improved in examination performance. Active discussion in the context of clinical scenarios may lead to longer retention of knowledge than that learned through rote memorization. Our findings support our hypothesis that TBL has a positive im- pact on students’ learning as demonstrated by improved test scores the following academic year. These results are likely multifactorial. Students must read preparatory materials prior to the TBL session, which may increase the amount of reading during the clerkship on subject matter that is directly linked to clerkship and institutional learning objectives, improving their preparation for examinations. The peer interactions during TBL sessions require students to choose best answers within teams and educate each other, which likely enhances personal under- standing of the material.17 The increased accountability to the group due to peer evaluation may have encouraged students to engage in more independent study.17 Across both years of study, students reported an increase in classroom engagement during TBL sessions as compared to lec- tures. This may also help explain the improvement in academic performance. As students critically examine clinical questions, they become invested in learning why certain choices are most appropriate, which likely invests them in learning.17 Because faculty can assess student knowledge from the IRAT and group discussion, faculty can then direct the ensuing discussion toward clarifying difficult concepts that were not well understood. Despite the positive effect on learning, students did not rate TBL as highly as didactic lectures. This confirms the results of another study which showed that although learners were more engaged during TBL, the perceived value of didactic sessions was higher.21 In that study, however, student performance was the same between TBL and didactic lectures, whereas our stu- dents had improved performance after TBL. We chose to keep only the most highly rated didactic lectures in our curriculum, which may have influenced our findings. In addition, our stu- dents have minimal exposure to TBL; introduction of a novel instructional method that requires more work on the part of the student may have affected student satisfaction. Orienting the students to TBL may be important in fostering acceptance. Overall, we are encouraged that the evaluations of TBL im- proved in the 2nd year. Another study has also demonstrated improvement in student perceptions of the merit of TBL teach- ing over a 3-year time span.35 Although faculty data were not formally collected, our faculty report increased comfort with the material and teaching style over time, which is supported by a qualitative study by Thompson et al. reporting that faculty needed time to become comfortable with the TBL method and faculty expertise was vital to successful TBL implementation.36 Faculty indicated that initial use of TBL was of limited success but that repeated use of TBL was essential to success.13 Our experience reminds us that creating the conditions for effective change require time, experience, and a shifting of attitudes for both students and faculty. The students involved in this study were primarily students from “Generation Me” who often require support when try- ing something new. It has been noted that this generation likes to know exactly what to do to get good grades and finds am- biguous instruction stressful.7 Our students may need additional preparation in order to accept TBL.23 We have created a more extensive orientation to TBL illustrating not only the process but the perceived benefits of this type of instruction, hoping to give students clearer expectations and improve their view of the process. The fact that medical decision making can yield mul- tiple correct yet different paths is also an intrinsic part of TBL, and one that may be difficult for junior students to accept. Even experienced clinicians may be uncomfortable with diagnostic uncertainty, despite it being part of clinical medicine. Although it may be uncomfortable for students—and may lead to low evaluations—it does not change the fact that this is important exposure for students and may be something they value in the long run—if not now. Our results support the findings of previous studies indicating that TBL can teach students to value teamwork. Students were more likely to appreciate using teamwork in class and, more important, see the value that comes from shared decision mak- ing. This promotes a collaborative decision-making process, an important value in this age of patient- and family-centered care. More research will be necessary to determine whether expo- sure to team learning impacts future views of teamwork in the clinical setting or ultimately impacts clinical care. This study has some limitations. Because TBL is a graded component of the 3rd-year curriculum, we were unable to cre- ate a control group for comparison; the entire class participated. This necessitated the use of historical controls for the analy- sis of academic performance. The yearly differences between classes in the medical school could explain the variation in Shelf, CCA, and ED clinical examination scores. Our mean student Shelf scores prior to the intervention, however, were constant. Demographic data that influence standardized test-taking ability were not included in the statistical analysis and could influence the scores. We have only 1 year of ED clinical exam data, as the examination was changed after that year, so we are unable to see if this improvement was sustained in the next academic year. All students were required to turn in the surveys, and so response or nonresponse bias was not a factor; however, the student surveys were not anonymous, which could have affected the students’ report on all measures. Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015
  • 7. LONG-TERM GAINS AFTER TBL IN PEDIATRIC CLERKSHIP 305 CONCLUSIONS This study adds to the growing body of evidence suggest- ing the effectiveness of TBL in achievement of knowledge ob- jectives for medical students and is the first to demonstrate knowledge gains extending into the next academic year. TBL addressed the needs of this generation of learners to be more actively involved in their education while allowing the fac- ulty member to provide “real-time” feedback as students work through clinical cases. Faculty need time to adapt to a differ- ent pedagogical method, but once practiced, the gains in ac- tive learning, teamwork, and increase in long-term performance seem well worth the investment. REFERENCES 1. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: A systematic review of 102 trials of interventions to improve profes- sional practice. Canadian Medical Association Journal 1995;153:1423– 31. 2. Richardson D, Birge B. Teaching physiology by combining passive (peda- gogical) and Active (andragogical) methods. Advances in Physiology Edu- cation 1995;13:S66–74. 3. Graffam B. Active learning in medical education: Strategies for beginning implementation. Medical Teacher 2007;29:38–42. 4. Kaufman DM. Applying educational theory in practice. British Medical Journal. 2003;326:213–6. 5. Liaison Committee on Medical Education. Functions and Struc- ture of a Medical School. 2008. Available at: http://www.lcme.org/ functions2008jun.pdf. Accessed December 7, 2009. 6. Armstrong EG, Mackey M, Spear SJ. Medical education as a process man- agement problem. Academic Medicine 2004;79:721–8. 7. Twenge JM. Generational changes and their impact in the classroom: Teach- ing Generation Me. Medical Education 2009;43:398–405. 8. Seidel CL, Richards BF. Application of team learning in a medical physi- ology course. Academic Medicine 2001;76:533–4. 9. Haidet P, O’Malley KJ, Richards B. An initial experience with ‘team learn- ing’ in medical education. Academic Medicine 2003;77:40–4. 10. Hunt DP, Haidet P, Coverdale JH, Richards BF. The effect of using team learning in an evidence-based medicine course for medical students. Teach- ing & Learning in Medicine 2003;15:131–9. 11. Levine RE, O’Boyle M, Haidet P, Lynn DJ, Stone MM, Wolf DV, et al. Transforming a clinical clerkship with team learning. Teaching & Learning in Medicine 2004;16:270–5. 12. Dunaway GA. Adaption of team learning to an introductory graduate phar- macology course. Teaching & Learning in Medicine 2005;17:56–62. 13. Thompson BM, Schneider VF, McMahon K, Levine R, Perkowski L, Richards BF. Team learning at ten medical schools: Two years later. Medi- cal Education 2007;41:250–7. 14. Tan NCK, Kandiah N, Chan YH, Umapathi T, Lee SH, Tan K. A con- trolled study of team-based learning for undergraduate clinical neurology education. BMC Medical Education 2011;11:91. 15. Zgheib NK, Simaan JA, Sabra R. Using team-based learning to teach phar- macology to second year medical students improves student performance. Medical Teacher 2010;32:130–5. 16. Touchet BK, Coon KA. A pilot use of team-based learning in psychiatry resident psychodynamic psychotherapy education. Academic Psychiatry 2005;29:293–6. 17. Koles PG, Stolfi A, Borges NJ, Nelson S, Parmelee DX. The impact of team-based learning on medical students’ academic performance. Academic Medicine 2010;85:1739–42. 18. Searle NS, Haidet P, Kelly A, Schneider VF, Seidel CL, Richards BF. Team learning in medical education: Initial experiences at ten institutions. Academic Medicine 2003;78:S55–8. 19. Vasan NS, DeFouw D. Team learning in a medical gross anatomy course. Medical Education 2005;39:524. 20. Kelly PA, Haidet P, Schneider V, Searle N, Seidel CL, Richards BF. A comparison on in-class learner engagement across lecture, problem-based learning and team learning using the STROBE classroom observation tool. Teaching & Learning in Medicine 2005;17:112–8. 21. Haidet P, Morgan RO, O’Malley K, Moran BJ, Richards BF. A controlled trial of active versus passive learning strategies in a large group setting. Advances in Health Sciences Education 2004;9:15–27. 22. Ravindranath D, Gay TL, Riba MB. Trainees as teachers in team-based learning. Academic Psychiatry 2010;34:294–7. 23. Thomas PA, Bowen CW. A controlled trial of team-based learning in an ambulatory medicine clerkship for medical students. Teaching & Learning in Medicine 2011;23:31–6. 24. Council on Medical Student Education in Pediatrics. Curriculum competencies and objectives. 2005. Available at: http://comsep.org/ educationalresources/currobjectives.cfm. Accessed December 15, 2009. 25. Warrier K, Schmidt J, Schiller J, Murphy T, Frei N. Pediatric Fever Team- Based Learning Module. MedEdPORTAL; 2010. Available from: http:// services.aamc.org/30/mededportal/servlet/s/segment/mededportal/?subid= 8356 26. Schiller J, Frei N, Keefer P, Warrier K, McBee Orzulak F. Pediatric Anemia Team-Based Learning Module. MedEdPORTAL; 2010. Available from: http://services.aamc.org/30/mededportal/servlet/s/segment/mededportal/? subid=8332 27. Warrier K, Mychaliska K, Schiller J, Mcbee-Orzulak F, Frei N. Pediatric Nutrition Team-Based Learning Module. MedEdPORTAL; 2010. Available from: http://services.aamc.org/30/mededportal/servlet/s/ segment/mededportal/?subid=8358 28. Keefer P, Frei N, Schiller J, Murphy T. Pediatric Abdominal Pain and Mass Team-Based Learning Module. MedEdPORTAL; 2010. Available from: http://services.aamc.org/30/mededportal/servlet/s/segment/mededportal/? subid=8371 29. McBee Orzulak F, Frei N, Murphy T, Keefer P, Schiller J. Pediatric Asthma Team-Based Learning Module. MedEdPORTAL; 2011. Available from: http://services.aamc.org/30/mededportal/servlet/s/segment/mededportal/? subid=8352 30. Frei N, Pituch K, Keefer P, Schiller J. Pediatric Pneumonia Team- Based Learning Module. MedEdPORTAL; 2011. Available from: http:// services.aamc.org/mededportal/servlet/s/segment/mededportal/?subid = 8353 31. Levine RE. Peer evaluation in team-based learning. In LK Michaelson, DX Parmelee, KK McMahon, & RE Levine (Eds.), Team-based learning for health professions education (pp. 107–14). Sterling, UK: Stylus, 2008. 32. Classroom engagement survey. In Team learning in medical education workshop handbook. Houston, TX: Baylor College of Medicine, Office of Curriculum, 2001. 33. Value of Teams Survey. In Team learning in medical education workshop handbook. Houston, TX: Baylor College of Medicine, Office of Curriculum, 2001. 34. Guide to using the Value of Teams Survey. In Team learning in medical education workshop handbook. Houston, TX: Baylor College of Medicine, Office of Curriculum, 2001. 35. Davidson LK. A 3-year experience implementing blended TBL: Active instructional methods can shift student attidues to learning. Medical Teacher 2011;33:750–3. 36. Thompson BM, Schneider VF, Haidet P, Perkowski LC, Richards BF. Factors influencing implementation of team-based learning in health science education. Academic Medicine 2007;82:S53–6. Downloadedby[UPSTATEMedicalUniversityHealthSciencesLibrary]at04:3610February2015