Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAssessmentTool.ReflectivePractice.2011

This article was downloaded by: [Syracuse University Library]
On: 10 February 2015, At: 07:31
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Reflective Practice: International and
Multidisciplinary Perspectives
Publication details, including instructions for authors and
subscription information:
http://www.tandfonline.com/loi/crep20
Faculty reflections on the
implementation of socio‐cultural
eportfolio assessment tool
Rachel L. Perlman
a
, Paula T. Ross
b
, Jennifer Christner
c e
&
Monica L. Lypson
a d e
a
Ann Arbor Veterans Affairs Medical Center, Ann Arbor, Michigan,
USA and Department of Internal Medicine , University of Michigan
Medical School , Ann Arbor, Michigan, USA
b
Department of Health Behavior and Health Education ,
University of Michigan School of Public Health , Ann Arbor,
Michigan, USA
c
Department of Pediatrics and Communicable Diseases ,
University of Michigan Medical School , Ann Arbor, Michigan, USA
d
University of Michigan Health System, Graduate Medical
Education , Ann Arbor, Michigan, USA
e
Department of Medical Education , University of Michigan
Medical School , Ann Arbor, Michigan, USA
Published online: 13 May 2011.
To cite this article: Rachel L. Perlman , Paula T. Ross , Jennifer Christner & Monica L. Lypson
(2011) Faculty reflections on the implementation of socio‐cultural eportfolio assessment
tool, Reflective Practice: International and Multidisciplinary Perspectives, 12:3, 375-388, DOI:
10.1080/14623943.2011.571868
To link to this article: http://dx.doi.org/10.1080/14623943.2011.571868
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the
“Content”) contained in the publications on our platform. However, Taylor & Francis,
our agents, and our licensors make no representations or warranties whatsoever as to
the accuracy, completeness, or suitability for any purpose of the Content. Any opinions
and views expressed in this publication are the opinions and views of the authors,
and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content
should not be relied upon and should be independently verified with primary sources

of information. Taylor and Francis shall not be liable for any losses, actions, claims,
proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or
arising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Conditions of access and use can be found at http://www.tandfonline.com/page/terms-
and-conditions
Downloadedby[SyracuseUniversityLibrary]at07:3110February2015

Reflective Practice
Vol. 12, No. 3, June 2011, 375–388
ISSN 1462-3943 print/ISSN 1470-1103 online
© 2011 Taylor & Francis
DOI: 10.1080/14623943.2011.571868
http://www.informaworld.com
Faculty reflections on the implementation of socio-cultural
eportfolio assessment tool
Rachel L. Perlmana, Paula T. Rossb, Jennifer Christnerc,e and Monica L. Lypsona,d,e*
a
Ann Arbor Veterans Affairs Medical Center, Ann Arbor, Michigan, USA and Department of
Internal Medicine, University of Michigan Medical School, Ann Arbor, Michigan, USA;
b
Department of Health Behavior and Health Education, University of Michigan School of
Public Health, Ann Arbor, Michigan, USA; c
Department of Pediatrics and Communicable
Diseases, University of Michigan Medical School, Ann Arbor, Michigan, USA; d
University of
Michigan Health System, Graduate Medical Education, Ann Arbor, Michigan, USA;
e
Department of Medical Education, University of Michigan Medical School, Ann Arbor,
Michigan, USA
Taylor and FrancisCREP_A_571868.sgm
(Received 1 July 2010; final version received 1 November 2010)
10.1080/14623943.2011.571868Reflective Practice1462-3943 (print)/1470-1103 (online)Article2011Taylor & Francis123000000July 2011Dr MonicaLypsonmlypson@umich.edu
Electronic portfolios (eportfolios) are increasingly used to assess student
development. The degree to which faculty in North American medical schools are
confident in their ability to evaluate such materials and their perceptions of the
utility of eportfolios are unclear. Thirteen medical educators developed skills
necessary to critically assess students’ reflective essays. Qualitative methods were
used to explore faculty experiences with a socio-cultural eportfolio. Despite self-
perceptions of being novice users, faculty were willing to engage with the
eportfolio and assume an unfamiliar role in guiding medical students’ personal
reflections. Additionally, we uncovered unique faculty expectations regarding the
quality of students’ essays and their thoughts about the utility of an eportfolio as
an assessment tool. Our findings indicate that given adequate time and training,
faculty will use an eportfolio. They believe it is an effective method for both
longitudinal assessment of student learning and achieving curricular objectives.
Our findings also illustrate the numerous unintended ethical challenges faculty
encounter as they engage in student assessment.
Keywords: faculty development; reflection; electronic portfolios; socio-cultural
curriculum; undergraduate medical education; qualitative methods
Introduction
There is a growing body of literature supporting the utility of electronic portfolios
(eportfolios) as an effective means of demonstrating both learning and reflection (Dries-
sen, van Tartwijk, Overeem, Vermunt, & van der Vleuten, 2005; Gruppen, Frohna,
Anderson, & Lowe, 2003). Portfolios have long been used as an authentic educational
tool to demonstrate learners’ development and achievement; they have recently been
incorporated into medical educational programs within the United States and Europe
(Austin & Braidman, 2008; Challis, 1999; Dannefer & Henson, 2007; Dekker et al.,
2009; Driessen, Overeem, van Tartwijk, van der Vleuten, & Muijtjens, 2006; Rees &
Sheard, 2004). While the review of student reflections is an established method for
*Corresponding author: Email: mlypson@umich.edu

376 R.L. Perlman et al.
demonstrating the occurrence of learning and identifying gaps in knowledge, two essen-
tial components have also been identified for the construction of successful portfolios:
(1) the opportunity for reflection; and (2) the development of a relationship between
student and teacher (Sandars, 2009). Further, extant research has found that collecting
a portfolio of previous work is a necessary yet insufficient step in characterizing an
educational portfolio. Rather, in order to develop this work into a meaningful portfolio
that will facilitate learning, a mentor or a teacher needs to help guide students through
the learning processes required (McMullan et al., 2003). Within the context of medical
education, faculty mentoring on the process of reflection can make important contri-
butions to the formal curricula as students prepare to become successful practitioners
(Carraccio & Englander, 2004; Dekker et al., 2009). Successful implementation of port-
folios has been found to depend on adequate faculty training to provide medical students
with meaningful feedback or coaching (Austin & Braidman, 2008; Driessen et al., 2005;
Gruppen et al., 2003). Others have determined that assessment of learner’s reflections
is inherently subjective and this activity is best filled by medical educators who are able
to engage in ‘mindful practice’ and can serve as role models to medical students and
residents (Gans, 2009). Under this guise, faculty frequently engage in reflective
exercises in which they must adopt the perspective of the other in order to fully embrace
alternative perspectives, and carefully consider the issues of power and responsibility
that emerge during the evaluation process (Swanson, 2010).
Whether reflection is a natural and inherent trait or teachable skill is the center of
much ongoing debate (Wald, Davis, Reis, Monroe, & Borkan, 2009). Nevertheless,
the use of electronic portfolios to facilitate reflection can help organize instructional
and assessment materials as well as provide space for students to reflect and receive
mentoring on their reflections. Educational programs must remain cognizant that
while it may be an important component of the curricula to require students to reflect
on their personal emotions and connect with the affective domain of learning, such
requirements make students quite vulnerable to criticism and judgment by their
reviewers who are responsible for monitoring those responses and progress. Both
partakers – faculty and students – might encounter ethical dilemmas as the author and
the reviewer in any portfolio experience (Ghaye, 2007).
Upon careful consideration of the key components that would properly assess a
student’s growth and development, we designed and implemented the Socio-cultural
ePortfolio Assessment Tool (SePAT). We recognized that this new educational tech-
nology would also require that faculty receive proper training to effectively evaluate
students’ reflective assignments within the eportfolio. Funding for this endeavor was
provided by the Gilbert Whitaker Fund for the Improvement of Teaching, an internal
grant competition funded by the University of Michigan’s Center for Research on
Learning and Teaching. Through a SAKAI-based, open-source and secured electronic
portfolio, students collected curriculum artifacts to demonstrate their achievement of
socio-cultural learning objectives (Table 1). The Socio-cultural Curriculum is
designed to teach and develop knowledge, skills and attitudes that contribute to socio-
cultural competence in medical education. This curriculum is also designed to meet
the Liaison Committee on Medical Education (LCME) mandate requiring medical
schools in North America to teach principles of culturally competent care and the
significance of health care disparities (Beach et al., 2005; Liaison Committee on
Medical Education, 2008). The Socio-cultural Curriculum is not a distinct course, but
includes dedicated lectures, assigned readings and small group discussions that have
been woven into the traditional medical school curriculum. This framework represents

Reflective Practice 377
a model curriculum for the use of an eportfolio (Kumagai & Lypson, 2009; Lypson,
Ross, & Kumagai, 2008; Tang, Fantone, Bozynski, & Adams, 2003). Each medical
school class comprises approximately 170 students per year, making delivering
individual feedback an exceptional challenge. The eportfolio is well suited to helping
faculty meet this challenge as well as to emphasizing the importance of the socio-
cultural learning objectives.
Students and faculty within many US medical schools are inexperienced in the
implementation and use of an eportfolio as an assessment tool. Thus we designed our
faculty development sessions using Kolb’s learning cycle, a framework which is
especially appropriate for adult learners (Kolb, 1984). All sessions included: (1) a
concrete hands-on experience with the tool; (2) faculty assignments requiring reflec-
tive observations of prior curricular and extra-curricular experiences related to socio-
cultural medicine; (3) development of a rating scale to aid their conceptualization of
distinct evidence of experiential learning and growth; and (4) practice exercises
reviewing sample essays similar to those they would actually review in the future.
Applying this framework to new technologies has been found to aid faculty in devel-
oping expertise in their assessment abilities (Chism, 2004).
While we recognize that amid the process of reviewing students’ reflections
faculty may encounter situations that require them to engage in the ethical decision
making, the ethical nature of these processes was not the focus of our planned
faculty development or subsequent inquiries (Campbell, 2008). Despite the absence
of matters of moral and ethical considerations within this context, medical school
faculty are uniquely familiar with numerous ethical issues such as confidentiality,
beneficence, and non-maleficence and thus we did not deem it necessary to reiterate
these principles here. In addition, it is not routine to include such fundamental
principles into faculty development or other pedagological exercises in these envi-
ronments. However, ethical concerns permeated many discussions and specific
issues that arose are discussed below which may need to be part of future faculty
Table 1. Year 1 socio-cultural curriculum artifacts.
Artifact Topic/area Assessment modality
Student Identity Reflective
Exercise (Essay)
Self-awareness, personal bias Faculty review of reflection
Socio-cultural Attitudes in
Medicine Inventory (SAMI)a
Attitudes toward socio-cultural
issues in medicine and
patient care
Review of student response
compared to peers
The Sprit Catches You and You
Fall Down (Essay)b
Health care system approach to
cultural differences
Faculty review of essay
Ms Graupera Social Historyc Case presentation on
Immigrate health, social
history taking skills
No assessment, mandatory
small group participation
and discussion
Tuskegee Syphilis Experiment
Lecture
Historical approach to
disparities & discrimination
Mandatory interactive
lecture with audience
response
End-of-Year Reflective
Exercise (Essay)
Reflective essay covering
personal growth and
development in the area of
socio-cultural medicine
Faculty summative
assessment of reflection
Sources: a
Tang et al. (2003); b
Fadiman (1997); c
Lypson, Perlman, Lash, Silveria, & Johnson (2009).

development efforts. As we report, these considerations must be included in future
work in this area.
Existing literature in this area has highlighted the utility of eportfolios in medical
education; however, few have investigated the perceptions of medical educators on
the implementation of this technology (Liaison Committee on Medical Education,
2008). Therefore, the purpose of this study was to explore the experiences and
attitudes of medical school faculty using an eportfolio for the first time and their
perceptions on assessing students’ reflective essays via an eportfolio.
Methods
At the end of their first year, students were required to write a 500–1500 word reflec-
tive essay expressing how they incorporated the socio-cultural learning objectives into
their concept of medicine. The eportfolio allowed faculty to provide written feedback
on student performance. Each faculty member was assigned 12 to 14 students and
given one month to read and evaluate the year-end summative essay.
Participants
The final purposeful sample included all 12 faculty members and one faculty director
who served as faculty mentors for the eportfolio. Faculty were recruited to become
mentors by one of the authors (MLL) based upon pre-existing knowledge of their
community service (e.g. volunteering at free clinics, providing care to the under-
served, etc.); institutional stewardship and research expertise around issues of health
care disparities, cultural competency or diversity; or prior teaching experience and
expertise in areas pertinent to socio-cultural medical education. All were practicing
physicians, except one medical educator with a doctorate in Education. The faculty
represented various departments: Internal Medicine (7), Education (1), Pediatrics and
Communicable Disease (3), Physical Medicine and Rehabilitation (1) and Surgery (1)
(Table 2). Most participants were assistant professors; several also had administrative
roles, such as clerkship director, residency program director, and assistant dean.
Faculty participated in two 2.5-hour training sessions and one faculty feedback
session designed to develop the skills paramount to the successful navigation of the
SePAT and delivery of feedback on student essays. The software design team also
participated in the training sessions to ensure faculty questions and potential software
problems were addressed. Time was allotted in these sessions for faculty to develop
and calibrate essay evaluation criteria, which helped to ensure the tool met curricular
objectives.
Data collection
After the SePAT was completed, we conducted focus groups and interviews of the
eportfolio faculty to better understand their expectations and experiences in this
project (Kuper, Martimianakis, McNaughton, Albert, & Hodges, 2010). One of the
authors (PTR) moderated the focus groups using exploratory, open-ended questions to
elicit details about navigating within the eportfolio system, the sufficiency of the train-
ing, and the utility of the program with regard to the students’ learning experience. All
faculty agreed to participate in the focus group; however, two were unable to attend
and were interviewed separately. The focus group lasted approximately 2.5 hours and

the individual interviews lasted 45 minutes to one hour. All data was collected elec-
tronically and transcribed verbatim. This study received exemption status from the
Medical School’s Institutional Review Board.
Data analysis
The focus group and two interview transcripts were analyzed using grounded theory
analysis techniques (Glaser & Strauss, 1967). Each investigator (RLP, PTR, JGC,
MLL) engaged in open coding by independently reviewing the transcripts and extract-
ing common themes. Investigators then met to review these common themes and
subsequently engaged in axial coding to identify relationships and patterns among the
common themes. Finally, textual descriptions were made of each theme pertinent to
the phenomenon under investigation – faculty perceptions and experiences with the
eportfolio assessment tool. All investigators met, and through this iterative process
validated final themes against transcripts and reached consensus.
Results
The thematic analysis of the focus group and interview transcripts revealed four major
themes highlighting participants’ experiences and perceptions using the eportfolio
(Table 3). Each participant was assigned a pseudonym for this study.
Faculty as novices
In many settings faculty are considered experts in their clinical, research and admin-
istrative roles; however, when they began the SePAT, they expressed having a sense
of role reversal that caused some discomfort. Many faculty expressed feelings of
unease, inexperience and uncertainty in grading students’ reflections.
Table 2. Eportfolio faculty characteristics.
Pseudonym Department Sub-specialty
Dr Dillard Internal Medicine General Medicine
Dr Owens Surgery Colorectal Surgery
Dr Robbins Internal Medicine General Medicine
Dr Wynn Pediatrics and Communicable
Disease
General Pediatrics
Dr Evans Pediatrics and Communicable
Disease
General Pediatrics
Dr Nielson Internal Medicine Infectious Disease
Dr Pugh Pediatrics and Communicable
Disease
Adolescent Medicine
Dr Hall Internal Medicine Pulmonology
Dr Brown Medical Education Higher Education
Dr Matthew Physical Medicine and
Rehabilitation
General Physical Medicine and
Rehabilitation
Dr Guthrie Internal Medicine Gastroenterology
Dr Adams Internal Medicine Nephrology
Dr Martin Internal Medicine General Medicine

Several faculty members acknowledged that they were assuming a role, providing
feedback to students on their writing, that they were not completely comfortable with.
Dr Hall commented:
I realized I did much better with the last 10 than I did than I did the first two. I think it
was just because I wasn’t sure what to expect and what to see the first time.
Similarly, faculty members recognized their inexperience, which caused many to
experience discomfort with providing feedback on students’ written reflections.
Dr Adams described:
I give my learners in other situations feedback on their medical performance but I
haven’t read student essays and given feedback on them in a long time. So I didn’t feel
like I was drawing on a skill that was totally in my comfort zone.
Another participant noted:
It took me like an hour for the very first one to even figure out how do I respond to some-
thing like this, without sort of minimizing the effort they [student] put into it. (Dr Dillard)
Many faculty also noted the subjectivity in grading reflective essays and, as such,
Dr Brown stated:
We all would read an essay [during the training session] and somebody would say that’s
really good and someone else would say well no, it doesn’t have this and it doesn’t have
that.
In addition:
… it is actually very hard to give people good feedback on their writing especially when
they’re writing about a personal experience, so I think having the rubric or some sort of
touchstones or standards to refer to is very useful. (Dr Adams)
Faculty struggled with the best way of providing useful feedback to students. One
mentor stated:
You really want to give them good feedback. They are making such an effort, really
reflecting on their entire year, so you don’t just want to say something [simple and easy].
(Dr Guthrie)
Table 3. Themes and descriptions.
Theme Description
Faculty as Novices Faculty perceived their evaluation skills at a novice level of
expertise.
Reflection on the Process Faculty reflected on numerous steps involved in completing the
evaluation and providing feedback to students.
Faculty Perceptions of
Student Essay Quality
Faculty recognized the varying quality of student essays and the
different approaches required as a result.
Utility of an ePortfolio
Assessment Tool
Faculty expressed their perspectives on the usefulness of the
eportfolio as an assessment tool.

These sentiments illustrate the differences clinical educators noticed between provid-
ing feedback on students’ reflections and feedback on clinical skills. Most of the
faculty were able to resolve their novice feelings as they gained experience and confi-
dence with the process. The use of the rubric also provided faculty with a foundation
from which to compare essay content, which helped to minimize the necessity to make
subjective judgments on students’ work.
Reflection on the process
In order to successfully provide feedback to stimulate further reflection, faculty under-
went several sessions learning to use the eportfolio and provide feedback on written
reflections. As it was their first experience using an electronic portfolio as an assess-
ment tool, participants shared that the workshop helped them gain familiarity with the
system prior to actually using it. They also expressed that these sessions gave them a
certain level of confidence to perform their new role.
Faculty described these sessions as important components of preparing for their
new role. Dr Adams commented that:
We had two training sessions and I went to both of them and yes, it really helped. From
both of what we were trying to accomplish and … also the practical [experience] of
literately how do you log on and how do you access it.
Another faculty member noted:
[thanks to the training sessions] I kind of bought into it. I had a deeper understanding of
the criteria than somebody saying this is the eight criteria we are going to use.
(Dr Robbins)
Several faculty commented on the amount of time they spent reviewing the essays:
‘Some of [the essays] you could do in 10 minutes, some of them took 45 minutes to
an hour’. Dr Martin stated that, ‘I personally was shocked about how long it took’.
Dr Dillard was then able to reach a resolution:
[the essays] seemed like such a personal effort by them, that I wanted to make my
response to them very personal. And I wasn’t sure if that was OK. I ended up … writing
them all basically letters. Like they wrote me a letter and I wrote them back a letter is
how I approached it.
Faculty were extended a significant amount of autonomy during the evaluation of
student essays, and as such were also given the option of having students re-do poor
quality essays. Even after identifying poor quality essays, faculty wrestled with asking
students to revise their work, eventually using their own judgment on an appropriate
course of action. Dr Davis commented:
I made at least one, maybe two of my students re-write their essays. I’m not sure if
that was good or bad. And the second essays were much better. When I wrote him
back I said look dude you mean to tell me you’ve had no change in your life in 12
months? He wrote about all these things, he gave me some stuff about his past and
wow, you came in with so many tools and you’ve not progressed, just breathing and
living and being a human being. And he really wrote a very apologetic, much more
thoughtful essay.

The faculty mentors appeared sincerely interested in the process of feedback and
expressed interest in receiving feedback on their feedback. As Dr Robbins stated:
I don’t have a sense of what [the faculty director] thought of my feedback or what the
students thought of my feedback – whether it was useful, whether it was accurate, or was
it enough … I don’t know if my feedback was useful to the students or useful to the
curriculum.
Faculty viewed their time on this novel educational modality as a responsibility in
addition to their existing teaching commitments. None of the faculty expressed feel-
ings that their time had been wasted, or in retrospect wished they had not participated.
Through this process faculty gained insight on knowing when to apply pressure and
when to accept what had been written. Further, through this process, faculty also
began to recognize and accept the diversity inherent in providing individual feedback.
They found that it was acceptable to provide different types of feedback to different
students, and that one universal standard was not required.
Faculty perceptions of student essays
Several instructors noticed that they needed to confront their biases or feelings about
students to fairly evaluate the students’ work. Faculty recognized that their role as
mentors involved ushering students into the field of medicine, despite students’
complaints. Dr Evans shared that:
All he said was oh, this is a stupid year and I didn’t learn anything and I’m no different
than I was, whatever. I said, oh, really… either you are a pathetic human being who is
not growing or else tell me how you’re growing through some contact. Whether you
hated this program or not tell me what’s going on in your life. You’re going to be a
doctor and you’re telling me you’re no more mature, no more sophisticated with inter-
action than you were 12 months ago.
Dr Hall noted:
Not all of them took it with the same seriousness or understood the ramifications. It’s
really easy to write something [without much thought and] every once in a while … I
found myself saying I know these kids are too smart for this, they just didn’t give it any
time because they just had so much to do.
As faculty reflected on the quality of the student essays, several questioned their
validity and whether students were simply writing what they believed would provide
a positive evaluation. One student later revealed: ‘I didn’t actually feel like I was able
to be honest … [in case the essay was read by] someone possibly important in my
future or something like that’. Despite this, a faculty mentor noted:
One thing I liked … you see a sense of honesty, and it’s very clear as you read these
essays [that the students] don’t care about the outcome. (Dr Guthrie)
Utility of an eportfolio assessment tool
The final prominent theme to emerge was faculty’s belief that the eportfolio was a
valuable addition to the curriculum. Not only did this system provide a means for

students to revisit previous assignments, but it also supported the overall aims and
goals of the medical school. One noted benefit was that it provided an additional
mechanism through which faculty could communicate with students about their work.
As Dr Martin stated:
I’m not sure they actually read the stuff that was there [as part of the multicultural curric-
ulum, when it was first assigned], but it’s very clear that it was a eureka moment. ‘Oh,
wow… I didn’t think I’d ever have to see this again’…but they had gone back to [it].
Dr Hall noticed that reading the essays provided new insight into medical students. He
stated:
It makes me even more cognizant of how they may be seeing what I consider a very stan-
dard, classic situation with a family or a patient that because of the complexity of every-
thing else floating around they might not see it. So it made me [think], well let’s walk
through the psycho-social portion of what we’re seeing as opposed to the blood gas.
The insight gained from reading essays gave Dr Hall a better understanding of the
students’ perspectives on a clinical rotation and helped him be more aware of their
learning needs in the in-patient setting.
The eportfolio assignment was also perceived as important to the medical school’s
educational mission:
[What] is really important is the reflection part. I think this ability to self-reflect makes
them cognizant of how they act, how they feel, how what they do impacts the situation
they’re in. Whether you’re having a good day or a bad day, how does that impact
the patient you’re taking care of … It makes them reflect on their place in the equation.
(Dr Hall)
Faculty suggested that the unique opportunity to review their previous work, including
previously assigned readings and their own writing encouraged students’ reflection.
This occurred by providing students with the opportunity to make connections
between various educational experiences:
I would like to see the students connecting the dots as well … the advantage of the
portfolio is we can put that all out there for them. We don’t have to ask them to keep a
paper binder or whatever, here it all is and I think that’s a huge advantage as long as the
technology can keep up. (Dr Brown)
Faculty also acknowledged the value of developing personal teaching skills and were
willing to participate in training to improve in this area. As one said about the work-
shops, needing to:
… be thoughtful about it and step back and think about it and give [the students] some-
thing concrete or something reflective to think about I think was a great exercise. Being
in a situation where you’re talking to students or giving them some type of feedback I
think practicing and doing that was really helpful. (Dr. Brown)
Finally, as an eportiolio may also serve as a useful tool for gaining access to impor-
tant, yet often missed opportunities for intervention, there are unintended conse-
quences of portfolio learning. After reviewing student essays, several faculty were
compelled to contact the course director as students’ submissions raised their concerns

regarding students’ emotional stability. In these cases, students were referred to the
medical school counseling team for further evaluation and treatment. While outside
the boundaries of education and scope of the assignment, students were grateful for
the faculty intervention, which would have otherwise been missed. Again, the ability
to recognize the signs of depression and social isolation were not subjects matters
raised during the training sessions, but are skills medical faculty have been profession-
ally trained to recognize. In these situations, attention to the student’s well-being
superseded the assignment.
Discussion
Although providing feedback on a learner’s thoughts or reflective process is not an
intuitive skill for many educators, this study demonstrated that given sufficient time
and training, faculty are able and willing to use an eportfolio in the context of a socio-
cultural curriculum to fulfill this task. Our findings parallel existing literature demon-
strating that portfolios can be a valuable tool in developing and assessing student
reflection (Driessen et al., 2005); in addition, we also determined that eportfolios can
serve as a tool for assessing student learning and ensuring curricular objectives are
achieved. Perhaps most importantly, this study emphasized the importance of faculty
training in areas in which they may lack expertise and the paradox of using a new
methodology with novice faculty to assess complex issues such as socio-cultural
medicine. As outlined in our study, faculty obtained sufficient experience to provide
effective feedback through interactive workshops with relevant exercises (Brukner,
Altkorn, Cook, Quinn, & McNabb, 1999).
Not all aspects of a portfolio were addressed during the faculty development work-
shops. Although we recognized that the learning process may as Ghaye notes,
‘contain, reframe and transform emotion’, students’ reflections may similarly
stimulate emotions which could subsequently impact faculty assessment (Ghaye,
2007). We did not specifically provide instruction on how faculty should address such
matters during the faculty development sessions. Several of the faculty commented on
the personal nature of the essays they were evaluating and the ways in which they
were challenged to address them. This concern has been noted as part of the inherent
tensions a reflective portfolio – specifically the tension between writing a personal and
a safe response and the tension between developing your own point of view and being
open to other points of view (Ghaye, 2007). Faculty modeling through their own
reflections may be a useful and ethical way to help students learn to address these
tensions. If faculty were to role-model their own reflections and share with students
their own vulnerabilities, that might serve as a template for students to pursue their
own reflections, especially in areas filled with controversy.
As noted above, two faculty members noticed, in reading the reflective essays,
clues that the student author was suffering from depression. This raised the issue of
student privacy and autonomy versus the administrative need to protect both the
individual student and the institution from harm. Each of these cases was discussed
first with the course director and then brought to the attention of the student counse-
lors, who operate under the Office of Student Affairs and follow each class of medi-
cal students longitudinally. Of note, both students are currently doing well in
medical school. Later, each student independently expressed appreciation to the
faculty and administration for the personal and concerned response he or she
received.

All medical students were required to, and successfully completed, a reflective
essay in order to move on to the second year of medical school. It is worth explicitly
noting the inherent power differential in the student-teacher relationship, which may
cause students to censor what they chose to share in an open essay (Colnerud, 1997).
However, the high-stakes nature of a graded essay may be somewhat tempered by the
fact that the University of Michigan Medical School operates on a pass/fail basis
during the first two years of study. Thus, concern over achieving an honors mark
likely would not drive the student essays.
The faculty who participated in the SePAT believed this experience positively
influenced their role of clinician educator. Nonetheless, teaching is a complicated
process and educators value and respond to affirmation (Palmer, 1998). The SePAT
workshops gave faculty educators the opportunity to explore a new aspect of medical
student education, and receive support from a cohort of faculty who similarly dedi-
cated time to focus on educational technique. Other research has found that small
group teaching or focused faculty development programs cannot only enhance their
satisfaction and motivation but also contribute to their own personal and professional
development (Stenfors-Hayes et al., 2010). This review of the reflective essays was
the first in a series of longitudinal exercises to assess student performance. As we
enter into the second year of the assessment of reflective essays, it is still unclear what
exactly faculty will expect to see or define as progressive development in this area.
Finally, our findings suggest that given the proper environment (e.g. support for
teaching activities, release time, etc.), faculty are indeed willing to invest significant
uncompensated teaching time and effort into educational endeavors that they believe
in. Furthermore, faculty willingness and participation is essential to providing effec-
tive mentorship to students (Brukner et al., 1999). This is an important consideration
since faculty time must be accounted for and ideally a mission-based budget would be
available for faculty compensation (Cooke, Irby, & O’Brien, 2010).
Limitations
In this context we did not explore the issue of responsibility as we had novice faculty
members review reflective exercises. One could question whether it was appropriate
or ethical to have novice faculty review potentially emotional and personal reflective
essays and whether we have conducted ourselves as responsible educators? It is
unclear whether we should have spent more time on faculty development activities to
ensure they had more experience in this area than the trainees they were assessing.
The pass/fail nature of the course, however, does not easily account for the other
issues noted in our findings. Throughout our discussions with faculty they continu-
ously noted the potential biases and misconceptions they bring to bear when evaluat-
ing the student’s reflective exercises. During the faculty development workshops,
consensus was reached regarding what an acceptable reflection would look like
However, in this domain of cultural competency or socio-cultural medicine, it is not
clear that there is a universal answer. We teach students that the patients’ perspective
at anytime or moment is the ‘appropriate approach’; however, in our work here we
assumed something quite different. We make the clear assumption that the ‘correct’
behavior or affective domain to master is one of tolerance, cultural acceptance and
personal humility. It might be that in another context this would not be the correct
answer. Many essays did discuss the difficulty students have with learning about and
dealing with patients and colleagues from distinctly different cultures. Faculty were

instructed that it was acceptable for a student to ‘disagree’ with this framework,
however, they had to posit alternative means of understanding and care. In this we
consciously made a choice about what was a ‘good’ reflection and what was a ‘bad’
reflection, noting this was reinforcing many of our own bias as faculty with expertise
in some area of socio-cuturalism in the United States.
Conclusions
In summary, our findings highlight unique and often overlooked components of imple-
mentation of assessment portfolios – faculty perspectives on the utility of eportfolios,
their ability to use this technology, and the importance of providing sufficient time and
training. Despite the numerous, useful guidelines for the general process of providing
feedback in medical education, faculty often seek specific strategies to aid them in this
task (Ende, 1983). Unanticipated ethical dilemmas did arise as faculty evaluated
student essays – these manifested amid decisions of whether to disclose personal infor-
mation revealed by a troubled student and when to make students remediate their work.
Physicians and medical educators are perhaps more likely to think of morality and
ethics as it relates to patient care and medical ethics. It is worth remembering that medi-
cal educators also need to meet the ethical demands placed on teachers.
Faculty who volunteered their time found the experience valuable, particularly in
their role as clinical educators. Our cohort of faculty was sufficiently satisfied with
their participation in the initial SePAT that each of the 13 members agreed to partici-
pate again in the upcoming academic year.
Notes on contributors
Rachel L. Perlman, MD, is chief of Nephrology, Ann Arbor Veterans Affairs Medical Center,
assistant professor, Department of Internal Medicine, University of Michigan Medical School,
Ann Arbor, Michigan.
Paula T. Ross, MA, is research area specialist, Department of Health Behavior and Health
Education, University of Michigan School of Public Health, Ann Arbor, Michigan.
Jennifer Christner, MD, is director of Pediatric Medical Student Education, assistant dean for
Longitudinal Learning and Assessment, and assistant professor, Department of Pediatrics and
Communicable Diseases.
Monica L. Lypson, MD, is associate professor, Department of Internal Medicine and Medical
Education, and also assistant dean for Graduate Medical Education, University of Michigan
Medical School. In addition, she is a staff pysician, Ann Arbor Veterans Affairs Medical
Center, Ann Arbor, Michigan.
References
Austin, C., & Braidman, I. (2008). Support for portfolio in the initial years of the undergraduate
medical school curriculum: What do the tutors think? Medical Teacher, 30(3), 265–271.
Beach, M.C., Prince, E.G., Gary, T.L., Robinson, K.A., Gozu, A., Palacio, A. et al. (2005).
Cultural competence: A systematic review of health care provider educational interven-
tions. Medical Care, 43(4), 356–373.
Brukner, H., Altkorn, D.L., Cook, S., Quinn, M.T., & McNabb, W.L. (1999). Giving effective
feedback to medical students: A workshop for faculty and house staff. Medical Teacher,
21(2), 161–165.

Campbell, E. (2008). The ethics of teaching as a moral profession. Curriculum Inquiry,
38(4), 357–385.
Carraccio, C., & Englander, R. (2004). Evaluating competence using a portfolio: A literature
review and web-based application to the ACGME competencies. Teaching and Learning
in Medicine, 16(4), 381–387.
Challis, M. (1999). Portfolio-based learning and assessment in medical education. Medical
Teacher, 21(4), 370–386.
Chism, N. (2004). Using a framework to engage faculty in instructional technologies. A
conceptual framework enables faculty development planners to better estimate the poten-
tial effectiveness of various strategies. Educause Quarterly, 30(2), 39–45.
Colnerud, G. (1997). Ethical conflicts in teaching. Teaching and Teacher Education, 13(6),
627–635.
Cooke, M., Irby, D.M., & O’Brien, B.C. (2010). Educating physicians: A call for reform of
medical school residency. Stanford, CA: Jossey-Bass.
Dannefer, E.F., & Henson, L.C. (2007). The portfolio approach to competency-based assess-
ment at the Cleveland Clinic Lerner College of Medicine. Academic Medicine, 82(5),
493–502.
Dekker, H., Driessen, E., Braak, E. Ter Scheele, F., Slaets, J., Van der Molen, T., et al.
(2009). Mentoring portfolio use in undergraduate and postgraduate medical education.
Medical Teacher, 31(10), 903–909.
Driessen,E.W.,Overeem,K.,vanTartwijk,J.,vanderVleuten,C.,&Muijtjens,A.(2006).Valid-
ity of portfolio assessment: Which qualifies determine ratings? Medical Education, 40(9),
862–866.
Driessen, E.W., van Tartwijk, J., Overeem, K., Vermunt, J.D., & van der Vleuten, C. (2005).
Conditions for successful reflective use of portfolios in undergraduate medical education.
Medical Education, 39, 1230–1235.
Ende, J. (1983). Feedback in clinical medical education. Journal of American Medical Associ-
ation, 250, 777–781.
Fadiman, A. (1997). The spirit catches you and you fall down: A Hmong child, her American
doctors, and the collision of two cultures. New York: Noonday Press.
Gans, R.O. (2009). Mentoring with a formative portfolio: A case for reflection as a separate
competency role. Medical Teacher, 31(10), 883–884.
Ghaye, T. (2007). Is reflective practice ethical? (The case of the reflective portfolio). Reflec-
tive Practice, 8(2), 151–162.
Glaser, B.G., & Strauss, A.L. (1967). The discovery of grounded theory: Strategies of quali-
ties research. New York: Aldine.
Gruppen, L.D., Frohna, A.Z., Anderson, R.M., & Lowe, K.D. (2003). Faculty development
for educational leadership and scholarship. Academic Medicine, 78, 137–141.
Kolb, D.A. (1984). Experiential learning: Experiences as the source of learning and develop-
ment. Englewood Cliffs, NJ: Prentice Hall.
Kumagai, A.K., & Lypson, M.L. (2009). Beyond cultural competence: Critical consciousness,
social justice and multicultural education. Academic Medicine, 84(6), 782–787.
Kuper, A., Martimianakis, M.A., McNaughton, N., Albert, M., & Hodges, B. (2010). Asking
new question with qualitative research: A reflection on AMEE 2009. Medical Teacher,
31, 1–2.
Liaison Committee on Medical Education, (2008). Structure for Accreditation of Medical
Education programs leading to the MD Degree. Functions and structure of a medical
school. Washington DC: Liaison Committee on Medical Education.
Lypson, M.L., Ross, P.T., & Kumagai, A.K. (2008). Medical students’ perspectives on a multi-
cultural curriculum. Journal of the National Medical Association, 100(9), 1078–1083.
Lypson, M.L., Perlman, R.L., Lash, R., Silveria, M., & Johnson, C. (2009). The social history
… it’s about the patient – culture and all. Retrieved from http://services.aamc.org/30/
mededportal/servlet/s/segment/mededportal/?subid=3149.
McMullan, M., Endacott, R., Gray, M.A., Jasper, M., Miller, C.M., Scholes, J. et al. (2003).
Portfolios and assessment of competence: A review of the literature. Journal of Advanced
Nursing, 41(3), 283–294.
Palmer, P.J. (1998). The courage to teach. San Francisco: Jossey-Bass.

Rees, C.E., & Sheard, C.E. (2004). The reliability of assessment criteria for undergraduate
medical students’ communication skills portfolios: The Nottingham experience. Medical
Education, 38, 138–144.
Sandars, J. (2009). The use of reflection in medical education: AMEE Guide No. 44. Medical
Teacher, 31(8), 685–695.
Stenfors-Hayes, T., Kalen, S., Hult, H., Dahlgren, L.O., Hindbeck, H., & Ponzer, S. (2010).
Being a mentor for undergraduate medical students enhances personal and professional
development. Medical Teacher, 32, 148–153.
Swanson, K.W. (2010). Constructing a learning portfolio in transformative teacher develop-
ment. Reflective Practice, 11(2), 259–269.
Tang, T.S., Fantone, J.C., Bozynski, M.E., & Adams, B.S. (2003). Implementation and evalu-
ation of an undergraduate sociocultural medicine program. Academic Medicine, 77(6),
587–585.
Wald, H., Davis, S., Reis, S., Monroe, A., & Borkan, J. (2009). Reflecting on reflections:
Enhancement of medical education curriculum with structured field notes and guided
feedback. Academic Medicine, 84(7), 830–837.

Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAssessmentTool.ReflectivePractice.2011

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (11)

Similar to Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAssessmentTool.ReflectivePractice.2011

Similar to Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAssessmentTool.ReflectivePractice.2011 (20)

Christner.FacultyReflectionsOnTheImplementationOfSocio-CulturalEportfolioAssessmentTool.ReflectivePractice.2011