This document explores the assessment of professionalism in healthcare professions. It begins with an icebreaker exercise and then defines professionalism as qualities characteristic of a profession including skill, knowledge, and work practices. The document outlines several domains of professionalism including patient care, medical knowledge, and ethics. It notes professionalism is complex to define and assess given it involves various attributes, behaviors, and attitudes that depend on context. Several tools for assessment are discussed, including written assessments, competency-based approaches like OSCEs, and performance-based methods like multi-source feedback. The document stresses the need for a validated definition of professionalism to guide assessment and emphasizes triangulation of multiple assessment methods over time for reliable and valid evaluation

2. Assessment of Professionalism
Dr Dalal ALQahtani
Session starts at 1.00 P.M.

3. Individual Expectations

4. Icebreaker Exercise
What constitutes having/being
professional?
One word

6. Aim
Explore the features of professionalism
and its assessment in health care
profession

7. Objectives
1. Define professionalism and its domains
2. Recognize the trouble with assessing
Professionalism
3. List and explain some tools used in
professionalism assessment
4. Articulate the reasons behind the need of
assessing professionalism

8. Professionalism is generally defined as the
body of qualities or features characteristic
of a profession
Oxford English Dictionaries, 1993

9. These qualities include a high degree of skill
and knowledge that is applied to the practice
of work

10. The Accreditation Council on Graduate Medical Education; 2009

11. Patient Care
Systems-based Medical
Practice Knowledge
Practice-based
Learning and
Professionalism Improvement
Interpersonal and
Communication
Skills

12. Demonstrate respect, compassion, and integrity; a
responsiveness to the needs of patients and society
that supersedes self-interest; accountability to
patients, society, and the profession; and a
commitment to excellence and on-going professional
development.

13. Demonstrate a commitment to ethical principles
pertaining to provision or withholding of clinical
care, confidentiality of patient information, informed
consent, and business practices

14. Demonstrate sensitivity and responsiveness to a
diverse patient population, including, but not limited
to diversity in gender, age, culture, race, religion, and
disabilities

15. American Board of Internal Medicine. American professionalismin medicine: issues and
opportunities. Definition and objectives.Available at: http://www.abim.org/pubs/p2/definitn.htm.
AccessedOctober 15, 2003.

16. It is very important competence
That means we need to
Teach and Assess

18. Complex construct
Second order competence
Context variable
Attitude and behavior

19. Complex construct
Attributes
How to measure ?

20. Complex construct
Attributes
How to measure ?

21. Complex construct
It encompasses a variety of
skills, knowledge and other attributes and
have complex relationships to observed
behavior

22. The Elements of Professionalism
By the American Board of Internal Medicine
But are these the only attributes of
professionalism?
NO

23. The Authors’ Classification of Themes
and Subthemes, Arising From Definitions
or Interpretations of Professionalism
Wilkinson et al;2009

24. So… We need to
Understand the various dimensions/attributes
of professionalism
Because

25. Without a validated definition of this construct, assessment of
professionalism within medical education will be compromised
It guides the selection of the educational material and provides the
criteria for any assessment system

26. Complex construct
Second order competence

27. Second order competence
‘can be expressed only via the performance of other
competences’
Verkerk MA et al;2007

28. Second order competence

29. Complex construct
Second order competence
Context variable

30. Context variable
Varies between contexts and individuals and
includes both normative and ideological aspects
It should reflect both local and national contexts
van Mook WNKA et al; 2009

31. Context variable
Professionalism, and the literature supporting it to
date, has arisen from Anglo-Saxon countries.
Caution should be used when transferring ideas to
other contexts and cultures.
International Ottawa Conference (Miami 2010)

32. Context variable
Professionalism is intrinsically related to the social
responsibility of the medical profession.
Thus, definition should reflect societal and health
care changes, and this is an important responsibility
of the profession and its educational institutions to
the public. International Ottawa Conference (Miami 2010)

33. Context variable

34. Context variable
A cross-sectional design
The findings point to a low perception among the
participating nurses regarding their professionalism

35. Context variable
Workplace
Personal background of the nurses, which includes
the personal interest in the nursing profession
Family
Society

36. 10 Focus group
What do you mean by professionalism?
To what extent have you developed these
qualities?

37.  Admitted were deficient in the acquisition of
professional values
 A Professionalism not taught or assessed
 Very few teachers as positive role models

38. WHY ?
Negative role modeling by the faculty
Deficiencies in the curriculum
Limited interaction with health team
Absence of feedback

39. Students Quote
I think 40–60% of the students enter medical college with
good professional qualities . . . unfortunately, they gradually
lose their qualities because of the influence of teachers and
senior students . . . they end up in behaving badly and
aggressively . . . (FG-05)

40. Students Quote
Feedback is totally missing in the system . . . Either from
teacher to the student . . . or from student to the teacher . . .
even when feedback is given to the faculty, they may not
implement, . . . they may even victimize the students . . . it
may be a good idea to take feedback anonymously . . . (FG-
10)

41. Context variable
Ginsburg et al conducted interviews with 30
clinicians after the clinicians had watched five
videotaped scenarios of professionally
challenging situations
They were asked what they thought students
should and should not do in these
situations, and they were also asked what
they would do themselves !
The authors found little agreement between clinicians. Ethical principles such as
honesty were defined differently across clinicians and within clinicians across
different scenarios
Suggesting that dishonest behavior could be interpreted as unprofessional or not
unprofessional depending on the context
Ginsburg S et al; 2004

42. Context variable
Ginsburg et al conducted interviews with 30
clinicians after the clinicians had watched five
Society
videotaped scenarios of professionally
challenging situations
They were asked what they thought students
Between individuals should and should not do in these
situations, and they were also asked what
they would do themselves !
Within individual
The authors found little agreement between clinicians. Ethical principles such as
honesty were defined differently across clinicians and within clinicians across
different scenarios
Suggesting that dishonest behavior could be interpreted as unprofessional or not
unprofessional depending on the context
Ginsburg S et al; 2004

43. Complex construct
Second order competence
Context variable
Attitude and behavior

44. Professionalism
Attitude Behavior
Mismatch

45. Mismatch
Attitude Behavior
Unprofessional attitude professional behavior
Faking
Professional attitude Unprofessional behavior
Social pressure to behave in a particular way

46. Mismatch
Attitude Behavior
As a result, we must be carful of making
assumptions about students’ professionalism on
the basis of observed behavior alone

47. Mismatch
Attitude Behavior
Therefore , when devising methods of
assessment; knowledge of the reasoning behind
the action is also required
Attitude

48. An attitude can be defined as “a favorable or
unfavorable evaluate reaction toward
something or someone, exhibited in one’s
beliefs, feelings, or intended behavior
Myers DG. Social Psychology. Boston: McGraw-
Hill College; 1999.

50. Summative assessment
Formative assessment
Conversation help students to reflect
critically on their behaviors
Help students to develop their professional
behaviors in the future

51. Mismatch
Attitude Behavior
Cognitive dissonance theory suggests
“when we behave in a manner contrary to our
attitudes, we experience dissonance, which we
seek to reduce, either by changing our attitudes
to match our behaviors or vice versa.”
Festinger L. A Theory of Cognitive Dissonance. Stanford, CA: StanfordUniversity Press;
1957.

52. Complex construct
Second order competence
Context variable
Attitude and behavior
HOW?

53. Both are important

54. Professionalism assessment
Written assessment
Competency-based assessment
Performance-based assessment
Portfolio

55. Written assessment
Selected • MCQ
response • Questionnaire
Constructed • Essays
response • Short answer questions

56. Written assessment
Ideal to assess
• Knowledge of the judicial, legislative and
administrative processes and ethical principle
• Reflective ability of medical students and junior
doctors

57. Written assessment
Questionnaires
Questionnaires are often based on vignettes or clinical
scenarios and may involve the description of critical
incidents
They show validity and reliability
Boenink AD et al; 2005

58. Written assessment
Critical incident report
This method asks the doctor to reflect on a
critical incident he or she has experienced or
witnessed

59. Written assessment
Critical incident report
• It can encourage reflection and attention to
elements of professionalism
• But it is dependent on the type of incident to
determine which aspect of professionalism is
being assessed

60. Competency-based assessment
It take place in controlled representations of
professional practice such as standard patient
encounters and objective structured, clinical
examinations (OSCEs)

61. Competency-based assessment
Useful component of a systematic approach to
assessing professionalism, especially in the earlier
stages of the curriculum

62. Competency-based assessment
These have the advantage over written assessment in
that they can be used to assess the ‘Showshow’ level
of Miller’s pyramid

63. Competency-based assessment
OSCE
OSCEs are also seen to be fair, as each student carries
out a standardized procedure during the assessment
and have high degrees of reliability because each
student is assessed by many different examiners and
several cases

64. Competency-based assessment
OSCE
BUT
• An artificial environment and therefore may
not reflect actual day-to-day clinical performance
•OSCEs are complex to organize

65. Competency-based assessment
OSCE
BUT
There are problems with the interpretation of student
behaviors by differing assessors, even when calibrated
and well trained
Mazor KM;2007

66. Performance-based assessment
Are those that take place within the natural clinical
setting and include work-based systems with direct
observation of the student

67. Performance-based assessment
• It measures the upper end, i.e. the ‘show’s how’ or even
‘does’, of Miller’s pyramid
•Authentic
•It enable professionalism to be assessed as a second-
order competence

68. Performance-based assessment
BUT
It is time-consuming and requires well-trained
observers and accurate criteria to work well

69. Performance-based assessment
360 feedback
Also known as multi-rater feedback or multi source
feedback (MSF)
Assessment by faculty, nursing or other members of staff
and by patients

70. Performance-based assessment
P-MEX
The Professionalism Mini-Evaluation Exercise (P-MEX)
Modifications of mini-CEX

71. This tool is used to assess a 15- to 30-minute observed
snapshot of a doctor/patient interaction that is conducted
within actual patient-care settings using real patients and
that has a structured marking sheet that covers predefined
generic areas

72. Assess four discrete areas : doctor–patient relationship skills
,reflective skills, time management, and interprofessional
relationship skills

73. ACTIVITY
ASSESS PROFESSIONAL BEHAVIOR
USING P-MEX

74. Portfolio
Portfolios provide evidence
of competence and progression
and may stimulate reflection

75. Portfolio
Suitable for both formative and summative
assessment of complex and multifaceted
skills and competences of professionalism

76. Portfolio

77. Wilkinson et al;2009
Review of Tools That Assess Elements of Professionalism

78. Wilkinson et al;2009
Blueprinting
For example :
The blueprint demonstrates that direct
observations (through the mini-CEX
and P-MEX) and collated views
(through MSF and patients’ opinions) are
crucial elements because they capture
many aspects in reliable, valid, and
feasible ways

79. Wilkinson et al;2009
Gaps
Attributes that would not be well assessed using
the current methods

80. 360 FEEDBACK P-MEX
Portfolios Critical incidents
OSCE Physician /patient
assessments

81. Criteria For Evaluation Of Instruments To Assess
Professionalism
What is the sample size, location, and demographics?
How are data recorded?
How is it scored?
Is it reliable?
Is it valid?
Were standards set/classifications made?
What is the feasibility? Length, Cost?
Do the data derived from the tool seem amenable to
change?
Does the tool educate users about the construct being
assessed?
What components of professionalism is this tool
measuring?

82. How should it be?
Less than half the articles retained by Jha et
al. demonstrated reliability or validity.
Many of the problems of reliability derive
from the fact that the assessment tools have
been developed for different purposes and
in different circumstances, perhaps
reflecting the varying contexts of
professionalism !
Jha et al;2007

83. How should it be?
NO single method of assessment has yet
emerged that is reliable and valid !
Triangulation of multiple assessments
by multiple assessors over time

84. Why professionalism assessment is
important?

85. Ronald and Edward;2002

86. Professionalism
Issues

87. Is professionalism a
state or a trait?
1

89. Do you think the results of
professionalism assessment
could change the attitude ?
2

90. Many medical educators hope that by constantly
monitoring students’ professional
behaviors, the students will eventually come to
internalize appropriate attitudes
Charlotte et al;2007
Reflecting on the writings of Immanuel
Kant, Sherman puts it another way:“decorum
can, in some cases, change inner states”
Sherman; 2005

91. What if someone ‘fails’ in
professionalism, what are the
penalties of ‘failing’? What
remedial measures do we have?
Would medical school
fail a student purely
based on professional
issues?
3

92. Other
Issues
Derived from literature

93.  Are the assessment tools looking
for a pattern of behaviors, or a
single behavior or a single
incident?
 Are there tools for students to
evaluate faculty professionalism?
 Is the purpose of the instrument to
identify professionalism or
unprofessional behaviors?
• Are we assessing professionalism
or personality?
• What is the public’s view of
professionalism?

94. Professionalism is a concept that varies across
cultural contexts

95. Elements of professionalism are vast and
include: individual
(attributes, characteristics, attitudes, behaviou
rs, identities), interpersonal (relations, group
dynamics, etc) and societal
(economic, political, etc).

96. There is need to develop concrete and
operationalizable definitions, and from them
effective teaching methods and defensible
assessment approaches are designed

97. A true evaluation of professionalism must
focus on the reasons for a behavior, rather
than just the behavior itself

98. Triangulation of multiple kinds of measures, by
multiple observers, synthesized over time with
data gathered in multiple, complex and
challenging contexts is likely to be appropriate
at all levels of analysis

99. While summative assessment is
important, formative methods should
predominate

100. No single method exists for the reliable and valid
evaluation of professional behavior

101. The overall assessment program is more important than
the individual tools. The best programs use a variety of
tools in a safe climate, provide rich feedback, anonymity
(when appropriate) and follow-up of behaviour change
over time

102. It may be more important to increase the depth and
quality of reliability and validity of a program existing
measures in various contexts than to continue to
develop new measures for single contexts

103. CLOSING AND EVALUATION

104. References
• Shaw D. Ethics, professionalism and fitness to practise: three concepts, not one. Br
Dent J 2009: 207: 59–62.
• Irvine D. The performance of doctors. I: professionalism and self-regulation in a
changing world. BMJ 1997: 314: 1540–1542.
• van Mook WNKA, de Grave WS, Wass V, et al. Professionalism: evolution of the
concept. Eur J Intern Med 2009: 20: e81–e84.
• American Board of Internal Medicine. American professionalismin medicine: issues
and opportunities. Definition and objectives.Available at:
http://www.abim.org/pubs/p2/definitn.htm. AccessedOctober 15, 2003.
• Messick S. The interplay of evidence and consequences in the vali-dation of
performance assessments. Educ Res 1994: 23: 13–23
• Verkerk MA, De Bree MJ, Mourits MJE. Reflective professional-ism: interpreting
CanMEDS’ ‘‘professionalism’’. J Med Ethics
• 2007: 33: 663–666
• Boenink AD, Jonge P, Small K, Oderwald A, Tilburg W. The effects of teaching
medical professionalism by means of vignettes: an exploratory study. Med Teach
2005: 27: 429–432.
• Mazor KM, Zanetti ML, Alper EJ, et al. Assessing professionalism in the context of
an objective structured clinical examination: an in-depth study of the rating
process. Med Educ 2007: 41: 331–340.

105. References
•International Ottawa Conference (Miami 2010) ;Professionalism Theme Working
Group (IOc-PwG) ; Post-Conference Draft Recommendations and Revisions v6
•Wilkinson TJ, Wade WB, Knock LD. A blueprint to assess professionalism: results
of a systematic review. Acad Med. 2009 May;84(5):551–8.
•Zijlstra-Shaw S, Robinson PG, Roberts T. Assessing professionalism within dental
education; the need for a definition. Eur J Dent Educ. 2012 Feb;16(1):e128–136.
•Cruess R, McIlroy JH, Cruess S, Ginsburg S, Steinert Y. The Professionalism Mini-
evaluation Exercise: a preliminary investigation. Acad Med. 2006 Oct;81(10
Suppl):S74–78.
•Rees CE, Knight LV. The trouble with assessing students’ professionalism:
theoretical insights from sociocognitive psychology. Acad Med. 2007
Jan;82(1):46–50.
Ginsburg S, Regehr G, Lingard L. Basing the evaluation of professionalism on
observable behaviors: a cautionary tale. Acad Med. 2004; 79:S1–S4.