Assessment in Dental Education.pdf

Learn
D E N T A L S E R I E S
P R E S E N T S

Y A S S E R A L I A L M O R T A D A A L W A S I F I
B D S , M S C , D D S , P H D
C O P Y R I G H T © 2 0 2 2 , A L W A S I F I , Y . A . A L L R I G H T S R E S E R V E D
ASSESSMENT IN DENTAL EDUCATION A LEARNING DEVELOPMENT TOOL

A S S E S S M E N T I N D E N TA L E D U C AT I O N - A L E A R N I N G D E V E L O P M E N T T O O L - C o p y r i g h t s © 2 0 2 2 , A LW A S I F I , Y. A . 3
- Egyptian dentist
- Researcher, Educator & International Lecturer
- Founder of LEARN - DENTAL SERIES @ YouTube
- Presentation skills coach
- Painter
- Life long learner -

D I S C L A I M E R
S T A T E M E N T I have NO financial disclosure or conflicts of interest with any of the
presented materials.
Yasser Alwasifi

C O P Y R I G H T
Copyright © 2022. All rights reserved. No part of this presentation may
be reproduced, distributed, or transmitted in any form or by any means,
including copying, recording, or other electronic or mechanical methods,
without the prior written permission of the author.
Yasser Alwasifi

E X C E P E C T E D
O U T C O M E S By the end of this workshop, we are expected to share our
cumulative experiences with each other to be able to:
1. Select the ideal assessment tool to identify the student’s strengths
and weaknesses
2. Indicate how and when that tool is used and by whom
3. Apply these tools to support learning and continuous professional
development of the practicing dentist
4. Design assessment blueprint

A S S E S S M E N T I N D E N TA L E D U C AT I O N - A L E A R N I N G D E V E L O P M E N T T O O L - C o p y r i g h t s © 2 0 2 2 , A LW A S I F I , Y. A .
PART I: PRINCIPLES OF ASSESSMENT 10
I.A. INTRODUCTION 11
I.B. PURPOSE OF ASSESSMENT 19
I.C. WHEN TO ASSESS 28
I.D. ASSESSMENT LEVELS 30
I.E. ASSESSMENT TYPES 32
I.F. GOOD ASSESSMENT 36
I.G. ASSESSMENT TASK SETTING 38
I.H. ASSESSMENT PSYCHOMETRICS 59
7
C O N T E N T S
Expected Time: 30 Min

PART II: ASSESSMENT TOOLS 82
II.A. INTRODUCTION 83
II.B. DEVELOPMENT OF COMPETENCIES 86
II.C. ASSESSMENT METHODS 91
II.D. DEVELOPING ASSESSMENT STRATEGY 154
8
C O N T E N T S

PART III: ASSESSMENT BLUEPRINT 157
III.A. WHAT IS ASSESSMENT BLUEPRINT 159
III.B. BENEFITS OF ASSESSMENT BLUEPRINT 162
III.C. HOW TO DESIGN ASSESSMENT BLUEPRINT 164
9
C O N T E N T S

PRINCIPLES OF ASSESSMENT
PART I:

I. PR I N C I P L E S O F
ASSESSMENT
I.A. INTRODUCTION
I.B. PURPOSE OF ASSESSMENT
I.D. ASSESSMENT LEVELS
I.E. ASSESSMENT TYPES
I.G. ASSESSMENT TASK SETTING
I.H. ASSESSMENT PSYCHOMETRICS
I.C. WHEN TO ASSESS
I.F. GOOD ASSESSMENT

H O L D T H I S
T H O U G H T
After Graduation
- Forget what you have studied
- Real practicing is different than academic study

I . A . I N T R O D U C T I O N

47% 32%
https://www.tommiemedia.com
DECEMBER 12, 2013 BY GRACE PASTOOR
Nearly half (47%) of college-
educated workers reported
their first job was not related to
their major
32% of college educated
workers reported they have
never found a job related to
their major
1 in 3 college-educated
workers do not work in
occupations related to their
major

QUALITY

My wife asked me “what is quality?” while she is making me a coffee.. and I replied
• Customer needs
• Effectiveness
• Timeliness
• Safety
• Efficiency
• Competency
• Respect & Care
• Decrease risk
• Customer satisfaction
• Improvement
• Once you asked me how many sugar you need in your coffee
• Once you made it as I requested exactly
• Once you serve it to me and it comes hot
• Once you made it without burning your hands
• Once you made it perfect from the first time
• Once you made it by yourself because you master it
• Once you serve it to me with smile
• Once you carry it and move cautiously
• Once I like its taste
• Every time you made it with better technique and shorter time

COMMUNITY
NEEDS
NQF
PROGRAM
SPECIFICATIONS
COURSE
SPECIFICATIONS
GRADUATE
ATTRIBUTES
ASSESSMENT

COMMUNITY
NEEDS
NQF
PROGRAM
SPECIFICATIONS
COURSE
SPECIFICATIONS
College Quality Unit
Curriculum Committee
College Quality Unit
Curriculum Committee
A c a d e m i c
Departments
GRADUATE ATTRIBUTES

ASSESSMENT
I.A. INTRODUCTION
I.C. WHEN TO ASSESS

I . B . P U R P O S E O F
A S S E S S M E N T

A S S E S S M E N T

• The term assessment has been used to refer to the
systematic determination of student/learner
achievement and performance (Schuwirth et al.,
2011)
• The term derives from the latin “assidere” meaning
to sit beside, suggesting that the assessor and the
student of dentistry travel together side by side on
the journey to aid learning
• So, assessment is important to ensure that we teach
well

• Assessment is a critical component of instruction:
• When properly used, it can aid in accomplishing key
curricular goals
• General primary purpose of testing is to:
1. Communicate to students material that is important
2. Motivate students to study
3. Identify areas of deficiency, in need of remediation,
or further learning
4. Determine final grades or make promotion decisions
5. Identify areas where instruction can be improved

WHY DO WE ASSESS OUR STUDENTS?
ACCOUNTABILITY
• To provide safe practitioners to the Society
FAIRNESS
• To ensure competency to progress

• Training the dental practitioner also requires
evaluation against a series of agreed standards
• Assessment is therefore distinct from evaluation of a
student for the purpose of certification, yet one
process certainly informs the other and the principle
of triangulation uses data from multiple sources to
determine the student’s readiness for practice

• Ongoing
• Improves quality
• Individualized
• Not scaled
• Provides feedback
• Process-oriented
1. ASSESSMENT
• Provides closure
• Judges quality
• Applied against standards
• Scaled
• Shows shortfalls
• Product-oriented
2. EVALUATION

ASSESSMENT
I.A. INTRODUCTION
I.C. WHEN TO ASSESS

IDEALLY:
• When we finish the curriculum
• When the student Strat practicing
PRACTICALLY:
• As per curriculum
• To assure that the competencies are built up in the
right way and you and your students are in the right
track
I.C. WHEN TO ASSESS

ASSESSMENT
I.A. INTRODUCTION
I.C. WHEN TO ASSESS

1
2
3
4
5
DIRECT CLOS
• Instructors
• Within the course
INDIRECT CLOS
• Department
• Student Satisfaction Feedback
• 2 weeks before the MID and Final Exams
DIRECT PLOS
• Quality Unit / PAC
• Within the Program
• Capstone Courses
INDIRECT PLOS
• Student Satisfaction Feedback
• 2 weeks before the MID and Final Exams
ALUMNI, STAKEHOLDERS &
BOARD OF TRUSTEES
• Community Feedback

ASSESSMENT
I.A. INTRODUCTION
I.C. WHEN TO ASSESS

1. ACCORDING TO ASSESSMENT MODE:
33
• Assesses a student’s strengths,
weaknesses, knowledge and skills
prior to instruction
A) DIAGNOSTIC
• Assesses a student’s performance
during instruction
• U s u a l l y o c c u r s r e g u l a r l y
throughout the instruction process
• Measures a student’s achievement
at the end of instruction
B) FORMATIVE C) SUMMATIVE

2. ACCORDING TO ASSESSMENT ASPECT:
34
• Compares a student’s performance
against a national or other “norm”
group or ranking them
A) NORM-REFERENCED
• Measures a student’s performance
against a goal, specific objective, or
standards
• Evaluates student’s performance at
periodic intervals, frequently at the
end of grading period
• Can predict student performance
on end-of-year summative test
B) CRITERION-REFERENCED C) INTERIM / BENCHMARK

3. ACCORDING TO ASSESSMENT SETTING:
35
• Formal
• Cheating control
• It could be paper-and-pencil or computer-based
• Couldn’t shuffle questions and distractors / student unless it is a
computer based
• Stressful
• Cost (printing, venues, facilities)
• Personels (administrators & invigilators)
• Security is questionable
• Post exam analysis is some how difficult
• Formality differs from institute to another
• Cheating control depends on the degree the instructor master the
technique and electronic invigilator software
• Shuffling of questions and distractors is could be automated so that
students versions do not match each other
• Lesser stress
• Lesser costs
• No need for additional personal, even invigilation could be automated
• More secure if the technology is applied professionally
• Immediate scoring and student feedback
• The system offers post exam analysis and grading
A) IN PERSON (DIRECT) B) ONLINE (INDIRECT OR DISTANT)

ASSESSMENT
I.A. INTRODUCTION
I.C. WHEN TO ASSESS

ACCOUNTABILITY:
• Mimic the realize situation as much as possible
FAIRNESS:
• Follow the validity and reliability principle
• Able to discriminate and evaluate the built
competencies
• Reflect learning outcomes achievement
• You should have:
BLUEPRINT

ASSESSMENT
I.A. INTRODUCTION
I.C. WHEN TO ASSESS

1. ESTABLISH TEST PURPOSE:
• A good first step in the development of an exam
program is to establish the test purpose
1. How will the test scores be used?
2. Is the exam intended for certification or for
licensure?
3. Will the emphasis of the test be on minimum
competency or on mastery of course content?
4. Overall, will the test be low-stakes, moderate-
stakes, or high-stakes for the examinees?
5. Will the results of the test have important
consequences for the examinees?
39

1. ESTABLISH TEST PURPOSE:
• The answers to these questions will have
implications for many aspects of the exam, such
as:
1. The overall length of the test (number of items or
tasks & time respond)
2. The average difficulty of the items
3. The conditions under which the test will be
administered
4. The type of information to be provided on the
score reports
• If you take the time at the beginning to establish a
clear, focused test purpose, your goals and priorities
will be more effectively met
40

2. CREATE THE TEST SPECIFICATIONS:
• Test specifications usually include:
a) Test description component
b) Test blueprint component
A) TEST DESCRIPTION COMPONENT:
• It is a written document that provides essential
background information about the planned exam
program
• This information is used to focus and guide the
remaining steps in the test development process
• The test description may simply indicate:
1. Who will be tested and what the purpose of the
exam program is
2. Overall test length
3. Test administration time limit
4. The item types that are expected to be used (e.g.,
multiple choice, essay)
5. Test administration mode (e.g., paper-and-pencil,
performance-based, computer-based)
6. If the test will include any items or tasks that will
need to be scored by human raters, it should include
plans for the scoring procedures and scoring rubrics
41

I . G . A S S E S S M E N T
T A S K S E T T I N G

- Scoring rubric minimizes students complains and conflicts with the
instructors
T A K E H O M E
M E S S A G E

2. CREATE THE TEST SPECIFICATIONS:
• Test specifications usually include:
a) Test description component
b) Test blueprint component
B) TEST BLUEPRINT COMPONENT:
• The content areas listed in the test blueprint, or table
of specifications, are drawn directly from the learning
outcomes
• These content areas comprise the knowledge, skills,
and abilities that have been determined to be the
essential elements of competency
• The test blueprint specifies the number or proportion
of items that are planned to be included on each test
form for each content area
• These proportions reflect the relative importance of
each content area to competency
52

Quiz 1 Quiz 2 Quiz 3 Quiz 4 Clinical
Cases
Presentatio
ns
Case Presentation
1.0 2.5% 2.5% 2.5% 2.5% 20% 25% 5% 20% 15%% 20% 100%
1.1 TOPIC 1: Preliminary considerations for clinical operative dentistry 3 5 8
1.1 TOPIC 2: Patient Assessment, Examination, Diagnosis & Treatment Planning. 1 8 9
1.1 TOPIC 3: Recent concepts in managing carious lesions 2 2
1.1 TOPIC 4: Conservative and minimal invasive restorative dentistry
1.1 TOPIC 5: Temporary restorations
1.1 TOPIC 6: Fundamental concept of enamel and dentin adhesion
1.1 TOPIC 7: Indirect metallic restorations
1.1 TOPIC 8: Indirect tooth-colored restorations
1.1 TOPIC 9: Management mutilated teeth
1.2 TOPIC 1: Preliminary considerations for clinical operative dentistry 7 7
1.2 TOPIC 2: Patient Assessment, Examination, Diagnosis & Treatment Planning. 1 1
1.2 TOPIC 3: Recent concepts in managing carious lesions 4 4
1.2 TOPIC 4: Conservative and minimal invasive restorative dentistry 4 8 12
1.2 TOPIC 5: Temporary restorations 1 1
1.2 TOPIC 6: Fundamental concept of enamel and dentin adhesion 3 13 16
1.2 TOPIC 7: Indirect metallic restorations 2 14 16
1.2 TOPIC 8: Indirect tooth-colored restorations 4 11 15
1.2 TOPIC 9: Management mutilated teeth 3 7 10
2.0 SKILLS
2.1 TOPIC 1: Preliminary considerations for clinical operative dentistry 1 3 4
2.1 TOPIC 2: Patient Assessment, Examination, Diagnosis & Treatment Planning. 5 9 14
2.1 TOPIC 3: Recent concepts in managing carious lesions 6 8 14
2.1 TOPIC 4: Conservative and minimal invasive restorative dentistry 4 4
2.1 TOPIC 6: Fundamental concept of enamel and dentin adhesion 4 5 9
2.1 TOPIC 7: Indirect metallic restorations 1 2 3
2.1 TOPIC 8: Indirect tooth-colored restorations 3 3 6
2.1 TOPIC 9: Management mutilated teeth 5 5
2.2 Record professionally the patient general
data and clinical findings using the
available clinical management electronic
system
10 2 12
2.3 Implement different restorative
treatment plans using safely and
efficiently all available instruments,
equipment and restorative materials
10 2 1 1 14
3.0 VALUES
3.1 Demonstrate satisfactory levels of
communication with patients, colleagues
and faculty members following the
College Code of Ethics and Code of
Conduct
10 2 12
10 10 10 10 60 10 2 60 1 1 174
QUIZZES (10%) Med-MCQs
(20%)
CLINICAL
REQUIREMENTS (30%)
Final-MCQs
(20%)
Final Clinical (20%)
CLOs TOPICS
ASSESSMENT BLUEPRINT - RDS 430
TOTAL
TOTAL
TASKS
KNOWLEDGE & UNDERSTANDING
Recall the diagnostic procedures for
restorative treatment planning
Indicate the restorative material and
techniques of different restorative
managements
Design professionally a restorative
treatment plan for different defects
affecting hard tooth structure based on
the data given by the patient and the
clinical findings
CONTINUOUS ASSESSMENT (60%) FINALS (40%) TOTAL
MARKS

K 1.1 K 1.2 S 2.1 S 2.2 S 2.3 V 3.1 TOTAL MCQs % of MCQs
TOPIC 1 8 7 4 0 0 0 19 11.88%
TOPIC 2 9 1 14 0 0 0 24 15.00%
TOPIC 3 2 4 14 0 0 0 20 12.50%
TOPIC 4 0 12 4 0 0 0 16 10.00%
TOPIC 5 0 1 0 0 0 0 1 0.63%
TOPIC 6 0 16 9 0 0 0 25 15.63%
TOPIC 7 0 16 3 0 0 0 19 11.88%
TOPIC 8 0 15 6 0 0 0 21 13.13%
TOPIC 9 0 10 5 0 0 0 15 9.38%
Clinical Cases 10 10 10
PPT 2 2 2
Final Case 1
Final PPT 1 TOTAL TASKS
Total Ass Tasks 19 82 59 12 14 12 174
Ass Tasks % 10.92% 47.13% 33.91% 6.90% 8.05% 6.90% 100.00%
SUMMARY OF ASSESSMENT BLUEPRINT - RDS 430
MCQs
CLINICAL
CLOs

T A K E A N O T E
- Once you designed an accurate blueprint for the course, the assessment tasks
and items could be easily changed not the blueprint
- Also, you could easily adjust and balance the marks distribution in relation to
the course learning outcome
- You could point out and trace learning outcome achievement and how to
improve

- Blueprint aligns assessment items with course learning outcomes
T A K E H O M E
M E S S A G E

3. DEVELOP THE INITIAL POOL OF ITEMS
4. REVIEW THE ITEMS
5. CONDUCT THE STANDARD SETTING
6. CONDUCT THE ITEM ANALYSIS
7. SCORE THE TEST
8. GIVE FEEDBACK
57

7%
93%
4%
96%
15%
85%
EXAM FEEDBACK SURVEY
• 93% think there should be
exam feedback
• 96% think exam feedback
would be useful
• 85% think exam feedback
is feasible

ASSESSMENT
I.A. INTRODUCTION
I.C. WHEN TO ASSESS

A S S E S S M E N T
PSYCHOMETRICS
1. VALIDITY
2. RELIABILITY
3. STANDARD ERROR OF MEASUREMENTS
4. DIFFICULTY INDEX
5. DISCRIMINATION INDEX

DESCRIPTION:
• Validity addresses the question of whether a test
measures what it is supposed to measure
CONSIDERATIONS:
• The following criteria need to be considered as
having an impact upon this unified concept of validity
1. Test content
2. Response process
3. Internal structure
4. Relationship to other tests
5. Effects / Outcome
61
I.H.1. VALIDITY

CONSIDERATIONS:
1. Test content:
• Refers to the purpose of the test and how it is
defined, i.e.
- Does the test content appropriately reflect the
learning objectives of the course/module?
- Is the assessment task both relevant and
representative of the work a dentist will do
• Blueprinting is key to this aspect of validity
2. Response Process:
• Refers to the type of testing formats being used i.e:
chosen or constructed
• There is more scope for error with constructed
responses as these currently cannot be electronically
scored and require examiners to read and mark
• Certain criteria should therefore be met to minimize
error, i.e. scoring rubrics, blinded double marking
62
I.H.1. VALIDITY

CONSIDERATIONS:
3. Internal structure:
• Refers to how a test is constructed and includes the
following criteria:
- Number of items (in a written test) or number of
stations (in a practical test)
- Format of the items
- Whether the format is appropriate for the domain
of skill being tested (e.g. MCQs for knowledge
tests, OSCEs for clinical skills)
- Sufficient sampling for the tests to be reliable
- Item analysis data (test score correlation, facility
indices, etc.)
- Weighting of certain parts of the test or equal
weighting of all parts
- Presence of a system of compensation
- Method of standard setting applied to determine
the pass mark
63
I.H.1. VALIDITY

CONSIDERATIONS:
4. Relationship to other tests:
• How do the results of a test compare to the results of
other tests taken by the same candidates?
5. Effects / Outcomes:
• Consider the implications and consequences of
decisions made on the basis of each test, e.g. Effect
on student learning
• Impact of failing - on students, on parents, on tutors,
on remediation and support staff
• Impact of passing - on students, on patients, relevant
health authorities, university reputation and the
regulatory body
64
I.H.1. VALIDITY

A S S E S S M E N T
PSYCHOMETRICS
1. VALIDITY
2. RELIABILITY
4. DIFFICULTY INDEX

DESCRIPTION:
• The reliability of a method reflects the reproducibility
of its results
• When a test is highly reliable, it is highly likely that it
produces the same result each time it is run
• This similarity can be explored from three slightly
different angles:
1. The absolute score
2. The relativity
3. The boundary
66
I.H.2. RELIABILITY

DESCRIPTION:
Example 1 – the absolute score:
• Student (A) score 78% in a test
• If Student (A) were to sit a similar but different test (also
called a parallel test), would he score 78% again?
Example 2 – the relativity:
• In a class of students, Student (A) has the best score
• Student (B) comes second and Student (C) has the worst
performance
• If the class were given a parallel test, would the ranking
remain the same?
Example 3 – the boundary:
• In a sitting, Student (A) passes the test with distinction
• The cut-off score for distinction was 70%
• In a parallel test would Student (A) pass with distinction
again?
67
I.H.2. RELIABILITY

DESCRIPTION:
Reliability Coefficient (KR20):
• Kuder–Richardson Formula 20 (KR-20) is a special
case of Cronbach’s ⍺ test for reliability that indicates
the consistency of the test
• This index can hold a value between 0 and 1
• Higher test reliability indicates that the test measures
whatever it measures in a consistent manner
68
I.H.2. RELIABILITY

DESCRIPTION:
Reliability Coefficient (KR20):
69
I.H.2. RELIABILITY

VALIDITY & RELIABILITY
Reliable
Not Valid
Low Reliability
Low Validity
Not Reliable
Not Valid
Reliable
Valid
P S Y C H O M E T R I C S

A S S E S S M E N T
PSYCHOMETRICS
1. VALIDITY
2. RELIABILITY
4. DIFFICULTY INDEX

DESCRIPTION:
• SEM is a measure of how much measured test scores
are spread around the “true” score
Observed score = “True Core” ± SEM
• How much error is associated with a student score
• How much error or fluctuation expected around true
score
• A student with a test score of 80, if it is not exactly 80,
it would be 80 ± SEM
• SEM is adversely related to test reliability
• If SEM is “large”, it means test and test scores are
unreliable
72
I.H.3. STANDARD ERROR OF MEASUREMENTS

DESCRIPTION:
• If test is reliable, How confident are we in test results?
• Approximately:
a) 68% confidence:
True score = [Score - (1*SEM) ~ Score + (1*SEM)]
b) 95% confidence:
c) 99% confidence:
• Example: A student scores 90 on a test with an
SEM=2
a) At 68% confidence: True score range 88 ~ 92
b) At 95% confidence: True score range 86 ~ 94
c) At 99% confidence: True score range 84 ~ 96
73
I.H.3. STANDARD ERROR OF MEASUREMENTS

A S S E S S M E N T
PSYCHOMETRICS
1. VALIDITY
2. RELIABILITY
4. DIFFICULTY INDEX

DESCRIPTION:
• Difficulty index =
• Difficulty index range and items distribution
75
Number of Students With Correct Answer
Total Number of Students Access the Exam
20% 60% 20%
Very Difficult Difficult Moderately Difficult Moderate Moderately Easy Easy Very Easy
0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
I.H.4. DIFFICULTY INDEX

- The recommended acceptable range of exam or item difficulty is:
0.3 - 0.8
T A K E A N O T E

DESCRIPTION:
Overall exam with high values:
• Occasionally everyone knows the answer
• This may be due to:
1. Exam leaks
2. Repetition of the items in previous exams
3. Cheating
4. Poorly constructed items that could be guessed
5. Excellent teaching and high standard student
performance
Overall exam with low values:
• This may be due to:
1. Stressful exam environment
2. Students didn’t understand the concept being
tested
3. Badly constructed items
4. Bad teaching
5. Careless students with low standard performance
77
I.H.4. DIFFICULTY INDEX

A S S E S S M E N T
PSYCHOMETRICS
1. VALIDITY
2. RELIABILITY
4. DIFFICULTY INDEX

I.H.5. DISCRIMINATION INDEX
DESCRIPTION:
• Also termed “Point Biserial”
• Discrimination index is calculated according to the
following equation:
• RU is the number of students in the upper quadrant
group who got the item right
• RL is the number of students in the lower quadrant
group who got the item right
• T is the total number of students included in the
analysis, i.e. got the item right
• The discrimination index value for an item ranges
from -1 to +1
D =
RU - RL
½ T

DESCRIPTION:
80
I.H.5. DISCRIMINATION INDEX

- The recommended acceptable range of exam or item discrimination index is:
0.2 - 0.9
T A K E A N O T E

ASSESSMENT TOOLS
PART II:

II.A. INTRODUCTION
II.B. DEVELOPMENT OF COMPETENCIES
II.C. ASSESSMENT METHODS
II.D. DEVELOPING ASSESSMENT STRATEGY
II. A S S E S S M E N T
T O O L S

• The Transition from Student to Practitioner
84
II.A. INTRODUCTION
NOVICE
ADVANCED
BEGINNER
COMPETENT PROFICIENT EXPERT
Dreyfus’ Five-stage model of Adult Skill Acquisition

II.A. INTRODUCTION
Rules Considers
everything
Detached
observer
Novice
Advanced
Beginner
Competent
Proficient
Expert
Intuition Relevant
focus
Part of the
system
Guidance
Skills
Novice
Advanced Beginner
Competent
Proficient
Expert

II.A. INTRODUCTION
T O O L S

Does
Shows
Knows how
Knows
Behavioral
Level
Cognitive
Level
Competencies
of the Dental
Practitioner
PSYCHOMOTOR
Skills
AFFECTIVE
Attitudinal
COGNITIVE
Knowledge-Based
Miller’s Pyramid Bloom’s Taxonomy of Educational Objectives

Does
Shows
Knows how
Knows
Miller’s Pyramid
• Complete the same task for a patient
• Performance of the task for a patient will usually involve skills such as:
communication, teamwork with the dental surgery assistant, gentle
anesthesia of the tooth, protection of the soft tissues, careful tooth
preparation, checking the occlusion
• Demonstrate that they can show this skill in a simulated setting such as
on a dental mannequin
• A dental student may first learn tooth morphology and dental
biomaterials
• Learn how to recreate this morphology using a restorative material

Does
Shows
Knows how
Knows
Miller’s Pyramid
• Mini-Clinical Evaluation Exercise (mini-CEX)
• Directly Observed Procedural Skills (DOPS)
• Dental Evaluation of Performance Test (ADEPT)
• Case-based Discussion (CbD)
• Observation on Clinics or Rotations
• Portfolios
• Objective Structured Practical / Clinical Examination (OSPE / OSCE)
• Practical or Clinical Test
• Multiple Choice Questions (MCQs)
• Extended Matching Questions (EMQs)
• Short-Answer Questions (SAQs)
• Essays

Bloom’s Taxonomy of Educational Objectives
PSYCHOMOTOR
Skills
AFFECTIVE
Attitudinal
COGNITIVE
Knowledge-Based

II.A. INTRODUCTION
T O O L S

ASSESSMENT
M E T H O D S
1. MILLER’S PYRAMID “KNOWS” AND “KNOWS HOW”
2. MILLER’S PYRAMID “SHOWS”
3. MILLER’S PYRAMID “DOES”

1. Multiple Choice Questions (MCQs)
2. Extended Matching Questions (EMQs)
3. Short-Answer Questions (SAQs)
4. Essay
5. Structured Oral / Viva / Viva Voce
6. The “Spotter” Test
7. Script Concordance Test (SCT)
8. Triple Jump Exercise (TJE)
93
II.C.1. MILLER’S PYRAMID “KNOWS” AND “KNOWS HOW”

1. MULTIPLE CHOICE QUESTIONS (MCQS):
Knowledge Assessed:
• Depending on the question, this can range from
“Knows to Knows How” levels of Miller’s pyramid and
“remembering to evaluating” levels of Bloom’s
taxonomy
Key Points:
• High reliability
• Computer marking saves time and resources
• Feedback often limited to overall score or score in
different sections (due to question security)
• Easy to blueprint comprehensively to the syllabus
• Requires significant staff training and quality
assurance
• Standard setting is time consuming
94

A Patient came back to you one week after you had finished a composite
resin restoration for a shallow simple occlusal cavity, complaining from
tooth sensitivity to biting and the clinical examination revealed a slight
tooth mobility that developed after tooth restoration which was originally
normal.
What is the possible cause of this condition?
a.    Improper bonding procedures
b.    Minor tooth crack
c.    Premature occlusal contact
d.    Periodontal pocket
Stem
Lead-in
Options
Options = Correct answer + Distractors
I I . C . A S S E S S M E N T
M E T H O D S

2. EXTENDED MATCHING QUESTIONS (EMQS):
Knowledge Assessed:
taxonomy
96

Description:
• The EMQ format has four components
1. Title or theme statement
2. List of ‘options’
3. Lead in statement
4. Question/s (item/s)
• The examinee has to respond to each question by
selecting the best answer from a large list (range from
5 up to 20 or more), where one or more answers are
potentially correct
• Number of options in the list should be more than
the number of questions
• Where there are several questions under one title,
each answer can be used once, more than once or
not at all
97

a. Amalgam alloy
b. Ceramics
c. Direct composite resin
d. Direct gold alloy
e. Glass ionomer cement
f. Indirect composite resin
g. Indirect gold alloy
h. Zinc oxide / eugenol
RESTORATIVE MATERIALS
For each of the following statements choose the most appropriate material from
the list above, you may use each option once or more or not at all
1. Excellent esthetics, brittle in thin sections, can be difficult to bond to tooth,
constructed indirectly, can cause wear of the opposing teeth if occlusion is
incorrect
2. Known to have eutecloidal phase diagram, and is available in high and low
copper variables. Can be used as a restorative material, as a core material
or in situations where moisture contamination and isolation is a problem
Theme
Options
Lead-in
Items
M E T H O D S

Key Points:
• Potentially higher reliability than MCQs
• Writing items that will test higher cognitive levels is
time consuming
• Linked items can reduce the choice of topics and
therefore reduce sampling across the curriculum
• Feedback is often limited to overall score or score in
different sections (due to maintaining question
security)
• Good discriminators at higher levels of ability
• If implemented online, shuffling of questions and
distractors should be disabled
99

3. SHORT-ANSWER QUESTIONS (SAQS):
Knowledge Assessed:
“remembering to creating” levels of Bloom’s
taxonomy
100

Description:
• A written test consisting of a series of questions that
require students to supply or formulate an answer
rather than choose from a list of options (as in MCQs)
• SAQ may require the examinee to construct a short
response (several sentences, a plan or a diagram) and
in some contexts write a short or structured version of
an essay
• Questioning can be directed to test a specific
objective or area
• The question format may be based on a case
scenario or set of data and may include additional
information e.g. images Sometimes several SAQs are
written as a linked series covering a particular topic
area
• Compared to MCQ/EMQ, there is no cueing effect,
as examinees are not presented with the correct
answer amongst a number of other choices
101

A case of minor diastema between the two central incisors that was planned to be
closed with bonded composite resin restoration traumatically without any
preparation and you have the choice to select either Etch & Rinse adhesive, Self-
Etch adhesive or Universal adhesive. Sort your choices from the best to the worst
and justify your choices?
1st Choice:
Why:
2nd Choice:
Why:
3rd Choice:
Why:
102
M E T H O D S

Key Points:
• Resource intensive marking compared to MCQ/EMQ
• Heterogeneity in interpretation of the term SAQ
• Reliability improved if structured marking schemes,
clear outline answers and independent double
scoring employed
• Has the advantage of no cueing effect
• Provision of written feedback possible but time
consuming
103

4. ESSAY:
Knowledge Assessed:
• Depending on the question can range from “Knows
to Knows How” levels of Miller’s pyramid and
taxonomy
104

4. ESSAY:
Description:
• An essay has been described as a short literary
composition on a particular theme or subject, usually
in prose and generally analytic, speculative, or
interpretative
• Sometimes also referred to as ‘long answer’ or
‘extended answer’ questions
• A variation is the modified essay question, which may
include e.g. an element of data handling
• It should be clear to students whether the essay is
being assessed / marked as a structured argument or
is being used as a means of testing knowledge
• For the latter, more efficient alternatives are
preferable
105

The selection of base material to protect the pulp is affected by the cavity depth
and the selected final restoration. Discuss?
”Drill, Fill and Bill” was an old description of operative dentistry which is recently
described as “Art & Science”. Discuss?
M E T H O D S

4. ESSAY:
Key Points:
• Resource intensive marking
• Low reliability
• Double marking recommended to improve reliability
• Heterogeneity in interpretation of the word ‘essay’
which can be confusing for students and make
comparison as a ‘method’ confusing
consuming
107

5. STRUCTURED ORAL / VIVA / VIVA VOCE:
Knowledge Assessed:
taxonomy
108

Description:
• The viva format involves the examinee being
questioned by one or more examiners using an
interview or discussion-like format typically to
ascertain knowledge of a subject area or the ability to
solve a clinical problem
• This is followed by discussion and questioning aimed
at probing the examinee’s depth and breadth of
knowledge, understanding, reasoning, and decision-
making process
• A viva can be used to explore ethical issues, assess
professionalism, attitudes and communication
skills
• As with several other forms of assessment, there is
considerable variation in the format and use of this
type of assessment, with standardized content and
structure (Structured Viva) more commonly used with
the aim of reducing bias
109

Considerations:
• If used as part of routine examinations for all
students, the time and resources required are
considerable
• This is even more of a problem when the number of
questions or cases presented is increased (as one way
of trying to improve reliability)
• Vivas (as well as other one-to-one encounters) can be
subject to “Halo effects” i.e. the effect whereby a
judgement on one aspect is influenced by an overall
impression of the person or where the judgement is
influenced by the performance of previous
candidates in contrast to the current candidate
110

Key Points:
• Heterogeneity in interpretation of the method
• Low reliability unless multiple examiners, multiple
cases and large testing time
• Often seen as having high authenticity to examiners
• Needs careful planning of subject matter and
appropriate coverage of the syllabus
• Resource intensive
• Immediate face to face feedback can be built in to
the process
• A good exam case should involve a relatively
common presenting problem with several plausible
diagnosis and should primarily test the student’s
problem solving skills
111

6. THE “SPOTTER” TEST:
Knowledge Assessed:
taxonomy
112

Description:
• This format has been a traditional assessment format
particularly for disciplines such as anatomy and
pathology
• The format is increasingly being replaced, in part or
completely, by computerized assessments using high
quality images
• It should not be confused with methods assessing at
the ‘Shows’ level of Miller’s pyramid
• The format usually has examinees moving around a
series of stations consisting of e.g. a specimen, a
labelled dissection or radiograph
• The answer may be one word or involve a response
that requires some level of deduction or diagnostic
skill
113

Key Points:
• Has been in common use but is being replaced by
computerized assessment and marking (where
possible)
• Heterogeneity in interpretation of the term
• Reliability improved if structured marking schemes
employed
consuming
• Consider using images within more reliable and
evidence-based forms of assessment
114

7. SCRIPT CONCORDANCE TEST (SCT):
Knowledge Assessed:
taxonomy
115

Description:
• Designed to assess assess decision-making and
clinical reasoning skills
• The SCT is a written exam that starts with a clinical
scenario or vignette that summarizes the case (e.g. a
patient presents with dental pain)
• This is followed by a proposed diagnosis or
suggested treatment or action
• Examinees have to rate the effect of further
information (e.g. the pain gets worse with hot and
better with cold) or findings on the probability of the
diagnosis / treatment being: more certain / likely,
unchanged or less certain / likely, using a 5-point
scale
• The answers are compared to those of a panel of
experts
116

A 39 years-old patient has a large decay under amalgam on tooth 16. After
amalgam removal and caries excavation, you would like to restore the lost
tooth structure.
M E T H O D S
If you were thinking And you see Hypothesis become
1. Direct composite resin -2 -1 1 2
2. Indirect composite resin -2 -1 1 2
3. Ceramic inlay -2 -1 1 2
4. Indirect gold inlay -2 -1 1 2
5. Endo-crown -2 -1 1 2
-2 = Strongly contraindicated, -1 = Not indicated, 1 = Indicated, 2 = Highly
indicated
Case
Description
Hypothesis
New
Information
4
levels
Likert’s
Scale

M E T H O D S

Key Points:
• Written test based on scenario / vignette
• Significant training required for item writing
• Considerable time needed to produce each question
• Good reliability if sufficient questions are used
• Requires suitably qualified panels of examiners to
produce scoring system
• Feedback often limited to overall score or score in
different sections (due to question security)
119

8. TRIPLE JUMP EXERCISE (TJE):
Knowledge Assessed:
“remembering to applying” levels of Bloom’s
taxonomy
120

Description:
• Originally developed at McMaster University in 1974
as a method of testing the critical thinking abilities of
the individual on an undergraduate program with an
emphasis upon group work and problem-based
learning (PBL)
• Each stage (jump) of the assessment is scored to
produce a cumulative score
• There are several variations:
a) Clinical TJE
b) Pre-Clinical TJE
121

Description:
a) Clinical TJE:
• Comprises a written patient history and examination,
followed by a write-up of the findings in the format of
subjective data, objective data, assessment and plan
(SOAP) to include evidence from the literature (the
second “jump”)
• The write-up is submitted to the member of the
faculty who observed the first “jump” (i.e. history and
examination) and this same faculty member also
conducts the third ”jump”, an oral examination of the
student to cover the pathophysiology, diagnosis, and
treatment of the patient including appropriate
research evidence
122

Description:
b) Pre-Clinical TJE:
• Uses a written patient scenario for which the student
must identify key issues and write a research question
in the PICO format (Problem, Intervention,
Comparison and Outcome)
• Researching the literature forms the second “jump”
with a report of their findings including the answer to
the research question and a critical appraisal of the
quality of the evidence to complete the third “jump”
123

Skills Assessed:
• The individual’s ability to search for evidence to support
their clinical practice with appropriate analysis and
application to health care problems i.e. reasoning and
learning skills
Key Points:
• Requires a significant amount of faculty time to conduct
and score
• Student performance may be case dependent
• More experienced students have been successfully used
as examiners
• No evidence to confirm whether a good performance in
this examination will be indicative of a similar approach
in the work place
• Usually used as a formative rather than summative
assessment
124

- MCQs design involves also double and triple jump strategies to assess the
cognitive and critical thinking skills of the students
T A K E A N O T E

- Reliability of “KNOW & KNOW HOW” assessment methods:
1. Script Concordance Test (SCT)
2. Multiple Choice Questions (MCQs) - if less than 0.7 - 0.8, items showed be reviewed
3. Extended Match Questions (EMQs) - if less than 0.7 - 0.8, items showed be reviewed
4. Short Answer Questions (SAQs)
5. The Spotter Test
6. Triple Jump Exercise (TJE)
7. Essay
8. Structured Oral / Viva / Viva Voce
CO N C LU S I O N S

ASSESSMENT
M E T H O D S

1. Objective Structured Clinical / Practical Exam
(OSCE / OSPE)
2. Practical Test in Simulated Clinical Setting or
Laboratory
128
II.C.2. MILLER’S PYRAMID “SHOWS”

1. OBJECTIVE STRUCTURED CLINICAL / PRACTICAL
EXAM (OSCE / OSPE):
Description:
• The exam consists of multiple mini-stations (typically
10 to 20), which the examinees rotate round in
sequence, completing a variety of tasks
• The examinee reads the scenario, then enters the
station and undertakes the task
• The station set-up varies and can include: simulated
patients, models, part-task trainers, laboratory
equipment and simulated work stations
• The selection of stations should be representative of,
and mapped (blueprinted) to, the taught course
• Blueprinting needs to consider several dimensions of
competence within each station including stages in a
clinical case and body systems
129

EXAM (OSCE / OSPE):
Marking:
1. Detailed Checklists:
• A very detailed checklist often with 15 to 25 items
that the examinee did or did not complete /
undertake
• Each item can be equally weighted i.e. 1 or 0,
although some critical steps (e.g. fatal errors, a break
in sterility, etc.) may carry a heavier weighting (more
marks) or be a requirement to pass the station
2. Global Rating Scales (GRS):
• Checklists are usually accompanied by a global rating
scale for the examiner to make a more subjective
judgement (selecting one of 4 to 7 categories with
descriptors across the spectrum from a bad fail to an
excellent pass)
130

EXAM (OSCE / OSPE):
Skills Assessed:
• Clinical / practical, technical and diagnostic skills,
treatment planning, and communication skills
Key Points:
• High reliability compared to a few long cases or
individual clinical examinations
• Potential to compromise validity by excessively
deconstructing tasks
• Resource intensive to establish, set up and run
• Can provide detailed specific feedback
131

2. PRACTICAL TEST IN SIMULATED CLINICAL SETTING
OR LABORATORY
Description:
• The student undertakes a defined piece of practical work
(e.g. Class II resin composite restoration) in the absence
of a patient
• This usually involves a phantom head/dental manikin and
synthetic teeth, although may also involve the
construction of laboratory items such as dentures,
crowns or inlays using a variety of materials
• Assessment may be at each stage of the process (e.g.
caries removal, cavity preparation, matrix band
placement, restoration placement, finishing) or for the
overall result
• Marking is usually by an expert clinician using a check list
(Rubric) of the required standards with a global rating
such as ideal, satisfactory, borderline or unsatisfactory
• An alternative method of marking is to use expert
opinion to inform software development for haptic
simulators which can then provide either summative or
formative feedback for the practical test
132

2. PRACTICAL TEST IN SIMULATED CLINICAL SETTING
OR LABORATORY
Skills Assessed:
• Manual dexterity and handling of dental materials in a
quasi-clinical setting such as a clinical skills laboratory
• The global rating reflects that the outcome is
underpinned by an appropriate level of knowledge and
professionalism which may be tested during the
examination with structured questioning and
observation, for example as part of the Overseas
Registration Examination (ORE)
Key Points:
• Clinical checklists, examiner training and the use of
multiple examiners improve reliability but will
increase the resource requirements
• The scores for each student from two or more
examiners can be compared to check correlation and
hence reliability of scoring for a particular assessment
133

- Reliability of “SHOW” assessment methods:
1. Objective Structured Clinical / Practical Exam (OSCE / OSPE):
High reliability if the number of stations is quite enough (15 - 20)
2. Practical Test in Simulated Clinical Setting or Laboratory
Without rubric, its reliability is questionable due to inter-examiner variability
CO N C LU S I O N S

ASSESSMENT
M E T H O D S

1. Mini-Clinical Evaluation Exercise (Mini-CEX)
2. Directly Observed Procedural Skills (DOPS)
3. Dental Evaluation of Performance Test (ADEPT)
4. 360o
(Multi-Source Feedback, MSF)
5. Case-based Discussion (CbD)
6. Observation on Clinics or Rotations
Longitudinal Evaluation of Performance (LEP)
7. Portfolios
8. Entrustable Professional Activities (EPAs)
136
II.C.3. MILLER’S PYRAMID “DOES”

1. MINI-CLINICAL EVALUATION EXERCISE (MINI-
CEX):
Description:
• Direct observation of a trainee by one examiner
during a clinical encounter with a real patient in the
normal work setting e.g. on a ward or in a dental
clinic
• In the mini-CEX, the “snap-shot” observation lasts 15
to 20 minutes and is followed by immediate feedback
from the examiner
• The observation is marked using a standardized tick
box (Rubric) form that is used to record information
about the case, setting, trainee and examiner
• Performance is rated for a list of skills as: at, above or
below expectation
• Mini-CEX are primarily used formatively with
feedback to produce an action plan that is structured
to support the trainee’s learning
137

1. MINI-CLINICAL EVALUATION EXERCISE (MINI-
CEX):
Skills Assessed:
• History taking, physical examination, diagnostic,
clinical judgement, decision making, communication
and time management
Key Points:
• High authenticity
• Reliability increases with number of examinations
(mini-CEXs) performed
138

2. DIRECTLY OBSERVED PROCEDURAL SKILLS
(DOPS)
Description:
• DOPS is designed specifically to assess practical skills
in a workplace setting
• A student is observed and scored by an assessor
while performing a routine practical procedure during
his / her normal clinical work
• The assessor uses a standard DOPS form to score the
technique (Rubric)
• The student is deemed either competent or
incompetent
• DOPS have been used as part of peer assessment
where dental students have been able to detect
improvement in the performance of their peers over
time
139

2. DIRECTLY OBSERVED PROCEDURAL SKILLS
(DOPS)
Skills Assessed:
• Practical / technical ability and manual dexterity in a
work place setting
Key Points:
• Multiple assessments of the same skill
• Present a valuable opportunity for formative feedback
with written marking
sheet
• Criterion referenced marking
• Emphasis upon testing psychomotor skills
• Resource intensive to conduct the assessment and need
suitable cases
140

3. DENTAL EVALUATION OF PERFORMANCE TEST
(ADEPT)
Description:
• A combination of DOPS and mini-CEX
• The evaluator observes the students performance within
a patient encounter and rates their performance against
a 6-point scale across several broad criteria such as
treatment planning, communication or time
management and organization
• The clinical competencies covered are recorded for each
case and following feedback
Key Points:
• Wide range of cases should be assessed, to cover all
major competencies of the clinical domain
• Used in the UK once a month for newly qualified
(foundation) dentists for the first year in supported
practice
141

4. 360O
(MULTI-SOURCE FEEDBACK, MSF)
Description:
• Involves collecting information about a clinician’s
performance in the workplace from those working
with that individual
• Feedback is gathered using a structured form or
questionnaire
• Different members of the clinical team assess the
individual’s performance and his or her professional
behavior
• Those ‘assessing' the individual include staff who are
more senior, more junior and peers; and
representatives of all groups in the clinician’s daily
working environment (not just co- professionals) e.g.
patients
142

4. 360O
(MULTI-SOURCE FEEDBACK, MSF)
Skills Assessed:
• Communication, team working, professionalism and
possibly insight
Key Points:
• Allows feedback from range of individuals (a variety
of staff +/- patients)
• Useful information may be gained about professional
behavior and insight
143

5. CASE-BASED DISCUSSION (CBD)
Description:
• A formal discussion between a trainee and an
assessor about a case that the trainee has managed
and for which they have been directly responsible
• During the discussion, the trainee refers to the case
records
• The assessor will probe the trainee’s depth of
understanding, decision-making and clinical
judgement
• A structured assessment form (Rubric) is used to
record basic case details and rate the key skill areas
144

5. CASE-BASED DISCUSSION (CBD)
Skills Assessed:
• Application of knowledge, decision making, clinical
judgement, professionalism
Key Points:
• Standardized rating system
• Mostly used formatively
• Low reliability
145

6. OBSERVATION ON CLINICS OR ROTATIONS:
LONGITUDINAL EVALUATION OF PERFORMANCE (LEP)
Description:
• The assessment is based on performance over a period
of time (days to weeks) and a number of skills can be
rated from basic factual knowledge to technical skills as
well as other aspects of professional behaviour
• Students are often assigned a grade at the end of the
rotation / placement, which can be derived from a global
rating form that includes general categories of
professional and clinical ability e.g. knowledge, clinical
skills, communication skills, case responsibility,
preparation and professionalism
• The assessment may be undertaken by one tutor or
several members of the team
• It may involve grading a log book, frequently
electronically, and taking a mean of the week-by-week
grades to give an overall grade at the end of a term or
placement
146

6. OBSERVATION ON CLINICS OR ROTATIONS:
LONGITUDINAL EVALUATION OF PERFORMANCE (LEP)
Skills Assessed:
• Knowledge, application of knowledge, clinical/
practical skills, diagnostic skills, clinical reasoning,
communication skills, attitudes and professionalism
Key Points:
• Based on observation of students in routine practice
• Low reliability
• Subjective and prone to ‘halo effects’
• Can provide useful opportunity for feedback
147

7. PORTFOLIOS
Description:
• This is a collection of work developed as a cumulative
‘body of evidence’ to demonstrate the student’s
learning and achievements
• It is not an examination format in its own right, rather
a receptacle containing a mixture of materials, each
piece assessable to predefined marking criteria which
may be graded or pass/fail
• Assessment of portfolios and the criteria applied are
quite variable and depend on content
• Assessment is often an on-going process which can
be formative and/or summative
• Interviews provide an opportunity to determine how
well the portfolio reflects the student’s achievements
• Provision of evidence that certain tasks have been
completed with a grade for engagement is important
148

7. PORTFOLIOS
Skills Assessed:
• K n o w l e d g e , k n o w l e d g e a p p l i c a t i o n a n d
interpretation, case recording and interpretation,
attitudes and professionalism (skills not always easy
to assess using other methods)
Key Points:
• Heterogeneity in meaning – covers many different
formats
• Assessing reflection is difficult and controversial
149

8. ENTRUSTABLE PROFESSIONAL ACTIVITIES (EPAS)
Description:
• The concept of EPAs has been adopted in many health
professions education programs since it was introduced
in 2005, EPAs) as a useful framework for assessment of
competency
• The EPAs framework expands upon the competency-
based education concept to facilitate assessment of
trainees’ abilities (competencies) and readiness for
practice through the assessment of performance of the
tasks associated with the job and its specified roles
within an authentic workplace setting
• Competencies are comprised of knowledge, skills and
attitudes
• The EPAs description should include multiple
assessment methods, and there should be guidance to
define for learners their stage of progress toward
independence and overall competence
• All focus on competency with maximum safety
150

- Reliability of “DOES” assessment methods:
Unless you have a well constructed rubric, reliability of all types of assessment is questionable due
to inter-examiner variability
CO N C LU S I O N S

- The selection of assessment type is affected by:
1. Type of education (Knowledge-based or Competency-based or Both)
2. Objectives and skills to be assessed
3. Number of students
4. Number of examiners
5. Resources
T A K E A N O T E

- Regardless the type of assessment
Blueprint & Rubric are Crucially Strategic
T A K E H O M E
M E S S A G E

T O O L S
II.A. INTRODUCTION

• When designing a new curriculum (or reviewing an
existing one) the development of an assessment
strategy is particularly valuable and should be an
integral part of the process
• The strategy should entail assessment of learning as:
1. Summative and for decision making
2. Formative, within a culture of feedback and
mentoring
• An assessment strategy aims to:
1. Provide a clear and comprehensive overview of all
assessments throughout
the curriculum
2. Demonstrate how assessments align with each
domain of competence and graduate outcomes
(GDC Preparing for Practice)
3. Include both assessment of learning (summative)
and for learning (formative)

• The strategy document is an extremely useful
management tool and can help support a
coordinated and consistent approach to all aspects of
assessment including:
1. Planning
2. Blueprinting
3. Standard setting including rubrics and structured
checklists for all types of assessments
4. Psychometrics
5. Feedback, etc.

ASSESSMENT BLUEPRINT
PART III:

H O L D T H I S
T H O U G H T
If you don’t know where you are going, every road
will get you nowhere
Henry Kissinger

III.A. WHAT IS ASSESSMENT BLUEPRINT
III.B. BENEFITS OF ASSESSMENT BLUEPRINT
III.C. HOW TO DESIGN ASSESSMENT BLUEPRINT
III. A S S E S S M E N T
B L U E P R I N T

• Assessment blueprint is a list of key components
defining your assessment including:
1. The purpose of the test
2. Content framework
3. Test time
4. Items format
5. Content weighting

1. The purpose of the test:
• It might be something simple, such as assessing
knowledge prior to instruction to a get a baseline of
what students know before taking a course
• Alternatively, the test purpose might be more complex,
such as assessing retention of material learned across
the course to determine eligibility for advancement
2. The content framework:
• Knowledge and skills to be assessed, i.e. topics or / and
tasks
3. The testing time:
• This includes amount of testing time available and the
need for breaks, as well as other logistical issues
related to the test administration
4. The content weighting:
• The number of questions per topic category should
reflect the importance of the topic; and the amount of
time spent on that topic in the course
5. The item formats:
• MCQs, EMQs, EASSY, ….

B L U E P R I N T

• Blueprinting can help you:
1. Assess the instructional objectives of the course or
instructional period
2. Avoid over- or under-representing a topic in your test
3. Use appropriate formats for the skills being assessed
4. Ensure consistent coverage of exam content from
year to year
5. Communicate course expectations to stakeholders
(e.g., trainees, other faculty, administration)
• Blueprinting can help students:
1. Construct meaning of the content by providing a
framework or mental schema
2. Plan their studying
3. See the topics you value most

B L U E P R I N T

1. CONTENT FRAMEWORK:
• List the followings:
1. Assessment Strategy
2. Course learning outcomes
3. Topics
4. Learning tasks and activities
5. Total required number of items
• Organize the relation between content

III.C. HOW TO DESIGN

1.0
1.1 TOPIC 1: Preliminary considerations for clinical operative dentistry
1.1 TOPIC 2: Patient Assessment, Examination, Diagnosis & Treatment Planning
1.1 TOPIC 3: Recent concepts in managing carious lesions
2.0 SKILLS
2.1 TOPIC 3: Recent concepts in managing carious lesions
3.0 VALUES
Demonstrate satisfactory levels of communication with
patients, colleagues and faculty members following the
10 Clinical cases
2 Case presentationns
10 Clinical cases
10 Clinical cases
2.2 Record professionally the patient general data and clinical
findings using the available clinical management electronic
system
2.3 Implement different restorative treatment plans using safely
and efficiently all available instruments, equipment and
restorative materials
3.1
CLOs TOPICS / TASKS
KNOWLEDGE & UNDERSTANDING
Recall the diagnostic procedures for restorative treatment
planning
Indicate the restorative material and techniques of different
restorative managements
Design professionally a restorative treatment plan for different
defects affecting hard tooth structure based on the data given
by the patient and the clinical findings

2. CONTENT WEIGHTING:
• The following questions will help you determine
the weight of each topic in the assessment:
1. How much of your lecture time do you spend
teaching this topic in your course?
2. How much time are examinees expected to
spend studying each topic area independently?
3. How much of the course textbook or lab time is
devoted to this topic?
4. How much of this topic will your students need
to know as they move through their training?
5. What other topics should get about the same
amount of test time as this topic?

3. CLASSIFYING TEST QUESTIONS:
• Classify your questions according to Bloom’s
Taxonomy of Educational Objectives into the
following categories:
1. Remembering
2. Understanding
3. Applying
4. Analyzing
5. Evaluating
6. Creating
• This allows for:
1. Easy pooling and tagging of questions
2. Find specific material from your pool more
efficiently and keep track of needs during the
item or case-writing process

Produce new or original work
design, assemble, construct, conjecture, develop, formulate, investigate, author
REMEMBER
UNDERSTAND
APPLY
ANALYZE
EVALUATE
CREATE
Recall facts and basic concepts
define, duplicate, list, memorize, repeat, state
Explain ideas or concepts
classify, describe, explain, identify, locate, recognize, report, select, translate
Use information in new situation
execute, implement, solve, use, demonstrate, interpret, operate, schedule, sketch
Draw connections among ideas
differentiate, organize, relate, compare, contrast, distinguish, examine, test
Justify a stand or decision
appraise, argue, defend, judge, select, support, value, critique, weigh
170

4. PUTTING IT ALL TOGETHER:
• Creating the blueprint

5. ITEMS ALIGNMENT & TAGGING:
• Create your items pool “Bank” and classify them to
be aligned with learning outcomes, then tagged
according to Bloom”s taxonomy and topic number

RDS 430
a) CLO 1.1
b) CLO 1.2
c) CLO 2.1
1. Remember
2. Understand
3. Apply
4. Analyze
5. Evaluate
6. Create
Topic 1
Topic 2
Topic 3
Topic 4
Topic 5
Topic 6
Topic 7
Topic 8
Topic 9

A Patient came back to you one week after you had finished a composite resin
restoration for a shallow simple occlusal cavity, complaining from tooth
sensitivity to biting and the clinical examination revealed a slight tooth mobility
that developed after tooth restoration which was originally normal
What is the possible cause of this condition?
a.    Improper bonding procedures
b.    Minor tooth crack
c.    Premature occlusal contact
d.    Periodontal pocket
C
.
5
.
T
6
-
1
.
C: CLO 2.1
5: Evaluate
T6: Topic 6
1: Question 1

Assessment in Dental Education.pdf

Assessment in Dental Education.pdf

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Assessment in Dental Education.pdf

Similar to Assessment in Dental Education.pdf (20)

More from Yasser Al-Wasifi

More from Yasser Al-Wasifi (20)

Recently uploaded

Recently uploaded (20)

Assessment in Dental Education.pdf