3. There is
nothing to fear
from changes in
assessment practices
What are the purposes of assessment?
• to measure students’ preparedness for further study or
professional accreditation
, ,
• to rank students, relative to one another, for the
purposes of competitive scholarships or other
opportunities
• to provide feedback on student learning for both
students and staff
3
4. What are the purposes of assessment?
• to provide feedback
to provide feedback
on teaching for staff
• to define and protect
academic standards
• to direct students’
learning
What are your experiences
in assessment?
4
6. Steps in Designing Learning Outcomes
Example
Solving problems and developing plans (Identifying problems,
posing problems, defining problems, analysing data, reviewing,
designing experiments, planning, applying information)
designing experiments planning applying information)
•Problem scenario
•Group work
•Work‐based problem
p
•Prepare a committee of enquiry report
•Draft a research bid to a realistic brief
•Analyse a case
TQF
Steps in Designing Learning Outcomes
Example
Demonstrating knowledge and understanding (Recalling,
describing, reporting, recounting, recognising, identifying,
relating and interrelating)
relating and interrelating)
•Written examination
•Oral examination
•Essay
p
•Report
•Devise an encyclopaedia entry (Wiki)
•Short‐answer questions: true/false/ multiple‐choice questions
(paper‐based or computer‐aided assessment
TQF
6
8. Core Principles of Effective Assessment
Objectives for higher education assessment
1. Assessment that guides and encourages effective
approaches to learning
2. Assessment that validly and reliably measures
expected learning outcomes, in particular the higher‐
order learning that characterises higher education
3. Assessment and grading that define and protect
academic standards
Adapted from the Centre for the Study of Higher Education, University of Melbourne
Core Principles of Effective Assessment
Well designed assessment should …
1. set clear expectations
2. establish a reasonable
workload (one that does not
push students into rote
reproductive approaches to
study)
3. provide opportunities for
3 id t iti f
students to self‐monitor,
rehearse, practise and
receive feedback.
TQF
8
10. Core Principles of Effective Assessment
16 INDICATORS OF EFFECTIVE ASSESSMENT IN HIGHER
EDUCATION
A checklist for quality in student assessment
A h kli f li i d
1. Assessment is treated by staff and students as an integral component
of the entire teaching and learning process.
2. The multiple roles of assessment are recognised. The powerful
motivating effect of assessment requirements on students is
understood and assessment tasks are designed to foster valued
understood and assessment tasks are designed to foster valued
study habits.
3. There is a faculty/departmental policy that guides assessment
practices. Subject assessment is integrated into an overall plan for
course assessment.
Core Principles of Effective Assessment
4. There is a clear alignment between expected learning outcomes, what is
taught and learnt, and the knowledge, skills (and attitudes) assessed.
5. Assessment tasks assess the capacity to analyse and synthesis new
information and concepts rather than simply recall information which
has been presented.
h b d
6. A variety of assessment methods is employed so that the limitations of
particular methods are minimised.
7. Assessment tasks are designed to assess relevant generic skills as well
as subject‐specific knowledge and skills.
10
11. Core Principles of Effective Assessment
8. There is a steady progression in the complexity and demands of assessment
requirements in the later years of courses.
9. There is provision for student choice in assessment tasks and weighting at
certain times.
certain times
10. Student and staff workloads are considered in the scheduling and design of
assessment tasks.
11. Excessive assessment is avoided. Assessment tasks are designed to sample
student learning.
12. Assessment tasks are weighted to balance the developmental (‘formative’)
and judgemental (‘summative’) roles of assessment. Early low‐stakes, low‐
and judgemental (‘summative’) roles of assessment Early low stakes low
weight assessment is used to provide students with feedback.
Core Principles of Effective Assessment
13. Grades are calculated and reported on the basis of clearly articulated
learning outcomes and criteria for levels of achievement.
p y g g
14. Students receive explanatory and diagnostic feedback as well as grades.
15. Assessment tasks are checked to ensure there are no inherent biases
that may disadvantage particular student groups.
16. Plagiarism is minimised through careful task design, explicit education
and appropriate monitoring of academic honesty.
11
12. Core Principles of Effective Assessment
What students value in assessment
• Unambiguous expectations
• ‘Authentic’ tasks
• Choice and flexibility
Feedback
Why Feedback?
•Without feedback, mistakes go uncorrected and good
performance is not reinforced.
•Without feedback, our learners may not know how
they are doing, if they are doing well, or if there are
elements of their performance that need to be
l t f th i f th t dt b
improved so that they can be competent.
12
13. Feedback
•Minimal Feedback/Reinforcement
Mi i l F db k/R i f t
•Evaluation
•Feedback
How do They Differ?
Feedback
Reinforcement/
Minimal Feedback
•Statements expressing positive (or
•Statements expressing positive (or
negative) reaction to a behaviour
which aims to increase (or
decrease) the likelihood of that
behaviour happening again
–“That was a great presentation”
–“You need to work on your
presentation skills”
i kill ”
•Often mistaken for feedback
–Timing is similar ‐ immediate
13
14. Feedback
Evaluation
•Qualitative judgement which ranks a learner’s
Q lit ti j d t hi h k l ’
performance in comparison to other learners
– 3.7 for professionalism competence
•Often the only measure of performance visible to the
learner
•Usually given after the performance is over
Feedback
Feedback
•(Reinforcement or correction) + Explanation
•(Reinforcement or correction) + Explanation
•Keeps you on course to meet goals
•Allows you to adjust your course to meet goals
•Given immediately after the performance or at some
time soon after, when the learner still has time to
time soon after, when the learner still has time to
demonstrate improvement
14
15. Barriers to Feedback
…as it is part of other training cultures
Barriers to Feedback
Barriers
•It is uncomfortable
•It is uncomfortable
–Avoid confrontation
–Learner will not be
receptive
–No one ever gave me
feedback
–Not quite sure how to do it
–I hated getting feedback
15
16. Barriers to Feedback
•I’m not sure of the goals or the
g
expected behaviour
•Not sure that the observed
behaviour is really a problem
–Need to see it twice
•It’s not my job
•It’s not that important
•There’s not enough time
Essential Components of Feedback
What are they?
•What was done well
•What could be done better
p
•What could be done to improve next time
16
17. Characteristics of Feedback
•Well timed and expected
•Based on first hand data
•Phrased in descriptive
language, based on specific
remediable behaviours
•Should be undertaken with
Should be undertaken with
teacher and learner working
as allies, with common goals
Ende J. Feedback in Clinical Medical Education. JAMA 1983;250:777-781.
The Old Feedback Sandwich
Praise
Criticism
Praise
Is it more palatable?
l bl ?
17
18. The NEW Feedback Sandwich
Ask
Tell
Ask
Adaptation of “The New Feedback Sandwich,’ common in patient-physician communication
literature; adapted by Lyuba Konopasek, MD, for use in feedback settings.
The New ‘Feedback Sandwich’
Ask
•Ask learner to assess own performance first
–What went well and what could have gone better?
–What were their goals?
–Have they ever seen a problem like this before?
•Begins a conversation
g
•Assesses learner’s level of insight
•Useful for second‐hand feedback
18
19. The New ‘Feedback Sandwich’
Tell
• Tell what you observed: diagnosis and
explanation
–React to the learner’s observation
–Include both positive and corrective elements
–“I observed….”
–Give reasons in the context of well-defined shared goals
The New ‘Feedback Sandwich’
Ask (again)
•Ask about recipients understanding
and strategies for improvement
–What could you do differently?
Give own suggestions
–Give own suggestions
–Perhaps even replay parts of the encounter: “show me”
–Commit to monitoring improvement together
19
20. The ‘Feedback Sandwich’
Limit the Quantity!!!
Checklist: The Feedback Dialogue
Ask learner to assess own performance first
What went well and what could have gone better?
Tell what you observed:diagnosis and explanation
React to the learner’s observation
Include both positive and constructive elements
Give reasons in the context of well‐defined shared goals
Regulate quantity
Ask about recipients understanding and strategies for improvement
What could you do differently?
What could you do differently?
Give own suggestions
Perhaps even replay parts of the encounter ‐ show me
Commit to monitoring improvement together
Adaptation of “The New Feedback Sandwich,’ common in patient-physician communication literature;
adapted by Lyuba Konopasek, MD, for use in medical resident feedback settings.
20
21. ACTIVITY
1. Return to your groups and yesterday’s scenarios
2. Look at your learner outcome.
Consider assessment needs and opportunities,
then write down suitable assessment activities
3. What feedback will you provide to students of
3 Wh f db k ill id d f
the subject?
Adaptation of “The New Feedback Sandwich,’ common in patient-physician communication literature;
adapted by Lyuba Konopasek, MD, for use in medical resident feedback settings.
Scenario 2.
Degree: Graduate Diploma of Business
Subject: AEC and Thailand
Learning Outcome: By the completion of this subject students
will be able to analyse the political,
social and economic impacts of the AEC
social and economic impacts of the AEC
TQF 42
21
22. Scenario 3.
Degree: Master of Arts in Political Science
Subject: Globalisation and Diversity
Learning Outcome: By the completion of this subject students
will be able to effectively apply
understanding of globalisation concepts to
understanding of globalisation concepts to
issues of cultural diversity, social justice
and ethical accountability
TQF 43
Scenario 4.
Degree: Bachelor of Science in Multimedia Studies
Subject: Website Development
Learning Outcome: By the completion of this subject students
will be able to design user‐friendly and
interactive web sites
interactive web sites
TQF 44
22
23. Scenario 5.
Degree: PhD
Subject: Research Methodology
Learning Outcome: By the completion of this subject students
will be able to critically evaluate a research
project and select appropriate research
project and select appropriate research
methods to undertake the study
TQF 45
Scenario 6.
Degree: Doctor of Medicine (MD)
Subject: Treatment of Bacterial Infection
Learning Outcome: By the completion of this subject students
will be able to correctly choose and deliver
effective treatment for bacterial infections
effective treatment for bacterial infections
TQF 46
23
