This document discusses assessment in higher education. It provides an overview of learning outcomes, linking learning outcomes to assessment, and using assessment for feedback. The document discusses the purposes of assessment, including measuring student learning, providing feedback, and defining academic standards. It also outlines principles of effective assessment, such as aligning assessment with learning outcomes and using a variety of assessment methods. The document emphasizes using assessment to guide student learning through feedback.

1. Workshop on
THE LINKAGES OF LEARNING OUTCOMES,
LEARNING ACTIVITIES AND ASSESSMENTS IN
HIGHER EDUCATION
Australian and Thai Perspectives
THAILAND 19 28 June 2012
THAILAND 19 – 28 June 2012
Thailand Qualification Framework
Learning Outcomes, Assessment and Feedback
Nattavud Pimpa
Tim Moore
1

2. Sessions 3 & 4: Learning Objective
By the end of these two sessions the
participants will be able to link learning
participants will be able to link learning
outcomes with assessment, and strengthen
teaching practice through effective feedback.
TQF
What are the purposes of assessment?
2

3. There is
nothing to fear
from changes in
assessment practices
What are the purposes of assessment?
• to measure students’ preparedness for further study or
professional accreditation
, ,
• to rank students, relative to one another, for the
purposes of competitive scholarships or other
opportunities
• to provide feedback on student learning for both
students and staff
3

4. What are the purposes of assessment?
• to provide feedback
to provide feedback
on teaching for staff
• to define and protect
academic standards
• to direct students’
learning
What are your experiences
in assessment?
4

5. Steps in Designing Learning Outcomes
Step 5: Determining Assessment
•Fair, equitable and transparent.
• Must reflect the learning outcomes
•Incorporate a range of types or modes of
assessment appropriate to the nature of the
unit, method of delivery and the students
involved
•Design appropriate assessment strategies
that ideally engage the learners in activities
they can relate to real‐life or workplace
situations
TQF
Steps in Designing Learning Outcomes
Example
Thinking critically and making judgments (Developing arguments,
reflecting, evaluating, assessing, judging)
•Essay
•Report
•Journal
•Letter of advice to … (about policy, public health matters … )
•Present a case for an interest group
•Prepare a committee briefing paper for a specific meeting
•Book review (or article) for a particular journal
•Write a newspaper article for a foreign newspaper
TQF
5

6. Steps in Designing Learning Outcomes
Example
Solving problems and developing plans (Identifying problems,
posing problems, defining problems, analysing data, reviewing,
designing experiments, planning, applying information)
designing experiments planning applying information)
•Problem scenario
•Group work
•Work‐based problem
p
•Prepare a committee of enquiry report
•Draft a research bid to a realistic brief
•Analyse a case
TQF
Steps in Designing Learning Outcomes
Example
Demonstrating knowledge and understanding (Recalling,
describing, reporting, recounting, recognising, identifying,
relating and interrelating)
relating and interrelating)
•Written examination
•Oral examination
•Essay
p
•Report
•Devise an encyclopaedia entry (Wiki)
•Short‐answer questions: true/false/ multiple‐choice questions
(paper‐based or computer‐aided assessment
TQF
6

7. Steps in Designing Learning Outcomes
Example
Designing, creating, performing (Imagining, visualising,
designing, producing, creating, innovating,
performing)
•Portfolio
•Performance
•Presentation
•Presentation
•‘Hypothetical’
•Projects
School/Department/Area
Steps in Designing Learning Outcomes
Check the Alignment
Checking to see that assessment and learning
outcomes align requires determining whether:
outcomes align requires determining whether:
• the assessment includes knowledge, understanding
or skills not in the learning outcomes then review the
learning outcomes. If after review you are still happy
with the learning outcomes then you will have to
change the assessment;
change the assessment;
• the assessment misses knowledge, understanding or
skills in the learning outcomes then review the
assessment.
TQF 14
7

8. Core Principles of Effective Assessment
Objectives for higher education assessment
1. Assessment that guides and encourages effective
approaches to learning
2. Assessment that validly and reliably measures
expected learning outcomes, in particular the higher‐
order learning that characterises higher education
3. Assessment and grading that define and protect
academic standards
Adapted from the Centre for the Study of Higher Education, University of Melbourne
Core Principles of Effective Assessment
Well designed assessment should …
1. set clear expectations
2. establish a reasonable
workload (one that does not
push students into rote
reproductive approaches to
study)
3. provide opportunities for
3 id t iti f
students to self‐monitor,
rehearse, practise and
receive feedback.
TQF
8

9. Core Principles of Effective Assessment
Core Principles of Effective Assessment
9

10. Core Principles of Effective Assessment
16 INDICATORS OF EFFECTIVE ASSESSMENT IN HIGHER
EDUCATION
A checklist for quality in student assessment
A h kli f li i d
1. Assessment is treated by staff and students as an integral component
of the entire teaching and learning process.
2. The multiple roles of assessment are recognised. The powerful
motivating effect of assessment requirements on students is
understood and assessment tasks are designed to foster valued
understood and assessment tasks are designed to foster valued
study habits.
3. There is a faculty/departmental policy that guides assessment
practices. Subject assessment is integrated into an overall plan for
course assessment.
Core Principles of Effective Assessment
4. There is a clear alignment between expected learning outcomes, what is
taught and learnt, and the knowledge, skills (and attitudes) assessed.
5. Assessment tasks assess the capacity to analyse and synthesis new
information and concepts rather than simply recall information which
has been presented.
h b d
6. A variety of assessment methods is employed so that the limitations of
particular methods are minimised.
7. Assessment tasks are designed to assess relevant generic skills as well
as subject‐specific knowledge and skills.
10

11. Core Principles of Effective Assessment
8. There is a steady progression in the complexity and demands of assessment
requirements in the later years of courses.
9. There is provision for student choice in assessment tasks and weighting at
certain times.
certain times
10. Student and staff workloads are considered in the scheduling and design of
assessment tasks.
11. Excessive assessment is avoided. Assessment tasks are designed to sample
student learning.
12. Assessment tasks are weighted to balance the developmental (‘formative’)
and judgemental (‘summative’) roles of assessment. Early low‐stakes, low‐
and judgemental (‘summative’) roles of assessment Early low stakes low
weight assessment is used to provide students with feedback.
Core Principles of Effective Assessment
13. Grades are calculated and reported on the basis of clearly articulated
learning outcomes and criteria for levels of achievement.
p y g g
14. Students receive explanatory and diagnostic feedback as well as grades.
15. Assessment tasks are checked to ensure there are no inherent biases
that may disadvantage particular student groups.
16. Plagiarism is minimised through careful task design, explicit education
and appropriate monitoring of academic honesty.
11

12. Core Principles of Effective Assessment
What students value in assessment
• Unambiguous expectations
• ‘Authentic’ tasks
• Choice and flexibility
Feedback
Why Feedback?
•Without feedback, mistakes go uncorrected and good
performance is not reinforced.
•Without feedback, our learners may not know how
they are doing, if they are doing well, or if there are
elements of their performance that need to be
l t f th i f th t dt b
improved so that they can be competent.
12

13. Feedback
•Minimal Feedback/Reinforcement
Mi i l F db k/R i f t
•Evaluation
•Feedback
How do They Differ?
Feedback
Reinforcement/
Minimal Feedback
•Statements expressing positive (or
•Statements expressing positive (or
negative) reaction to a behaviour
which aims to increase (or
decrease) the likelihood of that
behaviour happening again
–“That was a great presentation”
–“You need to work on your
presentation skills”
i kill ”
•Often mistaken for feedback
–Timing is similar ‐ immediate
13

14. Feedback
Evaluation
•Qualitative judgement which ranks a learner’s
Q lit ti j d t hi h k l ’
performance in comparison to other learners
– 3.7 for professionalism competence
•Often the only measure of performance visible to the
learner
•Usually given after the performance is over
Feedback
Feedback
•(Reinforcement or correction) + Explanation
•(Reinforcement or correction) + Explanation
•Keeps you on course to meet goals
•Allows you to adjust your course to meet goals
•Given immediately after the performance or at some
time soon after, when the learner still has time to
time soon after, when the learner still has time to
demonstrate improvement
14

15. Barriers to Feedback
…as it is part of other training cultures
Barriers to Feedback
Barriers
•It is uncomfortable
•It is uncomfortable
–Avoid confrontation
–Learner will not be
receptive
–No one ever gave me
feedback
–Not quite sure how to do it
–I hated getting feedback
15

16. Barriers to Feedback
•I’m not sure of the goals or the
g
expected behaviour
•Not sure that the observed
behaviour is really a problem
–Need to see it twice
•It’s not my job
•It’s not that important
•There’s not enough time
Essential Components of Feedback
What are they?
•What was done well
•What could be done better
p
•What could be done to improve next time
16

17. Characteristics of Feedback
•Well timed and expected
•Based on first hand data
•Phrased in descriptive
language, based on specific
remediable behaviours
•Should be undertaken with
Should be undertaken with
teacher and learner working
as allies, with common goals
Ende J. Feedback in Clinical Medical Education. JAMA 1983;250:777-781.
The Old Feedback Sandwich
Praise
Criticism
Praise
Is it more palatable?
l bl ?
17

18. The NEW Feedback Sandwich
Ask
Tell
Ask
Adaptation of “The New Feedback Sandwich,’ common in patient-physician communication
literature; adapted by Lyuba Konopasek, MD, for use in feedback settings.
The New ‘Feedback Sandwich’
Ask
•Ask learner to assess own performance first
–What went well and what could have gone better?
–What were their goals?
–Have they ever seen a problem like this before?
•Begins a conversation
g
•Assesses learner’s level of insight
•Useful for second‐hand feedback
18

19. The New ‘Feedback Sandwich’
Tell
• Tell what you observed: diagnosis and
explanation
–React to the learner’s observation
–Include both positive and corrective elements
–“I observed….”
–Give reasons in the context of well-defined shared goals
The New ‘Feedback Sandwich’
Ask (again)
•Ask about recipients understanding
and strategies for improvement
–What could you do differently?
Give own suggestions
–Give own suggestions
–Perhaps even replay parts of the encounter: “show me”
–Commit to monitoring improvement together
19

20. The ‘Feedback Sandwich’
Limit the Quantity!!!
Checklist: The Feedback Dialogue
Ask learner to assess own performance first
What went well and what could have gone better?
Tell what you observed:diagnosis and explanation
React to the learner’s observation
Include both positive and constructive elements
Give reasons in the context of well‐defined shared goals
Regulate quantity
Ask about recipients understanding and strategies for improvement
What could you do differently?
What could you do differently?
Give own suggestions
Perhaps even replay parts of the encounter ‐ show me
Commit to monitoring improvement together
Adaptation of “The New Feedback Sandwich,’ common in patient-physician communication literature;
adapted by Lyuba Konopasek, MD, for use in medical resident feedback settings.
20

21. ACTIVITY
1. Return to your groups and yesterday’s scenarios
2. Look at your learner outcome.
Consider assessment needs and opportunities,
then write down suitable assessment activities
3. What feedback will you provide to students of
3 Wh f db k ill id d f
the subject?
Adaptation of “The New Feedback Sandwich,’ common in patient-physician communication literature;
adapted by Lyuba Konopasek, MD, for use in medical resident feedback settings.
Scenario 2.
Degree: Graduate Diploma of Business
Subject: AEC and Thailand
Learning Outcome: By the completion of this subject students
will be able to analyse the political,
social and economic impacts of the AEC
social and economic impacts of the AEC
TQF 42
21

22. Scenario 3.
Degree: Master of Arts in Political Science
Subject: Globalisation and Diversity
Learning Outcome: By the completion of this subject students
will be able to effectively apply
understanding of globalisation concepts to
understanding of globalisation concepts to
issues of cultural diversity, social justice
and ethical accountability
TQF 43
Scenario 4.
Degree: Bachelor of Science in Multimedia Studies
Subject: Website Development
Learning Outcome: By the completion of this subject students
will be able to design user‐friendly and
interactive web sites
interactive web sites
TQF 44
22

23. Scenario 5.
Degree: PhD
Subject: Research Methodology
Learning Outcome: By the completion of this subject students
will be able to critically evaluate a research
project and select appropriate research
project and select appropriate research
methods to undertake the study
TQF 45
Scenario 6.
Degree: Doctor of Medicine (MD)
Subject: Treatment of Bacterial Infection
Learning Outcome: By the completion of this subject students
will be able to correctly choose and deliver
effective treatment for bacterial infections
effective treatment for bacterial infections
TQF 46
23

24. Feedback
What about feedback to teachers…?
Afterthoughts
48
24

25. Evaluations
THANK YOU
49
25