Measuring the quality

1. Quality of Assessment

2. Issues
• 1. How can the need for learners to pass
assessment distort the curriculum
content and teaching & learning?
• 2. How far do we assess only what we
can grade - especially ‘numerically’?
• 3. Does assessment act as a gate-
keeping mechanism to exclude rather
than include?
• 4. Why are there tensions between
‘unseen exams/tests’ and coursework?

3. Previously we discussed
Assessment methods
• Initial/ diagnostic
• Formative
• Summative

4. Mastery and Developmental
Mastery Objectives
• Essential skills and knowledge which underpin course
studied
• Assessed on pass/fail basis
Developmental Objectives
• Apply skills and knowledge in structured way to subject
and wider curriculum- e.g. transferable skills
• Evaluation, analysis, interpretation, synthesis
• Assessed on formative or graded basis
• http://pandora.cii.wwu.edu/cii/resources/outcomes/objectives.asp

5. MM DD

6. Bloom

7. Read Dunn’s Paper
• identify which of the outcome types are
‘mastery’ and which are ‘developmental’
• Suggest appropriate assessment methods
in each section which are relevant to your
subject specialism

8. Judging assessment quality
VACSR

9. Reliability
Getting the assessment right
• Consistent- the same result each time
Methods to ensure reliability: (what do you think?)
• Marking scheme
• Answer guide
Certain types of assessments tend to be more reliable:
(?)
• Oral questions with answer guide
• Multiple choice
• Closed questions
• Assignments with suggested evidence --- TASK 2

10. Validity
Validity is all about getting the right assessment. If
an assessment is valid it is measuring what it is
supposed to be measuring.
For example, within the psychomotor domain
(physical/manual skills) you would want them to
demonstrate the skill rather than write about it.

11. Validity and Reliability
• Validity is concerned with getting the right
assessment and reliability is concerned
with getting the assessment right.
Reece, I. & Walker, S. (2000) Teaching, Training and Learning.
Business Education Publishers, Sunderland (p. 420)

12. Sufficiency
This is to do with
the amount of work
the students have
to do for you to
determine whether
they can do
something or know
something to a
particular level.

13. Authenticity
“Is it real?”
On a horticultural course is performing a gardening task
indoors in a pot is authentic?
Plagiarism
Also we need to ask, is the work their own.
We can assess whether work is authentic as you get a feel
for the level that your learners are at.
So the work (as far as is possible) needs to be in an
Authentic setting as well as authentic to the person
that has produced it.
• Task 5

14. Currency
• Is it up to date?

15. …in class…

16. Today’s Targets
• Review previous session’s content
• Evaluate assessment types using key
terminology
• Distinguish mastery from developmental
objectives and assessment methods
• Define and exemplify key assessment
quality terms

17. Group Task
Identify one formative assessment and one
summative assessment used by one
member of the group. For each one make
judgements about it according to reliability,
validity, authenticity, sufficiency and fairness
VASK Doc

18. Review
• Look at your notes and the activities
• Text
• - one thing that is clear and useful
• - one thing that’s a bit muddy still