ACER U3 Session 2 - Challenges to making reliable and objective judgements.

1. Graduate Certificate in the
Assessment of Student Learning
Estimating Student Achievement
Challenges of making reliable and objective judgements

2. Reliable - dependable, consistent, trustworthy
Objective – impartial, detached, unbiased
Awareness of potential biases or judgement
‘errors’
2

3. 








Pre-judging
Confusing achievement with effort
Different standards different students
Cultural stereotyping
Gender stereotyping
‘halo’ effect
‘proximity’ error
‘central tendency’ error
‘severity/leniency’ error
3

4. Letting expectation rule over actual evidence of
learning when


A student exceeds our expectations
Doesn’t demonstrate the capacity we expect
To minimise potential to prejudge
Assess each piece of evidence of learning on
its own merits.
4

5. Effort does not always equate to ability
 bigger/longer is not necessarily better
 Appearance may not match response quality
To minimise potential to confuse achievement
with effort
Assess each piece of evidence of learning on
its own merits.
5

6. Building tolerance standards applied



‘More able’ students +/- 6mm
‘Average’ students +/- 8mm
‘Less able’ students +/- 10mm
Potentially fraught difference.
Might preclude some students meeting highest
standard.
6

7. Extensive research available

Klenowski & Gertz (2009)
 culture fair assessment

Kraiger & Ford (1985)
 74 studies race effect, bias to own race

Baker (2005)
 Potential cultural group sabotage

Perry & Delpit (1998)
 Potential to dismiss legitimate responses
7

8. 
Assuming girls and boys will perform at
higher/lower levels than each other

Gender effect on rating

Performance effect due to gender mix
Review items demonstrating diff.
8

9. 
Halo effect
◦ One aspect of performance influences rating on
another aspect

Proximity error
◦ Tendency to give similar rating to criteria located
near each other

‘central tendency’ error
◦ Rarely award very high or very low ratings
9

10. Comparability/reliability impacted

Tendency to be hard or easy marker
Congdon & McQueen (2000)
To minimise potential to be severe or lenient
assessor
Access to: work samples across levels; moderation;
and, professional dialogue.
10

11. 








Pre-judging
Confusing achievement with effort
Different standards different students
Cultural stereotyping
Gender stereotyping
‘halo’ effect
‘proximity’ error
‘central tendency’ error
‘severity/leniency’ error
11

12. 

Ensure fair and equitable assessment
practices
Learning to be established by the evidence
12

13. 


Note potential biases that may exist in your
context
Consider ways to expose biases
Consider ways to minimise biases/ ‘errors’
Share examples of best practice.
Initiate and build discussion, explore issues related
to providing fair and equitable assessment and
assessment results.
See participant requirements for further details.
13