Ipsative assessment
and motivation of
distance learners
Gwyneth Hughes and Megan Crawford
2
Contents
1 Why does assessment need radical reform?
2 Ipsative assessment -what are the possible benefits?
3 Results of ...
Why assessment reform?
Problems with current assessment practices
– Dominance of summative assessment and grading over for...
What if an obsession with criteria and
standards is at the root of the
problem?
‘if we wish to discover the truth about an...
Ipsative assessment as an alternative
to criteria-referenced assessment
5
Personal best
6
7
What is ipsative assessment ?
An ipsative assessment compares current performance with a previous
performance.
Ipsative ...
8
Possible benefits of ipsative
assessment
• Improves the usability of feedback e.g. through closing feedback loops
– See ...
Possible use with distance learners
• Can aid motivation at a distance as part of supportive open learning.
• Written feed...
Study on ipsative feedback and
learner motivation
MA in Applied Educational Leadership and Management (AELM) is
delivered ...
Tutor feedback profile
11
Criteria-related
Task-orientated
Criteria-related
Generic
Ipsative
Task-orientated
Ipsative
Gene...
Examples of feedback
Typical comments comparing the work with assessment criteria and
standards were:
“Have a look at the ...
Ipsative feedback: student views
• Feedback was perceived as vague lacking clarity
• Learners used grades and feedback to ...
Tutor feedback profile
14
Criteria-related
Task-orientated
Criteria-related
Generic
Ipsative
Task-orientated
Ipsative
Gene...
Recommended profile
15
Criteria-related
Task-orientated
Criteria-related
Generic
Ipsative
Task-orientated
Ipsative
Generic
Ipsative feedback: tutor views
“…if one tutor supervises the whole process of an assignment from the
draft to the final su...
Ipsative grades: student views
“I fully appreciate that my final grade will not be based on my efforts,
motivation or prog...
Ipsative grades: tutor views:
“ It (ipsative grading) is a radical proposal and…….It might motivate
(lower achievers)…the ...
Recommendations
Ipsative feedback could be introduced both to motivate students and to
reduce obsession with grades and cr...
g.hughes@ioe.ac.uk
mcrawford@brookes.ac.uk
20
21
References
Biggs, J. 2003. Teaching for Quality Learning at University.
Buckingham: Society for Research into Higher ed...
Upcoming SlideShare
Loading in …5
×

RIDE 2010 presentation - Ipsative assessment and motivation of distance learners

1,480 views

Published on

Research in Distance Education: impact on practice conference, 27 October 2010. Presentation in Assessment Strand by Dr Gwyneth Hughes, Institute of Education and Dr Megan Crawford, Oxford Brookes University.
More details at www.cde.london.ac.uk.

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,480
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • “They were like drug addicts, except they were addicted to grades - even though they knew it was bad for them,” said Professor Wiliam. In TES

    27 August, 2010 | By: Kerra Maddern
    Low-cost programme galvanises pupils with rewards, no grading and PE every morning



    More clarity useful but nor enough if learners have to close a huge gap to get there.

    Nicol and M Dick 7 principles include the importance of giving developmetnal feedback or feedforward –guiding students on what to do next. Retunr to this later.
  • Feedback on draft assignments was common practice.
  • Based on Hattie and Timperley model of feedback we distinguished feedback on where the student is now from feedforward on what the student needs to do next.
    For the feedback we used a tool to identify feedback statements as criteria-related or ipsative and task orientated (referring to this assignment) or generic (as could apply to other assignments like how to write correct references). This gives four possible categories.

    We counted statements in the sample in each category but because these numbers do not have numerical value –they are estimates depending on how the researchers interpreted the statements- we produced a profile for the feedback which shows the most common type of feedback in this sample.

    This was criteria related sometimes even quoting the written criteria and generic in that if referred to

    Virtually no ipsative feedback that mentioned previous work or progress since draft writign

  • Generic statement referring to the criteria of using critical analysis does not tell the student where this analysis was missed as it does not give any task related examples from the essay. Some students stated in interivews that they found the feedback unhelpful becuase it was too vague.
  • Many had little idea of whether or not they were making progress and would welcome ipsative feedback:
  • Based on Hattie and Timperley model of feedback we distinguished feedback on where the student is now from feedforward on what the student needs to do next.
    For the feedback we used a tool to identify feedback statements as criteria-related or ipsative and task orientated (referring to this assignment) or generic (as could apply to other assignments like how to write correct references). This gives four possible categories.

    We counted statements in the sample in each category but because these numbers do not have numerical value –they are estimates depending on how the researchers interpreted the statements- we produced a profile for the feedback which shows the most common type of feedback in this sample.

    This was criteria related sometimes even quoting the written criteria and generic in that if referred to

    Virtually no ipsative feedback that mentioned previous work or progress since draft writign

  • Ipsative grades were less popular as even those who would get higher grades felt they were resigned to the current system that only the best are rewarded – through high marks.
  • A shift towards ipsative feedback might help learners with self –assessment and could reduce dependency on grades and performance as the measurement of progress. Learners find this idea motivating.

    Ipsative grades could be introduced to motivate all learners, but would need to be carefully explained to learners who are very familiar with the more dominant criteria-referenced system and concerned about fairness.

    Any ipsative assessment should be incorporated into the assessment regime so that it does not add additional work for the assessors.
    Assessors may need staff development to appreciate the benefits of ipsative assessment and to be able to provide ipsative feedback within an adjusted assessment framework.
  • RIDE 2010 presentation - Ipsative assessment and motivation of distance learners

    1. 1. Ipsative assessment and motivation of distance learners Gwyneth Hughes and Megan Crawford
    2. 2. 2 Contents 1 Why does assessment need radical reform? 2 Ipsative assessment -what are the possible benefits? 3 Results of a study on ipsative feedback for distance learners 4 Recommendations
    3. 3. Why assessment reform? Problems with current assessment practices – Dominance of summative assessment and grading over formative (Gibbs, 2006) – Assessment can be demotivating (Broadfoot, 1996) – Feedback is not helpful for students so they don’t act (Lizzio & Wilson 2008) Suggestions for improving feedback practice – More clarity over standards and criteria – More dialogue – Improved formative feedback (Nicol & Macfarlane-Dick, 2006) 3
    4. 4. What if an obsession with criteria and standards is at the root of the problem? ‘if we wish to discover the truth about an educational system, we must first look to its assessment …’ (Rowntree 1987) 4
    5. 5. Ipsative assessment as an alternative to criteria-referenced assessment 5
    6. 6. Personal best 6
    7. 7. 7 What is ipsative assessment ? An ipsative assessment compares current performance with a previous performance. Ipsative feedback Ipsative grades
    8. 8. 8 Possible benefits of ipsative assessment • Improves the usability of feedback e.g. through closing feedback loops – See handout • Links up generic feedback across modules in a course • Improves self-esteem by rewarding progress • Ipsative grades might motivate learners to act on feedback
    9. 9. Possible use with distance learners • Can aid motivation at a distance as part of supportive open learning. • Written feedback is particularly important to students learning wholly at a distance- can help them to respond in revisions and following pieces of work. • The tutor can take on a coaching role in a very direct way and help their students personal response to feedback which at a distance may be even more tied up with emotions, identity and feelings of self-worth than F2F. 9
    10. 10. Study on ipsative feedback and learner motivation MA in Applied Educational Leadership and Management (AELM) is delivered wholly online to a fixed cohort. Our study looked at three tutors on the two core modules, where tutors had had specific training in giving good formative feedback. Module had been commended by External Examiner for its high quality written feedback. 10
    11. 11. Tutor feedback profile 11 Criteria-related Task-orientated Criteria-related Generic Ipsative Task-orientated Ipsative Generic
    12. 12. Examples of feedback Typical comments comparing the work with assessment criteria and standards were: “Have a look at the Assessment Criteria P. 37 to 39 of the Course Handbook: 1) Criteria A: you show a good level of understanding of the key issues and have begun to present extracts from the relevant literature..” (Tutor B) “What then might have taken the assignment to the next grade level? … you might have engaged rather more in a critical analysis of material in this field.” (Tutor A) 12
    13. 13. Ipsative feedback: student views • Feedback was perceived as vague lacking clarity • Learners used grades and feedback to self-assess on making progress but some would like more help e.g. “I do think about my own progress but of course I am never entirely confident in my own judgment of this!” “ if someone would read my first and second assignment and examine my progress as a paper writer and give me general comments about how I did or didn’t improve and what I needed to focus on that will be very helpful.” 13
    14. 14. Tutor feedback profile 14 Criteria-related Task-orientated Criteria-related Generic Ipsative Task-orientated Ipsative Generic
    15. 15. Recommended profile 15 Criteria-related Task-orientated Criteria-related Generic Ipsative Task-orientated Ipsative Generic
    16. 16. Ipsative feedback: tutor views “…if one tutor supervises the whole process of an assignment from the draft to the final submission, then ipsative assessment can be included fairly easily” Concerns were raised about needing staff development and that a new assessment approach might be time consuming. 16
    17. 17. Ipsative grades: student views “I fully appreciate that my final grade will not be based on my efforts, motivation or progress but simply on the work I submit, and that students who seem to have made far less effort may end up with better final grades if their work is of a higher quality. That is the nature of academic qualifications and it should remain this way if the end qualification is to maintain its current status as a well-respected academic certificate.” 17
    18. 18. Ipsative grades: tutor views: “ It (ipsative grading) is a radical proposal and…….It might motivate (lower achievers)…the only way I would see it working is if they get a grade for effort and for achievement. But I doubt that the effort would count towards the final mark. I haven’t had enough time to think about it.” 18
    19. 19. Recommendations Ipsative feedback could be introduced both to motivate students and to reduce obsession with grades and criteria Any ipsative grades would need careful explanation of the benefits Ipsative assessment would need to be included in the assessment regime and not provide extra work Assessors will probably need staff development and guidance 19
    20. 20. g.hughes@ioe.ac.uk mcrawford@brookes.ac.uk 20
    21. 21. 21 References Biggs, J. 2003. Teaching for Quality Learning at University. Buckingham: Society for Research into Higher education and Open University Press. Broadfoot, P. 1996. Education, Assessment and Society. Buckingham: Open University Press. Carless, D. 2006. Differing perceptions in the feedback process. Studies in Higher Education. 3, no. 2: 219-233. Hughes, G. & Crawford, M. 2009. Challenging Higher Education: knowledge, policy and practice. Society for Research in Higher Education conference papers, Newport, Wales. Nicol, D. & Macfarlane,-Dick, D. 2006. Formative assessment and self regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education 31 no. 2: 199-218. Institute of Education University of London 20 Bedford Way London WC1H 0AL Tel +44 (0)20 7612 6000 Fax +44 (0)20 7612 6126 Email info@ioe.ac.uk Web www.ioe.ac.uk

    ×