Cracking the challenges of
‘assessment for learning’
Tansy Jessop
LSE Education Symposium
10 May 2017
Today’s session
• Brief overview of TESTA
• Three themes
–Variations in assessment patterns
–High summative: low formative
–Disconnected feedback
• Towards an assessment paradigm shift
Mixed methods approach
Programme
Team
Meeting
Assessment
Experience
Questionnaire
(AEQ)
TESTA
Programme
Audit
Student
Focus Groups
TESTA….
“…is a way of thinking
about assessment and
feedback”
Graham Gibbs
A modular degree
Defining the terms
• Summative assessment carries a grade which
counts toward the degree classification.
• Formative assessment does not count
towards the degree, elicits feedback and is
required.
1. Variations in assessment patterns
• What is striking for
you about this data?
• How does it compare
with your context?
• Does variation
matter?
Characteristic Range
Summative 12 -227
Formative 0 - 116
Varieties of assessment 5 - 21
Proportion of examinations 0% - 87%
Time to return marks & feedback 10 - 42 days
Volume of oral feedback 37 -1800 minutes
Volume of written feedback 936 - 22,000 words
Variations in assessment diets (n=73 UG degrees in 14 UK universities)
Patterns on three year UG degrees
(n=73 programmes in 14 universities)
Characteristic Low Medium High
Volume of summative
assessment
Below 33 40-48 More than 48
Volume of formative only Below 1 5-19 More than 19
% of tasks by examinations Below 11% 22-31% More than 31%
Variety of assessment
methods
Below 8 11-15 More than 15
Written feedback in words Less than 3,800 6,000-7,600 More than 7,600
2. High summative: low formative
• High summative on UK, Irish, NZ and Indian degrees
• Summative as a ‘pedagogy of control’
• Low formative: ratio of 1:8 summative to formative
• Weakly practised and understood
Assessment Arms Race
What students say about summative
• A lot of people don’t do wider reading. You just focus
on your essay question.
• In Weeks 9 to 12 there is hardly anyone in our
lectures. I'd rather use those two hours of lectures to
get the assignment done.
• It’s been non-stop assignments, and I’m now free of
assignments until the exams – I’ve had to rush every
piece of work I’ve done.
What students say about formative
• If there weren’t loads of other assessments, I’d do it.
• If there are no actual consequences of not doing it,
most students are going to sit in the bar.
• It’s good to know you’re being graded because you
take it more seriously.
• The lecturers do formative assessment but we don’t
get any feedback on it.
Case Study 1: ‘Assessment Arms Race’
climb down
• All UG courses in business school
• Change from 2 x summative and 0 formative
per module
• …to 1 x summative and 3 x formative
• Formative required by all students
• Systematic shift: experimentation is less risky
together
Case Study 2: Public domain
• Education, Sociology and PGCert in HE degrees
• Problem: silent seminar, students not reading
• Blogging on current academic texts
• Threads and live discussion
• Linked to summative
Case study 3: Authentic, research-informed
• Problem: students’ lack of discrimination about
sources
• Students bring 1 x book, 1 x chapter, 1 x journal
article, 2 x pop culture articles
• Justify choices to group
• Reach consensus about five best sources
Learning-oriented summative?
https://www.youtube.com/watch?v=ZVFwQzlVFy0
3. Disconnected feedback
Lose-lose situation
It was heavy, tons of marking for
the tutor. It was such hard work.
It was criminal.
Media Course Leader
I’m really bad at reading
feedback. I’ll look at the mark
and then be like ‘well stuff it, I
can’t do anything about it’
Student, TESTA focus group
What students say…
It’s difficult because your assignments are so detached
from the next one you do for that subject. They don’t
relate to each other.
Because it’s at the end of the module, it doesn’t feed into
our future work.
Because they have to mark so many that our essay
becomes lost in the sea that they have to mark.
It was like ‘Who’s Holly?’ It’s that relationship where
you’re just a student.
Turning feedback into a dialogue
• Who starts the dialogue?
• Iterative cycles of reflection across modules
• Quick generic feedback
• Feedback synthesis tasks
• Peer feedback and self-evaluation
• Technology: audio, screencast and blogging
• From feedback as ‘telling’…
• … to feedback as clarifying conversation
From this educational paradigm…
Transmission Model
Social Constructivist Model
References
Barlow, A. and Jessop, T. 2016. “You can’t write a load of rubbish”: Why blogging works as formative
assessment. Educational Development. 17(3), 12-15. SEDA.
Berg, M. and Seeber, B. (2016) The Slow Professor: Challenging the Culture of Speed in the Academy.
Toronto. University of Toronto Press.
Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of
design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712.
Freire, P. 1972. Pedagogy of the Oppressed. Penguin Books. Harmondsworth.
Gibbs, G. & Simpson, C. (2004) Conditions which assessment supports students' learning. Learning and
Teaching in Higher Education. 1(1): 3-31.
Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2015) ‘An assessment arms race and its fallout:
High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Education 40(4)
528-541.
Jessop, T. and Tomas, C. 2016 The implications of programme assessment on student learning. Assessment
and Evaluation in Higher Education. Published online 2 August 2016.
Jessop, T. 2016. Seven years and still no itch – why TESTA keeps going. Educational Developments, 17(3) 5-8.
SEDA.
Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student learning: a
comparative study. Studies in Higher Education. Studies in Higher Education. 41(4) 696-711.
Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale
study of students’ learning in response to different assessment patterns. Assessment and Evaluation in
Higher Education. 39(1) 73-88.
Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher
education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517.
Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional Science,
18(2), pp. 119–144.
Torrance, H. (2007) Assessment as learning? How the use of explicit learning objectives, assessment criteria
and feedback in post-secondary education and training can come to dominate learning. Assessment in
Education 14(3) 281–294

LSE Assessment

  • 1.
    Cracking the challengesof ‘assessment for learning’ Tansy Jessop LSE Education Symposium 10 May 2017
  • 2.
    Today’s session • Briefoverview of TESTA • Three themes –Variations in assessment patterns –High summative: low formative –Disconnected feedback • Towards an assessment paradigm shift
  • 4.
  • 5.
    TESTA…. “…is a wayof thinking about assessment and feedback” Graham Gibbs
  • 6.
  • 7.
    Defining the terms •Summative assessment carries a grade which counts toward the degree classification. • Formative assessment does not count towards the degree, elicits feedback and is required.
  • 8.
    1. Variations inassessment patterns • What is striking for you about this data? • How does it compare with your context? • Does variation matter?
  • 9.
    Characteristic Range Summative 12-227 Formative 0 - 116 Varieties of assessment 5 - 21 Proportion of examinations 0% - 87% Time to return marks & feedback 10 - 42 days Volume of oral feedback 37 -1800 minutes Volume of written feedback 936 - 22,000 words Variations in assessment diets (n=73 UG degrees in 14 UK universities)
  • 10.
    Patterns on threeyear UG degrees (n=73 programmes in 14 universities) Characteristic Low Medium High Volume of summative assessment Below 33 40-48 More than 48 Volume of formative only Below 1 5-19 More than 19 % of tasks by examinations Below 11% 22-31% More than 31% Variety of assessment methods Below 8 11-15 More than 15 Written feedback in words Less than 3,800 6,000-7,600 More than 7,600
  • 11.
    2. High summative:low formative • High summative on UK, Irish, NZ and Indian degrees • Summative as a ‘pedagogy of control’ • Low formative: ratio of 1:8 summative to formative • Weakly practised and understood
  • 12.
  • 13.
    What students sayabout summative • A lot of people don’t do wider reading. You just focus on your essay question. • In Weeks 9 to 12 there is hardly anyone in our lectures. I'd rather use those two hours of lectures to get the assignment done. • It’s been non-stop assignments, and I’m now free of assignments until the exams – I’ve had to rush every piece of work I’ve done.
  • 14.
    What students sayabout formative • If there weren’t loads of other assessments, I’d do it. • If there are no actual consequences of not doing it, most students are going to sit in the bar. • It’s good to know you’re being graded because you take it more seriously. • The lecturers do formative assessment but we don’t get any feedback on it.
  • 15.
    Case Study 1:‘Assessment Arms Race’ climb down • All UG courses in business school • Change from 2 x summative and 0 formative per module • …to 1 x summative and 3 x formative • Formative required by all students • Systematic shift: experimentation is less risky together
  • 16.
    Case Study 2:Public domain • Education, Sociology and PGCert in HE degrees • Problem: silent seminar, students not reading • Blogging on current academic texts • Threads and live discussion • Linked to summative
  • 17.
    Case study 3:Authentic, research-informed • Problem: students’ lack of discrimination about sources • Students bring 1 x book, 1 x chapter, 1 x journal article, 2 x pop culture articles • Justify choices to group • Reach consensus about five best sources
  • 18.
  • 20.
  • 21.
    Lose-lose situation It washeavy, tons of marking for the tutor. It was such hard work. It was criminal. Media Course Leader I’m really bad at reading feedback. I’ll look at the mark and then be like ‘well stuff it, I can’t do anything about it’ Student, TESTA focus group
  • 22.
    What students say… It’sdifficult because your assignments are so detached from the next one you do for that subject. They don’t relate to each other. Because it’s at the end of the module, it doesn’t feed into our future work. Because they have to mark so many that our essay becomes lost in the sea that they have to mark. It was like ‘Who’s Holly?’ It’s that relationship where you’re just a student.
  • 23.
    Turning feedback intoa dialogue • Who starts the dialogue? • Iterative cycles of reflection across modules • Quick generic feedback • Feedback synthesis tasks • Peer feedback and self-evaluation • Technology: audio, screencast and blogging • From feedback as ‘telling’… • … to feedback as clarifying conversation
  • 24.
  • 25.
  • 26.
  • 27.
    References Barlow, A. andJessop, T. 2016. “You can’t write a load of rubbish”: Why blogging works as formative assessment. Educational Development. 17(3), 12-15. SEDA. Berg, M. and Seeber, B. (2016) The Slow Professor: Challenging the Culture of Speed in the Academy. Toronto. University of Toronto Press. Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712. Freire, P. 1972. Pedagogy of the Oppressed. Penguin Books. Harmondsworth. Gibbs, G. & Simpson, C. (2004) Conditions which assessment supports students' learning. Learning and Teaching in Higher Education. 1(1): 3-31. Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2015) ‘An assessment arms race and its fallout: High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Education 40(4) 528-541. Jessop, T. and Tomas, C. 2016 The implications of programme assessment on student learning. Assessment and Evaluation in Higher Education. Published online 2 August 2016. Jessop, T. 2016. Seven years and still no itch – why TESTA keeps going. Educational Developments, 17(3) 5-8. SEDA. Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student learning: a comparative study. Studies in Higher Education. Studies in Higher Education. 41(4) 696-711. Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale study of students’ learning in response to different assessment patterns. Assessment and Evaluation in Higher Education. 39(1) 73-88. Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517. Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional Science, 18(2), pp. 119–144. Torrance, H. (2007) Assessment as learning? How the use of explicit learning objectives, assessment criteria and feedback in post-secondary education and training can come to dominate learning. Assessment in Education 14(3) 281–294

Editor's Notes

  • #4 Research and change process. Three premises: assessment drives the curriculum; feedback is ‘the single most important factor in student learning’ and the programme is the most important place to influence change.
  • #6 What started as a research methodology has become a way of thinking. David Nicol – changing the discourse, the way we think about assessment and feedback; not only technical, research, mapping, also shaping our thinking. Evidence, assessment principles. Habermas framework.
  • #7 Academics operate in isolation from one another. Only see their part of the degree. Don’t see connections. Fragments into small tasks – hamster wheel. Curriculum design issue. The trouble is that students experience the whole elephant and it is often indigestible… Assessment is mainly sort of the topical knowledge and the topics never relate. We'll never do something again that we’ve already studied, like we learn something and then just move on (TESTA focus group data).
  • #13 Teach Less, learn more. Assess less, learn more.
  • #22 Feedback: all that effort, but what is the effect? Margaret Price But lots of projects and programmes do….