Download to read offline
Uploaded byTansy Jessop
Birmingham Assessment and Feedback Symposium
The document discusses the Testa methodology aimed at improving assessment practices in higher education through a collaborative approach. It identifies key problems in current assessment systems and presents evidence-based actions to enhance formative assessments, improve student feedback, and clarify academic standards. The ultimate goal is to foster a more cohesive learning environment that prioritizes student experiences and deep learning.
More Related Content
Similar to Birmingham Assessment and Feedback Symposium
More from Tansy Jessop
Birmingham Assessment and Feedback Symposium
- 1. Dethroning the gradeas king through TESTA’s programme approach Tansy Jessop @tansyjtweets @TESTAwin 24 May 2016
- 3. The TESTA Methodology 75PROGRAMME AUDITS Programme Team Meeting
- 4. Based on assessmentprinciples • ‘Time-on-task’ (Gibbs 2004) • Challenging and high expectations (Chickering and Gamson 1987) • Internalising goals and standards (Sadler 1989; Nicol and McFarlane-Dick 2006) • Prompt, detailed, specific, developmental, dialogic feedback (Gibbs 2004; Nicol 2010) • Deep learning (Marton and Saljo 1976).
- 5.
- 6. TESTA…. “…is a wayof thinking about assessment and feedback” Graham Gibbs
- 7. TESTA shifts inperspective from… • ‘my’ module to ‘our’ programme • from teacher-focused on module delivery to student experience of whole programme • from individualistic modular design to coherent team design • from the NSS to enhancement strategies
- 8. TESTA addresses threeproblems Problem 1: Knee-jerk problem Problem 2: Curriculum design problem Problem 3: Evidence to action problem
- 9. Problem 1: Theknee-jerk
- 10. Wow! Our students loveHistory! Fantastic!
- 11. Whoops there’s alittle problem here
- 12. Fix it! Ok, we’lllook especially at polishing up our feedback. Students seems to find that the least best thing.
- 13. Apply spit andpolish
- 14. Anyone for thefeedback sandwich? I cushion the blow! The hard truths are nicely disguised! Me too - nice and soft!
- 15. Problem 2: Curriculumdesign problem
- 16. Does IKEA 101work for complex learning?
- 17. Curriculum privileges ‘knowing’stuff “Content is often the most visible aspect for students, the control of which is frequently devolved to individual academics, who receive little or no training in curriculum design and planning” (Blackmore and Kandiko 2014, 7).
- 18.
- 19. Problem 3: Evidenceto action gap
- 20. Problem 3: Evidenceto action gap http://www.liberalarts.wabash.edu/study-overview/ Flawed Assumptions… • Main problem lack of high quality data • Logic will prevail • Systems will change
- 21. Proving is differentfrom improving “It is incredibly difficult to translate assessment evidence into improvements in student learning” “It’s far less risky and complicated to analyze data than it is to act” (Blaich & Wise, 2011)
- 22. Paradigm What itlooks like Technical rational Focus on data and tools Relational Focus on people Emancipatory Focus on systems and structures
- 23. TESTA themes andimpacts 1. Variations in assessment patterns 2. High summative: low formative 3. Disconnected feedback 4. Lack of clarity about goals and standards
- 24. 1. Huge variations •What is striking for you about this data? • How does it compare with your context? • Does variation matter?
- 25. Assessment features acrossa 3 year UG degree (n=75) Characteristic Range Summative 12 -227 Formative 0 - 116 Varieties of assessment 5 - 21 Proportion of examinations 0% - 87% Time to return marks & feedback 10 - 42 days Volume of oral feedback 37 -1800 minutes Volume of written feedback 936 - 22,000 words
- 26. And some patterns… CharacteristicLow Medium High Volume of summative assessment Below 33 40-48 More than 48 Volume of formative only Below 1 5-19 More than 19 % of tasks by examinations Below 11% 22-31% More than 31% Variety of assessment methods Below 8 11-15 More than 15 Written feedback in words Less than 3,800 6,000-7,600 More than 7,600
- 27. Actions based onevidence a) Reduction in summative b) Increase in formative c) Streamlined varieties d) More or less feedback depending… e) Quantifiable f) Every time a coconut with each feature
- 28. Theme 2: Highsummative: low formative • Summative ‘pedagogies of control’ • Circa 2 per module in UK • Ratio of 1:8 of formative to summative • Formative weakly understood and practised
- 29. What students say… •A lot of people don’t do wider reading. You just focus on your essay question. • In Weeks 9 to 12 there is hardly anyone in our lectures. I'd rather use those two hours of lectures to get the assignment done. • It’s been non-stop assignments, and I’m now free of assignments until the exams – I’ve had to rush every piece of work I’ve done.
- 30. What students sayabout formative • If there weren’t loads of other assessments, I’d do it. • If there are no actual consequences of not doing it, most students are going to sit in the bar. • It’s good to know you’re being graded because you take it more seriously. • The lecturers do formative assessment but we don’t get any feedback on it.
- 31.
- 32. Actions based onevidence 1. Rebalance summative and formative 2. Shared language: programme approach 3. Formative in the public domain 4. Linking formative and summative 5. Risky, creative, challenging tasks 6. Students reading and producing more 7. Deeper understanding of value of formative
- 33. Theme 3: Disconnectedfeedback
- 34. Take five • Choosea quote that strikes you. • What is the key issue? • What strategies might address this issue?
- 35. What students say… Thefeedback is generally focused on the module. It’s difficult because your assignments are so detached from the next one you do for that subject. They don’t relate to each other. Because it’s at the end of the module, it doesn’t feed into our future work. I read it and think “Well, that’s fine but I’ve already handed it in now and got the mark. It’s too late”.
- 36. Students say thefeedback relationship is broken… Because they have to mark so many that our essay becomes lost in the sea that they have to mark. It was like ‘Who’s Holly?’ It’s that relationship where you’re just a student. Here they say ‘Oh yes, I don’t know who you are. Got too many to remember, don’t really care, I’ll mark you on your assignment’.
- 37. Actions based onevidence • Conversation: who starts the dialogue? • Iterative cycles of reflection across modules • Quick generic feedback: the ‘Sherlock’ factor • Feedback synthesis tasks • Technology: audio, screencast and blogging • From feedback as ‘telling’… • … to feedback as asking questions
- 38. Theme 4: Confusionabout goals and standards • Consistently low scores on the AEQ for clear goals and standards • Alienation from the tools, especially criteria and guidelines • Symptoms: perceptions of marker variation, unfair standards and inconsistencies in practice
- 39. What students say… We’vegot two tutors- one marks completely differently to the other and it’s pot luck which one you get. They have different criteria, they build up their own criteria. It’s such a guessing game.... You don’t know what they expect from you.
- 40. What students say… Thereare criteria, but I find them really strange. There’s “writing coherently, making sure the argument that you present is backed up with evidence”. I get the impression that they don't even look at the marking criteria. They read the essay and then they get a general impression, then they pluck a mark from the air. I don’t have any idea of why it got that mark
- 41. Taking action: internalisinggoals and standards • Regular calibration exercises • Discussion and dialogue • Discipline specific criteria (no cut and paste) Staff Team • Rewrite/co-create criteria • Marking exercises (ASKE CETL) • Design and value formative Staff and students • Enter secret garden - peer review • Engage in drafting processes • Self-reflection Students
- 42. It’s about educationalparadigms…
- 43.
- 44.
- 45. Impacts at Winchester •Upwards trajectory on A&F scores on NSS on TESTA programmes – ‘Top 4’ University • TESTA ‘effect’ - people talk about formative • Experimentation in co-creation • Team approach to designing curricula • Design cycle for periodic review includes TESTA
- 46. References Blaich, C., &Wise, K. (2011). From Gathering to Using Assessment Results: Lessons from the Wabash National Study. Occasional Paper #8. University of Illinois: National Institution for Learning Outcomes Assessment. Boud, D. and Molloy, E. (2013) ‘Rethinking models of feedback for learning: The challenge of design’, Assessment & Evaluation in Higher Education, 38(6), pp. 698–712. doi: 10.1080/02602938.2012.691462. Gibbs, G. & Simpson, C. (2004) Conditions r which assessment supports students' learning. Learning and Teaching in Higher Education. 1(1): 3-31. Harland, T., McLean, A., Wass, R., Miller, E. and Sim, K. N. (2014) ‘An assessment arms race and its fallout: High-stakes grading and the case for slow scholarship’, Assessment & Evaluation in Higher Education, 40(4), pp. 528–541. doi: 10.1080/02602938.2014.931927. Hughes, G. (2014) Ipsative Assessment. Basingstoke. Palgrave MacMillan. Jessop, T. and Maleckar, B. (2014). The Influence of disciplinary assessment patterns on student learning: a comparative study. Studies in Higher Education. Published Online 27 August 2014 http://www.tandfonline.com/doi/abs/10.1080/03075079.2014.943170 Jessop, T. , El Hakim, Y. and Gibbs, G. (2014) The whole is greater than the sum of its parts: a large-scale study of students’ learning in response to different assessment patterns. Assessment and Evaluation in Higher Education. 39(1) 73-88. Nicol, D. (2010) From monologue to dialogue: improving written feedback processes in mass higher education, Assessment & Evaluation in Higher Education, 35: 5, 501 – 517. O'Donovan, B , Price, M. and Rust, C. (2008) 'Developing student understanding of assessment standards: a nested hierarchy of approaches', Teaching in Higher Education, 13: 2, 205 — 217 Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems’, Instructional Science, 18(2), pp. 119–144. doi: 10.1007/bf00117714.
Editor's Notes
- #6 How do you measure soft stuff? 5 day cricket match versus 20/20
- #7 What started as a research methodology has become a way of thinking. David Nicol – changing the discourse, the way we think about assessment and feedback; not only technical, research, mapping, also shaping our thinking. Evidence, assessment principles. Habermas framework.
- #8 Dominos fast food to Raymond Blanc slow learning
- #15 Honest dialogue vs tricks of dialogue to minimise damage
- #17 Hard to make connections, difficult to see the joins between assessments, much more assessment, much more assessment to accredit each little box. Multiplier effect. Less challenge, less integration. Lots of little neo-liberal tasks. The Assessment Arms Race.
- #19 Language of ‘covering material’ Should we be surprised?
- #20 Wabash study – 2005-2011, 17,000 students in 49 American colleges. 60-70 publications Critical thinking, moral reasoning, leadership towards social justice, engagement in diversity, deep intellectual work.
- #21 Wabash study – 2005-2011, 17,000 students in 49 American colleges. 60-70 publications Critical thinking, moral reasoning, leadership towards social justice, engagement in diversity, deep intellectual work.
- #23 TESTA has done the data and that’s been useful. Ideological compromises. Mixed methods approaches. Critical pedagogy sleeping with the enemy. Democratic, participatory, liberating curriculum and pedagogy. Teachers and students shape and change education. Resist managerialism and the market. Risky pedagogies.
- #32 Teach Less, learn more. Assess less, learn more.
- #37 Impoverished dialogue Nicol, Mass Higher Education; Relationship
- #42 Students can increase their understanding of the language of assessment through their active engagement in: ‘observation, imitation, dialogue and practice’ (Rust, Price, and O’Donovan 2003, 152), Dialogue, clever strategies, social practice, relationship building, relinquishing power.