1. The document summarizes the findings of the TESTA (Transforming the Experience of Students through Assessment) project, which examined assessment practices across multiple university programs. 2. The project found that most programs had high levels of summative assessment but low formative assessment. Feedback was often delivered too slowly to impact student learning and students found it unclear and not actionable. 3. Through analysis of assessment patterns, student surveys, and focus groups, the project identified ways to improve assessment practices, such as increasing formative assessment, clarifying standards, speeding up feedback, and helping students apply feedback to improve. The goal is to make assessment practices better support student learning.

1. TESTA: an evidence –
informed approach to
improving programme
assessment
Dr Tansy Jessop
TESTA Project Leader & Senior Fellow
Kingston University L&T Conference
20 April 2012

2. What is TESTA?
“Transforming the experience of Students through
Assessment”
HEA funded research project (2009-12)
Seven programmes in four partner universities
Mapping programme assessment
Listening to student voice
Engaging with Quality Assurance processes
Diagnosis - Intervention - Evaluation

3. TESTA ‘Cathedrals Group’ Universities

4. Two Paradigms
Transmission model
Expert to novice
Planned & ‘delivered’
Feedback by experts
Feedback to novices
Privatised
Monologue
Emphasis on measuring
Competition
Metaphor - machine
Social constructivist model
Participatory, democratic
Messy and process-oriented
Peer review
Self-evaluation
Social process
Dialogue
Emphasis on learning
Collaboration
Metaphor - the journey

5. 1. Programme assessment patterns
2. Robust research methods (Gibbs &
Dunbar-Goddet 2007,2009)
3. Educational principles (Gibbs and Simpson
2004).
4. Changing concepts and discourse (Nicol
2012)
5. Participatory social process
TESTA principles of transformation

6. Programme Audit
• How much summative assessment
• How much formative (reqd, formal,
feedback)
• How many varieties of assessment
• Proportion exams to coursework
• Word count of written feedback
• How much ‘formal’ oral feedback
• Criteria, learning outcomes, course docs

7. Assessment Experience Questionnaire
• 28 questions
• 5 point Likert scale where 5 = strongly
agree
• 9 scales & an overall satisfaction question
• Scales link to conditions of learning
• Examples:
• quantity and distribution of effort;
• use of feedback;
• quantity and quality of feedback;
• clear goals and standards

8. Focus groups
• Different kinds of assessment
• How assessment influences study
behaviour
• Whether and how students know what
quality work looks like
• The quality and timing of feedback
• The usefulness of feedback

9. Research Methodology
ASSESSMENT
EXPERIENCE
QUESTIONNAIRE
(AEQ n= 1200+)
FOCUS GROUPS
(n=50 with
301 students)
PROGRAMME AUDIT
(n =22)
Programme
Team
Meeting
Case Study

10. TESTA Case Study 1: what’s going
on?
• Lots of coursework, of very varied forms
• Very few exams
• Masses of written feedback on assignments
• Learning outcomes and criteria clearly specified
….looks like a ‘model’ assessment environment
But students:
• Don’t put in a lot of effort and distribute their effort across few
topics
• Don’t think there is a lot of feedback or that it very useful, and
don’t make use of it
• Don’t think it is at all clear what the goals and standards are

11. TESTA Case Study 2: what’s going
on?
• 35 summative assessments
• No formative assessment specified in documents
• Learning outcomes and criteria wordy and woolly
• Marking by global, tacit, professional judgements
• Teaching staff mainly part-time and hourly paid
….looks like a problematic assessment environment
But students:
• Put in a lot of effort and distribute their effort across topics
• Have a very clear idea of goals and standards
• Are self-regulating and have a good idea of how to close the
gap

12. Case Study 1:
• Staff do loads of work, and it doesn’t really
work for students.
• Students are unable to evaluate their own
performance.
• Students don’t take control of their own
learning.
• Summative assessment drives effort but not
necessarily engagement & learning.

13. Case Study 2:
• Students do loads of work, and it works well
as a programme.
• Students are continually engaged in
evaluating their own and others’
performance.
• Students control and manage their own
learning.
• Formative assessment drives effort,
engagement and learning.

14. Assessment trends on TESTA
• High summative, low formative (36:11?)
• High variety (ave 11; range 7-17)
• Written feedback (ave 7,153; range 2,869-
15,412 )
• Oral feedback (ave 6 hours, range 37 mins
to 30 hrs)
• Watertight documents, tacit standards
• Huge institutional and programme variations

15. Gibbs and Simpson’s effort principles
(2004)
1. Assessed tasks need to capture
sufficient student time and effort
2. These tasks should distribute effort
evenly across topics and weeks

16. TESTA evidence on effort
• Audit: Mean 36 summative assessments
• AEQ: Students from only 1/7 programmes agreed
that assessment encouraged regular effort
• Focus groups:
We could do with more assessments over the
course of the year to make sure that people are
actually doing stuff.
The more you write the better you become at it…
and if we’ve only written 40 pieces over three years
that’s not a lot.

17. • So you could have a great time doing nothing until
like a month before Christmas and you’d suddenly
panic. I prefer steady deadlines, there’s a gradual
move forward, rather than bam!
• If it’s not going to be in the exam, and it is quite
difficult, I wouldn’t bother with that.
• We get too much of this end or half way through the
term essay type things. Continual assessments
would be so much better.

18. Effort ideas
1. Multi-stage assessment
2. Social pressure – public work
3. Spread and co-ordinate hand in dates
4. Formative course requirements
5. Peer marking in class
6. Sampling
7. Set first year expectations
8. Frequent, authentic, innovative

19. Clear and high standards principles
3. Assessment communicates clear and high
expectations to students (Gibbs et al, 2004)
4. Students should:
(a)know what is expected,
(b)know how this relates to their actual
performance, and
(c)have some information about how to close
the gap (Sadler, 1989).

20. TESTA evidence on clarity
Audit: clear written statements mapped to
outcomes; low formative feedback; huge
variety of tasks
AEQ: Students on 0/7 programmes agreed.
Focus groups:
There are criteria, but I find them really strange.
There’s “writing coherently, making sure the
argument that you present is backed up with
evidence” but that isn’t enough.

21. I’m not a marker so I can’t really think like them... I
don’t have any idea of why it got that mark.
They have different criteria, build up their own
criteria. Some of them will mark more interested in
how you word things.
It’s easier to know if you’ve done a good essay than
creative piece... it’s very subjective.
You know who are going to give crap marks and
who are going to give decent marks.

22. Ideas for internalising standards
1. Marking workshops (Aske CETL, Oxford Brookes)
2. Students rewriting criteria in own words
3. Criteria-based peer review
4. Self-evaluation against criteria
5. Observation, imitation, participation, dialogue
(Bloxham and Campbell, 2010).
6. Contextualised Feedback – standards, criteria,
outcomes

24. Feedback principles (Gibbs & Simpson
2004)
5) Sufficient feedback is provided, both often
enough and in enough detail
6) It is quick enough for students to act on it
7) It focuses on learning rather than marks
8) It is understandable
9) Students attend to feedback and act on it to
improve their learning

25. TESTA evidence on feedback
• Audit: high volumes of written, low oral, delivered
slowly
• AEQ: Students from only 1/7 programmes agreed
that it was useful
I read it and think “Well that’s fine, but I’ve already handed it
in now and got the mark. It’s too late”.
It was about nine weeks before we got it back. I’d forgotten
what I’d written.
You know that twenty other people have got the same sort of
comment.

26. Once the deadline comes up to just look on the Internet and
say ‘Right, that’s my mark. I don’t need to know too much
about why I got it’.
I only apply feedback to that module because I have this fear
that if I transfer it to other modules it’s not going to transfer
smoothly.
You can’t carry forward most of the comments because you
might have an essay first & your next assignment might be a
poster.
Getting feedback from other students in my class, I can relate
more to what they’re saying and take it on board. I’d just shut
down if I was getting constant feedback from my lecturer.

27. TESTA: patterns of changesChanges through TESTAChanges through TESTA

28. Types of changesTypes of changes
• Reducing summative
• Increasing formative tasks
• Streamlining variety
• Student workload expectations
• Sequencing and linking tasks
• Practice based changes
• Changed conceptions of A&F

29. www.testa.ac.uk

30. References:
Gibbs, G. & Simpson, C. (2004) Conditions under which assessment
supports students' learning. Learning and Teaching in Higher
Education. 1(1): 3-31.
Gibbs, G. & Dunbar-Goddet, H. (2009). Characterising programme-
level assessment environments that support learning. Assessment &
Evaluation in Higher Education. 34,4: 481-489.
Jessop, T, McNab, N and Gubby, L. (2012) Mind the gap: An analysis
of how quality assurance processes influence programme assessment
patterns. Active Learning in Higher Education. 13(3).
Nicol, D. (2010) From monologue to dialogue: improving written
feedback processes in mass higher education, Assessment &
Evaluation in Higher Education, 35: 5, 501 – 517
Nicol, D. (2012) Assessment Principles Webinar on JISC.
Sadler, D.R. (1989) Formative assessment and the design of
instructional systems, Instructional Science, 18, 119-144.
Sambell, K (2011) Rethinking Feedback in Higher Education. Higher
Education Academy Escalate Subject Centre Publication.