Workshop at Trinity College Dublin for the National Forum T&L

1. National Forum Seminar Series: Engaging in
Meaningful Change to Assessment Practice
This seminar draws on ten years of research and change
from ‘Transforming the Experience of Students through
Assessment’ (TESTA), in order to explore changing
assessment cultures. Using evidence, models and stories of
change, the seminar will enable participants to weigh up the
benefits of making programmatic changes, troubleshoot
barriers, and identify enabling conditions for assessment
culture change. The workshop will also explore tangible and
practical assessment ideas, and begin to expand the
repertoire of approaches to programmatic assessment.

2. Engaging in meaningful change to
assessment practice
Trinity College Dublin
11 February 2020

3. The workshop
• Your thoughts about programme assessment
• Why programme assessment?
• What might a programme approach look like?
• Programme assessment tactics and approaches
• The challenge of institutional assessment change
• Enabling change: principles, caveats and tactics

4. Choose a picture that draws your eye

5. Reflective jottings
• How does the image express how you
feel about assessment?
• What hopeful thoughts does the
image suggest about the potential of
programme assessment?
• In relation to the picture explore your
fears about programme assessment.

6. Why programme assessment?
A modular
problem
A curriculum
problem
A high-speed
problem
A heavy-load
problem
An alienation
problem

7. A modular problem

8. A curriculum problem

9. An high-speed problem
Slow down!
I’m
exhausted.
So much
measuring!
So much
marking!
So much
material to
cover!

10. Summative
Exams
Tomas & Jessop 2018 Struggling & Juggling: comparing assessment loads

11. An Alienation
problem
Alienation n. the state of a
person who has disengaged
themselves or have been
alienated from the attentions
from a person, place, or thing
they once enjoyed.

12. What might
programme
assessment
look like?
Structure
Pedagogy
Culture

13. Models of programme
assessment:
Spot the flying pigs
1. 2. 3.
4.
5.

14. Model 1: The whole hog
• Teaching separate from
assessment
• Integrated programme
assessment (IPA) runs
through the degree

15. Brunel University: Integrated Programme
Assessment (IPA) in Biosciences

16. Here’s an example – from 3 to 1

17. Model 2: Half the hog
• Connective tissue units
• Research through-line
• Less summative makes
room for more formative

18. PPE degree
Politics
Philosophy
Economics
Integration between P, P, and E

19. Model 3: The other half of the hog
• Integrated assessment on
a few units
• Combines with fewer
summative and more
formative

20. Fashion Journalism
Modules Formative plus
50%
summative
Combined 50%
summative
Fashion
photography
Photo portfolio Magazine
(digital and
print)
combining
learning from
all three
modules
Fashion web
journalism
Blogs
Fashion print
media
Articles

21. Model 4: All
the hogs at
the trough
together
• Team approach to
assessment design
• Programme approach
leads to more formative,
less summative

23. Model 5: The warthog
• Block teaching and
assessment design
• No competing modules,
no assessment arms race

24. Netherlands
and
Plymouth
What is
programmatic
about this???
Module 1: Physics 101
Weeks 1-4 Intensive
Assessment
Module 2: Astrophysics 101
Weeks 5-8 Intensive
Assessment
Module 3: Physics labs
Weeks 9-12 Intensive
Assessment

25. Chat to a partner
• Which of these models resonates
for you and why?
• Which aligns most closely with
your view of programme-level
assessment?
• Which have you tried or are keen
to try and what might hold you
back?

26. What might
programme
assessment
look like?
Structure
Pedagogy
Culture

27. The interplay between curriculum
and pedagogy
In the end, adventurous
curricula will be brought off
through daring pedagogies,
and, in turn, imaginative
pedagogies will ultimately
point to a transformation of
curricula.
(Ron Barnett 2005)

28. There can be no curriculum development
without teacher development
Lawrence Stenhouse 1975

29. TESTA
definitions
Summative:
graded assessment which
counts towards the degree
Formative:
Does not count: ungraded,
required task with feedback

30. A preposition
problem:
assessment of
trumps for
and as
• Low formative to
summative ratio of
1:8 (UK, NZ, Ireland)
• Summative as
‘pedagogy of control’
• Formative weakly
practised and
understood

31. Assessment
Arms Race

32. A lot of people don’t do wider
reading. You just focus on your
essay question.
In Weeks 9 to 12 there is hardly
anyone in our lectures. I'd rather
use those two hours of lectures
to get the assignment done.
It’s been non-stop
assignments, and I’m now
free of assignments until
the exams – I’ve had to
rush every piece of work
I’ve done.
CONSEQUENCES
OF HIGH
SUMMATIVE

33. It was really useful. We
were assessed on it but we
weren’t officially given a
grade, but they did give us
feedback on how we did.
It didn’t actually count so
that helped quite a lot
because it was just a
practice and didn’t really
matter what we did and we
could learn from mistakes
so that was quite useful.
The benefits
of formative

34. If there weren’t loads
of other assessments,
I’d do it.
It’s good to know you’re
being graded because
you take it more
seriously.
BUT… If there are no actual
consequences of not doing
it, most students are going
to sit in the bar.
The lecturers do formative
assessment but we don’t get
any feedback on it.

35. Formative is the hardest nut to crack…
Go to www.menti.com and use the code 97 97 66
Type in three reasons why students may be
reluctant to invest time and energy in completing
formative assessment tasks

36. Yet
formative
is vital
Low-risk way of learning from
feedback (Sadler, 1989)
Fine-tune understanding of goals
(Boud 2000, Nicol 2006)
Feedback to lecturers to adapt
teaching (Hattie, 2009)
Cycles of reflection and collaboration
(Biggs 2003; Nicol & McFarlane Dick
2006)
Encourages and distributes student
effort (Gibbs 2004).

37. How do you encourage formative?
Go to www.menti.com and use the code 69 63 95
to choose three ideas that resonate most for you.

38. Case Study 1:
All or nothing
Systematic reduction of summative
across whole business school
Systematic ramping up of formative
All working to similar script
Whole department shift,
experimentation, less risky together

39. Case Study 2
• Problem: silent seminar
• Public platform blogging
• Current academic texts
• In-class
• Threads and live discussion
• Linked to summative

40. Case study 3
Two stage Exam
https://www.youtube.com/watch?v=ZVFwQzlVFy0

41. Case Study 4: Ceasing Continuous
Assessment

42. On the evils of
continuous assessment
Chris Adams

43. First year laboratory course set-up
2017-18
21 experiments, continuously assessed

44. Coming from a rubbish school (and having got into uni on a contextual
offer), I'd never set foot in a lab before, so the fact that the first labs
were assessed was insulting and stressful. The guy next to me in labs
had gone to Harrow, so got much better marks than me in the initial few
labs.
Too much pressure to get perfect
results, rather than it being a learning
process
I felt like we were constantly
being examined and weren't
given any time to just learn.

45. For many students, coming to grips with and then overcoming false starts, errors, bumbling
attempts and time spent going up blind alleys lead to deep understandings by the end of a course.

46. Attendance 2017-18
50
55
60
65
70
75
80
85
90
95
100
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Attendance(%)
Experiment

47. Course set-up
2017-18 2018-19
21 experiments, continuously summatively assessed 21 experiments
• 15 formative
• 6 summative
• Best 4 count

48. Dialogic feedback
I feel like this lab went well but there were a
lot of learning points. The TLC plate didn't
work to begin with because we had not
diluted the benzaldehyde so we had to redo
the TLC plate. Our melting point was 113.9
degrees Celsius which is close to the
recorded value of 111 degrees C. I would
give myself a mark of 62 for this lab.
Overall today you worked really well and achieved
the desired product which IR, TLC and melting point
proved was clean. When performing a TLC dilute
every sample you are spotting even if they are oils
otherwise, they will dominate the TLC plate. In your
lab book you need to record observations as you go
along as well as the experimental not at the end as
you may forget something or miss remember. In your
lab book the key IR peaks should also be labelled i.e.
peak at 1648 cm-1 carbonyl/C=O etc.

49. 0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
17-18 18-19 17-18 18-19 17-18 18-19 17-18 18-19 17-18 18-19 17-18 18-19
I found the laboratory course
to be interesting
It was clear to me what I
was supposed to learn from
completing the laboratory
component
Based on the feedback that I
have been provided during
the laboratory component, I
feel that I understand the
experiments and have
developed my laboratory
skills during the year
It was clear to me how the
laboratory experiments
would be assessed
The way in which the
laboratory component is
assessed helps me learn
and improve my
understanding of chemistry
I enjoyed the first-
year chemistry
laboratory
Chart Title
Strongly disagree Disgaree Neutral Agree Strongly agree

50. Are you afraid to tell someone when you've
made a mistake?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2017 2018
Yes Sometimes No

51. Attendance 2018-19
50
55
60
65
70
75
80
85
90
95
100
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Attendance(%)
Experiment

52. Attendance 2018-19
50
55
60
65
70
75
80
85
90
95
100
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Attendance(%)
Experiment

53. Number of summative experiments
performed
Summatively assessed
experiments performed
0 1 2 3 4 5 6
Number of students 1 0 1 2 16 44 52

54. Conclusions
 It is not necessary to continuously assess a course in order to get
>90% attendance.
 First year students prefer it if you don’t.

55. Take Five
• Chat to one another about the
principles embedded in the case
studies
• Write down principles on flipchart
paper
• How could you adapt them – or
improve on them?

56. A pronoun
problem with
feedback:
I write; I talk;
you listen.
• One-off feedback,
episodic
• Modular
• Feedback as a
product not a process
• Perpetuates the myth
of objectivity

57. The feedback is
generally focused
on the module
Because it’s at the end
of the module, it doesn’t
feed into our future
work.
If It’s difficult because your
assignments are so detached
from the next one you do for
that subject. They don’t
relate to each other.
I read it and think “Well,
that’s fine but I’ve already
handed it in now and got the
mark. It’s too late”.
STRUCTURAL

58. It was like ‘Who’s
Holly?’ It’s that
relationship where
you’re just a student.
Because they have to mark so
many that our essay becomes
lost in the sea that they have
to mark.
Here they say ‘Oh yes, I don’t
know who you are. Got too
many to remember, don’t
really care, I’ll mark you on
your assignment’.
RELATIONAL

59. A feedback dialogue

60. Your essay lacked structure and
your referencing is problematic
Your classes are boring and I
don’t really like you 
From this dialogue….

61. …to this kind of dialogue
Conversation:
who starts the
dialogue?
Quick generic
feedback to
the class
Time in class
on peer
feedback
Marking
formative
exercises
Audio
feedback
…from
feedback as
telling
To feedback as
a conversation

62. The key to dialogue

63. Students to lecturers:
Critical Incident Questionnaire
Stephen Brookfield’s Critical Incident Questionnaire http://bit.ly/1loUzq0

64. Two ways of seeing feedback…
Feedback as product Feedback as process
• Textual production • Oral, gesture and written
• Monologue • Dialogue
• One-off • Iterative
• Teacher main actor • Students main actors
• Students can be passive • Students act on feedback
• Final word • Unfinished business
• Evaluative: looking back • Developmental

65. A way of
thinking
about
assessment
and
feedback?

66. Jigsaw Activity

67. Weekend
plans
by David Holper

68. Coffee
time

69. What might
programme
assessment
look like?
Structure
Pedagogy
Culture

70. Write on the cards:
one idea per card
• What would enable more
programmatic assessment to
happen in your context?
• What are the barriers to more
programmatic assessment?

71. “All organizations resist change. After all, that’s
their job. The whole purpose of any organization is
to act in ways that are regular, consistent, and
predictable. And regularity, consistency, and
predictability are natural enemies of change.”
(Buller, 2015, 2)

72. Three big ideas
• Corporate models don’t
work well in HE
• Distributed culture means
need to see multiple
perspectives
• Needs based change wins
over sales talk and
benefits

73. Five propositions
1. Change is difficult. It often involves loss.
2. Change is about people: listen, respect & involve
them, and act together.
3. ‘Shrink the change’ and build on existing practices
4. Change is about taking risks. Jump together.
5. Sustainable change also addresses systems

74. Case Study 1: TESTA

75. It was heavy, tons of marking for
the tutor. It was such hard work.
It was criminal.
Media Course Leader
I’m really bad at reading
feedback. I’ll look at the mark
and then be like ‘well stuff it, I
can’t do anything about it’
Student, TESTA focus group
1. Responded to a felt need

76. The value was to look at what we do from a scientific
perspective and look at things objectively, and that is
really enabling us to re-think how we do things.
2. Evidence based perspective

77. 3. Shrink the change
It has fed into the changes that are going on in the
curriculum…this helps me go to the team and say
‘Well, look, we’ve got the evidence now to be able
to go ahead and do this’ rather than it being on a
whim…

78. 4. Jumping together
I don’t think it’s just the
tools. The tools are
good and they work
really, really well, but
..[the change] comes
through a kind of
collegiality.
It’s been a collaborative
thing. It hasn’t just been
me saying to the team
“We’re going to do this”
It’s “This is what they’ve
found out folks. What
are we going to do
about it? How are we
going to develop it?

79. A team approach – all pulling in same direction

80. Everybody has brought
in more formative. The
idea was to consolidate
the summative
assessment and bring in
more formative.
Do we want to continue
offering twenty different
types of assessment or
do we bite the bullet and
say “We want the
students to be able to
master five of them”?
There has been more of a
spacing of assessments.
5. It has practical impacts…
There is a lot more feed
forward, which is what
came out of the TESTA.

81. 6. Ownership – we, I, we, us
Already today I have seen some of yesterday’s
feedback being put into action across the team
and we are feeling excited about the changes
we are making.
Clearly some things will take a little longer but
yesterday’s meeting has bought about clarity
and given us an insight over the direction we
now want to take the programme.
Email Correspondence, Programme Leader

82. 7. It is not top-down

83. Case Study 2:
Bristol
Future’s
Curriculum
Framework

84. Jury is out
but a bit
more like
this….

85. Your change
ideas
• Write about one big idea
you can take away about
changing assessment
practices
• What loose ends are there
for you?
• What might you do next?