Nicky and Clare present a case study from the Assessment Literacy Project

1. Learning about A
Learning about Assessment Literacy:
A Case Study from the Assessment
Literacy Project
Nicky Spawls & Clare O’Donoghue
Dept of Education
School of Health and Education
16/07/15Slide 1

2. The Context
• BA Education Studies & BA Early Childhood Studies
• 2 Distinct programmes but with some combined
modules
• Approximately 150- 200 students per year in total
• Diverse student cohort:
– Some with A’ Levels
– Some with level 3 vocational diplomas
– Young and mature students
– Multicultural cohort, not all will have completed
secondary school in UK

3. 4 Models of Assessment Literacy
1. The Traditional Model = by Osmosis: Tacit standards absorbed over
time informally and serendipitously. The ability of students to
absorb these is seen as a mark of their academic ability to guess
what the lecturers want.
2. The Explicit Model: Standards explicitly articulated and presented
to students (with inevitable limitations of fuzzy description) for
transparency
3. The Social Constructivist Model: Actively engaging students in
formal processes to communicate tacit as well as explicit
understanding of standards
4. The ‘Cultivated’ Community of Practice Model: Tacit standards
communicated through informal knowledge exchange networks
seeded by specific activities, e.g. reading clubs, informal discussion
(O’Donovan, Price & Rust 2008)Slide 3

4. Analysis of our Starting Point
From analysis of our own courses we acknowledged some of the following :
• Much good practice taking place across the programme
• Tutors building in lots of support for assessment
• Variety of assessment methods being used
However….
• Lack of clear differentiation between formative and summative assessment
• Lack of real consideration of why one thing was formative and another
summative
• Neglecting to perceive the importance of formative as a building block to
summative
• …and in cases, having no obvious relationship between the two
• Scope for greater consistency of approach between modules / tutors etc
Slide 4

5. Students’ experiences of assessment can be…
Scary
Lonely
Confusing
Slide 5

6. Assessment Literacy: Staff & Students
• Appreciation of assessment’s relationship to learning
• Conceptual understanding of assessment
• Understanding the nature, meaning and levels of
assessment criteria and standards
• Develop student skills in self and peer assessment
• Familiarity with different assessment types
• Understanding the role of formative assessment
• Students understanding the value of feedback and how to
make use of it
• Possessing the intellectual ability to select and apply
appropriate techniques and approaches to assessed tasksSlide 6

7. Need for a Programme Level Approach
• Staff team need a programme view
‘Where there is a greater sense of the holistic
programme, students are more likely to achieve the
learning outcomes than students on programmes
with a more fragmented sense of the programme’.
(Havnes, 2007 in Price et al, 2012))
Slide 7

8. Our Interpretation
• To put Assessment Literacy at the heart of our team’s work in
delivering teaching and learning
• To engender a deep level of engagement in assessment that is
participatory and constructivist
• Through involvement with assessment criteria, to engage with
feedback and specifically formative feedback,
• Important to note that this is closely integrates with Academic
Literacy i.e. how to take good notes, how to write an essay, how to
write a report, how to carry out effective research, how to use
referencing.
• Also parallels the use of module level descriptorsmodule level descriptors to demonstrate how
expectations differ for each level of their studies and to make explicit
to students how these differences in expectation can be achieved
Slide 8

9. Our Goals in the Assessment Literacy Project
• Promote a discourse of assessment between staff and students;
‘a community of practice’
• Why ? student voice, NSS, BoS, PPR
• KPIs
• Authenticity in assessment
• Build on an existing dialogue of academic literacy, built into all
modules, to create systematic scaffolding in A.L. across
programmes
• To see the relationship between assessment for learning and the
development of successful self-regulated learners
• To create fair and transparent assessment practices
• To review and enhance assessment practices during this ‘window
of opportunity’ - the periodic reviewSlide 9

10. Engaging 1st
year students with AL
EDU 1305 Philosophy of Education
• Prior to term 1 assessment
– Students given past years’ assignment titles to brainstorm on
topics after relevant lectures.
– Students shown past examples of anonymised essays (1st
and
2:1 standard) and given grading scheme to read and mark
essays in seminars.
– Discussion of evaluation of essays against assessment criteria
starts the dialogic AL process
– Students complete self-evaluation questionnaire on term 1
and term 2 assignment to be submitted with assignments
– 83% pass rate (9% deferred, 6% failed, 2% X grade)
Slide 10

11. Student Self-Assessment (1)
• The self-assessment has 3 main components:
• Part 1: Academic Literacy – word count, sources, bibliography,
Turnitin originality score – paraphrasing and quotation
• Part 2 – Engagement with task – most interesting / most difficult
aspects
• Part 3 – Assessment Literacy
• Degree of confidence (confident / not confident) to which the
marking criteria for the assignment have been met.
• Areas to work on identified by previous feedback (term 2
assignment only)
• Confidence with extent to which these areas have (not) been
addressed
• Specific areas student requests feedback on

12. Student Self-Assessment (2) - Example
• Assignment feedback from term 1's work indicated that I should
work on the following areas of my assignment writing in order to
consolidate / improve my grades ..........
• I am confident that I have addressed the following areas of this
feedback in this assignment to consolidate / improve the quality
of my work ..........
• I am not very confident that I have made significant improvement
in the following areas of previous feedback in this assignment.....
• I would particularly welcome feedback on the following aspect of
my work in this assignment ..........
Slide 12

13. Engaging 2nd
year students with AL
EDU2309 ‘Insights into Play’
• Active engagement with the assessment standards
through marking example observations and essays
• Each summative assessment is preceded by a directly
linked formative assessment
• Feedback on formative assessment through individual
tutorials – dialogic process through which assessment
standards are constructed, as well as communicated
Slide 13

14. EDU2309 ‘Insights into Play’
• Term 1
Formative Assessment 1 - 1% first 45% good honours
Summative Assessment 1 - 14% first 49% good honours
• Term 2
Formative Assessment 2 – 22% first 50% good honours
Summative Assessment 2 – 22% first 58% good honours
Overall module grades - 92% pass rate, 54% good passes
(5% fail, 2% deferred, 1% X grade)
• 88% agreed or strongly agreed with the statement: ‘I have received
helpful feedback that has enabled me to make progress.’Slide 14

15. EDU 2308 Education & The Social World
An experiment prompted by staff workshop on AL.
Front-loading feedback:
• Term 1 essay only given detailed formative feedback but
no summative grade.
– On pressure from students, tutors gave general band
indicators for term 1 essays.
• Term 2 essay given summative grade & minimal
feedback
• Module pass rate = 91%, 55% good passes
(4% failed, 4% deferred)
Slide 15

16. EDU3309 ‘The Child in Context’
Strategies:
•Seminar on Level 6 Module Level Descriptors – unpacking
the language and expectations of Level 6 outcomes
•Student marking sample essays with marking criteria
followed by discussion of grades with discussion
•2014 - 77% pass rate, 55% ‘good’ grades 1-8
•2015 - 92% pass rate, 68% ‘good’ grades
Slide 16

17. Assessment Literacy Matrix
• The Team have developed an Assessment
Literacy Matrix
• In keeping with the tenets of assessment to
examine where different strategies are being
used and to ensure that they are not being
overused
• Realisation that over familiarity may lead to loss
of effectiveness and engagement
Slide 17

18. Summary of Activities to Promote Student
A.L.
• Student Activity
– Use of exemplars / grading exemplars against assessment criteria
– Use of previous cohort generic feedback, or individual examples of
previous task feedback for students to identify strengths and potential
pitfalls in completing assignment
– Peer evaluation and feedback
– Self assessment
– Tasks that require engagement with previous feedback
• Staff Activity
– Assignment preparation for students
– Formative feedback on drafts or plans
– Informative and timely feedback with feed-forward for future work
– Planning assessment tasks that build on previous assessment experience
(assessment mapping)
Slide 18

19. Summary of Ways to Promote Staff A.L.
• Staff ownership of assessment design in relation to
learning outcomes
• Collaborative staff-writing of assessment criteria
• Assessment marking workshops on sample work ->
exemplars of internal standards
• Explicit + and – feedback on student work to justify
grade awarded -> transparency
• Early moderation / double marking within the marking
cycle
• Awareness of intra-marker and inter-marker reliability
issuesSlide 19

20. Going forward…
• Within each module – how is assessment an integrated part of
the learning and how is it built towards? [assessment planning;
active engagement with assessment standards]
• Within each module – where is the opportunity to engage
students in a meaningful feedback process? [feedback timelines]
• Across the programme – how are assessments and feedback
opportunities related to each other across modules? [assessment
mapping]
• What are the ‘crisis’ moments for students and how can
assessment literacy be embedded to ease this?
Slide 20

21. Questions that were raised:
• Can you get enough information from a 10 minute
presentation to determine learning outcomes at Level 5?
• Can you get as much from a 1500 word essay as you can
from a 3,000 word essay?
• How much support is too much support?
Slide 21

22. References
Ball, Bew, Bloxham et al 2013 ‘A Marked Improvement:
Transforming Assessment in Higher Education’ HEA
https://www.heacademy.ac.uk/node/3950
O’Donovan, B., Price, M., Rust, C. (2008) ‘Developing Student
Understanding of Assessment Standards: A Nested Hierarchy of
Approaches’ Teaching in Higher Education 12/ 2: 205-217
Price, M., Rust, C., Donovan, B., Handley, K., & Bryant, R. (2012)
Assessment Literacy : The foundation for improving student learning
Oxford: ASKE
16/07/15Slide 22

23. • Thank you for listening
• Any Questions?
16/07/15Slide 23