1. Can assessment literacy be enhanced and does it lead
to improved student performance?
A case study of year one Business and Management
students at Middlesex University Business School
by Simon Roberts
Karim Qizalbash
Ana Marinica

2. Introduction
* Context for the study
* Course Design
* Evaluation Through an Assessment
Literacy lens
Assessment
Assessment Literacy
Why is assessment literacy important
Activity
A Business School Case Study
| 2
Session Overview

3. • Business School initiative to address the issue of lower than
expected recruitment target
• MBS0111 – Preparing for Business
• 12 week programme – 12 h/week
• 6h of Business Studies
• 6h of Academic Skills development
• The team: Business School Academics from LWO , LDU and a GTA
Context For The Study
| 3

4. • Rationale to mirror loosely what would come
• Workshops in lecture - seminar style
• Introduction to assessment likely to be experienced in year 1
with the addition of a reflection
• Introduction to both group work and individual assignments
• Support provided in parallel to be outlined by Karim
Course Design
| 4

5. Evaluation Through an Assessment Literacy lens
• Focus on preparing students for University studies
• Introducing them to different forms of assessment
• Providing them with experience of a variety of assessments
• Providing them with support and formative feedback
• Alignment of the assessments to what would follow ?
| 5

6. Assessment
• Brown and Knight (1994, p.7) state that ‘assessment defines what
the student regards as important, how they spend their time [..],
see themselves as students and then as graduates.
• A key issue in assessment is that students often do not
understand what is a better piece of work and do not understand
what is being asked of them particularly in terms of standards and
criteria.
| 6
(O’Donovan et al., 2001)

7. Assessment for learning
• engaging, meaningful assessment tasks; in line with the
learning outcomes of the module
• students developing as learners – effective attributes and skills
to self-assess and evaluate their own learning
• informal feedback (e.g. in-class group discussions, peer-review)
• formal feedback (a range of forms of feedback, used at a
number of stages)
• opportunities to learn and practice assessments
• formative and summative – appropriate balancing of these two
types of assessment
| 7
(Sambell et al, 2013)

8. Assessment Literacy
‘Literacy’ as a term:
• traditionally utilised in the context of skills
• at times referred to in terms of attributes such as confidence,
competence and fluency
• can also be seen as a threshold to further learning, deeper
understanding and engagement
| 8
(Price et al, 2012)

9. • refers to a student understanding and becoming fluent in
assessment terms
• encompasses an appreciation of an assessment’s relationship to
learning, understanding of assessment and feedback practices,
as well as terminology used, the type, meaning and level of
assessment criteria and standards
• equips one with an appreciation of the purpose and process of
assessment which enables one to engage deeply with
assessment standards to make a choice of which skill or area of
knowledge to apply and appreciate the appropriateness of each
to a specific task
| 9
Assessment Literacy cont’d
(Price et al, 2012)

10. Assessment Literacy cont’d
• assessment literates know the difference between sound and
unsound assessments. They are not intimidated by the
sometimes mysterious and always daunting technical world of
assessment.
• when assessment literate student undertake an assessment or
task they will already be familiar with the appropriate
assessment standards they do not discover the standard
through doing the task
| 10
(Stiggins, 1995)
(Price et al, 2012)

11. | 11

12. • the transition period has a major impact on student retention
• in the UK, for instance, about two-thirds of withdrawals happen during or at
the end of the first year
• a student population with diverse entry qualifications, abilities and learning
experiences
• traditional expectations towards students have not changed: they are to
manage their learning and acquire academic literacy independently
• under the pressure of league tables, students in secondary schools ‘tend to be
“spoon-fed” for longer, and are less equipped with “self-learning skills” ’
• the lack of preparation for and understanding of the type of learning that is
required makes it difficult for students to adjust to university life
(Drew, 2001; Gamache, 2002)
| 12
Why is Assessment Literacy important and why is it especially
important for first year undergraduates?
(Yorke, 2001)
(Wingate, 2007)
(National Audit Office, 2002, p. 15)

13. Activity
• Take 2 minutes to make notes with regard to the task below
• Share any key points with the audience
In what ways do you believe your current practice already
incorporates elements to facilitate students’ development of
assessment literacy?
| 13

14. | 14
• improvement in students’ assessed performance
• the more students know what is expected of them in
assessments the more effectively they will be able to meet those
requirements
• students are keen to know how to improve due to the fact that
assessment is a dominant influence on their learning
Why Assessment Literacy?
(Price et al, 2010)

15. Approaches to developing student understanding of
assessment standards
| 15

16. • My background – ELT & Applied Linguistics
• My initial aims for the Enhanced students – to be the bearer of
knowledge, while strengthening their academic skills and abilities.
• The Cohort – 2 X ~45 students 80% young males:
- reluctance
- resentment
• The 1st week.
• Some student comments:
“why do I have to do this course!”
“what’s the point of this?”
“how is this going to help me!”
Aims reworked – make everything relevant to the students’ assessment.
| 16

17. Types of Assessment & My Intervention 1
1. Essay (Report) please refer to the marking criteria - ESSAY
1st draft – feedback via 1-2-1 tutorial and annotated essay
• Some areas covered
- Academic writing conventions e.g. Referencing/citations, thesis
statements & paraphrasing
- Criticality
- Discursive vs Descriptive writing
• Instruction given by means of:
- general teacher-led instruction in seminar and/or lecture format
- Individual / group tasks
- Peer review
= “Dominant Logic” Explicit Model
+ The Social Constructivist Model
| 17

18. Types of Assessment & My Intervention 2
2. Presentation please refer to the marking criteria - PRESENTATION
Practice presentation - feedback – final presentation
• Some areas covered:
- Formulaic language
- Formality / Style / Register
• Instruction given by means of:
- Individual / group exercises
- general teacher-led instruction in seminar and/or lecture
format
= “Dominant Logic” Explicit Model
| 18

19. The Cultivated Community of Practice Model
Not desirable!
– too early in UG course for this approach
– even in some cases of using the Social Constructivist Model type tasks
students would imply that I was unprepared for the class or was being
“lazy” by getting the students to do the assessing
• The benefits of Assessment Literacy may not, however, be immediately
obvious to authority-dependent students who are motivated by certainty
and believe it is the duty of infallible assessors to ‘correct’ student work
• Indeed, assessment illiterate students may simply consider less authority -
dependant assessment processes (such as peer review and peer
assessment) ingenious ways for lazy tutors to shirk their marking
responsibilities.
| 19
(Baxter Magold,1992; King & Kitchener, 2004 in Price et al, 2012)
(Price et al, 2012)

20. • Quantitative
• Comparison of means regarding enhanced vs. Jan start
cohort:
– Grades individual assessments and overall grades
– Self efficacy scores
• Regressions regarding grades self efficacy at end of year 1
• Qualitative
• Focus groups (data yet to be analysed)
Methods of evaluation
| 20

21. Findings
• Grades of enhanced students were slightly lower across all
assessments and many were found to be statistically significant;
with the exception of all assessment in HRM1004
• From a cross tabulation no differences were found with regard
to the pass/fail ratio across the two groups with the exception of
MSO
• No statistically significant differences found with regard to
‘Understanding the requirements of different assessments’; in
fact scores of the Enhanced students were higher that January
starts in all instances but not statistically significant
• With regard to confidence levels there were only significantly
significant differences in the case of report writing and again
those of the Enhanced students were higher
| 21

22. Findings cont’d
From the regression analysis not many models were found, however:
• Understanding the requirements of writing an essay explained 13% of the
variance of the HRM essay grade and 30% of the essay grade within the
Enhanced students sample; no relationship was found regarding the January
start students
• For all students the confidence in doing online tests explained 6.6% of the
variance of the HRM online test grade
• Understanding the requirement of online tests explains 8.3% of the variance for
enhanced students’ grades; no model for Jan start students
• Confidence in doing online tests explains 4% of the variance in MSO Test1 grade
for the all students sample; 10% of the variance in MSO Test 1 grade for the
enhanced students sample
• Confidence in doing online tests predicts 4% of MSO test 2 grade for the all
students sample
• Interestingly, understanding the requirement of online tests predicts 11% of
Accounting and Finance test grade for the Jan start sample
No other models found
| 22

23. Discussion of findings
• The groups were compared due to convenience and an
assumed difference in entry points (MBS 180-220 pts and
January Start 240+ pts; this could not be verified yet)
• Results indicate that January start students appear to be out
performing their Enhanced cohort counterparts although not
significantly in the case of HRM1004
Why?
• Enhanced students had perceived higher clarity of the
requirements of each assessment and were slightly more
confident than their January start counterparts.
• The regression models indicate that there was a relationship
between awareness and performance of the Enhanced
students, especially in the case of the HRM1004 Essay
• 30% of the variance explained
| 23

24. Conclusions and lessons learnt
• We don’t know whether this was down to the intervention or not,
therefore pre and post interventions are required
• What is students’ understanding of the requirements of different forms
of assessments
• Better matching and alignment of assessments could lead to better
outcomes; is assessment literacy tied to subject discipline or is it
transferable?
• What can be done differently
• the team is now made up of academics across all the departments,
therefore provision of subject based criteria for assessments
• Is there a longer term impact, how do year two grades compare across
the two cohorts?
• Can staff be developed to ensure clarity of criteria in a relatively
consistent manner across and within subject disciplines?
| 24

25. References
Brown, S. and Knight, P. (1994). Assessing learners in higher education. London: Kogan Page.
Drew, S. (2001). Student Perceptions of What Helps Them Learn and Develop in Higher Education.
Teaching in Higher Education. 6(3), pp. 309–331.
Stiggins, R. (1995). Assessment literacy for the 21st century. Phi Delta Kappan.
Price, M., Rust, C., O’Donovan, B., Handley, K. and Bryant, R. (2012). Assessment Literacy: The
Foundation for Improving Student Learning. Oxford: Oxford Brookes University.
The Higher Education Academy. (2012). A Marked Improvement Transforming Assessment in Higher
Education. [online]. Available at:
https://www.heacademy.ac.uk/sites/default/files/A_Marked_Improvement.pdf. [Accessed 1 July 2015].
Yorke, M. (2001). Formative Assessment and Its Relevance in Retention. Higher Education Research and
Development. 20 (2), pp. 115–123.
Wingate, U. (2007). 'A Framework for Transition: Supporting ‘Learning to Learn’ in Higher
Education‘. Higher Education Quarterly. 61(3), pp. 391-405. [online]. Education Research Complete.
Available at: http://rt5vx6na7p.scholar.serialssolutions.com. [Accessed 1 July 2015].
| 25