Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Ofqual’s work on quality of marking 
October 2014
Introduction 
The proportion of GCSE and A/AS level papers where an EAR was 
requested has continually increased: 
4 
3.5 ...
Introduction 
There is a similar picture of grade changes resulting from EAR 
0.7 
0.6 
0.5 
0.4 
0.3 
0.2 
0.1 
% of GCSE...
Steps we are taking to improve the quality 
of marking 
1. Analysing this year’s Enquiries About Results data and 
requiri...
1. Analysing this year’s Enquiries About 
Results data and requiring more 
information from exam boards 
Issue 
 Provisio...
2. Improving the Enquiries About Results 
(EAR) and Appeals system to make it 
more simple, transparent and fair 
Issue 
...
3. Developing better ways for exam boards 
to measure and report on quality of 
marking in future 
Issue 
 EAR and Appeal...
4. Identifying best practice in mark 
scheme design 
Issue 
 Well designed mark schemes are 
important determinants of ma...
5. Requiring exam boards to upgrade 
action plans on quality of marking 
Issue 
 Before the summer we wrote to exam 
boar...
6. Requiring exam boards to improve their 
monitoring of markers as they mark 
Issue 
 Over 50,000 markers are involved i...
7. Evaluating the effectiveness of marker 
training 
Issue 
 Standardisation of markers is 
increasingly being carried ou...
Summary of actions 
Action Nov Dec Jan 
2015 
Feb Mar Summer 
2015 
Summer 
2016 
Summer 
2017 
Exam boards submit detaile...
Upcoming SlideShare
Loading in …5
×

Ofqual’s work on quality of marking

2,342 views

Published on

Analysing this year’s Enquiries About Results data and requiring more information form exam boards

Published in: Education
  • I've found your course really helpful and it's saved a lot of time so I can focus on my other subjects too such as English and Science. Your course has taught me a number of techniques to solve questions quicker. For instance, how to quickly turn recurring decimals into fractions. This has saved a lot of time when completing my mock exams. Thanks Jeevan. Your course is great! ▲▲▲ http://t.cn/AirrSv7D
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Ofqual’s work on quality of marking

  1. 1. Ofqual’s work on quality of marking October 2014
  2. 2. Introduction The proportion of GCSE and A/AS level papers where an EAR was requested has continually increased: 4 3.5 3 2.5 2 1.5 1 0.5 0 GCSE A level Total 2011 2012 2013 2014 % of scripts marked that received an enquiry % of GCSE and A level papers marked that received an EAR, 2011 - 2014
  3. 3. Introduction There is a similar picture of grade changes resulting from EAR 0.7 0.6 0.5 0.4 0.3 0.2 0.1 % of GCSE and A level qualification results involved in enquiries that had a grade change, summer exam series 2011 - 2014 0 2011 2012 2013 2014 % of total qualifications awarded resulting in a qualification grade change GCSE A level
  4. 4. Steps we are taking to improve the quality of marking 1. Analysing this year’s Enquiries About Results data and requiring more information from exam boards 2. Improving the Enquiries About Results (EAR) and Appeals system to make it more simple, transparent and fair 3. Developing better ways for exam boards to measure and report on quality of marking in future 4. Identifying best practice in mark scheme design 5. Requiring exam boards to upgrade action plans on quality of marking 6. Requiring exam boards to improve their monitoring of markers as they mark 7. Evaluating the effectiveness of marker training.
  5. 5. 1. Analysing this year’s Enquiries About Results data and requiring more information from exam boards Issue  Provisional data shows increased EAR requests and more grade changes  Increased EAR may reflect teacher anxiety around changes to qualifications and school performance measures  Grade changes may reflect small changes to marks just below a grade boundary…or may reflect more significant mark changes  In more subjective subjects, small mark changes may reflect legitimate differences of opinion  We need to understand whether there has been deterioration in the quality of marking this summer What we’re going to do  November: exam boards submit final EAR data to us, including:  Average size of mark adjustment for each paper  Detailed breakdown of the causes of any grade change  December: we will publish a report of the EAR data and analyses provided by exam boards  March 2015: deadline for all boards to have detailed quality of marking action plans based on their analyses  Summer 2015: boards must report on the impact of these plans on the summer exam series
  6. 6. 2. Improving the Enquiries About Results (EAR) and Appeals system to make it more simple, transparent and fair Issue  Some teachers lack confidence in the EAR and Appeals system  The EAR process fails to distinguish between true marking errors and legitimate differences of opinion  Exam boards have varying practice for the re-standardisation of markers  The basis upon which exam boards decide to authorise an extended review of marking is unclear and creates suspicion of unfairness  Teachers don’t think appeal panel hearings are sufficiently independent of the exam boards What we’re going to do  December: launch consultation on our proposals to overhaul the EAR and Appeals system, including:  All markers to be appropriately re-standardised  Boards to review and publish their principles for extending a review of marking  Boards to have procedures to demonstrate how they meet Appeals’ independence requirement  Summer 2015: we think some of our proposals can be put in place ready for the summer exam series, and the remainder by summer 2016
  7. 7. 3. Developing better ways for exam boards to measure and report on quality of marking in future Issue  EAR and Appeals data is not a good indicator of quality of marking  The current Appeals data lumps together small mark changes in subjectively marked subjects and large mark changes in objectively marked subjects…the latter are far more concerning than the former  There is poor visibility of the quality of marking. Teachers cannot make informed choices about exam boards on the basis of the quality of their marking What we’re going to do  Create standardised quality of marking indicators across exam boards to give a more accurate picture of marking quality  Summer 2017: indicators will be in place for the reformed GCSEs and revised A levels examined for the first time in 2017
  8. 8. 4. Identifying best practice in mark scheme design Issue  Well designed mark schemes are important determinants of marking quality, especially in more subjective subjects  Levels based mark schemes are often used in subjects associated with poorer consistency of marking  While a lot is known about how to write levels based mark schemes, there are still aspects about which there is little evidence What we’re going to do  We are conducting research studies into levels based mark schemes so we can require boards to follow best practice  Summer 2015: we will begin to report findings from this long term piece of work
  9. 9. 5. Requiring exam boards to upgrade action plans on quality of marking Issue  Before the summer we wrote to exam boards to check what actions they were taking in response to the issues raised in our Quality of Marking report, published in February 2014  Exam boards wrote back during the summer with action plans which we have reviewed What we’re going to do  We have identified the need for more detailed plans to be produced to address outstanding areas of concern raised in our original Quality of Marking report  We expect revised action plans to be with us by early November for review. We will then monitor their implementation as part of our regular audit activity.
  10. 10. 6. Requiring exam boards to improve their monitoring of markers as they mark Issue  Over 50,000 markers are involved in GCSE and A level marking: the vast majority behave with utmost professionalism  But some rogue markers may need to be weeded out  Online marking opens the possibility of marker probity systems: real time data analysis to identify patterns that would trigger investigation and action What we’re going to do  2015: require exam boards to:  Tell us how they currently monitor marker probity  Consider how they could improve these systems  Consider implementing new, more robust systems  Summer 2015: some improvements to marker monitoring will be in place  Summer 2016: noticeable improvements to quality control systems will have been made
  11. 11. 7. Evaluating the effectiveness of marker training Issue  Standardisation of markers is increasingly being carried out online  Online standardisation has cost and speed advantages and can be as good as face-to-face training  But online standardisation is unpopular with some markers  And exam boards vary greatly in how they deliver it – the quality of online standardisation is not always as high as it could be What we’re going to do  Conduct an extensive evaluation of the impact of online standardisation on marking quality:  Differences between exam boards  Where does it work  Where does it not work, and why  Audit exam boards’ standardisation processes in summer 2015  Carry out a research evaluation of the impact of online standardisation on marking quality  Investigate how boards monitor and improve their standardisation processes over time
  12. 12. Summary of actions Action Nov Dec Jan 2015 Feb Mar Summer 2015 Summer 2016 Summer 2017 Exam boards submit detailed EAR data analysis  We publish our report of the boards’ EAR data and analyses  We launch consultation on our proposals to overhaul the EAR and Appeals process  Deadline for boards to have detailed QoM action plans in place  Boards start reporting on the impact of their QoM plans on the summer exam series  We audit exam boards’ standardisation processes  Some improvements to examiner monitoring are in place  Some reforms to the EAR and Appeals process are in place  We begin to report findings from our research into levels based mark schemes  All reforms to EAR and Appeals process are implemented  Noticeable improvements to exam boards’ quality control systems are in place  Standardised QoM indicators are in place for reformed GCSEs and A levels 

×