Standards in public examinations

1,190 views
1,066 views

Published on

Isabel Nisbet spoke to the Guild of Educators on 29 January 2009 about standards in public examinations

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,190
On SlideShare
0
From Embeds
0
Number of Embeds
66
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Standards in public examinations

  1. 1. Standards in public examinations Isabel Nisbet, Acting CEO, Ofqual Guild of Educators, 29 January 2009
  2. 2. Some thoughts on standards <ul><ul><ul><li>“Standards are always out of date. That’s what makes them standards” (Alan Bennett, Forty Years On ) </li></ul></ul></ul>
  3. 3. Some thoughts on standards <ul><ul><ul><li>“Standards are always out of date. That’s what makes them standards” (Alan Bennett, Forty Years On ) </li></ul></ul></ul><ul><ul><ul><li>“So you’re the Minister for Standards? That must mean you’re the Minister for No Change” (Ken Boston at introductory meeting to new DfES Minister) </li></ul></ul></ul>
  4. 4. Some thoughts on standards <ul><ul><ul><li>“Standards are always out of date. That’s what makes them standards” (Alan Bennett, Forty Years On ) </li></ul></ul></ul><ul><ul><ul><li>“So you’re the Minister for Standards? That must mean you’re the Minister for No Change” (Ken Boston at introductory meeting to new DfES Minister) </li></ul></ul></ul><ul><ul><ul><li>“We should listen to the disinterested and dispirited researchers from the mathematics department at Coventry University who have concluded that an A-level candidate in mathematics who failed in the early 1990s would, if he or she were sitting the exam now, be awarded a grade B.” (Chris Woodhead, 2004) </li></ul></ul></ul>
  5. 5. More on standards <ul><li>“In 1984, 70.1% of candidates passed their A levels and 9.3% were awarded an A grade. This year, two decades on, 96.3% passed, 22.4% achieved the top grade and the controversy over the meaningfulness of these remarkable statistics rages once again in what has become, for everyone involved, a deeply dispiriting ritual” (Chris Woodhead, 2004) </li></ul><ul><li>“ The government says standards in science have improved year on year. According to the Royal Society of Chemistry, pupils are drilled to answer &quot;undemanding questions to satisfy the needs of league tables and national targets .”” (November 2008) </li></ul>
  6. 6. Fings ain’t wat they used to be <ul><ul><ul><li>“On one matter there was almost universal agreement; that among university entrants in general the ability to write English is disappointingly low.” (1960 JMB Annual Report) </li></ul></ul></ul><ul><ul><ul><li>“The only point that calls for report is the general weakness of a large proportion of the candidates” (Pure and Applied Mathematics, Higher School Certificate, JMB, 1924) </li></ul></ul></ul>
  7. 7. Outline <ul><li>Standards </li></ul><ul><li>Public examinations </li></ul><ul><li>What is the public interest in standards of assessment? </li></ul><ul><ul><ul><ul><li>Undergraduate medical education </li></ul></ul></ul></ul><ul><li>Maintaining standards </li></ul><ul><li>Ofqual’s role in maintaining standards </li></ul>
  8. 8. Standards of assessment and standards of performance
  9. 9. Standards of assessment <ul><li>The height of the hurdle, not how many people jump over the hurdle </li></ul><ul><li>Language of consistency, fairness, lack of change – compare the rhetoric of “driving up standards” [of performance against the standard] </li></ul><ul><li>Measuring change requires unchanging measures </li></ul><ul><li>No win for the student: </li></ul><ul><ul><ul><ul><li>“If fewer students than the previous year are awarded the grades then they were weak students, unable to reach the standard; if more students gain the grades then the examination was so easy that the grades are worthless” (Kathleen Tattersall, 2008) </li></ul></ul></ul></ul>
  10. 10. Public examinations – the statutory definition <ul><li>“In this Chapter references to a public examination (or a prescribed public examination) are references to such an examination as it applies in relation to persons who are entered for a syllabus for that examination with a view to meeting the examination requirements for that syllabus so as to qualify for assessment for the purposes of determining their achievements in that examination on any particular occasion in any year when an assessment takes place” (Education Act 1996, S462(4) </li></ul>
  11. 11. Public examinations – the statutory definition <ul><li>“In this Chapter references to a public examination (or a prescribed public examination) are references to such an examination as it applies in relation to persons who are entered for a syllabus for that examination with a view to meeting the examination requirements for that syllabus so as to qualify for assessment for the purposes of determining their achievements in that examination on any particular occasion in any year when an assessment takes place” (Education Act 1996, S462(4) </li></ul><ul><li>So that’s clear, then…. </li></ul>
  12. 12. What’s a public examination? <ul><li>An examination where aspects of the requirements for attainment are set by the State </li></ul><ul><li>Often by arm’s length bodies or professional regulators </li></ul><ul><li>Normally backed up by a responsibility to the same body to make sure that standards are maintained </li></ul><ul><li>Statutory basis for role of Government and the standard-setter/monitor </li></ul>
  13. 13. What is the public interest in examination assessment standards? <ul><li>For schooling – means to reinforce the National Curriculum (England only) </li></ul><ul><li>(all countries) As a vehicle for providing public assurance about the education of young people </li></ul><ul><li>To provide a measure for the attainment of: </li></ul><ul><ul><ul><ul><li>Individuals </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Schools </li></ul></ul></ul></ul><ul><ul><ul><ul><li>The nation? </li></ul></ul></ul></ul><ul><ul><ul><li>To provide a basis for discrimination among applicants: </li></ul></ul></ul><ul><ul><ul><ul><li>Further/Higher Education </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Employment </li></ul></ul></ul></ul><ul><ul><ul><ul><li>The exam content must be valid (in relation to the matters of concern to the selector) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>The exam results should be reliable </li></ul></ul></ul></ul>
  14. 14. The public interest (ctd) <ul><li>An assurance of competence/fitness to practise </li></ul><ul><li>Public protection/safety </li></ul><ul><ul><ul><ul><li>Invasion of the human body </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Work with vulnerable members of the public </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Work which affects the safety of the public </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Work required to provide public goods (e.g. roads, environmental infrastructure) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Consumer protection </li></ul></ul></ul></ul><ul><ul><ul><li>The traditional professions </li></ul></ul></ul><ul><ul><ul><ul><li>Law, medicine…. </li></ul></ul></ul></ul>
  15. 15. Initial medical education <ul><li>Primary legislation requires the regulatory body to: </li></ul><ul><ul><li>“Determine the extent of the knowledge and skill which is to be required for the granting of primary United Kingdom qualifications and secure that the instruction given in universities in the United Kingdom to persons studying for such qualifications is sufficient to equip them with knowledge and skill of that extent. </li></ul></ul><ul><ul><li>“ Determine the standard of proficiency which is to be required from candidates at qualifying examinations and secure the maintenance of that standard ” (Medical Act 1983) </li></ul></ul>
  16. 16. “Maintaining the standard” <ul><li>Making sure that the examinations (and other assessments) validly measure attainment against the requirements set by the regulator </li></ul><ul><li>Ensuring that the outcomes are reliable </li></ul><ul><ul><ul><ul><li>Our lives may depend on that </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Project-based learning V terminal exam </li></ul></ul></ul></ul><ul><ul><ul><li>Keeping the standard itself under review </li></ul></ul></ul><ul><ul><ul><ul><li>Content – validity: is it relevant to what doctors need to be able to do? </li></ul></ul></ul></ul><ul><ul><ul><li>A fair basis for discriminating among applicants for junior doctor posts </li></ul></ul></ul>
  17. 17. As medicine changes the standard may change <ul><li>“ Medicine used to be simple, ineffective and relatively safe. Now it is complex, effective and potentially dangerous” (Sir Cyril Chantler) </li></ul><ul><li>Continuity over time less important than fitness for purpose now </li></ul><ul><li>If medicine is more difficult now, then the standard of assessment should rise </li></ul><ul><li>If some techniques are easier now because of technology (e.g. technical drawing??) that doesn’t matter </li></ul>
  18. 18. Maintaining standards in public examinations <ul><li>Setting appropriate standards for the attainment to be shown </li></ul><ul><li>Defining how categories/grades are to be awarded </li></ul><ul><li>Ensuring that the award means what it says and can be understood/used </li></ul><ul><li>Ensuring that different versions of the SAME qualification (e.g. A level history) are genuinely comparable </li></ul><ul><li>Fairness to candidates </li></ul><ul><li>Commanding confidence </li></ul>
  19. 19. Maintaining exam standards – how to do it <ul><li>Ensure that the organisations running the exams are able to control their own standards (institutional audit) </li></ul><ul><li>Examination Design </li></ul><ul><ul><ul><ul><li>Number and design of units </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Grading structure </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Methods of assessment </li></ul></ul></ul></ul><ul><ul><ul><li>Adjustment during the live marking process </li></ul></ul></ul><ul><ul><ul><ul><li>QA of markers’ judgements </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Statistical checks and balances </li></ul></ul></ul></ul><ul><ul><li>Retrospective monitoring and feeding lessons into design </li></ul></ul>
  20. 20. Some challenges to confidence in standards <ul><li>Grade drift (or larger numbers getting higher grades) </li></ul><ul><ul><ul><ul><li>Re-calibration? </li></ul></ul></ul></ul><ul><ul><ul><li>Lack of continuity over time </li></ul></ul></ul><ul><ul><ul><li>Questioning of comparability between subjects: </li></ul></ul></ul><ul><ul><ul><ul><li>Research-based </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Urban myths </li></ul></ul></ul></ul><ul><ul><ul><li>Change in the system </li></ul></ul></ul><ul><ul><ul><li>Concerns about malpractice (e.g. internet plagiarism) </li></ul></ul></ul><ul><ul><ul><li>Perceived pressures to compromise standards </li></ul></ul></ul><ul><ul><ul><ul><li>To show increased performance </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Market pressures </li></ul></ul></ul></ul><ul><ul><ul><li>Wish to preserve the exam misery we suffered…. </li></ul></ul></ul>
  21. 21. The cold shower theory of exams
  22. 22. A levels – the gold standard? <ul><li>Increases in: </li></ul><ul><ul><ul><ul><li>Numbers/percentages passing </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Numbers/percentages getting A grades </li></ul></ul></ul></ul><ul><ul><ul><ul><li>University selection problems </li></ul></ul></ul></ul><ul><ul><ul><li>Better performance or dumbing down? </li></ul></ul></ul><ul><ul><ul><li>Hard subjects and soft subjects </li></ul></ul></ul><ul><ul><ul><li>The A* </li></ul></ul></ul><ul><ul><ul><li>Review in 2013 </li></ul></ul></ul><ul><ul><ul><ul><li>Case for re-calibrating? </li></ul></ul></ul></ul><ul><ul><ul><li>Rival qualifications </li></ul></ul></ul><ul><ul><ul><ul><li>Welcome competition </li></ul></ul></ul></ul>
  23. 24. About Ofqual <ul><ul><ul><li>Independent of Government and QCA </li></ul></ul></ul><ul><ul><ul><ul><li>Will report to Parliament </li></ul></ul></ul></ul><ul><ul><ul><li>To ensure standards and confidence in qualifications, exams and tests </li></ul></ul></ul><ul><ul><ul><li>Launched (in shadow form) in April 08 </li></ul></ul></ul><ul><ul><ul><li>Legislation in 08-09 session </li></ul></ul></ul><ul><ul><ul><li>Meantime distinct part of QCA </li></ul></ul></ul>
  24. 25. What we are trying to be <ul><li>The independent regulator of qualifications and assessments that are valued and trusted by learners, users and the wider public </li></ul><ul><li>Maintaining standards in existing, new and revised qualifications and assessments </li></ul>
  25. 26. Outline <ul><li>Standards </li></ul><ul><li>Public examinations </li></ul><ul><li>What is the public interest in standards of assessment? </li></ul><ul><ul><ul><ul><li>Undergraduate medical education </li></ul></ul></ul></ul><ul><li>Maintaining standards </li></ul><ul><li>Ofqual’s role in maintaining standards </li></ul>
  26. 27. Some conclusions <ul><li>This is about standards of ASSESSMENT – the height of the hurdle </li></ul><ul><li>Public examinations have a public interest in the setting and maintenance of standards – that interest is paramount </li></ul><ul><li>Undergraduate medical education and A levels – some common principles </li></ul><ul><li>Need to challenge some of the chatter: </li></ul><ul><ul><ul><ul><li>Continuity over time is all (it isn’t, but can sometimes be important) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>You can get a high grade writing gibberish (you can’t if writing’s being tested) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>University entrants can’t spell (they never could…) </li></ul></ul></ul></ul><ul><ul><ul><li>Independent regulator – there to assure standards and confidence </li></ul></ul></ul>

×