4. What do we do with event notifications?
Process
– acknowledge
– record
– assign
– ask for more information?
Evaluate
– are we sufficiently assured AO is mitigating incident/adverse
effect?
– are we sufficiently assured AO is addressing root cause?
– one off or part of a trend?
– management of event
Escalate
– incident management, investigation or regulatory action?
5. Analysis
Analysis to
– inform us of areas of concern
– build our understanding of the types of event notifications reported
– inform understanding of systemic risk and areas for further
monitoring and thematic review
Entity risk assessment
– we also analyse event notifications in the AO risk profiles, taking
into consideration the size of an awarding organisation and where
we have had no event notifications
Reporting
– monthly trend reports to Chief Regulator and Ofqual Board
6. What is an Adverse Effect?
Use these questions to judge if your incident could adversely effect:
– students – does this incident mean your current or potential
learners will be prejudiced?
– development, delivery and awarding – will this incident mean you
are not meeting the Conditions?
– standards – is the standard of your qualification compromised or
likely to be compromised?
– public confidence – will this situation affect public confidence in
your qualification?
7. Scenarios
One scenario per table – there are five different ones
Discuss the scenarios and think about:
– what information do you need to collect and how?
– what actions you will take and when?
– what are the potential adverse effects in your scenario?
– who will you inform of your actions and when?
You will receive additional information during the task. This might
mean your decisions and actions change
8. Feedback – Scenario one
Maladministration/malpractice
– security breaches and invigilation failings
Incident could adversely affect
– AOs ability to deliver and award qualifications in accordance with
the conditions
– performance standards
– public confidence
More information needed?
– do both incidents relate to door supervisor tests?
– are failings isolated or systemic across centre?
Action
– notify regulator and other AOs
– replace question paper, observe invigilation
– Investigate and apply sanction
9. Feedback – Scenario two
Breach of data protection law
– sensitive personal data shared and maybe lost
Incident could adversely affect
– AOs ability to deliver and award qualifications in accordance with
the conditions
– public confidence
More information needed?
– did the EV delete the information from the missing memory stick
before sending it?
– how was the missing memory stick sent and can it be tracked?
Action
– notify regulator
– seek legal advice about data protection responsibilities
– change centre and standardisation procedures
10. Feedback – Scenario three
Question paper error
– part question error causing difficulties for students
Incident could
– prejudice students
– adversely affect AOs ability to deliver and award qualifications in
accordance with the conditions; standards and public confidence
More information needed?
– how many students failed exam by getting 1a) wrong and not
completing the paper?
– is the qualification graded?
Action
– notify regulators and centres
– apply special consideration and review outcomes at awarding
– root cause analysis
11. Feedback – Scenario four
Marking error
– assessor transcription error not spotted by moderator
Incident has
– has prejudiced students
– adversely affect AOs ability to deliver and award qualifications in
accordance with the conditions; standards and public confidence
More information needed?
– does the error impact on just one component/unit or the whole
qualification?
– is the error isolated to one moderator?
Action
– notify regulator, centres and students
– re-issue results
– root cause analysis and mitigation
12. Feedback – Scenario five
Maladministration or malpractice
– ghost candidates?
Incident could
– adversely affect AOs ability to deliver and award qualifications in
accordance with the conditions; standards and public confidence
More information needed?
– why did the centre lose its funding contract?
– is the identity of the students genuine?
Action
– notify regulator and AOs
– apply sanction?
– investigate and sample evidence of assessment
13. Table discussion
Would you have done anything differently if you had known the
outcome at the beginning?
Is there anything more proactive you could have done to make your
decisions quicker and more effective?
Is there anything that has surprised you about:
– the way other tables have approached either the same task as
you, or different tasks?
– the way Ofqual has responded to the task?
Is there anything you disagree with?
Do you now know what, when and how to report to Ofqual?
14. Next steps for us
Review of non-statutory guidance
– include more information on what we would expect to receive
Review of statutory B3 guidance
– revised examples of positive and negative indicators
Introduce severity categorisation
– to inform analysis and reporting