Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Ofqual Audit Approach: April 2015

1,238 views

Published on

Audit update
Slides from a webinar to the Federation of Awarding Bodies on Monday 27 April 2015

Webinar hosted by Bryan Horne
Associate Director Standards for Vocational Qualifications and Apprenticeships


Published in: Education
  • Be the first to comment

  • Be the first to like this

Ofqual Audit Approach: April 2015

  1. 1. Audit update Bryan Horne - Associate Director Standards for Vocational Qualifications and Apprenticeships April 2015
  2. 2. Areas to cover What we are aiming to achieve How we use our risk model Statement of compliance 2015 Commissioning process Audits’ place amongst our regulatory ‘tool kit’ Spring audit programme - feedback What next?
  3. 3. What we are aiming to achieve We use a range of tools to gain assurance about whether awarding organisations are meeting our requirements and therefore delivering sufficiently valid qualifications. Our tools include: – annual statement of compliance – audit – technical evaluation, and – investigation. Through a risk-based programme of audit work - we gather evidence about how well an awarding organisation’s systems, processes and controls are meeting our regulatory requirements. Building a baseline evidence base – allowing for increasingly focussed activity in areas of highest risk.
  4. 4. Lifecycle
  5. 5. How we use our risk model Our risk framework allows us to identify risks to the provision of valid qualifications from: – an organisation’s governance, financial structure, internal processes, people and systems, and/or – external factors. We assess risk through: – qualification portfolio analysis, – risk assessments against defined risk indicators, – risk intelligence, – market and systemic risk analysis, and – audit, research and investigation outcomes. Targeting regulatory compliance activity - highest risk organisations are subject to stronger scrutiny. All have some form of scrutiny, but it is progressively less intense for lower- risk organisations.
  6. 6. Statement of compliance - 2015 Your annual statement is in accordance with the requirements set out in Condition B2. To make your submission refer to the instructions we issued on 2nd April. The submission window opens 1st June and closes 30th September. Your annual statement must be submitted during this window. We will tell you next month how to access the online form used to make your statement. We will use your statement in conjunction with the other information we hold about you to inform our regulatory approach.
  7. 7. Declaration
  8. 8. Details of Non compliance….
  9. 9. Commissioning process Risk model outputs Standards directorates determine priorities Audit programme determined Audits commissioned and agreed with audit team Audits carried out Audit outputs fed back to commissioner Next steps determined
  10. 10. Audits’ place amongst our regulatory ‘tool kit’ Annual statement of compliance – annual statement of compliance with the Conditions of Recognition. Audit – gathers evidence about how well an awarding organisation’s systems, processes and controls are meeting our regulatory requirements. Technical evaluation – appropriate research and evaluation methods to assess and make expert judgements about how well awarding organisations apply their processes so that qualifications are sufficiently valid. Investigation – we may investigate where there is information that leads to a suspicion of non-compliance with our Conditions of Recognition.
  11. 11. Audit approach We start with the assumption that an awarding organisation is fully compliant with the Conditions. An audit tests how this is achieved by examining processes, systems and resources. Because our Conditions are ‘outcome focussed’ – we may sometimes wish to examine products as well as the systems in place to produce them. A risk driven approach. Where we have a concern, we may provide feedback – an awarding organisation can then consider how to act upon this. In some cases – we may not provide feedback.
  12. 12. Testing systems and products Technical evaluation of assessment Tests: - systems - processes - resources Credible? Used? Assessment procedure? Used? Will do this Will do this Yes Fit for purpose qualification Yes Invalid assessment Invalid assessment NoYes Unlikely to deliver valid assessment No Unlikely to deliver valid assessment No Evaluates - assessment procedures NoYes
  13. 13. Spring audit programme - 1 Approximately 25 organisations selected Testing how organisations comply with: – processes for obtaining User support (Condition E1.3), – levels of resources and arrangements (Condition A5), and – assessment design (Condition E4.1 and E4.2). When the Regulatory Compliance team receives a commission to undertake an audit - awarding organisation will be notified of the following: – notification Regulatory Compliance has received the commissioned work, – the Commissioner, – the regulatory purpose of the audit, – the key lines of enquiry (KLOE) for the audit, – the relevant Condition(s) to be audited, and – the timetabling for the audit activity.
  14. 14. Spring audit programme - 2 Audit work Supplementary information provided (+ 5 days) Factual checking (+ 10 days) Evidence record agreed (+ 15 days) Findings record to Commissioner (+ 25 days) Next steps communication (+ 30 days) Representation Report agreed Findings analysed Audit Checking Next steps
  15. 15. Spring audit programme – feedback so far Good points Audits were handled well - two/three weeks notice of an audit. AOs invited to select two qualifications for audit - in this programme we selected two, but provided notice. The audit teams were friendly and approachable - AOs noted that the audits took less time than they had anticipated. To consider  Audits still ‘work in progress’ with a concern about inconsistency – feedback ‘on-site’/written.  Ofqual to provide written report sooner.  AOs gathered together more evidence than necessary in some cases.  AOs would like the opportunity to feedback on their experience of the audit.
  16. 16. What next? Audit programme will continue – building a baseline evidence base about how Conditions are complied with. Supported by outcomes of annual statement of compliance. Technical evaluation programme will begin. In some cases – organisations can expect a greater intensity of audit work. Where evidence builds and perceived risks become lower – the intensity of scrutiny will be lower. If we determine that there has been a breach of our requirements, we will decide what regulatory action to take. In some cases we will review our regulatory requirements to see if they can be improved. When we propose to change requirements, we will consult awarding organisations and the public.
  17. 17. Questions Bryan Horne Associate Director Standards for Vocational Qualifications and Apprenticeships, Ofqual

×