Downloaded 19 times
Uploaded byHigher Education Funding Council for England
Next steps for TEF - Chris Millward
Chris Millward from the Royal College of Physicians gave a presentation at the HEFCE Conference on lessons learned from the Teaching Excellence Framework (TEF). The key points were: - The TEF process was found to be fair and transparent, with clear findings based on metrics, submissions, and guidance. - Minor refinements will be made to the TEF for Year Three, but the overall structure and methodology will remain the same. - A subject pilot will explore rating subjects as well as institutions using two proposed models. - Recruitment of panel members, providers, and subjects is underway for participation in the TEF and subject pilot in the future.
More Related Content
Similar to Next steps for TEF - Chris Millward
More from Higher Education Funding Council for England
Next steps for TEF - Chris Millward
- 1. HEFCE Conference Chris Millward DirectorPolicy Royal College of Physicians 12 October 2017 Follow us on Twitter at #hefcecon2017
- 2. Teaching Excellence Framework:Lessons Learned • Process fair and transparent • Clear and robust findings on basis of metrics, submissions and guidance • Balance of evidence between metrics and submissions broadly right • No significant biases • No changes to overall structure or methodology • Some areas for improvement identified to strengthen accountability and fully recognise excellence.
- 3. TEF – theChair’s 10 conclusions • TEF focuses on things surveys say matter to students: routes to work, retention, assessment, quality of teaching. • TEF is part of an increased focus on outcomes for students, shifting from the focus on inputs, processes and outputs. • Metrics matter, but need to be benchmarked. Metrics generate hypotheses and anchor initial judgements • At the core of TEF is a strategic clarity: the relationship between institutional policies, institutional practices and student outcomes. • The best accounts did not describe initiatives but systematically demonstrated the difference they had made. • Initiatives carried conviction when part of a coherent strategy for improvement. • Technology tools matter less than how tools are used. • Genuine student involvement stood out: an embedded culture of engagement at every level. • Too many institutions used context as excuse rather than analysing and responding to the challenges it offered. • Excellence is distributed across the sector and matches its diversity.
- 4. Principal refinements forTEF Year Three Core: • NSS weighting; part-time provision; high & low absolute values Supplementary: • LEO & grade inflation Link with other processes: • Director of Fair Access; Quality Assessment There will be no changes to the overall structure or methodology of the TEF There will be the same method and process of assessment, with peer review central and the same criteria, ratings, core metrics and the same use of the provider submission.
- 5. TEF Year Three:Key dates Date Activity 9 October 2017 DfE publish TEF specification and Lessons Learned report 19 October 2017 HE providers have access to their metrics on TEF extranet HEFCE publishes TEF Year Three procedural guidance TEF extranet opens for provider applications to TEF Year Three November 2017 Provider webinars and briefing events 18 January 2018 Deadline for provider applications Early June 2018 Outcomes published
- 6. Subject Pilot specification ModelA Top-down, ‘by exception’ model gives provider-level rating and subjects the same rating as provider where metrics the same, with fuller assessment (and potentially different ratings) where metrics differ. Model B ‘Bottom-up’ model gives subject-level ratings, feeding into the provider-level assessment and rating. Subjects are grouped for submissions, but ratings are awarded at subject-level. Teaching intensity - piloting two measures in a selection of subjects in both models
- 7. Subject pilot providerrecruitment 83 providers have volunteered: • 77 England • 2 Scotland • 1 Northern Ireland • 3 Wales Mix of provider types • 46 HEIs (including 5 small and specialist) • 22 FECs (both franchised and own provision) • 15 Alternative providers A good spread of providers showed interest in participating in either one or both models
- 8. Panel member recruitment •3 groups of applicants • 3 types of panel • 15 types of role • 100 vacancies No of applications Experts (employer/PSRB/WP) 85 Academic panel members / assessors 595 Student panel members / assessors 169
- 9. Future of TEF •Ongoing Evaluative work • 1 January 2018 HERA clause 25 commences 1 January 2018 • January to April 2018 HEFCE implements TEF on behalf of OfS • April 2018 OfS responsible for TEF • AY 2018/19 Independent Review of TEF • AY 2019/20 TEF subject level specified (5 year cycle)
- 10. Copyright The copyright inthis presentation is held either by the Higher Education Funding Council for England (HEFCE) or by the originating authors. Please contact customerservices@hefce.ac.uk for further information and re-use requests.
- 11. Thank you forlistening c.millward@hefce.ac.uk Follow us on Twitter at #hefcecon2017
Editor's Notes
- #3 The operation of Year Two of the TEF has demonstrated that the process has operated in a fair and transparent way. Assessors were able to come to clear and robust findings on the basis of the metrics, provider submissions and guidance; the balance of evidence between metrics and submissions was broadly right; the moderation process worked well; and the results are generally perceived as credible and reflecting teaching excellence across the sector. Statistical analysis of the results has confirmed that there were no significant biases by region, gender, ethnicity, social disadvantage, subject breakdown and research income. We are therefore making no changes to the overall structure or methodology of the TEF, and will be retaining the method and process of assessment, the centrality of peer review, the criteria, the ratings, the core metrics and the use of the provider submission. The lessons learned exercise did, however, also highlight a number of areas for improvement: either refinements to the existing assessment process to ensure excellence is fully recognised, or steps to strengthen the way that TEF holds providers to account for delivering excellent teaching.
- #5 TESOF – may need to change this logo….. The NSS remains a key component of TEF but, in order to achieve a more balanced assessment, the weight of each NSS metric will be halved for the purposes of determining the initial hypothesis. Part-time provision In order to recognise excellence in part-time provision appropriately, we will offer providers with a majority of part-time provision (≤35%) the opportunity to submit an additional page of quantitative information alongside their metrics and refine the assessment procedure for providers with similar numbers of full-time and part-time students. Very high and low absolute values Whilst benchmarking will remain at the heart of TEF assessment, the flagging and benchmarking system has limitations at the extreme ends of the metrics. We will therefore explicitly mark the top and bottom 10% of absolute values for each metric. These indicators will be taken into account in the calculation of the initial hypothesis, where a provider is not already flagged. Longitudinal Education Outcomes (LEO) data We will include supplementary metrics derived from LEO data in order to exploit the rich data about graduate employment now available. As supplementary metrics, these will not affect the initial hypothesis but will be considered alongside the provider submission. Grade inflation metric - DfE very keen on implementing this strictly…. Expect it to be mandatory that you declare it if you have it. (but don’t worry if you don’t have it) and it will have to be only for providers with DAPs and presumably only for those students that the provider with DAPs teaches? Check spec carefully on this aspect when it is published. A new supplementary metric on grade inflation will be linked to the existing criterion on ‘Rigour and Stretch’ to aid assessors in making judgement in this area and allow providers that are taking genuine steps to tackle grade inflation to be recognised for doing so. Allow the Director for Fair Access, or successor, to comment on whether ‘gaming’ has taken place Power of referral Where the assessment process suggests concerns about the underlying quality of a provider, the TEF Chair will have the right to refer the provider to the Office for Students with a recommendation that the OfS should consider an investigation to confirm whether the provider continues to meet baseline quality requirements.
- #6 Plus 01 November 2017 Webinar: TEF Year Three Metrics 02 November 2017 Webinar: Introduction to TEF Year Three 7, 10, 14, 15 & 17 November TEF Year Three Briefings - Invite only events (1 member of staff & 1 student from each eligible provider will be invited to attend) and February - May 2018 Assessment period and June - July 2018 Appeals process August 2018 Publication of any changes to TEF outcomes as a result of appeals
- #9 WP and Employment experts and Employer/PSRB representatives 3 broad categories of applicants 15 different role types across TEF 3 and Pilot Main and Subject Panels 110 vacancies
- #10 Transition to the Office for Students and Independent Review 12. The Government intends to commence Clause 25 of the Higher Education and Research Act 2017 (HERA) on 1 January 2018, in line with the creation of the Office for Students (OfS). Between January and April 2018, HEFCE will implement the TEF on behalf of the OfS and, from April, the OfS will be solely responsible. This will not cause disruption for providers involved in the process or for students. 13. We therefore anticipate the Independent Review will take place in academic year 2018/19, in line with the timetable set out in HERA. We intend for the Independent Review to report in time to determine the assessment specification for 2019/20, which will also be the first year of assessments at subject-level. 14. The assessments taking place in academic year 2019/20 will therefore constitute the completion of the TEF development process. At this point, subject to the findings of the Independent Review, we anticipate that TEF will move to a five-year cycle: TEF ratings will last for five years, with providers able to reapply for assessment either three, four or five years after their last application. This will deliver value for money for the sector and be a proportionate approach to adopt once the TEF has been independently reviewed.