1) The document discusses potential metrics for measuring teaching excellence and learning gain in higher education. 2) It reports the results of a session where participants ranked various proposed metrics. The highest ranked metrics for teaching excellence included teaching awards, qualifications/fellowships, and evidence of sustained impact on student learning. 3) For learning gain, the highest ranked metrics included student grades, surveys measuring changes in attitudes/behaviors, and data on student engagement and experiences. 4) The document concludes that while NSS student feedback appears suitable for the Teaching Excellence Framework, other high ranked metrics could provide better measures, and contextualization is important.

1. Outcomes of session
SUMMARY
23 May 2017

2. Choosing the Metrics – number of responses
Learning Gain (LG)
• Grades (GPA) - entry to attainment (‘gap’) – 7
• Student feedback (eg NSS – 6; UKES – 1; bespoke
questionnaire/interview - 1)
• PDP portfolio/reflective log - 6 (eg self-efficacy,
student stories, co-curricular)
• Student engagement - attendance/analytics – 3
• Progress tracking (ALIS) – monitoring - 2
• Careers registration - DLHE - 2
• Employer or customer feedback - 2
• Employability awards - 2
• HEFCE projects (eg 13 LG pilots plus NMMLGP) –
pre/post tests and research - 1
• Professional accreditation - 1
Teaching Excellence (TE)
• Student feedback NSS – 7; PTES – 2; Module
feedback – 6
• Teaching qualifications/HEA fellowship – 7
• Student engagement – 6 (ie attendance -2; co-
curricular activities – 1; research – 1; focus group – 2)
• Awards ie student-led - 2 / internal TE awards – 6
• DLHE - 2
• HESA (non-continuation) - 2
• National Teaching Fellowship or awards - 2
• Evidence of sustained impact from teaching
interventions – 2; anecdotal evidence - 1
• Contact time – eg KIS indicators – 1
• HMRC? (Salary) – 0
The Holy Grail for HE - teaching excellence and learning gain

3. COMBINED RANK – relative importance
LEARNING GAIN
• Student attainment/Grades - 24
• Learning gain (ie longitudinal
surveys /self efficacy & stories) - 20
• Student feedback/satisfaction - 7
• Tracking (monitoring) data - 5
• Employment (DLHE) - 5
• Employer feedback - 5
• Student Engagement - 4
• Capability/competencies – 3
• Pre- and post-tests – 2
• UKES – 1
TEACHING EXELLENCE
• Teaching (NTF) awards – 21 (10) = 31
• Teaching quals./HEA fellowship - 19
• Sustained evidence of impact on learning
– 12
• Module/student evaluation - 4
• Attendance – 4
• Anecdotal evidence – 4
• Student achievement – 3
• Co-circular engagement – 3
• Focus groups – 2
• Employment (DLHE) – 1
The Holy Grail for HE - teaching excellence and learning gain

4. Conclusions
• Current TEF metrics - only NSS appears fit-4-purpose (DLHE and HESA non-
continuation rank much lower)
• Preferred metrics for Teaching Excellence include teaching qualifications or
recognition (HEA fellowships), awards for excellence and evidence of impact
on the student learning experience
• In terms of Learning Gain, grades are useful to measure the ‘distance
travelled,’ whilst longitudinal surveys of attitudes and behaviours (eg self-
efficacy) including the broader student experience (co-curricular) better reflect
the ‘student journey’
• Recognition that metrics must be contextualised (eg student ‘stories’ and
‘anecdotes’; institutional TEF submissions)
• Interestingly, employability/employment was not ranked highly –
• does this mean that the purpose of HE is about ‘learning for life’ not a specific job?
• does this reflect the complexity of comparing vocational with more academic disciplines?
• does this reflect the difficulties of benchmarking with complex geographical, demographical,
socio-economic and cultural impacts (beyond University)?
The Holy Grail for HE - teaching excellence and learning gain