3. 3 05/05/2016
Three common metrics (suitably
benchmarked):
Student satisfaction indicators – National
Student Survey (teaching quality and
learning environment)
Employment/destination – Destination of
Leavers from Higher Education, from
2017 results of Her Majesty’s Revenues
and Customs (tax returns)
Retention/continuation – student
outcomes (annual data returns to HESA)
Teaching Excellence Framework: proposed common metrics
4. Student engagement
Gibbs, G (2016) ‘Teaching’ in Response to the Higher Education
Green Paper HEPI
US research indicates student engagement is the key factor
is promoting student learning gain (e.g. Pascarella &
Terenzini)
Chickering & Gamson (1987) Seven principles for good
practice in undergraduate education
4 05/05/2016
5. Seven principles for good practice in undergraduate education
(Chickering and Gamson 1987) – with examples
Good practice…
…encourages student-faculty contact (e.g. students as partners in
research; in curriculum development/assessment – peer assessment)
…encourages cooperation among students (peer-assisted learning)
…encourages active learning (placements)
…gives prompt feedback
…emphasises time on task (directed student learning)
…communicates high expectations
…respects diverse talents and ways of learning (inclusive learning)
5 05/05/2016
6. HEA’s favoured metrics
Number of staff with accredited teaching qualifications (HEA Fellows) – HESA staff
return
UK Engagement Survey (UKES) - adopted and adapted questions from National
Survey of Student Engagement (NSSE) ‘assesses the extent to which students are
participating in educational practices that are strongly associated with high levels of
learning and development’ (Kuh 2001)
6 05/05/2016
7. UK Engagement Survey. Selection of questions
• Reflective and integrative learning
E.g. “How often have you connected your learning to societal problems or issues?”
• Time spent
E.g. “About how many hours do you spend in a typical 7-day week preparing for taught
sessions?”
• Skills development
E.g. “How much has your experience at this institution contributed to your knowledge, skills,
and personal development in solving complex real-world problems?”
• Engaging with research
E.g. “How much has your coursework emphasised learning about the results of current
research?”
• Creating knowledge
E.g. “How much has your coursework emphasised your active participation in creating
knowledge?”
7 05/05/2016
Editor's Notes
The GP reiterates that there is no accepted definition of TE and instead proposes 3 aspects of TE: TQ; LE and St Outcomes and LG.
It also provides an indicative list of criteria under each aspects. The sector has responded by saying that the three aspects and the proposed criteria, whilst they might have been differently phrased, do cover the areas of TE reasonably comprehensively.
HOWEVER, when it comes to the metrics to be deployed to assess institutional performance, concerns have been raised.
principles for metrics: valid; robust; comprehensive; credible; current
So three common metrics – known to be, at best, proxies for TE are proposed; are designedly OUTCOME measures;
Student satisfaction is not measure of teaching excellence; though the questions of assessment and feedback come closer to measuring the quality of one aspect of TQ.
Employment is a result of many factors unconnected to TQ e.g. student ability; social capital; institutional reputation; whether programme is vocation with good chance of employment on graduation; the DLHE is a short-term measure are misleading. Reliance on graduate salaries may well be misleading (e.g. some jobs start with very low pay e.g. film making etc)
a further outcome, or product, measure of quality most commonly cited – the proportion of students obtaining a first or upper second has been undermined by grade inflation.
Retention of students may relate in part to the quality of student support, but is mainly down to the quality of the student and their socio-economic background (issues of affordability; family circumstances etc). The weight of each would be very difficult to measure.
So there is considerable concern about the metrics which will be used in Year Two of the TEF (because they are considered to be the best available.)
As a survey NSSE annually assesses the extent to which students are participating in educational practices that are strongly associated with high levels of learning and personal development” (Kuh 2001)
“Several aspects of educational provision are known to predict both student performance and learning gains, independently of other variables such as resources, research performance and student entry standards…. The impact of these aspects of educational provision can be validly measured by examining student engagement and effort using the NSSE, and NSSE scores are sufficiently robust that they can act as proxies for the quality of educational provision.” (Gibbs 2012)