2. Aims of the report
• To influence senior managers and academic staff
on raising quality in undergraduate education
• To provide an evidential base for what constitutes
effective practice
• ‘To contribute to current debates about
educational quality’
• Author: Graham Gibbs, formerly Director of the
Oxford Learning Institute. Produced by and on
behalf of the HEA
3. The ‘3P model’ (Biggs 1993)
• Presage – the context before students start
learning (e.g. funding, selection, reputation)
• Process – what goes on when students learn (e.g.
class size, face to face contact hours, staff student
ratios, quality of student engagement and
feedback etc..)
• Product – the outcomes of the learning – e.g.
Degree classification, employability
(Similar to the input-environment–output model
favoured in the USA)
4. The wider context
• Recent reports (House of commons select committee
2009, QAA 2009 and HEPI 2006,07 ff) have expressed
concern that UK students work less hard and for less
hours than European counterparts
• Does it matter if there are less student contact hours?
• Are total student study hours an indicator of quality?
• Why are the University of Oxford and the Open
University nearly always at the top of NSS rankings
despite their fundamentally different models?
5. Quality – what is it?
• A relative, not an absolute term
• Relative to institutional purposes, customer,
satisfaction(NSS)?
• What gains do students make through their study
(transformation)? Harvey and Green (1993) i.e. is
their education effective? How is the student
enhanced?
• Product can be the measure of student
performance before and after their experience of
HE
6. Some limitations of the model
Class size – this may affect student performance
(process) but is a result (perhaps) of funding
levels (presage) But who decides how to
allocate the funds – HoDs? Programme
Directors?
Any others?
7. Presage dimensions of quality:
Funding
Cohort and class size are predictors of student performance
Teacher quality is also a result of funding decisions (buy better
teachers!)
The best students go to the best resourced institutions (Oxbridge)
However a series of large scale US studies found no easy correlation
between institutional levels of funding and measures of educational
gain, i.e. universities with similar level of funding can have very
different outcomes, and vice versa, institutions with very similar
performance in terms of graduation rates and student satisfaction
can have very different levels of funding – why?
The more effective institutions used funding ‘to produce a campus
ethos devoted to student success’ (Gansemer-Topf et al 2004)
8. Student: Staff ratios (SSRs)
Low SSRs mean potentially more contact between teachers
and students – this is a key predictor of educational gains
(Pascarella and Terenzini 2005), though low SSRs do not
always result in increased contact
The volume, quality and timeliness of student feedback are
also good predictors of educational gains –this can be
related to SSR but low SSR does not by itself guarantee
good or timely feedback
Small class size is also a good predictor of student
performance but low SSR doesn’t necessarily guarantee
this
Often a low SSR is only achieved in the third year (when NSS is
administered)
9. Quality of teaching staff
• What is the balance between teaching delivered
by tenured staff and research students?
• In most Russell group and pre-1992 universities
most small group teaching is carried out by
teachers other than academics (HEPI)
• How far are these ‘adjunct’ staff (VLs, p*art
timers, research students, involved in module or
curriculum development, meet students out of
class, attend departmental meetings – US
evidence indicates that use of such staff
negatively impacts student performance
10. Quality of Students
• Selection/ tariffs – how important are these as predictors of student performance?
• Evidence from both the UK and the USA suggest that A level point scores tell us
almost nothing about the quality of the educational process at university, or the
degree of student engagement with their studies.
• However it is also clear that there are benefits to students ‘being surrounded by
other able students’ – raises student expectations of themselves. In group work it
ids the previous educational attainment of the best student in the group that best
predicts the group grade, not the average level of prior attainment or the level of
the weakest student
• How far does the educational process engage the student in collaborative
learning? This is important as it is a key predictor of educational gains
“Students bring more to higher education than their A -level scores. It is likely that
their cultural capital, their aspirations, self-confidence and motivations all influence
their performance and interact with teaching and course design variables”
11. Process Dimensions 1: Class size
Large class sizes have a negative impact on student performance and engagement –
leads to ‘surface’ learning – also leads to a clear and negative impact on NSS –
same teachers het higher scores when teaching smaller classes ( so it’s not the
teachers!
Large classes also negatively impacts on library and other resources, promptness and
quality of feedback and nature of assessments. Also close contact with staff may
be more limited.
Large classes also associated with weak social cohesion, alienation, poor in class
behaviour, hiding library books etc..
But
Where out of class study is a major component of a course enrolment may be amore
crucial variable than class size
In the OU a course may have an enrolment of 10,000, but will have an average class
size of 24 – one factor in their high NSS score
Larger classes also limit the amount of time students can access specialist resources
e.g. Labs, studios
However there is a puzzle – in the UK overall student performance has increased in
recent years at the same time as class size has increased – why is this???
12. Process dimension 2 – class contact hours, independent
study and total hours
• The number of class contact hours per se has little to do
with educational quality – it’s what happens in those hours
• A 1997 review (Gardiner) found an average of 0.7 out of
class study for every hour on in class contact. At Oxford the
average is 11 hours of independent study for every contact
hour – Oxford students work harder than students at other
UK universities despite less contact hours – it’s the nature
of the class contact which counts – it tends to be up close
and personal!
• The OU has less contact hours but remains amongst the
highest in NSS – has this anything to do with the 7
principles?
13. Process dimension 2 - continued
This doesn't mean class contact hours can be cut and quality ensured – it is essential to change the
pedagogic model- if students read to prepare for a seminar and then the seminar is removed they
will read less and learn less
What matters most is the hours students put in, whether in or out of class, although hard evidence for
how much independent study students actually do is hard to come by
Are the students who study longer hours the ones that perform best? There is no straightforward
answer here (Stinebreckner and Stinebreckner 2008) because able students may do well on less
hours than less able students who study a lot, but without a clear focus – these latter are likely to
become disenchanted and take an increasingly surface approach
If the question is rephrased – If average study hours on a degree programme were higher would average
performance be better the answer is more clearly YES! This is confirmed in both US and EU studies
– under the Bologna process it is estimated that total student effort (in and out of class is between
4,500 and 5,200 hours for a 3 year undergraduate programme at Bachelor level
Multiple studies have shown that UK students work less hours than their EU counterparts – degree
programmmes in the UK have one third of the hours of EU universities EU students find UK degrees
‘less demanding’
Number of student effort hours (in and out of class) within and between programmes and institutions
can vary very widely - nor is it the case that weaker students in weaker institutions study for more
hours – often it is less!
14. Why are student study hours declining?
• 1. Where programmes rely heavily on coursework assessment they only focus
resources on assessed areas (there is more study where more marks are awarded
from a final exam)
• Where learning outcomes and assessment criteria are full and clear students are
likely to take this as indicating what they can safely ignore and focus solely on
assessment components – research on study diaries show students work
progressively less hours as they get through their 3 year degree – they become
more strategic and focus solely on what will be assessed
• Increase in part time work reduces course effort and grades – in the USA students
take longer to complete their courses and may study at several institutions
• Students who live at home and travel to local universities (often in urban
conurbations) have the lowest average study hours
• Universities with low study hours are also those with fewest resources, e.g.
library/space per student and there is a direct correlation between resource
allocation per student and average response to NSS question on the quality of
learning resources
• Many of these findings also relate to Masters programmes
15. Teaching Quality
• Teachers with a teaching qualification have
been rated more highly than those without
Little of no relationship between measures of
the quantity or quality of teacher’s research
and measures of the quality of their teaching (
Hattie and Marsh 1996) ‘the common belief
that teaching and research are inextricably
intertwined is an enduring myth. At best
teaching and research are loosely coupled.’
16. Teaching quality – judged by students
Student rating of teachers is often disparaged by academics (rate your
professor!)
But can be very reliable indicator because:
Students agree with each other on who the best teachers are, agree with
teachers peers and make similar judgements on different occasions. They
also can and do distinguish between teachers they like and teachers who
they think are good. So, student feedback is not just a popularity parade
However students may have different conceptions of what constitutes ‘good’
teaching and their conception may change over time
Thus an unsophisticated student might consider good someone who delivers
all the content in lectures and then tests for memory of that content
(surface) whereas a more sophisticated student might prefer someone
who promotes independent learning and the development of a personal
stance towards knowledge
How can this inform our student voice awards criteria?
17. Research environment
An active research environment can be one of the presage
factors but the best research departments aren’t
necessarily the best teaching environments – a college
whose faculty is research-orientated increases student
dissatisfaction (Astin 1993) BUT
Where undergraduate students are engaged with a real
research project this can really benefit student learning e.g.
At MIT and Oxford, where there is a deliberate policy to
engage undergraduate learning with research active staff
and this can lead to a deep approach to learning
However it is evident that RAE (and now REF) scores have no
correlation with improved educational quality
18. Level of intellectual challenge
There are three elements to this:
1. Level of the curriculum – usually determined by
the department unless there are external
professional requirements
2. Depth of approach to studying
Although many students take a surface approach to
learning it is widely accepted that a deep
approach is essential to long term and
meaningful outcomes from higher education
3. Student engagement
19. Surface/Deep learning
Students are not surface or deep learners – this
is a context dependent response to perceived
demands of the learning context
What factors can promote deep learning?
Good feedback, clear sense of learning
outcomes and goals of course, clear
understanding of expected standard
However none of these criteria appear on the
NSS
20. Student engagement
This is currently a key focus of interest in US studies
(National Survey of student engagement) NSSE –
large scale studies over three decades so results
important – a recent example is a study of 774
universities carried out in 2008
Findings indicate that the level of academic
challenge, of active and collaborative learning
and extent and quality of student/faculty
interaction are prominent among the important
factors. These and other factors are encapsulated
in the 7 principles
21. Chickering and Gamson’s Seven Principles of Good
practice in undergraduate education
Good practice encourages Student faculty contact
Good practice encourages cooperation among
students
Good practice encourages active learning
Good practice encourages prompt feedback
Good practice encourages time on task
Good practice communicates high expectations
Good practice respects diverse talents and ways of
learning
22. Formative assessment and feedback
Enhanced feedback increases student retention (Yorke 2001)
Greater use of Formative assessment increases the deep
approach to learning ( Gibbs and Dunbar-Goddet 2007)
Enormous variety in the amount of formative activity and
assessment both within and between universities from
twice in 3 years in one institution to 130 times in another
Volume of written feedback varies from 3,00 words over 3
years to 15, 00 words, for oral feedback from 12 minutes
per year per student to over 10 hours per year. These are
greater variables than SSRs, class contact hours,
independent study hours or funding per student. In the era
of massification it is formative activity and feedback which
has taken the biggest hit
23. Other dimensions of quality
Reputation – THES rankings – only marginally useful in terms of indicating educational
gains
Peer ratings and TQA (Teaching Quality Assessment ) is marginally better but still
relies on reputational factors and pays little attention to process
Student Support – difficult to measure as provision very varied between, and
sometimes within institutions e.g. Level of personal tutor support, effectiveness
can also depend on extent of demand and this in turn on presage factors (e.g.
tariff, % of students whose first language is not English)
QAA relies on external examiner system and student evaluation of teaching – has
some useful impact but not of itself sufficient to enhance quality
Collecting student feedback on teaching does not by itself lead to improvement in
teaching unless accompanied by other processes such as the teacher consulting
with an educational expert, especially when preceded by the expert observing the
teaching and interaction with students (Weimer and Lenze 1997 and Piccinin 1999)
Institutional learning and teaching strategies have not, thus far, been able to
demonstrate any perceptible impact on educational gains.
24. Product Dimensions of Quality
% of Students getting firsts and upper second class
degrees has increased markedly over time, at the same
time as SSRs, funding per student, amount of feedback,
class size close contact with teachers has declined.
How to explain this counter intuitive result? A major
factor may be decline in the robustness of the external
examiner system We now inhabit a bizarre world
where Maths graduates are three times more likely to
get a first than History graduates. It is argued that
comparing degree standards is no longer meaningful
(Brown 2010) and degree classifications are not a
sound basis for indicating the quality of educational
outcomes of a UK university
25. Student retention and persistence
Oxford has a 90% retention rate the OU about 50%, yet both score high in NSS
– age seems to be a key indicator here – the broader the age and ability
range of students the lower the retention rate
In the US students increasingly plan to drop out of and move between
institutions as they complete their degrees. Other factors affecting UK
retention rates include living on campus, working part time, but no clear
pattern so far in the research
In the US, and to some extent in the OU, research into student readiness for
learning before they start their programmes identifies those who are likely
to need the greater support so that scarce resources can be better
targeted.
UK studies confirm that collaborative and interactive learning together with
close contact with teachers increases retention rates, especially among
less able students.
However it is very difficult to predict levels of student motivation , either
extrinsic or intrinsic.
26. Employability and graduate
destinations
The mp=lyability of graduates is not a real
measure of educational grains or quality as
there are too many variables such as location,
degree type, qualifications and age on entry,
institutional reputation, age and social class
of student body, etc..
27. Conclusions...
• Those institutions that do best in NSS are those which
have evolved institutional pedagogies, however
radically different they may be – take for example the
OU commitment to openness and Oxford’s to academic
excellence. The focus in the UK has been on individual
teachers – the National Teaching Fellow scheme rather
than teaching teams or departments – other countries
have taken a different approach
• Do different subjects require different pedagogies? And
therefore different measures of quality? Both of these
would seem to be true, even though there may be
broad institutional guidelines and general principles as
revealed in this report.
28. Additional quality factors
• Is teaching valued and rewarded?
• Do teachers talk to each other about teaching
and its improvement?
• Is innovation supported and funded?
• Is educational effectiveness evaluated?
• How good is departmental leadership?
• Is there a departmental community of
practice?
