The document outlines a feedback survey for postgraduate taught (PGT) students, detailing aspects such as survey coverage, timing, data linking, and publication. It emphasizes the importance of a multi-point survey approach to capture representative data and discusses methods for publication and analysis of the survey results. Key considerations include ensuring meaningful breakdowns of responses and establishing benchmarks for comparison with existing performance indicators.

1. A feedback survey for taught
postgraduates
Workshop 2: Timing and publication
Catherine Cameron
Senior HE Policy Adviser, HEFCE
Richard Puttock
Head of Data and Management Information, HEFCE
Future Inn, Bristol City Centre
21 September 2017

2. Session Outline
1. Survey coverage (10 mins)
2. Survey timing (20 mins)
3. Data linking (10 mins)
4. Publication levels (20 mins)

3. Survey coverage
Initial starting point was whether there were
any groups of PGT students who should be
excluded
Our proposal
All students on PGT courses which lead to a
qualification
Excepting those on integrated Masters
Questions or comments?

4. Survey timing
Key consideration is to provide representative
and meaningful information
Options
• Single survey window
• Post completion survey
• Multi-point survey
• Rolling survey

5. Survey timing
End date analysis
Whilst June/July and
September/October cover most
students, there are differences for:
• Particular providers
• Some students on unstructured
courses
• Some students studying particular
subjects
End month Percentage
of students
Jan 3%
Feb 2%
Mar 3%
Apr 2%
May 4%
Jun 12%
Jul 16%
Aug 5%
Sep 34%
Oct 10%
Nov 6%
Dec 3%

6. Survey timing
Conclusion
• A multi-point or rolling survey is the only way
to achieve responses which reflect the
majority of a students’ academic experience
and a sufficiently high response rate.
• Running the survey at four points in the year
should be sufficient to capture a significant
majority of the PGT population close to the
end of their period of study.

7. Survey timing - questions
Your questions ….
Our questions
a. What are the challenges for institutions in
delivering a multi-point survey?
b. Is there anything we haven’t considered
that we should have?

8. Data linking
• In order to be able to analyse and represent
survey responses by student characteristics
we must be able to link back to individualised
student data
• This will also allow us to link to other
datasets: NSS, Graduate Outcomes for
analysis
• Data futures – will introduce in-year data
collection and potentially allow us to generate
target lists
Questions or comments?

9. Publication
• Intend to consult on some elements of
publication approach now and some following
successful piloting
• The key things that we think that we will need
to consider are:
• Publication routes
• Publication thresholds
• Publication level
• Breakdown of responses
• Benchmarking of responses
• Detailed approach to presentation of
information

10. Publication
Publication routes
Envisage publishing outcomes in similar way to
NSS:
• Publishing responses on the Office for Students (OfS) website
• Publishing on the central information source to support
decision-making being provided by the Office for Students
and the other funding bodies (assuming continuation)
• Publishing an open dataset so that other information
providers can make use of this information
• Making responses available to providers through a secure
system.

11. Publication
Publication thresholds
Propose to adopt same thresholds as used for publication
of NSS
• At least 10 students
• 50 per cent of sample
Question
Our analysis is based on these thresholds and we
propose not to consult on them. Are there any concerns
about this?

12. Publication
Potential coverage
However
Expected gaps for some particular types of provision, particularly at
FECs, due to very small cohort sizes
Publication level Estimated coverage
(70% response rate)
Estimated number of
students included
Course 19% 67%
HECoS CAH level 3 57% 94%
HECoS CAH level 2 70% 98%

13. Publication
Breakdown of responses
We also need to consider the course characteristics by
which the survey responses should be broken down to
best meet its purposes
For example, mode of study.
But …. we must be careful not to fragment to an extent
where we cannot publish data
Questions
• Are there ways in which is would be essential to break
down the data to make it meaningful?
• Are there ways in which it would be desirable to break
it down?

14. Publication
Benchmarking
• We will consider benchmarking as part of the second
stage of developing proposals.
• HEFCE is currently leading a fundamental review of
the approach to benchmarking used in the UK HE
Performance Indicators and the TEF to ensure it
meets best practice.
• This will influence our approach to benchmarking of
survey outcomes.

15. Other questions or
comments?

16. How to find out more
website www.hefce.ac.uk/lt/PGT/