SlideShare a Scribd company logo
1 of 16
Download to read offline
Effective Course Evaluation
The Future for Quality and Standards
in Higher Education




A summary report from interviews among
senior academics and student representatives
CONTENTS




Acknowledgements                        4

Introduction by Eric Bohms,
Managing Director, Electric Paper Ltd   5

Executive Summary                       6

Improving Response Rates                7

Improving Student Feedback              8

Improving Turnaround Time               9

Improving Survey Administration         10

Improving The Student Experience        11

Conclusion                              12

Evaluation Potential                    13-14
ACKNOWLEDGEMENTS




             The editors of this report, Phil Smith and Owen Morris at Communications Management,
             would like to thank the following people who were interviewed in July and August 2011:
             • Professor Susannah Quinsee, Director of Learning Development, City University
               London.
             • Professor Ian Marshall, Deputy Vice-Chancellor (Academic), Coventry University.
             • Professor David Coates, Dean of School of Life Sciences’ Learning and Teaching
               Division, University of Dundee.
             • Professor Glenn Burgess, Pro-Vice-Chancellor of Learning and Teaching, University
               of Hull.
             • Professor Andy Pitsillides, Chair of Teaching Quality Committee, Royal Veterinary
               College, University of London.
             • Professor Huw Morris, Pro-Vice-Chancellor (Academic), University of Salford.
             • Professor Alan Speight, Pro-Vice-Chancellor (Student Experience and Academic Quality
               Enhancement), University of Swansea, and member of Quality Assurance Agency Board
               of Directors.
             • Alex Bols, Head of Education and Quality, National Union of Students.
             • Alex Smith, former Academic Affairs Officer, University of Leicester Students’ Union,
               and member of Quality Assurance Agency Board of Directors.
             • Alex Nutt, current Academic Affairs Officer, University of Leicester Students’ Union.




4   Effective Course Evaluation
INTRODUCTION




The 2011-12 academic year is a big one for the Higher Education sector. This is the year
when, more than at any previous point at time, UK universities will have to provide clear
evidence of the ‘value’ of studying at their institution.
Universities are digesting the implications of the Higher Education White Paper, published in
June 2011, which has asked institutions to be more accountable to students on teaching quality
and to provide greater transparency in areas such as the student experience. Specifically the
paper says that “universities will be expected to publish online summary reports of student
surveys of lecture courses, aiding choice and stimulating competition between the best
academics”. Therefore, gaining effective student feedback on courses and lecturers to enable
transparent reporting is a must.
At Electric Paper we work with over 600 universities in the UK and worldwide to help them to
evaluate their courses through effective student feedback via our automated paper and online
survey management system EvaSys. We hope that this report will draw attention to the latest
issues and trends around effective course evaluation which, in turn, will help us to develop our
support for the Higher Education sector. But why is it so important that we get this right?
Firstly, and most importantly, with the increase in tuition fees from 2012, universities can no
longer afford to leave teaching and learning quality in the hands of academics alone. The
University’s executive needs to have visibility around the quality of the course and lecturer, and
establishing the best way to tackle the ‘process’ of getting student feedback is crucial.
Secondly, the National Student Survey (NSS) is currently the primary method of gathering
feedback on the quality of students’ courses in order to contribute to public accountability and
help inform the choices of future applicants. Gaining comprehensive, instant, feedback on
courses and lecturers can assist universities in improving overall quality, and NSS scores.
Thirdly, as many universities have taken the decision to charge the full £9,000, it will become
even more important for students to distinguish between institutions on issues of quality.
Quickly capturing feedback across all courses, providing students and their parents with
evidence-based feedback about the quality of teaching and learning, will be a big advantage.
Fourth, individual universities need to stand out from the crowd. With the introduction of ‘Key
Information Sets’ universities need to do everything they can to provide indicators of quality.
Capturing, and responding positively to, student feedback on course evaluation can only help
boost recruitment targets and reputation.
Finally, implementing an effective process for capturing student feedback around courses and
lecturers is a money-saver for universities. Institutions spend a huge amount of time and
money on survey administration, and approaches are inconsistent (e.g. with management of
surveys sitting centrally in one university, and departmentally in another). An effective process
for course evaluation can offer significant cost and efficiency savings, addressing the hidden
costs in survey administration and freeing up staff expertise for other areas.




Eric Bohms, Managing Director, Electric Paper Ltd

                                                            Effective Course Evaluation             5
EXECUTIVE SUMMARY




             The purpose of this report is to examine       • Universities need to work harder at
             the latest issues and trends in course           feeding back to students the actions they
             evaluation. In-depth interviews have been        will be taking as a result of input provided
             conducted with 10 senior academics and           for course and lecturer evaluation surveys.
             student representatives to draw out a
                                                            • End-of-module evaluation is a particular
             general sense of how universities are
                                                              stumbling block in the provision of
             currently approaching the ‘process’ of
                                                              feedback to students - and feedback can
             student surveys, and what improvements
                                                              be slow - but moving to mid-module
             need to be made to gain more effective
                                                              evaluation can help to improve the
             student feedback on courses and lecturers.
                                                              process.
                This qualitative research is particularly
                                                            • Ideally students want the opportunity to
             relevant in the context of the 2011 Higher
                                                              express their views on course
             Education White Paper, which has asked
                                                              improvements at a time that their
             universities to be more accountable to
                                                              feedback benefits them directly.
             students on teaching quality and to provide
             greater transparency in areas such as the      • Universities need to embrace new
             student experience. Rising tuition fees in       technologies to improve turnaround time
             2012 mean that gaining (and responding           - but effective feedback can be gained via
             effectively to) student feedback on courses      a combination of paper and online
             and lecturers is increasingly important for      surveys.
             universities as they have to provide clear     • Universities should establish a more
             evidence of the ‘value’ of studying at their     consistent (centralised) approach to survey
             institution.                                     administration - including a standard set
             This report finds that:                          of survey questions - to enable effective
                                                              benchmarking at course and institutional
             • Many universities seeking feedback on
                                                              level. However, individual departments
               courses and lecturers via surveys struggle
                                                              should have the flexibility to include
               to achieve a meaningful response from
                                                              bespoke questions for particular courses.
               students, which is partly due to students
               being inundated with requests to             • In-class student involvement in survey
               complete surveys.                              administration can increase commitment
                                                              as they are stakeholders in the process.
             • Student representatives have indicated
               that students are not effectively engaged    • Effective course evaluation is necessary
               in the feedback process and, for some,         for universities to provide a clear evidence
               providing feedback can even be                 base to demonstrate their ‘value’ to
               intimidating.                                  students.




6   Effective Course Evaluation
THE NEED TO...
IMPROVE RESPONSE RATES



Many universities seeking feedback on           Quality Assurance Agency Board of
courses and lecturers via surveys are still     Directors, agreed that there is a need for
struggling to achieve a meaningful              universities to ensure that students are
response from students - that is, a             “properly engaged” and even “incentivised”
response rate of over 50% that will ensure      to take part in the process. However,
statistical validity in order to evaluate       “providing feedback can be intimidating for
teaching quality and make improvements.         students,” he said. Alex Nutt, who
   Professor Glenn Burgess, Pro-Vice-           succeeded Smith at Leicester’s Students’
Chancellor of Learning and Teaching at the      Union added that “despite the process
University of Hull, said the problem was        being anonymous some students feel
that students were simply inundated with        intimidated by the idea of giving negative
requests to complete surveys. “A key issue      feedback in case it reflects on them”.
facing universities in relation to gaining
effective course feedback from students is      “We introduced mid-module surveys
around achieving significant response rates     and went back to paper and the
to any questionnaire-based approach.            response was super”
Students are increasingly overloaded with
surveys - from the National Student Survey        Coventry University has managed to
downwards - so you need a good reason to        increase its response rates by using paper
encourage participation. It’s also important    and moving to mid-module surveys.
to appreciate the qualitative/quantitative      “Historically universities conduct end-of-
divide in surveys. Often the most useful        course, or end-of-module surveys, but by
information is gained through qualitative       the time the feedback has been analysed
feedback, but this is also the most difficult   and results published the students have
to achieve as it requires more time from        gone away,” said Professor Ian Marshall,
respondents.”                                   the University’s Deputy Vice-Chancellor
   This is a view shared by Professor           (Academic). “We moved to online surveys,
Susannah Quinsee, Director of Learning          but the response was dreadful, so last year
Development at City University London,          we introduced mid-module surveys and
who said that “a lot of course evaluation” is   went back to paper. The response was
geared towards the National Student             super, and we are now able to turn around
Survey. “Ideally you want to engage in          feedback in two weeks maximum.” He
dialogue with students, because this is         added that if a module is perceived to be
more useful than knowing that half the          underperforming “we will focus on those”.
class love a module and half the class hate       At Coventry University mid-module
it. You want to know more about what the        surveys are handed out in class by 120
middle group think. You also need more          senior student representatives who are
open questions rather than tick box, but        interviewed and selected by the University’s
students are not so keen on these               Students’ Union. The University of Hull is
questions. So there is an issue over student    also working closely with its Students’
engagement, and an associated issue             Union to develop its approach to
around the extent to which students engage      improving response rates. “What we have
in the language used in evaluation surveys.”    found is that students are tolerant of
   Alex Smith, former Academic Affairs          problems, but they want us to listen, feed
Officer at the University of Leicester’s        back quickly, and respond,” said Professor
Students’ Union, and a member of the            Burgess.


                                                                             Effective Course Evaluation   7
THE NEED TO...
             IMPROVE STUDENT FEEDBACK



             Through this research a clear gap has           not going to feel the benefits of any
             emerged around the extent to which UK           improvements made,” said City’s Professor
             universities currently feed back to             Susannah Quinsee. “Ideally you would
             students the action they will be taking as      want to explore in-module evaluation,
             a result of input provided in course and        which we are doing, but that takes a lot of
             lecturer evaluation surveys.                    time both in terms of implementation and
               “Students need to provide feedback, then      analysis.”
             the universities need to take that on board       Student representatives agree that end-
             and do what they can with it, and then          of-module evaluation is no longer
             provide clear feedback to the students on       sufficient. “It’s a flawed system,” said
             the actions and outcomes,” said ex-             Smith. “In my experience universities often
             University of Leicester Students’ Union         ask for student feedback at the end of a
             Academic Affairs Officer, Alex Smith. “The      course, but by then it’s too late, and the
             feedback loop must underpin everything.”        students do not see any results from the
               Alex Bols, Head of Education and Quality      feedback they’ve given. Universities often
             at the National Union of Students (NUS),        profess a need to get the feedback system
             said that “in the majority of cases”            right but I don’t think this has necessarily
             students who participate in course              happened yet as, by only asking for
             evaluation surveys, in his experience, are      feedback at the end of a course, they are
             then not told what happens as a result of       gathering entirely new opinions from a
             the process. “It’s important for universities   different cohort of students and therefore
             to close the loop and tell students what        cannot compare like-for-like.”
             has happened - or hasn’t happened - as a          Professor Alan Speight, Pro-Vice-
             result of the feedback provided and why.        Chancellor (Student Experience and
             This should not be an autopsy at the end        Academic Quality Enhancement) at the
             of a course, but a process embedded             University of Swansea, and a member of
             through the learning experience so that it      the Quality Assurance Agency Board of
             is of benefit to the student giving the         Directors, said that establishing an
             feedback and their experience.”                 effective system for student feedback was
                                                             vital for universities’ reputation. “There is a
             “It's important for universities to close       need to dig down to obtain feedback as it
             the loop and tell students what has or          does not take much to make students feel
             hasn't happened”                                disgruntled. Universities are only as strong
                                                             as their weakest link and poor delivery by a
               End-of-module evaluation appears to be        subset of staff that goes unchecked could
             a particular stumbling block in the             make a lot of difference to the overall
             provision of feedback to students. For          teaching reputation of an institution.”
             example, City University London is three          “Students are more interested in
             years into a JISC-funded four-year project      outcomes - so it’s important for
             on curriculum design, and staff have            universities to be very clear on what they
             looked at the issue of effective course         are able to do, and equally be honest on
             evaluation. “What we’re finding is that         what they are not able to,” added
             evaluation for a lot of students is not         Coventry’s Professor Ian Marshall.
             meaningful because they are being asked
             to give feedback on a course or module
             they are just completing, and are therefore

8   Effective Course Evaluation
THE NEED TO...
IMPROVE TURNAROUND TIME



Relating to the issue of student feedback is     universities need this for both instant
the challenge that universities face in          feedback and for internal benchmarking
turning around the findings of course            leading to organisational improvement.
evaluation surveys.                                Student groups agree. “By examining
  Professor Andy Pitsillides, Chair of the       the process and proving that they are
Teaching Quality Committee at the Royal          interested, and capable of acting upon, the
Veterinary College, University of London,        feedback they receive in a timely manner
admitted that the process can be “very           universities could build partnerships with
slow” in Higher Education. “As academics         their students to add value all-round,” said
we know that students want quick                 the NUS’ Alex Bols. “Students should be
feedback as it helps their learning, yet we      able to express how their course could be
are restricted by the annual cycle of            improved while it impacts on them.”
surveying. The exercise is also paper-             Exploiting innovative new technologies
driven, so it is more cumbersome than we         could support the requirement to improve
might like, and we are looking at a process      turnaround time, according to Professor
of electronic feedback to augment this           Huw Morris, Pro-Vice-Chancellor
approach.”                                       (Academic) at the University of Salford,
                                                 who in his previous role as Dean of
“As academics we know that students              Manchester Metropolitan University
want quick feedback as it helps their            Business School led the trial of course
learning”                                        surveys via mobile phones. “Going forward
                                                 I anticipate the Higher Education sector
  It’s a major issue facing the sector,          will need to utilise online devices to
according to Swansea’s Professor Alan            capture student feedback, but at the same
Speight. “Turnaround time is vital, but a big    time ensure this is not done in an intrusive
problem at the moment. In some areas             manner. Some element of compulsion for
surveys may be paper-based and manually          students in providing feedback will also be
processed, which means that someone has          helpful in ensuring that the results are
to physically input the results and is open to   representative of underlying views.”
human error. Also the feedback may come            In addition to module evaluation, there is
back when it is too late for the staff to do     a clear need for feedback on the
anything about it as they are, by that time,     underlying quality of the students’
already committed to a teaching pattern for      educational experience, Professor Morris
the next academic session, which in turn         said. “The National Student Survey and, to
makes the students wonder why their              a lesser extent, the International Student
thoughts have not been listened to.”             Barometer, are helpful and most
  A survey of 100 university administrators      universities respond to these. But while we
by Electric Paper, conducted in 2011, found      know about student happiness and
74% of respondents agree that both paper         satisfaction, we don’t know that much
and online surveys is a requirement in           about how their educational experiences
Higher Education. It’s not a question of         prepare them in the longer term. Here
either/or, the respondents said, because         there is a need to draw on evidence from
both methodologies can provide high              other tools which assess the contribution
response rates and efficiency savings for        of the students’ educational experience to
the right surveys in the right context.          their career prospects and personal and
However, turnaround time is key as               social development in the future.”


                                                                              Effective Course Evaluation   9
THE NEED TO...
             IMPROVESURVEYADMINISTRATION



             To improve response rates, student feedback          Other universities are making moves
             and turnaround time, universities need to          towards centralisation, such as the University
             have a more consistent approach to survey          of Hull. “We have a central code of practice
             administration. Through this research it has       for survey administration which provides the
             become clear that the management of                foundation for all modular and programme
             surveys sits centrally in one university, and      surveys,” said Hull’s Professor Glenn
             departmentally in another. In addition, a 2011     Burgess. “But within the University there is
             Electric Paper survey found that academics         currently no consistency on the level of
             are managing their own survey                      approach and questions being asked locally,
             administration at 1 in 3 institutions - and that   and we have plans to change that. This
             most universities are facing ‘absolute chaos’      brings a pressure on resource for central
             around survey management.                          processes and for us to introduce a particular
               What is clear from interviews with senior        way of working. It also links to wider policy
             academics is that centralisation of course         issues - for example, how is feedback linked
             evaluation feedback analysis is required.          to staff appraisal, what happens in terms of
             Following the arrival of Vice-Chancellor,          responding to poor feedback, and how
             Professor Paul Curran, City University             public do you make this information.”
             London has introduced a centralised
             modular evaluation system. “We now have a          “A common set of questions should be
             standard set of questions for surveys,             owned institutionally, with subject areas
             managed centrally, which individual schools        able to select from optional additional
             can add to if they wish, and the results of        questions”
             these are now part of staff appraisals,” said
             City’s Professor Susannah Quinsee. The               The University of Swansea’s Professor Alan
             scheme being rolled out by Coventry                Speight said that there were historic
             University is managed by a central unit            challenges to overcome for many institutions
             based in student services, while the               in order for the feedback process to become
             University of Salford has established              centralised. “The difficulty has been that
             standard questionnaires for module, level          surveys carried out by different departments
             and programme-centred evaluation which             have evolved organically and been developed
             are co-ordinated by a central planning and         in silos that are tailored just to the needs of
             performance department.                            one faculty. This results in a lack of core
               Professor David Coates, Dean of the School       information across the institution, mixed
             of Life Sciences’ Learning and Teaching            responses and no consistent use of data.
             Division at the University of Dundee, said         There should be a unified approach that
             “the process of receiving and then                 includes core questions and specifies the way
             disseminating feedback needs to be clean to        the feedback is processed - which allows
             ensure that those doing the teaching will          benchmarking and consistency. A common
             receive the knowledge they need”. He added:        set of core questions should be owned
             “Firstly, there needs to be consistency in the     institutionally, with subject areas able to
             feedback the institution receives as a whole,      select from a bank of optional additional
             but equally courses should not always be           questions.”
             compared like-for-like. A hub-and-spoke              As with the Coventry University model,
             model works best where individual                  Professor Speight suggested that students
             departments are evaluated but then this            could be brought on board to help co-
             feeds into a holistic view of the institution.”    ordinate the process.

10   Effective Course Evaluation
THE NEED TO...
IMPROVE THE STUDENT EXPERIENCE



The volatile Higher Education landscape -       and lecturer feedback method so there
the increase in student tuition fees and        can be constant improvements that will
the focus on student-led decision-making        determine a student’s opinion of their
- means that universities are under             institution based on direct experience.”
pressure to be more accountable and               A wider issue beyond module evaluation
transparent on issues of quality.               is the need for UK universities to move
  Alex Nutt, Academic Affairs Officer at the    towards surveying “all student groups”,
University of Leicester Students’ Union,        according to Salford’s Professor Huw
said that prospective students and their        Morris. The National Student Survey and
parents “will want to know as much as           International Student Barometer do not,
possible” in order to make their university     he said, cover UK and other EU second
decision, citing examples from the quality      and third year students at Levels 4 and 5,
of lectures to resources in the library. “I     taught postgraduate UK and other EU
also think they will want to know that          students at level 7, or UK and other EU
institutions take the concerns of students      postgraduate research students at levels 7
seriously and that education is seen as a       and above.
collaborative partnership between the             The UK may also wish to take note of
university and the students, not just a         developments across the Atlantic,
business transaction.”                          Professor Morris said. “In the USA there
                                                are other survey methods which are more
“It’s brilliant news for students who to        comprehensive and which focus more
all intents and purposes are consumers          clearly on contribution. For example, Pew
and have the right to know this                 studies and the National Survey of
information”                                    Student Engagement provide more
                                                comprehensive and possibly more candid
  “There is a going to be a race to             and easily accessible measures of student
improve in Higher Education and the             experiences in particular institutions and
results will be there for all to see, whereas   particular colleges, schools and
previously things that were not done as         departments. With the changes in tuition
well may have been brushed under the            fees, I think we will see more information
carpet,” explained Nutt’s predecessor,          of this kind being expected by students
Alex Smith. “It’s brilliant news for            and their parents because they will want
students who to all intents and purposes        to measure return on investment.”
are consumers and have the right to know          However, City’s Professor Susannah
this information.”                              Quinsee, said she had “difficulty” with the
  Universities are only too aware of the        overall notion of students as consumers.
need to provide a clear evidence base to        “That implies a customer-supplier
demonstrate their ‘value’. “Of paramount        relationship where education is about
importance in the future will be the            partnership and dialogue. Universities do
student experience as this will determine       need to get students more involved in
how to promote the institution to the           programme design, and evaluation and
outside world and potential students,”          feedback is all part of that, but students
said Dundee’s Professor David Coates.           also need to work with universities to
“With the National Student Survey, and          tell us what data they find useful, what
the introduction of ‘Key Information Sets’,     they expect, and above all what they
it is vital that there is an effective course   find meaningful.”


                                                                            Effective Course Evaluation   11
CONCLUSION




              Interviews with senior academics and             However, universities also recognise that
              student representatives for this report have   current evaluation processes need
              uncovered a strong feeling that universities   development - there is a clear need to
              need to improve their approach to student      support academics in improving the quality
              feedback on courses and lecturers via          of a course by providing comprehensive,
              module surveys.                                instant, student feedback - and research
                 The core perspective coming out of          indicates that a combination of both paper
              universities is that they have difficulty in   and online surveys is a requirement in
              achieving a “meaningful” response from         Higher Education survey administration.
              students to surveys, and turnaround time         What is clear is that all universities need to
              from response to feedback is not as quick as   embrace this agenda. Prospective students
              it could or should be.                         (and their parents) applying for degree
                 From the student perspective there is an    programmes from 2012-13 will be looking
              issue around the extent to which students      for detailed and transparent information to
              feel engaged in the feedback process, not      help them distinguish between Higher
              least because universities do not do enough    Education institutions on issues of quality
              to feed back the outcomes of their survey      and standards in relation to courses.
              input, and the extent to which they feel the     Individual universities need to stand out
              benefit of any actions taken as a result of    from the crowd - and providing clear
              their feedback.                                baseline reporting about the quality of
                 Some universities are moving from end-      teaching and learning is going to be critical
              of-module evaluation to mid-module             going forward.
              evaluation, which may help to address these      As one interviewee for this report summed
              issues. Universities also recognise the need   up: “There is a lot of activity in the sector
              for a more consistent, and centralised         around understanding the student better
              approach to survey administration which        and communicating with them - and this
              will potentially enable more effective         could be a crucial differentiation for
              institutional benchmarking.                    universities in the next five years.”




12   Effective Course Evaluation
POTENTIAL FOR
PROCESS IMPROVEMENT



In this report senior academics and student        higher response rates.” Between 2009 and
representatives have expressed an interest         2010 more than 45,000 questionnaires were
in improving course evaluation practices in        completed, surveying over 400 graduate
universities.                                      teaching assistants, 600 courses, and 700
  Electric Paper Ltd, a software solution          permanent teachers. Once the completed
provider to the education sector based in the      surveys were received Electric Paper scanned
UK, Germany and Switzerland, has developed         up to 3,000 a day, providing an instant,
EvaSys Survey Management Suite, a hybrid           individual, daily report for each
paper and online survey automation system          course/teacher combination. Whereas
which enables Higher Education institutions        previously some staff had to wait weeks,
to improve and efficiently manage their            turnaround times improved dramatically
module evaluations and other student               with results returned quickly and in an easy-
feedback surveys.                                  to-read format. “The swift turnaround of
   By managing the process centrally through       clear and accurate results was a key
the web-based system, institutions are able        requirement for the new system and EvaSys
to reduce administration time and costs            more than met expectations,” Mr Page said.
whilst distributing detailed analysis to           “EvaSys was selected after an extensive
stakeholders in a timely manner. Using both        tendering exercise because it offered the
paper and online surveys, EvaSys utilises in-      most flexible yet dedicated system for our
class evaluations to maximise response rates       needs - it’s a clear winner in our eyes.”
and therefore validating results.
  In July 2011 Electric Paper announced its        Queen Mary, University of London
latest product release, EvaSys Education V5.0.     As part of its five-year strategic plan Queen
  At the time this report was published in         Mary, University of London (QMUL)
August 2011 Electric Paper was working with        required a system which would allow central
over 600 universities worldwide, and               collection and analysis of student
examples of its UK university partnerships         evaluations for its 16,000 students to ensure
are below.                                         rigorous monitoring and assessment of
                                                   taught programmes. In 2010 QMUL
London School of Economics and                     contracted Electric Paper’s EvaSys product,
Political Science                                  which provides flexible and automated
The London School of Economics and                 course and module evaluations including
Political Science (LSE) required a system          paper and online surveys and detailed
which would allow it to quickly and efficiently    reporting. “We wanted to ensure that all
survey its 9,000 students and gain feedback        students are invited to give anonymous
on its courses and teaching. LSE turned to         feedback on every module and that such
Electric Paper’s EvaSys survey management          feedback is collated and used systematically
system, which allows the flexibility to evaluate   to assure and enhance the quality of
modules using both online and paper-based          QMUL’s taught provision,” said Emma
surveys automatically and without the need         Wynne, Assistant Academic Registrar at
for manual data entry. “There were two main        QMUL. In the initial pilot roll-out, nine out
goals when introducing EvaSys at the               of 17 schools participated, surveying over
School,” explained Mike Page, Head of ARD          380 modules. Results were completed
Systems and Business Processes at LSE. “We         within three weeks with an average response
wanted to improve the timeliness and               rate of 59%, with some modules as high as
accuracy of survey results, and to encourage       73%. As the EvaSys system fulfilled the


                                                                                Effective Course Evaluation   13
POTENTIAL FOR
             PROCESS IMPROVEMENT



              objectives of the scheme, senior staff           overhaul of the process was needed. As a
              responsible for taught programmes and            result, Electric Paper’s EvaSys was purchased
              QMUL’s Vice-Principal recommended the            by St Andrews in November 2010 and used
              adoption of EvaSys throughout the                to produce that semester’s module
              University commencing in October 2011.           evaluations. In addition, a core set of
              “The feedback from the schools was that the      questions was developed to include
              reports generated by EvaSys are rapid,           compulsory questions relating to modules,
              informative and clearly presented,” Ms           individual lecturers and individual tutors,
              Wynne said. “From a programme                    supported by a maximum of eight optional
              management and quality enhancement               questions from a pool of 16. Once
              perspective, the presentation of results in      completed, the questionnaires were scanned
              graphical format - including School and          and instant analysis reports produced.
              College averages - was particularly useful in    Reports were automatically emailed to
              identifying areas of concern and examples        module coordinators and to lecturers to
              of excellent practice.”                          receive individual feedback tied to the
                                                               internal review of teaching and learning.
              University of St Andrews                         This alerted senior management to any
              The University of St Andrews in Scotland has     areas of concern, and to identify good
              established a reputation as one of Europe’s      practice that deserved commendation and
              leading and most distinctive centres for         dissemination. “The introduction of EvaSys
              teaching and research. Previously, while run     has simplified the module feedback process
              centrally, academics requested their own         immensely,” said Carol Morris, Director of
              individualised questionnaire resulting in        the Centre for Academic, Professional and
              over 1,000 unique versions with ensuing          Organisational Development at St Andrews.
              difficulties in longitudinal comparison of the   “Results are now available in days rather
              quality of delivery and content of the           than months allowing more time to monitor
              module. The University established that a        quality and implement enhancements to
              review of the feedback system and an             teaching and learning.”




14   Effective Course Evaluation
Need help
  in implementing
 centralised module
     evaluation?




Electric Paper works with over 600 universities to
   provide effective student feedback systems
          www.electricpaper.co.uk
Electric Paper Ltd
International House,
1-6 Yarmouth Place,
Mayfair, London, W1J 7BU
Telephone: 0203 145 3258
Email: info@electricpaper.co.uk
www.electricpaper.co.uk

More Related Content

What's hot

Academic audit presentation_-_introduction
Academic audit presentation_-_introductionAcademic audit presentation_-_introduction
Academic audit presentation_-_introductionDurgesh Ahirwar
 
ASSESSMENT AND ACCREDITATION:REVISED PROCESS -AFFILIATED/CONSTITUENT COLLEGES
ASSESSMENT AND ACCREDITATION:REVISED PROCESS -AFFILIATED/CONSTITUENT COLLEGESASSESSMENT AND ACCREDITATION:REVISED PROCESS -AFFILIATED/CONSTITUENT COLLEGES
ASSESSMENT AND ACCREDITATION:REVISED PROCESS -AFFILIATED/CONSTITUENT COLLEGES Abhay Khandagle
 
The use of technology in relation to the teaching excellence framework criteria
The use of technology in relation to the teaching excellence framework criteriaThe use of technology in relation to the teaching excellence framework criteria
The use of technology in relation to the teaching excellence framework criteriaSue Beckingham
 
Understanding GEM 2014 Presentation FINAL12
Understanding GEM 2014 Presentation FINAL12Understanding GEM 2014 Presentation FINAL12
Understanding GEM 2014 Presentation FINAL12Joshua LaFave
 
Presentation for faculty awareness about Quality Management System and Accred...
Presentation for faculty awareness about Quality Management System and Accred...Presentation for faculty awareness about Quality Management System and Accred...
Presentation for faculty awareness about Quality Management System and Accred...Ahmedani Shoaib
 
Enhancing the Undergraduate Experience Through A Collaborative Wiki Exercise ...
Enhancing the Undergraduate Experience Through A Collaborative Wiki Exercise ...Enhancing the Undergraduate Experience Through A Collaborative Wiki Exercise ...
Enhancing the Undergraduate Experience Through A Collaborative Wiki Exercise ...Iain Doherty
 
Presentation on External Academic Audit Report (2017 18) - R.D.Sivakumar
Presentation on External Academic Audit Report (2017 18) - R.D.SivakumarPresentation on External Academic Audit Report (2017 18) - R.D.Sivakumar
Presentation on External Academic Audit Report (2017 18) - R.D.SivakumarSivakumar R D .
 
Stephanie McKendry 'The conflicting priorities of blended and inclusive learn...
Stephanie McKendry 'The conflicting priorities of blended and inclusive learn...Stephanie McKendry 'The conflicting priorities of blended and inclusive learn...
Stephanie McKendry 'The conflicting priorities of blended and inclusive learn...johnroseadams1
 
Introduction of Medical Education Domain
Introduction of Medical Education Domain Introduction of Medical Education Domain
Introduction of Medical Education Domain Md Yunus
 
From Quality Assurance to Quality Enhancement (2006)
From Quality Assurance to Quality Enhancement (2006)From Quality Assurance to Quality Enhancement (2006)
From Quality Assurance to Quality Enhancement (2006)JamesDunphy
 
SAILing Takes You to Where Your Course Ought to Be - Course Technology Comput...
SAILing Takes You to Where Your Course Ought to Be - Course Technology Comput...SAILing Takes You to Where Your Course Ought to Be - Course Technology Comput...
SAILing Takes You to Where Your Course Ought to Be - Course Technology Comput...Cengage Learning
 
Criteria 2 NAAC New guideline ppt
Criteria 2   NAAC New guideline  ppt Criteria 2   NAAC New guideline  ppt
Criteria 2 NAAC New guideline ppt Anita GoswamiGiri
 
A Brief expression on National Assessment and Accreditation Council (NAAC) Ma...
A Brief expression on National Assessment and Accreditation Council (NAAC) Ma...A Brief expression on National Assessment and Accreditation Council (NAAC) Ma...
A Brief expression on National Assessment and Accreditation Council (NAAC) Ma...Sudhakar Babu
 
OECD Review on Evaluation and Assessment Frameworks for Improving School Outc...
OECD Review on Evaluation and Assessment Frameworks for Improving School Outc...OECD Review on Evaluation and Assessment Frameworks for Improving School Outc...
OECD Review on Evaluation and Assessment Frameworks for Improving School Outc...EduSkills OECD
 
Accreditation, Quality, And Graduate Attributes: A Case of Higher Education i...
Accreditation, Quality, And Graduate Attributes: A Case of Higher Education i...Accreditation, Quality, And Graduate Attributes: A Case of Higher Education i...
Accreditation, Quality, And Graduate Attributes: A Case of Higher Education i...HennaAnsari
 
Oct 21 Learning-oriented assessment
Oct 21 Learning-oriented assessmentOct 21 Learning-oriented assessment
Oct 21 Learning-oriented assessmentDavid Carless
 

What's hot (20)

Academic audit presentation_-_introduction
Academic audit presentation_-_introductionAcademic audit presentation_-_introduction
Academic audit presentation_-_introduction
 
ASSESSMENT AND ACCREDITATION:REVISED PROCESS -AFFILIATED/CONSTITUENT COLLEGES
ASSESSMENT AND ACCREDITATION:REVISED PROCESS -AFFILIATED/CONSTITUENT COLLEGESASSESSMENT AND ACCREDITATION:REVISED PROCESS -AFFILIATED/CONSTITUENT COLLEGES
ASSESSMENT AND ACCREDITATION:REVISED PROCESS -AFFILIATED/CONSTITUENT COLLEGES
 
The use of technology in relation to the teaching excellence framework criteria
The use of technology in relation to the teaching excellence framework criteriaThe use of technology in relation to the teaching excellence framework criteria
The use of technology in relation to the teaching excellence framework criteria
 
Understanding GEM 2014 Presentation FINAL12
Understanding GEM 2014 Presentation FINAL12Understanding GEM 2014 Presentation FINAL12
Understanding GEM 2014 Presentation FINAL12
 
Academic & administrative audit
Academic & administrative audit Academic & administrative audit
Academic & administrative audit
 
Presentation for faculty awareness about Quality Management System and Accred...
Presentation for faculty awareness about Quality Management System and Accred...Presentation for faculty awareness about Quality Management System and Accred...
Presentation for faculty awareness about Quality Management System and Accred...
 
Enhancing the Undergraduate Experience Through A Collaborative Wiki Exercise ...
Enhancing the Undergraduate Experience Through A Collaborative Wiki Exercise ...Enhancing the Undergraduate Experience Through A Collaborative Wiki Exercise ...
Enhancing the Undergraduate Experience Through A Collaborative Wiki Exercise ...
 
Presentation on External Academic Audit Report (2017 18) - R.D.Sivakumar
Presentation on External Academic Audit Report (2017 18) - R.D.SivakumarPresentation on External Academic Audit Report (2017 18) - R.D.Sivakumar
Presentation on External Academic Audit Report (2017 18) - R.D.Sivakumar
 
Stephanie McKendry 'The conflicting priorities of blended and inclusive learn...
Stephanie McKendry 'The conflicting priorities of blended and inclusive learn...Stephanie McKendry 'The conflicting priorities of blended and inclusive learn...
Stephanie McKendry 'The conflicting priorities of blended and inclusive learn...
 
Introduction of Medical Education Domain
Introduction of Medical Education Domain Introduction of Medical Education Domain
Introduction of Medical Education Domain
 
From Quality Assurance to Quality Enhancement (2006)
From Quality Assurance to Quality Enhancement (2006)From Quality Assurance to Quality Enhancement (2006)
From Quality Assurance to Quality Enhancement (2006)
 
SAILing Takes You to Where Your Course Ought to Be - Course Technology Comput...
SAILing Takes You to Where Your Course Ought to Be - Course Technology Comput...SAILing Takes You to Where Your Course Ought to Be - Course Technology Comput...
SAILing Takes You to Where Your Course Ought to Be - Course Technology Comput...
 
Criteria 2 NAAC New guideline ppt
Criteria 2   NAAC New guideline  ppt Criteria 2   NAAC New guideline  ppt
Criteria 2 NAAC New guideline ppt
 
A Brief expression on National Assessment and Accreditation Council (NAAC) Ma...
A Brief expression on National Assessment and Accreditation Council (NAAC) Ma...A Brief expression on National Assessment and Accreditation Council (NAAC) Ma...
A Brief expression on National Assessment and Accreditation Council (NAAC) Ma...
 
OECD Review on Evaluation and Assessment Frameworks for Improving School Outc...
OECD Review on Evaluation and Assessment Frameworks for Improving School Outc...OECD Review on Evaluation and Assessment Frameworks for Improving School Outc...
OECD Review on Evaluation and Assessment Frameworks for Improving School Outc...
 
Sloan-C Quality Score Card
Sloan-C Quality Score Card Sloan-C Quality Score Card
Sloan-C Quality Score Card
 
Accreditation, Quality, And Graduate Attributes: A Case of Higher Education i...
Accreditation, Quality, And Graduate Attributes: A Case of Higher Education i...Accreditation, Quality, And Graduate Attributes: A Case of Higher Education i...
Accreditation, Quality, And Graduate Attributes: A Case of Higher Education i...
 
Controlling3
Controlling3Controlling3
Controlling3
 
Lccc curriculum design handouts- 1-17-1045
Lccc   curriculum design handouts- 1-17-1045Lccc   curriculum design handouts- 1-17-1045
Lccc curriculum design handouts- 1-17-1045
 
Oct 21 Learning-oriented assessment
Oct 21 Learning-oriented assessmentOct 21 Learning-oriented assessment
Oct 21 Learning-oriented assessment
 

Viewers also liked

ConnectEDU Course Evaluation Solution: CoursEval
ConnectEDU Course Evaluation Solution: CoursEvalConnectEDU Course Evaluation Solution: CoursEval
ConnectEDU Course Evaluation Solution: CoursEvalConnectEDU
 
конференция!!!
конференция!!!конференция!!!
конференция!!!Q1E3T5U7
 
презентация учебника
презентация учебникапрезентация учебника
презентация учебникаQ1E3T5U7
 
José gabriel chèrrez herrera
José gabriel chèrrez herreraJosé gabriel chèrrez herrera
José gabriel chèrrez herrerajcherrezherrera
 
презентация икт
презентация иктпрезентация икт
презентация иктQ1E3T5U7
 
Добрая Дорога
Добрая ДорогаДобрая Дорога
Добрая ДорогаVitaly Rudenko
 
Eloadas bemutato jo3
Eloadas bemutato jo3Eloadas bemutato jo3
Eloadas bemutato jo3lynxs
 
Why Oral SPA sonic toothbrush
Why Oral SPA sonic toothbrushWhy Oral SPA sonic toothbrush
Why Oral SPA sonic toothbrushoralspa2u
 
Презентация икт
Презентация иктПрезентация икт
Презентация иктQ1E3T5U7
 
359557 stevanka peter-prezentacia
359557 stevanka peter-prezentacia359557 stevanka peter-prezentacia
359557 stevanka peter-prezentaciaPeter Števanka
 
SVILUPPO DI SERVIZI REST PER ANDROID
SVILUPPO DI SERVIZI REST PER ANDROIDSVILUPPO DI SERVIZI REST PER ANDROID
SVILUPPO DI SERVIZI REST PER ANDROIDLuca Masini
 
The Anatomy of Continuous Deployment at Scale
The Anatomy of Continuous Deployment at ScaleThe Anatomy of Continuous Deployment at Scale
The Anatomy of Continuous Deployment at ScaleJohn Viner
 
Beyond Siri on the iPhone: How could intelligent systems change the way we in...
Beyond Siri on the iPhone: How could intelligent systems change the way we in...Beyond Siri on the iPhone: How could intelligent systems change the way we in...
Beyond Siri on the iPhone: How could intelligent systems change the way we in...Yousif Almas
 
Presentacion tabla periodica
Presentacion tabla periodicaPresentacion tabla periodica
Presentacion tabla periodicaAngel Carrillo
 
Effective course evaluation
Effective course evaluationEffective course evaluation
Effective course evaluationsurveyresults
 

Viewers also liked (16)

ConnectEDU Course Evaluation Solution: CoursEval
ConnectEDU Course Evaluation Solution: CoursEvalConnectEDU Course Evaluation Solution: CoursEval
ConnectEDU Course Evaluation Solution: CoursEval
 
конференция!!!
конференция!!!конференция!!!
конференция!!!
 
презентация учебника
презентация учебникапрезентация учебника
презентация учебника
 
José gabriel chèrrez herrera
José gabriel chèrrez herreraJosé gabriel chèrrez herrera
José gabriel chèrrez herrera
 
презентация икт
презентация иктпрезентация икт
презентация икт
 
Добрая Дорога
Добрая ДорогаДобрая Дорога
Добрая Дорога
 
Eloadas bemutato jo3
Eloadas bemutato jo3Eloadas bemutato jo3
Eloadas bemutato jo3
 
Why Oral SPA sonic toothbrush
Why Oral SPA sonic toothbrushWhy Oral SPA sonic toothbrush
Why Oral SPA sonic toothbrush
 
Презентация икт
Презентация иктПрезентация икт
Презентация икт
 
359557 stevanka peter-prezentacia
359557 stevanka peter-prezentacia359557 stevanka peter-prezentacia
359557 stevanka peter-prezentacia
 
SVILUPPO DI SERVIZI REST PER ANDROID
SVILUPPO DI SERVIZI REST PER ANDROIDSVILUPPO DI SERVIZI REST PER ANDROID
SVILUPPO DI SERVIZI REST PER ANDROID
 
The Anatomy of Continuous Deployment at Scale
The Anatomy of Continuous Deployment at ScaleThe Anatomy of Continuous Deployment at Scale
The Anatomy of Continuous Deployment at Scale
 
Beyond Siri on the iPhone: How could intelligent systems change the way we in...
Beyond Siri on the iPhone: How could intelligent systems change the way we in...Beyond Siri on the iPhone: How could intelligent systems change the way we in...
Beyond Siri on the iPhone: How could intelligent systems change the way we in...
 
Presentacion tabla periodica
Presentacion tabla periodicaPresentacion tabla periodica
Presentacion tabla periodica
 
wedding decoration
wedding decorationwedding decoration
wedding decoration
 
Effective course evaluation
Effective course evaluationEffective course evaluation
Effective course evaluation
 

Similar to Effective course evaluation the future for quality and standards in higher education

perspective chapter-performance based assessment through IBL.pdf
perspective chapter-performance based assessment through IBL.pdfperspective chapter-performance based assessment through IBL.pdf
perspective chapter-performance based assessment through IBL.pdfAyshaAlShamsi11
 
Quality standards and enhancement in zimbabwean universities; the role of lec...
Quality standards and enhancement in zimbabwean universities; the role of lec...Quality standards and enhancement in zimbabwean universities; the role of lec...
Quality standards and enhancement in zimbabwean universities; the role of lec...Daisy Ifeoma
 
Issotl12 handout sw
Issotl12 handout swIssotl12 handout sw
Issotl12 handout swShan Wareing
 
Formative Assessment as an Essential Competence of University Teachers
Formative Assessment as an Essential Competence of University TeachersFormative Assessment as an Essential Competence of University Teachers
Formative Assessment as an Essential Competence of University Teachersiosrjce
 
Developing innovative assessment
Developing innovative assessmentDeveloping innovative assessment
Developing innovative assessmentRichard Oelmann
 
QAA Modelling and Managing Student Satisfaction: Use of student feedback to ...
QAA Modelling and Managing Student Satisfaction: Use of student feedback to ...QAA Modelling and Managing Student Satisfaction: Use of student feedback to ...
QAA Modelling and Managing Student Satisfaction: Use of student feedback to ...Bart Rienties
 
Feasibility Study for Empowering the System of Assessment and Feedback in Hig...
Feasibility Study for Empowering the System of Assessment and Feedback in Hig...Feasibility Study for Empowering the System of Assessment and Feedback in Hig...
Feasibility Study for Empowering the System of Assessment and Feedback in Hig...AJHSSR Journal
 
Tqf day 1 - learning outcomes and teaching strategies
Tqf   day 1 - learning outcomes and teaching strategiesTqf   day 1 - learning outcomes and teaching strategies
Tqf day 1 - learning outcomes and teaching strategiesRMIT
 
317 - Delivering a Professional Administration - A case study in fieldwork ma...
317 - Delivering a Professional Administration - A case study in fieldwork ma...317 - Delivering a Professional Administration - A case study in fieldwork ma...
317 - Delivering a Professional Administration - A case study in fieldwork ma...Association of University Administrators
 
Essential Elements for Optimising the Pedagogical Framework
Essential Elements for Optimising the Pedagogical FrameworkEssential Elements for Optimising the Pedagogical Framework
Essential Elements for Optimising the Pedagogical FrameworkAJHSSR Journal
 
Jisc webinar: Curriculum design: Changing the paradigm
Jisc webinar: Curriculum design: Changing the paradigmJisc webinar: Curriculum design: Changing the paradigm
Jisc webinar: Curriculum design: Changing the paradigmJisc
 
Sloan-C Effective Practice Award Winners Webinars
Sloan-C Effective Practice Award Winners WebinarsSloan-C Effective Practice Award Winners Webinars
Sloan-C Effective Practice Award Winners WebinarsSandra Coswatte-Mohr
 
Manchester Next Generation Spaces keynote
Manchester Next Generation Spaces keynoteManchester Next Generation Spaces keynote
Manchester Next Generation Spaces keynoteAlejandro Armellini
 
A Comparative Study Of Competency-Based Courses Demonstrating A Potential Mea...
A Comparative Study Of Competency-Based Courses Demonstrating A Potential Mea...A Comparative Study Of Competency-Based Courses Demonstrating A Potential Mea...
A Comparative Study Of Competency-Based Courses Demonstrating A Potential Mea...Sheila Sinclair
 
MM Bagali ......Training and FDP/ MDP...... Faculty Teachers Training.....
MM Bagali ......Training and FDP/ MDP...... Faculty Teachers Training.....MM Bagali ......Training and FDP/ MDP...... Faculty Teachers Training.....
MM Bagali ......Training and FDP/ MDP...... Faculty Teachers Training.....dr m m bagali, phd in hr
 
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATION
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATIONINTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATION
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATIONIJITE
 
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATION
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATIONINTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATION
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATIONIJITE
 
Assessment of Information Literacy Learning
Assessment of Information Literacy LearningAssessment of Information Literacy Learning
Assessment of Information Literacy LearningJohan Koren
 
Enhancing Student Learning Experience and Satisfaction Using Virtual Learning...
Enhancing Student Learning Experience and Satisfaction Using Virtual Learning...Enhancing Student Learning Experience and Satisfaction Using Virtual Learning...
Enhancing Student Learning Experience and Satisfaction Using Virtual Learning...M H
 

Similar to Effective course evaluation the future for quality and standards in higher education (20)

perspective chapter-performance based assessment through IBL.pdf
perspective chapter-performance based assessment through IBL.pdfperspective chapter-performance based assessment through IBL.pdf
perspective chapter-performance based assessment through IBL.pdf
 
Quality standards and enhancement in zimbabwean universities; the role of lec...
Quality standards and enhancement in zimbabwean universities; the role of lec...Quality standards and enhancement in zimbabwean universities; the role of lec...
Quality standards and enhancement in zimbabwean universities; the role of lec...
 
Issotl12 handout sw
Issotl12 handout swIssotl12 handout sw
Issotl12 handout sw
 
Formative Assessment as an Essential Competence of University Teachers
Formative Assessment as an Essential Competence of University TeachersFormative Assessment as an Essential Competence of University Teachers
Formative Assessment as an Essential Competence of University Teachers
 
Developing innovative assessment
Developing innovative assessmentDeveloping innovative assessment
Developing innovative assessment
 
QAA Modelling and Managing Student Satisfaction: Use of student feedback to ...
QAA Modelling and Managing Student Satisfaction: Use of student feedback to ...QAA Modelling and Managing Student Satisfaction: Use of student feedback to ...
QAA Modelling and Managing Student Satisfaction: Use of student feedback to ...
 
Feasibility Study for Empowering the System of Assessment and Feedback in Hig...
Feasibility Study for Empowering the System of Assessment and Feedback in Hig...Feasibility Study for Empowering the System of Assessment and Feedback in Hig...
Feasibility Study for Empowering the System of Assessment and Feedback in Hig...
 
Tqf day 1 - learning outcomes and teaching strategies
Tqf   day 1 - learning outcomes and teaching strategiesTqf   day 1 - learning outcomes and teaching strategies
Tqf day 1 - learning outcomes and teaching strategies
 
317 - Delivering a Professional Administration - A case study in fieldwork ma...
317 - Delivering a Professional Administration - A case study in fieldwork ma...317 - Delivering a Professional Administration - A case study in fieldwork ma...
317 - Delivering a Professional Administration - A case study in fieldwork ma...
 
Essential Elements for Optimising the Pedagogical Framework
Essential Elements for Optimising the Pedagogical FrameworkEssential Elements for Optimising the Pedagogical Framework
Essential Elements for Optimising the Pedagogical Framework
 
Jisc webinar: Curriculum design: Changing the paradigm
Jisc webinar: Curriculum design: Changing the paradigmJisc webinar: Curriculum design: Changing the paradigm
Jisc webinar: Curriculum design: Changing the paradigm
 
Sloan-C Effective Practice Award Winners Webinars
Sloan-C Effective Practice Award Winners WebinarsSloan-C Effective Practice Award Winners Webinars
Sloan-C Effective Practice Award Winners Webinars
 
Manchester Next Generation Spaces keynote
Manchester Next Generation Spaces keynoteManchester Next Generation Spaces keynote
Manchester Next Generation Spaces keynote
 
A Comparative Study Of Competency-Based Courses Demonstrating A Potential Mea...
A Comparative Study Of Competency-Based Courses Demonstrating A Potential Mea...A Comparative Study Of Competency-Based Courses Demonstrating A Potential Mea...
A Comparative Study Of Competency-Based Courses Demonstrating A Potential Mea...
 
Naac new
Naac newNaac new
Naac new
 
MM Bagali ......Training and FDP/ MDP...... Faculty Teachers Training.....
MM Bagali ......Training and FDP/ MDP...... Faculty Teachers Training.....MM Bagali ......Training and FDP/ MDP...... Faculty Teachers Training.....
MM Bagali ......Training and FDP/ MDP...... Faculty Teachers Training.....
 
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATION
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATIONINTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATION
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATION
 
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATION
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATIONINTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATION
INTERNSHIP ASSESSMENT AND EVALUATION IN HIGHER EDUCATION
 
Assessment of Information Literacy Learning
Assessment of Information Literacy LearningAssessment of Information Literacy Learning
Assessment of Information Literacy Learning
 
Enhancing Student Learning Experience and Satisfaction Using Virtual Learning...
Enhancing Student Learning Experience and Satisfaction Using Virtual Learning...Enhancing Student Learning Experience and Satisfaction Using Virtual Learning...
Enhancing Student Learning Experience and Satisfaction Using Virtual Learning...
 

Recently uploaded

CARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptxCARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptxGaneshChakor2
 
_Math 4-Q4 Week 5.pptx Steps in Collecting Data
_Math 4-Q4 Week 5.pptx Steps in Collecting Data_Math 4-Q4 Week 5.pptx Steps in Collecting Data
_Math 4-Q4 Week 5.pptx Steps in Collecting DataJhengPantaleon
 
Introduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxIntroduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxpboyjonauth
 
MENTAL STATUS EXAMINATION format.docx
MENTAL     STATUS EXAMINATION format.docxMENTAL     STATUS EXAMINATION format.docx
MENTAL STATUS EXAMINATION format.docxPoojaSen20
 
Presiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha electionsPresiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha electionsanshu789521
 
Mastering the Unannounced Regulatory Inspection
Mastering the Unannounced Regulatory InspectionMastering the Unannounced Regulatory Inspection
Mastering the Unannounced Regulatory InspectionSafetyChain Software
 
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdfBASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdfSoniaTolstoy
 
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...Marc Dusseiller Dusjagr
 
Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)eniolaolutunde
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Krashi Coaching
 
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17Incoming and Outgoing Shipments in 1 STEP Using Odoo 17
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17Celine George
 
Micromeritics - Fundamental and Derived Properties of Powders
Micromeritics - Fundamental and Derived Properties of PowdersMicromeritics - Fundamental and Derived Properties of Powders
Micromeritics - Fundamental and Derived Properties of PowdersChitralekhaTherkar
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13Steve Thomason
 
Introduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher EducationIntroduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher Educationpboyjonauth
 
Paris 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityParis 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityGeoBlogs
 
Separation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesSeparation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesFatimaKhan178732
 
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdfssuser54595a
 
APM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAPM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAssociation for Project Management
 

Recently uploaded (20)

TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
TataKelola dan KamSiber Kecerdasan Buatan v022.pdfTataKelola dan KamSiber Kecerdasan Buatan v022.pdf
TataKelola dan KamSiber Kecerdasan Buatan v022.pdf
 
CARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptxCARE OF CHILD IN INCUBATOR..........pptx
CARE OF CHILD IN INCUBATOR..........pptx
 
_Math 4-Q4 Week 5.pptx Steps in Collecting Data
_Math 4-Q4 Week 5.pptx Steps in Collecting Data_Math 4-Q4 Week 5.pptx Steps in Collecting Data
_Math 4-Q4 Week 5.pptx Steps in Collecting Data
 
Introduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxIntroduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptx
 
MENTAL STATUS EXAMINATION format.docx
MENTAL     STATUS EXAMINATION format.docxMENTAL     STATUS EXAMINATION format.docx
MENTAL STATUS EXAMINATION format.docx
 
Presiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha electionsPresiding Officer Training module 2024 lok sabha elections
Presiding Officer Training module 2024 lok sabha elections
 
Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1Código Creativo y Arte de Software | Unidad 1
Código Creativo y Arte de Software | Unidad 1
 
Mastering the Unannounced Regulatory Inspection
Mastering the Unannounced Regulatory InspectionMastering the Unannounced Regulatory Inspection
Mastering the Unannounced Regulatory Inspection
 
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdfBASLIQ CURRENT LOOKBOOK  LOOKBOOK(1) (1).pdf
BASLIQ CURRENT LOOKBOOK LOOKBOOK(1) (1).pdf
 
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
“Oh GOSH! Reflecting on Hackteria's Collaborative Practices in a Global Do-It...
 
Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)Software Engineering Methodologies (overview)
Software Engineering Methodologies (overview)
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
 
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17Incoming and Outgoing Shipments in 1 STEP Using Odoo 17
Incoming and Outgoing Shipments in 1 STEP Using Odoo 17
 
Micromeritics - Fundamental and Derived Properties of Powders
Micromeritics - Fundamental and Derived Properties of PowdersMicromeritics - Fundamental and Derived Properties of Powders
Micromeritics - Fundamental and Derived Properties of Powders
 
The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13The Most Excellent Way | 1 Corinthians 13
The Most Excellent Way | 1 Corinthians 13
 
Introduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher EducationIntroduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher Education
 
Paris 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activityParis 2024 Olympic Geographies - an activity
Paris 2024 Olympic Geographies - an activity
 
Separation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesSeparation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and Actinides
 
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
18-04-UA_REPORT_MEDIALITERAСY_INDEX-DM_23-1-final-eng.pdf
 
APM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across SectorsAPM Welcome, APM North West Network Conference, Synergies Across Sectors
APM Welcome, APM North West Network Conference, Synergies Across Sectors
 

Effective course evaluation the future for quality and standards in higher education

  • 1. Effective Course Evaluation The Future for Quality and Standards in Higher Education A summary report from interviews among senior academics and student representatives
  • 2.
  • 3. CONTENTS Acknowledgements 4 Introduction by Eric Bohms, Managing Director, Electric Paper Ltd 5 Executive Summary 6 Improving Response Rates 7 Improving Student Feedback 8 Improving Turnaround Time 9 Improving Survey Administration 10 Improving The Student Experience 11 Conclusion 12 Evaluation Potential 13-14
  • 4. ACKNOWLEDGEMENTS The editors of this report, Phil Smith and Owen Morris at Communications Management, would like to thank the following people who were interviewed in July and August 2011: • Professor Susannah Quinsee, Director of Learning Development, City University London. • Professor Ian Marshall, Deputy Vice-Chancellor (Academic), Coventry University. • Professor David Coates, Dean of School of Life Sciences’ Learning and Teaching Division, University of Dundee. • Professor Glenn Burgess, Pro-Vice-Chancellor of Learning and Teaching, University of Hull. • Professor Andy Pitsillides, Chair of Teaching Quality Committee, Royal Veterinary College, University of London. • Professor Huw Morris, Pro-Vice-Chancellor (Academic), University of Salford. • Professor Alan Speight, Pro-Vice-Chancellor (Student Experience and Academic Quality Enhancement), University of Swansea, and member of Quality Assurance Agency Board of Directors. • Alex Bols, Head of Education and Quality, National Union of Students. • Alex Smith, former Academic Affairs Officer, University of Leicester Students’ Union, and member of Quality Assurance Agency Board of Directors. • Alex Nutt, current Academic Affairs Officer, University of Leicester Students’ Union. 4 Effective Course Evaluation
  • 5. INTRODUCTION The 2011-12 academic year is a big one for the Higher Education sector. This is the year when, more than at any previous point at time, UK universities will have to provide clear evidence of the ‘value’ of studying at their institution. Universities are digesting the implications of the Higher Education White Paper, published in June 2011, which has asked institutions to be more accountable to students on teaching quality and to provide greater transparency in areas such as the student experience. Specifically the paper says that “universities will be expected to publish online summary reports of student surveys of lecture courses, aiding choice and stimulating competition between the best academics”. Therefore, gaining effective student feedback on courses and lecturers to enable transparent reporting is a must. At Electric Paper we work with over 600 universities in the UK and worldwide to help them to evaluate their courses through effective student feedback via our automated paper and online survey management system EvaSys. We hope that this report will draw attention to the latest issues and trends around effective course evaluation which, in turn, will help us to develop our support for the Higher Education sector. But why is it so important that we get this right? Firstly, and most importantly, with the increase in tuition fees from 2012, universities can no longer afford to leave teaching and learning quality in the hands of academics alone. The University’s executive needs to have visibility around the quality of the course and lecturer, and establishing the best way to tackle the ‘process’ of getting student feedback is crucial. Secondly, the National Student Survey (NSS) is currently the primary method of gathering feedback on the quality of students’ courses in order to contribute to public accountability and help inform the choices of future applicants. Gaining comprehensive, instant, feedback on courses and lecturers can assist universities in improving overall quality, and NSS scores. Thirdly, as many universities have taken the decision to charge the full £9,000, it will become even more important for students to distinguish between institutions on issues of quality. Quickly capturing feedback across all courses, providing students and their parents with evidence-based feedback about the quality of teaching and learning, will be a big advantage. Fourth, individual universities need to stand out from the crowd. With the introduction of ‘Key Information Sets’ universities need to do everything they can to provide indicators of quality. Capturing, and responding positively to, student feedback on course evaluation can only help boost recruitment targets and reputation. Finally, implementing an effective process for capturing student feedback around courses and lecturers is a money-saver for universities. Institutions spend a huge amount of time and money on survey administration, and approaches are inconsistent (e.g. with management of surveys sitting centrally in one university, and departmentally in another). An effective process for course evaluation can offer significant cost and efficiency savings, addressing the hidden costs in survey administration and freeing up staff expertise for other areas. Eric Bohms, Managing Director, Electric Paper Ltd Effective Course Evaluation 5
  • 6. EXECUTIVE SUMMARY The purpose of this report is to examine • Universities need to work harder at the latest issues and trends in course feeding back to students the actions they evaluation. In-depth interviews have been will be taking as a result of input provided conducted with 10 senior academics and for course and lecturer evaluation surveys. student representatives to draw out a • End-of-module evaluation is a particular general sense of how universities are stumbling block in the provision of currently approaching the ‘process’ of feedback to students - and feedback can student surveys, and what improvements be slow - but moving to mid-module need to be made to gain more effective evaluation can help to improve the student feedback on courses and lecturers. process. This qualitative research is particularly • Ideally students want the opportunity to relevant in the context of the 2011 Higher express their views on course Education White Paper, which has asked improvements at a time that their universities to be more accountable to feedback benefits them directly. students on teaching quality and to provide greater transparency in areas such as the • Universities need to embrace new student experience. Rising tuition fees in technologies to improve turnaround time 2012 mean that gaining (and responding - but effective feedback can be gained via effectively to) student feedback on courses a combination of paper and online and lecturers is increasingly important for surveys. universities as they have to provide clear • Universities should establish a more evidence of the ‘value’ of studying at their consistent (centralised) approach to survey institution. administration - including a standard set This report finds that: of survey questions - to enable effective benchmarking at course and institutional • Many universities seeking feedback on level. However, individual departments courses and lecturers via surveys struggle should have the flexibility to include to achieve a meaningful response from bespoke questions for particular courses. students, which is partly due to students being inundated with requests to • In-class student involvement in survey complete surveys. administration can increase commitment as they are stakeholders in the process. • Student representatives have indicated that students are not effectively engaged • Effective course evaluation is necessary in the feedback process and, for some, for universities to provide a clear evidence providing feedback can even be base to demonstrate their ‘value’ to intimidating. students. 6 Effective Course Evaluation
  • 7. THE NEED TO... IMPROVE RESPONSE RATES Many universities seeking feedback on Quality Assurance Agency Board of courses and lecturers via surveys are still Directors, agreed that there is a need for struggling to achieve a meaningful universities to ensure that students are response from students - that is, a “properly engaged” and even “incentivised” response rate of over 50% that will ensure to take part in the process. However, statistical validity in order to evaluate “providing feedback can be intimidating for teaching quality and make improvements. students,” he said. Alex Nutt, who Professor Glenn Burgess, Pro-Vice- succeeded Smith at Leicester’s Students’ Chancellor of Learning and Teaching at the Union added that “despite the process University of Hull, said the problem was being anonymous some students feel that students were simply inundated with intimidated by the idea of giving negative requests to complete surveys. “A key issue feedback in case it reflects on them”. facing universities in relation to gaining effective course feedback from students is “We introduced mid-module surveys around achieving significant response rates and went back to paper and the to any questionnaire-based approach. response was super” Students are increasingly overloaded with surveys - from the National Student Survey Coventry University has managed to downwards - so you need a good reason to increase its response rates by using paper encourage participation. It’s also important and moving to mid-module surveys. to appreciate the qualitative/quantitative “Historically universities conduct end-of- divide in surveys. Often the most useful course, or end-of-module surveys, but by information is gained through qualitative the time the feedback has been analysed feedback, but this is also the most difficult and results published the students have to achieve as it requires more time from gone away,” said Professor Ian Marshall, respondents.” the University’s Deputy Vice-Chancellor This is a view shared by Professor (Academic). “We moved to online surveys, Susannah Quinsee, Director of Learning but the response was dreadful, so last year Development at City University London, we introduced mid-module surveys and who said that “a lot of course evaluation” is went back to paper. The response was geared towards the National Student super, and we are now able to turn around Survey. “Ideally you want to engage in feedback in two weeks maximum.” He dialogue with students, because this is added that if a module is perceived to be more useful than knowing that half the underperforming “we will focus on those”. class love a module and half the class hate At Coventry University mid-module it. You want to know more about what the surveys are handed out in class by 120 middle group think. You also need more senior student representatives who are open questions rather than tick box, but interviewed and selected by the University’s students are not so keen on these Students’ Union. The University of Hull is questions. So there is an issue over student also working closely with its Students’ engagement, and an associated issue Union to develop its approach to around the extent to which students engage improving response rates. “What we have in the language used in evaluation surveys.” found is that students are tolerant of Alex Smith, former Academic Affairs problems, but they want us to listen, feed Officer at the University of Leicester’s back quickly, and respond,” said Professor Students’ Union, and a member of the Burgess. Effective Course Evaluation 7
  • 8. THE NEED TO... IMPROVE STUDENT FEEDBACK Through this research a clear gap has not going to feel the benefits of any emerged around the extent to which UK improvements made,” said City’s Professor universities currently feed back to Susannah Quinsee. “Ideally you would students the action they will be taking as want to explore in-module evaluation, a result of input provided in course and which we are doing, but that takes a lot of lecturer evaluation surveys. time both in terms of implementation and “Students need to provide feedback, then analysis.” the universities need to take that on board Student representatives agree that end- and do what they can with it, and then of-module evaluation is no longer provide clear feedback to the students on sufficient. “It’s a flawed system,” said the actions and outcomes,” said ex- Smith. “In my experience universities often University of Leicester Students’ Union ask for student feedback at the end of a Academic Affairs Officer, Alex Smith. “The course, but by then it’s too late, and the feedback loop must underpin everything.” students do not see any results from the Alex Bols, Head of Education and Quality feedback they’ve given. Universities often at the National Union of Students (NUS), profess a need to get the feedback system said that “in the majority of cases” right but I don’t think this has necessarily students who participate in course happened yet as, by only asking for evaluation surveys, in his experience, are feedback at the end of a course, they are then not told what happens as a result of gathering entirely new opinions from a the process. “It’s important for universities different cohort of students and therefore to close the loop and tell students what cannot compare like-for-like.” has happened - or hasn’t happened - as a Professor Alan Speight, Pro-Vice- result of the feedback provided and why. Chancellor (Student Experience and This should not be an autopsy at the end Academic Quality Enhancement) at the of a course, but a process embedded University of Swansea, and a member of through the learning experience so that it the Quality Assurance Agency Board of is of benefit to the student giving the Directors, said that establishing an feedback and their experience.” effective system for student feedback was vital for universities’ reputation. “There is a “It's important for universities to close need to dig down to obtain feedback as it the loop and tell students what has or does not take much to make students feel hasn't happened” disgruntled. Universities are only as strong as their weakest link and poor delivery by a End-of-module evaluation appears to be subset of staff that goes unchecked could a particular stumbling block in the make a lot of difference to the overall provision of feedback to students. For teaching reputation of an institution.” example, City University London is three “Students are more interested in years into a JISC-funded four-year project outcomes - so it’s important for on curriculum design, and staff have universities to be very clear on what they looked at the issue of effective course are able to do, and equally be honest on evaluation. “What we’re finding is that what they are not able to,” added evaluation for a lot of students is not Coventry’s Professor Ian Marshall. meaningful because they are being asked to give feedback on a course or module they are just completing, and are therefore 8 Effective Course Evaluation
  • 9. THE NEED TO... IMPROVE TURNAROUND TIME Relating to the issue of student feedback is universities need this for both instant the challenge that universities face in feedback and for internal benchmarking turning around the findings of course leading to organisational improvement. evaluation surveys. Student groups agree. “By examining Professor Andy Pitsillides, Chair of the the process and proving that they are Teaching Quality Committee at the Royal interested, and capable of acting upon, the Veterinary College, University of London, feedback they receive in a timely manner admitted that the process can be “very universities could build partnerships with slow” in Higher Education. “As academics their students to add value all-round,” said we know that students want quick the NUS’ Alex Bols. “Students should be feedback as it helps their learning, yet we able to express how their course could be are restricted by the annual cycle of improved while it impacts on them.” surveying. The exercise is also paper- Exploiting innovative new technologies driven, so it is more cumbersome than we could support the requirement to improve might like, and we are looking at a process turnaround time, according to Professor of electronic feedback to augment this Huw Morris, Pro-Vice-Chancellor approach.” (Academic) at the University of Salford, who in his previous role as Dean of “As academics we know that students Manchester Metropolitan University want quick feedback as it helps their Business School led the trial of course learning” surveys via mobile phones. “Going forward I anticipate the Higher Education sector It’s a major issue facing the sector, will need to utilise online devices to according to Swansea’s Professor Alan capture student feedback, but at the same Speight. “Turnaround time is vital, but a big time ensure this is not done in an intrusive problem at the moment. In some areas manner. Some element of compulsion for surveys may be paper-based and manually students in providing feedback will also be processed, which means that someone has helpful in ensuring that the results are to physically input the results and is open to representative of underlying views.” human error. Also the feedback may come In addition to module evaluation, there is back when it is too late for the staff to do a clear need for feedback on the anything about it as they are, by that time, underlying quality of the students’ already committed to a teaching pattern for educational experience, Professor Morris the next academic session, which in turn said. “The National Student Survey and, to makes the students wonder why their a lesser extent, the International Student thoughts have not been listened to.” Barometer, are helpful and most A survey of 100 university administrators universities respond to these. But while we by Electric Paper, conducted in 2011, found know about student happiness and 74% of respondents agree that both paper satisfaction, we don’t know that much and online surveys is a requirement in about how their educational experiences Higher Education. It’s not a question of prepare them in the longer term. Here either/or, the respondents said, because there is a need to draw on evidence from both methodologies can provide high other tools which assess the contribution response rates and efficiency savings for of the students’ educational experience to the right surveys in the right context. their career prospects and personal and However, turnaround time is key as social development in the future.” Effective Course Evaluation 9
  • 10. THE NEED TO... IMPROVESURVEYADMINISTRATION To improve response rates, student feedback Other universities are making moves and turnaround time, universities need to towards centralisation, such as the University have a more consistent approach to survey of Hull. “We have a central code of practice administration. Through this research it has for survey administration which provides the become clear that the management of foundation for all modular and programme surveys sits centrally in one university, and surveys,” said Hull’s Professor Glenn departmentally in another. In addition, a 2011 Burgess. “But within the University there is Electric Paper survey found that academics currently no consistency on the level of are managing their own survey approach and questions being asked locally, administration at 1 in 3 institutions - and that and we have plans to change that. This most universities are facing ‘absolute chaos’ brings a pressure on resource for central around survey management. processes and for us to introduce a particular What is clear from interviews with senior way of working. It also links to wider policy academics is that centralisation of course issues - for example, how is feedback linked evaluation feedback analysis is required. to staff appraisal, what happens in terms of Following the arrival of Vice-Chancellor, responding to poor feedback, and how Professor Paul Curran, City University public do you make this information.” London has introduced a centralised modular evaluation system. “We now have a “A common set of questions should be standard set of questions for surveys, owned institutionally, with subject areas managed centrally, which individual schools able to select from optional additional can add to if they wish, and the results of questions” these are now part of staff appraisals,” said City’s Professor Susannah Quinsee. The The University of Swansea’s Professor Alan scheme being rolled out by Coventry Speight said that there were historic University is managed by a central unit challenges to overcome for many institutions based in student services, while the in order for the feedback process to become University of Salford has established centralised. “The difficulty has been that standard questionnaires for module, level surveys carried out by different departments and programme-centred evaluation which have evolved organically and been developed are co-ordinated by a central planning and in silos that are tailored just to the needs of performance department. one faculty. This results in a lack of core Professor David Coates, Dean of the School information across the institution, mixed of Life Sciences’ Learning and Teaching responses and no consistent use of data. Division at the University of Dundee, said There should be a unified approach that “the process of receiving and then includes core questions and specifies the way disseminating feedback needs to be clean to the feedback is processed - which allows ensure that those doing the teaching will benchmarking and consistency. A common receive the knowledge they need”. He added: set of core questions should be owned “Firstly, there needs to be consistency in the institutionally, with subject areas able to feedback the institution receives as a whole, select from a bank of optional additional but equally courses should not always be questions.” compared like-for-like. A hub-and-spoke As with the Coventry University model, model works best where individual Professor Speight suggested that students departments are evaluated but then this could be brought on board to help co- feeds into a holistic view of the institution.” ordinate the process. 10 Effective Course Evaluation
  • 11. THE NEED TO... IMPROVE THE STUDENT EXPERIENCE The volatile Higher Education landscape - and lecturer feedback method so there the increase in student tuition fees and can be constant improvements that will the focus on student-led decision-making determine a student’s opinion of their - means that universities are under institution based on direct experience.” pressure to be more accountable and A wider issue beyond module evaluation transparent on issues of quality. is the need for UK universities to move Alex Nutt, Academic Affairs Officer at the towards surveying “all student groups”, University of Leicester Students’ Union, according to Salford’s Professor Huw said that prospective students and their Morris. The National Student Survey and parents “will want to know as much as International Student Barometer do not, possible” in order to make their university he said, cover UK and other EU second decision, citing examples from the quality and third year students at Levels 4 and 5, of lectures to resources in the library. “I taught postgraduate UK and other EU also think they will want to know that students at level 7, or UK and other EU institutions take the concerns of students postgraduate research students at levels 7 seriously and that education is seen as a and above. collaborative partnership between the The UK may also wish to take note of university and the students, not just a developments across the Atlantic, business transaction.” Professor Morris said. “In the USA there are other survey methods which are more “It’s brilliant news for students who to comprehensive and which focus more all intents and purposes are consumers clearly on contribution. For example, Pew and have the right to know this studies and the National Survey of information” Student Engagement provide more comprehensive and possibly more candid “There is a going to be a race to and easily accessible measures of student improve in Higher Education and the experiences in particular institutions and results will be there for all to see, whereas particular colleges, schools and previously things that were not done as departments. With the changes in tuition well may have been brushed under the fees, I think we will see more information carpet,” explained Nutt’s predecessor, of this kind being expected by students Alex Smith. “It’s brilliant news for and their parents because they will want students who to all intents and purposes to measure return on investment.” are consumers and have the right to know However, City’s Professor Susannah this information.” Quinsee, said she had “difficulty” with the Universities are only too aware of the overall notion of students as consumers. need to provide a clear evidence base to “That implies a customer-supplier demonstrate their ‘value’. “Of paramount relationship where education is about importance in the future will be the partnership and dialogue. Universities do student experience as this will determine need to get students more involved in how to promote the institution to the programme design, and evaluation and outside world and potential students,” feedback is all part of that, but students said Dundee’s Professor David Coates. also need to work with universities to “With the National Student Survey, and tell us what data they find useful, what the introduction of ‘Key Information Sets’, they expect, and above all what they it is vital that there is an effective course find meaningful.” Effective Course Evaluation 11
  • 12. CONCLUSION Interviews with senior academics and However, universities also recognise that student representatives for this report have current evaluation processes need uncovered a strong feeling that universities development - there is a clear need to need to improve their approach to student support academics in improving the quality feedback on courses and lecturers via of a course by providing comprehensive, module surveys. instant, student feedback - and research The core perspective coming out of indicates that a combination of both paper universities is that they have difficulty in and online surveys is a requirement in achieving a “meaningful” response from Higher Education survey administration. students to surveys, and turnaround time What is clear is that all universities need to from response to feedback is not as quick as embrace this agenda. Prospective students it could or should be. (and their parents) applying for degree From the student perspective there is an programmes from 2012-13 will be looking issue around the extent to which students for detailed and transparent information to feel engaged in the feedback process, not help them distinguish between Higher least because universities do not do enough Education institutions on issues of quality to feed back the outcomes of their survey and standards in relation to courses. input, and the extent to which they feel the Individual universities need to stand out benefit of any actions taken as a result of from the crowd - and providing clear their feedback. baseline reporting about the quality of Some universities are moving from end- teaching and learning is going to be critical of-module evaluation to mid-module going forward. evaluation, which may help to address these As one interviewee for this report summed issues. Universities also recognise the need up: “There is a lot of activity in the sector for a more consistent, and centralised around understanding the student better approach to survey administration which and communicating with them - and this will potentially enable more effective could be a crucial differentiation for institutional benchmarking. universities in the next five years.” 12 Effective Course Evaluation
  • 13. POTENTIAL FOR PROCESS IMPROVEMENT In this report senior academics and student higher response rates.” Between 2009 and representatives have expressed an interest 2010 more than 45,000 questionnaires were in improving course evaluation practices in completed, surveying over 400 graduate universities. teaching assistants, 600 courses, and 700 Electric Paper Ltd, a software solution permanent teachers. Once the completed provider to the education sector based in the surveys were received Electric Paper scanned UK, Germany and Switzerland, has developed up to 3,000 a day, providing an instant, EvaSys Survey Management Suite, a hybrid individual, daily report for each paper and online survey automation system course/teacher combination. Whereas which enables Higher Education institutions previously some staff had to wait weeks, to improve and efficiently manage their turnaround times improved dramatically module evaluations and other student with results returned quickly and in an easy- feedback surveys. to-read format. “The swift turnaround of By managing the process centrally through clear and accurate results was a key the web-based system, institutions are able requirement for the new system and EvaSys to reduce administration time and costs more than met expectations,” Mr Page said. whilst distributing detailed analysis to “EvaSys was selected after an extensive stakeholders in a timely manner. Using both tendering exercise because it offered the paper and online surveys, EvaSys utilises in- most flexible yet dedicated system for our class evaluations to maximise response rates needs - it’s a clear winner in our eyes.” and therefore validating results. In July 2011 Electric Paper announced its Queen Mary, University of London latest product release, EvaSys Education V5.0. As part of its five-year strategic plan Queen At the time this report was published in Mary, University of London (QMUL) August 2011 Electric Paper was working with required a system which would allow central over 600 universities worldwide, and collection and analysis of student examples of its UK university partnerships evaluations for its 16,000 students to ensure are below. rigorous monitoring and assessment of taught programmes. In 2010 QMUL London School of Economics and contracted Electric Paper’s EvaSys product, Political Science which provides flexible and automated The London School of Economics and course and module evaluations including Political Science (LSE) required a system paper and online surveys and detailed which would allow it to quickly and efficiently reporting. “We wanted to ensure that all survey its 9,000 students and gain feedback students are invited to give anonymous on its courses and teaching. LSE turned to feedback on every module and that such Electric Paper’s EvaSys survey management feedback is collated and used systematically system, which allows the flexibility to evaluate to assure and enhance the quality of modules using both online and paper-based QMUL’s taught provision,” said Emma surveys automatically and without the need Wynne, Assistant Academic Registrar at for manual data entry. “There were two main QMUL. In the initial pilot roll-out, nine out goals when introducing EvaSys at the of 17 schools participated, surveying over School,” explained Mike Page, Head of ARD 380 modules. Results were completed Systems and Business Processes at LSE. “We within three weeks with an average response wanted to improve the timeliness and rate of 59%, with some modules as high as accuracy of survey results, and to encourage 73%. As the EvaSys system fulfilled the Effective Course Evaluation 13
  • 14. POTENTIAL FOR PROCESS IMPROVEMENT objectives of the scheme, senior staff overhaul of the process was needed. As a responsible for taught programmes and result, Electric Paper’s EvaSys was purchased QMUL’s Vice-Principal recommended the by St Andrews in November 2010 and used adoption of EvaSys throughout the to produce that semester’s module University commencing in October 2011. evaluations. In addition, a core set of “The feedback from the schools was that the questions was developed to include reports generated by EvaSys are rapid, compulsory questions relating to modules, informative and clearly presented,” Ms individual lecturers and individual tutors, Wynne said. “From a programme supported by a maximum of eight optional management and quality enhancement questions from a pool of 16. Once perspective, the presentation of results in completed, the questionnaires were scanned graphical format - including School and and instant analysis reports produced. College averages - was particularly useful in Reports were automatically emailed to identifying areas of concern and examples module coordinators and to lecturers to of excellent practice.” receive individual feedback tied to the internal review of teaching and learning. University of St Andrews This alerted senior management to any The University of St Andrews in Scotland has areas of concern, and to identify good established a reputation as one of Europe’s practice that deserved commendation and leading and most distinctive centres for dissemination. “The introduction of EvaSys teaching and research. Previously, while run has simplified the module feedback process centrally, academics requested their own immensely,” said Carol Morris, Director of individualised questionnaire resulting in the Centre for Academic, Professional and over 1,000 unique versions with ensuing Organisational Development at St Andrews. difficulties in longitudinal comparison of the “Results are now available in days rather quality of delivery and content of the than months allowing more time to monitor module. The University established that a quality and implement enhancements to review of the feedback system and an teaching and learning.” 14 Effective Course Evaluation
  • 15. Need help in implementing centralised module evaluation? Electric Paper works with over 600 universities to provide effective student feedback systems www.electricpaper.co.uk
  • 16. Electric Paper Ltd International House, 1-6 Yarmouth Place, Mayfair, London, W1J 7BU Telephone: 0203 145 3258 Email: info@electricpaper.co.uk www.electricpaper.co.uk