Effective Course EvaluationThe Future for Quality and Standardsin Higher EducationA summary report from interviews amongse...
CONTENTSAcknowledgements                        4Introduction by Eric Bohms,Managing Director, Electric Paper Ltd   5Execu...
ACKNOWLEDGEMENTS             The editors of this report, Phil Smith and Owen Morris at Communications Management,         ...
INTRODUCTIONThe 2011-12 academic year is a big one for the Higher Education sector. This is the yearwhen, more than at any...
EXECUTIVE SUMMARY             The purpose of this report is to examine       • Universities need to work harder at        ...
THE NEED TO...IMPROVE RESPONSE RATESMany universities seeking feedback on           Quality Assurance Agency Board ofcours...
THE NEED TO...             IMPROVE STUDENT FEEDBACK             Through this research a clear gap has           not going ...
THE NEED TO...IMPROVE TURNAROUND TIMERelating to the issue of student feedback is     universities need this for both inst...
THE NEED TO...             IMPROVESURVEYADMINISTRATION             To improve response rates, student feedback          Ot...
THE NEED TO...IMPROVE THE STUDENT EXPERIENCEThe volatile Higher Education landscape -       and lecturer feedback method s...
CONCLUSION              Interviews with senior academics and             However, universities also recognise that        ...
POTENTIAL FORPROCESS IMPROVEMENTIn this report senior academics and student        higher response rates.” Between 2009 an...
POTENTIAL FOR             PROCESS IMPROVEMENT              objectives of the scheme, senior staff           overhaul of th...
Need help  in implementing centralised module     evaluation?Electric Paper works with over 600 universities to   provide ...
Electric Paper LtdInternational House,1-6 Yarmouth Place,Mayfair, London, W1J 7BUTelephone: 0203 145 3258Email: info@elect...
Effective course evaluation   the future for quality and standards in higher education
Upcoming SlideShare
Loading in...5

Effective course evaluation the future for quality and standards in higher education


Published on

The Future for Quality and Standards in Higher Education. Interviews with Senior Academics and Student Representatives on the topic of effective course and module evaluation

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Effective course evaluation the future for quality and standards in higher education

  1. 1. Effective Course EvaluationThe Future for Quality and Standardsin Higher EducationA summary report from interviews amongsenior academics and student representatives
  2. 2. CONTENTSAcknowledgements 4Introduction by Eric Bohms,Managing Director, Electric Paper Ltd 5Executive Summary 6Improving Response Rates 7Improving Student Feedback 8Improving Turnaround Time 9Improving Survey Administration 10Improving The Student Experience 11Conclusion 12Evaluation Potential 13-14
  3. 3. ACKNOWLEDGEMENTS The editors of this report, Phil Smith and Owen Morris at Communications Management, would like to thank the following people who were interviewed in July and August 2011: • Professor Susannah Quinsee, Director of Learning Development, City University London. • Professor Ian Marshall, Deputy Vice-Chancellor (Academic), Coventry University. • Professor David Coates, Dean of School of Life Sciences’ Learning and Teaching Division, University of Dundee. • Professor Glenn Burgess, Pro-Vice-Chancellor of Learning and Teaching, University of Hull. • Professor Andy Pitsillides, Chair of Teaching Quality Committee, Royal Veterinary College, University of London. • Professor Huw Morris, Pro-Vice-Chancellor (Academic), University of Salford. • Professor Alan Speight, Pro-Vice-Chancellor (Student Experience and Academic Quality Enhancement), University of Swansea, and member of Quality Assurance Agency Board of Directors. • Alex Bols, Head of Education and Quality, National Union of Students. • Alex Smith, former Academic Affairs Officer, University of Leicester Students’ Union, and member of Quality Assurance Agency Board of Directors. • Alex Nutt, current Academic Affairs Officer, University of Leicester Students’ Union.4 Effective Course Evaluation
  4. 4. INTRODUCTIONThe 2011-12 academic year is a big one for the Higher Education sector. This is the yearwhen, more than at any previous point at time, UK universities will have to provide clearevidence of the ‘value’ of studying at their institution.Universities are digesting the implications of the Higher Education White Paper, published inJune 2011, which has asked institutions to be more accountable to students on teaching qualityand to provide greater transparency in areas such as the student experience. Specifically thepaper says that “universities will be expected to publish online summary reports of studentsurveys of lecture courses, aiding choice and stimulating competition between the bestacademics”. Therefore, gaining effective student feedback on courses and lecturers to enabletransparent reporting is a must.At Electric Paper we work with over 600 universities in the UK and worldwide to help them toevaluate their courses through effective student feedback via our automated paper and onlinesurvey management system EvaSys. We hope that this report will draw attention to the latestissues and trends around effective course evaluation which, in turn, will help us to develop oursupport for the Higher Education sector. But why is it so important that we get this right?Firstly, and most importantly, with the increase in tuition fees from 2012, universities can nolonger afford to leave teaching and learning quality in the hands of academics alone. TheUniversity’s executive needs to have visibility around the quality of the course and lecturer, andestablishing the best way to tackle the ‘process’ of getting student feedback is crucial.Secondly, the National Student Survey (NSS) is currently the primary method of gatheringfeedback on the quality of students’ courses in order to contribute to public accountability andhelp inform the choices of future applicants. Gaining comprehensive, instant, feedback oncourses and lecturers can assist universities in improving overall quality, and NSS scores.Thirdly, as many universities have taken the decision to charge the full £9,000, it will becomeeven more important for students to distinguish between institutions on issues of quality.Quickly capturing feedback across all courses, providing students and their parents withevidence-based feedback about the quality of teaching and learning, will be a big advantage.Fourth, individual universities need to stand out from the crowd. With the introduction of ‘KeyInformation Sets’ universities need to do everything they can to provide indicators of quality.Capturing, and responding positively to, student feedback on course evaluation can only helpboost recruitment targets and reputation.Finally, implementing an effective process for capturing student feedback around courses andlecturers is a money-saver for universities. Institutions spend a huge amount of time andmoney on survey administration, and approaches are inconsistent (e.g. with management ofsurveys sitting centrally in one university, and departmentally in another). An effective processfor course evaluation can offer significant cost and efficiency savings, addressing the hiddencosts in survey administration and freeing up staff expertise for other areas.Eric Bohms, Managing Director, Electric Paper Ltd Effective Course Evaluation 5
  5. 5. EXECUTIVE SUMMARY The purpose of this report is to examine • Universities need to work harder at the latest issues and trends in course feeding back to students the actions they evaluation. In-depth interviews have been will be taking as a result of input provided conducted with 10 senior academics and for course and lecturer evaluation surveys. student representatives to draw out a • End-of-module evaluation is a particular general sense of how universities are stumbling block in the provision of currently approaching the ‘process’ of feedback to students - and feedback can student surveys, and what improvements be slow - but moving to mid-module need to be made to gain more effective evaluation can help to improve the student feedback on courses and lecturers. process. This qualitative research is particularly • Ideally students want the opportunity to relevant in the context of the 2011 Higher express their views on course Education White Paper, which has asked improvements at a time that their universities to be more accountable to feedback benefits them directly. students on teaching quality and to provide greater transparency in areas such as the • Universities need to embrace new student experience. Rising tuition fees in technologies to improve turnaround time 2012 mean that gaining (and responding - but effective feedback can be gained via effectively to) student feedback on courses a combination of paper and online and lecturers is increasingly important for surveys. universities as they have to provide clear • Universities should establish a more evidence of the ‘value’ of studying at their consistent (centralised) approach to survey institution. administration - including a standard set This report finds that: of survey questions - to enable effective benchmarking at course and institutional • Many universities seeking feedback on level. However, individual departments courses and lecturers via surveys struggle should have the flexibility to include to achieve a meaningful response from bespoke questions for particular courses. students, which is partly due to students being inundated with requests to • In-class student involvement in survey complete surveys. administration can increase commitment as they are stakeholders in the process. • Student representatives have indicated that students are not effectively engaged • Effective course evaluation is necessary in the feedback process and, for some, for universities to provide a clear evidence providing feedback can even be base to demonstrate their ‘value’ to intimidating. students.6 Effective Course Evaluation
  6. 6. THE NEED TO...IMPROVE RESPONSE RATESMany universities seeking feedback on Quality Assurance Agency Board ofcourses and lecturers via surveys are still Directors, agreed that there is a need forstruggling to achieve a meaningful universities to ensure that students areresponse from students - that is, a “properly engaged” and even “incentivised”response rate of over 50% that will ensure to take part in the process. However,statistical validity in order to evaluate “providing feedback can be intimidating forteaching quality and make improvements. students,” he said. Alex Nutt, who Professor Glenn Burgess, Pro-Vice- succeeded Smith at Leicester’s Students’Chancellor of Learning and Teaching at the Union added that “despite the processUniversity of Hull, said the problem was being anonymous some students feelthat students were simply inundated with intimidated by the idea of giving negativerequests to complete surveys. “A key issue feedback in case it reflects on them”.facing universities in relation to gainingeffective course feedback from students is “We introduced mid-module surveysaround achieving significant response rates and went back to paper and theto any questionnaire-based approach. response was super”Students are increasingly overloaded withsurveys - from the National Student Survey Coventry University has managed todownwards - so you need a good reason to increase its response rates by using paperencourage participation. It’s also important and moving to mid-module surveys.to appreciate the qualitative/quantitative “Historically universities conduct end-of-divide in surveys. Often the most useful course, or end-of-module surveys, but byinformation is gained through qualitative the time the feedback has been analysedfeedback, but this is also the most difficult and results published the students haveto achieve as it requires more time from gone away,” said Professor Ian Marshall,respondents.” the University’s Deputy Vice-Chancellor This is a view shared by Professor (Academic). “We moved to online surveys,Susannah Quinsee, Director of Learning but the response was dreadful, so last yearDevelopment at City University London, we introduced mid-module surveys andwho said that “a lot of course evaluation” is went back to paper. The response wasgeared towards the National Student super, and we are now able to turn aroundSurvey. “Ideally you want to engage in feedback in two weeks maximum.” Hedialogue with students, because this is added that if a module is perceived to bemore useful than knowing that half the underperforming “we will focus on those”.class love a module and half the class hate At Coventry University mid-moduleit. You want to know more about what the surveys are handed out in class by 120middle group think. You also need more senior student representatives who areopen questions rather than tick box, but interviewed and selected by the University’sstudents are not so keen on these Students’ Union. The University of Hull isquestions. So there is an issue over student also working closely with its Students’engagement, and an associated issue Union to develop its approach toaround the extent to which students engage improving response rates. “What we havein the language used in evaluation surveys.” found is that students are tolerant of Alex Smith, former Academic Affairs problems, but they want us to listen, feedOfficer at the University of Leicester’s back quickly, and respond,” said ProfessorStudents’ Union, and a member of the Burgess. Effective Course Evaluation 7
  7. 7. THE NEED TO... IMPROVE STUDENT FEEDBACK Through this research a clear gap has not going to feel the benefits of any emerged around the extent to which UK improvements made,” said City’s Professor universities currently feed back to Susannah Quinsee. “Ideally you would students the action they will be taking as want to explore in-module evaluation, a result of input provided in course and which we are doing, but that takes a lot of lecturer evaluation surveys. time both in terms of implementation and “Students need to provide feedback, then analysis.” the universities need to take that on board Student representatives agree that end- and do what they can with it, and then of-module evaluation is no longer provide clear feedback to the students on sufficient. “It’s a flawed system,” said the actions and outcomes,” said ex- Smith. “In my experience universities often University of Leicester Students’ Union ask for student feedback at the end of a Academic Affairs Officer, Alex Smith. “The course, but by then it’s too late, and the feedback loop must underpin everything.” students do not see any results from the Alex Bols, Head of Education and Quality feedback they’ve given. Universities often at the National Union of Students (NUS), profess a need to get the feedback system said that “in the majority of cases” right but I don’t think this has necessarily students who participate in course happened yet as, by only asking for evaluation surveys, in his experience, are feedback at the end of a course, they are then not told what happens as a result of gathering entirely new opinions from a the process. “It’s important for universities different cohort of students and therefore to close the loop and tell students what cannot compare like-for-like.” has happened - or hasn’t happened - as a Professor Alan Speight, Pro-Vice- result of the feedback provided and why. Chancellor (Student Experience and This should not be an autopsy at the end Academic Quality Enhancement) at the of a course, but a process embedded University of Swansea, and a member of through the learning experience so that it the Quality Assurance Agency Board of is of benefit to the student giving the Directors, said that establishing an feedback and their experience.” effective system for student feedback was vital for universities’ reputation. “There is a “Its important for universities to close need to dig down to obtain feedback as it the loop and tell students what has or does not take much to make students feel hasnt happened” disgruntled. Universities are only as strong as their weakest link and poor delivery by a End-of-module evaluation appears to be subset of staff that goes unchecked could a particular stumbling block in the make a lot of difference to the overall provision of feedback to students. For teaching reputation of an institution.” example, City University London is three “Students are more interested in years into a JISC-funded four-year project outcomes - so it’s important for on curriculum design, and staff have universities to be very clear on what they looked at the issue of effective course are able to do, and equally be honest on evaluation. “What we’re finding is that what they are not able to,” added evaluation for a lot of students is not Coventry’s Professor Ian Marshall. meaningful because they are being asked to give feedback on a course or module they are just completing, and are therefore8 Effective Course Evaluation
  8. 8. THE NEED TO...IMPROVE TURNAROUND TIMERelating to the issue of student feedback is universities need this for both instantthe challenge that universities face in feedback and for internal benchmarkingturning around the findings of course leading to organisational improvement.evaluation surveys. Student groups agree. “By examining Professor Andy Pitsillides, Chair of the the process and proving that they areTeaching Quality Committee at the Royal interested, and capable of acting upon, theVeterinary College, University of London, feedback they receive in a timely manneradmitted that the process can be “very universities could build partnerships withslow” in Higher Education. “As academics their students to add value all-round,” saidwe know that students want quick the NUS’ Alex Bols. “Students should befeedback as it helps their learning, yet we able to express how their course could beare restricted by the annual cycle of improved while it impacts on them.”surveying. The exercise is also paper- Exploiting innovative new technologiesdriven, so it is more cumbersome than we could support the requirement to improvemight like, and we are looking at a process turnaround time, according to Professorof electronic feedback to augment this Huw Morris, Pro-Vice-Chancellorapproach.” (Academic) at the University of Salford, who in his previous role as Dean of“As academics we know that students Manchester Metropolitan Universitywant quick feedback as it helps their Business School led the trial of courselearning” surveys via mobile phones. “Going forward I anticipate the Higher Education sector It’s a major issue facing the sector, will need to utilise online devices toaccording to Swansea’s Professor Alan capture student feedback, but at the sameSpeight. “Turnaround time is vital, but a big time ensure this is not done in an intrusiveproblem at the moment. In some areas manner. Some element of compulsion forsurveys may be paper-based and manually students in providing feedback will also beprocessed, which means that someone has helpful in ensuring that the results areto physically input the results and is open to representative of underlying views.”human error. Also the feedback may come In addition to module evaluation, there isback when it is too late for the staff to do a clear need for feedback on theanything about it as they are, by that time, underlying quality of the students’already committed to a teaching pattern for educational experience, Professor Morristhe next academic session, which in turn said. “The National Student Survey and, tomakes the students wonder why their a lesser extent, the International Studentthoughts have not been listened to.” Barometer, are helpful and most A survey of 100 university administrators universities respond to these. But while weby Electric Paper, conducted in 2011, found know about student happiness and74% of respondents agree that both paper satisfaction, we don’t know that muchand online surveys is a requirement in about how their educational experiencesHigher Education. It’s not a question of prepare them in the longer term. Hereeither/or, the respondents said, because there is a need to draw on evidence fromboth methodologies can provide high other tools which assess the contributionresponse rates and efficiency savings for of the students’ educational experience tothe right surveys in the right context. their career prospects and personal andHowever, turnaround time is key as social development in the future.” Effective Course Evaluation 9
  9. 9. THE NEED TO... IMPROVESURVEYADMINISTRATION To improve response rates, student feedback Other universities are making moves and turnaround time, universities need to towards centralisation, such as the University have a more consistent approach to survey of Hull. “We have a central code of practice administration. Through this research it has for survey administration which provides the become clear that the management of foundation for all modular and programme surveys sits centrally in one university, and surveys,” said Hull’s Professor Glenn departmentally in another. In addition, a 2011 Burgess. “But within the University there is Electric Paper survey found that academics currently no consistency on the level of are managing their own survey approach and questions being asked locally, administration at 1 in 3 institutions - and that and we have plans to change that. This most universities are facing ‘absolute chaos’ brings a pressure on resource for central around survey management. processes and for us to introduce a particular What is clear from interviews with senior way of working. It also links to wider policy academics is that centralisation of course issues - for example, how is feedback linked evaluation feedback analysis is required. to staff appraisal, what happens in terms of Following the arrival of Vice-Chancellor, responding to poor feedback, and how Professor Paul Curran, City University public do you make this information.” London has introduced a centralised modular evaluation system. “We now have a “A common set of questions should be standard set of questions for surveys, owned institutionally, with subject areas managed centrally, which individual schools able to select from optional additional can add to if they wish, and the results of questions” these are now part of staff appraisals,” said City’s Professor Susannah Quinsee. The The University of Swansea’s Professor Alan scheme being rolled out by Coventry Speight said that there were historic University is managed by a central unit challenges to overcome for many institutions based in student services, while the in order for the feedback process to become University of Salford has established centralised. “The difficulty has been that standard questionnaires for module, level surveys carried out by different departments and programme-centred evaluation which have evolved organically and been developed are co-ordinated by a central planning and in silos that are tailored just to the needs of performance department. one faculty. This results in a lack of core Professor David Coates, Dean of the School information across the institution, mixed of Life Sciences’ Learning and Teaching responses and no consistent use of data. Division at the University of Dundee, said There should be a unified approach that “the process of receiving and then includes core questions and specifies the way disseminating feedback needs to be clean to the feedback is processed - which allows ensure that those doing the teaching will benchmarking and consistency. A common receive the knowledge they need”. He added: set of core questions should be owned “Firstly, there needs to be consistency in the institutionally, with subject areas able to feedback the institution receives as a whole, select from a bank of optional additional but equally courses should not always be questions.” compared like-for-like. A hub-and-spoke As with the Coventry University model, model works best where individual Professor Speight suggested that students departments are evaluated but then this could be brought on board to help co- feeds into a holistic view of the institution.” ordinate the process.10 Effective Course Evaluation
  10. 10. THE NEED TO...IMPROVE THE STUDENT EXPERIENCEThe volatile Higher Education landscape - and lecturer feedback method so therethe increase in student tuition fees and can be constant improvements that willthe focus on student-led decision-making determine a student’s opinion of their- means that universities are under institution based on direct experience.”pressure to be more accountable and A wider issue beyond module evaluationtransparent on issues of quality. is the need for UK universities to move Alex Nutt, Academic Affairs Officer at the towards surveying “all student groups”,University of Leicester Students’ Union, according to Salford’s Professor Huwsaid that prospective students and their Morris. The National Student Survey andparents “will want to know as much as International Student Barometer do not,possible” in order to make their university he said, cover UK and other EU seconddecision, citing examples from the quality and third year students at Levels 4 and 5,of lectures to resources in the library. “I taught postgraduate UK and other EUalso think they will want to know that students at level 7, or UK and other EUinstitutions take the concerns of students postgraduate research students at levels 7seriously and that education is seen as a and above.collaborative partnership between the The UK may also wish to take note ofuniversity and the students, not just a developments across the Atlantic,business transaction.” Professor Morris said. “In the USA there are other survey methods which are more“It’s brilliant news for students who to comprehensive and which focus moreall intents and purposes are consumers clearly on contribution. For example, Pewand have the right to know this studies and the National Survey ofinformation” Student Engagement provide more comprehensive and possibly more candid “There is a going to be a race to and easily accessible measures of studentimprove in Higher Education and the experiences in particular institutions andresults will be there for all to see, whereas particular colleges, schools andpreviously things that were not done as departments. With the changes in tuitionwell may have been brushed under the fees, I think we will see more informationcarpet,” explained Nutt’s predecessor, of this kind being expected by studentsAlex Smith. “It’s brilliant news for and their parents because they will wantstudents who to all intents and purposes to measure return on investment.”are consumers and have the right to know However, City’s Professor Susannahthis information.” Quinsee, said she had “difficulty” with the Universities are only too aware of the overall notion of students as consumers.need to provide a clear evidence base to “That implies a customer-supplierdemonstrate their ‘value’. “Of paramount relationship where education is aboutimportance in the future will be the partnership and dialogue. Universities dostudent experience as this will determine need to get students more involved inhow to promote the institution to the programme design, and evaluation andoutside world and potential students,” feedback is all part of that, but studentssaid Dundee’s Professor David Coates. also need to work with universities to“With the National Student Survey, and tell us what data they find useful, whatthe introduction of ‘Key Information Sets’, they expect, and above all what theyit is vital that there is an effective course find meaningful.” Effective Course Evaluation 11
  11. 11. CONCLUSION Interviews with senior academics and However, universities also recognise that student representatives for this report have current evaluation processes need uncovered a strong feeling that universities development - there is a clear need to need to improve their approach to student support academics in improving the quality feedback on courses and lecturers via of a course by providing comprehensive, module surveys. instant, student feedback - and research The core perspective coming out of indicates that a combination of both paper universities is that they have difficulty in and online surveys is a requirement in achieving a “meaningful” response from Higher Education survey administration. students to surveys, and turnaround time What is clear is that all universities need to from response to feedback is not as quick as embrace this agenda. Prospective students it could or should be. (and their parents) applying for degree From the student perspective there is an programmes from 2012-13 will be looking issue around the extent to which students for detailed and transparent information to feel engaged in the feedback process, not help them distinguish between Higher least because universities do not do enough Education institutions on issues of quality to feed back the outcomes of their survey and standards in relation to courses. input, and the extent to which they feel the Individual universities need to stand out benefit of any actions taken as a result of from the crowd - and providing clear their feedback. baseline reporting about the quality of Some universities are moving from end- teaching and learning is going to be critical of-module evaluation to mid-module going forward. evaluation, which may help to address these As one interviewee for this report summed issues. Universities also recognise the need up: “There is a lot of activity in the sector for a more consistent, and centralised around understanding the student better approach to survey administration which and communicating with them - and this will potentially enable more effective could be a crucial differentiation for institutional benchmarking. universities in the next five years.”12 Effective Course Evaluation
  12. 12. POTENTIAL FORPROCESS IMPROVEMENTIn this report senior academics and student higher response rates.” Between 2009 andrepresentatives have expressed an interest 2010 more than 45,000 questionnaires werein improving course evaluation practices in completed, surveying over 400 graduateuniversities. teaching assistants, 600 courses, and 700 Electric Paper Ltd, a software solution permanent teachers. Once the completedprovider to the education sector based in the surveys were received Electric Paper scannedUK, Germany and Switzerland, has developed up to 3,000 a day, providing an instant,EvaSys Survey Management Suite, a hybrid individual, daily report for eachpaper and online survey automation system course/teacher combination. Whereaswhich enables Higher Education institutions previously some staff had to wait weeks,to improve and efficiently manage their turnaround times improved dramaticallymodule evaluations and other student with results returned quickly and in an easy-feedback surveys. to-read format. “The swift turnaround of By managing the process centrally through clear and accurate results was a keythe web-based system, institutions are able requirement for the new system and EvaSysto reduce administration time and costs more than met expectations,” Mr Page said.whilst distributing detailed analysis to “EvaSys was selected after an extensivestakeholders in a timely manner. Using both tendering exercise because it offered thepaper and online surveys, EvaSys utilises in- most flexible yet dedicated system for ourclass evaluations to maximise response rates needs - it’s a clear winner in our eyes.”and therefore validating results. In July 2011 Electric Paper announced its Queen Mary, University of Londonlatest product release, EvaSys Education V5.0. As part of its five-year strategic plan Queen At the time this report was published in Mary, University of London (QMUL)August 2011 Electric Paper was working with required a system which would allow centralover 600 universities worldwide, and collection and analysis of studentexamples of its UK university partnerships evaluations for its 16,000 students to ensureare below. rigorous monitoring and assessment of taught programmes. In 2010 QMULLondon School of Economics and contracted Electric Paper’s EvaSys product,Political Science which provides flexible and automatedThe London School of Economics and course and module evaluations includingPolitical Science (LSE) required a system paper and online surveys and detailedwhich would allow it to quickly and efficiently reporting. “We wanted to ensure that allsurvey its 9,000 students and gain feedback students are invited to give anonymouson its courses and teaching. LSE turned to feedback on every module and that suchElectric Paper’s EvaSys survey management feedback is collated and used systematicallysystem, which allows the flexibility to evaluate to assure and enhance the quality ofmodules using both online and paper-based QMUL’s taught provision,” said Emmasurveys automatically and without the need Wynne, Assistant Academic Registrar atfor manual data entry. “There were two main QMUL. In the initial pilot roll-out, nine outgoals when introducing EvaSys at the of 17 schools participated, surveying overSchool,” explained Mike Page, Head of ARD 380 modules. Results were completedSystems and Business Processes at LSE. “We within three weeks with an average responsewanted to improve the timeliness and rate of 59%, with some modules as high asaccuracy of survey results, and to encourage 73%. As the EvaSys system fulfilled the Effective Course Evaluation 13
  13. 13. POTENTIAL FOR PROCESS IMPROVEMENT objectives of the scheme, senior staff overhaul of the process was needed. As a responsible for taught programmes and result, Electric Paper’s EvaSys was purchased QMUL’s Vice-Principal recommended the by St Andrews in November 2010 and used adoption of EvaSys throughout the to produce that semester’s module University commencing in October 2011. evaluations. In addition, a core set of “The feedback from the schools was that the questions was developed to include reports generated by EvaSys are rapid, compulsory questions relating to modules, informative and clearly presented,” Ms individual lecturers and individual tutors, Wynne said. “From a programme supported by a maximum of eight optional management and quality enhancement questions from a pool of 16. Once perspective, the presentation of results in completed, the questionnaires were scanned graphical format - including School and and instant analysis reports produced. College averages - was particularly useful in Reports were automatically emailed to identifying areas of concern and examples module coordinators and to lecturers to of excellent practice.” receive individual feedback tied to the internal review of teaching and learning. University of St Andrews This alerted senior management to any The University of St Andrews in Scotland has areas of concern, and to identify good established a reputation as one of Europe’s practice that deserved commendation and leading and most distinctive centres for dissemination. “The introduction of EvaSys teaching and research. Previously, while run has simplified the module feedback process centrally, academics requested their own immensely,” said Carol Morris, Director of individualised questionnaire resulting in the Centre for Academic, Professional and over 1,000 unique versions with ensuing Organisational Development at St Andrews. difficulties in longitudinal comparison of the “Results are now available in days rather quality of delivery and content of the than months allowing more time to monitor module. The University established that a quality and implement enhancements to review of the feedback system and an teaching and learning.”14 Effective Course Evaluation
  14. 14. Need help in implementing centralised module evaluation?Electric Paper works with over 600 universities to provide effective student feedback systems www.electricpaper.co.uk
  15. 15. Electric Paper LtdInternational House,1-6 Yarmouth Place,Mayfair, London, W1J 7BUTelephone: 0203 145 3258Email: info@electricpaper.co.ukwww.electricpaper.co.uk