Download to read offline
Uploaded byAssociation of University Administrators
HE Course and Module Evaluation Conference -
This document discusses course evaluation at a university with over 24,000 students across 19 schools and 9 research institutes. It summarizes the current state of course evaluation, including piloting a new online system, and establishes a working group to review course evaluation practices. The working group identified multiple purposes of course evaluation, from summative reporting to facilitating continuous improvement. They reviewed hundreds of existing evaluation questions and developed a standardized set of 5 core questions, with optional supplementary question sets. The working group aims to provide clearer guidance on the purpose and use of course evaluation to improve both teaching and course design across the university.
More Related Content
Similar to HE Course and Module Evaluation Conference -
HE Course and Module Evaluation Conference -
- 1. Taking Course evaluationback to basics Just what are we evaluating and why?
- 2. Scene setting Student population –Undergraduate 19,096 – Postgraduate Taught 5,147 4 Colleges – Arts – Medicine, Veterinary and Life Sciences – Science and Engineering – Social Sciences Total of 19 Schools and 9 Research Institutes
- 3. Course evaluation Session 2012-13- EvaSys piloted – 1 School per College Working Group established to review Recommendations: • EvaSys rolled out on a voluntary basis • Review of course evaluation
- 4. Issues • Not compulsoryto use questionnaires for course evaluation • Previously introduced standard questionnaire – lasted one year! • Large and varied means of evaluating courses • When questionnaires were being used – a wide assortment of areas and questions were being asked.
- 5. Just what arewe evaluating? What do we use course evaluation to evaluate?
- 6. Areas evaluated • Therelevance of the course? • The student experience on the course? • The teaching content? • The quality of teaching? • The assessment methods? • The ‘popularity’ factor? Then, there is…..
- 7. Areas evaluated • Assessmentof dissertation/project supervisors • Assessment of placements • Assessment of learning and teaching support • Assessment of accommodation • Assessment of laboratories, seminars • Assessment of GTAs • Assessment of feedback received
- 8. What do wedo w What do we do with the information? • Management overview/auditing purposes • Provide feedback to lecturers • Provide feedback to course organisers and programme directors • Provide feedback to Head of School, Head of College • Provide information for marketing purposes • Provide feedback to students!
- 9. Working Group oncourse feedback questionnaires • To identify the purpose of course evaluation • The nature of feedback processes • Consideration of the use of questionnaire data • Administration of feedback processes
- 10. Summative • To providea snap-shot of the past, a description of how the course ran • To highlight existing good practice • To highlight exceptional quality that can be referred to in marketing and recruitment activities • To recognise teaching as evidence for teaching excellence awards or promotion.
- 11. Formative • To helpidentify where improvements can be made, and where there are unexpected problems that need to be solved • To ensure that the quality of teaching is sustained and improved • To identify where teaching staff might need additional support or resources • To demonstrate the use of feedback practices across the university for the purposes of audit
- 12. Purpose of CourseQuestionnaire evaluation Reviewed 300 questions! Discussed questionnaire design
- 13. Course feedback evaluationquestionnaire • 5 core questions to be used on all questionnaires • Teaching and course evaluation to be reviewed separately • Optional question sets to survey different aspects
- 14. Core Question Set CORE1a.The lecturer explained things well. • Strongly Agree □ □ □ □ □ Strongly Disagree • or • CORE1b. My project/dissertation/placement supervisor was helpful. • Strongly Agree □ □ □ □ □ Strongly Disagree • CORE2. The course was intellectually stimulating. • Strongly Agree □ □ □ □ □ Strongly Disagree • CORE3. I am satisfied with the overall quality of the course. • Strongly Agree □ □ □ □ □ Strongly Disagree • CORE4. What was good about the course? • OPEN • CORE5. How could this course be improved? • OPEN
- 15. Clarify what isbeing evaluated: Other Question sets • Teaching Quality set (9 qns) • Course Quality set (9 + 5) • PGT set (4) • Expectation/Marketing set (5) • Ad hoc set (up to max 22 qns)
- 16. What to dowith the information? • Course – Improving course design – Involving students with curriculum design and development • Teaching – Improving teaching – Provide source for recognition and award of teaching • Feedback – Improving student feedback
- 17. Linkage to otherprocesses • P&DR and Recognition of Excellent Teachers • Annual Monitoring • Internal Subject Review • Marketing
- 18. Moving Forward Establishment ofAdvisory Board – Strategic overview – Review potential developments and linkages to other processes – Identification and dissemination of good practice – Update and revise policy as required
- 19. And Finally…. Any questions? CatherineOmand Senior Academic Policy Manager Senate Office
Editor's Notes
- #3 Large, diverse institution – restructured in 2010 – moving from 8 Faculties to 4 Colleges. Each College having a substantial level of autonomy
- #4 Voluntary basis? As an registration system had been recently introduced that had not gone smoothly – academics would not have warmly received another “imposed” system As the Working Group reviewed the software –the benefits of EvaSys were recognised but for it to be effective – required a level of standardisation across the University for cross-Institution comparison for Quality Enhancement activities. This was currently not standard practice at Glasgow
- #5 Academics found the standard questionnaire too structured - considered unsuitable and inflexible. Working Group identified a large range of methods for course evaluation – focus groups, student interviews and question/answer sessions at end of class popular ways for obtaining feedback And when questionnaires where being used – wide range of questions were being used. So – Working Group recommended that both the role of course evaluation as well as using course evaluation questionnaires should be re-examined.
- #6 Seems like a very basic and stupid question but what is evaluated as part of course evaluation?
- #7 Some of these look similar, but actually are quite different…
- #8 Quite a lot going on – and often feedback was being sought on all of the above! And What where we doing with the information collated?
- #9 Supposed to be doing – but were we really? So another Working Group was established – Course Feedback Questionnaires It concluded that the purpose of course evaluation was -
- #10 Important to mention WG convened by Dr Helen Purchase (Computing Science) (and the EvaSys Working Group chaired by Dr Moira Fischbacher-Smith (Dean (L&T)) Invested substantial time in the group and establishing the report and final recommendations) So what did the Group conclude was the purpose of course feedback?
- #12 But how do we achieve this from a questionnaire and what questions should be asked?
- #13 The Group reviewed 300 questions Questions were placed into groups - which highlighted the different ways to ask the same thing! Often similar questions included on the same questionnaire, some misleading, some confusing and open for misinterpretation Not only looked at questions – but design of questionnaires (eg The appropriate number of tick boxes? (we went with 5!) Good Left or right? Psychology of completing questionnaires, the ideal length, etc Decided: what was the optimal number of questions? Ask group for ideal number of questions…. THEN…ask them to go into groups and discuss what they would think the 5 questions the Group came up with Once discussion has taken place ……
- #14 Out of 300 questions – agreed on 5 core questions! These are the only ones that are compulsory. In a group – take 10 minutes to discuss what questions you would consider core?
- #15 It was agreed that this met all requirements - This was the core information required to fulfil the needs of the University, the course organiser, the lecturer, the student. 3 scaled; 2 open questions The 5 questions would be the only compulsory element to the questionnaire. Schools could choose to use just these 5 questions. However, if more detail was required….
- #16 Question sets were identified to target specific areas Remember all those 300 questions – went through to find ideal questions for each set. 9 questions for Teaching set 9 + 5 for Course set 4 for PGT set 5 for Marketing set Ad hoc – any other questions the School wished to add Maximum number of questions – should not exceed 22 Okay – now we had a template for our questionnaires - often next stage of evaluation can go astray – what do we do with the information collated?
- #17 Next stage – establishing viable systems are in place to ensures outcome of course evaluation are effectively managed and used. Feedback on courses should fed into course design, should involve students Feedback on teaching should improve teaching and be recognised as part of the recognition and award schemes Students should be aware that their feedback is being taken on board
- #18 Also – potential to link into other processes Anything else? Next stage:
- #19 Now we have our questionnaire, we know what we want to do with our feedback – now need processes to ensure successful transfer of information and communication - Advisory Board recently created to oversee EvaSys implementation, updating and revising policy as required Oversee evaluation processes are working and enhancing other processes Identification and dissemination of good practice Identification of appropriate support and training And finally …. To oversee future developments This is the stage where we are currently at – Board is convened by the same person who chaired the Working Group – for continuality – representation from all Colleges both academic and administrative and from IT Services. Might have to be a future event – where we report back whether or not our strategy has been successful! Any questions?