Quality Assurance Measures• How do we measure what we are doing in this new service?• What do the students want from us?• How do we know whether we are doing well?• How do we know if our students are satisfied?• Let’s start with what we count?• What do we count?
What did we measure and count?• Headcounts • Enrolments • Matrix assessment• Gatecounts • Registrations • Number of appointments• SCONUL Returns • HESA Returns • Number of referrals• Annual Statistics • Number of coursework • Retention numbers• LibQUAL submissions • NSS feedback and• Feedback and Comments • Speed of enrolment comments scheme • Speed of coursework • Graduate destinations• Boards of Study processing• NSS feedback and • Enrolment survey comments • Boards of Study • Programme Boards Student Welfare andLibrary and IT Administration CareersSupport and Finance
Why?•What is it contributing to?•How do we join things up?•How do individual service areas feed into aninstitutional development plan?•Who do we benchmark against?•Where are the common standards?•Where are the common service levels?•Where are the common values?
Justifying the impact and value How do we know if what we are doing and what we are providing is successful?
Impact and Satisfaction• How do you know whether your customers are satisfied?• How do you know whether what you do has an impact?
Impact or Satisfaction?• “There is a tendency to confuse impact with customer/student satisfaction. Customer satisfaction focuses on measuring whether or not student like or are happy with the educational experience and services they receive”
Impact or Satisfaction?• “...Impact, however, is aimed at measuring whether or not the educational experience is making any difference to what they do” (AMOSHE Value and Impact Tool Kit)• What therefore is our scope?
Need to do something different? Massive increase in customersatisfaction – driven by changes in management behaviours Added value of staff and measuring the impact of the customer at the moment of truth!
Measure student satisfaction at key moments of truth Enrolment InductionInformation skills Study support Staff support Assessment
Measures of performance and satisfaction for 2010-11
Quality Framework – EnrolmentNumbers KPIs Feedback• No. of fastrack • Time taken to enrol • Bespoke enrolment enrolments • % of new starters satisfaction surveys• No. of online successfully completing • Fastrack enrolments fastrack enrolment • Non-Fastrack• No. of successful • % of successful online • Returning students enrolments for enrolments • Anomalies returning students • % of students satisfied with enrolment
Quality Framework – InductionNumbers KPIs Feedback• No. of Induction • % of eligible students • Standard pro-forma to be sessions receiving induction completed at each induction sessions (e.g.• No. of students eligible • % of students satisfied which students, how many, for induction with induction etc) • Induction survey to be completed by all students at induction to measure satisfaction with induction, perceived usefulness of induction
Quality Framework – Information LiteracyNumbers KPIs Feedback• Number of info skills • % of eligible students • Induction survey to be sessions receiving info skills completed by all students at induction to measure• Number of students • % of students satisfied satisfaction with induction, eligible for info skills with info skills perceived usefulness of • Correlation with LMS/e- induction resource usage • Comments in LibQUAL
Quality Framework – Study SupportNumbers KPIs Feedback• Number of different • Enquiry by student/type • Exit survey at key times in types of enquiry • Loans per student the year to measure satisfaction with enquiry• Number of book loans • Correlation with LMS/e- services• Number of Downloads resource usage • Feedback and comments in• LMF spend per student • Downloads per student LibQUAL and NSS• Time taken for re- • % of students satisfied shelving with enquiry service• Time taken for document delivery
Quality Framework – Staff SupportNumbers KPIs Feedback• Number of different • % of appointments • Feedback and comments types of enquiry attended on staff support within NSS, Mirror survey,• Number of • % of enquiries Comments scheme and appointments made / responded to within LibQUAL attended target time • Periodic enquiry• Time taken to respond • % of students satisfied satisfaction surveys to enquiries and with enquiry services • Student Finance online referrals transaction survey
Quality Framework – AssessmentNumbers KPIs Feedback• Number of coursework • No of pieces of • School office satisfaction submissions coursework processed survey• Times taken to get to within agreed service • Student satisfaction with standard - % satisfaction whole service School Offices rate • % of coursework arriving at School within standard time frame
Results – EnrolmentNumbers KPIs Feedback1,265 students were satisfied or very satisfied 100% of “Quick and easy”with the information they received students “Friendly, helpful staff” satisfied86 students (8%) were dissatisfied with the 0% “Staff were unsure”whole enrolment process dissatisfacti “Got sent to the wrong on place”1,017 students perceived a 10 minute or 100% less “Quick and easy”under wait than 10 minutes238 students perceived a wait of between 10 100% less “Too long spentand 30 minutes than 10 queuing” minutes25 students perceived a wait of 30 minutes 100% less “International studentsplus than 10 seem to get treated minutes unfairly”50 % of students stated that enrolment ?? “Service was muchexceeded expectations better and quicker than expected”
Results – SupportNumbers KPIs Feedback53% of enquiries at the Transaction 100% “Went to the Hub and was then sentDesk, 30% at the Hub, 8% at the to the Transaction Desk”Finance Office92% of customers satisfied with 100% “Lovely staff and always helpful”time taken to answer enquiry (8% “Always get a prompt response”dissatisfied)86% of customers satisfied with the 100% “The member of staff was rude andinformation they received (14% did not answer my enquiry”dissatisfied)92% of online enquiries referred to 100%and answered within 24 hours89% of telephone enquiries 100%referred answered within 24 hours
What now? Collate, analyse, report back on data from QAF AMOSSHE Value and Impact Tool Kit Staff understanding of Values driven Quality AssuranceCommitment to quality improvement and continual responsiveness approach to student feedback and satisfaction
Numbers? KPIs?• Can you think of anything different you might do with your numbers?• Do you need to start measuring different things using different methods?• Actions?
Thank You• Leo Appleton, Associate Director, Library and Student Support, Liverpool John Moores University.• firstname.lastname@example.org• Appleton, L., (2010) ‘Living through super-convergence: creating Library and Student Support at Liverpool John Moores University, SCONUL Focus, 49, pp. 67-70• Appleton, L., (2011) ‘Super-convergence. Liverpool John Moores University, 7th October 2010, SCONUL Focus, 51, pp. 85-89
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.