Your SlideShare is downloading. ×
Performance and innovation culture in academic libraries: the role of LibQUAL+ in enhancing quality
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Performance and innovation culture in academic libraries: the role of LibQUAL+ in enhancing quality

128

Published on

J. Stephen Town, Cranfield University. …

J. Stephen Town, Cranfield University.
Evaluation of Library & Information Services: Does it lead to innovation and effectiveness?
November 16-17
Vilnius, Lithuania

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
128
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Some background on the University – extracts from the University web-pages demonstrate the interest in peer groupings, U21 - an international grouping of universities dedicated to setting world-wide standards for higher education.
  • Transcript

    • 1. Performance and innovation culture in academic libraries: the role of LibQUAL+ in enhancing quality J. Stephen Town Cranfield University Evaluation of Library & Information Services: Does it lead to innovation and effectiveness? November 16-17 Vilnius, Lithuania
    • 2. Summary • Thoughts on performance and innovation • LibQUAL+ as a source of innovation and effectiveness – The UK national experience – Institutional experiences (Two case studies) • Innovation culture – Development issues
    • 3. Performance and innovation Is there a connection?
    • 4. Some dichotomies? • Performance or measurement? • Evaluation or innovation? • Indicators or improvements? • Accountability or activity? • Processes or projects? • Equity or equivocation? • Quality or acquiescence?
    • 5. The UK HE Quality & measurement context • Quality as QA – Teaching Quality Assessment • Quality as Peer Review – Research Assessment Exercise • Quality as batteries of performance indicators – Statistical collection and league tables • Quality as culture – TQM, IIP, Charter Mark
    • 6. Measurement Progress (Lancour, 1951) Three Phases of Academic Library development • “Storehouse” • “Service” • “Educational”
    • 7. The University Context Universities have two “bottom lines” 1. Financial (as in business) 2. Academic, largely through reputation in • Research (the priority in leading Universities) • Teaching (& maybe Learning)
    • 8. Library Pressures for Accountability The need is therefore to demonstrate the Library contribution in these two dimensions: 1. Financial, through “value for money” or related measures 2. Impact on research, teaching and learning This also implies that “competitive” data will be highly valued
    • 9. The UK & Ireland Experience The SCONUL Working Group on Performance Improvement • Ten years of “toolkit” development to assist in performance measurement and improvement • SCONUL ‘Top concern survey’ 2005 • The SCONUL Value and Impact Measurement Programme (VAMP)
    • 10. Examples of tools developed 1 • Integration • Efficiency & Comparability Quality assurance Guidelines SCONUL Statistics & interactive service HELMS national performance indicators E-measures project Benchmarking Manual
    • 11. Examples of tools developed 2 • Satisfaction • Impact SCONUL Satisfaction Survey SCONUL LibQUAL+ Consortium LIRG/SCONUL Impact Initiative Information Literacy Success Factors
    • 12. VAMP Objectives • New missing measurement instruments & frameworks • A full coherent framework for performance, improvement and innovation • Persuasive data for University Senior Managers, to prove value, impact, comparability, and worth
    • 13. Missing methods? • An impact tool or tools, for both teaching & learning and research • A robust Value for Money/Economic Impact tool • Staff measures • Process & operational costing tools
    • 14. Member Survey Findings • 38 respondents; 27% of population • 70% undertaken value or impact measurement • Main rationales are advocacy, service improvement, comparison • Half used in-house methodologies; half used standard techniques • Main barrier is lack of tools, making time an issue • Buy-in of stakeholders is an issue
    • 15. Some Conclusions … • There is a need to demonstrate value and that libraries make a difference • Measurement needs to show ‘real’ value • Need to link to University mission • Libraries are, and intend to be, ahead of the game • Impact may be difficult or impossible to measure – A pedagogic project needed? • Measurement is not innovation, and may inhibit it – Re-engineering needed first?
    • 16. Next Steps 1 “Content” Products 2.1 Value & Impact Guidelines 2.1.1 Institutional Value (eg VFM & Economic Impact) 2.1.2 Impact on Teaching & Learning 2.1.3 Impact on Research
    • 17. Next Steps 2 “Content” Products 2.2 Staffing & Operational Measures Guidelines 2.2.1 Staff Costing 2.2.2 Staff Added Value measures 2.2.3 Other operational costing methods 2.3 Re-branding & packaging of existing tools
    • 18. Next Steps 3 “Process” Products 3.1 Web Site 3.2 Community of practice establishment 3.3 Maintenance & sustainability strategy
    • 19. LibQUAL+ for enhancing quality A source of data to support performance and innovation
    • 20. What is LibQUAL+? • A web-based survey tool designed to measure Library quality • Provides comparable data with other institutions to help benchmark services • Provides detailed data to suggest a service improvement agenda, and longitudinal data to test improvement actions
    • 21. LibQUAL+ History • ARL New Measures initiative • Developed by Texas A&M University • Based on SERVQUAL • Piloted in 2000 • Now used by over 850 libraries worldwide
    • 22. LibQUAL+ in Europe • SCONUL (UK & Ireland) – 2003: Pilot with 20 member libraries – 2004: 17 participants – 2005: 17 participants – 2006: 22 participants – 55 different institutions over the 4 years • European Business Schools Librarians’ Group – 2004: Pilot with 5 member libraries – 2006: 12 participants in 7 European countries • National Health Service (UK) – 2006: Pilot with 12 member libraries
    • 23. Benefits of LibQUAL+ • Managed service – for delivery & analysis – cost • Web-based • Gap analysis • Permits benchmarking – Peers, nationally & internationally
    • 24. Time frame • Surveys can be run for a chosen duration in: – Session 1: January – June – Session 2: July – December • January / February – Training for Session 1 Participants – Results meeting for Session 2 Participants • July / August – Training for Session 2 Participants – Results meeting for Session 1 Participants
    • 25. Dimensions of Library Service Quality Empathy Information Control Responsiveness Symbol Utilitarianspace Assurance Scope of Content Ease of Navigation Self -Reliance Library asPlace Library Service Quality Model 3 Refuge Affectof Service Reliability Convenience Timeliness Equipment F. Heath, 2005
    • 26. The Survey Comprises of • 22 Core questions • 5 Local questions (selected by the institution) • 5 Information Literacy questions • 3 General Satisfaction questions • Demographic questions • A free-text comments box
    • 27. Sample Survey
    • 28. How it works • For the 22 “core” questions and 5 “local” questions users rate out of 1 – 9 their: – Minimum service level – Desired service level – Perceived service performance • This gives us a “Zone of Tolerance” for each question, and an “Adequacy Gap”
    • 29. Benefits of gap analysis AverageRating 9.0 8.0 7.0 6.0 5.0 4.0 3.0 2.0 Affect of Service Information Control Library as Place Perceived Range of Minimum to Desired Range of Minimum to Perceived (“Gap”)
    • 30. Comments box • Free-Text comments box at the end of the survey • About 40% of participants provide open-ended comments, and these are linked to demographics and quantitative data • Users elaborate the details of their concerns • Users feel the need to be constructive in their criticisms, and offer specific suggestions for action
    • 31. The SCONUL Experience
    • 32. LibQUAL+ Participants 2003 • University of Bath • Cranfield University • Royal Holloway & Bedford New College • University of Lancaster • University of Wales, Swansea • University of Edinburgh • University of Glasgow • University of Liverpool • University of London Library • University of Oxford • University College Northampton • University of Wales College Newport • University of Gloucestershire • De Montfort University • Leeds Metropolitan University • Liverpool John Moores University • Robert Gordon University • South Bank University • University of the West of England, Bristol • University of Wolverhampton
    • 33. LibQUAL+ Participants 2004 • Brunel University • Loughborough University • University of Strathclyde • University of York • Glasgow University • Sheffield University • Trinity College, Dublin • UMIST + University of Manchester • University of Liverpool • Anglia Polytechnic University • University of Westminster • London South Bank University • Napier University • Queen Margaret University College • University College Worcester • University of East London
    • 34. LibQUAL+ Participants 2005 • University of Exeter • University of Edinburgh • University of Dundee • University of Bath • University of Ulster • University College Northampton • University of Birmingham • Roehampton University • University of Glasgow • University of Surrey • Royal Holloway UoL • City University • Cranfield University • University of Luton • Dublin Institute of Technology • London South Bank University • Coventry University
    • 35. LibQUAL+ Participants 2006 • Cambridge University • Cranfield University • Goldsmiths College • Institute of Education • Institute of Technology Tallaght* • Queen Mary, University of London • Robert Gordon University • St. George's University of London • University of Aberdeen • University College for the Creative Arts • University of Central Lancashire • University of Gloucestershire • University of Leeds • University of Leicester • University of Liverpool • University of the West of England • University of Warwick • University of Westminster • London South Bank University • Scottish Royal Agricultural College • University of Birmingham • University of Glasgow
    • 36. Overall Potential UK Sample to 2006 • Full variety of institutions • 43% of institutions • 38% of HE students (>800,000) • 42% of Libraries • 48% of Library expenditure
    • 37. SCONUL Overall Results 2005
    • 38. Aims & purposes • Analysis compilation • Comparison to existing survey methods • A library focused survey • Benchmarking • Charter Mark application • Strategic planning aid • Real data as opposed to lobbying • To make adjustments where needed • To test improvement • “User satisfaction - as simple as that”
    • 39. Process Feedback • Straightforward • Publicity requires the most effort • Difficulty in obtaining email addresses • Difficulty in obtaining demographic data • Very simple to administer • Results as expected • More in-depth detail obtained • More ‘discriminatory’ than other surveys • Helped to strengthen Library’s case • Comments very specific & helpful
    • 40. Case Studies
    • 41. Cranfield University at DCMT • Cranfield’s Library services at the Defence College of Management & Technology • Contract situation demanding high quality services • Military and civilian education and research in defence, management & technology • About 1000 students, almost all postgraduate and post-experience
    • 42. DCMT Library Surveys • Student perspective (1993) • Exit questionnaires (1994-) • Information Services (Priority Search 1996) • DTC MSc & MA Students (1997) • Researchers Survey (Web based 1998) • SCONUL Survey Pilot (1999) • SCONUL Template (2001) • LibQUAL+ (2003, 2005, 2006)
    • 43. DCMT LibQUAL+ Surveys • 2003, 2005, 2006 • Increasing responses – 11%, 16%, 22% – Year on year 40% up • Increasing comments – 83, 153, 205 (almost 60% of respondents) • Improved performance across three years
    • 44. DCMT Overall 2006
    • 45. Agenda for Action 2003 • Information skills training • Improving staff specialist skills • Access to electronic resources • Customer care to different users
    • 46. DCMT Survey aims for 2005-06 • Test new Library building • Test launch of the new Library Web site • Test maintenance of other progress – Improved capability in data analysis & presentation • Develop a new strategy in line with changing academic needs
    • 47. Changes over three years
    • 48. Internal Benchmarking Affect of Service Information Control Library as Place Overall
    • 49. National Average External Benchmarking -1.6 -1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 Superiority Mean DCMT Superiority Mean -0.07 -0.83 -0.33 -0.41 UK Superiority Mean -0.91 -1.32 -1.49 -1.20 US Superiority Mean -0.73 -1.37 -1.17 -1.09 Affect of Service Information Control Library as Place Overall
    • 50. Peer-to-Peer Benchmarking Dimension Summary 4.00 4.50 5.00 5.50 6.00 6.50 7.00 7.50 8.00 8.50 Affect of Service Affect of Service Information Control Information Control Library as Place Library as Place Overall Overall DCMT Library 2 DCMT Library 2 DCMT Library 2 DCMT Library 2
    • 51. University of Glasgow • Founded in 1451 • Large research-led institution • About 20,000 students in 10 Faculties, and about 6,000 staff • Member of the Russell Group of major UK research-led Universities • Founder member of Universitas 21
    • 52. Survey Participation • Participation in LibQUAL+ 2006 will be Glasgow’s 4th successive year in the SCONUL Consortium • 2006 – 1,535 responses • 2005 – 1,423 responses • 2004 – 2,212 responses, 920 comments • 2003 – 502 responses, 402 comments
    • 53. Aims of Use of the Data • Strategic Service Developments – Data to support service development – Ability to identify where not meeting expectations – Measure if change has met need • Budget Discussions – Data to support bid for increased funding – Data to support case for change in emphasis (towards e-provision) • Marketing Position – Status of the library within the University – Importance of national & international benchmarking
    • 54. LibQUAL+ Outcomes • New Web Services Administrator • Increased opening Hours – Earlier Saturday morning opening – Sunday morning opening – Increased late opening hours (From January 2006 Mon-Thurs 08:00 – 02:00) • Now providing 222,578 seat hours per week
    • 55. Library Refurbishment Programme reinstated at costs in excess of £8 million From: To:
    • 56. Conclusions • LibQUAL+ is now a market leading survey tool for UK & Irish Academic & Research Libraries, and growing use in Europe • Use of the technique can strongly support and help develop an innovative culture, and provide evidence of impact as well as satisfaction • Some significant advantages over other survey methods • Additional support and data analysis is now available in Europe through ARL/Cranfield contract
    • 57. Innovation culture? • Customer Focus – Requires a satisfaction instrument – Requires further analysis & consultation • Systematic improvement – Requires success factors (& measures) for each – Requires project management alongside BAU processes • Total Involvement – Requires not only a quality culture, but also a meta-quality culture in which the culture itself can be assessed – Supported by other techniques eg situational leadership
    • 58. A Quality Maturity Model (Wilson & Town)
    • 59. The Level One Library 5 Optimising 4 Managed 3 Defined 2 Repeatable 1 Initial 2 3 4 5 1 The improvement process is characterised as ad hoc, and occasionally even chaotic. Innovation and evaluation tends to be one-off and success depends on individual efforts and heroics.
    • 60. Level Two 5 Optimising 4 Managed 3 Defined 2 Repeatable 1 Initial 2 3 4 5 1 Basic project management approaches are established to improve performance & service. The necessary discipline is in place to repeat earlier successes on innovation projects.
    • 61. Level Five: an innovative culture 5 Optimising 4 Managed 4 Defined 2 Repeatable 1 Initial 2 3 4 5 1 The Library has a fully defined, organised and managed approach to innovation and improvement, and is able to optimise fully its evaluation and improvement project effort to the benefit of customers. All staff understand and share the appropriate cultural assumptions.
    • 62. Overall Conclusions • There is an overall management challenge in creating an innovative culture for which we still lack a full range of measures • Measures are an essential aid, but not the culture itself • Effectiveness requires proof of impact, which may be impossible to measure • Building an innovative culture is unlikely to result from evaluation alone, but it is a necessary step • Staff development is more important than measurement, but they can be linked
    • 63. LibQUAL+ If you would like to know more about LibQUAL+, or are considering participating as a consortium or independently see: www.libqual.org Or contact: Selena Lock email: s.a.lock@cranfield.ac.uk Telephone: +44 (0) 1793 785561
    • 64. Acknowledgements • SCONUL and its Working Group on Performance Improvement • Selena Lock, R&D Officer, Cranfield University • Jacqui Dowd, Management Information Officer, Glasgow University • Bruce Thompson, Distinguished Professor of Educational Psychology and CEHD Distinguished Research Fellow, and Distinguished Professor of Library Science, Texas A&M University • Association of Research Libraries
    • 65. Seminar Questions? • Do measurement and evaluation guarantee performance or quality? • Does evaluation lead to innovation? • What are the key measures for accountability in an academic context? • What professional development requirements arise from these evaluation methods?

    ×