This document summarizes several initiatives by the Association of Research Libraries (ARL) related to assessing library services. It discusses LibQUAL+, a survey tool used to measure user perceptions of service quality, and how it has been implemented in the UK through SCONUL. It also introduces TechQUAL and ClimateQUAL as additional assessment tools for IT services and organizational culture, respectively, and provides examples of results from trials of these tools in UK universities.

1. LibQUAL+ and other ARL
initiatives
Stephen Town & Selena Killick

2. Summary
• ARL Background & History
• LibQUAL+: the SCONUL experience
• TechQUAL as a method for converged contexts
• ClimateQUAL for staff assessment: the
SCONUL trial
• Discussion

3. ARL Statistics & Assessment
• Leadership role in development, testing and
application of tools for management &
measurement
• Community development: the Library
Assessment Conference, Blog, email list, OPAL
(formerly ESP), Service Quality Evaluation
Academy
• Grew out of the “New Measures Initiative”

4. Tools & projects: StatsQUAL
• ARL Statistics • Lib-Value
• LibQUAL+ • E-Metrics
• ClimateQUAL • Organisational
• DigiQUAL Performance
• MINES for Libraries Assessment

5. LibQUAL+®: The SCONUL Experience
Selena Killick
Library Quality Officer, Cranfield University

7. &

8. Survey Composition
• 22 Core Questions
– Affect of Service
– Information Control
– Library as Place
• 5 Optional Questions
• Information Literacy
• General Satisfaction
• Demographics
• Free-text Comments

9. LibQUAL+ Lite
• Introduced in 2010
• Sampling Methodology
– All questions asked
– Respondents answer a
random sample only
• Reduced Survey Time
• Increase Response
Rates

10. Consortium Composition

11. Number of SCONUL LibQUAL+® Participants by
Year
25
20
15
10
5
0
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012*
LibQUAL+ Participants Unique LibQUAL+ Participants

12. SCONUL LibQUAL+® Libraries by HE Sector /
Sector Population
80
70
60
50
40
30
20
10 55% 33%
57% 7%
0
RLUK Pre-1992 Post-1992 HE colleges
LibQUAL+ Libraries Non-LibQUAL+ Libraries

13. LibQUAL+® Libraries by Mission Group
30
25
20
15
10
60% 68% 44%
5 82%
35%
0
Russell Group 1994 Group University Million+ CONUL
Alliance
LibQUAL+ Libraries Non-LibQUAL+ Libraries

14. SCONUL LibQUAL+® Repeat Participation by Style
Other Annual
18% 12%
Three-yearly
15%
Biennial
55%

15. Expectations

16. Highest Desired
ID Question Desired Years in
(2011) Top 5
Making electronic resources accessible from my
IC-1 home or office 8.30 9
LP-1 Library space that inspires study and learning 8.16 2
A library Web site enabling me to locate
IC-2 information on my own 8.10 9
Making information easily accessible for
IC-7 independent use 8.06 2
Print and/or electronic journal collections I
IC-8 require for my work 8.06 8

17. Lowest Desired
ID Question Desired Years in
(2011) Bottom 5
AS-2 Giving users individual attention 6.77 9
LP-5 Space for group learning and group study 7.32 9
AS-1 Library staff who instill confidence in users 7.49 8
AS-6 Library staff who deal with users in a caring
fashion 7.73 9
AS-7 Library staff who understand the needs of their
users 7.74 2

18. SCONUL LibQUAL+® Increasing Minimum Expectations
7.00
6.80
6.60
6.40
6.20
6.00
5.80
5.60
5.40
5.20
5.00
Space for group Making electronic Library staff who A comfortable and
learning and group resources deal with users in a inviting location
study accessible from my caring fashion
home or office
Minimum Mean 2004 Minimum Mean 2011

19. SCONUL LibQUAL+® Decreasing Desired Expectations
8.20
8.00
7.80
7.60
7.40
7.20
7.00
6.80
6.60
The printed Print and/or Giving users Quiet space for Modern
library materials I electronic journal individual individual work equipment that
need for my work collections I attention lets me easily
require for my access needed
work information
Desired Mean 2004 Desired Mean 2011

20. Perceptions

21. SCONUL Overall 2004

22. SCONUL Overall 2011

23. SCONUL LibQUAL+® Overall 2004 - 2011
8.50
8.00
7.50
Mean
7.00
6.50
6.00
5.50
2004 2005 2006 2007 2008 2009 2010 2011
Minimum Mean Desired Mean Perceived Mean

24. SCONUL LibQUAL+® Affect of Service 2004 - 2011
8.50
8.00
7.50
Mean
7.00
6.50
6.00
5.50
2004 2005 2006 2007 2008 2009 2010 2011
Minimum Mean Desired Mean Perceived Mean

25. SCONUL LibQUAL+® Information Control 2004 - 2011
8.50
8.00
7.50
Mean
7.00
6.50
6.00
5.50
2004 2005 2006 2007 2008 2009 2010 2011
Minimum Mean Desired Mean Perceived Mean

26. SCONUL LibQUAL+® Library as Place 2004 - 2011
8.50
8.00
7.50
Mean
7.00
6.50
6.00
5.50
2004 2005 2006 2007 2008 2009 2010 2011
Minimum Mean Desired Mean Perceived Mean

27. Comparisons

28. LibQUAL+ Desired Mean Comparisons 2011
8.20
8.00
7.80
7.60
7.40
Affect of Service Information Library as Place Overall
Control
SCONUL ARL

29. LibQUAL+ Perceived Mean Comparisons 2011
7.40
7.20
7.00
6.80
6.60
6.40
6.20
Affect of Service Information Library as Place Overall
Control
SCONUL ARL

30. LibQUAL+® Results in Action

31. The Library is one of the most oppressively
depressing locations on the face of the
earth; its ugly décor and offensive lighting
gnaw at the soul and the carpeting is an
invitation to madness
£14 million improvement programme now
underway to renovate the Library, expand the
collection and increase study space.

32. Feedback from SCONUL Participants
• Secured additional funding to increase
resources
• Increased liaison with Academic staff
• Improvements to the Library building &
facilities
• Developed information literacy training
• Increased help and support

33. National Outcomes
Libraries are being transformed into rowdy social
spaces… many of the changes have had unfortunate –
even disastrous – consequences for the place of
libraries in university scholarship and study.
Response published in the next edition of THE from
SCONUL outlining that LibQUAL+® results show that
academic staff perceptions of library buildings are
the highest they’ve ever been.
Sharpe, K. 2009. Quiet, please. Times Higher Education . 5 November. [Available Online].

34. Conclusions
• Greater use of LibQUAL+® by Pre-1992
Universities
• Biennial participation for ~half of users
• Expectations rising in most areas
• Perceptions rising also
• Not quite as good as ARL
• LibQUAL+® helps Libraries to develop and
improve services

35. TechQUAL and ClimateQUAL
Stephen Town
Director Of Information

36. TechQUAL IT Survey

37. Background
• Higher Education IT Survey Tool
• Developed in the US at Pepperdine University
• Based on the same “total market survey”
principles as SERVQual and LibQUAL
• Web based survey, used extensively in the US
(e.g. Boston, Yale)
• Free to use (but not serviced)
• First use in the UK at York 2011

38. The Instrument
• Currently 18 core questions and 6 local
questions, to which users give minimum,
desired and perceived ratings, and comments
on each
• Plans to reduce to 12 core questions
• One additional free text question

39. Example Questions
Connectivity & Access
1) Having adequate capacity (speed, bandwidth) when using the wired
network
2) Having adequate capacity (speed, bandwidth) when using the
wireless network
3) Having wireless network coverage in all the areas that are
important to me as a faculty, student, or staff member
4) Having a university network that is reliable, available, and performs
in an acceptable manner
5) Having access to important university provided technology services
from my mobile device
6) Having access to important university provided technology services
from off campus when at home or travelling

40. York Results
• Conducted between 28th November and 19th
December 2011
– Total respondents = 1,888
– Total response rate = 10.45%
– Total comments received = 2,855
– In comparison, Library survey 2011 received 1,865
responses

41. York Results

42. Sample Comments
• “Mac OSX is becoming increasingly prevalent
especially for laptops - staff should be well trained
for it.”
• “Web mail :( but gmail :-) and we are switching -
as have many many staff already.”
• “Main qualm here is with printing, the printers
often break/get jammed and it takes an age to fix
them.”
• “I'd like a York University app, where I can access
my timetable, yorkmail and contact details for
seminar tutors etc.”

43. Conclusions
• TechQUAL offers an acceptable tool for UK audiences
for IT service quality assessment
• TechQUAL provides focused, quantitative and
benchmarkable data on IT service quality
• TechQUAL is designed to provide specific commentary
• TechQUAL is very suitable for converged services to use
in tandem with LibQUAL+
• TechQUAL is not a serviced product
• At York has already had a significant impact on
resource decision-making, especially around wireless
provision, and has provided core data for the new
Information Strategy

44. ClimateQUAL: Organisational Culture and
Diversity Assessment Tool

45. Background
• Assessment tool developed for understanding
culture and diversity within libraries
• Developed by Organisational Psychologists at the
University of Maryland, administered by ARL
• Used by over 30 universities in the US
• Allows benchmarking against other universities
• Tailored for the UK market and being piloted with
York, Leicester, Nottingham Trent and UWE

46. Survey
• Based around 9 core scales:
– Organizational Climate for Justice
– Climate for Leadership
– Climate for Deep Diversity
– Climate for Demographic Diversity
– Climate for Innovation: Co-Workers
– Climate for Continual Learning
– Climate for Teamwork
– Climate for Customer Service
– Climate for Psychological Safety

47. Sample Questions
The following questions are designed to be answered on a 7-point (strongly disagree
to strongly agree) scale and are taken from a range of scales:
• This organization provides a clear understanding of the purpose of teams.
• I talk up this organization to my friends as a great organization to work for.
• My immediate supervisor has excellent interpersonal skills.
• The race of a team/division member does NOT affect how much attention is paid
to their opinions.
• Coworkers are able to provide reliable information about ways to improve job
performance.
• Library employees have the job knowledge and skills required to deliver superior
quality work and service.
• The work I do is very important to me.
• My impact on what happens in my team/division is large.

48. York Experience
• Ran the survey over a 3-week period in May/June 2012
• Promoted using emails directly to staff and managers
• Used incentive of Kindle Touch and £50 Amazon vouchers
• Over 100 staff completed the survey, out of a target of around
120 FTE
• Results confidential ( and not available yet!) but can be
benchmarked

49. General conclusions & questions
• ARL assessment products have been
successfully deployed in the UK & Ireland for
close to ten years (as well as internationally)
• Culture and language differences have not
been a significant barrier to take up
• Are there further opportunities for
collaboration?

50. Acknowledgements
• Amanda Conway, Dorothy Vuong, Karen
Ready, Jacqui Dowd, John MacColl and the
rest of the SCONUL LibQUAL+® Participants
• The ARL LibQUAL+® Participants
• SCONUL Working Group on Performance and
Quality
• Martha, Bruce, Colleen, David and the rest of
the LibQUAL+® team

51. Thank You!
Selena Killick
s.a.killick@cranfield.ac.uk
Twitter: @SelenaKillick
Tel: +44(0)1793 785561
J. Stephen Town
Director of Information, The University of York
stephen.town@york.ac.uk
http://www.arl.org/stats/