Your SlideShare is downloading. ×
0
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Sciences Po presentation eng

383

Published on

by Selena Killick

by Selena Killick

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
383
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • 1st Group – Look how large “Study” is in this word cloud2nd Group - Verbs
  • Once you have identified your peers, access their LibQUAL+® results in Analytics
  • Transcript

    • 1. Analysis and Interpretation of the LibQUAL+® Results Selena Killick Association of Research Libraries Cranfield University SciencesPo Paris, France 26th June 2012 www.libqual.org
    • 2. Objectives• An introduction to the LibQUAL+® results• Analyzing and interpreting results; – Internally – Externally – Longitudinally• Interpret the data and identify actionable items• Qualitative analysis of LibQUAL+® comments• How to conduct analysis on your survey results using LibQUAL+® Analytics• How to analyse Consortium results and compare them with local results• French specificities
    • 3. Programme• Understanding the LibQUAL+ Survey Results• Internal analysis: Interpretation of results and identifying actionable items• Internal analysis: Qualitative analysis of the LibQUAL+® comments• External benchmarking: How are we doing compared to the consortium?• External benchmarking: Identifying best practice• Action planning for change• Communicating your results to stakeholders• Longitudinal analysis: Assessing impact of change www.libqual.org
    • 4. Understanding the LibQUAL+® Survey Results www.libqual.org
    • 5. Data Repository
    • 6. Results Notebooks• Sections for Overall, Undergraduates, Graduates, Faculty, Staff, Library Staff include: – Demographic Summary – Core Questions Summary – Dimensions Summary – Local Questions – General Satisfaction Questions – Information Literacy Outcomes Questions – Library Use Summary• Appendix describing changes in the dimensions and the questions included in each dimension. www.libqual.org
    • 7. Results Notebook Demographic Summary www.libqual.org
    • 8. Results Notebook Core Question Summary www.libqual.org
    • 9. Results Notebook Core Question Radar Chart www.libqual.org
    • 10. Results Notebook Dimension SummaryKey Term:Zone of Tolerance www.libqual.org
    • 11. Results Notebook Local Questions Summary www.libqual.org
    • 12. Results NotebookSatisfaction & Outcome Questions Summary www.libqual.org
    • 13. Results Notebook Library Use Summary www.libqual.org
    • 14. Data in Electronic Format• LibQUAL+® Analytics• Raw data files• SPSS
    • 15. LibQUAL+® Analytics• Access to all institutional results• Breakdown the data by: – User group – Standard discipline• Export: – Radar charts – Data tables – Thermometer charts• Very easy to use 
    • 16. LibQUAL+® Analytics
    • 17. Data Selection
    • 18. Representativeness
    • 19. Core Questions
    • 20. Thermometer Chart & Library Use Summary
    • 21. Raw Data Files In Excel and SPSS Format Data Repository Link• You will receive access to your complete raw survey data in Excel format• You will also receive an SPSS syntax file that you can apply to the Excel file to analyze your data in SPSS• Instructions on how to create your own SPSS file are available in the Data Repository www.libqual.org
    • 22. SPSS Data Files Statistical Package for the Social Sciences• More complex analysis than Excel in an easier way – Excel can do a lot of things that SPSS can do but not everything• Can be complex to use www.libqual.org
    • 23. SPSS Data Files Analysis Possibilities:• Customized Discipline means and SD• Local question means and SD• Branch Library means and SD• Relationship between different variables• Validity and reliability analysis
    • 24. SPSS Options• Invest in training on SPSS for Library Staff• Work in partnership with colleagues in your institution who use SPSS already• Commission research from your students looking to use data in their studies• Commission LibQUAL+® to conduct the analysis for you www.libqual.org
    • 25. You Too Can Chart LQ Data! www.libqual.org
    • 26. Understanding Disciplines Standard Disciplines Custom Disciplines• LibQUAL+® provides standard • You may choose to create your discipline options that you can own discipline categories, customize to your institution‟s using your local terminology. If disciplines or other affiliations. you choose to add your own categories, each MUST be mapped to a LibQUAL+® standard discipline for data analysis purposes. (Please make sure your new term(s) relate to the standard disciplines. www.libqual.org
    • 27. Representativeness - Standard Disciplines An American Academic Library Respondents Population www.libqual.org
    • 28. Representativeness - Customized Disciplines An Academic Library Respondents Population www.libqual.org
    • 29. Before we go on…These LQ Constructs can be Challenging! • Means and Standard Deviation • The Zone of Tolerance • Radar Charts • Standard vs. Customized Disciplines • Representativeness www.libqual.org
    • 30. Understanding Means & Standard Deviation• Mean = arithmetic average – A measure of central tendency – Takes into account all scores – Sensitive to all values and affected by extreme scores• Standard Deviation = average distance – A measure of dispersion – Takes into account all scores – Sensitive to all values and affected by extreme scores www.libqual.org
    • 31. Understanding the Zone of Tolerance• For the 22 items LibQUAL+® asks users‟ to rate their – Minimum service level – Desired service level – Perceived service performance•This gives us a „Zone of Tolerance‟ for each question; the distance between minimally acceptable and desired service ratings• Perception ratings ideally fall within the Zone of Tolerance www.libqual.org
    • 32. Understanding the Zone of Tolerance Zone of Tolerance www.libqual.org
    • 33. Understanding Radar Charts www.libqual.org
    • 34. Understanding Radar Charts www.libqual.org
    • 35. What Do the Colors Mean?ExceedingDesiredExpectations Green And Blue www.libqual.org
    • 36. What Do the Colors Mean? RedNot MeetingMinimumExpectations www.libqual.org
    • 37. What Do the Colors Mean? Little YellowMeeting DesiredExpectations www.libqual.org
    • 38. What Do the Colors Mean? A lot of YellowExceeding MinimumExpectations, Just! www.libqual.org
    • 39. Understanding Thermometer Charts Perceived Perceived
    • 40. Internal AnalysisInterpretation of results www.libqual.org
    • 41. Interpreting Results How do users rate the Library? What „scores‟ do users give the Library‟s performance? Note: This is the simplest analysis and does not take advantage of the “gap analysis” opportunity
    • 42. Interpreting Results Identify areas where the user community thinks the Library is performing “best” and “worst”
    • 43. Minimum Desired Perceived Adequacy SuperiorityID Question Text Mean Mean Mean Mean MeanAffect of ServiceAS-1 Library staff who instill confidence in users 5.90 7.48 6.66 0.76 -0.82AS-2 Giving users individual attention 5.43 6.84 6.15 0.72 -0.69AS-3 Library staff who are consistently courteous 6.63 7.88 7.23 0.60 -0.65AS-4 Readiness to respond to users enquiries 6.65 7.88 7.17 0.52 -0.71AS-5 Library staff who have the knowledge to answer user questions 6.61 7.84 7.21 0.60 -0.63AS-6 Library staff who deal with users in a caring fashion 6.43 7.79 7.20 0.77 -0.59AS-7 Library staff who understand the needs of their users 6.53 7.78 6.99 0.46 -0.79AS-8 Willingness to help users 6.60 7.84 7.14 0.54 -0.70AS-9 Dependability in handling users service problems 6.53 7.73 6.89 0.36 -0.84Information ControlIC-1 Making electronic resources accessible from my home or office 6.81 8.27 6.85 0.04 -1.42IC-2 A library Web site enabling me to locate information on my own 6.77 8.09 6.95 0.18 -1.14IC-3 The printed library materials I need for my work 6.50 7.83 6.54 0.04 -1.29IC-4 The electronic information resources I need 6.41 7.98 6.71 0.30 -1.27IC-5 Modern equipment that lets me easily access needed information 6.68 7.99 6.93 0.25 -1.06IC-6 Easy-to-use access tools that allow me to find things on my own 6.66 8.01 6.92 0.26 -1.09IC-7 Making information easily accessible for independent use 6.72 8.02 6.98 0.26 -1.04IC-8 Print and/or electronic journal collections I require for my work 6.83 8.10 6.73 -0.10 -1.37Library as PlaceLP-1 Library space that inspires study and learning 6.12 7.77 6.15 0.03 -1.62LP-2 Quiet space for individual work 6.75 8.04 6.29 -0.46 -1.75LP-3 A comfortable and inviting location 6.35 7.82 6.77 0.42 -1.05LP-4 A haven for study, learning, or research 6.53 7.99 6.37 -0.16 -1.62LP-5 Space for group learning and group study 5.77 7.23 6.53 0.76 -0.70Overall 6.46 7.83 6.79 0.33 -1.04
    • 44. Perceived Mean Scores 9.00 8.00 BestMean 7.00 6.00 Worst Worst 5.00 AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5 Question www.libqual.org
    • 45. The Importance of the Zone of Tolerance 9.00 Best: Closest to Desired Mean 8.00Mean 7.00 6.00 Lowest Perceived & Lowest Desired Worst: Furthest from Minimum Mean 5.00 AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5 Question www.libqual.org
    • 46. Minimum Desired Perceived Adequacy SuperiorityID Question Text Mean Mean Mean Mean MeanAffect of ServiceAS-1 Library staff who instill confidence in users 5.90 7.48 6.66 0.76 -0.82AS-2 Giving users individual attention 5.43 6.84 6.15 0.72 -0.69AS-3 Library staff who are consistently courteous 6.63 7.88 7.23 0.60 -0.65AS-4 Readiness to respond to users enquiries 6.65 7.88 7.17 0.52 -0.71AS-5 Library staff who have the knowledge to answer user questions 6.61 7.84 7.21 0.60 -0.63AS-6 Library staff who deal with users in a caring fashion 6.43 7.79 7.20 0.77 -0.59AS-7 Library staff who understand the needs of their users 6.53 7.78 6.99 0.46 -0.79AS-8 Willingness to help users 6.60 7.84 7.14 0.54 -0.70AS-9 Dependability in handling users service problems 6.53 7.73 6.89 0.36 -0.84Information ControlIC-1 Making electronic resources accessible from my home or office 6.81 8.27 6.85 0.04 -1.42IC-2 A library Web site enabling me to locate information on my own 6.77 8.09 6.95 0.18 -1.14IC-3 The printed library materials I need for my work 6.50 7.83 6.54 0.04 -1.29IC-4 The electronic information resources I need 6.41 7.98 6.71 0.30 -1.27IC-5 Modern equipment that lets me easily access needed information 6.68 7.99 6.93 0.25 -1.06IC-6 Easy-to-use access tools that allow me to find things on my own 6.66 8.01 6.92 0.26 -1.09IC-7 Making information easily accessible for independent use 6.72 8.02 6.98 0.26 -1.04IC-8 Print and/or electronic journal collections I require for my work 6.83 8.10 6.73 -0.10 -1.37Library as PlaceLP-1 Library space that inspires study and learning 6.12 7.77 6.15 0.03 -1.62LP-2 Quiet space for individual work 6.75 8.04 6.29 -0.46 -1.75LP-3 A comfortable and inviting location 6.35 7.82 6.77 0.42 -1.05LP-4 A haven for study, learning, or research 6.53 7.99 6.37 -0.16 -1.62LP-5 Space for group learning and group study 5.77 7.23 6.53 0.76 -0.70Overall 6.46 7.83 6.79 0.33 -1.04
    • 47. Best & Worst• The highest Adequacy & Superiority Mean scores indicate the best performance• The lowest Adequacy & Superiority Mean scores indicate the worst performance• Highest Adequacy & Highest Superiority may not be the same item• Lowest Adequacy & Lowest Superiority may not be the same item www.libqual.org
    • 48. Quick Tip• Use LibQUAL+® Analytics to export your data into Excel• Use Data Sort to quickly highlight your highest and lowest scores www.libqual.org
    • 49. Interpreting Results Identify areas where the user community has the highest “wants” www.libqual.org
    • 50. Highest Wants Example Library Core Summary - 20099.008.007.006.005.00 AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5 2009 Identify users‟ highest “wants” www.libqual.org
    • 51. Interpreting Results Identify areas where the user community thinks the Library is “furthest from meeting minimum needs” www.libqual.org
    • 52. Lowest Adequacy Mean Example Library Core Summary - 2009 9.00 8.00 7.00 6.00 5.00 AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5 2009Identify areas where the Libraries are furthest from meeting users‟ minimum “needs” www.libqual.org
    • 53. Review Dimension Summary Example Library Dimension Summary - 2009987 Z.O.T. Per65 Affect of Service Information Control Library as Place 2009 Overall (Undergraduates, Graduates, & Faculty) www.libqual.org
    • 54. Interpreting Results Interpretation Frameworkscan also be applied to data gathered by separate User Groups, by Discipline affiliation and by primary library (Branch) use www.libqual.org
    • 55. Expectations of Postgraduates Example Library Core Summary by User Group (Grad Students) 9 8 7 6 5 4 AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5 2009 Graduates Identify highest “wants” for Grad StudentsIdentify areas where the Libraries are furthest from meeting Grad minimum “needs” www.libqual.org
    • 56. Explore Items by Subject Discipline Example Library Single Core Item by Discipline 9 8 7 6 5 Z.O.T. Law Humanities Other Education Architecture Performing & Fine Arts Business Undecided General Studies Communications / Journalism Health Sciences Social Sciences / Psychology Military / Naval Science Science / Math Engineering / Computer Science Agriculture / Environmental Studies Per IC-4 The electronic information resources I need Identify disciplines with highest “wants” for e-resourcesIdentify where the e-resources are furthest from meeting discipline minimum “needs” www.libqual.org
    • 57. Internal AnalysisIdentifying actionable items www.libqual.org
    • 58. Questions You Need Answering• Which items clearly stand out as the top wants/priorities of your users?• Which items clearly stand out and the lowest wants/priorities of your users?• Which items clearly stand out as the top weaknesses for your users?• Which items clearly stand out and the strengths for your users?
    • 59. Identifying Actionable Items Cross Tabulate Desired & Adequacy Mean Scores Highest Desired Lowest HighestAdequacy Adequacy Lowest Desired www.libqual.org
    • 60. Identifying Actionable Items #1 #3 MOST DESIRED MOST DESIRED and and MOST ADEQUATE LEAST ADEQUATE (lowest weakness/ (highest weakness) highest strength)ADEQUACY #2 #4 LEAST ADEQUATE MOST ADEQUATE (highest weakness) (lowest weakness/ and highest strength) LEAST DESIRED and DESIRED LEAST DESIRED www.libqual.org
    • 61. Identifying Actionable Items #1 #3 MOST DESIRED MOST DESIRED and and MOST ADEQUATE LEAST ADEQUATE (lowest weakness/ (highest weakness) highest strength)ADEQUACY #2 #4 LEAST ADEQUATE MOST ADEQUATE (highest weakness) (lowest weakness/ and highest strength) LEAST DESIRED and DESIRED LEAST DESIRED www.libqual.org
    • 62. Remember LibQUAL+® Analytics
    • 63. Identifying Actionable Items
    • 64. Considerations• Do all user groups have the same wants/priorities, or are there differences?• Plot graphs for different – user groups – disciplines – branch libraries• If you added local questions, included them in your graphs. www.libqual.org
    • 65. Internal AnalysisQualitative analysis of the LibQUAL+® comments www.libqual.org
    • 66. Qualitative Analysis: User Comments• Why the Box is so Important: – About half of participants provide open-ended comments, and these are linked to demographics and quantitative data – Users elaborate the details of their concerns – Users feel the need to be constructive in their criticisms & offer specific suggestions for action• User Comments available on the LibQUAL+® Web site – Download comments in Excel or text file – Skim the comments• Conduct analysis www.libqual.org
    • 67. Comments File Available from: Data Repository linkPlus, real-time access to the comments during Stage 2) Monitor Survey Progress Basic demographic information such as user group, age, sex, and library branch (if available) is provided with each comment. Comments are also tagged with a unique identification number that enables you to link each comment to the individual‟s survey response. Comments are not visible to other institutions participating in LibQUAL+®; only your institution has access to your comments. www.libqual.org
    • 68. Two Case Studies• Texas A&M University – Word clouds – Atlas.TI• Brown University – NVivo
    • 69. Texas A&M: Analysis of Undergraduate Comments Source: Colleen Cook, Presented at QQML 2009 in Chania http://www.wordle.com/ www.libqual.org
    • 70. www.libqual.orgLoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:AdminColleenServQual InterviewsTEXT Only01xxxxxxxxx.txt (redirected: c:zzatlastifred
    • 71. www.libqual.org
    • 72. Methodology for Coding Qualitative Datahttp://www.libqual.org/documents/admin/BrownU_2005_LQ_qual_method.pdf www.libqual.org
    • 73. Methodology for Coding Qualitative Data http://www.libqual.org/documents/admin/BrownU_2005_LQ_qual_method.pdf • Use NVivo software to code & analyze text • Review the comments as a whole • Create a taxonomy – a master list of themes (referred to as “nodes” by NVivo) and the specific keywords mentioned by users in the survey comments for each of the nodes – can generate a word frequency list from the comments file to facilitate the creation of the taxonomy • Import LQ Comments into NVivo • Run reports to assign nodes to comments based on the taxonomy/ keywords • Review the assigned nodes for each comment, making necessary corrections & additions • Evaluate the qualitative data for frequency statistics & analyses of sub-sets of the comments www.libqual.org
    • 74. Brown U: Methodology for Coding Qualitative DataThe following table shows the total distribution of all 4,197 individual comments (or “tags”) according to the 29 topics identified in the taxonomy. Topic (“Tag”) Number of comments with this tag Percentage of all comments Negative 479 59% Suggestion 465 58% Positive 321 40% Use 320 40% Location 290 36% Collection 264 33% Policies 233 29% Ambiance 226 28% Customer Service 218 27% Online content 176 22% Hours 156 19% Furnishings 134 16% Ease of Use 109 13% ILL 81 10% Web site 77 9% Quotable 68 8% Computer Equipment 64 8% Lighting 62 7% Comparison 61 7% Book Availability 56 7% Non-computer equipment 55 6% Catalog 53 6% Survey 49 6% Training 48 6% Off campus 43 5% Temperature 33 4% Named Staff 28 3% Financial www.libqual.org 26 3%
    • 75. External BenchmarkingHow are we doing compared to the consortium? www.libqual.org
    • 76. Consortium Results• PDF Notebook in Data Repository• Provides average results for all consortium members• Benchmarkable
    • 77. Questions to Ask• How do our results compare to the consortium?• Do our users have higher or lower minimum and desired means?• Do our users have higher or lower perceptions?• Are our top priorities (highest desires) the same as everyone else?
    • 78. Quick Tip• Consortium Notebooks in Excel Format – Free Online PDF to Excel convertor: http://www.pdftoexcelonline.com/• Not perfect, will require some checking of the data• Could save you data input time
    • 79. Consortium & Institution Comparisons Affect of Service Affect of Service 9 8 7Mean Score 6 5 4 AS-1 Average UniA Average UniA Average UniA Average UniA Average UniA Average UniA Average UniA Average UniA Average UniA AS-1 AS-2 AS-2 AS-3 AS-3 AS-4 AS-4 AS-5 AS-5 AS-6 AS-6 AS-7 AS-7 AS-8 AS-8 AS-9 AS-9 Question
    • 80. Consortium & Institution Comparisons Information Control Information Control 9 8 7Mean Score 6 5 4 IC-1 IC-1 Uni X IC-2 IC-2 Uni X IC-3 IC-3 Uni X IC-4 IC-4 Uni X IC-5 IC-5 Uni X IC-6 IC-6 Uni X IC-7 IC-7 Uni X IC-8 IC-8 Uni X Average Average Average Average Average Average Average Average Question
    • 81. Considerations• Benchmarking against the consortium will help you put your results in context• Review the individual consortium members listed in the notebook – Should your performance be the same as these institutions? – If not, who should you benchmark against? – Create your own peer-group if necessary
    • 82. External Benchmarking Identifying best practice www.libqual.org
    • 83. Peer Comparisons• How do I select peers? – Listen, talk to, or search web sites of your University Office of Institutional Research, Provost, President – Consortium members – Descriptive library statistics – Type of institution – Size of the faculty, student body (in specific disciplines)• Peer Group and/or Individual Institutions www.libqual.org
    • 84. LibQUAL+® Directory
    • 85. Summary Statistics in the Data Repository
    • 86. Summary Statistics in the Data Repository continued
    • 87. Summary Statistics in Notebook
    • 88. Remember: We assess to improve... ....not to proveInstitutions should NOT use other libraries data in ANY WAY that would compromiseand harm the reputation of other institutions. Institutions may use other libraries datain a confidential manner without disclosing the institutional identity of other libraries.
    • 89. Peer Comparisons: General Satisfaction Peer Comparison General Satisfaction *Data taken from last year of participation9876 H F A C B E LIBRARY G D X In general, I am satisfied with the way in which I am treated at the library. www.libqual.org
    • 90. Peer Comparisons: General Satisfaction Peer Comparison General Satisfaction *Data taken from last year of participation9876 H A C E F G LIBRARY D B X In general, I am satisfied with library support for my learning, research, and/or teaching needs. www.libqual.org
    • 91. Peer Comparisons: General Satisfaction Peer Comparison General Satisfaction *Data taken from last year of participation9876 H A C F B E G LIBRARY D X How would you rate the overall quality of the service provided by the library? Library‟s conclusion: There‟s still room for improvement! www.libqual.org
    • 92. Peer Comparisons: Focus on One Question LibQUAL+ 2006 Faculty Ratings of Journal Collections ARL Libraries 9.00 UVA 8.00 7.00 6.00 Top of Blue Bar = Desired Level of Service Bottom of Blue Bar = Minimum Level of Service Red Square = Perceived Service Performance 5.00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37Source: Jim Self, University of Virginia, Presented at Performance Measurement in Academic Libraries Workshop, EBLIP4, Durham, NorthCarolina, May 11, 2007 www.libqual.org
    • 93. Re m ot e ac ce Li ss br to ar el y ec w tr o eb ni si te c al re lo so -0.40 -0.20 0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 w ur in ce El ec Pr g s tro in au tl ib to ni no c ra m In in ry y fo m fo rm at rm at er at io ia io n ls n M re ea od so si Ea ur ly sy er n ce Pr ac -to eq s in ce -u ui ta ss se pm nd ib /o le ac en re fo ce t le ri nd ss ct to Em ro ep ol pl ni c en s oy jo de ee ur nt Em G s na us pl iv wh lc e oy in o ol le ee g in ct s us st io Re wh er ill ns o s co ad in nf in ar e di id es vi en s co du ce to ns al re is at sp te te on nt nt d ly io to co n us ur Kn te ow er ou Em le s s pl dg qu oy ab es ee le t io s em ns wh o Ca pl oy un rin g Adequacy Gap de ee Li rs em s br ta pl Wwww.libqual.org ar nd oy y De pe illi us ee sp ac nd ng er s ne s e ab ss ne th ilit at y to ed in in he s A sp ha lp The difference between the minimum and perceived score. ire nd us qu s lin er ie te g s ts ac pr pa hi ob A A ce ng le co fo an m ge m ri d s ta fo nd le wa rta ivi ar y bl du ni fo e al ng rs an tu d ac dy in t iv vi Adequacy Gap, All Questions ,l itie tin s ea rn g lo in ca g, t io Peer Group Comparisons: G or n ro re up se ar st ch ud y sp ac O e VE Source: Fred Heath, LibQUAL+™ Results Meeting, ALA Annual Conference, Seattle, WA, January 22, 2007 RA LL ARL Peers UT Austin
    • 94. Action Planning For Change www.libqual.org
    • 95. Cycle of Planning and Assessment www.libqual.org
    • 96. From Analysis to Action• From all of the data, determine what can and should be addressed• Prioritize some action items – Align with mission, vision and goals of parent organization – Address users‟ top priorities, by user group – Improve areas of strong user dissatisfaction – Build on strengths, if they are truly user needs and priorities – Identify work that can be de-emphasized and resources that can be reallocated www.libqual.org
    • 97. Pulling it all together• What are your actionable items?• How do these compare to: – Consortium – Peer Group – Individual institutions• Review the free-text comments relate to your actionable items – Coded – Linked to responses
    • 98. Focused Follow-up University of VA:• Who is unhappy? – Drilling down by college and discipline• Why are they unhappy? – Reading the comments – Conducting targeted interviews• Focus on areas with low scores• Diverse group of faculty• Asked for specific needs and wants – Including names of needed titles• Quick interviews Source: Jim Self, University of Virginia www.libqual.org
    • 99. Focused Follow-up University of VA:• Is the Library meeting your minimal level, regarding journal collections? – If not, what can we do?• Is the Library meeting your desired level? – If not, what can we do?• Does it matter if journals are print or electronic?• Any other comments about the Library? Source: Jim Self, University of Virginia www.libqual.org
    • 100. Identifying & Using Best Practices• Focus on your top actionable items• Who is performing better than you?• What are they doing that you could learn from?• Contact each other to discuss & share ideas for service improvements• Implementing improvements based upon best practices
    • 101. Communicating Results to Stakeholders www.libqual.org
    • 102. Why?• Communicating results effectively is vital for implementing actions• Goals, priorities, and resource allocation should be influenced by customer needs and wants• Decision makers, key stakeholders & budget holders need to understand the results
    • 103. Engaging Library Staff in Understanding & Using Data• Spread knowledge about LibQUAL+® to Library staff – Post Notebook on library web – Make staff aware of LQ tutorial: • http://www.libqual.org/Information/Tools/index.cfm – All-staff presentation – Enable key staff to access LibQUAL+® Analytics – Disseminate Comments to depts, units, groups, branches, librarians, etc.• Develop a culture of assessment and accountability for listening to customers and acting on user feedback www.libqual.org
    • 104. Overcoming Resistance“We know what’s best” “...only customers judge quality;“They are wrong” all other judgments are essentially irrelevant.”“We don’t have the resources to…” Zeithaml, Parasuraman, Berry.(1999). Delivering Quality Service. NY: The Free Press www.libqual.org
    • 105. Northeastern University Case Study“Comments were sorted by academic department, which were postedalong with department-specific radar charts and gap summaries, inthe Research & Instruction Department.The “Great Wall of LibQUAL+®” provoked discussion, and providedcollection managers with insight into the concerns of the faculty andstudents in the departments they were serving. www.libqual.org
    • 106. Presenting Results to Stakeholders• Identify all of the stakeholders or constituents who want and need to know about the survey results• Consider the “stake” of each of the above; what specific aspect of LibQUAL+® will be of most interest / concern• Determine how to communicate with each identified stakeholder• Speak in terms they will understand• Present data graphically in a format they can interpret quickly & easily www.libqual.org
    • 107. Communicate with your Customers (students, faculty, others)• Particularly those whom you asked to participate in the survey• Announce incentive award winners• Inform users of highlights of survey results• Focused follow-up for more specific insights• Most importantly, what the library intends to do “You asked for it…” www.libqual.org
    • 108. KU Libraries: We Listened to You! LibQUAL+ 2006Thanks to the more than 1,100 KU faculty, staff and students who completed the 2006LibQUAL+ survey, KU Libraries has made significant changes over the past few months tobetter meet your research and service needs.You requested:•Access to the electronic resources from your home or office •Print and/or electronic journal collections you require for your work •A Libraries Web site that enables you to locate information on your own more quickly and easily •Librarians and staff members who have the knowledge to answer your questions •Dependability in handling your service problemsWe delivered:•More access to print and electronic materials, including 30,000 journals and many otherprimary resources •The new Information Gateway, a primary tool for searching the Libraries proprietary online resources including databases, journals and images •A newly redesigned Web site •Access to electronic records for hundreds of thousands of previously inaccessible items•An ongoing commitment to enhancing service quality through comprehensive training andcontinuous evaluation www.libqual.org www.libqual.org
    • 109. Longitudinal Analysis:Assessing impact of change www.libqual.org
    • 110. Cycle of Planning and Assessment www.libqual.org
    • 111. Longitudinal Analysis Benchmarking Against Self, Longitudinally“Nobody is more like me than me!” --Anonymous www.libqual.org
    • 112. Longitudinal Analysis• An interpretation framework that allows you to compare performance over time• Are my scores increasing or decreasing?• Are my scores increasing or decreasing for specific subgroups of my population: faculty, grad, undergraduates, and/or disciplines or branches?• Have my actions affected my scores? www.libqual.org
    • 113. Longitudinal AnalysisExample Library Satisfaction (All Users, 2004 – 2008) General Satisfaction Overall 2004 to 2008 *All user groups (excluding Library Staff)9876 20045 20084321 In general, I am satisfied In general, I am satisfied How would you rate the with the way in which I am with library support for my overall quality of the treated at the library. learning, research, and/or service provided by the teaching needs. library? Increase in satisfaction in all 3 satisfaction measures for Total Population www.libqual.org
    • 114. Longitudinal Analysis Example Library Satisfaction (Faculty, 2004 – 2008) General Satisfaction Faculty 2004 to 2008 *All user groups (excluding Library Staff)9876 20045 20084321 In general, I am satisfied In general, I am satisfied How would you rate the with the way in which I am with library support for my overall quality of the treated at the library. learning, research, and/or service provided by the teaching needs. library? Increase in satisfaction in all 3 satisfaction measures for Faculty www.libqual.org
    • 115. Longitudinal AnalysisExample Library Satisfaction (Undergrads, 2004 – 2008) General Satisfaction Undergraduates 2004 to 2008 *All user groups (excluding Library Staff) 9 8 7 6 2004 5 2008 4 3 2 1 In general, I am satisfied In general, I am satisfied How would you rate the with the way in which I am with library support for my overall quality of the treated at the library. learning, research, and/or service provided by the teaching needs. library? Unchanged or decrease in satisfaction in all 3 satisfaction measures for Undergraduates www.libqual.org
    • 116. Z.O.T. + Longitudinal Example Library Changing Expectations (Faculty, 2003 – 2009)98765 2003 2006 2009 2003 2006 2009 2003 2006 2009 Affect of Service Information Control Library as Place Space and place needs are Faculty tolerance is narrowing: increasing over time, though their minimum needs are higher overall remain less critical for but their desires are stable faculty than other issues www.libqual.org
    • 117. Z.O.T. + LongitudinalExample Library Changing Expectations & Performance (Faculty, 2003 – 2009) 9 8 7 6 5 2003 2006 2009 2003 2006 2009 2003 2006 2009 Affect of Service Information Control Library as Place Wow, customer service for faculty Ouch, even though this Library This Library seems to be keeping is improving even while some improved slightly, the ability to pace with increasing faculty expectations are increasing! meet faculty needs for information expectations about library spaces and access is not keeping pace with expectations www.libqual.org
    • 118. Assessing Impact of Actions Explore Specific Questions IC-1 Making electronic resources accessible from my home or office Library as Place9.00 9.008.50 8.508.00 8.007.50 7.507.00 7.006.50 6.506.00 6.00 5.505.50 5.005.00 4.504.50 4.004.00 2005 2006 2007 2008 2009 2010 2011 2005 2006 2007 2008 2009 2010 2011 Minimum Mean Desired Mean Perceived Mean
    • 119. Assess Impact of Actions Have results improved?• Yes – Communicate to Library Staff, Key Stakeholders, the wider University community• No – Do you know why? – Explore further with your customers – What can you do to improve?
    • 120. Summing Up www.libqual.org
    • 121. Review• Understanding the LibQUAL+® Survey Results• Internal analysis: Interpretation of results and identifying actionable items• Internal analysis: Qualitative analysis of the LibQUAL+® comments• External benchmarking: How are we doing compared to the consortium?• External benchmarking: Identifying best practice• Action planning for change• Communicating your results to stakeholders• Longitudinal analysis: Assessing impact of change www.libqual.org
    • 122. LibQUAL+® Resources• LibQUAL+® Web site: http://www.libqual.org/home• Publications: http://www.libqual.org/Publications.aspx• Events and Training: http://www.libqual.org/events• Gap Theory/Radar Graph Introduction: http://libqual.org/about/about_survey/tools• LibQUAL+® Procedures Manual: http://www.libqual.org/documents/LibQual/publications/ProceduresMan ual.pdf• ARL YouTube Channel: http://www.youtube.com/user/arladmin?feature=watch www.libqual.org
    • 123. LibQUAL+® Team• Martha Kyrillidou - Senior Director, ARL Statistics and Service Quality Programs martha@arl.org• David Green - Library Relations Coordinator libqual@arl.org• Selena Killick – European Support s.a.killick@cranfield.ac.uk• And sometimes, consultant Raynna Bowlby raynna.bowlby@charter.net www.libqual.org
    • 124. Thank youSelena KillickLibQUAL+® European Support Officers.a.killick@cranfield.ac.uk+44 1793 785561 www.libqual.org

    ×