Analysis and Interpretation of the      LibQUAL+® Results              Selena Killick     Association of Research Librarie...
Objectives• An introduction to the LibQUAL+® results• Analyzing and interpreting results;  – Internally  – Externally  – L...
Programme• Understanding the LibQUAL+ Survey Results• Internal analysis: Interpretation of results and  identifying action...
Understanding the LibQUAL+®       Survey Results          www.libqual.org
Data Repository
Results Notebooks• Sections for  Overall, Undergraduates, Graduates, Faculty, Staff,  Library Staff include:   – Demograph...
Results Notebook  Demographic Summary        www.libqual.org
Results Notebook Core Question Summary        www.libqual.org
Results Notebook Core Question Radar Chart          www.libqual.org
Results Notebook                    Dimension SummaryKey Term:Zone of Tolerance                         www.libqual.org
Results Notebook Local Questions Summary         www.libqual.org
Results NotebookSatisfaction & Outcome Questions Summary                www.libqual.org
Results Notebook  Library Use Summary        www.libqual.org
Data in Electronic Format• LibQUAL+® Analytics• Raw data files• SPSS
LibQUAL+® Analytics• Access to all institutional results• Breakdown the data by:  – User group  – Standard discipline• Exp...
LibQUAL+® Analytics
Data Selection
Representativeness
Core Questions
Thermometer Chart & Library Use Summary
Raw Data Files             In Excel and SPSS Format                Data Repository Link• You will receive access to your c...
SPSS Data Files        Statistical Package for the Social Sciences• More complex analysis than Excel in an  easier way  – ...
SPSS Data Files             Analysis Possibilities:•   Customized Discipline means and SD•   Local question means and SD• ...
SPSS Options• Invest in training on SPSS for Library Staff• Work in partnership with colleagues in  your institution who u...
You Too Can Chart LQ Data!           www.libqual.org
Understanding Disciplines    Standard Disciplines                           Custom Disciplines• LibQUAL+® provides standar...
Representativeness - Standard Disciplines                An American Academic Library   Respondents   Population          ...
Representativeness - Customized Disciplines                 An Academic Library   Respondents   Population                ...
Before we go on…These LQ Constructs can be Challenging! •   Means and Standard Deviation •   The Zone of Tolerance •   Rad...
Understanding Means &           Standard Deviation• Mean = arithmetic average  – A measure of central tendency  – Takes in...
Understanding the Zone of Tolerance• For the 22 items LibQUAL+® asks users‟  to rate their  – Minimum service level  – Des...
Understanding the Zone of Tolerance                 Zone of Tolerance               www.libqual.org
Understanding Radar Charts           www.libqual.org
Understanding Radar Charts           www.libqual.org
What Do the Colors Mean?ExceedingDesiredExpectations                    Green                     And                    B...
What Do the Colors Mean?      RedNot MeetingMinimumExpectations                  www.libqual.org
What Do the Colors Mean?     Little YellowMeeting DesiredExpectations                     www.libqual.org
What Do the Colors Mean?   A lot of YellowExceeding MinimumExpectations, Just!                      www.libqual.org
Understanding Thermometer Charts              Perceived                          Perceived
Internal AnalysisInterpretation of results        www.libqual.org
Interpreting Results                   How do users                   rate the Library?                   What „scores‟ do...
Interpreting Results   Identify areas where   the user community   thinks the Library is    performing “best”       and “w...
Minimum Desired   Perceived Adequacy SuperiorityID          Question Text                                                 ...
Perceived Mean Scores       9.00       8.00                                    BestMean       7.00       6.00             ...
The Importance of the Zone of Tolerance       9.00                                        Best: Closest to                ...
Minimum Desired   Perceived Adequacy SuperiorityID          Question Text                                                 ...
Best & Worst• The highest Adequacy & Superiority Mean  scores indicate the best performance• The lowest Adequacy & Superio...
Quick Tip• Use LibQUAL+® Analytics to export your  data into Excel• Use Data Sort to quickly highlight your  highest and l...
Interpreting Results     Identify areas where     the user community       has the highest            “wants”        www.l...
Highest Wants                          Example Library Core Summary - 20099.008.007.006.005.00       AS-1 AS-2 AS-3 AS-4 A...
Interpreting Results   Identify areas where   the user community   thinks the Library is  “furthest from meeting     minim...
Lowest Adequacy Mean                            Example Library Core Summary - 2009   9.00   8.00   7.00   6.00   5.00    ...
Review Dimension Summary        Example Library Dimension Summary - 2009987                                               ...
Interpreting Results     Interpretation Frameworkscan also be applied to data gathered     by separate User Groups,    by ...
Expectations of Postgraduates      Example Library Core Summary by User Group (Grad Students)  9  8  7  6  5  4      AS-1 ...
Explore Items by Subject Discipline                                                             Example Library Single Cor...
Internal AnalysisIdentifying actionable items          www.libqual.org
Questions You Need Answering• Which items clearly stand out as the top  wants/priorities of your users?• Which items clear...
Identifying Actionable Items           Cross Tabulate Desired & Adequacy Mean Scores                               Highest...
Identifying Actionable Items                    #1                                     #3            MOST DESIRED         ...
Identifying Actionable Items                    #1                                     #3            MOST DESIRED         ...
Remember LibQUAL+® Analytics
Identifying Actionable Items
Considerations• Do all user groups have the same  wants/priorities, or are there differences?• Plot graphs for different  ...
Internal AnalysisQualitative analysis of the LibQUAL+®              comments              www.libqual.org
Qualitative Analysis: User Comments• Why the Box is so Important:   – About half of participants provide open-ended     co...
Comments File                        Available from: Data Repository linkPlus, real-time access to the comments during Sta...
Two Case Studies• Texas A&M University  – Word clouds  – Atlas.TI• Brown University  – NVivo
Texas A&M: Analysis of Undergraduate Comments Source: Colleen Cook, Presented at QQML 2009 in Chania                     h...
www.libqual.orgLoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:AdminColleenServQual InterviewsTEXT Only01xxxxxxxxx.txt (redirected: c:z...
www.libqual.org
Methodology for Coding Qualitative Datahttp://www.libqual.org/documents/admin/BrownU_2005_LQ_qual_method.pdf              ...
Methodology for Coding Qualitative Data http://www.libqual.org/documents/admin/BrownU_2005_LQ_qual_method.pdf  • Use NVivo...
Brown U: Methodology for Coding Qualitative DataThe following table shows the total distribution of all 4,197 individual c...
External BenchmarkingHow are we doing compared to the         consortium?            www.libqual.org
Consortium Results• PDF Notebook in Data Repository• Provides average results for all consortium  members• Benchmarkable
Questions to Ask• How do our results compare to the  consortium?• Do our users have higher or lower  minimum and desired m...
Quick Tip• Consortium Notebooks in Excel Format  – Free Online PDF to Excel convertor:    http://www.pdftoexcelonline.com/...
Consortium & Institution Comparisons                                                      Affect of Service               ...
Consortium & Institution Comparisons                                                       Information Control            ...
Considerations• Benchmarking against the consortium will  help you put your results in context• Review the individual cons...
External Benchmarking Identifying best practice         www.libqual.org
Peer Comparisons• How do I select peers?  – Listen, talk to, or search web sites of your    University Office of Instituti...
LibQUAL+® Directory
Summary Statistics in the Data Repository
Summary Statistics in the Data Repository                 continued
Summary Statistics in Notebook
Remember:         We assess to improve...                   ....not to proveInstitutions should NOT use other libraries da...
Peer Comparisons:                   General Satisfaction                                   Peer Comparison                ...
Peer Comparisons:                             General Satisfaction                                         Peer Comparison...
Peer Comparisons:                      General Satisfaction                                   Peer Comparison             ...
Peer Comparisons:                                    Focus on One Question                                                ...
Re                                                                                                                        ...
Action Planning For Change          www.libqual.org
Cycle of Planning and Assessment              www.libqual.org
From Analysis to Action• From all of the data, determine what can and  should be addressed• Prioritize some action items  ...
Pulling it all together• What are your actionable items?• How do these compare to:  – Consortium  – Peer Group  – Individu...
Focused Follow-up                    University of VA:• Who is unhappy?  – Drilling down by college and discipline• Why ar...
Focused Follow-up               University of VA:• Is the Library meeting your minimal  level, regarding journal collectio...
Identifying & Using Best Practices• Focus on your top actionable items• Who is performing better than you?• What are they ...
Communicating Results to    Stakeholders         www.libqual.org
Why?• Communicating results effectively  is vital for implementing actions• Goals, priorities, and resource  allocation sh...
Engaging Library Staff in Understanding &              Using Data• Spread knowledge about LibQUAL+® to Library  staff   – ...
Overcoming Resistance“We know what’s best”                                  “...only customers                            ...
Northeastern University Case Study“Comments were sorted by academic department, which were postedalong with department-spe...
Presenting Results to Stakeholders• Identify all of the stakeholders or constituents  who want and need to know about the ...
Communicate with your Customers             (students, faculty, others)• Particularly those whom you asked to participate ...
KU Libraries: We Listened to You!                                     LibQUAL+ 2006Thanks to the more than 1,100 KU facult...
Longitudinal Analysis:Assessing impact of change         www.libqual.org
Cycle of Planning and Assessment              www.libqual.org
Longitudinal Analysis     Benchmarking Against       Self, Longitudinally“Nobody is more like me than me!”          --Anon...
Longitudinal Analysis• An interpretation framework that allows  you to compare performance over time• Are my scores increa...
Longitudinal AnalysisExample Library Satisfaction (All Users, 2004 – 2008)                                       General S...
Longitudinal Analysis    Example Library Satisfaction (Faculty, 2004 – 2008)                                    General Sa...
Longitudinal AnalysisExample Library Satisfaction (Undergrads, 2004 – 2008)                                 General Satisf...
Z.O.T. + Longitudinal    Example Library Changing Expectations (Faculty, 2003 – 2009)98765      2003        2006          ...
Z.O.T. + LongitudinalExample Library Changing Expectations & Performance (Faculty, 2003 – 2009) 9 8 7 6 5      2003       ...
Assessing Impact of Actions                  Explore Specific Questions                         IC-1 Making electronic res...
Assess Impact of Actions        Have results improved?• Yes  – Communicate to Library Staff, Key    Stakeholders, the wide...
Summing Up  www.libqual.org
Review• Understanding the LibQUAL+® Survey Results• Internal analysis: Interpretation of results and  identifying actionab...
LibQUAL+® Resources• LibQUAL+® Web site:  http://www.libqual.org/home• Publications:  http://www.libqual.org/Publications....
LibQUAL+® Team• Martha Kyrillidou - Senior Director, ARL Statistics  and Service Quality Programs  martha@arl.org• David G...
Thank youSelena KillickLibQUAL+® European Support Officers.a.killick@cranfield.ac.uk+44 1793 785561                       ...
Sciences Po presentation eng
Sciences Po presentation eng
Sciences Po presentation eng
Upcoming SlideShare
Loading in …5
×

Sciences Po presentation eng

824 views

Published on

by Selena Killick

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
824
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 1st Group – Look how large “Study” is in this word cloud2nd Group - Verbs
  • Once you have identified your peers, access their LibQUAL+® results in Analytics
  • Sciences Po presentation eng

    1. 1. Analysis and Interpretation of the LibQUAL+® Results Selena Killick Association of Research Libraries Cranfield University SciencesPo Paris, France 26th June 2012 www.libqual.org
    2. 2. Objectives• An introduction to the LibQUAL+® results• Analyzing and interpreting results; – Internally – Externally – Longitudinally• Interpret the data and identify actionable items• Qualitative analysis of LibQUAL+® comments• How to conduct analysis on your survey results using LibQUAL+® Analytics• How to analyse Consortium results and compare them with local results• French specificities
    3. 3. Programme• Understanding the LibQUAL+ Survey Results• Internal analysis: Interpretation of results and identifying actionable items• Internal analysis: Qualitative analysis of the LibQUAL+® comments• External benchmarking: How are we doing compared to the consortium?• External benchmarking: Identifying best practice• Action planning for change• Communicating your results to stakeholders• Longitudinal analysis: Assessing impact of change www.libqual.org
    4. 4. Understanding the LibQUAL+® Survey Results www.libqual.org
    5. 5. Data Repository
    6. 6. Results Notebooks• Sections for Overall, Undergraduates, Graduates, Faculty, Staff, Library Staff include: – Demographic Summary – Core Questions Summary – Dimensions Summary – Local Questions – General Satisfaction Questions – Information Literacy Outcomes Questions – Library Use Summary• Appendix describing changes in the dimensions and the questions included in each dimension. www.libqual.org
    7. 7. Results Notebook Demographic Summary www.libqual.org
    8. 8. Results Notebook Core Question Summary www.libqual.org
    9. 9. Results Notebook Core Question Radar Chart www.libqual.org
    10. 10. Results Notebook Dimension SummaryKey Term:Zone of Tolerance www.libqual.org
    11. 11. Results Notebook Local Questions Summary www.libqual.org
    12. 12. Results NotebookSatisfaction & Outcome Questions Summary www.libqual.org
    13. 13. Results Notebook Library Use Summary www.libqual.org
    14. 14. Data in Electronic Format• LibQUAL+® Analytics• Raw data files• SPSS
    15. 15. LibQUAL+® Analytics• Access to all institutional results• Breakdown the data by: – User group – Standard discipline• Export: – Radar charts – Data tables – Thermometer charts• Very easy to use 
    16. 16. LibQUAL+® Analytics
    17. 17. Data Selection
    18. 18. Representativeness
    19. 19. Core Questions
    20. 20. Thermometer Chart & Library Use Summary
    21. 21. Raw Data Files In Excel and SPSS Format Data Repository Link• You will receive access to your complete raw survey data in Excel format• You will also receive an SPSS syntax file that you can apply to the Excel file to analyze your data in SPSS• Instructions on how to create your own SPSS file are available in the Data Repository www.libqual.org
    22. 22. SPSS Data Files Statistical Package for the Social Sciences• More complex analysis than Excel in an easier way – Excel can do a lot of things that SPSS can do but not everything• Can be complex to use www.libqual.org
    23. 23. SPSS Data Files Analysis Possibilities:• Customized Discipline means and SD• Local question means and SD• Branch Library means and SD• Relationship between different variables• Validity and reliability analysis
    24. 24. SPSS Options• Invest in training on SPSS for Library Staff• Work in partnership with colleagues in your institution who use SPSS already• Commission research from your students looking to use data in their studies• Commission LibQUAL+® to conduct the analysis for you www.libqual.org
    25. 25. You Too Can Chart LQ Data! www.libqual.org
    26. 26. Understanding Disciplines Standard Disciplines Custom Disciplines• LibQUAL+® provides standard • You may choose to create your discipline options that you can own discipline categories, customize to your institution‟s using your local terminology. If disciplines or other affiliations. you choose to add your own categories, each MUST be mapped to a LibQUAL+® standard discipline for data analysis purposes. (Please make sure your new term(s) relate to the standard disciplines. www.libqual.org
    27. 27. Representativeness - Standard Disciplines An American Academic Library Respondents Population www.libqual.org
    28. 28. Representativeness - Customized Disciplines An Academic Library Respondents Population www.libqual.org
    29. 29. Before we go on…These LQ Constructs can be Challenging! • Means and Standard Deviation • The Zone of Tolerance • Radar Charts • Standard vs. Customized Disciplines • Representativeness www.libqual.org
    30. 30. Understanding Means & Standard Deviation• Mean = arithmetic average – A measure of central tendency – Takes into account all scores – Sensitive to all values and affected by extreme scores• Standard Deviation = average distance – A measure of dispersion – Takes into account all scores – Sensitive to all values and affected by extreme scores www.libqual.org
    31. 31. Understanding the Zone of Tolerance• For the 22 items LibQUAL+® asks users‟ to rate their – Minimum service level – Desired service level – Perceived service performance•This gives us a „Zone of Tolerance‟ for each question; the distance between minimally acceptable and desired service ratings• Perception ratings ideally fall within the Zone of Tolerance www.libqual.org
    32. 32. Understanding the Zone of Tolerance Zone of Tolerance www.libqual.org
    33. 33. Understanding Radar Charts www.libqual.org
    34. 34. Understanding Radar Charts www.libqual.org
    35. 35. What Do the Colors Mean?ExceedingDesiredExpectations Green And Blue www.libqual.org
    36. 36. What Do the Colors Mean? RedNot MeetingMinimumExpectations www.libqual.org
    37. 37. What Do the Colors Mean? Little YellowMeeting DesiredExpectations www.libqual.org
    38. 38. What Do the Colors Mean? A lot of YellowExceeding MinimumExpectations, Just! www.libqual.org
    39. 39. Understanding Thermometer Charts Perceived Perceived
    40. 40. Internal AnalysisInterpretation of results www.libqual.org
    41. 41. Interpreting Results How do users rate the Library? What „scores‟ do users give the Library‟s performance? Note: This is the simplest analysis and does not take advantage of the “gap analysis” opportunity
    42. 42. Interpreting Results Identify areas where the user community thinks the Library is performing “best” and “worst”
    43. 43. Minimum Desired Perceived Adequacy SuperiorityID Question Text Mean Mean Mean Mean MeanAffect of ServiceAS-1 Library staff who instill confidence in users 5.90 7.48 6.66 0.76 -0.82AS-2 Giving users individual attention 5.43 6.84 6.15 0.72 -0.69AS-3 Library staff who are consistently courteous 6.63 7.88 7.23 0.60 -0.65AS-4 Readiness to respond to users enquiries 6.65 7.88 7.17 0.52 -0.71AS-5 Library staff who have the knowledge to answer user questions 6.61 7.84 7.21 0.60 -0.63AS-6 Library staff who deal with users in a caring fashion 6.43 7.79 7.20 0.77 -0.59AS-7 Library staff who understand the needs of their users 6.53 7.78 6.99 0.46 -0.79AS-8 Willingness to help users 6.60 7.84 7.14 0.54 -0.70AS-9 Dependability in handling users service problems 6.53 7.73 6.89 0.36 -0.84Information ControlIC-1 Making electronic resources accessible from my home or office 6.81 8.27 6.85 0.04 -1.42IC-2 A library Web site enabling me to locate information on my own 6.77 8.09 6.95 0.18 -1.14IC-3 The printed library materials I need for my work 6.50 7.83 6.54 0.04 -1.29IC-4 The electronic information resources I need 6.41 7.98 6.71 0.30 -1.27IC-5 Modern equipment that lets me easily access needed information 6.68 7.99 6.93 0.25 -1.06IC-6 Easy-to-use access tools that allow me to find things on my own 6.66 8.01 6.92 0.26 -1.09IC-7 Making information easily accessible for independent use 6.72 8.02 6.98 0.26 -1.04IC-8 Print and/or electronic journal collections I require for my work 6.83 8.10 6.73 -0.10 -1.37Library as PlaceLP-1 Library space that inspires study and learning 6.12 7.77 6.15 0.03 -1.62LP-2 Quiet space for individual work 6.75 8.04 6.29 -0.46 -1.75LP-3 A comfortable and inviting location 6.35 7.82 6.77 0.42 -1.05LP-4 A haven for study, learning, or research 6.53 7.99 6.37 -0.16 -1.62LP-5 Space for group learning and group study 5.77 7.23 6.53 0.76 -0.70Overall 6.46 7.83 6.79 0.33 -1.04
    44. 44. Perceived Mean Scores 9.00 8.00 BestMean 7.00 6.00 Worst Worst 5.00 AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5 Question www.libqual.org
    45. 45. The Importance of the Zone of Tolerance 9.00 Best: Closest to Desired Mean 8.00Mean 7.00 6.00 Lowest Perceived & Lowest Desired Worst: Furthest from Minimum Mean 5.00 AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5 Question www.libqual.org
    46. 46. Minimum Desired Perceived Adequacy SuperiorityID Question Text Mean Mean Mean Mean MeanAffect of ServiceAS-1 Library staff who instill confidence in users 5.90 7.48 6.66 0.76 -0.82AS-2 Giving users individual attention 5.43 6.84 6.15 0.72 -0.69AS-3 Library staff who are consistently courteous 6.63 7.88 7.23 0.60 -0.65AS-4 Readiness to respond to users enquiries 6.65 7.88 7.17 0.52 -0.71AS-5 Library staff who have the knowledge to answer user questions 6.61 7.84 7.21 0.60 -0.63AS-6 Library staff who deal with users in a caring fashion 6.43 7.79 7.20 0.77 -0.59AS-7 Library staff who understand the needs of their users 6.53 7.78 6.99 0.46 -0.79AS-8 Willingness to help users 6.60 7.84 7.14 0.54 -0.70AS-9 Dependability in handling users service problems 6.53 7.73 6.89 0.36 -0.84Information ControlIC-1 Making electronic resources accessible from my home or office 6.81 8.27 6.85 0.04 -1.42IC-2 A library Web site enabling me to locate information on my own 6.77 8.09 6.95 0.18 -1.14IC-3 The printed library materials I need for my work 6.50 7.83 6.54 0.04 -1.29IC-4 The electronic information resources I need 6.41 7.98 6.71 0.30 -1.27IC-5 Modern equipment that lets me easily access needed information 6.68 7.99 6.93 0.25 -1.06IC-6 Easy-to-use access tools that allow me to find things on my own 6.66 8.01 6.92 0.26 -1.09IC-7 Making information easily accessible for independent use 6.72 8.02 6.98 0.26 -1.04IC-8 Print and/or electronic journal collections I require for my work 6.83 8.10 6.73 -0.10 -1.37Library as PlaceLP-1 Library space that inspires study and learning 6.12 7.77 6.15 0.03 -1.62LP-2 Quiet space for individual work 6.75 8.04 6.29 -0.46 -1.75LP-3 A comfortable and inviting location 6.35 7.82 6.77 0.42 -1.05LP-4 A haven for study, learning, or research 6.53 7.99 6.37 -0.16 -1.62LP-5 Space for group learning and group study 5.77 7.23 6.53 0.76 -0.70Overall 6.46 7.83 6.79 0.33 -1.04
    47. 47. Best & Worst• The highest Adequacy & Superiority Mean scores indicate the best performance• The lowest Adequacy & Superiority Mean scores indicate the worst performance• Highest Adequacy & Highest Superiority may not be the same item• Lowest Adequacy & Lowest Superiority may not be the same item www.libqual.org
    48. 48. Quick Tip• Use LibQUAL+® Analytics to export your data into Excel• Use Data Sort to quickly highlight your highest and lowest scores www.libqual.org
    49. 49. Interpreting Results Identify areas where the user community has the highest “wants” www.libqual.org
    50. 50. Highest Wants Example Library Core Summary - 20099.008.007.006.005.00 AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5 2009 Identify users‟ highest “wants” www.libqual.org
    51. 51. Interpreting Results Identify areas where the user community thinks the Library is “furthest from meeting minimum needs” www.libqual.org
    52. 52. Lowest Adequacy Mean Example Library Core Summary - 2009 9.00 8.00 7.00 6.00 5.00 AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5 2009Identify areas where the Libraries are furthest from meeting users‟ minimum “needs” www.libqual.org
    53. 53. Review Dimension Summary Example Library Dimension Summary - 2009987 Z.O.T. Per65 Affect of Service Information Control Library as Place 2009 Overall (Undergraduates, Graduates, & Faculty) www.libqual.org
    54. 54. Interpreting Results Interpretation Frameworkscan also be applied to data gathered by separate User Groups, by Discipline affiliation and by primary library (Branch) use www.libqual.org
    55. 55. Expectations of Postgraduates Example Library Core Summary by User Group (Grad Students) 9 8 7 6 5 4 AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9 IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8 LP-1 LP-2 LP-3 LP-4 LP-5 2009 Graduates Identify highest “wants” for Grad StudentsIdentify areas where the Libraries are furthest from meeting Grad minimum “needs” www.libqual.org
    56. 56. Explore Items by Subject Discipline Example Library Single Core Item by Discipline 9 8 7 6 5 Z.O.T. Law Humanities Other Education Architecture Performing & Fine Arts Business Undecided General Studies Communications / Journalism Health Sciences Social Sciences / Psychology Military / Naval Science Science / Math Engineering / Computer Science Agriculture / Environmental Studies Per IC-4 The electronic information resources I need Identify disciplines with highest “wants” for e-resourcesIdentify where the e-resources are furthest from meeting discipline minimum “needs” www.libqual.org
    57. 57. Internal AnalysisIdentifying actionable items www.libqual.org
    58. 58. Questions You Need Answering• Which items clearly stand out as the top wants/priorities of your users?• Which items clearly stand out and the lowest wants/priorities of your users?• Which items clearly stand out as the top weaknesses for your users?• Which items clearly stand out and the strengths for your users?
    59. 59. Identifying Actionable Items Cross Tabulate Desired & Adequacy Mean Scores Highest Desired Lowest HighestAdequacy Adequacy Lowest Desired www.libqual.org
    60. 60. Identifying Actionable Items #1 #3 MOST DESIRED MOST DESIRED and and MOST ADEQUATE LEAST ADEQUATE (lowest weakness/ (highest weakness) highest strength)ADEQUACY #2 #4 LEAST ADEQUATE MOST ADEQUATE (highest weakness) (lowest weakness/ and highest strength) LEAST DESIRED and DESIRED LEAST DESIRED www.libqual.org
    61. 61. Identifying Actionable Items #1 #3 MOST DESIRED MOST DESIRED and and MOST ADEQUATE LEAST ADEQUATE (lowest weakness/ (highest weakness) highest strength)ADEQUACY #2 #4 LEAST ADEQUATE MOST ADEQUATE (highest weakness) (lowest weakness/ and highest strength) LEAST DESIRED and DESIRED LEAST DESIRED www.libqual.org
    62. 62. Remember LibQUAL+® Analytics
    63. 63. Identifying Actionable Items
    64. 64. Considerations• Do all user groups have the same wants/priorities, or are there differences?• Plot graphs for different – user groups – disciplines – branch libraries• If you added local questions, included them in your graphs. www.libqual.org
    65. 65. Internal AnalysisQualitative analysis of the LibQUAL+® comments www.libqual.org
    66. 66. Qualitative Analysis: User Comments• Why the Box is so Important: – About half of participants provide open-ended comments, and these are linked to demographics and quantitative data – Users elaborate the details of their concerns – Users feel the need to be constructive in their criticisms & offer specific suggestions for action• User Comments available on the LibQUAL+® Web site – Download comments in Excel or text file – Skim the comments• Conduct analysis www.libqual.org
    67. 67. Comments File Available from: Data Repository linkPlus, real-time access to the comments during Stage 2) Monitor Survey Progress Basic demographic information such as user group, age, sex, and library branch (if available) is provided with each comment. Comments are also tagged with a unique identification number that enables you to link each comment to the individual‟s survey response. Comments are not visible to other institutions participating in LibQUAL+®; only your institution has access to your comments. www.libqual.org
    68. 68. Two Case Studies• Texas A&M University – Word clouds – Atlas.TI• Brown University – NVivo
    69. 69. Texas A&M: Analysis of Undergraduate Comments Source: Colleen Cook, Presented at QQML 2009 in Chania http://www.wordle.com/ www.libqual.org
    70. 70. www.libqual.orgLoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:AdminColleenServQual InterviewsTEXT Only01xxxxxxxxx.txt (redirected: c:zzatlastifred
    71. 71. www.libqual.org
    72. 72. Methodology for Coding Qualitative Datahttp://www.libqual.org/documents/admin/BrownU_2005_LQ_qual_method.pdf www.libqual.org
    73. 73. Methodology for Coding Qualitative Data http://www.libqual.org/documents/admin/BrownU_2005_LQ_qual_method.pdf • Use NVivo software to code & analyze text • Review the comments as a whole • Create a taxonomy – a master list of themes (referred to as “nodes” by NVivo) and the specific keywords mentioned by users in the survey comments for each of the nodes – can generate a word frequency list from the comments file to facilitate the creation of the taxonomy • Import LQ Comments into NVivo • Run reports to assign nodes to comments based on the taxonomy/ keywords • Review the assigned nodes for each comment, making necessary corrections & additions • Evaluate the qualitative data for frequency statistics & analyses of sub-sets of the comments www.libqual.org
    74. 74. Brown U: Methodology for Coding Qualitative DataThe following table shows the total distribution of all 4,197 individual comments (or “tags”) according to the 29 topics identified in the taxonomy. Topic (“Tag”) Number of comments with this tag Percentage of all comments Negative 479 59% Suggestion 465 58% Positive 321 40% Use 320 40% Location 290 36% Collection 264 33% Policies 233 29% Ambiance 226 28% Customer Service 218 27% Online content 176 22% Hours 156 19% Furnishings 134 16% Ease of Use 109 13% ILL 81 10% Web site 77 9% Quotable 68 8% Computer Equipment 64 8% Lighting 62 7% Comparison 61 7% Book Availability 56 7% Non-computer equipment 55 6% Catalog 53 6% Survey 49 6% Training 48 6% Off campus 43 5% Temperature 33 4% Named Staff 28 3% Financial www.libqual.org 26 3%
    75. 75. External BenchmarkingHow are we doing compared to the consortium? www.libqual.org
    76. 76. Consortium Results• PDF Notebook in Data Repository• Provides average results for all consortium members• Benchmarkable
    77. 77. Questions to Ask• How do our results compare to the consortium?• Do our users have higher or lower minimum and desired means?• Do our users have higher or lower perceptions?• Are our top priorities (highest desires) the same as everyone else?
    78. 78. Quick Tip• Consortium Notebooks in Excel Format – Free Online PDF to Excel convertor: http://www.pdftoexcelonline.com/• Not perfect, will require some checking of the data• Could save you data input time
    79. 79. Consortium & Institution Comparisons Affect of Service Affect of Service 9 8 7Mean Score 6 5 4 AS-1 Average UniA Average UniA Average UniA Average UniA Average UniA Average UniA Average UniA Average UniA Average UniA AS-1 AS-2 AS-2 AS-3 AS-3 AS-4 AS-4 AS-5 AS-5 AS-6 AS-6 AS-7 AS-7 AS-8 AS-8 AS-9 AS-9 Question
    80. 80. Consortium & Institution Comparisons Information Control Information Control 9 8 7Mean Score 6 5 4 IC-1 IC-1 Uni X IC-2 IC-2 Uni X IC-3 IC-3 Uni X IC-4 IC-4 Uni X IC-5 IC-5 Uni X IC-6 IC-6 Uni X IC-7 IC-7 Uni X IC-8 IC-8 Uni X Average Average Average Average Average Average Average Average Question
    81. 81. Considerations• Benchmarking against the consortium will help you put your results in context• Review the individual consortium members listed in the notebook – Should your performance be the same as these institutions? – If not, who should you benchmark against? – Create your own peer-group if necessary
    82. 82. External Benchmarking Identifying best practice www.libqual.org
    83. 83. Peer Comparisons• How do I select peers? – Listen, talk to, or search web sites of your University Office of Institutional Research, Provost, President – Consortium members – Descriptive library statistics – Type of institution – Size of the faculty, student body (in specific disciplines)• Peer Group and/or Individual Institutions www.libqual.org
    84. 84. LibQUAL+® Directory
    85. 85. Summary Statistics in the Data Repository
    86. 86. Summary Statistics in the Data Repository continued
    87. 87. Summary Statistics in Notebook
    88. 88. Remember: We assess to improve... ....not to proveInstitutions should NOT use other libraries data in ANY WAY that would compromiseand harm the reputation of other institutions. Institutions may use other libraries datain a confidential manner without disclosing the institutional identity of other libraries.
    89. 89. Peer Comparisons: General Satisfaction Peer Comparison General Satisfaction *Data taken from last year of participation9876 H F A C B E LIBRARY G D X In general, I am satisfied with the way in which I am treated at the library. www.libqual.org
    90. 90. Peer Comparisons: General Satisfaction Peer Comparison General Satisfaction *Data taken from last year of participation9876 H A C E F G LIBRARY D B X In general, I am satisfied with library support for my learning, research, and/or teaching needs. www.libqual.org
    91. 91. Peer Comparisons: General Satisfaction Peer Comparison General Satisfaction *Data taken from last year of participation9876 H A C F B E G LIBRARY D X How would you rate the overall quality of the service provided by the library? Library‟s conclusion: There‟s still room for improvement! www.libqual.org
    92. 92. Peer Comparisons: Focus on One Question LibQUAL+ 2006 Faculty Ratings of Journal Collections ARL Libraries 9.00 UVA 8.00 7.00 6.00 Top of Blue Bar = Desired Level of Service Bottom of Blue Bar = Minimum Level of Service Red Square = Perceived Service Performance 5.00 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37Source: Jim Self, University of Virginia, Presented at Performance Measurement in Academic Libraries Workshop, EBLIP4, Durham, NorthCarolina, May 11, 2007 www.libqual.org
    93. 93. Re m ot e ac ce Li ss br to ar el y ec w tr o eb ni si te c al re lo so -0.40 -0.20 0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 w ur in ce El ec Pr g s tro in au tl ib to ni no c ra m In in ry y fo m fo rm at rm at er at io ia io n ls n M re ea od so si Ea ur ly sy er n ce Pr ac -to eq s in ce -u ui ta ss se pm nd ib /o le ac en re fo ce t le ri nd ss ct to Em ro ep ol pl ni c en s oy jo de ee ur nt Em G s na us pl iv wh lc e oy in o ol le ee g in ct s us st io Re wh er ill ns o s co ad in nf in ar e di id es vi en s co du ce to ns al re is at sp te te on nt nt d ly io to co n us ur Kn te ow er ou Em le s s pl dg qu oy ab es ee le t io s em ns wh o Ca pl oy un rin g Adequacy Gap de ee Li rs em s br ta pl Wwww.libqual.org ar nd oy y De pe illi us ee sp ac nd ng er s ne s e ab ss ne th ilit at y to ed in in he s A sp ha lp The difference between the minimum and perceived score. ire nd us qu s lin er ie te g s ts ac pr pa hi ob A A ce ng le co fo an m ge m ri d s ta fo nd le wa rta ivi ar y bl du ni fo e al ng rs an tu d ac dy in t iv vi Adequacy Gap, All Questions ,l itie tin s ea rn g lo in ca g, t io Peer Group Comparisons: G or n ro re up se ar st ch ud y sp ac O e VE Source: Fred Heath, LibQUAL+™ Results Meeting, ALA Annual Conference, Seattle, WA, January 22, 2007 RA LL ARL Peers UT Austin
    94. 94. Action Planning For Change www.libqual.org
    95. 95. Cycle of Planning and Assessment www.libqual.org
    96. 96. From Analysis to Action• From all of the data, determine what can and should be addressed• Prioritize some action items – Align with mission, vision and goals of parent organization – Address users‟ top priorities, by user group – Improve areas of strong user dissatisfaction – Build on strengths, if they are truly user needs and priorities – Identify work that can be de-emphasized and resources that can be reallocated www.libqual.org
    97. 97. Pulling it all together• What are your actionable items?• How do these compare to: – Consortium – Peer Group – Individual institutions• Review the free-text comments relate to your actionable items – Coded – Linked to responses
    98. 98. Focused Follow-up University of VA:• Who is unhappy? – Drilling down by college and discipline• Why are they unhappy? – Reading the comments – Conducting targeted interviews• Focus on areas with low scores• Diverse group of faculty• Asked for specific needs and wants – Including names of needed titles• Quick interviews Source: Jim Self, University of Virginia www.libqual.org
    99. 99. Focused Follow-up University of VA:• Is the Library meeting your minimal level, regarding journal collections? – If not, what can we do?• Is the Library meeting your desired level? – If not, what can we do?• Does it matter if journals are print or electronic?• Any other comments about the Library? Source: Jim Self, University of Virginia www.libqual.org
    100. 100. Identifying & Using Best Practices• Focus on your top actionable items• Who is performing better than you?• What are they doing that you could learn from?• Contact each other to discuss & share ideas for service improvements• Implementing improvements based upon best practices
    101. 101. Communicating Results to Stakeholders www.libqual.org
    102. 102. Why?• Communicating results effectively is vital for implementing actions• Goals, priorities, and resource allocation should be influenced by customer needs and wants• Decision makers, key stakeholders & budget holders need to understand the results
    103. 103. Engaging Library Staff in Understanding & Using Data• Spread knowledge about LibQUAL+® to Library staff – Post Notebook on library web – Make staff aware of LQ tutorial: • http://www.libqual.org/Information/Tools/index.cfm – All-staff presentation – Enable key staff to access LibQUAL+® Analytics – Disseminate Comments to depts, units, groups, branches, librarians, etc.• Develop a culture of assessment and accountability for listening to customers and acting on user feedback www.libqual.org
    104. 104. Overcoming Resistance“We know what’s best” “...only customers judge quality;“They are wrong” all other judgments are essentially irrelevant.”“We don’t have the resources to…” Zeithaml, Parasuraman, Berry.(1999). Delivering Quality Service. NY: The Free Press www.libqual.org
    105. 105. Northeastern University Case Study“Comments were sorted by academic department, which were postedalong with department-specific radar charts and gap summaries, inthe Research & Instruction Department.The “Great Wall of LibQUAL+®” provoked discussion, and providedcollection managers with insight into the concerns of the faculty andstudents in the departments they were serving. www.libqual.org
    106. 106. Presenting Results to Stakeholders• Identify all of the stakeholders or constituents who want and need to know about the survey results• Consider the “stake” of each of the above; what specific aspect of LibQUAL+® will be of most interest / concern• Determine how to communicate with each identified stakeholder• Speak in terms they will understand• Present data graphically in a format they can interpret quickly & easily www.libqual.org
    107. 107. Communicate with your Customers (students, faculty, others)• Particularly those whom you asked to participate in the survey• Announce incentive award winners• Inform users of highlights of survey results• Focused follow-up for more specific insights• Most importantly, what the library intends to do “You asked for it…” www.libqual.org
    108. 108. KU Libraries: We Listened to You! LibQUAL+ 2006Thanks to the more than 1,100 KU faculty, staff and students who completed the 2006LibQUAL+ survey, KU Libraries has made significant changes over the past few months tobetter meet your research and service needs.You requested:•Access to the electronic resources from your home or office •Print and/or electronic journal collections you require for your work •A Libraries Web site that enables you to locate information on your own more quickly and easily •Librarians and staff members who have the knowledge to answer your questions •Dependability in handling your service problemsWe delivered:•More access to print and electronic materials, including 30,000 journals and many otherprimary resources •The new Information Gateway, a primary tool for searching the Libraries proprietary online resources including databases, journals and images •A newly redesigned Web site •Access to electronic records for hundreds of thousands of previously inaccessible items•An ongoing commitment to enhancing service quality through comprehensive training andcontinuous evaluation www.libqual.org www.libqual.org
    109. 109. Longitudinal Analysis:Assessing impact of change www.libqual.org
    110. 110. Cycle of Planning and Assessment www.libqual.org
    111. 111. Longitudinal Analysis Benchmarking Against Self, Longitudinally“Nobody is more like me than me!” --Anonymous www.libqual.org
    112. 112. Longitudinal Analysis• An interpretation framework that allows you to compare performance over time• Are my scores increasing or decreasing?• Are my scores increasing or decreasing for specific subgroups of my population: faculty, grad, undergraduates, and/or disciplines or branches?• Have my actions affected my scores? www.libqual.org
    113. 113. Longitudinal AnalysisExample Library Satisfaction (All Users, 2004 – 2008) General Satisfaction Overall 2004 to 2008 *All user groups (excluding Library Staff)9876 20045 20084321 In general, I am satisfied In general, I am satisfied How would you rate the with the way in which I am with library support for my overall quality of the treated at the library. learning, research, and/or service provided by the teaching needs. library? Increase in satisfaction in all 3 satisfaction measures for Total Population www.libqual.org
    114. 114. Longitudinal Analysis Example Library Satisfaction (Faculty, 2004 – 2008) General Satisfaction Faculty 2004 to 2008 *All user groups (excluding Library Staff)9876 20045 20084321 In general, I am satisfied In general, I am satisfied How would you rate the with the way in which I am with library support for my overall quality of the treated at the library. learning, research, and/or service provided by the teaching needs. library? Increase in satisfaction in all 3 satisfaction measures for Faculty www.libqual.org
    115. 115. Longitudinal AnalysisExample Library Satisfaction (Undergrads, 2004 – 2008) General Satisfaction Undergraduates 2004 to 2008 *All user groups (excluding Library Staff) 9 8 7 6 2004 5 2008 4 3 2 1 In general, I am satisfied In general, I am satisfied How would you rate the with the way in which I am with library support for my overall quality of the treated at the library. learning, research, and/or service provided by the teaching needs. library? Unchanged or decrease in satisfaction in all 3 satisfaction measures for Undergraduates www.libqual.org
    116. 116. Z.O.T. + Longitudinal Example Library Changing Expectations (Faculty, 2003 – 2009)98765 2003 2006 2009 2003 2006 2009 2003 2006 2009 Affect of Service Information Control Library as Place Space and place needs are Faculty tolerance is narrowing: increasing over time, though their minimum needs are higher overall remain less critical for but their desires are stable faculty than other issues www.libqual.org
    117. 117. Z.O.T. + LongitudinalExample Library Changing Expectations & Performance (Faculty, 2003 – 2009) 9 8 7 6 5 2003 2006 2009 2003 2006 2009 2003 2006 2009 Affect of Service Information Control Library as Place Wow, customer service for faculty Ouch, even though this Library This Library seems to be keeping is improving even while some improved slightly, the ability to pace with increasing faculty expectations are increasing! meet faculty needs for information expectations about library spaces and access is not keeping pace with expectations www.libqual.org
    118. 118. Assessing Impact of Actions Explore Specific Questions IC-1 Making electronic resources accessible from my home or office Library as Place9.00 9.008.50 8.508.00 8.007.50 7.507.00 7.006.50 6.506.00 6.00 5.505.50 5.005.00 4.504.50 4.004.00 2005 2006 2007 2008 2009 2010 2011 2005 2006 2007 2008 2009 2010 2011 Minimum Mean Desired Mean Perceived Mean
    119. 119. Assess Impact of Actions Have results improved?• Yes – Communicate to Library Staff, Key Stakeholders, the wider University community• No – Do you know why? – Explore further with your customers – What can you do to improve?
    120. 120. Summing Up www.libqual.org
    121. 121. Review• Understanding the LibQUAL+® Survey Results• Internal analysis: Interpretation of results and identifying actionable items• Internal analysis: Qualitative analysis of the LibQUAL+® comments• External benchmarking: How are we doing compared to the consortium?• External benchmarking: Identifying best practice• Action planning for change• Communicating your results to stakeholders• Longitudinal analysis: Assessing impact of change www.libqual.org
    122. 122. LibQUAL+® Resources• LibQUAL+® Web site: http://www.libqual.org/home• Publications: http://www.libqual.org/Publications.aspx• Events and Training: http://www.libqual.org/events• Gap Theory/Radar Graph Introduction: http://libqual.org/about/about_survey/tools• LibQUAL+® Procedures Manual: http://www.libqual.org/documents/LibQual/publications/ProceduresMan ual.pdf• ARL YouTube Channel: http://www.youtube.com/user/arladmin?feature=watch www.libqual.org
    123. 123. LibQUAL+® Team• Martha Kyrillidou - Senior Director, ARL Statistics and Service Quality Programs martha@arl.org• David Green - Library Relations Coordinator libqual@arl.org• Selena Killick – European Support s.a.killick@cranfield.ac.uk• And sometimes, consultant Raynna Bowlby raynna.bowlby@charter.net www.libqual.org
    124. 124. Thank youSelena KillickLibQUAL+® European Support Officers.a.killick@cranfield.ac.uk+44 1793 785561 www.libqual.org

    ×