Strategic Metrics, presented at the Leadership Seminar on Strategy, Assessment and Service Development. University of Lund, Sweden. 19th September 2012. Presentation by Selena Killick, Cranfield University. Presentation discusses the need for assessment of Library Strategies and some of the techniques available to achieve this.

1. Strategic Metrics
Selena Killick
Library Quality Officer
Leadership Seminar on Strategy,
Assessment and Service Development.
University of Lund
19th September 2012.

2. Eras of Library Metrics
• Holdings
• Physical usage
• Customer satisfaction
• Electronic usage
• Value & Impact

3. Few libraries exist in a vacuum,
accountable only to themselves. There is
always a larger context for assessing
library quality, that is, what and how well
does the library contribute to achieving the
overall goals of the parent constituencies?
(S. Pritchard 1996)

4. Value & Impact Metrics
• Based on what matters the most to your key
stakeholders
• Identify where the Library impacts upon the goals of
the parent organisation
• Demonstrate how the library is contributing to the
mission of your institution

5. Strategic Metrics
• Start by defining your outcomes
• What do you want to demonstrate?
• What does your institution care about?
• SMART outcomes:
• Specific
• Measureable
• Attainable
• Relevant
• Time-bound
• If in doubt, KISS.

6. Good Strategic Metrics
• Linked to the University/Library mission & goals
• Provide useful information
• Reasonably accurate within the data limitations
• Ethical
• Cost effective

7. Mixed-Methods
Approach
Quantitative Qualitative
• Library Data • Customer Surveys
• Institutional Data • Focus groups
• National/International • Observational Studies
Data • Interviews
• Activity analysis
• Return on Investment
• Cost benefit

8. Quantitative Data
• Be prepared to take a fresh look at your data
collection
• Look beyond the Library
• Talk to other data providers in your institution
• Combine data sets
• Local (Library & student data), national,
international
• Check to see what is available before you start to
count things

9. Case Study: Cornell
University Library
Library Value Calculation
• Annual Cost to the institution $56,678,222
If CUL did not exist:
• Sourcing information and answering enquiries would
cost the institution $90,648,785
Source: Cornell University Library Research & Assessment Unit

10. Case Study: University
of West Florida
UWF University Libraries calculated its Return on Investment (ROI) for eight of
its services:
• students studying in the library;
• borrowing books, e-books, DVDs and laptops;
• students or faculty members asking reference questions or meeting with
reference librarians for individual research consultations;
• conducting library instruction sessions; and,
• students or faculty members using subscription databases when off-campus.
Calculations based on the number of occurrences (service outputs) multiplied by
an informal and conservative market value of the occurrence to calculate a
summed value for the services. The summed value was then divided by sum of
the University Libraries' personnel and operating expenditures.
• It was calculated that for every dollar expended by the University
Libraries, at least $5.89 was returned for the services identified.
Source: University of West Florida

11. Case Study: University
of Huddersfield
• Hypothesis: do the students who use the library the
most get the highest grades?
• Library Usage Data
• Book loans
• eResources usage
• Visits to the Library
• Student Attainment Data
• Final grade received when graduating

12. Non/Low Use Project
Source: Stone, G. (2011)
13

13. Measuring Library Impact
Source: Stone, G. (2011)
Analysis of the results consistently revealed a
correlation between e-resource use, book
borrowing and student attainment
This appears to be the case across all
disciplines

14. Thoughts
• Does correlation equal causation?
• Do our stakeholders like this impact data?

15. Qualitative Assessment
• Numbers are only ever half of the story
• Customer feedback can provide more insight that
data alone
• Customer quotes can have great impact on
stakeholders

16. The Library now has access
to 15,586 full text electronic
journals and 9,354 eBooks
When I started my distance learning
MSc it was the first time I had accessed
a digital library with such resources; you
have played a major positive role in my
effective learning.

17. Data Management
• Data Management is an important part of the
planning cycle
• Assessment management systems
• Qualitative coding software
• MS Excel / MS Access

18. Considerations
• Bring both the Qualitative and Quantitative together
• Collaborate with local experts
• Not all news will be good news
• Be realistic on what is achievable
How many staff-hours were spent
counting statistics this year?

19. Common Challenges
• Where do I start?!
• Finding the time
• Getting support from all Library staff
• Requesting data from other units
• Finding other data sets
• Difficulty in analysing and interpreting data
• Fear of change
• What if I fail?

21. Source: Megan Oakleaf, 2011,

22. Source: Megan Oakleaf, 2011,

23. Source: Megan Oakleaf, 2011,

24. Reporting Back
• Know your audience
• What do they care about?
• What is their level of understanding?
• Will they feel threatened by the data?
• Tell a story with the data
• Begin with the outcomes in mind
• Be mindful of the data limitations

25. Case Study: McMaster University
http://library.mcmaster.ca/library-scorecard

27. References
• Institutional Return on Investment. University of West Florida.
http://libguides.uwf.edu/content.php?pid=188487&sid=2184200
• Library Scorecard: Strategic Objectives, Measures & Initiatives 2011/2012. McMaster
University Library, revised April 23, 2012, http://library.mcmaster.ca/library-scorecard
• Library Value Calculations. Cornell University Library Research & Assessment Unit.
http://research.library.cornell.edu/value
• Oakleaf, M. and Matthews, J. (2011). Assessing the Impact of the Academic Library.
Presented at the 9th Northumbria International Conference on Performance
Measurement in Libraries and Information Services. York, England. 22nd – 26th August.
• Stone, G. (2011). The Library Impact Data Project: hit, miss or maybe. Presented at the
9th Northumbria International Conference on Performance Measurement in Libraries
and Information Services. York, England. 22nd – 26th August.

28. Thank you!
Selena Killick
s.a.killick@cranfield.ac.uk
Twitter: @SelenaKillick
Tel: +44(0)1793 785561