Vancouver Metrics Symposium

2. The Green Green Grass of Home: The Environment
• Association of Research Libraries
• Collections based
• Rankings “Goodness” of the Library
based on Collections and other input
factors
• Culture of Assessment - Statistics – Metrics
– Analytics –
• Statistics, statistics, statistics…

3. Changing Pastures: The Environment
• New Role for academic libraries
– Research Productivity/Output
• Reputation
• Funding
Research Support
– Student success
• Job placement
• Retention and completion
• Student experience
Teaching and Learning Support

4. Changing Pastures: The Environment
• Outcome and Impact measures linked to
the institutional strategic plan
• Demonstrating Value
• Balanced Scorecard
• Responsibility Centre Budgeting

5. The Environment
• Input Measures
– expenditures, collection size, number of staff
• Investment Index/Ranking
• Collection Index/Ranking
» # Journals
» Holdings
–Should be linked to strategic plan-
strategies/activities/operational plan

6. The Environment
• Output & Activity Measures
– reference activity**
– circulation & “hits”/downloads & interlibrary loans
– Gate counts
– Sessions/attendees
– # items uploaded to institutional repository
– # research data elements deposited
– Should be linked to strategic plan-
strategies/activities/operational plan

7. The Environment
• Outcome Measures
– Should be linked to strategic plan
– User focused – measuring change in client
• Impact Measures
– Should be linked to strategic plan
– Our contribution to the overall good

8. The Environment
Library intervention in information literacy
–Planned in advance
–Harder to collect – research ethics boards
–Research is often not truly or easily generalizable
–Research may not demonstrate causality
–Be prepared for results we don’t like
• Before and after surveys (outcome)
• Efficacy inventories (outcome)
• GPA studies, quality assessments (impact)
• Perception studies (impact)

9. Romancing the Wolf: Longitudinal Data
• Benefits of longitudinal data
– Comparison over time – improvement or
failure to make a difference
– Helps to identify hidden change and
understand our users or our environment
… but…
- Can inhibit change
- Can inhibit inforce undesirable behaviours

10. Romancing the Wolf: Longitudinal Data
• ARL collections data
• Highly respected rankings – stakeholders
– Years of work to respect the value of library
collections and preserve funding for those
collections
– Advent of electronic resources, potential for
shared repositories and collaborative
collection building

11. Romancing the Wolf: Assessment & Metrics

12. Romancing the Wolf: Taking Measures for Granted
• Understand the underlying data
– What is included
– What isn’t included
• Understand the potential uses
– How is it intended to be used
– How might it be used or interpreted
– Will it be used to make a decision or change
a course of action ?

13. Library Impact
Library
Collections
System of
Libraries

14. Romancing the Wolf: Taking Measures for Granted
• Setting realistic expectations
• What is acceptable?
• Understanding the population
• When what looks bad is actually good!

15. Sticking with the Flock: Adding Measures
• Strategic plans often completed without
contemplation of outcomes or
measurement activities
• Measures added as an afterthought
usually count activity
• Impact and Outcomes considered first

16. Using Indicators:
• Is there the research to support our use of
proxies and indicators?
– Collections strength as a proxy for research
support ?
– Is GPA a good proxy for student success?
– Beer & Pizza test?

17. Needs versus Wants
• “If I had asked people what they wanted they
would have said faster horses”
(Henry Ford)
• Frame of reference dictates the realm of
possibilities (Jobs, Einstein)
• Professional Judgment
• Outcomes for needs related services have
relevance beyond liking or happiness

18. Title of presentationumanitoba.ca