LIBRARY METRICS
AND MEASUREMENT:
COUNTING WHAT COUNTS &
MAKING IT MATTER
Emily Clasper
Wild Wisconsin Winter Web Conferenc...
Today’s Big Question
http://farm6.staticflickr.com/5478/11442225495_0ff925eda3_o.jpg
Library Statistics









Circulation Transactions
Door Count
Reference Questions
Web site visits
Program Attenda...
The Panicked Call…
We’ve been doing
so well! How can
our stats all be
down?

http://farm9.staticflickr.com/8006/7706143020...
Depending on Our Numbers









Securing Funds
Justifying expenses
Complying with mandates
Gaining support from
st...
Moving Beyond Circ Stats

http://connecttocrlibrary.files.wordpress.com/2010/12/graph.jpg
Asking The Right Questions

http://farm3.staticflickr.com/2318/2201907500_6f0741be40_o.jpg
The Wrong Question

We bought a 3-D printer
and it seems to get a lot
of use, but circulation
down!

http://static.flickr....
The Wrong Question

That long jumper seems
like a really strong
athlete, but she doesn’t
run very fast!

http://farm5.stat...
Asking New Questions


Is this a question that we can



Will the answer lead to new



Does this knowledge help us ass...
Go Back to Your Goals

http://farm3.staticflickr.com/2773/4318924726_85eae8b991_o.jpg
Outputs vs. Outcomes

http://farm5.staticflickr.com/4040/5143354092_e15831241a_b.jpg

http://static.flickr.com/2565/381023...
Outcomes Measurement
Process

Inputs

Activities

Library Perspective:
What Library
Does/Provides

Results of Process

Out...
What Defines Success?
What is it that we’re
actually trying to
DO?
How will we know if
we’ve accomplished
that goal?
Data Collection

http://static.flickr.com/2423/3971932491_27e5acf65b.jpg
What Data Do We Need?

http://farm9.staticflickr.com/8514/8450658347_06c56211f9_b.jpg
Key Performance Indicators







Measureable
Indicates progress towards goals
Actionable
Measured frequently and con...
12am-1am
3-4 am
4-5 am
5-6 am
6-7 am
7-8 am
8-9 am
9-10 am
10-11 am
11am-12pm
12-1 pm
1-2 pm
2-3 pm
3-4 pm
4-5 pm
5-6 pm
6...
KPIs as Strategic Goals








By 2015, 80% of users surveyed will indicate that
they waited no longer than three min...
Qualitative Measures

http://media-cache-ec0.pinimg.com/736x/5d/d2/f8/5dd2f8b2dae90faaf22a99548bf11da2.jpg
Standards

http://farm7.staticflickr.com/6049/6372676915_b651c4614d_o.jpg
Open Data

http://static.flickr.com/5002/5265955179_4a33d2f2cb.jpg
Available Tools

http://www.ronmartin.net/blog/wp-content/uploads/2009/07/door-counter.jpg http://smallbiztrends.com/wp-co...
Available Tools

http://online.wsj.com/news/articles/SB1000142405270230333290457923040103082772
Questions for Data Collection







Are we counting effectively?
What structures do libraries have in place for
gathe...
Use of Tools

http://farm4.staticflickr.com/3119/3132662146_b305c1c058_b.jpg
Accessing and Storing Data

http://farm4.staticflickr.com/3245/2294144289_3102e0de26_o.jpg
Analysis and Interpretation

http://farm3.staticflickr.com/2173/2121472112_8ac5d673ff_b.jpg
Now That We Have The Data…

http://static.flickr.com/6017/5961260280_f38d3cbee9.jpg
Drawing Conclusions

http://farm9.staticflickr.com/8389/8597760704_70f681d096_k.jpg
What Does This Really Mean?

http://farm3.staticflickr.com/2891/9354301543_30e405da0a_b.jpg
Using Data for Good, Not Evil

http://farm4.staticflickr.com/3674/10868394285_6a10dc9e99_b.jpg
Making it Matter

http://static.flickr.com/5213/5397741531_84b940fea1.jpg
Using and Presenting Data

http://farm9.staticflickr.com/8330/8436405977_d0613c3cac_b.jpg
Making Data Easy to
Understand

http://farm6.staticflickr.com/5144/5659510216_657431c231_b.jpg
Making Data Relatable

http://farm3.staticflickr.com/2842/10950781743_5ffecf486c_o.png
Making Decisions

http://www.nytimes.com/newsgraphics/2014/01/05/poverty-map/
Making Connections

http://farm8.staticflickr.com/7018/6785112263_1bdc801df8_o.png
Making a Point

http://farm9.staticflickr.com/8192/8112404383_669f0c715a_h.jpg
Making an Impression

http://farm2.staticflickr.com/1128/585861831_c3647f199d_b.jpg
Telling a Story

http://media-cacheak0.pinimg.com/736x/43/80/84/438084e70887229c890b9deae32a0db1.jpg

http://media-cacheec...
Promoting a Brand

http://obieosobalu.files.wordpress.com/2011/01/history-of-microsoft-final.jpg http://www.coca-colacompa...
http://www.thisblogrules.com/wp-content/uploads/2011/07/lib-550x317.jpg

http://www.heroicpixels.com/wpcontent/uploads/201...
Turning Outcomes Into Impact

http://media-cache-ec0.pinimg.com/736x/6d/23/29/6d232935bd4317fdf3547144ebad6dba.jpg
Moving Forward

http://farm5.staticflickr.com/4062/4529934744_98a8f4bcc7_o.png
Questions for Libraries






What areas of measurement might we get into in
response to changing organizational goals?...
Do We Have the Skills?

http://farm8.staticflickr.com/7171/6632168217_bdc32df214_b.jpg
Do We Have The Tools?

http://farm1.staticflickr.com/113/286709039_105881e4b9_o.jpg
Can We Make it Matter?

http://media-cache-ec0.pinimg.com/736x/6d/23/29/6d232935bd4317fdf3547144ebad6dba.jpg
Recommended Reading
•

Brophy, Peter. 2006. Measuring Library Performance: Principles and Techniques. London: Facet.

•

H...
EMILY CLASPER
HTTP://BIT.LY/CLASPERWWWC14
http://eclasper.com
@eclasper
Upcoming SlideShare
Loading in...5
×

Library Metrics and Measurement: Counting What Counts & Making it Matter

201

Published on

Wild Wisconsin Winter Web Conference 2014
Wednesday, January 15 2:30 - 3:30 pm
Register: https://www1.gotomeeting.com/register/278806136
#wwwc14

Published in: Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
201
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • This is a presentation riddled with questions, very few of which have concrete answers.
  • The public libraries I work with usually use combinations of these
  • But I’ve been getting more and more of this, lately
  • Many public Libraries depend on circ stats almost exclusively. In a world where these same liraries see over 50% of their circs come from DVDs and CDs, these libraries are in trouble.
  • We are doing ourselves a disservice by relying so heavily on Circ stats. We’re not telling our whole story. We’re not making our case effectively. We’re not arming ourselves with enough data to make truly informed decisions. Where do we go from here?
  • Start with the questions we using metrics to answer.
  • Actual question I got.
  • Not important to the competition at hand.When we rely on the same old data to measure new initiatives, it can lead us astray. We need to formulate questions that reflect on the goals we want to accomplish in a meaningful way.
  • When looking at what we want metrics to tell us, ask yourself: Is this a question that we can ANSWER? – scope, framingWill the answer lead to new KNOWLEDGE? - verifiableDoes this knowledge help us assess our PROGRESS? - Can this assessment be used to suggest ACTION? - Will the actions help lead us closer to our GOALS?
  • Strategic planning. Mission, Goals should drive everything. If goals are clearly identified, we can start asking questions that matter and determining new ways of answering those questions.
  • When considering the goals, we need to think about BOTH outputs and Outcomes.Outputs  What you’ve done (Make 100 Cupcakes)Outcomes  Impact of the Outputs (100 Happy, sugared up kids) – libraries usually focus on Outputs. It’s easier, funders ask for it, etc.Outcomes are harder to get at, but more in line with progressing towards our goals.
  • Outcomes measurement – see this excellent book for practical examplesEvaluation of both process and results. Results are harder to measure, but more important. ROI often focuses on Input/Output equation, which is valuable. However, it often fails to get at the true impact of the initiative. Resume writing assistance: Inputs: 100 hours staff time, specialized staff training Outputs: 35 resumes for jobless library membersOutcomes – interviews, jobs attained; customer satisfaction; community engagement and awareness
  • Come back to these questions. This leads to…
  • Deciding what to measure in order to do our ouput and Outcomes analyses?What can we get?What would indicate a successful outcome?
  • One way of organizing your data collection is to define KPIs in line with the strategic goals.Much documentation on this process. My own requirements…
  • Specific Circulation StatisticsComputer terminal, facilities, equipment usage Ratio of professional staff (FTE) per capitaWorkflow efficiency ratesCustomer satisfaction survey feedback
  • KPIs should be framed as specific goals with a defined timeframe. Assessment of progress can be measured against the degree of success in reaching these KPIs
  • Factors to consider in determining Data collection methodology:Qualitative measuresDon’t forget qualitative measures!How do we quantify the intangibles of our successes? Constant vigilance – don’t sacrifice quality. It is at the core of your mission.
  • Are there standards within the industry or community that can be applied?
  • What kind of data is available?Open data resources are multiplying!
  • Impacts what you decide to measure>ILS Tools – use and collection statisticsIndependent analysis products – GIS, demographic comparisonsObservational methods – “field work”
  • Hi Tech makes new measures possible: Movement Sensors (Measuring Behaviors: Attention, Movement, Location, Response) GeotrackingHow long did someone browse that section before making a selection? Issues of compiling and storing data – track length of time between visits, borrowing habits, behaviors – privacy issues
  • Effective measurement should not impact the data itself – hard with qualitative or subjective dataWhat are we doing already, or what can be adapted from current procedures>Look at models from inside and outside the industry!The ever-present issue of qualitative measurement
  • Love this data collection tool. Simple, easy, at point of service. Fallable, yes, but reasonably straightforward. We need to make sure that the tools we use are appropriate to the questio, data, and situation.
  • Data access and security, protecting patron privacy. Anonymization of data, security measures, privacy policies, staff training
  • Once the data has been collected, we need to do something with it. Often I see library directors and staff take a pile f statistics, survey results, etc. and then stop the process. They stare at the numbers, usually looking for a straightforward gain or decline. But what can this data really tell us?
  • Determination of value, implications for each measure (How much is a “Circ” worth in 2014?)Identification of trendsWhat skills are needed for this? What do we, as a profession, need to work on?
  • Popularity vs. Importance – More people attended the concert on the lawn than the college search seminar. Causation vs. Correlation
  • Gotta read the numbers right. Someone sent me a chart much like this one once, telling me that they had an exponential growth in new members. Nope. Reasonably steady new membership, growing overall membership. Time to cull out the inactive cards.
  • Use data responsibly. Guard privacy. Use to further favorable goals, not personal agendas.
  • Photo: Phil Bradley
  • Presentation of data connecting with stakeholdersMaking strategic decisionsMotivating staffHow can we better use statistics and measures to tell our stories and make our case? What can we do to make the data more meaningful to our stakeholders?
  • Bigger = MoreNot just a list of numbers
  • I know how big a soda can is. This is a measurement I can relate to. I can connect with this data because it means something real to me.
  • Compare with Library card data. Where should we focus more early literacy campaigns?Compare with results of recent Affordable Care Act info initiative. How successful were we in reaching target demographics?
  • Providing context.Social media = connections (I’ve talked little about social and Web analytics)
  • New York City's daily carbon dioxide emissions as one-ton spheresIllustrating your point to stakeholders. Envision something like this for circ stats instead of a number.
  • Make a bold statement to get results. We recently needed to make a point to our executive board about the complexity f our ILS setup. We made a chart of system elements that covered an entire wall… point taken. Action approved. How can you make an impression on funders, legislators, community members by translating metrics into something that they can’t ignore?
  • Telling that story. How can we show (not tell!) the impact of our services to those we need to sell it to. What is it that makes our library important to the community we serve?
  • More storytelling – What can we tell about the mission of an organization by looking at how they represent themselves via data?
  • Bottom line: Turning the outcome measurements into something that can make an impact. Using them to further our goals.
  • We’ve asked a lot of questions about library metrics, and brought up some concerns. So where do we go from here?
  • MORE QUESTIONS!To consider:
  • If not, were do we get them?
  • If not, what tools need to be developed? Can we engage in partnerships?
  • How can we use it to Change the World?
  • Library Metrics and Measurement: Counting What Counts & Making it Matter

    1. 1. LIBRARY METRICS AND MEASUREMENT: COUNTING WHAT COUNTS & MAKING IT MATTER Emily Clasper Wild Wisconsin Winter Web Conference 2014
    2. 2. Today’s Big Question
    3. 3. http://farm6.staticflickr.com/5478/11442225495_0ff925eda3_o.jpg
    4. 4. Library Statistics        Circulation Transactions Door Count Reference Questions Web site visits Program Attendance Collection Inventory Staff Actions http://farm7.staticflickr.com/6074/6033655868_6b53ef4ebb_b.jpg
    5. 5. The Panicked Call… We’ve been doing so well! How can our stats all be down? http://farm9.staticflickr.com/8006/7706143020_e533940f18_b.jpg
    6. 6. Depending on Our Numbers       Securing Funds Justifying expenses Complying with mandates Gaining support from stakeholders Demonstrating the value of services, ROI Data-driven decision making Allocation of our internal resources  Service development http://farm3.staticflickr.com/2602/3678747186_b63d82fe2c_o.jpg decisions 
    7. 7. Moving Beyond Circ Stats http://connecttocrlibrary.files.wordpress.com/2010/12/graph.jpg
    8. 8. Asking The Right Questions http://farm3.staticflickr.com/2318/2201907500_6f0741be40_o.jpg
    9. 9. The Wrong Question We bought a 3-D printer and it seems to get a lot of use, but circulation down! http://static.flickr.com/7062/6845000770_ebd5fb87d5.jpg
    10. 10. The Wrong Question That long jumper seems like a really strong athlete, but she doesn’t run very fast! http://farm5.staticflickr.com/4141/4817786416_282faf5324_o.jpg
    11. 11. Asking New Questions  Is this a question that we can  Will the answer lead to new  Does this knowledge help us assess our  Can this assessment be used to
    12. 12. Go Back to Your Goals http://farm3.staticflickr.com/2773/4318924726_85eae8b991_o.jpg
    13. 13. Outputs vs. Outcomes http://farm5.staticflickr.com/4040/5143354092_e15831241a_b.jpg http://static.flickr.com/2565/3810230521_35410d4b38.jpg
    14. 14. Outcomes Measurement Process Inputs Activities Library Perspective: What Library Does/Provides Results of Process Output Interim Outcomes Outcomes Goals User Perspective: Results of the User’s Experience Adapted from: Rubin, Rhea Joyce. 2006. Demonstrating Results: Using Outcome Measurement in Your Library. PLA Results Series. Chicago: American Library Association.
    15. 15. What Defines Success? What is it that we’re actually trying to DO? How will we know if we’ve accomplished that goal?
    16. 16. Data Collection http://static.flickr.com/2423/3971932491_27e5acf65b.jpg
    17. 17. What Data Do We Need? http://farm9.staticflickr.com/8514/8450658347_06c56211f9_b.jpg
    18. 18. Key Performance Indicators      Measureable Indicates progress towards goals Actionable Measured frequently and consistently May be objective or subjective
    19. 19. 12am-1am 3-4 am 4-5 am 5-6 am 6-7 am 7-8 am 8-9 am 9-10 am 10-11 am 11am-12pm 12-1 pm 1-2 pm 2-3 pm 3-4 pm 4-5 pm 5-6 pm 6-7 pm 7-8 pm 8-9 pm 9-10 pm 10-11 pm 11pm-12am Library KPIs Can Take Many Forms 250,000 200,000 150,000 100,000 50,000 0 http://www.bu.edu/library/files/2013/03/Faculty-Satisfaction-Services.bmp
    20. 20. KPIs as Strategic Goals     By 2015, 80% of users surveyed will indicate that they waited no longer than three minutes for staff assistance at a public service desk. By 2015, at least 85% of all purchased items will be invoiced, cataloged, processed, ad sent for shelving within two weeks of receipt. By 2016, the number of children aged 6-11 with library cards will increase from X to Y. Each year, a minimum of 80% of adults and teens who attend a training session on how to use technology to create content will indicate that the training session was very good or excellent.
    21. 21. Qualitative Measures http://media-cache-ec0.pinimg.com/736x/5d/d2/f8/5dd2f8b2dae90faaf22a99548bf11da2.jpg
    22. 22. Standards http://farm7.staticflickr.com/6049/6372676915_b651c4614d_o.jpg
    23. 23. Open Data http://static.flickr.com/5002/5265955179_4a33d2f2cb.jpg
    24. 24. Available Tools http://www.ronmartin.net/blog/wp-content/uploads/2009/07/door-counter.jpg http://smallbiztrends.com/wp-content/uploads/2011/02/rfid-large.jpg
    25. 25. Available Tools http://online.wsj.com/news/articles/SB1000142405270230333290457923040103082772
    26. 26. Questions for Data Collection     Are we counting effectively? What structures do libraries have in place for gathering performance data? What kind of models can we emulate for improving our data collection strategies? How do we get closer to gathering qualitative measures as well? http://farm3.staticflickr.com/2197/2200500024_33038620af_o.jpg
    27. 27. Use of Tools http://farm4.staticflickr.com/3119/3132662146_b305c1c058_b.jpg
    28. 28. Accessing and Storing Data http://farm4.staticflickr.com/3245/2294144289_3102e0de26_o.jpg
    29. 29. Analysis and Interpretation http://farm3.staticflickr.com/2173/2121472112_8ac5d673ff_b.jpg
    30. 30. Now That We Have The Data… http://static.flickr.com/6017/5961260280_f38d3cbee9.jpg
    31. 31. Drawing Conclusions http://farm9.staticflickr.com/8389/8597760704_70f681d096_k.jpg
    32. 32. What Does This Really Mean? http://farm3.staticflickr.com/2891/9354301543_30e405da0a_b.jpg
    33. 33. Using Data for Good, Not Evil http://farm4.staticflickr.com/3674/10868394285_6a10dc9e99_b.jpg
    34. 34. Making it Matter http://static.flickr.com/5213/5397741531_84b940fea1.jpg
    35. 35. Using and Presenting Data http://farm9.staticflickr.com/8330/8436405977_d0613c3cac_b.jpg
    36. 36. Making Data Easy to Understand http://farm6.staticflickr.com/5144/5659510216_657431c231_b.jpg
    37. 37. Making Data Relatable http://farm3.staticflickr.com/2842/10950781743_5ffecf486c_o.png
    38. 38. Making Decisions http://www.nytimes.com/newsgraphics/2014/01/05/poverty-map/
    39. 39. Making Connections http://farm8.staticflickr.com/7018/6785112263_1bdc801df8_o.png
    40. 40. Making a Point http://farm9.staticflickr.com/8192/8112404383_669f0c715a_h.jpg
    41. 41. Making an Impression http://farm2.staticflickr.com/1128/585861831_c3647f199d_b.jpg
    42. 42. Telling a Story http://media-cacheak0.pinimg.com/736x/43/80/84/438084e70887229c890b9deae32a0db1.jpg http://media-cacheec0.pinimg.com/736x/9f/c2/aa/9fc2aa3f178ff11634dfe236bd68b96c.jpg
    43. 43. Promoting a Brand http://obieosobalu.files.wordpress.com/2011/01/history-of-microsoft-final.jpg http://www.coca-colacompany.com/our-company/infographic-coca-cola-at-a-glance
    44. 44. http://www.thisblogrules.com/wp-content/uploads/2011/07/lib-550x317.jpg http://www.heroicpixels.com/wpcontent/uploads/2013/02/school_library_impact_blog13.jpg https://fbcdn-sphotos-e-a.akamaihd.net/hphotos-ak-ash3/1229853_10151835031660743_782695969_n.jpg
    45. 45. Turning Outcomes Into Impact http://media-cache-ec0.pinimg.com/736x/6d/23/29/6d232935bd4317fdf3547144ebad6dba.jpg
    46. 46. Moving Forward http://farm5.staticflickr.com/4062/4529934744_98a8f4bcc7_o.png
    47. 47. Questions for Libraries    What areas of measurement might we get into in response to changing organizational goals? How can we present the data we gather and the conclusions we draw from it to more effectively demonstrate success, promote the library and its goals, boost productivity, make more informed decisions, tell the library’s story to stakeholders? What skills do we, as Information Pros, Managers, and Advocates need to develop in order to effectively make analytics the powerful tool we need it to be?
    48. 48. Do We Have the Skills? http://farm8.staticflickr.com/7171/6632168217_bdc32df214_b.jpg
    49. 49. Do We Have The Tools? http://farm1.staticflickr.com/113/286709039_105881e4b9_o.jpg
    50. 50. Can We Make it Matter? http://media-cache-ec0.pinimg.com/736x/6d/23/29/6d232935bd4317fdf3547144ebad6dba.jpg
    51. 51. Recommended Reading • Brophy, Peter. 2006. Measuring Library Performance: Principles and Techniques. London: Facet. • Hernon, Peter, Robert E. Dugan, and Joseph R. Matthews. 2014. Getting Started with Evaluation. Chicago: American Library Association. • Ishak, Das. 2014. “Discovering the Right Key Performance Indicators in Libraries: A Review of Literatures.” Accessed January 14. http://www.academia.edu/1805631/Discovering_the_right_key_performance_indicators_in_libraries_a_ review_of_literatures. • Larkin, Richard. 2013. “Using Outcomes to Measure Nonprofit Success.” NPQ - Nonprofit Quarterly. July 2. http://www.nonprofitquarterly.org/management/22549-using-outcomes-to-measure-nonprofitsuccess.html. • Matthews, Joseph R. 2007. The Evaluation and Measurement of Library Services. Westport, Conn: Libraries Unlimited. • Rubin, Rhea Joyce. 2006. Demonstrating Results: Using Outcome Measurement in Your Library. PLA Results Series. Chicago: American Library Association. • Webbmedia Group. 20:12:02 UTC. “Key Performance Indicators For Libraries”. Education. http://www.slideshare.net/webbmedia/key-performance-indicators-for-libraries. • Wolf, Lesile. 2010. “What Gets Measured Gets Done: Key Performance Indicators.” California Digital Library. http://www.cdlib.org/cdlinfo/2010/09/15/what-gets-measured-gets-done-key-performanceindicators/. • Zappone, Marisa. 2013. “Why It Makes Sense for Nonprofits To Focus on Contribution, Not Causation.” Greenlights for Nonprofit Success. http://www.greenlights.org/blog/2013/09/16/why-it-makes-sense-for-
    52. 52. EMILY CLASPER HTTP://BIT.LY/CLASPERWWWC14 http://eclasper.com @eclasper
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×