Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
LIRG SCAN AWARD 2010 A REVIEW OF EXISTING QUANTITATIVE AND QUALITATIVE EVALUATION METHODOLOGIES FOR DEMONSTRATING THE VALU...
<ul><li>To define value </li></ul><ul><li>To present an overview of current methods for measuring performance </li></ul><u...
How do we define value? “ ... defining value  in the context of libraries is  complex , individual stakeholders are  uniqu...
Value is not fixed... <ul><ul><li>Personal, educational, professional circumstances </li></ul></ul><ul><ul><li>Seven Ages ...
 
Quantitative Methodologies
Audits... <ul><li>CIPFA Public Library Statistic Actuals Report </li></ul><ul><ul><li>Library managers complete a report a...
Pros & Cons... <ul><li>“ ...quantity of use and quality of performance do not yet prove that users benefitted from their i...
Return on Investment Studies (ROIs) <ul><li>Cost-Benefit Analysis (CBA) </li></ul><ul><ul><li>Enables quantifiable values,...
Input-Output Models (IOMs) <ul><ul><li>Evaluating  indirect   benefits  (e.g. impact that library has on the local economy...
Combining CBAs and IOMs... <ul><li>Suffolk Cooperative Library System (SCLS) </li></ul><ul><li>Benefit-to-cost-ratio metho...
Library Use Valuation Calculator... (Chelmsford Public Library, 2010)
 
But what about social value...? <ul><ul><li>(Rooney-Browne, 2009b).  </li></ul></ul>
Qualitative Methodologies
Social Impact Audit...
Ethnography... “ Ethnography is the art and science of describing a group or culture......the ethnographer is both storyte...
Tracking Value... <ul><li>The Engaged Library:  Chicago  Stories of Community Building </li></ul><ul><li>Prove that public...
Tracking Value... (UKOC, 2008, p.4).
Profiling... People behind the Numbers: Janet (UKOK, 2010, p.38).  <ul><li>Profiling  has enabled UKOC to translate facele...
Multiple Method Approaches...
Seattle Central Library  Economics Benefits Assessment... (SCL, 2005, p.10).
Methodology & Results... <ul><li>CBA  methods to estimate economic impact </li></ul><ul><li>Case studies, visitor and user...
Social Return on Investment ...the future? <ul><li>“ ...whether the capital provided is generating meaningful, real return...
Measuring SROI... <ul><li>Compares net benefits of a project to investment (financial & time) </li></ul><ul><ul><li>Social...
Conclusion... <ul><li>Perfect methodology does not exist but there are many possibilities </li></ul><ul><li>No general con...
 
THANK YOU CONTACT  ME: E-MAIL: [email_address] BLOG:  www.libraryofdigress.co.uk Twitter:  www.twitter.com/ librarydigress...
Upcoming SlideShare
Loading in …5
×

A Review of Methods for Measuring the Value of Public Libraries

4,059 views

Published on

Published in: Education, Technology
  • Wonderful slide show of the Umbrella session.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

A Review of Methods for Measuring the Value of Public Libraries

  1. 1. LIRG SCAN AWARD 2010 A REVIEW OF EXISTING QUANTITATIVE AND QUALITATIVE EVALUATION METHODOLOGIES FOR DEMONSTRATING THE VALUE OF PUBLIC LIBRARIES Christine Rooney-Browne PhD Researcher, University of Strathclyde, Glasgow Umbrella Conference 2011
  2. 2. <ul><li>To define value </li></ul><ul><li>To present an overview of current methods for measuring performance </li></ul><ul><li>To discuss quantitative and qualitative evaluations methodologies </li></ul><ul><li>To identify examples of successful library valuation projects </li></ul><ul><li>To introduce potential methods for measuring value from the non-profit sector </li></ul>Objectives...
  3. 3. How do we define value? “ ... defining value in the context of libraries is complex , individual stakeholders are unique , performance measurement is essentially spatial , and operating in an environment that is neither causal nor predictive creates complications” (Cram, 1999, p. 1).   © Christine Rooney-Browne (2009).
  4. 4. Value is not fixed... <ul><ul><li>Personal, educational, professional circumstances </li></ul></ul><ul><ul><li>Seven Ages of Library Use (Bohme and Spiller, 1999): </li></ul></ul><ul><li>Times of crisis: </li></ul><ul><ul><li>War, Recession, Natural Disasters (New Orleans) </li></ul></ul>“ Libraries can be valued in many different ways, from hard dollars to intangibles like community goodwill and historical significance ” (Elliott, 2005).
  5. 6. Quantitative Methodologies
  6. 7. Audits... <ul><li>CIPFA Public Library Statistic Actuals Report </li></ul><ul><ul><li>Library managers complete a report about expenditure and income, staff levels, service points, stock levels, issues, enquiries, visits, inter-library loans </li></ul></ul><ul><ul><li>Library & Information Statistics Unit (LISU) summarises key statistics from this report in their Libraries, Archives, Museums, Publishing, Online Statistics Table (LAMPOST) </li></ul></ul>(LISU, 2010)
  7. 8. Pros & Cons... <ul><li>“ ...quantity of use and quality of performance do not yet prove that users benefitted from their interaction with a library. Measuring impact or outcome means going a step further and trying to assess the affect of services on users and on society” (Poll and Boekhorst, 2007, p. 31). </li></ul><ul><li>How well the library is performing statistically </li></ul><ul><li>Statistics can be invaluable (budgets, staffing, stock) </li></ul><ul><li>Enables comparisons (league tables) </li></ul><ul><li>Disparity in funding </li></ul><ul><li>Geographical location </li></ul><ul><li>Overlook contribution to lives of individuals, local community and economy </li></ul>
  8. 9. Return on Investment Studies (ROIs) <ul><li>Cost-Benefit Analysis (CBA) </li></ul><ul><ul><li>Enables quantifiable values, such as cost or purchase price, to be applied to variables that are difficult to measure; used to measure direct benefits </li></ul></ul><ul><li>Consumer Surplus </li></ul><ul><ul><li>the value that consumers place on the consumption of a good or service in excess of what they must pay to get it </li></ul></ul><ul><li>Cost of Time and Effort </li></ul><ul><ul><li>measures time and effort expended by users </li></ul></ul><ul><li>Contingent Valuation (CV) </li></ul><ul><ul><li>measures the value of use and non-use of non-priced goods and services (e.g. public libraries) </li></ul></ul><ul><ul><li>Willingness to Pay (WTP) </li></ul></ul><ul><ul><li>Willingness to Accept (WTA) </li></ul></ul>
  9. 10. Input-Output Models (IOMs) <ul><ul><li>Evaluating indirect benefits (e.g. impact that library has on the local economy) </li></ul></ul><ul><ul><li>Benefits measured using mathematical software models which looks at cause and effect relationships (e.g. Regional Input-Output Modelling System II and the Regional Economic Models, Inc (REMI)). </li></ul></ul><ul><ul><li>Have been used in the past to measure the value of beaches and parks. </li></ul></ul>“ ...if we can demonstrate our worth, with numbers, our budget numbers will be all the more justifiable” (Finch and Warner, 1998, p.158).
  10. 11. Combining CBAs and IOMs... <ul><li>Suffolk Cooperative Library System (SCLS) </li></ul><ul><li>Benefit-to-cost-ratio methods to measure direct benefits: </li></ul><ul><li>Total Value of library services ÷ Tax dollars supporting service: </li></ul><ul><li>$509,415,038 ÷ $131,647,566 = $3.87 : 1 benefit/cost ratio </li></ul><ul><li>For every $1 invested the library returned $3.87 in direct benefits </li></ul><ul><li>Using input-output model called RIMS II to measure indirect benefits: </li></ul><ul><ul><li>Library generated $26 million in goods and services </li></ul></ul><ul><ul><li>Library enabled local earnings to increase by more than $50 million </li></ul></ul><ul><ul><li>Created more than 1,200 jobs for the local economy </li></ul></ul><ul><ul><li>Total multiplier effect of SCLS spending = $232 million . </li></ul></ul><ul><li>(Imholz and Arns, 2007, p. 19). </li></ul>
  11. 12. Library Use Valuation Calculator... (Chelmsford Public Library, 2010)
  12. 14. But what about social value...? <ul><ul><li>(Rooney-Browne, 2009b). </li></ul></ul>
  13. 15. Qualitative Methodologies
  14. 16. Social Impact Audit...
  15. 17. Ethnography... “ Ethnography is the art and science of describing a group or culture......the ethnographer is both storyteller and scientist; the closer the reader of an ethnography comes to understanding the native’s point of view, the better the story and the better the science” (Fetterman, 1998, pp. 1-2).
  16. 18. Tracking Value... <ul><li>The Engaged Library: Chicago Stories of Community Building </li></ul><ul><li>Prove that public libraries build social capital </li></ul><ul><li>Identify & connect the library’s assets to the community </li></ul><ul><li>Assess & strengthen the library’s connections with and use of community assets </li></ul><ul><li>Produce a toolkit for other libraries to adopt to </li></ul><ul><li>Mapping tools to perform an inventory services, identify areas for improvement and highlight library’s contribution to the community’s wider social, educational, cultural and economic goals. </li></ul>(Urban Libraries Council, 2006, pp.32-42).
  17. 19. Tracking Value... (UKOC, 2008, p.4).
  18. 20. Profiling... People behind the Numbers: Janet (UKOK, 2010, p.38). <ul><li>Profiling has enabled UKOC to translate faceless data into outcomes that are “tangible, practicable and workable” (UKOC, 2010, p.5). </li></ul><ul><li>Stakeholders and funders might be able to relate easier and have empathy for ‘Janet’ rather than a ‘C2DE female’. </li></ul>
  19. 21. Multiple Method Approaches...
  20. 22. Seattle Central Library Economics Benefits Assessment... (SCL, 2005, p.10).
  21. 23. Methodology & Results... <ul><li>CBA methods to estimate economic impact </li></ul><ul><li>Case studies, visitor and user surveys </li></ul><ul><li>Interviews with local businesses, developers and representatives from tourism </li></ul><ul><li>Analysis of visitor and usage statistics. </li></ul><ul><li>Generated $16 million in net new economic activity in its first year of operation </li></ul><ul><li>Projections next twenty years: $80m (5yrs), $155m (10 years) or $310m (20 yrs) </li></ul><ul><li>improves desirability of downtown area & Seattle as tourist destination </li></ul><ul><li>An icon </li></ul><ul><li>Findings used as an advocacy tool </li></ul>
  22. 24. Social Return on Investment ...the future? <ul><li>“ ...whether the capital provided is generating meaningful, real returns--returns for the manager, the investor and society as a whole” (Emerson, 2004). </li></ul><ul><li>Roberts Enterprise Development Fund (REDF) </li></ul><ul><ul><li>venture philanthropists working with non profits to create job opportunities for disenfranchised </li></ul></ul><ul><ul><li>Created SROI Model (2000): </li></ul></ul><ul><ul><ul><li>Measure success...achieving goals...inform decisions...convince others of our impact </li></ul></ul></ul><ul><ul><ul><li>(Javitis, 2008, p.1). </li></ul></ul></ul>
  23. 25. Measuring SROI... <ul><li>Compares net benefits of a project to investment (financial & time) </li></ul><ul><ul><li>Social Return Ratio (SRR): by combining net social benefits with cash flow of the business then dividing by the total value of the philanthropic investment </li></ul></ul><ul><ul><li>SROI Rate: by carrying out an Internal Rate of Return (IRR ) derived from total socio-economic value and total costs (Emerson and Cabaj, 2000, p. 11) </li></ul></ul><ul><li>The New Economics Foundation, London Business School & Small Business Service produced a SROI Primer: </li></ul><ul><ul><li>Helps organisations identify and measure social outputs, outcomes and impacts </li></ul></ul><ul><ul><li>Provides advice to help organisations use SROI Model to translate impacts into values (NEF, 2004). </li></ul></ul>
  24. 26. Conclusion... <ul><li>Perfect methodology does not exist but there are many possibilities </li></ul><ul><li>No general consensus as to the ideal model for measuring value </li></ul><ul><li>Competition for funding is fierce </li></ul><ul><li>Cuts are inevitable </li></ul><ul><ul><li>Find the methodology that best fits your project and your objectives </li></ul></ul><ul><li>“ In today’s climate of accountability, a better understanding of the value of public libraries is becoming essential to preserving and encouraging public and private investment” </li></ul><ul><li>(Imholz and Arns, 2007, p.12). </li></ul>
  25. 28. THANK YOU CONTACT ME: E-MAIL: [email_address] BLOG: www.libraryofdigress.co.uk Twitter: www.twitter.com/ librarydigress Voices for the Library: http://www.voicesforthelibrary.org.uk/wordpress/

×