Return on Investment
    (ROI) in Libraries
              Afternoon Session


     Universiti Malaysia Sarawak



                Joe Matthews
               September 28, 2012
Agenda

Financial impacts
 Return on Investment (ROI)
 Cost-Benefit Methodologies
 Direct Use Benefits
        Economic Impact Benefits
 Indirect Use Benefits
        Consumer Surplus Method
        Contingent Valuation Method
Agenda
ROI in Libraries
   Special
   Academic
   Public
   National
 What to do?
 Communicate Value
 Orr’s Fundamental Questions
Financia
l
Library Budgets




Competition for scarce resources
At the table or

On the menu?
Perspectives on Value
                                               Impacts
                     Direct                     Personal


                                               Organizational
           Use

                     Indirect                  Financial

Benefits
                     Option – Preservation of option for
                              future use by me

       Nonuse        Existence – Perceived value and
                     significance
                                  to the community
                     Legacy – Value of preservation for
                               future generations
Financial Impacts
• Direct use benefits
  – Cost savings
  – Access to other resources
  – Access to trained professionals
• Indirect use benefits
  – Estimated value
• Non-use benefits
  – Benefits that arise to you in the future
  – Benefits that arise to others in the
    future
What is ROI?
ROI Applications

 • Projects

 • Services

 • Organization
ROI in Library Contexts

• Demonstrating the value of libraries

• Evaluating existing services, collections,
  etc.

• Making the case for additional services or
  resources

• Recruiting support for a program or
  initiative
ROI Terminology
• Cost-Benefit Analysis (CBA)
  – Compares cost or purchase price with
    estimated value of variables that are difficult to
    measure
• Consumer Surplus
  – Value that consumers place on the
    consumption of a good or service in excess of
    what they paid for it
• Cost of Time and Effort
  – Measures time and effort expended by users
• Contingent Valuation
Cost/Benefit Methodologies

• Maximize the benefits for given costs
• Minimize the costs for a given level of
  benefits
• Maximize the ratio of benefits over costs
• Maximize the net benefits (present value
  of benefits minus the present value of
  costs)
• Maximize the internal rate of return
Ratio of Benefits to Costs

   Value of   benefits
          divided by

          Costs
   ROI = Benefit – Cost
          Cost
Challenge
   Estimating the
value of the benefits
Direct Use Benefits

• Cost savings – the “free” stuff

• Free or low-cost access to equipment,
  programs, meeting rooms, etc.

• Access to trained professional
  librarians
Identify a Competing Service

• Establish price of competing service

• Identify library’s annual volume or use

• Multiply to establish value of annual
  benefits
ROI 2.80:1
Challenge
What is the value of an item in a collection?

• Purchase price

• Purchase price + processing costs

• Discount the purchase price to 20 – 25 %
Challenge

• Difficult to identify all services



• Difficult to establish a value for some
  services
Generate Revenue

Ask the library user to:

• Estimate revenue generated

• Estimate costs saved

Library then calculates totals
Estimate Time Saved

Ask library customer to:

• Estimate time saved

• Multiple by hourly salary (plus
  benefits)

Library then calculates savings
Economic Impact Analysis


Compares local/regional economic activity
                of the library
  to the local/regional economic activity
             without the library
Economic Impact Analysis

• Salaries for library employees plus
  supplies/services purchased locally

• PLUS the multiplier or ripple effect

• The resulting data is entered into a
  input-output econometric model
A Destination
Major libraries and museums will attract
visitors from out-of-town. These visitors
spend money on:
  – Car rentals
  – Hotels
  – Restaurants
  – Tours
  – Etc.
Seattle Public Library
$16 Million Dollars Annually
Carnegie Library of Pittsburgh




    $9.8 to $15.6 Million
          Annually
Ripple Effect
Return on Capital Investment

   Valuing the physical assets of the
   library, including

     – Land
     – Buildings
     – Furniture
     – Equipment
     – Collections
$193.4 million ROI =
Indirect Use Benefits
Nonmarket Valuation
• Stated preference methods
  – Consumer surplus
  – Contingent valuation
  – Attribute-based methods
  – Paired choice comparisons
• Revealed preference methods
  – Travel cost models (Environmental quality)
  – Hedonic models (Property values)
  – Defensive behavior models (Offset effect of
    exposure)
Consumer Surplus

Monetary value consumers associate
     with a good or service in

            excess
   of any costs they incur to get it




   Data collected using a survey
Contingent Valuation
• Economic method of evaluation for non-
  priced goods and services

• Looks at the implications of not having the
  goods/services

• Identifies the cost to use alternative
  sources of information, should people
  choose to do so
Contingent Valuation

         Exercise

 Willingness-to-Pay (WTP)

Willingness-to-Accept (WTA)
Willingness-to-Pay (WTP)

Suppose that no libraries had ever existed
and taxes for libraries had never existed.
How many tax dollars or fees would your
  household be willing to pay annually
   to create and maintain your library
           as it exists today?
             $ __________
Willingness-to-Accept (WTA)

 Suppose that in the next election the ballot
   contained an option for closing all public
                    libraries.
 The budget savings would be used to lower
                      taxes.
 How much should the yearly tax saving be
in order for you to vote to close the libraries?
              $ _____________
Contingent Valuation Questions
• If there were no library, what would you do
  to obtain the information on this visit?
• How much time and money do you think it
  would take to find & use an alternative
  source?
• How far would you have had to travel to
  use this other source?
• Please estimate the additional costs of
  using this other source.
Contingent Valuation
          Challenges
• Depends on ability to pay
• Whose money? Yours or mine
• Immaterial goods (info) are undervalued
• Values influenced by the “warm glow”
  effect
• Are values suggested (multiple choice)
• WTP & WTA produce different results
• Telephone survey is long, costly
ROI in Libraries

            • Special

            • Public

            • Academic

            • National
Library Valuation Studies
     Country   National   State   Regional   Individual   Total

Australia         1                              1         2
Germany                                          1         1
New Zealand       1                                        1
Norway            1                                        1
South Korea                                      2         2
UK                2                                        2
USA                        7         8          19         34

      Totals      5        7         8          23         43
Special Libraries
              Library viewed as a:

• Asset that adds value for the organization

• Cost “sink hole” - so let’s get rid of it!
ROI in Special Libraries
• Time aspects

  – Manning, Texas Instruments        5.15:1

  – Griffiths & King        7.8:1 to 14.2:1

  – Harris & Marshall                 9:1

  – Strouse             $35 per library use
ROI in Special Libraries
• Cost aspects
  – Relative value
     • Griffiths & King – savings that resulted from the
       application of the information attained in reading
       17:1 to 26:1
     • Estabrook – 2:1 to 48:1
     • Koenig – 2.5:1 to 26:1


  − Consequential value – revenue increase, cost
    reduction
     − Strouse – Money saved: $42 per library use.
                   Revenue: $777 per library use
ROI in Medical Libraries

•   Assist in preventing patient deaths
•   Avoid surgery
•   Avoid additional tests
•   Reduce length of hospital stay
•   Avoid hospital admission
•   Change patient diagnosis
•   Change tests requested
•   Change prescriptions for
    medications
ROI in Banking

• Reduce risk of decisions

• Exploiting new business
  opportunities

• Saving time

• Avoiding loss of funds
US Dept. of Transportation

 • Reduces costs

 • Saves time

 • Improves decision making

 • Improves customer satisfaction
      (the drivers on the road)
ROI in Academic Libraries
Drexel University
University of Pittsburgh ROI
• If the library’s journal collection (print &
  electronic) were not available, faculty
  would use 250,000 hours and $2.1 million
  to find alternative sources for the articles


• It would cost the university 4.38 times
  the cost of the current library journal
  collections for the same amount of
  information gathering to be carried out
University of Illinois, Urbana-
         Champaign

 Connected citations to resources
 in
 the library’s collection to
 successful
 grant proposals, and the income
 the
 grants generated
ROI may provide us with calculations that
seek to document a financial relationship
between action and benefit, but too often
in the library community these studies are
poorly constructed, ineffectively
executed, and naïvely communicated.
And in the final analysis, do not respond
to the legitimate questions being raised
by our administrators and funders, and do
not advance the academic library as a
critical factor in institutional success.



                                James Neal
•   Comprehensive assessment of the library
•   ROI of the journal collection & readership
•   ROI for support of teaching & learning
•   ROI of digitized special collections
•   ROI of eBooks
•   Value of library commons
•   Bibliography
Bryant University – Faculty
            Access




Book ROI = 3.5:1
Article ROI = 3.2:1
Other publications ROI = 3.2:1
Syracuse University – ROI
               4.49:1
                              (Millions of Dollars)


                                               Faculty      Students
Economic
                  In Person                      $13.6         $23.1
                   Remote                         19.0          14.5
                     TOTAL                       $32.6         $37.6
Environmental
             Remote access                        $1.6          $3.7
           Read not-printed                           0.1        0.7
                     Social                            ?          ?
                     TOTAL                       $34.3         $42.0
Journal Collections & Reading
ROI for Support of Teaching &
          Learning
  Perceived Benefits –
  • Savings …
    – Of own time
    – Of own money
    – Of other resources – printing, copier
  • Improvements …
    – Teaching
    – Course-related materials
    – Student performance
ROI of Digitized Special Collections
 User
 • What is the value to a user in terms of
   time and money spent?
 Prestige
 • What is the prestige to the institution
   for high visibility digital special
   collection?
 Development
 • What value accrues to the
ROI of Digitized Special Collections
Environmental
• What is the value of the environmental
  savings from limited physical access to
  unique and often fragile material?
Scholars
• What value accrues from the role of special
  collections in attracting graduate students?
Collections
• What is the value of digital collection in
  attracting additional special collections?
ROI of eBooks
The Commons
ROI = 26:1
ROI in
 Public
Libraries
ROI in Public Libraries
• Value of direct use benefits usually
  determined by consumer surplus method

• Value of indirect use benefits determined
  by contingent valuation estimates

• Value of nonuse benefits determined by
  contingent valuation estimates
State of Florida
Total annual investment in Florida’s public libraries was $449 million

Total economic returns (direct and indirect) was $2.9 billion

       For every $1 invested the library returned $6.54 in direct benefits

Using input-output model called REMI to measure indirect benefits:


•   For every $6,448 spent on public libraries, 1 Job was created


•   For every dollar spent on public libraries, Gross Regional Product increased by
    $9.08
Suffolk Cooperative Library System

Benefit-to-cost-ratio methods to measure direct benefits:

         Total Value of library services Tax dollars supporting service:
         $509,415,038 $131,647,566 = $3.87 : 1 benefit/cost ratio

For every $1 invested the library returned $3.87 in direct benefits

Using input-output model called RIMS II to measure indirect benefits:


    • Library generated $26 million in goods and services
    • Library enabled local earnings to increase by more than $50 million

    • Created more than     1,200 jobs for the local economy
    • Total multiplier effect of SCLS spending = $232 million.
                                                     •
Carnegie Library of Pittsburgh
Consumer surplus method used to estimated direct benefits.

Library is the most visited regional destination


     For every $1 invested the library returned $3.00 in direct benefits

     $5.50 of benefits if only local tax dollars are considered.


Indirect benefit estimates suggested that:

•   Library provides some   700 jobs
•   $63 million in economic output is attributable to the library
•   $75 of benefits for every resident of the County
Philadelphia Free Library
          Economic value of library services that help Philadelphians:

Literacy
Learn to read and acquire working skills totals $21.8   million
Workforce Development
Locate job opportunities and develop career skills totals $6   million
979 people found jobs in one year ($30.4 million in earned income)
    generating $1.2 million annually in wage tax revenue for the city
Business Development
Develop or enhance their own businesses total $3.8 million
Value to Homes and Neighborhoods
Homes within ¼ mile of a library are worth $9,630 more than other homes
These homes produce an additional $18.5 million in property taxes
Charlotte Mecklenburg Library
Library Use Valuation Calculator
Priceless
Victoria Public
Libraries ROI = 3.56:1
Economic value
= 36% higher than budget


Economic benefit
= $ 4.24:1


Economic activity
= $ 2.82:1
New Zealand V+LM

 • Market price proxy

 • Replacement cost

 • Opportunity cost
ROI in Public Libraries
• Largest number of ROI studies

• Single library study, libraries with
  branches, & group of public libraries,
  state studies

• ROI ranges from a low of 1.02:1 to
  31.07:1
Social Return on Investment
           Models

           Expected Return =

Benefit X Probability of success   Cost
Social Return on Investment
              Models
• How are outcomes or benefits estimated?
  Over what timeframe?

• How are outcomes or benefits monetized?

• How are costs calculated?

• How are risks and uncertainties accounted
  for?
SROI Challenges
• The inconsistent use of language
• The lack of common measures in the
  social sector
• The lack of quality data on social
  impacts, outcomes, outposts, and cost
• The lack of incentives for transparency
• Unintended consequences
• Inadequate utilization
• The cost of measurement.
ROI in National Libraries
British Library




ROI = 4.4:1
Australian National Library




              ROI =
              2.0:1
New Zealand National Library


             ROI =
             3.50:1
National Library of Norway




      ROI = 4.0:1
New Zealand Parliamentary Library




             ROI ranges
             from
Latvia National Library

            WTP 3.88:1
            WTA 9.96:1
ROI Challenges


Lack of consistency in methodologies
limits the ability to replicate research,
    compare valuation results, and
      apply the research findings
Value of Benefits

• Materials for adults      35%

• Staff interactions        30%

• Materials for children   20%

• Information technology   15%
Library Valuation Frameworks

 • Marketing – promote communication
   between libraries and their
   stakeholders to affect the future state
   of a library

 • Evaluation – Describe the current state
   of the library and predict its future
   state
ROI
Strengths

  – Average ROI of $4:1 to $6:1
  – Wide variation in establishing a service value
  – Different user groups receive different levels of
    benefits
  – The value of a library will vary over time, as users
    and uses change
  – The consumer surplus method (or the shortcut
    method) has been used successfully by a number of
    libraries
  – Multiple methods lead to increased confidence in the
    results
ROI
Weaknesses
 – Assigning value for indirect (intangible)
   benefits is problematical
 – Comparing ROI studies is impossible
 – Variety of methodologies leads to variation
   in results
 – Policy-makers are uncertain of how to use
   the results
 – May involve large surveys (and costs)
 – ROI for small libraries may not be so good
Not only must a ROI study
       consider dollars and cents
            but it must also

make   dollars and sense
A great resource http://microsites.oii.ox.ac.uk/tidsr/welcome
What to do?
Good Assessment is Good
       Research
• An important question

• An approach to answer the
  question

• Data collection

• Analysis

• Report
Problems with Library
       Assessment
• Some studies correlate library use and
  retention, but no causative links

• Strength of correlations is weak – at
  best

• Problem with almost all existing library
  research are the small sample size

• Only a handful of research has been
“While satisfaction and service quality
measures like LibQUAL+ demonstrate
librarians commitment to user feedback,
they do not focus on the outcomes of
interaction with library services and
resources.”

                                Megan Oakleaf
Library Assessment


• Inward looking

• Focuses on collections and
  services


 This needs to change ….
Collaboration
Layers of Data

          Other University-wide Data
Student Surveys, Faculty Surveys, Alumni Surveys, …



               Performance Data
    Cumulative GPA, ACT score, CLA score, ….



              Demographic Data
  College, Level, Major, Gender, Ethnicity, Age, …



                  Library Data
    Circulation, Logins, Downloads, Reference,
                   Instruction, …
Privacy
    Use student IDs to match records from
    one data set to another and then

    Strip the student ID number from the
    combined record

    Work with Office of Institutional Research
&
    Institutional Review Board
• WCET project – data mining

• 6 institutions + 10 more (Phase 2)

• Analyzing 640,000 student records &
  more than 3 million course records
Next Steps
• Clearly understand the goals and
  concerns of top campus stakeholders
• Become engaged with the assessment
  process at your institution
• Partner with your campus Office of
  Institutional Research
• Create a library assessment plan
• Conduct “big picture” research
Next Steps
• Collect new data (individual students)
• Find “new” (& existing) data sets
• Combine existing data sets (library data &
  student information systems – registrar’s
  office)
• Use assessment management systems or
  a metric management system (LibPAS)
Past                                                  Future


        Building &      Focus         Engaging students &
        maintaining                         faculty
        collections
       Internal silos   Structure      University priorities



       Incremental      Change          Transformational


         Periodic        Outreach         Deliberative


         Top down        Decision            Shared
                         making


        Traditional     Measures of          Impact
                         Success
Library Value


• How integral it is to the community

• How well it supports learning and
  teaching

• How well it supports research
Communicating Value
Value Proposition




What your      What your
library        customers
does well      value
The Value Proposition

• An offer, not a demand

• Not what you value

• Only valuable perspective is the
  customer’s

• Valuable in a competitive environment
Or

The                 that a library makes to its
      promise
customers about what they can
                                   expect
      to receive in return for their time,
         their effort, their loyalty, and
           especially their dollars.
The library needs to …

       Focus on customers

and   what they want and need
           as well as

 how they want and need it.
Economi   Value
c
          Direct
Value     Indirect
Benefit   Nonuse
Stories + Stats =
Is the value in the glass, the wine
         or the savoring?
The financial crisis is looking even worse,
    but you will pleased to know that the
director reports that the library performance
         went up a half a point on the
     “library goodness scale” last week.

                               Michael Buckland
Joe@JoeMatthews.Org


  www.joematthews.org

          Joe Matthews
     Library Consultant
Cornell University Library Value

http://research.library.cornell.edu/value
The value is in
         the worth,
not in the number..

Roi perpun

  • 1.
    Return on Investment (ROI) in Libraries Afternoon Session Universiti Malaysia Sarawak Joe Matthews September 28, 2012
  • 2.
    Agenda Financial impacts  Returnon Investment (ROI)  Cost-Benefit Methodologies  Direct Use Benefits  Economic Impact Benefits  Indirect Use Benefits  Consumer Surplus Method  Contingent Valuation Method
  • 3.
    Agenda ROI in Libraries  Special  Academic  Public  National  What to do?  Communicate Value  Orr’s Fundamental Questions
  • 4.
  • 5.
  • 6.
    At the tableor On the menu?
  • 7.
    Perspectives on Value Impacts Direct Personal Organizational Use Indirect Financial Benefits Option – Preservation of option for future use by me Nonuse Existence – Perceived value and significance to the community Legacy – Value of preservation for future generations
  • 8.
    Financial Impacts • Directuse benefits – Cost savings – Access to other resources – Access to trained professionals • Indirect use benefits – Estimated value • Non-use benefits – Benefits that arise to you in the future – Benefits that arise to others in the future
  • 9.
  • 11.
    ROI Applications •Projects • Services • Organization
  • 12.
    ROI in LibraryContexts • Demonstrating the value of libraries • Evaluating existing services, collections, etc. • Making the case for additional services or resources • Recruiting support for a program or initiative
  • 13.
    ROI Terminology • Cost-BenefitAnalysis (CBA) – Compares cost or purchase price with estimated value of variables that are difficult to measure • Consumer Surplus – Value that consumers place on the consumption of a good or service in excess of what they paid for it • Cost of Time and Effort – Measures time and effort expended by users • Contingent Valuation
  • 14.
    Cost/Benefit Methodologies • Maximizethe benefits for given costs • Minimize the costs for a given level of benefits • Maximize the ratio of benefits over costs • Maximize the net benefits (present value of benefits minus the present value of costs) • Maximize the internal rate of return
  • 15.
    Ratio of Benefitsto Costs Value of benefits divided by Costs ROI = Benefit – Cost Cost
  • 16.
    Challenge Estimating the value of the benefits
  • 17.
    Direct Use Benefits •Cost savings – the “free” stuff • Free or low-cost access to equipment, programs, meeting rooms, etc. • Access to trained professional librarians
  • 19.
    Identify a CompetingService • Establish price of competing service • Identify library’s annual volume or use • Multiply to establish value of annual benefits
  • 21.
  • 22.
    Challenge What is thevalue of an item in a collection? • Purchase price • Purchase price + processing costs • Discount the purchase price to 20 – 25 %
  • 23.
    Challenge • Difficult toidentify all services • Difficult to establish a value for some services
  • 24.
    Generate Revenue Ask thelibrary user to: • Estimate revenue generated • Estimate costs saved Library then calculates totals
  • 25.
    Estimate Time Saved Asklibrary customer to: • Estimate time saved • Multiple by hourly salary (plus benefits) Library then calculates savings
  • 26.
    Economic Impact Analysis Compareslocal/regional economic activity of the library to the local/regional economic activity without the library
  • 27.
    Economic Impact Analysis •Salaries for library employees plus supplies/services purchased locally • PLUS the multiplier or ripple effect • The resulting data is entered into a input-output econometric model
  • 28.
    A Destination Major librariesand museums will attract visitors from out-of-town. These visitors spend money on: – Car rentals – Hotels – Restaurants – Tours – Etc.
  • 29.
    Seattle Public Library $16Million Dollars Annually
  • 30.
    Carnegie Library ofPittsburgh $9.8 to $15.6 Million Annually
  • 31.
  • 32.
    Return on CapitalInvestment Valuing the physical assets of the library, including – Land – Buildings – Furniture – Equipment – Collections
  • 33.
  • 34.
  • 36.
    Nonmarket Valuation • Statedpreference methods – Consumer surplus – Contingent valuation – Attribute-based methods – Paired choice comparisons • Revealed preference methods – Travel cost models (Environmental quality) – Hedonic models (Property values) – Defensive behavior models (Offset effect of exposure)
  • 37.
    Consumer Surplus Monetary valueconsumers associate with a good or service in excess of any costs they incur to get it Data collected using a survey
  • 38.
    Contingent Valuation • Economicmethod of evaluation for non- priced goods and services • Looks at the implications of not having the goods/services • Identifies the cost to use alternative sources of information, should people choose to do so
  • 39.
    Contingent Valuation Exercise Willingness-to-Pay (WTP) Willingness-to-Accept (WTA)
  • 40.
    Willingness-to-Pay (WTP) Suppose thatno libraries had ever existed and taxes for libraries had never existed. How many tax dollars or fees would your household be willing to pay annually to create and maintain your library as it exists today? $ __________
  • 41.
    Willingness-to-Accept (WTA) Supposethat in the next election the ballot contained an option for closing all public libraries. The budget savings would be used to lower taxes. How much should the yearly tax saving be in order for you to vote to close the libraries? $ _____________
  • 42.
    Contingent Valuation Questions •If there were no library, what would you do to obtain the information on this visit? • How much time and money do you think it would take to find & use an alternative source? • How far would you have had to travel to use this other source? • Please estimate the additional costs of using this other source.
  • 43.
    Contingent Valuation Challenges • Depends on ability to pay • Whose money? Yours or mine • Immaterial goods (info) are undervalued • Values influenced by the “warm glow” effect • Are values suggested (multiple choice) • WTP & WTA produce different results • Telephone survey is long, costly
  • 44.
    ROI in Libraries • Special • Public • Academic • National
  • 45.
    Library Valuation Studies Country National State Regional Individual Total Australia 1 1 2 Germany 1 1 New Zealand 1 1 Norway 1 1 South Korea 2 2 UK 2 2 USA 7 8 19 34 Totals 5 7 8 23 43
  • 46.
    Special Libraries Library viewed as a: • Asset that adds value for the organization • Cost “sink hole” - so let’s get rid of it!
  • 47.
    ROI in SpecialLibraries • Time aspects – Manning, Texas Instruments 5.15:1 – Griffiths & King 7.8:1 to 14.2:1 – Harris & Marshall 9:1 – Strouse $35 per library use
  • 48.
    ROI in SpecialLibraries • Cost aspects – Relative value • Griffiths & King – savings that resulted from the application of the information attained in reading 17:1 to 26:1 • Estabrook – 2:1 to 48:1 • Koenig – 2.5:1 to 26:1 − Consequential value – revenue increase, cost reduction − Strouse – Money saved: $42 per library use. Revenue: $777 per library use
  • 49.
    ROI in MedicalLibraries • Assist in preventing patient deaths • Avoid surgery • Avoid additional tests • Reduce length of hospital stay • Avoid hospital admission • Change patient diagnosis • Change tests requested • Change prescriptions for medications
  • 50.
    ROI in Banking •Reduce risk of decisions • Exploiting new business opportunities • Saving time • Avoiding loss of funds
  • 51.
    US Dept. ofTransportation • Reduces costs • Saves time • Improves decision making • Improves customer satisfaction (the drivers on the road)
  • 52.
    ROI in AcademicLibraries
  • 53.
  • 54.
    University of PittsburghROI • If the library’s journal collection (print & electronic) were not available, faculty would use 250,000 hours and $2.1 million to find alternative sources for the articles • It would cost the university 4.38 times the cost of the current library journal collections for the same amount of information gathering to be carried out
  • 55.
    University of Illinois,Urbana- Champaign Connected citations to resources in the library’s collection to successful grant proposals, and the income the grants generated
  • 58.
    ROI may provideus with calculations that seek to document a financial relationship between action and benefit, but too often in the library community these studies are poorly constructed, ineffectively executed, and naïvely communicated. And in the final analysis, do not respond to the legitimate questions being raised by our administrators and funders, and do not advance the academic library as a critical factor in institutional success. James Neal
  • 60.
    Comprehensive assessment of the library • ROI of the journal collection & readership • ROI for support of teaching & learning • ROI of digitized special collections • ROI of eBooks • Value of library commons • Bibliography
  • 61.
    Bryant University –Faculty Access Book ROI = 3.5:1 Article ROI = 3.2:1 Other publications ROI = 3.2:1
  • 62.
    Syracuse University –ROI 4.49:1 (Millions of Dollars) Faculty Students Economic In Person $13.6 $23.1 Remote 19.0 14.5 TOTAL $32.6 $37.6 Environmental Remote access $1.6 $3.7 Read not-printed 0.1 0.7 Social ? ? TOTAL $34.3 $42.0
  • 63.
  • 66.
    ROI for Supportof Teaching & Learning Perceived Benefits – • Savings … – Of own time – Of own money – Of other resources – printing, copier • Improvements … – Teaching – Course-related materials – Student performance
  • 67.
    ROI of DigitizedSpecial Collections User • What is the value to a user in terms of time and money spent? Prestige • What is the prestige to the institution for high visibility digital special collection? Development • What value accrues to the
  • 68.
    ROI of DigitizedSpecial Collections Environmental • What is the value of the environmental savings from limited physical access to unique and often fragile material? Scholars • What value accrues from the role of special collections in attracting graduate students? Collections • What is the value of digital collection in attracting additional special collections?
  • 69.
  • 70.
  • 71.
  • 72.
  • 73.
    ROI in PublicLibraries • Value of direct use benefits usually determined by consumer surplus method • Value of indirect use benefits determined by contingent valuation estimates • Value of nonuse benefits determined by contingent valuation estimates
  • 74.
    State of Florida Totalannual investment in Florida’s public libraries was $449 million Total economic returns (direct and indirect) was $2.9 billion For every $1 invested the library returned $6.54 in direct benefits Using input-output model called REMI to measure indirect benefits: • For every $6,448 spent on public libraries, 1 Job was created • For every dollar spent on public libraries, Gross Regional Product increased by $9.08
  • 75.
    Suffolk Cooperative LibrarySystem Benefit-to-cost-ratio methods to measure direct benefits: Total Value of library services Tax dollars supporting service: $509,415,038 $131,647,566 = $3.87 : 1 benefit/cost ratio For every $1 invested the library returned $3.87 in direct benefits Using input-output model called RIMS II to measure indirect benefits: • Library generated $26 million in goods and services • Library enabled local earnings to increase by more than $50 million • Created more than 1,200 jobs for the local economy • Total multiplier effect of SCLS spending = $232 million. •
  • 76.
    Carnegie Library ofPittsburgh Consumer surplus method used to estimated direct benefits. Library is the most visited regional destination For every $1 invested the library returned $3.00 in direct benefits $5.50 of benefits if only local tax dollars are considered. Indirect benefit estimates suggested that: • Library provides some 700 jobs • $63 million in economic output is attributable to the library • $75 of benefits for every resident of the County
  • 77.
    Philadelphia Free Library Economic value of library services that help Philadelphians: Literacy Learn to read and acquire working skills totals $21.8 million Workforce Development Locate job opportunities and develop career skills totals $6 million 979 people found jobs in one year ($30.4 million in earned income) generating $1.2 million annually in wage tax revenue for the city Business Development Develop or enhance their own businesses total $3.8 million Value to Homes and Neighborhoods Homes within ¼ mile of a library are worth $9,630 more than other homes These homes produce an additional $18.5 million in property taxes
  • 78.
  • 79.
  • 80.
  • 82.
  • 83.
    Economic value = 36%higher than budget Economic benefit = $ 4.24:1 Economic activity = $ 2.82:1
  • 84.
    New Zealand V+LM • Market price proxy • Replacement cost • Opportunity cost
  • 85.
    ROI in PublicLibraries • Largest number of ROI studies • Single library study, libraries with branches, & group of public libraries, state studies • ROI ranges from a low of 1.02:1 to 31.07:1
  • 88.
    Social Return onInvestment Models Expected Return = Benefit X Probability of success Cost
  • 89.
    Social Return onInvestment Models • How are outcomes or benefits estimated? Over what timeframe? • How are outcomes or benefits monetized? • How are costs calculated? • How are risks and uncertainties accounted for?
  • 90.
    SROI Challenges • Theinconsistent use of language • The lack of common measures in the social sector • The lack of quality data on social impacts, outcomes, outposts, and cost • The lack of incentives for transparency • Unintended consequences • Inadequate utilization • The cost of measurement.
  • 91.
    ROI in NationalLibraries
  • 92.
  • 93.
  • 94.
    New Zealand NationalLibrary ROI = 3.50:1
  • 95.
    National Library ofNorway ROI = 4.0:1
  • 96.
    New Zealand ParliamentaryLibrary ROI ranges from
  • 97.
    Latvia National Library WTP 3.88:1 WTA 9.96:1
  • 98.
    ROI Challenges Lack ofconsistency in methodologies limits the ability to replicate research, compare valuation results, and apply the research findings
  • 99.
    Value of Benefits •Materials for adults 35% • Staff interactions 30% • Materials for children 20% • Information technology 15%
  • 100.
    Library Valuation Frameworks • Marketing – promote communication between libraries and their stakeholders to affect the future state of a library • Evaluation – Describe the current state of the library and predict its future state
  • 101.
    ROI Strengths –Average ROI of $4:1 to $6:1 – Wide variation in establishing a service value – Different user groups receive different levels of benefits – The value of a library will vary over time, as users and uses change – The consumer surplus method (or the shortcut method) has been used successfully by a number of libraries – Multiple methods lead to increased confidence in the results
  • 102.
    ROI Weaknesses – Assigningvalue for indirect (intangible) benefits is problematical – Comparing ROI studies is impossible – Variety of methodologies leads to variation in results – Policy-makers are uncertain of how to use the results – May involve large surveys (and costs) – ROI for small libraries may not be so good
  • 103.
    Not only musta ROI study consider dollars and cents but it must also make dollars and sense
  • 104.
    A great resourcehttp://microsites.oii.ox.ac.uk/tidsr/welcome
  • 106.
  • 107.
    Good Assessment isGood Research • An important question • An approach to answer the question • Data collection • Analysis • Report
  • 108.
    Problems with Library Assessment • Some studies correlate library use and retention, but no causative links • Strength of correlations is weak – at best • Problem with almost all existing library research are the small sample size • Only a handful of research has been
  • 109.
    “While satisfaction andservice quality measures like LibQUAL+ demonstrate librarians commitment to user feedback, they do not focus on the outcomes of interaction with library services and resources.” Megan Oakleaf
  • 110.
    Library Assessment • Inwardlooking • Focuses on collections and services This needs to change ….
  • 111.
  • 112.
    Layers of Data Other University-wide Data Student Surveys, Faculty Surveys, Alumni Surveys, … Performance Data Cumulative GPA, ACT score, CLA score, …. Demographic Data College, Level, Major, Gender, Ethnicity, Age, … Library Data Circulation, Logins, Downloads, Reference, Instruction, …
  • 113.
    Privacy Use student IDs to match records from one data set to another and then Strip the student ID number from the combined record Work with Office of Institutional Research & Institutional Review Board
  • 114.
    • WCET project– data mining • 6 institutions + 10 more (Phase 2) • Analyzing 640,000 student records & more than 3 million course records
  • 115.
    Next Steps • Clearlyunderstand the goals and concerns of top campus stakeholders • Become engaged with the assessment process at your institution • Partner with your campus Office of Institutional Research • Create a library assessment plan • Conduct “big picture” research
  • 116.
    Next Steps • Collectnew data (individual students) • Find “new” (& existing) data sets • Combine existing data sets (library data & student information systems – registrar’s office) • Use assessment management systems or a metric management system (LibPAS)
  • 117.
    Past Future Building & Focus Engaging students & maintaining faculty collections Internal silos Structure University priorities Incremental Change Transformational Periodic Outreach Deliberative Top down Decision Shared making Traditional Measures of Impact Success
  • 118.
    Library Value • Howintegral it is to the community • How well it supports learning and teaching • How well it supports research
  • 119.
  • 123.
    Value Proposition What your What your library customers does well value
  • 124.
    The Value Proposition •An offer, not a demand • Not what you value • Only valuable perspective is the customer’s • Valuable in a competitive environment
  • 125.
    Or The that a library makes to its promise customers about what they can expect to receive in return for their time, their effort, their loyalty, and especially their dollars.
  • 126.
    The library needsto … Focus on customers and what they want and need as well as how they want and need it.
  • 131.
    Economi Value c Direct Value Indirect Benefit Nonuse
  • 132.
  • 135.
    Is the valuein the glass, the wine or the savoring?
  • 136.
    The financial crisisis looking even worse, but you will pleased to know that the director reports that the library performance went up a half a point on the “library goodness scale” last week. Michael Buckland
  • 139.
    Joe@JoeMatthews.Org www.joematthews.org Joe Matthews Library Consultant
  • 140.
    Cornell University LibraryValue http://research.library.cornell.edu/value
  • 141.
    The value isin the worth, not in the number..

Editor's Notes

  • #5 Flickr {e u g e n e } Financial Growth
  • #6 Libraries no longer being taken for granted as a “good thing”Increased availability of electronic resourcesGreater competition for finite resources (especially financial resources)Increased emphasis being placed on demonstrating value for moneyAre existing (or new) services of value?Value of the local collection is decliningGoogle images competition.jpgducharme.cc
  • #7 Is the library “at the table” with funding decision makersOr is it “on the menu”?
  • #10 What do you think? What is your definition of ROI?
  • #11 In short, ROI is a tool to help with decision-making in either planning or evaluation of services. Most often, it is a measure of the dollar amount of an investment compared to the estimated value in dollars and cents of the outcomes, outputs, or uses of the results. The most effective ROI analyses are, however, best described as stories that weave together qualitative and quantitative assessments of the value of the library and its services.
  • #12 Organizational level – ROI studies often called Library Valuation studiesIn financial circles, ROI often referred to as payback period, net present value, internal rate of return
  • #13 In planning, estimate of measurable outcomes expected to be achieved as a result of investment of resourcesIn evaluating performance, measurement of outcomes achieved as a result of investment of resources
  • #14 CBAConsumer surplus – cost of the vacation = X ringgits, value of the vacation experience = pricelessCost of Time and Effort – “Is it worth my time?”Contingent Valuation – more later
  • #15 Typically, analysis of competing projects- capital expenditure projectsROI is sometimes called ROCE – Return on Capital Employed
  • #16 In library studies, the ratio of benefits compared to costs is usedIn reality, it should be (value of benefits minus cost) divided by costs
  • #17 Ratio of benefits to costs – e.g. 4.50 to 1 or 4.50 Malaysian ringgits to 1 ringgit of cost (budget)For every ringgit supporting the library, the library sees a return on investment of X ringgitsThe challenge: Buying something (an alternative) is not the same thing as borrowing something from the library
  • #18 Estimating the value of the benefits – use (direct and indirect) and non-use value
  • #19 What is the connection?NYPL in 1922, DeWitt Wallace came in every day and read and wrote summaries of articles. – For more than 2 yearsBy 1929, it had 290,000 subscribersToday, 50 editions in 21 languages
  • #21 Shortcut method – Public Library
  • #22 University of Oregon and Cornell University
  • #28 Multiplier in some cases can be as high as 5-7
  • #29 Los Angeles’s J. Paul Getty MuseumNew York’s Public Library, American Museum of Natural History, MOMA, etc.The US Library of CongressThe British LibraryThe Huntington Library in Pasadena Calif
  • #30 Google images seattle‑public‑library.jpgplusmood.com
  • #31 Library most visited regional assetLibrary provides $75 worth of benefits for every resident of the CountyGoogle images carnegie‑library‑squirrel‑hill‑09‑600x450.jpgboringpittsburgh.com
  • #32 UK study looked at the economic impact on local businesses when a library was shut down on strike.Revenues went down 23 % Proctor, Usherwood and Sobczyk. What happens when a public library services closes down? Library Management, 18 (1), 1997, 59-64.
  • #33 Glen Holt and his colleaguesBaltimore County PL – 72%King County Library System – 94%Phoenix PL – 150%
  • #34 Michigan eLibrary (MeL) spent $3.9 million for online content for all residents in the state – saving the state $193.4 million82.7 million articles viewed or downloaded – estimated value @ $3 per article = 248 million (Cost = 5 cents per article)
  • #35 Flickr Kuala Lumpur : Petronas Twin Towers Jakir Ahmed [ Razu ]
  • #36 ALA Publishing - 2007
  • #37 Indirect Use Benefits & Non-Use BenefitsStated preference techniques are used by economists to estimate values when preferences cannot be revealed through the demand for goods and services at particular prices.Contingent valuation used to establish economic values and non-use values – range of technical and philosophical critiquesPaired choice comparisons can establish economic value of options and non-use valuesTravel costs – how much time are people willing to travel to visit a library – value of timeAttribute-based methods and paired comparisons – environmental compensation estimatesTravel cost method assumes that each user values their own time. The choice implicit in visiting a library (physically or virtually) rather than some other activity reflects the investment equal to the value of users time at the library. Travel time plus actual time at the library using the opportunity cost of this time, e.g., hourly wage rates or reported value of recreational time
  • #38 Survey costs can be high
  • #40 Your local Sungai Sarawak river
  • #44 Used to estimate Indirect use and non-use benefits“Warm glow” effect is the enjoyment derived when considering the benefits realized by the beneficiariesGenerally, WTA estimates are greater than WTP and are considered less reliableCosts can range from $20,000 to $100,000 +
  • #46 Public – 32Academic – 2Special – 2National – 5
  • #50 Quinn & Rodgers“Intangible” benefits obviously led to lowering costs (reduced hospital stay, no admission in the first place)Value of lifeSimilar results found by Marshall
  • #51 Joanne Marshall
  • #53 The Geisel Library at University of California San Diego (UCSD)Google images tumblr_ljt73g73W81qgn65ao1_500.jpgfuckyescalifornia.tumblr.com
  • #54 Challenge: How to translate capital cost savings (need for more space is diminished) to operational costs to support electronic resources
  • #55 Don King et al 2004 Contingent valuation ROI = 2.9.1:1
  • #56 Formula ? Convoluted –“Funding does not regenerate funding, but reputation does”TenopirPlus a study in Germany
  • #57 The calculations
  • #59 StoptheMadness:TheInsanityofROIandtheNeedforNewQualitativeMeasuresofAcademicLibrarySuccess.James G. NealACRL National Conference, March 30-April 2, 2011 Philadelphia, Pennsylvania.
  • #60 For-profit university – grown to over 100,000 students – all classes are taught onlineLibrary responsible for linking to online resources PLUSeBookstore (demands at least a 35% reduction in price for all eTextbooks and eliminates all shipping costs)APU ePress creates eTextbooks for high-demand classesOverall library achieves at least a 15:1 ROI
  • #61 Visit the project’s Web sitehttp://libvalue.cci.utk.edu/Comprehensive assessment of the library – Bryant University – Don King report Syracuse University – Bruce Kingma (surveys) Several NY universities – Megan Oakleaf – Web-based surveysROI of the journal collection & readership – selected US and UK universities – surveys, critical incident, open-ended Drexel UniversityROI for support of teaching & learning – Univ of Tennessee – survey, focus group, writing rubricROI of digitized special collections – Univ of TennesseeROI of eBooks – COUNTER stats, logbooks, surveys – Univ of Illinois, UCValue of library commons – Univ of Tennessee – surveys of undergrad and grad studentsProject bibliography
  • #62 Access to journal articlesPurpose of reading by faculty – 60% research, 17% teaching, 10% proposals, reportsPurpose of reading by students – 56% course assignment or paper, 20% required reading, 20% class reportFaculty do not get books from the library – 75% from other sourcesReference ROI = 1.50:1Also examined the costs and benefits of providing instruction and access to equipment, photocopiers, AV equipmentSurvey instruments included in the reportStudents do not rely on the library to read books – only 20% do so
  • #63 Faculty & student surveysBruce KingmaLIBValue Project ALA Midwinter 2012
  • #64 Annually, faculty spend 216 hours (27 8-hour days) reading articles, 148 hours reading books, and 84 hours reading other publications
  • #68 What are the values? Using contingent valuation survey and Google analytics
  • #70 eBooks offer value to the library in both a monetary way through documented “usefulness” to patrons – YMMD – Your mileage may differ - use is locally influencedeBooks are less expensive to own, circulate, maintain and preserve than print bookseBooks offer value to patrons, who “esteem” them due to:Accessibility and availability – 24/7, anywherePortabilitySearch and navigation capabilitiesRoom for improvement
  • #71 Survey of students and others who enterUniv of Tennessee library
  • #72 ROI for all of the State of Michigan’s Universities is quite highState funding of $1.5 Billion yield a benefit of $41 BillionYet, even this high ROI has done little to avoid the budget reductions in higher ed
  • #73 5 Recommendations
  • #75 Griffiths et al 2004Telephone survey of 883 users and nonusers; 1,505 in-library adult usersIn Pennsylvania – 1,128 telephone surveys, 2,500+ in-library visitor surveys, telephone surveys of 200+ organizationsUse of a REMI state input-output economic model
  • #76 *Circulation, reference transactions, programming, electronic resources multiplied by frequency of use (statistics derived from the New York State Annual Reports for Public and Association Libraries)**$ value that users would have been willing to pay for these services in the marketplace***Number of items multiplied by estimated market value.Overall, the combined method of CBA and IOMs led to a number of positive outcomes for SCLS:In the financial year following these results the SCLS received the highest amount of funding in its historyThe profile of SCLS was raised significantlySCLS were invited to work in partnership with other public service organisationsThe results of the study are still being used within the district as advocacy tools.This study inspired a number of similar smaller scale research projects within the state, all of which were supervised by Pearl Kamer (Kamer, 2006a, Kamer, 2006b, Kamer, 2006c); and all of which revealed similar positive outputs.
  • #77 Survey population of 1,300
  • #79 Combined direct & indirect range ROI for the library from 4.61:1 to 6.03:1Yet, suffered a 50% cut in funding
  • #80 Great marketing toolMakes customers feel good about using the libraryEasy to understandSo far – no impact in marshaling support for the library
  • #81 Priceless !!!
  • #82 10:1 ROI
  • #83 Victoria, Australia Dollars, Sense and Public Libraries 2008 all public libraries in VictoriaBenefit = $681 million to a cost of $191 millionLibrary customer willing to pay $72 per person per year – pay $36 now1,380 library customers included in the surveyFinancial savings, travel costs, contingent valuation
  • #84 Economic value – willingness to payEconomic benefit – savings from avoiding the purchase of goods and servicesEconomic activity – contribution to the local economy
  • #85 V+LM = Value Added Library MethodologyMarket price proxy – market price where there is a willing seller and a willing buyerReplacement cost – cost to replace a serviceOpportunity cost – value of time (less time when using a library)What adds greatest value, compares budget to value of each serviceManakau PL ROI = 1.5:1
  • #87 Walt Crawford. Give Us a Dollar and We’ll Give You Back Four (2012-13) Average ROI 4.89:1 8,659 public libraries IMLS dataCites & Insights Books
  • #89 Turin – Gates FoundationSometimes referred to as the “triple bottom line” – financial, social, and environmental
  • #93 Range of methodsContingent valuationInvestment in accessPrice elasticity in demandCost of alternativesGoogle images
  • #94 Google images national‑library‑of‑australia_nla_007.jpgccdact.com.au
  • #95  The National Library of New Zealand found a 3.5 : 1 benefit-cost ratio in their nationalbibliographic database through use of a Choice Modeling (conjoint analysis) survey
  • #97 V+LM method
  • #100 Public libraries – St. Louis study – more than 10 years ago – different for your library
  • #105 Impact will increasingly occur as a result of digitized resources
  • #106 TangibleIntangible Human capital IT capital Organizational capital
  • #112 Library needs to collaborate with other on campus to COMBINE dataOffice of Institutional Research
  • #115 Predictive Analytics Reporting FrameworkGates Foundation - $1 million“Big data”New students are at substantially higher levels of risk of dropping out or failing. 36% of students dropped out.Once a student has some success, they tend to keep going. Doing well the best predictor of continued enrollment. The more concurrent online courses, the more the student struggles.Demographic variables (race, gender, age) tend to lose significance in the presence of behavioral data.Developmental education (remedial) classes do little to help.Students who withdraw from a class will more likely withdraw from other classes in the future.
  • #117 Metric management system = LibPAS Counting Opinions
  • #118 Impact – use VERBS – Learning, collaborating, assisting, empowering
  • #120 How and what you say has a real impact!
  • #121 Is the library viewed as an ….
  • #122 Or is the library viewed as a budget sinkhole?
  • #123 What measures of library success will resonate in your organizational setting?Be visible with your funding decision makers – build personal connectionsIn your environment, how is value defined, measured & communicated?
  • #124 Pain gapGain gap - outcomes
  • #126 The Value Proposition must fill two requirements:Actionable by you and your organizationCredible and compelling to target audienceExamples – It’s the most technologically advanced and robust system on the marketWe improve communication and moraleWe offer training classes in a wide variety of areasMy product was rated the best-in-class by leading authorities
  • #128 Alignment Organization’s mission and goalsWhat is a valuable library? One that contributes to reaching the goals and objectives of the organization.
  • #129 Be intentional – planPerfect day at the beachUnderstanding how you library can better serve your community is essential to being relevant in the lives of those whom you serve.
  • #130 Partner with others
  • #131 Remember that ROI is only one piece of the puzzleFunding for libraries is a reflection of public or campus supportSupport (in the form of your budget) reflects the perceived value of the library to each family, student, faculty member –In short, to your communityHowever, the value is judged in the context of today’s economy and today’s society
  • #132 If you are going to do ROIEconomic value – willingness to payEconomic benefit – savings from avoiding the purchase of goods and servicesEconomic activity – contribution to the local economyAnd value has three components
  • #133 Tell your story in numbers and stories Your message – “We have contributed towards YOUR goals by ….”Harbor Bridge, Sydney, Australia
  • #134 Use lots of color and excitement to convey both written and spoken stories of the value of the library
  • #135 Gather evidence of the value we addUse the evidence to improve practicesTrumpet the value of the library
  • #138 Had enough bullet points?