Work Stream Two
Evaluating Social and Economic
            Impact

   European Congress on E-Inclusion: ECEI09
   Technolo...
Introduction

• Scope of Work Stream
  – How to evaluate social and economic
    impact of accessible technologies in
    ...
Key Questions

• What are current approaches to the
  evaluation of public library impact
  and how are they being applied...
Questions cont’d
• How have these evaluative tools
  affected the wider municipal, regional
  or national policy agendas?
...
Potential Outcomes
• Feedback on the range of current
  evaluation activities within public
  libraries
• Examples of eval...
Measuring Return-on-
     Investment
 in Public Libraries
Return-on-Investment (ROI) for
        Public Libraries
has been applied to many different types of
organizations and comm...
Importance of ROI for Public
           Libraries
• Tight budgets
• Competing national, regional and local
  interests
• M...
Why ROI is More Difficult to
     Calculate for Libraries
• Difficulty of quantifying benefits that vary
  from
     user ...
Early Efforts -
         Value Assessment
• Example: Griffiths/King study - 1982,
  Office of Scientific and Technical
  I...
Early Public Library
           Evaluations
• Example: Griffiths/King ROI studies–
  1989, U.K public libraries; 1991,
  M...
Costs, Outputs and Outcomes
Need to consider:
• The costs (investments) of the libraries
   and their services
• The outpu...
*
           Total Economic Value
• Use value - net willingness to pay
• Option value - willingness to pay for the
  optio...
Jobs and Income
• Center for Economic Development
  Research at the University of South Florida
  - economic contribution ...
Contingent Valuation
• economic method of evaluation for non-
  priced goods and services
• looks at the implications of n...
Case Study -
State of Florida
Methods
Used a variety of data collection and
 analytic methods including
  – data reported to the state by the libraries
...
Household Interviews/
        In-Library Survey
Collected information about:
   – cost to use the library
   – services us...
Organization Survey

Determined:
  – use of public library services
  – cost to use these services
  – savings resulting f...
Follow-up Survey
Obtained some information about:
  – use by tourists and school age children
  – interlibrary lending and...
REMI
• Addresses the economic effect a
  public organization or resource has on
  other economic sectors over time
• Econo...
Results and Presentation
Results
• Figured on lower bound - so results were at
  least as good as reported
• Emphasis on “bottom line” results (wit...
Florida’s Public Libraries
       Return $6.54
 for every $1.00 Invested
Return on                                        Economic                                                  Investments
   ...
Revenue Investment
Economic Return:
  $2.93 Billion


          Economic Returns Resulting
         From Florida's Public Libraries
         ...
New Jobs —
1 for every $6,448 invested
Florida’s Public Libraries
      Increase GRP
Florida Public Libraries
Increase Income in the State
      Income Increases from Public
        Support of Florida Public...
Economic Ripple Effect

• Effect of public investment in public
  libraries:
  – GRP         $4.0 billion increase
  – Wag...
Florida’s Public Libraries are
     Extensively Used —
 94 million In Person and Remote Visits
                           ...
Florida’s Public Libraries are Used
 by 11.8 Million People Annually
Communicating Results
   to Stakeholders
Variety of Report Formats
• Executive summary/overview report
• Detailed methodology and analysis
  document
• Additional ...
Variety of Presentations

Based on target audiences
• Results-oriented summary
• More detailed utilization data as well
  ...
Variety of Publications -
 General taxpayer audience
Variety of Publications -
Community/State leaders audience
Variety of Publications -
Library directors/leadership audience
Presentations and Press
From Economic to Social Impact

Social Return-on-Investment
• Economic value of cultural, social and
  environmental impac...
Public Library Valuation:
    Needs & Opportunities
• Study conducted in 2006-07
• Americans for Libraries Council
• Suppo...
Social Responsibility Models

• Balanced scorecard

• Triple-Bottom-Line Accounting

• Corporate Social Responsibility
  R...
Recommendations
• Improve coordination and communication
  among stakeholders
• Develop a comprehensive research agenda
  ...
Recommendations cont’d
• Define a national agenda for library
  valuation research as part of research
  coordination infr...
Economic Impact in the Arts -
            Matarasso
•   Voluntary labor and donations
•   Consumer spending
•   Employment...
Social Impact in the Arts -
             Matarasso
• Personal development
• Social cohesion
• Community empowerment and se...
Social Impact Measurement
           Areas - Matarasso
•   Making life better
•   Creating public arts projects
•   Involv...
IMLS Interconnections Study
• Conduct national survey of information
  needs of users and potential users of online
  info...
The Power of Trust

        Conclusion 1:
 Libraries and museums evoke
consistent, extraordinary public
trust among divers...
Libraries and museums are the most trusted
sources of information according to a survey
             of over 1,700 adults.
The Internet Does Not Kill
 Libraries and Museums


         Conclusion #2:
Internet use is positively related to
 in-pers...
Adults Who Use the Internet are
 More Likely to Visit Libraries and
            Museums
        Proportion of Public Libra...
Adults Who Use the Internet Visit
Libraries and Museums More Often
         Average Number of Public Library and Museum Vi...
Q. Why do we need
  museums and libraries
  if we have the Internet?
  A. Interconnections
           Conclusion #3:
     ...
To fulfill their need for information, most adults use
museums, public libraries, and the Internet. Museums
and public lib...
Interconnections
Use of Public Library
          Workstations
• 49 % of 149 million in-person visitors
  to public libraries used library-
...
Services Used from Public
       Library Workstations
•   Search engines                 70%
•   Obtaining info from libn ...
Reasons for Using Public
     Library Workstations
• Convenience/ease of use            87%
• Best source of information  ...
Time Spent Using Public
     Library Workstations
• 1 - 10 minutes            31%
• 11 – 20 minutes 19%
• 21 – 30 minutes ...
Purpose for Using Public
      Library Workstations
•   Meet educational needs          53%
•   Meet personal or family ne...
Satisfaction with Attributes of
    Public Library Workstations
        (scale: 1 – 5 – 5 is high)

• Hours of availabilit...
Outcomes of Using Public
      Library Workstations

• Obtained needed information
   – Got all information sought        ...
Jose Marie Griffiths Evaluating Social And Economic Impact
Jose Marie Griffiths Evaluating Social And Economic Impact
Upcoming SlideShare
Loading in …5
×

Jose Marie Griffiths Evaluating Social And Economic Impact

1,711 views

Published on

Public library: Evaluating social and economic impact

Published in: Education, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,711
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Jose Marie Griffiths Evaluating Social And Economic Impact

  1. 1. Work Stream Two Evaluating Social and Economic Impact European Congress on E-Inclusion: ECEI09 Technology and Beyond in Public Libraries Brussels October 22-23, 2009 José-Marie Griffiths School of Information and Library Science University of North Carolina at Chapel Hill
  2. 2. Introduction • Scope of Work Stream – How to evaluate social and economic impact of accessible technologies in public libraries – How libraries can demonstrate the impact of the investments made – Which criteria are needed to evaluate social and economic impact
  3. 3. Key Questions • What are current approaches to the evaluation of public library impact and how are they being applied? • What studies have been undertaken to assess specifically the performance of e-inclusion services within public libraries?
  4. 4. Questions cont’d • How have these evaluative tools affected the wider municipal, regional or national policy agendas? • What should be the future approaches to the evaluation of public libraries within the context of increasing direct access to networked services within communities
  5. 5. Potential Outcomes • Feedback on the range of current evaluation activities within public libraries • Examples of evaluation that has demonstrated the social and economic value of e-inclusion in public libraries • Future trends in terms of how public service can be evaluated successfully within the networked society
  6. 6. Measuring Return-on- Investment in Public Libraries
  7. 7. Return-on-Investment (ROI) for Public Libraries has been applied to many different types of organizations and community resources application of cost/benefit, cost- effectiveness, impact and return-on- investment measures for-profit sector - common to libraries, museums, schools and colleges, parks, etc. – not common
  8. 8. Importance of ROI for Public Libraries • Tight budgets • Competing national, regional and local interests • Must make the case in quantitative terms, not just anecdotal evidence • Economic valuation is a powerful tool for advocacy • Data must be collected and analyzed in the context of what is important to the communities within which the libraries operate
  9. 9. Why ROI is More Difficult to Calculate for Libraries • Difficulty of quantifying benefits that vary from user to user use to use from library to library (as service mixes vary) • The push for public libraries to develop services relevant to the needs of their local communities, has made the evaluation process more difficult. • As a result, libraries tended to focus on user satisfaction and other attitudinal measures.
  10. 10. Early Efforts - Value Assessment • Example: Griffiths/King study - 1982, Office of Scientific and Technical Information of the U.S. Department of Energy — develop approaches to assessing/measuring the value of the Energy Database • Three Levels of Value Assessment Willingness-to-pay or exchange value Use value Consequential value
  11. 11. Early Public Library Evaluations • Example: Griffiths/King ROI studies– 1989, U.K public libraries; 1991, Massachusetts public libraries; 1993 U.K policy briefing • Need to create an aggregate picture of library value • Compare this value to the total investment in libraries - return-on-investment
  12. 12. Costs, Outputs and Outcomes Need to consider: • The costs (investments) of the libraries and their services • The outputs produced • The use of the outputs • Outcomes resulting from that use – Improved quality of life – Support for lifelong learning – Support for the community’s economy
  13. 13. * Total Economic Value • Use value - net willingness to pay • Option value - willingness to pay for the option to use in the future • Existence value - willingness to pay for the good/service to exist even though no future use is contemplated • Bequest value - willingess to pay for the endowment of the good or service for future generations * North Carolina Blue Ridge Parkway Study, 1999-2002 *
  14. 14. Jobs and Income • Center for Economic Development Research at the University of South Florida - economic contribution or impact of various corporations and institutions to the state • Measured: – Jobs and jobs created (both paid and volunteer) – Personal income (wages and disposable income) – Local output (value of goods and services resulting from jobs created)
  15. 15. Contingent Valuation • economic method of evaluation for non- priced goods and services • looks at the implications of not having the goods/services. • includes – added cost to use alternatives sources of information, should people choose to do so (also called net benefit); – portion of direct economic contribution public libraries make to their communities that would be lost; – portion of economic benefits to the library users that would be lost.
  16. 16. Case Study - State of Florida
  17. 17. Methods Used a variety of data collection and analytic methods including – data reported to the state by the libraries – a statewide household telephone survey of adults – in-library user surveys of adults – a follow-up survey of the libraries – surveys of organizations – an input-output econometric model
  18. 18. Household Interviews/ In-Library Survey Collected information about: – cost to use the library – services used – reasons for using the services – importance of the services; ways the services were important – what visitors would do to obtain the needed information if there were no public library – estimated cost in time and money to use the alternatives – used critical incident approach
  19. 19. Organization Survey Determined: – use of public library services – cost to use these services – savings resulting from service use
  20. 20. Follow-up Survey Obtained some information about: – use by tourists and school age children – interlibrary lending and borrowing – expenditures and income – business-like operations run by the library, outside persons or vendors.
  21. 21. REMI • Addresses the economic effect a public organization or resource has on other economic sectors over time • Econometric input-output model • Extends the economic contribution of libraries beyond the actual users of the libraries to yield a set of direct, indirect and induced effects to the served communities
  22. 22. Results and Presentation
  23. 23. Results • Figured on lower bound - so results were at least as good as reported • Emphasis on “bottom line” results (with detail available for those who wanted it) • Correlation with interests important to state and community leadership and decision- makers • Provision of effective graphics in reports and presentations
  24. 24. Florida’s Public Libraries Return $6.54 for every $1.00 Invested
  25. 25. Return on Economic Investments Investment 6.54 to 1.0 = Return $2,993.660 million ÷ $448.903 million Net Benefit + Lost Use Benefit + Lost Community Benefits = Economic Return Lost Uses Lost Community Spending $2,993.660 million Use Benefits Community Benefits User Investment to Use the Library $1.7 billion Benefits (of having the library) To Individuals Organizations To community/ state individual To Halo $1,721 million $2.384 million users: organizatio availability ns as users of the Pass through Investments (costs) library: In-state spending To availabilitty individual users: Staff Federal State Local use Compensation Funding Funding Funding Cost to use alternatives Expenditures by the Other library Funding Multitype For For In-state In-state Print Electronic Individuals Organizations Staff Expenditures Expenditures Compensation In-state out of state out of state In-state In-state In-state Media Other Operating Capital Expenditures Expenditur Outlay out of state out of state es out of state
  26. 26. Revenue Investment
  27. 27. Economic Return: $2.93 Billion Economic Returns Resulting From Florida's Public Libraries (total $2.93 billion) 80% Added cost to use alternatives + 5% 15% Lost direct user Lost direct community economic benefits economic benefits
  28. 28. New Jobs — 1 for every $6,448 invested
  29. 29. Florida’s Public Libraries Increase GRP
  30. 30. Florida Public Libraries Increase Income in the State Income Increases from Public Support of Florida Public Libraries $14.00 $12.66 $12.00 $10.00 $8.00 $6.00 $4.00 $1.00 $2.00 $0.00 Public Florida support to income libraries increase
  31. 31. Economic Ripple Effect • Effect of public investment in public libraries: – GRP $4.0 billion increase – Wages $5.6 billion increase in personal income - Jobs 68,700 jobs created Net benefit: $4.9 billion or 3.7 to 1
  32. 32. Florida’s Public Libraries are Extensively Used — 94 million In Person and Remote Visits 68 million in-person visits 25.2 million remote Internet connections (not including children or tourists)
  33. 33. Florida’s Public Libraries are Used by 11.8 Million People Annually
  34. 34. Communicating Results to Stakeholders
  35. 35. Variety of Report Formats • Executive summary/overview report • Detailed methodology and analysis document • Additional analysis and survey instruments
  36. 36. Variety of Presentations Based on target audiences • Results-oriented summary • More detailed utilization data as well as results • Methodologies and detailed analysis
  37. 37. Variety of Publications - General taxpayer audience
  38. 38. Variety of Publications - Community/State leaders audience
  39. 39. Variety of Publications - Library directors/leadership audience
  40. 40. Presentations and Press
  41. 41. From Economic to Social Impact Social Return-on-Investment • Economic value of cultural, social and environmental impacts – How to value the cultural wealth created by public libraries? – How to value the contribution public libraries make to larger policy objectives? – How to value the social worth of public libraries?
  42. 42. Public Library Valuation: Needs & Opportunities • Study conducted in 2006-07 • Americans for Libraries Council • Support from The Bill & Melinda Gates Foundation • Excellent overview of economic valuation studies • Call for broader valuation to include social impact valuation
  43. 43. Social Responsibility Models • Balanced scorecard • Triple-Bottom-Line Accounting • Corporate Social Responsibility Reports
  44. 44. Recommendations • Improve coordination and communication among stakeholders • Develop a comprehensive research agenda that promotes systematic valuation of libraries’ contribution to education, civic participation and quality of life • Create a varied set of innovative tools • Take advantage of valuation and evaluation reporting lessons from other sectors and fields
  45. 45. Recommendations cont’d • Define a national agenda for library valuation research as part of research coordination infrastructure • Seek support for impact assessments enabling libraries, advocates and researchers to work together to evaluate the impact of different studies in different contexts
  46. 46. Economic Impact in the Arts - Matarasso • Voluntary labor and donations • Consumer spending • Employment and training for work • Investing in local communities • Savings in public expenditures
  47. 47. Social Impact in the Arts - Matarasso • Personal development • Social cohesion • Community empowerment and self- determination • Local image and identity • Imagination and vision • Health and well-being
  48. 48. Social Impact Measurement Areas - Matarasso • Making life better • Creating public arts projects • Involving local people • Developing people’s skills and resources • Strengthening partnertships
  49. 49. IMLS Interconnections Study • Conduct national survey of information needs of users and potential users of online information • Primary focus on museums, public libraries and the Internet as sources • Telephone surveys of adults (18 and over)
  50. 50. The Power of Trust Conclusion 1: Libraries and museums evoke consistent, extraordinary public trust among diverse adult users.
  51. 51. Libraries and museums are the most trusted sources of information according to a survey of over 1,700 adults.
  52. 52. The Internet Does Not Kill Libraries and Museums Conclusion #2: Internet use is positively related to in-person visits to museums and libraries.
  53. 53. Adults Who Use the Internet are More Likely to Visit Libraries and Museums Proportion of Public Library and Museum Visitors Who Use or Do Not Use the Internet 71.0% Public 47.4% Libraries 66.7% Internet users Non-users of the Internet 73.3% All adults 38.3% Museums 66.4% 0% 20% 40% 60% 80% 100% Proportion of Adult Visitors
  54. 54. Adults Who Use the Internet Visit Libraries and Museums More Often Average Number of Public Library and Museum Visits by Those Who Use and Do Not Use the Internet 3.44 Public 3.36 Libraries 3.42 Internet users Non-users of the Internet 3.46 All adults 1.34 Museums 3.14 1 2 3 4 5 Number of visits per adult
  55. 55. Q. Why do we need museums and libraries if we have the Internet? A. Interconnections Conclusion #3: Museums and public libraries in-person and online serve important and complementary roles in supporting a wide variety of information needs.
  56. 56. To fulfill their need for information, most adults use museums, public libraries, and the Internet. Museums and public libraries are used by 70%, the Internet is used by 83%, and nearly half (47%) use all three. Only 7% of adults do not use any of the three sources.
  57. 57. Interconnections
  58. 58. Use of Public Library Workstations • 49 % of 149 million in-person visitors to public libraries used library- provided workstations a total of 294 million times during the previous 12 months • 69% or 203 million of those 294 million uses were for Internet and other online resources and services
  59. 59. Services Used from Public Library Workstations • Search engines 70% • Obtaining info from libn or library 53% • Looking at other website 46% • Used e-mail 36% • Viewing/downloading articles 36% • Viewed blog 18% • Viewed/downloaded e-books 9% • Used chat or IM 7% • Other 5%
  60. 60. Reasons for Using Public Library Workstations • Convenience/ease of use 87% • Best source of information 74% • Information could be trusted 58% • Low cost (time and $$) 58% • Don’t own computer or currently unavailable 6% • Other 4%
  61. 61. Time Spent Using Public Library Workstations • 1 - 10 minutes 31% • 11 – 20 minutes 19% • 21 – 30 minutes 18% • 31 – 60 minutes 29% • 1 -2 hours 2% • Over 2 hours 2% Average time: Per in-library online visit: 29 minutes Per remote online visit: 63 minutes
  62. 62. Purpose for Using Public Library Workstations • Meet educational needs 53% • Meet personal or family needs 41% • Meet work-related needs 33% • Recreation or entertainment 27%
  63. 63. Satisfaction with Attributes of Public Library Workstations (scale: 1 – 5 – 5 is high) • Hours of availability 4.14 • Software available 3.92 • Hardware available 3.89 • Number of workstations and their accessibility 3.87 • Amount of time allowed in a single session 3.67
  64. 64. Outcomes of Using Public Library Workstations • Obtained needed information – Got all information sought 50% – Got some information sought 46% • Addressed reason for use – Completely 62% – Somewhat • Led to favorable outcome 94% • Led to negative outcome 4% • Too much irrelevant information 24%

×