Work Stream Two
Evaluating Social and Economic
Impact
European Congress on E-Inclusion: ECEI09
Technology and Beyond in Public Libraries
Brussels
October 22-23, 2009
José-Marie Griffiths
School of Information and Library Science
University of North Carolina at Chapel Hill

Introduction
• Scope of Work Stream
– How to evaluate social and economic
impact of accessible technologies in
public libraries
– How libraries can demonstrate the
impact of the investments made
– Which criteria are needed to evaluate
social and economic impact

Key Questions
• What are current approaches to the
evaluation of public library impact
and how are they being applied?
• What studies have been undertaken to
assess specifically the performance of
e-inclusion services within public
libraries?

Questions cont’d
• How have these evaluative tools
affected the wider municipal, regional
or national policy agendas?
• What should be the future approaches
to the evaluation of public libraries
within the context of increasing direct
access to networked services within
communities

Potential Outcomes
• Feedback on the range of current
evaluation activities within public
libraries
• Examples of evaluation that has
demonstrated the social and economic
value of e-inclusion in public libraries
• Future trends in terms of how public
service can be evaluated successfully
within the networked society

Measuring Return-on-
Investment
in Public Libraries

Return-on-Investment (ROI) for
Public Libraries
has been applied to many different types of
organizations and community resources
application of cost/benefit, cost-
effectiveness, impact and return-on-
investment measures
for-profit sector - common
to libraries, museums, schools and
colleges, parks, etc. – not common

Importance of ROI for Public
Libraries
• Tight budgets
• Competing national, regional and local
interests
• Must make the case in quantitative terms,
not just anecdotal evidence
• Economic valuation is a powerful tool for
advocacy
• Data must be collected and analyzed in the
context of what is important to the
communities within which the libraries
operate

Why ROI is More Difficult to
Calculate for Libraries
• Difficulty of quantifying benefits that vary
from
user to user
use to use
from library to library (as service mixes
vary)
• The push for public libraries to develop
services relevant to the needs of their local
communities, has made the evaluation
process more difficult.
• As a result, libraries tended to focus on user
satisfaction and other attitudinal measures.

Early Efforts -
Value Assessment
• Example: Griffiths/King study - 1982,
Office of Scientific and Technical
Information of the U.S. Department of
Energy — develop approaches to
assessing/measuring the value of the
Energy Database
• Three Levels of Value Assessment
Willingness-to-pay or exchange value
Use value
Consequential value

Early Public Library
Evaluations
• Example: Griffiths/King ROI studies–
1989, U.K public libraries; 1991,
Massachusetts public libraries; 1993 U.K
policy briefing
• Need to create an aggregate picture of
library value
• Compare this value to the total investment
in libraries - return-on-investment

Costs, Outputs and Outcomes
Need to consider:
• The costs (investments) of the libraries
and their services
• The outputs produced
• The use of the outputs
• Outcomes resulting from that use
– Improved quality of life
– Support for lifelong learning
– Support for the community’s economy

*
Total Economic Value
• Use value - net willingness to pay
• Option value - willingness to pay for the
option to use in the future
• Existence value - willingness to pay for the
good/service to exist even though no future
use is contemplated
• Bequest value - willingess to pay for the
endowment of the good or service for
future generations
* North Carolina Blue Ridge Parkway Study, 1999-2002
*

Jobs and Income
• Center for Economic Development
Research at the University of South Florida
- economic contribution or impact of
various corporations and institutions to the
state
• Measured:
– Jobs and jobs created (both paid and volunteer)
– Personal income (wages and disposable
income)
– Local output (value of goods and services
resulting from jobs created)

Contingent Valuation
• economic method of evaluation for non-
priced goods and services
• looks at the implications of not having the
goods/services.
• includes
– added cost to use alternatives sources of
information, should people choose to do so
(also called net benefit);
– portion of direct economic contribution public
libraries make to their communities that would
be lost;
– portion of economic benefits to the library
users that would be lost.

Case Study -
State of Florida

Methods
Used a variety of data collection and
analytic methods including
– data reported to the state by the libraries
– a statewide household telephone survey of
adults
– in-library user surveys of adults
– a follow-up survey of the libraries
– surveys of organizations
– an input-output econometric model

Household Interviews/
In-Library Survey
Collected information about:
– cost to use the library
– services used
– reasons for using the services
– importance of the services; ways the services
were important
– what visitors would do to obtain the needed
information if there were no public library
– estimated cost in time and money to use the
alternatives
– used critical incident approach

Organization Survey
Determined:
– use of public library services
– cost to use these services
– savings resulting from service use

Follow-up Survey
Obtained some information about:
– use by tourists and school age children
– interlibrary lending and borrowing
– expenditures and income
– business-like operations run by the library,
outside persons or vendors.

REMI
• Addresses the economic effect a
public organization or resource has on
other economic sectors over time
• Econometric input-output model
• Extends the economic contribution of
libraries beyond the actual users of
the libraries to yield a set of direct,
indirect and induced effects to the
served communities

Results and Presentation

Results
• Figured on lower bound - so results were at
least as good as reported
• Emphasis on “bottom line” results (with
detail available for those who wanted it)
• Correlation with interests important to state
and community leadership and decision-
makers
• Provision of effective graphics in reports
and presentations

Florida’s Public Libraries
Return $6.54
for every $1.00 Invested

Return on Economic Investments
Investment
6.54 to 1.0
= Return
$2,993.660
million
÷ $448.903 million
Net Benefit + Lost Use Benefit
+ Lost Community
Benefits = Economic
Return
Lost Uses Lost Community Spending
$2,993.660 million
Use Benefits Community Benefits
User Investment to Use the
Library $1.7 billion
Benefits (of having the library)
To
Individuals Organizations To community/
state
individual To Halo
$1,721 million $2.384 million users: organizatio
availability ns as users
of the
Pass through
Investments (costs)
library:
In-state spending
To availabilitty
individual
users: Staff
Federal State Local
use Compensation Funding Funding Funding
Cost to use alternatives
Expenditures by the Other
library Funding Multitype
For For In-state In-state
Print Electronic
Individuals Organizations Staff
Expenditures Expenditures
Compensation
In-state out of state out of state
In-state In-state In-state
Media Other
Operating Capital
Expenditures
Expenditur Outlay
out of state out of state
es out of state

Revenue Investment

Economic Return:
$2.93 Billion
Economic Returns Resulting
From Florida's Public Libraries
(total $2.93 billion)
80%
Added cost to use
alternatives
+
5% 15%
Lost direct user Lost direct community
economic benefits economic benefits

New Jobs —
1 for every $6,448 invested

Florida’s Public Libraries
Increase GRP

Florida Public Libraries
Increase Income in the State
Income Increases from Public
Support of Florida Public
Libraries
$14.00
$12.66
$12.00
$10.00
$8.00
$6.00
$4.00
$1.00
$2.00
$0.00
Public Florida
support to income
libraries increase

Economic Ripple Effect
• Effect of public investment in public
libraries:
– GRP $4.0 billion increase
– Wages $5.6 billion increase in
personal income
- Jobs 68,700 jobs created
Net benefit: $4.9 billion or 3.7 to 1

Florida’s Public Libraries are
Extensively Used —
94 million In Person and Remote Visits
68 million
in-person
visits
25.2 million
remote
Internet
connections
(not including
children or
tourists)

Florida’s Public Libraries are Used
by 11.8 Million People Annually

Communicating Results
to Stakeholders

Variety of Report Formats
• Executive summary/overview report
• Detailed methodology and analysis
document
• Additional analysis and survey
instruments

Variety of Presentations
Based on target audiences
• Results-oriented summary
• More detailed utilization data as well
as results
• Methodologies and detailed analysis

Variety of Publications -
General taxpayer audience

Variety of Publications -
Community/State leaders audience

Variety of Publications -
Library directors/leadership audience

Presentations and Press

From Economic to Social Impact
Social Return-on-Investment
• Economic value of cultural, social and
environmental impacts
– How to value the cultural wealth created
by public libraries?
– How to value the contribution public
libraries make to larger policy
objectives?
– How to value the social worth of public
libraries?

Public Library Valuation:
Needs & Opportunities
• Study conducted in 2006-07
• Americans for Libraries Council
• Support from The Bill & Melinda
Gates Foundation
• Excellent overview of economic
valuation studies
• Call for broader valuation to include
social impact valuation

Social Responsibility Models
• Balanced scorecard
• Triple-Bottom-Line Accounting
• Corporate Social Responsibility
Reports

Recommendations
• Improve coordination and communication
among stakeholders
• Develop a comprehensive research agenda
that promotes systematic valuation of
libraries’ contribution to education, civic
participation and quality of life
• Create a varied set of innovative tools
• Take advantage of valuation and evaluation
reporting lessons from other sectors and
fields

Recommendations cont’d
• Define a national agenda for library
valuation research as part of research
coordination infrastructure
• Seek support for impact assessments
enabling libraries, advocates and
researchers to work together to evaluate the
impact of different studies in different
contexts

Economic Impact in the Arts -
Matarasso
• Voluntary labor and donations
• Consumer spending
• Employment and training for work
• Investing in local communities
• Savings in public expenditures

Social Impact in the Arts -
Matarasso
• Personal development
• Social cohesion
• Community empowerment and self-
determination
• Local image and identity
• Imagination and vision
• Health and well-being

Social Impact Measurement
Areas - Matarasso
• Making life better
• Creating public arts projects
• Involving local people
• Developing people’s skills and resources
• Strengthening partnertships

IMLS Interconnections Study
• Conduct national survey of information
needs of users and potential users of online
information
• Primary focus on museums, public libraries
and the Internet as sources
• Telephone surveys of adults (18 and over)

The Power of Trust
Conclusion 1:
Libraries and museums evoke
consistent, extraordinary public
trust among diverse adult users.

Libraries and museums are the most trusted
sources of information according to a survey
of over 1,700 adults.

The Internet Does Not Kill
Libraries and Museums
Conclusion #2:
Internet use is positively related to
in-person visits to museums and
libraries.

Adults Who Use the Internet are
More Likely to Visit Libraries and
Museums
Proportion of Public Library and Museum Visitors Who
Use or Do Not Use the Internet
71.0%
Public 47.4%
Libraries
66.7%
Internet users
Non-users of the Internet
73.3%
All adults
38.3%
Museums
66.4%
0% 20% 40% 60% 80% 100%
Proportion of Adult Visitors

Adults Who Use the Internet Visit
Libraries and Museums More Often
Average Number of Public Library and Museum Visits by Those Who
Use and Do Not Use the Internet
3.44
Public 3.36
Libraries
3.42
Internet users
Non-users of the Internet
3.46
All adults
1.34
Museums
3.14
1 2 3 4 5
Number of visits per adult

Q. Why do we need
museums and libraries
if we have the Internet?
A. Interconnections
Conclusion #3:
Museums and public libraries
in-person and online serve important and
complementary roles in supporting a
wide variety of information needs.

To fulfill their need for information, most adults use
museums, public libraries, and the Internet. Museums
and public libraries are used by 70%, the Internet is
used by 83%, and nearly half (47%) use all three.
Only 7% of adults do not use any of the three sources.

Interconnections

Use of Public Library
Workstations
• 49 % of 149 million in-person visitors
to public libraries used library-
provided workstations a total of 294
million times during the previous 12
months
• 69% or 203 million of those 294
million uses were for Internet and
other online resources and services

Services Used from Public
Library Workstations
• Search engines 70%
• Obtaining info from libn or library 53%
• Looking at other website 46%
• Used e-mail 36%
• Viewing/downloading articles 36%
• Viewed blog 18%
• Viewed/downloaded e-books 9%
• Used chat or IM 7%
• Other 5%

Reasons for Using Public
Library Workstations
• Convenience/ease of use 87%
• Best source of information 74%
• Information could be trusted 58%
• Low cost (time and $$) 58%
• Don’t own computer or currently
unavailable 6%
• Other 4%

Time Spent Using Public
Library Workstations
• 1 - 10 minutes 31%
• 11 – 20 minutes 19%
• 21 – 30 minutes 18%
• 31 – 60 minutes 29%
• 1 -2 hours 2%
• Over 2 hours 2%
Average time:
Per in-library online visit: 29 minutes
Per remote online visit: 63 minutes

Purpose for Using Public
Library Workstations
• Meet educational needs 53%
• Meet personal or family needs 41%
• Meet work-related needs 33%
• Recreation or entertainment 27%

Satisfaction with Attributes of
Public Library Workstations
(scale: 1 – 5 – 5 is high)
• Hours of availability 4.14
• Software available 3.92
• Hardware available 3.89
• Number of workstations
and their accessibility 3.87
• Amount of time allowed in
a single session 3.67

Outcomes of Using Public
Library Workstations
• Obtained needed information
– Got all information sought 50%
– Got some information sought 46%
• Addressed reason for use
– Completely 62%
– Somewhat
• Led to favorable outcome 94%
• Led to negative outcome 4%
• Too much irrelevant information 24%