Presentation at the American Library Association Conference on June 28, 2014. Sponsored by counting Opinions

1. Data, Evidence and
Outcomes
Joe Matthews
American Library Association
Sponsored by Counting Opinions
June 28, 2014

2. Need for Data
 Making management decisions
 Increase program & service effectiveness
 Justifying budgets
 Evaluating performance
 Doing advocacy

3. Types of Data
 Quantitative
 Counting
 Surveys
 evaluation of services
 Customer satisfaction
 Qualitative

4. Purpose
The purpose of any data gathering activity
& analysis is to INFORM

5. Strategies
 User versus library’s perspective
 Examining trends
 Comparing libraries
 Social context
 Outcomes

6. User’s Perspective
 How well?
 How courteous?
 How responsive?
 How satisfied?

7. How courteous?
 Welcoming
 Attentive

8. How responsive?
 Anticipatory
 Helpful
 Empathetic

9. How satisfied?
 Expectations met
 Materials obtained
 Personal interaction
 Ease of use
 Equipment used
 Environment
 Comfort
 Willingness to return

10. Library’s Perspective
 How much?
 How many?
 How prompt?
 How economical?

11. How much?
 Magnitude
 Percent of change from last year
 Percent of overall change
 Cost

12. How many?
 Magnitude
 Change

13. How prompt?
 Cycle times
 Turnaround time
 Wait times
 Anticipatory

14. How economical?
 Resources used
 Units processed
 Productivity

15. Library & Customers Perspective
 How valuable?
 How reliable?
 How accurate?
 How well?

16. How valuable?
 Effort expended
 Costs
 Benefits obtained

17. How reliable?
 Dependability/Consistency
 Access
 Accuracy

18. How accurate?
 Completeness
 Comprehensiveness
 Currency

19. How well?
 Accuracy
 Promptness
 Courtesy
 Expertise

20. Outcomes
Change in a person’s life
Affective – attitudes, confidence, satisfaction
Behavioral
Knowledge-based
Competency-based – apply new skills

21. Tracking Trends
 Year-to-year trends
 Comparing trends
 Indexing dollars for inflation
 Indexing budgets for cost of living
 Projecting future needs

22. Comparing Libraries
 Identifying peers
 Individual vs. grouped data
 Rankings

23. Comparing Libraries Examples
 Comparing Peers – criteria ?
 Rankings
 Percentages/grouped data
 Summary stats
 Benchmarking/standards

24. Libraries in Social Context
 Social, political, & economic environment
 Focusing on decision-makers’ concerns
 Finding the right data
 Creating the “hook”

25. Social Context Examples
 PLs rank 2nd behind fire protection among local
government services
 Visits to PLs outnumber professional football
attendance of X annually by a factor of Y
 PLs loan 4.5 million items per day – FedEx delivers
about 1.75 million packages each day

26. Evidence
 How can evidence contribute to the quality
of the decision making process?
 What are the risks, costs, and benefits of an
evidence-based approach?
 Ensure that objective evidence is reflected
in the decision making process.

27. Evidence
 Depends on the need
 BIG DATA
 Combine data

28. Source of Data
 IMLS data
 ARL data
 PLAmetrics
 ACRLMetrics
 Edge Initiative
 Impact Survey – UW
 Evidence-based Library &
Information Practice

29. Collection Insights
 collectionHQ – selection, management, promotion
 Reports from your ILS

30. Patron Perspective
 Segmentation analytics
 Demographics
 Lifestyles
 Geography
 Benefit segmentation

31. Program Priorities
 Track program measures
 Provide programs tied to strategic goals –
aligned with community needs
 Early childhood education
 School age education support
 Economic development
 Community engagement
 . . .

32. Advocacy Hooks
 Use data for presentations
 Use data to target likely voters
 Use data to demonstrate value in the
life of your customers

33. Outcomes
Libraries cannot demonstrate value
until they define outcomes
of institutional relevance
and then measure the degree
to which they attain them

34. Outcomes
 Focus on the end results
 Logic Model
 Orr’s Input-Process-Output-Outcomes Model

35. Logic Model
use the if then exercise

36. If the library provides assistance to students with their
homework then the students will do better
with their schoolwork.
If they do better with their schoolwork
then they will get better grades and attend more regularly.
If they get better grades and attend more regularly
then they are more likely to graduate.
If they graduate then they are more
likely to become employed and have a
higher standard of living.

37. Orr’s Model
Input Process Output Outcome Impact

38. Gates
Common Impact Measurement System
 Digital inclusion
 Culture & Leisure
 Education
 Communication
 Health
 Economic Development

39. Academic Library Outcomes
 Student Learning
 Teaching Effectiveness
 Research

40. Public Library Outcomes
 PLA Performance Measurement Task Force
Focus is on programs and their outcomes
 Summer Reading Programs
 Early Childhood Education Programs
 Civic Engagement
 Digital Access & Learning
 Economic Development

41. Communication
 Know your customer needs
 What’s going to resonate
 Data + stories = success
 Focus on value – not how busy you are

42. Data, Evidence and Outcomes
Joe AT JoeMatthews.Org