The document discusses defining new metrics for library success. It notes that libraries typically focus on quantitative measures of past performance rather than qualitative measures of impact. It advocates identifying value through conversations with stakeholders and focusing on measures that demonstrate how programs and services add value. The document outlines a framework for measurement that involves understanding the context, aligning strategies and objectives, identifying services and programs, defining appropriate measures, managing measurement data, translating data into outcomes and impacts, and communicating results. It provides an example of measuring a job skills training program using inputs, outputs, outcomes, and impacts.

1. witnessed @ Defining New Metrics for Library Success
04.22.2014
presented by Moe Hosseini-Ara

2. Underlying assumptionsUnderlying assumptionsUnderlying assumptionsUnderlying assumptions
Few libraries have a “culture of assessment”…
• Tend to focus on quantitative“performance”
• Most measuresindicatepast performance
• Need to do more qualitativeperformancemeasures
• Difficultand complex (is the excuse given)
• Have we stopped and asked why we do what weHave we stopped and asked why we do what weHave we stopped and asked why we do what weHave we stopped and asked why we do what we
do?do?do?do?

3. Underlying assumptionsUnderlying assumptionsUnderlying assumptionsUnderlying assumptions
Identifying& illustratingvalue depends on
conversations with your stakeholders
…the first conversationshouldn’t be when measures
are presented – if they don’t know you exist…
“More” isn’tnecessarily better
…in fact,“more” clouds the issue and the message

4. Underlying assumptionsUnderlying assumptionsUnderlying assumptionsUnderlying assumptions
Noteverythingthatcountscanbe
countedANDnoteverythingthatcan
becountedcounts…
…Wait!What??
-Einstein

5. Questions driving measuresQuestions driving measuresQuestions driving measuresQuestions driving measures
• What is your library or information centredoing?
• How much is it doing?
• How well is it doing it?
• Who is it doing it for?
• What positive impact is theWhat positive impact is theWhat positive impact is theWhat positive impact is the
service having for them?service having for them?service having for them?service having for them?
• Most important questionMost important questionMost important questionMost important question ----
Are you adding VALUE?Are you adding VALUE?Are you adding VALUE?Are you adding VALUE?

6. Value
SatisfactionOperational
Three types of measuresThree types of measuresThree types of measuresThree types of measures

7. Operational/Usage MeasuresOperational/Usage MeasuresOperational/Usage MeasuresOperational/Usage Measures
• We’velong tracked“how much”
• We also need to look at differences:
• Peaks? Dips? Switches?
• What are thesestatistics
really telling us? Or, not?
• Who needs to see these
statistics?
• Are they telling a storythat
you want to tell?

8. Customer Satisfaction MeasuresCustomer Satisfaction MeasuresCustomer Satisfaction MeasuresCustomer Satisfaction Measures
• Tendsto focuson existing productsand/or services
• Usuallymeasured throughsurveys
• Are we also measuring
importance?
• Surveyburnout, plus the
expectationis to scoreall
“excellent”or “10 out of 10”

9. The VALUE proposition…Measures?The VALUE proposition…Measures?The VALUE proposition…Measures?The VALUE proposition…Measures?
• Are we adding value?
• What differenceare we making?
• What is the impactof our servicesand/or programs?
• How are wecontributing
to the successof….?

10. Meaningful measuresMeaningful measuresMeaningful measuresMeaningful measures
• Matterto you AND your stakeholders
• Are conveyedin thestakeholders’ language
• More effectivewhen defined using all three typesof measures
• Demonstratethat your programs and servicesadd value
• Focus attentionon what is most
importantfor theorganization
• Are criticalfor managing, planning
& decision-making
• Are organization-dependent

12. The Logic ModelThe Logic ModelThe Logic ModelThe Logic Model
…goes beyond documenting whatyou didand
measures whatdifference you made in the life of
your target audience…how has your audience
changed…specifically change in the human
condition…

13. Measuring Impact/ValueMeasuring Impact/ValueMeasuring Impact/ValueMeasuring Impact/Value
Change in the human condition?
• Behavior
• Skill
• Knowledge
• Attitude
• Circumstances
• Awareness
• Motivation
• Condition
• Status

14. IMPACT
Equipment
INPUTINPUTINPUTINPUT
resourceresourceresourceresource
perspectiveperspectiveperspectiveperspective
OUTPUTOUTPUTOUTPUTOUTPUT
operationaloperationaloperationaloperational
perspectiveperspectiveperspectiveperspective
OUTCOMEOUTCOMEOUTCOMEOUTCOME
useruseruseruser
perspectiveperspectiveperspectiveperspective
IMPACTIMPACTIMPACTIMPACT
stakeholderstakeholderstakeholderstakeholder
perspectiveperspectiveperspectiveperspective
It’s logical that:It’s logical that:It’s logical that:It’s logical that:

15. IMPACT
Equipment
INPUTINPUTINPUTINPUT
ResourceResourceResourceResource
PerspectivePerspectivePerspectivePerspective
• Equipment
• Space
• Supplies
• Funding
• Staff

16. IMPACT
Equipment
OUTPUTOUTPUTOUTPUTOUTPUT
OOOOperationalperationalperationalperational
PPPPerspectiveerspectiveerspectiveerspective
• Training module
• Program
• Report
• # of Program Attendees
• # of Programs Held

17. IMPACT
OUTCOMEOUTCOMEOUTCOMEOUTCOME
UserUserUserUser
PerspectivePerspectivePerspectivePerspective
• Increased skill
• Know-how or know-that
• Change in status
• Change in behavior or
attitude

18. IMPACT
IMPACTIMPACTIMPACTIMPACT
StakeholderStakeholderStakeholderStakeholder
PerspectivePerspectivePerspectivePerspective
• Lower dropout rates
• Increased employment
levels
• Increased recovery rates

19. Job SkillsJob SkillsJob SkillsJob Skills TrainingTrainingTrainingTraining
Input
(resources)
• 10 hours of staff time
• $100 budget required to create the program (supplies and
handouts)
Output
(operations)
• 2 hour job skills program created for student job seekers
• 4 programs offered over a period of a month
• total of 107 participants in attendance
Outcome
(user)
• Students gained resume and cover letter writing skills
• Knowledge of where to look for jobs and submit applications
• Ability to give interviews, proficiency in answering interview
questions
Impact
(stakeholder)
• 80% of the participants who attended the program reported that
they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following
graduation

20. Job SkillsJob SkillsJob SkillsJob Skills TrainingTrainingTrainingTraining
Input
(resources)
• 10 hours of staff time
• $100 budget required to create the program (supplies and
handouts)
Output
(operations)
• 2 hour job skills program created for student job seekers
• 4 programs offered over a period of a month
• total of 107 participants in attendance
Outcome
(user)
• Students gained resume and cover letter writing skills
• Knowledge of where to look for jobs and submit applications
• Ability to give interviews, proficiency in answering interview
questions
Impact
(stakeholder)
• 80% of the participants who attended the program reported that
they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following
graduation

21. Job SkillsJob SkillsJob SkillsJob Skills TrainingTrainingTrainingTraining
Input
(resources)
• 10 hours of staff time
• $100 budget required to create the program (supplies and
handouts)
Output
(operations)
• 2 hour job skills program created for job seekers
• 4 programs offered over a period of a month
• total of 107 participants in attendance
Outcome
(user)
• Students gained resume and cover letter writing skills
• Knowledge of where to look for jobs and submit applications
• Ability to give interviews, proficiency in answering interview
questions
Impact
(stakeholder)
• 80% of the participants who attended the program reported that
they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following
graduation

22. Job SkillsJob SkillsJob SkillsJob Skills TrainingTrainingTrainingTraining
Input
(resources)
• 10 hours of staff time
• $100 budget required to create the program (supplies and
handouts)
Output
(operations)
• 2 hour job skills program created for job seekers
• 4 programs offered over a period of a month
• total of 107 participants in attendance
Outcome
(user)
• Participants gained resume and cover letter writing skills
• Knowledge of where to look for jobs and submit applications
• Ability to give interviews, proficiency in answering interview
questions
Impact
(stakeholder)
• 80% of the participants who attended the program reported that
they were either able to get interviews or actual job placements
• University can claim that students are able to get jobs following
graduation

23. Job SkillsJob SkillsJob SkillsJob Skills TrainingTrainingTrainingTraining
Input
(resources)
• 10 hours of staff time
• $100 budget required to create the program (supplies and
handouts)
Output
(operations)
• 2 hour job skills program created for job seekers
• 4 programs offered over a period of a month
• total of 107 participants in attendance
Outcome
(user)
• Participants gained resume and cover letter writing skills
• Knowledge of where to look for jobs and submit applications
• Ability to give interviews, proficiency in answering interview
questions
Impact
(stakeholder)
• 80% of the participants who attended the program reported that
they were either able to get interviews or actual job placements
• Library can claim that as a result of their program residents are
able to get jobs, contribute to economy and lower unemployment

24. BEGINBEGINBEGINBEGINWITHWITHWITHWITH
THETHETHETHEIN MINDIN MINDIN MINDIN MIND
ENDENDENDEND
Covey 1989Covey 1989Covey 1989Covey 1989
80% of the participants who attended the program reported that they80% of the participants who attended the program reported that they80% of the participants who attended the program reported that they80% of the participants who attended the program reported that they
were either able to get interviews or actual jobwere either able to get interviews or actual jobwere either able to get interviews or actual jobwere either able to get interviews or actual job placementsplacementsplacementsplacements
Library can claim that as a result of their program residents are able to getLibrary can claim that as a result of their program residents are able to getLibrary can claim that as a result of their program residents are able to getLibrary can claim that as a result of their program residents are able to get
jobs, contribute to economy and lower unemploymentjobs, contribute to economy and lower unemploymentjobs, contribute to economy and lower unemploymentjobs, contribute to economy and lower unemployment

25. MeasurementMeasurementMeasurementMeasurement
FrameworkFrameworkFrameworkFramework
UnderstandUnderstandUnderstandUnderstand
the Contextthe Contextthe Contextthe Context
AlignAlignAlignAlign
Strategies &Strategies &Strategies &Strategies &
ObjectivesObjectivesObjectivesObjectives
IdentifyIdentifyIdentifyIdentify
Services &Services &Services &Services &
ProgramsProgramsProgramsPrograms
DefineDefineDefineDefine
MeasuresMeasuresMeasuresMeasures
ManageManageManageManage
MeasurementMeasurementMeasurementMeasurement
DataDataDataData
TranslateTranslateTranslateTranslate
Data intoData intoData intoData into
Outcomes &Outcomes &Outcomes &Outcomes &
ImpactsImpactsImpactsImpacts
CommunicateCommunicateCommunicateCommunicate
ResultsResultsResultsResults

26. Understand
the Context

27. Know the Context & theKnow the Context & theKnow the Context & theKnow the Context & the CCCContextersontextersontextersontexters
How do theyHow do theyHow do theyHow do they
measure theirmeasure theirmeasure theirmeasure their
Progress/success?Progress/success?Progress/success?Progress/success?
Know yourKnow yourKnow yourKnow your
Stakeholders/InflueStakeholders/InflueStakeholders/InflueStakeholders/Influe
ncersncersncersncers –––– who arewho arewho arewho are
they?they?they?they?
How is valueHow is valueHow is valueHow is value
defined,defined,defined,defined,
measured &measured &measured &measured &
communicated?communicated?communicated?communicated?

28. • Libraries days are numbered
• Libraries aren’t relevant
• Everyone has computers and internet
• Need information? It’s all online
• Research happens online not in libraries
• With eBooksand research online believes facilities
can be made smaller – no need for print material
Councillor Ural Doomed

29. Dean of Humanities
Dr. Kee Paper
• Influential in university & broader academic
circles
• Focus on research
• Has deep seated beliefs in traditional library
services
• Quiet
• No technology
• Libraries are institutionsfor reading &
research

30. Align
Strategies
&
Objectives

31. Align Strategies & ObjectivesAlign Strategies & ObjectivesAlign Strategies & ObjectivesAlign Strategies & Objectives
Stakeholder’s Goals & Objectives Your Goals & Objectives: How they
Contribute
Accelerate newcomers contribution
to the community & economy
Offer programs and services that
connect newcomers with relevant
agencies and services
Increased research grants Support the research process and
assist with submission and
completion of grants

32. Align objectivesAlign objectivesAlign objectivesAlign objectives
• Articulate & document your goals &
objectives
Whatareyoutryingtoaccomplish?
• Clarify how these contribute towards
your stakeholders’ objectives &
desired outcomes

33. Identify
Services &
Programs

34. Identify ServicesIdentify ServicesIdentify ServicesIdentify Services
What services,programs or productscontributeto your realizing
an objective?
Stakeholder Goals & Objectives
Your Goals & Objectives
Your Services

36. Identify ServicesIdentify ServicesIdentify ServicesIdentify Services
Library andLibrary andLibrary andLibrary and
UniversityUniversityUniversityUniversity
PrioritiesPrioritiesPrioritiesPriorities
What we do…What we do…What we do…What we do…

38. Define
Measures

39. Define successDefine successDefine successDefine success
• What will success look like?
• How will you track progress towardsmeeting goals?
• What indicators or measures will you use?
• Qualitative? Quantitative?
• What data needs to be collected?
• Keep it key
• Keep it simple
• Be practical
• Don’t get mired– or
overworkfrontlinestaff

40. Manage
Measurement
Data

41. Manage collectionManage collectionManage collectionManage collection
• Determine a collection schedule
• Work out agreements with other departments that
have needed data
• Are you currently collecting data or indicatorsthat
are no longer relevant?
• How long do you need to keep data?
• Who is responsible?

42. Translate
Data into
Outcomes &
Impacts

43. Define outcomes & impactDefine outcomes & impactDefine outcomes & impactDefine outcomes & impact
• What will success look like?
• And how will you “get” there?
OUTCOMEOUTCOMEOUTCOMEOUTCOME IMPACTIMPACTIMPACTIMPACT
INPUTINPUTINPUTINPUT OUTPUTOUTPUTOUTPUTOUTPUT

44. Define outcomes & impactDefine outcomes & impactDefine outcomes & impactDefine outcomes & impact
OUTCOMEOUTCOMEOUTCOMEOUTCOME
UserUserUserUser
PerspectivePerspectivePerspectivePerspective
• Increased skill
• Know-how or know-that
• Change in status
• Change in behaviour or
attitude
IMPACTIMPACTIMPACTIMPACT
StakeholderStakeholderStakeholderStakeholder
PerspectivePerspectivePerspectivePerspective
• Lower dropout rates
• Increased employment
levels
• Increased recovery rates

45. Communicate
Results

46. Communicate resultsCommunicate resultsCommunicate resultsCommunicate results
• Actuallybegins when you are understanding the context
• Your message: “wehave
contributedtowardsyourgoals
by……”
• If your goals & objectivesare meaningful for your planning
& decision-making, your measures will be indispensable

47. What do you need to stop doing?What do you need to stop doing?What do you need to stop doing?What do you need to stop doing?

48. Value
SatisfactionOperational
The Sweet SpotThe Sweet SpotThe Sweet SpotThe Sweet Spot

49. ResourcesResourcesResourcesResources
IMLSIMLSIMLSIMLS WebographyWebographyWebographyWebography
http://www.imls.gov/applicants/webography.aspx
ShapingShapingShapingShaping OutcomesOutcomesOutcomesOutcomes
http://www.shapingoutcomes.org/index.htm
OvercomingOvercomingOvercomingOvercoming Our Habits and Learning to MeasureOur Habits and Learning to MeasureOur Habits and Learning to MeasureOur Habits and Learning to Measure ImpactImpactImpactImpact by Moe Hosseini-Ara and Rebecca Jones.
Information Today, Inc. June 2013.
http://www.infotoday.com/cilmag/jun13/Hosseini-Ara_Jones--Overcoming-Our-Habits-and-Learning-to-
Measure-Impact.shtml
SevenSevenSevenSeven Easy Steps to Help You MeasureEasy Steps to Help You MeasureEasy Steps to Help You MeasureEasy Steps to Help You Measure UpUpUpUp by Moe Hosseini-Ara and Rebecca Jones. Information Today, Inc.
July/August 2013.
http://www.infotoday.com/cilmag/jul13/Hosseini-Ara_Jones--Seven-Easy-Steps-to-Help-You-Measure-Up.shtml

50. MOE
HOSSEINI-ARA
mhosse@markham.ca
Thank you!
Special thanks to Rebecca Jones (Dysart & Jones) for introducing me to the logic model and allowing
me the honor to teach this material with her at numerousconferences…