Reed Early's presentation on Choosing Performance Indicators from June 10 - see matched PPTX and DOCX files.

1. Performance Indicators WithinPerformance Indicators Within
and Across Community Settingsand Across Community Settings
Reed EarlyReed Early
Canadian Evaluation SocietyCanadian Evaluation Society
Conference 2013Conference 2013
TorontoToronto

2. 2
Activity 1 -Activity 1 - Obstacles toObstacles to
selecting indicators (3-4 min)selecting indicators (3-4 min)
1.1. Close eyes and write name on piece of paperClose eyes and write name on piece of paper
2.2. Turn it over and write place of birth using non-Turn it over and write place of birth using non-
dominant handdominant hand
3.3. Swap cards with person next & verballySwap cards with person next & verbally
instruct them to write your age in monthsinstruct them to write your age in months
without telling them what it actually iswithout telling them what it actually is
4.4. Optional - Provide lengthy written instructionsOptional - Provide lengthy written instructions
via a Policy and Procedure Manual on how tovia a Policy and Procedure Manual on how to
measure your success on the cardmeasure your success on the card

3. 3
Logic ModelLogic Model
Inputs Activities Outputs Outcomes Ultimate
Outcomes
Performance Indicators

4. 4
Organization
Ultimate indicators
Immediate
performance indicators
Beyond the sphere
of influence Beyond attribution
Program/Service Environment
Distant Environment
Performance Indicator ProximityPerformance Indicator Proximity

5. Management Choice of Indicators
Goals Objectives Indicators
Strategic Choice of Indicators
Strat-
egies
IndicatorsTargets
Consultative Choice of Indicators
Data
Collection IndicatorsAnalysis
Best Practice Choice of Indicators
Indicators
Borrow &
Adapt
Shared and
evaluated
Styles of Indicator SelectionStyles of Indicator Selection

6. 6
ManagementManagement
Indicators reflect goals and objectives –Indicators reflect goals and objectives –
i.e. related toi.e. related to logic model outcomeslogic model outcomes
Useful in service/prevention agenciesUseful in service/prevention agencies
Often mandated via legislationOften mandated via legislation
This style may be used by seniorThis style may be used by senior
managers to independently choose themanagers to independently choose the
indicatorsindicators

7. 7
StrategicStrategic
Indicators reflect strategies and targetsIndicators reflect strategies and targets
– i.e.– i.e. activities, outputs and nearactivities, outputs and near
outcomesoutcomes
Useful for quantifiable or tangibleUseful for quantifiable or tangible
productsproducts
Often result of accountability initiativeOften result of accountability initiative
This style may be used by performanceThis style may be used by performance
team in a larger agencyteam in a larger agency

8. 8
ConsultativeConsultative
Indicators based on focus group or survey ofIndicators based on focus group or survey of
employeesemployees (or clients)(or clients)
Useful for decentralized, flat organizationsUseful for decentralized, flat organizations
Requires time, effort and original dataRequires time, effort and original data
collectioncollection
May be used by smaller agencies practicingMay be used by smaller agencies practicing
more democratic managementmore democratic management

9. 9
Best PracticesBest Practices
Selecting indicators fromSelecting indicators from
literature/Internet, from similar leadingliterature/Internet, from similar leading
agencies, and adaptedagencies, and adapted
Useful to any agency with resources toUseful to any agency with resources to
search and researchsearch and research
Requires time, effort, and openness toRequires time, effort, and openness to
experimentationexperimentation
May provide benchmark comparabilityMay provide benchmark comparability

10. 10
Health IndicatorsHealth Indicators
Surgical wait time
Immunization rate
Infant mortality rate
Hospital acquired infections
Inpatient mortality rate
Cost per bed/day

11. 11
Social Services IndicatorsSocial Services Indicators
Child Behavior Checklist
Youth unemployment rate
Teen pregnancy and childbearing
School attendance or graduation rate
Children in families below poverty line
Cost per child in care

12. 12
Industry IndicatorsIndustry Indicators
SalesSales
Sales of new productsSales of new products
Value added to raw material consumedValue added to raw material consumed
Cost savings to industry i.e. reducedCost savings to industry i.e. reduced
training and down timetraining and down time
Increased market share %
Increased geographic penetration

13. 13
Continuing EducationContinuing Education
Reach Relevance
Staffing Overhead
Educational Financial
Efficiency ratio
Effectiveness ratio

14. 14
Performance MeasurementPerformance Measurement
SystemsSystems
community results based indicators, i.ecommunity results based indicators, i.e..
Tools For Action Series A resource guideTools For Action Series A resource guide
for designing a community indicatorfor designing a community indicator
projectproject. a report by SPARC BC, April 2008. a report by SPARC BC, April 2008
– Community Objective and PotentialCommunity Objective and Potential
IndicatorsIndicators, United Way of Greater, United Way of Greater
Milwaukee, Planning, Allocation &Milwaukee, Planning, Allocation &
Monitoring DivisionMonitoring Division

15. 15
How do you develop PI’sHow do you develop PI’s

16. 16
Conference Indicators…Conference Indicators…
– Participant satisfactionParticipant satisfaction
– Citations to conferenceCitations to conference
– Publications from conferencePublications from conference
– Networking and outside connectionsNetworking and outside connections
– Sleeper effectsSleeper effects
– Diffusion of benefitsDiffusion of benefits
– ……othersothers

17. 17
Criteria for IndicatorsCriteria for Indicators
Is it:Is it:
– Authoritative - commonly agreed to beAuthoritative - commonly agreed to be
truetrue
– Economical – only what’s neededEconomical – only what’s needed
– EthicalEthical
– Feasible – possible to measureFeasible – possible to measure
– Logical – outputs vs. outcomesLogical – outputs vs. outcomes

18. 18
Criteria Con’tCriteria Con’t
Is it:Is it:
– Manageable - suggest 10 -40Manageable - suggest 10 -40
– Measurable – qualitative or quantitativeMeasurable – qualitative or quantitative
– Reliable - accurateReliable - accurate
– Specific – at the right level of precisionSpecific – at the right level of precision
– Visible/accessible like a car dashboardVisible/accessible like a car dashboard
– Timely True and ValidTimely True and Valid

19. 19
Analog and ContinuousAnalog and Continuous
MeasuresMeasures

20. 20
Digital and Logical MeasuresDigital and Logical Measures

21. 21
Qualitative and SpacialQualitative and Spacial

22. 22
Ratio and Rate MeasuresRatio and Rate Measures

23. 23
Take away notesTake away notes
Performance indicators relate to logic modelsPerformance indicators relate to logic models
Indicator selection requires care, time andIndicator selection requires care, time and
conscious thoughtconscious thought
Different styles are:Different styles are:
– Management, strategic, consultative, best practiceManagement, strategic, consultative, best practice
Indicators should be selected and evaluatedIndicators should be selected and evaluated
against criteriaagainst criteria