The document outlines a framework for outcome measurement in program evaluation, emphasizing the importance of tracking participant changes to ensure accountability and efficiency. It discusses implementation steps, including building consensus and using logic models, as well as various data collection methods to assess program impact effectively. Recommendations include starting simple, assigning measurement responsibilities, and collecting data systematically over an 18-24 month timeline.

1. Outcome Measurement
Determining & Selecting Outcomes
A General Framework
12/6/2013
Rebecca Rodriguez
For the Philanthropist
1

2. Agenda








Purpose & Importance
Literature Review
How To implement
Outcomes
Logic Models
Data Collection Methods
Recommendations
References
12/6/2013
Rebecca Rodriguez
For the Philanthropist
2

3. Introduction

Outcome Measurement is the regular,
systematic tracking of the extent to
which program participants
experience the benefits or changes
intended.
- United Way of America
12/6/2013
Retrieved from the United Way of
America web site:
http://www.liveunited.org/outcomes
3

4. Digging Deeper…

The observance, tracking and analysis
of changes to participants in a program.



12/6/2013
Measurable & meaningful.
Directly linked to program activities.
Useful for program improvement.
Outcomes for Success! Organizational
Research Services, Inc. & Clegg and
Assoc., Inc.
4

5. Importance of Outcome
Measurement





Accountability, Responsiveness, Efficiency
Helps staff make informed decisions.
To learn that your efforts are helping and
not harming.
Helps organization learn to tell “their story”
effectively.
Can support fundraising efforts.
12/6/2013
Outcomes for Success!; United Way of
America, Center for What Works
5

6. Outcome Measurement is NOT




12/6/2013
Formal evaluation.
The end all and answer to all our
program questions/ problems.
A one-size- fits all solution.
A quick and easy fix.
6

7. Literature Review
Plethora of information that falls into two
main areas:
- Importance of
- Implementation Process


Common elements:
- Complexity; general lack of understanding
- Sharing of best practices
12/6/2013
7

8. Implementation Steps
7.
Build consensus – bring everyone to the table.
Start with a logic model; talk about theory of change.
Look at best practices already working!
Determine & select outcomes.
Determine data collection methods.
Collect, record, analyze data.
Reflect, talk about, brainstorm.
Make adjustments to program if necessary.
8.
Share your findings.
1.
2.
3.
4.
5.
6.
12/6/2013
Center for What Works;
www.innonet.org; Outcomes for
Success!; Instituting an Outcomes
Assessment Effort
8

9. Knowledge
Attitudes
Skills
Outcomes to Measure
Status
Behaviors
Awareness
12/6/2013
Outcomes - United Way of America
9

10. Logic Model Template
Planned Work
Resources/Inputs
12/6/2013
Intended Results
Activities
Outputs
Adapted from the Logic Model
Development Guide found at
www.kellogg.org
Outcomes
Goal
10

11. What Area Agencies on Aging
Measure

Time from referral to service – status

Provider quality – knowledge, behavior, skill,
attitude

Assisting consumers planning in advance for longterm care – knowledge, skills, attitudes, behavior

Positioning AAAs as the single point of entry for
long-term care – knowledge, awareness
12/6/2013
Personal Communication with The
Lewin Group
11

12. Data Collection Methods
Reliability vs. Validity
12/6/2013
validity
O
bs
er
va
t io
ns
Ca
se
Re
co
rd
s
ws
reliability
In
te
rv
ie
Su
rv
ey
s
90
80
70
60
50
40
30
20
10
0
Adapted from Outcomes for Success!
12

13. Elements for Success





Build consensus.
A strong desire to demonstrate impact.
Willingness to make change.
Be disciplined in collecting / analyzing
data.
Keep it simple.
12/6/2013
Rebecca Rodriguez
For thePhilanthropist
13

14. Recommendations
Proposed 18 - 24 month timeline:

Build consensus.

Assign an outcome measurement driver.

Start with a logic model.

Agree upon one/two outcome(s). Start simple!

Choose method of data collection: surveys or
interviews

Collect, record, analyze data.
12/6/2013
14

15. Resources









12/6/2013
www.whatworks.org
Collins, J. (2005). Good To Great And The Social Sectors. Boulder, CO: Jim Collins.
www.grantcraft.org
W.K. Kellogg Foundation
United Way of America. (2000). Agency Experiences with Outcome Measurement.
Virginia: United Way of America.
www.innonet.org (registration required)
Edna McConnell Foundation
Charting Impact – Initiative of Guidestar collaborative effort
American Evaluation Association
15