Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Why Evaluate? 
Before After 
www.usablellc.net
Organizations evaluate in order to find the number.
The number of kids who graduate 
from high school.
The number of people who are able to 
get and keep a new job for six months.
The number of points by which 
students’ math scores rise.
The number of points by which a 
psychiatric patient’s depression score 
decreases.
These numbers all describe outcomes.
Outcomes are about the results of a program.
Outputs are about the steps you 
followed to produce those 
outcomes.
It’s important to understand the difference.
But what is the number for?
Why do we need to know the number?
Well, existing funders want to know if 
their investments yielded results.
New funders want to invest 
in programs that work.
Board members, staff, clients and 
community stakeholders want to 
associate with programs that are 
effective.
But what really is the number?
To be useful, the number has to 
represent a comparison against 
expectations.
Expectations that without the program, 
people/ institutions/ communities would 
not have changed.
Expectations that without the program, 
people/ institutions/ communities would 
not have changed. 
* If you remember back...
So essentially, we have an experiment.
Differences between the number before 
the program and the number after the 
program have to be due to the 
program*. 
*Al...
That was easy, right?
Pretty reasonable, wouldn’t you say?
Did you ever sign up for an online service 
where you had to check a box indicating 
that you had read and agreed to all t...
Did you actually read all the fine print?
Neither did we.
But in this case you have to.
Differences between the number before 
the program and the number after the 
program have to be due to the 
program*. 
*Al...
But all other things are N E V E R equal.
The boys may do better than the girls.
Counselors may interpret the 
treatment modality differently.
Participants with different work 
histories may respond differently to 
the work readiness program.
Looking at the average 
change in: 
Math Grade 
Depression Score 
Months on the job 
may not be enough.
But wait, it gets even worse. 
What if the boys do better than the girls 
UNLESS the girls are taught by Ms. Smith 
in which case they do better?
What if more experienced counselors 
do better regardless of differences in 
treatment modality?
What if differences in work histories 
are only important for workers 
moving from traditional 
office work?
Organizations that want to improve their 
programs need to understand these 
differences.
Because maybe the girls need more 
teachers like Ms. Smith.
Because maybe the girls need more 
teachers like Ms. Smith. 
*Find out what Ms. Smith 
does that she’s so 
effective with ...
And only the less experienced 
counselors need more training.
And only the less experienced 
counselors need more training. 
* Devote resources to 
retraining just the less 
experience...
And the workforce readiness program 
needs to treat people with experience 
doing office work differently than 
those who ...
And the workforce readiness program 
needs to treat people with experience 
doing office work differently than 
those who ...
Maybe what this means is that 
evaluation is about more than just 
the number.
Maybe beyond asking 
the question “did the 
program work” it 
should ask: 
Where did it work? 
For whom? 
Under what circu...
We believe that effective organizations use 
evaluation to document positive outcomes….
& to inform continuous improvement.
If you found this useful, we’d love it if 
you would share it with your networks. 
www.usablellc.net
Upcoming SlideShare
Loading in …5
×

Why evaluate your program?

12,744 views

Published on

This slideshare explains why nonprofit organizations should evaluate their programs. Beyond documenting program outcomes, it argues that program evaluation can and should be a key component of program improvement.

Published in: Government & Nonprofit

Why evaluate your program?

  1. 1. Why Evaluate? Before After www.usablellc.net
  2. 2. Organizations evaluate in order to find the number.
  3. 3. The number of kids who graduate from high school.
  4. 4. The number of people who are able to get and keep a new job for six months.
  5. 5. The number of points by which students’ math scores rise.
  6. 6. The number of points by which a psychiatric patient’s depression score decreases.
  7. 7. These numbers all describe outcomes.
  8. 8. Outcomes are about the results of a program.
  9. 9. Outputs are about the steps you followed to produce those outcomes.
  10. 10. It’s important to understand the difference.
  11. 11. But what is the number for?
  12. 12. Why do we need to know the number?
  13. 13. Well, existing funders want to know if their investments yielded results.
  14. 14. New funders want to invest in programs that work.
  15. 15. Board members, staff, clients and community stakeholders want to associate with programs that are effective.
  16. 16. But what really is the number?
  17. 17. To be useful, the number has to represent a comparison against expectations.
  18. 18. Expectations that without the program, people/ institutions/ communities would not have changed.
  19. 19. Expectations that without the program, people/ institutions/ communities would not have changed. * If you remember back to your statistics course, this was called the null hypothesis.
  20. 20. So essentially, we have an experiment.
  21. 21. Differences between the number before the program and the number after the program have to be due to the program*. *All other things being equal
  22. 22. That was easy, right?
  23. 23. Pretty reasonable, wouldn’t you say?
  24. 24. Did you ever sign up for an online service where you had to check a box indicating that you had read and agreed to all the fine print?
  25. 25. Did you actually read all the fine print?
  26. 26. Neither did we.
  27. 27. But in this case you have to.
  28. 28. Differences between the number before the program and the number after the program have to be due to the program*. *All other things being equal
  29. 29. But all other things are N E V E R equal.
  30. 30. The boys may do better than the girls.
  31. 31. Counselors may interpret the treatment modality differently.
  32. 32. Participants with different work histories may respond differently to the work readiness program.
  33. 33. Looking at the average change in: Math Grade Depression Score Months on the job may not be enough.
  34. 34. But wait, it gets even worse. 
  35. 35. What if the boys do better than the girls UNLESS the girls are taught by Ms. Smith in which case they do better?
  36. 36. What if more experienced counselors do better regardless of differences in treatment modality?
  37. 37. What if differences in work histories are only important for workers moving from traditional office work?
  38. 38. Organizations that want to improve their programs need to understand these differences.
  39. 39. Because maybe the girls need more teachers like Ms. Smith.
  40. 40. Because maybe the girls need more teachers like Ms. Smith. *Find out what Ms. Smith does that she’s so effective with the girls.
  41. 41. And only the less experienced counselors need more training.
  42. 42. And only the less experienced counselors need more training. * Devote resources to retraining just the less experienced counselors.
  43. 43. And the workforce readiness program needs to treat people with experience doing office work differently than those who held blue collar jobs.
  44. 44. And the workforce readiness program needs to treat people with experience doing office work differently than those who held blue collar jobs. * Create two different programs, one for former office workers, the other for former blue collar workers.
  45. 45. Maybe what this means is that evaluation is about more than just the number.
  46. 46. Maybe beyond asking the question “did the program work” it should ask: Where did it work? For whom? Under what circumstances?
  47. 47. We believe that effective organizations use evaluation to document positive outcomes….
  48. 48. & to inform continuous improvement.
  49. 49. If you found this useful, we’d love it if you would share it with your networks. www.usablellc.net

×