Success and Failure in the Evaluation Process What do the terms “success” and “failure” really mean in the philanthropic world? Funders have taken different approaches to learning from initiatives that haven’t gone quite as they had hoped. Some funders want to learn from their mistakes, some provide technical assistance to lagging grantees, and some want to focus their light on “bright spots” and grantee successes. In this session, Kat Athanasiades from Innovation Network will discuss how and when her organization uses grant reports in evaluation; how and why getting good evaluation data from grant reports is difficult; and potential ways to make it easier for grantees to report on failure in a way that could be useful to evaluators. Session participants will: •Know how funders can embed “failure reporting” into grant reports in ways that are useful to evaluators. •Learn ways a foundation can combat some of the "structural" impediments, e.g., trust and communication, that may prevent proper reporting on failure. •Gain ideas from fellow participants on how to understand and appreciate grantmaking "failures" as well as successes.

1. Success and Failure
in the Evaluation Process
Kat Athanasiades
kathanasiades@innonet.org
@KatAthanasiades
Annual Conference 2015, National Harbor, MD
Grants Managers Network
Tuesday, March 17, 2015

2. About Innovation Network
White papers with tips for grantmakers
Kat
Evaluation consulting firm
working with foundations
and nonprofits
Research on
evaluation practice

3. What does
success
mean to you?
To your employer?

4. What does
failure
mean to you?
To your employer?

8. Agenda
Evaluation and grant reports
1
2
3
Evaluation and failure
Embedding “failure reporting”
in grant reports

9. Embedding “failure reporting”
in grant reports
Evaluation and grant reports
1
2
3
Agenda
Evaluation and failure

10. Planning
Data
collection
Analysis &
Reflection
Action &
Improvement
1

11. “failure reporting”

12. Risk
tolerance
1

13. 1

14. What’s
going on
over here?
1

15. 1

16. 1
How tolerant is your
comes to failure?
organization when it

17. Embedding “failure reporting”
in grant reports
Evaluation and failure1
2
3
Agenda
Evaluation and grant reports

18. Planning
Data
collection
Analysis &
Reflection
Action &
Improvement
2

19. Planning
Data
collection
Analysis &
Reflection
Action &
Improvement
2

20. 2

21. Planning
What does it take to achieve
goals? To achieve success?
Which strategies are most effective in moving
our organization closer to our long-term goal of
creating meaningful experiences?
2
Why are some grantees making
more progress than others?

22. Basic Grantee Information
Grantee Name
Grant Amount
Topic Area
Grant Terms
Accountability Assessment
Key Deliverables/Activities
Outputs
Barriers/Opportunities
Staff involvement
Grant report example 12

23. Impact/Success Assessment
Change Agent Targeted
Evidence of Influence
Recommended Next Steps
Additional Follow Up Recommended
No Follow Up
Grant report example 12

24. 1. Describe your organization’s advocacy activities and
how these activities relate to your organization’s
advocacy objectives
2. Describe your accomplishments in the first year of
GSA funding. What advocacy related results have
occurred in connection with MFH funding?
Grant report example 22

25. 3. How have the GSA grant funds increased your
organization’s capacity to perform advocacy
activities?
4. What have been your organization’s challenges
(frustrations, barriers, or disappointments) in the
past year? How were these challenges addressed or
redirected?
Grant report example 22

26. Organization
Outcomes
Accomplishments
Activities
Successes
Changing context—external factors
Evidence of influence
Grant report example 3
Updates on:
2

27. Why is it so hard to
use grant reports
for evaluation?

28. 2 The shoe (data)
doesn’t fit

29. Planning
Data
collection
Analysis &
Reflection
Action &
Improvement
2 This is where
grant reports
are often
useful

30. Planning
Data
collection
Analysis &
Reflection
Action &
Improvement
2 We often lack
the data for
the latter
stages of
evaluation

31. 2
Trust

32. 2
What pieces of your
reports can be useful
organization’s grant
to evaluators?

33. Evaluation and grant reports
Evaluation and failure1
2
3
Agenda
Embedding “failure reporting”
in grant reports

34. 3 Adjust the reporting structure
Adjust the fields

35. 3 Encourage risk tolerance
Build trust with grantees
Build trust with board

36. Agenda
Evaluation and grant reports
1
2
3
Evaluation and failure
Embedding “failure reporting”
in grant reports

37. What are your
grantmaking failures
suggestions for capturing
as well as successes?

38. Success and Failure
in the Evaluation Process
Kat Athanasiades
kathanasiades@innonet.org
@KatAthanasiades
Thank you!

39. Image credits
Slide Credit
5-6 Desk pictures c/o Kat Athanasiades
7 Children in cafeteria from USDAgov flickr, Creative Commons:
http://www.flickr.com/photos/usdagov/7995804478/
12 Roulette from Scott Beale / Laughing Squid flickr, Creative Commons: laughingsquid.com or
http://www.flickr.com/photos/laughingsquid/5249116537
13-14 Spotlight/light hole from abooth202 flickr, Creative Commons:
http://www.flickr.com/photos/abooth202/2628358466
15 Mountain landscape c/o Kat Athanasiades
20-21 Road map from sidelong flickr, Creative Commons:
http://www.flickr.com/photos/sidelong/234192777/
28 Child in shoes from rumpleteaser flickr, Creative Commons:
http://www.flickr.com/photos/rumpleteaser/2940847467
31 Man and tiger from dobs flickr, Creative Commons: http://www.flickr.com/photos/dobs/10726756606
34 Cranes from 96dpi flickr, Creative Commons: http://www.flickr.com/photos/96dpi/3227807209
35 Kitten and hedgehog from elenitaspics flickr, Creative Commons:
https://www.flickr.com/photos/elenitaspics/11254227916