Introduction to evaluation capacity and action research for development programs

1. Non-linear learning for projects
Originally presented in workshop on evaluation capacity building
with Education Innovation For India by Clare Strawn, PhD and Stéphan
Vincent-Lacrin, PhD of OECD
July 26-28, 2013
Hosted by Sri Aurobindo Society
Pondicherry, India

2. “We don’t really like the word
‘evaluation’ rather we are more
interested in creating dynamic
feedback mechanisms”
- EIFI grantee

3. What is evaluative capacity?
Evaluation capacity building involves the activities
and processes that help create, strengthen, and
sustain the learning of individuals and communities.
It is a foundation of sustainable communities.
• Increases capacity to monitor and adapt to
changing environment
• Creates shared knowledge to guide decision
making and planning
• Develops participation of staff, clients, and other
stakeholders in success of your endeavors

5. If the staff feels disconnected, they don’t see the whole picture. They
don’t know where the program is going. Once they buy into it they say, ‘‘I
will take leadership of this piece.’’

6. Organizations
may partake in
evaluation to
reduce
uncertainty and
secure
resources,
including
financial and
political
resources.

7. Evaluation capacity can also meet an internal
demand to achieve community goals.

8. Cultivating –
doing the work of the program

9. Harvesting the produce - data

10. Slicing and Dicing
to make it useful

11. Cooking –
interpreting to make meaning
Not the same as “cooking the books”

12. Presentations to different
stakeholders

13. Implementing
Community of practice

14. Continuous learning process
• Leadership vision and commitment
• Stakeholders – who should be at the table?
• Focus – define questions, priorities, uses.
What is your theory of change?
• Map activities and data flow. How will you
know if things are working and adapting?
• Plan data collection and management
integrated into activities
• Interpret findings collaboratively
• Implement recommendations

15. “The commitment of
organizational
leadership was
critical in
transforming the role
of evaluation from
one of basic
reporting and
accountability to a
true process of
continuous learning”
http://contentlibrary.theglobalfund.org/eLearning/me/01e
n/Index1.html
Leaders must “share responsibilities and find ways to integrate
evaluation into organizational life”

16. Non-linear program models
Its only new
because it
has been
missing for
so long:
Indigenous
Evaluation
Capacity
Building.
Anderson,
Chase,
Johnson III,
Mekiana,
McIntyre,
Ruerup Kerr.
American
Journal of
Evaluation
2012 33: 566

17. Example
Logic model for workshop
Needs Activities Outputs Outcomes
• Reliable reporting
on project impact
• Stories
• Baseline
assessment
• Report on state of
eval capacity
• Experience
evaluating global
projects
• Collectively
defined needs
• Asset mapping
• Critical thinking
• Skill development
& sharing
• Collaboration
• Project analysis
• Tools
• Review and
revision of
program data and
information plans.
• Adaptation of
tools and skills
for specific
program needs.
• Leadership and
confidence
• Transfer to
project staff
• Accurate data
reports
• Useful stories
• Sustainable
systems and
program
infrastructure
• Network of peer
to peer
collaboration for
program
improvement

18. Sample logic model template

19. Thinking about Data
• Data burden on participants
• How will it be used informs what you
collect.
• What “Unit of analysis” is needed to
answer your question?
• Check for consistent comparisons
• Cultural validity and construct validity
• Unintended consequence

20. From The Community Tool Box
http://ctb.ku.edu/en/tablecontents/sub_section_tools_1338.aspx