to interact with fellow participants focus on sharing experiences and problem-solving
Baggage exists – inaccessibility of M&E paradigm
M&E – research discipline, project-based Example: Would YOU know how to measure % of children age 6-59 months below -2 z score wt/ht?
by supporting organizational development Where they are - i.e. having realistic and appropriate expectations depending on the capacity level of a partner
Impact - such a loaded & over-used word communities can see ‘impact’ even if it’s difficult to describe… It’s their impact & success - not ours
This slide shows the trade-offs in cost and complexity among different methods of data collection. Can also think of M&E as regular/routine (part of everyday duties such as beneficiary records) or special/periodic (which require additional time, resources, planning such as household surveys) Consider the level of financial/human resources available. The effort expended should match the improvement in decision-making.
FLF made strategic decision to let small CBOs be. Orgs need to have strategy in place before they begin - middle capacity selected. Curriculum is focused on making monitoring activities “accessible” to grassroots organizations FLF made strategic decision not to tie our M&E training to reporting requirements. Of course, FLF hopes we will receive improved reports, but this is a long-term goal. 1 follow-up training in Lesotho. The training curriculum is extremely adaptable because it enables any organization to draw upon its current programming and monitoring activities. Help grantees tell their own story - build confidence in asking “so what?”
it’s about internalizing the importance of M&E – shifts in organizational culture from emergency/crisis mode to planning & reflection More important than standardized administrative procedures, M&E activities constitute an informed process of reflection.
These are abstract concepts - but understandable. Your logic does not equal your partners - other ways of knowing exist that are not empirical
M&E is about testing assumptions - that should be the new definition of building “evidence” Indicators - completely theoretical concept
Case studies, partners’ own data Follow-up training focused on design of data collection tools, data analysis and data management, most trainings end at developing M&E frameworks Respect that M&E activities must be relevant to grantees’ daily management of their organization
Takes $ and people to do good M&E. FLF working with in-country META facilitators Ultimately, M&E must be a learning/management tool for an organization. Process of transformation to a learning organization does not happen overnight. What happens within the organization is the most important factor in “institutionalizing” reflective practices.
Confident enough to train rest of organization and communities For those with frameworks or experience already - identified gaps and how it could be improved We have already observed that the training contributes to preparing grantee-partners to articulate their successes and challenges and to mobilize resources to sustain their programs, ultimately supporting a transition from Firelight funding. Groups have already demonstrated their use of more systematic and reflective monitoring practices, as well as a willingness to engage with local stakeholders based on the confidence gained from having a better understanding of their own programs’ outcomes.
1) The META training workshop was excellent and has given us insight in gathering data from the field and analyzing it before documenting it for various stakeholders. ~ISRL, Zambia 2) Tsosane “We use these 3 questions in every meeting from here on out. 1)Where are we now? 2)Where are we going? 3)How are we going to reach there?” 3) From LGGA final report, received 8/3/07: The META training has assisted the administrator to be able to gauge the organizational impact in terms of services offered (the field worker was able to account for the services she has planned), how can we expand, to what extend has the finances been utilized and how appropriate were they utilized. The roadmap was the guiding document for parameters setting and self –analysis for actual achievements.
Improved/adapted programming - LSC communities with more child neglect cases were targeted for awareness campaigns Strengthened fundraising - T’sosane Support Group was 12 out of 120 successful groups for USG grant. Only CBO to receive HR funding. Strengthened collaboration/networking – “We discuss all kinds of issues with more information at our fingertips.”
Assume nothing – transport (road and water), recordkeeping skills, volunteer staff, basic planning, beneficiary selection, funding insecurity, illiteracy, large catchment areas, financial mgmt the first issue each org different, different capacities/types of programming (e.g. orgs with smaller #s of case work vs. community mobilization) Not surprising considering output-level results are easier to comprehend and directly related to day-to-day activities. Outcomes is the new part.
already overburdened workloads Assume nothing – transport (road and water), recordkeeping skills, volunteer staff, basic planning, beneficiary selection, funding insecurity, illiteracy, large catchment areas, financial mgmt the first issue
Assume nothing – transport (road and water), recordkeeping skills, volunteer staff, basic planning, beneficiary selection, funding insecurity, illiteracy, large catchment areas, financial mgmt the first issue REALITY OF RELATIONSHIP-BASED GRANTMAKING
META Coordinator now a full-time position. Future adaptations for training - accompanying organizational capacity assessment, shorter, more frequent meetings conducted by in-country facilitators Groups beginning to see links with advocacy
Support your partners to sing their own praises!
Santa Cruz International Meet Up Mar 10
Making M&E Accessible to
Lessons Learned from Firelight’s
Capacity Building Program
Santa Cruz International
9 March 2010
• Overview of Firelight’s M&E training approach
• Engage in practical exercises to:
– Identify guiding principles for foundations’ M&E activities
– Identify how key M&E concepts and practicalities can be
better understood and utilized by grantees
• Sharing Firelight’s outcomes to date, challenges,
and way forward
Monitoring & Evaluation:
The International Development Paradigm
Methods / Data
Capacity Building Strategies
1) Training to build capacity of grantees, e.g.
“traditional” M&E skills
2) Special grants to support organization-specific
capacity development and learning needs
3) Partner learning exchanges and networking
4) Weekly “Newsflash” to share information with
5) As-needed requests
• Nurture strong, sustainable grassroots
• Meet groups “where they are.”
• Build partners’ ability and confidence to
monitor and evaluate their own work.
• Peer-to-peer technical assistance is most
• Grantees are already monitoring their work.
• Perfect beneficiary numbers are not our goal.
• Impact is NOT easy to prove due to:
• Causality and attribution
• M&E activities should never detract from the
work at hand, which is serving children.
Special or ’point’
Choosing M&E Methods
Keep expectations realistic.
• 43 partners
trained in 3
• Aims to build
own progress in
• Training is practical and experiential, to
apply principles to organizations’ day-to-
What did we hope to see?
Intended Long-Term Outcomes for Trained Grantees
• Organizations clearly identify strategies and intended
outcomes of their work.
∀ Monitoring is paired with ongoing, day-to-day work.
∀ Organizations have ownership of their monitoring framework,
tools, and procedures.
∀ Management/leaders makes changes in the organization’s
programs and activities based on what is learnt.
∀ The organization shares what it learns with local communities.
∀ Reports to an organization’s donors and other stakeholders are a
more complete depiction of the outcomes of their work.
Making M&E Accessible
What are we
we want to
How are we
going to get
What do we expect to
happen along the way?
OUTPUT OUTCOME IMPACT
we are on
OUTPUT OUTCOME IMPACTACTIVITIES
Have the activities
The very first
result of an
s have direct
s have less
s have very
Making M&E Accessible
Making M&E Accessible
Get to the practical.
Simple Steps in
Making M&E Accessible
“I appreciate that there will be support after the training,
and that we will not have carry on alone.”
Enthusiastic Feedback on the Training Curriculum/Approach
INCREASED CONFIDENCE - “We can see we are making progress.”
• “The simple words removed the fear and myths about M&E.”
• “I used to get indicators, outputs and outcomes confused but now
I know the difference.”
• “I learnt that we are already doing monitoring, and we can build
on that. But [we] need to make the process more systematic.”
RECOGNITION OF THE VALUE OF M&E
• “I learnt to think about more than outputs – that it is important to
think about the ‘So whats?’ of what we do.”
• “We need to monitor in order to be able to share our success
stories and learn from our mistakes!”
• “We are used to only thinking about monitoring at the end of
a project – this workshop has changed that!”
Evidence of “uptake” thus far
• Conceptualizing programming in terms of results
• Monitoring at the core of grantee’s planning - “We were doing
things randomly, jumping from here to there. Now we go forward
based on what our plan is.”
• Using the skills on projects not funded by Firelight - “The
knowledge we are getting is helping us improve our service
delivery, even on projects not funded by Firelight Foundation.”
• “Reporting is easier because you can go line by line, comparing
what you have done to what you planned.”
Evidence of “uptake” thus far
• Improved/adapted programming
• Strengthened fundraising - “[U.S. Embassy] officers were
shocked to find out that we are even capable of monitoring
and evaluating our project. Remember, we had written our
proposal by ourselves – there was no outside assistance.”
• Increased engagement with government frameworks
• Strengthened collaboration, networking, and advocacy
• Buy-in at organizational level - staff turnover, quality of
shared learning within organization, “Some people who
were attended were not the right people.”
• Management issues within organizations - planning
(strategic, workplans), human resources management,
• Efforts needed to ensure outcome indicators included in
grantees’ M&E frameworks
• Some baggage/anxiety about M&E remains - pleasing the
donor, some still searching for the “holy grail” of M&E
“Absence of computers
to process data and
store the information so
it can be easily
“We now need to be able
to cross the river more
• Lack of baseline data
• Data quality issues - verification needs
• Interest in participatory M&E with communities
• Data management and recordkeeping
• Lack of resources to devote to M&E (time, staff, $)
What do these challenges reveal to Firelight?
Grantees are excited.
Grantees are invested.
Grantees are “taking on” M&E within their organizations.
1. Conduct review of M&E training outcomes. Produce
publication on Firelight’s approach and lessons learned.
2. Test revised curricula for training in additional Firelight
focus countries in 2010/11.
It’s not about the indicators.
It’s about reflection and learning.
Good luck in your own M&E efforts!