1. Monitoring &
Evaluation

2. Agenda
• Introduction • M&E
• Basic concepts – Introduction to M&E
Definitions of M&E systems
Why do M&E? – How to measure
• Project Planning • What indicators to use
Theory of change • How to collect data
Result chain • What targets to set
Assumptions and risks – Putting it all together

3. Definitions
• Monitoring
Routine, systematic collection of data relating to project
performance
Provides regular feedback and early indications of progress (or lack
of progress)
• Evaluation
Episodic and objective assessment of project
Often done at mid-point and at end of project
Can determine whether objectives were achieved, how efficient the
project was, and what the project’s impact was

4. Why do M&E?
• Knowledge of impact
M&E tells you whether your projects are meeting objectives and
having results
• Improved decision-making
Information generated through M&E helps you make better
decisions about projects
• Greater accountability and transparency
M&E data allows donors and others to assess how well and
efficiently you are using resources

5. Theory of Change
• What is the theory of change for your project?
• A chain of so thats that lead to the change you hope for
• Example: we teach children how to wash their hands
So that they will understand the importance of washing their hands
AND will have the skills to wash their hands properly
So that they will wash their hands regularly
So that they will be less susceptible to certain illnesses
So that the incidence of certain illnesses will decrease
So that child mortality will decrease

6. Result Chain
Def: A picture of the chain of “so that”
relationships that lead to the desired outcomes
Source: University of Wisconsin Extension, Developing a Logic Model

7. Result Chain – Simplest Form
Purpose/ Goal/
Short- Long-
Inputs Activities Outputs
term term
objective objective

8. Result Chain – More Detailed
Inputs Activities Outputs Purpose/ Objectives Goal / Long-Term Objective
• What you invest • What you do •What you deliver • What the short- and • What the ultimate impact
• Staff • Specific actions you undertake •Products and/or services you medium-term results are is
• Volunteers to achieve particular outputs deliver (~1 yr) • Situations, conditions, behavio • Long-term impact of the
• Time •You should be able to ur changed as a consequence project (+ 5 years)
guarantee these of the project (~3-5 yrs) • Project contributes to goal, but
• Money
•Examples: • Outside your control likely doesn’t achieve it
• Research base
•Conduct workshops • Learning: • Conditions:
• Materials
•Deliver services • Changes in • Changes in
• Equipment
•Develop awareness, knowledge, attitud social, economic, civic, environ
• Technology es, skills, opinions, aspirations, mental conditions
• Partners products/curriculum/resources
motivations
•Train
• Action:
•Provide counseling
•Assess • Changes in
behaviour, practice, policies, d
•Facilitate ecision-making,
•Partner

9. Result Chain
Example: HIV Prevention for Sex Workers
Inputs Activities Outputs Purpose/Objectives Goal/Long-Term Objective
• Staff - 3 trainers • Make schedule of • Conduct 1 workshop • Learning: • HIV prevalence among
• Volunteers – national workshops every 2 months • Participants are aware of Cambodian sex workers
network of 8 volunteers • Coordinate with local • Target Cambodian sex the importance of HIV declines
• Time – 20 hours/month volunteers to recruit workers prevention
• Money – space workshop participants • Participants have
rental, snacks • Plan workshop curriculum knowledge of means of
purchase, transportation • Assemble workshop transmission, methods of
reimbursement materials prevention
• Research base – • Book workshop facilities • Participants have
knowledge of negotiation skills
STDs, knowledge of • Action:
training techniques • Participants practice safer
• Materials – workshop sex
materials

10. Assumptions and Risks
• How do you plan for events, conditions, and decisions
beyond your direct control?
• Assumptions
positive statements about what will go right
• Risks
negative statements about what might go wrong
Source: AusAid, AusGuideline 3.3 The Logical Framework Approach, 2005; NORAD, The Logical Framework
Approach (LFA): Handbook for objectives oriented planning (4th ed.), 1999.

11. Assumptions and Risks
Purpose/Short- Goal/Long-term
Inputs Activities Outputs
term objective objective
Assumptions Assumptions Assumptions Assumptions
Decreasing control
Decreasing control
Increasing significance of risks
Increasing significance of risks

12. Models and M&E
• A result chain says what results you expect to achieve
• M&E tells you if you are achieving the results you expected to
achieve

13. Good M&E Systems
• Dynamic
encourage `learning by doing’
• Participative and gender sensitive
seek to overcome barriers of gender, age, power, and culture
• Reflective
encourage creating regular space and time for analysing
information and reflecting on the theories of change
• Evolving
adapting and changing in order to keep them as light and simple
as possible while providing `real time’ information
Source: AusAid, Guidance on M&E for Civil Society Programs, December 2008

14. How to Measure: Indicators
• Indicators tell you how you will recognize success
• Indicators are a unit of measurement

15. Quantitative Indicators
• Need a balance between quantitative and qualitative
indicators
• Quantitative: Can be measured in numbers
number of water pumps installed
amount of rice harvested
percentage of children immunized
HIV prevalence rate
student-teacher ratio
frequency of attending class

16. Qualitative Indicators
• Qualitative: reflect people’s
judgments, opinions, perceptions, and attitudes
perception of well-being
appropriateness of intervention
quality of engagement
level of commitment
sense of empowerment

17. Indicators to Use
1) What questions do you want to answer? (i.e. your Key
Performance Questions)?
2) What data can provide the evidence you need to answer
your KPQs? Is the data high-quality? Is it participative?
3) What data can you collect? Do the benefits justify the
costs?

18. Indicators to Use
1) What questions do you want to answer? (i.e. your Key
Performance Questions)?
• This depends on
Your audience
– Management? Donors? Beneficiaries?
The purpose of measurement
– To assess impact? Efficiency? Equity?
Conflicts can arise between M&E for accountability and M&E for
learning

19. Indicators to Use
Level in Key Performance Questions - examples
result chain
Inputs • What interventions and resources are needed?
• Are resources being used efficiently?
Activities • What are we doing? Are we doing it right?
• Have planned activities been completed on time and within the budget?
Outputs • Are we implementing the project as planned?
• What direct tangible products or services has the project/programme delivered?
Purpose/ • Are interventions working or making a difference?
objectives • What changes have occurred as a result of the outputs? Are these changes contributing towards the
project purpose and desired impact?
• Has the project achieved the changes for which it can realistically be held accountable?
Goal/ long-term • Are we intervening on a large enough scale?
objective • To what extent has the project contributed towards its longer term goals? Why or why not?
• What unanticipated positive or negative consequences did the project have? Why did they arise?
Sources: International Federation of Red Cross and Red Crescent Societies, Monitoring and Evaluation in a Nutshell,
2007; Global Fund, Monitoring and Evaluation Toolkit, 2009

20. Indicators to Use
2) What data can provide the evidence you need to answer
your KPQs? Is the data high-quality? Is it participative?

21. Indicators to Use
Level in What to monitor & evaluate/ Key Key Performance
result Performance Questions Indicators - examples
chain
Inputs • Are resources being used efficiently? • Number of staff hours spent
• Money spent
Activities • Have planned activities been completed on time and within • Percent of milestones achieved on time
the budget? • Variance between budget and actual
• What unplanned activities have been completed?
Outputs • What direct tangible products or services has the project • Number of workshops held
delivered? • Number of sex workers trained
Purpose/ • What changes have occurred as a result of the outputs? To • Percent of workshop participants
objectives what extent are these likely to contribute towards the demonstrating a high level of
project purpose and desired impact? knowledge of HIV transmission and
• Has the project achieved the changes for which it can prevention
realistically be held accountable? • Percent of workshop participants
reporting using condoms “frequently”
or “always”
Goal/ long- • To what extent has the project contributed towards its • HIV prevalence rate among Cambodian
term objective longer term goals? Why or why not? sex workers
• What unanticiplated positive or negative consequences did
the project have? Why did they arise?

22. Indicators to Use
Ensure that high-quality data will be collected. It must be
• Accurate
• Reliable
• Precise
• Complete
• Timely
• With Integrity
• Respectful of confidentiality
Source: Global Fund, Monitoring and Evaluation Toolkit, 2009

23. Indicators to Use
3) What data can you reasonably collect?

24. Indicators to Use
Result Chain
Purpose/Short- Goal/Long-term
Inputs Activities Outputs
term objective objective
Increasing difficulty of measurement
Decreasing frequency of measurement

25. Indicators to Use
Hierarchy of Effects
Social-economic-environmental
Participation Reactions Learning Actions Improvements
•Number and •Degree of •Changes in •Changes in •Changes in
frequency of satisfaction knowledge, behaviours society
people with attitudes, ski and
reached programme lls, aspiratio practices
•Intensity of •Level of ns
contact interest
•Feelings
toward
activities,
educational
methods
Increasing difficulty of measurement
Source: Bennett and Rockwell, 1995, Targeting Outcomes of Programs

26. Indicators to Use
• Do the benefits justify the costs?
• Look for a balance between rigour and realism
Source: Results-Based Management Tools at CIDA: A How-to Guide, http://www.acdi-cida.gc.ca

27. How to Collect Data
Identify for each Performance Indicator:
Definition Examples
Data source(s) People, organisations, Beneficiaries, partners, government
documents providing data about documents
the indicator
Data collection How data will be collected Analysis of documents, surveys,
methods/tools interviews, focus groups, observation,
case study, tests, testimonials, expert
panel
Formula/ scale / How the data will be captured Different scales
assessment
method
Frequency and How often data will be collected Single time, continuously,
duration and for how long weekly/monthly/annually,
baseline/mid-term/final
Responsible Who will collect the data Staff, outside evaluators
person(s)

28. Indicators to Use
• Indicators ≠ Targets
• Indicators are units of measurement
E.g. HIV prevalence rate among Cambodian sex workers
• Targets specify a particular value for an indicator to be
accomplished by a specific date
E.g. HIV prevalence rate among Cambodian sex workers is reduced
by 50% by 2020
• Targets are essential: they are a concrete expression of
your goals and objectives

29. Targets to Set
• You can only set targets once you have baseline data
• Baseline data show the situation before you start the project

30. Targets to Set
• Targets can be set as
Absolute targets (e.g. increase by 5)
Proportional or percentage targets (e.g. increase by 5%)
Relative to benchmarks (e.g. be within the top three schools in our
area)
Relative to costs or budgets (e.g. increase or reduce by 5% same
level of budget).
Sources: Advanced Performance Institute, How to Design Key Performance Indicators, 2010; Results-Based
Management Tools at CIDA: A How-to Guide, http://www.acdi-cida.gc.ca

31. Targets to Set
• Targets should be SMART
Specific, Measurable, Attainable, Relevant, Time bound
Ambitious but realistic

32. Targets to Set
• Targets should account for
Trends and past performance
Predictable variation in performance (e.g. seasonal cycles)
National targets, best practice benchmarks, etc
Result chain (i.e. do not set outcome targets before you have set
input targets)
Time lag between intervention and effect
Dependence on others (e.g. partners, government actors)

33. Targets to Set
• Indicator - # of workshops held
• Add target group (for whom?) - # of workshops for sex workers
• Add quality (what? how well?) - # of workshops for sex workers
addressing HIV prevention
• Add quantity (how much?) - # of workshops for sex workers
addressing HIV prevention increased from 10 to 12
• Add time (by when?) - # of workshops for sex workers addressing
HIV prevention increased from 10 to 12 in 2012
• Add place (where?) -# of workshops for sex workers addressing HIV
prevention increased to 6 in Phnom Penh and 6 in the provinces in
2012
Source: NORAD, The Logical Framework Approach (LFA): Handbook for objectives oriented planning (4th
ed.), 1999.

34. Putting it All Together…
Key Performance Questions Key Performance Baseline Target Means of
Indicators Verification
Input Are resources being used # staff hours/workshop 90 in 2011 50 in 2012 Staff time sheets
efficiently?
Activities Have planned activities been % variance between 2012 2012 Records kept by
completed on time and within budgeted and actual budget budget accountant
the budget? spending
Outputs What direct tangible products or # workshops held 10/year in 12/year in Participant lists
services has the 2011 2012
project/programme delivered?
# sex workers reached 300/year in 480/year
2011 in 2012
Purpose/ What changes have occurred as % of workshop participants 50% before 90% after Questionnaires
Objectives a result of the outputs and to demonstrating a high level the the before and after
what extent are these likely to of knowledge of HIV workshop workshop workshop
contribute towards the transmission and prevention
project/programme purpose and
desired impact? % of workshop participants
reporting using condoms
Has the project/programme “frequently” or “always” 5% before 20% after Surveys before
achieved the changes for which it the 1 month workshop, after 1
can realistically be held workshop 10% after month, and after 6
accountable? 6 months months
Goal/ To what extent has the HIV prevalence rate among X% n 2011 Y% 2020 UNAIDS statistics
Long-term project/programme contributed Cambodian sex workers
objective towards its longer term goals?

35. Want to Keep Going?
Put Pari’s M&E Services to work for your organization!
• We can help you set up an M&E system from scratch or assess your
current system and help you improve it
• We’ll asses your team’s capacity and make an inventory of current
monitoring, evaluation and reporting activities
• We’ll work with your team to map out your programmes/projects, select
indicators, and create tools and schedules for tracking data and
engaging in evaluation
• We’ll help you pilot the new system, gather baseline data and set
targets
• We’ll work alongside your team to maintain the system
Email us at allie@thepariproject.com to learn more