Project Monitoring and Evaluation : Dr. Bijaya Bhusan Nanda, firstname.lastname@example.org
What is a project? a starting and an ending point clear objectives a plan of the work to be done a budget specific performance requirements that must be met
SYSTEM PROCESS RESULTSINPUTS- OUTPUTS- PRODUCTS/ SERVICES Outcome Impact
Project to Reduce Child Mortality Rate in A Particular Area R e Reduced mortality rates for Impact s children under 5 years old u l Improved use of ORT for managing t Outcome childhood diarrhea s 15 media campaigns completed Outputs 100 health professionals trained Increased maternal knowledge of ORT services Increased access to ORT Imple- Activities Launch media campaign to educatementation mother Train health professionals in ORT Trainers Inputs ORT supplies Funds Participants
What is Monitoring? A continuous management function. Provides programme/projects managers and key stakeholders with regular feedback and early indications of progress or lack thereof in the achievement of intended results. Tracks the actual performance against planned or expected. Involves collecting and analysing data on programme/project processes and results. And recommends corrective measures. Generally internal.
Regular feed backMonitoringCollecting & Analysing data
Purpose & Goals for the Monitoring Tool~Collect information regarding project quality~Support the stake holders in projectimplementation.~Lead the beneficiaries to possible solutions ofthe problems detected in the implementation.~ Be familiar with the real resources.~ Assess the progress of the projects.~ Disseminate examples of good practices.~ Establish a close relation between the Stakeholders and the organisation
What to Monitor Physical activity – Quality, timeliness Financial – Cost of component, target Performance – Overall performance of system
What to Monitor Inputs: Money, Material, human resources Activities: A set of tasks performed Process: A set of activities, towards a common purpose Output Outcomes: Series of effects of some action and activities (e.g. Crop diversity, reduced alkalinity, Increased Water table) Impact: Changes in peoples lives and livelihoods (e.g. Increased income, reduced mortality, Increased capacity to bargain)
Principles of developing Monitoring Demand driven System Simple and easy Participatory Not for policing or punishment rather for learning Mix of qualitative and quantitative data Aggregation of data 2 way feedback Meaningful use of analysed information at source
MONITORING DICHOTOMY Emphasis on Emphasis on physical the way the achievements target has vis-à-vis been targets achieved Progress monitoring or Input-output monitoring or Target-achievement Process monitoring monitoringA complete monitoring system will haveboth progress and process monitoring 12
Mechanism Reports: Regular , specific Visit/observations/ transects, both at field and office Meetings/reviews – Regular, specific Studies – Impact studies, other special studies Special tools – for measuring specific processes or outputs.Levels of monitoring are to be decided based on the project management level
Monitoring ToolManagement Information System (MIS) A system for Collection, Organisation, Maintenance, Analysis, Interpretation of data For assessment, Better decision, Corrective measures, for better performance at various level
Criteria of a good (MIS) Regularity, Continuity, Timeliness, Relevant Indicators, Forward & Backward feedback, Automated Hence a computerized MIS is a must Records & registers MPRs, QPRs, APRs
TRIPLE A Assessment: Identify the “what” - Analysis: Identify the “why” - Talking and understanding the causes Action: Identify the “how”- praise for good practice, agree on actions to improve the situation
Broken Triple AA supervisor during her field visit finds that in an Anganwadi Centre the number of children in grade II, III and IV has increased. She advises the AWW to take more care of these children. However in her next visit she finds that the situation has not improved.What is missing?
Broken Triple A A supervisor during her field visit finds that in an Anganwadi Centre the number of children in grade II, III and IV has increased. She discusses the causes with the AWW and is informed that a number of children are suffering from diarrhoea. She checks whether the AWW has stock of ORS and advises her to give ORS to the affected families. What is missing?
What is Evaluation?A time-bound exercise.Assess systematically and objectively therelevance, performance and success, or the lackthereof, of ongoing and completed programmes.Evaluation is undertaken selectively to answerspecific questions to guide decision-makers and/orprogramme managers, and to provide informationon whether underlying theories and assumptionsused in programme development were valid, whatworked and what did not work and why. Evaluationcommonly aims to determine the relevance, validityof design, efficiency, effectiveness, impact andsustainability of a programme.
Relevance Programme continues to meet needs Results vs. Efficiency Anticipated/ Unanticipat Evaluation- ed results costs concerned with Design Validity ofSource: ILO, 1997.
Why evaluate?•To inform decisions on operations, policy, or strategyrelated to ongoing or future programme interventions;•To demonstrate accountability to decision-makers•To enable learning and contribute to the body ofknowledge on what works and what does not work andwhy;•To verify/improve programme quality and management;•To identify successful strategies for extension/expansion/ replication;•To modify unsuccessful strategies;•To measure effects/benefits of programme and projectinterventions;•To give stakeholders the opportunity to have a say inprogramme output and quality;•To justify/validate programmes to donors, partners andother constituencies.
Three Common Evaluation Purposes To improve the design and performance of an ongoing programme – A formative evaluation. To make an overall judgment about the effectiveness of a completed programme, often to ensure accountability – A summative evaluation. To generate knowledge about good practices.
Monitoring vrs. Evaluation Monitoring Evaluation Continuous Periodic: Mid-term, End Keep track; oversight, term, Appraisal analyses and documents In-depth analysis compare progress planned with actual Focuses on inputs, achievement outputs, process, Focuses on out-puts in continued relevance, relation to inputs, results in likely results at purpose l relation to cost, processes, Translate Objectives to overall relevance, impact performance indicators and sustainability Collect data on Indicators Answers why and how routinely results were achieved. Report progress to stake Contributes to building holders & Alert them to theories and model problems and provides Provide managers with options for corrective strategy and policy options actions Internal and or External Internal Self assessment
Evaluation Evaluative Activities: Activities such as situational analysis, baseline surveys, applied research and diagnostic studies. Evaluation Questions: A set of questions developed by the evaluator, sponsor, and/or other stakeholders, which define the issues the evaluation will investigate and are stated in such terms that they can be answered in a way useful to stakeholders.
EvaluationEvaluation Standards: A set of criteria againstwhich the completeness and quality ofevaluation work can be assessed. Thestandards measure the utility, feasibility,propriety and accuracy of the evaluation.Evaluation standards must be established inconsultation with stakeholders prior to theevaluation.Ex-ante Evaluation: An evaluation that isperformed before implementation of adevelopment intervention. Related term:appraisal.
EvaluationEx-post Evaluation: A type of summativeevaluation of an intervention usually conductedafter it has been completed. Its purpose is tounderstand the factors of success or failure, toassess the outcome, impact and sustainabilityof results, and to draw conclusions that mayinform similar interventions in the future.External Evaluation: An evaluation conductedby individuals or entities free of control by thoseresponsible for the design and implementationof the development intervention to be evaluated(synonym: independent evaluation).
Participatory Monitoring & Evaluation Strengthening self development initiatives Public accountability of programmes to communities Encouraging institutional reform towards participatory structures Organization building and learning Capturing social dynamics
M&E StakeholdersFunding Implementing Beneficiaries NGOsagency agency Need & Interest Of all should be taken care of 29
Participatory monitoring and evaluationis a different approach which involves – Local People, Development Agencies, and Policy Makers deciding together how progress should be measured, and resultsacted upon. It can reveal valuable lessons and improve accountability. However, it is a challenging process for all concerned since itencourages people to examine their assumptions about whatconstitutes progress, and to face up to the contradictions andconflicts that can emerge. 30
At the heart of P M & E, however, are four broadprinciples:Participation –‘Negotiation to reach agreement about what will bemonitored or evaluated, how and when data will becollected and analysed, what the data actually means, andhow findings will be shared, and action taken.This leads to learning which becomes the basis forsubsequent improvement and corrective action.Since the number, role, and skills of stakeholders, theexternal environment, and other factors change over time,flexibility is essential. 31
Identify who Clarify participants Clarify if the PM&E should and want Expectations of the Process needs to be Process,and in what to be involved Way each person Sustained and Or group wants to If so how contribute Agree on howThe findings are Define the prioritiesTo be used and Steps involved in For M&E By whom PM&E (Clockwise) Identify indicators Analyse the That will provide information The information needed Agree on the methods, Collect the information Responsibilities and Timings of information collections 32
Key Characteristics of Participatory M&E- Draws on local resources and capacities Recognizes the innate wisdom and knowledgeof the end-usersDemonstrates that end-users are creative andknowledge about their environmentEnsures that stakeholders are part of thedecission-making processUses facilitators who act as catalysts and whoassist stakeholders in asking key questions 33
Benefits of participatory M&E All Stakeholders owns the M&E Process & Results. Correction, redesigning of the policy, plan of action, budgeting become easy.Better decision making by insidersInsiders develop evaluation skillsOutsiders have better understanding of insidersInsider to insider communication is strengthenedInformation is useful for ongoing management ofprojectEntry point for the participatory approach 34
STEPS FOR EVALUATIONFIRST Step: Review objectives and activitiesSecond step :Review reasons for evaluationThird step: Develop evaluation questionsFourth Step: Decide who will do the evaluation.Fifth step: Identify direct and indirectindicators.Sixth step :Identify the information sources forevaluation questions 35
Seventh Step: Determine the skillsand labour that are required to obtaininformationEighth step: Determine wheninformation gathering and analysiscan be done.Ninth step : Determine who willgather information.Tenth step: Analyze, present anduse results. 36