PCM - Project Cycle Management, Training on Evaluation
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
9,812
On Slideshare
9,765
From Embeds
47
Number of Embeds
6

Actions

Shares
Downloads
543
Comments
6
Likes
4

Embeds 47

http://www.e-presentations.us 20
http://www.slideshare.net 19
http://e-presentations.us 5
http://localhost 1
http://pmomale-ld1 1
http://www.slidesearchengine.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. P roject C ycle M anagement Training Course on " Evaluation"
  • 2. Definitions
    • Evaluation is the systematic assessment of the worth and merit of some object.
    • Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object.
  • 3. Evaluation: A Definition
    • An evaluation is an assessment, as systematic and objective as possible, of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, develop - mental efficiency, effectiveness, impact and sustainability.
    An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision - making process of both partner countries and donors. OECD, DAC, 1991
  • 4. The Project Cycle
    • Key characteristics:
    • breaks down the life of a project / programme into phases
    • defines key decisions, information requirements, & responsibilities at each phase
    • each phase must be completed for the next to be tackled with success
    • allows for corrective measures during implementation, based on monitoring and mid-term evaluation
    • draws on evaluation experience for the design of future programmes & projects
  • 5. When to do evaluations? Mid-term evaluation End of project/ programme evaluation Ex-post evaluation During implemen-tation At the end of implemen-tation After imple-mentation Also used as term: Ex ante evaluation
  • 6. Evaluations: What for?
    • To improve
      • Decision making
      • Resource allocation
      • Accountability
    • Through:
      • Informing the public
      • Informed key decision-making processes
      • Encouraging on-going organisational learning
  • 7. Logframe Basics ‘ ... IF results are delivered, AND assumptions hold true, THEN the project purpose will be achieved ...’ Objectively Verifiable Indicators Intervention Logic Sources of Verification Assumptions Overall Objectives Project Purpose Results Activities Means Cost Pre-condi-tions
  • 8. Good indicators Do partners and stakeholders agree that this indicator makes sense to use? Owned Will the data have utility for decision-making and learning? Useful Can data be collected easily, on a timely basis at reasonable costs? Accessible Is the data consistent or comparable over time? Reliable Is the definition precise and unambiguous about what is to be measured? Objective Does the indicator directly represent the objectice it is intended to measure? Valid
  • 9. Indicators: An Example
    • Objective: Pollution load of wastewater discharged into the Blue river is reduced
    • Select the indicator: Concentration of heavy metal compounds (Pb, Cd, Hg)
    • Define the targets:
      • Define the quantity: Co ncentration of heay metal compounds (Pb, Cd, Hg) is r educed by 75 % compared to year x levels … (particular attention should be paid to the availability of baseline information)
      • Define the quality: ... to meet the limits for irrigation water . ..
      • Define the target group: ... , used by the farmers of Blue village, ...
      • Define the place : ... in the Blue river section of the District ...
      • Determine the time: ... 2 years after the project has started
  • 10. Special challenges facing the donor agencies (1)
    • Work in many different countries and contexts
    • Have a wide diversity of projects in multiple sectors
    • Often focus on capacity building and policy reform, which are harder to measure than direct service delivery activities
  • 11. Special challenges facing the donor agencies (2)
    • Are moving into new areas such as good governance, where there's little performance measurement experience
    • Often lack standard indicators on results that can easily be aggregated across projects
    • Are usually only minor actors affecting impacts, with subsequent problems in attributing them to their agency's activities
  • 12. Special challenges facing the donor agencies (3)
    • Typically rely on outcome and impact data collected by partner countries, who have limited technical capacity and resources, with subsequent quality, coverage and timeliness problems.
  • 13. Criteria for Evaluating Development Assistance
      • Relevance
      • Efficiency
      • Effectiveness
      • Impact
      • Sustainability
  • 14. The Logframe and the 3 basic evaluation criteria Overall Objectives Project Purpose + Assumptions Results + Assumptions Activities + Assumptions + Pre-conditions allocation action utilisation change Efficiency Means Impact Overall Objectives + Assumptions Results + Assumptions Activities + Assumptions allocation action utilisation change + Pre-conditions Means Overall Objectives Project Purpose + Assumptions Results + Assumptions Activities + Assumptions allocation action utilisation change Effective- ness Project Purpose + Pre-conditions Means
  • 15. Logframe and basic evaluation criteria: Major questions Overall Objectives Project Purpose + Assumptions Results + Assumptions Activities + Assumptions + Pre-conditions allocation action utilisation change Means Efficiency Effective- ness Impact How were inputs and activities converted into Results? How well did the Results contribute to the achievement of the Project Purpose? Which benefits on society and sector?
  • 16. The 4th evaluation criterion: Relevance Overall Objectives Project Purpose + Assumptions Results + Assumptions Activities + Assumptions + Pre-conditions allocation action utilisation change Means Efficiency Effective- ness Impact Relevance Quality of planning and adaptation, including relevance of problems to correct beneficiaries, OVIs, means, cost, assumptions, risks How were inputs and activities converted into Results? How well did the Results contribute to the achievement of the Project Purpose? Which benefits on society and sector?
  • 17. The 5th evaluation criterion: Sustainability Overall Objectives Project Purpose + Assumptions Results + Assumptions Activities + Assumptions + Pre-conditions allocation action utilisation change Means Efficiency Effective- ness Impact Relevance Sustain-ability Have and will services and benefits be maintained? Quality of planning and adaptation, including relevance of problems to correct beneficiaries, OVIs, means, cost, assumptions, risks How were inputs and activities converted into Results? How well did the Results contribute to the achievement of the Project Purpose? Which benefits on society and sector?
  • 18. Summary: The 5 evaluation criteria The fact that the results were obtained at reasonable cost, i.e. t he cost, speed and management efficiency with which Means/ Inputs and Activities were converted into Results , and the q uality of the Results achieved . Efficiency The effect of the project on its wider environment, and its contribution to the wider objectives summarised in the Overall Objectives . Impact The appropriateness of project o bjectives to the real problems of the intended beneficiaries that it was supposed to address, and to the physical and policy environment within which it operate d . Relevance The likelihood of a continuation in the stream of benefits produced by the project after the period of external assistance has ended. Sustainability How well the Results contribute to the achievement of the Project Purpose , and how Assumptions have affected project achievements. Effectiveness
  • 19. Relevance
    • = The extent to which the aid intervention is suited to the priorities and policies of the target group, partner country and donor
    • Possible questions:
    • To what extent are the objectives of the programme still valid?
    • Are the activities and outputs of the programme consistent with the broader objectives and do they contribute to the attainment of these objectives?
    • Are the activities and outputs of the programme consistent with the intended impacts and effects?
  • 20. Efficiency
    • = Efficiency measures the outputs – qualitative and quantitative – in relation to the inputs. It is a term which signifies that the aid uses the least costly resources in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted
    • Possible questions:
    • Were the activities cost-efficient?
    • Were objectives achieved on time?
    • What were the major factors influencing the achievement or non-achievement of the objectives?
  • 21. Effectiveness
    • = A measure of the extent to which an aid intervention attains its objectives
    • Possible questions:
    • To what extent were the objectives achieved/are likely to be achieved?
    • What were the major factors influencing the achievement or non-achievement of the objectives?
  • 22. Impact
    • = The positive and negative changes produced by an intervention, directly or indirectly, intended or unintended.
    • Possible questions:
    • What has happened as a result of the programme or project?
    • What real difference has the intervention made to the beneficiaries?
    • How many people have been affected?
    • ……
    • ……
    • ……
  • 23. Sustainability
    • = Sustainability is concerned with measuring whether the benefits of an intervention are likely to continue after donor funding has been withdrawn.
    • Possible questions:
    • To what extent did the benefits of a programme or project continue after donor funding ceased?
    • What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project?
  • 24. Principles of evaluation
      • Integrative part of planning and management : closely connected to decision making processes
      • Impartiality & independence of the evaluation process in its function from the process concerned with policy making, the delivery and management of assistance (= separation of evaluation and responsibility for the project/ programme/policy)
      • Credibility depending on expertise and independence of the evaluators & transparency to be seeked through an open process, wide availability of results, distinction between findings and recommendations
      • Usefulness : relevant, presented in a clear and concise way, reflecting the interests and needs of the parties involved, easily accessible, timely and at the right moment  improved decision-making
      • Participation of stakeholders (donor, recipient...) ; if possible: views and expertise of groups affected should form integral part of the evaluation; involving all parties  capacity building
      • DAC 1991
  • 25. Evaluation, Monitoring and Audit (1)
      • once or twice , e ssentially at the end or 'ex-post' draw ing lessons from the past in order to orient future policies and actions but also during implementation: mid-term evaluation to (re-) orient implementation
    When?
      • external evaluators specialised in the subjects evaluated
    Who?
      • in-depth analysis
    How?
      • mainly analysis of the efficiency, effectiveness, impact, relevance and sustainability of aid policies and actions
    What? Evaluation:
  • 26. Evaluation, Monitoring and Audit (2)
      • regularly , several times per year
    When?
      • internal and external (staff, monitors.....)
    Who?
      • rapid and continuous analysis, immediately useful to improv e on-going actions ; of key importance to improving performance
    How?
      • mainly analysi s of efficiency and effectiveness (i.e. measuring actual against planned deliverables ) ; i s a systematic management activity
    What? Monitoring:
  • 27. Evaluation, Monitoring and Audit (3)
    • .
      • during or after implementation
    When?
      • external, professional auditors
    Who?
      • verification of financial records (financial audit)
    How?
      • traditionally checks whether financial operations and statements are in compliance with the legal and contractual obligations . M ore concerned with compliance, but better financial management can also contribute to improving current and future actions . More recently: Performance audit is strongly concerned with questions of efficiency amd good management
    What? Audit:
  • 28. Evaluation
    • Evaluation is the systematic assessment of the value and merit of some object (in this case, a project or a programme)
    • Evaluation is the systematic acquisition and assessment of information to provide useful feedback about the evaluated object.
  • 29. Evaluation: major criteria
      • Relevance
      • Efficiency
      • Effectiveness
      • Impact
      • Sustainability
  • 30. Relevance
    • = The extent to which the aid intervention is suited to the priorities and policies of the target group, partner country and donor
    • Possible questions:
    • To what extent are the objectives of the programme valid for the beneficiaries?
    • Are the activities and results of the programme consistent with the overall objectives?
  • 31. Efficiency
    • = Efficiency measures the outputs – qualitative and quantitative – in relation to the inputs. It is a term which signifies that the aid uses the least costly resources in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted
    • Possible questions:
    • Were the activities cost-efficient?
    • Are the unit costs comparable to …
  • 32. Effectiveness
    • = A measure of the extent to which an aid intervention reaches its objectives
    • Possible questions:
    • To what extent was the project purpose achieved/is likely to be achieved?
    • What were the major factors influencing the achievement or non-achievement of the project purpose?
  • 33. Impact
    • = The positive and negative changes produced by an intervention, directly or indirectly, intended or unintended.
    • Possible questions:
    • What real difference has the activity made to the beneficiaries?
    • How many people have been affected?
  • 34. Sustainability
    • = Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn.
    • Possible questions:
    • To what extent did the benefits of a programme or project continue after donor funding ceased?
    • What were the major factors which influenced the achievement or non-achievement of sustainability of the programme or project?