"Measuring Results and Monitoring Performance in IFAD "


Published on

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Public-private partnerships in developing value chains (input supply-farming-marketing-processing-retailing) Using farmer groups to manage water and irrigation, forests, rural finance, marketing Extensive water resource management and conservation Forest conservation and management, together with GEF Export sectors- specialty crops to generate income growth Land tenure Private provision of public services in partnership with government Targeted rural development packages for marginalized and excluded groups (indigenous people, women, elderly, farm labor…)
  • "Measuring Results and Monitoring Performance in IFAD "

    1. 1. <ul><li>IFAD’S RESULTS MEASUREMENT FRAMEWORK </li></ul><ul><ul><li>Kevin Cleaver, Associate Vice President, Programmes </li></ul></ul><ul><ul><li>Gary Howe, Director, Strategic Planning </li></ul></ul><ul><ul><li>April 2010 </li></ul></ul>
    2. 2. <ul><ul><li>Results areas and indicators that IFAD uses in managing its operations, resource mobilization, human resources, risks and efficiency </li></ul></ul><ul><ul><li>Measurement and reporting of results </li></ul></ul><ul><ul><li>Baseline results </li></ul></ul><ul><ul><li>Targets for 8 th Replenishment </li></ul></ul>Content of presentation
    3. 3. IFAD uses its loans and grants for <ul><li>Improve basic foods and staples </li></ul>Integrate livestock to match rising demand Develop private Agro-processing & mktg Include cash crops: exports are growing
    4. 4. RMF 2007-09: A snapshot of achievements compared to the 2005 Independent Evaluation of IFAD n.a. 20 n.a. Average time from project approval to first disbursement (months) n.a. 13.7 18 Average time from project approval to effectiveness (months) 90% 96% - % of country programs satisfactory for adherence to aide effectiveness agenda 90% 100% - Satisfactory overall at entry (effectiveness, poverty impact, sustainability, innovation) 70% 69% n.a. (ARRI) Satisfactory country strategy impact on food security, income empowerment 80% ARRI – 73% PCR – 75% 40% Sustainable at completion 65% ARRI – 100% PCR – 71% 25-55% Satisfactory impact in learning, scaling up and/or knowledge management at completion 70% ARRI – 91% PCR – 80% 37% Satisfactory impact on poverty at completion 80% ARRI – 82% PCR – 87% 61% Project effectiveness at completion Medium-term (2011) target established by IFAD’s Board (% of projects rated satisfactory) Latest results (RIDE, for Dec 2009 Board)(% of projects satisfactory) % of projects designed in 1995-2001 period rated satisfactory by IEE (2005) RESULTS
    5. 5. IFAD’s second Results Measurement Framework: the Results Chain Level 1: Macro outcomes Level 2: IFAD Country programme and project outcomes Level 3: IFAD Country programs and project outputs Level 4: IFAD Country programme and project design and implementation support Level 5: IFAD’s Institutional management & efficiency Outcome more relevant to IFAD Objectives Attribution to IFAD Strengthens
    6. 6. RMF in the context of IFAD: Operating Model
    7. 7. Level 1: Macro and sectoral Outcomes Baseline figures and global targets that IFAD will monitor Source: 1/. United Nations, The Millennium Development Goals Report, 2008 2/. World Bank, World Development Indicators, 2009 Tracked 4.1 2004 1.5. Agricultural value added (annual % growth) 2 / Tracked 112.4 2006 1.4. Crop production index (1999-2001 = 100) 2 / 17 27 2005 1.3. MDG 1: Children under 5, underweight % 1/ 10 17 2002 - 2004 1.2. MDG 1: Prevalence of under-nourishment in population (%) 1/ 21 26 2005 1.1. MDG 1: Population living on less than a $1.25 a day (%) 1/ 2012 target Baseline value Baseline year Indicator
    8. 8. Level 2: IFAD Country programmes & project outcomes <ul><li>Project performance measured at Project start-up and at completion by: </li></ul><ul><ul><li>Relevance – consistency of the project objectives with the priorities of poor rural people </li></ul></ul><ul><ul><li>Effectiveness – how well projects perform in delivering against their objectives; and </li></ul></ul><ul><ul><li>Efficiency – how economically resources are converted into results. </li></ul></ul><ul><ul><li>Sustainability , the net benefits sustained beyond the implementation period </li></ul></ul>[1] High-level Forum on Aid Effectiveness, final report of Roundtable 4. Accra, Ghana, 2008. [2] OE identifies them as ‘impact’, involving anticipated effects as well.
    9. 9. <ul><ul><li>Replication and scaling up , to add value to successful innovations and </li></ul></ul><ul><ul><li>Gender equality and women’s empowerment , integration of women’s as well as men’s concerns, so that women and men benefit equally. </li></ul></ul><ul><li>Poverty impact - the changes that have been perceived to have occurred in the lives of poor rural people. </li></ul><ul><ul><li>Food security, availability, access to food, and stability of access </li></ul></ul><ul><ul><li>Physical and financial assets, for income and crisis coping </li></ul></ul><ul><ul><li>Empowerment , as an end in itself, as well as a means of reducing poverty </li></ul></ul>Level 2: IFAD Country programmes & project outcomes contd …
    10. 10. Level 1: IFAD Country programmes & project outcomes: 2009 Baseline Values and 2012 Targets
    11. 11. Level 2: Data Sources and Reporting <ul><li>Data obtained from the project completion reports and project evaluations (summarised in ARRI) </li></ul><ul><li>Evaluation instruments use a six-point scale. </li></ul><ul><ul><ul><li>6= highly satisfactory, 5 = satisfactory </li></ul></ul></ul><ul><ul><ul><li>4= moderately satisfactory, 3= moderately unsatisfactory </li></ul></ul></ul><ul><ul><ul><li>2= unsatisfactory, 1 = highly unsatisfactory </li></ul></ul></ul><ul><li>Broadly, ratings of 4 and above are taken as acceptable and 3 or below as unsatisfactory </li></ul><ul><li>RMF targets are set mostly as percentage of projects rated 4 or above at start-up determined by QA findings and at completion </li></ul>
    12. 12. <ul><li>At the third level, RMF reports on outputs </li></ul><ul><li>Project outputs are reported on an absolute basis such as hectares of land brought under irrigation, number of active savers and borrowers. </li></ul><ul><li>Key outputs are aggregated and reported </li></ul><ul><ul><li>against IFAD’s six strategic areas </li></ul></ul><ul><ul><li>using the RIMS system </li></ul></ul><ul><ul><li>reporting on a sex-disaggregated basis, where applicable </li></ul></ul>Level 3: Project Outputs
    13. 13. Level 3: Project outputs against: Baseline Values and Targets Indicator Baseline year Baseline value Target 2012   470 thousand 2008 3.2 Area under constructed/rehabilitated irrigation schemes (ha)   3.86 million 2008 3.3 Common Property Resources (CPR) land under improved management practices (ha) 50:50 57:43 2007 Male : Female ratio 60 million 29 million 2007 3.1 People receiving services from IFAD-supported projects 1.07 million 35:65 2008 3.5 People trained in livestock production practices/technologies Male : Female ratio 1.72 million 50:50 2008 3.4 People trained in crop production practices/technologies Male : Female ratio
    14. 14. Level 3: Project outputs against: Baseline Values and Targets contd …   24 thousand 2008 3.13  Village/community action plans prepared (in '000)   38:62   Male : Female ratio   672 thousand 2008 3.12 People trained in community management topics (No)   19 thousand 2008 3.11  Enterprises accessing facilitated non-financial services (No)   53:47   Male : Female ratio   162 thousand 2008 3.10  People trained in business and entrepreneurship (Number)   25 thousand 2008 3.9    Marketing groups formed/strengthened (Number)   15 thousand 2008 3.8    Roads constructed/rehabilitated (‘000’ km)   51:49   Male : Female ratio   5.44 million 2008 3.7    Savers (Number)   52:48   Male : Female ratio   4.35 million 2008 3.6    Active borrowers (Number) Target Baseline Value Baseline year Indicator
    15. 15. Level 3: Data Sources and Reporting <ul><li>RIMS is IFAD’s trademark definition of key impact indicators, which it suggests be included in project M&E systems </li></ul><ul><li>RIMS operates at three levels of the results chain: outputs, outcomes and impact, based on the data from the Project M & E system </li></ul><ul><li>By providing common definition of indicators, RIMS permits aggregation of outputs across all IFAD projects, as presented in previous tables </li></ul>
    16. 16. Results and Impact Management System (RIMS) <ul><li>About 40 IFAD projects have established baselines for impact assessment using special RIMS surveys </li></ul><ul><li>RIMS was developed prior to the development of the self and independent evaluation systems: indicators are therefore not fully aligned </li></ul><ul><li>Management will review RIMS in order to bring about full coherence among results system </li></ul><ul><li>RIMS helps define indicators to be included in M&E systems; it does not substitute M&E system. </li></ul>Results and Impact Management System (RIMS)
    17. 17. <ul><li>At the fourth level IFAD measures the performance of country programmes and projects at entry and during implementation </li></ul><ul><li>The main focus is on: </li></ul><ul><ul><li>Quality at entry of country programmes for income, food security, empowerment of women and men </li></ul></ul><ul><ul><li>Adherence of aid effectiveness agenda in terms of country ownership, alignment and harmonisation </li></ul></ul><ul><ul><li>Quality at entry of projects for effectiveness, poverty impact, sustainability, & innovation, learning & scaling up </li></ul></ul><ul><ul><li>Better implementation support , measured by time elapsed for Board approval to 1 st disbursement, % of problem projects and pro-activity, time-overrun, and delay in processing withdrawal applications etc . </li></ul></ul>Level 4: Country Programme and project management
    18. 18. Level 4: IFAD Country Programme: Baseline and Targets 90 86 2008/09 QA 4.6 Innovation, learning &/or scaling up 90 81 200809 QA 4.5 Sustainability of benefits 90 91 2008/09 QA 4.4 Rural poverty impact among the target group, such as (a) physical and financial assets, (b) food security, (c) empowerment, and (d) gender equality 90 93 2008/09 QA 4.3 Effectiveness % of projects rated satisfactory or better at entry for: 90 96 2009 Client survey 4.2 Adherence to aid effectiveness agenda 90 >90 2008/09 QA 4.1 Contribution to increasing the incomes, improving the food security, and empowering poor rural women & men % of country programmes rated satisfactory or better for: 2012 target (%) Baseline value (%) Baseline year Source Indicator
    19. 19. Level 4: Project Implementation Support Indicator Baseline year Baseline value 2012 target Source 75 63 2008 Div PPR 4.9  % of problem projects in which major corrective actions are taken (Pro-activity index) 75 64 2008 ARRI 4.10 Percentage of projects for which IFAD performance rated 4 or better 65 61 2008 PPMS   Better Implementation Support 4.7  % of on-going projects actually receiving international co-financing 14 20 2008 PPMS 4.8 Average time from project approval to first disbursement (months) 15 19 2008-09 PPMS 4.11 % of problem projects in the ongoing portfolio 20  22 2008-09 PPMS 4.12 Percentage of time overrun for average project -10% over 2009 35 2008 WATS 4.13 Time taken for processing withdrawal applications (directly supervised)
    20. 20. Level 4: Data Sources and Reporting <ul><li>How undertaken: </li></ul><ul><li>The QA at entry rating is done independently by the consultants </li></ul><ul><li>The client survey, which reports on the perception of in-country stakeholders (‘clients’) comprising of representatives of governments, donor agencies and civil societies, undertaken annually, ensures anonymity of the respondents </li></ul><ul><li>Non-rated absolute numbers, such as average time elapsed, co-financing levels are generated through PPMS and WATS. </li></ul><ul><li>For day to day management monitoring in real time, Operations Dashboard has been developed. Brings data from different systems in a user-friendly interface. </li></ul>
    21. 21. Level 4: Data Sources and Reporting (cont’d)
    22. 22. <ul><li>Level 4 deals with what IFAD does operationally to achieve field level impact through its country programmes – in projects and at the macro-level of the MDGs </li></ul><ul><li>Level 5 addresses how well IFAD manages its resources to support these operations – specifically, how well: </li></ul><ul><ul><li>It focuses its workforce on the operational area </li></ul></ul><ul><ul><li>It engages the commitment of its workforce </li></ul></ul><ul><ul><li>It creates a diverse staff drawing on experience and capabilities </li></ul></ul><ul><ul><li>It achieves efficiency in expenditures </li></ul></ul><ul><ul><li>It manages risks </li></ul></ul>Level 5: Institutional management and efficiency
    23. 23. Level 5: Baseline and targets 20% 13.5% 76% 16.30% 2008 2008 ROL POW&B 5.8 % of high-priority internal audit recommendations that are overdue 5.9 Budgeted expenses per US$1 of loan and grant commitments 70% 56% 33% 30% 141 US$90 75% 65% Tracked 35% 100 Tracked 2008 2008 2008 2008 2007 2008 Global staff survey Peoplesoft HR Peoplesoft HR Peoplesoft HR HR Peoplesoft Financials 5.2 Staff engagement index (% staff positively engaged in IFAD objectives) 5.3 % of workforce in programmes (operations) 5.4 % of workforce from Lists B and C Member States 5.5 % of women in P5 posts and above 5.6 Average time to fill professional vacancies (days) 5.7 Cost per payslip (US$) 100 93 2008 Peoplesoft Financials 5.1 % achieved of replenishment pledges Indicator Baseline year Baseline value (%) 2012 target (%) Source
    24. 24. Managing to achieve results and impact <ul><li>The RMF targets are not just a basis for reporting, they are the points of orientation for the management of IFAD </li></ul><ul><li>Since 2007, IFAD has operated a comprehensive corporate planning and performance management system (CPPMS) that concentrates on the elements of performance directly managed by IFAD. </li></ul><ul><li>The CPPMS is IFAD’s system for managing to achieve the results. </li></ul><ul><ul><li>It establishes objectives for all managers </li></ul></ul><ul><ul><li>It focuses on what managers can actually manage and be directly accountable for </li></ul></ul><ul><li>Focuses on Levels 4 and 5 – on what IFAD directly manages, on the activities that lead to development and institutional efficiency results </li></ul><ul><ul><li>Project development and implementation support </li></ul></ul><ul><ul><li>The institutional platform for project-related work </li></ul></ul>
    25. 25. The results management matrix - 1 <ul><li>At the corporate level, management to achieve results revolves around targets and performance indicators for four activity clusters: </li></ul><ul><ul><li>Cluster 1. Country programme development and implementation </li></ul></ul><ul><ul><li>Cluster 2. High-level policy dialogue, resource mobilization and strategic communication </li></ul></ul><ul><ul><li>Cluster 3. Corporate management, reform and administration </li></ul></ul><ul><ul><li>Cluster 4. Support to Members’ governance activities </li></ul></ul><ul><li>Each of these groups has sub-clusters with specific targets (what we call Corporate Management Results – CMRs), e.g. Cluster 1 has: </li></ul><ul><ul><li>CMR 1 Better country programmes; CMR2 Better project design; and CMR3 Better project implementation support </li></ul></ul>
    26. 26. The results management matrix - 2 <ul><li>Each CMR includes an objective the achievement of which will contribute to better performance against the RMF targets </li></ul><ul><li>Each CMR has a basis for measuring performance - a number of Key Performance Indicators, e.g., </li></ul><ul><ul><li>CMR 1: Adherence to aid effectiveness agenda </li></ul></ul><ul><ul><li>CMR 2: % of projects rated satisfactory or better at entry for effectiveness </li></ul></ul><ul><ul><li>CMR 3: % of problem projects in the ongoing portfolio </li></ul></ul>
    27. 27. The results management process <ul><li>Each year every department prepares a plan organizing all activities around the CMRs they directly contribute to (according to function) – according to corporate level priorities </li></ul><ul><li>Every division prepares their own plan indicating their planned contribution to achievement of the corporate and departmental CMRs </li></ul><ul><li>Every individual prepares a work plan organizing their contribution to achievement of divisional plans </li></ul><ul><li>Performance is reviewed on a quarterly basis on corporate, departmental and divisional basis in performance conversations covering every department and every division, focused on : </li></ul><ul><ul><li>progress towards results </li></ul></ul><ul><ul><li>risks that need to be dealt with in achieving results </li></ul></ul><ul><ul><li>resource management issues </li></ul></ul><ul><li>The objective is identification of management actions (including financial and human resource deployment) that need to be taken in real time to keep performance on track </li></ul>
    28. 28. The performance conversation process - By 2nd week after close of qtr - Division Directors & staff - Review performance and risks - By 3rd week after close of qtr - Department Heads & Division Directors - Review performance and risks ( department risk registers ) - By 4th week after close of qtr - Operations Management Committee & Executive Management Committee - Results, resources and risks report ( corporate risk register )
    29. 29. Anchorage in day-to-day management systems - 1 <ul><li>This system is not designed to produce reports to the Executive Board. The reports on progress (RIDE) is produced from the information that IFAD needs and generates to manage itself for results on a day-to-day basis. </li></ul><ul><li>The key to the effective functioning of managing for results is accurate information on outputs and inputs. The Management Information System is being constantly upgraded: </li></ul><ul><ul><li>The CPPMS brings together all plans, KPIs and risk reports within the Peoplesoft system </li></ul></ul><ul><ul><li>Budget and financial information is drawn from the Peoplesoft financial and budget systems </li></ul></ul><ul><ul><li>Loan and grant information is drawn from the Loan and Grant System (to be replaced in 2011) </li></ul></ul>
    30. 30. Anchorage in day-to-day management systems - 2 <ul><li>Project performance and supervision data are drawn from the PPMS, supplemented by: </li></ul><ul><ul><li>Project Status Reports On-line </li></ul></ul><ul><ul><li>PMD Management Dashboard (reports) </li></ul></ul><ul><li>Human resource management data are drawn from the Peoplesoft HR database, supplemented for reporting and analysis by a new HR dashboard </li></ul><ul><li>Although all of these systems contribute to tracking performance against RMF, none are specifically for that purpose. They are all dedicated to normal processes of outcome, output and input tracking. </li></ul>
    31. 31. Bringing efficiency under scrutiny <ul><li>The results-based management system has been focused on development impact, and the monitoring system in this area is very robust – contributing to a major up-turn in IFAD’s project performance </li></ul><ul><li>There are still issues in management for development results – particularly data quality </li></ul><ul><li>The key issues for further elaboration lie in the efficiency area </li></ul><ul><ul><li>IFAD has been raising its overall efficiency – and has actually cut administrative budgets in real terms </li></ul></ul><ul><ul><li>It needs the efficiency drive to be based on detailed and accurate understanding of work load and volumes – to set more realistic targets and performance indicators </li></ul></ul><ul><ul><li>This work is under way, and we expect more comprehensive indicators to be in place by the end of the 2 nd Quarter 2010 </li></ul></ul>