Results-Based Management in UNDP

27,430 views

Published on

Patrick Grémillet, UNDP Bratislava Regional Center, Management Practice 2011

4 Comments
31 Likes
Statistics
Notes
No Downloads
Views
Total views
27,430
On SlideShare
0
From Embeds
0
Number of Embeds
2,602
Actions
Shares
0
Downloads
1,718
Comments
4
Likes
31
Embeds 0
No embeds

No notes for slide
  • Which issues, or which causes should we choose to work on?
  • Area 1 is sometimes referred to as the “Nike” or “Just do it” area.In area 3, care should be taken not to readily go into that area before doing a thorough review of whether another agency (UN, CSO, etc) can provide the support required. UNDP should operate in this zone only as a last resort, as it is outside of our mandate and comparative advantage.Question: The EU is sending a electoral observation mission to x country, to observe their national elections. They wish to negotiate with UNDP an arrangement for UNDP to contract their field personnel, and handle all logistics arrangements. The EU will do the recruitment and will supervise the staff. Which area (1,2,3 or 4) would this project fit in?
  • Discussion of examples of poor problem definition. Relationship to country analytical processes
  • Crucial to ensure that problems are not defined to fit the interest or capacity of an agency. Sometimes problems can be defined with the solutions in mind – e.g. first example above where someone may think that giving minorities the right to vote will solve the problem. If the problem was formulated in one of the other 2 ways, then the right to vote could be seen as one constraint. There could be others, including social norms, lack of access roads, etc.
  • In looking at causes, dig deeper into the issues to see what is the source of the problem.Are there policy impediments? Are institutions in place? Are they set up or structured properly? Do they collaborate effectively? Do they have the capacity? Are there social or cultural norms and practices that are creating impediments?
  • Before we decide on what to fund, we should have a clear understanding of our strengths, weaknesses and comparative advantagesBased on that, we define a clear vision of what needs to happen and what must in place for the goal to be achieved.We must be aware of what others will need to do to complement our efforts to achieve the goal.In most cases, a single project by one agency will not be adequate to achieve the goal: partnerships and collaboration will be critical.
  • Simplified results matrix: conflate the two outcome levels into one, with emphasis on strategic but specific enough to country context. Number of priorities & outcomes are optional.
  • Refer to pages 56 and 57 of HandbookPublic sector investment programme makes provision for substantial increase in the real allocation to gender equality and women’s empowerment. National budget allocation procedures and processes require gender analysis of all investment decisions.
  • Results-Based Management in UNDP

    1. 1. Results-Based Management<br />Patrick Grémillet<br />Bratislava Regional Center <br />Management Practice<br />2011<br />
    2. 2. Session Schedule<br />Results-based Management (RBM) Principles<br />Introduction to RBM<br />Setting a Results Management Strategy<br />Problem Analysis and Results Mapping<br />UNDP planning instruments<br />Results Matrix <br />Understanding the RBM Typology<br />Differentiating outcomes and outputs<br />Indicators, baseline and targets<br />August 2011<br />Management Practice - BRC<br />2<br />
    3. 3. What is a Result ?<br />August 2011<br />Management Practice - BRC<br />3<br />Results are changes in a state or condition that derive from a cause-and-effect relationship. There are three types of such changes (intended or unintended, positive and/or negative) that can be set in motion by a development intervention – outputs, outcomes and impacts.<br />- UNDG agreed RBM terminology<br />
    4. 4. What are Result?*<br />RESULTS<br />are changes in a state or condition that derive from a cause-and-effect relationship.<br />CONTROL & RESPOSIBILITY<br />MAKING A DIFFERENCE<br />* using UN harmonized terminology (based on OECD/DAC Glossary of Key Terms in Evaluation and Results-based Management)<br />
    5. 5. August 2011<br />Management Practice - BRC<br />5<br />What is results based management?<br />RBM is a management strategy by which all actors ensure that their processes, products and services contribute to the achievement of desired results (outputs, outcomes and higher level goals or impact) <br />A key component of RBM is performancemonitoring which is to<br />objectively measure how well results<br />are being achieved, and<br />report on measures taken to improve them.<br />
    6. 6. Why RBM?<br />Stated rationale/intended gains: <br />Improved focus on results instead of activities<br />Improved transparency<br />Improved accountability<br />Improved measurement of programme achievements (performance rather than utilization)<br />Enhanced strategic focus<br />Industry standard<br />To get more funds!! <br />August 2011<br />Management Practice - BRC<br />6<br />
    7. 7. Results-Based Management in practice <br />In your table groups, discuss and agree… <br /> What are the 3 main problems you have seen in trying to monitor and assess results as part of your work?<br /><ul><li>one concern/problem per card/post-it</li></ul>August 2011<br />Management Practice - BRC<br />7<br />
    8. 8. General problems when applying RBM?<br />Difficult to apply <br />Difficult to learn<br />Difficult to integrate<br />Difficult to revise (... or reluctance to revise? )<br />Difficult to measure<br />Difficult to ‘attribute’ <br />(at outcome level, the UN is accountable but not fully responsible)<br />August 2011<br />Management Practice - BRC<br />8<br />
    9. 9. Common elements of RBM<br />Problem analysis to understand causes<br />Stakeholder analysis<br />Structuring of programmes around a chain of desired results - addressing causes<br />Causality in the chain of results (if… then logic)<br />Use of ‘change language’ (future conditional)<br />Reliance on indicators to measure performance<br />Costing of results rather than isolated activity budgeting<br />August 2011<br />Management Practice - BRC<br />9<br />
    10. 10. The RBM life-cycle approach<br />August 2011<br />Management Practice - BRC<br />10<br />
    11. 11. Session Schedule<br />August 2011<br />Management Practice - BRC<br />11<br />Results-based Management (RBM) Principles<br /><ul><li>Introduction to RBM
    12. 12. Setting a Results Management Strategy
    13. 13. Problem Analysis and Results Mapping
    14. 14. Results Matrix
    15. 15. Understanding the RBM Typology
    16. 16. Differentiating outcomes and outputs
    17. 17. Indicators, baseline and targets</li></li></ul><li>Making Strategic Choices<br />August 2011<br />Management Practice - BRC<br />12<br />
    18. 18. Strategic Priority Setting<br />August 2011<br />Management Practice - BRC<br />13<br />Capacity<br />Value<br />1. “Just do it”/strategic priority<br />Major national challenge<br />UNDP comparative advantage<br />2. Potential high priority, if consensus can be built<br />3. Potential high priority, if others cannot meet demand and internal capacity development is feasible <br /> 4. Lower priority: does not meet major national challenge<br />Alignment of key actors to support action<br />Support<br />
    19. 19. Session Schedule<br />August 2011<br />Management Practice - BRC<br />14<br />Results-based Management (RBM) Principles<br /><ul><li>Introduction to RBM
    20. 20. Setting a Results Management Strategy
    21. 21. Problem Analysis and Results Mapping
    22. 22. Results Matrix
    23. 23. Understanding the RBM Typology
    24. 24. Differentiating outcomes and outputs
    25. 25. Indicators, baseline and targets</li></li></ul><li>What problem?<br />August 2011<br />Management Practice - BRC<br />15<br />33-35<br />
    26. 26. Formulate problem in a neutral manner<br />“Minorities and marginalized groups do not have the right to vote.” Versus<br />“Minorities and other marginalized groups do not participate in elections.” Or <br />“Low levels of participation by minorities in elections.”<br />August 2011<br />Management Practice - BRC<br />16<br />33-35<br />
    27. 27. Cause-effect analysis<br />August 2011<br />Management Practice - BRC<br />17<br />Effects<br />Causes<br />Are there capacity constraints? – policy level, institutional, individual<br />Are there Social or Cultural Constraints?<br />
    28. 28. Causality analysis – Why?<br />Negative outcomes, manifestations of problems, unfulfilled rights<br />Immediate causes<br />Affecting individuals and households<br />Underlying causesand capacity issues<br />Policies, laws, budgetsSystems for service deliveryBehaviors and practices, low household incomes<br />Root causes<br />Beliefs, attitudes, culture, traditionsNatural resources, natural disastersPolitical and economic systems, ideologies, conflict<br />August 2011<br />Management Practice - BRC<br />18<br />
    29. 29. Problem Tree<br />August 2011<br />Management Practice - BRC<br />19<br />Manifestations<br />Malnutrition &<br />death<br />Immediate<br />causes<br />effect<br />Inadequate dietary<br />intake<br />Disease<br />Underlying<br />causes<br />Insufficient <br />food security<br />Inadequate Maternal <br />& Child Care services<br />Insufficient health services<br />& unhealthy environment<br />Resource Control<br />+<br />Organizational structures<br />Root<br />causes<br />Political, Ideological,<br />Economic structures<br />cause<br />
    30. 30. Results Map<br />August 2011<br />Management Practice - BRC<br />20<br />Poor rural women earn meaningful wages and have improved life skills<br />Women gain marketable skills in traditional and non-traditional areas<br />Women learn good work attitudes and workplace behavior<br />Women develop self-confidence<br />Potential employers educated and committed to programme<br />Women trained in vocational skills<br />Some women receive traditional schooling<br />Advocacy and sensitization initiative<br />Women enroll<br />Long-term Outcome<br />Intermediate Outcomes<br />Partnerships<br />Outputs<br />Small stipend provided to women<br />Child care services provided<br />Programme info provided to women<br />Training agency, child care agency, NGO, and National Women’s Bureau etc collaborate to design and deliver programmes<br />
    31. 31. Session Schedule<br />Results-based Management (RBM)<br />Introduction to RBM<br />Key Principles<br />RBM Life cycle<br />UNDP planning instruments<br />Results Matrix <br />Understanding the RBM Typology<br />Differentiating outcomes and outputs<br />Indicators, baseline and targets<br />August 2011<br />Management Practice - BRC<br />21<br />
    32. 32. Planning and Monitoring instruments<br />August 2011<br />Management Practice - BRC<br />22<br />Simplification<br /><ul><li>UNDAF and CPD outcomes conflated into one
    33. 33. Results matrix streamlined</li></ul>PLANNING<br />MONITORING<br />National plans/strategies; PRSP; …<br />Country Analysis<br />UN Development Assistance Framework (UNDAF)<br />Country Programme Document (CPD)<br />Country Program Action Plan (CPAP)<br />Annual Work Plan (AWP)<br />Jointprogrammes<br />Projectdocument<br />
    34. 34. UNDAF BEFORE….<br />Good Governance and the protection of Human Rights<br />By 2013, achieve effective participation of citizens, and government accountability and integrity<br />Laws on public demonstrations and freedom of association improved in accordance with international standards.<br />Increased participation of civil society and citizens in decision making<br />Strengthened community participation in the planning and implementation of local development activities<br />Capacity of civil society strengthened through civic education and engagement with Parliament<br />Advocacy programmes conducted for accession to the UN convention against corruption.<br />Corruption in government management significantly reduced <br />Advocacy programmes conducted for the adoption of a national anti-corruption law <br />Effectiveness of decentralized government structures improved to deliver basic services<br />Government assisted to meet its obligation under the International Covenant on Economic, Social and Cultural Rights.<br />Capacities of communes for decentralized planning, management and delivery of public goods and services further strengthened.<br />August 2011<br />Management Practice - BRC<br />23<br />
    35. 35. UNDAF NOW…Single outcome layer, Outputs definition optional, integrated M&E data, no more separate M&E framework<br />August 2011<br />Management Practice - BRC<br />24<br />Strategic focus<br /><ul><li>Single Outcome to be strategic but specific enough to country context and UNDP programming needs
    36. 36. Approach supports cross-practice work</li></li></ul><li>New UNDP Country Programme Document(CPD) Results MatrixEffective October 2010<br />UNDP specific Indicators and Targets contributing to UNDAF Outcome<br /><ul><li>Only one Outcome layer.
    37. 37. Verbatim from UNDAF</li></ul>August 2011<br />Management Practice - BRC<br />25<br />
    38. 38. Session Schedule<br />Results-based Management (RBM)<br />Introduction to RBM<br />Key Principles<br />RBM Life cycle<br />UNDP planning instruments<br />Results Matrix <br />Understanding the RBM Typology<br />Differentiating outcomes and outputs<br />Indicators, baseline and targets<br />August 2011<br />Management Practice - BRC<br />26<br />
    39. 39. Ways how to depict the Results chain<br />Narrative<br />IMPACTA. XXXXXXXX<br />OUTCOMEA.1. XXXXXXXXA.2. XXXXXXX XXX<br />OUTPUTSA.1.1.XXX XXX XXXXA.1.2. XXXXXXXA.2.1. XXXXXXXX<br />Logframe<br />Results Tree<br />Results Pyramid<br />
    40. 40. Principles of RBM<br /><ul><li>“If-Then” causality between levels of results
    41. 41. Common results language to describe changes
    42. 42. Collective accountability increases as you move up the chain of results towards outcomes and impacts (Key message  No agency can do it alone!)
    43. 43. A results matrix is a mean not an end
    44. 44. A results matrix is contextual </li></ul>August 2011<br />Management Practice - BRC<br />28<br />
    45. 45. August 2011<br />Management Practice - BRC<br />29<br />Our counterparts may use different language <br />(i.e. logframe), but the basis is the same… <br />
    46. 46. CPAP Template<br />August 2011<br />Management Practice - BRC<br />30<br />UNDAF/CP<br />D outcome #1:<br />Extracted verbatim from the <br />UNDAF/CPD<br />Relevant UNDP Strategic Plan result: <br />Extracted verbatim from the <br />UNDAF<br />/<br />CPD<br />Indicative Resources by outcome (per year, US$)<br />Outcome<br />-<br />le<br />vel indicators, baselines and <br />Implementation modality(ies) and <br />targets for UNDP contribution to <br />Country programme outputs<br />implementing partner(s)<br />UNDAF/CPD outcomes<br />Year<br />Year<br />Year<br />Year<br />Year<br />Total<br />Output 1:<br />State the <br />implementing modality<br />Regular Resources<br />(i.e. NIM, UN Agency, NGO, IGO or <br />Annual Targets<br />(recommended for multi<br />-<br />DIM);<br />And<br />year outputs)<br />Government and/or UN <br />·<br />Gender Marker Rating <br />and <br />[2]<br />Implementing Partner<br />[1]<br />Motivation<br />Output 2:<br />State the <br />implementing modality<br />Other Resources<br />(i.e. NIM, UN Agency, NGO, IGO or <br />Annual Targets <br />(recommended for multi<br />-<br />DIM);<br />And<br />year outputs)<br />Government and/or UN <br />·<br />Gender Marker Rating and <br />Implementing Partner<br />Motivation<br />UNDAF/CPD outcome #1:<br />Extracted verbatim from the UNDAF/CPD<br />Relevant UNDP Strategic Plan result: <br />Extracted verbatim from the UNDAF<br />/<br />CPD<br />Regular Resources<br />Other Resources<br />[1]<br />Gender Rating:<br />3<br />-<br />Gender equality is a principal objective of the output; <br />2<br />-<br />Gender equality is a significant objective of the output; <br />1<br />-<br />Outputs that <br />will contribute in some way to gender <br />equality but not significantly; <br />0<br />-<br />Outputs that are not expected to contribute noticeably to gender equality.<br />Include a one sentence motivation as to the reason for the chosen rating.<br />[2]<br />State also IGO Implementing <br />Partners who have already signed SBEAA with UNDP. Other IGOs and NGOs will not be mentioned here in CPAP. They would be selec<br />ted through a <br />competitive process later.<br />
    47. 47.  <br />August 2011<br />Management Practice - BRC<br />31<br />UNDP Projects<br />Results & Resources Framework<br /> <br />
    48. 48. A few caveats<br />The results matrix is a storyline<br />It is a description of a programme or project strategy <br />Shows the intended paths we think are required to achieve desired results<br />The storyline comprises results, indicators & targets and assumptions & risks<br />August 2011<br />Management Practice - BRC<br />32<br />
    49. 49. Results chain problems<br />Results not logically linked<br />Results not sufficiently specific<br />Results are composites of several results <br />Results don’t express change (e.g. support provided to strengthen….)<br />Results statements are too wordy<br />Confusion between levels of results <br />Indicators <br /><ul><li>Not logically linked to the result
    50. 50. Not measurable
    51. 51. Are new results</li></ul>August 2011<br />Management Practice - BRC<br />33<br />
    52. 52. Results Chain<br />August 2011<br />Management Practice - BRC<br />34<br />Inputs<br />Outputs<br />outcomes<br />Impact<br />><br />Experts; equipment; funds<br />People capacities improved; Laws/policies drafted<br />Policies adopted; Laws enacted;<br />Conditions improved health/ longevity<br />Partnerships <br />and <br />other interventions<br />
    53. 53. Results Chain<br />August 2011<br />Management Practice - BRC<br />35<br />
    54. 54. Results Chain<br />August 2011<br />Management Practice - BRC<br />36<br />Positive and negative, primary and secondary<br />long-term effects produced by a<br />development intervention, directly or indirectly,<br />intended or unintended.<br />Impact:<br />Human Change<br />Outcome: <br />The likely or achieved short-term and<br />medium-term effects of an intervention’s<br />outputs. <br />Institutional & Behavioural Change<br />Outputs:<br />Tangible products and services which result from the completion of activities within a development intervention. <br />Products & Services<br />Skills & Abilities<br />
    55. 55. Results Chain<br />August 2011<br />Management Practice - BRC<br />37<br />Positive and negative, primary and secondary<br />long-term effects produced by a<br />development intervention, directly or indirectly,<br />intended or unintended.<br />Changes in the lives of people<br />Impact:<br />Human Change<br />Institutional Change: values, ethic, rules, laws – associated with/to institutional performance, access..<br />Behavioural change: knowledge, skills acquisition, practices (individual level)<br />Outcome: <br />The likely or achieved short-term and<br />medium-term effects of an intervention’s<br />outputs. <br />Institutional & Behavioural Change<br />Outputs:<br />Tangible products and services which result from the completion of activities within a development intervention. <br />Products & Services<br />Skills & Abilities<br />
    56. 56. Results Chain<br />August 2011<br />Management Practice - BRC<br />38<br />Positive and negative, primary and secondary<br />long-term effects produced by a<br />development intervention, directly or indirectly,<br />intended or unintended.<br />Changes in the lives of people<br />Reduced infant and maternal mortality by 2018<br />Impact:<br />Human Change<br />Institutional Change: values, ethic, rules, laws – associated with/to institutional performance, access..<br />Behavioural change: knowledge, skills acquisition, practices (individual level)<br />Outcome: <br />Improved provision of public sanitary services to rural communities by 2015<br />The likely or achieved short-term and<br />medium-term effects of an intervention’s<br />outputs. <br />Institutional & Behavioural Change<br />Outputs:<br />Tangible products and services which result from the completion of activities within a development intervention. <br />National Public Works Agency has the management systems, equipment, and skills to provide sanitation services to rural communities<br />Products & Services<br />Skills & Abilities<br />
    57. 57. August 2011<br />Management Practice - BRC<br />39<br />Results<br />Like…<br /> Focus <br />@<br />Timeframe <br />more<br /> 5-10 yrs <br />Impact<br />Impact<br />Impact<br />then<br />Collective Accountability<br />if<br /> 5 yrs <br />UN<br />Outcome<br />UN<br />Outcome<br />UN<br />Outcome<br />then<br /> <3 yrs <br />Output<br />Output<br />Output<br />if<br />then<br />less<br /> <1 yr <br />Activity<br />Activity<br />Activity<br />if<br />A Typology for RBM<br />HIV incidence reduced<br /> Human! <br />Assumptions<br />Leadership empowered<br />Institutional/ Behavioural<br />Skills of Nat’l Aids Comm. strengthened<br />Operational/ skills, abilities, products & services<br />Train 250 district AIDS officers<br />
    58. 58. August 2011<br />Management Practice - BRC<br />40<br />more<br />Collective Accountability<br />less<br />A Typology for RBM: Governance<br />Results<br />Like…<br />Focus<br />@<br />Timeframe<br />then<br />Impact<br />More transparent governance<br />5-10 yrs <br /> Human! <br />if<br />then<br />Outcome<br />National capacity to implement governance reforms increased<br />5 yrs <br />Institutional/ Behavioural<br />if<br />then<br />Output<br />Legislative code for local self-governance developed<br /> <5 yrs <br />Operational/ skills, abilities, products & services<br />if<br />Activity<br />- Training legislators <br />- National consultation<br />- Assessment of laws..<br /> <1 yr <br />
    59. 59. August 2011<br />Management Practice - BRC<br />41<br />more<br />Collective Accountability<br />less<br />A Typology for RBM: Poverty Reduction <br />Results<br />Like…<br />Focus<br />@<br />Timeframe<br />then<br />Impact<br />Poverty reduced<br />5-10 yrs <br /> Human! <br />if<br />then<br />Outcome<br />Employment and income generation increased<br />Institutional/ Behavioural<br />5 yrs<br />if<br />Regulatory environ. proposed to drive small enterprise development <br />then<br />Output<br /> <5 yrs <br />Operational/ skills, abilities, products & services<br />if<br /><ul><li> Economic assessment
    60. 60. Training for chambers of commerce </li></ul>- National consultation<br />Activity<br /> <1 yr <br />
    61. 61. August 2011<br />Management Practice - BRC<br />42<br />more<br />Collective Accountability<br />less<br />A Typology for RBM: Environment<br />Results<br />Like…<br />Focus<br />@<br />Timeframe<br />then<br />Impact<br />Loss of environmental resources reversed<br />5-10 yrs <br /> Human! <br />if<br />then<br />Outcome<br />Protected areas are designated<br />Institutional/ Behavioural<br />5 yrs<br />if<br />CBOs better abled to engage and mobilise communties<br />then<br />Output<br /> <5 yrs <br />Operational/ skills, abilities, products & services<br />if<br />Activity<br /><ul><li> CBO trained
    62. 62. Community orientation conducted
    63. 63. Small grants provided</li></ul> <1 yr <br />
    64. 64. Outcomes<br />Let’s look at some examples… <br />
    65. 65. How to articulate Outcomes<br />August 2011<br />Management Practice - BRC<br />44<br /><ul><li>Outcomes:</li></ul>Outcomes are actual or intended changes in development conditions that interventions are seeking to support. <br />Some guides: <br />Avoid action verbs – ”Strengthening”, “enhancing”, etc<br /> Avoid intentions – “To assist the government…”, <br />Use completed verbs: “…reduced”, “improved”, “have greater access to”, etc<br />Must signal that something has changed <br />The something which has changed must be important to the country/region/community, not just UNDP.<br />Avoid UN speak: gender mainstreamed<br />
    66. 66. Typical pitfalls<br />Wordy (..and no change language)<br /> To promote equitable economic development and democratic governance in accordance with international norms by strengthening national capacities at all levels and empowering citizens and increasing their participation in decision-making processes<br />Too ambitious<br /> Strengthened rule of law, equal access to justice and the promotion of rights<br />Containing multiple results<br /> The state improves its delivery of services and its protection of rights—with the involvement of civil society and in compliance with its international commitments<br />August 2011<br />Management Practice - BRC<br />45<br />
    67. 67. Typical pitfalls<br />Wishy-washy (ie. Support provided to improve..)<br />Support to institutional capacity building for improved governance<br />So general, they could mean anything<br /> To promote sustainable development and increase capacity at municipal level<br />Overlapping with National goals/ MDGs (impacts) <br /> Substantially reduce the level of poverty and income inequality in accordance with the MDGs and PRSP<br />Confusing means and ends<br /> Strengthen the protection of natural resources through the creation of an enabling environment that promotes sound resources management<br />August 2011<br />Management Practice - BRC<br />46<br />
    68. 68. Examples<br />Legal and regulatory framework reformed to provide people with better access to information and communication technologies. <br />The poor in x region have better access to capital and other financial services.<br />Reduction in the level of domestic violence against women by 2016<br />Increased regional and sub-regional trade <br />Higher and more sustainable employment and income for urban slum dwellers.<br />August 2011<br />Management Practice - BRC<br />47<br />
    69. 69. Examples<br />By the end of 2010, user-friendly and sustainable health care and nutrition services are provided in compliance with international standards at national and sub-national levels.<br />Increased access to and completion of basic education, especially for girls.<br />By 2012, increased and more equitable access to and utilization of quality, integrated and sustainable basic services by the poor and vulnerable.<br />August 2011<br />Management Practice - BRC<br />48<br />
    70. 70. Outputs<br />Tangible, deliverable, promises<br />
    71. 71. Outputs: Definition & key features<br />Outputs are deliverables/end-project results<br /><ul><li>Operational changes: new skills or abilities, the availability of new products and services
    72. 72. Must be achieved within the project period
    73. 73. Managers have a high degree of control</li></ul> If the result is mostly beyond the control or influence of the programme or project, it cannot be an output<br /><ul><li>Failure to deliver is failure of the project
    74. 74. 3 to 6 outputs per agency outcome
    75. 75. Unless under a joint programme, outputs are NOT collective results</li></ul>August 2011<br />Management Practice - BRC<br />50<br />
    76. 76. Measuring Outputs<br /><ul><li>Easier to measure than outcomes (tangible!)
    77. 77. Indicators usually coming from existing data, assessments, analysis or from routine progress reports
    78. 78. Indicators can be ‘yes-no’
    79. 79. Or qualitative
    80. 80. 1 to 4 indicator enough - the fewer the better to reduce costs</li></ul>August 2011<br />Management Practice - BRC<br />51<br />
    81. 81. Short exercise (10 mins)<br />Please reword/optimize the following draft output statement:<br />“To strengthen the capacity of civil servants to do X by undertaking Y”<br />August 2011<br />Management Practice - BRC<br />52<br />
    82. 82. Refining results…<br />August 2011<br />Management Practice - BRC<br />53<br />To strengthen the capacity of civil servants to do X by undertaking Y,…<br />Let’s use results language to emphasis the future condition we want to achieve.<br />The capacity of civil servants is strengthened to do X by undertaking Y,…<br />All civil servants, everywhere? Can you be more specific? Are there particularly weak or under-resourced civil servants we should emphasise?<br />The capacity of civil servants in the<br />4 poorest districts is strengthened to <br />do X by undertaking Y,…<br />We can take out information that relates to either strategy or activities. <br />The capacity of civil servants in the 4 poorest districts is strengthened to do X <br />by undertaking Y,…<br />Now, let’s try bringing the subject of change to the front, and shifting from passive to active language. <br />Civil servants in the 4 poorest districts are better able to X<br />
    83. 83. Typical pitfalls<br />Wordy<br />Unclear logic and confused indicators<br />Over-ambitious<br />Passive voice and wishy-washy wording (ie. Support provided to improve, including but not limited to..)<br />Overlapping with Country Program outcomes or repeating activities<br />August 2011<br />Management Practice - BRC<br />54<br />
    84. 84. Wordiness<br />August 2011<br />Management Practice - BRC<br />55<br />
    85. 85. Good Outputs examples (?)<br />Market-based vocational training programme developed<br />A legislative framework and code for local self governance is drafted<br />National budget process more effectively incorporates inputs from local governance structures<br />All immunization centers have a functioning cold-chain, and adequate supply of vaccines and Vitamin A.<br />August 2011<br />Management Practice - BRC<br />56<br />
    86. 86. August 2011<br />Management Practice - BRC<br />57<br />Good Output examples (cont’d)<br /><ul><li>National electoral body has adequate personnel, equipment and skills to administer free and fair national and local level elections by 2012.
    87. 87. Study of environment-poverty linkages completed
    88. 88. Police forces and judiciary trained in understanding gender violence
    89. 89. National, participatory forum convened to discuss draft national anti-poverty strategy
    90. 90. National human development report produced
    91. 91. Revised electoral dispute resolution mechanism established
    92. 92. Business processes reengineered
    93. 93. Compliance mechanisms established</li></li></ul><li>Indicators<br />Measures of performanceMonitoring = How are we doing?Indicators = How do we know?<br />
    94. 94. Indicators<br /><ul><li>Indicators describe how the intended results will be measured - accountability
    95. 95. Objectively verifiable, repeatable measures of a particular condition
    96. 96. They force clarification of what is meant by the result …….the fine print!
    97. 97. Must be accompanied by baselines and targets</li></ul>August 2011<br />Management Practice - BRC<br />59<br />
    98. 98. Baseline, Target and Achievement<br />August 2011<br />Management Practice - BRC<br />60<br />Performance<br />Commitment<br />Achievement <br />At end of period<br />Planned<br />Level of<br />Achievement<br />Target<br />Achievement<br />Current<br />Level of<br />Achievement<br />Baseline<br />
    99. 99. What to specify for each indicator?<br />Unit of analysis;<br />Existing baseline information;<br />Target for subsequent comparison;<br />Expected perceptions or judgments of progress by stakeholders;<br />Detailed description of expected conditions or situations to be observed;<br />1-4 indicators for each result are adequate – fewer the better to reduce cost.<br />August 2011<br />Management Practice - BRC<br />61<br />
    100. 100. Indicators ‘indicate’ that change is happening or not happening<br />They can<br />Clarify the scale and scope of a result in the results framework<br />Demonstrate progress when things go right<br />Provide early warning when things go wrong<br />Assist in identifying changes that need to be made in strategy and practice<br />Inform decision making<br />Facilitate effective evaluation<br />August 2011<br />Management Practice - BRC<br />62<br />
    101. 101. Indicators, Baseline, Target and Source of Data<br />August 2011<br />Management Practice - BRC<br />63<br />Indicators:<br />- # and proportion of the population with access to the Internet, disaggregated by gender<br />- % of population with computers and telephones<br />-Extent to which key policies on information technology are revised and promulgated<br />Legal and regulatory framework reformed to provide people with better access to information and communication technologies<br />Source of Data:<br /><ul><li> National Statistics
    102. 102. Monitoring reports
    103. 103. Surveys</li></ul>Baseline:<br />-(based on analysis, capacity assessment, reports, national statistics etc)<br />Target: actual value corresponding to each indicator, defined per year<br />
    104. 104. Types of indicators<br />August 2011<br />Management Practice - BRC<br />64<br />
    105. 105. August 2011<br />Management Practice - BRC<br />65<br />
    106. 106. Indirect (Proxy) Indicators<br />Direct indicators are preferable. However, they do sometimes not exist, are too expensive or inefficient to obtain. In such cases, indirect indicator have to be used.<br />August 2011<br />Management Practice - BRC<br />66<br />
    107. 107. August 2011<br />Management Practice - BRC<br />67<br />OUTPUT INDICATORS<br />
    108. 108. Indicators and Means of Verification<br />A key test in ensuring whether an indicator is really good is to define “means of verification” for each indicator; <br />Are data sources available?<br />What does it take to obtain the data? Who needs to be involved? How much will it cost to obtain the data?<br />If means of verification are unclear or unrealistic, the indicator and, possibly, the result need to be revisited;<br />The definition of results, indicators and means of verification is an iterative process.<br />August 2011<br />Management Practice - BRC<br />68<br />
    109. 109. Baselines and Targets<br />The indicator should be neutral <br />The target is what signals how much change and in what direction<br />The baseline and target should use the same unit of measurement as the indicator<br />August 2011<br />Management Practice - BRC<br />69<br />
    110. 110. The “SMART” way…<br />Specific<br />Measurable<br />Attainable<br />Relevant<br />Trackable<br />August 2011<br />Management Practice - BRC<br />70<br />
    111. 111. What are good indicators?<br />SPECIFIC<br />The indicator needs to be as specific as possible in terms of quantity, quality, time, location, target groups, baseline and target for the indicator<br />MEASURABLE<br />Will the indicator show desirable change?<br />Is it a reliable and clear measure of results?<br />Is it sensitive to changes in policies & programmes?<br />Do stakeholders agree on exactly what to measure?<br />ACHIEVABLE<br />Are the result(s) realistic and based on risk assessment, partnership strategy and other factors contributing to the underlying result<br />RELEVANT<br />Is it relevant to the intended result?<br /> Does it reflect the expectations and success criteria for change in the target groups?<br />TRACKABLE<br />Are data actually available at reasonable cost & effort?<br />Can proxy indicators be used?<br />Are data sources known? <br />August 2011<br />Management Practice - BRC<br />71<br />
    112. 112. Exercise for 10 mins<br />
    113. 113. August 2011<br />Management Practice - BRC<br />73<br />Exercise: Result Chain<br />
    114. 114. August 2011<br />Management Practice - BRC<br />74<br />Exercise: Result Chain<br />
    115. 115. Thank you!<br />Resource Person<br />Patrick Grémillet<br />Management Practice Coordinator<br />UNDP Bratislava Regional Center<br />Patrick.gremillet@undp.org<br />This presentation is using slides/inputs from training materials developed by the UN Staff College and RBM consultants<br />August 2011<br />Management Practice - BRC<br />75<br />

    ×