The Afrobarometer is a research project that measurespublic attitudes on economic, political, and social matters in sub-Saharan Africa. It is carried out through a partnership of the Institute for Democracy in South Africa (IDASA),the Ghana Center for Democratic Development (CDD), and the Department of Political Science at Michigan State University.Afrobarometer results are based on face-to-face and house-to-house interviews of individuals and are considered reliable and generalizable. As of October 19, 2006, Afrobarometer data and publications had been cited 216 times.
From UNDAF Guidelines, p15-16. See note on pros and cons of the different options.UNDAF Option 1B. Options: UNCTs have two options for the level of results in the UNDAF results matrix The UNCT, with the government, determines which option responds best to the country context. UNCTs have the flexibility to either keep the UNDAF results matrix at the outcome level (Option 1a), or develop a fuller results matrix, that includes outputs (Option 1b). Both options include indicators, baselines, targets, means of verification, risks and assumptions, role of partners and resources. The results chain and accountability system have to be agreed upon by all stakeholders. If a UNCT chooses Option 1b, and is not preparing an UNDAF Action Plan or a similar common operational plan, the results chain would then be: Outcomes & Outputs (UNDAF) ← Outputs & Key Actions (CPAPs/Project documents) ← Activities (Annual Work Plans). This option represents the conventional format of the UNDAF results matrix. Key Benefits:Transparency: Overview of the full extent of UN commitments in the country; Accountability: A detailed UNDAF results matrix provides the UNCT with a clear accountability framework since the division of labour between agencies only becomes apparent at that level: usually, only one agency is accountable for each output. Issues to consider:Transaction costs: Developing a realistic, coherent and logical results framework that includes both outcomes and outputs may be a difficult and time-consuming process. Yet, as experience has shown, this investment in the initial planning stage will greatly facilitate effective implementation, monitoring and evaluation throughout the programming cycle.Prioritization: Previous UNDAF generations had the tendency to contain far too many outputs and indicators in order to be useful as effective management tools. The UNCT needs to prioritize its results at every level (outcomes, outputs, activities) in line with its capacities and comparative advantage and without compromising the flow of the results chain.Regardless of the option chosen, agencies are accountable for their respective contributions towards the achievement of outcomes. Agencies are fully accountable for their respective outputs. Outputs can be shared by two or more agencies and their implementing partners only if they indicate an opportunity for joint programming. Outputs can be adjusted, where necessary, to take account of changes in the development environment, including changes in availability of resources. (See One Budgetary Framework)
In the area of anti-corruption, it is better to highlight risks and assumptions, because most of the time failure of many anti-corruption interventions is because of the enabling environment (such as political, economic, social, political and institutional factors), they influence the results.
Anti-corruption and Result-Based Management (RBM)UNDP2012
In plain language…RBM helps us to connect what we do to what we want to achieveRBM also tells us how we’ll know if we’ve achieved itWhat is results based management?A management strategy that aims at ensuring:• that activities achieve desired results(Performance monitoring is a critical element)• How well results are being achieved• What measures are needed to improve the process
ResultsGoalImpactOutcomeOutputsActivityIndicatorRBM Terminology (UNDG Approved)Changes in a state or condition which derive from a cause-and-effectrelationship. There are three types of such changes (intended or unintended,positive and/or negative) which can be set in motion by a development intervention– its output, outcome and impact.The higher-order objective to which a development intervention is intended tocontributePositive and negative long-term effects on identifiable population groups producedby a development intervention, directly or indirectly, intended or unintended. Theseeffects can be economic, socio-cultural, institutional, environmental, technologicalor of other typesThe intended or achieved short-term and medium-term effects of anintervention’s outputs, usually requiring the collective effort of partners.Outcomes represent changes in development conditions which occur between thecompletion of outputs and the achievement of impact.The products and services which result from the completion of activitieswithin a development interventionActions taken or work performed through which inputs, such as funds,technical assistance and other types of resources are mobilised to produce specificoutputsA tool to measure evidence of progress towards a result or that a result hasbeen achieved.
Advantages of RBM• Improved focus on results instead of activities• Improved transparency• Improved accountability• Improved measurement of programmeachievements (performance rather thanutilization)• Enhanced strategic focus
Challenges• Difficult to apply causal logic• Difficult to learn• Difficult to integrate• Difficult to revise (... or reluctance to revise? )• Difficult to measure• Difficult to ‘attribute’(at outcome level, the UN is partly accountable but not whollyresponsible) Go to typology
Linking analysis with results Impact: (e.g., corruption reduced) Outcome: (e.g., improvements in the performance ofgovernment institutions as well as monitoring mycitizens) Output: (e.g., tangible changes in any elements in thecapacity of government institutions and population(information, attitudes/motivation, risks, resources)
Linking Analysis to results:Translating the problem tree into resultsCausality Analysis Results chain(Problem tree) (UNDAF matrix)Immediate causes Impact (long-term)Underlying causes Outcomes (medium-term)Root causes Outputs (short-term)
1. To effectively analyze problems and their causes, usingevidence and lessons including key institutions/capacity gaps4. To establish performance monitoring and evaluation systems(including risk management)5. To capture and monitor actual results vis-à-vis targets6. To clearly identify issues/bottle necks, generating lessonsregarding what works/ what doesn’t work2. To develop right indicators that measure capacity changes andperformance improvements3. To determine baseline and setting targetsLinking analysis with results: Why results frameworksare relevant for anti-corruption in UNDAF?UNDAFs?
efficiencyeffectivenessImpactsOutcomesOutputsActivitiesInputsRESULTSINFLUENCEPerformanceindicatorsPlanningMonitoringLinking Analysis with Results: The Results Chain
Result Chain – Quick RecapInputsActivitiesOutputsOutcomesImpactIndicatorResources available - money, staff, facilities, equipment, technical expertiseWhat is done with the inputs - holding seminars, producing manualsServices or products produced as a result of activities – 100 staff traineds Changes, effects due to the activities and outputs – higher skill levelsLong-term results – quality of government services, less corruptionMeasure progress towards a result over time that indicates positive, negative,no change with respect to progress towards a stated target
What is an indicator?A tool to measure evidence of progress towards a result orthat a result has been achieved.
Types of IndicatorsQuantitative statisticalmeasures:• Number of• Frequency of• % of• Ratio of• Variance withQualitative judgments orperceptions:• Alignment with• Presence of• Quality of• Extent of• Level of
Considerations Indicators do not exist in a vacuum – they shouldalways be related to results Need a balance of quantitative and qualitative Some results are more suitable for indicators thanothers It takes time to get indicators right
A Key Challenge:Is it possible to meaningfully measure corruption, which hasno universal definition, but has different forms, typologies,manifestations, determinants, causes and symptoms?“What gets measured, gets managed.” - Peter Drucker. So,the UN should demonstrate results, results and results.
1. No indicator is perfect and thus standard; all haveadvantages and disadvantages. However, globalperception or cross-country indicators are not useful fortracking progress at the country level.2. In terms of quality and usefulness of data, it is advisablethat the country-based and nationally-owned corruptionmeasurement/assessment indicators be used.3. Similarly, experience-based indicators (victimizationsurveys) are advisable over the perception indices.However, victimization surveys data are expensive, too.
1. The perception data (e.g., score cards) at the country levelhas been often used as a proxy.2. If the country level data/indicator are not available, regionalsurveys such as Afrobarometer or Latin Barometer can alsobe useful.3. The selection of indicator mainly depends on anti-corruptionoutcomes and outputs, availability of data and the costassociated with gathering data.
DOs and DON’Ts of capturing adequate orsatisfactory resultsBad GoodNo supporting indicators, programmelogic or dataIncrease evaluability of anti-corruption projects, prospectiveevaluationsWeak, activity-oriented indicators Focus on a few good results oriented(outcome and output) indicators, anddisaggregate dataMajor cause-effect (attribution) leapsfrom output to impact levelStop reliance on macro-level globalindices (CPI) and develop countrylevel indicatorsNo evaluation/logical framework tocapture resultsMixed methods. Pragmatic approachto statistical design (best fit) andstrong qualitative supportWorst: Claiming impact when there isno result chain or frameworkAdapt impact evaluation methods tocharacteristics of anti-corruptionprojects
Performance Indicator Selection Criteria Validity - Does it measure the result? Reliability - Is it a consistent measure over time and, ifsupplied externally, will it continue to be available? Sensitivity - When a change occurs will it be sensitiveto those changes? Simplicity - Will it be easy to collect and analyze theinformation? Utility - Will the information be useful for decision-making and learning? Affordable – Do we have the resources to collect theinformation?
Illustrative example of the results chain of anAnti-corruption initiative (adapted from a country example)Stakeholder dialogue regarding corruption issues, review of current auditmechanism and gaps, assessment and development of the capacities ofsupervisory body, budgetary and funding support; etc.(money, staff, facilities, equipment, technical expertise)Governmentsupervisoryfunctions and auditprocesses areestablishedAnti-corruptioninstitutions/ supervisorybodies are enforcing thelawMedia haschannels to publishcorruption relatedinformationInformation regardingincidences of corruptionis accessible to thepublicPublic government officials, citizens, etc. have reduced theirengagement in corruption practicesReduced corruptionand increased public trust towardspublic institutionsCitizens havemechanism toreport oncorruptionincidencesGovernment agencies areexchanging informationabout corruption incidencesCorruption datamanagementsystem is put inplaceKnowledge, Innovation and Capacity Group, BDP, UNDP (April, 2013)InputsActivitiesOutputsOutcomesImpact
• Currently many UNDAFs restrict the aspect of anti-corruption to the objective of governance / human rights/ democracy. Anti-Corruotion is hardly an output in caseof MDGs, environment, conflict prevention, disasterreduction, etc.• Although corruption is identified in some of the UNDAFsdocuments, the RBM is very weak.Anti-corruption in UNDAF’s
Anti-corruption in UNDAF’sOption 1 A: UNDAF RESULT MATRIC WITH OUTCOME LEVEL ONLY (Mongolia)Strategic priority - Governance and Human rightsOutcomes Indicators, Baseline, TargetOutcome 1: Representation, accountability andtransparency of governing institutions strengthenedOutput 1.1. Enabling policy environment created foreffective decentralization and increased functionalcapacity of local governments to deliver service.Capacity of local governments to deliver servicesOutput 1.2.Increased capacity to implement the UN ConventionAgainst CorruptionCompliance with UNCAC provisions oncorruption preventionOutput 1.3. Increased civil society participation in keynational processes and strengthened state-citizenengagement for accountable and responsive governanceFeedback mechanism of state-CSOs-state Socialdialogue between government, workers andemployersBad or good?
What is the underlying vision behind thereform/policy? What are the objectives pursued?Which results do we expect to see? What is the logic underpinning the reform/policydesign? (from inputs to outputs, outcomes andimpact) What are the preconditions for success? Whichbehavioural assumptions are being made? Howwill political, cultural or economic factors affectpotential for change?Key questions you need to consider whenformulating the results matrix
Reduction incorruptionCorruptioncases areinvestigated torequiredstandardSubstantiatedcases are sentto court andprosecutedOther lawenforcementinst.cooperate.Witnessesare willingto comeforwardSufficientresources andmandateDonordemandsfor outputsLegislativeobstaclesResult Chain with Risks and Assumption
Group Work 2 (15 minutes)1. Put together a coherent, logically sequenced resultchain to introduce a Code of Conduct (CoC) inministries (yellow pieces)2. Identify where indicators fit at the right level of resultsin the chain.Inputs Activities Outputs Outcome ImpactsOutputindicatorsOutcomeindicatorsImpactindicators
Key messages• Corruption and anti-corruption measurements andassessments provide useful information for countryanalysis, which is an integral part of UNDAF process.• Corruption and anti-corruption measurements andassessments also strengthen results by providingindicators to measure, monitor, and report on change.• However, it is also important to make sure that theappropriate indicators are used to measure progress andresults.