-Page 38 and 39-Universal Indicators available on Capacity Corner
-Discuss difference btwn impacts, outputs, leading and lagging outcomes if necessary
Then discuss the step-by-step from page 38. Or we can make a new slide for the steps of this tool.
For outputs you will simply list the number of activities to be completed or people to be trained, etc. For leading and lagging outcomes, you will need to develop indicators for routine measurement as well as indicators for routine observation for particularly important changes or assumptions in the causal model.
Note that your causal model may include some changes that you do not want to share too broadly, particularly those associated with promoting sector-level changes within organizations, companies or government agencies that will not be directly engaged by the project
Discuss case example on page 42
Page 45 discusses logframes in detail
VC M&E Module 4 - Select Performance Indicators
VALUE CHAIN MONITORING AND EVALUATION GUIDEMODULE4 Select Performance Indicators
GETTING STARTED Routine Measurement Routine Observation •Indicators tracked using •Indicators tracked using non- traditional M&E tools – surveys, traditional M&E tools – staff focus group discussions, etc. observation, team reflections, •Traditional M&E timelines – meetings and after action monthly, quarterly, reviews beginning/middle/end of project, •Traditional M&E timelines or etc. whenever staff learn something Traditional, Rigid important about that indicator. Flexible, Reactive2May 22, 2012
GETTING STARTED Routinely Measured Indicators Routinely Observed Indicators •Specific •Open-ended, indentified through participatory exercises •Measurable •Focused on anticipated changes •Achievable such as impact group attitude and •Relevant behavior •Weighted against validity of •Time-bound ‘killer’ assumptions •Revised as needed to suit implementation needsCONSULT CARE’S UNIVERSAL INDICATORS FOR MARKETENGAGEMENT 4 May 22, 2012
MATERIALS / INPUTS RECOMMENDED Causal model M&E client map Donor indicator list CARE CO program and global indicator lists Key team members and/or partner representatives5May 22, 2012
STEP-BY-STEP GUIDE • DEFINE YOUR IMPACT INDICATORS 1 • IDENTIFY AND DEFINE THE CRITICAL ASSUMPTIONS IN YOUR CAUSAL MODEL 2 • DEVELOP INDICATORS FOR YOUR OUTPUTS AND LEADING AND LAGGING OUTCOMES 3 • REVIEW THE INDICATORS WITH KEY M&E CLIENTS 46May 22, 2012
STEP-BY-STEP GUIDE: STEP 1 Defining Indicators •These will ultimately measure program’s success •Household, Enterprise and Sectoral Levels •Should align with CO, Donor, National Government and CARE Universal Indicators •Again, DO NOT SELECT MORE7 THAN YOU CAN MEASURE!!!May 22, 2012
STEP-BY-STEP GUIDE • DEFINE YOUR IMPACT INDICATORS 1 • IDENTIFY AND DEFINE THE CRITICAL ASSUMPTIONS IN YOUR CAUSAL MODEL 2 • DEVELOP INDICATORS FOR YOUR OUTPUTS AND LEADING AND LAGGING OUTCOMES 3 • REVIEW THE INDICATORS WITH KEY M&E CLIENTS 48May 22, 2012
STEP-BY-STEP GUIDE: STEP 2 WORK THROUGH THE CAUSAL MODEL •Develop indicators at other levels: •Outputs •Leading outcomes •Lagging outcomes •To avoid piling up more indicators than we can measure: •Focus on priority clients of M&E system •Prioritize most critical assumptions from9 causal model •Use this participatory activity……………..May 22, 2012
STEP-BY-STEP GUIDE: STEP 2 & Prioritizing Assumptions Tool: Identifying Objective: This activity will help key system clients articulate and identify the most critical assumptions. Format: A group of 10-15 M&E clients (or more) broken into groups of 4-5. Ensure plenty of open wall space and a work table for each group. Materials: A projector, laptop, flip chart paper, markers and plenty of small stickers of different colors (for voting). Time: Will vary depending on the number of M&E clients and project complexity but should be no less than 1 hour for a simple exercise and can be ¾ of a day or more for a large-scale program.10May 22, 2012
STEP-BY-STEP GUIDE • DEFINE YOUR IMPACT INDICATORS 1 • IDENTIFY AND DEFINE THE CRITICAL ASSUMPTIONS IN YOUR CAUSAL MODEL 2 • DEVELOP INDICATORS FOR YOUR OUTPUTS AND LEADING AND LAGGING OUTCOMES 3 • REVIEW THE INDICATORS WITH KEY M&E CLIENTS 411May 22, 2012
STEP-BY-STEP GUIDE: STEP 3 Resources for ID’ing Indicators for Outputs and Outcomes•Donor requirements•Country Office Program Impact MeasurementFrameworks•CARE’s Universal Indicators for Market Engagement•Other relevant projects operating within the country•CARE’s global impact measurement system 12 May 22, 2012•Government measurement systems.
STEP-BY-STEP GUIDE: STEP 3 COMPLETING STEP 3•Start with a particular intervention and workthrough the causal pathway until you link toyour impact level indicators•Trace the causal pathway that linksindicator to outputs leading outcomes lagging outcomes•Choose a second and repeat the processuntil you have identified indicators for eachcausal pathway13May 22, 2012
STEP-BY-STEP GUIDE • DEFINE YOUR IMPACT INDICATORS 1 • IDENTIFY AND DEFINE THE CRITICAL ASSUMPTIONS IN YOUR CAUSAL MODEL 2 • DEVELOP INDICATORS FOR YOUR OUTPUTS AND LEADING AND LAGGING OUTCOMES 3 • REVIEW THE INDICATORS WITH KEY M&E CLIENTS 414May 22, 2012
STEP-BY-STEP GUIDE: STEP 4 Validate Indicators •Review the selected indicators with donor, project staff, managers, partner organizations and target participants •Group meetings, email reviews and comments, etc. •Whatever process you use should be transparent and M&E clients’ comments should be fully considered before15 finalizing your indicator setMay 22, 2012
CASE EXAMPLE SDVC Insert SDVC pic16May 22, 2012
COMMON PITFALLS•TOO MANY INDICATORS……..Don’t be afraid to negotiate.•Contradictions between donor requests and the good of the project (what to measure,approaches, etc.) . Use informed rationale to discuss these disparities with the donor!!!!!!•Projects are unwilling or unprepared to make difficult choices to weed out ‘nice but notnecessary’ indicators.•Projects fail to engage the donor in dialogue regarding expectations of the M&E system (highor low).•Projects focus too much on output indicators that will not provide insight into what the projectis accomplishing.•Projects do not use the SMART test on the indicators they plan to track through routinemeasurement, leading to indicator lists that are poorly aligned with killer assumptions and/ordifficult or impossible to measure reliably.•Projects fail to develop a set of indicators that will be tracked through routine observation atthe outset, leading to a lot of information being collected, very little of which can be aggregatedand/or applied to inform project decision making. 17 May 22, 2012
TEMPLATES AND SUPPORTINGMATERIALS Engagement Indicator Information CARE MarketResult Level Domain TableChange Key Indicator - Indicator - Target Data Users Data Source Collection Collection Responsible (Womens Emp, (Y/N) Measurement Observation (Clients) Method / Frequency for Collection Enterprise, Sector) ToolImpact Note: At this stage, you willLaggingoutcome not fill in the information in these three columns. YouLeading will do so afteroutcome the next Chapters in this guide.Output 18 May 22, 2012
TEMPLATES AND SUPPORTINGMATERIALS LOGICAL FRAMEWORKS Narrative Results Indicators Assumptions Summary Impacts Lagging Outcomes Leading Outcomes Outputs Interventions 19 May 22, 2012
TEMPLATES AND SUPPORTINGMATERIALS Measurement Tools Available by Poverty Country Country PPI PAT (USAID) Angola Bangladesh X X+ Benin Bolivia X X Cambodia X X Cote d’Ivoire Ecuador X X+ El Salvador X X+ Ethiopia X X Ghana X X+ Guatemala X X Haiti X X Honduras X India X X Kenya X X+ Liberia X X+ Malawi X X Mali X Mozambique Nepal X X Nicaragua X Niger Pakistan X Peru X X Rwanda X Sierra Leone Sri Lanka 20 X Tanzania May 22, 2012 Uganda X+ Vietnam X X
Want to Learn More?Multiple ways to continue the discussion and continue learning:• Initiate a monthly session on the M&E guide and case studies from across CARE. Contact email@example.com• Join the Market Engagement Community of Practice on LinkedIn.• Join a task force to review and refine the universal 22 indicators. Contact firstname.lastname@example.org May 22, 2012
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.