VC M&E Module 8 - Integrate Other Design Features into the M&E System

709 views
562 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
709
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
37
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • These additional features ensure the smooth functioning of the system and the validity of M&E information
  • These staff should be engaged in this process to ensure the project’s M&E system integrates with other standards within the country office.
  • Pilot testing is particularly important for instruments that are to be rolled-out widely across the project’s operating units, clients, or other value chain actors. Pilot testing is less important (and in some cases may not be cost effective) for ad hoc information gathering activities designed to be done quickly with a convenience or targeted sample of clients or other value chain actors. Wherever feasible and cost-effective, however, information gathering instruments should be pilot tested prior to full rollout.
  • Explain these terms from page 79
  • Pages 79 and 80 describes these procedures
  • These components are explained on page 81
  • VC M&E Module 8 - Integrate Other Design Features into the M&E System

    1. 1. VALUE CHAIN MONITORING AND EVALUATION GUIDEMODULE8 Integrating Other Design Features into the M&E System
    2. 2. GETTING STARTED1May 22, 2012
    3. 3. GETTING STARTED2May 22, 2012
    4. 4. MATERIALS / INPUTS RECOMMENDED All M&E system design pieces developed to this point. Senior M&E staff and/or country program M&E staff working on related3May 22, 2012 initiatives
    5. 5. STEP-BY-STEP GUIDE1 • Develop data collection instruments2 • Pilot test data collection instruments3 • Arrange data collection logistics4 • Establish data quality procedures5 • Develop Project Database and Data Entry Procedures6 • Establish data management procedures7 • Establish data analysis procedures8 • Establish information dissemination procedures9 May 22,•2012 Assign clearly defined roles 4
    6. 6. STEP-BY-STEP GUIDE: STEP 1 Data Collection Instruments:•Must be developed for every level - observation & measurement•Must be developed in a participatory way, in collaboration with futureusers: •Front-line staff members using them routinely •Project evaluators designing baseline, midterm and endline•Important that these tools are developed in local languages and tested foraccuracy•Must be accessible to project participants, staff and partners – whoeverwill be using them 5 May 22, 2012
    7. 7. STEP-BY-STEP GUIDE1 • Develop data collection instruments2 • Pilot test data collection instruments3 • Arrange data collection logistics4 • Establish data quality procedures5 • Develop Project Database and Data Entry Procedures6 • Establish data management procedures7 • Establish data analysis procedures8 • Establish information dissemination procedures9 May 22,•2012 Assign clearly defined roles 6
    8. 8. STEP-BY-STEP GUIDE: STEP 2 Pilot Test •Should be done prior to full rollout •Identifies inappropriate, poorly worded, or mistranslated questions •Validates the existing coding scheme •Identifies omitted questions •Trains field researchers in tool implementation7 • Varies in importanceMay 22, 2012
    9. 9. STEP-BY-STEP GUIDE1 • Develop data collection instruments2 • Pilot test data collection instruments3 • Arrange data collection logistics4 • Establish data quality procedures5 • Develop Project Database and Data Entry Procedures6 • Establish data management procedures7 • Establish data analysis procedures8 • Establish information dissemination procedures9 May 22,•2012 Assign clearly defined roles 8
    10. 10. STEP-BY-STEP GUIDE: STEP 3 Ensure all necessary logistical arrangements, to include:•Selecting research team members•Transportation to and from the research site•Meals and lodging•Security arrangements•Communication with respondents or potential respondents•Means of communication between research teams and withthe head office•Protocols for addressing problems or questions as they arise•Safeguarding of blank and completed information gatheringinstruments•Storage and transfer of research findings 9 May 22, 2012
    11. 11. STEP-BY-STEP GUIDE1 • Develop data collection instruments2 • Pilot test data collection instruments3 • Arrange data collection logistics4 • Establish data quality procedures5 • Develop Project Database and Data Entry Procedures6 • Establish data management procedures7 • Establish data analysis procedures8 • Establish information dissemination procedures9 May 22,•2012 Assign clearly defined roles 10
    12. 12. STEP-BY-STEP GUIDE: STEP 4 Establish data quality procedures to prevent: Security lapses Data collection errors Data tampering11May 22, 2012
    13. 13. STEP-BY-STEP GUIDE: STEP 4Quality control procedures that mitigate threats: Quality M&E System Training Collaboration with data collectors Routine Refreshers In-field Supervision Reviews of Data12 Collection Forms Random Quality ChecksMay 22, 2012
    14. 14. STEP-BY-STEP GUIDE1 • Develop data collection instruments2 • Pilot test data collection instruments3 • Arrange data collection logistics4 • Establish data quality procedures5 • Develop Project Database and Data Entry Procedures6 • Establish data management procedures7 • Establish data analysis procedures8 • Establish information dissemination procedures 13 • Assign clearly defined roles9 May 22, 2012
    15. 15. STEP-BY-STEP GUIDE: STEP 5 Electronic databases for survey/questionnaire results:•Common format is to list questions across columns,and respondents down rows•Data management programs most often used: -Microsoft Excel -SPSS -Microsoft Access•Should capture info in the order it was asked 14•Should include built in data controls May 22, 2012
    16. 16. STEP-BY-STEP GUIDE: STEP 5 Consolidation of Qualitative Data:•Should be compiled and managed in a central space•Carefully consider data collection tools Challenges Lessons learned Easy Access for the Reader Anecdotes Success 15 stories May 22, 2012
    17. 17. STEP-BY-STEP GUIDE1 • Develop data collection instruments2 • Pilot test data collection instruments3 • Arrange data collection logistics4 • Establish data quality procedures5 • Develop Project Database and Data Entry Procedures6 • Establish data management procedures7 • Establish data analysis procedures8 • Establish information dissemination procedures9 May 22,•2012 Assign clearly defined roles 16
    18. 18. STEP-BY-STEP GUIDE: STEP 6 Critical components of data management: Data Storage Data Backup Data Entry Data Access Data Revision17May 22, 2012
    19. 19. STEP-BY-STEP GUIDE1 • Develop data collection instruments2 • Pilot test data collection instruments3 • Arrange data collection logistics4 • Establish data quality procedures5 • Develop Project Database and Data Entry Procedures6 • Establish data management procedures7 • Establish data analysis procedures8 • Establish information dissemination procedures9 May 22,•2012 Assign clearly defined roles 18
    20. 20. STEP-BY-STEP GUIDE: STEP 7 UsefulRaw Data Informatio n FINAL M&E EVALUATION AWESOME PROJECT KENYA19May 22, 2012
    21. 21. STEP-BY-STEP GUIDE1 • Develop data collection instruments2 • Pilot test data collection instruments3 • Arrange data collection logistics4 • Establish data quality procedures5 • Develop Project Database and Data Entry Procedures6 • Establish data management procedures7 • Establish data analysis procedures8 • Establish information dissemination procedures9 May 22,•2012 Assign clearly defined roles 20
    22. 22. STEP-BY-STEP GUIDE: STEP 8 Disseminating Data to Internal/External Users:•Procedures include timing, content and formatof information•Externally, determined bydonor requirements•Internally, co-developed withstaff who will use the 21information May 22, 2012
    23. 23. STEP-BY-STEP GUIDE1 • Develop data collection instruments2 • Pilot test data collection instruments3 • Arrange data collection logistics4 • Establish data quality procedures5 • Develop Project Database and Data Entry Procedures6 • Establish data management procedures7 • Establish data analysis procedures8 • Establish information dissemination procedures9 May 22,•2012 Assign clearly defined roles 22
    24. 24. STEP-BY-STEP GUIDE: STEP 9 Effective operation requires that all involved understand roles and responsibilities clearly, which should be included in: •Staff job descriptions •MOUs with implementing partners •Contracts with external researcher23May 22, 2012
    25. 25. COMMON PITFALLS•Failure to consider all the features necessary for afunctioning and effective M&E system.•Projects do not pilot test data collection tools andinstruments.•Projects do not establish data quality procedures orconduct periodic assessments of data gatheringactivities and data quality.•Projects do not establish data managementprocedures or plan for staff turnover•Projects do not plan to and/or invest in conductingsufficient staff training •Including follow up mentoring•Too much focus on donor needs, not enough on otherM&E system clients. 24 May 22, 2012
    26. 26. QUESTIONS ? COMMENTS ?25May 22, 2012
    27. 27. Want to Learn More?Multiple ways to continue the discussion and continue learning:• Initiate a monthly session on the M&E guide and case studies from across CARE. Contact cpennotti@care.org• Join the Market Engagement Community of Practice on LinkedIn.• Join a task force to review and refine the universal 26 indicators. Contact nardi@careinternational.org May 22, 2012

    ×