Presented by Jason Smith, Stacey Gage and Susan Post at the MEASURE Evaluation End-of-Phase-III Event.

1. M&E Capacity Building in Phase III:
An Overview
Jason B. Smith, PhD, MPH
MEASURE Evaluation
End-of-Phase-III Event, May 22, 2014

2. Session Structure
 Overview
 Presentation 1 –
Regional Training
 Presentation 2 –
Organizational
Development
 Interactive Dialogue
 Wrap-Up

3. Phase III Capacity-
Building Goal
 To strengthenthe technical,
managerialand leadership
capacity of individuals and
institutions for the identification
of data needs,as wellas the
collection, analysis and use of
appropriateinformationto meet
thoseneeds.

4. Capacity Building:
System Level
 M&E system
assessments
 M&E system
operational plans
 Selected systemic
capacity-building
interventions

5. Capacity Building:
Organizational Level (1)
 Organizational
Developmentaddedin
PhaseIII
 Enablingenvironmentfor
application of knowledge
and skills

6. Capacity Building:
Organizational Level (2)
 Identified organizational barriers
 Conducted virtual and face-to-face
programs
 Assisted in development of
plans
 Strengthened capacity of regional
partners

7. Capacity Building:
Individual Level
 In-service training
 Pre-service training
 Distance learning
 Capacity for research
 Follow-up and
on-going support
of trainees

8. Capacity Building Through Technical
Assistance (CB thru TA)
“Every technical assistance activity is an opportunity to
improve M&E capacity”
Some CB thruTA processes:
 Identify opportunities
 Make the need for explicit
 Prepare/adapt tools
 Coaching/mentoring
 Documenting success

9. Selected Accomplishments:
Training
 43,000+ trainees
 Vast majority in-
country
 1,000+ training events

10. Selected Accomplishments:
Distance Learning
 8 new courses – 22 total offerings
 56,000+ people reached

11. Selected Accomplishments:
Communities of Practice (COP)
 10 new COPs /
current total = 13
 6900+ COP
members

12. Current Regional Training Partners
 TheNational Institute forPublic Health (Mexico)
 AddisContinentalInstitute forPublic Health (Ethiopia)
 University ofGhanaSchoolofPublic Health (Ghana)
 University ofPretoria (South Africa)
 African Graduate Studies CentreforManagement
(Senegal)
 Public Health Foundation ofIndia(India)

13. Selected Accomplishments:
Regional Training Partners (1)
 42 workshops with MEASURE Evaluation support
 70 workshops offered independently
 1 new MPH M&E track with 2 additional tracks
expected to launch in Fall 2014

14. Global Evaluation and Monitoring
Network for Health

15. Capacity Building with
Regional Training Partners
MEASURE Evaluation End-of-Phase-III Event
May 22, 2014

16. Objectives
 Discuss the potential audience, objectives,
processes, criteria for success, and
outcomes of M&E capacity building with
regional training partners
 Describe MEASURE Evaluation Phase III
experiences and lessons learned

17. Whose M&E capacity
should we build?

18. Capacity Building Objectives
INSTITUTIONAL LEVEL
 Enable training partners to become regional centers of
reference for M&E
 Conduct training programs (e.g., regional workshops, distance
learning, pre-service training)
 Provide technical assistance in M&E
 Conduct evaluation research
INDIVIDUAL LEVEL
 Build a cadre of well-qualified M&E professionals

19. M&E Technical Capacity
Strengthened
0%
20%
40%
60%
80%
100%
Total Sub-Saharan
Africa
Latin America Asia
National Govt. Research University
US Govt. USAID-funded Organizations
Other Donors MEASURE Evaluation
Other
Percent distribution of trainees by type of employer,
2008-2014

20. Percent of Women among Participants
at Regional Trainings, 2008-2014
31.1 33.3
27.2
64.0
35.6
30.9
46.6
38.0
0
20
40
60
80
100
AAU,
Ethiopia
ACIPH,
Ethiopia
CESAG,
Senegal
INSP,
Mexico
PHFI,
India
UG,
Ghana
UP,
South
Africa
Total
PercentFemale

21. What M&E capacity
should we build?

22. Knowledge Gained from Regional
Trainings: Fundamentals
 Basic concepts & practical
approaches for M&E
 State-of-the art tools & techniques
 Knowledge & skills in designing
M&E plans
 Decisions that can be informed
by M&E information
 Basic concepts & practical
approaches for M&E
 How to improve data quality
 How to define programmatic-
ally relevant indicators
PHN M&EWorkshop HIV M&EWorkshop

23. Knowledge Gained from Regional
Trainings: Higher Level Training
 Basic M&E concepts
 Impact evaluation questions
 Evaluation designs & estimation
techniques for program impact
 Criteria for choosing appropriate
estimation strategy
 How to interpret results & their
programmatic implications
 Fundamental concepts of GIS
use in public health context
 Practical experience in use of
GIS software & spatially
referenced data
 Analysis, interpretation &
presentation of GIS data
Impact EvaluationWorkshop GISWorkshop

24. Master’s Degree Programs
in Phase III
 New Programs Established
 Addis Continental Institute of Public Health
 Phase II Programs Supported
 INSP
 University of Pretoria
 Programs Under Development
 Public Health Foundation of India
 University of Ghana

25. What is the process for
enhancing M&E capacity of
regional training partners?

26. Process for Enhancing M&E Capacity
of Regional Training Partners
 Training-of-trainer workshops
 Learning exchanges
 Provision of books, software, teaching materials
 Academic coordination and mentoring
 South-to-South collaboration
 Training partner engagement in M&E TA
 Organizational development

27. Who should deliver regional
capacity-building activities?

28. Delivery of Regional Trainings
to Build M&E Capacity
2
4
13
7
13
4
9
3
2
7
1
0 0
18
0
4
8
12
16
20
AAU
2009/13
ACIPH
2011/14
CESAG
2009/14
INSP
2009/14
PHFI
2009/14
UG
2009/14
UP
2009/14
Number
Jointly Independently
Number of regional workshops offered jointly with
MEASURE Evaluation and independently, 2009-2014

29. What are the key ingredients
of a successful capacity
building collaboration?

30. Ingredients of a Successful Capacity
Building Collaboration
 Cultivating trust and confidence among partners
 Institutional/leadership commitment to M&E
 Identification of M&E champion/proponent in partner
organization
 Core team of M&E trainers
 Responsibility for adequate funding/business sense
 Connectedness to other centers of M&E activity

31. What would you want to
know about outcomes
among trainees?

32. 94% of Trainees Report Being Involved
in M&E Activities in their Current Job
“Firstly, I have trained my staff in M&E. Secondly, I
use forms and methodologies from the workshop to
manage data, evaluate and facilitate data use”
(2011.9).
“The workshop has helped me incorporate a Spatial Information
Management System in the National M&E System, as well as
incorporate the Community-Based Program Activity Reporting
Tool for the National AIDS Control Council…” (2012.5).
“With RDQA tools, we are monitoring nine major indicators that we
report to the Global Fund in every quarter. Thanks to the workshop
organizers to include RDQA in the workshop” (2011.11).

33. Which Countries do Trainees
Come From?
89 Countries from Around the World Represented
45 African Countries (all of sub-Saharan Africa,
except Somalia)
All but 5 Countries in South America
20 Countries from Asia

34. Global Reach of Regional
Training Events in Phase III

35. Organization Development:
Outstanding M&E Results
Susan E. Pritchett Post
Leadership and Organization
Development Specialist
MEASURE Evaluation
End-of-Phase-III Event, May 22, 2014

36. Session Objective:
Respond to Questions
 What is Organization
Development?
 What did we do?
 What was value added?

37. What is Organization
Development (OD)?
Organization Development comprises:
 Leadership Development: “Managers Who Lead”
mobilize teams to envision and create a positive future.
 Organizational effectiveness: Good leaders ensure that
management systems and plans (strategic, operational,
and business plans) support the success of the
organization.

38. What Did We Do?
Leadership Development
 Leadership Development
Programs
 Team-based
 Action learning
 Participatory and
experiential

39. Goal of Leadership Development:
Leader Shifts
From Managers Who Lead: A Handbook for Improving Health Services
Cambridge, MA: Management Sciences for Health, 2005

40. Leadership Development
Program (LDP) in
Cote d’Ivoire (1)
 M&E teams from 4 ministries
 Developed shared vision of a
national HIV/AIDS database
 Desired result – compendium of
harmonized indicators
 Governance challenge

41. LDP in Cote d’Ivoire: Results (1)
 Results
 Aligned PEPFAR and mobilized funding
 Held harmonization meeting
 Published compendium
 Sustainability – compendium of tools, strategic
plan

42. Virtual Leadership Development
Program (VLDP)
 Benefits of virtual
 9 VLDPs offered
 110 teams enrolled from
 86 teams/684 participants completed plans
 76% of teams implemented plans and achieved
results
 Commonly chosen topics

43. Reach of VLDPs
North Atlantic Ocean
Indian Ocean
Arctic Ocean
Arctic Ocean
North Pacific Ocean

44. Virtual Leadership Development
Program (VLDP)
Ethiopian Interfaith Forum for Development
Dialogue and Action (EIFDDA)
 Challenge – Data received not timely or accurate
 Root cause – Lack of M&E knowledge
 Exceeded Desired Result – Training program developed,
reached 270 local member staff members, 240 more than
expected
 M&E impact – 100% of members improved reporting to
EIFDDA, yielded better reporting to Global Fund

45. Organizational
Effectiveness
 Business Planning
 Management systems
assessments and
improvement
 Curriculum Development
 Community of Practice

46. Business Planning: Public
Health Foundation of
India (PHFI)
 Focus – Develop a regional
center of reference in M&E
 Process – Scope,
Product/Service
identification, Marketing,
Financial, etc.
 Results – Follow-up donor
conference

47. Other Activities
 Curriculum development –
Building Leadership for
Data Demand and Use
 Community of practice –
Creating Enabling
Environments for M&E
(CEEME) at
www.Leadernet.org

48. The research presented here has been supported by the President’s
Emergency Plan for AIDS Relief (PEPFAR) through the United
States Agency for International Development (USAID) under the
terms of MEASURE Evaluation cooperative agreement GHA-A-00-
08-00003-00. Views expressed are not necessarily those of PEPFAR,
USAID or the United States government.
MEASURE Evaluation is implemented by the Carolina Population
Center at the University of North Carolina at Chapel Hill in
partnership with Futures Group, ICF International, John Snow, Inc.,
Management Sciences for Health, and Tulane University.

49. www.measureevaluation.org/eop