Your SlideShare is downloading. ×
Monitoring and Evaluation of International Development Assistance to the Private Sector
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Monitoring and Evaluation of International Development Assistance to the Private Sector

234
views

Published on

Effective monitoring and evaluation (M&E) systems are essential to learning and accountability. M&E system reviews provides perspective on what is working well, where there are gaps in coverage or …

Effective monitoring and evaluation (M&E) systems are essential to learning and accountability. M&E system reviews provides perspective on what is working well, where there are gaps in coverage or weaknesses that need to be addressed, how the M&E information is actually used in decision making, and whether the system is efficient. This session will demonstrate the methods, tools and results in assessing the functioning of the M&E systems of the World Bank Group’s private sector operations in two specialized agencies: the International Finance Corporation and the Multilateral Investment Guarantee Agency.

Published in: Economy & Finance, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
234
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • IEG Private Sector Evaluation (IEGPE)*Evaluates the effectiveness of programs, projects, and activities and the strategies, policies and procedures that relate to them with particular attention to the achievement of objectives for private sector development Generates lessons for learningSupports the Board in ensuring WBG’s development effectiveness and accountability*Adapted from Director’s terms of reference approved by the Board (November 2005)
  • Transcript

    • 1. Monitoring and Evaluation ofPrivate Sector DevelopmentHiroyuki HatashimaPrivate Sector Evaluation,Independent Evaluation Group (IEG)World Bank GroupCanadian Evaluation Society ConferenceJune 11, 2013
    • 2. IEG is part of a larger array ofevaluation and control mechanisms2World Bank GroupPresident / ManagementManagement’s InternalAccountability FrameworkAnd Systems•Self Evaluation•Monitoring (QAG, DOTS, AS)•Institutional Integrity•Internal Audit•Compliance Advisor/ Ombudsman(IFC, MIGA)IndependentEvaluationGroupInspectionPanelBoard ofExecutive Directors
    • 3. IEG seeks to address bothaccountability and learning• Accountability for results:– Is the Bank Group doing what it says?– Is it achieving its objectives?– To what extent is it contributing to development outcomes?– With what results and development impact?• Learning:– What projects and programs achieve results –relevance, efficacy, and efficiency – and why?– How can M&E be strengthened in the Bank Group and inclient countries?3
    • 4. Three main messagesEvaluation Across Boundaries• Private Sector operations have specific datagathering advantage/disadvantage based ontheir business model• M&E systems should adopt to the businesspractices to be effective and efficient• M&E can influence learning, quality of workand outcome
    • 5. Outline of Presentation1. Context of Private Sector Development Assistance2. Types of World Bank Group Private SectorDevelopment interventions3. Technical Assistance projects M&E4. Private Sector Investment projects M&E5. Use and Influence of M&E
    • 6. 1.1 Context: Private Sector As Important Player1. The private sector has taken on an essentialrole in contributing a wide range ofdevelopment issues.2. Potential of private sector to contribute todevelopment solution is still underutilized.3. M&E essential for track progress, assessoutcome & performance and generatingknowledge of what works, what does not, andwhy.
    • 7. Are the monitoring and evaluation systems of IFCand MIGA equipped to inform the organizations ontheir performance and results?Systems functions• To what extent doesthe mechanism inplace ensure thatM&E systemsgeneratecredible, timely, andrelevantinformation?Use and Influence• To what extent doesM&E informationsupport evidence-based decisionmaking andlearning?Effectiveness andEfficiency• What has been theimpact of the M&Eoutputs and use onproject quality anddevelopmentoutcomes?1.2 Assessing M&E systems –Evaluation Questions
    • 8. Outline of Presentation1. Context of Private Sector Development Assistance2. Types of World Bank Group Private SectorDevelopment interventions3. Technical Assistance projects M&E4. Private sector Investment projects M&E5. Use and Influence of M&E
    • 9. The World Bank Group consists offive closely related institutions:• IBRD: The International Bank forReconstruction and Development• IDA: The International DevelopmentAssociation• IFC: International Finance Corporation• MIGA: The Multilateral InvestmentGuarantee Agency• ICSID: The International Center for theSettlement of Investment Disputes
    • 10. 1. Types of WBG Private Sector Developmentinterventions1. Interventions through the Public Sector to influencethe Private Sector World Band Financial and Private Sector Developmentprojects, IFC’s Advisory Services projects to government (InvestmentClimate, PPP)2. Direct interventions in partnership with the privatesector and public entities operating commercially IFC’s investments and MIGA’s guarantees3. Blend interventions IFC’s Advisory Services to private companies, but with astrong public good component
    • 11. Outline of Presentation1. Context of Private Sector Development Assistance2. Types of World Bank Group Private SectorDevelopment interventions3. Technical Assistance projects M&E4. Private Sector Investment projects M&E5. Business Models and M&E
    • 12. 3.1. Technical Assistances followlogical model results measurementImpactOutcomesOutputsActivitiesInputs Resources usedfor advisoryservices projects Programsassociated with anadvisory project Example:– Provideadvisoryservices tolocalcompanies Immediatedeliverables ofthe project Example:– Diagnosticreport andrecommendations made– 10 managerswere trained– Legislationdrafted Changes inknowledge, behaviors, and attitudesas a result ofintervention withattribution ofchanges to theproject Example:– Legislative orinst reforms– Changes inagriculturaltechniques, orCG practices Consequencesresulting from aninterventionincludingimprovements inclients’performance andeffects beyonddirectbeneficiaries Example:– increasedinvestment, sales, jobs
    • 13. Track indicatorsProject Supervision Report(PSR)SuperviseApprove Disburse CompletionCommitM&E systemLogframe,Identify indicatorsProjectCompletionReport(PCR)3.2. M&E follows project cycle
    • 14. Development Effectiveness FrameworkStrategicRelevanceWas it the rightproject at theright time?OutputAchievementWere “key”deliverablesachieved? Towhat extent wereclients satisfiedwith the advisoryservice?OutcomeAchievementWere intendedshort & medium-term changes inbehaviorachieved?EfficiencyWere costsreasonable vis-à-vis results?ImpactAchievementWere intended longer-term effects achieved?What would havehappened without theintervention? (What isthe counterfactual?)• These combine into a “synthesis” ratingIn addition: Assessment of IFC Role & Contribution:– Was IFC particularly catalytic or innovative in its advisory services?– Was IFC able to play a unique role not easily filled by others?3.2. Advisory Services ProjectCompletion Report as Self-Evaluation
    • 15. 3.4 Technical Assistance M&E –ways to adopt to business practicesProgress Integration with project cycle M&E officer playing important andformal roles in quality assurance Quality control cycleestablished M&E officers sign-off, DevelopmentImpact Dept reviews M&E quality improved Quality of logical framework, baselinedata, quality of results discussion – allimprovedChallenges Standard indicators notsufficient to track results Inadequate information inPCR to justifyoutcome/impact ratings Post completion follow-upsystem not systematic Work quality assessment notcovered in PCR
    • 16. Outline of Presentation1. Context of Private Sector Development Assistance2. Types of World Bank Group Private SectorDevelopment interventions3. IFC Advisory Services (Technical Assistance)projects M&E4. Private Sector investment projects M&E5. Use and Influence of M&E
    • 17. 4.1. Characteristics of Private Sector investmentssupported by IFC and MIGA Operations subject to market testDirect Beneficiaries pay for serviceSome degree of competitionSponsors motivated by profitability Third party effects (TPE) with no market feedbackEconomic distortionsPositive and negative externalitiesProvide rationale for public intervention Development Agency’s involvementConsistent with market principlesFocused on third party effects (TPE)
    • 18. Neighbors /EnvironmentCustomersGovernment /TaxpayersEmployees4.2 Assessing Development Results - StakeholderFrameworkCompetitorsProducers ofComplementaryProductsSuppliersNew EntrantsProjectCompany(financiers)
    • 19. 4.3. Performance dimensions of IFCinvestments and MIGA guaranteesObjective Stakeholders Criteria Standards AssessmentDevelopmentOutcomeFinanciers Businesssuccess(Market test)Marketbenchmarks forROICActual financialstatements andobjectives at approvalCustomers,suppliers,employees,governmentEconomicsustainability(Third partyeffect)Social benchmarksfor ERRActual financialstatements, adjustedby economic pricesand valuesNeighbors,environmentEnvironment andsocial effects(Third partyeffects)WBG performancestandardsActual impact andcompliance withperformancestandardsCompetitors, newentrants, producersof complementaryproductsPrivate sectordevelopment(Third partyeffect)Objectives oncompetition,ownershipstructure,demonstrationeffectsQualitativeassessment ofachievements ofobjectives oncompetition,ownership structure,demonstration effects
    • 20. Track indicators anddevelopment outcomesSuperviseDOTS**IFC’s Development Outcome Tracking SystemApprove Disburse Repaid / ExitCommitIdentify indicatorsExpandedSupervisionReport (XPSR)4.4. M&E to adopt investmentproject cycle
    • 21. 21Development Outcome FrameworkFinancialPerformanceFinanciersEconomicPerformanceGov’t, taxpayersemployees,customers,suppliersEnvi. & SocialPerformanceNeighbors &EnvironmentPrivate SectorDevelopmentDemo. effects,linkages, corp.gov, investmentclimate & newentrants+Standard Development Reach IndicatorsIFC-wide: Taxes, employment (incl. female)Plus sector-specific: e.g. MSME loans, utility connections• Development Outcome: Overall effect on country’s development• Framework: Derived from Good Practice Standards for Private SectorEvaluation (investments)• Each with specific indicators, benchmarks3.2. Performance Assessment
    • 22. 2.2 Investment project M&EChallenges•Difficult to gather info beyondcompany”•Gaps in tracking private sectordevelopment outcome– Missing indicators, discrepanciesbetween stated outcome andindicators•Reliance on client-submitteddata– Some data based largely onassumptions•Financial Institutions –challenges in getting sub-project level informationAdvantages•Financial data- good qualityand frequently updated;•Enforcement via contracts /covenants•Integration with project cycle– Use of standard indicators, withbaseline and target•Quality control cycleestablished–Dept. review, Development ImpactDept review, External Assurance forAnnual Report
    • 23. Political Risk Guarantee Business modeland M&E challenges (MIGA)Business Model•Core competency asguarantee underwriting,not financing•Arms-length relationshipto project enterprises;•No statutory reportingrequirement from projects;•Constraints to access ––No membership on projectcompany boards of directorsChallengesIn data gatheringIn communicating withproject entitiesChallenges incontextual analysisIndustry / sectorintelligence23
    • 24. Function of M&E in MIGA:significant progress in intro of self-evaluation andindicators.Achievements•New monitoring systemfor Environment & Social•Mainstreaming self-evaluation•Development indicatortracking systemintroducedAreas of improvementChallenges in data gatheringInconsistent E&S follow-up onmissing documentsDEIS –ex-ante data notcovering all indicators.Pool of evaluation notadequate to report portfolioresults24
    • 25. Outline of Presentation1. Context of Private Sector Development Assistance2. Types of WBG Private Sector Developmentinterventions3. Technical Assistance projects M&E4. Private Sector Investment projects M&E5. Use and influence of M&E
    • 26. 3.1 Lessons can have a positive influence onresultsIFC Investment•Lack of lessons / application oflessons at appraisal associatedwith poor outcome– 80% of unsuccessful projects – lowquality at entry was main factor– Low DO with low quality at entry =40% of them without lessons60% of them lessons were notintegrated in design.•Improved performance bylearning from past failures(Agribusiness case study)– Newer successful projects appliedlessons from earlier failed projectsIFC Advisory Lessons influence design andimplementation New template asks “howlessons have been applied toproject design” Wealth of lessons from PCR Segmented categories of lessons High use of past project PCR innew projects Low use of “SmartLessons” M&E officers review at entry
    • 27. 3.2 Issues have been detected duringsupervision and actions have been takenIFC Investment•High quality of supervision work:83% high (XPSR)•Half of XPSR identified issuesduring supervision, 90% of themwere detected, 63% of themwere fully addressed duringsupervision.•Credit and Environment risksystems as the most frequentlyused monitoring systemsIFC Advisory PSR widely used 95% of staff used PSR tomake adjustment. 88% of supervision issuesidentified in PCR captured inPSR. Poor supervision critical factorof 25% of projects rated low inDE
    • 28. 3.3 M&E information is increasinglyused in IFC Strategies• Increasing use of M&E data• Annual Strategy process– Development impact Dept. provide data andparticipate discussion.– Guidance: asks development issues to cover• Ratings and reach data used in strategy formulation– Historical DOTS rating comparisons betweensectors – prioritization of subsector– DOTS ratings vs. profitability– Gap analysis using “reach indicators” – strategy tosharpen focus, areas to reach (i.e. BOP)– Development return of different options using reachdata (reach per $ cost of capital)
    • 29. Three main messages• Private Sector operations have specific datagathering advantage/disadvantage based ontheir business model• M&E systems should adopt to the businesspractices to be effective and efficient• M&E can influence learning, quality of workand outcome
    • 30. Outstanding issues• Difficult to estimate benefit of M&E–Need innovative approach, someassumptions about counterfactuals;
    • 31. Monitoring and Evaluation ofPrivate Sector DevelopmentHiroyuki HatashimaPrivate Sector Evaluation,Independent Evaluation Group (IEG)World Bank GroupCanadian Evaluation Society ConferenceJune 11, 2013

    ×