Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center; 200 Panlong Road-Shanghai, October 12, 2008

1. Diagnostic Study of IFIs ’ M&E in China Main Findings • Considerations • Observations 中国 IFIs 项目 M&E 诊断 研究 主要发现 • 思考 • 启示 Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center 200 Panlong Road-Shanghai, October 12, 2008 Chen Zhaoying

9. Project Document ● PAD ● ICR ● IEG-Review Checklists Opinion C onclusion Causality Logical model for Case Diagnosis Fact S1 Project M&E framework S2 Project M&E indicators S3 Outcome indicators S4 Design & implementation of M&E S5 Data source and collection S6 M&E conclusions S7 Role of M&E Field visit information ● Workshops ● Interview Relevant Information ● T he reports from Chinese side ● Others Suggestion www.dadangsolihin.com

10. 项目诊断研究 Project Diagnosis 1 Gansu and Inner Mongolia Poverty 扶贫 WB Reduction 2 Southwest Poverty Reduction Poverty Reduction WB 3 Tarim Basin II 水资源 WB 4 Wanjiazhai Water Transfer Water Resource WB 5 China-National Highway IV 交通 WB 6 Hebei Roads Development Transport ADB 7 Liaoning Urban Transport 城市发展 WB 8 Yunnan Environment Urban Development WB 诊断项目清单 List of Projects Selected for Case Diagnosis 项目名称 Project Name 领域 Sector IFIs www.dadangsolihin.com

11. 诊断研究的主要发现 世行的成功经验 Strengths of M&E in WB www.dadangsolihin.com ■ M&E 经验的本土化 Localized M&E experiences through practice in China ■ M&E 的制度化 Institutionalization of project M&E system in WB ■ M&E 在项目管理中的作用 Support to core activities of the project management ■ 结果导向的 M&E 在中国的进展 Results-based M&E is making progress in Chin ■ M&E 的动态改进 Localized M&E experiences and improvement of WB-China cooperation in M&E

12. 成功经验 Strengths M&E 的 制度化 Institutionalization of M&E www.dadangsolihin.com ■ M&E 制度和项目管理制度 M&E 制度是项目管理制度的重要组成部分 The institutional arrangements for project M&E are set up within WB. ■ M&E 和项目周期 M&E 嵌入项目周期的相应环节 . M&E 是制度化的活动，而不是额外负担 The success of project M&E in WB was largely due to its full integration into the project cycle ■ 自评价和独立评价 通过独立评价对自评价进行有效的质量控制，高质量的自评价为独立评价提供的重要支撑 Self-evaluation and independent evaluation link to one system to ensure institutional framework of evaluation well established in WB.

13. 成功经验 Strengths M&E 的 制度化 Institutionalization of M&E 世行的项目评价体系 Evaluation System in WB ■ 自评价系统 。项目经理作为实施自评价的责任主体，项目启动时对自评价的范围、内容、关键指标提出明确的要求，并在项目完工后六个月内完成项目的自评价报告 (ICR) 。 ■ 独立评价系统 。独立评估组对全部项目的自评价报告逐一进行审核，并根据需要选择部分项目进行实地考察和复评。独立评估组选择约四分之一的项目进行独立的绩效评价，并向董事会报告。 ■ 项目绩效评价是其他评价活动的基础 。在选择绩效评价的候选项目时，也要考虑它们与国家、行业和主题层面评价的关联性。 www.dadangsolihin.com

14. 诊断研究的主要发现 世行的成功经验 Strengths of M&E in WB www.dadangsolihin.com ■ M&E 经验的本土化 Localized M&E experiences through practice in China ■ M&E 的制度化 Institutionalization of project M&E system in WB ■ M&E 在项目管理中的作用 Support to core activities of the project management ■ 结果导向的 M&E 在中国的进展 Results-based M&E is making progress in Chin ■ M&E 的动态改进 Localized M&E experiences and improvement of WB-China cooperation in M&E

15. 成功经验 Strengths M&E 在管理中的作用 Role of M&E www.dadangsolihin.com ■ 对 总体绩效的 保证作用 世行在华项目的总体绩效明显高于世行项目的平均水平 , 有效的监测与评价是项目顺利实施和实现预定目标的重要保证 The overall performance of the WB’s projects in China is obviously above the average level of the WB projects ■ 过程监测的作用 几乎所有的项目在实施周期内都实施了比较规范的过程监测，其结果作为指导项目实施和调整项目执行的重要依据 Monitoring information has been used for all projects to improve project management and support project mid-term adjustment ■ 对实现预定目标的作用 M&E 对大规模、综合性项目的作用尤为突出 The role of M&E in large-scale and comprehensive projects is significant

16. 成功经验 Strengths M&E 在管理中的作用 Role of M&E 成功案例 : 西南扶贫项目实施的贫困独立监测 是中国扶贫领域中第一个采用贫困独立监测的扶贫项目 . 在该项目的影响下，国家统计局从 1997 年开始对国家 592 个贫困县实施贫困监测，并且每年出版《中国农村贫困监测报告》。世行通过该项目实施，促进了知识转移和经验本土化，并为其他行业建立监测评价系统提供了有价值的参考 。 Case: Southwest Poverty Reduction Project, M&E is satisfactory The project adopts the independent monitoring at the national, provincial and county level. It is the first time to adopt independent poverty monitoring. With the influence of this project, starting from 1997, the National Bureau of Statistics has conducted poverty monitoring on 592 poverty-stricken counties and released the Poverty Monitoring Report of Rural China every year. www.dadangsolihin.com

17. 诊断研究的主要发现 世行的成功经验 Strengths of M&E in WB www.dadangsolihin.com ■ M&E 经验的本土化 Localized M&E experiences through practice in China ■ M&E 的制度化 Institutionalization of project M&E system in WB ■ M&E 在项目管理中的作用 Support to core activities of the project management ■ 结果导向的 M&E 在中国的进展 Results-based M&E is making progress in Chin ■ M&E 的动态改进 Localized M&E experiences and improvement of WB-China cooperation in M&E

18. 成功经验 Strengths 结果导向的 M&E Results-based M&E www.dadangsolihin.com ■ 项目设计阶段 预期成效、关键成效指标、基线数据、信息来源和假设条件等是项目设计的重要内容 All projects are required to specify the expected outcomes in project design ■ 项目实施过程中 项目实施中，重点监测项目实现成效的进展，及时观察项目是否向着实现成效的正确方向进展。聘请国家统计局开展定期的贫困监测 Focus on intermediate outcomes ■ 项目完工后 项目完工后进行的评价活动中，项目的成效，即项目的目标实现程度是评价的重点 Specified outcomes were discussed in in ICR and in IEG Review ■ 目标群体 关注项目为目标群体带来的可以确定的或可以测量的变化，包括目标群体的满意度 Pay attention to measurable changes that the projects have brought to the target group

19. 诊断研究的主要发现 存在问题 Weaknesses of M&E www.dadangsolihin.com ■ 项目全周期的 M&E 制度缺位 Absence of M&E institutional arrangement after the project completion ■ 中方对 M&E 自主性尚待提高 Weaker ownership of M&E by Chinese side ■ 项目 M&E 对中方政府部门管理决策的影响十分有限 Lower level of utilization of M&E by the government departments in China ■ 项目 M&E 的指标存在的问题 Concern on some unsatisfactory aspects in project M&E indicators ■ 中方关于 M&E 的经费渠道不明确 lack of clear regulations on the M&E funding

20. 存在问题 Weaknesses M&E 制度 M&E institutional arrangement www.dadangsolihin.com ■ 项目周期 世行的项目周期从项目提出到完工 ; 中方的项目周期从项目提出到贷款偿还结束 For the WB, from the project initiation to its completion; for the Chinese side, from the project initiation to the finish of loan repayment ■ 项目实施周期的 M &E 世行项目 M&E 制度主要针对项目的实施过程；中方与世行合作开展相关的 M&E 活动 Within the WB a relatively complete M&E system has been formed during the project implementation period ■ 项目完工后的 M&E 项目完工后， M&E 活动缺乏制度安排 , 责任机制不明确 , 没有工作计划和常规预算。 It is not clear on the responsibilities for M&E. There is no action planning and regular budget for the M&E activities after the project completion

21. 存在问题 Weaknesses M&E 结果 Lower level of utilization of M&E www.dadangsolihin.com ■ 项目全周期的 M&E 制度缺位 Absence of M&E institutional arrangement after the project completion ■ 中方对 M&E 自主性尚待提高 Weaker ownership of M&E by Chinese side ■ 项目 M&E 对中方政府部门管理决策的影响十分有限 Lower level of utilization of M&E by the government departments in China ■ 项目 M&E 的指标存在的问题 Concern on some unsatisfactory aspects in project M&E indicators ■ 中方关于 M&E 的经费渠道不明确 lack of clear regulations on the M&E funding

22. 存在问题 Weaknesses M&E 结果 Lower level of utilization of M&E www.dadangsolihin.com ■ 对评估报告的了解 中方对世行项目监测的备忘录比较重视。项目办对评估报告有一定的了解 , 大多数政府部门对评估报告不了解 Majority of PMOs are familiar with WB evaluation reports, few of the government officials know about the reports ■ M&E 对项目管理的作用 M&E 对项目 期中调整、项目实施进度控制和保障项目目标实现等方面发挥重要作用 M&E has played a critical role in the project management, including mid-term adjustment and progress control ■ M&E 对政府管理决策发挥的作用 M&E 对政府的管理决策的影响十分有限 仅在个别案例中 M&E 为政府决策提供了依据 M&E has a limited influences on the decision-making of the governmental departments

23. 诊断研究的主要发现 存在问题 Weaknesses of M&E www.dadangsolihin.com ■ 项目全周期的 M&E 制度缺位 Absence of M&E institutional arrangement after the project completion ■ 中方对 M&E 自主性尚待提高 Weaker ownership of M&E by Chinese side ■ 项目 M&E 对中方政府部门管理决策的影响十分有限 Lower level of utilization of M&E by the government departments in China ■ 项目 M&E 的指标存在的问题 Concern on some unsatisfactory aspects in project M&E indicators ■ 中方关于 M&E 的经费渠道不明确 lack of clear regulations on the M&E funding

24. 存在问题 Weaknesses M&E 指标 Unsatisfactory aspects in M&E indicators www.dadangsolihin.com ■ 指标缺失 缺乏对项目还贷状况的监测与评价 WB does not take into consideration the issue of repaying the loan in M&E ■ 假设条件 涉及制度建设、机构改革及价格调整等因素的假设条件在项目周期内往往难以实现 Some assumption in M&E cannot be realized during the project cycle ■ 产出和成效 产出指标和成效指标还存在某些模糊认识 There are some of confusion between the outcome indicator and the output indicator ■ 指标数量 某些项目设计时确定了较多的指标，但在评价报告中并没有采用 , 或难以操作 The indicators can be further screened ■ 其他因素 项目经理的个人因素对指标设计影响较大 uneven quality of M&E design influenced by project task managers

25. 存在问题 Weaknesses M&E 指标 Unsatisfactory aspects in M&E indicators 案例 : 云南环境项目两个重要假设条件未能实现 ■ 对于提供城市供水、污水收集和处理服务的机构，必须以公司或企业的方式运作， 不依靠政府的财政补贴。 ■ 云南省政府应该提供优惠的价格政策来保证这些机构的收入可以满足项目的运行和维护支出。 Example: Assumptions cannot be realized in Urban Environment Project ■ The institution providing urban water supply and sewage processing service should be run in the way of a company, instead of depending on the government financial subsidy. ■ The government should adjust their water prices, so that the price can meet the needs of a balanced finance for the project. www.dadangsolihin.com

26. 思考和启示 Consideration and Observations ■ 现状 : 如何认识中国 M&E 系统的现状 ? What is Current Status of M&E system in China? ■ 下阶段的思考 : 如何起步并 确定工作重点 ? Where is the starting point in building the M&E system in China and how to identify the priority? ■ 挑战 : 国际规范与中国国情 I nternational Norms vs Chinese Contexts www.dadangsolihin.com

27. 思考和启示 Consideration and Observations M&E 的现状 Current Status of M&E 如何认识中国 M&E 系统的现状 ( 就绪度 ) Current Status of M&E system in China ( Readiness) 启示 中国应在 IFIs 项目的绩效评价中发挥主导作用 ; 目前 IFIs 项目的绩效评价最有条件进行试点 Strengthen the ownership of project M&E by Chinese side; IFLs activities as the pilot projects for building government M&E system in China www.dadangsolihin.com 需求和共识 Driving Force for M&E 制度建设 Institutional Context 实践经验 Good Practices 能力建设 实施评估活动和应用评估结果的能力 Capacity Building The capacity to do evaluation and use evaluation results

28. ■ 建立 M&E 系统从哪里起步 ? 项目层次 , 计划层次 , 部门层次还是国家层次 Where is the starting point in building M&E system: from project level, program level, ministry level or country level? ■ 如何确定重点 ? 建议 : 侧重项目完工后的 绩效评估研究和试点 Priority setting: focus on the performance assessment after project completion and begin with pilot efforts 思考和启示 Consideration and Observations 下阶段的思考 Next Steps www.dadangsolihin.com

29. IDF 项目 ( 后续 ) ■ 现状研究 对 IFIs 和中国相关部门的项目绩效评价的现状进行调查研究 Review the status of project performance assessment in IFIs and government departments in China ■ 操作指南 研究结果导向的项目绩效评价的框架和方法 , 起草《中国 IFIs 项目绩效评价操作指南》，对全国的绩效评价工作进行业务指导 Develop an approach of performance assessment for IFIs projects in China and prepare the Draft Guidelines of Performance Assessment for IFIs Projects in China ■ 项目试点 对所选项目进行绩效评价的试点 Conduct pilot works of performance assessment on selected projects 思考和启示 Consideration and Observations 下阶段的思考 Next Steps www.dadangsolihin.com

30. 挑战 : 关于知识的转移 M&E 作为 产生于西方的管理工具 , 是否适应中国的情况 ? 或者在多大程度上适应中国的情况 ? 我们遇到的问题和其他国家有哪些相同和不同之处 ? The Challenges of Transferring Management Tools from Western to China As the tool produced from a “Western” management tradition, whether the M&E is applicable, or applicable to the same extent in China context. Are the lessons from China are consistent with other countries’ experience? 思考和启示 国际规范与中国国情 I nternational Norms vs Chinese Context www.dadangsolihin.com

31. 挑战 : 关于评价框架 如何建立符合国际规范并适应和中国实际情况的项目绩效评的框架和方法 ? How to develop a framework of performance assessment for IFIs projects in China, which should be in line with international norms also received by Chinese stakeholders? ■ 评价框架 : 评价准则、关键问题、数据信息及其来源 目标相关性 (Objectives Relevance); 效率 (Efficiency) 效果 (Effectiveness); 可持续性 ( Sustainability) ■ 重点内容 : 将项目的成效作为绩效评价的重点内容 , 影响评价对于多数情况不作为重点 Take the outcomes as the first priority. for most cases, performance assessment could focus more on the outcomes than the impacts. 思考和启示 国际规范与中国国情 I nternational Norms vs Chinese Context www.dadangsolihin.com

32. 挑战 : 关于评价的结果的应用 Use the M&E results carefully ■ 项目绩效评价的分数或等级告诉我们什么？ What do the performance ratings tell us? 提示作用 当分数或等级不满意时 , 提示该项目可能是 ” 问题项目 ” , 需要进一步调查核实 预警作用 某些指标的分数或等级不满意时 , 可以帮助预警该项目存在潜在的问题或风险 ■ 评价结果是可比的吗？ Is it possible to compare the performance ratings among different government departments? 纵向比较 : 有利于经验总结和改进管理 横向比较 : 有局限性 , 避免将绩效评价的重点放在横向比较和追究责任上 思考和启示 国际规范与中国国情 I nternational Norms vs Chinese Context www.dadangsolihin.com

33. 关于不同层次的绩效评价 Evaluation in various levels, such as project, sector, local and country ■ 行业评价是否等同于该行业的所有项目评价结果叠加？ Is a sector-level evaluation any different from evaluating all the projects in a sector and adding up the results? ■ “ 自下而上 ” 的评价结果和 “ 自上而下 ” 的评价结果有什么不同 ? The “bottom up”and “bottom up” evaluation ？ 思考和启示 国际规范与中国国情 I nternational Norms vs Chinese Context www.dadangsolihin.com 国家 项目 计划 部门 / 行业 地区