Independent evaluation at the ADB 亚行的独立评价

2,402 views

Published on

Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center; 200 Panlong Road-Shanghai, October 12, 2008

Published in: Education, Business, Technology
1 Comment
2 Likes
Statistics
Notes
No Downloads
Views
Total views
2,402
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide
  • Independent evaluation at the ADB 亚行的独立评价

    1. 1. Independent evaluation at the ADB 亚行的独立评价 Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center 200 Panlong Road-Shanghai, October 12, 2008 J. Scott Bayley
    2. 2. <ul><li>J. Scott Bayley </li></ul><ul><li>Evaluation Specialist - Operations Evaluation Department Asian Development Bank </li></ul><ul><li>6 ADB Avenue, Mandaluyong City </li></ul><ul><li>1550 Metro Manila, Philipines </li></ul><ul><li>sbayley@adb.org; www.adb.org /evaluation </li></ul>Dadang Solihin Indonesia Delegation Shanghai International Program for Development Evaluation Training
    3. 3. Overview 概览 <ul><li>About the Operations Evaluation Department (OED) </li></ul><ul><li>关于 业务评价局 (OED) </li></ul><ul><li>2. OED’s methodological approach </li></ul><ul><li>OED 使用的方法 </li></ul><ul><li>3. Evaluating the Shanghai-Nanpu bridge project </li></ul><ul><li>评价上海南浦大桥项目 </li></ul><ul><li>4. Further information – OED website </li></ul><ul><li>其他信息 – OED 网站 </li></ul>www.dadangsolihin.com
    4. 4. About OED 关于 OED <ul><li>ADB’s independent evaluation service </li></ul><ul><li>Accountable to the Board of Directors – Development Effectiveness Committee </li></ul><ul><li>Undertakes evaluation studies of ADB operations to draw lessons to improve future operations </li></ul><ul><li>Disseminates lessons to a wide audience – the Board, ADB operations, DMCs, other MDBs, and public in general </li></ul><ul><li>OED 是亚行的独立评价服务部门 </li></ul><ul><li>主管部门是亚行董事会,即发展有效性委员会 </li></ul><ul><li>承担亚行业务的评价研究,目的是吸取教训以便改善未来的业务 </li></ul><ul><li>将经验教训传播给更多的受众:董事会、亚行业务部、亚行发展中成员国、其他多边发展银行和公众 </li></ul>www.dadangsolihin.com
    5. 5. Evaluation’s Mission 评价的使命 <ul><li>To help ADB become a learning organization that continuously improves its development effectiveness and is accountable to its stakeholders. </li></ul><ul><li>让亚行成为持续改善发展有效性并对其相关利益人负责的学习组织。 </li></ul>www.dadangsolihin.com
    6. 6. <ul><li>Accountability , by assessing the effectiveness of ADB's operations. </li></ul><ul><li>Transparency , by independently reviewing operations and publicly reporting findings and recommendations. </li></ul><ul><li>Improved Performance , by helping ADB and its clients learn from past experience to enhance ongoing and future operations. </li></ul>OED supports three elements of good governance: OED 支持完善治理的三个要素: <ul><li>可靠 :通过评估亚行业务的有效性来实现。 </li></ul><ul><li>透明 :通过评审业务和公开其发现和建议来实现。 </li></ul><ul><li>改善绩效 :通过帮助亚行及其客户学习过去的经验,以提高目前和将来的业务工作。 </li></ul>www.dadangsolihin.com
    7. 7. Facts about OED 关于 OED 的情况 <ul><li>1978 – Postevaluation Office established </li></ul><ul><li>2001 – became Operations Evaluation Department </li></ul><ul><li>2004 – OED now independent, reporting directly to the Board through the Development Effectiveness Committee </li></ul><ul><li>24 professional staff with country and sector responsibilities </li></ul><ul><li>1978 – 项目后评价办公室建立 </li></ul><ul><li>2001 – 成为 业务评价局( OED ) </li></ul><ul><li>2004 –OED 现在是独立的,直接通过发展有效性委员会向董事会报告 </li></ul><ul><li>24 位专业人员负责不同国家和部门的业务 </li></ul>www.dadangsolihin.com
    8. 8. Evaluation Feedback Loop 评价反馈循环图 ADB Operations Departments (monitoring and evaluation, results agenda, applied learning) DMCs (use of evaluations, evaluation capacity development, joint evaluations) International Evaluation Community (harmonization and joint evaluations) ADB Management (use and follow-up of evaluations, influence on ADB directions) ADB Board of Directors (oversight) OED (evaluations, monitoring and evaluation, expertise self-assessment) ADB 业务部门 ( 监督和评价、结果日程、应用型学习 DMC ( 评价使用、评级能力发展、联合评价 ) 亚行管理部 ( 评价的使用和跟踪、对亚行指导的影响) 国际评价社区 ( 协调和联合评价) 亚行董事会 ( 监管 ) OED ( 评价、监督和评价、专家自我评估)
    9. 9. Audiences for Evaluations 关注评价的群体 Primary Audiences for Accountability 主要关注可靠性的群体 Primary Audiences for Learning 主要关注学习的 群体 Media / Civil Society Organizations 媒体 / 民间组织 Parliaments / National Audit Offices 议会 / 国家审计部门 Executive Branches (finance, external relations, trade, etc.) 行政部门 ( 金融、外交、贸易等) Other Development Agencies (including NGOs) 其他发展机构 ( 包括非政府组织 ) Board of Directors / Management 董事会 / 管理部门 Departments and Offices 各部门和办公室 Resident Missions 代表处 Operations Staff in Other Development Agencies 其他发展机构中的业务人员 Operations Staff in Agencies Implementing ADB Funded Projects 执行亚行资助项目的机构中的业务人员 Policy Makers in Partner Countries / Other In-country Development Agencies 合作国的政策制定者 / 其他国内的发展机构 Partner Governments 合作政府 Media and Civil Society Organizations 媒体和民间组织 Audiences in Donor Countries 捐助国中的群体 Audiences in Partner Countries 合作国中的群体 Audiences in ADB 亚行中的群体
    10. 10. OED’s Main Evaluation Products OED 的主要评价产品 <ul><li>Annual Evaluation Review </li></ul><ul><li>Annual Report on Portfolio Performance </li></ul><ul><li>Project Performance Evaluation Report </li></ul><ul><li>Technical Assistance Performance Evaluation Report </li></ul><ul><li>Project Completion Report Validation </li></ul><ul><li>Country Assistance Program Evaluation </li></ul><ul><li>Sector Assistance Program Evaluation </li></ul><ul><li>Special Evaluation Study </li></ul><ul><li>Impact Evaluation Study </li></ul><ul><li>Country Partnership Strategy Completion Report Validation </li></ul><ul><li>Evaluation Updates </li></ul><ul><li>Evaluation Briefs </li></ul>Broader evaluation studies 更广泛的评价研究 Evaluations of projects 项目评价 Annual reports 年度报告 <ul><li>国家帮助项目评价 </li></ul><ul><li>部门帮助项目评价 </li></ul><ul><li>特殊评价研究 </li></ul><ul><li>影响评价研究 </li></ul><ul><li>国家合作战略执行报告验证 </li></ul><ul><li>评价信息更新 </li></ul><ul><li>评价简报 </li></ul><ul><li>项目绩效评价报告 </li></ul><ul><li>技术支持绩效报告 </li></ul><ul><li>项目执行报告验证 </li></ul><ul><li>年度评价概况 </li></ul><ul><li>综合绩效年度报告 </li></ul>www.dadangsolihin.com
    11. 11. Evaluation Results at ADB 亚行的评价结果 <ul><li>As of 31 December 2007, 1,212 sovereign projects and programs had been independently or self-evaluated: </li></ul><ul><ul><li>63% were rated highly successful, or (generally) successful </li></ul></ul><ul><ul><li>29% were rated partly successful; and </li></ul></ul><ul><ul><li>8% were rated unsuccessful </li></ul></ul><ul><li>截至 2007 年 12 月 31 日,有 1212 个主权项目已经进行了独立评价或自我评价: </li></ul><ul><ul><li>63% 的项目被评定为非常成功,或(大体)成功 </li></ul></ul><ul><ul><li>29% 的项目被评定为部分成功 </li></ul></ul><ul><ul><li>8% 的项目被评定为不成功 </li></ul></ul>www.dadangsolihin.com
    12. 12. Who Evaluates What? 谁来评价?评价什么? Evaluation 评价 Manage- Ment 管理 Design 设计 Post Evaluation 评价后 Monitoring Reporting 监督报告 OD/OED OED OD OD/OED OD = ADB Operational Departments, OED = Operations Evaluation Department OD = ADB 业务部门 , OED = 业务评价部门 www.dadangsolihin.com Design and Monitoring Framework 设计和监督框架 Development Results 发展结果 Policies, Business Processes, Country, Sector, Project, 政策、业务流程、国家、部门、项目
    13. 13. Evaluation Guidelines 评价指导纲要 <ul><li>Preparation of Public Sector Project/Program Evaluation Report </li></ul><ul><li>Preparation for Country Assistance Program Evaluation Report </li></ul><ul><li>Evaluation Framework for Private Sector Operations </li></ul><ul><li>Validation reports </li></ul><ul><li>准备公共部门项目 / 计划评价报告 </li></ul><ul><li>准备国家援助项目评价报告 </li></ul><ul><li>准备私营部门业务评价框架 </li></ul><ul><li>验证报告 </li></ul>OED Guidelines at http://www.adb.org/Evaluation/about/methods.asp www.dadangsolihin.com
    14. 14. Four Criteria are Used to Rate Public Sector Projects 评价公共部门项目的四个标准 <ul><li>Relevance 相关性 </li></ul><ul><li>Effectiveness 有效性 </li></ul><ul><li>Efficiency 效率 </li></ul><ul><li>Sustainability 可持续性 </li></ul>www.dadangsolihin.com
    15. 15. Ratings & the Results Chain 评级与结果链 Outputs 产出 Outcome 结果 Impact 影响 Activities 活动 Inputs 投入 www.dadangsolihin.com
    16. 16. The Results Chain Explained 结果链解释 www.dadangsolihin.com Relevance 相关性 Efficiency 效率 Effectiveness 有效性 Sustainability 可持续性 Needs 需求 Objective 目的 Inputs 投入 Activities 活动 Outputs 产出 Outcome 结果 Impact 影响
    17. 17. Shanghai-Nanpu bridge project 上海南浦大桥项目 <ul><li>OED’s evaluation approach: </li></ul><ul><li>Preparation of an evaluation approach paper </li></ul><ul><li>Peer review, revision, approval to proceed </li></ul><ul><li>In-house preparation prior to field mission </li></ul><ul><li>OED 的评价方法: </li></ul><ul><li>评价方法文件的准备 </li></ul><ul><li>同行评审、修改、通过 </li></ul><ul><li>实地考察前的内部准备 </li></ul>www.dadangsolihin.com
    18. 18. Shanghai-Nanpu bridge project… 上海南浦大桥项目… <ul><li>OED’s evaluation approach: </li></ul><ul><li>Field mission </li></ul><ul><li>Presentation of preliminary findings </li></ul><ul><li>Preparation of 1 st draft report and peer review </li></ul><ul><li>OED 的评价方法: </li></ul><ul><li>实地考察 </li></ul><ul><li>展示初步成果 </li></ul><ul><li>准备第一稿报告草案并进行同行评审 </li></ul>www.dadangsolihin.com
    19. 19. Shanghai-Nanpu bridge project… 上海南浦大桥项目… <ul><li>OED’s evaluation approach: </li></ul><ul><li>Preparation of 2 nd draft report and circulation to ADB departments and borrower </li></ul><ul><li>Finalization of the report </li></ul><ul><li>Circulation of final report to ADB Board, management, borrower, and made available to the public. </li></ul><ul><li>OED 的评价方法: </li></ul><ul><li>准备第二稿报告草案并在亚行各部门和借方中传阅 </li></ul><ul><li>报告定稿 </li></ul><ul><li>报告终稿在亚行董事会、管理部门和借方中传阅,并公布给民众 </li></ul>www.dadangsolihin.com
    20. 20. Shanghai-Nanpu bridge project… 上海南浦大桥项目… <ul><li>Project background: </li></ul><ul><li>Project formulated in accordance with the Government’s 8 th 5-year plan (1991-1995) </li></ul><ul><li>Objective was to expand transport capacity across the Huangpu River and thus facilitate the development of Pudong New Area on the east bank </li></ul><ul><li>项目背景: </li></ul><ul><li>依据中国政府的第八个五年计划而成立 </li></ul><ul><li>目的是提高黄浦江两岸的交通能力,并借此促进东岸浦东新区的发展 </li></ul>www.dadangsolihin.com
    21. 21. Shanghai-Nanpu bridge project… 上海南浦大桥项目… <ul><li>Project background: </li></ul><ul><li>Constructing an 846 meter toll bridge, road approaches and related facilities, relocation of housing and businesses, operational and management support </li></ul><ul><li>Final project cost was $227 million, ADB provided a loan of $70m plus TA of $1.8m </li></ul><ul><li>Project completed in Dec 1992 </li></ul><ul><li>项目背景: </li></ul><ul><li>建设 846 米长的收费桥、道路入口和相关设施,住房和商业的拆迁,运营和管理的支持。 </li></ul><ul><li>最终项目成本是 2.27 亿美元,亚行提供了 7 千万美元的贷款和 180 万美元的技术援助。 </li></ul><ul><li>项目于 1992 年 12 月完成。 </li></ul>www.dadangsolihin.com
    22. 22. Shanghai-Nanpu bridge project… 上海南浦大桥项目… <ul><li>Evaluation methodology (1999): </li></ul><ul><li>评价方法论 (1999) : </li></ul><ul><li>http://www.adb.org/Documents/PERs/pe-529.pdf </li></ul><ul><li>Review of project documents </li></ul><ul><li>Employment of a consultant engineer </li></ul><ul><li>Site visit – 20 days </li></ul><ul><li>Examination of project costs, timelines, functionality </li></ul><ul><li>项目文件的审核 </li></ul><ul><li>招聘咨询工程师 </li></ul><ul><li>20 天的实地考察 </li></ul><ul><li>项目成本、时间和功效的审核 </li></ul>www.dadangsolihin.com
    23. 23. Shanghai-Nanpu bridge project… 上海南浦大桥项目… <ul><li>methodology: </li></ul><ul><li>Financial profitability analysis </li></ul><ul><li>Economic internal rate of return analysis </li></ul><ul><li>Interviews with government officials </li></ul><ul><li>Community consultations </li></ul><ul><li>Implementation, operational, financial, and economic analysis  social? environmental? </li></ul><ul><li>方法论: </li></ul><ul><li>分析财政盈利性 </li></ul><ul><li>分析内部经济回报率 </li></ul><ul><li>采访政府官员 </li></ul><ul><li>咨询社区公众 </li></ul><ul><li>执行、运作、财政和经济方面的分析  社会方面 ? 环境方面 ? </li></ul>www.dadangsolihin.com
    24. 24. Shanghai-Nanpu bridge project… 上海南浦大桥项目… <ul><li>Evaluation findings: </li></ul><ul><li>Bridge was properly designed </li></ul><ul><li>Construction was completed ahead of schedule and in accordance with the design specifications and within budget </li></ul><ul><li>Actual traffic volumes & savings in time and vehicle costs have exceeded forecasts </li></ul><ul><li>评价结果: </li></ul><ul><li>桥的设计合理 </li></ul><ul><li>提前竣工,符合设计规格要求,而且在预算之内 </li></ul><ul><li>实际交通流量以及节省的时间和车辆成本都超过预期 </li></ul>www.dadangsolihin.com
    25. 25. Shanghai-Nanpu bridge project… 上海南浦大桥项目… <ul><li>findings: </li></ul><ul><li>Bridge operation and maintenance is satisfactory </li></ul><ul><li>Resettlement of 4,200 households and 130 businesses was carried out efficiently and satisfactorily </li></ul><ul><li>Environment: bridge had no effect on river flow; problems with air and noise pollution </li></ul><ul><li>评价结果: </li></ul><ul><li>桥梁管理和维护令人满意 </li></ul><ul><li>圆满高效地完成了 4200 户居民和 130 家企业的动迁安置工作 </li></ul><ul><li>环境:桥梁对河流没有影响;空气污染和噪音污染问题 </li></ul>www.dadangsolihin.com
    26. 26. Shanghai-Nanpu bridge project… 上海南浦大桥项目… <ul><li>findings: </li></ul><ul><li>The general appearance of the bridge meets with community approval, e.g. flower boxes, decorative lighting </li></ul><ul><li>Bridge revenues are more than adequate to cover operating expenses and maintenance costs </li></ul><ul><li>Estimated economic internal rate of return = 16.4% </li></ul><ul><li>评价结果: </li></ul><ul><li>桥的总体外观,如花坛、装饰灯,得到了社区公众的认可。 </li></ul><ul><li>桥梁收入足以支付营运费用和维护成本 </li></ul><ul><li>估计内部经济回报率为 16.4% </li></ul>www.dadangsolihin.com
    27. 27. Shanghai-Nanpu bridge project… 上海南浦大桥项目… <ul><li>A very successful project! </li></ul><ul><li>是一个非常成功的项目! </li></ul>www.dadangsolihin.com
    28. 28. Evaluation Website ADB 评价网站: http://www.adb.org/evaluation www.dadangsolihin.com
    29. 29. Knowledge Products 知识产品 www.dadangsolihin.com
    30. 30. Evaluation Information System (EVIS) 评价信息系统 www.dadangsolihin.com
    31. 31. Thank you! 谢谢!
    32. 32. www.dadangsolihin.com <ul><li>Beside working as Assistant Professor at Graduate School of Asia-Pacific Studies, Waseda University, Tokyo, Japan, he also active as Associate Professor at University of Darma Persada, Jakarta, Indonesia. </li></ul><ul><li>He got various training around the globe, included Shanghai International Program for Development Evaluation Training (2008) , Public Officials Capacity Building Training Program for Government Innovation, Seoul –Korea (2007), Advanced International Training Programme of Information Technology Management, at Karlstad City, Sweden (2005); the Training Seminar on Land Use and Management, Taiwan (2004); Developing Multimedia Applications for Managers, Kuala Lumpur, Malaysia (2003); Applied Policy Development Training, Vancouver, Canada (2002); Local Government Administration Training Course, Hiroshima, Japan (2001); and Regional Development and Planning Training Course, Sapporo, Japan (1999). He published more than five books regarding local autonomous. </li></ul><ul><li>You can reach Dadang Solihin by email at dadangsol@yahoo.com or by his mobile at +62812 932 2202 </li></ul>Dadang Solihin currently is Director for Regional D evelopment Performance Evaluation at Indonesian National Development Planning Agency (Bappenas). He holds MA degree in Economics from University of Colorado, USA. His previous post is Director for System and Reporting of Development Performance Evaluation at Bappenas. Dadang Solihin’s Profile

    ×