• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统
 

Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统

on

  • 2,961 views

Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center; 200 Panlong Road-Shanghai, October 10, 2008

Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center; 200 Panlong Road-Shanghai, October 10, 2008

Statistics

Views

Total Views
2,961
Views on SlideShare
2,954
Embed Views
7

Actions

Likes
6
Downloads
0
Comments
2

1 Embed 7

http://www.slideshare.net 7

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统 Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统 Presentation Transcript

  • Building a Results-Based Monitoring and Evaluation System 建立面向结果的监测与评价系统 Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center 200 Panlong Road-Shanghai, October 10, 2008 Ray C. Rist
    • Ray C. Rist
    • Knowledge & Evaluation Capacity Development Advisor, Independent Evaluation Group of the World Bank
    • President International Development Evaluation Association (IDEAS)
    • rrist@worldbank.org;
    • www.worldbank.org/ieg
    Dadang Solihin Indonesia Delegation Shanghai International Program for Development Evaluation Training
  • Introduction 引言
    • Importance of Results-Based M&E
    • 面向结果的监测与评价的重要性
    • What Is Results-Based M&E?
    • 什么是面向结果的监测与评价?
    • Traditional vs. Results-Based M&E
    • 传统的监测与评价和面向结果的监测与评价的比较
    • Ten Steps to Building Results-Based M&E
    • 面向结果的监测与评价系统的十个步骤
    www.dadangsolihin.com
  • Importance 重要性
    • Growing pressures in developing countries to improve performance of public sectors
    • 发展中国家面临着越来越大的压力,要求其改善公共部门绩效
    • Tracks results of governments and other entities over time
    • 跟踪政府和其它实体的结果随时间的变化情况
    • -budgeting, human resources 预算,人力资源
    • organizational culture 组织文化
    • Is a management tool
    • 是一种管理工具
    www.dadangsolihin.com
  • www.dadangsolihin.com Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • The Power of Measuring Results 结果测算的效力
    • If you do not measure results, you can not tell success from failure
    • 如果不测算结果,就无法区分成功与失败
    • If you can not see success, you can not reward it
    • 不能预见成功,就无法对其进行奖励
    • If you can not reward success, you are probably rewarding failure
    • 不对成功进行奖励,就有可能奖励失败
    • If you can not see success, you can not learn from it
    • 不能预见成功,就无法从中汲取经验
    • If you can not recognize failure, you can not correct it
    • 没有认识到失败,就无法对其进行纠正
    • If you can demonstrate results, you can win public support
    • 如果能够证明结果,就能赢得公众支持
    www.dadangsolihin.com
  • Reasons to Do R-B M&E 实施面向结果的监测与评价的原因
    • Provides crucial information about public sector performance
    • 提供关于公共部门绩效的重要信息
    • Provides a view over time on the status of a project, program, or policy
    • 说明项目、计划或政策状态随时间变化的情况
    • Promotes credibility and public confidence by reporting on the results of programs
    • 通过报告计划的结果提高可信度、增强公众信心
    • Helps formulate and justify budget requests
    • 帮助制定和论证预算要求
    • Identifies potentially promising programs or practices by studying pilots
    • 通过研究试点发现具有潜力的计划或做法
    • (continued on next slide)
    • ( 转下页 )
    www.dadangsolihin.com
  • Reasons: (cont.) 原因 : ( 续 )
    • Focuses attention on achieving outcomes important to the organization and its stakeholders
    • 关注对组织和利益相关者具有重要意义的成果
    • Provides timely, frequent information to staff
    • 向工作人员提供及时而频繁的信息
    • Helps establish key goals and objectives
    • 帮助制定关键的长期目标和短期目标
    • Permits managers to identify and take action to correct weaknesses
    • 允许管理者发现并改正不足
    • Supports a development agenda that is shifting towards greater accountability for aid lending
    • 支持建立更为强大的援助贷款责任机制发展议程
    www.dadangsolihin.com
  • Results-Based Monitoring 面向结果的监测
    • Results-based monitoring (what we call “monitoring”) is a continuous process of collecting and analyzing information to compar ing actual results to expected results in order to measure how well a project, program, or policy is being implemented
    • 面向结果的监测(我们称之为“监测”)是一个收集和分析信息,从而将项目、计划或政策的实施情况与预期结果进行比较的 持续过程 。
    www.dadangsolihin.com
  • Results-Based Evaluation 面向结果的评价
    • Results-based evaluation is an assessment of a planned, ongoing, or completed intervention to determine its relevance, efficiency, effectiveness, impact, and sustainability
    • 面向结果的评价是对一项计划中的、正在实施的或已经完成的干预措施进行的评价,从而判定其相关性、效率、效果、影响和可持续性。
    www.dadangsolihin.com
  • Difference between Results-Based Monitoring and Evaluation 面向结果的监测和评价之间的区别
    • Monitoring is focused on tracking evidence of movement towards the achievement of specific, predetermined targets
    • 监测注重于跟踪取得预定的具体目标的进展证据
    • Evaluation takes a broader view of an intervention, considering not only progress toward stated goals, but also the logic of the initiative, as well as its consequences.
    • 评价对干预措施进行更广泛的考量,不仅考虑实现预定目标的进展情况,还考虑措施的逻辑以及其后果
    • Both are needed to be able to better manage policy, program, and project 两者都是为了更好地管理政策、计划和项目所必需的措施
    www.dadangsolihin.com
  • Comparison 比较
    • Traditional
    • 传统的监测与评价
    • inputs
    • 投入
    • activities
    • 活动
    • outputs
    • 产出
    • Results-Based
    • 面向结果的监测与评价
    • combines traditional with assessment results
    • 将传统的监测与评价和评价结果相结合
    • allows organization to modify and make adjustments to implementation processes
    • 允许组织对实施过程进行修改和调整
    www.dadangsolihin.com
  • Complementary Roles of and Monitoring Evaluation 监测与评价的互补作用
    • Monitoring 监测
    • Clarifies program objectives
    • 明确计划目标
    • Links activities and their resources to objectives
    • 建立各项活动及其资源投入与目标之间的联系
    • Translates objectives into performance indicators and set targets
    • 将目标转化为绩效指标并设定具体目标
    • Routinely collects data on these indicators, compares actual results with targets
    • 定期收集有关这些指标的数据,比较实际结果和具体目标
    • Reports progress to managers and alerts them to problems
    • 向管理者报告进展情况并提醒其注意出现的问题
    • Evaluation 评价
    • Analyzes why intended results were or were not achieved
    • 分析预期结果实现或未实现的原因
    • Assesses specific causal contributions of activities to results
    • 评价各项活动对结果产生的具有因果关系的特定作用
    • Examines implementation process
    • 检查实施过程
    •  Explores unintended results
    • 研究超出预期的结果
    •  Provides lessons, highlights significant accomplishment or program potential, and offers recommendations for improvement
    • 总结经验教训,突出重大成就或计划潜力,提出改进建议
    www.dadangsolihin.com
  • Brief Introduction to Theory of Change 变革理论:简介
    • Theory of change is a representation of how an organization or initiative is expected to lead to the results and identification of underlying assumptions being made
    • 变革理论代变现了一个组织或一项倡议是如何产生结果并明确重要假设的
    www.dadangsolihin.com
  • Components of Theory of Change 变革理论:组成
    • Activities 活动 – what we do 我们所作的
    • Outputs 产出– what we produce 我们生产的
    • Results 结果 – why we do it 我们为什么要做
    • Other optional components 其他选择组成
      • Inputs 投入 - what resources are used 使用了什么资源
      • Reach or Target groups – who the target is directed towards
      • Internal/External Factors – factors within and outside control or influence
      • 目标团体——目标是面向指向谁的
      • 内部 / 外部因素——在控制或影响之内和之外的因素
    www.dadangsolihin.com
  • Key Types of Monitoring 监测的主要类型 www.dadangsolihin.com Results Monitoring 结果监测 Implementation Monitoring (Means and Strategies) 实施情况监测 ( 手段和策略 ) Outcomes 成效 Impacts 影响 Results 结果 Inputs 投入 Activities 活动 Outputs 产出 Implementation 实施情况
  • Indicators 指标
    • A variable that allows the verification of changes in the development
    • 能够确认发展中变化的变量
    • intervention or shows results relative to what was planned
    • 干预活动或者显示相对于计划的结果
    • Examine the cumulative evidence of a cluster of indicators to see if a program is making progress
    • 检查一群指标的集合证据以此来判断一个项目是否有进展
    www.dadangsolihin.com
  • Ten Steps to Building a Results-Based M&E System 建立面向结果的监测与评价系统的十个步骤
  • Step 1: Conducting a Readiness Assessment 步骤 1: 实施就绪度评价 Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • What Is a Readiness Assessment? 什么是就绪度评价?
    • An analytical framework to assess a country’s ability to monitor and evaluate its development goals
    • 评价某国发展目标监测与评价能力的分析框架
    www.dadangsolihin.com
  • Incentives 激励机制
    • Sort out the answers to these questions :
    • 弄清楚这些问题的答案:
      • What is driving the need for building an M&E system?
      • 是什么正在推动建立监测与评价系统的需求?
      • Who are the champions for building and using an M&E system?
      • 谁是建立和使用监测与评价系统的拥护者?
      • What is motivating those who champion building and M&E system?
      • 是什么在激励那些拥护建立和使用监测与评价系统的人?
      • Who will benefit from the system?
      • 谁将会从该系统中受益?
      • Who will not benefit?
      • 谁不会从该系统中受益?
    www.dadangsolihin.com
  • Roles and Responsibilities 作用与责任
    • Ask:
    • 问题:
      • What are the roles of central and line ministries in assessing performance?
      • 中央部门和行业主管部门在评价绩效中的作用是什么?
      • What is the role of parliament?
      • 议会的作用是什么?
      • What is the role of the supreme audit agency?
      • 最高审计机构的作用是什么?
      • Do ministries and agencies share information with one another?
      • 政府部门和机构之间是否能实现信息共享?
      • Is there a political agenda behind the data produced?
      • 所提交的数据后面是否有一个政策议案?
      • Who in the country produces data?
      • 谁负责提交数据?
      • Where in the government are data used?
      • 政府内的那些部门会使用这些数据?
    www.dadangsolihin.com
  • Organizational Capacity 组织能力
    • Assess current capacity to monitor and evaluate:
    • 评价目前 实施监测与评价的能力 :
      • Technical skills
      • 技术能力
      • Managerial skills
      • 管理技能
      • Existing data systems and their quality
      • 现有的数据系统及其质量
      • Technology available
      • 可获得的技术
      • Fiscal resources available
      • 可获得的财政资源
      • Institutional experience
      • 机构经验
    www.dadangsolihin.com
  • Barriers to M&E 实施监测与评价的障碍
    • Do any of these immediate barriers to building and M&E system now exist?
    • 影响监测与评价系统建立的下述直接障碍是否依然存在?
      • Lack of fiscal resources
      • 缺乏财政资源
      • Lack of political will
      • 缺乏政治意愿
      • Lack of champion
      • 缺乏拥护者
      • Lack of expertise & knowledge
      • 缺乏专业技能和知识
      • Lack of strategy
      • 缺乏战略
      • Lack of prior experience
      • 缺乏经验
    • How do we confront these barriers?
    • 我们应该如何消除这些障碍?
    www.dadangsolihin.com
  • Key Questions for Predicting Success 用于预测成功的关键问题
    • Does a clear mandate exist for M&E?
      • PRSP?, Law? Civil Society? Other?
    • 是否存在一个针对监测与评价的清晰的授权?
      • 减贫战略文件( PRSP )、法律、公民社会、或其它?
    • Is there the presence of strong leadership at the most senior level of the government?
    • 在政府最高层中是否有强有力的领导?
    • Are resource and policy decisions linked to the budget?
    • 资源和政策决定是否与预算联系起来?
    • How reliable is information that may be used for policy and management decision making?
    • 可以用于政策制定和管理决策的信息的可靠性如何?
    • How involved is civil society as a partner with government?
    • 公民社会作为政府的伙伴是如何参与进来的?
    • Are there pockets of innovation that can serve as beginning practices or pilot programs?
    • 是否存在可作为启动实践或试点计划的创新 ?
    www.dadangsolihin.com
  • Step 2: Agreeing on Performance Outcomes to Monitor and Evaluate 步骤 2: 对要监测与评价的成效形成一致意见 Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • Why an Emphasis on Outcomes? 为什么要强调成效
    • Makes explicit the intended objectives of government action
      • (“Know where you are going before you get moving”)
    • 明确政府行动的预期目标
      • (“ 行动之前应该先明确目标” )
    • Outcomes are what produce benefits
    • 成效带来效益
    • Clearly setting outcomes is key to designing and building results-based M&E system
    • 设定清晰的成效对于设计和建立面向结果的监测与评价系统至关重要
    • Important! Budget to outputs, manage to outcomes!
    • 重要 ! 针对产出进行预算,针对成效进行管理!
    www.dadangsolihin.com
  • Developing Outcomes for One Policy Area: Education 政策领域成效设定:以教育为例 www.dadangsolihin.com 2. Primary school learning outcomes for children are improved 2. 提高儿童小学学习的成效 1. Nations children have improved access to pre-school programs 1. 使全国有更多的儿童加入学前教育计划 Targets: 具体目标 : Baselines: 基准 : Indicators: 指标 : Outcomes: 成效 :
  • Outcomes: 成效 :
    • Outcomes are usually not directly measured — only reported on
    • 通常无法对成效进行直接测量——只须报告
    • Outcomes must be translated to a set of key indicators
    • 必须将成效转化为一组关键指标
    • When choosing outcomes, “Do not go it alone!” – agreement is crucial
    • 当选择成效时,“切忌一意孤行!”——取得一致意见至关重要
    www.dadangsolihin.com
  • Step 3: Selecting Key Indicators to Monitor Outcomes 步骤 3: 选择监测成效的关键指标 Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • Indicators 指标
    • A specific measure (or set of data) that indicates progress (or not) toward a specific target
    • 用来表明特定目标实现进展(或偏离情况)的一种具体的测量方法(或一组数据)
    • Indicators asks the question:
      • How will we know success when we see it?
    • 指标提出这样一个问题:
      • 当我们看到成功时如何得知这就是成功?
    www.dadangsolihin.com
  • Indicator Development 指标开发
    • “ CREAM”
      • Clear
      • 清晰
      • Relevant
      • 相关
      • Economic
      • 经济
      • Adequate
      • 充足
      • Monitorable
      • 可监测
    www.dadangsolihin.com
  • Matrix for Selecting Indicators 指标选择矩阵 Indicator 指标 Data collection method 数据收集方法 Frequency & cost to collect 收集频率和成本 Who will analyze and report data? 谁负责分析和报告数据 Who will collect data? 谁负责收集数据 Difficulty to collect 收集的难度 www.dadangsolihin.com Who will use the data? 谁将使用数据 Data source 数据来源 1. 2. 3.                                          
  • Developing Set of Outcome Indicators for One Policy Area: Education 为教育,这一政策领域开发一套成效指标 www.dadangsolihin.com 1.% of Grade 6 students scoring70% or better on standardized math and science tests 1. 六年级学生在标准化的数学和自然科学考试中得到 70 分或以上成绩的人数比例 2. % of Grade 6 students scoring higher on standardized math and science tests in comparison to baseline data 2. 六年级学生在标准化的数学和自然科学考试中成绩高于基准数据的人数比例 1. % of eligible urban children enrolled in pre-school education 1. 城市适龄儿童接受学前教育的比例 2. % of eligible rural children enrolled in pre-school education 2. 农村适龄儿童接受学前教育的比例 Targets: 具体目标 : Baselines: 基准 : Indicators: 指标 : Outcomes: 成效 : 2. Primary school learning outcomes for children are improved 2. 提高儿童小学学习的成效 1. Nations children have improved access to pre-school programs 1. 使全国有更多的儿童加入学前教育计划
  • Developing Indicators 开发指标
    • Develop your own indicators to meet your needs
    • 开发满足自己需要的指标
    • Developing good indicators usually takes more than one try
    • 开发好的指标通常要进行多次尝试
    • Arriving at final indicators will take time
    • 开发出最终指标花费的时间
    • State all indicators neutrally – not “increase in…” or “decrease in…”
    • 用中性词语描述所有的指标,而非“在……增长了……”或“在……下降了……”之类的表达
    • Pilot, Pilot, and Pilot!
    • 试点、试点、再试点!
    www.dadangsolihin.com
  • Step 4: Gathering Baseline Data on Indicators 步骤 4: 收集指标的基准数据 Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • Baseline Data and Sources 基准数据及来源
    • Baseline data:
      • Measurement to find out where are we today?
    • 基准数据:
      • 明确我们现在处于什么情况的测量结果
    • Primary source: :
      • gathered specifically for the project
    • 主要数据来源:
      • 专门为项目收集的数据
    • Secondary source: :
      • collected for another purpose, but can be used
      • from: within organization, government, international data sources, etc.
      • can save money but be careful to ensure that it is truly the information you need
    • 辅助数据来源:
      • 为其它目的收集的数据,但也可以使用
      • 来源:组织内部、政府和国际数据来源等
      • 可以节约资金,但要注意确保这些信息是你真正需要的
    www.dadangsolihin.com
  • Possible Sources 可能的数据来源
    • Written records
    • 书面记录
    • Individuals involved with the intervention
    • 参与干预措施的个人
    • The general public
    • 公众
    • Trained observers
    • 训练有素的观察员
    • Mechanical measurements
    • 机械测量
    • Geographical information system
    • 地理信息系统
    www.dadangsolihin.com
  • www.dadangsolihin.com Conversation with concerned individuals 与相关 人员交谈 Community Interviews 社区访谈 Field visits 实地 调研 Reviews of official records (MIS and admin data) 查阅官方数据 ( 管理信息系统和管理数据 ) Participant Observation 参与式 观察 Key informant interviews 与关键知情 者面谈 Focus Group Interviews 焦点组 访谈 Questionnaires 问卷调查 Panel Surveys 同样本 调查 Census 人口普查 Field experiments 现场 试验 Informal/Less Structured Methods 非正式性 / 结构性较差的方法 More Structured/Formal Methods 更具结构性 / 正式的方法 One-Time Survey 一次性 调查 Direct observation 直接 观察
  • Continuing Example, Developing Baseline Data for One Policy Area: Education 开发基准数据:教育领域的一个例子 www.dadangsolihin.com
    • 47% in 2002 scored 70% or better in math. 50% in 2002 scored 70% or better
    • 1. 2002 年在数学和自然科学考试中取得 70 分或以上成绩的学生比例分别为 47% 和 50%
    • 2. Mean % score in 2002 for Grad 6 students for math was 68%, and 53% for science
    • 2. 2002 年六年级学生数学和自然科学平均成绩分别为 68 分和 53 分
    • 75% urban in 1999
    • 1999 年城市儿童接受学前教育的比例为 75%
    • 40% rural in 2000
    • 2. 2000 年农村儿童接受学前教育的比例为 40%
    1.% of Grade 6 students scoring70% or better on standardized math and science tests 1. 六年级学生在标准化的数学和自然科学考试中得到 70 分或以上成绩的人数比例 2. % of Grade 6 students scoring higher on standardized math and science tests in comparison to baseline data 2. 六年级学生在标准化的数学和自然科学考试中成绩高于基准数据的人数比例 1. % of eligible urban children enrolled in pre-school education 1. 城市适龄儿童接受学前教育的比例 2. % of eligible rural children enrolled in pre-school education 2. 农村适龄儿童接受学前教育的比例 Targets: 具体目标 : Baselines: 基准 : Indicators: 指标 : Outcomes: 成效 : 2. Primary school learning outcomes for children are improved 2. 提高儿童小学学习的成效 1. Nations children have improved access to pre-school programs 1. 使全国有更多的儿童加入学前教育计划
  • Step 5: Planning for Improvements — Setting Realistic Targets 步骤 5: 为改善绩效进行规 划——设定切合实际的具体目标 Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • Targets: 具体目标 :
    • The quantifiable levels of the indicators that a country or organization wants to achieve at a given point in time
    • 一个国家或组织希望在某一时点能够实现的经量化的指标数值
    • Example:
      • Agricultural exports will increase in the next three years by 20% over the baseline
    • 示例:
      • 未来 3 年农业出口将在基准出口水平的基础上增加 20%
    www.dadangsolihin.com
  • Identifying Expected or Desired Level of Improvement Requires Targets 确定期望或预计的改善水平需要设定具体目标 + = www.dadangsolihin.com Baseline Indicator Level 指标的基 准水平 Desired Level of Improvement 预计的改善水平 Assumes a finite and expected level of inputs, activities, and outputs 假设投入、活动和产出的期望 水平是给定的 Target Performance 具体的绩效目标 Desired level of performance to be reached within a specific time 特定时期内所要达到的预定绩效水平
  • Caution: 请注意 :
    • It takes time to observe the effects of improvements, therefore:
      • Be realistic when setting targets
    • 观察改善效应需要时间,因此:
      • 在设定具体目标时要切合实际
      • Avoid promising too much and thus programming yourself to fail
      • 避免过多承诺,不然会使自己陷于失败
    www.dadangsolihin.com
  • Continuing Example, Setting Performance Targets for One Policy Area: Education 设定具体目标:教育领域的一个例子 1.47% in 2002 scored 70% or better in math. 50%in 2002 scored 70% or better 1. 2002 年在数学和自然科学考试中取得 70 分或以上成绩的学生比例分别为 47% 和 50% 2. Mean % score in 2002 for Grad 6 students for math was 68%, and 53% for science 2. 2002 年六年级学生数学和自然科学平均成绩分别为 68 分和 53 分
    • 75% urban in 1999
    • 1.1999 年城市儿童接受学前教育的比例为 75%
    • 2. 40% rural in 2000
    • 2. 2000 年农村儿童接受学前教育的比例为 40%
    1.% of Grade 6 students scoring70% or better on standardized math and science tests 1. 六年级学生在标准化的数学和自然科学考试中得到 70 分或以上成绩的人数比例 2. % of Grade 6 students scoring higher on standardized math and science tests in comparison to baseline data 2. 六年级学生在标准化的数学和自然科学考试中成绩高于基准数据的人数比例 1. % of eligible urban children enrolled in pre-school education 1. 城市适龄儿童接受学前教育的比例 2. % of eligible rural children enrolled in pre-school education 2. 农村适龄儿童接受学前教育的比例 2. Primary school learning outcomes for children are improved 2. 提高儿童小学学习的成效 1. Nations children have improved access to pre-school programs 1. 使全国有更多的儿童加入学前教育计划 www.dadangsolihin.com 1.80% by 2006 in math-7% by 2006 1. 2006 年之前在数学和自然科学考试取得 70 分或以上成绩的学生比例要分别达到 80% 和 70% 2.Mean Math test score in 2006 is 78%--mean science score in 2006 is 65% 2.2006 年数学和自然科学平均成绩分别要达到 78 分和 65 分 1.85% urban in 2006 1.2006 年城市儿童接受学前教育的比例要达到 85% 2.60%rural in 2006 2.2006 年农村儿童接受学前教育的比例要达到 60% Targets: 具体目标 : Baselines: 基准 : Indicators: 指标 : Outcomes: 成效 :
  • Step 6: Monitoring for Results 步骤 6: 面向结果的监测 Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • Key Types of Monitoring 监测的主要类型 www.dadangsolihin.com Results Monitoring 结果监测 Implementation Monitoring (Means and Strategies) 实施情况监测 ( 手段和策略 ) Outcomes 成效 Impacts 影响 Results 结果 Inputs 投入 Activities 活动 Outputs 产出 Implementation 实施情况
  • Implementation Monitoring Links to Results Monitoring 实施监测与结果监测相连系 www.dadangsolihin.com Means and Strategies (Multi-year and Annual Work Plans) 方法和策略 ( 多年期和年度工作计划 ) Means and Strategies (Multi-year and Annual Work Plans) 方法和策略 ( 多年期和年度工作计划 ) Means and Strategies (Multi-year and Annual Work Plans) 方法和策略 ( 多年期和年度工作计划 ) Monitor Results 监测结果 Monitor Implementation 监测实施 Target 1 具体目标 1 Target 2 具体目标 2 Target 3 具体目标 3 Outcome 成效
  • www.dadangsolihin.com Target 1 具体目标 1 Partner1 合作伙伴 1 Partner2 合作伙伴 2 Partner3 合作伙伴 3 Means & Strategy 方法与策略 Partner1 合作伙伴 1 Partner2 合作伙伴 2 Partner3 合作伙伴 3 Means & Strategy 方法与策略 Partner1 合作伙伴 1 Partner2 合作伙伴 2 Partner3 合作伙伴 3 Means & Strategy 方法与策略 Target 2 具体目标 2 Outcome 成效 Outcome 成效 Outcome 成效 Impact 影响
  • Successful Monitoring Systems 成功的监测体系
    • To be successful, every monitoring system needs the following:
    • 要成功的话,每个监测体系都需要
      • Ownership 所有权
      • Management 管理
      • Maintenance 维护
      • Credibility 可信度
    www.dadangsolihin.com
  • Step 7: The Role of Evaluations 步骤 7: 评价的作用 Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • Evaluation Means Info on: 评价提供下列信息: www.dadangsolihin.com Strategy 战略
    • Whether we are doing the right things 我们是否在做正确的事
      • Rationale/justification 原理 / 证明
      • Clear theory of change 清晰的变革理论
    Operation 操作
    • Whether we are doing things right 我们是否在正确地做事
      • Effectiveness in achieving expected outcomes 现预期成效方面的效果
      • Efficiency in optimizing resources 优化使用资源方面的效率
      • Client satisfaction 客户满意度
    Learning 学习
    • Whether there are better ways of doing it 是否有更好的方法来做这件事
      • Alternatives 其它方法
      • Best practices 最佳实践
      • Lessons learned 经验教训
  • Evaluation and Monitoring – When to Use? 评价和监测——何时实施 ?
    • Any time there is an unexpected result that requires further investigation
    • 出现预期之外的结果并且需要进一步调查研究的时候
    • When resource or budget allocations are being made across projects, programs, or policies
    • 在不同项目、计划或政策之间进行资源或预算分配的时候
    • When a decision is being made whether or not to expand a pilot
    • 正在进行是否扩大试点范围决策的时候
    • When there is a long period with no improvement, and the reasons for this are not clear
    • 长期内未取得绩效改善,并且出现这种情况原因不明的时候
    • When similar programs or policies are reporting divergent outcomes
    • 类似计划或政策报告的成效各不相同时
    www.dadangsolihin.com
  • Step 8: Reporting Findings 步骤 8: 报告发现 Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • Analyzing and Reporting Data 分析数据和报告数据
    • Gives information on the status of projects, programs, and policies
    • 给出项目、计划和政策状态方面的信息
    • Provides clues to problems
    • 提供问题线索
    • Creates opportunities to consider improvements in the (projects, programs, or policy) implementation strategies
    • 为考虑改进(项目、计划或政策)实施策略创造机会
    • Provides important information over time on trends and directions
    • 提供随时间变化的发展趋势和方向方面的重要信息
    • Helps confirm or challenge theory of change behind the project ,program or policy
    • 帮助证实或质疑项目,计划或政策背后的变革理论
    www.dadangsolihin.com
  • Analyzing Data 分析数据
    • Examine changes over time
      • Compare present to past data to look for trends and other changes
      • The more data points you have, the more certain you are of your trends
    • 研究随时间发生的变化
      • 比较当前与过去的数据以观察趋势和其它变化
      • 拥有的数据点越多,对趋势的判断就越准确
    www.dadangsolihin.com ? Time Improving access to rural markets 时间 增加进入农村市场的机会 进入机会 Access Time Improving access to rural markets 时间 增加进入农村市场的机会 进入机会 Access
  • Presenting Data 呈报数据
    • Present most important data only
    • 只呈报最重要的数据
    • Use an appendix or a separate report for detailed technical data
    • 使用附件或单独报告来呈报详细的技术数据
    • Use visual presentations (charts, graphs, maps) to illustrate and highlight key points
    • 使用有形的呈报形式(图表、图形、地图)来解释和突出关键性数据
    • Avoid “data dumps”
    • 避免出现“数据转储”
    www.dadangsolihin.com
  • Step 9: Using Findings 步骤 9: 使用发现 Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • Ten Uses of Results Findings 结果发现的 10 个用途
    • Responds to elected officials’ and the public’s demands for accountability
    • 应对官员和公众提出的针对问责机制的要求
    • Helps formulate and justify budget requests
    • 帮助制定和论证预算要求
    • Helps in making operational resource allocation decisions
    • 帮助进行可用资源分配决策
    • Triggers in-depth examinations of what performance problems exist and what corrections are needed
    • 引发对存在的绩效问题和所需的改进方法进行深入研究
    • Helps motivate personnel to continue making program improvements
    • 有助于激励工作人员对计划进行持续性改进
    • (continued on next slide)( 转下页 )
    www.dadangsolihin.com
  • Ten Uses of Results Findings (cont.) 结果发现的 10 个用途 ( 续 )
    • Monitors the performance of contractors and grantees
    • 监测订约人和受让人的绩效
    • Provides data for special, in-depth program evaluations
    • 向特殊而深入的计划评价提供数据
    • Helps provide services more efficiently
    • 更有效地提供服务
    • Supports strategic and other long-term planning efforts (by providing baseline information and later tracking progress)
    • (通过提供基准信息和事后进展跟踪 ) 加大战略性的和其它长期性的规划力度
    • Communicates better with the public to build public trust
    • 更好地与公众进行沟通,从而建立公众信任
    www.dadangsolihin.com
  • Nine Strategies for Sharing Information 信息共享的九个策略
    • Empower the media
    • 授权媒体
    • Enact “Freedom of Information” legislation
    • 制定“信息自由”法
    • Institute E-government
    • 建立电子政务系统
    • Add information on internal and external internet sites
    • 在内部和外部网站上公开信息
    • Publish annual budget reports
    • 发布年度预算报告
    • Engage civil society and citizen groups
    • 使公民社会和市民团体参与进来
    • Strengthen parliamentary oversight
    • 加强议会监督
    • Strengthen the Office of the Auditor General
    • 加强审计署的职能
    • Share and compare results findings with development partners
    • 与开发合作伙伴共享成果并与之进行比较
    www.dadangsolihin.com
  • Step 10: Sustaining the M&E System within the Organization 步骤 10: 在组织内部维持监测与评价系统 Planning for Improvement  Selecting Realistic Targets 为改善绩效进行规划——选择 切合实际的具 体目标 5 1 Conducting a Readiness Assessment 实施就绪 度评价 3 Selecting Key Indicators to Monitor Outcomes 选择监测成效的关键指标 7 The Role of Evaluations 评价作用 9 Using Findings 使用发现 2 Agreeing on Outcomes to Monitor and Evaluate 对监测与评价成效达成一致意见 4 Baseline Data on Indicators — Where Are We Today? 指标的基准数 据——我们现在 处于什么情况? 6 Monitoring for Results 面向结果 的监测 8 Reporting Findings 发现 Sustaining the M&E System Within the Organization 组织内部维持监测与评价系统 10
  • Critical Components Crucial to Sustaining 维持系统的关键要素
    • Demand
    • 需求
    • Clear roles and responsibilities
    • 职责明确
    • Trustworthy and credible information
    • 可信而且可靠的信息
    • Accountability
    • 责任机制
    • Capacity
    • 能力
    • Incentives
    • 激励机制
    www.dadangsolihin.com
  • Last Reminders 最后的注意事项
    • The demand for capacity building never ends! The only way an organization can coast is downhill…
    • 能力建设的需求永无止境
    • 驻足不前,只会每况愈下
    • Keep your champions on your side and help them!
    • 争取拥护者并向其提供帮助!
    • Establish the understanding with the Ministry of Finance and the Parliament that an M&E system needs sustained resources
    • 使财政部和议会理解,监测和评价系统需要源源不断的资金投入
    • Look for every opportunity to link results information to budget and resource allocation decisions
    • 抓住每一个机会,将结果信息与预算和资源分配决策联系起来
    • (continued on next slide)
    • ( 转下页 )
    www.dadangsolihin.com
  • Last Reminders (cont.) 最后的注意事项 ( 续 )
    • Begin with pilot efforts to demonstrate effective results-based monitoring: i.e., begin with an enclave strategy (e.g., islands of innovation) as opposed to a whole-of-government approach.
    • 从试点开始,逐步展示有效的面向结果的监测:例如,先从局部范围开始(如局部创新),而不是一下子就在整个政府范围内建立这样的系统
    • Monitor both implementation progress and results achievements.
    • 对实施进展和取得的结果都要实施监测
    • Complement performance monitoring with evaluations to ensure better understanding of public sector results.
    • 以评价作为绩效监测的补充手段,确保更好地理解公共部门的结果
    www.dadangsolihin.com
  • Thank you! 谢谢!
    • Beside working as Assistant Professor at Graduate School of Asia-Pacific Studies, Waseda University, Tokyo, Japan, he also active as Associate Professor at University of Darma Persada, Jakarta, Indonesia.
    • He got various training around the globe, included Shanghai International Program for Development Evaluation Training (2008) , Public Officials Capacity Building Training Program for Government Innovation, Seoul –Korea (2007), Advanced International Training Programme of Information Technology Management, at Karlstad City, Sweden (2005); the Training Seminar on Land Use and Management, Taiwan (2004); Developing Multimedia Applications for Managers, Kuala Lumpur, Malaysia (2003); Applied Policy Development Training, Vancouver, Canada (2002); Local Government Administration Training Course, Hiroshima, Japan (2001); and Regional Development and Planning Training Course, Sapporo, Japan (1999). He published more than five books regarding local autonomous.
    • You can reach Dadang Solihin by email at [email_address] or by his mobile at +62812 932 2202
    Dadang Solihin currently is Director for Regional D evelopment Performance Evaluation at Indonesian National Development Planning Agency (Bappenas). He holds MA degree in Economics from University of Colorado, USA. His previous post is Director for System and Reporting of Development Performance Evaluation at Bappenas. Dadang Solihin’s Profile www.dadangsolihin.com