Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center; 200 Panlong Road-Shanghai, October 13, 2008

1. Developing Evaluation Questions & Starting the Design Matrix 制定评价问题选择设计矩阵 Shanghai International Program for Development Evaluation Training Asia-Pacific Finance and Development Center 200 Panlong Road-Shanghai, October 13, 2008 Linda Morra-Imas

6. Plus the Theory of Change: E.g. Logic Model to Questions 变革理论 问题的逻辑模型 www.dadangsolihin.com Inputs (Resources) 投入资源 Activities 活动 Outputs 产出 Short Term Results (Direct) 短期结果（直接） Intermediate Results (Indirect) 中期结果（间接） Long-Term Result 长期结果 Area of Control Internal to the Organization 组织内部的控制领域 Outputs Reach Direct Beneficiaries 直接受益人得到产出 Area of Influence External to the Organization 组织外部的影响领域 External Factors 外部因素 What aspects of our situation most shaped our ability to do the work we set out to do in our community 我们什么方面的情况最能够塑造我们执行既定工作的能力 What did our program accomplish in our community? 我们的计划在社会中取得了什么成就 What is our assessment of what resulted from our work in the community? 对于社会中我们工作所导致的事物我们如何评价？ What have we learned about doing this kind of work in a community like yours? 关于在像你们这样的社区做这样的工作我们学到了什么 Formative Evaluation 形成性评价 Summative Evaluation 总结性评价 and/or 和 / 或

22. Example of Classifying from Theory of Change 从逻辑模型进行问题分类的例子 Logic model for a micro-lending program 小额贷款计划的逻辑模型 Access to set-up funds for small businesses 获得小企业创业资金的机会 Income and employment for local people 为当地人们带来的收入和就业 Improved living conditions 生活条件的改善 Financial management advice and support 财务管理建议和支持 Skills in business & financial management 商业和财务管理的技能 Reduced family poverty 减少家庭贫困 www.dadangsolihin.com Questions about access to start up funds for small businesses 有关获得小企业创业资金的问题 Questions about income and employment for local people 有关对当地人们带来的收入和就业的问题

23. Example of Classifying from Theory of Change 从计划成效模型进行问题分类的例子 Questions about resources 有关资源的问题 Questions about services 有关服务的问题 www.dadangsolihin.com Changes 变化 • money • 资金 • staff • 人员 • volunteers • 自愿者 • supplies • 物品供应 Benefits 效益 • new knowledge • 新知识 • increased skills • 技能提高 • changed attitudes • 态度变化 • new employment opportunities • 新的就业机会 Products 产品 • total # of classes • 班级数量 • hours of service • 服务时间 • number of participants completing course • 完成课程的参与者人数 Services 服务 • training • 培训 • education • 教育 • counseling • 咨询服务 Resources 资源 • money • 资金 • staff • 人员 • volunteers • 自愿者 • supplies • 物品供应 Impacts 影响 Outcomes 成效 Outputs 产出 Activities 活动 Inputs 投入

27. Matrix for Ranking and Selecting Questions 排列和选择问题的矩阵 Would the evaluation question … 评价问题能否 …… Question # 问题编号 1 2 3 4 5 6 7 8 … 1 be of interest to key audiences? 引起关键报告对象的兴趣？ 2 reduce present uncertainty? 减小目前的不确定性？ 3 yield important information? 产生重要的信息？ 4 be of continuing interest? 具有持续的吸引力？ 5 be critical to the study ’ s scope and comprehensiveness? 对研究范围和综合性是否关键？ 6 have an impact on the course of events? 对事件过程产生影响？ 7 be answerable in terms of … 以 …… 来回答？

33. Approach to Development Evaluation 发展评价方法 Focus the Evaluation 关注评价 ▪ Identification and meeting with stakeholders 确认会见利益相关者 ▪ Purpose – meeting with the client 会见客户 ▪ Research – other studies and program documentation 其他研究和计划文件 ▪ Theory of Change 变革理论 ▪ Specification of evaluation questions 详细说明评价问题 ▪ Terms of Reference 任务大纲 Design & Methodology 设计和方法 ▪ Evaluation questions 评价问题 ▪ Measurement strategy 测量策略 ▪ Data collection design 数据收集设计 ▪ Data collection strategy 数据收集策略 ▪ Sampling strategy 抽样策略 ▪ Develop data collection instruments 制定数据收集工具 ▪ Develop analysis plan 制定分析计划 Brief client and stakeholders 主要客户和利益相关者 Gather & Analyze Data 收集 & 分析数据 ▪ Test instruments 测试工具 ▪ Develop protocols 制定协议 ▪ Train as needed 按需求培训 ▪ Gather data according to protocols 根据协议收集数据 ▪ Prepare data for analysis 准备分析用的数据 ▪ Analyze data 分析数据 ▪ Interpret data 解释数据 ▪ Hold message conference 开展通气会 ▪ Draft statement of findings 草拟发现说明 Report Findings 报告发现 ▪ Write report 写报告 ▪ Review and quality checks 审核质量核查 ▪ Make recommendations Incorporate feedback/ 做出建议 包含建议 refine 完善 ▪ Deliver 提交 Use Evaluation 使用评价 ▪ Develop communication strategy ▪ Brief on Evaluation Design ▪ Update brief on evaluation process ▪ Communicate Findings ▪ Feed-back ▪ Decision-making ▪ Action Plan ▪ Follow-up ▪ Recommendations ▪ Tracking www.dadangsolihin.com

37. www.dadangsolihin.com Main Evaluation Question/Issue: 主要评价问题 Page 1 A Questions 问题 Sub-Questions 次问题 Type of (sub)question 问题类型 Measure or Indicator 测量或指标 Target or Standard (if normative) 目标或标准（如果是规范性问题） Baseline Data? 基准数据

38. www.dadangsolihin.com General Approach: 一般方法 Page 1 B Design 设计 Data Sources 数据来源 Sample or Census 抽样或普查 Data Collection Instrument 数据收集工具 Data Analysis and Graphics 数据分析和图表 Comments 注释

39. Thank you! 谢谢！