Ilac 10 6 10


Published on

The presentation discusses the role of impact assessment in the CGIAR change process

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ilac 10 6 10

  1. 1. What we think is what matters: the role of impact assessment (IA) in the CGIAR change management<br />Javier M. Ekboir<br />1<br />
  2. 2. Overview of the presentation<br />Review of evaluation and IA approaches<br />How can philosophy of science contribute to evaluation and IA?<br />The nature of organizational change<br />Evaluation and IA in the history of the CGIAR <br />Evaluation and IA in the on-going change process<br />2<br />
  3. 3. Evaluation and IA can be categorized from two perspectives: goals and methods<br />Goals:<br />summative or judgment-oriented to determine the value of a program<br />formative or improvement-oriented to identify steps to improve a program<br />developmental to facilitate learning during implementation <br />3<br />
  4. 4. Evaluation and IA can be categorized from two perspectives (2)<br />Methods:<br />Qualitative<br />Quantitative <br /> rates of return and cost-benefit analysis<br /> other approaches, especially related to how to measure changes in indicators in non-experimental settings<br />Combination of qualitative and quantitative<br />4<br />
  5. 5. Which approaches and methods are appropriate depends on what important stakeholders think is appropriate!<br />5<br />
  6. 6. The role of data in learning<br />6<br />Argentina’s amazingdiscovery<br />Every data set (experimental or not) is open to an infinite number of hypotheses (underdetermination thesis)<br />Let me show you the picture of a bird<br />Or is it a rabbit? <br />Data are collected and have a meaning within a “theoretical” framework (“theory-ladenness” of data)<br />Data are important because they provide information to build and “confirm” theories<br />
  7. 7. Hypotheses cannot be tested (T, H, A1, A2 … An)<br />Then, what do researchers do? <br />They “build explanations”<br />But there is no consensus on what a valid explanation is<br />Scientists build narratives that reflect how they see the processes they study<br />7<br />
  8. 8. The social nature of knowledge<br />The acceptance or rejection of a narrative depends on the consensus among the majority of scientists and other stakeholders about what is a good explanation<br />8<br />Consensus about climate change<br />This does not mean that all knowledge is relative<br />Just that knowledge is determined by the interaction of data and social conventions<br />
  9. 9. Organizational change depends, among other factors, on:<br />The nature of leadership <br />9<br />The organization’s culture <br />The organization’s learning routines <br />Governance<br />In NGOs, also external consensus about what needs to be done <br />
  10. 10. The nature of change in organizations <br />Stakeholders influence organizational change according to their beliefs, resources and connections <br />10<br />Similar minded stakeholders form informal coalitions <br />The more effective a coalition, the greater its influence on the change process<br />
  11. 11. The nature of change in organizations (2) <br />Influence can be gained in two ways: <br />commanding resources <br />convincing other stakeholders by presenting a compelling argument about what change is needed and how to measure it<br />Evaluations do not bring organizational change by themselves, but influence the perception of what changes are needed<br />11<br />
  12. 12. Why was impact assessment implemented in the CGIAR?<br />In the 1990s, CGIAR centers had to justify their existence, i.e., accountability was the most important motive for IA <br />For decades, economists had calculated rates of return to research, which was considered by some professionals a rigorous methodto measure impacts <br />Some of these professionals occupied important positions among donors, in the science council and in the centers <br />Over time, other approaches for IA were developed by CGIAR and external scientists, but they remained at the fringes of the system<br />12<br />
  13. 13. Why was impact assessment implemented in the CGIAR? (2)<br />In recent years, some donors and scientists realized that an alternative to the rates of return was needed<br />New approaches for evaluation, IA and organizational learning are being adopted by donors, CGIAR scientists, other organizations working on development, and important segments of the evaluation community<br />The issue today is what type of IA meets scientific standards and is usefulfor the donors and the CGIAR<br />13<br />
  14. 14. IA and change management in the CGIAR<br />Evaluations influence the change management process by helping to create a consensus about the new role of the CGIAR and the changes needed<br />IA, as designed by the SC, did not induce change in the system <br />How the new approaches for evaluation can influence change in the CGIAR depends on the system’s ability to use the new information<br />This depends, among other factors, on the coalitions that influence the process<br />14<br />
  15. 15. How can new evaluation approaches help change in the CGIAR?<br />Generating new frameworks and information to help the Consortium, similarly-minded donors and DGs to provide effective leadership for change<br />Synthesizing the new approaches for evaluation and IA developed by CGIAR scientists and providing a unifying framework<br />15<br />
  16. 16. How can new evaluation approaches help change in the CGIAR? (2)<br />Developingnew approaches for IA based on novel frameworks for the analysis of social processes<br />Building a new, compelling argument for reassessing the role of science (including the CGIAR) and IA in development by bringing in new ideas and challenging orthodoxies <br />16<br />
  17. 17. On its own, IA will have little impact on the change process<br />To have an impact, it is necessary to work simultaneously along three lines of action:<br />Give more visibility to the new methods for evaluation and IA developed by CGIAR and/or external scientists <br />Develop a conceptual framework that unifies the new methods for evaluation and IA and provides a compelling argument for using evaluation as a tool for organizational learning<br />Build a coalition of similarly-minded stakeholders <br />17<br />
  18. 18. Thank you!<br />18<br />