But to do so, for our research processes to be catalytic, we need to learn how to work better in these systems. AAS and the other CRP 1s are ‘systems’ programs because we share, together with the other so-called NRM CRPs, the same view of the change progress that EIARD has helped embed in the CGIAR. The quotation comes from an epilogue article in a special issue of Agricultural System published in 2003 on impact assessment and evaluation in R4D. With respect to the model - our view is that while change processes may be uncertain, there is value in anticipating change (e.g., by making explicit our assumed theories of change) using our best available knowledge. We then start ‘doing’ – it is a mistake to over characterize and not get started (we learned that from the EcoRegional Approach in the 1990s). Once we intervene things start to happen. We seek to amplify positive emergence and dampen what is less positive, learning as we do. We revisit our theories of change and change our plans accordingly. And so the cycle continues.
Another way of looking at this is by realizing that the CGIAR has always worked in both ordered and complex systems. Much of our genetic improvement work appears ordered because it happens in well established vertically integrated networks that link scientists to farmers. Our relational work with networks is complex. But whether ordered or complex, ‘theory-made-explicit’ in project proposals, program design, reward structures and evaluation assume we operate in ordered systems To reach the billion left behind we need to trigger social potentialities. This means building partnerships and coalitions as well as doing cutting edge research biophysical research. We need to become more comfortable and effective engaging in complex systems. Our mental models, both individually and collectively, need to change to better match this need. Part of this requires us to see ourselves as part of the system, not apart from it. For this reason, CRP AAS has coined the term ‘research in development’ to capture the embeddedness idea. Putting it another way, the complex issues that hold rural people in poverty won’t fall to single interventions. It requires systemic change, and we in the CGIAR need to become much more comfortable and able to work in this arena. AND mirroring this, the approaches that we need to use will not be recipes or instruction manuals. They will be principles underpinned by networked learning and reflection.
I now want to give you some more detail on the evaluation and learning system we are building in AAS based on these foundation stones. Forgive me a straw-man for contrast. Generally speaking impact assessment, social research and M&E happen in isolation in most research organizations. M&E is usually understood to be about financial and milestone compliance to donors. Support to knowledge sharing and learning has been more rhetorical than real.
In AAS we begin with a commitment to KS&L led from the top, and legitimized in a research program that I lead (Research Theme 6 on Knowledge Sharing, Learning and Innovation). I depict this culture box as open to indicate that this cultural shift cuts across all aspects of program activity, not least research, open access to information, etc.
An expanded role for evaluation has a critical part to play in building the KS&L culture, not least because a big part of the required culture shift is learning to be critically self-evaluative and making time for standing back and reflecting. We see impact assessment as a subset of evaluation as do most evaluation theorists and practitioners, contributing to learning.
We see a much expanded role for social research, in particular to build and test theories of change. We want to know how our projects and programs act on, and change, patterns of interactions between people and their capacities. We want to know how better to use research as a lever for change both as adaptive managers and to influence policy.
And finally we seek to be clear about the roles monitoring has to play, in its contribution to testing theories of change, as part of real-time evaluation to guide adaptive management and, last but not least, in its traditional role to monitor milestones and financial compliance.
Impact evaluation has a leadership role to play in working to build project and program theories of change from the start. Our experience is that little builds common commitment to agreed priorities like a shared vision and creatively thinking through the pathways to achieve that vision. These pathways, made explicit as theories of change, can provide the basis of ex-ante impact assessment and formative mid-term evaluations, as well as providing the evaluation questions and causal explanations for ex-post impact assessment. In practice theories of change need to explain overall program logic and one level as well explaining how specific project interventions trigger change. In practice we need to work with ‘nested’ or ‘scaled’ theories of change.
Pathways to impact are through networks – through people working together. In most NRM projects, unlike many genetic improvement ones, relationships need to be built. To deal with uncertainty, NRM R4D impact evaluation must be staged and contribute to building and testing theories of change, as just discussed. Staging provides opportunity for reflection and theory testing. Indeed self-reflective learning is the hallmark of an evaluative culture needed to underpin our intent to work more effectively in systems.
2. aas csisa integration planning mtg may 2013 by charlie
AAS CRP introductionCSISA-AAS integration6-7 May 2013, Jessore, Bangladesh
System Level Objectives CRPs – primary focusReduced rural poverty Agricultural Systems: (Drylands;Humid Tropics; Aquatic) (1s)Improved food security Commodities (Wheat; Maize;Rice; Roots and Tubers; Pulses;Dryland cereals; Grain legumes;Milk, meat & fish) (3s)Improved nutrition and health Nutrition and Health (4)Sustainably managed naturalresourcesWater Land and Ecosystems (5)Forests and Trees (6)Policies and Institutions (2); Climate Change (7)Strategy and Results Framework
Aquatic Agriculture Systems•“those farming, fishing and herding systems where the annual production dynamics ofnatural freshwater and/or coastal ecosystems contribute significantly to householdlivelihood, including income and food security.”•“These include major wetlands, floodplains and deltas, and most coastal systems.”
Aquatic Agriculture SystemsThe Coral TriangleAsia mega deltasAfrican Inland• High numbers of poorHigh % of total population dependent on AAS• High vulnerability to change (climate/sea level/water)• Potential to scale out
CRP 1s and ruralpoverty• Reaching thoseleft behind by theGreen Revolution
Our research agendaGender transformative approachesGender transformative approaches
Impact Pathways of AAS CRP#1 Scaling up and scalingout#2 Societal learning andchange#3 Shifts in the practice ofresearch in development
Engage communities through Research in Development (RinD)• Meaningful participation by localwomen and men in research• Research that empowers• Shift focus from understanding tolearning how to achieve practicaloutcomes and local change• Get M&E and IA right – pursuechange through cycles of actionand reflection – flexibility to adjustas we learn• Recognize the need for long-termsite-based fieldwork andengagement
RinD in CRP AAS• emphasis on ParticipatoryAction Research at core ofmajor CGIAR program• Applying it in – and learningacross – a coherent set ofagricultural systems• If we show it works – make itcentral to what the CGIARdoes in agricultural systems• Explicit intent to learn fromthis and scale out by workingwith partners
“CRP 1.3 is a clear example of bestpractice” CGIAR Gender Scoping StudyGender transformativeapproach…… not just “gender accommodating”Actively examine, question and seekto change rigid gender norms andimbalance of powerEncourage critical awareness amongmen and women of gender roles andnormsAddress the distribution of resourcesand power relationships betweenwomen and others in the community…… not just “gender accommodating”Actively examine, question and seekto change rigid gender norms andimbalance of powerEncourage critical awareness amongmen and women of gender roles andnormsAddress the distribution of resourcesand power relationships betweenwomen and others in the community
Gender transformativeapproachInvest in gender analysisBuild gender capacityMake gender part of normal CRP AASpractice – not an add onExperiment systematically on how toovercome gender constraintsEvaluate to learn what works and scaleout successesInvest in gender analysisBuild gender capacityMake gender part of normal CRP AASpractice – not an add onExperiment systematically on how toovercome gender constraintsEvaluate to learn what works and scaleout successes
How change happensOrlikowski and Hofman, 1997Improvements in poverty alleviation, food security andthe state of natural resources result from dynamic,interactive, non-linear, and generally uncertain processesof innovation.”EIARD, 2003
Visualizing the innovation requiredM&EImpactAssessmentSocialresearchBusiness-as-usual AAS concept of an ME&IA system forlearning and accountability
Visualizing the innovation requiredM&EImpactAssessmentSocialresearchBusiness-as-usual AAS concept of an ME&IA system forlearning and accountabilityCulture of Knowledge Sharing and Learning
Visualizing the innovation requiredM&EImpactAssessmentSocialresearchBusiness-as-usualEvaluationAAS concept of an ME&IA system forlearning and accountabilityCulture of Knowledge Sharing and LearningImpactAssessmentEstablishingworth
Visualizing the innovation requiredM&EImpactAssessmentSocialresearchSocial researchBusiness-as-usualEvaluationAAS concept of an ME&IA system forlearning and accountabilityCulture of Knowledge Sharing and LearningBuilding andtesting ToCImpactAssessmentEstablishingworth
Visualizing the innovation requiredM&EImpactAssessmentSocialresearchSocial researchBusiness-as-usualMonitoringEvaluationReal-TimeEvaluationAAS concept of an ME&IA system forlearning and accountabilityCulture of Knowledge Sharing and LearningBuilding andtesting ToCImpactAssessmentEstablishingworth
M&E fundamentals: Build and test nestedtheories of change• With stakeholders• From the beginning
M&E fundamentals: Staged / iterativeapproach• Pathways unclear tobegin with• Staged approach• Ex-ante• Mid-term• Ex-post• In support oflearning andaccountability
HandbookRoll out goals– Set the tone– Start team building/training– Achieve coordination withexisting activities– Start or consolidatepartnerships– Produce the plans
Top Line Messages from site start up• First year start ups managedhuge learning curve• Second year start ups are betterplanned and will be bettermanaged thanks to the first year• A program learning culture is wellunderway• Roll out as designed is robustacross different settings• Partners are enthusiastic andcommunities are engaged
• Local teams recruited and working• Agreement with stakeholders on developmentchallenge• Agreement with stakeholders on strategic priorities toguide a proposed program of work• Communities selectedand engaged• 2013 work plan forimplementationOutputs
Barotse FloodPlain - ZambiaTo make more effective useof the seasonal flooding andnatural resources of theBarotse Flood Plain Systemthrough more productiveand diversified aquaticagricultural managementpractices and technologiesthat improve lives andlivelihoods of the poor.
Mailata – Solomon Islandsrising population and decliningquality and availability of marineand land resources.challenge is to improve theirlives through more productive,diversified livelihoods thatempower communities to bebetter able to adapt to changeand more effective use of theirresources.to develop and test alternativeapproaches to livelihooddiversification and resourcestewardship that will acceleratedevelopment and restore theproductivity of their resources.
Southern Bangladesh Polder ZoneThe AAS development challenge is to achieve sustainable and continualimprovements in agricultural productivity, livelihoods and nutrition of poorcommunities in the Southern Bangladesh Polder Zone in the face of increasingsalinity, changing hydrology and climate change
Outcomes• Explorations by programteams of what business not-as-usual means for AAS• Awareness of the programamong stakeholders• Hub-level partners willing toparticipate• Communities engaged andwilling to participate
Learning – community engagementFront line human resources strategy•Community facilitators•Program community engagement staffEngagement process and outputs•Strength based approach excellent for empowerment objectives•Good community visions but incomplete action plans•Major concern with expectations management
Coming up for roll out in 2013Implement work plans in first year sites•Complete detailed planning on initiatives•Continued engagement in communities•Capacity building of teams and stakeholdersRoll out in second year sitesContinued development of cross cutting themes