Extreme Realities and Imagination Barriers: Research in Somalia Savina Tessitore, Julie Lawson-McDowall, Giuseppe Simeon, FAO Somalia & FAO Rome. With Mohammed Jama Aden, Hoosh Banadir Sheikh, Omar Mohammed Hassan, Mire Abdullaahi Elmi, Sowdo Mude Billow, Farah Osman, Liban Mohammed, Mohyadin Omer, Khadar Abiib
Outline• Background: the Somali research conundrum• How has accountability in M&E in Somalia been attempted until now?• 2 cases of qualitative research around the FAO Cash for Work programme• Key methodological learning
The Somali research conundrum• The context demands and impedes intense learning and oversight• History of extensive diversion imposes compliance monitoring• Community targeting is a black box for outsiders• Gatekeepers e.g. local authorities “tax” incoming resources• Customary redistributiondistorts implementation objectives• Security imposes remote management employment of local researchers, but scarce research skills at local level• Clan affiliation of researchers, a key to access, may prove problematic
Accountability in M&E in Somalia• Quantitative impact evaluations with Randomized Control Trials are the gold standard for Cash Transfers assessment. Always the best solution?• In Somalia they inspired recent M&E of humanitarian interventions• Pros: large-scale, replicability, generalization and comparability between agencies and kinds of interventions• Cons: extremely “dirty” data, difficulties of remote managing large-scale data collection, inputs not always high enough value to be measurable, final results useful but too general?• How can in-depth qualitative work complement this methodology?
Researching FAO’s CFW: a virtuous cycle of learningCase 1: •Team: inexperienced male andEvaluation of FAO female researchers with localSomalia 2007/2012. access, led and supervised byQualitative impact experienced researcherstudy of FAO’s CFW •Methodology: adaptation of aprogramme familiar body of participatory toolsGedo, South-Central •Capacity building: substantial trainingSomalia input •Learning: constantCase 2: mentoring, debriefing andScoping mobile banking interrogation of research outputs.to poor rural Virtuous cycle of growth in the qualitybeneficiaries for next of researchers and their workphases of CFW •Sampling: reiteration of research inimplementation several sites to build up a more robustHargeisa, Somaliland understanding
CASE 1 CFW impact study: aims and methodology Context •Rapid and huge expansion of FAOSO CFW in response to crisis: 2011: $2m to 2012: $30m+ •Shifting emphasis: ag productivity food access + ag productivity Objectives •Study to focus on outcome and impact, beneficiary perspectives •+Learning -accountability •Feeding evidence into the programme’s new learning mechanisms Methodology •Tools: social mapping; wealth ranking and targeting analysis; income, expenditure and coping strategies matrix; ranking of preferred interventions; SSI with non-beneficiaries •Sampling: 7 villages, restricted access due to security and investigation of partner •Flexibility, build trust and capacity of team, rely on local knowledge, incorporate team’s suggestions and observations
CASE 1 CFW impact study: findingsImpact• CFW valued for its short and long-term benefits, relevant impact on FS and Ag productivity, multiplier effects, some impact on productive activities, some unintended negative impacts• Little community consultation (women, Bantus)• Systematic Inclusion and Exclusion Errors• Better planning required for work component and choice of infrastructuresMethodological learning• Qualitative approach allowed tight feedback loops into evolving methodology• In a research context where so much is unknown, being open to emerging issues is vital• Approach permitted interrogation of complex social relations
CASE 2 scoping the use of mobile banking: aims and methodology Hypothesis: mobile banking can be used a) as an alternative delivery mechanism b) to enhance M&E through direct access Methodology: • Desk study and preliminary research mission • Identification and training of team • Research in 3 communities around Hargeisa, 5 communities in Sanaag, 5 communities in Burao • Research tools: village mapping, wealth ranking, FGD on phone use Methodological learning: • Male and female team must be able to work in specific localities • Training, practice, debrief, review, practice • Daily debrief, by phone if necessary and systematize data collection for comparison • Keep interrogating key issues • Request phone numbers and call interviewees back if necessary
CASE 2 scoping the use of mobile banking: findings Topic FindingsNetwork Coverage Erratic, poor to non-existent in remote sites. Combined with charging problems makes use of phones a challengeCharging Dependent on generators, solar chargers, normal batteriesCurrent use of Spreading downwards from better off/shop owners, saves transport costsZaadRegistration Energetic marketing by phone companyOwnership and Ownership spreading fast amongst better off,AccessIlliteracy Only functional numeracy/literacy requiredBuying phone $20 upwards. Many second hand and inheritedCharging phone Major expense - $0.13-25 per chargeAirtime Relatively low costCONCLUSION:Can be piloted where network exists. Risky technology given infrastructuralchallenges. Partnership with mobile bank provider necessary
Key methodological learning I• The “extreme reality” of Somalia sheds light on methodological issues encountered elsewhere, and provides indications to overcome “imagination barriers”• Intense learning, supervision , search for gaps, reiteration and testing results in steady improvement in team performance and research outcomes• Mixed gender team crucial to learning
Key methodological learning II• Some questions particularly resistant to interrogation (clan, polygamous households, intra-household dynamics) will be easier to crack through refinement of qualitative methods• Though quantifying and attributing causality remains a challenge, this approach allows in-depth learning and acknowledges this complexity in social relations.
Conclusions • Cost-effective approach. • Repetition makes highly indicative findings more robust. • Rapid feedback of evidence into programming