This document discusses research challenges in Somalia and summarizes two case studies of qualitative research on FAO's Cash for Work program. The first case study evaluated the 2007-2012 Cash for Work program through qualitative impact assessments with a local research team. It found both short and long-term benefits to food security and agriculture, but also room for improved community consultation and targeting. The flexible methodology allowed feedback into the evolving research. The second case studied the potential for mobile banking as an alternative delivery mechanism. Research across communities found network coverage was poor but phone ownership increasing. Literacy was not a major barrier. A pilot was deemed possible where networks exist but infrastructure remains a challenge. Overall, the document finds that an

1. Extreme Realities and Imagination
Barriers: Research in Somalia
Savina Tessitore, Julie Lawson-McDowall, Giuseppe Simeon,
FAO Somalia & FAO Rome.
With Mohammed Jama Aden, Hoosh Banadir Sheikh, Omar Mohammed Hassan, Mire Abdullaahi Elmi,
Sowdo Mude Billow, Farah Osman, Liban Mohammed, Mohyadin Omer, Khadar Abiib

2. Outline
• Background: the Somali research conundrum
• How has accountability in M&E in Somalia
been attempted until now?
• 2 cases of qualitative research around the FAO
Cash for Work programme
• Key methodological learning

3. The Somali research conundrum
• The context demands and
impedes intense learning and
oversight
• History of extensive diversion
imposes compliance monitoring
• Community targeting is a black
box for outsiders
• Gatekeepers e.g. local
authorities “tax” incoming
resources
• Customary
redistributiondistorts
implementation objectives
• Security imposes remote management  employment of local
researchers, but scarce research skills at local level
• Clan affiliation of researchers, a key to access, may prove
problematic

4. Accountability in M&E in Somalia
• Quantitative impact evaluations with Randomized
Control Trials are the gold standard for Cash Transfers
assessment. Always the best solution?
• In Somalia they inspired recent M&E of humanitarian
interventions
• Pros: large-scale, replicability, generalization and
comparability between agencies and kinds of
interventions
• Cons: extremely “dirty” data, difficulties of remote
managing large-scale data collection, inputs not always
high enough value to be measurable, final results
useful but too general?
• How can in-depth qualitative work complement this
methodology?

5. Researching FAO’s CFW: a virtuous cycle
of learning
Case 1:
•Team: inexperienced male and
Evaluation of FAO female researchers with local
Somalia 2007/2012. access, led and supervised by
Qualitative impact experienced researcher
study of FAO’s CFW •Methodology: adaptation of a
programme familiar body of participatory tools
Gedo, South-Central •Capacity building: substantial training
Somalia
input
•Learning: constant
Case 2: mentoring, debriefing and
Scoping mobile banking interrogation of research outputs.
to poor rural Virtuous cycle of growth in the quality
beneficiaries for next of researchers and their work
phases of CFW •Sampling: reiteration of research in
implementation several sites to build up a more robust
Hargeisa, Somaliland understanding

6. CASE 1 CFW impact study: aims and methodology
Context
•Rapid and huge expansion of FAOSO CFW in response to crisis: 2011: $2m
to 2012: $30m+
•Shifting emphasis: ag productivity  food access + ag productivity
Objectives
•Study to focus on outcome and impact, beneficiary perspectives
•+Learning -accountability
•Feeding evidence into the programme’s new learning mechanisms
Methodology
•Tools: social mapping; wealth ranking and targeting analysis;
income, expenditure and coping strategies matrix; ranking of preferred
interventions; SSI with non-beneficiaries
•Sampling: 7 villages, restricted access due to security and investigation of
partner
•Flexibility, build trust and capacity of team, rely on local
knowledge, incorporate team’s suggestions and observations

7. CASE 1 CFW impact study: findings
Impact
• CFW valued for its short and long-term benefits, relevant
impact on FS and Ag productivity, multiplier effects, some
impact on productive activities, some unintended negative
impacts
• Little community consultation (women, Bantus)
• Systematic Inclusion and Exclusion Errors
• Better planning required for work component and choice of
infrastructures
Methodological learning
• Qualitative approach allowed tight feedback loops into
evolving methodology
• In a research context where so much is unknown, being open
to emerging issues is vital
• Approach permitted interrogation of complex social relations

8. CASE 2 scoping the use of mobile banking: aims and methodology
Hypothesis:
mobile banking can be used
a) as an alternative delivery mechanism
b) to enhance M&E through direct access
Methodology:
• Desk study and preliminary research mission
• Identification and training of team
• Research in 3 communities around Hargeisa, 5 communities in Sanaag, 5
communities in Burao
• Research tools: village mapping, wealth ranking, FGD on phone use
Methodological learning:
• Male and female team must be able to work in specific localities
• Training, practice, debrief, review, practice
• Daily debrief, by phone if necessary and systematize data collection for
comparison
• Keep interrogating key issues
• Request phone numbers and call interviewees back if necessary

9. CASE 2 scoping the use of mobile banking: findings
Topic Findings
Network Coverage Erratic, poor to non-existent in remote sites. Combined with charging
problems makes use of phones a challenge
Charging Dependent on generators, solar chargers, normal batteries
Current use of Spreading downwards from better off/shop owners, saves transport costs
Zaad
Registration Energetic marketing by phone company
Ownership and Ownership spreading fast amongst better off,
Access
Illiteracy Only functional numeracy/literacy required
Buying phone $20 upwards. Many second hand and inherited
Charging phone Major expense - $0.13-25 per charge
Airtime Relatively low cost
CONCLUSION:
Can be piloted where network exists. Risky technology given infrastructural
challenges. Partnership with mobile bank provider necessary

10. Key methodological learning I
• The “extreme reality” of Somalia sheds light on
methodological issues encountered elsewhere, and
provides indications to overcome “imagination
barriers”
• Intense
learning, supervision
, search for
gaps, reiteration and
testing results in
steady improvement
in team
performance and
research outcomes
• Mixed gender team
crucial to learning

11. Key methodological learning II
• Some questions particularly resistant to interrogation
(clan, polygamous households, intra-household dynamics)
will be easier to crack through refinement of qualitative
methods
• Though quantifying and attributing causality remains a
challenge, this approach allows in-depth learning and
acknowledges this complexity in social relations.

12. Conclusions
• Cost-effective
approach.
• Repetition makes
highly indicative
findings more
robust.
• Rapid feedback of
evidence into
programming