1. Common Impact Assessment Approaches and Learning Summarised for EDIAIS Case
Report by: Angela Motsa, WISE Representative/ Consultant
Updated October 2003
The tables below summarise some of the key Impact Assessment (IA) approaches and lessons
that can be learnt from impact assessment case studies found on the Enterprise Development
Impact Assessment Information System (EDIAIS) website. The summary includes:
o Table 1: Case study summaries
o Table 2: Common IA approaches
o Table 3: Key policy lessons per case study
o Table 4: Common issues across different IA case studies
The following abbreviations are used in the report.
AAP: Agribusiness Agent Programme
ApproTEC: Appropriate Technologies for Enterprise Creation
BDS: Business Development Services
BG: Bank Guarantees
CA: Contract Association
CB: Commercial Bank
CD: Capacity Development
CEEWA-UG: Council for Economic Empowerment for Women of Africa – Uganda Chapter
CERUDEB: Centenary Rural Development Bank
CO: Country Office
Cons. Consultancy Services
CRIMP: Central Region Infrastructure Maintenance Programme
CRISP: Credit for the Informal Sector
DCFU: Development Finance Company of Uganda
DG: DCFU Group
DM&E: Design Monitoring and Evaluation
EC: Employment creation
ED: Enterprise Development
EQ: Equipment purchase or leasing
ET: Enterprise training
FEBDEV: Foundation for Entrepreneurial and Business Development
FGD: Focus Group Discussion
FSA: FSA International
FT: Fair trade
HIV: Programmes with a focus on HIV/ AIDS
IA: Impact Assessment
INFR: Infrastructure development
IDPM: Institute for Development and Policy Management, University of Manchester
IS: Input Supplier
ILGP: ITDG’s Light Engineering Project
ITDG: Intermediate Technology Development Group
KF: Kashf Foundation
KK: Kuapo Kokoo
K-Rep: K-Rep Holdings Ltd Kenya
2. LED: Local Economic Development
MDA: Mineworkers Development Agency
M&E: Monitoring and Evaluation
MFI: Micro-finance institution
MIS: Management Information System
ML: Micro Leasing
MLO: Market linkages and opportunities/ Marketing Support
MR: Market Research
PLA: Participatory Learning in Action
PRA: Participatory Rapid Appraisal
PSSSEP: Port Sudan Small Scale Enterprise Programme
PWR: Participatory Wealth Ranking
OFT: Oxfam Fair Trade
OI: Opportunity International
OPR: Output to Purpose review
RF: Rural Focus
RI: Research International
RMU: Regional Management Unit
SBF: Small and Medium Business Finance
SBP: Small Business Project
SEF: Small Enterprise Foundation
SELFINA: SERO Lease and Finance Company
SFMC: SIBDI Foundation for Micro Credit
SWARMU: Southern and West African Regional Management Unit
TEEM: Training for Enterprise and Exports Malawi
UGAFODE: Uganda Agency for Development Ltd (UGAFODE)
ZHABS: Zambia HIV/ AIDS Business Sector Project
Tables 1-4 follow below.
3. Table 1: Case study summaries
Programme Type1 Description Key elements in IA approach include:
Agribusiness Agent MLO, IS Improving access to agri-inputs for small • Donor required reviews and evaluations (OPR).
Programme (AAP), holder farmers through: • Commitments to IA studies as a result of OPR.
Zimbabwe • sale of inputs by agents • Baseline survey.
• linking large input supply companies • MIS (as a result of OPR) – monitoring mechanisms.
to small rural traders (agents) • Indicators (geographical and poverty indicators).
• IA to provide future direction of programme.
• Research studies on agents’ operations.
Appropriate EQ Developing and promoting technologies • IA is integral to organisational strategy development.
Technologies for (equipment) to ensure profitable small • IA as customer service and market research tool.
Enterprise Creation scale enterprises. • Baseline of enterprise/ household levels – to deal with attribution.
(ApproTEC), Kenya • Continuous internal IA.
• Annual internal IA.
• Occasional/ external reviews (required by donor).
CARE International RMU Regional management unit for CARE’s • IA approach is being developed focusing on:
Southern and West Southern and West African programmes o Building capacities of CO for DM&E,
African Regional focused on strengthening the country o Generating MIS’s from log frames,
Management Unit offices’ (CO) design, monitoring and o Participatory methods of information gathering,
(CARE SWARMU), evaluation (DM&E) capacities. o Monitoring as part of ongoing project activities,
South Africa o Data analysis and information management, and
o Information sharing and experiential exchange.
CASHE MF, CD, CARE India / DFID project focused on • Impact indicators set up per goal as required by funder
RF improving the rural micro finance sector – Client satisfaction study based on key goals, activities and
including dealing with institutional, measures collectively agreed log frame. Used the following IA
methodological and regulatory issues. methodologies: document review, FGDs, questionnaires, statistical
Promotes client owned and managed analysis
savings and credit groups and then
federated organisations representing
Centenary Rural MF, CB Commercial bank offering a full range of • External Reviews – recommendations have been incorporated into new
Development Bank appropriate financial services including business plan.
(CERUDEB), Uganda savings, credit and funds transfer services • Ongoing collection of financial data through MIS.
Refer to table of abbreviations
4. Programme Type Description Key elements in IA approach include:
to economically-disadvantaged • Customer perception/ satisfaction surveys where IA is used for market
• Use of customer satisfaction surveys for staff incentive programme
Central Region CD, Community based road maintenance • Participatory livelihood assessment informs programme design.
Infrastructure INFR through community based contract • Baseline study for wealth ranking and as basis for information system.
Maintenance associations (CA), skills development and • Livelihood monitoring system using log frame.
Programme (CRIMP), the establishment of a network of small • Quarterly data collection, analysis and policy evaluation/ review.
Malawi scale community contractors. • Specific budget for M&E.
Council for Economic Adv, R, CEEWA-UG is involved in advocacy, • Baseline studies on various sectors (microfinance, agricultural extension,
Empowerment for CD research and training to increase information and communication technologies) have influenced
Women of Africa – women’s access and control over key government policy
Uganda Chapter economic resources such as land, • Use a variety of quantitative, qualitative and participatory methodologies
(CEEWA-UG), Uganda agriculture, finance, markets and including key informant interviews, FGDs and questionnaire based
Credit for the Informal MF, CB Commercial Bank providing micro-finance • External reviews.
Sector (CRISP) (savings and credit) to the economically • Regular assessment of profitability to determine feasibility of expansion.
Zimbabwe active poor. • Concern is mainly outreach and business sustainability.
• Donor interest in number of jobs created and sustained.
DCFU Group (DG), SBF, MF, Development finance company offering • Internal concern is profitability and sustainability.
Uganda ML small and medium business finance, • Donors (shareholders) require socio-economic assessment.
(formerly Development wholesale finance for MFIs, and leasing • Annual report.
Finance Company of products. • Client surveys as required by funders.
Foundation for ET, CD Focuses on providing and developing • IA approach is being developed. Baseline survey identified parameters,
Entrepreneurial and entrepreneurial skills to individuals, data capture methods and standard baseline data and records required
Business Development companies and educational institutions. for IA. Also identified social indicators, baseline information and on-going
(FEBDEV), South Africa Main programme focus on schools, M&E data questionnaire. Currently track achievement of objectives
colleges and enterprises. through progress reports and want to assess impact at 3 levels of
educators, learners and the institution based on existing M&E tools.
• Baseline snapshot included data collected through questionnaire, FGDs,
case studies and educational institution staff workshops.
• Challenge in reconciling donor vs. organisational needs for IA. A
comprehensive range of indicators is required by the donor including
impact on capacity, attitude, access and livelihoods.
• IA approach varies across programmes and levels of assessment
FSA International (FSA), MF Micro-finance services (shares, loans, • External evaluations of achievements required by funder (OPR) include
5. Programme Type Description Key elements in IA approach include:
Uganda savings account) through village banking. wider socio-economic impact.
• Monthly social audit report produced.
• Quarterly random surveys of stakeholders.
IDPM & EDA Rural R Research project to understand the • 2 research sites 1 urban 1 rural
Systems “Money market for financial services among the • 3 different techniques used over a period of a year:
Matters”, India poor in North India. Funded by DFID. – Financial diaries of 48 households compiled - record of financial
transactions compiled through open and unstructured interviews
– More detailed diaries for a further 92 households (2 rounds of
interviews on financial needs and preferences)
– Interviews with financial service providers which respondents had
said were especially important to them.
• Participatory Wealth Ranking used to identify a broad range of
ITDG’s Light MLO, Manufacturing and production equipment • Ongoing impact assessment within operations and the project
Engineering Project EQ, ML hire and purchase for artisans. Linkages management cycle.
(ILEP). Zimbabwe are also made between manufacturers • MIS to capture routine project data.
and the market. • Regular and random impact assessment of buyers.
• Surveys of wider impacts on livelihoods.
• IA used for strategic planning.
Just-X (JX), South FT, MLO Marketing support and fair trade • Evaluations through external consultants (some as a requirement of
Africa (closed) organisation assisting small producers to main funder) assessing effectiveness, sustainability, outreach, and
be more effective in local and export efficiency.
Kashf Foundation (KF), MF, CD, Provides micro-finance (loans and • At the early stages IA is seen as a capacity building and learning
Pakistan ADV deposits), social advocacy and capacity initiative to improve practice
building for poor women. • Client exit surveys.
• Client attitude survey.
• Poverty Assessment Study.
K-Rep Holdings Ltd (K- MF, Holding company for K-Rep Bank, which • In a learning organisation that innovates, tests and implements, IA is
Rep), Kenya Cons, R provides a range of microfinance products measured both from an organisational and product performance
and services (savings and credit to micro perspective using indicators related to profitability, portfolio quality, client
and small enterprises) and K-Rep records, client incomes, and outreach.
Development Agency which is an NGO • Methods used include: regular group meetings FSAs with staff
doing microfinance product development monitoring and FSA monthly board meetings minutes; regular staff
and experimentation. Has also set up 43 meetings, day-to-day programme performance indicators, external
6. Programme Type Description Key elements in IA approach include:
FSAs reviews and evaluations, and research.
Kuapo Kokoo (KK), FT, MLO, Member based fair trade cooperative • Donor funded impact assessment.
Ghana MF dealing with marketing problems and • M&E system.
increasing FT market share for cocoa • Framework for analysis including commodity chain analysis, livelihoods
producers in Ghana. Also have 2 credit analysis, sustainability and cost effectiveness analysis.
MicroSave-Africa CD, R MicroSave-Africa builds the capacity of • Market research (MR) with a focus is on improving (not proving) impact
(MicroSave), Uganda (MF) African MFIs to provide secure, high of MFI services through seeking clients’ views on products and services.
quality savings services for poor people MR toolkits use qualitative methods such as PRA and FGDs, quantitative
through research, curriculum methods such as questionnaires.
development, training and dissemination. • IA case studies used for comparative analysis and lesson learning. IA
Activities include: uses FGD, PRA and individual interviews to assess client perceptions.
• Impact assessment tools for use in • Part of 3-year East African microfinance impact assessment programme
market research by MFIs which aims to improve MFI services through increased effectiveness of
• Conducts IAs for MFIs as case impact assessment systems. Will include developing indicators for IA,
studies improving MIS, qualitative research and comparative analysis.
• Coordinates regional impact project
as part of ImpAct
Mineworkers CD, A trade union initiated Business • Client data base capturing quantitative information.
Development Agency MLO, Development Services organisation • External gender appraisal.
(MDA), South Africa EQ, IS focusing on assisting people affected by • First full IA donor driven, done by external consultant led to set up of
large scale mine retrenchments to start M&E unit.
income generating projects and small • Baseline surveys and forms for capturing data as part of M&E system.
businesses. • Organisational review also followed the IA.
Oxfam Fair Trade (OFT), FT, MLO Fair Trade programme targeting • Participatory methodologies link IA to organisational learning and
UK disadvantaged producers to help them operations.
build their capacities to access markets, • M&E and IA are seen are part of the project management cycle.
both export and domestic markets • Participatory monitoring.
• External pilot IA focusing on impact of FT on improving livelihoods,
sustainability and the profile of beneficiaries.
Port Sudan Small Scale MF An ACORD network project providing • 2 small exploratory gender impact assessment
Enterprise Programme Islamic modes of credit to refugees and – gender review using participatory methods was undertaken in 1995,
(PSSSEP) poor Sudanese living in the slum areas of – study of the impact of the training programmes in 1998. 18 women
Port Sudan. Credit is mainly provided for in two groups were interviewed using qualitative interview
self-employment and small business techniques as well as participant observation in training sessions.
although sometimes for other purposes
7. Programme Type Description Key elements in IA approach include:
such as housing and sanitation. • Following the potentially negative effects that the shift to financial
sustainability had on gender a number of specific strategies were
adopted - enhancing women's participation in credit administration,
additional support to encourage group-based lending and networking
activities among women, enhancing gender awareness, continuation of
formal and informal community development activities, and informal
support for women's centres
Research International R, MF Research agency - case study focuses on • Impact of micro-finance assessed in terms of the effect on the wider
(RI) East Africa, Kenya exploratory study based on case studies household and livelihoods.
of clients of 6 MFIs to assess impact of • Attribution based on qualitative interpretation of evidence, rather than
their services on businesses and statistical inference i.e. client perspective on impact.
households. • Exploratory rather than conclusive research.
SERO Lease and MF, Micro-finance and micro-leasing • Random Baseline study at the start of operations.
Finance Company, ML, company making capital available to • Informal, spontaneous evaluations via weekly FGDs and loan officers’
Tanzania (SELFINA) BG, women’s and indigenous businesses visits to clients.
• IA is closely linked to monitoring operations in terms of daily payment
reports, weekly arrears reports, and a yearly financial report by external
• Independent assessments of client satisfaction.
SHARE Microfin Limited MF SML was part of an impact assessment • Used five AIMS (Assessing the Impact of Micro finance services) tools
(SML) study of 21 MFIs in 14 countries initiated • Used quantitative (survey data collection) and qualitative tools (focus
by Ford Foundation and Institute for group discussions)
Development Studies • Approach incorporates tracking poverty reduction over time based on
important indicators, plotting the position of households on a composite
index of poverty at different points in time.
SIBDI Foundation for MFI A foundation incorporating 31 micro • 5 MFIs representative of 31 MFIs partners representing different regions,
Micro Credit (SFMC), finance partner institutions models of microfinance, age, outreach, range of services
India • 2-4 sample areas at each MFI representing a typical area of the MFI
• Within each sample area stratifies random sample of clients, non-clients
and dropouts. Wealth ranking used for stratification
• Techniques included focus group discussions, quantitative survey
• Findings on impact in terms of outreach using wealth ranking categories,
use of group vs. individual loans, credit needs and credit sources, impact
of MFI on money lender, women’s empowerment, environmental issues
Small Business Project MLO Corporate sector initiative focusing on • IA is a tool for reviewing and developing strategy
(SBP), South Africa facilitating linkages between large • M&E is a basis for IA and includes ongoing (monthly) data collection and
8. Programme Type Description Key elements in IA approach include:
companies and small firms and on LED. quarterly monitoring reporting
• IA is an integral part of project log frame.
• Staff involvement
Small Enterprise MFI, Adv, MFI focusing on the provision of • Impact monitoring system with an emphasis on participation and
Foundation (SEF), South CD sustainable microfinance services (credit organisational learning.
Africa and savings promotion) with poverty • Focus on improving policy rather than proving impact.
targeting and using group based • Elements include conventional IA studies, case study research and
(Grameen) methodology. continual impact monitoring by staff
• IA findings feed into programme policy; has led to changes in the
programme (e.g. loan terms, sizes and increments).
• Participatory data collection and analysis based on participatory wealth
ranking (PWR) using indicators partly developed by clients / communities
combining poverty, business and social indicators.
• Specific IA studies on profiles of businesses, reasons for drop outs etc
• Integrated into field staff responsibilities
Training for Enterprise MLO Facilitates SME exports through the • Learning approach where IA is a tool for improving programme.
and Exports (TEEM), provision of demand – led business • IA planned during project design
Malawi counselling, product advice and export • Impact of each type of service is measured as well as overall impact on
marketing services. poverty.
Uganda Agency for MF, ET, Provides savings and credit (Trust Bank • M&E required by network partner (OI) as condition of membership.
Development Ltd CD model, solidarity and individual loans), training • Emphasis on understanding clients’ needs and business conditions as a form of
(UGAFODE) and counselling to micro-entrepreneurs. Loan market research.
products include trust banks, solidarity group • Network partner supports establishment of M&E systems and IA studies.
and individual loans. UGAFODE is a member • Ongoing monitoring includes monitoring repayments and arrears, and loan
of Opportunity International (OI). officer’s routine assessments of client’s business performance, case studies
tracked over time, weekly business unit staff meetings.
• M&E and IA feeds into programme design and strategic planning. The IA tools
are also seen as forming a basis for improvement in policies, work methods and
product design in order to meet clients’ needs.
• Recent IA study used both: qualitative (such as business practices, access to
social facilities, quality of life, self esteem and decision making abilities, social
relations and contribution towards community activities, political participation,
gender equity, participation in religious activities) and quantitative indicators
(enterprise resource and asset base, input supply, stock turnover and financial
performance, household income, asset base, community progress in terms of
• The following IA tools are being tested: FGD tool to test client satisfaction; in-
depth individual interview tools to test client empowerment and use of loans,
9. Programme Type Description Key elements in IA approach include:
profits and savings over time; quantitative client exit survey tool; and quantitative
Zambia HIV/ AIDS HIV, CD HIV/ AIDS information and awareness • Baseline behavioural surveillance survey to be follow up survey after a year.
Business Sector raising for Zambian companies. • Data collected monthly on peer education sessions and at clinics.
(ZHABS) Project • Quarterly follow up at clinics.
• In depth clinic quality assessments
• Involvement of community in investigating programme expansion.
• External IA as required by funders as part of mid-term review (OPR)