PMSD proj design and evaluation, rev4

Markets and Livelihoods Programme Impact Assessment Framework Training material for practitioners (Working draft, Ver. 4, May 2008)

MARKETS AND LIVELIHOODS INTERNATIONAL PROGRAMME (IA2) THE THREE BUILDING BLOCKS OF OUR IMPACT ASSESSMENT FRAMEWORK Measuring the impact of our projects Measuring the influence of our approach Measuring our own learning performance

SECTION 1 Basic elements of a project

The basic levels of a project and the logic behind them

Goal

Outcomes

Outputs

Activities

Inputs

Every time we design or implement a project we are telling this story: With the resources available (INPUTS), we do things (ACTIVITIES) to promote processes (OUTPUTS) which will transform reality (OUTCOMES) in ways that, we strongly believe, will have a positive IMPACT on a certain group of people (our GOAL)

INPUTS

They are the building blocks of any project (they define the budget):

Human resources

Time

Money

In-kind (normally from the community)

We have control over them

Are easy to measure and monitor

ACTIVITIES

The things we do with the inputs we have available

Some examples are:

Organising meetings, workshops and fairs

Building infrastructure

Providing credit or subsidies

Producing leaflets and books

Creating a website

We begin to loose some control (but not much)

We know if they happened

We can quantify and qualify them well

OUTPUTS

They are the processes we want to promote with the activities we do

They have three basic characteristics:

Are done by others (WHO)

With a purpose (WHAT FOR)

And in a particular way (HOW)

We can know if they are happening BUT…

the more they gain a life of their own, the more difficult it is for us to control them

EXAMPLE OF AN OUTPUT

A group of influential, organised, and empowered farmers (WHO)

explore business opportunities with other actors and communicate those opportunities to their communities (WHAT FOR)

in a transparent, inspiring and effective way (HOW)

OUTCOMES

They are the transformations we believe will occur in the world because of the processes we promote

In reality, they are partially caused by the people involved in our outputs

We can still know if they are happening but… we cannot control them directly and they are very difficult to control indirectly

We can only hope that we did things in ways that the transformations happen almost as we imagine

GOAL

It reflects our dreams, visions and aspirations for a better society

We can never say that the goal was achieved because of our project, but…

we can contribute to make it happen

Sustainability is critical here

A summary Within life of project Totally attributable to project Us (we are accountable) What we have INPUTS Within life of project Totally attributable to project Us (we are accountable) What we do with what we have ACTIVITIES Within life of project (but they can go beyond) Partially attributable to project Us and the actors we involve in the activities (we are still accountable) Processes we promote with the activities we do OUTPUTS Within life of project (but they normally go beyond) Their emergence should be largely attributed to the project. We need to find ways to influence their evolution but very difficult as time goes by

Actors we involve in the activities

Actors they involve in their initiatives

Other influential actors

Sometimes ourselves (e.g., advocacy campaigns)

Transformations in the contexts, processes and actors that influence the people we are interested in OUTCOMES Sometimes within life of project but longer than that in general Attribution is very difficult (but we have to try as hard as we can to show some evidence of connection between the impacts and our project) A wide diversity of individuals and institutions, and influenced by a wide variety of forces Contribution to sustainable changes in people’s lives (in areas of our interest) GOAL TIME-FRAME ATTRIBUTED TO CAUSED BY SHORT DESCRIPTION PROJECT STRUCTURE

SECTION 2 LOGICAL FRAMEWORK: Putting the project in one page

Remember this?

Goal

Outcomes

Outputs

Activities

Inputs

Every time we design or implement a project we are telling this story: With the resources available (INPUTS), we do things (ACTIVITIES) to promote processes (OUTPUTS) which we believe can transform reality (OUTCOMES) in ways that have an IMPACT on a certain group of people (our GOAL)

Well…

A logframe is just a tool to tell the same story in a very simple, short and schematic way

It is not perfect (like any other tool) but we can use it to our advantage:

To help us filter out the ‘rubbish’ when we design a project (it promotes economic and simple design)

To help us communicate our ideas effectively and clearly to most donors

To monitor and evaluate the project

… and it is at the core of the project cycle Source: http://www.wau.boku.ac.at/fileadmin/_/H81/H811/Skripten/811332/811332_G3_log-framehandbook.pdf

A familiar image (?)… Source: http://www.mande.co.uk/docs/PIFA_Article_PDF.pdf

SECTION 3 The Market Map: Backbone of the Impact Assessment Framework

The Market Map model

A market is a complex and open system composed of market actors who are:

interacting through economic, social, political, and cultural relationships,

interested in a particular set of products or resources,

making the best they can with the skills, assets and resources they have available,

and influenced by contexts, issues and forces that affect their business decisions

Graphic Representation of a Market Source: Springfield Centre

A generic market map BUSINESS ENVIRONMENT MARKET CHAIN BUSINESS AND EXTENSION SERVICES

SECTION 4 Our Intervention Approach: Participatory Market System Development -PMSD

Our approach in words

Using promising market systems, we can improve the livelihoods of a significant number of marginalised small-scale producers (MSPs) if we:

(I) promote empowerment of MSPs so that they can explore/exploit new business opportunities with other market actors and influence on issues that affect them,

(II) promote interactions amongst market actors to improve critical issues and relations that can have positive impacts on farmers, and

(III) promote communication of lessons and business models emerging from the project to inspire others to adopt and adapt them

Our approach in a diagram PROMOTE EMPOWERMENT OF MARGINALISED SMALLSCALE PRODUCERS PROMOTE INTERACTIONS AMONGST MARKET ACTORS PROMOTE COMMUNICATION OF EMERGENT LESSONS AND BUSINESS MODELS

TRANSFORMATIONS IN ONE OR MORE OF THE THREE LEVELS OF THE MARKET SYSTEM :

BUSINESS ENVIRONMENT

MARKET CHAIN

INPUT AND SERVICE PROVISION

IMPROVEMENTS IN THE LIVELIHOODS OF A SIGNIFICANT NUMBER OF MARGINALISED, SMALLHOLDER RURAL PRODUCERS

SECTION 5 Making sure our PMSD approach is effective -Monitoring and evaluating what we do-

Upper levels of the logframe in our PMSD approach

Goal

Outcomes

Outputs

EMPOWERMENT FOR ENGAGEMENT EMPOWERED MSPs EXPLORING MORE/BETTER BUSINESS OPPORTUNITIES FOR THEMSELVES AND/OR THEIR COMMUNITIES INTERACTION FOR TRANSFORMATION MARKET ACTORS (INCLUDING MSPs AND POLICY-MAKERS) WORKING TOGETHER TO CHANGE CRITICAL ISSUES AND RELATIONS IN THE MARKET SYSTEM

TRANSFORMATIONS IN ONE OR MORE OF THE THREE LEVELS OF THE MARKET SYSTEM:

BUSINESS ENVIRONMENT

MARKET CHAIN

INPUT AND SERVICE PROVISION

IMPROVEMENTS IN THE LIVELIHOODS OF A SIGNIFICANT NUMBER OF MARGINALISED SMALL-SCALE RURAL PRODUCERS COMMUNICATION FOR UPTAKE INFLUENTIAL ACTORS INSPIRING OTHERS TO ADOPT AND ADAPT LESSONS AND BUSINESS MODELS EMERGING FROM THE PROJECT

Goal

Outcomes

Outputs

Activities

Project Inputs

EMPOWERMENT FOR ENGAGEMENT INTERACTION FOR TRANSFORMATION IMPROVEMENTS IN THE LIVELIHOODS OF A SIGNIFICANT NUMBER OF MARGINALISED SMALL-SCALE RURAL PRODUCERS COMMUNICATION FOR UPTAKE BUSINESS ENVIRONMENT MARKET CHAIN INPUT/SERVICE PROVISION This is our arena as facilitators. This is the playground where we use all of our creativity, knowledge, experience and resources to facilitate the emergence of the Three Basic PMSD Processes (Outputs) Staff Money Time Infrastructure, etc

Principles that guide our ACTIVITIES

We are a temporary facilitator of PMSD; we are NOT a market actor

Therefore, all our activities should be designed to:

Build capacity of MSPs and other strategic market actors and policy-makers

Allow participation of a wide variety of market actors

Create appropriate conditions for interaction and experimentation

Learn, generate evidence and inspire

CORE PMSD ACTIVITY PHASES

Preparation (within the team):

Preliminary Market Mapping sessions

Meetings or workshops for discussion, reflection, design, etc

Studies, research, assessments, evaluations, etc

Training (capacity building for the team)

Constitution, Strengthening and/or Promotion of MOGs

Participatory Market Mapping Workshops (normally more than one):

Stakeholder analysis (identification of participants, incentives, hooks)

Definition of agenda and logistics (invitations food, lodging, transport)

Production of materials

Identification and hiring of experts (keynote speakers, facilitators)

Constitution, Strengthening and/or Promotion of IFs

Follow-up, facilitation and promotion of Action Plans

Knowledge Management, M&E and Learning

Influence and production and/or diffusion of evidence

‘U2’ ACTIVITIES (EXAMPLES)

We can undertake them only if they are absolutely necessary to unlock and unleash (U2) the market system and making sure that we have an exit strategy within the project cycle :

Influence and advocacy meetings with influential actors

Vouchers, Guarantee Funds, Purchase Order Finance

Fairs (for seeds and other products)

Awareness raising campaigns and materials

Construction of infrastructure and provision of technologies for demonstration

Demonstrative farms

Training workshops, internships, exchanges

Meetings, seminars and conferences

Generic Activity Phases of PMSD “ UNLOCK AND UNLEASH” (U2) ACTIVITIES KM AND LEARNING PRODUCTION AND DIFFUSION OF EVIDENCE INFLUENCE (AT BOTH PROJECT AND PROGRAMME LEVELS) Elements in orange are CORE PMSD ACTIVITY PHASES PREPARATION M.O.Gs PMMWs I.F.s ACTION PLANS

EXAMPLE OF ACTIVITIES WE MUST AVOID

Providing or subsidising inputs and services without an exit strategy

Buying products at any point of the market chain

Doing marketing on behalf of any market actor

INPUTS

The same as for any other project:

human resources

money

time

in-kind (normally from the community)

SECTION 6 Indicators: telling a ‘good’ story

WHAT ARE INDICATORS?

Indicators tell us whether things are happening in the project

Indicators are the main elements that allow us to share the project with others and convince them of its benefits

They also tell us when things are not going well or as expected

DOING ‘SMART’ M&E

An M&E system (and the indicators it relies on) should be:

S pecific

M easurable

A chievable and Attributable

R elevant and Realistic

T ime-bound, Timely, Trackable, and Targeted

SECTION 6 Case study: The livestock project in Kassala (Sudan) – Wellcome Trust

Introduction

The Wellcome Trust is a very demanding donor: since 2003, in six different occasions, Practical Action applied to their funds without any success

This project was designed taking into account our most updated PMSD model and international experiences

The project was approved in 2005

REMEMBER THIS?

The GOAL reflects our dreams, visions and aspirations for a better society

We can never say that the goal was achieved because of our project, but…

we can contribute to make it happen

Sustainability is critical here

PROJECT GOAL

To contribute to

the long-term improvement

of the livelihoods and well-being

of 220,000 marginalised pastoralists

in the state of Kassala in Eastern Sudan

REMEMBER THIS?

OUTCOMES are the transformations we believe will occur in the world because of the processes we promote

THE WORLD WE ARE INTERESTED IN IS THE MARKET SYSTEM (AROUND A SPECIFIC SUBSECTOR)

PROJECT OUTCOME 1

BUSINESS ENVIRONMENT (influence and representation):

“To increase the organisation and political influence of the pastoralists around issues related to animal health and livestock markets at the local level, and improve their linkages with other stakeholders in the livestock subsector”

PROJECT OUTCOME 2

MARKET CHAIN (livestock market chain productivity):

“To increase the productivity and dynamism of the livestock market chain, paying special attention to the interactions and collaboration among key market stakeholders, and to critical problems related to livestock rearing and processing ”

PROJECT OUTCOME 3

SERVICE PROVISION:

“ To promote market-based service provision to pastoralists in the areas of information and training . Information contents will be focused on animal health and livestock market issues. Training will be focused on animal health and livestock improvement techniques, prioritising those techniques that are rooted in local or traditional knowledge”

REMEMBER THIS?

OUTPUTS are the processes we want to promote with the activities we do (the activities we do set them in motion)

They have 3 basic characteristics:

Are done by others (WHO)

With a purpose (WHAT FOR)

And in a particular way (HOW)

PROJECT OUTPUTS EMPOWERED SSPs EXPLORING MORE AND BETTER BUSINESS OPPORTUNITIES FOR THEIR COMMUNITIES (Key allies: Market Opportunity Groups) MARKET ACTORS (INCLUDING SSPs AND POLICY-MAKERS) WORKING TOGETHER TO CHANGE CRITICAL ISSUES AND RELATIONS IN THE MARKET SYSTEM (Key partners: Interest Groups and serv. provid.) The Pastoralist Marketing Group (MOG) The Livestock Forum (IG) Productivity Action Plan Productivity Workshops Itinerary Paravets Livestock Service Network INFLUENTIAL ACTORS INSPIRING OTHERS TO ADOPT AND ADAPT LESSONS AND BUSINESS MODELS EMERGING FROM THE PROJECT (Key allies: movers & shakers and mass media)

Productivity Workshops (example of OUTPUTS)

“ The project will create spaces for the pastoralists to discuss with their representatives in the Marketing Group about the early drafts and final version of the Productivity Action Plan. These workshops will be a communication mechanism that will increase the influence of the wider pastoralist communities in defining the Plan, which in turn is likely to become a mid-term navigation chart for a wide variety of stakeholders in the livestock subsector in Kassala”

EXAMPLE OF A PROJECT ACTIVITY