Basic monitoring and evaluation for beginners in Monitoring and Evaluation course.

1. INTRODUCTION TO MONITORING AND EVALUATION
1.0 What does monitoring and evaluation mean?
a) Monitoring is the routine and systematic collection of information against a
plan. The information might be about activities, products or services, users, or
about outside factors affecting the organization or project.
b) Evaluation is about making judgments about the value of any component
part of an organization or its products, services or benefits, or about the
organization as a whole.
Note: Monitoring and Evaluation is a continuous process.
1.1 Why is monitoring and evaluation important?
Organisations use monitoring and evaluation for two key purposes:
 to learn about their own activities and results, and to support internal
planning and development
 to be accountable to their stakeholders.
Organisations need evidence of their efficiency and effectiveness for funders,
commissioners and investors. They also need to communicate achievements to
a wider public. Charities are now specifically required by SORP to report the
achievements of the charity over the year. However, the role of monitoring and
evaluation in providing learning and in improving the organisation is of equal
importance.
1.2 The monitoring and evaluation cycle
There are four key phases in the monitoring and evaluation cycle:

2.  planning for monitoring and evaluation
 monitoring
 evaluation
 using the findings to learn and improve.
Figure 1: The Cycle

3. The first phase of the monitoring and evaluation cycle - planning.
1.3 The importance of planning
Planning is a key stage in the monitoring and evaluation cycle. Organisations
should think about:
 Why do we want to do it and for whom?
 What will we monitor and evaluate?
 How will we do it?
 Who will do it and do we have the right skills?
 When will we do it?
 What resources will we need, including outside support?
 What will we do with the information we get?
Organisations themselves, and their stakeholders, will usually want to find out
how the organization or project is meeting its planned aims and objectives.
These need to be clearly stated in the early stages of planning.
Defining aims and objectives will help you agree on planned outcomes. Many
funders and commissioners will ask for outcome information as part of their
monitoring requirements.
1.4 Integrating monitoring and evaluation
Monitoring and evaluation systems should be developed when an organisation
first starts up, so that collecting information becomes part of everyday life.
Often there are later opportunities to expand, develop or focus monitoring or to
carry out more developed evaluation:
 when an organisation is changing or developing
 if it is applying for or has received new funding
 when information is needed for assessment against quality standards

4.  when information is needed for strategic planning.
1.5 Developing a monitoring and evaluation plan
Drawing up a monitoring and evaluation plan will help the organisation to
think about how it will carry out monitoring and evaluation over the year or
over the lifetime of a project. It will consider:
 When will different information be collected?
 Who will carry out and manage the activities?
 How will information be collected, analysed and presented?
 How will monitoring and evaluation evaluation findings be used in short-
term planning and decision making, and quality reviews?
You will then be able to prioritise the information you need to collect regularly
and on an occasional basis by being clear about the key questions, such as:
 How are resources being used?
 How appropriate is the management structure?
 How well are we meeting identified needs?
 How do we fit within a network of services?
 How well have we met our expected outcomes?
 What were the unexpected outcomes?
 What lessons did we learn?
Organisations may focus on different questions at different points in time, and
they may need different types of information gathering.
The second stage of the monitoring and evaluation cycle: what and how to
monitor.

5. 1.6 A monitoring framework
All organisations keep records and notes, and discuss what they are doing.
This simple checking becomes monitoring when information is collected
routinely and systematically against a plan.
Strategic aims and objectives and annual operational plans will indicate the
broader areas and the more specific areas for monitoring. Funders and
commissioners may ask for information that is not felt useful by the
organisation, but there may be room for flexibility and negotiation.
Monitoring should be proportionate to the size of the organisation and level of
funding.
1.7 What do you monitor?
I. Inputs: As well as monitoring expenditure, some organisations will
monitor other inputs, such as staff and volunteer time.
II. Outputs: These will be specific products or elements of service provided
by the organization, such as advice sessions, training days or
publications.
III. Outcomes: These are related to the changes or difference made as a
result of the organization’s outputs.
IV. Impacts: These are related to longer-term or broader changes. These
might be more difficult to monitor on a routine basis.
Note-Read more about monitoring and evaluating outcomes and impact.

6. 1.8 Setting indicators
Once a broad framework for monitoring has been set up, making clear the
planned outputs and outcomes, some realistic indicators should be identified,
against which information can be collected.
1.8.1 Indicators are specific, observable characteristics that can be assessed
or measured to show the quality or quantity of aspects of the organization, its
resources, its process, or the results of its activities.
The most common indicators relate to outputs and outcomes, and you will
need to distinguish between the two.
1.8.2 Output indicators: these demonstrate the work the organization does
and show progress towards meeting its objectives.
1.8.3 Outcome indicators: these demonstrate changes which take place as a
result of the organization’s work, and show progress towards meeting specific
aims.
You may want to set targets against your indicators, and your monitoring will
demonstrate whether these have been achieved.
Performance indicators can be quantitative (numerical) or qualitative (related to
people's perception or experience). They are usually developed when setting
aims and objectives, although you can set them later on if necessary.
1.8.4 Things to consider when setting performance indicators
 What activities and services do you provide?
 What are you trying to achieve for each of your stakeholder groups?

7.  What would you expect to see at different points in the process and at
the end that indicates you will achieve or have achieved your aims and
objectives?
 What standards of performance achievement do you already have, for
example requirements from funders, local authorities or regulatory
bodies?
Your performance indicators should reflect the range of your activities and the
perspective of the stakeholders interested in your work.
Remember when setting performance indicators that 'what gets measured gets
done'. If you only set three performance indicators, this is what people will
focus on and other areas of your operations may be neglected. Aim to set
indicators which are both qualitative and quantitative and incorporate the
breadth and depth of your work.
1.8.5 Examples of performance indicators
An after-school club might have performance indicators which include:
 The number of children who attend the club. This will be of interest to
funders who want to see the number of children who benefit from their
investment.
 The number of staff to children ratio. Parents in particular will want to
know how many adults are keeping an eye on their children.
 The perceptions of children who attend. This will be of interest to a
number of stakeholders. In particular, parents, children, staff and
volunteers will want to know if the children enjoy the experience.
 The numbers of staff and volunteers who have been checked by the
Criminal Records Bureau. This again will be of interest to several
stakeholders, but is above all a regulatory requirement.

8. 1.9 Collecting monitoring information
Finding ways to collect monitoring information is a key part of monitoring and
evaluation. Tools for collecting data can be grouped under four basic heads:
 surveys and questionnaires
 observation
 interviews
 keeping records and notes.
1.9.1 Questions to ask when choosing information collection methods
Before deciding on data collection methods, you should address a number of
questions, for example:
 What depth and type of information do you want?
 How can you check the reliability of your information?
 How much time can you afford to spend?
 How much will it cost?
 How will you analyse the information?
1.9.2 Managing your monitoring information
Always bear in mind how you want to use the information before you start to
collect it. This will influence what you collect, how you collect it, and how you
store and process it. Make sure that everyone involved in collecting the
information is using the monitoring tools in the same way, so that the
information is consistent and there are no gaps. Small amounts of information
may be stored on paper and analyzed manually. However, many nonprofits now
use computers to manage their information.

9. If you don’t resolve practical issues concerning how you manage your
information, this can hold you back in terms of reporting and using monitoring
information to feed back into organizational improvement and management.
The third phase of the monitoring and evaluation cycle: internal and external
evaluation.
As well as finding out what evaluation means and who is responsible for it, this
page also provides information on:
1.10 What is evaluation?
Monitoring information may be reported on a quarterly or even a monthly
basis. Evaluation – taking stock and making judgments – is likely to be carried
out less frequently.
Evaluation often uses monitoring information that has been produced over a
period of time. However, this is not always available, and evaluation may use
one-off methods of enquiry. Often the best combination is bringing together
regular monitoring data with additional information gathered against specific
key questions.
An organization should think about evaluation from the start, so that
monitoring can be carried out with evaluation in mind.
1.10.1 Who does the evaluation?
Who carries out the evaluation may partly depend on the amount of money you
have available for it. There are other issues to consider, such as whether your
organization values building skills of analysis and reflection internally or
whether it values involving users.

10.  Self-evaluation is when an organization uses its own expertise to carry
out evaluation.
 External evaluation is when an outside evaluator is brought in to carry
out an evaluation for the organization.
These might be combined, depending on costs, the amount of time required
and available within the organization, the type of evaluation required and the
skills involved.
Self-evaluation
With self-evaluation, monitoring and evaluation is built into the everyday life of
the organization and is an essential part of organizational management and
development.
1.10.2 Why do self-evaluation?
 It can build on and complement internal reporting systems.
 It can be more easily integrated with other work processes.
 It can tap into internal knowledge more easily.
 Control and use of the information stays within the organization.
 It encourages greater learning and a more reflective approach.
 It is more likely to lead to change.
 The process can develop new skills in those involved.
 It may be cheaper.
1.10.3 External evaluation
External evaluation may be appropriate when an organization lacks the skills
or time to carry out an evaluation itself or wishes to carry out a more in-depth
evaluation than its own resources would permit. Sometimes funders will
require an external evaluation of a project as a condition of their funding.

11. Why have an external evaluation?
 External evaluators can bring an external perspective.
 The scale and scope of the evaluation may make it more suitable for
experienced evaluators.
 Funders may request an external evaluation and it may be more credible
to an external audience.
 It should involve less time commitment internally.
Managing external evaluation
A number of decisions need to be made when you are planning an external
evaluation:
 How will you look for and select an evaluator?
 How will the evaluator work with the organization?
 How much data collection will be done internally and when will internal
monitoring data be available?
 When will the report be needed?
 Who is the primary audience for the report?
1.11 Evaluation approaches
There is a significant difference in approaches to evaluation. Some
stakeholders may value a ‘scientific’ approach, looking for hard measurement
and objective evidence. Others may value evaluation that encompasses
description and different perspectives.
1.11.1 Evaluating aims and objectives: This approach focuses on monitoring
and evaluating an organization’s progress and effectiveness in delivering
planned outputs, outcomes and impacts. Information is gathered against
selected indicators.

12. 1.11.2 Case study evaluation: allows you to examine a particular individual,
event or activity in greater detail. This approach may be particularly
appropriate when evaluating certain types of projects or for certain audiences.
If you have a number of case studies or stories to tell, you will then be able to
compare and contrast them and pull out any common themes. You may want
to let participants tell their story in their own words. If so, you will need to tape
an interview or get permission to quote a diary record or other form of self-
report.
1.11.3 Appreciative enquiry: This approach focuses on what has worked well;
often telling stories in order to learn about a project and its outcomes.
1.11.4 User-led evaluation: In a user-led evaluation control passes to the
users themselves or to user-representatives. This may suit your organisation if
your work is particularly focused on building the capacity of users and
involving them in the delivery of services. However, it is important to provide
enough training and support to users who are leading the evaluation.
1.12 Ethical considerations in evaluation
The following are ethical considerations when carrying out monitoring and
evaluation.
 Storing information in accordance with data protection guidance
 Obtaining informed consent
 Ensuring that you don’t breach confidentiality
 Making sure that an individual or organisation is not harmed by how you
work or what you report
 Observing copyright and intellectual property when using secondary
materials (such as articles and other organization’s data)

13. Importance of evaluation findings
 Feedback for projects and activities
 Assess progress and results
 Document Lessons learnt
 Influence policy change.
1.13 Analyzing the information
Small amounts of information can be analysed manually, but a good ICT
system will save time and provide the information to meet different needs. The
kind of ICT system needed will depend on the size and type of organisation,
and needs to be assessed within the context of an overall ICT strategy.
Some of the software systems available have been designed for a specific
subsector. You will need to check whether the system can be used to assess
and report outcomes, and whether it will help you to report in different formats
and to meet different needs.
Interpretation means asking what the results mean in relation to the
evaluation questions and judging what is significant. Linking your findings to
the key questions posed will lead to conclusions and recommendations.
1.14 Presenting findings
You need to agree early on who your audience is and how you want to present
monitoring and evaluation findings. For example:
 in a formal report
 in a visual display
 on your website
 through discussion and workshop presentations.

14. Most reports will cover the following:
 an introduction
 the aims and objectives of the evaluation, including the main evaluation
questions
 how the evaluation was carried out
 findings
 Conclusions and recommendations.
1.14 Learning from evaluation
Organisations and their funders are increasingly thinking about how
evaluation findings can be shared and used to learn about approaches to
service delivery, so that success can be replicated and so that policy can be
influenced. Lessons can be learnt from initiatives and projects that don't work,
so these findings should be shared as well. Putting your reports on your
website and sharing findings in conferences and workshops can make them
doubly useful as a resource.
The importance of clarity before you start your monitoring and evaluation.
An organisation should be clear about its aims and objectives. Once aims and
objectives are clear, it will be easier to agree on planned outputs, outcomes and
impacts.
Aims
Aims are the changes that an organisation is trying to achieve.
Objectives
Objectives are the methods or the activities by which an organisation plans to
achieve its aims.

15. Description of diagram
The organisation's overall aim is directly linked to its impact. Objectives feed
into the 'what we do' of the organisation and inform its outputs. Specific aims
feed into the question of 'why we do it' and inform the organisation's outcomes.
Outcomes
Outcomes are the changes, benefits, learning or other effects that happen as a
result of your work. They can be wanted or unwanted, expected or unexpected.
For example, the outcomes for users of a refugee centre might include:
 improved access to services
 improved English language skills
 reduced isolation.
Outcomes can occur in many places, so it is important to think through who or
what your organisation is aiming to affect: what differences you are planning to
make. For example, change could occur in:
 individuals
 families
 communities
 organisations

16.  the environment
 policy.
What is impact?
The term 'impact' is used in different ways. It can be seen as the broad, longer-
term effects of your work, or the institutional changes it brings about. However,
for nonprofits it is also used to mean the outcomes achieved for your
immediate user group, whether they occur in the short or long term. The
crucial distinction to make is between what you do (that is, your outputs, such
as services offered) and what you achieve (that is, your outcomes, the changes
that result from what you do).
The importance of demonstrating impact
It is important to communicate with your stakeholders and the wider public
about your work and the difference you make. Information about outcomes has
also become part of regulatory requirements. Foundations and other
grantmakers are requiring evidence of measurable benefits to clients and
commissioners are also focusing more on outcomes. More than that, getting
information as you go along about how effective you are in achieving positive
change is crucial if you are to achieve even more for your client group.
Identify a realistic level of change
Certain types of work have intended impacts that take many years to achieve.
It may be difficult to get good information over time, so intermediate outcomes
may be more important to identify in order to do justice to your work, for
example in relation to improved skills and confidence and readiness for work.
You may also agree with your funding bodies certain outcomes that you can
report on which will contribute towards a desired area of impact.

17. Standards of evidence
Evidence of changed outcomes relies on comparing the situation before and
after your activities. Even with good outcomes information, it may be difficult
to demonstrate that the outcomes are a result of your intervention. To
overcome this, evaluations sometimes do a comparison study by setting up a
control group and some argue that this will provide a better standard of
evidence.
Potential funders and other stakeholders may find the results of an evaluation
using a comparison group more persuasive. But a matched comparison group
should be the same in all respects – in age, gender and socio-economic factors
– other than the fact that one group only receives the intervention and this can
be difficult to establish. It may be even more difficult to get reliable information
on the control group if you are relying on external sources for it. For most
small nonprofits a control group will not be appropriate, and even for larger
ones, the investment of time and resources should be set against what might
be limited gains.
Get help with evaluating your outcomes and impact
Many organisations are successfully working together with consultants over a
period of time to improve the quality of outcome and impact information, and
this can be a relatively cost-effective way of improving your reporting on
impact. Consultants can help you to establish a monitoring and evaluation
framework, to identify appropriate data collection tools or to develop your own.
They can carry out interviews to supplement internal monitoring, help with
data analysis and write reports.
What is outcomes-based accountability?
Central to this is a process called ‘turning the curve’ which compares progress
against what would have happened over time if nothing changed.

18. What is social accounting?
Social accounting is a framework for monitoring and evaluation that links a
systematic reporting of organisational activities to the issue of social impact
and the ethical behaviour of an organisation. The social accounting process
can be quite flexible and may look like, or incorporate, other evaluation
processes. However, there is greater emphasis on public accountability and a
social audit panel reviews the report or social accounts and only then will the
report be available to a wider public.
What is Social Return on Investment (SROI)?
SROI is a way to measure and account for the value you create with your work.
Currently there is a lot of interest in the approach from funders and nonprofits
and from both public and private sectors. It is a type of cost-benefit analysis,
which is playing an increasingly important role as nonprofits are required to
carry out more hard-headed analysis of the value and social impact of their
work.
SROI is an outcomes approach that puts monetary value on social and
environmental benefits relative to a given amount of investment. For example,
an organisation might have a ration of £4 of social value created for every £1
spent on its activities.
SROI may not be appropriate for many organisations, but the SROI
methodology could help make a good case for providing certain types of
services and is especially useful if an organisation’s funders require outcomes
information in financial terms. Organisations that have already done some
work on identifying and measuring their outcomes will find SROI much easier.

19. 1.15 Key terms in monitoring and evaluation
i) Aims
Areas of change or difference the organisation or project plans to bring about.
ii) Objectives
Areas of activity or steps an organisation or project plans to achieve its aims.
iii) Inputs
Inputs are the resources that are put into the project, such as staff and
volunteer time, funding and technical resources.
iv) Outputs
Products, services or facilities that result from an organisation or project’s
activities.
v) Outcomes
Outcomes are the changes, benefits, learning or other effects resulting from an
organisation’s activities.
vi) Impact
‘Impact’ can be used inclusively to mean the various levels of change or effects
of an organisation. It is also used to mean the longer-term, broader or
cumulative and sustained effects of an organisation’s outputs and outcomes.
vii) Indicators

20. Well-defined information that shows how an organisation or project is
performing.
viii) Target
A defined level of achievement which a project or organisation sets itself to
achieve in a specific period of time.
ix) Milestone
A well-defined and significant step towards achieving a target, output, outcome
or impact.