2. What we will do today
• Share best practice and network
• Introduce you to the tools and resources
• Discuss application and try tools
• Action plan
Inspiring Impact NI
3. Background to Inspiring Impact
• Collaborative UK wide programme
• Ten-year vision
• Challenging economic context
• Change in funding requirements
• Benefit of improved strategy and services
• More than just measuring impact
Inspiring Impact NI
5. Inspiring Impact NI programme
• Support to the VCSE
• Engagement with funders
• Establishing an Inspiring Impact Exchange
Inspiring Impact NI
6. NICVA’s experience
• Logic model
• Use of database
• Feedback from service users
• Longer term impact
• Continuous improvement
Inspiring Impact NI
7. Theory of change
Tool to help you describe:
• The need you are trying to address
• The changes you want to make
• What you plan to do
Often represented in a diagram or chart
Inspiring Impact NI
8. Elements in Theory of change
• Final goal
• Intermediate outcomes
• Activities
• Enablers
• Evidence
• Assumptions
Inspiring Impact NI
10. Group work
In groups please introduce yourselves and
discuss:
• What is impact?
• What is good impact practice?
• What are the challenges you face?
Inspiring Impact NI
14. Tips
• Collect the information you need
• Use a sample survey if collecting data from large groups
• Systematically record, manage and store it
• Have a clear idea of how you want to analyse, present, and
use the information you collect
• Be able to retrieve and analyse when needed
• Be able to report on the information in appropriate formats
Inspiring Impact NI
15. Assess
• It's impossible & unnecessary to measure and assess
everything you do –
• Analyse your findings
• Keep analyses simple
• Monitor and evaluate the full range of difference you are
making’, positive, negative and unanticipated impacts
Inspiring Impact NI
16. Review
• Learn from findings & use to improve what you do
• Disseminate findings – appropriate method to relevant
audiences
• Tell the whole story – the good , the bad, the ugly
• Build measurement into you everyday work
• Critique your methods of data collection and findings
Inspiring Impact NI
17. Disseminating and sharing learning
• Who
• How
http://www.actionforchildren.org.uk/media/9088373/impact-
report-2014.pdf
• Content
Inspiring Impact NI
19. Code of Good Impact practice
• Take responsibility
• Focus on purpose
• Involve others
• Proportionate and appropriate
• Consider full range of the difference
• Be honest and open
• Be willing to change
• Actively share
Inspiring Impact NI
28. Group Work for Code of Good Practice
Each group is given two of the principles to
discuss.
• What do you do currently related to these
principles?
• How can you improve what you do – think of
each stage of the Impact Cycle - Plan, Do,
Asses and Review
Inspiring Impact NI
29. Measuring Up
Easy-to-use impact measurement diagnostic system
Includes:
• Questions to help assess how well their current
measurement practices suit their needs
• Actions needed to improve impact practice
• Useful approaches and tools
Inspiring Impact NI
30. How Measuring Up can help you
• Identify your impact practice strengths and weaknesses
• Work towards an established standard of practice
• Improve the way you plan, evidence, understand,
communicate and learn from your impact
• Review your progress
• Understand what ‘good’ practice looks like for your
organisation
Inspiring Impact NI
31. Group work
1. Log onto the inspiring impact website
2. Choose one of the organisations round the
table as the “scenario” to use
3. Work your way through the questions under the
Plan section
4. Look at notes and add notes of your evidence
and action needed
Inspiring Impact NI
32. Impact Hub
Search by topic/sector/format/cost for the resources you
need to improve your impact practice.
Includes:
• Diagnostic tools
• Questionnaires
• Survey tools
• Performance management systems
• Research reports
Inspiring Impact NI
Less than a decade ago impact measurement was little-used and ill-defined, but now it has greater importance.
Measuring impact - two little words that have been embraced by some, but which can seem awfully cold and sterile to charities thinking about how to find out if what they do works.
A theory of change is a tool to help you describe the need you are trying to address, the changes you want to make (your outcomes), and what you plan to do (your activities). The approach can be used for organisations of all shapes and sizes—from service-delivery charities, to campaigning organisations, to funders.
Benefit to four keys areas
Strategy
Measurement
Communication
Partnership
Final goal: The broader social change a project or organisation is trying to achieve.
Intermediate outcomes: The short-term changes, benefits, learning or other effects that result from what a project/org does. These short-term steps will contribute to a final goal and may include changes in users’ knowledge, skills, attitudes and behaviour.
Activities: The things that an org/project does or the way it chooses to deliver a project day-to-day. Activities are within an org/project’s control.
Inputs: The resources that go into the project that a team or org needs to be able to carry out its activities.
Outputs: Products, services or facilities that result from an org/project’s activities. Expressed usually in numbers, number of users, how many sessions
Enablers: Conditions or factors that need to be present or absent to allow an org/project’s work to succeed. The presence or absence of enablers can help or hinder a project. There are 2 kinds of enablers:
• Internal enablers need to exist inside an org for a theory of change to work. Internal enablers describe the mechanisms by which an organisation delivers its work (such as the quality of services, relationships and the values and attitudes of staff).
• External enablers need to exist in the external environment for a theory of change to work, beyond an organisation or project’s immediate control. External enablers describe the context in which an org works (such as social, cultural, economic and political factors, laws, regulations, and working with other orgs).
Evidence: Info that you already have or plan to collect that is relevant to supporting or testing a theory of change.
Assumptions: The underlying beliefs about how a project will work, the people involved and the context. These are sometimes implicit in a logic model or theory of change, but it can be useful to state them explicitly.
Many orgs understand the need for good impact practice and want to get better at it. But they can encounter different definitions, confusing explanations, multiple methods, and sometimes contradictory advice. Staff understanding, in particular use of terminology is essential
Inputs (what we invest) what goes into the work – resources required. Staff, skills, budgets, equipment, buildings, policy environment
Activities/Processes - Project management/ operations, promotion, partnership working
Activities (the actual tasks we do)
Participation (who we serve; SU & stakeholders)
Engagement (how those we serve engage with the activities)
Outputs: what was done. Direct products or services, describing the activities delivered. Usually numeric, easy to count and are traditional form of reporting. How many people came on our programme, how many clients attained a qualification
Outcomes/Impacts: Outcome is the end result, something that follows from an action. Impact is the difference you make as a results of the work. Benefits gained by service users at a variety of levels - the consequences of outputs being made available
Short Term (learning: awareness, knowledge, skills, motivations)
Medium Term (action: behaviour, practice, decisions, policies)
Long Term (consequences: social, economic, environmental)
Impact
It’s a process not only a product
It’s a way to tell your story
It’s a way to measure success and explore potential for improvements
It’s a method for communications with stakeholders
It’s a area of increasing focus in government
Fish story
Input & Processes – parents get together fish, fresh vegetables, water, barley, species, pot and sour of heat
Activities – mother or father carefully prepare and cook ingredients
Output – children taste the most nourishing fresh soup in the world
Outcomes – Children consider the soup delicious and ask for it once a week
Impact – children group up healthy
Plan
Identify desired impact and how to deliver it
Know what to measure and data to collect
Do:
Delivery work
Collect information on the impact
Assess:
Analyse the information on impact
Draw conclusions on impact from findings
Review:
Communication you impact
Learn from findings and use to improve what you do
"Plan what you want to achieve before you measure it - and ask service users at the beginning what their targets are."
Deciding what to measure - is often done at the end of a programme, needs to be done at the beginning
Before you can begin measuring your impact, you need to be clear about what you are trying to achieve & what success looks like for your orf. Use the theory of change approach to help do this. Developing a theory of change clearly shows the key outcomes you need to measure to assess whether you are achieving your goals.
Many orgs setting out to measure their impact face common challenges, such as capturing intangible outcomes, accessing data & finding resources. Many are starting from scratch, using their own new methodologies, which can make it difficult to share and compare methods and results
Planning is essential:
Impact measurement can be made significantly cheaper if you plan from the outset. You must know what change you wish to achieve and understand the outcomes and outputs necessary to achieve your goals. Only then can you set up mechanisms to collect data at the appropriate time
Plan resources - How much should you spend:
A lot will depend on the scale, complexity, length and focus of your activity.
Many charities and funders already collect a lot of data about their activities and the people they help. But much of this data remains unused
Decide What is being collected by who, when, using what tool
Measuring impact starts at the beginning of each project (the when)
Once you have defined what you want to find out start thinking about what questions you need to ask
Choose the appropriate staff/team to gather the evidence (the who)
Choose an appropriate format for collecting information (interviews, questionnaires, focus groups, observations) (the tool)
Where possible use a case study to demonstrate your impact - where a person was before the intervention and what changed afterwards
Be sure you are robust on how you collect data to demonstrate impact
Evidence to demonstrate your impact and track your outputs is usually the most important information to gather.
Once you know the outcomes, research the best ways of measuring different types of social impact from developing questionnaires and surveys and accessing statutory data, to thinking through how to quantify soft outcomes
If you intend to show the ‘distance travelled’ by Service Users it will be important to do an initial interview, questionnaire, or survey when people first come into contact with your org & then at one or more specified points later on. You will want to ask about the same aspects of their lives – the areas in which your org hopes to make a change – at each point in time in the same way. These questions can be done either formally or informally as part of an ‘intake’ or initial induction process and as part of periodic check-ins or reviews at regular intervals.
Measuring up too using interviews and questionnaires to evaluate – data resource tool
What TOOL – interviews, questionnaires, focus groups, observations
Keep costs down - collect as much as possible in-house
Consider the resource implications
so don’t
Keep analyses simple - a spread sheet is useful for organising your information according to question asked and respondent– Excel, SPSS or when using qualitative data use typology (looking for trends or themes across your information
Build measurement into everyday work rather than viewing it as an unhelpful add-on, or something that comes at the end of a project or at the convenience of your funder.
Communicate your impact using appropriate formats
To who? – service users, staff, boards, funders, wider stakeholders
How: impact reports, annual reports, summary reports, info graphs, presentations, as part of induction and training sessions
(reports, impact statements, seminars, meetings, presentations, charts and infographs, case studies, websites, social media, video, podcasts)
content– report on the whole process, not just the numbers
Funders may require chapter and verse on every activity, they may want graphs and tables illustrating the data you have collected
Critique your methods of data collection and findings
