Doing Our Best: Measuring Impact

WELCOME & INTRODUCTIONS
Fintan O’Toole
CHAIRPERSON

Tweet your views about
the conference
#iii2012 BHP Insurances Ltd is regulated by The Central Bank of Ireland

Innovate Involve Inspire 2012 conference

ADDRESS FROM DEPARTMENT OF
ENVIRONMENT, COMMUNITY & LOCAL
GOVERNMENT
Rita McNulty
Assistant Secretary

the conference

Rita McNulty, Assistant Secretary,
Department of the Environment,
Community
and Local Government
The Wheel Annual Conference
2012

Tackling the Problem of Diminishing
Resources
What to do? • Demonstrate
• Make the most out Impact/Prove it
of available • Involve - Citizen
resources Participation
• Plan with the end • Be accountable,
in mind, best make democracy
practice evidence participative

HOW TO SQUARE THE CIRCLE?

• Two-thirds of
• Charities’ decrease charities have seen
in income: 60% in an increase in
last three years demand for services
• Reliance on
diminishing state
funding

Changes in the Community and Local
Government Sectors in the DECLG

Consider the changes
in the local government sector
& the alignment of its community,
local and enterprise remit
with that of local and community
development programmes

Changes in the Community and Local
Government Sectors in the DECLG

• What opportunities arise for the C&V
sector in relation to the second tier of
government?

The need for the C&V sector to
position itself to be a key player in
a reformed local government
system

The Forum on Philanthropy and
Fundraising – assisting with
sustainability in the C&V Sector

SESSION 1
INNOVATING
How can we deliver the best outcomes for people and greatest
impacts for communities?

the conference

ARE WE DOING THE BEST THAT CAN BE DONE?
Nuala Doherty
Chief Executive Officer, Centre for Effective Services

the conference

Doing our best
Are we basing our work on evidence of best
practice?

Nuala Doherty
31st May 2012

Overview

• What is good evidence?

• Best Practice - how are we doing?

• What are the challenges?

• Ways CES supports evidence into practice

14

What is ‘evidence-informed’?

‘An approach that helps people and
organisations make well-informed decisions
by putting the best available evidence at the
heart of practice development and service
delivery’.

(Sandra Nutley, 2010)
15

Types of evidence

Consultation

Research

Practice Policy
Wisdom

16

Finding the evidence you need

© The Centre for Effective Services 2011
17

The Why...

The puzzled practitioner?

18

What practitioners tell us…

19

What exactly are you looking for?

20

How are we doing ?
• Increasing focus across the world, including Ireland – PEIP GUI
• Commissioning of evidence reviews and ‘what works summaries’
by public policy bodies
• Practitioners more confident using evidence to strengthen their
work
• The public are more informed and knowledgeable
• Researchers getting better at applying findings to practice
• More understanding of achieving better outcomes for children ,
families and communities
• Greater awareness of measuring the impact of our services

21

Some challenges...

• The economic climate – it’s a ‘luxury’
• Limits to the evidence –Informed approach
• Limits to research literacy amongst practitioners
• Time constraints
• The best evidence in the world will not result in
better outcomes if poorly implemented

22

Some more technical challenges...

• Selecting and weighing evidence
• Contested definitions of evidence
• Research V other sources (experiential and
practice wisdom
• Dearth of research in some areas
• Generalisability and transportability of research
evidence can be limited (need to blend local and
international)

23

For practitioners

• Focus on outcomes
• Take a broad approach to evidence
• Find reliable sources of useable evidence
• Distil out the ‘active ingredients’ in effective
practice
• Evidence helps us make our assumptions explicit so
that we can check and test them
• Reflective practice.
24

For managers

• Drive an outcomes approach to the work
• Include measures of social impact
• Promote a culture of evidence
• Underpin own practice with evidence
• Creating spaces for reflection
• How to support, train and develop staff
• Join up learning across sectors

25

We need to do more by...

• Routinely gathering, collecting and using data in a
systematic way to inform our decision-making
• Doing better by knowing more and making
connections
• Be open minded and question accepted
orthodoxies
• ‘Evidence’ is not a magic bullet
• Implementation is key to better outcomes

26

If you are interested in ‘evidence based/informed practice’

• www.philanthropycapital.org
• www.ceni.org
• www.scie.org
• www.effectiveservices.org
• www.cssp.org

27

CES promoting evidence informed practice

• Prevention and Early Intervention Projects
• Practice seminars e.g. Sourcing Useable Evidence
• Strategic Advice and Support
• The What Works Process
• CES Library

28

Nuala Doherty

ndoherty@effectiveservices.org

ARE WE DOING OUR BEST TO BE THE BEST?
Sean Coughlan
Chief Executive Officer, Social Entrepreneurs Ireland

the conference

Are we doing our best
to be the best?

We need a plan

(and we need
to check it as
we go)

Are we doing our best to be
the best?



4 Key Questions  What
 Why
 How
 Is it working

taken from

Vision /
Ambition
Why

Mission / Purpose

What
Strategic priorities

Strategic objectives

Action Plan How
(activities, responsibilities, timelines)

Impact Measures & Key
Performance Indicators Is it working

common tools

Why What How Is it working

Theory of
Change  

Strategic Plan
 
Operations
Plan   
Measurement
Framework 
Logic Model   

Doing Good Is Not Good Enough …

… we must strive to be the best at what
we do

Present Better
future

A B
Strategy Communications

Operations Leadership

Sustainability Measurement

Present Better
future

1. We need an appropriate plan that can answer
4 questions: What, Why, How and Is it working

2. We need to follow the plan and course correct
when needed

Present Better
future

Thank you

HOW DO WE KNOW WE ARE DOING OUR BEST?
Sandra Velthuis
Whitebarn Consulting

the conference

How do we know
we are doing our best?

A presentation by Sandra Velthuis of
Whitebarn Consulting for The Wheel

31 May 2012

The maximum impact challenge
• Delivering impact is
more important than
measuring impact.
• But without
measurement, how do
you know you are
having an impact?

Measurement
• You can measure anything …
• But measured does not mean mattered.
• If you treasure it, measure it!
• Meaning is more important than metrics.

Some basic principles
• Understand how what you do leads to
change.
• Focus monitoring/evaluation activities
on those that demonstrate change.
• Stress intermediate outcomes
and distance travelled.
• Capture your organisation’s
full social value.
• Involve stakeholders.

Tracking progress
• Know what success ‘looks like’.
• Develop meaningful measurements and
indicators.
• Use 2-3 measurements or indicators per
variable.
• Quantitative and qualitative.
• Consult indicator banks and similar
organisations.
• Evidence, not proof.

Example: improved health
• Number of visits to doctor or
hospital.
• Weight gained or lost.
• Experience of specific
symptoms.
• Absence from work due to
sickness.

Example: policy change
• How often issue is discussed in traditional
and new media and how it is framed.
• Ratings of policy-makers’ interest in, and
influence on, issue.
• Public understanding of issue as assessed
by polls.
• Progress of a bill through
the Oireachtas.

Data collection and analysis
• Methods depend on what
information you need and what
resources you have.
• Existing records, questionnaire
surveys, one-to-one sessions,
group sessions, etc.
• Samples, baselines and controls.
• Organise raw data, spot
patterns/trends … learn!

The right tool for the job
• Quality tools (e.g. EFQM Excellence
Framework, ISO 9001, PQASSO)
• Strategic management tools (e.g.
balanced scorecard, RBM, SPEAK)
• Tools for measuring
outcomes, impacts and
social value

Outcomes, impacts, social value
• Individuals (e.g. Outcomes Star)
• Local communities and economies (e.g.
Local Multiplier 3 and Prove It!)
• Infrastructure organisations (e.g. Value of
Infrastructure Programme)
• Social value (e.g. SROI)

Communicating impact
• Internally to learn, improve and boost
morale.
• Externally to meet funding requirements,
attract support, increase credibility and
share lessons learnt.
• Stories and numbers.
• The good and the not-so-good.
• Appropriate to audience.

Thank you
Please feel free to contact me for further
information on outcomes, impacts and social
value:

Sandra Velthuis
01 – 296 7694
085 – 167 4425
www.whitebarn.info

TABLE EXERCISE
Developing Targets & Indicators

the conference

TABLE EXERCISE
1. Is everyone clear about the outcomes (as opposed to the outputs) their groups are
aiming to achieve through their work? Can you give a few examples?

2. Can everyone name some sources of evidence (studies, research reports, surveys,
theoretical models, etc) of what approaches deliver the best outcomes for the
people or communities they work with?

3. Would organisations be willing and able to do things differently if evidence
suggested there was a better way to do things? If yes, why? If not, why not?

4. What challenges would you anticipate in moving to more outcomes-based working?

5. What kinds of services, supports or initiatives might you find helpful in making this
transition?

the conference

KNOWING AND SHOWING YOUR
OUTCOMES AND IMPACTS

LAUNCHED
TODAY!

Visit Our Stand for
your copy 23
the conference

CLOSE OF SESSION 1
COFFEE BREAK
11.00am – 11.30am
Competitions in the exhibitor area

the conference

SESSION 2
INVOLVING
How do we maximise people’s participation in
decisions that affect them?

the conference

PUBLIC PARTICIPATION
Fergus O’Ferrall
Adelaide Lecturer in Health Policy TCD

the conference

SHARING SOCIAL RESPONSIBILITY
Sean Healy
Director, Social Justice Ireland

the conference

Sharing Social
Responsibility
Seán Healy

Director

Social Justice Ireland
seanhealy@seanhealy.com

Introduction
 Ireland Today?

 Ireland’s Development Model?

Material on these topics, as well as on other issues
addressed in this presentation, is available in

Shaping Ireland’s Future

Social Justice Ireland’s annual Socio-Economic Review
published April 2012. Available at www.socialjustice.ie

68

Current Policy-Making System
 Current situation?

 Link between evidence and policy-making?

 Responsibility for decision-making at

present?

69

Sharing Responsibility
 Place for stakeholders?
 Proposals for Social Dialogue?
 Council of Europe Charter.

Text of this Charter and other material on a range of
issues related to social responsibility in the future
(including a chapter by Ivan Cooper) is available in

Sharing Responsibility in Shaping the Future

Published by Social Justice Ireland’s in September 2011.
Available at www.socialjustice.ie

70

CHANGING THE NATURE OF PUBLIC SERVICES
Martin Sime
Chief Executive Officer,
Scottish Council of Voluntary Organisations

the conference

The Wheel: Innovate, Involve, Inspire

THE PUBLIC SERVICE PROBLEM IN SCOTLAND

• Economic recession generates demand
• Cuts in public expenditure to 2016
• 200 separate public sector bodies
• Cost of universal benefits
• Growing inequality


CHRISTIE COMMISSION DIAGNOSIS

• Public services unsustainable
• Up to 40% “failure demand”
• Need to focus on prevention
• Reorganisation only for the purpose of
improving the user experience


CHRISTIE COMMISSION PEOPLE & COMMUNITIES

• Helping people help themselves
• Asset based approaches
• Development Trusts
• Self directed support


CASE STUDIES

# Reshaping Care for Older People: Change Fund
•

# Long Term Conditions Alliance: Self-Management

# Roseneath Community Development Trust


ROLE OF THE VOLUNTARY SECTOR

• Includes Housing Associations and Credit
Unions (but not co-ops)
• Commissioned to deliver eg care
• Advocacy, public policy critique
• Adds value through fundraising, volunteers etc.


POLICY QUESTIONS

• How does the State help self-help?
• Valuing strong communities
• How to build sustainable community
infrastructure?
• Sector/State partnership needs redefined

WHY PARTICIPATE IN DEMOCRACY ANYWAY?
Sr Bernadette MacMahon
Director, Vincentian Partnership for Social Justice

the conference

 Established in 1996

 Working for social and economic change

 Tackling poverty and exclusion

Consists of:
 Society of St. Vincent de Paul
 Vincentian Congregation
 Daughters of Charity
 Sisters of the Holy Faith

Active Citizenship/Voter Education Programme

Research
Minimum Income Standards

Active citizenship requires the will and the
opportunity to participate at every level and in
every way in society – To be the arrow not the
target.
President M. Higgins

Loyalty to the state – carries specific
responsibilities, like bearing arms when the state
is under threat, observing the law, or treating
neighbours as you would be treated yourself.
And most importantly, a moral obligation to
participate fully and critically in civic life, voting,
joining speaking out, if necessary, marching.
Irish Times 4.2.12

Four Irish Studies/Reports

Supporting Voluntary Activity 1997
Taskforce on Active Citizenship 2006
Report Democracy Commission 2005
Power to the People – Assessing Democracy in
Ireland 2007

Re - thinking our Vision
Active citizenship refers to the role of people,
communities and voluntary organisations in decision -
making which directly affects them. This extends
the concept of formal citizenship and democratic
society from one base of basic civil, political, social
and economic rights to one of direct democratic
participation and responsibility. In this sense,
citizenship is a political activity which gives citizens
the opportunity to shape the society in which they live.

Supporting Voluntary Activity Green Paper, 1997 pages 24-25

Taskforce on Active Citizenship
established by the Taoiseach 2006

National conversation on the extent
to which citizens engage in issues
that affect them and their communities

The Taskforce travelled round the country

Recommendations
1. Participation in the Electoral Process

2. Public Service and Citizens

3. Community Engagement and Promoting a Sense
of Community

4. Education for Citizenship

5. Challenge of Engaging Newcomers

TASC The Report of the Democracy Commission and
Engaging Citizens 2005
Power to the People? Assessing Democracy in Ireland

2005
 Awareness of interdependence - all groups and individuals
 Civic self restraint – taking responsibility
 Openness to deliberative argument – how to engage in
politics
 Awareness and compatibility of Irish and EU citizenship

2007
Strengths
 Public Commitment to Democratic Values
 Social Partnership
 Electoral System
 Rule of Law

Weaknesses
 High level of poverty and inequality
 Lack of representatives in public life

Active Citizenship
Marshall 1950 and Kymlika and Norman 1991

Civil – deals with relations between duties and rights and
with what people can do for each other in voluntary groups
and organisations

Social – focus on respect for the rule of law as a necessary
condition of any kind of social order

Political – citizenship is not just a certain status defined by a
set of rights and responsibilities. It is also an identity, an
expression of one’s membership in political community.

Politics comes from the Greek word for Citizens,
which is polites

Politics can be understood as the work of all
citizens

“Bad officials are elected by good citizens who
do not vote”
George Jean Nathan

Issues of Participation

 Democracy: “Government of the people, for the people, by
the people”.

 Popular participation is at the core of any
democratic society

Voting Trends – General Elections

1969 1977 1987 1989 1992 1997 2002 2007 2011

76% 75.66% 72.69% 67.66% 67.46% 65.26% 62.05% 66.71% 70.00%

Non Voters: The Experience of the
Vincentian Partnership
 believe that their votes/voices have no value
 do not know how to complete a ballot sheet or how
to register as a voter.
 are intimidated by the language and the
presentation of politicians and candidates.
 have lost confidence in politicians because of the
scandals/ tribunals.
 Many people have given up voting because they
experience the growing gap between the richest and
poorest people in our society

What is The Programme?
 Unit 1 – Explore with people the reasons to Vote how
to register and how to Vote
 Unit 2 – considers ways of taking an informed stance
on important issues in society
 Unit 3 – presents an approach to choosing
candidates on an informed basis

The content and process reflect the approach of
Paulo Freire to Adult Education

To Date

Workshops have been facilitated in 24 of the 26
counties approximately 400 in 15 years

Approximately 150 ‘trainers trained’

Voting Trends
Ballyfermot/Cherry Orchard 49.17% increase of 21%

SWICN 43.64% increase of 13.21%

Finglas/Cabra 54.43% increase of 16.35%

Corduff 51% increase of 31%

Tallaght Central 54.88% increase of 24.11%

Bray 54.59% increase of 12.22%

Positive Outcomes

 Participants gained insight into the electoral and
political system
 Discovered they were the experts on their areas
 Acquired confidence in their capacity to speak up
 Recognised that politicians are ‘just other people’
 Some communities with further support achieved
changes for the local area – e.g. road safety
measures, safer parks and playing areas

Participants learned the importance of the 3Ps

 Many community leaders expanded and
developed the programme to include pre and post
election meetings with candidates/politicians

 More people recognise the power of their vote –
my vote is my voice and the number of voters
increased

 The more people know how they are governed the
better they will be governed Jefferson

“If we begin to see the world differently
we will begin to act in relation to it
differently.” Mark Dowd

Some Negative Developments

Present cutbacks affecting community work.
“Change comes about when small groups of
concerned people come together to work for it.
Indeed change rarely happens otherwise”
Margaret Meade

Experience of more broken promises – more
fruitless consultations – increasing the risk of
alienation from the democratic/electoral system -
“What is the point?”

Active Citizenship -
interactions between the three dimensions -
civil, social and political

Not simply volunteers
Not simply obeying law and order
Not simply voters
But
Citizens

Citizenship is good for you!

Dozens of painstaking studies have established
beyond doubt that social connectedness is one of
the most powerful determinants of well-being.
(personal and communal)
Bowling Alone - Robert D. Putnam, 2000

Civil Society

The conversation among people and societies to
determine the most effective way to create healthy
and sustainable public life – the engagement of
citizens ultimately determines the health of
public life.
Kumi Naidu 1991

My friends, no one is born a good citizen;
No nation is born a democracy;
Rather both are processes that continue
to evolve over a lifetime.
Kofi Annam 1998

Further Information
The Vincentian Partnership for Social Justice
Ozanam House
53 Mountjoy Square
Gardiner Street
Dublin 1
Tel: 01 8780425
Fax: 01 8780423

E-mail: vpj@eircom .net

Websites: www.vpsj.ie www.vote.ie www.budgeting.ie www.misc.ie

BUILDING ENGAGEMENT BY INVOLVING PEOPLE
Karl O’Connor
Senior Manager of Employee Engagement,
Human Resources at Ulster Bank

the conference

Building Engagement by
Involving Your People

Innovate Involve Inspire 2012 conference
Karl O’Connor, Senior Manager Culture & Engagement, Ulster Bank

“Let me look … I think I kept your ideas on file ! ”

Everyone likes to put their ….

use it to
your
advantage

INNOVATION
N
V
O
L
V
E NGAGE Slide 116

6 months

Engagement + 9%
Comms +17%

Engaging Our People
Using Continuous
Improvement

Responsible Employer Award

Involve Engage

Better
outcomes

TABLE EXERCISE, PANEL DISCUSSION and Q&A
Facilitated By

Karl O’Connor

the conference

CLOSE OF SESSION 2
LUNCH
1.00pm – 2.15pm
Competitions in the exhibitor area

the conference

SESSION 3
INSPIRING
Innovating & Involving; an Inspiring job for the sector!

the conference

TABLE EXERCISE
Improving Our Practice
What are the priorities now for the sector?
(1 – 5)

the conference

PANEL DISCUSSION
We talk a lot about it, but just how good
are we at innovating and involving?

the conference

• Mark Blake-Knox (CEO, Cheshire Ireland)
• Elaine Byrne (Academic Team, We the Citizens)
• Ivan Cooper (Director of Advocacy, The Wheel)
• Mike Allen (Director of Advocacy, Focus Ireland)
• Barbara O'Connell (CEO, Acquired Brain Injury Ireland).

the conference

PANEL DISCUSSION
FEEDBACK: What are the priorities now
for the sector?

the conference

THE WHEEL’S STRATEGY 2012 – 2016
Responding to the challenges

Deirdre Garvey
Chief Executive Officer, The Wheel

the conference

BEING THE CHANGE

The Wheel’s Strategic Plan 2012 - 2016

the conference

THEORY OF CHANGE MODEL
1. Describe the problem and provide the evidence for
how we know it exists.
2. Working ever deeper, explore the root causes of the
problem, and identify the things that would need to
change in the world for that problem to not exist any
more.
3. Identify which of these things The Wheel wants to
do, or wants to help to do.

the conference

taken from

Vision / Why
Ambition

Mission /
Purpose
What
Strategic
priorities

Strategic objectives

Action Plan How
(activities, responsibilities,
timelines)

Impact Measures & Key
Performance Indicators Is it working

WHY?
(Vision)

Our vision is of an Ireland where there is a shared
idea of, and commitment to realising, a fair and
just society; and where the role and contribution
of the community and voluntary sector in doing
this is valued and understood.

the conference

WHAT?
(Mission)
To increase the community and voluntary sector’s
capacity and capability to play its part in achieving a fair
and just society in Ireland.

We do this by:
1. Representing the shared interests of community and
voluntary organisations
2. Supporting these organisations to do their work
3. Promoting the importance of active citizenship as
vital in making Ireland a better place to live.
the conference

WHAT?
(Strategic Themes)

2. Recognising 3. Developing
1. Building a
and valuing the
truly fair and
the community
just society
community and voluntary
and voluntary sector
sector

4. Enhancing the effectiveness of The Wheel

the conference

WHAT? & HOW?
(Strategic Objectives)
BUILDING A FAIR AND JUST SOCIETY

Objective 1 Build an understanding of the importance of
participating in community life and in public
decision-making.

Objective 2 Maximise public participation in the development of
shared ideas of what a fair and just society is in
Ireland.

the conference

WHAT? & HOW?

RECOGNISING AND VALUING THE COMMUNITY AND VOLUNTARY
SECTOR

Objective 3 Maximise the effectiveness of the community and
voluntary sector’s voice so that the sector’s
importance is recognised by the state, the political
system and other sectors.

the conference

WHAT? & HOW?
DEVELOPING THE COMMUNITY AND VOLUNTARY SECTOR

Objective 4 Foster excellent leadership within the sector.

Objective 5 Promote quality and excellence in the community
and voluntary sector to deliver better outcomes for
people.

Objective 6 Maximise collaborative working in the sector.

the conference

WHAT? & HOW?
ENHANCING THE WHEEL’S EFFECTIVENESS

Objective 7 Ensure that The Wheel has the systems, structures,
processes, procedures and resources to deliver on our
strategic objectives.

Objective 8 Ensure The Wheel involves, adds value, and is responsive
to our members.

the conference

HOW? (continued)
Detailed Action Plan:
• Activities, responsibilities, timelines
• Budgets
• Other Resources

Annual Operations Plans

the conference

IS IT WORKING?
(Impact Measures & Key Performance Indicators)

The most challenging part?

the conference

OUTCOMES
• Based on the eight strategic objectives
• Will enable individual functional areas to measure their impact,
and also to enable The Wheel to track progress as an
organisation against a single strategic vision.
• Ultimate Outcomes: Longer term changes that The Wheel hopes to
achieve on behalf of and for our members (some straddle the scope of
this plan.)
• Intermediate Outcomes: Medium term changes that The Wheel hopes
to achieve on behalf of and for our members – and which are
measureable within the scope of this plan. They will also indicate
whether we are on course for achieving the changes described under
‘final outcomes’.
• Internal Outcomes: the things that make The Wheel effective at
achieving our intermediate and final outcomes. They are about the way
that we work and how we are perceived.
the conference

The Wheel’s activities

The Wheel’s internal outcomes

The Wheel’s external outcomes
The national policy Improved Best practice in governing
agenda responds to and outcomes for the and running CV
addresses the needs of people served by organisations is
the community & CV organisations maximised, standardised
voluntary sector and sustained

Shared public understanding and
awareness of what constitutes a
fair and just society
the conference

NEXT STEPS

• Consultation with members: June 11th
• Ratification at AGM: July 4th 12pm-2pm
• Implementation commencing immediately:
• Reaching out to collaborative partners
• Establishing the systems and processes for
measurement
• Organising resources around the 8 strategic
objectives

the conference

CHAIRMAN’S CONCLUDING ADDRESS
Fintan O’Toole

the conference

THANK YOU & SAFE HOME!

the conference

Doing Our Best: Measuring Impact

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (11)

Similar to Doing Our Best: Measuring Impact

Similar to Doing Our Best: Measuring Impact (20)

Recently uploaded

Recently uploaded (20)

Doing Our Best: Measuring Impact