The document discusses measuring the impact of social innovation. It makes three key points: 1. Measuring the impact of social innovation is challenging due to its complex, long-term nature and dependence on social contexts. 2. Universities should better support the evaluation of social innovation through monitoring inputs/outputs, using relevant indicators, and developing new evaluation models like developmental evaluation. 3. Networks like OLTIS, CLT, and RQIS in Quebec help catalyze social innovation and support its evaluation through knowledge transfer between researchers and communities.

1. Series 1- Social innovation, governance, impact,
accountability
Measuring the impact of social
innovation
Mireille Mathieu, Ph.D.
Professeure émérite de psychologie
Université de Montréal
1

2. Social innovation
Social innovation
• Social innovation is a dynamic that, in response to
current clearly defined social needs, offers more
appropriate and more sustainable solutions
• Social innovations taken up by an institution, an
organization or a community offer measurable
benefits for the collective, rather than simply for
certain individuals. The impact of social innovation is
transformative and systemic. By its inherent creativity,
social innovation represents a break from what is.
(Réseau québécois de l’innovation sociale)
2

3. Social innovation
Social innovation
“Despite its undeniable foundation in research,
social innovation is often, as we know, the
unexpected outcome of a social tinkering that
emerges from a context of need for change, for
adaptation or for a more radical transformation. It
calls for the involvement, conscious or not, but still
essential, of a plurality of actors from civil society,
from politics, public administration and research. “
(Louise Dandurand, 2013)
3

4. University and innovation
Innovation, social as well as technological,
should be part of the DNA
of the University
4

5. University and social innovation
University has two main roles:
• Produce social innovation
• Support the dissemination and implementation
of sustainable social innovation
5

6. 1. Produce social innovation
• Research, particularly, collaborative research,
provides basic knowledge and tools for social
innovation, for solving well defined problems
• Innovative training programs and innovative training
methods should be the trademark of our universities
• Lifelong learning initiatives, for ex: University of the
third age, MOOC (massive open online courses)
6

7. 2. Support the dissemination and the
implementation of social innovation
(Third mission)
• Life long learning
• Knowledge transfer and innovation
• Social engagement
7

8. Why should University measure effects
of social innovation ?
• “What gets measured gets valued”
• To improve program management
• To increase understanding of the « results »
• To achieve stronger communication of the value of
its work to the people that matter (internal and
external stake holders)
• To enhance attention to the social, economic and
environmental value created by the institution
8

9. Accountability
• Inputs - resources invested in the activity
• Outputs – the direct and tangible products
from the activity i.e. people trained, products
sold,
• Outcomes – changes to people resulting from
the activity i.e. a new job, increased income,
improved stability in life
• Impact – Outcomes less an estimate of what
would have happened anyway
9

10. Accountability
Who? How? With whom? For whom?
• Descriptive and performance Indicators (a few
examples)
– Number of research grants and publications by
faculty and by graduate students
– Number and description of innovative training
programs, number of students, number of graduates,
data on employment etc.
– Data on knowledge and innovation transfer (number
of products, number of target organizations, identity
and number of external stake holders etc.)
– Nature, number and degree of satisfaction of external
stakeholders involved as partners
10

11. Accountability
Who? How? With whom? For whom?
• Development of many, many set of indicators
over the last decade (see recent literature review
by RQIS)
• The more important challenge lies in the choice
of the relevant indicators at the very beginning of
the project
• Even though these sets of indicators don’t allow
for a satisfying evaluation of impact, they are the
first step towards that goal. They at least provide
a clear and reliable picture of inputs and outputs
11

12. Accountability
Who? How? With whom? For whom?
There is an urgent need for better and
comprehensive monitoring of inputs and
outputs by Universities.
It is as a necessary pre-requisite to evaluation
of impact of social innovation
12

13. Evaluation of outcomes and impact of social
innovation: a risky endeavor
By its very nature, innovation is:
• Risky;
• Unpredictable in terms of:
• which particular activity or intervention will work or
prove useful;
• who will benefit;
• when benefits, if any, will occur;
• under which particular set of circumstances an innovative
approach would be applicable;
• whether the discovery and application will be as
intended or of quite a different nature
13

14. Measuring impact: A risky endeavor
• “The indicators to measure the effects of social
innovation are probably those that are less developed .
Social and intangible impacts are particularly difficult
to measure due to various factors. Note, among others,
the specific nature of social innovations whose effects
are less in terms of economic performance when
compared to the technological innovations. The
complex and dynamic nature of social innovation
process also makes the task more difficult” (RQIS,
2015)
14

15. Measuring impact: A risky endeavor
• Social innovations are generally long term in nature,
sometimes very long term
• Social innovation never occurs alone but always in a context
of structured relationships, networks, infrastructures and in a
wider social and economic context
• It is tempting to want a simple matrix of aggregate measures.
But social change is messy. We can’t loose the context and
non measurable insights
• Even with a clear picture of outcomes it is another thing
entirely to tease out how the organization contributed.
15

16. A risky endeavor
• On average, good plans, people, and businesses succeed
only one in ten times.
• It should be considered a learning opportunity rather
than a problem that as many as 80–90 percent of
investments or projects do not work out well, or even
collapse completely
• Evaluation of impact of social innovation is also very
time-consuming and very expensive
16

17. Innovation to Impact:
Obama’s Social Innovation Fund
• Innovation to Impact:
Obama’s Social
Innovation Fund
• Evidence
• All programs funded by SIF must be able to
demonstrate a preliminary level of effectiveness and
then take part in a rigorous evaluation to strengthen
their base of evidence and to document and assess
whether their approach works more effectively, costs
less, or leads to better results for our country’s
communities. Too often, effective nonprofits lack the
expertise, resources, or infrastructure to conduct
meaningful evaluation that demonstrates impact and
can take their programs from promising to proven. In
fact, many grant programs do not adequately support
program assessments. However, SIF provides both
technical assistance and funding for required
evaluations of all selected programs. These evaluations
will provide robust information about which programs
and models generate strong results for individuals and
communities and how they can be replicated
successfully
17

18. Towards adequate evaluation of
outcomes and impact
• Focus on careful analysis of case studies so as
to learn from errors as well as from successes
and put the emphasis on best practices
• Use Developmental evaluation both for
monitoring the evolution of a given project
and as an innovative accountability strategy
for long term social innovation projects
18

19. Conditions for a successful developmental
evaluation (1))
• The organization is trying to solve a complex problem
with an adaptive solution
• The organization is open to testing new evaluation
approaches
• The organization’s leadership is willing to take risks,
be flexible and make necessary changes to the
initiative
19

20. Conditions for a successful developmental
evaluation (2)
• The organization’s values and culture support
innovation, continuous learning and adapting to its
environment
• The organization provides sufficient time, people ,
and financial resources for ongoing enquiry
• The organization is committed to open
communications and ensuring that information is
accessible and used internally and externally
20

21. Measuring impact:
social return on investment
• A way to monetise the value of the social impact in
financial terms
• Very often the result is expressed in avoided costs
rather than in an increase of funds (ex: quality of life at
work and depression)
• SROI is a way of thinking and we are not able to
monetize everything
• SROI is just a number that is one part of the story but it
is a powerful tool to convince stakeholders…
21

22. Self assessment of university leadership’s
support of social innovation
• Another important step towards a culture of
evaluation of outcomes and impact of social
innovation
22

23. Self assessment of university leadership’s
support of social innovation:
a few indicators
• Mission statements and strategy documents
• Faculty promotion criteria (social innovation,
knowledge transfer…)
• Effective monitoring and use of indicators of
knowledge transfer and social innovation in internal
and external reports
• Effective monitoring and use of indicators of social
engagement in internal and external reports 23

24. Conclusion
• Universities should:
– Put more adequate resources in evaluation of
social innovation
– Strongly support the development of new models
for better evaluation of impact of social
innovation
– Put more emphasis on their third mission:
revitalize their social contract with society
24

25. Conclusion
• Universities should:
– Monitor more extensively their effective
involvement in social innovation both in
production and in support of implementation
– Monitor the treatment by promotion committees
of knowledge transfer and innovation and take
relevant measures and rewards to foster a culture
of social innovation and of evaluation of social
impact
25

26. Conclusion
The prize for getting it right will be
considerable: more productive and
valued universities engaged in the
cultural and economic development of
their host societies.
26

27. Knowledge transfer and innovation in Québec
• Services aux collectivités (services to the community)
UQAM
• (CLT) Centres de liaison et de transfert -Centers for
liaison and transfer
• (OLTIS) Organismes de liaison et de transfert de
l’innovation sociale -Organizations of liaison and
transfer of social innovation
27

28. Social innovation: the Québec reality
OLTIS and CLT act as catalysts of social
innovation in support of the university
researchers, of practice settings and of the
community at large
28

29. CLT and OLTIS
CLT and OLTIS enhance the access of organizations
and communities to new knowledge and to the
results of research in the humanities and social
sciences generated by universities and public
research centers.
They also help the entire society benefit from the
effects of that knowledge on the quality of life of
individuals and communities and on the productivity
of organizations
(Government of Quebec)
29

30. OLTIS and CLT
• Networking between producers and users of research
and social innovation
• Animation in practice settings
• Analysis and dissemination of information
• Development and use of strategic intelligence tools
• Translation of research results into usable tools for implementation
of best practices
• Supporting sustainable appropriation of best practices by
individuals and organizations in the community
• Strong emphasis on evaluation of effects of social innovation
30

31. OLTIS supported by the Government of
Québec
• CLIPP: Centre de liaison sur l’intervention et la
prévention psychosociale
• CETREQ: Centre de transfert pour la réussite
éducative su Québec (CTREQ)
• TIESS-OLT: Territoires innovants en économie
sociale et solidaire Liaison et transfert
31

32. CLT supported by the Government of
Québec
• CEFRIO: Centre facilitant la recherche et
l’innovation dans les organisations l’aide des
technologies de l’information et de la
communication (TIC)
• CIRANO: Centre interuniversitaire de recherche
en analyse des organisations
32

34. Réseau québécois en innovation sociale
(RQIS)
The Québec Social Innovation Network (RQIS)
is a space for knowledge and experience
mobilization that helps make social innovation
an essential lever for sustainable development
of Quebec.
34

35. Réseau québécois en innovation sociale
(RQIS)
• A network of the main actors of social innovation in
Québec
• A very important and comprehensive literature review of
indicators for evaluation of social innovation (2015)
• The production of a Guide for choosing indicators of social
innovation (in progress)
• A laboratory for developing new methods for evaluation of
impact of social innovation
35