Virtual Workshop of the TCI Cluster Evaluation Working Group. With speakers Joan Martí, ACCIÓ, Catalonia, Federica Belfanti and Fernando Alberti, Liuc University, Italy and Emily Wise, Lund University, Sweden

1. On a screen near you
28/04/2020

2. Agenda
1. Welcome and introductions
2. Update to the work of TCI Evaluation WG
(Madeline Smith)
3. How do we evidence the wider impact of clusters?
Emily Wise, Lund University, Sweden
Joan Marti, ACCIÓ, Catalonia
Federica Belfanti and Fernando Alberti, Liuc University, Italy
Q&A: James Wilson, Orkestra, Basque Country
4. Sharing experiences and final reflections
• What strategic frameworks help drive more than economic contributions from
clusters in your region?
• What types of contributions to broader system-level improvements do clusters
make for Immediate Spillover Effects, Strategic Leadership Role, Higher Level
effects?
• How do clusters monitor and evidence the difference they make in these areas,
and how do they communicate to their different audiences?

3. Rules and requests
• Ask questions through the
chat function
• Presentation and questions
will be shared
• Recording the session

4. Cluster Evaluation Working Group
The Story so far…
28/04/2020

5. • A forum for learning collectively around common, complex & important
cluster evaluation challenges
• Sharing learning
• Identifying gaps
• Trying new approaches
www.tci-network.org/evaluation
TCI Cluster Evaluation Working Group

6. Despite the proliferation of cluster initiatives, there is a shortage of
evaluation research and practice
– Leaves cluster policies open to questions
– Prevents learning within clusters
– Prevents learning about policy
It is a challenge because the impacts of working with clusters are hard
to measure: they are intangible and difficult to isolate
– Clusters generate space for tackling issues that may only generate solutions in
the longer term
– Clusters are built on collaborative dynamics: trust and social capital
– Cluster activities spill over to agents outside the cluster initiative
– Cluster policies interact with other competitiveness policies
– Clusters are extremely heterogeneous
How do we develop better approaches to capture the
impact of clusters and cluster policies and foster
learning around how to improve them?
? ?
A shared challenge

7. Policy (resources)
Cluster Initiative
(association,
project, agency
…)
Change Behaviour
(innovate,
internationalise,
attract talent, …)
Improve Firm
Competitiveness
Impact on the Region
(economic & also
beyond GDP)
External Factors
Other
Policies
Time
Lag?
Controlling for
other factors?
Intangible ‘human’
elements?
Multi-level policy
mix ?
Cluster Policies: Not a Simple Logic Model
External Factors

9. 2013 2014 2015 2016 2017
Workshop
Forres
TCI
Conference
Kolding
TCI
Conference
Monterrey
TCI
Conference
Eindhoven
Workshop
Belfast
Workshop
Rzeszow
Workshop
Barcelona
TCI
Conference
Daegu
Workshop
Oslo
TCI
Conference
Bogota
2018
Workshop
Cork
TCI
Conference
Toronto
A long-run journey of collective learning
2019
TCI
Conference
Antwerp
2020
Virtual
Workshop
Workshop
Malmö

11. Three questions in focus for
TCI WG on Cluster Evaluation
1. How can we develop and share good practice and
techniques for cluster evaluation (Fruit Salad)?
2. How can we factor in the ”human element” in cluster
evaluation?
3. How do we evidence, back up and communicate the
critical success factors in clusters?

12. Further questions in focus for
TCI WG on Cluster Evaluation
1. How can we develop and share good practice and
techniques for cluster evaluation (Fruit Salad)?
2. How can we factor in the ”human element” in cluster
evaluation?
3. How do we evidence, back up and communicate the
critical success factors in clusters?
• M&E of clusters in relation to S3
• Cluster evaluation ‘beyond GDP’
• Evaluation levels (POPS - ACS Framework)

13. Cluster Programme Framework of Effects
Level
Individual
Actor (A)
Collaborative
Group/Cluster
Initiative (C)
Territorial
System (S)
Perceptions & Behaviours
- Competence development
- Knowledge exchange
- Capacity to innovate
- Involvement in collaborative activities
(Economic) Performance experienced by individual
actors
Perceptions & Behaviours
- Engagement of different actor groups
- Dynamics of linkages over time
- Perceived value of collaboration
Collaboration Infrastructure
- Quality of cluster management
- Leadership
- Processes
Competitiveness and international attractiveness of
innovation eco-system
- Changes in behaviours and performance of
system
Effectiveness of business and innovation support
system
- Changes in structures, policies and institutional
arrangements
Effectiveness of business and innovation
support system
- Changes in structures, policies and
institutional arrangements
Competitiveness and international
attractiveness of innovation eco-system
- Changes in behaviours and performance
of system
Short-term results (1-3 years)
Cluster participants
(direct/immediate results claimed by cluster participants)
Long term effects (3-10 years)
Cluster participants & beyond
(indirect/subsequent effects that can be observed over time)

14. Scope of Influence
• Direct Effects
• Spillover Effects
• Strategic/systemic Leadership
effects
(Catalyst, Anchor,
Spokesperson, Voice)
• Higher Level System Effects
(Changing the system, Changing
the world)
Higher Level
system effects
Strategic/Systemic
Leadership
Immediate
spillover effects
Project Direct
effect

15. Cluster Programme Framework of Effects
Short-term results (1-3 years)
Cluster participants
(direct/immediate results claimed by cluster participants)
Long term effects (3-10 years)
Cluster participants & beyond
(indirect/subsequent effects that can be observed over time)
Level
Individual
Actor (A)
Collaborative
Group/Cluster
Initiative (C)
Territorial
System (S)
Perceptions & Behaviours
- Competence development
- Knowledge exchange
- Capacity to innovate
- Involvement in collaborative activities
(Economic) Performance experienced by individual
actors
Perceptions & Behaviours
- Engagement of different actor groups
- Dynamics of linkages over time
- Perceived value of collaboration
Collaboration Infrastructure
- Quality of cluster management
- Leadership
- Processes
Spillover Effects
- Improving the competitiveness and
international attractiveness of the
innovation ecosystem
Strategic/Systemic Leadership Effects
- Contributing to increased effectiveness
(structures, policies and institutional
arrangements) of the innovation ecosystem
Strategic/Systemic Leadership Effects
- Contributing to increased effectiveness
(structures, policies and institutional
arrangements) of the innovation ecosystem
Spillover Effects
- Improving the competitiveness and
international attractiveness of the innovation
ecosystem
HigherLevelSystemEffects
Focus

16. Evidencing the wider impact of
clusters
• Three speakers, three questions
Emily Wise, Lund University, Sweden
Joan Marti, ACCIO, Catalonia
Federica Belfanti & Fernando Alberti, Liuc University, Italy
Group Sharing
• Survey responses to the same questions
• Q&A on the chat (James Wilson, Orkestra)
1. What strategic frameworks help drive more than economic contributions from
clusters in your region?
2. What types of contributions to broader system-level improvements do clusters
make for Immediate Spillover Effects, Strategic Leadership Role, Higher Level
effects?
3. How do clusters monitor and evidence the difference they make in these areas,
and how do they communicate to their different audiences?

17. What strategic frameworks help drive
more than economic contributions
from clusters in your region?

34. What strategic frameworks help drive more than economic contributions
from clusters in your region?
Shared Value
concept
National
Performance
Frameworks
Sustainable
Development
Goals
Smart
Specialisation
Strategies
Other Please give
details
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
What strategic frameworks help drive more than
economic contributions from clusters in your
region?
Responses
As a strategic framework linking clusters to wider
impacts, smart specialisation strategies lead the way
Pre-workshop survey, 51 responses

35. What types of contributions to
broader system-level improvements
do clusters make for Immediate
Spillover Effects, Strategic Leadership
Role, Higher Level effects?

36. A (preliminary) definition of
system level results
A system level result is a milestone, event or outcome
that has been influenced by (a set of) actions of the
cluster/collaborative initiative, and which has a lasting,
longer-term impact on several organisations/actors in
the territorial system (both within and beyond the
cluster/collaborative initiative).
Results on the system level typically spread to the
surrounding society and affect different types of systems
(sectoral, regulatory, social, environmental, etc.).

58. What types of contributions to broader system-level improvements do
clusters make for Immediate Spillover Effects, Strategic Leadership Role,
Higher Level effects?
The key immediate spillover effects from clusters are
in knowledge development & dissemination and
experimentation & entrepreneurship
Pre-workshop survey, 51 responses
Knowledge
Development and
Dissemination
Experimentation
and
Entrepreneurship
Investment Infrastructure OtherPlase give
details
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
What types of contributions to broader system-
level improvements do clusters make for
Immediate Spillover Effects?
Responses

59. What types of contributions to broader system-level improvements do
clusters make for Immediate Spillover Effects, Strategic Leadership Role,
Higher Level effects?
Clusters play a strategic leadership role across a
broad range of dimensions
Pre-workshop survey, 51 responses
Voice of the sector Developing
reputation and
position
Influencing policy
or strategy
Building strategic
partnerships
Other Please give
details
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
What types of contributions to broader system-level
improvements do clusters make for Strategic
Leadership Role?
Responses

60. What types of contributions to broader system-level improvements do
clusters make for Immediate Spillover Effects, Strategic Leadership Role,
Higher Level effects?
Contribution to environmental agendas dominate
the higher level effects that clusters make to
territorial systems
Pre-workshop survey, 51 responses
Environmental
agendas
Equality and diversity Social inclusion Wellbeing OtherPlease give
details
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
What types of contributions to broader system-level
improvements do clusters make for Higher Level
System Effects
Responses

61. How do clusters monitor and
evidence the difference they make in
these areas, and how do they
communicate to their different
audiences?

62. A structured approach for evidencing
system-level effects over time
• A guide ”How can we follow system-level results?” (explaining the why,
what and how)
- Tips on documenting, prioritising, categorising and communicating
important events within the regional innovation system
- Suggestions for support from action researchers
• Embedded in annual reporting – with increased structure and
harmonisation
• New possibilities for visualising, analysing and communicating ”system
result profiles”
- For individual initiatives
- For the full portfolio of Vinnväxt initiatives
- For tracking system-level changes over time

73. How do clusters monitor and evidence the difference they make in these
areas, and how do they communicate to their different audiences?
Case studies and success stories are by far the most
common form of evidencing the broader impacts of
clusters
Pre-workshop survey, 51 responses
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
How do clusters monitor and evidence the
difference they make in these areas?
Responses

74. How do clusters monitor and evidence the difference they make in these
areas, and how do they communicate to their different audiences?
The key audiences for communicating the broader
impacts of clusters are their memners and
national/regional governments
Pre-workshop survey, 51 responses
National / Regional
Governmental
funders
Local / City regions Cluster members Wider public Other (please
specify)
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
Who is the Audience for this wider evidence? And
how do you communicate with these audiences?
Responses

75. Cluster Programme Framework of Effects - indicators
Level
Individual
Actor (A)
Collaborative
Group/Cluster
Initiative (C)
Territorial
System (S)
 Introduction of new products/services
 Prototypes and patent applications
 Articles (academic, other)
 New markets and customers
 Change in strategy
 Revenue growth
 Productivity growth
 Employment growth
 Export growth
 Improved market share/position
 # and different types of actors engaged in the cluster initiative
 #, types and volume of collaborative activities
 New innovation partnerships
 Willingness/perceived value of collaborating around a shared strategic direction
 Labelling of cluster management quality (ESCA approach)
 Knowledge development and
dissemination
 Experimentation and
Entrepreneurship
 Attracting investment
 Developing physical (R&I)
infrastructure
 Building reputation and position
of the sector/thematic area
 Connecting with other system
leaders in new strategic
partnerships
 Influencing policy or strategy
 Initial milestones/key events on various
”impact pathways”
•Cleanenergyorclimateaction
•Equality,diversityandsocial
inclusion
•Healthandwellbeing
Short-term results (1-3 years)
Cluster participants
(direct/immediate results claimed by cluster participants)
Long term effects (3-10 years)
Cluster participants & beyond
(indirect/subsequent effects that can be observed over time)

76. Final Reflections

79. WG progress and plans
• Ongoing working group activities and outputs
• TCI Evaluation toolkit
• Conferences (virtual / physical)
• Evaluation WG contribution to TCI annual
conference
• Further journal articles…
• Join TCI to Keep collaborating…
You need a triple helix to tackle a triple helix
challenge

80. Thank you and stay safe