Generating Cluster Policy Learning
through Evaluation
Madeline Smith, James Wilson &
Emily Wise
Cluster Lab 2
Cluster Evaluation
Cluster Lab | Daegu, Republic of Korea
5th November 2015
The Story So Far
• Common Challenge
• Complex
• Increasing importance
• Sharing learning
• Identifying gaps
• Trying new
Cluster Evaluation Working Group
• Over three years of work
• Workshops, conference presentations & discussions
• Newsletter updates
• Website zone
March 2015
Working group meeting
Rzeszow, Poland
November 2015
TCI Conference
Cluster Lab
Daegu, South Korea
Kolding TCI 2013
Belfast March 2014
Digging Deeper into questions
Monterrey TCI 2014
Progress
Outputs
Mini Projects
Rzeszow March 2015
Human Element
• Framework
• Firm Level
• Test /iterate
Benchmarking
• Sharing
• Learning
On-line discussion
Daegu TCI 2015
This cluster lab is a forum to:
• Address sticky issues
• Leverage TCI knowledge
• Collaboratively tackle shared challenges
• Create better solutions to cluster evaluation
Agenda
• State of play of cluster evaluation
• Exercise: ‘the perfect cluster’
• Working on ‘sticky problems’
• Wrap up
The Challenge
Despite proliferation of cluster initiatives, there is a
shortage of evaluation research and practice
• Leaves cluster policies open to questions
• Prevents learning
How then do we develop better approaches to capture
the impact of cluster policies and foster learning
around how to improve them?
Methodological Difficulties
Direct outcomes are intangible & difficult to isolate
• Space for debating issues that may only
generate solutions in the longer term
• Higher trust and social capital
• Spillovers to agents outside the cluster initiative
This it is difficult to rigorously show whether or not
cluster policy has positive effects: “pick and mix of
research evidence” (Perry, 2005)
Heterogeneity & Policy Mixes
• Variety of policies in ‘cluster’ family
• Complex relations with other competitiveness
policies
Source:
Magro &
Wilson
(2013)
A generally accepted framework
...contribute to increased
interactive learning and
collaborative research and
innovation projects
...which contributes to increased
innovation, international
attractive-
ness, productivity and growth
Activities to
strengthen or
upgrade a
cluster/
innovation
environment...
Input/Resources Activities Results/Outcomes Effects
3-10 years >10 years
Structural
capital
(tangibles)
Social capital
(intangibles)
Results/Outcomes
3-10 years
Evaluation as Learning
• Evaluate across why, what & how
• Not audit – all about learning
• How to influence policy improvement and cluster
management improvement simultaneously
Emerging approaches
Quantative Impact
Studies
Qualitative Case
Studies
Towards
‘mixed methods &
evidence’
(smart
combination of
pieces of evidence
gathered by
different people)
Social network
analysis
Participatory
evaluation
Realist
evaluation
The Perfect Cluster
• A user journey story from the perspective of the
cluster
• If we were starting, growing, internationalising and
diversifying the perfect cluster, how would we
evidence its development?
The Perfect Cluster
Creation Growth Internationalisation Diversification
Activity: What is
happening?
Actors: Who is involved?
Resources: What money
& physical assets are
being used?
Social Capital: What
type of behaviour do we
see?
Results: What is being
generated?
Sticky Problems
1. How can we capture the right information
(qual/quant) without making evaluation excessive?
2. How can we measure/evidence trust building and
collaborative strength?
3. How do we ensure evaluation is about learning
that is useful to both clusters and policy makers?
Sticky Problem Groups
Experience Audit
• What
methods/approaches
have we used before?
Experience Reflection
• What was good about
these and what issues
have we had with
them?
Solution Generating
• Can we combine
experiences to
address gaps and find
better solutions?
Summary and Next Steps
Key learning from today?
How to take it forward?
Cluster Evaluation
Cluster Lab | Daegu, Republic of Korea
5th November 2015

TCI 2015 Generating Cluster Policy Learning through Evaluation

  • 1.
    Generating Cluster PolicyLearning through Evaluation Madeline Smith, James Wilson & Emily Wise Cluster Lab 2
  • 2.
    Cluster Evaluation Cluster Lab| Daegu, Republic of Korea 5th November 2015
  • 3.
    The Story SoFar • Common Challenge • Complex • Increasing importance • Sharing learning • Identifying gaps • Trying new
  • 4.
    Cluster Evaluation WorkingGroup • Over three years of work • Workshops, conference presentations & discussions • Newsletter updates • Website zone March 2015 Working group meeting Rzeszow, Poland November 2015 TCI Conference Cluster Lab Daegu, South Korea
  • 6.
  • 7.
    Belfast March 2014 DiggingDeeper into questions
  • 8.
  • 9.
    Rzeszow March 2015 HumanElement • Framework • Firm Level • Test /iterate Benchmarking • Sharing • Learning On-line discussion
  • 10.
    Daegu TCI 2015 Thiscluster lab is a forum to: • Address sticky issues • Leverage TCI knowledge • Collaboratively tackle shared challenges • Create better solutions to cluster evaluation
  • 11.
    Agenda • State ofplay of cluster evaluation • Exercise: ‘the perfect cluster’ • Working on ‘sticky problems’ • Wrap up
  • 12.
    The Challenge Despite proliferationof cluster initiatives, there is a shortage of evaluation research and practice • Leaves cluster policies open to questions • Prevents learning How then do we develop better approaches to capture the impact of cluster policies and foster learning around how to improve them?
  • 13.
    Methodological Difficulties Direct outcomesare intangible & difficult to isolate • Space for debating issues that may only generate solutions in the longer term • Higher trust and social capital • Spillovers to agents outside the cluster initiative This it is difficult to rigorously show whether or not cluster policy has positive effects: “pick and mix of research evidence” (Perry, 2005)
  • 14.
    Heterogeneity & PolicyMixes • Variety of policies in ‘cluster’ family • Complex relations with other competitiveness policies Source: Magro & Wilson (2013)
  • 15.
    A generally acceptedframework ...contribute to increased interactive learning and collaborative research and innovation projects ...which contributes to increased innovation, international attractive- ness, productivity and growth Activities to strengthen or upgrade a cluster/ innovation environment... Input/Resources Activities Results/Outcomes Effects 3-10 years >10 years Structural capital (tangibles) Social capital (intangibles) Results/Outcomes 3-10 years
  • 16.
    Evaluation as Learning •Evaluate across why, what & how • Not audit – all about learning • How to influence policy improvement and cluster management improvement simultaneously
  • 17.
    Emerging approaches Quantative Impact Studies QualitativeCase Studies Towards ‘mixed methods & evidence’ (smart combination of pieces of evidence gathered by different people) Social network analysis Participatory evaluation Realist evaluation
  • 18.
    The Perfect Cluster •A user journey story from the perspective of the cluster • If we were starting, growing, internationalising and diversifying the perfect cluster, how would we evidence its development?
  • 19.
    The Perfect Cluster CreationGrowth Internationalisation Diversification Activity: What is happening? Actors: Who is involved? Resources: What money & physical assets are being used? Social Capital: What type of behaviour do we see? Results: What is being generated?
  • 20.
    Sticky Problems 1. Howcan we capture the right information (qual/quant) without making evaluation excessive? 2. How can we measure/evidence trust building and collaborative strength? 3. How do we ensure evaluation is about learning that is useful to both clusters and policy makers?
  • 21.
    Sticky Problem Groups ExperienceAudit • What methods/approaches have we used before? Experience Reflection • What was good about these and what issues have we had with them? Solution Generating • Can we combine experiences to address gaps and find better solutions?
  • 22.
    Summary and NextSteps Key learning from today? How to take it forward?
  • 23.
    Cluster Evaluation Cluster Lab| Daegu, Republic of Korea 5th November 2015