Data, networks and experimentation
Hasan Bakhshi
Nesta Policy & Research Unit
IRC Annual Summit,
26th November, 2013
http://www.nesta.org.uk/
“Innovation policy would
work better, we suggest, if
modelled on experimental
science and directed to the
task of minimising the
uncertainty that
entrepreneurs face in the
discovery of opportunities
and constraints”
“…uncertainty is a defining
feature of emergent areas
subject to persistent
structural change like the
creative industries, and
should be dealt with in a
systematic way.”
Experimental programmes
Innovation policy as a process

Test a
hypothesis

Discover
what was
unknown

Test a
further
hypothesis
Data and evidence-based policy
Data

Programme

Ex post
Evaluation

Ex ante
evaluation

Programme

Data
CASE 1: CREATIVE CREDITS
Innovation
voucher
SME

Innovation project

RCT

Creative

SMEs receiving Credit 78% more
likely to undertake their project

✓

Strong evidence of S/T output
✓
additionality in terms of increased
innovations after six months Source: Bakhshi
et al (2011)
CASE 1: CREATIVE CREDITS
Innovation
voucher
SME

Innovation project

RCT

Creative

But no significant output
additionality after 12 months

X
X

No significant network or behavioral
additionality after 12 months Source: Bakhshi
et al (2013)
Manchester SME needs at turn of 2010
CASE 2: DIGITAL R&D FOR THE ARTS
Arts
organisations

Funding

Technology
companies

DIGITAL R&D
FUND
Digital R&D
Projects

Academic researchers

£7 million, 2012-15
50-60 R&D projects?
Sector-wide learning
Top 10% of
organisations
by how important
they judge digital
technology to be to
different activities
1736 new Twitter following
connections between attendees
after LeWeb’12 London
24% ↑ in total number of
following connections between
attendees

8% ↑ in total number of
following connections made by
attendees with non-attendees
Undertaking text analysis of tweets
between participants who connected at
LeWeb'12 London
THANKS!
Hasan.bakhshi@nesta.org.uk
@hasanbakhshi

Data networks and experimentation irc slides

  • 1.
    Data, networks andexperimentation Hasan Bakhshi Nesta Policy & Research Unit IRC Annual Summit, 26th November, 2013 http://www.nesta.org.uk/
  • 2.
    “Innovation policy would workbetter, we suggest, if modelled on experimental science and directed to the task of minimising the uncertainty that entrepreneurs face in the discovery of opportunities and constraints”
  • 3.
    “…uncertainty is adefining feature of emergent areas subject to persistent structural change like the creative industries, and should be dealt with in a systematic way.”
  • 4.
  • 5.
    Innovation policy asa process Test a hypothesis Discover what was unknown Test a further hypothesis
  • 6.
    Data and evidence-basedpolicy Data Programme Ex post Evaluation Ex ante evaluation Programme Data
  • 7.
    CASE 1: CREATIVECREDITS Innovation voucher SME Innovation project RCT Creative SMEs receiving Credit 78% more likely to undertake their project ✓ Strong evidence of S/T output ✓ additionality in terms of increased innovations after six months Source: Bakhshi et al (2011)
  • 8.
    CASE 1: CREATIVECREDITS Innovation voucher SME Innovation project RCT Creative But no significant output additionality after 12 months X X No significant network or behavioral additionality after 12 months Source: Bakhshi et al (2013)
  • 9.
    Manchester SME needsat turn of 2010
  • 10.
    CASE 2: DIGITALR&D FOR THE ARTS Arts organisations Funding Technology companies DIGITAL R&D FUND Digital R&D Projects Academic researchers £7 million, 2012-15 50-60 R&D projects? Sector-wide learning
  • 11.
    Top 10% of organisations byhow important they judge digital technology to be to different activities
  • 12.
    1736 new Twitterfollowing connections between attendees after LeWeb’12 London 24% ↑ in total number of following connections between attendees 8% ↑ in total number of following connections made by attendees with non-attendees
  • 13.
    Undertaking text analysisof tweets between participants who connected at LeWeb'12 London
  • 14.

Editor's Notes

  • #13 HASANGreen lines are the following connections between attendees at the LeWeb12 London conference formed in the three months after the conference. The light blue lines are the new following connections between attendees and speakers. The dark blue lines are the new following connections involving speakers.Within that three-month period, 1736 new following connections formed between attendees. After allowing for un-following activity, this represents a 24% increase in the number of connections between attendees. This compares with an 8% increase in the number of their following connections with non-attendees. Can we attribute all of this to the event? Unfortunately not! There is massive self-selection in event attendance – people go along tend to have common interests which means that they are more likely to form new connections with each other than with others. Running a controlled experiment where you randomly decide which events people can go to is not realistic! But we are exploring ways of using other data, including other properties of individuals’ Twitter networks, to control for their propensity to follow each other.
  • #14 HASANAdditional connections are valuable insofar as they lead to information flows that would not otherwise flow so directly and greater awareness. But on their own the connections are weak. We also want to know if they trigger stronger connections; content analysis of the tweets may give proximate indications. The frequency with which words like ‘meeting’ and ‘email’ appear in this wordcloud trivially illustrates what I’m getting at, but what we’re looking at in much greater depth.