Open Data: its value and lessons learned
Andrew Stott
UK Transparency Board
formerly Director, data.gov.uk

@dirdigeng
andrew.stott@dirdigeng.com

03 Feb 2014
Triple Objectives

Objectives of Open Data

More Transparent Government

Improved public services

New Economic and Social Value
2
Triple Objectives

Objectives of Open Data

More Transparent Government

Improved public services

New Economic and Social Value
3
4
5
Open Data as a Transport Investment in London
 ~500 Applications
(mobile, web, others)
 ~5000 people involved
in “app industry”

 As a transport project
alone, evaluated by
usual economic criteria:
ROI = 58:1
 Transport For London
have stopped making
their own apps
6
Anonymised medical prescription data
 Used experts in
 Health
 Data Analytics
 Analysed 35m
data records
 8 weeks
 £200m+/yr
savings
 Repeatable
 Could scale to
£1.5bn
7
National Information Infrastructure

8
Denmark: Open Address Data
Period

Benefits

Costs

Return on
Investment

2004-09
>€60m
(including
setup)

~€2m

22:1

2010
(steady
state)

~€0.2m 70:1

~€14.0m

9
Commercial Meteorology in the US and
Europe in 2004
The size of the US and EU economies are approximately the same
United States (1)

Europe (2)

(open)

(closed/charged)

$ 400-700 million

$ 30-50 million

Number of Firms

400

30

Number of
Employees

4000

300

$1,929m

$144m

Gross Receipts

Annual Value of
Weather Risk
Contracts (3)

Sources: Weiss 2004, using Commercial Weather Services Association (1), Meteoconsult
(2), Weather Risk Management Association (3)
10
UK Weather
Public
Weather
Service now
Open Data:
 Costs
£83m/yr
 Total
benefits
£614m/yr
Cadastral data
estate
agents/
realtors

Financial
services

builders and
other local
services
12
13
$930m business from Open Data
 Weather for 1m
points
 60 years of crop
yield data
 14 TB of soil
data
 Company
formed in 2006
 Sold to
Monsanto
October 2013 for
$930m cash
14
Roadworks Data
Councils:
 Cost £0.7m a year
 Benefits £6.3m a
year
 Fiscal ROI 9:1
Overall value
 Wider benefits of
additional £19m a
year.
 Overall ROI 28:1
15
16
17
18
Evidence base for EU Open Data Directive
 Open Gov Data in EU would
‒ increase business activity by up to €40 Bn/yr
‒ have total benefits up to €140 Bn/yr
 Open Data was reused 10x-100x more than
charged-for data
 Lowering charges may attract new types of reusers, in particular SMEs.
 Costs appear to increase very little: in fact, they
may eventually decrease
All economic analysis and case studies
point the same way
19
Deloitte POPSIS Case Studies
Case

Data Type

Country

Increase

BEV

Mapping

AT

Downloads: +200% to +7,000%

DECA

Addresses

DK

Uers: +10,000%

Destatis

Statistics

DE

Users: +1800%
Downloads: +800%

IGNCNIG

Mapping

ES

Volume: +200%
Users: +200%

KNMI

Weather

NL

Users: +1000%

Met.No

Weather

NO

Users: +3000%

Spanish
Cadaster

Cadastral

ES

Downloads: +800% to +1900%
20
Case Study: Statistics Germany
2004
Pay per use

2010
Free of charge

110,000 EUR

0 EUR

Premium
analyses

78,000 EUR

152,000 EUR

Downloads
Standard
Accounts
Premium
Accounts

130,300
1,800

1,093,000
3,000

36

69

Online
publications
Online sales

21
Case Study: Statistics Germany

Users: +1800% Downloads: +800%
22
Government can be an Open Data user too

Greater Manchester estimated £6.5m
savings from finding and using its own
data more easily

23
EU Inspire Directive on Geospatial Data

One Government reported fiscal ROI 8:1
in first 4 years, plus wider benefits

24
British Columbia Open Data
 Government
itself is #1
user of its
data
 33% of
downloads
come from
within BC
Government

25
Measuring Benefits is not easy
 Benefits take time to emerge
 Most benefits are from Open Data plus
innovation
 Difficult to measure consumer surplus, but
that’s where more of the value often is
 Difficult to value public sector benefits
 Benefits not predictable
 “National Information Infrastructure” is a good
source of benefits
26
Triple Objectives

Objectives of Open Data

More Transparent Government

Improved public services

New Economic and Social Value
27
Performance of individual schools

28
Performance of individual hospitals

Patient
ratings

12+ Weeks
MRSA-free

Blood
clots
2 recent
MRSA
Low
Mortality

Good C-Diff
record
29
Performance of local police and courts

30
Open Data used to drive Citizen Engagement
Accessible data on crime

It’s very local

Local team
How YOU
can get
involved
Local police
Twitter feed

Telephone, website, Facebook and Youtube ….

Attract

Inform

Engage

Action

31
“

“Police.uk is a success story”
“I am constantly afraid of becoming a victim of
crime, but this website has made me more
relaxed now that I know what has happened
and where. Crimes are not quite as rife in my
area as I imagined. I also feel that I have some
sort of link now with my local police. Well done”

“The new 'Draw Your Own
Area' function is a vast
improvement which will
allow regular comparisons
to be made about an entire
town or village.”

Site feedback: 70% of respondents
agree website is easy to use whilst
66% agree information is easy to
understand

“It's just great to have the
transparency because it will
encourage me and others to report
crime because the result of doing so
is now visual rather than notional.”

“Local information for interest, and
details of who my local officers are. It's
good to feel involved and informed. I
think this will help us all take more
responsibility for our own areas”

£300,000 set-up, £150,000/year to run

Over 56 million
visits since
January 2011
33% of adults
aware of and 11%
have used the
site
2/3 of users say
they’ll return to
the site to see if
crime goes up
and down
2/5 of users say
they’re now more
likely to take
steps to improve
their personal
32
safety
Triple Objectives

Objectives of Open Data

More Transparent Government

Improved public services

New Economic and Social Value
33
Is My Money Being Spent Well?

34
Organisational Transparency

Pay

Responsibil

Where the person
is in the structure

Contact details
35
International Corporate Transparency

36
First 4 years of data.gov.uk: Lessons Learned
 Over 10,000 datasets
 37 GB of geo data
 Public Data Principles
 Open Government
Licence
 Transparency of
salaries, spending,
contracts and tenders
 Four site versions, each
in response to user
feedback
37
Leadership inside and outside the Organisation

38
Demand from business and citizens

39
Clear, common, licensing approach

40
Standards

41
Make sure data is re-usable

42
Ensure Privacy of Personal Data

43
Focus on data on things that people care about

44
It’s not just about new data
Scope for “Open Data” also includes data
previously “published” but …
 in non-reusable format
 with restricted licence
 only aimed at specialist groups
 only for payment
 only in response to requests
 difficult to find
data.gov.uk contains a lot of data which
nobody knew was already published

45
Handling the concerns of data owners

“People hug their database, they don't want to
let it go. You have no idea the number of
excuses people come up with to hang onto
their data and not give it to you, even though
you've paid for it as a taxpayer.”
– Tim Berners-Lee
http://www.ted.com/talks/tim_berners_lee_on_the_next_web.html

46
Release data people want

47
Manage expectations, prepare for mistakes
“We’re making a small start next
week. But eventually, it’s going to
make a big difference.”
“The information we’re publishing
next week won’t be perfect, and
I’m sure there’ll be some mistakes.
But I want to get on with it.”
UK Prime Minister 29 May 2010

48
Delivery Incrementally

49
Identify datasets not yet released

50
Ensure continuing conformance

51
Not all Government data is accurate

52
Data Quality
 Release of data will
reveal issues of data
quality
 Surprisingly little
criticism
 Celebrate greater
checking of data!
 Use as stimulus to
 Measure
 Prioritise
 Improve
53
Promote Use of Data

54
Continuously engage with developers

Photos: @memespring,
@MadLabUK, @paul_clarke

55
Open Data Ecosystem

56
UK Open Data Institute
 Develop capability of UK
businesses to exploit value of
Open Data
 Engage developers/small
businesses to build Open Data
supply chains and commercial
outlets
 Help public sector use its own
data more effectively
 Ensure academic research in
Open Data technologies
57
UK Open Data Institute
 Running ~12 months
 £200m/yr savings identified
 5 startups incubated, 6 courses launched,
4 hackathons
 27 private-sector company paying members
 Over £2m of private sector funding secured
in 6 mths
 1,500 visitors to London space – and
provides “neutral meeting space” for
government and entrepreneurs
58
… and the biggest lesson of all

Overcome obstacles
practically
by doing,
not debating

59
Discussion?

60
End

61
62

Open Data: Its Value and Lessons Learned

  • 1.
    Open Data: itsvalue and lessons learned Andrew Stott UK Transparency Board formerly Director, data.gov.uk @dirdigeng andrew.stott@dirdigeng.com 03 Feb 2014
  • 2.
    Triple Objectives Objectives ofOpen Data More Transparent Government Improved public services New Economic and Social Value 2
  • 3.
    Triple Objectives Objectives ofOpen Data More Transparent Government Improved public services New Economic and Social Value 3
  • 4.
  • 5.
  • 6.
    Open Data asa Transport Investment in London  ~500 Applications (mobile, web, others)  ~5000 people involved in “app industry”  As a transport project alone, evaluated by usual economic criteria: ROI = 58:1  Transport For London have stopped making their own apps 6
  • 7.
    Anonymised medical prescriptiondata  Used experts in  Health  Data Analytics  Analysed 35m data records  8 weeks  £200m+/yr savings  Repeatable  Could scale to £1.5bn 7
  • 8.
  • 9.
    Denmark: Open AddressData Period Benefits Costs Return on Investment 2004-09 >€60m (including setup) ~€2m 22:1 2010 (steady state) ~€0.2m 70:1 ~€14.0m 9
  • 10.
    Commercial Meteorology inthe US and Europe in 2004 The size of the US and EU economies are approximately the same United States (1) Europe (2) (open) (closed/charged) $ 400-700 million $ 30-50 million Number of Firms 400 30 Number of Employees 4000 300 $1,929m $144m Gross Receipts Annual Value of Weather Risk Contracts (3) Sources: Weiss 2004, using Commercial Weather Services Association (1), Meteoconsult (2), Weather Risk Management Association (3) 10
  • 11.
    UK Weather Public Weather Service now OpenData:  Costs £83m/yr  Total benefits £614m/yr
  • 12.
  • 13.
  • 14.
    $930m business fromOpen Data  Weather for 1m points  60 years of crop yield data  14 TB of soil data  Company formed in 2006  Sold to Monsanto October 2013 for $930m cash 14
  • 15.
    Roadworks Data Councils:  Cost£0.7m a year  Benefits £6.3m a year  Fiscal ROI 9:1 Overall value  Wider benefits of additional £19m a year.  Overall ROI 28:1 15
  • 16.
  • 17.
  • 18.
  • 19.
    Evidence base forEU Open Data Directive  Open Gov Data in EU would ‒ increase business activity by up to €40 Bn/yr ‒ have total benefits up to €140 Bn/yr  Open Data was reused 10x-100x more than charged-for data  Lowering charges may attract new types of reusers, in particular SMEs.  Costs appear to increase very little: in fact, they may eventually decrease All economic analysis and case studies point the same way 19
  • 20.
    Deloitte POPSIS CaseStudies Case Data Type Country Increase BEV Mapping AT Downloads: +200% to +7,000% DECA Addresses DK Uers: +10,000% Destatis Statistics DE Users: +1800% Downloads: +800% IGNCNIG Mapping ES Volume: +200% Users: +200% KNMI Weather NL Users: +1000% Met.No Weather NO Users: +3000% Spanish Cadaster Cadastral ES Downloads: +800% to +1900% 20
  • 21.
    Case Study: StatisticsGermany 2004 Pay per use 2010 Free of charge 110,000 EUR 0 EUR Premium analyses 78,000 EUR 152,000 EUR Downloads Standard Accounts Premium Accounts 130,300 1,800 1,093,000 3,000 36 69 Online publications Online sales 21
  • 22.
    Case Study: StatisticsGermany Users: +1800% Downloads: +800% 22
  • 23.
    Government can bean Open Data user too Greater Manchester estimated £6.5m savings from finding and using its own data more easily 23
  • 24.
    EU Inspire Directiveon Geospatial Data One Government reported fiscal ROI 8:1 in first 4 years, plus wider benefits 24
  • 25.
    British Columbia OpenData  Government itself is #1 user of its data  33% of downloads come from within BC Government 25
  • 26.
    Measuring Benefits isnot easy  Benefits take time to emerge  Most benefits are from Open Data plus innovation  Difficult to measure consumer surplus, but that’s where more of the value often is  Difficult to value public sector benefits  Benefits not predictable  “National Information Infrastructure” is a good source of benefits 26
  • 27.
    Triple Objectives Objectives ofOpen Data More Transparent Government Improved public services New Economic and Social Value 27
  • 28.
  • 29.
    Performance of individualhospitals Patient ratings 12+ Weeks MRSA-free Blood clots 2 recent MRSA Low Mortality Good C-Diff record 29
  • 30.
    Performance of localpolice and courts 30
  • 31.
    Open Data usedto drive Citizen Engagement Accessible data on crime It’s very local Local team How YOU can get involved Local police Twitter feed Telephone, website, Facebook and Youtube …. Attract Inform Engage Action 31
  • 32.
    “ “Police.uk is asuccess story” “I am constantly afraid of becoming a victim of crime, but this website has made me more relaxed now that I know what has happened and where. Crimes are not quite as rife in my area as I imagined. I also feel that I have some sort of link now with my local police. Well done” “The new 'Draw Your Own Area' function is a vast improvement which will allow regular comparisons to be made about an entire town or village.” Site feedback: 70% of respondents agree website is easy to use whilst 66% agree information is easy to understand “It's just great to have the transparency because it will encourage me and others to report crime because the result of doing so is now visual rather than notional.” “Local information for interest, and details of who my local officers are. It's good to feel involved and informed. I think this will help us all take more responsibility for our own areas” £300,000 set-up, £150,000/year to run Over 56 million visits since January 2011 33% of adults aware of and 11% have used the site 2/3 of users say they’ll return to the site to see if crime goes up and down 2/5 of users say they’re now more likely to take steps to improve their personal 32 safety
  • 33.
    Triple Objectives Objectives ofOpen Data More Transparent Government Improved public services New Economic and Social Value 33
  • 34.
    Is My MoneyBeing Spent Well? 34
  • 35.
    Organisational Transparency Pay Responsibil Where theperson is in the structure Contact details 35
  • 36.
  • 37.
    First 4 yearsof data.gov.uk: Lessons Learned  Over 10,000 datasets  37 GB of geo data  Public Data Principles  Open Government Licence  Transparency of salaries, spending, contracts and tenders  Four site versions, each in response to user feedback 37
  • 38.
    Leadership inside andoutside the Organisation 38
  • 39.
    Demand from businessand citizens 39
  • 40.
  • 41.
  • 42.
    Make sure datais re-usable 42
  • 43.
    Ensure Privacy ofPersonal Data 43
  • 44.
    Focus on dataon things that people care about 44
  • 45.
    It’s not justabout new data Scope for “Open Data” also includes data previously “published” but …  in non-reusable format  with restricted licence  only aimed at specialist groups  only for payment  only in response to requests  difficult to find data.gov.uk contains a lot of data which nobody knew was already published 45
  • 46.
    Handling the concernsof data owners “People hug their database, they don't want to let it go. You have no idea the number of excuses people come up with to hang onto their data and not give it to you, even though you've paid for it as a taxpayer.” – Tim Berners-Lee http://www.ted.com/talks/tim_berners_lee_on_the_next_web.html 46
  • 47.
  • 48.
    Manage expectations, preparefor mistakes “We’re making a small start next week. But eventually, it’s going to make a big difference.” “The information we’re publishing next week won’t be perfect, and I’m sure there’ll be some mistakes. But I want to get on with it.” UK Prime Minister 29 May 2010 48
  • 49.
  • 50.
    Identify datasets notyet released 50
  • 51.
  • 52.
    Not all Governmentdata is accurate 52
  • 53.
    Data Quality  Releaseof data will reveal issues of data quality  Surprisingly little criticism  Celebrate greater checking of data!  Use as stimulus to  Measure  Prioritise  Improve 53
  • 54.
  • 55.
    Continuously engage withdevelopers Photos: @memespring, @MadLabUK, @paul_clarke 55
  • 56.
  • 57.
    UK Open DataInstitute  Develop capability of UK businesses to exploit value of Open Data  Engage developers/small businesses to build Open Data supply chains and commercial outlets  Help public sector use its own data more effectively  Ensure academic research in Open Data technologies 57
  • 58.
    UK Open DataInstitute  Running ~12 months  £200m/yr savings identified  5 startups incubated, 6 courses launched, 4 hackathons  27 private-sector company paying members  Over £2m of private sector funding secured in 6 mths  1,500 visitors to London space – and provides “neutral meeting space” for government and entrepreneurs 58
  • 59.
    … and thebiggest lesson of all Overcome obstacles practically by doing, not debating 59
  • 60.
  • 61.
  • 62.