Open Data & The Rewards of Failure

Open Data & The Rewards of Failure Future Everything Conference, Manchester, May 23 2010

Me: Chris Taggart @countculture

Me: Chris Taggart @countculture ✤ Developer of OpenlyLocal.com

Me: Chris Taggart @countculture ✤ Developer of OpenlyLocal.com ✤ Makes local data (esp govt) available in consistent and accessible form

Me: Chris Taggart @countculture ✤ Developer of OpenlyLocal.com ✤ Makes local data (esp govt) available in consistent and accessible form ✤ Scrapes council and govt websites

Me: Chris Taggart @countculture ✤ Developer of OpenlyLocal.com ✤ Makes local data (esp govt) available in consistent and accessible form ✤ Scrapes council and govt websites ✤ All open data

Me: Chris Taggart @countculture ✤ Developer of OpenlyLocal.com ✤ Makes local data (esp govt) available in consistent and accessible form ✤ Scrapes council and govt websites RDF JSON XML ✤ All open data

Me: Chris Taggart @countculture ✤ Developer of OpenlyLocal.com ✤ Makes local data (esp govt) available in consistent and accessible form ✤ Scrapes council and govt websites RDF JSON XML ✤ All open data ✤ Open Election Data project

Me: Chris Taggart @countculture ✤ Developer of OpenlyLocal.com ✤ Makes local data (esp govt) available in consistent and accessible form ✤ Scrapes council and govt websites RDF JSON XML ✤ All open data ✤ Open Election Data project ✤ Helping local authorities publish their election results as open data

Me: Chris Taggart @countculture ✤ Developer of OpenlyLocal.com ✤ Makes local data (esp govt) available in consistent and accessible form ✤ Scrapes council and govt websites RDF JSON XML ✤ All open data ✤ Open Election Data project ✤ Helping local authorities publish their election results as open data ✤ Tackling the open local data problem, one set at a time & learning lessons on the way

Me: Chris Taggart @countculture ✤ Developer of OpenlyLocal.com ✤ Makes local data (esp govt) available in consistent and accessible form ✤ Scrapes council and govt websites RDF JSON XML ✤ All open data ✤ Open Election Data project ✤ Helping local authorities publish their election results as open data ✤ Tackling the open local data problem, one set at a time & learning lessons on the way ✤ Member of Local Public Data Panel

Open data: why do we want it?

Open data: why do we want it? ✤ Stupid question: It’s our data

Open data: why do we want it? ✤ Stupid question: It’s our data ✤ Engagement – reducing barriers to getting involved

Open data: why do we want it? ✤ Stupid question: It’s our data ✤ Engagement – reducing barriers to getting involved ✤ Equality of access to information (and thus power). Much of this data is already available... for a price

Open data: why do we want it? ✤ Stupid question: It’s our data ✤ Engagement – reducing barriers to getting involved ✤ Equality of access to information (and thus power). Much of this data is already available... for a price ✤ Efﬁciency – Endless amount of duplication in all levels of government because they can’t/won’t share data

Open data: why do we want it? ✤ Stupid question: It’s our data ✤ Engagement – reducing barriers to getting involved ✤ Equality of access to information (and thus power). Much of this data is already available... for a price ✤ Efﬁciency – Endless amount of duplication in all levels of government because they can’t/won’t share data ✤ but most important of all...

Transparency so that we see what’s going on (and understand it too)

But wait a minute... Lord Mandelson photo: http://www.ﬂickr.com/photos/bisgovuk/4382508152/

But wait a minute... Lord Mandelson photo: http://www.ﬂickr.com/photos/bisgovuk/4382508152/ To some people and companies these are all reasons for not doing it. From their perspective, it’s all downside

Open data: why do we want it? ✤ Stupid question: It’s our data ✤ Engagement – reducing barriers to getting involved ✤ Equality of access to information (and thus power). Much of this data is already available... for a price ✤ Efﬁciency – Endless amount of duplication in all levels of government because they can’t/won’t share data ✤ but most important of all...

don’t the Open data: why do we want tit?ve it? o ha y want you ✤ Stupid question: It’s our data ✤ Engagement – reducing barriers to getting involved ✤ Equality of access to information (and thus power). Much of this data is already available... for a price ✤ Efﬁciency – Endless amount of duplication in all levels of government because they can’t/won’t share data ✤ but most important of all...

don’t the Open data: why do we want tit?ve it? o ha y want you t; we collected it, ✤ No it’s no It’svourus power Stupid question:t gi es data it’s wha ✤ Engagement – reducing barriers to getting involved ✤ Equality of access to information (and thus power). Much of this data is already available... for a price ✤ Efﬁciency – Endless amount of duplication in all levels of government because they can’t/won’t share data ✤ but most important of all...

don’t the Open data: why do we want tit?ve it? o ha y want you t; we collected it, ✤ No it’s no It’svourus power Stupid question:t gi es data it’s wha ✤ Engagement – duces power of to Rereducing barriers olitical/ p getting involvedpublic service elite ✤ Equality of access to information (and thus power). Much of this data is already available... for a price ✤ Efﬁciency – Endless amount of duplication in all levels of government because they can’t/won’t share data ✤ but most important of all...

don’t the Open data: why do we want tit?ve it? o ha y want you t; we collected it, ✤ No it’s no It’svourus power Stupid question:t gi es data it’s wha ✤ Engagement – duces power of to Rereducing barriers olitical/ p getting involvedpublic service elite ✤ Equality of access to information (and thus power).goes the competitive Bang Much of this data is already available... antage adv for a price ✤ Efﬁciency – Endless amount of duplication in all levels of government because they can’t/won’t share data ✤ but most important of all...

don’t the Open data: why do we want tit?ve it? o ha y want you t; we collected it, ✤ No it’s no It’svourus power Stupid question:t gi es data it’s wha ✤ Engagement – duces power of to Rereducing barriers olitical/ p getting involvedpublic service elite ✤ Equality of access to information (and thus power).goes the competitive Bang Much of this data is already available... antage adv for a price ✤ Efﬁciency – Endless amount of Fewer people = difﬁcult d ecisions, duplication in all levels of government less power, less prestige because they can’t/won’t share data ✤ but most important of all...

don’t the Open data: why do we want tit?ve it? o ha y want you t; we collected it, ✤ No it’s no It’svourus power Stupid question:t gi es data it’s wha ✤ Engagement – duces power of to Rereducing barriers olitical/ p getting involvedpublic service elite ✤ Equality of access to information (and thus power).goes the competitive Bang Much of this data is already available... antage adv for a price ✤ Efﬁciency – Endless amount of Fewer people = difﬁcult d ecisions, duplication in all levels of government less power, less prestige because they can’t/won’t share data ✤ but most important of all... scary

Transparency so that we see what’s going on (and understand it too)

Transparency an that jus ho what’s going so d see wetseew truly screwed on (and up things are. it too) understand

The rewards of failure today

The rewards of failure today ✤ Big projects/policy initiatives bring media attention, power & inﬂuence (and the key individuals will have moved on by the time its failure is publicly realized)

The rewards of failure today ✤ Big projects/policy initiatives bring media attention, power & inﬂuence (and the key individuals will have moved on by the time its failure is publicly realized) ✤ Spread the risk by emphasizing process over results, scoping everything out to the nth degree (you get very little beneﬁt from doing good in public service; lots of grief by visibly doing bad)

The rewards of failure today ✤ Big projects/policy initiatives bring media attention, power & influence (and the key individuals will have moved on by the time its failure is publicly realized) ✤ Spread the risk by emphasizing process over results, scoping everything out to the nth degree (you get very little benefit from doing good in public service; lots of grief by visibly doing bad) ✤ Don’t do small, quick, innovative projects (it’s too easy to see whether they have succeeded... or failed, and difficult to spread the blame)

The rewards of failure today ✤ Big projects/policy initiatives bring media attention, power & influence (and the key individuals will have moved on by the time its failure is publicly realized) ✤ Spread the risk by emphasizing process over results, scoping everything out to the nth degree (you get very little benefit from doing good in public service; lots of grief by visibly doing bad) ✤ Don’t do small, quick, innovative projects (it’s too easy to see whether they have succeeded... or failed, and difficult to spread the blame) ✤ Outsource everything (see ‘process over results’ above. Bonus: ‘commercial confidentiality’ means the facts can kept secret)

no better The private sector’s http://www.ﬂickr.com/photos/pingnews/2935021856/

http://www.ﬂickr.com/photos/pingnews/2935021856/ The private sector’s no better ✤ Finance industry incentivized to take risks, reap short-term rewards and let others bear the consequences

http://www.ﬂickr.com/photos/pingnews/2935021856/ The private sector’s no better ✤ Finance industry incentivized to take risks, reap short-term rewards and let others bear the consequences ✤ Bigger, more opaque and more complex schemes give protection to people & companies (until it all falls apart)

http://www.ﬂickr.com/photos/pingnews/2935021856/ The private sector’s no better ✤ Finance industry incentivized to take risks, reap short-term rewards and let others bear the consequences ✤ Bigger, more opaque and more complex schemes give protection to people & companies (until it all falls apart) ✤ Senior executives insulated from shareholders by diverse and diffuse nature of the reporting/ownership system

http://www.ﬂickr.com/photos/pingnews/2935021856/ The private sector’s no better ✤ Finance industry incentivized to take risks, reap short-term rewards and let others bear the consequences ✤ Bigger, more opaque and more complex schemes give protection to people & companies (until it all falls apart) ✤ Senior executives insulated from shareholders by diverse and diffuse nature of the reporting/ownership system ✤ People at the top are incentivized to make big changes (because intelligent comparison becomes difﬁcult); people at the bottom incentivized to do as they’re told

The rewards of failure done right

The rewards of failure done right ✤ The open-source movement has been founded on failure and ﬁxing that failure, with amazing results (and amazing stability)

The rewards of failure done right ✤ The open-source movement has been founded on failure and ﬁxing that failure, with amazing results (and amazing stability) ✤ More than that, it’s failure in public. Makes all bugs shallow; all faults visible

The rewards of failure done right ✤ The open-source movement has been founded on failure and ﬁxing that failure, with amazing results (and amazing stability) ✤ More than that, it’s failure in public. Makes all bugs shallow; all faults visible ✤ It also allows a meritocracy to develop, incentivizing innovation and quality, encouraging people to do their best work

The rewards of failure done right ✤ The open-source movement has been founded on failure and ﬁxing that failure, with amazing results (and amazing stability) ✤ More than that, it’s failure in public. Makes all bugs shallow; all faults visible ✤ It also allows a meritocracy to develop, incentivizing innovation and quality, encouraging people to do their best work ✤ The internet and the web are not only accepting of failure, the inevitability of failure is at the heart

How can Open Data help?

How can Open Data help? ✤ It helps change the incentives, from Fail Big, Fail Slow to Fail Small, Fail Fast, and Fail Forward (i.e. building on the failure)

How can Open Data help? ✤ It helps change the incentives, from Fail Big, Fail Slow to Fail Small, Fail Fast, and Fail Forward (i.e. building on the failure) ✤ It helps surface issues/problems making them less likely to be repeated (learning from other people’s failures)

How can Open Data help? ✤ It helps change the incentives, from Fail Big, Fail Slow to Fail Small, Fail Fast, and Fail Forward (i.e. building on the failure) ✤ It helps surface issues/problems making them less likely to be repeated (learning from other people’s failures) ✤ It allows you to stand on the shoulders of giants. Don’t reinvent the wheel – use or improve someone else’s

How can Open Data help? ✤ It helps change the incentives, from Fail Big, Fail Slow to Fail Small, Fail Fast, and Fail Forward (i.e. building on the failure) ✤ It helps surface issues/problems making them less likely to be repeated (learning from other people’s failures) ✤ It allows you to stand on the shoulders of giants. Don’t reinvent the wheel – use or improve someone else’s ✤ It increases the quality of data – reuse by other people and combining with other data identiﬁes (and helps solve) quality issues far more quickly and cheaply than internal checking

We need to recast the debate

We need to recast the debate ✤ Perhaps we need to talk about continuous improvement rather than failure; evolution (which has failure built in) rather than revolution

We need to recast the debate ✤ Perhaps we need to talk about continuous improvement rather than failure; evolution (which has failure built in) rather than revolution ✤ Be upfront about problems, knowledge gaps, issues

We need to recast the debate ✤ Perhaps we need to talk about continuous improvement rather than failure; evolution (which has failure built in) rather than revolution ✤ Be upfront about problems, knowledge gaps, issues ✤ Less centralised complexity; more distributed innovation

We need to recast the debate ✤ Perhaps we need to talk about continuous improvement rather than failure; evolution (which has failure built in) rather than revolution ✤ Be upfront about problems, knowledge gaps, issues ✤ Less centralised complexity; more distributed innovation ✤ Make public data open by default (and include outsourced work/ data), particularly anything to do with spending money

We need to recast the debate ✤ Perhaps we need to talk about continuous improvement rather than failure; evolution (which has failure built in) rather than revolution ✤ Be upfront about problems, knowledge gaps, issues ✤ Less centralised complexity; more distributed innovation ✤ Make public data open by default (and include outsourced work/ data), particularly anything to do with spending money ✤ Outsource commodity tasks, not skills and knowledge (and avoid the used-car buyer information trap). Stop the vendoritis

We need to recast the debate ✤ Perhaps we need to talk about continuous improvement rather than failure; evolution (which has failure built in) rather than revolution ✤ Be upfront about problems, knowledge gaps, issues ✤ Less centralised complexity; more distributed innovation ✤ Make public data open by default (and include outsourced work/ data), particularly anything to do with spending money ✤ Outsource commodity tasks, not skills and knowledge (and avoid the used-car buyer information trap). Stop the vendoritis ✤ Reward small-scale innovation and spending less money (not big projects and spending more money)

In short, less this...

...and more this 5 years of Linux kernel development in Git

http://reinikainen.co.uk	537
http://countculture.wordpress.com	207
http://www.futuregovnetwork.com	178
http://www.slideshare.net	66
http://www.theenvisioners.com	49
http://nuevosmedios.posterous.com	46
http://gobundlr.com	38
http://futuregovnetwork.com	34
http://www.africaopendata.org	27
http://davepress.net	20
http://bundlr.com	19
http://sarahhartley.wordpress.com	19
http://themancunianway.wordpress.com	18
http://sarahhartley.me.uk	8
http://www.sarahhartley.me.uk	5
http://ranksit.com	3
http://www.humancenteredict.nl	2
http://honyaku.yahoofs.jp	1
http://lanyrd.com	1
http://avantothink.net	1
http://theenvisioners.com	1
http://www.dmdlaw.co.uk	1
http://twittertim.es	1

Open Data & The Rewards of Failure

by Chris Taggart on May 13, 2010

Accessibility

Categories

Upload Details

Usage Rights

23 Embeds 1,282

Statistics

Open Data & The Rewards of Failure Presentation Transcript