Open local data presentation for okcon

Open Local Data, OpenlyLocal & the Open Election Data project Open Knowledge Conference, London, April 24, 2010

It’s been a good year for open government data data.gov.uk, Ordnance Survey, MPs expenses...

But what about local data?

But what about local data? • Frankly it’s a mess

But what about local data? • Frankly it’s a mess • Sporadically published by central government

But what about local data? • Frankly it’s a mess • Sporadically published by central government • Inaccessible & impenetrable council websites

But what about local data? • Frankly it’s a mess • Sporadically published by central government • Inaccessible & impenetrable council websites • Opaque local public bodies and NDPBs (quangos)

But what about local data? • Frankly it’s a mess • Sporadically published by central government • Inaccessible & impenetrable council websites • Opaque local public bodies and NDPBs (quangos) • At best unclear & at worst unusable legal situation

But what about local data? • Frankly it’s a mess • Sporadically published by central government • Inaccessible & impenetrable council websites • Opaque local public bodies and NDPBs (quangos) • At best unclear & at worst unusable legal situation • Start with the basics. Who are the councillors, where do they represent and what committees do they sit on?

But what about local data? • Frankly it’s a mess • Sporadically published by central government • Inaccessible & impenetrable council websites • Opaque local public bodies and NDPBs (quangos) • At best unclear & at worst unusable legal situation • Start with the basics. Who are the councillors, where do they represent and what committees do they sit on? • How easy it that information to ﬁnd & reuse?

Enter OpenlyLocal...

Enter OpenlyLocal... • Inspiration was a Manchester project, MCC Work For You. Doing something concrete to solve a problem. First code written 1 year ago

Enter OpenlyLocal... • Inspiration was a Manchester project, MCC Work For You. Doing something concrete to solve a problem. First code written 1 year ago • Screen-scrapes council websites. Now over 140 councils done, with basic information on all 434 in the UK

Enter OpenlyLocal... • Inspiration was a Manchester project, MCC Work For You. Doing something concrete to solve a problem. First code written 1 year ago • Screen-scrapes council websites. Now over 140 councils done, with basic information on all 434 in the UK • Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG...

Enter OpenlyLocal... • Inspiration was a Manchester project, MCC Work For You. Doing something concrete to solve a problem. First code written 1 year ago • Screen-scrapes council websites. Now over 140 councils done, with basic information on all 434 in the UK • Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG... • Building a graph of connections – councillors, committees, social networking, police forces, pension funds

Enter OpenlyLocal... • Inspiration was a Manchester project, MCC Work For You. Doing something concrete to solve a problem. First code written 1 year ago • Screen-scrapes council websites. Now over 140 councils done, with basic information on all 434 in the UK • Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG... • Building a graph of connections – councillors, committees, social networking, police forces, pension funds • All open data – free for reuse, including commercial reuse

Enter OpenlyLocal... • Inspiration was a Manchester project, MCC Work For You. Doing something concrete to solve a problem. First code written 1 year ago • Screen-scrapes council websites. Now over 140 councils done, with basic information on all 434 in the UK • Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG... • Building a graph of connections – councillors, committees, social networking, police forces, pension funds • All open data – free for reuse, including commercial reuse • 100% accessible. Data ﬁrst; bells & whistles later

RDF JSON

RDF JSON XML

OpenlyLocal: where next?

OpenlyLocal: where next? • More data. More councils. More connections.

OpenlyLocal: where next? • More data. More councils. More connections. • Election data, cabinet members, mayors, police authorities, health authorities, pension funds, etc... it’s all about the network

OpenlyLocal: where next? • More data. More councils. More connections. • Election data, cabinet members, mayors, police authorities, health authorities, pension funds, etc... it’s all about the network • Data used by Hyperlocal sites (inc Guardian). Ning app. Google Gadget. Widget. And now starting to get info in the other direction (i.e. connecting blog posts to councillors)

OpenlyLocal: where next? • More data. More councils. More connections. • Election data, cabinet members, mayors, police authorities, health authorities, pension funds, etc... it’s all about the network • Data used by Hyperlocal sites (inc Guardian). Ning app. Google Gadget. Widget. And now starting to get info in the other direction (i.e. connecting blog posts to councillors) • Some things can’t be done programmatically. Need crowdsourcing tools & help with tying info to Wikipedia, other sources

OpenlyLocal: where next? • More data. More councils. More connections. • Election data, cabinet members, mayors, police authorities, health authorities, pension funds, etc... it’s all about the network • Data used by Hyperlocal sites (inc Guardian). Ning app. Google Gadget. Widget. And now starting to get info in the other direction (i.e. connecting blog posts to councillors) • Some things can’t be done programmatically. Need crowdsourcing tools & help with tying info to Wikipedia, other sources • Some good visualisations done, but lots more potential, particularly now that we have OS geo data

Why does open local data matter?

Why does open local data matter? • Transparency – can we see what’s going on (and understand it too)

Why does open local data matter? • Transparency – can we see what’s going on (and understand it too) • Engagement – reducing barriers to getting involved

Why does open local data matter? • Transparency – can we see what’s going on (and understand it too) • Engagement – reducing barriers to getting involved • Equality of access to information (and thus power). Much of this data is already available... for a price

Why does open local data matter? • Transparency – can we see what’s going on (and understand it too) • Engagement – reducing barriers to getting involved • Equality of access to information (and thus power). Much of this data is already available... for a price • Efﬁciency – Don’t let ﬁve different branches of government needlessly do the same thing (aka keeping it DRY)

Simple questions, hard to answer

Simple questions, hard to answer • How does the budget of my council compare with similar ones?

Simple questions, hard to answer • How does the budget of my council compare with similar ones? • What’s the background of the chief executive of my council?

Simple questions, hard to answer • How does the budget of my council compare with similar ones? • What’s the background of the chief executive of my council? • Is my route to work going to affected by roadworks next week?

Simple questions, hard to answer • How does the budget of my council compare with similar ones? • What’s the background of the chief executive of my council? • Is my route to work going to affected by roadworks next week? • What are the connections between the companies awarded contracts and the councillors & senior management?

Simple questions, hard to answer • How does the budget of my council compare with similar ones? • What’s the background of the chief executive of my council? • Is my route to work going to affected by roadworks next week? • What are the connections between the companies awarded contracts and the councillors & senior management? • How will cuts in services be decided and what can I do to inﬂuence the decisions?

Transparency A brief example.

Transparency A brief example. • A story in Private Eye

Transparency A brief example. • A story in Private Eye • An investigation by the District Auditor

Transparency A brief example. • A story in Private Eye • An investigation by the District Auditor • A report buried in the nether recesses of the website...

Transparency A brief example. • A story in Private Eye • An investigation by the District Auditor • A report buried in the nether recesses of the website... • ...with no relevant heading

Transparency A brief example. • A story in Private Eye • An investigation by the District Auditor • A report buried in the nether recesses of the website... • ...with no relevant heading • Accessible only as a PDF of a scan of a document

Transparency A brief example. • A story in Private Eye • An investigation by the District Auditor • A report buried in the nether recesses of the website... • ...with no relevant heading • Accessible only as a PDF of a scan of a document • Is it any wonder suspicions are raised?

Engagement

Engagement • Local Authorities now have a duty to engage. This means • Reducing barriers to understanding and involvement

Engagement • Local Authorities now have a duty to engage. This means • Reducing barriers to understanding and involvement • Accessible for all

Engagement • Local Authorities now have a duty to engage. This means • Reducing barriers to understanding and involvement • Accessible for all • Available to use when, where & how we want to

Engagement • Local Authorities now have a duty to engage. This means • Reducing barriers to understanding and involvement • Accessible for all • Available to use when, where & how we want to • Publishing as data means easy to be repurposed – for mobile, for mashups, for ofﬂine use (e.g. Postcode Paper)

Equality

Equality • At the moment, all this information is available... at a cost

Equality ££ £ • At the moment, all this information is available... at a cost

Equality ££ £ • At the moment, all this information is available... at a cost • Huge asymmetry of information (and thus power)

Equality ££ £ • At the moment, all this information is available... at a cost • Huge asymmetry of information (and thus power) • Raises the barriers to involvement, and, critically, to challenge

Efficiency Now: cumbersome, wasteful, opaque, error prone Typical local data ﬂows

Efficiency Now: cumbersome, wasteful, opaque, error prone Typical local data ﬂows Council

Efficiency Now: cumbersome, wasteful, opaque, error prone Typical local data ﬂows Govt Dept A email web Council Govt Dept B upload Govt Dept C

Efficiency Now: cumbersome, wasteful, opaque, error prone Typical local data ﬂows Govt Dept A public (maybe) email web Council Govt Dept B upload Govt Dept C

Efficiency A better way? Council

Efficiency A better way? open data Council

Efficiency A better way? <tr rel='openelection:candidacy'>  <th scope="row" rel="openelection:candidate">Keith William Cockroft</th> <td rel="openelection:party" resource="http:// openelectiondata.org/id/parties/6">Labour</td>  <td property="openelection:candidateVoteCount" datatype="xsd:integer">330</td> <td>16.6%</td> open data Council <td property="openelection:elected" datatype="xsd:boolean" content="false">No</td> </tr> <tr rel='openelection:candidacy'> <th scope="row" rel="openelection:candidate"> Brenda Lilian Constable 37 Morley Road, Burntwood, Staffordshire WS7 2DE </th>

Efficiency A better way? <tr rel='openelection:candidacy'>  <th scope="row" rel="openelection:candidate">Keith William Cockroft</th> <td rel="openelection:party" resource="http:// openelectiondata.org/id/parties/6">Labour</td>  <td property="openelection:candidateVoteCount" open data datatype="xsd:integer">330</td> <td>16.6%</td> Govt Dept A Council <td property="openelection:elected" datatype="xsd:boolean" content="false">No</td> </tr> <tr rel='openelection:candidacy'> <th scope="row" rel="openelection:candidate"> Govt Dept B Brenda Lilian Constable 37 Morley Road, Govt Dept C Burntwood, Staffordshire WS7 2DE </th>

Efficiency A better way? <tr rel='openelection:candidacy'>  <th scope="row" rel="openelection:candidate">Keith William Cockroft</th> <td rel="openelection:party" resource="http:// openelectiondata.org/id/parties/6">Labour</td>  <td property="openelection:candidateVoteCount" open data datatype="xsd:integer">330</td> <td>16.6%</td> Govt Dept A Council <td property="openelection:elected" datatype="xsd:boolean" content="false">No</td> </tr> <tr rel='openelection:candidacy'> <th scope="row" rel="openelection:candidate"> Govt Dept B Brenda Lilian Constable 37 Morley Road, Govt Dept C Burntwood, Staffordshire WS7 2DE </th> Publish once, consume many times

Problems

Problems • IDs – councils don’t generally use them; central government uses many... inconsistently

Problems • IDs – councils don’t generally use them; central government uses many... inconsistently • Data tied up in PDFs (will we ever get this back?)

Problems • IDs – councils don’t generally use them; central government uses many... inconsistently • Data tied up in PDFs (will we ever get this back?) • Legacy systems. That nobody now understands

Problems • IDs – councils don’t generally use them; central government uses many... inconsistently • Data tied up in PDFs (will we ever get this back?) • Legacy systems. That nobody now understands • Private companies/JVs. No FoI. No access to data

Problems • IDs – councils don’t generally use them; central government uses many... inconsistently • Data tied up in PDFs (will we ever get this back?) • Legacy systems. That nobody now understands • Private companies/JVs. No FoI. No access to data • Outsourcing (councils have outsourced the skills & knowledge and now are often dumb consumers)

Problems • IDs – councils don’t generally use them; central government uses many... inconsistently • Data tied up in PDFs (will we ever get this back?) • Legacy systems. That nobody now understands • Private companies/JVs. No FoI. No access to data • Outsourcing (councils have outsourced the skills & knowledge and now are often dumb consumers) • Silos, especially of budgets. How do you get money from one budget to another (even if the whole beneﬁts)

The OpenElectionData project

The OpenElectionData project • Tackling the open local data problem, one set at a time & learning lessons on the way

The OpenElectionData project • Tackling the open local data problem, one set at a time & learning lessons on the way • Succeed, or fail forward

The OpenElectionData project • Tackling the open local data problem, one set at a time & learning lessons on the way • Succeed, or fail forward • No public database of local election results, only a commercial one (subsidized by the Electoral Commission)

The OpenElectionData project • Tackling the open local data problem, one set at a time & learning lessons on the way • Succeed, or fail forward • No public database of local election results, only a commercial one (subsidized by the Electoral Commission) • Allows even those with no prior knowledge of linked data/RDF/ semantic web(choose fave buzzword here) to take part

The OpenElectionData project • Tackling the open local data problem, one set at a time & learning lessons on the way • Succeed, or fail forward • No public database of local election results, only a commercial one (subsidized by the Electoral Commission) • Allows even those with no prior knowledge of linked data/RDF/ semantic web(choose fave buzzword here) to take part • Just need HTML competence

Open Election Data project How it works • Instead of publishing their election results as arbitrary HTML they publish it as HTML that has been semantically marked up. • e.g. <tr rel='openelection:candidacy'> <th scope="row" rel="openelection:candidate">Helen Elizabeth Fisher</th> <td rel="openelection:party" resource="http://openelectiondata.org/id/parties/25">Conservative</td> <td property="openelection:candidateVoteCount" datatype="xsd:integer">655</td> <td>33.0%</td> <td property="openelection:elected" datatype="xsd:boolean" content="true">Yes</td> </tr> instead of <tr class='zebraBandOdd'> <th scope="row" class='darkGreyCell'>Helen Elizabeth Fisher</th> <td class='lightCell'>Conservative</td> <td class='lightCell'>655</td> <td class='lightCell'>33.0%</td> <td class='lightCell'>Yes</td> </tr>

Open Election Data project How it works

Open Election Data project How it works • Allows:

Open Election Data project How it works • Allows: • data to be pulled from the page, and consumed by RDF reader

Open Election Data project How it works • Allows: • data to be pulled from the page, and consumed by RDF reader • Local Authorities to develop an understanding of the key issues of open linked data - e.g. URIs as identiﬁers

Open Election Data project How it works • Allows: • data to be pulled from the page, and consumed by RDF reader • Local Authorities to develop an understanding of the key issues of open linked data - e.g. URIs as identiﬁers • the beginnings of a complete database of local election results

Open Election Data project How it works • Allows: • data to be pulled from the page, and consumed by RDF reader • Local Authorities to develop an understanding of the key issues of open linked data - e.g. URIs as identiﬁers • the beginnings of a complete database of local election results • a concrete way of identifying the blockers & pain points when moving towards open local data

How’s it going? Too early to say but

How’s it going? Too early to say but • After the May 6 election (local elections at 166 councils) will be somewhere between 12 and 40 councils publishing their data this way (too early to give more accurate ﬁgure yet, lots joining at last minute)

How’s it going? Too early to say but • After the May 6 election (local elections at 166 councils) will be somewhere between 12 and 40 councils publishing their data this way (too early to give more accurate ﬁgure yet, lots joining at last minute) • Already we have increased knowledge and publishing of open data by councils by an order of magnitude, and have established informal network of people who get it

How’s it going? Too early to say but • After the May 6 election (local elections at 166 councils) will be somewhere between 12 and 40 councils publishing their data this way (too early to give more accurate ﬁgure yet, lots joining at last minute) • Already we have increased knowledge and publishing of open data by councils by an order of magnitude, and have established informal network of people who get it • Council people have begun to help each other, solving problems, writing scripts for internal CMSs, and it now has its own momentum

How’s it going? Too early to say but • After the May 6 election (local elections at 166 councils) will be somewhere between 12 and 40 councils publishing their data this way (too early to give more accurate ﬁgure yet, lots joining at last minute) • Already we have increased knowledge and publishing of open data by councils by an order of magnitude, and have established informal network of people who get it • Council people have begun to help each other, solving problems, writing scripts for internal CMSs, and it now has its own momentum • Core blockers already becoming clear: lack of basic skills (even HTML); intransigent middle managers; inﬂexible IT systems

Where next?

Where next? • A Freedom of Data act – too many different bodies

Where next? • A Freedom of Data act – too many different bodies • A recasting of how Government (central & local) does IT. Avoiding the big #fail

Where next? • A Freedom of Data act – too many different bodies • A recasting of how Government (central & local) does IT. Avoiding the big #fail • A change in relationship between central & local government

Where next? • A Freedom of Data act – too many different bodies • A recasting of how Government (central & local) does IT. Avoiding the big #fail • A change in relationship between central & local government • A change between between government and citizen

Where next? • A Freedom of Data act – too many different bodies • A recasting of how Government (central & local) does IT. Avoiding the big #fail • A change in relationship between central & local government • A change between between government and citizen • Some new business models – to support hyperlocal sites, innovative and supportive suppliers, public-interest groups

Where next? • A Freedom of Data act – too many different bodies • A recasting of how Government (central & local) does IT. Avoiding the big #fail • A change in relationship between central & local government • A change between between government and citizen • Some new business models – to support hyperlocal sites, innovative and supportive suppliers, public-interest groups • Develop the meme: enabler or blocker

Open local data presentation for okcon

by Chris Taggart on Apr 24, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 4

Statistics

Open local data presentation for okcon — Presentation Transcript