1. OpenlyLocal,
open local data
& the
Open Election Data project
Social Media Cafe Manchester, April 7, 2010

2. It’s been a good year for open
government data
data.gov.uk, Ordnance Survey, MPs expenses...

3. It’s been a good year for open
government data
data.gov.uk, Ordnance Survey, MPs expenses...

4. It’s been a good year for open
government data
data.gov.uk, Ordnance Survey, MPs expenses...

5. But what about local data?

6. But what about local data?
• Frankly it’s a mess

7. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government

8. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government
• Inaccessible & impenetrable council websites

9. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government
• Inaccessible & impenetrable council websites
• Opaque local public bodies and NDPBs (quangos)

10. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government
• Inaccessible & impenetrable council websites
• Opaque local public bodies and NDPBs (quangos)
• At best unclear & at worst unusable legal situation

11. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government
• Inaccessible & impenetrable council websites
• Opaque local public bodies and NDPBs (quangos)
• At best unclear & at worst unusable legal situation
• Start with the basics. Who are the councillors, where do they
represent and what committees do they sit on?

12. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government
• Inaccessible & impenetrable council websites
• Opaque local public bodies and NDPBs (quangos)
• At best unclear & at worst unusable legal situation
• Start with the basics. Who are the councillors, where do they
represent and what committees do they sit on?
• How easy it that information to find & reuse?

15. Enter OpenlyLocal...

16. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago

17. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago
• Screen-scrapes council websites. Now over 140 councils done, with
basic information on all 434 in the UK

18. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago
• Screen-scrapes council websites. Now over 140 councils done, with
basic information on all 434 in the UK
• Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG...

19. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago
• Screen-scrapes council websites. Now over 140 councils done, with
basic information on all 434 in the UK
• Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG...
• Building a graph of connections – councillors, committees, social
networking, police forces, pension funds

20. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago
• Screen-scrapes council websites. Now over 140 councils done, with
basic information on all 434 in the UK
• Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG...
• Building a graph of connections – councillors, committees, social
networking, police forces, pension funds
• All open data – free for reuse, including commercial reuse

21. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago
• Screen-scrapes council websites. Now over 140 councils done, with
basic information on all 434 in the UK
• Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG...
• Building a graph of connections – councillors, committees, social
networking, police forces, pension funds
• All open data – free for reuse, including commercial reuse
• 100% accessible. Data first; bells & whistles later

26. RDF

27. RDF
JSON

28. RDF
JSON
XML

29. OpenlyLocal: where next?

30. OpenlyLocal: where next?
• More data. More councils. More connections.

31. OpenlyLocal: where next?
• More data. More councils. More connections.
• Election data, cabinet members, mayors, police authorities, health authorities,
pension funds, etc...

32. OpenlyLocal: where next?
• More data. More councils. More connections.
• Election data, cabinet members, mayors, police authorities, health authorities,
pension funds, etc...
• Data used by Hyperlocal sites (inc Guardian). Ning app. Google
Gadget. But want more usage, more plugins written, to give the
(non-technical) community more power

33. OpenlyLocal: where next?
• More data. More councils. More connections.
• Election data, cabinet members, mayors, police authorities, health authorities,
pension funds, etc...
• Data used by Hyperlocal sites (inc Guardian). Ning app. Google
Gadget. But want more usage, more plugins written, to give the
(non-technical) community more power
• Some things can’t be done programmatically. Need crowdsourcing
tools & help with tying info to Wikipedia, other sources

34. OpenlyLocal: where next?
• More data. More councils. More connections.
• Election data, cabinet members, mayors, police authorities, health authorities,
pension funds, etc...
• Data used by Hyperlocal sites (inc Guardian). Ning app. Google
Gadget. But want more usage, more plugins written, to give the
(non-technical) community more power
• Some things can’t be done programmatically. Need crowdsourcing
tools & help with tying info to Wikipedia, other sources
• Some good visualisations done, but lots more potential, particularly
now that we have OS geo data

35. Why does open local data matter?

36. Why does open local data matter?
• Transparency – can we see (and understand) what’s going on?

37. Why does open local data matter?
• Transparency – can we see (and understand) what’s going on?
• Engagement – reducing barriers to getting involved

38. Why does open local data matter?
• Transparency – can we see (and understand) what’s going on?
• Engagement – reducing barriers to getting involved
• Equality of access to information (and thus power)

39. Why does open local data matter?
• Transparency – can we see (and understand) what’s going on?
• Engagement – reducing barriers to getting involved
• Equality of access to information (and thus power)
• Efficiency – Don’t let five different branches of government needlessly
do the same thing (aka keeping it DRY)

40. Transparency
A brief example.

41. Transparency
A brief example.
• A story in Private Eye

42. Transparency
A brief example.
• A story in Private Eye

43. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor

44. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...

45. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...

46. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...
• ...with no relevant heading

47. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...
• ...with no relevant heading

48. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...
• ...with no relevant heading
• Accessible only as a PDF of a
scan of a document

49. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...
• ...with no relevant heading
• Accessible only as a PDF of a
scan of a document

50. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...
• ...with no relevant heading
• Accessible only as a PDF of a
scan of a document
• Is it any wonder suspicions are
raised?

51. Engagement

52. Engagement
• Local Authorities now have a
duty to engage. This means
• Reducing barriers to understanding
and involvement

53. Engagement
• Local Authorities now have a
duty to engage. This means
• Reducing barriers to understanding
and involvement
• Accessible for all

54. Engagement
• Local Authorities now have a
duty to engage. This means
• Reducing barriers to understanding
and involvement
• Accessible for all
• Available to use when, where
& how we want to

55. Engagement
• Local Authorities now have a
duty to engage. This means
• Reducing barriers to understanding
and involvement
• Accessible for all
• Available to use when, where
& how we want to
• Publishing as data means easy
to be repurposed – for mobile,
for mashups, for offline use
(e.g. Postcode Paper)

56. Equality

57. Equality
• At the moment, all
this information
is available...
at a cost

58. Equality ££
£
• At the moment, all
this information
is available...
at a cost

59. Equality ££
£
• At the moment, all
this information
is available...
at a cost
• Huge asymmetry
of information
(and thus power)

60. Equality ££
£
• At the moment, all
this information
is available...
at a cost
• Huge asymmetry
of information
(and thus power)
• Raises the barriers
to involvement,
and, critically, to
challenge

61. Efficiency
Now: cumbersome, wasteful, opaque, error prone
Typical local data flows

62. Efficiency
Now: cumbersome, wasteful, opaque, error prone
Typical local data flows
Council

63. Efficiency
Now: cumbersome, wasteful, opaque, error prone
Typical local data flows
Govt Dept A
email
web
Council Govt Dept B
upload Govt Dept C

64. Efficiency
Now: cumbersome, wasteful, opaque, error prone
Typical local data flows
Govt Dept A public (maybe)
email
web
Council Govt Dept B
upload Govt Dept C

65. Efficiency
A better way?
Council

66. Efficiency
A better way?
open data
Council

67. Efficiency
A better way?
<tr rel='openelection:candidacy'>
<!-- Here we are saying the candidate has given names of
Keith William and a familyName Cockroft -->
<th scope="row" rel="openelection:candidate"><span
property="foaf:givenName">Keith William</span> <span
property="foaf:familyName">Cockroft</span></th>
<td rel="openelection:party" resource="http://
openelectiondata.org/id/parties/6"><span
property="rdfs:label">Labour</span></td>
<!-- The candidacy got 330 votes -->
<td property="openelection:candidateVoteCount"
datatype="xsd:integer">330</td>
<td>16.6%</td>
open data
Council <td property="openelection:elected"
datatype="xsd:boolean" content="false">No</td>
</tr>
<tr rel='openelection:candidacy'>
<th scope="row" rel="openelection:candidate">
<span typeof="openelection:Candidate"
property="foaf:name">Brenda Lilian Constable</span>
<span rel="openelection:address">
<span property="v:street-address">37 Morley Road</
span>,
<span property="v:locality">Burntwood</span>,
<span property="v:region">Staffordshire</span>
<span property="v:postal-code">WS7 2DE</span>
</span>
</th>

68. Efficiency
A better way?
<tr rel='openelection:candidacy'>
<!-- Here we are saying the candidate has given names of
Keith William and a familyName Cockroft -->
<th scope="row" rel="openelection:candidate"><span
public
property="foaf:givenName">Keith William</span> <span
property="foaf:familyName">Cockroft</span></th>
<td rel="openelection:party" resource="http://
openelectiondata.org/id/parties/6"><span
property="rdfs:label">Labour</span></td>
<!-- The candidacy got 330 votes -->
<td property="openelection:candidateVoteCount"
open data
datatype="xsd:integer">330</td>
<td>16.6%</td> Govt Dept A
Council <td property="openelection:elected"
datatype="xsd:boolean" content="false">No</td>
</tr>
<tr rel='openelection:candidacy'>
<th scope="row" rel="openelection:candidate">
Govt Dept B
<span typeof="openelection:Candidate"
property="foaf:name">Brenda Lilian Constable</span>
<span rel="openelection:address">
<span property="v:street-address">37 Morley Road</
span>,
Govt Dept C
<span property="v:locality">Burntwood</span>,
<span property="v:region">Staffordshire</span>
<span property="v:postal-code">WS7 2DE</span>
</span>
</th>

69. Efficiency
A better way?
<tr rel='openelection:candidacy'>
<!-- Here we are saying the candidate has given names of
Keith William and a familyName Cockroft -->
<th scope="row" rel="openelection:candidate"><span
public
property="foaf:givenName">Keith William</span> <span
property="foaf:familyName">Cockroft</span></th>
<td rel="openelection:party" resource="http://
openelectiondata.org/id/parties/6"><span
property="rdfs:label">Labour</span></td>
<!-- The candidacy got 330 votes -->
<td property="openelection:candidateVoteCount"
open data
datatype="xsd:integer">330</td>
<td>16.6%</td> Govt Dept A
Council <td property="openelection:elected"
datatype="xsd:boolean" content="false">No</td>
</tr>
<tr rel='openelection:candidacy'>
<th scope="row" rel="openelection:candidate">
Govt Dept B
<span typeof="openelection:Candidate"
property="foaf:name">Brenda Lilian Constable</span>
<span rel="openelection:address">
<span property="v:street-address">37 Morley Road</
span>,
Govt Dept C
<span property="v:locality">Burntwood</span>,
<span property="v:region">Staffordshire</span>
<span property="v:postal-code">WS7 2DE</span>
</span>
</th>
Publish once, consume many times

70. Problems

71. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently

72. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently
• Data tied up in PDFs (will we ever get this back?)

73. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently
• Data tied up in PDFs (will we ever get this back?)
• Legacy systems. That nobody now understands

74. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently
• Data tied up in PDFs (will we ever get this back?)
• Legacy systems. That nobody now understands
• Private companies/JVs. No FoI. No access to data

75. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently
• Data tied up in PDFs (will we ever get this back?)
• Legacy systems. That nobody now understands
• Private companies/JVs. No FoI. No access to data
• Outsourcing (councils have outsourced the skills & knowledge and
now are often dumb consumers)

76. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently
• Data tied up in PDFs (will we ever get this back?)
• Legacy systems. That nobody now understands
• Private companies/JVs. No FoI. No access to data
• Outsourcing (councils have outsourced the skills & knowledge and
now are often dumb consumers)
• Silos, especially of budgets. How do you get money from one budget
to another (even if the whole benefits)

77. The OpenElectionData project

78. The OpenElectionData project
• Tackling the open local data problem, one set at a time & learning
lessons on the way

79. The OpenElectionData project
• Tackling the open local data problem, one set at a time & learning
lessons on the way
• Succeed, or fail forward

80. The OpenElectionData project
• Tackling the open local data problem, one set at a time & learning
lessons on the way
• Succeed, or fail forward
• No public database of local election results, only a commercial one
(subsidized by the Electoral Commission)

81. The OpenElectionData project
• Tackling the open local data problem, one set at a time & learning
lessons on the way
• Succeed, or fail forward
• No public database of local election results, only a commercial one
(subsidized by the Electoral Commission)
• Allows even those with no prior knowledge of linked data/RDF/
semantic web(choose fave buzzword here) to take part

82. The OpenElectionData project
• Tackling the open local data problem, one set at a time & learning
lessons on the way
• Succeed, or fail forward
• No public database of local election results, only a commercial one
(subsidized by the Electoral Commission)
• Allows even those with no prior knowledge of linked data/RDF/
semantic web(choose fave buzzword here) to take part
• Just need HTML competence

84. Open Election Data project
How it works
• Instead of publishing their election results as arbitrary HTML they
publish it as HTML that has been semantically marked up.
• e.g.
<tr rel='openelection:candidacy'>
<th scope="row" rel="openelection:candidate"><span typeof="openelection:Candidate"
property="foaf:name">Helen Elizabeth Fisher</span></th>
<td rel="openelection:party" resource="http://openelectiondata.org/id/parties/25"><span
property="rdfs:label">Conservative</span></td>
<td property="openelection:candidateVoteCount" datatype="xsd:integer">655</td>
<td>33.0%</td>
<td property="openelection:elected" datatype="xsd:boolean" content="true">Yes</td>
</tr>
instead of
<tr class='zebraBandOdd'>
<th scope="row" class='darkGreyCell'><span class='boldEntry'>Helen Elizabeth Fisher</span></th>
<td class='lightCell'>Conservative</td>
<td class='lightCell'>655</td>
<td class='lightCell'>33.0%</td>
<td class='lightCell'>Yes</td>
</tr>

85. Open Election Data project
How it works

86. Open Election Data project
How it works
• Allows:

87. Open Election Data project
How it works
• Allows:
• data to be pulled from the page, and consumed by RDF reader

88. Open Election Data project
How it works
• Allows:
• data to be pulled from the page, and consumed by RDF reader
• Local Authorities to develop an understanding of the key issues of
open linked data - e.g. URIs as identifiers

89. Open Election Data project
How it works
• Allows:
• data to be pulled from the page, and consumed by RDF reader
• Local Authorities to develop an understanding of the key issues of
open linked data - e.g. URIs as identifiers
• the beginnings of a complete database of local election results

90. Open Election Data project
How it works
• Allows:
• data to be pulled from the page, and consumed by RDF reader
• Local Authorities to develop an understanding of the key issues of
open linked data - e.g. URIs as identifiers
• the beginnings of a complete database of local election results
• a concrete way of identifying the blockers & pain points when
moving towards open local data

91. Where next?

92. Where next?
• A Freedom of Data act

93. Where next?
• A Freedom of Data act
• A recasting of how Government (central & local) does IT. Avoid the
big #fail

94. Where next?
• A Freedom of Data act
• A recasting of how Government (central & local) does IT. Avoid the
big #fail
• A change in relationship between central & local government

95. Where next?
• A Freedom of Data act
• A recasting of how Government (central & local) does IT. Avoid the
big #fail
• A change in relationship between central & local government
• A change between between government and citizen

96. Where next?
• A Freedom of Data act
• A recasting of how Government (central & local) does IT. Avoid the
big #fail
• A change in relationship between central & local government
• A change between between government and citizen
• Some new business models – to support hyperlocal sites, innovative
and supportive suppliers, public-interest groups