Presentation given to OKCON 2010 (Open Knowledge Foundation Conference), held at ULU, London, April 24. Note this is a slightly updated and reworked version of the presentation given to the Manchester Social Media Cafe on April 6

1. Open Local Data,
OpenlyLocal & the
Open Election Data project
Open Knowledge Conference, London, April 24, 2010

2. It’s been a good year for open
government data
data.gov.uk, Ordnance Survey, MPs expenses...

3. It’s been a good year for open
government data
data.gov.uk, Ordnance Survey, MPs expenses...

4. It’s been a good year for open
government data
data.gov.uk, Ordnance Survey, MPs expenses...

5. But what about local data?

6. But what about local data?
• Frankly it’s a mess

7. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government

8. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government
• Inaccessible & impenetrable council websites

9. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government
• Inaccessible & impenetrable council websites
• Opaque local public bodies and NDPBs (quangos)

10. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government
• Inaccessible & impenetrable council websites
• Opaque local public bodies and NDPBs (quangos)
• At best unclear & at worst unusable legal situation

11. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government
• Inaccessible & impenetrable council websites
• Opaque local public bodies and NDPBs (quangos)
• At best unclear & at worst unusable legal situation
• Start with the basics. Who are the councillors, where do they
represent and what committees do they sit on?

12. But what about local data?
• Frankly it’s a mess
• Sporadically published by central government
• Inaccessible & impenetrable council websites
• Opaque local public bodies and NDPBs (quangos)
• At best unclear & at worst unusable legal situation
• Start with the basics. Who are the councillors, where do they
represent and what committees do they sit on?
• How easy it that information to find & reuse?

15. Enter OpenlyLocal...

16. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago

17. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago
• Screen-scrapes council websites. Now over 140 councils done, with
basic information on all 434 in the UK

18. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago
• Screen-scrapes council websites. Now over 140 councils done, with
basic information on all 434 in the UK
• Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG...

19. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago
• Screen-scrapes council websites. Now over 140 councils done, with
basic information on all 434 in the UK
• Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG...
• Building a graph of connections – councillors, committees, social
networking, police forces, pension funds

20. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago
• Screen-scrapes council websites. Now over 140 councils done, with
basic information on all 434 in the UK
• Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG...
• Building a graph of connections – councillors, committees, social
networking, police forces, pension funds
• All open data – free for reuse, including commercial reuse

21. Enter OpenlyLocal...
• Inspiration was a Manchester project, MCC Work For You. Doing
something concrete to solve a problem. First code written 1 year ago
• Screen-scrapes council websites. Now over 140 councils done, with
basic information on all 434 in the UK
• Pulls info from about 150 local councils, plus ONS, OS, NPIA, CLG...
• Building a graph of connections – councillors, committees, social
networking, police forces, pension funds
• All open data – free for reuse, including commercial reuse
• 100% accessible. Data first; bells & whistles later

26. RDF

27. RDF
JSON

28. RDF
JSON
XML

29. OpenlyLocal: where next?

30. OpenlyLocal: where next?
• More data. More councils. More connections.

31. OpenlyLocal: where next?
• More data. More councils. More connections.
• Election data, cabinet members, mayors, police authorities, health authorities,
pension funds, etc... it’s all about the network

32. OpenlyLocal: where next?
• More data. More councils. More connections.
• Election data, cabinet members, mayors, police authorities, health authorities,
pension funds, etc... it’s all about the network
• Data used by Hyperlocal sites (inc Guardian). Ning app. Google
Gadget. Widget. And now starting to get info in the other direction
(i.e. connecting blog posts to councillors)

33. OpenlyLocal: where next?
• More data. More councils. More connections.
• Election data, cabinet members, mayors, police authorities, health authorities,
pension funds, etc... it’s all about the network
• Data used by Hyperlocal sites (inc Guardian). Ning app. Google
Gadget. Widget. And now starting to get info in the other direction
(i.e. connecting blog posts to councillors)
• Some things can’t be done programmatically. Need crowdsourcing
tools & help with tying info to Wikipedia, other sources

34. OpenlyLocal: where next?
• More data. More councils. More connections.
• Election data, cabinet members, mayors, police authorities, health authorities,
pension funds, etc... it’s all about the network
• Data used by Hyperlocal sites (inc Guardian). Ning app. Google
Gadget. Widget. And now starting to get info in the other direction
(i.e. connecting blog posts to councillors)
• Some things can’t be done programmatically. Need crowdsourcing
tools & help with tying info to Wikipedia, other sources
• Some good visualisations done, but lots more potential, particularly
now that we have OS geo data

35. Why does open local data matter?

36. Why does open local data matter?
• Transparency – can we see what’s going on (and understand it too)

37. Why does open local data matter?
• Transparency – can we see what’s going on (and understand it too)
• Engagement – reducing barriers to getting involved

38. Why does open local data matter?
• Transparency – can we see what’s going on (and understand it too)
• Engagement – reducing barriers to getting involved
• Equality of access to information (and thus power). Much of this data
is already available... for a price

39. Why does open local data matter?
• Transparency – can we see what’s going on (and understand it too)
• Engagement – reducing barriers to getting involved
• Equality of access to information (and thus power). Much of this data
is already available... for a price
• Efficiency – Don’t let five different branches of government needlessly
do the same thing (aka keeping it DRY)

40. Simple questions, hard to answer

41. Simple questions, hard to answer
• How does the budget of my council compare with similar ones?

42. Simple questions, hard to answer
• How does the budget of my council compare with similar ones?
• What’s the background of the chief executive of my council?

43. Simple questions, hard to answer
• How does the budget of my council compare with similar ones?
• What’s the background of the chief executive of my council?
• Is my route to work going to affected by roadworks next week?

44. Simple questions, hard to answer
• How does the budget of my council compare with similar ones?
• What’s the background of the chief executive of my council?
• Is my route to work going to affected by roadworks next week?
• What are the connections between the companies awarded contracts
and the councillors & senior management?

45. Simple questions, hard to answer
• How does the budget of my council compare with similar ones?
• What’s the background of the chief executive of my council?
• Is my route to work going to affected by roadworks next week?
• What are the connections between the companies awarded contracts
and the councillors & senior management?
• How will cuts in services be decided and what can I do to influence
the decisions?

46. Transparency
A brief example.

47. Transparency
A brief example.
• A story in Private Eye

48. Transparency
A brief example.
• A story in Private Eye

49. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor

50. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...

51. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...

52. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...
• ...with no relevant heading

53. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...
• ...with no relevant heading

54. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...
• ...with no relevant heading
• Accessible only as a PDF of a
scan of a document

55. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...
• ...with no relevant heading
• Accessible only as a PDF of a
scan of a document

56. Transparency
A brief example.
• A story in Private Eye
• An investigation by the District
Auditor
• A report buried in the nether
recesses of the website...
• ...with no relevant heading
• Accessible only as a PDF of a
scan of a document
• Is it any wonder suspicions are
raised?

57. Engagement

58. Engagement
• Local Authorities now have a
duty to engage. This means
• Reducing barriers to understanding
and involvement

59. Engagement
• Local Authorities now have a
duty to engage. This means
• Reducing barriers to understanding
and involvement
• Accessible for all

60. Engagement
• Local Authorities now have a
duty to engage. This means
• Reducing barriers to understanding
and involvement
• Accessible for all
• Available to use when, where
& how we want to

61. Engagement
• Local Authorities now have a
duty to engage. This means
• Reducing barriers to understanding
and involvement
• Accessible for all
• Available to use when, where
& how we want to
• Publishing as data means easy
to be repurposed – for mobile,
for mashups, for offline use
(e.g. Postcode Paper)

62. Equality

63. Equality
• At the moment, all
this information
is available...
at a cost

64. Equality ££
£
• At the moment, all
this information
is available...
at a cost

65. Equality ££
£
• At the moment, all
this information
is available...
at a cost
• Huge asymmetry
of information
(and thus power)

66. Equality ££
£
• At the moment, all
this information
is available...
at a cost
• Huge asymmetry
of information
(and thus power)
• Raises the barriers
to involvement,
and, critically, to
challenge

67. Efficiency
Now: cumbersome, wasteful, opaque, error prone
Typical local data flows

68. Efficiency
Now: cumbersome, wasteful, opaque, error prone
Typical local data flows
Council

69. Efficiency
Now: cumbersome, wasteful, opaque, error prone
Typical local data flows
Govt Dept A
email
web
Council Govt Dept B
upload Govt Dept C

70. Efficiency
Now: cumbersome, wasteful, opaque, error prone
Typical local data flows
Govt Dept A public (maybe)
email
web
Council Govt Dept B
upload Govt Dept C

71. Efficiency
A better way?
Council

72. Efficiency
A better way?
open data
Council

73. Efficiency
A better way?
<tr rel='openelection:candidacy'>
<!-- Here we are saying the candidate has given names of
Keith William and a familyName Cockroft -->
<th scope="row" rel="openelection:candidate"><span
property="foaf:givenName">Keith William</span> <span
property="foaf:familyName">Cockroft</span></th>
<td rel="openelection:party" resource="http://
openelectiondata.org/id/parties/6"><span
property="rdfs:label">Labour</span></td>
<!-- The candidacy got 330 votes -->
<td property="openelection:candidateVoteCount"
datatype="xsd:integer">330</td>
<td>16.6%</td>
open data
Council <td property="openelection:elected"
datatype="xsd:boolean" content="false">No</td>
</tr>
<tr rel='openelection:candidacy'>
<th scope="row" rel="openelection:candidate">
<span typeof="openelection:Candidate"
property="foaf:name">Brenda Lilian Constable</span>
<span rel="openelection:address">
<span property="v:street-address">37 Morley Road</
span>,
<span property="v:locality">Burntwood</span>,
<span property="v:region">Staffordshire</span>
<span property="v:postal-code">WS7 2DE</span>
</span>
</th>

74. Efficiency
A better way?
<tr rel='openelection:candidacy'>
<!-- Here we are saying the candidate has given names of
Keith William and a familyName Cockroft -->
<th scope="row" rel="openelection:candidate"><span
public
property="foaf:givenName">Keith William</span> <span
property="foaf:familyName">Cockroft</span></th>
<td rel="openelection:party" resource="http://
openelectiondata.org/id/parties/6"><span
property="rdfs:label">Labour</span></td>
<!-- The candidacy got 330 votes -->
<td property="openelection:candidateVoteCount"
open data
datatype="xsd:integer">330</td>
<td>16.6%</td> Govt Dept A
Council <td property="openelection:elected"
datatype="xsd:boolean" content="false">No</td>
</tr>
<tr rel='openelection:candidacy'>
<th scope="row" rel="openelection:candidate">
Govt Dept B
<span typeof="openelection:Candidate"
property="foaf:name">Brenda Lilian Constable</span>
<span rel="openelection:address">
<span property="v:street-address">37 Morley Road</
span>,
Govt Dept C
<span property="v:locality">Burntwood</span>,
<span property="v:region">Staffordshire</span>
<span property="v:postal-code">WS7 2DE</span>
</span>
</th>

75. Efficiency
A better way?
<tr rel='openelection:candidacy'>
<!-- Here we are saying the candidate has given names of
Keith William and a familyName Cockroft -->
<th scope="row" rel="openelection:candidate"><span
public
property="foaf:givenName">Keith William</span> <span
property="foaf:familyName">Cockroft</span></th>
<td rel="openelection:party" resource="http://
openelectiondata.org/id/parties/6"><span
property="rdfs:label">Labour</span></td>
<!-- The candidacy got 330 votes -->
<td property="openelection:candidateVoteCount"
open data
datatype="xsd:integer">330</td>
<td>16.6%</td> Govt Dept A
Council <td property="openelection:elected"
datatype="xsd:boolean" content="false">No</td>
</tr>
<tr rel='openelection:candidacy'>
<th scope="row" rel="openelection:candidate">
Govt Dept B
<span typeof="openelection:Candidate"
property="foaf:name">Brenda Lilian Constable</span>
<span rel="openelection:address">
<span property="v:street-address">37 Morley Road</
span>,
Govt Dept C
<span property="v:locality">Burntwood</span>,
<span property="v:region">Staffordshire</span>
<span property="v:postal-code">WS7 2DE</span>
</span>
</th>
Publish once, consume many times

76. Problems

77. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently

78. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently
• Data tied up in PDFs (will we ever get this back?)

79. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently
• Data tied up in PDFs (will we ever get this back?)
• Legacy systems. That nobody now understands

80. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently
• Data tied up in PDFs (will we ever get this back?)
• Legacy systems. That nobody now understands
• Private companies/JVs. No FoI. No access to data

81. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently
• Data tied up in PDFs (will we ever get this back?)
• Legacy systems. That nobody now understands
• Private companies/JVs. No FoI. No access to data
• Outsourcing (councils have outsourced the skills & knowledge and
now are often dumb consumers)

82. Problems
• IDs – councils don’t generally use them; central government uses
many... inconsistently
• Data tied up in PDFs (will we ever get this back?)
• Legacy systems. That nobody now understands
• Private companies/JVs. No FoI. No access to data
• Outsourcing (councils have outsourced the skills & knowledge and
now are often dumb consumers)
• Silos, especially of budgets. How do you get money from one budget
to another (even if the whole benefits)

83. The OpenElectionData project

84. The OpenElectionData project
• Tackling the open local data problem, one set at a time & learning
lessons on the way

85. The OpenElectionData project
• Tackling the open local data problem, one set at a time & learning
lessons on the way
• Succeed, or fail forward

86. The OpenElectionData project
• Tackling the open local data problem, one set at a time & learning
lessons on the way
• Succeed, or fail forward
• No public database of local election results, only a commercial one
(subsidized by the Electoral Commission)

87. The OpenElectionData project
• Tackling the open local data problem, one set at a time & learning
lessons on the way
• Succeed, or fail forward
• No public database of local election results, only a commercial one
(subsidized by the Electoral Commission)
• Allows even those with no prior knowledge of linked data/RDF/
semantic web(choose fave buzzword here) to take part

88. The OpenElectionData project
• Tackling the open local data problem, one set at a time & learning
lessons on the way
• Succeed, or fail forward
• No public database of local election results, only a commercial one
(subsidized by the Electoral Commission)
• Allows even those with no prior knowledge of linked data/RDF/
semantic web(choose fave buzzword here) to take part
• Just need HTML competence

90. Open Election Data project
How it works
• Instead of publishing their election results as arbitrary HTML they
publish it as HTML that has been semantically marked up.
• e.g.
<tr rel='openelection:candidacy'>
<th scope="row" rel="openelection:candidate"><span typeof="openelection:Candidate"
property="foaf:name">Helen Elizabeth Fisher</span></th>
<td rel="openelection:party" resource="http://openelectiondata.org/id/parties/25"><span
property="rdfs:label">Conservative</span></td>
<td property="openelection:candidateVoteCount" datatype="xsd:integer">655</td>
<td>33.0%</td>
<td property="openelection:elected" datatype="xsd:boolean" content="true">Yes</td>
</tr>
instead of
<tr class='zebraBandOdd'>
<th scope="row" class='darkGreyCell'><span class='boldEntry'>Helen Elizabeth Fisher</span></th>
<td class='lightCell'>Conservative</td>
<td class='lightCell'>655</td>
<td class='lightCell'>33.0%</td>
<td class='lightCell'>Yes</td>
</tr>

91. Open Election Data project
How it works

92. Open Election Data project
How it works
• Allows:

93. Open Election Data project
How it works
• Allows:
• data to be pulled from the page, and consumed by RDF reader

94. Open Election Data project
How it works
• Allows:
• data to be pulled from the page, and consumed by RDF reader
• Local Authorities to develop an understanding of the key issues of
open linked data - e.g. URIs as identifiers

95. Open Election Data project
How it works
• Allows:
• data to be pulled from the page, and consumed by RDF reader
• Local Authorities to develop an understanding of the key issues of
open linked data - e.g. URIs as identifiers
• the beginnings of a complete database of local election results

96. Open Election Data project
How it works
• Allows:
• data to be pulled from the page, and consumed by RDF reader
• Local Authorities to develop an understanding of the key issues of
open linked data - e.g. URIs as identifiers
• the beginnings of a complete database of local election results
• a concrete way of identifying the blockers & pain points when
moving towards open local data

97. How’s it going? Too early to say but

98. How’s it going? Too early to say but
• After the May 6 election (local elections at 166 councils) will be
somewhere between 12 and 40 councils publishing their data this way
(too early to give more accurate figure yet, lots joining at last minute)

99. How’s it going? Too early to say but
• After the May 6 election (local elections at 166 councils) will be
somewhere between 12 and 40 councils publishing their data this way
(too early to give more accurate figure yet, lots joining at last minute)
• Already we have increased knowledge and publishing of open data
by councils by an order of magnitude, and have established informal
network of people who get it

100. How’s it going? Too early to say but
• After the May 6 election (local elections at 166 councils) will be
somewhere between 12 and 40 councils publishing their data this way
(too early to give more accurate figure yet, lots joining at last minute)
• Already we have increased knowledge and publishing of open data
by councils by an order of magnitude, and have established informal
network of people who get it
• Council people have begun to help each other, solving problems,
writing scripts for internal CMSs, and it now has its own momentum

101. How’s it going? Too early to say but
• After the May 6 election (local elections at 166 councils) will be
somewhere between 12 and 40 councils publishing their data this way
(too early to give more accurate figure yet, lots joining at last minute)
• Already we have increased knowledge and publishing of open data
by councils by an order of magnitude, and have established informal
network of people who get it
• Council people have begun to help each other, solving problems,
writing scripts for internal CMSs, and it now has its own momentum
• Core blockers already becoming clear: lack of basic skills (even
HTML); intransigent middle managers; inflexible IT systems

102. Where next?

103. Where next?
• A Freedom of Data act – too many different bodies

104. Where next?
• A Freedom of Data act – too many different bodies
• A recasting of how Government (central & local) does IT. Avoiding
the big #fail

105. Where next?
• A Freedom of Data act – too many different bodies
• A recasting of how Government (central & local) does IT. Avoiding
the big #fail
• A change in relationship between central & local government

106. Where next?
• A Freedom of Data act – too many different bodies
• A recasting of how Government (central & local) does IT. Avoiding
the big #fail
• A change in relationship between central & local government
• A change between between government and citizen

107. Where next?
• A Freedom of Data act – too many different bodies
• A recasting of how Government (central & local) does IT. Avoiding
the big #fail
• A change in relationship between central & local government
• A change between between government and citizen
• Some new business models – to support hyperlocal sites, innovative
and supportive suppliers, public-interest groups

108. Where next?
• A Freedom of Data act – too many different bodies
• A recasting of how Government (central & local) does IT. Avoiding
the big #fail
• A change in relationship between central & local government
• A change between between government and citizen
• Some new business models – to support hyperlocal sites, innovative
and supportive suppliers, public-interest groups
• Develop the meme: enabler or blocker